feat(core): slab + sparse motifs, slice 1 -- flat field, monolith, platform at 0.52 s (L56); research R25
Motif vocabulary v3: version-1 records keyed by (seed, strate, stable ID) with route keepouts, band clearance and pairwise spacing. A rounded capped-cylinder monolith (r 1.4 m, 24 m tall) and a rounded-box plinth, smooth-unioned into a flatField floor with a 0.55 m geometric rounding (measured fillets 0.73-1.17 m). Exact radial/box interval bounds, a BVH, isTileShapeFree aware (0 brute-force misses, false-proof controls rejected). Final-field route passes; whole-surface audit covers foot, cap and platform edges. Cold whole view 0.524 s. Owner world unchanged. fast/medium/full PASS (Clang and GCC). Research R25 (self-checked): FloatingIslands LOD0 column cache conflicts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
This commit is contained in:
Fr0zkaandClaude Opus 5.5 committed
1 parent
aee1487410
commit
4e64b75044
15 files changed
+1560
-12
No files matched your search
@@ -0,0 +1,84 @@
|
||||
# VoxelForge R25 — revised self-check report
|
||||
|
||||
## Self-check
|
||||
|
||||
The earlier “heavy thrashing” conclusion was too strong. The conflict counts alone do not prove that the column cache caused most of the grid time.
|
||||
|
||||
I reread the complete cache and FI density helper functions cited below, plus the mesher fill and X8 branches and the instrumentation blocks. I did not read every line of the large enclosing `GenerateMesh` and `World::generateTile` functions end to end; I rely only on the cited branches. This was a read-only review: no scripts, tests, builds, binaries, or edits.
|
||||
|
||||
## Cache mechanics and the access sequence
|
||||
|
||||
There is no separate local index function: `IslandColumnCache` computes its index inline. The cache key is **XY, not XYZ**. The equality check includes recipe and plan identities and hashes, seed, stable island ID, sample X/Y, and island/recipe shape parameters. It contains no Z. The cached value holds per-island XY-derived metrics—delta X/Y, radial distance, angle, lobe phase, and top surface—not the 3D density value. See [`FloatingIslandsInternal.h`]( /home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/FloatingIslandsInternal.h:497).
|
||||
|
||||
The local cache has four ways and 2,048 sets, or 8,192 entries per worker. The key-match lines include:
|
||||
|
||||
```cpp
|
||||
const bool bKeyMatches=Entry.bValid&&Entry.RecipeIdentity==&recipe
|
||||
&&Entry.PlanIdentity==&plan&&Entry.RecipeContentHash==recipe.contentHash
|
||||
&&Entry.PlanContentHash==plan.contentHash&&Entry.Seed==recipe.seed
|
||||
&&Entry.StableId==island.stableId
|
||||
&&Entry.SampleX==SampleX&&Entry.SampleY==SampleY
|
||||
```
|
||||
|
||||
The exact way selection is:
|
||||
|
||||
```cpp
|
||||
IndexKey^=IndexKey>>33;
|
||||
IndexKey*=0xff51afd7ed558ccdull;
|
||||
IndexKey^=IndexKey>>33;
|
||||
const std::size_t Base=static_cast<std::size_t>(IndexKey&(SetCount-1))*Ways;
|
||||
FIslandColumnCacheEntry* Replacement=&cache[Base+((IndexKey>>11)&(Ways-1))];
|
||||
```
|
||||
|
||||
Those lines are at 519, 520, 521, 522, and 523. If a way is invalid, the first invalid way is used; otherwise the preselected hash way is overwritten. This is four-way set associative, **not direct mapped**, and replacement is deterministic by hash bits, not LRU. Lines 545, 546, and 551 define the local “conflict”: a miss where all four ways in that set are already valid.
|
||||
|
||||
[`FLazy::Get`]( /home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/include/VoxelForgeCore/ThreadLocalStorage.h:21) lazily allocates the local cache once per worker. Its entries persist across that worker’s tile jobs; the per-tile instrumentation is reset, but the cache contents are not. That makes the near-equality unsurprising: on 27.683M local misses, only 86k were to sets with an invalid way; the other 27.597M landed in full sets. A full set can contain unrelated old keys, so this counter does **not** establish that a useful key was displaced or soon reused.
|
||||
|
||||
The shared cache is a world-owned second level with 16,384 sets × 4 ways. Its key also excludes Z and includes recipe/plan identity and hashes, XY, and the island stable ID as context. On insertion it uses the first empty way, then round-robins through `NextWay`; its conflicts likewise mean “set full when inserting.” A shared miss is counted before taking the lock; if the second lookup finds a value, that call increments both misses and hits. Thus shared counters are not mutually exclusive. See [`SharedColumnCache.h`]( /home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/include/VoxelForgeCore/SharedColumnCache.h:27), especially lines 120, 128, 131, 133, and 143–152.
|
||||
|
||||
For a concrete fill sequence, the mesher’s ordinary path has Z outermost, then Y and contiguous X batches; see [`VoxelMarchingCubesMesher.cpp`]( /home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:1319). An X8 batch is not SIMD evaluation of the FI field: [`DensityAtLatticeX8`]( /home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:3161) loops through lanes and calls `NativeFloatingIslandDensityAt` lane by lane. A logged tile has 6,125 X8 calls and 42,875 active lanes—seven lanes per call on average.
|
||||
|
||||
For a full 35³ sample grid and four islands, each point makes four XY-column lookups: 171,500 lookups. With Z outermost, the next Z plane revisits the same 4 × 35 × 35 = 4,900 island/XY keys after a plane’s worth of accesses. That is below the cache’s 8,192-entry *total* capacity, but per-set collisions and keys left by earlier tiles can still evict entries. The 4-way hash can also alias keys within a set. The code does not provide the set-occupancy or reuse trace needed to separate those causes.
|
||||
|
||||
The access sequence does **not** reproduce the aggregate counters by itself. For example, one logged tile with 42,875 samples has 147,201 local hits, 24,299 misses, and 19,713 conflicts—171,500 lookups, exactly four per sample. Repeating those tile counts across 512 tiles would give about 75.4M hits and 12.4M misses, not the measured 34.6M hits and 27.7M misses. That row is not representative of all tiles, but it shows why a single-tile sequence cannot explain the aggregate. See [`fi.log`]( /home/jahni/VoxelM/Plugins/VoxelForge/Saved/Linux/l55/timeline-static/fi.log:2464).
|
||||
|
||||
The aggregate ratio has a simpler explanation: the conflict counter is a full-set condition applied to misses, and both caches persist long enough for many target sets to be full. The remaining miss pattern is consistent with plane-first traversal, set collisions, prior tile contents, and other callers; the counters cannot say which dominates.
|
||||
|
||||
There are also important scope effects:
|
||||
|
||||
- The FI cache instrumentation is enabled for the whole tile generation and copied after mesh assembly, not only around grid fill. The X8 tile above has 188,864 local lookups—17,364 more than its 42,875 × 4 grid baseline—and 5,935 skirt density probes. Skirt probes call back into shape density and can reach the FI field.
|
||||
- `NativeFloatingIslandDensityAt` has a passage-mouth filter. Near the mouth and within its density threshold, it samples a 3×3×3 neighborhood (26 additional field samples), and may make up to six more fragment probes. Those calls also reach the column cache.
|
||||
- The 892k proof skips are **cells**, not necessarily density samples avoided. There is a separate `proof_density_samples` counter. One logged tile skips 8,192 air cells and 9,800 density samples; those are different units. The aggregate cell count alone cannot quantify grid-fill savings.
|
||||
- The four-island L55 plan does not activate the tile feature-query route: `candidateEvaluationUseful` is set only when the plan has more than four islands, and the scoped query ignores it otherwise. The ordinary `FieldDensity` path therefore scans all four islands and their connector groups. Each full island evaluation runs five root-lobe distances; for four islands that is 20 root-lobe evaluations per field point, plus the plan’s connectors. The root parameter constants are cached separately, but the point-dependent distances are recomputed. The lobe contour uses three sine harmonics varying with Z; I found no separate helix evaluator.
|
||||
|
||||
## Cost split
|
||||
|
||||
| Quantity | Status | Reading |
|
||||
|---|---|---|
|
||||
| FI LOD0 total 29.710 worker-s; grid 27.610; skirts 1.163 | **Measured** | The grid is 92.9% of LOD0 total. The remaining 0.937 worker-s is derived. |
|
||||
| Local lookups 62.259M; hits 55.5%, misses 44.5%; conflicts are 99.69% of misses | **Derived** | From 34.576M hits, 27.683M misses, and 27.597M conflicts. |
|
||||
| Shared computes 3.733M; shared hits 23.951M; shared wait 150.689 ms | **Measured** | Most local misses are served by the shared cache. Wait time is about 0.55% of grid time. |
|
||||
| Field arithmetic versus all cache lookup overhead | **Not measured** | The 150.689 ms is lock wait only. It excludes local probes, shared atomic reads, hashing, and other cache work. No counter times FI field math separately. |
|
||||
|
||||
The 3.733M shared computes are cached XY-column builds, not full FI density evaluations. The measured counts and code show substantial field work, but they do not support a defensible percentage split. My estimate is that the repeated island/root evaluation is substantial and may dominate; the 23.951M shared hits still take atomic read work, so cache overhead could also be material. **No numerical field-versus-cache fraction can be justified from L55.**
|
||||
|
||||
## Ranked fixes
|
||||
|
||||
The savings ratings are qualitative; L55 did not measure any of these changes.
|
||||
|
||||
| Rank by potential worker-second saving | Fix | Risk and exactness |
|
||||
|---|---|---|
|
||||
| 1 | **Coarse per-island sampling with refinement near the surface.** It has the largest potential because it could reduce the number of expensive field evaluations. | High implementation risk. Interpolating coarse values changes density, normals, and likely hashes; exactness is not preserved unless the refinement scheme proves and samples every required value. |
|
||||
| 2 | **Exact per-island culling with the existing tile feature bounds, or a tighter per-island BVH.** The current four-island plan takes the full-scan path. Testing the exact candidate/lower-bound path could avoid whole island/root evaluations for distant positive-density points. | Medium risk and exact if conservative bounds are maintained. Actual savings are uncertain; the code currently marks this path uneconomic at four islands. |
|
||||
| 3 | **Test XY-contiguous (Z-inner) traversal at LOD0 before changing the hash or ways.** It would revisit each island/XY key immediately across Z, reducing the current plane-sized reuse distance. The current vertical traversal threshold is only used for tiles larger than 512 voxels; LOD0 is 32. | Low correctness risk: samples still write to their lattice indices. Strided grid writes may cost more elsewhere. It removes cache misses, not root-lobe work, so it may not approach the roughly 11 worker-s LOD0 target by itself. |
|
||||
| 4 | **Tighten exact interval proofs / narrow-band sampling.** The proof path currently skips 892k of 16.777M LOD0 candidate cells (5.3%) and certifies no solid cells. Tighter intervals or smaller blocks may help. | Exact with conservative bounds, but current cell skips suggest limited headroom; smaller blocks also add classification overhead. The cell count is not a sample-savings measure. |
|
||||
| 5 | **Precomputed island distance volumes.** Could help if values are reused across tiles or LODs. | High memory and build cost. Exact stored lattice values preserve exactness at those points; interpolated values do not. Sparse hierarchical storage is an option only if the volume is sparse enough to benefit; [OpenVDB’s overview](https://www.openvdb.org/documentation/doxygen/overview.html) describes that trade space. |
|
||||
| 6 | **Per-tile 3D brick cache of exact island-local values.** | Exact, but likely low benefit for the one-pass grid, where each 3D lattice point is usually sampled once. It is more promising if measurement shows repeated mouth-filter, skirt, or overlapping-tile queries. |
|
||||
|
||||
The local index already mixes XY and island identity before selecting a set, so I would not label it a simple aliasing-stride bug without a set/reuse trace. Increasing associativity may help, but expands the per-worker cache and increases probe work. Keep the shared cache in the first A/B: it serves most local misses, and its measured lock wait is small; the atomic hit path remains unmeasured.
|
||||
|
||||
## Confirming measurement
|
||||
|
||||
Add one instrumented LOD0 run that tags FI field calls by caller—grid fill, skirt, passage filter, and boundary work—and separately records sampled cycles for `IslandColumnCache` local probes, shared reads/lock/build, and the remaining `FieldDensity` work. Add a small per-worker reuse/eviction trace for local misses, including set index and whether the key was seen before.
|
||||
|
||||
That will show whether the 27.6M full-set misses are soon-reused keys evicted by plane traversal, mostly one-off XY keys, or extra non-grid callers—and whether cache work or island/root evaluation consumes the 27.610 grid worker-s. No such attribution exists in L55, so the present cause ranking remains a hypothesis.
|
||||
@@ -0,0 +1,40 @@
|
||||
# R25 — Why FloatingIslands’ LOD0 grid is expensive
|
||||
|
||||
## Cache mechanics
|
||||
|
||||
`IslandColumnCache` in [FloatingIslandsInternal.h](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/FloatingIslandsInternal.h:497) is a **four-way set-associative** cache with 2,048 sets (8,192 entries) per worker. It hashes recipe and plan identity, island ID, seed, and integer X/Y. Equality also checks the island’s geometry parameters. The key deliberately has no Z or LOD.
|
||||
|
||||
On a miss, it uses the first invalid way; once the set is full, it replaces the way selected by bits of the key hash. It has no LRU policy. The worker cache is allocated lazily on first use and lasts until that thread exits ([ThreadLocalStorage.h](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/include/VoxelForgeCore/ThreadLocalStorage.h:9)). The shared second-level cache has 16,384 four-way sets (65,536 entries); once full, it replaces ways round-robin ([SharedColumnCache.h](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/include/VoxelForgeCore/SharedColumnCache.h:53)).
|
||||
|
||||
A FI “column” here is **per-island XY math**, not a stored density column or heightfield: delta X/Y, radius, angle, lobe phase, and top surface. The SDF still evaluates its Z-dependent shape, roots, detail, and connectors at each sample ([IslandDensity](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/FloatingIslandsInternal.h:776)).
|
||||
|
||||
The grid normally scans XY rows plane by plane across Z ([mesher traversal](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:1299)). Each XY/island key is reused vertically, but the full XY plane intervenes. Meanwhile, worker caches persist across tiles, and both cache levels have bounded, set-associative capacity. When a miss finds all four ways occupied, the counter records a conflict replacement. Thus, conflicts being nearly equal to misses means the caches were already warm and their selected sets were full; it does not by itself prove a direct-map aliasing stride. The code uses four ways, and adding Z to the key would defeat the intended vertical reuse.
|
||||
|
||||
## Cost split
|
||||
|
||||
L55 assigns **27.61 worker-s** to FI LOD0 grid work. The cache recorded 62.26M local requests: 34.58M hits and 27.68M misses. Of those misses, 23.95M hit the shared cache; only 3.73M caused a shared compute. The 151 ms of measured shared-lock wait is about **0.55%** of grid time. It does not include the cost of local probes, shared atomic reads, hashing, or comparisons, so the counters cannot give a reliable cache-time percentage.
|
||||
|
||||
The field path is also substantial: L55’s plan has four islands and three connectors. `RootCount` is five, so a full scan evaluates up to 20 root-lobe SDFs per density query, plus island contours and connector fields ([root count and scan](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/FloatingIslandsInternal.h:626), [field composition](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/FloatingIslandsInternal.h:1134)). The contour warp uses three Z-varying angular sine terms; there is no separately named helix evaluator in this path ([LobeOffsetWithPhase](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/FloatingIslandsInternal.h:401)).
|
||||
|
||||
There is no timer splitting field arithmetic from cache probes. The supported estimate is that cache **recomputation and lock waiting** are a small part of the cost, while the 62M lookup path may still be material. The code’s repeated root and feature evaluation is expensive, but an exact time fraction needs a direct measurement.
|
||||
|
||||
## Ranked fixes
|
||||
|
||||
The 37% LOD0 target requires saving about **10.2 worker-s**. These are ordinal estimates; no A/B timing supports precise savings yet.
|
||||
|
||||
| Rank | Fix | Expected saving | Risk and exactness |
|
||||
|---|---|---|---|
|
||||
| 1 | Use tile-scoped per-island candidates and tighten their bounds. The four-island plan marks candidate evaluation “not useful,” so sampling falls back to the full scan even though `FieldDensityForTile` has ordered group bounds ([tile-list hint](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/FloatingIslandsInternal.h:2211)). | Very high: a skipped group avoids five root SDFs and its feature work. | Medium risk; exact with conservative bounds and canonical composition order. |
|
||||
| 2 | Refine interval proofs adaptively around uncertain blocks. Current 8-cell proofs skip 892,416 air cells—about 6.2% of LOD0 candidates—and no solid cells. Large blocks and conservative bounds are plausible reasons for the low skip count; the logs do not report verdicts by feature or bound. | High if more empty cells can be certified. | Medium risk; exact when every skipped region has a valid sign proof. |
|
||||
| 3 | Evaluate coarse per-island bounds, then refine near the surface. | High potential for avoiding full-resolution work in air. | High risk if coarse values become final interpolated densities; exact only when they guide conservative proofs and all uncertain samples are refined. |
|
||||
| 4 | Replace hashed column lookup with a tile-local XY table, or increase local set count/associativity. The 27.68M local misses currently trigger shared lookups, most of which are shared hits. | Medium: reduces lookup/hash/atomic traffic, not the root and connector evaluations. | Low risk; exact. A direct XY table also matches what the cached values actually represent. |
|
||||
| 5 | Precompute per-island distance volumes. | Potentially high if reused across many tiles or LODs. | High memory and setup cost; exact at stored lattice points if canonical values are retained, approximate under interpolation. |
|
||||
| 6 | Add a per-tile 3D cache of island-local values. | Low to medium for this grid pass: each density-grid point is generally sampled once, and the mesher already stores final densities. | Exact if values are stored unchanged; adds memory and pays off only where other passes reuse those samples. |
|
||||
|
||||
Sparse 3D blocks are a well-established fit for localized, high-frequency volume data; OpenVDB documents hierarchical sparse storage and active narrow bands. That supports the structure as a design option, not a speedup prediction for this workload. [OpenVDB overview](https://www.openvdb.org/documentation/doxygen/overview.html)
|
||||
|
||||
## Confirming measurement
|
||||
|
||||
Add sampled cycle timings for the local probe, shared lookup, shared compute, and the island/root/connector field work, reported per tile and LOD. Pair those with set-occupancy or eviction-reuse-distance counts. This will show whether cache probes or SDF arithmetic dominate the 27.61 worker-s, and whether the full-set replacements evict keys needed again in the next Z plane.
|
||||
|
||||
Read-only review complete; no scripts, tests, builds, binaries, or source edits were made.
|
||||
Reference in new issue
Block a user