VoxelForge/Standalone/Research/WorldAssemblerM3/Astra1/REPORT-pass2.md

# Astra 1, pass 2 — whole-chain review (gpt-6-astra, effort medium, read-only), 2026-10-27

Claude verified: PackedDensityAt (ShapeWorldInternal.h ~5909) applies ApplyResolvedStrateComposition outside the L76 term scope; FScopedDensityQueryProfile times whole invocations crossing a 256-point boundary (batch-biased).

**Stage-by-stage cost map**

[MEASURED] The L73 stage timings below are historical, before the later skips. They describe where that run spent time; they cannot be subtracted from L76 term timings to produce a current residual. I also inspected the saved L76 results without executing generation.

[CODE] A major architectural detail: the “compiled-analytic” world **already contains precomputed packed signs and quantized surface-distance proxies**. Many lattice queries never enter `AnalyticDensityAt`.

| Stage | Work and frequency | Cost and necessity |
|---|---|---|
| **World/plan resolution** | [CODE] Headless loads the owner export, resolves/validates its layout, constructs a snapshot, then constructs `World`. Once per cold run. UE’s serialization entry is [`VoxelForgeCoreAdapter.cpp:2062`](/home/jahni/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelForgeCoreAdapter.cpp:2062); headless startup begins at [`StreamBenchmarkHeadless.cpp:1659`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/StreamBenchmarkHeadless.cpp:1659). | [MEASURED] Saved L76 FI owner-data loading takes about **25 ms**. No complete isolated resolution cost is supplied. [INFERENCE] Necessary, but not a plausible home for worker time measured inside tile-field queries. |
| **Shape compilation** | [CODE] `World::compile` builds primitives/bins, packed rock intervals, candidate bands, detailed distance proxies, sign-edge repairs, passage validation and far parents. Once for the base shape; additional pages when needed. [`VoxelForgeCore.cpp:1394`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/VoxelForgeCore.cpp:1394), [`ShapeWorldInternal.h:2724`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:2724). | [MEASURED] Saved L76 FI: **313 ms wall**, including approximately 48 ms shapes, 64 ms intervals/signs, 62 ms surface band and 119 ms far parents. [INFERENCE] Important to a one-second cold budget, but separate from the missing tile-field attribution. It already amortizes substantial base-field work. |
| **Tile-key selection** | [CODE] `desiredTiles` builds clipmap keys, vertical bounds and additional far surface columns. Once initially, then on streaming-set changes; about 2,250 tiles for FI/TN. [`VoxelForgeCore.cpp:1904`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/VoxelForgeCore.cpp:1904). | [INFERENCE] Thousands of key/set operations, not millions of field evaluations. No measured isolated cost. Necessary selection; exact tile count remains a design choice. |
| **Whole-tile classification** | [CODE] Each assembly tile calls `ClassifyBoxDetailed`, including a one-step halo. It delegates to `ClassifyBoundsDetailed`, with conditional slicing and conservative vetoes. [`VoxelForgeCore.cpp:2377`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/VoxelForgeCore.cpp:2377), [`ShapeWorldInternal.h:6385`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:6385). | [MEASURED] L73: only **4/3 ms total FI/TN**, roughly **2/1 μs per tile**, but ineffective legacy skips then. These are not current classifier costs. [INFERENCE] Necessary only when avoided work pays for it; retain current profitable proofs. |
| **Density dispatch and base evaluation** | [CODE] Scalar lattice queries probe native/legacy-surface paths, then choose packed lookup or analytic evaluation. Batches have additional fast paths. [`ShapeWorldInternal.h:3475`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:3475), [`:3602`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:3602), [`:5909`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:5909). | [MEASURED] L73’s approximately 52M calls and 39.9/33.2 field worker-seconds imply roughly **0.77/0.64 μs per counted point**, averaged over heterogeneous paths. [INFERENCE] Fetching/evaluating the base value is necessary; repeated dispatch is not inherently necessary. |
| **Composer** | [CODE] Packed and scalar analytic paths both call `ApplyResolvedStrateComposition`: cavern graph → boundary → passage/support → stamps → compatibility floor caps. [`ShapeWorldInternal.h:1993`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:1993). Frequency depends on the actual route, not just the strate label. | [MEASURED] L76 measures approximately **6.5/5.5 worker-seconds** only within its eligible analytic scopes. [CODE] Its passage/support bucket also includes `ApplyResolvedCavernFloorCaps` at [`:2053`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:2053). [INFERENCE] Composition is necessary where influential; universal invocation is not. |
| **Grid fill** | [CODE] One cached lattice per generated tile: typically **35³ = 42,875** points at LOD0/1, including halo. Scalar, X8 or Z-run traversal fills it. Storage is thread-local and reused. [`Standalone mesher:430`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:430), [`:957`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:957), [`:1296`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:1296). | [MEASURED] L73: **36.14/29.37 s**, approximately **16.1/13.2 ms per visited tile**, including evaluation. [INFERENCE] Dense fill is avoidable in proven-uniform regions. This is the primary workload, not merely array writes. |
| **Marching cubes** | [CODE] Classifies cells using cached corners, computes geometry only for surface cells, interpolates edges and deduplicates vertices. LOD0 midpoint correction can issue extra unclamped queries. [`Mesher:1879`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:1879), [`:1943`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:1943). | [MEASURED] L73 MC aggregate: **40.58/32.63 s**, including grid work. Subtracting grid gives about **4.44/3.26 s**, not another 40 seconds. [INFERENCE] Necessary extraction, but replacing MC is not the first lever. |
| **Normals** | [CODE] Central differences normally read the existing grid. At Step 1, proxy-band fallback values trigger `DensityAtUnclamped`; optional gradient memoization avoids repeated corner gradients. [`Mesher:1654`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:1654). | [MEASURED] L73: **0.306/0.246 s**, about **0.14/0.11 ms per visited tile**. [INFERENCE] Some repeated field queries exist, but six new field evaluations per vertex is not the general path. Normals cannot plausibly explain 30 seconds under these measurements. |
| **Skirts** | [CODE] Boundary-edge detection, extrusion and density probes prevent unsuitable skirt geometry. Work scales with boundary geometry, not tile volume. [`Mesher:2111`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:2111). | [MEASURED] L73: **1.079/1.091 s**, approximately **0.48/0.49 ms per visited tile**. [INFERENCE] Necessary for the current seam solution; bounded opportunity, not the missing dominant cost. |
| **Ground/cap refinement and far shells** | [CODE] Far stratum/connector repairs and native ground/cap refinement run under eligibility gates, with refinement at LOD≥4 and no overlapping edits. [`VoxelForgeCore.cpp:2792`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/VoxelForgeCore.cpp:2792), [`:2875`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/VoxelForgeCore.cpp:2875). | [MEASURED] L73’s combined “Other,” including these and assembly, is **2.86/2.78 s**. [INFERENCE] Required by the current far representation, but partly corrective work that a better far representation could avoid. Not an explanation for missing grid time. |
| **Payload** | [CODE] Core assembles material/vertex/index outputs; the headless wrapper converts vertices and moves indices/masks. Per output vertex/index, once per tile. [`VoxelForgeCore.cpp:2969`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/VoxelForgeCore.cpp:2969), [`StreamingWorld.h:934`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/StreamingWorld.h:934). | [MEASURED] The separately timed headless conversion is **12 ms total**, about **5 μs per tile**. [INFERENCE] This does not include all Core assembly or Unreal upload/collision costs. It is not a useful primary target. |
| **Unreal publication** | [CODE] Worker generation → prepared streams → result queue → budgeted game-thread component/upload/section work → asynchronous collision completion. [`VoxelWorld.cpp:13303`](/home/jahni/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:13303), [`:9400`](/home/jahni/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:9400), [`:14010`](/home/jahni/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:14010), [`:14279`](/home/jahni/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:14279). | [MEASURED] SW has **1.33 s additional readiness latency**. [INFERENCE] That is not 1.33 s of CPU publication work. Per-tile apply/cook costs are not established here. Essential to the target, but cannot explain headless field time. |

**Where the “unattributed 30 seconds” most plausibly went**

[CODE] **First: packed-field composition is outside L76’s measurement scope.** The relevant chain is:

`DensityAtLattice` → `PackedDensityAt` → `ApplyResolvedStrateComposition`.

`PackedDensityAt` reads `RockWords`, `CandidateWords` and possibly `SurfaceDistanceQ`, then applies composition. It never enters the `FScopedLegacyFieldTermScope` created later inside `AnalyticDensityAt` at [`ShapeWorldInternal.h:5967`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:5967).

[CODE] Term timers immediately return when that scope is inactive: [`ShapeWorldInternal.h:2275`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:2275). Thus an expensive passage query can be counted by the outer field profiler but entirely absent from L76’s passage timing.

[CODE] `DensityDispatchPathAt` returns “compiled-analytic/FloatingIslands” from the bound generator kind without distinguishing packed from analytic execution: [`ShapeWorldInternal.h:3455`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:3455). That label is insufficient for performance attribution.

[CODE] There are further differences:

- X8 can return packed proxy values directly without invoking the scalar composer; other conditions make it fall back to scalar calls. [`ShapeWorldInternal.h:3675`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:3675)
- FI LOD0 preferentially uses scalar vertical traversal. [`Mesher:1288`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:1288)
- Z-run dispatch can fall back to scalar lattice queries, including packed queries. Its analytic term scope is reached only after those fallbacks. [`ShapeWorldInternal.h:11460`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:11460)
- Native/surface branches and non-FI/TN samples are excluded from the legacy-term population.

[INFERENCE] Therefore, the correction establishes that **6.5 seconds is measured passage cost within one subset**, not that total passage cost is only 15%. Packed composition is my leading explanation for substantial uninstrumented work. I cannot assign it 30 seconds from source alone.

[MEASURED] **Second: the residual mixes different generations of the workload.** Existing [L76 saved results](/home/jahni/VoxelM/Plugins/VoxelForge/Saved/Linux/l76/term-profile-before/summary.json) contain:

| Same saved L76 run | FI | TN |
|---|---:|---:|
| Density point calls | 26.99M | 25.13M |
| Eligible legacy points | 6.01M | 5.56M |
| Reported field estimate | 18.19 worker-s | 15.49 worker-s |
| Tile-worker total | 33.80 worker-s | 28.56 worker-s |
| Whole-view wall | 4.621 s | 3.977 s |

[INFERENCE] The approximately 46M unseen calls belong to subtracting L76’s eligible count from L73’s pre-skip population. They are not an established current population. Likewise, `40−measured_terms` is not a coherent current residual.

[CODE] **Third: the aggregate field estimator can be biased by batching.** `FScopedDensityQueryProfile` samples when the point sequence crosses a multiple of 256, timing the entire invocation and recording its point count. The reducer pools sampled time/points and scales by all points. See [`ShapeWorldInternal.h:2355`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:2355) and [`StreamBenchmarkPipeline.h:5611`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/StreamBenchmarkPipeline.h:5611).

[INFERENCE] A batch of eight has approximately eight times a scalar invocation’s chance of being selected, and contributes eight sampled points when selected. Pooling both without inclusion-probability correction overrepresents batch points. If packed batches are cheap and scalar composition is expensive, the estimate can substantially understate field time. Periodic sampling also risks correlation with spatial traversal.

[MEASURED] This deserves attention: L76 FI LOD0 records **12.70 s grid fill but only 5.96 s estimated field evaluation**. Conversely, LOD3 estimates **3.04 s field time against 2.58 s total tile-worker time**. Those are diagnostic inconsistencies, not proof of where the discrepancy lies. [Saved log](/home/jahni/VoxelM/Plugins/VoxelForge/Saved/Linux/l76/term-profile-before/strate1-w8.log:12880)

[INFERENCE] **Lower-ranked explanations:**

- **Dispatch/lookups:** real repeated work. Stratum lookup uses binary search and coordinate conversion, sometimes repeatedly per sample: [`ShapeWorldInternal.h:437`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:437). But attributing 30 seconds to this alone means roughly **0.65 μs per purported 46M calls**. That needs evidence; hidden composer execution is more concrete.
- **Allocation/cache misses:** possible contributors, but grid allocation is reused and packed access is direct indexing. There is no evidence of one heap allocation per ordinary density point.
- **Halos/neighbor duplication:** significant but bounded. `35³−33³ = 6,938`, or **16.2%** of a dense tile’s samples, is the gradient halo. Including shared faces, an ideal large same-LOD region could reduce `35³` toward `32³` unique samples per tile: approximately **23.6%** savings before edges and synchronization. This multiplies existing evaluation cost; it does not create a separate unmeasured stage.
- **Normals/midpoints/skirts:** issue additional calls, but their measured stage budgets exclude them as the dominant explanation. Cross-LOD duplication remains unquantified.

**Cheapest decisive measurement**

[INFERENCE] First take a sampling call-stack profile of the clean production run, focused on grid fill. It requires no new per-sample timers and should immediately distinguish packed composition from dispatch or memory stalls.

[INFERENCE] The next instrumentation change should be a **mutually exclusive dispatch census**, not another individual-term profiler. Count and sample time for packed-scalar, packed-X8, analytic-scalar, analytic-batch, native and legacy-surface routes, split by LOD and caller stage. Measure composition inside packed-scalar explicitly.

Use equal-probability invocation sampling with correct expansion, or stratify by route **and batch length**. Avoid nested double-counting; compare the resulting total against directly timed grid fill. Separate passage carving from compatibility floor caps.

[INFERENCE] This falsifies my leading explanation if packed-scalar composition occupies little time. Only then prioritize dispatch, cache misses or other overhead.

**Revised ranked recommendations**

**1. Establish complete attribution, then specialize the dominant lattice route.**

[INFERENCE] This replaces my unconditional “passages first” ranking. The first implementation round should produce a closed cost account and identify which route dominates LOD0/1.

If packed composition dominates, give each tile/brick a stable local composition program and an identity fast path. Extend `FScopedPassageTileQueries`, `PackedDensityAt` and `ApplyResolvedStrateComposition`. Hoist Z-only dispatch decisions into plane/run contexts. Before copying the X8 shortcut into scalar traversal, establish why their composition behavior differs and which semantics are required.

[INFERENCE] Effort: small for attribution, medium for specialization. Risks: incorrect support envelopes, changed composition order, scalar/batch disagreement. Preserve peer determinism and audit clearance, support and grade; old-version hashes remain an exactness diagnostic.

[MEASURED → INFERENCE] **Corrected passage arithmetic:** using your historical budget strictly as an illustration:

- FI, removing 80–90% of **6.5/46** worker-seconds: `4.6 × (1−0.8…0.9 × 6.5/46)` = **4.08–4.02 s**.
- TN, removing 80–90% of **5.5/39**: approximately **3.81–3.75 s**.

These replace my erroneous approximately two-second estimates. They are not forecasts because the operands span rounds. Using the contemporaneous saved L76 totals instead gives approximately **3.91–3.83 s FI**, **3.37–3.29 s TN**, under proportional scaling.

[INFERENCE] Total passage savings could be larger if the missing packed-route work is substantial. Measure it before assigning a percentage.

**2. Remove repeated composed sampling; retain MC.**

[INFERENCE] Keep the retained-proof/local-program recommendation, but build on the existing packed representation. Priorities:

- Retain successful slice/brick proofs rather than requiring a uniform whole-tile verdict.
- Reuse final composed lattice samples across neighboring tiles where inputs and semantics match.
- Evaluate exact surface endpoints and normal stencils; leave certified interiors implicit.

The relevant changes remain `ClassifyBoundsDetailed`, `World::generateTile` and the standalone mesher’s grid fill. Sample-cache keys must include evaluator version, edits and relevant resolution/context—not coordinates alone.

[INFERENCE] An alternative worth testing is applying composition once to a shared near-region lattice. Count its cold construction cost: moving the same work earlier is not a speedup. It wins through deduplication, localization or cheaper batching.

[INFERENCE] Effort: medium-to-large. Risks: stale edit dependencies, seams and false uniform proofs. Cheap falsifier: one representative region spanning adjacent tiles and a strate boundary; compare total construction plus meshing cost against independent tiles.

[INFERENCE] Withdraw the earlier combined one-second projection. At unchanged throughput, FI’s current 4.6 seconds requires approximately **78% critical-path reduction**. Border sharing alone cannot supply that; broad sampling elimination still has to prove its economics.

**3. Preserve a separate arrival deadline and reduce end-to-end latency.**

[INFERENCE] Keep dependency-aware scheduling through collision publication, rather than merely increasing queue depth. The existing near-priority and hot-swap mechanisms already provide the foundation.

Shape compilation’s approximately 0.31 seconds matters once generation approaches one second. Far repair/refinement becomes material too. Optimize these after the dominant grid route is understood; do not hide their cost outside the cold timer.

[MEASURED → INFERENCE] SW’s scheduling gap remains a possible **0–1.33-second** latency opportunity, not a guaranteed CPU saving. Scheduling cannot remove required whole-view work.

[INFERENCE] Effort: medium. Risks: starving far coverage or publishing visuals before safe collision. Cheap falsifier: an arrival-only admission experiment plus prerequisite/publication tracing. If its uncontended critical path exceeds one second, scheduling alone cannot meet arrival readiness.

[INFERENCE] GPU porting, SIMD, native-generator replacement and a new mesher stay below these three. The newly visible distinction is between **cheap cached base density and potentially expensive repeated composition**, not simply “52M analytic evaluations.”

Read-only inspection completed; no edits, builds, tests, commits or stashes.