VoxelForge/Standalone/Research/WorldAssemblerM3/Astra1/REPORT-pass2.md

Astra 1, pass 2 — whole-chain review (gpt-6-astra, effort medium, read-only), 2026-10-27

Claude verified: PackedDensityAt (ShapeWorldInternal.h ~5909) applies ApplyResolvedStrateComposition outside the L76 term scope; FScopedDensityQueryProfile times whole invocations crossing a 256-point boundary (batch-biased).

Stage-by-stage cost map

[MEASURED] The L73 stage timings below are historical, before the later skips. They describe where that run spent time; they cannot be subtracted from L76 term timings to produce a current residual. I also inspected the saved L76 results without executing generation.

[CODE] A major architectural detail: the “compiled-analytic” world already contains precomputed packed signs and quantized surface-distance proxies. Many lattice queries never enter AnalyticDensityAt.

Stage	Work and frequency	Cost and necessity
World/plan resolution	[CODE] Headless loads the owner export, resolves/validates its layout, constructs a snapshot, then constructs World. Once per cold run. UE’s serialization entry is VoxelForgeCoreAdapter.cpp:2062; headless startup begins at StreamBenchmarkHeadless.cpp:1659.	[MEASURED] Saved L76 FI owner-data loading takes about 25 ms. No complete isolated resolution cost is supplied. [INFERENCE] Necessary, but not a plausible home for worker time measured inside tile-field queries.
Shape compilation	[CODE] World::compile builds primitives/bins, packed rock intervals, candidate bands, detailed distance proxies, sign-edge repairs, passage validation and far parents. Once for the base shape; additional pages when needed. VoxelForgeCore.cpp:1394, ShapeWorldInternal.h:2724.	[MEASURED] Saved L76 FI: 313 ms wall, including approximately 48 ms shapes, 64 ms intervals/signs, 62 ms surface band and 119 ms far parents. [INFERENCE] Important to a one-second cold budget, but separate from the missing tile-field attribution. It already amortizes substantial base-field work.
Tile-key selection	[CODE] desiredTiles builds clipmap keys, vertical bounds and additional far surface columns. Once initially, then on streaming-set changes; about 2,250 tiles for FI/TN. VoxelForgeCore.cpp:1904.	[INFERENCE] Thousands of key/set operations, not millions of field evaluations. No measured isolated cost. Necessary selection; exact tile count remains a design choice.
Whole-tile classification	[CODE] Each assembly tile calls ClassifyBoxDetailed, including a one-step halo. It delegates to ClassifyBoundsDetailed, with conditional slicing and conservative vetoes. VoxelForgeCore.cpp:2377, ShapeWorldInternal.h:6385.	[MEASURED] L73: only 4/3 ms total FI/TN, roughly 2/1 μs per tile, but ineffective legacy skips then. These are not current classifier costs. [INFERENCE] Necessary only when avoided work pays for it; retain current profitable proofs.
Density dispatch and base evaluation	[CODE] Scalar lattice queries probe native/legacy-surface paths, then choose packed lookup or analytic evaluation. Batches have additional fast paths. ShapeWorldInternal.h:3475, :3602, :5909.	[MEASURED] L73’s approximately 52M calls and 39.9/33.2 field worker-seconds imply roughly 0.77/0.64 μs per counted point, averaged over heterogeneous paths. [INFERENCE] Fetching/evaluating the base value is necessary; repeated dispatch is not inherently necessary.
Composer	[CODE] Packed and scalar analytic paths both call ApplyResolvedStrateComposition: cavern graph → boundary → passage/support → stamps → compatibility floor caps. ShapeWorldInternal.h:1993. Frequency depends on the actual route, not just the strate label.	[MEASURED] L76 measures approximately 6.5/5.5 worker-seconds only within its eligible analytic scopes. [CODE] Its passage/support bucket also includes ApplyResolvedCavernFloorCaps at :2053. [INFERENCE] Composition is necessary where influential; universal invocation is not.
Grid fill	[CODE] One cached lattice per generated tile: typically 35³ = 42,875 points at LOD0/1, including halo. Scalar, X8 or Z-run traversal fills it. Storage is thread-local and reused. Standalone mesher:430, :957, :1296.	[MEASURED] L73: 36.14/29.37 s, approximately 16.1/13.2 ms per visited tile, including evaluation. [INFERENCE] Dense fill is avoidable in proven-uniform regions. This is the primary workload, not merely array writes.
Marching cubes	[CODE] Classifies cells using cached corners, computes geometry only for surface cells, interpolates edges and deduplicates vertices. LOD0 midpoint correction can issue extra unclamped queries. Mesher:1879, :1943.	[MEASURED] L73 MC aggregate: 40.58/32.63 s, including grid work. Subtracting grid gives about 4.44/3.26 s, not another 40 seconds. [INFERENCE] Necessary extraction, but replacing MC is not the first lever.
Normals	[CODE] Central differences normally read the existing grid. At Step 1, proxy-band fallback values trigger DensityAtUnclamped; optional gradient memoization avoids repeated corner gradients. Mesher:1654.	[MEASURED] L73: 0.306/0.246 s, about 0.14/0.11 ms per visited tile. [INFERENCE] Some repeated field queries exist, but six new field evaluations per vertex is not the general path. Normals cannot plausibly explain 30 seconds under these measurements.
Skirts	[CODE] Boundary-edge detection, extrusion and density probes prevent unsuitable skirt geometry. Work scales with boundary geometry, not tile volume. Mesher:2111.	[MEASURED] L73: 1.079/1.091 s, approximately 0.48/0.49 ms per visited tile. [INFERENCE] Necessary for the current seam solution; bounded opportunity, not the missing dominant cost.
Ground/cap refinement and far shells	[CODE] Far stratum/connector repairs and native ground/cap refinement run under eligibility gates, with refinement at LOD≥4 and no overlapping edits. VoxelForgeCore.cpp:2792, :2875.	[MEASURED] L73’s combined “Other,” including these and assembly, is 2.86/2.78 s. [INFERENCE] Required by the current far representation, but partly corrective work that a better far representation could avoid. Not an explanation for missing grid time.
Payload	[CODE] Core assembles material/vertex/index outputs; the headless wrapper converts vertices and moves indices/masks. Per output vertex/index, once per tile. VoxelForgeCore.cpp:2969, StreamingWorld.h:934.	[MEASURED] The separately timed headless conversion is 12 ms total, about 5 μs per tile. [INFERENCE] This does not include all Core assembly or Unreal upload/collision costs. It is not a useful primary target.
Unreal publication	[CODE] Worker generation → prepared streams → result queue → budgeted game-thread component/upload/section work → asynchronous collision completion. VoxelWorld.cpp:13303, :9400, :14010, :14279.	[MEASURED] SW has 1.33 s additional readiness latency. [INFERENCE] That is not 1.33 s of CPU publication work. Per-tile apply/cook costs are not established here. Essential to the target, but cannot explain headless field time.

Where the “unattributed 30 seconds” most plausibly went

[CODE] First: packed-field composition is outside L76’s measurement scope. The relevant chain is:

DensityAtLattice → PackedDensityAt → ApplyResolvedStrateComposition.

PackedDensityAt reads RockWords, CandidateWords and possibly SurfaceDistanceQ, then applies composition. It never enters the FScopedLegacyFieldTermScope created later inside AnalyticDensityAt at ShapeWorldInternal.h:5967.

[CODE] Term timers immediately return when that scope is inactive: ShapeWorldInternal.h:2275. Thus an expensive passage query can be counted by the outer field profiler but entirely absent from L76’s passage timing.

[CODE] DensityDispatchPathAt returns “compiled-analytic/FloatingIslands” from the bound generator kind without distinguishing packed from analytic execution: ShapeWorldInternal.h:3455. That label is insufficient for performance attribution.

[CODE] There are further differences:

• X8 can return packed proxy values directly without invoking the scalar composer; other conditions make it fall back to scalar calls. ShapeWorldInternal.h:3675
• FI LOD0 preferentially uses scalar vertical traversal. Mesher:1288
• Z-run dispatch can fall back to scalar lattice queries, including packed queries. Its analytic term scope is reached only after those fallbacks. ShapeWorldInternal.h:11460
• Native/surface branches and non-FI/TN samples are excluded from the legacy-term population.

[INFERENCE] Therefore, the correction establishes that 6.5 seconds is measured passage cost within one subset, not that total passage cost is only 15%. Packed composition is my leading explanation for substantial uninstrumented work. I cannot assign it 30 seconds from source alone.

[MEASURED] Second: the residual mixes different generations of the workload. Existing L76 saved results contain:

Same saved L76 run	FI	TN
Density point calls	26.99M	25.13M
Eligible legacy points	6.01M	5.56M
Reported field estimate	18.19 worker-s	15.49 worker-s
Tile-worker total	33.80 worker-s	28.56 worker-s
Whole-view wall	4.621 s	3.977 s

[INFERENCE] The approximately 46M unseen calls belong to subtracting L76’s eligible count from L73’s pre-skip population. They are not an established current population. Likewise, 40−measured_terms is not a coherent current residual.

[CODE] Third: the aggregate field estimator can be biased by batching. FScopedDensityQueryProfile samples when the point sequence crosses a multiple of 256, timing the entire invocation and recording its point count. The reducer pools sampled time/points and scales by all points. See ShapeWorldInternal.h:2355 and StreamBenchmarkPipeline.h:5611.

[INFERENCE] A batch of eight has approximately eight times a scalar invocation’s chance of being selected, and contributes eight sampled points when selected. Pooling both without inclusion-probability correction overrepresents batch points. If packed batches are cheap and scalar composition is expensive, the estimate can substantially understate field time. Periodic sampling also risks correlation with spatial traversal.

[MEASURED] This deserves attention: L76 FI LOD0 records 12.70 s grid fill but only 5.96 s estimated field evaluation. Conversely, LOD3 estimates 3.04 s field time against 2.58 s total tile-worker time. Those are diagnostic inconsistencies, not proof of where the discrepancy lies. Saved log

[INFERENCE] Lower-ranked explanations:

• Dispatch/lookups: real repeated work. Stratum lookup uses binary search and coordinate conversion, sometimes repeatedly per sample: ShapeWorldInternal.h:437. But attributing 30 seconds to this alone means roughly 0.65 μs per purported 46M calls. That needs evidence; hidden composer execution is more concrete.
• Allocation/cache misses: possible contributors, but grid allocation is reused and packed access is direct indexing. There is no evidence of one heap allocation per ordinary density point.
• Halos/neighbor duplication: significant but bounded. 35³−33³ = 6,938, or 16.2% of a dense tile’s samples, is the gradient halo. Including shared faces, an ideal large same-LOD region could reduce 35³ toward 32³ unique samples per tile: approximately 23.6% savings before edges and synchronization. This multiplies existing evaluation cost; it does not create a separate unmeasured stage.
• Normals/midpoints/skirts: issue additional calls, but their measured stage budgets exclude them as the dominant explanation. Cross-LOD duplication remains unquantified.

Cheapest decisive measurement

[INFERENCE] First take a sampling call-stack profile of the clean production run, focused on grid fill. It requires no new per-sample timers and should immediately distinguish packed composition from dispatch or memory stalls.

[INFERENCE] The next instrumentation change should be a mutually exclusive dispatch census, not another individual-term profiler. Count and sample time for packed-scalar, packed-X8, analytic-scalar, analytic-batch, native and legacy-surface routes, split by LOD and caller stage. Measure composition inside packed-scalar explicitly.

Use equal-probability invocation sampling with correct expansion, or stratify by route and batch length. Avoid nested double-counting; compare the resulting total against directly timed grid fill. Separate passage carving from compatibility floor caps.

[INFERENCE] This falsifies my leading explanation if packed-scalar composition occupies little time. Only then prioritize dispatch, cache misses or other overhead.

Revised ranked recommendations

1. Establish complete attribution, then specialize the dominant lattice route.

[INFERENCE] This replaces my unconditional “passages first” ranking. The first implementation round should produce a closed cost account and identify which route dominates LOD0/1.

If packed composition dominates, give each tile/brick a stable local composition program and an identity fast path. Extend FScopedPassageTileQueries, PackedDensityAt and ApplyResolvedStrateComposition. Hoist Z-only dispatch decisions into plane/run contexts. Before copying the X8 shortcut into scalar traversal, establish why their composition behavior differs and which semantics are required.

[INFERENCE] Effort: small for attribution, medium for specialization. Risks: incorrect support envelopes, changed composition order, scalar/batch disagreement. Preserve peer determinism and audit clearance, support and grade; old-version hashes remain an exactness diagnostic.

[MEASURED → INFERENCE] Corrected passage arithmetic: using your historical budget strictly as an illustration:

• FI, removing 80–90% of 6.5/46 worker-seconds: 4.6 × (1−0.8…0.9 × 6.5/46) = 4.08–4.02 s.
• TN, removing 80–90% of 5.5/39: approximately 3.81–3.75 s.

These replace my erroneous approximately two-second estimates. They are not forecasts because the operands span rounds. Using the contemporaneous saved L76 totals instead gives approximately 3.91–3.83 s FI, 3.37–3.29 s TN, under proportional scaling.

[INFERENCE] Total passage savings could be larger if the missing packed-route work is substantial. Measure it before assigning a percentage.

2. Remove repeated composed sampling; retain MC.

[INFERENCE] Keep the retained-proof/local-program recommendation, but build on the existing packed representation. Priorities:

• Retain successful slice/brick proofs rather than requiring a uniform whole-tile verdict.
• Reuse final composed lattice samples across neighboring tiles where inputs and semantics match.
• Evaluate exact surface endpoints and normal stencils; leave certified interiors implicit.

The relevant changes remain ClassifyBoundsDetailed, World::generateTile and the standalone mesher’s grid fill. Sample-cache keys must include evaluator version, edits and relevant resolution/context—not coordinates alone.

[INFERENCE] An alternative worth testing is applying composition once to a shared near-region lattice. Count its cold construction cost: moving the same work earlier is not a speedup. It wins through deduplication, localization or cheaper batching.

[INFERENCE] Effort: medium-to-large. Risks: stale edit dependencies, seams and false uniform proofs. Cheap falsifier: one representative region spanning adjacent tiles and a strate boundary; compare total construction plus meshing cost against independent tiles.

[INFERENCE] Withdraw the earlier combined one-second projection. At unchanged throughput, FI’s current 4.6 seconds requires approximately 78% critical-path reduction. Border sharing alone cannot supply that; broad sampling elimination still has to prove its economics.

3. Preserve a separate arrival deadline and reduce end-to-end latency.

[INFERENCE] Keep dependency-aware scheduling through collision publication, rather than merely increasing queue depth. The existing near-priority and hot-swap mechanisms already provide the foundation.

Shape compilation’s approximately 0.31 seconds matters once generation approaches one second. Far repair/refinement becomes material too. Optimize these after the dominant grid route is understood; do not hide their cost outside the cold timer.

[MEASURED → INFERENCE] SW’s scheduling gap remains a possible 0–1.33-second latency opportunity, not a guaranteed CPU saving. Scheduling cannot remove required whole-view work.

[INFERENCE] Effort: medium. Risks: starving far coverage or publishing visuals before safe collision. Cheap falsifier: an arrival-only admission experiment plus prerequisite/publication tracing. If its uncontended critical path exceeds one second, scheduling alone cannot meet arrival readiness.

[INFERENCE] GPU porting, SIMD, native-generator replacement and a new mesher stay below these three. The newly visible distinction is between cheap cached base density and potentially expensive repeated composition, not simply “52M analytic evaluations.”

Read-only inspection completed; no edits, builds, tests, commits or stashes.