VoxelForge/Standalone/Research/WorldAssemblerM3/Linux10/REPORT.md

Linux10 — startup and region crossing

Result: no world-generation optimization was shipped. The dominant spawn gate is full shape compilation; the current Core API does not let tiles consume a partially built shape safely. L10 adds opt-in phase and crossing telemetry to the headless research path.

Runs used CPUs 0–11, normal priority, at most -j12, a 6 GiB address-space cap, and the unchanged powersave governor. Peak monitored RSS was 624,996 KiB (610.3 MiB). Final builds: Clang 19 with FP contraction off, SHA-256 7d5360e2baa637f8b2b4679df9616580bffe689afd101470e311260f0ecd4975; GCC 14 default, 5041cf15019766f453eb59fe8ff7fc4cd98ca6a904208b645e3d7ca4584e1195. The L9 Clang timing/hash baseline is Standalone/bin/linux/VoxelForgeCoreHeadless-linux9-clang, SHA-256 0cd3f0ef993ced2c615418c6e2b37b83f57eff6d024d7d1568c30ea51490384a.

1. Startup phases

SurfaceWorld seed 0, W12, no hash; five runs on the optimized Clang build. Wall and process CPU are medians. Process CPU sums work across threads. First-tile time is from cold start; its CPU column is CPU since pool start.

Phase	Wall ms	Process CPU ms	Threads used
Owner data load and resolve	5.05	5.05	1
Assembly plan build	51.04	51.03	1
Core snapshot	0.53	0.54	1
First desired-set build, 2,553 tiles	0.74	0.74	1
Shape request setup	0.03	0.03	1
Shape compile	222.59	856.79	caller + up to 8 scheduler workers
Streamer setup	0.01	0.01	1
Worker-pool start	0.41	1.27	12 requested; 8–11 had entered at print
First tile starts at 299.62 ms	—	1.59 since pool start	12/12 entered

Shape-compile wall ranged 215.89–324.88 ms; first tile started at 286.56–409.76 ms. Owner load, plan, local shape setup, and far-parent construction are serial. Before the pool starts, there are no sleeping stream workers, but most of the 12 allowed CPUs are unused. During the interval and detail-band phases the scheduler uses at most 8 workers, leaving up to 4 CPUs unused.

The phase-trace mode’s whole-view median was 1.436 s, versus 1.372 s with tracing off. The phase runs are for decomposition only; the performance ranking and A/B below use the optimized build with tracing disabled and hashing off. In that flag-off path, shape wall time was 0.222 s median, consistent with the phase trace.

2. Shape contents and first-tile dependencies

Shape phase	Wall ms	Process CPU ms	Work / concurrency
Local owner shapes and spatial bins	40.46	40.44	Extracts owner primitives; 1 thread
Rock intervals, signs, candidate bands	40.20	187.14	Packed column signs and candidate masks; up to 8 workers
Detail band and sign-edge distances	81.00	568.78	Applies coherent detail to candidate nodes and builds distance proxies; up to 8 workers
Route validation and shared-field setup	1.12	1.13	5,642 passage samples; then shared primitive form
Far parents and bins	55.56	55.57	6,050 far rooms / 18,348 links; 1 thread

The compile extracts 18,348 owner primitives (71 rooms, 74 tunnels, 2 passages), covers 138,188,413 sign nodes, allocates 36.71 MiB of packed words and 263.57 MiB of proxy address capacity, and visits 214,538 detail candidates (6,098 flips).

For this SurfaceWorld fixture, the first LOD0 tile’s density queries take the native Heightfield2D path before packed-shape lookup. Far-parent shapes are not needed by that first tile. However, World::generateTile rejects requests until the full base shape is marked built, and compileTileRegion also requires that base shape. So the current API gates the near tile on the interval, detail, validation, and far-parent work even when that tile does not sample those arrays. Other recipes and fallback/query paths do use the analytic primitives and packed signs/proxies.

There is no seed-independent owner shape payload to cache at build time. The assembly plan is built from owner data; region pages also re-run owner primitive extraction, spatial-bin construction, and shared-field packing. Those immutable owner-derived structures are a possible per-world reuse target, but the measured short crossings below built no pages, so this run does not price that work.

3. Moving crossing

Hashing off, W12, same camera and already-streamed initial view. Region compile was called, but all entering fine keys were covered by the initial shape bounds; it built zero regions.

Camera step	New tiles	LOD breakdown	Region compile	Tile work wall	Crossing total
8 m	144	LOD2: 128, LOD3: 16	0.010 ms	12.61 ms	12.81 ms
16 m	288	LOD0: 128, LOD1: 16, LOD2: 128, LOD3: 16	0.010 ms	60.49 ms	60.68 ms

The 16 m crossing’s worker-time sum was 633 ms; its wall time was 60.5 ms. No plan or full shape compile repeated. A 24 m attempt terminated with std::bad_alloc after base shape compilation and before crossing metrics; the failing allocation site is unknown.

4. Ranked levers

Rank	Lever	Upper bound from these measurements	Assessment
1	Overlap near-tile work with shape compilation	222.6 ms, 16.2% of 1.372 s whole-view-ready; first tile could start earlier by up to the compile time	Largest opportunity. Not built: tiles and region compilation currently require a fully built base shape. A safe version needs immutable partial-page publication and a wait before any tile reads an unfinished sign, proxy, primitive bin, or far field.
2	Defer far-parent construction until far tiles need it	55.6 ms, at most 4.1% of whole-view-ready and 18.6% of the first-tile delay	Could improve spawn, but far and near paths share mutable spatial-bin state today. Needs separate immutable bins and a synchronized readiness state; no crossing benefit was measured.
3	Parallelize assembly-plan construction	51.0 ms, at most 3.7% of whole-view-ready	Serial and seed/owner-specific. No safe deterministic parallel split was established.
4	Raise shape scheduler cap from 8 to 12	About 121 ms currently lies in the two worker phases; ideal linear scaling saves about 40 ms (2.9% of whole-view-ready)	Below the 3% bar before overhead, and no crossing benefit. Not built.

For any future partial or deferred build, hashes can fail if a tile observes a partially written sign/proxy array, if region publication races with tile lookup, or if shared bins change while a tile is sampling. The page must be immutable before publication; worker-count and timing tests must compare W1 with W12 and repeat W12. Any asynchronous result must also follow the Epoch/MPSC shutdown-drain rules.

5. Final-build timing and hashes

Five interleaved W12 runs, no hash. Pair values are whole_view_ready / stream_wall in seconds.

Pair	L9 7650abb Clang	L10 Clang	Delta, ms
1	1.336 / 1.050	1.346 / 1.060	+10 / +10
2	1.351 / 1.062	1.365 / 1.073	+14 / +11
3	1.359 / 1.073	1.374 / 1.083	+15 / +10
4	1.363 / 1.077	1.375 / 1.081	+12 / +4
5	1.387 / 1.089	1.372 / 1.092	−15 / +3
Median	1.359 / 1.073	1.372 / 1.081	+13 / +8

The measurement-only candidate is +1.0% whole-view-ready and +0.7% stream median; this is no demonstrated speedup. Hashing was off for these timings.

Final rebuilt binaries passed SurfaceWorld and FloatingIslands seed-0 hash checks at W1, W8, and five W12 runs per compiler: Clang FP contraction off and GCC default. Every full-detail, whole-view, LOD0–8, and exact-query row matches the compiler-matched L9 baseline. Audit files: Saved/Linux/l10_final_surface_hash_audit.txt and Saved/Linux/l10_islands_hash_audit.txt.

Recipe / scope	Density	Mesh	Payload	Topology
SurfaceWorld full detail	0x0ee0579bd15cbbe4	0x32e5d4964d610fee	0x1932b96ee4b1d910	0x69743c6bad3447dc
SurfaceWorld whole view	0xc10697a8f48380c7	0xe75c9119a1498cd5	0x49a9a8d1835a6c8c	0xedcb23288b5e8724
FloatingIslands full detail	0x6d842d2878b08ef6	0x4b868a3184f08035	0x2cedf2236788752d	0xeac17d5e0cb5044a
FloatingIslands whole view	0x5be4d8cde79b2a76	0x96f75041f6861b3a	0x3700a516cc89d50c	0x6b99948d6eab4f21

Exact-query hashes also match: SurfaceWorld density 0xe4d4a8eb95b35933, detail 0xefcba20c0c70c415, collision 0xe604823a249029bf; FloatingIslands density 0x61daded9c303becb, detail 0xefcba20c0c70c415, collision 0x84d37463be9b0573.

6. Failed or retried runs

• One phase smoke omitted the explicit owner-data path and resolved it under Standalone/bin/data; reran with Standalone/data/owner_world.json.
• A no-camera phase smoke produced 2,370 tiles instead of the 2,553-tile fixture; discarded it and reran with the fixed camera.
• A malformed --workers=w8 command exited during argument parsing; reran with numeric --workers=8.
• The first FloatingIslands matrix invocation omitted the three required streaming-mode switches. All 14 attempts exited before generation; corrected the command and all 14 final runs passed.
• The 24 m crossing attempt failed with std::bad_alloc as described above.
• The first hash-audit parser expected 13 rows; the actual audit contains 12 scopes. Corrected the parser and all final comparisons passed.

At spawn, the first tile starts about 300 ms after cold start, with roughly 223 ms in full shape compilation and 51 ms in the single-threaded plan; this round did not make that wait shorter. While moving, the 16 m crossing completed 288 entering tiles in about 61 ms and repeated no shape compile, so the measured delay was tile work rather than region setup.

Orchestrator review (Claude)

• Not committed: the L10 telemetry code. It is +1.0% whole-view median with tracing off (4 of 5 pairs slower), and the rule is zero cost when off. Saved as Saved/Linux/claude/l10-telemetry.patch (gitignored) for reuse; the tree is reset to 7650abb.
• Key finding, for L11: the first SurfaceWorld tiles take the native Heightfield2D path and don't read the cave shape, yet World::generateTile waits for the full base shape (~223 ms). A per-tile readiness check (start tiles whose recipe path provably never touches shape arrays) is lever #1 in safe form.
• Open bug, for L11: the 24 m crossing died with std::bad_alloc under the 6 GiB address-space cap (RLIMIT_AS). That cap counts reservations, not resident memory; L10 also reports 263.57 MiB of proxy address capacity. It is not yet known whether this is a real memory blow-up on larger crossings or the cap counting reserved address space. L7's 32 m step also hit the 160M-node interval bound.