VoxelForge/Standalone/Research/WorldAssemblerM3/Linux7/REPORT.md

Linux7 — shared terrain columns

Decision: keep the shared second tier. On the L1 view it cut median W12 whole-view time by 19.0%, well above the 3% threshold, and the 8 m camera crossing was also faster. All final output hashes match the 9898e05 binary.

Runs used the linux6 base, Clang 19 with FP contraction off, GCC 14 defaults, CPUs 0–11, normal priority, a 6 GiB address-space cap, and the unchanged powersave governor. Static timings below are three alternating base/candidate W12 runs of 2,553 tiles.

1. What was recomputed

Three complete W12 captures per mode on the L1 view. Counts below are medians of actual column formula executions after worker-local cache misses, grouped by LOD. “Same worker” means the worker had computed that XY before and later missed its local entry. Halo/normal repeats are separated before worker attribution; the parenthesized value is the total computed halo/normal probes, including unique ones. Each field's median is independent, so row medians can differ slightly from their sums.

LOD	Base total / unique XY	Same worker	Different worker	Halo repeats / probes	Shared total / unique XY
0	556,134 / 128,240	22,307	358,908	46,679 / 70,972	135,720 / 127,370
1	452,626 / 104,284	16,393	287,791	39,245 / 59,082	85,658 / 85,479
2	144,251 / 32,274	14,667	74,377	22,933 / 30,942	28,019 / 28,019
3	91,158 / 33,256	0	46,387	11,564 / 20,339	16,213 / 16,213
4	272,776 / 91,325	100	167,083	14,266 / 28,089	90,105 / 90,065
5	178,511 / 71,957	1	98,687	7,892 / 20,042	70,932 / 70,930
6	342,763 / 280,334	20	56,204	6,205 / 19,977	279,266 / 279,202
7	79,400 / 71,468	1	4,951	2,980 / 14,329	70,492 / 70,492
8	137,062 / 106,219	0	22,138	8,704 / 22,609	106,219 / 106,219

Across the per-LOD medians, the base made 2.255 million formula evaluations: 919 thousand unique XYs and 1.336 million repeats. Of the repeats, 83.6% were from another worker, 4.0% were same-worker evictions, and 12.0% were halo/normal probes. The shared candidate made 883 thousand evaluations, 60.9% fewer; about 8.6 thousand repeats remained. This is mostly a worker-sharing problem, not a local-cache-capacity problem.

2. Design

SharedColumnCache.h is a bounded, four-way second-tier value cache used by both Heightfield2D and FloatingIslands. Its key includes recipe and plan pointer identity plus content hash, exact XY, and an optional exact sub-domain identity; FloatingIslands supplies the island stable ID because different anchors can use the same XY. LOD is omitted because the column value is LOD-independent. The heightfield table has 524,288 slots and the island table has 65,536. Both are allocated with the World::Impl and destroyed with it. The Standalone streamer stops and joins workers before world teardown; engine shutdown must keep the same drain-before-destroy ordering as the existing generator state.

The S14 FLazy thread-local caches stay first, so their hits remain worker-private. Shared hits read versioned atomic entries without taking a lock; a miss takes a set-local spin lock, rechecks, then computes and publishes. A hit racing with a slot replacement can fall through as a miss. Fixed capacity bounds memory and replacement bounds reuse as the player moves. No page ring, prefetch, or neighbor affinity was needed. The shared table is not an immutable page store; atomic fields and version checks give readers a consistent value while a slot can be replaced. Workers only read recipe/plan identity and content hashes; they do not mutate generator state or call UObject/engine APIs. Epoch, shutdown, and MPSC result handling were not changed. The L3 FP-contract preamble and self-test remain intact, as does the FLazy rule for large thread-local storage.

3. Speed and memory

W12 static whole view	Base	Shared	Change
whole_view_ready median	2.165 s	1.754 s	−19.0%
Peak RSS median (max)	401.5 MiB (402.6)	467.8 MiB (468.0)	+66.3 MiB median

Per-LOD worker sum is the sum of tile worker seconds, not wall time. Values are medians across the same three static runs.

LOD	Base worker sum	Shared worker sum	Change
0	3.830 s	2.667 s	−30.4%
1	5.301 s	4.085 s	−22.9%
2	1.296 s	0.795 s	−38.7%
3	1.116 s	0.725 s	−35.1%
4	2.692 s	1.674 s	−37.8%
5	1.911 s	1.459 s	−23.7%
6	2.903 s	2.621 s	−9.7%
7	1.242 s	1.227 s	−1.2%
8	2.318 s	2.271 s	−2.0%

The moving trial warmed the first view, moved the camera 8 m, then generated 144 entering tiles (128 LOD0, 16 LOD1) on the same world and worker pool. It ran three times per cache mode.

Moving check	Shared	Thread-local	Change
New-tile wall median	52.39 ms	82.00 ms	−36.1%
LOD0 worker sum median	403.44 ms	693.67 ms	−41.8%
LOD1 worker sum median	138.34 ms	182.29 ms	−24.1%
Peak RSS median (max)	466.1 MiB (466.9)	466.1 MiB (467.6)	Same cache allocation in both modes

4. Hashes

Every W1/W12 result matched the base for Clang FP contraction off and GCC default: full detail, whole view, each LOD, and exact queries. The L1 base hashes are:

Scope	Density	Mesh	Payload	Topology
Full detail, 960 tiles	0x0ee0579bd15cbbe4	0x32e5d4964d610fee	0x1932b96ee4b1d910	0x69743c6bad3447dc
Whole view, 2,553 tiles	0xc10697a8f48380c7	0xe75c9119a1498cd5	0x49a9a8d1835a6c8c	0xedcb23288b5e8724

LOD	Density	Mesh	Payload	Topology
0	0x769f2a706d585a72	0x33eeeb39fa443816	0x645621a1d968983f	0xc2457ed905036c9d
1	0x69cc5a5c0a2c60af	0xb48335f5410c57d5	0xd15c5480162ab8ba	0xfeca1bd025823604
2	0x85deed5683a77202	0x53469895987aeea2	0xe55f8ae7e1532033	0xd5feb12a0e85c09b
3	0x6518caefb3be7e7f	0xec513c90b853a614	0xf62fa1137ff17f82	0xccf1dcf681965816
4	0x62d4be96021992f4	0xf1e4c6efcc2615a9	0x9dc5f9c88be02f33	0xfe6f55125a07c3bf
5	0x5e4a8c38b8cea6fc	0x08ee56998aaf21e5	0x541eea8558f26cc8	0x6539374a8b3336d8
6	0xe036e95b9006945b	0x624a33f941a99b02	0x81b73e14f68bdf86	0x4de4c63ce1f80a56
7	0x8ee358af56a30d8d	0x46ab3c6184048b7c	0x0a6164b7e68e80c5	0x0c26b85bb03bc831
8	0xc3458e1cbdb9c471	0x46c03d0994d75192	0x8f89fc5bde12d230	0xd16dc0cf62031898

Exact queries (9 probes): density and unclamped 0xe4d4a8eb95b35933, detail 0xefcba20c0c70c415, collision 0xe604823a249029bf. The FloatingIslands seed-0 fixture also passes at Clang/GCC W1/W12 with recipe hash 0x39a24198ad8cf88f; its whole-view hash matches the base: density 0xa8fc5f8f4eaf77a3, mesh 0xfff7d8ac8923eb9a, payload 0xd730c51d959c1c59, topology 0xbb33ddb76cd386be.

5. Transfer estimate and Windows work

The majority of the opportunity is cross-worker, so it does not disappear just because the 5800X3D has a larger L3. Same-worker evictions are only 4% of baseline repeats; that capacity-driven part may shrink. The candidate adds about 66 MiB peak RSS, and a 96 MiB L3 could keep much of the shared table resident, though other game data will compete for it. I expect the win to transfer and clear 3%, likely landing in the mid-single to low-double digits; that percentage is an estimate until the 5800X3D run.

Windows to-do:

1. Build the Core route with MSVC /fp:precise; keep the FP-contract self-test and check the 64-bit atomic slot path on the supported Windows target.
2. Run the W1/W12 whole-view, full-detail, per-LOD, exact-query, and seed-0 FloatingIslands checks against the base binary.
3. Interleave three static W12 A/B runs on the 5800X3D, then repeat the 8 m camera crossing; record peak RSS and the machine's unchanged governor.
4. Exercise real streaming shutdown/profile changes and confirm workers drain before world/cache destruction, with Epoch checks and MPSC result queues still intact.

6. Failed and corrected runs

• The first Clang W1 hash invocation omitted the explicit data path; it exited 2 after resolving Standalone/bin/data/owner_world.json. The retry passed with Standalone/data/owner_world.json supplied.
• The first moving step was 32 m and failed the shape interval 160-million-node safety bound. The benchmark was retried at 8 m, which completed three times in each cache mode.
• The first shared FloatingIslands cache key omitted the island stable ID. The W12 fixture exposed an LOD4 hash mismatch. Adding the exact sub-domain ID fixed it; final hashes match at W1/W12 for both compilers.
• An early fixture command passed --seed=0, which also overrode the surface-world seed. The final fixture comparison omits that override; FloatingIslands itself still runs its seed-0 check.
• No run exceeded 30 minutes or remained stuck for 15 minutes. All final Clang and GCC builds succeeded with 12 jobs.

Owner note

Linux measured a 19.0% W12 whole-view gain and a 36.1% gain on the 8 m crossing, at about 66 MiB more peak RSS. Please run the Windows hash matrix and static/moving A/B on the 5800X3D before using the laptop percentage as an owner-machine estimate.

Orchestrator review (Claude)

• Concurrency read: a seqlock over all-atomic fields (seq_cst), with the version checked before and after the key/value loads; readers fall back to a miss on any mismatch. Writers hold the set spin lock and mark the version odd while writing. There is no non-atomic shared access, so a reader can't return a torn value. The per-tile scoped setThreadActive/restore binds each thread to its own world's table. Default on (useSharedColumnCache = true).
• Re-run by me (W12, L1 view, interleaved): base linux6 2.014/2.059/2.092 s vs L7 linux7shared 1.657/1.730/1.729 s, then 1.718/1.705/1.722 s (median 2.059 → 1.722 s, -16%). The whole-view hash was identical in all 8 L7 runs at W12, plus W1 (8.036 s) and W8 (1.799 s).
• Luna's island-key bug was caught by the W12 hash fixture before review, so the determinism check works as intended.