VoxelForge/Standalone/Research/WorldAssemblerM3/Linux2/REPORT.md

WorldAssemblerM3 Linux2

Result and changes

The hash check can go red. -ffp-contract=on, fast, and GCC's default each change the first stage hash (density) on the L1 view; contraction off still reproduces all four L1 hashes at W1 and W12. No Core cache or density source was changed.

The standalone now accepts --seed, --recipe, and --render-distance-chunks; the viewer uses those same options for Windows replay. Linux standalone workers use 32 MiB pthread stacks, while the process retains its ordinary 8 MiB stack limit. The build script still defaults to contraction off; --fp-contract=default omits that override for the compiler-default probe.

Environment: i9-12900H, CPUs 0–11, Clang 19.1.7, GCC 14.2.0, governor powersave, EPP balance_performance. Runs used taskset -c 0-11, RLIMIT_AS=6,442,450,944, normal priority, and no 64 MiB stack limit. Peak wait4 RSS was 539,648 KiB on the 2,553-tile checks and 293,460 KiB in the seed sweep.

Commands

Build variants, with --fp-contract=off, on, and fast as named; default leaves the compiler flag unset:

bash Standalone/BuildLinux.sh --compiler=clang++-19 --variant=clang19-l2-off --jobs=12 --fp-contract=off
bash Standalone/BuildLinux.sh --compiler=clang++-19 --variant=clang19-l2-on --jobs=12 --fp-contract=on
bash Standalone/BuildLinux.sh --compiler=clang++-19 --variant=clang19-l2-fast --jobs=12 --fp-contract=fast
bash Standalone/BuildLinux.sh --compiler=g++-14 --variant=gcc14-l2-off --jobs=12 --fp-contract=off
bash Standalone/BuildLinux.sh --compiler=g++-14 --variant=gcc14-l2-default --jobs=12 --fp-contract=default

The L1-view call was taskset -c 0-11 prlimit --as=6442450944 -- python3 Standalone/Research/WorldAssemblerM3/Linux2/MeasureProcess.py <binary> Standalone/data/owner_world.json --workers=<1|12> --world-assembler-prototype --stream-benchmark --stream-pipeline --stream-hash --force-generate --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --far-horizon-lod8-step=0. The per-variant commands and resource records are in Saved/Linux/linux2-contract-final-*.log.

The seed matrix ran with:

python3 Standalone/Research/WorldAssemblerM3/Linux2/RunSeedSweep.py 1800

Contraction sensitivity

All rows cover the same 2,553-tile L1 view, W12. g++-14 -Q --help=optimizers -O2 -fno-fast-math reports -ffp-contract=fast when no override is given.

Build	Density	Mesh	Payload	Topology	First divergence from off
Clang 19 off; GCC 14 off	c10697a8f48380c7	e75c9119a1498cd5	49a9a8d1835a6c8c	edcb23288b5e8724	—
Clang 19 on	acfa7023c3671dd6	adacc34b568be881	7c1de424738166e3	78e77fd29674920f	density
Clang 19 fast	f04bb87094a92a25	07596ba1a83f91d5	1292f277745a876d	b07b82bfb52166f5	density
GCC 14 default (effective fast)	03ed0b826a9ff773	07596ba1a83f91d5	fb6f82e779be56b2	9e5d549138ef2bc0	density

The hashes come from the whole-view pipeline stages in order: density, mesh, payload, topology. The same contraction-off binaries reproduce L1 at W1 as well as W12.

Seed sweep

seeds.txt contains seed 0 plus 47 distinct values sampled without replacement from signed int32 using random.Random(0x564F58454C464F52). With no mismatch among the 47 random draws, the one-sided 95% binomial upper bound is 6.18% for a random seed to mismatch on this exact matrix; seed 0 is included as the fixed L1 baseline.

Recipe	Seeds	Linux combinations	Coverage per seed
SurfaceWorld	48	Clang/GCC × W1/W12 (192 rows)	27 LOD0 tiles; voxels (-32,-192,-1984) to (64,-96,-1888), a 96³ cube (24 m per side)
FloatingIslands	48	Clang/GCC × W1/W12 (192 rows)	27 LOD0 tiles; voxels (-32,-32,-160) to (64,64,-64), a 96³ cube (24 m per side)

All 384 runs passed. Clang/GCC matched at both worker counts in 192 comparisons; W1/W12 also matched within each compiler in 192 comparisons. The full per-seed, per-stage hashes, tile extents, wall time, and RSS are in seed_hashes.csv; the summary is in seed_sweep_summary.txt. The sweep took 290.1 seconds; its maximum per-process peak RSS was 293,460 KiB. Coverage is limited to these fixed local cubes.

For seed 0, SurfaceWorld hashes are e93491225d0202f9 / 6b2a630951428df2 / fc2798d6a5478b25 / 0c08d7d980a84301; FloatingIslands hashes are ea74d3354b5ce671 / 476d49ae2a4ba2a6 / 16078fcd36b68ff0 / cfb6c412b7118172. Each sequence is density / mesh / payload / topology and matched across both compilers and worker counts.

The viewer seed override was added in StandaloneViewer.cpp and the standalone recipe helper; no Core change is needed. windows_commands.txt contains all 192 PowerShell commands. Example from the repository root:

& ".\Standalone\bin\VoxelForgeCoreStandalone.exe" ".\Standalone\data\owner_world.json" --seed=0 --recipe=surface-world --render-distance-chunks=0 --workers=1 --world-assembler-prototype --stream-benchmark --stream-pipeline --stream-hash --force-generate --clip-radius=1 --max-clip-level=0 --camera=0,-35,-487.94,0,-22 --far-horizon-lod8-step=0 --process-affinity=4095 --process-priority=Normal --frame-limit=60

Build the viewer with powershell -ExecutionPolicy Bypass -File Standalone\Build.ps1, then run the full replay with powershell -ExecutionPolicy Bypass -File Standalone\Research\WorldAssemblerM3\Linux2\WindowsReplay.ps1; it writes logs under Saved\Linux. Windows/MSVC compilation was unavailable here. Portability risk spots are the new C++20 helpers in WorldAssemblyPrototype.h, strtoll argument parsing in StandaloneViewer.cpp, and the __linux__/std::thread split in StandaloneWorkerThread.h; the pthread path is Linux-guarded.

TLS cost and follow-up

readelf -lW reports PT_TLS FileSiz 6,093,024 bytes (5.81 MiB initialized image) and MemSiz 18,155,149 bytes (17.31 MiB per thread). Three 12-worker probes under the ordinary 8 MiB process stack measured a median spawn-to-ready time of 5.63 ms per thread (5.07–6.44 ms), and a resident increase of 17,736 KiB per worker after the first (17,936 KiB for the first). At 12 workers, /proc/self/statm rose by 213,036 KiB (208.0 MiB); the 32 MiB worker stacks reserve a further 32,772 KiB of virtual address space each. The largest wait4 peak across TLS probes was 268,476 KiB.

The large TLS symbols include HeightfieldColumnCache (0x680000 bytes, about 6.8 MB), IslandColumnCache (0x110000), plus duplicated GFloorReliefCache (0x140000 each), GRoomGraphCache (0xed000 each), and GSurfColCache (0xc9e78 each). GPlayerFitMemo is also duplicated as a process-wide object in ordinary .bss. core_sources.txt has 18 translation units: VoxelForgeCore.cpp is compiled once; each other 17 is compiled both as a vfcore::detail wrapper for Core and as a global legacy object for the standalone owner loader/mesher. This isolates the Core copy from Unreal-shaped global symbols and preserves the legacy reference path; the duplicate objects also duplicate cache state.

The L2 executable's static TLS measurement does not establish the Unreal plugin's model. Inference: a runtime-loaded shared object using general-dynamic TLS can obtain per-thread blocks through the dynamic loader, so it may avoid this exact static-TLS-in-stack footprint; the actual UBT .so flags and relocations must be inspected. glibc documents the TLS/dlopen compatibility concern for initial-exec TLS and recommends global-dynamic for more general shared-object loading (glibc Dynamic Linker Hardening); GCC describes the per-thread storage model and its loader/libc support (GCC Thread-Local Storage).

After S14, prototype a small per-thread TLS handle backed by a bounded worker scratch pool or explicit generation context; share immutable initialized data and remove the Core/legacy duplicate cache where practical. This is a proposal only. The standalone stack wrapper already avoids the external 64 MiB process-stack setting: its own Linux workers request 32 MiB, and the L1 hashes stayed unchanged.

Failed and retried runs

• The first Clang-off build failed because one join loop still named std::thread&; it was changed to FStandaloneWorkerThread& and rebuilt successfully (linux2-build-clang-off.log, then the retry logs). One later GCC-off build attempt stopped with headless_object: unbound variable (linux2-build-gcc-off-final2.log); an immediate identical build under the same CPU/address-space limits succeeded (linux2-build-gcc-off-debug.log), and the failure did not recur.
• The first FloatingIslands smoke used the default far ring and planned 186,911 tiles; it was stopped after about 10 seconds before generation (linux2-floating-smoke-clang-w1.log). With --render-distance-chunks=0, a radius-2/max-LOD-4 smoke completed 791 tiles in 18.94 seconds (linux2-floating-smoke-clang-w1-r2.log); the sweep uses radius 1/max-LOD 0 (27 tiles).
• The first contraction pass omitted the L1 camera and covered 2,370 tiles (linux2-contract-*-w12.log), so those hashes were discarded. The exact L1 camera rerun covered 2,553 tiles and produced the table above (linux2-contract-final-*.log).
• An optional one-ULP diagnostic attempt did not trigger (no injection line and no hash change; linux2-negative-one-ulp-clang-off-w12.log), so that switch was removed. It was unnecessary after the contraction variants made the check go red. The seed sweep had no failed or retried runs.

Note for the owner

Contraction off reproduces L1 at W1/W12, while Clang on/fast and GCC default diverge first at density, so the Unreal Linux build's effective FP flags are decisive. The 48-seed sample matched across compilers and worker counts on 24 m-sided cubes in both recipes; the Windows replay commands are ready, with their MSVC build still unverified.

Orchestrator review (Claude)

• Surface sweep coverage is 14 effective seeds, not 48. In seed_hashes.csv, 34 of the 48 SurfaceWorld seeds give seed 0's exact density hash 0xe93491225d0202f9, i.e. the fixed 24 m cube carries no seed-dependent surface for them. Only 14 seeds exercise distinct terrain, so the 6.18% bound applies to FloatingIslands (48 distinct hashes) but not to SurfaceWorld. Future sweeps should place the cube on each seed's surface and report a per-seed non-empty-mesh count, so an empty cube can't count as a pass.
• Re-run by me: VoxelForgeCoreHeadless-clang19-l2-off, W12, plain 8 MiB ulimit -s, L1 view: whole view 3.361 s, hashes c10697a8f48380c7 / e75c9119a1498cd5 / 49a9a8d1835a6c8c / edcb23288b5e8724.
• Consequence of the contraction table: Core's world now provably depends on the toolchain's FP-contract setting, which every build outside Build.ps1/BuildLinux.sh (UBT on Windows and Linux) sets for itself. Next: make Core immune to, or loudly detect, contraction instead of relying on flags.