Fr0zkaandClaude Opus 5.5
303b7b9f24
Clang -ffp-contract=on/fast and GCC's default (fast) all change the density hash on the L1 view; contraction off reproduces L1 (= Windows S13) at W1/W12. Seed sweep: 48 FloatingIslands seeds and 14 effective SurfaceWorld seeds (34 of 48 fell on a seed-independent cube) match across Clang/GCC and W1/W12. Standalone gains --seed/--recipe/--render-distance-chunks (viewer + headless) and a Windows replay script. Linux standalone workers get 32 MiB pthread stacks (TLS is 17.3 MiB/thread), so no external stack limit is needed. No Core or world-output change; MSVC compile unverified. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
98 lines
11 KiB
Markdown
98 lines
11 KiB
Markdown
# WorldAssemblerM3 Linux2
|
||
|
||
## Result and changes
|
||
|
||
The hash check can go red. `-ffp-contract=on`, `fast`, and GCC's default each change the first stage hash (density) on the L1 view; contraction off still reproduces all four L1 hashes at W1 and W12. No Core cache or density source was changed.
|
||
|
||
The standalone now accepts `--seed`, `--recipe`, and `--render-distance-chunks`; the viewer uses those same options for Windows replay. Linux standalone workers use 32 MiB pthread stacks, while the process retains its ordinary 8 MiB stack limit. The build script still defaults to contraction off; `--fp-contract=default` omits that override for the compiler-default probe.
|
||
|
||
Environment: i9-12900H, CPUs 0–11, Clang 19.1.7, GCC 14.2.0, governor `powersave`, EPP `balance_performance`. Runs used `taskset -c 0-11`, `RLIMIT_AS=6,442,450,944`, normal priority, and no 64 MiB stack limit. Peak `wait4` RSS was 539,648 KiB on the 2,553-tile checks and 293,460 KiB in the seed sweep.
|
||
|
||
## Commands
|
||
|
||
Build variants, with `--fp-contract=off`, `on`, and `fast` as named; `default` leaves the compiler flag unset:
|
||
|
||
```bash
|
||
bash Standalone/BuildLinux.sh --compiler=clang++-19 --variant=clang19-l2-off --jobs=12 --fp-contract=off
|
||
bash Standalone/BuildLinux.sh --compiler=clang++-19 --variant=clang19-l2-on --jobs=12 --fp-contract=on
|
||
bash Standalone/BuildLinux.sh --compiler=clang++-19 --variant=clang19-l2-fast --jobs=12 --fp-contract=fast
|
||
bash Standalone/BuildLinux.sh --compiler=g++-14 --variant=gcc14-l2-off --jobs=12 --fp-contract=off
|
||
bash Standalone/BuildLinux.sh --compiler=g++-14 --variant=gcc14-l2-default --jobs=12 --fp-contract=default
|
||
```
|
||
|
||
The L1-view call was `taskset -c 0-11 prlimit --as=6442450944 -- python3 Standalone/Research/WorldAssemblerM3/Linux2/MeasureProcess.py <binary> Standalone/data/owner_world.json --workers=<1|12> --world-assembler-prototype --stream-benchmark --stream-pipeline --stream-hash --force-generate --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --far-horizon-lod8-step=0`. The per-variant commands and resource records are in `Saved/Linux/linux2-contract-final-*.log`.
|
||
|
||
The seed matrix ran with:
|
||
|
||
```bash
|
||
python3 Standalone/Research/WorldAssemblerM3/Linux2/RunSeedSweep.py 1800
|
||
```
|
||
|
||
## Contraction sensitivity
|
||
|
||
All rows cover the same 2,553-tile L1 view, W12. `g++-14 -Q --help=optimizers -O2 -fno-fast-math` reports `-ffp-contract=fast` when no override is given.
|
||
|
||
| Build | Density | Mesh | Payload | Topology | First divergence from off |
|
||
|---|---|---|---|---|---|
|
||
| Clang 19 off; GCC 14 off | `c10697a8f48380c7` | `e75c9119a1498cd5` | `49a9a8d1835a6c8c` | `edcb23288b5e8724` | — |
|
||
| Clang 19 on | `acfa7023c3671dd6` | `adacc34b568be881` | `7c1de424738166e3` | `78e77fd29674920f` | density |
|
||
| Clang 19 fast | `f04bb87094a92a25` | `07596ba1a83f91d5` | `1292f277745a876d` | `b07b82bfb52166f5` | density |
|
||
| GCC 14 default (effective fast) | `03ed0b826a9ff773` | `07596ba1a83f91d5` | `fb6f82e779be56b2` | `9e5d549138ef2bc0` | density |
|
||
|
||
The hashes come from the whole-view pipeline stages in order: density, mesh, payload, topology. The same contraction-off binaries reproduce L1 at W1 as well as W12.
|
||
|
||
## Seed sweep
|
||
|
||
`seeds.txt` contains seed 0 plus 47 distinct values sampled without replacement from signed int32 using `random.Random(0x564F58454C464F52)`. With no mismatch among the 47 random draws, the one-sided 95% binomial upper bound is 6.18% for a random seed to mismatch on this exact matrix; seed 0 is included as the fixed L1 baseline.
|
||
|
||
| Recipe | Seeds | Linux combinations | Coverage per seed |
|
||
|---|---:|---|---|
|
||
| SurfaceWorld | 48 | Clang/GCC × W1/W12 (192 rows) | 27 LOD0 tiles; voxels `(-32,-192,-1984)` to `(64,-96,-1888)`, a 96³ cube (24 m per side) |
|
||
| FloatingIslands | 48 | Clang/GCC × W1/W12 (192 rows) | 27 LOD0 tiles; voxels `(-32,-32,-160)` to `(64,64,-64)`, a 96³ cube (24 m per side) |
|
||
|
||
All 384 runs passed. Clang/GCC matched at both worker counts in 192 comparisons; W1/W12 also matched within each compiler in 192 comparisons. The full per-seed, per-stage hashes, tile extents, wall time, and RSS are in [seed_hashes.csv](seed_hashes.csv); the summary is in [seed_sweep_summary.txt](seed_sweep_summary.txt). The sweep took 290.1 seconds; its maximum per-process peak RSS was 293,460 KiB. Coverage is limited to these fixed local cubes.
|
||
|
||
For seed 0, SurfaceWorld hashes are `e93491225d0202f9 / 6b2a630951428df2 / fc2798d6a5478b25 / 0c08d7d980a84301`; FloatingIslands hashes are `ea74d3354b5ce671 / 476d49ae2a4ba2a6 / 16078fcd36b68ff0 / cfb6c412b7118172`. Each sequence is density / mesh / payload / topology and matched across both compilers and worker counts.
|
||
|
||
The viewer seed override was added in `StandaloneViewer.cpp` and the standalone recipe helper; no Core change is needed. [windows_commands.txt](windows_commands.txt) contains all 192 PowerShell commands. Example from the repository root:
|
||
|
||
```powershell
|
||
& ".\Standalone\bin\VoxelForgeCoreStandalone.exe" ".\Standalone\data\owner_world.json" --seed=0 --recipe=surface-world --render-distance-chunks=0 --workers=1 --world-assembler-prototype --stream-benchmark --stream-pipeline --stream-hash --force-generate --clip-radius=1 --max-clip-level=0 --camera=0,-35,-487.94,0,-22 --far-horizon-lod8-step=0 --process-affinity=4095 --process-priority=Normal --frame-limit=60
|
||
```
|
||
|
||
Build the viewer with `powershell -ExecutionPolicy Bypass -File Standalone\Build.ps1`, then run the full replay with `powershell -ExecutionPolicy Bypass -File Standalone\Research\WorldAssemblerM3\Linux2\WindowsReplay.ps1`; it writes logs under `Saved\Linux`. Windows/MSVC compilation was unavailable here. Portability risk spots are the new C++20 helpers in `WorldAssemblyPrototype.h`, `strtoll` argument parsing in `StandaloneViewer.cpp`, and the `__linux__`/`std::thread` split in `StandaloneWorkerThread.h`; the pthread path is Linux-guarded.
|
||
|
||
## TLS cost and follow-up
|
||
|
||
`readelf -lW` reports `PT_TLS` FileSiz 6,093,024 bytes (5.81 MiB initialized image) and MemSiz 18,155,149 bytes (17.31 MiB per thread). Three 12-worker probes under the ordinary 8 MiB process stack measured a median spawn-to-ready time of 5.63 ms per thread (5.07–6.44 ms), and a resident increase of 17,736 KiB per worker after the first (17,936 KiB for the first). At 12 workers, `/proc/self/statm` rose by 213,036 KiB (208.0 MiB); the 32 MiB worker stacks reserve a further 32,772 KiB of virtual address space each. The largest wait4 peak across TLS probes was 268,476 KiB.
|
||
|
||
The large TLS symbols include `HeightfieldColumnCache` (0x680000 bytes, about 6.8 MB), `IslandColumnCache` (0x110000), plus duplicated `GFloorReliefCache` (0x140000 each), `GRoomGraphCache` (0xed000 each), and `GSurfColCache` (0xc9e78 each). `GPlayerFitMemo` is also duplicated as a process-wide object in ordinary `.bss`. [core_sources.txt](../../../Core/core_sources.txt) has 18 translation units: `VoxelForgeCore.cpp` is compiled once; each other 17 is compiled both as a `vfcore::detail` wrapper for Core and as a global legacy object for the standalone owner loader/mesher. This isolates the Core copy from Unreal-shaped global symbols and preserves the legacy reference path; the duplicate objects also duplicate cache state.
|
||
|
||
The L2 executable's static TLS measurement does not establish the Unreal plugin's model. Inference: a runtime-loaded shared object using general-dynamic TLS can obtain per-thread blocks through the dynamic loader, so it may avoid this exact static-TLS-in-stack footprint; the actual UBT `.so` flags and relocations must be inspected. glibc documents the TLS/dlopen compatibility concern for initial-exec TLS and recommends global-dynamic for more general shared-object loading ([glibc Dynamic Linker Hardening](https://www.sourceware.org/glibc/manual/latest/html_node/Dynamic-Linker-Hardening.html)); GCC describes the per-thread storage model and its loader/libc support ([GCC Thread-Local Storage](https://gcc.gnu.org/onlinedocs/gcc-13.3.0/gcc/Thread-Local.html)).
|
||
|
||
After S14, prototype a small per-thread TLS handle backed by a bounded worker scratch pool or explicit generation context; share immutable initialized data and remove the Core/legacy duplicate cache where practical. This is a proposal only. The standalone stack wrapper already avoids the external 64 MiB process-stack setting: its own Linux workers request 32 MiB, and the L1 hashes stayed unchanged.
|
||
|
||
## Failed and retried runs
|
||
|
||
- The first Clang-off build failed because one join loop still named `std::thread&`; it was changed to `FStandaloneWorkerThread&` and rebuilt successfully (`linux2-build-clang-off.log`, then the retry logs). One later GCC-off build attempt stopped with `headless_object: unbound variable` (`linux2-build-gcc-off-final2.log`); an immediate identical build under the same CPU/address-space limits succeeded (`linux2-build-gcc-off-debug.log`), and the failure did not recur.
|
||
- The first FloatingIslands smoke used the default far ring and planned 186,911 tiles; it was stopped after about 10 seconds before generation (`linux2-floating-smoke-clang-w1.log`). With `--render-distance-chunks=0`, a radius-2/max-LOD-4 smoke completed 791 tiles in 18.94 seconds (`linux2-floating-smoke-clang-w1-r2.log`); the sweep uses radius 1/max-LOD 0 (27 tiles).
|
||
- The first contraction pass omitted the L1 camera and covered 2,370 tiles (`linux2-contract-*-w12.log`), so those hashes were discarded. The exact L1 camera rerun covered 2,553 tiles and produced the table above (`linux2-contract-final-*.log`).
|
||
- An optional one-ULP diagnostic attempt did not trigger (no injection line and no hash change; `linux2-negative-one-ulp-clang-off-w12.log`), so that switch was removed. It was unnecessary after the contraction variants made the check go red. The seed sweep had no failed or retried runs.
|
||
|
||
## Note for the owner
|
||
|
||
Contraction off reproduces L1 at W1/W12, while Clang on/fast and GCC default diverge first at density, so the Unreal Linux build's effective FP flags are decisive. The 48-seed sample matched across compilers and worker counts on 24 m-sided cubes in both recipes; the Windows replay commands are ready, with their MSVC build still unverified.
|
||
|
||
## Orchestrator review (Claude)
|
||
|
||
- **Surface sweep coverage is 14 effective seeds, not 48.** In `seed_hashes.csv`, 34 of the 48 SurfaceWorld
|
||
seeds give seed 0's exact density hash `0xe93491225d0202f9`, i.e. the fixed 24 m cube carries no
|
||
seed-dependent surface for them. Only 14 seeds exercise distinct terrain, so the 6.18% bound applies to
|
||
FloatingIslands (48 distinct hashes) but not to SurfaceWorld. Future sweeps should place the cube on each
|
||
seed's surface and report a per-seed non-empty-mesh count, so an empty cube can't count as a pass.
|
||
- Re-run by me: `VoxelForgeCoreHeadless-clang19-l2-off`, W12, plain 8 MiB `ulimit -s`, L1 view:
|
||
whole view 3.361 s, hashes `c10697a8f48380c7 / e75c9119a1498cd5 / 49a9a8d1835a6c8c / edcb23288b5e8724`.
|
||
- **Consequence of the contraction table:** Core's world now provably depends on the toolchain's FP-contract
|
||
setting, which every build outside `Build.ps1`/`BuildLinux.sh` (UBT on Windows and Linux) sets for itself.
|
||
Next: make Core immune to, or loudly detect, contraction instead of relying on flags.
|