Fr0zka/VoxelForge
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(assembler): round L10 report -- spawn waits on the full shape; 24 m crossing hits bad_alloc

Browse Source 
First tile starts ~300 ms after cold start: plan 51 ms (serial), shape compile 223 ms (caves,
signs, detail band, far parents). The first SurfaceWorld tiles use the native heightfield and do
not read that shape, but generateTile requires the full base shape. 8/16 m crossings repeat no
compile (61 ms for 288 tiles); a 24 m crossing died with std::bad_alloc under the 6 GiB
address-space cap (cause unknown). Telemetry not committed (+1% when off).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CSFYgsdW8ejZSyiMf4qRSX
This commit is contained in:
Fr0zkaandClaude Opus 5.5 committed 2026-10-19 23:44:52 +02:00
1 parent 7650abb755
commit b370908174
1 file changed
+115
Download Patch File Download Diff File Expand all files Collapse all files
Standalone/Research/WorldAssemblerM3/Linux10/REPORT.md
+115
View File
@@ -0,0 +1,115 @@
# Linux10 — startup and region crossing
**Result:** no world-generation optimization was shipped. The dominant spawn gate is full shape compilation; the current Core API does not let tiles consume a partially built shape safely. L10 adds opt-in phase and crossing telemetry to the headless research path.
Runs used CPUs 0–11, normal priority, at most `-j12`, a 6 GiB address-space cap, and the unchanged `powersave` governor. Peak monitored RSS was 624,996 KiB (610.3 MiB). Final builds: Clang 19 with FP contraction off, SHA-256 `7d5360e2baa637f8b2b4679df9616580bffe689afd101470e311260f0ecd4975`; GCC 14 default, `5041cf15019766f453eb59fe8ff7fc4cd98ca6a904208b645e3d7ca4584e1195`. The L9 Clang timing/hash baseline is `Standalone/bin/linux/VoxelForgeCoreHeadless-linux9-clang`, SHA-256 `0cd3f0ef993ced2c615418c6e2b37b83f57eff6d024d7d1568c30ea51490384a`.
## 1. Startup phases
SurfaceWorld seed 0, W12, no hash; five runs on the optimized Clang build. Wall and process CPU are medians. Process CPU sums work across threads. First-tile time is from cold start; its CPU column is CPU since pool start.
| Phase | Wall ms | Process CPU ms | Threads used |
|---|---:|---:|---|
| Owner data load and resolve | 5.05 | 5.05 | 1 |
| Assembly plan build | 51.04 | 51.03 | 1 |
| Core snapshot | 0.53 | 0.54 | 1 |
| First desired-set build, 2,553 tiles | 0.74 | 0.74 | 1 |
| Shape request setup | 0.03 | 0.03 | 1 |
| Shape compile | 222.59 | 856.79 | caller + up to 8 scheduler workers |
| Streamer setup | 0.01 | 0.01 | 1 |
| Worker-pool start | 0.41 | 1.27 | 12 requested; 8–11 had entered at print |
| First tile starts at 299.62 ms | — | 1.59 since pool start | 12/12 entered |
Shape-compile wall ranged 215.89–324.88 ms; first tile started at 286.56–409.76 ms. Owner load, plan, local shape setup, and far-parent construction are serial. Before the pool starts, there are no sleeping stream workers, but most of the 12 allowed CPUs are unused. During the interval and detail-band phases the scheduler uses at most 8 workers, leaving up to 4 CPUs unused.
The phase-trace mode’s whole-view median was 1.436 s, versus 1.372 s with tracing off. The phase runs are for decomposition only; the performance ranking and A/B below use the optimized build with tracing disabled and hashing off. In that flag-off path, shape wall time was 0.222 s median, consistent with the phase trace.
## 2. Shape contents and first-tile dependencies
| Shape phase | Wall ms | Process CPU ms | Work / concurrency |
|---|---:|---:|---|
| Local owner shapes and spatial bins | 40.46 | 40.44 | Extracts owner primitives; 1 thread |
| Rock intervals, signs, candidate bands | 40.20 | 187.14 | Packed column signs and candidate masks; up to 8 workers |
| Detail band and sign-edge distances | 81.00 | 568.78 | Applies coherent detail to candidate nodes and builds distance proxies; up to 8 workers |
| Route validation and shared-field setup | 1.12 | 1.13 | 5,642 passage samples; then shared primitive form |
| Far parents and bins | 55.56 | 55.57 | 6,050 far rooms / 18,348 links; 1 thread |
The compile extracts 18,348 owner primitives (71 rooms, 74 tunnels, 2 passages), covers 138,188,413 sign nodes, allocates 36.71 MiB of packed words and 263.57 MiB of proxy address capacity, and visits 214,538 detail candidates (6,098 flips).
For this SurfaceWorld fixture, the first LOD0 tile’s density queries take the native `Heightfield2D` path before packed-shape lookup. Far-parent shapes are not needed by that first tile. However, `World::generateTile` rejects requests until the full base shape is marked built, and `compileTileRegion` also requires that base shape. So the current API gates the near tile on the interval, detail, validation, and far-parent work even when that tile does not sample those arrays. Other recipes and fallback/query paths do use the analytic primitives and packed signs/proxies.
There is no seed-independent owner shape payload to cache at build time. The assembly plan is built from owner data; region pages also re-run owner primitive extraction, spatial-bin construction, and shared-field packing. Those immutable owner-derived structures are a possible per-world reuse target, but the measured short crossings below built no pages, so this run does not price that work.
## 3. Moving crossing
Hashing off, W12, same camera and already-streamed initial view. Region compile was called, but all entering fine keys were covered by the initial shape bounds; it built zero regions.
| Camera step | New tiles | LOD breakdown | Region compile | Tile work wall | Crossing total |
|---|---:|---|---:|---:|---:|
| 8 m | 144 | LOD2: 128, LOD3: 16 | 0.010 ms | 12.61 ms | 12.81 ms |
| 16 m | 288 | LOD0: 128, LOD1: 16, LOD2: 128, LOD3: 16 | 0.010 ms | 60.49 ms | 60.68 ms |
The 16 m crossing’s worker-time sum was 633 ms; its wall time was 60.5 ms. No plan or full shape compile repeated. A 24 m attempt terminated with `std::bad_alloc` after base shape compilation and before crossing metrics; the failing allocation site is unknown.
## 4. Ranked levers
| Rank | Lever | Upper bound from these measurements | Assessment |
|---:|---|---|---|
| 1 | Overlap near-tile work with shape compilation | 222.6 ms, 16.2% of 1.372 s whole-view-ready; first tile could start earlier by up to the compile time | Largest opportunity. Not built: tiles and region compilation currently require a fully built base shape. A safe version needs immutable partial-page publication and a wait before any tile reads an unfinished sign, proxy, primitive bin, or far field. |
| 2 | Defer far-parent construction until far tiles need it | 55.6 ms, at most 4.1% of whole-view-ready and 18.6% of the first-tile delay | Could improve spawn, but far and near paths share mutable spatial-bin state today. Needs separate immutable bins and a synchronized readiness state; no crossing benefit was measured. |
| 3 | Parallelize assembly-plan construction | 51.0 ms, at most 3.7% of whole-view-ready | Serial and seed/owner-specific. No safe deterministic parallel split was established. |
| 4 | Raise shape scheduler cap from 8 to 12 | About 121 ms currently lies in the two worker phases; ideal linear scaling saves about 40 ms (2.9% of whole-view-ready) | Below the 3% bar before overhead, and no crossing benefit. Not built. |
For any future partial or deferred build, hashes can fail if a tile observes a partially written sign/proxy array, if region publication races with tile lookup, or if shared bins change while a tile is sampling. The page must be immutable before publication; worker-count and timing tests must compare W1 with W12 and repeat W12. Any asynchronous result must also follow the Epoch/MPSC shutdown-drain rules.
## 5. Final-build timing and hashes
Five interleaved W12 runs, no hash. Pair values are `whole_view_ready / stream_wall` in seconds.
| Pair | L9 `7650abb` Clang | L10 Clang | Delta, ms |
|---:|---:|---:|---:|
| 1 | 1.336 / 1.050 | 1.346 / 1.060 | +10 / +10 |
| 2 | 1.351 / 1.062 | 1.365 / 1.073 | +14 / +11 |
| 3 | 1.359 / 1.073 | 1.374 / 1.083 | +15 / +10 |
| 4 | 1.363 / 1.077 | 1.375 / 1.081 | +12 / +4 |
| 5 | 1.387 / 1.089 | 1.372 / 1.092 | −15 / +3 |
| **Median** | **1.359 / 1.073** | **1.372 / 1.081** | **+13 / +8** |
The measurement-only candidate is +1.0% whole-view-ready and +0.7% stream median; this is no demonstrated speedup. Hashing was off for these timings.
Final rebuilt binaries passed SurfaceWorld and FloatingIslands seed-0 hash checks at W1, W8, and five W12 runs per compiler: Clang FP contraction off and GCC default. Every full-detail, whole-view, LOD0–8, and exact-query row matches the compiler-matched L9 baseline. Audit files: `Saved/Linux/l10_final_surface_hash_audit.txt` and `Saved/Linux/l10_islands_hash_audit.txt`.
| Recipe / scope | Density | Mesh | Payload | Topology |
|---|---|---|---|---|
| SurfaceWorld full detail | `0x0ee0579bd15cbbe4` | `0x32e5d4964d610fee` | `0x1932b96ee4b1d910` | `0x69743c6bad3447dc` |
| SurfaceWorld whole view | `0xc10697a8f48380c7` | `0xe75c9119a1498cd5` | `0x49a9a8d1835a6c8c` | `0xedcb23288b5e8724` |
| FloatingIslands full detail | `0x6d842d2878b08ef6` | `0x4b868a3184f08035` | `0x2cedf2236788752d` | `0xeac17d5e0cb5044a` |
| FloatingIslands whole view | `0x5be4d8cde79b2a76` | `0x96f75041f6861b3a` | `0x3700a516cc89d50c` | `0x6b99948d6eab4f21` |
Exact-query hashes also match: SurfaceWorld density `0xe4d4a8eb95b35933`, detail `0xefcba20c0c70c415`, collision `0xe604823a249029bf`; FloatingIslands density `0x61daded9c303becb`, detail `0xefcba20c0c70c415`, collision `0x84d37463be9b0573`.
## 6. Failed or retried runs
- One phase smoke omitted the explicit owner-data path and resolved it under `Standalone/bin/data`; reran with `Standalone/data/owner_world.json`.
- A no-camera phase smoke produced 2,370 tiles instead of the 2,553-tile fixture; discarded it and reran with the fixed camera.
- A malformed `--workers=w8` command exited during argument parsing; reran with numeric `--workers=8`.
- The first FloatingIslands matrix invocation omitted the three required streaming-mode switches. All 14 attempts exited before generation; corrected the command and all 14 final runs passed.
- The 24 m crossing attempt failed with `std::bad_alloc` as described above.
- The first hash-audit parser expected 13 rows; the actual audit contains 12 scopes. Corrected the parser and all final comparisons passed.
At spawn, the first tile starts about 300 ms after cold start, with roughly 223 ms in full shape compilation and 51 ms in the single-threaded plan; this round did not make that wait shorter. While moving, the 16 m crossing completed 288 entering tiles in about 61 ms and repeated no shape compile, so the measured delay was tile work rather than region setup.
## Orchestrator review (Claude)
- **Not committed:** the L10 telemetry code. It is +1.0% whole-view median with tracing off (4 of 5
pairs slower), and the rule is zero cost when off. Saved as `Saved/Linux/claude/l10-telemetry.patch`
(gitignored) for reuse; the tree is reset to `7650abb`.
- **Key finding, for L11:** the first SurfaceWorld tiles take the native `Heightfield2D` path and don't
read the cave shape, yet `World::generateTile` waits for the full base shape (~223 ms). A per-tile
readiness check (start tiles whose recipe path provably never touches shape arrays) is lever #1 in
safe form.
- **Open bug, for L11:** the 24 m crossing died with `std::bad_alloc` under the 6 GiB **address-space**
cap (`RLIMIT_AS`). That cap counts reservations, not resident memory; L10 also reports 263.57 MiB
of proxy address *capacity*. It is not yet known whether this is a real memory blow-up on larger
crossings or the cap counting reserved address space. L7's 32 m step also hit the 160M-node
interval bound.