wip(assembler): round L9 -- LOD1 skips proven-uniform blocks: 95% fewer samples, view ~2% faster

Step-2 (LOD1) tiles of native SurfaceWorld now prove 8-cell blocks uniform from exact heightfield column extrema over the padded block (cells + normal halo) and skip their density fill; features, edits, mixed strata and Unknown verdicts still fill. LOD1 samples 42,875 -> 2,063 per tile. Audit: 80,502 skipped blocks (3 seeds) + 9,216 island blocks force-filled, 0 violations. All SurfaceWorld and FloatingIslands hashes unchanged at W1/W8/W12, Clang off and GCC default. Reviewer interleaved W12, no hashing: whole view 1.439 -> 1.409 s. L8 phase probes removed; --profile-lod-work and --audit-uniform-block-proofs are opt-in. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CSFYgsdW8ejZSyiMf4qRSX
2026-10-19 23:14:16 +02:00
1 parent 82813fb13b
commit 7650abb755
@@ -0,0 +1,149 @@
+# Linux9: LOD1 uniform-block proof
+
+LOD1 now omits density fills only for 8-cell blocks proved uniform by the Core SurfaceWorld heightfield. It fills 924,232 samples across the 448 tiles (2,063 per tile), down from 19,208,000 (42,875 per tile): 95.2% fewer. All output hashes match the `4b1108b` baseline.
+
+## Why the old gate excluded steps
+
+`git blame` points to the original mesher gate: steps 2, 16 and 32 were disabled because the per-block classifier was expected to cost more than the smaller grids saved, and to preserve the old coarse-LOD latency. It gives no correctness reason at step 2; the history does not cite block edges, normal halos or skirts. Step 1 and step 8 were already enabled (step 8 is LOD2). This round enables step 2 for native SurfaceWorld tiles; steps 16/32 remain off pending a separate cost profile.
+
+## Proof and fallback
+
+The tile and each proof box must stay inside one native SurfaceWorld stratum, with no overlapping edit. For every XY lattice point in the padded 8-cell box, `Heightfield2D::classifyLatticeBox` evaluates the exact ground and cap columns at the step-2 stride. For density `min(z-ground, cap-z)`, `minZ > maxGround && maxZ < minCap` proves air; `maxZ < minGround || minZ > maxCap` proves solid. These extrema bound every discrete MC density sample; the box includes the cell samples and the one-sample normal halo (`-1..+9`). No nominal noise parameter is used as a bound.
+
+Feature boxes that can add volume, edits, invalid input, mixed strata, and any Mixed/Unknown verdict stay on the exact fill path. The default FloatingIslands view does not use this Step-2 tile gate; its Unknown whole boxes (1,635) remain filled. The native classifier path is shared, and a W12 island audit also forced 9,216 uniform Step-2 candidates through fill-and-check.
+
+The generic `Heightfield2D::interval` was sound but too broad here: it gave bounds `[-2,883,809, 1,032,090]` subvoxels around z=-2,114, so all 28,672 Step-2 blocks were Mixed. The exact lattice-column extrema produced useful verdicts instead.
+
+## Soundness sweep
+
+The audit bypassed block skipping and explicitly evaluated every density sample in each candidate's padded block, asserting that none contradicted the verdict. It covered three SurfaceWorld seeds and one FloatingIslands audit:
+
+| Recipe / seed | Blocks checked | Density samples checked | Mixed fallbacks | Unknown | Violations |
+|---|---:|---:|---:|---:|---:|
+| SurfaceWorld 0 | 27,479 | 36,574,549 | 1,193 | 0 | 0 |
+| SurfaceWorld 1 | 26,069 | 34,697,839 | 1,451 | 0 | 0 |
+| SurfaceWorld 17 | 26,954 | 35,875,774 | 1,718 | 0 | 0 |
+| SurfaceWorld total | 80,502 | 107,148,162 | 4,362 | 0 | 0 |
+| FloatingIslands 0 | 9,216 | 12,266,496 | 0 | 0 | 0 |
+
+A violation would be possible if the native density dispatch or `point()` stopped matching this column formula, if a new volume/edit overlay bypassed the Mixed guard, or if the padded lattice/normal halo changed without updating the proof bounds. Zero density is treated as Mixed by the audit.
+
+## W12 samples and worker time
+
+Sample counts compare L8's SurfaceWorld profile with the final L9 profile. Worker seconds are medians of five W12 no-hash runs against `linux7shared` and the Clang candidate. They are per-LOD sums across workers.
+
+| LOD | L9 filled samples | Worker seconds baseline → L9 | Change |
+|---:|---:|---:|---:|
+| 0 | 5,316,500 | 2.398 → 2.395 | -0.1% |
+| 1 | 924,232 | 1.213 → 0.920 | -24.1% |
+| 2 | 932,824 | 0.452 → 0.456 | +0.8% |
+| 3 | 1,858,789 | 0.540 → 0.549 | +1.6% |
+| 4 | 2,085,136 | 1.535 → 1.545 | +0.6% |
+| 5 | 987,696 | 1.378 → 1.395 | +1.2% |
+| 6 | 658,464 | 2.559 → 2.564 | +0.2% |
+| 7 | 329,232 | 1.134 → 1.139 | +0.4% |
+| 8 | 1,049,427 | 2.055 → 2.082 | +1.3% |
+
+LOD1 per tile is 42,875 → 2,063 samples.
+
+LOD0 and LOD2–8 stayed within 1.6% of baseline worker time. LOD1's 4.064 worker-seconds in the L8 profile did not reproduce in the specified `linux7shared` binary's worker diagnostics: the five-run median there was 1.213 seconds. The exact-binary interleaved timing below is the comparable whole-view measure.
+
+## W12 interleaved stream time
+
+Baseline is `Standalone/bin/linux/VoxelForgeCoreHeadless-linux7shared`, SHA256 `fbbd468e351c8a0e00c903a7988a3765158e9400ed6a0111898da4d7f5ae6c85`. Each pair used the same seed, camera, 2,553 tiles and W12 settings.
+
+| Pair | Baseline | Clang `-ffp-contract=off` | Delta |
+|---:|---:|---:|---:|
+| 1 | 1.079 s | 1.066 s | -1.2% |
+| 2 | 1.109 s | 1.083 s | -2.3% |
+| 3 | 1.145 s | 1.100 s | -3.9% |
+| 4 | 1.125 s | 1.099 s | -2.3% |
+| 5 | 1.126 s | 1.113 s | -1.2% |
+| Median | 1.125 s | 1.099 s | -2.3% |
+
+| Pair | Baseline | GCC 14 default | Delta |
+|---:|---:|---:|---:|
+| 1 | 1.073 s | 1.100 s | +2.5% |
+| 2 | 1.091 s | 1.136 s | +4.1% |
+| 3 | 1.104 s | 1.129 s | +2.3% |
+| 4 | 1.123 s | 1.140 s | +1.5% |
+| 5 | 1.123 s | 1.150 s | +2.4% |
+| Median | 1.104 s | 1.136 s | +2.9% |
+
+The baseline ELF contains GCC 14 and Clang 19 objects; the candidate GCC binary is GCC-only. As a compiler-matched control, five W12 pairs against the GCC-only `linux7gcc` baseline measured 1.154 s → 1.116 s (-3.3%). Thus the specified-binary GCC comparison is slower, while the matched GCC control is faster; the mixed-toolchain delta cannot be attributed to this change. The Clang candidate is faster than the specified binary.
+
+FloatingIslands seed 0 hashes match at W1/W12 for both compilers. Five W12 no-hash pairs against `linux7shared` gave Clang 10.836 → 11.005 s (+1.6%, pair range -0.5% to +3.0%). GCC was 11.223 → 12.618 s (+12.4%), while five compiler-matched GCC pairs against `linux7gcc` were 12.498 → 12.529 s (+0.25%, pair range -0.5% to +1.3%). The default island path has no measurable same-compiler slowdown; the large `linux7shared` GCC delta is the same toolchain mismatch.
+
+## Hashes
+
+Every SurfaceWorld hash below matched the baseline at W1/W8/W12 for both compilers; W12 was repeated five times per compiler. FloatingIslands seed 0 matched at W1/W12 for both compilers. Per-LOD tuples are density / mesh / payload / topology.
+
+SurfaceWorld full detail: density `0x0ee0579bd15cbbe4`, mesh `0x32e5d4964d610fee`, payload `0x1932b96ee4b1d910`, topology `0x69743c6bad3447dc`.
+
+SurfaceWorld whole view: density `0xc10697a8f48380c7`, mesh `0xe75c9119a1498cd5`, payload `0x49a9a8d1835a6c8c`, topology `0xedcb23288b5e8724`.
+
+SurfaceWorld exact queries (9 probes): density/unclamped `0xe4d4a8eb95b35933`, detail `0xefcba20c0c70c415`, collision `0xe604823a249029bf`.
+
+| LOD | Density | Mesh | Payload | Topology |
+|---:|---|---|---|---|
+| 0 | `0x769f2a706d585a72` | `0x33eeeb39fa443816` | `0x645621a1d968983f` | `0xc2457ed905036c9d` |
+| 1 | `0x69cc5a5c0a2c60af` | `0xb48335f5410c57d5` | `0xd15c5480162ab8ba` | `0xfeca1bd025823604` |
+| 2 | `0x85deed5683a77202` | `0x53469895987aeea2` | `0xe55f8ae7e1532033` | `0xd5feb12a0e85c09b` |
+| 3 | `0x6518caefb3be7e7f` | `0xec513c90b853a614` | `0xf62fa1137ff17f82` | `0xccf1dcf681965816` |
+| 4 | `0x62d4be96021992f4` | `0xf1e4c6efcc2615a9` | `0x9dc5f9c88be02f33` | `0xfe6f55125a07c3bf` |
+| 5 | `0x5e4a8c38b8cea6fc` | `0x08ee56998aaf21e5` | `0x541eea8558f26cc8` | `0x6539374a8b3336d8` |
+| 6 | `0xe036e95b9006945b` | `0x624a33f941a99b02` | `0x81b73e14f68bdf86` | `0x4de4c63ce1f80a56` |
+| 7 | `0x8ee358af56a30d8d` | `0x46ab3c6184048b7c` | `0x0a6164b7e68e80c5` | `0x0c26b85bb03bc831` |
+| 8 | `0xc3458e1cbdb9c471` | `0x46c03d0994d75192` | `0x8f89fc5bde12d230` | `0xd16dc0cf62031898` |
+
+FloatingIslands full detail: density `0x6d842d2878b08ef6`, mesh `0x4b868a3184f08035`, payload `0x2cedf2236788752d`, topology `0xeac17d5e0cb5044a`.
+
+FloatingIslands whole view: density `0x5be4d8cde79b2a76`, mesh `0x96f75041f6861b3a`, payload `0x3700a516cc89d50c`, topology `0x6b99948d6eab4f21`.
+
+FloatingIslands exact queries (9 probes): density/unclamped `0x61daded9c303becb`, detail `0xefcba20c0c70c415`, collision `0x84d37463be9b0573`.
+
+| LOD | Density | Mesh | Payload | Topology |
+|---:|---|---|---|---|
+| 0 | `0xf2524906044ef994` | `0x15b2828701a06591` | `0x4cbbf7ade54321f7` | `0x2c0382f590d69610` |
+| 1 | `0x71b4c8327ad91da7` | `0xfd2280177adb70c9` | `0xc6f695b969ca8573` | `0xb7b026366bc5b627` |
+| 2 | `0x23925de354d86f14` | `0xd4b470b6aae75cab` | `0x38c39feddc72ca0c` | `0x8496f76d94d0e578` |
+| 3 | `0x4cb68a60bc8a5103` | `0x30514ad73d41fd87` | `0x39a209d2ec7695b1` | `0xa2ffa199ba1e1215` |
+| 4 | `0x0e95c8abbcf6fb37` | `0x966b891fbe67794a` | `0xf01627522109d362` | `0xd7bfb2fea0137c0c` |
+| 5 | `0xf444ffed34a1bd20` | `0x7f3073ed3a711113` | `0xad32e4ea4a1b4d50` | `0x6fdba8d6e549193c` |
+| 6 | `0x6a6b4f75b9583ec7` | `0xdbb540e17a46b981` | `0xe2374a66e9d12b22` | `0xd384b7c0c0247982` |
+| 7 | `0x5c4e824aaa7e5f73` | `0x44857e6de8e89645` | `0xa4ac0bf1da59e406` | `0x94f425b0d77004c8` |
+| 8 | `0x8ce5f96cb24bc365` | `0xa1406a9a6dcae02b` | `0x8b5e182d38fe352b` | `0x098f65c1a8cde739` |
+
+## Instrumentation, builds and limits
+
+Removed the L8 phase/stage probes from the normal path. Kept `--profile-lod-work` for per-LOD density sample counts and added `--audit-uniform-block-proofs`; both are off by default. Normal runs do not perform the audit fill-and-check pass or sample-count aggregation. The optional LOD0 MC (1.348 worker-s) and shape compile (215 ms) were counted in L8 and not changed.
+
+No async task or epoch path changed. The L3 FP-contract preamble/self-test, L4 `FLazy`, L7 shared cache, ARCHITECTURE §8.10 two-pass MC/SSE noise/clipmap-streaming invariants, and generator density output remain unchanged.
+
+Clang 19 (`-ffp-contract=off`) and GCC 14 default builds used `-j12`; SHA256s are `0cd3f0ef993ced2c615418c6e2b37b83f57eff6d024d7d1568c30ea51490384a` and `23fe40de558b1b6931469cc5e976a0e4f2ed5fb6c362bc746dcc64a38fa40e08`. Peak build RSS was 901,940 KiB; highest measured runtime RSS was 623,928 KiB. Runs used CPUs 0–11, a 6 GiB address-space limit, normal priority, and the unchanged `powersave` governor. No run exceeded 30 minutes or remained stuck for 15 minutes.
+
+## Failed and retried runs
+
+- The first build wrapper failed because `/usr/bin/time` is absent; direct builds succeeded. One RSS sampling wrapper hung on zombie handling and was interrupted; later measurements used a corrected monitor.
+- Initial proof-audit commands exited 2 because the positional data file and required stream modes were missing. Corrected commands completed. The first broad global interval returned Mixed for all 28,672 blocks and was replaced by the exact lattice-column proof.
+- An early all-step audit found two legacy step-8 uniform candidates whose exact filled-grid signs disagreed; the existing post-fill validator cleared them before skip. The new audit is scoped to the native step-2 proof; step 8 is unchanged.
+- The first Clang retry failed to compile because `SampleGridPoint` was outside its lambda scope; it was corrected to call `DensityAtLattice` directly and rebuilt.
+- The first hash comparison differed because a uniform-tile early return bypassed debug density capture. Step-2 hash runs now refill omitted samples after exercising sparse MC. A separate FloatingIslands mismatch came from changing `>= 0` to `> 0`; the legacy `>= 0` comparison was restored and both compiler hashes then matched.
+
+**Owner note:** LOD1 now fills 95% fewer samples with zero proof violations, and all SurfaceWorld and FloatingIslands hashes match. The GCC W12 comparison against `linux7shared` is toolchain-confounded; GCC-to-GCC controls are flat or faster, while the Clang candidate is faster than the requested baseline.
+
+## Orchestrator review (Claude)
+
+- **Re-run by me, 5 interleaved W12 pairs, L1 view.** Without `--stream-hash` (game-like):
+  whole view L7 1.379/1.485/1.430/1.439/1.449 s vs L9 1.390/1.409/1.401/1.423/1.429 s (median
+  1.439 → 1.409 s, −2.1%); stream wall 1.140 → 1.112 s (−2.5%). With hashing on, the difference is
+  inside noise (1.748 vs 1.744 s), because hashing adds ~0.3 s. The whole-view hash is identical in
+  all runs.
+- **The L8 lever was inflated by L8's own instrumentation.** L8's profile build reported LOD1 at
+  4.06 worker-s; the plain `linux7shared` binary spends 1.21. Per-LOD shares from instrumented builds
+  must be re-checked on a plain build before they're ranked.
+- The "baseline ELF contains GCC and Clang objects" remark is expected: crt/libstdc++ objects always
+  come from GCC. The compiler-matched controls are the right comparison, as Luna did.
+- Proof read: for native SurfaceWorld step-2 tiles the verdict uses exact column extrema over the
+  padded block (cells plus the −1..+9 normal halo) for density `min(z−ground, cap−z)`. Features, edits,
+  mixed strata and Unknown verdicts all fill. The 80,502-block audit found 0 violations.