streaming: incremental overlay materialization and same-center priority merge (L166)

Persistent Core/overlay/candidate sets updated from deltas; same-center priority order merges changed keys (full sort
only on center change). Set and order equality audits 28/28 + 28/28, injected mismatch fails. 30 m/s UE, 7 pairs vs
9c70053: p99 10.52 -> 7.42 ms (all pairs), worst tick 12.98 -> 11.65 ms (6/7), near 42.0 -> 43.7% and forward cone
14.8 -> 16.1% (all pairs), far holes 3.20 -> 2.74% median (4/7), arrival flat. Source/: VoxelWorld.cpp, VoxelWorld.h.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1

Standalone/Research/WorldAssemblerM3/Linux166/REPORT.md

@@ -0,0 +1,58 @@
+# L166 — Incremental streaming update
+
+**Result:** Core and overlay candidate materialization now apply entering/leaving deltas, and same-center priority order updates merge only changed keys. Seven 30 m/s W8 pairs show lower median rebuild p95, tick p99, worst tick, near readiness, forward-cone readiness, and far holes. The strict pairwise bar is not fully clean: one worst-tick pair, three far-hole pairs, and four arrival pairs regressed slightly. Set/order audits and the fast tier pass.
+
+## Count first
+
+Stage costs below are milliseconds per desired rebuild, reported as the median of the seven runs' p50 and p95 values. Each run's window is its 12-second move at 30 m/s; there were 44–45 center crossings per run.
+
+| Stage | `assembler/world` p50 / p95 | Incremental p50 / p95 |
+| --- | ---: | ---: |
+| Core query/preparation | 0.832 / 2.229 | 0.0002 / 0.142 |
+| Core crossing delta | — | 0.000 / 0.138 |
+| UE overlay materialization | 0.350 / 0.723 | 0.103 / 0.356 |
+| Priority sort/order | 0.306 / 0.656 | 0.038 / 0.437 |
+| Critical admission order | 0.063 / 0.134 | 0.065 / 0.222 |
+
+The incremental Core path updated on 98.4% of desired rebuilds. Priority order merged key deltas on 52.2%, reused the cached order unchanged on 33.5%, and fully sorted on the remaining 14.3% (the center-crossing updates). LOD4+ crossings caused five full Core-query fallbacks per 12-second move; their median per-run maximum was 1.19 ms and their median total was 5.25 ms spread across those five events.
+
+The median worst tick was 11.652 ms versus 12.980 ms for the baseline. The largest candidate worst-tick sample was 12.764 ms at a center crossing: its priority sort took 1.311 ms, overlay materialization 0.915 ms, and Core delta 0.601 ms; no LOD4+ fallback occurred on that tick. Priority sorting remained the largest measured substage in that worst tick. Fallbacks were not the source of the remaining worst ticks.
+
+## Change and equality checks
+
+`BuildCoreDesiredTiles` now retains the Core visual set, UE overlay set, candidate set, and priority order. It applies Core and overlay membership deltas to the persistent sets. Live anchor, gate, and support inputs are still walked into a temporary overlay union on each trigger. At an unchanged center, it removes leaving keys and merges sorted entering keys into the cached priority order. Center changes still require a full priority sort because the distance comparator changes. The critical-key promotion list is still rebuilt on each trigger.
+
+The equality audit independently obtains a full Core desired set, rebuilds the overlay reference, and compares candidate and collision-only sets. A second audit builds the full prioritized and critical order and compares it with the incremental result. Across the seven measured candidate runs, all 28 sampled set audits and all 28 order audits passed. A positive materialization-control run passed 13 additional set and order audits. The injected mismatch control logged `missing=1` and failed with a nonzero UE exit (`-11`), as expected.
+
+## Seven interleaved pairs
+
+Each pair compared `assembler/world` at `9c70053` against this worktree, with a 12-second 30 m/s move, 5-second dwell, and eight effective tile slots. Values are medians across seven runs; paired gates are stated separately.
+
+| Metric | `assembler/world` | Incremental | Paired result |
+| --- | ---: | ---: | --- |
+| `BuildDesiredTiles` p95 | 7.281 ms | 3.738 ms | 48.7% lower median |
+| Game-thread tick p99 | 10.521 ms | 7.420 ms | Lower in all 7 pairs; candidate range 7.031–7.526 ms |
+| Worst tick | 12.980 ms | 11.652 ms | Lower in 6/7 pairs; pair 5 was 0.347 ms higher |
+| Near LOD0–2 readiness | 41.995% | 43.668% | Better in all 7 pairs |
+| 55° forward-cone readiness | 14.779% | 16.116% | Better in all 7 pairs |
+| Far holes while moving | 3.197% | 2.739% | Lower median; better in 4/7 pairs |
+| First settled LOD0–2 arrival | 2.432 s | 2.439 s | Paired median delta +0.002 s; later in 4/7 pairs |
+
+The paired far-hole deltas were −0.072, +0.144, −0.059, +0.301, −1.052, −0.773, and +0.069 percentage points. The L165 +0.291 pp result is within this observed spread; L166 also had one +0.301 pp pair, but the seven-pair median delta was −0.059 pp and four pairs improved. This is consistent with substantial run-to-run variation, not a consistent far-hole regression; it does not establish a strict no-regression guarantee.
+
+Null-RHI produced no camera-forward samples, so the 55° cone uses the travel-direction fallback. The arrival measure also varied sharply between pairs, including two candidate runs that settled 0.22–0.26 s earlier. Its paired median was effectively flat, but the strict per-pair arrival gate did not pass.
+
+## Checks and limits
+
+- VFHost Linux Development build succeeded with `-NoUBA -MaxParallelActions=4`; `VFHost/PluginDeps/VoxelForge/Source` was restored to `/home/jahni/VoxelM/Plugins/VoxelForge/Source`.
+- `verify.py fast` passed: 268 checks, 0 failures, 5 diagnostics, 281.5 s. Clang/GCC W1/W8/W12 determinism and canonical geometry checks passed.
+- The live movement `TileHashDump` hook was enabled but wrote zero records for both conditions, so that direct runtime geometry comparison is inconclusive. No meshing or geometry-generation code changed; the fast-tier geometry hashes and sampled full-rebuild order audit passed.
+- `git diff --check` and runner `py_compile` passed. W12 movement timing, camera-forward cone capture, and separate movement runs on each of the four owner layers were not run.
+- The strict movement summary fails `paired_worst_tick_no_worse`, `paired_far_holes_no_worse`, and `paired_arrival_no_worse`; p99, near readiness, forward-cone readiness, audits, and median rebuild-p95 gates pass.
+
+Changed Unreal `Source/` files:
+
+- `Source/VoxelForge/Private/VoxelWorld.cpp`
+- `Source/VoxelForge/Public/VoxelWorld.h`
+
+Logs, traces, summary, and verifier artifacts are under `Saved/Linux/l166/`.