Compare commits

...
Author SHA1 Message Date
Fr0zka 82eb5b1c66 Merge branch 'wip/l157-speed' into wip/combined 2026-11-02 20:27:26 +01:00
Fr0zkaandClaude Opus 5.5 52cb3d7a23 streaming: restore same-level refill; early admission on by default; fixed UE harness (L172)
Scheduler stable-sorts the whole level and checks capacity per admission (refill when slots free mid-pass); regression
test with the old partial-sort path as negative control. Harness repoints Source/Standalone/uplugin together, records
root HEAD/status, builds baselines from fresh worktrees, gates on DWARF struct sizes (480/480/480).
Frame-resolution arrival: A/A p90 36.8 ms; fixed-on vs fixed-off -12.2 ms median (4/10 slower). 30 m/s: near 43.3 ->
55.3%, far holes 1.8 -> 0%, p99 <= 7.46 ms, worst <= 12.3 ms.
Defaults: StreamingFastTravelWaitForAllCriticalTiles=0, StreamingFastTravelApplyBudgetMilliseconds=1.0,
StreamingIncrementalDesiredSet=1, CoreRegionConcurrentAdmission=1, StreamingFastTravelForwardFirst=0,
StreamingRebuildOnCoreRegionCompletion=1, StreamingCoreDesiredCacheByCenter=0, audits 0.
Source/: VoxelWorld.cpp, VoxelStreamingAdmissionOrder.h (new).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
2026-11-02 20:27:24 +01:00
Fr0zka 4d7b470ebc Merge branch 'wip/living-plane' into wip/combined 2026-11-02 19:01:43 +01:00
Fr0zka c3a1610b22 Merge branch 'wip/l141-liminal' into wip/combined 2026-11-02 19:01:43 +01:00
Fr0zkaandClaude Opus 5.5 9d088ef96c research: UE arrival A/A could not run (L171)
Two identical ca0bf76 VFHost builds staged together crash at BeginPlay after snapshot parsing
(surface_strates=858993460 = 0x33333333, std::length_error). No arrival pairs measured.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
2026-11-02 18:08:43 +01:00
Fr0zkaandClaude Opus 5.5 24803ab771 streaming: latch fast-travel early admission until the arrival set settles (L170, still opt-in)
Switches verified inactive before first settlement; moving gains hold (near 43.7 -> 55.5%, far holes 1.8 -> 0%, p99
7.21 -> 6.48 ms, worst 12.32 -> 12.27 ms). Arrival median +1.28% (4/7 pairs slower) though the switches did no work
then; defaults remain off (WaitForAllCriticalTiles=1, ApplyBudget=0). Source/: VoxelWorld.cpp, VoxelWorld.h.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
2026-11-02 17:28:52 +01:00
Fr0zkaandClaude Opus 5.5 786ecc4cdf streaming: bounded early admission - partial level sort, apply preflight, sorted transition index (L169, opt-in)
With StreamingFastTravelWaitForAllCriticalTiles=0 and ApplyBudget=1 ms (test values; defaults stay 1 and 0), 7 pairs
vs ca0bf76: near 43.3 -> 56.1%, forward 15.5 -> 17.9%, far holes 1.8 -> 0.0%, p99 7.06 -> 6.52 ms, worst 12.2 ->
11.8 ms, every run p99 <= 7.4 and worst <= 13.8 ms; arrival 1.671 -> 1.683 s (slower in 7/7) so not promoted.
Source/: VoxelWorld.cpp, VoxelWorld.h.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
2026-11-02 16:45:20 +01:00
Fr0zkaandClaude Opus 5.5 ca0bf767ca streaming: incremental desired set on by default (L168)
Defaults after this commit: StreamingIncrementalDesiredSet=1 (Core delta, overlay, priority merge);
StreamingCoreDesiredCacheByCenter=0, StreamingRebuildOnCoreRegionCompletion=1, StreamingFastTravelForwardFirst=0,
StreamingFastTravelWaitForAllCriticalTiles=1, StreamingFastTravelApplyBudgetMilliseconds=0 (L157/L160 stay off);
audits 0. Default-only 30 m/s UE, 5 pairs vs c6357b5: p99 10.07 -> 7.01 ms (all pairs), worst tick 13.24 -> 11.63 ms
(4/5), near +1.4 pp, far holes -0.19 pp, arrival flat. Audit run 307/307/302 equal. Source/: VoxelWorld.cpp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
2026-11-02 14:34:39 +01:00
Fr0zkaandClaude Opus 5.5 ec89b29853 research: re-test L157/L160 on the incremental base (L167, none promoted)
7 runs each: L157 near 55.4% but a 16.7 ms worst tick (UpdateChunks/level sort/overlay); L160 tick-safe but lower
all-around near and later arrival; both: forward cone 25.2% and p99 <= 11 ms, but worst tick up and a 14.3 ms run.
Also found: voxel.StreamingIncrementalDesiredSet defaults to 0, so L165/L166 gains need it enabled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
2026-11-02 13:44:40 +01:00
Fr0zkaandClaude Opus 5.5 c6357b573f streaming: incremental overlay materialization and same-center priority merge (L166)
Persistent Core/overlay/candidate sets updated from deltas; same-center priority order merges changed keys (full sort
only on center change). Set and order equality audits 28/28 + 28/28, injected mismatch fails. 30 m/s UE, 7 pairs vs
9c70053: p99 10.52 -> 7.42 ms (all pairs), worst tick 12.98 -> 11.65 ms (6/7), near 42.0 -> 43.7% and forward cone
14.8 -> 16.1% (all pairs), far holes 3.20 -> 2.74% median (4/7), arrival flat. Source/: VoxelWorld.cpp, VoxelWorld.h.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
2026-11-02 13:00:55 +01:00
Fr0zkaandClaude Opus 5.5 9c70053543 streaming: incremental clipmap desired-set delta for adjacent crossings (L165)
BuildDelta emits entering/leaving slabs per LOD (incl. coarser-ring coverage changes); cached Core key set updated
from the delta at one-tile crossings, full rebuild for larger jumps or LOD>=4 changes. Equality audit vs full rebuild
(45/45 crossings, 20/20 sampled; injected mismatch asserts) + 630 automation cases. 30 m/s UE, 5 pairs:
BuildDesiredTiles p95 7.80 -> 4.08 ms, p99 11.46 -> 7.97 ms, worst tick 13.73 -> 12.11 ms (all pairs), near and
forward readiness and arrival better in all pairs; far holes better in 4/5 (pair 5 +0.29 pp). Geometry unchanged.
Source/: VoxelClipmapDesiredTiles.h, VoxelWorld.cpp, VoxelWorld.h, Tests/VoxelForgeClipmapNoOverlapTest.cpp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
2026-11-02 12:01:25 +01:00
Fr0zkaandClaude Opus 5.5 3b71d35eba experiment: LivingPlane repeatable + standalone UE runtime module (L164)
Readback barrier fix (GL_TEXTURE_UPDATE_BARRIER_BIT before glGetTexImage); check_repeatability.py compares full-field
checksums across query/capture settings - Intel GPU PASS (3b9d45905c7aafff), llvmpipe PASS, mismatch control fails.
New runtime module VoxelForgeLivingPlane (separate from generation): RDG global shader (Shaders/LivingPlane.usf),
ping-pong RGBA32F render targets, actor with WPO plane and object inputs, no CPU readback. Compiles in Linux UE 5.7.
Windows build: Source/VoxelForgeLivingPlane/*, Shaders/LivingPlane.usf, VoxelForge.uplugin. Setup: UE-SETUP.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
2026-11-02 11:10:43 +01:00
Fr0zkaandClaude Opus 5.5 e1d87eed26 experiment: LivingPlane readable + alive mode (L163)
RGBA32F displacement/velocity/imprint field; rolling base relief, grazing light, elevation colours, 7.5 cm contours,
follow camera; scenes: roll, rest and wobble, lift-off, second sphere; alive mode = travelling swells + viscoelastic
imprint. Intel ADL GT2: ~0.17 ms/step at 512^2. Open: run-to-run difference with queries on/off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
2026-11-02 10:55:32 +01:00
Fr0zkaandClaude Opus 5.5 5b15595e4a experiment: LivingPlane - GPU wave-equation height field reacting to a moving sphere (L162)
Standalone EGL/OpenGL 4.3 compute runner (runtime GLSL, no shader compiler needed), ping-pong height/velocity field,
damped wave + foundation spring + Gaussian sphere push + optional breathing, vertex-displaced render to PNG.
Independent of generation. Intel ADL GT2: ~0.08 ms/step at 512^2, ~0.29 ms/step at 1024^2 (wall-time difference).
.gitignore now allows Experiments/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
2026-11-02 10:35:37 +01:00
Fr0zkaandClaude Opus 5.5 aa458d3607 streaming: ahead-first admission, same-center rebuild suppression, Core key cache; off by default (L160)
30 m/s UE, 5 pairs: forward-cone readiness 14.2 -> 18.2% (+3.75 pp), all-around 39.5 -> 36.6%; p99 <= 11 ms in all
candidate runs, but worst tick, far holes and arrival regressed in some pairs. Rebuilds per crossing 7.2 -> 4.4; per
rebuild cost unchanged (not yet incremental). Source/: VoxelWorld.cpp, VoxelWorld.h (defaults unchanged).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
2026-11-02 09:29:39 +01:00
Fr0zkaandClaude Opus 5.5 323fbe3736 streaming: early-admission + apply-cap experiment controls, off by default; forward-cone metric (L157)
30 m/s UE, 5 pairs: near coverage 36.1 -> 48.6% but 2/5 runs had p99 15.8/18.2 ms (worst 19.9 ms), so controls stay
off. Measured: only ~23% of desired LOD0-2 tiles lie in the 55 deg forward cone; forward readiness is 12.1% (base) /
16.9%, vs 50.9/68.5% behind. Slow frames are dominated by desired-set rebuild/UpdateChunks, not apply. Source/:
VoxelWorld.cpp (default behaviour unchanged).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
2026-11-02 03:21:19 +01:00
42 changed files with 9882 additions and 143 deletions

No files matched your search

+8
View File
@@ -9,12 +9,20 @@
!/Source/ !/Source/
!/Source/** !/Source/**
# Plugin runtime shader source.
!/Shaders/
!/Shaders/**
# Portable CPU generator and D3D11 viewer. # Portable CPU generator and D3D11 viewer.
!/Standalone/ !/Standalone/
!/Standalone/** !/Standalone/**
/Standalone/Build/ /Standalone/Build/
/Standalone/bin/ /Standalone/bin/
# Research and prototype experiments.
!/Experiments/
!/Experiments/**
# The guarded single-entry test harness is source, not generated output. # The guarded single-entry test harness is source, not generated output.
!/Tools/ !/Tools/
!/Tools/** !/Tools/**
+1 -1
View File
@@ -7,7 +7,7 @@ As of 2026-10-31. L131 is on `assembler/world` as `8107196`; this L132 worktree
- **Core:** evaluator version **227**. Unreal's `AVoxelWorld::bUseVoxelForgeCore` is opt-in and defaults to `false`; the legacy generator remains the default runtime path. [Core version](Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h), [runtime switch](Source/VoxelForge/Public/VoxelWorld.h), [L130 report](Standalone/Research/WorldAssemblerM3/Linux130/REPORT.md). - **Core:** evaluator version **227**. Unreal's `AVoxelWorld::bUseVoxelForgeCore` is opt-in and defaults to `false`; the legacy generator remains the default runtime path. [Core version](Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h), [runtime switch](Source/VoxelForge/Public/VoxelWorld.h), [L130 report](Standalone/Research/WorldAssemblerM3/Linux130/REPORT.md).
- **Verified owner4 generator dispatch (L132):** SW binds `SurfaceWorld-native` because the owner export contains a matching `SurfaceWorld.Heightfield2D` v2 plan. FI binds `compiled-analytic/FloatingIslands` because its `FloatingIslands.ClusterField` v0 layout has no matching native island plan. TN and VS bind `compiled-analytic/TunnelNetwork` and `compiled-analytic/VerticalShafts`; their legacy recipe IDs provide neither a `NativeCaves`/`NativeLattice` graph recipe nor native field capability. The L132 fast-tier report prints the binding and generated tile-center share at each owner strate. [Dispatch](Standalone/Core/src/ShapeWorldInternal.h), [adapter recipe checks](Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h), [L132 report](Standalone/Research/WorldAssemblerM3/Linux132/REPORT.md). - **Verified owner4 generator dispatch (L132):** SW binds `SurfaceWorld-native` because the owner export contains a matching `SurfaceWorld.Heightfield2D` v2 plan. FI binds `compiled-analytic/FloatingIslands` because its `FloatingIslands.ClusterField` v0 layout has no matching native island plan. TN and VS bind `compiled-analytic/TunnelNetwork` and `compiled-analytic/VerticalShafts`; their legacy recipe IDs provide neither a `NativeCaves`/`NativeLattice` graph recipe nor native field capability. The L132 fast-tier report prints the binding and generated tile-center share at each owner strate. [Dispatch](Standalone/Core/src/ShapeWorldInternal.h), [adapter recipe checks](Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h), [L132 report](Standalone/Research/WorldAssemblerM3/Linux132/REPORT.md).
- **Passages:** continuous smooth swept tubes with a continuous floor profile. L125 stamps conservative route slabs and outside sentinels into the tile cache; exact route geometry is still evaluated. All 21,425 route samples passed the open-air walk check; the nine-station floor probe's maximum root offset was 7.884 mm. L130 sets the shared influence reach to **1.0533 m** (0.75 m shell + 0.21 m relief + 0.50 m smooth-max support / 6 + 0.01 m margin), converted once from metres to Core voxels for the proof, feature, fallback, and off-grid bounds. [L125](Standalone/Research/WorldAssemblerM3/Linux125/REPORT.md), [L130](Standalone/Research/WorldAssemblerM3/Linux130/REPORT.md). - **Passages:** continuous smooth swept tubes with a continuous floor profile. L125 stamps conservative route slabs and outside sentinels into the tile cache; exact route geometry is still evaluated. All 21,425 route samples passed the open-air walk check; the nine-station floor probe's maximum root offset was 7.884 mm. L130 sets the shared influence reach to **1.0533 m** (0.75 m shell + 0.21 m relief + 0.50 m smooth-max support / 6 + 0.01 m margin), converted once from metres to Core voxels for the proof, feature, fallback, and off-grid bounds. [L125](Standalone/Research/WorldAssemblerM3/Linux125/REPORT.md), [L130](Standalone/Research/WorldAssemblerM3/Linux130/REPORT.md).
- **Streaming:** arrival uses a bounded startup apply burst: `voxel.StreamingStartupApplyBoost=1` by default, up to 24 visible meshes or 14 ms per frame for at most 30 seconds, until the initial desired set and walkable ring are ready. `voxel.StreamingFarApplySupplement=1` is also default-on: after the normal visible apply cap, at most one far result applies when under half the apply-time budget; it is inactive during the startup burst. Dynamic near reserve defaults on; coarse-first remains off; the legacy pending-result cap defaults off. The far-apply lane is Unreal-only. [CVar source](Source/VoxelForge/Private/VoxelWorld.cpp), [L121 report](Standalone/Research/WorldAssemblerM3/Linux121/REPORT.md), [architecture](ARCHITECTURE.md). - **Streaming:** arrival uses a bounded startup apply burst: `voxel.StreamingStartupApplyBoost=1` by default, up to 24 visible meshes or 14 ms per frame for at most 30 seconds, until the initial desired set and walkable ring are ready. `voxel.StreamingFarApplySupplement=1` is also default-on: after the normal visible apply cap, at most one far result applies when under half the apply-time budget; it is inactive during the startup burst. Dynamic near reserve defaults on; coarse-first remains off; the legacy pending-result cap defaults off. The L165/L166 incremental desired-set path now defaults on (`voxel.StreamingIncrementalDesiredSet=1`): Core clipmap keys, overlay membership, and priority order update incrementally, with same-center Core keys cached. No separate overlay or priority incremental switches are required. Audits stay off (`StreamingIncrementalDesiredAuditEvery=0`, `StreamingIncrementalDesiredMaterializationAuditEvery=0`, mismatch injection `=0`). L157 early admission and apply-cap controls stay off (`StreamingFastTravelWaitForAllCriticalTiles=1`, `StreamingFastTravelApplyBudgetMilliseconds=0`). L160 ahead-first and same-center rebuild suppression stay off (`StreamingFastTravelForwardFirst=0`, `StreamingRebuildOnCoreRegionCompletion=1`). Its separate Core desired-cache switch stays off (`StreamingCoreDesiredCacheByCenter=0`); the new incremental path enables its own same-center Core cache. The far-apply lane is Unreal-only. [CVar source](Source/VoxelForge/Private/VoxelWorld.cpp), [L121 report](Standalone/Research/WorldAssemblerM3/Linux121/REPORT.md), [architecture](ARCHITECTURE.md).
- **Determinism:** proven on Linux with Clang and GCC at W1/W8/W12. L131's unchanged-output comparison covers owner4's SW/FI/TN/VS views in 24 compiler/worker cases; the canonical owner4/owner2 recipe matrix also passes. Windows determinism is unverified. [L131 checks](Standalone/Research/WorldAssemblerM3/Linux131/REPORT.md), [v227 canonical hashes](Standalone/Tools/verify/baselines/owner-hash/canonical/). - **Determinism:** proven on Linux with Clang and GCC at W1/W8/W12. L131's unchanged-output comparison covers owner4's SW/FI/TN/VS views in 24 compiler/worker cases; the canonical owner4/owner2 recipe matrix also passes. Windows determinism is unverified. [L131 checks](Standalone/Research/WorldAssemblerM3/Linux131/REPORT.md), [v227 canonical hashes](Standalone/Tools/verify/baselines/owner-hash/canonical/).
## Current performance references ## Current performance references
File diff suppressed because it is too large. Load diff
+201
View File
@@ -0,0 +1,201 @@
# L162 — Living Plane
The EGL/OpenGL 4.3 runner builds and passes its llvmpipe correctness and frame checks. The sandbox has no accessible DRM render node, so it did not run or report Intel GPU performance. The runner refuses `--device=gpu` here instead of falling back to software.
## Implementation
`LivingPlane.cpp` runs a ping-pong `RG32F` height/velocity field through a runtime-compiled GLSL compute shader. The damped wave update includes a weak foundation spring, a Gaussian sphere push, and an optional slow breathing force. The spring is needed because a persistent local push otherwise accelerates the zero-frequency height mode and sinks the whole plane. The sphere moves across the plane, stops, and bobs. A vertex-displaced mesh and lit sphere render to an EGL pbuffer-backed framebuffer and save as PNGs.
The timestep is selected from the largest 2D grid eigenfrequency:
`dt = 1.30 / sqrt(8*c*c/(dx*dx) + foundationStiffness)`
The resulting spectral step ratio is 1.30, below the explicit update's 2.0 stability limit. The runner warms the compute kernel for eight unmeasured dispatches, then records one `GL_TIME_ELAPSED` query per requested simulation step in `timing.json`.
The first-step equation audit also caught a GL state error during bring-up: frame rendering left the graphics program active before the next compute dispatch. `Dispatch` now binds the compute program before every step, so the audit fails if a frame silently stops simulation.
Device selection enumerates EGL devices and uses `EGL_EXT_platform_device` for an accessible Intel render node first. On this host, the known integrated device is Intel on `renderD128`. Automatic mode falls back to a surfaceless Mesa context with `LIBGL_ALWAYS_SOFTWARE=1`; explicit GPU mode fails when no accessible hardware node is present. EGL does not expose Vulkan's integrated/discrete enum, so Intel vendor/render-node identity is the selection heuristic.
The Unreal handoff is in [UNREAL-MAPPING.md](UNREAL-MAPPING.md). It keeps the prototype render-only and describes the density mapping if the surface later needs to affect voxel queries. No Unreal `Source/` files changed.
## Measurements and checks
- Built with `Experiments/LivingPlane/build.sh` using the installed EGL, desktop GL, and libpng development files; no shader compiler or package install was needed.
- Final llvmpipe run: 128² field, 360 steps, `dt=9.56814 ms`, spectral ratio 1.30. The first-step equation audit and finite-field/dynamics audit passed. Final field range was −0.139533 m to +0.0366704 m. It wrote 16 128×128 PNGs and a JSON file containing 360 per-step query samples. The frames were inspected; the moving sphere and wake are visible.
- Auto-selection smoke check: 32², 12 steps; selected llvmpipe and passed both audits. Breathing smoke check: 64², 180 steps with `--breathing=on`; passed both audits.
- Negative device-selection control: `--device=gpu` returned an error because the sandbox has no accessible EGL DRM render node.
- Existing fast verification tier passed: 278.9 seconds, ending in `SUMMARY PASS`. Its build and logs are under `Saved/Linux/l162/verify/` and `Saved/Linux/l162/`.
- The llvmpipe `GL_TIME_ELAPSED` samples are preserved in JSON but are software-driver query values, not GPU timings. The requested hardware timing pass remains to be run outside the sandbox.
- No Unreal build was run; no Unreal `Source/` files changed.
## Intel GPU pass
Run these from the plugin root as `jahni`. The frame run writes 512×512 PNGs to `Saved/Linux/l162/gpu-frames-512/`; the two timing runs write `timing.json` with one GPU query per step and skip PNG capture. They still read the field for the first-step and final-state correctness audits.
Before the timing runs, check `uptime` and run them on a quiet machine (load below about 1.5, with no other build active).
```sh
uptime
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s ./Saved/Linux/l162/living-plane --device=gpu --size=512 --steps=1800 --capture-every=12 --out=Saved/Linux/l162/gpu-frames-512
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s ./Saved/Linux/l162/living-plane --device=gpu --size=512 --steps=1800 --capture-every=0 --out=Saved/Linux/l162/gpu-timing-512
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s ./Saved/Linux/l162/living-plane --device=gpu --size=1024 --steps=1800 --capture-every=0 --out=Saved/Linux/l162/gpu-timing-1024
```
Encode the captured sequence as MP4 with:
```sh
ffmpeg -y -framerate 30 -i Saved/Linux/l162/gpu-frames-512/frame_%06d.png -c:v libx264 -pix_fmt yuv420p Saved/Linux/l162/living-plane-512.mp4
```
## Files and remaining risk
The only changed file outside `Experiments/LivingPlane/` is `.gitignore`, which now allows `Experiments/` and its contents. GPU timings and Intel render-node selection remain unverified until the commands above run on the host outside this sandbox.
## Intel GPU pass (run by the orchestrator outside the sandbox, 2026-11-02)
- The device was selected correctly: `Mesa Intel(R) Graphics (ADL GT2)`, OpenGL 4.6, `/dev/dri/renderD128`. 512²,
1,800 steps, 151 frames; the equation audit passes. MP4: `Saved/Linux/l162/living-plane-512.mp4` (local).
- **The `GL_TIME_ELAPSED` per-dispatch values (~0.0001 ms) are not credible** for 262k cells, so they aren't
reported as timings.
- Wall-time difference between 1,800 and 200 steps, divided by the 1,600 steps (start-up cancels out; this includes
submission, so it's an upper bound): **0.080 ms/step at 512²** and **0.292 ms/step at 1024²**.
- Visual read: the effect is very subtle in the current render (a slight dip under the sphere, faint rings). It needs
a larger push amplitude, grazing light or height colouring, and a closer camera to read.
## L163 — readable, living plane
The plane now reads as rolling terrain, and the objects leave visible troughs and wakes. `alive` adds travelling
breathing and a viscoelastic imprint that remains after lift-off. The simulation still runs in one GPU dispatch per
step; CPU work only supplies the two object states.
`LivingPlane.cpp` now keeps the dynamic displacement, velocity, and imprint in an `RGBA32F` ping-pong field. The
surface shader adds broad static terrain relief, calculates normals from relief plus displacement, and uses grazing
light, elevation colours, slope tint, and 7.5 cm contour bands. The camera follows the first sphere at a closer angle.
The wake scene rolls the first sphere, pauses and wobbles it, lifts it, then sends a second sphere through the wake.
The alive scene runs that same sequence with low-frequency travelling swells and a decaying contact imprint. Its
defaults are breathing strength 4.8 m/s², imprint capture 0.15 m/s, recovery time 6 s, and maximum imprint depth
0.34 m. These values have command-line overrides.
The timer query check found no early read or reuse in L162: it allocated a unique query per step, called `glFinish`,
then read every result. The saved Intel/Mesa 25.0.7 samples are 0 or 52 ns for most dispatches (512² median 0,
p95 52 ns; largest sample 17.552 µs), which cannot be a credible per-dispatch cost. The old usage did not reuse
query objects, so lifetime is not the cause; the saved data point to Intel/Mesa's timer-query reporting or timing
scope, but the sandbox cannot isolate which. L163 uses a 64-query ring, reads each query before reusing its slot,
checks availability after `glFinish`, and records `GL_QUERY_COUNTER_BITS`. On llvmpipe the availability check passed
with 64 counter bits and zero waits. The Intel query result still needs the GPU diagnostic command below.
### Measurements and checks
- llvmpipe wake: 128², 600 steps (`dt=9.56814 ms`, 5.741 s simulation window), 151 frames at 30 fps. First-step
equation audit and timer-query lifetime check passed; final displacement range was −0.177488 m to +0.104854 m.
- llvmpipe alive: same grid, steps, and window; equation audit and timer-query lifetime check passed. Final
displacement range was −0.200437 m to +0.115904 m, with a −0.111667 m stored imprint.
- llvmpipe cost: 512², 1,800 steps, `--queries=off`, no frame capture: **0.767473 ms/step** in one timed
simulation-loop window. `timing.json` includes dispatch submission, the step-0 equation readback, and final
`glFinish`, but excludes setup and frame capture. This is a software-renderer measurement, not the Intel GPU cost.
- Both five-second llvmpipe previews are in `Saved/Linux/l163/llvmpipe-wake.mp4` and
`Saved/Linux/l163/llvmpipe-alive.mp4`. They are 128² review previews; use the GPU commands below for the owner
videos.
- Equation-audit negative control: a temporary copy with its expected first-step height forced to zero exited FAIL
at actual −0.0488777 m. The final source passes that same check.
- The fast verification tier passed in 278.5 s. Build and logs are under `Saved/Linux/l163/verify/`.
- No Unreal `Source/` files changed and no Unreal build ran.
### Intel GPU run
Run from the plugin root as `jahni`. First build the runner. For timed runs, check `uptime` and keep the machine
quiet. The wall-time field is valid only when `--capture-every=0`; use `--queries=off` for the step-cost measurement.
```sh
bash Experiments/LivingPlane/build.sh
uptime
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s ./Saved/Linux/l163/living-plane --device=gpu --scene=wake --size=512 --steps=2400 --capture-every=16 --queries=off --out=Saved/Linux/l163/gpu-wake-512
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s ./Saved/Linux/l163/living-plane --device=gpu --scene=alive --size=512 --steps=2400 --capture-every=16 --queries=off --out=Saved/Linux/l163/gpu-alive-512
ffmpeg -y -framerate 30 -i Saved/Linux/l163/gpu-wake-512/frame_%06d.png -c:v libx264 -pix_fmt yuv420p Saved/Linux/l163/living-plane-wake-512.mp4
ffmpeg -y -framerate 30 -i Saved/Linux/l163/gpu-alive-512/frame_%06d.png -c:v libx264 -pix_fmt yuv420p Saved/Linux/l163/living-plane-alive-512.mp4
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s ./Saved/Linux/l163/living-plane --device=gpu --scene=alive --size=512 --steps=1800 --capture-every=0 --queries=off --out=Saved/Linux/l163/gpu-timing-512
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s ./Saved/Linux/l163/living-plane --device=gpu --scene=alive --size=512 --steps=1800 --capture-every=0 --queries=on --out=Saved/Linux/l163/gpu-query-check-512
```
`gpu-timing-512/timing.json` reports the 512² batch wall time per step, including dispatch submission and completion.
`gpu-query-check-512/timing.json` contains query counter bits, waits before slot reuse, availability status, and raw
per-step query samples. The sandbox has no accessible Intel DRM node, so the current Intel step cost and new query
diagnostic remain unmeasured here.
### L163 Intel GPU pass (run by the orchestrator, 2026-11-02)
- Intel ADL GT2 (Mesa 25.0.7, `renderD128`): wake and alive scenes, 512², 2,400 steps, 151 frames each; equation
audits PASS. Videos (local): `Saved/Linux/l163/living-plane-wake-512.mp4`, `living-plane-alive-512.mp4`.
- Step cost, alive scene, 512², 1,800 steps, no capture: **0.172 ms/step** with queries off, 0.152 ms/step with
queries on (batch wall time per step).
- The Intel `GL_TIME_ELAPSED` median is still 0 with 64 counter bits and the ring/availability checks passing. That
looks like a Mesa/Intel reporting issue; use wall time.
- **Open issue:** the same scene with the same steps gave different final fields (min −0.637 m with queries off vs
−0.327 m with queries on). Something depends on more than the step index (e.g. wall-clock time); investigate.
## L164 — repeatable field readback and UE runtime actor
**Result:** the repeatability check passes on llvmpipe, and the new `VoxelForgeLivingPlane` runtime module compiles
in the Linux UE 5.7 host. The surface actor is independent of VoxelForge generation and sends no field readback to
the CPU.
### Repeatability finding
`uTime` is `Step * dt`, and `GetBodiesAt` receives that same value. Neither the object schedule nor the breathing
force uses wall time or a rendered-frame counter. Every compute step already issued an image-access barrier before
the next ping-pong image read. The defect was on the diagnostic side: the step-zero and final `glGetTexImage`
readbacks followed shader image writes without `GL_TEXTURE_UPDATE_BARRIER_BIT`. The Khronos OpenGL specification
requires that barrier for `GetTexImage` to observe earlier shader writes ([§7.12](https://registry.khronos.org/OpenGL/specs/gl/glspec44.compatibility.pdf)). Query/capture pacing could therefore affect the reported field values without changing any simulation input. The barrier is now issued before both readbacks.
`check_repeatability.py` runs the same 512² alive scene for 1,800 steps twice: queries off with no captures, then
queries on with captures every 900 steps. The checksum covers every final RGBA32F field value using canonical
little-endian float bytes.
- llvmpipe, queries off/capture off: field range −0.637382 m to +0.109308 m; checksum
`1ba08f5dafc01f82`.
- llvmpipe, queries on/capture every 900: the same range and checksum; equation and timer-query audits pass; three
frames written.
- Negative control: changing one saved checksum made the comparison exit 1 with `FAIL field checksum mismatch`.
- The llvmpipe runs were each one 1,800-step simulation-loop window. Logs and JSON are under
`Saved/Linux/l164/repeatability/`.
The Intel run is not available in this sandbox. The owner should repeat the checksum comparison on the GPU to
confirm that the missing transfer-readback barrier accounts for the earlier Intel min mismatch:
```sh
python3 Experiments/LivingPlane/check_repeatability.py --device=gpu --out=Saved/Linux/l164/gpu-repeatability
```
### Unreal implementation and checks
The plugin descriptor adds a separate `VoxelForgeLivingPlane` runtime module. Its RDG global shader is in
`Shaders/LivingPlane.usf`; module startup maps `/Plugin/VoxelForgeLivingPlane` to the plugin shader directory. The
actor owns two UAV-capable RGBA32F render targets, advances them at the stable fixed step, and supplies the current
texture to a movable plane mesh material. Its Blueprint/C++ object input includes world position, radius, velocity,
base push, and active state. The CPU converts those values to plane-local metres and uploads uniforms; it does not
read the field back. The ignore rules now include `Shaders/` so that the shader ships in the worktree diff.
`Experiments/LivingPlane/UE-SETUP.md` gives level setup, the full material graph, and object driving steps.
- Linux UE 5.7 `VFHostEditor Linux Development`: **PASS**, final incremental build compiled
`VoxelForgeLivingPlaneActor.cpp` and linked `libUnrealEditor-VoxelForgeLivingPlane.so`. Command used
`-NoUBA -MaxParallelActions=4`, `taskset -c 0-11`, and no `prlimit`. Log:
`Saved/Linux/l164/ue-build-final.log`. The initial attempt exposed an incorrect RDG texture registration overload;
it had already compiled `VoxelForgeLivingPlaneModule.cpp` and the generated module code. The corrected final
build passed.
- The host `PluginDeps/VoxelForge/Source` and `.uplugin` links were temporarily pointed at this worktree for the
build and restored to `/home/jahni/VoxelM/Plugins/VoxelForge` afterward. The temporary shader link was removed.
- Standalone build, final repeatability comparison, checksum-mismatch negative control, plugin JSON parse, and
`git diff --check` passed.
- Not run: fast generation verification (generation sources are untouched), Intel GPU checksum comparison, or a UE
editor render/shader-runtime check. The Linux UBT build verifies C++ and module integration, not first-use shader
compilation or material appearance.
New Unreal source and shader files for the owner's Windows build:
- `Source/VoxelForgeLivingPlane/VoxelForgeLivingPlane.Build.cs`
- `Source/VoxelForgeLivingPlane/Public/VoxelForgeLivingPlaneActor.h`
- `Source/VoxelForgeLivingPlane/Private/VoxelForgeLivingPlaneActor.cpp`
- `Source/VoxelForgeLivingPlane/Private/VoxelForgeLivingPlaneModule.cpp`
- `Shaders/LivingPlane.usf`
No VoxelForge generation module files changed. Remaining risks are the Intel checksum confirmation and first-use UE
shader/material rendering, which require the owner's GPU/editor run.
+55
View File
@@ -0,0 +1,55 @@
# Living Plane in Unreal Engine 5.7
`VoxelForgeLivingPlane` is a separate runtime module. It does not call the VoxelForge terrain generators and it does not read the simulated texture back to the CPU. The actor owns two transient `RGBA32f` render targets and advances them with an RDG compute pass. The CPU sends object state and fixed-step uniforms only.
## Add it to a level
1. Enable the VoxelForge plugin and restart the editor so the `VoxelForgeLivingPlane` runtime module and its shader mapping load.
2. Create a material named `M_LivingPlane` using the graph below.
3. Place `VoxelForgeLivingPlaneActor` in the level. Assign `M_LivingPlane` to **Plane Material**.
4. Keep the actor on the world XY plane with unit actor scale. `Plane Length Meters` controls the mesh width and simulation width; the built-in plane mesh is scaled to that size.
5. The actor creates and clears its two GPU state targets on `BeginPlay`. The mesh has collision disabled; this surface is render-only and does not change VoxelForge density or collision queries.
## Material graph
Use a Surface, Opaque material. The actor binds these parameters on its dynamic material instance:
- `LivingField`: Texture Object Parameter. The texture is the current RGBA32f simulation state; **R** is displacement in metres.
- `PlaneLengthMeters`: Scalar Parameter.
- `FieldSize`: Scalar Parameter, equal to the render target width and height.
For World Position Offset:
1. Read `LivingField` at `TextureCoordinate[0]` with a Texture Sample.
2. Multiply its **R** output by `100` to convert metres to Unreal centimetres.
3. Append `(0, 0)` to that value, making `(0, 0, heightCm)`, and connect it to **World Position Offset**.
For the displaced normal, sample the height at four UV offsets using the same texture object:
- Left/right UV: `TexCoord + (-1/FieldSize, 0)` and `TexCoord + (1/FieldSize, 0)`.
- Down/up UV: `TexCoord + (0, -1/FieldSize)` and `TexCoord + (0, 1/FieldSize)`.
- `dHdx = (Right.R - Left.R) * FieldSize / (2 * PlaneLengthMeters)`.
- `dHdy = (Up.R - Down.R) * FieldSize / (2 * PlaneLengthMeters)`.
- Make and normalize the tangent-space vector `(-dHdx, -dHdy, 1)`, then connect it to **Normal**.
The built-in plane's U/V tangent directions align with local X/Y. Use a neutral base colour and roughness around `0.7` to start. The graph assumes a horizontal plane; add a local-to-world transform if the plane needs arbitrary rotation.
## Drive objects
`ObjectInputs` has two slots. Each `FVFLivingPlaneObjectInput` contains world position in centimetres, world velocity in centimetres per second, radius in centimetres, base push acceleration in metres per second squared, and an active flag. Blueprint can use **Set Object State** or set the array entry; C++ can call the same method. Set the object's center about one radius above the plane while it contacts it, and mark the slot inactive after lift-off.
The compute shader uses the radius to scale the Gaussian contact footprint. The original wave equation keeps the L163 wave speed, damping, foundation spring, travelling breathing force, and recovering imprint. The base push acceleration supplies the object force; lateral speed adds `0.25 m/s²` per `1 m/s`, and downward speed adds `0.5 m/s²` per `1 m/s`. Upward velocity does not add force. Object positions and velocities are transformed into the plane's local frame and converted from centimetres to metres before upload.
Each step uses `dt = 1.30 / sqrt(8*c*c/(dx*dx) + foundationStiffness)`, with `dx = PlaneLengthMeters / FieldSize`. Simulation time passed to breathing is the fixed step index times `dt`. Game-thread delta time only determines how many fixed steps are queued; no GPU field data is read back in the normal path.
## Build files
The owner's Windows build needs every added source and shader file:
- `Source/VoxelForgeLivingPlane/VoxelForgeLivingPlane.Build.cs`
- `Source/VoxelForgeLivingPlane/Public/VoxelForgeLivingPlaneActor.h`
- `Source/VoxelForgeLivingPlane/Private/VoxelForgeLivingPlaneActor.cpp`
- `Source/VoxelForgeLivingPlane/Private/VoxelForgeLivingPlaneModule.cpp`
- `Shaders/LivingPlane.usf`
The `VoxelForge.uplugin` runtime-module entry must ship with them.
+33
View File
@@ -0,0 +1,33 @@
# Unreal mapping note
The prototype stores height, vertical velocity, a recovering imprint, and a spare
channel in a pair of `RGBA32F` textures. A world-owned Unreal component can keep
equivalent transient render graph textures and advance them with a compute pass. It
should own simulation time and clear or restore its buffers when the world is recreated.
The sphere path becomes a world-space interaction event: position, contact radius,
and impulse strength are supplied to the compute pass. For multiplayer or deterministic
replay, replicate the compact events and simulation tick, or record the resulting field;
do not assume that independent GPU floating-point runs will produce byte-identical
textures.
The rendered top surface can remain a render-only displaced mesh. If it must participate
in voxel collision or queries, its height must also reach the density evaluator. For a
height `h(x,y)`, the plugin's density convention is negative inside solid and positive
in air, so the plane's signed density is `z - h(x,y)`. Keep this calculation in metres
until the existing voxel/Core boundary converts with the active voxel size (25 cm by
default); do not mix metre distances and voxel counts in the wave solver.
The current sample is an isolated visual field. Wiring this into `AVoxelWorld` would
need an explicit owner for simulation lifetime, collision/update cadence, save/load,
and the replication policy before it could affect shipped terrain.
## L164 render-only Unreal implementation
`Source/VoxelForgeLivingPlane/` now implements this render-only path as a separate
runtime module. `AVoxelForgeLivingPlaneActor` owns two UAV-capable RGBA32F render
targets, advances the wave, breathing, and imprint equations with an RDG global shader,
and binds the current texture to its plane material. Blueprint/C++ supplies object
position, radius, velocity, push, and active state. The actor does not call the
generation modules and does not read the field back. Level setup and the material graph
are in [UE-SETUP.md](UE-SETUP.md).
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
PLUGIN_ROOT="$(cd -- "${SCRIPT_DIR}/../.." && pwd)"
OUTPUT_DIR="${PLUGIN_ROOT}/Saved/Linux/l164"
CXX="${CXX:-g++}"
command -v "${CXX}" >/dev/null || { echo "C++ compiler not found: ${CXX}" >&2; exit 2; }
pkg-config --exists egl gl libpng || {
echo "EGL, desktop OpenGL, and libpng development files are required" >&2
exit 2
}
mkdir -p "${OUTPUT_DIR}"
read -r -a GRAPHICS_FLAGS <<<"$(pkg-config --cflags --libs egl gl libpng)"
taskset -c 0-11 prlimit --as=6442450944 -- \
"${CXX}" -std=c++20 -O2 -g -Wall -Wextra -Wpedantic \
"${SCRIPT_DIR}/LivingPlane.cpp" -o "${OUTPUT_DIR}/living-plane" \
"${GRAPHICS_FLAGS[@]}"
echo "built ${OUTPUT_DIR}/living-plane"
+132
View File
@@ -0,0 +1,132 @@
#!/usr/bin/env python3
"""Compare final field checksums across different query and capture settings."""
from __future__ import annotations
import argparse
import json
import pathlib
import shutil
import subprocess
import sys
PLUGIN_ROOT = pathlib.Path(__file__).resolve().parents[2]
def read_checksum(path: pathlib.Path) -> str:
try:
with path.open(encoding="utf-8") as source:
result = json.load(source)
except (OSError, json.JSONDecodeError) as error:
raise RuntimeError(f"could not read {path}: {error}") from error
checksum = result.get("field_checksum_fnv1a64")
if not isinstance(checksum, str) or len(checksum) != 16:
raise RuntimeError(f"{path} has no 16-digit field checksum")
return checksum
def compare(left: pathlib.Path, right: pathlib.Path) -> bool:
left_checksum = read_checksum(left)
right_checksum = read_checksum(right)
if left_checksum != right_checksum:
print(
"FAIL field checksum mismatch: "
f"{left}={left_checksum}, {right}={right_checksum}",
file=sys.stderr,
)
return False
print(f"PASS field checksum matches: {left_checksum}")
return True
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--binary",
type=pathlib.Path,
default=PLUGIN_ROOT / "Saved/Linux/l164/living-plane",
)
parser.add_argument(
"--out",
type=pathlib.Path,
default=PLUGIN_ROOT / "Saved/Linux/l164/repeatability",
)
parser.add_argument("--device", choices=("auto", "cpu", "gpu"), default="cpu")
parser.add_argument("--size", type=int, default=512)
parser.add_argument("--steps", type=int, default=1800)
parser.add_argument(
"--compare-only",
nargs=2,
type=pathlib.Path,
metavar=("LEFT_JSON", "RIGHT_JSON"),
help="compare existing timing.json files; used to exercise the failure path",
)
options = parser.parse_args()
if options.compare_only:
return 0 if compare(*options.compare_only) else 1
if not options.binary.is_file():
parser.error(f"binary not found: {options.binary}; run Experiments/LivingPlane/build.sh")
options.out.mkdir(parents=True, exist_ok=True)
off_dir = options.out / "queries-off_capture-off"
on_dir = options.out / "queries-on_capture-every-900"
for directory in (off_dir, on_dir):
if directory.exists():
shutil.rmtree(directory)
common = [
f"--device={options.device}",
"--scene=alive",
f"--size={options.size}",
f"--steps={options.steps}",
]
runs = (
("--capture-every=0", "--queries=off", off_dir),
("--capture-every=900", "--queries=on", on_dir),
)
for capture, queries, output in runs:
command = [
"taskset",
"-c",
"0-11",
"prlimit",
"--as=6442450944",
"--",
"timeout",
"1740s",
str(options.binary.resolve()),
*common,
capture,
queries,
f"--out={output}",
]
print("RUN " + " ".join(command), flush=True)
log_path = options.out / f"{output.name}.log"
with log_path.open("w", encoding="utf-8") as log_file:
try:
subprocess.run(
command,
cwd=PLUGIN_ROOT,
check=True,
stdout=log_file,
stderr=subprocess.STDOUT,
)
except subprocess.CalledProcessError:
print(log_path.read_text(encoding="utf-8"), file=sys.stderr)
raise
print(log_path.read_text(encoding="utf-8").rstrip())
left = off_dir / "timing.json"
right = on_dir / "timing.json"
return 0 if compare(left, right) else 1
if __name__ == "__main__":
try:
raise SystemExit(main())
except (OSError, RuntimeError, subprocess.CalledProcessError) as error:
print(f"FAIL repeatability check: {error}", file=sys.stderr)
raise SystemExit(1)
+85
View File
@@ -0,0 +1,85 @@
Texture2D<float4> SourceState;
RWTexture2D<float4> TargetState;
int2 GridSize;
float Dt;
float Dx;
float PlaneLength;
float WaveSpeed;
float Damping;
float FoundationStiffness;
float TimeSeconds;
float2 BreathingSettings;
float4 ImprintSettings;
float4 ObjectPositionRadius[2];
float4 ObjectVelocityPush[2];
[numthreads(8, 8, 1)]
void MainCS(uint3 DispatchThreadId : SV_DispatchThreadID)
{
int2 Pixel = int2(DispatchThreadId.xy);
if (Pixel.x >= GridSize.x || Pixel.y >= GridSize.y)
{
return;
}
int2 Lo = int2(0, 0);
int2 Hi = GridSize - 1;
float4 State = SourceState.Load(int3(Pixel, 0));
float Height = State.r;
float Velocity = State.g;
float Imprint = State.b;
float Left = SourceState.Load(int3(max(Pixel + int2(-1, 0), Lo), 0)).r;
float Right = SourceState.Load(int3(min(Pixel + int2(1, 0), Hi), 0)).r;
float Down = SourceState.Load(int3(max(Pixel + int2(0, -1), Lo), 0)).r;
float Up = SourceState.Load(int3(min(Pixel + int2(0, 1), Hi), 0)).r;
float Laplacian = (Left + Right + Down + Up - 4.0 * Height) / max(Dx * Dx, 1.0e-8);
float2 World = (float2(Pixel) + 0.5) * Dx - PlaneLength * 0.5;
float Interaction = 0.0;
float Contact = 0.0;
for (int ObjectIndex = 0; ObjectIndex < 2; ++ObjectIndex)
{
float4 PositionRadius = ObjectPositionRadius[ObjectIndex];
if (PositionRadius.w <= 0.0)
{
continue;
}
float4 VelocityPush = ObjectVelocityPush[ObjectIndex];
float Radius = PositionRadius.w;
float Sigma = max(Radius * (0.58 / 0.43), Dx);
float2 Offset = World - PositionRadius.xy;
float Weight = exp(-dot(Offset, Offset) / max(2.0 * Sigma * Sigma, 1.0e-8));
float HeightAboveContact = max(PositionRadius.z - Radius, 0.0);
float ContactHeight = 1.0 - smoothstep(0.0, max(Radius, Dx), HeightAboveContact);
float ImpactSpeed = length(VelocityPush.xy) * 0.25 + max(-VelocityPush.z, 0.0) * 0.5;
float Push = max(VelocityPush.w, 0.0) + ImpactSpeed;
Interaction -= Push * Weight * ContactHeight;
if (Push > 0.0)
{
Contact = max(Contact, Weight * ContactHeight);
}
}
float Breathing = 0.0;
if (BreathingSettings.x != 0.0)
{
float Swell = sin(World.x * 0.48 - TimeSeconds * 0.22)
* cos(World.y * 0.42 + TimeSeconds * 0.16);
Breathing = BreathingSettings.y * Swell * sin(TimeSeconds * 0.48);
}
float NextImprint = Imprint;
if (ImprintSettings.x != 0.0)
{
NextImprint = clamp(Imprint * exp(-Dt / max(ImprintSettings.z, 1.0e-4))
- Contact * ImprintSettings.y * Dt, -ImprintSettings.w, 0.0);
}
float FoundationTarget = ImprintSettings.x != 0.0 ? Imprint : 0.0;
float Acceleration = WaveSpeed * WaveSpeed * Laplacian
- FoundationStiffness * (Height - FoundationTarget) + Interaction + Breathing;
float NextVelocity = (Velocity + Acceleration * Dt) * exp(-Damping * Dt);
float NextHeight = Height + NextVelocity * Dt;
TargetState[Pixel] = float4(NextHeight, NextVelocity, NextImprint, 0.0);
}
@@ -322,6 +322,108 @@ bool FVoxelForgeClipmapNoOverlapTest::RunTest(const FString& Parameters)
FirstFailure.IsEmpty() ? TEXT("none") : *FirstFailure)); FirstFailure.IsEmpty() ? TEXT("none") : *FirstFailure));
TestEqual(TEXT("all swept desired sets have zero overlapping pairs"), OverlapCases, 0); TestEqual(TEXT("all swept desired sets have zero overlapping pairs"), OverlapCases, 0);
TestEqual(TEXT("all swept desired sets cover the legacy footprint"), CoverageFailures, 0); TestEqual(TEXT("all swept desired sets cover the legacy footprint"), CoverageFailures, 0);
static const FIntVector DeltaCenters[] = {
FIntVector(-5, -2, -1), FIntVector(-3, 1, 2), FIntVector(-1, -3, 0),
FIntVector(0, 0, 0), FIntVector(1, 3, -2), FIntVector(3, -1, 1),
FIntVector(5, 2, -3),
};
static const FIntVector DeltaSteps[] = {
FIntVector(-1, 0, 0), FIntVector(1, 0, 0),
FIntVector(0, -1, 0), FIntVector(0, 1, 0),
FIntVector(0, 0, -1), FIntVector(0, 0, 1),
};
int32 DeltaCases = 0;
int32 DeltaFailures = 0;
FString FirstDeltaFailure;
for (const FIntVector& Center : DeltaCenters)
for (const FSweepSettings& Sweep : SweepSettings)
for (int32 ClampCase = 0; ClampCase < 3; ++ClampCase)
{
FParameters P;
P.ClipRadius = Sweep.Radius;
P.MaxClipLevel = Sweep.MaxLevel;
P.RenderDistanceChunks = Sweep.RenderDistance;
const auto GetVerticalBounds = [&](int32 CenterZ, int32& OutMinZ, int32& OutMaxZ)
{
OutMinZ = MIN_int32;
OutMaxZ = MAX_int32;
if (ClampCase == 1)
{
OutMinZ = CenterZ - 2;
OutMaxZ = CenterZ + 3;
}
else if (ClampCase == 2)
{
const int32 Band = VoxelClipmapDesiredTiles::FloorDiv(
CenterZ, 1 << Sweep.MaxLevel);
const int64 BandStart = static_cast<int64>(Band)
* (int64{1} << static_cast<uint32>(Sweep.MaxLevel));
OutMinZ = static_cast<int32>(BandStart - 1);
OutMaxZ = OutMinZ + 6;
}
};
int32 PreviousMinZ = 0;
int32 PreviousMaxZ = 0;
GetVerticalBounds(Center.Z, PreviousMinZ, PreviousMaxZ);
TArray<FVoxelTileKey> PreviousTiles;
VoxelClipmapDesiredTiles::Build(Center, PreviousMinZ, PreviousMaxZ, P, PreviousTiles);
for (const FIntVector& Step : DeltaSteps)
{
const FIntVector NextCenter = Center + Step;
int32 NextMinZ = 0;
int32 NextMaxZ = 0;
GetVerticalBounds(NextCenter.Z, NextMinZ, NextMaxZ);
TArray<FVoxelTileKey> Entering;
TArray<FVoxelTileKey> Leaving;
VoxelClipmapDesiredTiles::BuildDelta(
Center, PreviousMinZ, PreviousMaxZ,
NextCenter, NextMinZ, NextMaxZ, P, Entering, Leaving);
TSet<FVoxelTileKey> Updated;
for (const FVoxelTileKey& Tile : PreviousTiles) Updated.Add(Tile);
for (const FVoxelTileKey& Tile : Leaving) Updated.Remove(Tile);
for (const FVoxelTileKey& Tile : Entering) Updated.Add(Tile);
TArray<FVoxelTileKey> FullNextTiles;
VoxelClipmapDesiredTiles::Build(
NextCenter, NextMinZ, NextMaxZ, P, FullNextTiles);
TSet<FVoxelTileKey> FullNextSet;
for (const FVoxelTileKey& Tile : FullNextTiles) FullNextSet.Add(Tile);
bool bMatches = Updated.Num() == FullNextSet.Num();
if (bMatches)
{
for (const FVoxelTileKey& Tile : Updated)
{
if (!FullNextSet.Contains(Tile))
{
bMatches = false;
break;
}
}
}
if (!bMatches)
{
++DeltaFailures;
if (FirstDeltaFailure.IsEmpty())
{
FirstDeltaFailure = FString::Printf(
TEXT("center=(%d,%d,%d) step=(%d,%d,%d) radius=%d level=%d clamp=%d entering=%d leaving=%d full=%d updated=%d"),
Center.X, Center.Y, Center.Z, Step.X, Step.Y, Step.Z,
Sweep.Radius, Sweep.MaxLevel, ClampCase,
Entering.Num(), Leaving.Num(), FullNextSet.Num(), Updated.Num());
}
}
++DeltaCases;
}
}
AddInfo(FString::Printf(
TEXT("clipmap delta cases=%d failures=%d first=%s"), DeltaCases, DeltaFailures,
FirstDeltaFailure.IsEmpty() ? TEXT("none") : *FirstDeltaFailure));
TestEqual(TEXT("clipmap delta update equals a full rebuild on every tested crossing"),
DeltaFailures, 0);
return true; return true;
} }
@@ -182,4 +182,218 @@ namespace VoxelClipmapDesiredTiles
PreviousRingMax = RingMax; PreviousRingMax = RingMax;
} }
} }
// Compute the exact key difference between two adjacent clipmap centers without visiting
// unchanged ring cells. A coarser ring can change membership when its finer child ring moves,
// even when the coarser ring bounds themselves stay fixed; the cover-box delta below includes
// those boundary keys as well.
FORCEINLINE void BuildDelta(
const FIntVector& PreviousCenter, int32 PreviousMinChunkZ, int32 PreviousMaxChunkZ,
const FIntVector& Center, int32 MinChunkZ, int32 MaxChunkZ,
const FParameters& Parameters,
TArray<FVoxelTileKey>& OutEntering, TArray<FVoxelTileKey>& OutLeaving)
{
struct FBox
{
FIntVector Min = FIntVector::ZeroValue;
FIntVector Max = FIntVector::ZeroValue;
bool bValid = false;
};
OutEntering.Reset();
OutLeaving.Reset();
const int32 Radius = FMath::Max(1, Parameters.ClipRadius);
const int32 MaxLevel = FMath::Clamp(Parameters.MaxClipLevel, 0, 8);
const int32 OuterRadius = OuterShellRadius(Parameters, Radius, MaxLevel);
FBox PreviousValid[9];
FBox CurrentValid[9];
FBox PreviousCovered[9];
FBox CurrentCovered[9];
const auto BuildRings = [&](const FIntVector& RingCenter, int32 RingMinChunkZ,
int32 RingMaxChunkZ, FBox* OutValid, FBox* OutCovered)
{
FIntVector ChildRingMin = FIntVector::ZeroValue;
FIntVector ChildRingMax = FIntVector::ZeroValue;
for (int32 Level = 0; Level <= MaxLevel; ++Level)
{
const int32 Pow = static_cast<int32>(uint32{1} << static_cast<uint32>(Level));
const FIntVector CenterLevel = FloorDiv(RingCenter, Pow);
const int32 LevelRadius = (Level == MaxLevel) ? OuterRadius : Radius;
const int32 RingAlignment = Level < MaxLevel ? 2 : 1;
FIntVector RingMin, RingMax;
AlignedRingBounds(CenterLevel, LevelRadius, RingAlignment, RingMin, RingMax);
FBox Valid;
Valid.Min = RingMin;
Valid.Max = RingMax;
Valid.Min.Z = FMath::Max(Valid.Min.Z, FloorDiv(RingMinChunkZ, Pow));
Valid.Max.Z = FMath::Min(Valid.Max.Z, FloorDiv(RingMaxChunkZ, Pow));
Valid.bValid = Valid.Min.X <= Valid.Max.X
&& Valid.Min.Y <= Valid.Max.Y && Valid.Min.Z <= Valid.Max.Z;
OutValid[Level] = Valid;
FBox Covered;
if (Level > 0)
{
Covered.Min = FIntVector(
CeilDiv(ChildRingMin.X, 2), CeilDiv(ChildRingMin.Y, 2),
CeilDiv(ChildRingMin.Z, 2));
Covered.Max = FIntVector(
ClampInt32(FloorDivWide(static_cast<int64>(ChildRingMax.X) - 1, 2)),
ClampInt32(FloorDivWide(static_cast<int64>(ChildRingMax.Y) - 1, 2)),
ClampInt32(FloorDivWide(static_cast<int64>(ChildRingMax.Z) - 1, 2)));
Covered.bValid = Covered.Min.X <= Covered.Max.X
&& Covered.Min.Y <= Covered.Max.Y && Covered.Min.Z <= Covered.Max.Z;
}
OutCovered[Level] = Covered;
ChildRingMin = RingMin;
ChildRingMax = RingMax;
}
};
BuildRings(PreviousCenter, PreviousMinChunkZ, PreviousMaxChunkZ,
PreviousValid, PreviousCovered);
BuildRings(Center, MinChunkZ, MaxChunkZ, CurrentValid, CurrentCovered);
const auto Intersect = [](const FBox& A, const FBox& B)
{
FBox Result;
if (!A.bValid || !B.bValid) return Result;
Result.Min = FIntVector(
FMath::Max(A.Min.X, B.Min.X), FMath::Max(A.Min.Y, B.Min.Y),
FMath::Max(A.Min.Z, B.Min.Z));
Result.Max = FIntVector(
FMath::Min(A.Max.X, B.Max.X), FMath::Min(A.Max.Y, B.Max.Y),
FMath::Min(A.Max.Z, B.Max.Z));
Result.bValid = Result.Min.X <= Result.Max.X
&& Result.Min.Y <= Result.Max.Y && Result.Min.Z <= Result.Max.Z;
return Result;
};
const auto AppendDifference = [&Intersect](const FBox& A, const FBox& B, auto& OutBoxes)
{
if (!A.bValid) return;
const FBox Common = Intersect(A, B);
if (!Common.bValid)
{
OutBoxes.Add(A);
return;
}
// Six disjoint slabs cover A minus Common. The later slabs are narrowed to the
// common extent on each already-partitioned axis, so no key is emitted twice.
FBox Slab = A;
if (A.Min.X < Common.Min.X)
{
Slab.Max.X = Common.Min.X - 1;
Slab.bValid = Slab.Min.X <= Slab.Max.X;
if (Slab.bValid) OutBoxes.Add(Slab);
}
Slab = A;
if (Common.Max.X < A.Max.X)
{
Slab.Min.X = Common.Max.X + 1;
Slab.bValid = Slab.Min.X <= Slab.Max.X;
if (Slab.bValid) OutBoxes.Add(Slab);
}
Slab = A;
Slab.Min.X = Common.Min.X;
Slab.Max.X = Common.Max.X;
if (A.Min.Y < Common.Min.Y)
{
Slab.Max.Y = Common.Min.Y - 1;
Slab.bValid = Slab.Min.Y <= Slab.Max.Y;
if (Slab.bValid) OutBoxes.Add(Slab);
}
Slab = A;
Slab.Min.X = Common.Min.X;
Slab.Max.X = Common.Max.X;
if (Common.Max.Y < A.Max.Y)
{
Slab.Min.Y = Common.Max.Y + 1;
Slab.bValid = Slab.Min.Y <= Slab.Max.Y;
if (Slab.bValid) OutBoxes.Add(Slab);
}
Slab = A;
Slab.Min.X = Common.Min.X;
Slab.Max.X = Common.Max.X;
Slab.Min.Y = Common.Min.Y;
Slab.Max.Y = Common.Max.Y;
if (A.Min.Z < Common.Min.Z)
{
Slab.Max.Z = Common.Min.Z - 1;
Slab.bValid = Slab.Min.Z <= Slab.Max.Z;
if (Slab.bValid) OutBoxes.Add(Slab);
}
Slab = A;
Slab.Min.X = Common.Min.X;
Slab.Max.X = Common.Max.X;
Slab.Min.Y = Common.Min.Y;
Slab.Max.Y = Common.Max.Y;
if (Common.Max.Z < A.Max.Z)
{
Slab.Min.Z = Common.Max.Z + 1;
Slab.bValid = Slab.Min.Z <= Slab.Max.Z;
if (Slab.bValid) OutBoxes.Add(Slab);
}
};
const auto Contains = [](const FBox& Box, const FIntVector& Point)
{
return Box.bValid && Point.X >= Box.Min.X && Point.X <= Box.Max.X
&& Point.Y >= Box.Min.Y && Point.Y <= Box.Max.Y
&& Point.Z >= Box.Min.Z && Point.Z <= Box.Max.Z;
};
const auto ForEachBox = [](const FBox& Box, const auto& Callback)
{
if (!Box.bValid) return;
for (int64 Z = Box.Min.Z; Z <= Box.Max.Z; ++Z)
for (int64 Y = Box.Min.Y; Y <= Box.Max.Y; ++Y)
for (int64 X = Box.Min.X; X <= Box.Max.X; ++X)
{
Callback(FIntVector(static_cast<int32>(X), static_cast<int32>(Y),
static_cast<int32>(Z)));
}
};
for (int32 Level = 0; Level <= MaxLevel; ++Level)
{
TArray<FBox, TInlineAllocator<16>> ChangedBoxes;
AppendDifference(CurrentValid[Level], PreviousValid[Level], ChangedBoxes);
AppendDifference(PreviousValid[Level], CurrentValid[Level], ChangedBoxes);
if (Level > 0)
{
const FBox CommonValid = Intersect(CurrentValid[Level], PreviousValid[Level]);
if (CommonValid.bValid)
{
TArray<FBox, TInlineAllocator<16>> CoverDifferences;
AppendDifference(CurrentCovered[Level], PreviousCovered[Level], CoverDifferences);
AppendDifference(PreviousCovered[Level], CurrentCovered[Level], CoverDifferences);
for (const FBox& CoverDifference : CoverDifferences)
{
const FBox ChangedCover = Intersect(CoverDifference, CommonValid);
if (ChangedCover.bValid) ChangedBoxes.Add(ChangedCover);
}
}
}
for (const FBox& ChangedBox : ChangedBoxes)
{
ForEachBox(ChangedBox, [&](const FIntVector& Coord)
{
const bool bCurrentMember = Contains(CurrentValid[Level], Coord)
&& !Contains(CurrentCovered[Level], Coord);
const bool bPreviousMember = Contains(PreviousValid[Level], Coord)
&& !Contains(PreviousCovered[Level], Coord);
if (bCurrentMember && !bPreviousMember)
{
OutEntering.Emplace(Coord, Level);
}
else if (bPreviousMember && !bCurrentMember)
{
OutLeaving.Emplace(Coord, Level);
}
});
}
}
}
} }
@@ -0,0 +1,28 @@
#pragma once
#include <algorithm>
namespace VoxelStreamingAdmissionOrder
{
template <typename Iterator, typename Less>
void StableSortCandidates(Iterator First, Iterator Last, Less Compare)
{
std::stable_sort(First, Last, Compare);
}
template <typename Range, typename IsAtCapacity, typename SubmitCandidate,
typename OnCapacityReached>
void AdmitInOrder(const Range& Candidates, IsAtCapacity AtCapacity,
SubmitCandidate Submit, OnCapacityReached CapacityReached)
{
for (const auto& Candidate : Candidates)
{
if (AtCapacity())
{
CapacityReached();
break;
}
Submit(Candidate);
}
}
}
File diff suppressed because it is too large. Load diff
+18 -1
View File
@@ -1247,6 +1247,8 @@ public:
bool bSampleStacksStarted = false; bool bSampleStacksStarted = false;
bool bStartupApplyBoostStarted = false; bool bStartupApplyBoostStarted = false;
bool bStartupApplyBoostEnded = false; bool bStartupApplyBoostEnded = false;
// Fast-travel admission stays disabled until the initial near LOD and collision set is ready.
bool bInitialArrivalSetSettled = false;
uint64 StartupApplyBoostStartCycles = 0; uint64 StartupApplyBoostStartCycles = 0;
// Trace the resolved startup budget once per world. / Tracer une fois le budget de demarrage resolu par monde. // Trace the resolved startup budget once per world. / Tracer une fois le budget de demarrage resolu par monde.
bool bStartupApplyConfigTraced = false; bool bStartupApplyConfigTraced = false;
@@ -1254,6 +1256,21 @@ public:
float StartupTraceSpeedCmPerSecond = 0.0f; float StartupTraceSpeedCmPerSecond = 0.0f;
bool bSpeedAwareTravelMode = false; bool bSpeedAwareTravelMode = false;
bool bSpeedAwareFastMode = false; bool bSpeedAwareFastMode = false;
TArray<FVoxelTileKey> CachedCoreDesiredTiles;
TSet<FVoxelTileKey> CachedCoreDesiredTileSet;
FIntVector CachedCoreDesiredCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
bool bHasCachedCoreDesiredTiles = false;
TArray<FVoxelTileKey> CachedDesiredCandidateKeys;
TSet<FVoxelTileKey> CachedDesiredCandidateSet;
TSet<FVoxelTileKey> CachedDesiredBaseVisualTiles;
TSet<FVoxelTileKey> CachedDesiredOverlayTiles;
FIntVector CachedDesiredMaterializationCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
bool bHasCachedDesiredMaterialization = false;
TArray<FVoxelTileKey> CachedDesiredPriorityOrder;
FIntVector CachedDesiredPriorityCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
bool bHasCachedDesiredPriorityOrder = false;
int32 IncrementalDesiredCenterCrossings = 0;
int32 IncrementalDesiredMaterializationUpdates = 0;
TArray<FVoxelTileKey> DesiredSorted; // critical floor prefix, then true-pawn distance-first TArray<FVoxelTileKey> DesiredSorted; // critical floor prefix, then true-pawn distance-first
TArray<FVoxelTileKey> CriticalDesiredTiles; // occupied/support tile, then heading tile TArray<FVoxelTileKey> CriticalDesiredTiles; // occupied/support tile, then heading tile
// Ancestor -> desired descendants, rebuilt with DesiredSorted. Reconciliation uses this // Ancestor -> desired descendants, rebuilt with DesiredSorted. Reconciliation uses this
@@ -1602,7 +1619,7 @@ public:
// ONE frame → a game-thread teardown spike. Drained by ProcessUnloadQueue (catch-up scaled). // ONE frame → a game-thread teardown spike. Drained by ProcessUnloadQueue (catch-up scaled).
TSet<FVoxelTileKey> PendingUnload; TSet<FVoxelTileKey> PendingUnload;
void ProcessPendingChunks(); void ProcessPendingChunks(uint64 TickStartCycles);
void ProcessUnloadQueue(); // budgeted teardown drain (see PendingUnload) void ProcessUnloadQueue(); // budgeted teardown drain (see PendingUnload)
void QueuePendingUnload(const FVoxelTileKey& Tile); void QueuePendingUnload(const FVoxelTileKey& Tile);
void HideOverlappingDesiredVisuals(const FVoxelTileKey& RetiredTile); void HideOverlappingDesiredVisuals(const FVoxelTileKey& RetiredTile);
@@ -0,0 +1,303 @@
#include "VoxelForgeLivingPlaneActor.h"
#include "Engine/TextureRenderTarget2D.h"
#include "Engine/TextureRenderTarget.h"
#include "Engine/StaticMesh.h"
#include "Engine/World.h"
#include "Materials/MaterialInstanceDynamic.h"
#include "Materials/MaterialInterface.h"
#include "RenderGraphBuilder.h"
#include "RenderGraphUtils.h"
#include "RHICommandList.h"
#include "RHIResources.h"
#include "DataDrivenShaderPlatformInfo.h"
#include "GlobalShader.h"
#include "ShaderParameterStruct.h"
#include "UObject/ConstructorHelpers.h"
#include "Components/StaticMeshComponent.h"
namespace
{
constexpr int32 LivingPlaneThreadGroupSize = 8;
struct FLivingPlaneStepInput
{
FVector4f ObjectPositionRadius[2] = {FVector4f(0, 0, 0, -1), FVector4f(0, 0, 0, -1)};
FVector4f ObjectVelocityPush[2] = {FVector4f(0, 0, 0, 0), FVector4f(0, 0, 0, 0)};
float TimeSeconds = 0.0f;
};
class FVFLivingPlaneStepCS final : public FGlobalShader
{
DECLARE_GLOBAL_SHADER(FVFLivingPlaneStepCS);
SHADER_USE_PARAMETER_STRUCT(FVFLivingPlaneStepCS, FGlobalShader);
BEGIN_SHADER_PARAMETER_STRUCT(FParameters, )
SHADER_PARAMETER(FIntPoint, GridSize)
SHADER_PARAMETER(float, Dt)
SHADER_PARAMETER(float, Dx)
SHADER_PARAMETER(float, PlaneLength)
SHADER_PARAMETER(float, WaveSpeed)
SHADER_PARAMETER(float, Damping)
SHADER_PARAMETER(float, FoundationStiffness)
SHADER_PARAMETER(float, TimeSeconds)
SHADER_PARAMETER(FVector2f, BreathingSettings)
SHADER_PARAMETER(FVector4f, ImprintSettings)
SHADER_PARAMETER_ARRAY(FVector4f, ObjectPositionRadius, [2])
SHADER_PARAMETER_ARRAY(FVector4f, ObjectVelocityPush, [2])
SHADER_PARAMETER_RDG_TEXTURE_SRV(Texture2D<float4>, SourceState)
SHADER_PARAMETER_RDG_TEXTURE_UAV(RWTexture2D<float4>, TargetState)
END_SHADER_PARAMETER_STRUCT()
public:
static bool ShouldCompilePermutation(const FGlobalShaderPermutationParameters& Parameters)
{
return IsFeatureLevelSupported(Parameters.Platform, ERHIFeatureLevel::SM5);
}
};
IMPLEMENT_GLOBAL_SHADER(FVFLivingPlaneStepCS,
"/Plugin/VoxelForgeLivingPlane/LivingPlane.usf", "MainCS", SF_Compute)
UTextureRenderTarget2D* CreateFieldTarget(UObject* Outer, int32 Size, FName Name)
{
UTextureRenderTarget2D* Target = NewObject<UTextureRenderTarget2D>(Outer, Name, RF_Transient);
Target->RenderTargetFormat = RTF_RGBA32f;
Target->bCanCreateUAV = true;
Target->bAutoGenerateMips = false;
Target->ClearColor = FLinearColor::Transparent;
Target->Filter = TF_Bilinear;
Target->AddressX = TA_Clamp;
Target->AddressY = TA_Clamp;
Target->InitAutoFormat(Size, Size);
Target->UpdateResourceImmediate(true);
return Target;
}
}
AVoxelForgeLivingPlaneActor::AVoxelForgeLivingPlaneActor()
{
PrimaryActorTick.bCanEverTick = true;
PlaneMesh = CreateDefaultSubobject<UStaticMeshComponent>(TEXT("LivingPlaneMesh"));
SetRootComponent(PlaneMesh);
PlaneMesh->SetCollisionEnabled(ECollisionEnabled::NoCollision);
PlaneMesh->SetMobility(EComponentMobility::Movable);
static ConstructorHelpers::FObjectFinder<UStaticMesh> PlaneAsset(
TEXT("/Engine/BasicShapes/Plane.Plane"));
if (PlaneAsset.Succeeded())
{
PlaneMesh->SetStaticMesh(PlaneAsset.Object);
}
ObjectInputs.SetNum(2);
}
void AVoxelForgeLivingPlaneActor::OnConstruction(const FTransform& Transform)
{
Super::OnConstruction(Transform);
if (PlaneMesh)
{
// /Engine/BasicShapes/Plane is 100 cm square; one Unreal scale unit
// therefore corresponds to one metre of authored plane length.
PlaneMesh->SetRelativeScale3D(FVector(PlaneLengthMeters, PlaneLengthMeters, 1.0));
}
}
float AVoxelForgeLivingPlaneActor::GetStableStepSeconds() const
{
const float Dx = PlaneLengthMeters / static_cast<float>(FMath::Max(GridSize, 16));
const float EigenFrequencySquared = 8.0f * WaveSpeedMetersPerSecond * WaveSpeedMetersPerSecond
/ FMath::Max(Dx * Dx, 1.0e-8f) + FoundationStiffnessPerSecondSquared;
return 1.30f / FMath::Sqrt(FMath::Max(EigenFrequencySquared, 1.0e-8f));
}
void AVoxelForgeLivingPlaneActor::BeginPlay()
{
Super::BeginPlay();
GridSize = FMath::Clamp(GridSize, 16, 2048);
PlaneLengthMeters = FMath::Max(PlaneLengthMeters, 1.0f);
FixedStepSeconds = GetStableStepSeconds();
CreateFieldTargets();
if (PlaneMaterial && PlaneMesh)
{
DynamicPlaneMaterial = PlaneMesh->CreateDynamicMaterialInstance(0, PlaneMaterial);
if (DynamicPlaneMaterial)
{
DynamicPlaneMaterial->SetScalarParameterValue(TEXT("PlaneLengthMeters"), PlaneLengthMeters);
DynamicPlaneMaterial->SetScalarParameterValue(TEXT("FieldSize"), static_cast<float>(GridSize));
}
}
UpdateMaterialField();
}
void AVoxelForgeLivingPlaneActor::CreateFieldTargets()
{
FieldTargets[0] = CreateFieldTarget(this, GridSize, TEXT("LivingPlaneStateA"));
FieldTargets[1] = CreateFieldTarget(this, GridSize, TEXT("LivingPlaneStateB"));
CurrentFieldIndex = 0;
SimulationStepCount = 0;
SimulationAccumulatorSeconds = 0.0;
}
void AVoxelForgeLivingPlaneActor::SetObjectState(int32 ObjectIndex,
const FVFLivingPlaneObjectInput& State)
{
if (!ObjectInputs.IsValidIndex(ObjectIndex) || ObjectIndex >= 2)
{
return;
}
ObjectInputs[ObjectIndex] = State;
}
void AVoxelForgeLivingPlaneActor::ResetSimulation()
{
if (!IsInGameThread() || !FieldTargets[0] || !FieldTargets[1])
{
return;
}
CreateFieldTargets();
FixedStepSeconds = GetStableStepSeconds();
UpdateMaterialField();
}
void AVoxelForgeLivingPlaneActor::UpdateMaterialField()
{
if (DynamicPlaneMaterial && FieldTargets[CurrentFieldIndex])
{
DynamicPlaneMaterial->SetTextureParameterValue(TEXT("LivingField"), FieldTargets[CurrentFieldIndex]);
}
}
void AVoxelForgeLivingPlaneActor::EnqueueSimulationSteps(int32 NumSteps)
{
if (NumSteps <= 0 || !FieldTargets[0] || !FieldTargets[1])
{
return;
}
FTextureRenderTargetResource* ResourceA = FieldTargets[0]->GameThread_GetRenderTargetResource();
FTextureRenderTargetResource* ResourceB = FieldTargets[1]->GameThread_GetRenderTargetResource();
if (!ResourceA || !ResourceB)
{
return;
}
TArray<FLivingPlaneStepInput> Steps;
Steps.Reserve(NumSteps);
const FTransform PlaneTransform = GetActorTransform();
for (int32 Step = 0; Step < NumSteps; ++Step)
{
FLivingPlaneStepInput& Input = Steps.AddDefaulted_GetRef();
Input.TimeSeconds = static_cast<float>(SimulationStepCount + Step) * FixedStepSeconds;
for (int32 ObjectIndex = 0; ObjectIndex < 2; ++ObjectIndex)
{
if (!ObjectInputs.IsValidIndex(ObjectIndex) || !ObjectInputs[ObjectIndex].bActive)
{
continue;
}
const FVFLivingPlaneObjectInput& Object = ObjectInputs[ObjectIndex];
const FVector LocalPosition = PlaneTransform.InverseTransformPosition(Object.WorldPositionCm);
const FVector LocalVelocity = PlaneTransform.InverseTransformVector(Object.WorldVelocityCmPerSecond);
const float RadiusMeters = FMath::Max(Object.RadiusCm * 0.01f, 0.001f);
Input.ObjectPositionRadius[ObjectIndex] = FVector4f(
static_cast<float>(LocalPosition.X * 0.01),
static_cast<float>(LocalPosition.Y * 0.01),
static_cast<float>(LocalPosition.Z * 0.01),
RadiusMeters);
Input.ObjectVelocityPush[ObjectIndex] = FVector4f(
static_cast<float>(LocalVelocity.X * 0.01),
static_cast<float>(LocalVelocity.Y * 0.01),
static_cast<float>(LocalVelocity.Z * 0.01),
FMath::Max(Object.PushAccelerationMetersPerSecondSquared, 0.0f));
}
}
const int32 Resolution = GridSize;
const int32 InitialFieldIndex = CurrentFieldIndex;
const float Dt = static_cast<float>(FixedStepSeconds);
const float Dx = PlaneLengthMeters / static_cast<float>(Resolution);
const float PlaneLength = PlaneLengthMeters;
const float WaveSpeed = WaveSpeedMetersPerSecond;
const float Damping = DampingPerSecond;
const float Foundation = FoundationStiffnessPerSecondSquared;
const float BreathingStrength = bEnableBreathing ? BreathingStrengthMetersPerSecondSquared : 0.0f;
const FVector2f BreathingSettings(bEnableBreathing ? 1.0f : 0.0f, BreathingStrength);
const FVector4f ImprintSettings(bEnableImprint ? 1.0f : 0.0f,
ImprintCaptureMetersPerSecond, FMath::Max(ImprintRecoverySeconds, 1.0e-4f),
FMath::Max(ImprintMaxDepthMeters, 0.0f));
ENQUEUE_RENDER_COMMAND(VoxelForgeLivingPlaneStep)(
[ResourceA, ResourceB, Steps = MoveTemp(Steps), InitialFieldIndex, Resolution, Dt, Dx, PlaneLength,
WaveSpeed, Damping, Foundation, BreathingSettings, ImprintSettings]
(FRHICommandListImmediate& RHICmdList) mutable
{
FRDGBuilder GraphBuilder(RHICmdList, RDG_EVENT_NAME("VoxelForgeLivingPlane"));
FRDGTextureRef Fields[2] = {
RegisterExternalTexture(GraphBuilder, ResourceA->GetRenderTargetTexture().GetReference(), TEXT("LivingPlaneStateA")),
RegisterExternalTexture(GraphBuilder, ResourceB->GetRenderTargetTexture().GetReference(), TEXT("LivingPlaneStateB"))};
TShaderMapRef<FVFLivingPlaneStepCS> ComputeShader(GetGlobalShaderMap(GMaxRHIFeatureLevel));
int32 CurrentIndex = InitialFieldIndex;
for (const FLivingPlaneStepInput& StepInput : Steps)
{
const int32 TargetIndex = 1 - CurrentIndex;
FVFLivingPlaneStepCS::FParameters* PassParameters =
GraphBuilder.AllocParameters<FVFLivingPlaneStepCS::FParameters>();
PassParameters->GridSize = FIntPoint(Resolution, Resolution);
PassParameters->Dt = Dt;
PassParameters->Dx = Dx;
PassParameters->PlaneLength = PlaneLength;
PassParameters->WaveSpeed = WaveSpeed;
PassParameters->Damping = Damping;
PassParameters->FoundationStiffness = Foundation;
PassParameters->TimeSeconds = StepInput.TimeSeconds;
PassParameters->BreathingSettings = BreathingSettings;
PassParameters->ImprintSettings = ImprintSettings;
for (int32 ObjectIndex = 0; ObjectIndex < 2; ++ObjectIndex)
{
PassParameters->ObjectPositionRadius[ObjectIndex] = StepInput.ObjectPositionRadius[ObjectIndex];
PassParameters->ObjectVelocityPush[ObjectIndex] = StepInput.ObjectVelocityPush[ObjectIndex];
}
PassParameters->SourceState = GraphBuilder.CreateSRV(Fields[CurrentIndex]);
PassParameters->TargetState = GraphBuilder.CreateUAV(Fields[TargetIndex]);
FComputeShaderUtils::AddPass(GraphBuilder,
RDG_EVENT_NAME("LivingPlaneStep"),
ComputeShader, PassParameters,
FIntVector(FMath::DivideAndRoundUp(Resolution, LivingPlaneThreadGroupSize),
FMath::DivideAndRoundUp(Resolution, LivingPlaneThreadGroupSize), 1));
CurrentIndex = TargetIndex;
}
GraphBuilder.Execute();
});
CurrentFieldIndex = (InitialFieldIndex + NumSteps) & 1;
SimulationStepCount += NumSteps;
UpdateMaterialField();
}
void AVoxelForgeLivingPlaneActor::Tick(float DeltaSeconds)
{
Super::Tick(DeltaSeconds);
if (FixedStepSeconds <= 0.0 || !FieldTargets[0] || !FieldTargets[1])
{
return;
}
SimulationAccumulatorSeconds += FMath::Max(static_cast<double>(DeltaSeconds), 0.0);
const int32 StepsToRun = FMath::Min(
static_cast<int32>(SimulationAccumulatorSeconds / FixedStepSeconds), 32);
if (StepsToRun > 0)
{
SimulationAccumulatorSeconds -= StepsToRun * FixedStepSeconds;
EnqueueSimulationSteps(StepsToRun);
}
}
void AVoxelForgeLivingPlaneActor::EndPlay(const EEndPlayReason::Type EndPlayReason)
{
Super::EndPlay(EndPlayReason);
DynamicPlaneMaterial = nullptr;
FieldTargets[0] = nullptr;
FieldTargets[1] = nullptr;
}
@@ -0,0 +1,25 @@
#include "Modules/ModuleManager.h"
#include "Interfaces/IPluginManager.h"
#include "Misc/Paths.h"
#include "ShaderCore.h"
class FVoxelForgeLivingPlaneModule final : public IModuleInterface
{
public:
virtual void StartupModule() override
{
const TSharedPtr<IPlugin> Plugin = IPluginManager::Get().FindPlugin(TEXT("VoxelForge"));
if (!Plugin.IsValid())
{
UE_LOG(LogTemp, Error, TEXT("VoxelForgeLivingPlane could not find its owning plugin."));
return;
}
AddShaderSourceDirectoryMapping(
TEXT("/Plugin/VoxelForgeLivingPlane"),
FPaths::Combine(Plugin->GetBaseDir(), TEXT("Shaders")));
}
};
IMPLEMENT_MODULE(FVoxelForgeLivingPlaneModule, VoxelForgeLivingPlane)
@@ -0,0 +1,114 @@
#pragma once
#include "CoreMinimal.h"
#include "GameFramework/Actor.h"
#include "VoxelForgeLivingPlaneActor.generated.h"
class UMaterialInterface;
class UMaterialInstanceDynamic;
class UStaticMeshComponent;
class UTextureRenderTarget2D;
USTRUCT(BlueprintType)
struct FVFLivingPlaneObjectInput
{
GENERATED_BODY()
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Object")
FVector WorldPositionCm = FVector(0.0, 0.0, 43.0);
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Object")
FVector WorldVelocityCmPerSecond = FVector::ZeroVector;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Object", meta=(ClampMin="0.1"))
float RadiusCm = 43.0f;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Object", meta=(ClampMin="0.0"))
float PushAccelerationMetersPerSecondSquared = 36.0f;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Object")
bool bActive = false;
};
/** A render-only, GPU-stepped heightfield plane. Object state is uploaded; the field is never read back. */
UCLASS(BlueprintType, Blueprintable)
class VOXELFORGELIVINGPLANE_API AVoxelForgeLivingPlaneActor : public AActor
{
GENERATED_BODY()
public:
AVoxelForgeLivingPlaneActor();
virtual void BeginPlay() override;
virtual void Tick(float DeltaSeconds) override;
virtual void OnConstruction(const FTransform& Transform) override;
virtual void EndPlay(const EEndPlayReason::Type EndPlayReason) override;
UFUNCTION(BlueprintCallable, Category="Living Plane|Object")
void SetObjectState(int32 ObjectIndex, const FVFLivingPlaneObjectInput& State);
UFUNCTION(BlueprintCallable, Category="Living Plane")
void ResetSimulation();
UFUNCTION(BlueprintPure, Category="Living Plane")
int64 GetSimulationStepCount() const { return SimulationStepCount; }
UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category="Living Plane")
TObjectPtr<UStaticMeshComponent> PlaneMesh;
UPROPERTY(EditAnywhere, BlueprintReadOnly, Category="Living Plane|Rendering")
TObjectPtr<UMaterialInterface> PlaneMaterial;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Simulation", meta=(ClampMin="16", ClampMax="2048"))
int32 GridSize = 256;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Simulation", meta=(ClampMin="1.0", ClampMax="100.0"))
float PlaneLengthMeters = 12.0f;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Simulation", meta=(ClampMin="0.0"))
float WaveSpeedMetersPerSecond = 4.5f;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Simulation", meta=(ClampMin="0.0"))
float DampingPerSecond = 1.35f;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Simulation", meta=(ClampMin="0.0"))
float FoundationStiffnessPerSecondSquared = 28.0f;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Simulation")
bool bEnableBreathing = true;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Simulation", meta=(ClampMin="0.0", ClampMax="30.0"))
float BreathingStrengthMetersPerSecondSquared = 4.8f;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Imprint")
bool bEnableImprint = true;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Imprint", meta=(ClampMin="0.0", ClampMax="1.0"))
float ImprintCaptureMetersPerSecond = 0.15f;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Imprint", meta=(ClampMin="0.1", ClampMax="60.0"))
float ImprintRecoverySeconds = 6.0f;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Imprint", meta=(ClampMin="0.0", ClampMax="1.0"))
float ImprintMaxDepthMeters = 0.34f;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Object")
TArray<FVFLivingPlaneObjectInput> ObjectInputs;
private:
void CreateFieldTargets();
void EnqueueSimulationSteps(int32 NumSteps);
void UpdateMaterialField();
float GetStableStepSeconds() const;
UPROPERTY(Transient)
TObjectPtr<UTextureRenderTarget2D> FieldTargets[2];
UPROPERTY(Transient)
TObjectPtr<UMaterialInstanceDynamic> DynamicPlaneMaterial;
int32 CurrentFieldIndex = 0;
int64 SimulationStepCount = 0;
double SimulationAccumulatorSeconds = 0.0;
double FixedStepSeconds = 0.0;
};
@@ -0,0 +1,28 @@
using UnrealBuildTool;
public class VoxelForgeLivingPlane : ModuleRules
{
public VoxelForgeLivingPlane(ReadOnlyTargetRules Target) : base(Target)
{
PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs;
if (Target.Platform == UnrealTargetPlatform.Linux)
{
bUseUnity = false;
}
PublicDependencyModuleNames.AddRange(new string[]
{
"Core",
"CoreUObject",
"Engine"
});
PrivateDependencyModuleNames.AddRange(new string[]
{
"Projects",
"RenderCore",
"RHI"
});
}
}
@@ -0,0 +1,38 @@
# L157 — near terrain at 30 m/s
**Result:** Early admission raised moving near coverage, but the W8 experiment did not meet the full latency and worst-tick bar. Keep both experiment controls disabled by default. The UE host was built from `/home/jahni/VoxelM/Plugins/VoxelForge/Saved/Linux/l157-tree/Source`; the VFHost source symlink has since been restored.
## Measurements
Five interleaved pairs used the same 30 m/s, 12 s movement plus 5 s dwell, 8 effective tile slots, and four-strate owner plan. Each run had 45–46 moving samples. The uncounted candidate smoke run warmed the same UE binary. The machine was quiet before the matrix (uptime load averages 1.09/1.40/1.22).
| Moving metric | `assembler/world` median | Early admission + 1 ms cap median | Paired median delta |
| --- | ---: | ---: | ---: |
| LOD0–2 ready | 36.06% | 48.64% | +10.76 pp |
| 55° forward-cone ready | 12.12% | 16.86% | +5.95 pp |
| Game-thread tick p99 | 12.51 ms | 10.93 ms | +0.07 ms |
| Worst tick per run, median | 14.67 ms | 14.52 ms | +0.18 ms |
| Far holes | 4.00% | 5.82% | −0.13 pp |
| Initial near-set arrival | 2.80 s | 3.21 s | +0.01 s |
The p99 median is under 11 ms, but only three candidate runs were under that limit; the other two were 15.75 and 18.17 ms. The worst observed candidate tick was 19.88 ms versus 17.75 ms for baseline. Far-hole results were mixed: the paired median improved slightly, while two candidate pairs regressed and the worst candidate run had 12.48% holes versus a 7.08% baseline worst. Arrival was effectively tied by paired median, with one candidate pair 0.88 s slower. The combined bar therefore fails on repeatability and worst tick; far holes are not robustly no-worse.
The apply cap lowered median apply p99 from 2.11 to 1.40 ms, but it is not a hard per-frame ceiling: candidate apply p99 reached 6.61 ms and the maximum was 7.58 ms. The loop checks elapsed time after finishing a result. Candidate stream-update p99 reached 15.07 and 13.52 ms in the same two outlier runs. Historical L122 frame joins likewise show that applying extra results adds work on candidate tail frames (about 1.0–1.5 ms), while `UpdateChunks`/desired-set work forms the larger tail; `level_admit` and `submit_tiles` p99 were not consistently higher. The cap cannot make the whole tick fit when update work already exceeds the frame budget.
## Direction and next lever
The old metric counts every desired LOD0–2 tile equally, including tiles behind the player. Desired tiles split about 50.44% ahead and 49.56% behind; only 23.20% fell inside the 55° forward cone. Behind readiness was 50.93% baseline / 68.49% candidate, versus 21.71% / 30.60% ahead. The all-around count therefore overstates what is ready in the direction of travel.
For VR, readiness against the HMD/camera frustum is the useful visibility metric. The velocity-forward cone measured here is a movement proxy, not a head-pose measurement. Report it beside the old all-around residency count. No additional speed fix is validated; a narrow forward-cone admission probe is the next cheap experiment, while desired-set rebuild/update work remains the measured latency target.
## Checks and limits
- UBT built `VFHostEditor` successfully, 98/98 actions, using `-NoUBA -MaxParallelActions=4`. It compiled `Source/VoxelForge/Private/VoxelWorld.cpp` from the L157 worktree. Three existing RealtimeMesh deprecation warnings remained.
- Fast tier: 268 PASS, 0 FAIL, 5 older-baseline diagnostics. Current canonical owner hashes match L130; W1/W12 hashes match for both pinned owners. Random-camera tile hashes also match across W1/W8/W12.
- UE `TileHashDump` comparison was inconclusive: both runs logged `records=0`, despite streamed meshes. No direct UE tile-hash claim is made. The L157 changes are limited to admission/apply controls and trace accounting; no generator or mesher source changed.
- W12 movement timing and HMD-pose visibility were not run. W12 was skipped after W8 failed the worst-tick bar; direct UE hash capture emitted no records.
- `RunL157Movement.py` passed `py_compile`; `git diff --check` passed.
Logs and paired data are in `Saved/Linux/l157/`; movement traces and summary are in `Saved/Linux/l157/ue/movement/`.
Changed Unreal source: `Source/VoxelForge/Private/VoxelWorld.cpp`.
@@ -0,0 +1,400 @@
#!/usr/bin/env python3
"""Interleaved L157 UE movement runs for critical-gate and apply-budget ablations."""
from __future__ import annotations
import argparse
import json
import math
import statistics
import subprocess
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
HOST = Path("/home/jahni/VFHost")
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
OUT = ROOT / "Saved/Linux/l157/ue/movement"
HOST_SAVED = HOST / "Saved/Linux/l157"
HOST_PLUGIN_SAVED = HOST / "Plugins/VoxelForge/Saved/Linux/l157"
SPAWN_M = (19.166015625, -14.2001953125, -753.3740234375)
MOVE_START_S = 6.0
CONDITIONS = {
"assembler_world": {"wait_for_critical": 1, "apply_budget_ms": 0.0},
"early_admission_apply_1ms": {"wait_for_critical": 0, "apply_budget_ms": 1.0},
}
ORDER = {
1: ("assembler_world", "early_admission_apply_1ms"),
2: ("early_admission_apply_1ms", "assembler_world"),
3: ("assembler_world", "early_admission_apply_1ms"),
4: ("early_admission_apply_1ms", "assembler_world"),
5: ("assembler_world", "early_admission_apply_1ms"),
}
def percentile(values: list[float], fraction: float = 0.99) -> float | None:
if not values:
return None
ordered = sorted(values)
return ordered[max(0, min(len(ordered) - 1, math.ceil(len(ordered) * fraction) - 1))]
def read_events(path: Path) -> list[dict]:
rows = []
if path.is_file():
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
if not line.startswith("{"):
continue
try:
rows.append(json.loads(line))
except json.JSONDecodeError:
continue
return rows
def mean(values: list[float]) -> float | None:
return statistics.mean(values) if values else None
def percent(numerator: int | float, denominator: int | float) -> float | None:
return 100.0 * numerator / denominator if denominator > 0 else None
def run(condition: str, repetition: int, tile_hash_dump: Path | None = None) -> dict:
config = CONDITIONS[condition]
run_name = f"fast-r{repetition}"
out = OUT / condition
host_out = HOST_PLUGIN_SAVED / condition / "output"
userdir = HOST_SAVED / "user" / condition / run_name
zendata = HOST_SAVED / "zen" / condition / run_name
ddc = HOST_SAVED / "DDC"
trace = out / f"{run_name}.trace.jsonl"
abslog = out / f"{run_name}.log"
stdout = out / f"{run_name}.stdout.log"
export = host_out / run_name
for directory in (out, export, userdir, zendata, ddc):
directory.mkdir(parents=True, exist_ok=True)
trace.unlink(missing_ok=True)
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWN_M)
command = [
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
"-notraceserver", "-nocrashreports", "-DisablePlugins=AndroidFileServer",
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
"-archetype=TunnelNetwork", "-slot=0", "-modes=export", "-gameconfig=1",
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0",
"-coreplantest=0", "-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings",
"-corehybridmesher=0", f"-out={export}", "-voxel.StreamingCoarseFirst=1",
"-voxel.StreamingDynamicNearReserve=1",
"-voxel.StreamingLegacyPendingResultCap=0",
"-voxel.CoreRegionConcurrentAdmission=1",
"-voxel.StreamingResultBacklogCap=256", "-voxel.StreamingReadyQueueDepth=8",
"-voxel.StreamingBackgroundWorkerCap=1", "-voxel.CoreColumnCacheDirectMapped=0",
"-voxel.StreamingFarApplySupplement=1",
f"-voxel.StreamingFastTravelWaitForAllCriticalTiles={config['wait_for_critical']}",
f"-voxel.StreamingFastTravelApplyBudgetMilliseconds={config['apply_budget_ms']:.1f}",
"-voxel.CoreMCGradientMemoization=1", "-voxel.StreamingCollisionLookahead=1",
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=3000.0",
f"-voxel.TestMoveStartSeconds={MOVE_START_S:.1f}",
"-voxel.TestMoveDurationSeconds=12.0", "-voxel.TestExitSeconds=23.0",
"-voxel.TestMoveDirection=1,0,0", "-voxel.StreamingSchedulerPolicy=3",
"-voxel.TestExitOnSteady=0", "-coreworldstreamtrace=1",
f"-voxel.StartupTraceSpawnCm={spawn_cm}",
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
f"-abslog={abslog}",
]
if tile_hash_dump is not None:
tile_hash_dump.parent.mkdir(parents=True, exist_ok=True)
tile_hash_dump.unlink(missing_ok=True)
command.append(f"-voxel.TileHashDump={tile_hash_dump.resolve()}")
wrapped = ["taskset", "-c", "0-11", "timeout", "1740s", *command]
started = time.monotonic()
with stdout.open("w", encoding="utf-8") as stream:
process = subprocess.run(
wrapped, cwd=ROOT, stdout=stream, stderr=subprocess.STDOUT,
check=False, timeout=1740)
events = read_events(trace)
samples = [event for event in events if event.get("type") == "streaming_sample"]
moving = [event for event in samples if float(event.get("speed_cm_s", 0.0)) >= 50.0]
stop_event = next((event for event in events
if event.get("type") == "streaming_test_move_stopped"), None)
backend = next((event for event in events if event.get("type") == "worker_backend_config"), {})
worker_cap = next((event for event in events
if event.get("type") == "streaming_worker_cap_readback"), {})
if stop_event is None:
raise RuntimeError(f"missing move-stop event: {trace}")
stop_s = float(stop_event.get("t_s", 0.0))
move_start_s = min((float(event.get("t_s", 0.0)) for event in moving), default=float("inf"))
dwell = [event for event in samples if float(event.get("t_s", 0.0)) >= stop_s
and float(event.get("speed_cm_s", 0.0)) < 50.0]
arrival = [event for event in samples if float(event.get("t_s", 0.0)) < move_start_s
and float(event.get("speed_cm_s", 0.0)) < 50.0]
if process.returncode or len(moving) < 40 or len(dwell) < 16:
raise RuntimeError(f"incomplete {condition} run: {trace}; exit={process.returncode}; "
f"move={len(moving)} dwell={len(dwell)}")
if (int(backend.get("streaming_fast_travel_wait_for_all_critical_tiles", -1))
!= config["wait_for_critical"]
or abs(float(backend.get("streaming_fast_travel_apply_budget_ms", -1.0))
- config["apply_budget_ms"]) > 0.01
or int(worker_cap.get("streaming_background_worker_cap", -1)) != 1
or int(backend.get("effective_tile_slots", -1)) != 8
or int(backend.get("streaming_far_apply_supplement", -1)) != 1):
raise RuntimeError(f"CVar/worker readback mismatch: {backend}; {worker_cap}")
moving_metrics = []
for event in moving:
desired = event.get("desired_by_lod", [0] * 9)
resident = event.get("resident_by_lod", [0] * 9)
near_desired = sum(desired[:3])
cone_desired = int(event.get("near_forward_cone_desired", 0))
weighted_desired = float(event.get("near_forward_weighted_desired", 0.0))
moving_metrics.append({
"near_ready_pct": percent(sum(resident[:3]), near_desired) or 0.0,
"lod0_ready_pct": percent(resident[0], desired[0]) or 0.0,
"lod1_ready_pct": percent(resident[1], desired[1]) or 0.0,
"lod2_ready_pct": percent(resident[2], desired[2]) or 0.0,
"far_hole_pct": percent(
sum(desired[3:]) - sum(event.get("covered_by_lod", resident)[3:]),
sum(desired[3:])) or 0.0,
"ahead_share_pct": percent(int(event.get("near_ahead_desired", 0)), near_desired),
"behind_share_pct": percent(int(event.get("near_behind_desired", 0)), near_desired),
"side_share_pct": 100.0 - (
percent(int(event.get("near_ahead_desired", 0)), near_desired) or 0.0
) - (
percent(int(event.get("near_behind_desired", 0)), near_desired) or 0.0
),
"ahead_ready_pct": percent(
int(event.get("near_ahead_resident", 0)),
int(event.get("near_ahead_desired", 0))),
"behind_ready_pct": percent(
int(event.get("near_behind_resident", 0)),
int(event.get("near_behind_desired", 0))),
"forward_cone_ready_pct": percent(
int(event.get("near_forward_cone_resident", 0)), cone_desired),
"forward_cone_share_pct": percent(cone_desired, near_desired),
"forward_weighted_ready_pct": percent(
float(event.get("near_forward_weighted_resident", 0.0)), weighted_desired),
})
arrival_near_settled = next((event for event in arrival
if sum(event.get("desired_by_lod", [0] * 9)[:3]) > 0
and sum(event.get("settled_by_lod", [0] * 9)[:3])
>= sum(event.get("desired_by_lod", [0] * 9)[:3])), None)
dwell_complete = next((event for event in dwell
if sum(event.get("resident_by_lod", [0] * 9)[:3])
>= sum(event.get("desired_by_lod", [0] * 9)[:3])
and sum(event.get("desired_by_lod", [0] * 9)[3:])
<= sum(event.get("covered_by_lod", [0] * 9)[3:])), None)
work = [event for event in events if event.get("type") == "game_thread_work_frame"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
applies = {event.get("frame"): event for event in events
if event.get("type") == "apply_frame"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s}
scheduler = {event.get("frame"): event for event in events
if event.get("type") == "scheduler_admission"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s}
frames = []
for event in work:
apply = applies.get(event.get("frame"), {})
admit = scheduler.get(event.get("frame"), {})
frames.append({
"frame": event.get("frame"),
"tick_ms": float(event.get("tick_ms", 0.0)),
"update_chunks_ms": float(event.get("update_chunks_ms", 0.0)),
"pending_drain_ms": float(event.get("pending_drain_ms", 0.0)),
"apply_ms": float(apply.get("apply_ms", 0.0)),
"apply_budget_ms": float(apply.get("apply_budget_ms", 0.0)),
"meshes_applied": int(apply.get("meshes_applied", 0)),
"time_budget_hit": int(apply.get("time_limit_reached", 0)),
"desired_rebuild_ms": float(admit.get("desired_rebuild_ms", 0.0)),
"submit_tiles_ms": float(admit.get("submit_tiles_ms", 0.0)),
"level_admit_ms": float(admit.get("level_admit_ms", 0.0)),
})
p99_field = lambda key: percentile([float(row[key]) for row in frames])
max_field = lambda key: max((float(row[key]) for row in frames), default=None)
dwell_complete_s = (float(dwell_complete.get("t_s", 0.0)) - stop_s
if dwell_complete else None)
row = {
"condition": condition,
"repetition": repetition,
"speed_mps": 30.0,
"wait_for_all_critical_tiles": config["wait_for_critical"],
"fast_travel_apply_budget_ms": config["apply_budget_ms"],
"effective_tile_slots": backend.get("effective_tile_slots"),
"exit_code": process.returncode,
"wall_s": round(time.monotonic() - started, 3),
"moving_samples": len(moving),
"dwell_samples": len(dwell),
"arrival_samples": len(arrival),
"near_ready_moving_mean_pct": mean([m["near_ready_pct"] for m in moving_metrics]),
"lod0_ready_moving_mean_pct": mean([m["lod0_ready_pct"] for m in moving_metrics]),
"lod1_ready_moving_mean_pct": mean([m["lod1_ready_pct"] for m in moving_metrics]),
"lod2_ready_moving_mean_pct": mean([m["lod2_ready_pct"] for m in moving_metrics]),
"far_holes_moving_mean_pct": mean([m["far_hole_pct"] for m in moving_metrics]),
"near_ahead_desired_share_pct": mean([m["ahead_share_pct"] for m in moving_metrics
if m["ahead_share_pct"] is not None]),
"near_behind_desired_share_pct": mean([m["behind_share_pct"] for m in moving_metrics
if m["behind_share_pct"] is not None]),
"near_side_desired_share_pct": mean([m["side_share_pct"] for m in moving_metrics]),
"near_ahead_ready_pct": mean([m["ahead_ready_pct"] for m in moving_metrics
if m["ahead_ready_pct"] is not None]),
"near_behind_ready_pct": mean([m["behind_ready_pct"] for m in moving_metrics
if m["behind_ready_pct"] is not None]),
"near_forward_cone_half_angle_deg": 55,
"near_forward_cone_desired_share_pct": mean([m["forward_cone_share_pct"]
for m in moving_metrics
if m["forward_cone_share_pct"] is not None]),
"near_forward_cone_ready_pct": mean([m["forward_cone_ready_pct"]
for m in moving_metrics
if m["forward_cone_ready_pct"] is not None]),
"near_forward_cosine_weighted_ready_pct": mean(
[m["forward_weighted_ready_pct"] for m in moving_metrics
if m["forward_weighted_ready_pct"] is not None]),
"arrival_lod02_settled_t_s": (
float(arrival_near_settled.get("t_s", 0.0)) if arrival_near_settled else None),
"near_ready_dwell_end_pct": (100.0 * sum(dwell[-1].get("resident_by_lod", [0] * 9)[:3])
/ max(1, sum(dwell[-1].get("desired_by_lod", [0] * 9)[:3]))),
"far_holes_dwell_end_pct": (100.0 * (
sum(dwell[-1].get("desired_by_lod", [0] * 9)[3:])
- sum(dwell[-1].get("covered_by_lod", [0] * 9)[3:]))
/ max(1, sum(dwell[-1].get("desired_by_lod", [0] * 9)[3:]))),
"dwell_complete_after_stop_s": dwell_complete_s,
"game_thread_tick_p99_ms": p99_field("tick_ms"),
"game_thread_tick_max_ms": max_field("tick_ms"),
"stream_update_p99_ms": p99_field("update_chunks_ms"),
"pending_drain_p99_ms": p99_field("pending_drain_ms"),
"apply_p99_ms": p99_field("apply_ms"),
"apply_max_ms": max_field("apply_ms"),
"apply_budget_min_ms": min((f["apply_budget_ms"] for f in frames), default=None),
"apply_budget_max_ms": max((f["apply_budget_ms"] for f in frames), default=None),
"apply_time_budget_hit_frames": sum(f["time_budget_hit"] for f in frames),
"apply_frames": sum(f["meshes_applied"] > 0 for f in frames),
"apply_meshes_total": sum(f["meshes_applied"] for f in frames),
"desired_rebuild_p99_ms": p99_field("desired_rebuild_ms"),
"submit_tiles_p99_ms": p99_field("submit_tiles_ms"),
"level_admit_p99_ms": p99_field("level_admit_ms"),
"top_tick_frames": sorted(frames, key=lambda f: f["tick_ms"], reverse=True)[:5],
"trace": str(trace.relative_to(ROOT)),
"log": str(abslog.relative_to(ROOT)),
}
print("[L157Movement] " + json.dumps(row, sort_keys=True), flush=True)
return row
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--smoke", action="store_true",
help="run one candidate trace to validate CVar readback and event fields")
parser.add_argument("--geometry-hash-check", action="store_true",
help="compare UE per-tile geometry hashes for baseline and early admission")
args = parser.parse_args()
missing = [path for path in (EDITOR, HOST / "VFHost.uproject") if not path.exists()]
if missing:
raise SystemExit(f"missing UE runner input: {missing}")
OUT.mkdir(parents=True, exist_ok=True)
if args.geometry_hash_check:
hash_dir = OUT / "geometry-hash"
baseline_path = hash_dir / "assembler-world.json"
candidate_path = hash_dir / "early-admission-apply-1ms.json"
run("assembler_world", 99, baseline_path)
run("early_admission_apply_1ms", 99, candidate_path)
baseline_data = json.loads(baseline_path.read_text(encoding="utf-8"))
candidate_data = json.loads(candidate_path.read_text(encoding="utf-8"))
key_fields = ("level", "tile_x", "tile_y", "tile_z")
def indexed(data: dict) -> dict[tuple[int, ...], tuple]:
result = {}
for record in data.get("records", []):
key = tuple(int(record[field]) for field in key_fields)
result[key] = (str(record.get("geometry_hash", "")),
int(record.get("triangle_count", -1)),
bool(record.get("empty", False)))
return result
baseline_hashes = indexed(baseline_data)
candidate_hashes = indexed(candidate_data)
common = baseline_hashes.keys() & candidate_hashes.keys()
mismatches = [key for key in common
if baseline_hashes[key] != candidate_hashes[key]]
result = {
"baseline_records": len(baseline_hashes),
"candidate_records": len(candidate_hashes),
"common_records": len(common),
"hash_mismatches": len(mismatches),
"baseline_only_records": len(baseline_hashes.keys() - candidate_hashes.keys()),
"candidate_only_records": len(candidate_hashes.keys() - baseline_hashes.keys()),
"baseline_dump": str(baseline_path.relative_to(ROOT)),
"candidate_dump": str(candidate_path.relative_to(ROOT)),
}
(hash_dir / "comparison.json").write_text(
json.dumps(result, indent=2, sort_keys=True) + "\n", encoding="utf-8")
print("[L157GeometryHash] " + json.dumps(result, sort_keys=True), flush=True)
return 0 if len(common) >= 100 and not mismatches else 1
if args.smoke:
run("early_admission_apply_1ms", 0)
return 0
rows = []
partial = OUT / "runs-partial.json"
for repetition in range(1, 6):
for condition in ORDER[repetition]:
row = run(condition, repetition)
rows.append(row)
partial.write_text(json.dumps(rows, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
metrics = (
"near_ready_moving_mean_pct", "far_holes_moving_mean_pct",
"game_thread_tick_p99_ms", "game_thread_tick_max_ms",
"arrival_lod02_settled_t_s", "dwell_complete_after_stop_s",
"near_ahead_desired_share_pct", "near_behind_desired_share_pct",
"near_side_desired_share_pct", "near_forward_cone_desired_share_pct",
"near_ahead_ready_pct", "near_behind_ready_pct",
"near_forward_cone_ready_pct", "near_forward_cosine_weighted_ready_pct",
"apply_p99_ms", "desired_rebuild_p99_ms", "level_admit_p99_ms",
)
medians = {
condition: {metric: statistics.median(
row[metric] for row in rows if row["condition"] == condition)
for metric in metrics}
for condition in CONDITIONS
}
pairs = []
for repetition in range(1, 6):
by_condition = {row["condition"]: row for row in rows
if row["repetition"] == repetition}
for candidate in ("early_admission_apply_1ms",):
pairs.append({
"repetition": repetition,
"candidate": candidate,
**{f"{metric}_delta": by_condition[candidate][metric]
- by_condition["assembler_world"][metric]
for metric in metrics if by_condition[candidate][metric] is not None
and by_condition["assembler_world"][metric] is not None},
})
summary = {
"source_commit": subprocess.run(
["git", "-c", f"safe.directory={ROOT}", "rev-parse", "HEAD"],
cwd=ROOT, check=True, capture_output=True, text=True).stdout.strip(),
"protocol": "Five interleaved 30 m/s UE runs per condition; 12 s move plus 5 s dwell; 8 effective tile slots.",
"conditions": CONDITIONS,
"movement_medians": medians,
"paired_deltas_vs_assembler_world": pairs,
"runs": rows,
}
(OUT / "summary.json").write_text(json.dumps(summary, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
print("[L157Summary] " + json.dumps({"movement_medians": medians,
"runs": len(rows)}, sort_keys=True), flush=True)
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,45 @@
# L160 — ahead-first admission and desired rebuild cost
**Result:** Forward-first admission plus same-center rebuild suppression and Core-key caching raised median 55° travel-cone readiness by 3.75 percentage points. All five candidate game-thread p99s were at or below 11 ms, but the full bar failed: worst ticks regressed in 3/5 pairs, far holes in 3/5, and arrival in 2/5. Median all-around LOD0–2 readiness also fell. Keep the experiment controls off by default.
## Measurements
Five interleaved W8 UE pairs ran at 30 m/s for 12 s plus a 5 s dwell, with eight effective tile slots. The pre-run load averages were 0.15 / 1.12 / 1.49. Each run recorded 45 center crossings. The forward-cone value uses travel direction: null-RHI supplied no camera-forward samples. These are desired-residency counts, not measured HMD visibility.
| Moving readiness | `assembler/world` median | Candidate median | Paired median change |
| --- | ---: | ---: | ---: |
| All-around LOD0–2 | 39.46% | 36.59% | −3.07 pp |
| 55° forward cone | 14.19% | 18.24% | +3.75 pp |
The cone covered about 23% of desired LOD0–2 tiles in both conditions. The cone improvement is clear, while the all-around average fell; behind readiness was 55.52% baseline versus 50.03% candidate.
| Gate metric | `assembler/world` median | Candidate median | Candidate result |
| --- | ---: | ---: | --- |
| Game-thread tick p99 | 10.571 ms | 10.509 ms | All five candidate runs: 10.106–10.560 ms; pass ≤11 ms |
| Worst tick per run | 13.615 ms | 14.057 ms | No-worse failed in pairs 1, 2, 4; observed maxima 15.092 / 15.141 ms |
| Far holes while moving | 3.238% | 3.260% | No-worse failed in pairs 2, 4, 5 |
| First settled LOD0–2 arrival | 2.458 s | 2.460 s | No-worse failed in pairs 1 and 3 |
The median `UpdateChunks` p99 changed from 9.30 to 9.18 ms. This small difference did not remove the worst-tick regressions.
### Admission order
The baseline’s first twelve submitted near tiles were four travel-ahead critical tiles (direction class 1), then three forward-cone critical tiles (class 0), then five forward-cone regular tiles. Candidate order put the three cone critical tiles first, followed by the four travel-ahead critical tiles, then the same cone regular lane. Within cone and ahead classes, entries were ordered by distance with coordinate tie-breaks; side and behind entries retained stable order. Here class 0 is the 55° camera cone when available, or the travel cone in these null-RHI runs; class 1 is travel-ahead outside the cone.
### Desired-set rebuilds
The set was not rebuilt only at center crossings. With exactly 45 crossings per run, baseline rebuilt on 313–334 frames (median 324, or 7.20 rebuilds per crossing). The candidate rebuilt on 193–211 frames (median 200, or 4.44 per crossing). Baseline recorded 171–198 same-center Core-region requests per run; candidate recorded 148–166.
The nonzero `BuildDesiredTiles` p95 was 7.08 ms baseline and 7.52 ms candidate (median across runs); its per-rebuild cost did not fall. The median Core `desiredTiles` query p95 fell from 1.96 to 1.25 ms, with a 77.5% median same-center cache-hit share. Median rebuild p50 was 3.91 / 3.98 ms. The candidate suppresses a forced rebuild after Core region compilation completes and reuses the Core base key list at an unchanged center. `BuildDesiredTiles` still rebuilds the UE desired and overlay sets on each trigger; this is neither an incremental full-set delta nor a time-sliced rebuild, so the remaining rebuild tail persists.
## Checks and limits
- Built `VFHostEditor` from the worktree with `taskset -c 0-11`, `-NoUBA`, and `-MaxParallelActions=4`; final build succeeded (3/3 actions). The VFHost `PluginDeps/VoxelForge/Source` symlink was restored to `/home/jahni/VoxelM/Plugins/VoxelForge/Source` afterward.
- Fast tier passed: 268 PASS, 0 FAIL, 5 existing diagnostics. Canonical owner hashes match the L130 baseline; W1/W12 owner hashes and randomized W1/W8/W12 worker comparisons match. No generator or mesher source changed.
- `RunL160Movement.py` passed `py_compile`; `git diff --check` passed. Final `git diff --check` was also run after writing this report.
- HMD/camera-pose readiness and W12 movement timing were not measured. The directional proxy falls back to travel direction because the null-RHI run has no local camera sample.
- The candidate combines admission ordering, region-completion rebuild suppression, and same-center Core-key caching, so the five-pair matrix does not isolate each effect. Geometry remained unchanged; no `WorldAssemblyEvaluatorVersion` bump was needed.
Logs and paired traces are in `Saved/Linux/l160/`; the matrix summary is `Saved/Linux/l160/ue/movement/summary.json`.
Changed Unreal source: `Source/VoxelForge/Private/VoxelWorld.cpp`, `Source/VoxelForge/Public/VoxelWorld.h`.
@@ -0,0 +1,521 @@
#!/usr/bin/env python3
"""Interleaved L160 UE movement runs for forward admission and Core rebuild ablations."""
from __future__ import annotations
import argparse
import json
import math
import statistics
import subprocess
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
HOST = Path("/home/jahni/VFHost")
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
OUT = ROOT / "Saved/Linux/l160/ue/movement"
HOST_SAVED = HOST / "Saved/Linux/l160"
HOST_PLUGIN_SAVED = HOST / "Plugins/VoxelForge/Saved/Linux/l160"
SPAWN_M = (19.166015625, -14.2001953125, -753.3740234375)
MOVE_START_S = 6.0
CONDITIONS = {
"assembler_world": {
"wait_for_critical": 1, "apply_budget_ms": 0.0,
"forward_first": 0, "rebuild_on_region_completion": 1,
"core_desired_cache": 0,
},
"forward_first": {
"wait_for_critical": 1, "apply_budget_ms": 0.0,
"forward_first": 1, "rebuild_on_region_completion": 0,
"core_desired_cache": 1,
},
}
ORDER = {
1: ("assembler_world", "forward_first"),
2: ("forward_first", "assembler_world"),
3: ("assembler_world", "forward_first"),
4: ("forward_first", "assembler_world"),
5: ("assembler_world", "forward_first"),
}
def percentile(values: list[float], fraction: float = 0.99) -> float | None:
if not values:
return None
ordered = sorted(values)
return ordered[max(0, min(len(ordered) - 1, math.ceil(len(ordered) * fraction) - 1))]
def read_events(path: Path) -> list[dict]:
rows = []
if path.is_file():
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
if not line.startswith("{"):
continue
try:
rows.append(json.loads(line))
except json.JSONDecodeError:
continue
return rows
def mean(values: list[float]) -> float | None:
return statistics.mean(values) if values else None
def percent(numerator: int | float, denominator: int | float) -> float | None:
return 100.0 * numerator / denominator if denominator > 0 else None
def run(condition: str, repetition: int, tile_hash_dump: Path | None = None) -> dict:
config = CONDITIONS[condition]
run_name = f"fast-r{repetition}"
out = OUT / condition
host_out = HOST_PLUGIN_SAVED / condition / "output"
userdir = HOST_SAVED / "user" / condition / run_name
zendata = HOST_SAVED / "zen" / condition / run_name
ddc = HOST_SAVED / "DDC"
trace = out / f"{run_name}.trace.jsonl"
abslog = out / f"{run_name}.log"
stdout = out / f"{run_name}.stdout.log"
export = host_out / run_name
for directory in (out, export, userdir, zendata, ddc):
directory.mkdir(parents=True, exist_ok=True)
trace.unlink(missing_ok=True)
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWN_M)
command = [
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
"-notraceserver", "-nocrashreports", "-DisablePlugins=AndroidFileServer",
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
"-archetype=TunnelNetwork", "-slot=0", "-modes=export", "-gameconfig=1",
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0",
"-coreplantest=0", "-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings",
"-corehybridmesher=0", f"-out={export}", "-voxel.StreamingCoarseFirst=1",
"-voxel.StreamingDynamicNearReserve=1",
"-voxel.StreamingLegacyPendingResultCap=0",
"-voxel.CoreRegionConcurrentAdmission=1",
"-voxel.StreamingResultBacklogCap=256", "-voxel.StreamingReadyQueueDepth=8",
"-voxel.StreamingBackgroundWorkerCap=1", "-voxel.CoreColumnCacheDirectMapped=0",
"-voxel.StreamingFarApplySupplement=1",
f"-voxel.StreamingFastTravelWaitForAllCriticalTiles={config['wait_for_critical']}",
f"-voxel.StreamingFastTravelApplyBudgetMilliseconds={config['apply_budget_ms']:.1f}",
f"-voxel.StreamingFastTravelForwardFirst={config['forward_first']}",
f"-voxel.StreamingRebuildOnCoreRegionCompletion={config['rebuild_on_region_completion']}",
f"-voxel.StreamingCoreDesiredCacheByCenter={config['core_desired_cache']}",
"-voxel.CoreMCGradientMemoization=1", "-voxel.StreamingCollisionLookahead=1",
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=3000.0",
f"-voxel.TestMoveStartSeconds={MOVE_START_S:.1f}",
"-voxel.TestMoveDurationSeconds=12.0", "-voxel.TestExitSeconds=23.0",
"-voxel.TestMoveDirection=1,0,0", "-voxel.StreamingSchedulerPolicy=3",
"-voxel.TestExitOnSteady=0", "-coreworldstreamtrace=1",
f"-voxel.StartupTraceSpawnCm={spawn_cm}",
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
f"-abslog={abslog}",
]
if tile_hash_dump is not None:
tile_hash_dump.parent.mkdir(parents=True, exist_ok=True)
tile_hash_dump.unlink(missing_ok=True)
command.append(f"-voxel.TileHashDump={tile_hash_dump.resolve()}")
wrapped = ["taskset", "-c", "0-11", "timeout", "1740s", *command]
started = time.monotonic()
with stdout.open("w", encoding="utf-8") as stream:
process = subprocess.run(
wrapped, cwd=ROOT, stdout=stream, stderr=subprocess.STDOUT,
check=False, timeout=1740)
events = read_events(trace)
samples = [event for event in events if event.get("type") == "streaming_sample"]
moving = [event for event in samples if float(event.get("speed_cm_s", 0.0)) >= 50.0]
stop_event = next((event for event in events
if event.get("type") == "streaming_test_move_stopped"), None)
backend = next((event for event in events if event.get("type") == "worker_backend_config"), {})
scheduler_config = next((event for event in events
if event.get("type") == "scheduler_admission"), {})
worker_cap = next((event for event in events
if event.get("type") == "streaming_worker_cap_readback"), {})
if stop_event is None:
raise RuntimeError(f"missing move-stop event: {trace}")
stop_s = float(stop_event.get("t_s", 0.0))
move_start_s = min((float(event.get("t_s", 0.0)) for event in moving), default=float("inf"))
dwell = [event for event in samples if float(event.get("t_s", 0.0)) >= stop_s
and float(event.get("speed_cm_s", 0.0)) < 50.0]
arrival = [event for event in samples if float(event.get("t_s", 0.0)) < move_start_s
and float(event.get("speed_cm_s", 0.0)) < 50.0]
if process.returncode or len(moving) < 40 or len(dwell) < 16:
raise RuntimeError(f"incomplete {condition} run: {trace}; exit={process.returncode}; "
f"move={len(moving)} dwell={len(dwell)}")
if (int(backend.get("streaming_fast_travel_wait_for_all_critical_tiles", -1))
!= config["wait_for_critical"]
or abs(float(backend.get("streaming_fast_travel_apply_budget_ms", -1.0))
- config["apply_budget_ms"]) > 0.01
or int(worker_cap.get("streaming_background_worker_cap", -1)) != 1
or int(backend.get("effective_tile_slots", -1)) != 8
or int(backend.get("streaming_far_apply_supplement", -1)) != 1
or int(scheduler_config.get("forward_first", -1)) != config["forward_first"]
or int(scheduler_config.get("rebuild_on_core_region_completion", -1))
!= config["rebuild_on_region_completion"]
or int(scheduler_config.get("core_desired_cache_by_center", -1))
!= config["core_desired_cache"]):
raise RuntimeError(f"CVar/worker readback mismatch: {backend}; {worker_cap}; {scheduler_config}")
moving_metrics = []
for event in moving:
desired = event.get("desired_by_lod", [0] * 9)
resident = event.get("resident_by_lod", [0] * 9)
near_desired = sum(desired[:3])
cone_desired = int(event.get("near_forward_cone_desired", 0))
view_cone_desired = int(event.get("near_view_forward_cone_desired", 0))
view_valid = bool(event.get("near_view_forward_cone_valid", 0))
weighted_desired = float(event.get("near_forward_weighted_desired", 0.0))
travel_cone_ready = percent(
int(event.get("near_forward_cone_resident", 0)), cone_desired)
view_cone_ready = percent(
int(event.get("near_view_forward_cone_resident", 0)), view_cone_desired)
moving_metrics.append({
"near_ready_pct": percent(sum(resident[:3]), near_desired) or 0.0,
"lod0_ready_pct": percent(resident[0], desired[0]) or 0.0,
"lod1_ready_pct": percent(resident[1], desired[1]) or 0.0,
"lod2_ready_pct": percent(resident[2], desired[2]) or 0.0,
"far_hole_pct": percent(
sum(desired[3:]) - sum(event.get("covered_by_lod", resident)[3:]),
sum(desired[3:])) or 0.0,
"ahead_share_pct": percent(int(event.get("near_ahead_desired", 0)), near_desired),
"behind_share_pct": percent(int(event.get("near_behind_desired", 0)), near_desired),
"side_share_pct": 100.0 - (
percent(int(event.get("near_ahead_desired", 0)), near_desired) or 0.0
) - (
percent(int(event.get("near_behind_desired", 0)), near_desired) or 0.0
),
"ahead_ready_pct": percent(
int(event.get("near_ahead_resident", 0)),
int(event.get("near_ahead_desired", 0))),
"behind_ready_pct": percent(
int(event.get("near_behind_resident", 0)),
int(event.get("near_behind_desired", 0))),
"travel_forward_cone_ready_pct": travel_cone_ready,
"view_forward_cone_ready_pct": view_cone_ready,
"forward_cone_ready_pct": view_cone_ready if view_valid else travel_cone_ready,
"forward_cone_source": "camera" if view_valid else "travel_fallback",
"forward_cone_share_pct": percent(cone_desired, near_desired),
"forward_weighted_ready_pct": percent(
float(event.get("near_forward_weighted_resident", 0.0)), weighted_desired),
})
arrival_near_settled = next((event for event in arrival
if sum(event.get("desired_by_lod", [0] * 9)[:3]) > 0
and sum(event.get("settled_by_lod", [0] * 9)[:3])
>= sum(event.get("desired_by_lod", [0] * 9)[:3])), None)
dwell_complete = next((event for event in dwell
if sum(event.get("resident_by_lod", [0] * 9)[:3])
>= sum(event.get("desired_by_lod", [0] * 9)[:3])
and sum(event.get("desired_by_lod", [0] * 9)[3:])
<= sum(event.get("covered_by_lod", [0] * 9)[3:])), None)
work = [event for event in events if event.get("type") == "game_thread_work_frame"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
applies = {event.get("frame"): event for event in events
if event.get("type") == "apply_frame"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s}
scheduler = {event.get("frame"): event for event in events
if event.get("type") == "scheduler_admission"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s}
frames = []
for event in work:
apply = applies.get(event.get("frame"), {})
admit = scheduler.get(event.get("frame"), {})
frames.append({
"frame": event.get("frame"),
"tick_ms": float(event.get("tick_ms", 0.0)),
"update_chunks_ms": float(event.get("update_chunks_ms", 0.0)),
"pending_drain_ms": float(event.get("pending_drain_ms", 0.0)),
"apply_ms": float(apply.get("apply_ms", 0.0)),
"apply_budget_ms": float(apply.get("apply_budget_ms", 0.0)),
"meshes_applied": int(apply.get("meshes_applied", 0)),
"time_budget_hit": int(apply.get("time_limit_reached", 0)),
"desired_rebuild_ms": float(admit.get("desired_rebuild_ms", 0.0)),
"desired_rebuilt": int(admit.get("desired_rebuilt", 0)),
"submit_tiles_ms": float(admit.get("submit_tiles_ms", 0.0)),
"level_admit_ms": float(admit.get("level_admit_ms", 0.0)),
})
p99_field = lambda key: percentile([float(row[key]) for row in frames])
max_field = lambda key: max((float(row[key]) for row in frames), default=None)
rebuild_frames = [row for row in frames if row["desired_rebuilt"]]
rebuild_costs = [float(row["desired_rebuild_ms"]) for row in rebuild_frames]
moving_scheduler = [event for event in scheduler.values()]
center_crossings = 0
previous_center = None
for event in sorted(moving_scheduler, key=lambda item: int(item.get("frame", 0))):
center = tuple(event.get("center", ()))
if previous_center is not None and center != previous_center:
center_crossings += 1
previous_center = center
core_requests = [event for event in events
if event.get("type") == "core_region_compile_request"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
same_center_core_requests = sum(
event.get("reason") == "desired_set_changed_same_center" for event in core_requests)
core_desired_queries = [event for event in events
if event.get("type") == "core_desired_cache"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
core_desired_query_ms = [float(event.get("query_ms", 0.0))
for event in core_desired_queries if float(event.get("query_ms", 0.0)) > 0.0]
core_desired_cache_hits = sum(int(event.get("hit", 0)) for event in core_desired_queries)
direction_submissions = [event.get("near_submitted_by_direction", [0, 0, 0, 0])
for event in moving_scheduler]
direction_submissions = [counts for counts in direction_submissions
if isinstance(counts, list) and len(counts) == 4]
direction_submission_totals = [
sum(int(counts[index]) for counts in direction_submissions) for index in range(4)
]
first_admission_order = next((event.get("near_admission_order_first", [])
for event in moving_scheduler
if event.get("near_admission_order_first")), [])
camera_samples = sum(bool(event.get("near_view_forward_cone_valid", 0)) for event in moving)
dwell_complete_s = (float(dwell_complete.get("t_s", 0.0)) - stop_s
if dwell_complete else None)
row = {
"condition": condition,
"repetition": repetition,
"speed_mps": 30.0,
"wait_for_all_critical_tiles": config["wait_for_critical"],
"fast_travel_apply_budget_ms": config["apply_budget_ms"],
"forward_first": config["forward_first"],
"rebuild_on_core_region_completion": config["rebuild_on_region_completion"],
"core_desired_cache_by_center": config["core_desired_cache"],
"effective_tile_slots": backend.get("effective_tile_slots"),
"exit_code": process.returncode,
"wall_s": round(time.monotonic() - started, 3),
"moving_samples": len(moving),
"dwell_samples": len(dwell),
"arrival_samples": len(arrival),
"near_ready_moving_mean_pct": mean([m["near_ready_pct"] for m in moving_metrics]),
"lod0_ready_moving_mean_pct": mean([m["lod0_ready_pct"] for m in moving_metrics]),
"lod1_ready_moving_mean_pct": mean([m["lod1_ready_pct"] for m in moving_metrics]),
"lod2_ready_moving_mean_pct": mean([m["lod2_ready_pct"] for m in moving_metrics]),
"far_holes_moving_mean_pct": mean([m["far_hole_pct"] for m in moving_metrics]),
"near_ahead_desired_share_pct": mean([m["ahead_share_pct"] for m in moving_metrics
if m["ahead_share_pct"] is not None]),
"near_behind_desired_share_pct": mean([m["behind_share_pct"] for m in moving_metrics
if m["behind_share_pct"] is not None]),
"near_side_desired_share_pct": mean([m["side_share_pct"] for m in moving_metrics]),
"near_ahead_ready_pct": mean([m["ahead_ready_pct"] for m in moving_metrics
if m["ahead_ready_pct"] is not None]),
"near_behind_ready_pct": mean([m["behind_ready_pct"] for m in moving_metrics
if m["behind_ready_pct"] is not None]),
"near_forward_cone_half_angle_deg": 55,
"near_forward_cone_desired_share_pct": mean([m["forward_cone_share_pct"]
for m in moving_metrics
if m["forward_cone_share_pct"] is not None]),
"near_forward_cone_ready_pct": mean([m["forward_cone_ready_pct"]
for m in moving_metrics
if m["forward_cone_ready_pct"] is not None]),
"near_travel_forward_cone_ready_pct": mean(
[m["travel_forward_cone_ready_pct"] for m in moving_metrics
if m["travel_forward_cone_ready_pct"] is not None]),
"near_view_forward_cone_ready_pct": mean(
[m["view_forward_cone_ready_pct"] for m in moving_metrics
if m["view_forward_cone_ready_pct"] is not None]),
"camera_forward_samples": camera_samples,
"camera_forward_sample_share_pct": percent(camera_samples, len(moving)),
"near_forward_cosine_weighted_ready_pct": mean(
[m["forward_weighted_ready_pct"] for m in moving_metrics
if m["forward_weighted_ready_pct"] is not None]),
"arrival_lod02_settled_t_s": (
float(arrival_near_settled.get("t_s", 0.0)) if arrival_near_settled else None),
"near_ready_dwell_end_pct": (100.0 * sum(dwell[-1].get("resident_by_lod", [0] * 9)[:3])
/ max(1, sum(dwell[-1].get("desired_by_lod", [0] * 9)[:3]))),
"far_holes_dwell_end_pct": (100.0 * (
sum(dwell[-1].get("desired_by_lod", [0] * 9)[3:])
- sum(dwell[-1].get("covered_by_lod", [0] * 9)[3:]))
/ max(1, sum(dwell[-1].get("desired_by_lod", [0] * 9)[3:]))),
"dwell_complete_after_stop_s": dwell_complete_s,
"game_thread_tick_p99_ms": p99_field("tick_ms"),
"game_thread_tick_max_ms": max_field("tick_ms"),
"stream_update_p99_ms": p99_field("update_chunks_ms"),
"pending_drain_p99_ms": p99_field("pending_drain_ms"),
"apply_p99_ms": p99_field("apply_ms"),
"apply_max_ms": max_field("apply_ms"),
"apply_budget_min_ms": min((f["apply_budget_ms"] for f in frames), default=None),
"apply_budget_max_ms": max((f["apply_budget_ms"] for f in frames), default=None),
"apply_time_budget_hit_frames": sum(f["time_budget_hit"] for f in frames),
"apply_frames": sum(f["meshes_applied"] > 0 for f in frames),
"apply_meshes_total": sum(f["meshes_applied"] for f in frames),
"desired_rebuild_p99_ms": p99_field("desired_rebuild_ms"),
"desired_rebuild_count": len(rebuild_frames),
"desired_rebuilds_per_second": len(rebuild_frames) / 12.0,
"center_crossings": center_crossings,
"same_center_core_region_requests": same_center_core_requests,
"core_desired_cache_hit_count": core_desired_cache_hits,
"core_desired_cache_hit_share_pct": percent(
core_desired_cache_hits, len(core_desired_queries)),
"core_desired_query_nonzero_p50_ms": statistics.median(core_desired_query_ms)
if core_desired_query_ms else None,
"core_desired_query_nonzero_p95_ms": percentile(core_desired_query_ms, 0.95),
"core_desired_query_nonzero_max_ms": max(core_desired_query_ms)
if core_desired_query_ms else None,
"desired_rebuild_nonzero_p50_ms": statistics.median(rebuild_costs)
if rebuild_costs else None,
"desired_rebuild_nonzero_p95_ms": percentile(rebuild_costs, 0.95),
"desired_rebuild_nonzero_max_ms": max(rebuild_costs) if rebuild_costs else None,
"near_submitted_by_direction": direction_submission_totals,
"near_admission_first12": first_admission_order,
"submit_tiles_p99_ms": p99_field("submit_tiles_ms"),
"level_admit_p99_ms": p99_field("level_admit_ms"),
"top_tick_frames": sorted(frames, key=lambda f: f["tick_ms"], reverse=True)[:5],
"trace": str(trace.relative_to(ROOT)),
"log": str(abslog.relative_to(ROOT)),
}
print("[L160Movement] " + json.dumps(row, sort_keys=True), flush=True)
return row
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--smoke", action="store_true",
help="run one forward-first candidate trace to validate CVar readback and event fields")
parser.add_argument("--geometry-hash-check", action="store_true",
help="compare UE per-tile geometry hashes for baseline and forward-first admission")
args = parser.parse_args()
missing = [path for path in (EDITOR, HOST / "VFHost.uproject") if not path.exists()]
if missing:
raise SystemExit(f"missing UE runner input: {missing}")
OUT.mkdir(parents=True, exist_ok=True)
if args.geometry_hash_check:
hash_dir = OUT / "geometry-hash"
baseline_path = hash_dir / "assembler-world.json"
candidate_path = hash_dir / "forward-first.json"
run("assembler_world", 99, baseline_path)
run("forward_first", 99, candidate_path)
baseline_data = json.loads(baseline_path.read_text(encoding="utf-8"))
candidate_data = json.loads(candidate_path.read_text(encoding="utf-8"))
key_fields = ("level", "tile_x", "tile_y", "tile_z")
def indexed(data: dict) -> dict[tuple[int, ...], tuple]:
result = {}
for record in data.get("records", []):
key = tuple(int(record[field]) for field in key_fields)
result[key] = (str(record.get("geometry_hash", "")),
int(record.get("triangle_count", -1)),
bool(record.get("empty", False)))
return result
baseline_hashes = indexed(baseline_data)
candidate_hashes = indexed(candidate_data)
common = baseline_hashes.keys() & candidate_hashes.keys()
mismatches = [key for key in common
if baseline_hashes[key] != candidate_hashes[key]]
result = {
"baseline_records": len(baseline_hashes),
"candidate_records": len(candidate_hashes),
"common_records": len(common),
"hash_mismatches": len(mismatches),
"baseline_only_records": len(baseline_hashes.keys() - candidate_hashes.keys()),
"candidate_only_records": len(candidate_hashes.keys() - baseline_hashes.keys()),
"baseline_dump": str(baseline_path.relative_to(ROOT)),
"candidate_dump": str(candidate_path.relative_to(ROOT)),
}
(hash_dir / "comparison.json").write_text(
json.dumps(result, indent=2, sort_keys=True) + "\n", encoding="utf-8")
print("[L160GeometryHash] " + json.dumps(result, sort_keys=True), flush=True)
return 0 if len(common) >= 100 and not mismatches else 1
if args.smoke:
run("forward_first", 0)
return 0
rows = []
partial = OUT / "runs-partial.json"
for repetition in range(1, 6):
for condition in ORDER[repetition]:
row = run(condition, repetition)
rows.append(row)
partial.write_text(json.dumps(rows, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
metrics = (
"near_ready_moving_mean_pct", "far_holes_moving_mean_pct",
"game_thread_tick_p99_ms", "game_thread_tick_max_ms",
"arrival_lod02_settled_t_s", "dwell_complete_after_stop_s",
"near_ahead_desired_share_pct", "near_behind_desired_share_pct",
"near_side_desired_share_pct", "near_forward_cone_desired_share_pct",
"near_ahead_ready_pct", "near_behind_ready_pct",
"near_forward_cone_ready_pct", "near_forward_cosine_weighted_ready_pct",
"near_travel_forward_cone_ready_pct",
"apply_p99_ms", "desired_rebuild_p99_ms", "desired_rebuild_count",
"desired_rebuild_nonzero_p95_ms", "center_crossings",
"same_center_core_region_requests", "level_admit_p99_ms",
"core_desired_cache_hit_count", "core_desired_query_nonzero_p95_ms",
)
medians = {
condition: {metric: statistics.median(
row[metric] for row in rows if row["condition"] == condition)
for metric in metrics}
for condition in CONDITIONS
}
pairs = []
for repetition in range(1, 6):
by_condition = {row["condition"]: row for row in rows
if row["repetition"] == repetition}
candidate = "forward_first"
pairs.append({
"repetition": repetition,
"candidate": candidate,
**{f"{metric}_delta": by_condition[candidate][metric]
- by_condition["assembler_world"][metric]
for metric in metrics if by_condition[candidate][metric] is not None
and by_condition["assembler_world"][metric] is not None},
})
by_pair = {
repetition: {row["condition"]: row for row in rows if row["repetition"] == repetition}
for repetition in range(1, 6)
}
candidate_rows = [row for row in rows if row["condition"] == "forward_first"]
candidate_gates = {
"every_candidate_run_p99_le_11_ms": all(
row["game_thread_tick_p99_ms"] <= 11.0 for row in candidate_rows),
"paired_worst_tick_no_worse": all(
by_pair[rep]["forward_first"]["game_thread_tick_max_ms"]
<= by_pair[rep]["assembler_world"]["game_thread_tick_max_ms"]
for rep in by_pair),
"paired_far_holes_no_worse": all(
by_pair[rep]["forward_first"]["far_holes_moving_mean_pct"]
<= by_pair[rep]["assembler_world"]["far_holes_moving_mean_pct"]
for rep in by_pair),
"paired_arrival_no_worse": all(
by_pair[rep]["forward_first"]["arrival_lod02_settled_t_s"] is not None
and by_pair[rep]["assembler_world"]["arrival_lod02_settled_t_s"] is not None
and by_pair[rep]["forward_first"]["arrival_lod02_settled_t_s"]
<= by_pair[rep]["assembler_world"]["arrival_lod02_settled_t_s"]
for rep in by_pair),
"forward_cone_median_delta_pp": statistics.median(
by_pair[rep]["forward_first"]["near_forward_cone_ready_pct"]
- by_pair[rep]["assembler_world"]["near_forward_cone_ready_pct"]
for rep in by_pair),
}
summary = {
"source_commit": subprocess.run(
["git", "-c", f"safe.directory={ROOT}", "rev-parse", "HEAD"],
cwd=ROOT, check=True, capture_output=True, text=True).stdout.strip(),
"protocol": "Five interleaved 30 m/s UE pairs; 12 s move plus 5 s dwell; 8 effective tile slots. L157 critical wait remains on and apply cap remains 0 ms.",
"conditions": CONDITIONS,
"movement_medians": medians,
"paired_deltas_vs_assembler_world": pairs,
"candidate_gates": candidate_gates,
"runs": rows,
}
(OUT / "summary.json").write_text(json.dumps(summary, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
print("[L160Summary] " + json.dumps({"movement_medians": medians,
"candidate_gates": candidate_gates,
"runs": len(rows)}, sort_keys=True), flush=True)
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,58 @@
# L165 — incremental desired-set rebuild
**Result:** The Core clipmap key update is incremental for ordinary adjacent crossings and cuts median nonzero `BuildDesiredTiles` p95 by 47.7% (7.80 to 4.08 ms). Equality checks pass. Worst tick, p99, near readiness, arrival, and five-pair median far-hole coverage all improved. Far holes regressed by 0.291 percentage points in pair 5, so the strict paired no-regression gate is not clean across all five pairs.
## Count first
A count-only trace of the baseline run covered movement from 6.174 to 17.981 s: 316 desired-set updates, including 44 center crossings. At a crossing, 93.813% of prior desired keys were unchanged in aggregate (median crossing 93.405%; range 93.384–97.318%). Same-center updates had 99.955% median overlap.
| LOD | Prior keys unchanged at crossings |
| --- | ---: |
| 0 | 86.929% |
| 1 | 91.071% |
| 2 | 94.981% |
| 3 | 97.017% |
| 4 | 98.674% |
| 5 | 99.053% |
| 6 | 99.811% |
| 7 | 100.000% |
| 8 | 100.000% |
This trace was used only for key counts. Its commandlet ended before its separate readiness check, so its timings are excluded.
## Change and movement results
`BuildDelta` computes entering and leaving box slabs per LOD, including coarser-ring coverage changes. The runtime caches the Core key array and set, applies the delta at one-tile crossings, and updates the transition-descendant index from entering/leaving keys. It falls back to `World::desiredTiles` for larger jumps or when a changed key touches LOD4 or coarser. The UE overlay sets and priority/admission order are still rebuilt on each trigger; this round does not make those materialization steps incremental.
Five interleaved W8 UE pairs ran at 30 m/s, 45 crossings each, with the L160 `assembler/world` baseline settings and forward-first disabled. The machine was quiet before timing (load averages 0.35 / 1.46 / 1.25). Metrics below are medians of the five runs; rebuild p50/p95 are over nonzero `BuildDesiredTiles` samples in the 12 s movement window.
| Metric | `assembler/world` | Incremental | Paired result |
| --- | ---: | ---: | --- |
| `BuildDesiredTiles` p50 | 3.96 ms | 1.28 ms | lower in all five pairs |
| `BuildDesiredTiles` p95 | 7.80 ms | 4.08 ms | −3.72 ms median; lower in all five pairs |
| Game-thread p99 | 11.46 ms | 7.97 ms | every candidate run was 7.90–8.26 ms |
| Worst tick | 13.73 ms | 12.11 ms | lower in all five pairs by 0.86–3.94 ms |
| Near LOD0–2 readiness | 41.41% | 42.90% | better in all five pairs |
| 55° forward-cone readiness | 14.69% | 15.37% | better in all five pairs |
| Far holes while moving | 3.09% | 2.61% | better in four pairs; pair 5 was +0.291 pp |
| First settled LOD0–2 arrival | 2.453 s | 2.437 s | earlier in all five pairs |
The cone uses the travel-direction fallback because null-RHI produced no camera-forward samples. Far-hole coverage measures loaded desired coverage; the desired key set itself matched in every-crossing control checks.
## Checks and limits
- Built `VFHostEditor` from this worktree with `taskset -c 0-11`, `-NoUBA`, and `-MaxParallelActions=4`; final build succeeded. Restored `VFHost/PluginDeps/VoxelForge/Source` to `/home/jahni/VoxelM/Plugins/VoxelForge/Source`.
- Fast tier passed: 268 PASS, 0 FAIL, 5 diagnostics. Both canonical owner hash sets match the L130 baseline exactly (12/12 rows each); W1/W8/W12 determinism checks passed.
- Clipmap automation passed 630 delta cases across six unit steps and three vertical-clamp settings. Its log reports `Result=Success`, failures=0; UE returned exit status 1 when `TestExit=Automation Test Queue Empty` fired.
- The runtime equality audit passed all 45 crossings with interval 1, and all 20 sampled audits across the five measured candidate runs (four per run). Injecting an impossible reference key failed at crossing 1 with `missing=1` and triggered the assertion/nonzero UE exit.
- `git diff --check` and runner `py_compile` passed. The movement summarizer reports the strict pairwise far-hole gate as failed in pair 5; the five-pair median still improved. The other movement gates passed. HMD/camera-forward readiness and W12 movement were not measured.
- The final transition-index range correction only changes configurations whose desired max LOD is below 8; this W8 scene includes LOD8, so the measured code path is unchanged. The forward-first default remains off.
Changed Unreal `Source/` files:
- `Source/VoxelForge/Private/VoxelClipmapDesiredTiles.h`
- `Source/VoxelForge/Private/VoxelWorld.cpp`
- `Source/VoxelForge/Private/Tests/VoxelForgeClipmapNoOverlapTest.cpp`
- `Source/VoxelForge/Public/VoxelWorld.h`
Logs and summaries are in `Saved/Linux/l165/`.
@@ -0,0 +1,606 @@
#!/usr/bin/env python3
"""Interleaved L165 UE movement runs for incremental desired-set rebuilds."""
from __future__ import annotations
import argparse
import json
import math
import statistics
import subprocess
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
HOST = Path("/home/jahni/VFHost")
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
OUT = ROOT / "Saved/Linux/l165/ue/movement"
HOST_SAVED = HOST / "Saved/Linux/l165"
HOST_PLUGIN_SAVED = HOST / "Plugins/VoxelForge/Saved/Linux/l165"
SPAWN_M = (19.166015625, -14.2001953125, -753.3740234375)
MOVE_START_S = 6.0
CONDITIONS = {
"assembler_world": {
"wait_for_critical": 1, "apply_budget_ms": 0.0,
"forward_first": 0, "rebuild_on_region_completion": 1,
"core_desired_cache": 0, "incremental_desired": 0,
"audit_every": 0, "inject_mismatch": 0,
},
"incremental": {
"wait_for_critical": 1, "apply_budget_ms": 0.0,
"forward_first": 0, "rebuild_on_region_completion": 1,
"core_desired_cache": 0, "incremental_desired": 1,
"audit_every": 10, "inject_mismatch": 0,
},
}
ORDER = {
1: ("assembler_world", "incremental"),
2: ("incremental", "assembler_world"),
3: ("assembler_world", "incremental"),
4: ("incremental", "assembler_world"),
5: ("assembler_world", "incremental"),
}
def percentile(values: list[float], fraction: float = 0.99) -> float | None:
if not values:
return None
ordered = sorted(values)
return ordered[max(0, min(len(ordered) - 1, math.ceil(len(ordered) * fraction) - 1))]
def read_events(path: Path) -> list[dict]:
rows = []
if path.is_file():
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
if not line.startswith("{"):
continue
try:
rows.append(json.loads(line))
except json.JSONDecodeError:
continue
return rows
def mean(values: list[float]) -> float | None:
return statistics.mean(values) if values else None
def percent(numerator: int | float, denominator: int | float) -> float | None:
return 100.0 * numerator / denominator if denominator > 0 else None
def run(condition: str, repetition: int, tile_hash_dump: Path | None = None,
expected_mismatch: bool = False) -> dict:
config = CONDITIONS[condition]
run_name = f"fast-r{repetition}"
out = OUT / condition
host_out = HOST_PLUGIN_SAVED / condition / "output"
userdir = HOST_SAVED / "user" / condition / run_name
zendata = HOST_SAVED / "zen" / condition / run_name
ddc = HOST_SAVED / "DDC"
trace = out / f"{run_name}.trace.jsonl"
abslog = out / f"{run_name}.log"
stdout = out / f"{run_name}.stdout.log"
export = host_out / run_name
for directory in (out, export, userdir, zendata, ddc):
directory.mkdir(parents=True, exist_ok=True)
trace.unlink(missing_ok=True)
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWN_M)
command = [
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
"-notraceserver", "-nocrashreports", "-DisablePlugins=AndroidFileServer",
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
"-archetype=TunnelNetwork", "-slot=0", "-modes=export", "-gameconfig=1",
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0",
"-coreplantest=0", "-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings",
"-corehybridmesher=0", f"-out={export}", "-voxel.StreamingCoarseFirst=1",
"-voxel.StreamingDynamicNearReserve=1",
"-voxel.StreamingLegacyPendingResultCap=0",
"-voxel.CoreRegionConcurrentAdmission=1",
"-voxel.StreamingResultBacklogCap=256", "-voxel.StreamingReadyQueueDepth=8",
"-voxel.StreamingBackgroundWorkerCap=1", "-voxel.CoreColumnCacheDirectMapped=0",
"-voxel.StreamingFarApplySupplement=1",
f"-voxel.StreamingFastTravelWaitForAllCriticalTiles={config['wait_for_critical']}",
f"-voxel.StreamingFastTravelApplyBudgetMilliseconds={config['apply_budget_ms']:.1f}",
f"-voxel.StreamingFastTravelForwardFirst={config['forward_first']}",
f"-voxel.StreamingRebuildOnCoreRegionCompletion={config['rebuild_on_region_completion']}",
f"-voxel.StreamingCoreDesiredCacheByCenter={config['core_desired_cache']}",
f"-voxel.StreamingIncrementalDesiredSet={config['incremental_desired']}",
f"-voxel.StreamingIncrementalDesiredAuditEvery={config['audit_every']}",
f"-voxel.StreamingIncrementalDesiredAuditInjectMismatch={config['inject_mismatch']}",
"-voxel.CoreMCGradientMemoization=1", "-voxel.StreamingCollisionLookahead=1",
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=3000.0",
f"-voxel.TestMoveStartSeconds={MOVE_START_S:.1f}",
"-voxel.TestMoveDurationSeconds=12.0", "-voxel.TestExitSeconds=0.0",
"-voxel.TestMoveDirection=1,0,0", "-voxel.StreamingSchedulerPolicy=3",
"-voxel.TestExitOnSteady=0", "-coreworldstreamtrace=1",
f"-voxel.StartupTraceSpawnCm={spawn_cm}",
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
f"-abslog={abslog}",
]
if tile_hash_dump is not None:
tile_hash_dump.parent.mkdir(parents=True, exist_ok=True)
tile_hash_dump.unlink(missing_ok=True)
command.append(f"-voxel.TileHashDump={tile_hash_dump.resolve()}")
wrapped = ["taskset", "-c", "0-11", "timeout", "1740s", *command]
started = time.monotonic()
with stdout.open("w", encoding="utf-8") as stream:
process = subprocess.run(
wrapped, cwd=ROOT, stdout=stream, stderr=subprocess.STDOUT,
check=False, timeout=1740)
events = read_events(trace)
if expected_mismatch:
diagnostic = "VoxelForgeIncrementalDesiredAudit"
logs = "\n".join(path.read_text(encoding="utf-8", errors="replace")
for path in (abslog, stdout) if path.is_file())
audit_events = [event for event in events
if event.get("type") == "incremental_desired_audit"]
failed_audit = any(int(event.get("pass", 1)) == 0 for event in audit_events)
if process.returncode == 0 or diagnostic not in logs or "mismatch" not in logs.lower():
raise RuntimeError(
f"injected mismatch did not fail as expected: exit={process.returncode}; "
f"audit_events={len(audit_events)}; log={abslog}")
print("[L165MismatchControl] " + json.dumps({
"exit_code": process.returncode,
"diagnostic_seen": True,
"failed_audit_event_seen": failed_audit,
"trace": str(trace.relative_to(ROOT)),
"log": str(abslog.relative_to(ROOT)),
}, sort_keys=True), flush=True)
return {"condition": condition, "exit_code": process.returncode,
"failed_audit_event_seen": failed_audit}
samples = [event for event in events if event.get("type") == "streaming_sample"]
moving = [event for event in samples if float(event.get("speed_cm_s", 0.0)) >= 50.0]
stop_event = next((event for event in events
if event.get("type") == "streaming_test_move_stopped"), None)
backend = next((event for event in events if event.get("type") == "worker_backend_config"), {})
scheduler_config = next((event for event in events
if event.get("type") == "scheduler_admission"), {})
worker_cap = next((event for event in events
if event.get("type") == "streaming_worker_cap_readback"), {})
if stop_event is None:
raise RuntimeError(f"missing move-stop event: {trace}")
stop_s = float(stop_event.get("t_s", 0.0))
move_start_s = min((float(event.get("t_s", 0.0)) for event in moving), default=float("inf"))
dwell = [event for event in samples if float(event.get("t_s", 0.0)) >= stop_s
and float(event.get("speed_cm_s", 0.0)) < 50.0]
arrival = [event for event in samples if float(event.get("t_s", 0.0)) < move_start_s
and float(event.get("speed_cm_s", 0.0)) < 50.0]
if process.returncode or len(moving) < 40 or len(dwell) < 16:
raise RuntimeError(f"incomplete {condition} run: {trace}; exit={process.returncode}; "
f"move={len(moving)} dwell={len(dwell)}")
if (int(backend.get("streaming_fast_travel_wait_for_all_critical_tiles", -1))
!= config["wait_for_critical"]
or abs(float(backend.get("streaming_fast_travel_apply_budget_ms", -1.0))
- config["apply_budget_ms"]) > 0.01
or int(worker_cap.get("streaming_background_worker_cap", -1)) != 1
or int(backend.get("effective_tile_slots", -1)) != 8
or int(backend.get("streaming_far_apply_supplement", -1)) != 1
or int(scheduler_config.get("forward_first", -1)) != config["forward_first"]
or int(scheduler_config.get("rebuild_on_core_region_completion", -1))
!= config["rebuild_on_region_completion"]
or int(scheduler_config.get("core_desired_cache_by_center", -1))
!= config["core_desired_cache"]
or int(scheduler_config.get("incremental_desired_set", -1))
!= config["incremental_desired"]
or int(scheduler_config.get("incremental_desired_audit_every", -1))
!= config["audit_every"]
or int(scheduler_config.get("incremental_desired_audit_inject_mismatch", -1))
!= config["inject_mismatch"]):
raise RuntimeError(f"CVar/worker readback mismatch: {backend}; {worker_cap}; {scheduler_config}")
moving_metrics = []
for event in moving:
desired = event.get("desired_by_lod", [0] * 9)
resident = event.get("resident_by_lod", [0] * 9)
near_desired = sum(desired[:3])
cone_desired = int(event.get("near_forward_cone_desired", 0))
view_cone_desired = int(event.get("near_view_forward_cone_desired", 0))
view_valid = bool(event.get("near_view_forward_cone_valid", 0))
weighted_desired = float(event.get("near_forward_weighted_desired", 0.0))
travel_cone_ready = percent(
int(event.get("near_forward_cone_resident", 0)), cone_desired)
view_cone_ready = percent(
int(event.get("near_view_forward_cone_resident", 0)), view_cone_desired)
moving_metrics.append({
"near_ready_pct": percent(sum(resident[:3]), near_desired) or 0.0,
"lod0_ready_pct": percent(resident[0], desired[0]) or 0.0,
"lod1_ready_pct": percent(resident[1], desired[1]) or 0.0,
"lod2_ready_pct": percent(resident[2], desired[2]) or 0.0,
"far_hole_pct": percent(
sum(desired[3:]) - sum(event.get("covered_by_lod", resident)[3:]),
sum(desired[3:])) or 0.0,
"ahead_share_pct": percent(int(event.get("near_ahead_desired", 0)), near_desired),
"behind_share_pct": percent(int(event.get("near_behind_desired", 0)), near_desired),
"side_share_pct": 100.0 - (
percent(int(event.get("near_ahead_desired", 0)), near_desired) or 0.0
) - (
percent(int(event.get("near_behind_desired", 0)), near_desired) or 0.0
),
"ahead_ready_pct": percent(
int(event.get("near_ahead_resident", 0)),
int(event.get("near_ahead_desired", 0))),
"behind_ready_pct": percent(
int(event.get("near_behind_resident", 0)),
int(event.get("near_behind_desired", 0))),
"travel_forward_cone_ready_pct": travel_cone_ready,
"view_forward_cone_ready_pct": view_cone_ready,
"forward_cone_ready_pct": view_cone_ready if view_valid else travel_cone_ready,
"forward_cone_source": "camera" if view_valid else "travel_fallback",
"forward_cone_share_pct": percent(cone_desired, near_desired),
"forward_weighted_ready_pct": percent(
float(event.get("near_forward_weighted_resident", 0.0)), weighted_desired),
})
arrival_near_settled = next((event for event in arrival
if sum(event.get("desired_by_lod", [0] * 9)[:3]) > 0
and sum(event.get("settled_by_lod", [0] * 9)[:3])
>= sum(event.get("desired_by_lod", [0] * 9)[:3])), None)
dwell_complete = next((event for event in dwell
if sum(event.get("resident_by_lod", [0] * 9)[:3])
>= sum(event.get("desired_by_lod", [0] * 9)[:3])
and sum(event.get("desired_by_lod", [0] * 9)[3:])
<= sum(event.get("covered_by_lod", [0] * 9)[3:])), None)
work = [event for event in events if event.get("type") == "game_thread_work_frame"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
applies = {event.get("frame"): event for event in events
if event.get("type") == "apply_frame"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s}
scheduler = {event.get("frame"): event for event in events
if event.get("type") == "scheduler_admission"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s}
frames = []
for event in work:
apply = applies.get(event.get("frame"), {})
admit = scheduler.get(event.get("frame"), {})
frames.append({
"frame": event.get("frame"),
"tick_ms": float(event.get("tick_ms", 0.0)),
"update_chunks_ms": float(event.get("update_chunks_ms", 0.0)),
"pending_drain_ms": float(event.get("pending_drain_ms", 0.0)),
"apply_ms": float(apply.get("apply_ms", 0.0)),
"apply_budget_ms": float(apply.get("apply_budget_ms", 0.0)),
"meshes_applied": int(apply.get("meshes_applied", 0)),
"time_budget_hit": int(apply.get("time_limit_reached", 0)),
"desired_rebuild_ms": float(admit.get("desired_rebuild_ms", 0.0)),
"desired_rebuilt": int(admit.get("desired_rebuilt", 0)),
"submit_tiles_ms": float(admit.get("submit_tiles_ms", 0.0)),
"level_admit_ms": float(admit.get("level_admit_ms", 0.0)),
})
p99_field = lambda key: percentile([float(row[key]) for row in frames])
max_field = lambda key: max((float(row[key]) for row in frames), default=None)
rebuild_frames = [row for row in frames if row["desired_rebuilt"]]
rebuild_costs = [float(row["desired_rebuild_ms"]) for row in rebuild_frames]
moving_scheduler = [event for event in scheduler.values()]
center_crossings = 0
previous_center = None
for event in sorted(moving_scheduler, key=lambda item: int(item.get("frame", 0))):
center = tuple(event.get("center", ()))
if previous_center is not None and center != previous_center:
center_crossings += 1
previous_center = center
core_requests = [event for event in events
if event.get("type") == "core_region_compile_request"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
same_center_core_requests = sum(
event.get("reason") == "desired_set_changed_same_center" for event in core_requests)
core_desired_queries = [event for event in events
if event.get("type") == "core_desired_cache"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
core_desired_query_ms = [float(event.get("query_ms", 0.0))
for event in core_desired_queries if float(event.get("query_ms", 0.0)) > 0.0]
core_desired_cache_hits = sum(int(event.get("hit", 0)) for event in core_desired_queries)
incremental_audits = [event for event in events
if event.get("type") == "incremental_desired_audit"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
incremental_audit_failures = sum(int(event.get("pass", 1)) == 0
for event in incremental_audits)
if config["incremental_desired"] and (
not incremental_audits or incremental_audit_failures):
raise RuntimeError(
f"incremental desired equality audit missing/failed: {trace}; "
f"audits={len(incremental_audits)} failures={incremental_audit_failures}")
direction_submissions = [event.get("near_submitted_by_direction", [0, 0, 0, 0])
for event in moving_scheduler]
direction_submissions = [counts for counts in direction_submissions
if isinstance(counts, list) and len(counts) == 4]
direction_submission_totals = [
sum(int(counts[index]) for counts in direction_submissions) for index in range(4)
]
first_admission_order = next((event.get("near_admission_order_first", [])
for event in moving_scheduler
if event.get("near_admission_order_first")), [])
camera_samples = sum(bool(event.get("near_view_forward_cone_valid", 0)) for event in moving)
dwell_complete_s = (float(dwell_complete.get("t_s", 0.0)) - stop_s
if dwell_complete else None)
row = {
"condition": condition,
"repetition": repetition,
"speed_mps": 30.0,
"wait_for_all_critical_tiles": config["wait_for_critical"],
"fast_travel_apply_budget_ms": config["apply_budget_ms"],
"forward_first": config["forward_first"],
"rebuild_on_core_region_completion": config["rebuild_on_region_completion"],
"core_desired_cache_by_center": config["core_desired_cache"],
"incremental_desired_set": config["incremental_desired"],
"incremental_desired_audit_every": config["audit_every"],
"incremental_desired_audit_count": len(incremental_audits),
"incremental_desired_audit_failures": incremental_audit_failures,
"effective_tile_slots": backend.get("effective_tile_slots"),
"exit_code": process.returncode,
"wall_s": round(time.monotonic() - started, 3),
"moving_samples": len(moving),
"dwell_samples": len(dwell),
"arrival_samples": len(arrival),
"near_ready_moving_mean_pct": mean([m["near_ready_pct"] for m in moving_metrics]),
"lod0_ready_moving_mean_pct": mean([m["lod0_ready_pct"] for m in moving_metrics]),
"lod1_ready_moving_mean_pct": mean([m["lod1_ready_pct"] for m in moving_metrics]),
"lod2_ready_moving_mean_pct": mean([m["lod2_ready_pct"] for m in moving_metrics]),
"far_holes_moving_mean_pct": mean([m["far_hole_pct"] for m in moving_metrics]),
"near_ahead_desired_share_pct": mean([m["ahead_share_pct"] for m in moving_metrics
if m["ahead_share_pct"] is not None]),
"near_behind_desired_share_pct": mean([m["behind_share_pct"] for m in moving_metrics
if m["behind_share_pct"] is not None]),
"near_side_desired_share_pct": mean([m["side_share_pct"] for m in moving_metrics]),
"near_ahead_ready_pct": mean([m["ahead_ready_pct"] for m in moving_metrics
if m["ahead_ready_pct"] is not None]),
"near_behind_ready_pct": mean([m["behind_ready_pct"] for m in moving_metrics
if m["behind_ready_pct"] is not None]),
"near_forward_cone_half_angle_deg": 55,
"near_forward_cone_desired_share_pct": mean([m["forward_cone_share_pct"]
for m in moving_metrics
if m["forward_cone_share_pct"] is not None]),
"near_forward_cone_ready_pct": mean([m["forward_cone_ready_pct"]
for m in moving_metrics
if m["forward_cone_ready_pct"] is not None]),
"near_travel_forward_cone_ready_pct": mean(
[m["travel_forward_cone_ready_pct"] for m in moving_metrics
if m["travel_forward_cone_ready_pct"] is not None]),
"near_view_forward_cone_ready_pct": mean(
[m["view_forward_cone_ready_pct"] for m in moving_metrics
if m["view_forward_cone_ready_pct"] is not None]),
"camera_forward_samples": camera_samples,
"camera_forward_sample_share_pct": percent(camera_samples, len(moving)),
"near_forward_cosine_weighted_ready_pct": mean(
[m["forward_weighted_ready_pct"] for m in moving_metrics
if m["forward_weighted_ready_pct"] is not None]),
"arrival_lod02_settled_t_s": (
float(arrival_near_settled.get("t_s", 0.0)) if arrival_near_settled else None),
"near_ready_dwell_end_pct": (100.0 * sum(dwell[-1].get("resident_by_lod", [0] * 9)[:3])
/ max(1, sum(dwell[-1].get("desired_by_lod", [0] * 9)[:3]))),
"far_holes_dwell_end_pct": (100.0 * (
sum(dwell[-1].get("desired_by_lod", [0] * 9)[3:])
- sum(dwell[-1].get("covered_by_lod", [0] * 9)[3:]))
/ max(1, sum(dwell[-1].get("desired_by_lod", [0] * 9)[3:]))),
"dwell_complete_after_stop_s": dwell_complete_s,
"game_thread_tick_p99_ms": p99_field("tick_ms"),
"game_thread_tick_max_ms": max_field("tick_ms"),
"stream_update_p99_ms": p99_field("update_chunks_ms"),
"pending_drain_p99_ms": p99_field("pending_drain_ms"),
"apply_p99_ms": p99_field("apply_ms"),
"apply_max_ms": max_field("apply_ms"),
"apply_budget_min_ms": min((f["apply_budget_ms"] for f in frames), default=None),
"apply_budget_max_ms": max((f["apply_budget_ms"] for f in frames), default=None),
"apply_time_budget_hit_frames": sum(f["time_budget_hit"] for f in frames),
"apply_frames": sum(f["meshes_applied"] > 0 for f in frames),
"apply_meshes_total": sum(f["meshes_applied"] for f in frames),
"desired_rebuild_p99_ms": p99_field("desired_rebuild_ms"),
"desired_rebuild_count": len(rebuild_frames),
"desired_rebuilds_per_second": len(rebuild_frames) / 12.0,
"center_crossings": center_crossings,
"same_center_core_region_requests": same_center_core_requests,
"core_desired_cache_hit_count": core_desired_cache_hits,
"core_desired_cache_hit_share_pct": percent(
core_desired_cache_hits, len(core_desired_queries)),
"core_desired_query_nonzero_p50_ms": statistics.median(core_desired_query_ms)
if core_desired_query_ms else None,
"core_desired_query_nonzero_p95_ms": percentile(core_desired_query_ms, 0.95),
"core_desired_query_nonzero_max_ms": max(core_desired_query_ms)
if core_desired_query_ms else None,
"desired_rebuild_nonzero_p50_ms": statistics.median(rebuild_costs)
if rebuild_costs else None,
"desired_rebuild_nonzero_p95_ms": percentile(rebuild_costs, 0.95),
"desired_rebuild_nonzero_max_ms": max(rebuild_costs) if rebuild_costs else None,
"near_submitted_by_direction": direction_submission_totals,
"near_admission_first12": first_admission_order,
"submit_tiles_p99_ms": p99_field("submit_tiles_ms"),
"level_admit_p99_ms": p99_field("level_admit_ms"),
"top_tick_frames": sorted(frames, key=lambda f: f["tick_ms"], reverse=True)[:5],
"trace": str(trace.relative_to(ROOT)),
"log": str(abslog.relative_to(ROOT)),
}
print("[L165Movement] " + json.dumps(row, sort_keys=True), flush=True)
return row
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--smoke", action="store_true",
help="run one incremental trace to validate readback and equality audits")
parser.add_argument("--geometry-hash-check", action="store_true",
help="compare UE per-tile geometry hashes for baseline and incremental desired sets")
parser.add_argument("--controls-only", action="store_true",
help="run a per-crossing equality audit and injected mismatch negative control")
parser.add_argument("--summarize-existing", action="store_true",
help="summarize the ten completed rows in runs-partial.json without rerunning UE")
args = parser.parse_args()
missing = [path for path in (EDITOR, HOST / "VFHost.uproject") if not path.exists()]
if missing:
raise SystemExit(f"missing UE runner input: {missing}")
OUT.mkdir(parents=True, exist_ok=True)
if args.geometry_hash_check:
hash_dir = OUT / "geometry-hash"
baseline_path = hash_dir / "assembler-world.json"
candidate_path = hash_dir / "incremental.json"
run("assembler_world", 99, baseline_path)
run("incremental", 99, candidate_path)
baseline_data = json.loads(baseline_path.read_text(encoding="utf-8"))
candidate_data = json.loads(candidate_path.read_text(encoding="utf-8"))
key_fields = ("level", "tile_x", "tile_y", "tile_z")
def indexed(data: dict) -> dict[tuple[int, ...], tuple]:
result = {}
for record in data.get("records", []):
key = tuple(int(record[field]) for field in key_fields)
result[key] = (str(record.get("geometry_hash", "")),
int(record.get("triangle_count", -1)),
bool(record.get("empty", False)))
return result
baseline_hashes = indexed(baseline_data)
candidate_hashes = indexed(candidate_data)
common = baseline_hashes.keys() & candidate_hashes.keys()
mismatches = [key for key in common
if baseline_hashes[key] != candidate_hashes[key]]
result = {
"baseline_records": len(baseline_hashes),
"candidate_records": len(candidate_hashes),
"common_records": len(common),
"hash_mismatches": len(mismatches),
"baseline_only_records": len(baseline_hashes.keys() - candidate_hashes.keys()),
"candidate_only_records": len(candidate_hashes.keys() - baseline_hashes.keys()),
"baseline_dump": str(baseline_path.relative_to(ROOT)),
"candidate_dump": str(candidate_path.relative_to(ROOT)),
}
(hash_dir / "comparison.json").write_text(
json.dumps(result, indent=2, sort_keys=True) + "\n", encoding="utf-8")
print("[L165GeometryHash] " + json.dumps(result, sort_keys=True), flush=True)
return 0 if len(common) >= 100 and not mismatches else 1
if args.smoke:
run("incremental", 0)
return 0
if args.controls_only:
control = dict(CONDITIONS["incremental"])
control["audit_every"] = 1
control["inject_mismatch"] = 0
CONDITIONS["audit_every_1_control"] = control
run("audit_every_1_control", 0)
mismatch = dict(control)
mismatch["inject_mismatch"] = 1
CONDITIONS["injected_mismatch_control"] = mismatch
run("injected_mismatch_control", 0, expected_mismatch=True)
return 0
partial = OUT / "runs-partial.json"
if args.summarize_existing:
if not partial.is_file():
raise RuntimeError(f"no saved movement rows: {partial}")
rows = json.loads(partial.read_text(encoding="utf-8"))
if len(rows) != 10:
raise RuntimeError(f"expected ten completed movement rows, found {len(rows)}")
else:
rows = []
# Both conditions use this editor binary. Warm each path before measured pairs.
run("assembler_world", 0)
run("incremental", 0)
for repetition in range(1, 6):
for condition in ORDER[repetition]:
row = run(condition, repetition)
rows.append(row)
partial.write_text(json.dumps(rows, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
metrics = (
"near_ready_moving_mean_pct", "far_holes_moving_mean_pct",
"game_thread_tick_p99_ms", "game_thread_tick_max_ms",
"arrival_lod02_settled_t_s", "dwell_complete_after_stop_s",
"near_ahead_desired_share_pct", "near_behind_desired_share_pct",
"near_side_desired_share_pct", "near_forward_cone_desired_share_pct",
"near_ahead_ready_pct", "near_behind_ready_pct",
"near_forward_cone_ready_pct", "near_forward_cosine_weighted_ready_pct",
"near_travel_forward_cone_ready_pct",
"apply_p99_ms", "desired_rebuild_p99_ms", "desired_rebuild_count",
"desired_rebuild_nonzero_p95_ms", "center_crossings",
"same_center_core_region_requests", "level_admit_p99_ms",
"core_desired_cache_hit_count", "core_desired_query_nonzero_p95_ms",
)
medians = {}
for condition in CONDITIONS:
medians[condition] = {}
for metric in metrics:
values = [row[metric] for row in rows
if row["condition"] == condition and row[metric] is not None]
medians[condition][metric] = statistics.median(values) if values else None
pairs = []
for repetition in range(1, 6):
by_condition = {row["condition"]: row for row in rows
if row["repetition"] == repetition}
candidate = "incremental"
pairs.append({
"repetition": repetition,
"candidate": candidate,
**{f"{metric}_delta": by_condition[candidate][metric]
- by_condition["assembler_world"][metric]
for metric in metrics if by_condition[candidate][metric] is not None
and by_condition["assembler_world"][metric] is not None},
})
by_pair = {
repetition: {row["condition"]: row for row in rows if row["repetition"] == repetition}
for repetition in range(1, 6)
}
candidate_rows = [row for row in rows if row["condition"] == "incremental"]
baseline_rows = [row for row in rows if row["condition"] == "assembler_world"]
candidate_gates = {
"every_candidate_run_p99_le_11_ms": all(
row["game_thread_tick_p99_ms"] <= 11.0 for row in candidate_rows),
"paired_worst_tick_no_worse": all(
by_pair[rep]["incremental"]["game_thread_tick_max_ms"]
<= by_pair[rep]["assembler_world"]["game_thread_tick_max_ms"]
for rep in by_pair),
"paired_near_all_around_no_worse": all(
by_pair[rep]["incremental"]["near_ready_moving_mean_pct"]
>= by_pair[rep]["assembler_world"]["near_ready_moving_mean_pct"]
for rep in by_pair),
"paired_forward_cone_no_worse": all(
by_pair[rep]["incremental"]["near_forward_cone_ready_pct"]
>= by_pair[rep]["assembler_world"]["near_forward_cone_ready_pct"]
for rep in by_pair),
"paired_far_holes_no_worse": all(
by_pair[rep]["incremental"]["far_holes_moving_mean_pct"]
<= by_pair[rep]["assembler_world"]["far_holes_moving_mean_pct"]
for rep in by_pair),
"paired_arrival_no_worse": all(
by_pair[rep]["incremental"]["arrival_lod02_settled_t_s"] is not None
and by_pair[rep]["assembler_world"]["arrival_lod02_settled_t_s"] is not None
and by_pair[rep]["incremental"]["arrival_lod02_settled_t_s"]
<= by_pair[rep]["assembler_world"]["arrival_lod02_settled_t_s"]
for rep in by_pair),
"all_candidate_equality_audits_pass": all(
row["incremental_desired_audit_count"] > 0
and row["incremental_desired_audit_failures"] == 0
for row in candidate_rows),
"median_build_desired_p95_down": statistics.median(
row["desired_rebuild_nonzero_p95_ms"] for row in candidate_rows)
< statistics.median(row["desired_rebuild_nonzero_p95_ms"]
for row in baseline_rows),
}
summary = {
"source_commit": subprocess.run(
["git", "-c", f"safe.directory={ROOT}", "rev-parse", "HEAD"],
cwd=ROOT, check=True, capture_output=True, text=True).stdout.strip(),
"protocol": "Five interleaved 30 m/s UE pairs; 12 s move plus 5 s dwell; 8 effective tile slots. L157 critical wait remains on and apply cap remains 0 ms.",
"conditions": CONDITIONS,
"movement_medians": medians,
"paired_deltas_vs_assembler_world": pairs,
"candidate_gates": candidate_gates,
"runs": rows,
}
(OUT / "summary.json").write_text(json.dumps(summary, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
print("[L165Summary] " + json.dumps({"movement_medians": medians,
"candidate_gates": candidate_gates,
"runs": len(rows)}, sort_keys=True), flush=True)
return 0 if all(candidate_gates.values()) else 1
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,58 @@
# L166 — Incremental streaming update
**Result:** Core and overlay candidate materialization now apply entering/leaving deltas, and same-center priority order updates merge only changed keys. Seven 30 m/s W8 pairs show lower median rebuild p95, tick p99, worst tick, near readiness, forward-cone readiness, and far holes. The strict pairwise bar is not fully clean: one worst-tick pair, three far-hole pairs, and four arrival pairs regressed slightly. Set/order audits and the fast tier pass.
## Count first
Stage costs below are milliseconds per desired rebuild, reported as the median of the seven runs' p50 and p95 values. Each run's window is its 12-second move at 30 m/s; there were 44–45 center crossings per run.
| Stage | `assembler/world` p50 / p95 | Incremental p50 / p95 |
| --- | ---: | ---: |
| Core query/preparation | 0.832 / 2.229 | 0.0002 / 0.142 |
| Core crossing delta | — | 0.000 / 0.138 |
| UE overlay materialization | 0.350 / 0.723 | 0.103 / 0.356 |
| Priority sort/order | 0.306 / 0.656 | 0.038 / 0.437 |
| Critical admission order | 0.063 / 0.134 | 0.065 / 0.222 |
The incremental Core path updated on 98.4% of desired rebuilds. Priority order merged key deltas on 52.2%, reused the cached order unchanged on 33.5%, and fully sorted on the remaining 14.3% (the center-crossing updates). LOD4+ crossings caused five full Core-query fallbacks per 12-second move; their median per-run maximum was 1.19 ms and their median total was 5.25 ms spread across those five events.
The median worst tick was 11.652 ms versus 12.980 ms for the baseline. The largest candidate worst-tick sample was 12.764 ms at a center crossing: its priority sort took 1.311 ms, overlay materialization 0.915 ms, and Core delta 0.601 ms; no LOD4+ fallback occurred on that tick. Priority sorting remained the largest measured substage in that worst tick. Fallbacks were not the source of the remaining worst ticks.
## Change and equality checks
`BuildCoreDesiredTiles` now retains the Core visual set, UE overlay set, candidate set, and priority order. It applies Core and overlay membership deltas to the persistent sets. Live anchor, gate, and support inputs are still walked into a temporary overlay union on each trigger. At an unchanged center, it removes leaving keys and merges sorted entering keys into the cached priority order. Center changes still require a full priority sort because the distance comparator changes. The critical-key promotion list is still rebuilt on each trigger.
The equality audit independently obtains a full Core desired set, rebuilds the overlay reference, and compares candidate and collision-only sets. A second audit builds the full prioritized and critical order and compares it with the incremental result. Across the seven measured candidate runs, all 28 sampled set audits and all 28 order audits passed. A positive materialization-control run passed 13 additional set and order audits. The injected mismatch control logged `missing=1` and failed with a nonzero UE exit (`-11`), as expected.
## Seven interleaved pairs
Each pair compared `assembler/world` at `9c70053` against this worktree, with a 12-second 30 m/s move, 5-second dwell, and eight effective tile slots. Values are medians across seven runs; paired gates are stated separately.
| Metric | `assembler/world` | Incremental | Paired result |
| --- | ---: | ---: | --- |
| `BuildDesiredTiles` p95 | 7.281 ms | 3.738 ms | 48.7% lower median |
| Game-thread tick p99 | 10.521 ms | 7.420 ms | Lower in all 7 pairs; candidate range 7.031–7.526 ms |
| Worst tick | 12.980 ms | 11.652 ms | Lower in 6/7 pairs; pair 5 was 0.347 ms higher |
| Near LOD0–2 readiness | 41.995% | 43.668% | Better in all 7 pairs |
| 55° forward-cone readiness | 14.779% | 16.116% | Better in all 7 pairs |
| Far holes while moving | 3.197% | 2.739% | Lower median; better in 4/7 pairs |
| First settled LOD0–2 arrival | 2.432 s | 2.439 s | Paired median delta +0.002 s; later in 4/7 pairs |
The paired far-hole deltas were −0.072, +0.144, −0.059, +0.301, −1.052, −0.773, and +0.069 percentage points. The L165 +0.291 pp result is within this observed spread; L166 also had one +0.301 pp pair, but the seven-pair median delta was −0.059 pp and four pairs improved. This is consistent with substantial run-to-run variation, not a consistent far-hole regression; it does not establish a strict no-regression guarantee.
Null-RHI produced no camera-forward samples, so the 55° cone uses the travel-direction fallback. The arrival measure also varied sharply between pairs, including two candidate runs that settled 0.22–0.26 s earlier. Its paired median was effectively flat, but the strict per-pair arrival gate did not pass.
## Checks and limits
- VFHost Linux Development build succeeded with `-NoUBA -MaxParallelActions=4`; `VFHost/PluginDeps/VoxelForge/Source` was restored to `/home/jahni/VoxelM/Plugins/VoxelForge/Source`.
- `verify.py fast` passed: 268 checks, 0 failures, 5 diagnostics, 281.5 s. Clang/GCC W1/W8/W12 determinism and canonical geometry checks passed.
- The live movement `TileHashDump` hook was enabled but wrote zero records for both conditions, so that direct runtime geometry comparison is inconclusive. No meshing or geometry-generation code changed; the fast-tier geometry hashes and sampled full-rebuild order audit passed.
- `git diff --check` and runner `py_compile` passed. W12 movement timing, camera-forward cone capture, and separate movement runs on each of the four owner layers were not run.
- The strict movement summary fails `paired_worst_tick_no_worse`, `paired_far_holes_no_worse`, and `paired_arrival_no_worse`; p99, near readiness, forward-cone readiness, audits, and median rebuild-p95 gates pass.
Changed Unreal `Source/` files:
- `Source/VoxelForge/Private/VoxelWorld.cpp`
- `Source/VoxelForge/Public/VoxelWorld.h`
Logs, traces, summary, and verifier artifacts are under `Saved/Linux/l166/`.
@@ -0,0 +1,685 @@
#!/usr/bin/env python3
"""Interleaved L166 UE movement runs for incremental desired-set rebuilds."""
from __future__ import annotations
import argparse
import json
import math
import statistics
import subprocess
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
HOST = Path("/home/jahni/VFHost")
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
OUT = ROOT / "Saved/Linux/l166/ue/movement"
HOST_SAVED = HOST / "Saved/Linux/l166"
HOST_PLUGIN_SAVED = HOST / "Plugins/VoxelForge/Saved/Linux/l166"
SPAWN_M = (19.166015625, -14.2001953125, -753.3740234375)
MOVE_START_S = 6.0
CONDITIONS = {
"assembler_world": {
"wait_for_critical": 1, "apply_budget_ms": 0.0,
"forward_first": 0, "rebuild_on_region_completion": 1,
"core_desired_cache": 0, "incremental_desired": 0,
"audit_every": 0, "materialization_audit_every": 0, "inject_mismatch": 0,
},
"incremental": {
"wait_for_critical": 1, "apply_budget_ms": 0.0,
"forward_first": 0, "rebuild_on_region_completion": 1,
"core_desired_cache": 0, "incremental_desired": 1,
"audit_every": 10, "materialization_audit_every": 0, "inject_mismatch": 0,
},
}
ORDER = {
1: ("assembler_world", "incremental"),
2: ("incremental", "assembler_world"),
3: ("assembler_world", "incremental"),
4: ("incremental", "assembler_world"),
5: ("assembler_world", "incremental"),
6: ("incremental", "assembler_world"),
7: ("assembler_world", "incremental"),
}
def percentile(values: list[float], fraction: float = 0.99) -> float | None:
if not values:
return None
ordered = sorted(values)
return ordered[max(0, min(len(ordered) - 1, math.ceil(len(ordered) * fraction) - 1))]
def read_events(path: Path) -> list[dict]:
rows = []
if path.is_file():
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
if not line.startswith("{"):
continue
try:
rows.append(json.loads(line))
except json.JSONDecodeError:
continue
return rows
def mean(values: list[float]) -> float | None:
return statistics.mean(values) if values else None
def desired_breakdown(events: list[dict], move_start_s: float, stop_s: float) -> dict:
rows = [event for event in events
if event.get("type") == "desired_rebuild_breakdown"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
keys = ("total_ms", "core_prepare_ms", "core_delta_ms", "fallback_ms",
"overlay_materialize_ms", "priority_sort_ms", "critical_order_ms")
result = {"rebuild_count": len(rows)}
result["incremental_materialization_update_count"] = sum(
int(event.get("materialization_update", 0)) != 0 for event in rows)
result["priority_order_reuse_count"] = sum(
int(event.get("priority_order_reused", 0)) != 0 for event in rows)
result["priority_order_update_count"] = sum(
int(event.get("priority_order_updated", 0)) != 0 for event in rows)
result["incremental_materialization_update_share_pct"] = percent(
result["incremental_materialization_update_count"], len(rows))
result["priority_order_reuse_share_pct"] = percent(
result["priority_order_reuse_count"], len(rows))
result["priority_order_update_share_pct"] = percent(
result["priority_order_update_count"], len(rows))
for key in keys:
values = [float(event.get(key, 0.0)) for event in rows]
result[f"{key}_p50"] = statistics.median(values) if values else None
result[f"{key}_p95"] = percentile(values, 0.95)
result[f"{key}_max"] = max(values) if values else None
fallback_rows = [event for event in events
if event.get("type") == "core_desired_cache"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s
and int(event.get("full_fallback", 0)) != 0]
result["fallback_count"] = len(fallback_rows)
result["lod4plus_fallback_count"] = sum(
int(event.get("fallback_reason", 0)) == 1 for event in fallback_rows)
result["other_fallback_count"] = sum(
int(event.get("fallback_reason", 0)) == 2 for event in fallback_rows)
result["fallback_cost_sum_ms"] = sum(
float(event.get("fallback_ms", 0.0)) for event in fallback_rows)
result["fallback_cost_max_ms"] = max(
(float(event.get("fallback_ms", 0.0)) for event in fallback_rows), default=None)
return result
def percent(numerator: int | float, denominator: int | float) -> float | None:
return 100.0 * numerator / denominator if denominator > 0 else None
def run(condition: str, repetition: int, tile_hash_dump: Path | None = None,
expected_mismatch: bool = False) -> dict:
config = CONDITIONS[condition]
run_name = f"fast-r{repetition}"
out = OUT / condition
host_out = HOST_PLUGIN_SAVED / condition / "output"
userdir = HOST_SAVED / "user" / condition / run_name
zendata = HOST_SAVED / "zen" / condition / run_name
ddc = HOST_SAVED / "DDC"
trace = out / f"{run_name}.trace.jsonl"
abslog = out / f"{run_name}.log"
stdout = out / f"{run_name}.stdout.log"
export = host_out / run_name
for directory in (out, export, userdir, zendata, ddc):
directory.mkdir(parents=True, exist_ok=True)
trace.unlink(missing_ok=True)
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWN_M)
command = [
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
"-notraceserver", "-nocrashreports", "-DisablePlugins=AndroidFileServer",
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
"-archetype=TunnelNetwork", "-slot=0", "-modes=export", "-gameconfig=1",
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0",
"-coreplantest=0", "-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings",
"-corehybridmesher=0", f"-out={export}", "-voxel.StreamingCoarseFirst=1",
"-voxel.StreamingDynamicNearReserve=1",
"-voxel.StreamingLegacyPendingResultCap=0",
"-voxel.CoreRegionConcurrentAdmission=1",
"-voxel.StreamingResultBacklogCap=256", "-voxel.StreamingReadyQueueDepth=8",
"-voxel.StreamingBackgroundWorkerCap=1", "-voxel.CoreColumnCacheDirectMapped=0",
"-voxel.StreamingFarApplySupplement=1",
f"-voxel.StreamingFastTravelWaitForAllCriticalTiles={config['wait_for_critical']}",
f"-voxel.StreamingFastTravelApplyBudgetMilliseconds={config['apply_budget_ms']:.1f}",
f"-voxel.StreamingFastTravelForwardFirst={config['forward_first']}",
f"-voxel.StreamingRebuildOnCoreRegionCompletion={config['rebuild_on_region_completion']}",
f"-voxel.StreamingCoreDesiredCacheByCenter={config['core_desired_cache']}",
f"-voxel.StreamingIncrementalDesiredSet={config['incremental_desired']}",
f"-voxel.StreamingIncrementalDesiredAuditEvery={config['audit_every']}",
f"-voxel.StreamingIncrementalDesiredMaterializationAuditEvery={config['materialization_audit_every']}",
f"-voxel.StreamingIncrementalDesiredAuditInjectMismatch={config['inject_mismatch']}",
"-voxel.CoreMCGradientMemoization=1", "-voxel.StreamingCollisionLookahead=1",
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=3000.0",
f"-voxel.TestMoveStartSeconds={MOVE_START_S:.1f}",
"-voxel.TestMoveDurationSeconds=12.0", "-voxel.TestExitSeconds=0.0",
"-voxel.TestMoveDirection=1,0,0", "-voxel.StreamingSchedulerPolicy=3",
"-voxel.TestExitOnSteady=0", "-coreworldstreamtrace=1",
f"-voxel.StartupTraceSpawnCm={spawn_cm}",
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
f"-abslog={abslog}",
]
if tile_hash_dump is not None:
tile_hash_dump.parent.mkdir(parents=True, exist_ok=True)
tile_hash_dump.unlink(missing_ok=True)
command.append(f"-voxel.TileHashDump={tile_hash_dump.resolve()}")
wrapped = ["taskset", "-c", "0-11", "timeout", "1740s", *command]
started = time.monotonic()
with stdout.open("w", encoding="utf-8") as stream:
process = subprocess.run(
wrapped, cwd=ROOT, stdout=stream, stderr=subprocess.STDOUT,
check=False, timeout=1740)
events = read_events(trace)
if expected_mismatch:
diagnostic = "VoxelForgeIncrementalDesiredAudit"
logs = "\n".join(path.read_text(encoding="utf-8", errors="replace")
for path in (abslog, stdout) if path.is_file())
audit_events = [event for event in events
if event.get("type") in {"incremental_desired_audit",
"incremental_desired_order_audit"}]
failed_audit = any(int(event.get("pass", 1)) == 0 for event in audit_events)
failed_audit = failed_audit or (
diagnostic in logs and "mismatch" in logs.lower())
if process.returncode == 0 or diagnostic not in logs or "mismatch" not in logs.lower():
raise RuntimeError(
f"injected mismatch did not fail as expected: exit={process.returncode}; "
f"audit_events={len(audit_events)}; log={abslog}")
print("[L166MismatchControl] " + json.dumps({
"exit_code": process.returncode,
"diagnostic_seen": True,
"failed_audit_event_seen": failed_audit,
"trace": str(trace.relative_to(ROOT)),
"log": str(abslog.relative_to(ROOT)),
}, sort_keys=True), flush=True)
return {"condition": condition, "exit_code": process.returncode,
"failed_audit_event_seen": failed_audit}
samples = [event for event in events if event.get("type") == "streaming_sample"]
moving = [event for event in samples if float(event.get("speed_cm_s", 0.0)) >= 50.0]
stop_event = next((event for event in events
if event.get("type") == "streaming_test_move_stopped"), None)
backend = next((event for event in events if event.get("type") == "worker_backend_config"), {})
scheduler_config = next((event for event in events
if event.get("type") == "scheduler_admission"), {})
worker_cap = next((event for event in events
if event.get("type") == "streaming_worker_cap_readback"), {})
if stop_event is None:
raise RuntimeError(f"missing move-stop event: {trace}")
stop_s = float(stop_event.get("t_s", 0.0))
move_start_s = min((float(event.get("t_s", 0.0)) for event in moving), default=float("inf"))
dwell = [event for event in samples if float(event.get("t_s", 0.0)) >= stop_s
and float(event.get("speed_cm_s", 0.0)) < 50.0]
arrival = [event for event in samples if float(event.get("t_s", 0.0)) < move_start_s
and float(event.get("speed_cm_s", 0.0)) < 50.0]
desired_stage_breakdown = desired_breakdown(events, move_start_s, stop_s)
if process.returncode or len(moving) < 40 or len(dwell) < 16:
raise RuntimeError(f"incomplete {condition} run: {trace}; exit={process.returncode}; "
f"move={len(moving)} dwell={len(dwell)}")
if (int(backend.get("streaming_fast_travel_wait_for_all_critical_tiles", -1))
!= config["wait_for_critical"]
or abs(float(backend.get("streaming_fast_travel_apply_budget_ms", -1.0))
- config["apply_budget_ms"]) > 0.01
or int(worker_cap.get("streaming_background_worker_cap", -1)) != 1
or int(backend.get("effective_tile_slots", -1)) != 8
or int(backend.get("streaming_far_apply_supplement", -1)) != 1
or int(scheduler_config.get("forward_first", -1)) != config["forward_first"]
or int(scheduler_config.get("rebuild_on_core_region_completion", -1))
!= config["rebuild_on_region_completion"]
or int(scheduler_config.get("core_desired_cache_by_center", -1))
!= config["core_desired_cache"]
or int(scheduler_config.get("incremental_desired_set", -1))
!= config["incremental_desired"]
or int(scheduler_config.get("incremental_desired_audit_every", -1))
!= config["audit_every"]
or int(scheduler_config.get("incremental_desired_materialization_audit_every", -1))
!= config["materialization_audit_every"]
or int(scheduler_config.get("incremental_desired_audit_inject_mismatch", -1))
!= config["inject_mismatch"]):
raise RuntimeError(f"CVar/worker readback mismatch: {backend}; {worker_cap}; {scheduler_config}")
moving_metrics = []
for event in moving:
desired = event.get("desired_by_lod", [0] * 9)
resident = event.get("resident_by_lod", [0] * 9)
near_desired = sum(desired[:3])
cone_desired = int(event.get("near_forward_cone_desired", 0))
view_cone_desired = int(event.get("near_view_forward_cone_desired", 0))
view_valid = bool(event.get("near_view_forward_cone_valid", 0))
weighted_desired = float(event.get("near_forward_weighted_desired", 0.0))
travel_cone_ready = percent(
int(event.get("near_forward_cone_resident", 0)), cone_desired)
view_cone_ready = percent(
int(event.get("near_view_forward_cone_resident", 0)), view_cone_desired)
moving_metrics.append({
"near_ready_pct": percent(sum(resident[:3]), near_desired) or 0.0,
"lod0_ready_pct": percent(resident[0], desired[0]) or 0.0,
"lod1_ready_pct": percent(resident[1], desired[1]) or 0.0,
"lod2_ready_pct": percent(resident[2], desired[2]) or 0.0,
"far_hole_pct": percent(
sum(desired[3:]) - sum(event.get("covered_by_lod", resident)[3:]),
sum(desired[3:])) or 0.0,
"ahead_share_pct": percent(int(event.get("near_ahead_desired", 0)), near_desired),
"behind_share_pct": percent(int(event.get("near_behind_desired", 0)), near_desired),
"side_share_pct": 100.0 - (
percent(int(event.get("near_ahead_desired", 0)), near_desired) or 0.0
) - (
percent(int(event.get("near_behind_desired", 0)), near_desired) or 0.0
),
"ahead_ready_pct": percent(
int(event.get("near_ahead_resident", 0)),
int(event.get("near_ahead_desired", 0))),
"behind_ready_pct": percent(
int(event.get("near_behind_resident", 0)),
int(event.get("near_behind_desired", 0))),
"travel_forward_cone_ready_pct": travel_cone_ready,
"view_forward_cone_ready_pct": view_cone_ready,
"forward_cone_ready_pct": view_cone_ready if view_valid else travel_cone_ready,
"forward_cone_source": "camera" if view_valid else "travel_fallback",
"forward_cone_share_pct": percent(cone_desired, near_desired),
"forward_weighted_ready_pct": percent(
float(event.get("near_forward_weighted_resident", 0.0)), weighted_desired),
})
arrival_near_settled = next((event for event in arrival
if sum(event.get("desired_by_lod", [0] * 9)[:3]) > 0
and sum(event.get("settled_by_lod", [0] * 9)[:3])
>= sum(event.get("desired_by_lod", [0] * 9)[:3])), None)
dwell_complete = next((event for event in dwell
if sum(event.get("resident_by_lod", [0] * 9)[:3])
>= sum(event.get("desired_by_lod", [0] * 9)[:3])
and sum(event.get("desired_by_lod", [0] * 9)[3:])
<= sum(event.get("covered_by_lod", [0] * 9)[3:])), None)
work = [event for event in events if event.get("type") == "game_thread_work_frame"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
applies = {event.get("frame"): event for event in events
if event.get("type") == "apply_frame"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s}
scheduler = {event.get("frame"): event for event in events
if event.get("type") == "scheduler_admission"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s}
frames = []
for event in work:
apply = applies.get(event.get("frame"), {})
admit = scheduler.get(event.get("frame"), {})
frames.append({
"frame": event.get("frame"),
"tick_ms": float(event.get("tick_ms", 0.0)),
"update_chunks_ms": float(event.get("update_chunks_ms", 0.0)),
"pending_drain_ms": float(event.get("pending_drain_ms", 0.0)),
"apply_ms": float(apply.get("apply_ms", 0.0)),
"apply_budget_ms": float(apply.get("apply_budget_ms", 0.0)),
"meshes_applied": int(apply.get("meshes_applied", 0)),
"time_budget_hit": int(apply.get("time_limit_reached", 0)),
"desired_rebuild_ms": float(admit.get("desired_rebuild_ms", 0.0)),
"desired_rebuilt": int(admit.get("desired_rebuilt", 0)),
"submit_tiles_ms": float(admit.get("submit_tiles_ms", 0.0)),
"level_admit_ms": float(admit.get("level_admit_ms", 0.0)),
})
p99_field = lambda key: percentile([float(row[key]) for row in frames])
max_field = lambda key: max((float(row[key]) for row in frames), default=None)
rebuild_frames = [row for row in frames if row["desired_rebuilt"]]
rebuild_costs = [float(row["desired_rebuild_ms"]) for row in rebuild_frames]
moving_scheduler = [event for event in scheduler.values()]
center_crossings = 0
previous_center = None
for event in sorted(moving_scheduler, key=lambda item: int(item.get("frame", 0))):
center = tuple(event.get("center", ()))
if previous_center is not None and center != previous_center:
center_crossings += 1
previous_center = center
core_requests = [event for event in events
if event.get("type") == "core_region_compile_request"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
same_center_core_requests = sum(
event.get("reason") == "desired_set_changed_same_center" for event in core_requests)
core_desired_queries = [event for event in events
if event.get("type") == "core_desired_cache"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
core_desired_query_ms = [float(event.get("query_ms", 0.0))
for event in core_desired_queries if float(event.get("query_ms", 0.0)) > 0.0]
core_desired_cache_hits = sum(int(event.get("hit", 0)) for event in core_desired_queries)
incremental_audits = [event for event in events
if event.get("type") == "incremental_desired_audit"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
incremental_audit_failures = sum(int(event.get("pass", 1)) == 0
for event in incremental_audits)
order_audits = [event for event in events
if event.get("type") == "incremental_desired_order_audit"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
order_audit_failures = sum(int(event.get("pass", 1)) == 0 for event in order_audits)
materialization_audits = [event for event in incremental_audits
if int(event.get("materialization_audit", 0)) != 0]
if config["incremental_desired"] and (
not incremental_audits or incremental_audit_failures
or not order_audits or order_audit_failures
or (config["materialization_audit_every"] > 0
and not materialization_audits)):
raise RuntimeError(
f"incremental desired equality audit missing/failed: {trace}; "
f"set={len(incremental_audits)}/{incremental_audit_failures} "
f"order={len(order_audits)}/{order_audit_failures} "
f"materialization={len(materialization_audits)}")
direction_submissions = [event.get("near_submitted_by_direction", [0, 0, 0, 0])
for event in moving_scheduler]
direction_submissions = [counts for counts in direction_submissions
if isinstance(counts, list) and len(counts) == 4]
direction_submission_totals = [
sum(int(counts[index]) for counts in direction_submissions) for index in range(4)
]
first_admission_order = next((event.get("near_admission_order_first", [])
for event in moving_scheduler
if event.get("near_admission_order_first")), [])
camera_samples = sum(bool(event.get("near_view_forward_cone_valid", 0)) for event in moving)
dwell_complete_s = (float(dwell_complete.get("t_s", 0.0)) - stop_s
if dwell_complete else None)
row = {
"condition": condition,
"repetition": repetition,
"speed_mps": 30.0,
"wait_for_all_critical_tiles": config["wait_for_critical"],
"fast_travel_apply_budget_ms": config["apply_budget_ms"],
"forward_first": config["forward_first"],
"rebuild_on_core_region_completion": config["rebuild_on_region_completion"],
"core_desired_cache_by_center": config["core_desired_cache"],
"incremental_desired_set": config["incremental_desired"],
"incremental_desired_audit_every": config["audit_every"],
"incremental_desired_materialization_audit_every": config["materialization_audit_every"],
"incremental_desired_audit_count": len(incremental_audits),
"incremental_desired_audit_failures": incremental_audit_failures,
"incremental_desired_order_audit_count": len(order_audits),
"incremental_desired_order_audit_failures": order_audit_failures,
"incremental_desired_materialization_audit_count": len(materialization_audits),
"effective_tile_slots": backend.get("effective_tile_slots"),
"exit_code": process.returncode,
"wall_s": round(time.monotonic() - started, 3),
"moving_samples": len(moving),
"dwell_samples": len(dwell),
"arrival_samples": len(arrival),
"near_ready_moving_mean_pct": mean([m["near_ready_pct"] for m in moving_metrics]),
"lod0_ready_moving_mean_pct": mean([m["lod0_ready_pct"] for m in moving_metrics]),
"lod1_ready_moving_mean_pct": mean([m["lod1_ready_pct"] for m in moving_metrics]),
"lod2_ready_moving_mean_pct": mean([m["lod2_ready_pct"] for m in moving_metrics]),
"far_holes_moving_mean_pct": mean([m["far_hole_pct"] for m in moving_metrics]),
"near_ahead_desired_share_pct": mean([m["ahead_share_pct"] for m in moving_metrics
if m["ahead_share_pct"] is not None]),
"near_behind_desired_share_pct": mean([m["behind_share_pct"] for m in moving_metrics
if m["behind_share_pct"] is not None]),
"near_side_desired_share_pct": mean([m["side_share_pct"] for m in moving_metrics]),
"near_ahead_ready_pct": mean([m["ahead_ready_pct"] for m in moving_metrics
if m["ahead_ready_pct"] is not None]),
"near_behind_ready_pct": mean([m["behind_ready_pct"] for m in moving_metrics
if m["behind_ready_pct"] is not None]),
"near_forward_cone_half_angle_deg": 55,
"near_forward_cone_desired_share_pct": mean([m["forward_cone_share_pct"]
for m in moving_metrics
if m["forward_cone_share_pct"] is not None]),
"near_forward_cone_ready_pct": mean([m["forward_cone_ready_pct"]
for m in moving_metrics
if m["forward_cone_ready_pct"] is not None]),
"near_travel_forward_cone_ready_pct": mean(
[m["travel_forward_cone_ready_pct"] for m in moving_metrics
if m["travel_forward_cone_ready_pct"] is not None]),
"near_view_forward_cone_ready_pct": mean(
[m["view_forward_cone_ready_pct"] for m in moving_metrics
if m["view_forward_cone_ready_pct"] is not None]),
"camera_forward_samples": camera_samples,
"camera_forward_sample_share_pct": percent(camera_samples, len(moving)),
"near_forward_cosine_weighted_ready_pct": mean(
[m["forward_weighted_ready_pct"] for m in moving_metrics
if m["forward_weighted_ready_pct"] is not None]),
"arrival_lod02_settled_t_s": (
float(arrival_near_settled.get("t_s", 0.0)) if arrival_near_settled else None),
"near_ready_dwell_end_pct": (100.0 * sum(dwell[-1].get("resident_by_lod", [0] * 9)[:3])
/ max(1, sum(dwell[-1].get("desired_by_lod", [0] * 9)[:3]))),
"far_holes_dwell_end_pct": (100.0 * (
sum(dwell[-1].get("desired_by_lod", [0] * 9)[3:])
- sum(dwell[-1].get("covered_by_lod", [0] * 9)[3:]))
/ max(1, sum(dwell[-1].get("desired_by_lod", [0] * 9)[3:]))),
"dwell_complete_after_stop_s": dwell_complete_s,
"game_thread_tick_p99_ms": p99_field("tick_ms"),
"game_thread_tick_max_ms": max_field("tick_ms"),
"stream_update_p99_ms": p99_field("update_chunks_ms"),
"pending_drain_p99_ms": p99_field("pending_drain_ms"),
"apply_p99_ms": p99_field("apply_ms"),
"apply_max_ms": max_field("apply_ms"),
"apply_budget_min_ms": min((f["apply_budget_ms"] for f in frames), default=None),
"apply_budget_max_ms": max((f["apply_budget_ms"] for f in frames), default=None),
"apply_time_budget_hit_frames": sum(f["time_budget_hit"] for f in frames),
"apply_frames": sum(f["meshes_applied"] > 0 for f in frames),
"apply_meshes_total": sum(f["meshes_applied"] for f in frames),
"desired_rebuild_p99_ms": p99_field("desired_rebuild_ms"),
"desired_rebuild_count": len(rebuild_frames),
"desired_rebuilds_per_second": len(rebuild_frames) / 12.0,
"center_crossings": center_crossings,
"same_center_core_region_requests": same_center_core_requests,
"core_desired_cache_hit_count": core_desired_cache_hits,
"core_desired_cache_hit_share_pct": percent(
core_desired_cache_hits, len(core_desired_queries)),
"core_desired_query_nonzero_p50_ms": statistics.median(core_desired_query_ms)
if core_desired_query_ms else None,
"core_desired_query_nonzero_p95_ms": percentile(core_desired_query_ms, 0.95),
"core_desired_query_nonzero_max_ms": max(core_desired_query_ms)
if core_desired_query_ms else None,
"desired_rebuild_nonzero_p50_ms": statistics.median(rebuild_costs)
if rebuild_costs else None,
"desired_rebuild_nonzero_p95_ms": percentile(rebuild_costs, 0.95),
"desired_rebuild_nonzero_max_ms": max(rebuild_costs) if rebuild_costs else None,
"desired_breakdown": desired_stage_breakdown,
"near_submitted_by_direction": direction_submission_totals,
"near_admission_first12": first_admission_order,
"submit_tiles_p99_ms": p99_field("submit_tiles_ms"),
"level_admit_p99_ms": p99_field("level_admit_ms"),
"top_tick_frames": sorted(frames, key=lambda f: f["tick_ms"], reverse=True)[:5],
"trace": str(trace.relative_to(ROOT)),
"log": str(abslog.relative_to(ROOT)),
}
print("[L166Movement] " + json.dumps(row, sort_keys=True), flush=True)
return row
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--smoke", action="store_true",
help="run one incremental trace to validate readback and equality audits")
parser.add_argument("--count-only", action="store_true",
help="run one baseline/candidate pair for desired-stage cost attribution")
parser.add_argument("--geometry-hash-check", action="store_true",
help="compare UE per-tile geometry hashes for baseline and incremental desired sets")
parser.add_argument("--controls-only", action="store_true",
help="run a per-crossing equality audit and injected mismatch negative control")
parser.add_argument("--summarize-existing", action="store_true",
help="summarize the fourteen completed rows in runs-partial.json without rerunning UE")
args = parser.parse_args()
missing = [path for path in (EDITOR, HOST / "VFHost.uproject") if not path.exists()]
if missing:
raise SystemExit(f"missing UE runner input: {missing}")
OUT.mkdir(parents=True, exist_ok=True)
if args.geometry_hash_check:
hash_dir = OUT / "geometry-hash"
baseline_path = hash_dir / "assembler-world.json"
candidate_path = hash_dir / "incremental.json"
run("assembler_world", 99, baseline_path)
run("incremental", 99, candidate_path)
baseline_data = json.loads(baseline_path.read_text(encoding="utf-8"))
candidate_data = json.loads(candidate_path.read_text(encoding="utf-8"))
key_fields = ("level", "tile_x", "tile_y", "tile_z")
def indexed(data: dict) -> dict[tuple[int, ...], tuple]:
result = {}
for record in data.get("records", []):
key = tuple(int(record[field]) for field in key_fields)
result[key] = (str(record.get("geometry_hash", "")),
int(record.get("triangle_count", -1)),
bool(record.get("empty", False)))
return result
baseline_hashes = indexed(baseline_data)
candidate_hashes = indexed(candidate_data)
common = baseline_hashes.keys() & candidate_hashes.keys()
mismatches = [key for key in common
if baseline_hashes[key] != candidate_hashes[key]]
result = {
"baseline_records": len(baseline_hashes),
"candidate_records": len(candidate_hashes),
"common_records": len(common),
"hash_mismatches": len(mismatches),
"baseline_only_records": len(baseline_hashes.keys() - candidate_hashes.keys()),
"candidate_only_records": len(candidate_hashes.keys() - baseline_hashes.keys()),
"baseline_dump": str(baseline_path.relative_to(ROOT)),
"candidate_dump": str(candidate_path.relative_to(ROOT)),
}
(hash_dir / "comparison.json").write_text(
json.dumps(result, indent=2, sort_keys=True) + "\n", encoding="utf-8")
print("[L166GeometryHash] " + json.dumps(result, sort_keys=True), flush=True)
return 0 if len(common) >= 100 and not mismatches else 1
if args.count_only:
run("assembler_world", 99)
run("incremental", 99)
return 0
if args.smoke:
run("incremental", 0)
return 0
if args.controls_only:
control = dict(CONDITIONS["incremental"])
control["audit_every"] = 0
control["materialization_audit_every"] = 25
control["inject_mismatch"] = 0
CONDITIONS["materialization_audit_control"] = control
run("materialization_audit_control", 0)
mismatch = dict(control)
mismatch["materialization_audit_every"] = 1
mismatch["inject_mismatch"] = 1
CONDITIONS["injected_materialization_mismatch_control"] = mismatch
run("injected_materialization_mismatch_control", 0, expected_mismatch=True)
return 0
partial = OUT / "runs-partial.json"
if args.summarize_existing:
if not partial.is_file():
raise RuntimeError(f"no saved movement rows: {partial}")
rows = json.loads(partial.read_text(encoding="utf-8"))
if len(rows) != 14:
raise RuntimeError(f"expected fourteen completed movement rows, found {len(rows)}")
else:
rows = []
# Both conditions use this editor binary. Warm each path before measured pairs.
run("assembler_world", 0)
run("incremental", 0)
for repetition in range(1, 8):
for condition in ORDER[repetition]:
row = run(condition, repetition)
rows.append(row)
partial.write_text(json.dumps(rows, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
metrics = (
"near_ready_moving_mean_pct", "far_holes_moving_mean_pct",
"game_thread_tick_p99_ms", "game_thread_tick_max_ms",
"arrival_lod02_settled_t_s", "dwell_complete_after_stop_s",
"near_ahead_desired_share_pct", "near_behind_desired_share_pct",
"near_side_desired_share_pct", "near_forward_cone_desired_share_pct",
"near_ahead_ready_pct", "near_behind_ready_pct",
"near_forward_cone_ready_pct", "near_forward_cosine_weighted_ready_pct",
"near_travel_forward_cone_ready_pct",
"apply_p99_ms", "desired_rebuild_p99_ms", "desired_rebuild_count",
"desired_rebuild_nonzero_p95_ms", "center_crossings",
"same_center_core_region_requests", "level_admit_p99_ms",
"core_desired_cache_hit_count", "core_desired_query_nonzero_p95_ms",
)
medians = {}
for condition in CONDITIONS:
medians[condition] = {}
for metric in metrics:
values = [row[metric] for row in rows
if row["condition"] == condition and row[metric] is not None]
medians[condition][metric] = statistics.median(values) if values else None
pairs = []
for repetition in range(1, 8):
by_condition = {row["condition"]: row for row in rows
if row["repetition"] == repetition}
candidate = "incremental"
pairs.append({
"repetition": repetition,
"candidate": candidate,
**{f"{metric}_delta": by_condition[candidate][metric]
- by_condition["assembler_world"][metric]
for metric in metrics if by_condition[candidate][metric] is not None
and by_condition["assembler_world"][metric] is not None},
})
by_pair = {
repetition: {row["condition"]: row for row in rows if row["repetition"] == repetition}
for repetition in range(1, 8)
}
candidate_rows = [row for row in rows if row["condition"] == "incremental"]
baseline_rows = [row for row in rows if row["condition"] == "assembler_world"]
candidate_gates = {
"every_candidate_run_p99_le_11_ms": all(
row["game_thread_tick_p99_ms"] <= 11.0 for row in candidate_rows),
"paired_p99_no_worse": all(
by_pair[rep]["incremental"]["game_thread_tick_p99_ms"]
<= by_pair[rep]["assembler_world"]["game_thread_tick_p99_ms"]
for rep in by_pair),
"paired_worst_tick_no_worse": all(
by_pair[rep]["incremental"]["game_thread_tick_max_ms"]
<= by_pair[rep]["assembler_world"]["game_thread_tick_max_ms"]
for rep in by_pair),
"paired_near_all_around_no_worse": all(
by_pair[rep]["incremental"]["near_ready_moving_mean_pct"]
>= by_pair[rep]["assembler_world"]["near_ready_moving_mean_pct"]
for rep in by_pair),
"paired_forward_cone_no_worse": all(
by_pair[rep]["incremental"]["near_forward_cone_ready_pct"]
>= by_pair[rep]["assembler_world"]["near_forward_cone_ready_pct"]
for rep in by_pair),
"paired_far_holes_no_worse": all(
by_pair[rep]["incremental"]["far_holes_moving_mean_pct"]
<= by_pair[rep]["assembler_world"]["far_holes_moving_mean_pct"]
for rep in by_pair),
"paired_arrival_no_worse": all(
by_pair[rep]["incremental"]["arrival_lod02_settled_t_s"] is not None
and by_pair[rep]["assembler_world"]["arrival_lod02_settled_t_s"] is not None
and by_pair[rep]["incremental"]["arrival_lod02_settled_t_s"]
<= by_pair[rep]["assembler_world"]["arrival_lod02_settled_t_s"]
for rep in by_pair),
"all_candidate_equality_audits_pass": all(
row["incremental_desired_audit_count"] > 0
and row["incremental_desired_audit_failures"] == 0
and row["incremental_desired_order_audit_count"] > 0
and row["incremental_desired_order_audit_failures"] == 0
for row in candidate_rows),
"median_build_desired_p95_down": statistics.median(
row["desired_rebuild_nonzero_p95_ms"] for row in candidate_rows)
< statistics.median(row["desired_rebuild_nonzero_p95_ms"]
for row in baseline_rows),
}
summary = {
"source_commit": subprocess.run(
["git", "-c", f"safe.directory={ROOT}", "rev-parse", "HEAD"],
cwd=ROOT, check=True, capture_output=True, text=True).stdout.strip(),
"protocol": "Seven interleaved 30 m/s UE pairs; 12 s move plus 5 s dwell; 8 effective tile slots. L157 critical wait remains on and apply cap remains 0 ms.",
"conditions": CONDITIONS,
"movement_medians": medians,
"paired_deltas_vs_assembler_world": pairs,
"candidate_gates": candidate_gates,
"runs": rows,
}
(OUT / "summary.json").write_text(json.dumps(summary, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
print("[L166Summary] " + json.dumps({"movement_medians": medians,
"candidate_gates": candidate_gates,
"runs": len(rows)}, sort_keys=True), flush=True)
return 0 if all(candidate_gates.values()) else 1
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,39 @@
# L167 — Re-test L157 and L160 on the faster base
**Result:** No candidate meets the full default bar. Leave L157 and L160 disabled. L160 is tick-safe in this W8 run but lowers all-around near readiness and delays arrival. L157 improves readiness but still spikes; combining them improves readiness most, but worsens worst ticks and delays arrival.
## Measurement
Ran base, L157, L160, and both in seven interleaved repetitions each at 30 m/s. Each measured run covered a 12-second move and 5-second dwell, with eight effective tile slots and the four-strata owner plan. One base warm-up was excluded. Timing began with 1/5/15-minute loads at 0.77/0.96/1.46 and no other build or editor process running. Values below are medians across seven runs; p99 range and maximum worst tick show the full run spread.
The common base had `StreamingIncrementalDesiredSet=1` with audits off, matching the faster L165/L166 path. The checked-in CVar default remains off; this was a controlled runtime setting for the comparison. Null-RHI provided no camera-forward samples, so the 55-degree forward cone uses travel direction.
| Condition | Tick p99 median (run range), ms | Worst tick median / run max, ms | Near readiness, % | Forward cone, % | Far holes, % | LOD0–2 arrival, s |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| Base | 7.289 (6.786–7.513) | 12.061 / 12.465 | 43.39 | 15.77 | 3.00 | 2.230 |
| L157 | 8.582 (8.046–12.462) | 12.462 / 16.689 | 55.44 | 18.61 | 2.64 | 2.469 |
| L160 | 7.187 (6.962–8.009) | 11.747 / 12.503 | 38.03 | 20.32 | 2.59 | 2.447 |
| Both | 8.932 (8.064–10.973) | 12.594 / 14.278 | 45.55 | 25.24 | 2.85 | 2.461 |
Near readiness, cone readiness, and far holes improved for L157 and the combination; L160 lowered all-around near readiness. All three candidates delayed median arrival by 0.216–0.239 seconds. L157 had one run above the 11 ms p99 limit and a 16.689 ms worst tick. The combination stayed within the p99 limit in all seven runs, but its median worst tick was 0.533 ms worse than base and one run reached 14.278 ms. L160 passed the tick limits but failed readiness and arrival gates. Therefore none qualifies as default.
## Spike count and cause
Trace data attributes the L157 spikes to main-thread desired-set and scheduler work, not the 1 ms apply budget:
- At the 16.689 ms worst tick, `UpdateChunks` took 15.500 ms. `SubmitTiles` took 8.807 ms, including 8.336 ms in level sorting; desired rebuild took 5.353 ms. Apply took only 0.374 ms. The scheduler trace counted 912 reserve-declined tiles and three near tasks submitted.
- A second 16.198 ms tick had 13.852 ms in `UpdateChunks` and an 11.777 ms desired rebuild, including 6.679 ms of overlay materialization. `SubmitTiles` took 0.813 ms and apply took 1.087 ms.
- The combined run's 14.278 ms maximum had 10.279 ms in `UpdateChunks`, 2.859 ms draining pending results, and one applied result taking 2.799 ms against the 1 ms budget (`time_limit_reached=1`). Its level sort was only 0.095 ms. The budget is checked after a result completes, so it cannot bound one expensive apply.
These traces show more than one source: level sorting under L157 admission, expensive desired-set rebuild/materialization, and individual result-apply cost in the combined mode. No tuning change was attempted because the measured work does not support a single cap adjustment that would address all three; all candidates also fail another promotion gate.
## Checks and limits
- VFHost Linux Development build succeeded with `-NoUBA -MaxParallelActions=4`; the temporary VFHost plugin source link was restored afterward.
- `verify.py fast` passed in 298.0 seconds: 268 pass, 0 fail, 5 diagnostics. Its 84 binary runs had 82 pass and 2 expected failures. Geometry checks and Clang/GCC W1/W8/W12 determinism passed. The L167 source stamp matched the L166 verifier binary (`ba79279c16018d2996e724df0f7c0e8a210929baebc1aa14bef6a76fbda96faa`).
- Movement timings used W8, the effective slot count read back by this VFHost command. W12 movement was not run; the command's 12-core affinity exposed eight TaskGraph background workers and eight effective streaming slots. Fast-tier W12 determinism did run. The live movement `TileHashDump` hook was not enabled; no geometry-generation source changed.
- Runner `py_compile` and `git diff --check` passed on the final files. No Unreal `Source/` files changed.
Changed Unreal `Source/` files: none.
Artifacts are under `Saved/Linux/l167/`; the interleaved movement runner is `RunL167Movement.py` alongside this report.
@@ -0,0 +1,639 @@
#!/usr/bin/env python3
"""Interleaved L167 UE movement runs for the L157/L160 levers on the L165/L166 base."""
from __future__ import annotations
import argparse
import json
import math
import statistics
import subprocess
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
HOST = Path("/home/jahni/VFHost")
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
OUT = ROOT / "Saved/Linux/l167/ue/movement"
HOST_SAVED = HOST / "Saved/Linux/l167"
HOST_PLUGIN_SAVED = HOST / "Plugins/VoxelForge/Saved/Linux/l167"
SPAWN_M = (19.166015625, -14.2001953125, -753.3740234375)
MOVE_START_S = 6.0
_COMMON = {
# Keep the faster L165/L166 path active for every condition; omit its audit
# during timing so the audit itself cannot create a frame spike.
"incremental_desired": 1, "audit_every": 0,
"materialization_audit_every": 0, "inject_mismatch": 0,
}
CONDITIONS = {
"base": {
**_COMMON, "wait_for_critical": 1, "apply_budget_ms": 0.0,
"forward_first": 0, "rebuild_on_region_completion": 1,
"core_desired_cache": 0,
},
"l157": {
**_COMMON, "wait_for_critical": 0, "apply_budget_ms": 1.0,
"forward_first": 0, "rebuild_on_region_completion": 1,
"core_desired_cache": 0,
},
"l160": {
**_COMMON, "wait_for_critical": 1, "apply_budget_ms": 0.0,
"forward_first": 1, "rebuild_on_region_completion": 0,
"core_desired_cache": 1,
},
"both": {
**_COMMON, "wait_for_critical": 0, "apply_budget_ms": 1.0,
"forward_first": 1, "rebuild_on_region_completion": 0,
"core_desired_cache": 1,
},
}
ORDER = {
1: ("base", "l157", "l160", "both"),
2: ("l157", "l160", "both", "base"),
3: ("l160", "both", "base", "l157"),
4: ("both", "base", "l157", "l160"),
5: ("l160", "l157", "base", "both"),
6: ("both", "l160", "l157", "base"),
7: ("base", "both", "l160", "l157"),
}
def percentile(values: list[float], fraction: float = 0.99) -> float | None:
if not values:
return None
ordered = sorted(values)
return ordered[max(0, min(len(ordered) - 1, math.ceil(len(ordered) * fraction) - 1))]
def read_events(path: Path) -> list[dict]:
rows = []
if path.is_file():
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
if not line.startswith("{"):
continue
try:
rows.append(json.loads(line))
except json.JSONDecodeError:
continue
return rows
def mean(values: list[float]) -> float | None:
return statistics.mean(values) if values else None
def desired_breakdown(events: list[dict], move_start_s: float, stop_s: float) -> dict:
rows = [event for event in events
if event.get("type") == "desired_rebuild_breakdown"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
keys = ("total_ms", "core_prepare_ms", "core_delta_ms", "fallback_ms",
"overlay_materialize_ms", "priority_sort_ms", "critical_order_ms")
result = {"rebuild_count": len(rows)}
result["incremental_materialization_update_count"] = sum(
int(event.get("materialization_update", 0)) != 0 for event in rows)
result["priority_order_reuse_count"] = sum(
int(event.get("priority_order_reused", 0)) != 0 for event in rows)
result["priority_order_update_count"] = sum(
int(event.get("priority_order_updated", 0)) != 0 for event in rows)
result["incremental_materialization_update_share_pct"] = percent(
result["incremental_materialization_update_count"], len(rows))
result["priority_order_reuse_share_pct"] = percent(
result["priority_order_reuse_count"], len(rows))
result["priority_order_update_share_pct"] = percent(
result["priority_order_update_count"], len(rows))
for key in keys:
values = [float(event.get(key, 0.0)) for event in rows]
result[f"{key}_p50"] = statistics.median(values) if values else None
result[f"{key}_p95"] = percentile(values, 0.95)
result[f"{key}_max"] = max(values) if values else None
fallback_rows = [event for event in events
if event.get("type") == "core_desired_cache"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s
and int(event.get("full_fallback", 0)) != 0]
result["fallback_count"] = len(fallback_rows)
result["lod4plus_fallback_count"] = sum(
int(event.get("fallback_reason", 0)) == 1 for event in fallback_rows)
result["other_fallback_count"] = sum(
int(event.get("fallback_reason", 0)) == 2 for event in fallback_rows)
result["fallback_cost_sum_ms"] = sum(
float(event.get("fallback_ms", 0.0)) for event in fallback_rows)
result["fallback_cost_max_ms"] = max(
(float(event.get("fallback_ms", 0.0)) for event in fallback_rows), default=None)
return result
def percent(numerator: int | float, denominator: int | float) -> float | None:
return 100.0 * numerator / denominator if denominator > 0 else None
def run(condition: str, repetition: int, tile_hash_dump: Path | None = None,
expected_mismatch: bool = False) -> dict:
config = CONDITIONS[condition]
run_name = f"fast-r{repetition}"
out = OUT / condition
host_out = HOST_PLUGIN_SAVED / condition / "output"
userdir = HOST_SAVED / "user" / condition / run_name
zendata = HOST_SAVED / "zen" / condition / run_name
ddc = HOST_SAVED / "DDC"
trace = out / f"{run_name}.trace.jsonl"
abslog = out / f"{run_name}.log"
stdout = out / f"{run_name}.stdout.log"
export = host_out / run_name
for directory in (out, export, userdir, zendata, ddc):
directory.mkdir(parents=True, exist_ok=True)
trace.unlink(missing_ok=True)
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWN_M)
command = [
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
"-notraceserver", "-nocrashreports", "-DisablePlugins=AndroidFileServer",
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
"-archetype=TunnelNetwork", "-slot=0", "-modes=export", "-gameconfig=1",
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0",
"-coreplantest=0", "-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings",
"-corehybridmesher=0", f"-out={export}", "-voxel.StreamingCoarseFirst=1",
"-voxel.StreamingDynamicNearReserve=1",
"-voxel.StreamingLegacyPendingResultCap=0",
"-voxel.CoreRegionConcurrentAdmission=1",
"-voxel.StreamingResultBacklogCap=256", "-voxel.StreamingReadyQueueDepth=8",
"-voxel.StreamingBackgroundWorkerCap=1", "-voxel.CoreColumnCacheDirectMapped=0",
"-voxel.StreamingFarApplySupplement=1",
f"-voxel.StreamingFastTravelWaitForAllCriticalTiles={config['wait_for_critical']}",
f"-voxel.StreamingFastTravelApplyBudgetMilliseconds={config['apply_budget_ms']:.1f}",
f"-voxel.StreamingFastTravelForwardFirst={config['forward_first']}",
f"-voxel.StreamingRebuildOnCoreRegionCompletion={config['rebuild_on_region_completion']}",
f"-voxel.StreamingCoreDesiredCacheByCenter={config['core_desired_cache']}",
f"-voxel.StreamingIncrementalDesiredSet={config['incremental_desired']}",
f"-voxel.StreamingIncrementalDesiredAuditEvery={config['audit_every']}",
f"-voxel.StreamingIncrementalDesiredMaterializationAuditEvery={config['materialization_audit_every']}",
f"-voxel.StreamingIncrementalDesiredAuditInjectMismatch={config['inject_mismatch']}",
"-voxel.CoreMCGradientMemoization=1", "-voxel.StreamingCollisionLookahead=1",
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=3000.0",
f"-voxel.TestMoveStartSeconds={MOVE_START_S:.1f}",
"-voxel.TestMoveDurationSeconds=12.0", "-voxel.TestExitSeconds=0.0",
"-voxel.TestMoveDirection=1,0,0", "-voxel.StreamingSchedulerPolicy=3",
"-voxel.TestExitOnSteady=0", "-coreworldstreamtrace=1",
f"-voxel.StartupTraceSpawnCm={spawn_cm}",
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
f"-abslog={abslog}",
]
if tile_hash_dump is not None:
tile_hash_dump.parent.mkdir(parents=True, exist_ok=True)
tile_hash_dump.unlink(missing_ok=True)
command.append(f"-voxel.TileHashDump={tile_hash_dump.resolve()}")
wrapped = ["taskset", "-c", "0-11", "timeout", "1740s", *command]
started = time.monotonic()
with stdout.open("w", encoding="utf-8") as stream:
process = subprocess.run(
wrapped, cwd=ROOT, stdout=stream, stderr=subprocess.STDOUT,
check=False, timeout=1740)
events = read_events(trace)
if expected_mismatch:
diagnostic = "VoxelForgeIncrementalDesiredAudit"
logs = "\n".join(path.read_text(encoding="utf-8", errors="replace")
for path in (abslog, stdout) if path.is_file())
audit_events = [event for event in events
if event.get("type") in {"incremental_desired_audit",
"incremental_desired_order_audit"}]
failed_audit = any(int(event.get("pass", 1)) == 0 for event in audit_events)
failed_audit = failed_audit or (
diagnostic in logs and "mismatch" in logs.lower())
if process.returncode == 0 or diagnostic not in logs or "mismatch" not in logs.lower():
raise RuntimeError(
f"injected mismatch did not fail as expected: exit={process.returncode}; "
f"audit_events={len(audit_events)}; log={abslog}")
print("[L166MismatchControl] " + json.dumps({
"exit_code": process.returncode,
"diagnostic_seen": True,
"failed_audit_event_seen": failed_audit,
"trace": str(trace.relative_to(ROOT)),
"log": str(abslog.relative_to(ROOT)),
}, sort_keys=True), flush=True)
return {"condition": condition, "exit_code": process.returncode,
"failed_audit_event_seen": failed_audit}
samples = [event for event in events if event.get("type") == "streaming_sample"]
moving = [event for event in samples if float(event.get("speed_cm_s", 0.0)) >= 50.0]
stop_event = next((event for event in events
if event.get("type") == "streaming_test_move_stopped"), None)
backend = next((event for event in events if event.get("type") == "worker_backend_config"), {})
scheduler_config = next((event for event in events
if event.get("type") == "scheduler_admission"), {})
worker_cap = next((event for event in events
if event.get("type") == "streaming_worker_cap_readback"), {})
if stop_event is None:
raise RuntimeError(f"missing move-stop event: {trace}")
stop_s = float(stop_event.get("t_s", 0.0))
move_start_s = min((float(event.get("t_s", 0.0)) for event in moving), default=float("inf"))
dwell = [event for event in samples if float(event.get("t_s", 0.0)) >= stop_s
and float(event.get("speed_cm_s", 0.0)) < 50.0]
arrival = [event for event in samples if float(event.get("t_s", 0.0)) < move_start_s
and float(event.get("speed_cm_s", 0.0)) < 50.0]
desired_stage_breakdown = desired_breakdown(events, move_start_s, stop_s)
if process.returncode or len(moving) < 40 or len(dwell) < 16:
raise RuntimeError(f"incomplete {condition} run: {trace}; exit={process.returncode}; "
f"move={len(moving)} dwell={len(dwell)}")
if (int(backend.get("streaming_fast_travel_wait_for_all_critical_tiles", -1))
!= config["wait_for_critical"]
or abs(float(backend.get("streaming_fast_travel_apply_budget_ms", -1.0))
- config["apply_budget_ms"]) > 0.01
or int(worker_cap.get("streaming_background_worker_cap", -1)) != 1
or int(backend.get("effective_tile_slots", -1)) != 8
or int(backend.get("streaming_far_apply_supplement", -1)) != 1
or int(scheduler_config.get("forward_first", -1)) != config["forward_first"]
or int(scheduler_config.get("rebuild_on_core_region_completion", -1))
!= config["rebuild_on_region_completion"]
or int(scheduler_config.get("core_desired_cache_by_center", -1))
!= config["core_desired_cache"]
or int(scheduler_config.get("incremental_desired_set", -1))
!= config["incremental_desired"]
or int(scheduler_config.get("incremental_desired_audit_every", -1))
!= config["audit_every"]
or int(scheduler_config.get("incremental_desired_materialization_audit_every", -1))
!= config["materialization_audit_every"]
or int(scheduler_config.get("incremental_desired_audit_inject_mismatch", -1))
!= config["inject_mismatch"]):
raise RuntimeError(f"CVar/worker readback mismatch: {backend}; {worker_cap}; {scheduler_config}")
moving_metrics = []
for event in moving:
desired = event.get("desired_by_lod", [0] * 9)
resident = event.get("resident_by_lod", [0] * 9)
near_desired = sum(desired[:3])
cone_desired = int(event.get("near_forward_cone_desired", 0))
view_cone_desired = int(event.get("near_view_forward_cone_desired", 0))
view_valid = bool(event.get("near_view_forward_cone_valid", 0))
weighted_desired = float(event.get("near_forward_weighted_desired", 0.0))
travel_cone_ready = percent(
int(event.get("near_forward_cone_resident", 0)), cone_desired)
view_cone_ready = percent(
int(event.get("near_view_forward_cone_resident", 0)), view_cone_desired)
moving_metrics.append({
"near_ready_pct": percent(sum(resident[:3]), near_desired) or 0.0,
"lod0_ready_pct": percent(resident[0], desired[0]) or 0.0,
"lod1_ready_pct": percent(resident[1], desired[1]) or 0.0,
"lod2_ready_pct": percent(resident[2], desired[2]) or 0.0,
"far_hole_pct": percent(
sum(desired[3:]) - sum(event.get("covered_by_lod", resident)[3:]),
sum(desired[3:])) or 0.0,
"ahead_share_pct": percent(int(event.get("near_ahead_desired", 0)), near_desired),
"behind_share_pct": percent(int(event.get("near_behind_desired", 0)), near_desired),
"side_share_pct": 100.0 - (
percent(int(event.get("near_ahead_desired", 0)), near_desired) or 0.0
) - (
percent(int(event.get("near_behind_desired", 0)), near_desired) or 0.0
),
"ahead_ready_pct": percent(
int(event.get("near_ahead_resident", 0)),
int(event.get("near_ahead_desired", 0))),
"behind_ready_pct": percent(
int(event.get("near_behind_resident", 0)),
int(event.get("near_behind_desired", 0))),
"travel_forward_cone_ready_pct": travel_cone_ready,
"view_forward_cone_ready_pct": view_cone_ready,
"forward_cone_ready_pct": view_cone_ready if view_valid else travel_cone_ready,
"forward_cone_source": "camera" if view_valid else "travel_fallback",
"forward_cone_share_pct": percent(cone_desired, near_desired),
"forward_weighted_ready_pct": percent(
float(event.get("near_forward_weighted_resident", 0.0)), weighted_desired),
})
arrival_near_settled = next((event for event in arrival
if sum(event.get("desired_by_lod", [0] * 9)[:3]) > 0
and sum(event.get("settled_by_lod", [0] * 9)[:3])
>= sum(event.get("desired_by_lod", [0] * 9)[:3])), None)
dwell_complete = next((event for event in dwell
if sum(event.get("resident_by_lod", [0] * 9)[:3])
>= sum(event.get("desired_by_lod", [0] * 9)[:3])
and sum(event.get("desired_by_lod", [0] * 9)[3:])
<= sum(event.get("covered_by_lod", [0] * 9)[3:])), None)
work = [event for event in events if event.get("type") == "game_thread_work_frame"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
applies = {event.get("frame"): event for event in events
if event.get("type") == "apply_frame"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s}
scheduler = {event.get("frame"): event for event in events
if event.get("type") == "scheduler_admission"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s}
frames = []
for event in work:
apply = applies.get(event.get("frame"), {})
admit = scheduler.get(event.get("frame"), {})
frames.append({
"frame": event.get("frame"),
"tick_ms": float(event.get("tick_ms", 0.0)),
"update_chunks_ms": float(event.get("update_chunks_ms", 0.0)),
"pending_drain_ms": float(event.get("pending_drain_ms", 0.0)),
"apply_ms": float(apply.get("apply_ms", 0.0)),
"apply_budget_ms": float(apply.get("apply_budget_ms", 0.0)),
"meshes_applied": int(apply.get("meshes_applied", 0)),
"time_budget_hit": int(apply.get("time_limit_reached", 0)),
"desired_rebuild_ms": float(admit.get("desired_rebuild_ms", 0.0)),
"desired_rebuilt": int(admit.get("desired_rebuilt", 0)),
"submit_tiles_ms": float(admit.get("submit_tiles_ms", 0.0)),
"level_admit_ms": float(admit.get("level_admit_ms", 0.0)),
})
p99_field = lambda key: percentile([float(row[key]) for row in frames])
max_field = lambda key: max((float(row[key]) for row in frames), default=None)
rebuild_frames = [row for row in frames if row["desired_rebuilt"]]
rebuild_costs = [float(row["desired_rebuild_ms"]) for row in rebuild_frames]
moving_scheduler = [event for event in scheduler.values()]
center_crossings = 0
previous_center = None
for event in sorted(moving_scheduler, key=lambda item: int(item.get("frame", 0))):
center = tuple(event.get("center", ()))
if previous_center is not None and center != previous_center:
center_crossings += 1
previous_center = center
core_requests = [event for event in events
if event.get("type") == "core_region_compile_request"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
same_center_core_requests = sum(
event.get("reason") == "desired_set_changed_same_center" for event in core_requests)
core_desired_queries = [event for event in events
if event.get("type") == "core_desired_cache"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
core_desired_query_ms = [float(event.get("query_ms", 0.0))
for event in core_desired_queries if float(event.get("query_ms", 0.0)) > 0.0]
core_desired_cache_hits = sum(int(event.get("hit", 0)) for event in core_desired_queries)
incremental_audits = [event for event in events
if event.get("type") == "incremental_desired_audit"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
incremental_audit_failures = sum(int(event.get("pass", 1)) == 0
for event in incremental_audits)
order_audits = [event for event in events
if event.get("type") == "incremental_desired_order_audit"
and move_start_s <= float(event.get("t_s", 0.0)) < stop_s]
order_audit_failures = sum(int(event.get("pass", 1)) == 0 for event in order_audits)
materialization_audits = [event for event in incremental_audits
if int(event.get("materialization_audit", 0)) != 0]
if config["audit_every"] > 0 and (
not incremental_audits or incremental_audit_failures
or not order_audits or order_audit_failures
or (config["materialization_audit_every"] > 0
and not materialization_audits)):
raise RuntimeError(
f"incremental desired equality audit missing/failed: {trace}; "
f"set={len(incremental_audits)}/{incremental_audit_failures} "
f"order={len(order_audits)}/{order_audit_failures} "
f"materialization={len(materialization_audits)}")
direction_submissions = [event.get("near_submitted_by_direction", [0, 0, 0, 0])
for event in moving_scheduler]
direction_submissions = [counts for counts in direction_submissions
if isinstance(counts, list) and len(counts) == 4]
direction_submission_totals = [
sum(int(counts[index]) for counts in direction_submissions) for index in range(4)
]
first_admission_order = next((event.get("near_admission_order_first", [])
for event in moving_scheduler
if event.get("near_admission_order_first")), [])
camera_samples = sum(bool(event.get("near_view_forward_cone_valid", 0)) for event in moving)
dwell_complete_s = (float(dwell_complete.get("t_s", 0.0)) - stop_s
if dwell_complete else None)
row = {
"condition": condition,
"repetition": repetition,
"speed_mps": 30.0,
"wait_for_all_critical_tiles": config["wait_for_critical"],
"fast_travel_apply_budget_ms": config["apply_budget_ms"],
"forward_first": config["forward_first"],
"rebuild_on_core_region_completion": config["rebuild_on_region_completion"],
"core_desired_cache_by_center": config["core_desired_cache"],
"incremental_desired_set": config["incremental_desired"],
"incremental_desired_audit_every": config["audit_every"],
"incremental_desired_materialization_audit_every": config["materialization_audit_every"],
"incremental_desired_audit_count": len(incremental_audits),
"incremental_desired_audit_failures": incremental_audit_failures,
"incremental_desired_order_audit_count": len(order_audits),
"incremental_desired_order_audit_failures": order_audit_failures,
"incremental_desired_materialization_audit_count": len(materialization_audits),
"effective_tile_slots": backend.get("effective_tile_slots"),
"exit_code": process.returncode,
"wall_s": round(time.monotonic() - started, 3),
"moving_samples": len(moving),
"dwell_samples": len(dwell),
"arrival_samples": len(arrival),
"near_ready_moving_mean_pct": mean([m["near_ready_pct"] for m in moving_metrics]),
"lod0_ready_moving_mean_pct": mean([m["lod0_ready_pct"] for m in moving_metrics]),
"lod1_ready_moving_mean_pct": mean([m["lod1_ready_pct"] for m in moving_metrics]),
"lod2_ready_moving_mean_pct": mean([m["lod2_ready_pct"] for m in moving_metrics]),
"far_holes_moving_mean_pct": mean([m["far_hole_pct"] for m in moving_metrics]),
"near_ahead_desired_share_pct": mean([m["ahead_share_pct"] for m in moving_metrics
if m["ahead_share_pct"] is not None]),
"near_behind_desired_share_pct": mean([m["behind_share_pct"] for m in moving_metrics
if m["behind_share_pct"] is not None]),
"near_side_desired_share_pct": mean([m["side_share_pct"] for m in moving_metrics]),
"near_ahead_ready_pct": mean([m["ahead_ready_pct"] for m in moving_metrics
if m["ahead_ready_pct"] is not None]),
"near_behind_ready_pct": mean([m["behind_ready_pct"] for m in moving_metrics
if m["behind_ready_pct"] is not None]),
"near_forward_cone_half_angle_deg": 55,
"near_forward_cone_desired_share_pct": mean([m["forward_cone_share_pct"]
for m in moving_metrics
if m["forward_cone_share_pct"] is not None]),
"near_forward_cone_ready_pct": mean([m["forward_cone_ready_pct"]
for m in moving_metrics
if m["forward_cone_ready_pct"] is not None]),
"near_travel_forward_cone_ready_pct": mean(
[m["travel_forward_cone_ready_pct"] for m in moving_metrics
if m["travel_forward_cone_ready_pct"] is not None]),
"near_view_forward_cone_ready_pct": mean(
[m["view_forward_cone_ready_pct"] for m in moving_metrics
if m["view_forward_cone_ready_pct"] is not None]),
"camera_forward_samples": camera_samples,
"camera_forward_sample_share_pct": percent(camera_samples, len(moving)),
"near_forward_cosine_weighted_ready_pct": mean(
[m["forward_weighted_ready_pct"] for m in moving_metrics
if m["forward_weighted_ready_pct"] is not None]),
"arrival_lod02_settled_t_s": (
float(arrival_near_settled.get("t_s", 0.0)) if arrival_near_settled else None),
"near_ready_dwell_end_pct": (100.0 * sum(dwell[-1].get("resident_by_lod", [0] * 9)[:3])
/ max(1, sum(dwell[-1].get("desired_by_lod", [0] * 9)[:3]))),
"far_holes_dwell_end_pct": (100.0 * (
sum(dwell[-1].get("desired_by_lod", [0] * 9)[3:])
- sum(dwell[-1].get("covered_by_lod", [0] * 9)[3:]))
/ max(1, sum(dwell[-1].get("desired_by_lod", [0] * 9)[3:]))),
"dwell_complete_after_stop_s": dwell_complete_s,
"game_thread_tick_p99_ms": p99_field("tick_ms"),
"game_thread_tick_max_ms": max_field("tick_ms"),
"stream_update_p99_ms": p99_field("update_chunks_ms"),
"pending_drain_p99_ms": p99_field("pending_drain_ms"),
"apply_p99_ms": p99_field("apply_ms"),
"apply_max_ms": max_field("apply_ms"),
"apply_budget_min_ms": min((f["apply_budget_ms"] for f in frames), default=None),
"apply_budget_max_ms": max((f["apply_budget_ms"] for f in frames), default=None),
"apply_time_budget_hit_frames": sum(f["time_budget_hit"] for f in frames),
"apply_frames": sum(f["meshes_applied"] > 0 for f in frames),
"apply_meshes_total": sum(f["meshes_applied"] for f in frames),
"desired_rebuild_p99_ms": p99_field("desired_rebuild_ms"),
"desired_rebuild_count": len(rebuild_frames),
"desired_rebuilds_per_second": len(rebuild_frames) / 12.0,
"center_crossings": center_crossings,
"same_center_core_region_requests": same_center_core_requests,
"core_desired_cache_hit_count": core_desired_cache_hits,
"core_desired_cache_hit_share_pct": percent(
core_desired_cache_hits, len(core_desired_queries)),
"core_desired_query_nonzero_p50_ms": statistics.median(core_desired_query_ms)
if core_desired_query_ms else None,
"core_desired_query_nonzero_p95_ms": percentile(core_desired_query_ms, 0.95),
"core_desired_query_nonzero_max_ms": max(core_desired_query_ms)
if core_desired_query_ms else None,
"desired_rebuild_nonzero_p50_ms": statistics.median(rebuild_costs)
if rebuild_costs else None,
"desired_rebuild_nonzero_p95_ms": percentile(rebuild_costs, 0.95),
"desired_rebuild_nonzero_max_ms": max(rebuild_costs) if rebuild_costs else None,
"desired_breakdown": desired_stage_breakdown,
"near_submitted_by_direction": direction_submission_totals,
"near_admission_first12": first_admission_order,
"submit_tiles_p99_ms": p99_field("submit_tiles_ms"),
"level_admit_p99_ms": p99_field("level_admit_ms"),
"top_tick_frames": sorted(frames, key=lambda f: f["tick_ms"], reverse=True)[:5],
"trace": str(trace.relative_to(ROOT)),
"log": str(abslog.relative_to(ROOT)),
}
print("[L167Movement] " + json.dumps(row, sort_keys=True), flush=True)
return row
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--smoke", action="store_true",
help="run one unmeasured base warm-up and validate CVar readback")
parser.add_argument("--resume", action="store_true",
help="resume an incomplete matrix from runs-partial.json")
parser.add_argument("--summarize-existing", action="store_true",
help="summarize 28 completed matrix rows without rerunning UE")
args = parser.parse_args()
missing = [path for path in (EDITOR, HOST / "VFHost.uproject") if not path.exists()]
if missing:
raise SystemExit(f"missing UE runner input: {missing}")
OUT.mkdir(parents=True, exist_ok=True)
partial = OUT / "runs-partial.json"
if args.smoke:
run("base", 0)
return 0
if args.summarize_existing:
if not partial.is_file():
raise RuntimeError(f"no saved movement rows: {partial}")
rows = json.loads(partial.read_text(encoding="utf-8"))
else:
if partial.is_file() and not args.resume:
raise RuntimeError(f"partial matrix exists; use --resume or move it first: {partial}")
rows = json.loads(partial.read_text(encoding="utf-8")) if partial.is_file() else []
if not rows:
# One editor binary is used by every condition, so one warm-up is enough.
run("base", 0)
completed = {(row["repetition"], row["condition"]) for row in rows}
for repetition in range(1, 8):
for condition in ORDER[repetition]:
if (repetition, condition) in completed:
continue
row = run(condition, repetition)
rows.append(row)
partial.write_text(json.dumps(rows, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
metrics = (
"near_ready_moving_mean_pct", "far_holes_moving_mean_pct",
"game_thread_tick_p99_ms", "game_thread_tick_max_ms",
"arrival_lod02_settled_t_s", "dwell_complete_after_stop_s",
"near_ahead_ready_pct", "near_behind_ready_pct",
"near_forward_cone_ready_pct", "near_forward_cosine_weighted_ready_pct",
"apply_p99_ms", "apply_max_ms", "desired_rebuild_nonzero_p95_ms",
"stream_update_p99_ms", "pending_drain_p99_ms", "submit_tiles_p99_ms",
"level_admit_p99_ms", "desired_rebuild_count", "center_crossings",
)
def median_for(condition: str, metric: str) -> float | None:
values = [row[metric] for row in rows
if row["condition"] == condition and row.get(metric) is not None]
return statistics.median(values) if values else None
medians = {
condition: {metric: median_for(condition, metric) for metric in metrics}
for condition in CONDITIONS
}
gates = {}
paired_deltas = {}
for candidate in ("l157", "l160", "both"):
candidate_rows = [row for row in rows if row["condition"] == candidate]
candidate_median_worst = median_for(candidate, "game_thread_tick_max_ms")
base_median_worst = median_for("base", "game_thread_tick_max_ms")
candidate_near = median_for(candidate, "near_ready_moving_mean_pct")
base_near = median_for("base", "near_ready_moving_mean_pct")
candidate_cone = median_for(candidate, "near_forward_cone_ready_pct")
base_cone = median_for("base", "near_forward_cone_ready_pct")
candidate_far = median_for(candidate, "far_holes_moving_mean_pct")
base_far = median_for("base", "far_holes_moving_mean_pct")
candidate_arrival = median_for(candidate, "arrival_lod02_settled_t_s")
base_arrival = median_for("base", "arrival_lod02_settled_t_s")
candidate_gates = {
"seven_runs_complete": len(candidate_rows) == 7,
"every_run_p99_le_11_ms": len(candidate_rows) == 7 and all(
row["game_thread_tick_p99_ms"] <= 11.0 for row in candidate_rows),
"median_worst_tick_no_worse": (
candidate_median_worst is not None and base_median_worst is not None
and candidate_median_worst <= base_median_worst),
"no_run_worst_tick_above_14_ms": len(candidate_rows) == 7 and all(
row["game_thread_tick_max_ms"] <= 14.0 for row in candidate_rows),
"median_near_readiness_up": (
candidate_near is not None and base_near is not None
and candidate_near > base_near),
"median_forward_cone_readiness_up": (
candidate_cone is not None and base_cone is not None
and candidate_cone > base_cone),
"median_far_holes_no_worse": (
candidate_far is not None and base_far is not None
and candidate_far <= base_far),
"median_arrival_no_worse": (
candidate_arrival is not None and base_arrival is not None
and candidate_arrival <= base_arrival),
}
gates[candidate] = {
**candidate_gates,
"passes_default_bar": all(candidate_gates.values()),
}
paired_deltas[candidate] = []
for repetition in range(1, 8):
by_condition = {row["condition"]: row for row in rows
if row["repetition"] == repetition}
if candidate not in by_condition or "base" not in by_condition:
continue
paired_deltas[candidate].append({
"repetition": repetition,
**{f"{metric}_delta": by_condition[candidate][metric]
- by_condition["base"][metric]
for metric in metrics if by_condition[candidate].get(metric) is not None
and by_condition["base"].get(metric) is not None},
})
summary = {
"source_commit": subprocess.run(
["git", "-c", f"safe.directory={ROOT}", "rev-parse", "HEAD"],
cwd=ROOT, check=True, capture_output=True, text=True).stdout.strip(),
"protocol": "Seven interleaved 30 m/s W8 UE runs per condition; 12 s move plus 5 s dwell; 8 effective tile slots; one base warm-up.",
"conditions": CONDITIONS,
"movement_medians": medians,
"candidate_gates": gates,
"paired_deltas_vs_base": paired_deltas,
"runs": rows,
}
(OUT / "summary.json").write_text(
json.dumps(summary, indent=2, sort_keys=True) + "\n", encoding="utf-8")
print("[L167Summary] " + json.dumps({
"movement_medians": medians,
"candidate_gates": gates,
"runs": len(rows),
}, sort_keys=True), flush=True)
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,41 @@
# L168 — Default the incremental desired set
**Result:** `voxel.StreamingIncrementalDesiredSet` now defaults to `1`. In five interleaved 30 m/s UE pairs against `assembler/world` `c6357b5`, candidate tick p99 fell from **10.07 to 7.01 ms** and median worst tick from **13.24 to 11.63 ms**. P99 was lower in all five pairs; worst tick was lower in four, with one +0.06 ms pair.
## Defaults and trace
The main switch gates the Core clipmap delta, overlay membership updates, priority-order merge/reuse, and the incremental path's same-center Core-key cache. Overlay and priority updates have no separate switches. Defaults after the change:
| CVar | Default | Meaning |
|---|---:|---|
| `StreamingIncrementalDesiredSet` | 1 | Core, overlay, and priority updates are incremental. |
| `StreamingCoreDesiredCacheByCenter` | 0 | Separate L160 cache stays off; the incremental path has its own cache. |
| `StreamingRebuildOnCoreRegionCompletion` | 1 | L160 same-center rebuild suppression stays off. |
| `StreamingFastTravelForwardFirst` | 0 | L160 ahead-first stays off. |
| `StreamingFastTravelWaitForAllCriticalTiles` | 1 | L157 early admission stays off. |
| `StreamingFastTravelApplyBudgetMilliseconds` | 0 | L157 fast-travel apply cap stays off. |
| `StreamingIncrementalDesiredAuditEvery` | 0 | Set/order audit stays off. |
| `StreamingIncrementalDesiredMaterializationAuditEvery` | 0 | Materialization audit stays off. |
| `StreamingIncrementalDesiredAuditInjectMismatch` | 0 | Negative-control injection stays off. |
The candidate warm-up was a default 30 m/s run with no streaming-configuration CVar overrides. Readback showed `incremental_desired_set=1`; its 12-second movement trace recorded 40 Core deltas, 313 incremental materialization updates, 168 priority-order updates, and 105 priority-order reuses. Baseline readback was `incremental_desired_set=0`. Every timing run also read back the L157/L160 and audit defaults in the table; only the VFHost movement and trace harness arguments were supplied.
## Measurement and checks
Five interleaved W8 pairs used the four-strata owner plan, a 12-second move plus 5-second dwell, and eight effective streaming slots. Tick metrics below are medians of per-run values over the 12-second movement window. One warm-up per binary was excluded. I discarded an initial provisional matrix because the longer load averages exceeded the quiet gate; the reported matrix began at **0.45 / 0.69 / 1.47** (1/5/15-minute averages), with no build running.
| Metric | `c6357b5` | Default-on | Paired result |
|---|---:|---:|---|
| Tick p99 | 10.07 ms | 7.01 ms | Lower in all 5 pairs |
| Worst tick | 13.24 ms | 11.63 ms | Lower in 4/5; one +0.06 ms pair |
| Near LOD0–2 readiness | 42.33% | 43.73% | +1.40 pp median |
| Moving far holes | 2.04% | 1.85% | −0.19 pp median |
| First settled LOD0–2 | 1.677 s | 1.672 s | 5 ms earlier median |
The separate audit run enabled both audit intervals at 1: **307** desired-set/collision-only comparisons, **307** priority-order comparisons, and **302** materialization comparisons passed; there were zero mismatches. Audit-run metrics are excluded from timing.
`verify.py fast` passed in **301.7 s** (268 pass, 0 fail, 5 diagnostics; 84 binary runs). VFHost Linux Development builds succeeded for the baseline and candidate with `-NoUBA -MaxParallelActions=4`. Runner `py_compile` and `git diff --check` passed. Fast-tier determinism covers W1/W8/W12. UE W12 movement was not run because the same 12-core VFHost affinity exposes eight TaskGraph background workers/effective streaming slots; movement used W8.
Changed Unreal `Source/` files:
- `Source/VoxelForge/Private/VoxelWorld.cpp`
@@ -0,0 +1,301 @@
#!/usr/bin/env python3
"""Five interleaved L168 30 m/s UE pairs, using source-default streaming CVars."""
from __future__ import annotations
import json
import math
import shutil
import statistics
import subprocess
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
HOST = Path("/home/jahni/VFHost")
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
OUT = ROOT / "Saved/Linux/l168/ue/movement"
BIN_DIR = ROOT / "Saved/Linux/l168/bin"
HOST_PLUGIN = HOST / "Binaries/Linux/libUnrealEditor-VoxelForge.so"
SPAWN_M = (19.166015625, -14.2001953125, -753.3740234375)
MOVE_START_S = 6.0
MOVE_DURATION_S = 12.0
CONDITIONS = {
"assembler_world": {
"binary": BIN_DIR / "assembler-world/libUnrealEditor-VoxelForge.so",
"incremental": 0,
},
"incremental_default": {
"binary": BIN_DIR / "incremental-default/libUnrealEditor-VoxelForge.so",
"incremental": 1,
},
}
ORDER = {
1: ("assembler_world", "incremental_default"),
2: ("incremental_default", "assembler_world"),
3: ("assembler_world", "incremental_default"),
4: ("incremental_default", "assembler_world"),
5: ("assembler_world", "incremental_default"),
}
def percentile(values: list[float], fraction: float = 0.99) -> float | None:
if not values:
return None
ordered = sorted(values)
return ordered[max(0, min(len(ordered) - 1, math.ceil(len(ordered) * fraction) - 1))]
def percent(numerator: int | float, denominator: int | float) -> float | None:
return 100.0 * numerator / denominator if denominator > 0 else None
def read_events(path: Path) -> list[dict]:
rows = []
if path.is_file():
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
if not line.startswith("{"):
continue
try:
rows.append(json.loads(line))
except json.JSONDecodeError:
continue
return rows
def summarize_run(condition: str, run_name: str, trace: Path, log: Path,
exit_code: int, wall_s: float, audit: bool = False) -> dict:
expected_incremental = CONDITIONS[condition]["incremental"]
events = read_events(trace)
backend = next((e for e in events if e.get("type") == "worker_backend_config"), {})
scheduler = [e for e in events if e.get("type") == "scheduler_admission"]
if not scheduler:
raise RuntimeError(f"missing scheduler readback: {trace}")
if any(int(e.get("incremental_desired_set", -1)) != expected_incremental for e in scheduler):
raise RuntimeError(f"incremental default readback mismatch: {trace}")
if any(int(e.get("forward_first", -1)) != 0
or int(e.get("rebuild_on_core_region_completion", -1)) != 1
or int(e.get("core_desired_cache_by_center", -1)) != 0
for e in scheduler):
raise RuntimeError(f"L160 control default readback mismatch: {trace}")
if any(int(e.get("incremental_desired_audit_every", -1)) != (1 if audit else 0)
or int(e.get("incremental_desired_materialization_audit_every", -1))
!= (1 if audit else 0)
or int(e.get("incremental_desired_audit_inject_mismatch", -1)) != 0
for e in scheduler):
raise RuntimeError(f"audit CVar readback mismatch: {trace}")
if (int(backend.get("streaming_fast_travel_wait_for_all_critical_tiles", -1)) != 1
or abs(float(backend.get("streaming_fast_travel_apply_budget_ms", -1.0))) > 0.01
or int(backend.get("effective_tile_slots", -1)) != 8):
raise RuntimeError(f"L157/default worker readback mismatch: {trace}; {backend}")
move_stop = next((e for e in events if e.get("type") == "streaming_test_move_stopped"), None)
if move_stop is None:
raise RuntimeError(f"missing move-stop event: {trace}")
stop_s = float(move_stop.get("t_s", 0.0))
samples = [e for e in events if e.get("type") == "streaming_sample"]
moving = [e for e in samples if MOVE_START_S <= float(e.get("t_s", 0.0)) < stop_s
and float(e.get("speed_cm_s", 0.0)) >= 50.0]
dwell = [e for e in samples if float(e.get("t_s", 0.0)) >= stop_s
and float(e.get("speed_cm_s", 0.0)) < 50.0]
if exit_code != 0 or len(moving) < 40 or len(dwell) < 16:
raise RuntimeError(f"incomplete run {run_name}: exit={exit_code}, move={len(moving)}, dwell={len(dwell)}")
move_start = min(float(e.get("t_s", 0.0)) for e in moving)
work = [e for e in events if e.get("type") == "game_thread_work_frame"
and move_start <= float(e.get("t_s", 0.0)) < stop_s]
ticks = [float(e.get("tick_ms", 0.0)) for e in work]
rebuilds = [e for e in events if e.get("type") == "desired_rebuild_breakdown"
and move_start <= float(e.get("t_s", 0.0)) < stop_s]
core_events = [e for e in events if e.get("type") == "core_desired_cache"
and move_start <= float(e.get("t_s", 0.0)) < stop_s]
audits = [e for e in events if e.get("type") == "incremental_desired_audit"
and move_start <= float(e.get("t_s", 0.0)) < stop_s]
order_audits = [e for e in events if e.get("type") == "incremental_desired_order_audit"
and move_start <= float(e.get("t_s", 0.0)) < stop_s]
materialization_audits = [e for e in audits if int(e.get("materialization_audit", 0)) != 0]
if audit:
if (not audits or not order_audits or not materialization_audits
or any(int(e.get("pass", 0)) != 1 for e in audits + order_audits)):
raise RuntimeError(
f"equality audit missing/failed: {trace}; set={len(audits)} "
f"order={len(order_audits)} materialization={len(materialization_audits)}")
near_values = []
far_holes = []
for event in moving:
desired = event.get("desired_by_lod", [0] * 9)
resident = event.get("resident_by_lod", [0] * 9)
covered = event.get("covered_by_lod", resident)
near_values.append(percent(sum(resident[:3]), sum(desired[:3])) or 0.0)
far_holes.append(percent(sum(desired[3:]) - sum(covered[3:]), sum(desired[3:])) or 0.0)
settled = next((e for e in samples if float(e.get("t_s", 0.0)) < move_start
and float(e.get("speed_cm_s", 0.0)) < 50.0
and sum(e.get("settled_by_lod", [0] * 9)[:3])
>= sum(e.get("desired_by_lod", [0] * 9)[:3]) > 0), None)
row = {
"condition": condition,
"run": run_name,
"audit_run": audit,
"source_default_incremental": expected_incremental,
"speed_mps": 30,
"effective_tile_slots": backend.get("effective_tile_slots"),
"moving_samples": len(moving),
"dwell_samples": len(dwell),
"game_thread_tick_p99_ms": percentile(ticks),
"game_thread_tick_worst_ms": max(ticks, default=None),
"near_readiness_moving_mean_pct": statistics.mean(near_values) if near_values else None,
"far_holes_moving_mean_pct": statistics.mean(far_holes) if far_holes else None,
"first_settled_lod02_s": float(settled.get("t_s", 0.0)) if settled else None,
"desired_rebuild_count": len(rebuilds),
"incremental_materialization_update_count": sum(
int(e.get("materialization_update", 0)) != 0 for e in rebuilds),
"priority_order_update_count": sum(
int(e.get("priority_order_updated", 0)) != 0 for e in rebuilds),
"priority_order_reuse_count": sum(
int(e.get("priority_order_reused", 0)) != 0 for e in rebuilds),
"core_incremental_delta_count": sum(int(e.get("incremental_delta", 0)) != 0
for e in core_events),
"core_cache_hit_count": sum(int(e.get("hit", 0)) != 0 for e in core_events),
"set_audits": len(audits),
"order_audits": len(order_audits),
"materialization_audits": len(materialization_audits),
"audit_failures": sum(int(e.get("pass", 1)) != 1 for e in audits + order_audits),
"trace": str(trace.relative_to(ROOT)),
"log": str(log.relative_to(ROOT)),
"wall_s": round(wall_s, 3),
}
if expected_incremental and not audit and (
row["incremental_materialization_update_count"] == 0
or row["core_incremental_delta_count"] == 0):
raise RuntimeError(f"default incremental trace counter stayed zero: {trace}; {row}")
return row
def run(condition: str, run_name: str, audit: bool = False) -> dict:
binary = CONDITIONS[condition]["binary"]
if not binary.is_file():
raise FileNotFoundError(binary)
shutil.copy2(binary, HOST_PLUGIN)
out = OUT / condition
export = HOST / "Plugins/VoxelForge/Saved/Linux/l168" / condition / run_name
userdir = HOST / "Saved/Linux/l168/user" / condition / run_name
zendata = HOST / "Saved/Linux/l168/zen" / condition / run_name
ddc = HOST / "Saved/Linux/l168/DDC"
trace = out / f"{run_name}.trace.jsonl"
log = out / f"{run_name}.log"
stdout = out / f"{run_name}.stdout.log"
for directory in (out, export, userdir, zendata, ddc):
directory.mkdir(parents=True, exist_ok=True)
trace.unlink(missing_ok=True)
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWN_M)
command = [
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
"-notraceserver", "-nocrashreports", "-DisablePlugins=AndroidFileServer",
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
"-archetype=TunnelNetwork", "-slot=0", "-modes=export", "-gameconfig=1",
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0",
"-coreplantest=0", "-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings",
"-corehybridmesher=0", f"-out={export}",
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=3000.0",
f"-voxel.TestMoveStartSeconds={MOVE_START_S:.1f}",
f"-voxel.TestMoveDurationSeconds={MOVE_DURATION_S:.1f}",
"-voxel.TestExitSeconds=0.0", "-voxel.TestMoveDirection=1,0,0",
"-voxel.TestExitOnSteady=0", "-coreworldstreamtrace=1",
f"-voxel.StartupTraceSpawnCm={spawn_cm}",
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
f"-abslog={log}",
]
if audit:
command += ["-voxel.StreamingIncrementalDesiredAuditEvery=1",
"-voxel.StreamingIncrementalDesiredMaterializationAuditEvery=1"]
if any("StreamingIncrementalDesiredSet=" in arg for arg in command):
raise RuntimeError("timing/default proof must not override incremental desired-set CVar")
started = time.monotonic()
with stdout.open("w", encoding="utf-8") as stream:
process = subprocess.run(["taskset", "-c", "0-11", "timeout", "1740s", *command],
cwd=ROOT, stdout=stream, stderr=subprocess.STDOUT,
check=False, timeout=1740)
return summarize_run(condition, run_name, trace, log, process.returncode,
time.monotonic() - started, audit)
def main() -> int:
missing = [p for p in (EDITOR, HOST / "VFHost.uproject", HOST_PLUGIN)
if not p.exists()]
if missing:
raise SystemExit(f"missing UE runner input: {missing}")
OUT.mkdir(parents=True, exist_ok=True)
BIN_DIR.mkdir(parents=True, exist_ok=True)
# One warm-up for each binary, excluded from the paired summary. The candidate warm-up
# is also the no-CVar-override proof that the compiled default executes incremental updates.
warmups = [run("assembler_world", "warmup"),
run("incremental_default", "warmup-default")]
rows = []
partial = OUT / "runs-partial.json"
for pair, sequence in ORDER.items():
for condition in sequence:
rows.append(run(condition, f"pair-{pair}"))
partial.write_text(json.dumps(rows, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
# One audited candidate run, outside the timing matrix.
audit_row = run("incremental_default", "equality-audit", audit=True)
(OUT / "warmups.json").write_text(json.dumps(warmups, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
(OUT / "equality-audit.json").write_text(
json.dumps(audit_row, indent=2, sort_keys=True) + "\n", encoding="utf-8")
metrics = ("game_thread_tick_p99_ms", "game_thread_tick_worst_ms",
"near_readiness_moving_mean_pct", "far_holes_moving_mean_pct",
"first_settled_lod02_s", "incremental_materialization_update_count",
"core_incremental_delta_count")
by_pair = {pair: {row["condition"]: row for row in rows
if row["run"] == f"pair-{pair}"} for pair in ORDER}
medians = {}
for condition in CONDITIONS:
selected = [row for row in rows if row["condition"] == condition]
medians[condition] = {metric: statistics.median(
row[metric] for row in selected if row[metric] is not None) for metric in metrics}
deltas = []
for pair in ORDER:
base = by_pair[pair]["assembler_world"]
candidate = by_pair[pair]["incremental_default"]
deltas.append({"pair": pair, **{f"{m}_delta": candidate[m] - base[m]
for m in metrics
if candidate[m] is not None and base[m] is not None}})
summary = {
"protocol": "Five interleaved 30 m/s UE pairs; 12 s move plus 5 s dwell; 8 effective tile slots; four-strata owner plan. No streaming configuration CVar overrides during warm-up or timing runs. One warm-up per binary excluded. Equality audits run once afterward.",
"default_readback": {
"incremental_desired_set": 1,
"incremental_desired_audit_every": 0,
"incremental_desired_materialization_audit_every": 0,
"incremental_desired_audit_inject_mismatch": 0,
"fast_travel_wait_for_all_critical_tiles": 1,
"fast_travel_apply_budget_ms": 0,
"forward_first": 0,
"rebuild_on_core_region_completion": 1,
"core_desired_cache_by_center": 0,
},
"warmups": warmups,
"movement_medians": medians,
"paired_deltas_candidate_minus_baseline": deltas,
"equality_audit": audit_row,
"runs": rows,
}
(OUT / "summary.json").write_text(json.dumps(summary, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
print("[L168Summary] " + json.dumps({"movement_medians": medians,
"audit": audit_row,
"pairs": len(ORDER)}, sort_keys=True), flush=True)
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,54 @@
# L169 — Early admission stays opt-in
**Result:** Do not promote L157 early admission. Six of seven promotion gates passed, but candidate LOD0–2 arrival was slower in every pair: median **1.683 s** versus **1.671 s** on `assembler/world` `ca0bf76` (+12.4 ms). The source defaults are restored to wait-for-critical **1** and apply-budget **0 ms**. The candidate settings remain explicit overrides in the runner.
## Measurement
Seven interleaved W8 pairs used the four-strata owner plan, 25 cm core voxels, 30 m/s movement, a 12-second move from t=6 to t=18, and a 5-second dwell. Each run had about 46 moving samples; one warm-up per binary was excluded. No product CVar was changed except the two L157 switches under test. At measurement time, the candidate binary compiled those settings as defaults; the runner now supplies the same values explicitly so the experiment can be repeated with shipping defaults off.
| Moving-window metric (median across seven runs) | `ca0bf76` | Candidate | Result |
|---|---:|---:|---:|
| Tick p99 | 7.060 ms | 6.519 ms | lower |
| Worst tick | 12.219 ms | 11.776 ms | lower |
| Near LOD0–2 readiness | 43.28% | 56.12% | +12.84 pp |
| Forward-cone readiness | 15.52% | 17.93% | +2.41 pp |
| Far holes | 1.80% | 0.00% | lower |
| First settled LOD0–2 | 1.671 s | 1.683 s | **12.4 ms slower** |
Every run stayed below 11 ms p99 and 14 ms worst tick. The largest observed p99 was 7.400 ms; the largest worst tick was 13.776 ms. Candidate worst-tick median was not worse. Arrival remained slower after subtracting each run’s first streaming sample (1.305 s vs 1.299 s), so the raw arrival failure is not explained entirely by trace start time.
## Spike count and changes
All counts below use the 12-second moving window unless identified as a startup diagnostic.
- **Level sorting:** With L157 on at the base defaults, the largest full-sort sample was 0.865 ms and no sample exceeded 1 ms; the L167 8.3 ms spike did not recur. Candidate partial sorting reduced the largest sample to 0.123 ms, with no sample above 1 ms.
- **Desired rebuild and overlay:** Base per-run desired-rebuild peaks had a 7.476 ms median and reached 8.110 ms. Candidate peaks had a 6.928 ms median and reached 7.260 ms; rebuilds over 5 ms still occurred (median four per run, maximum five). Overlay materialization never exceeded 2 ms; its candidate maximum was 1.094 ms. The transition descendant index now uses sorted arrays and binary-search deltas. In a separate pre-move startup trace with 2,233 Core keys, the initial index build took 1.605 ms; the maximum moving-window index phase was 3.626 ms.
- **Mesh apply:** The 2.8 ms historical apply spike did not recur. Maximum per-frame apply time fell from 2.548 ms at base to 1.590 ms candidate; largest individual candidate apply was 1.512 ms. The preflight deferred 21–145 results per run (median 109), with a largest estimate of 3.447 ms for 8,979 triangles. RealtimeMesh submission remains atomic, so the estimate is a heuristic and does not split a single upload.
The cold-start arrival gate prevents default promotion even though movement readiness and tick behavior improved. No further adjustment was made after the seven-pair result.
## Defaults and checks
| CVar | Final default | Candidate test value |
|---|---:|---:|
| `voxel.StreamingFastTravelWaitForAllCriticalTiles` | 1 | 0 |
| `voxel.StreamingFastTravelApplyBudgetMilliseconds` | 0 ms | 1 ms |
| `voxel.StreamingIncrementalDesiredSet` | 1 | 1 |
| `voxel.StreamingFastTravelForwardFirst` | 0 | 0 |
| `voxel.StreamingRebuildOnCoreRegionCompletion` | 1 | 1 |
| `voxel.StreamingCoreDesiredCacheByCenter` | 0 | 0 |
| `voxel.StreamingIncrementalDesiredAuditEvery` | 0 | 0 |
| `voxel.StreamingIncrementalDesiredMaterializationAuditEvery` | 0 | 0 |
| `voxel.StreamingIncrementalDesiredAuditInjectMismatch` | 0 | 0 |
- Final VFHost Linux Development build passed with `-NoUBA -MaxParallelActions=4`; three existing deprecated RealtimeMesh API warnings remain.
- Final-source `verify.py fast` passed in 289.8 s: 268 passes, 0 failures, 5 diagnostics. Clang and GCC geometry hashes and W1/W8/W12 determinism passed.
- `git diff --check` and the runner AST syntax check passed. The direct live `TileHashDump` hook returned zero records for the Core path on both binaries and is inconclusive; the fast-tier geometry and determinism checks passed.
- UE W12 movement was not run: this VFHost procedure exposed eight effective tile slots. Fast-tier W12 determinism was run.
Remaining risk: desired rebuilds still reach 7.260 ms in the candidate, and the apply estimate cannot bound an atomic RMC upload. Most importantly, the arrival gate failed, so early admission remains opt-in.
Changed Unreal `Source/` files:
- `Source/VoxelForge/Private/VoxelWorld.cpp`
- `Source/VoxelForge/Public/VoxelWorld.h`
@@ -0,0 +1,442 @@
#!/usr/bin/env python3
"""L169 default-based UE movement comparison and early-admission spike trace."""
from __future__ import annotations
import argparse
import json
import math
import shutil
import statistics
import subprocess
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
HOST = Path("/home/jahni/VFHost")
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
HOST_PLUGIN = HOST / "Binaries/Linux/libUnrealEditor-VoxelForge.so"
OUT = ROOT / "Saved/Linux/l169/ue/movement"
BIN_DIR = ROOT / "Saved/Linux/l169/bin"
SPAWN_M = (19.166015625, -14.2001953125, -753.3740234375)
MOVE_START_S = 6.0
MOVE_DURATION_S = 12.0
BASELINE_BINARY = BIN_DIR / "ca0bf76/libUnrealEditor-VoxelForge.so"
CANDIDATE_BINARY = BIN_DIR / "early-admission/libUnrealEditor-VoxelForge.so"
DIAGNOSTIC_BINARY = BIN_DIR / "apply-preflight/libUnrealEditor-VoxelForge.so"
EXPECTED = {
"baseline": {"wait": 1, "apply_budget_ms": 0.0, "incremental": 1},
"early_override": {"wait": 0, "apply_budget_ms": 1.0, "incremental": 1},
"candidate": {"wait": 0, "apply_budget_ms": 1.0, "incremental": 1},
}
ORDER = {
1: ("baseline", "candidate"),
2: ("candidate", "baseline"),
3: ("baseline", "candidate"),
4: ("candidate", "baseline"),
5: ("baseline", "candidate"),
6: ("candidate", "baseline"),
7: ("baseline", "candidate"),
}
def percentile(values: list[float], fraction: float = 0.99) -> float | None:
if not values:
return None
ordered = sorted(values)
index = max(0, min(len(ordered) - 1, math.ceil(len(ordered) * fraction) - 1))
return ordered[index]
def percent(numerator: int | float, denominator: int | float) -> float | None:
return 100.0 * numerator / denominator if denominator > 0 else None
def read_events(path: Path) -> list[dict]:
rows = []
if path.is_file():
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
if not line.startswith("{"):
continue
try:
rows.append(json.loads(line))
except json.JSONDecodeError:
continue
return rows
def summarize(condition: str, run_name: str, trace: Path, log: Path, exit_code: int,
wall_s: float, audit_or_diag: bool = False) -> dict:
expected = EXPECTED[condition]
events = read_events(trace)
backend = next((e for e in events if e.get("type") == "worker_backend_config"), {})
scheduler_all = [e for e in events if e.get("type") == "scheduler_admission"]
if not scheduler_all:
raise RuntimeError(f"missing scheduler readback: {trace}")
if any(int(e.get("incremental_desired_set", -1)) != expected["incremental"]
for e in scheduler_all):
raise RuntimeError(f"incremental desired-set readback mismatch: {trace}")
if any(int(e.get("forward_first", -1)) != 0
or int(e.get("rebuild_on_core_region_completion", -1)) != 1
or int(e.get("core_desired_cache_by_center", -1)) != 0
or int(e.get("incremental_desired_audit_every", -1)) != 0
or int(e.get("incremental_desired_materialization_audit_every", -1)) != 0
or int(e.get("incremental_desired_audit_inject_mismatch", -1)) != 0
for e in scheduler_all):
raise RuntimeError(f"non-default L160/audit CVar readback: {trace}")
if (int(backend.get("streaming_fast_travel_wait_for_all_critical_tiles", -1))
!= expected["wait"]
or abs(float(backend.get("streaming_fast_travel_apply_budget_ms", -999.0))
- expected["apply_budget_ms"]) > 0.01
or int(backend.get("effective_tile_slots", -1)) != 8):
raise RuntimeError(f"early-admission/worker readback mismatch: {trace}; {backend}")
move_stop = next((e for e in events if e.get("type") == "streaming_test_move_stopped"), None)
if move_stop is None:
raise RuntimeError(f"missing move-stop event: {trace}")
stop_s = float(move_stop.get("t_s", 0.0))
samples = [e for e in events if e.get("type") == "streaming_sample"]
moving = [e for e in samples if MOVE_START_S <= float(e.get("t_s", 0.0)) < stop_s
and float(e.get("speed_cm_s", 0.0)) >= 50.0]
dwell = [e for e in samples if float(e.get("t_s", 0.0)) >= stop_s
and float(e.get("speed_cm_s", 0.0)) < 50.0]
if exit_code != 0 or len(moving) < 40 or len(dwell) < 16:
raise RuntimeError(f"incomplete {run_name}: exit={exit_code}, move={len(moving)}, "
f"dwell={len(dwell)}")
move_start = min(float(e.get("t_s", 0.0)) for e in moving)
work = [e for e in events if e.get("type") == "game_thread_work_frame"
and move_start <= float(e.get("t_s", 0.0)) < stop_s]
scheduler_moving = [e for e in scheduler_all
if move_start <= float(e.get("t_s", 0.0)) < stop_s]
rebuild_events = [e for e in events if e.get("type") == "desired_rebuild_breakdown"
and move_start <= float(e.get("t_s", 0.0)) < stop_s]
applies = {int(e.get("frame", -1)): e for e in events
if e.get("type") == "apply_frame"
and move_start <= float(e.get("t_s", 0.0)) < stop_s}
scheduler = {int(e.get("frame", -1)): e for e in scheduler_all
if move_start <= float(e.get("t_s", 0.0)) < stop_s}
desired = {int(e.get("frame", -1)): e for e in events
if e.get("type") == "desired_rebuild_breakdown"
and move_start <= float(e.get("t_s", 0.0)) < stop_s}
moving_tiles = [e for e in events if e.get("type") == "tile"
and move_start <= float(e.get("t_s", 0.0)) < stop_s]
apply_deferrals = [e for e in events if e.get("type") == "tile_apply_deferred"
and move_start <= float(e.get("t_s", 0.0)) < stop_s]
frames = []
for event in work:
frame = int(event.get("frame", -1))
apply = applies.get(frame, {})
admit = scheduler.get(frame, {})
rebuild = desired.get(frame, {})
frames.append({
"frame": frame,
"tick_ms": float(event.get("tick_ms", 0.0)),
"update_chunks_ms": float(event.get("update_chunks_ms", 0.0)),
"pending_drain_ms": float(event.get("pending_drain_ms", 0.0)),
"apply_ms": float(apply.get("apply_ms", 0.0)),
"apply_budget_ms": float(apply.get("apply_budget_ms", 0.0)),
"meshes_applied": int(apply.get("meshes_applied", 0)),
"time_budget_hit": int(apply.get("time_limit_reached", 0)),
"desired_rebuild_ms": float(admit.get("desired_rebuild_ms", 0.0)),
"desired_rebuild_event_ms": float(rebuild.get("total_ms", 0.0)),
"overlay_materialize_ms": float(rebuild.get("overlay_materialize_ms", 0.0)),
"core_prepare_ms": float(rebuild.get("core_prepare_ms", 0.0)),
"core_delta_ms": float(rebuild.get("core_delta_ms", 0.0)),
"priority_sort_ms": float(rebuild.get("priority_sort_ms", 0.0)),
"critical_order_ms": float(rebuild.get("critical_order_ms", 0.0)),
"collision_stamp_ms": float(rebuild.get("collision_stamp_ms", 0.0)),
"transition_index_ms": float(rebuild.get("transition_index_ms", 0.0)),
"retained_region_ms": float(rebuild.get("retained_region_ms", 0.0)),
"region_reconcile_ms": float(rebuild.get("region_reconcile_ms", 0.0)),
"level_sort_ms": float(admit.get("level_sort_ms", 0.0)),
"level_candidate_scan_ms": float(admit.get("level_candidate_scan_ms", 0.0)),
"submit_tiles_ms": float(admit.get("submit_tiles_ms", 0.0)),
})
near = []
cone = []
far_holes = []
for event in moving:
desired_by_lod = event.get("desired_by_lod", [0] * 9)
resident_by_lod = event.get("resident_by_lod", [0] * 9)
covered_by_lod = event.get("covered_by_lod", resident_by_lod)
near.append(percent(sum(resident_by_lod[:3]), sum(desired_by_lod[:3])) or 0.0)
cone.append(percent(int(event.get("near_forward_cone_resident", 0)),
int(event.get("near_forward_cone_desired", 0))) or 0.0)
far_holes.append(percent(sum(desired_by_lod[3:]) - sum(covered_by_lod[3:]),
sum(desired_by_lod[3:])) or 0.0)
settled = next((e for e in samples if float(e.get("t_s", 0.0)) < move_start
and float(e.get("speed_cm_s", 0.0)) < 50.0
and sum(e.get("settled_by_lod", [0] * 9)[:3])
>= sum(e.get("desired_by_lod", [0] * 9)[:3]) > 0), None)
p99 = percentile([r["tick_ms"] for r in frames])
worst = max(frames, key=lambda r: r["tick_ms"], default={})
row = {
"condition": condition,
"run": run_name,
"diagnostic_override": audit_or_diag,
"source_default_incremental": expected["incremental"],
"early_admission_wait_for_critical": expected["wait"],
"early_admission_apply_budget_ms": expected["apply_budget_ms"],
"speed_mps": 30,
"effective_tile_slots": backend.get("effective_tile_slots"),
"moving_samples": len(moving),
"dwell_samples": len(dwell),
"tick_p99_ms": p99,
"tick_worst_ms": worst.get("tick_ms"),
"worst_frame": worst,
"near_readiness_moving_mean_pct": statistics.mean(near) if near else None,
"forward_cone_readiness_moving_mean_pct": statistics.mean(cone) if cone else None,
"far_holes_moving_median_pct": statistics.median(far_holes) if far_holes else None,
"arrival_lod02_settled_s": float(settled.get("t_s", 0.0)) if settled else None,
"desired_rebuild_count": len(rebuild_events),
"incremental_materialization_updates": sum(
int(e.get("materialization_update", 0)) != 0 for e in events
if e.get("type") == "desired_rebuild_breakdown"
and move_start <= float(e.get("t_s", 0.0)) < stop_s),
"core_incremental_deltas": sum(
int(e.get("incremental_delta", 0)) != 0 for e in events
if e.get("type") == "core_desired_cache"
and move_start <= float(e.get("t_s", 0.0)) < stop_s),
"level_sort_samples_over_1ms": sum(
float(e.get("level_sort_ms", 0.0)) > 1.0 for e in scheduler_moving),
"level_sort_max_ms": max((float(e.get("level_sort_ms", 0.0))
for e in scheduler_moving), default=0.0),
"desired_rebuild_events_over_5ms": sum(
float(e.get("total_ms", 0.0)) > 5.0 for e in rebuild_events),
"desired_rebuild_max_ms": max(
(float(e.get("total_ms", 0.0)) for e in rebuild_events), default=0.0),
"overlay_materialize_events_over_2ms": sum(
float(e.get("overlay_materialize_ms", 0.0)) > 2.0 for e in rebuild_events),
"overlay_materialize_max_ms": max(
(float(e.get("overlay_materialize_ms", 0.0)) for e in rebuild_events), default=0.0),
"desired_subphase_max_ms": {
key: max((float(e.get(key, 0.0)) for e in rebuild_events), default=0.0)
for key in ("core_prepare_ms", "overlay_materialize_ms", "priority_sort_ms",
"critical_order_ms", "collision_stamp_ms", "transition_index_ms",
"retained_region_ms", "region_reconcile_ms")},
"apply_budget_hit_frames": sum(f["time_budget_hit"] for f in frames),
"apply_frame_count": sum(f["meshes_applied"] > 0 for f in frames),
"apply_ms_max": max((f["apply_ms"] for f in frames), default=0.0),
"tiles_over_1ms_apply": sum(float(e.get("apply_s", 0.0)) > 0.001
for e in moving_tiles),
"apply_preflight_deferred_count": len(apply_deferrals),
"apply_preflight_deferred_max_estimate_ms": max(
(float(e.get("estimated_apply_ms", 0.0)) for e in apply_deferrals), default=0.0),
"apply_preflight_deferred_max_triangles": max(
(int(e.get("triangles", 0)) for e in apply_deferrals), default=0),
"largest_tile_applies": sorted(({
"apply_ms": float(e.get("apply_s", 0.0)) * 1000.0,
"triangles": int(e.get("triangles", 0)),
"vertices": int(e.get("vertices", 0)),
"level": int(e.get("level", -1)),
} for e in moving_tiles),
key=lambda e: e["apply_ms"], reverse=True)[:8],
"top_ticks": sorted(frames, key=lambda r: r["tick_ms"], reverse=True)[:8],
"trace": str(trace.relative_to(ROOT)),
"log": str(log.relative_to(ROOT)),
"wall_s": round(wall_s, 3),
}
return row
def run(condition: str, run_name: str, diagnostic_override: bool = False,
tile_hash_dump: Path | None = None) -> dict:
if condition == "baseline":
binary = BASELINE_BINARY
elif condition == "early_override":
binary = DIAGNOSTIC_BINARY if DIAGNOSTIC_BINARY.is_file() else BASELINE_BINARY
elif condition == "candidate":
binary = CANDIDATE_BINARY
else:
raise ValueError(condition)
if not binary.is_file():
raise FileNotFoundError(binary)
HOST_PLUGIN.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(binary, HOST_PLUGIN)
out = OUT / condition
export = HOST / "Plugins/VoxelForge/Saved/Linux/l169" / condition / run_name
userdir = HOST / "Saved/Linux/l169/user" / condition / run_name
zendata = HOST / "Saved/Linux/l169/zen" / condition / run_name
ddc = HOST / "Saved/Linux/l169/DDC"
trace = out / f"{run_name}.trace.jsonl"
log = out / f"{run_name}.log"
stdout = out / f"{run_name}.stdout.log"
for directory in (out, export, userdir, zendata, ddc):
directory.mkdir(parents=True, exist_ok=True)
trace.unlink(missing_ok=True)
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWN_M)
command = [
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
"-notraceserver", "-nocrashreports", "-DisablePlugins=AndroidFileServer",
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
"-archetype=TunnelNetwork", "-slot=0", "-modes=export", "-gameconfig=1",
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0",
"-coreplantest=0", "-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings",
"-corehybridmesher=0", f"-out={export}",
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=3000.0",
f"-voxel.TestMoveStartSeconds={MOVE_START_S:.1f}",
f"-voxel.TestMoveDurationSeconds={MOVE_DURATION_S:.1f}",
"-voxel.TestExitSeconds=0.0", "-voxel.TestMoveDirection=1,0,0",
"-voxel.TestExitOnSteady=0", "-coreworldstreamtrace=1",
f"-voxel.StartupTraceSpawnCm={spawn_cm}",
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
f"-abslog={log}",
]
if condition in ("early_override", "candidate"):
command += ["-voxel.StreamingFastTravelWaitForAllCriticalTiles=0",
"-voxel.StreamingFastTravelApplyBudgetMilliseconds=1.0"]
if tile_hash_dump is not None:
tile_hash_dump.parent.mkdir(parents=True, exist_ok=True)
tile_hash_dump.unlink(missing_ok=True)
command.append(f"-voxel.TileHashDump={tile_hash_dump.resolve()}")
started = time.monotonic()
with stdout.open("w", encoding="utf-8") as stream:
process = subprocess.run(
["taskset", "-c", "0-11", "timeout", "1740s", *command],
cwd=ROOT, stdout=stream, stderr=subprocess.STDOUT, check=False, timeout=1740)
return summarize(condition, run_name, trace, log, process.returncode,
time.monotonic() - started, diagnostic_override)
def main() -> int:
parser = argparse.ArgumentParser()
modes = parser.add_mutually_exclusive_group(required=True)
modes.add_argument("--diagnostic", action="store_true",
help="one early-admission trace on the selected diagnostic binary")
modes.add_argument("--matrix", action="store_true",
help="warm both binaries, then run seven interleaved pairs with only the early-admission switches overridden")
modes.add_argument("--geometry-hash-check", action="store_true",
help="compare shared tile hashes with defaults vs the promoted switches")
args = parser.parse_args()
missing = [p for p in (EDITOR, HOST / "VFHost.uproject", ROOT / "VoxelForge.uplugin")
if not p.exists()]
if missing:
raise SystemExit(f"missing UE runner input: {missing}")
OUT.mkdir(parents=True, exist_ok=True)
if args.diagnostic:
warmup = run("early_override", "diagnostic-warmup", diagnostic_override=True)
row = run("early_override", "l169-default-base-diagnostic", diagnostic_override=True)
row["warmup"] = warmup
path = OUT / "diagnostic.json"
path.write_text(json.dumps(row, indent=2, sort_keys=True) + "\n", encoding="utf-8")
print("[L169Diagnostic] " + json.dumps(row, sort_keys=True), flush=True)
return 0
if args.geometry_hash_check:
hash_dir = OUT / "geometry-hash"
baseline_dump = hash_dir / "baseline.json"
candidate_dump = hash_dir / "candidate.json"
run("baseline", "geometry-hash", tile_hash_dump=baseline_dump)
run("candidate", "geometry-hash", tile_hash_dump=candidate_dump)
baseline_data = json.loads(baseline_dump.read_text(encoding="utf-8"))
candidate_data = json.loads(candidate_dump.read_text(encoding="utf-8"))
key_fields = ("level", "tile_x", "tile_y", "tile_z")
def indexed(data: dict) -> dict[tuple[int, ...], tuple]:
return {tuple(int(record[field]) for field in key_fields): (
str(record.get("geometry_hash", "")),
int(record.get("triangle_count", -1)),
bool(record.get("empty", False)))
for record in data.get("records", [])}
base = indexed(baseline_data)
candidate = indexed(candidate_data)
common = base.keys() & candidate.keys()
mismatches = [key for key in common if base[key] != candidate[key]]
result = {"baseline_records": len(base), "candidate_records": len(candidate),
"common_records": len(common), "hash_mismatches": len(mismatches),
"baseline_only_records": len(base.keys() - candidate.keys()),
"candidate_only_records": len(candidate.keys() - base.keys()),
"baseline_dump": str(baseline_dump.relative_to(ROOT)),
"candidate_dump": str(candidate_dump.relative_to(ROOT))}
(hash_dir / "comparison.json").write_text(
json.dumps(result, indent=2, sort_keys=True) + "\n", encoding="utf-8")
print("[L169GeometryHash] " + json.dumps(result, sort_keys=True), flush=True)
return 0 if len(common) >= 100 and not mismatches else 1
warmups = [run("baseline", "warmup"), run("candidate", "warmup")]
rows = []
partial = OUT / "runs-partial.json"
for pair, sequence in ORDER.items():
for condition in sequence:
rows.append(run(condition, f"pair-{pair}"))
partial.write_text(json.dumps(rows, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
metrics = ("tick_p99_ms", "tick_worst_ms", "near_readiness_moving_mean_pct",
"forward_cone_readiness_moving_mean_pct", "far_holes_moving_median_pct",
"arrival_lod02_settled_s")
medians = {condition: {metric: statistics.median(
row[metric] for row in rows if row["condition"] == condition
and row.get(metric) is not None) for metric in metrics}
for condition in ("baseline", "candidate")}
by_pair = {pair: {row["condition"]: row for row in rows if row["run"] == f"pair-{pair}"}
for pair in ORDER}
deltas = [
{"pair": pair, **{f"{metric}_delta":
by_pair[pair]["candidate"][metric] - by_pair[pair]["baseline"][metric]
for metric in metrics}}
for pair in ORDER
]
summary = {"protocol": "Seven interleaved 30 m/s UE pairs using base defaults except for "
"the two early-admission switches at candidate values; 12 s move plus 5 s "
"dwell; eight effective tile slots; one warm-up per binary excluded.",
"defaults": {"incremental_desired_set": 1,
"streaming_fast_travel_wait_for_all_critical_tiles": 1,
"streaming_fast_travel_apply_budget_ms": 0.0,
"forward_first": 0, "rebuild_on_core_region_completion": 1,
"core_desired_cache_by_center": 0,
"incremental_desired_audit_every": 0,
"incremental_desired_materialization_audit_every": 0,
"incremental_desired_audit_inject_mismatch": 0},
"candidate_switch_overrides": {
"streaming_fast_travel_wait_for_all_critical_tiles": 0,
"streaming_fast_travel_apply_budget_ms": 1.0},
"warmups": warmups, "movement_medians": medians,
"paired_deltas_candidate_minus_baseline": deltas, "runs": rows}
(OUT / "summary.json").write_text(json.dumps(summary, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
print("[L169Summary] " + json.dumps({"movement_medians": medians,
"paired_deltas": deltas}, sort_keys=True), flush=True)
all_runs_p99_pass = all(row["tick_p99_ms"] is not None and row["tick_p99_ms"] <= 11.0
for row in rows)
all_runs_worst_pass = all(row["tick_worst_ms"] is not None
and row["tick_worst_ms"] <= 14.0 for row in rows)
candidate_worst = [row["tick_worst_ms"] for row in rows
if row["condition"] == "candidate"]
baseline_worst = [row["tick_worst_ms"] for row in rows
if row["condition"] == "baseline"]
gates = {
"every_run_p99_le_11_ms": all_runs_p99_pass,
"every_run_worst_le_14_ms": all_runs_worst_pass,
"candidate_worst_tick_median_not_worse": bool(candidate_worst and baseline_worst)
and statistics.median(candidate_worst) <= statistics.median(baseline_worst),
"near_readiness_up": medians["candidate"]["near_readiness_moving_mean_pct"]
> medians["baseline"]["near_readiness_moving_mean_pct"],
"forward_cone_readiness_up": medians["candidate"][
"forward_cone_readiness_moving_mean_pct"]
> medians["baseline"]["forward_cone_readiness_moving_mean_pct"],
"far_holes_not_worse": medians["candidate"]["far_holes_moving_median_pct"]
<= medians["baseline"]["far_holes_moving_median_pct"],
"arrival_not_worse": medians["candidate"]["arrival_lod02_settled_s"]
<= medians["baseline"]["arrival_lod02_settled_s"],
}
summary["promotion_gates"] = gates
(OUT / "summary.json").write_text(json.dumps(summary, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
print("[L169Gates] " + json.dumps(gates, sort_keys=True), flush=True)
return 0 if all(gates.values()) else 1
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,55 @@
# L170 — Gate early admission on settled arrival
**Result:** The initial-arrival latch works: fast-travel admission and its 1 ms apply budget stayed inactive until the LOD0–2 desired set and required collision set settled, then became active during 30 m/s movement. Promotion still fails the arrival gate, so the source defaults remain off. The candidate was 1.282% slower on the per-condition arrival medians and was slower in four of seven pairs.
## First-arrival census
The UE matrix used seven interleaved W8 pairs against `assembler/world` `ca0bf76`, 30 m/s, 12 seconds of movement plus 5 seconds of dwell, and one excluded warm-up per binary. The candidate binary was compiled with the two early-admission defaults on; there were no CVar command-line overrides. Its raw first-settled LOD0–2 median was 1.691450 s versus 1.670039 s for baseline (+21.411 ms, +1.282%). The median paired delta was +12.176 ms; the candidate was not slower in only three pairs. Times from the first streaming sample to settlement had medians 1.2965 s and 1.3132 s respectively.
I counted startup-window trace timers from each run's first streaming sample through its first settled LOD0–2 sample. Candidate fast-travel mode was active in zero pre-settlement scheduler samples, and its apply budget remained at the normal 2 ms. During that same window:
- Level candidate scanning summed to 2.896 ms baseline and 3.758 ms candidate (paired median delta +0.895 ms). Partial level sorting fell from 2.195 ms to 0.727 ms (−1.452 ms paired median).
- Desired rebuild totals were effectively unchanged: 3.046 ms versus 3.047 ms per window (paired median delta −0.035 ms). The candidate spent 0.149 ms in its transition-index phase; the baseline has no matching phase.
- Result-drain time was 127.986 ms versus 131.521 ms (paired median delta +5.001 ms). Apply time was 121.509 ms versus 125.095 ms (+5.416 ms paired median), while the candidate had applied 10 fewer meshes by settlement at the paired median.
These timers locate the largest measured difference in result draining and apply work, but they do not isolate why that work costs more; some phase timers may overlap and should not be added. The early-admission switches themselves did no work before settlement, so the census does not attribute the remaining arrival regression to those switches.
## Movement results
Medians over the seven 30 m/s moving windows:
| Metric | `ca0bf76` | Candidate | Change |
|---|---:|---:|---:|
| Near LOD0–2 readiness | 43.65% | 55.46% | +11.82 pp |
| Forward-cone readiness | 15.79% | 18.69% | +2.90 pp |
| Far holes | 1.80% | 0.00% | −1.80 pp |
| Tick p99 | 7.214 ms | 6.476 ms | lower |
| Worst tick | 12.315 ms | 12.272 ms | lower |
All runs met p99 ≤ 11 ms and worst tick ≤ 14 ms; the largest observed values across both conditions were 7.281 ms and 12.881 ms. Candidate fast-travel mode and the 1 ms budget were active in every moving run. The movement gains met the L169-relative gates; only the two arrival gates failed.
## Defaults and checks
| CVar | Final source default | Candidate test default |
|---|---:|---:|
| `voxel.StreamingFastTravelWaitForAllCriticalTiles` | 1 | 0 |
| `voxel.StreamingFastTravelApplyBudgetMilliseconds` | 0 ms | 1 ms |
| `voxel.StreamingIncrementalDesiredSet` | 1 | 1 |
| `voxel.StreamingFastTravelForwardFirst` | 0 | 0 |
| `voxel.StreamingRebuildOnCoreRegionCompletion` | 1 | 1 |
| `voxel.StreamingCoreDesiredCacheByCenter` | 0 | 0 |
| `voxel.StreamingIncrementalDesiredAuditEvery` | 0 | 0 |
| `voxel.StreamingIncrementalDesiredMaterializationAuditEvery` | 0 | 0 |
| `voxel.StreamingIncrementalDesiredAuditInjectMismatch` | 0 | 0 |
- VFHost UE Linux Development builds passed for the candidate-on measurement binary and final defaults-off source (`-NoUBA -MaxParallelActions=4`); three existing deprecated RealtimeMesh API warnings remain.
- `Standalone/Tools/verify/verify.py fast` passed on the final source in 280.7 s: 268 passes, zero failures, five non-gating hash-baseline diagnostics. Worker-count determinism checks passed.
- The L170 movement runner completed all seven pairs. Its exit was nonzero only because the two arrival promotion gates failed. Runner AST parsing and `git diff --check` passed.
- UE W12 movement timing was not run; the VFHost procedure reported eight effective tile slots. Fast-tier W1/W8/W12 determinism checks ran.
No geometry or evaluator code changed. Remaining risk is the unexplained arrival regression in result-drain/apply work; promotion should wait for a measurement that isolates that cost.
Changed Unreal `Source/` files:
- `Source/VoxelForge/Private/VoxelWorld.cpp`
- `Source/VoxelForge/Public/VoxelWorld.h`
@@ -0,0 +1,490 @@
#!/usr/bin/env python3
"""L170 compiled-default UE movement comparison and startup-gate trace."""
from __future__ import annotations
import argparse
import json
import math
import shutil
import statistics
import subprocess
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
HOST = Path("/home/jahni/VFHost")
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
HOST_PLUGIN = HOST / "Binaries/Linux/libUnrealEditor-VoxelForge.so"
OUT = ROOT / "Saved/Linux/l170/ue/movement"
BIN_DIR = ROOT / "Saved/Linux/l170/bin"
SPAWN_M = (19.166015625, -14.2001953125, -753.3740234375)
MOVE_START_S = 6.0
MOVE_DURATION_S = 12.0
BASELINE_BINARY = ROOT / "Saved/Linux/l169/bin/ca0bf76/libUnrealEditor-VoxelForge.so"
CANDIDATE_BINARY = BIN_DIR / "candidate-default-on/libUnrealEditor-VoxelForge.so"
DIAGNOSTIC_BINARY = CANDIDATE_BINARY
EXPECTED = {
"baseline": {"wait": 1, "apply_budget_ms": 0.0, "incremental": 1},
"candidate": {"wait": 0, "apply_budget_ms": 1.0, "incremental": 1},
}
ORDER = {
1: ("baseline", "candidate"),
2: ("candidate", "baseline"),
3: ("baseline", "candidate"),
4: ("candidate", "baseline"),
5: ("baseline", "candidate"),
6: ("candidate", "baseline"),
7: ("baseline", "candidate"),
}
def percentile(values: list[float], fraction: float = 0.99) -> float | None:
if not values:
return None
ordered = sorted(values)
index = max(0, min(len(ordered) - 1, math.ceil(len(ordered) * fraction) - 1))
return ordered[index]
def percent(numerator: int | float, denominator: int | float) -> float | None:
return 100.0 * numerator / denominator if denominator > 0 else None
def read_events(path: Path) -> list[dict]:
rows = []
if path.is_file():
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
if not line.startswith("{"):
continue
try:
rows.append(json.loads(line))
except json.JSONDecodeError:
continue
return rows
def summarize(condition: str, run_name: str, trace: Path, log: Path, exit_code: int,
wall_s: float, audit_or_diag: bool = False) -> dict:
expected = EXPECTED[condition]
events = read_events(trace)
backend = next((e for e in events if e.get("type") == "worker_backend_config"), {})
scheduler_all = [e for e in events if e.get("type") == "scheduler_admission"]
if not scheduler_all:
raise RuntimeError(f"missing scheduler readback: {trace}")
if any(int(e.get("incremental_desired_set", -1)) != expected["incremental"]
for e in scheduler_all):
raise RuntimeError(f"incremental desired-set readback mismatch: {trace}")
if any(int(e.get("forward_first", -1)) != 0
or int(e.get("rebuild_on_core_region_completion", -1)) != 1
or int(e.get("core_desired_cache_by_center", -1)) != 0
or int(e.get("incremental_desired_audit_every", -1)) != 0
or int(e.get("incremental_desired_materialization_audit_every", -1)) != 0
or int(e.get("incremental_desired_audit_inject_mismatch", -1)) != 0
for e in scheduler_all):
raise RuntimeError(f"non-default L160/audit CVar readback: {trace}")
if (int(backend.get("streaming_fast_travel_wait_for_all_critical_tiles", -1))
!= expected["wait"]
or abs(float(backend.get("streaming_fast_travel_apply_budget_ms", -999.0))
- expected["apply_budget_ms"]) > 0.01
or int(backend.get("effective_tile_slots", -1)) != 8):
raise RuntimeError(f"early-admission/worker readback mismatch: {trace}; {backend}")
if condition == "candidate" and any(
int(e.get("wait_for_all_critical_tiles", -1)) != expected["wait"]
or "initial_arrival_set_settled" not in e
or "fast_travel_mode" not in e
for e in scheduler_all):
raise RuntimeError(f"missing L170 arrival-gate readback: {trace}")
move_stop = next((e for e in events if e.get("type") == "streaming_test_move_stopped"), None)
if move_stop is None:
raise RuntimeError(f"missing move-stop event: {trace}")
stop_s = float(move_stop.get("t_s", 0.0))
samples = [e for e in events if e.get("type") == "streaming_sample"]
moving = [e for e in samples if MOVE_START_S <= float(e.get("t_s", 0.0)) < stop_s
and float(e.get("speed_cm_s", 0.0)) >= 50.0]
dwell = [e for e in samples if float(e.get("t_s", 0.0)) >= stop_s
and float(e.get("speed_cm_s", 0.0)) < 50.0]
if exit_code != 0 or len(moving) < 40 or len(dwell) < 16:
raise RuntimeError(f"incomplete {run_name}: exit={exit_code}, move={len(moving)}, "
f"dwell={len(dwell)}")
move_start = min(float(e.get("t_s", 0.0)) for e in moving)
work = [e for e in events if e.get("type") == "game_thread_work_frame"
and move_start <= float(e.get("t_s", 0.0)) < stop_s]
scheduler_moving = [e for e in scheduler_all
if move_start <= float(e.get("t_s", 0.0)) < stop_s]
rebuild_events = [e for e in events if e.get("type") == "desired_rebuild_breakdown"
and move_start <= float(e.get("t_s", 0.0)) < stop_s]
applies = {int(e.get("frame", -1)): e for e in events
if e.get("type") == "apply_frame"
and move_start <= float(e.get("t_s", 0.0)) < stop_s}
scheduler = {int(e.get("frame", -1)): e for e in scheduler_all
if move_start <= float(e.get("t_s", 0.0)) < stop_s}
desired = {int(e.get("frame", -1)): e for e in events
if e.get("type") == "desired_rebuild_breakdown"
and move_start <= float(e.get("t_s", 0.0)) < stop_s}
moving_tiles = [e for e in events if e.get("type") == "tile"
and move_start <= float(e.get("t_s", 0.0)) < stop_s]
apply_deferrals = [e for e in events if e.get("type") == "tile_apply_deferred"
and move_start <= float(e.get("t_s", 0.0)) < stop_s]
frames = []
for event in work:
frame = int(event.get("frame", -1))
apply = applies.get(frame, {})
admit = scheduler.get(frame, {})
rebuild = desired.get(frame, {})
frames.append({
"frame": frame,
"tick_ms": float(event.get("tick_ms", 0.0)),
"update_chunks_ms": float(event.get("update_chunks_ms", 0.0)),
"pending_drain_ms": float(event.get("pending_drain_ms", 0.0)),
"apply_ms": float(apply.get("apply_ms", 0.0)),
"apply_budget_ms": float(apply.get("apply_budget_ms", 0.0)),
"meshes_applied": int(apply.get("meshes_applied", 0)),
"time_budget_hit": int(apply.get("time_limit_reached", 0)),
"desired_rebuild_ms": float(admit.get("desired_rebuild_ms", 0.0)),
"desired_rebuild_event_ms": float(rebuild.get("total_ms", 0.0)),
"overlay_materialize_ms": float(rebuild.get("overlay_materialize_ms", 0.0)),
"core_prepare_ms": float(rebuild.get("core_prepare_ms", 0.0)),
"core_delta_ms": float(rebuild.get("core_delta_ms", 0.0)),
"priority_sort_ms": float(rebuild.get("priority_sort_ms", 0.0)),
"critical_order_ms": float(rebuild.get("critical_order_ms", 0.0)),
"collision_stamp_ms": float(rebuild.get("collision_stamp_ms", 0.0)),
"transition_index_ms": float(rebuild.get("transition_index_ms", 0.0)),
"retained_region_ms": float(rebuild.get("retained_region_ms", 0.0)),
"region_reconcile_ms": float(rebuild.get("region_reconcile_ms", 0.0)),
"level_sort_ms": float(admit.get("level_sort_ms", 0.0)),
"level_candidate_scan_ms": float(admit.get("level_candidate_scan_ms", 0.0)),
"submit_tiles_ms": float(admit.get("submit_tiles_ms", 0.0)),
})
near = []
cone = []
far_holes = []
for event in moving:
desired_by_lod = event.get("desired_by_lod", [0] * 9)
resident_by_lod = event.get("resident_by_lod", [0] * 9)
covered_by_lod = event.get("covered_by_lod", resident_by_lod)
near.append(percent(sum(resident_by_lod[:3]), sum(desired_by_lod[:3])) or 0.0)
cone.append(percent(int(event.get("near_forward_cone_resident", 0)),
int(event.get("near_forward_cone_desired", 0))) or 0.0)
far_holes.append(percent(sum(desired_by_lod[3:]) - sum(covered_by_lod[3:]),
sum(desired_by_lod[3:])) or 0.0)
settled = next((e for e in samples if float(e.get("t_s", 0.0)) < move_start
and float(e.get("speed_cm_s", 0.0)) < 50.0
and sum(e.get("settled_by_lod", [0] * 9)[:3])
>= sum(e.get("desired_by_lod", [0] * 9)[:3]) > 0), None)
if settled is None:
raise RuntimeError(f"missing initial LOD0-2 settled sample: {trace}")
arrival_settle_s = float(settled["t_s"])
arrival_scheduler = [e for e in scheduler_all
if float(e.get("t_s", 0.0)) <= arrival_settle_s]
apply_all = [e for e in events if e.get("type") == "apply_frame"]
if condition == "candidate":
if any(int(e.get("fast_travel_mode", 0)) != 0 for e in arrival_scheduler):
raise RuntimeError(f"fast-travel admission activated before initial arrival settled: {trace}")
if any(float(e.get("apply_budget_ms", 0.0)) < 1.99
and float(e.get("t_s", 0.0)) <= arrival_settle_s for e in apply_all):
raise RuntimeError(f"fast-travel apply cap activated before initial arrival settled: {trace}")
if not any(int(e.get("initial_arrival_set_settled", 0)) == 1
and int(e.get("fast_travel_mode", 0)) == 1
for e in scheduler_moving):
raise RuntimeError(f"fast-travel mode never activated after initial arrival settled: {trace}")
if not any(float(e.get("apply_budget_ms", 0.0)) <= 1.001
for e in apply_all
if move_start <= float(e.get("t_s", 0.0)) < stop_s):
raise RuntimeError(f"compiled 1 ms apply cap never activated while moving: {trace}")
p99 = percentile([r["tick_ms"] for r in frames])
worst = max(frames, key=lambda r: r["tick_ms"], default={})
row = {
"condition": condition,
"run": run_name,
"diagnostic_override": audit_or_diag,
"source_default_incremental": expected["incremental"],
"early_admission_wait_for_critical": expected["wait"],
"early_admission_apply_budget_ms": expected["apply_budget_ms"],
"speed_mps": 30,
"effective_tile_slots": backend.get("effective_tile_slots"),
"moving_samples": len(moving),
"dwell_samples": len(dwell),
"tick_p99_ms": p99,
"tick_worst_ms": worst.get("tick_ms"),
"worst_frame": worst,
"near_readiness_moving_mean_pct": statistics.mean(near) if near else None,
"forward_cone_readiness_moving_mean_pct": statistics.mean(cone) if cone else None,
"far_holes_moving_median_pct": statistics.median(far_holes) if far_holes else None,
"arrival_lod02_settled_s": float(settled.get("t_s", 0.0)) if settled else None,
"arrival_fast_travel_modes_before_settle": sum(
int(e.get("fast_travel_mode", 0)) != 0 for e in arrival_scheduler),
"arrival_gate_was_settled_before_move": any(
int(e.get("initial_arrival_set_settled", 0)) != 0
for e in scheduler_all if float(e.get("t_s", 0.0)) < move_start),
"moving_fast_travel_mode_frames": sum(
int(e.get("fast_travel_mode", 0)) != 0 for e in scheduler_moving),
"moving_one_ms_apply_budget_frames": sum(
float(e.get("apply_budget_ms", 0.0)) <= 1.001
for e in apply_all if move_start <= float(e.get("t_s", 0.0)) < stop_s),
"desired_rebuild_count": len(rebuild_events),
"incremental_materialization_updates": sum(
int(e.get("materialization_update", 0)) != 0 for e in events
if e.get("type") == "desired_rebuild_breakdown"
and move_start <= float(e.get("t_s", 0.0)) < stop_s),
"core_incremental_deltas": sum(
int(e.get("incremental_delta", 0)) != 0 for e in events
if e.get("type") == "core_desired_cache"
and move_start <= float(e.get("t_s", 0.0)) < stop_s),
"level_sort_samples_over_1ms": sum(
float(e.get("level_sort_ms", 0.0)) > 1.0 for e in scheduler_moving),
"level_sort_max_ms": max((float(e.get("level_sort_ms", 0.0))
for e in scheduler_moving), default=0.0),
"desired_rebuild_events_over_5ms": sum(
float(e.get("total_ms", 0.0)) > 5.0 for e in rebuild_events),
"desired_rebuild_max_ms": max(
(float(e.get("total_ms", 0.0)) for e in rebuild_events), default=0.0),
"overlay_materialize_events_over_2ms": sum(
float(e.get("overlay_materialize_ms", 0.0)) > 2.0 for e in rebuild_events),
"overlay_materialize_max_ms": max(
(float(e.get("overlay_materialize_ms", 0.0)) for e in rebuild_events), default=0.0),
"desired_subphase_max_ms": {
key: max((float(e.get(key, 0.0)) for e in rebuild_events), default=0.0)
for key in ("core_prepare_ms", "overlay_materialize_ms", "priority_sort_ms",
"critical_order_ms", "collision_stamp_ms", "transition_index_ms",
"retained_region_ms", "region_reconcile_ms")},
"apply_budget_hit_frames": sum(f["time_budget_hit"] for f in frames),
"apply_frame_count": sum(f["meshes_applied"] > 0 for f in frames),
"apply_ms_max": max((f["apply_ms"] for f in frames), default=0.0),
"tiles_over_1ms_apply": sum(float(e.get("apply_s", 0.0)) > 0.001
for e in moving_tiles),
"apply_preflight_deferred_count": len(apply_deferrals),
"apply_preflight_deferred_max_estimate_ms": max(
(float(e.get("estimated_apply_ms", 0.0)) for e in apply_deferrals), default=0.0),
"apply_preflight_deferred_max_triangles": max(
(int(e.get("triangles", 0)) for e in apply_deferrals), default=0),
"largest_tile_applies": sorted(({
"apply_ms": float(e.get("apply_s", 0.0)) * 1000.0,
"triangles": int(e.get("triangles", 0)),
"vertices": int(e.get("vertices", 0)),
"level": int(e.get("level", -1)),
} for e in moving_tiles),
key=lambda e: e["apply_ms"], reverse=True)[:8],
"top_ticks": sorted(frames, key=lambda r: r["tick_ms"], reverse=True)[:8],
"trace": str(trace.relative_to(ROOT)),
"log": str(log.relative_to(ROOT)),
"wall_s": round(wall_s, 3),
}
return row
def run(condition: str, run_name: str, diagnostic_override: bool = False,
tile_hash_dump: Path | None = None) -> dict:
if condition == "baseline":
binary = BASELINE_BINARY
elif condition == "candidate":
binary = CANDIDATE_BINARY
else:
raise ValueError(condition)
if not binary.is_file():
raise FileNotFoundError(binary)
HOST_PLUGIN.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(binary, HOST_PLUGIN)
out = OUT / condition
export = HOST / "Plugins/VoxelForge/Saved/Linux/l170" / condition / run_name
userdir = HOST / "Saved/Linux/l170/user" / condition / run_name
zendata = HOST / "Saved/Linux/l170/zen" / condition / run_name
ddc = HOST / "Saved/Linux/l170/DDC"
trace = out / f"{run_name}.trace.jsonl"
log = out / f"{run_name}.log"
stdout = out / f"{run_name}.stdout.log"
for directory in (out, export, userdir, zendata, ddc):
directory.mkdir(parents=True, exist_ok=True)
trace.unlink(missing_ok=True)
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWN_M)
command = [
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
"-notraceserver", "-nocrashreports", "-DisablePlugins=AndroidFileServer",
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
"-archetype=TunnelNetwork", "-slot=0", "-modes=export", "-gameconfig=1",
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0",
"-coreplantest=0", "-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings",
"-corehybridmesher=0", f"-out={export}",
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=3000.0",
f"-voxel.TestMoveStartSeconds={MOVE_START_S:.1f}",
f"-voxel.TestMoveDurationSeconds={MOVE_DURATION_S:.1f}",
"-voxel.TestExitSeconds=0.0", "-voxel.TestMoveDirection=1,0,0",
"-voxel.TestExitOnSteady=0", "-coreworldstreamtrace=1",
f"-voxel.StartupTraceSpawnCm={spawn_cm}",
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
f"-abslog={log}",
]
if tile_hash_dump is not None:
tile_hash_dump.parent.mkdir(parents=True, exist_ok=True)
tile_hash_dump.unlink(missing_ok=True)
command.append(f"-voxel.TileHashDump={tile_hash_dump.resolve()}")
started = time.monotonic()
with stdout.open("w", encoding="utf-8") as stream:
process = subprocess.run(
["taskset", "-c", "0-11", "timeout", "1740s", *command],
cwd=ROOT, stdout=stream, stderr=subprocess.STDOUT, check=False, timeout=1740)
return summarize(condition, run_name, trace, log, process.returncode,
time.monotonic() - started, diagnostic_override)
def main() -> int:
parser = argparse.ArgumentParser()
modes = parser.add_mutually_exclusive_group(required=True)
modes.add_argument("--diagnostic", action="store_true",
help="one candidate trace with compiled defaults and no command-line switch overrides")
modes.add_argument("--matrix", action="store_true",
help="warm both binaries, then run seven interleaved pairs with compiled defaults and no switch overrides")
modes.add_argument("--geometry-hash-check", action="store_true",
help="compare shared tile hashes from the compiled-default binaries")
args = parser.parse_args()
missing = [p for p in (EDITOR, HOST / "VFHost.uproject", ROOT / "VoxelForge.uplugin")
if not p.exists()]
if missing:
raise SystemExit(f"missing UE runner input: {missing}")
OUT.mkdir(parents=True, exist_ok=True)
if args.diagnostic:
warmup = run("candidate", "diagnostic-warmup", diagnostic_override=True)
row = run("candidate", "l170-default-diagnostic", diagnostic_override=True)
row["warmup"] = warmup
path = OUT / "diagnostic.json"
path.write_text(json.dumps(row, indent=2, sort_keys=True) + "\n", encoding="utf-8")
print("[L169Diagnostic] " + json.dumps(row, sort_keys=True), flush=True)
return 0
if args.geometry_hash_check:
hash_dir = OUT / "geometry-hash"
baseline_dump = hash_dir / "baseline.json"
candidate_dump = hash_dir / "candidate.json"
run("baseline", "geometry-hash", tile_hash_dump=baseline_dump)
run("candidate", "geometry-hash", tile_hash_dump=candidate_dump)
baseline_data = json.loads(baseline_dump.read_text(encoding="utf-8"))
candidate_data = json.loads(candidate_dump.read_text(encoding="utf-8"))
key_fields = ("level", "tile_x", "tile_y", "tile_z")
def indexed(data: dict) -> dict[tuple[int, ...], tuple]:
return {tuple(int(record[field]) for field in key_fields): (
str(record.get("geometry_hash", "")),
int(record.get("triangle_count", -1)),
bool(record.get("empty", False)))
for record in data.get("records", [])}
base = indexed(baseline_data)
candidate = indexed(candidate_data)
common = base.keys() & candidate.keys()
mismatches = [key for key in common if base[key] != candidate[key]]
result = {"baseline_records": len(base), "candidate_records": len(candidate),
"common_records": len(common), "hash_mismatches": len(mismatches),
"baseline_only_records": len(base.keys() - candidate.keys()),
"candidate_only_records": len(candidate.keys() - base.keys()),
"baseline_dump": str(baseline_dump.relative_to(ROOT)),
"candidate_dump": str(candidate_dump.relative_to(ROOT))}
(hash_dir / "comparison.json").write_text(
json.dumps(result, indent=2, sort_keys=True) + "\n", encoding="utf-8")
print("[L169GeometryHash] " + json.dumps(result, sort_keys=True), flush=True)
return 0 if len(common) >= 100 and not mismatches else 1
warmups = [run("baseline", "warmup"), run("candidate", "warmup")]
rows = []
partial = OUT / "runs-partial.json"
for pair, sequence in ORDER.items():
for condition in sequence:
rows.append(run(condition, f"pair-{pair}"))
partial.write_text(json.dumps(rows, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
metrics = ("tick_p99_ms", "tick_worst_ms", "near_readiness_moving_mean_pct",
"forward_cone_readiness_moving_mean_pct", "far_holes_moving_median_pct",
"arrival_lod02_settled_s")
medians = {condition: {metric: statistics.median(
row[metric] for row in rows if row["condition"] == condition
and row.get(metric) is not None) for metric in metrics}
for condition in ("baseline", "candidate")}
by_pair = {pair: {row["condition"]: row for row in rows if row["run"] == f"pair-{pair}"}
for pair in ORDER}
deltas = [
{"pair": pair, **{f"{metric}_delta":
by_pair[pair]["candidate"][metric] - by_pair[pair]["baseline"][metric]
for metric in metrics}}
for pair in ORDER
]
summary = {"protocol": "Seven interleaved 30 m/s UE pairs using compiled defaults; "
"candidate defaults enable the two arrival-gated fast-travel switches; no "
"command-line switch overrides; 12 s move plus 5 s dwell; eight effective "
"tile slots; one warm-up per binary excluded.",
"defaults": {"incremental_desired_set": 1,
"streaming_fast_travel_wait_for_all_critical_tiles": 0,
"streaming_fast_travel_apply_budget_ms": 1.0,
"forward_first": 0, "rebuild_on_core_region_completion": 1,
"core_desired_cache_by_center": 0,
"incremental_desired_audit_every": 0,
"incremental_desired_materialization_audit_every": 0,
"incremental_desired_audit_inject_mismatch": 0},
"warmups": warmups, "movement_medians": medians,
"paired_deltas_candidate_minus_baseline": deltas, "runs": rows}
(OUT / "summary.json").write_text(json.dumps(summary, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
print("[L170Summary] " + json.dumps({"movement_medians": medians,
"paired_deltas": deltas}, sort_keys=True), flush=True)
all_runs_p99_pass = all(row["tick_p99_ms"] is not None and row["tick_p99_ms"] <= 11.0
for row in rows)
all_runs_worst_pass = all(row["tick_worst_ms"] is not None
and row["tick_worst_ms"] <= 14.0 for row in rows)
candidate_worst = [row["tick_worst_ms"] for row in rows
if row["condition"] == "candidate"]
baseline_worst = [row["tick_worst_ms"] for row in rows
if row["condition"] == "baseline"]
arrival_pairs_not_slower = sum(
by_pair[pair]["candidate"]["arrival_lod02_settled_s"]
<= by_pair[pair]["baseline"]["arrival_lod02_settled_s"]
for pair in ORDER)
arrival_median_delta_pct = 100.0 * (
medians["candidate"]["arrival_lod02_settled_s"]
/ medians["baseline"]["arrival_lod02_settled_s"] - 1.0)
near_gain_pp = (medians["candidate"]["near_readiness_moving_mean_pct"]
- medians["baseline"]["near_readiness_moving_mean_pct"])
forward_gain_pp = (medians["candidate"]["forward_cone_readiness_moving_mean_pct"]
- medians["baseline"]["forward_cone_readiness_moving_mean_pct"])
far_hole_reduction_pp = (medians["baseline"]["far_holes_moving_median_pct"]
- medians["candidate"]["far_holes_moving_median_pct"])
gate_metrics = {
"arrival_delta_pct": arrival_median_delta_pct,
"arrival_pairs_not_slower": arrival_pairs_not_slower,
"near_gain_pp": near_gain_pp,
"forward_gain_pp": forward_gain_pp,
"far_hole_reduction_pp": far_hole_reduction_pp,
}
gates = {
"every_run_p99_le_11_ms": all_runs_p99_pass,
"every_run_worst_le_14_ms": all_runs_worst_pass,
"candidate_p99_median_not_worse": medians["candidate"]["tick_p99_ms"]
<= medians["baseline"]["tick_p99_ms"],
"candidate_worst_tick_median_not_worse": bool(candidate_worst and baseline_worst)
and statistics.median(candidate_worst) <= statistics.median(baseline_worst),
"near_gain_at_least_75pct_of_l169": near_gain_pp >= 9.63,
"forward_gain_at_least_75pct_of_l169": forward_gain_pp >= 1.81,
"far_hole_reduction_at_least_75pct_of_l169": far_hole_reduction_pp >= 1.35,
"arrival_median_within_0_5pct": abs(arrival_median_delta_pct) <= 0.5,
"arrival_not_slower_in_majority_of_pairs": arrival_pairs_not_slower >= 4,
}
summary["promotion_gates"] = gates
summary["promotion_gate_metrics"] = gate_metrics
(OUT / "summary.json").write_text(json.dumps(summary, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
print("[L170Gates] " + json.dumps({"checks": gates, "metrics": gate_metrics},
sort_keys=True), flush=True)
return 0 if all(gates.values()) else 1
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,9 @@
# L171 — UE arrival measurement
**Result:** Not measured. The A/A warm-up aborts before the first streaming sample, so this round cannot say whether L170’s +1.28% arrival difference exceeds UE noise or identify its cause.
Two `assembler/world` `ca0bf76` VFHost Development builds completed and produced identical VoxelForge, VoxelForgeCore, and VoxelForgeEditor binaries. With that matched module set staged, the A/A warm-up reaches `world_assembly_plan` with one surface stratum, then logs `surface_strates=858993460` after snapshot parsing and terminates with `std::length_error: vector` during `AVoxelWorld::BeginPlay`. The trace has no scheduler readback or streaming sample. This points to a Core snapshot/module compatibility failure; the exact cause is unproven.
Valid arrival pairs: A/A **0/15**, switches-on **0/15**, switches-off **0/15**. No medians, paired spread, or slower counts are available. The A/B batches were not started because the A/A control failed validation. Failed launches are logged under `Saved/Linux/l171/ue/arrival/`.
The baseline and L170 HEAD-off VFHost builds passed. The runner compiled and its CLI loaded; `git diff --check` passed. No Unreal `Source/` files changed. The full arrival matrix and fast verification did not run: the UE control never reached measurement.
@@ -0,0 +1,62 @@
# L172 — repair the UE harness and restore level refill
**Result:** The scheduler now keeps the full stable candidate order and refills a level when a slot opens during its pass. The UE harness uses all three links from one tree and rejects mixed DWARF layouts. The 10-pair decision passed, so early admission is now the default. The final default-on UE build and fast tier passed.
## L171 findings and harness
The reviewer’s mixed-build diagnosis matches the saved evidence. L171’s A/A warm-up logged `surface_strates=1`, then `858993460` (`0x33333334`) before throwing `std::length_error: vector`. Its `aa-build-1.log` has only two link actions and no `Compile VoxelWorld.cpp`; the staged `aa-build-1`, `aa-build-2`, and `l170-head-off` plugin binaries contain the L170-only `initial_arrival_set_settled` and `tile_apply_deferred` UTF-16 strings, while the saved `ca0bf76` binary does not. The L171 DWARF inspection showed `WorldAssemblyStratumPlanV1` as 600 bytes in the stale `VoxelWorld.cpp.o` and 480 bytes in the adapter and Core shim. This is consistent with the vector corruption being an ABI mismatch.
At L172 start, `Source`, `Standalone`, and `VoxelForge.uplugin` in `VFHost/PluginDeps/VoxelForge` all resolved to the main checkout on `wip/l141-liminal`. Earlier round reports document restoring `Source` alone. L172 switches all three links together and asserts their common root. Each run manifest records that root’s `HEAD` and `git status --porcelain`. The `ca0bf76` comparison used a fresh detached `git worktree` with its own `Source` and `Standalone`; it built `VoxelWorld.cpp`, passed the 480/480/480-byte DWARF gate, and ran the owner world in its warm-up and all ten pairs. Fixed-tree builds also compiled `VoxelWorld.cpp` and passed the same gate. All three VFHost links were restored to the main checkout after the last build and run.
## Scheduler change
`VoxelWorld.cpp` no longer sorts only the pass-start free-slot prefix and truncates `LevelTiles`. It stable-sorts the complete level and checks current capacity as each candidate is admitted. This preserves full-sort order and lets the same level refill when work completes mid-pass. The regression test frees a slot after the first submission, checks the submitted sequence against the full-sort order, and includes the old partial-sort/truncation path as a negative control.
The frame-resolution arrival is the first `scheduler_admission` frame with both `desired_missing_by_lod[0:3]` and `pending_by_lod[0:3]` summing to zero. Re-scoring saved L169 and L170 traces gives paired median deltas of **+48.762 ms** and **+60.475 ms** respectively; each candidate was slower in all 7 pairs (14/14 combined). The 0.25 s `streaming_sample` arrival had obscured this difference.
## L172 movement results
Runs used the same fixed binary for both CVar arms, 30 m/s, one warm-up per arm, and ten counterbalanced pairs per batch. The measured movement window was 12 s (first moving samples near 6.05 s, stop near 18.02 s). The trace continued through 45.00 s after movement; those post-move samples were excluded from movement metrics. The pre-run load was 0.45 / 1.43 / 2.00 (1/5/15 min), with no UE or build process running. The harness reported eight effective tile slots.
| Check | Result |
| --- | ---: |
| A/A absolute paired-delta p90 | 36.815 ms |
| Fixed-off slower than true `ca0bf76` | 6/10 pairs (stop rule: 8/10) |
| Fixed-on minus fixed-off arrival, paired median | −12.203 ms |
| Fixed-on slower on arrival | 4/10 pairs |
| Arrival tolerance | 36.815 ms |
| Near readiness change, fixed-on vs fixed-off | +11.951 pp (55.276% vs 43.325%) |
| Near readiness change, fixed-on vs `ca0bf76` | +12.457 pp (55.276% vs 42.820%) |
| Far-hole change, fixed-on vs either control | −1.800 pp (0.000% vs 1.800%) |
| Fixed-on maximum p99 / worst tick | 7.460 / 12.305 ms |
| Maximum p99 / worst tick across both on/off arms | 7.460 / 13.266 ms |
All decision gates pass. For the CVar promotion, movement gates use the fixed-on treatment against fixed-off, and the fixed-on arm also clears the L169-relative near/far gates against `ca0bf76`. The A/A and true-baseline runs are measurement controls, not the promotion treatment; the baseline control contained one 16.7 ms worst-tick outlier. Both on/off arms stayed below 14 ms. The per-pair traces, manifests, summaries, and decision are in `Saved/Linux/l172/ue/`.
The default-on smoke run omitted both command-line CVar overrides. Its backend readback was wait=0 and apply budget=1.0 ms; frame arrival was 1.500 s, p99 6.971 ms, and worst tick 13.052 ms.
## Defaults, checks, and limits
| CVar | Final default |
| --- | ---: |
| `voxel.StreamingFastTravelWaitForAllCriticalTiles` | 0 |
| `voxel.StreamingFastTravelApplyBudgetMilliseconds` | 1.0 ms |
| `voxel.CoreRegionConcurrentAdmission` | 1 |
| `voxel.StreamingIncrementalDesiredSet` | 1 |
| `voxel.StreamingFastTravelForwardFirst` | 0 |
| `voxel.StreamingRebuildOnCoreRegionCompletion` | 1 |
| `voxel.StreamingCoreDesiredCacheByCenter` | 0 |
| `voxel.StreamingIncrementalDesiredAuditEvery` | 0 |
| `voxel.StreamingIncrementalDesiredMaterializationAuditEvery` | 0 |
| `voxel.StreamingIncrementalDesiredAuditInjectMismatch` | 0 |
- Fast tier: **PASS**, 269 checks and 5 non-gating diagnostics, 284.7 s. The new mid-pass refill regression and owner W1/W8/W12 determinism checks passed.
- Final default-on VFHost build: **PASS**, 150.58 s; `VoxelWorld.cpp` compiled and the DWARF sizes were 480/480/480 bytes.
- Default-on owner-world smoke: **PASS**, 52.33 s. Saved L169/L170 rescoring, all three 10-pair batches, runner syntax compilation, and `git diff --check` passed.
- Two failed preflight launches were excluded: UE first inherited the standalone 6 GiB address-space limit, then the runner used an export path outside the plugin Saved directory. An initial fixed-off warm-up produced a valid trace but exposed an overstrict latch assertion; the check was corrected and the final warm-up and measured pairs passed. None of these launches contributed paired timing values.
- UE W12 movement was not run; the harness exposes eight effective tile slots. The timing experiment covers the owner four-stratum plan under Null RHI.
Changed Unreal `Source/` files:
- `Source/VoxelForge/Private/VoxelWorld.cpp`
- `Source/VoxelForge/Private/VoxelStreamingAdmissionOrder.h`
@@ -0,0 +1,868 @@
#!/usr/bin/env python3
"""Build isolated L172 UE trees, measure admission, and rescore saved traces."""
from __future__ import annotations
import argparse
import datetime as dt
import hashlib
import json
import math
import os
from pathlib import Path
import re
import shutil
import statistics
import subprocess
import time
from typing import Any
ROOT = Path(__file__).resolve().parents[4]
HOST = Path("/home/jahni/VFHost")
ENGINE = Path("/home/jahni/UnrealEngine")
MAIN_CHECKOUT = Path("/home/jahni/VoxelM/Plugins/VoxelForge")
OUT = ROOT / "Saved/Linux/l172"
UE_OUT = OUT / "ue"
BUILD_OUT = OUT / "builds"
BIN_OUT = OUT / "bin"
HOST_PLUGIN_LINK = HOST / "PluginDeps/VoxelForge"
HOST_PLUGIN_BINARY = HOST / "Binaries/Linux/libUnrealEditor-VoxelForge.so"
EDITOR = ENGINE / "Engine/Binaries/Linux/UnrealEditor-Cmd"
BUILD_SH = ENGINE / "Engine/Build/BatchFiles/Linux/Build.sh"
DWARFDUMP = Path("/usr/lib/llvm-19/bin/llvm-dwarfdump")
INTERMEDIATE = HOST / "Intermediate"
MOVE_START_S = 6.0
MOVE_DURATION_S = 12.0
SPAWN_M = (19.166015625, -14.2001953125, -753.3740234375)
LINK_NAMES = ("Source", "Standalone", "VoxelForge.uplugin")
OBJECTS = {
"voxel_world": "External/Build/Linux/x64/UnrealEditor/Development/VoxelForge/VoxelWorld.cpp.o",
"core_adapter": "External/Build/Linux/x64/UnrealEditor/Development/VoxelForge/VoxelForgeCoreAdapter.cpp.o",
"core_api_shim": "External/Build/Linux/x64/UnrealEditor/Development/VoxelForgeCore/VFCore_Core_src_VoxelForgeCore_cpp.cpp.o",
}
WAIT_OFF = 1
APPLY_OFF_MS = 0.0
WAIT_ON = 0
APPLY_ON_MS = 1.0
def utc_now() -> str:
return dt.datetime.now(dt.timezone.utc).isoformat(timespec="seconds")
def write_json(path: Path, value: Any) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
temporary = path.with_suffix(path.suffix + ".tmp")
temporary.write_text(json.dumps(value, indent=2, sort_keys=True) + "\n", encoding="utf-8")
temporary.replace(path)
def git(root: Path, *args: str) -> str:
result = subprocess.run(["git", "-C", str(root), *args], text=True,
capture_output=True, check=False)
if result.returncode != 0:
raise RuntimeError(f"git -C {root} {' '.join(args)} failed: {result.stderr.strip()}")
return result.stdout.rstrip("\n")
def root_info(root: Path) -> dict[str, Any]:
actual_root = Path(git(root, "rev-parse", "--show-toplevel")).resolve()
expected_root = root.resolve()
if actual_root != expected_root:
raise RuntimeError(f"plugin root mismatch: requested={expected_root}, git={actual_root}")
return {
"plugin_root": str(actual_root),
"git_head": git(actual_root, "rev-parse", "HEAD"),
"git_status_porcelain": git(actual_root, "status", "--porcelain"),
}
def link_roots() -> dict[str, str]:
roots = {}
for name in LINK_NAMES:
link = HOST_PLUGIN_LINK / name
if not link.is_symlink():
raise RuntimeError(f"VFHost plugin dependency is not a symlink: {link}")
resolved = link.resolve(strict=True)
roots[name] = str(resolved.parent)
return roots
def assert_shared_root(expected_root: Path) -> dict[str, str]:
expected = expected_root.resolve(strict=True)
roots = link_roots()
if any(Path(root) != expected for root in roots.values()):
raise RuntimeError(f"VFHost plugin symlinks do not share expected root {expected}: {roots}")
return roots
def _move_plugin_intermediates(old_root: str | None, new_root: Path) -> list[dict[str, str]]:
matches = []
if INTERMEDIATE.exists():
for current, dirs, _files in os.walk(INTERMEDIATE, topdown=True):
for dirname in list(dirs):
if dirname.startswith("VoxelForge"):
matches.append(Path(current) / dirname)
dirs.remove(dirname)
if not matches:
return []
head = "unknown"
if old_root:
try:
head = root_info(Path(old_root))["git_head"][:12]
except Exception:
head = Path(old_root).name
archive = OUT / "intermediate-archive" / f"{time.time_ns()}-{head}-to-{new_root.name}"
moved = []
for source in sorted(matches, key=lambda path: len(path.parts)):
relative = source.relative_to(INTERMEDIATE)
target = archive / relative
target.parent.mkdir(parents=True, exist_ok=True)
shutil.move(str(source), str(target))
moved.append({"from": str(source), "to": str(target)})
return moved
def switch_plugin_root(target_root: Path) -> dict[str, Any]:
target = target_root.resolve(strict=True)
if not (target / "Source").is_dir() or not (target / "Standalone").is_dir():
raise RuntimeError(f"tree does not contain its own Source and Standalone: {target}")
if not (target / "VoxelForge.uplugin").is_file():
raise RuntimeError(f"tree has no plugin descriptor: {target}")
current_roots = set(link_roots().values())
old_root = next(iter(current_roots)) if len(current_roots) == 1 else None
moved = []
if old_root != str(target):
moved = _move_plugin_intermediates(old_root, target)
for name in LINK_NAMES:
link = HOST_PLUGIN_LINK / name
temporary = link.with_name(f"{link.name}.l172-{os.getpid()}.tmp")
if temporary.exists() or temporary.is_symlink():
temporary.unlink()
temporary.symlink_to(target / name)
os.replace(temporary, link)
roots = assert_shared_root(target)
return {"active_root": str(target), "links": roots, "intermediates_moved": moved}
def restore_main_checkout() -> dict[str, Any]:
return switch_plugin_root(MAIN_CHECKOUT)
def sha256_file(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as stream:
for block in iter(lambda: stream.read(1024 * 1024), b""):
digest.update(block)
return digest.hexdigest()
def dwarf_struct_size(path: Path) -> int:
if not DWARFDUMP.is_file():
raise RuntimeError(f"llvm-dwarfdump is missing: {DWARFDUMP}")
result = subprocess.run([str(DWARFDUMP), "--name=WorldAssemblyStratumPlanV1", str(path)],
text=True, capture_output=True, check=False)
if result.returncode != 0:
raise RuntimeError(f"llvm-dwarfdump failed for {path}: {result.stderr.strip()}")
dies = re.split(r"(?=0x[0-9a-fA-F]+: DW_TAG_)", result.stdout)
matches = []
for die in dies:
if not re.search(r'DW_AT_name\s+\("WorldAssemblyStratumPlanV1"\)', die):
continue
size = re.search(r"DW_AT_byte_size\s+\(0x([0-9a-fA-F]+)\)", die)
if size:
matches.append(int(size.group(1), 16))
if len(matches) != 1:
raise RuntimeError(f"expected one complete plan type in {path}; sizes={matches}")
return matches[0]
def dwarf_pre_run_gate() -> dict[str, Any]:
sizes = {}
for label, relative in OBJECTS.items():
path = INTERMEDIATE / relative
if not path.is_file():
raise RuntimeError(f"pre-run DWARF gate object is missing: {path}")
sizes[label] = {"object": str(path), "byte_size": dwarf_struct_size(path)}
unique = {item["byte_size"] for item in sizes.values()}
if len(unique) != 1:
raise RuntimeError(f"WorldAssemblyStratumPlanV1 size mismatch: {sizes}")
return {"status": "PASS", "struct": "WorldAssemblyStratumPlanV1", "objects": sizes}
def create_fresh_baseline_worktree() -> Path:
timestamp = str(time.time_ns())
repo_host = OUT / "worktree-hosts" / f"ca0bf76-git-{timestamp}"
parent = OUT / "worktrees"
parent.mkdir(parents=True, exist_ok=True)
repo_host.parent.mkdir(parents=True, exist_ok=True)
commit = git(ROOT, "rev-parse", "ca0bf76")
clone = subprocess.run(["git", "clone", "--shared", "--no-checkout",
str(MAIN_CHECKOUT), str(repo_host)], text=True,
capture_output=True, check=False)
if clone.returncode != 0:
raise RuntimeError(f"isolated git worktree host clone failed: {clone.stderr.strip()}")
fetch = subprocess.run(["git", "-C", str(repo_host), "fetch", "--no-tags",
str(MAIN_CHECKOUT), commit], text=True,
capture_output=True, check=False)
if fetch.returncode != 0:
raise RuntimeError(f"ca0bf76 fetch into isolated worktree host failed: "
f"{fetch.stderr.strip()}")
path = parent / f"assembler-world-ca0bf76-fresh-{timestamp}"
result = subprocess.run(["git", "-C", str(repo_host), "worktree", "add", "--detach",
str(path), commit], text=True, capture_output=True, check=False)
if result.returncode != 0:
raise RuntimeError(f"fresh ca0bf76 worktree add failed: {result.stderr.strip()}")
info = root_info(path)
if not info["git_head"].startswith("ca0bf767ca7afe40723bf8bb6987d9efc9ea7523"):
raise RuntimeError(f"fresh baseline worktree has unexpected HEAD: {info}")
return path.resolve()
def build_tree(root: Path, label: str) -> dict[str, Any]:
switch = switch_plugin_root(root)
source = root_info(root)
if len(set(switch["links"].values())) != 1:
raise RuntimeError(f"link roots diverged before build: {switch}")
log = BUILD_OUT / f"ubt-{label}.log"
log.parent.mkdir(parents=True, exist_ok=True)
command = [str(BUILD_SH), "VFHostEditor", "Linux", "Development",
f"-Project={HOST / 'VFHost.uproject'}", "-NoUBA", "-MaxParallelActions=4",
f"-Log={log}"]
started = time.monotonic()
with log.open("w", encoding="utf-8") as stream:
result = subprocess.run(["taskset", "-c", "0-11", "timeout", "1800s", *command],
cwd=ROOT, stdout=stream, stderr=subprocess.STDOUT,
check=False, timeout=1800)
elapsed = time.monotonic() - started
output = log.read_text(encoding="utf-8", errors="replace")
if result.returncode != 0:
raise RuntimeError(f"UE build {label} failed ({result.returncode}); log={log}")
if not re.search(r"Compile VoxelWorld\.cpp", output):
raise RuntimeError(f"UE build {label} did not compile VoxelWorld.cpp; log={log}")
gate = dwarf_pre_run_gate()
if not HOST_PLUGIN_BINARY.is_file():
raise RuntimeError(f"UE build did not produce {HOST_PLUGIN_BINARY}")
artifact_dir = BIN_OUT / label
artifact_dir.mkdir(parents=True, exist_ok=True)
artifact = artifact_dir / "libUnrealEditor-VoxelForge.so"
shutil.copy2(HOST_PLUGIN_BINARY, artifact)
return {
**source,
"build_command": command,
"build_log": str(log),
"compiled_voxel_world_cpp": True,
"elapsed_s": round(elapsed, 3),
"dwarf_pre_run_gate": gate,
"binary": str(artifact),
"binary_sha256": sha256_file(artifact),
"active_links": switch["links"],
"intermediates_moved_before_build": switch["intermediates_moved"],
}
def prepare() -> int:
OUT.mkdir(parents=True, exist_ok=True)
baseline_root = None
builds = {}
try:
baseline_root = create_fresh_baseline_worktree()
builds["ca0bf76"] = build_tree(baseline_root, "ca0bf76")
builds["ca0bf76"]["worktree_root"] = str(baseline_root)
builds["ca0bf76"]["worktree_created_with"] = [
"git", "worktree", "add", "--detach", str(baseline_root), "ca0bf76"]
write_json(OUT / "builds.json", {"status": "BASELINE_READY", **builds})
builds["fixed"] = build_tree(ROOT, "fixed")
builds["fixed"]["worktree_root"] = str(ROOT.resolve())
write_json(OUT / "builds.json", {"status": "BOTH_TREES_READY", **builds})
print("[L172Builds] " + json.dumps({
key: {"head": value["git_head"], "root": value["plugin_root"],
"binary_sha256": value["binary_sha256"],
"dwarf_sizes": {name: item["byte_size"]
for name, item in value["dwarf_pre_run_gate"]["objects"].items()}}
for key, value in builds.items() if key != "status"}, sort_keys=True), flush=True)
return 0
finally:
restored = restore_main_checkout()
write_json(OUT / "restored-main-links.json", {
"time_utc": utc_now(), "main_root": str(MAIN_CHECKOUT.resolve()),
"links": restored["links"], "intermediates_moved": restored["intermediates_moved"]})
def read_events(path: Path) -> list[dict[str, Any]]:
rows = []
if not path.is_file():
return rows
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
if not line.startswith("{"):
continue
try:
rows.append(json.loads(line))
except json.JSONDecodeError:
continue
return rows
def first_frame_arrival(events: list[dict[str, Any]]) -> dict[str, Any] | None:
for event in events:
if event.get("type") != "scheduler_admission":
continue
missing = event.get("desired_missing_by_lod", [])
pending = event.get("pending_by_lod", [])
if (len(missing) >= 3 and len(pending) >= 3
and sum(int(value) for value in missing[:3]) == 0
and sum(int(value) for value in pending[:3]) == 0):
return event
return None
def percentile(values: list[float], fraction: float = 0.99) -> float | None:
if not values:
return None
ordered = sorted(values)
index = max(0, min(len(ordered) - 1, math.ceil(len(ordered) * fraction) - 1))
return ordered[index]
def percent(numerator: int | float, denominator: int | float) -> float | None:
return 100.0 * numerator / denominator if denominator > 0 else None
def summarize_trial(condition: str, trace: Path, result: subprocess.CompletedProcess[str],
wall_s: float, expected_wait: int, expected_budget_ms: float) -> dict[str, Any]:
events = read_events(trace)
backend = next((event for event in events if event.get("type") == "worker_backend_config"), {})
scheduler = [event for event in events if event.get("type") == "scheduler_admission"]
if not scheduler:
raise RuntimeError(f"no scheduler_admission readback in {trace}")
if (int(backend.get("streaming_fast_travel_wait_for_all_critical_tiles", -1))
!= expected_wait
or abs(float(backend.get("streaming_fast_travel_apply_budget_ms", -999.0))
- expected_budget_ms) > 0.01
or int(backend.get("effective_tile_slots", -1)) != 8):
raise RuntimeError(f"switch or worker readback mismatch in {trace}: {backend}")
if any(int(event.get("forward_first", -1)) != 0
or int(event.get("rebuild_on_core_region_completion", -1)) != 1
or int(event.get("core_desired_cache_by_center", -1)) != 0
or int(event.get("incremental_desired_set", -1)) != 1
or int(event.get("incremental_desired_audit_every", -1)) != 0
or int(event.get("incremental_desired_materialization_audit_every", -1)) != 0
or int(event.get("incremental_desired_audit_inject_mismatch", -1)) != 0
for event in scheduler):
raise RuntimeError(f"non-default scheduler CVar readback in {trace}")
arrival = first_frame_arrival(scheduler)
if arrival is None:
raise RuntimeError(f"missing frame-resolution LOD0-2 settled event in {trace}")
if condition == "fixed-off":
arrival_s = float(arrival["t_s"])
pre_arrival = [event for event in scheduler
if float(event.get("t_s", 0.0)) < arrival_s]
post_arrival = [event for event in scheduler
if float(event.get("t_s", 0.0)) >= arrival_s]
if any(int(event.get("fast_travel_mode", 0)) != 0 for event in pre_arrival):
raise RuntimeError(f"fixed-off entered fast-travel mode before initial arrival: {trace}")
if not any(int(event.get("fast_travel_mode", 0)) == 1 for event in post_arrival):
raise RuntimeError(f"fixed-off never latched fast-travel mode after initial arrival: {trace}")
if condition == "fixed-on":
arrival_s = float(arrival["t_s"])
pre_arrival = [event for event in scheduler if float(event.get("t_s", 0.0)) < arrival_s]
apply_events = [event for event in events if event.get("type") == "apply_frame"]
if any(int(event.get("fast_travel_mode", 0)) != 0 for event in pre_arrival):
raise RuntimeError(f"fast-travel mode activated before frame-settled arrival: {trace}")
if any(float(event.get("apply_budget_ms", 0.0)) < 1.99
and float(event.get("t_s", 0.0)) <= arrival_s for event in apply_events):
raise RuntimeError(f"fast-travel apply budget activated before arrival: {trace}")
if not any(int(event.get("initial_arrival_set_settled", 0)) == 1
and int(event.get("fast_travel_mode", 0)) == 1
for event in scheduler):
raise RuntimeError(f"fixed-on latch did not activate after settled arrival: {trace}")
if not any(float(event.get("apply_budget_ms", 0.0)) <= 1.001
for event in apply_events):
raise RuntimeError(f"fixed-on 1 ms apply budget never activated: {trace}")
move_stop = next((event for event in events
if event.get("type") == "streaming_test_move_stopped"), None)
if move_stop is None:
raise RuntimeError(f"missing move-stop event in {trace}")
stop_s = float(move_stop.get("t_s", 0.0))
samples = [event for event in events if event.get("type") == "streaming_sample"]
moving = [event for event in samples
if MOVE_START_S <= float(event.get("t_s", 0.0)) < stop_s
and float(event.get("speed_cm_s", 0.0)) >= 50.0]
dwell = [event for event in samples
if float(event.get("t_s", 0.0)) >= stop_s
and float(event.get("speed_cm_s", 0.0)) < 50.0]
if result.returncode != 0 or len(moving) < 40 or len(dwell) < 16:
raise RuntimeError(f"incomplete run {trace}: exit={result.returncode}, "
f"moving={len(moving)}, dwell={len(dwell)}")
work_start = min(float(event.get("t_s", 0.0)) for event in moving)
frames = [event for event in events if event.get("type") == "game_thread_work_frame"
and work_start <= float(event.get("t_s", 0.0)) < stop_s]
if not frames:
raise RuntimeError(f"missing moving game-thread frames in {trace}")
near, far_holes = [], []
for event in moving:
desired = event.get("desired_by_lod", [0] * 9)
resident = event.get("resident_by_lod", [0] * 9)
covered = event.get("covered_by_lod", resident)
near.append(percent(sum(resident[:3]), sum(desired[:3])) or 0.0)
far_holes.append(percent(sum(desired[3:]) - sum(covered[3:]),
sum(desired[3:])) or 0.0)
settled_sample = next((event for event in samples
if float(event.get("t_s", 0.0)) < work_start
and float(event.get("speed_cm_s", 0.0)) < 50.0
and sum(event.get("settled_by_lod", [0] * 9)[:3])
>= sum(event.get("desired_by_lod", [0] * 9)[:3]) > 0), None)
return {
"condition": condition,
"moving_samples": len(moving),
"dwell_samples": len(dwell),
"effective_tile_slots": backend.get("effective_tile_slots"),
"frame_resolution_arrival_s": float(arrival["t_s"]),
"frame_resolution_arrival_frame": int(arrival.get("frame", -1)),
"streaming_sample_arrival_s": (float(settled_sample["t_s"])
if settled_sample else None),
"near_readiness_moving_mean_pct": statistics.mean(near),
"far_holes_moving_median_pct": statistics.median(far_holes),
"tick_p99_ms": percentile([float(frame.get("tick_ms", 0.0)) for frame in frames]),
"tick_worst_ms": max(float(frame.get("tick_ms", 0.0)) for frame in frames),
"early_admission_wait_for_critical": expected_wait,
"early_admission_apply_budget_ms": expected_budget_ms,
"trace": str(trace.relative_to(ROOT)),
"wall_s": round(wall_s, 3),
}
def run_trial(name: str, batch: str, condition: str, root: Path, binary: Path,
wait: int, apply_ms: float,
include_switch_overrides: bool = True) -> dict[str, Any]:
switch = switch_plugin_root(root)
identity = root_info(root)
assert_shared_root(root)
if not binary.is_file():
raise FileNotFoundError(binary)
binary_hash = sha256_file(binary)
shutil.copy2(binary, HOST_PLUGIN_BINARY)
run_dir = UE_OUT / batch / condition
run_dir.mkdir(parents=True, exist_ok=True)
export = HOST / "Plugins/VoxelForge/Saved/Linux/l172/export" / batch / condition / name
userdir = HOST / "Saved/Linux/l172/user" / batch / condition / name
zendata = HOST / "Saved/Linux/l172/zen" / batch / condition / name
ddc = HOST / "Saved/Linux/l172/DDC"
for directory in (export, userdir, zendata, ddc):
directory.mkdir(parents=True, exist_ok=True)
trace = run_dir / f"{name}.trace.jsonl"
log = run_dir / f"{name}.log"
stdout = run_dir / f"{name}.stdout.log"
manifest_path = run_dir / f"{name}.manifest.json"
trace.unlink(missing_ok=True)
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWN_M)
command = [
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
"-notraceserver", "-nocrashreports", "-DisablePlugins=AndroidFileServer",
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
"-archetype=TunnelNetwork", "-slot=0", "-modes=export", "-gameconfig=1",
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0",
"-coreplantest=0", "-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings",
"-corehybridmesher=0", f"-out={export}",
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=3000.0",
f"-voxel.TestMoveStartSeconds={MOVE_START_S:.1f}",
f"-voxel.TestMoveDurationSeconds={MOVE_DURATION_S:.1f}",
"-voxel.TestExitSeconds=0.0", "-voxel.TestMoveDirection=1,0,0",
"-voxel.TestExitOnSteady=0", "-coreworldstreamtrace=1",
f"-voxel.StartupTraceSpawnCm={spawn_cm}",
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
f"-abslog={log}",
]
if include_switch_overrides:
command.extend((
f"-voxel.StreamingFastTravelWaitForAllCriticalTiles={wait}",
f"-voxel.StreamingFastTravelApplyBudgetMilliseconds={apply_ms:.1f}",
))
manifest = {
**identity,
"created_utc": utc_now(),
"batch": batch,
"condition": condition,
"run": name,
"active_links": switch["links"],
"intermediates_moved_on_tree_switch": switch["intermediates_moved"],
"binary": str(binary),
"binary_sha256": binary_hash,
"staged_host_binary_sha256": sha256_file(HOST_PLUGIN_BINARY),
"expected_switches": {"wait_for_all_critical_tiles": wait,
"apply_budget_ms": apply_ms},
"switch_source": ("command_line_overrides" if include_switch_overrides
else "compiled_defaults"),
"command": command,
"trace": str(trace),
"unreal_log": str(log),
}
write_json(manifest_path, manifest)
started = time.monotonic()
with stdout.open("w", encoding="utf-8") as stream:
result = subprocess.run(["taskset", "-c", "0-11", "timeout", "1740s", *command],
cwd=ROOT, stdout=stream, stderr=subprocess.STDOUT,
check=False, timeout=1740)
wall_s = time.monotonic() - started
try:
row = summarize_trial(condition, trace, result, wall_s, wait, apply_ms)
row["run"] = name
manifest["result"] = {"exit_code": result.returncode, "wall_s": round(wall_s, 3),
"status": "PASS", "owner_world_ran": True}
row["manifest"] = str(manifest_path.relative_to(ROOT))
write_json(manifest_path, manifest)
return row
except Exception as error:
manifest["result"] = {"exit_code": result.returncode, "wall_s": round(wall_s, 3),
"status": "FAIL", "error": str(error)}
write_json(manifest_path, manifest)
raise
def frame_arrival_from_trace(path: Path) -> float | None:
event = first_frame_arrival(read_events(path))
return float(event["t_s"]) if event is not None else None
def rescore_saved_traces() -> dict[str, Any]:
result: dict[str, Any] = {"metric": "first scheduler_admission with "
"sum(desired_missing_by_lod[0:3]) == 0 and "
"sum(pending_by_lod[0:3]) == 0; t_s at that scheduler frame"}
for round_no in ("169", "170"):
root = ROOT / f"Saved/Linux/l{round_no}/ue/movement"
arms = {}
for arm in ("baseline", "candidate"):
values = {}
for trace in sorted((root / arm).glob("pair-*.trace.jsonl")):
arrival = frame_arrival_from_trace(trace)
if arrival is not None:
number = int(re.search(r"pair-(\d+)", trace.name).group(1))
values[number] = {"arrival_s": arrival, "trace": str(trace.relative_to(ROOT))}
arms[arm] = values
pairs = []
for number in sorted(arms["baseline"].keys() & arms["candidate"].keys()):
delta = arms["candidate"][number]["arrival_s"] - arms["baseline"][number]["arrival_s"]
pairs.append({"pair": number, "delta_s_candidate_minus_baseline": delta,
"candidate_slower": delta > 0,
"baseline": arms["baseline"][number],
"candidate": arms["candidate"][number]})
deltas = [pair["delta_s_candidate_minus_baseline"] for pair in pairs]
result[round_no] = {
"pairs": pairs,
"baseline_median_arrival_s": statistics.median(
[value["arrival_s"] for value in arms["baseline"].values()])
if arms["baseline"] else None,
"candidate_median_arrival_s": statistics.median(
[value["arrival_s"] for value in arms["candidate"].values()])
if arms["candidate"] else None,
"paired_median_delta_ms": 1000.0 * statistics.median(deltas) if deltas else None,
"candidate_slower_pairs": sum(delta > 0 for delta in deltas),
"pair_count": len(pairs),
}
path = OUT / "arrival-rescore-l169-l170.json"
write_json(path, result)
print("[L172ArrivalRescore] " + json.dumps({
key: {metric: value for metric, value in data.items() if metric != "pairs"}
for key, data in result.items() if key in ("169", "170")}, sort_keys=True), flush=True)
return result
def percentile_nearest_rank(values: list[float], fraction: float) -> float:
ordered = sorted(values)
if not ordered:
raise ValueError("percentile requires values")
return ordered[max(0, min(len(ordered) - 1, math.ceil(len(ordered) * fraction) - 1))]
def paired_deltas(rows: list[dict[str, Any]], first: str, second: str,
metric: str) -> list[dict[str, Any]]:
by_pair: dict[int, dict[str, dict[str, Any]]] = {}
for row in rows:
pair_match = re.search(r"pair-(\d+)", row["run"])
if not pair_match:
continue
pair = int(pair_match.group(1))
by_pair.setdefault(pair, {})[row["condition"]] = row
result = []
for pair, arms in sorted(by_pair.items()):
if first not in arms or second not in arms:
continue
a, b = arms[first][metric], arms[second][metric]
result.append({"pair": pair, "first": first, "second": second,
"delta_s_second_minus_first": b - a,
"first_value": a, "second_value": b})
return result
def run_batch_arm(rows: list[dict[str, Any]], batch: str, condition: str,
root: Path, binary: Path, wait: int, apply_ms: float,
run_name: str) -> dict[str, Any]:
row = run_trial(run_name, batch, condition, root, binary, wait, apply_ms)
rows.append(row)
write_json(UE_OUT / batch / "runs-partial.json", rows)
return row
def load_partial_rows(batch: str) -> list[dict[str, Any]]:
path = UE_OUT / batch / "runs-partial.json"
if not path.is_file():
return []
rows = json.loads(path.read_text(encoding="utf-8"))
if not isinstance(rows, list):
raise RuntimeError(f"invalid partial rows file: {path}")
return rows
def has_run(rows: list[dict[str, Any]], condition: str, run_name: str) -> bool:
return any(row.get("condition") == condition and row.get("run") == run_name
for row in rows)
def require_quiet_machine() -> dict[str, Any]:
uptime = subprocess.run(["uptime"], text=True, capture_output=True, check=False).stdout.strip()
load = os.getloadavg()
process_list = subprocess.run(["ps", "-eo", "args="], text=True, capture_output=True,
check=False).stdout.splitlines()
busy = [line.strip() for line in process_list
if any(token in line for token in ("UnrealBuildTool", "Build.sh", "UnrealEditor-Cmd"))]
record = {"uptime": uptime, "load_1_5_15": load, "other_build_or_editor_processes": busy,
"checked_utc": utc_now()}
if load[0] >= 1.5 or load[1] >= 1.5 or busy:
raise RuntimeError(f"machine is not quiet enough for timing: {record}")
return record
def run_matrix() -> int:
builds_path = OUT / "builds.json"
if not builds_path.is_file():
raise RuntimeError("missing build manifests; run this script prepare first")
builds = json.loads(builds_path.read_text(encoding="utf-8"))
fixed_root = Path(builds["fixed"]["plugin_root"])
baseline_root = Path(builds["ca0bf76"]["plugin_root"])
fixed_binary = Path(builds["fixed"]["binary"])
baseline_binary = Path(builds["ca0bf76"]["binary"])
for build in (builds["fixed"], builds["ca0bf76"]):
if not build.get("compiled_voxel_world_cpp"):
raise RuntimeError("build manifest lacks a full VoxelWorld.cpp compile")
sizes = {item["byte_size"] for item in build["dwarf_pre_run_gate"]["objects"].values()}
if len(sizes) != 1:
raise RuntimeError(f"saved DWARF gate failed: {build['plugin_root']} {sizes}")
if sha256_file(fixed_binary) != builds["fixed"]["binary_sha256"]:
raise RuntimeError("fixed binary does not match its build manifest")
if sha256_file(baseline_binary) != builds["ca0bf76"]["binary_sha256"]:
raise RuntimeError("ca0bf76 binary does not match its build manifest")
quiet = require_quiet_machine()
write_json(OUT / "timing-machine.json", quiet)
all_rows: dict[str, list[dict[str, Any]]] = {
"aa": load_partial_rows("aa"),
"off_vs_ca0bf76": load_partial_rows("off-vs-ca0bf76"),
"on_vs_off": load_partial_rows("on-vs-off"),
}
try:
# A/A arms use the exact same staged fixed binary and the same CVar values.
aa = all_rows["aa"]
for arm in ("aa-a", "aa-b"):
if not has_run(aa, arm, "warmup"):
run_batch_arm(aa, "aa", arm, fixed_root, fixed_binary, WAIT_OFF, APPLY_OFF_MS,
"warmup")
for pair in range(1, 11):
for condition in ("aa-a", "aa-b"):
if not has_run(aa, condition, f"pair-{pair}"):
run_batch_arm(aa, "aa", condition, fixed_root, fixed_binary,
WAIT_OFF, APPLY_OFF_MS, f"pair-{pair}")
aa_pairs = paired_deltas(aa, "aa-a", "aa-b", "frame_resolution_arrival_s")
aa_abs_p90_ms = 1000.0 * percentile_nearest_rank(
[abs(item["delta_s_second_minus_first"]) for item in aa_pairs], 0.90)
aa_summary = {"pairs": aa_pairs, "pair_count": len(aa_pairs),
"absolute_delta_p90_ms": aa_abs_p90_ms}
write_json(UE_OUT / "aa" / "summary.json", aa_summary)
off_vs_base = all_rows["off_vs_ca0bf76"]
for condition, root, binary in (("fixed-off", fixed_root, fixed_binary),
("ca0bf76", baseline_root, baseline_binary)):
if not has_run(off_vs_base, condition, "warmup"):
run_batch_arm(off_vs_base, "off-vs-ca0bf76", condition, root, binary,
WAIT_OFF, APPLY_OFF_MS, "warmup")
for pair in range(1, 11):
control_order = (("fixed-off", "ca0bf76") if pair % 2
else ("ca0bf76", "fixed-off"))
for condition in control_order:
if has_run(off_vs_base, condition, f"pair-{pair}"):
continue
root, binary = ((fixed_root, fixed_binary) if condition == "fixed-off"
else (baseline_root, baseline_binary))
run_batch_arm(off_vs_base, "off-vs-ca0bf76", condition, root, binary,
WAIT_OFF, APPLY_OFF_MS, f"pair-{pair}")
control_pairs = paired_deltas(off_vs_base, "ca0bf76", "fixed-off",
"frame_resolution_arrival_s")
control_slower = sum(item["delta_s_second_minus_first"] > 0 for item in control_pairs)
control_summary = {"pairs": control_pairs, "pair_count": len(control_pairs),
"fixed_off_slower_pairs": control_slower,
"stop_at_eight_slower": control_slower >= 8}
write_json(UE_OUT / "off-vs-ca0bf76" / "summary.json", control_summary)
if control_slower >= 8:
final = {"status": "STOPPED_CONTROL_FAILED", "timing_machine": quiet,
"aa": aa_summary, "fixed_off_vs_ca0bf76": control_summary,
"fixed_on_vs_off": None,
"reason": "fixed-off was slower than true ca0bf76 in at least 8/10 pairs"}
write_json(UE_OUT / "decision-summary.json", final)
print("[L172Decision] " + json.dumps({key: value for key, value in final.items()
if key not in ("aa", "fixed_off_vs_ca0bf76")},
sort_keys=True), flush=True)
return 0
on_vs_off = all_rows["on_vs_off"]
for condition, wait, budget in (("fixed-on", WAIT_ON, APPLY_ON_MS),
("fixed-off", WAIT_OFF, APPLY_OFF_MS)):
if not has_run(on_vs_off, condition, "warmup"):
run_batch_arm(on_vs_off, "on-vs-off", condition, fixed_root, fixed_binary,
wait, budget, "warmup")
settings = {"fixed-on": (WAIT_ON, APPLY_ON_MS),
"fixed-off": (WAIT_OFF, APPLY_OFF_MS)}
for pair in range(1, 11):
on_off_order = (("fixed-on", "fixed-off") if pair % 2
else ("fixed-off", "fixed-on"))
for condition in on_off_order:
if has_run(on_vs_off, condition, f"pair-{pair}"):
continue
wait, budget = settings[condition]
run_batch_arm(on_vs_off, "on-vs-off", condition, fixed_root, fixed_binary,
wait, budget, f"pair-{pair}")
promotion_pairs = paired_deltas(on_vs_off, "fixed-off", "fixed-on",
"frame_resolution_arrival_s")
arrival_deltas_ms = [1000.0 * item["delta_s_second_minus_first"]
for item in promotion_pairs]
paired_median_delta_ms = statistics.median(arrival_deltas_ms)
slower_pairs = sum(value > 0 for value in arrival_deltas_ms)
tolerance_ms = max(aa_abs_p90_ms, 17.0)
fixed_off_rows = [row for row in on_vs_off if row["condition"] == "fixed-off"
and row["run"].startswith("pair-")]
fixed_on_rows = [row for row in on_vs_off if row["condition"] == "fixed-on"
and row["run"].startswith("pair-")]
near_gain = (statistics.median(row["near_readiness_moving_mean_pct"]
for row in fixed_on_rows)
- statistics.median(row["near_readiness_moving_mean_pct"]
for row in fixed_off_rows))
far_change = (statistics.median(row["far_holes_moving_median_pct"]
for row in fixed_on_rows)
- statistics.median(row["far_holes_moving_median_pct"]
for row in fixed_off_rows))
moving_gate_rows = [row for row in on_vs_off if row["run"].startswith("pair-")]
gates = {
"arrival_median_delta_within_tolerance": paired_median_delta_ms <= tolerance_ms,
"arrival_candidate_slower_in_at_most_6_of_10": slower_pairs <= 6,
"near_gain_at_least_9_6_pp": near_gain >= 9.6,
"far_hole_change_at_most_minus_1_35_pp": far_change <= -1.35,
"on_off_moving_runs_p99_at_most_11_ms": all(
row["tick_p99_ms"] is not None and row["tick_p99_ms"] <= 11.0
for row in moving_gate_rows),
"on_off_moving_runs_worst_at_most_14_ms": all(
row["tick_worst_ms"] is not None and row["tick_worst_ms"] <= 14.0
for row in moving_gate_rows),
}
on_off_summary = {
"pairs": promotion_pairs,
"paired_median_delta_ms": paired_median_delta_ms,
"candidate_slower_pairs": slower_pairs,
"arrival_tolerance_ms": tolerance_ms,
"aa_absolute_delta_p90_ms": aa_abs_p90_ms,
"near_gain_pp": near_gain,
"far_hole_change_pp": far_change,
"gates": gates,
}
write_json(UE_OUT / "on-vs-off" / "summary.json", on_off_summary)
promote = all(gates.values()) and not control_summary["stop_at_eight_slower"]
final = {
"status": "PROMOTE" if promote else "KEEP_DEFAULTS_OFF",
"timing_machine": quiet,
"aa": aa_summary,
"fixed_off_vs_ca0bf76": control_summary,
"fixed_on_vs_off": on_off_summary,
"moving_metrics_medians": {
"fixed_off_near_pct": statistics.median(
row["near_readiness_moving_mean_pct"] for row in fixed_off_rows),
"fixed_on_near_pct": statistics.median(
row["near_readiness_moving_mean_pct"] for row in fixed_on_rows),
"fixed_off_far_holes_pct": statistics.median(
row["far_holes_moving_median_pct"] for row in fixed_off_rows),
"fixed_on_far_holes_pct": statistics.median(
row["far_holes_moving_median_pct"] for row in fixed_on_rows),
"fixed_on_p99_ms_max": max(row["tick_p99_ms"] for row in fixed_on_rows),
"fixed_on_worst_tick_ms_max": max(row["tick_worst_ms"] for row in fixed_on_rows),
},
"promotion_gates": gates,
"all_runs": all_rows,
"switch_defaults": {
"voxel.StreamingFastTravelWaitForAllCriticalTiles": 0 if promote else 1,
"voxel.StreamingFastTravelApplyBudgetMilliseconds": 1.0 if promote else 0.0,
"voxel.StreamingIncrementalDesiredSet": 1,
"voxel.StreamingFastTravelForwardFirst": 0,
"voxel.StreamingRebuildOnCoreRegionCompletion": 1,
"voxel.StreamingCoreDesiredCacheByCenter": 0,
"voxel.StreamingIncrementalDesiredAuditEvery": 0,
"voxel.StreamingIncrementalDesiredMaterializationAuditEvery": 0,
"voxel.StreamingIncrementalDesiredAuditInjectMismatch": 0,
},
}
write_json(UE_OUT / "decision-summary.json", final)
print("[L172Decision] " + json.dumps({
"status": final["status"], "aa_abs_p90_ms": aa_abs_p90_ms,
"arrival_median_delta_ms": paired_median_delta_ms,
"arrival_tolerance_ms": tolerance_ms, "slower_pairs": slower_pairs,
"near_gain_pp": near_gain, "far_hole_change_pp": far_change,
"gates": gates}, sort_keys=True), flush=True)
return 0
finally:
restored = restore_main_checkout()
write_json(OUT / "restored-main-links.json", {
"time_utc": utc_now(), "main_root": str(MAIN_CHECKOUT.resolve()),
"links": restored["links"], "intermediates_moved": restored["intermediates_moved"]})
def run_default_smoke() -> int:
build = json.loads((OUT / "build-fixed-default-on.json").read_text(encoding="utf-8"))
try:
row = run_trial("default-smoke", "default-on-smoke", "fixed-on", ROOT,
Path(build["binary"]), WAIT_ON, APPLY_ON_MS,
include_switch_overrides=False)
write_json(UE_OUT / "default-on-smoke" / "summary.json", row)
print("[L172DefaultSmoke] " + json.dumps(row, sort_keys=True), flush=True)
return 0
finally:
restored = restore_main_checkout()
write_json(OUT / "restored-main-links.json", {
"time_utc": utc_now(), "main_root": str(MAIN_CHECKOUT.resolve()),
"links": restored["links"],
"intermediates_moved": restored["intermediates_moved"]})
def main() -> int:
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(dest="command", required=True)
subparsers.add_parser("prepare", help="make a fresh ca0bf76 worktree and clean UE builds")
subparsers.add_parser("rescore", help="rescore saved L169 and L170 traces by scheduler frame")
subparsers.add_parser("matrix", help="run the L172 A/A and counterbalanced 10-pair matrices")
subparsers.add_parser("default-smoke", help="run the built default-on binary without CVar overrides")
args = parser.parse_args()
if args.command == "prepare":
return prepare()
if args.command == "rescore":
rescore_saved_traces()
return 0
if args.command == "matrix":
return run_matrix()
if args.command == "default-smoke":
return run_default_smoke()
return 2
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,59 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import os
from pathlib import Path
import shutil
import subprocess
ROOT = Path(__file__).resolve().parents[4]
SOURCE = ROOT / "Source/VoxelForge/Private/VoxelWorld.cpp"
HEADER_DIR = ROOT / "Source/VoxelForge/Private"
TEST_SOURCE = Path(__file__).with_name("TestStreamingAdmissionOrder.cpp")
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--out", type=Path, required=True)
args = parser.parse_args()
output = args.out.resolve()
output.mkdir(parents=True, exist_ok=True)
source = SOURCE.read_text(encoding="utf-8")
required = (
"VoxelStreamingAdmissionOrder::StableSortCandidates(",
"LevelTiles.GetData(), LevelTiles.GetData() + LevelTiles.Num()",
"VoxelStreamingAdmissionOrder::AdmitInOrder(LevelTiles",
)
if any(token not in source for token in required):
raise SystemExit("VoxelWorld.cpp is not using the tested sort/admission helpers")
if ("LevelSubmissionCount" in source or "LevelTiles.SetNum" in source
or "std::partial_sort(" in source):
raise SystemExit("pass-start truncation returned to the production scheduler")
compiler = os.environ.get("CXX") or shutil.which("clang++-19") or shutil.which("clang++")
if not compiler:
raise SystemExit("clang++ is required for the streaming admission regression")
executable = output / "StreamingAdmissionOrderTest"
build_log = output / "streaming-admission-order-build.log"
run_log = output / "streaming-admission-order.log"
command = [compiler, "-std=c++20", "-O1", "-Wall", "-Wextra", "-Werror",
f"-I{HEADER_DIR}", str(TEST_SOURCE), "-o", str(executable)]
built = subprocess.run(command, cwd=ROOT, text=True, capture_output=True, check=False)
build_log.write_text(built.stdout + built.stderr, encoding="utf-8")
if built.returncode != 0:
raise SystemExit(f"admission regression compile failed; see {build_log}")
ran = subprocess.run([str(executable)], cwd=ROOT, text=True, capture_output=True,
check=False)
run_log.write_text(ran.stdout + ran.stderr, encoding="utf-8")
if ran.returncode != 0:
raise SystemExit(f"admission regression failed; see {run_log}")
print(ran.stdout.strip())
print(f"[StreamingAdmissionOrder] production_source_guard=PASS build_log={build_log}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,97 @@
#include "VoxelStreamingAdmissionOrder.h"
#include <algorithm>
#include <cstdlib>
#include <iostream>
#include <vector>
struct FCandidate
{
int Priority;
int Id;
int Level;
};
static bool LessCandidate(const FCandidate& A, const FCandidate& B)
{
if (A.Priority != B.Priority) return A.Priority < B.Priority;
return A.Id < B.Id;
}
static std::vector<int> RunPass(std::vector<FCandidate>& Candidates,
int InitiallyActive, int Capacity)
{
int Active = InitiallyActive;
bool FreedOneSlot = false;
std::vector<int> Submitted;
VoxelStreamingAdmissionOrder::AdmitInOrder(
Candidates,
[&]() { return Active >= Capacity; },
[&](const FCandidate& Candidate)
{
if (Candidate.Level != 0)
{
std::cerr << "candidate escaped its level pass\n";
std::exit(2);
}
Submitted.push_back(Candidate.Id);
++Active;
if (!FreedOneSlot && Submitted.size() == 1)
{
--Active; // A worker finishes one task while this level pass is still running.
FreedOneSlot = true;
}
},
[]() {});
return Submitted;
}
int main()
{
const std::vector<FCandidate> Input = {
{4, 4, 0}, {0, 0, 0}, {3, 3, 0}, {1, 1, 0}, {2, 2, 0}};
constexpr int InitiallyActive = 1;
constexpr int Capacity = 2;
const int PassStartFreeSlots = Capacity - InitiallyActive;
std::vector<FCandidate> FullSort = Input;
std::stable_sort(FullSort.begin(), FullSort.end(), LessCandidate);
const std::vector<int> Expected = {FullSort[0].Id, FullSort[1].Id};
std::vector<FCandidate> FixedCandidates = Input;
VoxelStreamingAdmissionOrder::StableSortCandidates(
FixedCandidates.begin(), FixedCandidates.end(), LessCandidate);
const std::vector<int> FixedSubmitted =
RunPass(FixedCandidates, InitiallyActive, Capacity);
if (FixedSubmitted != Expected || FixedSubmitted.size() != 2
|| FixedCandidates[1].Level != FixedCandidates[0].Level)
{
std::cerr << "fixed submission order/refill mismatch\n";
return 1;
}
// Negative control: the old pass-start partial_sort + SetNum truncation loses the
// second candidate even though a slot frees after the first submission.
std::vector<FCandidate> OldCandidates = Input;
const std::size_t InitialCount = std::min(
OldCandidates.size(), static_cast<std::size_t>(PassStartFreeSlots));
std::partial_sort(OldCandidates.begin(), OldCandidates.begin() + InitialCount,
OldCandidates.end(), LessCandidate);
OldCandidates.resize(InitialCount);
const std::vector<int> OldSubmitted =
RunPass(OldCandidates, InitiallyActive, Capacity);
if (OldSubmitted == Expected)
{
std::cerr << "negative control did not reproduce the lost refill\n";
return 1;
}
std::cout << "[StreamingAdmissionOrder] full_order=";
for (const int Id : Expected) std::cout << Id;
std::cout << " fixed_submissions=";
for (const int Id : FixedSubmitted) std::cout << Id;
std::cout << " old_submissions=";
for (const int Id : OldSubmitted) std::cout << Id;
std::cout << " same_level_refill=PASS negative_control=PASS status=PASS\n";
return 0;
}
+15
View File
@@ -3230,6 +3230,20 @@ def _tier_deadline(_signum: int, _frame: object) -> None:
f"{ARGS.tier} tier exceeded its {ARGS.overall_timeout}s overall timeout") f"{ARGS.tier} tier exceeded its {ARGS.overall_timeout}s overall timeout")
def run_streaming_admission_order_regression(runner: VerificationRunner) -> None:
script = ROOT / "Standalone/Research/WorldAssemblerM3/Linux172/RunStreamingAdmissionOrderTest.py"
log = OUT / "streaming-admission-order.log"
command = [sys.executable, str(script), "--out", str(OUT / "streaming-admission-order")]
with log.open("w", encoding="utf-8") as stream:
result = subprocess.run(command, cwd=ROOT, stdout=stream, stderr=subprocess.STDOUT,
check=False, timeout=120)
output = log.read_text(encoding="utf-8", errors="replace")
runner.require("streaming-admission-order-mid-pass-refill",
result.returncode == 0 and "same_level_refill=PASS" in output
and "negative_control=PASS" in output,
f"log={log}")
def main() -> int: def main() -> int:
runner = VerificationRunner(ARGS.tier) runner = VerificationRunner(ARGS.tier)
started = time.monotonic() started = time.monotonic()
@@ -3251,6 +3265,7 @@ def main() -> int:
if ARGS.tier == "sanitize": if ARGS.tier == "sanitize":
run_sanitizer_stage(runner) run_sanitizer_stage(runner)
elif ARGS.tier == "fast": elif ARGS.tier == "fast":
run_streaming_admission_order_regression(runner)
owners = run_owner_passages(runner, (CLANG,)) owners = run_owner_passages(runner, (CLANG,))
new_plan_failure = None new_plan_failure = None
try: try:
+5
View File
@@ -31,6 +31,11 @@
"Type": "Runtime", "Type": "Runtime",
"LoadingPhase": "Default" "LoadingPhase": "Default"
}, },
{
"Name": "VoxelForgeLivingPlane",
"Type": "Runtime",
"LoadingPhase": "Default"
},
{ {
"Name": "VoxelForgeEditor", "Name": "VoxelForgeEditor",
"Type": "Editor", "Type": "Editor",