Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3b71d35eba | ||
|
|
e1d87eed26 | ||
|
|
5b15595e4a | ||
|
|
eb47d72258 | ||
|
|
95bf654fbc | ||
|
|
0065ee68e6 | ||
|
|
ebae71b142 | ||
|
|
81071969fc | ||
|
|
cbbfd46bd3 | ||
|
|
044cd6d9c5 | ||
|
|
b96dd739b8 | ||
|
|
4041d4bb4b | ||
|
|
aeb60bbda3 | ||
|
|
c743f2abc7 | ||
|
|
5e7e0eb1c1 | ||
|
|
bae1144ee4 | ||
|
|
5c519f5206 | ||
|
|
161565936c |
No files matched your search
@@ -9,12 +9,20 @@
|
|||||||
!/Source/
|
!/Source/
|
||||||
!/Source/**
|
!/Source/**
|
||||||
|
|
||||||
|
# Plugin runtime shader source.
|
||||||
|
!/Shaders/
|
||||||
|
!/Shaders/**
|
||||||
|
|
||||||
# Portable CPU generator and D3D11 viewer.
|
# Portable CPU generator and D3D11 viewer.
|
||||||
!/Standalone/
|
!/Standalone/
|
||||||
!/Standalone/**
|
!/Standalone/**
|
||||||
/Standalone/Build/
|
/Standalone/Build/
|
||||||
/Standalone/bin/
|
/Standalone/bin/
|
||||||
|
|
||||||
|
# Research and prototype experiments.
|
||||||
|
!/Experiments/
|
||||||
|
!/Experiments/**
|
||||||
|
|
||||||
# The guarded single-entry test harness is source, not generated output.
|
# The guarded single-entry test harness is source, not generated output.
|
||||||
!/Tools/
|
!/Tools/
|
||||||
!/Tools/**
|
!/Tools/**
|
||||||
|
|||||||
@@ -0,0 +1,57 @@
|
|||||||
|
# AGENTS.md — VoxelForge (read automatically by Codex)
|
||||||
|
|
||||||
|
You are the implementation agent. The orchestrator (Claude) writes each round's brief, reviews your diff against the
|
||||||
|
code, and commits. The owner decides direction. Be warm and direct; credit what worked; say plainly what didn't.
|
||||||
|
|
||||||
|
Read first, as the task needs: `CURRENT-STATE.md` (what ships, reference numbers, **closed experiments: don't reopen
|
||||||
|
them without new evidence**), then `CODEMAP.md` and `ARCHITECTURE.md`. Round reports live in
|
||||||
|
`Standalone/Research/WorldAssemblerM3/Linux*/REPORT.md`.
|
||||||
|
|
||||||
|
## Standing rules (every round, unless the brief says otherwise)
|
||||||
|
|
||||||
|
- Run as user `jahni`. Write only inside this plugin, plus `/home/jahni/VFHost` and `/home/jahni/UnrealEngine` when
|
||||||
|
the brief asks for a UE build. No installs, no system changes.
|
||||||
|
- **No `git commit`, `push` or `stash`.** Leave changes in the working tree; the orchestrator reviews and commits.
|
||||||
|
- Headless processes and builds: `taskset -c 0-11 prlimit --as=6442450944` (12 P-core threads, 6 GiB). Never use all
|
||||||
|
cores.
|
||||||
|
- Unreal (UBT, the editor, commandlets): `taskset -c 0-11` only, **never `prlimit`** (CoreCLR fails). UE builds use
|
||||||
|
`-NoUBA -MaxParallelActions=4`, never while timing.
|
||||||
|
- Stop any single run that passes 30 minutes. Logs go in `Saved/Linux/l<round>/`.
|
||||||
|
- **Timing:** quiet machine only (check `uptime` first; load < ~1.5, nothing else building). L80 protocol: one warm-up
|
||||||
|
per binary, ≥5 interleaved runs, W8 and W12, all four owner layers. Every number states the window that produced it.
|
||||||
|
- **Same world for everyone:** a seed must give the same world on every machine and worker count (Clang and GCC,
|
||||||
|
W1/W8/W12). Byte-identity with older versions is NOT required. If the output changes: bump
|
||||||
|
`WorldAssemblyEvaluatorVersion`, add baselines, and show captures.
|
||||||
|
- Density sign: negative = solid, positive = air. Core voxels are 25 cm. Never mix metres and voxels without an
|
||||||
|
explicit conversion (v225 shipped a reach bug from exactly that).
|
||||||
|
- The fast tier (`Standalone/Tools/verify/verify.py fast`) must stay green. Don't loosen an audit to make a change pass.
|
||||||
|
|
||||||
|
## How to work
|
||||||
|
|
||||||
|
1. **Count before you build.** Measure where the time or the failure actually is (stack sampler, census, counters).
|
||||||
|
State which parts are measured and which are inferred.
|
||||||
|
2. **Suspect the measurement first.** An alarming or contradictory number is the instrument until a control rules it
|
||||||
|
out (A/A runs, an ablation that verifiably removes the work).
|
||||||
|
3. **No fix without a root cause.** Reproduce it, trace it to its source, fix the cause. A guard that hides the symptom
|
||||||
|
is not a fix.
|
||||||
|
4. **One change at a time,** each with its own check.
|
||||||
|
5. **Subtract before you add.** Prefer removing work to reorganising it. The smallest edit that solves the problem.
|
||||||
|
6. **Attack the premise.** When two fixes sharing one premise fail the same gate, write that premise in one sentence
|
||||||
|
and question it before trying a third. **After three failed fixes, stop:** it is probably the design, not the
|
||||||
|
hypothesis. Report it instead of trying fix #4.
|
||||||
|
7. **Tests must be able to fail.** Name what would make each check fail. Add a negative control when you add an audit.
|
||||||
|
8. **Refactors:** pin the behaviour (hashes), change the structure, prove nothing changed.
|
||||||
|
9. **Verify before you claim.** Don't report a check as passed unless it ran on the final source. Say which checks
|
||||||
|
didn't run and why.
|
||||||
|
10. Changes that speed things up go in the shared layer, so every generator benefits.
|
||||||
|
|
||||||
|
## Report
|
||||||
|
|
||||||
|
Write the round's `REPORT.md`, short and plain:
|
||||||
|
- the result first;
|
||||||
|
- what changed and why;
|
||||||
|
- the measurements, with their windows;
|
||||||
|
- the checks run, and those not run;
|
||||||
|
- remaining risks.
|
||||||
|
|
||||||
|
List any changed Unreal `Source/` files, because the owner compiles them on Windows.
|
||||||
@@ -1522,11 +1522,12 @@ collision ring became walkable at 0.632 s. `voxel.StreamingCoarseFirst=1` with
|
|||||||
residency by about one third on the staged plan, but null-RHI residency does not prove rendered
|
residency by about one third on the staged plan, but null-RHI residency does not prove rendered
|
||||||
enclosure. Keep coarse-first opt-in until that view is checked in a rendered owner-plan run.
|
enclosure. Keep coarse-first opt-in until that view is checked in a rendered owner-plan run.
|
||||||
|
|
||||||
The separate `voxel.StreamingStartupApplyBoost` switch stays default-off. At `1`, it temporarily allows
|
The separate `voxel.StreamingStartupApplyBoost` switch is default-on in the current code. At `1`, it
|
||||||
12 visible mesh submissions / 14 ms while the initial desired set and walkable ring are incomplete,
|
temporarily allows up to 24 visible mesh submissions / 14 ms while the initial desired set and
|
||||||
with a 30 second ceiling; at `0`, the authored budget is unchanged. On the matched legacy-order test,
|
walkable ring are incomplete, with a 30 second ceiling; at `0`, the authored budget is unchanged.
|
||||||
the boost changed full-set readiness from 17.412 s to 17.346 s and removed count-cap frames, but did
|
The historical matched S6 run changed full-set readiness from 17.412 s to 17.346 s and removed
|
||||||
not improve end-to-end residency. Keep the normal 4 / 2 ms guard for play.
|
count-cap frames, but did not improve end-to-end residency. These current defaults are defined in
|
||||||
|
`Source/VoxelForge/Private/VoxelWorld.cpp`; the S6 measurement remains historical context.
|
||||||
|
|
||||||
The S6 trace found the shared pool was not saturated: its 10-slot idle estimate was about 132
|
The S6 trace found the shared pool was not saturated: its 10-slot idle estimate was about 132
|
||||||
slot-seconds on legacy order and 73 on gated coarse-first, while game-thread tile applies totaled only
|
slot-seconds on legacy order and 73 on gated coarse-first, while game-thread tile applies totaled only
|
||||||
@@ -1556,8 +1557,8 @@ by its RAII guard when the worker lambda exits. This still counts queued and run
|
|||||||
results do not consume an admission slot. `voxel.StreamingLegacyPendingResultCap=1` restores the old
|
results do not consume an admission slot. `voxel.StreamingLegacyPendingResultCap=1` restores the old
|
||||||
`PendingTiles` accounting; the test harness's `NearReserveMode=Legacy` also sets
|
`PendingTiles` accounting; the test harness's `NearReserveMode=Legacy` also sets
|
||||||
`voxel.StreamingDynamicNearReserve=0`, reproducing the S6 path. `StreamingDynamicNearReserve` now
|
`voxel.StreamingDynamicNearReserve=0`, reproducing the S6 path. `StreamingDynamicNearReserve` now
|
||||||
defaults to `1`; `StreamingCoarseFirst` remains `0`, and the apply boost remains `0` in the matched
|
defaults to `1`; `StreamingCoarseFirst` remains `0`. That historical S6 matched measurement disabled
|
||||||
measurement.
|
the startup apply boost (`voxel.StreamingStartupApplyBoost=0`); the current default is on (§8.27).
|
||||||
|
|
||||||
On the 2,578-key owner SurfaceWorld trace, the old-path control reached the full set in 17.527 s,
|
On the 2,578-key owner SurfaceWorld trace, the old-path control reached the full set in 17.527 s,
|
||||||
walkable ring in 0.650 s, and full LOD0 in 2.281 s. Dynamic admission reached them in 7.780 s,
|
walkable ring in 0.650 s, and full LOD0 in 2.281 s. Dynamic admission reached them in 7.780 s,
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ then wait for his results / pasted errors. Same for in-editor checks — ask for
|
|||||||
try to run anything.
|
try to run anything.
|
||||||
|
|
||||||
## Read first (in order, load only what the task needs)
|
## Read first (in order, load only what the task needs)
|
||||||
|
- **[CURRENT-STATE.md](CURRENT-STATE.md) — current shipping snapshot and verification entry point.**
|
||||||
- **[CODEMAP.md](CODEMAP.md) — ALWAYS.** Navigation: what it is, data-flow (§2), symbol→`file:line`
|
- **[CODEMAP.md](CODEMAP.md) — ALWAYS.** Navigation: what it is, data-flow (§2), symbol→`file:line`
|
||||||
index (§3), "I want to change X → go here" (§5), conventions & gotchas (§6). Trust **symbol names
|
index (§3), "I want to change X → go here" (§5), conventions & gotchas (§6). Trust **symbol names
|
||||||
over line numbers** (lines drift).
|
over line numbers** (lines drift).
|
||||||
|
|||||||
@@ -388,7 +388,7 @@ casse cette arithmetique *structurellement* — `BeginPlay` loggue une **Error**
|
|||||||
| `AdvanceHeadlessDigMotionTest` / `RecordHeadlessDigMotionFrame` | 3060 / 3217 | Explicit `-voxel.TestDigMotion=1` repro: optional near digs, then repeats underfoot digs and samples `Generator->GetDensityAt` immediately after each edit, on its first subsequent simulated frame, and on every following observation frame. `-voxel.TestDigMotionFieldOverride=1` is a measurement-only prototype that clears `CurrentFloor` and switches CMC to falling when the sampled post-edit density is air; it is inactive by default. The test counts false support, false air and downward velocity beyond one gravity step. `-voxel.TestDigMotionUnderfootDigCount` is clamped to 1–100; `-voxel.TestDigMotionUnderfootIntervalSeconds=0` performs per-frame rapid edits. The accompanying gap audit is scoped to the dig session. |
|
| `AdvanceHeadlessDigMotionTest` / `RecordHeadlessDigMotionFrame` | 3060 / 3217 | Explicit `-voxel.TestDigMotion=1` repro: optional near digs, then repeats underfoot digs and samples `Generator->GetDensityAt` immediately after each edit, on its first subsequent simulated frame, and on every following observation frame. `-voxel.TestDigMotionFieldOverride=1` is a measurement-only prototype that clears `CurrentFloor` and switches CMC to falling when the sampled post-edit density is air; it is inactive by default. The test counts false support, false air and downward velocity beyond one gravity step. `-voxel.TestDigMotionUnderfootDigCount` is clamped to 1–100; `-voxel.TestDigMotionUnderfootIntervalSeconds=0` performs per-frame rapid edits. The accompanying gap audit is scoped to the dig session. |
|
||||||
| `MaybeFinishHeadlessStreamingTest` | — | The existing `voxel.TestExitSeconds` diagnostic emits `VoxelForgeStreamingExitDiag` at most every 30 elapsed seconds, including threshold and early-return test flags; no output when the timer is unconfigured. |
|
| `MaybeFinishHeadlessStreamingTest` | — | The existing `voxel.TestExitSeconds` diagnostic emits `VoxelForgeStreamingExitDiag` at most every 30 elapsed seconds, including threshold and early-return test flags; no output when the timer is unconfigured. |
|
||||||
| `UpdateVisualStalenessAudit` | 5034 | Test-only camera-frustum coverage audit: tracks up to 24 real pre-skirt isosurface points per visible component, then counts frames where no visible tile component covers a previously visible, still-desired point (excluding edited bounds). `voxel.TestVisualGapInjectEarlyHide=1` hides a valid component for three frames and must be caught. `voxel.TestGapAuditLegacyHandoff=1` restores old handoff culling for A/B baselines. |
|
| `UpdateVisualStalenessAudit` | 5034 | Test-only camera-frustum coverage audit: tracks up to 24 real pre-skirt isosurface points per visible component, then counts frames where no visible tile component covers a previously visible, still-desired point (excluding edited bounds). `voxel.TestVisualGapInjectEarlyHide=1` hides a valid component for three frames and must be caught. `voxel.TestGapAuditLegacyHandoff=1` restores old handoff culling for A/B baselines. |
|
||||||
| `ProcessPendingChunks` | 7435 | Drains `CriticalProcessQueue`, `EditedProcessQueue`, `NearProcessQueue`, then `ProcessQueue`; applies under the authored 4-mesh/2 ms budget during normal play and the bounded shared startup budget (12 meshes/14 ms) until the initial desired set and walkable ring are ready. `voxel.StreamingStartupApplyBoost=0` restores the authored budget. `startup_apply_config` traces the actor-read CVar and both budgets; `startup_apply_budget` records activation/end. Reads per-request cancellation before removing its map entry, traces stale/obsolete/aborted discards with epoch and ordinal, then applies via `ApplyTileResult`. Step 1 prequeue still releases here and emits `VFStep1UploadCorpusReleased`. |
|
| `ProcessPendingChunks` | 7435 | Drains `CriticalProcessQueue`, `EditedProcessQueue`, `NearProcessQueue`, then `ProcessQueue`; applies under the authored 4-mesh/2 ms budget during normal play and the default-on bounded startup budget (24 meshes/14 ms, at most 30 s) until the initial desired set and walkable ring are ready. `voxel.StreamingStartupApplyBoost=0` restores the authored budget. `startup_apply_config` traces the actor-read CVar and both budgets; `startup_apply_budget` records activation/end. Reads per-request cancellation before removing its map entry, traces stale/obsolete/aborted discards with epoch and ordinal, then applies via `ApplyTileResult`. Step 1 prequeue still releases here and emits `VFStep1UploadCorpusReleased`. |
|
||||||
| `ApplyTileResult` | — | **Shared game-thread apply** for one `FChunkResult` (async drain + sync carve): discards stale generation epochs and keys no longer desired, marks loaded, ingests capture, EMPTY releases the tile's existing component (a re-gen can flip content→empty on band change — old geometry must not linger), else `ApplyMeshToTile`. Returns true iff a visible mesh uploaded (counts the budget). Doesn't touch `PendingTiles` (caller's). |
|
| `ApplyTileResult` | — | **Shared game-thread apply** for one `FChunkResult` (async drain + sync carve): discards stale generation epochs and keys no longer desired, marks loaded, ingests capture, EMPTY releases the tile's existing component (a re-gen can flip content→empty on band change — old geometry must not linger), else `ApplyMeshToTile`. Returns true iff a visible mesh uploaded (counts the budget). Doesn't touch `PendingTiles` (caller's). |
|
||||||
| `FVoxelStackSampler` / `FScopedVoxelStackRegistration` (`VoxelStackSampler.h/.cpp`) | — | **Diagnostic raw stack sampler, off by default.** A sampler thread captures registered generation workers' stacks (`CaptureThreadStackBackTrace`) every N µs and tags each with the tile's LOD. It writes bounded raw PCs to CSV plus a symbolization-free summary under `Saved/`; symbol resolution is deferred until after process exit so the measured process never initializes/calls DbgHelp for this diagnostic. Lifecycle teardown is serialized and registration TLS is session-tagged. Registration is per tile TASK in `LoadTile` (and the export's tasks), never per sample; disabled costs one relaxed atomic load. Game `-voxel.SampleStacks=<us>`, commandlet `-samplestacks[=us]`. |
|
| `FVoxelStackSampler` / `FScopedVoxelStackRegistration` (`VoxelStackSampler.h/.cpp`) | — | **Diagnostic raw stack sampler, off by default.** A sampler thread captures registered generation workers' stacks (`CaptureThreadStackBackTrace`) every N µs and tags each with the tile's LOD. It writes bounded raw PCs to CSV plus a symbolization-free summary under `Saved/`; symbol resolution is deferred until after process exit so the measured process never initializes/calls DbgHelp for this diagnostic. Lifecycle teardown is serialized and registration TLS is session-tagged. Registration is per tile TASK in `LoadTile` (and the export's tasks), never per sample; disabled costs one relaxed atomic load. Game `-voxel.SampleStacks=<us>`, commandlet `-samplestacks[=us]`. |
|
||||||
| Room-cache performance (2026-09-13): `VF_FindPlayerFitPointForRoomMemoized` / `GPlayerFitMemo` (`VoxelCaveMorphology.cpp`), `voxel.TileCacheWindow` + `VoxelGenLOD::TileOriginVoxels/TileCellsPerAxis` (`VoxelGenerator.h`), `FChunkSDFSpatialIndex` | — | **Player-fit memo:** exact 97-word key, process-wide, `voxel.PlayerFitMemo`. **Tile cache window:** the room cache covers the requesting tile's footprint plus a fixed collect margin. It is set via `TGuardValue` in `GenerateMesh`/classify, and is window-invariant by construction (the margin does not grow with the tile). **LOD-gated:** fused path at LOD ≥ `voxel.TileCacheWindowMinLOD` (3), op-stack at LOD ≥ `voxel.TileCacheWindowOpStackMinLOD` (4); lower LODs keep the 4-chunk worker window, because the tile window slowed LOD0 evaluation ~6%. Strates whose authored terrain ops exceed the proven 256-voxel extent (or are malformed/unloaded) fall back to the legacy window (`VF_ComputeTerrainOpEnvelope`). `voxel.SpatialIndex` is -1 auto (off at fused LOD0, on elsewhere), 0 or 1. **Spatial index:** ordered XY buckets over cache primitives, exact rejection. The `static_assert` in `VoxelStrateTypes.h` ties `VF_STRATE_PARAM_FIELDS` to `FStrateGenerationParams` (83 fields + 8 B padding). |
|
| Room-cache performance (2026-09-13): `VF_FindPlayerFitPointForRoomMemoized` / `GPlayerFitMemo` (`VoxelCaveMorphology.cpp`), `voxel.TileCacheWindow` + `VoxelGenLOD::TileOriginVoxels/TileCellsPerAxis` (`VoxelGenerator.h`), `FChunkSDFSpatialIndex` | — | **Player-fit memo:** exact 97-word key, process-wide, `voxel.PlayerFitMemo`. **Tile cache window:** the room cache covers the requesting tile's footprint plus a fixed collect margin. It is set via `TGuardValue` in `GenerateMesh`/classify, and is window-invariant by construction (the margin does not grow with the tile). **LOD-gated:** fused path at LOD ≥ `voxel.TileCacheWindowMinLOD` (3), op-stack at LOD ≥ `voxel.TileCacheWindowOpStackMinLOD` (4); lower LODs keep the 4-chunk worker window, because the tile window slowed LOD0 evaluation ~6%. Strates whose authored terrain ops exceed the proven 256-voxel extent (or are malformed/unloaded) fall back to the legacy window (`VF_ComputeTerrainOpEnvelope`). `voxel.SpatialIndex` is -1 auto (off at fused LOD0, on elsewhere), 0 or 1. **Spatial index:** ordered XY buckets over cache primitives, exact rejection. The `static_assert` in `VoxelStrateTypes.h` ties `VF_STRATE_PARAM_FIELDS` to `FStrateGenerationParams` (83 fields + 8 B padding). |
|
||||||
@@ -396,7 +396,7 @@ casse cette arithmetique *structurellement* — `BeginPlay` loggue une **Error**
|
|||||||
| `RecordTileHash` / `WriteTileHashDump` | — | Headless `-voxel.TileHashDump=<absolute path>` parity instrument. Records the exact `FVoxelMeshData` used by the streaming result, including empty/classified tiles, then writes sorted absolute tile keys, LOD step/cells, content-band limits, settings digest context, and the shared `VoxelGeometryHash` at `EndPlay`. |
|
| `RecordTileHash` / `WriteTileHashDump` | — | Headless `-voxel.TileHashDump=<absolute path>` parity instrument. Records the exact `FVoxelMeshData` used by the streaming result, including empty/classified tiles, then writes sorted absolute tile keys, LOD step/cells, content-band limits, settings digest context, and the shared `VoxelGeometryHash` at `EndPlay`. |
|
||||||
| `SyncRemeshTile` | — | **INSTANT DIG**: level-0 same-frame re-mesh on the game thread. Uses `PreEditDensityCache` when the tile is resident in the bounded local cache, replays the current DiffLayer over that exact procedural lattice, then meshes/applies inline so a carve is visible THIS frame. A cache miss uses normal generation; it captures the unedited lattice if the tile needs meshing, while uniform-tile proofs remain enabled. |
|
| `SyncRemeshTile` | — | **INSTANT DIG**: level-0 same-frame re-mesh on the game thread. Uses `PreEditDensityCache` when the tile is resident in the bounded local cache, replays the current DiffLayer over that exact procedural lattice, then meshes/applies inline so a carve is visible THIS frame. A cache miss uses normal generation; it captures the unedited lattice if the tile needs meshing, while uniform-tile proofs remain enabled. |
|
||||||
| `PreEditDensityCache` / `VoxelForgePreEditDensityCache::BuildEditedGrid` | — | Per-world cache of the pawn's current L0 tile: exact 35³ pre-edit samples, max **171,500 bytes**. It is pruned when the pawn crosses chunk boundaries and cleared on `RegenerateAllChunks`. `BuildEditedGrid` snapshots touched diff chunks once and reproduces `GetDensityAt`'s final `pre-edit - EvaluateMods` operation before the per-call shared-grid mesher path. |
|
| `PreEditDensityCache` / `VoxelForgePreEditDensityCache::BuildEditedGrid` | — | Per-world cache of the pawn's current L0 tile: exact 35³ pre-edit samples, max **171,500 bytes**. It is pruned when the pawn crosses chunk boundaries and cleared on `RegenerateAllChunks`. `BuildEditedGrid` snapshots touched diff chunks once and reproduces `GetDensityAt`'s final `pre-edit - EvaluateMods` operation before the per-call shared-grid mesher path. |
|
||||||
| `UpdateChunksAroundPosition` / `GetGenerationAdmissionCount` | 10990 / 10971 | Shared Core/legacy scheduler. Builds the desired set and delta-culls leavers; admits the occupied/support tile plus collision-required LOD0 tiles in the pawn's 3×3 XY ring as `CollisionCritical`. S5 coarse-first admission stays opt-in (`voxel.StreamingCoarseFirst=1`). `voxel.StreamingDynamicNearReserve=1` is now the default: it keeps the authored near reserve while moving or until the stationary walkable ring is ready, then makes those slots available to coarse work. Admission counts submitted queued/running tasks, not completed results waiting in the MPSC result queues; `voxel.StreamingLegacyPendingResultCap=1` restores the old `PendingTiles`-based cap for the negative control. `PendingTiles` still lasts through game-thread apply for duplicate suppression and result bookkeeping. The clipmap stays non-overlapping and load-before-unload / `TransitionHold` reconciliation remains authoritative. Standalone has no mirror of this actor scheduler. §8.10 + §8.26–8.28. |
|
| `UpdateChunksAroundPosition` / `GetGenerationAdmissionCount` | 10990 / 10971 | Shared Core/legacy scheduler. Builds the desired set and delta-culls leavers; admits the occupied/support tile plus collision-required LOD0 tiles in the pawn's 3×3 XY ring as `CollisionCritical`. S5 coarse-first admission stays opt-in (`voxel.StreamingCoarseFirst=1`). `voxel.StreamingDynamicNearReserve=1` is the default: it keeps the authored near reserve while moving or until the stationary walkable ring is ready, then makes those slots available to coarse work. Admission counts submitted queued/running tasks, not completed results waiting in the MPSC result queues; `voxel.StreamingLegacyPendingResultCap=1` restores the old `PendingTiles`-based cap for the negative control. `voxel.StreamingFarApplySupplement=1` is the default L121 lane: after the visible-apply cap, at most one far result is admitted when under half the apply-time budget and startup boost is inactive. `PendingTiles` still lasts through game-thread apply for duplicate suppression and result bookkeeping. The clipmap stays non-overlapping and load-before-unload / `TransitionHold` reconciliation remains authoritative. Standalone has no mirror of this actor scheduler. §8.10 + §8.26–8.28. |
|
||||||
| `AdvanceHeadlessCollisionGateStressTest` / `ObserveHeadlessCollisionGateStressTest` / `MaybeFinishHeadlessCollisionGateStressTest` / `MeasureHeadlessCollisionGateStressTerrain` | — | Explicit `-voxel.TestCollisionGateStress=1` harness only. The normal spawn-height cvar offsets from the measured density surface; `voxel.TestCollisionGateStressDeepDrop=1` carves a process-local box shaft from above the top to just above the floor tile, extending along the post-landing route so the pawn does not run into the shaft sidewall, temporarily raises only the transient test DiffLayer brush-radius allowance for that fixture, removes only the local pawn's redundant streaming anchor for this isolated repro, proves every sample in two complete outside-ring level-0 tiles is air before seeding their known-empty status, prewarms the known-empty in-ring column, and asserts that the support walk reaches the in-layout floor outside the aligned level-0 render ring before the pawn drops. The log reports whether the fixed gate retains that otherwise out-of-ring target. Other streaming anchors remain active. The fixture rejects a target or floor outside the layout, a non-air empty-tile proof, or a floor inside the aligned level-0 ring. The `GetDensityAt` column at capsule feet defines the local air/solid surface; the deep-drop column crossing is cached after its first exact scan. Each frame records actor position, velocity, direct foot tile and gate support readiness, gate state, signed feet clearance, and a ≤0.5-voxel sweep between positions. PASS requires no measured clearance below −5 cm; deep-drop gate-on also requires horizontal travel during a hold. Result logs include the terminal density column, first crossing below spawn height, hold count/time, and gated horizontal progress. |
|
| `AdvanceHeadlessCollisionGateStressTest` / `ObserveHeadlessCollisionGateStressTest` / `MaybeFinishHeadlessCollisionGateStressTest` / `MeasureHeadlessCollisionGateStressTerrain` | — | Explicit `-voxel.TestCollisionGateStress=1` harness only. The normal spawn-height cvar offsets from the measured density surface; `voxel.TestCollisionGateStressDeepDrop=1` carves a process-local box shaft from above the top to just above the floor tile, extending along the post-landing route so the pawn does not run into the shaft sidewall, temporarily raises only the transient test DiffLayer brush-radius allowance for that fixture, removes only the local pawn's redundant streaming anchor for this isolated repro, proves every sample in two complete outside-ring level-0 tiles is air before seeding their known-empty status, prewarms the known-empty in-ring column, and asserts that the support walk reaches the in-layout floor outside the aligned level-0 render ring before the pawn drops. The log reports whether the fixed gate retains that otherwise out-of-ring target. Other streaming anchors remain active. The fixture rejects a target or floor outside the layout, a non-air empty-tile proof, or a floor inside the aligned level-0 ring. The `GetDensityAt` column at capsule feet defines the local air/solid surface; the deep-drop column crossing is cached after its first exact scan. Each frame records actor position, velocity, direct foot tile and gate support readiness, gate state, signed feet clearance, and a ≤0.5-voxel sweep between positions. PASS requires no measured clearance below −5 cm; deep-drop gate-on also requires horizontal travel during a hold. Result logs include the terminal density column, first crossing below spawn height, hold count/time, and gated horizontal progress. |
|
||||||
| `BuildDesiredTiles` | 6482 | Builds `DesiredSorted`+`DesiredStamped` (player clipmap shells), then adds streaming anchors and the pending player's collision-only support column before the leaver sweep. It orders from the true local pawn position and builds `CriticalDesiredTiles` for the unresolved support target, occupied/support and heading tiles. Also rebuilds `DesiredTransitionDescendants` (ancestor → desired descendants, including face/edge-touching replacements) for the reconcile pass. |
|
| `BuildDesiredTiles` | 6482 | Builds `DesiredSorted`+`DesiredStamped` (player clipmap shells), then adds streaming anchors and the pending player's collision-only support column before the leaver sweep. It orders from the true local pawn position and builds `CriticalDesiredTiles` for the unresolved support target, occupied/support and heading tiles. Also rebuilds `DesiredTransitionDescendants` (ancestor → desired descendants, including face/edge-touching replacements) for the reconcile pass. |
|
||||||
| `CollisionGateDesiredTiles` / `RequestCollisionGateSupportTile` | 5062 | Retains an unresolved support target and up to three level-0 tiles below it across desired-set rebuilds. The additional keys are hidden collision-only tiles, queued ahead of ordinary streaming, and pruned once resolved or outside the active column. |
|
| `CollisionGateDesiredTiles` / `RequestCollisionGateSupportTile` | 5062 | Retains an unresolved support target and up to three level-0 tiles below it across desired-set rebuilds. The additional keys are hidden collision-only tiles, queued ahead of ordinary streaming, and pruned once resolved or outside the active column. |
|
||||||
|
|||||||
@@ -0,0 +1,57 @@
|
|||||||
|
# VoxelForge current state
|
||||||
|
|
||||||
|
As of 2026-10-31. L131 is on `assembler/world` as `8107196`; this L132 worktree is on `wip/l132-dispatch-truth`, starting at `ebae71b`. Current-state claims below point to their code or measured report.
|
||||||
|
|
||||||
|
## What ships today
|
||||||
|
|
||||||
|
- **Core:** evaluator version **227**. Unreal's `AVoxelWorld::bUseVoxelForgeCore` is opt-in and defaults to `false`; the legacy generator remains the default runtime path. [Core version](Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h), [runtime switch](Source/VoxelForge/Public/VoxelWorld.h), [L130 report](Standalone/Research/WorldAssemblerM3/Linux130/REPORT.md).
|
||||||
|
- **Verified owner4 generator dispatch (L132):** SW binds `SurfaceWorld-native` because the owner export contains a matching `SurfaceWorld.Heightfield2D` v2 plan. FI binds `compiled-analytic/FloatingIslands` because its `FloatingIslands.ClusterField` v0 layout has no matching native island plan. TN and VS bind `compiled-analytic/TunnelNetwork` and `compiled-analytic/VerticalShafts`; their legacy recipe IDs provide neither a `NativeCaves`/`NativeLattice` graph recipe nor native field capability. The L132 fast-tier report prints the binding and generated tile-center share at each owner strate. [Dispatch](Standalone/Core/src/ShapeWorldInternal.h), [adapter recipe checks](Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h), [L132 report](Standalone/Research/WorldAssemblerM3/Linux132/REPORT.md).
|
||||||
|
- **Passages:** continuous smooth swept tubes with a continuous floor profile. L125 stamps conservative route slabs and outside sentinels into the tile cache; exact route geometry is still evaluated. All 21,425 route samples passed the open-air walk check; the nine-station floor probe's maximum root offset was 7.884 mm. L130 sets the shared influence reach to **1.0533 m** (0.75 m shell + 0.21 m relief + 0.50 m smooth-max support / 6 + 0.01 m margin), converted once from metres to Core voxels for the proof, feature, fallback, and off-grid bounds. [L125](Standalone/Research/WorldAssemblerM3/Linux125/REPORT.md), [L130](Standalone/Research/WorldAssemblerM3/Linux130/REPORT.md).
|
||||||
|
- **Streaming:** arrival uses a bounded startup apply burst: `voxel.StreamingStartupApplyBoost=1` by default, up to 24 visible meshes or 14 ms per frame for at most 30 seconds, until the initial desired set and walkable ring are ready. `voxel.StreamingFarApplySupplement=1` is also default-on: after the normal visible apply cap, at most one far result applies when under half the apply-time budget; it is inactive during the startup burst. Dynamic near reserve defaults on; coarse-first remains off; the legacy pending-result cap defaults off. The far-apply lane is Unreal-only. [CVar source](Source/VoxelForge/Private/VoxelWorld.cpp), [L121 report](Standalone/Research/WorldAssemblerM3/Linux121/REPORT.md), [architecture](ARCHITECTURE.md).
|
||||||
|
- **Determinism:** proven on Linux with Clang and GCC at W1/W8/W12. L131's unchanged-output comparison covers owner4's SW/FI/TN/VS views in 24 compiler/worker cases; the canonical owner4/owner2 recipe matrix also passes. Windows determinism is unverified. [L131 checks](Standalone/Research/WorldAssemblerM3/Linux131/REPORT.md), [v227 canonical hashes](Standalone/Tools/verify/baselines/owner-hash/canonical/).
|
||||||
|
|
||||||
|
## Current performance references
|
||||||
|
|
||||||
|
Headless medians in seconds, five interleaved L80-style measurements per version in each cell. Near-ready means the complete LOD0–2 set; whole-view is the full requested view. Values below are the edited L131 binary; the paired pre-cleanup comparison is in the [L131 report](Standalone/Research/WorldAssemblerM3/Linux131/REPORT.md) and [timing summary](Saved/Linux/l131/timing/summary.json).
|
||||||
|
|
||||||
|
| Stratum | Near-ready W8 / W12 | Whole-view W8 / W12 |
|
||||||
|
|---|---:|---:|
|
||||||
|
| S0 | 0.804 / 0.848 | 1.811 / 1.832 |
|
||||||
|
| S1 | 1.014 / 1.002 | 1.993 / 1.886 |
|
||||||
|
| S2 | 1.060 / 1.012 | 1.960 / 1.865 |
|
||||||
|
| S3 | 0.917 / 0.884 | 1.887 / 1.782 |
|
||||||
|
|
||||||
|
Across 40 paired runs per metric, the mean relative change from pre-cleanup L130 was −0.22% near-ready (95% CI −1.02% to +0.57%) and −0.29% whole-view (−1.04% to +0.46%); neither indicates a measurable slowdown. The machine gate was load 0.81 and 99.2% idle. [L80 protocol](Standalone/Research/WorldAssemblerM3/Linux80/REPORT.md), [runner and raw runs](Standalone/Research/WorldAssemblerM3/Linux131/RunL131Timing.py).
|
||||||
|
|
||||||
|
The latest recorded Linux Unreal start-to-LOD0–2 values are L127's one run per stratum: S0 2.456 s, S1 2.231 s, S2 2.514 s, S3 2.221 s. L130 did not rerun UE startup or movement. Five L127 runs at 30 m/s, 12 s moving plus 5 s dwell, had median moving near coverage **40.22%**, far holes **3.46%**, tick p99 **10.80 ms**, and worst tick **13.27 ms**. After stopping, near coverage reached 100% and far holes reached 0%. These are the latest UE references, not v227 UE measurements. [L127 report](Standalone/Research/WorldAssemblerM3/Linux127/REPORT.md), [L127 movement summary](Saved/Linux/l127/ue/movement-pairs-summary.json), [L127 start summaries](Saved/Linux/l127/ue/merge/start-summary.json).
|
||||||
|
|
||||||
|
## Open gaps
|
||||||
|
|
||||||
|
- Multiplayer world lifecycle is unfinished: seed/layout replication, authoritative modification RPCs, and serialized diff replay for late join are still open. See [ARCHITECTURE §9.5 and §9.7](ARCHITECTURE.md).
|
||||||
|
- Determinism has not been verified on Windows. Linux Clang/GCC agreement does not establish MSVC agreement.
|
||||||
|
- NullRHI measures residency and apply timing, not the first presented pixel or rendered enclosure. L127 also found four low-contrast views among 23 otherwise verified captures. [L121](Standalone/Research/WorldAssemblerM3/Linux121/REPORT.md), [L127](Standalone/Research/WorldAssemblerM3/Linux127/REPORT.md).
|
||||||
|
- UE startup/movement numbers above predate L130's v227 reach change; there is no L130/L131 UE movement comparison.
|
||||||
|
|
||||||
|
## How to verify
|
||||||
|
|
||||||
|
- Build standalone Clang or GCC with [`Standalone/BuildLinux.sh`](Standalone/BuildLinux.sh), for example `bash Standalone/BuildLinux.sh --compiler=clang++-19 --variant=l132-clang --fp-contract=off --jobs=4`. The script pins CPUs 0–11 and applies `prlimit --as=6442450944` itself.
|
||||||
|
- Fast tier: `python3 Standalone/Tools/verify/verify.py fast --out Saved/Linux/l132 --overall-timeout 1740`. It includes the L132 four-pose owner density-dispatch report; per-run logs and the summary stay under `Saved/Linux/l132`.
|
||||||
|
- Canonical output baselines are the v227 files in `Standalone/Tools/verify/baselines/owner-hash/canonical/`. L131 also provides reproducible pre-cleanup identity matrices for owner4 tile hashes and owner4/owner2 canonical hashes under `Standalone/Research/WorldAssemblerM3/Linux131/`.
|
||||||
|
- Timing follows L80: quiet machine, no overlapping build, one warm-up per version, five interleaved runs per stratum and worker count, W8/W12. Each headless run uses `taskset -c 0-11`, `prlimit --as=6442450944`, and a 1,740 s timeout. See [`RunL131Timing.py`](Standalone/Research/WorldAssemblerM3/Linux131/RunL131Timing.py).
|
||||||
|
- Linux host build: `/home/jahni/UnrealEngine/Engine/Build/BatchFiles/Linux/Build.sh VFHostEditor Linux Development -Project=/home/jahni/VFHost/VFHost.uproject -NoUBA -MaxParallelActions=4 -Log=/home/jahni/VoxelM/Plugins/VoxelForge/Saved/Linux/l131/ubt.log`. Run Unreal without `prlimit`.
|
||||||
|
|
||||||
|
## Closed experiments
|
||||||
|
|
||||||
|
These rejected speed/streaming proposals stay closed unless new evidence changes their measured failure. Reports and surviving patches are linked in the [L131 inventory](Standalone/Research/WorldAssemblerM3/Linux131/REPORT.md).
|
||||||
|
|
||||||
|
| Line | Result |
|
||||||
|
|---|---|
|
||||||
|
| L116 tunnel chord bound and z cache | Pair count fell 20.8%, but stayed 1.96× L109 and readiness was 15–30% slower. Patch: [`chord-bound-zcolumn-not-merged.patch`](Standalone/Research/WorldAssemblerM3/Linux116/patches/chord-bound-zcolumn-not-merged.patch), commit `89504ec`; see [report](Standalone/Research/WorldAssemblerM3/Linux116/REPORT.md). |
|
||||||
|
| L118 pair pruning / deferred floor work | Pruned 17–18% of candidate pairs but remained 17.2–25.7% behind L109 near-ready. [Report](Standalone/Research/WorldAssemblerM3/Linux118/REPORT.md); patch: [`l118-designs.patch`](Standalone/Research/WorldAssemblerM3/Linux118/l118-designs.patch). |
|
||||||
|
| L126 floor-call filters | Cut floor calls 20–23% without reducing pair counts; all eight cells missed L109. Candidate source was restored and no standalone patch was kept; see [report](Standalone/Research/WorldAssemblerM3/Linux126/REPORT.md) and commit `01a5094`. |
|
||||||
|
| L128 half-open slab ownership | Ownership audit passed, but readiness was 14.8–28.0% behind L109 and moving coverage dropped. Report and patch `l128-slab-ownership.patch` are in archived commit `7022d02`. |
|
||||||
|
| L129 2 m physical reach | v225's metre/Core-voxel bounds mismatch made the correctly bounded candidate do more work; it was slower and failed UE movement gates. Report and patch `l129-reach-2m.patch` are in archived commit `0d1140c`. |
|
||||||
|
| L120 priority split / extra worker slots | Neither met coverage and tick gates; 10 slots reached a 16.67 ms worst tick against 15.43 ms baseline. [Report](Standalone/Research/WorldAssemblerM3/Linux120/REPORT.md); no production patch retained. |
|
||||||
|
| L122 admission relaxation | Early admission missed p99 or far-hole gates; no behavior retained. [Report](Standalone/Research/WorldAssemblerM3/Linux122/REPORT.md); patch: [`l122-trace.patch`](Standalone/Research/WorldAssemblerM3/Linux122/l122-trace.patch). |
|
||||||
|
| L123 lookahead / completion gate | Lookahead failed p99 and worsened near/far coverage; queue-only also failed p99. [Report](Standalone/Research/WorldAssemblerM3/Linux123/REPORT.md); patch: [`l123-queue-lookahead.patch`](Standalone/Research/WorldAssemblerM3/Linux123/l123-queue-lookahead.patch). |
|
||||||
|
| L124 early LOD0 pages | Optimistic gain stayed below 100 ms except negative cases; no publication/scheduling implementation was made. [Report](Standalone/Research/WorldAssemblerM3/Linux124/REPORT.md). |
|
||||||
@@ -0,0 +1,201 @@
|
|||||||
|
# L162 — Living Plane
|
||||||
|
|
||||||
|
The EGL/OpenGL 4.3 runner builds and passes its llvmpipe correctness and frame checks. The sandbox has no accessible DRM render node, so it did not run or report Intel GPU performance. The runner refuses `--device=gpu` here instead of falling back to software.
|
||||||
|
|
||||||
|
## Implementation
|
||||||
|
|
||||||
|
`LivingPlane.cpp` runs a ping-pong `RG32F` height/velocity field through a runtime-compiled GLSL compute shader. The damped wave update includes a weak foundation spring, a Gaussian sphere push, and an optional slow breathing force. The spring is needed because a persistent local push otherwise accelerates the zero-frequency height mode and sinks the whole plane. The sphere moves across the plane, stops, and bobs. A vertex-displaced mesh and lit sphere render to an EGL pbuffer-backed framebuffer and save as PNGs.
|
||||||
|
|
||||||
|
The timestep is selected from the largest 2D grid eigenfrequency:
|
||||||
|
|
||||||
|
`dt = 1.30 / sqrt(8*c*c/(dx*dx) + foundationStiffness)`
|
||||||
|
|
||||||
|
The resulting spectral step ratio is 1.30, below the explicit update's 2.0 stability limit. The runner warms the compute kernel for eight unmeasured dispatches, then records one `GL_TIME_ELAPSED` query per requested simulation step in `timing.json`.
|
||||||
|
|
||||||
|
The first-step equation audit also caught a GL state error during bring-up: frame rendering left the graphics program active before the next compute dispatch. `Dispatch` now binds the compute program before every step, so the audit fails if a frame silently stops simulation.
|
||||||
|
|
||||||
|
Device selection enumerates EGL devices and uses `EGL_EXT_platform_device` for an accessible Intel render node first. On this host, the known integrated device is Intel on `renderD128`. Automatic mode falls back to a surfaceless Mesa context with `LIBGL_ALWAYS_SOFTWARE=1`; explicit GPU mode fails when no accessible hardware node is present. EGL does not expose Vulkan's integrated/discrete enum, so Intel vendor/render-node identity is the selection heuristic.
|
||||||
|
|
||||||
|
The Unreal handoff is in [UNREAL-MAPPING.md](UNREAL-MAPPING.md). It keeps the prototype render-only and describes the density mapping if the surface later needs to affect voxel queries. No Unreal `Source/` files changed.
|
||||||
|
|
||||||
|
## Measurements and checks
|
||||||
|
|
||||||
|
- Built with `Experiments/LivingPlane/build.sh` using the installed EGL, desktop GL, and libpng development files; no shader compiler or package install was needed.
|
||||||
|
- Final llvmpipe run: 128² field, 360 steps, `dt=9.56814 ms`, spectral ratio 1.30. The first-step equation audit and finite-field/dynamics audit passed. Final field range was −0.139533 m to +0.0366704 m. It wrote 16 128×128 PNGs and a JSON file containing 360 per-step query samples. The frames were inspected; the moving sphere and wake are visible.
|
||||||
|
- Auto-selection smoke check: 32², 12 steps; selected llvmpipe and passed both audits. Breathing smoke check: 64², 180 steps with `--breathing=on`; passed both audits.
|
||||||
|
- Negative device-selection control: `--device=gpu` returned an error because the sandbox has no accessible EGL DRM render node.
|
||||||
|
- Existing fast verification tier passed: 278.9 seconds, ending in `SUMMARY PASS`. Its build and logs are under `Saved/Linux/l162/verify/` and `Saved/Linux/l162/`.
|
||||||
|
- The llvmpipe `GL_TIME_ELAPSED` samples are preserved in JSON but are software-driver query values, not GPU timings. The requested hardware timing pass remains to be run outside the sandbox.
|
||||||
|
- No Unreal build was run; no Unreal `Source/` files changed.
|
||||||
|
|
||||||
|
## Intel GPU pass
|
||||||
|
|
||||||
|
Run these from the plugin root as `jahni`. The frame run writes 512×512 PNGs to `Saved/Linux/l162/gpu-frames-512/`; the two timing runs write `timing.json` with one GPU query per step and skip PNG capture. They still read the field for the first-step and final-state correctness audits.
|
||||||
|
Before the timing runs, check `uptime` and run them on a quiet machine (load below about 1.5, with no other build active).
|
||||||
|
|
||||||
|
```sh
|
||||||
|
uptime
|
||||||
|
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s ./Saved/Linux/l162/living-plane --device=gpu --size=512 --steps=1800 --capture-every=12 --out=Saved/Linux/l162/gpu-frames-512
|
||||||
|
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s ./Saved/Linux/l162/living-plane --device=gpu --size=512 --steps=1800 --capture-every=0 --out=Saved/Linux/l162/gpu-timing-512
|
||||||
|
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s ./Saved/Linux/l162/living-plane --device=gpu --size=1024 --steps=1800 --capture-every=0 --out=Saved/Linux/l162/gpu-timing-1024
|
||||||
|
```
|
||||||
|
|
||||||
|
Encode the captured sequence as MP4 with:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
ffmpeg -y -framerate 30 -i Saved/Linux/l162/gpu-frames-512/frame_%06d.png -c:v libx264 -pix_fmt yuv420p Saved/Linux/l162/living-plane-512.mp4
|
||||||
|
```
|
||||||
|
|
||||||
|
## Files and remaining risk
|
||||||
|
|
||||||
|
The only changed file outside `Experiments/LivingPlane/` is `.gitignore`, which now allows `Experiments/` and its contents. GPU timings and Intel render-node selection remain unverified until the commands above run on the host outside this sandbox.
|
||||||
|
|
||||||
|
## Intel GPU pass (run by the orchestrator outside the sandbox, 2026-11-02)
|
||||||
|
|
||||||
|
- The device was selected correctly: `Mesa Intel(R) Graphics (ADL GT2)`, OpenGL 4.6, `/dev/dri/renderD128`. 512²,
|
||||||
|
1,800 steps, 151 frames; the equation audit passes. MP4: `Saved/Linux/l162/living-plane-512.mp4` (local).
|
||||||
|
- **The `GL_TIME_ELAPSED` per-dispatch values (~0.0001 ms) are not credible** for 262k cells, so they aren't
|
||||||
|
reported as timings.
|
||||||
|
- Wall-time difference between 1,800 and 200 steps, divided by the 1,600 steps (start-up cancels out; this includes
|
||||||
|
submission, so it's an upper bound): **0.080 ms/step at 512²** and **0.292 ms/step at 1024²**.
|
||||||
|
- Visual read: the effect is very subtle in the current render (a slight dip under the sphere, faint rings). It needs
|
||||||
|
a larger push amplitude, grazing light or height colouring, and a closer camera to read.
|
||||||
|
|
||||||
|
## L163 — readable, living plane
|
||||||
|
|
||||||
|
The plane now reads as rolling terrain, and the objects leave visible troughs and wakes. `alive` adds travelling
|
||||||
|
breathing and a viscoelastic imprint that remains after lift-off. The simulation still runs in one GPU dispatch per
|
||||||
|
step; CPU work only supplies the two object states.
|
||||||
|
|
||||||
|
`LivingPlane.cpp` now keeps the dynamic displacement, velocity, and imprint in an `RGBA32F` ping-pong field. The
|
||||||
|
surface shader adds broad static terrain relief, calculates normals from relief plus displacement, and uses grazing
|
||||||
|
light, elevation colours, slope tint, and 7.5 cm contour bands. The camera follows the first sphere at a closer angle.
|
||||||
|
The wake scene rolls the first sphere, pauses and wobbles it, lifts it, then sends a second sphere through the wake.
|
||||||
|
The alive scene runs that same sequence with low-frequency travelling swells and a decaying contact imprint. Its
|
||||||
|
defaults are breathing strength 4.8 m/s², imprint capture 0.15 m/s, recovery time 6 s, and maximum imprint depth
|
||||||
|
0.34 m. These values have command-line overrides.
|
||||||
|
|
||||||
|
The timer query check found no early read or reuse in L162: it allocated a unique query per step, called `glFinish`,
|
||||||
|
then read every result. The saved Intel/Mesa 25.0.7 samples are 0 or 52 ns for most dispatches (512² median 0,
|
||||||
|
p95 52 ns; largest sample 17.552 µs), which cannot be a credible per-dispatch cost. The old usage did not reuse
|
||||||
|
query objects, so lifetime is not the cause; the saved data point to Intel/Mesa's timer-query reporting or timing
|
||||||
|
scope, but the sandbox cannot isolate which. L163 uses a 64-query ring, reads each query before reusing its slot,
|
||||||
|
checks availability after `glFinish`, and records `GL_QUERY_COUNTER_BITS`. On llvmpipe the availability check passed
|
||||||
|
with 64 counter bits and zero waits. The Intel query result still needs the GPU diagnostic command below.
|
||||||
|
|
||||||
|
### Measurements and checks
|
||||||
|
|
||||||
|
- llvmpipe wake: 128², 600 steps (`dt=9.56814 ms`, 5.741 s simulation window), 151 frames at 30 fps. First-step
|
||||||
|
equation audit and timer-query lifetime check passed; final displacement range was −0.177488 m to +0.104854 m.
|
||||||
|
- llvmpipe alive: same grid, steps, and window; equation audit and timer-query lifetime check passed. Final
|
||||||
|
displacement range was −0.200437 m to +0.115904 m, with a −0.111667 m stored imprint.
|
||||||
|
- llvmpipe cost: 512², 1,800 steps, `--queries=off`, no frame capture: **0.767473 ms/step** in one timed
|
||||||
|
simulation-loop window. `timing.json` includes dispatch submission, the step-0 equation readback, and final
|
||||||
|
`glFinish`, but excludes setup and frame capture. This is a software-renderer measurement, not the Intel GPU cost.
|
||||||
|
- Both five-second llvmpipe previews are in `Saved/Linux/l163/llvmpipe-wake.mp4` and
|
||||||
|
`Saved/Linux/l163/llvmpipe-alive.mp4`. They are 128² review previews; use the GPU commands below for the owner
|
||||||
|
videos.
|
||||||
|
- Equation-audit negative control: a temporary copy with its expected first-step height forced to zero exited FAIL
|
||||||
|
at actual −0.0488777 m. The final source passes that same check.
|
||||||
|
- The fast verification tier passed in 278.5 s. Build and logs are under `Saved/Linux/l163/verify/`.
|
||||||
|
- No Unreal `Source/` files changed and no Unreal build ran.
|
||||||
|
|
||||||
|
### Intel GPU run
|
||||||
|
|
||||||
|
Run from the plugin root as `jahni`. First build the runner. For timed runs, check `uptime` and keep the machine
|
||||||
|
quiet. The wall-time field is valid only when `--capture-every=0`; use `--queries=off` for the step-cost measurement.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
bash Experiments/LivingPlane/build.sh
|
||||||
|
uptime
|
||||||
|
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s ./Saved/Linux/l163/living-plane --device=gpu --scene=wake --size=512 --steps=2400 --capture-every=16 --queries=off --out=Saved/Linux/l163/gpu-wake-512
|
||||||
|
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s ./Saved/Linux/l163/living-plane --device=gpu --scene=alive --size=512 --steps=2400 --capture-every=16 --queries=off --out=Saved/Linux/l163/gpu-alive-512
|
||||||
|
ffmpeg -y -framerate 30 -i Saved/Linux/l163/gpu-wake-512/frame_%06d.png -c:v libx264 -pix_fmt yuv420p Saved/Linux/l163/living-plane-wake-512.mp4
|
||||||
|
ffmpeg -y -framerate 30 -i Saved/Linux/l163/gpu-alive-512/frame_%06d.png -c:v libx264 -pix_fmt yuv420p Saved/Linux/l163/living-plane-alive-512.mp4
|
||||||
|
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s ./Saved/Linux/l163/living-plane --device=gpu --scene=alive --size=512 --steps=1800 --capture-every=0 --queries=off --out=Saved/Linux/l163/gpu-timing-512
|
||||||
|
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s ./Saved/Linux/l163/living-plane --device=gpu --scene=alive --size=512 --steps=1800 --capture-every=0 --queries=on --out=Saved/Linux/l163/gpu-query-check-512
|
||||||
|
```
|
||||||
|
|
||||||
|
`gpu-timing-512/timing.json` reports the 512² batch wall time per step, including dispatch submission and completion.
|
||||||
|
`gpu-query-check-512/timing.json` contains query counter bits, waits before slot reuse, availability status, and raw
|
||||||
|
per-step query samples. The sandbox has no accessible Intel DRM node, so the current Intel step cost and new query
|
||||||
|
diagnostic remain unmeasured here.
|
||||||
|
|
||||||
|
### L163 Intel GPU pass (run by the orchestrator, 2026-11-02)
|
||||||
|
|
||||||
|
- Intel ADL GT2 (Mesa 25.0.7, `renderD128`): wake and alive scenes, 512², 2,400 steps, 151 frames each; equation
|
||||||
|
audits PASS. Videos (local): `Saved/Linux/l163/living-plane-wake-512.mp4`, `living-plane-alive-512.mp4`.
|
||||||
|
- Step cost, alive scene, 512², 1,800 steps, no capture: **0.172 ms/step** with queries off, 0.152 ms/step with
|
||||||
|
queries on (batch wall time per step).
|
||||||
|
- The Intel `GL_TIME_ELAPSED` median is still 0 with 64 counter bits and the ring/availability checks passing. That
|
||||||
|
looks like a Mesa/Intel reporting issue; use wall time.
|
||||||
|
- **Open issue:** the same scene with the same steps gave different final fields (min −0.637 m with queries off vs
|
||||||
|
−0.327 m with queries on). Something depends on more than the step index (e.g. wall-clock time); investigate.
|
||||||
|
|
||||||
|
## L164 — repeatable field readback and UE runtime actor
|
||||||
|
|
||||||
|
**Result:** the repeatability check passes on llvmpipe, and the new `VoxelForgeLivingPlane` runtime module compiles
|
||||||
|
in the Linux UE 5.7 host. The surface actor is independent of VoxelForge generation and sends no field readback to
|
||||||
|
the CPU.
|
||||||
|
|
||||||
|
### Repeatability finding
|
||||||
|
|
||||||
|
`uTime` is `Step * dt`, and `GetBodiesAt` receives that same value. Neither the object schedule nor the breathing
|
||||||
|
force uses wall time or a rendered-frame counter. Every compute step already issued an image-access barrier before
|
||||||
|
the next ping-pong image read. The defect was on the diagnostic side: the step-zero and final `glGetTexImage`
|
||||||
|
readbacks followed shader image writes without `GL_TEXTURE_UPDATE_BARRIER_BIT`. The Khronos OpenGL specification
|
||||||
|
requires that barrier for `GetTexImage` to observe earlier shader writes ([§7.12](https://registry.khronos.org/OpenGL/specs/gl/glspec44.compatibility.pdf)). Query/capture pacing could therefore affect the reported field values without changing any simulation input. The barrier is now issued before both readbacks.
|
||||||
|
|
||||||
|
`check_repeatability.py` runs the same 512² alive scene for 1,800 steps twice: queries off with no captures, then
|
||||||
|
queries on with captures every 900 steps. The checksum covers every final RGBA32F field value using canonical
|
||||||
|
little-endian float bytes.
|
||||||
|
|
||||||
|
- llvmpipe, queries off/capture off: field range −0.637382 m to +0.109308 m; checksum
|
||||||
|
`1ba08f5dafc01f82`.
|
||||||
|
- llvmpipe, queries on/capture every 900: the same range and checksum; equation and timer-query audits pass; three
|
||||||
|
frames written.
|
||||||
|
- Negative control: changing one saved checksum made the comparison exit 1 with `FAIL field checksum mismatch`.
|
||||||
|
- The llvmpipe runs were each one 1,800-step simulation-loop window. Logs and JSON are under
|
||||||
|
`Saved/Linux/l164/repeatability/`.
|
||||||
|
|
||||||
|
The Intel run is not available in this sandbox. The owner should repeat the checksum comparison on the GPU to
|
||||||
|
confirm that the missing transfer-readback barrier accounts for the earlier Intel min mismatch:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python3 Experiments/LivingPlane/check_repeatability.py --device=gpu --out=Saved/Linux/l164/gpu-repeatability
|
||||||
|
```
|
||||||
|
|
||||||
|
### Unreal implementation and checks
|
||||||
|
|
||||||
|
The plugin descriptor adds a separate `VoxelForgeLivingPlane` runtime module. Its RDG global shader is in
|
||||||
|
`Shaders/LivingPlane.usf`; module startup maps `/Plugin/VoxelForgeLivingPlane` to the plugin shader directory. The
|
||||||
|
actor owns two UAV-capable RGBA32F render targets, advances them at the stable fixed step, and supplies the current
|
||||||
|
texture to a movable plane mesh material. Its Blueprint/C++ object input includes world position, radius, velocity,
|
||||||
|
base push, and active state. The CPU converts those values to plane-local metres and uploads uniforms; it does not
|
||||||
|
read the field back. The ignore rules now include `Shaders/` so that the shader ships in the worktree diff.
|
||||||
|
`Experiments/LivingPlane/UE-SETUP.md` gives level setup, the full material graph, and object driving steps.
|
||||||
|
|
||||||
|
- Linux UE 5.7 `VFHostEditor Linux Development`: **PASS**, final incremental build compiled
|
||||||
|
`VoxelForgeLivingPlaneActor.cpp` and linked `libUnrealEditor-VoxelForgeLivingPlane.so`. Command used
|
||||||
|
`-NoUBA -MaxParallelActions=4`, `taskset -c 0-11`, and no `prlimit`. Log:
|
||||||
|
`Saved/Linux/l164/ue-build-final.log`. The initial attempt exposed an incorrect RDG texture registration overload;
|
||||||
|
it had already compiled `VoxelForgeLivingPlaneModule.cpp` and the generated module code. The corrected final
|
||||||
|
build passed.
|
||||||
|
- The host `PluginDeps/VoxelForge/Source` and `.uplugin` links were temporarily pointed at this worktree for the
|
||||||
|
build and restored to `/home/jahni/VoxelM/Plugins/VoxelForge` afterward. The temporary shader link was removed.
|
||||||
|
- Standalone build, final repeatability comparison, checksum-mismatch negative control, plugin JSON parse, and
|
||||||
|
`git diff --check` passed.
|
||||||
|
- Not run: fast generation verification (generation sources are untouched), Intel GPU checksum comparison, or a UE
|
||||||
|
editor render/shader-runtime check. The Linux UBT build verifies C++ and module integration, not first-use shader
|
||||||
|
compilation or material appearance.
|
||||||
|
|
||||||
|
New Unreal source and shader files for the owner's Windows build:
|
||||||
|
|
||||||
|
- `Source/VoxelForgeLivingPlane/VoxelForgeLivingPlane.Build.cs`
|
||||||
|
- `Source/VoxelForgeLivingPlane/Public/VoxelForgeLivingPlaneActor.h`
|
||||||
|
- `Source/VoxelForgeLivingPlane/Private/VoxelForgeLivingPlaneActor.cpp`
|
||||||
|
- `Source/VoxelForgeLivingPlane/Private/VoxelForgeLivingPlaneModule.cpp`
|
||||||
|
- `Shaders/LivingPlane.usf`
|
||||||
|
|
||||||
|
No VoxelForge generation module files changed. Remaining risks are the Intel checksum confirmation and first-use UE
|
||||||
|
shader/material rendering, which require the owner's GPU/editor run.
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
# Living Plane in Unreal Engine 5.7
|
||||||
|
|
||||||
|
`VoxelForgeLivingPlane` is a separate runtime module. It does not call the VoxelForge terrain generators and it does not read the simulated texture back to the CPU. The actor owns two transient `RGBA32f` render targets and advances them with an RDG compute pass. The CPU sends object state and fixed-step uniforms only.
|
||||||
|
|
||||||
|
## Add it to a level
|
||||||
|
|
||||||
|
1. Enable the VoxelForge plugin and restart the editor so the `VoxelForgeLivingPlane` runtime module and its shader mapping load.
|
||||||
|
2. Create a material named `M_LivingPlane` using the graph below.
|
||||||
|
3. Place `VoxelForgeLivingPlaneActor` in the level. Assign `M_LivingPlane` to **Plane Material**.
|
||||||
|
4. Keep the actor on the world XY plane with unit actor scale. `Plane Length Meters` controls the mesh width and simulation width; the built-in plane mesh is scaled to that size.
|
||||||
|
5. The actor creates and clears its two GPU state targets on `BeginPlay`. The mesh has collision disabled; this surface is render-only and does not change VoxelForge density or collision queries.
|
||||||
|
|
||||||
|
## Material graph
|
||||||
|
|
||||||
|
Use a Surface, Opaque material. The actor binds these parameters on its dynamic material instance:
|
||||||
|
|
||||||
|
- `LivingField`: Texture Object Parameter. The texture is the current RGBA32f simulation state; **R** is displacement in metres.
|
||||||
|
- `PlaneLengthMeters`: Scalar Parameter.
|
||||||
|
- `FieldSize`: Scalar Parameter, equal to the render target width and height.
|
||||||
|
|
||||||
|
For World Position Offset:
|
||||||
|
|
||||||
|
1. Read `LivingField` at `TextureCoordinate[0]` with a Texture Sample.
|
||||||
|
2. Multiply its **R** output by `100` to convert metres to Unreal centimetres.
|
||||||
|
3. Append `(0, 0)` to that value, making `(0, 0, heightCm)`, and connect it to **World Position Offset**.
|
||||||
|
|
||||||
|
For the displaced normal, sample the height at four UV offsets using the same texture object:
|
||||||
|
|
||||||
|
- Left/right UV: `TexCoord + (-1/FieldSize, 0)` and `TexCoord + (1/FieldSize, 0)`.
|
||||||
|
- Down/up UV: `TexCoord + (0, -1/FieldSize)` and `TexCoord + (0, 1/FieldSize)`.
|
||||||
|
- `dHdx = (Right.R - Left.R) * FieldSize / (2 * PlaneLengthMeters)`.
|
||||||
|
- `dHdy = (Up.R - Down.R) * FieldSize / (2 * PlaneLengthMeters)`.
|
||||||
|
- Make and normalize the tangent-space vector `(-dHdx, -dHdy, 1)`, then connect it to **Normal**.
|
||||||
|
|
||||||
|
The built-in plane's U/V tangent directions align with local X/Y. Use a neutral base colour and roughness around `0.7` to start. The graph assumes a horizontal plane; add a local-to-world transform if the plane needs arbitrary rotation.
|
||||||
|
|
||||||
|
## Drive objects
|
||||||
|
|
||||||
|
`ObjectInputs` has two slots. Each `FVFLivingPlaneObjectInput` contains world position in centimetres, world velocity in centimetres per second, radius in centimetres, base push acceleration in metres per second squared, and an active flag. Blueprint can use **Set Object State** or set the array entry; C++ can call the same method. Set the object's center about one radius above the plane while it contacts it, and mark the slot inactive after lift-off.
|
||||||
|
|
||||||
|
The compute shader uses the radius to scale the Gaussian contact footprint. The original wave equation keeps the L163 wave speed, damping, foundation spring, travelling breathing force, and recovering imprint. The base push acceleration supplies the object force; lateral speed adds `0.25 m/s²` per `1 m/s`, and downward speed adds `0.5 m/s²` per `1 m/s`. Upward velocity does not add force. Object positions and velocities are transformed into the plane's local frame and converted from centimetres to metres before upload.
|
||||||
|
|
||||||
|
Each step uses `dt = 1.30 / sqrt(8*c*c/(dx*dx) + foundationStiffness)`, with `dx = PlaneLengthMeters / FieldSize`. Simulation time passed to breathing is the fixed step index times `dt`. Game-thread delta time only determines how many fixed steps are queued; no GPU field data is read back in the normal path.
|
||||||
|
|
||||||
|
## Build files
|
||||||
|
|
||||||
|
The owner's Windows build needs every added source and shader file:
|
||||||
|
|
||||||
|
- `Source/VoxelForgeLivingPlane/VoxelForgeLivingPlane.Build.cs`
|
||||||
|
- `Source/VoxelForgeLivingPlane/Public/VoxelForgeLivingPlaneActor.h`
|
||||||
|
- `Source/VoxelForgeLivingPlane/Private/VoxelForgeLivingPlaneActor.cpp`
|
||||||
|
- `Source/VoxelForgeLivingPlane/Private/VoxelForgeLivingPlaneModule.cpp`
|
||||||
|
- `Shaders/LivingPlane.usf`
|
||||||
|
|
||||||
|
The `VoxelForge.uplugin` runtime-module entry must ship with them.
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
# Unreal mapping note
|
||||||
|
|
||||||
|
The prototype stores height, vertical velocity, a recovering imprint, and a spare
|
||||||
|
channel in a pair of `RGBA32F` textures. A world-owned Unreal component can keep
|
||||||
|
equivalent transient render graph textures and advance them with a compute pass. It
|
||||||
|
should own simulation time and clear or restore its buffers when the world is recreated.
|
||||||
|
|
||||||
|
The sphere path becomes a world-space interaction event: position, contact radius,
|
||||||
|
and impulse strength are supplied to the compute pass. For multiplayer or deterministic
|
||||||
|
replay, replicate the compact events and simulation tick, or record the resulting field;
|
||||||
|
do not assume that independent GPU floating-point runs will produce byte-identical
|
||||||
|
textures.
|
||||||
|
|
||||||
|
The rendered top surface can remain a render-only displaced mesh. If it must participate
|
||||||
|
in voxel collision or queries, its height must also reach the density evaluator. For a
|
||||||
|
height `h(x,y)`, the plugin's density convention is negative inside solid and positive
|
||||||
|
in air, so the plane's signed density is `z - h(x,y)`. Keep this calculation in metres
|
||||||
|
until the existing voxel/Core boundary converts with the active voxel size (25 cm by
|
||||||
|
default); do not mix metre distances and voxel counts in the wave solver.
|
||||||
|
|
||||||
|
The current sample is an isolated visual field. Wiring this into `AVoxelWorld` would
|
||||||
|
need an explicit owner for simulation lifetime, collision/update cadence, save/load,
|
||||||
|
and the replication policy before it could affect shipped terrain.
|
||||||
|
|
||||||
|
## L164 render-only Unreal implementation
|
||||||
|
|
||||||
|
`Source/VoxelForgeLivingPlane/` now implements this render-only path as a separate
|
||||||
|
runtime module. `AVoxelForgeLivingPlaneActor` owns two UAV-capable RGBA32F render
|
||||||
|
targets, advances the wave, breathing, and imprint equations with an RDG global shader,
|
||||||
|
and binds the current texture to its plane material. Blueprint/C++ supplies object
|
||||||
|
position, radius, velocity, push, and active state. The actor does not call the
|
||||||
|
generation modules and does not read the field back. Level setup and the material graph
|
||||||
|
are in [UE-SETUP.md](UE-SETUP.md).
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
PLUGIN_ROOT="$(cd -- "${SCRIPT_DIR}/../.." && pwd)"
|
||||||
|
OUTPUT_DIR="${PLUGIN_ROOT}/Saved/Linux/l164"
|
||||||
|
CXX="${CXX:-g++}"
|
||||||
|
|
||||||
|
command -v "${CXX}" >/dev/null || { echo "C++ compiler not found: ${CXX}" >&2; exit 2; }
|
||||||
|
pkg-config --exists egl gl libpng || {
|
||||||
|
echo "EGL, desktop OpenGL, and libpng development files are required" >&2
|
||||||
|
exit 2
|
||||||
|
}
|
||||||
|
mkdir -p "${OUTPUT_DIR}"
|
||||||
|
read -r -a GRAPHICS_FLAGS <<<"$(pkg-config --cflags --libs egl gl libpng)"
|
||||||
|
|
||||||
|
taskset -c 0-11 prlimit --as=6442450944 -- \
|
||||||
|
"${CXX}" -std=c++20 -O2 -g -Wall -Wextra -Wpedantic \
|
||||||
|
"${SCRIPT_DIR}/LivingPlane.cpp" -o "${OUTPUT_DIR}/living-plane" \
|
||||||
|
"${GRAPHICS_FLAGS[@]}"
|
||||||
|
|
||||||
|
echo "built ${OUTPUT_DIR}/living-plane"
|
||||||
@@ -0,0 +1,132 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Compare final field checksums across different query and capture settings."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import pathlib
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
|
||||||
|
|
||||||
|
PLUGIN_ROOT = pathlib.Path(__file__).resolve().parents[2]
|
||||||
|
|
||||||
|
|
||||||
|
def read_checksum(path: pathlib.Path) -> str:
|
||||||
|
try:
|
||||||
|
with path.open(encoding="utf-8") as source:
|
||||||
|
result = json.load(source)
|
||||||
|
except (OSError, json.JSONDecodeError) as error:
|
||||||
|
raise RuntimeError(f"could not read {path}: {error}") from error
|
||||||
|
checksum = result.get("field_checksum_fnv1a64")
|
||||||
|
if not isinstance(checksum, str) or len(checksum) != 16:
|
||||||
|
raise RuntimeError(f"{path} has no 16-digit field checksum")
|
||||||
|
return checksum
|
||||||
|
|
||||||
|
|
||||||
|
def compare(left: pathlib.Path, right: pathlib.Path) -> bool:
|
||||||
|
left_checksum = read_checksum(left)
|
||||||
|
right_checksum = read_checksum(right)
|
||||||
|
if left_checksum != right_checksum:
|
||||||
|
print(
|
||||||
|
"FAIL field checksum mismatch: "
|
||||||
|
f"{left}={left_checksum}, {right}={right_checksum}",
|
||||||
|
file=sys.stderr,
|
||||||
|
)
|
||||||
|
return False
|
||||||
|
print(f"PASS field checksum matches: {left_checksum}")
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument(
|
||||||
|
"--binary",
|
||||||
|
type=pathlib.Path,
|
||||||
|
default=PLUGIN_ROOT / "Saved/Linux/l164/living-plane",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--out",
|
||||||
|
type=pathlib.Path,
|
||||||
|
default=PLUGIN_ROOT / "Saved/Linux/l164/repeatability",
|
||||||
|
)
|
||||||
|
parser.add_argument("--device", choices=("auto", "cpu", "gpu"), default="cpu")
|
||||||
|
parser.add_argument("--size", type=int, default=512)
|
||||||
|
parser.add_argument("--steps", type=int, default=1800)
|
||||||
|
parser.add_argument(
|
||||||
|
"--compare-only",
|
||||||
|
nargs=2,
|
||||||
|
type=pathlib.Path,
|
||||||
|
metavar=("LEFT_JSON", "RIGHT_JSON"),
|
||||||
|
help="compare existing timing.json files; used to exercise the failure path",
|
||||||
|
)
|
||||||
|
options = parser.parse_args()
|
||||||
|
|
||||||
|
if options.compare_only:
|
||||||
|
return 0 if compare(*options.compare_only) else 1
|
||||||
|
|
||||||
|
if not options.binary.is_file():
|
||||||
|
parser.error(f"binary not found: {options.binary}; run Experiments/LivingPlane/build.sh")
|
||||||
|
|
||||||
|
options.out.mkdir(parents=True, exist_ok=True)
|
||||||
|
off_dir = options.out / "queries-off_capture-off"
|
||||||
|
on_dir = options.out / "queries-on_capture-every-900"
|
||||||
|
for directory in (off_dir, on_dir):
|
||||||
|
if directory.exists():
|
||||||
|
shutil.rmtree(directory)
|
||||||
|
|
||||||
|
common = [
|
||||||
|
f"--device={options.device}",
|
||||||
|
"--scene=alive",
|
||||||
|
f"--size={options.size}",
|
||||||
|
f"--steps={options.steps}",
|
||||||
|
]
|
||||||
|
runs = (
|
||||||
|
("--capture-every=0", "--queries=off", off_dir),
|
||||||
|
("--capture-every=900", "--queries=on", on_dir),
|
||||||
|
)
|
||||||
|
for capture, queries, output in runs:
|
||||||
|
command = [
|
||||||
|
"taskset",
|
||||||
|
"-c",
|
||||||
|
"0-11",
|
||||||
|
"prlimit",
|
||||||
|
"--as=6442450944",
|
||||||
|
"--",
|
||||||
|
"timeout",
|
||||||
|
"1740s",
|
||||||
|
str(options.binary.resolve()),
|
||||||
|
*common,
|
||||||
|
capture,
|
||||||
|
queries,
|
||||||
|
f"--out={output}",
|
||||||
|
]
|
||||||
|
print("RUN " + " ".join(command), flush=True)
|
||||||
|
log_path = options.out / f"{output.name}.log"
|
||||||
|
with log_path.open("w", encoding="utf-8") as log_file:
|
||||||
|
try:
|
||||||
|
subprocess.run(
|
||||||
|
command,
|
||||||
|
cwd=PLUGIN_ROOT,
|
||||||
|
check=True,
|
||||||
|
stdout=log_file,
|
||||||
|
stderr=subprocess.STDOUT,
|
||||||
|
)
|
||||||
|
except subprocess.CalledProcessError:
|
||||||
|
print(log_path.read_text(encoding="utf-8"), file=sys.stderr)
|
||||||
|
raise
|
||||||
|
print(log_path.read_text(encoding="utf-8").rstrip())
|
||||||
|
|
||||||
|
left = off_dir / "timing.json"
|
||||||
|
right = on_dir / "timing.json"
|
||||||
|
return 0 if compare(left, right) else 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
try:
|
||||||
|
raise SystemExit(main())
|
||||||
|
except (OSError, RuntimeError, subprocess.CalledProcessError) as error:
|
||||||
|
print(f"FAIL repeatability check: {error}", file=sys.stderr)
|
||||||
|
raise SystemExit(1)
|
||||||
@@ -0,0 +1,85 @@
|
|||||||
|
Texture2D<float4> SourceState;
|
||||||
|
RWTexture2D<float4> TargetState;
|
||||||
|
|
||||||
|
int2 GridSize;
|
||||||
|
float Dt;
|
||||||
|
float Dx;
|
||||||
|
float PlaneLength;
|
||||||
|
float WaveSpeed;
|
||||||
|
float Damping;
|
||||||
|
float FoundationStiffness;
|
||||||
|
float TimeSeconds;
|
||||||
|
float2 BreathingSettings;
|
||||||
|
float4 ImprintSettings;
|
||||||
|
float4 ObjectPositionRadius[2];
|
||||||
|
float4 ObjectVelocityPush[2];
|
||||||
|
|
||||||
|
[numthreads(8, 8, 1)]
|
||||||
|
void MainCS(uint3 DispatchThreadId : SV_DispatchThreadID)
|
||||||
|
{
|
||||||
|
int2 Pixel = int2(DispatchThreadId.xy);
|
||||||
|
if (Pixel.x >= GridSize.x || Pixel.y >= GridSize.y)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
int2 Lo = int2(0, 0);
|
||||||
|
int2 Hi = GridSize - 1;
|
||||||
|
float4 State = SourceState.Load(int3(Pixel, 0));
|
||||||
|
float Height = State.r;
|
||||||
|
float Velocity = State.g;
|
||||||
|
float Imprint = State.b;
|
||||||
|
float Left = SourceState.Load(int3(max(Pixel + int2(-1, 0), Lo), 0)).r;
|
||||||
|
float Right = SourceState.Load(int3(min(Pixel + int2(1, 0), Hi), 0)).r;
|
||||||
|
float Down = SourceState.Load(int3(max(Pixel + int2(0, -1), Lo), 0)).r;
|
||||||
|
float Up = SourceState.Load(int3(min(Pixel + int2(0, 1), Hi), 0)).r;
|
||||||
|
float Laplacian = (Left + Right + Down + Up - 4.0 * Height) / max(Dx * Dx, 1.0e-8);
|
||||||
|
float2 World = (float2(Pixel) + 0.5) * Dx - PlaneLength * 0.5;
|
||||||
|
|
||||||
|
float Interaction = 0.0;
|
||||||
|
float Contact = 0.0;
|
||||||
|
for (int ObjectIndex = 0; ObjectIndex < 2; ++ObjectIndex)
|
||||||
|
{
|
||||||
|
float4 PositionRadius = ObjectPositionRadius[ObjectIndex];
|
||||||
|
if (PositionRadius.w <= 0.0)
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
float4 VelocityPush = ObjectVelocityPush[ObjectIndex];
|
||||||
|
float Radius = PositionRadius.w;
|
||||||
|
float Sigma = max(Radius * (0.58 / 0.43), Dx);
|
||||||
|
float2 Offset = World - PositionRadius.xy;
|
||||||
|
float Weight = exp(-dot(Offset, Offset) / max(2.0 * Sigma * Sigma, 1.0e-8));
|
||||||
|
float HeightAboveContact = max(PositionRadius.z - Radius, 0.0);
|
||||||
|
float ContactHeight = 1.0 - smoothstep(0.0, max(Radius, Dx), HeightAboveContact);
|
||||||
|
float ImpactSpeed = length(VelocityPush.xy) * 0.25 + max(-VelocityPush.z, 0.0) * 0.5;
|
||||||
|
float Push = max(VelocityPush.w, 0.0) + ImpactSpeed;
|
||||||
|
Interaction -= Push * Weight * ContactHeight;
|
||||||
|
if (Push > 0.0)
|
||||||
|
{
|
||||||
|
Contact = max(Contact, Weight * ContactHeight);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
float Breathing = 0.0;
|
||||||
|
if (BreathingSettings.x != 0.0)
|
||||||
|
{
|
||||||
|
float Swell = sin(World.x * 0.48 - TimeSeconds * 0.22)
|
||||||
|
* cos(World.y * 0.42 + TimeSeconds * 0.16);
|
||||||
|
Breathing = BreathingSettings.y * Swell * sin(TimeSeconds * 0.48);
|
||||||
|
}
|
||||||
|
|
||||||
|
float NextImprint = Imprint;
|
||||||
|
if (ImprintSettings.x != 0.0)
|
||||||
|
{
|
||||||
|
NextImprint = clamp(Imprint * exp(-Dt / max(ImprintSettings.z, 1.0e-4))
|
||||||
|
- Contact * ImprintSettings.y * Dt, -ImprintSettings.w, 0.0);
|
||||||
|
}
|
||||||
|
float FoundationTarget = ImprintSettings.x != 0.0 ? Imprint : 0.0;
|
||||||
|
float Acceleration = WaveSpeed * WaveSpeed * Laplacian
|
||||||
|
- FoundationStiffness * (Height - FoundationTarget) + Interaction + Breathing;
|
||||||
|
float NextVelocity = (Velocity + Acceleration * Dt) * exp(-Damping * Dt);
|
||||||
|
float NextHeight = Height + NextVelocity * Dt;
|
||||||
|
TargetState[Pixel] = float4(NextHeight, NextVelocity, NextImprint, 0.0);
|
||||||
|
}
|
||||||
@@ -423,6 +423,7 @@ void RecordApplyFrame(const FApplyFrameSample& Sample)
|
|||||||
"\"active_tasks\":%d,\"running_workers\":%d,\"task_cap\":%d,"
|
"\"active_tasks\":%d,\"running_workers\":%d,\"task_cap\":%d,"
|
||||||
"\"pending_tiles\":%d,\"pending_results_estimate\":%d,"
|
"\"pending_tiles\":%d,\"pending_results_estimate\":%d,"
|
||||||
"\"results_drained\":%d,\"apply_results\":%d,\"meshes_applied\":%d,\"max_applies\":%d,"
|
"\"results_drained\":%d,\"apply_results\":%d,\"meshes_applied\":%d,\"max_applies\":%d,"
|
||||||
|
"\"supplemental_far_applies\":%d,"
|
||||||
"\"apply_budget_ms\":%s,\"count_limit_reached\":%d,\"time_limit_reached\":%d,"
|
"\"apply_budget_ms\":%s,\"count_limit_reached\":%d,\"time_limit_reached\":%d,"
|
||||||
"\"pending_by_lod\":%s,\"apply_results_by_lod\":%s,"
|
"\"pending_by_lod\":%s,\"apply_results_by_lod\":%s,"
|
||||||
"\"meshes_by_lod\":%s,\"apply_ms_by_lod\":%s"),
|
"\"meshes_by_lod\":%s,\"apply_ms_by_lod\":%s"),
|
||||||
@@ -432,7 +433,8 @@ void RecordApplyFrame(const FApplyFrameSample& Sample)
|
|||||||
Sample.RunningWorkerTasks, Sample.TaskCap, Sample.PendingTiles,
|
Sample.RunningWorkerTasks, Sample.TaskCap, Sample.PendingTiles,
|
||||||
Sample.PendingResultsEstimate, Sample.ResultsDrained, Sample.ApplyResultsByLevelTotal,
|
Sample.PendingResultsEstimate, Sample.ResultsDrained, Sample.ApplyResultsByLevelTotal,
|
||||||
Sample.MeshesApplied,
|
Sample.MeshesApplied,
|
||||||
Sample.MaxApplies, *Number(Sample.ApplyBudgetMilliseconds),
|
Sample.MaxApplies, Sample.SupplementalFarApplies,
|
||||||
|
*Number(Sample.ApplyBudgetMilliseconds),
|
||||||
Sample.bCountLimitReached ? 1 : 0, Sample.bTimeLimitReached ? 1 : 0,
|
Sample.bCountLimitReached ? 1 : 0, Sample.bTimeLimitReached ? 1 : 0,
|
||||||
*PendingByLevel, *ApplyResultsByLevel, *MeshesByLevel, *ApplyMillisecondsByLevel));
|
*PendingByLevel, *ApplyResultsByLevel, *MeshesByLevel, *ApplyMillisecondsByLevel));
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -268,6 +268,12 @@ namespace
|
|||||||
GVoxelForgeStreamingResultBacklogCap,
|
GVoxelForgeStreamingResultBacklogCap,
|
||||||
TEXT("Maximum queued results plus admitted tile tasks; 0 disables the safety cap. This is separate from the worker-slot cap."));
|
TEXT("Maximum queued results plus admitted tile tasks; 0 disables the safety cap. This is separate from the worker-slot cap."));
|
||||||
|
|
||||||
|
int32 GVoxelForgeStreamingFarApplySupplement = 1;
|
||||||
|
FAutoConsoleVariableRef CVarVoxelForgeStreamingFarApplySupplement(
|
||||||
|
TEXT("voxel.StreamingFarApplySupplement"),
|
||||||
|
GVoxelForgeStreamingFarApplySupplement,
|
||||||
|
TEXT("After the normal visible-apply count is reached, allow one far mesh apply when half of the frame time budget remains; 0 disables the supplemental far lane."));
|
||||||
|
|
||||||
int32 GVoxelForgeStreamingReadyQueueDepth = 64;
|
int32 GVoxelForgeStreamingReadyQueueDepth = 64;
|
||||||
FAutoConsoleVariableRef CVarVoxelForgeStreamingReadyQueueDepth(
|
FAutoConsoleVariableRef CVarVoxelForgeStreamingReadyQueueDepth(
|
||||||
TEXT("voxel.StreamingReadyQueueDepth"),
|
TEXT("voxel.StreamingReadyQueueDepth"),
|
||||||
@@ -2223,6 +2229,12 @@ void AVoxelWorld::BeginPlay()
|
|||||||
}
|
}
|
||||||
FParse::Value(FCommandLine::Get(), TEXT("voxel.StreamingStartupApplyBoost="),
|
FParse::Value(FCommandLine::Get(), TEXT("voxel.StreamingStartupApplyBoost="),
|
||||||
GVoxelForgeStreamingStartupApplyBoost);
|
GVoxelForgeStreamingStartupApplyBoost);
|
||||||
|
if (FParse::Value(FCommandLine::Get(), TEXT("voxel.StreamingFarApplySupplement="),
|
||||||
|
GVoxelForgeStreamingFarApplySupplement))
|
||||||
|
{
|
||||||
|
GVoxelForgeStreamingFarApplySupplement =
|
||||||
|
GVoxelForgeStreamingFarApplySupplement != 0 ? 1 : 0;
|
||||||
|
}
|
||||||
// Headless BuildHost does not reliably apply arbitrary console-variable command-line tokens
|
// Headless BuildHost does not reliably apply arbitrary console-variable command-line tokens
|
||||||
// before BeginPlay; parse the test switches explicitly so A/B runs exercise the requested path.
|
// before BeginPlay; parse the test switches explicitly so A/B runs exercise the requested path.
|
||||||
// BuildHost sans interface n'applique pas toujours les CVar arbitraires avant BeginPlay ;
|
// BuildHost sans interface n'applique pas toujours les CVar arbitraires avant BeginPlay ;
|
||||||
@@ -2737,6 +2749,7 @@ void AVoxelWorld::BeginPlay()
|
|||||||
"\"streaming_collision_lookahead\":%d,\"core_region_concurrent_admission\":%d,"
|
"\"streaming_collision_lookahead\":%d,\"core_region_concurrent_admission\":%d,"
|
||||||
"\"collision_gate\":%d,"
|
"\"collision_gate\":%d,"
|
||||||
"\"streaming_result_backlog_cap\":%d,\"streaming_startup_apply_boost\":%d,"
|
"\"streaming_result_backlog_cap\":%d,\"streaming_startup_apply_boost\":%d,"
|
||||||
|
"\"streaming_far_apply_supplement\":%d,"
|
||||||
"\"streaming_background_worker_reserve\":%d,\"core_parallel_for_background_priority\":%d,"
|
"\"streaming_background_worker_reserve\":%d,\"core_parallel_for_background_priority\":%d,"
|
||||||
"\"streaming_tile_background_low\":%d,\"streaming_tile_background_normal\":%d,"
|
"\"streaming_tile_background_low\":%d,\"streaming_tile_background_normal\":%d,"
|
||||||
"\"core_column_cache_direct_mapped\":%d,"
|
"\"core_column_cache_direct_mapped\":%d,"
|
||||||
@@ -2758,6 +2771,7 @@ void AVoxelWorld::BeginPlay()
|
|||||||
GVoxelForgeStreamingCollisionLookahead, GVoxelForgeCoreRegionConcurrentAdmission,
|
GVoxelForgeStreamingCollisionLookahead, GVoxelForgeCoreRegionConcurrentAdmission,
|
||||||
GVoxelForgeCollisionGate,
|
GVoxelForgeCollisionGate,
|
||||||
GVoxelForgeStreamingResultBacklogCap, GVoxelForgeStreamingStartupApplyBoost,
|
GVoxelForgeStreamingResultBacklogCap, GVoxelForgeStreamingStartupApplyBoost,
|
||||||
|
GVoxelForgeStreamingFarApplySupplement,
|
||||||
GVoxelForgeStreamingBackgroundWorkerReserve,
|
GVoxelForgeStreamingBackgroundWorkerReserve,
|
||||||
GVoxelForgeCoreParallelForBackgroundPriority,
|
GVoxelForgeCoreParallelForBackgroundPriority,
|
||||||
GVoxelForgeStreamingTileBackgroundLow,
|
GVoxelForgeStreamingTileBackgroundLow,
|
||||||
@@ -9881,6 +9895,7 @@ void AVoxelWorld::ProcessPendingChunks()
|
|||||||
const double ApplyBudgetSeconds =
|
const double ApplyBudgetSeconds =
|
||||||
static_cast<double>(ApplyBudgetMilliseconds) * 0.001;
|
static_cast<double>(ApplyBudgetMilliseconds) * 0.001;
|
||||||
int32 MeshesApplied = 0;
|
int32 MeshesApplied = 0;
|
||||||
|
int32 SupplementalFarApplies = 0;
|
||||||
int32 ResultsDrained = 0;
|
int32 ResultsDrained = 0;
|
||||||
int32 ApplyResultsByLevelTotal = 0;
|
int32 ApplyResultsByLevelTotal = 0;
|
||||||
double ApplySeconds = 0.0;
|
double ApplySeconds = 0.0;
|
||||||
@@ -9888,6 +9903,8 @@ void AVoxelWorld::ProcessPendingChunks()
|
|||||||
TStaticArray<int32, 9> MeshesAppliedByLevel{};
|
TStaticArray<int32, 9> MeshesAppliedByLevel{};
|
||||||
TStaticArray<double, 9> ApplySecondsByLevel{};
|
TStaticArray<double, 9> ApplySecondsByLevel{};
|
||||||
bool bTimeBudgetHit = false;
|
bool bTimeBudgetHit = false;
|
||||||
|
bool bForceSupplementalFarDequeue = false;
|
||||||
|
bool bDequeuedSupplementalFar = false;
|
||||||
const auto RecordApplyFrame = [&]()
|
const auto RecordApplyFrame = [&]()
|
||||||
{
|
{
|
||||||
if (!VoxelForgeStartupTrace::IsActive()) return;
|
if (!VoxelForgeStartupTrace::IsActive()) return;
|
||||||
@@ -9911,6 +9928,7 @@ void AVoxelWorld::ProcessPendingChunks()
|
|||||||
Sample.ResultsDrained = ResultsDrained;
|
Sample.ResultsDrained = ResultsDrained;
|
||||||
Sample.ApplyResultsByLevelTotal = ApplyResultsByLevelTotal;
|
Sample.ApplyResultsByLevelTotal = ApplyResultsByLevelTotal;
|
||||||
Sample.MeshesApplied = MeshesApplied;
|
Sample.MeshesApplied = MeshesApplied;
|
||||||
|
Sample.SupplementalFarApplies = SupplementalFarApplies;
|
||||||
Sample.MaxApplies = MaxApplies;
|
Sample.MaxApplies = MaxApplies;
|
||||||
Sample.ApplyBudgetMilliseconds = ApplyBudgetMilliseconds;
|
Sample.ApplyBudgetMilliseconds = ApplyBudgetMilliseconds;
|
||||||
Sample.bCountLimitReached = MeshesApplied >= MaxApplies;
|
Sample.bCountLimitReached = MeshesApplied >= MaxApplies;
|
||||||
@@ -9939,6 +9957,7 @@ void AVoxelWorld::ProcessPendingChunks()
|
|||||||
FChunkResult DequeuedChunk;
|
FChunkResult DequeuedChunk;
|
||||||
const auto DequeueNextResult = [&]() -> bool
|
const auto DequeueNextResult = [&]() -> bool
|
||||||
{
|
{
|
||||||
|
bDequeuedSupplementalFar = false;
|
||||||
if (bStep1UploadCorpusReleased
|
if (bStep1UploadCorpusReleased
|
||||||
&& Step1HeldApplyCursor < Step1HeldApplyResults.Num())
|
&& Step1HeldApplyCursor < Step1HeldApplyResults.Num())
|
||||||
{
|
{
|
||||||
@@ -9954,8 +9973,19 @@ void AVoxelWorld::ProcessPendingChunks()
|
|||||||
// Mirror the admission order. A horizon result that completed earlier must not consume
|
// Mirror the admission order. A horizon result that completed earlier must not consume
|
||||||
// the game-thread apply budget ahead of collision support, an edit, or ordinary LOD0.
|
// the game-thread apply budget ahead of collision support, an edit, or ordinary LOD0.
|
||||||
if (CriticalProcessQueue.Dequeue(DequeuedChunk)
|
if (CriticalProcessQueue.Dequeue(DequeuedChunk)
|
||||||
|| EditedProcessQueue.Dequeue(DequeuedChunk)
|
|| EditedProcessQueue.Dequeue(DequeuedChunk))
|
||||||
|| NearProcessQueue.Dequeue(DequeuedChunk)
|
{
|
||||||
|
RecordStreamingResultDequeued(DequeuedChunk);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (bForceSupplementalFarDequeue)
|
||||||
|
{
|
||||||
|
if (!ProcessQueue.Dequeue(DequeuedChunk)) return false;
|
||||||
|
bDequeuedSupplementalFar = true;
|
||||||
|
RecordStreamingResultDequeued(DequeuedChunk);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (NearProcessQueue.Dequeue(DequeuedChunk)
|
||||||
|| ProcessQueue.Dequeue(DequeuedChunk))
|
|| ProcessQueue.Dequeue(DequeuedChunk))
|
||||||
{
|
{
|
||||||
RecordStreamingResultDequeued(DequeuedChunk);
|
RecordStreamingResultDequeued(DequeuedChunk);
|
||||||
@@ -10104,6 +10134,10 @@ void AVoxelWorld::ProcessPendingChunks()
|
|||||||
if (bMeshApplied)
|
if (bMeshApplied)
|
||||||
{
|
{
|
||||||
++MeshesApplied;
|
++MeshesApplied;
|
||||||
|
if (bDequeuedSupplementalFar && ResultLevel >= 3)
|
||||||
|
{
|
||||||
|
++SupplementalFarApplies;
|
||||||
|
}
|
||||||
if (ResultLevel >= 0 && ResultLevel < 9)
|
if (ResultLevel >= 0 && ResultLevel < 9)
|
||||||
{
|
{
|
||||||
++MeshesAppliedByLevel[ResultLevel];
|
++MeshesAppliedByLevel[ResultLevel];
|
||||||
@@ -10122,6 +10156,19 @@ void AVoxelWorld::ProcessPendingChunks()
|
|||||||
}
|
}
|
||||||
if (MeshesApplied >= MaxApplies)
|
if (MeshesApplied >= MaxApplies)
|
||||||
{
|
{
|
||||||
|
const double DrainElapsedSeconds = FPlatformTime::ToSeconds64(
|
||||||
|
FPlatformTime::Cycles64() - DrainStartCycles);
|
||||||
|
const bool bCanSupplementFar = MeshesApplied == MaxApplies
|
||||||
|
&& SupplementalFarApplies == 0
|
||||||
|
&& !bUseStartupApplyBoost
|
||||||
|
&& GVoxelForgeStreamingFarApplySupplement != 0
|
||||||
|
&& ApplyBudgetSeconds > 0.0
|
||||||
|
&& DrainElapsedSeconds < ApplyBudgetSeconds * 0.5;
|
||||||
|
if (bCanSupplementFar)
|
||||||
|
{
|
||||||
|
bForceSupplementalFarDequeue = true;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -15360,6 +15407,8 @@ bool AVoxelWorld::ApplyMeshToTile(FChunkResult& Result)
|
|||||||
const int32 NumIndices = Result.NumTriangles * 3;
|
const int32 NumIndices = Result.NumTriangles * 3;
|
||||||
const int32 FirstCapIndex = FMath::Clamp(
|
const int32 FirstCapIndex = FMath::Clamp(
|
||||||
NumIndices - Result.NumCeilingTriangles * 3, 0, NumIndices);
|
NumIndices - Result.NumCeilingTriangles * 3, 0, NumIndices);
|
||||||
|
TFuture<ERealtimeMeshProxyUpdateStatus> ProxyPublicationFuture;
|
||||||
|
bool bHasProxyPublicationFuture = false;
|
||||||
{
|
{
|
||||||
TRACE_CPUPROFILER_EVENT_SCOPE(VoxelForge_RMC_CreateSections);
|
TRACE_CPUPROFILER_EVENT_SCOPE(VoxelForge_RMC_CreateSections);
|
||||||
if (bHasGroundTris)
|
if (bHasGroundTris)
|
||||||
@@ -15369,9 +15418,10 @@ bool AVoxelWorld::ApplyMeshToTile(FChunkResult& Result)
|
|||||||
const FRealtimeMeshStreamRange GroundRange =
|
const FRealtimeMeshStreamRange GroundRange =
|
||||||
URealtimeMeshBlueprintFunctionLibrary::MakeStreamRange(
|
URealtimeMeshBlueprintFunctionLibrary::MakeStreamRange(
|
||||||
0, Result.NumVertices, 0, FirstCapIndex);
|
0, Result.NumVertices, 0, FirstCapIndex);
|
||||||
RTMesh->CreateSection(
|
ProxyPublicationFuture = RTMesh->CreateSection(
|
||||||
FRealtimeMeshSectionKey::CreateForPolyGroup(GroupKey, 0),
|
FRealtimeMeshSectionKey::CreateForPolyGroup(GroupKey, 0),
|
||||||
GroundConfig, GroundRange, /*bShouldCreateCollision*/ bWantsCollision);
|
GroundConfig, GroundRange, /*bShouldCreateCollision*/ bWantsCollision);
|
||||||
|
bHasProxyPublicationFuture = true;
|
||||||
}
|
}
|
||||||
if (bHasCeilingTris)
|
if (bHasCeilingTris)
|
||||||
{
|
{
|
||||||
@@ -15380,11 +15430,48 @@ bool AVoxelWorld::ApplyMeshToTile(FChunkResult& Result)
|
|||||||
const FRealtimeMeshStreamRange CapRange =
|
const FRealtimeMeshStreamRange CapRange =
|
||||||
URealtimeMeshBlueprintFunctionLibrary::MakeStreamRange(
|
URealtimeMeshBlueprintFunctionLibrary::MakeStreamRange(
|
||||||
0, Result.NumVertices, FirstCapIndex, NumIndices);
|
0, Result.NumVertices, FirstCapIndex, NumIndices);
|
||||||
RTMesh->CreateSection(
|
ProxyPublicationFuture = RTMesh->CreateSection(
|
||||||
FRealtimeMeshSectionKey::CreateForPolyGroup(GroupKey, 1),
|
FRealtimeMeshSectionKey::CreateForPolyGroup(GroupKey, 1),
|
||||||
CapConfig, CapRange, /*bShouldCreateCollision*/ bWantsCollision);
|
CapConfig, CapRange, /*bShouldCreateCollision*/ bWantsCollision);
|
||||||
|
bHasProxyPublicationFuture = true;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
const uint64 ProxyPublicationSubmitCycles = FPlatformTime::Cycles64();
|
||||||
|
if (bTraceRmcTiming && bHasProxyPublicationFuture)
|
||||||
|
{
|
||||||
|
const TWeakObjectPtr<AVoxelWorld> WeakWorld(this);
|
||||||
|
const TWeakObjectPtr<URealtimeMeshComponent> WeakComponent(MeshComp);
|
||||||
|
const FVoxelTileKey PublishedTile = Tile;
|
||||||
|
const uint64 RequestOrdinal = Result.RequestOrdinal;
|
||||||
|
const uint64 ApplyStartedCycles = ApplyStartCycles;
|
||||||
|
ProxyPublicationFuture.Next(
|
||||||
|
[WeakWorld, WeakComponent, PublishedTile, RequestOrdinal,
|
||||||
|
ApplyStartedCycles, ProxyPublicationSubmitCycles](
|
||||||
|
ERealtimeMeshProxyUpdateStatus Status)
|
||||||
|
{
|
||||||
|
if (!VoxelForgeStartupTrace::IsActive()) return;
|
||||||
|
const uint64 PublishedCycles = FPlatformTime::Cycles64();
|
||||||
|
const URealtimeMeshComponent* Component = WeakComponent.Get();
|
||||||
|
VoxelForgeStartupTrace::RecordEvent(TEXT("rmc_proxy_published"), FString::Printf(
|
||||||
|
TEXT("\"tile\":[%d,%d,%d],\"level\":%d,\"request_ordinal\":%llu,"
|
||||||
|
"\"proxy_update_status\":%d,\"apply_to_proxy_ms\":%.3f,"
|
||||||
|
"\"section_submit_to_proxy_ms\":%.3f,\"component_visible\":%d,"
|
||||||
|
"\"world_valid\":%d"),
|
||||||
|
PublishedTile.Coord.X, PublishedTile.Coord.Y, PublishedTile.Coord.Z,
|
||||||
|
PublishedTile.Level, static_cast<unsigned long long>(RequestOrdinal),
|
||||||
|
static_cast<int32>(Status),
|
||||||
|
ApplyStartedCycles != 0 && PublishedCycles >= ApplyStartedCycles
|
||||||
|
? FPlatformTime::ToSeconds64(PublishedCycles - ApplyStartedCycles) * 1000.0
|
||||||
|
: 0.0,
|
||||||
|
ProxyPublicationSubmitCycles != 0
|
||||||
|
&& PublishedCycles >= ProxyPublicationSubmitCycles
|
||||||
|
? FPlatformTime::ToSeconds64(
|
||||||
|
PublishedCycles - ProxyPublicationSubmitCycles) * 1000.0
|
||||||
|
: 0.0,
|
||||||
|
Component && Component->IsVisible() ? 1 : 0,
|
||||||
|
WeakWorld.IsValid() ? 1 : 0));
|
||||||
|
});
|
||||||
|
}
|
||||||
if (bTraceRmcTiming)
|
if (bTraceRmcTiming)
|
||||||
{
|
{
|
||||||
TraceSectionConfigMilliseconds = FPlatformTime::ToSeconds64(
|
TraceSectionConfigMilliseconds = FPlatformTime::ToSeconds64(
|
||||||
|
|||||||
@@ -59,6 +59,7 @@ namespace VoxelForgeStartupTrace
|
|||||||
int32 ResultsDrained = 0;
|
int32 ResultsDrained = 0;
|
||||||
int32 ApplyResultsByLevelTotal = 0;
|
int32 ApplyResultsByLevelTotal = 0;
|
||||||
int32 MeshesApplied = 0;
|
int32 MeshesApplied = 0;
|
||||||
|
int32 SupplementalFarApplies = 0;
|
||||||
int32 MaxApplies = 0;
|
int32 MaxApplies = 0;
|
||||||
double ApplyBudgetMilliseconds = 0.0;
|
double ApplyBudgetMilliseconds = 0.0;
|
||||||
bool bCountLimitReached = false;
|
bool bCountLimitReached = false;
|
||||||
|
|||||||
@@ -0,0 +1,303 @@
|
|||||||
|
#include "VoxelForgeLivingPlaneActor.h"
|
||||||
|
|
||||||
|
#include "Engine/TextureRenderTarget2D.h"
|
||||||
|
#include "Engine/TextureRenderTarget.h"
|
||||||
|
#include "Engine/StaticMesh.h"
|
||||||
|
#include "Engine/World.h"
|
||||||
|
#include "Materials/MaterialInstanceDynamic.h"
|
||||||
|
#include "Materials/MaterialInterface.h"
|
||||||
|
#include "RenderGraphBuilder.h"
|
||||||
|
#include "RenderGraphUtils.h"
|
||||||
|
#include "RHICommandList.h"
|
||||||
|
#include "RHIResources.h"
|
||||||
|
#include "DataDrivenShaderPlatformInfo.h"
|
||||||
|
#include "GlobalShader.h"
|
||||||
|
#include "ShaderParameterStruct.h"
|
||||||
|
#include "UObject/ConstructorHelpers.h"
|
||||||
|
#include "Components/StaticMeshComponent.h"
|
||||||
|
|
||||||
|
namespace
|
||||||
|
{
|
||||||
|
constexpr int32 LivingPlaneThreadGroupSize = 8;
|
||||||
|
|
||||||
|
struct FLivingPlaneStepInput
|
||||||
|
{
|
||||||
|
FVector4f ObjectPositionRadius[2] = {FVector4f(0, 0, 0, -1), FVector4f(0, 0, 0, -1)};
|
||||||
|
FVector4f ObjectVelocityPush[2] = {FVector4f(0, 0, 0, 0), FVector4f(0, 0, 0, 0)};
|
||||||
|
float TimeSeconds = 0.0f;
|
||||||
|
};
|
||||||
|
|
||||||
|
class FVFLivingPlaneStepCS final : public FGlobalShader
|
||||||
|
{
|
||||||
|
DECLARE_GLOBAL_SHADER(FVFLivingPlaneStepCS);
|
||||||
|
SHADER_USE_PARAMETER_STRUCT(FVFLivingPlaneStepCS, FGlobalShader);
|
||||||
|
|
||||||
|
BEGIN_SHADER_PARAMETER_STRUCT(FParameters, )
|
||||||
|
SHADER_PARAMETER(FIntPoint, GridSize)
|
||||||
|
SHADER_PARAMETER(float, Dt)
|
||||||
|
SHADER_PARAMETER(float, Dx)
|
||||||
|
SHADER_PARAMETER(float, PlaneLength)
|
||||||
|
SHADER_PARAMETER(float, WaveSpeed)
|
||||||
|
SHADER_PARAMETER(float, Damping)
|
||||||
|
SHADER_PARAMETER(float, FoundationStiffness)
|
||||||
|
SHADER_PARAMETER(float, TimeSeconds)
|
||||||
|
SHADER_PARAMETER(FVector2f, BreathingSettings)
|
||||||
|
SHADER_PARAMETER(FVector4f, ImprintSettings)
|
||||||
|
SHADER_PARAMETER_ARRAY(FVector4f, ObjectPositionRadius, [2])
|
||||||
|
SHADER_PARAMETER_ARRAY(FVector4f, ObjectVelocityPush, [2])
|
||||||
|
SHADER_PARAMETER_RDG_TEXTURE_SRV(Texture2D<float4>, SourceState)
|
||||||
|
SHADER_PARAMETER_RDG_TEXTURE_UAV(RWTexture2D<float4>, TargetState)
|
||||||
|
END_SHADER_PARAMETER_STRUCT()
|
||||||
|
|
||||||
|
public:
|
||||||
|
static bool ShouldCompilePermutation(const FGlobalShaderPermutationParameters& Parameters)
|
||||||
|
{
|
||||||
|
return IsFeatureLevelSupported(Parameters.Platform, ERHIFeatureLevel::SM5);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
IMPLEMENT_GLOBAL_SHADER(FVFLivingPlaneStepCS,
|
||||||
|
"/Plugin/VoxelForgeLivingPlane/LivingPlane.usf", "MainCS", SF_Compute)
|
||||||
|
|
||||||
|
UTextureRenderTarget2D* CreateFieldTarget(UObject* Outer, int32 Size, FName Name)
|
||||||
|
{
|
||||||
|
UTextureRenderTarget2D* Target = NewObject<UTextureRenderTarget2D>(Outer, Name, RF_Transient);
|
||||||
|
Target->RenderTargetFormat = RTF_RGBA32f;
|
||||||
|
Target->bCanCreateUAV = true;
|
||||||
|
Target->bAutoGenerateMips = false;
|
||||||
|
Target->ClearColor = FLinearColor::Transparent;
|
||||||
|
Target->Filter = TF_Bilinear;
|
||||||
|
Target->AddressX = TA_Clamp;
|
||||||
|
Target->AddressY = TA_Clamp;
|
||||||
|
Target->InitAutoFormat(Size, Size);
|
||||||
|
Target->UpdateResourceImmediate(true);
|
||||||
|
return Target;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
AVoxelForgeLivingPlaneActor::AVoxelForgeLivingPlaneActor()
|
||||||
|
{
|
||||||
|
PrimaryActorTick.bCanEverTick = true;
|
||||||
|
PlaneMesh = CreateDefaultSubobject<UStaticMeshComponent>(TEXT("LivingPlaneMesh"));
|
||||||
|
SetRootComponent(PlaneMesh);
|
||||||
|
PlaneMesh->SetCollisionEnabled(ECollisionEnabled::NoCollision);
|
||||||
|
PlaneMesh->SetMobility(EComponentMobility::Movable);
|
||||||
|
static ConstructorHelpers::FObjectFinder<UStaticMesh> PlaneAsset(
|
||||||
|
TEXT("/Engine/BasicShapes/Plane.Plane"));
|
||||||
|
if (PlaneAsset.Succeeded())
|
||||||
|
{
|
||||||
|
PlaneMesh->SetStaticMesh(PlaneAsset.Object);
|
||||||
|
}
|
||||||
|
|
||||||
|
ObjectInputs.SetNum(2);
|
||||||
|
}
|
||||||
|
|
||||||
|
void AVoxelForgeLivingPlaneActor::OnConstruction(const FTransform& Transform)
|
||||||
|
{
|
||||||
|
Super::OnConstruction(Transform);
|
||||||
|
if (PlaneMesh)
|
||||||
|
{
|
||||||
|
// /Engine/BasicShapes/Plane is 100 cm square; one Unreal scale unit
|
||||||
|
// therefore corresponds to one metre of authored plane length.
|
||||||
|
PlaneMesh->SetRelativeScale3D(FVector(PlaneLengthMeters, PlaneLengthMeters, 1.0));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
float AVoxelForgeLivingPlaneActor::GetStableStepSeconds() const
|
||||||
|
{
|
||||||
|
const float Dx = PlaneLengthMeters / static_cast<float>(FMath::Max(GridSize, 16));
|
||||||
|
const float EigenFrequencySquared = 8.0f * WaveSpeedMetersPerSecond * WaveSpeedMetersPerSecond
|
||||||
|
/ FMath::Max(Dx * Dx, 1.0e-8f) + FoundationStiffnessPerSecondSquared;
|
||||||
|
return 1.30f / FMath::Sqrt(FMath::Max(EigenFrequencySquared, 1.0e-8f));
|
||||||
|
}
|
||||||
|
|
||||||
|
void AVoxelForgeLivingPlaneActor::BeginPlay()
|
||||||
|
{
|
||||||
|
Super::BeginPlay();
|
||||||
|
GridSize = FMath::Clamp(GridSize, 16, 2048);
|
||||||
|
PlaneLengthMeters = FMath::Max(PlaneLengthMeters, 1.0f);
|
||||||
|
FixedStepSeconds = GetStableStepSeconds();
|
||||||
|
CreateFieldTargets();
|
||||||
|
|
||||||
|
if (PlaneMaterial && PlaneMesh)
|
||||||
|
{
|
||||||
|
DynamicPlaneMaterial = PlaneMesh->CreateDynamicMaterialInstance(0, PlaneMaterial);
|
||||||
|
if (DynamicPlaneMaterial)
|
||||||
|
{
|
||||||
|
DynamicPlaneMaterial->SetScalarParameterValue(TEXT("PlaneLengthMeters"), PlaneLengthMeters);
|
||||||
|
DynamicPlaneMaterial->SetScalarParameterValue(TEXT("FieldSize"), static_cast<float>(GridSize));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
UpdateMaterialField();
|
||||||
|
}
|
||||||
|
|
||||||
|
void AVoxelForgeLivingPlaneActor::CreateFieldTargets()
|
||||||
|
{
|
||||||
|
FieldTargets[0] = CreateFieldTarget(this, GridSize, TEXT("LivingPlaneStateA"));
|
||||||
|
FieldTargets[1] = CreateFieldTarget(this, GridSize, TEXT("LivingPlaneStateB"));
|
||||||
|
CurrentFieldIndex = 0;
|
||||||
|
SimulationStepCount = 0;
|
||||||
|
SimulationAccumulatorSeconds = 0.0;
|
||||||
|
}
|
||||||
|
|
||||||
|
void AVoxelForgeLivingPlaneActor::SetObjectState(int32 ObjectIndex,
|
||||||
|
const FVFLivingPlaneObjectInput& State)
|
||||||
|
{
|
||||||
|
if (!ObjectInputs.IsValidIndex(ObjectIndex) || ObjectIndex >= 2)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
ObjectInputs[ObjectIndex] = State;
|
||||||
|
}
|
||||||
|
|
||||||
|
void AVoxelForgeLivingPlaneActor::ResetSimulation()
|
||||||
|
{
|
||||||
|
if (!IsInGameThread() || !FieldTargets[0] || !FieldTargets[1])
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
CreateFieldTargets();
|
||||||
|
FixedStepSeconds = GetStableStepSeconds();
|
||||||
|
UpdateMaterialField();
|
||||||
|
}
|
||||||
|
|
||||||
|
void AVoxelForgeLivingPlaneActor::UpdateMaterialField()
|
||||||
|
{
|
||||||
|
if (DynamicPlaneMaterial && FieldTargets[CurrentFieldIndex])
|
||||||
|
{
|
||||||
|
DynamicPlaneMaterial->SetTextureParameterValue(TEXT("LivingField"), FieldTargets[CurrentFieldIndex]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void AVoxelForgeLivingPlaneActor::EnqueueSimulationSteps(int32 NumSteps)
|
||||||
|
{
|
||||||
|
if (NumSteps <= 0 || !FieldTargets[0] || !FieldTargets[1])
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
FTextureRenderTargetResource* ResourceA = FieldTargets[0]->GameThread_GetRenderTargetResource();
|
||||||
|
FTextureRenderTargetResource* ResourceB = FieldTargets[1]->GameThread_GetRenderTargetResource();
|
||||||
|
if (!ResourceA || !ResourceB)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
TArray<FLivingPlaneStepInput> Steps;
|
||||||
|
Steps.Reserve(NumSteps);
|
||||||
|
const FTransform PlaneTransform = GetActorTransform();
|
||||||
|
for (int32 Step = 0; Step < NumSteps; ++Step)
|
||||||
|
{
|
||||||
|
FLivingPlaneStepInput& Input = Steps.AddDefaulted_GetRef();
|
||||||
|
Input.TimeSeconds = static_cast<float>(SimulationStepCount + Step) * FixedStepSeconds;
|
||||||
|
for (int32 ObjectIndex = 0; ObjectIndex < 2; ++ObjectIndex)
|
||||||
|
{
|
||||||
|
if (!ObjectInputs.IsValidIndex(ObjectIndex) || !ObjectInputs[ObjectIndex].bActive)
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
const FVFLivingPlaneObjectInput& Object = ObjectInputs[ObjectIndex];
|
||||||
|
const FVector LocalPosition = PlaneTransform.InverseTransformPosition(Object.WorldPositionCm);
|
||||||
|
const FVector LocalVelocity = PlaneTransform.InverseTransformVector(Object.WorldVelocityCmPerSecond);
|
||||||
|
const float RadiusMeters = FMath::Max(Object.RadiusCm * 0.01f, 0.001f);
|
||||||
|
Input.ObjectPositionRadius[ObjectIndex] = FVector4f(
|
||||||
|
static_cast<float>(LocalPosition.X * 0.01),
|
||||||
|
static_cast<float>(LocalPosition.Y * 0.01),
|
||||||
|
static_cast<float>(LocalPosition.Z * 0.01),
|
||||||
|
RadiusMeters);
|
||||||
|
Input.ObjectVelocityPush[ObjectIndex] = FVector4f(
|
||||||
|
static_cast<float>(LocalVelocity.X * 0.01),
|
||||||
|
static_cast<float>(LocalVelocity.Y * 0.01),
|
||||||
|
static_cast<float>(LocalVelocity.Z * 0.01),
|
||||||
|
FMath::Max(Object.PushAccelerationMetersPerSecondSquared, 0.0f));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const int32 Resolution = GridSize;
|
||||||
|
const int32 InitialFieldIndex = CurrentFieldIndex;
|
||||||
|
const float Dt = static_cast<float>(FixedStepSeconds);
|
||||||
|
const float Dx = PlaneLengthMeters / static_cast<float>(Resolution);
|
||||||
|
const float PlaneLength = PlaneLengthMeters;
|
||||||
|
const float WaveSpeed = WaveSpeedMetersPerSecond;
|
||||||
|
const float Damping = DampingPerSecond;
|
||||||
|
const float Foundation = FoundationStiffnessPerSecondSquared;
|
||||||
|
const float BreathingStrength = bEnableBreathing ? BreathingStrengthMetersPerSecondSquared : 0.0f;
|
||||||
|
const FVector2f BreathingSettings(bEnableBreathing ? 1.0f : 0.0f, BreathingStrength);
|
||||||
|
const FVector4f ImprintSettings(bEnableImprint ? 1.0f : 0.0f,
|
||||||
|
ImprintCaptureMetersPerSecond, FMath::Max(ImprintRecoverySeconds, 1.0e-4f),
|
||||||
|
FMath::Max(ImprintMaxDepthMeters, 0.0f));
|
||||||
|
|
||||||
|
ENQUEUE_RENDER_COMMAND(VoxelForgeLivingPlaneStep)(
|
||||||
|
[ResourceA, ResourceB, Steps = MoveTemp(Steps), InitialFieldIndex, Resolution, Dt, Dx, PlaneLength,
|
||||||
|
WaveSpeed, Damping, Foundation, BreathingSettings, ImprintSettings]
|
||||||
|
(FRHICommandListImmediate& RHICmdList) mutable
|
||||||
|
{
|
||||||
|
FRDGBuilder GraphBuilder(RHICmdList, RDG_EVENT_NAME("VoxelForgeLivingPlane"));
|
||||||
|
FRDGTextureRef Fields[2] = {
|
||||||
|
RegisterExternalTexture(GraphBuilder, ResourceA->GetRenderTargetTexture().GetReference(), TEXT("LivingPlaneStateA")),
|
||||||
|
RegisterExternalTexture(GraphBuilder, ResourceB->GetRenderTargetTexture().GetReference(), TEXT("LivingPlaneStateB"))};
|
||||||
|
TShaderMapRef<FVFLivingPlaneStepCS> ComputeShader(GetGlobalShaderMap(GMaxRHIFeatureLevel));
|
||||||
|
int32 CurrentIndex = InitialFieldIndex;
|
||||||
|
for (const FLivingPlaneStepInput& StepInput : Steps)
|
||||||
|
{
|
||||||
|
const int32 TargetIndex = 1 - CurrentIndex;
|
||||||
|
FVFLivingPlaneStepCS::FParameters* PassParameters =
|
||||||
|
GraphBuilder.AllocParameters<FVFLivingPlaneStepCS::FParameters>();
|
||||||
|
PassParameters->GridSize = FIntPoint(Resolution, Resolution);
|
||||||
|
PassParameters->Dt = Dt;
|
||||||
|
PassParameters->Dx = Dx;
|
||||||
|
PassParameters->PlaneLength = PlaneLength;
|
||||||
|
PassParameters->WaveSpeed = WaveSpeed;
|
||||||
|
PassParameters->Damping = Damping;
|
||||||
|
PassParameters->FoundationStiffness = Foundation;
|
||||||
|
PassParameters->TimeSeconds = StepInput.TimeSeconds;
|
||||||
|
PassParameters->BreathingSettings = BreathingSettings;
|
||||||
|
PassParameters->ImprintSettings = ImprintSettings;
|
||||||
|
for (int32 ObjectIndex = 0; ObjectIndex < 2; ++ObjectIndex)
|
||||||
|
{
|
||||||
|
PassParameters->ObjectPositionRadius[ObjectIndex] = StepInput.ObjectPositionRadius[ObjectIndex];
|
||||||
|
PassParameters->ObjectVelocityPush[ObjectIndex] = StepInput.ObjectVelocityPush[ObjectIndex];
|
||||||
|
}
|
||||||
|
PassParameters->SourceState = GraphBuilder.CreateSRV(Fields[CurrentIndex]);
|
||||||
|
PassParameters->TargetState = GraphBuilder.CreateUAV(Fields[TargetIndex]);
|
||||||
|
FComputeShaderUtils::AddPass(GraphBuilder,
|
||||||
|
RDG_EVENT_NAME("LivingPlaneStep"),
|
||||||
|
ComputeShader, PassParameters,
|
||||||
|
FIntVector(FMath::DivideAndRoundUp(Resolution, LivingPlaneThreadGroupSize),
|
||||||
|
FMath::DivideAndRoundUp(Resolution, LivingPlaneThreadGroupSize), 1));
|
||||||
|
CurrentIndex = TargetIndex;
|
||||||
|
}
|
||||||
|
GraphBuilder.Execute();
|
||||||
|
});
|
||||||
|
|
||||||
|
CurrentFieldIndex = (InitialFieldIndex + NumSteps) & 1;
|
||||||
|
SimulationStepCount += NumSteps;
|
||||||
|
UpdateMaterialField();
|
||||||
|
}
|
||||||
|
|
||||||
|
void AVoxelForgeLivingPlaneActor::Tick(float DeltaSeconds)
|
||||||
|
{
|
||||||
|
Super::Tick(DeltaSeconds);
|
||||||
|
if (FixedStepSeconds <= 0.0 || !FieldTargets[0] || !FieldTargets[1])
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
SimulationAccumulatorSeconds += FMath::Max(static_cast<double>(DeltaSeconds), 0.0);
|
||||||
|
const int32 StepsToRun = FMath::Min(
|
||||||
|
static_cast<int32>(SimulationAccumulatorSeconds / FixedStepSeconds), 32);
|
||||||
|
if (StepsToRun > 0)
|
||||||
|
{
|
||||||
|
SimulationAccumulatorSeconds -= StepsToRun * FixedStepSeconds;
|
||||||
|
EnqueueSimulationSteps(StepsToRun);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void AVoxelForgeLivingPlaneActor::EndPlay(const EEndPlayReason::Type EndPlayReason)
|
||||||
|
{
|
||||||
|
Super::EndPlay(EndPlayReason);
|
||||||
|
DynamicPlaneMaterial = nullptr;
|
||||||
|
FieldTargets[0] = nullptr;
|
||||||
|
FieldTargets[1] = nullptr;
|
||||||
|
}
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
#include "Modules/ModuleManager.h"
|
||||||
|
|
||||||
|
#include "Interfaces/IPluginManager.h"
|
||||||
|
#include "Misc/Paths.h"
|
||||||
|
#include "ShaderCore.h"
|
||||||
|
|
||||||
|
class FVoxelForgeLivingPlaneModule final : public IModuleInterface
|
||||||
|
{
|
||||||
|
public:
|
||||||
|
virtual void StartupModule() override
|
||||||
|
{
|
||||||
|
const TSharedPtr<IPlugin> Plugin = IPluginManager::Get().FindPlugin(TEXT("VoxelForge"));
|
||||||
|
if (!Plugin.IsValid())
|
||||||
|
{
|
||||||
|
UE_LOG(LogTemp, Error, TEXT("VoxelForgeLivingPlane could not find its owning plugin."));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
AddShaderSourceDirectoryMapping(
|
||||||
|
TEXT("/Plugin/VoxelForgeLivingPlane"),
|
||||||
|
FPaths::Combine(Plugin->GetBaseDir(), TEXT("Shaders")));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
IMPLEMENT_MODULE(FVoxelForgeLivingPlaneModule, VoxelForgeLivingPlane)
|
||||||
@@ -0,0 +1,114 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include "CoreMinimal.h"
|
||||||
|
#include "GameFramework/Actor.h"
|
||||||
|
#include "VoxelForgeLivingPlaneActor.generated.h"
|
||||||
|
|
||||||
|
class UMaterialInterface;
|
||||||
|
class UMaterialInstanceDynamic;
|
||||||
|
class UStaticMeshComponent;
|
||||||
|
class UTextureRenderTarget2D;
|
||||||
|
|
||||||
|
USTRUCT(BlueprintType)
|
||||||
|
struct FVFLivingPlaneObjectInput
|
||||||
|
{
|
||||||
|
GENERATED_BODY()
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Object")
|
||||||
|
FVector WorldPositionCm = FVector(0.0, 0.0, 43.0);
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Object")
|
||||||
|
FVector WorldVelocityCmPerSecond = FVector::ZeroVector;
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Object", meta=(ClampMin="0.1"))
|
||||||
|
float RadiusCm = 43.0f;
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Object", meta=(ClampMin="0.0"))
|
||||||
|
float PushAccelerationMetersPerSecondSquared = 36.0f;
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Object")
|
||||||
|
bool bActive = false;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** A render-only, GPU-stepped heightfield plane. Object state is uploaded; the field is never read back. */
|
||||||
|
UCLASS(BlueprintType, Blueprintable)
|
||||||
|
class VOXELFORGELIVINGPLANE_API AVoxelForgeLivingPlaneActor : public AActor
|
||||||
|
{
|
||||||
|
GENERATED_BODY()
|
||||||
|
|
||||||
|
public:
|
||||||
|
AVoxelForgeLivingPlaneActor();
|
||||||
|
|
||||||
|
virtual void BeginPlay() override;
|
||||||
|
virtual void Tick(float DeltaSeconds) override;
|
||||||
|
virtual void OnConstruction(const FTransform& Transform) override;
|
||||||
|
virtual void EndPlay(const EEndPlayReason::Type EndPlayReason) override;
|
||||||
|
|
||||||
|
UFUNCTION(BlueprintCallable, Category="Living Plane|Object")
|
||||||
|
void SetObjectState(int32 ObjectIndex, const FVFLivingPlaneObjectInput& State);
|
||||||
|
|
||||||
|
UFUNCTION(BlueprintCallable, Category="Living Plane")
|
||||||
|
void ResetSimulation();
|
||||||
|
|
||||||
|
UFUNCTION(BlueprintPure, Category="Living Plane")
|
||||||
|
int64 GetSimulationStepCount() const { return SimulationStepCount; }
|
||||||
|
|
||||||
|
UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category="Living Plane")
|
||||||
|
TObjectPtr<UStaticMeshComponent> PlaneMesh;
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadOnly, Category="Living Plane|Rendering")
|
||||||
|
TObjectPtr<UMaterialInterface> PlaneMaterial;
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Simulation", meta=(ClampMin="16", ClampMax="2048"))
|
||||||
|
int32 GridSize = 256;
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Simulation", meta=(ClampMin="1.0", ClampMax="100.0"))
|
||||||
|
float PlaneLengthMeters = 12.0f;
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Simulation", meta=(ClampMin="0.0"))
|
||||||
|
float WaveSpeedMetersPerSecond = 4.5f;
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Simulation", meta=(ClampMin="0.0"))
|
||||||
|
float DampingPerSecond = 1.35f;
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Simulation", meta=(ClampMin="0.0"))
|
||||||
|
float FoundationStiffnessPerSecondSquared = 28.0f;
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Simulation")
|
||||||
|
bool bEnableBreathing = true;
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Simulation", meta=(ClampMin="0.0", ClampMax="30.0"))
|
||||||
|
float BreathingStrengthMetersPerSecondSquared = 4.8f;
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Imprint")
|
||||||
|
bool bEnableImprint = true;
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Imprint", meta=(ClampMin="0.0", ClampMax="1.0"))
|
||||||
|
float ImprintCaptureMetersPerSecond = 0.15f;
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Imprint", meta=(ClampMin="0.1", ClampMax="60.0"))
|
||||||
|
float ImprintRecoverySeconds = 6.0f;
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Imprint", meta=(ClampMin="0.0", ClampMax="1.0"))
|
||||||
|
float ImprintMaxDepthMeters = 0.34f;
|
||||||
|
|
||||||
|
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Living Plane|Object")
|
||||||
|
TArray<FVFLivingPlaneObjectInput> ObjectInputs;
|
||||||
|
|
||||||
|
private:
|
||||||
|
void CreateFieldTargets();
|
||||||
|
void EnqueueSimulationSteps(int32 NumSteps);
|
||||||
|
void UpdateMaterialField();
|
||||||
|
float GetStableStepSeconds() const;
|
||||||
|
|
||||||
|
UPROPERTY(Transient)
|
||||||
|
TObjectPtr<UTextureRenderTarget2D> FieldTargets[2];
|
||||||
|
|
||||||
|
UPROPERTY(Transient)
|
||||||
|
TObjectPtr<UMaterialInstanceDynamic> DynamicPlaneMaterial;
|
||||||
|
|
||||||
|
int32 CurrentFieldIndex = 0;
|
||||||
|
int64 SimulationStepCount = 0;
|
||||||
|
double SimulationAccumulatorSeconds = 0.0;
|
||||||
|
double FixedStepSeconds = 0.0;
|
||||||
|
};
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
using UnrealBuildTool;
|
||||||
|
|
||||||
|
public class VoxelForgeLivingPlane : ModuleRules
|
||||||
|
{
|
||||||
|
public VoxelForgeLivingPlane(ReadOnlyTargetRules Target) : base(Target)
|
||||||
|
{
|
||||||
|
PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs;
|
||||||
|
|
||||||
|
if (Target.Platform == UnrealTargetPlatform.Linux)
|
||||||
|
{
|
||||||
|
bUseUnity = false;
|
||||||
|
}
|
||||||
|
|
||||||
|
PublicDependencyModuleNames.AddRange(new string[]
|
||||||
|
{
|
||||||
|
"Core",
|
||||||
|
"CoreUObject",
|
||||||
|
"Engine"
|
||||||
|
});
|
||||||
|
|
||||||
|
PrivateDependencyModuleNames.AddRange(new string[]
|
||||||
|
{
|
||||||
|
"Projects",
|
||||||
|
"RenderCore",
|
||||||
|
"RHI"
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -494,7 +494,10 @@ namespace VoxelForgeCore
|
|||||||
// Version 223 projects the floor onto one continuous arc-length profile and groups
|
// Version 223 projects the floor onto one continuous arc-length profile and groups
|
||||||
// dense passage samples into coarse tile candidates. Version 224 keeps the same
|
// dense passage samples into coarse tile candidates. Version 224 keeps the same
|
||||||
// continuous floor while relaxing the spline chord tolerance and bounding shell work.
|
// continuous floor while relaxing the spline chord tolerance and bounding shell work.
|
||||||
inline constexpr std::uint32_t WorldAssemblyEvaluatorVersion = 225;
|
// Version 226 unifies the passage influence reach in metres and converts it to core
|
||||||
|
// voxels for every broadphase and proof bound. Version 227 uses the exact smooth-max
|
||||||
|
// support width/6 when deriving that reach.
|
||||||
|
inline constexpr std::uint32_t WorldAssemblyEvaluatorVersion = 227;
|
||||||
|
|
||||||
enum class FieldChannel : std::uint32_t
|
enum class FieldChannel : std::uint32_t
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -33,50 +33,41 @@
|
|||||||
#include <variant>
|
#include <variant>
|
||||||
#include <vector>
|
#include <vector>
|
||||||
|
|
||||||
// L88-only composition split. These switches let the timing harness remove passage work and
|
static bool VF_IsDiagnosticEnvSwitchEnabled(const char* Name)
|
||||||
// resolved boundary-band work independently. They are diagnostic only, never world settings.
|
|
||||||
static bool VF_IsL88DiagnosticSwitchEnabled(const char* Name)
|
|
||||||
{
|
{
|
||||||
const char* Value=std::getenv(Name);
|
const char* Value=std::getenv(Name);
|
||||||
return Value&&Value[0]=='1'&&Value[1]=='\0';
|
return Value&&Value[0]=='1'&&Value[1]=='\0';
|
||||||
}
|
}
|
||||||
|
|
||||||
static bool VF_IsL88PassageCompositionDisabled()
|
|
||||||
{
|
|
||||||
static const bool bDisabled=VF_IsL88DiagnosticSwitchEnabled(
|
|
||||||
"VOXELFORGE_L88_DISABLE_PASSAGES")
|
|
||||||
||VF_IsL88DiagnosticSwitchEnabled("VOXELFORGE_L86_DISABLE_PASSAGE_COMPOSITION");
|
|
||||||
return bDisabled;
|
|
||||||
}
|
|
||||||
|
|
||||||
static bool VF_IsL88BoundaryBandsDisabled()
|
|
||||||
{
|
|
||||||
static const bool bDisabled=VF_IsL88DiagnosticSwitchEnabled(
|
|
||||||
"VOXELFORGE_L88_DISABLE_BOUNDARY_BANDS")
|
|
||||||
||VF_IsL88DiagnosticSwitchEnabled("VOXELFORGE_L86_DISABLE_PASSAGE_COMPOSITION");
|
|
||||||
return bDisabled;
|
|
||||||
}
|
|
||||||
|
|
||||||
static bool VF_IsL88AllowPassageAblationWalkabilityFailure()
|
|
||||||
{
|
|
||||||
static const bool bEnabled=VF_IsL88DiagnosticSwitchEnabled(
|
|
||||||
"VOXELFORGE_L88_ALLOW_PASSAGE_ABLATION_WALKABILITY_FAILURE");
|
|
||||||
return bEnabled;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
static bool VF_IsL98PassagePrepareTraceEnabled()
|
static bool VF_IsL98PassagePrepareTraceEnabled()
|
||||||
{
|
{
|
||||||
static const bool bEnabled=VF_IsL88DiagnosticSwitchEnabled(
|
static const bool bEnabled=VF_IsDiagnosticEnvSwitchEnabled(
|
||||||
"VOXELFORGE_L98_PASSAGE_PREP_TRACE");
|
"VOXELFORGE_L98_PASSAGE_PREP_TRACE");
|
||||||
return bEnabled;
|
return bEnabled;
|
||||||
}
|
}
|
||||||
|
|
||||||
static double VF_L93PassageCarveReachMetres()
|
static bool VF_IsL130PassageReachProofAuditEnabled()
|
||||||
{
|
{
|
||||||
// The generated world uses a fixed bounded tube influence. An explicit diagnostic
|
static const bool bEnabled=VF_IsDiagnosticEnvSwitchEnabled(
|
||||||
// option can still select the legacy point-evaluation path for comparisons.
|
"VOXELFORGE_L130_PASSAGE_REACH_PROOF_AUDIT");
|
||||||
return 4.0;
|
return bEnabled;
|
||||||
|
}
|
||||||
|
|
||||||
|
static constexpr double VF_PassageShellThicknessMetres=0.75;
|
||||||
|
static constexpr double VF_PassageWallReliefMetres=0.21;
|
||||||
|
static constexpr double VF_PassageCornerBlendMetres=0.50;
|
||||||
|
static constexpr double VF_PassageReachMarginMetres=0.01;
|
||||||
|
// Radial support in metres: shell + maximum relief + the smooth-max support
|
||||||
|
// (corner-blend width / 6) + a 1 cm guard for float and bound rounding.
|
||||||
|
static constexpr double VF_PassageInfluenceReachMetres=
|
||||||
|
VF_PassageShellThicknessMetres+VF_PassageWallReliefMetres
|
||||||
|
+VF_PassageCornerBlendMetres/6.0+VF_PassageReachMarginMetres;
|
||||||
|
|
||||||
|
static double VF_PassageInfluenceReachCoreVoxels(
|
||||||
|
std::uint32_t coreVoxelSizeMicrometres)
|
||||||
|
{
|
||||||
|
return VF_PassageInfluenceReachMetres*1000000.0
|
||||||
|
/std::max<std::uint32_t>(1,coreVoxelSizeMicrometres);
|
||||||
}
|
}
|
||||||
|
|
||||||
enum class EPassageDensityQueryCaller : std::size_t
|
enum class EPassageDensityQueryCaller : std::size_t
|
||||||
@@ -306,22 +297,18 @@ public:
|
|||||||
profileStats_(profileStats),profileLod_(profileLod),
|
profileStats_(profileStats),profileLod_(profileLod),
|
||||||
passageCarveReachMetres_(std::isfinite(passageCarveReachMetres)
|
passageCarveReachMetres_(std::isfinite(passageCarveReachMetres)
|
||||||
&&passageCarveReachMetres>=0.0?passageCarveReachMetres
|
&&passageCarveReachMetres>=0.0?passageCarveReachMetres
|
||||||
:VF_L93PassageCarveReachMetres())
|
:VF_PassageInfluenceReachMetres)
|
||||||
{
|
{
|
||||||
mayContainResolvedBoundaryBand_=shape.ResolvedBoundaryBandsIntersectSubvoxelRange(
|
mayContainResolvedBoundaryBand_=shape.ResolvedBoundaryBandsIntersectSubvoxelRange(
|
||||||
minimumSubvoxels_[2],maximumSubvoxels_[2]);
|
minimumSubvoxels_[2],maximumSubvoxels_[2]);
|
||||||
const bool disablePassageComposition=VF_IsL88PassageCompositionDisabled();
|
mayContainTransitionPassageFade_=shape.ResolvedTransitionPassageFadeIntersectsTileXY(
|
||||||
if(VF_IsL88BoundaryBandsDisabled())mayContainResolvedBoundaryBand_=false;
|
minimumSubvoxels_,maximumSubvoxels_);
|
||||||
mayContainTransitionPassageFade_=!disablePassageComposition
|
featureIndices_=shape.BuildPassageFeatureTileList(
|
||||||
&&shape.ResolvedTransitionPassageFadeIntersectsTileXY(
|
minimumSubvoxels_,maximumSubvoxels_);
|
||||||
minimumSubvoxels_,maximumSubvoxels_);
|
|
||||||
if(!disablePassageComposition)
|
|
||||||
featureIndices_=shape.BuildPassageFeatureTileList(
|
|
||||||
minimumSubvoxels_,maximumSubvoxels_);
|
|
||||||
stampIndices_=shape.BuildStampTileList(minimumSubvoxels_,maximumSubvoxels_);
|
stampIndices_=shape.BuildStampTileList(minimumSubvoxels_,maximumSubvoxels_);
|
||||||
cavernFeatureIndices_=shape.BuildCavernFeatureTileList(
|
cavernFeatureIndices_=shape.BuildCavernFeatureTileList(
|
||||||
minimumSubvoxels_,maximumSubvoxels_);
|
minimumSubvoxels_,maximumSubvoxels_);
|
||||||
passageRoutes_.assign(shape.ResolvedLayout&&!disablePassageComposition
|
passageRoutes_.assign(shape.ResolvedLayout
|
||||||
?shape.ResolvedLayout->passages.size():0,0);
|
?shape.ResolvedLayout->passages.size():0,0);
|
||||||
passageRouteSegments_.resize(passageRoutes_.size());
|
passageRouteSegments_.resize(passageRoutes_.size());
|
||||||
for(const std::uint32_t featureIndex:featureIndices_)
|
for(const std::uint32_t featureIndex:featureIndices_)
|
||||||
@@ -1204,8 +1191,8 @@ public:
|
|||||||
if(!ResolvedLayout||ResolvedLayout->passages.empty()||!std::isfinite(X)
|
if(!ResolvedLayout||ResolvedLayout->passages.empty()||!std::isfinite(X)
|
||||||
||!std::isfinite(Y)||!std::isfinite(Z))return density;
|
||!std::isfinite(Y)||!std::isfinite(Z))return density;
|
||||||
constexpr double Q=static_cast<double>(VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel);
|
constexpr double Q=static_cast<double>(VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel);
|
||||||
constexpr double FloorShellThicknessMetres=0.75;
|
constexpr double FloorShellThicknessMetres=VF_PassageShellThicknessMetres;
|
||||||
constexpr double WallReliefMetres=0.21;
|
constexpr double WallReliefMetres=VF_PassageWallReliefMetres;
|
||||||
constexpr double OpenAirSideWallHeightMetres=1.0;
|
constexpr double OpenAirSideWallHeightMetres=1.0;
|
||||||
constexpr double MouthShellBlendMetres=5.0;
|
constexpr double MouthShellBlendMetres=5.0;
|
||||||
const double qPerMetre=Q*1000000.0/
|
const double qPerMetre=Q*1000000.0/
|
||||||
@@ -1544,12 +1531,8 @@ public:
|
|||||||
// the generic stratum transition distance creates a discontinuity at a
|
// the generic stratum transition distance creates a discontinuity at a
|
||||||
// mouth that crosses a wide transition band. Keep that opt-in field whole;
|
// mouth that crosses a wide transition band. Keep that opt-in field whole;
|
||||||
// its host-rock value already supplies the ceiling and bedrock.
|
// its host-rock value already supplies the ceiling and bedrock.
|
||||||
const bool disablePassageComposition=!nativeLattice
|
|
||||||
&&VF_IsL88PassageCompositionDisabled();
|
|
||||||
const bool disableBoundaryBands=!nativeLattice
|
|
||||||
&&VF_IsL88BoundaryBandsDisabled();
|
|
||||||
float boundary=graphDensity;
|
float boundary=graphDensity;
|
||||||
if(!nativeLattice&&!disableBoundaryBands)
|
if(!nativeLattice)
|
||||||
{
|
{
|
||||||
VoxelForgeCore::Linux77Profiler::FScopedPhase l77BandsPhase(
|
VoxelForgeCore::Linux77Profiler::FScopedPhase l77BandsPhase(
|
||||||
VoxelForgeCore::Linux77Profiler::EPhase::Bands);
|
VoxelForgeCore::Linux77Profiler::EPhase::Bands);
|
||||||
@@ -1578,7 +1561,6 @@ public:
|
|||||||
?bound->Resolved->passageCapabilities.hasNativeCaveField
|
?bound->Resolved->passageCapabilities.hasNativeCaveField
|
||||||
:bound->Resolved->passageCapabilities.hasNativeLatticeField))
|
:bound->Resolved->passageCapabilities.hasNativeLatticeField))
|
||||||
{
|
{
|
||||||
if(!disablePassageComposition)
|
|
||||||
{
|
{
|
||||||
VoxelForgeCore::Linux77Profiler::FScopedPhase l77PassagePhase(
|
VoxelForgeCore::Linux77Profiler::FScopedPhase l77PassagePhase(
|
||||||
VoxelForgeCore::Linux77Profiler::EPhase::Passage);
|
VoxelForgeCore::Linux77Profiler::EPhase::Passage);
|
||||||
@@ -1609,7 +1591,6 @@ public:
|
|||||||
ELegacyFieldTerm::ComposerPassage);
|
ELegacyFieldTerm::ComposerPassage);
|
||||||
final=ApplyResolvedCavernFloorCaps(stamped,X,Y,Z);
|
final=ApplyResolvedCavernFloorCaps(stamped,X,Y,Z);
|
||||||
}
|
}
|
||||||
if(!disablePassageComposition)
|
|
||||||
{
|
{
|
||||||
VoxelForgeCore::Linux77Profiler::FScopedPhase l77PassagePhase(
|
VoxelForgeCore::Linux77Profiler::FScopedPhase l77PassagePhase(
|
||||||
VoxelForgeCore::Linux77Profiler::EPhase::Passage);
|
VoxelForgeCore::Linux77Profiler::EPhase::Passage);
|
||||||
@@ -1630,8 +1611,8 @@ public:
|
|||||||
{
|
{
|
||||||
constexpr double Q=static_cast<double>(
|
constexpr double Q=static_cast<double>(
|
||||||
VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel);
|
VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel);
|
||||||
const double qPerMetre=Q*1000000.0/
|
const double reachCoreVoxels=VF_PassageInfluenceReachCoreVoxels(
|
||||||
std::max<std::uint32_t>(1,ResolvedLayout->coreVoxelSizeMicrometres);
|
ResolvedLayout->coreVoxelSizeMicrometres);
|
||||||
const auto overlaps=[&](const double low[3],const double high[3])
|
const auto overlaps=[&](const double low[3],const double high[3])
|
||||||
{
|
{
|
||||||
for(int axis=0;axis<3;++axis)
|
for(int axis=0;axis<3;++axis)
|
||||||
@@ -1648,7 +1629,8 @@ public:
|
|||||||
const auto& p3=passage.controlPoints[
|
const auto& p3=passage.controlPoints[
|
||||||
std::min(segment+2,passage.controlPoints.size()-1)];
|
std::min(segment+2,passage.controlPoints.size()-1)];
|
||||||
const double radius=std::max(static_cast<double>(a.radiusSubvoxels),
|
const double radius=std::max(static_cast<double>(a.radiusSubvoxels),
|
||||||
static_cast<double>(b.radiusSubvoxels))/Q+1.0;
|
static_cast<double>(b.radiusSubvoxels))/Q
|
||||||
|
+reachCoreVoxels+0.01;
|
||||||
double low[3]{},high[3]{};
|
double low[3]{},high[3]{};
|
||||||
for(int axis=0;axis<3;++axis)
|
for(int axis=0;axis<3;++axis)
|
||||||
{
|
{
|
||||||
@@ -2314,18 +2296,8 @@ public:
|
|||||||
const bool bPassageRoutesValid=!bValidatePassageRoutes||ValidatePassageRoutes();
|
const bool bPassageRoutesValid=!bValidatePassageRoutes||ValidatePassageRoutes();
|
||||||
if(!bPassageRoutesValid)
|
if(!bPassageRoutesValid)
|
||||||
{
|
{
|
||||||
if(VF_IsL88PassageCompositionDisabled()
|
Error="compiled shape world contains a solid passage centreline sample";
|
||||||
&&!VF_IsL88BoundaryBandsDisabled()
|
return false;
|
||||||
&&VF_IsL88AllowPassageAblationWalkabilityFailure())
|
|
||||||
{
|
|
||||||
std::printf("[L88PassageAblation] route walkability failed as expected; "
|
|
||||||
"continuing only to measure passage-off/bands-on readiness\n");
|
|
||||||
}
|
|
||||||
else
|
|
||||||
{
|
|
||||||
Error="compiled shape world contains a solid passage centreline sample";
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
Stats.PassageValidationSeconds=std::chrono::duration<double>(Clock::now()-Substage).count();
|
Stats.PassageValidationSeconds=std::chrono::duration<double>(Clock::now()-Substage).count();
|
||||||
Built=true;
|
Built=true;
|
||||||
@@ -3499,7 +3471,7 @@ private:
|
|||||||
double centerZSubvoxels,double queryZSubvoxels,double floorDepthSubvoxels,
|
double centerZSubvoxels,double queryZSubvoxels,double floorDepthSubvoxels,
|
||||||
double qPerMetre)
|
double qPerMetre)
|
||||||
{
|
{
|
||||||
constexpr double CornerBlendMetres=0.50;
|
constexpr double CornerBlendMetres=VF_PassageCornerBlendMetres;
|
||||||
const float floorDistance=static_cast<float>(centerZSubvoxels
|
const float floorDistance=static_cast<float>(centerZSubvoxels
|
||||||
-floorDepthSubvoxels-queryZSubvoxels);
|
-floorDepthSubvoxels-queryZSubvoxels);
|
||||||
return SmoothMaxDistance(roundDistanceSubvoxels,floorDistance,
|
return SmoothMaxDistance(roundDistanceSubvoxels,floorDistance,
|
||||||
@@ -4811,13 +4783,9 @@ private:
|
|||||||
PassageNearestNodes.resize(ResolvedLayout?ResolvedLayout->passages.size():0);
|
PassageNearestNodes.resize(ResolvedLayout?ResolvedLayout->passages.size():0);
|
||||||
if(!ResolvedLayout||ResolvedLayout->passages.empty())return;
|
if(!ResolvedLayout||ResolvedLayout->passages.empty())return;
|
||||||
constexpr double Q=static_cast<double>(VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel);
|
constexpr double Q=static_cast<double>(VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel);
|
||||||
constexpr double PassageShellMetres=0.75;
|
|
||||||
constexpr double ReliefMetres=0.21;
|
|
||||||
constexpr double SweepReachMetres=4.0;
|
|
||||||
constexpr double CornerBlendMetres=0.50;
|
|
||||||
constexpr std::size_t CandidateFineSamplesPerSpan=5;
|
constexpr std::size_t CandidateFineSamplesPerSpan=5;
|
||||||
const double requiredReach=std::max(SweepReachMetres+ReliefMetres
|
const double requiredReachCoreVoxels=VF_PassageInfluenceReachCoreVoxels(
|
||||||
+CornerBlendMetres*0.25,PassageShellMetres+ReliefMetres);
|
ResolvedLayout->coreVoxelSizeMicrometres);
|
||||||
for(std::size_t passageIndex=0;passageIndex<ResolvedLayout->passages.size();
|
for(std::size_t passageIndex=0;passageIndex<ResolvedLayout->passages.size();
|
||||||
++passageIndex)
|
++passageIndex)
|
||||||
{
|
{
|
||||||
@@ -4854,7 +4822,7 @@ private:
|
|||||||
routeMinimum[axis]=std::min(routeMinimum[axis],localMinimum[axis]);
|
routeMinimum[axis]=std::min(routeMinimum[axis],localMinimum[axis]);
|
||||||
routeMaximum[axis]=std::max(routeMaximum[axis],localMaximum[axis]);
|
routeMaximum[axis]=std::max(routeMaximum[axis],localMaximum[axis]);
|
||||||
}
|
}
|
||||||
const double localExtent=requiredReach
|
const double localExtent=requiredReachCoreVoxels
|
||||||
+std::max(prepared.radiusA,prepared.radiusB)/Q+0.01;
|
+std::max(prepared.radiusA,prepared.radiusB)/Q+0.01;
|
||||||
FPassageProofBound proof;
|
FPassageProofBound proof;
|
||||||
for(std::size_t axis=0;axis<3;++axis)
|
for(std::size_t axis=0;axis<3;++axis)
|
||||||
@@ -4872,7 +4840,8 @@ private:
|
|||||||
std::ceil((localMaximum[axis]+localExtent)*Q));
|
std::ceil((localMaximum[axis]+localExtent)*Q));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
const double routeExtent=maximumRadiusSubvoxels/Q+requiredReach+0.01;
|
const double routeExtent=maximumRadiusSubvoxels/Q
|
||||||
|
+requiredReachCoreVoxels+0.01;
|
||||||
const double tileExtent=routeExtent;
|
const double tileExtent=routeExtent;
|
||||||
std::array<std::int64_t,3> featureMinimum{},featureMaximum{};
|
std::array<std::int64_t,3> featureMinimum{},featureMaximum{};
|
||||||
for(std::size_t axis=0;axis<3;++axis)
|
for(std::size_t axis=0;axis<3;++axis)
|
||||||
@@ -4899,6 +4868,113 @@ private:
|
|||||||
BuildPassageFeatureNode(0,PassageFeatureOrder.size());
|
BuildPassageFeatureNode(0,PassageFeatureOrder.size());
|
||||||
if(!PassageProofBounds.empty())
|
if(!PassageProofBounds.empty())
|
||||||
BuildPassageProofNode(0,PassageProofBounds.size());
|
BuildPassageProofNode(0,PassageProofBounds.size());
|
||||||
|
|
||||||
|
if(VF_IsL130PassageReachProofAuditEnabled())
|
||||||
|
{
|
||||||
|
constexpr double Q=static_cast<double>(
|
||||||
|
VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel);
|
||||||
|
constexpr std::array<std::array<double,2>,8> RadialDirections={{
|
||||||
|
{1.0,0.0},{0.7071067811865475,0.7071067811865475},
|
||||||
|
{0.0,1.0},{-0.7071067811865475,0.7071067811865475},
|
||||||
|
{-1.0,0.0},{-0.7071067811865475,-0.7071067811865475},
|
||||||
|
{0.0,-1.0},{0.7071067811865475,-0.7071067811865475}}};
|
||||||
|
constexpr std::array<double,2> CrestHeightsMetres={0.625,0.875};
|
||||||
|
const double qPerMetre=Q*1000000.0/
|
||||||
|
std::max<std::uint32_t>(1,ResolvedLayout->coreVoxelSizeMicrometres);
|
||||||
|
const std::array<double,6> shellOffsetsMetres={0.70,0.78,0.86,0.94,
|
||||||
|
VF_PassageInfluenceReachMetres-0.005,
|
||||||
|
VF_PassageInfluenceReachMetres+0.005};
|
||||||
|
std::uint64_t sampled=0,signRelevant=0,proofMisses=0,featureMisses=0;
|
||||||
|
for(std::size_t passageIndex=0;
|
||||||
|
passageIndex<PassageSegmentData.size();++passageIndex)
|
||||||
|
{
|
||||||
|
const auto& segments=PassageSegmentData[passageIndex];
|
||||||
|
if(segments.empty())continue;
|
||||||
|
const auto& constants=PassageComposerConstants[passageIndex];
|
||||||
|
const std::size_t stride=std::max<std::size_t>(1,
|
||||||
|
(segments.size()+127)/128);
|
||||||
|
for(std::size_t segmentIndex=0;segmentIndex<segments.size();
|
||||||
|
segmentIndex+=stride)
|
||||||
|
{
|
||||||
|
const auto& segment=segments[segmentIndex];
|
||||||
|
if(segment.lengthSq<=1.0e-12)continue;
|
||||||
|
constexpr double T=0.5;
|
||||||
|
const double centerX=segment.a[0]+T*segment.delta[0];
|
||||||
|
const double centerY=segment.a[1]+T*segment.delta[1];
|
||||||
|
const double centerZ=segment.a[2]+T*segment.delta[2];
|
||||||
|
const double radius=segment.radiusA+T*segment.radiusDelta;
|
||||||
|
const double floorDepth=PassageFlatFloorDepthSubvoxels(
|
||||||
|
constants,segmentIndex,T,radius,qPerMetre);
|
||||||
|
const double floorCenterZ=PassageFlatFloorCenterZSubvoxels(
|
||||||
|
constants,segments,segmentIndex,T,segment,centerX,centerY,
|
||||||
|
floorDepth,qPerMetre);
|
||||||
|
for(const double crestHeightMetres:CrestHeightsMetres)
|
||||||
|
{
|
||||||
|
const double sampleZ=floorCenterZ-floorDepth
|
||||||
|
+crestHeightMetres*qPerMetre;
|
||||||
|
const double verticalDelta=sampleZ-centerZ;
|
||||||
|
for(const double shellOffsetMetres:shellOffsetsMetres)
|
||||||
|
{
|
||||||
|
const double radialDistance=radius
|
||||||
|
+shellOffsetMetres*qPerMetre;
|
||||||
|
if(radialDistance*radialDistance
|
||||||
|
<=verticalDelta*verticalDelta)continue;
|
||||||
|
const double horizontalDistance=std::sqrt(
|
||||||
|
radialDistance*radialDistance
|
||||||
|
-verticalDelta*verticalDelta);
|
||||||
|
for(const auto& direction:RadialDirections)
|
||||||
|
{
|
||||||
|
const double sampleQ[3]={
|
||||||
|
centerX+direction[0]*horizontalDistance,
|
||||||
|
centerY+direction[1]*horizontalDistance,sampleZ};
|
||||||
|
const double sampleVoxel[3]={sampleQ[0]/Q,
|
||||||
|
sampleQ[1]/Q,sampleQ[2]/Q};
|
||||||
|
const float contribution=ApplyResolvedStratePassageCarves(
|
||||||
|
0.0f,static_cast<float>(sampleVoxel[0]),
|
||||||
|
static_cast<float>(sampleVoxel[1]),
|
||||||
|
static_cast<float>(sampleVoxel[2]));
|
||||||
|
++sampled;
|
||||||
|
if(std::abs(contribution)<=1.0e-6f)continue;
|
||||||
|
++signRelevant;
|
||||||
|
if(PassageProofNodes.empty()
|
||||||
|
||!PassageProofBoundsOverlapNode(0,sampleVoxel,
|
||||||
|
sampleVoxel))
|
||||||
|
++proofMisses;
|
||||||
|
std::array<std::int64_t,3> sampleMinimum{},sampleMaximum{};
|
||||||
|
for(std::size_t axis=0;axis<3;++axis)
|
||||||
|
{
|
||||||
|
sampleMinimum[axis]=static_cast<std::int64_t>(
|
||||||
|
std::floor(sampleQ[axis]));
|
||||||
|
sampleMaximum[axis]=static_cast<std::int64_t>(
|
||||||
|
std::ceil(sampleQ[axis]));
|
||||||
|
}
|
||||||
|
const auto candidates=BuildPassageFeatureTileList(
|
||||||
|
sampleMinimum,sampleMaximum);
|
||||||
|
const bool featureContainsSample=std::any_of(
|
||||||
|
candidates.begin(),candidates.end(),
|
||||||
|
[&](std::uint32_t featureIndex)
|
||||||
|
{
|
||||||
|
return featureIndex<PassageTileFeatures.size()
|
||||||
|
&&PassageTileFeatures[featureIndex].passageIndex
|
||||||
|
==passageIndex;
|
||||||
|
});
|
||||||
|
if(!featureContainsSample)++featureMisses;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const bool auditPassed=sampled>0&&signRelevant>0
|
||||||
|
&&proofMisses==0&&featureMisses==0;
|
||||||
|
std::printf("[L130PassageReachProofAudit] reach_m=%.3f sampled=%llu sign_relevant=%llu proof_misses=%llu feature_misses=%llu status=%s\n",
|
||||||
|
VF_PassageInfluenceReachMetres,
|
||||||
|
static_cast<unsigned long long>(sampled),
|
||||||
|
static_cast<unsigned long long>(signRelevant),
|
||||||
|
static_cast<unsigned long long>(proofMisses),
|
||||||
|
static_cast<unsigned long long>(featureMisses),
|
||||||
|
auditPassed?"PASS":"FAIL");
|
||||||
|
if(!auditPassed)std::abort();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
bool PassageProofBoundsOverlapNode(int nodeIndex,const double minimum[3],
|
bool PassageProofBoundsOverlapNode(int nodeIndex,const double minimum[3],
|
||||||
@@ -11442,10 +11518,9 @@ inline void FStandaloneShapeWorld::FScopedPassageTileQueries::prepareSweptTubeBa
|
|||||||
const FIntVector& origin,std::int32_t step,std::int32_t cells,std::uint32_t seed)
|
const FIntVector& origin,std::int32_t step,std::int32_t cells,std::uint32_t seed)
|
||||||
{
|
{
|
||||||
constexpr std::int64_t Q=VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel;
|
constexpr std::int64_t Q=VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel;
|
||||||
constexpr double InfluenceMetres=4.0;
|
|
||||||
constexpr double OpenAirSideWallHeightMetres=1.0;
|
constexpr double OpenAirSideWallHeightMetres=1.0;
|
||||||
if(!shape_||step<=0||cells<=0||!std::isfinite(passageCarveReachMetres_)
|
if(!shape_||step<=0||cells<=0||!std::isfinite(passageCarveReachMetres_)
|
||||||
||std::abs(passageCarveReachMetres_-InfluenceMetres)>1.0e-9)return;
|
||std::abs(passageCarveReachMetres_-VF_PassageInfluenceReachMetres)>1.0e-9)return;
|
||||||
const bool profile=profileStats_!=nullptr;
|
const bool profile=profileStats_!=nullptr;
|
||||||
const auto prepareStart=profile?std::chrono::steady_clock::now()
|
const auto prepareStart=profile?std::chrono::steady_clock::now()
|
||||||
:std::chrono::steady_clock::time_point{};
|
:std::chrono::steady_clock::time_point{};
|
||||||
@@ -11459,7 +11534,7 @@ inline void FStandaloneShapeWorld::FScopedPassageTileQueries::prepareSweptTubeBa
|
|||||||
sweep->dimension=cells+3;
|
sweep->dimension=cells+3;
|
||||||
const std::size_t gridSampleCount=static_cast<std::size_t>(sweep->dimension)
|
const std::size_t gridSampleCount=static_cast<std::size_t>(sweep->dimension)
|
||||||
*static_cast<std::size_t>(sweep->dimension)*static_cast<std::size_t>(sweep->dimension);
|
*static_cast<std::size_t>(sweep->dimension)*static_cast<std::size_t>(sweep->dimension);
|
||||||
sweep->influenceSubvoxels=InfluenceMetres*static_cast<double>(Q)*1000000.0
|
sweep->influenceSubvoxels=passageCarveReachMetres_*static_cast<double>(Q)*1000000.0
|
||||||
/std::max<std::uint32_t>(1,shape_->ResolvedLayout
|
/std::max<std::uint32_t>(1,shape_->ResolvedLayout
|
||||||
?shape_->ResolvedLayout->coreVoxelSizeMicrometres:250000);
|
?shape_->ResolvedLayout->coreVoxelSizeMicrometres:250000);
|
||||||
sweep->passageToSweep.assign(passageRouteSegments_.size(),-1);
|
sweep->passageToSweep.assign(passageRouteSegments_.size(),-1);
|
||||||
@@ -11473,7 +11548,7 @@ inline void FStandaloneShapeWorld::FScopedPassageTileQueries::prepareSweptTubeBa
|
|||||||
const double qPerMetre=static_cast<double>(Q)*1000000.0/
|
const double qPerMetre=static_cast<double>(Q)*1000000.0/
|
||||||
std::max<std::uint32_t>(1,shape_->ResolvedLayout
|
std::max<std::uint32_t>(1,shape_->ResolvedLayout
|
||||||
?shape_->ResolvedLayout->coreVoxelSizeMicrometres:250000);
|
?shape_->ResolvedLayout->coreVoxelSizeMicrometres:250000);
|
||||||
const double shellThickness=0.75*qPerMetre;
|
const double shellThickness=VF_PassageShellThicknessMetres*qPerMetre;
|
||||||
const double stampedOutsideDistance=sweep->influenceSubvoxels+1.0;
|
const double stampedOutsideDistance=sweep->influenceSubvoxels+1.0;
|
||||||
constexpr double MouthShellBlendMetres=5.0;
|
constexpr double MouthShellBlendMetres=5.0;
|
||||||
const double wallReliefShell=0.41*qPerMetre;
|
const double wallReliefShell=0.41*qPerMetre;
|
||||||
@@ -11633,7 +11708,7 @@ inline void FStandaloneShapeWorld::FScopedPassageTileQueries::prepareSweptTubeBa
|
|||||||
// more than cornerWidth/6. Skip its work for pairs outside this exact
|
// more than cornerWidth/6. Skip its work for pairs outside this exact
|
||||||
// support, with a small subvoxel guard for float rounding. The final
|
// support, with a small subvoxel guard for float rounding. The final
|
||||||
// shell union only consumes negative distances.
|
// shell union only consumes negative distances.
|
||||||
constexpr double CornerBlendMetres=0.50;
|
constexpr double CornerBlendMetres=VF_PassageCornerBlendMetres;
|
||||||
const float roundDistanceFloat=static_cast<float>(signedDistance);
|
const float roundDistanceFloat=static_cast<float>(signedDistance);
|
||||||
const float floorDistanceFloat=static_cast<float>(floorCenterZ-floorDepth-qz);
|
const float floorDistanceFloat=static_cast<float>(floorCenterZ-floorDepth-qz);
|
||||||
const double flatDistanceLowerBound=std::max(
|
const double flatDistanceLowerBound=std::max(
|
||||||
|
|||||||
@@ -0,0 +1,53 @@
|
|||||||
|
# VoxelForge — read-only design review: per-block bit flags (masks) to skip work
|
||||||
|
|
||||||
|
You are reviewing an idea against real code. **Do not edit, create or delete any file, and do not build or run
|
||||||
|
anything.** Read code and reason. Be thorough; mark each claim as [CODE] (with file:line), [MEASURED] (from a report),
|
||||||
|
or [INFERENCE].
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
VoxelForge Core (`Standalone/Core`) generates a density field: negative = solid, positive = air, 25 cm voxels, 32³ tiles,
|
||||||
|
LOD0–8 clipmap, marching cubes. The production mesher for the standalone path is in `Standalone/Source/...`, and Core is
|
||||||
|
also compiled into Unreal. Same seed must give the same world on every machine and worker count; byte-identity with
|
||||||
|
older versions is NOT required.
|
||||||
|
|
||||||
|
Branch `assembler/world` (HEAD `7022d02`'s parent line, currently `4041d4b`) has the smooth inter-strate tunnels.
|
||||||
|
Readiness target: LOD0–2 near terrain in ~1 s on arrival. Measured today: SW ~0.7–0.8 s, VS ~0.9 s, FI/TN ~1.0 s
|
||||||
|
headless. In Linux UE, start→LOD0–2 is ~2.2–2.5 s. At 30 m/s in UE, near coverage is ~40%.
|
||||||
|
|
||||||
|
Reports to consult: `Standalone/Research/WorldAssemblerM3/Linux1*/REPORT.md`, especially:
|
||||||
|
- L77/L87/L117, cost census;
|
||||||
|
- L124, shape gate and per-strate timings;
|
||||||
|
- L125/L126/L128, tunnel prepare: per-segment work, 67–73% pair overlap, support up to 7.9 m;
|
||||||
|
- L128, where an exact one-owner slab scheme halved pairs but got slower because of per-sample ownership tests.
|
||||||
|
|
||||||
|
Tile skipping already exists at tile level: box verdicts / classify, where all-solid or all-air tiles are skipped.
|
||||||
|
|
||||||
|
## The idea (from the owner)
|
||||||
|
|
||||||
|
Keep bit flags per block (1 bit per voxel, or per small brick of e.g. 4³ or 8³ voxels): for example "certainly deep
|
||||||
|
solid", "certainly deep air", "near a surface", "touched by a tunnel", "already evaluated". Use them to skip work in
|
||||||
|
generation and meshing, and possibly elsewhere (collision, streaming, edits, AI/nav, lighting).
|
||||||
|
|
||||||
|
## Questions
|
||||||
|
|
||||||
|
1. Where does time actually go per tile today, and which of that work a mask could skip? Cover density fill,
|
||||||
|
classification, mesher (marching cubes, normals/gradients, skirts, seam refinement), passage prepare, and composition.
|
||||||
|
Use measured census numbers where they exist.
|
||||||
|
2. Which flags are sound? A flag must be a proof, never a guess. A mask "certainly solid" must be derived from a
|
||||||
|
conservative bound (the bound must be the supremum of what the evaluator produces), or the mesher will delete
|
||||||
|
surfaces. Which bounds already exist in the code (box verdicts, interval bounds) that a brick-level mask could reuse?
|
||||||
|
Name the existing hazards. For example, `Identity` means `Sdf >= T`, and `ClassifyTile` proving below-TerrainZ solid
|
||||||
|
would delete 3D caves.
|
||||||
|
3. Granularity: per-voxel bits vs per-brick (4³/8³) vs per-column. What does each cost to build, versus what it
|
||||||
|
saves? Consider SIMD and cache behaviour. Marching cubes needs only cells whose corner signs differ, and gradients
|
||||||
|
need a halo.
|
||||||
|
4. "Already evaluated" or "checked" flags: where would they be correct (memoisation of identical queries), and where
|
||||||
|
would they be wrong (nearest-of-many queries like the tunnel segments)? Is there repeated identical evaluation today,
|
||||||
|
for example halo samples shared between neighbouring tiles, or skirt/seam re-queries?
|
||||||
|
5. Other uses: player edits (diff layer), collision-only server tiles, streaming priority, nav. Which would a mask
|
||||||
|
genuinely help, given the multiplayer rule (replicate seed + layout + diff, never geometry)?
|
||||||
|
6. Recommendation: the one or two mask designs most likely to pay off, with the expected gain (a range, marked
|
||||||
|
inference), the measurement that would prove or kill each, and the risks. Keep it short.
|
||||||
|
|
||||||
|
Write plain Markdown, at most ~1,200 words.
|
||||||
@@ -0,0 +1,83 @@
|
|||||||
|
[INFERENCE] **Improve the existing brick-proof system; don’t add a general “checked voxel” mask.** The promising savings are avoided field/composition work and passage preparation, rather than marching-cubes classification.
|
||||||
|
|
||||||
|
References below abbreviate these files: [M = production mesher](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp), [S = ShapeWorldInternal.h](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h), [C = VoxelForgeCore.cpp](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/VoxelForgeCore.cpp), [O = VoxelDensityOpStack.cpp](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/VoxelForge/Private/VoxelDensityOpStack.cpp). L-numbers identify `Standalone/Research/WorldAssemblerM3/LinuxN/REPORT.md`.
|
||||||
|
|
||||||
|
[INFERENCE] Evidence limitation: Linux128 is absent from this checkout. Its supplied 7.9 m support and slower one-owner result remain unverified here. The reports below span different revisions; their percentages must not be presented as today’s census.
|
||||||
|
|
||||||
|
**1. Where time goes**
|
||||||
|
|
||||||
|
[MEASURED] L77’s exclusive FI/TN LOD0–2 worker account:
|
||||||
|
|
||||||
|
| Work | FI | TN |
|
||||||
|
|---|---:|---:|
|
||||||
|
| Total worker CPU/view | 23.02 s | 21.50 s |
|
||||||
|
| Passage composition | 65.9% | 65.4% |
|
||||||
|
| Support | 14.2% | 13.5% |
|
||||||
|
| Base field | 3.2% | 3.3% |
|
||||||
|
| Bands / dispatch | 2.3% / 3.7% | 2.6% / 3.6% |
|
||||||
|
| MC / skirts / refinement, excluding field work | 1.6% / 0.1% / 7.5% | 1.2% / 0.2% / 8.6% |
|
||||||
|
|
||||||
|
[MEASURED] Grid fill occupied 19.058/17.377 worker-seconds, **overlapping** those phase categories. L117’s later census found support zero; its smooth-tunnel regression concentrated about 95% of added tile CPU in passage-feature tiles. Cell classification stayed flat; gradients became cheaper. Thus L77 identifies historical leverage, not current percentages.
|
||||||
|
|
||||||
|
[MEASURED] L125 preparation averaged 514/130/4.1 ms worker work at LOD0/1/2. LOD0 evaluated 3.11–8.49 million segment/sample pairs with 67–72% overlap, although the exact band occupied only 5–7% of grid samples. L126 reduced floor-profile calls 20–23% without useful readiness improvement.
|
||||||
|
|
||||||
|
[CODE] Brick skipping already exists: configurable block verdicts, required-sample masks, and active-block gradient halos (M:646,1053). MC already rejects cells without sign crossings before interpolation/gradients (M:2186). Gradient memoisation exists (M:1956). Skirts issue separate density queries (M:2491); refinement has its own stage (C:2975).
|
||||||
|
|
||||||
|
[INFERENCE] A pre-evaluation proof can save density, composition, and subsequent MC work. A sign mask constructed after density evaluation saves only downstream scanning. Classification itself costs work; no isolated current classification percentage is established by these reports.
|
||||||
|
|
||||||
|
[MEASURED] L124’s shape gate alone took 179–226 ms, with no near-tile admissions before completion. L87 additionally measured a 107–153 ms far-parent gate and no headless collision wait. Tile masks cannot directly remove these gates or explain the supplied UE/headless gap.
|
||||||
|
|
||||||
|
**2. Sound flags and hazards**
|
||||||
|
|
||||||
|
[INFERENCE] For final density \(D\), solid requires an upper bound covering the evaluator’s supremum, \(U<0\); air requires a valid lower bound \(L\ge0\). “Deep” requires an explicit margin. Unknown must stay unknown. A crossing proves an MC-active cell; uniform sampled signs do not prove an entire continuous volume surface-free.
|
||||||
|
|
||||||
|
[CODE] Reusable proofs include:
|
||||||
|
|
||||||
|
- S:5428: halo-expanded box classification.
|
||||||
|
- S:5974: exact surface-column extrema over the MC lattice.
|
||||||
|
- S:6019: floating-island density intervals.
|
||||||
|
- S:5197,6032: native cave intervals.
|
||||||
|
- S:5949: stamp interval composition with outward numerical padding.
|
||||||
|
- O:6998: operation-specific inactivity from SDF lower bounds.
|
||||||
|
|
||||||
|
[CODE] These proofs have guards: passage overlap returns unknown (S:5687); unsupported stamp composition returns unknown (S:5655); stratum crossings are rejected (S:5900). The mesher distinguishes uniform candidates from `bSkipSafe` certificates (M:715,756), and otherwise validates candidates against actual samples (M:1838).
|
||||||
|
|
||||||
|
[CODE] For gated modifiers, `Identity` can mean `Sdf >= T`, not `abs(Sdf) >= T` (O:1496,6998). Noise bounds cannot assume nominal `[-1,1]`: FBM and unclamped cellular bounds are explicitly discussed at O:848,301.
|
||||||
|
|
||||||
|
[INFERENCE] Additional hazards: confusing internal SDF with final density; certifying base terrain before caves, tunnels, shells, stamps or edits; using lattice proofs for off-grid queries; ignoring rounding or gradient halos. A below-`TerrainZ` solid rule applied to 3D caves deletes caves. “Touched by tunnel” should mean conservative candidate overlap; only **proven untouched** authorizes omission.
|
||||||
|
|
||||||
|
**3. Granularity**
|
||||||
|
|
||||||
|
[INFERENCE] Storage per flag for 32³ cells:
|
||||||
|
|
||||||
|
| Granularity | Storage | Construction/tradeoff |
|
||||||
|
|---|---:|---|
|
||||||
|
| Per cell | 4 KiB | 32,768 decisions; expensive unless derived during necessary work |
|
||||||
|
| 4³ bricks | 64 B | 512 proofs; tighter boundaries, more proof/halo overhead |
|
||||||
|
| 8³ bricks | 8 B | 64 proofs; cheaper, more mixed bricks |
|
||||||
|
| XY columns | 128 B | 1,024 decisions; useful mainly for heightfield-specific proofs |
|
||||||
|
|
||||||
|
[INFERENCE] These exclude corner/halo samples, metadata and additional flags. A column bit cannot represent multiple cave intervals. Brick runs preserve contiguous X sampling and SIMD better than scattered per-sample branches; bit packing alone does not establish a speedup.
|
||||||
|
|
||||||
|
[CODE] The existing implementation uses byte arrays and preserves exact samples around active bricks (M:1053–1083). Replacing densities with arbitrary same-sign constants near active cells would corrupt interpolated positions or normals.
|
||||||
|
|
||||||
|
**4. “Already evaluated”**
|
||||||
|
|
||||||
|
[CODE] Passage preparation precedes meshing (C:2747). Its valid sentinel is initialization, not completion: subsequent segments update floor selection, shell minima and round-distance winners, with deterministic tie-breaking (S:11520,11615,11688).
|
||||||
|
|
||||||
|
[INFERENCE] Therefore first-visitor wins is invalid. Memoisation is valid only for identical queries against identical seed/layout/evaluator/edit revision and query semantics; cache the value, not just a bit.
|
||||||
|
|
||||||
|
[MEASURED] L77 counted 8.6–8.9% halo points, without measuring duplicate coordinates. L125 verified 4,900 shared samples across four neighbour pairs and still recorded substantial off-grid skirt/midpoint fallbacks.
|
||||||
|
|
||||||
|
[CODE] Shared-grid reuse and gradient validity caches already exist (M:1031,1956). [INFERENCE] Skirt/seam queries are reuse candidates only when coordinates and semantics actually match; “same voxel” is insufficient.
|
||||||
|
|
||||||
|
**5. Other uses**
|
||||||
|
|
||||||
|
[INFERENCE] Edit-dirty bricks can localize regeneration, with invalidation expanded through normals, seams and collision dependencies. Collision-only servers can skip certified empty regions, but still need boundary geometry. Streaming can prioritize potentially occupied bricks; nav and lighting need clearance/connectivity or visibility data beyond occupancy. Keep masks derived locally from replicated seed + layout + diff, versioned with that state.
|
||||||
|
|
||||||
|
**6. Recommendation**
|
||||||
|
|
||||||
|
- [INFERENCE] **Tune existing 4³/8³ certificates**, caching proofs and emitting contiguous required-sample runs. Plausible incremental near-ready saving: **0–5%**, with regressions possible. Measure proof time, samples avoided, halo expansion, total worker CPU and UE readiness.
|
||||||
|
- [INFERENCE] **Pass required-region information into passage preparation**, rejecting whole segment/brick regions conservatively. Plausible near-ready saving: **0–10%**, highly uncertain because passage overlap currently vetoes proofs. Kill it if pair reduction fails to reduce preparation and end-to-end time.
|
||||||
|
|
||||||
|
[INFERENCE] Both require seam/off-grid validation, adversarial bound checks, edit invalidation tests, and cross-machine/worker determinism. Preserve canonical tie-breaking; old-version byte identity need not constrain a sound redesign.
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
# VoxelForge — open read-only review: what should be done and focused on now?
|
||||||
|
|
||||||
|
You are an outside reviewer. **Do not edit, create or delete any file, and do not build or run anything.** Read the
|
||||||
|
repository and its reports and reason. Be thorough; mark each claim as [CODE] (with file:line), [MEASURED] (from a
|
||||||
|
report), or [INFERENCE].
|
||||||
|
|
||||||
|
## The project, in the owner's words and goals
|
||||||
|
|
||||||
|
- VoxelForge is a UE5 density-field voxel plugin for a **multiplayer** game. The world is made of stacked "strates"
|
||||||
|
(layers) linked by tunnels. The standalone C++ generator is "Core" (`Standalone/Core`), and it is also compiled into
|
||||||
|
Unreal. Read `CLAUDE.md`, `CODEMAP.md`, `ARCHITECTURE.md`, `GDD.md`, `COMPOSER-NOTES.md` and `fable-idea.md` at the
|
||||||
|
plugin root for the game and design intent. The research reports are in
|
||||||
|
`Standalone/Research/WorldAssemblerM3/Linux*/REPORT.md` and the `Astra*/` reviews.
|
||||||
|
- Owner goals stated over time:
|
||||||
|
- the near terrain (LOD0–2) ready in ~1 s on arrival, then far terrain keeps filling in at low priority;
|
||||||
|
- same seed gives the same world on every machine (byte-identity with older versions is NOT required);
|
||||||
|
- smooth for VR and high-speed travel;
|
||||||
|
- all generators (surface, floating islands, caves, dreamcore/liminal, etc.) working in Core;
|
||||||
|
- the system should eventually *invent* strates and worlds from tagged authored content plus a seed;
|
||||||
|
- keep things simple and avoid over-complication.
|
||||||
|
- The current main line is branch `assembler/world`.
|
||||||
|
|
||||||
|
## Your task
|
||||||
|
|
||||||
|
Look at the whole: code, reports, design docs, and the direction of the last few dozen rounds. Tell the owner what
|
||||||
|
should be done and focused on now, whatever it is. That may be performance, features, generation quality, gameplay
|
||||||
|
foundations, architecture, technical debt, tooling, process, or stopping something. Don't assume the recent focus is
|
||||||
|
the right one; question it if the evidence says so.
|
||||||
|
|
||||||
|
Answer:
|
||||||
|
1. **Where the project actually is**: what works, what's fragile, what's missing for the game to exist.
|
||||||
|
2. **The top 3–5 things to focus on now**, ranked, each with why, a rough size, what "done" looks like, and how to
|
||||||
|
tell it worked.
|
||||||
|
3. **What to stop or deprioritise**, and why.
|
||||||
|
4. **Risks or debts** that will bite later if ignored.
|
||||||
|
|
||||||
|
Write plain Markdown, at most ~1,500 words, readable by a non-specialist owner (explain jargon briefly).
|
||||||
@@ -0,0 +1,72 @@
|
|||||||
|
[INFERENCE] **Focus now on a small, persistent multiplayer descent, supported by end-to-end Unreal performance work.** VoxelForge has substantial terrain technology. The larger uncertainty is whether it delivers a reliable, interesting shared expedition. Another sequence of tunnel optimizations will not resolve that.
|
||||||
|
|
||||||
|
Read-only review of `assembler/world` at `044cd6d`; no files changed and no builds, tests, or project execution. [CODE] identifies repository evidence; design-document references establish intent, not implementation. [MEASURED] means reported results, not independently reproduced measurements.
|
||||||
|
|
||||||
|
**1. Where the project actually is**
|
||||||
|
|
||||||
|
[CODE] **Core is real and integrated, but generator implementation remains mixed.** Unreal compiles the canonical standalone sources through validated wrappers. Core dispatches to surface, floating-island and native graph evaluators, with a legacy-generator fallback when appropriate. That is useful coverage, but does not mean every family has a complete native implementation. See [VoxelForgeCore.Build.cs:15](/home/jahni/VoxelM/Plugins/VoxelForge/Source/VoxelForgeCore/VoxelForgeCore.Build.cs:15) and [ShapeWorldInternal.h:600](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:600).
|
||||||
|
|
||||||
|
[MEASURED] **Passage correctness and speed have improved substantially.** L109 simplified tunnels and removed support-restoration machinery, reducing floating-island and tunnel-network near-generation times by approximately 31–38%. L125 reports 21,425/21,425 open route samples, a maximum sampled floor offset of 7.884 mm, and matching cross-compiler/worker hashes. These are meaningful engineering achievements. [L109](Standalone/Research/WorldAssemblerM3/Linux109/REPORT.md), [L125](Standalone/Research/WorldAssemblerM3/Linux125/REPORT.md).
|
||||||
|
|
||||||
|
[MEASURED] **The one-second player experience is not achieved.** L125’s standalone near-ready medians were approximately 0.78–1.03 seconds. L127’s Unreal startup took 2.22–2.51 seconds; at 30 m/s, median near coverage fell from 43.36% to 40.22% after the smooth-tunnel merge. Both versions recovered after stopping. L130 found no measurable timing improvement and skipped Unreal movement. These are different workloads; subtracting their times would not isolate a particular overhead. [L125](Standalone/Research/WorldAssemblerM3/Linux125/REPORT.md), [L127](Standalone/Research/WorldAssemblerM3/Linux127/REPORT.md), [L130](Standalone/Research/WorldAssemblerM3/Linux130/REPORT.md).
|
||||||
|
|
||||||
|
[CODE] **Multiplayer foundations exist; the multiplayer lifecycle remains unfinished here.** There are season hashes, local terrain-edit entry points, and streaming-anchor infrastructure. The architecture explicitly leaves join replication, authoritative edit messages, late-join snapshots and networked season changes open. The edit entry point directly dispatches to local modification paths. [VoxelWorld.h:574](/home/jahni/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelWorld.h:574), [VoxelWorld.cpp:15771](/home/jahni/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:15771), [ARCHITECTURE.md:2408](/home/jahni/VoxelM/Plugins/VoxelForge/ARCHITECTURE.md:2408).
|
||||||
|
|
||||||
|
[CODE] **The composer is more than a proposal, but less than the full content vision.** It rolls candidates, measures them and rejects failures. The intended system additionally combines tagged geometry, materials, decorations, creatures and audio; theme coherence remains an open design item. [VoxelSeasonManifest.cpp:2523](/home/jahni/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelSeasonManifest.cpp:2523), [COMPOSER-NOTES.md:1512](/home/jahni/VoxelM/Plugins/VoxelForge/COMPOSER-NOTES.md:1512).
|
||||||
|
|
||||||
|
[INFERENCE] **This repository demonstrates a capable terrain platform, not yet a validated game loop.** I cannot establish what gameplay exists outside the plugin. The GDD itself identifies “Is the descent fun?” as its first unresolved question. [GDD.md:441](/home/jahni/VoxelM/Plugins/VoxelForge/GDD.md:441).
|
||||||
|
|
||||||
|
**2. Ranked priorities**
|
||||||
|
|
||||||
|
All sizes below are [INFERENCE]: rough focused engineering estimates, excluding substantial art production.
|
||||||
|
|
||||||
|
**1 — Deliver one multiplayer expedition that survives reconnecting. Size: 3–6 weeks.**
|
||||||
|
|
||||||
|
[INFERENCE] Use two contrasting strates and their connecting tunnel. Two players descend, find one destination, uncover or collect something, return, and retain their changes after restart. Include a third player joining after terrain has changed. Avoid expanding immediately into the city economy or community backend.
|
||||||
|
|
||||||
|
[INFERENCE] The necessary foundation is a server-approved world identity, ordered authoritative edits, a saved snapshot plus subsequent edits, and collision around separated players. Give edits stable identifiers so retransmission cannot apply them twice. Reuse the existing local edit and anchor systems.
|
||||||
|
|
||||||
|
[INFERENCE] **Done:** host/client edits agree; a late joiner reconstructs the same terrain; restart preserves it; incompatible world versions fail clearly; remote players have ground when away from the host.
|
||||||
|
|
||||||
|
[INFERENCE] **Success:** a complete shared expedition without terrain disagreement, lost progress or collision failures—and players can explain what made them want to descend again. This tests the project’s central assumptions much earlier than a complete RPG would.
|
||||||
|
|
||||||
|
**2 — Close the Unreal arrival and movement gap. Size: 1–3 weeks initially.**
|
||||||
|
|
||||||
|
[MEASURED] L122 found idle generation capacity while admission was blocked; L123 measured approximately 19 ms from region request to game-thread observation despite sub-millisecond median compilation. Simple lookahead and admission relaxations failed their combined coverage/frame-time gates. L121’s modest extra far-apply lane did improve far coverage. [L121](Standalone/Research/WorldAssemblerM3/Linux121/REPORT.md), [L122](Standalone/Research/WorldAssemblerM3/Linux122/REPORT.md), [L123](Standalone/Research/WorldAssemblerM3/Linux123/REPORT.md).
|
||||||
|
|
||||||
|
[INFERENCE] First establish one current-build, rendered Unreal baseline on intended hardware. Measure arrival through **visible terrain and usable collision**, then walking, 30 m/s travel, turns, teleporting and edits. Record near coverage, far gaps, frame-time spikes and memory separately. NullRHI measurements, which omit rendering, cannot certify VR comfort.
|
||||||
|
|
||||||
|
[INFERENCE] Investigate region completion, admission and publication before another broad density rewrite. A game-thread continuation is a candidate experiment, not a proven solution; keep world mutation on its owning thread.
|
||||||
|
|
||||||
|
[INFERENCE] **Done:** approximately one-second LOD0–2 readiness on the agreed hardware, safe collision, steadily filling far terrain, and an explicit movement-coverage target without worsening frame spikes. **Success:** repeated rendered runs meet those limits, including with remote-player collision work. Do not redefine “ready” as only the nearest handful of tiles.
|
||||||
|
|
||||||
|
**3 — Turn generator breadth into playable variety. Size: 2–4 weeks for the first content slice.**
|
||||||
|
|
||||||
|
[INFERENCE] Establish a compact capability matrix for each family: Core dispatch, entrances/exits, interior connectivity, edits, materials/content, far terrain and performance. Include dreamcore/liminal explicitly; motif generation alone does not establish a convincing anomalous place.
|
||||||
|
|
||||||
|
[MEASURED] L58 illustrates the distinction: its lattice route passed reachability checks, but the review described a corkscrew tunnel rather than the intended open vertical shaft. Technical validity and identity are separate acceptance criteria. [L58](Standalone/Research/WorldAssemblerM3/Linux58/REPORT.md).
|
||||||
|
|
||||||
|
[INFERENCE] Extend the existing offline composer with a small tagged library: a grounded cave, an island environment and one liminal identity, each with coherent materials, sound and a destination. Preserve owner review before publishing a season.
|
||||||
|
|
||||||
|
[INFERENCE] **Done:** several seeds produce recognizably different, coherent places with findable progression and usable encounter spaces. **Success:** players navigate them without developer guidance and distinguish their identities without reading labels. Expand families after this works.
|
||||||
|
|
||||||
|
**4 — Consolidate the evidence and remove obsolete complexity incrementally. Size: 2–4 days, then ongoing.**
|
||||||
|
|
||||||
|
[INFERENCE] Maintain one short current-state page: shipped behavior, open defects, active baseline, hardware and next acceptance test. Keep historical reports as history. Remove rejected experimental paths when they no longer serve a specific comparison; extract stable responsibilities when touching oversized files.
|
||||||
|
|
||||||
|
[INFERENCE] **Done:** a contributor can identify the current evaluator, runtime path, multiplayer gaps and performance limits without reconstructing 130 rounds. **Success:** new work no longer repeats rejected experiments or treats old measurements as current.
|
||||||
|
|
||||||
|
**3. Stop or deprioritise**
|
||||||
|
|
||||||
|
- [INFERENCE] **Pause open-ended tunnel micro-optimization.** L125/L126 missed the older speed bar; L130’s reach correction produced no measurable speedup. Keep correctness fixes, but require a current cost account and a bounded experiment before further optimization. [MEASURED: L126](Standalone/Research/WorldAssemblerM3/Linux126/REPORT.md), [L130](Standalone/Research/WorldAssemblerM3/Linux130/REPORT.md).
|
||||||
|
- [INFERENCE] **Do not implement Astra recommendations mechanically.** Astra1’s support-heavy model predates L109’s simplification. Astra2/3 offer hypotheses, not measured wins for today’s code. Re-profile before adopting them.
|
||||||
|
- [INFERENCE] **Defer GPU generation, a mesher replacement, elaborate discovery systems and large-scale backend work.** None currently resolves the missing shared expedition.
|
||||||
|
- [INFERENCE] **Stop requiring old-version byte identity for deliberate generation changes.** Preserve peer agreement within a version and explicit versioned saves; permit reviewed geometry improvements.
|
||||||
|
|
||||||
|
**4. Debts that will bite later**
|
||||||
|
|
||||||
|
[INFERENCE] **Determinism needs a deployment contract.** Linux Clang/GCC and worker-count agreement is strong evidence, not proof across every intended machine. Include Windows/compiler/CPU combinations actually supported. World identity must cover evaluator, content, layout, scale and edit state—not merely seed. [MEASURED: L130](Standalone/Research/WorldAssemblerM3/Linux130/REPORT.md).
|
||||||
|
|
||||||
|
[CODE] **Scale and fallback complexity remain active integration risks.** Core is opt-in, and runtime queries bridge different coordinate conventions. [VoxelWorld.h:300](/home/jahni/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelWorld.h:300), [VoxelWorld.cpp:15612](/home/jahni/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:15612). [INFERENCE] Test edits, content placement and collision across those boundaries before broadening authoring.
|
||||||
|
|
||||||
|
[INFERENCE] **A passing route audit does not prove a whole strate is enjoyable, connected or suitable for six-player combat.** Retain geometric checks, but add actual traversal, destination discovery and encounter-space evaluation. Those are now more valuable than another polished tunnel benchmark.
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
# Linux 120 — arrival and far streaming
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Keep the current scheduler defaults. Neither tested design meets the full bar. No production C++ change or version bump is retained.
|
||||||
|
|
||||||
|
## Findings
|
||||||
|
|
||||||
|
1. **Far holes can coexist with idle generation workers.** In five UE 30 m/s runs, 575 of 2,153 scheduler samples with missing far tiles had zero generation workers running. In 497 of those 575 samples, completed results were queued. During movement, far tiles waited a median 1.12 s and p95 3.21 s from worker completion to apply; their generation time was p95 10 ms. The main wait is the game-thread result queue and scheduler admission, not expensive far-tile generation. The separate far-parent build also leaves the tile pool idle for 33–34 ms; see the L105 timeline.
|
||||||
|
|
||||||
|
2. **Background priority is not tile preemption.** VR mode assigns both near and horizon tile tasks `BackgroundLow`; UE can defer queued background tasks, but a running tile finishes or is cancelled before another task takes its worker. The tested split (near `BackgroundNormal`, horizon `BackgroundLow`) changed paired near-ready by only +0.01 points and increased far holes by 0.40 points. The scheduler already reserves an 80/20 near/far admission share during fast travel.
|
||||||
|
|
||||||
|
Raising the existing worker cap from 8 to 10 effective tile slots improved paired UE moving near-ready by 1.10 points and reduced far holes by 0.48 points. Headless gains were +0.35 points and −0.41 points. This increases admitted work, not UE background threads (eight were available). The UE max tick reached 16.67 ms in one 10-slot run versus 15.43 ms in the 8-slot set, so the strict worst-tick gate failed.
|
||||||
|
|
||||||
|
3. **LOD2 still depends on far parents.** `LoadCoreTile` rejects LOD2 while required far-world data is not ready. That data contributes to the density field LOD2 meshes sample. Removing the gate without proving a tile’s dependency set would change geometry. The far-world build is deferred until LOD0–1 are ready, so this dependency is on the arrival path.
|
||||||
|
|
||||||
|
4. **At 30 m/s, generation and result throughput limit near readiness.** With the production ready-queue depth, LOD0–2 settled about 1.31 s after first admission. The 10-slot arrival pair improved near settlement by only 19 ms paired-median; whole-view/far-clear time changed by only 18 ms, within run variation, and both arms had zero far holes by 20 s. The larger depth-8 arrival gain does not represent the production default. At depth 8, the 10-slot movement trial did raise near-ready, but its worst UE tick failed the bar. Keep the cap unchanged.
|
||||||
|
|
||||||
|
## Checks
|
||||||
|
|
||||||
|
- Five interleaved 30 m/s movement pairs per platform and five interleaved FI arrival pairs at default ready depth; quiet load stayed below 2.2 on 20 logical CPUs. UE used `taskset` only; headless used `taskset` and the 6 GiB address-space limit. Each commandlet finished well under 30 minutes.
|
||||||
|
- Fast tier: **PASS**, 274 checks / 80 runs, 267.1 s.
|
||||||
|
- Exact hashes: **24/24 PASS** for SW/FI/TN/VS × Clang/GCC × W1/W8/W12; tile and whole-view hashes match the L104 baseline and Clang W1. Source stamp `f3ec37a…`; no version bump is needed.
|
||||||
|
- Unreal build passed with `-NoUBA -MaxParallelActions=4`. `git diff --check` is clean.
|
||||||
|
|
||||||
|
Detailed summaries and logs are under `Saved/Linux/l120/`; the movement and arrival comparisons are in `movement/worker-slots-verified/summary.json`, `movement/priority-split-verified/summary.json`, and `arrival/pairs-rd64.json`.
|
||||||
@@ -0,0 +1,487 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Run one pinned null-RHI FI arrival for Linux 120."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from collections import Counter
|
||||||
|
from datetime import datetime
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
HOST = Path("/home/jahni/VFHost")
|
||||||
|
HOST_PLUGIN_SAVED = HOST / "Plugins/VoxelForge/Saved/Linux/l120"
|
||||||
|
HOST_SAVED = HOST / "Saved/Linux/l120"
|
||||||
|
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
|
||||||
|
OUT = ROOT / "Saved/Linux/l120/arrival"
|
||||||
|
SPAWNS_M = {
|
||||||
|
"FI": (25.8369140625, 3.19921875, -737.0),
|
||||||
|
"SW": (25.8369140625, 3.19921875, -200.0),
|
||||||
|
}
|
||||||
|
MOVE_START_S = 20.0
|
||||||
|
MOVE_DURATION_S = 12.0
|
||||||
|
EXIT_S = MOVE_START_S + MOVE_DURATION_S + 5.0
|
||||||
|
RUN_TIMEOUT_S = 1740
|
||||||
|
|
||||||
|
|
||||||
|
def read_events(path: Path) -> list[dict]:
|
||||||
|
events = []
|
||||||
|
if path.is_file():
|
||||||
|
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||||
|
if line.startswith("{"):
|
||||||
|
try:
|
||||||
|
events.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
return events
|
||||||
|
|
||||||
|
|
||||||
|
def percentile(values: list[float], fraction: float) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
ordered = sorted(values)
|
||||||
|
return ordered[min(len(ordered) - 1, int(round((len(ordered) - 1) * fraction)))]
|
||||||
|
|
||||||
|
|
||||||
|
def sample_unreal_peak_rss() -> tuple[int, int] | None:
|
||||||
|
executable = str(EDITOR).encode()
|
||||||
|
project = str(HOST / "VFHost.uproject").encode()
|
||||||
|
peak: tuple[int, int] | None = None
|
||||||
|
for process in Path("/proc").glob("[0-9]*"):
|
||||||
|
try:
|
||||||
|
command_line = (process / "cmdline").read_bytes()
|
||||||
|
if executable not in command_line or project not in command_line:
|
||||||
|
continue
|
||||||
|
status = (process / "status").read_text(encoding="ascii", errors="ignore")
|
||||||
|
match = re.search(r"^VmHWM:\s+(\d+)\s+kB$", status, re.MULTILINE)
|
||||||
|
if match:
|
||||||
|
rss_kib = int(match.group(1))
|
||||||
|
if peak is None or rss_kib > peak[0]:
|
||||||
|
peak = (rss_kib, int(process.name))
|
||||||
|
except (FileNotFoundError, PermissionError, ProcessLookupError, OSError):
|
||||||
|
continue
|
||||||
|
return peak
|
||||||
|
|
||||||
|
|
||||||
|
def endphysics_gap(stdout: Path) -> tuple[float | None, int | None]:
|
||||||
|
release_times: dict[int, dict[int, datetime]] = {}
|
||||||
|
pattern = re.compile(
|
||||||
|
r"^\[(\d{4}\.\d{2}\.\d{2}-\d{2}\.\d{2}\.\d{2}:\d{3})\].*"
|
||||||
|
r"LogTick: tick\s+(\d+)\s+-+\s+Release tick group ([34])")
|
||||||
|
with stdout.open(encoding="utf-8", errors="replace") as stream:
|
||||||
|
for line in stream:
|
||||||
|
match = pattern.match(line)
|
||||||
|
if not match:
|
||||||
|
continue
|
||||||
|
timestamp = datetime.strptime(match.group(1), "%Y.%m.%d-%H.%M.%S:%f")
|
||||||
|
frame = int(match.group(2))
|
||||||
|
group = int(match.group(3))
|
||||||
|
release_times.setdefault(frame, {})[group] = timestamp
|
||||||
|
gaps = [(round((times[4] - times[3]).total_seconds() * 1000.0, 3), frame)
|
||||||
|
for frame, times in release_times.items() if 3 in times and 4 in times]
|
||||||
|
return max(gaps, default=(None, None))
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, repeat: int, phase: str, ready_depth: int,
|
||||||
|
background_cap: int, worker_reserve: int, helper_background: int,
|
||||||
|
tile_background_low: int, tile_background_normal: int, startup_boost: int,
|
||||||
|
tick_logs: bool, insights: bool) -> dict:
|
||||||
|
label = (f"{phase}-{stratum.lower()}-rd{ready_depth}-bg{background_cap}"
|
||||||
|
f"-reserve{worker_reserve}-helperbg{helper_background}"
|
||||||
|
f"-tilelow{tile_background_low}-tilenormal{tile_background_normal}"
|
||||||
|
f"-ticklog{int(tick_logs)}"
|
||||||
|
f"-boost{startup_boost}-{repeat}")
|
||||||
|
trace = OUT / f"{label}.trace.jsonl"
|
||||||
|
utrace = OUT / f"{label}.utrace"
|
||||||
|
abslog = OUT / f"{label}.log"
|
||||||
|
stdout = OUT / f"{label}.stdout.log"
|
||||||
|
output = HOST_PLUGIN_SAVED / "output" / label
|
||||||
|
userdir = HOST_SAVED / "user" / label
|
||||||
|
zendata = HOST_SAVED / "zen" / label
|
||||||
|
ddc = HOST_SAVED / "DDC"
|
||||||
|
for directory in (OUT, output, userdir, zendata, ddc):
|
||||||
|
directory.mkdir(parents=True, exist_ok=True)
|
||||||
|
trace.unlink(missing_ok=True)
|
||||||
|
utrace.unlink(missing_ok=True)
|
||||||
|
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWNS_M[stratum])
|
||||||
|
command = [
|
||||||
|
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
|
||||||
|
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
|
||||||
|
"-notraceserver", "-nocrashreports",
|
||||||
|
"-DisablePlugins=AndroidFileServer",
|
||||||
|
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
|
||||||
|
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
|
||||||
|
"-archetype=TunnelNetwork", f"-slot={1 if stratum == 'FI' else 0}",
|
||||||
|
"-modes=export", "-gameconfig=1", "-coreworldtest=1",
|
||||||
|
"-ownerfourstrateplan=1", "-corevoxelsizecm=25", "-coreislandtest=0",
|
||||||
|
"-corefallbacktest=0", "-coreofftest=0", "-coreplantest=0",
|
||||||
|
"-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings", "-corehybridmesher=0",
|
||||||
|
f"-out={output}", "-voxel.StreamingCoarseFirst=1",
|
||||||
|
"-voxel.StreamingDynamicNearReserve=1",
|
||||||
|
"-voxel.StreamingLegacyPendingResultCap=0",
|
||||||
|
"-voxel.CoreRegionConcurrentAdmission=1",
|
||||||
|
"-voxel.StreamingResultBacklogCap=256",
|
||||||
|
f"-voxel.StreamingReadyQueueDepth={ready_depth}",
|
||||||
|
f"-voxel.StreamingBackgroundWorkerCap={background_cap}",
|
||||||
|
f"-voxel.StreamingBackgroundWorkerReserve={worker_reserve}",
|
||||||
|
f"-voxel.CoreParallelForBackgroundPriority={helper_background}",
|
||||||
|
f"-voxel.StreamingTileBackgroundLow={tile_background_low}",
|
||||||
|
f"-voxel.StreamingTileBackgroundNormal={tile_background_normal}",
|
||||||
|
"-voxel.StreamingTilePrioritySplit=0",
|
||||||
|
f"-voxel.TestLogTickFunctions={int(tick_logs)}",
|
||||||
|
"-voxel.CoreColumnCacheDirectMapped=0", "-voxel.CoreMCGradientMemoization=1",
|
||||||
|
"-voxel.StreamingCollisionLookahead=1",
|
||||||
|
f"-voxel.StreamingStartupApplyBoost={startup_boost}",
|
||||||
|
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=140.0",
|
||||||
|
f"-voxel.TestMoveStartSeconds={MOVE_START_S:.1f}",
|
||||||
|
f"-voxel.TestMoveDurationSeconds={MOVE_DURATION_S:.1f}",
|
||||||
|
f"-voxel.TestExitSeconds={EXIT_S:.1f}", "-voxel.TestMoveDirection=1,0,0",
|
||||||
|
"-voxel.StreamingSchedulerPolicy=3", "-voxel.TestExitOnSteady=0",
|
||||||
|
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
|
||||||
|
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
|
||||||
|
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
|
||||||
|
f"-abslog={abslog}",
|
||||||
|
]
|
||||||
|
if insights:
|
||||||
|
command += ["-trace=cpu,task,frame", "-statnamedevents",
|
||||||
|
f"-tracefile={utrace}", "-tracefiletrunc"]
|
||||||
|
wrapped = ["taskset", "-c", "0-11", "timeout", f"{RUN_TIMEOUT_S}s", *command]
|
||||||
|
started = time.monotonic()
|
||||||
|
with stdout.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.Popen(wrapped, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT)
|
||||||
|
peak_rss = None
|
||||||
|
while process.poll() is None:
|
||||||
|
observed = sample_unreal_peak_rss()
|
||||||
|
if observed and (peak_rss is None or observed[0] > peak_rss[0]):
|
||||||
|
peak_rss = observed
|
||||||
|
time.sleep(0.25)
|
||||||
|
exit_code = process.wait()
|
||||||
|
observed = sample_unreal_peak_rss()
|
||||||
|
if observed and (peak_rss is None or observed[0] > peak_rss[0]):
|
||||||
|
peak_rss = observed
|
||||||
|
|
||||||
|
events = read_events(trace)
|
||||||
|
worker_backend_config = next(
|
||||||
|
(e for e in events if e.get("type") == "worker_backend_config"), {})
|
||||||
|
worker_cap_readback = next(
|
||||||
|
(e for e in events if e.get("type") == "streaming_worker_cap_readback"), {})
|
||||||
|
samples = [e for e in events if e.get("type") == "streaming_sample"]
|
||||||
|
arrival = [e for e in samples if float(e.get("t_s", 0.0)) < MOVE_START_S]
|
||||||
|
full = next((e for e in arrival
|
||||||
|
if sum(e.get("desired_by_lod", [0] * 9)[:3]) > 0
|
||||||
|
and sum(e.get("settled_by_lod", [0] * 9)[:3])
|
||||||
|
>= sum(e.get("desired_by_lod", [0] * 9)[:3])), None)
|
||||||
|
first_admission = next((e for e in events if e.get("type") == "scheduler_admission"
|
||||||
|
and e.get("desired_rebuilt") == 1), None)
|
||||||
|
request_events = [e for e in events if e.get("type") == "request"]
|
||||||
|
initial_request_events = [e for e in request_events if e.get("epoch") == 0]
|
||||||
|
tile_by_ordinal = {e.get("request_ordinal"): e for e in events
|
||||||
|
if e.get("type") == "tile"}
|
||||||
|
collision_count = sum(1 for e in events if e.get("type") == "collision_ready"
|
||||||
|
and e.get("level", 9) <= 2
|
||||||
|
and full and e.get("t_s", 1e30) <= full["t_s"])
|
||||||
|
initial_tiles = [tile_by_ordinal[r.get("request_ordinal")] for r in initial_request_events
|
||||||
|
if full and r.get("t_s", 1e30) <= full["t_s"]
|
||||||
|
and r.get("request_ordinal") in tile_by_ordinal
|
||||||
|
and tile_by_ordinal[r["request_ordinal"]].get("level", 9) <= 2]
|
||||||
|
burst_start_s = float(first_admission["t_s"]) if first_admission else -1.0
|
||||||
|
burst_end_s = float(full["t_s"]) if full else -1.0
|
||||||
|
work_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "game_thread_work_frame"}
|
||||||
|
work_values = [e for e in work_by_frame.values()
|
||||||
|
if burst_start_s <= float(e.get("t_s", -1.0)) <= burst_end_s]
|
||||||
|
arrival_frame_ids = {int(e.get("frame", -1)) for e in work_values}
|
||||||
|
scheduler_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "scheduler_admission"}
|
||||||
|
scheduler_values = [e for e in events if e.get("type") == "scheduler_admission"
|
||||||
|
and burst_start_s <= float(e.get("t_s", -1.0)) <= burst_end_s]
|
||||||
|
apply_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "apply_frame"}
|
||||||
|
commandlet_tick_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "commandlet_world_tick_frame"}
|
||||||
|
garbage_collect_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "garbage_collect"}
|
||||||
|
# apply_frame N+1 records the interval containing AVoxelWorld::Tick N. Its apply_ms
|
||||||
|
# belongs to Tick N+1, so use apply_by_frame[N] for the same-tick apply breakdown.
|
||||||
|
near_intervals = [apply_by_frame[frame + 1] for frame in arrival_frame_ids
|
||||||
|
if frame + 1 in apply_by_frame]
|
||||||
|
rmc_by_frame: dict[int, dict[str, float]] = {}
|
||||||
|
for event in events:
|
||||||
|
frame = int(event.get("frame", -1))
|
||||||
|
if event.get("type") != "rmc_submit" or frame not in arrival_frame_ids:
|
||||||
|
continue
|
||||||
|
values = rmc_by_frame.setdefault(frame, {
|
||||||
|
"component_acquire_ms": 0.0,
|
||||||
|
"create_section_group_ms": 0.0,
|
||||||
|
"create_sections_ms": 0.0,
|
||||||
|
"collision_handoff_ms": 0.0,
|
||||||
|
})
|
||||||
|
for key in values:
|
||||||
|
values[key] += float(event.get(key, 0.0))
|
||||||
|
collision_completion_by_frame: dict[int, float] = {}
|
||||||
|
for event in events:
|
||||||
|
frame = int(event.get("frame", -1))
|
||||||
|
if event.get("type") == "collision_completion" and frame in arrival_frame_ids:
|
||||||
|
collision_completion_by_frame[frame] = collision_completion_by_frame.get(frame, 0.0) \
|
||||||
|
+ float(event.get("handler_ms", 0.0))
|
||||||
|
near_requests = [r for r in request_events if full and r.get("t_s", 1e30) <= full["t_s"]
|
||||||
|
and r.get("level", 9) <= 2]
|
||||||
|
initial_lod0_counts = Counter(
|
||||||
|
tuple(r.get("tile", ())) for r in near_requests if r.get("level") == 0)
|
||||||
|
repeated_lod0_keys = [
|
||||||
|
{"tile": list(tile), "requests": count}
|
||||||
|
for tile, count in sorted(initial_lod0_counts.items()) if count > 1]
|
||||||
|
initial_scheduler_frames = [e for e in events
|
||||||
|
if e.get("type") == "scheduler_admission" and full
|
||||||
|
and e.get("t_s", 1e30) <= full["t_s"]]
|
||||||
|
worst_interval = max(near_intervals, key=lambda e: float(e.get("frame_ms", 0.0)), default=None)
|
||||||
|
worst_interval_frame_id = int(worst_interval.get("frame", -1)) if worst_interval else -1
|
||||||
|
worst_work_frame_id = worst_interval_frame_id - 1
|
||||||
|
worst_work = work_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_apply = apply_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_scheduler = scheduler_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_rmc = rmc_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_collision_completion_ms = collision_completion_by_frame.get(worst_work_frame_id, 0.0)
|
||||||
|
rmc_values = [rmc_by_frame.get(frame, {}) for frame in arrival_frame_ids]
|
||||||
|
apply_values = [apply_by_frame[frame] for frame in arrival_frame_ids
|
||||||
|
if frame in apply_by_frame]
|
||||||
|
commandlet_tick_values = [commandlet_tick_by_frame[frame] for frame in arrival_frame_ids
|
||||||
|
if frame in commandlet_tick_by_frame]
|
||||||
|
garbage_collect_values = [garbage_collect_by_frame[frame] for frame in arrival_frame_ids
|
||||||
|
if frame in garbage_collect_by_frame]
|
||||||
|
|
||||||
|
def work_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in work_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
def scheduler_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in scheduler_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
def rmc_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in rmc_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
def commandlet_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in commandlet_tick_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
row = {
|
||||||
|
"label": label, "stratum": stratum, "repeat": repeat, "phase": phase,
|
||||||
|
"ready_queue_depth": ready_depth,
|
||||||
|
"requested_background_worker_cap": background_cap,
|
||||||
|
"background_worker_cap": worker_cap_readback.get(
|
||||||
|
"streaming_background_worker_cap", background_cap),
|
||||||
|
"requested_background_worker_reserve": worker_reserve,
|
||||||
|
"background_worker_reserve": worker_backend_config.get(
|
||||||
|
"streaming_background_worker_reserve", worker_cap_readback.get(
|
||||||
|
"background_worker_reserve", worker_reserve)),
|
||||||
|
"requested_core_parallel_for_background_priority": helper_background,
|
||||||
|
"core_parallel_for_background_priority": worker_backend_config.get(
|
||||||
|
"core_parallel_for_background_priority", helper_background),
|
||||||
|
"requested_streaming_tile_background_low": tile_background_low,
|
||||||
|
"streaming_tile_background_low": worker_backend_config.get(
|
||||||
|
"streaming_tile_background_low", tile_background_low),
|
||||||
|
"requested_streaming_tile_background_normal": tile_background_normal,
|
||||||
|
"streaming_tile_background_normal": worker_backend_config.get(
|
||||||
|
"streaming_tile_background_normal", tile_background_normal),
|
||||||
|
"streaming_tile_priority_split": worker_backend_config.get(
|
||||||
|
"streaming_tile_priority_split"),
|
||||||
|
"effective_tile_slots": worker_backend_config.get("effective_tile_slots"),
|
||||||
|
"background_worker_threads": worker_backend_config.get("taskgraph_background_threads"),
|
||||||
|
"startup_apply_boost": startup_boost, "insights": insights,
|
||||||
|
"engine_tick_logs": tick_logs,
|
||||||
|
"exit_code": exit_code,
|
||||||
|
"launch_wrapper": "taskset -c 0-11 timeout 1740s (no address-space limit)",
|
||||||
|
"peak_rss_kib": peak_rss[0] if peak_rss else None,
|
||||||
|
"peak_rss_mib": round(peak_rss[0] / 1024.0, 1) if peak_rss else None,
|
||||||
|
"peak_rss_pid": peak_rss[1] if peak_rss else None,
|
||||||
|
"endphysics_to_postphysics_gap_ms_max": None,
|
||||||
|
"endphysics_gap_frame": None,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"desired_t_s": next((e.get("t_s") for e in events if e.get("type") == "desired_set"), None),
|
||||||
|
"first_admission_t_s": first_admission.get("t_s") if first_admission else None,
|
||||||
|
"lod02_full_settled_t_s": full.get("t_s") if full else None,
|
||||||
|
"near_settle_from_first_admission_s": round(
|
||||||
|
float(full["t_s"]) - float(first_admission["t_s"]), 4)
|
||||||
|
if full and first_admission else None,
|
||||||
|
"desired_by_lod": full.get("desired_by_lod", [])[:3] if full else None,
|
||||||
|
"settled_by_lod": full.get("settled_by_lod", [])[:3] if full else None,
|
||||||
|
"near_admitted_tiles": len(near_requests),
|
||||||
|
"initial_lod0_admissions": sum(initial_lod0_counts.values()),
|
||||||
|
"repeated_lod0_request_keys_before_settle": repeated_lod0_keys,
|
||||||
|
"peak_result_backlog_before_settle": max(
|
||||||
|
(int(e.get("result_backlog", 0)) for e in initial_scheduler_frames), default=0),
|
||||||
|
"last_admission_t_s": max((e.get("t_s", 0.0) for e in near_requests), default=None),
|
||||||
|
"near_tile_events": len(initial_tiles),
|
||||||
|
"near_worker_seconds": round(sum(e.get("worker_task_s", 0.0) for e in initial_tiles), 4),
|
||||||
|
"near_core_seconds": round(sum(e.get("core_generate_s", 0.0) for e in initial_tiles), 4),
|
||||||
|
"collision_ready_by_settle": collision_count,
|
||||||
|
"burst_frame_count": len(near_intervals),
|
||||||
|
"burst_frame_ms_p95": round(sorted(e.get("frame_ms", 0.0) for e in near_intervals)[
|
||||||
|
min(len(near_intervals) - 1, int(.95 * (len(near_intervals) - 1)))], 3)
|
||||||
|
if near_intervals else None,
|
||||||
|
"burst_frame_ms_max": round(max((e.get("frame_ms", 0.0) for e in near_intervals), default=0.0), 3),
|
||||||
|
"burst_full_world_tick_ms_p99": round(commandlet_stat("world_tick_ms", .99) or 0.0, 3),
|
||||||
|
"burst_full_world_tick_ms_max": round(commandlet_stat("world_tick_ms") or 0.0, 3),
|
||||||
|
"burst_world_pre_actor_ms_p99": round(commandlet_stat("world_pre_actor_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_pre_actor_ms_max": round(commandlet_stat("world_pre_actor_ms") or 0.0, 3),
|
||||||
|
"burst_world_actor_phase_ms_p99": round(commandlet_stat("world_actor_phase_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_actor_phase_ms_max": round(commandlet_stat("world_actor_phase_ms") or 0.0, 3),
|
||||||
|
"burst_world_post_actor_ms_p99": round(commandlet_stat("world_post_actor_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_post_actor_ms_max": round(commandlet_stat("world_post_actor_ms") or 0.0, 3),
|
||||||
|
"burst_commandlet_readiness_check_ms_max": round(
|
||||||
|
commandlet_stat("readiness_check_ms") or 0.0, 3),
|
||||||
|
"burst_commandlet_sleep_ms_p99": round(commandlet_stat("sleep_ms", .99) or 0.0, 3),
|
||||||
|
"burst_commandlet_sleep_ms_max": round(commandlet_stat("sleep_ms") or 0.0, 3),
|
||||||
|
"burst_commandlet_loop_ms_max": round(commandlet_stat("loop_ms") or 0.0, 3),
|
||||||
|
"burst_gc_count": len(garbage_collect_values),
|
||||||
|
"burst_gc_duration_ms_max": round(max(
|
||||||
|
(float(e.get("duration_ms", 0.0)) for e in garbage_collect_values), default=0.0), 3),
|
||||||
|
"burst_max_meshes_per_frame": max((e.get("meshes_applied", 0) for e in apply_values), default=0),
|
||||||
|
"burst_max_apply_ms": round(max((e.get("apply_ms", 0.0) for e in apply_values), default=0.0), 3),
|
||||||
|
"burst_max_admit_per_frame": max((sum(e.get("submitted_by_lod", [0] * 9)[:3])
|
||||||
|
for e in scheduler_values), default=0),
|
||||||
|
"burst_frame_timing_count": len(work_values),
|
||||||
|
"burst_active_generation_workers_p95": round(work_stat("active_generation_workers", .95) or 0.0, 3),
|
||||||
|
"burst_active_generation_workers_max": round(work_stat("active_generation_workers") or 0.0, 3),
|
||||||
|
"burst_active_tile_tasks_max": round(work_stat("active_tile_tasks") or 0.0, 3),
|
||||||
|
"burst_world_tick_ms_p99": round(work_stat("tick_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_tick_ms_max": round(work_stat("tick_ms") or 0.0, 3),
|
||||||
|
"burst_update_chunks_ms_p99": round(work_stat("update_chunks_ms", .99) or 0.0, 3),
|
||||||
|
"burst_update_chunks_ms_max": round(work_stat("update_chunks_ms") or 0.0, 3),
|
||||||
|
"burst_pending_drain_ms_p99": round(work_stat("pending_drain_ms", .99) or 0.0, 3),
|
||||||
|
"burst_pending_drain_ms_max": round(work_stat("pending_drain_ms") or 0.0, 3),
|
||||||
|
"burst_post_apply_callbacks_ms_p99": round(work_stat("post_apply_callbacks_ms", .99) or 0.0, 3),
|
||||||
|
"burst_post_apply_callbacks_ms_max": round(work_stat("post_apply_callbacks_ms") or 0.0, 3),
|
||||||
|
"burst_visibility_reconcile_ms_p99": round(work_stat("visibility_reconcile_ms", .99) or 0.0, 3),
|
||||||
|
"burst_visibility_reconcile_ms_max": round(work_stat("visibility_reconcile_ms") or 0.0, 3),
|
||||||
|
"burst_collision_probe_ms_max": round(work_stat("collision_probe_ms") or 0.0, 3),
|
||||||
|
"burst_visible_apply_probe_ms_max": round(work_stat("visible_apply_probe_ms") or 0.0, 3),
|
||||||
|
"burst_lod0_ready_ms_max": round(work_stat("lod0_ready_ms") or 0.0, 3),
|
||||||
|
"burst_steady_trace_ms_max": round(work_stat("steady_trace_ms") or 0.0, 3),
|
||||||
|
"burst_modification_ms_max": round(work_stat("modification_ms") or 0.0, 3),
|
||||||
|
"burst_lifecycle_audit_ms_p99": round(work_stat("lifecycle_audit_ms", .99) or 0.0, 3),
|
||||||
|
"burst_lifecycle_audit_ms_max": round(work_stat("lifecycle_audit_ms") or 0.0, 3),
|
||||||
|
"burst_collision_gate_ms_p99": round(work_stat("collision_gate_ms", .99) or 0.0, 3),
|
||||||
|
"burst_collision_gate_ms_max": round(work_stat("collision_gate_ms") or 0.0, 3),
|
||||||
|
"burst_unload_ms_p99": round(work_stat("unload_ms", .99) or 0.0, 3),
|
||||||
|
"burst_unload_ms_max": round(work_stat("unload_ms") or 0.0, 3),
|
||||||
|
"burst_trace_json_ms_p99": round(work_stat("trace_json_ms", .99) or 0.0, 3),
|
||||||
|
"burst_trace_json_ms_max": round(work_stat("trace_json_ms") or 0.0, 3),
|
||||||
|
"burst_desired_rebuild_ms_max": round(scheduler_stat("desired_rebuild_ms") or 0.0, 3),
|
||||||
|
"burst_rmc_create_group_ms_p99": round(rmc_stat("create_section_group_ms", .99) or 0.0, 3),
|
||||||
|
"burst_rmc_create_group_ms_max": round(rmc_stat("create_section_group_ms") or 0.0, 3),
|
||||||
|
"burst_rmc_collision_handoff_ms_max": round(rmc_stat("collision_handoff_ms") or 0.0, 3),
|
||||||
|
"burst_collision_completion_ms_max": round(max(
|
||||||
|
collision_completion_by_frame.values(), default=0.0), 3),
|
||||||
|
"worst_frame_breakdown": {
|
||||||
|
"frame": worst_work_frame_id,
|
||||||
|
"world_tick_frame": worst_work_frame_id,
|
||||||
|
"frame_interval_end_frame": worst_interval_frame_id,
|
||||||
|
"frame_interval_ms": round(float(worst_interval.get("frame_ms", 0.0)), 3)
|
||||||
|
if worst_interval else None,
|
||||||
|
"apply_ms": round(float(worst_apply.get("apply_ms", 0.0)), 3)
|
||||||
|
if worst_apply else None,
|
||||||
|
"result_drain_ms": round(float(worst_apply.get("drain_ms", 0.0)), 3)
|
||||||
|
if worst_apply else None,
|
||||||
|
"outside_world_tick_ms": round(max(0.0, float(worst_interval.get("frame_ms", 0.0))
|
||||||
|
- float(worst_work.get("tick_ms", 0.0))), 3)
|
||||||
|
if worst_interval else None,
|
||||||
|
"outside_full_world_tick_ms": round(max(0.0, float(worst_interval.get("frame_ms", 0.0))
|
||||||
|
- float(commandlet_tick_by_frame.get(worst_work_frame_id, {}).get("world_tick_ms", 0.0))), 3)
|
||||||
|
if worst_interval else None,
|
||||||
|
"full_world_tick_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_tick_ms", 0.0)), 3),
|
||||||
|
"world_pre_actor_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_pre_actor_ms", 0.0)), 3),
|
||||||
|
"world_actor_phase_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_actor_phase_ms", 0.0)), 3),
|
||||||
|
"world_post_actor_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_post_actor_ms", 0.0)), 3),
|
||||||
|
"commandlet_readiness_check_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("readiness_check_ms", 0.0)), 3),
|
||||||
|
"commandlet_sleep_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("sleep_ms", 0.0)), 3),
|
||||||
|
"commandlet_loop_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("loop_ms", 0.0)), 3),
|
||||||
|
"gc_duration_ms": round(float(garbage_collect_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("duration_ms", 0.0)), 3),
|
||||||
|
"world_tick_ms": round(float(worst_work.get("tick_ms", 0.0)), 3),
|
||||||
|
"active_generation_workers": int(worst_work.get("active_generation_workers", 0)),
|
||||||
|
"active_tile_tasks": int(worst_work.get("active_tile_tasks", 0)),
|
||||||
|
"collision_gate_ms": round(float(worst_work.get("collision_gate_ms", 0.0)), 3),
|
||||||
|
"update_chunks_ms": round(float(worst_work.get("update_chunks_ms", 0.0)), 3),
|
||||||
|
"desired_rebuild_ms": round(float(worst_scheduler.get("desired_rebuild_ms", 0.0)), 3),
|
||||||
|
"pending_drain_ms": round(float(worst_work.get("pending_drain_ms", 0.0)), 3),
|
||||||
|
"post_apply_callbacks_ms": round(float(worst_work.get("post_apply_callbacks_ms", 0.0)), 3),
|
||||||
|
"collision_probe_ms": round(float(worst_work.get("collision_probe_ms", 0.0)), 3),
|
||||||
|
"visibility_reconcile_ms": round(float(worst_work.get("visibility_reconcile_ms", 0.0)), 3),
|
||||||
|
"visible_apply_probe_ms": round(float(worst_work.get("visible_apply_probe_ms", 0.0)), 3),
|
||||||
|
"lod0_ready_ms": round(float(worst_work.get("lod0_ready_ms", 0.0)), 3),
|
||||||
|
"unload_ms": round(float(worst_work.get("unload_ms", 0.0)), 3),
|
||||||
|
"steady_trace_ms": round(float(worst_work.get("steady_trace_ms", 0.0)), 3),
|
||||||
|
"modification_ms": round(float(worst_work.get("modification_ms", 0.0)), 3),
|
||||||
|
"lifecycle_audit_ms": round(float(worst_work.get("lifecycle_audit_ms", 0.0)), 3),
|
||||||
|
"trace_json_ms": round(float(worst_work.get("trace_json_ms", 0.0)), 3),
|
||||||
|
"trace_archive_serialize_ms": round(float(
|
||||||
|
worst_work.get("trace_archive_serialize_ms", 0.0)), 3),
|
||||||
|
"collision_completion_ms": round(worst_collision_completion_ms, 3),
|
||||||
|
**{key: round(float(worst_rmc.get(key, 0.0)), 3)
|
||||||
|
for key in ("component_acquire_ms", "create_section_group_ms",
|
||||||
|
"create_sections_ms", "collision_handoff_ms")},
|
||||||
|
},
|
||||||
|
"trace": str(trace.relative_to(ROOT)), "utrace": str(utrace.relative_to(ROOT)),
|
||||||
|
"abslog": str(abslog.relative_to(ROOT)), "stdout": str(stdout.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
stdout_text = stdout.read_text(encoding="utf-8", errors="replace")
|
||||||
|
gap_ms, gap_frame = endphysics_gap(stdout)
|
||||||
|
row["endphysics_to_postphysics_gap_ms_max"] = gap_ms
|
||||||
|
row["endphysics_gap_frame"] = gap_frame
|
||||||
|
failure = re.search(r"\[VoxelForgeCoreWorldTest\] failed: ([^\r\n]+)", stdout_text)
|
||||||
|
if failure:
|
||||||
|
row["world_test_failure"] = failure.group(1)
|
||||||
|
with (OUT / "runs.jsonl").open("a", encoding="utf-8") as stream:
|
||||||
|
stream.write(json.dumps(row, sort_keys=True) + "\n")
|
||||||
|
(OUT / f"{label}.summary.json").write_text(json.dumps(row, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
print("[L105UnrealArrival] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
if exit_code:
|
||||||
|
raise SystemExit(f"Unreal commandlet failed: {stdout}; exit={exit_code}")
|
||||||
|
if not full or len(arrival) < 8:
|
||||||
|
raise SystemExit(f"Unreal arrival trace incomplete: {stdout}; exit={exit_code}")
|
||||||
|
if insights and (not utrace.is_file() or utrace.stat().st_size < 1024):
|
||||||
|
raise SystemExit(f"Unreal Insights trace missing or empty: {utrace}")
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--stratum", choices=("FI", "SW"), required=True)
|
||||||
|
parser.add_argument("--repeat", type=int, default=1)
|
||||||
|
parser.add_argument("--phase", choices=("probe", "before", "after", "snapshot"), default="probe")
|
||||||
|
parser.add_argument("--ready-depth", type=int, default=64)
|
||||||
|
parser.add_argument("--background-cap", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--worker-reserve", type=int, choices=(0, 1, 2, 3), default=0)
|
||||||
|
parser.add_argument("--helper-background", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--tile-background-low", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--tile-background-normal", type=int, choices=(0, 1), default=0)
|
||||||
|
parser.add_argument("--startup-boost", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--tick-logs", action="store_true")
|
||||||
|
parser.add_argument("--insights", action="store_true")
|
||||||
|
args = parser.parse_args()
|
||||||
|
if not EDITOR.is_file():
|
||||||
|
raise SystemExit(f"missing Unreal commandlet executable: {EDITOR}")
|
||||||
|
run(args.stratum, args.repeat, args.phase, args.ready_depth,
|
||||||
|
args.background_cap, args.worker_reserve, args.helper_background,
|
||||||
|
args.tile_background_low, args.tile_background_normal, args.startup_boost,
|
||||||
|
args.tick_logs, args.insights)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,103 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Five interleaved FI arrival pairs, comparing 8 and 10 effective tile slots."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
RUNNER_PATH = Path(__file__).with_name("RunL120Arrival.py")
|
||||||
|
SPEC = importlib.util.spec_from_file_location("l120_arrival", RUNNER_PATH)
|
||||||
|
assert SPEC and SPEC.loader
|
||||||
|
ARRIVAL = importlib.util.module_from_spec(SPEC)
|
||||||
|
SPEC.loader.exec_module(ARRIVAL)
|
||||||
|
|
||||||
|
|
||||||
|
def add_view_metrics(row: dict) -> dict:
|
||||||
|
trace_path = ROOT / row["trace"]
|
||||||
|
events = ARRIVAL.read_events(trace_path)
|
||||||
|
config = next((e for e in events if e.get("type") == "worker_backend_config"), {})
|
||||||
|
row["effective_tile_slots"] = config.get("effective_tile_slots")
|
||||||
|
row["near_priority"] = config.get("near_priority")
|
||||||
|
row["horizon_priority"] = config.get("horizon_priority")
|
||||||
|
samples = [e for e in events if e.get("type") == "streaming_sample"
|
||||||
|
and float(e.get("t_s", 0.0)) < ARRIVAL.MOVE_START_S
|
||||||
|
and float(e.get("speed_cm_s", 0.0)) < 50.0]
|
||||||
|
|
||||||
|
def total(values: list[int]) -> int:
|
||||||
|
return sum(values)
|
||||||
|
|
||||||
|
def near_complete(event: dict) -> bool:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
settled = event.get("settled_by_lod", [0] * 9)
|
||||||
|
return total(desired[:3]) > 0 and total(settled[:3]) >= total(desired[:3])
|
||||||
|
|
||||||
|
def whole_complete(event: dict) -> bool:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
settled = event.get("settled_by_lod", [0] * 9)
|
||||||
|
return total(desired) > 0 and total(settled) >= total(desired)
|
||||||
|
|
||||||
|
def far_holes(event: dict) -> float | None:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)[3:]
|
||||||
|
covered = event.get("covered_by_lod", [0] * 9)[3:]
|
||||||
|
denominator = total(desired)
|
||||||
|
return (100.0 * (denominator - total(covered)) / denominator
|
||||||
|
if denominator else None)
|
||||||
|
|
||||||
|
first_near = next((e for e in samples if near_complete(e)), None)
|
||||||
|
first_whole = next((e for e in samples if whole_complete(e)), None)
|
||||||
|
first_far_clear = next((e for e in samples if total(e.get("desired_by_lod", [0] * 9)[3:])
|
||||||
|
and far_holes(e) == 0.0), None)
|
||||||
|
last_sample = samples[-1] if samples else None
|
||||||
|
row["lod02_settled_t_s"] = float(first_near["t_s"]) if first_near else None
|
||||||
|
row["whole_view_settled_t_s"] = float(first_whole["t_s"]) if first_whole else None
|
||||||
|
row["far_clear_t_s"] = float(first_far_clear["t_s"]) if first_far_clear else None
|
||||||
|
row["far_holes_at_20s_pct"] = far_holes(last_sample) if last_sample else None
|
||||||
|
row["arrival_sample_count"] = len(samples)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def run_condition(condition: str, repetition: int, ready_depth: int) -> dict:
|
||||||
|
cap = 1 if condition == "baseline" else 0
|
||||||
|
expected_slots = 8 if condition == "baseline" else 10
|
||||||
|
row = ARRIVAL.run(
|
||||||
|
"FI", repetition, condition, ready_depth, cap, 0, 1, 1, 0, 1, False, False)
|
||||||
|
row = add_view_metrics(row)
|
||||||
|
if row["effective_tile_slots"] != expected_slots:
|
||||||
|
raise RuntimeError(f"worker slot readback mismatch: {row}")
|
||||||
|
if row["streaming_tile_priority_split"] != 0:
|
||||||
|
raise RuntimeError(f"priority condition mismatch: {row}")
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--ready-depth", type=int, choices=(8, 64), default=8)
|
||||||
|
ready_depth = parser.parse_args().ready_depth
|
||||||
|
OUT = ROOT / "Saved/Linux/l120/arrival"
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
results = []
|
||||||
|
for condition in ("baseline", "slots10"):
|
||||||
|
warmup = run_condition(condition, 0, ready_depth)
|
||||||
|
warmup["warmup"] = True
|
||||||
|
results.append(warmup)
|
||||||
|
|
||||||
|
for repetition in range(1, 6):
|
||||||
|
order = ("baseline", "slots10") if repetition % 2 else ("slots10", "baseline")
|
||||||
|
for condition in order:
|
||||||
|
row = run_condition(condition, repetition, ready_depth)
|
||||||
|
row["warmup"] = False
|
||||||
|
results.append(row)
|
||||||
|
print("[L120ArrivalPair] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
|
||||||
|
(OUT / f"pairs-rd{ready_depth}.json").write_text(json.dumps({
|
||||||
|
"design": f"FI arrival, 8 versus 10 effective tile slots, priority split off, ready depth {ready_depth}",
|
||||||
|
"runs": results,
|
||||||
|
}, indent=2, sort_keys=True) + "\n", encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,117 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Compare L120 exact tile hashes with L104 across all strata, compilers and workers."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
OUT = ROOT / "Saved/Linux/l120/hashes"
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
OLD = ROOT / "Saved/Linux/l104/hashes"
|
||||||
|
BINARIES = {
|
||||||
|
"clang": ROOT / "Saved/Linux/l120/bin/VoxelForgeCoreHeadless-l120-clang",
|
||||||
|
"gcc": ROOT / "Saved/Linux/l120/bin/VoxelForgeCoreHeadless-l120-gcc",
|
||||||
|
}
|
||||||
|
CAMERAS = {
|
||||||
|
"sw": ((8.0, 16.0, -372.2626953125), 2850),
|
||||||
|
"fi": ((25.8369140625, 3.19921875, -737.0), 2249),
|
||||||
|
"tn": ((19.166015625, -14.2001953125, -753.3740234375), 2233),
|
||||||
|
"vs": ((-20.0625, -5.068359375, -900.353515625), 2329),
|
||||||
|
}
|
||||||
|
TILE = re.compile(
|
||||||
|
r"^\[PipelineTileHash\] lod=(-?\d+) x=(-?\d+) y=(-?\d+) z=(-?\d+) "
|
||||||
|
r"density=(0x[0-9a-f]+) mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) "
|
||||||
|
r"topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
WHOLE = re.compile(
|
||||||
|
r"^\[PipelineWholeViewHash\] tiles=(\d+) density=(0x[0-9a-f]+) "
|
||||||
|
r"mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M)
|
||||||
|
|
||||||
|
|
||||||
|
def parse(path: Path) -> tuple[dict, tuple[str, ...], str]:
|
||||||
|
content = path.read_text(encoding="utf-8", errors="replace")
|
||||||
|
tiles = {(int(m.group(1)), int(m.group(2)), int(m.group(3)), int(m.group(4))):
|
||||||
|
m.group(5, 6, 7, 8) for m in TILE.finditer(content)}
|
||||||
|
whole, stamp = WHOLE.search(content), STAMP.search(content)
|
||||||
|
if whole is None or stamp is None or len(tiles) != int(whole.group(1)):
|
||||||
|
raise RuntimeError(f"incomplete hash output: {path}")
|
||||||
|
return tiles, whole.groups(), stamp.group(1)
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, compiler: str, workers: int) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = OUT / f"{stratum}-{compiler}-w{workers}.log"
|
||||||
|
camera, expected_tiles = CAMERAS[stratum]
|
||||||
|
camera_arg = ",".join(f"{v:.12f}" for v in camera) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--", "timeout", "1740s",
|
||||||
|
str(BINARIES[compiler]), str(DATA), "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", f"--workers={workers}",
|
||||||
|
"--clip-radius=3", "--max-clip-level=8", "--max-level=8",
|
||||||
|
f"--camera={camera_arg}", "--stream-hash-tiles",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
if process.returncode:
|
||||||
|
raise RuntimeError(f"hash process exited {process.returncode}: {log}")
|
||||||
|
tiles, whole, stamp = parse(log)
|
||||||
|
if len(tiles) != expected_tiles:
|
||||||
|
raise RuntimeError(f"expected {expected_tiles} tiles, got {len(tiles)}: {log}")
|
||||||
|
old_log = OLD / f"{stratum}-{compiler}-w{workers}.log"
|
||||||
|
old_tiles, old_whole, _old_stamp = parse(old_log)
|
||||||
|
clang_reference, clang_whole, _ = parse(OLD / f"{stratum}-clang-w1.log")
|
||||||
|
|
||||||
|
def difference(reference: dict, candidate: dict) -> tuple[int, int]:
|
||||||
|
keys = set(reference) | set(candidate)
|
||||||
|
mismatches = sum(reference.get(key) != candidate.get(key) for key in keys)
|
||||||
|
return mismatches, len(keys)
|
||||||
|
|
||||||
|
old_mismatches, old_keys = difference(old_tiles, tiles)
|
||||||
|
cross_mismatches, cross_keys = difference(clang_reference, tiles)
|
||||||
|
row = {
|
||||||
|
"stratum": stratum, "compiler": compiler, "workers": workers,
|
||||||
|
"source_stamp": stamp, "tiles": len(tiles),
|
||||||
|
"l104_same_cell_tile_mismatches": old_mismatches,
|
||||||
|
"l104_same_cell_key_count": old_keys,
|
||||||
|
"l104_same_cell_whole_hash_match": whole == old_whole,
|
||||||
|
"clang_w1_tile_mismatches": cross_mismatches,
|
||||||
|
"clang_w1_key_count": cross_keys,
|
||||||
|
"clang_w1_whole_hash_match": whole == clang_whole,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L105HashRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
row["tile_hashes"] = {"|".join(map(str, key)): value for key, value in tiles.items()}
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
for path in (*BINARIES.values(), DATA):
|
||||||
|
if not path.is_file():
|
||||||
|
raise SystemExit(f"missing input: {path}")
|
||||||
|
rows = []
|
||||||
|
for stratum in CAMERAS:
|
||||||
|
for compiler in BINARIES:
|
||||||
|
for workers in (1, 8, 12):
|
||||||
|
rows.append(run(stratum, compiler, workers))
|
||||||
|
(OUT / "runs-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
failures = [row for row in rows if row["l104_same_cell_tile_mismatches"]
|
||||||
|
or not row["l104_same_cell_whole_hash_match"]
|
||||||
|
or row["clang_w1_tile_mismatches"]
|
||||||
|
or not row["clang_w1_whole_hash_match"]]
|
||||||
|
summary = {"runs": [{key: value for key, value in row.items() if key != "tile_hashes"}
|
||||||
|
for row in rows], "failed_runs": len(failures)}
|
||||||
|
(OUT / "summary.json").write_text(json.dumps(summary, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 1 if failures else 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,292 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Five-run interleaved L120 priority-split comparison at 30 m/s."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import re
|
||||||
|
import statistics
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
HOST = Path("/home/jahni/VFHost")
|
||||||
|
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
HEADLESS = ROOT / "Saved/Linux/l108/bin/VoxelForgeCoreHeadless-l108-after-clang"
|
||||||
|
OUT = ROOT / "Saved/Linux/l120/movement/priority-split-verified"
|
||||||
|
HOST_SAVED = HOST / "Saved/Linux/l120"
|
||||||
|
HOST_OUT = HOST / "Plugins/VoxelForge/Saved/Linux/l120/priority-split-verified"
|
||||||
|
ACTIVE_DESIGN = "priority_split"
|
||||||
|
SPAWN_M = (19.166015625, -14.2001953125, -753.3740234375)
|
||||||
|
MOVING_SAMPLE = re.compile(r"^\[L79MovingSample\] (.*)$", re.M)
|
||||||
|
MOVING_SUMMARY = re.compile(r"^\[L79MovingSummary\] (.*)$", re.M)
|
||||||
|
FIELDS = re.compile(r"([a-zA-Z0-9_]+)=([^ ]+)")
|
||||||
|
|
||||||
|
|
||||||
|
def p99(values: list[float]) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
values.sort()
|
||||||
|
return values[max(0, math.ceil(0.99 * len(values)) - 1)]
|
||||||
|
|
||||||
|
|
||||||
|
def parse_events(path: Path) -> list[dict]:
|
||||||
|
events = []
|
||||||
|
if not path.is_file():
|
||||||
|
return events
|
||||||
|
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||||
|
if not line.startswith("{"):
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
events.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
return events
|
||||||
|
|
||||||
|
|
||||||
|
def summarize(rows: list[dict], phase: str) -> dict:
|
||||||
|
moving = [row for row in rows if row["phase"] == "move"]
|
||||||
|
dwell = [row for row in rows if row["phase"] == "dwell"]
|
||||||
|
def moving_mean(name: str) -> float | None:
|
||||||
|
values = [row[name] for row in moving if name in row]
|
||||||
|
return statistics.mean(values) if values else None
|
||||||
|
complete = next((row for row in dwell
|
||||||
|
if row["near_ready_pct"] >= 99.999
|
||||||
|
and row["far_hole_pct"] <= 0.001), None)
|
||||||
|
end = dwell[-1] if dwell else {}
|
||||||
|
return {
|
||||||
|
"moving_samples": len(moving),
|
||||||
|
"dwell_samples": len(dwell),
|
||||||
|
"near_ready_moving_mean_pct": moving_mean("near_ready_pct"),
|
||||||
|
"lod0_ready_moving_mean_pct": moving_mean("lod0_ready_pct"),
|
||||||
|
"lod1_ready_moving_mean_pct": moving_mean("lod1_ready_pct"),
|
||||||
|
"lod2_ready_moving_mean_pct": moving_mean("lod2_ready_pct"),
|
||||||
|
"far_holes_moving_mean_pct": moving_mean("far_hole_pct"),
|
||||||
|
"near_ready_dwell_end_pct": end.get("near_ready_pct"),
|
||||||
|
"far_holes_dwell_end_pct": end.get("far_hole_pct"),
|
||||||
|
"dwell_complete_after_stop_s": complete.get("since_stop_s") if complete else None,
|
||||||
|
"phase": phase,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def run_headless(condition: str, repetition: int) -> dict:
|
||||||
|
worker_count = 8 if condition == "baseline" else 10
|
||||||
|
folder = OUT / condition / "headless"
|
||||||
|
folder.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = folder / f"fast-r{repetition}.log"
|
||||||
|
camera = ",".join(f"{value:.12f}" for value in SPAWN_M) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", str(HEADLESS), str(DATA),
|
||||||
|
"--world-assembler-prototype", "--stream-benchmark", "--stream-pipeline",
|
||||||
|
f"--workers={worker_count}", "--clip-radius=3", "--max-clip-level=8",
|
||||||
|
"--max-level=8", f"--camera={camera}", "--profile-lod-work",
|
||||||
|
"--stream-scheduler-policy=3", "--moving-camera-speed-mps=30.0",
|
||||||
|
"--moving-camera-duration-s=12", "--moving-camera-dwell-s=5",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
result = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
content = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
rows = []
|
||||||
|
for match in MOVING_SAMPLE.finditer(content):
|
||||||
|
row = dict(FIELDS.findall(match.group(1)))
|
||||||
|
for name in ("t_s", "near_ready_pct", "walkable_ring_ready_pct",
|
||||||
|
"far_hole_pct", "queue_update_ms"):
|
||||||
|
if name in row:
|
||||||
|
row[name] = float(row[name])
|
||||||
|
row["phase"] = row.get("phase", "move")
|
||||||
|
row["since_stop_s"] = max(0.0, row.get("t_s", 0.0) - 12.0)
|
||||||
|
rows.append(row)
|
||||||
|
engine_summary = {}
|
||||||
|
if match := MOVING_SUMMARY.search(content):
|
||||||
|
engine_summary = dict(FIELDS.findall(match.group(1)))
|
||||||
|
if result.returncode or not rows or int(engine_summary.get("generation_failures", "-1")) != 0:
|
||||||
|
raise RuntimeError(f"headless run failed: {log}; exit={result.returncode}")
|
||||||
|
moving_updates = [row["queue_update_ms"] for row in rows
|
||||||
|
if row["phase"] == "move" and "queue_update_ms" in row]
|
||||||
|
return {
|
||||||
|
"condition": condition, "platform": "headless", "speed_mps": 30.0,
|
||||||
|
"repetition": repetition, "worker_count": worker_count,
|
||||||
|
"exit_code": result.returncode,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"update_work_p99_ms": p99(moving_updates),
|
||||||
|
"update_work_max_ms": max(moving_updates, default=None),
|
||||||
|
"log": str(log.relative_to(ROOT)), **summarize(rows, "headless"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def run_unreal(condition: str, repetition: int) -> dict:
|
||||||
|
core_limit = 12
|
||||||
|
if ACTIVE_DESIGN == "priority_split":
|
||||||
|
worker_cap = 1
|
||||||
|
priority_split = 0 if condition == "baseline" else 1
|
||||||
|
else:
|
||||||
|
worker_cap = 1 if condition == "baseline" else 0
|
||||||
|
priority_split = 0
|
||||||
|
folder = OUT / condition / "unreal"
|
||||||
|
host_folder = HOST_OUT / condition / "output"
|
||||||
|
folder.mkdir(parents=True, exist_ok=True)
|
||||||
|
trace = folder / f"fast-r{repetition}.trace.jsonl"
|
||||||
|
abslog = folder / f"fast-r{repetition}.log"
|
||||||
|
stdout = folder / f"fast-r{repetition}.stdout.log"
|
||||||
|
export = host_folder / f"fast-r{repetition}"
|
||||||
|
userdir = HOST_SAVED / "user" / condition / f"fast-r{repetition}"
|
||||||
|
zendata = HOST_SAVED / "zen" / condition / f"fast-r{repetition}"
|
||||||
|
ddc = HOST_SAVED / "DDC"
|
||||||
|
for path in (export, userdir, zendata, ddc):
|
||||||
|
path.mkdir(parents=True, exist_ok=True)
|
||||||
|
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWN_M)
|
||||||
|
command = [
|
||||||
|
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
|
||||||
|
"-nop4", "-nosplash", "-NoSound", f"-corelimit={core_limit}",
|
||||||
|
"-NoZenAutoLaunch", "-notraceserver", "-nocrashreports",
|
||||||
|
"-DisablePlugins=AndroidFileServer", "-ddc=NoZenLocalFallback",
|
||||||
|
f"-userdir={userdir}", f"-ZenDataPath={zendata}",
|
||||||
|
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
|
||||||
|
"-archetype=TunnelNetwork", "-slot=0", "-modes=export", "-gameconfig=1",
|
||||||
|
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
|
||||||
|
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0",
|
||||||
|
"-coreplantest=0", "-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings",
|
||||||
|
"-corehybridmesher=0", f"-out={export}",
|
||||||
|
"-voxel.StreamingCoarseFirst=1", "-voxel.StreamingDynamicNearReserve=1",
|
||||||
|
"-voxel.StreamingLegacyPendingResultCap=0", "-voxel.CoreRegionConcurrentAdmission=1",
|
||||||
|
"-voxel.StreamingResultBacklogCap=256", "-voxel.StreamingReadyQueueDepth=8",
|
||||||
|
f"-voxel.StreamingBackgroundWorkerCap={worker_cap}",
|
||||||
|
"-voxel.CoreColumnCacheDirectMapped=0",
|
||||||
|
f"-voxel.StreamingTilePrioritySplit={priority_split}",
|
||||||
|
"-voxel.CoreMCGradientMemoization=1", "-voxel.StreamingCollisionLookahead=1",
|
||||||
|
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=3000.0",
|
||||||
|
"-voxel.TestMoveStartSeconds=6.0", "-voxel.TestMoveDurationSeconds=12.0",
|
||||||
|
"-voxel.TestExitSeconds=23.0", "-voxel.TestMoveDirection=1,0,0",
|
||||||
|
"-voxel.StreamingSchedulerPolicy=3", "-voxel.TestExitOnSteady=0",
|
||||||
|
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
|
||||||
|
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
|
||||||
|
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
|
||||||
|
f"-abslog={abslog}",
|
||||||
|
]
|
||||||
|
wrapped = ["taskset", "-c", "0-11", "timeout", "1740s", *command]
|
||||||
|
started = time.monotonic()
|
||||||
|
with stdout.open("w", encoding="utf-8") as stream:
|
||||||
|
result = subprocess.run(wrapped, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
events = parse_events(trace)
|
||||||
|
samples = [event for event in events if event.get("type") == "streaming_sample"]
|
||||||
|
moving_samples = [event for event in samples
|
||||||
|
if float(event.get("speed_cm_s", 0.0)) >= 50.0]
|
||||||
|
move_start = min((float(event.get("t_s", 0.0)) for event in moving_samples),
|
||||||
|
default=float("inf"))
|
||||||
|
stop_event = next((event for event in events
|
||||||
|
if event.get("type") == "streaming_test_move_stopped"), None)
|
||||||
|
stop = float(stop_event.get("t_s", 0.0)) if stop_event else float("inf")
|
||||||
|
rows = []
|
||||||
|
for event in samples:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
resident = event.get("resident_by_lod", [0] * 9)
|
||||||
|
covered = event.get("covered_by_lod", resident)
|
||||||
|
near_desired = sum(desired[:3])
|
||||||
|
near_resident = sum(resident[:3])
|
||||||
|
far_desired = sum(desired[3:])
|
||||||
|
far_covered = sum(covered[3:])
|
||||||
|
t_s = float(event.get("t_s", 0.0))
|
||||||
|
speed = float(event.get("speed_cm_s", 0.0))
|
||||||
|
phase = "move" if speed >= 50.0 else ("dwell" if t_s >= stop else "arrival")
|
||||||
|
rows.append({
|
||||||
|
"phase": phase, "t_s": t_s,
|
||||||
|
"since_stop_s": max(0.0, t_s - stop) if phase == "dwell" else None,
|
||||||
|
"near_ready_pct": 100.0 * near_resident / max(1, near_desired),
|
||||||
|
"lod0_ready_pct": 100.0 * resident[0] / max(1, desired[0]),
|
||||||
|
"lod1_ready_pct": 100.0 * resident[1] / max(1, desired[1]),
|
||||||
|
"lod2_ready_pct": 100.0 * resident[2] / max(1, desired[2]),
|
||||||
|
"far_hole_pct": 100.0 * (far_desired - far_covered) / max(1, far_desired),
|
||||||
|
})
|
||||||
|
dwell_rows = [row for row in rows if row["phase"] == "dwell"]
|
||||||
|
complete = next((row for row in dwell_rows
|
||||||
|
if row["near_ready_pct"] >= 99.999
|
||||||
|
and row["far_hole_pct"] <= 0.001), None)
|
||||||
|
moving_work = [event for event in events
|
||||||
|
if event.get("type") == "game_thread_work_frame"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop]
|
||||||
|
ticks = [float(event.get("tick_ms", 0.0)) for event in moving_work]
|
||||||
|
updates = [float(event.get("update_chunks_ms", 0.0)) for event in moving_work]
|
||||||
|
backend = next((event for event in events
|
||||||
|
if event.get("type") == "worker_backend_config"), {})
|
||||||
|
worker_cap_readback = next((event for event in events
|
||||||
|
if event.get("type") == "streaming_worker_cap_readback"), {})
|
||||||
|
if result.returncode or len(moving_samples) < 40 or len(dwell_rows) < 16:
|
||||||
|
raise RuntimeError(f"Unreal run failed/incomplete: {trace}; exit={result.returncode}; "
|
||||||
|
f"move={len(moving_samples)} dwell={len(dwell_rows)}")
|
||||||
|
expected_split = (0 if condition == "baseline" else 1) \
|
||||||
|
if ACTIVE_DESIGN == "priority_split" else 0
|
||||||
|
expected_near_priority = "BackgroundLow" if expected_split == 0 else "BackgroundNormal"
|
||||||
|
if (int(backend.get("streaming_tile_priority_split", -1)) != expected_split
|
||||||
|
or backend.get("near_priority") != expected_near_priority
|
||||||
|
or backend.get("horizon_priority") != "BackgroundLow"
|
||||||
|
or int(worker_cap_readback.get("streaming_background_worker_cap", -1)) != worker_cap
|
||||||
|
or int(backend.get("effective_tile_slots", -1)) != (8 if worker_cap else 10)):
|
||||||
|
raise RuntimeError(f"priority CVar readback mismatch in {trace}: {backend}")
|
||||||
|
return {
|
||||||
|
"condition": condition, "platform": "unreal", "speed_mps": 30.0,
|
||||||
|
"repetition": repetition, "core_limit": core_limit,
|
||||||
|
"effective_tile_slots": backend.get("effective_tile_slots"),
|
||||||
|
"streaming_background_worker_cap": worker_cap_readback.get(
|
||||||
|
"streaming_background_worker_cap"),
|
||||||
|
"taskgraph_background_threads": backend.get("taskgraph_background_threads"),
|
||||||
|
"exit_code": result.returncode,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"game_thread_tick_p99_ms": p99(ticks),
|
||||||
|
"game_thread_tick_max_ms": max(ticks, default=None),
|
||||||
|
"stream_update_p99_ms": p99(updates),
|
||||||
|
"stream_update_max_ms": max(updates, default=None),
|
||||||
|
"dwell_complete_after_stop_s": complete.get("since_stop_s") if complete else None,
|
||||||
|
"moving_samples": len(moving_samples), "dwell_samples": len(dwell_rows),
|
||||||
|
"trace": str(trace.relative_to(ROOT)),
|
||||||
|
"log": str(abslog.relative_to(ROOT)),
|
||||||
|
**summarize(rows, "unreal"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
global ACTIVE_DESIGN, OUT, HOST_SAVED, HOST_OUT
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--platform", choices=("all", "headless", "unreal"), default="all")
|
||||||
|
parser.add_argument("--design", choices=("priority_split", "worker_slots"),
|
||||||
|
default="priority_split")
|
||||||
|
args = parser.parse_args()
|
||||||
|
ACTIVE_DESIGN = args.design
|
||||||
|
design_path = "priority-split-verified" if ACTIVE_DESIGN == "priority_split" \
|
||||||
|
else "worker-slots-verified"
|
||||||
|
OUT = ROOT / "Saved/Linux/l120/movement" / design_path
|
||||||
|
HOST_SAVED = HOST / "Saved/Linux/l120" / design_path
|
||||||
|
HOST_OUT = HOST / "Plugins/VoxelForge/Saved/Linux/l120" / design_path
|
||||||
|
if not DATA.is_file() or not HEADLESS.is_file() or not EDITOR.is_file():
|
||||||
|
raise SystemExit("missing owner data, L108 headless binary, or Unreal editor")
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
results = []
|
||||||
|
partial = OUT / "runs-partial.json"
|
||||||
|
for repetition in range(1, 6):
|
||||||
|
candidate = "priority_split" if ACTIVE_DESIGN == "priority_split" else "slots10"
|
||||||
|
conditions = ("baseline", candidate) if repetition % 2 else (candidate, "baseline")
|
||||||
|
platforms = (("headless", "unreal") if repetition % 2 else ("unreal", "headless")) \
|
||||||
|
if args.platform == "all" else (args.platform,)
|
||||||
|
for condition in conditions:
|
||||||
|
for platform in platforms:
|
||||||
|
row = (run_headless(condition, repetition) if platform == "headless"
|
||||||
|
else run_unreal(condition, repetition))
|
||||||
|
results.append(row)
|
||||||
|
partial.write_text(json.dumps(results, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print("[L120Measurement] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
design_description = "near BGNormal vs horizon BackgroundLow" \
|
||||||
|
if ACTIVE_DESIGN == "priority_split" else "effective UE tile slots 8 vs 10"
|
||||||
|
(OUT / "summary.json").write_text(
|
||||||
|
json.dumps({"design": design_description,
|
||||||
|
"runs": results}, indent=2) + "\n", encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
# Linux 121 — finished far tiles
|
||||||
|
|
||||||
|
One change: after four normal visible applies, allow one far apply if less than half of the 2 ms budget is used. Disable the lane during startup boost. Geometry and worker limits stay unchanged.
|
||||||
|
|
||||||
|
Five interleaved UE pairs at 30 m/s (8 slots, ready depth 8) reduced median far coverage holes from 4.38% to 2.70%; near settled coverage rose from 45.12% to 45.43%. Far worker-finish-to-apply wait fell from 1.105 s / 3.25 s p50/p95 to 0.114 s / 2.40 s. The lane applied 672 extra far tiles across the five runs.
|
||||||
|
|
||||||
|
The worst tick was 14.74 ms, below L120's 15.43 ms ceiling. The worst per-run p99 was 11.00 ms versus 11.58 ms for baseline. Five 140 cm/s arrival pairs improved paired median LOD0–2 settlement by 20 ms and whole-view/far-clear by 313 ms. Both arms had zero far coverage holes at 20 s.
|
||||||
|
|
||||||
|
## Stage timings
|
||||||
|
|
||||||
|
Five paired UE traces per arm. Result wait is worker finish to `ApplyTileResult` (p50/p95 ms); generation, apply, and proxy are p95 ms. Proxy lag is reconstructed from callback and tile-apply timestamps because profile-only proxy-lag fields are zero when profiling is off.
|
||||||
|
|
||||||
|
| Speed / LOD | Far generation p95 | Result wait baseline → lane | Apply baseline → lane | Proxy baseline → lane |
|
||||||
|
|---|---:|---:|---:|---:|
|
||||||
|
| 1.4 m/s near | — | 12 / 43 → 9 / 37 | 0.362 → 0.340 | 0.411 → 0.406 |
|
||||||
|
| 1.4 m/s far | 5.6 → 6.3 | 209 / 229 → 142 / 209 | 0.314 → 0.252 | 0.221 → 0.196 |
|
||||||
|
| 30 m/s near | — | 12 / 125 → 12 / 120 | 0.206 → 0.197 | 0.250 → 0.237 |
|
||||||
|
| 30 m/s far | 10.0 → 10.3 | 1,105 / 3,250 → 114 / 2,397 | 0.351 → 0.324 | 0.335 → 0.312 |
|
||||||
|
|
||||||
|
The four-apply count limit was hit on 44% of baseline moving frames; the 2 ms time limit was hit on under 1%. Stale and obsolete results were discarded before apply and were under 1% of tile events.
|
||||||
|
|
||||||
|
Admission had idle-worker/queued-result windows while far coverage was missing: 504/2,135 baseline and 446/2,109 lane samples. The 256-result cap did not cause those windows: backlog p95/max was 101/206, with zero cap declines in them. Core-region compilation was pending in 64%/62% of those windows.
|
||||||
|
|
||||||
|
RMC reported `Updated` after proxy publication, but many replacement components were still hidden by the coverage handoff. NullRHI runs cannot measure the first presented pixel; the trace's far coverage is explicitly not visual enclosure.
|
||||||
|
|
||||||
|
UE build passed with `-NoUBA -MaxParallelActions=4`; fast tier passed (269 gates, 5 diagnostics). Clang/GCC W1/W8/W12 matched L104 across four strata: 24/24 exact tile and whole-view hashes. Headless has no UE apply budget, so the lane has no headless effect. Logs are under `Saved/Linux/l121`; timing used CPUs 0–11 with no build running.
|
||||||
@@ -0,0 +1,488 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Run one pinned null-RHI FI arrival for Linux 121."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from collections import Counter
|
||||||
|
from datetime import datetime
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
HOST = Path("/home/jahni/VFHost")
|
||||||
|
HOST_PLUGIN_SAVED = HOST / "Plugins/VoxelForge/Saved/Linux/l121"
|
||||||
|
HOST_SAVED = HOST / "Saved/Linux/l121"
|
||||||
|
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
|
||||||
|
OUT = ROOT / "Saved/Linux/l121/arrival"
|
||||||
|
SPAWNS_M = {
|
||||||
|
"FI": (25.8369140625, 3.19921875, -737.0),
|
||||||
|
"SW": (25.8369140625, 3.19921875, -200.0),
|
||||||
|
}
|
||||||
|
MOVE_START_S = 20.0
|
||||||
|
MOVE_DURATION_S = 12.0
|
||||||
|
EXIT_S = MOVE_START_S + MOVE_DURATION_S + 5.0
|
||||||
|
RUN_TIMEOUT_S = 1740
|
||||||
|
|
||||||
|
|
||||||
|
def read_events(path: Path) -> list[dict]:
|
||||||
|
events = []
|
||||||
|
if path.is_file():
|
||||||
|
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||||
|
if line.startswith("{"):
|
||||||
|
try:
|
||||||
|
events.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
return events
|
||||||
|
|
||||||
|
|
||||||
|
def percentile(values: list[float], fraction: float) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
ordered = sorted(values)
|
||||||
|
return ordered[min(len(ordered) - 1, int(round((len(ordered) - 1) * fraction)))]
|
||||||
|
|
||||||
|
|
||||||
|
def sample_unreal_peak_rss() -> tuple[int, int] | None:
|
||||||
|
executable = str(EDITOR).encode()
|
||||||
|
project = str(HOST / "VFHost.uproject").encode()
|
||||||
|
peak: tuple[int, int] | None = None
|
||||||
|
for process in Path("/proc").glob("[0-9]*"):
|
||||||
|
try:
|
||||||
|
command_line = (process / "cmdline").read_bytes()
|
||||||
|
if executable not in command_line or project not in command_line:
|
||||||
|
continue
|
||||||
|
status = (process / "status").read_text(encoding="ascii", errors="ignore")
|
||||||
|
match = re.search(r"^VmHWM:\s+(\d+)\s+kB$", status, re.MULTILINE)
|
||||||
|
if match:
|
||||||
|
rss_kib = int(match.group(1))
|
||||||
|
if peak is None or rss_kib > peak[0]:
|
||||||
|
peak = (rss_kib, int(process.name))
|
||||||
|
except (FileNotFoundError, PermissionError, ProcessLookupError, OSError):
|
||||||
|
continue
|
||||||
|
return peak
|
||||||
|
|
||||||
|
|
||||||
|
def endphysics_gap(stdout: Path) -> tuple[float | None, int | None]:
|
||||||
|
release_times: dict[int, dict[int, datetime]] = {}
|
||||||
|
pattern = re.compile(
|
||||||
|
r"^\[(\d{4}\.\d{2}\.\d{2}-\d{2}\.\d{2}\.\d{2}:\d{3})\].*"
|
||||||
|
r"LogTick: tick\s+(\d+)\s+-+\s+Release tick group ([34])")
|
||||||
|
with stdout.open(encoding="utf-8", errors="replace") as stream:
|
||||||
|
for line in stream:
|
||||||
|
match = pattern.match(line)
|
||||||
|
if not match:
|
||||||
|
continue
|
||||||
|
timestamp = datetime.strptime(match.group(1), "%Y.%m.%d-%H.%M.%S:%f")
|
||||||
|
frame = int(match.group(2))
|
||||||
|
group = int(match.group(3))
|
||||||
|
release_times.setdefault(frame, {})[group] = timestamp
|
||||||
|
gaps = [(round((times[4] - times[3]).total_seconds() * 1000.0, 3), frame)
|
||||||
|
for frame, times in release_times.items() if 3 in times and 4 in times]
|
||||||
|
return max(gaps, default=(None, None))
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, repeat: int, phase: str, ready_depth: int,
|
||||||
|
background_cap: int, worker_reserve: int, helper_background: int,
|
||||||
|
tile_background_low: int, tile_background_normal: int, startup_boost: int,
|
||||||
|
tick_logs: bool, insights: bool, far_apply_supplement: int) -> dict:
|
||||||
|
label = (f"{phase}-{stratum.lower()}-rd{ready_depth}-bg{background_cap}"
|
||||||
|
f"-reserve{worker_reserve}-helperbg{helper_background}"
|
||||||
|
f"-tilelow{tile_background_low}-tilenormal{tile_background_normal}"
|
||||||
|
f"-ticklog{int(tick_logs)}"
|
||||||
|
f"-farapply{far_apply_supplement}-boost{startup_boost}-{repeat}")
|
||||||
|
trace = OUT / f"{label}.trace.jsonl"
|
||||||
|
utrace = OUT / f"{label}.utrace"
|
||||||
|
abslog = OUT / f"{label}.log"
|
||||||
|
stdout = OUT / f"{label}.stdout.log"
|
||||||
|
output = HOST_PLUGIN_SAVED / "output" / label
|
||||||
|
userdir = HOST_SAVED / "user" / label
|
||||||
|
zendata = HOST_SAVED / "zen" / label
|
||||||
|
ddc = HOST_SAVED / "DDC"
|
||||||
|
for directory in (OUT, output, userdir, zendata, ddc):
|
||||||
|
directory.mkdir(parents=True, exist_ok=True)
|
||||||
|
trace.unlink(missing_ok=True)
|
||||||
|
utrace.unlink(missing_ok=True)
|
||||||
|
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWNS_M[stratum])
|
||||||
|
command = [
|
||||||
|
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
|
||||||
|
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
|
||||||
|
"-notraceserver", "-nocrashreports",
|
||||||
|
"-DisablePlugins=AndroidFileServer",
|
||||||
|
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
|
||||||
|
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
|
||||||
|
"-archetype=TunnelNetwork", f"-slot={1 if stratum == 'FI' else 0}",
|
||||||
|
"-modes=export", "-gameconfig=1", "-coreworldtest=1",
|
||||||
|
"-ownerfourstrateplan=1", "-corevoxelsizecm=25", "-coreislandtest=0",
|
||||||
|
"-corefallbacktest=0", "-coreofftest=0", "-coreplantest=0",
|
||||||
|
"-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings", "-corehybridmesher=0",
|
||||||
|
f"-out={output}", "-voxel.StreamingCoarseFirst=1",
|
||||||
|
"-voxel.StreamingDynamicNearReserve=1",
|
||||||
|
"-voxel.StreamingLegacyPendingResultCap=0",
|
||||||
|
"-voxel.CoreRegionConcurrentAdmission=1",
|
||||||
|
"-voxel.StreamingResultBacklogCap=256",
|
||||||
|
f"-voxel.StreamingReadyQueueDepth={ready_depth}",
|
||||||
|
f"-voxel.StreamingBackgroundWorkerCap={background_cap}",
|
||||||
|
f"-voxel.StreamingBackgroundWorkerReserve={worker_reserve}",
|
||||||
|
f"-voxel.CoreParallelForBackgroundPriority={helper_background}",
|
||||||
|
f"-voxel.StreamingTileBackgroundLow={tile_background_low}",
|
||||||
|
f"-voxel.StreamingTileBackgroundNormal={tile_background_normal}",
|
||||||
|
f"-voxel.TestLogTickFunctions={int(tick_logs)}",
|
||||||
|
"-voxel.CoreColumnCacheDirectMapped=0", "-voxel.CoreMCGradientMemoization=1",
|
||||||
|
"-voxel.StreamingCollisionLookahead=1",
|
||||||
|
f"-voxel.StreamingStartupApplyBoost={startup_boost}",
|
||||||
|
f"-voxel.StreamingFarApplySupplement={far_apply_supplement}",
|
||||||
|
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=140.0",
|
||||||
|
f"-voxel.TestMoveStartSeconds={MOVE_START_S:.1f}",
|
||||||
|
f"-voxel.TestMoveDurationSeconds={MOVE_DURATION_S:.1f}",
|
||||||
|
f"-voxel.TestExitSeconds={EXIT_S:.1f}", "-voxel.TestMoveDirection=1,0,0",
|
||||||
|
"-voxel.StreamingSchedulerPolicy=3", "-voxel.TestExitOnSteady=0",
|
||||||
|
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
|
||||||
|
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
|
||||||
|
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
|
||||||
|
f"-abslog={abslog}",
|
||||||
|
]
|
||||||
|
if insights:
|
||||||
|
command += ["-trace=cpu,task,frame", "-statnamedevents",
|
||||||
|
f"-tracefile={utrace}", "-tracefiletrunc"]
|
||||||
|
wrapped = ["taskset", "-c", "0-11", "timeout", f"{RUN_TIMEOUT_S}s", *command]
|
||||||
|
started = time.monotonic()
|
||||||
|
with stdout.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.Popen(wrapped, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT)
|
||||||
|
peak_rss = None
|
||||||
|
while process.poll() is None:
|
||||||
|
observed = sample_unreal_peak_rss()
|
||||||
|
if observed and (peak_rss is None or observed[0] > peak_rss[0]):
|
||||||
|
peak_rss = observed
|
||||||
|
time.sleep(0.25)
|
||||||
|
exit_code = process.wait()
|
||||||
|
observed = sample_unreal_peak_rss()
|
||||||
|
if observed and (peak_rss is None or observed[0] > peak_rss[0]):
|
||||||
|
peak_rss = observed
|
||||||
|
|
||||||
|
events = read_events(trace)
|
||||||
|
worker_backend_config = next(
|
||||||
|
(e for e in events if e.get("type") == "worker_backend_config"), {})
|
||||||
|
worker_cap_readback = next(
|
||||||
|
(e for e in events if e.get("type") == "streaming_worker_cap_readback"), {})
|
||||||
|
samples = [e for e in events if e.get("type") == "streaming_sample"]
|
||||||
|
arrival = [e for e in samples if float(e.get("t_s", 0.0)) < MOVE_START_S]
|
||||||
|
full = next((e for e in arrival
|
||||||
|
if sum(e.get("desired_by_lod", [0] * 9)[:3]) > 0
|
||||||
|
and sum(e.get("settled_by_lod", [0] * 9)[:3])
|
||||||
|
>= sum(e.get("desired_by_lod", [0] * 9)[:3])), None)
|
||||||
|
first_admission = next((e for e in events if e.get("type") == "scheduler_admission"
|
||||||
|
and e.get("desired_rebuilt") == 1), None)
|
||||||
|
request_events = [e for e in events if e.get("type") == "request"]
|
||||||
|
initial_request_events = [e for e in request_events if e.get("epoch") == 0]
|
||||||
|
tile_by_ordinal = {e.get("request_ordinal"): e for e in events
|
||||||
|
if e.get("type") == "tile"}
|
||||||
|
collision_count = sum(1 for e in events if e.get("type") == "collision_ready"
|
||||||
|
and e.get("level", 9) <= 2
|
||||||
|
and full and e.get("t_s", 1e30) <= full["t_s"])
|
||||||
|
initial_tiles = [tile_by_ordinal[r.get("request_ordinal")] for r in initial_request_events
|
||||||
|
if full and r.get("t_s", 1e30) <= full["t_s"]
|
||||||
|
and r.get("request_ordinal") in tile_by_ordinal
|
||||||
|
and tile_by_ordinal[r["request_ordinal"]].get("level", 9) <= 2]
|
||||||
|
burst_start_s = float(first_admission["t_s"]) if first_admission else -1.0
|
||||||
|
burst_end_s = float(full["t_s"]) if full else -1.0
|
||||||
|
work_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "game_thread_work_frame"}
|
||||||
|
work_values = [e for e in work_by_frame.values()
|
||||||
|
if burst_start_s <= float(e.get("t_s", -1.0)) <= burst_end_s]
|
||||||
|
arrival_frame_ids = {int(e.get("frame", -1)) for e in work_values}
|
||||||
|
scheduler_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "scheduler_admission"}
|
||||||
|
scheduler_values = [e for e in events if e.get("type") == "scheduler_admission"
|
||||||
|
and burst_start_s <= float(e.get("t_s", -1.0)) <= burst_end_s]
|
||||||
|
apply_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "apply_frame"}
|
||||||
|
commandlet_tick_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "commandlet_world_tick_frame"}
|
||||||
|
garbage_collect_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "garbage_collect"}
|
||||||
|
# apply_frame N+1 records the interval containing AVoxelWorld::Tick N. Its apply_ms
|
||||||
|
# belongs to Tick N+1, so use apply_by_frame[N] for the same-tick apply breakdown.
|
||||||
|
near_intervals = [apply_by_frame[frame + 1] for frame in arrival_frame_ids
|
||||||
|
if frame + 1 in apply_by_frame]
|
||||||
|
rmc_by_frame: dict[int, dict[str, float]] = {}
|
||||||
|
for event in events:
|
||||||
|
frame = int(event.get("frame", -1))
|
||||||
|
if event.get("type") != "rmc_submit" or frame not in arrival_frame_ids:
|
||||||
|
continue
|
||||||
|
values = rmc_by_frame.setdefault(frame, {
|
||||||
|
"component_acquire_ms": 0.0,
|
||||||
|
"create_section_group_ms": 0.0,
|
||||||
|
"create_sections_ms": 0.0,
|
||||||
|
"collision_handoff_ms": 0.0,
|
||||||
|
})
|
||||||
|
for key in values:
|
||||||
|
values[key] += float(event.get(key, 0.0))
|
||||||
|
collision_completion_by_frame: dict[int, float] = {}
|
||||||
|
for event in events:
|
||||||
|
frame = int(event.get("frame", -1))
|
||||||
|
if event.get("type") == "collision_completion" and frame in arrival_frame_ids:
|
||||||
|
collision_completion_by_frame[frame] = collision_completion_by_frame.get(frame, 0.0) \
|
||||||
|
+ float(event.get("handler_ms", 0.0))
|
||||||
|
near_requests = [r for r in request_events if full and r.get("t_s", 1e30) <= full["t_s"]
|
||||||
|
and r.get("level", 9) <= 2]
|
||||||
|
initial_lod0_counts = Counter(
|
||||||
|
tuple(r.get("tile", ())) for r in near_requests if r.get("level") == 0)
|
||||||
|
repeated_lod0_keys = [
|
||||||
|
{"tile": list(tile), "requests": count}
|
||||||
|
for tile, count in sorted(initial_lod0_counts.items()) if count > 1]
|
||||||
|
initial_scheduler_frames = [e for e in events
|
||||||
|
if e.get("type") == "scheduler_admission" and full
|
||||||
|
and e.get("t_s", 1e30) <= full["t_s"]]
|
||||||
|
worst_interval = max(near_intervals, key=lambda e: float(e.get("frame_ms", 0.0)), default=None)
|
||||||
|
worst_interval_frame_id = int(worst_interval.get("frame", -1)) if worst_interval else -1
|
||||||
|
worst_work_frame_id = worst_interval_frame_id - 1
|
||||||
|
worst_work = work_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_apply = apply_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_scheduler = scheduler_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_rmc = rmc_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_collision_completion_ms = collision_completion_by_frame.get(worst_work_frame_id, 0.0)
|
||||||
|
rmc_values = [rmc_by_frame.get(frame, {}) for frame in arrival_frame_ids]
|
||||||
|
apply_values = [apply_by_frame[frame] for frame in arrival_frame_ids
|
||||||
|
if frame in apply_by_frame]
|
||||||
|
commandlet_tick_values = [commandlet_tick_by_frame[frame] for frame in arrival_frame_ids
|
||||||
|
if frame in commandlet_tick_by_frame]
|
||||||
|
garbage_collect_values = [garbage_collect_by_frame[frame] for frame in arrival_frame_ids
|
||||||
|
if frame in garbage_collect_by_frame]
|
||||||
|
|
||||||
|
def work_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in work_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
def scheduler_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in scheduler_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
def rmc_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in rmc_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
def commandlet_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in commandlet_tick_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
row = {
|
||||||
|
"label": label, "stratum": stratum, "repeat": repeat, "phase": phase,
|
||||||
|
"ready_queue_depth": ready_depth,
|
||||||
|
"requested_background_worker_cap": background_cap,
|
||||||
|
"background_worker_cap": worker_cap_readback.get(
|
||||||
|
"streaming_background_worker_cap", background_cap),
|
||||||
|
"requested_background_worker_reserve": worker_reserve,
|
||||||
|
"background_worker_reserve": worker_backend_config.get(
|
||||||
|
"streaming_background_worker_reserve", worker_cap_readback.get(
|
||||||
|
"background_worker_reserve", worker_reserve)),
|
||||||
|
"requested_core_parallel_for_background_priority": helper_background,
|
||||||
|
"core_parallel_for_background_priority": worker_backend_config.get(
|
||||||
|
"core_parallel_for_background_priority", helper_background),
|
||||||
|
"requested_streaming_tile_background_low": tile_background_low,
|
||||||
|
"streaming_tile_background_low": worker_backend_config.get(
|
||||||
|
"streaming_tile_background_low", tile_background_low),
|
||||||
|
"requested_streaming_tile_background_normal": tile_background_normal,
|
||||||
|
"streaming_tile_background_normal": worker_backend_config.get(
|
||||||
|
"streaming_tile_background_normal", tile_background_normal),
|
||||||
|
"effective_tile_slots": worker_backend_config.get("effective_tile_slots"),
|
||||||
|
"background_worker_threads": worker_backend_config.get("taskgraph_background_threads"),
|
||||||
|
"startup_apply_boost": startup_boost,
|
||||||
|
"streaming_far_apply_supplement": worker_backend_config.get(
|
||||||
|
"streaming_far_apply_supplement"),
|
||||||
|
"insights": insights,
|
||||||
|
"engine_tick_logs": tick_logs,
|
||||||
|
"exit_code": exit_code,
|
||||||
|
"launch_wrapper": "taskset -c 0-11 timeout 1740s (no address-space limit)",
|
||||||
|
"peak_rss_kib": peak_rss[0] if peak_rss else None,
|
||||||
|
"peak_rss_mib": round(peak_rss[0] / 1024.0, 1) if peak_rss else None,
|
||||||
|
"peak_rss_pid": peak_rss[1] if peak_rss else None,
|
||||||
|
"endphysics_to_postphysics_gap_ms_max": None,
|
||||||
|
"endphysics_gap_frame": None,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"desired_t_s": next((e.get("t_s") for e in events if e.get("type") == "desired_set"), None),
|
||||||
|
"first_admission_t_s": first_admission.get("t_s") if first_admission else None,
|
||||||
|
"lod02_full_settled_t_s": full.get("t_s") if full else None,
|
||||||
|
"near_settle_from_first_admission_s": round(
|
||||||
|
float(full["t_s"]) - float(first_admission["t_s"]), 4)
|
||||||
|
if full and first_admission else None,
|
||||||
|
"desired_by_lod": full.get("desired_by_lod", [])[:3] if full else None,
|
||||||
|
"settled_by_lod": full.get("settled_by_lod", [])[:3] if full else None,
|
||||||
|
"near_admitted_tiles": len(near_requests),
|
||||||
|
"initial_lod0_admissions": sum(initial_lod0_counts.values()),
|
||||||
|
"repeated_lod0_request_keys_before_settle": repeated_lod0_keys,
|
||||||
|
"peak_result_backlog_before_settle": max(
|
||||||
|
(int(e.get("result_backlog", 0)) for e in initial_scheduler_frames), default=0),
|
||||||
|
"last_admission_t_s": max((e.get("t_s", 0.0) for e in near_requests), default=None),
|
||||||
|
"near_tile_events": len(initial_tiles),
|
||||||
|
"near_worker_seconds": round(sum(e.get("worker_task_s", 0.0) for e in initial_tiles), 4),
|
||||||
|
"near_core_seconds": round(sum(e.get("core_generate_s", 0.0) for e in initial_tiles), 4),
|
||||||
|
"collision_ready_by_settle": collision_count,
|
||||||
|
"burst_frame_count": len(near_intervals),
|
||||||
|
"burst_frame_ms_p95": round(sorted(e.get("frame_ms", 0.0) for e in near_intervals)[
|
||||||
|
min(len(near_intervals) - 1, int(.95 * (len(near_intervals) - 1)))], 3)
|
||||||
|
if near_intervals else None,
|
||||||
|
"burst_frame_ms_max": round(max((e.get("frame_ms", 0.0) for e in near_intervals), default=0.0), 3),
|
||||||
|
"burst_full_world_tick_ms_p99": round(commandlet_stat("world_tick_ms", .99) or 0.0, 3),
|
||||||
|
"burst_full_world_tick_ms_max": round(commandlet_stat("world_tick_ms") or 0.0, 3),
|
||||||
|
"burst_world_pre_actor_ms_p99": round(commandlet_stat("world_pre_actor_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_pre_actor_ms_max": round(commandlet_stat("world_pre_actor_ms") or 0.0, 3),
|
||||||
|
"burst_world_actor_phase_ms_p99": round(commandlet_stat("world_actor_phase_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_actor_phase_ms_max": round(commandlet_stat("world_actor_phase_ms") or 0.0, 3),
|
||||||
|
"burst_world_post_actor_ms_p99": round(commandlet_stat("world_post_actor_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_post_actor_ms_max": round(commandlet_stat("world_post_actor_ms") or 0.0, 3),
|
||||||
|
"burst_commandlet_readiness_check_ms_max": round(
|
||||||
|
commandlet_stat("readiness_check_ms") or 0.0, 3),
|
||||||
|
"burst_commandlet_sleep_ms_p99": round(commandlet_stat("sleep_ms", .99) or 0.0, 3),
|
||||||
|
"burst_commandlet_sleep_ms_max": round(commandlet_stat("sleep_ms") or 0.0, 3),
|
||||||
|
"burst_commandlet_loop_ms_max": round(commandlet_stat("loop_ms") or 0.0, 3),
|
||||||
|
"burst_gc_count": len(garbage_collect_values),
|
||||||
|
"burst_gc_duration_ms_max": round(max(
|
||||||
|
(float(e.get("duration_ms", 0.0)) for e in garbage_collect_values), default=0.0), 3),
|
||||||
|
"burst_max_meshes_per_frame": max((e.get("meshes_applied", 0) for e in apply_values), default=0),
|
||||||
|
"burst_max_apply_ms": round(max((e.get("apply_ms", 0.0) for e in apply_values), default=0.0), 3),
|
||||||
|
"burst_max_admit_per_frame": max((sum(e.get("submitted_by_lod", [0] * 9)[:3])
|
||||||
|
for e in scheduler_values), default=0),
|
||||||
|
"burst_frame_timing_count": len(work_values),
|
||||||
|
"burst_active_generation_workers_p95": round(work_stat("active_generation_workers", .95) or 0.0, 3),
|
||||||
|
"burst_active_generation_workers_max": round(work_stat("active_generation_workers") or 0.0, 3),
|
||||||
|
"burst_active_tile_tasks_max": round(work_stat("active_tile_tasks") or 0.0, 3),
|
||||||
|
"burst_world_tick_ms_p99": round(work_stat("tick_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_tick_ms_max": round(work_stat("tick_ms") or 0.0, 3),
|
||||||
|
"burst_update_chunks_ms_p99": round(work_stat("update_chunks_ms", .99) or 0.0, 3),
|
||||||
|
"burst_update_chunks_ms_max": round(work_stat("update_chunks_ms") or 0.0, 3),
|
||||||
|
"burst_pending_drain_ms_p99": round(work_stat("pending_drain_ms", .99) or 0.0, 3),
|
||||||
|
"burst_pending_drain_ms_max": round(work_stat("pending_drain_ms") or 0.0, 3),
|
||||||
|
"burst_post_apply_callbacks_ms_p99": round(work_stat("post_apply_callbacks_ms", .99) or 0.0, 3),
|
||||||
|
"burst_post_apply_callbacks_ms_max": round(work_stat("post_apply_callbacks_ms") or 0.0, 3),
|
||||||
|
"burst_visibility_reconcile_ms_p99": round(work_stat("visibility_reconcile_ms", .99) or 0.0, 3),
|
||||||
|
"burst_visibility_reconcile_ms_max": round(work_stat("visibility_reconcile_ms") or 0.0, 3),
|
||||||
|
"burst_collision_probe_ms_max": round(work_stat("collision_probe_ms") or 0.0, 3),
|
||||||
|
"burst_visible_apply_probe_ms_max": round(work_stat("visible_apply_probe_ms") or 0.0, 3),
|
||||||
|
"burst_lod0_ready_ms_max": round(work_stat("lod0_ready_ms") or 0.0, 3),
|
||||||
|
"burst_steady_trace_ms_max": round(work_stat("steady_trace_ms") or 0.0, 3),
|
||||||
|
"burst_modification_ms_max": round(work_stat("modification_ms") or 0.0, 3),
|
||||||
|
"burst_lifecycle_audit_ms_p99": round(work_stat("lifecycle_audit_ms", .99) or 0.0, 3),
|
||||||
|
"burst_lifecycle_audit_ms_max": round(work_stat("lifecycle_audit_ms") or 0.0, 3),
|
||||||
|
"burst_collision_gate_ms_p99": round(work_stat("collision_gate_ms", .99) or 0.0, 3),
|
||||||
|
"burst_collision_gate_ms_max": round(work_stat("collision_gate_ms") or 0.0, 3),
|
||||||
|
"burst_unload_ms_p99": round(work_stat("unload_ms", .99) or 0.0, 3),
|
||||||
|
"burst_unload_ms_max": round(work_stat("unload_ms") or 0.0, 3),
|
||||||
|
"burst_trace_json_ms_p99": round(work_stat("trace_json_ms", .99) or 0.0, 3),
|
||||||
|
"burst_trace_json_ms_max": round(work_stat("trace_json_ms") or 0.0, 3),
|
||||||
|
"burst_desired_rebuild_ms_max": round(scheduler_stat("desired_rebuild_ms") or 0.0, 3),
|
||||||
|
"burst_rmc_create_group_ms_p99": round(rmc_stat("create_section_group_ms", .99) or 0.0, 3),
|
||||||
|
"burst_rmc_create_group_ms_max": round(rmc_stat("create_section_group_ms") or 0.0, 3),
|
||||||
|
"burst_rmc_collision_handoff_ms_max": round(rmc_stat("collision_handoff_ms") or 0.0, 3),
|
||||||
|
"burst_collision_completion_ms_max": round(max(
|
||||||
|
collision_completion_by_frame.values(), default=0.0), 3),
|
||||||
|
"worst_frame_breakdown": {
|
||||||
|
"frame": worst_work_frame_id,
|
||||||
|
"world_tick_frame": worst_work_frame_id,
|
||||||
|
"frame_interval_end_frame": worst_interval_frame_id,
|
||||||
|
"frame_interval_ms": round(float(worst_interval.get("frame_ms", 0.0)), 3)
|
||||||
|
if worst_interval else None,
|
||||||
|
"apply_ms": round(float(worst_apply.get("apply_ms", 0.0)), 3)
|
||||||
|
if worst_apply else None,
|
||||||
|
"result_drain_ms": round(float(worst_apply.get("drain_ms", 0.0)), 3)
|
||||||
|
if worst_apply else None,
|
||||||
|
"outside_world_tick_ms": round(max(0.0, float(worst_interval.get("frame_ms", 0.0))
|
||||||
|
- float(worst_work.get("tick_ms", 0.0))), 3)
|
||||||
|
if worst_interval else None,
|
||||||
|
"outside_full_world_tick_ms": round(max(0.0, float(worst_interval.get("frame_ms", 0.0))
|
||||||
|
- float(commandlet_tick_by_frame.get(worst_work_frame_id, {}).get("world_tick_ms", 0.0))), 3)
|
||||||
|
if worst_interval else None,
|
||||||
|
"full_world_tick_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_tick_ms", 0.0)), 3),
|
||||||
|
"world_pre_actor_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_pre_actor_ms", 0.0)), 3),
|
||||||
|
"world_actor_phase_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_actor_phase_ms", 0.0)), 3),
|
||||||
|
"world_post_actor_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_post_actor_ms", 0.0)), 3),
|
||||||
|
"commandlet_readiness_check_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("readiness_check_ms", 0.0)), 3),
|
||||||
|
"commandlet_sleep_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("sleep_ms", 0.0)), 3),
|
||||||
|
"commandlet_loop_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("loop_ms", 0.0)), 3),
|
||||||
|
"gc_duration_ms": round(float(garbage_collect_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("duration_ms", 0.0)), 3),
|
||||||
|
"world_tick_ms": round(float(worst_work.get("tick_ms", 0.0)), 3),
|
||||||
|
"active_generation_workers": int(worst_work.get("active_generation_workers", 0)),
|
||||||
|
"active_tile_tasks": int(worst_work.get("active_tile_tasks", 0)),
|
||||||
|
"collision_gate_ms": round(float(worst_work.get("collision_gate_ms", 0.0)), 3),
|
||||||
|
"update_chunks_ms": round(float(worst_work.get("update_chunks_ms", 0.0)), 3),
|
||||||
|
"desired_rebuild_ms": round(float(worst_scheduler.get("desired_rebuild_ms", 0.0)), 3),
|
||||||
|
"pending_drain_ms": round(float(worst_work.get("pending_drain_ms", 0.0)), 3),
|
||||||
|
"post_apply_callbacks_ms": round(float(worst_work.get("post_apply_callbacks_ms", 0.0)), 3),
|
||||||
|
"collision_probe_ms": round(float(worst_work.get("collision_probe_ms", 0.0)), 3),
|
||||||
|
"visibility_reconcile_ms": round(float(worst_work.get("visibility_reconcile_ms", 0.0)), 3),
|
||||||
|
"visible_apply_probe_ms": round(float(worst_work.get("visible_apply_probe_ms", 0.0)), 3),
|
||||||
|
"lod0_ready_ms": round(float(worst_work.get("lod0_ready_ms", 0.0)), 3),
|
||||||
|
"unload_ms": round(float(worst_work.get("unload_ms", 0.0)), 3),
|
||||||
|
"steady_trace_ms": round(float(worst_work.get("steady_trace_ms", 0.0)), 3),
|
||||||
|
"modification_ms": round(float(worst_work.get("modification_ms", 0.0)), 3),
|
||||||
|
"lifecycle_audit_ms": round(float(worst_work.get("lifecycle_audit_ms", 0.0)), 3),
|
||||||
|
"trace_json_ms": round(float(worst_work.get("trace_json_ms", 0.0)), 3),
|
||||||
|
"trace_archive_serialize_ms": round(float(
|
||||||
|
worst_work.get("trace_archive_serialize_ms", 0.0)), 3),
|
||||||
|
"collision_completion_ms": round(worst_collision_completion_ms, 3),
|
||||||
|
**{key: round(float(worst_rmc.get(key, 0.0)), 3)
|
||||||
|
for key in ("component_acquire_ms", "create_section_group_ms",
|
||||||
|
"create_sections_ms", "collision_handoff_ms")},
|
||||||
|
},
|
||||||
|
"trace": str(trace.relative_to(ROOT)), "utrace": str(utrace.relative_to(ROOT)),
|
||||||
|
"abslog": str(abslog.relative_to(ROOT)), "stdout": str(stdout.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
stdout_text = stdout.read_text(encoding="utf-8", errors="replace")
|
||||||
|
gap_ms, gap_frame = endphysics_gap(stdout)
|
||||||
|
row["endphysics_to_postphysics_gap_ms_max"] = gap_ms
|
||||||
|
row["endphysics_gap_frame"] = gap_frame
|
||||||
|
failure = re.search(r"\[VoxelForgeCoreWorldTest\] failed: ([^\r\n]+)", stdout_text)
|
||||||
|
if failure:
|
||||||
|
row["world_test_failure"] = failure.group(1)
|
||||||
|
with (OUT / "runs.jsonl").open("a", encoding="utf-8") as stream:
|
||||||
|
stream.write(json.dumps(row, sort_keys=True) + "\n")
|
||||||
|
(OUT / f"{label}.summary.json").write_text(json.dumps(row, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
print("[L105UnrealArrival] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
if exit_code:
|
||||||
|
raise SystemExit(f"Unreal commandlet failed: {stdout}; exit={exit_code}")
|
||||||
|
if not full or len(arrival) < 8:
|
||||||
|
raise SystemExit(f"Unreal arrival trace incomplete: {stdout}; exit={exit_code}")
|
||||||
|
if insights and (not utrace.is_file() or utrace.stat().st_size < 1024):
|
||||||
|
raise SystemExit(f"Unreal Insights trace missing or empty: {utrace}")
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--stratum", choices=("FI", "SW"), required=True)
|
||||||
|
parser.add_argument("--repeat", type=int, default=1)
|
||||||
|
parser.add_argument("--phase", choices=("probe", "before", "after", "snapshot"), default="probe")
|
||||||
|
parser.add_argument("--ready-depth", type=int, default=64)
|
||||||
|
parser.add_argument("--background-cap", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--worker-reserve", type=int, choices=(0, 1, 2, 3), default=0)
|
||||||
|
parser.add_argument("--helper-background", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--tile-background-low", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--tile-background-normal", type=int, choices=(0, 1), default=0)
|
||||||
|
parser.add_argument("--startup-boost", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--tick-logs", action="store_true")
|
||||||
|
parser.add_argument("--insights", action="store_true")
|
||||||
|
args = parser.parse_args()
|
||||||
|
if not EDITOR.is_file():
|
||||||
|
raise SystemExit(f"missing Unreal commandlet executable: {EDITOR}")
|
||||||
|
run(args.stratum, args.repeat, args.phase, args.ready_depth,
|
||||||
|
args.background_cap, args.worker_reserve, args.helper_background,
|
||||||
|
args.tile_background_low, args.tile_background_normal, args.startup_boost,
|
||||||
|
args.tick_logs, args.insights)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,104 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Five interleaved FI arrival pairs, comparing normal applies with one supplemental far apply."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
RUNNER_PATH = Path(__file__).with_name("RunL121Arrival.py")
|
||||||
|
SPEC = importlib.util.spec_from_file_location("l121_arrival", RUNNER_PATH)
|
||||||
|
assert SPEC and SPEC.loader
|
||||||
|
ARRIVAL = importlib.util.module_from_spec(SPEC)
|
||||||
|
SPEC.loader.exec_module(ARRIVAL)
|
||||||
|
|
||||||
|
|
||||||
|
def add_view_metrics(row: dict) -> dict:
|
||||||
|
trace_path = ROOT / row["trace"]
|
||||||
|
events = ARRIVAL.read_events(trace_path)
|
||||||
|
config = next((e for e in events if e.get("type") == "worker_backend_config"), {})
|
||||||
|
row["effective_tile_slots"] = config.get("effective_tile_slots")
|
||||||
|
row["near_priority"] = config.get("near_priority")
|
||||||
|
row["horizon_priority"] = config.get("horizon_priority")
|
||||||
|
samples = [e for e in events if e.get("type") == "streaming_sample"
|
||||||
|
and float(e.get("t_s", 0.0)) < ARRIVAL.MOVE_START_S
|
||||||
|
and float(e.get("speed_cm_s", 0.0)) < 50.0]
|
||||||
|
|
||||||
|
def total(values: list[int]) -> int:
|
||||||
|
return sum(values)
|
||||||
|
|
||||||
|
def near_complete(event: dict) -> bool:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
settled = event.get("settled_by_lod", [0] * 9)
|
||||||
|
return total(desired[:3]) > 0 and total(settled[:3]) >= total(desired[:3])
|
||||||
|
|
||||||
|
def whole_complete(event: dict) -> bool:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
settled = event.get("settled_by_lod", [0] * 9)
|
||||||
|
return total(desired) > 0 and total(settled) >= total(desired)
|
||||||
|
|
||||||
|
def far_holes(event: dict) -> float | None:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)[3:]
|
||||||
|
covered = event.get("covered_by_lod", [0] * 9)[3:]
|
||||||
|
denominator = total(desired)
|
||||||
|
return (100.0 * (denominator - total(covered)) / denominator
|
||||||
|
if denominator else None)
|
||||||
|
|
||||||
|
first_near = next((e for e in samples if near_complete(e)), None)
|
||||||
|
first_whole = next((e for e in samples if whole_complete(e)), None)
|
||||||
|
first_far_clear = next((e for e in samples if total(e.get("desired_by_lod", [0] * 9)[3:])
|
||||||
|
and far_holes(e) == 0.0), None)
|
||||||
|
last_sample = samples[-1] if samples else None
|
||||||
|
row["lod02_settled_t_s"] = float(first_near["t_s"]) if first_near else None
|
||||||
|
row["whole_view_settled_t_s"] = float(first_whole["t_s"]) if first_whole else None
|
||||||
|
row["far_clear_t_s"] = float(first_far_clear["t_s"]) if first_far_clear else None
|
||||||
|
row["far_holes_at_20s_pct"] = far_holes(last_sample) if last_sample else None
|
||||||
|
row["arrival_sample_count"] = len(samples)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def run_condition(condition: str, repetition: int, ready_depth: int) -> dict:
|
||||||
|
cap = 1
|
||||||
|
expected_slots = 8
|
||||||
|
row = ARRIVAL.run(
|
||||||
|
"FI", repetition, condition, ready_depth, cap, 0, 1, 1, 0, 1, False, False,
|
||||||
|
0 if condition == "baseline" else 1)
|
||||||
|
row = add_view_metrics(row)
|
||||||
|
if row["effective_tile_slots"] != expected_slots:
|
||||||
|
raise RuntimeError(f"worker slot readback mismatch: {row}")
|
||||||
|
if row["streaming_far_apply_supplement"] != (0 if condition == "baseline" else 1):
|
||||||
|
raise RuntimeError(f"far apply supplement CVar mismatch: {row}")
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--ready-depth", type=int, choices=(8, 64), default=8)
|
||||||
|
ready_depth = parser.parse_args().ready_depth
|
||||||
|
OUT = ROOT / "Saved/Linux/l121/arrival"
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
results = []
|
||||||
|
for condition in ("baseline", "far_supplement"):
|
||||||
|
warmup = run_condition(condition, 0, ready_depth)
|
||||||
|
warmup["warmup"] = True
|
||||||
|
results.append(warmup)
|
||||||
|
|
||||||
|
for repetition in range(1, 6):
|
||||||
|
order = ("baseline", "far_supplement") if repetition % 2 else ("far_supplement", "baseline")
|
||||||
|
for condition in order:
|
||||||
|
row = run_condition(condition, repetition, ready_depth)
|
||||||
|
row["warmup"] = False
|
||||||
|
results.append(row)
|
||||||
|
print("[L121ArrivalPair] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
|
||||||
|
(OUT / f"pairs-rd{ready_depth}.json").write_text(json.dumps({
|
||||||
|
"design": f"FI arrival, normal four applies versus one supplemental far apply, ready depth {ready_depth}",
|
||||||
|
"runs": results,
|
||||||
|
}, indent=2, sort_keys=True) + "\n", encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,117 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Compare L121 exact tile hashes with L104 across all strata, compilers and workers."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
OUT = ROOT / "Saved/Linux/l121/hashes"
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
OLD = ROOT / "Saved/Linux/l104/hashes"
|
||||||
|
BINARIES = {
|
||||||
|
"clang": ROOT / "Saved/Linux/l121/verify/bin/VoxelForgeCoreHeadless-verify-clang",
|
||||||
|
"gcc": ROOT / "Saved/Linux/l121/verify/bin/VoxelForgeCoreHeadless-verify-gcc",
|
||||||
|
}
|
||||||
|
CAMERAS = {
|
||||||
|
"sw": ((8.0, 16.0, -372.2626953125), 2850),
|
||||||
|
"fi": ((25.8369140625, 3.19921875, -737.0), 2249),
|
||||||
|
"tn": ((19.166015625, -14.2001953125, -753.3740234375), 2233),
|
||||||
|
"vs": ((-20.0625, -5.068359375, -900.353515625), 2329),
|
||||||
|
}
|
||||||
|
TILE = re.compile(
|
||||||
|
r"^\[PipelineTileHash\] lod=(-?\d+) x=(-?\d+) y=(-?\d+) z=(-?\d+) "
|
||||||
|
r"density=(0x[0-9a-f]+) mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) "
|
||||||
|
r"topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
WHOLE = re.compile(
|
||||||
|
r"^\[PipelineWholeViewHash\] tiles=(\d+) density=(0x[0-9a-f]+) "
|
||||||
|
r"mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M)
|
||||||
|
|
||||||
|
|
||||||
|
def parse(path: Path) -> tuple[dict, tuple[str, ...], str]:
|
||||||
|
content = path.read_text(encoding="utf-8", errors="replace")
|
||||||
|
tiles = {(int(m.group(1)), int(m.group(2)), int(m.group(3)), int(m.group(4))):
|
||||||
|
m.group(5, 6, 7, 8) for m in TILE.finditer(content)}
|
||||||
|
whole, stamp = WHOLE.search(content), STAMP.search(content)
|
||||||
|
if whole is None or stamp is None or len(tiles) != int(whole.group(1)):
|
||||||
|
raise RuntimeError(f"incomplete hash output: {path}")
|
||||||
|
return tiles, whole.groups(), stamp.group(1)
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, compiler: str, workers: int) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = OUT / f"{stratum}-{compiler}-w{workers}.log"
|
||||||
|
camera, expected_tiles = CAMERAS[stratum]
|
||||||
|
camera_arg = ",".join(f"{v:.12f}" for v in camera) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--", "timeout", "1740s",
|
||||||
|
str(BINARIES[compiler]), str(DATA), "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", f"--workers={workers}",
|
||||||
|
"--clip-radius=3", "--max-clip-level=8", "--max-level=8",
|
||||||
|
f"--camera={camera_arg}", "--stream-hash-tiles",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
if process.returncode:
|
||||||
|
raise RuntimeError(f"hash process exited {process.returncode}: {log}")
|
||||||
|
tiles, whole, stamp = parse(log)
|
||||||
|
if len(tiles) != expected_tiles:
|
||||||
|
raise RuntimeError(f"expected {expected_tiles} tiles, got {len(tiles)}: {log}")
|
||||||
|
old_log = OLD / f"{stratum}-{compiler}-w{workers}.log"
|
||||||
|
old_tiles, old_whole, _old_stamp = parse(old_log)
|
||||||
|
clang_reference, clang_whole, _ = parse(OLD / f"{stratum}-clang-w1.log")
|
||||||
|
|
||||||
|
def difference(reference: dict, candidate: dict) -> tuple[int, int]:
|
||||||
|
keys = set(reference) | set(candidate)
|
||||||
|
mismatches = sum(reference.get(key) != candidate.get(key) for key in keys)
|
||||||
|
return mismatches, len(keys)
|
||||||
|
|
||||||
|
old_mismatches, old_keys = difference(old_tiles, tiles)
|
||||||
|
cross_mismatches, cross_keys = difference(clang_reference, tiles)
|
||||||
|
row = {
|
||||||
|
"stratum": stratum, "compiler": compiler, "workers": workers,
|
||||||
|
"source_stamp": stamp, "tiles": len(tiles),
|
||||||
|
"l104_same_cell_tile_mismatches": old_mismatches,
|
||||||
|
"l104_same_cell_key_count": old_keys,
|
||||||
|
"l104_same_cell_whole_hash_match": whole == old_whole,
|
||||||
|
"clang_w1_tile_mismatches": cross_mismatches,
|
||||||
|
"clang_w1_key_count": cross_keys,
|
||||||
|
"clang_w1_whole_hash_match": whole == clang_whole,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L121HashRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
row["tile_hashes"] = {"|".join(map(str, key)): value for key, value in tiles.items()}
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
for path in (*BINARIES.values(), DATA):
|
||||||
|
if not path.is_file():
|
||||||
|
raise SystemExit(f"missing input: {path}")
|
||||||
|
rows = []
|
||||||
|
for stratum in CAMERAS:
|
||||||
|
for compiler in BINARIES:
|
||||||
|
for workers in (1, 8, 12):
|
||||||
|
rows.append(run(stratum, compiler, workers))
|
||||||
|
(OUT / "runs-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
failures = [row for row in rows if row["l104_same_cell_tile_mismatches"]
|
||||||
|
or not row["l104_same_cell_whole_hash_match"]
|
||||||
|
or row["clang_w1_tile_mismatches"]
|
||||||
|
or not row["clang_w1_whole_hash_match"]]
|
||||||
|
summary = {"runs": [{key: value for key, value in row.items() if key != "tile_hashes"}
|
||||||
|
for row in rows], "failed_runs": len(failures)}
|
||||||
|
(OUT / "summary.json").write_text(json.dumps(summary, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 1 if failures else 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,285 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Five interleaved L121 apply-supplement pairs at 30 m/s."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import re
|
||||||
|
import statistics
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
HOST = Path("/home/jahni/VFHost")
|
||||||
|
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
HEADLESS = ROOT / "Saved/Linux/l108/bin/VoxelForgeCoreHeadless-l108-after-clang"
|
||||||
|
OUT = ROOT / "Saved/Linux/l121/movement/far-apply-supplement"
|
||||||
|
HOST_SAVED = HOST / "Saved/Linux/l121"
|
||||||
|
HOST_OUT = HOST / "Plugins/VoxelForge/Saved/Linux/l121/far-apply-supplement"
|
||||||
|
ACTIVE_DESIGN = "far_apply_supplement"
|
||||||
|
SPAWN_M = (19.166015625, -14.2001953125, -753.3740234375)
|
||||||
|
MOVING_SAMPLE = re.compile(r"^\[L79MovingSample\] (.*)$", re.M)
|
||||||
|
MOVING_SUMMARY = re.compile(r"^\[L79MovingSummary\] (.*)$", re.M)
|
||||||
|
FIELDS = re.compile(r"([a-zA-Z0-9_]+)=([^ ]+)")
|
||||||
|
|
||||||
|
|
||||||
|
def p99(values: list[float]) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
values.sort()
|
||||||
|
return values[max(0, math.ceil(0.99 * len(values)) - 1)]
|
||||||
|
|
||||||
|
|
||||||
|
def parse_events(path: Path) -> list[dict]:
|
||||||
|
events = []
|
||||||
|
if not path.is_file():
|
||||||
|
return events
|
||||||
|
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||||
|
if not line.startswith("{"):
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
events.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
return events
|
||||||
|
|
||||||
|
|
||||||
|
def summarize(rows: list[dict], phase: str) -> dict:
|
||||||
|
moving = [row for row in rows if row["phase"] == "move"]
|
||||||
|
dwell = [row for row in rows if row["phase"] == "dwell"]
|
||||||
|
def moving_mean(name: str) -> float | None:
|
||||||
|
values = [row[name] for row in moving if name in row]
|
||||||
|
return statistics.mean(values) if values else None
|
||||||
|
complete = next((row for row in dwell
|
||||||
|
if row["near_ready_pct"] >= 99.999
|
||||||
|
and row["far_hole_pct"] <= 0.001), None)
|
||||||
|
end = dwell[-1] if dwell else {}
|
||||||
|
return {
|
||||||
|
"moving_samples": len(moving),
|
||||||
|
"dwell_samples": len(dwell),
|
||||||
|
"near_ready_moving_mean_pct": moving_mean("near_ready_pct"),
|
||||||
|
"lod0_ready_moving_mean_pct": moving_mean("lod0_ready_pct"),
|
||||||
|
"lod1_ready_moving_mean_pct": moving_mean("lod1_ready_pct"),
|
||||||
|
"lod2_ready_moving_mean_pct": moving_mean("lod2_ready_pct"),
|
||||||
|
"far_holes_moving_mean_pct": moving_mean("far_hole_pct"),
|
||||||
|
"near_ready_dwell_end_pct": end.get("near_ready_pct"),
|
||||||
|
"far_holes_dwell_end_pct": end.get("far_hole_pct"),
|
||||||
|
"dwell_complete_after_stop_s": complete.get("since_stop_s") if complete else None,
|
||||||
|
"phase": phase,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def run_headless(condition: str, repetition: int) -> dict:
|
||||||
|
worker_count = 8
|
||||||
|
folder = OUT / condition / "headless"
|
||||||
|
folder.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = folder / f"fast-r{repetition}.log"
|
||||||
|
camera = ",".join(f"{value:.12f}" for value in SPAWN_M) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", str(HEADLESS), str(DATA),
|
||||||
|
"--world-assembler-prototype", "--stream-benchmark", "--stream-pipeline",
|
||||||
|
f"--workers={worker_count}", "--clip-radius=3", "--max-clip-level=8",
|
||||||
|
"--max-level=8", f"--camera={camera}", "--profile-lod-work",
|
||||||
|
"--stream-scheduler-policy=3", "--moving-camera-speed-mps=30.0",
|
||||||
|
"--moving-camera-duration-s=12", "--moving-camera-dwell-s=5",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
result = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
content = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
rows = []
|
||||||
|
for match in MOVING_SAMPLE.finditer(content):
|
||||||
|
row = dict(FIELDS.findall(match.group(1)))
|
||||||
|
for name in ("t_s", "near_ready_pct", "walkable_ring_ready_pct",
|
||||||
|
"far_hole_pct", "queue_update_ms"):
|
||||||
|
if name in row:
|
||||||
|
row[name] = float(row[name])
|
||||||
|
row["phase"] = row.get("phase", "move")
|
||||||
|
row["since_stop_s"] = max(0.0, row.get("t_s", 0.0) - 12.0)
|
||||||
|
rows.append(row)
|
||||||
|
engine_summary = {}
|
||||||
|
if match := MOVING_SUMMARY.search(content):
|
||||||
|
engine_summary = dict(FIELDS.findall(match.group(1)))
|
||||||
|
if result.returncode or not rows or int(engine_summary.get("generation_failures", "-1")) != 0:
|
||||||
|
raise RuntimeError(f"headless run failed: {log}; exit={result.returncode}")
|
||||||
|
moving_updates = [row["queue_update_ms"] for row in rows
|
||||||
|
if row["phase"] == "move" and "queue_update_ms" in row]
|
||||||
|
return {
|
||||||
|
"condition": condition, "platform": "headless", "speed_mps": 30.0,
|
||||||
|
"repetition": repetition, "worker_count": worker_count,
|
||||||
|
"exit_code": result.returncode,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"update_work_p99_ms": p99(moving_updates),
|
||||||
|
"update_work_max_ms": max(moving_updates, default=None),
|
||||||
|
"log": str(log.relative_to(ROOT)), **summarize(rows, "headless"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def run_unreal(condition: str, repetition: int) -> dict:
|
||||||
|
core_limit = 12
|
||||||
|
worker_cap = 1
|
||||||
|
far_apply_supplement = 0 if condition == "baseline" else 1
|
||||||
|
folder = OUT / condition / "unreal"
|
||||||
|
host_folder = HOST_OUT / condition / "output"
|
||||||
|
folder.mkdir(parents=True, exist_ok=True)
|
||||||
|
trace = folder / f"fast-r{repetition}.trace.jsonl"
|
||||||
|
abslog = folder / f"fast-r{repetition}.log"
|
||||||
|
stdout = folder / f"fast-r{repetition}.stdout.log"
|
||||||
|
export = host_folder / f"fast-r{repetition}"
|
||||||
|
userdir = HOST_SAVED / "user" / condition / f"fast-r{repetition}"
|
||||||
|
zendata = HOST_SAVED / "zen" / condition / f"fast-r{repetition}"
|
||||||
|
ddc = HOST_SAVED / "DDC"
|
||||||
|
for path in (export, userdir, zendata, ddc):
|
||||||
|
path.mkdir(parents=True, exist_ok=True)
|
||||||
|
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWN_M)
|
||||||
|
command = [
|
||||||
|
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
|
||||||
|
"-nop4", "-nosplash", "-NoSound", f"-corelimit={core_limit}",
|
||||||
|
"-NoZenAutoLaunch", "-notraceserver", "-nocrashreports",
|
||||||
|
"-DisablePlugins=AndroidFileServer", "-ddc=NoZenLocalFallback",
|
||||||
|
f"-userdir={userdir}", f"-ZenDataPath={zendata}",
|
||||||
|
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
|
||||||
|
"-archetype=TunnelNetwork", "-slot=0", "-modes=export", "-gameconfig=1",
|
||||||
|
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
|
||||||
|
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0",
|
||||||
|
"-coreplantest=0", "-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings",
|
||||||
|
"-corehybridmesher=0", f"-out={export}",
|
||||||
|
"-voxel.StreamingCoarseFirst=1", "-voxel.StreamingDynamicNearReserve=1",
|
||||||
|
"-voxel.StreamingLegacyPendingResultCap=0", "-voxel.CoreRegionConcurrentAdmission=1",
|
||||||
|
"-voxel.StreamingResultBacklogCap=256", "-voxel.StreamingReadyQueueDepth=8",
|
||||||
|
f"-voxel.StreamingBackgroundWorkerCap={worker_cap}",
|
||||||
|
"-voxel.CoreColumnCacheDirectMapped=0",
|
||||||
|
f"-voxel.StreamingFarApplySupplement={far_apply_supplement}",
|
||||||
|
"-voxel.CoreMCGradientMemoization=1", "-voxel.StreamingCollisionLookahead=1",
|
||||||
|
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=3000.0",
|
||||||
|
"-voxel.TestMoveStartSeconds=6.0", "-voxel.TestMoveDurationSeconds=12.0",
|
||||||
|
"-voxel.TestExitSeconds=23.0", "-voxel.TestMoveDirection=1,0,0",
|
||||||
|
"-voxel.StreamingSchedulerPolicy=3", "-voxel.TestExitOnSteady=0",
|
||||||
|
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
|
||||||
|
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
|
||||||
|
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
|
||||||
|
f"-abslog={abslog}",
|
||||||
|
]
|
||||||
|
wrapped = ["taskset", "-c", "0-11", "timeout", "1740s", *command]
|
||||||
|
started = time.monotonic()
|
||||||
|
with stdout.open("w", encoding="utf-8") as stream:
|
||||||
|
result = subprocess.run(wrapped, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
events = parse_events(trace)
|
||||||
|
samples = [event for event in events if event.get("type") == "streaming_sample"]
|
||||||
|
moving_samples = [event for event in samples
|
||||||
|
if float(event.get("speed_cm_s", 0.0)) >= 50.0]
|
||||||
|
move_start = min((float(event.get("t_s", 0.0)) for event in moving_samples),
|
||||||
|
default=float("inf"))
|
||||||
|
stop_event = next((event for event in events
|
||||||
|
if event.get("type") == "streaming_test_move_stopped"), None)
|
||||||
|
stop = float(stop_event.get("t_s", 0.0)) if stop_event else float("inf")
|
||||||
|
rows = []
|
||||||
|
for event in samples:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
resident = event.get("resident_by_lod", [0] * 9)
|
||||||
|
covered = event.get("covered_by_lod", resident)
|
||||||
|
near_desired = sum(desired[:3])
|
||||||
|
near_resident = sum(resident[:3])
|
||||||
|
far_desired = sum(desired[3:])
|
||||||
|
far_covered = sum(covered[3:])
|
||||||
|
t_s = float(event.get("t_s", 0.0))
|
||||||
|
speed = float(event.get("speed_cm_s", 0.0))
|
||||||
|
phase = "move" if speed >= 50.0 else ("dwell" if t_s >= stop else "arrival")
|
||||||
|
rows.append({
|
||||||
|
"phase": phase, "t_s": t_s,
|
||||||
|
"since_stop_s": max(0.0, t_s - stop) if phase == "dwell" else None,
|
||||||
|
"near_ready_pct": 100.0 * near_resident / max(1, near_desired),
|
||||||
|
"lod0_ready_pct": 100.0 * resident[0] / max(1, desired[0]),
|
||||||
|
"lod1_ready_pct": 100.0 * resident[1] / max(1, desired[1]),
|
||||||
|
"lod2_ready_pct": 100.0 * resident[2] / max(1, desired[2]),
|
||||||
|
"far_hole_pct": 100.0 * (far_desired - far_covered) / max(1, far_desired),
|
||||||
|
})
|
||||||
|
dwell_rows = [row for row in rows if row["phase"] == "dwell"]
|
||||||
|
complete = next((row for row in dwell_rows
|
||||||
|
if row["near_ready_pct"] >= 99.999
|
||||||
|
and row["far_hole_pct"] <= 0.001), None)
|
||||||
|
moving_work = [event for event in events
|
||||||
|
if event.get("type") == "game_thread_work_frame"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop]
|
||||||
|
ticks = [float(event.get("tick_ms", 0.0)) for event in moving_work]
|
||||||
|
updates = [float(event.get("update_chunks_ms", 0.0)) for event in moving_work]
|
||||||
|
backend = next((event for event in events
|
||||||
|
if event.get("type") == "worker_backend_config"), {})
|
||||||
|
worker_cap_readback = next((event for event in events
|
||||||
|
if event.get("type") == "streaming_worker_cap_readback"), {})
|
||||||
|
if result.returncode or len(moving_samples) < 40 or len(dwell_rows) < 16:
|
||||||
|
raise RuntimeError(f"Unreal run failed/incomplete: {trace}; exit={result.returncode}; "
|
||||||
|
f"move={len(moving_samples)} dwell={len(dwell_rows)}")
|
||||||
|
expected_far_supplement = 0 if condition == "baseline" else 1
|
||||||
|
expected_near_priority = "BackgroundLow"
|
||||||
|
if (backend.get("near_priority") != expected_near_priority
|
||||||
|
or backend.get("horizon_priority") != "BackgroundLow"
|
||||||
|
or int(worker_cap_readback.get("streaming_background_worker_cap", -1)) != worker_cap
|
||||||
|
or int(backend.get("streaming_far_apply_supplement", -1)) != expected_far_supplement
|
||||||
|
or int(backend.get("effective_tile_slots", -1)) != (8 if worker_cap else 10)):
|
||||||
|
raise RuntimeError(f"priority CVar readback mismatch in {trace}: {backend}")
|
||||||
|
return {
|
||||||
|
"condition": condition, "platform": "unreal", "speed_mps": 30.0,
|
||||||
|
"repetition": repetition, "core_limit": core_limit,
|
||||||
|
"effective_tile_slots": backend.get("effective_tile_slots"),
|
||||||
|
"streaming_background_worker_cap": worker_cap_readback.get(
|
||||||
|
"streaming_background_worker_cap"),
|
||||||
|
"taskgraph_background_threads": backend.get("taskgraph_background_threads"),
|
||||||
|
"exit_code": result.returncode,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"game_thread_tick_p99_ms": p99(ticks),
|
||||||
|
"game_thread_tick_max_ms": max(ticks, default=None),
|
||||||
|
"stream_update_p99_ms": p99(updates),
|
||||||
|
"stream_update_max_ms": max(updates, default=None),
|
||||||
|
"dwell_complete_after_stop_s": complete.get("since_stop_s") if complete else None,
|
||||||
|
"moving_samples": len(moving_samples), "dwell_samples": len(dwell_rows),
|
||||||
|
"trace": str(trace.relative_to(ROOT)),
|
||||||
|
"log": str(abslog.relative_to(ROOT)),
|
||||||
|
**summarize(rows, "unreal"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
global ACTIVE_DESIGN, OUT, HOST_SAVED, HOST_OUT
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--platform", choices=("all", "headless", "unreal"), default="all")
|
||||||
|
parser.add_argument("--design", choices=("far_apply_supplement",),
|
||||||
|
default="far_apply_supplement")
|
||||||
|
args = parser.parse_args()
|
||||||
|
ACTIVE_DESIGN = args.design
|
||||||
|
design_path = "far-apply-supplement"
|
||||||
|
OUT = ROOT / "Saved/Linux/l121/movement" / design_path
|
||||||
|
HOST_SAVED = HOST / "Saved/Linux/l121" / design_path
|
||||||
|
HOST_OUT = HOST / "Plugins/VoxelForge/Saved/Linux/l121" / design_path
|
||||||
|
if not DATA.is_file() or not HEADLESS.is_file() or not EDITOR.is_file():
|
||||||
|
raise SystemExit("missing owner data, L108 headless binary, or Unreal editor")
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
results = []
|
||||||
|
partial = OUT / "runs-partial.json"
|
||||||
|
for repetition in range(1, 6):
|
||||||
|
candidate = "far_supplement"
|
||||||
|
conditions = ("baseline", candidate) if repetition % 2 else (candidate, "baseline")
|
||||||
|
platforms = (("headless", "unreal") if repetition % 2 else ("unreal", "headless")) \
|
||||||
|
if args.platform == "all" else (args.platform,)
|
||||||
|
for condition in conditions:
|
||||||
|
for platform in platforms:
|
||||||
|
row = (run_headless(condition, repetition) if platform == "headless"
|
||||||
|
else run_unreal(condition, repetition))
|
||||||
|
results.append(row)
|
||||||
|
partial.write_text(json.dumps(results, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print("[L121Movement] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
design_description = "one supplemental far result after the normal four applies, with the 2 ms time budget"
|
||||||
|
(OUT / "summary.json").write_text(
|
||||||
|
json.dumps({"design": design_description,
|
||||||
|
"runs": results}, indent=2) + "\n", encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,198 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Summarize L122 scheduler blockers and Core region compile timing."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import statistics
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
TRACE_DIR = ROOT / "Saved/Linux/l122/diagnostic/unreal"
|
||||||
|
OUT = ROOT / "Saved/Linux/l122/diagnostic/analysis.json"
|
||||||
|
|
||||||
|
|
||||||
|
def read_events(path: Path) -> list[dict]:
|
||||||
|
rows = []
|
||||||
|
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||||
|
if line.startswith("{"):
|
||||||
|
try:
|
||||||
|
rows.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
continue
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def percentile(values: list[float], fraction: float) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
ordered = sorted(values)
|
||||||
|
index = max(0, min(len(ordered) - 1, math.ceil(len(ordered) * fraction) - 1))
|
||||||
|
return ordered[index]
|
||||||
|
|
||||||
|
|
||||||
|
def distribution(values: list[float]) -> dict:
|
||||||
|
if not values:
|
||||||
|
return {"n": 0, "p50": None, "p95": None, "max": None}
|
||||||
|
return {
|
||||||
|
"n": len(values), "p50": statistics.median(values),
|
||||||
|
"p95": percentile(values, 0.95), "max": max(values),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def summarize_run(path: Path) -> tuple[dict, list[dict], list[dict]]:
|
||||||
|
events = read_events(path)
|
||||||
|
stop = next(float(event["t_s"]) for event in events
|
||||||
|
if event.get("type") == "streaming_test_move_stopped")
|
||||||
|
move_start = min(float(event["t_s"]) for event in events
|
||||||
|
if event.get("type") == "streaming_sample"
|
||||||
|
and float(event.get("speed_cm_s", 0.0)) >= 50.0)
|
||||||
|
scheduler = [event for event in events if event.get("type") == "scheduler_admission"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop
|
||||||
|
and event.get("moving", 0)]
|
||||||
|
idle = [event for event in scheduler
|
||||||
|
if event.get("running_workers", 0) == 0
|
||||||
|
and sum(event.get("desired_missing_by_lod", [0] * 9)[:3])
|
||||||
|
+ sum(event.get("pending_by_lod", [0] * 9)[:3]) > 0]
|
||||||
|
samples = [event for event in events if event.get("type") == "streaming_sample"
|
||||||
|
and float(event.get("speed_cm_s", 0.0)) >= 50.0]
|
||||||
|
near = []
|
||||||
|
far_holes = []
|
||||||
|
for event in samples:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
resident = event.get("resident_by_lod", [0] * 9)
|
||||||
|
covered = event.get("covered_by_lod", resident)
|
||||||
|
near.append(100.0 * sum(resident[:3]) / max(1, sum(desired[:3])))
|
||||||
|
far_holes.append(100.0 * (sum(desired[3:]) - sum(covered[3:]))
|
||||||
|
/ max(1, sum(desired[3:])))
|
||||||
|
ticks = [float(event.get("tick_ms", 0.0)) for event in events
|
||||||
|
if event.get("type") == "game_thread_work_frame"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop]
|
||||||
|
|
||||||
|
def near_array(event: dict, name: str) -> list[int]:
|
||||||
|
return event.get(name, [0] * 9)[:3]
|
||||||
|
|
||||||
|
def near_count(event: dict, name: str) -> int:
|
||||||
|
return sum(near_array(event, name))
|
||||||
|
|
||||||
|
reason_counts = {
|
||||||
|
"core_region_pending": sum(event.get("core_region_compile_pending", 0) > 0
|
||||||
|
for event in idle),
|
||||||
|
"core_region_exact_near_decline": sum(
|
||||||
|
sum(near_array(event, "region_compile_declines_by_lod")) > 0 for event in idle),
|
||||||
|
"far_parent_near_decline": sum(
|
||||||
|
sum(near_array(event, "far_parent_declines_by_lod")) > 0 for event in idle),
|
||||||
|
"near_cap_decline": sum(any(value in (1, 2)
|
||||||
|
for value in event.get("declined_rule_by_lod", [0] * 9)[:3])
|
||||||
|
for event in idle),
|
||||||
|
"critical_set_not_ready": sum(event.get("critical_tiles_ready", 1) == 0
|
||||||
|
for event in idle),
|
||||||
|
"walkable_ring_not_ready": sum(event.get("walkable_ready", 1) == 0 for event in idle),
|
||||||
|
"pawn_collision_gate_held": sum(event.get("pawn_collision_gate_engaged", 0) > 0
|
||||||
|
for event in idle),
|
||||||
|
"desired_center_stale": sum(event.get("desired_center_current", 1) == 0 for event in idle),
|
||||||
|
"only_pending_near_work": sum(near_count(event, "desired_missing_by_lod") == 0
|
||||||
|
and near_count(event, "pending_by_lod") > 0 for event in idle),
|
||||||
|
"no_near_admission_this_frame": sum(
|
||||||
|
near_count(event, "submitted_by_lod") == 0 for event in idle),
|
||||||
|
}
|
||||||
|
|
||||||
|
compiles = [event for event in events if event.get("type") == "core_region_compile_complete"]
|
||||||
|
requests = {event.get("request_serial"): event for event in events
|
||||||
|
if event.get("type") == "core_region_compile_request"}
|
||||||
|
starts = [event for event in events if event.get("type") == "core_region_compile_start"]
|
||||||
|
request_start = [
|
||||||
|
(float(event["t_s"]) - float(requests[event["request_serial"]]["t_s"])) * 1000.0
|
||||||
|
for event in starts if event.get("request_serial") in requests
|
||||||
|
]
|
||||||
|
near_missing = [near_count(event, "desired_missing_by_lod") for event in idle]
|
||||||
|
near_pending = [near_count(event, "pending_by_lod") for event in idle]
|
||||||
|
occupancy = [float(event.get("admission_occupancy", 0)) for event in idle]
|
||||||
|
row = {
|
||||||
|
"trace": str(path.relative_to(ROOT)),
|
||||||
|
"moving_scheduler_samples": len(scheduler),
|
||||||
|
"idle_missing_near_windows": len(idle),
|
||||||
|
"idle_window_share_pct": 100.0 * len(idle) / max(1, len(scheduler)),
|
||||||
|
"near_ready_moving_mean_pct": statistics.mean(near) if near else None,
|
||||||
|
"far_holes_moving_mean_pct": statistics.mean(far_holes) if far_holes else None,
|
||||||
|
"tick_p99_ms": percentile(ticks, 0.99), "tick_max_ms": max(ticks, default=None),
|
||||||
|
"idle_near_unadmitted_mean": statistics.mean(near_missing) if near_missing else None,
|
||||||
|
"idle_near_pending_mean": statistics.mean(near_pending) if near_pending else None,
|
||||||
|
"idle_admission_occupancy_p95": percentile(occupancy, 0.95),
|
||||||
|
"idle_admission_occupancy_max": max(occupancy, default=None),
|
||||||
|
"idle_critical_missing_p50": statistics.median(
|
||||||
|
[float(event.get("critical_missing", 0)) for event in idle]) if idle else None,
|
||||||
|
"idle_reasons": reason_counts,
|
||||||
|
"region_compile_request_start_ms": distribution(request_start),
|
||||||
|
"region_compile_task_queue_ms": distribution(
|
||||||
|
[float(event.get("compile_queue_ms", 0)) for event in compiles]),
|
||||||
|
"region_compile_task_runtime_ms": distribution(
|
||||||
|
[float(event.get("compile_task_ms", 0)) for event in compiles]),
|
||||||
|
"region_compile_core_wall_ms": distribution(
|
||||||
|
[float(event.get("compile_core_wall_ms", 0)) for event in compiles]),
|
||||||
|
"region_compile_pending_gate_ms": distribution(
|
||||||
|
[float(event.get("pending_gate_elapsed_ms", 0)) for event in compiles]),
|
||||||
|
"region_compiled_tile_batches": sum(event.get("compiled_tiles", 0) > 0
|
||||||
|
for event in compiles),
|
||||||
|
"region_compile_worker_thread_count": len({event.get("compile_worker_thread_id")
|
||||||
|
for event in compiles}),
|
||||||
|
"region_compile_submitted_while_generation_idle": sum(
|
||||||
|
event.get("active_generation_workers", -1) == 0 for event in starts),
|
||||||
|
"region_compile_requests": len(requests),
|
||||||
|
"region_compile_starts": len(starts),
|
||||||
|
"region_compile_completions": len(compiles),
|
||||||
|
}
|
||||||
|
return row, idle, compiles
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
traces = sorted(TRACE_DIR.glob("fast-r*.trace.jsonl"))
|
||||||
|
if len(traces) != 5:
|
||||||
|
raise SystemExit(f"expected five diagnostic traces, found {len(traces)} in {TRACE_DIR}")
|
||||||
|
runs = [summarize_run(path) for path in traces]
|
||||||
|
rows = [run[0] for run in runs]
|
||||||
|
idles = [event for run in runs for event in run[1]]
|
||||||
|
compiles = [event for run in runs for event in run[2]]
|
||||||
|
total_scheduler = sum(row["moving_scheduler_samples"] for row in rows)
|
||||||
|
reason_names = rows[0]["idle_reasons"].keys()
|
||||||
|
summary = {
|
||||||
|
"runs": rows,
|
||||||
|
"pooled_idle_windows": len(idles),
|
||||||
|
"pooled_scheduler_samples": total_scheduler,
|
||||||
|
"pooled_idle_window_share_pct": 100.0 * len(idles) / max(1, total_scheduler),
|
||||||
|
"pooled_idle_reasons": {
|
||||||
|
reason: sum(row["idle_reasons"][reason] for row in rows)
|
||||||
|
for reason in reason_names
|
||||||
|
},
|
||||||
|
"pooled_near_ready_moving_mean_pct": statistics.mean(
|
||||||
|
row["near_ready_moving_mean_pct"] for row in rows),
|
||||||
|
"pooled_tick_p99_ms": distribution(
|
||||||
|
[float(row["tick_p99_ms"]) for row in rows if row["tick_p99_ms"] is not None]),
|
||||||
|
"pooled_tick_max_ms": max(float(row["tick_max_ms"]) for row in rows
|
||||||
|
if row["tick_max_ms"] is not None),
|
||||||
|
"pooled_region_compile_task_queue_ms": distribution(
|
||||||
|
[float(event.get("compile_queue_ms", 0)) for event in compiles]),
|
||||||
|
"pooled_region_compile_task_runtime_ms": distribution(
|
||||||
|
[float(event.get("compile_task_ms", 0)) for event in compiles]),
|
||||||
|
"pooled_region_compile_core_wall_ms": distribution(
|
||||||
|
[float(event.get("compile_core_wall_ms", 0)) for event in compiles]),
|
||||||
|
"pooled_region_compile_pending_gate_ms": distribution(
|
||||||
|
[float(event.get("pending_gate_elapsed_ms", 0)) for event in compiles]),
|
||||||
|
"pooled_region_compiled_tile_batches": sum(
|
||||||
|
event.get("compiled_tiles", 0) > 0 for event in compiles),
|
||||||
|
"pooled_region_compile_worker_thread_count": len({
|
||||||
|
event.get("compile_worker_thread_id") for event in compiles}),
|
||||||
|
"pooled_region_compile_submitted_while_generation_idle": sum(
|
||||||
|
event.get("active_generation_workers", -1) == 0
|
||||||
|
for run in runs for event in read_events(ROOT / run[0]["trace"])
|
||||||
|
if event.get("type") == "core_region_compile_start"),
|
||||||
|
"pooled_region_compile_requests": sum(row["region_compile_requests"] for row in rows),
|
||||||
|
}
|
||||||
|
OUT.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
OUT.write_text(json.dumps(summary, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print(json.dumps(summary, indent=2))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
# Linux 122 — idle workers at speed
|
||||||
|
|
||||||
|
**Decision:** keep the L121 admission policy. Neither tested fast-travel relaxation met every coverage and tick limit.
|
||||||
|
|
||||||
|
Five diagnostic runs at 30 m/s had 686 idle-worker windows with near tiles missing. In 673 (98%) the expanded critical set was not ready; 462 (67%) also had Core-region compilation pending, and 441 (64%) declined an exact region-dependent near tile. These counts overlap. Far-parent, near-cap, walkable-ring, pawn-collision, and stale-center holds were all zero. Only 13 windows had pending near work and no unadmitted near tile. The desired center was current in every window.
|
||||||
|
|
||||||
|
Region compile requests scheduled immediately (request to task submission p50/p95 **0.008/0.021 ms**). UE task queue delay was **0.122/14.856 ms** p50/p95 (max 54.4 ms); task runtime was **0.485/4.572 ms**, with Core compile wall time **0.259/3.679 ms**. The task runs on UE background workers, not the Game Thread. It was submitted while generation workers were idle in 680/1,158 requests. The compile targets the current desired center; there is no velocity look-ahead. The completion gate remained pending **18.216/21.136 ms** p50/p95 while the Game Thread observed completion.
|
||||||
|
|
||||||
|
The UE/headless coverage gap is not a denominator mismatch: both use resident near tiles / desired LOD0–2 tiles, with about 1,346–1,348 desired near tiles in UE versus 1,344 in the saved headless runs. Headless was about 82.6%; UE was 45.0%. The measured idle windows point to UE admission. Apply work is small (L121 near apply and proxy p95 were about 0.20 and 0.24 ms), though worker-finish-to-apply wait reached 120 ms p95 and may add secondary delay.
|
||||||
|
|
||||||
|
| Fast-travel design | Near change | Far-hole change | Paired p99 change | Worst tick |
|
||||||
|
|---|---:|---:|---:|---:|
|
||||||
|
| Admit normal near/far shares before expanded critical readiness | +15.42 pp | −0.28 pp | +0.28 ms | 14.52 ms |
|
||||||
|
| Admit near LOD0–2 only until critical readiness | +9.89 pp | +1.48 pp | −0.56 ms | 14.38 ms |
|
||||||
|
|
||||||
|
The first design exceeded the p99 bar (worst per-run p99 13.36 ms versus L121's 11.00 ms); the second increased far holes. Both stayed under L121's 14.74 ms worst-tick ceiling. No admission behavior was retained, so arrival behavior remains L121.
|
||||||
|
|
||||||
|
UE build passed with `-NoUBA -MaxParallelActions=4`. Exact tile and whole-view hashes matched L104 in **24/24** Clang/GCC W1/W8/W12 runs. Fast tier passed (**269 gates, 5 diagnostics**). Logs are under `Saved/Linux/l122`.
|
||||||
@@ -0,0 +1,111 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Capture five 30 m/s UE runs with admission-reason tracing enabled."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
HOST = Path("/home/jahni/VFHost")
|
||||||
|
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
|
||||||
|
OUT = ROOT / "Saved/Linux/l122/diagnostic/unreal"
|
||||||
|
HOST_OUT = HOST / "Plugins/VoxelForge/Saved/Linux/l122/diagnostic"
|
||||||
|
HOST_SAVED = HOST / "Saved/Linux/l122"
|
||||||
|
SPAWN_M = (19.166015625, -14.2001953125, -753.3740234375)
|
||||||
|
|
||||||
|
|
||||||
|
def run(repetition: int) -> dict:
|
||||||
|
trace = OUT / f"fast-r{repetition}.trace.jsonl"
|
||||||
|
abslog = OUT / f"fast-r{repetition}.log"
|
||||||
|
stdout = OUT / f"fast-r{repetition}.stdout.log"
|
||||||
|
export = HOST_OUT / f"fast-r{repetition}"
|
||||||
|
userdir = HOST_SAVED / "user" / f"fast-r{repetition}"
|
||||||
|
zendata = HOST_SAVED / "zen" / f"fast-r{repetition}"
|
||||||
|
ddc = HOST_SAVED / "DDC"
|
||||||
|
for path in (export, userdir, zendata, ddc):
|
||||||
|
path.mkdir(parents=True, exist_ok=True)
|
||||||
|
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWN_M)
|
||||||
|
command = [
|
||||||
|
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
|
||||||
|
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
|
||||||
|
"-notraceserver", "-nocrashreports", "-DisablePlugins=AndroidFileServer",
|
||||||
|
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
|
||||||
|
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
|
||||||
|
"-archetype=TunnelNetwork", "-slot=0", "-modes=export", "-gameconfig=1",
|
||||||
|
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
|
||||||
|
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0", "-coreplantest=0",
|
||||||
|
"-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings", "-corehybridmesher=0",
|
||||||
|
f"-out={export}", "-voxel.StreamingCoarseFirst=1",
|
||||||
|
"-voxel.StreamingDynamicNearReserve=1",
|
||||||
|
"-voxel.StreamingLegacyPendingResultCap=0",
|
||||||
|
"-voxel.CoreRegionConcurrentAdmission=1", "-voxel.StreamingResultBacklogCap=256",
|
||||||
|
"-voxel.StreamingReadyQueueDepth=8", "-voxel.StreamingBackgroundWorkerCap=1",
|
||||||
|
"-voxel.CoreColumnCacheDirectMapped=0", "-voxel.StreamingFarApplySupplement=1",
|
||||||
|
"-voxel.CoreMCGradientMemoization=1", "-voxel.StreamingCollisionLookahead=1",
|
||||||
|
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=3000.0",
|
||||||
|
"-voxel.TestMoveStartSeconds=6.0", "-voxel.TestMoveDurationSeconds=12.0",
|
||||||
|
"-voxel.TestExitSeconds=23.0", "-voxel.TestMoveDirection=1,0,0",
|
||||||
|
"-voxel.StreamingSchedulerPolicy=3", "-voxel.TestExitOnSteady=0",
|
||||||
|
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
|
||||||
|
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
|
||||||
|
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
|
||||||
|
f"-abslog={abslog}",
|
||||||
|
]
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
wrapped = ["taskset", "-c", "0-11", "timeout", "1740s", *command]
|
||||||
|
started = time.monotonic()
|
||||||
|
with stdout.open("w", encoding="utf-8") as stream:
|
||||||
|
result = subprocess.run(wrapped, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
events = []
|
||||||
|
if trace.is_file():
|
||||||
|
for line in trace.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||||
|
if line.startswith("{"):
|
||||||
|
try:
|
||||||
|
events.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
continue
|
||||||
|
moving = [event for event in events if event.get("type") == "streaming_sample"
|
||||||
|
and float(event.get("speed_cm_s", 0.0)) >= 50.0]
|
||||||
|
dwell = [event for event in events if event.get("type") == "streaming_sample"
|
||||||
|
and float(event.get("speed_cm_s", 0.0)) < 50.0
|
||||||
|
and float(event.get("t_s", 0.0)) >= 18.0]
|
||||||
|
backend = next((event for event in events if event.get("type") == "worker_backend_config"), {})
|
||||||
|
cap = next((event for event in events if event.get("type") == "streaming_worker_cap_readback"), {})
|
||||||
|
if result.returncode or len(moving) < 40 or len(dwell) < 16:
|
||||||
|
raise RuntimeError(f"incomplete Unreal run {repetition}: exit={result.returncode}, "
|
||||||
|
f"move={len(moving)}, dwell={len(dwell)}, trace={trace}")
|
||||||
|
if (int(backend.get("streaming_far_apply_supplement", -1)) != 1
|
||||||
|
or int(cap.get("streaming_background_worker_cap", -1)) != 1
|
||||||
|
or int(backend.get("effective_tile_slots", -1)) != 8):
|
||||||
|
raise RuntimeError(f"configuration readback mismatch: {backend}; {cap}")
|
||||||
|
if not any(event.get("type") == "scheduler_admission"
|
||||||
|
and "region_compile_declines_by_lod" in event
|
||||||
|
for event in events):
|
||||||
|
raise RuntimeError(f"admission diagnostics missing from {trace}")
|
||||||
|
row = {
|
||||||
|
"platform": "unreal", "phase": "diagnostic_baseline", "repetition": repetition,
|
||||||
|
"exit_code": result.returncode, "wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"moving_samples": len(moving), "dwell_samples": len(dwell),
|
||||||
|
"trace": str(trace.relative_to(ROOT)), "log": str(abslog.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
with (OUT / "runs.jsonl").open("a", encoding="utf-8") as stream:
|
||||||
|
stream.write(json.dumps(row, sort_keys=True) + "\n")
|
||||||
|
print("[L122Admission] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
if not (ROOT / "Standalone/data/owner_world.json").is_file() or not EDITOR.is_file():
|
||||||
|
raise SystemExit("missing owner world data or Unreal editor")
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
runs = [run(repetition) for repetition in range(1, 6)]
|
||||||
|
(OUT / "summary.json").write_text(json.dumps({"runs": runs}, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,117 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Compare L122 exact tile hashes with L104 across all strata, compilers and workers."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
OUT = ROOT / "Saved/Linux/l122/hashes"
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
OLD = ROOT / "Saved/Linux/l104/hashes"
|
||||||
|
BINARIES = {
|
||||||
|
"clang": ROOT / "Saved/Linux/l122/verify/bin/VoxelForgeCoreHeadless-verify-clang",
|
||||||
|
"gcc": ROOT / "Saved/Linux/l122/verify/bin/VoxelForgeCoreHeadless-verify-gcc",
|
||||||
|
}
|
||||||
|
CAMERAS = {
|
||||||
|
"sw": ((8.0, 16.0, -372.2626953125), 2850),
|
||||||
|
"fi": ((25.8369140625, 3.19921875, -737.0), 2249),
|
||||||
|
"tn": ((19.166015625, -14.2001953125, -753.3740234375), 2233),
|
||||||
|
"vs": ((-20.0625, -5.068359375, -900.353515625), 2329),
|
||||||
|
}
|
||||||
|
TILE = re.compile(
|
||||||
|
r"^\[PipelineTileHash\] lod=(-?\d+) x=(-?\d+) y=(-?\d+) z=(-?\d+) "
|
||||||
|
r"density=(0x[0-9a-f]+) mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) "
|
||||||
|
r"topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
WHOLE = re.compile(
|
||||||
|
r"^\[PipelineWholeViewHash\] tiles=(\d+) density=(0x[0-9a-f]+) "
|
||||||
|
r"mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M)
|
||||||
|
|
||||||
|
|
||||||
|
def parse(path: Path) -> tuple[dict, tuple[str, ...], str]:
|
||||||
|
content = path.read_text(encoding="utf-8", errors="replace")
|
||||||
|
tiles = {(int(m.group(1)), int(m.group(2)), int(m.group(3)), int(m.group(4))):
|
||||||
|
m.group(5, 6, 7, 8) for m in TILE.finditer(content)}
|
||||||
|
whole, stamp = WHOLE.search(content), STAMP.search(content)
|
||||||
|
if whole is None or stamp is None or len(tiles) != int(whole.group(1)):
|
||||||
|
raise RuntimeError(f"incomplete hash output: {path}")
|
||||||
|
return tiles, whole.groups(), stamp.group(1)
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, compiler: str, workers: int) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = OUT / f"{stratum}-{compiler}-w{workers}.log"
|
||||||
|
camera, expected_tiles = CAMERAS[stratum]
|
||||||
|
camera_arg = ",".join(f"{v:.12f}" for v in camera) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--", "timeout", "1740s",
|
||||||
|
str(BINARIES[compiler]), str(DATA), "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", f"--workers={workers}",
|
||||||
|
"--clip-radius=3", "--max-clip-level=8", "--max-level=8",
|
||||||
|
f"--camera={camera_arg}", "--stream-hash-tiles",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
if process.returncode:
|
||||||
|
raise RuntimeError(f"hash process exited {process.returncode}: {log}")
|
||||||
|
tiles, whole, stamp = parse(log)
|
||||||
|
if len(tiles) != expected_tiles:
|
||||||
|
raise RuntimeError(f"expected {expected_tiles} tiles, got {len(tiles)}: {log}")
|
||||||
|
old_log = OLD / f"{stratum}-{compiler}-w{workers}.log"
|
||||||
|
old_tiles, old_whole, _old_stamp = parse(old_log)
|
||||||
|
clang_reference, clang_whole, _ = parse(OLD / f"{stratum}-clang-w1.log")
|
||||||
|
|
||||||
|
def difference(reference: dict, candidate: dict) -> tuple[int, int]:
|
||||||
|
keys = set(reference) | set(candidate)
|
||||||
|
mismatches = sum(reference.get(key) != candidate.get(key) for key in keys)
|
||||||
|
return mismatches, len(keys)
|
||||||
|
|
||||||
|
old_mismatches, old_keys = difference(old_tiles, tiles)
|
||||||
|
cross_mismatches, cross_keys = difference(clang_reference, tiles)
|
||||||
|
row = {
|
||||||
|
"stratum": stratum, "compiler": compiler, "workers": workers,
|
||||||
|
"source_stamp": stamp, "tiles": len(tiles),
|
||||||
|
"l104_same_cell_tile_mismatches": old_mismatches,
|
||||||
|
"l104_same_cell_key_count": old_keys,
|
||||||
|
"l104_same_cell_whole_hash_match": whole == old_whole,
|
||||||
|
"clang_w1_tile_mismatches": cross_mismatches,
|
||||||
|
"clang_w1_key_count": cross_keys,
|
||||||
|
"clang_w1_whole_hash_match": whole == clang_whole,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L122HashRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
row["tile_hashes"] = {"|".join(map(str, key)): value for key, value in tiles.items()}
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
for path in (*BINARIES.values(), DATA):
|
||||||
|
if not path.is_file():
|
||||||
|
raise SystemExit(f"missing input: {path}")
|
||||||
|
rows = []
|
||||||
|
for stratum in CAMERAS:
|
||||||
|
for compiler in BINARIES:
|
||||||
|
for workers in (1, 8, 12):
|
||||||
|
rows.append(run(stratum, compiler, workers))
|
||||||
|
(OUT / "runs-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
failures = [row for row in rows if row["l104_same_cell_tile_mismatches"]
|
||||||
|
or not row["l104_same_cell_whole_hash_match"]
|
||||||
|
or row["clang_w1_tile_mismatches"]
|
||||||
|
or not row["clang_w1_whole_hash_match"]]
|
||||||
|
summary = {"runs": [{key: value for key, value in row.items() if key != "tile_hashes"}
|
||||||
|
for row in rows], "failed_runs": len(failures)}
|
||||||
|
(OUT / "summary.json").write_text(json.dumps(summary, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 1 if failures else 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,298 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Archived five-pair harness for transient L122 fast-travel admission candidates.
|
||||||
|
|
||||||
|
The candidate CVar branches were removed after neither design met the quality gate.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import statistics
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
HOST = Path("/home/jahni/VFHost")
|
||||||
|
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
|
||||||
|
SPAWN_TN_M = (19.166015625, -14.2001953125, -753.3740234375)
|
||||||
|
SPAWN_FI_M = (25.8369140625, 3.19921875, -737.0)
|
||||||
|
DESIGN = "relax_expanded_critical_gate"
|
||||||
|
NEAR_ONLY_DESIGN = "near_only_expanded_critical_gate"
|
||||||
|
|
||||||
|
|
||||||
|
def percentile(values: list[float], fraction: float) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
ordered = sorted(values)
|
||||||
|
return ordered[max(0, min(len(ordered) - 1, math.ceil(len(ordered) * fraction) - 1))]
|
||||||
|
|
||||||
|
|
||||||
|
def read_events(path: Path) -> list[dict]:
|
||||||
|
rows = []
|
||||||
|
if path.is_file():
|
||||||
|
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||||
|
if not line.startswith("{"):
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
rows.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
continue
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def run(scenario: str, condition: str, repetition: int, design: str) -> dict:
|
||||||
|
candidate = condition == design
|
||||||
|
wait_for_critical = 0 if candidate else 1
|
||||||
|
near_only_while_critical_pending = int(candidate and design == NEAR_ONLY_DESIGN)
|
||||||
|
if scenario == "movement":
|
||||||
|
speed_cm_s, start_s, duration_s, exit_s, slot = 3000.0, 6.0, 12.0, 23.0, 0
|
||||||
|
spawn = SPAWN_TN_M
|
||||||
|
else:
|
||||||
|
speed_cm_s, start_s, duration_s, exit_s, slot = 140.0, 20.0, 12.0, 37.0, 1
|
||||||
|
spawn = SPAWN_FI_M
|
||||||
|
|
||||||
|
output_name = scenario + ("-near-only" if design == NEAR_ONLY_DESIGN else "")
|
||||||
|
root_out = ROOT / f"Saved/Linux/l122/{output_name}"
|
||||||
|
out = root_out / condition / "unreal"
|
||||||
|
host_out = HOST / "Plugins/VoxelForge/Saved/Linux/l122" / output_name / condition
|
||||||
|
host_saved = HOST / "Saved/Linux/l122"
|
||||||
|
label = f"{scenario}-r{repetition}"
|
||||||
|
trace = out / f"{label}.trace.jsonl"
|
||||||
|
abslog = out / f"{label}.log"
|
||||||
|
stdout = out / f"{label}.stdout.log"
|
||||||
|
export = host_out / "output" / label
|
||||||
|
userdir = host_saved / "user" / scenario / condition / label
|
||||||
|
zendata = host_saved / "zen" / scenario / condition / label
|
||||||
|
ddc = host_saved / "DDC"
|
||||||
|
for directory in (out, export, userdir, zendata, ddc):
|
||||||
|
directory.mkdir(parents=True, exist_ok=True)
|
||||||
|
trace.unlink(missing_ok=True)
|
||||||
|
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in spawn)
|
||||||
|
command = [
|
||||||
|
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
|
||||||
|
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
|
||||||
|
"-notraceserver", "-nocrashreports", "-DisablePlugins=AndroidFileServer",
|
||||||
|
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
|
||||||
|
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
|
||||||
|
"-archetype=TunnelNetwork", f"-slot={slot}", "-modes=export", "-gameconfig=1",
|
||||||
|
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
|
||||||
|
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0", "-coreplantest=0",
|
||||||
|
"-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings", "-corehybridmesher=0",
|
||||||
|
f"-out={export}", "-voxel.StreamingCoarseFirst=1",
|
||||||
|
"-voxel.StreamingDynamicNearReserve=1",
|
||||||
|
"-voxel.StreamingLegacyPendingResultCap=0",
|
||||||
|
"-voxel.CoreRegionConcurrentAdmission=1", "-voxel.StreamingResultBacklogCap=256",
|
||||||
|
"-voxel.StreamingReadyQueueDepth=8", "-voxel.StreamingBackgroundWorkerCap=1",
|
||||||
|
"-voxel.CoreColumnCacheDirectMapped=0", "-voxel.StreamingFarApplySupplement=1",
|
||||||
|
f"-voxel.StreamingFastTravelWaitForAllCriticalTiles={wait_for_critical}",
|
||||||
|
f"-voxel.StreamingFastTravelNearOnlyWhileCriticalPending={near_only_while_critical_pending}",
|
||||||
|
"-voxel.CoreMCGradientMemoization=1", "-voxel.StreamingCollisionLookahead=1",
|
||||||
|
"-voxel.TestMove=1", f"-voxel.TestMoveSpeedCmPerSecond={speed_cm_s:.1f}",
|
||||||
|
f"-voxel.TestMoveStartSeconds={start_s:.1f}",
|
||||||
|
f"-voxel.TestMoveDurationSeconds={duration_s:.1f}",
|
||||||
|
f"-voxel.TestExitSeconds={exit_s:.1f}", "-voxel.TestMoveDirection=1,0,0",
|
||||||
|
"-voxel.StreamingSchedulerPolicy=3", "-voxel.TestExitOnSteady=0",
|
||||||
|
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
|
||||||
|
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
|
||||||
|
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
|
||||||
|
f"-abslog={abslog}",
|
||||||
|
]
|
||||||
|
wrapped = ["taskset", "-c", "0-11", "timeout", "1740s", *command]
|
||||||
|
started = time.monotonic()
|
||||||
|
with stdout.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(wrapped, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
events = read_events(trace)
|
||||||
|
samples = [event for event in events if event.get("type") == "streaming_sample"]
|
||||||
|
moving = [event for event in samples if float(event.get("speed_cm_s", 0.0)) >= 50.0]
|
||||||
|
stop = next((event for event in events if event.get("type") == "streaming_test_move_stopped"), {})
|
||||||
|
stop_s = float(stop.get("t_s", 0.0))
|
||||||
|
move_start = min((float(event.get("t_s", 0.0)) for event in moving), default=float("inf"))
|
||||||
|
dwell = [event for event in samples if float(event.get("t_s", 0.0)) >= stop_s
|
||||||
|
and float(event.get("speed_cm_s", 0.0)) < 50.0]
|
||||||
|
backend = next((event for event in events if event.get("type") == "worker_backend_config"), {})
|
||||||
|
cap = next((event for event in events if event.get("type") == "streaming_worker_cap_readback"), {})
|
||||||
|
if process.returncode or len(moving) < 40 or len(dwell) < 16:
|
||||||
|
raise RuntimeError(f"incomplete {scenario} run: {trace}; exit={process.returncode}; "
|
||||||
|
f"move={len(moving)} dwell={len(dwell)}")
|
||||||
|
if (int(backend.get("streaming_fast_travel_wait_for_all_critical_tiles", -1))
|
||||||
|
!= wait_for_critical
|
||||||
|
or int(backend.get("streaming_fast_travel_near_only_while_critical_pending", -1))
|
||||||
|
!= near_only_while_critical_pending
|
||||||
|
or int(backend.get("streaming_far_apply_supplement", -1)) != 1
|
||||||
|
or int(cap.get("streaming_background_worker_cap", -1)) != 1
|
||||||
|
or int(backend.get("effective_tile_slots", -1)) != 8):
|
||||||
|
raise RuntimeError(f"configuration readback mismatch: {backend}; {cap}")
|
||||||
|
|
||||||
|
moving_scheduler = [event for event in events if event.get("type") == "scheduler_admission"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop_s
|
||||||
|
and event.get("moving", 0)]
|
||||||
|
idle = [event for event in moving_scheduler
|
||||||
|
if event.get("running_workers", 0) == 0
|
||||||
|
and sum(event.get("desired_missing_by_lod", [0] * 9)[:3])
|
||||||
|
+ sum(event.get("pending_by_lod", [0] * 9)[:3]) > 0]
|
||||||
|
move_rows = []
|
||||||
|
for event in moving:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
resident = event.get("resident_by_lod", [0] * 9)
|
||||||
|
covered = event.get("covered_by_lod", resident)
|
||||||
|
move_rows.append({
|
||||||
|
"near_ready_pct": 100.0 * sum(resident[:3]) / max(1, sum(desired[:3])),
|
||||||
|
"lod0_ready_pct": 100.0 * resident[0] / max(1, desired[0]),
|
||||||
|
"lod1_ready_pct": 100.0 * resident[1] / max(1, desired[1]),
|
||||||
|
"lod2_ready_pct": 100.0 * resident[2] / max(1, desired[2]),
|
||||||
|
"far_hole_pct": 100.0 * (sum(desired[3:]) - sum(covered[3:]))
|
||||||
|
/ max(1, sum(desired[3:])),
|
||||||
|
})
|
||||||
|
work_frames = [event for event in events if event.get("type") == "game_thread_work_frame"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop_s]
|
||||||
|
tick_values = [float(event.get("tick_ms", 0.0)) for event in work_frames]
|
||||||
|
update_values = [float(event.get("update_chunks_ms", 0.0)) for event in work_frames]
|
||||||
|
requests = [event for event in events if event.get("type") == "request"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop_s
|
||||||
|
and event.get("level", 9) <= 2]
|
||||||
|
|
||||||
|
arrival_rows = [event for event in samples if float(event.get("t_s", 0.0)) < start_s
|
||||||
|
and float(event.get("speed_cm_s", 0.0)) < 50.0]
|
||||||
|
|
||||||
|
def near_complete(event: dict) -> bool:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
settled = event.get("settled_by_lod", [0] * 9)
|
||||||
|
return sum(desired[:3]) > 0 and sum(settled[:3]) >= sum(desired[:3])
|
||||||
|
|
||||||
|
def whole_complete(event: dict) -> bool:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
settled = event.get("settled_by_lod", [0] * 9)
|
||||||
|
return sum(desired) > 0 and sum(settled) >= sum(desired)
|
||||||
|
|
||||||
|
def far_holes(event: dict) -> float | None:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)[3:]
|
||||||
|
covered = event.get("covered_by_lod", [0] * 9)[3:]
|
||||||
|
return (100.0 * (sum(desired) - sum(covered)) / sum(desired)
|
||||||
|
if sum(desired) else None)
|
||||||
|
|
||||||
|
near_settled = next((event for event in arrival_rows if near_complete(event)), None)
|
||||||
|
whole_settled = next((event for event in arrival_rows if whole_complete(event)), None)
|
||||||
|
far_clear = next((event for event in arrival_rows if far_holes(event) == 0.0), None)
|
||||||
|
|
||||||
|
def mean_metric(name: str, rows: list[dict]) -> float | None:
|
||||||
|
values = [float(row[name]) for row in rows if row.get(name) is not None]
|
||||||
|
return statistics.mean(values) if values else None
|
||||||
|
|
||||||
|
row = {
|
||||||
|
"scenario": scenario, "condition": condition, "repetition": repetition,
|
||||||
|
"design": design, "speed_mps": speed_cm_s / 100.0,
|
||||||
|
"wait_for_all_critical_tiles": wait_for_critical,
|
||||||
|
"near_only_while_critical_pending": near_only_while_critical_pending,
|
||||||
|
"exit_code": process.returncode, "wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"moving_samples": len(moving), "dwell_samples": len(dwell),
|
||||||
|
"near_ready_moving_mean_pct": mean_metric("near_ready_pct", move_rows),
|
||||||
|
"lod0_ready_moving_mean_pct": mean_metric("lod0_ready_pct", move_rows),
|
||||||
|
"lod1_ready_moving_mean_pct": mean_metric("lod1_ready_pct", move_rows),
|
||||||
|
"lod2_ready_moving_mean_pct": mean_metric("lod2_ready_pct", move_rows),
|
||||||
|
"far_holes_moving_mean_pct": mean_metric("far_hole_pct", move_rows),
|
||||||
|
"game_thread_tick_p99_ms": percentile(tick_values, 0.99),
|
||||||
|
"game_thread_tick_max_ms": max(tick_values, default=None),
|
||||||
|
"stream_update_p99_ms": percentile(update_values, 0.99),
|
||||||
|
"stream_update_max_ms": max(update_values, default=None),
|
||||||
|
"near_requests_during_move": len(requests),
|
||||||
|
"arrival_sample_count": len(arrival_rows),
|
||||||
|
"lod02_settled_t_s": float(near_settled["t_s"]) if near_settled else None,
|
||||||
|
"whole_view_settled_t_s": float(whole_settled["t_s"]) if whole_settled else None,
|
||||||
|
"far_clear_t_s": float(far_clear["t_s"]) if far_clear else None,
|
||||||
|
"far_holes_at_pre_move_pct": far_holes(arrival_rows[-1]) if arrival_rows else None,
|
||||||
|
"idle_missing_near_windows": len(idle),
|
||||||
|
"idle_critical_gate_closed": sum(not event.get("critical_tiles_ready", 1)
|
||||||
|
for event in idle),
|
||||||
|
"idle_exact_region_declines": sum(
|
||||||
|
sum(event.get("region_compile_declines_by_lod", [0] * 9)[:3]) > 0
|
||||||
|
for event in idle),
|
||||||
|
"idle_far_parent_declines": sum(
|
||||||
|
sum(event.get("far_parent_declines_by_lod", [0] * 9)[:3]) > 0
|
||||||
|
for event in idle),
|
||||||
|
"idle_near_cap_declines": sum(
|
||||||
|
any(value in (1, 2) for value in event.get("declined_rule_by_lod", [0] * 9)[:3])
|
||||||
|
for event in idle),
|
||||||
|
"idle_desired_center_stale": sum(not event.get("desired_center_current", 1)
|
||||||
|
for event in idle),
|
||||||
|
"trace": str(trace.relative_to(ROOT)), "log": str(abslog.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L122Pair] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def pairs_for(runs: list[dict], scenario: str, design: str) -> list[dict]:
|
||||||
|
rows = []
|
||||||
|
for repetition in range(1, 6):
|
||||||
|
baseline = next(row for row in runs if row["scenario"] == scenario
|
||||||
|
and row["condition"] == "baseline" and row["repetition"] == repetition)
|
||||||
|
candidate = next(row for row in runs if row["scenario"] == scenario
|
||||||
|
and row["condition"] == design and row["repetition"] == repetition)
|
||||||
|
rows.append({
|
||||||
|
"repetition": repetition,
|
||||||
|
"near_ready_delta_pct": candidate.get("near_ready_moving_mean_pct", 0.0)
|
||||||
|
- baseline.get("near_ready_moving_mean_pct", 0.0),
|
||||||
|
"lod0_ready_delta_pct": candidate.get("lod0_ready_moving_mean_pct", 0.0)
|
||||||
|
- baseline.get("lod0_ready_moving_mean_pct", 0.0),
|
||||||
|
"lod1_ready_delta_pct": candidate.get("lod1_ready_moving_mean_pct", 0.0)
|
||||||
|
- baseline.get("lod1_ready_moving_mean_pct", 0.0),
|
||||||
|
"lod2_ready_delta_pct": candidate.get("lod2_ready_moving_mean_pct", 0.0)
|
||||||
|
- baseline.get("lod2_ready_moving_mean_pct", 0.0),
|
||||||
|
"far_holes_delta_pct": candidate.get("far_holes_moving_mean_pct", 0.0)
|
||||||
|
- baseline.get("far_holes_moving_mean_pct", 0.0),
|
||||||
|
"tick_p99_delta_ms": candidate.get("game_thread_tick_p99_ms", 0.0)
|
||||||
|
- baseline.get("game_thread_tick_p99_ms", 0.0),
|
||||||
|
"tick_max_delta_ms": candidate.get("game_thread_tick_max_ms", 0.0)
|
||||||
|
- baseline.get("game_thread_tick_max_ms", 0.0),
|
||||||
|
})
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--scenario", choices=("movement", "arrival"), required=True)
|
||||||
|
parser.add_argument("--design", choices=(DESIGN, NEAR_ONLY_DESIGN), default=DESIGN)
|
||||||
|
args = parser.parse_args()
|
||||||
|
if not EDITOR.is_file():
|
||||||
|
raise SystemExit(f"missing Unreal editor: {EDITOR}")
|
||||||
|
output_name = args.scenario + ("-near-only" if args.design == NEAR_ONLY_DESIGN else "")
|
||||||
|
out = ROOT / f"Saved/Linux/l122/{output_name}"
|
||||||
|
out.mkdir(parents=True, exist_ok=True)
|
||||||
|
runs = []
|
||||||
|
for repetition in range(1, 6):
|
||||||
|
order = ("baseline", args.design) if repetition % 2 else (args.design, "baseline")
|
||||||
|
for condition in order:
|
||||||
|
runs.append(run(args.scenario, condition, repetition, args.design))
|
||||||
|
(out / "runs-partial.json").write_text(json.dumps(runs, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
pairs = pairs_for(runs, args.scenario, args.design)
|
||||||
|
summary = {
|
||||||
|
"design_id": args.design,
|
||||||
|
"design": ("Allow fast-travel near LOD0-2 admission after the 3x3 walkable ring is ready "
|
||||||
|
"while other expanded critical tiles remain unresident; skip directional sort "
|
||||||
|
"and far admission until the critical set is ready."
|
||||||
|
if args.design == NEAR_ONLY_DESIGN else
|
||||||
|
"Allow fast-travel near/far admissions after the 3x3 walkable ring is ready, "
|
||||||
|
"even while other expanded critical tiles remain unresident; critical requests "
|
||||||
|
"still submit first."),
|
||||||
|
"scenario": args.scenario,
|
||||||
|
"runs": runs,
|
||||||
|
"pairs": pairs,
|
||||||
|
"paired_median_near_ready_delta_pct": statistics.median(
|
||||||
|
pair["near_ready_delta_pct"] for pair in pairs),
|
||||||
|
"paired_median_far_holes_delta_pct": statistics.median(
|
||||||
|
pair["far_holes_delta_pct"] for pair in pairs),
|
||||||
|
"paired_median_tick_p99_delta_ms": statistics.median(
|
||||||
|
pair["tick_p99_delta_ms"] for pair in pairs),
|
||||||
|
"paired_median_tick_max_delta_ms": statistics.median(
|
||||||
|
pair["tick_max_delta_ms"] for pair in pairs),
|
||||||
|
}
|
||||||
|
(out / "summary.json").write_text(json.dumps(summary, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,218 @@
|
|||||||
|
diff --git a/Source/VoxelForge/Private/VoxelWorld.cpp b/Source/VoxelForge/Private/VoxelWorld.cpp
|
||||||
|
index 42ac6f5..5c436ee 100644
|
||||||
|
--- a/Source/VoxelForge/Private/VoxelWorld.cpp
|
||||||
|
+++ b/Source/VoxelForge/Private/VoxelWorld.cpp
|
||||||
|
@@ -195,10 +195,13 @@ public:
|
||||||
|
std::atomic<bool> bShuttingDown{false};
|
||||||
|
std::atomic<uint64> NextEditSequence{1};
|
||||||
|
std::atomic<uint64> RegionCompileStartCycles{0};
|
||||||
|
+ std::atomic<uint64> RegionCompileWorkerStartCycles{0};
|
||||||
|
std::atomic<uint64> RegionCompileEndCycles{0};
|
||||||
|
std::atomic<int32> RegionCompileTileCount{0};
|
||||||
|
std::atomic<int32> RegionCompileSerial{0};
|
||||||
|
+ std::atomic<uint32> RegionCompileWorkerThreadId{0};
|
||||||
|
std::atomic<double> RegionCompileWorkerSeconds{0.0};
|
||||||
|
+ std::atomic<double> RegionCompileWallSeconds{0.0};
|
||||||
|
int32 FullResolutionLodCount = 0;
|
||||||
|
FString InitialFailureReason;
|
||||||
|
};
|
||||||
|
@@ -11381,20 +11384,25 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||||
|
1, std::memory_order_relaxed) + 1;
|
||||||
|
State->RegionCompileStartCycles.store(ActiveCoreRegionCompileStartCycles,
|
||||||
|
std::memory_order_release);
|
||||||
|
+ State->RegionCompileWorkerStartCycles.store(0, std::memory_order_release);
|
||||||
|
State->RegionCompileEndCycles.store(0, std::memory_order_release);
|
||||||
|
State->RegionCompileTileCount.store(0, std::memory_order_release);
|
||||||
|
+ State->RegionCompileWorkerThreadId.store(0, std::memory_order_release);
|
||||||
|
State->RegionCompileWorkerSeconds.store(0.0, std::memory_order_release);
|
||||||
|
+ State->RegionCompileWallSeconds.store(0.0, std::memory_order_release);
|
||||||
|
if (VoxelForgeStartupTrace::IsActive())
|
||||||
|
{
|
||||||
|
VoxelForgeStartupTrace::RecordEvent(TEXT("core_region_compile_start"), FString::Printf(
|
||||||
|
TEXT("\"compile_serial\":%d,\"request_serial\":%d,\"center\":[%d,%d,%d],"
|
||||||
|
"\"missing_region_tiles\":%d,\"retained_tiles\":%d,\"concurrent_admission\":%d,"
|
||||||
|
- "\"active_tile_tasks\":%d,\"active_task_guard\":%d"),
|
||||||
|
+ "\"active_tile_tasks\":%d,\"active_generation_workers\":%d,"
|
||||||
|
+ "\"active_task_guard\":%d,\"game_thread_id\":%u"),
|
||||||
|
CompileSerial, CoreRegionCompileRequestSerial,
|
||||||
|
CompileCenter.X, CompileCenter.Y, CompileCenter.Z,
|
||||||
|
MissingTiles.Num(), RetainedTiles.Num(), bAllowConcurrentGeneration ? 1 : 0,
|
||||||
|
ActiveTileTaskCount.load(std::memory_order_relaxed),
|
||||||
|
- ActiveTaskCount.load(std::memory_order_relaxed)));
|
||||||
|
+ ActiveGenerationWorkerCount.load(std::memory_order_relaxed),
|
||||||
|
+ ActiveTaskCount.load(std::memory_order_relaxed), FPlatformTLS::GetCurrentThreadId()));
|
||||||
|
}
|
||||||
|
ActiveTaskCount.fetch_add(1, std::memory_order_relaxed);
|
||||||
|
UE::Tasks::Launch(TEXT("VoxelForgeCoreTileRegion"),
|
||||||
|
@@ -11407,6 +11415,11 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||||
|
~FTaskGuard() { Counter.fetch_sub(1, std::memory_order_release); }
|
||||||
|
} Guard{ActiveTaskCount};
|
||||||
|
|
||||||
|
+ State->RegionCompileWorkerStartCycles.store(FPlatformTime::Cycles64(),
|
||||||
|
+ std::memory_order_release);
|
||||||
|
+ State->RegionCompileWorkerThreadId.store(FPlatformTLS::GetCurrentThreadId(),
|
||||||
|
+ std::memory_order_release);
|
||||||
|
+
|
||||||
|
if (bShuttingDown.load(std::memory_order_acquire)
|
||||||
|
|| State->bShuttingDown.load(std::memory_order_acquire))
|
||||||
|
{
|
||||||
|
@@ -11458,6 +11471,8 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||||
|
}
|
||||||
|
State->RegionCompileWorkerSeconds.store(CompileStats.workerSeconds,
|
||||||
|
std::memory_order_release);
|
||||||
|
+ State->RegionCompileWallSeconds.store(CompileStats.wallSeconds,
|
||||||
|
+ std::memory_order_release);
|
||||||
|
State->bRegionCompileSucceeded.store(bCompiled, std::memory_order_release);
|
||||||
|
if (!bCompiled)
|
||||||
|
{
|
||||||
|
@@ -12345,9 +12360,17 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
State->RegionCompileStartCycles.load(std::memory_order_acquire);
|
||||||
|
const uint64 CompileEndCycles =
|
||||||
|
State->RegionCompileEndCycles.load(std::memory_order_acquire);
|
||||||
|
+ const uint64 CompileWorkerStartCycles =
|
||||||
|
+ State->RegionCompileWorkerStartCycles.load(std::memory_order_acquire);
|
||||||
|
const double CompileElapsedMilliseconds = CompileEndCycles >= CompileStartCycles
|
||||||
|
? FPlatformTime::ToSeconds64(CompileEndCycles - CompileStartCycles) * 1000.0
|
||||||
|
: 0.0;
|
||||||
|
+ const double CompileQueueMilliseconds = CompileWorkerStartCycles >= CompileStartCycles
|
||||||
|
+ ? FPlatformTime::ToSeconds64(CompileWorkerStartCycles - CompileStartCycles) * 1000.0
|
||||||
|
+ : 0.0;
|
||||||
|
+ const double CompileTaskMilliseconds = CompileEndCycles >= CompileWorkerStartCycles
|
||||||
|
+ ? FPlatformTime::ToSeconds64(CompileEndCycles - CompileWorkerStartCycles) * 1000.0
|
||||||
|
+ : 0.0;
|
||||||
|
const double GateElapsedMilliseconds = CoreRegionPendingSinceSeconds > 0.0
|
||||||
|
? FMath::Max(0.0, FPlatformTime::Seconds() - CoreRegionPendingSinceSeconds) * 1000.0
|
||||||
|
: 0.0;
|
||||||
|
@@ -12355,6 +12378,10 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
State->RegionCompileTileCount.load(std::memory_order_acquire);
|
||||||
|
const double CompileWorkerSeconds =
|
||||||
|
State->RegionCompileWorkerSeconds.load(std::memory_order_acquire);
|
||||||
|
+ const double CompileCoreWallSeconds =
|
||||||
|
+ State->RegionCompileWallSeconds.load(std::memory_order_acquire);
|
||||||
|
+ const uint32 CompileWorkerThreadId =
|
||||||
|
+ State->RegionCompileWorkerThreadId.load(std::memory_order_acquire);
|
||||||
|
const int32 CompileSerial =
|
||||||
|
State->RegionCompileSerial.load(std::memory_order_acquire);
|
||||||
|
const uint64 IndependentSubmissions = CoreRegionIndependentSubmissions;
|
||||||
|
@@ -12364,14 +12391,20 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
VoxelForgeStartupTrace::RecordEvent(TEXT("core_region_compile_complete"), FString::Printf(
|
||||||
|
TEXT("\"compile_serial\":%d,\"request_serial\":%d,\"center\":[%d,%d,%d],"
|
||||||
|
"\"success\":%d,\"compiled_tiles\":%d,\"compile_elapsed_ms\":%s,"
|
||||||
|
- "\"compile_worker_s\":%s,\"pending_gate_elapsed_ms\":%s,"
|
||||||
|
+ "\"compile_queue_ms\":%s,\"compile_task_ms\":%s,"
|
||||||
|
+ "\"compile_core_wall_ms\":%s,\"compile_worker_s\":%s,"
|
||||||
|
+ "\"compile_worker_thread_id\":%u,\"pending_gate_elapsed_ms\":%s,"
|
||||||
|
"\"independent_submissions\":%llu,"
|
||||||
|
"\"dependent_declines\":%llu,\"concurrent_admission\":%d"),
|
||||||
|
CompileSerial, ActiveCoreRegionCompileRequestSerial,
|
||||||
|
ActiveCoreRegionCompileCenter.X, ActiveCoreRegionCompileCenter.Y,
|
||||||
|
ActiveCoreRegionCompileCenter.Z, bRegionCompileSucceeded ? 1 : 0,
|
||||||
|
CompiledTileCount, *FString::SanitizeFloat(CompileElapsedMilliseconds),
|
||||||
|
+ *FString::SanitizeFloat(CompileQueueMilliseconds),
|
||||||
|
+ *FString::SanitizeFloat(CompileTaskMilliseconds),
|
||||||
|
+ *FString::SanitizeFloat(CompileCoreWallSeconds * 1000.0),
|
||||||
|
*FString::SanitizeFloat(CompileWorkerSeconds),
|
||||||
|
+ CompileWorkerThreadId,
|
||||||
|
*FString::SanitizeFloat(GateElapsedMilliseconds),
|
||||||
|
static_cast<unsigned long long>(IndependentSubmissions),
|
||||||
|
static_cast<unsigned long long>(DependentDeclines),
|
||||||
|
@@ -12584,6 +12617,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
&& bWalkableRingReadyForReserve;
|
||||||
|
int32 SchedulerSubmittedByLevel[9] = {};
|
||||||
|
int32 SchedulerLoadAttemptsByLevel[9] = {};
|
||||||
|
+ int32 SchedulerRegionCompileDeclinesByLevel[9] = {};
|
||||||
|
+ int32 SchedulerFarParentDeclinesByLevel[9] = {};
|
||||||
|
int32 SchedulerLevelCapReturns[9] = {};
|
||||||
|
int32 SchedulerLevelCapBreaks[9] = {};
|
||||||
|
int32 SchedulerCriticalFullCapBreaks = 0;
|
||||||
|
@@ -12951,7 +12986,21 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
if(CoreState.IsValid()
|
||||||
|
&&CoreState->bFarWorldRequired.load(std::memory_order_acquire)
|
||||||
|
&&!CoreState->bFarWorldReady.load(std::memory_order_acquire))
|
||||||
|
+ {
|
||||||
|
+ if (bTraceStreamingScheduler && Tile.Level >= 0 && Tile.Level < 9)
|
||||||
|
+ {
|
||||||
|
+ ++SchedulerFarParentDeclinesByLevel[Tile.Level];
|
||||||
|
+ }
|
||||||
|
return;
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ if (bTraceStreamingScheduler && bUseVoxelForgeCore && bCoreRegionCompilePending
|
||||||
|
+ && GVoxelForgeCoreRegionConcurrentAdmission != 0
|
||||||
|
+ && (PendingCoreRegionTileSet.Contains(Tile)
|
||||||
|
+ || CoreRegionCompileTilesInFlight.Contains(Tile))
|
||||||
|
+ && Tile.Level >= 0 && Tile.Level < 9)
|
||||||
|
+ {
|
||||||
|
+ ++SchedulerRegionCompileDeclinesByLevel[Tile.Level];
|
||||||
|
}
|
||||||
|
if (Tile.Level >= 0 && Tile.Level < 9)
|
||||||
|
{
|
||||||
|
@@ -13661,10 +13710,37 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
const double BusySeconds = FPlatformTime::ToSeconds64(WorkerCyclesDelta);
|
||||||
|
const double SchedulerUpdateMilliseconds = (SchedulerTraceNow
|
||||||
|
- StreamingUpdateStartSeconds) * 1000.0;
|
||||||
|
+ int32 CriticalMissingTiles = 0;
|
||||||
|
+ for (const FVoxelTileKey& Tile : CriticalDesiredTiles)
|
||||||
|
+ {
|
||||||
|
+ if (!LoadedTiles.Contains(Tile) && !CollisionNotRequiredTiles.Contains(Tile))
|
||||||
|
+ {
|
||||||
|
+ ++CriticalMissingTiles;
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ int32 CollisionGateMissingTiles = 0;
|
||||||
|
+ for (const FVoxelTileKey& Tile : CollisionGateDesiredTiles)
|
||||||
|
+ {
|
||||||
|
+ if (!LoadedTiles.Contains(Tile) && !CollisionNotRequiredTiles.Contains(Tile))
|
||||||
|
+ {
|
||||||
|
+ ++CollisionGateMissingTiles;
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ const FIntVector DesiredCenterLag = DesiredEpochCenterChunk - CenterChunk;
|
||||||
|
+ const TSharedPtr<FVoxelForgeCoreRuntimeState, ESPMode::ThreadSafe> TraceCoreState =
|
||||||
|
+ VoxelForgeCoreState;
|
||||||
|
+ const bool bFarWorldRequiredForTrace = TraceCoreState.IsValid()
|
||||||
|
+ && TraceCoreState->bFarWorldRequired.load(std::memory_order_relaxed);
|
||||||
|
+ const bool bFarWorldReadyForTrace = TraceCoreState.IsValid()
|
||||||
|
+ && TraceCoreState->bFarWorldReady.load(std::memory_order_relaxed);
|
||||||
|
VoxelForgeStartupTrace::RecordEvent(TEXT("scheduler_admission"), FString::Printf(
|
||||||
|
TEXT("\"frame\":%llu,\"center\":[%d,%d,%d],\"desired_rebuilt\":%d,"
|
||||||
|
"\"submission_loop_ran\":%d,\"all_chunks_loaded\":%d,"
|
||||||
|
"\"walkable_ready\":%d,\"moving\":%d,"
|
||||||
|
+ "\"pawn_collision_gate_engaged\":%d,\"critical_tiles_ready\":%d,"
|
||||||
|
+ "\"critical_missing\":%d,\"collision_gate_missing\":%d,"
|
||||||
|
+ "\"desired_center_lag\":[%d,%d,%d],\"desired_center_current\":%d,"
|
||||||
|
+ "\"far_world_required\":%d,\"far_world_ready\":%d,"
|
||||||
|
"\"core_region_compile_pending\":%d,\"core_ready\":%d,\"coarse_first\":%d,"
|
||||||
|
"\"dynamic_near_reserve\":%d,\"legacy_pending_result_cap\":%d,"
|
||||||
|
"\"core_region_concurrent_admission\":%d,"
|
||||||
|
@@ -13685,6 +13761,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
"\"level_candidate_scan_ms\":%s,\"passage_score_ms\":%s,"
|
||||||
|
"\"level_sort_ms\":%s,\"level_admit_ms\":%s,"
|
||||||
|
"\"desired_missing_by_lod\":%s,\"loaded_by_lod\":%s,\"pending_by_lod\":%s,"
|
||||||
|
+ "\"region_compile_declines_by_lod\":%s,"
|
||||||
|
+ "\"far_parent_declines_by_lod\":%s,"
|
||||||
|
"\"submitted_by_lod\":%s,\"load_attempts_by_lod\":%s,"
|
||||||
|
"\"admission_cap_by_lod\":%s,\"declined_rule_by_lod\":%s,"
|
||||||
|
"\"level_cap_returns_by_lod\":%s,\"level_cap_breaks_by_lod\":%s,"
|
||||||
|
@@ -13695,6 +13773,11 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
static_cast<unsigned long long>(GFrameCounter), CenterChunk.X, CenterChunk.Y, CenterChunk.Z,
|
||||||
|
bDesiredRebuiltThisUpdate ? 1 : 0, bSubmissionLoopRan ? 1 : 0, bAllChunksLoaded ? 1 : 0,
|
||||||
|
bWalkableRingReadyForReserve ? 1 : 0, bPawnMoving ? 1 : 0,
|
||||||
|
+ bPawnCollisionGateEngaged ? 1 : 0, CriticalMissingTiles == 0 ? 1 : 0,
|
||||||
|
+ CriticalMissingTiles, CollisionGateMissingTiles,
|
||||||
|
+ DesiredCenterLag.X, DesiredCenterLag.Y, DesiredCenterLag.Z,
|
||||||
|
+ DesiredCenterLag == FIntVector::ZeroValue ? 1 : 0,
|
||||||
|
+ bFarWorldRequiredForTrace ? 1 : 0, bFarWorldReadyForTrace ? 1 : 0,
|
||||||
|
bCoreRegionCompilePending ? 1 : 0, IsVoxelForgeCoreReady() ? 1 : 0,
|
||||||
|
GVoxelForgeStreamingCoarseFirst, GVoxelForgeStreamingDynamicNearReserve,
|
||||||
|
GVoxelForgeStreamingLegacyPendingResultCap,
|
||||||
|
@@ -13721,7 +13804,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
*FString::SanitizeFloat(LevelSortMilliseconds),
|
||||||
|
*FString::SanitizeFloat(LevelAdmissionMilliseconds),
|
||||||
|
*IntArrayJson(DesiredMissingByLevel), *IntArrayJson(LoadedByLevel),
|
||||||
|
- *IntArrayJson(PendingByLevel), *IntArrayJson(SchedulerSubmittedByLevel),
|
||||||
|
+ *IntArrayJson(PendingByLevel), *IntArrayJson(SchedulerRegionCompileDeclinesByLevel),
|
||||||
|
+ *IntArrayJson(SchedulerFarParentDeclinesByLevel), *IntArrayJson(SchedulerSubmittedByLevel),
|
||||||
|
*IntArrayJson(SchedulerLoadAttemptsByLevel), *IntArrayJson(AdmissionCapByLevel),
|
||||||
|
*IntArrayJson(DeclinedRuleByLevel), *IntArrayJson(SchedulerLevelCapReturns),
|
||||||
|
*IntArrayJson(SchedulerLevelCapBreaks), FullCapDeclinedTiles, ReserveDeclinedTiles,
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
# L123: region completion gate
|
||||||
|
|
||||||
|
Five interleaved 30 m/s UE pairs show the gate is a game-thread poll delay. Core compile wall time is p50 0.251 ms (worker queue plus task p50 0.475 ms); request-to-observe is p50 18.89 ms, p95 21.68 ms. In 95% of traced completions, the next scheduler pass drains the MPSC result and resumes admission in that same pass; a small tail waits 2–4 scheduler frames. This is one tick interval, not separate compile, observe, and admit ticks.
|
||||||
|
|
||||||
|
The worker can safely hand off an immutable result through MPSC with its runtime state and `GenerationEpoch`. The game thread validates both before updating state and admitting tiles. Direct worker-side admission would touch game-thread state unsafely. This queue handoff preserves ownership but still polls once per tick; a game-thread wakeup/continuation is the safe way to try removing that delay. A wakeup was not timed in this round.
|
||||||
|
|
||||||
|
One-step velocity lookahead did not remove the wait (gate p50 19.0 ms). Paired near readiness fell 0.79 percentage points; paired far holes rose 0.14 points. Its worst-run p99 was 11.77 ms (limit 11.00) and worst tick 15.11 ms (limit 14.74). After a 90° turn, 252 of 960 retained ahead keys became stale. At 1.4 m/s it compiled 128 ahead tiles per run; paired near arrival was 2.6 ms faster, while far-clear arrival was 14 ms slower. Keep lookahead off.
|
||||||
|
|
||||||
|
The queue-only runs also missed the p99 limit (worst 11.46 ms), though their worst tick was 14.19 ms. Neither measured option meets the full bar. Headless 30 m/s completed with zero generation failures (82.71% near readiness, 14.21% far holes); this path has no UE region-completion gate. Geometry checks passed 24/24 for Clang/GCC W1/W8/W12, and the fast tier passed.
|
||||||
@@ -0,0 +1,115 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Compare L123 exact tile hashes with L104 across strata, compilers, and worker counts."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
OUT = ROOT / "Saved/Linux/l123/hashes"
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
OLD = ROOT / "Saved/Linux/l104/hashes"
|
||||||
|
BINARIES = {
|
||||||
|
"clang": ROOT / "Saved/Linux/l123/bin/VoxelForgeCoreHeadless-l123-clang",
|
||||||
|
"gcc": ROOT / "Saved/Linux/l123/bin/VoxelForgeCoreHeadless-l123-gcc",
|
||||||
|
}
|
||||||
|
CAMERAS = {
|
||||||
|
"sw": ((8.0, 16.0, -372.2626953125), 2850),
|
||||||
|
"fi": ((25.8369140625, 3.19921875, -737.0), 2249),
|
||||||
|
"tn": ((19.166015625, -14.2001953125, -753.3740234375), 2233),
|
||||||
|
"vs": ((-20.0625, -5.068359375, -900.353515625), 2329),
|
||||||
|
}
|
||||||
|
TILE = re.compile(
|
||||||
|
r"^\[PipelineTileHash\] lod=(-?\d+) x=(-?\d+) y=(-?\d+) z=(-?\d+) "
|
||||||
|
r"density=(0x[0-9a-f]+) mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) "
|
||||||
|
r"topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
WHOLE = re.compile(
|
||||||
|
r"^\[PipelineWholeViewHash\] tiles=(\d+) density=(0x[0-9a-f]+) "
|
||||||
|
r"mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M)
|
||||||
|
|
||||||
|
|
||||||
|
def parse(path: Path) -> tuple[dict, tuple[str, ...], str]:
|
||||||
|
content = path.read_text(encoding="utf-8", errors="replace")
|
||||||
|
tiles = {(int(m.group(1)), int(m.group(2)), int(m.group(3)), int(m.group(4))):
|
||||||
|
m.group(5, 6, 7, 8) for m in TILE.finditer(content)}
|
||||||
|
whole, stamp = WHOLE.search(content), STAMP.search(content)
|
||||||
|
if whole is None or stamp is None or len(tiles) != int(whole.group(1)):
|
||||||
|
raise RuntimeError(f"incomplete hash output: {path}")
|
||||||
|
return tiles, whole.groups(), stamp.group(1)
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, compiler: str, workers: int) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = OUT / f"{stratum}-{compiler}-w{workers}.log"
|
||||||
|
camera, expected_tiles = CAMERAS[stratum]
|
||||||
|
camera_arg = ",".join(f"{v:.12f}" for v in camera) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", str(BINARIES[compiler]), str(DATA),
|
||||||
|
"--world-assembler-prototype", "--stream-benchmark", "--stream-pipeline",
|
||||||
|
f"--workers={workers}", "--clip-radius=3", "--max-clip-level=8",
|
||||||
|
"--max-level=8", f"--camera={camera_arg}", "--stream-hash-tiles",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
if process.returncode:
|
||||||
|
raise RuntimeError(f"hash process exited {process.returncode}: {log}")
|
||||||
|
tiles, whole, stamp = parse(log)
|
||||||
|
if len(tiles) != expected_tiles:
|
||||||
|
raise RuntimeError(f"expected {expected_tiles} tiles, got {len(tiles)}: {log}")
|
||||||
|
old_tiles, old_whole, _ = parse(OLD / f"{stratum}-{compiler}-w{workers}.log")
|
||||||
|
clang_tiles, clang_whole, _ = parse(OLD / f"{stratum}-clang-w1.log")
|
||||||
|
|
||||||
|
def difference(reference: dict, candidate: dict) -> tuple[int, int]:
|
||||||
|
keys = set(reference) | set(candidate)
|
||||||
|
return sum(reference.get(key) != candidate.get(key) for key in keys), len(keys)
|
||||||
|
|
||||||
|
old_mismatches, old_keys = difference(old_tiles, tiles)
|
||||||
|
cross_mismatches, cross_keys = difference(clang_tiles, tiles)
|
||||||
|
row = {
|
||||||
|
"stratum": stratum, "compiler": compiler, "workers": workers,
|
||||||
|
"source_stamp": stamp, "tiles": len(tiles),
|
||||||
|
"l104_same_cell_tile_mismatches": old_mismatches,
|
||||||
|
"l104_same_cell_key_count": old_keys,
|
||||||
|
"l104_same_cell_whole_hash_match": whole == old_whole,
|
||||||
|
"clang_w1_tile_mismatches": cross_mismatches,
|
||||||
|
"clang_w1_key_count": cross_keys,
|
||||||
|
"clang_w1_whole_hash_match": whole == clang_whole,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L123HashRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
row["tile_hashes"] = {"|".join(map(str, key)): value for key, value in tiles.items()}
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
for path in (*BINARIES.values(), DATA):
|
||||||
|
if not path.is_file():
|
||||||
|
raise SystemExit(f"missing input: {path}")
|
||||||
|
rows = []
|
||||||
|
for stratum in CAMERAS:
|
||||||
|
for compiler in BINARIES:
|
||||||
|
for workers in (1, 8, 12):
|
||||||
|
rows.append(run(stratum, compiler, workers))
|
||||||
|
(OUT / "runs-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
failures = [row for row in rows if row["l104_same_cell_tile_mismatches"]
|
||||||
|
or not row["l104_same_cell_whole_hash_match"]
|
||||||
|
or row["clang_w1_tile_mismatches"]
|
||||||
|
or not row["clang_w1_whole_hash_match"]]
|
||||||
|
summary = {"runs": [{key: value for key, value in row.items() if key != "tile_hashes"}
|
||||||
|
for row in rows], "failed_runs": len(failures)}
|
||||||
|
(OUT / "summary.json").write_text(json.dumps(summary, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 1 if failures else 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""One 30 m/s headless movement run for the UE-only region gate experiment."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
BIN = ROOT / "Saved/Linux/l108/bin/VoxelForgeCoreHeadless-l108-after-clang"
|
||||||
|
OUT = ROOT / "Saved/Linux/l123/headless"
|
||||||
|
SAMPLE = re.compile(r"^\[L79MovingSample\] (.*)$", re.M)
|
||||||
|
SUMMARY = re.compile(r"^\[L79MovingSummary\] (.*)$", re.M)
|
||||||
|
FIELDS = re.compile(r"([a-zA-Z0-9_]+)=([^ ]+)")
|
||||||
|
|
||||||
|
|
||||||
|
def p99(values: list[float]) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
values.sort()
|
||||||
|
return values[max(0, math.ceil(0.99 * len(values)) - 1)]
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = OUT / "movement-30mps-r1.log"
|
||||||
|
camera = "19.166015625000,-14.200195312500,-753.374023437500,0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", str(BIN), str(DATA), "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", "--workers=8", "--clip-radius=3",
|
||||||
|
"--max-clip-level=8", "--max-level=8", f"--camera={camera}",
|
||||||
|
"--profile-lod-work", "--stream-scheduler-policy=3",
|
||||||
|
"--moving-camera-speed-mps=30.0", "--moving-camera-duration-s=12",
|
||||||
|
"--moving-camera-dwell-s=5",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
content = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
rows = []
|
||||||
|
for match in SAMPLE.finditer(content):
|
||||||
|
row = dict(FIELDS.findall(match.group(1)))
|
||||||
|
for name in ("near_ready_pct", "far_hole_pct", "queue_update_ms"):
|
||||||
|
if name in row:
|
||||||
|
row[name] = float(row[name])
|
||||||
|
row["phase"] = row.get("phase", "move")
|
||||||
|
rows.append(row)
|
||||||
|
summary = dict(FIELDS.findall(SUMMARY.search(content).group(1))) if SUMMARY.search(content) else {}
|
||||||
|
moving = [row for row in rows if row["phase"] == "move"]
|
||||||
|
updates = [row["queue_update_ms"] for row in moving if "queue_update_ms" in row]
|
||||||
|
result = {
|
||||||
|
"exit_code": process.returncode,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"moving_samples": len(moving),
|
||||||
|
"near_ready_moving_mean_pct": sum(r["near_ready_pct"] for r in moving) / max(1, len(moving)),
|
||||||
|
"far_holes_moving_mean_pct": sum(r["far_hole_pct"] for r in moving) / max(1, len(moving)),
|
||||||
|
"update_work_p99_ms": p99(updates),
|
||||||
|
"update_work_max_ms": max(updates, default=None),
|
||||||
|
"generation_failures": int(summary.get("generation_failures", -1)),
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
(OUT / "summary.json").write_text(json.dumps(result, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
print("[L123Headless] " + json.dumps(result, sort_keys=True), flush=True)
|
||||||
|
if process.returncode or not moving or result["generation_failures"] != 0:
|
||||||
|
return 1
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,262 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Five interleaved 30 m/s UE pairs for the Core region lookahead experiment."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import statistics
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
HOST = Path("/home/jahni/VFHost")
|
||||||
|
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
|
||||||
|
SPAWN_TN_M = (19.166015625, -14.2001953125, -753.3740234375)
|
||||||
|
SPAWN_FI_M = (25.8369140625, 3.19921875, -737.0)
|
||||||
|
|
||||||
|
|
||||||
|
def percentile(values: list[float], fraction: float) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
ordered = sorted(values)
|
||||||
|
return ordered[max(0, min(len(ordered) - 1, math.ceil(len(ordered) * fraction) - 1))]
|
||||||
|
|
||||||
|
|
||||||
|
def read_events(path: Path) -> list[dict]:
|
||||||
|
rows = []
|
||||||
|
if path.is_file():
|
||||||
|
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||||
|
if not line.startswith("{"):
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
rows.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def run(scenario: str, condition: str, repetition: int, turn: bool = False) -> dict:
|
||||||
|
lookahead = int(condition in ("lookahead", "turn_lookahead"))
|
||||||
|
if scenario == "movement":
|
||||||
|
speed, start, duration, exit_s, slot, spawn = (
|
||||||
|
3000.0, 6.0, 12.0, 23.0, 0, SPAWN_TN_M)
|
||||||
|
else:
|
||||||
|
speed, start, duration, exit_s, slot, spawn = (
|
||||||
|
140.0, 20.0, 12.0, 37.0, 1, SPAWN_FI_M)
|
||||||
|
|
||||||
|
label = f"{scenario}-{condition}-r{repetition}"
|
||||||
|
output_group = "turn" if turn else scenario
|
||||||
|
out = ROOT / "Saved/Linux/l123" / output_group / condition / "unreal"
|
||||||
|
host_out = HOST / "Plugins/VoxelForge/Saved/Linux/l123" / output_group / condition
|
||||||
|
host_saved = HOST / "Saved/Linux/l123"
|
||||||
|
trace = out / f"{label}.trace.jsonl"
|
||||||
|
abslog = out / f"{label}.log"
|
||||||
|
stdout = out / f"{label}.stdout.log"
|
||||||
|
export = host_out / "output" / label
|
||||||
|
userdir = host_saved / "user" / label
|
||||||
|
zendata = host_saved / "zen" / label
|
||||||
|
ddc = host_saved / "DDC"
|
||||||
|
for path in (out, export, userdir, zendata, ddc):
|
||||||
|
path.mkdir(parents=True, exist_ok=True)
|
||||||
|
trace.unlink(missing_ok=True)
|
||||||
|
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in spawn)
|
||||||
|
command = [
|
||||||
|
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
|
||||||
|
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
|
||||||
|
"-notraceserver", "-nocrashreports", "-DisablePlugins=AndroidFileServer",
|
||||||
|
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
|
||||||
|
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
|
||||||
|
"-archetype=TunnelNetwork", f"-slot={slot}", "-modes=export", "-gameconfig=1",
|
||||||
|
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
|
||||||
|
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0", "-coreplantest=0",
|
||||||
|
"-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings", "-corehybridmesher=0",
|
||||||
|
f"-out={export}", "-voxel.StreamingCoarseFirst=1",
|
||||||
|
"-voxel.StreamingDynamicNearReserve=1", "-voxel.StreamingLegacyPendingResultCap=0",
|
||||||
|
"-voxel.CoreRegionConcurrentAdmission=1", "-voxel.StreamingResultBacklogCap=256",
|
||||||
|
"-voxel.StreamingReadyQueueDepth=8", "-voxel.StreamingBackgroundWorkerCap=1",
|
||||||
|
"-voxel.CoreColumnCacheDirectMapped=0", "-voxel.StreamingFarApplySupplement=1",
|
||||||
|
"-voxel.CoreMCGradientMemoization=1", "-voxel.StreamingCollisionLookahead=1",
|
||||||
|
f"-voxel.CoreRegionLookahead={lookahead}", "-voxel.TestMove=1",
|
||||||
|
f"-voxel.TestMoveSpeedCmPerSecond={speed:.1f}",
|
||||||
|
f"-voxel.TestMoveStartSeconds={start:.1f}",
|
||||||
|
f"-voxel.TestMoveDurationSeconds={duration:.1f}",
|
||||||
|
f"-voxel.TestExitSeconds={exit_s:.1f}", "-voxel.TestMoveDirection=1,0,0",
|
||||||
|
"-voxel.StreamingSchedulerPolicy=3", "-voxel.TestExitOnSteady=0",
|
||||||
|
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
|
||||||
|
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
|
||||||
|
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
|
||||||
|
f"-abslog={abslog}",
|
||||||
|
]
|
||||||
|
if turn:
|
||||||
|
command.extend(["-voxel.TestMoveTurnAfterSeconds=6.0",
|
||||||
|
"-voxel.TestMoveTurnDirection=0,1,0"])
|
||||||
|
started = time.monotonic()
|
||||||
|
wrapped = ["taskset", "-c", "0-11", "timeout", "1740s", *command]
|
||||||
|
with stdout.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(wrapped, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
events = read_events(trace)
|
||||||
|
samples = [event for event in events if event.get("type") == "streaming_sample"]
|
||||||
|
moving = [event for event in samples if float(event.get("speed_cm_s", 0.0)) >= 50.0]
|
||||||
|
stop_event = next((event for event in events
|
||||||
|
if event.get("type") == "streaming_test_move_stopped"), {})
|
||||||
|
stop_s = float(stop_event.get("t_s", 0.0))
|
||||||
|
move_start = min((float(event.get("t_s", 0.0)) for event in moving), default=float("inf"))
|
||||||
|
dwell = [event for event in samples if float(event.get("t_s", 0.0)) >= stop_s
|
||||||
|
and float(event.get("speed_cm_s", 0.0)) < 50.0]
|
||||||
|
backend = next((event for event in events if event.get("type") == "worker_backend_config"), {})
|
||||||
|
cap = next((event for event in events if event.get("type") == "streaming_worker_cap_readback"), {})
|
||||||
|
if process.returncode or len(moving) < 40 or len(dwell) < 16:
|
||||||
|
raise RuntimeError(f"incomplete UE run {label}: exit={process.returncode}, "
|
||||||
|
f"move={len(moving)}, dwell={len(dwell)}, trace={trace}")
|
||||||
|
if (int(backend.get("core_region_lookahead", -1)) != lookahead
|
||||||
|
or int(backend.get("streaming_far_apply_supplement", -1)) != 1
|
||||||
|
or int(cap.get("streaming_background_worker_cap", -1)) != 1
|
||||||
|
or int(backend.get("effective_tile_slots", -1)) != 8):
|
||||||
|
raise RuntimeError(f"configuration readback mismatch: {backend}; {cap}")
|
||||||
|
|
||||||
|
moving_rows = []
|
||||||
|
for event in moving:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
resident = event.get("resident_by_lod", [0] * 9)
|
||||||
|
covered = event.get("covered_by_lod", resident)
|
||||||
|
moving_rows.append({
|
||||||
|
"near_ready_pct": 100.0 * sum(resident[:3]) / max(1, sum(desired[:3])),
|
||||||
|
"lod0_ready_pct": 100.0 * resident[0] / max(1, desired[0]),
|
||||||
|
"lod1_ready_pct": 100.0 * resident[1] / max(1, desired[1]),
|
||||||
|
"lod2_ready_pct": 100.0 * resident[2] / max(1, desired[2]),
|
||||||
|
"far_hole_pct": 100.0 * (sum(desired[3:]) - sum(covered[3:]))
|
||||||
|
/ max(1, sum(desired[3:])),
|
||||||
|
})
|
||||||
|
work = [event for event in events if event.get("type") == "game_thread_work_frame"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop_s]
|
||||||
|
ticks = [float(event.get("tick_ms", 0.0)) for event in work]
|
||||||
|
updates = [float(event.get("update_chunks_ms", 0.0)) for event in work]
|
||||||
|
arrival = [event for event in samples if float(event.get("t_s", 0.0)) < start
|
||||||
|
and float(event.get("speed_cm_s", 0.0)) < 50.0]
|
||||||
|
|
||||||
|
def settled(event: dict, end: int) -> bool:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
actual = event.get("settled_by_lod", [0] * 9)
|
||||||
|
return sum(desired[:end]) > 0 and sum(actual[:end]) >= sum(desired[:end])
|
||||||
|
|
||||||
|
def far_holes(event: dict) -> float | None:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)[3:]
|
||||||
|
covered = event.get("covered_by_lod", [0] * 9)[3:]
|
||||||
|
return 100.0 * (sum(desired) - sum(covered)) / sum(desired) if sum(desired) else None
|
||||||
|
|
||||||
|
near_settled = next((event for event in arrival if settled(event, 3)), None)
|
||||||
|
whole_settled = next((event for event in arrival if settled(event, 9)), None)
|
||||||
|
far_clear = next((event for event in arrival if far_holes(event) == 0.0), None)
|
||||||
|
compile_done = [event for event in events if event.get("type") == "core_region_compile_complete"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop_s]
|
||||||
|
compile_requests = [event for event in events
|
||||||
|
if event.get("type") == "core_region_compile_request"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop_s]
|
||||||
|
lookahead_turns = [event for event in events
|
||||||
|
if event.get("type") == "core_region_lookahead_turn"]
|
||||||
|
row = {
|
||||||
|
"scenario": scenario, "condition": condition, "repetition": repetition,
|
||||||
|
"speed_mps": speed / 100.0, "core_region_lookahead": lookahead,
|
||||||
|
"exit_code": process.returncode, "wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"moving_samples": len(moving), "dwell_samples": len(dwell),
|
||||||
|
"near_ready_moving_mean_pct": statistics.mean(r["near_ready_pct"] for r in moving_rows),
|
||||||
|
"lod0_ready_moving_mean_pct": statistics.mean(r["lod0_ready_pct"] for r in moving_rows),
|
||||||
|
"lod1_ready_moving_mean_pct": statistics.mean(r["lod1_ready_pct"] for r in moving_rows),
|
||||||
|
"lod2_ready_moving_mean_pct": statistics.mean(r["lod2_ready_pct"] for r in moving_rows),
|
||||||
|
"far_holes_moving_mean_pct": statistics.mean(r["far_hole_pct"] for r in moving_rows),
|
||||||
|
"game_thread_tick_p99_ms": percentile(ticks, 0.99),
|
||||||
|
"game_thread_tick_max_ms": max(ticks, default=None),
|
||||||
|
"stream_update_p99_ms": percentile(updates, 0.99),
|
||||||
|
"stream_update_max_ms": max(updates, default=None),
|
||||||
|
"lod02_settled_t_s": float(near_settled["t_s"]) if near_settled else None,
|
||||||
|
"whole_view_settled_t_s": float(whole_settled["t_s"]) if whole_settled else None,
|
||||||
|
"far_clear_t_s": float(far_clear["t_s"]) if far_clear else None,
|
||||||
|
"far_holes_at_pre_move_pct": far_holes(arrival[-1]) if arrival else None,
|
||||||
|
"pending_gate_p50_ms": percentile(
|
||||||
|
[float(e.get("pending_gate_elapsed_ms", 0.0)) for e in compile_done], 0.50),
|
||||||
|
"compile_queue_p95_ms": percentile(
|
||||||
|
[float(e.get("compile_queue_ms", 0.0)) for e in compile_done], 0.95),
|
||||||
|
"compile_task_p95_ms": percentile(
|
||||||
|
[float(e.get("compile_task_ms", 0.0)) for e in compile_done], 0.95),
|
||||||
|
"compiled_tiles_total": sum(int(e.get("compiled_tiles", 0)) for e in compile_done),
|
||||||
|
"lookahead_tiles_peak": max((int(e.get("lookahead_region_tiles", 0))
|
||||||
|
for e in compile_requests), default=0),
|
||||||
|
"wasted_prefetch_tiles": sum(int(e.get("wasted_prefetch_tiles", 0))
|
||||||
|
for e in compile_done),
|
||||||
|
"lookahead_turn_events": lookahead_turns,
|
||||||
|
"compile_completion_count": len(compile_done),
|
||||||
|
"trace": str(trace.relative_to(ROOT)), "log": str(abslog.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
with (ROOT / "Saved/Linux/l123" / "runs.jsonl").open("a", encoding="utf-8") as stream:
|
||||||
|
stream.write(json.dumps(row, sort_keys=True) + "\n")
|
||||||
|
print("[L123Pair] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def pairs_for(runs: list[dict], scenario: str) -> list[dict]:
|
||||||
|
pairs = []
|
||||||
|
for repetition in range(1, 6):
|
||||||
|
baseline = next(row for row in runs if row["scenario"] == scenario
|
||||||
|
and row["condition"] == "baseline" and row["repetition"] == repetition)
|
||||||
|
candidate = next(row for row in runs if row["scenario"] == scenario
|
||||||
|
and row["condition"] == "lookahead" and row["repetition"] == repetition)
|
||||||
|
pairs.append({"repetition": repetition,
|
||||||
|
"near_ready_delta_pct": candidate["near_ready_moving_mean_pct"]
|
||||||
|
- baseline["near_ready_moving_mean_pct"],
|
||||||
|
"far_holes_delta_pct": candidate["far_holes_moving_mean_pct"]
|
||||||
|
- baseline["far_holes_moving_mean_pct"],
|
||||||
|
"tick_p99_delta_ms": candidate["game_thread_tick_p99_ms"]
|
||||||
|
- baseline["game_thread_tick_p99_ms"],
|
||||||
|
"tick_max_delta_ms": candidate["game_thread_tick_max_ms"]
|
||||||
|
- baseline["game_thread_tick_max_ms"]})
|
||||||
|
return pairs
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--scenario", choices=("movement", "arrival"), required=True)
|
||||||
|
parser.add_argument("--turn-diagnostic", action="store_true")
|
||||||
|
args = parser.parse_args()
|
||||||
|
if not EDITOR.is_file():
|
||||||
|
raise SystemExit(f"missing Unreal editor: {EDITOR}")
|
||||||
|
root_out = ROOT / "Saved/Linux/l123" / args.scenario
|
||||||
|
root_out.mkdir(parents=True, exist_ok=True)
|
||||||
|
(ROOT / "Saved/Linux/l123").mkdir(parents=True, exist_ok=True)
|
||||||
|
if args.turn_diagnostic:
|
||||||
|
row = run("movement", "turn_lookahead", 1, turn=True)
|
||||||
|
turn_out = ROOT / "Saved/Linux/l123/turn"
|
||||||
|
turn_out.mkdir(parents=True, exist_ok=True)
|
||||||
|
(turn_out / "summary.json").write_text(
|
||||||
|
json.dumps({"run": row}, indent=2) + "\n", encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
runs = []
|
||||||
|
for repetition in range(1, 6):
|
||||||
|
order = ("baseline", "lookahead") if repetition % 2 else ("lookahead", "baseline")
|
||||||
|
for condition in order:
|
||||||
|
runs.append(run(args.scenario, condition, repetition))
|
||||||
|
(root_out / "runs-partial.json").write_text(
|
||||||
|
json.dumps(runs, indent=2) + "\n", encoding="utf-8")
|
||||||
|
pairs = pairs_for(runs, args.scenario)
|
||||||
|
summary = {
|
||||||
|
"design": "Compile/retain the current near region set plus one step along velocity; "
|
||||||
|
"return completion through an Epoch-tagged MPSC queue.",
|
||||||
|
"scenario": args.scenario, "runs": runs, "pairs": pairs,
|
||||||
|
"paired_median_near_ready_delta_pct": statistics.median(
|
||||||
|
pair["near_ready_delta_pct"] for pair in pairs),
|
||||||
|
"paired_median_far_holes_delta_pct": statistics.median(
|
||||||
|
pair["far_holes_delta_pct"] for pair in pairs),
|
||||||
|
"paired_median_tick_p99_delta_ms": statistics.median(
|
||||||
|
pair["tick_p99_delta_ms"] for pair in pairs),
|
||||||
|
"paired_median_tick_max_delta_ms": statistics.median(
|
||||||
|
pair["tick_max_delta_ms"] for pair in pairs),
|
||||||
|
}
|
||||||
|
(root_out / "summary.json").write_text(json.dumps(summary, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,825 @@
|
|||||||
|
diff --git a/Source/VoxelForge/Private/VoxelWorld.cpp b/Source/VoxelForge/Private/VoxelWorld.cpp
|
||||||
|
index 42ac6f5..2c7046b 100644
|
||||||
|
--- a/Source/VoxelForge/Private/VoxelWorld.cpp
|
||||||
|
+++ b/Source/VoxelForge/Private/VoxelWorld.cpp
|
||||||
|
@@ -195,10 +195,13 @@ public:
|
||||||
|
std::atomic<bool> bShuttingDown{false};
|
||||||
|
std::atomic<uint64> NextEditSequence{1};
|
||||||
|
std::atomic<uint64> RegionCompileStartCycles{0};
|
||||||
|
+ std::atomic<uint64> RegionCompileWorkerStartCycles{0};
|
||||||
|
std::atomic<uint64> RegionCompileEndCycles{0};
|
||||||
|
std::atomic<int32> RegionCompileTileCount{0};
|
||||||
|
std::atomic<int32> RegionCompileSerial{0};
|
||||||
|
+ std::atomic<uint32> RegionCompileWorkerThreadId{0};
|
||||||
|
std::atomic<double> RegionCompileWorkerSeconds{0.0};
|
||||||
|
+ std::atomic<double> RegionCompileWallSeconds{0.0};
|
||||||
|
int32 FullResolutionLodCount = 0;
|
||||||
|
FString InitialFailureReason;
|
||||||
|
};
|
||||||
|
@@ -262,6 +265,12 @@ namespace
|
||||||
|
GVoxelForgeCoreRegionConcurrentAdmission,
|
||||||
|
TEXT("Allow tiles independent of a pending Core region compile to proceed; 0 restores the global streaming stop."));
|
||||||
|
|
||||||
|
+ int32 GVoxelForgeCoreRegionLookahead = 0;
|
||||||
|
+ FAutoConsoleVariableRef CVarVoxelForgeCoreRegionLookahead(
|
||||||
|
+ TEXT("voxel.CoreRegionLookahead"),
|
||||||
|
+ GVoxelForgeCoreRegionLookahead,
|
||||||
|
+ TEXT("Retain and compile the next near region set along velocity; 0 disables region lookahead."));
|
||||||
|
+
|
||||||
|
int32 GVoxelForgeStreamingResultBacklogCap = 256;
|
||||||
|
FAutoConsoleVariableRef CVarVoxelForgeStreamingResultBacklogCap(
|
||||||
|
TEXT("voxel.StreamingResultBacklogCap"),
|
||||||
|
@@ -1745,6 +1754,10 @@ void AVoxelWorld::EndPlay(const EEndPlayReason::Type EndPlayReason)
|
||||||
|
{
|
||||||
|
if (Discard.bObsolete) { ++ObsoleteTileAbortCount; }
|
||||||
|
}
|
||||||
|
+ FCoreRegionCompileResult DiscardedRegionCompile;
|
||||||
|
+ while (CoreRegionCompileResults.Dequeue(DiscardedRegionCompile))
|
||||||
|
+ {
|
||||||
|
+ }
|
||||||
|
UE_LOG(LogTemp, Display, TEXT("[VoxelWorld] EndPlay: workers drained in %.6fs; obsolete_tile_aborts=%d"),
|
||||||
|
FPlatformTime::Seconds() - EndPlayStartSeconds, ObsoleteTileAbortCount);
|
||||||
|
VoxelForgeCoreState.Reset();
|
||||||
|
@@ -2093,6 +2106,11 @@ void AVoxelWorld::BeginPlay()
|
||||||
|
PendingCoreRegionTiles.Reset();
|
||||||
|
PendingCoreRegionTileSet.Reset();
|
||||||
|
PendingCoreRetainedTiles.Reset();
|
||||||
|
+ CoreRegionRetainedDesiredTiles.Reset();
|
||||||
|
+ CoreRegionLookaheadTiles.Reset();
|
||||||
|
+ LastCoreRegionLookaheadCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
|
||||||
|
+ LastCoreRegionLookaheadStep = FIntVector::ZeroValue;
|
||||||
|
+ LastCoreRegionLookaheadGenerationEpoch = GenerationEpoch;
|
||||||
|
CoreRegionCompileTilesInFlight.Reset();
|
||||||
|
LastCoreRegionRequestCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
|
||||||
|
PendingCoreRegionRequestCenter = FIntVector::ZeroValue;
|
||||||
|
@@ -2149,6 +2167,7 @@ void AVoxelWorld::BeginPlay()
|
||||||
|
int32 CommandLineSampleStacks = GVoxelForgeSampleStacks;
|
||||||
|
int32 CommandLineCollisionGate = GVoxelForgeCollisionGate;
|
||||||
|
int32 CommandLineCoreRegionConcurrentAdmission = GVoxelForgeCoreRegionConcurrentAdmission;
|
||||||
|
+ int32 CommandLineCoreRegionLookahead = GVoxelForgeCoreRegionLookahead;
|
||||||
|
int32 CommandLineStreamingCollisionLookahead = GVoxelForgeStreamingCollisionLookahead;
|
||||||
|
int32 CommandLineStreamingResultBacklogCap = GVoxelForgeStreamingResultBacklogCap;
|
||||||
|
int32 CommandLineStreamingReadyQueueDepth = GVoxelForgeStreamingReadyQueueDepth;
|
||||||
|
@@ -2257,6 +2276,11 @@ void AVoxelWorld::BeginPlay()
|
||||||
|
GVoxelForgeCoreRegionConcurrentAdmission =
|
||||||
|
CommandLineCoreRegionConcurrentAdmission != 0 ? 1 : 0;
|
||||||
|
}
|
||||||
|
+ if (FParse::Value(FCommandLine::Get(), TEXT("voxel.CoreRegionLookahead="),
|
||||||
|
+ CommandLineCoreRegionLookahead))
|
||||||
|
+ {
|
||||||
|
+ GVoxelForgeCoreRegionLookahead = CommandLineCoreRegionLookahead != 0 ? 1 : 0;
|
||||||
|
+ }
|
||||||
|
if (FParse::Value(FCommandLine::Get(), TEXT("voxel.StreamingResultBacklogCap="),
|
||||||
|
CommandLineStreamingResultBacklogCap))
|
||||||
|
{
|
||||||
|
@@ -2747,6 +2771,7 @@ void AVoxelWorld::BeginPlay()
|
||||||
|
"\"streaming_coarse_first\":%d,\"streaming_dynamic_near_reserve\":%d,"
|
||||||
|
"\"streaming_legacy_pending_result_cap\":%d,"
|
||||||
|
"\"streaming_collision_lookahead\":%d,\"core_region_concurrent_admission\":%d,"
|
||||||
|
+ "\"core_region_lookahead\":%d,"
|
||||||
|
"\"collision_gate\":%d,"
|
||||||
|
"\"streaming_result_backlog_cap\":%d,\"streaming_startup_apply_boost\":%d,"
|
||||||
|
"\"streaming_far_apply_supplement\":%d,"
|
||||||
|
@@ -2769,6 +2794,7 @@ void AVoxelWorld::BeginPlay()
|
||||||
|
GVoxelForgeStreamingCoarseFirst, GVoxelForgeStreamingDynamicNearReserve,
|
||||||
|
GVoxelForgeStreamingLegacyPendingResultCap,
|
||||||
|
GVoxelForgeStreamingCollisionLookahead, GVoxelForgeCoreRegionConcurrentAdmission,
|
||||||
|
+ GVoxelForgeCoreRegionLookahead,
|
||||||
|
GVoxelForgeCollisionGate,
|
||||||
|
GVoxelForgeStreamingResultBacklogCap, GVoxelForgeStreamingStartupApplyBoost,
|
||||||
|
GVoxelForgeStreamingFarApplySupplement,
|
||||||
|
@@ -3246,11 +3272,13 @@ void AVoxelWorld::ConfigureHeadlessStreamingTest()
|
||||||
|
float Speed = 800.0f;
|
||||||
|
float StartSeconds = 20.0f;
|
||||||
|
float MoveDurationSeconds = 0.0f;
|
||||||
|
+ float MoveTurnAfterSeconds = 0.0f;
|
||||||
|
float ExitSeconds = 0.0f;
|
||||||
|
int32 ExitOnSteadyValue = 0;
|
||||||
|
FParse::Value(CommandLine, TEXT("voxel.TestMoveSpeedCmPerSecond="), Speed);
|
||||||
|
FParse::Value(CommandLine, TEXT("voxel.TestMoveStartSeconds="), StartSeconds);
|
||||||
|
FParse::Value(CommandLine, TEXT("voxel.TestMoveDurationSeconds="), MoveDurationSeconds);
|
||||||
|
+ FParse::Value(CommandLine, TEXT("voxel.TestMoveTurnAfterSeconds="), MoveTurnAfterSeconds);
|
||||||
|
FParse::Value(CommandLine, TEXT("voxel.TestExitSeconds="), ExitSeconds);
|
||||||
|
FParse::Value(CommandLine, TEXT("voxel.TestTeleportAfterSeconds="),
|
||||||
|
HeadlessStreamingTestTeleportAtSeconds);
|
||||||
|
@@ -3418,9 +3446,44 @@ void AVoxelWorld::ConfigureHeadlessStreamingTest()
|
||||||
|
{
|
||||||
|
HeadlessStreamingTestDirection = FVector::XAxisVector;
|
||||||
|
}
|
||||||
|
+ FString TurnDirectionText;
|
||||||
|
+ const FString TurnDirectionKey(TEXT("voxel.TestMoveTurnDirection="));
|
||||||
|
+ const int32 TurnDirectionKeyIndex = CommandLineText.Find(
|
||||||
|
+ TurnDirectionKey, ESearchCase::IgnoreCase, ESearchDir::FromStart);
|
||||||
|
+ if (TurnDirectionKeyIndex != INDEX_NONE)
|
||||||
|
+ {
|
||||||
|
+ TurnDirectionText = CommandLineText.Mid(
|
||||||
|
+ TurnDirectionKeyIndex + TurnDirectionKey.Len());
|
||||||
|
+ const int32 DirectionEnd = TurnDirectionText.Find(
|
||||||
|
+ TEXT(" "), ESearchCase::CaseSensitive, ESearchDir::FromStart);
|
||||||
|
+ if (DirectionEnd != INDEX_NONE)
|
||||||
|
+ {
|
||||||
|
+ TurnDirectionText.LeftInline(DirectionEnd, EAllowShrinking::No);
|
||||||
|
+ }
|
||||||
|
+ TurnDirectionText.TrimStartAndEndInline();
|
||||||
|
+ TurnDirectionText.RemoveFromStart(TEXT("\""));
|
||||||
|
+ TurnDirectionText.RemoveFromEnd(TEXT("\""));
|
||||||
|
+ TArray<FString> Components;
|
||||||
|
+ TurnDirectionText.ParseIntoArray(Components, TEXT(","), true);
|
||||||
|
+ if (Components.Num() == 3)
|
||||||
|
+ {
|
||||||
|
+ HeadlessStreamingTestTurnDirection = FVector(
|
||||||
|
+ FCString::Atof(*Components[0]),
|
||||||
|
+ FCString::Atof(*Components[1]),
|
||||||
|
+ FCString::Atof(*Components[2]));
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ HeadlessStreamingTestTurnDirection =
|
||||||
|
+ HeadlessStreamingTestTurnDirection.GetSafeNormal();
|
||||||
|
+ if (HeadlessStreamingTestTurnDirection.IsNearlyZero())
|
||||||
|
+ {
|
||||||
|
+ HeadlessStreamingTestTurnDirection = FVector::YAxisVector;
|
||||||
|
+ }
|
||||||
|
HeadlessStreamingTestSpeedCmPerSecond = FMath::Max(Speed, 0.0f);
|
||||||
|
HeadlessStreamingTestStartSeconds = FMath::Max(StartSeconds, 0.0f);
|
||||||
|
HeadlessStreamingTestMoveDurationSeconds = FMath::Max(MoveDurationSeconds, 0.0f);
|
||||||
|
+ HeadlessStreamingTestTurnAfterSeconds = FMath::Max(MoveTurnAfterSeconds, 0.0f);
|
||||||
|
+ bHeadlessStreamingTestTurnDone = false;
|
||||||
|
bHeadlessStreamingTestMovement = HeadlessStreamingTestSpeedCmPerSecond > 0.0f;
|
||||||
|
}
|
||||||
|
|
||||||
|
@@ -3431,15 +3494,20 @@ void AVoxelWorld::ConfigureHeadlessStreamingTest()
|
||||||
|
HeadlessStreamingTestBeginSeconds = FPlatformTime::Seconds();
|
||||||
|
UE_LOG(LogTemp, Display,
|
||||||
|
TEXT("[VoxelForgeStreamingTest] clean=1 movement=%d speed_cm_s=%.3f start_s=%.3f "
|
||||||
|
- "move_duration_s=%.3f exit_s=%.3f direction=(%.3f,%.3f,%.3f)"),
|
||||||
|
+ "move_duration_s=%.3f turn_after_s=%.3f exit_s=%.3f "
|
||||||
|
+ "direction=(%.3f,%.3f,%.3f) turn_direction=(%.3f,%.3f,%.3f)"),
|
||||||
|
bHeadlessStreamingTestMovement ? 1 : 0,
|
||||||
|
HeadlessStreamingTestSpeedCmPerSecond,
|
||||||
|
HeadlessStreamingTestStartSeconds,
|
||||||
|
HeadlessStreamingTestMoveDurationSeconds,
|
||||||
|
+ HeadlessStreamingTestTurnAfterSeconds,
|
||||||
|
HeadlessStreamingTestExitSeconds,
|
||||||
|
HeadlessStreamingTestDirection.X,
|
||||||
|
HeadlessStreamingTestDirection.Y,
|
||||||
|
- HeadlessStreamingTestDirection.Z);
|
||||||
|
+ HeadlessStreamingTestDirection.Z,
|
||||||
|
+ HeadlessStreamingTestTurnDirection.X,
|
||||||
|
+ HeadlessStreamingTestTurnDirection.Y,
|
||||||
|
+ HeadlessStreamingTestTurnDirection.Z);
|
||||||
|
}
|
||||||
|
|
||||||
|
int32 CollisionGateStressValue = 0;
|
||||||
|
@@ -4503,6 +4571,32 @@ void AVoxelWorld::AdvanceHeadlessStreamingTest(
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
+ if (!bHeadlessStreamingTestTurnDone && HeadlessStreamingTestTurnAfterSeconds > 0.0f
|
||||||
|
+ && ElapsedSeconds - static_cast<double>(HeadlessStreamingTestStartSeconds)
|
||||||
|
+ >= static_cast<double>(HeadlessStreamingTestTurnAfterSeconds))
|
||||||
|
+ {
|
||||||
|
+ bHeadlessStreamingTestTurnDone = true;
|
||||||
|
+ const FVector PreviousDirection = HeadlessStreamingTestDirection;
|
||||||
|
+ HeadlessStreamingTestDirection = HeadlessStreamingTestTurnDirection;
|
||||||
|
+ if (VoxelForgeStartupTrace::IsActive())
|
||||||
|
+ {
|
||||||
|
+ VoxelForgeStartupTrace::RecordEvent(TEXT("streaming_test_move_turned"),
|
||||||
|
+ FString::Printf(
|
||||||
|
+ TEXT("\"elapsed_s\":%s,\"from\":[%.0f,%.0f,%.0f],"
|
||||||
|
+ "\"to\":[%.0f,%.0f,%.0f]"),
|
||||||
|
+ *FString::SanitizeFloat(ElapsedSeconds),
|
||||||
|
+ PreviousDirection.X, PreviousDirection.Y, PreviousDirection.Z,
|
||||||
|
+ HeadlessStreamingTestDirection.X, HeadlessStreamingTestDirection.Y,
|
||||||
|
+ HeadlessStreamingTestDirection.Z));
|
||||||
|
+ }
|
||||||
|
+ UE_LOG(LogTemp, Display,
|
||||||
|
+ TEXT("[VoxelForgeStreamingTest] turn=1 elapsed_s=%.3f from=(%.0f,%.0f,%.0f) "
|
||||||
|
+ "to=(%.0f,%.0f,%.0f)"),
|
||||||
|
+ ElapsedSeconds, PreviousDirection.X, PreviousDirection.Y, PreviousDirection.Z,
|
||||||
|
+ HeadlessStreamingTestDirection.X, HeadlessStreamingTestDirection.Y,
|
||||||
|
+ HeadlessStreamingTestDirection.Z);
|
||||||
|
+ }
|
||||||
|
+
|
||||||
|
if (HeadlessStreamingTestMoveDurationSeconds > 0.0f
|
||||||
|
&& ElapsedSeconds - static_cast<double>(HeadlessStreamingTestStartSeconds)
|
||||||
|
>= static_cast<double>(HeadlessStreamingTestMoveDurationSeconds))
|
||||||
|
@@ -11258,16 +11352,113 @@ void AVoxelWorld::BuildCoreDesiredTiles(const FIntVector& Center, const FVector&
|
||||||
|
}
|
||||||
|
BuildDesiredTransitionIndex();
|
||||||
|
|
||||||
|
- bool bRetainedSetChanged = false;
|
||||||
|
- int32 RetainedTileCount = 0;
|
||||||
|
+ FIntVector RegionLookaheadStep = FIntVector::ZeroValue;
|
||||||
|
+ if (GVoxelForgeCoreRegionLookahead != 0 && PlayerHeading.Size() >= 50.0f)
|
||||||
|
+ {
|
||||||
|
+ RegionLookaheadStep = GetStreamingHeadingStep(PlayerHeading, PlayerPawn);
|
||||||
|
+ }
|
||||||
|
+ const bool bRegionLookaheadTurn = RegionLookaheadStep != FIntVector::ZeroValue
|
||||||
|
+ && LastCoreRegionLookaheadStep != FIntVector::ZeroValue
|
||||||
|
+ && LastCoreRegionLookaheadStep != RegionLookaheadStep
|
||||||
|
+ && LastCoreRegionLookaheadGenerationEpoch == GenerationEpoch;
|
||||||
|
+ const FIntVector PreviousRegionLookaheadStep = LastCoreRegionLookaheadStep;
|
||||||
|
+ const TArray<FVoxelTileKey> PreviousRegionLookaheadTiles = bRegionLookaheadTurn
|
||||||
|
+ ? CoreRegionLookaheadTiles : TArray<FVoxelTileKey>();
|
||||||
|
+ if (RegionLookaheadStep != FIntVector::ZeroValue
|
||||||
|
+ && (LastCoreRegionLookaheadCenter != Center
|
||||||
|
+ || LastCoreRegionLookaheadStep != RegionLookaheadStep
|
||||||
|
+ || LastCoreRegionLookaheadGenerationEpoch != GenerationEpoch))
|
||||||
|
+ {
|
||||||
|
+ const FVector AheadCameraVoxels = CameraVoxels + FVector(
|
||||||
|
+ static_cast<float>(RegionLookaheadStep.X),
|
||||||
|
+ static_cast<float>(RegionLookaheadStep.Y),
|
||||||
|
+ static_cast<float>(RegionLookaheadStep.Z)) * static_cast<float>(CHUNK_SIZE);
|
||||||
|
+ std::vector<VoxelForgeCore::TileKey> AheadDesired;
|
||||||
|
+ {
|
||||||
|
+ FReadScopeLock Lock(State->WorldLock);
|
||||||
|
+ if (State->World)
|
||||||
|
+ {
|
||||||
|
+ AheadDesired = State->World->desiredTiles(
|
||||||
|
+ {AheadCameraVoxels.X, AheadCameraVoxels.Y, AheadCameraVoxels.Z});
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ CoreRegionLookaheadTiles.Reset();
|
||||||
|
+ TSet<FVoxelTileKey> AheadSet;
|
||||||
|
+ for (const VoxelForgeCore::TileKey& Key : AheadDesired)
|
||||||
|
+ {
|
||||||
|
+ if (Key.lod >= State->FullResolutionLodCount) continue;
|
||||||
|
+ const FVoxelTileKey RuntimeKey(
|
||||||
|
+ FIntVector(Key.coordinate.x, Key.coordinate.y, Key.coordinate.z), Key.lod);
|
||||||
|
+ if (!AheadSet.Contains(RuntimeKey))
|
||||||
|
+ {
|
||||||
|
+ AheadSet.Add(RuntimeKey);
|
||||||
|
+ CoreRegionLookaheadTiles.Add(RuntimeKey);
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ if (bRegionLookaheadTurn)
|
||||||
|
+ {
|
||||||
|
+ TSet<FVoxelTileKey> ActualSet;
|
||||||
|
+ for (const FVoxelTileKey& Key : DesiredSorted) ActualSet.Add(Key);
|
||||||
|
+ int32 StaleRetainedLookaheadTiles = 0;
|
||||||
|
+ {
|
||||||
|
+ FScopeLock RetainedLock(&State->RetainedTilesLock);
|
||||||
|
+ for (const FVoxelTileKey& Key : PreviousRegionLookaheadTiles)
|
||||||
|
+ {
|
||||||
|
+ if (ActualSet.Contains(Key) || AheadSet.Contains(Key)) continue;
|
||||||
|
+ const VoxelForgeCore::TileKey CoreKey{
|
||||||
|
+ {Key.Coord.X, Key.Coord.Y, Key.Coord.Z}, Key.Level};
|
||||||
|
+ if (State->RetainedTiles.contains(CoreKey))
|
||||||
|
+ {
|
||||||
|
+ ++StaleRetainedLookaheadTiles;
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ if (VoxelForgeStartupTrace::IsActive())
|
||||||
|
+ {
|
||||||
|
+ VoxelForgeStartupTrace::RecordEvent(TEXT("core_region_lookahead_turn"),
|
||||||
|
+ FString::Printf(
|
||||||
|
+ TEXT("\"previous_step\":[%d,%d,%d],\"new_step\":[%d,%d,%d],"
|
||||||
|
+ "\"previous_prefetch_tiles\":%d,"
|
||||||
|
+ "\"stale_retained_lookahead_tiles\":%d"),
|
||||||
|
+ PreviousRegionLookaheadStep.X, PreviousRegionLookaheadStep.Y,
|
||||||
|
+ PreviousRegionLookaheadStep.Z, RegionLookaheadStep.X,
|
||||||
|
+ RegionLookaheadStep.Y, RegionLookaheadStep.Z,
|
||||||
|
+ PreviousRegionLookaheadTiles.Num(), StaleRetainedLookaheadTiles));
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ LastCoreRegionLookaheadCenter = Center;
|
||||||
|
+ LastCoreRegionLookaheadStep = RegionLookaheadStep;
|
||||||
|
+ LastCoreRegionLookaheadGenerationEpoch = GenerationEpoch;
|
||||||
|
+ }
|
||||||
|
+ else if (GVoxelForgeCoreRegionLookahead == 0)
|
||||||
|
+ {
|
||||||
|
+ CoreRegionLookaheadTiles.Reset();
|
||||||
|
+ LastCoreRegionLookaheadCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
|
||||||
|
+ LastCoreRegionLookaheadStep = FIntVector::ZeroValue;
|
||||||
|
+ LastCoreRegionLookaheadGenerationEpoch = GenerationEpoch;
|
||||||
|
+ }
|
||||||
|
+
|
||||||
|
+ CoreRegionRetainedDesiredTiles = DesiredSorted;
|
||||||
|
TSet<FVoxelTileKey> DesiredSet;
|
||||||
|
+ DesiredSet.Reserve(DesiredSorted.Num() + CoreRegionLookaheadTiles.Num());
|
||||||
|
for (const FVoxelTileKey& Key : DesiredSorted) DesiredSet.Add(Key);
|
||||||
|
+ for (const FVoxelTileKey& Key : CoreRegionLookaheadTiles)
|
||||||
|
+ {
|
||||||
|
+ if (!DesiredSet.Contains(Key))
|
||||||
|
+ {
|
||||||
|
+ DesiredSet.Add(Key);
|
||||||
|
+ CoreRegionRetainedDesiredTiles.Add(Key);
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+
|
||||||
|
+ bool bRetainedSetChanged = false;
|
||||||
|
+ int32 RetainedTileCount = 0;
|
||||||
|
TArray<FVoxelTileKey> MissingRetainedFineTiles;
|
||||||
|
{
|
||||||
|
FScopeLock Lock(&State->RetainedTilesLock);
|
||||||
|
RetainedTileCount = static_cast<int32>(State->RetainedTiles.size());
|
||||||
|
bRetainedSetChanged = DesiredSet.Num() != RetainedTileCount;
|
||||||
|
- for (const FVoxelTileKey& Key : DesiredSorted)
|
||||||
|
+ for (const FVoxelTileKey& Key : CoreRegionRetainedDesiredTiles)
|
||||||
|
{
|
||||||
|
const VoxelForgeCore::TileKey CoreKey{{Key.Coord.X, Key.Coord.Y, Key.Coord.Z}, Key.Level};
|
||||||
|
const bool bAlreadyRetained = State->RetainedTiles.contains(CoreKey);
|
||||||
|
@@ -11313,7 +11504,7 @@ void AVoxelWorld::BuildCoreDesiredTiles(const FIntVector& Center, const FVector&
|
||||||
|
{
|
||||||
|
PendingCoreRegionTileSet.Add(Key);
|
||||||
|
}
|
||||||
|
- PendingCoreRetainedTiles = DesiredSorted;
|
||||||
|
+ PendingCoreRetainedTiles = CoreRegionRetainedDesiredTiles;
|
||||||
|
bCoreRegionCompilePending = true;
|
||||||
|
++CoreRegionCompileRequestSerial;
|
||||||
|
if (VoxelForgeStartupTrace::IsActive())
|
||||||
|
@@ -11322,6 +11513,7 @@ void AVoxelWorld::BuildCoreDesiredTiles(const FIntVector& Center, const FVector&
|
||||||
|
TEXT("\"request_serial\":%d,\"reason\":\"%s\",\"center\":[%d,%d,%d],"
|
||||||
|
"\"previous_center\":[%d,%d,%d],\"center_delta\":[%d,%d,%d],"
|
||||||
|
"\"desired_tiles\":%d,\"retained_tiles\":%d,\"missing_region_tiles\":%d,"
|
||||||
|
+ "\"lookahead_region_tiles\":%d,\"lookahead_step\":[%d,%d,%d],"
|
||||||
|
"\"desired_epoch\":%u,\"active_tile_tasks\":%d"),
|
||||||
|
CoreRegionCompileRequestSerial,
|
||||||
|
!bHadPreviousCenter ? TEXT("initial_desired_region_set")
|
||||||
|
@@ -11332,7 +11524,9 @@ void AVoxelWorld::BuildCoreDesiredTiles(const FIntVector& Center, const FVector&
|
||||||
|
bCenterMoved ? Center.X - PreviousCenter.X : 0,
|
||||||
|
bCenterMoved ? Center.Y - PreviousCenter.Y : 0,
|
||||||
|
bCenterMoved ? Center.Z - PreviousCenter.Z : 0,
|
||||||
|
- DesiredSorted.Num(), RetainedTileCount, PendingCoreRegionTiles.Num(), DesiredEpoch,
|
||||||
|
+ DesiredSorted.Num(), RetainedTileCount, PendingCoreRegionTiles.Num(),
|
||||||
|
+ CoreRegionLookaheadTiles.Num(), RegionLookaheadStep.X, RegionLookaheadStep.Y,
|
||||||
|
+ RegionLookaheadStep.Z, DesiredEpoch,
|
||||||
|
ActiveTileTaskCount.load(std::memory_order_relaxed)));
|
||||||
|
}
|
||||||
|
StartPendingCoreRegionCompile();
|
||||||
|
@@ -11363,13 +11557,27 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||||
|
State->bRegionCompileSucceeded.store(false, std::memory_order_release);
|
||||||
|
const TArray<FVoxelTileKey> MissingTiles = PendingCoreRegionTiles;
|
||||||
|
const TArray<FVoxelTileKey> RetainedTiles = PendingCoreRetainedTiles;
|
||||||
|
+ TSet<FVoxelTileKey> ActualDesiredSet;
|
||||||
|
+ TSet<FVoxelTileKey> PrefetchSet;
|
||||||
|
+ for (const FVoxelTileKey& Key : DesiredSorted) ActualDesiredSet.Add(Key);
|
||||||
|
+ for (const FVoxelTileKey& Key : CoreRegionLookaheadTiles) PrefetchSet.Add(Key);
|
||||||
|
+ TArray<FVoxelTileKey> PrefetchTiles;
|
||||||
|
+ for (const FVoxelTileKey& Key : MissingTiles)
|
||||||
|
+ {
|
||||||
|
+ if (PrefetchSet.Contains(Key) && !ActualDesiredSet.Contains(Key))
|
||||||
|
+ {
|
||||||
|
+ PrefetchTiles.Add(Key);
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
const FIntVector CompileCenter = PendingCoreRegionRequestCenter;
|
||||||
|
const bool bAllowConcurrentGeneration = GVoxelForgeCoreRegionConcurrentAdmission != 0;
|
||||||
|
const uint32 TaskEpoch = GenerationEpoch;
|
||||||
|
const uint32 TaskDesiredEpoch = DesiredEpoch;
|
||||||
|
+ const int32 RequestSerial = CoreRegionCompileRequestSerial;
|
||||||
|
const uint32 WorkerLimit = static_cast<uint32>(FMath::Clamp(GetMaxConcurrentTasks(), 1, 16));
|
||||||
|
ActiveCoreRegionCompileCenter = CompileCenter;
|
||||||
|
ActiveCoreRegionCompileRequestSerial = CoreRegionCompileRequestSerial;
|
||||||
|
+ ActiveCoreRegionWastedPrefetchTiles = 0;
|
||||||
|
ActiveCoreRegionCompileStartSeconds = FPlatformTime::Seconds();
|
||||||
|
ActiveCoreRegionCompileStartCycles = FPlatformTime::Cycles64();
|
||||||
|
CoreRegionCompileTilesInFlight.Reset();
|
||||||
|
@@ -11381,25 +11589,32 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||||
|
1, std::memory_order_relaxed) + 1;
|
||||||
|
State->RegionCompileStartCycles.store(ActiveCoreRegionCompileStartCycles,
|
||||||
|
std::memory_order_release);
|
||||||
|
+ State->RegionCompileWorkerStartCycles.store(0, std::memory_order_release);
|
||||||
|
State->RegionCompileEndCycles.store(0, std::memory_order_release);
|
||||||
|
State->RegionCompileTileCount.store(0, std::memory_order_release);
|
||||||
|
+ State->RegionCompileWorkerThreadId.store(0, std::memory_order_release);
|
||||||
|
State->RegionCompileWorkerSeconds.store(0.0, std::memory_order_release);
|
||||||
|
+ State->RegionCompileWallSeconds.store(0.0, std::memory_order_release);
|
||||||
|
if (VoxelForgeStartupTrace::IsActive())
|
||||||
|
{
|
||||||
|
VoxelForgeStartupTrace::RecordEvent(TEXT("core_region_compile_start"), FString::Printf(
|
||||||
|
TEXT("\"compile_serial\":%d,\"request_serial\":%d,\"center\":[%d,%d,%d],"
|
||||||
|
"\"missing_region_tiles\":%d,\"retained_tiles\":%d,\"concurrent_admission\":%d,"
|
||||||
|
- "\"active_tile_tasks\":%d,\"active_task_guard\":%d"),
|
||||||
|
+ "\"prefetch_compile_tiles\":%d,"
|
||||||
|
+ "\"active_tile_tasks\":%d,\"active_generation_workers\":%d,"
|
||||||
|
+ "\"active_task_guard\":%d,\"game_thread_id\":%u"),
|
||||||
|
CompileSerial, CoreRegionCompileRequestSerial,
|
||||||
|
CompileCenter.X, CompileCenter.Y, CompileCenter.Z,
|
||||||
|
MissingTiles.Num(), RetainedTiles.Num(), bAllowConcurrentGeneration ? 1 : 0,
|
||||||
|
+ PrefetchTiles.Num(),
|
||||||
|
ActiveTileTaskCount.load(std::memory_order_relaxed),
|
||||||
|
- ActiveTaskCount.load(std::memory_order_relaxed)));
|
||||||
|
+ ActiveGenerationWorkerCount.load(std::memory_order_relaxed),
|
||||||
|
+ ActiveTaskCount.load(std::memory_order_relaxed), FPlatformTLS::GetCurrentThreadId()));
|
||||||
|
}
|
||||||
|
ActiveTaskCount.fetch_add(1, std::memory_order_relaxed);
|
||||||
|
UE::Tasks::Launch(TEXT("VoxelForgeCoreTileRegion"),
|
||||||
|
- [this, State, MissingTiles, RetainedTiles, TaskEpoch, TaskDesiredEpoch, WorkerLimit,
|
||||||
|
- bAllowConcurrentGeneration, CompileSerial]()
|
||||||
|
+ [this, State, MissingTiles, RetainedTiles, PrefetchTiles, CompileCenter, TaskEpoch,
|
||||||
|
+ TaskDesiredEpoch, RequestSerial, WorkerLimit, bAllowConcurrentGeneration, CompileSerial]()
|
||||||
|
{
|
||||||
|
struct FTaskGuard
|
||||||
|
{
|
||||||
|
@@ -11407,12 +11622,23 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||||
|
~FTaskGuard() { Counter.fetch_sub(1, std::memory_order_release); }
|
||||||
|
} Guard{ActiveTaskCount};
|
||||||
|
|
||||||
|
+ FCoreRegionCompileResult Result;
|
||||||
|
+ Result.RuntimeState = State;
|
||||||
|
+ Result.GenerationEpoch = TaskEpoch;
|
||||||
|
+ Result.DesiredEpoch = TaskDesiredEpoch;
|
||||||
|
+ Result.Center = CompileCenter;
|
||||||
|
+ Result.RequestSerial = RequestSerial;
|
||||||
|
+ Result.CompileSerial = CompileSerial;
|
||||||
|
+ Result.PrefetchTiles = PrefetchTiles;
|
||||||
|
+ Result.WorkerStartCycles = FPlatformTime::Cycles64();
|
||||||
|
+ Result.WorkerThreadId = FPlatformTLS::GetCurrentThreadId();
|
||||||
|
+
|
||||||
|
if (bShuttingDown.load(std::memory_order_acquire)
|
||||||
|
|| State->bShuttingDown.load(std::memory_order_acquire))
|
||||||
|
{
|
||||||
|
- State->RegionCompileEndCycles.store(FPlatformTime::Cycles64(), std::memory_order_release);
|
||||||
|
- State->bRegionCompileInFlight.store(false, std::memory_order_release);
|
||||||
|
- State->bRegionCompileFinished.store(true, std::memory_order_release);
|
||||||
|
+ Result.bAborted = true;
|
||||||
|
+ Result.WorkerEndCycles = FPlatformTime::Cycles64();
|
||||||
|
+ CoreRegionCompileResults.Enqueue(MoveTemp(Result));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
std::vector<VoxelForgeCore::TileKey> Missing;
|
||||||
|
@@ -11456,38 +11682,14 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||||
|
FWriteScopeLock Lock(State->WorldLock);
|
||||||
|
CompileAndRetain();
|
||||||
|
}
|
||||||
|
- State->RegionCompileWorkerSeconds.store(CompileStats.workerSeconds,
|
||||||
|
- std::memory_order_release);
|
||||||
|
- State->bRegionCompileSucceeded.store(bCompiled, std::memory_order_release);
|
||||||
|
- if (!bCompiled)
|
||||||
|
- {
|
||||||
|
- State->bRegionCompileFailed.store(true, std::memory_order_release);
|
||||||
|
- UE_LOG(LogTemp, Error,
|
||||||
|
- TEXT("[VoxelForgeCoreRuntime] tile-region compile failed epoch=%u desired_epoch=%u: %s"),
|
||||||
|
- TaskEpoch, TaskDesiredEpoch, UTF8_TO_TCHAR(Error.c_str()));
|
||||||
|
- }
|
||||||
|
- else
|
||||||
|
- {
|
||||||
|
- {
|
||||||
|
- FScopeLock RetainedLock(&State->RetainedTilesLock);
|
||||||
|
- State->RetainedTiles.clear();
|
||||||
|
- for (const VoxelForgeCore::TileKey& Key : Retained)
|
||||||
|
- {
|
||||||
|
- State->RetainedTiles.insert(Key);
|
||||||
|
- }
|
||||||
|
- }
|
||||||
|
- State->RegionCompileTileCount.store(
|
||||||
|
- static_cast<int32>(CompiledTileCount), std::memory_order_release);
|
||||||
|
- State->bRebuildDesired.store(true, std::memory_order_release);
|
||||||
|
- UE_LOG(LogTemp, Display,
|
||||||
|
- TEXT("[VoxelForgeCoreRuntime] tile_regions_compiled=%d retained=%d epoch=%u desired_epoch=%u "
|
||||||
|
- "wall_ms=%.3f worker_s=%.3f serial=%d"),
|
||||||
|
- static_cast<int32>(CompiledTileCount), RetainedTiles.Num(), TaskEpoch, TaskDesiredEpoch,
|
||||||
|
- CompileStats.wallSeconds * 1000.0, CompileStats.workerSeconds, CompileSerial);
|
||||||
|
- }
|
||||||
|
- State->RegionCompileEndCycles.store(FPlatformTime::Cycles64(), std::memory_order_release);
|
||||||
|
- State->bRegionCompileInFlight.store(false, std::memory_order_release);
|
||||||
|
- State->bRegionCompileFinished.store(true, std::memory_order_release);
|
||||||
|
+ Result.bSucceeded = bCompiled;
|
||||||
|
+ Result.RetainedTiles = RetainedTiles;
|
||||||
|
+ Result.Error = UTF8_TO_TCHAR(Error.c_str());
|
||||||
|
+ Result.CompiledTileCount = static_cast<int32>(CompiledTileCount);
|
||||||
|
+ Result.WorkerSeconds = CompileStats.workerSeconds;
|
||||||
|
+ Result.CoreWallSeconds = CompileStats.wallSeconds;
|
||||||
|
+ Result.WorkerEndCycles = FPlatformTime::Cycles64();
|
||||||
|
+ CoreRegionCompileResults.Enqueue(MoveTemp(Result));
|
||||||
|
}, UE::Tasks::ETaskPriority::BackgroundNormal);
|
||||||
|
}
|
||||||
|
|
||||||
|
@@ -12331,6 +12533,88 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
RecordSchedulerSkip(TEXT("core_not_ready"));
|
||||||
|
return; // Core mode never falls through to the legacy generator.
|
||||||
|
}
|
||||||
|
+ FCoreRegionCompileResult RegionResult;
|
||||||
|
+ while (CoreRegionCompileResults.Dequeue(RegionResult))
|
||||||
|
+ {
|
||||||
|
+ if (RegionResult.RuntimeState.IsValid())
|
||||||
|
+ {
|
||||||
|
+ RegionResult.RuntimeState->bRegionCompileInFlight.store(
|
||||||
|
+ false, std::memory_order_release);
|
||||||
|
+ }
|
||||||
|
+ if (RegionResult.bAborted
|
||||||
|
+ || RegionResult.RuntimeState != State
|
||||||
|
+ || RegionResult.GenerationEpoch != GenerationEpoch)
|
||||||
|
+ {
|
||||||
|
+ if (VoxelForgeStartupTrace::IsActive())
|
||||||
|
+ {
|
||||||
|
+ VoxelForgeStartupTrace::RecordEvent(TEXT("core_region_compile_discard"),
|
||||||
|
+ FString::Printf(
|
||||||
|
+ TEXT("\"request_serial\":%d,\"compile_serial\":%d,"
|
||||||
|
+ "\"result_epoch\":%u,\"current_epoch\":%u,\"aborted\":%d"),
|
||||||
|
+ RegionResult.RequestSerial, RegionResult.CompileSerial,
|
||||||
|
+ RegionResult.GenerationEpoch, GenerationEpoch,
|
||||||
|
+ RegionResult.bAborted ? 1 : 0));
|
||||||
|
+ }
|
||||||
|
+ continue;
|
||||||
|
+ }
|
||||||
|
+
|
||||||
|
+ State->RegionCompileWorkerStartCycles.store(
|
||||||
|
+ RegionResult.WorkerStartCycles, std::memory_order_release);
|
||||||
|
+ State->RegionCompileEndCycles.store(
|
||||||
|
+ RegionResult.WorkerEndCycles, std::memory_order_release);
|
||||||
|
+ State->RegionCompileTileCount.store(
|
||||||
|
+ RegionResult.CompiledTileCount, std::memory_order_release);
|
||||||
|
+ State->RegionCompileWorkerThreadId.store(
|
||||||
|
+ RegionResult.WorkerThreadId, std::memory_order_release);
|
||||||
|
+ State->RegionCompileWorkerSeconds.store(
|
||||||
|
+ RegionResult.WorkerSeconds, std::memory_order_release);
|
||||||
|
+ State->RegionCompileWallSeconds.store(
|
||||||
|
+ RegionResult.CoreWallSeconds, std::memory_order_release);
|
||||||
|
+ State->bRegionCompileSucceeded.store(
|
||||||
|
+ RegionResult.bSucceeded, std::memory_order_release);
|
||||||
|
+ ActiveCoreRegionCompileCenter = RegionResult.Center;
|
||||||
|
+ ActiveCoreRegionCompileRequestSerial = RegionResult.RequestSerial;
|
||||||
|
+ TSet<FVoxelTileKey> CurrentDesiredSet;
|
||||||
|
+ for (const FVoxelTileKey& Key : DesiredSorted) CurrentDesiredSet.Add(Key);
|
||||||
|
+ ActiveCoreRegionWastedPrefetchTiles = 0;
|
||||||
|
+ for (const FVoxelTileKey& Key : RegionResult.PrefetchTiles)
|
||||||
|
+ {
|
||||||
|
+ if (!CurrentDesiredSet.Contains(Key) && !CoreRegionLookaheadTiles.Contains(Key))
|
||||||
|
+ {
|
||||||
|
+ ++ActiveCoreRegionWastedPrefetchTiles;
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ if (RegionResult.bSucceeded)
|
||||||
|
+ {
|
||||||
|
+ {
|
||||||
|
+ FScopeLock RetainedLock(&State->RetainedTilesLock);
|
||||||
|
+ State->RetainedTiles.clear();
|
||||||
|
+ for (const FVoxelTileKey& Key : RegionResult.RetainedTiles)
|
||||||
|
+ {
|
||||||
|
+ State->RetainedTiles.insert({
|
||||||
|
+ {Key.Coord.X, Key.Coord.Y, Key.Coord.Z}, Key.Level});
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ State->bRebuildDesired.store(true, std::memory_order_release);
|
||||||
|
+ UE_LOG(LogTemp, Display,
|
||||||
|
+ TEXT("[VoxelForgeCoreRuntime] tile_regions_compiled=%d retained=%d "
|
||||||
|
+ "epoch=%u desired_epoch=%u wall_ms=%.3f worker_s=%.3f serial=%d"),
|
||||||
|
+ RegionResult.CompiledTileCount, RegionResult.RetainedTiles.Num(),
|
||||||
|
+ RegionResult.GenerationEpoch, RegionResult.DesiredEpoch,
|
||||||
|
+ RegionResult.CoreWallSeconds * 1000.0, RegionResult.WorkerSeconds,
|
||||||
|
+ RegionResult.CompileSerial);
|
||||||
|
+ }
|
||||||
|
+ else
|
||||||
|
+ {
|
||||||
|
+ State->bRegionCompileFailed.store(true, std::memory_order_release);
|
||||||
|
+ UE_LOG(LogTemp, Error,
|
||||||
|
+ TEXT("[VoxelForgeCoreRuntime] tile-region compile failed epoch=%u "
|
||||||
|
+ "desired_epoch=%u: %s"),
|
||||||
|
+ RegionResult.GenerationEpoch, RegionResult.DesiredEpoch,
|
||||||
|
+ *RegionResult.Error);
|
||||||
|
+ }
|
||||||
|
+ State->bRegionCompileFinished.store(true, std::memory_order_release);
|
||||||
|
+ }
|
||||||
|
if (State->bRebuildDesired.exchange(false, std::memory_order_acq_rel))
|
||||||
|
{
|
||||||
|
bForceDesiredRebuild = true;
|
||||||
|
@@ -12345,9 +12629,17 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
State->RegionCompileStartCycles.load(std::memory_order_acquire);
|
||||||
|
const uint64 CompileEndCycles =
|
||||||
|
State->RegionCompileEndCycles.load(std::memory_order_acquire);
|
||||||
|
+ const uint64 CompileWorkerStartCycles =
|
||||||
|
+ State->RegionCompileWorkerStartCycles.load(std::memory_order_acquire);
|
||||||
|
const double CompileElapsedMilliseconds = CompileEndCycles >= CompileStartCycles
|
||||||
|
? FPlatformTime::ToSeconds64(CompileEndCycles - CompileStartCycles) * 1000.0
|
||||||
|
: 0.0;
|
||||||
|
+ const double CompileQueueMilliseconds = CompileWorkerStartCycles >= CompileStartCycles
|
||||||
|
+ ? FPlatformTime::ToSeconds64(CompileWorkerStartCycles - CompileStartCycles) * 1000.0
|
||||||
|
+ : 0.0;
|
||||||
|
+ const double CompileTaskMilliseconds = CompileEndCycles >= CompileWorkerStartCycles
|
||||||
|
+ ? FPlatformTime::ToSeconds64(CompileEndCycles - CompileWorkerStartCycles) * 1000.0
|
||||||
|
+ : 0.0;
|
||||||
|
const double GateElapsedMilliseconds = CoreRegionPendingSinceSeconds > 0.0
|
||||||
|
? FMath::Max(0.0, FPlatformTime::Seconds() - CoreRegionPendingSinceSeconds) * 1000.0
|
||||||
|
: 0.0;
|
||||||
|
@@ -12355,6 +12647,10 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
State->RegionCompileTileCount.load(std::memory_order_acquire);
|
||||||
|
const double CompileWorkerSeconds =
|
||||||
|
State->RegionCompileWorkerSeconds.load(std::memory_order_acquire);
|
||||||
|
+ const double CompileCoreWallSeconds =
|
||||||
|
+ State->RegionCompileWallSeconds.load(std::memory_order_acquire);
|
||||||
|
+ const uint32 CompileWorkerThreadId =
|
||||||
|
+ State->RegionCompileWorkerThreadId.load(std::memory_order_acquire);
|
||||||
|
const int32 CompileSerial =
|
||||||
|
State->RegionCompileSerial.load(std::memory_order_acquire);
|
||||||
|
const uint64 IndependentSubmissions = CoreRegionIndependentSubmissions;
|
||||||
|
@@ -12364,16 +12660,23 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
VoxelForgeStartupTrace::RecordEvent(TEXT("core_region_compile_complete"), FString::Printf(
|
||||||
|
TEXT("\"compile_serial\":%d,\"request_serial\":%d,\"center\":[%d,%d,%d],"
|
||||||
|
"\"success\":%d,\"compiled_tiles\":%d,\"compile_elapsed_ms\":%s,"
|
||||||
|
- "\"compile_worker_s\":%s,\"pending_gate_elapsed_ms\":%s,"
|
||||||
|
- "\"independent_submissions\":%llu,"
|
||||||
|
+ "\"compile_queue_ms\":%s,\"compile_task_ms\":%s,"
|
||||||
|
+ "\"compile_core_wall_ms\":%s,\"compile_worker_s\":%s,"
|
||||||
|
+ "\"compile_worker_thread_id\":%u,\"pending_gate_elapsed_ms\":%s,"
|
||||||
|
+ "\"independent_submissions\":%llu,\"wasted_prefetch_tiles\":%d,"
|
||||||
|
"\"dependent_declines\":%llu,\"concurrent_admission\":%d"),
|
||||||
|
CompileSerial, ActiveCoreRegionCompileRequestSerial,
|
||||||
|
ActiveCoreRegionCompileCenter.X, ActiveCoreRegionCompileCenter.Y,
|
||||||
|
ActiveCoreRegionCompileCenter.Z, bRegionCompileSucceeded ? 1 : 0,
|
||||||
|
CompiledTileCount, *FString::SanitizeFloat(CompileElapsedMilliseconds),
|
||||||
|
+ *FString::SanitizeFloat(CompileQueueMilliseconds),
|
||||||
|
+ *FString::SanitizeFloat(CompileTaskMilliseconds),
|
||||||
|
+ *FString::SanitizeFloat(CompileCoreWallSeconds * 1000.0),
|
||||||
|
*FString::SanitizeFloat(CompileWorkerSeconds),
|
||||||
|
+ CompileWorkerThreadId,
|
||||||
|
*FString::SanitizeFloat(GateElapsedMilliseconds),
|
||||||
|
static_cast<unsigned long long>(IndependentSubmissions),
|
||||||
|
+ ActiveCoreRegionWastedPrefetchTiles,
|
||||||
|
static_cast<unsigned long long>(DependentDeclines),
|
||||||
|
GVoxelForgeCoreRegionConcurrentAdmission != 0 ? 1 : 0));
|
||||||
|
}
|
||||||
|
@@ -12388,8 +12691,9 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
{
|
||||||
|
FScopeLock RetainedLock(&State->RetainedTilesLock);
|
||||||
|
bRetainedSetMatchesDesired =
|
||||||
|
- DesiredSorted.Num() == static_cast<int32>(State->RetainedTiles.size());
|
||||||
|
- for (const FVoxelTileKey& Key : DesiredSorted)
|
||||||
|
+ CoreRegionRetainedDesiredTiles.Num()
|
||||||
|
+ == static_cast<int32>(State->RetainedTiles.size());
|
||||||
|
+ for (const FVoxelTileKey& Key : CoreRegionRetainedDesiredTiles)
|
||||||
|
{
|
||||||
|
const VoxelForgeCore::TileKey CoreKey{
|
||||||
|
{Key.Coord.X, Key.Coord.Y, Key.Coord.Z}, Key.Level};
|
||||||
|
@@ -12431,7 +12735,7 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
{
|
||||||
|
PendingCoreRegionTileSet.Add(Key);
|
||||||
|
}
|
||||||
|
- PendingCoreRetainedTiles = DesiredSorted;
|
||||||
|
+ PendingCoreRetainedTiles = CoreRegionRetainedDesiredTiles;
|
||||||
|
PendingCoreRegionRequestCenter = DesiredEpochCenterChunk;
|
||||||
|
bCoreRegionCompilePending = true;
|
||||||
|
StartPendingCoreRegionCompile();
|
||||||
|
@@ -12584,6 +12888,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
&& bWalkableRingReadyForReserve;
|
||||||
|
int32 SchedulerSubmittedByLevel[9] = {};
|
||||||
|
int32 SchedulerLoadAttemptsByLevel[9] = {};
|
||||||
|
+ int32 SchedulerRegionCompileDeclinesByLevel[9] = {};
|
||||||
|
+ int32 SchedulerFarParentDeclinesByLevel[9] = {};
|
||||||
|
int32 SchedulerLevelCapReturns[9] = {};
|
||||||
|
int32 SchedulerLevelCapBreaks[9] = {};
|
||||||
|
int32 SchedulerCriticalFullCapBreaks = 0;
|
||||||
|
@@ -12951,7 +13257,21 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
if(CoreState.IsValid()
|
||||||
|
&&CoreState->bFarWorldRequired.load(std::memory_order_acquire)
|
||||||
|
&&!CoreState->bFarWorldReady.load(std::memory_order_acquire))
|
||||||
|
+ {
|
||||||
|
+ if (bTraceStreamingScheduler && Tile.Level >= 0 && Tile.Level < 9)
|
||||||
|
+ {
|
||||||
|
+ ++SchedulerFarParentDeclinesByLevel[Tile.Level];
|
||||||
|
+ }
|
||||||
|
return;
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ if (bTraceStreamingScheduler && bUseVoxelForgeCore && bCoreRegionCompilePending
|
||||||
|
+ && GVoxelForgeCoreRegionConcurrentAdmission != 0
|
||||||
|
+ && (PendingCoreRegionTileSet.Contains(Tile)
|
||||||
|
+ || CoreRegionCompileTilesInFlight.Contains(Tile))
|
||||||
|
+ && Tile.Level >= 0 && Tile.Level < 9)
|
||||||
|
+ {
|
||||||
|
+ ++SchedulerRegionCompileDeclinesByLevel[Tile.Level];
|
||||||
|
}
|
||||||
|
if (Tile.Level >= 0 && Tile.Level < 9)
|
||||||
|
{
|
||||||
|
@@ -13661,10 +13981,37 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
const double BusySeconds = FPlatformTime::ToSeconds64(WorkerCyclesDelta);
|
||||||
|
const double SchedulerUpdateMilliseconds = (SchedulerTraceNow
|
||||||
|
- StreamingUpdateStartSeconds) * 1000.0;
|
||||||
|
+ int32 CriticalMissingTiles = 0;
|
||||||
|
+ for (const FVoxelTileKey& Tile : CriticalDesiredTiles)
|
||||||
|
+ {
|
||||||
|
+ if (!LoadedTiles.Contains(Tile) && !CollisionNotRequiredTiles.Contains(Tile))
|
||||||
|
+ {
|
||||||
|
+ ++CriticalMissingTiles;
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ int32 CollisionGateMissingTiles = 0;
|
||||||
|
+ for (const FVoxelTileKey& Tile : CollisionGateDesiredTiles)
|
||||||
|
+ {
|
||||||
|
+ if (!LoadedTiles.Contains(Tile) && !CollisionNotRequiredTiles.Contains(Tile))
|
||||||
|
+ {
|
||||||
|
+ ++CollisionGateMissingTiles;
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ const FIntVector DesiredCenterLag = DesiredEpochCenterChunk - CenterChunk;
|
||||||
|
+ const TSharedPtr<FVoxelForgeCoreRuntimeState, ESPMode::ThreadSafe> TraceCoreState =
|
||||||
|
+ VoxelForgeCoreState;
|
||||||
|
+ const bool bFarWorldRequiredForTrace = TraceCoreState.IsValid()
|
||||||
|
+ && TraceCoreState->bFarWorldRequired.load(std::memory_order_relaxed);
|
||||||
|
+ const bool bFarWorldReadyForTrace = TraceCoreState.IsValid()
|
||||||
|
+ && TraceCoreState->bFarWorldReady.load(std::memory_order_relaxed);
|
||||||
|
VoxelForgeStartupTrace::RecordEvent(TEXT("scheduler_admission"), FString::Printf(
|
||||||
|
TEXT("\"frame\":%llu,\"center\":[%d,%d,%d],\"desired_rebuilt\":%d,"
|
||||||
|
"\"submission_loop_ran\":%d,\"all_chunks_loaded\":%d,"
|
||||||
|
"\"walkable_ready\":%d,\"moving\":%d,"
|
||||||
|
+ "\"pawn_collision_gate_engaged\":%d,\"critical_tiles_ready\":%d,"
|
||||||
|
+ "\"critical_missing\":%d,\"collision_gate_missing\":%d,"
|
||||||
|
+ "\"desired_center_lag\":[%d,%d,%d],\"desired_center_current\":%d,"
|
||||||
|
+ "\"far_world_required\":%d,\"far_world_ready\":%d,"
|
||||||
|
"\"core_region_compile_pending\":%d,\"core_ready\":%d,\"coarse_first\":%d,"
|
||||||
|
"\"dynamic_near_reserve\":%d,\"legacy_pending_result_cap\":%d,"
|
||||||
|
"\"core_region_concurrent_admission\":%d,"
|
||||||
|
@@ -13685,6 +14032,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
"\"level_candidate_scan_ms\":%s,\"passage_score_ms\":%s,"
|
||||||
|
"\"level_sort_ms\":%s,\"level_admit_ms\":%s,"
|
||||||
|
"\"desired_missing_by_lod\":%s,\"loaded_by_lod\":%s,\"pending_by_lod\":%s,"
|
||||||
|
+ "\"region_compile_declines_by_lod\":%s,"
|
||||||
|
+ "\"far_parent_declines_by_lod\":%s,"
|
||||||
|
"\"submitted_by_lod\":%s,\"load_attempts_by_lod\":%s,"
|
||||||
|
"\"admission_cap_by_lod\":%s,\"declined_rule_by_lod\":%s,"
|
||||||
|
"\"level_cap_returns_by_lod\":%s,\"level_cap_breaks_by_lod\":%s,"
|
||||||
|
@@ -13695,6 +14044,11 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
static_cast<unsigned long long>(GFrameCounter), CenterChunk.X, CenterChunk.Y, CenterChunk.Z,
|
||||||
|
bDesiredRebuiltThisUpdate ? 1 : 0, bSubmissionLoopRan ? 1 : 0, bAllChunksLoaded ? 1 : 0,
|
||||||
|
bWalkableRingReadyForReserve ? 1 : 0, bPawnMoving ? 1 : 0,
|
||||||
|
+ bPawnCollisionGateEngaged ? 1 : 0, CriticalMissingTiles == 0 ? 1 : 0,
|
||||||
|
+ CriticalMissingTiles, CollisionGateMissingTiles,
|
||||||
|
+ DesiredCenterLag.X, DesiredCenterLag.Y, DesiredCenterLag.Z,
|
||||||
|
+ DesiredCenterLag == FIntVector::ZeroValue ? 1 : 0,
|
||||||
|
+ bFarWorldRequiredForTrace ? 1 : 0, bFarWorldReadyForTrace ? 1 : 0,
|
||||||
|
bCoreRegionCompilePending ? 1 : 0, IsVoxelForgeCoreReady() ? 1 : 0,
|
||||||
|
GVoxelForgeStreamingCoarseFirst, GVoxelForgeStreamingDynamicNearReserve,
|
||||||
|
GVoxelForgeStreamingLegacyPendingResultCap,
|
||||||
|
@@ -13721,7 +14075,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
*FString::SanitizeFloat(LevelSortMilliseconds),
|
||||||
|
*FString::SanitizeFloat(LevelAdmissionMilliseconds),
|
||||||
|
*IntArrayJson(DesiredMissingByLevel), *IntArrayJson(LoadedByLevel),
|
||||||
|
- *IntArrayJson(PendingByLevel), *IntArrayJson(SchedulerSubmittedByLevel),
|
||||||
|
+ *IntArrayJson(PendingByLevel), *IntArrayJson(SchedulerRegionCompileDeclinesByLevel),
|
||||||
|
+ *IntArrayJson(SchedulerFarParentDeclinesByLevel), *IntArrayJson(SchedulerSubmittedByLevel),
|
||||||
|
*IntArrayJson(SchedulerLoadAttemptsByLevel), *IntArrayJson(AdmissionCapByLevel),
|
||||||
|
*IntArrayJson(DeclinedRuleByLevel), *IntArrayJson(SchedulerLevelCapReturns),
|
||||||
|
*IntArrayJson(SchedulerLevelCapBreaks), FullCapDeclinedTiles, ReserveDeclinedTiles,
|
||||||
|
diff --git a/Source/VoxelForge/Public/VoxelWorld.h b/Source/VoxelForge/Public/VoxelWorld.h
|
||||||
|
index b9fa34c..5979f29 100644
|
||||||
|
--- a/Source/VoxelForge/Public/VoxelWorld.h
|
||||||
|
+++ b/Source/VoxelForge/Public/VoxelWorld.h
|
||||||
|
@@ -207,6 +207,28 @@ struct FChunkResult
|
||||||
|
TArray<FVector> SurfaceWitnesses;
|
||||||
|
};
|
||||||
|
|
||||||
|
+/** Worker-to-game-thread completion for immutable Core tile-region publication. */
|
||||||
|
+struct FCoreRegionCompileResult
|
||||||
|
+{
|
||||||
|
+ TSharedPtr<FVoxelForgeCoreRuntimeState, ESPMode::ThreadSafe> RuntimeState;
|
||||||
|
+ TArray<FVoxelTileKey> RetainedTiles;
|
||||||
|
+ TArray<FVoxelTileKey> PrefetchTiles;
|
||||||
|
+ FString Error;
|
||||||
|
+ FIntVector Center = FIntVector::ZeroValue;
|
||||||
|
+ uint32 GenerationEpoch = 0;
|
||||||
|
+ uint32 DesiredEpoch = 0;
|
||||||
|
+ uint64 WorkerStartCycles = 0;
|
||||||
|
+ uint64 WorkerEndCycles = 0;
|
||||||
|
+ uint32 WorkerThreadId = 0;
|
||||||
|
+ int32 CompileSerial = 0;
|
||||||
|
+ int32 RequestSerial = 0;
|
||||||
|
+ int32 CompiledTileCount = 0;
|
||||||
|
+ double WorkerSeconds = 0.0;
|
||||||
|
+ double CoreWallSeconds = 0.0;
|
||||||
|
+ bool bSucceeded = false;
|
||||||
|
+ bool bAborted = false;
|
||||||
|
+};
|
||||||
|
+
|
||||||
|
/** One worker-produced mesh identity emitted only by the explicit parity harness. */
|
||||||
|
struct FVoxelTileHashDumpRecord
|
||||||
|
{
|
||||||
|
@@ -798,6 +820,11 @@ private:
|
||||||
|
TArray<FVoxelTileKey> PendingCoreRegionTiles;
|
||||||
|
TSet<FVoxelTileKey> PendingCoreRegionTileSet;
|
||||||
|
TArray<FVoxelTileKey> PendingCoreRetainedTiles;
|
||||||
|
+ TArray<FVoxelTileKey> CoreRegionRetainedDesiredTiles;
|
||||||
|
+ TArray<FVoxelTileKey> CoreRegionLookaheadTiles;
|
||||||
|
+ FIntVector LastCoreRegionLookaheadCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
|
||||||
|
+ FIntVector LastCoreRegionLookaheadStep = FIntVector::ZeroValue;
|
||||||
|
+ uint32 LastCoreRegionLookaheadGenerationEpoch = 0;
|
||||||
|
TSet<FVoxelTileKey> CoreRegionCompileTilesInFlight;
|
||||||
|
FIntVector LastCoreRegionRequestCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
|
||||||
|
FIntVector PendingCoreRegionRequestCenter = FIntVector::ZeroValue;
|
||||||
|
@@ -810,6 +837,7 @@ private:
|
||||||
|
uint64 CoreRegionDependentDeclines = 0;
|
||||||
|
int32 CoreRegionCompileRequestSerial = 0;
|
||||||
|
int32 ActiveCoreRegionCompileRequestSerial = 0;
|
||||||
|
+ int32 ActiveCoreRegionWastedPrefetchTiles = 0;
|
||||||
|
|
||||||
|
public:
|
||||||
|
|
||||||
|
@@ -1139,6 +1167,7 @@ public:
|
||||||
|
TQueue<FChunkResult, EQueueMode::Mpsc> EditedProcessQueue;
|
||||||
|
TQueue<FChunkResult, EQueueMode::Mpsc> NearProcessQueue;
|
||||||
|
TQueue<FChunkResult, EQueueMode::Mpsc> ProcessQueue;
|
||||||
|
+ TQueue<FCoreRegionCompileResult, EQueueMode::Mpsc> CoreRegionCompileResults;
|
||||||
|
// The cap counts completed MPSC results plus admitted tile tasks, so a slow game-thread apply
|
||||||
|
// cannot accumulate an unbounded payload queue while completed work never consumes worker slots.
|
||||||
|
// Le plafond compte les resultats MPSC et les taches admises; un apply lent ne peut donc pas
|
||||||
|
@@ -1362,6 +1391,9 @@ public:
|
||||||
|
float HeadlessStreamingTestSpeedCmPerSecond = 0.0f;
|
||||||
|
float HeadlessStreamingTestStartSeconds = 0.0f;
|
||||||
|
float HeadlessStreamingTestMoveDurationSeconds = 0.0f;
|
||||||
|
+ float HeadlessStreamingTestTurnAfterSeconds = 0.0f;
|
||||||
|
+ bool bHeadlessStreamingTestTurnDone = false;
|
||||||
|
+ FVector HeadlessStreamingTestTurnDirection = FVector::YAxisVector;
|
||||||
|
float HeadlessStreamingTestExitSeconds = 0.0f;
|
||||||
|
float HeadlessStreamingTestTeleportAtSeconds = 0.0f;
|
||||||
|
float HeadlessStreamingTestTeleportDistanceCm = 0.0f;
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
# L124 — early arrival tiles
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Measured **not worth shipping** the design that independently builds pages for the nearest 27 LOD0 tiles. The existing shape gate is real, but this page set can remove only a small amount of work from the LOD0–2 completion path. Its estimated best case is below 100 ms for every stratum/worker pair except the unadjusted FI W8 tile-work ceiling; after page work, the largest estimate is 62 ms. VS is negative. No second design or runtime change was pursued.
|
||||||
|
|
||||||
|
## Current L80 timings
|
||||||
|
|
||||||
|
Five interleaved W8/W12 runs per stratum, Clang 19, policy 3, L80 protocol, source stamp `3f48d5ef6dfd7e735afcd824b742f997179810acd3ca76601a99c1131efb482e`. Values below are medians. `Near ready` is the complete LOD0–2 set.
|
||||||
|
|
||||||
|
The host load was 0.5–1.7 before baseline timing. The unchanged headless binary was built first; no build overlapped a timing run.
|
||||||
|
|
||||||
|
| Stratum | LOD0–2 tiles | Shape wall W8/W12 | First tile ready W8/W12 | Near ready W8/W12 |
|
||||||
|
|---|---:|---:|---:|---:|
|
||||||
|
| SW | 1,408 | 208/204 ms | 280/281 ms | 709/690 ms |
|
||||||
|
| FI | 1,216 | 182/179 ms | 261/261 ms | 1,441/1,329 ms |
|
||||||
|
| TN | 1,344 | 205/210 ms | 284/275 ms | 1,372/1,243 ms |
|
||||||
|
| VS | 1,344 | 226/221 ms | 291/283 ms | 992/835 ms |
|
||||||
|
|
||||||
|
A separate cold timeline trace put shape-build start at 58–60 ms after process start and shape ready at 233–279 ms. The first LOD0–2 admission followed 0.65–0.70 ms later; there were zero LOD0–2 admissions before shape ready. The exact per-run records are in `Saved/Linux/l124/timing/` and `Saved/Linux/l124/workers/`.
|
||||||
|
|
||||||
|
## What a tile reads
|
||||||
|
|
||||||
|
`World::compile` publishes `shapeReady` only after `FStandaloneShapeWorld::Build` finishes. Shape-dependent `generateTile` calls wait for that publication. A tile’s mesher uses `ClassifyBoxDetailed`, lattice density queries, and off-grid `DensityAtUnclamped` queries for gradients and seam normals. Density dispatch can read native surface/island/lattice fields, or packed `RockWords`, `CandidateWords`, and `SurfaceDistanceQ`; fallback queries use analytic density. Composition also reads stratum transitions, cavern fields, stamps/edits, and passage carve data. The tile query scope includes a one-step normal halo and selects passage features, while passage void-crossing plans and primitive/field indices must already be valid.
|
||||||
|
|
||||||
|
An exact local page is feasible: the existing full page builder constructs those dependencies. A temporary probe built a 128-voxel LOD0 page independently after base compilation and compared the center tile’s density, mesh, payload, and topology hashes. All four matched for SW, FI, TN, and VS at W8 and W12. This covered one tile per fixture, not the full 27-tile ring. The normal `compileTileRegion` path still requires the base shape and forwards page queries to it, so it cannot currently publish that page early.
|
||||||
|
|
||||||
|
## Cost count
|
||||||
|
|
||||||
|
An exact 128-voxel page took a median 7.5–12.6 ms wall per page (three interleaved W8/W12 probes); reported parallel-stage worker time was about 27–63 ms. The nearest 27 LOD0 tiles span 1 SW page, 8 FI pages, 1 TN page, and 4 VS pages. Their measured tile-service CPU and an optimistic cost-adjusted budget are:
|
||||||
|
|
||||||
|
| Stratum | Ring tile CPU W8/W12 | Estimated net budget W8/W12 |
|
||||||
|
|---|---:|---:|
|
||||||
|
| SW | 439/770 ms | 51/62 ms |
|
||||||
|
| FI | 818/1,134 ms | 41/57 ms |
|
||||||
|
| TN | 402/577 ms | 43/43 ms |
|
||||||
|
| VS | 96/175 ms | −20/−5 ms |
|
||||||
|
|
||||||
|
The budget subtracts reported page worker time for the required page count, then divides by stream workers. That counter excludes local primitive-build CPU. It also assumes all 27 tile jobs leave the post-shape queue and ignores contention, so it is an optimistic estimate, not an observed A/B speedup.
|
||||||
|
|
||||||
|
During the base build, stream workers were idle for 8.00/8 or 12.00/12 worker-equivalents. Shape helper stages averaged 7.1–7.97 active workers of the configured 8; detail work alone occupied 52–87 ms wall. Whole-build process use averaged 4.6–5.3 of the 12 pinned CPUs. Early page and tile work would overlap the periods where the shape helper pool is full. Given the sub-100 ms estimate and VS’s negative budget, I stopped before adding publication and scheduling code.
|
||||||
|
|
||||||
|
## Validation and artifacts
|
||||||
|
|
||||||
|
The temporary page-cost/equivalence hooks were removed and the headless binary rebuilt against the unchanged L123 source stamp. No 24/24 hash matrix, unpublished-page poison assertion, 30 m/s trace, fast tier, or Linux UE candidate timing was run because no runtime change was retained. The independent-page hash probe was only a viability check after full shape publication. Baseline logs are in `Saved/Linux/l124/{timing,workers,page-cost,page-equivalence}/`.
|
||||||
@@ -0,0 +1,148 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Five interleaved L80-protocol W8/W12 cold-readiness pairs per owner stratum."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import statistics
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
BIN = ROOT / "Saved/Linux/l124/bin/VoxelForgeCoreHeadless-l124-clang"
|
||||||
|
OUT = ROOT / "Saved/Linux/l124/timing"
|
||||||
|
STAMP_TOOL = ROOT / "Standalone/Tools/verify/source_stamp.py"
|
||||||
|
CAMERAS = {
|
||||||
|
"sw": (8.0, 16.0, -372.2626953125),
|
||||||
|
"fi": (25.8369140625, 3.19921875, -737.0),
|
||||||
|
"tn": (19.166015625, -14.2001953125, -753.3740234375),
|
||||||
|
"vs": (-20.0625, -5.068359375, -900.353515625),
|
||||||
|
}
|
||||||
|
FIELDS = re.compile(r"([a-zA-Z0-9_+.-]+)=([^ ]+)")
|
||||||
|
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M)
|
||||||
|
SHAPE_FREE = re.compile(
|
||||||
|
r"^\[ShapeReadiness\] shape_free_desired=(\d+)/(\d+) near_ring=(\d+)/27",
|
||||||
|
re.M,
|
||||||
|
)
|
||||||
|
PROFILE = re.compile(r"^\[ShapeProfile\] (.*)$", re.M)
|
||||||
|
STAGE = re.compile(r"^\[ShapeStage\] (.*)$", re.M)
|
||||||
|
MILESTONE = re.compile(r"^\[PipelineMilestone\] (.*)$", re.M)
|
||||||
|
NEAR = re.compile(
|
||||||
|
r"^\[WorldAssemblerNearLODReady\] lods=0-2 tiles=(\d+)/(\d+) cold_ready_s=([0-9.]+)$",
|
||||||
|
re.M,
|
||||||
|
)
|
||||||
|
PIPELINE = re.compile(r"^\[StreamPipeline\] (.*)$", re.M)
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, workers: int, phase: str, replicate: int) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
label = f"{stratum}-w{workers}-{phase}{replicate}"
|
||||||
|
log = OUT / f"{label}.log"
|
||||||
|
camera = ",".join(f"{value:.12f}" for value in CAMERAS[stratum]) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", str(BIN), str(DATA), "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", f"--workers={workers}",
|
||||||
|
"--clip-radius=3", "--max-clip-level=8", "--max-level=8",
|
||||||
|
f"--camera={camera}", "--profile-lod-work", "--stream-scheduler-policy=3",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as output:
|
||||||
|
result = subprocess.run(command, cwd=ROOT, stdout=output,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
content = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
near, pipeline_match = NEAR.search(content), PIPELINE.search(content)
|
||||||
|
profile, stage, free = PROFILE.search(content), STAGE.search(content), SHAPE_FREE.search(content)
|
||||||
|
stamp = STAMP.search(content)
|
||||||
|
milestones = dict(FIELDS.findall(MILESTONE.search(content).group(1))) if MILESTONE.search(content) else {}
|
||||||
|
if result.returncode or near is None or pipeline_match is None or not all((profile, stage, free, stamp)):
|
||||||
|
raise RuntimeError(f"baseline run incomplete: {log}; exit={result.returncode}")
|
||||||
|
if stamp.group(1) != EXPECTED_STAMP:
|
||||||
|
raise RuntimeError(f"source stamp mismatch: {log}; got={stamp.group(1)} expected={EXPECTED_STAMP}")
|
||||||
|
pipeline = dict(FIELDS.findall(pipeline_match.group(1)))
|
||||||
|
profile_values = dict(FIELDS.findall(profile.group(1)))
|
||||||
|
stage_values = dict(FIELDS.findall(stage.group(1)))
|
||||||
|
row = {
|
||||||
|
"stratum": stratum,
|
||||||
|
"workers": workers,
|
||||||
|
"phase": phase,
|
||||||
|
"replicate": replicate,
|
||||||
|
"exit_code": result.returncode,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"source_stamp": stamp.group(1),
|
||||||
|
"shape_free_tiles": int(free.group(1)),
|
||||||
|
"desired_tiles": int(free.group(2)),
|
||||||
|
"shape_free_near_ring": int(free.group(3)),
|
||||||
|
"shape_compile_s": float(pipeline["shape"].removesuffix("s")),
|
||||||
|
"first_tile_ready_s": float(pipeline["first_tile_ready"].removesuffix("s")),
|
||||||
|
"near_ready_s": float(near.group(3)),
|
||||||
|
"near_ready_tiles": int(near.group(1)),
|
||||||
|
"near_desired_tiles": int(near.group(2)),
|
||||||
|
"worker_shape_stages_ms": {
|
||||||
|
key: float(value.removesuffix("ms"))
|
||||||
|
for key, value in profile_values.items() if key.endswith("_worker")
|
||||||
|
},
|
||||||
|
"shape_stage_wall_ms": {
|
||||||
|
key: float(value)
|
||||||
|
for key, value in stage_values.items()
|
||||||
|
if key in {"local-shapes", "intervals+signs", "surface-band", "far-parents", "total"}
|
||||||
|
},
|
||||||
|
"milestones": milestones,
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L124HeadlessRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
global EXPECTED_STAMP
|
||||||
|
expected = subprocess.run(
|
||||||
|
["python3", "-B", str(STAMP_TOOL), "stamp", "--root", str(ROOT)],
|
||||||
|
cwd=ROOT, text=True, stdout=subprocess.PIPE, check=True,
|
||||||
|
).stdout.strip()
|
||||||
|
EXPECTED_STAMP = expected
|
||||||
|
if not DATA.is_file() or not BIN.is_file():
|
||||||
|
raise SystemExit(f"missing input: {DATA} or {BIN}")
|
||||||
|
rows: list[dict] = []
|
||||||
|
for stratum in CAMERAS:
|
||||||
|
for workers in (8, 12):
|
||||||
|
run(stratum, workers, "warmup", 0)
|
||||||
|
orders = ((8, 12), (12, 8), (8, 12), (12, 8), (8, 12))
|
||||||
|
for stratum in CAMERAS:
|
||||||
|
for replicate, order in enumerate(orders, 1):
|
||||||
|
paired = {}
|
||||||
|
for workers in order:
|
||||||
|
row = run(stratum, workers, "measured", replicate)
|
||||||
|
rows.append(row)
|
||||||
|
paired[workers] = row
|
||||||
|
(OUT / "runs-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print("[L124HeadlessPair] " + json.dumps({
|
||||||
|
"stratum": stratum,
|
||||||
|
"replicate": replicate,
|
||||||
|
"w8_near_ready_s": paired[8]["near_ready_s"],
|
||||||
|
"w12_near_ready_s": paired[12]["near_ready_s"],
|
||||||
|
}, sort_keys=True), flush=True)
|
||||||
|
summary = []
|
||||||
|
for stratum in CAMERAS:
|
||||||
|
entry = {"stratum": stratum, "interleaved_pairs": 5}
|
||||||
|
for workers in (8, 12):
|
||||||
|
subset = [r for r in rows if r["stratum"] == stratum and r["workers"] == workers]
|
||||||
|
entry[f"w{workers}_near_ready_s"] = [r["near_ready_s"] for r in subset]
|
||||||
|
entry[f"w{workers}_near_ready_median_s"] = statistics.median(
|
||||||
|
r["near_ready_s"] for r in subset)
|
||||||
|
entry[f"w{workers}_shape_s"] = [r["shape_compile_s"] for r in subset]
|
||||||
|
entry[f"w{workers}_shape_median_s"] = statistics.median(
|
||||||
|
r["shape_compile_s"] for r in subset)
|
||||||
|
summary.append(entry)
|
||||||
|
(OUT / "summary.json").write_text(
|
||||||
|
json.dumps({"source_stamp": EXPECTED_STAMP, "runs": rows, "summary": summary}, indent=2) + "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,151 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Collect the separate worker-idle and admission timeline for L124's unchanged headless path."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
BIN = ROOT / "Saved/Linux/l124/bin/VoxelForgeCoreHeadless-l124-clang"
|
||||||
|
OUT = ROOT / "Saved/Linux/l124/workers"
|
||||||
|
STAMP_TOOL = ROOT / "Standalone/Tools/verify/source_stamp.py"
|
||||||
|
CAMERAS = {
|
||||||
|
"sw": (8.0, 16.0, -372.2626953125),
|
||||||
|
"fi": (25.8369140625, 3.19921875, -737.0),
|
||||||
|
"tn": (19.166015625, -14.2001953125, -753.3740234375),
|
||||||
|
"vs": (-20.0625, -5.068359375, -900.353515625),
|
||||||
|
}
|
||||||
|
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M)
|
||||||
|
COLD = re.compile(
|
||||||
|
r"^\[ColdTimeline\] event=(\S+) edge=(\S+) at_ms=([0-9.]+) cpu_ms=([0-9.]+)$",
|
||||||
|
re.M,
|
||||||
|
)
|
||||||
|
TIMELINE = re.compile(
|
||||||
|
r"^\[CpuTileTimeline\] event=(\S+) key=\(([-0-9]+),([-0-9]+),([-0-9]+),lod(\d+)\) "
|
||||||
|
r"at_ms=([0-9.-]+) frame=\d+ worker_id=(-?\d+) queue_depth=(\d+) "
|
||||||
|
r"active_workers=(\d+) priority_rank=(\d+)", re.M,
|
||||||
|
)
|
||||||
|
PROFILE = re.compile(r"^\[ShapeProfile\] (.*)$", re.M)
|
||||||
|
BREAKDOWN = re.compile(r"^\[ShapeBreakdown\] (.*)$", re.M)
|
||||||
|
NEAR = re.compile(
|
||||||
|
r"^\[WorldAssemblerNearLODReady\] lods=0-2 tiles=(\d+)/(\d+) cold_ready_s=([0-9.]+)$",
|
||||||
|
re.M,
|
||||||
|
)
|
||||||
|
PAIR = re.compile(r"([a-zA-Z0-9_+.-]+)=([0-9.]+)(?:ms)?")
|
||||||
|
|
||||||
|
|
||||||
|
def intervals(events: list[tuple[float, float]], start: float, end: float) -> float:
|
||||||
|
return sum(max(0.0, min(finish, end) - max(begin, start))
|
||||||
|
for begin, finish in events)
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, workers: int, expected_stamp: str) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = OUT / f"{stratum}-w{workers}.log"
|
||||||
|
camera = ",".join(f"{value:.12f}" for value in CAMERAS[stratum]) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", str(BIN), str(DATA), "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", f"--workers={workers}",
|
||||||
|
"--clip-radius=3", "--max-clip-level=8", "--max-level=8",
|
||||||
|
f"--camera={camera}", "--profile-lod-work", "--stream-scheduler-policy=3",
|
||||||
|
"--cold-worker-timeline",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as output:
|
||||||
|
result = subprocess.run(command, cwd=ROOT, stdout=output,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
content = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
stamp = STAMP.search(content)
|
||||||
|
cold = {(event, edge): (float(at), float(cpu))
|
||||||
|
for event, edge, at, cpu in COLD.findall(content)}
|
||||||
|
timeline = [
|
||||||
|
{"event": event, "key": (int(x), int(y), int(z), int(lod)),
|
||||||
|
"at_ms": float(at), "worker_id": int(worker),
|
||||||
|
"queue_depth": int(depth), "active_workers": int(active),
|
||||||
|
"priority_rank": int(rank)}
|
||||||
|
for event, x, y, z, lod, at, worker, depth, active, rank in TIMELINE.findall(content)
|
||||||
|
]
|
||||||
|
base_begin, base_end = cold.get(("base_compile", "begin")), cold.get(("base_compile", "end"))
|
||||||
|
near = NEAR.search(content)
|
||||||
|
profile_match, breakdown_match = PROFILE.search(content), BREAKDOWN.search(content)
|
||||||
|
if (result.returncode or stamp is None or stamp.group(1) != expected_stamp
|
||||||
|
or base_begin is None or base_end is None or near is None
|
||||||
|
or profile_match is None or breakdown_match is None):
|
||||||
|
raise RuntimeError(f"worker diagnostic incomplete: {log}; exit={result.returncode}")
|
||||||
|
begin_ms, begin_cpu = base_begin
|
||||||
|
end_ms, end_cpu = base_end
|
||||||
|
admitted = [event for event in timeline if event["event"] == "admitted" and event["key"][3] <= 2]
|
||||||
|
post_compile = [event for event in admitted if event["at_ms"] >= end_ms]
|
||||||
|
idle_by_worker: dict[int, list[tuple[float, float]]] = {}
|
||||||
|
idle_begin: dict[int, float] = {}
|
||||||
|
for event in timeline:
|
||||||
|
worker = event["worker_id"]
|
||||||
|
if worker < 0:
|
||||||
|
continue
|
||||||
|
if event["event"] == "idle_begin":
|
||||||
|
idle_begin[worker] = event["at_ms"]
|
||||||
|
elif event["event"] == "idle_end" and worker in idle_begin:
|
||||||
|
idle_by_worker.setdefault(worker, []).append((idle_begin.pop(worker), event["at_ms"]))
|
||||||
|
for worker, idle_start in idle_begin.items():
|
||||||
|
idle_by_worker.setdefault(worker, []).append((idle_start, end_ms))
|
||||||
|
idle_worker_ms = sum(intervals(spans, begin_ms, end_ms)
|
||||||
|
for spans in idle_by_worker.values())
|
||||||
|
profile = {key: float(value) for key, value in PAIR.findall(profile_match.group(1))}
|
||||||
|
breakdown = {key: float(value) for key, value in PAIR.findall(breakdown_match.group(1))}
|
||||||
|
parallel_stages = {}
|
||||||
|
for name in ("rock_lines", "candidate_axes", "candidate_merge", "detail_lines"):
|
||||||
|
wall, worker = profile.get(f"{name}_wall"), profile.get(f"{name}_worker")
|
||||||
|
if wall is not None and worker is not None and wall > 0.0:
|
||||||
|
parallel_stages[name] = {
|
||||||
|
"wall_ms": wall,
|
||||||
|
"worker_ms": worker,
|
||||||
|
"average_busy_workers": worker / wall,
|
||||||
|
}
|
||||||
|
row = {
|
||||||
|
"stratum": stratum,
|
||||||
|
"workers": workers,
|
||||||
|
"exit_code": result.returncode,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"source_stamp": stamp.group(1),
|
||||||
|
"base_compile_start_ms": begin_ms,
|
||||||
|
"base_shape_ready_ms": end_ms,
|
||||||
|
"base_compile_wall_ms": end_ms - begin_ms,
|
||||||
|
"process_cpu_ms_during_compile": end_cpu - begin_cpu,
|
||||||
|
"average_process_cores_during_compile": (end_cpu - begin_cpu) / max(1e-9, end_ms - begin_ms),
|
||||||
|
"stream_workers_idle_ms_during_compile": idle_worker_ms,
|
||||||
|
"stream_idle_worker_equivalents": idle_worker_ms / max(1e-9, end_ms - begin_ms),
|
||||||
|
"first_lod0_2_admission_ms": min((event["at_ms"] for event in admitted), default=None),
|
||||||
|
"first_lod0_2_shape_dependent_admission_ms": min(
|
||||||
|
(event["at_ms"] for event in post_compile), default=None),
|
||||||
|
"near_ready_s": float(near.group(3)),
|
||||||
|
"lod0_2_admissions": len(admitted),
|
||||||
|
"lod0_2_admissions_before_shape_ready": sum(event["at_ms"] < end_ms for event in admitted),
|
||||||
|
"shape_parallel_stages": parallel_stages,
|
||||||
|
"owner_shape_worker_ms": breakdown.get("owner_shape_worker"),
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L124WorkerRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
expected_stamp = subprocess.run(
|
||||||
|
["python3", "-B", str(STAMP_TOOL), "stamp", "--root", str(ROOT)],
|
||||||
|
cwd=ROOT, text=True, stdout=subprocess.PIPE, check=True,
|
||||||
|
).stdout.strip()
|
||||||
|
rows = [run(stratum, workers, expected_stamp)
|
||||||
|
for stratum in CAMERAS for workers in (8, 12)]
|
||||||
|
(OUT / "summary.json").write_text(
|
||||||
|
json.dumps({"source_stamp": expected_stamp, "runs": rows}, indent=2) + "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
# Linux127 — smooth-tunnel merge
|
||||||
|
|
||||||
|
Target: `aeb60bb` on `merge/smooth-tunnels`.
|
||||||
|
|
||||||
|
Build, fast-tier, walkability, seam, floor, determinism, and capture checks pass. The UE moving comparison shows a repeatable coverage drop and is the item to review before landing.
|
||||||
|
|
||||||
|
- Clang and GCC headless builds pass. UE builds pass: c743 baseline 157/157 actions; merge 29/29, using `-NoUBA -MaxParallelActions=4`.
|
||||||
|
- Fast tier passes (259 checks; 5 diagnostic checks). Walkability is 21,425/21,425. Four seam pairs (4,900 shared samples) are bit-identical. Floor probe maximum is 7.884 mm.
|
||||||
|
- All 18 owner4/owner2 Clang/GCC W1/W8/W12 hash runs match the L125 v225 baselines.
|
||||||
|
- L80 timing protocol on a quiet machine (load 0.69, CPU idle 99.4%): five interleaved measurements per cell, W8/W12 across all four strates. Merge vs L125 is within -4.05% to +2.83% near-ready and -4.58% to +0.95% whole-view. Vs c743, S0 is slower (+10.90% to +14.33% near, +1.26% to +3.52% whole); the other six cells are faster.
|
||||||
|
- UE startup reaches full LOD0–2 in all four strates: c743 2.232–2.726 s; merge 2.221–2.514 s. Far-apply CVar reads back as 1 and produces applies in every start and moving run.
|
||||||
|
- Five 30 m/s movement runs per build: median near coverage 43.36% → 40.22%; far holes 3.14% → 3.46%; tick p99 11.25 → 10.80 ms; worst tick 13.97 → 13.27 ms. Both builds reach 100% near coverage and zero far holes after stopping. The lane works; moving coverage remains lower on the merge.
|
||||||
|
- 23/23 L125 poses rendered and verified from the merge binary; 19 pass pixel quality, 4 show visible low-contrast geometry. See [`captures/INDEX.md`](captures/INDEX.md) and [`captures/COMPARE.html`](captures/COMPARE.html).
|
||||||
|
|
||||||
|
No source fix was made. Logs and summaries are in `Saved/Linux/l127`. An early probe wrapper wrote to the L125 probe output; the L125 binary regenerated it, with restoration checksums recorded in `Saved/Linux/l127/restore-l125-probe.log`.
|
||||||
@@ -0,0 +1,103 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Render the exact 23 L119 camera views with the L127 merge candidate."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
sys.dont_write_bytecode = True
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
RESEARCH = Path(__file__).resolve().parent
|
||||||
|
CAPTURES = RESEARCH / "captures"
|
||||||
|
SAVED = ROOT / "Saved/Linux/l127/captures"
|
||||||
|
BASELINE_INDEX = (ROOT / "Standalone/Research/WorldAssemblerM3/Linux119"
|
||||||
|
/ "captures/L119/capture-index.json")
|
||||||
|
LAYOUT = ROOT / "Saved/Linux/l127/hashes/owner4-surface-world-w12-clang.layout.json"
|
||||||
|
CLANG = ROOT / os.environ.get("L127_CAPTURE_CLANG",
|
||||||
|
"Saved/Linux/l127/bin-clang/VoxelForgeCoreHeadless-l127-clang")
|
||||||
|
RENDERER = ROOT / os.environ.get("L127_CAPTURE_RENDERER",
|
||||||
|
"Saved/Linux/l127/bin-clang/VoxelForgeOffscreenCapture-l127-clang")
|
||||||
|
HELPER_PATH = ROOT / "Standalone/Research/WorldAssemblerM3/Linux113/RunL113Captures.py"
|
||||||
|
|
||||||
|
SPEC = importlib.util.spec_from_file_location("l113_capture_helpers", HELPER_PATH)
|
||||||
|
if SPEC is None or SPEC.loader is None:
|
||||||
|
raise SystemExit("cannot load the verified L113 capture helpers")
|
||||||
|
HELPERS = importlib.util.module_from_spec(SPEC)
|
||||||
|
SPEC.loader.exec_module(HELPERS)
|
||||||
|
HELPERS.RESEARCH = RESEARCH
|
||||||
|
HELPERS.CAPTURES = CAPTURES
|
||||||
|
HELPERS.SAVED = SAVED
|
||||||
|
HELPERS.BINARIES = {"L127": CLANG}
|
||||||
|
HELPERS.RENDERERS = {"L127": RENDERER}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
if any(not path.is_file() for path in (BASELINE_INDEX, LAYOUT, CLANG, RENDERER)):
|
||||||
|
raise SystemExit("missing L119 camera set, L127 layout, or capture binary")
|
||||||
|
baseline = json.loads(BASELINE_INDEX.read_text(encoding="utf-8"))
|
||||||
|
layout = json.loads(LAYOUT.read_text(encoding="utf-8"))
|
||||||
|
rows = []
|
||||||
|
CAPTURES.mkdir(parents=True, exist_ok=True)
|
||||||
|
SAVED.mkdir(parents=True, exist_ok=True)
|
||||||
|
for view in baseline["captures"]:
|
||||||
|
row = HELPERS.capture("L127", view, layout)
|
||||||
|
row["variant"] = "L127-merge-v225"
|
||||||
|
if view["kind"] == "rock-floor-close":
|
||||||
|
scene = ROOT / row["scene"]
|
||||||
|
anchor = HELPERS.nearest_surface_anchor(scene, view["target"], margin_m=5.0,
|
||||||
|
preferred_normal=(0.0, 0.0, 1.0), minimum_alignment=0.85)
|
||||||
|
alignment = HELPERS.dot(anchor["normal"], (0.0, 0.0, 1.0))
|
||||||
|
camera_distance = math.dist(row["camera_actual_m"], anchor["position_m"])
|
||||||
|
row.update({"floor_anchor_position_m": anchor["position_m"],
|
||||||
|
"floor_anchor_normal": anchor["normal"],
|
||||||
|
"closeup_camera_to_floor_distance_m": camera_distance,
|
||||||
|
"closeup_floor_normal_alignment": alignment,
|
||||||
|
"verified_upward_floor_normal": alignment >= 0.85})
|
||||||
|
if (not row["verified"] or not 2.0 <= camera_distance <= 3.2
|
||||||
|
or anchor["distance_m"] > 0.25 or alignment < 0.85
|
||||||
|
or row["drawn_triangles"] <= 0
|
||||||
|
or row["pixel_quality_status"] not in
|
||||||
|
("pass", "low_contrast_geometry_visible")):
|
||||||
|
raise RuntimeError(f"L127 floor close-up failed at {view['name']}")
|
||||||
|
rows.append(row)
|
||||||
|
if len(rows) != len(baseline["captures"]) or len(rows) != 23:
|
||||||
|
raise RuntimeError(f"expected all 23 L119 views, captured {len(rows)}")
|
||||||
|
index = CAPTURES / "L127" / "capture-index.json"
|
||||||
|
index.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
index.write_text(json.dumps({"captures": rows}, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
|
||||||
|
html = ["<!doctype html><meta charset='utf-8'>",
|
||||||
|
"<title>L119 / L127 tunnel capture comparison</title>",
|
||||||
|
"<style>body{font:14px system-ui;margin:24px;background:#17191d;color:#eee}"
|
||||||
|
"section{margin:28px 0}h2{font-size:18px}div{display:grid;"
|
||||||
|
"grid-template-columns:1fr 1fr;gap:12px}img{width:100%;height:auto}"
|
||||||
|
"p{color:#bbb}</style>",
|
||||||
|
"<h1>L119 / L127 merge: identical views</h1>"]
|
||||||
|
old_by_name = {Path(row["capture"]).name: row for row in baseline["captures"]}
|
||||||
|
for row in rows:
|
||||||
|
name = Path(row["capture"]).name
|
||||||
|
old = old_by_name[name]
|
||||||
|
old_path = os.path.relpath(ROOT / "Standalone/Research/WorldAssemblerM3/Linux119"
|
||||||
|
/ old["capture"], CAPTURES)
|
||||||
|
new_path = os.path.relpath(RESEARCH / row["capture"], CAPTURES)
|
||||||
|
html.extend(["<section>", f"<h2>{row['passage']} — {row['station']}</h2>",
|
||||||
|
f"<p>{row['kind']} · L119 {old['pixel_quality_status']} · "
|
||||||
|
f"L127 {row['pixel_quality_status']}</p>", "<div>",
|
||||||
|
f"<figure><figcaption>L119</figcaption><img src='{old_path}'></figure>",
|
||||||
|
f"<figure><figcaption>L127 merge</figcaption><img src='{new_path}'></figure>",
|
||||||
|
"</div>", "</section>"])
|
||||||
|
(CAPTURES / "COMPARE.html").write_text("\n".join(html) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
print(f"[L127Captures] rendered={len(rows)} views index={index.relative_to(ROOT)}",
|
||||||
|
flush=True)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,188 @@
|
|||||||
|
<!doctype html><meta charset='utf-8'>
|
||||||
|
<title>L119 / L127 tunnel capture comparison</title>
|
||||||
|
<style>body{font:14px system-ui;margin:24px;background:#17191d;color:#eee}section{margin:28px 0}h2{font-size:18px}div{display:grid;grid-template-columns:1fr 1fr;gap:12px}img{width:100%;height:auto}p{color:#bbb}</style>
|
||||||
|
<h1>L119 / L127 merge: identical views</h1>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — rock-floor/401.7m</h2>
|
||||||
|
<p>rock-floor-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-rock-floor-station-1.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-rock-floor-station-1.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — rock-floor/792.8m</h2>
|
||||||
|
<p>rock-floor-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-rock-floor-station-2.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-rock-floor-station-2.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — rock-floor/1634.5m</h2>
|
||||||
|
<p>rock-floor-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-rock-floor-station-3.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-rock-floor-station-3.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/1-2/0 — rock-floor/27.0m</h2>
|
||||||
|
<p>rock-floor-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-1-2-0-rock-floor-station-1.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-1-2-0-rock-floor-station-1.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/1-2/0 — rock-floor/52.9m</h2>
|
||||||
|
<p>rock-floor-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-1-2-0-rock-floor-station-2.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-1-2-0-rock-floor-station-2.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/1-2/0 — rock-floor/78.9m</h2>
|
||||||
|
<p>rock-floor-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-1-2-0-rock-floor-station-3.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-1-2-0-rock-floor-station-3.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/2-3/0 — rock-floor/323.4m</h2>
|
||||||
|
<p>rock-floor-close · L119 low_contrast_geometry_visible · L127 low_contrast_geometry_visible</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-2-3-0-rock-floor-station-1.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-2-3-0-rock-floor-station-1.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/2-3/0 — rock-floor/333.4m</h2>
|
||||||
|
<p>rock-floor-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-2-3-0-rock-floor-station-2.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-2-3-0-rock-floor-station-2.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/2-3/0 — rock-floor/343.4m</h2>
|
||||||
|
<p>rock-floor-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-2-3-0-rock-floor-station-3.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-2-3-0-rock-floor-station-3.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — section-1/4m-outside</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-openair-1-4m-outside.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-openair-1-4m-outside.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — section-1/4m-interior-wall</h2>
|
||||||
|
<p>open-air-close · L119 low_contrast_geometry_visible · L127 low_contrast_geometry_visible</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-openair-1-4m-interior-wall.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-openair-1-4m-interior-wall.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — section-2/4m-outside</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-openair-2-4m-outside.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-openair-2-4m-outside.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — section-2/4m-interior-wall</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-openair-2-4m-interior-wall.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-openair-2-4m-interior-wall.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — section-3/4m-outside</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-openair-3-4m-outside.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-openair-3-4m-outside.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — section-3/4m-interior-wall</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-openair-3-4m-interior-wall.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-openair-3-4m-interior-wall.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/1-2/0 — section-1/4m-outside</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-1-2-0-openair-1-4m-outside.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-1-2-0-openair-1-4m-outside.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/1-2/0 — section-1/4m-interior-wall</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-1-2-0-openair-1-4m-interior-wall.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-1-2-0-openair-1-4m-interior-wall.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/1-2/0 — section-2/4m-outside</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-1-2-0-openair-2-4m-outside.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-1-2-0-openair-2-4m-outside.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/1-2/0 — section-2/4m-interior-wall</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-1-2-0-openair-2-4m-interior-wall.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-1-2-0-openair-2-4m-interior-wall.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/2-3/0 — section-1/4m-outside</h2>
|
||||||
|
<p>open-air-close · L119 low_contrast_geometry_visible · L127 low_contrast_geometry_visible</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-2-3-0-openair-1-4m-outside.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-2-3-0-openair-1-4m-outside.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/2-3/0 — section-1/4m-interior-wall</h2>
|
||||||
|
<p>open-air-close · L119 low_contrast_geometry_visible · L127 low_contrast_geometry_visible</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-2-3-0-openair-1-4m-interior-wall.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-2-3-0-openair-1-4m-interior-wall.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/2-3/0 — section-2/4m-outside</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-2-3-0-openair-2-4m-outside.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-2-3-0-openair-2-4m-outside.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/2-3/0 — section-2/4m-interior-wall</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-2-3-0-openair-2-4m-interior-wall.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-2-3-0-openair-2-4m-interior-wall.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
# L127 merge captures
|
||||||
|
|
||||||
|
These are the 23 L125 camera poses, rendered from the merge binary.
|
||||||
|
|
||||||
|
[Open the L119/L127 side-by-side comparison](COMPARE.html).
|
||||||
|
|
||||||
|
- [passage/0-1/0 — rock-floor/401.7m (pass)](L127/passage-0-1-0-rock-floor-station-1.png)
|
||||||
|
- [passage/0-1/0 — rock-floor/792.8m (pass)](L127/passage-0-1-0-rock-floor-station-2.png)
|
||||||
|
- [passage/0-1/0 — rock-floor/1634.5m (pass)](L127/passage-0-1-0-rock-floor-station-3.png)
|
||||||
|
- [passage/1-2/0 — rock-floor/27.0m (pass)](L127/passage-1-2-0-rock-floor-station-1.png)
|
||||||
|
- [passage/1-2/0 — rock-floor/52.9m (pass)](L127/passage-1-2-0-rock-floor-station-2.png)
|
||||||
|
- [passage/1-2/0 — rock-floor/78.9m (pass)](L127/passage-1-2-0-rock-floor-station-3.png)
|
||||||
|
- [passage/2-3/0 — rock-floor/323.4m (low_contrast_geometry_visible)](L127/passage-2-3-0-rock-floor-station-1.png)
|
||||||
|
- [passage/2-3/0 — rock-floor/333.4m (pass)](L127/passage-2-3-0-rock-floor-station-2.png)
|
||||||
|
- [passage/2-3/0 — rock-floor/343.4m (pass)](L127/passage-2-3-0-rock-floor-station-3.png)
|
||||||
|
- [passage/0-1/0 — section-1/4m-outside (pass)](L127/passage-0-1-0-openair-1-4m-outside.png)
|
||||||
|
- [passage/0-1/0 — section-1/4m-interior-wall (low_contrast_geometry_visible)](L127/passage-0-1-0-openair-1-4m-interior-wall.png)
|
||||||
|
- [passage/0-1/0 — section-2/4m-outside (pass)](L127/passage-0-1-0-openair-2-4m-outside.png)
|
||||||
|
- [passage/0-1/0 — section-2/4m-interior-wall (pass)](L127/passage-0-1-0-openair-2-4m-interior-wall.png)
|
||||||
|
- [passage/0-1/0 — section-3/4m-outside (pass)](L127/passage-0-1-0-openair-3-4m-outside.png)
|
||||||
|
- [passage/0-1/0 — section-3/4m-interior-wall (pass)](L127/passage-0-1-0-openair-3-4m-interior-wall.png)
|
||||||
|
- [passage/1-2/0 — section-1/4m-outside (pass)](L127/passage-1-2-0-openair-1-4m-outside.png)
|
||||||
|
- [passage/1-2/0 — section-1/4m-interior-wall (pass)](L127/passage-1-2-0-openair-1-4m-interior-wall.png)
|
||||||
|
- [passage/1-2/0 — section-2/4m-outside (pass)](L127/passage-1-2-0-openair-2-4m-outside.png)
|
||||||
|
- [passage/1-2/0 — section-2/4m-interior-wall (pass)](L127/passage-1-2-0-openair-2-4m-interior-wall.png)
|
||||||
|
- [passage/2-3/0 — section-1/4m-outside (low_contrast_geometry_visible)](L127/passage-2-3-0-openair-1-4m-outside.png)
|
||||||
|
- [passage/2-3/0 — section-1/4m-interior-wall (low_contrast_geometry_visible)](L127/passage-2-3-0-openair-1-4m-interior-wall.png)
|
||||||
|
- [passage/2-3/0 — section-2/4m-outside (pass)](L127/passage-2-3-0-openair-2-4m-outside.png)
|
||||||
|
- [passage/2-3/0 — section-2/4m-interior-wall (pass)](L127/passage-2-3-0-openair-2-4m-interior-wall.png)
|
||||||
|
After Width: | Height: | Size: 11 KiB |
|
After Width: | Height: | Size: 98 KiB |
|
After Width: | Height: | Size: 63 KiB |
|
After Width: | Height: | Size: 111 KiB |
|
After Width: | Height: | Size: 225 KiB |
|
After Width: | Height: | Size: 177 KiB |
|
After Width: | Height: | Size: 125 KiB |
|
After Width: | Height: | Size: 112 KiB |
|
After Width: | Height: | Size: 116 KiB |
|
After Width: | Height: | Size: 50 KiB |
|
After Width: | Height: | Size: 184 KiB |
|
After Width: | Height: | Size: 34 KiB |
|
After Width: | Height: | Size: 121 KiB |
|
After Width: | Height: | Size: 111 KiB |
|
After Width: | Height: | Size: 163 KiB |
|
After Width: | Height: | Size: 132 KiB |
|
After Width: | Height: | Size: 37 KiB |
|
After Width: | Height: | Size: 55 KiB |
|
After Width: | Height: | Size: 16 KiB |
|
After Width: | Height: | Size: 67 KiB |
|
After Width: | Height: | Size: 21 KiB |
|
After Width: | Height: | Size: 146 KiB |
|
After Width: | Height: | Size: 106 KiB |
@@ -0,0 +1,9 @@
|
|||||||
|
# Linux130 — passage reach
|
||||||
|
|
||||||
|
On `wip/l130-reach-units` from `b96dd73`, passage influence is now **1.0533 m**: 0.75 m shell + 0.21 m relief + 0.50 m smooth-max support (`width/6`) + 0.01 m margin. One metres-to-core-voxels conversion feeds proof, feature, fallback and off-grid bounds; the sweep cutoff uses the same metre value. The dead 4 m cutoff is removed. Evaluator version is 227.
|
||||||
|
|
||||||
|
The debug shell/crest audit sampled 25,728 points, including 5 mm inside and outside the reach; 18,993 had sign-relevant passage contribution. Proof and feature bounds missed none. The nine-station floor probe still peaks at **7.884 mm**, matching L125. The 23 L125 views and six L129 mouth close-ups are byte-identical. Fast verification, walkability and shell audits pass; the seam audit found 0 mismatches across 4 pairs / 4,900 shared samples.
|
||||||
|
|
||||||
|
Clang/GCC W1/W8/W12 hashes agree across all 18 runs. Versus v225, hashes changed for owner4 full-detail, whole-view and LOD0–3; owner2 surface-world and floating-islands whole-view and LOD2–3. Exact-query hashes stayed unchanged. The v227 canonical baselines are recorded; changed hashes reflect the shorter density influence halo, while sampled sign-relevant bounds and captured surfaces remain covered.
|
||||||
|
|
||||||
|
L80 timing used 10 interleaved pairs per stratum/worker count. Across the eight case medians, the weighted deltas were +0.33% for LOD0–2 and +0.31% whole-view. Across 80 paired runs, mean deltas were +0.26% (95% interval −0.44% to +0.96%) and +0.47% (−0.06% to +1.00%); both intervals include zero. No measurable timing change; UE movement was skipped. The Linux UE host build succeeded in 76.30 s with `-NoUBA -MaxParallelActions=4`, without `prlimit`. `ARCHITECTURE.md` and `CODEMAP.md` had no note for this optimized influence reach, so they were unchanged.
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Run Clang/GCC owner hashes at W1/W8/W12 and compare with b96dd73."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
OUT = ROOT / "Saved/Linux/l130/hashes"
|
||||||
|
BASELINE = json.loads((ROOT / "Saved/Linux/l125/hashes/summary.json").read_text())
|
||||||
|
DATASETS = {
|
||||||
|
"owner4": ROOT / "Standalone/data/owner_world.json",
|
||||||
|
"owner2": ROOT / "Standalone/data/regression/owner_world_2strate_l61.json",
|
||||||
|
}
|
||||||
|
RECIPES = {"owner4": ("surface-world",),
|
||||||
|
"owner2": ("surface-world", "floating-islands")}
|
||||||
|
BINARIES = {
|
||||||
|
"clang": ROOT / "Saved/Linux/l130/bin/VoxelForgeCoreHeadless-l130-clang",
|
||||||
|
"gcc": ROOT / "Saved/Linux/l130/bin/VoxelForgeCoreHeadless-l130-gcc",
|
||||||
|
}
|
||||||
|
HASH_RE = re.compile(r"^\[Pipeline(?:FullDetailHash|WholeViewHash|LODHash|ExactQueryHash)\].*$", re.M)
|
||||||
|
STAMP_RE = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=(\S+)$", re.M)
|
||||||
|
|
||||||
|
|
||||||
|
def run(dataset: str, recipe: str, workers: int, compiler: str) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
label = f"{dataset}-{recipe}-w{workers}-{compiler}"
|
||||||
|
log = OUT / f"{label}.log"
|
||||||
|
layout = OUT / f"{label}.layout.json"
|
||||||
|
command = ["taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", str(BINARIES[compiler]), str(DATASETS[dataset]),
|
||||||
|
f"--workers={workers}", "--world-assembler-prototype", "--stream-benchmark",
|
||||||
|
"--stream-pipeline", "--force-generate", "--clip-radius=3",
|
||||||
|
"--max-clip-level=8", "--max-level=8", f"--recipe={recipe}",
|
||||||
|
"--stream-hash", f"--resolved-layout-output={layout}"]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
result = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
output = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
hashes = HASH_RE.findall(output)
|
||||||
|
if result.returncode or len(hashes) != 12:
|
||||||
|
raise SystemExit(f"hash run failed: {label}; exit={result.returncode} "
|
||||||
|
f"hashes={len(hashes)} log={log}")
|
||||||
|
stamp = STAMP_RE.search(output)
|
||||||
|
row = {"dataset": dataset, "recipe": recipe, "workers": workers,
|
||||||
|
"compiler": compiler, "exit_code": result.returncode, "hashes": hashes,
|
||||||
|
"source_stamp": stamp.group(1) if stamp else None,
|
||||||
|
"elapsed_s": round(time.monotonic() - started, 3),
|
||||||
|
"log": str(log.relative_to(ROOT))}
|
||||||
|
print("[L130HashRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
if any(not path.is_file() for path in (*DATASETS.values(), *BINARIES.values())):
|
||||||
|
raise SystemExit("missing L130 hash input")
|
||||||
|
old = {(row["dataset"], row["recipe"], row["workers"], row["compiler"]): row
|
||||||
|
for row in BASELINE["runs"]}
|
||||||
|
rows = []
|
||||||
|
for dataset, recipes in RECIPES.items():
|
||||||
|
for recipe in recipes:
|
||||||
|
for workers in (1, 8, 12):
|
||||||
|
for compiler in ("clang", "gcc"):
|
||||||
|
rows.append(run(dataset, recipe, workers, compiler))
|
||||||
|
(OUT / "runs-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
groups = {}
|
||||||
|
for dataset, recipes in RECIPES.items():
|
||||||
|
for recipe in recipes:
|
||||||
|
group = [row for row in rows if row["dataset"] == dataset
|
||||||
|
and row["recipe"] == recipe]
|
||||||
|
key = f"{dataset}/{recipe}"
|
||||||
|
stable = all(row["hashes"] == group[0]["hashes"] for row in group)
|
||||||
|
baseline = old[(dataset, recipe, 12, "clang")]["hashes"]
|
||||||
|
changed = [i for i, (before, after) in enumerate(zip(baseline,
|
||||||
|
group[0]["hashes"])) if before != after]
|
||||||
|
groups[key] = {"all_workers_and_compilers_match": stable,
|
||||||
|
"matches_b96dd73_v225": not changed,
|
||||||
|
"changed_hash_rows_from_v225": changed,
|
||||||
|
"source_stamps": sorted({row["source_stamp"] for row in group})}
|
||||||
|
if not stable:
|
||||||
|
raise SystemExit(f"worker/compiler hash mismatch: {key}")
|
||||||
|
(OUT / "summary.json").write_text(
|
||||||
|
json.dumps({"baseline_source": "b96dd73", "runs": rows,
|
||||||
|
"groups": groups}, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print("[L130HashSummary] " + json.dumps(groups, sort_keys=True), flush=True)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,136 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Render the six L129 b96dd73 mouth views against the L130 meshes."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import html
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
SOURCE = ROOT / "Saved/Linux/l129/captures/mouth-closeups/before"
|
||||||
|
SAVED = ROOT / "Saved/Linux/l130/captures/mouth-closeups"
|
||||||
|
CAPTURES = ROOT / "Standalone/Research/WorldAssemblerM3/Linux130/captures/mouth-closeups"
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
HEADLESS = ROOT / "Saved/Linux/l130/bin/VoxelForgeCoreHeadless-l130-clang"
|
||||||
|
RENDERER = ROOT / "Saved/Linux/l130/bin/VoxelForgeOffscreenCapture-l130-clang"
|
||||||
|
HEADLESS_CONFIG = re.compile(
|
||||||
|
r"^\[HeadlessConfig\].*?camera_m=\(([^)]*)\) camera_rotation_deg=\(([^)]*)\)", re.M)
|
||||||
|
SCENE = re.compile(r"^\[OffscreenScene\].*?height_center_m=(-?[\d.]+)", re.M)
|
||||||
|
DRAWN = re.compile(r"drawn_triangles=(\d+)")
|
||||||
|
|
||||||
|
|
||||||
|
def wrapped(command: list[str]) -> list[str]:
|
||||||
|
return ["taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", *map(str, command)]
|
||||||
|
|
||||||
|
|
||||||
|
def run(label: str, command: list[str], log: Path) -> str:
|
||||||
|
started = time.monotonic()
|
||||||
|
log.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with log.open("w", encoding="utf-8") as output:
|
||||||
|
result = subprocess.run(wrapped(command), cwd=ROOT, stdout=output,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
content = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
if result.returncode:
|
||||||
|
raise SystemExit(f"{label} failed exit={result.returncode}: {log}")
|
||||||
|
print(f"[L130MouthProcess] {label} elapsed_s={time.monotonic()-started:.2f}",
|
||||||
|
flush=True)
|
||||||
|
return content
|
||||||
|
|
||||||
|
|
||||||
|
def parse_vector(text: str) -> tuple[float, ...]:
|
||||||
|
return tuple(float(value) for value in text.split(","))
|
||||||
|
|
||||||
|
|
||||||
|
def sha256(path: Path) -> str:
|
||||||
|
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
if any(not path.is_file() for path in (DATA, HEADLESS, RENDERER)):
|
||||||
|
raise SystemExit("missing L130 mouth capture input")
|
||||||
|
(CAPTURES / "b96dd73").mkdir(parents=True, exist_ok=True)
|
||||||
|
(CAPTURES / "L130").mkdir(parents=True, exist_ok=True)
|
||||||
|
rows = []
|
||||||
|
for source_log in sorted(SOURCE.glob("*-mouth-mesh.log")):
|
||||||
|
name = source_log.name.removesuffix("-mesh.log")
|
||||||
|
source_render = SOURCE / f"{name}-render.log"
|
||||||
|
baseline_mesh = SOURCE / f"{name}.vfmesh"
|
||||||
|
if not all(path.is_file() for path in (source_render, baseline_mesh)):
|
||||||
|
raise SystemExit(f"missing b96dd73 source artifact for {name}")
|
||||||
|
config = HEADLESS_CONFIG.search(source_log.read_text())
|
||||||
|
scene = SCENE.search(source_render.read_text())
|
||||||
|
if config is None or scene is None:
|
||||||
|
raise SystemExit(f"missing camera metadata for {name}")
|
||||||
|
camera = parse_vector(config.group(1))
|
||||||
|
yaw, pitch = parse_vector(config.group(2))
|
||||||
|
height_center = float(scene.group(1))
|
||||||
|
camera_arg = ",".join(f"{value:.6f}" for value in (*camera, yaw, pitch))
|
||||||
|
candidate_mesh = SAVED / "L130" / f"{name}.vfmesh"
|
||||||
|
candidate_mesh.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
run(f"{name}-mesh", [HEADLESS, DATA, "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", "--workers=12",
|
||||||
|
"--clip-radius=3", "--max-clip-level=0", "--max-level=0",
|
||||||
|
"--force-generate", "--recipe=surface-world", f"--camera={camera_arg}",
|
||||||
|
"--render-distance-chunks=1", f"--mesh-export={candidate_mesh}"],
|
||||||
|
SAVED / "L130" / f"{name}-mesh.log")
|
||||||
|
if not candidate_mesh.is_file() or candidate_mesh.stat().st_size == 0:
|
||||||
|
raise SystemExit(f"empty L130 mesh for {name}")
|
||||||
|
|
||||||
|
render_args = ["--size=1920x1080",
|
||||||
|
f"--camera-m={','.join(f'{value:.6f}' for value in camera)}",
|
||||||
|
f"--yaw-deg={yaw:.6f}", f"--pitch-deg={pitch:.6f}",
|
||||||
|
"--shade=readable", "--headlamp=on", "--height-tint=on",
|
||||||
|
f"--height-center-m={height_center:.6f}", "--height-range-m=32"]
|
||||||
|
images = {}
|
||||||
|
draw_counts = {}
|
||||||
|
for variant, mesh in (("b96dd73", baseline_mesh), ("L130", candidate_mesh)):
|
||||||
|
image = CAPTURES / variant / f"{name}.png"
|
||||||
|
render_log = SAVED / variant / f"{name}-render.log"
|
||||||
|
content = run(f"{name}-{variant}-render",
|
||||||
|
[RENDERER, mesh, image, *render_args], render_log)
|
||||||
|
drawn = DRAWN.search(content)
|
||||||
|
if drawn is None or int(drawn.group(1)) <= 0 or not image.is_file():
|
||||||
|
raise SystemExit(f"render has no visible triangles for {name}/{variant}")
|
||||||
|
images[variant] = image
|
||||||
|
draw_counts[variant] = int(drawn.group(1))
|
||||||
|
equal = images["b96dd73"].read_bytes() == images["L130"].read_bytes()
|
||||||
|
row = {"name": name, "camera_m": camera, "yaw_deg": yaw,
|
||||||
|
"pitch_deg": pitch, "height_center_m": height_center,
|
||||||
|
"source_mesh": str(baseline_mesh.relative_to(ROOT)),
|
||||||
|
"candidate_mesh": str(candidate_mesh.relative_to(ROOT)),
|
||||||
|
"baseline_png": str(images["b96dd73"].relative_to(ROOT)),
|
||||||
|
"candidate_png": str(images["L130"].relative_to(ROOT)),
|
||||||
|
"baseline_sha256": sha256(images["b96dd73"]),
|
||||||
|
"candidate_sha256": sha256(images["L130"]),
|
||||||
|
"byte_identical": equal, "drawn_triangles": draw_counts}
|
||||||
|
rows.append(row)
|
||||||
|
print(f"[L130MouthCompare] {name} byte_identical={equal} "
|
||||||
|
f"sha256={row['candidate_sha256']}", flush=True)
|
||||||
|
|
||||||
|
if len(rows) != 6:
|
||||||
|
raise SystemExit(f"expected six L129 mouth views, found {len(rows)}")
|
||||||
|
(CAPTURES / "summary.json").write_text(json.dumps(rows, indent=2) + "\n")
|
||||||
|
sections = []
|
||||||
|
for row in rows:
|
||||||
|
name = html.escape(row["name"])
|
||||||
|
sections.append(f"<section><h2>{name}</h2><p>byte identical: "
|
||||||
|
f"{str(row['byte_identical']).lower()}</p><div>"
|
||||||
|
f"<figure><figcaption>b96dd73 source mesh</figcaption><img src='b96dd73/{name}.png'></figure>"
|
||||||
|
f"<figure><figcaption>L130 reach units</figcaption><img src='L130/{name}.png'></figure>"
|
||||||
|
"</div></section>")
|
||||||
|
page = ("<!doctype html><meta charset='utf-8'><title>L130 mouth closeups</title>"
|
||||||
|
"<style>body{font:14px system-ui;margin:24px;background:#17191d;color:#eee}"
|
||||||
|
"section{margin:28px 0}div{display:grid;grid-template-columns:1fr 1fr;gap:12px}"
|
||||||
|
"img{width:100%;height:auto}p{color:#bbb}</style>"
|
||||||
|
"<h1>L129 mouth closeups: b96dd73 vs L130</h1>" + "\n".join(sections))
|
||||||
|
(CAPTURES / "COMPARE.html").write_text(page)
|
||||||
|
return 0 if all(row["byte_identical"] for row in rows) else 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,98 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Render the 23 L125 camera poses with the L130 evaluator and compare them."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import html
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
import shutil
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
RESEARCH = Path(__file__).resolve().parent
|
||||||
|
CAPTURES = RESEARCH / "captures"
|
||||||
|
SAVED = ROOT / "Saved/Linux/l130/captures"
|
||||||
|
BASELINE_INDEX = (ROOT / "Standalone/Research/WorldAssemblerM3/Linux119"
|
||||||
|
/ "captures/L119/capture-index.json")
|
||||||
|
REFERENCE = ROOT / "Standalone/Research/WorldAssemblerM3/Linux125/captures/L125"
|
||||||
|
LAYOUT = ROOT / "Saved/Linux/l130/hashes/owner4-surface-world-w12-clang.layout.json"
|
||||||
|
HEADLESS = ROOT / "Saved/Linux/l130/bin/VoxelForgeCoreHeadless-l130-clang"
|
||||||
|
RENDERER = ROOT / "Saved/Linux/l130/bin/VoxelForgeOffscreenCapture-l130-clang"
|
||||||
|
HELPER_PATH = ROOT / "Standalone/Research/WorldAssemblerM3/Linux113/RunL113Captures.py"
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
if any(not path.is_file() for path in
|
||||||
|
(BASELINE_INDEX, REFERENCE / "passage-0-1-0-rock-floor-station-1.png",
|
||||||
|
LAYOUT, HEADLESS, RENDERER)):
|
||||||
|
raise SystemExit("missing pose, layout, or L130 capture input")
|
||||||
|
spec = importlib.util.spec_from_file_location("l125_capture_runner",
|
||||||
|
ROOT / "Standalone/Research/WorldAssemblerM3/Linux125/RunL125Captures.py")
|
||||||
|
if spec is None or spec.loader is None:
|
||||||
|
raise SystemExit("cannot load the verified L125 pose set")
|
||||||
|
runner = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(runner)
|
||||||
|
runner.RESEARCH = RESEARCH
|
||||||
|
runner.CAPTURES = CAPTURES
|
||||||
|
runner.SAVED = SAVED
|
||||||
|
runner.LAYOUT = LAYOUT
|
||||||
|
runner.CLANG = HEADLESS
|
||||||
|
runner.RENDERER = RENDERER
|
||||||
|
runner.HELPERS.RESEARCH = RESEARCH
|
||||||
|
runner.HELPERS.CAPTURES = CAPTURES
|
||||||
|
runner.HELPERS.SAVED = SAVED
|
||||||
|
runner.HELPERS.BINARIES = {"L125": HEADLESS}
|
||||||
|
runner.HELPERS.RENDERERS = {"L125": RENDERER}
|
||||||
|
if runner.main() != 0:
|
||||||
|
raise SystemExit("L130 pose generation failed")
|
||||||
|
|
||||||
|
new_captures = CAPTURES / "L125"
|
||||||
|
new_saved = SAVED / "L125"
|
||||||
|
index_path = new_captures / "capture-index.json"
|
||||||
|
data = json.loads(index_path.read_text())
|
||||||
|
if len(data["captures"]) != 23:
|
||||||
|
raise SystemExit(f"expected 23 poses, found {len(data['captures'])}")
|
||||||
|
identical = 0
|
||||||
|
sections = []
|
||||||
|
for row in data["captures"]:
|
||||||
|
name = row["name"]
|
||||||
|
old_image = REFERENCE / f"{name}.png"
|
||||||
|
new_image = new_captures / f"{name}.png"
|
||||||
|
if not old_image.is_file() or not new_image.is_file():
|
||||||
|
raise SystemExit(f"missing comparison image for {name}")
|
||||||
|
same = old_image.read_bytes() == new_image.read_bytes()
|
||||||
|
identical += int(same)
|
||||||
|
row["variant"] = "L130-v227"
|
||||||
|
row["capture"] = row["capture"].replace("captures/L125/", "captures/L130/")
|
||||||
|
row["scene"] = row["scene"].replace("captures/L125/", "captures/L130/")
|
||||||
|
row["logs"] = [path.replace("captures/L125/", "captures/L130/")
|
||||||
|
for path in row.get("logs", [])]
|
||||||
|
sections.append(f"<section><h2>{html.escape(row['passage'])} — "
|
||||||
|
f"{html.escape(row.get('station', ''))}</h2><p>{html.escape(row['kind'])} · "
|
||||||
|
f"byte identical: {str(same).lower()}</p><div>"
|
||||||
|
f"<figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/{html.escape(name)}.png'></figure>"
|
||||||
|
f"<figure><figcaption>L130 v227</figcaption><img src='L130/{html.escape(name)}.png'></figure>"
|
||||||
|
"</div></section>")
|
||||||
|
if identical != 23:
|
||||||
|
raise SystemExit(f"only {identical}/23 L125 pose captures are byte-identical")
|
||||||
|
|
||||||
|
index_path.write_text(json.dumps(data, indent=2) + "\n")
|
||||||
|
old_captures, old_saved = CAPTURES / "L130", SAVED / "L130"
|
||||||
|
if old_captures.exists():
|
||||||
|
shutil.rmtree(old_captures)
|
||||||
|
if old_saved.exists():
|
||||||
|
shutil.rmtree(old_saved)
|
||||||
|
new_captures.rename(old_captures)
|
||||||
|
new_saved.rename(old_saved)
|
||||||
|
page = ("<!doctype html><meta charset='utf-8'><title>L130 passage captures</title>"
|
||||||
|
"<style>body{font:14px system-ui;margin:24px;background:#17191d;color:#eee}"
|
||||||
|
"section{margin:28px 0}div{display:grid;grid-template-columns:1fr 1fr;gap:12px}"
|
||||||
|
"img{width:100%;height:auto}p{color:#bbb}</style>"
|
||||||
|
"<h1>L125 v225 vs L130 v227: passage captures</h1>" + "\n".join(sections))
|
||||||
|
(CAPTURES / "COMPARE.html").write_text(page)
|
||||||
|
print(f"[L130PoseCapture] poses=23 byte_identical={identical}/23", flush=True)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,137 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Five interleaved L80 near-ready runs per stratum/worker vs b96dd73."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import statistics
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
OUT = ROOT / "Saved/Linux/l130/timing"
|
||||||
|
PASS_ID = int(os.environ.get("VOXELFORGE_L130_TIMING_PASS", "1"))
|
||||||
|
REPLICATE_FIRST = (PASS_ID - 1) * 5 + 1
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
BINARIES = {
|
||||||
|
"b96dd73": ROOT / "Saved/Linux/l125/bin/VoxelForgeCoreHeadless-l125-clang",
|
||||||
|
"L130": ROOT / "Saved/Linux/l130/bin/VoxelForgeCoreHeadless-l130-clang",
|
||||||
|
}
|
||||||
|
CAMERAS = {
|
||||||
|
0: (8.0, 16.0, -372.2626953125),
|
||||||
|
1: (25.8369140625, 3.19921875, -737.0),
|
||||||
|
2: (19.166015625, -14.2001953125, -753.3740234375),
|
||||||
|
3: (-20.0625, -5.068359375, -900.353515625),
|
||||||
|
}
|
||||||
|
PIPELINE_RE = re.compile(r"^\[StreamPipeline\] (.*)$", re.M)
|
||||||
|
FIELDS_RE = re.compile(r"([a-zA-Z0-9_]+)=([^ ]+)")
|
||||||
|
NEAR_RE = re.compile(
|
||||||
|
r"^\[WorldAssemblerNearLODReady\] lods=0-2 tiles=(\d+)/(\d+) cold_ready_s=([0-9.]+)$",
|
||||||
|
re.M)
|
||||||
|
|
||||||
|
|
||||||
|
def run(strate: int, workers: int, version: str, phase: str, replicate: int) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
pass_suffix = "" if PASS_ID == 1 else f"-p{PASS_ID}"
|
||||||
|
label = f"s{strate}-w{workers}-{version}-{phase}{replicate}-l130-v227{pass_suffix}"
|
||||||
|
log = OUT / f"{label}.log"
|
||||||
|
camera = ",".join(f"{value:.12f}" for value in CAMERAS[strate]) + ",0,0"
|
||||||
|
command = ["taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", str(BINARIES[version]), str(DATA),
|
||||||
|
"--world-assembler-prototype", "--stream-benchmark", "--stream-pipeline",
|
||||||
|
f"--workers={workers}", "--clip-radius=3", "--max-clip-level=8",
|
||||||
|
"--max-level=8", f"--camera={camera}", "--profile-lod-work",
|
||||||
|
"--stream-scheduler-policy=3"]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as output:
|
||||||
|
result = subprocess.run(command, cwd=ROOT, stdout=output,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
content = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
pipeline, near = PIPELINE_RE.search(content), NEAR_RE.search(content)
|
||||||
|
if result.returncode or pipeline is None or near is None:
|
||||||
|
raise SystemExit(f"timing run failed: {log}; exit={result.returncode}")
|
||||||
|
fields = dict(FIELDS_RE.findall(pipeline.group(1)))
|
||||||
|
row = {"strate": strate, "workers": workers, "version": version,
|
||||||
|
"phase": phase, "replicate": replicate, "exit_code": result.returncode,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"lod02_ready_s": float(near.group(3)),
|
||||||
|
"whole_view_ready_s": float(fields["whole_view_ready"].removesuffix("s")),
|
||||||
|
"log": str(log.relative_to(ROOT))}
|
||||||
|
print("[L130TimingRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
if any(not path.is_file() for path in (DATA, *BINARIES.values())):
|
||||||
|
raise SystemExit("missing L130 timing input")
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
load_1m = __import__("os").getloadavg()[0]
|
||||||
|
def cpu_idle(interval: float = 2.0) -> float:
|
||||||
|
def totals() -> tuple[int, int]:
|
||||||
|
fields = Path("/proc/stat").read_text().splitlines()[0].split()
|
||||||
|
values = [int(value) for value in fields[1:]]
|
||||||
|
return sum(values), values[3] + (values[4] if len(values) > 4 else 0)
|
||||||
|
before = totals()
|
||||||
|
time.sleep(interval)
|
||||||
|
after = totals()
|
||||||
|
return 100.0 * (after[1] - before[1]) / max(1, after[0] - before[0])
|
||||||
|
idle = cpu_idle()
|
||||||
|
pass_suffix = "" if PASS_ID == 1 else f"-p{PASS_ID}"
|
||||||
|
(OUT / f"machine-load-v227{pass_suffix}-before.txt").write_text(
|
||||||
|
f"1-minute load: {load_1m:.2f}\n2-second CPU idle: {idle:.1f}%\n")
|
||||||
|
print(f"[L130TimingGate] load_1m={load_1m:.2f} cpu_idle_2s={idle:.1f}%",
|
||||||
|
flush=True)
|
||||||
|
if load_1m >= 0.9 or idle < 97.0:
|
||||||
|
raise SystemExit("timing refused: machine is above the quiet-load gate")
|
||||||
|
|
||||||
|
rows = []
|
||||||
|
for strate in range(4):
|
||||||
|
for workers in (8, 12):
|
||||||
|
for version in BINARIES:
|
||||||
|
run(strate, workers, version, "warmup", 0)
|
||||||
|
for replicate in range(REPLICATE_FIRST, REPLICATE_FIRST + 5):
|
||||||
|
order = tuple(BINARIES) if replicate % 2 else tuple(reversed(BINARIES))
|
||||||
|
for version in order:
|
||||||
|
rows.append(run(strate, workers, version, "measured", replicate))
|
||||||
|
(OUT / f"runs-v227{pass_suffix}-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
prior_runs = []
|
||||||
|
if PASS_ID > 1:
|
||||||
|
previous = OUT / ("v227-summary.json" if PASS_ID == 2
|
||||||
|
else "v227-combined-summary.json")
|
||||||
|
if previous.is_file():
|
||||||
|
prior_runs = json.loads(previous.read_text(encoding="utf-8")).get("runs", [])
|
||||||
|
all_rows = prior_runs + rows
|
||||||
|
measured_rows = [row for row in all_rows if row["phase"] == "measured"]
|
||||||
|
summary = []
|
||||||
|
for strate in range(4):
|
||||||
|
for workers in (8, 12):
|
||||||
|
group_size = sum(row["strate"] == strate and row["workers"] == workers
|
||||||
|
for row in measured_rows) // 2
|
||||||
|
item = {"strate": strate, "workers": workers,
|
||||||
|
"replicates_per_version": group_size}
|
||||||
|
medians = {}
|
||||||
|
for version in BINARIES:
|
||||||
|
group = [row for row in measured_rows if row["strate"] == strate
|
||||||
|
and row["workers"] == workers and row["version"] == version]
|
||||||
|
medians[version] = {}
|
||||||
|
for key in ("lod02_ready_s", "whole_view_ready_s"):
|
||||||
|
values = [row[key] for row in group]
|
||||||
|
medians[version][key] = statistics.median(values)
|
||||||
|
item[f"{version}_{key}_values"] = values
|
||||||
|
item[f"{version}_{key}_median"] = medians[version][key]
|
||||||
|
for key in ("lod02_ready_s", "whole_view_ready_s"):
|
||||||
|
item[f"L130_vs_b96dd73_{key}_delta_pct"] = round(
|
||||||
|
100.0*(medians["L130"][key]/medians["b96dd73"][key]-1.0), 2)
|
||||||
|
summary.append(item)
|
||||||
|
summary_path = OUT / ("v227-summary.json" if PASS_ID == 1
|
||||||
|
else f"v227-combined-summary.json")
|
||||||
|
summary_path.write_text(json.dumps(
|
||||||
|
{"runs": all_rows, "summary": summary}, indent=2) + "\n", encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
<!doctype html><meta charset='utf-8'><title>L130 passage captures</title><style>body{font:14px system-ui;margin:24px;background:#17191d;color:#eee}section{margin:28px 0}div{display:grid;grid-template-columns:1fr 1fr;gap:12px}img{width:100%;height:auto}p{color:#bbb}</style><h1>L125 v225 vs L130 v227: passage captures</h1><section><h2>passage/0-1/0 — rock-floor/401.7m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-rock-floor-station-1.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-rock-floor-station-1.png'></figure></div></section>
|
||||||
|
<section><h2>passage/0-1/0 — rock-floor/792.8m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-rock-floor-station-2.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-rock-floor-station-2.png'></figure></div></section>
|
||||||
|
<section><h2>passage/0-1/0 — rock-floor/1634.5m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-rock-floor-station-3.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-rock-floor-station-3.png'></figure></div></section>
|
||||||
|
<section><h2>passage/1-2/0 — rock-floor/27.0m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-1-2-0-rock-floor-station-1.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-1-2-0-rock-floor-station-1.png'></figure></div></section>
|
||||||
|
<section><h2>passage/1-2/0 — rock-floor/52.9m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-1-2-0-rock-floor-station-2.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-1-2-0-rock-floor-station-2.png'></figure></div></section>
|
||||||
|
<section><h2>passage/1-2/0 — rock-floor/78.9m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-1-2-0-rock-floor-station-3.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-1-2-0-rock-floor-station-3.png'></figure></div></section>
|
||||||
|
<section><h2>passage/2-3/0 — rock-floor/323.4m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-2-3-0-rock-floor-station-1.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-2-3-0-rock-floor-station-1.png'></figure></div></section>
|
||||||
|
<section><h2>passage/2-3/0 — rock-floor/333.4m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-2-3-0-rock-floor-station-2.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-2-3-0-rock-floor-station-2.png'></figure></div></section>
|
||||||
|
<section><h2>passage/2-3/0 — rock-floor/343.4m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-2-3-0-rock-floor-station-3.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-2-3-0-rock-floor-station-3.png'></figure></div></section>
|
||||||
|
<section><h2>passage/0-1/0 — section-1/4m-outside</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-openair-1-4m-outside.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-openair-1-4m-outside.png'></figure></div></section>
|
||||||
|
<section><h2>passage/0-1/0 — section-1/4m-interior-wall</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-openair-1-4m-interior-wall.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-openair-1-4m-interior-wall.png'></figure></div></section>
|
||||||
|
<section><h2>passage/0-1/0 — section-2/4m-outside</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-openair-2-4m-outside.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-openair-2-4m-outside.png'></figure></div></section>
|
||||||
|
<section><h2>passage/0-1/0 — section-2/4m-interior-wall</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-openair-2-4m-interior-wall.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-openair-2-4m-interior-wall.png'></figure></div></section>
|
||||||
|
<section><h2>passage/0-1/0 — section-3/4m-outside</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-openair-3-4m-outside.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-openair-3-4m-outside.png'></figure></div></section>
|
||||||
|
<section><h2>passage/0-1/0 — section-3/4m-interior-wall</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-openair-3-4m-interior-wall.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-openair-3-4m-interior-wall.png'></figure></div></section>
|
||||||
|
<section><h2>passage/1-2/0 — section-1/4m-outside</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-1-2-0-openair-1-4m-outside.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-1-2-0-openair-1-4m-outside.png'></figure></div></section>
|
||||||
|
<section><h2>passage/1-2/0 — section-1/4m-interior-wall</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-1-2-0-openair-1-4m-interior-wall.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-1-2-0-openair-1-4m-interior-wall.png'></figure></div></section>
|
||||||
|
<section><h2>passage/1-2/0 — section-2/4m-outside</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-1-2-0-openair-2-4m-outside.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-1-2-0-openair-2-4m-outside.png'></figure></div></section>
|
||||||
|
<section><h2>passage/1-2/0 — section-2/4m-interior-wall</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-1-2-0-openair-2-4m-interior-wall.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-1-2-0-openair-2-4m-interior-wall.png'></figure></div></section>
|
||||||
|
<section><h2>passage/2-3/0 — section-1/4m-outside</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-2-3-0-openair-1-4m-outside.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-2-3-0-openair-1-4m-outside.png'></figure></div></section>
|
||||||
|
<section><h2>passage/2-3/0 — section-1/4m-interior-wall</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-2-3-0-openair-1-4m-interior-wall.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-2-3-0-openair-1-4m-interior-wall.png'></figure></div></section>
|
||||||
|
<section><h2>passage/2-3/0 — section-2/4m-outside</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-2-3-0-openair-2-4m-outside.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-2-3-0-openair-2-4m-outside.png'></figure></div></section>
|
||||||
|
<section><h2>passage/2-3/0 — section-2/4m-interior-wall</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-2-3-0-openair-2-4m-interior-wall.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-2-3-0-openair-2-4m-interior-wall.png'></figure></div></section>
|
||||||
|
After Width: | Height: | Size: 11 KiB |
|
After Width: | Height: | Size: 98 KiB |
|
After Width: | Height: | Size: 63 KiB |
|
After Width: | Height: | Size: 111 KiB |
|
After Width: | Height: | Size: 225 KiB |
|
After Width: | Height: | Size: 177 KiB |
|
After Width: | Height: | Size: 125 KiB |
|
After Width: | Height: | Size: 112 KiB |
|
After Width: | Height: | Size: 116 KiB |
|
After Width: | Height: | Size: 50 KiB |
|
After Width: | Height: | Size: 184 KiB |
|
After Width: | Height: | Size: 34 KiB |
|
After Width: | Height: | Size: 121 KiB |
|
After Width: | Height: | Size: 111 KiB |
|
After Width: | Height: | Size: 163 KiB |