Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f8cba7c36a | ||
|
|
0065ee68e6 | ||
|
|
ebae71b142 | ||
|
|
81071969fc | ||
|
|
cbbfd46bd3 | ||
|
|
044cd6d9c5 | ||
|
|
b96dd739b8 | ||
|
|
4041d4bb4b | ||
|
|
aeb60bbda3 | ||
|
|
c743f2abc7 | ||
|
|
5e7e0eb1c1 | ||
|
|
bae1144ee4 | ||
|
|
5c519f5206 | ||
|
|
161565936c |
No files matched your search
@@ -1522,11 +1522,12 @@ collision ring became walkable at 0.632 s. `voxel.StreamingCoarseFirst=1` with
|
|||||||
residency by about one third on the staged plan, but null-RHI residency does not prove rendered
|
residency by about one third on the staged plan, but null-RHI residency does not prove rendered
|
||||||
enclosure. Keep coarse-first opt-in until that view is checked in a rendered owner-plan run.
|
enclosure. Keep coarse-first opt-in until that view is checked in a rendered owner-plan run.
|
||||||
|
|
||||||
The separate `voxel.StreamingStartupApplyBoost` switch stays default-off. At `1`, it temporarily allows
|
The separate `voxel.StreamingStartupApplyBoost` switch is default-on in the current code. At `1`, it
|
||||||
12 visible mesh submissions / 14 ms while the initial desired set and walkable ring are incomplete,
|
temporarily allows up to 24 visible mesh submissions / 14 ms while the initial desired set and
|
||||||
with a 30 second ceiling; at `0`, the authored budget is unchanged. On the matched legacy-order test,
|
walkable ring are incomplete, with a 30 second ceiling; at `0`, the authored budget is unchanged.
|
||||||
the boost changed full-set readiness from 17.412 s to 17.346 s and removed count-cap frames, but did
|
The historical matched S6 run changed full-set readiness from 17.412 s to 17.346 s and removed
|
||||||
not improve end-to-end residency. Keep the normal 4 / 2 ms guard for play.
|
count-cap frames, but did not improve end-to-end residency. These current defaults are defined in
|
||||||
|
`Source/VoxelForge/Private/VoxelWorld.cpp`; the S6 measurement remains historical context.
|
||||||
|
|
||||||
The S6 trace found the shared pool was not saturated: its 10-slot idle estimate was about 132
|
The S6 trace found the shared pool was not saturated: its 10-slot idle estimate was about 132
|
||||||
slot-seconds on legacy order and 73 on gated coarse-first, while game-thread tile applies totaled only
|
slot-seconds on legacy order and 73 on gated coarse-first, while game-thread tile applies totaled only
|
||||||
@@ -1556,8 +1557,8 @@ by its RAII guard when the worker lambda exits. This still counts queued and run
|
|||||||
results do not consume an admission slot. `voxel.StreamingLegacyPendingResultCap=1` restores the old
|
results do not consume an admission slot. `voxel.StreamingLegacyPendingResultCap=1` restores the old
|
||||||
`PendingTiles` accounting; the test harness's `NearReserveMode=Legacy` also sets
|
`PendingTiles` accounting; the test harness's `NearReserveMode=Legacy` also sets
|
||||||
`voxel.StreamingDynamicNearReserve=0`, reproducing the S6 path. `StreamingDynamicNearReserve` now
|
`voxel.StreamingDynamicNearReserve=0`, reproducing the S6 path. `StreamingDynamicNearReserve` now
|
||||||
defaults to `1`; `StreamingCoarseFirst` remains `0`, and the apply boost remains `0` in the matched
|
defaults to `1`; `StreamingCoarseFirst` remains `0`. That historical S6 matched measurement disabled
|
||||||
measurement.
|
the startup apply boost (`voxel.StreamingStartupApplyBoost=0`); the current default is on (§8.27).
|
||||||
|
|
||||||
On the 2,578-key owner SurfaceWorld trace, the old-path control reached the full set in 17.527 s,
|
On the 2,578-key owner SurfaceWorld trace, the old-path control reached the full set in 17.527 s,
|
||||||
walkable ring in 0.650 s, and full LOD0 in 2.281 s. Dynamic admission reached them in 7.780 s,
|
walkable ring in 0.650 s, and full LOD0 in 2.281 s. Dynamic admission reached them in 7.780 s,
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ then wait for his results / pasted errors. Same for in-editor checks — ask for
|
|||||||
try to run anything.
|
try to run anything.
|
||||||
|
|
||||||
## Read first (in order, load only what the task needs)
|
## Read first (in order, load only what the task needs)
|
||||||
|
- **[CURRENT-STATE.md](CURRENT-STATE.md) — current shipping snapshot and verification entry point.**
|
||||||
- **[CODEMAP.md](CODEMAP.md) — ALWAYS.** Navigation: what it is, data-flow (§2), symbol→`file:line`
|
- **[CODEMAP.md](CODEMAP.md) — ALWAYS.** Navigation: what it is, data-flow (§2), symbol→`file:line`
|
||||||
index (§3), "I want to change X → go here" (§5), conventions & gotchas (§6). Trust **symbol names
|
index (§3), "I want to change X → go here" (§5), conventions & gotchas (§6). Trust **symbol names
|
||||||
over line numbers** (lines drift).
|
over line numbers** (lines drift).
|
||||||
|
|||||||
@@ -388,7 +388,7 @@ casse cette arithmetique *structurellement* — `BeginPlay` loggue une **Error**
|
|||||||
| `AdvanceHeadlessDigMotionTest` / `RecordHeadlessDigMotionFrame` | 3060 / 3217 | Explicit `-voxel.TestDigMotion=1` repro: optional near digs, then repeats underfoot digs and samples `Generator->GetDensityAt` immediately after each edit, on its first subsequent simulated frame, and on every following observation frame. `-voxel.TestDigMotionFieldOverride=1` is a measurement-only prototype that clears `CurrentFloor` and switches CMC to falling when the sampled post-edit density is air; it is inactive by default. The test counts false support, false air and downward velocity beyond one gravity step. `-voxel.TestDigMotionUnderfootDigCount` is clamped to 1–100; `-voxel.TestDigMotionUnderfootIntervalSeconds=0` performs per-frame rapid edits. The accompanying gap audit is scoped to the dig session. |
|
| `AdvanceHeadlessDigMotionTest` / `RecordHeadlessDigMotionFrame` | 3060 / 3217 | Explicit `-voxel.TestDigMotion=1` repro: optional near digs, then repeats underfoot digs and samples `Generator->GetDensityAt` immediately after each edit, on its first subsequent simulated frame, and on every following observation frame. `-voxel.TestDigMotionFieldOverride=1` is a measurement-only prototype that clears `CurrentFloor` and switches CMC to falling when the sampled post-edit density is air; it is inactive by default. The test counts false support, false air and downward velocity beyond one gravity step. `-voxel.TestDigMotionUnderfootDigCount` is clamped to 1–100; `-voxel.TestDigMotionUnderfootIntervalSeconds=0` performs per-frame rapid edits. The accompanying gap audit is scoped to the dig session. |
|
||||||
| `MaybeFinishHeadlessStreamingTest` | — | The existing `voxel.TestExitSeconds` diagnostic emits `VoxelForgeStreamingExitDiag` at most every 30 elapsed seconds, including threshold and early-return test flags; no output when the timer is unconfigured. |
|
| `MaybeFinishHeadlessStreamingTest` | — | The existing `voxel.TestExitSeconds` diagnostic emits `VoxelForgeStreamingExitDiag` at most every 30 elapsed seconds, including threshold and early-return test flags; no output when the timer is unconfigured. |
|
||||||
| `UpdateVisualStalenessAudit` | 5034 | Test-only camera-frustum coverage audit: tracks up to 24 real pre-skirt isosurface points per visible component, then counts frames where no visible tile component covers a previously visible, still-desired point (excluding edited bounds). `voxel.TestVisualGapInjectEarlyHide=1` hides a valid component for three frames and must be caught. `voxel.TestGapAuditLegacyHandoff=1` restores old handoff culling for A/B baselines. |
|
| `UpdateVisualStalenessAudit` | 5034 | Test-only camera-frustum coverage audit: tracks up to 24 real pre-skirt isosurface points per visible component, then counts frames where no visible tile component covers a previously visible, still-desired point (excluding edited bounds). `voxel.TestVisualGapInjectEarlyHide=1` hides a valid component for three frames and must be caught. `voxel.TestGapAuditLegacyHandoff=1` restores old handoff culling for A/B baselines. |
|
||||||
| `ProcessPendingChunks` | 7435 | Drains `CriticalProcessQueue`, `EditedProcessQueue`, `NearProcessQueue`, then `ProcessQueue`; applies under the authored 4-mesh/2 ms budget during normal play and the bounded shared startup budget (12 meshes/14 ms) until the initial desired set and walkable ring are ready. `voxel.StreamingStartupApplyBoost=0` restores the authored budget. `startup_apply_config` traces the actor-read CVar and both budgets; `startup_apply_budget` records activation/end. Reads per-request cancellation before removing its map entry, traces stale/obsolete/aborted discards with epoch and ordinal, then applies via `ApplyTileResult`. Step 1 prequeue still releases here and emits `VFStep1UploadCorpusReleased`. |
|
| `ProcessPendingChunks` | 7435 | Drains `CriticalProcessQueue`, `EditedProcessQueue`, `NearProcessQueue`, then `ProcessQueue`; applies under the authored 4-mesh/2 ms budget during normal play and the default-on bounded startup budget (24 meshes/14 ms, at most 30 s) until the initial desired set and walkable ring are ready. `voxel.StreamingStartupApplyBoost=0` restores the authored budget. `startup_apply_config` traces the actor-read CVar and both budgets; `startup_apply_budget` records activation/end. Reads per-request cancellation before removing its map entry, traces stale/obsolete/aborted discards with epoch and ordinal, then applies via `ApplyTileResult`. Step 1 prequeue still releases here and emits `VFStep1UploadCorpusReleased`. |
|
||||||
| `ApplyTileResult` | — | **Shared game-thread apply** for one `FChunkResult` (async drain + sync carve): discards stale generation epochs and keys no longer desired, marks loaded, ingests capture, EMPTY releases the tile's existing component (a re-gen can flip content→empty on band change — old geometry must not linger), else `ApplyMeshToTile`. Returns true iff a visible mesh uploaded (counts the budget). Doesn't touch `PendingTiles` (caller's). |
|
| `ApplyTileResult` | — | **Shared game-thread apply** for one `FChunkResult` (async drain + sync carve): discards stale generation epochs and keys no longer desired, marks loaded, ingests capture, EMPTY releases the tile's existing component (a re-gen can flip content→empty on band change — old geometry must not linger), else `ApplyMeshToTile`. Returns true iff a visible mesh uploaded (counts the budget). Doesn't touch `PendingTiles` (caller's). |
|
||||||
| `FVoxelStackSampler` / `FScopedVoxelStackRegistration` (`VoxelStackSampler.h/.cpp`) | — | **Diagnostic raw stack sampler, off by default.** A sampler thread captures registered generation workers' stacks (`CaptureThreadStackBackTrace`) every N µs and tags each with the tile's LOD. It writes bounded raw PCs to CSV plus a symbolization-free summary under `Saved/`; symbol resolution is deferred until after process exit so the measured process never initializes/calls DbgHelp for this diagnostic. Lifecycle teardown is serialized and registration TLS is session-tagged. Registration is per tile TASK in `LoadTile` (and the export's tasks), never per sample; disabled costs one relaxed atomic load. Game `-voxel.SampleStacks=<us>`, commandlet `-samplestacks[=us]`. |
|
| `FVoxelStackSampler` / `FScopedVoxelStackRegistration` (`VoxelStackSampler.h/.cpp`) | — | **Diagnostic raw stack sampler, off by default.** A sampler thread captures registered generation workers' stacks (`CaptureThreadStackBackTrace`) every N µs and tags each with the tile's LOD. It writes bounded raw PCs to CSV plus a symbolization-free summary under `Saved/`; symbol resolution is deferred until after process exit so the measured process never initializes/calls DbgHelp for this diagnostic. Lifecycle teardown is serialized and registration TLS is session-tagged. Registration is per tile TASK in `LoadTile` (and the export's tasks), never per sample; disabled costs one relaxed atomic load. Game `-voxel.SampleStacks=<us>`, commandlet `-samplestacks[=us]`. |
|
||||||
| Room-cache performance (2026-09-13): `VF_FindPlayerFitPointForRoomMemoized` / `GPlayerFitMemo` (`VoxelCaveMorphology.cpp`), `voxel.TileCacheWindow` + `VoxelGenLOD::TileOriginVoxels/TileCellsPerAxis` (`VoxelGenerator.h`), `FChunkSDFSpatialIndex` | — | **Player-fit memo:** exact 97-word key, process-wide, `voxel.PlayerFitMemo`. **Tile cache window:** the room cache covers the requesting tile's footprint plus a fixed collect margin. It is set via `TGuardValue` in `GenerateMesh`/classify, and is window-invariant by construction (the margin does not grow with the tile). **LOD-gated:** fused path at LOD ≥ `voxel.TileCacheWindowMinLOD` (3), op-stack at LOD ≥ `voxel.TileCacheWindowOpStackMinLOD` (4); lower LODs keep the 4-chunk worker window, because the tile window slowed LOD0 evaluation ~6%. Strates whose authored terrain ops exceed the proven 256-voxel extent (or are malformed/unloaded) fall back to the legacy window (`VF_ComputeTerrainOpEnvelope`). `voxel.SpatialIndex` is -1 auto (off at fused LOD0, on elsewhere), 0 or 1. **Spatial index:** ordered XY buckets over cache primitives, exact rejection. The `static_assert` in `VoxelStrateTypes.h` ties `VF_STRATE_PARAM_FIELDS` to `FStrateGenerationParams` (83 fields + 8 B padding). |
|
| Room-cache performance (2026-09-13): `VF_FindPlayerFitPointForRoomMemoized` / `GPlayerFitMemo` (`VoxelCaveMorphology.cpp`), `voxel.TileCacheWindow` + `VoxelGenLOD::TileOriginVoxels/TileCellsPerAxis` (`VoxelGenerator.h`), `FChunkSDFSpatialIndex` | — | **Player-fit memo:** exact 97-word key, process-wide, `voxel.PlayerFitMemo`. **Tile cache window:** the room cache covers the requesting tile's footprint plus a fixed collect margin. It is set via `TGuardValue` in `GenerateMesh`/classify, and is window-invariant by construction (the margin does not grow with the tile). **LOD-gated:** fused path at LOD ≥ `voxel.TileCacheWindowMinLOD` (3), op-stack at LOD ≥ `voxel.TileCacheWindowOpStackMinLOD` (4); lower LODs keep the 4-chunk worker window, because the tile window slowed LOD0 evaluation ~6%. Strates whose authored terrain ops exceed the proven 256-voxel extent (or are malformed/unloaded) fall back to the legacy window (`VF_ComputeTerrainOpEnvelope`). `voxel.SpatialIndex` is -1 auto (off at fused LOD0, on elsewhere), 0 or 1. **Spatial index:** ordered XY buckets over cache primitives, exact rejection. The `static_assert` in `VoxelStrateTypes.h` ties `VF_STRATE_PARAM_FIELDS` to `FStrateGenerationParams` (83 fields + 8 B padding). |
|
||||||
@@ -396,7 +396,7 @@ casse cette arithmetique *structurellement* — `BeginPlay` loggue une **Error**
|
|||||||
| `RecordTileHash` / `WriteTileHashDump` | — | Headless `-voxel.TileHashDump=<absolute path>` parity instrument. Records the exact `FVoxelMeshData` used by the streaming result, including empty/classified tiles, then writes sorted absolute tile keys, LOD step/cells, content-band limits, settings digest context, and the shared `VoxelGeometryHash` at `EndPlay`. |
|
| `RecordTileHash` / `WriteTileHashDump` | — | Headless `-voxel.TileHashDump=<absolute path>` parity instrument. Records the exact `FVoxelMeshData` used by the streaming result, including empty/classified tiles, then writes sorted absolute tile keys, LOD step/cells, content-band limits, settings digest context, and the shared `VoxelGeometryHash` at `EndPlay`. |
|
||||||
| `SyncRemeshTile` | — | **INSTANT DIG**: level-0 same-frame re-mesh on the game thread. Uses `PreEditDensityCache` when the tile is resident in the bounded local cache, replays the current DiffLayer over that exact procedural lattice, then meshes/applies inline so a carve is visible THIS frame. A cache miss uses normal generation; it captures the unedited lattice if the tile needs meshing, while uniform-tile proofs remain enabled. |
|
| `SyncRemeshTile` | — | **INSTANT DIG**: level-0 same-frame re-mesh on the game thread. Uses `PreEditDensityCache` when the tile is resident in the bounded local cache, replays the current DiffLayer over that exact procedural lattice, then meshes/applies inline so a carve is visible THIS frame. A cache miss uses normal generation; it captures the unedited lattice if the tile needs meshing, while uniform-tile proofs remain enabled. |
|
||||||
| `PreEditDensityCache` / `VoxelForgePreEditDensityCache::BuildEditedGrid` | — | Per-world cache of the pawn's current L0 tile: exact 35³ pre-edit samples, max **171,500 bytes**. It is pruned when the pawn crosses chunk boundaries and cleared on `RegenerateAllChunks`. `BuildEditedGrid` snapshots touched diff chunks once and reproduces `GetDensityAt`'s final `pre-edit - EvaluateMods` operation before the per-call shared-grid mesher path. |
|
| `PreEditDensityCache` / `VoxelForgePreEditDensityCache::BuildEditedGrid` | — | Per-world cache of the pawn's current L0 tile: exact 35³ pre-edit samples, max **171,500 bytes**. It is pruned when the pawn crosses chunk boundaries and cleared on `RegenerateAllChunks`. `BuildEditedGrid` snapshots touched diff chunks once and reproduces `GetDensityAt`'s final `pre-edit - EvaluateMods` operation before the per-call shared-grid mesher path. |
|
||||||
| `UpdateChunksAroundPosition` / `GetGenerationAdmissionCount` | 10990 / 10971 | Shared Core/legacy scheduler. Builds the desired set and delta-culls leavers; admits the occupied/support tile plus collision-required LOD0 tiles in the pawn's 3×3 XY ring as `CollisionCritical`. S5 coarse-first admission stays opt-in (`voxel.StreamingCoarseFirst=1`). `voxel.StreamingDynamicNearReserve=1` is now the default: it keeps the authored near reserve while moving or until the stationary walkable ring is ready, then makes those slots available to coarse work. Admission counts submitted queued/running tasks, not completed results waiting in the MPSC result queues; `voxel.StreamingLegacyPendingResultCap=1` restores the old `PendingTiles`-based cap for the negative control. `PendingTiles` still lasts through game-thread apply for duplicate suppression and result bookkeeping. The clipmap stays non-overlapping and load-before-unload / `TransitionHold` reconciliation remains authoritative. Standalone has no mirror of this actor scheduler. §8.10 + §8.26–8.28. |
|
| `UpdateChunksAroundPosition` / `GetGenerationAdmissionCount` | 10990 / 10971 | Shared Core/legacy scheduler. Builds the desired set and delta-culls leavers; admits the occupied/support tile plus collision-required LOD0 tiles in the pawn's 3×3 XY ring as `CollisionCritical`. S5 coarse-first admission stays opt-in (`voxel.StreamingCoarseFirst=1`). `voxel.StreamingDynamicNearReserve=1` is the default: it keeps the authored near reserve while moving or until the stationary walkable ring is ready, then makes those slots available to coarse work. Admission counts submitted queued/running tasks, not completed results waiting in the MPSC result queues; `voxel.StreamingLegacyPendingResultCap=1` restores the old `PendingTiles`-based cap for the negative control. `voxel.StreamingFarApplySupplement=1` is the default L121 lane: after the visible-apply cap, at most one far result is admitted when under half the apply-time budget and startup boost is inactive. `PendingTiles` still lasts through game-thread apply for duplicate suppression and result bookkeeping. The clipmap stays non-overlapping and load-before-unload / `TransitionHold` reconciliation remains authoritative. Standalone has no mirror of this actor scheduler. §8.10 + §8.26–8.28. |
|
||||||
| `AdvanceHeadlessCollisionGateStressTest` / `ObserveHeadlessCollisionGateStressTest` / `MaybeFinishHeadlessCollisionGateStressTest` / `MeasureHeadlessCollisionGateStressTerrain` | — | Explicit `-voxel.TestCollisionGateStress=1` harness only. The normal spawn-height cvar offsets from the measured density surface; `voxel.TestCollisionGateStressDeepDrop=1` carves a process-local box shaft from above the top to just above the floor tile, extending along the post-landing route so the pawn does not run into the shaft sidewall, temporarily raises only the transient test DiffLayer brush-radius allowance for that fixture, removes only the local pawn's redundant streaming anchor for this isolated repro, proves every sample in two complete outside-ring level-0 tiles is air before seeding their known-empty status, prewarms the known-empty in-ring column, and asserts that the support walk reaches the in-layout floor outside the aligned level-0 render ring before the pawn drops. The log reports whether the fixed gate retains that otherwise out-of-ring target. Other streaming anchors remain active. The fixture rejects a target or floor outside the layout, a non-air empty-tile proof, or a floor inside the aligned level-0 ring. The `GetDensityAt` column at capsule feet defines the local air/solid surface; the deep-drop column crossing is cached after its first exact scan. Each frame records actor position, velocity, direct foot tile and gate support readiness, gate state, signed feet clearance, and a ≤0.5-voxel sweep between positions. PASS requires no measured clearance below −5 cm; deep-drop gate-on also requires horizontal travel during a hold. Result logs include the terminal density column, first crossing below spawn height, hold count/time, and gated horizontal progress. |
|
| `AdvanceHeadlessCollisionGateStressTest` / `ObserveHeadlessCollisionGateStressTest` / `MaybeFinishHeadlessCollisionGateStressTest` / `MeasureHeadlessCollisionGateStressTerrain` | — | Explicit `-voxel.TestCollisionGateStress=1` harness only. The normal spawn-height cvar offsets from the measured density surface; `voxel.TestCollisionGateStressDeepDrop=1` carves a process-local box shaft from above the top to just above the floor tile, extending along the post-landing route so the pawn does not run into the shaft sidewall, temporarily raises only the transient test DiffLayer brush-radius allowance for that fixture, removes only the local pawn's redundant streaming anchor for this isolated repro, proves every sample in two complete outside-ring level-0 tiles is air before seeding their known-empty status, prewarms the known-empty in-ring column, and asserts that the support walk reaches the in-layout floor outside the aligned level-0 render ring before the pawn drops. The log reports whether the fixed gate retains that otherwise out-of-ring target. Other streaming anchors remain active. The fixture rejects a target or floor outside the layout, a non-air empty-tile proof, or a floor inside the aligned level-0 ring. The `GetDensityAt` column at capsule feet defines the local air/solid surface; the deep-drop column crossing is cached after its first exact scan. Each frame records actor position, velocity, direct foot tile and gate support readiness, gate state, signed feet clearance, and a ≤0.5-voxel sweep between positions. PASS requires no measured clearance below −5 cm; deep-drop gate-on also requires horizontal travel during a hold. Result logs include the terminal density column, first crossing below spawn height, hold count/time, and gated horizontal progress. |
|
||||||
| `BuildDesiredTiles` | 6482 | Builds `DesiredSorted`+`DesiredStamped` (player clipmap shells), then adds streaming anchors and the pending player's collision-only support column before the leaver sweep. It orders from the true local pawn position and builds `CriticalDesiredTiles` for the unresolved support target, occupied/support and heading tiles. Also rebuilds `DesiredTransitionDescendants` (ancestor → desired descendants, including face/edge-touching replacements) for the reconcile pass. |
|
| `BuildDesiredTiles` | 6482 | Builds `DesiredSorted`+`DesiredStamped` (player clipmap shells), then adds streaming anchors and the pending player's collision-only support column before the leaver sweep. It orders from the true local pawn position and builds `CriticalDesiredTiles` for the unresolved support target, occupied/support and heading tiles. Also rebuilds `DesiredTransitionDescendants` (ancestor → desired descendants, including face/edge-touching replacements) for the reconcile pass. |
|
||||||
| `CollisionGateDesiredTiles` / `RequestCollisionGateSupportTile` | 5062 | Retains an unresolved support target and up to three level-0 tiles below it across desired-set rebuilds. The additional keys are hidden collision-only tiles, queued ahead of ordinary streaming, and pruned once resolved or outside the active column. |
|
| `CollisionGateDesiredTiles` / `RequestCollisionGateSupportTile` | 5062 | Retains an unresolved support target and up to three level-0 tiles below it across desired-set rebuilds. The additional keys are hidden collision-only tiles, queued ahead of ordinary streaming, and pruned once resolved or outside the active column. |
|
||||||
|
|||||||
@@ -0,0 +1,57 @@
|
|||||||
|
# VoxelForge current state
|
||||||
|
|
||||||
|
As of 2026-10-31. L131 is on `assembler/world` as `8107196`; this L132 worktree is on `wip/l132-dispatch-truth`, starting at `ebae71b`. Current-state claims below point to their code or measured report.
|
||||||
|
|
||||||
|
## What ships today
|
||||||
|
|
||||||
|
- **Core:** evaluator version **227**. Unreal's `AVoxelWorld::bUseVoxelForgeCore` is opt-in and defaults to `false`; the legacy generator remains the default runtime path. [Core version](Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h), [runtime switch](Source/VoxelForge/Public/VoxelWorld.h), [L130 report](Standalone/Research/WorldAssemblerM3/Linux130/REPORT.md).
|
||||||
|
- **Verified owner4 generator dispatch (L132):** SW binds `SurfaceWorld-native` because the owner export contains a matching `SurfaceWorld.Heightfield2D` v2 plan. FI binds `compiled-analytic/FloatingIslands` because its `FloatingIslands.ClusterField` v0 layout has no matching native island plan. TN and VS bind `compiled-analytic/TunnelNetwork` and `compiled-analytic/VerticalShafts`; their legacy recipe IDs provide neither a `NativeCaves`/`NativeLattice` graph recipe nor native field capability. The L132 fast-tier report prints the binding and generated tile-center share at each owner strate. [Dispatch](Standalone/Core/src/ShapeWorldInternal.h), [adapter recipe checks](Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h), [L132 report](Standalone/Research/WorldAssemblerM3/Linux132/REPORT.md).
|
||||||
|
- **Passages:** continuous smooth swept tubes with a continuous floor profile. L125 stamps conservative route slabs and outside sentinels into the tile cache; exact route geometry is still evaluated. All 21,425 route samples passed the open-air walk check; the nine-station floor probe's maximum root offset was 7.884 mm. L130 sets the shared influence reach to **1.0533 m** (0.75 m shell + 0.21 m relief + 0.50 m smooth-max support / 6 + 0.01 m margin), converted once from metres to Core voxels for the proof, feature, fallback, and off-grid bounds. [L125](Standalone/Research/WorldAssemblerM3/Linux125/REPORT.md), [L130](Standalone/Research/WorldAssemblerM3/Linux130/REPORT.md).
|
||||||
|
- **Streaming:** arrival uses a bounded startup apply burst: `voxel.StreamingStartupApplyBoost=1` by default, up to 24 visible meshes or 14 ms per frame for at most 30 seconds, until the initial desired set and walkable ring are ready. `voxel.StreamingFarApplySupplement=1` is also default-on: after the normal visible apply cap, at most one far result applies when under half the apply-time budget; it is inactive during the startup burst. Dynamic near reserve defaults on; coarse-first remains off; the legacy pending-result cap defaults off. The far-apply lane is Unreal-only. [CVar source](Source/VoxelForge/Private/VoxelWorld.cpp), [L121 report](Standalone/Research/WorldAssemblerM3/Linux121/REPORT.md), [architecture](ARCHITECTURE.md).
|
||||||
|
- **Determinism:** proven on Linux with Clang and GCC at W1/W8/W12. L131's unchanged-output comparison covers owner4's SW/FI/TN/VS views in 24 compiler/worker cases; the canonical owner4/owner2 recipe matrix also passes. Windows determinism is unverified. [L131 checks](Standalone/Research/WorldAssemblerM3/Linux131/REPORT.md), [v227 canonical hashes](Standalone/Tools/verify/baselines/owner-hash/canonical/).
|
||||||
|
|
||||||
|
## Current performance references
|
||||||
|
|
||||||
|
Headless medians in seconds, five interleaved L80-style measurements per version in each cell. Near-ready means the complete LOD0–2 set; whole-view is the full requested view. Values below are the edited L131 binary; the paired pre-cleanup comparison is in the [L131 report](Standalone/Research/WorldAssemblerM3/Linux131/REPORT.md) and [timing summary](Saved/Linux/l131/timing/summary.json).
|
||||||
|
|
||||||
|
| Stratum | Near-ready W8 / W12 | Whole-view W8 / W12 |
|
||||||
|
|---|---:|---:|
|
||||||
|
| S0 | 0.804 / 0.848 | 1.811 / 1.832 |
|
||||||
|
| S1 | 1.014 / 1.002 | 1.993 / 1.886 |
|
||||||
|
| S2 | 1.060 / 1.012 | 1.960 / 1.865 |
|
||||||
|
| S3 | 0.917 / 0.884 | 1.887 / 1.782 |
|
||||||
|
|
||||||
|
Across 40 paired runs per metric, the mean relative change from pre-cleanup L130 was −0.22% near-ready (95% CI −1.02% to +0.57%) and −0.29% whole-view (−1.04% to +0.46%); neither indicates a measurable slowdown. The machine gate was load 0.81 and 99.2% idle. [L80 protocol](Standalone/Research/WorldAssemblerM3/Linux80/REPORT.md), [runner and raw runs](Standalone/Research/WorldAssemblerM3/Linux131/RunL131Timing.py).
|
||||||
|
|
||||||
|
The latest recorded Linux Unreal start-to-LOD0–2 values are L127's one run per stratum: S0 2.456 s, S1 2.231 s, S2 2.514 s, S3 2.221 s. L130 did not rerun UE startup or movement. Five L127 runs at 30 m/s, 12 s moving plus 5 s dwell, had median moving near coverage **40.22%**, far holes **3.46%**, tick p99 **10.80 ms**, and worst tick **13.27 ms**. After stopping, near coverage reached 100% and far holes reached 0%. These are the latest UE references, not v227 UE measurements. [L127 report](Standalone/Research/WorldAssemblerM3/Linux127/REPORT.md), [L127 movement summary](Saved/Linux/l127/ue/movement-pairs-summary.json), [L127 start summaries](Saved/Linux/l127/ue/merge/start-summary.json).
|
||||||
|
|
||||||
|
## Open gaps
|
||||||
|
|
||||||
|
- Multiplayer world lifecycle is unfinished: seed/layout replication, authoritative modification RPCs, and serialized diff replay for late join are still open. See [ARCHITECTURE §9.5 and §9.7](ARCHITECTURE.md).
|
||||||
|
- Determinism has not been verified on Windows. Linux Clang/GCC agreement does not establish MSVC agreement.
|
||||||
|
- NullRHI measures residency and apply timing, not the first presented pixel or rendered enclosure. L127 also found four low-contrast views among 23 otherwise verified captures. [L121](Standalone/Research/WorldAssemblerM3/Linux121/REPORT.md), [L127](Standalone/Research/WorldAssemblerM3/Linux127/REPORT.md).
|
||||||
|
- UE startup/movement numbers above predate L130's v227 reach change; there is no L130/L131 UE movement comparison.
|
||||||
|
|
||||||
|
## How to verify
|
||||||
|
|
||||||
|
- Build standalone Clang or GCC with [`Standalone/BuildLinux.sh`](Standalone/BuildLinux.sh), for example `bash Standalone/BuildLinux.sh --compiler=clang++-19 --variant=l132-clang --fp-contract=off --jobs=4`. The script pins CPUs 0–11 and applies `prlimit --as=6442450944` itself.
|
||||||
|
- Fast tier: `python3 Standalone/Tools/verify/verify.py fast --out Saved/Linux/l132 --overall-timeout 1740`. It includes the L132 four-pose owner density-dispatch report; per-run logs and the summary stay under `Saved/Linux/l132`.
|
||||||
|
- Canonical output baselines are the v227 files in `Standalone/Tools/verify/baselines/owner-hash/canonical/`. L131 also provides reproducible pre-cleanup identity matrices for owner4 tile hashes and owner4/owner2 canonical hashes under `Standalone/Research/WorldAssemblerM3/Linux131/`.
|
||||||
|
- Timing follows L80: quiet machine, no overlapping build, one warm-up per version, five interleaved runs per stratum and worker count, W8/W12. Each headless run uses `taskset -c 0-11`, `prlimit --as=6442450944`, and a 1,740 s timeout. See [`RunL131Timing.py`](Standalone/Research/WorldAssemblerM3/Linux131/RunL131Timing.py).
|
||||||
|
- Linux host build: `/home/jahni/UnrealEngine/Engine/Build/BatchFiles/Linux/Build.sh VFHostEditor Linux Development -Project=/home/jahni/VFHost/VFHost.uproject -NoUBA -MaxParallelActions=4 -Log=/home/jahni/VoxelM/Plugins/VoxelForge/Saved/Linux/l131/ubt.log`. Run Unreal without `prlimit`.
|
||||||
|
|
||||||
|
## Closed experiments
|
||||||
|
|
||||||
|
These rejected speed/streaming proposals stay closed unless new evidence changes their measured failure. Reports and surviving patches are linked in the [L131 inventory](Standalone/Research/WorldAssemblerM3/Linux131/REPORT.md).
|
||||||
|
|
||||||
|
| Line | Result |
|
||||||
|
|---|---|
|
||||||
|
| L116 tunnel chord bound and z cache | Pair count fell 20.8%, but stayed 1.96× L109 and readiness was 15–30% slower. Patch: [`chord-bound-zcolumn-not-merged.patch`](Standalone/Research/WorldAssemblerM3/Linux116/patches/chord-bound-zcolumn-not-merged.patch), commit `89504ec`; see [report](Standalone/Research/WorldAssemblerM3/Linux116/REPORT.md). |
|
||||||
|
| L118 pair pruning / deferred floor work | Pruned 17–18% of candidate pairs but remained 17.2–25.7% behind L109 near-ready. [Report](Standalone/Research/WorldAssemblerM3/Linux118/REPORT.md); patch: [`l118-designs.patch`](Standalone/Research/WorldAssemblerM3/Linux118/l118-designs.patch). |
|
||||||
|
| L126 floor-call filters | Cut floor calls 20–23% without reducing pair counts; all eight cells missed L109. Candidate source was restored and no standalone patch was kept; see [report](Standalone/Research/WorldAssemblerM3/Linux126/REPORT.md) and commit `01a5094`. |
|
||||||
|
| L128 half-open slab ownership | Ownership audit passed, but readiness was 14.8–28.0% behind L109 and moving coverage dropped. Report and patch `l128-slab-ownership.patch` are in archived commit `7022d02`. |
|
||||||
|
| L129 2 m physical reach | v225's metre/Core-voxel bounds mismatch made the correctly bounded candidate do more work; it was slower and failed UE movement gates. Report and patch `l129-reach-2m.patch` are in archived commit `0d1140c`. |
|
||||||
|
| L120 priority split / extra worker slots | Neither met coverage and tick gates; 10 slots reached a 16.67 ms worst tick against 15.43 ms baseline. [Report](Standalone/Research/WorldAssemblerM3/Linux120/REPORT.md); no production patch retained. |
|
||||||
|
| L122 admission relaxation | Early admission missed p99 or far-hole gates; no behavior retained. [Report](Standalone/Research/WorldAssemblerM3/Linux122/REPORT.md); patch: [`l122-trace.patch`](Standalone/Research/WorldAssemblerM3/Linux122/l122-trace.patch). |
|
||||||
|
| L123 lookahead / completion gate | Lookahead failed p99 and worsened near/far coverage; queue-only also failed p99. [Report](Standalone/Research/WorldAssemblerM3/Linux123/REPORT.md); patch: [`l123-queue-lookahead.patch`](Standalone/Research/WorldAssemblerM3/Linux123/l123-queue-lookahead.patch). |
|
||||||
|
| L124 early LOD0 pages | Optimistic gain stayed below 100 ms except negative cases; no publication/scheduling implementation was made. [Report](Standalone/Research/WorldAssemblerM3/Linux124/REPORT.md). |
|
||||||
@@ -423,6 +423,7 @@ void RecordApplyFrame(const FApplyFrameSample& Sample)
|
|||||||
"\"active_tasks\":%d,\"running_workers\":%d,\"task_cap\":%d,"
|
"\"active_tasks\":%d,\"running_workers\":%d,\"task_cap\":%d,"
|
||||||
"\"pending_tiles\":%d,\"pending_results_estimate\":%d,"
|
"\"pending_tiles\":%d,\"pending_results_estimate\":%d,"
|
||||||
"\"results_drained\":%d,\"apply_results\":%d,\"meshes_applied\":%d,\"max_applies\":%d,"
|
"\"results_drained\":%d,\"apply_results\":%d,\"meshes_applied\":%d,\"max_applies\":%d,"
|
||||||
|
"\"supplemental_far_applies\":%d,"
|
||||||
"\"apply_budget_ms\":%s,\"count_limit_reached\":%d,\"time_limit_reached\":%d,"
|
"\"apply_budget_ms\":%s,\"count_limit_reached\":%d,\"time_limit_reached\":%d,"
|
||||||
"\"pending_by_lod\":%s,\"apply_results_by_lod\":%s,"
|
"\"pending_by_lod\":%s,\"apply_results_by_lod\":%s,"
|
||||||
"\"meshes_by_lod\":%s,\"apply_ms_by_lod\":%s"),
|
"\"meshes_by_lod\":%s,\"apply_ms_by_lod\":%s"),
|
||||||
@@ -432,7 +433,8 @@ void RecordApplyFrame(const FApplyFrameSample& Sample)
|
|||||||
Sample.RunningWorkerTasks, Sample.TaskCap, Sample.PendingTiles,
|
Sample.RunningWorkerTasks, Sample.TaskCap, Sample.PendingTiles,
|
||||||
Sample.PendingResultsEstimate, Sample.ResultsDrained, Sample.ApplyResultsByLevelTotal,
|
Sample.PendingResultsEstimate, Sample.ResultsDrained, Sample.ApplyResultsByLevelTotal,
|
||||||
Sample.MeshesApplied,
|
Sample.MeshesApplied,
|
||||||
Sample.MaxApplies, *Number(Sample.ApplyBudgetMilliseconds),
|
Sample.MaxApplies, Sample.SupplementalFarApplies,
|
||||||
|
*Number(Sample.ApplyBudgetMilliseconds),
|
||||||
Sample.bCountLimitReached ? 1 : 0, Sample.bTimeLimitReached ? 1 : 0,
|
Sample.bCountLimitReached ? 1 : 0, Sample.bTimeLimitReached ? 1 : 0,
|
||||||
*PendingByLevel, *ApplyResultsByLevel, *MeshesByLevel, *ApplyMillisecondsByLevel));
|
*PendingByLevel, *ApplyResultsByLevel, *MeshesByLevel, *ApplyMillisecondsByLevel));
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -268,6 +268,12 @@ namespace
|
|||||||
GVoxelForgeStreamingResultBacklogCap,
|
GVoxelForgeStreamingResultBacklogCap,
|
||||||
TEXT("Maximum queued results plus admitted tile tasks; 0 disables the safety cap. This is separate from the worker-slot cap."));
|
TEXT("Maximum queued results plus admitted tile tasks; 0 disables the safety cap. This is separate from the worker-slot cap."));
|
||||||
|
|
||||||
|
int32 GVoxelForgeStreamingFarApplySupplement = 1;
|
||||||
|
FAutoConsoleVariableRef CVarVoxelForgeStreamingFarApplySupplement(
|
||||||
|
TEXT("voxel.StreamingFarApplySupplement"),
|
||||||
|
GVoxelForgeStreamingFarApplySupplement,
|
||||||
|
TEXT("After the normal visible-apply count is reached, allow one far mesh apply when half of the frame time budget remains; 0 disables the supplemental far lane."));
|
||||||
|
|
||||||
int32 GVoxelForgeStreamingReadyQueueDepth = 64;
|
int32 GVoxelForgeStreamingReadyQueueDepth = 64;
|
||||||
FAutoConsoleVariableRef CVarVoxelForgeStreamingReadyQueueDepth(
|
FAutoConsoleVariableRef CVarVoxelForgeStreamingReadyQueueDepth(
|
||||||
TEXT("voxel.StreamingReadyQueueDepth"),
|
TEXT("voxel.StreamingReadyQueueDepth"),
|
||||||
@@ -2223,6 +2229,12 @@ void AVoxelWorld::BeginPlay()
|
|||||||
}
|
}
|
||||||
FParse::Value(FCommandLine::Get(), TEXT("voxel.StreamingStartupApplyBoost="),
|
FParse::Value(FCommandLine::Get(), TEXT("voxel.StreamingStartupApplyBoost="),
|
||||||
GVoxelForgeStreamingStartupApplyBoost);
|
GVoxelForgeStreamingStartupApplyBoost);
|
||||||
|
if (FParse::Value(FCommandLine::Get(), TEXT("voxel.StreamingFarApplySupplement="),
|
||||||
|
GVoxelForgeStreamingFarApplySupplement))
|
||||||
|
{
|
||||||
|
GVoxelForgeStreamingFarApplySupplement =
|
||||||
|
GVoxelForgeStreamingFarApplySupplement != 0 ? 1 : 0;
|
||||||
|
}
|
||||||
// Headless BuildHost does not reliably apply arbitrary console-variable command-line tokens
|
// Headless BuildHost does not reliably apply arbitrary console-variable command-line tokens
|
||||||
// before BeginPlay; parse the test switches explicitly so A/B runs exercise the requested path.
|
// before BeginPlay; parse the test switches explicitly so A/B runs exercise the requested path.
|
||||||
// BuildHost sans interface n'applique pas toujours les CVar arbitraires avant BeginPlay ;
|
// BuildHost sans interface n'applique pas toujours les CVar arbitraires avant BeginPlay ;
|
||||||
@@ -2737,6 +2749,7 @@ void AVoxelWorld::BeginPlay()
|
|||||||
"\"streaming_collision_lookahead\":%d,\"core_region_concurrent_admission\":%d,"
|
"\"streaming_collision_lookahead\":%d,\"core_region_concurrent_admission\":%d,"
|
||||||
"\"collision_gate\":%d,"
|
"\"collision_gate\":%d,"
|
||||||
"\"streaming_result_backlog_cap\":%d,\"streaming_startup_apply_boost\":%d,"
|
"\"streaming_result_backlog_cap\":%d,\"streaming_startup_apply_boost\":%d,"
|
||||||
|
"\"streaming_far_apply_supplement\":%d,"
|
||||||
"\"streaming_background_worker_reserve\":%d,\"core_parallel_for_background_priority\":%d,"
|
"\"streaming_background_worker_reserve\":%d,\"core_parallel_for_background_priority\":%d,"
|
||||||
"\"streaming_tile_background_low\":%d,\"streaming_tile_background_normal\":%d,"
|
"\"streaming_tile_background_low\":%d,\"streaming_tile_background_normal\":%d,"
|
||||||
"\"core_column_cache_direct_mapped\":%d,"
|
"\"core_column_cache_direct_mapped\":%d,"
|
||||||
@@ -2758,6 +2771,7 @@ void AVoxelWorld::BeginPlay()
|
|||||||
GVoxelForgeStreamingCollisionLookahead, GVoxelForgeCoreRegionConcurrentAdmission,
|
GVoxelForgeStreamingCollisionLookahead, GVoxelForgeCoreRegionConcurrentAdmission,
|
||||||
GVoxelForgeCollisionGate,
|
GVoxelForgeCollisionGate,
|
||||||
GVoxelForgeStreamingResultBacklogCap, GVoxelForgeStreamingStartupApplyBoost,
|
GVoxelForgeStreamingResultBacklogCap, GVoxelForgeStreamingStartupApplyBoost,
|
||||||
|
GVoxelForgeStreamingFarApplySupplement,
|
||||||
GVoxelForgeStreamingBackgroundWorkerReserve,
|
GVoxelForgeStreamingBackgroundWorkerReserve,
|
||||||
GVoxelForgeCoreParallelForBackgroundPriority,
|
GVoxelForgeCoreParallelForBackgroundPriority,
|
||||||
GVoxelForgeStreamingTileBackgroundLow,
|
GVoxelForgeStreamingTileBackgroundLow,
|
||||||
@@ -9881,6 +9895,7 @@ void AVoxelWorld::ProcessPendingChunks()
|
|||||||
const double ApplyBudgetSeconds =
|
const double ApplyBudgetSeconds =
|
||||||
static_cast<double>(ApplyBudgetMilliseconds) * 0.001;
|
static_cast<double>(ApplyBudgetMilliseconds) * 0.001;
|
||||||
int32 MeshesApplied = 0;
|
int32 MeshesApplied = 0;
|
||||||
|
int32 SupplementalFarApplies = 0;
|
||||||
int32 ResultsDrained = 0;
|
int32 ResultsDrained = 0;
|
||||||
int32 ApplyResultsByLevelTotal = 0;
|
int32 ApplyResultsByLevelTotal = 0;
|
||||||
double ApplySeconds = 0.0;
|
double ApplySeconds = 0.0;
|
||||||
@@ -9888,6 +9903,8 @@ void AVoxelWorld::ProcessPendingChunks()
|
|||||||
TStaticArray<int32, 9> MeshesAppliedByLevel{};
|
TStaticArray<int32, 9> MeshesAppliedByLevel{};
|
||||||
TStaticArray<double, 9> ApplySecondsByLevel{};
|
TStaticArray<double, 9> ApplySecondsByLevel{};
|
||||||
bool bTimeBudgetHit = false;
|
bool bTimeBudgetHit = false;
|
||||||
|
bool bForceSupplementalFarDequeue = false;
|
||||||
|
bool bDequeuedSupplementalFar = false;
|
||||||
const auto RecordApplyFrame = [&]()
|
const auto RecordApplyFrame = [&]()
|
||||||
{
|
{
|
||||||
if (!VoxelForgeStartupTrace::IsActive()) return;
|
if (!VoxelForgeStartupTrace::IsActive()) return;
|
||||||
@@ -9911,6 +9928,7 @@ void AVoxelWorld::ProcessPendingChunks()
|
|||||||
Sample.ResultsDrained = ResultsDrained;
|
Sample.ResultsDrained = ResultsDrained;
|
||||||
Sample.ApplyResultsByLevelTotal = ApplyResultsByLevelTotal;
|
Sample.ApplyResultsByLevelTotal = ApplyResultsByLevelTotal;
|
||||||
Sample.MeshesApplied = MeshesApplied;
|
Sample.MeshesApplied = MeshesApplied;
|
||||||
|
Sample.SupplementalFarApplies = SupplementalFarApplies;
|
||||||
Sample.MaxApplies = MaxApplies;
|
Sample.MaxApplies = MaxApplies;
|
||||||
Sample.ApplyBudgetMilliseconds = ApplyBudgetMilliseconds;
|
Sample.ApplyBudgetMilliseconds = ApplyBudgetMilliseconds;
|
||||||
Sample.bCountLimitReached = MeshesApplied >= MaxApplies;
|
Sample.bCountLimitReached = MeshesApplied >= MaxApplies;
|
||||||
@@ -9939,6 +9957,7 @@ void AVoxelWorld::ProcessPendingChunks()
|
|||||||
FChunkResult DequeuedChunk;
|
FChunkResult DequeuedChunk;
|
||||||
const auto DequeueNextResult = [&]() -> bool
|
const auto DequeueNextResult = [&]() -> bool
|
||||||
{
|
{
|
||||||
|
bDequeuedSupplementalFar = false;
|
||||||
if (bStep1UploadCorpusReleased
|
if (bStep1UploadCorpusReleased
|
||||||
&& Step1HeldApplyCursor < Step1HeldApplyResults.Num())
|
&& Step1HeldApplyCursor < Step1HeldApplyResults.Num())
|
||||||
{
|
{
|
||||||
@@ -9954,8 +9973,19 @@ void AVoxelWorld::ProcessPendingChunks()
|
|||||||
// Mirror the admission order. A horizon result that completed earlier must not consume
|
// Mirror the admission order. A horizon result that completed earlier must not consume
|
||||||
// the game-thread apply budget ahead of collision support, an edit, or ordinary LOD0.
|
// the game-thread apply budget ahead of collision support, an edit, or ordinary LOD0.
|
||||||
if (CriticalProcessQueue.Dequeue(DequeuedChunk)
|
if (CriticalProcessQueue.Dequeue(DequeuedChunk)
|
||||||
|| EditedProcessQueue.Dequeue(DequeuedChunk)
|
|| EditedProcessQueue.Dequeue(DequeuedChunk))
|
||||||
|| NearProcessQueue.Dequeue(DequeuedChunk)
|
{
|
||||||
|
RecordStreamingResultDequeued(DequeuedChunk);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (bForceSupplementalFarDequeue)
|
||||||
|
{
|
||||||
|
if (!ProcessQueue.Dequeue(DequeuedChunk)) return false;
|
||||||
|
bDequeuedSupplementalFar = true;
|
||||||
|
RecordStreamingResultDequeued(DequeuedChunk);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (NearProcessQueue.Dequeue(DequeuedChunk)
|
||||||
|| ProcessQueue.Dequeue(DequeuedChunk))
|
|| ProcessQueue.Dequeue(DequeuedChunk))
|
||||||
{
|
{
|
||||||
RecordStreamingResultDequeued(DequeuedChunk);
|
RecordStreamingResultDequeued(DequeuedChunk);
|
||||||
@@ -10104,6 +10134,10 @@ void AVoxelWorld::ProcessPendingChunks()
|
|||||||
if (bMeshApplied)
|
if (bMeshApplied)
|
||||||
{
|
{
|
||||||
++MeshesApplied;
|
++MeshesApplied;
|
||||||
|
if (bDequeuedSupplementalFar && ResultLevel >= 3)
|
||||||
|
{
|
||||||
|
++SupplementalFarApplies;
|
||||||
|
}
|
||||||
if (ResultLevel >= 0 && ResultLevel < 9)
|
if (ResultLevel >= 0 && ResultLevel < 9)
|
||||||
{
|
{
|
||||||
++MeshesAppliedByLevel[ResultLevel];
|
++MeshesAppliedByLevel[ResultLevel];
|
||||||
@@ -10122,6 +10156,19 @@ void AVoxelWorld::ProcessPendingChunks()
|
|||||||
}
|
}
|
||||||
if (MeshesApplied >= MaxApplies)
|
if (MeshesApplied >= MaxApplies)
|
||||||
{
|
{
|
||||||
|
const double DrainElapsedSeconds = FPlatformTime::ToSeconds64(
|
||||||
|
FPlatformTime::Cycles64() - DrainStartCycles);
|
||||||
|
const bool bCanSupplementFar = MeshesApplied == MaxApplies
|
||||||
|
&& SupplementalFarApplies == 0
|
||||||
|
&& !bUseStartupApplyBoost
|
||||||
|
&& GVoxelForgeStreamingFarApplySupplement != 0
|
||||||
|
&& ApplyBudgetSeconds > 0.0
|
||||||
|
&& DrainElapsedSeconds < ApplyBudgetSeconds * 0.5;
|
||||||
|
if (bCanSupplementFar)
|
||||||
|
{
|
||||||
|
bForceSupplementalFarDequeue = true;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -15360,6 +15407,8 @@ bool AVoxelWorld::ApplyMeshToTile(FChunkResult& Result)
|
|||||||
const int32 NumIndices = Result.NumTriangles * 3;
|
const int32 NumIndices = Result.NumTriangles * 3;
|
||||||
const int32 FirstCapIndex = FMath::Clamp(
|
const int32 FirstCapIndex = FMath::Clamp(
|
||||||
NumIndices - Result.NumCeilingTriangles * 3, 0, NumIndices);
|
NumIndices - Result.NumCeilingTriangles * 3, 0, NumIndices);
|
||||||
|
TFuture<ERealtimeMeshProxyUpdateStatus> ProxyPublicationFuture;
|
||||||
|
bool bHasProxyPublicationFuture = false;
|
||||||
{
|
{
|
||||||
TRACE_CPUPROFILER_EVENT_SCOPE(VoxelForge_RMC_CreateSections);
|
TRACE_CPUPROFILER_EVENT_SCOPE(VoxelForge_RMC_CreateSections);
|
||||||
if (bHasGroundTris)
|
if (bHasGroundTris)
|
||||||
@@ -15369,9 +15418,10 @@ bool AVoxelWorld::ApplyMeshToTile(FChunkResult& Result)
|
|||||||
const FRealtimeMeshStreamRange GroundRange =
|
const FRealtimeMeshStreamRange GroundRange =
|
||||||
URealtimeMeshBlueprintFunctionLibrary::MakeStreamRange(
|
URealtimeMeshBlueprintFunctionLibrary::MakeStreamRange(
|
||||||
0, Result.NumVertices, 0, FirstCapIndex);
|
0, Result.NumVertices, 0, FirstCapIndex);
|
||||||
RTMesh->CreateSection(
|
ProxyPublicationFuture = RTMesh->CreateSection(
|
||||||
FRealtimeMeshSectionKey::CreateForPolyGroup(GroupKey, 0),
|
FRealtimeMeshSectionKey::CreateForPolyGroup(GroupKey, 0),
|
||||||
GroundConfig, GroundRange, /*bShouldCreateCollision*/ bWantsCollision);
|
GroundConfig, GroundRange, /*bShouldCreateCollision*/ bWantsCollision);
|
||||||
|
bHasProxyPublicationFuture = true;
|
||||||
}
|
}
|
||||||
if (bHasCeilingTris)
|
if (bHasCeilingTris)
|
||||||
{
|
{
|
||||||
@@ -15380,11 +15430,48 @@ bool AVoxelWorld::ApplyMeshToTile(FChunkResult& Result)
|
|||||||
const FRealtimeMeshStreamRange CapRange =
|
const FRealtimeMeshStreamRange CapRange =
|
||||||
URealtimeMeshBlueprintFunctionLibrary::MakeStreamRange(
|
URealtimeMeshBlueprintFunctionLibrary::MakeStreamRange(
|
||||||
0, Result.NumVertices, FirstCapIndex, NumIndices);
|
0, Result.NumVertices, FirstCapIndex, NumIndices);
|
||||||
RTMesh->CreateSection(
|
ProxyPublicationFuture = RTMesh->CreateSection(
|
||||||
FRealtimeMeshSectionKey::CreateForPolyGroup(GroupKey, 1),
|
FRealtimeMeshSectionKey::CreateForPolyGroup(GroupKey, 1),
|
||||||
CapConfig, CapRange, /*bShouldCreateCollision*/ bWantsCollision);
|
CapConfig, CapRange, /*bShouldCreateCollision*/ bWantsCollision);
|
||||||
|
bHasProxyPublicationFuture = true;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
const uint64 ProxyPublicationSubmitCycles = FPlatformTime::Cycles64();
|
||||||
|
if (bTraceRmcTiming && bHasProxyPublicationFuture)
|
||||||
|
{
|
||||||
|
const TWeakObjectPtr<AVoxelWorld> WeakWorld(this);
|
||||||
|
const TWeakObjectPtr<URealtimeMeshComponent> WeakComponent(MeshComp);
|
||||||
|
const FVoxelTileKey PublishedTile = Tile;
|
||||||
|
const uint64 RequestOrdinal = Result.RequestOrdinal;
|
||||||
|
const uint64 ApplyStartedCycles = ApplyStartCycles;
|
||||||
|
ProxyPublicationFuture.Next(
|
||||||
|
[WeakWorld, WeakComponent, PublishedTile, RequestOrdinal,
|
||||||
|
ApplyStartedCycles, ProxyPublicationSubmitCycles](
|
||||||
|
ERealtimeMeshProxyUpdateStatus Status)
|
||||||
|
{
|
||||||
|
if (!VoxelForgeStartupTrace::IsActive()) return;
|
||||||
|
const uint64 PublishedCycles = FPlatformTime::Cycles64();
|
||||||
|
const URealtimeMeshComponent* Component = WeakComponent.Get();
|
||||||
|
VoxelForgeStartupTrace::RecordEvent(TEXT("rmc_proxy_published"), FString::Printf(
|
||||||
|
TEXT("\"tile\":[%d,%d,%d],\"level\":%d,\"request_ordinal\":%llu,"
|
||||||
|
"\"proxy_update_status\":%d,\"apply_to_proxy_ms\":%.3f,"
|
||||||
|
"\"section_submit_to_proxy_ms\":%.3f,\"component_visible\":%d,"
|
||||||
|
"\"world_valid\":%d"),
|
||||||
|
PublishedTile.Coord.X, PublishedTile.Coord.Y, PublishedTile.Coord.Z,
|
||||||
|
PublishedTile.Level, static_cast<unsigned long long>(RequestOrdinal),
|
||||||
|
static_cast<int32>(Status),
|
||||||
|
ApplyStartedCycles != 0 && PublishedCycles >= ApplyStartedCycles
|
||||||
|
? FPlatformTime::ToSeconds64(PublishedCycles - ApplyStartedCycles) * 1000.0
|
||||||
|
: 0.0,
|
||||||
|
ProxyPublicationSubmitCycles != 0
|
||||||
|
&& PublishedCycles >= ProxyPublicationSubmitCycles
|
||||||
|
? FPlatformTime::ToSeconds64(
|
||||||
|
PublishedCycles - ProxyPublicationSubmitCycles) * 1000.0
|
||||||
|
: 0.0,
|
||||||
|
Component && Component->IsVisible() ? 1 : 0,
|
||||||
|
WeakWorld.IsValid() ? 1 : 0));
|
||||||
|
});
|
||||||
|
}
|
||||||
if (bTraceRmcTiming)
|
if (bTraceRmcTiming)
|
||||||
{
|
{
|
||||||
TraceSectionConfigMilliseconds = FPlatformTime::ToSeconds64(
|
TraceSectionConfigMilliseconds = FPlatformTime::ToSeconds64(
|
||||||
|
|||||||
@@ -59,6 +59,7 @@ namespace VoxelForgeStartupTrace
|
|||||||
int32 ResultsDrained = 0;
|
int32 ResultsDrained = 0;
|
||||||
int32 ApplyResultsByLevelTotal = 0;
|
int32 ApplyResultsByLevelTotal = 0;
|
||||||
int32 MeshesApplied = 0;
|
int32 MeshesApplied = 0;
|
||||||
|
int32 SupplementalFarApplies = 0;
|
||||||
int32 MaxApplies = 0;
|
int32 MaxApplies = 0;
|
||||||
double ApplyBudgetMilliseconds = 0.0;
|
double ApplyBudgetMilliseconds = 0.0;
|
||||||
bool bCountLimitReached = false;
|
bool bCountLimitReached = false;
|
||||||
|
|||||||
@@ -494,7 +494,10 @@ namespace VoxelForgeCore
|
|||||||
// Version 223 projects the floor onto one continuous arc-length profile and groups
|
// Version 223 projects the floor onto one continuous arc-length profile and groups
|
||||||
// dense passage samples into coarse tile candidates. Version 224 keeps the same
|
// dense passage samples into coarse tile candidates. Version 224 keeps the same
|
||||||
// continuous floor while relaxing the spline chord tolerance and bounding shell work.
|
// continuous floor while relaxing the spline chord tolerance and bounding shell work.
|
||||||
inline constexpr std::uint32_t WorldAssemblyEvaluatorVersion = 225;
|
// Version 226 unifies the passage influence reach in metres and converts it to core
|
||||||
|
// voxels for every broadphase and proof bound. Version 227 uses the exact smooth-max
|
||||||
|
// support width/6 when deriving that reach.
|
||||||
|
inline constexpr std::uint32_t WorldAssemblyEvaluatorVersion = 227;
|
||||||
|
|
||||||
enum class FieldChannel : std::uint32_t
|
enum class FieldChannel : std::uint32_t
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -33,50 +33,41 @@
|
|||||||
#include <variant>
|
#include <variant>
|
||||||
#include <vector>
|
#include <vector>
|
||||||
|
|
||||||
// L88-only composition split. These switches let the timing harness remove passage work and
|
static bool VF_IsDiagnosticEnvSwitchEnabled(const char* Name)
|
||||||
// resolved boundary-band work independently. They are diagnostic only, never world settings.
|
|
||||||
static bool VF_IsL88DiagnosticSwitchEnabled(const char* Name)
|
|
||||||
{
|
{
|
||||||
const char* Value=std::getenv(Name);
|
const char* Value=std::getenv(Name);
|
||||||
return Value&&Value[0]=='1'&&Value[1]=='\0';
|
return Value&&Value[0]=='1'&&Value[1]=='\0';
|
||||||
}
|
}
|
||||||
|
|
||||||
static bool VF_IsL88PassageCompositionDisabled()
|
|
||||||
{
|
|
||||||
static const bool bDisabled=VF_IsL88DiagnosticSwitchEnabled(
|
|
||||||
"VOXELFORGE_L88_DISABLE_PASSAGES")
|
|
||||||
||VF_IsL88DiagnosticSwitchEnabled("VOXELFORGE_L86_DISABLE_PASSAGE_COMPOSITION");
|
|
||||||
return bDisabled;
|
|
||||||
}
|
|
||||||
|
|
||||||
static bool VF_IsL88BoundaryBandsDisabled()
|
|
||||||
{
|
|
||||||
static const bool bDisabled=VF_IsL88DiagnosticSwitchEnabled(
|
|
||||||
"VOXELFORGE_L88_DISABLE_BOUNDARY_BANDS")
|
|
||||||
||VF_IsL88DiagnosticSwitchEnabled("VOXELFORGE_L86_DISABLE_PASSAGE_COMPOSITION");
|
|
||||||
return bDisabled;
|
|
||||||
}
|
|
||||||
|
|
||||||
static bool VF_IsL88AllowPassageAblationWalkabilityFailure()
|
|
||||||
{
|
|
||||||
static const bool bEnabled=VF_IsL88DiagnosticSwitchEnabled(
|
|
||||||
"VOXELFORGE_L88_ALLOW_PASSAGE_ABLATION_WALKABILITY_FAILURE");
|
|
||||||
return bEnabled;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
static bool VF_IsL98PassagePrepareTraceEnabled()
|
static bool VF_IsL98PassagePrepareTraceEnabled()
|
||||||
{
|
{
|
||||||
static const bool bEnabled=VF_IsL88DiagnosticSwitchEnabled(
|
static const bool bEnabled=VF_IsDiagnosticEnvSwitchEnabled(
|
||||||
"VOXELFORGE_L98_PASSAGE_PREP_TRACE");
|
"VOXELFORGE_L98_PASSAGE_PREP_TRACE");
|
||||||
return bEnabled;
|
return bEnabled;
|
||||||
}
|
}
|
||||||
|
|
||||||
static double VF_L93PassageCarveReachMetres()
|
static bool VF_IsL130PassageReachProofAuditEnabled()
|
||||||
{
|
{
|
||||||
// The generated world uses a fixed bounded tube influence. An explicit diagnostic
|
static const bool bEnabled=VF_IsDiagnosticEnvSwitchEnabled(
|
||||||
// option can still select the legacy point-evaluation path for comparisons.
|
"VOXELFORGE_L130_PASSAGE_REACH_PROOF_AUDIT");
|
||||||
return 4.0;
|
return bEnabled;
|
||||||
|
}
|
||||||
|
|
||||||
|
static constexpr double VF_PassageShellThicknessMetres=0.75;
|
||||||
|
static constexpr double VF_PassageWallReliefMetres=0.21;
|
||||||
|
static constexpr double VF_PassageCornerBlendMetres=0.50;
|
||||||
|
static constexpr double VF_PassageReachMarginMetres=0.01;
|
||||||
|
// Radial support in metres: shell + maximum relief + the smooth-max support
|
||||||
|
// (corner-blend width / 6) + a 1 cm guard for float and bound rounding.
|
||||||
|
static constexpr double VF_PassageInfluenceReachMetres=
|
||||||
|
VF_PassageShellThicknessMetres+VF_PassageWallReliefMetres
|
||||||
|
+VF_PassageCornerBlendMetres/6.0+VF_PassageReachMarginMetres;
|
||||||
|
|
||||||
|
static double VF_PassageInfluenceReachCoreVoxels(
|
||||||
|
std::uint32_t coreVoxelSizeMicrometres)
|
||||||
|
{
|
||||||
|
return VF_PassageInfluenceReachMetres*1000000.0
|
||||||
|
/std::max<std::uint32_t>(1,coreVoxelSizeMicrometres);
|
||||||
}
|
}
|
||||||
|
|
||||||
enum class EPassageDensityQueryCaller : std::size_t
|
enum class EPassageDensityQueryCaller : std::size_t
|
||||||
@@ -306,22 +297,18 @@ public:
|
|||||||
profileStats_(profileStats),profileLod_(profileLod),
|
profileStats_(profileStats),profileLod_(profileLod),
|
||||||
passageCarveReachMetres_(std::isfinite(passageCarveReachMetres)
|
passageCarveReachMetres_(std::isfinite(passageCarveReachMetres)
|
||||||
&&passageCarveReachMetres>=0.0?passageCarveReachMetres
|
&&passageCarveReachMetres>=0.0?passageCarveReachMetres
|
||||||
:VF_L93PassageCarveReachMetres())
|
:VF_PassageInfluenceReachMetres)
|
||||||
{
|
{
|
||||||
mayContainResolvedBoundaryBand_=shape.ResolvedBoundaryBandsIntersectSubvoxelRange(
|
mayContainResolvedBoundaryBand_=shape.ResolvedBoundaryBandsIntersectSubvoxelRange(
|
||||||
minimumSubvoxels_[2],maximumSubvoxels_[2]);
|
minimumSubvoxels_[2],maximumSubvoxels_[2]);
|
||||||
const bool disablePassageComposition=VF_IsL88PassageCompositionDisabled();
|
mayContainTransitionPassageFade_=shape.ResolvedTransitionPassageFadeIntersectsTileXY(
|
||||||
if(VF_IsL88BoundaryBandsDisabled())mayContainResolvedBoundaryBand_=false;
|
minimumSubvoxels_,maximumSubvoxels_);
|
||||||
mayContainTransitionPassageFade_=!disablePassageComposition
|
featureIndices_=shape.BuildPassageFeatureTileList(
|
||||||
&&shape.ResolvedTransitionPassageFadeIntersectsTileXY(
|
minimumSubvoxels_,maximumSubvoxels_);
|
||||||
minimumSubvoxels_,maximumSubvoxels_);
|
|
||||||
if(!disablePassageComposition)
|
|
||||||
featureIndices_=shape.BuildPassageFeatureTileList(
|
|
||||||
minimumSubvoxels_,maximumSubvoxels_);
|
|
||||||
stampIndices_=shape.BuildStampTileList(minimumSubvoxels_,maximumSubvoxels_);
|
stampIndices_=shape.BuildStampTileList(minimumSubvoxels_,maximumSubvoxels_);
|
||||||
cavernFeatureIndices_=shape.BuildCavernFeatureTileList(
|
cavernFeatureIndices_=shape.BuildCavernFeatureTileList(
|
||||||
minimumSubvoxels_,maximumSubvoxels_);
|
minimumSubvoxels_,maximumSubvoxels_);
|
||||||
passageRoutes_.assign(shape.ResolvedLayout&&!disablePassageComposition
|
passageRoutes_.assign(shape.ResolvedLayout
|
||||||
?shape.ResolvedLayout->passages.size():0,0);
|
?shape.ResolvedLayout->passages.size():0,0);
|
||||||
passageRouteSegments_.resize(passageRoutes_.size());
|
passageRouteSegments_.resize(passageRoutes_.size());
|
||||||
for(const std::uint32_t featureIndex:featureIndices_)
|
for(const std::uint32_t featureIndex:featureIndices_)
|
||||||
@@ -1204,8 +1191,8 @@ public:
|
|||||||
if(!ResolvedLayout||ResolvedLayout->passages.empty()||!std::isfinite(X)
|
if(!ResolvedLayout||ResolvedLayout->passages.empty()||!std::isfinite(X)
|
||||||
||!std::isfinite(Y)||!std::isfinite(Z))return density;
|
||!std::isfinite(Y)||!std::isfinite(Z))return density;
|
||||||
constexpr double Q=static_cast<double>(VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel);
|
constexpr double Q=static_cast<double>(VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel);
|
||||||
constexpr double FloorShellThicknessMetres=0.75;
|
constexpr double FloorShellThicknessMetres=VF_PassageShellThicknessMetres;
|
||||||
constexpr double WallReliefMetres=0.21;
|
constexpr double WallReliefMetres=VF_PassageWallReliefMetres;
|
||||||
constexpr double OpenAirSideWallHeightMetres=1.0;
|
constexpr double OpenAirSideWallHeightMetres=1.0;
|
||||||
constexpr double MouthShellBlendMetres=5.0;
|
constexpr double MouthShellBlendMetres=5.0;
|
||||||
const double qPerMetre=Q*1000000.0/
|
const double qPerMetre=Q*1000000.0/
|
||||||
@@ -1544,12 +1531,8 @@ public:
|
|||||||
// the generic stratum transition distance creates a discontinuity at a
|
// the generic stratum transition distance creates a discontinuity at a
|
||||||
// mouth that crosses a wide transition band. Keep that opt-in field whole;
|
// mouth that crosses a wide transition band. Keep that opt-in field whole;
|
||||||
// its host-rock value already supplies the ceiling and bedrock.
|
// its host-rock value already supplies the ceiling and bedrock.
|
||||||
const bool disablePassageComposition=!nativeLattice
|
|
||||||
&&VF_IsL88PassageCompositionDisabled();
|
|
||||||
const bool disableBoundaryBands=!nativeLattice
|
|
||||||
&&VF_IsL88BoundaryBandsDisabled();
|
|
||||||
float boundary=graphDensity;
|
float boundary=graphDensity;
|
||||||
if(!nativeLattice&&!disableBoundaryBands)
|
if(!nativeLattice)
|
||||||
{
|
{
|
||||||
VoxelForgeCore::Linux77Profiler::FScopedPhase l77BandsPhase(
|
VoxelForgeCore::Linux77Profiler::FScopedPhase l77BandsPhase(
|
||||||
VoxelForgeCore::Linux77Profiler::EPhase::Bands);
|
VoxelForgeCore::Linux77Profiler::EPhase::Bands);
|
||||||
@@ -1578,7 +1561,6 @@ public:
|
|||||||
?bound->Resolved->passageCapabilities.hasNativeCaveField
|
?bound->Resolved->passageCapabilities.hasNativeCaveField
|
||||||
:bound->Resolved->passageCapabilities.hasNativeLatticeField))
|
:bound->Resolved->passageCapabilities.hasNativeLatticeField))
|
||||||
{
|
{
|
||||||
if(!disablePassageComposition)
|
|
||||||
{
|
{
|
||||||
VoxelForgeCore::Linux77Profiler::FScopedPhase l77PassagePhase(
|
VoxelForgeCore::Linux77Profiler::FScopedPhase l77PassagePhase(
|
||||||
VoxelForgeCore::Linux77Profiler::EPhase::Passage);
|
VoxelForgeCore::Linux77Profiler::EPhase::Passage);
|
||||||
@@ -1609,7 +1591,6 @@ public:
|
|||||||
ELegacyFieldTerm::ComposerPassage);
|
ELegacyFieldTerm::ComposerPassage);
|
||||||
final=ApplyResolvedCavernFloorCaps(stamped,X,Y,Z);
|
final=ApplyResolvedCavernFloorCaps(stamped,X,Y,Z);
|
||||||
}
|
}
|
||||||
if(!disablePassageComposition)
|
|
||||||
{
|
{
|
||||||
VoxelForgeCore::Linux77Profiler::FScopedPhase l77PassagePhase(
|
VoxelForgeCore::Linux77Profiler::FScopedPhase l77PassagePhase(
|
||||||
VoxelForgeCore::Linux77Profiler::EPhase::Passage);
|
VoxelForgeCore::Linux77Profiler::EPhase::Passage);
|
||||||
@@ -1630,8 +1611,8 @@ public:
|
|||||||
{
|
{
|
||||||
constexpr double Q=static_cast<double>(
|
constexpr double Q=static_cast<double>(
|
||||||
VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel);
|
VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel);
|
||||||
const double qPerMetre=Q*1000000.0/
|
const double reachCoreVoxels=VF_PassageInfluenceReachCoreVoxels(
|
||||||
std::max<std::uint32_t>(1,ResolvedLayout->coreVoxelSizeMicrometres);
|
ResolvedLayout->coreVoxelSizeMicrometres);
|
||||||
const auto overlaps=[&](const double low[3],const double high[3])
|
const auto overlaps=[&](const double low[3],const double high[3])
|
||||||
{
|
{
|
||||||
for(int axis=0;axis<3;++axis)
|
for(int axis=0;axis<3;++axis)
|
||||||
@@ -1648,7 +1629,8 @@ public:
|
|||||||
const auto& p3=passage.controlPoints[
|
const auto& p3=passage.controlPoints[
|
||||||
std::min(segment+2,passage.controlPoints.size()-1)];
|
std::min(segment+2,passage.controlPoints.size()-1)];
|
||||||
const double radius=std::max(static_cast<double>(a.radiusSubvoxels),
|
const double radius=std::max(static_cast<double>(a.radiusSubvoxels),
|
||||||
static_cast<double>(b.radiusSubvoxels))/Q+1.0;
|
static_cast<double>(b.radiusSubvoxels))/Q
|
||||||
|
+reachCoreVoxels+0.01;
|
||||||
double low[3]{},high[3]{};
|
double low[3]{},high[3]{};
|
||||||
for(int axis=0;axis<3;++axis)
|
for(int axis=0;axis<3;++axis)
|
||||||
{
|
{
|
||||||
@@ -2314,18 +2296,8 @@ public:
|
|||||||
const bool bPassageRoutesValid=!bValidatePassageRoutes||ValidatePassageRoutes();
|
const bool bPassageRoutesValid=!bValidatePassageRoutes||ValidatePassageRoutes();
|
||||||
if(!bPassageRoutesValid)
|
if(!bPassageRoutesValid)
|
||||||
{
|
{
|
||||||
if(VF_IsL88PassageCompositionDisabled()
|
Error="compiled shape world contains a solid passage centreline sample";
|
||||||
&&!VF_IsL88BoundaryBandsDisabled()
|
return false;
|
||||||
&&VF_IsL88AllowPassageAblationWalkabilityFailure())
|
|
||||||
{
|
|
||||||
std::printf("[L88PassageAblation] route walkability failed as expected; "
|
|
||||||
"continuing only to measure passage-off/bands-on readiness\n");
|
|
||||||
}
|
|
||||||
else
|
|
||||||
{
|
|
||||||
Error="compiled shape world contains a solid passage centreline sample";
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
Stats.PassageValidationSeconds=std::chrono::duration<double>(Clock::now()-Substage).count();
|
Stats.PassageValidationSeconds=std::chrono::duration<double>(Clock::now()-Substage).count();
|
||||||
Built=true;
|
Built=true;
|
||||||
@@ -3499,7 +3471,7 @@ private:
|
|||||||
double centerZSubvoxels,double queryZSubvoxels,double floorDepthSubvoxels,
|
double centerZSubvoxels,double queryZSubvoxels,double floorDepthSubvoxels,
|
||||||
double qPerMetre)
|
double qPerMetre)
|
||||||
{
|
{
|
||||||
constexpr double CornerBlendMetres=0.50;
|
constexpr double CornerBlendMetres=VF_PassageCornerBlendMetres;
|
||||||
const float floorDistance=static_cast<float>(centerZSubvoxels
|
const float floorDistance=static_cast<float>(centerZSubvoxels
|
||||||
-floorDepthSubvoxels-queryZSubvoxels);
|
-floorDepthSubvoxels-queryZSubvoxels);
|
||||||
return SmoothMaxDistance(roundDistanceSubvoxels,floorDistance,
|
return SmoothMaxDistance(roundDistanceSubvoxels,floorDistance,
|
||||||
@@ -4811,13 +4783,9 @@ private:
|
|||||||
PassageNearestNodes.resize(ResolvedLayout?ResolvedLayout->passages.size():0);
|
PassageNearestNodes.resize(ResolvedLayout?ResolvedLayout->passages.size():0);
|
||||||
if(!ResolvedLayout||ResolvedLayout->passages.empty())return;
|
if(!ResolvedLayout||ResolvedLayout->passages.empty())return;
|
||||||
constexpr double Q=static_cast<double>(VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel);
|
constexpr double Q=static_cast<double>(VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel);
|
||||||
constexpr double PassageShellMetres=0.75;
|
|
||||||
constexpr double ReliefMetres=0.21;
|
|
||||||
constexpr double SweepReachMetres=4.0;
|
|
||||||
constexpr double CornerBlendMetres=0.50;
|
|
||||||
constexpr std::size_t CandidateFineSamplesPerSpan=5;
|
constexpr std::size_t CandidateFineSamplesPerSpan=5;
|
||||||
const double requiredReach=std::max(SweepReachMetres+ReliefMetres
|
const double requiredReachCoreVoxels=VF_PassageInfluenceReachCoreVoxels(
|
||||||
+CornerBlendMetres*0.25,PassageShellMetres+ReliefMetres);
|
ResolvedLayout->coreVoxelSizeMicrometres);
|
||||||
for(std::size_t passageIndex=0;passageIndex<ResolvedLayout->passages.size();
|
for(std::size_t passageIndex=0;passageIndex<ResolvedLayout->passages.size();
|
||||||
++passageIndex)
|
++passageIndex)
|
||||||
{
|
{
|
||||||
@@ -4854,7 +4822,7 @@ private:
|
|||||||
routeMinimum[axis]=std::min(routeMinimum[axis],localMinimum[axis]);
|
routeMinimum[axis]=std::min(routeMinimum[axis],localMinimum[axis]);
|
||||||
routeMaximum[axis]=std::max(routeMaximum[axis],localMaximum[axis]);
|
routeMaximum[axis]=std::max(routeMaximum[axis],localMaximum[axis]);
|
||||||
}
|
}
|
||||||
const double localExtent=requiredReach
|
const double localExtent=requiredReachCoreVoxels
|
||||||
+std::max(prepared.radiusA,prepared.radiusB)/Q+0.01;
|
+std::max(prepared.radiusA,prepared.radiusB)/Q+0.01;
|
||||||
FPassageProofBound proof;
|
FPassageProofBound proof;
|
||||||
for(std::size_t axis=0;axis<3;++axis)
|
for(std::size_t axis=0;axis<3;++axis)
|
||||||
@@ -4872,7 +4840,8 @@ private:
|
|||||||
std::ceil((localMaximum[axis]+localExtent)*Q));
|
std::ceil((localMaximum[axis]+localExtent)*Q));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
const double routeExtent=maximumRadiusSubvoxels/Q+requiredReach+0.01;
|
const double routeExtent=maximumRadiusSubvoxels/Q
|
||||||
|
+requiredReachCoreVoxels+0.01;
|
||||||
const double tileExtent=routeExtent;
|
const double tileExtent=routeExtent;
|
||||||
std::array<std::int64_t,3> featureMinimum{},featureMaximum{};
|
std::array<std::int64_t,3> featureMinimum{},featureMaximum{};
|
||||||
for(std::size_t axis=0;axis<3;++axis)
|
for(std::size_t axis=0;axis<3;++axis)
|
||||||
@@ -4899,6 +4868,113 @@ private:
|
|||||||
BuildPassageFeatureNode(0,PassageFeatureOrder.size());
|
BuildPassageFeatureNode(0,PassageFeatureOrder.size());
|
||||||
if(!PassageProofBounds.empty())
|
if(!PassageProofBounds.empty())
|
||||||
BuildPassageProofNode(0,PassageProofBounds.size());
|
BuildPassageProofNode(0,PassageProofBounds.size());
|
||||||
|
|
||||||
|
if(VF_IsL130PassageReachProofAuditEnabled())
|
||||||
|
{
|
||||||
|
constexpr double Q=static_cast<double>(
|
||||||
|
VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel);
|
||||||
|
constexpr std::array<std::array<double,2>,8> RadialDirections={{
|
||||||
|
{1.0,0.0},{0.7071067811865475,0.7071067811865475},
|
||||||
|
{0.0,1.0},{-0.7071067811865475,0.7071067811865475},
|
||||||
|
{-1.0,0.0},{-0.7071067811865475,-0.7071067811865475},
|
||||||
|
{0.0,-1.0},{0.7071067811865475,-0.7071067811865475}}};
|
||||||
|
constexpr std::array<double,2> CrestHeightsMetres={0.625,0.875};
|
||||||
|
const double qPerMetre=Q*1000000.0/
|
||||||
|
std::max<std::uint32_t>(1,ResolvedLayout->coreVoxelSizeMicrometres);
|
||||||
|
const std::array<double,6> shellOffsetsMetres={0.70,0.78,0.86,0.94,
|
||||||
|
VF_PassageInfluenceReachMetres-0.005,
|
||||||
|
VF_PassageInfluenceReachMetres+0.005};
|
||||||
|
std::uint64_t sampled=0,signRelevant=0,proofMisses=0,featureMisses=0;
|
||||||
|
for(std::size_t passageIndex=0;
|
||||||
|
passageIndex<PassageSegmentData.size();++passageIndex)
|
||||||
|
{
|
||||||
|
const auto& segments=PassageSegmentData[passageIndex];
|
||||||
|
if(segments.empty())continue;
|
||||||
|
const auto& constants=PassageComposerConstants[passageIndex];
|
||||||
|
const std::size_t stride=std::max<std::size_t>(1,
|
||||||
|
(segments.size()+127)/128);
|
||||||
|
for(std::size_t segmentIndex=0;segmentIndex<segments.size();
|
||||||
|
segmentIndex+=stride)
|
||||||
|
{
|
||||||
|
const auto& segment=segments[segmentIndex];
|
||||||
|
if(segment.lengthSq<=1.0e-12)continue;
|
||||||
|
constexpr double T=0.5;
|
||||||
|
const double centerX=segment.a[0]+T*segment.delta[0];
|
||||||
|
const double centerY=segment.a[1]+T*segment.delta[1];
|
||||||
|
const double centerZ=segment.a[2]+T*segment.delta[2];
|
||||||
|
const double radius=segment.radiusA+T*segment.radiusDelta;
|
||||||
|
const double floorDepth=PassageFlatFloorDepthSubvoxels(
|
||||||
|
constants,segmentIndex,T,radius,qPerMetre);
|
||||||
|
const double floorCenterZ=PassageFlatFloorCenterZSubvoxels(
|
||||||
|
constants,segments,segmentIndex,T,segment,centerX,centerY,
|
||||||
|
floorDepth,qPerMetre);
|
||||||
|
for(const double crestHeightMetres:CrestHeightsMetres)
|
||||||
|
{
|
||||||
|
const double sampleZ=floorCenterZ-floorDepth
|
||||||
|
+crestHeightMetres*qPerMetre;
|
||||||
|
const double verticalDelta=sampleZ-centerZ;
|
||||||
|
for(const double shellOffsetMetres:shellOffsetsMetres)
|
||||||
|
{
|
||||||
|
const double radialDistance=radius
|
||||||
|
+shellOffsetMetres*qPerMetre;
|
||||||
|
if(radialDistance*radialDistance
|
||||||
|
<=verticalDelta*verticalDelta)continue;
|
||||||
|
const double horizontalDistance=std::sqrt(
|
||||||
|
radialDistance*radialDistance
|
||||||
|
-verticalDelta*verticalDelta);
|
||||||
|
for(const auto& direction:RadialDirections)
|
||||||
|
{
|
||||||
|
const double sampleQ[3]={
|
||||||
|
centerX+direction[0]*horizontalDistance,
|
||||||
|
centerY+direction[1]*horizontalDistance,sampleZ};
|
||||||
|
const double sampleVoxel[3]={sampleQ[0]/Q,
|
||||||
|
sampleQ[1]/Q,sampleQ[2]/Q};
|
||||||
|
const float contribution=ApplyResolvedStratePassageCarves(
|
||||||
|
0.0f,static_cast<float>(sampleVoxel[0]),
|
||||||
|
static_cast<float>(sampleVoxel[1]),
|
||||||
|
static_cast<float>(sampleVoxel[2]));
|
||||||
|
++sampled;
|
||||||
|
if(std::abs(contribution)<=1.0e-6f)continue;
|
||||||
|
++signRelevant;
|
||||||
|
if(PassageProofNodes.empty()
|
||||||
|
||!PassageProofBoundsOverlapNode(0,sampleVoxel,
|
||||||
|
sampleVoxel))
|
||||||
|
++proofMisses;
|
||||||
|
std::array<std::int64_t,3> sampleMinimum{},sampleMaximum{};
|
||||||
|
for(std::size_t axis=0;axis<3;++axis)
|
||||||
|
{
|
||||||
|
sampleMinimum[axis]=static_cast<std::int64_t>(
|
||||||
|
std::floor(sampleQ[axis]));
|
||||||
|
sampleMaximum[axis]=static_cast<std::int64_t>(
|
||||||
|
std::ceil(sampleQ[axis]));
|
||||||
|
}
|
||||||
|
const auto candidates=BuildPassageFeatureTileList(
|
||||||
|
sampleMinimum,sampleMaximum);
|
||||||
|
const bool featureContainsSample=std::any_of(
|
||||||
|
candidates.begin(),candidates.end(),
|
||||||
|
[&](std::uint32_t featureIndex)
|
||||||
|
{
|
||||||
|
return featureIndex<PassageTileFeatures.size()
|
||||||
|
&&PassageTileFeatures[featureIndex].passageIndex
|
||||||
|
==passageIndex;
|
||||||
|
});
|
||||||
|
if(!featureContainsSample)++featureMisses;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const bool auditPassed=sampled>0&&signRelevant>0
|
||||||
|
&&proofMisses==0&&featureMisses==0;
|
||||||
|
std::printf("[L130PassageReachProofAudit] reach_m=%.3f sampled=%llu sign_relevant=%llu proof_misses=%llu feature_misses=%llu status=%s\n",
|
||||||
|
VF_PassageInfluenceReachMetres,
|
||||||
|
static_cast<unsigned long long>(sampled),
|
||||||
|
static_cast<unsigned long long>(signRelevant),
|
||||||
|
static_cast<unsigned long long>(proofMisses),
|
||||||
|
static_cast<unsigned long long>(featureMisses),
|
||||||
|
auditPassed?"PASS":"FAIL");
|
||||||
|
if(!auditPassed)std::abort();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
bool PassageProofBoundsOverlapNode(int nodeIndex,const double minimum[3],
|
bool PassageProofBoundsOverlapNode(int nodeIndex,const double minimum[3],
|
||||||
@@ -11442,10 +11518,9 @@ inline void FStandaloneShapeWorld::FScopedPassageTileQueries::prepareSweptTubeBa
|
|||||||
const FIntVector& origin,std::int32_t step,std::int32_t cells,std::uint32_t seed)
|
const FIntVector& origin,std::int32_t step,std::int32_t cells,std::uint32_t seed)
|
||||||
{
|
{
|
||||||
constexpr std::int64_t Q=VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel;
|
constexpr std::int64_t Q=VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel;
|
||||||
constexpr double InfluenceMetres=4.0;
|
|
||||||
constexpr double OpenAirSideWallHeightMetres=1.0;
|
constexpr double OpenAirSideWallHeightMetres=1.0;
|
||||||
if(!shape_||step<=0||cells<=0||!std::isfinite(passageCarveReachMetres_)
|
if(!shape_||step<=0||cells<=0||!std::isfinite(passageCarveReachMetres_)
|
||||||
||std::abs(passageCarveReachMetres_-InfluenceMetres)>1.0e-9)return;
|
||std::abs(passageCarveReachMetres_-VF_PassageInfluenceReachMetres)>1.0e-9)return;
|
||||||
const bool profile=profileStats_!=nullptr;
|
const bool profile=profileStats_!=nullptr;
|
||||||
const auto prepareStart=profile?std::chrono::steady_clock::now()
|
const auto prepareStart=profile?std::chrono::steady_clock::now()
|
||||||
:std::chrono::steady_clock::time_point{};
|
:std::chrono::steady_clock::time_point{};
|
||||||
@@ -11459,7 +11534,7 @@ inline void FStandaloneShapeWorld::FScopedPassageTileQueries::prepareSweptTubeBa
|
|||||||
sweep->dimension=cells+3;
|
sweep->dimension=cells+3;
|
||||||
const std::size_t gridSampleCount=static_cast<std::size_t>(sweep->dimension)
|
const std::size_t gridSampleCount=static_cast<std::size_t>(sweep->dimension)
|
||||||
*static_cast<std::size_t>(sweep->dimension)*static_cast<std::size_t>(sweep->dimension);
|
*static_cast<std::size_t>(sweep->dimension)*static_cast<std::size_t>(sweep->dimension);
|
||||||
sweep->influenceSubvoxels=InfluenceMetres*static_cast<double>(Q)*1000000.0
|
sweep->influenceSubvoxels=passageCarveReachMetres_*static_cast<double>(Q)*1000000.0
|
||||||
/std::max<std::uint32_t>(1,shape_->ResolvedLayout
|
/std::max<std::uint32_t>(1,shape_->ResolvedLayout
|
||||||
?shape_->ResolvedLayout->coreVoxelSizeMicrometres:250000);
|
?shape_->ResolvedLayout->coreVoxelSizeMicrometres:250000);
|
||||||
sweep->passageToSweep.assign(passageRouteSegments_.size(),-1);
|
sweep->passageToSweep.assign(passageRouteSegments_.size(),-1);
|
||||||
@@ -11473,7 +11548,7 @@ inline void FStandaloneShapeWorld::FScopedPassageTileQueries::prepareSweptTubeBa
|
|||||||
const double qPerMetre=static_cast<double>(Q)*1000000.0/
|
const double qPerMetre=static_cast<double>(Q)*1000000.0/
|
||||||
std::max<std::uint32_t>(1,shape_->ResolvedLayout
|
std::max<std::uint32_t>(1,shape_->ResolvedLayout
|
||||||
?shape_->ResolvedLayout->coreVoxelSizeMicrometres:250000);
|
?shape_->ResolvedLayout->coreVoxelSizeMicrometres:250000);
|
||||||
const double shellThickness=0.75*qPerMetre;
|
const double shellThickness=VF_PassageShellThicknessMetres*qPerMetre;
|
||||||
const double stampedOutsideDistance=sweep->influenceSubvoxels+1.0;
|
const double stampedOutsideDistance=sweep->influenceSubvoxels+1.0;
|
||||||
constexpr double MouthShellBlendMetres=5.0;
|
constexpr double MouthShellBlendMetres=5.0;
|
||||||
const double wallReliefShell=0.41*qPerMetre;
|
const double wallReliefShell=0.41*qPerMetre;
|
||||||
@@ -11633,7 +11708,7 @@ inline void FStandaloneShapeWorld::FScopedPassageTileQueries::prepareSweptTubeBa
|
|||||||
// more than cornerWidth/6. Skip its work for pairs outside this exact
|
// more than cornerWidth/6. Skip its work for pairs outside this exact
|
||||||
// support, with a small subvoxel guard for float rounding. The final
|
// support, with a small subvoxel guard for float rounding. The final
|
||||||
// shell union only consumes negative distances.
|
// shell union only consumes negative distances.
|
||||||
constexpr double CornerBlendMetres=0.50;
|
constexpr double CornerBlendMetres=VF_PassageCornerBlendMetres;
|
||||||
const float roundDistanceFloat=static_cast<float>(signedDistance);
|
const float roundDistanceFloat=static_cast<float>(signedDistance);
|
||||||
const float floorDistanceFloat=static_cast<float>(floorCenterZ-floorDepth-qz);
|
const float floorDistanceFloat=static_cast<float>(floorCenterZ-floorDepth-qz);
|
||||||
const double flatDistanceLowerBound=std::max(
|
const double flatDistanceLowerBound=std::max(
|
||||||
|
|||||||
@@ -0,0 +1,53 @@
|
|||||||
|
# VoxelForge — read-only design review: per-block bit flags (masks) to skip work
|
||||||
|
|
||||||
|
You are reviewing an idea against real code. **Do not edit, create or delete any file, and do not build or run
|
||||||
|
anything.** Read code and reason. Be thorough; mark each claim as [CODE] (with file:line), [MEASURED] (from a report),
|
||||||
|
or [INFERENCE].
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
VoxelForge Core (`Standalone/Core`) generates a density field: negative = solid, positive = air, 25 cm voxels, 32³ tiles,
|
||||||
|
LOD0–8 clipmap, marching cubes. The production mesher for the standalone path is in `Standalone/Source/...`, and Core is
|
||||||
|
also compiled into Unreal. Same seed must give the same world on every machine and worker count; byte-identity with
|
||||||
|
older versions is NOT required.
|
||||||
|
|
||||||
|
Branch `assembler/world` (HEAD `7022d02`'s parent line, currently `4041d4b`) has the smooth inter-strate tunnels.
|
||||||
|
Readiness target: LOD0–2 near terrain in ~1 s on arrival. Measured today: SW ~0.7–0.8 s, VS ~0.9 s, FI/TN ~1.0 s
|
||||||
|
headless. In Linux UE, start→LOD0–2 is ~2.2–2.5 s. At 30 m/s in UE, near coverage is ~40%.
|
||||||
|
|
||||||
|
Reports to consult: `Standalone/Research/WorldAssemblerM3/Linux1*/REPORT.md`, especially:
|
||||||
|
- L77/L87/L117, cost census;
|
||||||
|
- L124, shape gate and per-strate timings;
|
||||||
|
- L125/L126/L128, tunnel prepare: per-segment work, 67–73% pair overlap, support up to 7.9 m;
|
||||||
|
- L128, where an exact one-owner slab scheme halved pairs but got slower because of per-sample ownership tests.
|
||||||
|
|
||||||
|
Tile skipping already exists at tile level: box verdicts / classify, where all-solid or all-air tiles are skipped.
|
||||||
|
|
||||||
|
## The idea (from the owner)
|
||||||
|
|
||||||
|
Keep bit flags per block (1 bit per voxel, or per small brick of e.g. 4³ or 8³ voxels): for example "certainly deep
|
||||||
|
solid", "certainly deep air", "near a surface", "touched by a tunnel", "already evaluated". Use them to skip work in
|
||||||
|
generation and meshing, and possibly elsewhere (collision, streaming, edits, AI/nav, lighting).
|
||||||
|
|
||||||
|
## Questions
|
||||||
|
|
||||||
|
1. Where does time actually go per tile today, and which of that work a mask could skip? Cover density fill,
|
||||||
|
classification, mesher (marching cubes, normals/gradients, skirts, seam refinement), passage prepare, and composition.
|
||||||
|
Use measured census numbers where they exist.
|
||||||
|
2. Which flags are sound? A flag must be a proof, never a guess. A mask "certainly solid" must be derived from a
|
||||||
|
conservative bound (the bound must be the supremum of what the evaluator produces), or the mesher will delete
|
||||||
|
surfaces. Which bounds already exist in the code (box verdicts, interval bounds) that a brick-level mask could reuse?
|
||||||
|
Name the existing hazards. For example, `Identity` means `Sdf >= T`, and `ClassifyTile` proving below-TerrainZ solid
|
||||||
|
would delete 3D caves.
|
||||||
|
3. Granularity: per-voxel bits vs per-brick (4³/8³) vs per-column. What does each cost to build, versus what it
|
||||||
|
saves? Consider SIMD and cache behaviour. Marching cubes needs only cells whose corner signs differ, and gradients
|
||||||
|
need a halo.
|
||||||
|
4. "Already evaluated" or "checked" flags: where would they be correct (memoisation of identical queries), and where
|
||||||
|
would they be wrong (nearest-of-many queries like the tunnel segments)? Is there repeated identical evaluation today,
|
||||||
|
for example halo samples shared between neighbouring tiles, or skirt/seam re-queries?
|
||||||
|
5. Other uses: player edits (diff layer), collision-only server tiles, streaming priority, nav. Which would a mask
|
||||||
|
genuinely help, given the multiplayer rule (replicate seed + layout + diff, never geometry)?
|
||||||
|
6. Recommendation: the one or two mask designs most likely to pay off, with the expected gain (a range, marked
|
||||||
|
inference), the measurement that would prove or kill each, and the risks. Keep it short.
|
||||||
|
|
||||||
|
Write plain Markdown, at most ~1,200 words.
|
||||||
@@ -0,0 +1,83 @@
|
|||||||
|
[INFERENCE] **Improve the existing brick-proof system; don’t add a general “checked voxel” mask.** The promising savings are avoided field/composition work and passage preparation, rather than marching-cubes classification.
|
||||||
|
|
||||||
|
References below abbreviate these files: [M = production mesher](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp), [S = ShapeWorldInternal.h](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h), [C = VoxelForgeCore.cpp](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/VoxelForgeCore.cpp), [O = VoxelDensityOpStack.cpp](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/VoxelForge/Private/VoxelDensityOpStack.cpp). L-numbers identify `Standalone/Research/WorldAssemblerM3/LinuxN/REPORT.md`.
|
||||||
|
|
||||||
|
[INFERENCE] Evidence limitation: Linux128 is absent from this checkout. Its supplied 7.9 m support and slower one-owner result remain unverified here. The reports below span different revisions; their percentages must not be presented as today’s census.
|
||||||
|
|
||||||
|
**1. Where time goes**
|
||||||
|
|
||||||
|
[MEASURED] L77’s exclusive FI/TN LOD0–2 worker account:
|
||||||
|
|
||||||
|
| Work | FI | TN |
|
||||||
|
|---|---:|---:|
|
||||||
|
| Total worker CPU/view | 23.02 s | 21.50 s |
|
||||||
|
| Passage composition | 65.9% | 65.4% |
|
||||||
|
| Support | 14.2% | 13.5% |
|
||||||
|
| Base field | 3.2% | 3.3% |
|
||||||
|
| Bands / dispatch | 2.3% / 3.7% | 2.6% / 3.6% |
|
||||||
|
| MC / skirts / refinement, excluding field work | 1.6% / 0.1% / 7.5% | 1.2% / 0.2% / 8.6% |
|
||||||
|
|
||||||
|
[MEASURED] Grid fill occupied 19.058/17.377 worker-seconds, **overlapping** those phase categories. L117’s later census found support zero; its smooth-tunnel regression concentrated about 95% of added tile CPU in passage-feature tiles. Cell classification stayed flat; gradients became cheaper. Thus L77 identifies historical leverage, not current percentages.
|
||||||
|
|
||||||
|
[MEASURED] L125 preparation averaged 514/130/4.1 ms worker work at LOD0/1/2. LOD0 evaluated 3.11–8.49 million segment/sample pairs with 67–72% overlap, although the exact band occupied only 5–7% of grid samples. L126 reduced floor-profile calls 20–23% without useful readiness improvement.
|
||||||
|
|
||||||
|
[CODE] Brick skipping already exists: configurable block verdicts, required-sample masks, and active-block gradient halos (M:646,1053). MC already rejects cells without sign crossings before interpolation/gradients (M:2186). Gradient memoisation exists (M:1956). Skirts issue separate density queries (M:2491); refinement has its own stage (C:2975).
|
||||||
|
|
||||||
|
[INFERENCE] A pre-evaluation proof can save density, composition, and subsequent MC work. A sign mask constructed after density evaluation saves only downstream scanning. Classification itself costs work; no isolated current classification percentage is established by these reports.
|
||||||
|
|
||||||
|
[MEASURED] L124’s shape gate alone took 179–226 ms, with no near-tile admissions before completion. L87 additionally measured a 107–153 ms far-parent gate and no headless collision wait. Tile masks cannot directly remove these gates or explain the supplied UE/headless gap.
|
||||||
|
|
||||||
|
**2. Sound flags and hazards**
|
||||||
|
|
||||||
|
[INFERENCE] For final density \(D\), solid requires an upper bound covering the evaluator’s supremum, \(U<0\); air requires a valid lower bound \(L\ge0\). “Deep” requires an explicit margin. Unknown must stay unknown. A crossing proves an MC-active cell; uniform sampled signs do not prove an entire continuous volume surface-free.
|
||||||
|
|
||||||
|
[CODE] Reusable proofs include:
|
||||||
|
|
||||||
|
- S:5428: halo-expanded box classification.
|
||||||
|
- S:5974: exact surface-column extrema over the MC lattice.
|
||||||
|
- S:6019: floating-island density intervals.
|
||||||
|
- S:5197,6032: native cave intervals.
|
||||||
|
- S:5949: stamp interval composition with outward numerical padding.
|
||||||
|
- O:6998: operation-specific inactivity from SDF lower bounds.
|
||||||
|
|
||||||
|
[CODE] These proofs have guards: passage overlap returns unknown (S:5687); unsupported stamp composition returns unknown (S:5655); stratum crossings are rejected (S:5900). The mesher distinguishes uniform candidates from `bSkipSafe` certificates (M:715,756), and otherwise validates candidates against actual samples (M:1838).
|
||||||
|
|
||||||
|
[CODE] For gated modifiers, `Identity` can mean `Sdf >= T`, not `abs(Sdf) >= T` (O:1496,6998). Noise bounds cannot assume nominal `[-1,1]`: FBM and unclamped cellular bounds are explicitly discussed at O:848,301.
|
||||||
|
|
||||||
|
[INFERENCE] Additional hazards: confusing internal SDF with final density; certifying base terrain before caves, tunnels, shells, stamps or edits; using lattice proofs for off-grid queries; ignoring rounding or gradient halos. A below-`TerrainZ` solid rule applied to 3D caves deletes caves. “Touched by tunnel” should mean conservative candidate overlap; only **proven untouched** authorizes omission.
|
||||||
|
|
||||||
|
**3. Granularity**
|
||||||
|
|
||||||
|
[INFERENCE] Storage per flag for 32³ cells:
|
||||||
|
|
||||||
|
| Granularity | Storage | Construction/tradeoff |
|
||||||
|
|---|---:|---|
|
||||||
|
| Per cell | 4 KiB | 32,768 decisions; expensive unless derived during necessary work |
|
||||||
|
| 4³ bricks | 64 B | 512 proofs; tighter boundaries, more proof/halo overhead |
|
||||||
|
| 8³ bricks | 8 B | 64 proofs; cheaper, more mixed bricks |
|
||||||
|
| XY columns | 128 B | 1,024 decisions; useful mainly for heightfield-specific proofs |
|
||||||
|
|
||||||
|
[INFERENCE] These exclude corner/halo samples, metadata and additional flags. A column bit cannot represent multiple cave intervals. Brick runs preserve contiguous X sampling and SIMD better than scattered per-sample branches; bit packing alone does not establish a speedup.
|
||||||
|
|
||||||
|
[CODE] The existing implementation uses byte arrays and preserves exact samples around active bricks (M:1053–1083). Replacing densities with arbitrary same-sign constants near active cells would corrupt interpolated positions or normals.
|
||||||
|
|
||||||
|
**4. “Already evaluated”**
|
||||||
|
|
||||||
|
[CODE] Passage preparation precedes meshing (C:2747). Its valid sentinel is initialization, not completion: subsequent segments update floor selection, shell minima and round-distance winners, with deterministic tie-breaking (S:11520,11615,11688).
|
||||||
|
|
||||||
|
[INFERENCE] Therefore first-visitor wins is invalid. Memoisation is valid only for identical queries against identical seed/layout/evaluator/edit revision and query semantics; cache the value, not just a bit.
|
||||||
|
|
||||||
|
[MEASURED] L77 counted 8.6–8.9% halo points, without measuring duplicate coordinates. L125 verified 4,900 shared samples across four neighbour pairs and still recorded substantial off-grid skirt/midpoint fallbacks.
|
||||||
|
|
||||||
|
[CODE] Shared-grid reuse and gradient validity caches already exist (M:1031,1956). [INFERENCE] Skirt/seam queries are reuse candidates only when coordinates and semantics actually match; “same voxel” is insufficient.
|
||||||
|
|
||||||
|
**5. Other uses**
|
||||||
|
|
||||||
|
[INFERENCE] Edit-dirty bricks can localize regeneration, with invalidation expanded through normals, seams and collision dependencies. Collision-only servers can skip certified empty regions, but still need boundary geometry. Streaming can prioritize potentially occupied bricks; nav and lighting need clearance/connectivity or visibility data beyond occupancy. Keep masks derived locally from replicated seed + layout + diff, versioned with that state.
|
||||||
|
|
||||||
|
**6. Recommendation**
|
||||||
|
|
||||||
|
- [INFERENCE] **Tune existing 4³/8³ certificates**, caching proofs and emitting contiguous required-sample runs. Plausible incremental near-ready saving: **0–5%**, with regressions possible. Measure proof time, samples avoided, halo expansion, total worker CPU and UE readiness.
|
||||||
|
- [INFERENCE] **Pass required-region information into passage preparation**, rejecting whole segment/brick regions conservatively. Plausible near-ready saving: **0–10%**, highly uncertain because passage overlap currently vetoes proofs. Kill it if pair reduction fails to reduce preparation and end-to-end time.
|
||||||
|
|
||||||
|
[INFERENCE] Both require seam/off-grid validation, adversarial bound checks, edit invalidation tests, and cross-machine/worker determinism. Preserve canonical tie-breaking; old-version byte identity need not constrain a sound redesign.
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
# VoxelForge — open read-only review: what should be done and focused on now?
|
||||||
|
|
||||||
|
You are an outside reviewer. **Do not edit, create or delete any file, and do not build or run anything.** Read the
|
||||||
|
repository and its reports and reason. Be thorough; mark each claim as [CODE] (with file:line), [MEASURED] (from a
|
||||||
|
report), or [INFERENCE].
|
||||||
|
|
||||||
|
## The project, in the owner's words and goals
|
||||||
|
|
||||||
|
- VoxelForge is a UE5 density-field voxel plugin for a **multiplayer** game. The world is made of stacked "strates"
|
||||||
|
(layers) linked by tunnels. The standalone C++ generator is "Core" (`Standalone/Core`), and it is also compiled into
|
||||||
|
Unreal. Read `CLAUDE.md`, `CODEMAP.md`, `ARCHITECTURE.md`, `GDD.md`, `COMPOSER-NOTES.md` and `fable-idea.md` at the
|
||||||
|
plugin root for the game and design intent. The research reports are in
|
||||||
|
`Standalone/Research/WorldAssemblerM3/Linux*/REPORT.md` and the `Astra*/` reviews.
|
||||||
|
- Owner goals stated over time:
|
||||||
|
- the near terrain (LOD0–2) ready in ~1 s on arrival, then far terrain keeps filling in at low priority;
|
||||||
|
- same seed gives the same world on every machine (byte-identity with older versions is NOT required);
|
||||||
|
- smooth for VR and high-speed travel;
|
||||||
|
- all generators (surface, floating islands, caves, dreamcore/liminal, etc.) working in Core;
|
||||||
|
- the system should eventually *invent* strates and worlds from tagged authored content plus a seed;
|
||||||
|
- keep things simple and avoid over-complication.
|
||||||
|
- The current main line is branch `assembler/world`.
|
||||||
|
|
||||||
|
## Your task
|
||||||
|
|
||||||
|
Look at the whole: code, reports, design docs, and the direction of the last few dozen rounds. Tell the owner what
|
||||||
|
should be done and focused on now, whatever it is. That may be performance, features, generation quality, gameplay
|
||||||
|
foundations, architecture, technical debt, tooling, process, or stopping something. Don't assume the recent focus is
|
||||||
|
the right one; question it if the evidence says so.
|
||||||
|
|
||||||
|
Answer:
|
||||||
|
1. **Where the project actually is**: what works, what's fragile, what's missing for the game to exist.
|
||||||
|
2. **The top 3–5 things to focus on now**, ranked, each with why, a rough size, what "done" looks like, and how to
|
||||||
|
tell it worked.
|
||||||
|
3. **What to stop or deprioritise**, and why.
|
||||||
|
4. **Risks or debts** that will bite later if ignored.
|
||||||
|
|
||||||
|
Write plain Markdown, at most ~1,500 words, readable by a non-specialist owner (explain jargon briefly).
|
||||||
@@ -0,0 +1,72 @@
|
|||||||
|
[INFERENCE] **Focus now on a small, persistent multiplayer descent, supported by end-to-end Unreal performance work.** VoxelForge has substantial terrain technology. The larger uncertainty is whether it delivers a reliable, interesting shared expedition. Another sequence of tunnel optimizations will not resolve that.
|
||||||
|
|
||||||
|
Read-only review of `assembler/world` at `044cd6d`; no files changed and no builds, tests, or project execution. [CODE] identifies repository evidence; design-document references establish intent, not implementation. [MEASURED] means reported results, not independently reproduced measurements.
|
||||||
|
|
||||||
|
**1. Where the project actually is**
|
||||||
|
|
||||||
|
[CODE] **Core is real and integrated, but generator implementation remains mixed.** Unreal compiles the canonical standalone sources through validated wrappers. Core dispatches to surface, floating-island and native graph evaluators, with a legacy-generator fallback when appropriate. That is useful coverage, but does not mean every family has a complete native implementation. See [VoxelForgeCore.Build.cs:15](/home/jahni/VoxelM/Plugins/VoxelForge/Source/VoxelForgeCore/VoxelForgeCore.Build.cs:15) and [ShapeWorldInternal.h:600](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:600).
|
||||||
|
|
||||||
|
[MEASURED] **Passage correctness and speed have improved substantially.** L109 simplified tunnels and removed support-restoration machinery, reducing floating-island and tunnel-network near-generation times by approximately 31–38%. L125 reports 21,425/21,425 open route samples, a maximum sampled floor offset of 7.884 mm, and matching cross-compiler/worker hashes. These are meaningful engineering achievements. [L109](Standalone/Research/WorldAssemblerM3/Linux109/REPORT.md), [L125](Standalone/Research/WorldAssemblerM3/Linux125/REPORT.md).
|
||||||
|
|
||||||
|
[MEASURED] **The one-second player experience is not achieved.** L125’s standalone near-ready medians were approximately 0.78–1.03 seconds. L127’s Unreal startup took 2.22–2.51 seconds; at 30 m/s, median near coverage fell from 43.36% to 40.22% after the smooth-tunnel merge. Both versions recovered after stopping. L130 found no measurable timing improvement and skipped Unreal movement. These are different workloads; subtracting their times would not isolate a particular overhead. [L125](Standalone/Research/WorldAssemblerM3/Linux125/REPORT.md), [L127](Standalone/Research/WorldAssemblerM3/Linux127/REPORT.md), [L130](Standalone/Research/WorldAssemblerM3/Linux130/REPORT.md).
|
||||||
|
|
||||||
|
[CODE] **Multiplayer foundations exist; the multiplayer lifecycle remains unfinished here.** There are season hashes, local terrain-edit entry points, and streaming-anchor infrastructure. The architecture explicitly leaves join replication, authoritative edit messages, late-join snapshots and networked season changes open. The edit entry point directly dispatches to local modification paths. [VoxelWorld.h:574](/home/jahni/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelWorld.h:574), [VoxelWorld.cpp:15771](/home/jahni/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:15771), [ARCHITECTURE.md:2408](/home/jahni/VoxelM/Plugins/VoxelForge/ARCHITECTURE.md:2408).
|
||||||
|
|
||||||
|
[CODE] **The composer is more than a proposal, but less than the full content vision.** It rolls candidates, measures them and rejects failures. The intended system additionally combines tagged geometry, materials, decorations, creatures and audio; theme coherence remains an open design item. [VoxelSeasonManifest.cpp:2523](/home/jahni/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelSeasonManifest.cpp:2523), [COMPOSER-NOTES.md:1512](/home/jahni/VoxelM/Plugins/VoxelForge/COMPOSER-NOTES.md:1512).
|
||||||
|
|
||||||
|
[INFERENCE] **This repository demonstrates a capable terrain platform, not yet a validated game loop.** I cannot establish what gameplay exists outside the plugin. The GDD itself identifies “Is the descent fun?” as its first unresolved question. [GDD.md:441](/home/jahni/VoxelM/Plugins/VoxelForge/GDD.md:441).
|
||||||
|
|
||||||
|
**2. Ranked priorities**
|
||||||
|
|
||||||
|
All sizes below are [INFERENCE]: rough focused engineering estimates, excluding substantial art production.
|
||||||
|
|
||||||
|
**1 — Deliver one multiplayer expedition that survives reconnecting. Size: 3–6 weeks.**
|
||||||
|
|
||||||
|
[INFERENCE] Use two contrasting strates and their connecting tunnel. Two players descend, find one destination, uncover or collect something, return, and retain their changes after restart. Include a third player joining after terrain has changed. Avoid expanding immediately into the city economy or community backend.
|
||||||
|
|
||||||
|
[INFERENCE] The necessary foundation is a server-approved world identity, ordered authoritative edits, a saved snapshot plus subsequent edits, and collision around separated players. Give edits stable identifiers so retransmission cannot apply them twice. Reuse the existing local edit and anchor systems.
|
||||||
|
|
||||||
|
[INFERENCE] **Done:** host/client edits agree; a late joiner reconstructs the same terrain; restart preserves it; incompatible world versions fail clearly; remote players have ground when away from the host.
|
||||||
|
|
||||||
|
[INFERENCE] **Success:** a complete shared expedition without terrain disagreement, lost progress or collision failures—and players can explain what made them want to descend again. This tests the project’s central assumptions much earlier than a complete RPG would.
|
||||||
|
|
||||||
|
**2 — Close the Unreal arrival and movement gap. Size: 1–3 weeks initially.**
|
||||||
|
|
||||||
|
[MEASURED] L122 found idle generation capacity while admission was blocked; L123 measured approximately 19 ms from region request to game-thread observation despite sub-millisecond median compilation. Simple lookahead and admission relaxations failed their combined coverage/frame-time gates. L121’s modest extra far-apply lane did improve far coverage. [L121](Standalone/Research/WorldAssemblerM3/Linux121/REPORT.md), [L122](Standalone/Research/WorldAssemblerM3/Linux122/REPORT.md), [L123](Standalone/Research/WorldAssemblerM3/Linux123/REPORT.md).
|
||||||
|
|
||||||
|
[INFERENCE] First establish one current-build, rendered Unreal baseline on intended hardware. Measure arrival through **visible terrain and usable collision**, then walking, 30 m/s travel, turns, teleporting and edits. Record near coverage, far gaps, frame-time spikes and memory separately. NullRHI measurements, which omit rendering, cannot certify VR comfort.
|
||||||
|
|
||||||
|
[INFERENCE] Investigate region completion, admission and publication before another broad density rewrite. A game-thread continuation is a candidate experiment, not a proven solution; keep world mutation on its owning thread.
|
||||||
|
|
||||||
|
[INFERENCE] **Done:** approximately one-second LOD0–2 readiness on the agreed hardware, safe collision, steadily filling far terrain, and an explicit movement-coverage target without worsening frame spikes. **Success:** repeated rendered runs meet those limits, including with remote-player collision work. Do not redefine “ready” as only the nearest handful of tiles.
|
||||||
|
|
||||||
|
**3 — Turn generator breadth into playable variety. Size: 2–4 weeks for the first content slice.**
|
||||||
|
|
||||||
|
[INFERENCE] Establish a compact capability matrix for each family: Core dispatch, entrances/exits, interior connectivity, edits, materials/content, far terrain and performance. Include dreamcore/liminal explicitly; motif generation alone does not establish a convincing anomalous place.
|
||||||
|
|
||||||
|
[MEASURED] L58 illustrates the distinction: its lattice route passed reachability checks, but the review described a corkscrew tunnel rather than the intended open vertical shaft. Technical validity and identity are separate acceptance criteria. [L58](Standalone/Research/WorldAssemblerM3/Linux58/REPORT.md).
|
||||||
|
|
||||||
|
[INFERENCE] Extend the existing offline composer with a small tagged library: a grounded cave, an island environment and one liminal identity, each with coherent materials, sound and a destination. Preserve owner review before publishing a season.
|
||||||
|
|
||||||
|
[INFERENCE] **Done:** several seeds produce recognizably different, coherent places with findable progression and usable encounter spaces. **Success:** players navigate them without developer guidance and distinguish their identities without reading labels. Expand families after this works.
|
||||||
|
|
||||||
|
**4 — Consolidate the evidence and remove obsolete complexity incrementally. Size: 2–4 days, then ongoing.**
|
||||||
|
|
||||||
|
[INFERENCE] Maintain one short current-state page: shipped behavior, open defects, active baseline, hardware and next acceptance test. Keep historical reports as history. Remove rejected experimental paths when they no longer serve a specific comparison; extract stable responsibilities when touching oversized files.
|
||||||
|
|
||||||
|
[INFERENCE] **Done:** a contributor can identify the current evaluator, runtime path, multiplayer gaps and performance limits without reconstructing 130 rounds. **Success:** new work no longer repeats rejected experiments or treats old measurements as current.
|
||||||
|
|
||||||
|
**3. Stop or deprioritise**
|
||||||
|
|
||||||
|
- [INFERENCE] **Pause open-ended tunnel micro-optimization.** L125/L126 missed the older speed bar; L130’s reach correction produced no measurable speedup. Keep correctness fixes, but require a current cost account and a bounded experiment before further optimization. [MEASURED: L126](Standalone/Research/WorldAssemblerM3/Linux126/REPORT.md), [L130](Standalone/Research/WorldAssemblerM3/Linux130/REPORT.md).
|
||||||
|
- [INFERENCE] **Do not implement Astra recommendations mechanically.** Astra1’s support-heavy model predates L109’s simplification. Astra2/3 offer hypotheses, not measured wins for today’s code. Re-profile before adopting them.
|
||||||
|
- [INFERENCE] **Defer GPU generation, a mesher replacement, elaborate discovery systems and large-scale backend work.** None currently resolves the missing shared expedition.
|
||||||
|
- [INFERENCE] **Stop requiring old-version byte identity for deliberate generation changes.** Preserve peer agreement within a version and explicit versioned saves; permit reviewed geometry improvements.
|
||||||
|
|
||||||
|
**4. Debts that will bite later**
|
||||||
|
|
||||||
|
[INFERENCE] **Determinism needs a deployment contract.** Linux Clang/GCC and worker-count agreement is strong evidence, not proof across every intended machine. Include Windows/compiler/CPU combinations actually supported. World identity must cover evaluator, content, layout, scale and edit state—not merely seed. [MEASURED: L130](Standalone/Research/WorldAssemblerM3/Linux130/REPORT.md).
|
||||||
|
|
||||||
|
[CODE] **Scale and fallback complexity remain active integration risks.** Core is opt-in, and runtime queries bridge different coordinate conventions. [VoxelWorld.h:300](/home/jahni/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelWorld.h:300), [VoxelWorld.cpp:15612](/home/jahni/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:15612). [INFERENCE] Test edits, content placement and collision across those boundaries before broadening authoring.
|
||||||
|
|
||||||
|
[INFERENCE] **A passing route audit does not prove a whole strate is enjoyable, connected or suitable for six-player combat.** Retain geometric checks, but add actual traversal, destination discovery and encounter-space evaluation. Those are now more valuable than another polished tunnel benchmark.
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
# Linux 120 — arrival and far streaming
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Keep the current scheduler defaults. Neither tested design meets the full bar. No production C++ change or version bump is retained.
|
||||||
|
|
||||||
|
## Findings
|
||||||
|
|
||||||
|
1. **Far holes can coexist with idle generation workers.** In five UE 30 m/s runs, 575 of 2,153 scheduler samples with missing far tiles had zero generation workers running. In 497 of those 575 samples, completed results were queued. During movement, far tiles waited a median 1.12 s and p95 3.21 s from worker completion to apply; their generation time was p95 10 ms. The main wait is the game-thread result queue and scheduler admission, not expensive far-tile generation. The separate far-parent build also leaves the tile pool idle for 33–34 ms; see the L105 timeline.
|
||||||
|
|
||||||
|
2. **Background priority is not tile preemption.** VR mode assigns both near and horizon tile tasks `BackgroundLow`; UE can defer queued background tasks, but a running tile finishes or is cancelled before another task takes its worker. The tested split (near `BackgroundNormal`, horizon `BackgroundLow`) changed paired near-ready by only +0.01 points and increased far holes by 0.40 points. The scheduler already reserves an 80/20 near/far admission share during fast travel.
|
||||||
|
|
||||||
|
Raising the existing worker cap from 8 to 10 effective tile slots improved paired UE moving near-ready by 1.10 points and reduced far holes by 0.48 points. Headless gains were +0.35 points and −0.41 points. This increases admitted work, not UE background threads (eight were available). The UE max tick reached 16.67 ms in one 10-slot run versus 15.43 ms in the 8-slot set, so the strict worst-tick gate failed.
|
||||||
|
|
||||||
|
3. **LOD2 still depends on far parents.** `LoadCoreTile` rejects LOD2 while required far-world data is not ready. That data contributes to the density field LOD2 meshes sample. Removing the gate without proving a tile’s dependency set would change geometry. The far-world build is deferred until LOD0–1 are ready, so this dependency is on the arrival path.
|
||||||
|
|
||||||
|
4. **At 30 m/s, generation and result throughput limit near readiness.** With the production ready-queue depth, LOD0–2 settled about 1.31 s after first admission. The 10-slot arrival pair improved near settlement by only 19 ms paired-median; whole-view/far-clear time changed by only 18 ms, within run variation, and both arms had zero far holes by 20 s. The larger depth-8 arrival gain does not represent the production default. At depth 8, the 10-slot movement trial did raise near-ready, but its worst UE tick failed the bar. Keep the cap unchanged.
|
||||||
|
|
||||||
|
## Checks
|
||||||
|
|
||||||
|
- Five interleaved 30 m/s movement pairs per platform and five interleaved FI arrival pairs at default ready depth; quiet load stayed below 2.2 on 20 logical CPUs. UE used `taskset` only; headless used `taskset` and the 6 GiB address-space limit. Each commandlet finished well under 30 minutes.
|
||||||
|
- Fast tier: **PASS**, 274 checks / 80 runs, 267.1 s.
|
||||||
|
- Exact hashes: **24/24 PASS** for SW/FI/TN/VS × Clang/GCC × W1/W8/W12; tile and whole-view hashes match the L104 baseline and Clang W1. Source stamp `f3ec37a…`; no version bump is needed.
|
||||||
|
- Unreal build passed with `-NoUBA -MaxParallelActions=4`. `git diff --check` is clean.
|
||||||
|
|
||||||
|
Detailed summaries and logs are under `Saved/Linux/l120/`; the movement and arrival comparisons are in `movement/worker-slots-verified/summary.json`, `movement/priority-split-verified/summary.json`, and `arrival/pairs-rd64.json`.
|
||||||
@@ -0,0 +1,487 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Run one pinned null-RHI FI arrival for Linux 120."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from collections import Counter
|
||||||
|
from datetime import datetime
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
HOST = Path("/home/jahni/VFHost")
|
||||||
|
HOST_PLUGIN_SAVED = HOST / "Plugins/VoxelForge/Saved/Linux/l120"
|
||||||
|
HOST_SAVED = HOST / "Saved/Linux/l120"
|
||||||
|
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
|
||||||
|
OUT = ROOT / "Saved/Linux/l120/arrival"
|
||||||
|
SPAWNS_M = {
|
||||||
|
"FI": (25.8369140625, 3.19921875, -737.0),
|
||||||
|
"SW": (25.8369140625, 3.19921875, -200.0),
|
||||||
|
}
|
||||||
|
MOVE_START_S = 20.0
|
||||||
|
MOVE_DURATION_S = 12.0
|
||||||
|
EXIT_S = MOVE_START_S + MOVE_DURATION_S + 5.0
|
||||||
|
RUN_TIMEOUT_S = 1740
|
||||||
|
|
||||||
|
|
||||||
|
def read_events(path: Path) -> list[dict]:
|
||||||
|
events = []
|
||||||
|
if path.is_file():
|
||||||
|
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||||
|
if line.startswith("{"):
|
||||||
|
try:
|
||||||
|
events.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
return events
|
||||||
|
|
||||||
|
|
||||||
|
def percentile(values: list[float], fraction: float) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
ordered = sorted(values)
|
||||||
|
return ordered[min(len(ordered) - 1, int(round((len(ordered) - 1) * fraction)))]
|
||||||
|
|
||||||
|
|
||||||
|
def sample_unreal_peak_rss() -> tuple[int, int] | None:
|
||||||
|
executable = str(EDITOR).encode()
|
||||||
|
project = str(HOST / "VFHost.uproject").encode()
|
||||||
|
peak: tuple[int, int] | None = None
|
||||||
|
for process in Path("/proc").glob("[0-9]*"):
|
||||||
|
try:
|
||||||
|
command_line = (process / "cmdline").read_bytes()
|
||||||
|
if executable not in command_line or project not in command_line:
|
||||||
|
continue
|
||||||
|
status = (process / "status").read_text(encoding="ascii", errors="ignore")
|
||||||
|
match = re.search(r"^VmHWM:\s+(\d+)\s+kB$", status, re.MULTILINE)
|
||||||
|
if match:
|
||||||
|
rss_kib = int(match.group(1))
|
||||||
|
if peak is None or rss_kib > peak[0]:
|
||||||
|
peak = (rss_kib, int(process.name))
|
||||||
|
except (FileNotFoundError, PermissionError, ProcessLookupError, OSError):
|
||||||
|
continue
|
||||||
|
return peak
|
||||||
|
|
||||||
|
|
||||||
|
def endphysics_gap(stdout: Path) -> tuple[float | None, int | None]:
|
||||||
|
release_times: dict[int, dict[int, datetime]] = {}
|
||||||
|
pattern = re.compile(
|
||||||
|
r"^\[(\d{4}\.\d{2}\.\d{2}-\d{2}\.\d{2}\.\d{2}:\d{3})\].*"
|
||||||
|
r"LogTick: tick\s+(\d+)\s+-+\s+Release tick group ([34])")
|
||||||
|
with stdout.open(encoding="utf-8", errors="replace") as stream:
|
||||||
|
for line in stream:
|
||||||
|
match = pattern.match(line)
|
||||||
|
if not match:
|
||||||
|
continue
|
||||||
|
timestamp = datetime.strptime(match.group(1), "%Y.%m.%d-%H.%M.%S:%f")
|
||||||
|
frame = int(match.group(2))
|
||||||
|
group = int(match.group(3))
|
||||||
|
release_times.setdefault(frame, {})[group] = timestamp
|
||||||
|
gaps = [(round((times[4] - times[3]).total_seconds() * 1000.0, 3), frame)
|
||||||
|
for frame, times in release_times.items() if 3 in times and 4 in times]
|
||||||
|
return max(gaps, default=(None, None))
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, repeat: int, phase: str, ready_depth: int,
|
||||||
|
background_cap: int, worker_reserve: int, helper_background: int,
|
||||||
|
tile_background_low: int, tile_background_normal: int, startup_boost: int,
|
||||||
|
tick_logs: bool, insights: bool) -> dict:
|
||||||
|
label = (f"{phase}-{stratum.lower()}-rd{ready_depth}-bg{background_cap}"
|
||||||
|
f"-reserve{worker_reserve}-helperbg{helper_background}"
|
||||||
|
f"-tilelow{tile_background_low}-tilenormal{tile_background_normal}"
|
||||||
|
f"-ticklog{int(tick_logs)}"
|
||||||
|
f"-boost{startup_boost}-{repeat}")
|
||||||
|
trace = OUT / f"{label}.trace.jsonl"
|
||||||
|
utrace = OUT / f"{label}.utrace"
|
||||||
|
abslog = OUT / f"{label}.log"
|
||||||
|
stdout = OUT / f"{label}.stdout.log"
|
||||||
|
output = HOST_PLUGIN_SAVED / "output" / label
|
||||||
|
userdir = HOST_SAVED / "user" / label
|
||||||
|
zendata = HOST_SAVED / "zen" / label
|
||||||
|
ddc = HOST_SAVED / "DDC"
|
||||||
|
for directory in (OUT, output, userdir, zendata, ddc):
|
||||||
|
directory.mkdir(parents=True, exist_ok=True)
|
||||||
|
trace.unlink(missing_ok=True)
|
||||||
|
utrace.unlink(missing_ok=True)
|
||||||
|
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWNS_M[stratum])
|
||||||
|
command = [
|
||||||
|
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
|
||||||
|
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
|
||||||
|
"-notraceserver", "-nocrashreports",
|
||||||
|
"-DisablePlugins=AndroidFileServer",
|
||||||
|
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
|
||||||
|
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
|
||||||
|
"-archetype=TunnelNetwork", f"-slot={1 if stratum == 'FI' else 0}",
|
||||||
|
"-modes=export", "-gameconfig=1", "-coreworldtest=1",
|
||||||
|
"-ownerfourstrateplan=1", "-corevoxelsizecm=25", "-coreislandtest=0",
|
||||||
|
"-corefallbacktest=0", "-coreofftest=0", "-coreplantest=0",
|
||||||
|
"-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings", "-corehybridmesher=0",
|
||||||
|
f"-out={output}", "-voxel.StreamingCoarseFirst=1",
|
||||||
|
"-voxel.StreamingDynamicNearReserve=1",
|
||||||
|
"-voxel.StreamingLegacyPendingResultCap=0",
|
||||||
|
"-voxel.CoreRegionConcurrentAdmission=1",
|
||||||
|
"-voxel.StreamingResultBacklogCap=256",
|
||||||
|
f"-voxel.StreamingReadyQueueDepth={ready_depth}",
|
||||||
|
f"-voxel.StreamingBackgroundWorkerCap={background_cap}",
|
||||||
|
f"-voxel.StreamingBackgroundWorkerReserve={worker_reserve}",
|
||||||
|
f"-voxel.CoreParallelForBackgroundPriority={helper_background}",
|
||||||
|
f"-voxel.StreamingTileBackgroundLow={tile_background_low}",
|
||||||
|
f"-voxel.StreamingTileBackgroundNormal={tile_background_normal}",
|
||||||
|
"-voxel.StreamingTilePrioritySplit=0",
|
||||||
|
f"-voxel.TestLogTickFunctions={int(tick_logs)}",
|
||||||
|
"-voxel.CoreColumnCacheDirectMapped=0", "-voxel.CoreMCGradientMemoization=1",
|
||||||
|
"-voxel.StreamingCollisionLookahead=1",
|
||||||
|
f"-voxel.StreamingStartupApplyBoost={startup_boost}",
|
||||||
|
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=140.0",
|
||||||
|
f"-voxel.TestMoveStartSeconds={MOVE_START_S:.1f}",
|
||||||
|
f"-voxel.TestMoveDurationSeconds={MOVE_DURATION_S:.1f}",
|
||||||
|
f"-voxel.TestExitSeconds={EXIT_S:.1f}", "-voxel.TestMoveDirection=1,0,0",
|
||||||
|
"-voxel.StreamingSchedulerPolicy=3", "-voxel.TestExitOnSteady=0",
|
||||||
|
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
|
||||||
|
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
|
||||||
|
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
|
||||||
|
f"-abslog={abslog}",
|
||||||
|
]
|
||||||
|
if insights:
|
||||||
|
command += ["-trace=cpu,task,frame", "-statnamedevents",
|
||||||
|
f"-tracefile={utrace}", "-tracefiletrunc"]
|
||||||
|
wrapped = ["taskset", "-c", "0-11", "timeout", f"{RUN_TIMEOUT_S}s", *command]
|
||||||
|
started = time.monotonic()
|
||||||
|
with stdout.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.Popen(wrapped, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT)
|
||||||
|
peak_rss = None
|
||||||
|
while process.poll() is None:
|
||||||
|
observed = sample_unreal_peak_rss()
|
||||||
|
if observed and (peak_rss is None or observed[0] > peak_rss[0]):
|
||||||
|
peak_rss = observed
|
||||||
|
time.sleep(0.25)
|
||||||
|
exit_code = process.wait()
|
||||||
|
observed = sample_unreal_peak_rss()
|
||||||
|
if observed and (peak_rss is None or observed[0] > peak_rss[0]):
|
||||||
|
peak_rss = observed
|
||||||
|
|
||||||
|
events = read_events(trace)
|
||||||
|
worker_backend_config = next(
|
||||||
|
(e for e in events if e.get("type") == "worker_backend_config"), {})
|
||||||
|
worker_cap_readback = next(
|
||||||
|
(e for e in events if e.get("type") == "streaming_worker_cap_readback"), {})
|
||||||
|
samples = [e for e in events if e.get("type") == "streaming_sample"]
|
||||||
|
arrival = [e for e in samples if float(e.get("t_s", 0.0)) < MOVE_START_S]
|
||||||
|
full = next((e for e in arrival
|
||||||
|
if sum(e.get("desired_by_lod", [0] * 9)[:3]) > 0
|
||||||
|
and sum(e.get("settled_by_lod", [0] * 9)[:3])
|
||||||
|
>= sum(e.get("desired_by_lod", [0] * 9)[:3])), None)
|
||||||
|
first_admission = next((e for e in events if e.get("type") == "scheduler_admission"
|
||||||
|
and e.get("desired_rebuilt") == 1), None)
|
||||||
|
request_events = [e for e in events if e.get("type") == "request"]
|
||||||
|
initial_request_events = [e for e in request_events if e.get("epoch") == 0]
|
||||||
|
tile_by_ordinal = {e.get("request_ordinal"): e for e in events
|
||||||
|
if e.get("type") == "tile"}
|
||||||
|
collision_count = sum(1 for e in events if e.get("type") == "collision_ready"
|
||||||
|
and e.get("level", 9) <= 2
|
||||||
|
and full and e.get("t_s", 1e30) <= full["t_s"])
|
||||||
|
initial_tiles = [tile_by_ordinal[r.get("request_ordinal")] for r in initial_request_events
|
||||||
|
if full and r.get("t_s", 1e30) <= full["t_s"]
|
||||||
|
and r.get("request_ordinal") in tile_by_ordinal
|
||||||
|
and tile_by_ordinal[r["request_ordinal"]].get("level", 9) <= 2]
|
||||||
|
burst_start_s = float(first_admission["t_s"]) if first_admission else -1.0
|
||||||
|
burst_end_s = float(full["t_s"]) if full else -1.0
|
||||||
|
work_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "game_thread_work_frame"}
|
||||||
|
work_values = [e for e in work_by_frame.values()
|
||||||
|
if burst_start_s <= float(e.get("t_s", -1.0)) <= burst_end_s]
|
||||||
|
arrival_frame_ids = {int(e.get("frame", -1)) for e in work_values}
|
||||||
|
scheduler_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "scheduler_admission"}
|
||||||
|
scheduler_values = [e for e in events if e.get("type") == "scheduler_admission"
|
||||||
|
and burst_start_s <= float(e.get("t_s", -1.0)) <= burst_end_s]
|
||||||
|
apply_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "apply_frame"}
|
||||||
|
commandlet_tick_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "commandlet_world_tick_frame"}
|
||||||
|
garbage_collect_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "garbage_collect"}
|
||||||
|
# apply_frame N+1 records the interval containing AVoxelWorld::Tick N. Its apply_ms
|
||||||
|
# belongs to Tick N+1, so use apply_by_frame[N] for the same-tick apply breakdown.
|
||||||
|
near_intervals = [apply_by_frame[frame + 1] for frame in arrival_frame_ids
|
||||||
|
if frame + 1 in apply_by_frame]
|
||||||
|
rmc_by_frame: dict[int, dict[str, float]] = {}
|
||||||
|
for event in events:
|
||||||
|
frame = int(event.get("frame", -1))
|
||||||
|
if event.get("type") != "rmc_submit" or frame not in arrival_frame_ids:
|
||||||
|
continue
|
||||||
|
values = rmc_by_frame.setdefault(frame, {
|
||||||
|
"component_acquire_ms": 0.0,
|
||||||
|
"create_section_group_ms": 0.0,
|
||||||
|
"create_sections_ms": 0.0,
|
||||||
|
"collision_handoff_ms": 0.0,
|
||||||
|
})
|
||||||
|
for key in values:
|
||||||
|
values[key] += float(event.get(key, 0.0))
|
||||||
|
collision_completion_by_frame: dict[int, float] = {}
|
||||||
|
for event in events:
|
||||||
|
frame = int(event.get("frame", -1))
|
||||||
|
if event.get("type") == "collision_completion" and frame in arrival_frame_ids:
|
||||||
|
collision_completion_by_frame[frame] = collision_completion_by_frame.get(frame, 0.0) \
|
||||||
|
+ float(event.get("handler_ms", 0.0))
|
||||||
|
near_requests = [r for r in request_events if full and r.get("t_s", 1e30) <= full["t_s"]
|
||||||
|
and r.get("level", 9) <= 2]
|
||||||
|
initial_lod0_counts = Counter(
|
||||||
|
tuple(r.get("tile", ())) for r in near_requests if r.get("level") == 0)
|
||||||
|
repeated_lod0_keys = [
|
||||||
|
{"tile": list(tile), "requests": count}
|
||||||
|
for tile, count in sorted(initial_lod0_counts.items()) if count > 1]
|
||||||
|
initial_scheduler_frames = [e for e in events
|
||||||
|
if e.get("type") == "scheduler_admission" and full
|
||||||
|
and e.get("t_s", 1e30) <= full["t_s"]]
|
||||||
|
worst_interval = max(near_intervals, key=lambda e: float(e.get("frame_ms", 0.0)), default=None)
|
||||||
|
worst_interval_frame_id = int(worst_interval.get("frame", -1)) if worst_interval else -1
|
||||||
|
worst_work_frame_id = worst_interval_frame_id - 1
|
||||||
|
worst_work = work_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_apply = apply_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_scheduler = scheduler_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_rmc = rmc_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_collision_completion_ms = collision_completion_by_frame.get(worst_work_frame_id, 0.0)
|
||||||
|
rmc_values = [rmc_by_frame.get(frame, {}) for frame in arrival_frame_ids]
|
||||||
|
apply_values = [apply_by_frame[frame] for frame in arrival_frame_ids
|
||||||
|
if frame in apply_by_frame]
|
||||||
|
commandlet_tick_values = [commandlet_tick_by_frame[frame] for frame in arrival_frame_ids
|
||||||
|
if frame in commandlet_tick_by_frame]
|
||||||
|
garbage_collect_values = [garbage_collect_by_frame[frame] for frame in arrival_frame_ids
|
||||||
|
if frame in garbage_collect_by_frame]
|
||||||
|
|
||||||
|
def work_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in work_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
def scheduler_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in scheduler_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
def rmc_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in rmc_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
def commandlet_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in commandlet_tick_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
row = {
|
||||||
|
"label": label, "stratum": stratum, "repeat": repeat, "phase": phase,
|
||||||
|
"ready_queue_depth": ready_depth,
|
||||||
|
"requested_background_worker_cap": background_cap,
|
||||||
|
"background_worker_cap": worker_cap_readback.get(
|
||||||
|
"streaming_background_worker_cap", background_cap),
|
||||||
|
"requested_background_worker_reserve": worker_reserve,
|
||||||
|
"background_worker_reserve": worker_backend_config.get(
|
||||||
|
"streaming_background_worker_reserve", worker_cap_readback.get(
|
||||||
|
"background_worker_reserve", worker_reserve)),
|
||||||
|
"requested_core_parallel_for_background_priority": helper_background,
|
||||||
|
"core_parallel_for_background_priority": worker_backend_config.get(
|
||||||
|
"core_parallel_for_background_priority", helper_background),
|
||||||
|
"requested_streaming_tile_background_low": tile_background_low,
|
||||||
|
"streaming_tile_background_low": worker_backend_config.get(
|
||||||
|
"streaming_tile_background_low", tile_background_low),
|
||||||
|
"requested_streaming_tile_background_normal": tile_background_normal,
|
||||||
|
"streaming_tile_background_normal": worker_backend_config.get(
|
||||||
|
"streaming_tile_background_normal", tile_background_normal),
|
||||||
|
"streaming_tile_priority_split": worker_backend_config.get(
|
||||||
|
"streaming_tile_priority_split"),
|
||||||
|
"effective_tile_slots": worker_backend_config.get("effective_tile_slots"),
|
||||||
|
"background_worker_threads": worker_backend_config.get("taskgraph_background_threads"),
|
||||||
|
"startup_apply_boost": startup_boost, "insights": insights,
|
||||||
|
"engine_tick_logs": tick_logs,
|
||||||
|
"exit_code": exit_code,
|
||||||
|
"launch_wrapper": "taskset -c 0-11 timeout 1740s (no address-space limit)",
|
||||||
|
"peak_rss_kib": peak_rss[0] if peak_rss else None,
|
||||||
|
"peak_rss_mib": round(peak_rss[0] / 1024.0, 1) if peak_rss else None,
|
||||||
|
"peak_rss_pid": peak_rss[1] if peak_rss else None,
|
||||||
|
"endphysics_to_postphysics_gap_ms_max": None,
|
||||||
|
"endphysics_gap_frame": None,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"desired_t_s": next((e.get("t_s") for e in events if e.get("type") == "desired_set"), None),
|
||||||
|
"first_admission_t_s": first_admission.get("t_s") if first_admission else None,
|
||||||
|
"lod02_full_settled_t_s": full.get("t_s") if full else None,
|
||||||
|
"near_settle_from_first_admission_s": round(
|
||||||
|
float(full["t_s"]) - float(first_admission["t_s"]), 4)
|
||||||
|
if full and first_admission else None,
|
||||||
|
"desired_by_lod": full.get("desired_by_lod", [])[:3] if full else None,
|
||||||
|
"settled_by_lod": full.get("settled_by_lod", [])[:3] if full else None,
|
||||||
|
"near_admitted_tiles": len(near_requests),
|
||||||
|
"initial_lod0_admissions": sum(initial_lod0_counts.values()),
|
||||||
|
"repeated_lod0_request_keys_before_settle": repeated_lod0_keys,
|
||||||
|
"peak_result_backlog_before_settle": max(
|
||||||
|
(int(e.get("result_backlog", 0)) for e in initial_scheduler_frames), default=0),
|
||||||
|
"last_admission_t_s": max((e.get("t_s", 0.0) for e in near_requests), default=None),
|
||||||
|
"near_tile_events": len(initial_tiles),
|
||||||
|
"near_worker_seconds": round(sum(e.get("worker_task_s", 0.0) for e in initial_tiles), 4),
|
||||||
|
"near_core_seconds": round(sum(e.get("core_generate_s", 0.0) for e in initial_tiles), 4),
|
||||||
|
"collision_ready_by_settle": collision_count,
|
||||||
|
"burst_frame_count": len(near_intervals),
|
||||||
|
"burst_frame_ms_p95": round(sorted(e.get("frame_ms", 0.0) for e in near_intervals)[
|
||||||
|
min(len(near_intervals) - 1, int(.95 * (len(near_intervals) - 1)))], 3)
|
||||||
|
if near_intervals else None,
|
||||||
|
"burst_frame_ms_max": round(max((e.get("frame_ms", 0.0) for e in near_intervals), default=0.0), 3),
|
||||||
|
"burst_full_world_tick_ms_p99": round(commandlet_stat("world_tick_ms", .99) or 0.0, 3),
|
||||||
|
"burst_full_world_tick_ms_max": round(commandlet_stat("world_tick_ms") or 0.0, 3),
|
||||||
|
"burst_world_pre_actor_ms_p99": round(commandlet_stat("world_pre_actor_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_pre_actor_ms_max": round(commandlet_stat("world_pre_actor_ms") or 0.0, 3),
|
||||||
|
"burst_world_actor_phase_ms_p99": round(commandlet_stat("world_actor_phase_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_actor_phase_ms_max": round(commandlet_stat("world_actor_phase_ms") or 0.0, 3),
|
||||||
|
"burst_world_post_actor_ms_p99": round(commandlet_stat("world_post_actor_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_post_actor_ms_max": round(commandlet_stat("world_post_actor_ms") or 0.0, 3),
|
||||||
|
"burst_commandlet_readiness_check_ms_max": round(
|
||||||
|
commandlet_stat("readiness_check_ms") or 0.0, 3),
|
||||||
|
"burst_commandlet_sleep_ms_p99": round(commandlet_stat("sleep_ms", .99) or 0.0, 3),
|
||||||
|
"burst_commandlet_sleep_ms_max": round(commandlet_stat("sleep_ms") or 0.0, 3),
|
||||||
|
"burst_commandlet_loop_ms_max": round(commandlet_stat("loop_ms") or 0.0, 3),
|
||||||
|
"burst_gc_count": len(garbage_collect_values),
|
||||||
|
"burst_gc_duration_ms_max": round(max(
|
||||||
|
(float(e.get("duration_ms", 0.0)) for e in garbage_collect_values), default=0.0), 3),
|
||||||
|
"burst_max_meshes_per_frame": max((e.get("meshes_applied", 0) for e in apply_values), default=0),
|
||||||
|
"burst_max_apply_ms": round(max((e.get("apply_ms", 0.0) for e in apply_values), default=0.0), 3),
|
||||||
|
"burst_max_admit_per_frame": max((sum(e.get("submitted_by_lod", [0] * 9)[:3])
|
||||||
|
for e in scheduler_values), default=0),
|
||||||
|
"burst_frame_timing_count": len(work_values),
|
||||||
|
"burst_active_generation_workers_p95": round(work_stat("active_generation_workers", .95) or 0.0, 3),
|
||||||
|
"burst_active_generation_workers_max": round(work_stat("active_generation_workers") or 0.0, 3),
|
||||||
|
"burst_active_tile_tasks_max": round(work_stat("active_tile_tasks") or 0.0, 3),
|
||||||
|
"burst_world_tick_ms_p99": round(work_stat("tick_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_tick_ms_max": round(work_stat("tick_ms") or 0.0, 3),
|
||||||
|
"burst_update_chunks_ms_p99": round(work_stat("update_chunks_ms", .99) or 0.0, 3),
|
||||||
|
"burst_update_chunks_ms_max": round(work_stat("update_chunks_ms") or 0.0, 3),
|
||||||
|
"burst_pending_drain_ms_p99": round(work_stat("pending_drain_ms", .99) or 0.0, 3),
|
||||||
|
"burst_pending_drain_ms_max": round(work_stat("pending_drain_ms") or 0.0, 3),
|
||||||
|
"burst_post_apply_callbacks_ms_p99": round(work_stat("post_apply_callbacks_ms", .99) or 0.0, 3),
|
||||||
|
"burst_post_apply_callbacks_ms_max": round(work_stat("post_apply_callbacks_ms") or 0.0, 3),
|
||||||
|
"burst_visibility_reconcile_ms_p99": round(work_stat("visibility_reconcile_ms", .99) or 0.0, 3),
|
||||||
|
"burst_visibility_reconcile_ms_max": round(work_stat("visibility_reconcile_ms") or 0.0, 3),
|
||||||
|
"burst_collision_probe_ms_max": round(work_stat("collision_probe_ms") or 0.0, 3),
|
||||||
|
"burst_visible_apply_probe_ms_max": round(work_stat("visible_apply_probe_ms") or 0.0, 3),
|
||||||
|
"burst_lod0_ready_ms_max": round(work_stat("lod0_ready_ms") or 0.0, 3),
|
||||||
|
"burst_steady_trace_ms_max": round(work_stat("steady_trace_ms") or 0.0, 3),
|
||||||
|
"burst_modification_ms_max": round(work_stat("modification_ms") or 0.0, 3),
|
||||||
|
"burst_lifecycle_audit_ms_p99": round(work_stat("lifecycle_audit_ms", .99) or 0.0, 3),
|
||||||
|
"burst_lifecycle_audit_ms_max": round(work_stat("lifecycle_audit_ms") or 0.0, 3),
|
||||||
|
"burst_collision_gate_ms_p99": round(work_stat("collision_gate_ms", .99) or 0.0, 3),
|
||||||
|
"burst_collision_gate_ms_max": round(work_stat("collision_gate_ms") or 0.0, 3),
|
||||||
|
"burst_unload_ms_p99": round(work_stat("unload_ms", .99) or 0.0, 3),
|
||||||
|
"burst_unload_ms_max": round(work_stat("unload_ms") or 0.0, 3),
|
||||||
|
"burst_trace_json_ms_p99": round(work_stat("trace_json_ms", .99) or 0.0, 3),
|
||||||
|
"burst_trace_json_ms_max": round(work_stat("trace_json_ms") or 0.0, 3),
|
||||||
|
"burst_desired_rebuild_ms_max": round(scheduler_stat("desired_rebuild_ms") or 0.0, 3),
|
||||||
|
"burst_rmc_create_group_ms_p99": round(rmc_stat("create_section_group_ms", .99) or 0.0, 3),
|
||||||
|
"burst_rmc_create_group_ms_max": round(rmc_stat("create_section_group_ms") or 0.0, 3),
|
||||||
|
"burst_rmc_collision_handoff_ms_max": round(rmc_stat("collision_handoff_ms") or 0.0, 3),
|
||||||
|
"burst_collision_completion_ms_max": round(max(
|
||||||
|
collision_completion_by_frame.values(), default=0.0), 3),
|
||||||
|
"worst_frame_breakdown": {
|
||||||
|
"frame": worst_work_frame_id,
|
||||||
|
"world_tick_frame": worst_work_frame_id,
|
||||||
|
"frame_interval_end_frame": worst_interval_frame_id,
|
||||||
|
"frame_interval_ms": round(float(worst_interval.get("frame_ms", 0.0)), 3)
|
||||||
|
if worst_interval else None,
|
||||||
|
"apply_ms": round(float(worst_apply.get("apply_ms", 0.0)), 3)
|
||||||
|
if worst_apply else None,
|
||||||
|
"result_drain_ms": round(float(worst_apply.get("drain_ms", 0.0)), 3)
|
||||||
|
if worst_apply else None,
|
||||||
|
"outside_world_tick_ms": round(max(0.0, float(worst_interval.get("frame_ms", 0.0))
|
||||||
|
- float(worst_work.get("tick_ms", 0.0))), 3)
|
||||||
|
if worst_interval else None,
|
||||||
|
"outside_full_world_tick_ms": round(max(0.0, float(worst_interval.get("frame_ms", 0.0))
|
||||||
|
- float(commandlet_tick_by_frame.get(worst_work_frame_id, {}).get("world_tick_ms", 0.0))), 3)
|
||||||
|
if worst_interval else None,
|
||||||
|
"full_world_tick_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_tick_ms", 0.0)), 3),
|
||||||
|
"world_pre_actor_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_pre_actor_ms", 0.0)), 3),
|
||||||
|
"world_actor_phase_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_actor_phase_ms", 0.0)), 3),
|
||||||
|
"world_post_actor_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_post_actor_ms", 0.0)), 3),
|
||||||
|
"commandlet_readiness_check_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("readiness_check_ms", 0.0)), 3),
|
||||||
|
"commandlet_sleep_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("sleep_ms", 0.0)), 3),
|
||||||
|
"commandlet_loop_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("loop_ms", 0.0)), 3),
|
||||||
|
"gc_duration_ms": round(float(garbage_collect_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("duration_ms", 0.0)), 3),
|
||||||
|
"world_tick_ms": round(float(worst_work.get("tick_ms", 0.0)), 3),
|
||||||
|
"active_generation_workers": int(worst_work.get("active_generation_workers", 0)),
|
||||||
|
"active_tile_tasks": int(worst_work.get("active_tile_tasks", 0)),
|
||||||
|
"collision_gate_ms": round(float(worst_work.get("collision_gate_ms", 0.0)), 3),
|
||||||
|
"update_chunks_ms": round(float(worst_work.get("update_chunks_ms", 0.0)), 3),
|
||||||
|
"desired_rebuild_ms": round(float(worst_scheduler.get("desired_rebuild_ms", 0.0)), 3),
|
||||||
|
"pending_drain_ms": round(float(worst_work.get("pending_drain_ms", 0.0)), 3),
|
||||||
|
"post_apply_callbacks_ms": round(float(worst_work.get("post_apply_callbacks_ms", 0.0)), 3),
|
||||||
|
"collision_probe_ms": round(float(worst_work.get("collision_probe_ms", 0.0)), 3),
|
||||||
|
"visibility_reconcile_ms": round(float(worst_work.get("visibility_reconcile_ms", 0.0)), 3),
|
||||||
|
"visible_apply_probe_ms": round(float(worst_work.get("visible_apply_probe_ms", 0.0)), 3),
|
||||||
|
"lod0_ready_ms": round(float(worst_work.get("lod0_ready_ms", 0.0)), 3),
|
||||||
|
"unload_ms": round(float(worst_work.get("unload_ms", 0.0)), 3),
|
||||||
|
"steady_trace_ms": round(float(worst_work.get("steady_trace_ms", 0.0)), 3),
|
||||||
|
"modification_ms": round(float(worst_work.get("modification_ms", 0.0)), 3),
|
||||||
|
"lifecycle_audit_ms": round(float(worst_work.get("lifecycle_audit_ms", 0.0)), 3),
|
||||||
|
"trace_json_ms": round(float(worst_work.get("trace_json_ms", 0.0)), 3),
|
||||||
|
"trace_archive_serialize_ms": round(float(
|
||||||
|
worst_work.get("trace_archive_serialize_ms", 0.0)), 3),
|
||||||
|
"collision_completion_ms": round(worst_collision_completion_ms, 3),
|
||||||
|
**{key: round(float(worst_rmc.get(key, 0.0)), 3)
|
||||||
|
for key in ("component_acquire_ms", "create_section_group_ms",
|
||||||
|
"create_sections_ms", "collision_handoff_ms")},
|
||||||
|
},
|
||||||
|
"trace": str(trace.relative_to(ROOT)), "utrace": str(utrace.relative_to(ROOT)),
|
||||||
|
"abslog": str(abslog.relative_to(ROOT)), "stdout": str(stdout.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
stdout_text = stdout.read_text(encoding="utf-8", errors="replace")
|
||||||
|
gap_ms, gap_frame = endphysics_gap(stdout)
|
||||||
|
row["endphysics_to_postphysics_gap_ms_max"] = gap_ms
|
||||||
|
row["endphysics_gap_frame"] = gap_frame
|
||||||
|
failure = re.search(r"\[VoxelForgeCoreWorldTest\] failed: ([^\r\n]+)", stdout_text)
|
||||||
|
if failure:
|
||||||
|
row["world_test_failure"] = failure.group(1)
|
||||||
|
with (OUT / "runs.jsonl").open("a", encoding="utf-8") as stream:
|
||||||
|
stream.write(json.dumps(row, sort_keys=True) + "\n")
|
||||||
|
(OUT / f"{label}.summary.json").write_text(json.dumps(row, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
print("[L105UnrealArrival] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
if exit_code:
|
||||||
|
raise SystemExit(f"Unreal commandlet failed: {stdout}; exit={exit_code}")
|
||||||
|
if not full or len(arrival) < 8:
|
||||||
|
raise SystemExit(f"Unreal arrival trace incomplete: {stdout}; exit={exit_code}")
|
||||||
|
if insights and (not utrace.is_file() or utrace.stat().st_size < 1024):
|
||||||
|
raise SystemExit(f"Unreal Insights trace missing or empty: {utrace}")
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--stratum", choices=("FI", "SW"), required=True)
|
||||||
|
parser.add_argument("--repeat", type=int, default=1)
|
||||||
|
parser.add_argument("--phase", choices=("probe", "before", "after", "snapshot"), default="probe")
|
||||||
|
parser.add_argument("--ready-depth", type=int, default=64)
|
||||||
|
parser.add_argument("--background-cap", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--worker-reserve", type=int, choices=(0, 1, 2, 3), default=0)
|
||||||
|
parser.add_argument("--helper-background", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--tile-background-low", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--tile-background-normal", type=int, choices=(0, 1), default=0)
|
||||||
|
parser.add_argument("--startup-boost", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--tick-logs", action="store_true")
|
||||||
|
parser.add_argument("--insights", action="store_true")
|
||||||
|
args = parser.parse_args()
|
||||||
|
if not EDITOR.is_file():
|
||||||
|
raise SystemExit(f"missing Unreal commandlet executable: {EDITOR}")
|
||||||
|
run(args.stratum, args.repeat, args.phase, args.ready_depth,
|
||||||
|
args.background_cap, args.worker_reserve, args.helper_background,
|
||||||
|
args.tile_background_low, args.tile_background_normal, args.startup_boost,
|
||||||
|
args.tick_logs, args.insights)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,103 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Five interleaved FI arrival pairs, comparing 8 and 10 effective tile slots."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
RUNNER_PATH = Path(__file__).with_name("RunL120Arrival.py")
|
||||||
|
SPEC = importlib.util.spec_from_file_location("l120_arrival", RUNNER_PATH)
|
||||||
|
assert SPEC and SPEC.loader
|
||||||
|
ARRIVAL = importlib.util.module_from_spec(SPEC)
|
||||||
|
SPEC.loader.exec_module(ARRIVAL)
|
||||||
|
|
||||||
|
|
||||||
|
def add_view_metrics(row: dict) -> dict:
|
||||||
|
trace_path = ROOT / row["trace"]
|
||||||
|
events = ARRIVAL.read_events(trace_path)
|
||||||
|
config = next((e for e in events if e.get("type") == "worker_backend_config"), {})
|
||||||
|
row["effective_tile_slots"] = config.get("effective_tile_slots")
|
||||||
|
row["near_priority"] = config.get("near_priority")
|
||||||
|
row["horizon_priority"] = config.get("horizon_priority")
|
||||||
|
samples = [e for e in events if e.get("type") == "streaming_sample"
|
||||||
|
and float(e.get("t_s", 0.0)) < ARRIVAL.MOVE_START_S
|
||||||
|
and float(e.get("speed_cm_s", 0.0)) < 50.0]
|
||||||
|
|
||||||
|
def total(values: list[int]) -> int:
|
||||||
|
return sum(values)
|
||||||
|
|
||||||
|
def near_complete(event: dict) -> bool:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
settled = event.get("settled_by_lod", [0] * 9)
|
||||||
|
return total(desired[:3]) > 0 and total(settled[:3]) >= total(desired[:3])
|
||||||
|
|
||||||
|
def whole_complete(event: dict) -> bool:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
settled = event.get("settled_by_lod", [0] * 9)
|
||||||
|
return total(desired) > 0 and total(settled) >= total(desired)
|
||||||
|
|
||||||
|
def far_holes(event: dict) -> float | None:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)[3:]
|
||||||
|
covered = event.get("covered_by_lod", [0] * 9)[3:]
|
||||||
|
denominator = total(desired)
|
||||||
|
return (100.0 * (denominator - total(covered)) / denominator
|
||||||
|
if denominator else None)
|
||||||
|
|
||||||
|
first_near = next((e for e in samples if near_complete(e)), None)
|
||||||
|
first_whole = next((e for e in samples if whole_complete(e)), None)
|
||||||
|
first_far_clear = next((e for e in samples if total(e.get("desired_by_lod", [0] * 9)[3:])
|
||||||
|
and far_holes(e) == 0.0), None)
|
||||||
|
last_sample = samples[-1] if samples else None
|
||||||
|
row["lod02_settled_t_s"] = float(first_near["t_s"]) if first_near else None
|
||||||
|
row["whole_view_settled_t_s"] = float(first_whole["t_s"]) if first_whole else None
|
||||||
|
row["far_clear_t_s"] = float(first_far_clear["t_s"]) if first_far_clear else None
|
||||||
|
row["far_holes_at_20s_pct"] = far_holes(last_sample) if last_sample else None
|
||||||
|
row["arrival_sample_count"] = len(samples)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def run_condition(condition: str, repetition: int, ready_depth: int) -> dict:
|
||||||
|
cap = 1 if condition == "baseline" else 0
|
||||||
|
expected_slots = 8 if condition == "baseline" else 10
|
||||||
|
row = ARRIVAL.run(
|
||||||
|
"FI", repetition, condition, ready_depth, cap, 0, 1, 1, 0, 1, False, False)
|
||||||
|
row = add_view_metrics(row)
|
||||||
|
if row["effective_tile_slots"] != expected_slots:
|
||||||
|
raise RuntimeError(f"worker slot readback mismatch: {row}")
|
||||||
|
if row["streaming_tile_priority_split"] != 0:
|
||||||
|
raise RuntimeError(f"priority condition mismatch: {row}")
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--ready-depth", type=int, choices=(8, 64), default=8)
|
||||||
|
ready_depth = parser.parse_args().ready_depth
|
||||||
|
OUT = ROOT / "Saved/Linux/l120/arrival"
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
results = []
|
||||||
|
for condition in ("baseline", "slots10"):
|
||||||
|
warmup = run_condition(condition, 0, ready_depth)
|
||||||
|
warmup["warmup"] = True
|
||||||
|
results.append(warmup)
|
||||||
|
|
||||||
|
for repetition in range(1, 6):
|
||||||
|
order = ("baseline", "slots10") if repetition % 2 else ("slots10", "baseline")
|
||||||
|
for condition in order:
|
||||||
|
row = run_condition(condition, repetition, ready_depth)
|
||||||
|
row["warmup"] = False
|
||||||
|
results.append(row)
|
||||||
|
print("[L120ArrivalPair] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
|
||||||
|
(OUT / f"pairs-rd{ready_depth}.json").write_text(json.dumps({
|
||||||
|
"design": f"FI arrival, 8 versus 10 effective tile slots, priority split off, ready depth {ready_depth}",
|
||||||
|
"runs": results,
|
||||||
|
}, indent=2, sort_keys=True) + "\n", encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,117 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Compare L120 exact tile hashes with L104 across all strata, compilers and workers."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
OUT = ROOT / "Saved/Linux/l120/hashes"
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
OLD = ROOT / "Saved/Linux/l104/hashes"
|
||||||
|
BINARIES = {
|
||||||
|
"clang": ROOT / "Saved/Linux/l120/bin/VoxelForgeCoreHeadless-l120-clang",
|
||||||
|
"gcc": ROOT / "Saved/Linux/l120/bin/VoxelForgeCoreHeadless-l120-gcc",
|
||||||
|
}
|
||||||
|
CAMERAS = {
|
||||||
|
"sw": ((8.0, 16.0, -372.2626953125), 2850),
|
||||||
|
"fi": ((25.8369140625, 3.19921875, -737.0), 2249),
|
||||||
|
"tn": ((19.166015625, -14.2001953125, -753.3740234375), 2233),
|
||||||
|
"vs": ((-20.0625, -5.068359375, -900.353515625), 2329),
|
||||||
|
}
|
||||||
|
TILE = re.compile(
|
||||||
|
r"^\[PipelineTileHash\] lod=(-?\d+) x=(-?\d+) y=(-?\d+) z=(-?\d+) "
|
||||||
|
r"density=(0x[0-9a-f]+) mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) "
|
||||||
|
r"topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
WHOLE = re.compile(
|
||||||
|
r"^\[PipelineWholeViewHash\] tiles=(\d+) density=(0x[0-9a-f]+) "
|
||||||
|
r"mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M)
|
||||||
|
|
||||||
|
|
||||||
|
def parse(path: Path) -> tuple[dict, tuple[str, ...], str]:
|
||||||
|
content = path.read_text(encoding="utf-8", errors="replace")
|
||||||
|
tiles = {(int(m.group(1)), int(m.group(2)), int(m.group(3)), int(m.group(4))):
|
||||||
|
m.group(5, 6, 7, 8) for m in TILE.finditer(content)}
|
||||||
|
whole, stamp = WHOLE.search(content), STAMP.search(content)
|
||||||
|
if whole is None or stamp is None or len(tiles) != int(whole.group(1)):
|
||||||
|
raise RuntimeError(f"incomplete hash output: {path}")
|
||||||
|
return tiles, whole.groups(), stamp.group(1)
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, compiler: str, workers: int) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = OUT / f"{stratum}-{compiler}-w{workers}.log"
|
||||||
|
camera, expected_tiles = CAMERAS[stratum]
|
||||||
|
camera_arg = ",".join(f"{v:.12f}" for v in camera) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--", "timeout", "1740s",
|
||||||
|
str(BINARIES[compiler]), str(DATA), "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", f"--workers={workers}",
|
||||||
|
"--clip-radius=3", "--max-clip-level=8", "--max-level=8",
|
||||||
|
f"--camera={camera_arg}", "--stream-hash-tiles",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
if process.returncode:
|
||||||
|
raise RuntimeError(f"hash process exited {process.returncode}: {log}")
|
||||||
|
tiles, whole, stamp = parse(log)
|
||||||
|
if len(tiles) != expected_tiles:
|
||||||
|
raise RuntimeError(f"expected {expected_tiles} tiles, got {len(tiles)}: {log}")
|
||||||
|
old_log = OLD / f"{stratum}-{compiler}-w{workers}.log"
|
||||||
|
old_tiles, old_whole, _old_stamp = parse(old_log)
|
||||||
|
clang_reference, clang_whole, _ = parse(OLD / f"{stratum}-clang-w1.log")
|
||||||
|
|
||||||
|
def difference(reference: dict, candidate: dict) -> tuple[int, int]:
|
||||||
|
keys = set(reference) | set(candidate)
|
||||||
|
mismatches = sum(reference.get(key) != candidate.get(key) for key in keys)
|
||||||
|
return mismatches, len(keys)
|
||||||
|
|
||||||
|
old_mismatches, old_keys = difference(old_tiles, tiles)
|
||||||
|
cross_mismatches, cross_keys = difference(clang_reference, tiles)
|
||||||
|
row = {
|
||||||
|
"stratum": stratum, "compiler": compiler, "workers": workers,
|
||||||
|
"source_stamp": stamp, "tiles": len(tiles),
|
||||||
|
"l104_same_cell_tile_mismatches": old_mismatches,
|
||||||
|
"l104_same_cell_key_count": old_keys,
|
||||||
|
"l104_same_cell_whole_hash_match": whole == old_whole,
|
||||||
|
"clang_w1_tile_mismatches": cross_mismatches,
|
||||||
|
"clang_w1_key_count": cross_keys,
|
||||||
|
"clang_w1_whole_hash_match": whole == clang_whole,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L105HashRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
row["tile_hashes"] = {"|".join(map(str, key)): value for key, value in tiles.items()}
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
for path in (*BINARIES.values(), DATA):
|
||||||
|
if not path.is_file():
|
||||||
|
raise SystemExit(f"missing input: {path}")
|
||||||
|
rows = []
|
||||||
|
for stratum in CAMERAS:
|
||||||
|
for compiler in BINARIES:
|
||||||
|
for workers in (1, 8, 12):
|
||||||
|
rows.append(run(stratum, compiler, workers))
|
||||||
|
(OUT / "runs-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
failures = [row for row in rows if row["l104_same_cell_tile_mismatches"]
|
||||||
|
or not row["l104_same_cell_whole_hash_match"]
|
||||||
|
or row["clang_w1_tile_mismatches"]
|
||||||
|
or not row["clang_w1_whole_hash_match"]]
|
||||||
|
summary = {"runs": [{key: value for key, value in row.items() if key != "tile_hashes"}
|
||||||
|
for row in rows], "failed_runs": len(failures)}
|
||||||
|
(OUT / "summary.json").write_text(json.dumps(summary, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 1 if failures else 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,292 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Five-run interleaved L120 priority-split comparison at 30 m/s."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import re
|
||||||
|
import statistics
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
HOST = Path("/home/jahni/VFHost")
|
||||||
|
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
HEADLESS = ROOT / "Saved/Linux/l108/bin/VoxelForgeCoreHeadless-l108-after-clang"
|
||||||
|
OUT = ROOT / "Saved/Linux/l120/movement/priority-split-verified"
|
||||||
|
HOST_SAVED = HOST / "Saved/Linux/l120"
|
||||||
|
HOST_OUT = HOST / "Plugins/VoxelForge/Saved/Linux/l120/priority-split-verified"
|
||||||
|
ACTIVE_DESIGN = "priority_split"
|
||||||
|
SPAWN_M = (19.166015625, -14.2001953125, -753.3740234375)
|
||||||
|
MOVING_SAMPLE = re.compile(r"^\[L79MovingSample\] (.*)$", re.M)
|
||||||
|
MOVING_SUMMARY = re.compile(r"^\[L79MovingSummary\] (.*)$", re.M)
|
||||||
|
FIELDS = re.compile(r"([a-zA-Z0-9_]+)=([^ ]+)")
|
||||||
|
|
||||||
|
|
||||||
|
def p99(values: list[float]) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
values.sort()
|
||||||
|
return values[max(0, math.ceil(0.99 * len(values)) - 1)]
|
||||||
|
|
||||||
|
|
||||||
|
def parse_events(path: Path) -> list[dict]:
|
||||||
|
events = []
|
||||||
|
if not path.is_file():
|
||||||
|
return events
|
||||||
|
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||||
|
if not line.startswith("{"):
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
events.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
return events
|
||||||
|
|
||||||
|
|
||||||
|
def summarize(rows: list[dict], phase: str) -> dict:
|
||||||
|
moving = [row for row in rows if row["phase"] == "move"]
|
||||||
|
dwell = [row for row in rows if row["phase"] == "dwell"]
|
||||||
|
def moving_mean(name: str) -> float | None:
|
||||||
|
values = [row[name] for row in moving if name in row]
|
||||||
|
return statistics.mean(values) if values else None
|
||||||
|
complete = next((row for row in dwell
|
||||||
|
if row["near_ready_pct"] >= 99.999
|
||||||
|
and row["far_hole_pct"] <= 0.001), None)
|
||||||
|
end = dwell[-1] if dwell else {}
|
||||||
|
return {
|
||||||
|
"moving_samples": len(moving),
|
||||||
|
"dwell_samples": len(dwell),
|
||||||
|
"near_ready_moving_mean_pct": moving_mean("near_ready_pct"),
|
||||||
|
"lod0_ready_moving_mean_pct": moving_mean("lod0_ready_pct"),
|
||||||
|
"lod1_ready_moving_mean_pct": moving_mean("lod1_ready_pct"),
|
||||||
|
"lod2_ready_moving_mean_pct": moving_mean("lod2_ready_pct"),
|
||||||
|
"far_holes_moving_mean_pct": moving_mean("far_hole_pct"),
|
||||||
|
"near_ready_dwell_end_pct": end.get("near_ready_pct"),
|
||||||
|
"far_holes_dwell_end_pct": end.get("far_hole_pct"),
|
||||||
|
"dwell_complete_after_stop_s": complete.get("since_stop_s") if complete else None,
|
||||||
|
"phase": phase,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def run_headless(condition: str, repetition: int) -> dict:
|
||||||
|
worker_count = 8 if condition == "baseline" else 10
|
||||||
|
folder = OUT / condition / "headless"
|
||||||
|
folder.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = folder / f"fast-r{repetition}.log"
|
||||||
|
camera = ",".join(f"{value:.12f}" for value in SPAWN_M) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", str(HEADLESS), str(DATA),
|
||||||
|
"--world-assembler-prototype", "--stream-benchmark", "--stream-pipeline",
|
||||||
|
f"--workers={worker_count}", "--clip-radius=3", "--max-clip-level=8",
|
||||||
|
"--max-level=8", f"--camera={camera}", "--profile-lod-work",
|
||||||
|
"--stream-scheduler-policy=3", "--moving-camera-speed-mps=30.0",
|
||||||
|
"--moving-camera-duration-s=12", "--moving-camera-dwell-s=5",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
result = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
content = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
rows = []
|
||||||
|
for match in MOVING_SAMPLE.finditer(content):
|
||||||
|
row = dict(FIELDS.findall(match.group(1)))
|
||||||
|
for name in ("t_s", "near_ready_pct", "walkable_ring_ready_pct",
|
||||||
|
"far_hole_pct", "queue_update_ms"):
|
||||||
|
if name in row:
|
||||||
|
row[name] = float(row[name])
|
||||||
|
row["phase"] = row.get("phase", "move")
|
||||||
|
row["since_stop_s"] = max(0.0, row.get("t_s", 0.0) - 12.0)
|
||||||
|
rows.append(row)
|
||||||
|
engine_summary = {}
|
||||||
|
if match := MOVING_SUMMARY.search(content):
|
||||||
|
engine_summary = dict(FIELDS.findall(match.group(1)))
|
||||||
|
if result.returncode or not rows or int(engine_summary.get("generation_failures", "-1")) != 0:
|
||||||
|
raise RuntimeError(f"headless run failed: {log}; exit={result.returncode}")
|
||||||
|
moving_updates = [row["queue_update_ms"] for row in rows
|
||||||
|
if row["phase"] == "move" and "queue_update_ms" in row]
|
||||||
|
return {
|
||||||
|
"condition": condition, "platform": "headless", "speed_mps": 30.0,
|
||||||
|
"repetition": repetition, "worker_count": worker_count,
|
||||||
|
"exit_code": result.returncode,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"update_work_p99_ms": p99(moving_updates),
|
||||||
|
"update_work_max_ms": max(moving_updates, default=None),
|
||||||
|
"log": str(log.relative_to(ROOT)), **summarize(rows, "headless"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def run_unreal(condition: str, repetition: int) -> dict:
|
||||||
|
core_limit = 12
|
||||||
|
if ACTIVE_DESIGN == "priority_split":
|
||||||
|
worker_cap = 1
|
||||||
|
priority_split = 0 if condition == "baseline" else 1
|
||||||
|
else:
|
||||||
|
worker_cap = 1 if condition == "baseline" else 0
|
||||||
|
priority_split = 0
|
||||||
|
folder = OUT / condition / "unreal"
|
||||||
|
host_folder = HOST_OUT / condition / "output"
|
||||||
|
folder.mkdir(parents=True, exist_ok=True)
|
||||||
|
trace = folder / f"fast-r{repetition}.trace.jsonl"
|
||||||
|
abslog = folder / f"fast-r{repetition}.log"
|
||||||
|
stdout = folder / f"fast-r{repetition}.stdout.log"
|
||||||
|
export = host_folder / f"fast-r{repetition}"
|
||||||
|
userdir = HOST_SAVED / "user" / condition / f"fast-r{repetition}"
|
||||||
|
zendata = HOST_SAVED / "zen" / condition / f"fast-r{repetition}"
|
||||||
|
ddc = HOST_SAVED / "DDC"
|
||||||
|
for path in (export, userdir, zendata, ddc):
|
||||||
|
path.mkdir(parents=True, exist_ok=True)
|
||||||
|
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWN_M)
|
||||||
|
command = [
|
||||||
|
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
|
||||||
|
"-nop4", "-nosplash", "-NoSound", f"-corelimit={core_limit}",
|
||||||
|
"-NoZenAutoLaunch", "-notraceserver", "-nocrashreports",
|
||||||
|
"-DisablePlugins=AndroidFileServer", "-ddc=NoZenLocalFallback",
|
||||||
|
f"-userdir={userdir}", f"-ZenDataPath={zendata}",
|
||||||
|
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
|
||||||
|
"-archetype=TunnelNetwork", "-slot=0", "-modes=export", "-gameconfig=1",
|
||||||
|
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
|
||||||
|
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0",
|
||||||
|
"-coreplantest=0", "-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings",
|
||||||
|
"-corehybridmesher=0", f"-out={export}",
|
||||||
|
"-voxel.StreamingCoarseFirst=1", "-voxel.StreamingDynamicNearReserve=1",
|
||||||
|
"-voxel.StreamingLegacyPendingResultCap=0", "-voxel.CoreRegionConcurrentAdmission=1",
|
||||||
|
"-voxel.StreamingResultBacklogCap=256", "-voxel.StreamingReadyQueueDepth=8",
|
||||||
|
f"-voxel.StreamingBackgroundWorkerCap={worker_cap}",
|
||||||
|
"-voxel.CoreColumnCacheDirectMapped=0",
|
||||||
|
f"-voxel.StreamingTilePrioritySplit={priority_split}",
|
||||||
|
"-voxel.CoreMCGradientMemoization=1", "-voxel.StreamingCollisionLookahead=1",
|
||||||
|
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=3000.0",
|
||||||
|
"-voxel.TestMoveStartSeconds=6.0", "-voxel.TestMoveDurationSeconds=12.0",
|
||||||
|
"-voxel.TestExitSeconds=23.0", "-voxel.TestMoveDirection=1,0,0",
|
||||||
|
"-voxel.StreamingSchedulerPolicy=3", "-voxel.TestExitOnSteady=0",
|
||||||
|
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
|
||||||
|
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
|
||||||
|
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
|
||||||
|
f"-abslog={abslog}",
|
||||||
|
]
|
||||||
|
wrapped = ["taskset", "-c", "0-11", "timeout", "1740s", *command]
|
||||||
|
started = time.monotonic()
|
||||||
|
with stdout.open("w", encoding="utf-8") as stream:
|
||||||
|
result = subprocess.run(wrapped, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
events = parse_events(trace)
|
||||||
|
samples = [event for event in events if event.get("type") == "streaming_sample"]
|
||||||
|
moving_samples = [event for event in samples
|
||||||
|
if float(event.get("speed_cm_s", 0.0)) >= 50.0]
|
||||||
|
move_start = min((float(event.get("t_s", 0.0)) for event in moving_samples),
|
||||||
|
default=float("inf"))
|
||||||
|
stop_event = next((event for event in events
|
||||||
|
if event.get("type") == "streaming_test_move_stopped"), None)
|
||||||
|
stop = float(stop_event.get("t_s", 0.0)) if stop_event else float("inf")
|
||||||
|
rows = []
|
||||||
|
for event in samples:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
resident = event.get("resident_by_lod", [0] * 9)
|
||||||
|
covered = event.get("covered_by_lod", resident)
|
||||||
|
near_desired = sum(desired[:3])
|
||||||
|
near_resident = sum(resident[:3])
|
||||||
|
far_desired = sum(desired[3:])
|
||||||
|
far_covered = sum(covered[3:])
|
||||||
|
t_s = float(event.get("t_s", 0.0))
|
||||||
|
speed = float(event.get("speed_cm_s", 0.0))
|
||||||
|
phase = "move" if speed >= 50.0 else ("dwell" if t_s >= stop else "arrival")
|
||||||
|
rows.append({
|
||||||
|
"phase": phase, "t_s": t_s,
|
||||||
|
"since_stop_s": max(0.0, t_s - stop) if phase == "dwell" else None,
|
||||||
|
"near_ready_pct": 100.0 * near_resident / max(1, near_desired),
|
||||||
|
"lod0_ready_pct": 100.0 * resident[0] / max(1, desired[0]),
|
||||||
|
"lod1_ready_pct": 100.0 * resident[1] / max(1, desired[1]),
|
||||||
|
"lod2_ready_pct": 100.0 * resident[2] / max(1, desired[2]),
|
||||||
|
"far_hole_pct": 100.0 * (far_desired - far_covered) / max(1, far_desired),
|
||||||
|
})
|
||||||
|
dwell_rows = [row for row in rows if row["phase"] == "dwell"]
|
||||||
|
complete = next((row for row in dwell_rows
|
||||||
|
if row["near_ready_pct"] >= 99.999
|
||||||
|
and row["far_hole_pct"] <= 0.001), None)
|
||||||
|
moving_work = [event for event in events
|
||||||
|
if event.get("type") == "game_thread_work_frame"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop]
|
||||||
|
ticks = [float(event.get("tick_ms", 0.0)) for event in moving_work]
|
||||||
|
updates = [float(event.get("update_chunks_ms", 0.0)) for event in moving_work]
|
||||||
|
backend = next((event for event in events
|
||||||
|
if event.get("type") == "worker_backend_config"), {})
|
||||||
|
worker_cap_readback = next((event for event in events
|
||||||
|
if event.get("type") == "streaming_worker_cap_readback"), {})
|
||||||
|
if result.returncode or len(moving_samples) < 40 or len(dwell_rows) < 16:
|
||||||
|
raise RuntimeError(f"Unreal run failed/incomplete: {trace}; exit={result.returncode}; "
|
||||||
|
f"move={len(moving_samples)} dwell={len(dwell_rows)}")
|
||||||
|
expected_split = (0 if condition == "baseline" else 1) \
|
||||||
|
if ACTIVE_DESIGN == "priority_split" else 0
|
||||||
|
expected_near_priority = "BackgroundLow" if expected_split == 0 else "BackgroundNormal"
|
||||||
|
if (int(backend.get("streaming_tile_priority_split", -1)) != expected_split
|
||||||
|
or backend.get("near_priority") != expected_near_priority
|
||||||
|
or backend.get("horizon_priority") != "BackgroundLow"
|
||||||
|
or int(worker_cap_readback.get("streaming_background_worker_cap", -1)) != worker_cap
|
||||||
|
or int(backend.get("effective_tile_slots", -1)) != (8 if worker_cap else 10)):
|
||||||
|
raise RuntimeError(f"priority CVar readback mismatch in {trace}: {backend}")
|
||||||
|
return {
|
||||||
|
"condition": condition, "platform": "unreal", "speed_mps": 30.0,
|
||||||
|
"repetition": repetition, "core_limit": core_limit,
|
||||||
|
"effective_tile_slots": backend.get("effective_tile_slots"),
|
||||||
|
"streaming_background_worker_cap": worker_cap_readback.get(
|
||||||
|
"streaming_background_worker_cap"),
|
||||||
|
"taskgraph_background_threads": backend.get("taskgraph_background_threads"),
|
||||||
|
"exit_code": result.returncode,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"game_thread_tick_p99_ms": p99(ticks),
|
||||||
|
"game_thread_tick_max_ms": max(ticks, default=None),
|
||||||
|
"stream_update_p99_ms": p99(updates),
|
||||||
|
"stream_update_max_ms": max(updates, default=None),
|
||||||
|
"dwell_complete_after_stop_s": complete.get("since_stop_s") if complete else None,
|
||||||
|
"moving_samples": len(moving_samples), "dwell_samples": len(dwell_rows),
|
||||||
|
"trace": str(trace.relative_to(ROOT)),
|
||||||
|
"log": str(abslog.relative_to(ROOT)),
|
||||||
|
**summarize(rows, "unreal"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
global ACTIVE_DESIGN, OUT, HOST_SAVED, HOST_OUT
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--platform", choices=("all", "headless", "unreal"), default="all")
|
||||||
|
parser.add_argument("--design", choices=("priority_split", "worker_slots"),
|
||||||
|
default="priority_split")
|
||||||
|
args = parser.parse_args()
|
||||||
|
ACTIVE_DESIGN = args.design
|
||||||
|
design_path = "priority-split-verified" if ACTIVE_DESIGN == "priority_split" \
|
||||||
|
else "worker-slots-verified"
|
||||||
|
OUT = ROOT / "Saved/Linux/l120/movement" / design_path
|
||||||
|
HOST_SAVED = HOST / "Saved/Linux/l120" / design_path
|
||||||
|
HOST_OUT = HOST / "Plugins/VoxelForge/Saved/Linux/l120" / design_path
|
||||||
|
if not DATA.is_file() or not HEADLESS.is_file() or not EDITOR.is_file():
|
||||||
|
raise SystemExit("missing owner data, L108 headless binary, or Unreal editor")
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
results = []
|
||||||
|
partial = OUT / "runs-partial.json"
|
||||||
|
for repetition in range(1, 6):
|
||||||
|
candidate = "priority_split" if ACTIVE_DESIGN == "priority_split" else "slots10"
|
||||||
|
conditions = ("baseline", candidate) if repetition % 2 else (candidate, "baseline")
|
||||||
|
platforms = (("headless", "unreal") if repetition % 2 else ("unreal", "headless")) \
|
||||||
|
if args.platform == "all" else (args.platform,)
|
||||||
|
for condition in conditions:
|
||||||
|
for platform in platforms:
|
||||||
|
row = (run_headless(condition, repetition) if platform == "headless"
|
||||||
|
else run_unreal(condition, repetition))
|
||||||
|
results.append(row)
|
||||||
|
partial.write_text(json.dumps(results, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print("[L120Measurement] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
design_description = "near BGNormal vs horizon BackgroundLow" \
|
||||||
|
if ACTIVE_DESIGN == "priority_split" else "effective UE tile slots 8 vs 10"
|
||||||
|
(OUT / "summary.json").write_text(
|
||||||
|
json.dumps({"design": design_description,
|
||||||
|
"runs": results}, indent=2) + "\n", encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
# Linux 121 — finished far tiles
|
||||||
|
|
||||||
|
One change: after four normal visible applies, allow one far apply if less than half of the 2 ms budget is used. Disable the lane during startup boost. Geometry and worker limits stay unchanged.
|
||||||
|
|
||||||
|
Five interleaved UE pairs at 30 m/s (8 slots, ready depth 8) reduced median far coverage holes from 4.38% to 2.70%; near settled coverage rose from 45.12% to 45.43%. Far worker-finish-to-apply wait fell from 1.105 s / 3.25 s p50/p95 to 0.114 s / 2.40 s. The lane applied 672 extra far tiles across the five runs.
|
||||||
|
|
||||||
|
The worst tick was 14.74 ms, below L120's 15.43 ms ceiling. The worst per-run p99 was 11.00 ms versus 11.58 ms for baseline. Five 140 cm/s arrival pairs improved paired median LOD0–2 settlement by 20 ms and whole-view/far-clear by 313 ms. Both arms had zero far coverage holes at 20 s.
|
||||||
|
|
||||||
|
## Stage timings
|
||||||
|
|
||||||
|
Five paired UE traces per arm. Result wait is worker finish to `ApplyTileResult` (p50/p95 ms); generation, apply, and proxy are p95 ms. Proxy lag is reconstructed from callback and tile-apply timestamps because profile-only proxy-lag fields are zero when profiling is off.
|
||||||
|
|
||||||
|
| Speed / LOD | Far generation p95 | Result wait baseline → lane | Apply baseline → lane | Proxy baseline → lane |
|
||||||
|
|---|---:|---:|---:|---:|
|
||||||
|
| 1.4 m/s near | — | 12 / 43 → 9 / 37 | 0.362 → 0.340 | 0.411 → 0.406 |
|
||||||
|
| 1.4 m/s far | 5.6 → 6.3 | 209 / 229 → 142 / 209 | 0.314 → 0.252 | 0.221 → 0.196 |
|
||||||
|
| 30 m/s near | — | 12 / 125 → 12 / 120 | 0.206 → 0.197 | 0.250 → 0.237 |
|
||||||
|
| 30 m/s far | 10.0 → 10.3 | 1,105 / 3,250 → 114 / 2,397 | 0.351 → 0.324 | 0.335 → 0.312 |
|
||||||
|
|
||||||
|
The four-apply count limit was hit on 44% of baseline moving frames; the 2 ms time limit was hit on under 1%. Stale and obsolete results were discarded before apply and were under 1% of tile events.
|
||||||
|
|
||||||
|
Admission had idle-worker/queued-result windows while far coverage was missing: 504/2,135 baseline and 446/2,109 lane samples. The 256-result cap did not cause those windows: backlog p95/max was 101/206, with zero cap declines in them. Core-region compilation was pending in 64%/62% of those windows.
|
||||||
|
|
||||||
|
RMC reported `Updated` after proxy publication, but many replacement components were still hidden by the coverage handoff. NullRHI runs cannot measure the first presented pixel; the trace's far coverage is explicitly not visual enclosure.
|
||||||
|
|
||||||
|
UE build passed with `-NoUBA -MaxParallelActions=4`; fast tier passed (269 gates, 5 diagnostics). Clang/GCC W1/W8/W12 matched L104 across four strata: 24/24 exact tile and whole-view hashes. Headless has no UE apply budget, so the lane has no headless effect. Logs are under `Saved/Linux/l121`; timing used CPUs 0–11 with no build running.
|
||||||
@@ -0,0 +1,488 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Run one pinned null-RHI FI arrival for Linux 121."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from collections import Counter
|
||||||
|
from datetime import datetime
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
HOST = Path("/home/jahni/VFHost")
|
||||||
|
HOST_PLUGIN_SAVED = HOST / "Plugins/VoxelForge/Saved/Linux/l121"
|
||||||
|
HOST_SAVED = HOST / "Saved/Linux/l121"
|
||||||
|
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
|
||||||
|
OUT = ROOT / "Saved/Linux/l121/arrival"
|
||||||
|
SPAWNS_M = {
|
||||||
|
"FI": (25.8369140625, 3.19921875, -737.0),
|
||||||
|
"SW": (25.8369140625, 3.19921875, -200.0),
|
||||||
|
}
|
||||||
|
MOVE_START_S = 20.0
|
||||||
|
MOVE_DURATION_S = 12.0
|
||||||
|
EXIT_S = MOVE_START_S + MOVE_DURATION_S + 5.0
|
||||||
|
RUN_TIMEOUT_S = 1740
|
||||||
|
|
||||||
|
|
||||||
|
def read_events(path: Path) -> list[dict]:
|
||||||
|
events = []
|
||||||
|
if path.is_file():
|
||||||
|
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||||
|
if line.startswith("{"):
|
||||||
|
try:
|
||||||
|
events.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
return events
|
||||||
|
|
||||||
|
|
||||||
|
def percentile(values: list[float], fraction: float) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
ordered = sorted(values)
|
||||||
|
return ordered[min(len(ordered) - 1, int(round((len(ordered) - 1) * fraction)))]
|
||||||
|
|
||||||
|
|
||||||
|
def sample_unreal_peak_rss() -> tuple[int, int] | None:
|
||||||
|
executable = str(EDITOR).encode()
|
||||||
|
project = str(HOST / "VFHost.uproject").encode()
|
||||||
|
peak: tuple[int, int] | None = None
|
||||||
|
for process in Path("/proc").glob("[0-9]*"):
|
||||||
|
try:
|
||||||
|
command_line = (process / "cmdline").read_bytes()
|
||||||
|
if executable not in command_line or project not in command_line:
|
||||||
|
continue
|
||||||
|
status = (process / "status").read_text(encoding="ascii", errors="ignore")
|
||||||
|
match = re.search(r"^VmHWM:\s+(\d+)\s+kB$", status, re.MULTILINE)
|
||||||
|
if match:
|
||||||
|
rss_kib = int(match.group(1))
|
||||||
|
if peak is None or rss_kib > peak[0]:
|
||||||
|
peak = (rss_kib, int(process.name))
|
||||||
|
except (FileNotFoundError, PermissionError, ProcessLookupError, OSError):
|
||||||
|
continue
|
||||||
|
return peak
|
||||||
|
|
||||||
|
|
||||||
|
def endphysics_gap(stdout: Path) -> tuple[float | None, int | None]:
|
||||||
|
release_times: dict[int, dict[int, datetime]] = {}
|
||||||
|
pattern = re.compile(
|
||||||
|
r"^\[(\d{4}\.\d{2}\.\d{2}-\d{2}\.\d{2}\.\d{2}:\d{3})\].*"
|
||||||
|
r"LogTick: tick\s+(\d+)\s+-+\s+Release tick group ([34])")
|
||||||
|
with stdout.open(encoding="utf-8", errors="replace") as stream:
|
||||||
|
for line in stream:
|
||||||
|
match = pattern.match(line)
|
||||||
|
if not match:
|
||||||
|
continue
|
||||||
|
timestamp = datetime.strptime(match.group(1), "%Y.%m.%d-%H.%M.%S:%f")
|
||||||
|
frame = int(match.group(2))
|
||||||
|
group = int(match.group(3))
|
||||||
|
release_times.setdefault(frame, {})[group] = timestamp
|
||||||
|
gaps = [(round((times[4] - times[3]).total_seconds() * 1000.0, 3), frame)
|
||||||
|
for frame, times in release_times.items() if 3 in times and 4 in times]
|
||||||
|
return max(gaps, default=(None, None))
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, repeat: int, phase: str, ready_depth: int,
|
||||||
|
background_cap: int, worker_reserve: int, helper_background: int,
|
||||||
|
tile_background_low: int, tile_background_normal: int, startup_boost: int,
|
||||||
|
tick_logs: bool, insights: bool, far_apply_supplement: int) -> dict:
|
||||||
|
label = (f"{phase}-{stratum.lower()}-rd{ready_depth}-bg{background_cap}"
|
||||||
|
f"-reserve{worker_reserve}-helperbg{helper_background}"
|
||||||
|
f"-tilelow{tile_background_low}-tilenormal{tile_background_normal}"
|
||||||
|
f"-ticklog{int(tick_logs)}"
|
||||||
|
f"-farapply{far_apply_supplement}-boost{startup_boost}-{repeat}")
|
||||||
|
trace = OUT / f"{label}.trace.jsonl"
|
||||||
|
utrace = OUT / f"{label}.utrace"
|
||||||
|
abslog = OUT / f"{label}.log"
|
||||||
|
stdout = OUT / f"{label}.stdout.log"
|
||||||
|
output = HOST_PLUGIN_SAVED / "output" / label
|
||||||
|
userdir = HOST_SAVED / "user" / label
|
||||||
|
zendata = HOST_SAVED / "zen" / label
|
||||||
|
ddc = HOST_SAVED / "DDC"
|
||||||
|
for directory in (OUT, output, userdir, zendata, ddc):
|
||||||
|
directory.mkdir(parents=True, exist_ok=True)
|
||||||
|
trace.unlink(missing_ok=True)
|
||||||
|
utrace.unlink(missing_ok=True)
|
||||||
|
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWNS_M[stratum])
|
||||||
|
command = [
|
||||||
|
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
|
||||||
|
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
|
||||||
|
"-notraceserver", "-nocrashreports",
|
||||||
|
"-DisablePlugins=AndroidFileServer",
|
||||||
|
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
|
||||||
|
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
|
||||||
|
"-archetype=TunnelNetwork", f"-slot={1 if stratum == 'FI' else 0}",
|
||||||
|
"-modes=export", "-gameconfig=1", "-coreworldtest=1",
|
||||||
|
"-ownerfourstrateplan=1", "-corevoxelsizecm=25", "-coreislandtest=0",
|
||||||
|
"-corefallbacktest=0", "-coreofftest=0", "-coreplantest=0",
|
||||||
|
"-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings", "-corehybridmesher=0",
|
||||||
|
f"-out={output}", "-voxel.StreamingCoarseFirst=1",
|
||||||
|
"-voxel.StreamingDynamicNearReserve=1",
|
||||||
|
"-voxel.StreamingLegacyPendingResultCap=0",
|
||||||
|
"-voxel.CoreRegionConcurrentAdmission=1",
|
||||||
|
"-voxel.StreamingResultBacklogCap=256",
|
||||||
|
f"-voxel.StreamingReadyQueueDepth={ready_depth}",
|
||||||
|
f"-voxel.StreamingBackgroundWorkerCap={background_cap}",
|
||||||
|
f"-voxel.StreamingBackgroundWorkerReserve={worker_reserve}",
|
||||||
|
f"-voxel.CoreParallelForBackgroundPriority={helper_background}",
|
||||||
|
f"-voxel.StreamingTileBackgroundLow={tile_background_low}",
|
||||||
|
f"-voxel.StreamingTileBackgroundNormal={tile_background_normal}",
|
||||||
|
f"-voxel.TestLogTickFunctions={int(tick_logs)}",
|
||||||
|
"-voxel.CoreColumnCacheDirectMapped=0", "-voxel.CoreMCGradientMemoization=1",
|
||||||
|
"-voxel.StreamingCollisionLookahead=1",
|
||||||
|
f"-voxel.StreamingStartupApplyBoost={startup_boost}",
|
||||||
|
f"-voxel.StreamingFarApplySupplement={far_apply_supplement}",
|
||||||
|
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=140.0",
|
||||||
|
f"-voxel.TestMoveStartSeconds={MOVE_START_S:.1f}",
|
||||||
|
f"-voxel.TestMoveDurationSeconds={MOVE_DURATION_S:.1f}",
|
||||||
|
f"-voxel.TestExitSeconds={EXIT_S:.1f}", "-voxel.TestMoveDirection=1,0,0",
|
||||||
|
"-voxel.StreamingSchedulerPolicy=3", "-voxel.TestExitOnSteady=0",
|
||||||
|
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
|
||||||
|
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
|
||||||
|
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
|
||||||
|
f"-abslog={abslog}",
|
||||||
|
]
|
||||||
|
if insights:
|
||||||
|
command += ["-trace=cpu,task,frame", "-statnamedevents",
|
||||||
|
f"-tracefile={utrace}", "-tracefiletrunc"]
|
||||||
|
wrapped = ["taskset", "-c", "0-11", "timeout", f"{RUN_TIMEOUT_S}s", *command]
|
||||||
|
started = time.monotonic()
|
||||||
|
with stdout.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.Popen(wrapped, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT)
|
||||||
|
peak_rss = None
|
||||||
|
while process.poll() is None:
|
||||||
|
observed = sample_unreal_peak_rss()
|
||||||
|
if observed and (peak_rss is None or observed[0] > peak_rss[0]):
|
||||||
|
peak_rss = observed
|
||||||
|
time.sleep(0.25)
|
||||||
|
exit_code = process.wait()
|
||||||
|
observed = sample_unreal_peak_rss()
|
||||||
|
if observed and (peak_rss is None or observed[0] > peak_rss[0]):
|
||||||
|
peak_rss = observed
|
||||||
|
|
||||||
|
events = read_events(trace)
|
||||||
|
worker_backend_config = next(
|
||||||
|
(e for e in events if e.get("type") == "worker_backend_config"), {})
|
||||||
|
worker_cap_readback = next(
|
||||||
|
(e for e in events if e.get("type") == "streaming_worker_cap_readback"), {})
|
||||||
|
samples = [e for e in events if e.get("type") == "streaming_sample"]
|
||||||
|
arrival = [e for e in samples if float(e.get("t_s", 0.0)) < MOVE_START_S]
|
||||||
|
full = next((e for e in arrival
|
||||||
|
if sum(e.get("desired_by_lod", [0] * 9)[:3]) > 0
|
||||||
|
and sum(e.get("settled_by_lod", [0] * 9)[:3])
|
||||||
|
>= sum(e.get("desired_by_lod", [0] * 9)[:3])), None)
|
||||||
|
first_admission = next((e for e in events if e.get("type") == "scheduler_admission"
|
||||||
|
and e.get("desired_rebuilt") == 1), None)
|
||||||
|
request_events = [e for e in events if e.get("type") == "request"]
|
||||||
|
initial_request_events = [e for e in request_events if e.get("epoch") == 0]
|
||||||
|
tile_by_ordinal = {e.get("request_ordinal"): e for e in events
|
||||||
|
if e.get("type") == "tile"}
|
||||||
|
collision_count = sum(1 for e in events if e.get("type") == "collision_ready"
|
||||||
|
and e.get("level", 9) <= 2
|
||||||
|
and full and e.get("t_s", 1e30) <= full["t_s"])
|
||||||
|
initial_tiles = [tile_by_ordinal[r.get("request_ordinal")] for r in initial_request_events
|
||||||
|
if full and r.get("t_s", 1e30) <= full["t_s"]
|
||||||
|
and r.get("request_ordinal") in tile_by_ordinal
|
||||||
|
and tile_by_ordinal[r["request_ordinal"]].get("level", 9) <= 2]
|
||||||
|
burst_start_s = float(first_admission["t_s"]) if first_admission else -1.0
|
||||||
|
burst_end_s = float(full["t_s"]) if full else -1.0
|
||||||
|
work_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "game_thread_work_frame"}
|
||||||
|
work_values = [e for e in work_by_frame.values()
|
||||||
|
if burst_start_s <= float(e.get("t_s", -1.0)) <= burst_end_s]
|
||||||
|
arrival_frame_ids = {int(e.get("frame", -1)) for e in work_values}
|
||||||
|
scheduler_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "scheduler_admission"}
|
||||||
|
scheduler_values = [e for e in events if e.get("type") == "scheduler_admission"
|
||||||
|
and burst_start_s <= float(e.get("t_s", -1.0)) <= burst_end_s]
|
||||||
|
apply_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "apply_frame"}
|
||||||
|
commandlet_tick_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "commandlet_world_tick_frame"}
|
||||||
|
garbage_collect_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "garbage_collect"}
|
||||||
|
# apply_frame N+1 records the interval containing AVoxelWorld::Tick N. Its apply_ms
|
||||||
|
# belongs to Tick N+1, so use apply_by_frame[N] for the same-tick apply breakdown.
|
||||||
|
near_intervals = [apply_by_frame[frame + 1] for frame in arrival_frame_ids
|
||||||
|
if frame + 1 in apply_by_frame]
|
||||||
|
rmc_by_frame: dict[int, dict[str, float]] = {}
|
||||||
|
for event in events:
|
||||||
|
frame = int(event.get("frame", -1))
|
||||||
|
if event.get("type") != "rmc_submit" or frame not in arrival_frame_ids:
|
||||||
|
continue
|
||||||
|
values = rmc_by_frame.setdefault(frame, {
|
||||||
|
"component_acquire_ms": 0.0,
|
||||||
|
"create_section_group_ms": 0.0,
|
||||||
|
"create_sections_ms": 0.0,
|
||||||
|
"collision_handoff_ms": 0.0,
|
||||||
|
})
|
||||||
|
for key in values:
|
||||||
|
values[key] += float(event.get(key, 0.0))
|
||||||
|
collision_completion_by_frame: dict[int, float] = {}
|
||||||
|
for event in events:
|
||||||
|
frame = int(event.get("frame", -1))
|
||||||
|
if event.get("type") == "collision_completion" and frame in arrival_frame_ids:
|
||||||
|
collision_completion_by_frame[frame] = collision_completion_by_frame.get(frame, 0.0) \
|
||||||
|
+ float(event.get("handler_ms", 0.0))
|
||||||
|
near_requests = [r for r in request_events if full and r.get("t_s", 1e30) <= full["t_s"]
|
||||||
|
and r.get("level", 9) <= 2]
|
||||||
|
initial_lod0_counts = Counter(
|
||||||
|
tuple(r.get("tile", ())) for r in near_requests if r.get("level") == 0)
|
||||||
|
repeated_lod0_keys = [
|
||||||
|
{"tile": list(tile), "requests": count}
|
||||||
|
for tile, count in sorted(initial_lod0_counts.items()) if count > 1]
|
||||||
|
initial_scheduler_frames = [e for e in events
|
||||||
|
if e.get("type") == "scheduler_admission" and full
|
||||||
|
and e.get("t_s", 1e30) <= full["t_s"]]
|
||||||
|
worst_interval = max(near_intervals, key=lambda e: float(e.get("frame_ms", 0.0)), default=None)
|
||||||
|
worst_interval_frame_id = int(worst_interval.get("frame", -1)) if worst_interval else -1
|
||||||
|
worst_work_frame_id = worst_interval_frame_id - 1
|
||||||
|
worst_work = work_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_apply = apply_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_scheduler = scheduler_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_rmc = rmc_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_collision_completion_ms = collision_completion_by_frame.get(worst_work_frame_id, 0.0)
|
||||||
|
rmc_values = [rmc_by_frame.get(frame, {}) for frame in arrival_frame_ids]
|
||||||
|
apply_values = [apply_by_frame[frame] for frame in arrival_frame_ids
|
||||||
|
if frame in apply_by_frame]
|
||||||
|
commandlet_tick_values = [commandlet_tick_by_frame[frame] for frame in arrival_frame_ids
|
||||||
|
if frame in commandlet_tick_by_frame]
|
||||||
|
garbage_collect_values = [garbage_collect_by_frame[frame] for frame in arrival_frame_ids
|
||||||
|
if frame in garbage_collect_by_frame]
|
||||||
|
|
||||||
|
def work_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in work_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
def scheduler_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in scheduler_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
def rmc_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in rmc_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
def commandlet_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in commandlet_tick_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
row = {
|
||||||
|
"label": label, "stratum": stratum, "repeat": repeat, "phase": phase,
|
||||||
|
"ready_queue_depth": ready_depth,
|
||||||
|
"requested_background_worker_cap": background_cap,
|
||||||
|
"background_worker_cap": worker_cap_readback.get(
|
||||||
|
"streaming_background_worker_cap", background_cap),
|
||||||
|
"requested_background_worker_reserve": worker_reserve,
|
||||||
|
"background_worker_reserve": worker_backend_config.get(
|
||||||
|
"streaming_background_worker_reserve", worker_cap_readback.get(
|
||||||
|
"background_worker_reserve", worker_reserve)),
|
||||||
|
"requested_core_parallel_for_background_priority": helper_background,
|
||||||
|
"core_parallel_for_background_priority": worker_backend_config.get(
|
||||||
|
"core_parallel_for_background_priority", helper_background),
|
||||||
|
"requested_streaming_tile_background_low": tile_background_low,
|
||||||
|
"streaming_tile_background_low": worker_backend_config.get(
|
||||||
|
"streaming_tile_background_low", tile_background_low),
|
||||||
|
"requested_streaming_tile_background_normal": tile_background_normal,
|
||||||
|
"streaming_tile_background_normal": worker_backend_config.get(
|
||||||
|
"streaming_tile_background_normal", tile_background_normal),
|
||||||
|
"effective_tile_slots": worker_backend_config.get("effective_tile_slots"),
|
||||||
|
"background_worker_threads": worker_backend_config.get("taskgraph_background_threads"),
|
||||||
|
"startup_apply_boost": startup_boost,
|
||||||
|
"streaming_far_apply_supplement": worker_backend_config.get(
|
||||||
|
"streaming_far_apply_supplement"),
|
||||||
|
"insights": insights,
|
||||||
|
"engine_tick_logs": tick_logs,
|
||||||
|
"exit_code": exit_code,
|
||||||
|
"launch_wrapper": "taskset -c 0-11 timeout 1740s (no address-space limit)",
|
||||||
|
"peak_rss_kib": peak_rss[0] if peak_rss else None,
|
||||||
|
"peak_rss_mib": round(peak_rss[0] / 1024.0, 1) if peak_rss else None,
|
||||||
|
"peak_rss_pid": peak_rss[1] if peak_rss else None,
|
||||||
|
"endphysics_to_postphysics_gap_ms_max": None,
|
||||||
|
"endphysics_gap_frame": None,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"desired_t_s": next((e.get("t_s") for e in events if e.get("type") == "desired_set"), None),
|
||||||
|
"first_admission_t_s": first_admission.get("t_s") if first_admission else None,
|
||||||
|
"lod02_full_settled_t_s": full.get("t_s") if full else None,
|
||||||
|
"near_settle_from_first_admission_s": round(
|
||||||
|
float(full["t_s"]) - float(first_admission["t_s"]), 4)
|
||||||
|
if full and first_admission else None,
|
||||||
|
"desired_by_lod": full.get("desired_by_lod", [])[:3] if full else None,
|
||||||
|
"settled_by_lod": full.get("settled_by_lod", [])[:3] if full else None,
|
||||||
|
"near_admitted_tiles": len(near_requests),
|
||||||
|
"initial_lod0_admissions": sum(initial_lod0_counts.values()),
|
||||||
|
"repeated_lod0_request_keys_before_settle": repeated_lod0_keys,
|
||||||
|
"peak_result_backlog_before_settle": max(
|
||||||
|
(int(e.get("result_backlog", 0)) for e in initial_scheduler_frames), default=0),
|
||||||
|
"last_admission_t_s": max((e.get("t_s", 0.0) for e in near_requests), default=None),
|
||||||
|
"near_tile_events": len(initial_tiles),
|
||||||
|
"near_worker_seconds": round(sum(e.get("worker_task_s", 0.0) for e in initial_tiles), 4),
|
||||||
|
"near_core_seconds": round(sum(e.get("core_generate_s", 0.0) for e in initial_tiles), 4),
|
||||||
|
"collision_ready_by_settle": collision_count,
|
||||||
|
"burst_frame_count": len(near_intervals),
|
||||||
|
"burst_frame_ms_p95": round(sorted(e.get("frame_ms", 0.0) for e in near_intervals)[
|
||||||
|
min(len(near_intervals) - 1, int(.95 * (len(near_intervals) - 1)))], 3)
|
||||||
|
if near_intervals else None,
|
||||||
|
"burst_frame_ms_max": round(max((e.get("frame_ms", 0.0) for e in near_intervals), default=0.0), 3),
|
||||||
|
"burst_full_world_tick_ms_p99": round(commandlet_stat("world_tick_ms", .99) or 0.0, 3),
|
||||||
|
"burst_full_world_tick_ms_max": round(commandlet_stat("world_tick_ms") or 0.0, 3),
|
||||||
|
"burst_world_pre_actor_ms_p99": round(commandlet_stat("world_pre_actor_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_pre_actor_ms_max": round(commandlet_stat("world_pre_actor_ms") or 0.0, 3),
|
||||||
|
"burst_world_actor_phase_ms_p99": round(commandlet_stat("world_actor_phase_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_actor_phase_ms_max": round(commandlet_stat("world_actor_phase_ms") or 0.0, 3),
|
||||||
|
"burst_world_post_actor_ms_p99": round(commandlet_stat("world_post_actor_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_post_actor_ms_max": round(commandlet_stat("world_post_actor_ms") or 0.0, 3),
|
||||||
|
"burst_commandlet_readiness_check_ms_max": round(
|
||||||
|
commandlet_stat("readiness_check_ms") or 0.0, 3),
|
||||||
|
"burst_commandlet_sleep_ms_p99": round(commandlet_stat("sleep_ms", .99) or 0.0, 3),
|
||||||
|
"burst_commandlet_sleep_ms_max": round(commandlet_stat("sleep_ms") or 0.0, 3),
|
||||||
|
"burst_commandlet_loop_ms_max": round(commandlet_stat("loop_ms") or 0.0, 3),
|
||||||
|
"burst_gc_count": len(garbage_collect_values),
|
||||||
|
"burst_gc_duration_ms_max": round(max(
|
||||||
|
(float(e.get("duration_ms", 0.0)) for e in garbage_collect_values), default=0.0), 3),
|
||||||
|
"burst_max_meshes_per_frame": max((e.get("meshes_applied", 0) for e in apply_values), default=0),
|
||||||
|
"burst_max_apply_ms": round(max((e.get("apply_ms", 0.0) for e in apply_values), default=0.0), 3),
|
||||||
|
"burst_max_admit_per_frame": max((sum(e.get("submitted_by_lod", [0] * 9)[:3])
|
||||||
|
for e in scheduler_values), default=0),
|
||||||
|
"burst_frame_timing_count": len(work_values),
|
||||||
|
"burst_active_generation_workers_p95": round(work_stat("active_generation_workers", .95) or 0.0, 3),
|
||||||
|
"burst_active_generation_workers_max": round(work_stat("active_generation_workers") or 0.0, 3),
|
||||||
|
"burst_active_tile_tasks_max": round(work_stat("active_tile_tasks") or 0.0, 3),
|
||||||
|
"burst_world_tick_ms_p99": round(work_stat("tick_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_tick_ms_max": round(work_stat("tick_ms") or 0.0, 3),
|
||||||
|
"burst_update_chunks_ms_p99": round(work_stat("update_chunks_ms", .99) or 0.0, 3),
|
||||||
|
"burst_update_chunks_ms_max": round(work_stat("update_chunks_ms") or 0.0, 3),
|
||||||
|
"burst_pending_drain_ms_p99": round(work_stat("pending_drain_ms", .99) or 0.0, 3),
|
||||||
|
"burst_pending_drain_ms_max": round(work_stat("pending_drain_ms") or 0.0, 3),
|
||||||
|
"burst_post_apply_callbacks_ms_p99": round(work_stat("post_apply_callbacks_ms", .99) or 0.0, 3),
|
||||||
|
"burst_post_apply_callbacks_ms_max": round(work_stat("post_apply_callbacks_ms") or 0.0, 3),
|
||||||
|
"burst_visibility_reconcile_ms_p99": round(work_stat("visibility_reconcile_ms", .99) or 0.0, 3),
|
||||||
|
"burst_visibility_reconcile_ms_max": round(work_stat("visibility_reconcile_ms") or 0.0, 3),
|
||||||
|
"burst_collision_probe_ms_max": round(work_stat("collision_probe_ms") or 0.0, 3),
|
||||||
|
"burst_visible_apply_probe_ms_max": round(work_stat("visible_apply_probe_ms") or 0.0, 3),
|
||||||
|
"burst_lod0_ready_ms_max": round(work_stat("lod0_ready_ms") or 0.0, 3),
|
||||||
|
"burst_steady_trace_ms_max": round(work_stat("steady_trace_ms") or 0.0, 3),
|
||||||
|
"burst_modification_ms_max": round(work_stat("modification_ms") or 0.0, 3),
|
||||||
|
"burst_lifecycle_audit_ms_p99": round(work_stat("lifecycle_audit_ms", .99) or 0.0, 3),
|
||||||
|
"burst_lifecycle_audit_ms_max": round(work_stat("lifecycle_audit_ms") or 0.0, 3),
|
||||||
|
"burst_collision_gate_ms_p99": round(work_stat("collision_gate_ms", .99) or 0.0, 3),
|
||||||
|
"burst_collision_gate_ms_max": round(work_stat("collision_gate_ms") or 0.0, 3),
|
||||||
|
"burst_unload_ms_p99": round(work_stat("unload_ms", .99) or 0.0, 3),
|
||||||
|
"burst_unload_ms_max": round(work_stat("unload_ms") or 0.0, 3),
|
||||||
|
"burst_trace_json_ms_p99": round(work_stat("trace_json_ms", .99) or 0.0, 3),
|
||||||
|
"burst_trace_json_ms_max": round(work_stat("trace_json_ms") or 0.0, 3),
|
||||||
|
"burst_desired_rebuild_ms_max": round(scheduler_stat("desired_rebuild_ms") or 0.0, 3),
|
||||||
|
"burst_rmc_create_group_ms_p99": round(rmc_stat("create_section_group_ms", .99) or 0.0, 3),
|
||||||
|
"burst_rmc_create_group_ms_max": round(rmc_stat("create_section_group_ms") or 0.0, 3),
|
||||||
|
"burst_rmc_collision_handoff_ms_max": round(rmc_stat("collision_handoff_ms") or 0.0, 3),
|
||||||
|
"burst_collision_completion_ms_max": round(max(
|
||||||
|
collision_completion_by_frame.values(), default=0.0), 3),
|
||||||
|
"worst_frame_breakdown": {
|
||||||
|
"frame": worst_work_frame_id,
|
||||||
|
"world_tick_frame": worst_work_frame_id,
|
||||||
|
"frame_interval_end_frame": worst_interval_frame_id,
|
||||||
|
"frame_interval_ms": round(float(worst_interval.get("frame_ms", 0.0)), 3)
|
||||||
|
if worst_interval else None,
|
||||||
|
"apply_ms": round(float(worst_apply.get("apply_ms", 0.0)), 3)
|
||||||
|
if worst_apply else None,
|
||||||
|
"result_drain_ms": round(float(worst_apply.get("drain_ms", 0.0)), 3)
|
||||||
|
if worst_apply else None,
|
||||||
|
"outside_world_tick_ms": round(max(0.0, float(worst_interval.get("frame_ms", 0.0))
|
||||||
|
- float(worst_work.get("tick_ms", 0.0))), 3)
|
||||||
|
if worst_interval else None,
|
||||||
|
"outside_full_world_tick_ms": round(max(0.0, float(worst_interval.get("frame_ms", 0.0))
|
||||||
|
- float(commandlet_tick_by_frame.get(worst_work_frame_id, {}).get("world_tick_ms", 0.0))), 3)
|
||||||
|
if worst_interval else None,
|
||||||
|
"full_world_tick_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_tick_ms", 0.0)), 3),
|
||||||
|
"world_pre_actor_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_pre_actor_ms", 0.0)), 3),
|
||||||
|
"world_actor_phase_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_actor_phase_ms", 0.0)), 3),
|
||||||
|
"world_post_actor_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_post_actor_ms", 0.0)), 3),
|
||||||
|
"commandlet_readiness_check_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("readiness_check_ms", 0.0)), 3),
|
||||||
|
"commandlet_sleep_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("sleep_ms", 0.0)), 3),
|
||||||
|
"commandlet_loop_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("loop_ms", 0.0)), 3),
|
||||||
|
"gc_duration_ms": round(float(garbage_collect_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("duration_ms", 0.0)), 3),
|
||||||
|
"world_tick_ms": round(float(worst_work.get("tick_ms", 0.0)), 3),
|
||||||
|
"active_generation_workers": int(worst_work.get("active_generation_workers", 0)),
|
||||||
|
"active_tile_tasks": int(worst_work.get("active_tile_tasks", 0)),
|
||||||
|
"collision_gate_ms": round(float(worst_work.get("collision_gate_ms", 0.0)), 3),
|
||||||
|
"update_chunks_ms": round(float(worst_work.get("update_chunks_ms", 0.0)), 3),
|
||||||
|
"desired_rebuild_ms": round(float(worst_scheduler.get("desired_rebuild_ms", 0.0)), 3),
|
||||||
|
"pending_drain_ms": round(float(worst_work.get("pending_drain_ms", 0.0)), 3),
|
||||||
|
"post_apply_callbacks_ms": round(float(worst_work.get("post_apply_callbacks_ms", 0.0)), 3),
|
||||||
|
"collision_probe_ms": round(float(worst_work.get("collision_probe_ms", 0.0)), 3),
|
||||||
|
"visibility_reconcile_ms": round(float(worst_work.get("visibility_reconcile_ms", 0.0)), 3),
|
||||||
|
"visible_apply_probe_ms": round(float(worst_work.get("visible_apply_probe_ms", 0.0)), 3),
|
||||||
|
"lod0_ready_ms": round(float(worst_work.get("lod0_ready_ms", 0.0)), 3),
|
||||||
|
"unload_ms": round(float(worst_work.get("unload_ms", 0.0)), 3),
|
||||||
|
"steady_trace_ms": round(float(worst_work.get("steady_trace_ms", 0.0)), 3),
|
||||||
|
"modification_ms": round(float(worst_work.get("modification_ms", 0.0)), 3),
|
||||||
|
"lifecycle_audit_ms": round(float(worst_work.get("lifecycle_audit_ms", 0.0)), 3),
|
||||||
|
"trace_json_ms": round(float(worst_work.get("trace_json_ms", 0.0)), 3),
|
||||||
|
"trace_archive_serialize_ms": round(float(
|
||||||
|
worst_work.get("trace_archive_serialize_ms", 0.0)), 3),
|
||||||
|
"collision_completion_ms": round(worst_collision_completion_ms, 3),
|
||||||
|
**{key: round(float(worst_rmc.get(key, 0.0)), 3)
|
||||||
|
for key in ("component_acquire_ms", "create_section_group_ms",
|
||||||
|
"create_sections_ms", "collision_handoff_ms")},
|
||||||
|
},
|
||||||
|
"trace": str(trace.relative_to(ROOT)), "utrace": str(utrace.relative_to(ROOT)),
|
||||||
|
"abslog": str(abslog.relative_to(ROOT)), "stdout": str(stdout.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
stdout_text = stdout.read_text(encoding="utf-8", errors="replace")
|
||||||
|
gap_ms, gap_frame = endphysics_gap(stdout)
|
||||||
|
row["endphysics_to_postphysics_gap_ms_max"] = gap_ms
|
||||||
|
row["endphysics_gap_frame"] = gap_frame
|
||||||
|
failure = re.search(r"\[VoxelForgeCoreWorldTest\] failed: ([^\r\n]+)", stdout_text)
|
||||||
|
if failure:
|
||||||
|
row["world_test_failure"] = failure.group(1)
|
||||||
|
with (OUT / "runs.jsonl").open("a", encoding="utf-8") as stream:
|
||||||
|
stream.write(json.dumps(row, sort_keys=True) + "\n")
|
||||||
|
(OUT / f"{label}.summary.json").write_text(json.dumps(row, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
print("[L105UnrealArrival] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
if exit_code:
|
||||||
|
raise SystemExit(f"Unreal commandlet failed: {stdout}; exit={exit_code}")
|
||||||
|
if not full or len(arrival) < 8:
|
||||||
|
raise SystemExit(f"Unreal arrival trace incomplete: {stdout}; exit={exit_code}")
|
||||||
|
if insights and (not utrace.is_file() or utrace.stat().st_size < 1024):
|
||||||
|
raise SystemExit(f"Unreal Insights trace missing or empty: {utrace}")
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--stratum", choices=("FI", "SW"), required=True)
|
||||||
|
parser.add_argument("--repeat", type=int, default=1)
|
||||||
|
parser.add_argument("--phase", choices=("probe", "before", "after", "snapshot"), default="probe")
|
||||||
|
parser.add_argument("--ready-depth", type=int, default=64)
|
||||||
|
parser.add_argument("--background-cap", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--worker-reserve", type=int, choices=(0, 1, 2, 3), default=0)
|
||||||
|
parser.add_argument("--helper-background", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--tile-background-low", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--tile-background-normal", type=int, choices=(0, 1), default=0)
|
||||||
|
parser.add_argument("--startup-boost", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--tick-logs", action="store_true")
|
||||||
|
parser.add_argument("--insights", action="store_true")
|
||||||
|
args = parser.parse_args()
|
||||||
|
if not EDITOR.is_file():
|
||||||
|
raise SystemExit(f"missing Unreal commandlet executable: {EDITOR}")
|
||||||
|
run(args.stratum, args.repeat, args.phase, args.ready_depth,
|
||||||
|
args.background_cap, args.worker_reserve, args.helper_background,
|
||||||
|
args.tile_background_low, args.tile_background_normal, args.startup_boost,
|
||||||
|
args.tick_logs, args.insights)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,104 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Five interleaved FI arrival pairs, comparing normal applies with one supplemental far apply."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
RUNNER_PATH = Path(__file__).with_name("RunL121Arrival.py")
|
||||||
|
SPEC = importlib.util.spec_from_file_location("l121_arrival", RUNNER_PATH)
|
||||||
|
assert SPEC and SPEC.loader
|
||||||
|
ARRIVAL = importlib.util.module_from_spec(SPEC)
|
||||||
|
SPEC.loader.exec_module(ARRIVAL)
|
||||||
|
|
||||||
|
|
||||||
|
def add_view_metrics(row: dict) -> dict:
|
||||||
|
trace_path = ROOT / row["trace"]
|
||||||
|
events = ARRIVAL.read_events(trace_path)
|
||||||
|
config = next((e for e in events if e.get("type") == "worker_backend_config"), {})
|
||||||
|
row["effective_tile_slots"] = config.get("effective_tile_slots")
|
||||||
|
row["near_priority"] = config.get("near_priority")
|
||||||
|
row["horizon_priority"] = config.get("horizon_priority")
|
||||||
|
samples = [e for e in events if e.get("type") == "streaming_sample"
|
||||||
|
and float(e.get("t_s", 0.0)) < ARRIVAL.MOVE_START_S
|
||||||
|
and float(e.get("speed_cm_s", 0.0)) < 50.0]
|
||||||
|
|
||||||
|
def total(values: list[int]) -> int:
|
||||||
|
return sum(values)
|
||||||
|
|
||||||
|
def near_complete(event: dict) -> bool:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
settled = event.get("settled_by_lod", [0] * 9)
|
||||||
|
return total(desired[:3]) > 0 and total(settled[:3]) >= total(desired[:3])
|
||||||
|
|
||||||
|
def whole_complete(event: dict) -> bool:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
settled = event.get("settled_by_lod", [0] * 9)
|
||||||
|
return total(desired) > 0 and total(settled) >= total(desired)
|
||||||
|
|
||||||
|
def far_holes(event: dict) -> float | None:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)[3:]
|
||||||
|
covered = event.get("covered_by_lod", [0] * 9)[3:]
|
||||||
|
denominator = total(desired)
|
||||||
|
return (100.0 * (denominator - total(covered)) / denominator
|
||||||
|
if denominator else None)
|
||||||
|
|
||||||
|
first_near = next((e for e in samples if near_complete(e)), None)
|
||||||
|
first_whole = next((e for e in samples if whole_complete(e)), None)
|
||||||
|
first_far_clear = next((e for e in samples if total(e.get("desired_by_lod", [0] * 9)[3:])
|
||||||
|
and far_holes(e) == 0.0), None)
|
||||||
|
last_sample = samples[-1] if samples else None
|
||||||
|
row["lod02_settled_t_s"] = float(first_near["t_s"]) if first_near else None
|
||||||
|
row["whole_view_settled_t_s"] = float(first_whole["t_s"]) if first_whole else None
|
||||||
|
row["far_clear_t_s"] = float(first_far_clear["t_s"]) if first_far_clear else None
|
||||||
|
row["far_holes_at_20s_pct"] = far_holes(last_sample) if last_sample else None
|
||||||
|
row["arrival_sample_count"] = len(samples)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def run_condition(condition: str, repetition: int, ready_depth: int) -> dict:
|
||||||
|
cap = 1
|
||||||
|
expected_slots = 8
|
||||||
|
row = ARRIVAL.run(
|
||||||
|
"FI", repetition, condition, ready_depth, cap, 0, 1, 1, 0, 1, False, False,
|
||||||
|
0 if condition == "baseline" else 1)
|
||||||
|
row = add_view_metrics(row)
|
||||||
|
if row["effective_tile_slots"] != expected_slots:
|
||||||
|
raise RuntimeError(f"worker slot readback mismatch: {row}")
|
||||||
|
if row["streaming_far_apply_supplement"] != (0 if condition == "baseline" else 1):
|
||||||
|
raise RuntimeError(f"far apply supplement CVar mismatch: {row}")
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--ready-depth", type=int, choices=(8, 64), default=8)
|
||||||
|
ready_depth = parser.parse_args().ready_depth
|
||||||
|
OUT = ROOT / "Saved/Linux/l121/arrival"
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
results = []
|
||||||
|
for condition in ("baseline", "far_supplement"):
|
||||||
|
warmup = run_condition(condition, 0, ready_depth)
|
||||||
|
warmup["warmup"] = True
|
||||||
|
results.append(warmup)
|
||||||
|
|
||||||
|
for repetition in range(1, 6):
|
||||||
|
order = ("baseline", "far_supplement") if repetition % 2 else ("far_supplement", "baseline")
|
||||||
|
for condition in order:
|
||||||
|
row = run_condition(condition, repetition, ready_depth)
|
||||||
|
row["warmup"] = False
|
||||||
|
results.append(row)
|
||||||
|
print("[L121ArrivalPair] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
|
||||||
|
(OUT / f"pairs-rd{ready_depth}.json").write_text(json.dumps({
|
||||||
|
"design": f"FI arrival, normal four applies versus one supplemental far apply, ready depth {ready_depth}",
|
||||||
|
"runs": results,
|
||||||
|
}, indent=2, sort_keys=True) + "\n", encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,117 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Compare L121 exact tile hashes with L104 across all strata, compilers and workers."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
OUT = ROOT / "Saved/Linux/l121/hashes"
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
OLD = ROOT / "Saved/Linux/l104/hashes"
|
||||||
|
BINARIES = {
|
||||||
|
"clang": ROOT / "Saved/Linux/l121/verify/bin/VoxelForgeCoreHeadless-verify-clang",
|
||||||
|
"gcc": ROOT / "Saved/Linux/l121/verify/bin/VoxelForgeCoreHeadless-verify-gcc",
|
||||||
|
}
|
||||||
|
CAMERAS = {
|
||||||
|
"sw": ((8.0, 16.0, -372.2626953125), 2850),
|
||||||
|
"fi": ((25.8369140625, 3.19921875, -737.0), 2249),
|
||||||
|
"tn": ((19.166015625, -14.2001953125, -753.3740234375), 2233),
|
||||||
|
"vs": ((-20.0625, -5.068359375, -900.353515625), 2329),
|
||||||
|
}
|
||||||
|
TILE = re.compile(
|
||||||
|
r"^\[PipelineTileHash\] lod=(-?\d+) x=(-?\d+) y=(-?\d+) z=(-?\d+) "
|
||||||
|
r"density=(0x[0-9a-f]+) mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) "
|
||||||
|
r"topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
WHOLE = re.compile(
|
||||||
|
r"^\[PipelineWholeViewHash\] tiles=(\d+) density=(0x[0-9a-f]+) "
|
||||||
|
r"mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M)
|
||||||
|
|
||||||
|
|
||||||
|
def parse(path: Path) -> tuple[dict, tuple[str, ...], str]:
|
||||||
|
content = path.read_text(encoding="utf-8", errors="replace")
|
||||||
|
tiles = {(int(m.group(1)), int(m.group(2)), int(m.group(3)), int(m.group(4))):
|
||||||
|
m.group(5, 6, 7, 8) for m in TILE.finditer(content)}
|
||||||
|
whole, stamp = WHOLE.search(content), STAMP.search(content)
|
||||||
|
if whole is None or stamp is None or len(tiles) != int(whole.group(1)):
|
||||||
|
raise RuntimeError(f"incomplete hash output: {path}")
|
||||||
|
return tiles, whole.groups(), stamp.group(1)
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, compiler: str, workers: int) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = OUT / f"{stratum}-{compiler}-w{workers}.log"
|
||||||
|
camera, expected_tiles = CAMERAS[stratum]
|
||||||
|
camera_arg = ",".join(f"{v:.12f}" for v in camera) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--", "timeout", "1740s",
|
||||||
|
str(BINARIES[compiler]), str(DATA), "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", f"--workers={workers}",
|
||||||
|
"--clip-radius=3", "--max-clip-level=8", "--max-level=8",
|
||||||
|
f"--camera={camera_arg}", "--stream-hash-tiles",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
if process.returncode:
|
||||||
|
raise RuntimeError(f"hash process exited {process.returncode}: {log}")
|
||||||
|
tiles, whole, stamp = parse(log)
|
||||||
|
if len(tiles) != expected_tiles:
|
||||||
|
raise RuntimeError(f"expected {expected_tiles} tiles, got {len(tiles)}: {log}")
|
||||||
|
old_log = OLD / f"{stratum}-{compiler}-w{workers}.log"
|
||||||
|
old_tiles, old_whole, _old_stamp = parse(old_log)
|
||||||
|
clang_reference, clang_whole, _ = parse(OLD / f"{stratum}-clang-w1.log")
|
||||||
|
|
||||||
|
def difference(reference: dict, candidate: dict) -> tuple[int, int]:
|
||||||
|
keys = set(reference) | set(candidate)
|
||||||
|
mismatches = sum(reference.get(key) != candidate.get(key) for key in keys)
|
||||||
|
return mismatches, len(keys)
|
||||||
|
|
||||||
|
old_mismatches, old_keys = difference(old_tiles, tiles)
|
||||||
|
cross_mismatches, cross_keys = difference(clang_reference, tiles)
|
||||||
|
row = {
|
||||||
|
"stratum": stratum, "compiler": compiler, "workers": workers,
|
||||||
|
"source_stamp": stamp, "tiles": len(tiles),
|
||||||
|
"l104_same_cell_tile_mismatches": old_mismatches,
|
||||||
|
"l104_same_cell_key_count": old_keys,
|
||||||
|
"l104_same_cell_whole_hash_match": whole == old_whole,
|
||||||
|
"clang_w1_tile_mismatches": cross_mismatches,
|
||||||
|
"clang_w1_key_count": cross_keys,
|
||||||
|
"clang_w1_whole_hash_match": whole == clang_whole,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L121HashRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
row["tile_hashes"] = {"|".join(map(str, key)): value for key, value in tiles.items()}
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
for path in (*BINARIES.values(), DATA):
|
||||||
|
if not path.is_file():
|
||||||
|
raise SystemExit(f"missing input: {path}")
|
||||||
|
rows = []
|
||||||
|
for stratum in CAMERAS:
|
||||||
|
for compiler in BINARIES:
|
||||||
|
for workers in (1, 8, 12):
|
||||||
|
rows.append(run(stratum, compiler, workers))
|
||||||
|
(OUT / "runs-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
failures = [row for row in rows if row["l104_same_cell_tile_mismatches"]
|
||||||
|
or not row["l104_same_cell_whole_hash_match"]
|
||||||
|
or row["clang_w1_tile_mismatches"]
|
||||||
|
or not row["clang_w1_whole_hash_match"]]
|
||||||
|
summary = {"runs": [{key: value for key, value in row.items() if key != "tile_hashes"}
|
||||||
|
for row in rows], "failed_runs": len(failures)}
|
||||||
|
(OUT / "summary.json").write_text(json.dumps(summary, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 1 if failures else 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,285 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Five interleaved L121 apply-supplement pairs at 30 m/s."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import re
|
||||||
|
import statistics
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
HOST = Path("/home/jahni/VFHost")
|
||||||
|
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
HEADLESS = ROOT / "Saved/Linux/l108/bin/VoxelForgeCoreHeadless-l108-after-clang"
|
||||||
|
OUT = ROOT / "Saved/Linux/l121/movement/far-apply-supplement"
|
||||||
|
HOST_SAVED = HOST / "Saved/Linux/l121"
|
||||||
|
HOST_OUT = HOST / "Plugins/VoxelForge/Saved/Linux/l121/far-apply-supplement"
|
||||||
|
ACTIVE_DESIGN = "far_apply_supplement"
|
||||||
|
SPAWN_M = (19.166015625, -14.2001953125, -753.3740234375)
|
||||||
|
MOVING_SAMPLE = re.compile(r"^\[L79MovingSample\] (.*)$", re.M)
|
||||||
|
MOVING_SUMMARY = re.compile(r"^\[L79MovingSummary\] (.*)$", re.M)
|
||||||
|
FIELDS = re.compile(r"([a-zA-Z0-9_]+)=([^ ]+)")
|
||||||
|
|
||||||
|
|
||||||
|
def p99(values: list[float]) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
values.sort()
|
||||||
|
return values[max(0, math.ceil(0.99 * len(values)) - 1)]
|
||||||
|
|
||||||
|
|
||||||
|
def parse_events(path: Path) -> list[dict]:
|
||||||
|
events = []
|
||||||
|
if not path.is_file():
|
||||||
|
return events
|
||||||
|
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||||
|
if not line.startswith("{"):
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
events.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
return events
|
||||||
|
|
||||||
|
|
||||||
|
def summarize(rows: list[dict], phase: str) -> dict:
|
||||||
|
moving = [row for row in rows if row["phase"] == "move"]
|
||||||
|
dwell = [row for row in rows if row["phase"] == "dwell"]
|
||||||
|
def moving_mean(name: str) -> float | None:
|
||||||
|
values = [row[name] for row in moving if name in row]
|
||||||
|
return statistics.mean(values) if values else None
|
||||||
|
complete = next((row for row in dwell
|
||||||
|
if row["near_ready_pct"] >= 99.999
|
||||||
|
and row["far_hole_pct"] <= 0.001), None)
|
||||||
|
end = dwell[-1] if dwell else {}
|
||||||
|
return {
|
||||||
|
"moving_samples": len(moving),
|
||||||
|
"dwell_samples": len(dwell),
|
||||||
|
"near_ready_moving_mean_pct": moving_mean("near_ready_pct"),
|
||||||
|
"lod0_ready_moving_mean_pct": moving_mean("lod0_ready_pct"),
|
||||||
|
"lod1_ready_moving_mean_pct": moving_mean("lod1_ready_pct"),
|
||||||
|
"lod2_ready_moving_mean_pct": moving_mean("lod2_ready_pct"),
|
||||||
|
"far_holes_moving_mean_pct": moving_mean("far_hole_pct"),
|
||||||
|
"near_ready_dwell_end_pct": end.get("near_ready_pct"),
|
||||||
|
"far_holes_dwell_end_pct": end.get("far_hole_pct"),
|
||||||
|
"dwell_complete_after_stop_s": complete.get("since_stop_s") if complete else None,
|
||||||
|
"phase": phase,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def run_headless(condition: str, repetition: int) -> dict:
|
||||||
|
worker_count = 8
|
||||||
|
folder = OUT / condition / "headless"
|
||||||
|
folder.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = folder / f"fast-r{repetition}.log"
|
||||||
|
camera = ",".join(f"{value:.12f}" for value in SPAWN_M) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", str(HEADLESS), str(DATA),
|
||||||
|
"--world-assembler-prototype", "--stream-benchmark", "--stream-pipeline",
|
||||||
|
f"--workers={worker_count}", "--clip-radius=3", "--max-clip-level=8",
|
||||||
|
"--max-level=8", f"--camera={camera}", "--profile-lod-work",
|
||||||
|
"--stream-scheduler-policy=3", "--moving-camera-speed-mps=30.0",
|
||||||
|
"--moving-camera-duration-s=12", "--moving-camera-dwell-s=5",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
result = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
content = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
rows = []
|
||||||
|
for match in MOVING_SAMPLE.finditer(content):
|
||||||
|
row = dict(FIELDS.findall(match.group(1)))
|
||||||
|
for name in ("t_s", "near_ready_pct", "walkable_ring_ready_pct",
|
||||||
|
"far_hole_pct", "queue_update_ms"):
|
||||||
|
if name in row:
|
||||||
|
row[name] = float(row[name])
|
||||||
|
row["phase"] = row.get("phase", "move")
|
||||||
|
row["since_stop_s"] = max(0.0, row.get("t_s", 0.0) - 12.0)
|
||||||
|
rows.append(row)
|
||||||
|
engine_summary = {}
|
||||||
|
if match := MOVING_SUMMARY.search(content):
|
||||||
|
engine_summary = dict(FIELDS.findall(match.group(1)))
|
||||||
|
if result.returncode or not rows or int(engine_summary.get("generation_failures", "-1")) != 0:
|
||||||
|
raise RuntimeError(f"headless run failed: {log}; exit={result.returncode}")
|
||||||
|
moving_updates = [row["queue_update_ms"] for row in rows
|
||||||
|
if row["phase"] == "move" and "queue_update_ms" in row]
|
||||||
|
return {
|
||||||
|
"condition": condition, "platform": "headless", "speed_mps": 30.0,
|
||||||
|
"repetition": repetition, "worker_count": worker_count,
|
||||||
|
"exit_code": result.returncode,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"update_work_p99_ms": p99(moving_updates),
|
||||||
|
"update_work_max_ms": max(moving_updates, default=None),
|
||||||
|
"log": str(log.relative_to(ROOT)), **summarize(rows, "headless"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def run_unreal(condition: str, repetition: int) -> dict:
|
||||||
|
core_limit = 12
|
||||||
|
worker_cap = 1
|
||||||
|
far_apply_supplement = 0 if condition == "baseline" else 1
|
||||||
|
folder = OUT / condition / "unreal"
|
||||||
|
host_folder = HOST_OUT / condition / "output"
|
||||||
|
folder.mkdir(parents=True, exist_ok=True)
|
||||||
|
trace = folder / f"fast-r{repetition}.trace.jsonl"
|
||||||
|
abslog = folder / f"fast-r{repetition}.log"
|
||||||
|
stdout = folder / f"fast-r{repetition}.stdout.log"
|
||||||
|
export = host_folder / f"fast-r{repetition}"
|
||||||
|
userdir = HOST_SAVED / "user" / condition / f"fast-r{repetition}"
|
||||||
|
zendata = HOST_SAVED / "zen" / condition / f"fast-r{repetition}"
|
||||||
|
ddc = HOST_SAVED / "DDC"
|
||||||
|
for path in (export, userdir, zendata, ddc):
|
||||||
|
path.mkdir(parents=True, exist_ok=True)
|
||||||
|
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWN_M)
|
||||||
|
command = [
|
||||||
|
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
|
||||||
|
"-nop4", "-nosplash", "-NoSound", f"-corelimit={core_limit}",
|
||||||
|
"-NoZenAutoLaunch", "-notraceserver", "-nocrashreports",
|
||||||
|
"-DisablePlugins=AndroidFileServer", "-ddc=NoZenLocalFallback",
|
||||||
|
f"-userdir={userdir}", f"-ZenDataPath={zendata}",
|
||||||
|
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
|
||||||
|
"-archetype=TunnelNetwork", "-slot=0", "-modes=export", "-gameconfig=1",
|
||||||
|
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
|
||||||
|
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0",
|
||||||
|
"-coreplantest=0", "-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings",
|
||||||
|
"-corehybridmesher=0", f"-out={export}",
|
||||||
|
"-voxel.StreamingCoarseFirst=1", "-voxel.StreamingDynamicNearReserve=1",
|
||||||
|
"-voxel.StreamingLegacyPendingResultCap=0", "-voxel.CoreRegionConcurrentAdmission=1",
|
||||||
|
"-voxel.StreamingResultBacklogCap=256", "-voxel.StreamingReadyQueueDepth=8",
|
||||||
|
f"-voxel.StreamingBackgroundWorkerCap={worker_cap}",
|
||||||
|
"-voxel.CoreColumnCacheDirectMapped=0",
|
||||||
|
f"-voxel.StreamingFarApplySupplement={far_apply_supplement}",
|
||||||
|
"-voxel.CoreMCGradientMemoization=1", "-voxel.StreamingCollisionLookahead=1",
|
||||||
|
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=3000.0",
|
||||||
|
"-voxel.TestMoveStartSeconds=6.0", "-voxel.TestMoveDurationSeconds=12.0",
|
||||||
|
"-voxel.TestExitSeconds=23.0", "-voxel.TestMoveDirection=1,0,0",
|
||||||
|
"-voxel.StreamingSchedulerPolicy=3", "-voxel.TestExitOnSteady=0",
|
||||||
|
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
|
||||||
|
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
|
||||||
|
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
|
||||||
|
f"-abslog={abslog}",
|
||||||
|
]
|
||||||
|
wrapped = ["taskset", "-c", "0-11", "timeout", "1740s", *command]
|
||||||
|
started = time.monotonic()
|
||||||
|
with stdout.open("w", encoding="utf-8") as stream:
|
||||||
|
result = subprocess.run(wrapped, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
events = parse_events(trace)
|
||||||
|
samples = [event for event in events if event.get("type") == "streaming_sample"]
|
||||||
|
moving_samples = [event for event in samples
|
||||||
|
if float(event.get("speed_cm_s", 0.0)) >= 50.0]
|
||||||
|
move_start = min((float(event.get("t_s", 0.0)) for event in moving_samples),
|
||||||
|
default=float("inf"))
|
||||||
|
stop_event = next((event for event in events
|
||||||
|
if event.get("type") == "streaming_test_move_stopped"), None)
|
||||||
|
stop = float(stop_event.get("t_s", 0.0)) if stop_event else float("inf")
|
||||||
|
rows = []
|
||||||
|
for event in samples:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
resident = event.get("resident_by_lod", [0] * 9)
|
||||||
|
covered = event.get("covered_by_lod", resident)
|
||||||
|
near_desired = sum(desired[:3])
|
||||||
|
near_resident = sum(resident[:3])
|
||||||
|
far_desired = sum(desired[3:])
|
||||||
|
far_covered = sum(covered[3:])
|
||||||
|
t_s = float(event.get("t_s", 0.0))
|
||||||
|
speed = float(event.get("speed_cm_s", 0.0))
|
||||||
|
phase = "move" if speed >= 50.0 else ("dwell" if t_s >= stop else "arrival")
|
||||||
|
rows.append({
|
||||||
|
"phase": phase, "t_s": t_s,
|
||||||
|
"since_stop_s": max(0.0, t_s - stop) if phase == "dwell" else None,
|
||||||
|
"near_ready_pct": 100.0 * near_resident / max(1, near_desired),
|
||||||
|
"lod0_ready_pct": 100.0 * resident[0] / max(1, desired[0]),
|
||||||
|
"lod1_ready_pct": 100.0 * resident[1] / max(1, desired[1]),
|
||||||
|
"lod2_ready_pct": 100.0 * resident[2] / max(1, desired[2]),
|
||||||
|
"far_hole_pct": 100.0 * (far_desired - far_covered) / max(1, far_desired),
|
||||||
|
})
|
||||||
|
dwell_rows = [row for row in rows if row["phase"] == "dwell"]
|
||||||
|
complete = next((row for row in dwell_rows
|
||||||
|
if row["near_ready_pct"] >= 99.999
|
||||||
|
and row["far_hole_pct"] <= 0.001), None)
|
||||||
|
moving_work = [event for event in events
|
||||||
|
if event.get("type") == "game_thread_work_frame"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop]
|
||||||
|
ticks = [float(event.get("tick_ms", 0.0)) for event in moving_work]
|
||||||
|
updates = [float(event.get("update_chunks_ms", 0.0)) for event in moving_work]
|
||||||
|
backend = next((event for event in events
|
||||||
|
if event.get("type") == "worker_backend_config"), {})
|
||||||
|
worker_cap_readback = next((event for event in events
|
||||||
|
if event.get("type") == "streaming_worker_cap_readback"), {})
|
||||||
|
if result.returncode or len(moving_samples) < 40 or len(dwell_rows) < 16:
|
||||||
|
raise RuntimeError(f"Unreal run failed/incomplete: {trace}; exit={result.returncode}; "
|
||||||
|
f"move={len(moving_samples)} dwell={len(dwell_rows)}")
|
||||||
|
expected_far_supplement = 0 if condition == "baseline" else 1
|
||||||
|
expected_near_priority = "BackgroundLow"
|
||||||
|
if (backend.get("near_priority") != expected_near_priority
|
||||||
|
or backend.get("horizon_priority") != "BackgroundLow"
|
||||||
|
or int(worker_cap_readback.get("streaming_background_worker_cap", -1)) != worker_cap
|
||||||
|
or int(backend.get("streaming_far_apply_supplement", -1)) != expected_far_supplement
|
||||||
|
or int(backend.get("effective_tile_slots", -1)) != (8 if worker_cap else 10)):
|
||||||
|
raise RuntimeError(f"priority CVar readback mismatch in {trace}: {backend}")
|
||||||
|
return {
|
||||||
|
"condition": condition, "platform": "unreal", "speed_mps": 30.0,
|
||||||
|
"repetition": repetition, "core_limit": core_limit,
|
||||||
|
"effective_tile_slots": backend.get("effective_tile_slots"),
|
||||||
|
"streaming_background_worker_cap": worker_cap_readback.get(
|
||||||
|
"streaming_background_worker_cap"),
|
||||||
|
"taskgraph_background_threads": backend.get("taskgraph_background_threads"),
|
||||||
|
"exit_code": result.returncode,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"game_thread_tick_p99_ms": p99(ticks),
|
||||||
|
"game_thread_tick_max_ms": max(ticks, default=None),
|
||||||
|
"stream_update_p99_ms": p99(updates),
|
||||||
|
"stream_update_max_ms": max(updates, default=None),
|
||||||
|
"dwell_complete_after_stop_s": complete.get("since_stop_s") if complete else None,
|
||||||
|
"moving_samples": len(moving_samples), "dwell_samples": len(dwell_rows),
|
||||||
|
"trace": str(trace.relative_to(ROOT)),
|
||||||
|
"log": str(abslog.relative_to(ROOT)),
|
||||||
|
**summarize(rows, "unreal"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
global ACTIVE_DESIGN, OUT, HOST_SAVED, HOST_OUT
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--platform", choices=("all", "headless", "unreal"), default="all")
|
||||||
|
parser.add_argument("--design", choices=("far_apply_supplement",),
|
||||||
|
default="far_apply_supplement")
|
||||||
|
args = parser.parse_args()
|
||||||
|
ACTIVE_DESIGN = args.design
|
||||||
|
design_path = "far-apply-supplement"
|
||||||
|
OUT = ROOT / "Saved/Linux/l121/movement" / design_path
|
||||||
|
HOST_SAVED = HOST / "Saved/Linux/l121" / design_path
|
||||||
|
HOST_OUT = HOST / "Plugins/VoxelForge/Saved/Linux/l121" / design_path
|
||||||
|
if not DATA.is_file() or not HEADLESS.is_file() or not EDITOR.is_file():
|
||||||
|
raise SystemExit("missing owner data, L108 headless binary, or Unreal editor")
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
results = []
|
||||||
|
partial = OUT / "runs-partial.json"
|
||||||
|
for repetition in range(1, 6):
|
||||||
|
candidate = "far_supplement"
|
||||||
|
conditions = ("baseline", candidate) if repetition % 2 else (candidate, "baseline")
|
||||||
|
platforms = (("headless", "unreal") if repetition % 2 else ("unreal", "headless")) \
|
||||||
|
if args.platform == "all" else (args.platform,)
|
||||||
|
for condition in conditions:
|
||||||
|
for platform in platforms:
|
||||||
|
row = (run_headless(condition, repetition) if platform == "headless"
|
||||||
|
else run_unreal(condition, repetition))
|
||||||
|
results.append(row)
|
||||||
|
partial.write_text(json.dumps(results, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print("[L121Movement] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
design_description = "one supplemental far result after the normal four applies, with the 2 ms time budget"
|
||||||
|
(OUT / "summary.json").write_text(
|
||||||
|
json.dumps({"design": design_description,
|
||||||
|
"runs": results}, indent=2) + "\n", encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,198 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Summarize L122 scheduler blockers and Core region compile timing."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import statistics
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
TRACE_DIR = ROOT / "Saved/Linux/l122/diagnostic/unreal"
|
||||||
|
OUT = ROOT / "Saved/Linux/l122/diagnostic/analysis.json"
|
||||||
|
|
||||||
|
|
||||||
|
def read_events(path: Path) -> list[dict]:
|
||||||
|
rows = []
|
||||||
|
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||||
|
if line.startswith("{"):
|
||||||
|
try:
|
||||||
|
rows.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
continue
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def percentile(values: list[float], fraction: float) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
ordered = sorted(values)
|
||||||
|
index = max(0, min(len(ordered) - 1, math.ceil(len(ordered) * fraction) - 1))
|
||||||
|
return ordered[index]
|
||||||
|
|
||||||
|
|
||||||
|
def distribution(values: list[float]) -> dict:
|
||||||
|
if not values:
|
||||||
|
return {"n": 0, "p50": None, "p95": None, "max": None}
|
||||||
|
return {
|
||||||
|
"n": len(values), "p50": statistics.median(values),
|
||||||
|
"p95": percentile(values, 0.95), "max": max(values),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def summarize_run(path: Path) -> tuple[dict, list[dict], list[dict]]:
|
||||||
|
events = read_events(path)
|
||||||
|
stop = next(float(event["t_s"]) for event in events
|
||||||
|
if event.get("type") == "streaming_test_move_stopped")
|
||||||
|
move_start = min(float(event["t_s"]) for event in events
|
||||||
|
if event.get("type") == "streaming_sample"
|
||||||
|
and float(event.get("speed_cm_s", 0.0)) >= 50.0)
|
||||||
|
scheduler = [event for event in events if event.get("type") == "scheduler_admission"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop
|
||||||
|
and event.get("moving", 0)]
|
||||||
|
idle = [event for event in scheduler
|
||||||
|
if event.get("running_workers", 0) == 0
|
||||||
|
and sum(event.get("desired_missing_by_lod", [0] * 9)[:3])
|
||||||
|
+ sum(event.get("pending_by_lod", [0] * 9)[:3]) > 0]
|
||||||
|
samples = [event for event in events if event.get("type") == "streaming_sample"
|
||||||
|
and float(event.get("speed_cm_s", 0.0)) >= 50.0]
|
||||||
|
near = []
|
||||||
|
far_holes = []
|
||||||
|
for event in samples:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
resident = event.get("resident_by_lod", [0] * 9)
|
||||||
|
covered = event.get("covered_by_lod", resident)
|
||||||
|
near.append(100.0 * sum(resident[:3]) / max(1, sum(desired[:3])))
|
||||||
|
far_holes.append(100.0 * (sum(desired[3:]) - sum(covered[3:]))
|
||||||
|
/ max(1, sum(desired[3:])))
|
||||||
|
ticks = [float(event.get("tick_ms", 0.0)) for event in events
|
||||||
|
if event.get("type") == "game_thread_work_frame"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop]
|
||||||
|
|
||||||
|
def near_array(event: dict, name: str) -> list[int]:
|
||||||
|
return event.get(name, [0] * 9)[:3]
|
||||||
|
|
||||||
|
def near_count(event: dict, name: str) -> int:
|
||||||
|
return sum(near_array(event, name))
|
||||||
|
|
||||||
|
reason_counts = {
|
||||||
|
"core_region_pending": sum(event.get("core_region_compile_pending", 0) > 0
|
||||||
|
for event in idle),
|
||||||
|
"core_region_exact_near_decline": sum(
|
||||||
|
sum(near_array(event, "region_compile_declines_by_lod")) > 0 for event in idle),
|
||||||
|
"far_parent_near_decline": sum(
|
||||||
|
sum(near_array(event, "far_parent_declines_by_lod")) > 0 for event in idle),
|
||||||
|
"near_cap_decline": sum(any(value in (1, 2)
|
||||||
|
for value in event.get("declined_rule_by_lod", [0] * 9)[:3])
|
||||||
|
for event in idle),
|
||||||
|
"critical_set_not_ready": sum(event.get("critical_tiles_ready", 1) == 0
|
||||||
|
for event in idle),
|
||||||
|
"walkable_ring_not_ready": sum(event.get("walkable_ready", 1) == 0 for event in idle),
|
||||||
|
"pawn_collision_gate_held": sum(event.get("pawn_collision_gate_engaged", 0) > 0
|
||||||
|
for event in idle),
|
||||||
|
"desired_center_stale": sum(event.get("desired_center_current", 1) == 0 for event in idle),
|
||||||
|
"only_pending_near_work": sum(near_count(event, "desired_missing_by_lod") == 0
|
||||||
|
and near_count(event, "pending_by_lod") > 0 for event in idle),
|
||||||
|
"no_near_admission_this_frame": sum(
|
||||||
|
near_count(event, "submitted_by_lod") == 0 for event in idle),
|
||||||
|
}
|
||||||
|
|
||||||
|
compiles = [event for event in events if event.get("type") == "core_region_compile_complete"]
|
||||||
|
requests = {event.get("request_serial"): event for event in events
|
||||||
|
if event.get("type") == "core_region_compile_request"}
|
||||||
|
starts = [event for event in events if event.get("type") == "core_region_compile_start"]
|
||||||
|
request_start = [
|
||||||
|
(float(event["t_s"]) - float(requests[event["request_serial"]]["t_s"])) * 1000.0
|
||||||
|
for event in starts if event.get("request_serial") in requests
|
||||||
|
]
|
||||||
|
near_missing = [near_count(event, "desired_missing_by_lod") for event in idle]
|
||||||
|
near_pending = [near_count(event, "pending_by_lod") for event in idle]
|
||||||
|
occupancy = [float(event.get("admission_occupancy", 0)) for event in idle]
|
||||||
|
row = {
|
||||||
|
"trace": str(path.relative_to(ROOT)),
|
||||||
|
"moving_scheduler_samples": len(scheduler),
|
||||||
|
"idle_missing_near_windows": len(idle),
|
||||||
|
"idle_window_share_pct": 100.0 * len(idle) / max(1, len(scheduler)),
|
||||||
|
"near_ready_moving_mean_pct": statistics.mean(near) if near else None,
|
||||||
|
"far_holes_moving_mean_pct": statistics.mean(far_holes) if far_holes else None,
|
||||||
|
"tick_p99_ms": percentile(ticks, 0.99), "tick_max_ms": max(ticks, default=None),
|
||||||
|
"idle_near_unadmitted_mean": statistics.mean(near_missing) if near_missing else None,
|
||||||
|
"idle_near_pending_mean": statistics.mean(near_pending) if near_pending else None,
|
||||||
|
"idle_admission_occupancy_p95": percentile(occupancy, 0.95),
|
||||||
|
"idle_admission_occupancy_max": max(occupancy, default=None),
|
||||||
|
"idle_critical_missing_p50": statistics.median(
|
||||||
|
[float(event.get("critical_missing", 0)) for event in idle]) if idle else None,
|
||||||
|
"idle_reasons": reason_counts,
|
||||||
|
"region_compile_request_start_ms": distribution(request_start),
|
||||||
|
"region_compile_task_queue_ms": distribution(
|
||||||
|
[float(event.get("compile_queue_ms", 0)) for event in compiles]),
|
||||||
|
"region_compile_task_runtime_ms": distribution(
|
||||||
|
[float(event.get("compile_task_ms", 0)) for event in compiles]),
|
||||||
|
"region_compile_core_wall_ms": distribution(
|
||||||
|
[float(event.get("compile_core_wall_ms", 0)) for event in compiles]),
|
||||||
|
"region_compile_pending_gate_ms": distribution(
|
||||||
|
[float(event.get("pending_gate_elapsed_ms", 0)) for event in compiles]),
|
||||||
|
"region_compiled_tile_batches": sum(event.get("compiled_tiles", 0) > 0
|
||||||
|
for event in compiles),
|
||||||
|
"region_compile_worker_thread_count": len({event.get("compile_worker_thread_id")
|
||||||
|
for event in compiles}),
|
||||||
|
"region_compile_submitted_while_generation_idle": sum(
|
||||||
|
event.get("active_generation_workers", -1) == 0 for event in starts),
|
||||||
|
"region_compile_requests": len(requests),
|
||||||
|
"region_compile_starts": len(starts),
|
||||||
|
"region_compile_completions": len(compiles),
|
||||||
|
}
|
||||||
|
return row, idle, compiles
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
traces = sorted(TRACE_DIR.glob("fast-r*.trace.jsonl"))
|
||||||
|
if len(traces) != 5:
|
||||||
|
raise SystemExit(f"expected five diagnostic traces, found {len(traces)} in {TRACE_DIR}")
|
||||||
|
runs = [summarize_run(path) for path in traces]
|
||||||
|
rows = [run[0] for run in runs]
|
||||||
|
idles = [event for run in runs for event in run[1]]
|
||||||
|
compiles = [event for run in runs for event in run[2]]
|
||||||
|
total_scheduler = sum(row["moving_scheduler_samples"] for row in rows)
|
||||||
|
reason_names = rows[0]["idle_reasons"].keys()
|
||||||
|
summary = {
|
||||||
|
"runs": rows,
|
||||||
|
"pooled_idle_windows": len(idles),
|
||||||
|
"pooled_scheduler_samples": total_scheduler,
|
||||||
|
"pooled_idle_window_share_pct": 100.0 * len(idles) / max(1, total_scheduler),
|
||||||
|
"pooled_idle_reasons": {
|
||||||
|
reason: sum(row["idle_reasons"][reason] for row in rows)
|
||||||
|
for reason in reason_names
|
||||||
|
},
|
||||||
|
"pooled_near_ready_moving_mean_pct": statistics.mean(
|
||||||
|
row["near_ready_moving_mean_pct"] for row in rows),
|
||||||
|
"pooled_tick_p99_ms": distribution(
|
||||||
|
[float(row["tick_p99_ms"]) for row in rows if row["tick_p99_ms"] is not None]),
|
||||||
|
"pooled_tick_max_ms": max(float(row["tick_max_ms"]) for row in rows
|
||||||
|
if row["tick_max_ms"] is not None),
|
||||||
|
"pooled_region_compile_task_queue_ms": distribution(
|
||||||
|
[float(event.get("compile_queue_ms", 0)) for event in compiles]),
|
||||||
|
"pooled_region_compile_task_runtime_ms": distribution(
|
||||||
|
[float(event.get("compile_task_ms", 0)) for event in compiles]),
|
||||||
|
"pooled_region_compile_core_wall_ms": distribution(
|
||||||
|
[float(event.get("compile_core_wall_ms", 0)) for event in compiles]),
|
||||||
|
"pooled_region_compile_pending_gate_ms": distribution(
|
||||||
|
[float(event.get("pending_gate_elapsed_ms", 0)) for event in compiles]),
|
||||||
|
"pooled_region_compiled_tile_batches": sum(
|
||||||
|
event.get("compiled_tiles", 0) > 0 for event in compiles),
|
||||||
|
"pooled_region_compile_worker_thread_count": len({
|
||||||
|
event.get("compile_worker_thread_id") for event in compiles}),
|
||||||
|
"pooled_region_compile_submitted_while_generation_idle": sum(
|
||||||
|
event.get("active_generation_workers", -1) == 0
|
||||||
|
for run in runs for event in read_events(ROOT / run[0]["trace"])
|
||||||
|
if event.get("type") == "core_region_compile_start"),
|
||||||
|
"pooled_region_compile_requests": sum(row["region_compile_requests"] for row in rows),
|
||||||
|
}
|
||||||
|
OUT.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
OUT.write_text(json.dumps(summary, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print(json.dumps(summary, indent=2))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
# Linux 122 — idle workers at speed
|
||||||
|
|
||||||
|
**Decision:** keep the L121 admission policy. Neither tested fast-travel relaxation met every coverage and tick limit.
|
||||||
|
|
||||||
|
Five diagnostic runs at 30 m/s had 686 idle-worker windows with near tiles missing. In 673 (98%) the expanded critical set was not ready; 462 (67%) also had Core-region compilation pending, and 441 (64%) declined an exact region-dependent near tile. These counts overlap. Far-parent, near-cap, walkable-ring, pawn-collision, and stale-center holds were all zero. Only 13 windows had pending near work and no unadmitted near tile. The desired center was current in every window.
|
||||||
|
|
||||||
|
Region compile requests scheduled immediately (request to task submission p50/p95 **0.008/0.021 ms**). UE task queue delay was **0.122/14.856 ms** p50/p95 (max 54.4 ms); task runtime was **0.485/4.572 ms**, with Core compile wall time **0.259/3.679 ms**. The task runs on UE background workers, not the Game Thread. It was submitted while generation workers were idle in 680/1,158 requests. The compile targets the current desired center; there is no velocity look-ahead. The completion gate remained pending **18.216/21.136 ms** p50/p95 while the Game Thread observed completion.
|
||||||
|
|
||||||
|
The UE/headless coverage gap is not a denominator mismatch: both use resident near tiles / desired LOD0–2 tiles, with about 1,346–1,348 desired near tiles in UE versus 1,344 in the saved headless runs. Headless was about 82.6%; UE was 45.0%. The measured idle windows point to UE admission. Apply work is small (L121 near apply and proxy p95 were about 0.20 and 0.24 ms), though worker-finish-to-apply wait reached 120 ms p95 and may add secondary delay.
|
||||||
|
|
||||||
|
| Fast-travel design | Near change | Far-hole change | Paired p99 change | Worst tick |
|
||||||
|
|---|---:|---:|---:|---:|
|
||||||
|
| Admit normal near/far shares before expanded critical readiness | +15.42 pp | −0.28 pp | +0.28 ms | 14.52 ms |
|
||||||
|
| Admit near LOD0–2 only until critical readiness | +9.89 pp | +1.48 pp | −0.56 ms | 14.38 ms |
|
||||||
|
|
||||||
|
The first design exceeded the p99 bar (worst per-run p99 13.36 ms versus L121's 11.00 ms); the second increased far holes. Both stayed under L121's 14.74 ms worst-tick ceiling. No admission behavior was retained, so arrival behavior remains L121.
|
||||||
|
|
||||||
|
UE build passed with `-NoUBA -MaxParallelActions=4`. Exact tile and whole-view hashes matched L104 in **24/24** Clang/GCC W1/W8/W12 runs. Fast tier passed (**269 gates, 5 diagnostics**). Logs are under `Saved/Linux/l122`.
|
||||||
@@ -0,0 +1,111 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Capture five 30 m/s UE runs with admission-reason tracing enabled."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
HOST = Path("/home/jahni/VFHost")
|
||||||
|
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
|
||||||
|
OUT = ROOT / "Saved/Linux/l122/diagnostic/unreal"
|
||||||
|
HOST_OUT = HOST / "Plugins/VoxelForge/Saved/Linux/l122/diagnostic"
|
||||||
|
HOST_SAVED = HOST / "Saved/Linux/l122"
|
||||||
|
SPAWN_M = (19.166015625, -14.2001953125, -753.3740234375)
|
||||||
|
|
||||||
|
|
||||||
|
def run(repetition: int) -> dict:
|
||||||
|
trace = OUT / f"fast-r{repetition}.trace.jsonl"
|
||||||
|
abslog = OUT / f"fast-r{repetition}.log"
|
||||||
|
stdout = OUT / f"fast-r{repetition}.stdout.log"
|
||||||
|
export = HOST_OUT / f"fast-r{repetition}"
|
||||||
|
userdir = HOST_SAVED / "user" / f"fast-r{repetition}"
|
||||||
|
zendata = HOST_SAVED / "zen" / f"fast-r{repetition}"
|
||||||
|
ddc = HOST_SAVED / "DDC"
|
||||||
|
for path in (export, userdir, zendata, ddc):
|
||||||
|
path.mkdir(parents=True, exist_ok=True)
|
||||||
|
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWN_M)
|
||||||
|
command = [
|
||||||
|
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
|
||||||
|
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
|
||||||
|
"-notraceserver", "-nocrashreports", "-DisablePlugins=AndroidFileServer",
|
||||||
|
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
|
||||||
|
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
|
||||||
|
"-archetype=TunnelNetwork", "-slot=0", "-modes=export", "-gameconfig=1",
|
||||||
|
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
|
||||||
|
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0", "-coreplantest=0",
|
||||||
|
"-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings", "-corehybridmesher=0",
|
||||||
|
f"-out={export}", "-voxel.StreamingCoarseFirst=1",
|
||||||
|
"-voxel.StreamingDynamicNearReserve=1",
|
||||||
|
"-voxel.StreamingLegacyPendingResultCap=0",
|
||||||
|
"-voxel.CoreRegionConcurrentAdmission=1", "-voxel.StreamingResultBacklogCap=256",
|
||||||
|
"-voxel.StreamingReadyQueueDepth=8", "-voxel.StreamingBackgroundWorkerCap=1",
|
||||||
|
"-voxel.CoreColumnCacheDirectMapped=0", "-voxel.StreamingFarApplySupplement=1",
|
||||||
|
"-voxel.CoreMCGradientMemoization=1", "-voxel.StreamingCollisionLookahead=1",
|
||||||
|
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=3000.0",
|
||||||
|
"-voxel.TestMoveStartSeconds=6.0", "-voxel.TestMoveDurationSeconds=12.0",
|
||||||
|
"-voxel.TestExitSeconds=23.0", "-voxel.TestMoveDirection=1,0,0",
|
||||||
|
"-voxel.StreamingSchedulerPolicy=3", "-voxel.TestExitOnSteady=0",
|
||||||
|
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
|
||||||
|
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
|
||||||
|
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
|
||||||
|
f"-abslog={abslog}",
|
||||||
|
]
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
wrapped = ["taskset", "-c", "0-11", "timeout", "1740s", *command]
|
||||||
|
started = time.monotonic()
|
||||||
|
with stdout.open("w", encoding="utf-8") as stream:
|
||||||
|
result = subprocess.run(wrapped, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
events = []
|
||||||
|
if trace.is_file():
|
||||||
|
for line in trace.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||||
|
if line.startswith("{"):
|
||||||
|
try:
|
||||||
|
events.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
continue
|
||||||
|
moving = [event for event in events if event.get("type") == "streaming_sample"
|
||||||
|
and float(event.get("speed_cm_s", 0.0)) >= 50.0]
|
||||||
|
dwell = [event for event in events if event.get("type") == "streaming_sample"
|
||||||
|
and float(event.get("speed_cm_s", 0.0)) < 50.0
|
||||||
|
and float(event.get("t_s", 0.0)) >= 18.0]
|
||||||
|
backend = next((event for event in events if event.get("type") == "worker_backend_config"), {})
|
||||||
|
cap = next((event for event in events if event.get("type") == "streaming_worker_cap_readback"), {})
|
||||||
|
if result.returncode or len(moving) < 40 or len(dwell) < 16:
|
||||||
|
raise RuntimeError(f"incomplete Unreal run {repetition}: exit={result.returncode}, "
|
||||||
|
f"move={len(moving)}, dwell={len(dwell)}, trace={trace}")
|
||||||
|
if (int(backend.get("streaming_far_apply_supplement", -1)) != 1
|
||||||
|
or int(cap.get("streaming_background_worker_cap", -1)) != 1
|
||||||
|
or int(backend.get("effective_tile_slots", -1)) != 8):
|
||||||
|
raise RuntimeError(f"configuration readback mismatch: {backend}; {cap}")
|
||||||
|
if not any(event.get("type") == "scheduler_admission"
|
||||||
|
and "region_compile_declines_by_lod" in event
|
||||||
|
for event in events):
|
||||||
|
raise RuntimeError(f"admission diagnostics missing from {trace}")
|
||||||
|
row = {
|
||||||
|
"platform": "unreal", "phase": "diagnostic_baseline", "repetition": repetition,
|
||||||
|
"exit_code": result.returncode, "wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"moving_samples": len(moving), "dwell_samples": len(dwell),
|
||||||
|
"trace": str(trace.relative_to(ROOT)), "log": str(abslog.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
with (OUT / "runs.jsonl").open("a", encoding="utf-8") as stream:
|
||||||
|
stream.write(json.dumps(row, sort_keys=True) + "\n")
|
||||||
|
print("[L122Admission] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
if not (ROOT / "Standalone/data/owner_world.json").is_file() or not EDITOR.is_file():
|
||||||
|
raise SystemExit("missing owner world data or Unreal editor")
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
runs = [run(repetition) for repetition in range(1, 6)]
|
||||||
|
(OUT / "summary.json").write_text(json.dumps({"runs": runs}, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,117 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Compare L122 exact tile hashes with L104 across all strata, compilers and workers."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
OUT = ROOT / "Saved/Linux/l122/hashes"
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
OLD = ROOT / "Saved/Linux/l104/hashes"
|
||||||
|
BINARIES = {
|
||||||
|
"clang": ROOT / "Saved/Linux/l122/verify/bin/VoxelForgeCoreHeadless-verify-clang",
|
||||||
|
"gcc": ROOT / "Saved/Linux/l122/verify/bin/VoxelForgeCoreHeadless-verify-gcc",
|
||||||
|
}
|
||||||
|
CAMERAS = {
|
||||||
|
"sw": ((8.0, 16.0, -372.2626953125), 2850),
|
||||||
|
"fi": ((25.8369140625, 3.19921875, -737.0), 2249),
|
||||||
|
"tn": ((19.166015625, -14.2001953125, -753.3740234375), 2233),
|
||||||
|
"vs": ((-20.0625, -5.068359375, -900.353515625), 2329),
|
||||||
|
}
|
||||||
|
TILE = re.compile(
|
||||||
|
r"^\[PipelineTileHash\] lod=(-?\d+) x=(-?\d+) y=(-?\d+) z=(-?\d+) "
|
||||||
|
r"density=(0x[0-9a-f]+) mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) "
|
||||||
|
r"topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
WHOLE = re.compile(
|
||||||
|
r"^\[PipelineWholeViewHash\] tiles=(\d+) density=(0x[0-9a-f]+) "
|
||||||
|
r"mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M)
|
||||||
|
|
||||||
|
|
||||||
|
def parse(path: Path) -> tuple[dict, tuple[str, ...], str]:
|
||||||
|
content = path.read_text(encoding="utf-8", errors="replace")
|
||||||
|
tiles = {(int(m.group(1)), int(m.group(2)), int(m.group(3)), int(m.group(4))):
|
||||||
|
m.group(5, 6, 7, 8) for m in TILE.finditer(content)}
|
||||||
|
whole, stamp = WHOLE.search(content), STAMP.search(content)
|
||||||
|
if whole is None or stamp is None or len(tiles) != int(whole.group(1)):
|
||||||
|
raise RuntimeError(f"incomplete hash output: {path}")
|
||||||
|
return tiles, whole.groups(), stamp.group(1)
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, compiler: str, workers: int) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = OUT / f"{stratum}-{compiler}-w{workers}.log"
|
||||||
|
camera, expected_tiles = CAMERAS[stratum]
|
||||||
|
camera_arg = ",".join(f"{v:.12f}" for v in camera) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--", "timeout", "1740s",
|
||||||
|
str(BINARIES[compiler]), str(DATA), "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", f"--workers={workers}",
|
||||||
|
"--clip-radius=3", "--max-clip-level=8", "--max-level=8",
|
||||||
|
f"--camera={camera_arg}", "--stream-hash-tiles",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
if process.returncode:
|
||||||
|
raise RuntimeError(f"hash process exited {process.returncode}: {log}")
|
||||||
|
tiles, whole, stamp = parse(log)
|
||||||
|
if len(tiles) != expected_tiles:
|
||||||
|
raise RuntimeError(f"expected {expected_tiles} tiles, got {len(tiles)}: {log}")
|
||||||
|
old_log = OLD / f"{stratum}-{compiler}-w{workers}.log"
|
||||||
|
old_tiles, old_whole, _old_stamp = parse(old_log)
|
||||||
|
clang_reference, clang_whole, _ = parse(OLD / f"{stratum}-clang-w1.log")
|
||||||
|
|
||||||
|
def difference(reference: dict, candidate: dict) -> tuple[int, int]:
|
||||||
|
keys = set(reference) | set(candidate)
|
||||||
|
mismatches = sum(reference.get(key) != candidate.get(key) for key in keys)
|
||||||
|
return mismatches, len(keys)
|
||||||
|
|
||||||
|
old_mismatches, old_keys = difference(old_tiles, tiles)
|
||||||
|
cross_mismatches, cross_keys = difference(clang_reference, tiles)
|
||||||
|
row = {
|
||||||
|
"stratum": stratum, "compiler": compiler, "workers": workers,
|
||||||
|
"source_stamp": stamp, "tiles": len(tiles),
|
||||||
|
"l104_same_cell_tile_mismatches": old_mismatches,
|
||||||
|
"l104_same_cell_key_count": old_keys,
|
||||||
|
"l104_same_cell_whole_hash_match": whole == old_whole,
|
||||||
|
"clang_w1_tile_mismatches": cross_mismatches,
|
||||||
|
"clang_w1_key_count": cross_keys,
|
||||||
|
"clang_w1_whole_hash_match": whole == clang_whole,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L122HashRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
row["tile_hashes"] = {"|".join(map(str, key)): value for key, value in tiles.items()}
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
for path in (*BINARIES.values(), DATA):
|
||||||
|
if not path.is_file():
|
||||||
|
raise SystemExit(f"missing input: {path}")
|
||||||
|
rows = []
|
||||||
|
for stratum in CAMERAS:
|
||||||
|
for compiler in BINARIES:
|
||||||
|
for workers in (1, 8, 12):
|
||||||
|
rows.append(run(stratum, compiler, workers))
|
||||||
|
(OUT / "runs-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
failures = [row for row in rows if row["l104_same_cell_tile_mismatches"]
|
||||||
|
or not row["l104_same_cell_whole_hash_match"]
|
||||||
|
or row["clang_w1_tile_mismatches"]
|
||||||
|
or not row["clang_w1_whole_hash_match"]]
|
||||||
|
summary = {"runs": [{key: value for key, value in row.items() if key != "tile_hashes"}
|
||||||
|
for row in rows], "failed_runs": len(failures)}
|
||||||
|
(OUT / "summary.json").write_text(json.dumps(summary, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 1 if failures else 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,298 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Archived five-pair harness for transient L122 fast-travel admission candidates.
|
||||||
|
|
||||||
|
The candidate CVar branches were removed after neither design met the quality gate.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import statistics
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
HOST = Path("/home/jahni/VFHost")
|
||||||
|
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
|
||||||
|
SPAWN_TN_M = (19.166015625, -14.2001953125, -753.3740234375)
|
||||||
|
SPAWN_FI_M = (25.8369140625, 3.19921875, -737.0)
|
||||||
|
DESIGN = "relax_expanded_critical_gate"
|
||||||
|
NEAR_ONLY_DESIGN = "near_only_expanded_critical_gate"
|
||||||
|
|
||||||
|
|
||||||
|
def percentile(values: list[float], fraction: float) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
ordered = sorted(values)
|
||||||
|
return ordered[max(0, min(len(ordered) - 1, math.ceil(len(ordered) * fraction) - 1))]
|
||||||
|
|
||||||
|
|
||||||
|
def read_events(path: Path) -> list[dict]:
|
||||||
|
rows = []
|
||||||
|
if path.is_file():
|
||||||
|
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||||
|
if not line.startswith("{"):
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
rows.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
continue
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def run(scenario: str, condition: str, repetition: int, design: str) -> dict:
|
||||||
|
candidate = condition == design
|
||||||
|
wait_for_critical = 0 if candidate else 1
|
||||||
|
near_only_while_critical_pending = int(candidate and design == NEAR_ONLY_DESIGN)
|
||||||
|
if scenario == "movement":
|
||||||
|
speed_cm_s, start_s, duration_s, exit_s, slot = 3000.0, 6.0, 12.0, 23.0, 0
|
||||||
|
spawn = SPAWN_TN_M
|
||||||
|
else:
|
||||||
|
speed_cm_s, start_s, duration_s, exit_s, slot = 140.0, 20.0, 12.0, 37.0, 1
|
||||||
|
spawn = SPAWN_FI_M
|
||||||
|
|
||||||
|
output_name = scenario + ("-near-only" if design == NEAR_ONLY_DESIGN else "")
|
||||||
|
root_out = ROOT / f"Saved/Linux/l122/{output_name}"
|
||||||
|
out = root_out / condition / "unreal"
|
||||||
|
host_out = HOST / "Plugins/VoxelForge/Saved/Linux/l122" / output_name / condition
|
||||||
|
host_saved = HOST / "Saved/Linux/l122"
|
||||||
|
label = f"{scenario}-r{repetition}"
|
||||||
|
trace = out / f"{label}.trace.jsonl"
|
||||||
|
abslog = out / f"{label}.log"
|
||||||
|
stdout = out / f"{label}.stdout.log"
|
||||||
|
export = host_out / "output" / label
|
||||||
|
userdir = host_saved / "user" / scenario / condition / label
|
||||||
|
zendata = host_saved / "zen" / scenario / condition / label
|
||||||
|
ddc = host_saved / "DDC"
|
||||||
|
for directory in (out, export, userdir, zendata, ddc):
|
||||||
|
directory.mkdir(parents=True, exist_ok=True)
|
||||||
|
trace.unlink(missing_ok=True)
|
||||||
|
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in spawn)
|
||||||
|
command = [
|
||||||
|
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
|
||||||
|
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
|
||||||
|
"-notraceserver", "-nocrashreports", "-DisablePlugins=AndroidFileServer",
|
||||||
|
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
|
||||||
|
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
|
||||||
|
"-archetype=TunnelNetwork", f"-slot={slot}", "-modes=export", "-gameconfig=1",
|
||||||
|
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
|
||||||
|
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0", "-coreplantest=0",
|
||||||
|
"-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings", "-corehybridmesher=0",
|
||||||
|
f"-out={export}", "-voxel.StreamingCoarseFirst=1",
|
||||||
|
"-voxel.StreamingDynamicNearReserve=1",
|
||||||
|
"-voxel.StreamingLegacyPendingResultCap=0",
|
||||||
|
"-voxel.CoreRegionConcurrentAdmission=1", "-voxel.StreamingResultBacklogCap=256",
|
||||||
|
"-voxel.StreamingReadyQueueDepth=8", "-voxel.StreamingBackgroundWorkerCap=1",
|
||||||
|
"-voxel.CoreColumnCacheDirectMapped=0", "-voxel.StreamingFarApplySupplement=1",
|
||||||
|
f"-voxel.StreamingFastTravelWaitForAllCriticalTiles={wait_for_critical}",
|
||||||
|
f"-voxel.StreamingFastTravelNearOnlyWhileCriticalPending={near_only_while_critical_pending}",
|
||||||
|
"-voxel.CoreMCGradientMemoization=1", "-voxel.StreamingCollisionLookahead=1",
|
||||||
|
"-voxel.TestMove=1", f"-voxel.TestMoveSpeedCmPerSecond={speed_cm_s:.1f}",
|
||||||
|
f"-voxel.TestMoveStartSeconds={start_s:.1f}",
|
||||||
|
f"-voxel.TestMoveDurationSeconds={duration_s:.1f}",
|
||||||
|
f"-voxel.TestExitSeconds={exit_s:.1f}", "-voxel.TestMoveDirection=1,0,0",
|
||||||
|
"-voxel.StreamingSchedulerPolicy=3", "-voxel.TestExitOnSteady=0",
|
||||||
|
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
|
||||||
|
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
|
||||||
|
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
|
||||||
|
f"-abslog={abslog}",
|
||||||
|
]
|
||||||
|
wrapped = ["taskset", "-c", "0-11", "timeout", "1740s", *command]
|
||||||
|
started = time.monotonic()
|
||||||
|
with stdout.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(wrapped, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
events = read_events(trace)
|
||||||
|
samples = [event for event in events if event.get("type") == "streaming_sample"]
|
||||||
|
moving = [event for event in samples if float(event.get("speed_cm_s", 0.0)) >= 50.0]
|
||||||
|
stop = next((event for event in events if event.get("type") == "streaming_test_move_stopped"), {})
|
||||||
|
stop_s = float(stop.get("t_s", 0.0))
|
||||||
|
move_start = min((float(event.get("t_s", 0.0)) for event in moving), default=float("inf"))
|
||||||
|
dwell = [event for event in samples if float(event.get("t_s", 0.0)) >= stop_s
|
||||||
|
and float(event.get("speed_cm_s", 0.0)) < 50.0]
|
||||||
|
backend = next((event for event in events if event.get("type") == "worker_backend_config"), {})
|
||||||
|
cap = next((event for event in events if event.get("type") == "streaming_worker_cap_readback"), {})
|
||||||
|
if process.returncode or len(moving) < 40 or len(dwell) < 16:
|
||||||
|
raise RuntimeError(f"incomplete {scenario} run: {trace}; exit={process.returncode}; "
|
||||||
|
f"move={len(moving)} dwell={len(dwell)}")
|
||||||
|
if (int(backend.get("streaming_fast_travel_wait_for_all_critical_tiles", -1))
|
||||||
|
!= wait_for_critical
|
||||||
|
or int(backend.get("streaming_fast_travel_near_only_while_critical_pending", -1))
|
||||||
|
!= near_only_while_critical_pending
|
||||||
|
or int(backend.get("streaming_far_apply_supplement", -1)) != 1
|
||||||
|
or int(cap.get("streaming_background_worker_cap", -1)) != 1
|
||||||
|
or int(backend.get("effective_tile_slots", -1)) != 8):
|
||||||
|
raise RuntimeError(f"configuration readback mismatch: {backend}; {cap}")
|
||||||
|
|
||||||
|
moving_scheduler = [event for event in events if event.get("type") == "scheduler_admission"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop_s
|
||||||
|
and event.get("moving", 0)]
|
||||||
|
idle = [event for event in moving_scheduler
|
||||||
|
if event.get("running_workers", 0) == 0
|
||||||
|
and sum(event.get("desired_missing_by_lod", [0] * 9)[:3])
|
||||||
|
+ sum(event.get("pending_by_lod", [0] * 9)[:3]) > 0]
|
||||||
|
move_rows = []
|
||||||
|
for event in moving:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
resident = event.get("resident_by_lod", [0] * 9)
|
||||||
|
covered = event.get("covered_by_lod", resident)
|
||||||
|
move_rows.append({
|
||||||
|
"near_ready_pct": 100.0 * sum(resident[:3]) / max(1, sum(desired[:3])),
|
||||||
|
"lod0_ready_pct": 100.0 * resident[0] / max(1, desired[0]),
|
||||||
|
"lod1_ready_pct": 100.0 * resident[1] / max(1, desired[1]),
|
||||||
|
"lod2_ready_pct": 100.0 * resident[2] / max(1, desired[2]),
|
||||||
|
"far_hole_pct": 100.0 * (sum(desired[3:]) - sum(covered[3:]))
|
||||||
|
/ max(1, sum(desired[3:])),
|
||||||
|
})
|
||||||
|
work_frames = [event for event in events if event.get("type") == "game_thread_work_frame"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop_s]
|
||||||
|
tick_values = [float(event.get("tick_ms", 0.0)) for event in work_frames]
|
||||||
|
update_values = [float(event.get("update_chunks_ms", 0.0)) for event in work_frames]
|
||||||
|
requests = [event for event in events if event.get("type") == "request"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop_s
|
||||||
|
and event.get("level", 9) <= 2]
|
||||||
|
|
||||||
|
arrival_rows = [event for event in samples if float(event.get("t_s", 0.0)) < start_s
|
||||||
|
and float(event.get("speed_cm_s", 0.0)) < 50.0]
|
||||||
|
|
||||||
|
def near_complete(event: dict) -> bool:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
settled = event.get("settled_by_lod", [0] * 9)
|
||||||
|
return sum(desired[:3]) > 0 and sum(settled[:3]) >= sum(desired[:3])
|
||||||
|
|
||||||
|
def whole_complete(event: dict) -> bool:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
settled = event.get("settled_by_lod", [0] * 9)
|
||||||
|
return sum(desired) > 0 and sum(settled) >= sum(desired)
|
||||||
|
|
||||||
|
def far_holes(event: dict) -> float | None:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)[3:]
|
||||||
|
covered = event.get("covered_by_lod", [0] * 9)[3:]
|
||||||
|
return (100.0 * (sum(desired) - sum(covered)) / sum(desired)
|
||||||
|
if sum(desired) else None)
|
||||||
|
|
||||||
|
near_settled = next((event for event in arrival_rows if near_complete(event)), None)
|
||||||
|
whole_settled = next((event for event in arrival_rows if whole_complete(event)), None)
|
||||||
|
far_clear = next((event for event in arrival_rows if far_holes(event) == 0.0), None)
|
||||||
|
|
||||||
|
def mean_metric(name: str, rows: list[dict]) -> float | None:
|
||||||
|
values = [float(row[name]) for row in rows if row.get(name) is not None]
|
||||||
|
return statistics.mean(values) if values else None
|
||||||
|
|
||||||
|
row = {
|
||||||
|
"scenario": scenario, "condition": condition, "repetition": repetition,
|
||||||
|
"design": design, "speed_mps": speed_cm_s / 100.0,
|
||||||
|
"wait_for_all_critical_tiles": wait_for_critical,
|
||||||
|
"near_only_while_critical_pending": near_only_while_critical_pending,
|
||||||
|
"exit_code": process.returncode, "wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"moving_samples": len(moving), "dwell_samples": len(dwell),
|
||||||
|
"near_ready_moving_mean_pct": mean_metric("near_ready_pct", move_rows),
|
||||||
|
"lod0_ready_moving_mean_pct": mean_metric("lod0_ready_pct", move_rows),
|
||||||
|
"lod1_ready_moving_mean_pct": mean_metric("lod1_ready_pct", move_rows),
|
||||||
|
"lod2_ready_moving_mean_pct": mean_metric("lod2_ready_pct", move_rows),
|
||||||
|
"far_holes_moving_mean_pct": mean_metric("far_hole_pct", move_rows),
|
||||||
|
"game_thread_tick_p99_ms": percentile(tick_values, 0.99),
|
||||||
|
"game_thread_tick_max_ms": max(tick_values, default=None),
|
||||||
|
"stream_update_p99_ms": percentile(update_values, 0.99),
|
||||||
|
"stream_update_max_ms": max(update_values, default=None),
|
||||||
|
"near_requests_during_move": len(requests),
|
||||||
|
"arrival_sample_count": len(arrival_rows),
|
||||||
|
"lod02_settled_t_s": float(near_settled["t_s"]) if near_settled else None,
|
||||||
|
"whole_view_settled_t_s": float(whole_settled["t_s"]) if whole_settled else None,
|
||||||
|
"far_clear_t_s": float(far_clear["t_s"]) if far_clear else None,
|
||||||
|
"far_holes_at_pre_move_pct": far_holes(arrival_rows[-1]) if arrival_rows else None,
|
||||||
|
"idle_missing_near_windows": len(idle),
|
||||||
|
"idle_critical_gate_closed": sum(not event.get("critical_tiles_ready", 1)
|
||||||
|
for event in idle),
|
||||||
|
"idle_exact_region_declines": sum(
|
||||||
|
sum(event.get("region_compile_declines_by_lod", [0] * 9)[:3]) > 0
|
||||||
|
for event in idle),
|
||||||
|
"idle_far_parent_declines": sum(
|
||||||
|
sum(event.get("far_parent_declines_by_lod", [0] * 9)[:3]) > 0
|
||||||
|
for event in idle),
|
||||||
|
"idle_near_cap_declines": sum(
|
||||||
|
any(value in (1, 2) for value in event.get("declined_rule_by_lod", [0] * 9)[:3])
|
||||||
|
for event in idle),
|
||||||
|
"idle_desired_center_stale": sum(not event.get("desired_center_current", 1)
|
||||||
|
for event in idle),
|
||||||
|
"trace": str(trace.relative_to(ROOT)), "log": str(abslog.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L122Pair] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def pairs_for(runs: list[dict], scenario: str, design: str) -> list[dict]:
|
||||||
|
rows = []
|
||||||
|
for repetition in range(1, 6):
|
||||||
|
baseline = next(row for row in runs if row["scenario"] == scenario
|
||||||
|
and row["condition"] == "baseline" and row["repetition"] == repetition)
|
||||||
|
candidate = next(row for row in runs if row["scenario"] == scenario
|
||||||
|
and row["condition"] == design and row["repetition"] == repetition)
|
||||||
|
rows.append({
|
||||||
|
"repetition": repetition,
|
||||||
|
"near_ready_delta_pct": candidate.get("near_ready_moving_mean_pct", 0.0)
|
||||||
|
- baseline.get("near_ready_moving_mean_pct", 0.0),
|
||||||
|
"lod0_ready_delta_pct": candidate.get("lod0_ready_moving_mean_pct", 0.0)
|
||||||
|
- baseline.get("lod0_ready_moving_mean_pct", 0.0),
|
||||||
|
"lod1_ready_delta_pct": candidate.get("lod1_ready_moving_mean_pct", 0.0)
|
||||||
|
- baseline.get("lod1_ready_moving_mean_pct", 0.0),
|
||||||
|
"lod2_ready_delta_pct": candidate.get("lod2_ready_moving_mean_pct", 0.0)
|
||||||
|
- baseline.get("lod2_ready_moving_mean_pct", 0.0),
|
||||||
|
"far_holes_delta_pct": candidate.get("far_holes_moving_mean_pct", 0.0)
|
||||||
|
- baseline.get("far_holes_moving_mean_pct", 0.0),
|
||||||
|
"tick_p99_delta_ms": candidate.get("game_thread_tick_p99_ms", 0.0)
|
||||||
|
- baseline.get("game_thread_tick_p99_ms", 0.0),
|
||||||
|
"tick_max_delta_ms": candidate.get("game_thread_tick_max_ms", 0.0)
|
||||||
|
- baseline.get("game_thread_tick_max_ms", 0.0),
|
||||||
|
})
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--scenario", choices=("movement", "arrival"), required=True)
|
||||||
|
parser.add_argument("--design", choices=(DESIGN, NEAR_ONLY_DESIGN), default=DESIGN)
|
||||||
|
args = parser.parse_args()
|
||||||
|
if not EDITOR.is_file():
|
||||||
|
raise SystemExit(f"missing Unreal editor: {EDITOR}")
|
||||||
|
output_name = args.scenario + ("-near-only" if args.design == NEAR_ONLY_DESIGN else "")
|
||||||
|
out = ROOT / f"Saved/Linux/l122/{output_name}"
|
||||||
|
out.mkdir(parents=True, exist_ok=True)
|
||||||
|
runs = []
|
||||||
|
for repetition in range(1, 6):
|
||||||
|
order = ("baseline", args.design) if repetition % 2 else (args.design, "baseline")
|
||||||
|
for condition in order:
|
||||||
|
runs.append(run(args.scenario, condition, repetition, args.design))
|
||||||
|
(out / "runs-partial.json").write_text(json.dumps(runs, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
pairs = pairs_for(runs, args.scenario, args.design)
|
||||||
|
summary = {
|
||||||
|
"design_id": args.design,
|
||||||
|
"design": ("Allow fast-travel near LOD0-2 admission after the 3x3 walkable ring is ready "
|
||||||
|
"while other expanded critical tiles remain unresident; skip directional sort "
|
||||||
|
"and far admission until the critical set is ready."
|
||||||
|
if args.design == NEAR_ONLY_DESIGN else
|
||||||
|
"Allow fast-travel near/far admissions after the 3x3 walkable ring is ready, "
|
||||||
|
"even while other expanded critical tiles remain unresident; critical requests "
|
||||||
|
"still submit first."),
|
||||||
|
"scenario": args.scenario,
|
||||||
|
"runs": runs,
|
||||||
|
"pairs": pairs,
|
||||||
|
"paired_median_near_ready_delta_pct": statistics.median(
|
||||||
|
pair["near_ready_delta_pct"] for pair in pairs),
|
||||||
|
"paired_median_far_holes_delta_pct": statistics.median(
|
||||||
|
pair["far_holes_delta_pct"] for pair in pairs),
|
||||||
|
"paired_median_tick_p99_delta_ms": statistics.median(
|
||||||
|
pair["tick_p99_delta_ms"] for pair in pairs),
|
||||||
|
"paired_median_tick_max_delta_ms": statistics.median(
|
||||||
|
pair["tick_max_delta_ms"] for pair in pairs),
|
||||||
|
}
|
||||||
|
(out / "summary.json").write_text(json.dumps(summary, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,218 @@
|
|||||||
|
diff --git a/Source/VoxelForge/Private/VoxelWorld.cpp b/Source/VoxelForge/Private/VoxelWorld.cpp
|
||||||
|
index 42ac6f5..5c436ee 100644
|
||||||
|
--- a/Source/VoxelForge/Private/VoxelWorld.cpp
|
||||||
|
+++ b/Source/VoxelForge/Private/VoxelWorld.cpp
|
||||||
|
@@ -195,10 +195,13 @@ public:
|
||||||
|
std::atomic<bool> bShuttingDown{false};
|
||||||
|
std::atomic<uint64> NextEditSequence{1};
|
||||||
|
std::atomic<uint64> RegionCompileStartCycles{0};
|
||||||
|
+ std::atomic<uint64> RegionCompileWorkerStartCycles{0};
|
||||||
|
std::atomic<uint64> RegionCompileEndCycles{0};
|
||||||
|
std::atomic<int32> RegionCompileTileCount{0};
|
||||||
|
std::atomic<int32> RegionCompileSerial{0};
|
||||||
|
+ std::atomic<uint32> RegionCompileWorkerThreadId{0};
|
||||||
|
std::atomic<double> RegionCompileWorkerSeconds{0.0};
|
||||||
|
+ std::atomic<double> RegionCompileWallSeconds{0.0};
|
||||||
|
int32 FullResolutionLodCount = 0;
|
||||||
|
FString InitialFailureReason;
|
||||||
|
};
|
||||||
|
@@ -11381,20 +11384,25 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||||
|
1, std::memory_order_relaxed) + 1;
|
||||||
|
State->RegionCompileStartCycles.store(ActiveCoreRegionCompileStartCycles,
|
||||||
|
std::memory_order_release);
|
||||||
|
+ State->RegionCompileWorkerStartCycles.store(0, std::memory_order_release);
|
||||||
|
State->RegionCompileEndCycles.store(0, std::memory_order_release);
|
||||||
|
State->RegionCompileTileCount.store(0, std::memory_order_release);
|
||||||
|
+ State->RegionCompileWorkerThreadId.store(0, std::memory_order_release);
|
||||||
|
State->RegionCompileWorkerSeconds.store(0.0, std::memory_order_release);
|
||||||
|
+ State->RegionCompileWallSeconds.store(0.0, std::memory_order_release);
|
||||||
|
if (VoxelForgeStartupTrace::IsActive())
|
||||||
|
{
|
||||||
|
VoxelForgeStartupTrace::RecordEvent(TEXT("core_region_compile_start"), FString::Printf(
|
||||||
|
TEXT("\"compile_serial\":%d,\"request_serial\":%d,\"center\":[%d,%d,%d],"
|
||||||
|
"\"missing_region_tiles\":%d,\"retained_tiles\":%d,\"concurrent_admission\":%d,"
|
||||||
|
- "\"active_tile_tasks\":%d,\"active_task_guard\":%d"),
|
||||||
|
+ "\"active_tile_tasks\":%d,\"active_generation_workers\":%d,"
|
||||||
|
+ "\"active_task_guard\":%d,\"game_thread_id\":%u"),
|
||||||
|
CompileSerial, CoreRegionCompileRequestSerial,
|
||||||
|
CompileCenter.X, CompileCenter.Y, CompileCenter.Z,
|
||||||
|
MissingTiles.Num(), RetainedTiles.Num(), bAllowConcurrentGeneration ? 1 : 0,
|
||||||
|
ActiveTileTaskCount.load(std::memory_order_relaxed),
|
||||||
|
- ActiveTaskCount.load(std::memory_order_relaxed)));
|
||||||
|
+ ActiveGenerationWorkerCount.load(std::memory_order_relaxed),
|
||||||
|
+ ActiveTaskCount.load(std::memory_order_relaxed), FPlatformTLS::GetCurrentThreadId()));
|
||||||
|
}
|
||||||
|
ActiveTaskCount.fetch_add(1, std::memory_order_relaxed);
|
||||||
|
UE::Tasks::Launch(TEXT("VoxelForgeCoreTileRegion"),
|
||||||
|
@@ -11407,6 +11415,11 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||||
|
~FTaskGuard() { Counter.fetch_sub(1, std::memory_order_release); }
|
||||||
|
} Guard{ActiveTaskCount};
|
||||||
|
|
||||||
|
+ State->RegionCompileWorkerStartCycles.store(FPlatformTime::Cycles64(),
|
||||||
|
+ std::memory_order_release);
|
||||||
|
+ State->RegionCompileWorkerThreadId.store(FPlatformTLS::GetCurrentThreadId(),
|
||||||
|
+ std::memory_order_release);
|
||||||
|
+
|
||||||
|
if (bShuttingDown.load(std::memory_order_acquire)
|
||||||
|
|| State->bShuttingDown.load(std::memory_order_acquire))
|
||||||
|
{
|
||||||
|
@@ -11458,6 +11471,8 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||||
|
}
|
||||||
|
State->RegionCompileWorkerSeconds.store(CompileStats.workerSeconds,
|
||||||
|
std::memory_order_release);
|
||||||
|
+ State->RegionCompileWallSeconds.store(CompileStats.wallSeconds,
|
||||||
|
+ std::memory_order_release);
|
||||||
|
State->bRegionCompileSucceeded.store(bCompiled, std::memory_order_release);
|
||||||
|
if (!bCompiled)
|
||||||
|
{
|
||||||
|
@@ -12345,9 +12360,17 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
State->RegionCompileStartCycles.load(std::memory_order_acquire);
|
||||||
|
const uint64 CompileEndCycles =
|
||||||
|
State->RegionCompileEndCycles.load(std::memory_order_acquire);
|
||||||
|
+ const uint64 CompileWorkerStartCycles =
|
||||||
|
+ State->RegionCompileWorkerStartCycles.load(std::memory_order_acquire);
|
||||||
|
const double CompileElapsedMilliseconds = CompileEndCycles >= CompileStartCycles
|
||||||
|
? FPlatformTime::ToSeconds64(CompileEndCycles - CompileStartCycles) * 1000.0
|
||||||
|
: 0.0;
|
||||||
|
+ const double CompileQueueMilliseconds = CompileWorkerStartCycles >= CompileStartCycles
|
||||||
|
+ ? FPlatformTime::ToSeconds64(CompileWorkerStartCycles - CompileStartCycles) * 1000.0
|
||||||
|
+ : 0.0;
|
||||||
|
+ const double CompileTaskMilliseconds = CompileEndCycles >= CompileWorkerStartCycles
|
||||||
|
+ ? FPlatformTime::ToSeconds64(CompileEndCycles - CompileWorkerStartCycles) * 1000.0
|
||||||
|
+ : 0.0;
|
||||||
|
const double GateElapsedMilliseconds = CoreRegionPendingSinceSeconds > 0.0
|
||||||
|
? FMath::Max(0.0, FPlatformTime::Seconds() - CoreRegionPendingSinceSeconds) * 1000.0
|
||||||
|
: 0.0;
|
||||||
|
@@ -12355,6 +12378,10 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
State->RegionCompileTileCount.load(std::memory_order_acquire);
|
||||||
|
const double CompileWorkerSeconds =
|
||||||
|
State->RegionCompileWorkerSeconds.load(std::memory_order_acquire);
|
||||||
|
+ const double CompileCoreWallSeconds =
|
||||||
|
+ State->RegionCompileWallSeconds.load(std::memory_order_acquire);
|
||||||
|
+ const uint32 CompileWorkerThreadId =
|
||||||
|
+ State->RegionCompileWorkerThreadId.load(std::memory_order_acquire);
|
||||||
|
const int32 CompileSerial =
|
||||||
|
State->RegionCompileSerial.load(std::memory_order_acquire);
|
||||||
|
const uint64 IndependentSubmissions = CoreRegionIndependentSubmissions;
|
||||||
|
@@ -12364,14 +12391,20 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
VoxelForgeStartupTrace::RecordEvent(TEXT("core_region_compile_complete"), FString::Printf(
|
||||||
|
TEXT("\"compile_serial\":%d,\"request_serial\":%d,\"center\":[%d,%d,%d],"
|
||||||
|
"\"success\":%d,\"compiled_tiles\":%d,\"compile_elapsed_ms\":%s,"
|
||||||
|
- "\"compile_worker_s\":%s,\"pending_gate_elapsed_ms\":%s,"
|
||||||
|
+ "\"compile_queue_ms\":%s,\"compile_task_ms\":%s,"
|
||||||
|
+ "\"compile_core_wall_ms\":%s,\"compile_worker_s\":%s,"
|
||||||
|
+ "\"compile_worker_thread_id\":%u,\"pending_gate_elapsed_ms\":%s,"
|
||||||
|
"\"independent_submissions\":%llu,"
|
||||||
|
"\"dependent_declines\":%llu,\"concurrent_admission\":%d"),
|
||||||
|
CompileSerial, ActiveCoreRegionCompileRequestSerial,
|
||||||
|
ActiveCoreRegionCompileCenter.X, ActiveCoreRegionCompileCenter.Y,
|
||||||
|
ActiveCoreRegionCompileCenter.Z, bRegionCompileSucceeded ? 1 : 0,
|
||||||
|
CompiledTileCount, *FString::SanitizeFloat(CompileElapsedMilliseconds),
|
||||||
|
+ *FString::SanitizeFloat(CompileQueueMilliseconds),
|
||||||
|
+ *FString::SanitizeFloat(CompileTaskMilliseconds),
|
||||||
|
+ *FString::SanitizeFloat(CompileCoreWallSeconds * 1000.0),
|
||||||
|
*FString::SanitizeFloat(CompileWorkerSeconds),
|
||||||
|
+ CompileWorkerThreadId,
|
||||||
|
*FString::SanitizeFloat(GateElapsedMilliseconds),
|
||||||
|
static_cast<unsigned long long>(IndependentSubmissions),
|
||||||
|
static_cast<unsigned long long>(DependentDeclines),
|
||||||
|
@@ -12584,6 +12617,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
&& bWalkableRingReadyForReserve;
|
||||||
|
int32 SchedulerSubmittedByLevel[9] = {};
|
||||||
|
int32 SchedulerLoadAttemptsByLevel[9] = {};
|
||||||
|
+ int32 SchedulerRegionCompileDeclinesByLevel[9] = {};
|
||||||
|
+ int32 SchedulerFarParentDeclinesByLevel[9] = {};
|
||||||
|
int32 SchedulerLevelCapReturns[9] = {};
|
||||||
|
int32 SchedulerLevelCapBreaks[9] = {};
|
||||||
|
int32 SchedulerCriticalFullCapBreaks = 0;
|
||||||
|
@@ -12951,7 +12986,21 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
if(CoreState.IsValid()
|
||||||
|
&&CoreState->bFarWorldRequired.load(std::memory_order_acquire)
|
||||||
|
&&!CoreState->bFarWorldReady.load(std::memory_order_acquire))
|
||||||
|
+ {
|
||||||
|
+ if (bTraceStreamingScheduler && Tile.Level >= 0 && Tile.Level < 9)
|
||||||
|
+ {
|
||||||
|
+ ++SchedulerFarParentDeclinesByLevel[Tile.Level];
|
||||||
|
+ }
|
||||||
|
return;
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ if (bTraceStreamingScheduler && bUseVoxelForgeCore && bCoreRegionCompilePending
|
||||||
|
+ && GVoxelForgeCoreRegionConcurrentAdmission != 0
|
||||||
|
+ && (PendingCoreRegionTileSet.Contains(Tile)
|
||||||
|
+ || CoreRegionCompileTilesInFlight.Contains(Tile))
|
||||||
|
+ && Tile.Level >= 0 && Tile.Level < 9)
|
||||||
|
+ {
|
||||||
|
+ ++SchedulerRegionCompileDeclinesByLevel[Tile.Level];
|
||||||
|
}
|
||||||
|
if (Tile.Level >= 0 && Tile.Level < 9)
|
||||||
|
{
|
||||||
|
@@ -13661,10 +13710,37 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
const double BusySeconds = FPlatformTime::ToSeconds64(WorkerCyclesDelta);
|
||||||
|
const double SchedulerUpdateMilliseconds = (SchedulerTraceNow
|
||||||
|
- StreamingUpdateStartSeconds) * 1000.0;
|
||||||
|
+ int32 CriticalMissingTiles = 0;
|
||||||
|
+ for (const FVoxelTileKey& Tile : CriticalDesiredTiles)
|
||||||
|
+ {
|
||||||
|
+ if (!LoadedTiles.Contains(Tile) && !CollisionNotRequiredTiles.Contains(Tile))
|
||||||
|
+ {
|
||||||
|
+ ++CriticalMissingTiles;
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ int32 CollisionGateMissingTiles = 0;
|
||||||
|
+ for (const FVoxelTileKey& Tile : CollisionGateDesiredTiles)
|
||||||
|
+ {
|
||||||
|
+ if (!LoadedTiles.Contains(Tile) && !CollisionNotRequiredTiles.Contains(Tile))
|
||||||
|
+ {
|
||||||
|
+ ++CollisionGateMissingTiles;
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ const FIntVector DesiredCenterLag = DesiredEpochCenterChunk - CenterChunk;
|
||||||
|
+ const TSharedPtr<FVoxelForgeCoreRuntimeState, ESPMode::ThreadSafe> TraceCoreState =
|
||||||
|
+ VoxelForgeCoreState;
|
||||||
|
+ const bool bFarWorldRequiredForTrace = TraceCoreState.IsValid()
|
||||||
|
+ && TraceCoreState->bFarWorldRequired.load(std::memory_order_relaxed);
|
||||||
|
+ const bool bFarWorldReadyForTrace = TraceCoreState.IsValid()
|
||||||
|
+ && TraceCoreState->bFarWorldReady.load(std::memory_order_relaxed);
|
||||||
|
VoxelForgeStartupTrace::RecordEvent(TEXT("scheduler_admission"), FString::Printf(
|
||||||
|
TEXT("\"frame\":%llu,\"center\":[%d,%d,%d],\"desired_rebuilt\":%d,"
|
||||||
|
"\"submission_loop_ran\":%d,\"all_chunks_loaded\":%d,"
|
||||||
|
"\"walkable_ready\":%d,\"moving\":%d,"
|
||||||
|
+ "\"pawn_collision_gate_engaged\":%d,\"critical_tiles_ready\":%d,"
|
||||||
|
+ "\"critical_missing\":%d,\"collision_gate_missing\":%d,"
|
||||||
|
+ "\"desired_center_lag\":[%d,%d,%d],\"desired_center_current\":%d,"
|
||||||
|
+ "\"far_world_required\":%d,\"far_world_ready\":%d,"
|
||||||
|
"\"core_region_compile_pending\":%d,\"core_ready\":%d,\"coarse_first\":%d,"
|
||||||
|
"\"dynamic_near_reserve\":%d,\"legacy_pending_result_cap\":%d,"
|
||||||
|
"\"core_region_concurrent_admission\":%d,"
|
||||||
|
@@ -13685,6 +13761,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
"\"level_candidate_scan_ms\":%s,\"passage_score_ms\":%s,"
|
||||||
|
"\"level_sort_ms\":%s,\"level_admit_ms\":%s,"
|
||||||
|
"\"desired_missing_by_lod\":%s,\"loaded_by_lod\":%s,\"pending_by_lod\":%s,"
|
||||||
|
+ "\"region_compile_declines_by_lod\":%s,"
|
||||||
|
+ "\"far_parent_declines_by_lod\":%s,"
|
||||||
|
"\"submitted_by_lod\":%s,\"load_attempts_by_lod\":%s,"
|
||||||
|
"\"admission_cap_by_lod\":%s,\"declined_rule_by_lod\":%s,"
|
||||||
|
"\"level_cap_returns_by_lod\":%s,\"level_cap_breaks_by_lod\":%s,"
|
||||||
|
@@ -13695,6 +13773,11 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
static_cast<unsigned long long>(GFrameCounter), CenterChunk.X, CenterChunk.Y, CenterChunk.Z,
|
||||||
|
bDesiredRebuiltThisUpdate ? 1 : 0, bSubmissionLoopRan ? 1 : 0, bAllChunksLoaded ? 1 : 0,
|
||||||
|
bWalkableRingReadyForReserve ? 1 : 0, bPawnMoving ? 1 : 0,
|
||||||
|
+ bPawnCollisionGateEngaged ? 1 : 0, CriticalMissingTiles == 0 ? 1 : 0,
|
||||||
|
+ CriticalMissingTiles, CollisionGateMissingTiles,
|
||||||
|
+ DesiredCenterLag.X, DesiredCenterLag.Y, DesiredCenterLag.Z,
|
||||||
|
+ DesiredCenterLag == FIntVector::ZeroValue ? 1 : 0,
|
||||||
|
+ bFarWorldRequiredForTrace ? 1 : 0, bFarWorldReadyForTrace ? 1 : 0,
|
||||||
|
bCoreRegionCompilePending ? 1 : 0, IsVoxelForgeCoreReady() ? 1 : 0,
|
||||||
|
GVoxelForgeStreamingCoarseFirst, GVoxelForgeStreamingDynamicNearReserve,
|
||||||
|
GVoxelForgeStreamingLegacyPendingResultCap,
|
||||||
|
@@ -13721,7 +13804,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
*FString::SanitizeFloat(LevelSortMilliseconds),
|
||||||
|
*FString::SanitizeFloat(LevelAdmissionMilliseconds),
|
||||||
|
*IntArrayJson(DesiredMissingByLevel), *IntArrayJson(LoadedByLevel),
|
||||||
|
- *IntArrayJson(PendingByLevel), *IntArrayJson(SchedulerSubmittedByLevel),
|
||||||
|
+ *IntArrayJson(PendingByLevel), *IntArrayJson(SchedulerRegionCompileDeclinesByLevel),
|
||||||
|
+ *IntArrayJson(SchedulerFarParentDeclinesByLevel), *IntArrayJson(SchedulerSubmittedByLevel),
|
||||||
|
*IntArrayJson(SchedulerLoadAttemptsByLevel), *IntArrayJson(AdmissionCapByLevel),
|
||||||
|
*IntArrayJson(DeclinedRuleByLevel), *IntArrayJson(SchedulerLevelCapReturns),
|
||||||
|
*IntArrayJson(SchedulerLevelCapBreaks), FullCapDeclinedTiles, ReserveDeclinedTiles,
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
# L123: region completion gate
|
||||||
|
|
||||||
|
Five interleaved 30 m/s UE pairs show the gate is a game-thread poll delay. Core compile wall time is p50 0.251 ms (worker queue plus task p50 0.475 ms); request-to-observe is p50 18.89 ms, p95 21.68 ms. In 95% of traced completions, the next scheduler pass drains the MPSC result and resumes admission in that same pass; a small tail waits 2–4 scheduler frames. This is one tick interval, not separate compile, observe, and admit ticks.
|
||||||
|
|
||||||
|
The worker can safely hand off an immutable result through MPSC with its runtime state and `GenerationEpoch`. The game thread validates both before updating state and admitting tiles. Direct worker-side admission would touch game-thread state unsafely. This queue handoff preserves ownership but still polls once per tick; a game-thread wakeup/continuation is the safe way to try removing that delay. A wakeup was not timed in this round.
|
||||||
|
|
||||||
|
One-step velocity lookahead did not remove the wait (gate p50 19.0 ms). Paired near readiness fell 0.79 percentage points; paired far holes rose 0.14 points. Its worst-run p99 was 11.77 ms (limit 11.00) and worst tick 15.11 ms (limit 14.74). After a 90° turn, 252 of 960 retained ahead keys became stale. At 1.4 m/s it compiled 128 ahead tiles per run; paired near arrival was 2.6 ms faster, while far-clear arrival was 14 ms slower. Keep lookahead off.
|
||||||
|
|
||||||
|
The queue-only runs also missed the p99 limit (worst 11.46 ms), though their worst tick was 14.19 ms. Neither measured option meets the full bar. Headless 30 m/s completed with zero generation failures (82.71% near readiness, 14.21% far holes); this path has no UE region-completion gate. Geometry checks passed 24/24 for Clang/GCC W1/W8/W12, and the fast tier passed.
|
||||||
@@ -0,0 +1,115 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Compare L123 exact tile hashes with L104 across strata, compilers, and worker counts."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
OUT = ROOT / "Saved/Linux/l123/hashes"
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
OLD = ROOT / "Saved/Linux/l104/hashes"
|
||||||
|
BINARIES = {
|
||||||
|
"clang": ROOT / "Saved/Linux/l123/bin/VoxelForgeCoreHeadless-l123-clang",
|
||||||
|
"gcc": ROOT / "Saved/Linux/l123/bin/VoxelForgeCoreHeadless-l123-gcc",
|
||||||
|
}
|
||||||
|
CAMERAS = {
|
||||||
|
"sw": ((8.0, 16.0, -372.2626953125), 2850),
|
||||||
|
"fi": ((25.8369140625, 3.19921875, -737.0), 2249),
|
||||||
|
"tn": ((19.166015625, -14.2001953125, -753.3740234375), 2233),
|
||||||
|
"vs": ((-20.0625, -5.068359375, -900.353515625), 2329),
|
||||||
|
}
|
||||||
|
TILE = re.compile(
|
||||||
|
r"^\[PipelineTileHash\] lod=(-?\d+) x=(-?\d+) y=(-?\d+) z=(-?\d+) "
|
||||||
|
r"density=(0x[0-9a-f]+) mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) "
|
||||||
|
r"topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
WHOLE = re.compile(
|
||||||
|
r"^\[PipelineWholeViewHash\] tiles=(\d+) density=(0x[0-9a-f]+) "
|
||||||
|
r"mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M)
|
||||||
|
|
||||||
|
|
||||||
|
def parse(path: Path) -> tuple[dict, tuple[str, ...], str]:
|
||||||
|
content = path.read_text(encoding="utf-8", errors="replace")
|
||||||
|
tiles = {(int(m.group(1)), int(m.group(2)), int(m.group(3)), int(m.group(4))):
|
||||||
|
m.group(5, 6, 7, 8) for m in TILE.finditer(content)}
|
||||||
|
whole, stamp = WHOLE.search(content), STAMP.search(content)
|
||||||
|
if whole is None or stamp is None or len(tiles) != int(whole.group(1)):
|
||||||
|
raise RuntimeError(f"incomplete hash output: {path}")
|
||||||
|
return tiles, whole.groups(), stamp.group(1)
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, compiler: str, workers: int) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = OUT / f"{stratum}-{compiler}-w{workers}.log"
|
||||||
|
camera, expected_tiles = CAMERAS[stratum]
|
||||||
|
camera_arg = ",".join(f"{v:.12f}" for v in camera) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", str(BINARIES[compiler]), str(DATA),
|
||||||
|
"--world-assembler-prototype", "--stream-benchmark", "--stream-pipeline",
|
||||||
|
f"--workers={workers}", "--clip-radius=3", "--max-clip-level=8",
|
||||||
|
"--max-level=8", f"--camera={camera_arg}", "--stream-hash-tiles",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
if process.returncode:
|
||||||
|
raise RuntimeError(f"hash process exited {process.returncode}: {log}")
|
||||||
|
tiles, whole, stamp = parse(log)
|
||||||
|
if len(tiles) != expected_tiles:
|
||||||
|
raise RuntimeError(f"expected {expected_tiles} tiles, got {len(tiles)}: {log}")
|
||||||
|
old_tiles, old_whole, _ = parse(OLD / f"{stratum}-{compiler}-w{workers}.log")
|
||||||
|
clang_tiles, clang_whole, _ = parse(OLD / f"{stratum}-clang-w1.log")
|
||||||
|
|
||||||
|
def difference(reference: dict, candidate: dict) -> tuple[int, int]:
|
||||||
|
keys = set(reference) | set(candidate)
|
||||||
|
return sum(reference.get(key) != candidate.get(key) for key in keys), len(keys)
|
||||||
|
|
||||||
|
old_mismatches, old_keys = difference(old_tiles, tiles)
|
||||||
|
cross_mismatches, cross_keys = difference(clang_tiles, tiles)
|
||||||
|
row = {
|
||||||
|
"stratum": stratum, "compiler": compiler, "workers": workers,
|
||||||
|
"source_stamp": stamp, "tiles": len(tiles),
|
||||||
|
"l104_same_cell_tile_mismatches": old_mismatches,
|
||||||
|
"l104_same_cell_key_count": old_keys,
|
||||||
|
"l104_same_cell_whole_hash_match": whole == old_whole,
|
||||||
|
"clang_w1_tile_mismatches": cross_mismatches,
|
||||||
|
"clang_w1_key_count": cross_keys,
|
||||||
|
"clang_w1_whole_hash_match": whole == clang_whole,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L123HashRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
row["tile_hashes"] = {"|".join(map(str, key)): value for key, value in tiles.items()}
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
for path in (*BINARIES.values(), DATA):
|
||||||
|
if not path.is_file():
|
||||||
|
raise SystemExit(f"missing input: {path}")
|
||||||
|
rows = []
|
||||||
|
for stratum in CAMERAS:
|
||||||
|
for compiler in BINARIES:
|
||||||
|
for workers in (1, 8, 12):
|
||||||
|
rows.append(run(stratum, compiler, workers))
|
||||||
|
(OUT / "runs-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
failures = [row for row in rows if row["l104_same_cell_tile_mismatches"]
|
||||||
|
or not row["l104_same_cell_whole_hash_match"]
|
||||||
|
or row["clang_w1_tile_mismatches"]
|
||||||
|
or not row["clang_w1_whole_hash_match"]]
|
||||||
|
summary = {"runs": [{key: value for key, value in row.items() if key != "tile_hashes"}
|
||||||
|
for row in rows], "failed_runs": len(failures)}
|
||||||
|
(OUT / "summary.json").write_text(json.dumps(summary, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 1 if failures else 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""One 30 m/s headless movement run for the UE-only region gate experiment."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
BIN = ROOT / "Saved/Linux/l108/bin/VoxelForgeCoreHeadless-l108-after-clang"
|
||||||
|
OUT = ROOT / "Saved/Linux/l123/headless"
|
||||||
|
SAMPLE = re.compile(r"^\[L79MovingSample\] (.*)$", re.M)
|
||||||
|
SUMMARY = re.compile(r"^\[L79MovingSummary\] (.*)$", re.M)
|
||||||
|
FIELDS = re.compile(r"([a-zA-Z0-9_]+)=([^ ]+)")
|
||||||
|
|
||||||
|
|
||||||
|
def p99(values: list[float]) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
values.sort()
|
||||||
|
return values[max(0, math.ceil(0.99 * len(values)) - 1)]
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = OUT / "movement-30mps-r1.log"
|
||||||
|
camera = "19.166015625000,-14.200195312500,-753.374023437500,0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", str(BIN), str(DATA), "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", "--workers=8", "--clip-radius=3",
|
||||||
|
"--max-clip-level=8", "--max-level=8", f"--camera={camera}",
|
||||||
|
"--profile-lod-work", "--stream-scheduler-policy=3",
|
||||||
|
"--moving-camera-speed-mps=30.0", "--moving-camera-duration-s=12",
|
||||||
|
"--moving-camera-dwell-s=5",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
content = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
rows = []
|
||||||
|
for match in SAMPLE.finditer(content):
|
||||||
|
row = dict(FIELDS.findall(match.group(1)))
|
||||||
|
for name in ("near_ready_pct", "far_hole_pct", "queue_update_ms"):
|
||||||
|
if name in row:
|
||||||
|
row[name] = float(row[name])
|
||||||
|
row["phase"] = row.get("phase", "move")
|
||||||
|
rows.append(row)
|
||||||
|
summary = dict(FIELDS.findall(SUMMARY.search(content).group(1))) if SUMMARY.search(content) else {}
|
||||||
|
moving = [row for row in rows if row["phase"] == "move"]
|
||||||
|
updates = [row["queue_update_ms"] for row in moving if "queue_update_ms" in row]
|
||||||
|
result = {
|
||||||
|
"exit_code": process.returncode,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"moving_samples": len(moving),
|
||||||
|
"near_ready_moving_mean_pct": sum(r["near_ready_pct"] for r in moving) / max(1, len(moving)),
|
||||||
|
"far_holes_moving_mean_pct": sum(r["far_hole_pct"] for r in moving) / max(1, len(moving)),
|
||||||
|
"update_work_p99_ms": p99(updates),
|
||||||
|
"update_work_max_ms": max(updates, default=None),
|
||||||
|
"generation_failures": int(summary.get("generation_failures", -1)),
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
(OUT / "summary.json").write_text(json.dumps(result, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
print("[L123Headless] " + json.dumps(result, sort_keys=True), flush=True)
|
||||||
|
if process.returncode or not moving or result["generation_failures"] != 0:
|
||||||
|
return 1
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,262 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Five interleaved 30 m/s UE pairs for the Core region lookahead experiment."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import statistics
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
HOST = Path("/home/jahni/VFHost")
|
||||||
|
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
|
||||||
|
SPAWN_TN_M = (19.166015625, -14.2001953125, -753.3740234375)
|
||||||
|
SPAWN_FI_M = (25.8369140625, 3.19921875, -737.0)
|
||||||
|
|
||||||
|
|
||||||
|
def percentile(values: list[float], fraction: float) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
ordered = sorted(values)
|
||||||
|
return ordered[max(0, min(len(ordered) - 1, math.ceil(len(ordered) * fraction) - 1))]
|
||||||
|
|
||||||
|
|
||||||
|
def read_events(path: Path) -> list[dict]:
|
||||||
|
rows = []
|
||||||
|
if path.is_file():
|
||||||
|
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||||
|
if not line.startswith("{"):
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
rows.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def run(scenario: str, condition: str, repetition: int, turn: bool = False) -> dict:
|
||||||
|
lookahead = int(condition in ("lookahead", "turn_lookahead"))
|
||||||
|
if scenario == "movement":
|
||||||
|
speed, start, duration, exit_s, slot, spawn = (
|
||||||
|
3000.0, 6.0, 12.0, 23.0, 0, SPAWN_TN_M)
|
||||||
|
else:
|
||||||
|
speed, start, duration, exit_s, slot, spawn = (
|
||||||
|
140.0, 20.0, 12.0, 37.0, 1, SPAWN_FI_M)
|
||||||
|
|
||||||
|
label = f"{scenario}-{condition}-r{repetition}"
|
||||||
|
output_group = "turn" if turn else scenario
|
||||||
|
out = ROOT / "Saved/Linux/l123" / output_group / condition / "unreal"
|
||||||
|
host_out = HOST / "Plugins/VoxelForge/Saved/Linux/l123" / output_group / condition
|
||||||
|
host_saved = HOST / "Saved/Linux/l123"
|
||||||
|
trace = out / f"{label}.trace.jsonl"
|
||||||
|
abslog = out / f"{label}.log"
|
||||||
|
stdout = out / f"{label}.stdout.log"
|
||||||
|
export = host_out / "output" / label
|
||||||
|
userdir = host_saved / "user" / label
|
||||||
|
zendata = host_saved / "zen" / label
|
||||||
|
ddc = host_saved / "DDC"
|
||||||
|
for path in (out, export, userdir, zendata, ddc):
|
||||||
|
path.mkdir(parents=True, exist_ok=True)
|
||||||
|
trace.unlink(missing_ok=True)
|
||||||
|
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in spawn)
|
||||||
|
command = [
|
||||||
|
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
|
||||||
|
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
|
||||||
|
"-notraceserver", "-nocrashreports", "-DisablePlugins=AndroidFileServer",
|
||||||
|
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
|
||||||
|
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
|
||||||
|
"-archetype=TunnelNetwork", f"-slot={slot}", "-modes=export", "-gameconfig=1",
|
||||||
|
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
|
||||||
|
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0", "-coreplantest=0",
|
||||||
|
"-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings", "-corehybridmesher=0",
|
||||||
|
f"-out={export}", "-voxel.StreamingCoarseFirst=1",
|
||||||
|
"-voxel.StreamingDynamicNearReserve=1", "-voxel.StreamingLegacyPendingResultCap=0",
|
||||||
|
"-voxel.CoreRegionConcurrentAdmission=1", "-voxel.StreamingResultBacklogCap=256",
|
||||||
|
"-voxel.StreamingReadyQueueDepth=8", "-voxel.StreamingBackgroundWorkerCap=1",
|
||||||
|
"-voxel.CoreColumnCacheDirectMapped=0", "-voxel.StreamingFarApplySupplement=1",
|
||||||
|
"-voxel.CoreMCGradientMemoization=1", "-voxel.StreamingCollisionLookahead=1",
|
||||||
|
f"-voxel.CoreRegionLookahead={lookahead}", "-voxel.TestMove=1",
|
||||||
|
f"-voxel.TestMoveSpeedCmPerSecond={speed:.1f}",
|
||||||
|
f"-voxel.TestMoveStartSeconds={start:.1f}",
|
||||||
|
f"-voxel.TestMoveDurationSeconds={duration:.1f}",
|
||||||
|
f"-voxel.TestExitSeconds={exit_s:.1f}", "-voxel.TestMoveDirection=1,0,0",
|
||||||
|
"-voxel.StreamingSchedulerPolicy=3", "-voxel.TestExitOnSteady=0",
|
||||||
|
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
|
||||||
|
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
|
||||||
|
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
|
||||||
|
f"-abslog={abslog}",
|
||||||
|
]
|
||||||
|
if turn:
|
||||||
|
command.extend(["-voxel.TestMoveTurnAfterSeconds=6.0",
|
||||||
|
"-voxel.TestMoveTurnDirection=0,1,0"])
|
||||||
|
started = time.monotonic()
|
||||||
|
wrapped = ["taskset", "-c", "0-11", "timeout", "1740s", *command]
|
||||||
|
with stdout.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(wrapped, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
events = read_events(trace)
|
||||||
|
samples = [event for event in events if event.get("type") == "streaming_sample"]
|
||||||
|
moving = [event for event in samples if float(event.get("speed_cm_s", 0.0)) >= 50.0]
|
||||||
|
stop_event = next((event for event in events
|
||||||
|
if event.get("type") == "streaming_test_move_stopped"), {})
|
||||||
|
stop_s = float(stop_event.get("t_s", 0.0))
|
||||||
|
move_start = min((float(event.get("t_s", 0.0)) for event in moving), default=float("inf"))
|
||||||
|
dwell = [event for event in samples if float(event.get("t_s", 0.0)) >= stop_s
|
||||||
|
and float(event.get("speed_cm_s", 0.0)) < 50.0]
|
||||||
|
backend = next((event for event in events if event.get("type") == "worker_backend_config"), {})
|
||||||
|
cap = next((event for event in events if event.get("type") == "streaming_worker_cap_readback"), {})
|
||||||
|
if process.returncode or len(moving) < 40 or len(dwell) < 16:
|
||||||
|
raise RuntimeError(f"incomplete UE run {label}: exit={process.returncode}, "
|
||||||
|
f"move={len(moving)}, dwell={len(dwell)}, trace={trace}")
|
||||||
|
if (int(backend.get("core_region_lookahead", -1)) != lookahead
|
||||||
|
or int(backend.get("streaming_far_apply_supplement", -1)) != 1
|
||||||
|
or int(cap.get("streaming_background_worker_cap", -1)) != 1
|
||||||
|
or int(backend.get("effective_tile_slots", -1)) != 8):
|
||||||
|
raise RuntimeError(f"configuration readback mismatch: {backend}; {cap}")
|
||||||
|
|
||||||
|
moving_rows = []
|
||||||
|
for event in moving:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
resident = event.get("resident_by_lod", [0] * 9)
|
||||||
|
covered = event.get("covered_by_lod", resident)
|
||||||
|
moving_rows.append({
|
||||||
|
"near_ready_pct": 100.0 * sum(resident[:3]) / max(1, sum(desired[:3])),
|
||||||
|
"lod0_ready_pct": 100.0 * resident[0] / max(1, desired[0]),
|
||||||
|
"lod1_ready_pct": 100.0 * resident[1] / max(1, desired[1]),
|
||||||
|
"lod2_ready_pct": 100.0 * resident[2] / max(1, desired[2]),
|
||||||
|
"far_hole_pct": 100.0 * (sum(desired[3:]) - sum(covered[3:]))
|
||||||
|
/ max(1, sum(desired[3:])),
|
||||||
|
})
|
||||||
|
work = [event for event in events if event.get("type") == "game_thread_work_frame"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop_s]
|
||||||
|
ticks = [float(event.get("tick_ms", 0.0)) for event in work]
|
||||||
|
updates = [float(event.get("update_chunks_ms", 0.0)) for event in work]
|
||||||
|
arrival = [event for event in samples if float(event.get("t_s", 0.0)) < start
|
||||||
|
and float(event.get("speed_cm_s", 0.0)) < 50.0]
|
||||||
|
|
||||||
|
def settled(event: dict, end: int) -> bool:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)
|
||||||
|
actual = event.get("settled_by_lod", [0] * 9)
|
||||||
|
return sum(desired[:end]) > 0 and sum(actual[:end]) >= sum(desired[:end])
|
||||||
|
|
||||||
|
def far_holes(event: dict) -> float | None:
|
||||||
|
desired = event.get("desired_by_lod", [0] * 9)[3:]
|
||||||
|
covered = event.get("covered_by_lod", [0] * 9)[3:]
|
||||||
|
return 100.0 * (sum(desired) - sum(covered)) / sum(desired) if sum(desired) else None
|
||||||
|
|
||||||
|
near_settled = next((event for event in arrival if settled(event, 3)), None)
|
||||||
|
whole_settled = next((event for event in arrival if settled(event, 9)), None)
|
||||||
|
far_clear = next((event for event in arrival if far_holes(event) == 0.0), None)
|
||||||
|
compile_done = [event for event in events if event.get("type") == "core_region_compile_complete"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop_s]
|
||||||
|
compile_requests = [event for event in events
|
||||||
|
if event.get("type") == "core_region_compile_request"
|
||||||
|
and move_start <= float(event.get("t_s", 0.0)) < stop_s]
|
||||||
|
lookahead_turns = [event for event in events
|
||||||
|
if event.get("type") == "core_region_lookahead_turn"]
|
||||||
|
row = {
|
||||||
|
"scenario": scenario, "condition": condition, "repetition": repetition,
|
||||||
|
"speed_mps": speed / 100.0, "core_region_lookahead": lookahead,
|
||||||
|
"exit_code": process.returncode, "wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"moving_samples": len(moving), "dwell_samples": len(dwell),
|
||||||
|
"near_ready_moving_mean_pct": statistics.mean(r["near_ready_pct"] for r in moving_rows),
|
||||||
|
"lod0_ready_moving_mean_pct": statistics.mean(r["lod0_ready_pct"] for r in moving_rows),
|
||||||
|
"lod1_ready_moving_mean_pct": statistics.mean(r["lod1_ready_pct"] for r in moving_rows),
|
||||||
|
"lod2_ready_moving_mean_pct": statistics.mean(r["lod2_ready_pct"] for r in moving_rows),
|
||||||
|
"far_holes_moving_mean_pct": statistics.mean(r["far_hole_pct"] for r in moving_rows),
|
||||||
|
"game_thread_tick_p99_ms": percentile(ticks, 0.99),
|
||||||
|
"game_thread_tick_max_ms": max(ticks, default=None),
|
||||||
|
"stream_update_p99_ms": percentile(updates, 0.99),
|
||||||
|
"stream_update_max_ms": max(updates, default=None),
|
||||||
|
"lod02_settled_t_s": float(near_settled["t_s"]) if near_settled else None,
|
||||||
|
"whole_view_settled_t_s": float(whole_settled["t_s"]) if whole_settled else None,
|
||||||
|
"far_clear_t_s": float(far_clear["t_s"]) if far_clear else None,
|
||||||
|
"far_holes_at_pre_move_pct": far_holes(arrival[-1]) if arrival else None,
|
||||||
|
"pending_gate_p50_ms": percentile(
|
||||||
|
[float(e.get("pending_gate_elapsed_ms", 0.0)) for e in compile_done], 0.50),
|
||||||
|
"compile_queue_p95_ms": percentile(
|
||||||
|
[float(e.get("compile_queue_ms", 0.0)) for e in compile_done], 0.95),
|
||||||
|
"compile_task_p95_ms": percentile(
|
||||||
|
[float(e.get("compile_task_ms", 0.0)) for e in compile_done], 0.95),
|
||||||
|
"compiled_tiles_total": sum(int(e.get("compiled_tiles", 0)) for e in compile_done),
|
||||||
|
"lookahead_tiles_peak": max((int(e.get("lookahead_region_tiles", 0))
|
||||||
|
for e in compile_requests), default=0),
|
||||||
|
"wasted_prefetch_tiles": sum(int(e.get("wasted_prefetch_tiles", 0))
|
||||||
|
for e in compile_done),
|
||||||
|
"lookahead_turn_events": lookahead_turns,
|
||||||
|
"compile_completion_count": len(compile_done),
|
||||||
|
"trace": str(trace.relative_to(ROOT)), "log": str(abslog.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
with (ROOT / "Saved/Linux/l123" / "runs.jsonl").open("a", encoding="utf-8") as stream:
|
||||||
|
stream.write(json.dumps(row, sort_keys=True) + "\n")
|
||||||
|
print("[L123Pair] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def pairs_for(runs: list[dict], scenario: str) -> list[dict]:
|
||||||
|
pairs = []
|
||||||
|
for repetition in range(1, 6):
|
||||||
|
baseline = next(row for row in runs if row["scenario"] == scenario
|
||||||
|
and row["condition"] == "baseline" and row["repetition"] == repetition)
|
||||||
|
candidate = next(row for row in runs if row["scenario"] == scenario
|
||||||
|
and row["condition"] == "lookahead" and row["repetition"] == repetition)
|
||||||
|
pairs.append({"repetition": repetition,
|
||||||
|
"near_ready_delta_pct": candidate["near_ready_moving_mean_pct"]
|
||||||
|
- baseline["near_ready_moving_mean_pct"],
|
||||||
|
"far_holes_delta_pct": candidate["far_holes_moving_mean_pct"]
|
||||||
|
- baseline["far_holes_moving_mean_pct"],
|
||||||
|
"tick_p99_delta_ms": candidate["game_thread_tick_p99_ms"]
|
||||||
|
- baseline["game_thread_tick_p99_ms"],
|
||||||
|
"tick_max_delta_ms": candidate["game_thread_tick_max_ms"]
|
||||||
|
- baseline["game_thread_tick_max_ms"]})
|
||||||
|
return pairs
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--scenario", choices=("movement", "arrival"), required=True)
|
||||||
|
parser.add_argument("--turn-diagnostic", action="store_true")
|
||||||
|
args = parser.parse_args()
|
||||||
|
if not EDITOR.is_file():
|
||||||
|
raise SystemExit(f"missing Unreal editor: {EDITOR}")
|
||||||
|
root_out = ROOT / "Saved/Linux/l123" / args.scenario
|
||||||
|
root_out.mkdir(parents=True, exist_ok=True)
|
||||||
|
(ROOT / "Saved/Linux/l123").mkdir(parents=True, exist_ok=True)
|
||||||
|
if args.turn_diagnostic:
|
||||||
|
row = run("movement", "turn_lookahead", 1, turn=True)
|
||||||
|
turn_out = ROOT / "Saved/Linux/l123/turn"
|
||||||
|
turn_out.mkdir(parents=True, exist_ok=True)
|
||||||
|
(turn_out / "summary.json").write_text(
|
||||||
|
json.dumps({"run": row}, indent=2) + "\n", encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
runs = []
|
||||||
|
for repetition in range(1, 6):
|
||||||
|
order = ("baseline", "lookahead") if repetition % 2 else ("lookahead", "baseline")
|
||||||
|
for condition in order:
|
||||||
|
runs.append(run(args.scenario, condition, repetition))
|
||||||
|
(root_out / "runs-partial.json").write_text(
|
||||||
|
json.dumps(runs, indent=2) + "\n", encoding="utf-8")
|
||||||
|
pairs = pairs_for(runs, args.scenario)
|
||||||
|
summary = {
|
||||||
|
"design": "Compile/retain the current near region set plus one step along velocity; "
|
||||||
|
"return completion through an Epoch-tagged MPSC queue.",
|
||||||
|
"scenario": args.scenario, "runs": runs, "pairs": pairs,
|
||||||
|
"paired_median_near_ready_delta_pct": statistics.median(
|
||||||
|
pair["near_ready_delta_pct"] for pair in pairs),
|
||||||
|
"paired_median_far_holes_delta_pct": statistics.median(
|
||||||
|
pair["far_holes_delta_pct"] for pair in pairs),
|
||||||
|
"paired_median_tick_p99_delta_ms": statistics.median(
|
||||||
|
pair["tick_p99_delta_ms"] for pair in pairs),
|
||||||
|
"paired_median_tick_max_delta_ms": statistics.median(
|
||||||
|
pair["tick_max_delta_ms"] for pair in pairs),
|
||||||
|
}
|
||||||
|
(root_out / "summary.json").write_text(json.dumps(summary, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,825 @@
|
|||||||
|
diff --git a/Source/VoxelForge/Private/VoxelWorld.cpp b/Source/VoxelForge/Private/VoxelWorld.cpp
|
||||||
|
index 42ac6f5..2c7046b 100644
|
||||||
|
--- a/Source/VoxelForge/Private/VoxelWorld.cpp
|
||||||
|
+++ b/Source/VoxelForge/Private/VoxelWorld.cpp
|
||||||
|
@@ -195,10 +195,13 @@ public:
|
||||||
|
std::atomic<bool> bShuttingDown{false};
|
||||||
|
std::atomic<uint64> NextEditSequence{1};
|
||||||
|
std::atomic<uint64> RegionCompileStartCycles{0};
|
||||||
|
+ std::atomic<uint64> RegionCompileWorkerStartCycles{0};
|
||||||
|
std::atomic<uint64> RegionCompileEndCycles{0};
|
||||||
|
std::atomic<int32> RegionCompileTileCount{0};
|
||||||
|
std::atomic<int32> RegionCompileSerial{0};
|
||||||
|
+ std::atomic<uint32> RegionCompileWorkerThreadId{0};
|
||||||
|
std::atomic<double> RegionCompileWorkerSeconds{0.0};
|
||||||
|
+ std::atomic<double> RegionCompileWallSeconds{0.0};
|
||||||
|
int32 FullResolutionLodCount = 0;
|
||||||
|
FString InitialFailureReason;
|
||||||
|
};
|
||||||
|
@@ -262,6 +265,12 @@ namespace
|
||||||
|
GVoxelForgeCoreRegionConcurrentAdmission,
|
||||||
|
TEXT("Allow tiles independent of a pending Core region compile to proceed; 0 restores the global streaming stop."));
|
||||||
|
|
||||||
|
+ int32 GVoxelForgeCoreRegionLookahead = 0;
|
||||||
|
+ FAutoConsoleVariableRef CVarVoxelForgeCoreRegionLookahead(
|
||||||
|
+ TEXT("voxel.CoreRegionLookahead"),
|
||||||
|
+ GVoxelForgeCoreRegionLookahead,
|
||||||
|
+ TEXT("Retain and compile the next near region set along velocity; 0 disables region lookahead."));
|
||||||
|
+
|
||||||
|
int32 GVoxelForgeStreamingResultBacklogCap = 256;
|
||||||
|
FAutoConsoleVariableRef CVarVoxelForgeStreamingResultBacklogCap(
|
||||||
|
TEXT("voxel.StreamingResultBacklogCap"),
|
||||||
|
@@ -1745,6 +1754,10 @@ void AVoxelWorld::EndPlay(const EEndPlayReason::Type EndPlayReason)
|
||||||
|
{
|
||||||
|
if (Discard.bObsolete) { ++ObsoleteTileAbortCount; }
|
||||||
|
}
|
||||||
|
+ FCoreRegionCompileResult DiscardedRegionCompile;
|
||||||
|
+ while (CoreRegionCompileResults.Dequeue(DiscardedRegionCompile))
|
||||||
|
+ {
|
||||||
|
+ }
|
||||||
|
UE_LOG(LogTemp, Display, TEXT("[VoxelWorld] EndPlay: workers drained in %.6fs; obsolete_tile_aborts=%d"),
|
||||||
|
FPlatformTime::Seconds() - EndPlayStartSeconds, ObsoleteTileAbortCount);
|
||||||
|
VoxelForgeCoreState.Reset();
|
||||||
|
@@ -2093,6 +2106,11 @@ void AVoxelWorld::BeginPlay()
|
||||||
|
PendingCoreRegionTiles.Reset();
|
||||||
|
PendingCoreRegionTileSet.Reset();
|
||||||
|
PendingCoreRetainedTiles.Reset();
|
||||||
|
+ CoreRegionRetainedDesiredTiles.Reset();
|
||||||
|
+ CoreRegionLookaheadTiles.Reset();
|
||||||
|
+ LastCoreRegionLookaheadCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
|
||||||
|
+ LastCoreRegionLookaheadStep = FIntVector::ZeroValue;
|
||||||
|
+ LastCoreRegionLookaheadGenerationEpoch = GenerationEpoch;
|
||||||
|
CoreRegionCompileTilesInFlight.Reset();
|
||||||
|
LastCoreRegionRequestCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
|
||||||
|
PendingCoreRegionRequestCenter = FIntVector::ZeroValue;
|
||||||
|
@@ -2149,6 +2167,7 @@ void AVoxelWorld::BeginPlay()
|
||||||
|
int32 CommandLineSampleStacks = GVoxelForgeSampleStacks;
|
||||||
|
int32 CommandLineCollisionGate = GVoxelForgeCollisionGate;
|
||||||
|
int32 CommandLineCoreRegionConcurrentAdmission = GVoxelForgeCoreRegionConcurrentAdmission;
|
||||||
|
+ int32 CommandLineCoreRegionLookahead = GVoxelForgeCoreRegionLookahead;
|
||||||
|
int32 CommandLineStreamingCollisionLookahead = GVoxelForgeStreamingCollisionLookahead;
|
||||||
|
int32 CommandLineStreamingResultBacklogCap = GVoxelForgeStreamingResultBacklogCap;
|
||||||
|
int32 CommandLineStreamingReadyQueueDepth = GVoxelForgeStreamingReadyQueueDepth;
|
||||||
|
@@ -2257,6 +2276,11 @@ void AVoxelWorld::BeginPlay()
|
||||||
|
GVoxelForgeCoreRegionConcurrentAdmission =
|
||||||
|
CommandLineCoreRegionConcurrentAdmission != 0 ? 1 : 0;
|
||||||
|
}
|
||||||
|
+ if (FParse::Value(FCommandLine::Get(), TEXT("voxel.CoreRegionLookahead="),
|
||||||
|
+ CommandLineCoreRegionLookahead))
|
||||||
|
+ {
|
||||||
|
+ GVoxelForgeCoreRegionLookahead = CommandLineCoreRegionLookahead != 0 ? 1 : 0;
|
||||||
|
+ }
|
||||||
|
if (FParse::Value(FCommandLine::Get(), TEXT("voxel.StreamingResultBacklogCap="),
|
||||||
|
CommandLineStreamingResultBacklogCap))
|
||||||
|
{
|
||||||
|
@@ -2747,6 +2771,7 @@ void AVoxelWorld::BeginPlay()
|
||||||
|
"\"streaming_coarse_first\":%d,\"streaming_dynamic_near_reserve\":%d,"
|
||||||
|
"\"streaming_legacy_pending_result_cap\":%d,"
|
||||||
|
"\"streaming_collision_lookahead\":%d,\"core_region_concurrent_admission\":%d,"
|
||||||
|
+ "\"core_region_lookahead\":%d,"
|
||||||
|
"\"collision_gate\":%d,"
|
||||||
|
"\"streaming_result_backlog_cap\":%d,\"streaming_startup_apply_boost\":%d,"
|
||||||
|
"\"streaming_far_apply_supplement\":%d,"
|
||||||
|
@@ -2769,6 +2794,7 @@ void AVoxelWorld::BeginPlay()
|
||||||
|
GVoxelForgeStreamingCoarseFirst, GVoxelForgeStreamingDynamicNearReserve,
|
||||||
|
GVoxelForgeStreamingLegacyPendingResultCap,
|
||||||
|
GVoxelForgeStreamingCollisionLookahead, GVoxelForgeCoreRegionConcurrentAdmission,
|
||||||
|
+ GVoxelForgeCoreRegionLookahead,
|
||||||
|
GVoxelForgeCollisionGate,
|
||||||
|
GVoxelForgeStreamingResultBacklogCap, GVoxelForgeStreamingStartupApplyBoost,
|
||||||
|
GVoxelForgeStreamingFarApplySupplement,
|
||||||
|
@@ -3246,11 +3272,13 @@ void AVoxelWorld::ConfigureHeadlessStreamingTest()
|
||||||
|
float Speed = 800.0f;
|
||||||
|
float StartSeconds = 20.0f;
|
||||||
|
float MoveDurationSeconds = 0.0f;
|
||||||
|
+ float MoveTurnAfterSeconds = 0.0f;
|
||||||
|
float ExitSeconds = 0.0f;
|
||||||
|
int32 ExitOnSteadyValue = 0;
|
||||||
|
FParse::Value(CommandLine, TEXT("voxel.TestMoveSpeedCmPerSecond="), Speed);
|
||||||
|
FParse::Value(CommandLine, TEXT("voxel.TestMoveStartSeconds="), StartSeconds);
|
||||||
|
FParse::Value(CommandLine, TEXT("voxel.TestMoveDurationSeconds="), MoveDurationSeconds);
|
||||||
|
+ FParse::Value(CommandLine, TEXT("voxel.TestMoveTurnAfterSeconds="), MoveTurnAfterSeconds);
|
||||||
|
FParse::Value(CommandLine, TEXT("voxel.TestExitSeconds="), ExitSeconds);
|
||||||
|
FParse::Value(CommandLine, TEXT("voxel.TestTeleportAfterSeconds="),
|
||||||
|
HeadlessStreamingTestTeleportAtSeconds);
|
||||||
|
@@ -3418,9 +3446,44 @@ void AVoxelWorld::ConfigureHeadlessStreamingTest()
|
||||||
|
{
|
||||||
|
HeadlessStreamingTestDirection = FVector::XAxisVector;
|
||||||
|
}
|
||||||
|
+ FString TurnDirectionText;
|
||||||
|
+ const FString TurnDirectionKey(TEXT("voxel.TestMoveTurnDirection="));
|
||||||
|
+ const int32 TurnDirectionKeyIndex = CommandLineText.Find(
|
||||||
|
+ TurnDirectionKey, ESearchCase::IgnoreCase, ESearchDir::FromStart);
|
||||||
|
+ if (TurnDirectionKeyIndex != INDEX_NONE)
|
||||||
|
+ {
|
||||||
|
+ TurnDirectionText = CommandLineText.Mid(
|
||||||
|
+ TurnDirectionKeyIndex + TurnDirectionKey.Len());
|
||||||
|
+ const int32 DirectionEnd = TurnDirectionText.Find(
|
||||||
|
+ TEXT(" "), ESearchCase::CaseSensitive, ESearchDir::FromStart);
|
||||||
|
+ if (DirectionEnd != INDEX_NONE)
|
||||||
|
+ {
|
||||||
|
+ TurnDirectionText.LeftInline(DirectionEnd, EAllowShrinking::No);
|
||||||
|
+ }
|
||||||
|
+ TurnDirectionText.TrimStartAndEndInline();
|
||||||
|
+ TurnDirectionText.RemoveFromStart(TEXT("\""));
|
||||||
|
+ TurnDirectionText.RemoveFromEnd(TEXT("\""));
|
||||||
|
+ TArray<FString> Components;
|
||||||
|
+ TurnDirectionText.ParseIntoArray(Components, TEXT(","), true);
|
||||||
|
+ if (Components.Num() == 3)
|
||||||
|
+ {
|
||||||
|
+ HeadlessStreamingTestTurnDirection = FVector(
|
||||||
|
+ FCString::Atof(*Components[0]),
|
||||||
|
+ FCString::Atof(*Components[1]),
|
||||||
|
+ FCString::Atof(*Components[2]));
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ HeadlessStreamingTestTurnDirection =
|
||||||
|
+ HeadlessStreamingTestTurnDirection.GetSafeNormal();
|
||||||
|
+ if (HeadlessStreamingTestTurnDirection.IsNearlyZero())
|
||||||
|
+ {
|
||||||
|
+ HeadlessStreamingTestTurnDirection = FVector::YAxisVector;
|
||||||
|
+ }
|
||||||
|
HeadlessStreamingTestSpeedCmPerSecond = FMath::Max(Speed, 0.0f);
|
||||||
|
HeadlessStreamingTestStartSeconds = FMath::Max(StartSeconds, 0.0f);
|
||||||
|
HeadlessStreamingTestMoveDurationSeconds = FMath::Max(MoveDurationSeconds, 0.0f);
|
||||||
|
+ HeadlessStreamingTestTurnAfterSeconds = FMath::Max(MoveTurnAfterSeconds, 0.0f);
|
||||||
|
+ bHeadlessStreamingTestTurnDone = false;
|
||||||
|
bHeadlessStreamingTestMovement = HeadlessStreamingTestSpeedCmPerSecond > 0.0f;
|
||||||
|
}
|
||||||
|
|
||||||
|
@@ -3431,15 +3494,20 @@ void AVoxelWorld::ConfigureHeadlessStreamingTest()
|
||||||
|
HeadlessStreamingTestBeginSeconds = FPlatformTime::Seconds();
|
||||||
|
UE_LOG(LogTemp, Display,
|
||||||
|
TEXT("[VoxelForgeStreamingTest] clean=1 movement=%d speed_cm_s=%.3f start_s=%.3f "
|
||||||
|
- "move_duration_s=%.3f exit_s=%.3f direction=(%.3f,%.3f,%.3f)"),
|
||||||
|
+ "move_duration_s=%.3f turn_after_s=%.3f exit_s=%.3f "
|
||||||
|
+ "direction=(%.3f,%.3f,%.3f) turn_direction=(%.3f,%.3f,%.3f)"),
|
||||||
|
bHeadlessStreamingTestMovement ? 1 : 0,
|
||||||
|
HeadlessStreamingTestSpeedCmPerSecond,
|
||||||
|
HeadlessStreamingTestStartSeconds,
|
||||||
|
HeadlessStreamingTestMoveDurationSeconds,
|
||||||
|
+ HeadlessStreamingTestTurnAfterSeconds,
|
||||||
|
HeadlessStreamingTestExitSeconds,
|
||||||
|
HeadlessStreamingTestDirection.X,
|
||||||
|
HeadlessStreamingTestDirection.Y,
|
||||||
|
- HeadlessStreamingTestDirection.Z);
|
||||||
|
+ HeadlessStreamingTestDirection.Z,
|
||||||
|
+ HeadlessStreamingTestTurnDirection.X,
|
||||||
|
+ HeadlessStreamingTestTurnDirection.Y,
|
||||||
|
+ HeadlessStreamingTestTurnDirection.Z);
|
||||||
|
}
|
||||||
|
|
||||||
|
int32 CollisionGateStressValue = 0;
|
||||||
|
@@ -4503,6 +4571,32 @@ void AVoxelWorld::AdvanceHeadlessStreamingTest(
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
+ if (!bHeadlessStreamingTestTurnDone && HeadlessStreamingTestTurnAfterSeconds > 0.0f
|
||||||
|
+ && ElapsedSeconds - static_cast<double>(HeadlessStreamingTestStartSeconds)
|
||||||
|
+ >= static_cast<double>(HeadlessStreamingTestTurnAfterSeconds))
|
||||||
|
+ {
|
||||||
|
+ bHeadlessStreamingTestTurnDone = true;
|
||||||
|
+ const FVector PreviousDirection = HeadlessStreamingTestDirection;
|
||||||
|
+ HeadlessStreamingTestDirection = HeadlessStreamingTestTurnDirection;
|
||||||
|
+ if (VoxelForgeStartupTrace::IsActive())
|
||||||
|
+ {
|
||||||
|
+ VoxelForgeStartupTrace::RecordEvent(TEXT("streaming_test_move_turned"),
|
||||||
|
+ FString::Printf(
|
||||||
|
+ TEXT("\"elapsed_s\":%s,\"from\":[%.0f,%.0f,%.0f],"
|
||||||
|
+ "\"to\":[%.0f,%.0f,%.0f]"),
|
||||||
|
+ *FString::SanitizeFloat(ElapsedSeconds),
|
||||||
|
+ PreviousDirection.X, PreviousDirection.Y, PreviousDirection.Z,
|
||||||
|
+ HeadlessStreamingTestDirection.X, HeadlessStreamingTestDirection.Y,
|
||||||
|
+ HeadlessStreamingTestDirection.Z));
|
||||||
|
+ }
|
||||||
|
+ UE_LOG(LogTemp, Display,
|
||||||
|
+ TEXT("[VoxelForgeStreamingTest] turn=1 elapsed_s=%.3f from=(%.0f,%.0f,%.0f) "
|
||||||
|
+ "to=(%.0f,%.0f,%.0f)"),
|
||||||
|
+ ElapsedSeconds, PreviousDirection.X, PreviousDirection.Y, PreviousDirection.Z,
|
||||||
|
+ HeadlessStreamingTestDirection.X, HeadlessStreamingTestDirection.Y,
|
||||||
|
+ HeadlessStreamingTestDirection.Z);
|
||||||
|
+ }
|
||||||
|
+
|
||||||
|
if (HeadlessStreamingTestMoveDurationSeconds > 0.0f
|
||||||
|
&& ElapsedSeconds - static_cast<double>(HeadlessStreamingTestStartSeconds)
|
||||||
|
>= static_cast<double>(HeadlessStreamingTestMoveDurationSeconds))
|
||||||
|
@@ -11258,16 +11352,113 @@ void AVoxelWorld::BuildCoreDesiredTiles(const FIntVector& Center, const FVector&
|
||||||
|
}
|
||||||
|
BuildDesiredTransitionIndex();
|
||||||
|
|
||||||
|
- bool bRetainedSetChanged = false;
|
||||||
|
- int32 RetainedTileCount = 0;
|
||||||
|
+ FIntVector RegionLookaheadStep = FIntVector::ZeroValue;
|
||||||
|
+ if (GVoxelForgeCoreRegionLookahead != 0 && PlayerHeading.Size() >= 50.0f)
|
||||||
|
+ {
|
||||||
|
+ RegionLookaheadStep = GetStreamingHeadingStep(PlayerHeading, PlayerPawn);
|
||||||
|
+ }
|
||||||
|
+ const bool bRegionLookaheadTurn = RegionLookaheadStep != FIntVector::ZeroValue
|
||||||
|
+ && LastCoreRegionLookaheadStep != FIntVector::ZeroValue
|
||||||
|
+ && LastCoreRegionLookaheadStep != RegionLookaheadStep
|
||||||
|
+ && LastCoreRegionLookaheadGenerationEpoch == GenerationEpoch;
|
||||||
|
+ const FIntVector PreviousRegionLookaheadStep = LastCoreRegionLookaheadStep;
|
||||||
|
+ const TArray<FVoxelTileKey> PreviousRegionLookaheadTiles = bRegionLookaheadTurn
|
||||||
|
+ ? CoreRegionLookaheadTiles : TArray<FVoxelTileKey>();
|
||||||
|
+ if (RegionLookaheadStep != FIntVector::ZeroValue
|
||||||
|
+ && (LastCoreRegionLookaheadCenter != Center
|
||||||
|
+ || LastCoreRegionLookaheadStep != RegionLookaheadStep
|
||||||
|
+ || LastCoreRegionLookaheadGenerationEpoch != GenerationEpoch))
|
||||||
|
+ {
|
||||||
|
+ const FVector AheadCameraVoxels = CameraVoxels + FVector(
|
||||||
|
+ static_cast<float>(RegionLookaheadStep.X),
|
||||||
|
+ static_cast<float>(RegionLookaheadStep.Y),
|
||||||
|
+ static_cast<float>(RegionLookaheadStep.Z)) * static_cast<float>(CHUNK_SIZE);
|
||||||
|
+ std::vector<VoxelForgeCore::TileKey> AheadDesired;
|
||||||
|
+ {
|
||||||
|
+ FReadScopeLock Lock(State->WorldLock);
|
||||||
|
+ if (State->World)
|
||||||
|
+ {
|
||||||
|
+ AheadDesired = State->World->desiredTiles(
|
||||||
|
+ {AheadCameraVoxels.X, AheadCameraVoxels.Y, AheadCameraVoxels.Z});
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ CoreRegionLookaheadTiles.Reset();
|
||||||
|
+ TSet<FVoxelTileKey> AheadSet;
|
||||||
|
+ for (const VoxelForgeCore::TileKey& Key : AheadDesired)
|
||||||
|
+ {
|
||||||
|
+ if (Key.lod >= State->FullResolutionLodCount) continue;
|
||||||
|
+ const FVoxelTileKey RuntimeKey(
|
||||||
|
+ FIntVector(Key.coordinate.x, Key.coordinate.y, Key.coordinate.z), Key.lod);
|
||||||
|
+ if (!AheadSet.Contains(RuntimeKey))
|
||||||
|
+ {
|
||||||
|
+ AheadSet.Add(RuntimeKey);
|
||||||
|
+ CoreRegionLookaheadTiles.Add(RuntimeKey);
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ if (bRegionLookaheadTurn)
|
||||||
|
+ {
|
||||||
|
+ TSet<FVoxelTileKey> ActualSet;
|
||||||
|
+ for (const FVoxelTileKey& Key : DesiredSorted) ActualSet.Add(Key);
|
||||||
|
+ int32 StaleRetainedLookaheadTiles = 0;
|
||||||
|
+ {
|
||||||
|
+ FScopeLock RetainedLock(&State->RetainedTilesLock);
|
||||||
|
+ for (const FVoxelTileKey& Key : PreviousRegionLookaheadTiles)
|
||||||
|
+ {
|
||||||
|
+ if (ActualSet.Contains(Key) || AheadSet.Contains(Key)) continue;
|
||||||
|
+ const VoxelForgeCore::TileKey CoreKey{
|
||||||
|
+ {Key.Coord.X, Key.Coord.Y, Key.Coord.Z}, Key.Level};
|
||||||
|
+ if (State->RetainedTiles.contains(CoreKey))
|
||||||
|
+ {
|
||||||
|
+ ++StaleRetainedLookaheadTiles;
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ if (VoxelForgeStartupTrace::IsActive())
|
||||||
|
+ {
|
||||||
|
+ VoxelForgeStartupTrace::RecordEvent(TEXT("core_region_lookahead_turn"),
|
||||||
|
+ FString::Printf(
|
||||||
|
+ TEXT("\"previous_step\":[%d,%d,%d],\"new_step\":[%d,%d,%d],"
|
||||||
|
+ "\"previous_prefetch_tiles\":%d,"
|
||||||
|
+ "\"stale_retained_lookahead_tiles\":%d"),
|
||||||
|
+ PreviousRegionLookaheadStep.X, PreviousRegionLookaheadStep.Y,
|
||||||
|
+ PreviousRegionLookaheadStep.Z, RegionLookaheadStep.X,
|
||||||
|
+ RegionLookaheadStep.Y, RegionLookaheadStep.Z,
|
||||||
|
+ PreviousRegionLookaheadTiles.Num(), StaleRetainedLookaheadTiles));
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ LastCoreRegionLookaheadCenter = Center;
|
||||||
|
+ LastCoreRegionLookaheadStep = RegionLookaheadStep;
|
||||||
|
+ LastCoreRegionLookaheadGenerationEpoch = GenerationEpoch;
|
||||||
|
+ }
|
||||||
|
+ else if (GVoxelForgeCoreRegionLookahead == 0)
|
||||||
|
+ {
|
||||||
|
+ CoreRegionLookaheadTiles.Reset();
|
||||||
|
+ LastCoreRegionLookaheadCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
|
||||||
|
+ LastCoreRegionLookaheadStep = FIntVector::ZeroValue;
|
||||||
|
+ LastCoreRegionLookaheadGenerationEpoch = GenerationEpoch;
|
||||||
|
+ }
|
||||||
|
+
|
||||||
|
+ CoreRegionRetainedDesiredTiles = DesiredSorted;
|
||||||
|
TSet<FVoxelTileKey> DesiredSet;
|
||||||
|
+ DesiredSet.Reserve(DesiredSorted.Num() + CoreRegionLookaheadTiles.Num());
|
||||||
|
for (const FVoxelTileKey& Key : DesiredSorted) DesiredSet.Add(Key);
|
||||||
|
+ for (const FVoxelTileKey& Key : CoreRegionLookaheadTiles)
|
||||||
|
+ {
|
||||||
|
+ if (!DesiredSet.Contains(Key))
|
||||||
|
+ {
|
||||||
|
+ DesiredSet.Add(Key);
|
||||||
|
+ CoreRegionRetainedDesiredTiles.Add(Key);
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+
|
||||||
|
+ bool bRetainedSetChanged = false;
|
||||||
|
+ int32 RetainedTileCount = 0;
|
||||||
|
TArray<FVoxelTileKey> MissingRetainedFineTiles;
|
||||||
|
{
|
||||||
|
FScopeLock Lock(&State->RetainedTilesLock);
|
||||||
|
RetainedTileCount = static_cast<int32>(State->RetainedTiles.size());
|
||||||
|
bRetainedSetChanged = DesiredSet.Num() != RetainedTileCount;
|
||||||
|
- for (const FVoxelTileKey& Key : DesiredSorted)
|
||||||
|
+ for (const FVoxelTileKey& Key : CoreRegionRetainedDesiredTiles)
|
||||||
|
{
|
||||||
|
const VoxelForgeCore::TileKey CoreKey{{Key.Coord.X, Key.Coord.Y, Key.Coord.Z}, Key.Level};
|
||||||
|
const bool bAlreadyRetained = State->RetainedTiles.contains(CoreKey);
|
||||||
|
@@ -11313,7 +11504,7 @@ void AVoxelWorld::BuildCoreDesiredTiles(const FIntVector& Center, const FVector&
|
||||||
|
{
|
||||||
|
PendingCoreRegionTileSet.Add(Key);
|
||||||
|
}
|
||||||
|
- PendingCoreRetainedTiles = DesiredSorted;
|
||||||
|
+ PendingCoreRetainedTiles = CoreRegionRetainedDesiredTiles;
|
||||||
|
bCoreRegionCompilePending = true;
|
||||||
|
++CoreRegionCompileRequestSerial;
|
||||||
|
if (VoxelForgeStartupTrace::IsActive())
|
||||||
|
@@ -11322,6 +11513,7 @@ void AVoxelWorld::BuildCoreDesiredTiles(const FIntVector& Center, const FVector&
|
||||||
|
TEXT("\"request_serial\":%d,\"reason\":\"%s\",\"center\":[%d,%d,%d],"
|
||||||
|
"\"previous_center\":[%d,%d,%d],\"center_delta\":[%d,%d,%d],"
|
||||||
|
"\"desired_tiles\":%d,\"retained_tiles\":%d,\"missing_region_tiles\":%d,"
|
||||||
|
+ "\"lookahead_region_tiles\":%d,\"lookahead_step\":[%d,%d,%d],"
|
||||||
|
"\"desired_epoch\":%u,\"active_tile_tasks\":%d"),
|
||||||
|
CoreRegionCompileRequestSerial,
|
||||||
|
!bHadPreviousCenter ? TEXT("initial_desired_region_set")
|
||||||
|
@@ -11332,7 +11524,9 @@ void AVoxelWorld::BuildCoreDesiredTiles(const FIntVector& Center, const FVector&
|
||||||
|
bCenterMoved ? Center.X - PreviousCenter.X : 0,
|
||||||
|
bCenterMoved ? Center.Y - PreviousCenter.Y : 0,
|
||||||
|
bCenterMoved ? Center.Z - PreviousCenter.Z : 0,
|
||||||
|
- DesiredSorted.Num(), RetainedTileCount, PendingCoreRegionTiles.Num(), DesiredEpoch,
|
||||||
|
+ DesiredSorted.Num(), RetainedTileCount, PendingCoreRegionTiles.Num(),
|
||||||
|
+ CoreRegionLookaheadTiles.Num(), RegionLookaheadStep.X, RegionLookaheadStep.Y,
|
||||||
|
+ RegionLookaheadStep.Z, DesiredEpoch,
|
||||||
|
ActiveTileTaskCount.load(std::memory_order_relaxed)));
|
||||||
|
}
|
||||||
|
StartPendingCoreRegionCompile();
|
||||||
|
@@ -11363,13 +11557,27 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||||
|
State->bRegionCompileSucceeded.store(false, std::memory_order_release);
|
||||||
|
const TArray<FVoxelTileKey> MissingTiles = PendingCoreRegionTiles;
|
||||||
|
const TArray<FVoxelTileKey> RetainedTiles = PendingCoreRetainedTiles;
|
||||||
|
+ TSet<FVoxelTileKey> ActualDesiredSet;
|
||||||
|
+ TSet<FVoxelTileKey> PrefetchSet;
|
||||||
|
+ for (const FVoxelTileKey& Key : DesiredSorted) ActualDesiredSet.Add(Key);
|
||||||
|
+ for (const FVoxelTileKey& Key : CoreRegionLookaheadTiles) PrefetchSet.Add(Key);
|
||||||
|
+ TArray<FVoxelTileKey> PrefetchTiles;
|
||||||
|
+ for (const FVoxelTileKey& Key : MissingTiles)
|
||||||
|
+ {
|
||||||
|
+ if (PrefetchSet.Contains(Key) && !ActualDesiredSet.Contains(Key))
|
||||||
|
+ {
|
||||||
|
+ PrefetchTiles.Add(Key);
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
const FIntVector CompileCenter = PendingCoreRegionRequestCenter;
|
||||||
|
const bool bAllowConcurrentGeneration = GVoxelForgeCoreRegionConcurrentAdmission != 0;
|
||||||
|
const uint32 TaskEpoch = GenerationEpoch;
|
||||||
|
const uint32 TaskDesiredEpoch = DesiredEpoch;
|
||||||
|
+ const int32 RequestSerial = CoreRegionCompileRequestSerial;
|
||||||
|
const uint32 WorkerLimit = static_cast<uint32>(FMath::Clamp(GetMaxConcurrentTasks(), 1, 16));
|
||||||
|
ActiveCoreRegionCompileCenter = CompileCenter;
|
||||||
|
ActiveCoreRegionCompileRequestSerial = CoreRegionCompileRequestSerial;
|
||||||
|
+ ActiveCoreRegionWastedPrefetchTiles = 0;
|
||||||
|
ActiveCoreRegionCompileStartSeconds = FPlatformTime::Seconds();
|
||||||
|
ActiveCoreRegionCompileStartCycles = FPlatformTime::Cycles64();
|
||||||
|
CoreRegionCompileTilesInFlight.Reset();
|
||||||
|
@@ -11381,25 +11589,32 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||||
|
1, std::memory_order_relaxed) + 1;
|
||||||
|
State->RegionCompileStartCycles.store(ActiveCoreRegionCompileStartCycles,
|
||||||
|
std::memory_order_release);
|
||||||
|
+ State->RegionCompileWorkerStartCycles.store(0, std::memory_order_release);
|
||||||
|
State->RegionCompileEndCycles.store(0, std::memory_order_release);
|
||||||
|
State->RegionCompileTileCount.store(0, std::memory_order_release);
|
||||||
|
+ State->RegionCompileWorkerThreadId.store(0, std::memory_order_release);
|
||||||
|
State->RegionCompileWorkerSeconds.store(0.0, std::memory_order_release);
|
||||||
|
+ State->RegionCompileWallSeconds.store(0.0, std::memory_order_release);
|
||||||
|
if (VoxelForgeStartupTrace::IsActive())
|
||||||
|
{
|
||||||
|
VoxelForgeStartupTrace::RecordEvent(TEXT("core_region_compile_start"), FString::Printf(
|
||||||
|
TEXT("\"compile_serial\":%d,\"request_serial\":%d,\"center\":[%d,%d,%d],"
|
||||||
|
"\"missing_region_tiles\":%d,\"retained_tiles\":%d,\"concurrent_admission\":%d,"
|
||||||
|
- "\"active_tile_tasks\":%d,\"active_task_guard\":%d"),
|
||||||
|
+ "\"prefetch_compile_tiles\":%d,"
|
||||||
|
+ "\"active_tile_tasks\":%d,\"active_generation_workers\":%d,"
|
||||||
|
+ "\"active_task_guard\":%d,\"game_thread_id\":%u"),
|
||||||
|
CompileSerial, CoreRegionCompileRequestSerial,
|
||||||
|
CompileCenter.X, CompileCenter.Y, CompileCenter.Z,
|
||||||
|
MissingTiles.Num(), RetainedTiles.Num(), bAllowConcurrentGeneration ? 1 : 0,
|
||||||
|
+ PrefetchTiles.Num(),
|
||||||
|
ActiveTileTaskCount.load(std::memory_order_relaxed),
|
||||||
|
- ActiveTaskCount.load(std::memory_order_relaxed)));
|
||||||
|
+ ActiveGenerationWorkerCount.load(std::memory_order_relaxed),
|
||||||
|
+ ActiveTaskCount.load(std::memory_order_relaxed), FPlatformTLS::GetCurrentThreadId()));
|
||||||
|
}
|
||||||
|
ActiveTaskCount.fetch_add(1, std::memory_order_relaxed);
|
||||||
|
UE::Tasks::Launch(TEXT("VoxelForgeCoreTileRegion"),
|
||||||
|
- [this, State, MissingTiles, RetainedTiles, TaskEpoch, TaskDesiredEpoch, WorkerLimit,
|
||||||
|
- bAllowConcurrentGeneration, CompileSerial]()
|
||||||
|
+ [this, State, MissingTiles, RetainedTiles, PrefetchTiles, CompileCenter, TaskEpoch,
|
||||||
|
+ TaskDesiredEpoch, RequestSerial, WorkerLimit, bAllowConcurrentGeneration, CompileSerial]()
|
||||||
|
{
|
||||||
|
struct FTaskGuard
|
||||||
|
{
|
||||||
|
@@ -11407,12 +11622,23 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||||
|
~FTaskGuard() { Counter.fetch_sub(1, std::memory_order_release); }
|
||||||
|
} Guard{ActiveTaskCount};
|
||||||
|
|
||||||
|
+ FCoreRegionCompileResult Result;
|
||||||
|
+ Result.RuntimeState = State;
|
||||||
|
+ Result.GenerationEpoch = TaskEpoch;
|
||||||
|
+ Result.DesiredEpoch = TaskDesiredEpoch;
|
||||||
|
+ Result.Center = CompileCenter;
|
||||||
|
+ Result.RequestSerial = RequestSerial;
|
||||||
|
+ Result.CompileSerial = CompileSerial;
|
||||||
|
+ Result.PrefetchTiles = PrefetchTiles;
|
||||||
|
+ Result.WorkerStartCycles = FPlatformTime::Cycles64();
|
||||||
|
+ Result.WorkerThreadId = FPlatformTLS::GetCurrentThreadId();
|
||||||
|
+
|
||||||
|
if (bShuttingDown.load(std::memory_order_acquire)
|
||||||
|
|| State->bShuttingDown.load(std::memory_order_acquire))
|
||||||
|
{
|
||||||
|
- State->RegionCompileEndCycles.store(FPlatformTime::Cycles64(), std::memory_order_release);
|
||||||
|
- State->bRegionCompileInFlight.store(false, std::memory_order_release);
|
||||||
|
- State->bRegionCompileFinished.store(true, std::memory_order_release);
|
||||||
|
+ Result.bAborted = true;
|
||||||
|
+ Result.WorkerEndCycles = FPlatformTime::Cycles64();
|
||||||
|
+ CoreRegionCompileResults.Enqueue(MoveTemp(Result));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
std::vector<VoxelForgeCore::TileKey> Missing;
|
||||||
|
@@ -11456,38 +11682,14 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||||
|
FWriteScopeLock Lock(State->WorldLock);
|
||||||
|
CompileAndRetain();
|
||||||
|
}
|
||||||
|
- State->RegionCompileWorkerSeconds.store(CompileStats.workerSeconds,
|
||||||
|
- std::memory_order_release);
|
||||||
|
- State->bRegionCompileSucceeded.store(bCompiled, std::memory_order_release);
|
||||||
|
- if (!bCompiled)
|
||||||
|
- {
|
||||||
|
- State->bRegionCompileFailed.store(true, std::memory_order_release);
|
||||||
|
- UE_LOG(LogTemp, Error,
|
||||||
|
- TEXT("[VoxelForgeCoreRuntime] tile-region compile failed epoch=%u desired_epoch=%u: %s"),
|
||||||
|
- TaskEpoch, TaskDesiredEpoch, UTF8_TO_TCHAR(Error.c_str()));
|
||||||
|
- }
|
||||||
|
- else
|
||||||
|
- {
|
||||||
|
- {
|
||||||
|
- FScopeLock RetainedLock(&State->RetainedTilesLock);
|
||||||
|
- State->RetainedTiles.clear();
|
||||||
|
- for (const VoxelForgeCore::TileKey& Key : Retained)
|
||||||
|
- {
|
||||||
|
- State->RetainedTiles.insert(Key);
|
||||||
|
- }
|
||||||
|
- }
|
||||||
|
- State->RegionCompileTileCount.store(
|
||||||
|
- static_cast<int32>(CompiledTileCount), std::memory_order_release);
|
||||||
|
- State->bRebuildDesired.store(true, std::memory_order_release);
|
||||||
|
- UE_LOG(LogTemp, Display,
|
||||||
|
- TEXT("[VoxelForgeCoreRuntime] tile_regions_compiled=%d retained=%d epoch=%u desired_epoch=%u "
|
||||||
|
- "wall_ms=%.3f worker_s=%.3f serial=%d"),
|
||||||
|
- static_cast<int32>(CompiledTileCount), RetainedTiles.Num(), TaskEpoch, TaskDesiredEpoch,
|
||||||
|
- CompileStats.wallSeconds * 1000.0, CompileStats.workerSeconds, CompileSerial);
|
||||||
|
- }
|
||||||
|
- State->RegionCompileEndCycles.store(FPlatformTime::Cycles64(), std::memory_order_release);
|
||||||
|
- State->bRegionCompileInFlight.store(false, std::memory_order_release);
|
||||||
|
- State->bRegionCompileFinished.store(true, std::memory_order_release);
|
||||||
|
+ Result.bSucceeded = bCompiled;
|
||||||
|
+ Result.RetainedTiles = RetainedTiles;
|
||||||
|
+ Result.Error = UTF8_TO_TCHAR(Error.c_str());
|
||||||
|
+ Result.CompiledTileCount = static_cast<int32>(CompiledTileCount);
|
||||||
|
+ Result.WorkerSeconds = CompileStats.workerSeconds;
|
||||||
|
+ Result.CoreWallSeconds = CompileStats.wallSeconds;
|
||||||
|
+ Result.WorkerEndCycles = FPlatformTime::Cycles64();
|
||||||
|
+ CoreRegionCompileResults.Enqueue(MoveTemp(Result));
|
||||||
|
}, UE::Tasks::ETaskPriority::BackgroundNormal);
|
||||||
|
}
|
||||||
|
|
||||||
|
@@ -12331,6 +12533,88 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
RecordSchedulerSkip(TEXT("core_not_ready"));
|
||||||
|
return; // Core mode never falls through to the legacy generator.
|
||||||
|
}
|
||||||
|
+ FCoreRegionCompileResult RegionResult;
|
||||||
|
+ while (CoreRegionCompileResults.Dequeue(RegionResult))
|
||||||
|
+ {
|
||||||
|
+ if (RegionResult.RuntimeState.IsValid())
|
||||||
|
+ {
|
||||||
|
+ RegionResult.RuntimeState->bRegionCompileInFlight.store(
|
||||||
|
+ false, std::memory_order_release);
|
||||||
|
+ }
|
||||||
|
+ if (RegionResult.bAborted
|
||||||
|
+ || RegionResult.RuntimeState != State
|
||||||
|
+ || RegionResult.GenerationEpoch != GenerationEpoch)
|
||||||
|
+ {
|
||||||
|
+ if (VoxelForgeStartupTrace::IsActive())
|
||||||
|
+ {
|
||||||
|
+ VoxelForgeStartupTrace::RecordEvent(TEXT("core_region_compile_discard"),
|
||||||
|
+ FString::Printf(
|
||||||
|
+ TEXT("\"request_serial\":%d,\"compile_serial\":%d,"
|
||||||
|
+ "\"result_epoch\":%u,\"current_epoch\":%u,\"aborted\":%d"),
|
||||||
|
+ RegionResult.RequestSerial, RegionResult.CompileSerial,
|
||||||
|
+ RegionResult.GenerationEpoch, GenerationEpoch,
|
||||||
|
+ RegionResult.bAborted ? 1 : 0));
|
||||||
|
+ }
|
||||||
|
+ continue;
|
||||||
|
+ }
|
||||||
|
+
|
||||||
|
+ State->RegionCompileWorkerStartCycles.store(
|
||||||
|
+ RegionResult.WorkerStartCycles, std::memory_order_release);
|
||||||
|
+ State->RegionCompileEndCycles.store(
|
||||||
|
+ RegionResult.WorkerEndCycles, std::memory_order_release);
|
||||||
|
+ State->RegionCompileTileCount.store(
|
||||||
|
+ RegionResult.CompiledTileCount, std::memory_order_release);
|
||||||
|
+ State->RegionCompileWorkerThreadId.store(
|
||||||
|
+ RegionResult.WorkerThreadId, std::memory_order_release);
|
||||||
|
+ State->RegionCompileWorkerSeconds.store(
|
||||||
|
+ RegionResult.WorkerSeconds, std::memory_order_release);
|
||||||
|
+ State->RegionCompileWallSeconds.store(
|
||||||
|
+ RegionResult.CoreWallSeconds, std::memory_order_release);
|
||||||
|
+ State->bRegionCompileSucceeded.store(
|
||||||
|
+ RegionResult.bSucceeded, std::memory_order_release);
|
||||||
|
+ ActiveCoreRegionCompileCenter = RegionResult.Center;
|
||||||
|
+ ActiveCoreRegionCompileRequestSerial = RegionResult.RequestSerial;
|
||||||
|
+ TSet<FVoxelTileKey> CurrentDesiredSet;
|
||||||
|
+ for (const FVoxelTileKey& Key : DesiredSorted) CurrentDesiredSet.Add(Key);
|
||||||
|
+ ActiveCoreRegionWastedPrefetchTiles = 0;
|
||||||
|
+ for (const FVoxelTileKey& Key : RegionResult.PrefetchTiles)
|
||||||
|
+ {
|
||||||
|
+ if (!CurrentDesiredSet.Contains(Key) && !CoreRegionLookaheadTiles.Contains(Key))
|
||||||
|
+ {
|
||||||
|
+ ++ActiveCoreRegionWastedPrefetchTiles;
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ if (RegionResult.bSucceeded)
|
||||||
|
+ {
|
||||||
|
+ {
|
||||||
|
+ FScopeLock RetainedLock(&State->RetainedTilesLock);
|
||||||
|
+ State->RetainedTiles.clear();
|
||||||
|
+ for (const FVoxelTileKey& Key : RegionResult.RetainedTiles)
|
||||||
|
+ {
|
||||||
|
+ State->RetainedTiles.insert({
|
||||||
|
+ {Key.Coord.X, Key.Coord.Y, Key.Coord.Z}, Key.Level});
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ State->bRebuildDesired.store(true, std::memory_order_release);
|
||||||
|
+ UE_LOG(LogTemp, Display,
|
||||||
|
+ TEXT("[VoxelForgeCoreRuntime] tile_regions_compiled=%d retained=%d "
|
||||||
|
+ "epoch=%u desired_epoch=%u wall_ms=%.3f worker_s=%.3f serial=%d"),
|
||||||
|
+ RegionResult.CompiledTileCount, RegionResult.RetainedTiles.Num(),
|
||||||
|
+ RegionResult.GenerationEpoch, RegionResult.DesiredEpoch,
|
||||||
|
+ RegionResult.CoreWallSeconds * 1000.0, RegionResult.WorkerSeconds,
|
||||||
|
+ RegionResult.CompileSerial);
|
||||||
|
+ }
|
||||||
|
+ else
|
||||||
|
+ {
|
||||||
|
+ State->bRegionCompileFailed.store(true, std::memory_order_release);
|
||||||
|
+ UE_LOG(LogTemp, Error,
|
||||||
|
+ TEXT("[VoxelForgeCoreRuntime] tile-region compile failed epoch=%u "
|
||||||
|
+ "desired_epoch=%u: %s"),
|
||||||
|
+ RegionResult.GenerationEpoch, RegionResult.DesiredEpoch,
|
||||||
|
+ *RegionResult.Error);
|
||||||
|
+ }
|
||||||
|
+ State->bRegionCompileFinished.store(true, std::memory_order_release);
|
||||||
|
+ }
|
||||||
|
if (State->bRebuildDesired.exchange(false, std::memory_order_acq_rel))
|
||||||
|
{
|
||||||
|
bForceDesiredRebuild = true;
|
||||||
|
@@ -12345,9 +12629,17 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
State->RegionCompileStartCycles.load(std::memory_order_acquire);
|
||||||
|
const uint64 CompileEndCycles =
|
||||||
|
State->RegionCompileEndCycles.load(std::memory_order_acquire);
|
||||||
|
+ const uint64 CompileWorkerStartCycles =
|
||||||
|
+ State->RegionCompileWorkerStartCycles.load(std::memory_order_acquire);
|
||||||
|
const double CompileElapsedMilliseconds = CompileEndCycles >= CompileStartCycles
|
||||||
|
? FPlatformTime::ToSeconds64(CompileEndCycles - CompileStartCycles) * 1000.0
|
||||||
|
: 0.0;
|
||||||
|
+ const double CompileQueueMilliseconds = CompileWorkerStartCycles >= CompileStartCycles
|
||||||
|
+ ? FPlatformTime::ToSeconds64(CompileWorkerStartCycles - CompileStartCycles) * 1000.0
|
||||||
|
+ : 0.0;
|
||||||
|
+ const double CompileTaskMilliseconds = CompileEndCycles >= CompileWorkerStartCycles
|
||||||
|
+ ? FPlatformTime::ToSeconds64(CompileEndCycles - CompileWorkerStartCycles) * 1000.0
|
||||||
|
+ : 0.0;
|
||||||
|
const double GateElapsedMilliseconds = CoreRegionPendingSinceSeconds > 0.0
|
||||||
|
? FMath::Max(0.0, FPlatformTime::Seconds() - CoreRegionPendingSinceSeconds) * 1000.0
|
||||||
|
: 0.0;
|
||||||
|
@@ -12355,6 +12647,10 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
State->RegionCompileTileCount.load(std::memory_order_acquire);
|
||||||
|
const double CompileWorkerSeconds =
|
||||||
|
State->RegionCompileWorkerSeconds.load(std::memory_order_acquire);
|
||||||
|
+ const double CompileCoreWallSeconds =
|
||||||
|
+ State->RegionCompileWallSeconds.load(std::memory_order_acquire);
|
||||||
|
+ const uint32 CompileWorkerThreadId =
|
||||||
|
+ State->RegionCompileWorkerThreadId.load(std::memory_order_acquire);
|
||||||
|
const int32 CompileSerial =
|
||||||
|
State->RegionCompileSerial.load(std::memory_order_acquire);
|
||||||
|
const uint64 IndependentSubmissions = CoreRegionIndependentSubmissions;
|
||||||
|
@@ -12364,16 +12660,23 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
VoxelForgeStartupTrace::RecordEvent(TEXT("core_region_compile_complete"), FString::Printf(
|
||||||
|
TEXT("\"compile_serial\":%d,\"request_serial\":%d,\"center\":[%d,%d,%d],"
|
||||||
|
"\"success\":%d,\"compiled_tiles\":%d,\"compile_elapsed_ms\":%s,"
|
||||||
|
- "\"compile_worker_s\":%s,\"pending_gate_elapsed_ms\":%s,"
|
||||||
|
- "\"independent_submissions\":%llu,"
|
||||||
|
+ "\"compile_queue_ms\":%s,\"compile_task_ms\":%s,"
|
||||||
|
+ "\"compile_core_wall_ms\":%s,\"compile_worker_s\":%s,"
|
||||||
|
+ "\"compile_worker_thread_id\":%u,\"pending_gate_elapsed_ms\":%s,"
|
||||||
|
+ "\"independent_submissions\":%llu,\"wasted_prefetch_tiles\":%d,"
|
||||||
|
"\"dependent_declines\":%llu,\"concurrent_admission\":%d"),
|
||||||
|
CompileSerial, ActiveCoreRegionCompileRequestSerial,
|
||||||
|
ActiveCoreRegionCompileCenter.X, ActiveCoreRegionCompileCenter.Y,
|
||||||
|
ActiveCoreRegionCompileCenter.Z, bRegionCompileSucceeded ? 1 : 0,
|
||||||
|
CompiledTileCount, *FString::SanitizeFloat(CompileElapsedMilliseconds),
|
||||||
|
+ *FString::SanitizeFloat(CompileQueueMilliseconds),
|
||||||
|
+ *FString::SanitizeFloat(CompileTaskMilliseconds),
|
||||||
|
+ *FString::SanitizeFloat(CompileCoreWallSeconds * 1000.0),
|
||||||
|
*FString::SanitizeFloat(CompileWorkerSeconds),
|
||||||
|
+ CompileWorkerThreadId,
|
||||||
|
*FString::SanitizeFloat(GateElapsedMilliseconds),
|
||||||
|
static_cast<unsigned long long>(IndependentSubmissions),
|
||||||
|
+ ActiveCoreRegionWastedPrefetchTiles,
|
||||||
|
static_cast<unsigned long long>(DependentDeclines),
|
||||||
|
GVoxelForgeCoreRegionConcurrentAdmission != 0 ? 1 : 0));
|
||||||
|
}
|
||||||
|
@@ -12388,8 +12691,9 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
{
|
||||||
|
FScopeLock RetainedLock(&State->RetainedTilesLock);
|
||||||
|
bRetainedSetMatchesDesired =
|
||||||
|
- DesiredSorted.Num() == static_cast<int32>(State->RetainedTiles.size());
|
||||||
|
- for (const FVoxelTileKey& Key : DesiredSorted)
|
||||||
|
+ CoreRegionRetainedDesiredTiles.Num()
|
||||||
|
+ == static_cast<int32>(State->RetainedTiles.size());
|
||||||
|
+ for (const FVoxelTileKey& Key : CoreRegionRetainedDesiredTiles)
|
||||||
|
{
|
||||||
|
const VoxelForgeCore::TileKey CoreKey{
|
||||||
|
{Key.Coord.X, Key.Coord.Y, Key.Coord.Z}, Key.Level};
|
||||||
|
@@ -12431,7 +12735,7 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
{
|
||||||
|
PendingCoreRegionTileSet.Add(Key);
|
||||||
|
}
|
||||||
|
- PendingCoreRetainedTiles = DesiredSorted;
|
||||||
|
+ PendingCoreRetainedTiles = CoreRegionRetainedDesiredTiles;
|
||||||
|
PendingCoreRegionRequestCenter = DesiredEpochCenterChunk;
|
||||||
|
bCoreRegionCompilePending = true;
|
||||||
|
StartPendingCoreRegionCompile();
|
||||||
|
@@ -12584,6 +12888,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
&& bWalkableRingReadyForReserve;
|
||||||
|
int32 SchedulerSubmittedByLevel[9] = {};
|
||||||
|
int32 SchedulerLoadAttemptsByLevel[9] = {};
|
||||||
|
+ int32 SchedulerRegionCompileDeclinesByLevel[9] = {};
|
||||||
|
+ int32 SchedulerFarParentDeclinesByLevel[9] = {};
|
||||||
|
int32 SchedulerLevelCapReturns[9] = {};
|
||||||
|
int32 SchedulerLevelCapBreaks[9] = {};
|
||||||
|
int32 SchedulerCriticalFullCapBreaks = 0;
|
||||||
|
@@ -12951,7 +13257,21 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
if(CoreState.IsValid()
|
||||||
|
&&CoreState->bFarWorldRequired.load(std::memory_order_acquire)
|
||||||
|
&&!CoreState->bFarWorldReady.load(std::memory_order_acquire))
|
||||||
|
+ {
|
||||||
|
+ if (bTraceStreamingScheduler && Tile.Level >= 0 && Tile.Level < 9)
|
||||||
|
+ {
|
||||||
|
+ ++SchedulerFarParentDeclinesByLevel[Tile.Level];
|
||||||
|
+ }
|
||||||
|
return;
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ if (bTraceStreamingScheduler && bUseVoxelForgeCore && bCoreRegionCompilePending
|
||||||
|
+ && GVoxelForgeCoreRegionConcurrentAdmission != 0
|
||||||
|
+ && (PendingCoreRegionTileSet.Contains(Tile)
|
||||||
|
+ || CoreRegionCompileTilesInFlight.Contains(Tile))
|
||||||
|
+ && Tile.Level >= 0 && Tile.Level < 9)
|
||||||
|
+ {
|
||||||
|
+ ++SchedulerRegionCompileDeclinesByLevel[Tile.Level];
|
||||||
|
}
|
||||||
|
if (Tile.Level >= 0 && Tile.Level < 9)
|
||||||
|
{
|
||||||
|
@@ -13661,10 +13981,37 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
const double BusySeconds = FPlatformTime::ToSeconds64(WorkerCyclesDelta);
|
||||||
|
const double SchedulerUpdateMilliseconds = (SchedulerTraceNow
|
||||||
|
- StreamingUpdateStartSeconds) * 1000.0;
|
||||||
|
+ int32 CriticalMissingTiles = 0;
|
||||||
|
+ for (const FVoxelTileKey& Tile : CriticalDesiredTiles)
|
||||||
|
+ {
|
||||||
|
+ if (!LoadedTiles.Contains(Tile) && !CollisionNotRequiredTiles.Contains(Tile))
|
||||||
|
+ {
|
||||||
|
+ ++CriticalMissingTiles;
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ int32 CollisionGateMissingTiles = 0;
|
||||||
|
+ for (const FVoxelTileKey& Tile : CollisionGateDesiredTiles)
|
||||||
|
+ {
|
||||||
|
+ if (!LoadedTiles.Contains(Tile) && !CollisionNotRequiredTiles.Contains(Tile))
|
||||||
|
+ {
|
||||||
|
+ ++CollisionGateMissingTiles;
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ const FIntVector DesiredCenterLag = DesiredEpochCenterChunk - CenterChunk;
|
||||||
|
+ const TSharedPtr<FVoxelForgeCoreRuntimeState, ESPMode::ThreadSafe> TraceCoreState =
|
||||||
|
+ VoxelForgeCoreState;
|
||||||
|
+ const bool bFarWorldRequiredForTrace = TraceCoreState.IsValid()
|
||||||
|
+ && TraceCoreState->bFarWorldRequired.load(std::memory_order_relaxed);
|
||||||
|
+ const bool bFarWorldReadyForTrace = TraceCoreState.IsValid()
|
||||||
|
+ && TraceCoreState->bFarWorldReady.load(std::memory_order_relaxed);
|
||||||
|
VoxelForgeStartupTrace::RecordEvent(TEXT("scheduler_admission"), FString::Printf(
|
||||||
|
TEXT("\"frame\":%llu,\"center\":[%d,%d,%d],\"desired_rebuilt\":%d,"
|
||||||
|
"\"submission_loop_ran\":%d,\"all_chunks_loaded\":%d,"
|
||||||
|
"\"walkable_ready\":%d,\"moving\":%d,"
|
||||||
|
+ "\"pawn_collision_gate_engaged\":%d,\"critical_tiles_ready\":%d,"
|
||||||
|
+ "\"critical_missing\":%d,\"collision_gate_missing\":%d,"
|
||||||
|
+ "\"desired_center_lag\":[%d,%d,%d],\"desired_center_current\":%d,"
|
||||||
|
+ "\"far_world_required\":%d,\"far_world_ready\":%d,"
|
||||||
|
"\"core_region_compile_pending\":%d,\"core_ready\":%d,\"coarse_first\":%d,"
|
||||||
|
"\"dynamic_near_reserve\":%d,\"legacy_pending_result_cap\":%d,"
|
||||||
|
"\"core_region_concurrent_admission\":%d,"
|
||||||
|
@@ -13685,6 +14032,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
"\"level_candidate_scan_ms\":%s,\"passage_score_ms\":%s,"
|
||||||
|
"\"level_sort_ms\":%s,\"level_admit_ms\":%s,"
|
||||||
|
"\"desired_missing_by_lod\":%s,\"loaded_by_lod\":%s,\"pending_by_lod\":%s,"
|
||||||
|
+ "\"region_compile_declines_by_lod\":%s,"
|
||||||
|
+ "\"far_parent_declines_by_lod\":%s,"
|
||||||
|
"\"submitted_by_lod\":%s,\"load_attempts_by_lod\":%s,"
|
||||||
|
"\"admission_cap_by_lod\":%s,\"declined_rule_by_lod\":%s,"
|
||||||
|
"\"level_cap_returns_by_lod\":%s,\"level_cap_breaks_by_lod\":%s,"
|
||||||
|
@@ -13695,6 +14044,11 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
static_cast<unsigned long long>(GFrameCounter), CenterChunk.X, CenterChunk.Y, CenterChunk.Z,
|
||||||
|
bDesiredRebuiltThisUpdate ? 1 : 0, bSubmissionLoopRan ? 1 : 0, bAllChunksLoaded ? 1 : 0,
|
||||||
|
bWalkableRingReadyForReserve ? 1 : 0, bPawnMoving ? 1 : 0,
|
||||||
|
+ bPawnCollisionGateEngaged ? 1 : 0, CriticalMissingTiles == 0 ? 1 : 0,
|
||||||
|
+ CriticalMissingTiles, CollisionGateMissingTiles,
|
||||||
|
+ DesiredCenterLag.X, DesiredCenterLag.Y, DesiredCenterLag.Z,
|
||||||
|
+ DesiredCenterLag == FIntVector::ZeroValue ? 1 : 0,
|
||||||
|
+ bFarWorldRequiredForTrace ? 1 : 0, bFarWorldReadyForTrace ? 1 : 0,
|
||||||
|
bCoreRegionCompilePending ? 1 : 0, IsVoxelForgeCoreReady() ? 1 : 0,
|
||||||
|
GVoxelForgeStreamingCoarseFirst, GVoxelForgeStreamingDynamicNearReserve,
|
||||||
|
GVoxelForgeStreamingLegacyPendingResultCap,
|
||||||
|
@@ -13721,7 +14075,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||||
|
*FString::SanitizeFloat(LevelSortMilliseconds),
|
||||||
|
*FString::SanitizeFloat(LevelAdmissionMilliseconds),
|
||||||
|
*IntArrayJson(DesiredMissingByLevel), *IntArrayJson(LoadedByLevel),
|
||||||
|
- *IntArrayJson(PendingByLevel), *IntArrayJson(SchedulerSubmittedByLevel),
|
||||||
|
+ *IntArrayJson(PendingByLevel), *IntArrayJson(SchedulerRegionCompileDeclinesByLevel),
|
||||||
|
+ *IntArrayJson(SchedulerFarParentDeclinesByLevel), *IntArrayJson(SchedulerSubmittedByLevel),
|
||||||
|
*IntArrayJson(SchedulerLoadAttemptsByLevel), *IntArrayJson(AdmissionCapByLevel),
|
||||||
|
*IntArrayJson(DeclinedRuleByLevel), *IntArrayJson(SchedulerLevelCapReturns),
|
||||||
|
*IntArrayJson(SchedulerLevelCapBreaks), FullCapDeclinedTiles, ReserveDeclinedTiles,
|
||||||
|
diff --git a/Source/VoxelForge/Public/VoxelWorld.h b/Source/VoxelForge/Public/VoxelWorld.h
|
||||||
|
index b9fa34c..5979f29 100644
|
||||||
|
--- a/Source/VoxelForge/Public/VoxelWorld.h
|
||||||
|
+++ b/Source/VoxelForge/Public/VoxelWorld.h
|
||||||
|
@@ -207,6 +207,28 @@ struct FChunkResult
|
||||||
|
TArray<FVector> SurfaceWitnesses;
|
||||||
|
};
|
||||||
|
|
||||||
|
+/** Worker-to-game-thread completion for immutable Core tile-region publication. */
|
||||||
|
+struct FCoreRegionCompileResult
|
||||||
|
+{
|
||||||
|
+ TSharedPtr<FVoxelForgeCoreRuntimeState, ESPMode::ThreadSafe> RuntimeState;
|
||||||
|
+ TArray<FVoxelTileKey> RetainedTiles;
|
||||||
|
+ TArray<FVoxelTileKey> PrefetchTiles;
|
||||||
|
+ FString Error;
|
||||||
|
+ FIntVector Center = FIntVector::ZeroValue;
|
||||||
|
+ uint32 GenerationEpoch = 0;
|
||||||
|
+ uint32 DesiredEpoch = 0;
|
||||||
|
+ uint64 WorkerStartCycles = 0;
|
||||||
|
+ uint64 WorkerEndCycles = 0;
|
||||||
|
+ uint32 WorkerThreadId = 0;
|
||||||
|
+ int32 CompileSerial = 0;
|
||||||
|
+ int32 RequestSerial = 0;
|
||||||
|
+ int32 CompiledTileCount = 0;
|
||||||
|
+ double WorkerSeconds = 0.0;
|
||||||
|
+ double CoreWallSeconds = 0.0;
|
||||||
|
+ bool bSucceeded = false;
|
||||||
|
+ bool bAborted = false;
|
||||||
|
+};
|
||||||
|
+
|
||||||
|
/** One worker-produced mesh identity emitted only by the explicit parity harness. */
|
||||||
|
struct FVoxelTileHashDumpRecord
|
||||||
|
{
|
||||||
|
@@ -798,6 +820,11 @@ private:
|
||||||
|
TArray<FVoxelTileKey> PendingCoreRegionTiles;
|
||||||
|
TSet<FVoxelTileKey> PendingCoreRegionTileSet;
|
||||||
|
TArray<FVoxelTileKey> PendingCoreRetainedTiles;
|
||||||
|
+ TArray<FVoxelTileKey> CoreRegionRetainedDesiredTiles;
|
||||||
|
+ TArray<FVoxelTileKey> CoreRegionLookaheadTiles;
|
||||||
|
+ FIntVector LastCoreRegionLookaheadCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
|
||||||
|
+ FIntVector LastCoreRegionLookaheadStep = FIntVector::ZeroValue;
|
||||||
|
+ uint32 LastCoreRegionLookaheadGenerationEpoch = 0;
|
||||||
|
TSet<FVoxelTileKey> CoreRegionCompileTilesInFlight;
|
||||||
|
FIntVector LastCoreRegionRequestCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
|
||||||
|
FIntVector PendingCoreRegionRequestCenter = FIntVector::ZeroValue;
|
||||||
|
@@ -810,6 +837,7 @@ private:
|
||||||
|
uint64 CoreRegionDependentDeclines = 0;
|
||||||
|
int32 CoreRegionCompileRequestSerial = 0;
|
||||||
|
int32 ActiveCoreRegionCompileRequestSerial = 0;
|
||||||
|
+ int32 ActiveCoreRegionWastedPrefetchTiles = 0;
|
||||||
|
|
||||||
|
public:
|
||||||
|
|
||||||
|
@@ -1139,6 +1167,7 @@ public:
|
||||||
|
TQueue<FChunkResult, EQueueMode::Mpsc> EditedProcessQueue;
|
||||||
|
TQueue<FChunkResult, EQueueMode::Mpsc> NearProcessQueue;
|
||||||
|
TQueue<FChunkResult, EQueueMode::Mpsc> ProcessQueue;
|
||||||
|
+ TQueue<FCoreRegionCompileResult, EQueueMode::Mpsc> CoreRegionCompileResults;
|
||||||
|
// The cap counts completed MPSC results plus admitted tile tasks, so a slow game-thread apply
|
||||||
|
// cannot accumulate an unbounded payload queue while completed work never consumes worker slots.
|
||||||
|
// Le plafond compte les resultats MPSC et les taches admises; un apply lent ne peut donc pas
|
||||||
|
@@ -1362,6 +1391,9 @@ public:
|
||||||
|
float HeadlessStreamingTestSpeedCmPerSecond = 0.0f;
|
||||||
|
float HeadlessStreamingTestStartSeconds = 0.0f;
|
||||||
|
float HeadlessStreamingTestMoveDurationSeconds = 0.0f;
|
||||||
|
+ float HeadlessStreamingTestTurnAfterSeconds = 0.0f;
|
||||||
|
+ bool bHeadlessStreamingTestTurnDone = false;
|
||||||
|
+ FVector HeadlessStreamingTestTurnDirection = FVector::YAxisVector;
|
||||||
|
float HeadlessStreamingTestExitSeconds = 0.0f;
|
||||||
|
float HeadlessStreamingTestTeleportAtSeconds = 0.0f;
|
||||||
|
float HeadlessStreamingTestTeleportDistanceCm = 0.0f;
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
# L124 — early arrival tiles
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Measured **not worth shipping** the design that independently builds pages for the nearest 27 LOD0 tiles. The existing shape gate is real, but this page set can remove only a small amount of work from the LOD0–2 completion path. Its estimated best case is below 100 ms for every stratum/worker pair except the unadjusted FI W8 tile-work ceiling; after page work, the largest estimate is 62 ms. VS is negative. No second design or runtime change was pursued.
|
||||||
|
|
||||||
|
## Current L80 timings
|
||||||
|
|
||||||
|
Five interleaved W8/W12 runs per stratum, Clang 19, policy 3, L80 protocol, source stamp `3f48d5ef6dfd7e735afcd824b742f997179810acd3ca76601a99c1131efb482e`. Values below are medians. `Near ready` is the complete LOD0–2 set.
|
||||||
|
|
||||||
|
The host load was 0.5–1.7 before baseline timing. The unchanged headless binary was built first; no build overlapped a timing run.
|
||||||
|
|
||||||
|
| Stratum | LOD0–2 tiles | Shape wall W8/W12 | First tile ready W8/W12 | Near ready W8/W12 |
|
||||||
|
|---|---:|---:|---:|---:|
|
||||||
|
| SW | 1,408 | 208/204 ms | 280/281 ms | 709/690 ms |
|
||||||
|
| FI | 1,216 | 182/179 ms | 261/261 ms | 1,441/1,329 ms |
|
||||||
|
| TN | 1,344 | 205/210 ms | 284/275 ms | 1,372/1,243 ms |
|
||||||
|
| VS | 1,344 | 226/221 ms | 291/283 ms | 992/835 ms |
|
||||||
|
|
||||||
|
A separate cold timeline trace put shape-build start at 58–60 ms after process start and shape ready at 233–279 ms. The first LOD0–2 admission followed 0.65–0.70 ms later; there were zero LOD0–2 admissions before shape ready. The exact per-run records are in `Saved/Linux/l124/timing/` and `Saved/Linux/l124/workers/`.
|
||||||
|
|
||||||
|
## What a tile reads
|
||||||
|
|
||||||
|
`World::compile` publishes `shapeReady` only after `FStandaloneShapeWorld::Build` finishes. Shape-dependent `generateTile` calls wait for that publication. A tile’s mesher uses `ClassifyBoxDetailed`, lattice density queries, and off-grid `DensityAtUnclamped` queries for gradients and seam normals. Density dispatch can read native surface/island/lattice fields, or packed `RockWords`, `CandidateWords`, and `SurfaceDistanceQ`; fallback queries use analytic density. Composition also reads stratum transitions, cavern fields, stamps/edits, and passage carve data. The tile query scope includes a one-step normal halo and selects passage features, while passage void-crossing plans and primitive/field indices must already be valid.
|
||||||
|
|
||||||
|
An exact local page is feasible: the existing full page builder constructs those dependencies. A temporary probe built a 128-voxel LOD0 page independently after base compilation and compared the center tile’s density, mesh, payload, and topology hashes. All four matched for SW, FI, TN, and VS at W8 and W12. This covered one tile per fixture, not the full 27-tile ring. The normal `compileTileRegion` path still requires the base shape and forwards page queries to it, so it cannot currently publish that page early.
|
||||||
|
|
||||||
|
## Cost count
|
||||||
|
|
||||||
|
An exact 128-voxel page took a median 7.5–12.6 ms wall per page (three interleaved W8/W12 probes); reported parallel-stage worker time was about 27–63 ms. The nearest 27 LOD0 tiles span 1 SW page, 8 FI pages, 1 TN page, and 4 VS pages. Their measured tile-service CPU and an optimistic cost-adjusted budget are:
|
||||||
|
|
||||||
|
| Stratum | Ring tile CPU W8/W12 | Estimated net budget W8/W12 |
|
||||||
|
|---|---:|---:|
|
||||||
|
| SW | 439/770 ms | 51/62 ms |
|
||||||
|
| FI | 818/1,134 ms | 41/57 ms |
|
||||||
|
| TN | 402/577 ms | 43/43 ms |
|
||||||
|
| VS | 96/175 ms | −20/−5 ms |
|
||||||
|
|
||||||
|
The budget subtracts reported page worker time for the required page count, then divides by stream workers. That counter excludes local primitive-build CPU. It also assumes all 27 tile jobs leave the post-shape queue and ignores contention, so it is an optimistic estimate, not an observed A/B speedup.
|
||||||
|
|
||||||
|
During the base build, stream workers were idle for 8.00/8 or 12.00/12 worker-equivalents. Shape helper stages averaged 7.1–7.97 active workers of the configured 8; detail work alone occupied 52–87 ms wall. Whole-build process use averaged 4.6–5.3 of the 12 pinned CPUs. Early page and tile work would overlap the periods where the shape helper pool is full. Given the sub-100 ms estimate and VS’s negative budget, I stopped before adding publication and scheduling code.
|
||||||
|
|
||||||
|
## Validation and artifacts
|
||||||
|
|
||||||
|
The temporary page-cost/equivalence hooks were removed and the headless binary rebuilt against the unchanged L123 source stamp. No 24/24 hash matrix, unpublished-page poison assertion, 30 m/s trace, fast tier, or Linux UE candidate timing was run because no runtime change was retained. The independent-page hash probe was only a viability check after full shape publication. Baseline logs are in `Saved/Linux/l124/{timing,workers,page-cost,page-equivalence}/`.
|
||||||
@@ -0,0 +1,148 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Five interleaved L80-protocol W8/W12 cold-readiness pairs per owner stratum."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import statistics
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
BIN = ROOT / "Saved/Linux/l124/bin/VoxelForgeCoreHeadless-l124-clang"
|
||||||
|
OUT = ROOT / "Saved/Linux/l124/timing"
|
||||||
|
STAMP_TOOL = ROOT / "Standalone/Tools/verify/source_stamp.py"
|
||||||
|
CAMERAS = {
|
||||||
|
"sw": (8.0, 16.0, -372.2626953125),
|
||||||
|
"fi": (25.8369140625, 3.19921875, -737.0),
|
||||||
|
"tn": (19.166015625, -14.2001953125, -753.3740234375),
|
||||||
|
"vs": (-20.0625, -5.068359375, -900.353515625),
|
||||||
|
}
|
||||||
|
FIELDS = re.compile(r"([a-zA-Z0-9_+.-]+)=([^ ]+)")
|
||||||
|
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M)
|
||||||
|
SHAPE_FREE = re.compile(
|
||||||
|
r"^\[ShapeReadiness\] shape_free_desired=(\d+)/(\d+) near_ring=(\d+)/27",
|
||||||
|
re.M,
|
||||||
|
)
|
||||||
|
PROFILE = re.compile(r"^\[ShapeProfile\] (.*)$", re.M)
|
||||||
|
STAGE = re.compile(r"^\[ShapeStage\] (.*)$", re.M)
|
||||||
|
MILESTONE = re.compile(r"^\[PipelineMilestone\] (.*)$", re.M)
|
||||||
|
NEAR = re.compile(
|
||||||
|
r"^\[WorldAssemblerNearLODReady\] lods=0-2 tiles=(\d+)/(\d+) cold_ready_s=([0-9.]+)$",
|
||||||
|
re.M,
|
||||||
|
)
|
||||||
|
PIPELINE = re.compile(r"^\[StreamPipeline\] (.*)$", re.M)
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, workers: int, phase: str, replicate: int) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
label = f"{stratum}-w{workers}-{phase}{replicate}"
|
||||||
|
log = OUT / f"{label}.log"
|
||||||
|
camera = ",".join(f"{value:.12f}" for value in CAMERAS[stratum]) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", str(BIN), str(DATA), "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", f"--workers={workers}",
|
||||||
|
"--clip-radius=3", "--max-clip-level=8", "--max-level=8",
|
||||||
|
f"--camera={camera}", "--profile-lod-work", "--stream-scheduler-policy=3",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as output:
|
||||||
|
result = subprocess.run(command, cwd=ROOT, stdout=output,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
content = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
near, pipeline_match = NEAR.search(content), PIPELINE.search(content)
|
||||||
|
profile, stage, free = PROFILE.search(content), STAGE.search(content), SHAPE_FREE.search(content)
|
||||||
|
stamp = STAMP.search(content)
|
||||||
|
milestones = dict(FIELDS.findall(MILESTONE.search(content).group(1))) if MILESTONE.search(content) else {}
|
||||||
|
if result.returncode or near is None or pipeline_match is None or not all((profile, stage, free, stamp)):
|
||||||
|
raise RuntimeError(f"baseline run incomplete: {log}; exit={result.returncode}")
|
||||||
|
if stamp.group(1) != EXPECTED_STAMP:
|
||||||
|
raise RuntimeError(f"source stamp mismatch: {log}; got={stamp.group(1)} expected={EXPECTED_STAMP}")
|
||||||
|
pipeline = dict(FIELDS.findall(pipeline_match.group(1)))
|
||||||
|
profile_values = dict(FIELDS.findall(profile.group(1)))
|
||||||
|
stage_values = dict(FIELDS.findall(stage.group(1)))
|
||||||
|
row = {
|
||||||
|
"stratum": stratum,
|
||||||
|
"workers": workers,
|
||||||
|
"phase": phase,
|
||||||
|
"replicate": replicate,
|
||||||
|
"exit_code": result.returncode,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"source_stamp": stamp.group(1),
|
||||||
|
"shape_free_tiles": int(free.group(1)),
|
||||||
|
"desired_tiles": int(free.group(2)),
|
||||||
|
"shape_free_near_ring": int(free.group(3)),
|
||||||
|
"shape_compile_s": float(pipeline["shape"].removesuffix("s")),
|
||||||
|
"first_tile_ready_s": float(pipeline["first_tile_ready"].removesuffix("s")),
|
||||||
|
"near_ready_s": float(near.group(3)),
|
||||||
|
"near_ready_tiles": int(near.group(1)),
|
||||||
|
"near_desired_tiles": int(near.group(2)),
|
||||||
|
"worker_shape_stages_ms": {
|
||||||
|
key: float(value.removesuffix("ms"))
|
||||||
|
for key, value in profile_values.items() if key.endswith("_worker")
|
||||||
|
},
|
||||||
|
"shape_stage_wall_ms": {
|
||||||
|
key: float(value)
|
||||||
|
for key, value in stage_values.items()
|
||||||
|
if key in {"local-shapes", "intervals+signs", "surface-band", "far-parents", "total"}
|
||||||
|
},
|
||||||
|
"milestones": milestones,
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L124HeadlessRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
global EXPECTED_STAMP
|
||||||
|
expected = subprocess.run(
|
||||||
|
["python3", "-B", str(STAMP_TOOL), "stamp", "--root", str(ROOT)],
|
||||||
|
cwd=ROOT, text=True, stdout=subprocess.PIPE, check=True,
|
||||||
|
).stdout.strip()
|
||||||
|
EXPECTED_STAMP = expected
|
||||||
|
if not DATA.is_file() or not BIN.is_file():
|
||||||
|
raise SystemExit(f"missing input: {DATA} or {BIN}")
|
||||||
|
rows: list[dict] = []
|
||||||
|
for stratum in CAMERAS:
|
||||||
|
for workers in (8, 12):
|
||||||
|
run(stratum, workers, "warmup", 0)
|
||||||
|
orders = ((8, 12), (12, 8), (8, 12), (12, 8), (8, 12))
|
||||||
|
for stratum in CAMERAS:
|
||||||
|
for replicate, order in enumerate(orders, 1):
|
||||||
|
paired = {}
|
||||||
|
for workers in order:
|
||||||
|
row = run(stratum, workers, "measured", replicate)
|
||||||
|
rows.append(row)
|
||||||
|
paired[workers] = row
|
||||||
|
(OUT / "runs-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print("[L124HeadlessPair] " + json.dumps({
|
||||||
|
"stratum": stratum,
|
||||||
|
"replicate": replicate,
|
||||||
|
"w8_near_ready_s": paired[8]["near_ready_s"],
|
||||||
|
"w12_near_ready_s": paired[12]["near_ready_s"],
|
||||||
|
}, sort_keys=True), flush=True)
|
||||||
|
summary = []
|
||||||
|
for stratum in CAMERAS:
|
||||||
|
entry = {"stratum": stratum, "interleaved_pairs": 5}
|
||||||
|
for workers in (8, 12):
|
||||||
|
subset = [r for r in rows if r["stratum"] == stratum and r["workers"] == workers]
|
||||||
|
entry[f"w{workers}_near_ready_s"] = [r["near_ready_s"] for r in subset]
|
||||||
|
entry[f"w{workers}_near_ready_median_s"] = statistics.median(
|
||||||
|
r["near_ready_s"] for r in subset)
|
||||||
|
entry[f"w{workers}_shape_s"] = [r["shape_compile_s"] for r in subset]
|
||||||
|
entry[f"w{workers}_shape_median_s"] = statistics.median(
|
||||||
|
r["shape_compile_s"] for r in subset)
|
||||||
|
summary.append(entry)
|
||||||
|
(OUT / "summary.json").write_text(
|
||||||
|
json.dumps({"source_stamp": EXPECTED_STAMP, "runs": rows, "summary": summary}, indent=2) + "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,151 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Collect the separate worker-idle and admission timeline for L124's unchanged headless path."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
BIN = ROOT / "Saved/Linux/l124/bin/VoxelForgeCoreHeadless-l124-clang"
|
||||||
|
OUT = ROOT / "Saved/Linux/l124/workers"
|
||||||
|
STAMP_TOOL = ROOT / "Standalone/Tools/verify/source_stamp.py"
|
||||||
|
CAMERAS = {
|
||||||
|
"sw": (8.0, 16.0, -372.2626953125),
|
||||||
|
"fi": (25.8369140625, 3.19921875, -737.0),
|
||||||
|
"tn": (19.166015625, -14.2001953125, -753.3740234375),
|
||||||
|
"vs": (-20.0625, -5.068359375, -900.353515625),
|
||||||
|
}
|
||||||
|
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M)
|
||||||
|
COLD = re.compile(
|
||||||
|
r"^\[ColdTimeline\] event=(\S+) edge=(\S+) at_ms=([0-9.]+) cpu_ms=([0-9.]+)$",
|
||||||
|
re.M,
|
||||||
|
)
|
||||||
|
TIMELINE = re.compile(
|
||||||
|
r"^\[CpuTileTimeline\] event=(\S+) key=\(([-0-9]+),([-0-9]+),([-0-9]+),lod(\d+)\) "
|
||||||
|
r"at_ms=([0-9.-]+) frame=\d+ worker_id=(-?\d+) queue_depth=(\d+) "
|
||||||
|
r"active_workers=(\d+) priority_rank=(\d+)", re.M,
|
||||||
|
)
|
||||||
|
PROFILE = re.compile(r"^\[ShapeProfile\] (.*)$", re.M)
|
||||||
|
BREAKDOWN = re.compile(r"^\[ShapeBreakdown\] (.*)$", re.M)
|
||||||
|
NEAR = re.compile(
|
||||||
|
r"^\[WorldAssemblerNearLODReady\] lods=0-2 tiles=(\d+)/(\d+) cold_ready_s=([0-9.]+)$",
|
||||||
|
re.M,
|
||||||
|
)
|
||||||
|
PAIR = re.compile(r"([a-zA-Z0-9_+.-]+)=([0-9.]+)(?:ms)?")
|
||||||
|
|
||||||
|
|
||||||
|
def intervals(events: list[tuple[float, float]], start: float, end: float) -> float:
|
||||||
|
return sum(max(0.0, min(finish, end) - max(begin, start))
|
||||||
|
for begin, finish in events)
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, workers: int, expected_stamp: str) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = OUT / f"{stratum}-w{workers}.log"
|
||||||
|
camera = ",".join(f"{value:.12f}" for value in CAMERAS[stratum]) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", str(BIN), str(DATA), "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", f"--workers={workers}",
|
||||||
|
"--clip-radius=3", "--max-clip-level=8", "--max-level=8",
|
||||||
|
f"--camera={camera}", "--profile-lod-work", "--stream-scheduler-policy=3",
|
||||||
|
"--cold-worker-timeline",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as output:
|
||||||
|
result = subprocess.run(command, cwd=ROOT, stdout=output,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
content = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
stamp = STAMP.search(content)
|
||||||
|
cold = {(event, edge): (float(at), float(cpu))
|
||||||
|
for event, edge, at, cpu in COLD.findall(content)}
|
||||||
|
timeline = [
|
||||||
|
{"event": event, "key": (int(x), int(y), int(z), int(lod)),
|
||||||
|
"at_ms": float(at), "worker_id": int(worker),
|
||||||
|
"queue_depth": int(depth), "active_workers": int(active),
|
||||||
|
"priority_rank": int(rank)}
|
||||||
|
for event, x, y, z, lod, at, worker, depth, active, rank in TIMELINE.findall(content)
|
||||||
|
]
|
||||||
|
base_begin, base_end = cold.get(("base_compile", "begin")), cold.get(("base_compile", "end"))
|
||||||
|
near = NEAR.search(content)
|
||||||
|
profile_match, breakdown_match = PROFILE.search(content), BREAKDOWN.search(content)
|
||||||
|
if (result.returncode or stamp is None or stamp.group(1) != expected_stamp
|
||||||
|
or base_begin is None or base_end is None or near is None
|
||||||
|
or profile_match is None or breakdown_match is None):
|
||||||
|
raise RuntimeError(f"worker diagnostic incomplete: {log}; exit={result.returncode}")
|
||||||
|
begin_ms, begin_cpu = base_begin
|
||||||
|
end_ms, end_cpu = base_end
|
||||||
|
admitted = [event for event in timeline if event["event"] == "admitted" and event["key"][3] <= 2]
|
||||||
|
post_compile = [event for event in admitted if event["at_ms"] >= end_ms]
|
||||||
|
idle_by_worker: dict[int, list[tuple[float, float]]] = {}
|
||||||
|
idle_begin: dict[int, float] = {}
|
||||||
|
for event in timeline:
|
||||||
|
worker = event["worker_id"]
|
||||||
|
if worker < 0:
|
||||||
|
continue
|
||||||
|
if event["event"] == "idle_begin":
|
||||||
|
idle_begin[worker] = event["at_ms"]
|
||||||
|
elif event["event"] == "idle_end" and worker in idle_begin:
|
||||||
|
idle_by_worker.setdefault(worker, []).append((idle_begin.pop(worker), event["at_ms"]))
|
||||||
|
for worker, idle_start in idle_begin.items():
|
||||||
|
idle_by_worker.setdefault(worker, []).append((idle_start, end_ms))
|
||||||
|
idle_worker_ms = sum(intervals(spans, begin_ms, end_ms)
|
||||||
|
for spans in idle_by_worker.values())
|
||||||
|
profile = {key: float(value) for key, value in PAIR.findall(profile_match.group(1))}
|
||||||
|
breakdown = {key: float(value) for key, value in PAIR.findall(breakdown_match.group(1))}
|
||||||
|
parallel_stages = {}
|
||||||
|
for name in ("rock_lines", "candidate_axes", "candidate_merge", "detail_lines"):
|
||||||
|
wall, worker = profile.get(f"{name}_wall"), profile.get(f"{name}_worker")
|
||||||
|
if wall is not None and worker is not None and wall > 0.0:
|
||||||
|
parallel_stages[name] = {
|
||||||
|
"wall_ms": wall,
|
||||||
|
"worker_ms": worker,
|
||||||
|
"average_busy_workers": worker / wall,
|
||||||
|
}
|
||||||
|
row = {
|
||||||
|
"stratum": stratum,
|
||||||
|
"workers": workers,
|
||||||
|
"exit_code": result.returncode,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"source_stamp": stamp.group(1),
|
||||||
|
"base_compile_start_ms": begin_ms,
|
||||||
|
"base_shape_ready_ms": end_ms,
|
||||||
|
"base_compile_wall_ms": end_ms - begin_ms,
|
||||||
|
"process_cpu_ms_during_compile": end_cpu - begin_cpu,
|
||||||
|
"average_process_cores_during_compile": (end_cpu - begin_cpu) / max(1e-9, end_ms - begin_ms),
|
||||||
|
"stream_workers_idle_ms_during_compile": idle_worker_ms,
|
||||||
|
"stream_idle_worker_equivalents": idle_worker_ms / max(1e-9, end_ms - begin_ms),
|
||||||
|
"first_lod0_2_admission_ms": min((event["at_ms"] for event in admitted), default=None),
|
||||||
|
"first_lod0_2_shape_dependent_admission_ms": min(
|
||||||
|
(event["at_ms"] for event in post_compile), default=None),
|
||||||
|
"near_ready_s": float(near.group(3)),
|
||||||
|
"lod0_2_admissions": len(admitted),
|
||||||
|
"lod0_2_admissions_before_shape_ready": sum(event["at_ms"] < end_ms for event in admitted),
|
||||||
|
"shape_parallel_stages": parallel_stages,
|
||||||
|
"owner_shape_worker_ms": breakdown.get("owner_shape_worker"),
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L124WorkerRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
expected_stamp = subprocess.run(
|
||||||
|
["python3", "-B", str(STAMP_TOOL), "stamp", "--root", str(ROOT)],
|
||||||
|
cwd=ROOT, text=True, stdout=subprocess.PIPE, check=True,
|
||||||
|
).stdout.strip()
|
||||||
|
rows = [run(stratum, workers, expected_stamp)
|
||||||
|
for stratum in CAMERAS for workers in (8, 12)]
|
||||||
|
(OUT / "summary.json").write_text(
|
||||||
|
json.dumps({"source_stamp": expected_stamp, "runs": rows}, indent=2) + "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
# Linux127 — smooth-tunnel merge
|
||||||
|
|
||||||
|
Target: `aeb60bb` on `merge/smooth-tunnels`.
|
||||||
|
|
||||||
|
Build, fast-tier, walkability, seam, floor, determinism, and capture checks pass. The UE moving comparison shows a repeatable coverage drop and is the item to review before landing.
|
||||||
|
|
||||||
|
- Clang and GCC headless builds pass. UE builds pass: c743 baseline 157/157 actions; merge 29/29, using `-NoUBA -MaxParallelActions=4`.
|
||||||
|
- Fast tier passes (259 checks; 5 diagnostic checks). Walkability is 21,425/21,425. Four seam pairs (4,900 shared samples) are bit-identical. Floor probe maximum is 7.884 mm.
|
||||||
|
- All 18 owner4/owner2 Clang/GCC W1/W8/W12 hash runs match the L125 v225 baselines.
|
||||||
|
- L80 timing protocol on a quiet machine (load 0.69, CPU idle 99.4%): five interleaved measurements per cell, W8/W12 across all four strates. Merge vs L125 is within -4.05% to +2.83% near-ready and -4.58% to +0.95% whole-view. Vs c743, S0 is slower (+10.90% to +14.33% near, +1.26% to +3.52% whole); the other six cells are faster.
|
||||||
|
- UE startup reaches full LOD0–2 in all four strates: c743 2.232–2.726 s; merge 2.221–2.514 s. Far-apply CVar reads back as 1 and produces applies in every start and moving run.
|
||||||
|
- Five 30 m/s movement runs per build: median near coverage 43.36% → 40.22%; far holes 3.14% → 3.46%; tick p99 11.25 → 10.80 ms; worst tick 13.97 → 13.27 ms. Both builds reach 100% near coverage and zero far holes after stopping. The lane works; moving coverage remains lower on the merge.
|
||||||
|
- 23/23 L125 poses rendered and verified from the merge binary; 19 pass pixel quality, 4 show visible low-contrast geometry. See [`captures/INDEX.md`](captures/INDEX.md) and [`captures/COMPARE.html`](captures/COMPARE.html).
|
||||||
|
|
||||||
|
No source fix was made. Logs and summaries are in `Saved/Linux/l127`. An early probe wrapper wrote to the L125 probe output; the L125 binary regenerated it, with restoration checksums recorded in `Saved/Linux/l127/restore-l125-probe.log`.
|
||||||
@@ -0,0 +1,103 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Render the exact 23 L119 camera views with the L127 merge candidate."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
sys.dont_write_bytecode = True
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
RESEARCH = Path(__file__).resolve().parent
|
||||||
|
CAPTURES = RESEARCH / "captures"
|
||||||
|
SAVED = ROOT / "Saved/Linux/l127/captures"
|
||||||
|
BASELINE_INDEX = (ROOT / "Standalone/Research/WorldAssemblerM3/Linux119"
|
||||||
|
/ "captures/L119/capture-index.json")
|
||||||
|
LAYOUT = ROOT / "Saved/Linux/l127/hashes/owner4-surface-world-w12-clang.layout.json"
|
||||||
|
CLANG = ROOT / os.environ.get("L127_CAPTURE_CLANG",
|
||||||
|
"Saved/Linux/l127/bin-clang/VoxelForgeCoreHeadless-l127-clang")
|
||||||
|
RENDERER = ROOT / os.environ.get("L127_CAPTURE_RENDERER",
|
||||||
|
"Saved/Linux/l127/bin-clang/VoxelForgeOffscreenCapture-l127-clang")
|
||||||
|
HELPER_PATH = ROOT / "Standalone/Research/WorldAssemblerM3/Linux113/RunL113Captures.py"
|
||||||
|
|
||||||
|
SPEC = importlib.util.spec_from_file_location("l113_capture_helpers", HELPER_PATH)
|
||||||
|
if SPEC is None or SPEC.loader is None:
|
||||||
|
raise SystemExit("cannot load the verified L113 capture helpers")
|
||||||
|
HELPERS = importlib.util.module_from_spec(SPEC)
|
||||||
|
SPEC.loader.exec_module(HELPERS)
|
||||||
|
HELPERS.RESEARCH = RESEARCH
|
||||||
|
HELPERS.CAPTURES = CAPTURES
|
||||||
|
HELPERS.SAVED = SAVED
|
||||||
|
HELPERS.BINARIES = {"L127": CLANG}
|
||||||
|
HELPERS.RENDERERS = {"L127": RENDERER}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
if any(not path.is_file() for path in (BASELINE_INDEX, LAYOUT, CLANG, RENDERER)):
|
||||||
|
raise SystemExit("missing L119 camera set, L127 layout, or capture binary")
|
||||||
|
baseline = json.loads(BASELINE_INDEX.read_text(encoding="utf-8"))
|
||||||
|
layout = json.loads(LAYOUT.read_text(encoding="utf-8"))
|
||||||
|
rows = []
|
||||||
|
CAPTURES.mkdir(parents=True, exist_ok=True)
|
||||||
|
SAVED.mkdir(parents=True, exist_ok=True)
|
||||||
|
for view in baseline["captures"]:
|
||||||
|
row = HELPERS.capture("L127", view, layout)
|
||||||
|
row["variant"] = "L127-merge-v225"
|
||||||
|
if view["kind"] == "rock-floor-close":
|
||||||
|
scene = ROOT / row["scene"]
|
||||||
|
anchor = HELPERS.nearest_surface_anchor(scene, view["target"], margin_m=5.0,
|
||||||
|
preferred_normal=(0.0, 0.0, 1.0), minimum_alignment=0.85)
|
||||||
|
alignment = HELPERS.dot(anchor["normal"], (0.0, 0.0, 1.0))
|
||||||
|
camera_distance = math.dist(row["camera_actual_m"], anchor["position_m"])
|
||||||
|
row.update({"floor_anchor_position_m": anchor["position_m"],
|
||||||
|
"floor_anchor_normal": anchor["normal"],
|
||||||
|
"closeup_camera_to_floor_distance_m": camera_distance,
|
||||||
|
"closeup_floor_normal_alignment": alignment,
|
||||||
|
"verified_upward_floor_normal": alignment >= 0.85})
|
||||||
|
if (not row["verified"] or not 2.0 <= camera_distance <= 3.2
|
||||||
|
or anchor["distance_m"] > 0.25 or alignment < 0.85
|
||||||
|
or row["drawn_triangles"] <= 0
|
||||||
|
or row["pixel_quality_status"] not in
|
||||||
|
("pass", "low_contrast_geometry_visible")):
|
||||||
|
raise RuntimeError(f"L127 floor close-up failed at {view['name']}")
|
||||||
|
rows.append(row)
|
||||||
|
if len(rows) != len(baseline["captures"]) or len(rows) != 23:
|
||||||
|
raise RuntimeError(f"expected all 23 L119 views, captured {len(rows)}")
|
||||||
|
index = CAPTURES / "L127" / "capture-index.json"
|
||||||
|
index.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
index.write_text(json.dumps({"captures": rows}, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
|
||||||
|
html = ["<!doctype html><meta charset='utf-8'>",
|
||||||
|
"<title>L119 / L127 tunnel capture comparison</title>",
|
||||||
|
"<style>body{font:14px system-ui;margin:24px;background:#17191d;color:#eee}"
|
||||||
|
"section{margin:28px 0}h2{font-size:18px}div{display:grid;"
|
||||||
|
"grid-template-columns:1fr 1fr;gap:12px}img{width:100%;height:auto}"
|
||||||
|
"p{color:#bbb}</style>",
|
||||||
|
"<h1>L119 / L127 merge: identical views</h1>"]
|
||||||
|
old_by_name = {Path(row["capture"]).name: row for row in baseline["captures"]}
|
||||||
|
for row in rows:
|
||||||
|
name = Path(row["capture"]).name
|
||||||
|
old = old_by_name[name]
|
||||||
|
old_path = os.path.relpath(ROOT / "Standalone/Research/WorldAssemblerM3/Linux119"
|
||||||
|
/ old["capture"], CAPTURES)
|
||||||
|
new_path = os.path.relpath(RESEARCH / row["capture"], CAPTURES)
|
||||||
|
html.extend(["<section>", f"<h2>{row['passage']} — {row['station']}</h2>",
|
||||||
|
f"<p>{row['kind']} · L119 {old['pixel_quality_status']} · "
|
||||||
|
f"L127 {row['pixel_quality_status']}</p>", "<div>",
|
||||||
|
f"<figure><figcaption>L119</figcaption><img src='{old_path}'></figure>",
|
||||||
|
f"<figure><figcaption>L127 merge</figcaption><img src='{new_path}'></figure>",
|
||||||
|
"</div>", "</section>"])
|
||||||
|
(CAPTURES / "COMPARE.html").write_text("\n".join(html) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
print(f"[L127Captures] rendered={len(rows)} views index={index.relative_to(ROOT)}",
|
||||||
|
flush=True)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,188 @@
|
|||||||
|
<!doctype html><meta charset='utf-8'>
|
||||||
|
<title>L119 / L127 tunnel capture comparison</title>
|
||||||
|
<style>body{font:14px system-ui;margin:24px;background:#17191d;color:#eee}section{margin:28px 0}h2{font-size:18px}div{display:grid;grid-template-columns:1fr 1fr;gap:12px}img{width:100%;height:auto}p{color:#bbb}</style>
|
||||||
|
<h1>L119 / L127 merge: identical views</h1>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — rock-floor/401.7m</h2>
|
||||||
|
<p>rock-floor-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-rock-floor-station-1.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-rock-floor-station-1.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — rock-floor/792.8m</h2>
|
||||||
|
<p>rock-floor-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-rock-floor-station-2.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-rock-floor-station-2.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — rock-floor/1634.5m</h2>
|
||||||
|
<p>rock-floor-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-rock-floor-station-3.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-rock-floor-station-3.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/1-2/0 — rock-floor/27.0m</h2>
|
||||||
|
<p>rock-floor-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-1-2-0-rock-floor-station-1.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-1-2-0-rock-floor-station-1.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/1-2/0 — rock-floor/52.9m</h2>
|
||||||
|
<p>rock-floor-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-1-2-0-rock-floor-station-2.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-1-2-0-rock-floor-station-2.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/1-2/0 — rock-floor/78.9m</h2>
|
||||||
|
<p>rock-floor-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-1-2-0-rock-floor-station-3.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-1-2-0-rock-floor-station-3.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/2-3/0 — rock-floor/323.4m</h2>
|
||||||
|
<p>rock-floor-close · L119 low_contrast_geometry_visible · L127 low_contrast_geometry_visible</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-2-3-0-rock-floor-station-1.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-2-3-0-rock-floor-station-1.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/2-3/0 — rock-floor/333.4m</h2>
|
||||||
|
<p>rock-floor-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-2-3-0-rock-floor-station-2.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-2-3-0-rock-floor-station-2.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/2-3/0 — rock-floor/343.4m</h2>
|
||||||
|
<p>rock-floor-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-2-3-0-rock-floor-station-3.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-2-3-0-rock-floor-station-3.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — section-1/4m-outside</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-openair-1-4m-outside.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-openair-1-4m-outside.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — section-1/4m-interior-wall</h2>
|
||||||
|
<p>open-air-close · L119 low_contrast_geometry_visible · L127 low_contrast_geometry_visible</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-openair-1-4m-interior-wall.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-openair-1-4m-interior-wall.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — section-2/4m-outside</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-openair-2-4m-outside.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-openair-2-4m-outside.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — section-2/4m-interior-wall</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-openair-2-4m-interior-wall.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-openair-2-4m-interior-wall.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — section-3/4m-outside</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-openair-3-4m-outside.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-openair-3-4m-outside.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/0-1/0 — section-3/4m-interior-wall</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-0-1-0-openair-3-4m-interior-wall.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-0-1-0-openair-3-4m-interior-wall.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/1-2/0 — section-1/4m-outside</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-1-2-0-openair-1-4m-outside.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-1-2-0-openair-1-4m-outside.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/1-2/0 — section-1/4m-interior-wall</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-1-2-0-openair-1-4m-interior-wall.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-1-2-0-openair-1-4m-interior-wall.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/1-2/0 — section-2/4m-outside</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-1-2-0-openair-2-4m-outside.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-1-2-0-openair-2-4m-outside.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/1-2/0 — section-2/4m-interior-wall</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-1-2-0-openair-2-4m-interior-wall.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-1-2-0-openair-2-4m-interior-wall.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/2-3/0 — section-1/4m-outside</h2>
|
||||||
|
<p>open-air-close · L119 low_contrast_geometry_visible · L127 low_contrast_geometry_visible</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-2-3-0-openair-1-4m-outside.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-2-3-0-openair-1-4m-outside.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/2-3/0 — section-1/4m-interior-wall</h2>
|
||||||
|
<p>open-air-close · L119 low_contrast_geometry_visible · L127 low_contrast_geometry_visible</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-2-3-0-openair-1-4m-interior-wall.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-2-3-0-openair-1-4m-interior-wall.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/2-3/0 — section-2/4m-outside</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-2-3-0-openair-2-4m-outside.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-2-3-0-openair-2-4m-outside.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<section>
|
||||||
|
<h2>passage/2-3/0 — section-2/4m-interior-wall</h2>
|
||||||
|
<p>open-air-close · L119 pass · L127 pass</p>
|
||||||
|
<div>
|
||||||
|
<figure><figcaption>L119</figcaption><img src='../../Linux119/captures/L119/L115/L115/passage-2-3-0-openair-2-4m-interior-wall.png'></figure>
|
||||||
|
<figure><figcaption>L127 merge</figcaption><img src='L127/passage-2-3-0-openair-2-4m-interior-wall.png'></figure>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
# L127 merge captures
|
||||||
|
|
||||||
|
These are the 23 L125 camera poses, rendered from the merge binary.
|
||||||
|
|
||||||
|
[Open the L119/L127 side-by-side comparison](COMPARE.html).
|
||||||
|
|
||||||
|
- [passage/0-1/0 — rock-floor/401.7m (pass)](L127/passage-0-1-0-rock-floor-station-1.png)
|
||||||
|
- [passage/0-1/0 — rock-floor/792.8m (pass)](L127/passage-0-1-0-rock-floor-station-2.png)
|
||||||
|
- [passage/0-1/0 — rock-floor/1634.5m (pass)](L127/passage-0-1-0-rock-floor-station-3.png)
|
||||||
|
- [passage/1-2/0 — rock-floor/27.0m (pass)](L127/passage-1-2-0-rock-floor-station-1.png)
|
||||||
|
- [passage/1-2/0 — rock-floor/52.9m (pass)](L127/passage-1-2-0-rock-floor-station-2.png)
|
||||||
|
- [passage/1-2/0 — rock-floor/78.9m (pass)](L127/passage-1-2-0-rock-floor-station-3.png)
|
||||||
|
- [passage/2-3/0 — rock-floor/323.4m (low_contrast_geometry_visible)](L127/passage-2-3-0-rock-floor-station-1.png)
|
||||||
|
- [passage/2-3/0 — rock-floor/333.4m (pass)](L127/passage-2-3-0-rock-floor-station-2.png)
|
||||||
|
- [passage/2-3/0 — rock-floor/343.4m (pass)](L127/passage-2-3-0-rock-floor-station-3.png)
|
||||||
|
- [passage/0-1/0 — section-1/4m-outside (pass)](L127/passage-0-1-0-openair-1-4m-outside.png)
|
||||||
|
- [passage/0-1/0 — section-1/4m-interior-wall (low_contrast_geometry_visible)](L127/passage-0-1-0-openair-1-4m-interior-wall.png)
|
||||||
|
- [passage/0-1/0 — section-2/4m-outside (pass)](L127/passage-0-1-0-openair-2-4m-outside.png)
|
||||||
|
- [passage/0-1/0 — section-2/4m-interior-wall (pass)](L127/passage-0-1-0-openair-2-4m-interior-wall.png)
|
||||||
|
- [passage/0-1/0 — section-3/4m-outside (pass)](L127/passage-0-1-0-openair-3-4m-outside.png)
|
||||||
|
- [passage/0-1/0 — section-3/4m-interior-wall (pass)](L127/passage-0-1-0-openair-3-4m-interior-wall.png)
|
||||||
|
- [passage/1-2/0 — section-1/4m-outside (pass)](L127/passage-1-2-0-openair-1-4m-outside.png)
|
||||||
|
- [passage/1-2/0 — section-1/4m-interior-wall (pass)](L127/passage-1-2-0-openair-1-4m-interior-wall.png)
|
||||||
|
- [passage/1-2/0 — section-2/4m-outside (pass)](L127/passage-1-2-0-openair-2-4m-outside.png)
|
||||||
|
- [passage/1-2/0 — section-2/4m-interior-wall (pass)](L127/passage-1-2-0-openair-2-4m-interior-wall.png)
|
||||||
|
- [passage/2-3/0 — section-1/4m-outside (low_contrast_geometry_visible)](L127/passage-2-3-0-openair-1-4m-outside.png)
|
||||||
|
- [passage/2-3/0 — section-1/4m-interior-wall (low_contrast_geometry_visible)](L127/passage-2-3-0-openair-1-4m-interior-wall.png)
|
||||||
|
- [passage/2-3/0 — section-2/4m-outside (pass)](L127/passage-2-3-0-openair-2-4m-outside.png)
|
||||||
|
- [passage/2-3/0 — section-2/4m-interior-wall (pass)](L127/passage-2-3-0-openair-2-4m-interior-wall.png)
|
||||||
|
After Width: | Height: | Size: 11 KiB |
|
After Width: | Height: | Size: 98 KiB |
|
After Width: | Height: | Size: 63 KiB |
|
After Width: | Height: | Size: 111 KiB |
|
After Width: | Height: | Size: 225 KiB |
|
After Width: | Height: | Size: 177 KiB |
|
After Width: | Height: | Size: 125 KiB |
|
After Width: | Height: | Size: 112 KiB |
|
After Width: | Height: | Size: 116 KiB |
|
After Width: | Height: | Size: 50 KiB |
|
After Width: | Height: | Size: 184 KiB |
|
After Width: | Height: | Size: 34 KiB |
|
After Width: | Height: | Size: 121 KiB |
|
After Width: | Height: | Size: 111 KiB |
|
After Width: | Height: | Size: 163 KiB |
|
After Width: | Height: | Size: 132 KiB |
|
After Width: | Height: | Size: 37 KiB |
|
After Width: | Height: | Size: 55 KiB |
|
After Width: | Height: | Size: 16 KiB |
|
After Width: | Height: | Size: 67 KiB |
|
After Width: | Height: | Size: 21 KiB |
|
After Width: | Height: | Size: 146 KiB |
|
After Width: | Height: | Size: 106 KiB |
@@ -0,0 +1,9 @@
|
|||||||
|
# Linux130 — passage reach
|
||||||
|
|
||||||
|
On `wip/l130-reach-units` from `b96dd73`, passage influence is now **1.0533 m**: 0.75 m shell + 0.21 m relief + 0.50 m smooth-max support (`width/6`) + 0.01 m margin. One metres-to-core-voxels conversion feeds proof, feature, fallback and off-grid bounds; the sweep cutoff uses the same metre value. The dead 4 m cutoff is removed. Evaluator version is 227.
|
||||||
|
|
||||||
|
The debug shell/crest audit sampled 25,728 points, including 5 mm inside and outside the reach; 18,993 had sign-relevant passage contribution. Proof and feature bounds missed none. The nine-station floor probe still peaks at **7.884 mm**, matching L125. The 23 L125 views and six L129 mouth close-ups are byte-identical. Fast verification, walkability and shell audits pass; the seam audit found 0 mismatches across 4 pairs / 4,900 shared samples.
|
||||||
|
|
||||||
|
Clang/GCC W1/W8/W12 hashes agree across all 18 runs. Versus v225, hashes changed for owner4 full-detail, whole-view and LOD0–3; owner2 surface-world and floating-islands whole-view and LOD2–3. Exact-query hashes stayed unchanged. The v227 canonical baselines are recorded; changed hashes reflect the shorter density influence halo, while sampled sign-relevant bounds and captured surfaces remain covered.
|
||||||
|
|
||||||
|
L80 timing used 10 interleaved pairs per stratum/worker count. Across the eight case medians, the weighted deltas were +0.33% for LOD0–2 and +0.31% whole-view. Across 80 paired runs, mean deltas were +0.26% (95% interval −0.44% to +0.96%) and +0.47% (−0.06% to +1.00%); both intervals include zero. No measurable timing change; UE movement was skipped. The Linux UE host build succeeded in 76.30 s with `-NoUBA -MaxParallelActions=4`, without `prlimit`. `ARCHITECTURE.md` and `CODEMAP.md` had no note for this optimized influence reach, so they were unchanged.
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Run Clang/GCC owner hashes at W1/W8/W12 and compare with b96dd73."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
OUT = ROOT / "Saved/Linux/l130/hashes"
|
||||||
|
BASELINE = json.loads((ROOT / "Saved/Linux/l125/hashes/summary.json").read_text())
|
||||||
|
DATASETS = {
|
||||||
|
"owner4": ROOT / "Standalone/data/owner_world.json",
|
||||||
|
"owner2": ROOT / "Standalone/data/regression/owner_world_2strate_l61.json",
|
||||||
|
}
|
||||||
|
RECIPES = {"owner4": ("surface-world",),
|
||||||
|
"owner2": ("surface-world", "floating-islands")}
|
||||||
|
BINARIES = {
|
||||||
|
"clang": ROOT / "Saved/Linux/l130/bin/VoxelForgeCoreHeadless-l130-clang",
|
||||||
|
"gcc": ROOT / "Saved/Linux/l130/bin/VoxelForgeCoreHeadless-l130-gcc",
|
||||||
|
}
|
||||||
|
HASH_RE = re.compile(r"^\[Pipeline(?:FullDetailHash|WholeViewHash|LODHash|ExactQueryHash)\].*$", re.M)
|
||||||
|
STAMP_RE = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=(\S+)$", re.M)
|
||||||
|
|
||||||
|
|
||||||
|
def run(dataset: str, recipe: str, workers: int, compiler: str) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
label = f"{dataset}-{recipe}-w{workers}-{compiler}"
|
||||||
|
log = OUT / f"{label}.log"
|
||||||
|
layout = OUT / f"{label}.layout.json"
|
||||||
|
command = ["taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", str(BINARIES[compiler]), str(DATASETS[dataset]),
|
||||||
|
f"--workers={workers}", "--world-assembler-prototype", "--stream-benchmark",
|
||||||
|
"--stream-pipeline", "--force-generate", "--clip-radius=3",
|
||||||
|
"--max-clip-level=8", "--max-level=8", f"--recipe={recipe}",
|
||||||
|
"--stream-hash", f"--resolved-layout-output={layout}"]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
result = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
output = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
hashes = HASH_RE.findall(output)
|
||||||
|
if result.returncode or len(hashes) != 12:
|
||||||
|
raise SystemExit(f"hash run failed: {label}; exit={result.returncode} "
|
||||||
|
f"hashes={len(hashes)} log={log}")
|
||||||
|
stamp = STAMP_RE.search(output)
|
||||||
|
row = {"dataset": dataset, "recipe": recipe, "workers": workers,
|
||||||
|
"compiler": compiler, "exit_code": result.returncode, "hashes": hashes,
|
||||||
|
"source_stamp": stamp.group(1) if stamp else None,
|
||||||
|
"elapsed_s": round(time.monotonic() - started, 3),
|
||||||
|
"log": str(log.relative_to(ROOT))}
|
||||||
|
print("[L130HashRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
if any(not path.is_file() for path in (*DATASETS.values(), *BINARIES.values())):
|
||||||
|
raise SystemExit("missing L130 hash input")
|
||||||
|
old = {(row["dataset"], row["recipe"], row["workers"], row["compiler"]): row
|
||||||
|
for row in BASELINE["runs"]}
|
||||||
|
rows = []
|
||||||
|
for dataset, recipes in RECIPES.items():
|
||||||
|
for recipe in recipes:
|
||||||
|
for workers in (1, 8, 12):
|
||||||
|
for compiler in ("clang", "gcc"):
|
||||||
|
rows.append(run(dataset, recipe, workers, compiler))
|
||||||
|
(OUT / "runs-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
groups = {}
|
||||||
|
for dataset, recipes in RECIPES.items():
|
||||||
|
for recipe in recipes:
|
||||||
|
group = [row for row in rows if row["dataset"] == dataset
|
||||||
|
and row["recipe"] == recipe]
|
||||||
|
key = f"{dataset}/{recipe}"
|
||||||
|
stable = all(row["hashes"] == group[0]["hashes"] for row in group)
|
||||||
|
baseline = old[(dataset, recipe, 12, "clang")]["hashes"]
|
||||||
|
changed = [i for i, (before, after) in enumerate(zip(baseline,
|
||||||
|
group[0]["hashes"])) if before != after]
|
||||||
|
groups[key] = {"all_workers_and_compilers_match": stable,
|
||||||
|
"matches_b96dd73_v225": not changed,
|
||||||
|
"changed_hash_rows_from_v225": changed,
|
||||||
|
"source_stamps": sorted({row["source_stamp"] for row in group})}
|
||||||
|
if not stable:
|
||||||
|
raise SystemExit(f"worker/compiler hash mismatch: {key}")
|
||||||
|
(OUT / "summary.json").write_text(
|
||||||
|
json.dumps({"baseline_source": "b96dd73", "runs": rows,
|
||||||
|
"groups": groups}, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print("[L130HashSummary] " + json.dumps(groups, sort_keys=True), flush=True)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,136 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Render the six L129 b96dd73 mouth views against the L130 meshes."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import html
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
SOURCE = ROOT / "Saved/Linux/l129/captures/mouth-closeups/before"
|
||||||
|
SAVED = ROOT / "Saved/Linux/l130/captures/mouth-closeups"
|
||||||
|
CAPTURES = ROOT / "Standalone/Research/WorldAssemblerM3/Linux130/captures/mouth-closeups"
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
HEADLESS = ROOT / "Saved/Linux/l130/bin/VoxelForgeCoreHeadless-l130-clang"
|
||||||
|
RENDERER = ROOT / "Saved/Linux/l130/bin/VoxelForgeOffscreenCapture-l130-clang"
|
||||||
|
HEADLESS_CONFIG = re.compile(
|
||||||
|
r"^\[HeadlessConfig\].*?camera_m=\(([^)]*)\) camera_rotation_deg=\(([^)]*)\)", re.M)
|
||||||
|
SCENE = re.compile(r"^\[OffscreenScene\].*?height_center_m=(-?[\d.]+)", re.M)
|
||||||
|
DRAWN = re.compile(r"drawn_triangles=(\d+)")
|
||||||
|
|
||||||
|
|
||||||
|
def wrapped(command: list[str]) -> list[str]:
|
||||||
|
return ["taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", *map(str, command)]
|
||||||
|
|
||||||
|
|
||||||
|
def run(label: str, command: list[str], log: Path) -> str:
|
||||||
|
started = time.monotonic()
|
||||||
|
log.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with log.open("w", encoding="utf-8") as output:
|
||||||
|
result = subprocess.run(wrapped(command), cwd=ROOT, stdout=output,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
content = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
if result.returncode:
|
||||||
|
raise SystemExit(f"{label} failed exit={result.returncode}: {log}")
|
||||||
|
print(f"[L130MouthProcess] {label} elapsed_s={time.monotonic()-started:.2f}",
|
||||||
|
flush=True)
|
||||||
|
return content
|
||||||
|
|
||||||
|
|
||||||
|
def parse_vector(text: str) -> tuple[float, ...]:
|
||||||
|
return tuple(float(value) for value in text.split(","))
|
||||||
|
|
||||||
|
|
||||||
|
def sha256(path: Path) -> str:
|
||||||
|
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
if any(not path.is_file() for path in (DATA, HEADLESS, RENDERER)):
|
||||||
|
raise SystemExit("missing L130 mouth capture input")
|
||||||
|
(CAPTURES / "b96dd73").mkdir(parents=True, exist_ok=True)
|
||||||
|
(CAPTURES / "L130").mkdir(parents=True, exist_ok=True)
|
||||||
|
rows = []
|
||||||
|
for source_log in sorted(SOURCE.glob("*-mouth-mesh.log")):
|
||||||
|
name = source_log.name.removesuffix("-mesh.log")
|
||||||
|
source_render = SOURCE / f"{name}-render.log"
|
||||||
|
baseline_mesh = SOURCE / f"{name}.vfmesh"
|
||||||
|
if not all(path.is_file() for path in (source_render, baseline_mesh)):
|
||||||
|
raise SystemExit(f"missing b96dd73 source artifact for {name}")
|
||||||
|
config = HEADLESS_CONFIG.search(source_log.read_text())
|
||||||
|
scene = SCENE.search(source_render.read_text())
|
||||||
|
if config is None or scene is None:
|
||||||
|
raise SystemExit(f"missing camera metadata for {name}")
|
||||||
|
camera = parse_vector(config.group(1))
|
||||||
|
yaw, pitch = parse_vector(config.group(2))
|
||||||
|
height_center = float(scene.group(1))
|
||||||
|
camera_arg = ",".join(f"{value:.6f}" for value in (*camera, yaw, pitch))
|
||||||
|
candidate_mesh = SAVED / "L130" / f"{name}.vfmesh"
|
||||||
|
candidate_mesh.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
run(f"{name}-mesh", [HEADLESS, DATA, "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", "--workers=12",
|
||||||
|
"--clip-radius=3", "--max-clip-level=0", "--max-level=0",
|
||||||
|
"--force-generate", "--recipe=surface-world", f"--camera={camera_arg}",
|
||||||
|
"--render-distance-chunks=1", f"--mesh-export={candidate_mesh}"],
|
||||||
|
SAVED / "L130" / f"{name}-mesh.log")
|
||||||
|
if not candidate_mesh.is_file() or candidate_mesh.stat().st_size == 0:
|
||||||
|
raise SystemExit(f"empty L130 mesh for {name}")
|
||||||
|
|
||||||
|
render_args = ["--size=1920x1080",
|
||||||
|
f"--camera-m={','.join(f'{value:.6f}' for value in camera)}",
|
||||||
|
f"--yaw-deg={yaw:.6f}", f"--pitch-deg={pitch:.6f}",
|
||||||
|
"--shade=readable", "--headlamp=on", "--height-tint=on",
|
||||||
|
f"--height-center-m={height_center:.6f}", "--height-range-m=32"]
|
||||||
|
images = {}
|
||||||
|
draw_counts = {}
|
||||||
|
for variant, mesh in (("b96dd73", baseline_mesh), ("L130", candidate_mesh)):
|
||||||
|
image = CAPTURES / variant / f"{name}.png"
|
||||||
|
render_log = SAVED / variant / f"{name}-render.log"
|
||||||
|
content = run(f"{name}-{variant}-render",
|
||||||
|
[RENDERER, mesh, image, *render_args], render_log)
|
||||||
|
drawn = DRAWN.search(content)
|
||||||
|
if drawn is None or int(drawn.group(1)) <= 0 or not image.is_file():
|
||||||
|
raise SystemExit(f"render has no visible triangles for {name}/{variant}")
|
||||||
|
images[variant] = image
|
||||||
|
draw_counts[variant] = int(drawn.group(1))
|
||||||
|
equal = images["b96dd73"].read_bytes() == images["L130"].read_bytes()
|
||||||
|
row = {"name": name, "camera_m": camera, "yaw_deg": yaw,
|
||||||
|
"pitch_deg": pitch, "height_center_m": height_center,
|
||||||
|
"source_mesh": str(baseline_mesh.relative_to(ROOT)),
|
||||||
|
"candidate_mesh": str(candidate_mesh.relative_to(ROOT)),
|
||||||
|
"baseline_png": str(images["b96dd73"].relative_to(ROOT)),
|
||||||
|
"candidate_png": str(images["L130"].relative_to(ROOT)),
|
||||||
|
"baseline_sha256": sha256(images["b96dd73"]),
|
||||||
|
"candidate_sha256": sha256(images["L130"]),
|
||||||
|
"byte_identical": equal, "drawn_triangles": draw_counts}
|
||||||
|
rows.append(row)
|
||||||
|
print(f"[L130MouthCompare] {name} byte_identical={equal} "
|
||||||
|
f"sha256={row['candidate_sha256']}", flush=True)
|
||||||
|
|
||||||
|
if len(rows) != 6:
|
||||||
|
raise SystemExit(f"expected six L129 mouth views, found {len(rows)}")
|
||||||
|
(CAPTURES / "summary.json").write_text(json.dumps(rows, indent=2) + "\n")
|
||||||
|
sections = []
|
||||||
|
for row in rows:
|
||||||
|
name = html.escape(row["name"])
|
||||||
|
sections.append(f"<section><h2>{name}</h2><p>byte identical: "
|
||||||
|
f"{str(row['byte_identical']).lower()}</p><div>"
|
||||||
|
f"<figure><figcaption>b96dd73 source mesh</figcaption><img src='b96dd73/{name}.png'></figure>"
|
||||||
|
f"<figure><figcaption>L130 reach units</figcaption><img src='L130/{name}.png'></figure>"
|
||||||
|
"</div></section>")
|
||||||
|
page = ("<!doctype html><meta charset='utf-8'><title>L130 mouth closeups</title>"
|
||||||
|
"<style>body{font:14px system-ui;margin:24px;background:#17191d;color:#eee}"
|
||||||
|
"section{margin:28px 0}div{display:grid;grid-template-columns:1fr 1fr;gap:12px}"
|
||||||
|
"img{width:100%;height:auto}p{color:#bbb}</style>"
|
||||||
|
"<h1>L129 mouth closeups: b96dd73 vs L130</h1>" + "\n".join(sections))
|
||||||
|
(CAPTURES / "COMPARE.html").write_text(page)
|
||||||
|
return 0 if all(row["byte_identical"] for row in rows) else 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,98 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Render the 23 L125 camera poses with the L130 evaluator and compare them."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import html
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
import shutil
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
RESEARCH = Path(__file__).resolve().parent
|
||||||
|
CAPTURES = RESEARCH / "captures"
|
||||||
|
SAVED = ROOT / "Saved/Linux/l130/captures"
|
||||||
|
BASELINE_INDEX = (ROOT / "Standalone/Research/WorldAssemblerM3/Linux119"
|
||||||
|
/ "captures/L119/capture-index.json")
|
||||||
|
REFERENCE = ROOT / "Standalone/Research/WorldAssemblerM3/Linux125/captures/L125"
|
||||||
|
LAYOUT = ROOT / "Saved/Linux/l130/hashes/owner4-surface-world-w12-clang.layout.json"
|
||||||
|
HEADLESS = ROOT / "Saved/Linux/l130/bin/VoxelForgeCoreHeadless-l130-clang"
|
||||||
|
RENDERER = ROOT / "Saved/Linux/l130/bin/VoxelForgeOffscreenCapture-l130-clang"
|
||||||
|
HELPER_PATH = ROOT / "Standalone/Research/WorldAssemblerM3/Linux113/RunL113Captures.py"
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
if any(not path.is_file() for path in
|
||||||
|
(BASELINE_INDEX, REFERENCE / "passage-0-1-0-rock-floor-station-1.png",
|
||||||
|
LAYOUT, HEADLESS, RENDERER)):
|
||||||
|
raise SystemExit("missing pose, layout, or L130 capture input")
|
||||||
|
spec = importlib.util.spec_from_file_location("l125_capture_runner",
|
||||||
|
ROOT / "Standalone/Research/WorldAssemblerM3/Linux125/RunL125Captures.py")
|
||||||
|
if spec is None or spec.loader is None:
|
||||||
|
raise SystemExit("cannot load the verified L125 pose set")
|
||||||
|
runner = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(runner)
|
||||||
|
runner.RESEARCH = RESEARCH
|
||||||
|
runner.CAPTURES = CAPTURES
|
||||||
|
runner.SAVED = SAVED
|
||||||
|
runner.LAYOUT = LAYOUT
|
||||||
|
runner.CLANG = HEADLESS
|
||||||
|
runner.RENDERER = RENDERER
|
||||||
|
runner.HELPERS.RESEARCH = RESEARCH
|
||||||
|
runner.HELPERS.CAPTURES = CAPTURES
|
||||||
|
runner.HELPERS.SAVED = SAVED
|
||||||
|
runner.HELPERS.BINARIES = {"L125": HEADLESS}
|
||||||
|
runner.HELPERS.RENDERERS = {"L125": RENDERER}
|
||||||
|
if runner.main() != 0:
|
||||||
|
raise SystemExit("L130 pose generation failed")
|
||||||
|
|
||||||
|
new_captures = CAPTURES / "L125"
|
||||||
|
new_saved = SAVED / "L125"
|
||||||
|
index_path = new_captures / "capture-index.json"
|
||||||
|
data = json.loads(index_path.read_text())
|
||||||
|
if len(data["captures"]) != 23:
|
||||||
|
raise SystemExit(f"expected 23 poses, found {len(data['captures'])}")
|
||||||
|
identical = 0
|
||||||
|
sections = []
|
||||||
|
for row in data["captures"]:
|
||||||
|
name = row["name"]
|
||||||
|
old_image = REFERENCE / f"{name}.png"
|
||||||
|
new_image = new_captures / f"{name}.png"
|
||||||
|
if not old_image.is_file() or not new_image.is_file():
|
||||||
|
raise SystemExit(f"missing comparison image for {name}")
|
||||||
|
same = old_image.read_bytes() == new_image.read_bytes()
|
||||||
|
identical += int(same)
|
||||||
|
row["variant"] = "L130-v227"
|
||||||
|
row["capture"] = row["capture"].replace("captures/L125/", "captures/L130/")
|
||||||
|
row["scene"] = row["scene"].replace("captures/L125/", "captures/L130/")
|
||||||
|
row["logs"] = [path.replace("captures/L125/", "captures/L130/")
|
||||||
|
for path in row.get("logs", [])]
|
||||||
|
sections.append(f"<section><h2>{html.escape(row['passage'])} — "
|
||||||
|
f"{html.escape(row.get('station', ''))}</h2><p>{html.escape(row['kind'])} · "
|
||||||
|
f"byte identical: {str(same).lower()}</p><div>"
|
||||||
|
f"<figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/{html.escape(name)}.png'></figure>"
|
||||||
|
f"<figure><figcaption>L130 v227</figcaption><img src='L130/{html.escape(name)}.png'></figure>"
|
||||||
|
"</div></section>")
|
||||||
|
if identical != 23:
|
||||||
|
raise SystemExit(f"only {identical}/23 L125 pose captures are byte-identical")
|
||||||
|
|
||||||
|
index_path.write_text(json.dumps(data, indent=2) + "\n")
|
||||||
|
old_captures, old_saved = CAPTURES / "L130", SAVED / "L130"
|
||||||
|
if old_captures.exists():
|
||||||
|
shutil.rmtree(old_captures)
|
||||||
|
if old_saved.exists():
|
||||||
|
shutil.rmtree(old_saved)
|
||||||
|
new_captures.rename(old_captures)
|
||||||
|
new_saved.rename(old_saved)
|
||||||
|
page = ("<!doctype html><meta charset='utf-8'><title>L130 passage captures</title>"
|
||||||
|
"<style>body{font:14px system-ui;margin:24px;background:#17191d;color:#eee}"
|
||||||
|
"section{margin:28px 0}div{display:grid;grid-template-columns:1fr 1fr;gap:12px}"
|
||||||
|
"img{width:100%;height:auto}p{color:#bbb}</style>"
|
||||||
|
"<h1>L125 v225 vs L130 v227: passage captures</h1>" + "\n".join(sections))
|
||||||
|
(CAPTURES / "COMPARE.html").write_text(page)
|
||||||
|
print(f"[L130PoseCapture] poses=23 byte_identical={identical}/23", flush=True)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,137 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Five interleaved L80 near-ready runs per stratum/worker vs b96dd73."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import statistics
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
OUT = ROOT / "Saved/Linux/l130/timing"
|
||||||
|
PASS_ID = int(os.environ.get("VOXELFORGE_L130_TIMING_PASS", "1"))
|
||||||
|
REPLICATE_FIRST = (PASS_ID - 1) * 5 + 1
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
BINARIES = {
|
||||||
|
"b96dd73": ROOT / "Saved/Linux/l125/bin/VoxelForgeCoreHeadless-l125-clang",
|
||||||
|
"L130": ROOT / "Saved/Linux/l130/bin/VoxelForgeCoreHeadless-l130-clang",
|
||||||
|
}
|
||||||
|
CAMERAS = {
|
||||||
|
0: (8.0, 16.0, -372.2626953125),
|
||||||
|
1: (25.8369140625, 3.19921875, -737.0),
|
||||||
|
2: (19.166015625, -14.2001953125, -753.3740234375),
|
||||||
|
3: (-20.0625, -5.068359375, -900.353515625),
|
||||||
|
}
|
||||||
|
PIPELINE_RE = re.compile(r"^\[StreamPipeline\] (.*)$", re.M)
|
||||||
|
FIELDS_RE = re.compile(r"([a-zA-Z0-9_]+)=([^ ]+)")
|
||||||
|
NEAR_RE = re.compile(
|
||||||
|
r"^\[WorldAssemblerNearLODReady\] lods=0-2 tiles=(\d+)/(\d+) cold_ready_s=([0-9.]+)$",
|
||||||
|
re.M)
|
||||||
|
|
||||||
|
|
||||||
|
def run(strate: int, workers: int, version: str, phase: str, replicate: int) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
pass_suffix = "" if PASS_ID == 1 else f"-p{PASS_ID}"
|
||||||
|
label = f"s{strate}-w{workers}-{version}-{phase}{replicate}-l130-v227{pass_suffix}"
|
||||||
|
log = OUT / f"{label}.log"
|
||||||
|
camera = ",".join(f"{value:.12f}" for value in CAMERAS[strate]) + ",0,0"
|
||||||
|
command = ["taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||||
|
"timeout", "1740s", str(BINARIES[version]), str(DATA),
|
||||||
|
"--world-assembler-prototype", "--stream-benchmark", "--stream-pipeline",
|
||||||
|
f"--workers={workers}", "--clip-radius=3", "--max-clip-level=8",
|
||||||
|
"--max-level=8", f"--camera={camera}", "--profile-lod-work",
|
||||||
|
"--stream-scheduler-policy=3"]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as output:
|
||||||
|
result = subprocess.run(command, cwd=ROOT, stdout=output,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
content = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
pipeline, near = PIPELINE_RE.search(content), NEAR_RE.search(content)
|
||||||
|
if result.returncode or pipeline is None or near is None:
|
||||||
|
raise SystemExit(f"timing run failed: {log}; exit={result.returncode}")
|
||||||
|
fields = dict(FIELDS_RE.findall(pipeline.group(1)))
|
||||||
|
row = {"strate": strate, "workers": workers, "version": version,
|
||||||
|
"phase": phase, "replicate": replicate, "exit_code": result.returncode,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"lod02_ready_s": float(near.group(3)),
|
||||||
|
"whole_view_ready_s": float(fields["whole_view_ready"].removesuffix("s")),
|
||||||
|
"log": str(log.relative_to(ROOT))}
|
||||||
|
print("[L130TimingRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
if any(not path.is_file() for path in (DATA, *BINARIES.values())):
|
||||||
|
raise SystemExit("missing L130 timing input")
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
load_1m = __import__("os").getloadavg()[0]
|
||||||
|
def cpu_idle(interval: float = 2.0) -> float:
|
||||||
|
def totals() -> tuple[int, int]:
|
||||||
|
fields = Path("/proc/stat").read_text().splitlines()[0].split()
|
||||||
|
values = [int(value) for value in fields[1:]]
|
||||||
|
return sum(values), values[3] + (values[4] if len(values) > 4 else 0)
|
||||||
|
before = totals()
|
||||||
|
time.sleep(interval)
|
||||||
|
after = totals()
|
||||||
|
return 100.0 * (after[1] - before[1]) / max(1, after[0] - before[0])
|
||||||
|
idle = cpu_idle()
|
||||||
|
pass_suffix = "" if PASS_ID == 1 else f"-p{PASS_ID}"
|
||||||
|
(OUT / f"machine-load-v227{pass_suffix}-before.txt").write_text(
|
||||||
|
f"1-minute load: {load_1m:.2f}\n2-second CPU idle: {idle:.1f}%\n")
|
||||||
|
print(f"[L130TimingGate] load_1m={load_1m:.2f} cpu_idle_2s={idle:.1f}%",
|
||||||
|
flush=True)
|
||||||
|
if load_1m >= 0.9 or idle < 97.0:
|
||||||
|
raise SystemExit("timing refused: machine is above the quiet-load gate")
|
||||||
|
|
||||||
|
rows = []
|
||||||
|
for strate in range(4):
|
||||||
|
for workers in (8, 12):
|
||||||
|
for version in BINARIES:
|
||||||
|
run(strate, workers, version, "warmup", 0)
|
||||||
|
for replicate in range(REPLICATE_FIRST, REPLICATE_FIRST + 5):
|
||||||
|
order = tuple(BINARIES) if replicate % 2 else tuple(reversed(BINARIES))
|
||||||
|
for version in order:
|
||||||
|
rows.append(run(strate, workers, version, "measured", replicate))
|
||||||
|
(OUT / f"runs-v227{pass_suffix}-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
prior_runs = []
|
||||||
|
if PASS_ID > 1:
|
||||||
|
previous = OUT / ("v227-summary.json" if PASS_ID == 2
|
||||||
|
else "v227-combined-summary.json")
|
||||||
|
if previous.is_file():
|
||||||
|
prior_runs = json.loads(previous.read_text(encoding="utf-8")).get("runs", [])
|
||||||
|
all_rows = prior_runs + rows
|
||||||
|
measured_rows = [row for row in all_rows if row["phase"] == "measured"]
|
||||||
|
summary = []
|
||||||
|
for strate in range(4):
|
||||||
|
for workers in (8, 12):
|
||||||
|
group_size = sum(row["strate"] == strate and row["workers"] == workers
|
||||||
|
for row in measured_rows) // 2
|
||||||
|
item = {"strate": strate, "workers": workers,
|
||||||
|
"replicates_per_version": group_size}
|
||||||
|
medians = {}
|
||||||
|
for version in BINARIES:
|
||||||
|
group = [row for row in measured_rows if row["strate"] == strate
|
||||||
|
and row["workers"] == workers and row["version"] == version]
|
||||||
|
medians[version] = {}
|
||||||
|
for key in ("lod02_ready_s", "whole_view_ready_s"):
|
||||||
|
values = [row[key] for row in group]
|
||||||
|
medians[version][key] = statistics.median(values)
|
||||||
|
item[f"{version}_{key}_values"] = values
|
||||||
|
item[f"{version}_{key}_median"] = medians[version][key]
|
||||||
|
for key in ("lod02_ready_s", "whole_view_ready_s"):
|
||||||
|
item[f"L130_vs_b96dd73_{key}_delta_pct"] = round(
|
||||||
|
100.0*(medians["L130"][key]/medians["b96dd73"][key]-1.0), 2)
|
||||||
|
summary.append(item)
|
||||||
|
summary_path = OUT / ("v227-summary.json" if PASS_ID == 1
|
||||||
|
else f"v227-combined-summary.json")
|
||||||
|
summary_path.write_text(json.dumps(
|
||||||
|
{"runs": all_rows, "summary": summary}, indent=2) + "\n", encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
<!doctype html><meta charset='utf-8'><title>L130 passage captures</title><style>body{font:14px system-ui;margin:24px;background:#17191d;color:#eee}section{margin:28px 0}div{display:grid;grid-template-columns:1fr 1fr;gap:12px}img{width:100%;height:auto}p{color:#bbb}</style><h1>L125 v225 vs L130 v227: passage captures</h1><section><h2>passage/0-1/0 — rock-floor/401.7m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-rock-floor-station-1.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-rock-floor-station-1.png'></figure></div></section>
|
||||||
|
<section><h2>passage/0-1/0 — rock-floor/792.8m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-rock-floor-station-2.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-rock-floor-station-2.png'></figure></div></section>
|
||||||
|
<section><h2>passage/0-1/0 — rock-floor/1634.5m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-rock-floor-station-3.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-rock-floor-station-3.png'></figure></div></section>
|
||||||
|
<section><h2>passage/1-2/0 — rock-floor/27.0m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-1-2-0-rock-floor-station-1.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-1-2-0-rock-floor-station-1.png'></figure></div></section>
|
||||||
|
<section><h2>passage/1-2/0 — rock-floor/52.9m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-1-2-0-rock-floor-station-2.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-1-2-0-rock-floor-station-2.png'></figure></div></section>
|
||||||
|
<section><h2>passage/1-2/0 — rock-floor/78.9m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-1-2-0-rock-floor-station-3.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-1-2-0-rock-floor-station-3.png'></figure></div></section>
|
||||||
|
<section><h2>passage/2-3/0 — rock-floor/323.4m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-2-3-0-rock-floor-station-1.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-2-3-0-rock-floor-station-1.png'></figure></div></section>
|
||||||
|
<section><h2>passage/2-3/0 — rock-floor/333.4m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-2-3-0-rock-floor-station-2.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-2-3-0-rock-floor-station-2.png'></figure></div></section>
|
||||||
|
<section><h2>passage/2-3/0 — rock-floor/343.4m</h2><p>rock-floor-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-2-3-0-rock-floor-station-3.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-2-3-0-rock-floor-station-3.png'></figure></div></section>
|
||||||
|
<section><h2>passage/0-1/0 — section-1/4m-outside</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-openair-1-4m-outside.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-openair-1-4m-outside.png'></figure></div></section>
|
||||||
|
<section><h2>passage/0-1/0 — section-1/4m-interior-wall</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-openair-1-4m-interior-wall.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-openair-1-4m-interior-wall.png'></figure></div></section>
|
||||||
|
<section><h2>passage/0-1/0 — section-2/4m-outside</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-openair-2-4m-outside.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-openair-2-4m-outside.png'></figure></div></section>
|
||||||
|
<section><h2>passage/0-1/0 — section-2/4m-interior-wall</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-openair-2-4m-interior-wall.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-openair-2-4m-interior-wall.png'></figure></div></section>
|
||||||
|
<section><h2>passage/0-1/0 — section-3/4m-outside</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-openair-3-4m-outside.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-openair-3-4m-outside.png'></figure></div></section>
|
||||||
|
<section><h2>passage/0-1/0 — section-3/4m-interior-wall</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-0-1-0-openair-3-4m-interior-wall.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-0-1-0-openair-3-4m-interior-wall.png'></figure></div></section>
|
||||||
|
<section><h2>passage/1-2/0 — section-1/4m-outside</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-1-2-0-openair-1-4m-outside.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-1-2-0-openair-1-4m-outside.png'></figure></div></section>
|
||||||
|
<section><h2>passage/1-2/0 — section-1/4m-interior-wall</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-1-2-0-openair-1-4m-interior-wall.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-1-2-0-openair-1-4m-interior-wall.png'></figure></div></section>
|
||||||
|
<section><h2>passage/1-2/0 — section-2/4m-outside</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-1-2-0-openair-2-4m-outside.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-1-2-0-openair-2-4m-outside.png'></figure></div></section>
|
||||||
|
<section><h2>passage/1-2/0 — section-2/4m-interior-wall</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-1-2-0-openair-2-4m-interior-wall.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-1-2-0-openair-2-4m-interior-wall.png'></figure></div></section>
|
||||||
|
<section><h2>passage/2-3/0 — section-1/4m-outside</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-2-3-0-openair-1-4m-outside.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-2-3-0-openair-1-4m-outside.png'></figure></div></section>
|
||||||
|
<section><h2>passage/2-3/0 — section-1/4m-interior-wall</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-2-3-0-openair-1-4m-interior-wall.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-2-3-0-openair-1-4m-interior-wall.png'></figure></div></section>
|
||||||
|
<section><h2>passage/2-3/0 — section-2/4m-outside</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-2-3-0-openair-2-4m-outside.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-2-3-0-openair-2-4m-outside.png'></figure></div></section>
|
||||||
|
<section><h2>passage/2-3/0 — section-2/4m-interior-wall</h2><p>open-air-close · byte identical: true</p><div><figure><figcaption>L125 v225</figcaption><img src='../../Linux125/captures/L125/passage-2-3-0-openair-2-4m-interior-wall.png'></figure><figure><figcaption>L130 v227</figcaption><img src='L130/passage-2-3-0-openair-2-4m-interior-wall.png'></figure></div></section>
|
||||||
|
After Width: | Height: | Size: 11 KiB |
|
After Width: | Height: | Size: 98 KiB |
|
After Width: | Height: | Size: 63 KiB |
|
After Width: | Height: | Size: 111 KiB |
|
After Width: | Height: | Size: 225 KiB |
|
After Width: | Height: | Size: 177 KiB |
|
After Width: | Height: | Size: 125 KiB |
|
After Width: | Height: | Size: 112 KiB |
|
After Width: | Height: | Size: 116 KiB |
|
After Width: | Height: | Size: 50 KiB |
|
After Width: | Height: | Size: 184 KiB |
|
After Width: | Height: | Size: 34 KiB |
|
After Width: | Height: | Size: 121 KiB |
|
After Width: | Height: | Size: 111 KiB |
|
After Width: | Height: | Size: 163 KiB |
|
After Width: | Height: | Size: 132 KiB |
|
After Width: | Height: | Size: 37 KiB |
|
After Width: | Height: | Size: 55 KiB |
|
After Width: | Height: | Size: 16 KiB |
|
After Width: | Height: | Size: 67 KiB |
|
After Width: | Height: | Size: 21 KiB |
|
After Width: | Height: | Size: 146 KiB |
|
After Width: | Height: | Size: 106 KiB |
@@ -0,0 +1,6 @@
|
|||||||
|
<!doctype html><meta charset='utf-8'><title>L130 mouth closeups</title><style>body{font:14px system-ui;margin:24px;background:#17191d;color:#eee}section{margin:28px 0}div{display:grid;grid-template-columns:1fr 1fr;gap:12px}img{width:100%;height:auto}p{color:#bbb}</style><h1>L129 mouth closeups: b96dd73 vs L130</h1><section><h2>passage-0-1-0-lower-mouth</h2><p>byte identical: true</p><div><figure><figcaption>b96dd73 source mesh</figcaption><img src='b96dd73/passage-0-1-0-lower-mouth.png'></figure><figure><figcaption>L130 reach units</figcaption><img src='L130/passage-0-1-0-lower-mouth.png'></figure></div></section>
|
||||||
|
<section><h2>passage-0-1-0-upper-mouth</h2><p>byte identical: true</p><div><figure><figcaption>b96dd73 source mesh</figcaption><img src='b96dd73/passage-0-1-0-upper-mouth.png'></figure><figure><figcaption>L130 reach units</figcaption><img src='L130/passage-0-1-0-upper-mouth.png'></figure></div></section>
|
||||||
|
<section><h2>passage-1-2-0-lower-mouth</h2><p>byte identical: true</p><div><figure><figcaption>b96dd73 source mesh</figcaption><img src='b96dd73/passage-1-2-0-lower-mouth.png'></figure><figure><figcaption>L130 reach units</figcaption><img src='L130/passage-1-2-0-lower-mouth.png'></figure></div></section>
|
||||||
|
<section><h2>passage-1-2-0-upper-mouth</h2><p>byte identical: true</p><div><figure><figcaption>b96dd73 source mesh</figcaption><img src='b96dd73/passage-1-2-0-upper-mouth.png'></figure><figure><figcaption>L130 reach units</figcaption><img src='L130/passage-1-2-0-upper-mouth.png'></figure></div></section>
|
||||||
|
<section><h2>passage-2-3-0-lower-mouth</h2><p>byte identical: true</p><div><figure><figcaption>b96dd73 source mesh</figcaption><img src='b96dd73/passage-2-3-0-lower-mouth.png'></figure><figure><figcaption>L130 reach units</figcaption><img src='L130/passage-2-3-0-lower-mouth.png'></figure></div></section>
|
||||||
|
<section><h2>passage-2-3-0-upper-mouth</h2><p>byte identical: true</p><div><figure><figcaption>b96dd73 source mesh</figcaption><img src='b96dd73/passage-2-3-0-upper-mouth.png'></figure><figure><figcaption>L130 reach units</figcaption><img src='L130/passage-2-3-0-upper-mouth.png'></figure></div></section>
|
||||||
|
After Width: | Height: | Size: 37 KiB |
|
After Width: | Height: | Size: 225 KiB |
|
After Width: | Height: | Size: 39 KiB |
|
After Width: | Height: | Size: 187 KiB |