measure(step1): cook is 1.5-2.3 ms, RMC's route is 8-11 s; CMC cannot take a field floor; cap 16 uploads

Rebuild plan step 1 (RESEARCH-SPEED-2026-09.md), all behind default-off switches.
A. Direct cook on a dedicated thread, 27-tile support cohort: core cook p50 1.50 ms, p95 2.28 ms;
   a 3,949-tri tile 1.67 ms extract+cook. PASSES the 10 ms gate. Through RMC's route the same
   support bodies became ready 8.46 s p50 / 10.80 s p95 after their meshes existed: FAILS 150 ms.
   Exact-target capsule sweep confirmed (0.75 ms after body update).
B. 100 rapid underfoot digs on CMC with a field override: 883 false-support frames, 0 false-air,
   0 invented velocity. CMC's floor comes from collision-scene traces -> step 4 = field-aware Mover.
C. 341 prequeued tiles, render on: cap 4 = 238 applies/s, cap 8 = 476, cap 16 = 639 (drain p95
   2.22 ms vs 2 ms gate); first visible 1 frame after release; no RT/RHI backlog. Tune drain,
   no grouping needed yet.
D. Same 128^3 region, 64/64 tile hashes equal in export and game. The "263 ns/sample" was export
   WALL time with 15 workers sharing a lattice (~3.4 us CPU/sample); game tile time includes
   meshing and duplicated borders (+22%); contention cap 1 vs 14: 74 vs 112 ms per tile.
   Profiling is not the cause.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MmSoF64BbRFmk9C3p1VsBJ
This commit is contained in:
Fr0zkaandClaude Opus 5.5 committed 2026-09-30 00:07:59 +02:00
1 parent 97a1874498
commit 41b001b652
5 files changed
+481 -55

No files matched your search

+12 -9
View File
@@ -117,9 +117,9 @@ remains `0`, and lateral regions remain gated off.
|------|------|
| `../../VoxelForge.uplugin` | Plugin manifest. Runtime module `VoxelForge` plus the Editor-only `VoxelForgeEditor` commandlet module. Beta. |
| `VoxelForge.Build.cs` | Deps: Core, CoreUObject, Engine, **GameplayTags**, **RealtimeMeshComponent**. |
| `Tools/VoxelForgeTest.ps1` | The single guarded test entry point. `-Scenario canonical|owner|probe|perf|tests|surface-fall|crossing|gate-stress|visual-edit|visual-lod|visual-gap-walk|visual-gap-inject|dig-motion|parity|horizon|horizon-render|determinism`, optional `-Build`, `-Assets owner|default`, `-Cvars`, absolute `-Out`, `-Label`, `-HorizonLevel`, `-HorizonRenderDistanceChunks`, and `-SharedDDC`; stages the host/assets, isolates user/DDC/config state, serializes one Unreal launch at a time, enforces the 8 GiB/30-minute guards, and emits compact `result.json`/`summary.txt` plus full logs. `visual-gap-walk` records camera-frustum surface-point coverage at 6000 cm/s; `visual-gap-inject` proves an early component hide is detected; `dig-motion` logs per-frame vertical velocity, gate, support readiness, and movement mode. `-Cvars @{ 'voxel.TestGapAuditLegacyHandoff'=1 }` replays the old visibility handoff for A/B audit baselines. Horizon traces report desired/loaded tiles, resident stream geometry, worker seconds, steady wall time, and peak private memory; `horizon-render` captures the owner view; `determinism` compares per-tile hashes across the default CRT path and `-voxel.CrtFma3=0`. |
| `Tools/VoxelForgeTest.ps1` | The single guarded test entry point. `-Scenario canonical|owner|probe|perf|tests|surface-fall|crossing|gate-stress|visual-edit|visual-lod|visual-gap-walk|visual-gap-inject|dig-motion|parity|horizon|horizon-render|determinism`, optional `-Build`, `-Assets owner|default`, `-Cvars`, absolute `-Out`, `-Label`, `-HorizonLevel`, `-HorizonRenderDistanceChunks`, `-ParityExportSize=32|64|128`, `-ParityExportTile=x,y,z`, and `-SharedDDC`; stages the host/assets, isolates user/DDC/config state, serializes one Unreal launch at a time, enforces the 8 GiB/30-minute guards, and emits compact `result.json`/`summary.txt` plus full logs. Parity can render its game leg with `-EnableRender` and compares the selected export region against the exact streamed tile hashes. `visual-gap-walk` records camera-frustum surface-point coverage at 6000 cm/s; `visual-gap-inject` proves an early component hide is detected; `dig-motion` logs per-frame vertical velocity, gate, support readiness, and movement mode. `-Cvars @{ 'voxel.TestGapAuditLegacyHandoff'=1 }` replays the old visibility handoff for A/B audit baselines. Horizon traces report desired/loaded tiles, resident stream geometry, worker seconds, steady wall time, and peak private memory; `horizon-render` captures the owner view; `determinism` compares per-tile hashes across the default CRT path and `-voxel.CrtFma3=0`. |
| `Public/VoxelForgeModule.h` / `Private/VoxelForgeModule.cpp` | `FVoxelForgeModule` boilerplate (Startup/Shutdown just log). |
| `Source/VoxelForgeEditor/VoxelForgeEditor.Build.cs` / `Private/VoxelForgeExploreCommandlet.*` | Editor/commandlet-only `-run=VoxelForgeExplore`; owns bounded render/`VF_` walk/OBJ export orchestration and is not linked by Game/Shipping. |
| `Source/VoxelForgeEditor/VoxelForgeEditor.Build.cs` / `Private/VoxelForgeExploreCommandlet.*` | Editor/commandlet-only `-run=VoxelForgeExplore`; owns bounded render/`VF_` walk/OBJ export orchestration and is not linked by Game/Shipping. Optional `-exporttilex/y/z` sets an exact L0 tile origin for controlled export/game parity; omitting all three preserves the existing center-derived origin. |
| `Public/VoxelGeometryHash.h` | Header-only geometry digest shared by commandlet export and runtime tile-hash dump; hashes mesh counts and the vertex/normal/UV/color/triangle arrays with the historical CRC order. |
| `Public/VoxelStats.h` / `Private/VoxelStats.cpp` | `stat VoxelForge` DWORD counters for tile classification, skipping, meshing, operator-stack verdicts, and cave-bail diagnosis. Each `Cave Bail *` counter names one guard; `Cave Bail No Stack` counts slots with no buildable stack (no definition, or a degenerate strate). |
@@ -305,9 +305,10 @@ casse cette arithmetique *structurellement* — `BeginPlay` loggue une **Error**
| `RebuildStrates` | 724 | Reinitializes the volume only when enabled; when disabled, shuts down and clears any existing volume and reapplies neutral terrain material params. |
| `Tick` | 5715 | `UpdateChunksAroundPosition(player, heading)` + `ProcessPendingChunks()`; the clean LOD0 latency summary is flushed at `EndPlay`. Density-volume update/material work is inside the non-null volume branch, so it is absent when disabled. |
| `GetPlayerPosition` | 4831 | Pawn position or zero. |
| `AdvanceHeadlessDigMotionTest` / `RecordHeadlessDigMotionFrame` | 3060 / 3217 | Explicit `-voxel.TestDigMotion=1` repro: digs eight times beside a walking pawn, records each frame's vertical velocity, gate, support tile/readiness, movement mode, then digs underfoot and requires ordinary falling. The accompanying gap audit is scoped to the dig session. |
| `AdvanceHeadlessDigMotionTest` / `RecordHeadlessDigMotionFrame` | 3060 / 3217 | Explicit `-voxel.TestDigMotion=1` repro: optional near digs, then repeats underfoot digs and samples `Generator->GetDensityAt` immediately after each edit, on its first subsequent simulated frame, and on every following observation frame. `-voxel.TestDigMotionFieldOverride=1` is a measurement-only prototype that clears `CurrentFloor` and switches CMC to falling when the sampled post-edit density is air; it is inactive by default. The test counts false support, false air and downward velocity beyond one gravity step. `-voxel.TestDigMotionUnderfootDigCount` is clamped to 1–100; `-voxel.TestDigMotionUnderfootIntervalSeconds=0` performs per-frame rapid edits. The accompanying gap audit is scoped to the dig session. |
| `MaybeFinishHeadlessStreamingTest` | — | The existing `voxel.TestExitSeconds` diagnostic emits `VoxelForgeStreamingExitDiag` at most every 30 elapsed seconds, including threshold and early-return test flags; no output when the timer is unconfigured. |
| `UpdateVisualStalenessAudit` | 5034 | Test-only camera-frustum coverage audit: tracks up to 24 real pre-skirt isosurface points per visible component, then counts frames where no visible tile component covers a previously visible, still-desired point (excluding edited bounds). `voxel.TestVisualGapInjectEarlyHide=1` hides a valid component for three frames and must be caught. `voxel.TestGapAuditLegacyHandoff=1` restores old handoff culling for A/B baselines. |
| `ProcessPendingChunks` | 7435 | Drains critical, edited, near, then ordinary result queues under the per-frame budget; reads the per-request cancellation token before removing its map entry, drops obsolete/aborted results, then applies each live result via `ApplyTileResult`. Collision profiling timestamps the result's arrival here to separate worker completion from game-thread apply wait. Step 1's `voxel.Step1PrequeueVisibleLOD0` holds completed results until the requested non-empty L0 count is ready, then releases them through this same drain. |
| `ProcessPendingChunks` | 7435 | Drains critical, edited, near, then ordinary result queues under the per-frame budget; reads the per-request cancellation token before removing its map entry, drops obsolete/aborted results, then applies each live result via `ApplyTileResult`. Collision profiling timestamps the result's arrival here to separate worker completion from game-thread apply wait. Step 1's `voxel.Step1PrequeueVisibleLOD0` holds completed results until the requested non-empty L0 count is ready, then releases them through this same drain and emits the `VFStep1UploadCorpusReleased` CSV event for UE's delayed CSV capture. |
| `ApplyTileResult` | — | **Shared game-thread apply** for one `FChunkResult` (async drain + sync carve): discards stale generation epochs and keys no longer desired, marks loaded, ingests capture, EMPTY releases the tile's existing component (a re-gen can flip content→empty on band change — old geometry must not linger), else `ApplyMeshToTile`. Returns true iff a visible mesh uploaded (counts the budget). Doesn't touch `PendingTiles` (caller's). |
| `FVoxelStackSampler` / `FScopedVoxelStackRegistration` (`VoxelStackSampler.h/.cpp`) | — | **Diagnostic raw stack sampler, off by default.** A sampler thread captures registered generation workers' stacks (`CaptureThreadStackBackTrace`) every N µs and tags each with the tile's LOD. It writes bounded raw PCs to CSV plus a symbolization-free summary under `Saved/`; symbol resolution is deferred until after process exit so the measured process never initializes/calls DbgHelp for this diagnostic. Lifecycle teardown is serialized and registration TLS is session-tagged. Registration is per tile TASK in `LoadTile` (and the export's tasks), never per sample; disabled costs one relaxed atomic load. Game `-voxel.SampleStacks=<us>`, commandlet `-samplestacks[=us]`. |
| Room-cache performance (2026-09-13): `VF_FindPlayerFitPointForRoomMemoized` / `GPlayerFitMemo` (`VoxelCaveMorphology.cpp`), `voxel.TileCacheWindow` + `VoxelGenLOD::TileOriginVoxels/TileCellsPerAxis` (`VoxelGenerator.h`), `FChunkSDFSpatialIndex` | — | **Player-fit memo:** exact 97-word key, process-wide, `voxel.PlayerFitMemo`. **Tile cache window:** the room cache covers the requesting tile's footprint plus a fixed collect margin. It is set via `TGuardValue` in `GenerateMesh`/classify, and is window-invariant by construction (the margin does not grow with the tile). **LOD-gated:** fused path at LOD ≥ `voxel.TileCacheWindowMinLOD` (3), op-stack at LOD ≥ `voxel.TileCacheWindowOpStackMinLOD` (4); lower LODs keep the 4-chunk worker window, because the tile window slowed LOD0 evaluation ~6%. Strates whose authored terrain ops exceed the proven 256-voxel extent (or are malformed/unloaded) fall back to the legacy window (`VF_ComputeTerrainOpEnvelope`). `voxel.SpatialIndex` is -1 auto (off at fused LOD0, on elsewhere), 0 or 1. **Spatial index:** ordered XY buckets over cache primitives, exact rejection. The `static_assert` in `VoxelStrateTypes.h` ties `VF_STRATE_PARAM_FIELDS` to `FStrateGenerationParams` (83 fields + 8 B padding). |
@@ -331,9 +332,9 @@ casse cette arithmetique *structurellement* — `BeginPlay` loggue une **Error**
| `LoadTile` | 9033 | Budget check → `UE::Tasks::Launch` background gen+mesh; RAII task guard. Worker runs `Generator->ClassifyTile` first (T1.d): AllSolid/AllAir ⇒ skip `GenerateMesh`, tile stays empty (capture tiles always generate). It carries `GenerationEpoch`/desired epoch and checks `bShuttingDown`, generation pause, and its read-only cancellation token between expensive stages; obsolete work returns through `ProcessQueue`. STRATE CONTENT CUT: tiles ≥ `StrateContentCutMinLevel` pass the player-strate band (`MeshBandChunkLo/Hi` → voxels) to `GenerateMesh` + stamp it on `FChunkResult::BandChunkLo/Hi`; band change re-queues via `BandRemeshQueue` (see `UpdateChunksAroundPosition`). `voxel.TestFullResClipLevels` is a measurement-only runtime override; default -1 uses the authored setting. TOO-COARSE SKIP: if one cell is taller than the band (`Step > band height` — level ≥7 territory) the tile is enqueued EMPTY without launching a task (cell-granular cut could only render garbage). §8.10. |
| `UnloadTile` | — | Clears tile state; marks any matching generation request obsolete without removing its pending slot, so a late result cannot clear a replacement request; the component is PARKED in the pool (T2.c), not destroyed. |
| `ApplyMeshToTile` | 9838 | Upload geometry. One component per tile (clipmap keeps count low; supersedes the old region batching); worker-built streams (T1.f) → `CreateSectionGroup(MoveTemp)`. Reuses the component's existing `URealtimeMesh` (no per-apply mesh alloc). **F17: two polygroups** (0 ground / 1 sky-cap, per-triangle class from the mesher) → RMC auto-section per non-empty group; slot 0 = override/default material, slot 1 = `CeilingMaterial` (fallback ground); config gated by `FChunkResult::bHasGroundTris/bHasCeilingTris`. Takes `FChunkResult&`; strate lookups clamp Z into `Result.BandChunkLo/Hi` (strate content cut) — ground at clamped bottom chunk, cap at clamped top (mid = gap fallback). Collision level-0 only (T1.c) both groups; shadow per SECTION: ground casts at level≤1, cap never. Ordinary same-tile re-cooks retain the installed body while it is valid. A carve intersecting the pawn's current support tile and capsule bounds suppresses that old body until the replacement cook is installed. `voxel.ProfileCollisionCookStages` reports generation/apply waits, combined RMC time, completion hops, triangle/skirt counts and collision settings. With `voxel.Step1PrequeueVisibleLOD0>0`, logs apply frame and waits for `GetLastRenderTime` to advance for first-visible frame; `voxel.Step1UploadDisableCollision=1` isolates render upload. Routes each terrain base through shared MIDs; disabled volume MIDs bind neutral `VolP*` and null `VolTex*`. **§9.4:** `SetVisibility(false)` when the tile is in `CollisionOnlyTiles` (anchor-only, hidden — collision still cooks). §8.10 + ARCHITECTURE SurfaceWorld row. |
| `HandleRealtimeMeshCollisionBodyUpdated` / `HandleTileCollisionCookComplete` / `PollStep1SupportSweepProbes` | 6964 / 7192 / — | Correlate RMC body-install event and collision-future completion. `voxel.ProfileCollisionCookStages=1` logs worker queue/gen, result wait, apply/upload/config submission, RMC submit→body update, body event→future, future→game-thread hop, submit frames, pending VoxelForge cooks, priority, collision flags, and triangle/skirt totals. `voxel.Step1CollisionProbe=1` adds the support component's physics-state-ready timestamp and polls a real downward capsule sweep until that component is hit. RMC's end-frame dequeue, async worker start, geometry extraction and core cook remain inside the external plugin and are not individually timestampable from this module. |
| `FVoxelStep1CoreCookWork` / `FStep1CoreCookPoolOwner` | — | Optional `voxel.Step1CoreCookBenchmark=1` copies critical L0 streams after meshing, appends collision triangles and calls RMC's public `CookComplexMesh` on a dedicated one-thread high-priority pool. It records queue delay, extraction and core cook separately. The benchmark sidecar is not installed as the tile BodySetup and is disabled by default. |
| Step 1 measurement CVars | — | `voxel.Step1GenerationCap=-1` uses the normal cap (`0` pauses, positive values cap workers); `voxel.Step1PauseGenerationAfterSeconds=0` disables the timed pause; `voxel.Step1MaxMeshApplies=0` uses the asset cap; `voxel.Step1PrequeueVisibleLOD0=0` disables the held upload corpus; `voxel.Step1UploadDisableCollision=0`, `voxel.Step1CollisionProbe=0`, and `voxel.Step1CoreCookBenchmark=0`. All default off. |
| `HandleRealtimeMeshCollisionBodyUpdated` / `HandleTileCollisionCookComplete` / `PollStep1SupportSweepProbes` | 6964 / 7192 / — | Correlate RMC body-install event and collision-future completion. `voxel.ProfileCollisionCookStages=1` logs worker queue/gen, result wait, apply/upload/config submission, RMC submit→body update, body event→future, future→game-thread hop, submit frames, pending VoxelForge cooks, priority, collision flags, and triangle/skirt totals; it also reports mesh-generation-end→ready. `voxel.Step1CollisionProbe=1` sweeps the pawn capsule against the exact component pointer, logs both the first hit and whether it is the target body, and checks every body in the captured support cohort. RMC's end-frame dequeue, async worker start, geometry extraction and core cook remain inside the external plugin and are not individually timestampable from this module. |
| `FVoxelStep1CoreCookWork` / `FStep1CoreCookPoolOwner` | — | Optional `voxel.Step1CoreCookBenchmark=1` captures the pawn's first exact radius box (normally 27 L0 tiles) on the game thread, tags those results through worker meshing, and sends their streams plus any out-of-box critical support tile to a dedicated one-thread high-priority pool. It records queue delay, extraction and core cook separately. One approximately 4k-triangle cohort stream is retained and replayed after `Step1PauseGenerationAfterSeconds` pauses generation. The benchmark sidecar is not installed as the tile BodySetup and is disabled by default. |
| Step 1 measurement CVars | — | `voxel.Step1GenerationCap=-1` uses the normal cap (`0` pauses, positive values cap worker tasks); `voxel.Step1ExitAfterFrames=0` disables a frame-counted clean test exit; `voxel.Step1PauseGenerationAfterSeconds=0` disables the timed pause; `voxel.Step1MaxMeshApplies=0` uses the asset cap; `voxel.Step1PrequeueVisibleLOD0=0` disables the held upload corpus; `voxel.Step1UploadDisableCollision=0`, `voxel.Step1CollisionProbe=0`, `voxel.Step1CoreCookBenchmark=0`, and `voxel.Step1CsvCaptureOnBeginPlay=0`. All default off. The launch-only `voxel.TestClipRadius` transiently duplicates settings and clamps a positive requested radius to 1–16; it is absent by default and leaves the authored asset unchanged. The prequeue release emits the default-global `VFStep1UploadCorpusReleased` CSV event; `voxel.Step1CsvCaptureOnBeginPlay=1` emits `VFStep1GameBeginPlay`. Use UE's `-csvStartOnEvent=<event>` to begin capture after RHI initialization. |
| `AcquireTileComponent` / `ReleaseTileComponent` | — | **T2.c component pool** (`TileComponentPool`, bounded): park on unload (geometry+collision stripped, hidden, stays registered), pop on apply — no `NewObject`/`RegisterComponent`/GC churn during travel & regen. §8.10. |
| `GetStrateAtPosition` | 8351 | Gameplay query → strate index. |
| `GetBiomeAtWorldLocation` | — | **BlueprintCallable** biome probe at a world point (undoes actor xf → voxel → `Generator::QueryBiomeAt`). Returns `FVoxelBiomeQuery` for BP debug ("what biome / how many decos under the cursor?"). |
@@ -443,8 +444,10 @@ serialization before they are written. The commandlet keeps `WorldRadiusVoxels=0
authored asset or diff layer, and enforces the default 25-minute / maximum 30-minute wall-clock
budget, reporting partial artifacts and completed modes on truncation. `-gameconfig=1` opts into the
authored settings' skirts/LOD octave drop and permits export-only parity of a single-strate asset;
the historical interior-slot requirement remains in force for walk/probe runs. `VoxelGeometryHash.h`
is the shared digest used by `Tools/VoxelForgeTest.ps1 -Scenario parity`.
the optional `-exporttilex/y/z` arguments choose a specific level-0 tile for export/game parity,
while omitting them preserves center-derived region selection. The historical interior-slot
requirement remains in force for walk/probe runs. `VoxelGeometryHash.h` is the shared digest used
by `Tools/VoxelForgeTest.ps1 -Scenario parity`.
### 3.7 Cave morphology (SDF rooms/tunnels) — `Public/VoxelCaveMorphology.h` + `.cpp`
Header is rich with inline docs. Two namespaces + a per-chunk cache system.