diff --git a/CODEMAP.md b/CODEMAP.md index 0cdad09..3dcb9e0 100644 --- a/CODEMAP.md +++ b/CODEMAP.md @@ -297,6 +297,7 @@ casse cette arithmetique *structurellement* — `BeginPlay` loggue une **Error** | `RegenerateAllChunks()` | 530 | Bumps epoch, unloads all → Tick reloads. CallInEditor button. | | `ValidateDeterminism()` | — | **F2 CallInEditor button (PIE)**: re-samples boundary points under left- vs right-chunk cache warm-ups + a same-alignment repeat; any non-zero delta = window-invariance regression (§8.4). Run after every "bit-identical" hot-path refactor. | | `GetMaxConcurrentTasks()` | — | T2.d — asset `MaxConcurrentTasks` capped to logical cores − 2 (all three budget checks use it). | +| `GetActiveGenerationTaskCount()` | — | Normal mode returns the existing pending count. With `voxel.Step1PrequeueVisibleLOD0>0`, completed held results are excluded from active worker admission so a full render corpus can be generated before the upload release. | | `PostEditChangeProperty` | 1000 | Editor live-edit hook. | | `OnObjectModifiedInEditor` | 1022 | Regenerates when a strate asset is edited (if `bLiveEditStrates`). | | `EndPlay` | 1118 | Sets `bShuttingDown`, **waits for `ActiveTaskCount`→0**, unbinds delegate. | @@ -306,7 +307,7 @@ casse cette arithmetique *structurellement* — `BeginPlay` loggue une **Error** | `GetPlayerPosition` | 4831 | Pawn position or zero. | | `AdvanceHeadlessDigMotionTest` / `RecordHeadlessDigMotionFrame` | 3060 / 3217 | Explicit `-voxel.TestDigMotion=1` repro: digs eight times beside a walking pawn, records each frame's vertical velocity, gate, support tile/readiness, movement mode, then digs underfoot and requires ordinary falling. The accompanying gap audit is scoped to the dig session. | | `UpdateVisualStalenessAudit` | 5034 | Test-only camera-frustum coverage audit: tracks up to 24 real pre-skirt isosurface points per visible component, then counts frames where no visible tile component covers a previously visible, still-desired point (excluding edited bounds). `voxel.TestVisualGapInjectEarlyHide=1` hides a valid component for three frames and must be caught. `voxel.TestGapAuditLegacyHandoff=1` restores old handoff culling for A/B baselines. | -| `ProcessPendingChunks` | 7435 | Drains critical, edited, near, then ordinary result queues under the per-frame budget; reads the per-request cancellation token before removing its map entry, drops obsolete/aborted results, then applies each live result via `ApplyTileResult`. Collision profiling timestamps the result's arrival here to separate worker completion from game-thread apply wait. | +| `ProcessPendingChunks` | 7435 | Drains critical, edited, near, then ordinary result queues under the per-frame budget; reads the per-request cancellation token before removing its map entry, drops obsolete/aborted results, then applies each live result via `ApplyTileResult`. Collision profiling timestamps the result's arrival here to separate worker completion from game-thread apply wait. Step 1's `voxel.Step1PrequeueVisibleLOD0` holds completed results until the requested non-empty L0 count is ready, then releases them through this same drain. | | `ApplyTileResult` | — | **Shared game-thread apply** for one `FChunkResult` (async drain + sync carve): discards stale generation epochs and keys no longer desired, marks loaded, ingests capture, EMPTY releases the tile's existing component (a re-gen can flip content→empty on band change — old geometry must not linger), else `ApplyMeshToTile`. Returns true iff a visible mesh uploaded (counts the budget). Doesn't touch `PendingTiles` (caller's). | | `FVoxelStackSampler` / `FScopedVoxelStackRegistration` (`VoxelStackSampler.h/.cpp`) | — | **Diagnostic raw stack sampler, off by default.** A sampler thread captures registered generation workers' stacks (`CaptureThreadStackBackTrace`) every N µs and tags each with the tile's LOD. It writes bounded raw PCs to CSV plus a symbolization-free summary under `Saved/`; symbol resolution is deferred until after process exit so the measured process never initializes/calls DbgHelp for this diagnostic. Lifecycle teardown is serialized and registration TLS is session-tagged. Registration is per tile TASK in `LoadTile` (and the export's tasks), never per sample; disabled costs one relaxed atomic load. Game `-voxel.SampleStacks=`, commandlet `-samplestacks[=us]`. | | Room-cache performance (2026-09-13): `VF_FindPlayerFitPointForRoomMemoized` / `GPlayerFitMemo` (`VoxelCaveMorphology.cpp`), `voxel.TileCacheWindow` + `VoxelGenLOD::TileOriginVoxels/TileCellsPerAxis` (`VoxelGenerator.h`), `FChunkSDFSpatialIndex` | — | **Player-fit memo:** exact 97-word key, process-wide, `voxel.PlayerFitMemo`. **Tile cache window:** the room cache covers the requesting tile's footprint plus a fixed collect margin. It is set via `TGuardValue` in `GenerateMesh`/classify, and is window-invariant by construction (the margin does not grow with the tile). **LOD-gated:** fused path at LOD ≥ `voxel.TileCacheWindowMinLOD` (3), op-stack at LOD ≥ `voxel.TileCacheWindowOpStackMinLOD` (4); lower LODs keep the 4-chunk worker window, because the tile window slowed LOD0 evaluation ~6%. Strates whose authored terrain ops exceed the proven 256-voxel extent (or are malformed/unloaded) fall back to the legacy window (`VF_ComputeTerrainOpEnvelope`). `voxel.SpatialIndex` is -1 auto (off at fused LOD0, on elsewhere), 0 or 1. **Spatial index:** ordered XY buckets over cache primitives, exact rejection. The `static_assert` in `VoxelStrateTypes.h` ties `VF_STRATE_PARAM_FIELDS` to `FStrateGenerationParams` (83 fields + 8 B padding). | @@ -329,8 +330,10 @@ casse cette arithmetique *structurellement* — `BeginPlay` loggue une **Error** | `RegisterStreamingAnchor` / `UnregisterStreamingAnchor` | — | **BlueprintCallable §9.3:** add/remove an actor as a streaming anchor (`EVoxelAnchorPolicy` CollisionOnly/FullVisual + XY/ZBelow/ZAbove box). Idempotent; forces a rebuild next Tick. §9.4: CollisionOnly tiles cook collision but are hidden (no draw/VSM) unless the player clipmap wants them too. | | `LoadTile` | 9033 | Budget check → `UE::Tasks::Launch` background gen+mesh; RAII task guard. Worker runs `Generator->ClassifyTile` first (T1.d): AllSolid/AllAir ⇒ skip `GenerateMesh`, tile stays empty (capture tiles always generate). It carries `GenerationEpoch`/desired epoch and checks `bShuttingDown`, generation pause, and its read-only cancellation token between expensive stages; obsolete work returns through `ProcessQueue`. STRATE CONTENT CUT: tiles ≥ `StrateContentCutMinLevel` pass the player-strate band (`MeshBandChunkLo/Hi` → voxels) to `GenerateMesh` + stamp it on `FChunkResult::BandChunkLo/Hi`; band change re-queues via `BandRemeshQueue` (see `UpdateChunksAroundPosition`). `voxel.TestFullResClipLevels` is a measurement-only runtime override; default -1 uses the authored setting. TOO-COARSE SKIP: if one cell is taller than the band (`Step > band height` — level ≥7 territory) the tile is enqueued EMPTY without launching a task (cell-granular cut could only render garbage). §8.10. | | `UnloadTile` | — | Clears tile state; marks any matching generation request obsolete without removing its pending slot, so a late result cannot clear a replacement request; the component is PARKED in the pool (T2.c), not destroyed. | -| `ApplyMeshToTile` | 9838 | Upload geometry. One component per tile (clipmap keeps count low; supersedes the old region batching); worker-built streams (T1.f) → `CreateSectionGroup(MoveTemp)`. Reuses the component's existing `URealtimeMesh` (no per-apply mesh alloc). **F17: two polygroups** (0 ground / 1 sky-cap, per-triangle class from the mesher) → RMC auto-section per non-empty group; slot 0 = override/default material, slot 1 = `CeilingMaterial` (fallback ground); config gated by `FChunkResult::bHasGroundTris/bHasCeilingTris`. Takes `FChunkResult&`; strate lookups clamp Z into `Result.BandChunkLo/Hi` (strate content cut) — ground at clamped bottom chunk, cap at clamped top (mid = gap fallback). Collision level-0 only (T1.c) both groups; shadow per SECTION: ground casts at level≤1, cap never. Ordinary same-tile re-cooks retain the installed body while it is valid. A carve intersecting the pawn's current support tile and capsule bounds suppresses that old body until the replacement cook is installed. `voxel.ProfileCollisionCookStages` reports generation/apply waits, combined RMC time, completion hops, triangle/skirt counts and collision settings. Routes each terrain base through shared MIDs; disabled volume MIDs bind neutral `VolP*` and null `VolTex*`. **§9.4:** `SetVisibility(false)` when the tile is in `CollisionOnlyTiles` (anchor-only, hidden — collision still cooks). §8.10 + ARCHITECTURE SurfaceWorld row. | -| `HandleRealtimeMeshCollisionBodyUpdated` / `HandleTileCollisionCookComplete` | 6964 / 7192 | Correlate RMC body-install event and collision-future completion. `voxel.ProfileCollisionCookStages=1` logs worker queue/gen, result wait, apply/upload/config submission, RMC submit→body update, body event→future, future→game-thread hop, submit frames, pending VoxelForge cooks, priority, collision flags, and triangle/skirt totals. RMC's internal end-of-frame, async-pool wait, and cook remain one combined interval because they are inside the external RMC plugin. | +| `ApplyMeshToTile` | 9838 | Upload geometry. One component per tile (clipmap keeps count low; supersedes the old region batching); worker-built streams (T1.f) → `CreateSectionGroup(MoveTemp)`. Reuses the component's existing `URealtimeMesh` (no per-apply mesh alloc). **F17: two polygroups** (0 ground / 1 sky-cap, per-triangle class from the mesher) → RMC auto-section per non-empty group; slot 0 = override/default material, slot 1 = `CeilingMaterial` (fallback ground); config gated by `FChunkResult::bHasGroundTris/bHasCeilingTris`. Takes `FChunkResult&`; strate lookups clamp Z into `Result.BandChunkLo/Hi` (strate content cut) — ground at clamped bottom chunk, cap at clamped top (mid = gap fallback). Collision level-0 only (T1.c) both groups; shadow per SECTION: ground casts at level≤1, cap never. Ordinary same-tile re-cooks retain the installed body while it is valid. A carve intersecting the pawn's current support tile and capsule bounds suppresses that old body until the replacement cook is installed. `voxel.ProfileCollisionCookStages` reports generation/apply waits, combined RMC time, completion hops, triangle/skirt counts and collision settings. With `voxel.Step1PrequeueVisibleLOD0>0`, logs apply frame and waits for `GetLastRenderTime` to advance for first-visible frame; `voxel.Step1UploadDisableCollision=1` isolates render upload. Routes each terrain base through shared MIDs; disabled volume MIDs bind neutral `VolP*` and null `VolTex*`. **§9.4:** `SetVisibility(false)` when the tile is in `CollisionOnlyTiles` (anchor-only, hidden — collision still cooks). §8.10 + ARCHITECTURE SurfaceWorld row. | +| `HandleRealtimeMeshCollisionBodyUpdated` / `HandleTileCollisionCookComplete` / `PollStep1SupportSweepProbes` | 6964 / 7192 / — | Correlate RMC body-install event and collision-future completion. `voxel.ProfileCollisionCookStages=1` logs worker queue/gen, result wait, apply/upload/config submission, RMC submit→body update, body event→future, future→game-thread hop, submit frames, pending VoxelForge cooks, priority, collision flags, and triangle/skirt totals. `voxel.Step1CollisionProbe=1` adds the support component's physics-state-ready timestamp and polls a real downward capsule sweep until that component is hit. RMC's end-frame dequeue, async worker start, geometry extraction and core cook remain inside the external plugin and are not individually timestampable from this module. | +| `FVoxelStep1CoreCookWork` / `FStep1CoreCookPoolOwner` | — | Optional `voxel.Step1CoreCookBenchmark=1` copies critical L0 streams after meshing, appends collision triangles and calls RMC's public `CookComplexMesh` on a dedicated one-thread high-priority pool. It records queue delay, extraction and core cook separately. The benchmark sidecar is not installed as the tile BodySetup and is disabled by default. | +| Step 1 measurement CVars | — | `voxel.Step1GenerationCap=-1` uses the normal cap (`0` pauses, positive values cap workers); `voxel.Step1PauseGenerationAfterSeconds=0` disables the timed pause; `voxel.Step1MaxMeshApplies=0` uses the asset cap; `voxel.Step1PrequeueVisibleLOD0=0` disables the held upload corpus; `voxel.Step1UploadDisableCollision=0`, `voxel.Step1CollisionProbe=0`, and `voxel.Step1CoreCookBenchmark=0`. All default off. | | `AcquireTileComponent` / `ReleaseTileComponent` | — | **T2.c component pool** (`TileComponentPool`, bounded): park on unload (geometry+collision stripped, hidden, stays registered), pop on apply — no `NewObject`/`RegisterComponent`/GC churn during travel & regen. §8.10. | | `GetStrateAtPosition` | 8351 | Gameplay query → strate index. | | `GetBiomeAtWorldLocation` | — | **BlueprintCallable** biome probe at a world point (undoes actor xf → voxel → `Generator::QueryBiomeAt`). Returns `FVoxelBiomeQuery` for BP debug ("what biome / how many decos under the cursor?"). | diff --git a/RESEARCH-SPEED-2026-09.md b/RESEARCH-SPEED-2026-09.md index 81ff56b..ce9d9b6 100644 --- a/RESEARCH-SPEED-2026-09.md +++ b/RESEARCH-SPEED-2026-09.md @@ -34,7 +34,7 @@ The steps below describe **future plugin experiments**. This research pass makes 1. **Day-one falsification: stage the existing collision and upload clocks.** Add timestamps for collision request, RMC end-of-frame dequeue, `GThreadPool` worker start, geometry extraction end, `CookComplexMesh` start/end, game-thread `ApplyCollisionUpdate` start/end, physics-state ready and first successful capsule sweep. Keep existing generation, mesh and gameplay behavior while profiling. Test one ~4,000-triangle tile and the pawn's current 3³=27-tile support neighborhood, idle at the current 14 generation tasks, and with the generation cap reduced to 12. If the cook's queue wait dominates, test a dedicated priority cook dispatched directly after player-support mesh extraction; compare its core cook and install times with RMC's route. In parallel, prequeue a 341-tile render corpus and run caps 4/8/16 with the same 2 ms drain guard; record visible applies/s, game/render/RHI clocks and first-visible frame. **Pass the cook shortcut only if** core cook p95 ≤10 ms, the 27-tile player support is ready ≤150 ms after meshes exist under load, and an edit-time field override can switch floor/support within the first simulated/rendered frame with zero invented velocity in 100 repeated underfoot digs. If any edit test fails, implement field-aware Mover as the player authority; if the shortcut passes all tests, use prioritized cooks for steady terrain plus field queries while an edit's cook is pending. **Pass the ungrouped upload path only if** it sustains ≥525 visible applies/s at 60 Hz with game-thread drain ≤2 ms/frame and no render/RHI backlog. These are decision thresholds, not existing measurements. 2. **Standalone exact-float evaluator ablation.** Extract representative *real* TunnelNetwork and composer plans, including warped rooms, passages, stratum transitions, route floors and diff edits, into `Saved/Research` only. Keep the plugin's active evaluator and worlds untouched. Compare baseline point API to a prepared float region/brick API at the same lattice coordinates and op order. The reference is the chosen **new-world float definition**, not old-season geometry. **Pass:** zero 16-bit quantized-value disagreements and zero route/capsule regression across at least five seeds; ≤**600 ms** for the full 259³ cold ring on six workers before certificates. Record candidate visits, cache checks and worker CPU. Failure means local compilation did not remove enough work; investigate before rewriting numeric math. 3. **Add certificates, shared lattice and contouring.** Prove final-sign bounds after all writers and edits; classify 8³ bricks, evaluate uncertain ones, share global corners, and mesh them with today's marching-cubes table. Keep the old tile route as a versioned reference while the new route is validated. **Pass:** zero false-uniform bricks and zero sign mismatch at all 17,373,979 ring lattice sites for the test corpus, cold worker generation ≤**350 ms**, and max render-visible/full ring ≤**900 ms** in the later UE gate. If certification consumes more time than it saves, fall back to evaluating uncertain regions without that proof; do not repeat the archived 0.588→2.057 s interval-control mistake. -3b. **AVX2 check against the new norm (owner's request, 2026-09-29).** Once step 3's brick evaluator is the baseline, try an explicit 8-lane AVX2 version of its hot loop (the plugin's current noise path is 4-lane SSE4.1). Float lanes stay deterministic only with no FMA contraction and each lane performing exactly the scalar op sequence. **Keep it only if** it is faster on the step-3 corpus and the quantized lattice hash is identical to the scalar path on two runs; otherwise record the number and drop it. The round-two integer AVX2 probe (4.25–5.86× over its own scalar) is the reference for what explicit vectorisation can buy. +3b. **AVX2 check against the new norm (owner's request, 2026-09-29).** Once step 3's brick evaluator is the baseline, try an explicit 8-lane AVX2 version of its hot loop (the plugin's current noise path is 4-lane SSE4.1). **The owner does not require AVX2 to match the scalar path bit for bit** (clarified 2026-09-29): only the shipped product must be deterministic across players' machines. If adopted, the AVX2 path *is* the canonical world definition (FMA allowed, as long as it is always used), and every machine must run that same path: require AVX2 at startup rather than falling back to a scalar path that would produce a different world. **Keep it only if** it is faster on the step-3 corpus and the AVX2 path gives the same lattice hash on two runs and on two different AVX2 CPU models; otherwise record the number and drop it. The round-two integer AVX2 probe (4.25–5.86× over its own scalar) is the reference for what explicit vectorisation can buy. 4. **Choose and implement player collision.** Use step 1 to select prioritized Chaos+edit overlay or field-aware Mover; keep ordinary collision with nonterrain actors and keep NPC/projectile field-query backup. Version floor/contact state by edit sequence. **Pass:** at every teleport the player can perform an actual blocked capsule movement on the intended terrain ≤**900 ms** after the request, 100 underfoot digs add **0** spurious velocity impulses, and 100 excavations with no cave below have **0** frames of false support or false air. The full visible ring must also meet the separate ≤1,000 ms hard deadline. A successful cook timing without the edit tests is insufficient. 5. **Decide render batching from measured throughput.** Keep per-tile RMC if it sustains ≥525 visible applies/s and the first-visible gate ≤900 ms. Otherwise group 2³ tiles per page while retaining tile-addressable sections and the same materials/LOD transition topology; rerun the same corpus and require ≥**99 page applies/s** for at most 64 pages in 0.65 s, ≤2 ms/frame game-thread drain and no render-thread tail past the 1 s deadline. If grouping causes whole-page reupload on small edits or still misses, use a dedicated batched render proxy. Compare first-visible time against the same ≤900 ms gate. 6. **Version the edit and GPU visual protocol.** Quantize float-produced lattice density with specified rounding/saturation; use fixed integer edit composition, stable per-cell triangle counts/prefix sums, global edge ownership and one edit sequence across touched pages/LODs. Keep CPU procedural generation, composer, route graph and ordered edit replication as the authority. Test AMD and NVIDIA output hashes against CPU reference; on a single machine, two full cold runs must yield the **same field/sign/mesh hash**. **Pass:** a radius-10 m crater (up to 714 bricks/64 tiles) appears throughout its region in the **first submitted render frame containing the edit**, no gap/ghost/ripple at borders or LOD seams, CPU/GPU sign disagreement **0**, and added GPU cost ≤**2 ms** within an 11.1 ms 90 Hz VR frame in 20 varied positions. If direct GPU remeshing exceeds that, test the complete-region raymarch overlay with the same visual and frame gates; do not publish a partial mesh. diff --git a/Source/VoxelForge/Private/VoxelWorld.cpp b/Source/VoxelForge/Private/VoxelWorld.cpp index 100eaa2..f91f40f 100644 --- a/Source/VoxelForge/Private/VoxelWorld.cpp +++ b/Source/VoxelForge/Private/VoxelWorld.cpp @@ -36,6 +36,11 @@ #include "GameFramework/CharacterMovementComponent.h" #include "Components/CapsuleComponent.h" #include "Components/PrimitiveComponent.h" +#include "CollisionQueryParams.h" +#include "Engine/World.h" +#include "Misc/IQueuedWork.h" +#include "Misc/QueuedThreadPool.h" +#include "RealtimeMeshCollisionLibrary.h" #include "Camera/PlayerCameraManager.h" #include "Async/Async.h" #include "Materials/MaterialInstanceDynamic.h" @@ -90,6 +95,140 @@ namespace GVoxelForgeProfileCollisionCookStages, TEXT("Log per-tile collision queue/apply/RMC/future timing and cooked geometry counts.")); + int32 GVoxelForgeStep1GenerationCap = -1; + FAutoConsoleVariableRef CVarVoxelForgeStep1GenerationCap( + TEXT("voxel.Step1GenerationCap"), + GVoxelForgeStep1GenerationCap, + TEXT("Step 1 measurement override: -1 uses the normal cap, 0 pauses generation, positive values cap worker tasks.")); + + float GVoxelForgeStep1PauseGenerationAfterSeconds = 0.0f; + FAutoConsoleVariableRef CVarVoxelForgeStep1PauseGenerationAfterSeconds( + TEXT("voxel.Step1PauseGenerationAfterSeconds"), + GVoxelForgeStep1PauseGenerationAfterSeconds, + TEXT("Step 1 measurement: switch the generation cap to paused after this many run seconds; 0 disables the timed pause.")); + double GVoxelForgeStep1StartSeconds = 0.0; + bool bVoxelForgeStep1GenerationPauseApplied = false; + + int32 GVoxelForgeStep1MaxMeshApplies = 0; + FAutoConsoleVariableRef CVarVoxelForgeStep1MaxMeshApplies( + TEXT("voxel.Step1MaxMeshApplies"), + GVoxelForgeStep1MaxMeshApplies, + TEXT("Step 1 measurement override: 0 uses the asset MaxMeshAppliesPerFrame, otherwise caps visible applies per frame.")); + + int32 GVoxelForgeStep1PrequeueVisibleLOD0 = 0; + FAutoConsoleVariableRef CVarVoxelForgeStep1PrequeueVisibleLOD0( + TEXT("voxel.Step1PrequeueVisibleLOD0"), + GVoxelForgeStep1PrequeueVisibleLOD0, + TEXT("Step 1 measurement: hold generated results until this many non-empty level-0 meshes are ready, then release them for upload.")); + + int32 GVoxelForgeStep1UploadDisableCollision = 0; + FAutoConsoleVariableRef CVarVoxelForgeStep1UploadDisableCollision( + TEXT("voxel.Step1UploadDisableCollision"), + GVoxelForgeStep1UploadDisableCollision, + TEXT("Step 1 upload isolation: suppress terrain collision requests while measuring prequeued render uploads.")); + + int32 GVoxelForgeStep1CollisionProbe = 0; + FAutoConsoleVariableRef CVarVoxelForgeStep1CollisionProbe( + TEXT("voxel.Step1CollisionProbe"), + GVoxelForgeStep1CollisionProbe, + TEXT("Step 1 measurement: observe physics-state readiness and the first support capsule sweep after RMC body install.")); + + int32 GVoxelForgeStep1CoreCookBenchmark = 0; + FAutoConsoleVariableRef CVarVoxelForgeStep1CoreCookBenchmark( + TEXT("voxel.Step1CoreCookBenchmark"), + GVoxelForgeStep1CoreCookBenchmark, + TEXT("Step 1 measurement: directly extract and CookComplexMesh for critical L0 support tiles on a dedicated one-thread pool; does not install this sidecar body.")); + + struct FStep1CoreCookPoolOwner + { + FCriticalSection Guard; + FQueuedThreadPool* Pool = nullptr; + + FQueuedThreadPool* GetOrCreate() + { + FScopeLock Lock(&Guard); + if (Pool == nullptr) + { + Pool = FQueuedThreadPool::Allocate(); + if (Pool == nullptr + || !Pool->Create(1, 32 * 1024, TPri_AboveNormal, + TEXT("VoxelForgeStep1Cook"))) + { + if (Pool != nullptr) + { + delete Pool; + Pool = nullptr; + } + } + } + return Pool; + } + + ~FStep1CoreCookPoolOwner() + { + if (Pool != nullptr) + { + Pool->Destroy(); + delete Pool; + Pool = nullptr; + } + } + }; + + FStep1CoreCookPoolOwner GVoxelForgeStep1CoreCookPool; + + class FVoxelStep1CoreCookWork final : public IQueuedWork + { + public: + FVoxelStep1CoreCookWork( + TSharedPtr InStreams, + FVoxelTileKey InTile, int32 InTriangleCount, uint64 InQueueCycles) + : Streams(MoveTemp(InStreams)), Tile(InTile), TriangleCount(InTriangleCount), + QueueCycles(InQueueCycles) + { + } + + virtual void DoThreadedWork() override + { + const uint64 WorkerStartCycles = FPlatformTime::Cycles64(); + FRealtimeMeshCollisionMesh CollisionMesh; + const uint64 ExtractionStartCycles = FPlatformTime::Cycles64(); + const bool bExtracted = Streams.IsValid() + && URealtimeMeshCollisionTools::AppendStreamsToCollisionMesh( + CollisionMesh, *Streams, 0); + const uint64 ExtractionEndCycles = FPlatformTime::Cycles64(); + const uint64 CookStartCycles = FPlatformTime::Cycles64(); + if (bExtracted) + { + URealtimeMeshCollisionTools::CookComplexMesh(CollisionMesh); + } + const uint64 CookEndCycles = FPlatformTime::Cycles64(); + UE_LOG(LogTemp, Display, + TEXT("[VoxelForgeStep1CoreCook] tile=(%d,%d,%d) level=%d extracted=%d " + "triangles=%d collision_vertices=%d queue_to_worker_ms=%.3f " + "stream_extract_ms=%.3f cook_ms=%.3f total_ms=%.3f thread_id=%u"), + Tile.Coord.X, Tile.Coord.Y, Tile.Coord.Z, Tile.Level, + bExtracted ? 1 : 0, TriangleCount, CollisionMesh.GetVertices().Num(), + FPlatformTime::ToSeconds64(WorkerStartCycles - QueueCycles) * 1000.0, + FPlatformTime::ToSeconds64(ExtractionEndCycles - ExtractionStartCycles) * 1000.0, + FPlatformTime::ToSeconds64(CookEndCycles - CookStartCycles) * 1000.0, + FPlatformTime::ToSeconds64(CookEndCycles - WorkerStartCycles) * 1000.0, + FPlatformTLS::GetCurrentThreadId()); + delete this; + } + + virtual void Abandon() override + { + delete this; + } + + private: + TSharedPtr Streams; + FVoxelTileKey Tile; + int32 TriangleCount = 0; + uint64 QueueCycles = 0; + }; + int32 GVoxelForgeTestFullResClipLevels = -1; FAutoConsoleVariableRef CVarVoxelForgeTestFullResClipLevels( TEXT("voxel.TestFullResClipLevels"), @@ -1638,6 +1777,14 @@ void AVoxelWorld::BeginPlay() int32 CommandLineSealedSolidProof = GVoxelForgeSealedSolidProof; int32 CommandLineSampleStacks = GVoxelForgeSampleStacks; int32 CommandLineCollisionGate = GVoxelForgeCollisionGate; + int32 CommandLineStep1GenerationCap = GVoxelForgeStep1GenerationCap; + float CommandLineStep1PauseGenerationAfterSeconds = + GVoxelForgeStep1PauseGenerationAfterSeconds; + int32 CommandLineStep1MaxMeshApplies = GVoxelForgeStep1MaxMeshApplies; + int32 CommandLineStep1PrequeueVisibleLOD0 = GVoxelForgeStep1PrequeueVisibleLOD0; + int32 CommandLineStep1UploadDisableCollision = GVoxelForgeStep1UploadDisableCollision; + int32 CommandLineStep1CollisionProbe = GVoxelForgeStep1CollisionProbe; + int32 CommandLineStep1CoreCookBenchmark = GVoxelForgeStep1CoreCookBenchmark; if (FParse::Value(FCommandLine::Get(), TEXT("voxel.ProfileDensity="), CommandLineProfile)) { GVoxelForgeProfileDensity = CommandLineProfile; @@ -1664,6 +1811,49 @@ void AVoxelWorld::BeginPlay() { GVoxelForgeProfileCollisionCookStages = CommandLineProfileCollisionCookStages; } + if (FParse::Value(FCommandLine::Get(), TEXT("voxel.Step1GenerationCap="), + CommandLineStep1GenerationCap)) + { + GVoxelForgeStep1GenerationCap = CommandLineStep1GenerationCap; + } + if (FParse::Value(FCommandLine::Get(), TEXT("voxel.Step1PauseGenerationAfterSeconds="), + CommandLineStep1PauseGenerationAfterSeconds)) + { + GVoxelForgeStep1PauseGenerationAfterSeconds = FMath::Max( + 0.0f, CommandLineStep1PauseGenerationAfterSeconds); + } + if (FParse::Value(FCommandLine::Get(), TEXT("voxel.Step1MaxMeshApplies="), + CommandLineStep1MaxMeshApplies)) + { + GVoxelForgeStep1MaxMeshApplies = CommandLineStep1MaxMeshApplies; + } + if (FParse::Value(FCommandLine::Get(), TEXT("voxel.Step1PrequeueVisibleLOD0="), + CommandLineStep1PrequeueVisibleLOD0)) + { + GVoxelForgeStep1PrequeueVisibleLOD0 = FMath::Max(0, CommandLineStep1PrequeueVisibleLOD0); + } + if (FParse::Value(FCommandLine::Get(), TEXT("voxel.Step1UploadDisableCollision="), + CommandLineStep1UploadDisableCollision)) + { + GVoxelForgeStep1UploadDisableCollision = CommandLineStep1UploadDisableCollision; + } + if (FParse::Value(FCommandLine::Get(), TEXT("voxel.Step1CollisionProbe="), + CommandLineStep1CollisionProbe)) + { + GVoxelForgeStep1CollisionProbe = CommandLineStep1CollisionProbe; + } + if (FParse::Value(FCommandLine::Get(), TEXT("voxel.Step1CoreCookBenchmark="), + CommandLineStep1CoreCookBenchmark)) + { + GVoxelForgeStep1CoreCookBenchmark = CommandLineStep1CoreCookBenchmark; + } + UE_LOG(LogTemp, Display, + TEXT("[VoxelForgeStep1] gen_cap=%d pause_after_s=%.3f max_applies=%d prequeue_l0=%d " + "disable_collision=%d collision_probe=%d core_cook_benchmark=%d"), + GVoxelForgeStep1GenerationCap, GVoxelForgeStep1PauseGenerationAfterSeconds, + GVoxelForgeStep1MaxMeshApplies, + GVoxelForgeStep1PrequeueVisibleLOD0, GVoxelForgeStep1UploadDisableCollision, + GVoxelForgeStep1CollisionProbe, GVoxelForgeStep1CoreCookBenchmark); if (FParse::Value( FCommandLine::Get(), TEXT("voxel.TestFullResClipLevels="), @@ -5726,6 +5916,26 @@ void AVoxelWorld::Tick(float DeltaTime) APawn* PlayerPawn = nullptr; const bool bHasPlayer = TryGetPlayerPosition(PlayerLastPos, &PlayerPawn); + if (GVoxelForgeStep1PauseGenerationAfterSeconds > 0.0f + && !bVoxelForgeStep1GenerationPauseApplied) + { + const double NowSeconds = FPlatformTime::Seconds(); + if (GVoxelForgeStep1StartSeconds <= 0.0) + { + GVoxelForgeStep1StartSeconds = NowSeconds; + } + else if (NowSeconds - GVoxelForgeStep1StartSeconds + >= static_cast(GVoxelForgeStep1PauseGenerationAfterSeconds)) + { + GVoxelForgeStep1GenerationCap = 0; + bVoxelForgeStep1GenerationPauseApplied = true; + UE_LOG(LogTemp, Display, + TEXT("[VoxelForgeStep1] generation_paused=1 elapsed_s=%.3f pending=%d held_results=%d"), + NowSeconds - GVoxelForgeStep1StartSeconds, PendingTiles.Num(), + Step1HeldApplyResults.Num()); + } + } + if (bHasPlayer) { // Capture velocity before the collision gate can deliberately zero it while the support // tile is cooking. The scheduler uses this as the movement heading for its next-tile @@ -5823,7 +6033,9 @@ void AVoxelWorld::Tick(float DeltaTime) } } ProcessPendingChunks(); + PollStep1SupportSweepProbes(); ReconcileReadyTransitionVisibility(); + PollStep1VisibleApplyProbes(); // Near-field evidence is independent of the full horizon settle point. Record one completion // for each desired-set epoch after this frame's result drain, including moving crossings. RecordLOD0RingReadyIfComplete(); @@ -6978,11 +7190,215 @@ void AVoxelWorld::HandleRealtimeMeshCollisionBodyUpdated(URealtimeMesh* Mesh, UB Pending.bBodyUpdatedEventSeen = true; Pending.BodyUpdatedEventCycles = FPlatformTime::Cycles64(); Pending.BodyUpdatedEventFrame = GFrameCounter; + if (GVoxelForgeStep1CollisionProbe != 0) + { + APawn* Pawn = nullptr; + FVector PlayerPosition = FVector::ZeroVector; + FVoxelTileKey CurrentSupportTile; + if (TryGetPlayerPosition(PlayerPosition, &Pawn) + && GetPlayerSupportTile(Pawn, PlayerPosition, CurrentSupportTile) + && CurrentSupportTile == Pair.Key) + { + const URealtimeMeshComponent* Component = TileComponents.FindRef(Pair.Key); + Pending.bPhysicsStateReadyAtBodyUpdate = IsValid(Component) + && Component->IsPhysicsStateCreated(); + if (Pending.bPhysicsStateReadyAtBodyUpdate) + { + Pending.PhysicsStateReadyCycles = FPlatformTime::Cycles64(); + } + FStep1SupportSweepProbe& Probe = Step1SupportSweepProbes.Add(Pair.Key); + Probe.CollisionRequestCycles = Pending.CollisionSubmittedCycles; + Probe.BodyUpdatedCycles = Pending.BodyUpdatedEventCycles; + Probe.PhysicsStateReadyCycles = Pending.PhysicsStateReadyCycles; + UE_LOG(LogTemp, Display, + TEXT("[VoxelForgeCollisionProbe] stage=body_updated tile=(%d,%d,%d) " + "request_to_body_ms=%.3f physics_ready=%d body_to_physics_ms=%.3f"), + Pair.Key.Coord.X, Pair.Key.Coord.Y, Pair.Key.Coord.Z, + FPlatformTime::ToSeconds64( + Probe.BodyUpdatedCycles - Probe.CollisionRequestCycles) * 1000.0, + Pending.bPhysicsStateReadyAtBodyUpdate ? 1 : 0, + Pending.bPhysicsStateReadyAtBodyUpdate + ? FPlatformTime::ToSeconds64( + Probe.PhysicsStateReadyCycles - Probe.BodyUpdatedCycles) * 1000.0 + : -1.0); + PollStep1SupportSweepProbes(); + } + } return; } } } +void AVoxelWorld::PollStep1SupportSweepProbes() +{ + if (GVoxelForgeStep1CollisionProbe == 0 || Step1SupportSweepProbes.IsEmpty()) + { + return; + } + + APawn* Pawn = nullptr; + FVector PlayerPosition = FVector::ZeroVector; + FVoxelTileKey CurrentSupportTile; + if (!TryGetPlayerPosition(PlayerPosition, &Pawn)) + { + return; + } + const bool bHaveCurrentSupportTile = GetPlayerSupportTile( + Pawn, PlayerPosition, CurrentSupportTile); + + const uint64 NowCycles = FPlatformTime::Cycles64(); + for (auto It = Step1SupportSweepProbes.CreateIterator(); It; ++It) + { + FStep1SupportSweepProbe& Probe = It.Value(); + URealtimeMeshComponent* Component = TileComponents.FindRef(It.Key()); + if (!IsValid(Component)) + { + It.RemoveCurrent(); + continue; + } + + if (Component->IsPhysicsStateCreated() && Probe.PhysicsStateReadyCycles == 0) + { + Probe.PhysicsStateReadyCycles = NowCycles; + UE_LOG(LogTemp, Display, + TEXT("[VoxelForgeCollisionProbe] stage=physics_ready tile=(%d,%d,%d) " + "request_to_physics_ms=%.3f body_to_physics_ms=%.3f"), + It.Key().Coord.X, It.Key().Coord.Y, It.Key().Coord.Z, + FPlatformTime::ToSeconds64( + Probe.PhysicsStateReadyCycles - Probe.CollisionRequestCycles) * 1000.0, + FPlatformTime::ToSeconds64( + Probe.PhysicsStateReadyCycles - Probe.BodyUpdatedCycles) * 1000.0); + } + + ACharacter* Character = Cast(Pawn); + UCapsuleComponent* Capsule = Character ? Character->GetCapsuleComponent() : nullptr; + UWorld* World = GetWorld(); + if (Probe.PhysicsStateReadyCycles != 0 && IsValid(Capsule) && World != nullptr) + { + const float Radius = Capsule->GetScaledCapsuleRadius(); + const float HalfHeight = Capsule->GetScaledCapsuleHalfHeight(); + const FVector Start = Pawn->GetActorLocation() + FVector(0.0, 0.0, 5.0); + const FVector End = Start - FVector(0.0, 0.0, 200.0); + const FCollisionShape CapsuleShape = FCollisionShape::MakeCapsule( + Radius, FMath::Max(Radius, HalfHeight - 2.0f)); + FCollisionQueryParams QueryParams( + SCENE_QUERY_STAT(VoxelForgeStep1SupportSweep), false, Pawn); + FHitResult Hit; + const bool bHit = World->SweepSingleByChannel( + Hit, Start, End, FQuat::Identity, ECC_Pawn, CapsuleShape, QueryParams); + FHitResult ObjectHit; + FCollisionObjectQueryParams ObjectQueryParams; + ObjectQueryParams.AddObjectTypesToQuery(Component->GetCollisionObjectType()); + const bool bObjectHit = World->SweepSingleByObjectType( + ObjectHit, Start, End, FQuat::Identity, ObjectQueryParams, + CapsuleShape, QueryParams); + if (!Probe.bFirstSweepAttemptLogged) + { + const UPrimitiveComponent* HitComponent = Hit.GetComponent(); + const UPrimitiveComponent* ObjectHitComponent = ObjectHit.GetComponent(); + const FBoxSphereBounds ComponentBounds = Component->Bounds; + UE_LOG(LogTemp, Display, + TEXT("[VoxelForgeCollisionProbe] stage=first_sweep_attempt tile=(%d,%d,%d) " + "pawn_channel_hit=%d pawn_channel_target=%d pawn_hit_component=%s " + "object_query_hit=%d object_query_target=%d object_hit_component=%s " + "collision_enabled=%d pawn_response=%d object_type=%d physics_state=%d " + "pawn=(%.1f,%.1f,%.1f) sweep_start_z=%.1f sweep_end_z=%.1f " + "component_bounds=(%.1f,%.1f,%.1f;%.1f,%.1f,%.1f) " + "current_support_match=%d"), + It.Key().Coord.X, It.Key().Coord.Y, It.Key().Coord.Z, + bHit ? 1 : 0, (bHit && HitComponent == Component) ? 1 : 0, + HitComponent ? *HitComponent->GetName() : TEXT("none"), + bObjectHit ? 1 : 0, + (bObjectHit && ObjectHitComponent == Component) ? 1 : 0, + ObjectHitComponent ? *ObjectHitComponent->GetName() : TEXT("none"), + static_cast(Component->GetCollisionEnabled()), + static_cast(Component->GetCollisionResponseToChannel(ECC_Pawn)), + static_cast(Component->GetCollisionObjectType()), + Component->IsPhysicsStateCreated() ? 1 : 0, + PlayerPosition.X, PlayerPosition.Y, PlayerPosition.Z, Start.Z, End.Z, + ComponentBounds.Origin.X, ComponentBounds.Origin.Y, ComponentBounds.Origin.Z, + ComponentBounds.BoxExtent.X, ComponentBounds.BoxExtent.Y, + ComponentBounds.BoxExtent.Z, + bHaveCurrentSupportTile && CurrentSupportTile == It.Key() ? 1 : 0); + Probe.bFirstSweepAttemptLogged = true; + } + if (bHit && Hit.GetComponent() == Component) + { + UE_LOG(LogTemp, Display, + TEXT("[VoxelForgeCollisionProbe] stage=first_support_capsule_sweep tile=(%d,%d,%d) " + "request_to_sweep_ms=%.3f body_to_sweep_ms=%.3f hit_z=%.3f"), + It.Key().Coord.X, It.Key().Coord.Y, It.Key().Coord.Z, + FPlatformTime::ToSeconds64(NowCycles - Probe.CollisionRequestCycles) * 1000.0, + FPlatformTime::ToSeconds64(NowCycles - Probe.BodyUpdatedCycles) * 1000.0, + Hit.ImpactPoint.Z); + It.RemoveCurrent(); + continue; + } + } + + if (Probe.BodyUpdatedCycles != 0 && NowCycles >= Probe.BodyUpdatedCycles + && FPlatformTime::ToSeconds64(NowCycles - Probe.BodyUpdatedCycles) > 10.0) + { + UE_LOG(LogTemp, Warning, + TEXT("[VoxelForgeCollisionProbe] stage=sweep_timeout tile=(%d,%d,%d) " + "physics_ready=%d support_match=%d body_to_timeout_ms=%.3f"), + It.Key().Coord.X, It.Key().Coord.Y, It.Key().Coord.Z, + Component->IsPhysicsStateCreated() ? 1 : 0, + bHaveCurrentSupportTile && CurrentSupportTile == It.Key() ? 1 : 0, + FPlatformTime::ToSeconds64(NowCycles - Probe.BodyUpdatedCycles) * 1000.0); + It.RemoveCurrent(); + } + } +} + +void AVoxelWorld::PollStep1VisibleApplyProbes() +{ + if (GVoxelForgeStep1PrequeueVisibleLOD0 <= 0 || Step1PendingVisibleApplyProbes.IsEmpty()) + { + return; + } + + const uint64 NowCycles = FPlatformTime::Cycles64(); + for (auto It = Step1PendingVisibleApplyProbes.CreateIterator(); It; ++It) + { + URealtimeMeshComponent* Component = TileComponents.FindRef(It.Key()); + if (!IsValid(Component)) + { + It.RemoveCurrent(); + continue; + } + + const float CurrentLastRenderTime = Component->GetLastRenderTime(); + if (Component->IsVisible() + && CurrentLastRenderTime > It.Value().LastRenderTimeBeforeApply + SMALL_NUMBER) + { + UE_LOG(LogTemp, Display, + TEXT("[VoxelForgeFirstVisible] tile=(%d,%d,%d) level=0 " + "applied_frame=%llu visible_frame=%llu frame_delta=%llu apply_to_visible_ms=%.3f"), + It.Key().Coord.X, It.Key().Coord.Y, It.Key().Coord.Z, + static_cast(It.Value().AppliedFrame), + static_cast(GFrameCounter), + static_cast(GFrameCounter - It.Value().AppliedFrame), + FPlatformTime::ToSeconds64(NowCycles - It.Value().AppliedCycles) * 1000.0); + It.RemoveCurrent(); + continue; + } + + if (NowCycles >= It.Value().AppliedCycles + && FPlatformTime::ToSeconds64(NowCycles - It.Value().AppliedCycles) > 10.0) + { + UE_LOG(LogTemp, Warning, + TEXT("[VoxelForgeFirstVisible] tile=(%d,%d,%d) level=0 state=not_seen " + "applied_frame=%llu waited_ms=%.3f visible=%d"), + It.Key().Coord.X, It.Key().Coord.Y, It.Key().Coord.Z, + static_cast(It.Value().AppliedFrame), + FPlatformTime::ToSeconds64(NowCycles - It.Value().AppliedCycles) * 1000.0, + Component->IsVisible() ? 1 : 0); + It.RemoveCurrent(); + } + } +} + void AVoxelWorld::RecordLOD0ReadySample( uint64 RequestStartCycles, uint64 GenerationStartCycles, uint64 GenerationEndCycles, uint64 ApplyStartCycles, uint64 ReadyCycles, uint64 CollisionSubmittedCycles) @@ -7232,6 +7648,8 @@ void AVoxelWorld::HandleTileCollisionCookComplete( const bool bBodyUpdatedEventSeen = Pending->bBodyUpdatedEventSeen; const uint64 BodyUpdatedEventCycles = Pending->BodyUpdatedEventCycles; const uint64 BodyUpdatedEventFrame = Pending->BodyUpdatedEventFrame; + const bool bPhysicsStateReadyAtBodyUpdate = Pending->bPhysicsStateReadyAtBodyUpdate; + const uint64 PhysicsStateReadyCycles = Pending->PhysicsStateReadyCycles; const uint64 MeasuredCollisionSubmittedFrame = Pending->CollisionSubmittedFrame; const uint64 MeasuredRequestStartCycles = Pending->RequestStartCycles; const uint64 MeasuredGenerationStartCycles = Pending->GenerationStartCycles; @@ -7325,6 +7743,9 @@ void AVoxelWorld::HandleTileCollisionCookComplete( const double BodyEventToFutureSeconds = bBodyUpdatedEventSeen && ReadyCycles >= BodyUpdatedEventCycles ? FPlatformTime::ToSeconds64(ReadyCycles - BodyUpdatedEventCycles) : -1.0; + const double BodyEventToPhysicsReadySeconds = bPhysicsStateReadyAtBodyUpdate + && PhysicsStateReadyCycles >= BodyUpdatedEventCycles + ? FPlatformTime::ToSeconds64(PhysicsStateReadyCycles - BodyUpdatedEventCycles) : -1.0; const auto SecondsBetween = [](uint64 EndCycles, uint64 StartCycles) -> double { return StartCycles != 0 && EndCycles >= StartCycles @@ -7371,6 +7792,7 @@ void AVoxelWorld::HandleTileCollisionCookComplete( "merge=%d request_to_worker_s=%.6f worker_gen_s=%.6f worker_to_apply_s=%.6f " "apply_to_mesh_submit_s=%.6f mesh_to_collision_submit_s=%.6f " "rmc_submit_to_body_update_s=%.6f submit_to_body_frames=%lld " + "physics_ready_at_body=%d body_update_to_physics_ready_s=%.6f " "body_update_to_future_callback_s=%.6f future_callback_to_gt_s=%.6f " "suppressed_to_ready_s=%.6f mesh_submit_to_ready=%.6f " "collision_request_to_ready=%.6f body_event_to_future=%.6f"), @@ -7386,6 +7808,7 @@ void AVoxelWorld::HandleTileCollisionCookComplete( RequestToWorkerStartSeconds, WorkerGenerationSeconds, WorkerResultToApplySeconds, ApplyToMeshSubmitSeconds, MeshSubmitToCollisionSubmitSeconds, RMCSubmitToBodyUpdateSeconds, FramesToBodyUpdate, + bPhysicsStateReadyAtBodyUpdate ? 1 : 0, BodyEventToPhysicsReadySeconds, BodyUpdateToFutureCallbackSeconds, FutureCallbackToGameThreadSeconds, SuppressedToReadySeconds, MeshToCookSeconds, CollisionRequestToCookSeconds, BodyEventToFutureSeconds); @@ -7459,9 +7882,11 @@ void AVoxelWorld::ProcessPendingChunks() // Budget limits how many VISIBLE mesh applies we do per frame (GPU upload cost). // Empty meshes and stale results are free to drain — don't count them. const bool bProfileApply = GVoxelForgeProfileTileGeneration != 0; - const uint64 DrainStartCycles = FPlatformTime::Cycles64(); - const int32 MaxApplies = FMath::Max( + uint64 DrainStartCycles = FPlatformTime::Cycles64(); + const int32 ConfiguredMaxApplies = FMath::Max( Settings ? Settings->MaxMeshAppliesPerFrame : 4, 1); + const int32 MaxApplies = GVoxelForgeStep1MaxMeshApplies > 0 + ? GVoxelForgeStep1MaxMeshApplies : ConfiguredMaxApplies; const float ApplyBudgetMilliseconds = Settings ? FMath::Max(Settings->MaxMeshApplyMilliseconds, 0.0f) : 2.0f; @@ -7473,6 +7898,18 @@ void AVoxelWorld::ProcessPendingChunks() FChunkResult DequeuedChunk; const auto DequeueNextResult = [&]() -> bool { + if (bStep1UploadCorpusReleased + && Step1HeldApplyCursor < Step1HeldApplyResults.Num()) + { + DequeuedChunk = MoveTemp(Step1HeldApplyResults[Step1HeldApplyCursor++]); + Step1HeldApplyTiles.Remove(DequeuedChunk.Tile); + return true; + } + if (bStep1UploadCorpusReleased && Step1HeldApplyResults.Num() > 0) + { + Step1HeldApplyResults.Empty(); + Step1HeldApplyCursor = 0; + } // Mirror the admission order. A horizon result that completed earlier must not consume // the game-thread apply budget ahead of collision support, an edit, or ordinary LOD0. return CriticalProcessQueue.Dequeue(DequeuedChunk) @@ -7480,6 +7917,71 @@ void AVoxelWorld::ProcessPendingChunks() || NearProcessQueue.Dequeue(DequeuedChunk) || ProcessQueue.Dequeue(DequeuedChunk); }; + + if (GVoxelForgeStep1PrequeueVisibleLOD0 > 0 && !bStep1UploadCorpusReleased) + { + while (DequeueNextResult()) + { + const TSharedPtr Cancellation = + PendingTileCancellation.FindRef(DequeuedChunk.Tile); + const bool bRequestObsolete = DequeuedChunk.bObsolete + || (Cancellation.IsValid() + && Cancellation->bObsolete.load(std::memory_order_acquire)); + if (DequeuedChunk.bAborted || bRequestObsolete + || DequeuedChunk.Epoch != GenerationEpoch || !IsDesired(DequeuedChunk.Tile)) + { + PendingTileCancellation.Remove(DequeuedChunk.Tile); + PendingTiles.Remove(DequeuedChunk.Tile); + if (bRequestObsolete) + { + ++ObsoleteTileAbortCount; + } + continue; + } + + if (!Step1HeldApplyTiles.Contains(DequeuedChunk.Tile)) + { + Step1HeldApplyTiles.Add(DequeuedChunk.Tile); + if (DequeuedChunk.Tile.Level == 0 && !DequeuedChunk.bEmpty + && DequeuedChunk.Streams.IsValid() && DequeuedChunk.NumTriangles > 0) + { + ++Step1HeldVisibleLOD0Count; + } + Step1HeldApplyResults.Add(MoveTemp(DequeuedChunk)); + if (Step1HeldVisibleLOD0Count >= GVoxelForgeStep1PrequeueVisibleLOD0) + { + break; + } + } + } + + if (Step1HeldVisibleLOD0Count >= GVoxelForgeStep1PrequeueVisibleLOD0) + { + bStep1UploadCorpusReleased = true; + UE_LOG(LogTemp, Display, + TEXT("[VoxelForgeUploadPrequeue] state=released target_visible_l0=%d " + "held_visible_l0=%d held_results=%d active_generation=%d pending=%d frame=%llu"), + GVoxelForgeStep1PrequeueVisibleLOD0, Step1HeldVisibleLOD0Count, + Step1HeldApplyResults.Num(), GetActiveGenerationTaskCount(), PendingTiles.Num(), + static_cast(GFrameCounter)); + DrainStartCycles = FPlatformTime::Cycles64(); + } + else + { + if (Step1LastLoggedPrequeueCount < 0 + || Step1HeldVisibleLOD0Count >= Step1LastLoggedPrequeueCount + 50) + { + UE_LOG(LogTemp, Display, + TEXT("[VoxelForgeUploadPrequeue] state=collecting target_visible_l0=%d " + "held_visible_l0=%d held_results=%d active_generation=%d pending=%d"), + GVoxelForgeStep1PrequeueVisibleLOD0, Step1HeldVisibleLOD0Count, + Step1HeldApplyResults.Num(), GetActiveGenerationTaskCount(), PendingTiles.Num()); + Step1LastLoggedPrequeueCount = Step1HeldVisibleLOD0Count; + } + return; + } + } + while (DequeueNextResult()) { ++ResultsDrained; @@ -7769,6 +8271,23 @@ bool AVoxelWorld::ApplyTileResult(FChunkResult& Result) LogTileApplied(); return false; } + if (GVoxelForgeStep1PrequeueVisibleLOD0 > 0 && Result.Tile.Level == 0) + { + if (URealtimeMeshComponent* Component = TileComponents.FindRef(Result.Tile)) + { + FStep1VisibleApplyProbe& Probe = Step1PendingVisibleApplyProbes.Add(Result.Tile); + Probe.AppliedCycles = FPlatformTime::Cycles64(); + Probe.AppliedFrame = GFrameCounter; + Probe.LastRenderTimeBeforeApply = Component->GetLastRenderTime(); + UE_LOG(LogTemp, Display, + TEXT("[VoxelForgeUploadApplied] tile=(%d,%d,%d) triangles=%d frame=%llu " + "apply_begin_to_submit_ms=%.3f"), + Result.Tile.Coord.X, Result.Tile.Coord.Y, Result.Tile.Coord.Z, + Result.NumTriangles, + static_cast(Probe.AppliedFrame), + FPlatformTime::ToSeconds64(Probe.AppliedCycles - ApplyStartCycles) * 1000.0); + } + } LoadedTiles.Add(Result.Tile); AppliedModificationEpoch.Add(Result.Tile, Result.ModificationEpoch); LoadedTileGeometry.Add(Result.Tile, FLoadedTileGeometryStats{ @@ -8589,7 +9108,21 @@ int32 AVoxelWorld::GetMaxConcurrentTasks() const const int32 CoreCap = AuthoredCoreCap > 0 ? FMath::Clamp(AuthoredCoreCap, 1, LogicalCores) : AutomaticCap; - return FMath::Clamp(Asset, 1, CoreCap); + const int32 NormalCap = FMath::Clamp(Asset, 1, CoreCap); + if (GVoxelForgeStep1GenerationCap == 0) + { + return 0; + } + return GVoxelForgeStep1GenerationCap > 0 + ? FMath::Min(NormalCap, GVoxelForgeStep1GenerationCap) + : NormalCap; +} + +int32 AVoxelWorld::GetActiveGenerationTaskCount() const +{ + const int32 HeldCompletedResults = GVoxelForgeStep1PrequeueVisibleLOD0 > 0 + ? Step1HeldApplyTiles.Num() : 0; + return FMath::Max(0, PendingTiles.Num() - HeldCompletedResults); } int32 AVoxelWorld::GetNearTaskReserve(int32 MaxTasks) const @@ -8618,6 +9151,7 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw UE_LOG(LogTemp, Display, TEXT("[VoxelForgeStreamingBudget] logical_cores=%d automatic_core_cap=%d " "authored_core_cap=%d asset_task_cap=%d effective_generation_cap=%d " + "step1_generation_cap=%d step1_max_mesh_applies=%d " "near_task_reserve=%d coarse_task_cap=%d " "max_mesh_applies=%d max_apply_ms=%.3f max_unloads=%d"), LogicalCores, @@ -8625,6 +9159,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw AuthoredCoreCap, AssetCap, MaxTasks, + GVoxelForgeStep1GenerationCap, + GVoxelForgeStep1MaxMeshApplies, NearTaskReserve, CoarseTaskCap, Settings ? Settings->MaxMeshAppliesPerFrame : 4, @@ -8920,7 +9456,7 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw // They run at BackgroundHigh so the pawn's floor gets the first available worker. for (const FVoxelTileKey& T : CriticalDesiredTiles) { - if (PendingTiles.Num() >= MaxTasks) break; + if (GetActiveGenerationTaskCount() >= MaxTasks) break; if (PendingTiles.Contains(T) || LoadedTiles.Contains(T)) continue; LoadTile(T, EVoxelTileWorkPriority::CollisionCritical); ++Submitted; @@ -8930,7 +9466,7 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw // still in flight is KEPT queued (retried next frame) so its stale pre-carve result is corrected. for (auto It = DirtyRemeshQueue.CreateIterator(); It; ++It) { - if (PendingTiles.Num() >= MaxTasks) break; + if (GetActiveGenerationTaskCount() >= MaxTasks) break; const FVoxelTileKey T = *It; if (!IsDesired(T)) { It.RemoveCurrent(); continue; } // obsolete — drop if (PendingTiles.Contains(T)) { continue; } // in flight — retry after it lands @@ -8945,7 +9481,7 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw for (const FVoxelTileKey& T : DesiredSorted) { if (T.Level != 0) continue; - if (PendingTiles.Num() >= MaxTasks) break; + if (GetActiveGenerationTaskCount() >= MaxTasks) break; if (PendingTiles.Contains(T)) continue; // in flight if (LoadedTiles.Contains(T)) continue; // already loaded (level is in the key — no LOD remesh) LoadTile(T, EVoxelTileWorkPriority::LOD0); @@ -8967,13 +9503,13 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw for (const FVoxelTileKey& T : DesiredSorted) { if (T.Level != Level) continue; - if (PendingTiles.Num() >= CoarseTaskCap) break; + if (GetActiveGenerationTaskCount() >= CoarseTaskCap) break; if (PendingTiles.Contains(T)) continue; // in flight if (LoadedTiles.Contains(T)) continue; // already loaded LoadTile(T); ++Submitted; } - if (PendingTiles.Num() >= CoarseTaskCap) break; + if (GetActiveGenerationTaskCount() >= CoarseTaskCap) break; } if (bMeasureReadyTransitionVisibility) { @@ -8987,7 +9523,7 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw // mesh reste visible jusqu'au résultat — même schéma que RemeshDirtyChunks). for (auto It = BandRemeshQueue.CreateIterator(); It; ++It) { - if (PendingTiles.Num() >= CoarseTaskCap) break; + if (GetActiveGenerationTaskCount() >= CoarseTaskCap) break; const FVoxelTileKey T = *It; It.RemoveCurrent(); if (!IsDesired(T) || PendingTiles.Contains(T) || !LoadedTiles.Contains(T)) continue; @@ -9004,7 +9540,7 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw GVoxelForgeMaxPendingTilesObserved, MaxTasks, Submitted); } if (GVoxelForgeProfileTileGeneration != 0 - && PendingTiles.Num() >= CoarseTaskCap + && GetActiveGenerationTaskCount() >= CoarseTaskCap && !GVoxelForgeGenerationCapHitReported) { UE_LOG(LogTemp, Display, @@ -9032,6 +9568,7 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw void AVoxelWorld::LoadTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority WorkPriority) { + if (GVoxelForgeStep1GenerationCap == 0) return; if (PendingTiles.Contains(Tile)) return; const int32 MaxTasks = GetMaxConcurrentTasks(); // T2.d — core-clamped @@ -9039,7 +9576,7 @@ void AVoxelWorld::LoadTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority Wor const int32 CoarseTaskCap = FMath::Max(1, MaxTasks - GetNearTaskReserve(MaxTasks)); const int32 AdmissionCap = WorkPriority == EVoxelTileWorkPriority::Horizon ? CoarseTaskCap : MaxTasks; - if (PendingTiles.Num() >= AdmissionCap) + if (GetActiveGenerationTaskCount() >= AdmissionCap) { if (GVoxelForgeProfileTileGeneration != 0 && !GVoxelForgeGenerationCapHitReported) @@ -9695,6 +10232,26 @@ void AVoxelWorld::GenerateTileResult(const FVoxelTileKey& Tile, const FIntVector Result.NumSkirtTriangles = MeshData.NumSkirtTriangles; Result.bHasCeilingTris = MeshData.NumCeilingTriangles > 0; Result.bHasGroundTris = NumTris > MeshData.NumCeilingTriangles; + + if (GVoxelForgeStep1CoreCookBenchmark != 0 + && Result.Tile.Level == 0 + && Result.WorkPriority == EVoxelTileWorkPriority::CollisionCritical) + { + if (FQueuedThreadPool* CookPool = GVoxelForgeStep1CoreCookPool.GetOrCreate()) + { + CookPool->AddQueuedWork( + new FVoxelStep1CoreCookWork( + MakeShared(*Result.Streams), + Result.Tile, Result.NumTriangles, + FPlatformTime::Cycles64()), + EQueuedWorkPriority::High); + } + else + { + UE_LOG(LogTemp, Error, + TEXT("[VoxelForgeStep1CoreCook] dedicated_pool_create_failed=1")); + } + } } // Parity instrumentation observes the same FVoxelMeshData that the runtime stream builder @@ -9850,7 +10407,8 @@ bool AVoxelWorld::ApplyMeshToTile(FChunkResult& Result) const bool bHasGroundTris = Result.bHasGroundTris; const bool bHasCeilingTris = Result.bHasCeilingTris; const bool bLevel0 = (Tile.Level == 0); - const bool bWantsCollision = bLevel0 && IsTileCollisionDesired(Tile); + const bool bWantsCollision = bLevel0 && IsTileCollisionDesired(Tile) + && GVoxelForgeStep1UploadDisableCollision == 0; const bool bCollisionSuppressed = CollisionSuppressedUntilCook.Contains(Tile); // RMC keeps its installed BodySetup active while the replacement collision cooks. Preserve diff --git a/Source/VoxelForge/Public/VoxelWorld.h b/Source/VoxelForge/Public/VoxelWorld.h index 2d5d09e..3b00994 100644 --- a/Source/VoxelForge/Public/VoxelWorld.h +++ b/Source/VoxelForge/Public/VoxelWorld.h @@ -294,6 +294,8 @@ public: bool bBodyUpdatedEventSeen = false; uint64 BodyUpdatedEventCycles = 0; uint64 BodyUpdatedEventFrame = 0; + bool bPhysicsStateReadyAtBodyUpdate = false; + uint64 PhysicsStateReadyCycles = 0; uint64 FutureCallbackCycles = 0; uint64 CompletionGameThreadCycles = 0; // Game-thread-only timing copied from the generation result for the clean LOD0 summary. @@ -320,6 +322,29 @@ public: uint64 NextCollisionSubmissionId = 0; TMap CollisionBodyUpdatedHandles; + struct FStep1SupportSweepProbe + { + uint64 CollisionRequestCycles = 0; + uint64 BodyUpdatedCycles = 0; + uint64 PhysicsStateReadyCycles = 0; + bool bFirstSweepAttemptLogged = false; + }; + TMap Step1SupportSweepProbes; + + struct FStep1VisibleApplyProbe + { + uint64 AppliedCycles = 0; + uint64 AppliedFrame = 0; + float LastRenderTimeBeforeApply = 0.0f; + }; + TMap Step1PendingVisibleApplyProbes; + TArray Step1HeldApplyResults; + TSet Step1HeldApplyTiles; + int32 Step1HeldVisibleLOD0Count = 0; + int32 Step1LastLoggedPrequeueCount = -1; + int32 Step1HeldApplyCursor = 0; + bool bStep1UploadCorpusReleased = false; + /** Render component per NON-empty loaded tile (GC-safe via actor ownership; not UPROPERTY * because FVoxelTileKey isn't a USTRUCT key). */ TMap TileComponents; @@ -347,6 +372,7 @@ public: * capped to (logical cores − 2) so small CPUs don't thrash on a flat 16 (background * priority stops frame starvation, not the context-switch overhead). */ int32 GetMaxConcurrentTasks() const; + int32 GetActiveGenerationTaskCount() const; /** Reserve task slots for collision-critical/edit/LOD0 work; far work uses the remainder. */ int32 GetNearTaskReserve(int32 MaxTasks) const; @@ -920,6 +946,8 @@ public: void BindRealtimeMeshCollisionEvent(URealtimeMesh* Mesh); void UnbindRealtimeMeshCollisionEvent(URealtimeMesh* Mesh); void HandleRealtimeMeshCollisionBodyUpdated(URealtimeMesh* Mesh, UBodySetup* BodySetup); + void PollStep1SupportSweepProbes(); + void PollStep1VisibleApplyProbes(); void RecordLOD0ReadySample(uint64 RequestStartCycles, uint64 GenerationStartCycles, uint64 GenerationEndCycles, uint64 ApplyStartCycles, uint64 ReadyCycles, uint64 CollisionSubmittedCycles); diff --git a/Tools/VoxelForgeTest.ps1 b/Tools/VoxelForgeTest.ps1 index bf50d0a..667283d 100644 --- a/Tools/VoxelForgeTest.ps1 +++ b/Tools/VoxelForgeTest.ps1 @@ -3,6 +3,7 @@ param( [ValidateSet('canonical', 'owner', 'probe', 'perf', 'visual-edit', 'visual-lod', 'visual-gap-walk', 'visual-gap-inject', 'dig-motion', 'tests', 'surface-fall', 'crossing', 'gate-stress', 'parity', 'horizon', 'horizon-render', 'determinism', 'symbolize-stacks')] [string]$Scenario = 'canonical', [switch]$Build, + [switch]$EnableRender, [ValidateSet('owner', 'default')] [string]$Assets = 'owner', [hashtable]$Cvars = @{}, @@ -465,7 +466,7 @@ function Invoke-BuildProcess([string[]]$Arguments, [string]$LogPath, [string]$Er try { Stop-Process -Id ([int]$process.Id) -Force -ErrorAction Stop } catch { } throw "Could not lower owned build PID $($process.Id) to BelowNormal: $($_.Exception.Message)" } - $wait = Wait-OwnedProcess $process $LaunchTimeoutSeconds + $wait = Wait-OwnedProcess $process $LaunchTimeoutSeconds -EnforceMemoryLimit } finally { if ($null -eq $oldDotnetCliHome) { Remove-Item Env:DOTNET_CLI_HOME -ErrorAction SilentlyContinue } else { $env:DOTNET_CLI_HOME = $oldDotnetCliHome } if ($null -eq $oldDotnetFirstUse) { Remove-Item Env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE -ErrorAction SilentlyContinue } else { $env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE = $oldDotnetFirstUse } @@ -479,12 +480,16 @@ function Invoke-BuildProcess([string[]]$Arguments, [string]$LogPath, [string]$Er ended_utc = [datetime]::UtcNow.ToString('o') exit_code = $wait.exit_code timed_out = [bool]$wait.timed_out + memory_limit_hit = [bool]$wait.memory_limit_hit + peak_private_bytes = [int64]$wait.peak_private_bytes log = $LogPath stdout = $StdoutPath error_log = $ErrorPath arguments = @($Arguments) isolated_local_app_data = $isolatedLocalAppData - status = if ($wait.timed_out) { 'timeout' } elseif ($wait.exit_code -eq 0) { 'passed' } else { 'failed' } + status = if ($wait.timed_out) { 'timeout' } + elseif ($wait.memory_limit_hit) { 'memory_limit' } + elseif ($wait.exit_code -eq 0) { 'passed' } else { 'failed' } } } @@ -1490,7 +1495,7 @@ try { $baseCvars['voxel.UseBlockEarlyOut'] = 0 $baseCvars['voxel.OuterClassifierMode'] = 0 $baseCvars['voxel.SealedSolidProof'] = 0 - $args = New-CommonArguments 'game' (Join-Path $RunRoot 'Logs\perf.log') (Merge-Cvars $baseCvars) + $args = New-CommonArguments 'game' (Join-Path $RunRoot 'Logs\perf.log') (Merge-Cvars $baseCvars) $EnableRender.IsPresent $gameLedger = Invoke-UnrealLaunch 'perf-game' $args ` (Join-Path $RunRoot 'Logs\perf.log') (Join-Path $RunRoot 'Logs\perf.stdout.log') (Join-Path $RunRoot 'Logs\perf.stderr.log') $GameMetrics = Get-StreamingMetrics (Join-Path $RunRoot 'Logs\perf.log')