From b9c13c37c0bff46fe29a964a1d20dda01fe1904b Mon Sep 17 00:00:00 2001 From: Fr0zka Date: Mon, 19 Oct 2026 06:46:54 +0200 Subject: [PATCH] wip(assembler): round S6 -- in-game cold view counted; workers mostly idle, apply cap not the floor Direct per-tile trace on the BuildHost actor (owner plan, 2,578 keys): full set 17.4 s legacy; gated coarse-first (after the 9-tile walkable ring) 11.7 s with the ring still at 0.63 s, but full LOD0 slips 2.3 -> 10.2 s. Startup apply boost changed nothing (0.4%). 37.6 worker-s over 17.4 s on 10 slots = ~3.3 busy: the floor is admission/work supply. S5's 251 ms staircase was the trace sampler. Fixed CVar parsing in headless BeginPlay and the 60 m/s crossing instrument. All three knobs stay opt-in. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01MmSoF64BbRFmk9C3p1VsBJ --- ARCHITECTURE.md | 37 +++ CODEMAP.md | 14 + .../VoxelForge/Private/VoxelStartupTrace.cpp | 62 ++++- Source/VoxelForge/Private/VoxelWorld.cpp | 234 +++++++++++++++-- Source/VoxelForge/Public/VoxelStartupTrace.h | 30 +++ Source/VoxelForge/Public/VoxelWorld.h | 12 + .../Private/VoxelForgeExploreCommandlet.cpp | 245 ++++++++++++++++++ .../gpu_field_port/LUNA_DIALOGUE.md | 18 ++ .../WorldAssemblerM3/Round10/REPORT.md | 218 ++++++++++++++++ Standalone/Source/StandaloneViewer.cpp | 24 +- Tools/TestVoxelForgeCoreWorldHost.ps1 | 40 ++- 11 files changed, 903 insertions(+), 31 deletions(-) create mode 100644 Standalone/Research/WorldAssemblerM3/Round10/REPORT.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index fed997d..512e43e 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1476,6 +1476,43 @@ rather than proof of screen presentation. Keep S5 opt-in until an owner-visible three gates and closes that gap. Full results and limits are in `Standalone/Research/WorldAssemblerM3/Round9/REPORT.md`. +### 8.27 Cold-view throughput trace and gated coarse admission — World Assembler S6 / M3 Round 10 + +`UpdateChunksAroundPosition` samples residency at intervals of at least 0.25 seconds. A `streaming_sample` +timestamp is therefore a quantized observation, not a tile-completion timestamp or an admission throttle. +Use the direct per-tile apply records for completion time, and `apply_frame` records for the game-thread +budget, queued-result estimate and running-worker count. `collision_ready` measures the asynchronous +RealtimeMesh submit-to-body-update interval; it is not an isolated Chaos CPU-cook timer. + +The staged SurfaceWorld actor trace contained 2,578 desired keys. With legacy order and the authored +4-visible-mesh / 2 ms apply budget, the full desired set became resident at 17.412 s and the required +collision ring became walkable at 0.632 s. `voxel.StreamingCoarseFirst=1` with +`voxel.StreamingDynamicNearReserve=1` completed the same set at 11.704 s while the ring gate remained +0.627 s. Coarse admissions begin only after the nine-tile walkable gate. This reduces measured cold +residency by about one third on the staged plan, but null-RHI residency does not prove rendered +enclosure. Keep coarse-first opt-in until that view is checked in a rendered owner-plan run. + +The separate `voxel.StreamingStartupApplyBoost` switch stays default-off. At `1`, it temporarily allows +12 visible mesh submissions / 14 ms while the initial desired set and walkable ring are incomplete, +with a 30 second ceiling; at `0`, the authored budget is unchanged. On the matched legacy-order test, +the boost changed full-set readiness from 17.412 s to 17.346 s and removed count-cap frames, but did +not improve end-to-end residency. Keep the normal 4 / 2 ms guard for play. + +The trace found the shared worker pool was not saturated: the 10-slot idle estimate was about 132 +slot-seconds on legacy order and 73 on gated coarse-first, while game-thread tile applies totaled only +about 0.24 seconds during cold residency. Results waited in the result queue, and some frames reached +the four-mesh cap, but expanding the apply budget alone did not move the wall clock. The remaining +floor is worker-side work supply/admission; this trace does not isolate why available worker slots +stay idle. The 1.57 s target remains unmet; the headless trace is not a render-time measurement. + +The report records per-LOD worker, queue, result-queue and apply totals, collision latency, frame +percentiles, hashes, regressions and the command lines. The standalone 60 m/s viewer now counts +forward +Y center crossings across multi-chunk deltas; that route reports valid crossing deltas, but +its latency gate still fails. These are scheduling and instrumentation changes only: deterministic +W1/W8/W16 world and LOD hashes are unchanged, as are `Epoch`, cancellation, `ModsLock`, MPSC queue, +shutdown-drain and game-thread UObject-creation rules. See +[`Standalone/Research/WorldAssemblerM3/Round10/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round10/REPORT.md). + ## 9. Multiplayer model (listen-server first, dedicated-friendly) > **Status: transport is DESIGN ONLY — nothing is networked in-tree yet** (no `Replicated`/`HasAuthority`/RPCs; a diff --git a/CODEMAP.md b/CODEMAP.md index 5bfadb9..55346fc 100644 --- a/CODEMAP.md +++ b/CODEMAP.md @@ -996,6 +996,20 @@ exact-density-root LOD7/8 agreement audit with its datum-offset negative control ARCHITECTURE §8.25 and [`Standalone/Research/WorldAssemblerM3/Round8/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round8/REPORT.md). +### 3.14 Cold streaming apply trace and gated coarse order — World Assembler S6 / M3 Round 10 + +`AVoxelWorld::ProcessPendingChunks` records per-frame result draining, visible-mesh applies, the +authored/startup budgets, per-LOD apply counts and worker occupancy through +`VoxelStartupTrace::RecordApplyFrame`. Direct `tile` events separate request-to-worker queue, +generation work, worker-result queue and game-thread apply time; `collision_ready` records the +RealtimeMesh body-update latency. `AVoxelWorld::UpdateChunksAroundPosition` samples resident counts +every 250 ms (for observation only) and admits coarse-first work only after the required nine-tile +walkable ring is ready. The order and startup apply boost are opt-in CVars; the boost is default-off. +`VoxelForgeExploreCommandlet.cpp` drives the staged owner-plan trace and moving actor check. The +standalone viewer's 60 m/s crossing counter now accepts multi-chunk forward deltas. See +ARCHITECTURE §8.27 and +[`Standalone/Research/WorldAssemblerM3/Round10/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round10/REPORT.md). + ## 4. The density pipeline (most-edited hot path) ### 4.1 `GetDensityWithParams` (TunnelNetwork) — VoxelGenerator.cpp:3749 diff --git a/Source/VoxelForge/Private/VoxelStartupTrace.cpp b/Source/VoxelForge/Private/VoxelStartupTrace.cpp index 6607a0c..1f7bb1b 100644 --- a/Source/VoxelForge/Private/VoxelStartupTrace.cpp +++ b/Source/VoxelForge/Private/VoxelStartupTrace.cpp @@ -20,6 +20,7 @@ namespace int32 TileRecordsSeen = 0; int32 TileRecordsWritten = 0; int32 TileRecordsSuppressed = 0; + int32 ApplyFrameRecords = 0; TArray Samples; TMap StageStarts; TMap StageTotals; @@ -94,7 +95,7 @@ namespace FString BuildTileGroupJson(const TArray& Samples, int32 Level) { - TArray Request, Queue, Generation, Classify, Mesh, Apply; + TArray Request, Queue, WorkerTask, Generation, Classify, Mesh, Apply; int32 Count = 0; int32 Empty = 0; int32 AllAir = 0; @@ -117,6 +118,7 @@ namespace GeometryBytes += Sample.GeometryBytes; Request.Add(Sample.RequestToApplySeconds); Queue.Add(Sample.QueueWaitSeconds); + WorkerTask.Add(Sample.WorkerTaskSeconds); Generation.Add(Sample.GenerationSeconds); Classify.Add(Sample.ClassifySeconds); Mesh.Add(Sample.MeshSeconds); @@ -127,6 +129,7 @@ namespace TEXT("{\"count\":%d,\"empty\":%d,\"all_air\":%d,\"all_solid\":%d,\"mixed\":%d,\"vertices\":%lld,\"triangles\":%lld,\"geometry_bytes\":%llu," "\"request_apply_s\":{\"p50\":%s,\"p95\":%s,\"max\":%s}," "\"queue_s\":{\"p50\":%s,\"p95\":%s,\"max\":%s}," + "\"worker_task_s\":{\"p50\":%s,\"p95\":%s,\"max\":%s}," "\"generation_s\":{\"p50\":%s,\"p95\":%s,\"max\":%s}," "\"classify_s\":{\"p50\":%s,\"p95\":%s,\"max\":%s}," "\"mesh_s\":{\"p50\":%s,\"p95\":%s,\"max\":%s}," @@ -136,6 +139,7 @@ namespace static_cast(GeometryBytes), *Number(Percentile(Request, 0.50)), *Number(Percentile(Request, 0.95)), *Number(Maximum(Request)), *Number(Percentile(Queue, 0.50)), *Number(Percentile(Queue, 0.95)), *Number(Maximum(Queue)), + *Number(Percentile(WorkerTask, 0.50)), *Number(Percentile(WorkerTask, 0.95)), *Number(Maximum(WorkerTask)), *Number(Percentile(Generation, 0.50)), *Number(Percentile(Generation, 0.95)), *Number(Maximum(Generation)), *Number(Percentile(Classify, 0.50)), *Number(Percentile(Classify, 0.95)), *Number(Maximum(Classify)), *Number(Percentile(Mesh, 0.50)), *Number(Percentile(Mesh, 0.95)), *Number(Maximum(Mesh)), @@ -173,11 +177,12 @@ namespace return FString::Printf( TEXT("\"reason\":\"%s\",\"wall_s\":%s,\"tile_records_seen\":%d," "\"tile_records_written\":%d,\"tile_records_suppressed\":%d," - "\"tile_record_cap\":%d,\"stage_seconds\":%s,\"tile_percentiles\":{%s}"), + "\"tile_record_cap\":%d,\"apply_frame_records\":%d," + "\"stage_seconds\":%s,\"tile_percentiles\":{%s}"), *JsonEscape(Reason ? FString(Reason) : FString(TEXT("unknown"))), *Number(FPlatformTime::Seconds() - State.StartSeconds), State.TileRecordsSeen, State.TileRecordsWritten, State.TileRecordsSuppressed, - State.TileRecordCap, *Stages, *Groups); + State.TileRecordCap, State.ApplyFrameRecords, *Stages, *Groups); } } @@ -283,7 +288,7 @@ void RecordTile(const FTileSample& Sample) TEXT("\"tile\":[%d,%d,%d],\"level\":%d,\"priority\":%d," "\"request_ordinal\":%llu,\"generation_ordinal\":%llu,\"resident_at_request\":%d," "\"verdict\":%d,\"proof\":%d,\"empty\":%d,\"vertices\":%d,\"triangles\":%d,\"geometry_bytes\":%llu," - "\"request_to_apply_s\":%s,\"queue_wait_s\":%s,\"worker_queue_s\":%s,\"result_queue_s\":%s," + "\"request_to_apply_s\":%s,\"queue_wait_s\":%s,\"worker_queue_s\":%s,\"result_queue_s\":%s,\"worker_task_s\":%s," "\"generation_s\":%s,\"classify_s\":%s,\"mesh_s\":%s,\"stream_s\":%s,\"apply_s\":%s"), Sample.TileX, Sample.TileY, Sample.TileZ, Sample.Level, Sample.WorkPriority, @@ -295,10 +300,59 @@ void RecordTile(const FTileSample& Sample) static_cast(Sample.GeometryBytes), *Number(Sample.RequestToApplySeconds), *Number(Sample.QueueWaitSeconds), *Number(Sample.WorkerQueueSeconds), *Number(Sample.ResultQueueSeconds), + *Number(Sample.WorkerTaskSeconds), *Number(Sample.GenerationSeconds), *Number(Sample.ClassifySeconds), *Number(Sample.MeshSeconds), *Number(Sample.StreamSeconds), *Number(Sample.ApplySeconds))); } +void RecordApplyFrame(const FApplyFrameSample& Sample) +{ + FScopeLock Lock(>raceLock); + if (!GState.IsValid() || !GState->bActive) return; + + FString PendingByLevel = TEXT("["); + FString ApplyResultsByLevel = TEXT("["); + FString MeshesByLevel = TEXT("["); + FString ApplyMillisecondsByLevel = TEXT("["); + for (int32 Level = 0; Level < 9; ++Level) + { + if (Level > 0) + { + PendingByLevel += TEXT(","); + ApplyResultsByLevel += TEXT(","); + MeshesByLevel += TEXT(","); + ApplyMillisecondsByLevel += TEXT(","); + } + PendingByLevel += FString::FromInt(Sample.PendingByLevel[Level]); + ApplyResultsByLevel += FString::FromInt(Sample.ApplyResultsByLevel[Level]); + MeshesByLevel += FString::FromInt(Sample.MeshesAppliedByLevel[Level]); + ApplyMillisecondsByLevel += Number(Sample.ApplySecondsByLevel[Level] * 1000.0); + } + PendingByLevel += TEXT("]"); + ApplyResultsByLevel += TEXT("]"); + MeshesByLevel += TEXT("]"); + ApplyMillisecondsByLevel += TEXT("]"); + + ++GState->ApplyFrameRecords; + WriteEventLocked(*GState, TEXT("apply_frame"), FString::Printf( + TEXT("\"frame\":%llu,\"frame_ms\":%s,\"speed_cm_s\":%s,\"apply_ms\":%s,\"drain_ms\":%s," + "\"active_tasks\":%d,\"running_workers\":%d,\"task_cap\":%d," + "\"pending_tiles\":%d,\"pending_results_estimate\":%d," + "\"results_drained\":%d,\"apply_results\":%d,\"meshes_applied\":%d,\"max_applies\":%d," + "\"apply_budget_ms\":%s,\"count_limit_reached\":%d,\"time_limit_reached\":%d," + "\"pending_by_lod\":%s,\"apply_results_by_lod\":%s," + "\"meshes_by_lod\":%s,\"apply_ms_by_lod\":%s"), + static_cast(Sample.Frame), + *Number(Sample.FrameSeconds * 1000.0), *Number(Sample.SpeedCmPerSecond), + *Number(Sample.ApplySeconds * 1000.0), *Number(Sample.DrainSeconds * 1000.0), Sample.ActiveTasks, + Sample.RunningWorkerTasks, Sample.TaskCap, Sample.PendingTiles, + Sample.PendingResultsEstimate, Sample.ResultsDrained, Sample.ApplyResultsByLevelTotal, + Sample.MeshesApplied, + Sample.MaxApplies, *Number(Sample.ApplyBudgetMilliseconds), + Sample.bCountLimitReached ? 1 : 0, Sample.bTimeLimitReached ? 1 : 0, + *PendingByLevel, *ApplyResultsByLevel, *MeshesByLevel, *ApplyMillisecondsByLevel)); +} + void Finish(const TCHAR* Reason) { FScopeLock Lock(>raceLock); diff --git a/Source/VoxelForge/Private/VoxelWorld.cpp b/Source/VoxelForge/Private/VoxelWorld.cpp index 5010fb9..b8c3b45 100644 --- a/Source/VoxelForge/Private/VoxelWorld.cpp +++ b/Source/VoxelForge/Private/VoxelWorld.cpp @@ -237,6 +237,15 @@ namespace GVoxelForgeStep1MaxMeshApplies, TEXT("Step 1 measurement override: 0 uses the asset MaxMeshAppliesPerFrame, otherwise caps visible applies per frame.")); + int32 GVoxelForgeStreamingStartupApplyBoost = 0; + FAutoConsoleVariableRef CVarVoxelForgeStreamingStartupApplyBoost( + TEXT("voxel.StreamingStartupApplyBoost"), + GVoxelForgeStreamingStartupApplyBoost, + TEXT("Opt-in cold-view upload experiment (default off): 1 allows a 14 ms / 12-mesh budget until the initial desired set and walkable ring are ready; 0 uses the authored per-frame budget.")); + constexpr int32 StreamingStartupMaxMeshApplies = 12; + constexpr float StreamingStartupApplyBudgetMilliseconds = 14.0f; + constexpr double StreamingStartupApplyMaxSeconds = 30.0; + int32 GVoxelForgeStep1PrequeueVisibleLOD0 = 0; FAutoConsoleVariableRef CVarVoxelForgeStep1PrequeueVisibleLOD0( TEXT("voxel.Step1PrequeueVisibleLOD0"), @@ -2041,6 +2050,16 @@ void AVoxelWorld::BeginPlay() { GVoxelForgeStep1MaxMeshApplies = CommandLineStep1MaxMeshApplies; } + FParse::Value(FCommandLine::Get(), TEXT("voxel.StreamingStartupApplyBoost="), + GVoxelForgeStreamingStartupApplyBoost); + // Headless BuildHost does not reliably apply arbitrary console-variable command-line tokens + // before BeginPlay; parse the test switches explicitly so A/B runs exercise the requested path. + // BuildHost sans interface n'applique pas toujours les CVar arbitraires avant BeginPlay ; + // lire explicitement ces options afin que les essais A/B utilisent bien le chemin demande. + FParse::Value(FCommandLine::Get(), TEXT("voxel.StreamingCoarseFirst="), + GVoxelForgeStreamingCoarseFirst); + FParse::Value(FCommandLine::Get(), TEXT("voxel.StreamingDynamicNearReserve="), + GVoxelForgeStreamingDynamicNearReserve); if (FParse::Value(FCommandLine::Get(), TEXT("voxel.Step1PrequeueVisibleLOD0="), CommandLineStep1PrequeueVisibleLOD0)) { @@ -3026,9 +3045,26 @@ void AVoxelWorld::ConfigureHeadlessStreamingTest() if (bMoveRequested && !bHeadlessStrateCrossingTest) { + // FParse::Value treats commas as separators in some commandlet argument builds; read the + // whole vector token so the requested X,Y,Z direction survives intact. + // FParse::Value separe parfois les virgules dans les commandes ; lire le jeton X,Y,Z complet. FString DirectionText; - if (FParse::Value(CommandLine, TEXT("voxel.TestMoveDirection="), DirectionText)) + const FString CommandLineText(CommandLine); + const FString DirectionKey(TEXT("voxel.TestMoveDirection=")); + const int32 DirectionKeyIndex = CommandLineText.Find( + DirectionKey, ESearchCase::IgnoreCase, ESearchDir::FromStart); + if (DirectionKeyIndex != INDEX_NONE) { + DirectionText = CommandLineText.Mid(DirectionKeyIndex + DirectionKey.Len()); + const int32 DirectionEnd = DirectionText.Find( + TEXT(" "), ESearchCase::CaseSensitive, ESearchDir::FromStart); + if (DirectionEnd != INDEX_NONE) + { + DirectionText.LeftInline(DirectionEnd, EAllowShrinking::No); + } + DirectionText.TrimStartAndEndInline(); + DirectionText.RemoveFromStart(TEXT("\"")); + DirectionText.RemoveFromEnd(TEXT("\"")); TArray Components; DirectionText.ParseIntoArray(Components, TEXT(","), true); if (Components.Num() == 3) @@ -6717,6 +6753,7 @@ void AVoxelWorld::Tick(float DeltaTime) AdvanceHeadlessDigMotionTest(PlayerLastPos, PlayerPawn); PeakObservedPawnSpeedCmPerSecond = FMath::Max( PeakObservedPawnSpeedCmPerSecond, PlayerHeading.Size2D()); + StartupTraceSpeedCmPerSecond = PlayerHeading.Size2D(); UpdatePawnCollisionGate(PlayerPawn, PlayerLastPos, DeltaTime); RecordHeadlessDigMotionFrame(PlayerPawn, PlayerLastPos, DeltaTime); ObserveHeadlessStrateCrossingTest(PlayerLastPos, PlayerPawn); @@ -9064,19 +9101,121 @@ void AVoxelWorld::ProcessPendingChunks() // Budget limits how many VISIBLE mesh applies we do per frame (GPU upload cost). // Empty meshes and stale results are free to drain — don't count them. const bool bProfileApply = GVoxelForgeProfileTileGeneration != 0; - uint64 DrainStartCycles = FPlatformTime::Cycles64(); + const uint64 ApplyFrameStartCycles = FPlatformTime::Cycles64(); + const double FrameSeconds = LastApplyFrameCycles != 0 + && ApplyFrameStartCycles >= LastApplyFrameCycles + ? FPlatformTime::ToSeconds64(ApplyFrameStartCycles - LastApplyFrameCycles) + : 0.0; + LastApplyFrameCycles = ApplyFrameStartCycles; + + const double StartupElapsedSeconds = DesiredEpochStartCycles != 0 + && ApplyFrameStartCycles >= DesiredEpochStartCycles + ? FPlatformTime::ToSeconds64(ApplyFrameStartCycles - DesiredEpochStartCycles) + : -1.0; + const bool bInitialDesiredSetReady = DesiredEpoch == 1 + && bAllChunksLoaded + && PendingTiles.Num() == 0 + && PendingCollisionCooks.Num() == 0 + && LastLOD0RingReadyEpoch == DesiredEpoch; + if (bStartupApplyBoostStarted && !bStartupApplyBoostEnded + && (bInitialDesiredSetReady || DesiredEpoch != 1 + || GVoxelForgeStreamingStartupApplyBoost <= 0 + || GVoxelForgeStep1MaxMeshApplies > 0 + || StartupElapsedSeconds >= StreamingStartupApplyMaxSeconds)) + { + bStartupApplyBoostEnded = true; + if (VoxelForgeStartupTrace::IsActive()) + { + VoxelForgeStartupTrace::RecordEvent(TEXT("startup_apply_budget"), FString::Printf( + TEXT("\"state\":\"normal\",\"reason\":\"%s\",\"elapsed_s\":%s"), + bInitialDesiredSetReady ? TEXT("initial_set_and_walkable_ring_ready") + : (StartupElapsedSeconds >= StreamingStartupApplyMaxSeconds + ? TEXT("time_limit") + : (DesiredEpoch != 1 ? TEXT("desired_epoch_changed") + : TEXT("override_disabled"))), + *FString::SanitizeFloat(StartupElapsedSeconds))); + } + } + const bool bUseStartupApplyBoost = GVoxelForgeStreamingStartupApplyBoost > 0 + && GVoxelForgeStep1MaxMeshApplies <= 0 + && !bStartupApplyBoostEnded + && DesiredEpoch == 1 + && StartupElapsedSeconds >= 0.0 + && !bInitialDesiredSetReady + && StartupElapsedSeconds < StreamingStartupApplyMaxSeconds; + if (bUseStartupApplyBoost && !bStartupApplyBoostStarted) + { + bStartupApplyBoostStarted = true; + if (VoxelForgeStartupTrace::IsActive()) + { + VoxelForgeStartupTrace::RecordEvent(TEXT("startup_apply_budget"), FString::Printf( + TEXT("\"state\":\"boosted\",\"max_applies\":%d,\"budget_ms\":%.3f," + "\"max_duration_s\":%.3f,\"desired_epoch\":%u"), + StreamingStartupMaxMeshApplies, + StreamingStartupApplyBudgetMilliseconds, + StreamingStartupApplyMaxSeconds, + DesiredEpoch)); + } + } + + uint64 DrainStartCycles = ApplyFrameStartCycles; const int32 ConfiguredMaxApplies = FMath::Max( Settings ? Settings->MaxMeshAppliesPerFrame : 4, 1); const int32 MaxApplies = GVoxelForgeStep1MaxMeshApplies > 0 - ? GVoxelForgeStep1MaxMeshApplies : ConfiguredMaxApplies; - const float ApplyBudgetMilliseconds = Settings - ? FMath::Max(Settings->MaxMeshApplyMilliseconds, 0.0f) - : 2.0f; + ? GVoxelForgeStep1MaxMeshApplies + : (bUseStartupApplyBoost ? StreamingStartupMaxMeshApplies : ConfiguredMaxApplies); + const float ApplyBudgetMilliseconds = bUseStartupApplyBoost + ? StreamingStartupApplyBudgetMilliseconds + : (Settings ? FMath::Max(Settings->MaxMeshApplyMilliseconds, 0.0f) : 2.0f); const double ApplyBudgetSeconds = static_cast(ApplyBudgetMilliseconds) * 0.001; int32 MeshesApplied = 0; int32 ResultsDrained = 0; + int32 ApplyResultsByLevelTotal = 0; + double ApplySeconds = 0.0; + TStaticArray ApplyResultsByLevel{}; + TStaticArray MeshesAppliedByLevel{}; + TStaticArray ApplySecondsByLevel{}; bool bTimeBudgetHit = false; + const auto RecordApplyFrame = [&]() + { + if (!VoxelForgeStartupTrace::IsActive()) return; + VoxelForgeStartupTrace::FApplyFrameSample Sample; + Sample.Frame = GFrameCounter; + Sample.FrameSeconds = FrameSeconds; + Sample.SpeedCmPerSecond = StartupTraceSpeedCmPerSecond; + Sample.ApplySeconds = ApplySeconds; + Sample.DrainSeconds = FPlatformTime::ToSeconds64( + FPlatformTime::Cycles64() - DrainStartCycles); + // `GetActiveGenerationTaskCount` includes completed tiles until the game-thread apply removes + // their PendingTiles entry. Use the task guard's counter here so the difference estimates + // results waiting in the MPSC queues instead of reporting zero by construction. + // `GetActiveGenerationTaskCount` inclut les resultats termines jusqu'a leur application. + // Utiliser le compteur RAII pour mesurer les resultats en attente dans les files MPSC. + Sample.ActiveTasks = ActiveTaskCount.load(std::memory_order_relaxed); + Sample.RunningWorkerTasks = ActiveGenerationWorkerCount.load(std::memory_order_relaxed); + Sample.TaskCap = GetMaxConcurrentTasks(); + Sample.PendingTiles = PendingTiles.Num(); + Sample.PendingResultsEstimate = FMath::Max(0, Sample.PendingTiles - Sample.ActiveTasks); + Sample.ResultsDrained = ResultsDrained; + Sample.ApplyResultsByLevelTotal = ApplyResultsByLevelTotal; + Sample.MeshesApplied = MeshesApplied; + Sample.MaxApplies = MaxApplies; + Sample.ApplyBudgetMilliseconds = ApplyBudgetMilliseconds; + Sample.bCountLimitReached = MeshesApplied >= MaxApplies; + Sample.bTimeLimitReached = bTimeBudgetHit; + Sample.ApplyResultsByLevel = ApplyResultsByLevel; + Sample.MeshesAppliedByLevel = MeshesAppliedByLevel; + Sample.ApplySecondsByLevel = ApplySecondsByLevel; + for (const FVoxelTileKey& PendingTile : PendingTiles) + { + if (PendingTile.Level >= 0 && PendingTile.Level < 9) + { + ++Sample.PendingByLevel[PendingTile.Level]; + } + } + VoxelForgeStartupTrace::RecordApplyFrame(Sample); + }; FChunkResult DequeuedChunk; const auto DequeueNextResult = [&]() -> bool { @@ -9177,6 +9316,7 @@ void AVoxelWorld::ProcessPendingChunks() Step1HeldApplyResults.Num(), GetActiveGenerationTaskCount(), PendingTiles.Num()); Step1LastLoggedPrequeueCount = Step1HeldVisibleLOD0Count; } + RecordApplyFrame(); return; } } @@ -9222,9 +9362,27 @@ void AVoxelWorld::ProcessPendingChunks() // ApplyTileResult does epoch check, capture ingest, empty-release / mesh submission, and // only then records the tile as loaded. Only a real (visible) upload counts against the // per-frame budget — stale/empty drains are free. - if (ApplyTileResult(DequeuedChunk)) + const int32 ResultLevel = DequeuedChunk.Tile.Level; + const uint64 TileApplyStartCycles = FPlatformTime::Cycles64(); + const bool bMeshApplied = ApplyTileResult(DequeuedChunk); + const uint64 TileApplyEndCycles = FPlatformTime::Cycles64(); + const double TileApplySeconds = TileApplyEndCycles >= TileApplyStartCycles + ? FPlatformTime::ToSeconds64(TileApplyEndCycles - TileApplyStartCycles) + : 0.0; + ApplySeconds += TileApplySeconds; + if (ResultLevel >= 0 && ResultLevel < 9) + { + ++ApplyResultsByLevelTotal; + ++ApplyResultsByLevel[ResultLevel]; + ApplySecondsByLevel[ResultLevel] += TileApplySeconds; + } + if (bMeshApplied) { ++MeshesApplied; + if (ResultLevel >= 0 && ResultLevel < 9) + { + ++MeshesAppliedByLevel[ResultLevel]; + } } // Time-box the entire game-thread drain, not only visible uploads. Empty releases and @@ -9255,6 +9413,7 @@ void AVoxelWorld::ProcessPendingChunks() ApplyBudgetMilliseconds, bTimeBudgetHit ? 1 : 0); } + RecordApplyFrame(); } // Game-thread apply for one gen result. Shared by ProcessPendingChunks (async drain) and @@ -9379,31 +9538,44 @@ bool AVoxelWorld::ApplyTileResult(FChunkResult& Result) const uint64 AppliedCycles = FPlatformTime::Cycles64(); const bool bHaveWorkerTiming = Result.GenerationStartCycles != 0 && Result.GenerationEndCycles >= Result.GenerationStartCycles; + const bool bHaveWorkerTaskTiming = Result.WorkerTaskStartCycles != 0 + && Result.WorkerTaskEndCycles >= Result.WorkerTaskStartCycles; const double RequestToApply = FPlatformTime::ToSeconds64( AppliedCycles - Result.RequestStartCycles); const double ApplySeconds = ApplyStartCycles != 0 ? FPlatformTime::ToSeconds64(AppliedCycles - ApplyStartCycles) : 0.0; + const double WorkerTaskSeconds = bHaveWorkerTaskTiming + ? FPlatformTime::ToSeconds64( + Result.WorkerTaskEndCycles - Result.WorkerTaskStartCycles) + : (bHaveWorkerTiming ? FPlatformTime::ToSeconds64( + Result.GenerationEndCycles - Result.GenerationStartCycles) : 0.0); const double GenerationSeconds = bHaveWorkerTiming ? FPlatformTime::ToSeconds64( Result.GenerationEndCycles - Result.GenerationStartCycles) : 0.0; const double WorkerQueueSeconds = bHaveWorkerTiming ? FPlatformTime::ToSeconds64( - Result.GenerationStartCycles - Result.RequestStartCycles) : 0.0; - const double ResultQueueSeconds = bHaveWorkerTiming + (bHaveWorkerTaskTiming ? Result.WorkerTaskStartCycles + : Result.GenerationStartCycles) - Result.RequestStartCycles) : 0.0; + const uint64 WorkerTaskEndCycles = bHaveWorkerTaskTiming + ? Result.WorkerTaskEndCycles : Result.GenerationEndCycles; + const double ResultQueueSeconds = bHaveWorkerTiming && ApplyStartCycles != 0 + && ApplyStartCycles >= WorkerTaskEndCycles + ? FPlatformTime::ToSeconds64(ApplyStartCycles - WorkerTaskEndCycles) + : (bHaveWorkerTiming ? FMath::Max(0.0, RequestToApply - WorkerQueueSeconds - - GenerationSeconds - ApplySeconds) - : FMath::Max(0.0, RequestToApply - ApplySeconds); + - WorkerTaskSeconds - ApplySeconds) + : FMath::Max(0.0, RequestToApply - ApplySeconds)); if (GVoxelForgeProfileTileGeneration != 0) { UE_LOG(LogTemp, Display, TEXT("[VoxelForgeTileApplied] tile=(%d,%d,%d) level=%d empty=%d triangles=%d " "band=(%d,%d) " - "request_to_apply=%.6f worker_queue=%.6f generation=%.6f " + "request_to_apply=%.6f worker_queue=%.6f worker_task=%.6f generation=%.6f " "result_queue=%.6f apply=%.6f"), Result.Tile.Coord.X, Result.Tile.Coord.Y, Result.Tile.Coord.Z, Result.Tile.Level, bResultEmpty ? 1 : 0, Result.NumTriangles, Result.BandChunkLo, Result.BandChunkHi, - RequestToApply, WorkerQueueSeconds, GenerationSeconds, + RequestToApply, WorkerQueueSeconds, WorkerTaskSeconds, GenerationSeconds, ResultQueueSeconds, ApplySeconds); } if (bTraceTile) @@ -9427,6 +9599,7 @@ bool AVoxelWorld::ApplyTileResult(FChunkResult& Result) Sample.WorkerQueueSeconds = WorkerQueueSeconds; Sample.ResultQueueSeconds = ResultQueueSeconds; Sample.QueueWaitSeconds = WorkerQueueSeconds + ResultQueueSeconds; + Sample.WorkerTaskSeconds = WorkerTaskSeconds; Sample.GenerationSeconds = GenerationSeconds; Sample.ClassifySeconds = Result.ClassifySeconds; Sample.MeshSeconds = Result.MeshSeconds; @@ -11307,7 +11480,12 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw // refinement. The opt-out reproduces the S4 LOD0-to-LOD8 submission order for a control. // Le shell lointain passe apres l'anneau de collision, puis le raffinement fin. L'option // inverse restaure l'ordre LOD0-vers-LOD8 de S4 comme controle negatif. - if (GVoxelForgeStreamingCoarseFirst != 0) + // Keep the walkable ring on the legacy near-first path; only spend its reserved task slots + // on coarse work after all nine collision gates are ready. + // Garder l'anneau praticable en priorite ; les taches grossieres commencent apres ses neuf collisions. + const bool bUseCoarseFirstOrder = GVoxelForgeStreamingCoarseFirst != 0 + && bWalkableRingReadyForReserve; + if (bUseCoarseFirstOrder) { const int32 FirstCoarseLevel = FMath::Max( 2, Settings ? Settings->FullResClipLevels : 2); @@ -11426,7 +11604,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw TEXT("\"scope\":\"desired_residency_not_visual_enclosure\",\"center\":[%d,%d,%d]," "\"position_cm\":[%s,%s,%s],\"speed_cm_s\":%s,\"walkable_ready\":%d," "\"desired_by_lod\":%s,\"resident_by_lod\":%s,\"pending_by_lod\":%s," - "\"active_generation\":%d,\"task_cap\":%d,\"near_reserve\":%d,\"epoch\":%u," + "\"active_generation\":%d,\"task_cap\":%d,\"near_reserve\":%d," + "\"coarse_first\":%d,\"dynamic_near_reserve\":%d,\"epoch\":%u," "\"update_chunks_ms\":%s,\"desired_rebuild_ms\":%s"), CenterChunk.X, CenterChunk.Y, CenterChunk.Z, *FString::SanitizeFloat(LocalPawn.X), *FString::SanitizeFloat(LocalPawn.Y), @@ -11434,7 +11613,9 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw *FString::SanitizeFloat(PlayerHeading.Size2D()), bWalkableRingReadyForReserve ? 1 : 0, *DesiredCounts, *ResidentCounts, *PendingCounts, - GetActiveGenerationTaskCount(), MaxTasks, NearTaskReserve, GenerationEpoch, + GetActiveGenerationTaskCount(), MaxTasks, NearTaskReserve, + GVoxelForgeStreamingCoarseFirst, GVoxelForgeStreamingDynamicNearReserve, + GenerationEpoch, *FString::SanitizeFloat(StreamingUpdateMilliseconds), *FString::SanitizeFloat(DesiredRebuildMilliseconds))); LastStreamingTraceSampleSeconds = TraceNow; @@ -11626,8 +11807,16 @@ void AVoxelWorld::LoadTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority Wor std::atomic& Counter; ~FTaskGuard() { Counter.fetch_sub(1, std::memory_order_relaxed); } } Guard{ActiveTaskCount}; + ActiveGenerationWorkerCount.fetch_add(1, std::memory_order_relaxed); + const uint64 WorkerTaskStartCycles = FPlatformTime::Cycles64(); + struct FWorkerGuard + { + std::atomic& Counter; + ~FWorkerGuard() { Counter.fetch_sub(1, std::memory_order_relaxed); } + } WorkerGuard{ActiveGenerationWorkerCount}; FChunkResult Result; + Result.WorkerTaskStartCycles = WorkerTaskStartCycles; Result.RequestOrdinal = RequestOrdinal; Result.bResidentAtRequest = bResidentAtRequest; Result.WorkPriority = WorkPriority; @@ -11647,6 +11836,8 @@ void AVoxelWorld::LoadTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority Wor Canceled.RequestStartCycles = RequestStartCycles; Canceled.RequestOrdinal = RequestOrdinal; Canceled.GenerationOrdinal = Result.GenerationOrdinal; + Canceled.WorkerTaskStartCycles = Result.WorkerTaskStartCycles; + Canceled.WorkerTaskEndCycles = FPlatformTime::Cycles64(); Canceled.GenerationStartCycles = Result.GenerationStartCycles; Canceled.GenerationEndCycles = FPlatformTime::Cycles64(); Canceled.bResidentAtRequest = bResidentAtRequest; @@ -11723,6 +11914,7 @@ void AVoxelWorld::LoadTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority Wor if (!ShouldAbortWork()) { + Result.WorkerTaskEndCycles = FPlatformTime::Cycles64(); if (WorkPriority == EVoxelTileWorkPriority::CollisionCritical) CriticalProcessQueue.Enqueue(MoveTemp(Result)); else if (WorkPriority == EVoxelTileWorkPriority::Edited) @@ -11790,9 +11982,17 @@ void AVoxelWorld::LoadCoreTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority std::atomic& Counter; ~FTaskGuard() { Counter.fetch_sub(1, std::memory_order_relaxed); } } Guard{ActiveTaskCount}; + ActiveGenerationWorkerCount.fetch_add(1, std::memory_order_relaxed); + const uint64 WorkerTaskStartCycles = FPlatformTime::Cycles64(); + struct FWorkerGuard + { + std::atomic& Counter; + ~FWorkerGuard() { Counter.fetch_sub(1, std::memory_order_relaxed); } + } WorkerGuard{ActiveGenerationWorkerCount}; auto Enqueue = [this, WorkPriority](FChunkResult&& Result) { + Result.WorkerTaskEndCycles = FPlatformTime::Cycles64(); if (WorkPriority == EVoxelTileWorkPriority::CollisionCritical) CriticalProcessQueue.Enqueue(MoveTemp(Result)); else if (WorkPriority == EVoxelTileWorkPriority::Edited) @@ -11810,6 +12010,7 @@ void AVoxelWorld::LoadCoreTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority } FChunkResult Result; + Result.WorkerTaskStartCycles = WorkerTaskStartCycles; Result.Tile = Tile; Result.Epoch = TaskEpoch; Result.DesiredEpoch = TaskDesiredEpoch; @@ -11831,6 +12032,7 @@ void AVoxelWorld::LoadCoreTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority Canceled.RequestStartCycles = RequestStartCycles; Canceled.RequestOrdinal = RequestOrdinal; Canceled.GenerationOrdinal = Result.GenerationOrdinal; + Canceled.WorkerTaskStartCycles = Result.WorkerTaskStartCycles; Canceled.bResidentAtRequest = bResidentAtRequest; Canceled.GenerationStartCycles = Result.GenerationStartCycles; Canceled.GenerationEndCycles = FPlatformTime::Cycles64(); diff --git a/Source/VoxelForge/Public/VoxelStartupTrace.h b/Source/VoxelForge/Public/VoxelStartupTrace.h index 63439e0..92a6a7f 100644 --- a/Source/VoxelForge/Public/VoxelStartupTrace.h +++ b/Source/VoxelForge/Public/VoxelStartupTrace.h @@ -29,6 +29,7 @@ namespace VoxelForgeStartupTrace double QueueWaitSeconds = 0.0; double WorkerQueueSeconds = 0.0; double ResultQueueSeconds = 0.0; + double WorkerTaskSeconds = 0.0; double GenerationSeconds = 0.0; double ClassifySeconds = 0.0; double MeshSeconds = 0.0; @@ -36,6 +37,31 @@ namespace VoxelForgeStartupTrace double ApplySeconds = 0.0; }; + struct VOXELFORGE_API FApplyFrameSample + { + uint64 Frame = 0; + double FrameSeconds = 0.0; + double SpeedCmPerSecond = 0.0; + double ApplySeconds = 0.0; + double DrainSeconds = 0.0; + int32 ActiveTasks = 0; + int32 RunningWorkerTasks = 0; + int32 TaskCap = 0; + int32 PendingTiles = 0; + int32 PendingResultsEstimate = 0; + int32 ResultsDrained = 0; + int32 ApplyResultsByLevelTotal = 0; + int32 MeshesApplied = 0; + int32 MaxApplies = 0; + double ApplyBudgetMilliseconds = 0.0; + bool bCountLimitReached = false; + bool bTimeLimitReached = false; + TStaticArray PendingByLevel{}; + TStaticArray ApplyResultsByLevel{}; + TStaticArray MeshesAppliedByLevel{}; + TStaticArray ApplySecondsByLevel{}; + }; + /** Starts when -voxel.StartupTraceFile= is present. */ VOXELFORGE_API void BeginFromCommandLine(); @@ -51,6 +77,10 @@ namespace VoxelForgeStartupTrace VOXELFORGE_API void RecordTile(const FTileSample& Sample); + /** One game-thread result-drain sample, including worker occupancy and per-LOD applies. */ + /** Echantillon de vidage des resultats, avec occupation des travailleurs et applications par LOD. */ + VOXELFORGE_API void RecordApplyFrame(const FApplyFrameSample& Sample); + /** Writes the final bounded summary and closes the file. Safe to call more than once. */ VOXELFORGE_API void Finish(const TCHAR* Reason); diff --git a/Source/VoxelForge/Public/VoxelWorld.h b/Source/VoxelForge/Public/VoxelWorld.h index 717b4bb..ec0d8b6 100644 --- a/Source/VoxelForge/Public/VoxelWorld.h +++ b/Source/VoxelForge/Public/VoxelWorld.h @@ -132,6 +132,11 @@ struct FChunkResult // opt-in profile. It is carried through the worker queue so the report can separate scheduler // wait from generation time. uint64 RequestStartCycles = 0; + // Full background-task boundaries include stream conversion and result packaging around the + // generator call; they let the trace reconcile per-tile worker-seconds with worker occupancy. + // Les bornes de tâche incluent la conversion et la préparation du résultat autour du générateur. + uint64 WorkerTaskStartCycles = 0; + uint64 WorkerTaskEndCycles = 0; // Worker timing boundaries for separating scheduler wait from generation and result/apply wait. // These are captured with low overhead for the clean streaming-latency summary; expensive // per-tile profiling remains opt-in. @@ -1118,6 +1123,9 @@ public: // Number of async tasks currently running — EndPlay waits for this to reach 0 std::atomic ActiveTaskCount{0}; + // Active workers are counted inside the launched tile task, distinct from tasks queued by UE. + // Les travailleurs actifs sont comptes dans la tache de tuile, distincts des taches en attente UE. + std::atomic ActiveGenerationWorkerCount{0}; FORCEINLINE bool ShouldAbortWork(const std::atomic* ObsoleteFlag = nullptr) const { @@ -1175,6 +1183,10 @@ public: bool bAllChunksLoaded = false; bool bStartupTraceDesiredRecorded = false; bool bSampleStacksStarted = false; + bool bStartupApplyBoostStarted = false; + bool bStartupApplyBoostEnded = false; + uint64 LastApplyFrameCycles = 0; + float StartupTraceSpeedCmPerSecond = 0.0f; TArray DesiredSorted; // critical floor prefix, then true-pawn distance-first TArray CriticalDesiredTiles; // occupied/support tile, then heading tile // Ancestor -> desired descendants, rebuilt with DesiredSorted. Reconciliation uses this diff --git a/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp b/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp index 2fe6973..a6d6e10 100644 --- a/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp +++ b/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp @@ -72,6 +72,7 @@ #include "VoxelTerrainOpDefinition.h" #include "VoxelTypes.h" #include "VoxelWorld.h" +#include "VoxelStartupTrace.h" #include "VoxelForgeCoreAssetAdapter.h" #include "VoxelForgeCore/VoxelForgeCore.h" @@ -878,6 +879,7 @@ struct FExploreArguments int32 PassageFloorWrites = 1; bool bStandaloneWorldExport = false; bool bCoreWorldTest = false; + bool bCoreWorldStreamTraceOnly = false; bool bRenderCameraOverride = false; FVector RenderCamera = FVector::ZeroVector; bool bRenderTargetOverride = false; @@ -923,6 +925,9 @@ bool ParseArguments(const FString& Params, FExploreArguments& OutArguments, FStr int32 CoreWorldTest = 0; FParse::Value(*Params, TEXT("coreworldtest="), CoreWorldTest); OutArguments.bCoreWorldTest = CoreWorldTest != 0; + int32 CoreWorldStreamTraceOnly = 0; + FParse::Value(*Params, TEXT("coreworldstreamtrace="), CoreWorldStreamTraceOnly); + OutArguments.bCoreWorldStreamTraceOnly = CoreWorldStreamTraceOnly != 0; FParse::Value(*Params, TEXT("corevoxelsizecm="), OutArguments.CoreVoxelSizeCm); if (!FMath::IsFinite(OutArguments.CoreVoxelSizeCm) || OutArguments.CoreVoxelSizeCm < VOXEL_SIZE @@ -1499,6 +1504,11 @@ bool ParseArguments(const FString& Params, FExploreArguments& OutArguments, FStr OutError = TEXT("coreworldtest requires settingsref and must omit strateref."); return false; } + if (OutArguments.bCoreWorldStreamTraceOnly && !OutArguments.bCoreWorldTest) + { + OutError = TEXT("coreworldstreamtrace requires coreworldtest."); + return false; + } if ((OutArguments.bCoreHybridMesher || OutArguments.bCoreHybridFixturePlacement || OutArguments.bCoreHybridCliffOnly || OutArguments.bCoreHybridFixturesOnly || OutArguments.bCoreFloatingIslandTest || OutArguments.bCoreFallbackTest @@ -17329,6 +17339,241 @@ bool RunCoreWorldActorIntegrationTest(const FExploreArguments& Arguments, FStrin return false; } + if (Arguments.bCoreWorldStreamTraceOnly) + { + const UWorld::InitializationValues WorldInitialization = UWorld::InitializationValues() + .AllowAudioPlayback(false) + .RequiresHitProxies(false) + .CreatePhysicsScene(true) + .CreateNavigation(false) + .CreateAISystem(false) + .ShouldSimulatePhysics(true) + .SetDefaultGameMode(AGameModeBase::StaticClass()) + .SetTransactional(false); + UWorld* TestWorld = UWorld::CreateWorld( + EWorldType::Game, false, FName(TEXT("VoxelForgeCoreStreamTraceWorld")), + GetTransientPackage(), false, ERHIFeatureLevel::Num, &WorldInitialization); + if (TestWorld == nullptr) + { + OutError = TEXT("Could not create the transient BuildHost stream-trace world."); + return false; + } + UGameInstance* TestGameInstance = NewObject(GEngine); + FWorldContext& TestWorldContext = GEngine->CreateNewWorldContext(EWorldType::Game); + TestWorldContext.OwningGameInstance = TestGameInstance; + TestWorldContext.SetCurrentWorld(TestWorld); + TestWorld->SetGameInstance(TestGameInstance); + TestWorld->SetShouldTick(false); + TestGameInstance->Init(); + TestWorld->AddToRoot(); + struct FStreamTraceWorldCleanup + { + UWorld* World = nullptr; + uint64 PreviousFrameCounter = GFrameCounter; + ~FStreamTraceWorldCleanup() + { + if (World != nullptr) + { + if (World->HasBegunPlay()) + { + World->BeginTearingDown(); + World->EndPlay(EEndPlayReason::Quit); + } + if (UGameInstance* GameInstance = World->GetGameInstance()) + { + GameInstance->Shutdown(); + } + if (GEngine != nullptr) GEngine->DestroyWorldContext(World); + World->DestroyWorld(false); + if (World->IsRooted()) World->RemoveFromRoot(); + } + GFrameCounter = PreviousFrameCounter; + } + } Cleanup{TestWorld, GFrameCounter}; + + if (!TestWorld->SetGameMode(FURL())) + { + OutError = TEXT("Could not create the transient BuildHost GameMode for stream tracing."); + return false; + } + if (AGameModeBase* GameMode = TestWorld->GetAuthGameMode()) + { + GameMode->DefaultPawnClass = nullptr; + } + UPlayer* TestPlayer = NewObject(GEngine, NAME_None, RF_Transient); + APlayerController* PlayerController = TestWorld->SpawnActor( + APlayerController::StaticClass(), FTransform::Identity, FActorSpawnParameters()); + if (TestPlayer == nullptr || PlayerController == nullptr) + { + OutError = TEXT("Could not create the BuildHost stream-trace player controller."); + return false; + } + PlayerController->SetPlayer(TestPlayer); + TestWorld->AddController(PlayerController); + APawn* PlayerPawn = TestWorld->SpawnActor( + APawn::StaticClass(), FVector::ZeroVector, FRotator::ZeroRotator); + if (PlayerPawn == nullptr) + { + OutError = TEXT("Could not create the BuildHost stream-trace pawn."); + return false; + } + USceneComponent* PawnRoot = NewObject(PlayerPawn, NAME_None, RF_Transient); + if (PawnRoot == nullptr) + { + OutError = TEXT("Could not create the BuildHost stream-trace pawn root."); + return false; + } + PlayerPawn->SetRootComponent(PawnRoot); + PawnRoot->RegisterComponent(); + PlayerPawn->SetActorLocation(FVector::ZeroVector, false, nullptr, ETeleportType::TeleportPhysics); + PlayerController->Possess(PlayerPawn); + if (TestWorld->GetFirstPlayerController() != PlayerController + || PlayerController->GetPawn() != PlayerPawn) + { + OutError = TEXT("BuildHost stream-trace controller did not retain its pawn."); + return false; + } + + const FTransform ActorTransform = FTransform::Identity; + AVoxelWorld* Actor = TestWorld->SpawnActorDeferred( + AVoxelWorld::StaticClass(), ActorTransform, nullptr, nullptr, + ESpawnActorCollisionHandlingMethod::AlwaysSpawn); + if (Actor == nullptr) + { + OutError = TEXT("Could not spawn the transient AVoxelWorld for stream tracing."); + return false; + } + Actor->Settings = AuthoredSettings; + Actor->bManageAtmosphere = false; + Actor->bUseVoxelForgeCore = true; + Actor->CoreVoxelSizeCm = Arguments.CoreVoxelSizeCm; + Actor->CoreMesherRecipe = Arguments.CoreHybridMesherVersion == 2 + ? EVoxelForgeCoreMesherRecipe::WorldAssemblerHybridV2 + : (Arguments.CoreHybridMesherVersion == 1 + ? EVoxelForgeCoreMesherRecipe::WorldAssemblerHybridV1 + : EVoxelForgeCoreMesherRecipe::MarchingCubesV1); + Actor->CoreClipRadius = 3; + Actor->CoreMaxClipLevel = 8; + Actor->bCaptureVoxelForgeCoreHashesForValidation = true; + Actor->FinishSpawning(ActorTransform); + TestWorld->InitializeActorsForPlay(FURL()); + TestWorld->BeginPlay(); + if (!VoxelForgeStartupTrace::IsActive()) + { + OutError = TEXT("Core stream trace mode requires -voxel.StartupTraceFile."); + return false; + } + Actor->SetActorTickEnabled(true); + + auto DesiredTilesLoaded = [](const AVoxelWorld* World) + { + if (World == nullptr || World->DesiredSorted.IsEmpty()) return false; + for (const FVoxelTileKey& Key : World->DesiredSorted) + { + if (!World->LoadedTiles.Contains(Key)) return false; + } + return true; + }; + auto CollisionSetSettled = [](const AVoxelWorld* World) + { + if (World == nullptr) return false; + for (const FVoxelTileKey& Key : World->CollisionDesiredTiles) + { + if (!World->CollisionReadyTiles.Contains(Key) + && !World->CollisionNotRequiredTiles.Contains(Key)) return false; + } + return true; + }; + const double ActorBeginSeconds = FPlatformTime::Seconds(); + constexpr double WalkStartSeconds = 30.0; + constexpr double RunDurationSeconds = 45.0; + TArray SpawnTickMilliseconds; + TArray WalkTickMilliseconds; + TArray> TickSamples; + bool bInitialSetReady = false; + double InitialSetReadySeconds = -1.0; + while (FPlatformTime::Seconds() - ActorBeginSeconds < RunDurationSeconds) + { + const double TickStartSeconds = FPlatformTime::Seconds(); + TestWorld->Tick(LEVELTICK_All, 0.016f); + const double TickMilliseconds = + (FPlatformTime::Seconds() - TickStartSeconds) * 1000.0; + ++GFrameCounter; + const double ElapsedSeconds = FPlatformTime::Seconds() - ActorBeginSeconds; + TickSamples.Emplace(ElapsedSeconds, TickMilliseconds); + + if (!bInitialSetReady && Actor->IsVoxelForgeCoreReady() + && Actor->bAllChunksLoaded && Actor->PendingTiles.IsEmpty() + && Actor->PendingCollisionCooks.IsEmpty() && Actor->PendingUnload.IsEmpty() + && Actor->ActiveTaskCount.load(std::memory_order_acquire) == 0 + && DesiredTilesLoaded(Actor) && CollisionSetSettled(Actor)) + { + bInitialSetReady = true; + InitialSetReadySeconds = ElapsedSeconds; + VoxelForgeStartupTrace::RecordEvent(TEXT("stream_trace_milestone"), FString::Printf( + TEXT("\"name\":\"initial_set_ready\",\"elapsed_s\":%s,\"desired\":%d,\"loaded\":%d,\"collision_desired\":%d,\"collision_ready\":%d"), + *FString::SanitizeFloat(ElapsedSeconds), Actor->DesiredSorted.Num(), + Actor->LoadedTiles.Num(), Actor->CollisionDesiredTiles.Num(), + Actor->CollisionReadyTiles.Num())); + } + FPlatformProcess::Sleep(0.016f); + } + + for (const TPair& Sample : TickSamples) + { + if (bInitialSetReady && Sample.Key <= InitialSetReadySeconds) + { + SpawnTickMilliseconds.Add(Sample.Value); + } + else if (Sample.Key >= WalkStartSeconds) + { + WalkTickMilliseconds.Add(Sample.Value); + } + } + + auto PercentileMilliseconds = [](TArray Values, double Fraction) + { + if (Values.IsEmpty()) return 0.0; + Values.Sort(); + const int32 Index = FMath::Clamp( + FMath::CeilToInt(Fraction * static_cast(Values.Num() - 1)), + 0, Values.Num() - 1); + return Values[Index]; + }; + const auto MaxMilliseconds = [](const TArray& Values) + { + double Maximum = 0.0; + for (const double Value : Values) Maximum = FMath::Max(Maximum, Value); + return Maximum; + }; + UE_LOG(LogTemp, Display, + TEXT("[VoxelForgeCoreWorldStreamTrace] initial_ready=%d initial_ready_s=%.6f walk_start_s=%.3f run_s=%.3f desired=%d loaded=%d " + "spawn_tick_ms_p95=%.3f spawn_tick_ms_max=%.3f walk_tick_ms_p95=%.3f walk_tick_ms_max=%.3f"), + bInitialSetReady ? 1 : 0, InitialSetReadySeconds, WalkStartSeconds, + RunDurationSeconds, Actor->DesiredSorted.Num(), Actor->LoadedTiles.Num(), + PercentileMilliseconds(SpawnTickMilliseconds, 0.95), + MaxMilliseconds(SpawnTickMilliseconds), + PercentileMilliseconds(WalkTickMilliseconds, 0.95), + MaxMilliseconds(WalkTickMilliseconds)); + if (!bInitialSetReady) + { + OutError = FString::Printf( + TEXT("Actor-only Core stream trace did not reach initial readiness: ready=%d desired=%d loaded=%d pending=%d cooks=%d active=%d."), + Actor->IsVoxelForgeCoreReady() ? 1 : 0, Actor->DesiredSorted.Num(), + Actor->LoadedTiles.Num(), Actor->PendingTiles.Num(), + Actor->PendingCollisionCooks.Num(), + Actor->ActiveTaskCount.load(std::memory_order_acquire)); + return false; + } + UE_LOG(LogTemp, Display, + TEXT("[VoxelForgeCoreWorldStreamTrace] pass complete=1 recipe=%s voxel_cm=%.3f walk_speed_cm_s=600 walk_direction=+Y"), + Arguments.CoreHybridMesherVersion == 2 ? TEXT("WorldAssemblerHybridV2") + : (Arguments.CoreHybridMesherVersion == 1 ? TEXT("WorldAssemblerHybridV1") + : TEXT("MarchingCubesV1")), + Arguments.CoreVoxelSizeCm); + return true; + } + // Build one Standalone reference before creating the actor world so the two full Core worlds // never overlap in memory. The reference also precomputes the exact edit result and hashes. // Construire la référence Standalone avant le monde acteur pour ne jamais garder deux mondes diff --git a/Standalone/Research/OwnerShapeWorld/gpu_field_port/LUNA_DIALOGUE.md b/Standalone/Research/OwnerShapeWorld/gpu_field_port/LUNA_DIALOGUE.md index 284f098..721de9b 100644 --- a/Standalone/Research/OwnerShapeWorld/gpu_field_port/LUNA_DIALOGUE.md +++ b/Standalone/Research/OwnerShapeWorld/gpu_field_port/LUNA_DIALOGUE.md @@ -1265,3 +1265,21 @@ and every failed or corrected run in [WorldAssemblerM3/Round9/REPORT.md](../../W No owner editor/project or asset was touched; no commit, push or stash was made. — Luna + + +## World assembler S6 / M3 Round 10 — 2026-10-19 + +The 251 ms staircase was the residency sampler cadence; direct tile apply events are the completion +clock. The legacy-order control completed the staged 2,578-key set in 17.412 s. Gating coarse-first +admission behind the nine-tile walkable ring reduced it to 11.704 s without delaying that gate +(0.632 → 0.627 s). The 12-mesh / 14 ms spawn apply boost did not improve the legacy timeline, so it +remains opt-in and the normal 4-mesh / 2 ms guard stays in place. Worker-side work supply/admission +remains the measured floor; the trace does not isolate why worker slots stay idle. The 1.57 s aim is +still unmet and rendered enclosure remains unproven in the null-RHI harness. + +The corrected standalone 60 m/s route now counts +Y crossings across multi-chunk deltas. Its +crossing data is valid, though the latency gate still fails. W1/W8/W16 and all nine LOD hashes match +S5. The new trace, regressions, run ledger, exact log-captured command lines, hashes and compile +spots are in [WorldAssemblerM3/Round10/REPORT.md](../../WorldAssemblerM3/Round10/REPORT.md). + +— Luna diff --git a/Standalone/Research/WorldAssemblerM3/Round10/REPORT.md b/Standalone/Research/WorldAssemblerM3/Round10/REPORT.md new file mode 100644 index 0000000..a41cc08 --- /dev/null +++ b/Standalone/Research/WorldAssemblerM3/Round10/REPORT.md @@ -0,0 +1,218 @@ +# World assembler S6 / M3 Round 10 — in-game cold-view time + +**Result:** the 251 ms staircase was trace sampling cadence, not a scheduler tick. On the staged +2,578-key SurfaceWorld actor, the default legacy path reached full residency in 17.412 s. Gated +coarse-first admission reached it in 11.704 s while the walkable ring time stayed at 0.627 s versus +0.632 s. A 12-mesh / 14 ms startup apply budget did not improve the legacy timeline. The remaining +floor is on worker-side work supply/admission, not the game-thread apply budget; the trace does not +isolate why available worker slots stay idle. The 1.57 s aim remains unmet. +This is null-RHI residency evidence, not rendered enclosure proof. + +## 1. Cadence and comparison with S5 + +`AVoxelWorld::UpdateChunksAroundPosition` emits `streaming_sample` only when at least 0.25 s has +elapsed since its previous sample. S5’s near-251 ms per-LOD staircase was quantized by that observer; +it was not a one-LOD-per-tick scheduler throttle. S6 records direct tile-result/apply timestamps and +per-frame `ProcessPendingChunks` samples, so the completion table below does not depend on that +sampler. + +S5 reported 3.352–6.367 s for its per-LOD residency samples. The Round 9 report did not retain the +exact BuildHost command line, executable hash, or raw direct-tile trace for that run. S6 uses the +same staged `DA_Settings` SurfaceWorld plan, 25 cm Core scale, 2,578 desired keys, 12-task host setup, +and legacy-order controls, but its direct full-set result is 17.412 s. The 250 ms sampling interval +cannot explain this larger difference. Treat S5’s 3.35–6.37 s values as a different/unidentified +measurement route, not a matched baseline; S6 preserves the direct command lines and hashes below. +The Standalone 3.58–3.64 s S5 measurement was CPU-only over 2,553 keys, so it is also not a matched +BuildHost apply/collision measurement. + +## 2. Matched BuildHost actor runs + +Fresh sequential processes used the staged owner plan, `CoreVoxelSizeCm=25`, 2,578 desired tiles, +`-corelimit=12` (10 active task slots in the actor trace), null RHI, a manually ticked `UWorld` with +a 16 ms step, and a +Y 600 cm/s move beginning at 30 s. There was no overlapping Unreal process or +build. Full-set time is the actor’s initial desired-set milestone. “LOD0 full” means all 512 visual +LOD0 keys, separate from the 9/9 collision walkable gate. + +| Configuration | Full set ready | Walkable ring 9/9 | Full 512-key LOD0 | Count-cap frames / backlog frames | Visible meshes/frame p95/max | Apply CPU p95/max | `UWorld::Tick` spawn p95/max | `UWorld::Tick` walk p95/max | +|---|---:|---:|---:|---:|---:|---:|---:|---:| +| Legacy order, authored 4 / 2 ms budget | 17.412 s | 0.632 s | 2.252 s | 78 / 22 | 4 / 4 | 0.77 / 1.62 ms | 1.406 / 5.059 ms | 1.771 / 29.247 ms | +| Legacy order, startup boost on (12 / 14 ms) | 17.346 s | 0.611 s | 2.190 s | 0 / 0 | 4 / 6 | 0.70 / 1.49 ms | 1.301 / 5.429 ms | 1.798 / 28.489 ms | +| Coarse-first after ring gate, authored 4 / 2 ms | **11.704 s** | **0.627 s** | 10.201 s | 118 / 48 | 4 / 4 | 0.82 / 1.44 ms | 1.396 / 5.027 ms | 1.911 / 28.276 ms | + +Coarse-first shortened the full desired-set tail by 5.708 s (32.8%) without a walkable-gate +regression. It did not make all near detail ready sooner: full LOD0 moved from 2.252 s to 10.201 s. +The option therefore stays opt-in pending a rendered view/enclosure check. The apply boost removed +cap frames but changed full-set time by only 66 ms (0.4%), so it also stays opt-in. Normal play +continues to use the authored 4 visible-mesh / 2 ms limit. These are `UWorld::Tick` CPU samples in a +manual null-RHI harness, not full rendered frame times. Walk p95 stayed below 2 ms, but the observed +~28–29 ms maxima remain and were not isolated to a single subsystem. Per-frame apply maxima were +1.62 ms legacy and 1.44 ms gated coarse-first. + +### Per-LOD cold set + +Each pair is **legacy → gated coarse-first**, from direct tile completion events up to the initial +desired-set milestone. Queue values are cumulative tile-seconds, not elapsed wall seconds. The +initial set contains 2,578 tile results; empty results are included in the result count and apply +time. + +| LOD | Last tile apply (s) | Worker task (s) | Worker queue (tile-s) | Result queue (tile-s) | GT apply (ms) / results | +|---:|---:|---:|---:|---:|---:| +| 0 | 2.252 → 10.201 | 3.39 → 3.39 | 0.028 → 0.033 | 5.94 → 5.87 | 27.9 → 35.0 / 512 | +| 1 | 4.785 → 11.765 | 3.21 → 3.05 | 0.007 → 0.006 | 4.43 → 4.79 | 17.0 → 20.4 / 448 | +| 2 | 6.609 → 8.567 | 0.70 → 0.71 | 0.012 → 0.006 | 4.77 → 4.90 | 10.9 → 13.0 / 320 | +| 3 | 8.162 → 7.460 | 1.81 → 1.78 | 0.005 → 0.005 | 2.86 → 3.07 | 10.5 → 11.8 / 272 | +| 4 | 10.317 → 6.530 | 6.02 → 6.08 | 0.004 → 0.006 | 2.16 → 2.62 | 26.5 → 24.9 / 272 | +| 5 | 11.494 → 5.182 | 1.42 → 1.42 | 0.003 → 0.003 | 2.20 → 2.40 | 18.4 → 15.9 / 208 | +| 6 | 12.483 → 4.438 | 1.85 → 1.90 | 0.002 → 0.002 | 1.58 → 1.65 | 15.2 → 12.8 / 144 | +| 7 | 13.198 → 3.852 | 1.74 → 1.75 | 0.001 → 0.001 | 0.88 → 1.02 | 16.7 → 13.5 / 96 | +| 8 | 17.476 → 3.432 | 17.50 → 19.29 | 0.005 → 0.465 | 2.59 → 2.62 | 95.0 → 87.0 / 306 | + +Per-frame visible mesh counts and the largest per-LOD apply sample were: + +| LOD | Visible meshes/frame p95/max, legacy → coarse | Max LOD apply/frame, legacy → coarse | +|---:|---:|---:| +| 0 | 2 / 4 → 3 / 4 | 0.762 → 0.934 ms | +| 1 | 1 / 4 → 1 / 4 | 0.630 → 0.809 ms | +| 2 | 0 / 4 → 1 / 4 | 0.533 → 0.639 ms | +| 3 | 0 / 4 → 1 / 4 | 0.515 → 0.570 ms | +| 4 | 2 / 4 → 3 / 4 | 0.605 → 1.354 ms | +| 5 | 1 / 3 → 2 / 4 | 1.272 → 0.742 ms | +| 6 | 0 / 4 → 1 / 4 | 0.687 → 0.708 ms | +| 7 | 0 / 4 → 0 / 4 | 0.974 → 0.732 ms | +| 8 | 2 / 4 → 3 / 4 | 1.625 → 1.439 ms | + +The 10-task-slot slack estimate was 132.1 slot-seconds on legacy order and 73.3 on coarse-first, +computed as ten slots × (first request to full-set ready) minus measured worker-task seconds. The +shared pool averaged 3.27 / 4.21 running workers on frames with estimated pending results; one +legacy backlog frame had no running worker, versus none in the coarse-first run. Idle worker time +and apply-cap frames are shared-pool/world properties, so a per-LOD split would double-count them. +Legacy used 37.64 worker-seconds and coarse-first 39.36; pre-start worker-queue wait was only +0.067 / 0.527 tile-seconds. Result-queue wait summed to 27.41 / 28.94 tile-seconds. Game-thread +tile-apply time was about 0.24 / 0.23 seconds for the cold set. The boost run had no apply-cap or +result-backlog frames and still took 17.346 s. Thus the apply cap is not the end-to-end floor. The +remaining floor is worker-side work supply/admission; the trace does not isolate why available +worker slots stay idle or identify a safe single worker-side change for this round. + +Collision is currently required only on the LOD0 path. The trace captured 60 body-update events in +each cold run; 60 of 125 collision-desired entries were ready by the full-set milestone, while +non-required/empty entries were classified separately. RealtimeMesh submit-to-body-update latency +(not isolated Chaos CPU cooking) was legacy p50/p95/max 17.02 / 17.89 / 18.04 ms, boost 16.80 / +17.51 / 17.70 ms, and coarse-first 17.08 / 18.01 / 18.30 ms. The nine required ring tiles were ready +at the gate times above. LOD1–8 had no collision-ready events in this trace. + +## 3. Shared change and standalone crossing instrument + +`voxel.StreamingCoarseFirst=1` now starts far-level admissions only after all nine required +walkable-ring collision gates are ready. `voxel.StreamingDynamicNearReserve=1` keeps the authored +near reserve while moving or before that gate; after a stationary gate it lets coarse work use the +reserved slots. Both are explicit, default-off controls. `voxel.StreamingStartupApplyBoost=1` allows +12 visible-mesh applies / 14 ms during the initial desired-set and walkable-ring phase, with a 30 s +ceiling; zero keeps the authored budget. It is not preemptive within one apply, so it remains opt-in. +This behavior is in shared `AVoxelWorld::UpdateChunksAroundPosition` / `ProcessPendingChunks` and +does not change density, tile output, or collision policy. + +The standalone 60 m/s flight instrument now counts forward +Y chunk crossings using the actual +center delta, including multi-chunk jumps. The valid viewer route completed 250/250 center samples, +with 226 center updates and max forward delta +2. It reports `crossing_deltas=forward-+Y`, +resident 2,553/2,553 and no missing tiles. Latency still **fails**: nearest-ready p50/p95/max was +9.087 / 20.594 / 128.785 ms and maximum crossing wall time 42.931 s. The earlier +`--stream-benchmark` and `--stream-pipeline` invocations emitted no Flight markers; only the viewer +route is used for this result. + +## 4. Hashes, regression checks and async audit + +Standalone W1/W8/W16 fresh processes produced identical full-world and all nine per-LOD hashes; +all match S5. Full world identity: density `c10697a8f48380c7`, mesh `430d2be2bdd8b988`, payload +`b7b33e14092225c9`, topology `c5c46d66606a819e`. + +- Core-off legacy at 12.5 cm: **PASS**, `VOXELFORGE_CORE_OFF_LEGACY_PASS`. +- M2 V1 and V2: **PASS**, five fixtures plus same-LOD border for each. +- SurfaceWorld and FloatingIslands BuildHost at 25 cm: **PASS**. FloatingIslands W1/W8/W16 + matched Standalone; the final Standalone W1/W8/W16 matrix also matched across all nine LODs and S5. +- S4 cap census/agreement and ray coverage: **PASS**; 551 requested/emitted cap bands, 1,584/1,584 + coverage rays, 128-point cap agreement max 0.0504 cm against 2 cm tolerance, negative control + detected. +- S3 ground and S4 LOD7/8 ground agreement: **PASS**; max ground delta 0.0427 cm, cross-LOD 0.0 cm, + and the negative control was detected. +- Water remains fixed for the season; player carve/fill stays unrestricted. + +Diff audit: `GenerationEpoch` is carried and stale epochs are rejected; the four result queues remain +`EQueueMode::Mpsc`; `EndPlay` sets shutdown, waits for active work and drains the queues; workers +check `bShuttingDown`; `ModsLock` behavior is unchanged; all UObject/RealtimeMesh component creation +and mutation stays on the game thread. No density-sign or determinism path changed. The added C++ +comments are French and English. No generated header, `Binaries/`, or `Intermediate/` source edits. + +## 5. Exact commands, binaries and run ledger + +BuildHost actor runs used `E:\Program Files\Epic Games\UE_5.7\Engine\Binaries\Win64\UnrealEditor-Cmd.exe` +with `-nullrhi -unattended -corelimit=12 -run=VoxelForgeExplore -seed=0 -archetype=TunnelNetwork +-coreworldtest=1 -corevoxelsizecm=25 -settingsref=/Game/VoxelForge/DA_Settings.DA_Settings +-coreworldstreamtrace=1 -voxel.TestMove=1 -voxel.TestMoveSpeedCmPerSecond=600 +-voxel.TestMoveStartSeconds=30 -voxel.TestMoveDirection=0,1,0 +-voxel.StartupTraceUntilEndPlay=1 -voxel.CollisionGate=0`. The exact expanded `LogInit: Command Line:` +including project, DDC, trace and output paths is preserved verbatim in each [legacy run](../../../../Saved/BuildHost/HostProject/Plugins/VoxelForge/Saved/CoreWorldValidation/Logs/CoreWorldValidation-owner_surface_stream_trace_only_applyboost_0_legacy_order_trace.stdout.log), +[boost run](../../../../Saved/BuildHost/HostProject/Plugins/VoxelForge/Saved/CoreWorldValidation/Logs/CoreWorldValidation-owner_surface_stream_trace_only_applyboost_1_legacy_order_trace.stdout.log), +and [gated coarse run](../../../../Saved/BuildHost/HostProject/Plugins/VoxelForge/Saved/CoreWorldValidation/Logs/CoreWorldValidation-owner_surface_stream_trace_only_applyboost_0_coarse_first_trace.stdout.log). +Their CVar overrides were respectively: + +| Run | `StreamingCoarseFirst` | `StreamingDynamicNearReserve` | `StreamingStartupApplyBoost` | +|---|---:|---:|---:| +| Legacy negative control | 0 | 0 | 0 | +| Apply-budget A/B | 0 | 0 | 1 | +| Gated coarse-first | 1 | 1 | 0 | + +Standalone 60 m/s exact command (also in [commands.txt](../../../../Saved/BuildHost/S6/standalone/commands.txt)): + +```text +Standalone\bin\VoxelForgeCoreStandalone.exe --world-assembler-prototype --force-generate --workers=12 --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --test-flight-crossings=250 --flight-speed-mps=60 --process-priority=Normal --process-affinity=4095 --frame-limit=60 +``` + +| Executable / module | SHA-256 | +|---|---| +| BuildHost `UnrealEditor-Cmd.exe` | `ECCE9996191F7F7E69B3D104EECDD0E678AE69176FDCCD79828C3E75A0654D6D` | +| BuildHost `UnrealEditor-VoxelForge.dll` | `1C083446ED360FBD798EAE4C4D3CA9E5994E906770B5017A51418BFEF46C30FA` | +| BuildHost `UnrealEditor-VoxelForgeCore.dll` | `54259552340FE81EA05640C622E9E5E5C0AC68C95F87DC718326D25EF133B2AC` | +| Standalone `VoxelForgeCoreStandalone.exe` | `1F445760AFDE264E2F28F8F2FB4CB68DBD0C8988AACD5A493B065E74C1827529` | + +BuildHost UBT passed twice at 12 actions / Normal priority. `& .\Standalone\Build.ps1 -Target All +-CompileWorkers 12 -BuildPriority Normal` passed. Existing vswhere/C4838 warnings did not fail the +Standalone build. The S6 hash and 60 m/s logs are in [Saved/BuildHost/S6/standalone](../../../../Saved/BuildHost/S6/standalone/). + +Failed/retried attempts, including setup failures: + +- An earlier full actor attempt reached its 600 s timeout in the Standalone-reference preamble; it + left a partial trace. A first trace-only command omitted its required startup-trace output path + and exited before the actor run; the helper now makes `StreamingTraceOnly` imply trace output. +- Two fresh-shell memory preflights lacked the local `Add-Type` helper and stopped before launching + Unreal. The corrected `GlobalMemoryStatusEx` preflight ran before later launches. +- Early coarse-first attempts did not parse the requested CVar in headless `BeginPlay`; trace values + showed legacy LOD0-first order. Explicit CVar parsing fixed the A/B. An ungated coarse-first run + then delayed the walkable gate to 1.653 s; admission was changed to wait for 9/9 and rerun. +- FloatingIslands at 25 cm hit its 120 s test timeout; the bounded 600 s retry passed in 192 s. +- The first `--stream-benchmark` and `--stream-pipeline` sprint commands emitted no Flight markers. + The corrected viewer command above produced valid crossing deltas, with the latency failure + reported rather than hidden. +- Unsupported `-RunName`, a fresh-shell type reuse, and one PowerShell metric-parser expression + failed before/after measurement without changing an Unreal run. The corrected command/parser + produced the data above. + +Every Unreal launch was sequential, with more than 17 GiB free memory at the final launch checks; +actor-harness peak was about 3.5 GiB and all measured processes stayed under 8 GiB. No run returned +`0xC0000142`. Crash audit found no new UECC folder; the only folder is the pre-existing +`UECC-Windows-F1A3EFD5469F5DD62E68EBBACAC84376_0000` dated 2026-09-17. Test stdout and JSONL traces +remain under `Saved/BuildHost/...`; none were copied into `Research/`. + +## 6. Files and compile spots + +No new C++ source file was added. Modified source files and likely compile spots: + +- `Source/VoxelForge/Private/VoxelWorld.cpp`, `Source/VoxelForge/Public/VoxelWorld.h`, + `Source/VoxelForge/Private/VoxelStartupTrace.cpp`, + `Source/VoxelForge/Public/VoxelStartupTrace.h` — VoxelForge runtime module; BuildHost UBT. +- `Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp` — editor module; BuildHost UBT. +- `Standalone/Source/StandaloneViewer.cpp` — `Standalone/Build.ps1 -Target All`. +- `Tools/TestVoxelForgeCoreWorldHost.ps1` — harness argument/default changes; no C++ compile. +- `CODEMAP.md` §3, `ARCHITECTURE.md` §8.27, this report, and + `Standalone/Research/OwnerShapeWorld/gpu_field_port/LUNA_DIALOGUE.md` — documentation. + +No owner file or asset was touched; no commit, push or stash was made. diff --git a/Standalone/Source/StandaloneViewer.cpp b/Standalone/Source/StandaloneViewer.cpp index a935e75..1dd6464 100644 --- a/Standalone/Source/StandaloneViewer.cpp +++ b/Standalone/Source/StandaloneViewer.cpp @@ -1691,6 +1691,8 @@ namespace bool bDensityHash=false; int32 FlightTargetCrossings=0; int32 FlightObservedCrossings=0; + int32 FlightCenterUpdates=0; + int32 FlightMaximumForwardDelta=0; uint64 FlightStartCrossingIndex=0; float FlightSpeedMps=16.0f; std::string FlightCaptureSequencePrefix; @@ -4890,9 +4892,15 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75; NewDesired.CenterChunk.Y-Previous.Y,NewDesired.CenterChunk.Z-Previous.Z); if(FlightTargetCrossings>0&&bFlightStarted&&!bFlightFinished) { - ++FlightObservedCrossings; - if(CenterDelta.X!=0||CenterDelta.Y!=1||CenterDelta.Z!=0) + ++FlightCenterUpdates; + if(CenterDelta.X!=0||CenterDelta.Y<=0||CenterDelta.Z!=0) bFlightCrossingsValid=false; + else + { + FlightObservedCrossings+=CenterDelta.Y; + FlightMaximumForwardDelta=std::max( + FlightMaximumForwardDelta,CenterDelta.Y); + } } std::unordered_set NewSet; NewSet.reserve(NewDesired.Tiles.size()*2+1); @@ -5353,13 +5361,14 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75; const double NearestMaximum=NearestReadyMilliseconds.empty()?-1.0: NearestReadyMilliseconds.back(); const std::size_t MissingDesired=DesiredTiles.size()-DesiredResidentCount; - const bool bPass=FlightObservedCrossings==FlightTargetCrossings + const bool bPass=FlightObservedCrossings>=FlightTargetCrossings &&bFlightCrossingsValid&&CacheHits==0&&bCrossingLatencyUnderTarget &&MissingDesired==0&&DeferredLeaverTiles.empty()&&DirtyBatchSet.empty(); - std::printf("[FlightBenchmarkComplete] crossings=%d/%d speed=%.2fm/s elapsed=%.3fs cache_hits=%llu crossing_deltas=%s nearest_ready_p50=%.3fms nearest_ready_p95=%.3fms nearest_ready_max=%.3fms max_crossing_wall=%.3fs max_worker_cpu_sum=%.3fs resident=%zu/%zu missing_tiles=%zu deferred_leavers=%zu uploads_drained=%s status=%s\n", - FlightObservedCrossings,FlightTargetCrossings,FlightSpeedMps,Elapsed, + std::printf("[FlightBenchmarkComplete] center_chunks=%d/%d center_updates=%d max_forward_delta_y=%d speed=%.2fm/s elapsed=%.3fs cache_hits=%llu crossing_deltas=%s nearest_ready_p50=%.3fms nearest_ready_p95=%.3fms nearest_ready_max=%.3fms max_crossing_wall=%.3fs max_worker_cpu_sum=%.3fs resident=%zu/%zu missing_tiles=%zu deferred_leavers=%zu uploads_drained=%s status=%s\n", + FlightObservedCrossings,FlightTargetCrossings, + FlightCenterUpdates,FlightMaximumForwardDelta,FlightSpeedMps,Elapsed, static_cast(CacheHits), - bFlightCrossingsValid?"unit-+Y":"invalid", + bFlightCrossingsValid?"forward-+Y":"invalid", Percentile(50),Percentile(95),NearestMaximum,MaxCrossingWallSeconds, MaxWorkerCpuSeconds,DesiredResidentCount,DesiredTiles.size(),MissingDesired, DeferredLeaverTiles.size(),DirtyBatchSet.empty()?"yes":"no", @@ -5692,8 +5701,9 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75; { bFlightStarted=true;FlightStartTime=Now;FlightStartPosition=Position; FlightStartCrossingIndex=CrossingIndex;FlightObservedCrossings=0; + FlightCenterUpdates=0;FlightMaximumForwardDelta=0; bFlightCrossingsValid=true; - std::printf("[FlightBenchmarkStart] crossings=%d speed=%.2fm/s direction=+Y start=(%.2f,%.2f,%.2f)m cache_mode=none\n", + std::printf("[FlightBenchmarkStart] center_chunks=%d speed=%.2fm/s direction=+Y start=(%.2f,%.2f,%.2f)m cache_mode=none\n", FlightTargetCrossings,FlightSpeedMps,Position.X,Position.Y,Position.Z); std::fflush(stdout); } diff --git a/Tools/TestVoxelForgeCoreWorldHost.ps1 b/Tools/TestVoxelForgeCoreWorldHost.ps1 index f89e0a9..3395110 100644 --- a/Tools/TestVoxelForgeCoreWorldHost.ps1 +++ b/Tools/TestVoxelForgeCoreWorldHost.ps1 @@ -2,9 +2,11 @@ param( [ValidateRange(60, 900)][int]$TimeoutSeconds = 900, [ValidateRange(12.5, 25.0)][double]$CoreVoxelSizeCm = 25.0, - [ValidateSet('CoarseFirst', 'Legacy')][string]$StreamingOrder = 'CoarseFirst', + [ValidateSet('CoarseFirst', 'Legacy')][string]$StreamingOrder = 'Legacy', [switch]$EnforceCollisionGate, [switch]$StartupTrace, + [ValidateRange(0, 1)][int]$StartupApplyBoost = 0, + [switch]$StreamingTraceOnly, [string]$WorldAssemblyManifestPath = '', [ValidateSet('MarchingCubesV1', 'WorldAssemblerHybridV1', 'WorldAssemblerHybridV2')][string]$CoreMesherRecipe = 'MarchingCubesV1', [switch]$EnableHybridFixturePlacement, @@ -50,11 +52,13 @@ $RunName = if ($FloatingIslandsOnly) { "floating_islands$CoreSizeSuffix" } elseif ($CoreOffTest) { "core_off$CoreSizeSuffix" } elseif ($CorePlanOnly) { if ($CoreTestSeed -ne [int]::MinValue) { "plan_seed_$CoreTestSeed$CoreSizeSuffix" } else { "plan_only$CoreSizeSuffix" } } else { "owner_surface$CoreSizeSuffix" } +$RunName += $(if ($StreamingTraceOnly) { '_stream_trace_only' } else { '' }) +$RunName += "_applyboost_$StartupApplyBoost" if ($MismatchedManifestFallback) { $CoreFallbackTest = $true } $SelectedModeFlags = @( [bool]$FloatingIslandsOnly, [bool]$CoreFallbackTest, - [bool]$CoreOffTest, [bool]$CorePlanOnly + [bool]$CoreOffTest, [bool]$CorePlanOnly, [bool]$StreamingTraceOnly ) $SelectedModeCount = @($SelectedModeFlags | Where-Object { $_ }).Count if ($SelectedModeCount -gt 1) { throw 'Select one Core test mode per BuildHost launch.' } @@ -64,6 +68,10 @@ if ($CorePlanOnly -and $CoreTestSeed -eq [int]::MinValue) { if ($CoreTestSeed -ne [int]::MinValue -and -not $CorePlanOnly) { throw '-CoreTestSeed is valid only with -CorePlanOnly.' } +if ($StreamingTraceOnly -and ($SharedFeatureAuditOnly -or $EnableHybridFixturePlacement -or + $HybridCliffOnly -or $HybridFixturesOnly -or $WorldAssemblyManifestPath)) { + throw '-StreamingTraceOnly uses the staged owner plan without additional fixture or manifest overrides.' +} function Assert-UnderPath([string]$Candidate, [string]$Root) { $candidateFull = [IO.Path]::GetFullPath($Candidate).TrimEnd('\', '/') @@ -138,7 +146,7 @@ if ($SharedFeatureAuditOnly) { } $RunName += if ($StreamingOrder -eq 'Legacy') { '_legacy_order' } else { '_coarse_first' } if ($EnforceCollisionGate) { $RunName += '_collision_gate' } -if ($StartupTrace) { $RunName += '_trace' } +if ($StartupTrace -or $StreamingTraceOnly) { $RunName += '_trace' } $foreignEditors = @(Get-Process -ErrorAction SilentlyContinue | Where-Object { $_.ProcessName -like 'UnrealEditor*' }) @@ -257,7 +265,15 @@ $CoarseFirstValue = if ($StreamingOrder -eq 'CoarseFirst') { 1 } else { 0 } $DynamicReserveValue = if ($StreamingOrder -eq 'CoarseFirst') { 1 } else { 0 } $BaseArguments += "-voxel.StreamingCoarseFirst=$CoarseFirstValue" $BaseArguments += "-voxel.StreamingDynamicNearReserve=$DynamicReserveValue" -if ($StartupTrace) { +$BaseArguments += "-voxel.StreamingStartupApplyBoost=$StartupApplyBoost" +if ($StreamingTraceOnly) { + $BaseArguments += '-coreworldstreamtrace=1' + $BaseArguments += '-voxel.TestMove=1' + $BaseArguments += '-voxel.TestMoveSpeedCmPerSecond=600' + $BaseArguments += '-voxel.TestMoveStartSeconds=30' + $BaseArguments += '-voxel.TestMoveDirection=0,1,0' +} +if ($StartupTrace -or $StreamingTraceOnly) { $TracePath = Join-Path $LogDirectory "CoreWorldValidation-$RunName.trace.jsonl" $BaseArguments += ('-voxel.StartupTraceFile="{0}"' -f $TracePath) $BaseArguments += '-voxel.StartupTraceUntilEndPlay=1' @@ -387,6 +403,22 @@ if ($SharedFeatureAuditOnly) { Write-Host "Peak commandlet private bytes sampled: $PeakPrivateBytes" return } +if ($StreamingTraceOnly) { + if ($Text -notmatch '\[VoxelForgeCoreWorldStreamTrace\] initial_ready=1 .*walk_tick_ms_p95=.*walk_tick_ms_max=' -or + $Text -notmatch '\[VoxelForgeCoreWorldStreamTrace\] pass complete=1 .*walk_speed_cm_s=600 walk_direction=\+Y' -or + $Text -notmatch '\[VoxelForgeCoreWorldTest\] pass complete=1') { + throw "BuildHost actor-only stream trace did not prove initial readiness and a completed walk sample. See $FinalLog." + } + $NewCrashes = @(Get-NewCrashFolders) + if ($NewCrashes.Count -gt 0) { + throw "New UECC crash folders appeared during the stream trace: $($NewCrashes -join '; ')" + } + Write-Host "VOXELFORGE_CORE_STREAM_TRACE_PASS boost=$StartupApplyBoost order=$StreamingOrder" + Write-Host "Core actor log: $FinalLog" + Write-Host "Startup trace: $TracePath" + Write-Host "Peak commandlet private bytes sampled: $PeakPrivateBytes" + return +} if ($CoreFallbackTest) { foreach ($Marker in @( '[VoxelForgeCoreFallbackTest] fallback=PASS core_disabled=1 legacy_generator=1',