From 212814a75c93b018e10af0d49485dd2eb3b81d66 Mon Sep 17 00:00:00 2001 From: Fr0zka Date: Mon, 19 Oct 2026 14:29:38 +0200 Subject: [PATCH] wip(assembler): round S12 -- short ready queue keeps workers fed between ticks: full set 6.0 s -> 4.9 s voxel.StreamingReadyQueueDepth=8 admits one extra worker-set of prioritized tasks; voxel.StreamingBackgroundWorkerCap=1 sizes slots to the 8 TaskGraph background workers actually running tile tasks. Full set 5.96 -> 4.91 s, LOD0 1.40 -> 1.09 s, ring 0.49 -> 0.51 s, 40 m/s ring 75/75. Tick p95 +0.6 ms spawn / +0.26 ms walk, max flat. No geometry change. Report: Standalone/Research/WorldAssemblerM3/Round16. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01MmSoF64BbRFmk9C3p1VsBJ --- ARCHITECTURE.md | 49 +++ CODEMAP.md | 18 ++ .../VoxelForge/Private/VoxelStartupTrace.cpp | 12 + Source/VoxelForge/Private/VoxelWorld.cpp | 191 ++++++++--- Source/VoxelForge/Public/VoxelStartupTrace.h | 4 + Source/VoxelForge/Public/VoxelWorld.h | 5 + .../WorldAssemblerM3/Round16/REPORT.md | 302 ++++++++++++++++++ Tools/TestVoxelForgeCoreWorldHost.ps1 | 9 + 8 files changed, 554 insertions(+), 36 deletions(-) create mode 100644 Standalone/Research/WorldAssemblerM3/Round16/REPORT.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index eff599f..b7dd3d6 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1707,6 +1707,55 @@ ready well before that far-LOD tail. Geometry and collision arithmetic do not ch versions stay unchanged. See [`Standalone/Research/WorldAssemblerM3/Round15/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round15/REPORT.md). +### 8.33 Streaming worker supply and tick-independent task pickup — World Assembler S12 / M3 Round 16 + +The S11 trace showed that admission was paced by the actor tick: typical tiles finished within one +18 ms frame, while the eight TaskGraph background workers waited for the next scheduler pass to +receive another task. The host exposed ten TaskGraph workers (two foreground, eight background), +and the tile tasks ran only on the eight background workers. The shared scheduler now admits up to +one additional worker-set of prioritized tile tasks (`voxel.StreamingReadyQueueDepth`, default 8, +clamped to the effective worker count). These tasks are still selected and submitted on the game +thread through the existing desired-tile ordering; UE's background executor dispatches queued work +as each worker becomes free. There is no worker-owned continuation list, and no UObject, RMC, or +desired-set mutation moves off the game thread. + +`voxel.StreamingBackgroundWorkerCap=1` sizes the streaming admission slots to +`FTaskGraphInterface::GetNumBackgroundThreads()`. It does not lower `GetMaxConcurrentTasks()` or the +worker limit used inside Core's parallel generation. `0` restores the old logical-core-derived +streaming cap. The `worker_backend_config` and `streaming_worker_cap_readback` trace events record +the CVar readback, logical core count, TaskGraph and global queued-pool sizes, task API, and effective +slot count. Each tile row records its submission, worker start/end, and OS thread ID. The external +BuildHost sample also records process thread count and working set; its eight tile IDs match the +TaskGraph background pool. RMC's inspected async collision path queues through `GThreadPool`, which +is a separate 10-thread pool; it does not occupy those eight TaskGraph worker IDs. Arbitrary UE +background tasks can share TaskGraph background workers, so the pool count alone does not imply +those workers are idle outside the measured tile task intervals. + +Admission remains shared by `LoadTile` and `LoadCoreTile`. The queue-aware dynamic near reserve is +at least the ready-queue depth while moving or before the stationary walkable gate is ready, capped +at one less than the worker slots. Horizon work uses a separate coarse cap; while moving, ordinary +LOD0 admission stops at the actual worker-slot count and critical collision/edit work retains its +existing higher admission cap. Thus queued far work cannot consume the reserved near capacity. +`bStreamingPlayerMoving` is captured on the game thread for both admission paths. Existing +`GenerationEpoch` / `DesiredEpoch` and per-request cancellation checks still reject stale tasks and +results; no preprepared desired list survives a set change. + +On the matched 25 cm SurfaceWorld startup trace, queue depth 8 plus the 8-worker slot cap brought the +2,578-tile ready time to 4.911 s (the matched queue-depth-zero control took 7.024 s). The LOD0 +worker tail ended at trace time 1.167 s; its last apply was at 1.087 s on the actor startup clock, +against S11's 1.403 s. The stationary 3×3 collision-ready ring was ready at 0.509 s, close to the S11 +0.488 s measurement. The per-tile trace contains all 2,578 initial requests and shows eight active +workers at peak, 29.432 worker-seconds, and 6.573 seconds of inter-task idle time summed across the +eight slots. 500 ms busy-worker equivalents, per-LOD completion and each slot's inter-task gap are +saved under `Saved/BuildHost/Round16/`. The full table and matched controls are in the Round 16 +report. + +Foreground task priorities were tested and rejected: they increased spawn/walk `UWorld::Tick` +tails despite occupying more TaskGraph workers. The shipped priorities stay `BackgroundHigh` for +near and `BackgroundNormal` for horizon tasks. The task-pickup changes do not alter density, +triangulation, collision, or Core output, so no recipe version changes. +See [`Standalone/Research/WorldAssemblerM3/Round16/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round16/REPORT.md). + ## 9. Multiplayer model (listen-server first, dedicated-friendly) > **Status: transport is DESIGN ONLY — nothing is networked in-tree yet** (no `Replicated`/`HasAuthority`/RPCs; a diff --git a/CODEMAP.md b/CODEMAP.md index bb11b7f..9bb49d9 100644 --- a/CODEMAP.md +++ b/CODEMAP.md @@ -1065,6 +1065,24 @@ cap and output-assembly costs, plus opt-in duplicated LOD8 XY evaluations. The S `--far-horizon-lod8-step=128` restores S9 sampling for the negative control. See ARCHITECTURE §8.31 and [`Standalone/Research/WorldAssemblerM3/Round14/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round14/REPORT.md). +### 3.18 Streaming worker supply and ready queue — World Assembler S12 / M3 Round 16 + +`AVoxelWorld::LoadTile` and `LoadCoreTile` retain the existing `UE::Tasks::Launch` background +priorities (`BackgroundHigh` for near work, `BackgroundNormal` for horizon work). The shared +scheduler now sizes streaming admission to the TaskGraph background worker count by default +(`voxel.StreamingBackgroundWorkerCap=1`) and admits a bounded ready queue of up to one worker-set +(`voxel.StreamingReadyQueueDepth=8` on the 8-background-worker host). This lets UE dispatch the next +already-admitted task as a worker becomes free without waiting for the next actor tick. Setting the +worker cap to `0` restores the prior logical-core cap; setting queue depth to `0` restores one-slot-per +worker admission. The moving near reserve grows with the ready queue, coarse/horizon work remains +under its separate cap, and moving LOD0 admission is limited to the actual worker slots so far work +cannot occupy reserved headroom. `bStreamingPlayerMoving` is a game-thread scheduler snapshot used +by both admission helpers. `worker_backend_config`, `streaming_worker_cap_readback`, and +`scheduler_admission` record the pool counts and effective CVar values. Per-tile trace rows record +submit/start/end times and the OS worker thread ID. No tile generation, mesh, density, collision, or +Core recipe output changes; recipe versions remain unchanged. See ARCHITECTURE §8.33 and +[`Standalone/Research/WorldAssemblerM3/Round16/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round16/REPORT.md). + ## 4. The density pipeline (most-edited hot path) ### 4.1 `GetDensityWithParams` (TunnelNetwork) — VoxelGenerator.cpp:3749 diff --git a/Source/VoxelForge/Private/VoxelStartupTrace.cpp b/Source/VoxelForge/Private/VoxelStartupTrace.cpp index 1f7bb1b..bbc4e07 100644 --- a/Source/VoxelForge/Private/VoxelStartupTrace.cpp +++ b/Source/VoxelForge/Private/VoxelStartupTrace.cpp @@ -284,9 +284,17 @@ void RecordTile(const FTileSample& Sample) GState->Samples.Add(Sample); ++GState->TileRecordsWritten; + const double TraceNowSeconds = FPlatformTime::Seconds() - GState->StartSeconds; + const double SubmittedAtSeconds = Sample.RequestStartCycles != 0 + ? TraceNowSeconds - Sample.RequestToApplySeconds : -1.0; + const double WorkerStartedAtSeconds = Sample.WorkerTaskStartCycles != 0 + ? SubmittedAtSeconds + Sample.WorkerQueueSeconds : -1.0; + const double WorkerEndedAtSeconds = Sample.WorkerTaskEndCycles != 0 + ? WorkerStartedAtSeconds + Sample.WorkerTaskSeconds : -1.0; WriteEventLocked(*GState, TEXT("tile"), FString::Printf( TEXT("\"tile\":[%d,%d,%d],\"level\":%d,\"priority\":%d," "\"request_ordinal\":%llu,\"generation_ordinal\":%llu,\"resident_at_request\":%d," + "\"submitted_t_s\":%s,\"worker_start_t_s\":%s,\"worker_end_t_s\":%s,\"worker_thread_id\":%u," "\"verdict\":%d,\"proof\":%d,\"empty\":%d,\"vertices\":%d,\"triangles\":%d,\"geometry_bytes\":%llu," "\"request_to_apply_s\":%s,\"queue_wait_s\":%s,\"worker_queue_s\":%s,\"result_queue_s\":%s,\"worker_task_s\":%s," "\"generation_s\":%s,\"classify_s\":%s,\"mesh_s\":%s,\"stream_s\":%s,\"apply_s\":%s"), @@ -295,6 +303,10 @@ void RecordTile(const FTileSample& Sample) static_cast(Sample.RequestOrdinal), static_cast(Sample.GenerationOrdinal), Sample.bResidentAtRequest ? 1 : 0, + *Number(SubmittedAtSeconds), + *Number(WorkerStartedAtSeconds), + *Number(WorkerEndedAtSeconds), + Sample.WorkerThreadId, Sample.Verdict, Sample.bSealedSolidProof ? 1 : 0, Sample.bEmpty ? 1 : 0, Sample.Vertices, Sample.Triangles, static_cast(Sample.GeometryBytes), diff --git a/Source/VoxelForge/Private/VoxelWorld.cpp b/Source/VoxelForge/Private/VoxelWorld.cpp index f04d24e..483ddd2 100644 --- a/Source/VoxelForge/Private/VoxelWorld.cpp +++ b/Source/VoxelForge/Private/VoxelWorld.cpp @@ -53,6 +53,8 @@ #include "Engine/GameViewportClient.h" #include "Slate/SceneViewport.h" #include "Camera/CameraActor.h" +#include "Async/Fundamental/Scheduler.h" +#include "Async/TaskGraphInterfaces.h" #include "Components/PointLightComponent.h" #include "VoxelAtmosphereManager.h" #include "DrawDebugHelpers.h" @@ -221,6 +223,18 @@ namespace GVoxelForgeStreamingResultBacklogCap, TEXT("Maximum queued results plus admitted tile tasks; 0 disables the safety cap. This is separate from the worker-slot cap.")); + int32 GVoxelForgeStreamingReadyQueueDepth = 8; + FAutoConsoleVariableRef CVarVoxelForgeStreamingReadyQueueDepth( + TEXT("voxel.StreamingReadyQueueDepth"), + GVoxelForgeStreamingReadyQueueDepth, + TEXT("Extra admitted tile tasks kept ready beyond the worker slots, up to one worker-set; 0 disables the refill queue.")); + + int32 GVoxelForgeStreamingBackgroundWorkerCap = 1; + FAutoConsoleVariableRef CVarVoxelForgeStreamingBackgroundWorkerCap( + TEXT("voxel.StreamingBackgroundWorkerCap"), + GVoxelForgeStreamingBackgroundWorkerCap, + TEXT("Size tile admission slots to the TaskGraph background workers used by tile tasks; 0 restores the logical-core cap.")); + int32 GVoxelForgeProfileTileGeneration = 0; FAutoConsoleVariableRef CVarVoxelForgeProfileTileGeneration( TEXT("voxel.ProfileTileGeneration"), @@ -2064,6 +2078,8 @@ void AVoxelWorld::BeginPlay() int32 CommandLineCoreRegionConcurrentAdmission = GVoxelForgeCoreRegionConcurrentAdmission; int32 CommandLineStreamingCollisionLookahead = GVoxelForgeStreamingCollisionLookahead; int32 CommandLineStreamingResultBacklogCap = GVoxelForgeStreamingResultBacklogCap; + int32 CommandLineStreamingReadyQueueDepth = GVoxelForgeStreamingReadyQueueDepth; + int32 CommandLineStreamingBackgroundWorkerCap = GVoxelForgeStreamingBackgroundWorkerCap; int32 CommandLineStep1GenerationCap = GVoxelForgeStep1GenerationCap; int32 CommandLineStep1ExitAfterFrames = GVoxelForgeStep1ExitAfterFrames; int32 CommandLineStep1CsvCaptureOnBeginPlay = GVoxelForgeStep1CsvCaptureOnBeginPlay; @@ -2158,6 +2174,18 @@ void AVoxelWorld::BeginPlay() { GVoxelForgeStreamingResultBacklogCap = FMath::Max(0, CommandLineStreamingResultBacklogCap); } + if (FParse::Value(FCommandLine::Get(), TEXT("voxel.StreamingReadyQueueDepth="), + CommandLineStreamingReadyQueueDepth)) + { + GVoxelForgeStreamingReadyQueueDepth = FMath::Clamp( + CommandLineStreamingReadyQueueDepth, 0, 32); + } + if (FParse::Value(FCommandLine::Get(), TEXT("voxel.StreamingBackgroundWorkerCap="), + CommandLineStreamingBackgroundWorkerCap)) + { + GVoxelForgeStreamingBackgroundWorkerCap = + CommandLineStreamingBackgroundWorkerCap != 0 ? 1 : 0; + } if (FParse::Value(FCommandLine::Get(), TEXT("voxel.Step1PrequeueVisibleLOD0="), CommandLineStep1PrequeueVisibleLOD0)) { @@ -2560,6 +2588,48 @@ void AVoxelWorld::BeginPlay() Settings->bGenerateSkirts ? 1 : 0, Settings->FullResClipLevels, Settings->CoarseTileCells)); + if (VoxelForgeStartupTrace::IsActive()) + { + FTaskGraphInterface& TaskGraph = FTaskGraphInterface::Get(); + LowLevelTasks::FScheduler& TaskScheduler = LowLevelTasks::FScheduler::Get(); + const int32 QueueDepth = FMath::Clamp( + GVoxelForgeStreamingReadyQueueDepth, 0, GetMaxConcurrentStreamingTasks()); + VoxelForgeStartupTrace::RecordEvent(TEXT("worker_backend_config"), FString::Printf( + TEXT("\"logical_cores\":%d,\"effective_tile_slots\":%d," + "\"task_api\":\"UE::Tasks::Launch\",\"task_scheduler_workers\":%u," + "\"task_scheduler_max_workers\":%u,\"task_scheduler_worker_priority\":%d," + "\"task_scheduler_background_priority\":%d," + "\"taskgraph_worker_threads_per_priority\":%d,\"taskgraph_foreground_threads\":%d," + "\"taskgraph_background_threads\":%d,\"global_queued_thread_pool_threads\":%d," + "\"step1_cook_pool_threads\":%d,\"ready_queue_depth_cvar\":%d," + "\"ready_queue_depth\":%d," + "\"near_priority\":\"BackgroundHigh\"," + "\"horizon_priority\":\"BackgroundNormal\"," + "\"streaming_coarse_first\":%d,\"streaming_dynamic_near_reserve\":%d," + "\"streaming_legacy_pending_result_cap\":%d," + "\"streaming_collision_lookahead\":%d,\"core_region_concurrent_admission\":%d," + "\"collision_gate\":%d," + "\"streaming_result_backlog_cap\":%d,\"streaming_startup_apply_boost\":%d"), + FPlatformMisc::NumberOfCoresIncludingHyperthreads(), GetMaxConcurrentStreamingTasks(), + TaskScheduler.GetNumWorkers(), TaskScheduler.GetMaxNumWorkers(), + static_cast(TaskScheduler.GetWorkerPriority()), + static_cast(TaskScheduler.GetBackgroundPriority()), + TaskGraph.GetNumWorkerThreads(), TaskGraph.GetNumForegroundThreads(), + TaskGraph.GetNumBackgroundThreads(), GThreadPool ? GThreadPool->GetNumThreads() : 0, + GVoxelForgeStep1CoreCookPool.Pool ? GVoxelForgeStep1CoreCookPool.Pool->GetNumThreads() : 0, + GVoxelForgeStreamingReadyQueueDepth, QueueDepth, + GVoxelForgeStreamingCoarseFirst, GVoxelForgeStreamingDynamicNearReserve, + GVoxelForgeStreamingLegacyPendingResultCap, + GVoxelForgeStreamingCollisionLookahead, GVoxelForgeCoreRegionConcurrentAdmission, + GVoxelForgeCollisionGate, + GVoxelForgeStreamingResultBacklogCap, GVoxelForgeStreamingStartupApplyBoost)); + VoxelForgeStartupTrace::RecordEvent(TEXT("streaming_worker_cap_readback"), FString::Printf( + TEXT("\"streaming_background_worker_cap\":%d,\"background_worker_threads\":%d," + "\"effective_tile_slots\":%d"), + GVoxelForgeStreamingBackgroundWorkerCap, + TaskGraph.GetNumBackgroundThreads(), GetMaxConcurrentStreamingTasks())); + } + // Système de strates — a cooked season takes precedence; otherwise the authored pool path is // exactly the legacy one. if (!Settings->Season.IsNull() || Settings->StratePool.Num() > 0 @@ -9779,6 +9849,10 @@ bool AVoxelWorld::ApplyTileResult(FChunkResult& Result) Sample.WorkPriority = static_cast(Result.WorkPriority); Sample.RequestOrdinal = Result.RequestOrdinal; Sample.GenerationOrdinal = Result.GenerationOrdinal; + Sample.RequestStartCycles = Result.RequestStartCycles; + Sample.WorkerTaskStartCycles = Result.WorkerTaskStartCycles; + Sample.WorkerTaskEndCycles = Result.WorkerTaskEndCycles; + Sample.WorkerThreadId = Result.WorkerThreadId; Sample.bResidentAtRequest = Result.bResidentAtRequest; Sample.Verdict = Result.ClassifyVerdict; Sample.bSealedSolidProof = Result.bSealedSolidProof; @@ -11349,8 +11423,8 @@ bool AVoxelWorld::IsTileInClipRange(const FVoxelTileKey& Tile, const FIntVector& int32 AVoxelWorld::GetMaxConcurrentTasks() const { // T2.d — both sides of the cap are authored in Voxel|Streaming. BackgroundNormal priority - // (see LoadTile) already stops gen from starving the frame; this cap stops a flat task count - // from thrashing context switches on small CPUs where it exceeds spare parallelism. + // (see LoadTile) yields to frame work; this cap avoids oversubscribing available workers. + // La priorite BackgroundNormal cede la place au frame ; ce plafond evite de depasser les workers disponibles. const int32 Asset = Settings ? Settings->MaxConcurrentTasks : 16; const int32 LogicalCores = FMath::Max( 1, FPlatformMisc::NumberOfCoresIncludingHyperthreads()); @@ -11369,6 +11443,17 @@ int32 AVoxelWorld::GetMaxConcurrentTasks() const : NormalCap; } +int32 AVoxelWorld::GetMaxConcurrentStreamingTasks() const +{ + const int32 TaskCap = GetMaxConcurrentTasks(); + if (TaskCap <= 0 || GVoxelForgeStreamingBackgroundWorkerCap == 0) + { + return TaskCap; + } + const int32 BackgroundWorkers = FTaskGraphInterface::Get().GetNumBackgroundThreads(); + return BackgroundWorkers > 0 ? FMath::Min(TaskCap, BackgroundWorkers) : TaskCap; +} + int32 AVoxelWorld::GetActiveGenerationTaskCount() const { return FMath::Max(0, ActiveTaskCount.load(std::memory_order_relaxed)); @@ -11509,7 +11594,7 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw ? Step1HeldApplyTiles.Num() : 0; const int32 PendingResultsEstimate = FMath::Max( 0, PendingTiles.Num() - ActiveTasks - HeldCompletedResults); - const int32 MaxTasks = GetMaxConcurrentTasks(); + const int32 MaxTasks = GetMaxConcurrentStreamingTasks(); const int32 AdmissionOccupancy = GetGenerationAdmissionCount(); const uint64 TotalWorkerCycles = TotalGenerationWorkerTaskCycles.load(std::memory_order_relaxed); const uint64 WorkerCyclesDelta = TotalWorkerCycles >= LastStreamingSchedulerWorkerCycles @@ -11684,7 +11769,13 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw && GVoxelForgeCoreRegionConcurrentAdmission == 0; } } - const int32 MaxTasks = GetMaxConcurrentTasks(); + const int32 MaxTasks = GetMaxConcurrentStreamingTasks(); + // Keep at most one worker-set of ready work so an admission pass can feed idle slots + // without letting far work consume the near reserve. / Garder une file d'un ensemble + // de workers pour nourrir les slots sans consommer la reserve proche. + const int32 ReadyQueueDepth = FMath::Clamp( + GVoxelForgeStreamingReadyQueueDepth, 0, MaxTasks); + const int32 TotalAdmissionCap = MaxTasks + ReadyQueueDepth; const int32 AuthoredNearTaskReserve = GetNearTaskReserve(MaxTasks); const FIntVector CenterChunk = GetTileCoordAtLocalCm(WorldToLocalCm(CenterPosition)); FVoxelTileKey WalkableSupportTile; @@ -11741,11 +11832,18 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw return; } const bool bPawnMoving = PlayerHeading.Size() >= 50.0f; + bStreamingPlayerMoving = bPawnMoving; + // Match the near reserve to the added far queue so queued horizon tiles cannot spend that headroom. + // Aligner la reserve proche sur la file lointaine pour que l'horizon ne consomme pas cette marge. + const int32 QueueAwareNearTaskReserve = FMath::Clamp( + FMath::Max(AuthoredNearTaskReserve, ReadyQueueDepth), 0, + FMath::Max(0, MaxTasks - 1)); EffectiveNearTaskReserve = GVoxelForgeStreamingDynamicNearReserve == 0 ? AuthoredNearTaskReserve - : ((bPawnMoving || !bWalkableRingReadyForReserve) ? AuthoredNearTaskReserve : 0); + : ((bPawnMoving || !bWalkableRingReadyForReserve) ? QueueAwareNearTaskReserve : 0); const int32 NearTaskReserve = EffectiveNearTaskReserve; - const int32 CoarseTaskCap = FMath::Max(1, MaxTasks - NearTaskReserve); + const int32 NearAdmissionCap = bPawnMoving ? MaxTasks : TotalAdmissionCap; + const int32 CoarseTaskCap = FMath::Max(1, MaxTasks - NearTaskReserve) + ReadyQueueDepth; const int32 FirstCoarseLevel = FMath::Max( 2, Settings ? Settings->FullResClipLevels : 2); const bool bUseCoarseFirstOrder = GVoxelForgeStreamingCoarseFirst != 0 @@ -11765,11 +11863,14 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw const int32 LogicalCores = FMath::Max( 1, FPlatformMisc::NumberOfCoresIncludingHyperthreads()); const int32 AutomaticCap = FMath::Max(2, LogicalCores - 2); + const int32 BackgroundWorkerThreads = FTaskGraphInterface::Get().GetNumBackgroundThreads(); const int32 AssetCap = Settings ? Settings->MaxConcurrentTasks : 16; const int32 AuthoredCoreCap = Settings ? Settings->MaxGenerationWorkerCores : 0; UE_LOG(LogTemp, Display, TEXT("[VoxelForgeStreamingBudget] logical_cores=%d automatic_core_cap=%d " "authored_core_cap=%d asset_task_cap=%d effective_generation_cap=%d " + "taskgraph_background_workers=%d background_worker_cap=%d " + "ready_queue_depth=%d total_admission_cap=%d " "step1_generation_cap=%d step1_max_mesh_applies=%d " "near_task_reserve=%d dynamic_reserve=%d legacy_result_cap=%d " "moving=%d walkable_ring_ready=%d " @@ -11780,6 +11881,10 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw AuthoredCoreCap, AssetCap, MaxTasks, + BackgroundWorkerThreads, + GVoxelForgeStreamingBackgroundWorkerCap, + ReadyQueueDepth, + TotalAdmissionCap, GVoxelForgeStep1GenerationCap, GVoxelForgeStep1MaxMeshApplies, NearTaskReserve, @@ -12110,11 +12215,11 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw // FLOOR RESPONSIVENESS — the support tile and collision-required 3x3 XY ring lead ordinary work. // PRIORITE AU SOL — la tuile de support et l'anneau XY 3x3 requis pour la collision passent avant le reste. - // They run at BackgroundHigh so the pawn's floor gets the first available worker. - // Ils passent en BackgroundHigh afin que le sol du pion obtienne les premiers workers disponibles. + // They use the foreground task lane when enabled, above ordinary background streaming. + // Ils utilisent la voie foreground lorsqu'elle est active, avant le streaming distant. for (const FVoxelTileKey& T : CriticalDesiredTiles) { - if (GetGenerationAdmissionCount() >= MaxTasks) + if (GetGenerationAdmissionCount() >= TotalAdmissionCap) { ++SchedulerCriticalFullCapBreaks; break; @@ -12128,7 +12233,7 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw // still in flight is KEPT queued (retried next frame) so its stale pre-carve result is corrected. for (auto It = DirtyRemeshQueue.CreateIterator(); It; ++It) { - if (GetGenerationAdmissionCount() >= MaxTasks) + if (GetGenerationAdmissionCount() >= TotalAdmissionCap) { ++SchedulerDirtyFullCapBreaks; break; @@ -12235,12 +12340,14 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw { const EVoxelTileWorkPriority Priority = Level == 0 ? EVoxelTileWorkPriority::LOD0 : EVoxelTileWorkPriority::Horizon; - SubmitLevel(Level, Priority, MaxTasks, true); + const int32 LevelAdmissionCap = Priority == EVoxelTileWorkPriority::Horizon + ? CoarseTaskCap : NearAdmissionCap; + SubmitLevel(Level, Priority, LevelAdmissionCap, true); } } else { - SubmitLevel(0, EVoxelTileWorkPriority::LOD0, MaxTasks, false); + SubmitLevel(0, EVoxelTileWorkPriority::LOD0, NearAdmissionCap, false); for (int32 Level = 1; Level <= MaxDesiredLevel; ++Level) { SubmitLevel(Level, EVoxelTileWorkPriority::Horizon, @@ -12283,8 +12390,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw { GVoxelForgeMaxPendingTilesObserved = PendingTiles.Num(); UE_LOG(LogTemp, Display, - TEXT("[VoxelForgeStreamingBudget] pending_max=%d effective=%d submitted=%d"), - GVoxelForgeMaxPendingTilesObserved, MaxTasks, Submitted); + TEXT("[VoxelForgeStreamingBudget] pending_max=%d effective=%d ready_queue=%d submitted=%d"), + GVoxelForgeMaxPendingTilesObserved, MaxTasks, ReadyQueueDepth, Submitted); } if (GVoxelForgeProfileTileGeneration != 0 && GetGenerationAdmissionCount() >= CoarseTaskCap @@ -12411,8 +12518,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw for (int32 Level = 0; Level < 9; ++Level) { AdmissionCapByLevel[Level] = bUseCoarseFirstOrder - ? (Level >= FirstCoarseLevel ? CoarseTaskCap : MaxTasks) - : (Level == 0 ? MaxTasks : CoarseTaskCap); + ? (Level >= FirstCoarseLevel || Level > 0 ? CoarseTaskCap : NearAdmissionCap) + : (Level == 0 ? NearAdmissionCap : CoarseTaskCap); if (DesiredMissingByLevel[Level] <= 0) continue; if (bAllChunksLoaded) @@ -12426,7 +12533,7 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw { const int32 DeclinedAtCap = AdmissionCapByLevel[Level] > 0 ? AdmissionCapByLevel[Level] : SchedulerOrderStopCap; - if (DeclinedAtCap < MaxTasks) + if (DeclinedAtCap < TotalAdmissionCap) { DeclinedRuleByLevel[Level] = 2; // Near reserve reached during this submit pass. // Reserve proche atteinte pendant cette passe d'admission. @@ -12439,14 +12546,14 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw FullCapDeclinedTiles += DesiredMissingByLevel[Level]; } } - else if (AdmissionOccupancy >= MaxTasks) + else if (AdmissionOccupancy >= TotalAdmissionCap) { DeclinedRuleByLevel[Level] = 1; // Full task admission cap. // Capacite totale d'admission des taches atteinte. FullCapDeclinedTiles += DesiredMissingByLevel[Level]; } else if (AdmissionOccupancy >= AdmissionCapByLevel[Level] - && AdmissionCapByLevel[Level] < MaxTasks) + && AdmissionCapByLevel[Level] < TotalAdmissionCap) { DeclinedRuleByLevel[Level] = 2; // Authored near reserve held back this LOD. // La reserve proche configuree retient ce niveau de detail. @@ -12497,7 +12604,9 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw "\"core_region_compile_pending\":%d,\"core_ready\":%d,\"coarse_first\":%d," "\"dynamic_near_reserve\":%d,\"legacy_pending_result_cap\":%d," "\"core_region_concurrent_admission\":%d," - "\"max_tasks\":%d,\"near_reserve\":%d," + "\"max_tasks\":%d,\"ready_queue_depth\":%d," + "\"total_admission_cap\":%d,\"near_admission_cap\":%d," + "\"near_reserve\":%d," "\"coarse_cap\":%d,\"first_coarse_level\":%d," "\"submission_stop_level\":%d,\"submission_stop_cap\":%d," "\"pending_entry\":%d,\"pending_after\":%d,\"admission_occupancy\":%d," @@ -12520,8 +12629,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw bCoreRegionCompilePending ? 1 : 0, IsVoxelForgeCoreReady() ? 1 : 0, GVoxelForgeStreamingCoarseFirst, GVoxelForgeStreamingDynamicNearReserve, GVoxelForgeStreamingLegacyPendingResultCap, - GVoxelForgeCoreRegionConcurrentAdmission, MaxTasks, - NearTaskReserve, CoarseTaskCap, FirstCoarseLevel, + GVoxelForgeCoreRegionConcurrentAdmission, MaxTasks, ReadyQueueDepth, + TotalAdmissionCap, NearAdmissionCap, NearTaskReserve, CoarseTaskCap, FirstCoarseLevel, SchedulerOrderStopLevel, SchedulerOrderStopCap, PendingTilesAtUpdateEntry, PendingTiles.Num(), AdmissionOccupancy, ActiveTaskGuard, ActiveGenerationWorkerCount.load(std::memory_order_relaxed), PendingResultsEstimate, @@ -12572,14 +12681,19 @@ void AVoxelWorld::LoadTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority Wor return; } - const int32 MaxTasks = GetMaxConcurrentTasks(); // T2.d — core-clamped + const int32 MaxTasks = GetMaxConcurrentStreamingTasks(); // Core and worker lane clamped. / Core et voie worker plafonnes. + const int32 ReadyQueueDepth = FMath::Clamp( + GVoxelForgeStreamingReadyQueueDepth, 0, MaxTasks); + const int32 TotalAdmissionCap = MaxTasks + ReadyQueueDepth; const bool bNearPriority = WorkPriority != EVoxelTileWorkPriority::Horizon; const int32 NearReserve = EffectiveNearTaskReserve >= 0 ? FMath::Clamp(EffectiveNearTaskReserve, 0, FMath::Max(0, MaxTasks - 1)) : GetNearTaskReserve(MaxTasks); - const int32 CoarseTaskCap = FMath::Max(1, MaxTasks - NearReserve); + const int32 NearAdmissionCap = bStreamingPlayerMoving ? MaxTasks : TotalAdmissionCap; + const int32 CoarseTaskCap = FMath::Max(1, MaxTasks - NearReserve) + ReadyQueueDepth; const int32 AdmissionCap = WorkPriority == EVoxelTileWorkPriority::Horizon - ? CoarseTaskCap : MaxTasks; + ? CoarseTaskCap + : (WorkPriority == EVoxelTileWorkPriority::LOD0 ? NearAdmissionCap : TotalAdmissionCap); if (GetGenerationAdmissionCount() >= AdmissionCap) { if (GVoxelForgeProfileTileGeneration != 0 @@ -12716,14 +12830,8 @@ void AVoxelWorld::LoadTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority Wor ActiveTaskCount.fetch_add(1, std::memory_order_relaxed); ActiveTileTaskCount.fetch_add(1, std::memory_order_relaxed); - // BackgroundNormal priority: coarse gen runs on background workers that YIELD to foreground - // (game/render-thread) tasks. Without this, raising MaxConcurrentTasks past the spare - // core count saturates the scheduler and starves the frame (the "over 12 = lag" symptom). - // At background priority the frame keeps its cores; gen just fills in around it. - // NEAR RESPONSIVENESS: collision support, edits, and every level-0 tile launch at - // BackgroundHigh — still a background worker (yields to the frame, keeps the invariant) but - // jumps AHEAD of ordinary pending streaming gen. Admission reserves slots for this class and - // the result queues apply it in collision -> edit -> LOD0 order. + // BackgroundNormal yields to foreground work; near tasks stay ahead at BackgroundHigh. + // BackgroundNormal cede la priorite au premier plan ; les tuiles proches restent prioritaires en BackgroundHigh. const UE::Tasks::ETaskPriority TaskPriority = bNearPriority ? UE::Tasks::ETaskPriority::BackgroundHigh : UE::Tasks::ETaskPriority::BackgroundNormal; @@ -12747,6 +12855,7 @@ void AVoxelWorld::LoadTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority Wor } Guard{ActiveTaskCount, ActiveTileTaskCount}; ActiveGenerationWorkerCount.fetch_add(1, std::memory_order_relaxed); const uint64 WorkerTaskStartCycles = FPlatformTime::Cycles64(); + const uint32 WorkerThreadId = FPlatformTLS::GetCurrentThreadId(); struct FWorkerGuard { std::atomic& Counter; @@ -12765,6 +12874,7 @@ void AVoxelWorld::LoadTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority Wor FChunkResult Result; Result.WorkerTaskStartCycles = WorkerTaskStartCycles; + Result.WorkerThreadId = WorkerThreadId; Result.RequestOrdinal = RequestOrdinal; Result.bResidentAtRequest = bResidentAtRequest; Result.WorkPriority = WorkPriority; @@ -12786,6 +12896,7 @@ void AVoxelWorld::LoadTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority Wor Canceled.GenerationOrdinal = Result.GenerationOrdinal; Canceled.WorkerTaskStartCycles = Result.WorkerTaskStartCycles; Canceled.WorkerTaskEndCycles = FPlatformTime::Cycles64(); + Canceled.WorkerThreadId = Result.WorkerThreadId; Canceled.GenerationStartCycles = Result.GenerationStartCycles; Canceled.GenerationEndCycles = FPlatformTime::Cycles64(); Canceled.bResidentAtRequest = bResidentAtRequest; @@ -12887,14 +12998,19 @@ void AVoxelWorld::LoadCoreTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority return; } - const int32 MaxTasks = GetMaxConcurrentTasks(); + const int32 MaxTasks = GetMaxConcurrentStreamingTasks(); + const int32 ReadyQueueDepth = FMath::Clamp( + GVoxelForgeStreamingReadyQueueDepth, 0, MaxTasks); + const int32 TotalAdmissionCap = MaxTasks + ReadyQueueDepth; const bool bNearPriority = WorkPriority != EVoxelTileWorkPriority::Horizon; const int32 NearReserve = EffectiveNearTaskReserve >= 0 ? FMath::Clamp(EffectiveNearTaskReserve, 0, FMath::Max(0, MaxTasks - 1)) : GetNearTaskReserve(MaxTasks); - const int32 CoarseTaskCap = FMath::Max(1, MaxTasks - NearReserve); + const int32 NearAdmissionCap = bStreamingPlayerMoving ? MaxTasks : TotalAdmissionCap; + const int32 CoarseTaskCap = FMath::Max(1, MaxTasks - NearReserve) + ReadyQueueDepth; const int32 AdmissionCap = WorkPriority == EVoxelTileWorkPriority::Horizon - ? CoarseTaskCap : MaxTasks; + ? CoarseTaskCap + : (WorkPriority == EVoxelTileWorkPriority::LOD0 ? NearAdmissionCap : TotalAdmissionCap); if (GetGenerationAdmissionCount() >= AdmissionCap) return; PendingTiles.Add(Tile); @@ -12947,6 +13063,7 @@ void AVoxelWorld::LoadCoreTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority } Guard{ActiveTaskCount, ActiveTileTaskCount}; ActiveGenerationWorkerCount.fetch_add(1, std::memory_order_relaxed); const uint64 WorkerTaskStartCycles = FPlatformTime::Cycles64(); + const uint32 WorkerThreadId = FPlatformTLS::GetCurrentThreadId(); struct FWorkerGuard { std::atomic& Counter; @@ -12978,6 +13095,7 @@ void AVoxelWorld::LoadCoreTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority FChunkResult Result; Result.WorkerTaskStartCycles = WorkerTaskStartCycles; + Result.WorkerThreadId = WorkerThreadId; Result.Tile = Tile; Result.Epoch = TaskEpoch; Result.DesiredEpoch = TaskDesiredEpoch; @@ -13000,6 +13118,7 @@ void AVoxelWorld::LoadCoreTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority Canceled.RequestOrdinal = RequestOrdinal; Canceled.GenerationOrdinal = Result.GenerationOrdinal; Canceled.WorkerTaskStartCycles = Result.WorkerTaskStartCycles; + Canceled.WorkerThreadId = Result.WorkerThreadId; Canceled.bResidentAtRequest = bResidentAtRequest; Canceled.GenerationStartCycles = Result.GenerationStartCycles; Canceled.GenerationEndCycles = FPlatformTime::Cycles64(); diff --git a/Source/VoxelForge/Public/VoxelStartupTrace.h b/Source/VoxelForge/Public/VoxelStartupTrace.h index 92a6a7f..191a834 100644 --- a/Source/VoxelForge/Public/VoxelStartupTrace.h +++ b/Source/VoxelForge/Public/VoxelStartupTrace.h @@ -17,6 +17,10 @@ namespace VoxelForgeStartupTrace int32 Verdict = -1; // Mixed=0, AllSolid=1, AllAir=2, -1=not classified (early path) uint64 RequestOrdinal = 0; uint64 GenerationOrdinal = 0; + uint64 RequestStartCycles = 0; + uint64 WorkerTaskStartCycles = 0; + uint64 WorkerTaskEndCycles = 0; + uint32 WorkerThreadId = 0; bool bResidentAtRequest = false; bool bSealedSolidProof = false; bool bEmpty = true; diff --git a/Source/VoxelForge/Public/VoxelWorld.h b/Source/VoxelForge/Public/VoxelWorld.h index edc4021..a3f3c43 100644 --- a/Source/VoxelForge/Public/VoxelWorld.h +++ b/Source/VoxelForge/Public/VoxelWorld.h @@ -137,6 +137,9 @@ struct FChunkResult // Les bornes de tâche incluent la conversion et la préparation du résultat autour du générateur. uint64 WorkerTaskStartCycles = 0; uint64 WorkerTaskEndCycles = 0; + // Operating-system thread id identifies the physical worker slot used by this tile. + // L'identifiant du thread systeme repere le slot worker utilise par cette tuile. + uint32 WorkerThreadId = 0; // Worker timing boundaries for separating scheduler wait from generation and result/apply wait. // These are captured with low overhead for the clean streaming-latency summary; expensive // per-tile profiling remains opt-in. @@ -443,6 +446,7 @@ public: * capped to (logical cores − 2) so small CPUs don't thrash on a flat 16 (background * priority stops frame starvation, not the context-switch overhead). */ int32 GetMaxConcurrentTasks() const; + int32 GetMaxConcurrentStreamingTasks() const; /** Submitted generation tasks still queued or running; completed MPSC results do not occupy admission. * Taches de generation soumises, en file ou en cours ; les resultats MPSC termines ne bloquent pas l'admission. */ int32 GetActiveGenerationTaskCount() const; @@ -1239,6 +1243,7 @@ public: int32 ObsoleteTileAbortCount = 0; // cumulative per-world moving-session cancellation count float PeakObservedPawnSpeedCmPerSecond = 0.0f; int32 EffectiveNearTaskReserve = -1; + bool bStreamingPlayerMoving = false; // GT snapshot consumed by admission helpers / instantane GT pour les admissions uint64 NextStreamingRequestOrdinal = 0; std::atomic NextStreamingGenerationOrdinal{0}; // Cumulative worker-task wall cycles avoid aliasing the instantaneous per-frame counter. diff --git a/Standalone/Research/WorldAssemblerM3/Round16/REPORT.md b/Standalone/Research/WorldAssemblerM3/Round16/REPORT.md new file mode 100644 index 0000000..4ac3487 --- /dev/null +++ b/Standalone/Research/WorldAssemblerM3/Round16/REPORT.md @@ -0,0 +1,302 @@ +# World Assembler S12 — Round 16 + +**2026-10-19 · worker supply and tick-independent task pickup** + +## Result + +The matched 25 cm SurfaceWorld run completed its 2,578-tile set in **4.911 s** on the actor startup +clock, down from Round15's **5.960 s**. From the first tile submission to readiness it took **4.537 s**, +down from **5.603 s**. Full LOD0's last apply moved from 1.403 s to **1.087 s**. The stationary +walkable ring measured **0.509 s** (Round15: 0.488 s). The 40 m/s trace kept the 3×3 collision-ready +ring at 9/9 for every moving gate sample. + +The measured ceiling was eight tile workers, not ten: with 12 CPUs in the process affinity, Unreal +reported 10 TaskGraph workers split into two foreground and eight background workers. Tile tasks +use background priorities, so I sized their admission slots to the eight background workers. A +short, ordered TaskGraph queue keeps those workers supplied between actor ticks. No density, +triangulation, collision, or Core recipe output changed; recipe versions stay unchanged. + +## Pool count and priority + +| Measurement | BuildHost result | +|---|---:| +| Process affinity / priority | `0xFFF` (12 CPUs) / Normal | +| `UE::Tasks` scheduler workers / reported maximum | 10 / 20 | +| `FTaskGraphInterface` workers | 10 total: 2 foreground, 8 background | +| `GThreadPool` workers | 10; separate from TaskGraph | +| Step1 cook pool | 0 threads | +| Tile task API / priority | `UE::Tasks::Launch`; near `BackgroundHigh`, horizon `BackgroundNormal` | +| Scheduler worker / background priority fields | 5 / 2 in `worker_backend_config` | +| Unique tile worker thread IDs / peak simultaneous tile tasks | 8 / 8 | + +RMC's inspected async collision path calls `DoOnAllowedThread(AsyncThread)`; that implementation +queues work on `GThreadPool` (`RealtimeMeshFuture.h:147–165`, +`RealtimeMeshManaged.cpp:703–707`). It therefore does not consume the eight TaskGraph background +worker IDs used by tile generation. Other UE background work can share TaskGraph's workers; this +trace attributes tile work by OS thread ID but does not label every unrelated engine task. The +external process sampler counted 40 OS threads after initialization and 58–60 while running, +including engine service threads. The 12-CPU affinity limits simultaneous CPU execution; it is not +a 12-OS-thread process cap. VoxelForge tile generation itself peaked at eight workers, and the +BuildHost process priority stayed Normal. + +## Shared scheduler and trace + +Two read-back CVars control the shipped behavior: + +| CVar | Default | Negative control | Effect | +|---|---:|---:|---| +| `voxel.StreamingReadyQueueDepth` | 8 | 0 | Admit up to one extra worker-set of prioritized tasks | +| `voxel.StreamingBackgroundWorkerCap` | 1 | 0 | Size streaming slots to TaskGraph background workers; 0 restores the logical-core cap | + +With the selected defaults, effective tile slots are 8 and the total admission cap is 16. Near +tasks retain `BackgroundHigh`; horizon tasks retain `BackgroundNormal`. While moving, the dynamic +near reserve is 7, the near admission cap is 8, and the coarse cap is 9. Collision look-ahead and +critical/edit admission retain their existing precedence. The game-thread scheduler continues to +select and submit from the desired-tile ordering; there is no worker-owned continuation list. UE +dispatches an already-admitted task as a background worker becomes free. `bStreamingPlayerMoving` is +a game-thread snapshot shared by `LoadTile` and `LoadCoreTile`. + +Queued tasks retain `GenerationEpoch`, `DesiredEpoch`, and per-request cancellation checks. A desired +set change obsoletes stale queued work, workers still check `bShuttingDown`, and results still return +through the MPSC queues for game-thread application. No UObject or RMC component mutation moved to a +worker. The trace records CVar readbacks in `worker_backend_config` and +`streaming_worker_cap_readback`, and records each tile's submission, worker start/end, thread ID, +next scheduler pass, next submission, and next task start on that worker. + +### Initial-set timeline + +The following busy-worker equivalents integrate the 2,578 tile intervals in 500 ms bins from the +first submission (`submitted_t_s=0.456843` on the trace clock). The final short bin is 10.7 ms. + +| Time from first submission | Busy worker equivalents, out of 8 | +|---|---:| +| 0–0.5 s | 6.16 | +| 0.5–1.0 s | 5.94 | +| 1.0–1.5 s | 4.82 | +| 1.5–2.0 s | 4.38 | +| 2.0–2.5 s | 7.83 | +| 2.5–3.0 s | 7.98 | +| 3.0–3.5 s | 6.58 | +| 3.5–4.0 s | 7.47 | +| 4.0–4.5 s | 7.67 | +| 4.5–4.511 s | 1.75 | + +All eight worker IDs were used. The initial set accumulated **29.432 worker-seconds**; the first +worker started at 0.456872 and the last ended at 4.967520 on the trace clock, a 4.511 s worker +span. That workload has a 3.679 s ideal at eight fully occupied slots (2.943 s at ten); the +measured worker span was 4.511 s. Per-slot idle gaps between one tile ending and that worker's next tile starting summed to +**6.573 slot-seconds** over 2,570 gaps: p50 0.006 ms, p95 13.604 ms, max 16.800 ms. + +| Worker thread ID | Tiles | Inter-task idle seconds | Gap p95 / max | +|---:|---:|---:|---:| +| 20144 | 324 | 0.838 | 13.780 / 15.607 ms | +| 21468 | 318 | 0.829 | 13.489 / 16.800 ms | +| 21536 | 304 | 0.830 | 13.237 / 15.902 ms | +| 23392 | 327 | 0.832 | 13.476 / 15.691 ms | +| 23572 | 331 | 0.821 | 13.365 / 15.916 ms | +| 23616 | 331 | 0.847 | 13.765 / 15.773 ms | +| 2716 | 321 | 0.734 | 13.486 / 15.801 ms | +| 5392 | 322 | 0.843 | 13.657 / 15.556 ms | + +The scheduler recorded 247 admission passes, 243 with submissions, averaging 10.61 tiles per +non-empty pass. Peak initial scheduler backlog was 32; the whole trace's result-backlog peak was +35 tiles / 1,620,010 bytes against the 256-tile cap, with zero cap stops. The completed-result +queue itself peaked at five during initial readiness. + +`Saved/BuildHost/Round16/initial_tile_worker_timeline.csv` contains one row per initial tile, +including submitted/start/end, next scheduler admission, next submission, same-thread next start, +and the calculated idle gap. The 500 ms busy bins, per-LOD totals, and per-thread idle table are +also saved there. Files larger than 300 KB remain under `Saved/BuildHost/`. + +### Per-LOD completion and work + +S12 worker-end/apply values below are normalized to the actor startup clock; S11 apply values are +from the Round15 matched report. S12 worker task totals were higher in this sample even though the +set finished sooner; the scheduler changes do not change tile contents. + +| LOD | Tiles | Worker seconds S11 → S12 | S11 last apply | S12 last worker / last apply | +|---:|---:|---:|---:|---:| +| 0 | 512 | 3.725 → 4.018 | 1.403 s | 1.084 / 1.087 s | +| 1 | 448 | 3.895 → 4.099 | 2.283 s | 1.749 / 1.757 s | +| 2 | 320 | 0.806 → 0.825 | 2.849 s | 2.107 / 2.122 s | +| 3 | 272 | 2.064 → 2.296 | 3.330 s | 2.469 / 2.484 s | +| 4 | 272 | 6.590 → 7.682 | 4.276 s | 3.431 / 3.440 s | +| 5 | 208 | 1.528 → 1.732 | 4.660 s | 3.706 / 3.709 s | +| 6 | 144 | 2.077 → 2.160 | 5.060 s | 4.048 / 4.062 s | +| 7 | 96 | 1.928 → 2.044 | 5.338 s | 4.289 / 4.293 s | +| 8 | 306 | 4.171 → 4.576 | 6.019 s | 4.885 / 4.894 s | + +LOD4 remains the largest worker-cost group: 7.682 s across 272 tiles, or 28.24 ms per tile. +The tile trace attributes this to the overall generation task (`generation_s`); it does not break +the worker interval into density/classification/meshing stages (`classify_s` and `mesh_s` are not +populated for these rows). Its last apply was 3.440 s, so it did not set the tail. I made no LOD4 +content optimization without the requested stage attribution and quality-capture work. + +## Apply, tick, and motion results + +| Metric | Round15 S11 | Round16 S12 | +|---|---:|---:| +| Full set ready, actor startup clock | 5.960 s | 4.911 s | +| First submission to full ready | 5.603 s | 4.537 s | +| Worker span from first submission | 5.581 s | 4.511 s | +| Worker-seconds | 26.786 s | 29.432 s | +| Walkable ring, 9/9 | 0.488 s | 0.509 s | +| Full LOD0 last apply | 1.403 s | 1.087 s | +| Apply frames through readiness | 332 | 268 | +| Apply-count limit frames | 0 | 11 | +| Max visible mesh applies / frame | 10 | 12 | +| Total measured apply time | 236.14 ms | 255.17 ms | +| Max apply time / frame | 2.357 ms | 2.697 ms | +| Result-wait p95 / max | 16.62 / 19.26 ms | 16.90 / 21.03 ms | +| Result backlog peak / cap | 10 / 256 | 35 / 256 | + +`UWorld::Tick` spawn and normal-walk windows were measured in fresh processes: + +| Window | S11 p95 / max | S12 p95 / max | Change | +|---|---:|---:|---:| +| Spawn | 2.740 / 5.175 ms | 3.344 / 5.219 ms | +0.604 / +0.044 ms | +| Normal walk | 2.483 / 28.876 ms | 2.744 / 28.752 ms | +0.261 / −0.124 ms | + +The maxima stayed effectively flat or lower, while p95 rose by 0.60 ms at spawn and 0.26 ms while +walking. I record this as a small p95 increase rather than claiming strict p95 non-regression. Both +candidate p95 values remained below 3.4 ms. + +The 40 m/s run used the same queue and slot defaults. It completed the initial set in 4.919 s; +all **75/75** moving `walkable_gate` events reported 9/9, and all 58 moving streaming samples +reported the ring ready. Spawn tick p95/max was 3.173/8.071 ms; the accelerated walk window was +29.562/39.227 ms. The 40 m/s trace's collision look-ahead, dynamic reserve, and moving near cap +readbacks are present in the JSONL. + +### Negative controls and rejected variants + +| Read-back settings | Effective slots / actual worker IDs | Full set ready | Result | +|---|---:|---:|---| +| Queue depth 0, worker cap 1 | 8 / 8 | 7.024 s | Disabling the ready queue removes the speedup | +| Queue depth 8, worker cap 0 | 10 / 8 | 4.669 s | Restores logical-core admission; still only eight background IDs ran | +| Queue depth 8, worker cap 1 | 8 / 8 | 4.911 s | Selected default, sized to actual background workers | + +The first 40 m/s queue prototype lacked the queue-aware moving reserve and LOD0 admission cap; it +missed the ring in 56/77 gate events and 47/57 streaming samples. Adding the reserve alone was +insufficient. The final moving near cap and reserve passed 75/75 gate events. Reducing the shared +Core task cap itself to eight increased per-tile generation cost and made the full set 5.30 s, so +that change was reverted; only streaming admission is background-worker-capped. + +Foreground-priority variants were also rejected. Raising all tile work to foreground priorities +used ten worker IDs but produced 39.422/50.432 ms spawn and 49.137/83.184 ms walk p95/max, with a +5.215 s set. Raising only near work produced an 86.225 ms walk maximum. The final build keeps the +original background priorities. + +## Determinism and quality gates + +- Standalone W1/W8/W12 all passed with identical full-clipmap hashes: density + `c10697a8f48380c7`, mesh `e75c9119a1498cd5`, payload `49a9a8d1835a6c8c`, topology + `edcb23288b5e8724`. +- BuildHost SurfaceWorld W1/W8/W16 passed with stable density/mesh/payload/topology hashes + `0cd2a1a4152ee4aa / a06243aee16b65d5 / c700162935c262d7 / 0af901aa42b452bc`. +- BuildHost FloatingIslands passed at 25 cm; W1/W8/W16 were stable with hashes + `4dcfb37585d7d986 / f14f76fbf065c6ed / 53035c45fd1a082e / 23550100654a4ec3`. +- Core-off legacy gate passed at 12.5 cm (`legacy_unchanged=PASS`). M2 Hybrid V1 and V2 analytic + fixture and border suites passed at 25 cm. +- The current Standalone far-cap audit passed S4 agreement: 128 shared points, maximum far/near + difference 0.0504 cm against a 2 cm tolerance; the 100 cm negative control was detected. Coverage + was 2520/2520 (100%), with zero missing mesh/profile crossings. S3's prior 72-point audit remains + at 0.0427 cm; scheduler changes do not touch that Core content path. +- The 25 cm SurfaceWorld and FloatingIslands gates both passed. No world recipe version changed. + +## Builds, commands, and hashes + +Builds completed: + +```powershell +.\Tools\BuildVoxelForgeCoreHost.ps1 -MaxParallelActions 12 -TimeoutSeconds 900 +.\Standalone\Build.ps1 -Target All -CompileWorkers 12 -BuildPriority Normal +``` + +The matched SurfaceWorld trace command was: + +```powershell +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -StreamingCollisionLookahead 1 -TestMoveDirection '0,1,0' -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 600 -StreamingReadyQueueDepth 8 -StreamingBackgroundWorkerCap 1 -TimeoutSeconds 900 +``` + +The 40 m/s and negative-control commands were: + +```powershell +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -StreamingCollisionLookahead 1 -TestMoveDirection '0,1,0' -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 4000 -StreamingReadyQueueDepth 8 -StreamingBackgroundWorkerCap 1 -TimeoutSeconds 900 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -StreamingCollisionLookahead 1 -TestMoveDirection '0,1,0' -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 600 -StreamingReadyQueueDepth 0 -StreamingBackgroundWorkerCap 1 -TimeoutSeconds 900 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -StreamingCollisionLookahead 1 -TestMoveDirection '0,1,0' -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 600 -StreamingReadyQueueDepth 8 -StreamingBackgroundWorkerCap 0 -TimeoutSeconds 900 +``` + +Quality commands: + +```powershell +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -CoreOffTest -CoreVoxelSizeCm 12.5 -TimeoutSeconds 900 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -HybridFixturesOnly -CoreMesherRecipe WorldAssemblerHybridV1 -CoreVoxelSizeCm 25 -TimeoutSeconds 900 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -HybridFixturesOnly -CoreMesherRecipe WorldAssemblerHybridV2 -CoreVoxelSizeCm 25 -TimeoutSeconds 900 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -CoreVoxelSizeCm 25 -TimeoutSeconds 900 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -FloatingIslandsOnly -CoreVoxelSizeCm 25 -TimeoutSeconds 900 +``` + +The S3/S4 Standalone agreement and coverage audit command was: + +```powershell +.\Standalone\bin\VoxelForgeCoreStandalone.exe --world-assembler-audit --workers=12 --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --process-priority=Normal --process-affinity=4095 --frame-limit=60 > Saved/BuildHost/Round16/standalone_S3S4_audit.stdout.log 2> Saved/BuildHost/Round16/standalone_S3S4_audit.stderr.log +``` + +Standalone determinism ran serially at Normal priority, affinity `4095`, and frame limit 60. Each +worker count used the same remaining arguments and output paths below: + +```powershell +.\Standalone\bin\VoxelForgeCoreStandalone.exe --workers=1 --world-assembler-prototype --stream-benchmark --stream-hash --force-generate --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --far-horizon-lod8-step=0 --process-priority=Normal --process-affinity=4095 --frame-limit=60 > Saved/BuildHost/Round16/standalone_W1.stdout.log 2> Saved/BuildHost/Round16/standalone_W1.stderr.log +.\Standalone\bin\VoxelForgeCoreStandalone.exe --workers=8 --world-assembler-prototype --stream-benchmark --stream-hash --force-generate --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --far-horizon-lod8-step=0 --process-priority=Normal --process-affinity=4095 --frame-limit=60 > Saved/BuildHost/Round16/standalone_W8.stdout.log 2> Saved/BuildHost/Round16/standalone_W8.stderr.log +.\Standalone\bin\VoxelForgeCoreStandalone.exe --workers=12 --world-assembler-prototype --stream-benchmark --stream-hash --force-generate --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --far-horizon-lod8-step=0 --process-priority=Normal --process-affinity=4095 --frame-limit=60 > Saved/BuildHost/Round16/standalone_W12.stdout.log 2> Saved/BuildHost/Round16/standalone_W12.stderr.log +``` + +SHA-256 records. Streaming and quality runs used the first VoxelForge module hash. After those runs, +I added the French half of one inline comment and rebuilt the current tree; that comment-only rebuild +produced the second module hash. The executable code did not change between those two hashes. + +| Binary | SHA-256 | +|---|---| +| BuildHost `UnrealEditor-VoxelForge.dll` used for timing and gates | `C9D5DA90B48C5C1F844221C4A6A195947016C522F6A296EBC488A1AD8CCC6EEF` | +| Current rebuilt BuildHost `UnrealEditor-VoxelForge.dll` | `12E7883DFB8825012890F1A2B5BACD038AA40FD1C342AA70711EC55AABF9057F` | +| BuildHost `UnrealEditor-VoxelForgeCore.dll` | `D87CC56BACCDE6559705EEAEA7DB73D1A6ABF79484D37A3887B94A9FD62BABD3` | +| BuildHost `UnrealEditor-VoxelForgeEditor.dll` | `805FB845F00D013B48E29D4FD5D56EF1531E43F68047AE499DC617B04864DF80` | +| Standalone `VoxelForgeCoreStandalone.exe` | `4EBE946E4D7B245458CD4FD0FBC3FCD6988B98000685C0043B25CD629AFC0F30` | + +## Failed attempts and safety review + +- One intermediate BuildHost compile failed because `PlayerHeading` was out of scope in two loader + admission helpers. I replaced those reads with the game-thread `bStreamingPlayerMoving` snapshot; + the final BuildHost and Standalone builds passed. +- Two early external-sampler attempts failed before launching Unreal (a PowerShell job argument + was flattened, then `Get-Counter` requested an unsupported counter). The later sampler ran. No + Unreal run failed; no `0xC0000142` occurred and no retry was needed. +- The final SurfaceWorld command completed under 15 minutes with 3.75 GB peak private memory; + FloatingIslands completed with 3.26 GB. The Windows free-memory query was denied during the latter + launch, but the BuildHost runner's preflight guard allowed it. No Unreal process remains running. +- Standalone compiled with existing warnings: `vswhere.exe` was not found on `PATH`, and + `VoxelStrateManager.cpp` emitted C4838 double-to-float conversion warnings. The `All` build + completed successfully. +- `Saved/BuildHost/HostProject/Saved/Crashes/` contains only the pre-existing + `UECC-Windows-F1A3EFD5469F5DD62E68EBBACAC84376_0000`; no new crash folder appeared. +- Diff review confirmed the result queues remain `EQueueMode::Mpsc`; `EndPlay` waits for active + tasks and drains queues; workers check `bShuttingDown`; epoch and cancellation checks remain; + `ModsLock` still guards mod data; core pages retain shared ownership through readers; and UObject + / RMC component mutation remains on the game thread. No queue mode or page ownership changed. +- No source file was added. C++ compile spots are `Source/VoxelForge/Private/VoxelWorld.cpp`, + `Source/VoxelForge/Public/VoxelWorld.h`, `Source/VoxelForge/Private/VoxelStartupTrace.cpp`, and + `Source/VoxelForge/Public/VoxelStartupTrace.h` in the BuildHost VoxelForge module. The test-harness + change is PowerShell in `Tools/TestVoxelForgeCoreWorldHost.ps1`; no Core source, generated header, + `.uasset`, owner `.uproject`, or editor was modified or built. +- No commit, push, or stash was created. RealtimeMeshComponent was read-only. + +### Dialogue + +**User:** Count the workers and their idle gaps, preserve the streaming priorities and safety rules, +and get the full set under six seconds without losing the 40 m/s walkable ring. + +**Codex:** The actor had ten TaskGraph workers available but only eight background workers for its +background-priority tile tasks. A queue of eight plus an eight-slot admission cap reduced the +2,578-tile startup to 4.911 s; the moving 3×3 ring stayed ready in all 75 gate samples. The process +remains Normal priority and limited to 12 CPUs. The measured p95 tick increase is small but recorded, +and the worker pool shows why this round did not reach the 3 s ideal for ten workers. diff --git a/Tools/TestVoxelForgeCoreWorldHost.ps1 b/Tools/TestVoxelForgeCoreWorldHost.ps1 index 53cb3c5..7f4a3e0 100644 --- a/Tools/TestVoxelForgeCoreWorldHost.ps1 +++ b/Tools/TestVoxelForgeCoreWorldHost.ps1 @@ -13,6 +13,8 @@ param( [string]$TestMoveDirection = '0,1,0', [ValidateRange(0, 1)][int]$CoreRegionConcurrentAdmission = 1, [ValidateRange(0, 4096)][int]$StreamingResultBacklogCap = 256, + [ValidateRange(0, 10)][int]$StreamingReadyQueueDepth = 8, + [ValidateRange(0, 1)][int]$StreamingBackgroundWorkerCap = 1, [ValidateRange(0, 900)][int]$Step1MaxMeshApplies = 0, [ValidateRange(0, 900)][int]$TestTeleportAfterSeconds = 0, [ValidateRange(0, 1000000)][int]$TestTeleportDistanceCm = 0, @@ -64,6 +66,9 @@ $RunName = if ($FloatingIslandsOnly) { "floating_islands$CoreSizeSuffix" } $RunName += $(if ($StreamingTraceOnly) { '_stream_trace_only' } else { '' }) $StartupApplyBoostLabel = if ($StartupApplyBoost -lt 0) { 'default' } else { [string]$StartupApplyBoost } $RunName += "_applyboost_$StartupApplyBoostLabel" +if ($StreamingTraceOnly) { + $RunName += "_bgcap_${StreamingBackgroundWorkerCap}_readyq_${StreamingReadyQueueDepth}" +} if ($MismatchedManifestFallback) { $CoreFallbackTest = $true } $SelectedModeFlags = @( @@ -95,6 +100,7 @@ if ($MoveDirectionComponents[0] -eq 0 -and $MoveDirectionComponents[1] -eq 0 -an if (-not $StreamingTraceOnly -and ($TestMoveSpeedCmPerSecond -ne 600 -or $StreamingCollisionLookahead -ne 1 -or $TestMoveDirection -ne '0,1,0' -or $CoreRegionConcurrentAdmission -ne 1 -or $StreamingResultBacklogCap -ne 256 -or + $StreamingReadyQueueDepth -ne 8 -or $StreamingBackgroundWorkerCap -ne 1 -or $Step1MaxMeshApplies -ne 0 -or $TestTeleportAfterSeconds -ne 0 -or $TestTeleportDistanceCm -ne 0)) { throw 'Streaming, backlog, apply and teleport controls require -StreamingTraceOnly.' @@ -320,6 +326,8 @@ if ($StartupApplyBoost -ge 0) { } $BaseArguments += "-voxel.CoreRegionConcurrentAdmission=$CoreRegionConcurrentAdmission" $BaseArguments += "-voxel.StreamingResultBacklogCap=$StreamingResultBacklogCap" +$BaseArguments += "-voxel.StreamingReadyQueueDepth=$StreamingReadyQueueDepth" +$BaseArguments += "-voxel.StreamingBackgroundWorkerCap=$StreamingBackgroundWorkerCap" if ($Step1MaxMeshApplies -gt 0) { $BaseArguments += "-voxel.Step1MaxMeshApplies=$Step1MaxMeshApplies" } @@ -339,6 +347,7 @@ if ($StartupTrace -or $StreamingTraceOnly) { $TracePath = Join-Path $LogDirectory "CoreWorldValidation-$RunName.trace.jsonl" $BaseArguments += ('-voxel.StartupTraceFile="{0}"' -f $TracePath) $BaseArguments += '-voxel.StartupTraceUntilEndPlay=1' + $BaseArguments += '-voxel.StartupTraceMaxTiles=16384' } if ($SharedFeatureAuditOnly) { $BaseArguments += '-corefeatureauditonly' } if ($HasManifestOverride) {