wip(assembler): round S8 -- region rebuilds no longer stall admission; result backlog capped

voxel.CoreRegionConcurrentAdmission (default on): tiles independent of the
entering region keep submitting while a region page compiles; pages are
immutable shared_ptr owners so running readers survive eviction. 6 m/s: 11
whole-pass skips -> 0, walkable ring ready at every centre change.
voxel.StreamingResultBacklogCap=256 bounds finished results (slow-apply worst
case 125 -> 76 MB). Spawn unchanged at ~7.7 s: LOD8 is 46% of worker time.
At 60 m/s + teleport the collision ring is outrun (102/113 centres).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MmSoF64BbRFmk9C3p1VsBJ
This commit is contained in:
Fr0zkaandClaude Opus 5.5 committed 2026-10-19 09:43:03 +02:00
1 parent 5eb9ee76ff
commit 8440adc742
10 files changed
+1011 -78

No files matched your search

+44
View File
@@ -584,6 +584,13 @@ driven by `EditorBrush*` props.
admission count. `voxel.StreamingDynamicNearReserve=1` holds the authored near reserve while moving admission count. `voxel.StreamingDynamicNearReserve=1` holds the authored near reserve while moving
or before the walkable gate, then releases it to coarse work after the stationary ring is ready. or before the walkable gate, then releases it to coarse work after the stationary ring is ready.
Stationary player ≈ free. (Old per-frame O(loaded×desired) scan = 22ms.) Stationary player ≈ free. (Old per-frame O(loaded×desired) scan = 22ms.)
Core region compilation is a separate resource gate. With the default
`voxel.CoreRegionConcurrentAdmission=1`, tile keys that need the pending region are withheld, while
independent keys may use remaining worker slots. `0` restores the all-admission wait. Region pages
are built off the lookup map, published as immutable shared pages, and reclaimed when the final
reader releases its shared owner. `voxel.StreamingResultBacklogCap=256` independently bounds queued
result payloads plus active tile tasks and held prequeue results; `0` disables this memory backstop.
Result count is not used as a worker-slot count.
- **LOD** changes HOT-SWAP (`LoadTile` only, never unload-first) → no holes. LOD - **LOD** changes HOT-SWAP (`LoadTile` only, never unload-first) → no holes. LOD
reconciliation lives in the PERSISTENT per-frame submit loop (same loop as new-chunk loads), reconciliation lives in the PERSISTENT per-frame submit loop (same loop as new-chunk loads),
NOT as a one-shot on the boundary-cross frame — a one-shot drops every chunk past the task NOT as a one-shot on the boundary-cross frame — a one-shot drops every chunk past the task
@@ -1565,6 +1572,43 @@ the recipe version remains unchanged. Null-RHI runs do not prove player-visible
[`Standalone/Research/WorldAssemblerM3/Round11/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round11/REPORT.md) [`Standalone/Research/WorldAssemblerM3/Round11/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round11/REPORT.md)
for the audit counts, RMC review, quality matrix, commands and binary hashes. for the audit counts, RMC review, quality matrix, commands and binary hashes.
### 8.29 Concurrent region admission and bounded result backlog — World Assembler S8 / M3 Round 12
The S7 moving trace found `bCoreRegionCompilePending` returning before all tile submissions. Each
such update could leave most slots unused while a new player center requested a region page. The
default `voxel.CoreRegionConcurrentAdmission=1` lets `LoadCoreTile` continue for keys outside the
pending and in-flight region sets. Keys that need those sets wait for publication; setting the CVar
to `0` restores the old all-admission gate for an A/B control. This changes scheduling only.
`World::compileTileRegion` builds region pages without mutating pages already in use, then publishes
them under the region-map lock. `generateTile` copies a `shared_ptr` while holding the map's shared
lock and keeps it for the whole mesh read. Evicting or replacing a map entry therefore cannot free a
page under an existing worker. The actor's shared `WorldLock` prevents a world edit/reset while a
tile reader or region compiler is active; `EndPlay` waits for the shared active-task count before
destroying the actor's UObject state.
The result backlog has its own backpressure control. `voxel.StreamingResultBacklogCap=256` counts
queued MPSC results, active tile-generation tasks and held visible-LOD0 prequeue results. Reaching
the cap stops new tile admissions until apply drains results or active tasks retire. It does not make
finished results consume worker slots in the normal admission count. `0` is the uncapped negative
control; the Core-off legacy test also exercises the default cap.
On the 25 cm owner SurfaceWorld plan, the matched 6 m/s CVar-off/on runs both completed 11 region
requests (one initial desired set plus ten center moves). The old gate recorded 11 global scheduler
skips; concurrent admission recorded none and submitted 110 unrelated tiles with no dependent-key
declines. All eleven changing support centers had the nine-tile walkable ring ready immediately.
Accumulated pending windows were about 0.43 s, but measured compiler wall time totaled about 11 ms.
At 60 m/s with a 400 m teleport, the trace recorded 113 region requests, 782 independent submissions
and 983 dependent-key declines. Most support centers advanced before their nine-tile ring could
become ready, so the high-speed gate still fails its travel-time target.
The shared result trace now records `queued_peak`, peak payload bytes, the configurable cap, and cap
stops. A slow-apply teleport stress run peaked at 256 results / 76.2 MB with the default cap; the
uncapped control reached 1,577 / 125.1 MB. Payload bytes exclude queue-node and allocator overhead.
The cap kept the observed backlog finite without reinstating the S6/S7 finished-result worker-slot
decline. Full timings, per-center records, quality checks and hashes are in
[`Standalone/Research/WorldAssemblerM3/Round12/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round12/REPORT.md).
## 9. Multiplayer model (listen-server first, dedicated-friendly) ## 9. Multiplayer model (listen-server first, dedicated-friendly)
> **Status: transport is DESIGN ONLY — nothing is networked in-tree yet** (no `Replicated`/`HasAuthority`/RPCs; a > **Status: transport is DESIGN ONLY — nothing is networked in-tree yet** (no `Replicated`/`HasAuthority`/RPCs; a
+25
View File
@@ -331,6 +331,11 @@ game-thread `PendingTileCancellation` map whose per-request atomic flags are rea
S5 assigns request and worker-start ordinals to each tile, preserving epoch, resident-at-request, S5 assigns request and worker-start ordinals to each tile, preserving epoch, resident-at-request,
priority, queue-wait and build timing in the opt-in startup trace. All four result queues remain MPSC; priority, queue-wait and build timing in the opt-in startup trace. All four result queues remain MPSC;
EndPlay still waits for workers and drains every queue. EndPlay still waits for workers and drains every queue.
S8 adds `StartPendingCoreRegionCompile`: by default `voxel.CoreRegionConcurrentAdmission=1` lets
`LoadCoreTile` admit keys independent of a pending region page; `0` restores the global compile gate.
`voxel.StreamingResultBacklogCap=256` limits queued result payloads + active tile tasks + held
prequeue results without folding completed results back into worker-slot admission; `0` disables it.
Core region pages are immutable shared owners while readers are active. See §8.29 and §3.15.
**⭐ Espace ACTEUR / ACTOR SPACE (2026-08-17).** Le champ de densite est defini en espace **acteur** : **⭐ Espace ACTEUR / ACTOR SPACE (2026-08-17).** Le champ de densite est defini en espace **acteur** :
`(0,0)` est l origine de l acteur `AVoxelWorld`, pas celle du monde Unreal. `WorldToLocalCm` / `(0,0)` est l origine de l acteur `AVoxelWorld`, pas celle du monde Unreal. `WorldToLocalCm` /
@@ -352,6 +357,8 @@ casse cette arithmetique *structurellement* — `BeginPlay` loggue une **Error**
| `ValidateDeterminism()` | — | **F2 CallInEditor button (PIE)**: re-samples boundary points under left- vs right-chunk cache warm-ups + a same-alignment repeat; any non-zero delta = window-invariance regression (§8.4). Run after every "bit-identical" hot-path refactor. | | `ValidateDeterminism()` | — | **F2 CallInEditor button (PIE)**: re-samples boundary points under left- vs right-chunk cache warm-ups + a same-alignment repeat; any non-zero delta = window-invariance regression (§8.4). Run after every "bit-identical" hot-path refactor. |
| `GetMaxConcurrentTasks()` | — | T2.d — asset `MaxConcurrentTasks` capped to logical cores − 2 (all three budget checks use it). | | `GetMaxConcurrentTasks()` | — | T2.d — asset `MaxConcurrentTasks` capped to logical cores − 2 (all three budget checks use it). |
| `GetActiveGenerationTaskCount()` | — | Normal mode returns the existing pending count. With `voxel.Step1PrequeueVisibleLOD0>0`, completed held results are excluded from active worker admission so a full render corpus can be generated before the upload release. | | `GetActiveGenerationTaskCount()` | — | Normal mode returns the existing pending count. With `voxel.Step1PrequeueVisibleLOD0>0`, completed held results are excluded from active worker admission so a full render corpus can be generated before the upload release. |
| `StartPendingCoreRegionCompile()` / `LoadCoreTile()` | — | Background Core region-page compile. `voxel.CoreRegionConcurrentAdmission=0` is the global-stop control; default `1` withholds only keys in the pending/in-flight region sets. Independent tiles can proceed. Core pages use immutable `shared_ptr` ownership under a separate shared mutex. §8.29. |
| `EnqueueStreamingResult()` / `IsStreamingResultBacklogCapped()` | — | Central result accounting for all four MPSC queues. `voxel.StreamingResultBacklogCap=256` caps queued results + active tile tasks + held prequeue results; `0` disables the cap. This bounds result-payload growth separately from worker-slot admission. §8.29. |
| `PostEditChangeProperty` | 1000 | Editor live-edit hook. | | `PostEditChangeProperty` | 1000 | Editor live-edit hook. |
| `OnObjectModifiedInEditor` | 1022 | Regenerates when a strate asset is edited (if `bLiveEditStrates`). | | `OnObjectModifiedInEditor` | 1022 | Regenerates when a strate asset is edited (if `bLiveEditStrates`). |
| `EndPlay` | 1118 | Sets `bShuttingDown`, **waits for `ActiveTaskCount`→0**, unbinds delegate. | | `EndPlay` | 1118 | Sets `bShuttingDown`, **waits for `ActiveTaskCount`→0**, unbinds delegate. |
@@ -1016,6 +1023,24 @@ cumulative task intervals for worker-seconds. `VoxelForgeExploreCommandlet.cpp`
owner-plan trace and moving actor check. See ARCHITECTURE §8.27–8.28 and owner-plan trace and moving actor check. See ARCHITECTURE §8.27–8.28 and
[`Standalone/Research/WorldAssemblerM3/Round11/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round11/REPORT.md). [`Standalone/Research/WorldAssemblerM3/Round11/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round11/REPORT.md).
### 3.15 Moving region admission and result-backlog safety trace — World Assembler S8 / M3 Round 12
`core_region_compile_request/start/complete` record why the desired region is being built (initial
set, player-center move, or same-center desired-set change), center delta, pending tiles, compile wall
time, accumulated pending interval, independent submissions and dependent-key declines. While a
compile is pending, `scheduler_skip.reason=core_region_compile_pending` records per-LOD missing
counts, motion and speed. The default concurrent mode should have no global region-pending skip;
its negative control is `voxel.CoreRegionConcurrentAdmission=0`.
`walkable_gate` logs every support-center change and whether all nine required LOD0 collision tiles
are ready there. `result_queue_frame` samples queued results, payload bytes, active tile tasks and
the cap; `result_queue_summary` stores peak queue length/bytes and admission stops. These counters
include slow-apply, teleport and high-speed controls added to `VoxelForgeExploreCommandlet.cpp` and
`Tools/TestVoxelForgeCoreWorldHost.ps1`. Keep the cap negative control (`voxel.StreamingResultBacklogCap=0`)
separate from the region-gate control. Round12 found 11/11 ready support centers at 6 m/s and 8/113
immediately ready at 60 m/s with a 400 m teleport; see the traces under `Saved/BuildHost` and
[`Standalone/Research/WorldAssemblerM3/Round12/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round12/REPORT.md).
## 4. The density pipeline (most-edited hot path) ## 4. The density pipeline (most-edited hot path)
### 4.1 `GetDensityWithParams` (TunnelNetwork) — VoxelGenerator.cpp:3749 ### 4.1 `GetDensityWithParams` (TunnelNetwork) — VoxelGenerator.cpp:3749
File diff suppressed because it is too large. Load diff
+31
View File
@@ -704,6 +704,11 @@ private:
TArray<FVoxelTileKey>& OutLeavers); TArray<FVoxelTileKey>& OutLeavers);
void StartPendingCoreRegionCompile(); void StartPendingCoreRegionCompile();
void LoadCoreTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority WorkPriority); void LoadCoreTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority WorkPriority);
void EnqueueStreamingResult(FChunkResult&& Result);
void RecordStreamingResultDequeued(const FChunkResult& Result);
uint64 GetStreamingResultPayloadBytes(const FChunkResult& Result) const;
int32 GetStreamingResultBacklogCount() const;
bool IsStreamingResultBacklogCapped() const;
void ApplyCoreModification(const FVoxelModification& Modification); void ApplyCoreModification(const FVoxelModification& Modification);
void BuildDesiredTransitionIndex(); void BuildDesiredTransitionIndex();
float GetActiveVoxelSizeCm() const; float GetActiveVoxelSizeCm() const;
@@ -768,7 +773,20 @@ private:
FVector CoreFallbackCenterLocalCm = FVector::ZeroVector; FVector CoreFallbackCenterLocalCm = FVector::ZeroVector;
bool bCoreRegionCompilePending = false; bool bCoreRegionCompilePending = false;
TArray<FVoxelTileKey> PendingCoreRegionTiles; TArray<FVoxelTileKey> PendingCoreRegionTiles;
TSet<FVoxelTileKey> PendingCoreRegionTileSet;
TArray<FVoxelTileKey> PendingCoreRetainedTiles; TArray<FVoxelTileKey> PendingCoreRetainedTiles;
TSet<FVoxelTileKey> CoreRegionCompileTilesInFlight;
FIntVector LastCoreRegionRequestCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
FIntVector PendingCoreRegionRequestCenter = FIntVector::ZeroValue;
FIntVector ActiveCoreRegionCompileCenter = FIntVector::ZeroValue;
bool bHasLastCoreRegionRequestCenter = false;
double CoreRegionPendingSinceSeconds = 0.0;
double ActiveCoreRegionCompileStartSeconds = 0.0;
uint64 ActiveCoreRegionCompileStartCycles = 0;
uint64 CoreRegionIndependentSubmissions = 0;
uint64 CoreRegionDependentDeclines = 0;
int32 CoreRegionCompileRequestSerial = 0;
int32 ActiveCoreRegionCompileRequestSerial = 0;
public: public:
@@ -1098,6 +1116,16 @@ public:
TQueue<FChunkResult, EQueueMode::Mpsc> EditedProcessQueue; TQueue<FChunkResult, EQueueMode::Mpsc> EditedProcessQueue;
TQueue<FChunkResult, EQueueMode::Mpsc> NearProcessQueue; TQueue<FChunkResult, EQueueMode::Mpsc> NearProcessQueue;
TQueue<FChunkResult, EQueueMode::Mpsc> ProcessQueue; TQueue<FChunkResult, EQueueMode::Mpsc> ProcessQueue;
// The cap counts completed MPSC results plus admitted tile tasks, so a slow game-thread apply
// cannot accumulate an unbounded payload queue while completed work never consumes worker slots.
// Le plafond compte les resultats MPSC et les taches admises; un apply lent ne peut donc pas
// accumuler sans limite, et les resultats termines ne reprennent pas de slot worker.
std::atomic<int32> QueuedStreamingResultCount{0};
std::atomic<uint64> QueuedStreamingResultBytes{0};
std::atomic<int32> PeakQueuedStreamingResultCount{0};
std::atomic<uint64> PeakQueuedStreamingResultBytes{0};
std::atomic<int32> ActiveTileTaskCount{0};
std::atomic<uint64> StreamingResultBacklogCapStops{0};
// Keys stay pending from task submission until the game thread applies the result; completed results can wait here. // Keys stay pending from task submission until the game thread applies the result; completed results can wait here.
// Les cles restent en attente de la soumission jusqu'a l'application sur le thread de jeu ; les resultats termines y attendent. // Les cles restent en attente de la soumission jusqu'a l'application sur le thread de jeu ; les resultats termines y attendent.
TSet<FVoxelTileKey> PendingTiles; TSet<FVoxelTileKey> PendingTiles;
@@ -1302,6 +1330,9 @@ public:
float HeadlessStreamingTestSpeedCmPerSecond = 0.0f; float HeadlessStreamingTestSpeedCmPerSecond = 0.0f;
float HeadlessStreamingTestStartSeconds = 0.0f; float HeadlessStreamingTestStartSeconds = 0.0f;
float HeadlessStreamingTestExitSeconds = 0.0f; float HeadlessStreamingTestExitSeconds = 0.0f;
float HeadlessStreamingTestTeleportAtSeconds = 0.0f;
float HeadlessStreamingTestTeleportDistanceCm = 0.0f;
bool bHeadlessStreamingTestTeleportDone = false;
FVector HeadlessStreamingTestDirection = FVector::XAxisVector; FVector HeadlessStreamingTestDirection = FVector::XAxisVector;
FVector HeadlessStreamingTestLastActualPosition = FVector::ZeroVector; FVector HeadlessStreamingTestLastActualPosition = FVector::ZeroVector;
double HeadlessStreamingTestBeginSeconds = 0.0; double HeadlessStreamingTestBeginSeconds = 0.0;
@@ -17487,6 +17487,9 @@ bool RunCoreWorldActorIntegrationTest(const FExploreArguments& Arguments, FStrin
const double ActorBeginSeconds = FPlatformTime::Seconds(); const double ActorBeginSeconds = FPlatformTime::Seconds();
constexpr double WalkStartSeconds = 30.0; constexpr double WalkStartSeconds = 30.0;
constexpr double RunDurationSeconds = 45.0; constexpr double RunDurationSeconds = 45.0;
float WalkSpeedCmPerSecond = 600.0f;
FParse::Value(FCommandLine::Get(), TEXT("voxel.TestMoveSpeedCmPerSecond="),
WalkSpeedCmPerSecond);
TArray<double> SpawnTickMilliseconds; TArray<double> SpawnTickMilliseconds;
TArray<double> WalkTickMilliseconds; TArray<double> WalkTickMilliseconds;
TArray<TPair<double, double>> TickSamples; TArray<TPair<double, double>> TickSamples;
@@ -17566,11 +17569,11 @@ bool RunCoreWorldActorIntegrationTest(const FExploreArguments& Arguments, FStrin
return false; return false;
} }
UE_LOG(LogTemp, Display, UE_LOG(LogTemp, Display,
TEXT("[VoxelForgeCoreWorldStreamTrace] pass complete=1 recipe=%s voxel_cm=%.3f walk_speed_cm_s=600 walk_direction=+Y"), TEXT("[VoxelForgeCoreWorldStreamTrace] pass complete=1 recipe=%s voxel_cm=%.3f walk_speed_cm_s=%.0f walk_direction=+Y"),
Arguments.CoreHybridMesherVersion == 2 ? TEXT("WorldAssemblerHybridV2") Arguments.CoreHybridMesherVersion == 2 ? TEXT("WorldAssemblerHybridV2")
: (Arguments.CoreHybridMesherVersion == 1 ? TEXT("WorldAssemblerHybridV1") : (Arguments.CoreHybridMesherVersion == 1 ? TEXT("WorldAssemblerHybridV1")
: TEXT("MarchingCubesV1")), : TEXT("MarchingCubesV1")),
Arguments.CoreVoxelSizeCm); Arguments.CoreVoxelSizeCm, WalkSpeedCmPerSecond);
return true; return true;
} }
@@ -1021,6 +1021,9 @@ namespace VoxelForgeCore
bool compile(const ShapeBuildRequest& request, const JobScheduler& scheduler, bool compile(const ShapeBuildRequest& request, const JobScheduler& scheduler,
std::string& error); std::string& error);
// True only when a fine tile needs a separately compiled region before generation.
// Vrai seulement si une tuile fine doit attendre la compilation d'une region dediee.
bool needsTileRegion(const TileKey& tile) const;
// Compile missing detail regions for entering fine tiles. Generated tile keys map to // Compile missing detail regions for entering fine tiles. Generated tile keys map to
// shared world-coordinate regions. The default analyticOnly path coalesces each entering // shared world-coordinate regions. The default analyticOnly path coalesces each entering
// LOD slab, builds its primitives/bins, and evaluates density on demand without // LOD slab, builds its primitives/bins, and evaluates density on demand without
+10
View File
@@ -909,6 +909,16 @@ namespace VoxelForgeCore
return ok; return ok;
} }
bool World::needsTileRegion(const TileKey& key) const
{
if(!impl_||!impl_->owner||!impl_->shape||!impl_->shape->IsBuilt()
||key.lod<0||key.lod>=impl_->configuration.fullResolutionLodCount)
return false;
if(CoveredByBaseShape(impl_->buildRequest,impl_->configuration,key))return false;
std::shared_lock lock(impl_->tileRegionMutex);
return !impl_->tileRegions.contains(key);
}
bool World::compileTileRegion(std::span<const TileKey> tiles,std::uint32_t workerLimit, bool World::compileTileRegion(std::span<const TileKey> tiles,std::uint32_t workerLimit,
const JobScheduler& scheduler,std::string& error, const JobScheduler& scheduler,std::string& error,
std::size_t* outCompiledTiles, std::size_t* outCompiledTiles,
@@ -1310,3 +1310,28 @@ No new source file was added. The likely compile spots are `VoxelWorld.cpp` and
file was modified. No commit, push or stash was made. file was modified. No commit, push or stash was made.
— Luna — Luna
## World assembler S8 / M3 Round 12 — 2026-10-19
The S7 trace's moving-path `bCoreRegionCompilePending` stop was real: the matched 6 m/s negative
control rejected 11 whole scheduler passes while region pages were rebuilt. With concurrent
admission enabled, those passes disappeared and 110 unrelated tiles submitted; all 11 support
centers still had a ready nine-tile ring. Overall moving worker use stayed near 2.7 of 10 slots, so
this closes the region gate without changing the separate spawn bottleneck.
The 60 m/s + 400 m teleport run admitted independent work too, but most center changes outran the
walkable ring. Spawn remains 7.7 s: 306 LOD8 jobs make a broad tail, while no one job exceeds
95 ms. The measured queue cap held the slow-apply backlog at 256 results / 76.2 MB; disabling it
reached 1,577 / 125.1 MB. Core-off, M2 fixtures, SurfaceWorld, FloatingIslands, S3/S4 and W1/W8/W16
checks passed. The optional tick subsystem marker was not added, leaving the 29 ms walk maximum
unattributed.
Full metrics, the matched controls, hashes, commands and async audit are in
[`WorldAssemblerM3/Round12/REPORT.md`](../../WorldAssemblerM3/Round12/REPORT.md).
No new source file was added. Compile spots are `VoxelWorld.cpp/.h`, the Core public header and
`VoxelForgeCore.cpp`, `VoxelForgeExploreCommandlet.cpp`, and the BuildHost harness. No owner
project/editor, asset or external plugin file was modified; no commit, push or stash was made.
— Luna
@@ -0,0 +1,233 @@
# World assembler S8 / M3 Round 12 — moving region admission and bounded results
**Result:** the S7 region-pending stop is removed for independent tiles. At 6 m/s, the matched
negative control logged 11 global admission skips; the new mode logged none and admitted 110
independent tiles during those windows. All 11 changed support centers had a ready 9/9 walkable
ring. This did not materially change overall moving worker occupancy: about 110 of 150 moving
slot-seconds remained idle. A 60 m/s teleport run admitted unrelated tiles too, but most centers
still outran their walkable ring.
The result queue is now separately bounded at 256 queued results + admitted tile tasks + held
prequeue results. Under one-result-per-frame apply with a 60 m/s run and 400 m teleport, the tracked
result payload peak was 76.2 MB; the uncapped control reached 125.1 MB. Spawn remains about 7.7 s,
not the roughly 4.1 s ideal from 41 worker-seconds / 10 slots. The timeline shows a broad LOD8 tail,
not one abnormally long tile. No safe low-cost spawn fix was supported by this measurement.
## 1. Region gate measurement and change
`BuildCoreDesiredTiles` asks for region pages when the desired tile center changes or its initial
desired region set is established. The old `bCoreRegionCompilePending` path stopped the whole Core
submission pass. The new default `voxel.CoreRegionConcurrentAdmission=1` continues through
`LoadCoreTile` and refuses only keys in `PendingCoreRegionTileSet` or
`CoreRegionCompileTilesInFlight`. `voxel.CoreRegionConcurrentAdmission=0` restores the old global
gate as the negative control. Core tile arithmetic and recipes are unchanged.
Region pages are constructed away from the lookup map and published under `tileRegionMutex` as
immutable shared owners. `generateTile` copies the matching `shared_ptr` while holding the map's
shared lock, then keeps it for the complete read. Map eviction cannot free a page while an already
running tile task holds it. The actor's shared `WorldLock` prevents edits/resets from mutating the
Core world while readers or a region compile are active.
Matched fresh BuildHost processes used the staged owner `SurfaceWorld` settings, seed 0, 25 cm Core
scale, 2,578-key desired set, Legacy order, Dynamic near reserve, apply boost 0, 10 task slots,
Normal priority, `-corelimit=12`, and a +Y move starting at 30 s. Both modes used backlog cap 256.
| Moving 6 m/s | Region CVar 0 | Region CVar 1 |
|---|---:|---:|
| Region requests / completions | 11 / 11 | 11 / 11 |
| Reasons | 1 initial set, 10 center moves | 1 initial set, 10 center moves |
| Whole-scheduler region skips | 11 | 0 |
| Pending interval sum / worst interval | 426.3 / 40.9 ms | 433.1 / 41.4 ms |
| Compile wall sum / slowest compile | 10.8 / 2.5 ms | 10.5 / 2.4 ms |
| Independent tile submissions / dependent declines | 0 / 0 | 110 / 0 |
| Changed support centers ready 9/9 at the change | 11 / 11 | 11 / 11 |
| Walkable lag at those centers | 0 ms | 0 ms |
| Moving frames with an LOD0 key missing | 173 | 163 |
| Moving worker busy / idle slot-seconds (15 s × 10 slots) | 39.7 / 110.3 s | 39.9 / 110.1 s |
The pending interval is the request-to-completion time recorded for each compile; it is not all
global blocking in the new mode. The compiler's measured wall time summed to only about 11 ms over
the run. The default mode removes the full-pass skip and submits keys independent of the entering
region. Its total worker supply remains low: moving work averages about 2.66 of 10 busy workers.
Each of the 11 negative-control whole-pass skips had 92–144 desired keys missing and zero active
tile tasks. The CVar1 run has no corresponding global skip.
Per-LOD counts added to `scheduler_skip` make the LOD0-missing-frame count include the 6 globally
gated frames that had LOD0 work missing in the negative control.
The fast trace used 60 m/s and teleported 400 m after 2 s of the move. It recorded 113 region
requests (initial set plus 112 center moves), all completed, with 381.4 ms aggregate compile wall
time (30.8 ms worst) and 4,523.7 ms accumulated pending interval (69.9 ms worst). Concurrent
admission recorded 782 independent submissions, 983 dependent-key declines, and zero global
region skips. Of 113 changing support centers, 8 were already 9/9 ready on the change event, 3
became ready before the next center change, and 102 did not reach 9/9 before leaving or test end.
The worst resolved lag was 106.3 ms. LOD0 was missing on 480 of 488 moving scheduler frames. This
speed is beyond what the current collision ring can follow.
## 2. Spawn, worker timeline and tail
| Fresh owner-plan spawn | S7 Dynamic reference | Round12 region CVar 0 | Round12 region CVar 1 |
|---|---:|---:|---:|
| Full set (2,578 keys, from BeginPlay) | 7.780 s | 7.820 s | 7.706 s |
| Walkable 9/9 ring | 0.581 s | 0.583 s | 0.566 s |
| Full LOD0 (512 tiles) | 1.439 s | 1.442 s | 1.424 s |
| Worker task-seconds to full set | 41.364 | 41.966 | 41.229 |
| Integrated busy workers / 10 (first request → ready) | 5.63 | 5.63 | 5.60 |
| Idle slot-seconds (same interval) | 32.056 | 32.63 | 32.37 |
| `UWorld::Tick` spawn p95 / max | 2.262 / 5.230 ms | 2.387 / 5.273 ms | 2.359 / 5.383 ms |
| `UWorld::Tick` move p95 / max | 2.512 / 28.893 ms | 2.472 / 28.257 ms | 2.377 / 28.967 ms |
The CVar A/B changes the spawn full-set time by only 0.11 s; the region issue occurs during the
later move. The Round12 CVar1 first-request-to-ready span was 7.360 s with 41.229 worker-seconds.
In one-second bins its worker task time was 3.94, 3.91, 3.34, 6.51, 5.73, 7.75 and 7.81 worker-
seconds, followed by 2.24 worker-seconds in the final 0.36 s. Average occupancy was 5.60/10 and
idle capacity was 32.37 slot-seconds. No sustained ten-worker plateau appears. That is about
16.0 ms of elapsed task time per initial tile, similar to S7's 16.04 ms and still about 9% above
the S7 old-path control's 14.62 ms. It does not show whether the difference is CPU contention or
other elapsed-time variation.
The 306 LOD8 jobs consumed 18.798 worker-seconds (45.6% of total), averaging 61.4 ms each; their
worker-time p95 was 79.8 ms and max 94.7 ms. The final 20 completed tile records were all LOD8 and
landed between 7.587 and 7.765 s. This is a group tail of ordinary large jobs, not one serial tile
to split. Across the initial set, worker queue wait averaged 1.71 ms per tile (p95 9.71 ms, max
41.44 ms), and game-thread mesh apply totaled 0.250 s. The trace records elapsed time, not CPU
cycles, cache misses or task-pool wakeup causes; it cannot identify memory bandwidth or thread-local
cache contention. The 10-slot cap reflects 12 logical cores with two cores reserved by
`GetMaxConcurrentTasks`; I left that reserve in place without evidence that using all 12 improves
the spawn while retaining tick time.
The optional actor-tick subsystem markers were not added. The matched run reports the 28.967 ms
maximum walk tick, but this trace has no per-call marker at that maximum, so it remains unattributed.
These are headless/null-RHI actor tick measurements, not rendered frame timings.
## 3. Result backlog bound
`voxel.StreamingResultBacklogCap` defaults to 256 and counts queued results, active tile tasks, and
held visible-LOD0 prequeue results. It is distinct from worker-slot admission; finished results
still do not occupy slots. Zero disables the cap and is the safety negative control.
| Run | Queue cap | Peak result count | Peak payload bytes | Cap stops | Peak private memory |
|---|---:|---:|---:|---:|---:|
| 6 m/s normal move | 256 | 116 | 17,024,546 | 0 | 3.48 GB |
| 60 m/s + 400 m teleport | 256 | 114 | 17,153,276 | 0 | 3.80 GB |
| 60 m/s + teleport, one mesh apply/frame | 256 | 256 | 76,201,412 | 843,198 | 3.60 GB |
| Same slow-apply control, cap disabled | 0 | 1,577 | 125,098,792 | 0 | 3.63 GB |
The slow-apply test still reached its initial set: 23.890 s capped versus 23.493 s uncapped. The
cap reduced the tracked peak by 1,321 results and 48.9 MB of payload, at a 0.397 s startup
difference in this pair. `cap_stops` counts repeated admission attempts, not unique tile keys. The
atomic peak tracker saw 256 / 1,577 results; periodic `result_queue_frame` samples saw 255 / 1,576,
respectively, because a frame sample can miss the brief peak. Bytes are result payload estimates,
excluding queue-node and allocator overhead. The default stops new admissions as work drains; it
does not reduce active task capacity when the queue is below the cap.
## 4. Quality, determinism and async audit
- BuildHost Core Off legacy passed at 12.5 cm. HybridV1 and HybridV2 each passed five fixtures and
same-LOD border checks at 25 cm.
- Final-module SurfaceWorld passed the full actor, collision, anchor-move and carve checks at 25 cm.
Its W1/W8/W16 worker matrix matched: density `0cd2a1a4152ee4aa`, mesh `80a5b054b904b1a2`, payload
`bf2f1b40dff38acc`, topology `56074198682d5e18`. S3 ground agreement passed 128 points: far7/8
maximum error 0.0427 cm, cross-LOD 0.0000 cm, 2 cm tolerance; the 100 cm offset control was
detected.
- FloatingIslands passed the final BuildHost W1/W8/W16 actor matrix at 25 cm, with fixed water,
collision, edits and LOD0/1 coverage. The standalone W1/W8/W16 `--stream-hash` runs matched on all
nine LODs. Its host scheduler clamps `parallel_for` to at most eight actual workers; W16 is the
requested worker-limit input routed through that eight-thread executor. The S4 audit passed:
385/385 far-cap bands emitted, 128 agreement points within 0.0504 cm (2 cm tolerance), 100 cm
offset detected, and 2,520/2,520 coverage rays hit.
- No field, mesh or recipe output changed; recipe version remains unchanged. Water remains fixed
for the season, and player carve/fill remains unrestricted.
Async review of the diff:
- `FChunkResult::Epoch` and desired/modification epochs remain carried and checked. The region task
captures generation and desired epochs plus its compile serial.
- All four result queues remain `EQueueMode::Mpsc`; result insertion is centralized in
`EnqueueStreamingResult`.
- Tile and region workers check `bShuttingDown`; `EndPlay` sets both shutdown flags, waits for
`ActiveTaskCount` to reach zero, then drains all four result queues.
- `ModsLock` is unchanged. Workers generate data only; UObject and RMC mutation remains on the game
thread. The region map has its own shared mutex, and readers retain immutable pages with
`shared_ptr` until their tile job ends.
## 5. Build, commands and run record
BuildHost compiled at Normal priority with 12 actions. Link-safety passed (96,279 dumpbin lines
checked against 421 names); the five staged asset hashes and 18 manifest shims passed. The final
SurfaceWorld quality run took about 10 minutes and peaked at 3.896 GB private memory. The standalone
full build passed with the existing `vswhere.exe` lookup warning and existing C4838 conversions in
`VoxelStrateManager.cpp:3974–3975`.
Build commands:
```powershell
.\Tools\BuildVoxelForgeCoreHost.ps1 -MaxParallelActions 12 -TimeoutSeconds 900
.\Standalone\Build.ps1 -Target All -CompileWorkers 12 -BuildPriority Normal
```
Matched region-gate commands (cap held at 256 for both sides):
```powershell
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StartupApplyBoost 0 -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -CoreRegionConcurrentAdmission 0 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 600 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StartupApplyBoost 0 -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 600 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StartupApplyBoost 0 -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 6000 -TestTeleportAfterSeconds 2 -TestTeleportDistanceCm 40000 -TimeoutSeconds 900
```
Slow-apply backlog controls used these commands; the cap-off run is the backlog negative control:
```powershell
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StartupApplyBoost 0 -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 6000 -TestTeleportAfterSeconds 2 -TestTeleportDistanceCm 40000 -Step1MaxMeshApplies 1 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StartupApplyBoost 0 -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 0 -TestMoveSpeedCmPerSecond 6000 -TestTeleportAfterSeconds 2 -TestTeleportDistanceCm 40000 -Step1MaxMeshApplies 1 -TimeoutSeconds 900
```
Other BuildHost checks were:
```powershell
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -CoreOffTest -CoreVoxelSizeCm 12.5 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -HybridFixturesOnly -CoreMesherRecipe WorldAssemblerHybridV1 -CoreVoxelSizeCm 25 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -HybridFixturesOnly -CoreMesherRecipe WorldAssemblerHybridV2 -CoreVoxelSizeCm 25 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -CoreVoxelSizeCm 25 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -FloatingIslandsOnly -CoreVoxelSizeCm 25 -TimeoutSeconds 900
```
Standalone determinism used three fresh processes (`--workers=1`, `8`, and `16`), each with:
```powershell
.\Standalone\bin\VoxelForgeCoreStandalone.exe --workers=1 --world-assembler-prototype --stream-benchmark --stream-hash --force-generate --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --process-priority=Normal --process-affinity=4095 --frame-limit=60
```
The S4 standalone audit command was:
```powershell
.\Standalone\bin\VoxelForgeCoreStandalone.exe --world-assembler-audit --workers=12 --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --process-priority=Normal --process-affinity=4095 --frame-limit=60
```
SHA-256 of the executables/modules used by the final runs:
| Binary | SHA-256 |
|---|---|
| UE 5.7 `UnrealEditor-Cmd.exe` | `ECCE9996191F7F7E69B3D104EECDD0E678AE69176FDCCD79828C3E75A0654D6D` |
| BuildHost `UnrealEditor-VoxelForge.dll` | `8B82EE6C6FBB511AE766EB3D8646A7E9491952F41445CB62D49D71D0B9783418` |
| BuildHost `UnrealEditor-VoxelForgeCore.dll` | `0A1EB7BB27C3C74A4AA0996E0A24E74224D7B8FFFFF113B1071FA42E18E8375A` |
| BuildHost `UnrealEditor-VoxelForgeEditor.dll` | `B3BC5E61172C5B21D928A5ACB5F22F8D9E00179B481B675804838DE9A620010D` |
| `Standalone/bin/VoxelForgeCoreStandalone.exe` | `9C1601D7FCACCAAC16A2BA9DF0E7826E18746028A4D08FD23EEC479D2F5745BE` |
Trace logs and queue records are under
`Saved/BuildHost/HostProject/Plugins/VoxelForge/Saved/CoreWorldValidation/Logs/`. The final quality
logs and standalone determinism/audit stdout are under `Saved/BuildHost/Round12/`. No new file over
300 KB was placed in `Research/`.
BuildHost and standalone launch guards checked free physical memory before each process and waited
for at least 4 GiB. Explicit preflights for the final matched run and SurfaceWorld actor test saw
17.33 and 17.11 GiB free. All processes stayed under 8 GiB and used Normal priority / at most 12
threads. The only crash-folder entry is the pre-existing
`UECC-Windows-F1A3EFD5469F5DD62E68EBBACAC84376_0000` from 2026-09-17; no new UECC or
`0xC0000142` occurred. Three exploratory PowerShell RAM-probe definitions failed before launching
Unreal; after correcting the interop declaration, the probe and all actual build/test runs passed
on attempt 1. There were no failed Unreal runs or retries.
No new source file was added. Likely compile spots are `Source/VoxelForge/Private/VoxelWorld.cpp`,
`Source/VoxelForge/Public/VoxelWorld.h`, `Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp`,
`Standalone/Core/src/VoxelForgeCore.cpp` and its public header, plus the PowerShell harness. I did
not open the owner's editor, build `VoxelM.uproject`, edit any `.uasset`, or modify
RealtimeMeshComponent. No commit, push or stash was made.
+33 -2
View File
@@ -8,6 +8,12 @@ param(
[switch]$StartupTrace, [switch]$StartupTrace,
[ValidateRange(0, 1)][int]$StartupApplyBoost = 0, [ValidateRange(0, 1)][int]$StartupApplyBoost = 0,
[switch]$StreamingTraceOnly, [switch]$StreamingTraceOnly,
[ValidateRange(1, 6000)][int]$TestMoveSpeedCmPerSecond = 600,
[ValidateRange(0, 1)][int]$CoreRegionConcurrentAdmission = 1,
[ValidateRange(0, 4096)][int]$StreamingResultBacklogCap = 256,
[ValidateRange(0, 900)][int]$Step1MaxMeshApplies = 0,
[ValidateRange(0, 900)][int]$TestTeleportAfterSeconds = 0,
[ValidateRange(0, 1000000)][int]$TestTeleportDistanceCm = 0,
[string]$WorldAssemblyManifestPath = '', [string]$WorldAssemblyManifestPath = '',
[ValidateSet('MarchingCubesV1', 'WorldAssemblerHybridV1', 'WorldAssemblerHybridV2')][string]$CoreMesherRecipe = 'MarchingCubesV1', [ValidateSet('MarchingCubesV1', 'WorldAssemblerHybridV1', 'WorldAssemblerHybridV2')][string]$CoreMesherRecipe = 'MarchingCubesV1',
[switch]$EnableHybridFixturePlacement, [switch]$EnableHybridFixturePlacement,
@@ -73,6 +79,15 @@ if ($StreamingTraceOnly -and ($SharedFeatureAuditOnly -or $EnableHybridFixturePl
$HybridCliffOnly -or $HybridFixturesOnly -or $WorldAssemblyManifestPath)) { $HybridCliffOnly -or $HybridFixturesOnly -or $WorldAssemblyManifestPath)) {
throw '-StreamingTraceOnly uses the staged owner plan without additional fixture or manifest overrides.' throw '-StreamingTraceOnly uses the staged owner plan without additional fixture or manifest overrides.'
} }
if (-not $StreamingTraceOnly -and ($TestMoveSpeedCmPerSecond -ne 600 -or
$CoreRegionConcurrentAdmission -ne 1 -or $StreamingResultBacklogCap -ne 256 -or
$Step1MaxMeshApplies -ne 0 -or $TestTeleportAfterSeconds -ne 0 -or
$TestTeleportDistanceCm -ne 0)) {
throw 'Streaming, backlog, apply and teleport controls require -StreamingTraceOnly.'
}
if (($TestTeleportAfterSeconds -eq 0) -ne ($TestTeleportDistanceCm -eq 0)) {
throw 'Set both -TestTeleportAfterSeconds and -TestTeleportDistanceCm for the teleport probe.'
}
function Assert-UnderPath([string]$Candidate, [string]$Root) { function Assert-UnderPath([string]$Candidate, [string]$Root) {
$candidateFull = [IO.Path]::GetFullPath($Candidate).TrimEnd('\', '/') $candidateFull = [IO.Path]::GetFullPath($Candidate).TrimEnd('\', '/')
@@ -153,6 +168,13 @@ $RunName += switch ($NearReserveMode) {
} }
if ($EnforceCollisionGate) { $RunName += '_collision_gate' } if ($EnforceCollisionGate) { $RunName += '_collision_gate' }
if ($StartupTrace -or $StreamingTraceOnly) { $RunName += '_trace' } if ($StartupTrace -or $StreamingTraceOnly) { $RunName += '_trace' }
if ($StreamingTraceOnly) {
$RunName += "_move_${TestMoveSpeedCmPerSecond}_region_${CoreRegionConcurrentAdmission}_backlog_${StreamingResultBacklogCap}"
if ($Step1MaxMeshApplies -gt 0) { $RunName += "_applymax_$Step1MaxMeshApplies" }
if ($TestTeleportAfterSeconds -gt 0) {
$RunName += "_teleport_${TestTeleportAfterSeconds}s_${TestTeleportDistanceCm}cm"
}
}
$foreignEditors = @(Get-Process -ErrorAction SilentlyContinue | $foreignEditors = @(Get-Process -ErrorAction SilentlyContinue |
Where-Object { $_.ProcessName -like 'UnrealEditor*' }) Where-Object { $_.ProcessName -like 'UnrealEditor*' })
@@ -278,12 +300,21 @@ $BaseArguments += "-voxel.StreamingCoarseFirst=$CoarseFirstValue"
$BaseArguments += "-voxel.StreamingDynamicNearReserve=$DynamicReserveValue" $BaseArguments += "-voxel.StreamingDynamicNearReserve=$DynamicReserveValue"
$BaseArguments += "-voxel.StreamingLegacyPendingResultCap=$LegacyPendingResultCapValue" $BaseArguments += "-voxel.StreamingLegacyPendingResultCap=$LegacyPendingResultCapValue"
$BaseArguments += "-voxel.StreamingStartupApplyBoost=$StartupApplyBoost" $BaseArguments += "-voxel.StreamingStartupApplyBoost=$StartupApplyBoost"
$BaseArguments += "-voxel.CoreRegionConcurrentAdmission=$CoreRegionConcurrentAdmission"
$BaseArguments += "-voxel.StreamingResultBacklogCap=$StreamingResultBacklogCap"
if ($Step1MaxMeshApplies -gt 0) {
$BaseArguments += "-voxel.Step1MaxMeshApplies=$Step1MaxMeshApplies"
}
if ($StreamingTraceOnly) { if ($StreamingTraceOnly) {
$BaseArguments += '-coreworldstreamtrace=1' $BaseArguments += '-coreworldstreamtrace=1'
$BaseArguments += '-voxel.TestMove=1' $BaseArguments += '-voxel.TestMove=1'
$BaseArguments += '-voxel.TestMoveSpeedCmPerSecond=600' $BaseArguments += "-voxel.TestMoveSpeedCmPerSecond=$TestMoveSpeedCmPerSecond"
$BaseArguments += '-voxel.TestMoveStartSeconds=30' $BaseArguments += '-voxel.TestMoveStartSeconds=30'
$BaseArguments += '-voxel.TestMoveDirection=0,1,0' $BaseArguments += '-voxel.TestMoveDirection=0,1,0'
if ($TestTeleportAfterSeconds -gt 0) {
$BaseArguments += "-voxel.TestTeleportAfterSeconds=$TestTeleportAfterSeconds"
$BaseArguments += "-voxel.TestTeleportDistanceCm=$TestTeleportDistanceCm"
}
} }
if ($StartupTrace -or $StreamingTraceOnly) { if ($StartupTrace -or $StreamingTraceOnly) {
$TracePath = Join-Path $LogDirectory "CoreWorldValidation-$RunName.trace.jsonl" $TracePath = Join-Path $LogDirectory "CoreWorldValidation-$RunName.trace.jsonl"
@@ -417,7 +448,7 @@ if ($SharedFeatureAuditOnly) {
} }
if ($StreamingTraceOnly) { if ($StreamingTraceOnly) {
if ($Text -notmatch '\[VoxelForgeCoreWorldStreamTrace\] initial_ready=1 .*walk_tick_ms_p95=.*walk_tick_ms_max=' -or if ($Text -notmatch '\[VoxelForgeCoreWorldStreamTrace\] initial_ready=1 .*walk_tick_ms_p95=.*walk_tick_ms_max=' -or
$Text -notmatch '\[VoxelForgeCoreWorldStreamTrace\] pass complete=1 .*walk_speed_cm_s=600 walk_direction=\+Y' -or $Text -notmatch ("\[VoxelForgeCoreWorldStreamTrace\] pass complete=1 .*walk_speed_cm_s=$TestMoveSpeedCmPerSecond walk_direction=\+Y") -or
$Text -notmatch '\[VoxelForgeCoreWorldTest\] pass complete=1') { $Text -notmatch '\[VoxelForgeCoreWorldTest\] pass complete=1') {
throw "BuildHost actor-only stream trace did not prove initial readiness and a completed walk sample. See $FinalLog." throw "BuildHost actor-only stream trace did not prove initial readiness and a completed walk sample. See $FinalLog."
} }