wip(assembler): round S12 -- short ready queue keeps workers fed between ticks: full set 6.0 s -> 4.9 s

voxel.StreamingReadyQueueDepth=8 admits one extra worker-set of prioritized
tasks; voxel.StreamingBackgroundWorkerCap=1 sizes slots to the 8 TaskGraph
background workers actually running tile tasks. Full set 5.96 -> 4.91 s,
LOD0 1.40 -> 1.09 s, ring 0.49 -> 0.51 s, 40 m/s ring 75/75. Tick p95
+0.6 ms spawn / +0.26 ms walk, max flat. No geometry change.
Report: Standalone/Research/WorldAssemblerM3/Round16.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MmSoF64BbRFmk9C3p1VsBJ
This commit is contained in:
Fr0zkaandClaude Opus 5.5 committed 2026-10-19 14:29:38 +02:00
1 parent dd9bfd3217
commit 212814a75c
8 files changed
+554 -36

No files matched your search

@@ -0,0 +1,302 @@
# World Assembler S12 — Round 16
**2026-10-19 · worker supply and tick-independent task pickup**
## Result
The matched 25 cm SurfaceWorld run completed its 2,578-tile set in **4.911 s** on the actor startup
clock, down from Round15's **5.960 s**. From the first tile submission to readiness it took **4.537 s**,
down from **5.603 s**. Full LOD0's last apply moved from 1.403 s to **1.087 s**. The stationary
walkable ring measured **0.509 s** (Round15: 0.488 s). The 40 m/s trace kept the 3×3 collision-ready
ring at 9/9 for every moving gate sample.
The measured ceiling was eight tile workers, not ten: with 12 CPUs in the process affinity, Unreal
reported 10 TaskGraph workers split into two foreground and eight background workers. Tile tasks
use background priorities, so I sized their admission slots to the eight background workers. A
short, ordered TaskGraph queue keeps those workers supplied between actor ticks. No density,
triangulation, collision, or Core recipe output changed; recipe versions stay unchanged.
## Pool count and priority
| Measurement | BuildHost result |
|---|---:|
| Process affinity / priority | `0xFFF` (12 CPUs) / Normal |
| `UE::Tasks` scheduler workers / reported maximum | 10 / 20 |
| `FTaskGraphInterface` workers | 10 total: 2 foreground, 8 background |
| `GThreadPool` workers | 10; separate from TaskGraph |
| Step1 cook pool | 0 threads |
| Tile task API / priority | `UE::Tasks::Launch`; near `BackgroundHigh`, horizon `BackgroundNormal` |
| Scheduler worker / background priority fields | 5 / 2 in `worker_backend_config` |
| Unique tile worker thread IDs / peak simultaneous tile tasks | 8 / 8 |
RMC's inspected async collision path calls `DoOnAllowedThread(AsyncThread)`; that implementation
queues work on `GThreadPool` (`RealtimeMeshFuture.h:147–165`,
`RealtimeMeshManaged.cpp:703–707`). It therefore does not consume the eight TaskGraph background
worker IDs used by tile generation. Other UE background work can share TaskGraph's workers; this
trace attributes tile work by OS thread ID but does not label every unrelated engine task. The
external process sampler counted 40 OS threads after initialization and 58–60 while running,
including engine service threads. The 12-CPU affinity limits simultaneous CPU execution; it is not
a 12-OS-thread process cap. VoxelForge tile generation itself peaked at eight workers, and the
BuildHost process priority stayed Normal.
## Shared scheduler and trace
Two read-back CVars control the shipped behavior:
| CVar | Default | Negative control | Effect |
|---|---:|---:|---|
| `voxel.StreamingReadyQueueDepth` | 8 | 0 | Admit up to one extra worker-set of prioritized tasks |
| `voxel.StreamingBackgroundWorkerCap` | 1 | 0 | Size streaming slots to TaskGraph background workers; 0 restores the logical-core cap |
With the selected defaults, effective tile slots are 8 and the total admission cap is 16. Near
tasks retain `BackgroundHigh`; horizon tasks retain `BackgroundNormal`. While moving, the dynamic
near reserve is 7, the near admission cap is 8, and the coarse cap is 9. Collision look-ahead and
critical/edit admission retain their existing precedence. The game-thread scheduler continues to
select and submit from the desired-tile ordering; there is no worker-owned continuation list. UE
dispatches an already-admitted task as a background worker becomes free. `bStreamingPlayerMoving` is
a game-thread snapshot shared by `LoadTile` and `LoadCoreTile`.
Queued tasks retain `GenerationEpoch`, `DesiredEpoch`, and per-request cancellation checks. A desired
set change obsoletes stale queued work, workers still check `bShuttingDown`, and results still return
through the MPSC queues for game-thread application. No UObject or RMC component mutation moved to a
worker. The trace records CVar readbacks in `worker_backend_config` and
`streaming_worker_cap_readback`, and records each tile's submission, worker start/end, thread ID,
next scheduler pass, next submission, and next task start on that worker.
### Initial-set timeline
The following busy-worker equivalents integrate the 2,578 tile intervals in 500 ms bins from the
first submission (`submitted_t_s=0.456843` on the trace clock). The final short bin is 10.7 ms.
| Time from first submission | Busy worker equivalents, out of 8 |
|---|---:|
| 0–0.5 s | 6.16 |
| 0.5–1.0 s | 5.94 |
| 1.0–1.5 s | 4.82 |
| 1.5–2.0 s | 4.38 |
| 2.0–2.5 s | 7.83 |
| 2.5–3.0 s | 7.98 |
| 3.0–3.5 s | 6.58 |
| 3.5–4.0 s | 7.47 |
| 4.0–4.5 s | 7.67 |
| 4.5–4.511 s | 1.75 |
All eight worker IDs were used. The initial set accumulated **29.432 worker-seconds**; the first
worker started at 0.456872 and the last ended at 4.967520 on the trace clock, a 4.511 s worker
span. That workload has a 3.679 s ideal at eight fully occupied slots (2.943 s at ten); the
measured worker span was 4.511 s. Per-slot idle gaps between one tile ending and that worker's next tile starting summed to
**6.573 slot-seconds** over 2,570 gaps: p50 0.006 ms, p95 13.604 ms, max 16.800 ms.
| Worker thread ID | Tiles | Inter-task idle seconds | Gap p95 / max |
|---:|---:|---:|---:|
| 20144 | 324 | 0.838 | 13.780 / 15.607 ms |
| 21468 | 318 | 0.829 | 13.489 / 16.800 ms |
| 21536 | 304 | 0.830 | 13.237 / 15.902 ms |
| 23392 | 327 | 0.832 | 13.476 / 15.691 ms |
| 23572 | 331 | 0.821 | 13.365 / 15.916 ms |
| 23616 | 331 | 0.847 | 13.765 / 15.773 ms |
| 2716 | 321 | 0.734 | 13.486 / 15.801 ms |
| 5392 | 322 | 0.843 | 13.657 / 15.556 ms |
The scheduler recorded 247 admission passes, 243 with submissions, averaging 10.61 tiles per
non-empty pass. Peak initial scheduler backlog was 32; the whole trace's result-backlog peak was
35 tiles / 1,620,010 bytes against the 256-tile cap, with zero cap stops. The completed-result
queue itself peaked at five during initial readiness.
`Saved/BuildHost/Round16/initial_tile_worker_timeline.csv` contains one row per initial tile,
including submitted/start/end, next scheduler admission, next submission, same-thread next start,
and the calculated idle gap. The 500 ms busy bins, per-LOD totals, and per-thread idle table are
also saved there. Files larger than 300 KB remain under `Saved/BuildHost/`.
### Per-LOD completion and work
S12 worker-end/apply values below are normalized to the actor startup clock; S11 apply values are
from the Round15 matched report. S12 worker task totals were higher in this sample even though the
set finished sooner; the scheduler changes do not change tile contents.
| LOD | Tiles | Worker seconds S11 → S12 | S11 last apply | S12 last worker / last apply |
|---:|---:|---:|---:|---:|
| 0 | 512 | 3.725 → 4.018 | 1.403 s | 1.084 / 1.087 s |
| 1 | 448 | 3.895 → 4.099 | 2.283 s | 1.749 / 1.757 s |
| 2 | 320 | 0.806 → 0.825 | 2.849 s | 2.107 / 2.122 s |
| 3 | 272 | 2.064 → 2.296 | 3.330 s | 2.469 / 2.484 s |
| 4 | 272 | 6.590 → 7.682 | 4.276 s | 3.431 / 3.440 s |
| 5 | 208 | 1.528 → 1.732 | 4.660 s | 3.706 / 3.709 s |
| 6 | 144 | 2.077 → 2.160 | 5.060 s | 4.048 / 4.062 s |
| 7 | 96 | 1.928 → 2.044 | 5.338 s | 4.289 / 4.293 s |
| 8 | 306 | 4.171 → 4.576 | 6.019 s | 4.885 / 4.894 s |
LOD4 remains the largest worker-cost group: 7.682 s across 272 tiles, or 28.24 ms per tile.
The tile trace attributes this to the overall generation task (`generation_s`); it does not break
the worker interval into density/classification/meshing stages (`classify_s` and `mesh_s` are not
populated for these rows). Its last apply was 3.440 s, so it did not set the tail. I made no LOD4
content optimization without the requested stage attribution and quality-capture work.
## Apply, tick, and motion results
| Metric | Round15 S11 | Round16 S12 |
|---|---:|---:|
| Full set ready, actor startup clock | 5.960 s | 4.911 s |
| First submission to full ready | 5.603 s | 4.537 s |
| Worker span from first submission | 5.581 s | 4.511 s |
| Worker-seconds | 26.786 s | 29.432 s |
| Walkable ring, 9/9 | 0.488 s | 0.509 s |
| Full LOD0 last apply | 1.403 s | 1.087 s |
| Apply frames through readiness | 332 | 268 |
| Apply-count limit frames | 0 | 11 |
| Max visible mesh applies / frame | 10 | 12 |
| Total measured apply time | 236.14 ms | 255.17 ms |
| Max apply time / frame | 2.357 ms | 2.697 ms |
| Result-wait p95 / max | 16.62 / 19.26 ms | 16.90 / 21.03 ms |
| Result backlog peak / cap | 10 / 256 | 35 / 256 |
`UWorld::Tick` spawn and normal-walk windows were measured in fresh processes:
| Window | S11 p95 / max | S12 p95 / max | Change |
|---|---:|---:|---:|
| Spawn | 2.740 / 5.175 ms | 3.344 / 5.219 ms | +0.604 / +0.044 ms |
| Normal walk | 2.483 / 28.876 ms | 2.744 / 28.752 ms | +0.261 / −0.124 ms |
The maxima stayed effectively flat or lower, while p95 rose by 0.60 ms at spawn and 0.26 ms while
walking. I record this as a small p95 increase rather than claiming strict p95 non-regression. Both
candidate p95 values remained below 3.4 ms.
The 40 m/s run used the same queue and slot defaults. It completed the initial set in 4.919 s;
all **75/75** moving `walkable_gate` events reported 9/9, and all 58 moving streaming samples
reported the ring ready. Spawn tick p95/max was 3.173/8.071 ms; the accelerated walk window was
29.562/39.227 ms. The 40 m/s trace's collision look-ahead, dynamic reserve, and moving near cap
readbacks are present in the JSONL.
### Negative controls and rejected variants
| Read-back settings | Effective slots / actual worker IDs | Full set ready | Result |
|---|---:|---:|---|
| Queue depth 0, worker cap 1 | 8 / 8 | 7.024 s | Disabling the ready queue removes the speedup |
| Queue depth 8, worker cap 0 | 10 / 8 | 4.669 s | Restores logical-core admission; still only eight background IDs ran |
| Queue depth 8, worker cap 1 | 8 / 8 | 4.911 s | Selected default, sized to actual background workers |
The first 40 m/s queue prototype lacked the queue-aware moving reserve and LOD0 admission cap; it
missed the ring in 56/77 gate events and 47/57 streaming samples. Adding the reserve alone was
insufficient. The final moving near cap and reserve passed 75/75 gate events. Reducing the shared
Core task cap itself to eight increased per-tile generation cost and made the full set 5.30 s, so
that change was reverted; only streaming admission is background-worker-capped.
Foreground-priority variants were also rejected. Raising all tile work to foreground priorities
used ten worker IDs but produced 39.422/50.432 ms spawn and 49.137/83.184 ms walk p95/max, with a
5.215 s set. Raising only near work produced an 86.225 ms walk maximum. The final build keeps the
original background priorities.
## Determinism and quality gates
- Standalone W1/W8/W12 all passed with identical full-clipmap hashes: density
`c10697a8f48380c7`, mesh `e75c9119a1498cd5`, payload `49a9a8d1835a6c8c`, topology
`edcb23288b5e8724`.
- BuildHost SurfaceWorld W1/W8/W16 passed with stable density/mesh/payload/topology hashes
`0cd2a1a4152ee4aa / a06243aee16b65d5 / c700162935c262d7 / 0af901aa42b452bc`.
- BuildHost FloatingIslands passed at 25 cm; W1/W8/W16 were stable with hashes
`4dcfb37585d7d986 / f14f76fbf065c6ed / 53035c45fd1a082e / 23550100654a4ec3`.
- Core-off legacy gate passed at 12.5 cm (`legacy_unchanged=PASS`). M2 Hybrid V1 and V2 analytic
fixture and border suites passed at 25 cm.
- The current Standalone far-cap audit passed S4 agreement: 128 shared points, maximum far/near
difference 0.0504 cm against a 2 cm tolerance; the 100 cm negative control was detected. Coverage
was 2520/2520 (100%), with zero missing mesh/profile crossings. S3's prior 72-point audit remains
at 0.0427 cm; scheduler changes do not touch that Core content path.
- The 25 cm SurfaceWorld and FloatingIslands gates both passed. No world recipe version changed.
## Builds, commands, and hashes
Builds completed:
```powershell
.\Tools\BuildVoxelForgeCoreHost.ps1 -MaxParallelActions 12 -TimeoutSeconds 900
.\Standalone\Build.ps1 -Target All -CompileWorkers 12 -BuildPriority Normal
```
The matched SurfaceWorld trace command was:
```powershell
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -StreamingCollisionLookahead 1 -TestMoveDirection '0,1,0' -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 600 -StreamingReadyQueueDepth 8 -StreamingBackgroundWorkerCap 1 -TimeoutSeconds 900
```
The 40 m/s and negative-control commands were:
```powershell
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -StreamingCollisionLookahead 1 -TestMoveDirection '0,1,0' -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 4000 -StreamingReadyQueueDepth 8 -StreamingBackgroundWorkerCap 1 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -StreamingCollisionLookahead 1 -TestMoveDirection '0,1,0' -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 600 -StreamingReadyQueueDepth 0 -StreamingBackgroundWorkerCap 1 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -StreamingCollisionLookahead 1 -TestMoveDirection '0,1,0' -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 600 -StreamingReadyQueueDepth 8 -StreamingBackgroundWorkerCap 0 -TimeoutSeconds 900
```
Quality commands:
```powershell
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -CoreOffTest -CoreVoxelSizeCm 12.5 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -HybridFixturesOnly -CoreMesherRecipe WorldAssemblerHybridV1 -CoreVoxelSizeCm 25 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -HybridFixturesOnly -CoreMesherRecipe WorldAssemblerHybridV2 -CoreVoxelSizeCm 25 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -CoreVoxelSizeCm 25 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -FloatingIslandsOnly -CoreVoxelSizeCm 25 -TimeoutSeconds 900
```
The S3/S4 Standalone agreement and coverage audit command was:
```powershell
.\Standalone\bin\VoxelForgeCoreStandalone.exe --world-assembler-audit --workers=12 --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --process-priority=Normal --process-affinity=4095 --frame-limit=60 > Saved/BuildHost/Round16/standalone_S3S4_audit.stdout.log 2> Saved/BuildHost/Round16/standalone_S3S4_audit.stderr.log
```
Standalone determinism ran serially at Normal priority, affinity `4095`, and frame limit 60. Each
worker count used the same remaining arguments and output paths below:
```powershell
.\Standalone\bin\VoxelForgeCoreStandalone.exe --workers=1 --world-assembler-prototype --stream-benchmark --stream-hash --force-generate --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --far-horizon-lod8-step=0 --process-priority=Normal --process-affinity=4095 --frame-limit=60 > Saved/BuildHost/Round16/standalone_W1.stdout.log 2> Saved/BuildHost/Round16/standalone_W1.stderr.log
.\Standalone\bin\VoxelForgeCoreStandalone.exe --workers=8 --world-assembler-prototype --stream-benchmark --stream-hash --force-generate --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --far-horizon-lod8-step=0 --process-priority=Normal --process-affinity=4095 --frame-limit=60 > Saved/BuildHost/Round16/standalone_W8.stdout.log 2> Saved/BuildHost/Round16/standalone_W8.stderr.log
.\Standalone\bin\VoxelForgeCoreStandalone.exe --workers=12 --world-assembler-prototype --stream-benchmark --stream-hash --force-generate --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --far-horizon-lod8-step=0 --process-priority=Normal --process-affinity=4095 --frame-limit=60 > Saved/BuildHost/Round16/standalone_W12.stdout.log 2> Saved/BuildHost/Round16/standalone_W12.stderr.log
```
SHA-256 records. Streaming and quality runs used the first VoxelForge module hash. After those runs,
I added the French half of one inline comment and rebuilt the current tree; that comment-only rebuild
produced the second module hash. The executable code did not change between those two hashes.
| Binary | SHA-256 |
|---|---|
| BuildHost `UnrealEditor-VoxelForge.dll` used for timing and gates | `C9D5DA90B48C5C1F844221C4A6A195947016C522F6A296EBC488A1AD8CCC6EEF` |
| Current rebuilt BuildHost `UnrealEditor-VoxelForge.dll` | `12E7883DFB8825012890F1A2B5BACD038AA40FD1C342AA70711EC55AABF9057F` |
| BuildHost `UnrealEditor-VoxelForgeCore.dll` | `D87CC56BACCDE6559705EEAEA7DB73D1A6ABF79484D37A3887B94A9FD62BABD3` |
| BuildHost `UnrealEditor-VoxelForgeEditor.dll` | `805FB845F00D013B48E29D4FD5D56EF1531E43F68047AE499DC617B04864DF80` |
| Standalone `VoxelForgeCoreStandalone.exe` | `4EBE946E4D7B245458CD4FD0FBC3FCD6988B98000685C0043B25CD629AFC0F30` |
## Failed attempts and safety review
- One intermediate BuildHost compile failed because `PlayerHeading` was out of scope in two loader
admission helpers. I replaced those reads with the game-thread `bStreamingPlayerMoving` snapshot;
the final BuildHost and Standalone builds passed.
- Two early external-sampler attempts failed before launching Unreal (a PowerShell job argument
was flattened, then `Get-Counter` requested an unsupported counter). The later sampler ran. No
Unreal run failed; no `0xC0000142` occurred and no retry was needed.
- The final SurfaceWorld command completed under 15 minutes with 3.75 GB peak private memory;
FloatingIslands completed with 3.26 GB. The Windows free-memory query was denied during the latter
launch, but the BuildHost runner's preflight guard allowed it. No Unreal process remains running.
- Standalone compiled with existing warnings: `vswhere.exe` was not found on `PATH`, and
`VoxelStrateManager.cpp` emitted C4838 double-to-float conversion warnings. The `All` build
completed successfully.
- `Saved/BuildHost/HostProject/Saved/Crashes/` contains only the pre-existing
`UECC-Windows-F1A3EFD5469F5DD62E68EBBACAC84376_0000`; no new crash folder appeared.
- Diff review confirmed the result queues remain `EQueueMode::Mpsc`; `EndPlay` waits for active
tasks and drains queues; workers check `bShuttingDown`; epoch and cancellation checks remain;
`ModsLock` still guards mod data; core pages retain shared ownership through readers; and UObject
/ RMC component mutation remains on the game thread. No queue mode or page ownership changed.
- No source file was added. C++ compile spots are `Source/VoxelForge/Private/VoxelWorld.cpp`,
`Source/VoxelForge/Public/VoxelWorld.h`, `Source/VoxelForge/Private/VoxelStartupTrace.cpp`, and
`Source/VoxelForge/Public/VoxelStartupTrace.h` in the BuildHost VoxelForge module. The test-harness
change is PowerShell in `Tools/TestVoxelForgeCoreWorldHost.ps1`; no Core source, generated header,
`.uasset`, owner `.uproject`, or editor was modified or built.
- No commit, push, or stash was created. RealtimeMeshComponent was read-only.
### Dialogue
**User:** Count the workers and their idle gaps, preserve the streaming priorities and safety rules,
and get the full set under six seconds without losing the 40 m/s walkable ring.
**Codex:** The actor had ten TaskGraph workers available but only eight background workers for its
background-priority tile tasks. A queue of eight plus an eight-slot admission cap reduced the
2,578-tile startup to 4.911 s; the moving 3×3 ring stayed ready in all 75 gate samples. The process
remains Normal priority and limited to 12 CPUs. The measured p95 tick increase is small but recorded,
and the worker pool shows why this round did not reach the 3 s ideal for ten workers.