Files
Fr0zkaandClaude Opus 5.5 8440adc742 wip(assembler): round S8 -- region rebuilds no longer stall admission; result backlog capped
voxel.CoreRegionConcurrentAdmission (default on): tiles independent of the
entering region keep submitting while a region page compiles; pages are
immutable shared_ptr owners so running readers survive eviction. 6 m/s: 11
whole-pass skips -> 0, walkable ring ready at every centre change.
voxel.StreamingResultBacklogCap=256 bounds finished results (slow-apply worst
case 125 -> 76 MB). Spawn unchanged at ~7.7 s: LOD8 is 46% of worker time.
At 60 m/s + teleport the collision ring is outrun (102/113 centres).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MmSoF64BbRFmk9C3p1VsBJ
2026-10-19 09:43:03 +02:00

16 KiB
Raw Permalink Blame History

World assembler S8 / M3 Round 12 — moving region admission and bounded results

Result: the S7 region-pending stop is removed for independent tiles. At 6 m/s, the matched negative control logged 11 global admission skips; the new mode logged none and admitted 110 independent tiles during those windows. All 11 changed support centers had a ready 9/9 walkable ring. This did not materially change overall moving worker occupancy: about 110 of 150 moving slot-seconds remained idle. A 60 m/s teleport run admitted unrelated tiles too, but most centers still outran their walkable ring.

The result queue is now separately bounded at 256 queued results + admitted tile tasks + held prequeue results. Under one-result-per-frame apply with a 60 m/s run and 400 m teleport, the tracked result payload peak was 76.2 MB; the uncapped control reached 125.1 MB. Spawn remains about 7.7 s, not the roughly 4.1 s ideal from 41 worker-seconds / 10 slots. The timeline shows a broad LOD8 tail, not one abnormally long tile. No safe low-cost spawn fix was supported by this measurement.

1. Region gate measurement and change

BuildCoreDesiredTiles asks for region pages when the desired tile center changes or its initial desired region set is established. The old bCoreRegionCompilePending path stopped the whole Core submission pass. The new default voxel.CoreRegionConcurrentAdmission=1 continues through LoadCoreTile and refuses only keys in PendingCoreRegionTileSet or CoreRegionCompileTilesInFlight. voxel.CoreRegionConcurrentAdmission=0 restores the old global gate as the negative control. Core tile arithmetic and recipes are unchanged.

Region pages are constructed away from the lookup map and published under tileRegionMutex as immutable shared owners. generateTile copies the matching shared_ptr while holding the map's shared lock, then keeps it for the complete read. Map eviction cannot free a page while an already running tile task holds it. The actor's shared WorldLock prevents edits/resets from mutating the Core world while readers or a region compile are active.

Matched fresh BuildHost processes used the staged owner SurfaceWorld settings, seed 0, 25 cm Core scale, 2,578-key desired set, Legacy order, Dynamic near reserve, apply boost 0, 10 task slots, Normal priority, -corelimit=12, and a +Y move starting at 30 s. Both modes used backlog cap 256.

Moving 6 m/s Region CVar 0 Region CVar 1
Region requests / completions 11 / 11 11 / 11
Reasons 1 initial set, 10 center moves 1 initial set, 10 center moves
Whole-scheduler region skips 11 0
Pending interval sum / worst interval 426.3 / 40.9 ms 433.1 / 41.4 ms
Compile wall sum / slowest compile 10.8 / 2.5 ms 10.5 / 2.4 ms
Independent tile submissions / dependent declines 0 / 0 110 / 0
Changed support centers ready 9/9 at the change 11 / 11 11 / 11
Walkable lag at those centers 0 ms 0 ms
Moving frames with an LOD0 key missing 173 163
Moving worker busy / idle slot-seconds (15 s × 10 slots) 39.7 / 110.3 s 39.9 / 110.1 s

The pending interval is the request-to-completion time recorded for each compile; it is not all global blocking in the new mode. The compiler's measured wall time summed to only about 11 ms over the run. The default mode removes the full-pass skip and submits keys independent of the entering region. Its total worker supply remains low: moving work averages about 2.66 of 10 busy workers. Each of the 11 negative-control whole-pass skips had 92–144 desired keys missing and zero active tile tasks. The CVar1 run has no corresponding global skip. Per-LOD counts added to scheduler_skip make the LOD0-missing-frame count include the 6 globally gated frames that had LOD0 work missing in the negative control.

The fast trace used 60 m/s and teleported 400 m after 2 s of the move. It recorded 113 region requests (initial set plus 112 center moves), all completed, with 381.4 ms aggregate compile wall time (30.8 ms worst) and 4,523.7 ms accumulated pending interval (69.9 ms worst). Concurrent admission recorded 782 independent submissions, 983 dependent-key declines, and zero global region skips. Of 113 changing support centers, 8 were already 9/9 ready on the change event, 3 became ready before the next center change, and 102 did not reach 9/9 before leaving or test end. The worst resolved lag was 106.3 ms. LOD0 was missing on 480 of 488 moving scheduler frames. This speed is beyond what the current collision ring can follow.

2. Spawn, worker timeline and tail

Fresh owner-plan spawn S7 Dynamic reference Round12 region CVar 0 Round12 region CVar 1
Full set (2,578 keys, from BeginPlay) 7.780 s 7.820 s 7.706 s
Walkable 9/9 ring 0.581 s 0.583 s 0.566 s
Full LOD0 (512 tiles) 1.439 s 1.442 s 1.424 s
Worker task-seconds to full set 41.364 41.966 41.229
Integrated busy workers / 10 (first request → ready) 5.63 5.63 5.60
Idle slot-seconds (same interval) 32.056 32.63 32.37
UWorld::Tick spawn p95 / max 2.262 / 5.230 ms 2.387 / 5.273 ms 2.359 / 5.383 ms
UWorld::Tick move p95 / max 2.512 / 28.893 ms 2.472 / 28.257 ms 2.377 / 28.967 ms

The CVar A/B changes the spawn full-set time by only 0.11 s; the region issue occurs during the later move. The Round12 CVar1 first-request-to-ready span was 7.360 s with 41.229 worker-seconds. In one-second bins its worker task time was 3.94, 3.91, 3.34, 6.51, 5.73, 7.75 and 7.81 worker- seconds, followed by 2.24 worker-seconds in the final 0.36 s. Average occupancy was 5.60/10 and idle capacity was 32.37 slot-seconds. No sustained ten-worker plateau appears. That is about 16.0 ms of elapsed task time per initial tile, similar to S7's 16.04 ms and still about 9% above the S7 old-path control's 14.62 ms. It does not show whether the difference is CPU contention or other elapsed-time variation.

The 306 LOD8 jobs consumed 18.798 worker-seconds (45.6% of total), averaging 61.4 ms each; their worker-time p95 was 79.8 ms and max 94.7 ms. The final 20 completed tile records were all LOD8 and landed between 7.587 and 7.765 s. This is a group tail of ordinary large jobs, not one serial tile to split. Across the initial set, worker queue wait averaged 1.71 ms per tile (p95 9.71 ms, max 41.44 ms), and game-thread mesh apply totaled 0.250 s. The trace records elapsed time, not CPU cycles, cache misses or task-pool wakeup causes; it cannot identify memory bandwidth or thread-local cache contention. The 10-slot cap reflects 12 logical cores with two cores reserved by GetMaxConcurrentTasks; I left that reserve in place without evidence that using all 12 improves the spawn while retaining tick time.

The optional actor-tick subsystem markers were not added. The matched run reports the 28.967 ms maximum walk tick, but this trace has no per-call marker at that maximum, so it remains unattributed. These are headless/null-RHI actor tick measurements, not rendered frame timings.

3. Result backlog bound

voxel.StreamingResultBacklogCap defaults to 256 and counts queued results, active tile tasks, and held visible-LOD0 prequeue results. It is distinct from worker-slot admission; finished results still do not occupy slots. Zero disables the cap and is the safety negative control.

Run Queue cap Peak result count Peak payload bytes Cap stops Peak private memory
6 m/s normal move 256 116 17,024,546 0 3.48 GB
60 m/s + 400 m teleport 256 114 17,153,276 0 3.80 GB
60 m/s + teleport, one mesh apply/frame 256 256 76,201,412 843,198 3.60 GB
Same slow-apply control, cap disabled 0 1,577 125,098,792 0 3.63 GB

The slow-apply test still reached its initial set: 23.890 s capped versus 23.493 s uncapped. The cap reduced the tracked peak by 1,321 results and 48.9 MB of payload, at a 0.397 s startup difference in this pair. cap_stops counts repeated admission attempts, not unique tile keys. The atomic peak tracker saw 256 / 1,577 results; periodic result_queue_frame samples saw 255 / 1,576, respectively, because a frame sample can miss the brief peak. Bytes are result payload estimates, excluding queue-node and allocator overhead. The default stops new admissions as work drains; it does not reduce active task capacity when the queue is below the cap.

4. Quality, determinism and async audit

  • BuildHost Core Off legacy passed at 12.5 cm. HybridV1 and HybridV2 each passed five fixtures and same-LOD border checks at 25 cm.
  • Final-module SurfaceWorld passed the full actor, collision, anchor-move and carve checks at 25 cm. Its W1/W8/W16 worker matrix matched: density 0cd2a1a4152ee4aa, mesh 80a5b054b904b1a2, payload bf2f1b40dff38acc, topology 56074198682d5e18. S3 ground agreement passed 128 points: far7/8 maximum error 0.0427 cm, cross-LOD 0.0000 cm, 2 cm tolerance; the 100 cm offset control was detected.
  • FloatingIslands passed the final BuildHost W1/W8/W16 actor matrix at 25 cm, with fixed water, collision, edits and LOD0/1 coverage. The standalone W1/W8/W16 --stream-hash runs matched on all nine LODs. Its host scheduler clamps parallel_for to at most eight actual workers; W16 is the requested worker-limit input routed through that eight-thread executor. The S4 audit passed: 385/385 far-cap bands emitted, 128 agreement points within 0.0504 cm (2 cm tolerance), 100 cm offset detected, and 2,520/2,520 coverage rays hit.
  • No field, mesh or recipe output changed; recipe version remains unchanged. Water remains fixed for the season, and player carve/fill remains unrestricted.

Async review of the diff:

  • FChunkResult::Epoch and desired/modification epochs remain carried and checked. The region task captures generation and desired epochs plus its compile serial.
  • All four result queues remain EQueueMode::Mpsc; result insertion is centralized in EnqueueStreamingResult.
  • Tile and region workers check bShuttingDown; EndPlay sets both shutdown flags, waits for ActiveTaskCount to reach zero, then drains all four result queues.
  • ModsLock is unchanged. Workers generate data only; UObject and RMC mutation remains on the game thread. The region map has its own shared mutex, and readers retain immutable pages with shared_ptr until their tile job ends.

5. Build, commands and run record

BuildHost compiled at Normal priority with 12 actions. Link-safety passed (96,279 dumpbin lines checked against 421 names); the five staged asset hashes and 18 manifest shims passed. The final SurfaceWorld quality run took about 10 minutes and peaked at 3.896 GB private memory. The standalone full build passed with the existing vswhere.exe lookup warning and existing C4838 conversions in VoxelStrateManager.cpp:3974–3975.

Build commands:

.\Tools\BuildVoxelForgeCoreHost.ps1 -MaxParallelActions 12 -TimeoutSeconds 900
.\Standalone\Build.ps1 -Target All -CompileWorkers 12 -BuildPriority Normal

Matched region-gate commands (cap held at 256 for both sides):

.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StartupApplyBoost 0 -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -CoreRegionConcurrentAdmission 0 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 600 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StartupApplyBoost 0 -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 600 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StartupApplyBoost 0 -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 6000 -TestTeleportAfterSeconds 2 -TestTeleportDistanceCm 40000 -TimeoutSeconds 900

Slow-apply backlog controls used these commands; the cap-off run is the backlog negative control:

.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StartupApplyBoost 0 -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 6000 -TestTeleportAfterSeconds 2 -TestTeleportDistanceCm 40000 -Step1MaxMeshApplies 1 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StartupApplyBoost 0 -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 0 -TestMoveSpeedCmPerSecond 6000 -TestTeleportAfterSeconds 2 -TestTeleportDistanceCm 40000 -Step1MaxMeshApplies 1 -TimeoutSeconds 900

Other BuildHost checks were:

.\Tools\TestVoxelForgeCoreWorldHost.ps1 -CoreOffTest -CoreVoxelSizeCm 12.5 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -HybridFixturesOnly -CoreMesherRecipe WorldAssemblerHybridV1 -CoreVoxelSizeCm 25 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -HybridFixturesOnly -CoreMesherRecipe WorldAssemblerHybridV2 -CoreVoxelSizeCm 25 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -CoreVoxelSizeCm 25 -TimeoutSeconds 900
.\Tools\TestVoxelForgeCoreWorldHost.ps1 -FloatingIslandsOnly -CoreVoxelSizeCm 25 -TimeoutSeconds 900

Standalone determinism used three fresh processes (--workers=1, 8, and 16), each with:

.\Standalone\bin\VoxelForgeCoreStandalone.exe --workers=1 --world-assembler-prototype --stream-benchmark --stream-hash --force-generate --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --process-priority=Normal --process-affinity=4095 --frame-limit=60

The S4 standalone audit command was:

.\Standalone\bin\VoxelForgeCoreStandalone.exe --world-assembler-audit --workers=12 --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --process-priority=Normal --process-affinity=4095 --frame-limit=60

SHA-256 of the executables/modules used by the final runs:

Binary SHA-256
UE 5.7 UnrealEditor-Cmd.exe ECCE9996191F7F7E69B3D104EECDD0E678AE69176FDCCD79828C3E75A0654D6D
BuildHost UnrealEditor-VoxelForge.dll 8B82EE6C6FBB511AE766EB3D8646A7E9491952F41445CB62D49D71D0B9783418
BuildHost UnrealEditor-VoxelForgeCore.dll 0A1EB7BB27C3C74A4AA0996E0A24E74224D7B8FFFFF113B1071FA42E18E8375A
BuildHost UnrealEditor-VoxelForgeEditor.dll B3BC5E61172C5B21D928A5ACB5F22F8D9E00179B481B675804838DE9A620010D
Standalone/bin/VoxelForgeCoreStandalone.exe 9C1601D7FCACCAAC16A2BA9DF0E7826E18746028A4D08FD23EEC479D2F5745BE

Trace logs and queue records are under Saved/BuildHost/HostProject/Plugins/VoxelForge/Saved/CoreWorldValidation/Logs/. The final quality logs and standalone determinism/audit stdout are under Saved/BuildHost/Round12/. No new file over 300 KB was placed in Research/.

BuildHost and standalone launch guards checked free physical memory before each process and waited for at least 4 GiB. Explicit preflights for the final matched run and SurfaceWorld actor test saw 17.33 and 17.11 GiB free. All processes stayed under 8 GiB and used Normal priority / at most 12 threads. The only crash-folder entry is the pre-existing UECC-Windows-F1A3EFD5469F5DD62E68EBBACAC84376_0000 from 2026-09-17; no new UECC or 0xC0000142 occurred. Three exploratory PowerShell RAM-probe definitions failed before launching Unreal; after correcting the interop declaration, the probe and all actual build/test runs passed on attempt 1. There were no failed Unreal runs or retries.

No new source file was added. Likely compile spots are Source/VoxelForge/Private/VoxelWorld.cpp, Source/VoxelForge/Public/VoxelWorld.h, Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp, Standalone/Core/src/VoxelForgeCore.cpp and its public header, plus the PowerShell harness. I did not open the owner's editor, build VoxelM.uproject, edit any .uasset, or modify RealtimeMeshComponent. No commit, push or stash was made.