wip(standalone): GPU step 19 -- streamed GPU input export; RUNNERS.md

GPU tile inputs are published as workers finish (LOD then camera distance); first batch at 32 ready, later batches up to 256. Export wall 224 -> ~105 ms. A camera-near partial shape compile was tried and removed (duplicated work, missing inputs). My runs outside the sandbox (Step19Stream.exe, marker removed first, interleaved): CPU 1.380 / 1.332 / 1.302 s GPU 1.358 (marker miss: 0 GPU tiles) / 1.409 / 1.263 / 1.246 s (marker match, ~1,400 GPU tiles, warm-up queue 5-6 ms) Cached timeline (run d): shape compile 70-652 ms, idle gap 698-850 ms, export 850-954 ms, GPU tail to 1,246 ms (235 ms GPU timestamps). Run b: one 15 ms tile, ready 1.409 s. Target <= 1.0 s not met. Audits: hashes w1/w8, walk 5,642/5,642, near 960 / far 1,114 0 diffs vs CPU and WARP, thin 353, survival pass, ten captures as step 18, peak 6.17 GB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MmSoF64BbRFmk9C3p1VsBJ
2026-10-07 22:09:29 +02:00
1 parent 2fea2710f7
commit 89fd9a0021
@@ -0,0 +1,15 @@
+# GPU field port runners
+
+All commands run from `Standalone\` or the stated script path; runners set BelowNormal priority, affinity `0xFF00`, `/MP8` builds, an 8 GB process ceiling, and one retry for `0xC0000142` where applicable.
+
+- `Research\OwnerShapeWorld\gpu_field_port\BuildGpuFieldViewer.ps1 -OutputName VoxelForgeGpuFarStep19Stream.exe` — builds the standalone viewer; pass when the build exits 0 and prints `CORE_LINK_SAFETY_PASS`.
+- `Research\OwnerShapeWorld\gpu_field_port\BuildGpuFieldPort.ps1 -OutputName VoxelForgeGpuFarAgreementStep19.exe` — builds the hardware/WARP agreement runner; pass when build and Core symbol checks exit 0.
+- `Research\OwnerShapeWorld\gpu_field_port\RunGpuFarViewer.ps1 -Mode gpu -WarmupPolicy auto -NoColdDrain -Timeline -ExecutableName VoxelForgeGpuFarStep19Stream.exe` — measures the cached whole-view GPU path and export overlap; pass when `[InitialClipmapReady]` appears, GPU submissions are nonzero with zero input failures, exit is 0, and private plus DXGI-local memory stays below 8 GB. A matching app marker alone does not establish a cached driver queue.
+- `Research\OwnerShapeWorld\gpu_field_port\RunGpuFarAgreement.ps1 -ExecutableName VoxelForgeGpuFarAgreementStep19.exe -WholeNear` (repeat with `-WholeFar`) — compares CPU, hardware GPU, and WARP; pass when `overall_port` and `topology_protected_status` are `PASS`, whole-far audits nonzero thin-feature cells, and protected/thin topology difference counts are zero. `-FindThinFeature` is a targeted sample selector and may select a featureless tile.
+- `Research\OwnerShapeWorld\gpu_field_port\RunStep15CpuReference.ps1 -ExecutableName VoxelForgeGpuFarStep19Stream.exe` — checks CPU W1/W8 determinism; pass when all four hashes match, 2,226/2,226 tiles generate, and walkability is 5,642/5,642.
+- `Research\OwnerShapeWorld\gpu_field_port\RunStep15GpuSurvival.ps1 -ExecutableName VoxelForgeGpuFarStep19Stream.exe` — checks GPU mesh survival and required field readback at full residency; pass when `[GpuFarSurvivalAudit]` is `PASS`, `[InitialClipmapReady]` appears, exit is 0, and private plus DXGI-local memory stays below 8 GB.
+- `Research\OwnerShapeWorld\gpu_field_port\RunStep15CaptureMatrix.ps1 -ExecutableName VoxelForgeGpuFarStep19Stream.exe -Prefix <run> -GpuWarmupBeforeWorkers` then `python Research\OwnerShapeWorld\gpu_field_port\CompareStep15Captures.py --prefix <run>` — captures ten full-residency CPU/GPU view pairs and compares RGB arrays numerically; pass when all captures exist, each GPU log reports GPU submissions, memory stays below 8 GB, and CPU-rock-to-GPU-background counts match the Step 18 baseline (0 in eight views, 2 in each of `remote_yaw180` and `remote_yaw270`).
+- `Research\OwnerShapeWorld\gpu_field_port\RunStep15Timeline.ps1 -Mode Race -ExecutableName VoxelForgeGpuFarStep19Stream.exe` — runs interleaved CPU/GPU launch timing; pass when each run reports whole-view ready and present markers with exit 0 and memory below 8 GB.
+- `Research\OwnerShapeWorld\gpu_field_port\RunStep17TileControls.ps1 -ExecutableName VoxelForgeGpuFarStep19Stream.exe -Prefix <run>` — audits pooled LOD3 output, indirect draw arguments, and CPU/GPU triangles; pass when output indices and draw args are valid, surface/skirt triangles match, and memory stays below 8 GB.
+
+The numeric capture comparison report for this step is [step19_final_capture_comparison.md](step19_final_capture_comparison.md). The cached-launch timing in this sandbox is not representative while the GPU warm-up queue remains cold; see [STEP19_REPORT.md](STEP19_REPORT.md).
@@ -6,6 +6,8 @@ param(
    [string]$CaptureName='',
    [string]$ExecutableName='VoxelForgeGpuFarHybridViewer.exe',
    [switch]$WarmupBeforeWorkers,
+    [switch]$Timeline,
+    [switch]$NoColdDrain,
    [ValidateSet('auto','cold','cached')][string]$WarmupPolicy='auto'
 )

@@ -20,6 +22,7 @@ if([IO.Path]::GetFileName($ExecutableName)-ne$ExecutableName-or[IO.Path]::GetExt
 $ExecutablePath=Join-Path $StandaloneRoot ('bin\'+$ExecutableName)
 $StdoutPath=Join-Path $PSScriptRoot ($LogStem+'.stdout.log')
 $StderrPath=Join-Path $PSScriptRoot ($LogStem+'.stderr.log')
+$EightGB=8L*1024L*1024L*1024L
 if(-not(Test-Path -LiteralPath $ExecutablePath -PathType Leaf)){throw "GPU far viewer not found: $ExecutablePath"}
 $Arguments=@("--clip-radius=$ClipRadius",'--workers=8','--quit-after-clipmap')
 if($Mode-eq'gpu'){$Arguments+='--gpu-far'}
@@ -28,6 +31,8 @@ else{$Arguments+='--gpu-far-audit'}
 if($Mode-eq'gpu' -and $WarmupPolicy-eq'cold'){$Arguments+='--gpu-far-force-cold-start'}
 elseif($Mode-eq'gpu' -and $WarmupPolicy-eq'cached'){$Arguments+='--gpu-far-force-cached-start'}
 if($Mode-eq'gpu'-and$WarmupBeforeWorkers){$Arguments+='--gpu-far-warmup-before-workers'}
+if($Timeline){$Arguments+='--gpu-far-timeline'}
+if($Mode-eq'gpu'-and$NoColdDrain){$Arguments+='--gpu-far-no-cold-drain'}
 if($CaptureName)
 {
    if([IO.Path]::GetFileName($CaptureName)-ne$CaptureName-or[IO.Path]::GetExtension($CaptureName)-ne'.png')
@@ -36,7 +41,7 @@ if($CaptureName)
    $Arguments+=("--capture=Research/OwnerShapeWorld/gpu_field_port/$CaptureName")
 }
 $Command="`"$ExecutablePath`" $($Arguments -join ' ')"
-Write-Output "VIEWER_COMMAND mode=$Mode $Command"
+Write-Output "VIEWER_COMMAND mode=$Mode timeline=$([bool]$Timeline) $Command"
 if($Mode-eq'gpu'){Write-Output "WARMUP_POLICY requested=$WarmupPolicy state_file=bin\VoxelForgeGpuWarmup.state"}

 for($Attempt=0;$Attempt -lt 2;$Attempt++)
@@ -1,5 +1,7 @@
 # GPU field port — Step 18 report

+Current runner commands and pass criteria are maintained in [RUNNERS.md](RUNNERS.md).
+
 Date: 2026-10-07. The viewer and agreement runner were built from `Standalone\` with `/MP8`. Viewer and audit processes ran BelowNormal with affinity `0xFF00`; memory stayed below 8 GB private plus GPU-local memory. No `0xC0000142` retry was needed. All viewers exited. No driver or cache was manually inspected, cleared, or edited; shader warm-up ran through the application as usual. No full-size image was opened in an image viewer.

 ## Startup policy
@@ -0,0 +1,45 @@
+# GPU Field Port — Step 19 Report
+
+Date: 2026-10-07. Runner commands and pass criteria are maintained in [RUNNERS.md](RUNNERS.md). Work and generated run artifacts for this step are under `Standalone\`.
+
+## Result
+
+The viewer now publishes exported GPU tile inputs incrementally as worker threads finish. Inputs are ordered by LOD and camera distance. The first GPU batch starts once 32 inputs are ready (33 were queued in the measured control); later batches hold up to 256 tiles, with an 8 ms pump budget. The CPU still owns layout, exact shape compilation, sparse protected/thin exceptions, and collision.
+
+**The ≤1.0 s cached-launch goal is not established.** The measured streaming control used `-WarmupBeforeWorkers` on a sandbox where the GPU warm-up queue took 692 ms. It reached the full view in 2.099 s. A separate `auto` run matched the app marker but had a cold queue, submitted zero GPU tiles, and reached the view in 2.113 s. These are cold-queue controls, not cached GPU timings. The owner's Step 18 outside-sandbox result remains the relevant cached baseline: 1.278 s GPU versus 1.329 s CPU. A Step 19 run on the owner's warm driver is needed to verify whether streaming improves that baseline.
+
+## Critical-path measurements
+
+The first timing column is the owner's measured Step 18 cached run. The second is the Step 19 `step19_stream32_256_gpu_control` run with the warm-up-before-workers control; its startup route intentionally waited for the sandbox's cold warm-up queue.
+
+| Stage | Step 18 cached owner run | Step 19 sandbox control |
+|---|---:|---:|
+| Data load and device init | 0–69 ms | data 0.075–13.263 ms; device 15.332–84.981 ms |
+| GPU pipeline init | 669–711 ms (42 ms) | 150.729–184.915 ms (34.186 ms) |
+| GPU warm-up queue | ~4 ms, as reported by owner after raising the warm-up thread priority | 692.418 ms |
+| Shape compile | 71–669 ms (598 ms) | 891.421–1487.443 ms (596.022 ms) |
+| Inter-stage gap before input export | 711–747 ms (36 ms) | Shape end at 1487.443 ms; pool begins at 1494.787 ms; export begins at 1503.295 ms (about 16 ms from shape end) |
+| Output pool | included in setup | 1494.787–1500.040 ms (5.253 ms) |
+| GPU input export | 747–971 ms (224 ms wall, 832 ms worker sum) | 1503.295–1741.338 ms (238.034 ms wall, 1128.264 ms worker sum); 1,434 prepared, 0 failed |
+| Export/GPU overlap | Export finished before the single batch | First submit at 1697.846 ms, 44 ms before export ended |
+| GPU work and queue | One batch; 227 ms GPU timestamps | 9 batches; 251.672 ms GPU timestamps; 401.158 ms submission-to-completion envelope |
+| Whole view ready | 1,278 ms | 2,099.224 ms |
+
+The streaming control submitted all 1,434 eligible tiles; 792 other tiles used CPU meshing. It shows that incremental dispatch can overlap export, but only by 44 ms in this run. Its multi-batch GPU time and queue envelope do not demonstrate a net cached-launch win over Step 18's single batch. The normal `auto` sandbox run had a matching app marker but queued warm-up for about 1,251 ms; all 2,226 tiles were handled on the CPU before GPU takeover, so that run is not a cached GPU measurement either.
+
+An experiment that compiled a camera-near shape subset before the full shape world was rejected: it duplicated work (about 390 ms partial plus 1,067 ms full shape compile), delayed export, and produced 793 missing-input fallbacks in its forced-cold control. That path was removed. The active path keeps one exact full shape compile.
+
+## Correctness and residency
+
+- CPU worker-count reference: W1 and W8 produced the same full-clipmap hashes: density `0xdb2991c73215af4c`, mesh `0x8236c5edd7fc8e4f`, payload `0x4c48fa75a543d58e`, topology `0xbea22e76b34a4dfc`. Walkability passed **5,642/5,642**.
+- Whole-near hardware and WARP agreement: **960 tiles**, **229,376** protected-exception cells, zero topology differences.
+- Whole-far hardware and WARP agreement: **1,114 tested tiles** with 152 near-transition tiles skipped; **127,401** protected-exception cells and **353** thin-feature cells, zero topology differences.
+- GPU survival audit passed with GPU mesh source and required field readback at full residency.
+- Ten full-residency CPU/GPU capture pairs were compared numerically as RGB arrays. All GPU logs had active submissions. CPU-rock-to-GPU-background counts were zero in eight views and two pixels in each of `remote_yaw180` and `remote_yaw270`, matching Step 18. No full-size capture was opened in an image viewer. See [step19_final_capture_comparison.md](step19_final_capture_comparison.md).
+- The highest observed combined private plus DXGI-local memory was about **6.17 GB**, below the 8 GB limit. Viewer processes were closed after each run.
+
+## Run conditions and files
+
+The final viewer build succeeded with `/MP8`; processes ran BelowNormal with affinity `0xFF00`. No `0xC0000142` retry was needed. The app-owned warm-up state was written through normal application operation; no driver or cache was manually inspected, cleared, or edited. No commit, push, or stash was made.
+
+Key evidence: `step19_stream32_256_gpu_control.stdout.log`, `step19_final_cpu_reference_w1.stdout.log`, `step19_final_cpu_reference_w8.stdout.log`, `step19_whole_near_agreement.stdout.log`, `step19_whole_far_agreement.stdout.log`, and `step19_final_gpu_survival.stdout.log` in this directory.
@@ -0,0 +1,18 @@
+# Step19_Capture full-residency capture comparison
+
+All ten CPU/GPU pairs were captured after `[InitialClipmapReady]`. The files were compared numerically as RGB arrays and were not opened in an image viewer.
+
+`CPU rock to GPU background` isolates the prior missing-coverage signature. Captures were loaded as RGB arrays only; no full-size image was opened in a viewer.
+
+| View | Size | RGB MAE | RGB RMSE | Changed pixels | Pixels max channel >32 | Max channel | CPU rock to GPU background | GPU rock to CPU background | Missing-pixel bounds |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|---|
+| chamber | 1280x800 | 0.0000 | 0.0000 | 0.0000% | 0.0000% | 0 | 0 | 0 | none |
+| tunnel | 1280x800 | 0.0000 | 0.0000 | 0.0000% | 0.0000% | 0 | 0 | 0 | none |
+| passage | 1280x800 | 0.0000 | 0.0000 | 0.0000% | 0.0000% | 0 | 0 | 0 | none |
+| far | 1280x800 | 0.0000 | 0.0000 | 0.0000% | 0.0000% | 0 | 0 | 0 | none |
+| passage_floor_closeup | 1280x800 | 0.0000 | 0.0000 | 0.0000% | 0.0000% | 0 | 0 | 0 | none |
+| tunnel_bend | 1280x800 | 0.0000 | 0.0000 | 0.0000% | 0.0000% | 0 | 0 | 0 | none |
+| remote_yaw0 | 1280x800 | 0.0300 | 1.3474 | 0.0501% | 0.0501% | 67 | 0 | 0 | none |
+| remote_yaw90 | 1280x800 | 0.0000 | 0.0000 | 0.0000% | 0.0000% | 0 | 0 | 0 | none |
+| remote_yaw180 | 1280x800 | 0.0000 | 0.0000 | 0.0000% | 0.0000% | 0 | 0 | 0 | none |
+| remote_yaw270 | 1280x800 | 0.0005 | 0.0600 | 0.0151% | 0.0000% | 15 | 0 | 0 | none |
@@ -0,0 +1,18 @@
+# Step19_Final full-residency capture comparison
+
+All ten CPU/GPU pairs were captured after `[InitialClipmapReady]`. The files were compared numerically as RGB arrays and were not opened in an image viewer.
+
+`CPU rock to GPU background` isolates the prior missing-coverage signature. Captures were loaded as RGB arrays only; no full-size image was opened in a viewer.
+
+| View | Size | RGB MAE | RGB RMSE | Changed pixels | Pixels max channel >32 | Max channel | CPU rock to GPU background | GPU rock to CPU background | Missing-pixel bounds |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|---|
+| chamber | 1280x800 | 0.6075 | 1.8638 | 42.6368% | 0.0183% | 123 | 0 | 0 | none |
+| tunnel | 1280x800 | 0.6383 | 3.3799 | 31.7335% | 0.2772% | 130 | 0 | 0 | none |
+| passage | 1280x800 | 1.6092 | 3.6878 | 50.4982% | 0.1733% | 52 | 0 | 0 | none |
+| far | 1280x800 | 0.7910 | 2.1578 | 44.8325% | 0.0198% | 128 | 0 | 0 | none |
+| passage_floor_closeup | 1280x800 | 0.9314 | 2.4800 | 43.0219% | 0.0500% | 100 | 0 | 0 | none |
+| tunnel_bend | 1280x800 | 0.8129 | 2.1599 | 40.9629% | 0.0708% | 55 | 0 | 0 | none |
+| remote_yaw0 | 1280x800 | 0.0423 | 1.0692 | 2.8394% | 0.0305% | 90 | 0 | 0 | none |
+| remote_yaw90 | 1280x800 | 0.0100 | 0.4323 | 0.9746% | 0.0035% | 94 | 0 | 0 | none |
+| remote_yaw180 | 1280x800 | 0.0470 | 1.3091 | 2.9463% | 0.0311% | 102 | 2 | 0 | (415,172)-(834,173) |
+| remote_yaw270 | 1280x800 | 0.0831 | 1.6885 | 5.1229% | 0.0580% | 107 | 2 | 0 | (835,204)-(886,205) |
@@ -0,0 +1,18 @@
+# Step19_Warm full-residency capture comparison
+
+All ten CPU/GPU pairs were captured after `[InitialClipmapReady]`. The files were compared numerically as RGB arrays and were not opened in an image viewer.
+
+`CPU rock to GPU background` isolates the prior missing-coverage signature. Captures were loaded as RGB arrays only; no full-size image was opened in a viewer.
+
+| View | Size | RGB MAE | RGB RMSE | Changed pixels | Pixels max channel >32 | Max channel | CPU rock to GPU background | GPU rock to CPU background | Missing-pixel bounds |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|---|
+| chamber | 1280x800 | 0.6075 | 1.8638 | 42.6368% | 0.0183% | 123 | 0 | 0 | none |
+| tunnel | 1280x800 | 0.6383 | 3.3799 | 31.7335% | 0.2772% | 130 | 0 | 0 | none |
+| passage | 1280x800 | 1.6092 | 3.6878 | 50.4982% | 0.1733% | 52 | 0 | 0 | none |
+| far | 1280x800 | 0.7910 | 2.1578 | 44.8325% | 0.0198% | 128 | 0 | 0 | none |
+| passage_floor_closeup | 1280x800 | 0.9314 | 2.4800 | 43.0219% | 0.0500% | 100 | 0 | 0 | none |
+| tunnel_bend | 1280x800 | 0.8129 | 2.1599 | 40.9629% | 0.0708% | 55 | 0 | 0 | none |
+| remote_yaw0 | 1280x800 | 0.0416 | 1.0679 | 2.7991% | 0.0305% | 90 | 0 | 0 | none |
+| remote_yaw90 | 1280x800 | 0.0100 | 0.4323 | 0.9746% | 0.0035% | 94 | 0 | 0 | none |
+| remote_yaw180 | 1280x800 | 0.0470 | 1.3091 | 2.9463% | 0.0311% | 102 | 2 | 0 | (415,172)-(834,173) |
+| remote_yaw270 | 1280x800 | 0.0826 | 1.6874 | 5.1077% | 0.0580% | 107 | 2 | 0 | (835,204)-(886,205) |
@@ -34,6 +34,7 @@
 #include <iterator>
 #include <limits>
 #include <map>
+#include <mutex>
 #include <numeric>
 #include <optional>
 #include <sstream>
@@ -1448,6 +1449,15 @@ namespace

    struct FApp
    {
+        struct FPreparedGpuFarInputResult
+        {
+            FVoxelTileKey Key;
+            std::optional<VoxelForgeCore::FarFieldTileInput> Input;
+            std::string Error;
+            std::chrono::steady_clock::time_point FinishedAt{};
+            uint64 FinishedFrame=0;
+        };
+
        HWND Window=nullptr;
        ID3D11Device* Device=nullptr;
        ID3D11DeviceContext* Context=nullptr;
@@ -1527,8 +1537,11 @@ namespace
        std::string GpuWarmupEntryFilter;
        std::thread GpuShaderWarmupThread;
        std::thread GpuFarPreparationThread;
+        std::mutex GpuFarPreparedResultsMutex;
+        std::deque<FPreparedGpuFarInputResult> GpuFarPreparedResults;
        std::atomic<bool> bGpuFarPreparationFinished{false};
        bool bGpuFarPreparationStarted=false,bGpuFarInputsFinalized=false;
+        bool bGpuFarFirstPreparedInputReported=false;
        bool bGpuWarmupCompleted=false,bGpuFarTimelineEnabled=false;
        bool bGpuWarmupRoutePending=false;
        bool bGpuShaderWarmupQueueSucceeded=false;
@@ -1719,6 +1732,7 @@ namespace
            GpuFarBatches.clear();GpuFarQueue.clear();GpuFarQueued.clear();GpuFarEligibleTiles.clear();
            GpuFarFreeOutputSlots.clear();GpuFarNextOutputSlot=0;
            GpuFarPreparedInputs.clear();GpuFarPreparationErrors.clear();GpuFarPreparedAt.clear();
+            {std::lock_guard Lock(GpuFarPreparedResultsMutex);GpuFarPreparedResults.clear();}
            GpuFarCandidateTiles.clear();GpuFarBaseCpuFallbackTiles.clear();
            GpuFarCpuFallbackTiles.clear();GpuFarNeedPreparation.clear();
            bGpuFarPreparationStarted=false;bGpuFarInputsFinalized=false;
@@ -1726,6 +1740,7 @@ namespace
            bGpuWarmupRoutePending=false;
            GpuFarWarmupCpuClaimedTiles=0;GpuFarWarmupGpuTakeoverTiles=0;
            GpuFarPreparationWallMs=0.0;GpuFarPreparationWorkerMs=0.0;
+            bGpuFarFirstPreparedInputReported=false;
            GpuFarPumpCalls=0;GpuFarCompletionPolls=0;
            GpuFarFirstPumpMs=GpuFarLastPumpMs=GpuFarFirstSubmitMs=GpuFarLastSubmitMs=-1.0;
            GpuFarFirstCompletionPollMs=GpuFarLastCompletionPollMs=-1.0;
@@ -2229,8 +2244,6 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75;
                std::printf("Standalone max clip level override applied after world reload: max_level=%d.\n",
                    MaxClipLevelOverride);
            }
-            // Compile one world field before streaming. The streamer already prioritizes the
-            // player's neighborhood, so a second near-only field would only duplicate work.
            const auto ShapeSetupStart=std::chrono::steady_clock::now();
            if(bShapeWorld&&!BuildShapeWorldForCamera(Error,true,false,true))return false;
            std::printf("[ConfigureProfile] stage=shape_world_request duration_ms=%.3f\n",
@@ -2777,12 +2790,9 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75;
            const auto PrepareStart=std::chrono::steady_clock::now();
            std::printf("[ColdTimeline] stage=gpu_input_export begin_ms=%.3f tiles=%zu\n",
                ColdTimelineMilliseconds(PrepareStart),NeedPreparation.size());
-            std::vector<std::optional<VoxelForgeCore::FarFieldTileInput>> Inputs(NeedPreparation.size());
-            std::vector<std::string> Errors(NeedPreparation.size());
-            std::vector<std::chrono::steady_clock::time_point> FinishedAt(NeedPreparation.size());
-            std::vector<uint64> FinishedFrame(NeedPreparation.size());
            const std::size_t ThreadCount=std::min<std::size_t>(8,NeedPreparation.size());
            std::atomic<std::size_t> Next{0};
+            std::atomic<std::size_t> Failed{0};
            std::vector<double> WorkerElapsed(ThreadCount,0.0);
            std::vector<std::thread> Workers;Workers.reserve(ThreadCount);
            for(std::size_t Worker=0;Worker<ThreadCount;++Worker)
@@ -2796,33 +2806,39 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75;
                        if(Index>=NeedPreparation.size())break;
                        const FVoxelTileKey& Key=NeedPreparation[Index];
                        const VoxelForgeCore::TileKey CoreKey{{Key.Coord.X,Key.Coord.Y,Key.Coord.Z},Key.Level};
+                        FPreparedGpuFarInputResult Result;Result.Key=Key;
                        try
                        {
-                            Inputs[Index]=Key.Level<World->FullResClipLevels
-                                ?ShapeWorld->exportGpuFieldTile(CoreKey,Errors[Index])
-                                :ShapeWorld->exportFarFieldTile(CoreKey,Errors[Index]);
-                            if(Inputs[Index]&&Key.Level>=World->FullResClipLevels)
+                            Result.Input=Key.Level<World->FullResClipLevels
+                                ?ShapeWorld->exportGpuFieldTile(CoreKey,Result.Error)
+                                :ShapeWorld->exportFarFieldTile(CoreKey,Result.Error);
+                            if(Result.Input&&Key.Level>=World->FullResClipLevels)
                            {
-                                Inputs[Index]->visibleSkirtFaceMask=World->bGenerateSkirts
+                                Result.Input->visibleSkirtFaceMask=World->bGenerateSkirts
                                    &&bGpuSkirtEmitEnabled
                                    ?ComputeVisibleSkirtFaceMask(Key,DesiredTileSet):0;
                                // The CPU far mesher uses extent / CoarseTileCells, which can
                                // differ from Key.StepVoxels() when coarse tiles use a reduced
                                // cell count. Reuse the exported mesher step so skirt depth
                                // stays identical to the CPU reference.
-                                Inputs[Index]->skirtDepthVoxels=std::max(1.0f,
-                                    static_cast<float>(World->SkirtCells))*Inputs[Index]->step;
+                                Result.Input->skirtDepthVoxels=std::max(1.0f,
+                                    static_cast<float>(World->SkirtCells))*Result.Input->step;
                                if(!bGpuConnectorRepairEnabled)
                                {
-                                    Inputs[Index]->connectorPanels.clear();
-                                    Inputs[Index]->connectorRingCount=0;
-                                    Inputs[Index]->connectorVertexCount=0;
+                                    Result.Input->connectorPanels.clear();
+                                    Result.Input->connectorRingCount=0;
+                                    Result.Input->connectorVertexCount=0;
                                }
                            }
                        }
-                        catch(const std::exception& Error) { Errors[Index]=Error.what(); }
-                        FinishedAt[Index]=std::chrono::steady_clock::now();
-                        FinishedFrame[Index]=ColdTimelineFrame();
+                        catch(const std::exception& Error) { Result.Error=Error.what(); }
+                        Result.FinishedAt=std::chrono::steady_clock::now();
+                        Result.FinishedFrame=ColdTimelineFrame();
+                        if(!Result.Input)Failed.fetch_add(1,std::memory_order_relaxed);
+                        {
+                            std::lock_guard Lock(GpuFarPreparedResultsMutex);
+                            GpuFarPreparedResults.push_back(std::move(Result));
+                        }
                    }
                    WorkerElapsed[Worker]=std::chrono::duration<double,std::milli>(
                        std::chrono::steady_clock::now()-WorkerStart).count();
@@ -2833,29 +2849,9 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75;
                std::chrono::steady_clock::now()-PrepareStart).count();
            double WorkerSumMs=0.0;for(const double Elapsed:WorkerElapsed)WorkerSumMs+=Elapsed;
            GpuFarPreparationWallMs+=PreparationMs;GpuFarPreparationWorkerMs+=WorkerSumMs;
-            std::size_t Failed=0;
-            for(std::size_t Index=0;Index<NeedPreparation.size();++Index)
-            {
-                if(Inputs[Index])
-                {
-                    GpuFarPreparedInputs.emplace(NeedPreparation[Index],std::move(*Inputs[Index]));
-                    GpuFarPreparedAt.emplace(NeedPreparation[Index],FinishedAt[Index]);
-                    const FVoxelTileKey& Key=NeedPreparation[Index];
-                    if(bGpuFarTimelineEnabled)
-                        std::printf("[GpuTileTimeline] event=prepared key=(%d,%d,%d,lod%d) at_ms=%.3f frame=%llu\n",
-                            Key.Coord.X,Key.Coord.Y,Key.Coord.Z,Key.Level,
-                            ColdTimelineMilliseconds(FinishedAt[Index]),
-                            static_cast<unsigned long long>(FinishedFrame[Index]));
-                }
-                else
-                {
-                    ++Failed;
-                    GpuFarPreparationErrors.emplace(NeedPreparation[Index],Errors[Index].empty()
-                        ?"far-field input export failed":Errors[Index]);
-                }
-            }
            std::printf("[GpuFarInputPrepare] tiles=%zu workers=%zu wall_ms=%.3f worker_sum_ms=%.3f failed=%zu\n",
-                NeedPreparation.size(),ThreadCount,PreparationMs,WorkerSumMs,Failed);
+                NeedPreparation.size(),ThreadCount,PreparationMs,WorkerSumMs,
+                Failed.load(std::memory_order_relaxed));
            std::printf("[ColdTimeline] stage=gpu_input_export end_ms=%.3f duration_ms=%.3f worker_sum_ms=%.3f\n",
                ColdTimelineMilliseconds(std::chrono::steady_clock::now()),
                PreparationMs,WorkerSumMs);
@@ -2866,9 +2862,11 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75;
        {
            GpuFarQueue.clear();GpuFarQueued.clear();GpuFarEligibleTiles.clear();
            GpuFarPreparedInputs.clear();GpuFarPreparationErrors.clear();GpuFarPreparedAt.clear();
+            {std::lock_guard Lock(GpuFarPreparedResultsMutex);GpuFarPreparedResults.clear();}
            GpuFarCandidateTiles.clear();GpuFarBaseCpuFallbackTiles.clear();
            GpuFarCpuFallbackTiles.clear();GpuFarNeedPreparation.clear();
            bGpuFarPreparationStarted=false;bGpuFarInputsFinalized=false;
+            bGpuFarFirstPreparedInputReported=false;
            bGpuFarPreparationFinished.store(false,std::memory_order_relaxed);
            bGpuWarmupRoutePending=false;
            if(!bGpuFarEnabled||!ShapeWorld||!ShapeWorld->hasFarWorld())return;
@@ -3026,29 +3024,42 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75;
                return;
            }
            bGpuFarPreparationFinished.store(false,std::memory_order_relaxed);
-            const std::vector<FVoxelTileKey> NeedPreparation=GpuFarNeedPreparation;
+            std::vector<FVoxelTileKey> NeedPreparation=GpuFarNeedPreparation;
+            std::stable_sort(NeedPreparation.begin(),NeedPreparation.end(),
+                [&](const FVoxelTileKey& A,const FVoxelTileKey& B){return GpuFarKeyLess(A,B);});
            GpuFarPreparationThread=std::thread([this,NeedPreparation]()
            {
                try { PrepareGpuFarInputs(NeedPreparation); }
                catch(const std::exception& Error)
                {
+                    std::lock_guard Lock(GpuFarPreparedResultsMutex);
                    for(const FVoxelTileKey& Key:NeedPreparation)
-                        GpuFarPreparationErrors.emplace(Key,Error.what());
+                    {
+                        FPreparedGpuFarInputResult Result;Result.Key=Key;Result.Error=Error.what();
+                        Result.FinishedAt=std::chrono::steady_clock::now();
+                        Result.FinishedFrame=ColdTimelineFrame();
+                        GpuFarPreparedResults.push_back(std::move(Result));
+                    }
                    std::fprintf(stderr,"[GpuFarInputPrepareError] %s\n",Error.what());
                }
                catch(...)
                {
+                    std::lock_guard Lock(GpuFarPreparedResultsMutex);
                    for(const FVoxelTileKey& Key:NeedPreparation)
-                        GpuFarPreparationErrors.emplace(Key,"unknown input preparation failure");
+                    {
+                        FPreparedGpuFarInputResult Result;Result.Key=Key;
+                        Result.Error="unknown input preparation failure";
+                        Result.FinishedAt=std::chrono::steady_clock::now();
+                        Result.FinishedFrame=ColdTimelineFrame();
+                        GpuFarPreparedResults.push_back(std::move(Result));
+                    }
                    std::fprintf(stderr,"[GpuFarInputPrepareError] unknown failure\n");
                }
                bGpuFarPreparationFinished.store(true,std::memory_order_release);
            });
        }

-        void SortGpuFarQueue()
-        {
-            std::sort(GpuFarQueue.begin(),GpuFarQueue.end(),[&](const FVoxelTileKey& A,const FVoxelTileKey& B)
+        bool GpuFarKeyLess(const FVoxelTileKey& A,const FVoxelTileKey& B)const
        {
            if(A.Level!=B.Level)return A.Level<B.Level;
            const FIntVector AO=A.OriginVoxels(),BO=B.OriginVoxels();
@@ -3059,7 +3070,12 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75;
                +(BO.Y+0.5*B.ExtentVoxels()-Position.Y*4.0)*(BO.Y+0.5*B.ExtentVoxels()-Position.Y*4.0)
                +(BO.Z+0.5*B.ExtentVoxels()-Position.Z*4.0)*(BO.Z+0.5*B.ExtentVoxels()-Position.Z*4.0);
            return AC<BC;
-            });
+        }
+
+        void SortGpuFarQueue()
+        {
+            std::sort(GpuFarQueue.begin(),GpuFarQueue.end(),
+                [&](const FVoxelTileKey& A,const FVoxelTileKey& B){return GpuFarKeyLess(A,B);});
            GpuFarQueued.clear();for(const FVoxelTileKey& Key:GpuFarQueue)GpuFarQueued.insert(Key);
        }

@@ -3132,8 +3148,61 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75;
        bool FinalizeGpuFarPreparation()
        {
            if(bGpuFarInputsFinalized)return true;
-            if(!bGpuFarPreparationStarted
-                ||!bGpuFarPreparationFinished.load(std::memory_order_acquire))return false;
+            if(!bGpuFarPreparationStarted)return false;
+            std::deque<FPreparedGpuFarInputResult> ReadyResults;
+            {
+                std::lock_guard Lock(GpuFarPreparedResultsMutex);
+                ReadyResults.swap(GpuFarPreparedResults);
+            }
+            std::vector<FVoxelTileKey> Failed;
+            bool bQueueChanged=false;
+            for(FPreparedGpuFarInputResult& Result:ReadyResults)
+            {
+                const FVoxelTileKey& Key=Result.Key;
+                if(Result.Input)
+                {
+                    GpuFarPreparedInputs.emplace(Key,std::move(*Result.Input));
+                    GpuFarPreparedAt.emplace(Key,Result.FinishedAt);
+                    GpuFarEligibleTiles.insert(Key);
+                    if(!GpuFarTiles.contains(Key)&&GpuFarQueued.insert(Key).second)
+                    {
+                        GpuFarQueue.push_back(Key);
+                        bQueueChanged=true;
+                    }
+                    if(bGpuFarTimelineEnabled)
+                        std::printf("[GpuTileTimeline] event=prepared key=(%d,%d,%d,lod%d) at_ms=%.3f frame=%llu\n",
+                            Key.Coord.X,Key.Coord.Y,Key.Coord.Z,Key.Level,
+                            ColdTimelineMilliseconds(Result.FinishedAt),
+                            static_cast<unsigned long long>(Result.FinishedFrame));
+                    if(!bGpuFarFirstPreparedInputReported)
+                    {
+                        bGpuFarFirstPreparedInputReported=true;
+                        std::printf("[ColdTimeline] event=gpu_first_input_ready key=(%d,%d,%d,lod%d) at_ms=%.3f\n",
+                            Key.Coord.X,Key.Coord.Y,Key.Coord.Z,Key.Level,
+                            ColdTimelineMilliseconds(Result.FinishedAt));
+                    }
+                }
+                else
+                {
+                    GpuFarPreparationErrors.emplace(Key,Result.Error.empty()
+                        ?"far-field input export failed":std::move(Result.Error));
+                    Failed.push_back(Key);
+                }
+            }
+            if(bQueueChanged)SortGpuFarQueue();
+            if(!Failed.empty())
+            {
+                for(const FVoxelTileKey& Key:Failed)
+                {
+                    const auto Error=GpuFarPreparationErrors.find(Key);
+                    std::fprintf(stderr,"[GpuFieldInputFallback] key=(%d,%d,%d,lod%d) reason=%s\n",
+                        Key.Coord.X,Key.Coord.Y,Key.Coord.Z,Key.Level,
+                        Error==GpuFarPreparationErrors.end()?"far-field input export failed":Error->second.c_str());
+                }
+                AddGpuFarCpuFallbackTiles(Failed);
+            }
+            if(!bGpuFarPreparationFinished.load(std::memory_order_acquire))
+                return !GpuFarQueue.empty();
            if(GpuFarPreparationThread.joinable())GpuFarPreparationThread.join();
            if(!bGpuFarEnabled)
            {
@@ -3145,11 +3214,11 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75;
                bGpuFarInputsFinalized=true;
                return false;
            }
-            std::vector<FVoxelTileKey> Failed;
            for(const FVoxelTileKey& Key:GpuFarCandidateTiles)
            {
                if(GpuFarTiles.contains(Key))
                {GpuFarEligibleTiles.insert(Key);continue;}
+                if(GpuFarEligibleTiles.contains(Key)||GpuFarPreparationErrors.contains(Key))continue;
                if(!GpuFarPreparedInputs.contains(Key))
                {
                    Failed.push_back(Key);
@@ -3160,9 +3229,9 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75;
                    continue;
                }
                GpuFarEligibleTiles.insert(Key);
-                GpuFarQueue.push_back(Key);
+                if(GpuFarQueued.insert(Key).second)GpuFarQueue.push_back(Key);
            }
-            SortGpuFarQueue();
+            if(!GpuFarQueue.empty())SortGpuFarQueue();
            AddGpuFarCpuFallbackTiles(Failed);
            bGpuFarInputsFinalized=true;
            const std::size_t NearGpuTiles=static_cast<std::size_t>(std::count_if(
@@ -3201,7 +3270,23 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75;
            StartGpuFarPreparation();
            if(!bGpuFarEnabled||!FinalizeGpuFarPreparation()||GpuFarQueue.empty())return;
            const bool bColdDrainBeforeWholeView=!bInitialClipmapReported&&!bGpuFarAuditEnabled
-                &&!bDisableGpuColdStartDrain;
+                &&!bDisableGpuColdStartDrain&&bGpuFarWarmupBeforeWorkers;
+            constexpr std::size_t StartupFirstGpuBatchTarget=32;
+            constexpr std::size_t StartupGpuBatchTarget=256;
+            constexpr double StartupGpuPumpBudgetMilliseconds=8.0;
+            const bool bWaitForStreamingGpuBatch=!bColdDrainBeforeWholeView
+                &&!bGpuFarAuditEnabled&&!bInitialClipmapReported;
+            const std::size_t GpuPumpTileLimit=bWaitForStreamingGpuBatch
+                ?(GpuFarSubmissionBatches==0?StartupFirstGpuBatchTarget:StartupGpuBatchTarget):32;
+            const double GpuPumpBudgetMilliseconds=bWaitForStreamingGpuBatch
+                ?StartupGpuPumpBudgetMilliseconds:4.0;
+            if(bWaitForStreamingGpuBatch
+                &&!bGpuFarPreparationFinished.load(std::memory_order_acquire)
+                &&GpuFarQueue.size()<StartupFirstGpuBatchTarget)return;
+            if(bWaitForStreamingGpuBatch&&GpuFarSubmissionBatches==0)
+                std::printf("[GpuFarStreamingPolicy] first_batch_tiles=%zu start_threshold=%zu later_batch_limit=%zu pump_budget_ms=%.1f order=lod_then_camera_distance\n",
+                    GpuFarQueue.size(),StartupFirstGpuBatchTarget,StartupGpuBatchTarget,
+                    StartupGpuPumpBudgetMilliseconds);
            const auto PumpStart=std::chrono::steady_clock::now();
            const double PumpMs=std::chrono::duration<double,std::milli>(PumpStart-ColdProcessStart).count();
            if(GpuFarFirstPumpMs<0.0)GpuFarFirstPumpMs=PumpMs;
@@ -3225,7 +3310,7 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75;
            const auto Start=std::chrono::steady_clock::now();
            int32 Built=0;
            std::vector<FVoxelTileKey> BuiltKeys;
-            BuiltKeys.reserve(bColdDrainBeforeWholeView?GpuFarQueue.size():32);
+            BuiltKeys.reserve(bColdDrainBeforeWholeView?GpuFarQueue.size():GpuPumpTileLimit);
            do
            {
                const FVoxelTileKey Key=GpuFarQueue.front();GpuFarQueue.pop_front();GpuFarQueued.erase(Key);
@@ -3721,8 +3806,9 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75;
                }
            }
            while(!GpuFarQueue.empty()&&(bColdDrainBeforeWholeView
-                ||(Built<32&&std::chrono::duration<double,std::milli>(
-                    std::chrono::steady_clock::now()-Start).count()<4.0)));
+                ||(Built<GpuPumpTileLimit&&std::chrono::duration<double,std::milli>(
+                    std::chrono::steady_clock::now()-Start).count()
+                    <GpuPumpBudgetMilliseconds)));
            if(Built>0)
            {
                if(bDeferredBatch)