wip(standalone): GPU step 4 -- batched far generation, connector panels on GPU; far survival passes

Up to 32 tiles per deferred command list, one polled event per batch, no per-tile waits; CPU
upload/enqueue 723 -> 46 ms. GPU far meshes pass far survival (7,200 strata boundaries, 10,976
connectors, 0 missing/holes). But hybrid cold 4.36 s vs CPU 2.29 s: whole-tile GPU timestamps sum to
646 ms (0.58 ms/tile; step 3's 0.03 ms covered part of the work only), divergent per-thread shape
loops; CPU far-input export 1.6 worker-s. CPU 44/44 hashes unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MmSoF64BbRFmk9C3p1VsBJ

Standalone/Research/OwnerShapeWorld/gpu_field_port/BuildGpuFieldViewer.ps1

@@ -9,8 +9,11 @@ $BuildHostProcess = Get-Process -Id $PID
 $BuildHostProcess.PriorityClass = 'BelowNormal'
 $BuildHostProcess.ProcessorAffinity = [IntPtr]0xFF00
 $Root = [IO.Path]::GetFullPath((Join-Path $PSScriptRoot '..\..\..'))
-$BuildDir = Join-Path $Root 'Build\VoxelForgeGpuFarHybridViewer'
+$BuildStem = [IO.Path]::GetFileNameWithoutExtension($OutputName)
+$BuildDir = Join-Path $Root ('Build\'+$BuildStem)
 $BinDir = Join-Path $Root 'bin'
+$ShaderSource = Join-Path $Root 'Research\OwnerShapeWorld\gpu_first_port\VoxelForgeGpuMesher.hlsl'
+$PrecompiledShaderDir = Join-Path $Root 'Research\OwnerShapeWorld\gpu_field_port\precompiled'
 [IO.Directory]::CreateDirectory($BuildDir) | Out-Null
 [IO.Directory]::CreateDirectory($BinDir) | Out-Null
 
@@ -110,6 +113,22 @@ if ($Target -eq 'All') {
     $LegacyObjects = @($LegacySources | ForEach-Object { '"{0}.obj"' -f [IO.Path]::GetFileNameWithoutExtension($_) })
     $LegacyRspLines = $CommonFlags + @('/c') + @($LegacySources | ForEach-Object { '"{0}"' -f $_ })
     [IO.File]::WriteAllLines($LegacyRsp,$LegacyRspLines,[Text.UTF8Encoding]::new($false))
+    if (-not (Test-Path -LiteralPath $ShaderSource -PathType Leaf)) { throw "GPU shader source is missing: $ShaderSource" }
+    $FxCRoot = 'C:\Program Files (x86)\Windows Kits\10\bin'
+    $FxC = Get-ChildItem -LiteralPath $FxCRoot -Filter fxc.exe -Recurse -ErrorAction SilentlyContinue |
+        Where-Object { $_.FullName -match '\\x64\\fxc\.exe$' } |
+        Sort-Object -Property FullName -Descending | Select-Object -First 1
+    if ($FxC) {
+        [IO.Directory]::CreateDirectory($PrecompiledShaderDir) | Out-Null
+        foreach ($Entry in @('EvaluateFarFieldCS','ClassifyCS','ClassifyFarCS','ScanGroupsCS','EmitCS','NoiseProbeCS','BuildFarConnectorRingsCS','EmitFarConnectorPanelsCS')) {
+            $ShaderBinary = Join-Path $PrecompiledShaderDir ($Entry+'.cso')
+            $CommandLine = '"{0}" /nologo /T cs_5_0 /E {1} /O3 /Gis /Ges /Fo "{2}" "{3}"' -f $FxC.FullName,$Entry,$ShaderBinary,$ShaderSource
+            Invoke-FieldViewerBuildCommand $CommandLine "precompile HLSL entry $Entry"
+        }
+        Write-Host "Precompiled viewer compute shaders into $PrecompiledShaderDir"
+    } else {
+        Write-Warning 'fxc.exe was not found; the viewer will fall back to one-time runtime shader compilation.'
+    }
 }
 
 $ViewerRsp = Join-Path $BuildDir 'viewer-link.rsp'

Standalone/Research/OwnerShapeWorld/gpu_field_port/GpuFarViewer.h

@@ -15,14 +15,39 @@ namespace VoxelForgeGpuFarViewer
 {
     struct CpuVertex { float position[3]{}, normal[3]{}; };
     struct CpuMesh { std::vector<CpuVertex> vertices; std::vector<std::uint32_t> indices; };
+    struct GpuTimingQueries
+    {
+        ID3D11Query* start=nullptr;
+        ID3D11Query* end=nullptr;
+        ID3D11Query* disjoint=nullptr;
+    };
+    bool ReserveFarOutputPool(ID3D11Device* device,ID3D11DeviceContext* context,
+        const std::filesystem::path& shaderPath,std::uint32_t tileSlots,std::uint32_t cells,
+        std::string& error);
+    bool QueryFarGpuMemory(ID3D11Device* device,ID3D11DeviceContext* context,
+        const std::filesystem::path& shaderPath,std::uint64_t& usageBytes,
+        std::uint64_t& budgetBytes,std::string& error);
+    bool BeginFarTileBatch(ID3D11Device* device,ID3D11DeviceContext* immediateContext,
+        const std::filesystem::path& shaderPath,std::string& error);
     // Returned D3D11 buffers carry one reference each; the caller releases them.
     // Vertex records keep the 32-byte compute layout (position/normal/uv). The
     // first 24 bytes match the standalone viewer's input layout.
     bool BuildFarTile(ID3D11Device* device, ID3D11DeviceContext* context,
         VoxelForgeCore::World& world, const VoxelForgeCore::TileKey& key,
+        const VoxelForgeCore::FarFieldTileInput* preparedInput,
         const std::filesystem::path& shaderPath,
         ID3D11Buffer** vertices, ID3D11Buffer** indices, ID3D11Buffer** indirectArgs,
-        ID3D11Query** completionEvent,
+        ID3D11Buffer** connectorVertices,ID3D11Buffer** connectorIndices,
+        std::uint32_t outputSlot, std::uint32_t outputCapacitySlots,
+        std::uint32_t* indirectArgsByteOffset,
+        ID3D11Query** completionEvent,GpuTimingQueries* timingQueries,
         CpuMesh* auditReadback,
         double* enqueueWallMs, double* gpuDispatchMs, std::string& error);
+    bool SubmitFarTileBatch(ID3D11Device* device,ID3D11DeviceContext* immediateContext,
+        const std::filesystem::path& shaderPath,ID3D11Query** completionEvent,
+        GpuTimingQueries* timingQueries,std::string& error);
+    bool ResolveGpuTiming(ID3D11DeviceContext* context,GpuTimingQueries& queries,
+        double& dispatchMs,std::uint64_t* timestampFrequency=nullptr);
+    bool ResolveGpuTimestampPair(ID3D11DeviceContext* context,GpuTimingQueries& queries,
+        std::uint64_t timestampFrequency,double& dispatchMs);
 }

Standalone/Research/OwnerShapeWorld/gpu_field_port/RunGpuFarViewer.ps1

@@ -3,7 +3,8 @@ param(
     [ValidateSet('gpu','cpu','audit')][string]$Mode='gpu',
     [string]$LogStem='gpu_far_viewer',
     [int]$ClipRadius=3,
-    [string]$CaptureName=''
+    [string]$CaptureName='',
+    [string]$ExecutableName='VoxelForgeGpuFarHybridViewer.exe'
 )
 
 $ErrorActionPreference='Stop'
@@ -12,7 +13,9 @@ $HostProcess=Get-Process -Id $PID
 $HostProcess.PriorityClass='BelowNormal'
 $HostProcess.ProcessorAffinity=[IntPtr]0xFF00
 $StandaloneRoot=[IO.Path]::GetFullPath((Join-Path $PSScriptRoot '..\..\..'))
-$ExecutablePath=Join-Path $StandaloneRoot 'bin\VoxelForgeGpuFarHybridViewer.exe'
+if([IO.Path]::GetFileName($ExecutableName)-ne$ExecutableName-or[IO.Path]::GetExtension($ExecutableName)-ne'.exe')
+{throw 'ExecutableName must be an executable filename, not a path.'}
+$ExecutablePath=Join-Path $StandaloneRoot ('bin\'+$ExecutableName)
 $StdoutPath=Join-Path $PSScriptRoot ($LogStem+'.stdout.log')
 $StderrPath=Join-Path $PSScriptRoot ($LogStem+'.stderr.log')
 if(-not(Test-Path -LiteralPath $ExecutablePath -PathType Leaf)){throw "GPU far viewer not found: $ExecutablePath"}

Standalone/Research/OwnerShapeWorld/gpu_field_port/STEP4_REPORT.md

@@ -0,0 +1,76 @@
+# GPU far streaming, step 4
+
+## Result
+
+The GPU far viewer now records up to 32 tiles into a deferred D3D11 command list, executes one list per update batch, and polls one completion event per batch. The normal path has no per-tile GPU wait or readback. Persistent pooled mesh buffers remain in use, including a separate connector-panel pool drawn with the base far mesh. **This removes most of the CPU submission overhead, but the measured whole-view cold time is still slower than all-CPU.**
+
+The viewer keeps CPU generation for LOD0/1 and 152 transition tiles. GPU field evaluation and marching cubes cover the other 1,114 far tiles. Press **H** to switch between CPU-far and GPU-far.
+
+## Connector and topology evidence
+
+The final viewer binary passed `RunGpuFarViewer.ps1 -Mode audit -ClipRadius 3`. The GPU mesh survival audit reported:
+
+- 7,200/7,200 strata-boundary surfaces, with zero missing, extra, or hole counts.
+- 10,976/10,976 far connectors audited; 178,351 connector samples and 710,047 rays, with zero missing samples and zero connector holes.
+- `status=PASS` and `mesh_source=GPU field_readback=required`.
+
+The audit reports 49,585 connector-ray `extra` hits; those are tolerated by the existing repair-panel survival criteria. The missing and hole counts, which gate survival, are zero.
+
+The batch refactor only changes viewer submission and timing. It does not change field evaluation, sign overrides, or marching-cubes cases. The prior step-3 whole-far agreement covered all 1,114 tiles: zero ordinary sign differences, zero cell-configuration differences against the RX 7900 XTX and WARP, zero protected/thin topology differences, and repeat RX output was bit-identical. Those agreement results are recorded in [the step-3 report](REPORT.md); they were not rerun for this submission-only change.
+
+`CompareFarCpuHashes.ps1` still reports 44/44 hash fields equal with zero differences against the retained W8 reference logs. That script compares the saved `566d1f3` reference log to the saved current-path log; it is not a fresh CPU-path hash run.
+
+## Cold timings
+
+Commands, all with `--workers=8`, BelowNormal priority, affinity `0xFF00`, and clip radius 3:
+
+```powershell
+RunGpuFarViewer.ps1 -Mode cpu -LogStem step4_memory_cpu_A -ClipRadius 3 -ExecutableName VoxelForgeGpuFarStep4Viewer.exe
+RunGpuFarViewer.ps1 -Mode gpu -LogStem step4_memory_gpu_A -ClipRadius 3 -ExecutableName VoxelForgeGpuFarStep4Viewer.exe
+RunGpuFarViewer.ps1 -Mode gpu -LogStem step4_memory_gpu_B -ClipRadius 3 -ExecutableName VoxelForgeGpuFarStep4Viewer.exe
+RunGpuFarViewer.ps1 -Mode cpu -LogStem step4_memory_cpu_B -ClipRadius 3 -ExecutableName VoxelForgeGpuFarStep4Viewer.exe
+```
+
+| Cold clipmap | A | B | Mean |
+| --- | ---: | ---: | ---: |
+| All CPU, 2,226 tiles | 2.189 s | 2.387 s | 2.288 s |
+| CPU near + GPU far, 1,114 GPU tiles | 4.359 s | 4.361 s | 4.360 s |
+
+The hybrid is 2.072 s slower on average (1.91x). CPU near/transition work took 3.214 s and 3.114 s wall, despite 1.566 s and 1.698 s of worker time summed across workers. The all-CPU runs took 2.093 s and 2.154 s wall for the full 2,226 tiles. This near-stream timing remains a major unexplained contributor to the hybrid result.
+
+CPU tile-worker time fell from a 4.745 s mean to 1.632 s, saving 3.113 worker-seconds. Parallel far-input export used another 1.598 worker-seconds on average, leaving about 1.515 worker-seconds net before main-thread upload/enqueue and pool work. This worker-time reduction did not reduce whole-view wall time.
+
+The batch design reduced summed CPU upload/enqueue time from about 723 ms before batching to 46.5 ms and 45.0 ms in the final runs. Each normal run used 37 submissions for 1,114 tiles, at most 32 tiles per submission. The command list still contains per-tile compute dispatch sequences; this is command-list batching rather than a single multi-tile shader dispatch.
+
+## GPU timing and 90 Hz slicing
+
+Two query levels are reported separately. Per-tile timestamp pairs bracket each tile's field, classify, emit, and connector compute sequence. A separate timestamp set brackets command-list execution, including its deferred resource updates.
+
+| Measurement across 1,114 GPU tiles | Run A | Run B | Mean |
+| --- | ---: | ---: | ---: |
+| Tile dispatch timestamp sum | 640.7 ms | 650.5 ms | 645.6 ms |
+| Mean dispatch time per tile | 0.575 ms | 0.584 ms | 0.580 ms |
+| Slowest tile dispatch | 3.924 ms | 3.990 ms | 3.990 ms |
+| Command-list execution timestamp sum | 762.4 ms | 817.3 ms | 789.8 ms |
+| Slowest command-list span (up to 32 tiles) | 57.6 ms | 62.5 ms | 60.1 ms |
+
+For 0.5 ms of compute per 90 Hz frame, the measured dispatch sum needs about 1,292 frame slices (14.4 s). For 1 ms per frame it needs about 646 slices (7.2 s). At the mean tile cost, that is 0.86 tiles per 0.5 ms slice or 1.72 tiles per 1 ms slice. These are budget estimates from summed dispatch timestamps, not a VR frame-pacing run; a tile cannot be divided by the current scheduler, and the slowest tile alone took about 4 ms. The current 32-tile batches are not frame-budgeted: their largest measured execution span was 62.5 ms.
+
+## Memory and limits
+
+The final timed runs peaked at 0.94 GiB and 0.85 GiB working set, and 4.66 GiB and 4.60 GiB private process memory. DXGI sampled local GPU-memory peaks of 3,689.8 MiB and 3,690.9 MiB (3.60 GiB) against a reported 23,749 MiB (23.2 GiB) budget. The higher run's working set plus local GPU residency was about 4.54 GiB. Private-byte totals and DXGI residency can overlap because the driver accounts for resource backing separately; they are reported separately rather than added as if disjoint.
+
+No `0xC0000142` occurred. The final viewer and audit runners exited with code 0. The audit mode uses immediate-context readback and is intentionally synchronous, so its 41.6 s cold time and 1.50 s query sum are not comparable to the normal no-readback viewer path.
+
+## GTX 1060-class estimate
+
+Using the user's 4.4 TFLOPS figure and AMD's published 61.4 TFLOPS FP32 peak for the RX 7900 XTX, simple throughput scaling gives roughly a 14x ratio ([AMD RX 7900 XTX specifications](https://www.amd.com/en/products/graphics/desktops/radeon/7000-series/amd-radeon-rx-7900xtx.html)). Applied to these measurements, that estimates about 8.1 ms average GPU dispatch per far tile and about 9.0 s summed dispatch time for 1,114 tiles. Pascal's slower 32-bit integer multiply and this shader's divergent shape loops make **roughly 9–15 s for the far dispatch work** a more practical, still very coarse estimate. This is not a GTX 1060 measurement. It implies no whole-view speedup over the measured 2.29 s CPU path.
+
+## Logs and implementation
+
+- Final interleaved timings: `step4_memory_cpu_A/B.stdout.log`, `step4_memory_gpu_A/B.stdout.log`.
+- Final GPU connector audit: `step4_final_gpu_survival_audit.stdout.log`.
+- Batching and timestamp code: [GpuFarViewer.h](GpuFarViewer.h), [GpuFieldPort.cpp](GpuFieldPort.cpp), and [StandaloneViewer.cpp](../../../Source/StandaloneViewer.cpp).
+- Connector repair compute stages: [VoxelForgeGpuMesher.hlsl](../gpu_first_port/VoxelForgeGpuMesher.hlsl).
+
+The CPU reference path was left unchanged. All work and generated output remain under `Standalone/`; nothing was committed, pushed, or stashed.

Standalone/Research/OwnerShapeWorld/gpu_field_port/StartGpuFarViewer.ps1

@@ -1,9 +1,11 @@
 [CmdletBinding()]
-param([int]$ClipRadius=3)
+param([int]$ClipRadius=3,[string]$ExecutableName='VoxelForgeGpuFarHybridViewer.exe')
 
 $ErrorActionPreference='Stop'
 $StandaloneRoot=[IO.Path]::GetFullPath((Join-Path $PSScriptRoot '..\..\..'))
-$ExecutablePath=Join-Path $StandaloneRoot 'bin\VoxelForgeGpuFarHybridViewer.exe'
+if([IO.Path]::GetFileName($ExecutableName)-ne$ExecutableName-or[IO.Path]::GetExtension($ExecutableName)-ne'.exe')
+{throw 'ExecutableName must be an executable filename, not a path.'}
+$ExecutablePath=Join-Path $StandaloneRoot ('bin\'+$ExecutableName)
 $StdoutPath=Join-Path $PSScriptRoot 'owner_hybrid_viewer.stdout.log'
 $StderrPath=Join-Path $PSScriptRoot 'owner_hybrid_viewer.stderr.log'
 if(-not(Test-Path -LiteralPath $ExecutablePath -PathType Leaf)){throw "GPU far viewer not found: $ExecutablePath"}