VoxelForge/Standalone/README.md

# VoxelForge standalone world

This C++20 program loads the owner snapshot from `data/owner_world.json`, runs the VoxelForge CPU
pipeline, streams the clipmap with up to sixteen host worker threads, and displays it through Direct3D
11. `Core/` contains the Unreal-free `VoxelForgeCore` API and static-library target used by the
standalone shape-world path. Unreal is not linked or needed at run time.

## Build and launch

From the plugin root, run:

```powershell
.\Standalone\Build.ps1 -Target All
.\Standalone\bin\VoxelForgeCoreStandalone.exe --shape-world
```

`Build.ps1 -Target Core` builds only `Build\VoxelForgeCore\VoxelForgeCore.lib`; `-Target All`
builds that library and the viewer. `Core\core_sources.txt` is the shared translation-unit
manifest. The defaults use MSVC `/std:c++20`, `/O2`, `/arch:AVX2`, `/fp:precise`, `/MP16`, Normal
priority and affinity `0xFFFF`. Use `-CompileWorkers 8 -BuildPriority BelowNormal` for a lower-impact
build. Pass another executable name to build beside any viewer that may be open by setting `-OutputName`, for example
`VoxelForgeCoreStandalone_test.exe`. The executable finds the JSON under `Standalone\data\`.
The isolated viewer links `Source\GpuFarViewerLinkStubs.cpp`; GPU Far rendering is not part of this
CPU viewer target.

### BuildHost resource controls

The standalone viewer accepts `--workers=1..16`, `--process-priority=Normal` and
`--affinity-all-cpu`. Its draw loop also has a 60 FPS ceiling by default, including when DXGI reports
an occluded window; pass `--frame-limit=1..60` to choose a lower ceiling for captures and benchmarks.
VSync remains enabled unless `--no-vsync` is passed. The verification runner in
`Core\verification\RunMachine.ps1` retains its
separate BelowNormal/8-worker limits and refuses to overwrite an existing log stem.

## Controls

- Hold right mouse button and move to look.
- WASD moves; Space/Ctrl or E/Q moves vertically; Shift moves faster; the wheel changes speed.
- F toggles wireframe.
- 1 through 7 toggle clipmap levels 0 through 6 individually; 0 toggles every level.
- N/P changes the seed. T switches between the owner's base world and the added research-shaped
  round, winding tunnels; `--round-tunnels` starts in that variant. G switches between legacy smooth
  and detailed/smoothed tunnel walls. Either action rebuilds the clipmap.
  Crossing a chunk recenters streaming:
  tiles still desired stay resident, only new tiles are generated, and leaving tiles are dropped.
- Esc exits.

The console reports density parity at startup, each level's generation time, and the total time.

## Copied code and compatibility layer

`Source\VoxelForge\` contains the plugin's density evaluator and operator stack, cave morphology,
passage geometry, strate definitions/layout/manager/composer, noise, voxel math and types,
marching-cubes tables/mesher, and clipmap tile selection. These are copies. Plugin runtime source
is untouched apart from the allowed editor commandlet export addition.

`Compat\CoreMinimal.h` supplies the UE-style arrays/maps/sets, vectors and boxes, math helpers,
strings, smart pointers, locks, logging/stat macros, UObject-like data holders, and platform helpers
needed by the copied CPU code. The `Compat` subfolders cover the UE include paths used by those
copies. Perlin 2D/3D uses the copied UE 5.7 implementation in `UnrealPerlin.cpp` and
`UnrealPerlinHelpers.inl`. `OwnerWorldLoader.h` maps the exported settings, resolved strate
definitions, biomes, and terrain operations into copied holders; soft references resolve through a
small local asset registry. The copied strate manager reconstructs runtime passages from that data
and seed. The export also records their geometry in JSON for reference.

## Behavior differences

- Standalone `VOXEL_SIZE` is 25 cm; the plugin's checked-out value remains 12.5 cm. No other voxel
  conversion constant needed changing. Mesh vertices stay in centimetres in the copied mesher and
  the viewer converts them to metres at upload.
- The compatibility `TMap` uses an open-addressed hash index with stable pair storage;
  `TSet` uses that index for lookup. Iteration follows insertion order rather than Unreal hash-table
  order. Owner strate slots and passages are read from ordered JSON arrays. Perlin noise and
  deterministic `DetSinCos` are copied unchanged; MSVC uses `/fp:precise` without fast-math/FMA
  contraction.
- Unreal reflection and UObject construction are replaced by explicit JSON-to-holder mapping.
  Density-relevant settings, resolved strate layouts, passages, biome parameters, and terrain-op
  assets are loaded. Runtime actor/material assets are not instantiated; the viewer uses one lit
  rock material for all generated geometry.
- The owner export has no active season. Standalone season loading is not implemented; an active
  season would need to be exported and mapped before it could affect this viewer.
- `WITH_EDITOR` research/debug branches, Unreal logging, startup trace, profiling counters, and
  reflection metadata do not run in the standalone. The base generator path is the default.
- T selects a standalone-only tunnel-shape port. It preserves the owner's rooms, tunnels, and
  passages, then adds the research field's cubic winding centerline and round elliptical tunnel
  profile (144-voxel pitch, 28-voxel radius, 26-voxel half-height, deterministic variable radius
  and vertical wave) at the fixed world height -80 voxels. Detailed walls use deterministic relief
  at 8-, 16-, and 32-voxel wavelengths; the fixed-height field and copied `DetSin` keep the world
  independent of camera placement and CRT trigonometry. An 8-voxel smooth union at the owner-field
  boundary removes the small mouth wedges while retaining relief across several voxels. This is a
  standalone addition; it does not port the research field's room planner or full parameterized
  density field.

## Owner data and parity

`data/owner_world.json` came from one default-off `VoxelForgeExploreCommandlet` export using the
guarded `Tools\VoxelForgeTest.ps1 -Scenario standalone-export -Build -Assets owner` path. The
snapshot contains the resolved settings/strates/passages and 3,436 plugin density samples. For that
commandlet build, `VOXEL_SIZE` was temporarily set to 25 cm in the worktree and restored afterward;
the exported JSON records 25 cm and the checked-out plugin source still records 12.5 cm.

Standalone comparison: **3,436/3,436 bit-identical, zero mismatches**. Samples covered strate
boundaries (2,420), varied world coordinates including negative and far-from-origin points (768),
tunnel interiors (222), room interiors (12), inter-strate passages (10), and passage landings (4).

## Baseline clipmap timing

Owner seed 0, base world, 25 cm voxels, six workers; the final viewer run generated the full
seven-level clipmap (12,054 tiles, 6,605,189 vertices, 36,307,137 indices) in 515.117 seconds.
These level timers cover CPU meshing and queue handoff; D3D11 buffer upload is separate. An earlier
headless pass over the same tiles took 408.719 seconds, so use one run mode consistently when
comparing later speed changes.

| Level | Seconds | Tiles | Vertices | Density samples |
|---:|---:|---:|---:|---:|
| 0 | 70.201 | 2,744 | 957,655 | 117.65 M |
| 1 | 65.376 | 2,009 | 657,423 | 86.14 M |
| 2 | 9.042 | 1,274 | 292,267 | 8.74 M |
| 3 | 25.689 | 588 | 235,755 | 4.03 M |
| 4 | 24.248 | 441 | 229,720 | 3.02 M |
| 5 | 23.809 | 294 | 199,503 | 2.02 M |
| 6 | 296.304 | 4,704 | 4,032,866 | 32.26 M |
| **Total** | **515.117** | **12,054** | **6,605,189** | **253.86 M** |

## Profiling result and speed change

The compatibility layer's old `TMap`/`TSet` linear lookups and heap-only vector growth were
replaced with an open-addressed hash index, stable pair storage, small inline storage, and a
thread-local pooled allocator. That removes known container and allocation mismatches with the
Unreal path, but did not improve the earlier LOD0 timing (72.1 s before, 74.1 s after); profiling
showed those were not the dominant cost. The same audit also found two recursive room-graph walks
wrapped in `std::function`, which could allocate and dispatch indirectly on every graph build. They
now use self-recursive generic lambdas with the same depth-first visit order, removing that wrapper
overhead from the generation hot path. In matched profile runs, RoomGraphBuild fell from 4.531 to
4.417 s for 64 LOD5 tiles and from 5.210 to 5.038 s for 64 LOD6 tiles; graph build counts and
density hashes were unchanged.

Per-stage timers and cache counters found the real hotspot in `GetDensityAt`: the room/SDF graph
cache is keyed by exact XY bounds, and the original X-fast lattice walk kept rebuilding it. On one
LOD3 tile, cache-region grouping cut graph builds from 1,084 to 229 while writing samples to their
original lattice indices. A 64-tile LOD6 profile attributed 13.189 s of 19.874 s to 36,480 room
graph builds. For tiles wider than 16 chunks, keeping each XY column's Z run contiguous reduced
that to 12,160 builds and 5.210 s of 10.972 s, with the same lattice hash. The final recursive-
lambda profile measured 5.038 s in room-graph builds and 10.600 s total for the same 64 LOD6 tiles.
Finer tiles retain the plane-first traversal, which is faster for their shorter cache runs.

Cache-slot experiments found a second scale-specific case: 32-chunk LOD5 tiles improve with two
room graph cache slots (26,926 to 9,700 builds; profiled RoomGraphBuild 10.151 to 4.475 s across
64 tiles). Two slots regress 64-chunk LOD6 tiles (12,160 to 23,104 builds), so the generator uses
two only for 32-chunk tile contexts and one otherwise. The selected-tile density hash is unchanged.

The controlled before/after run below meshes the 64 nearest tiles for each LOD serially, without
profiling or hash collection. It uses the same owner seed and camera-centered desired set. The
selected-tile lattice hashes from a separate final run are identical at every LOD.

| LOD | 64-tile density hash |
|---:|---:|
| 0 | `0x3d31a090650399d3` |
| 1 | `0x6150c607c7a5c76f` |
| 2 | `0x9f1d560bc14e67f1` |
| 3 | `0xdd3e76020ce69cd7` |
| 4 | `0x59a209a5db034b46` |
| 5 | `0x1fdeb60160e4216f` |
| 6 | `0xbac69461f812493d` |

| LOD | Before ms/tile | After ms/tile | After p50 | After p95 |
|---:|---:|---:|---:|---:|
| 0 | 50.4 | 50.551 | 50.521 | 63.148 |
| 1 | 112.3 | 110.175 | 33.030 | 407.767 |
| 2 | 23.9 | 22.430 | 12.015 | 51.078 |
| 3 | 223.0 | 73.482 | 69.614 | 94.331 |
| 4 | 269.0 | 133.512 | 98.003 | 214.854 |
| 5 | 393.1 | 127.325 | 116.386 | 173.401 |
| 6 | 282.8 | 145.457 | 140.952 | 183.234 |

Run mode: BelowNormal priority, serial headless benchmark, 64 nearest tiles per LOD, profiling and
hash collection off; final executable `VoxelForgeStandalone_night20.exe`. The before timings are
from the earlier 64-tile baseline. The user's prior plugin reference is about 95 ms worker time for
a 25 cm LOD0 tile. The final nearest-tile LOD0 average is 50.6 ms. In the complete streamed run,
LOD0 median was 82.9 ms, but its mean was 197.6 ms and p95 was 668.8 ms. The plugin was not timed
on the same tile coordinates, so the rough reference comparison is not apples-to-apples and the
streamed tail does not meet that target yet.

With 16 BelowNormal workers, the final full run completed all 12,054 tiles in 175.056 s, down from
the historical 515.117 s six-worker run. It reached full detail after 66.565 s. Worker time per
level in the final run was:

| LOD | Worker sum | Tiles | Avg/tile | p50 | p95 |
|---:|---:|---:|---:|---:|---:|
| 0 | 542.284 s | 2,744 | 197.625 ms | 82.946 ms | 668.796 ms |
| 1 | 501.518 s | 2,009 | 249.635 ms | 89.990 ms | 742.258 ms |
| 2 | 68.712 s | 1,274 | 53.934 ms | 55.633 ms | 120.102 ms |
| 3 | 77.055 s | 588 | 131.047 ms | 122.697 ms | 212.157 ms |
| 4 | 103.552 s | 441 | 234.812 ms | 192.137 ms | 418.249 ms |
| 5 | 66.689 s | 294 | 226.834 ms | 223.139 ms | 328.030 ms |
| 6 | 1,439.125 s | 4,704 | 305.936 ms | 303.062 ms | 391.440 ms |

The worker sums are concurrent wall-time measurements and therefore add to more than the 175.056 s
clipmap wall time. The historical and final full runs used different worker counts; the serial
64-tile table is the controlled per-tile comparison. The full run also passed density parity at
3,436/3,436 samples with zero mismatches.

## Streaming and uploads

The viewer rebuilds the desired tile set at each chunk crossing, keeps the intersection resident,
cancels queued jobs that left the set, and enqueues only missing tiles. A heap orders by camera
distance, then prefers finer LOD at equal distance. Workers generate CPU meshes concurrently. D3D11
uploads are limited to four batches or 2 ms per frame. A leaving or changed batch keeps its prior GPU
buffer until its replacement is uploaded, avoiding a hole while the batch is rebuilt.

The final 16-worker run reached full detail 66.565 s after spawn (4,753 LOD0–1 desired tiles; 5,173
resident through LOD1) and the full clipmap at 175.056 s. In a controlled one-chunk crossing on the
same executable, 4,856 tiles stayed resident, 441 left, and only 441 new jobs were submitted. Those
jobs covered 18.91 M density samples, used 88.573 worker-seconds, and finished in 6.108 s wall time;
all 441 were generated with no stale results or queued cancellations. The upload budget remained
four batches / 2 ms per frame.

For a deterministic movement check, `--test-one-chunk-crossing` moves once after full-detail
readiness and closes the test viewer after that crossing's jobs and bounded uploads drain.

## Lighting comparison

The copied plugin mesher already computes central-difference density-gradient normals on the sampled
density lattice and interpolates them along marching-cubes edges. The standalone passes those
normals through to D3D11, so replacing face normals was unnecessary. Profiling measured this existing
gradient stage at 2.575 ms across 64 LOD0 benchmark tiles (about 72 ms over all 2,744 streamed LOD0
tiles); this round adds no density samples or meshing work for normals.

The visible striping was amplified by the viewer's squared, high-intensity camera-facing light. The
shader now uses a broad linear camera fill at similar average strength. Geometry, density, and LOD
are unchanged; the shading path uses fewer multiplications. The D3D11 pixel-shader adjustment has
no CPU mesh-generation cost.

Both captures use the default seed, default camera, 1280x800 view, and are saved from the D3D11 back
buffer after all full-detail tiles upload:

| Before | After |
|---|---|
| ![Before lighting](data/night_lighting_before.png) | ![After lighting](data/night_lighting_after.png) |

## Tunnel palette pass

As a separate, render-only visual pass, the neutral blue-gray rock constants were changed to warm
limestone and ochre, with a softer cyan camera fill. This follows the color balance of the approved
cave reference while leaving its mesh, 25 cm resolution, and detail untouched. The shader changes
only existing constant values and adds no texture reads or operations; CPU density and meshing cost is
zero. The matched 16-worker captures generated the same 7,521,175 vertices and 41,506,623 indices;
full-clipmap wall times were 172.772 s before and 171.676 s after, within run variance.

Both images show seed 0 with the round/winding tunnel field, at 1280x800 after the full clipmap
uploaded. Camera: `(-11.75, -123.25, -19.85)` m, yaw 90 degrees, pitch 0. Capture mode used 16
BelowNormal workers:

| Before palette pass | After palette pass |
|---|---|
| ![Before tunnel palette](data/night18_tunnel_view_before.png) | ![After tunnel palette](data/night19_tunnel_view_after.png) |

## 2026-10 speed round

The owner export still says `ClipRadius=6`, which selects a 112 m full-detail cube. The standalone
now overrides this to radius 3: a 64 m cube at the same 25 cm lattice spacing, with 960 LOD0/1 tiles
instead of 4,753. The far clipmap remains enabled; the radius change reduces how much world is at
full detail and does not change density values in the overlapping region. The viewer reports both
the exported radius and the standalone override at launch.

| Path | Before | After |
|---|---:|---:|
| Full detail, uncached | 112.243 s, radius 6, 4,753 tiles, 8 workers | 20.266 s, radius 3, 960 tiles, 6 workers |
| Full detail, seed-0 base cache hit | — | 0.270 s cache read + 0.028 s headless handoff; viewer full-detail ready at 0.059 s |
| Full clipmap, uncached | 175.056 s, radius 6, 12,054 tiles, 16-worker historical night run | 170.642 s, radius 3, 6,498 tiles, 8 workers |
| Full clipmap, seed-0 base cache hit | — | 0.266 s cache read + 0.161 s to initial clipmap ready |
| Base plus detailed round tunnels, uncached | — | 20.724 s full detail; 173.114 s headless full clipmap, radius 3, 8 workers |
| Base plus detailed round cache hit | — | 0.282 s cache read + 0.028 s headless detail; viewer ready at 0.058 s |
| Detailed round full clipmap, cache hit | — | 0.282 s cache read + 0.092 s headless; viewer ready at 0.165 s |

All new timings use BelowNormal priority and affinity `0xFF00`; worker counts are shown above. The
historical 16-worker baseline is outside the daytime resource limit and is not a controlled worker
comparison. Raw generation still misses both time targets. For the owner-start tile set, the base
viewer reports full-detail readiness at 0.059 s and full-clipmap readiness at 0.161 s after
generation starts. The detailed round viewer reports 0.058 s and 0.165 s. Including their 0.266 s /
0.284 s cache reads, both modes are below 0.45 s from cache-read start. The base cache is
`data/VoxelForge_seed0_r3_base.vfmesh` (215.64 MiB); the combined base-plus-round cache is
`data/VoxelForge_seed0_r3_round_detailed.vfmesh` (226.55 MiB). Their raw bakes took 170.642 s and
173.114 s. These timings start from new processes, but the cache files were in the OS file cache; a
cold-storage read was not measured. Cache metadata includes seed, radius, source data, and tunnel
mode. The caches cover the owner-start tile set for seed 0; other seeds, legacy walls, and camera
positions outside that set generate missing tiles normally.

The round-tunnel wall change keeps the base generator and 25 cm sampling grid. Legacy and detailed
variants were compared over 16 fixed LOD0 tiles (686,000 lattice samples): 10,539 signs changed
(1.5363%). Symmetric nearest-vertex distance was 0.1955 m mean and 2.0457 m maximum, with no
unmatched vertices. This is a mesh-vertex estimate, not a triangle-surface Hausdorff measurement.
The larger maximum is local to tunnel/room joins. Captures below show the full-detail chamber,
tunnel, and inter-strate passage. The far pair waits for all 6,498 clipmap tiles to upload.

| Fixed view | Legacy walls | Detailed/smoothed walls |
|---|---|---|
| Chamber `(-11.75,-123.25,-19.85) m`, yaw 0° | [capture](Research/VoxelSpeedRound/final/chamber_legacy.png) | [capture](Research/VoxelSpeedRound/final/chamber_detailed.png) |
| Tunnel `(-11.75,-123.25,-19.85) m`, yaw 90° | [capture](Research/VoxelSpeedRound/final/tunnel_legacy.png) | [capture](Research/VoxelSpeedRound/final/tunnel_detailed.png) |
| Passage `(-39.19,6.27,-28.51) m`, yaw 63° | [capture](Research/VoxelSpeedRound/final/passage_legacy.png) | [capture](Research/VoxelSpeedRound/final/passage_detailed.png) |
| Full clipmap `(-11.75,-123.25,-19.85) m`, yaw 90° | [capture](Research/VoxelSpeedRound/final/farclip_legacy.png) | [capture](Research/VoxelSpeedRound/final/farclip_detailed.png) |

The detailed wall field uses seeded value noise at 8, 16, and 32 voxels (2, 4, and 8 m), avoiding
per-voxel speckle. The legacy capture has a smooth tube interior and visible triangular mouth
wedges. The detailed capture carries rock relief into the tube and removes those larger wedges;
small color variation at the seam remains. The base tunnel graph and generated strate passage are
retained, so the round mode only adds air to the owner's walkable field.

`Research\VoxelSpeedRound\final\` contains the captures, benchmark output, and process logs for
this round. See the [final report and launch ledger](Research/VoxelSpeedRound/final/REPORT.md) for
PIDs and discarded runs. Each viewer/benchmark was started with `Build\LaunchBelowNormal.ps1`, which
prints its PID, BelowNormal priority, and `0xFF00` affinity. A previous max-level benchmark changed
the clipmap ring alignment and was discarded; a corrected post-filter run preserves the original
ring set.