VoxelForge/Standalone

VoxelForge standalone world

This C++20 program loads the owner snapshot from data/owner_world.json, runs the VoxelForge CPU pipeline, streams the clipmap with up to sixteen host worker threads, and displays it through Direct3D 11. Core/ contains the Unreal-free VoxelForgeCore API and static-library target used by the standalone shape-world path. Unreal is not linked or needed at run time.

Build and launch

From the plugin root, run:

.\Standalone\Build.ps1 -Target All
.\Standalone\bin\VoxelForgeCoreStandalone.exe --shape-world

Build.ps1 -Target Core builds only Build\VoxelForgeCore\VoxelForgeCore.lib; -Target All builds that library and the viewer. Core\core_sources.txt is the shared translation-unit manifest. The defaults use MSVC /std:c++20, /O2, /arch:AVX2, /fp:precise, /MP16, Normal priority and affinity 0xFFFF. Use -CompileWorkers 8 -BuildPriority BelowNormal for a lower-impact build. Pass another executable name to build beside any viewer that may be open by setting -OutputName, for example VoxelForgeCoreStandalone_test.exe. The executable finds the JSON under Standalone\data\. The isolated viewer links Source\GpuFarViewerLinkStubs.cpp; GPU Far rendering is not part of this CPU viewer target.

BuildHost resource controls

The standalone viewer accepts --workers=1..16, --process-priority=Normal and --affinity-all-cpu. Its draw loop also has a 60 FPS ceiling by default, including when DXGI reports an occluded window; pass --frame-limit=1..60 to choose a lower ceiling for captures and benchmarks. VSync remains enabled unless --no-vsync is passed. The verification runner in Core\verification\RunMachine.ps1 retains its separate BelowNormal/8-worker limits and refuses to overwrite an existing log stem.

Controls

• Hold right mouse button and move to look.
• WASD moves; Space/Ctrl or E/Q moves vertically; Shift moves faster; the wheel changes speed.
• F toggles wireframe.
• 1 through 7 toggle clipmap levels 0 through 6 individually; 0 toggles every level.
• N/P changes the seed. T switches between the owner's base world and the added research-shaped round, winding tunnels; --round-tunnels starts in that variant. G switches between legacy smooth and detailed/smoothed tunnel walls. Either action rebuilds the clipmap. Crossing a chunk recenters streaming: tiles still desired stay resident, only new tiles are generated, and leaving tiles are dropped.
• Esc exits.

The console reports density parity at startup, each level's generation time, and the total time.

Copied code and compatibility layer

Source\VoxelForge\ contains the plugin's density evaluator and operator stack, cave morphology, passage geometry, strate definitions/layout/manager/composer, noise, voxel math and types, marching-cubes tables/mesher, and clipmap tile selection. These are copies. Plugin runtime source is untouched apart from the allowed editor commandlet export addition.

Compat\CoreMinimal.h supplies the UE-style arrays/maps/sets, vectors and boxes, math helpers, strings, smart pointers, locks, logging/stat macros, UObject-like data holders, and platform helpers needed by the copied CPU code. The Compat subfolders cover the UE include paths used by those copies. Perlin 2D/3D uses the copied UE 5.7 implementation in UnrealPerlin.cpp and UnrealPerlinHelpers.inl. OwnerWorldLoader.h maps the exported settings, resolved strate definitions, biomes, and terrain operations into copied holders; soft references resolve through a small local asset registry. The copied strate manager reconstructs runtime passages from that data and seed. The export also records their geometry in JSON for reference.

Behavior differences

• Standalone VOXEL_SIZE is 25 cm; the plugin's checked-out value remains 12.5 cm. No other voxel conversion constant needed changing. Mesh vertices stay in centimetres in the copied mesher and the viewer converts them to metres at upload.
• The compatibility TMap uses an open-addressed hash index with stable pair storage; TSet uses that index for lookup. Iteration follows insertion order rather than Unreal hash-table order. Owner strate slots and passages are read from ordered JSON arrays. Perlin noise and deterministic DetSinCos are copied unchanged; MSVC uses /fp:precise without fast-math/FMA contraction.
• Unreal reflection and UObject construction are replaced by explicit JSON-to-holder mapping. Density-relevant settings, resolved strate layouts, passages, biome parameters, and terrain-op assets are loaded. Runtime actor/material assets are not instantiated; the viewer uses one lit rock material for all generated geometry.
• The owner export has no active season. Standalone season loading is not implemented; an active season would need to be exported and mapped before it could affect this viewer.
• WITH_EDITOR research/debug branches, Unreal logging, startup trace, profiling counters, and reflection metadata do not run in the standalone. The base generator path is the default.
• T selects a standalone-only tunnel-shape port. It preserves the owner's rooms, tunnels, and passages, then adds the research field's cubic winding centerline and round elliptical tunnel profile (144-voxel pitch, 28-voxel radius, 26-voxel half-height, deterministic variable radius and vertical wave) at the fixed world height -80 voxels. Detailed walls use deterministic relief at 8-, 16-, and 32-voxel wavelengths; the fixed-height field and copied DetSin keep the world independent of camera placement and CRT trigonometry. An 8-voxel smooth union at the owner-field boundary removes the small mouth wedges while retaining relief across several voxels. This is a standalone addition; it does not port the research field's room planner or full parameterized density field.

Owner data and parity

data/owner_world.json came from one default-off VoxelForgeExploreCommandlet export using the guarded Tools\VoxelForgeTest.ps1 -Scenario standalone-export -Build -Assets owner path. The snapshot contains the resolved settings/strates/passages and 3,436 plugin density samples. For that commandlet build, VOXEL_SIZE was temporarily set to 25 cm in the worktree and restored afterward; the exported JSON records 25 cm and the checked-out plugin source still records 12.5 cm.

Standalone comparison: 3,436/3,436 bit-identical, zero mismatches. Samples covered strate boundaries (2,420), varied world coordinates including negative and far-from-origin points (768), tunnel interiors (222), room interiors (12), inter-strate passages (10), and passage landings (4).

Baseline clipmap timing

Owner seed 0, base world, 25 cm voxels, six workers; the final viewer run generated the full seven-level clipmap (12,054 tiles, 6,605,189 vertices, 36,307,137 indices) in 515.117 seconds. These level timers cover CPU meshing and queue handoff; D3D11 buffer upload is separate. An earlier headless pass over the same tiles took 408.719 seconds, so use one run mode consistently when comparing later speed changes.

Level	Seconds	Tiles	Vertices	Density samples
0	70.201	2,744	957,655	117.65 M
1	65.376	2,009	657,423	86.14 M
2	9.042	1,274	292,267	8.74 M
3	25.689	588	235,755	4.03 M
4	24.248	441	229,720	3.02 M
5	23.809	294	199,503	2.02 M
6	296.304	4,704	4,032,866	32.26 M
Total	515.117	12,054	6,605,189	253.86 M

Profiling result and speed change

The compatibility layer's old TMap/TSet linear lookups and heap-only vector growth were replaced with an open-addressed hash index, stable pair storage, small inline storage, and a thread-local pooled allocator. That removes known container and allocation mismatches with the Unreal path, but did not improve the earlier LOD0 timing (72.1 s before, 74.1 s after); profiling showed those were not the dominant cost. The same audit also found two recursive room-graph walks wrapped in std::function, which could allocate and dispatch indirectly on every graph build. They now use self-recursive generic lambdas with the same depth-first visit order, removing that wrapper overhead from the generation hot path. In matched profile runs, RoomGraphBuild fell from 4.531 to 4.417 s for 64 LOD5 tiles and from 5.210 to 5.038 s for 64 LOD6 tiles; graph build counts and density hashes were unchanged.

Per-stage timers and cache counters found the real hotspot in GetDensityAt: the room/SDF graph cache is keyed by exact XY bounds, and the original X-fast lattice walk kept rebuilding it. On one LOD3 tile, cache-region grouping cut graph builds from 1,084 to 229 while writing samples to their original lattice indices. A 64-tile LOD6 profile attributed 13.189 s of 19.874 s to 36,480 room graph builds. For tiles wider than 16 chunks, keeping each XY column's Z run contiguous reduced that to 12,160 builds and 5.210 s of 10.972 s, with the same lattice hash. The final recursive- lambda profile measured 5.038 s in room-graph builds and 10.600 s total for the same 64 LOD6 tiles. Finer tiles retain the plane-first traversal, which is faster for their shorter cache runs.

Cache-slot experiments found a second scale-specific case: 32-chunk LOD5 tiles improve with two room graph cache slots (26,926 to 9,700 builds; profiled RoomGraphBuild 10.151 to 4.475 s across 64 tiles). Two slots regress 64-chunk LOD6 tiles (12,160 to 23,104 builds), so the generator uses two only for 32-chunk tile contexts and one otherwise. The selected-tile density hash is unchanged.

The controlled before/after run below meshes the 64 nearest tiles for each LOD serially, without profiling or hash collection. It uses the same owner seed and camera-centered desired set. The selected-tile lattice hashes from a separate final run are identical at every LOD.

LOD	64-tile density hash
0	0x3d31a090650399d3
1	0x6150c607c7a5c76f
2	0x9f1d560bc14e67f1
3	0xdd3e76020ce69cd7
4	0x59a209a5db034b46
5	0x1fdeb60160e4216f
6	0xbac69461f812493d

LOD	Before ms/tile	After ms/tile	After p50	After p95
0	50.4	50.551	50.521	63.148
1	112.3	110.175	33.030	407.767
2	23.9	22.430	12.015	51.078
3	223.0	73.482	69.614	94.331
4	269.0	133.512	98.003	214.854
5	393.1	127.325	116.386	173.401
6	282.8	145.457	140.952	183.234

Run mode: BelowNormal priority, serial headless benchmark, 64 nearest tiles per LOD, profiling and hash collection off; final executable VoxelForgeStandalone_night20.exe. The before timings are from the earlier 64-tile baseline. The user's prior plugin reference is about 95 ms worker time for a 25 cm LOD0 tile. The final nearest-tile LOD0 average is 50.6 ms. In the complete streamed run, LOD0 median was 82.9 ms, but its mean was 197.6 ms and p95 was 668.8 ms. The plugin was not timed on the same tile coordinates, so the rough reference comparison is not apples-to-apples and the streamed tail does not meet that target yet.

With 16 BelowNormal workers, the final full run completed all 12,054 tiles in 175.056 s, down from the historical 515.117 s six-worker run. It reached full detail after 66.565 s. Worker time per level in the final run was:

LOD	Worker sum	Tiles	Avg/tile	p50	p95
0	542.284 s	2,744	197.625 ms	82.946 ms	668.796 ms
1	501.518 s	2,009	249.635 ms	89.990 ms	742.258 ms
2	68.712 s	1,274	53.934 ms	55.633 ms	120.102 ms
3	77.055 s	588	131.047 ms	122.697 ms	212.157 ms
4	103.552 s	441	234.812 ms	192.137 ms	418.249 ms
5	66.689 s	294	226.834 ms	223.139 ms	328.030 ms
6	1,439.125 s	4,704	305.936 ms	303.062 ms	391.440 ms

The worker sums are concurrent wall-time measurements and therefore add to more than the 175.056 s clipmap wall time. The historical and final full runs used different worker counts; the serial 64-tile table is the controlled per-tile comparison. The full run also passed density parity at 3,436/3,436 samples with zero mismatches.

Streaming and uploads

The viewer rebuilds the desired tile set at each chunk crossing, keeps the intersection resident, cancels queued jobs that left the set, and enqueues only missing tiles. A heap orders by camera distance, then prefers finer LOD at equal distance. Workers generate CPU meshes concurrently. D3D11 uploads are limited to four batches or 2 ms per frame. A leaving or changed batch keeps its prior GPU buffer until its replacement is uploaded, avoiding a hole while the batch is rebuilt.

The final 16-worker run reached full detail 66.565 s after spawn (4,753 LOD0–1 desired tiles; 5,173 resident through LOD1) and the full clipmap at 175.056 s. In a controlled one-chunk crossing on the same executable, 4,856 tiles stayed resident, 441 left, and only 441 new jobs were submitted. Those jobs covered 18.91 M density samples, used 88.573 worker-seconds, and finished in 6.108 s wall time; all 441 were generated with no stale results or queued cancellations. The upload budget remained four batches / 2 ms per frame.

For a deterministic movement check, --test-one-chunk-crossing moves once after full-detail readiness and closes the test viewer after that crossing's jobs and bounded uploads drain.

Lighting comparison

The copied plugin mesher already computes central-difference density-gradient normals on the sampled density lattice and interpolates them along marching-cubes edges. The standalone passes those normals through to D3D11, so replacing face normals was unnecessary. Profiling measured this existing gradient stage at 2.575 ms across 64 LOD0 benchmark tiles (about 72 ms over all 2,744 streamed LOD0 tiles); this round adds no density samples or meshing work for normals.

The visible striping was amplified by the viewer's squared, high-intensity camera-facing light. The shader now uses a broad linear camera fill at similar average strength. Geometry, density, and LOD are unchanged; the shading path uses fewer multiplications. The D3D11 pixel-shader adjustment has no CPU mesh-generation cost.

Both captures use the default seed, default camera, 1280x800 view, and are saved from the D3D11 back buffer after all full-detail tiles upload:

Before	After

Tunnel palette pass

As a separate, render-only visual pass, the neutral blue-gray rock constants were changed to warm limestone and ochre, with a softer cyan camera fill. This follows the color balance of the approved cave reference while leaving its mesh, 25 cm resolution, and detail untouched. The shader changes only existing constant values and adds no texture reads or operations; CPU density and meshing cost is zero. The matched 16-worker captures generated the same 7,521,175 vertices and 41,506,623 indices; full-clipmap wall times were 172.772 s before and 171.676 s after, within run variance.

Both images show seed 0 with the round/winding tunnel field, at 1280x800 after the full clipmap uploaded. Camera: (-11.75, -123.25, -19.85) m, yaw 90 degrees, pitch 0. Capture mode used 16 BelowNormal workers:

Before palette pass	After palette pass

2026-10 speed round

The owner export still says ClipRadius=6, which selects a 112 m full-detail cube. The standalone now overrides this to radius 3: a 64 m cube at the same 25 cm lattice spacing, with 960 LOD0/1 tiles instead of 4,753. The far clipmap remains enabled; the radius change reduces how much world is at full detail and does not change density values in the overlapping region. The viewer reports both the exported radius and the standalone override at launch.

Path	Before	After
Full detail, uncached	112.243 s, radius 6, 4,753 tiles, 8 workers	20.266 s, radius 3, 960 tiles, 6 workers
Full detail, seed-0 base cache hit	—	0.270 s cache read + 0.028 s headless handoff; viewer full-detail ready at 0.059 s
Full clipmap, uncached	175.056 s, radius 6, 12,054 tiles, 16-worker historical night run	170.642 s, radius 3, 6,498 tiles, 8 workers
Full clipmap, seed-0 base cache hit	—	0.266 s cache read + 0.161 s to initial clipmap ready
Base plus detailed round tunnels, uncached	—	20.724 s full detail; 173.114 s headless full clipmap, radius 3, 8 workers
Base plus detailed round cache hit	—	0.282 s cache read + 0.028 s headless detail; viewer ready at 0.058 s
Detailed round full clipmap, cache hit	—	0.282 s cache read + 0.092 s headless; viewer ready at 0.165 s

All new timings use BelowNormal priority and affinity 0xFF00; worker counts are shown above. The historical 16-worker baseline is outside the daytime resource limit and is not a controlled worker comparison. Raw generation still misses both time targets. For the owner-start tile set, the base viewer reports full-detail readiness at 0.059 s and full-clipmap readiness at 0.161 s after generation starts. The detailed round viewer reports 0.058 s and 0.165 s. Including their 0.266 s / 0.284 s cache reads, both modes are below 0.45 s from cache-read start. The base cache is data/VoxelForge_seed0_r3_base.vfmesh (215.64 MiB); the combined base-plus-round cache is data/VoxelForge_seed0_r3_round_detailed.vfmesh (226.55 MiB). Their raw bakes took 170.642 s and 173.114 s. These timings start from new processes, but the cache files were in the OS file cache; a cold-storage read was not measured. Cache metadata includes seed, radius, source data, and tunnel mode. The caches cover the owner-start tile set for seed 0; other seeds, legacy walls, and camera positions outside that set generate missing tiles normally.

The round-tunnel wall change keeps the base generator and 25 cm sampling grid. Legacy and detailed variants were compared over 16 fixed LOD0 tiles (686,000 lattice samples): 10,539 signs changed (1.5363%). Symmetric nearest-vertex distance was 0.1955 m mean and 2.0457 m maximum, with no unmatched vertices. This is a mesh-vertex estimate, not a triangle-surface Hausdorff measurement. The larger maximum is local to tunnel/room joins. Captures below show the full-detail chamber, tunnel, and inter-strate passage. The far pair waits for all 6,498 clipmap tiles to upload.

Fixed view	Legacy walls	Detailed/smoothed walls
Chamber (-11.75,-123.25,-19.85) m, yaw 0°	capture	capture
Tunnel (-11.75,-123.25,-19.85) m, yaw 90°	capture	capture
Passage (-39.19,6.27,-28.51) m, yaw 63°	capture	capture
Full clipmap (-11.75,-123.25,-19.85) m, yaw 90°	capture	capture

The detailed wall field uses seeded value noise at 8, 16, and 32 voxels (2, 4, and 8 m), avoiding per-voxel speckle. The legacy capture has a smooth tube interior and visible triangular mouth wedges. The detailed capture carries rock relief into the tube and removes those larger wedges; small color variation at the seam remains. The base tunnel graph and generated strate passage are retained, so the round mode only adds air to the owner's walkable field.

Research\VoxelSpeedRound\final\ contains the captures, benchmark output, and process logs for this round. See the final report and launch ledger for PIDs and discarded runs. Each viewer/benchmark was started with Build\LaunchBelowNormal.ps1, which prints its PID, BelowNormal priority, and 0xFF00 affinity. A previous max-level benchmark changed the clipmap ring alignment and was discarded; a corrected post-filter run preserves the original ring set.