standalone: VoxelForgeCore library -- the fast generator behind a clean C++20 API, output unchanged

- Standalone/Core: public header VoxelForgeCore.h (no Unreal types), static library target,
  ResolvedWorldSnapshot (from the schema-v1 JSON), World (compile, generateTile, density/sign,
  point/segment solid, classifyBox), host-supplied JobScheduler (the core starts no threads),
  sequenced, idempotent EditTransaction returning affected tiles per LOD.
- The viewer, benchmarks, edits and audits run on Core; StreamingWorld stays host code.
- Same as round 11: 11 hash rows at W1 and W8, routes 5642/5642, look/passage/far audits pass,
  bad-shell control fails, six fixed captures byte-identical; cold W8 1.025 s vs 1.041 s reference.
- SHARED_CPP_LIBRARY_PLAN.md rewritten for a UBT module compiling the same sources via shims.
Open issue before Unreal: Core's implementation still compiles the Compat stand-ins named like
Unreal types (TArray, FVector...); inside a UE binary that is an ODR clash. Must be namespaced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MmSoF64BbRFmk9C3p1VsBJ

Standalone/SHARED_CPP_LIBRARY_PLAN.md

@@ -1,74 +1,134 @@
-# Shared C++ library plan
+# VoxelForgeCore shared-library plan
 
-This round keeps the standalone copy in place. The next extraction should make one implementation
-serve both hosts; it should not turn the compatibility layer into a second public API.
+This plan describes the extracted shape-world pipeline and the next Unreal integration. The
+standalone implementation is under `Standalone/Core`; no file in the plugin's `Source` tree was
+changed.
 
-## Library contents
+## Current layout and build
 
-Move the deterministic CPU pipeline into `VoxelForgeCore`:
+- `Core/include/VoxelForgeCore/VoxelForgeCore.h` is the public C++20 API. It includes only standard
+  C++ headers and value types.
+- `Core/src/VoxelForgeCore.cpp` adapts the copied density evaluator and marching-cubes mesher to the
+  public API. `ShapeWorldInternal.h` owns shape compilation, density/detail evaluation, edits and
+  audits. `FarShellsInternal.h` owns far-parent repairs and strata/connector shells.
+- `Core/core_sources.txt` is the translation-unit manifest. `Build.ps1` reads it for both
+  `-Target Core` and `-Target All`; the latter links the viewer against
+  `Build/VoxelForgeCore/VoxelForgeCore.lib`. Use `-OutputName` to keep another executable open.
+  Builds use `/std:c++20 /O2 /arch:AVX2 /fp:precise /MP8`, BelowNormal priority and affinity
+  `0xFF00`.
+- `Source/StandaloneShapeWorld.h` and `Source/StandaloneResearchTunnels.h` are forwarding headers
+  for old standalone include sites. `StreamingWorld.h` remains host code: it owns worker threads,
+  queued jobs, caches, cancellation, D3D upload state and per-frame upload budgets. When a core
+  world is supplied, it gets desired tiles and their priority from the library and submits each
+  tile to `World::generateTile`. Shape-world `--stream-benchmark`, `--stream-pipeline`, edit,
+  audit, and `--benchmark-level` runs use the Core world. Explicit evaluator and owner-comparison
+  modes retain the compatibility generator only as the comparison side of those diagnostics.
+- The build manifest compiles the copied CPU generator and mesher with the private compatibility
+  headers under `Compat`. Those headers and their Unreal-shaped stand-ins are implementation
+  details. They are absent from the public include tree and public API.
 
-- Plain voxel, tile, vector, bounds, and resolved-world input types.
-- Deterministic math, noise, density operations, cave and passage fields, strate composition, and
-  the room/SDF caches they use.
-- Marching-cubes tables and meshing, including the density-lattice gradient normals.
-- Clipmap desired-set calculation and the shared density evaluation entry points.
+## API and ownership
 
-Keep JSON and Unreal asset loading, material/actor creation, D3D11 and Unreal rendering, collision,
-worker scheduling, cancellation policy, and per-frame mesh upload budgets in the host applications.
-The cache-region traversal found in this round belongs with the density/meshing implementation.
-The worker-local room graph cache is keyed by exact XY bounds and stratum. Grouping samples by the
-legacy cache regions reduced a profiled LOD3 tile from 1,084 graph builds to 229. At coarser sizes,
-plane-first traversal still thrashes the same XY windows across a tall Z run, so tiles wider than 16
-chunks keep each XY column's Z samples contiguous. The profiled 64-tile LOD6 case fell from 36,480
-graph builds (13.189 s in graph construction) to 12,160 (5.210 s), with the same density hash.
+`ResolvedWorldSnapshot::FromJson` takes the normalized UTF-8 schema-v1 JSON produced by the
+standalone owner export. It resolves the input into owned read-only data. `World` owns the compiled
+shape field and edit state for that snapshot. See [Core/API.md](Core/API.md) for signatures, units,
+and examples.
 
-Cache capacity is also scale-dependent: a 32-chunk LOD5 tile benefits from two cached stratum
-windows (26,926 to 9,700 builds across 64 tiles), while a 64-chunk LOD6 tile regresses with two
-slots (12,160 to 23,104), so the current generator selects two only for 32-chunk tile contexts and
-one otherwise. Replacing the room-graph `std::function` recursion then reduced profiled LOD5
-RoomGraphBuild from 4.531 to 4.417 s and LOD6 from 5.210 to 5.038 s across the same 64 tiles,
-without changing graph build counts or density hashes. These timings are profiling evidence, not a
-cross-host benchmark. The shared API
-should own the traversal and cache policy, with parity tests covering both traversal paths while
-writing the same original density-grid indices.
+The host supplies a `JobScheduler`. The core calls its synchronous `parallel_for` during shape
+compilation; the host chooses the pool, fibers, task graph or serial execution. Core code starts no
+threads. After compilation, hosts may run independent tile generation and query calls on their own
+workers. Serialize compilation and edit application against reads/generation of the same `World`;
+there is no internal lock or cancellation policy.
 
-The compatibility-map and allocator change did not move the original LOD0 run; those changes remove
-known linear-lookup and heap-growth gaps, but stage profiling put the cost in density evaluation and
-room-graph cache churn. The hot room-graph walks also used an owning `std::function` compatibility
-alias for recursion; replacing them with generic self-recursive lambdas kept the same depth-first
-order and removed repeated type-erasure/allocation overhead. Keep that code in the core rather than
-carrying the UE-style callback alias into it. Viewer lighting and its palette remain host renderer
-choices and stay outside the shared generator.
+Tile meshes own positions, normals, UVs, colors and indices. Positions are centimetres; query and
+tile coordinates are voxels. Group counts expose the ground index run, ceiling index run, and skirt
+triangle subset. Density/sign, point-solid, segment-solid, and tile-classification queries support
+host gameplay and collision checks without returning engine objects.
 
-## Compatibility boundary
+Edit transactions contain an authority-supplied increasing sequence and an ordered brush list. The
+core hashes those exact values, accepts an identical replay idempotently, rejects sequence reuse
+with different data and out-of-order transactions, and returns affected desired tiles grouped by
+LOD. The host schedules `generateTile` for those keys and drops results made stale by its own
+cancellation/version policy. Every client must apply the same sequence and brush order.
 
-The core should expose standard C++ value types, explicit immutable resolved inputs, and owned mesh
-results. No Unreal headers, reflection macros, `UObject`, Unreal containers, renderer types, or host
-allocator should cross the boundary. The plugin adapter converts settings and loaded assets into
-the resolved input snapshot, then converts the returned vertices, indices, and normals into Unreal
-mesh types. The standalone JSON loader maps its exported snapshot into the same input snapshot.
+## Unreal integration: UBT module
 
-Do not move scheduling into the library. Each host should pass an immutable generation context to a
-worker and receive an owned result; this lets the plugin retain its task admission and game-thread
-rules while the viewer retains its worker pool and bounded D3D11 upload pump.
+Add a `VoxelForgeCore` runtime module to the plugin. Its `Build.cs` sets
+`CppStandardVersion.Cpp20`, exports only `Standalone/Core/include`, and keeps these private include
+roots available to the core implementation:
 
-## Plugin integration
+- `Standalone/Core/src`
+- `Standalone/Compat`
+- `Standalone/Source`
+- `Standalone/Source/VoxelForge/Public`
+- `Standalone/Source/VoxelForge/Private`
 
-Add a `VoxelForgeCore` C++20 target and make both builds compile or link that target from the same
-source files. For Unreal, first prototype the Unreal Build Tool integration: either a small core
-module that compiles the library sources or a platform-built static library with explicit build
-rules. Prefer source integration if packaging a binary per Unreal platform would add maintenance.
-The plugin then replaces its copied evaluator and mesher calls with the core API behind a thin
-adapter. Preserve plugin-owned UObject resolution, task cancellation, collision, game-thread mesh
-application, and upload budgets. The standalone keeps its JSON loader and D3D11 renderer.
+UBT discovers translation units under a module's `Source` tree. Add one thin `.cpp` include shim per
+row of `Core/core_sources.txt`, with each shim including exactly one canonical source file by a
+relative path into `Standalone`. Keeping one included source per shim preserves the translation
+unit boundaries used by `Build.ps1`. A manifest check in the UBT module should fail when a manifest
+row lacks a shim or a shim names a different file. Do not copy implementation files into the
+module. The standalone and UBT builds then compile the same source bytes.
 
-## Preventing drift before extraction
+The existing `VoxelForge` module owns the adapter and calls the new module. Its adapter runs on the
+game thread, resolves soft references and asset defaults, preserves array order, and serializes
+settings, biome and terrain-op records, strata layouts, passages, seed, voxel size, clip limits and
+viewer start into the exact schema-v1 document accepted by `FromJson`. It must not serialize
+UObject pointers or depend on unordered map iteration. Snapshot creation returns a descriptive
+error for unresolved or unsupported assets. Once created, workers only see the immutable core
+snapshot and standard C++ results.
 
-Until the common target exists, keep a manifest of mirrored files and a sync/check script that names
-the canonical source for each file, compares hashes, and refuses when both copies have changed.
-Avoid hand-editing only one copy without recording that the edit is pending synchronization. After
-each density or meshing change, run the 3,436-sample bitwise corpus and the selected-tile density
-hash check. Keep `/fp:precise`; do not accept evaluator changes that alter density bits as a speed
-shortcut. A standalone-only optimization must be marked as pending synchronization until it is
-ported or explicitly rejected for the canonical core. Delete the duplicate standalone source only
-after both host builds use the common target and the parity corpus passes through each adapter.
+The plugin supplies a `JobScheduler` backed by its chosen task system. The adapter submits shape
+build work and tile jobs to that scheduler, observes generation IDs for cancellation, and discards
+stale results. It copies completed `TileMesh` data into `FVoxelMeshData` or the plugin's current
+mesh payload on the host side, preserving centimetre positions, normals, UVs, colors, indices and
+ground/ceiling/skirt group counts. UObject creation, collision cooking, render-resource changes,
+game-thread application, residency and upload budgets remain in the plugin.
+
+## What the plugin pipeline replaces
+
+The current plugin's `AVoxelWorld::GenerateTileResult` worker path performs proofs and
+`UVoxelGenerator::ClassifyTile`, calls `UVoxelMarchingCubesMesher::GenerateMesh`, builds
+`RealtimeMesh` streams with `BuildTileStreamSet`, then publishes an epoch-tagged `FChunkResult` for
+game-thread upload/collision handling. `AVoxelWorld::CarveAtPosition`, `CarveBox`, and
+`CarveCapsule` route soft `FVoxelModification` brushes through `UVoxelDiffLayer::ApplyModification`
+and remesh the affected chunks. The adapter replaces those CPU-field and edit operations as
+follows:
+
+| Current call or state | Core API replacement | Keep in the plugin |
+| --- | --- | --- |
+| Shape interval/detail-band build and any active shape-world dispatch | `World::compile` with explicit bounds/detail options | Asset resolution, build admission and cancellation |
+| `GetDensityAt`, lattice sign tests, and point/segment checks in meshing/gameplay | `densityAt`, `densityAtLattice`, `isSolidAt`, `isSolidAlongSegment` | Gameplay policy and physics objects |
+| `ClassifyTile` / tile proof | `classifyBox` | Clipmap residency and tile lifetime |
+| `GenerateMesh` plus shape-world far-parent repair and shell stitching | `generateTile` | Convert owned result to `FVoxelMeshData`, call `BuildTileStreamSet`, and upload it |
+| `VoxelClipmapDesiredTiles::Build` tile-set calculation | `desiredTiles` and `prioritizeTiles` | Decide when to request, cancel, retain or evict jobs |
+| `UVoxelDiffLayer::ApplyModification` and affected-chunk scan | `applyEditTransaction` with `densityOffset` brushes carrying signed strength and falloff; regenerate returned LOD keys | Brush budgets, authority sequence, replication, and result acceptance |
+
+Delete plugin-side active-world dispatch when routing these calls; the `World` context is an
+explicit object passed to each operation. The plugin's epoch/cancellation checks still reject stale
+results, but cannot interrupt an executing core call. Because edit application replaces field edit
+state, the host must drain in-flight core calls before applying a transaction and then submit the
+affected tiles under a new epoch. Do not move worker admission, collision cooking, actor or material
+creation, or rendering into the library.
+
+## Acceptance record
+
+The retained Round 11 executable is the reference at checkout `631dd57`. The final Core build
+matched its 11 hash rows (full detail, whole view and LOD0–8) at both W1 and W8. It retained the
+5,642/5,642 route audit, passed the shape/look and far-survival audits, and the damaged-shell control
+still reports 512 removed triangles, 16 missing boundary surfaces, 16 holes, and exit 14. Six
+fixed 1280×800 captures compare byte-for-byte with the Round 11 references. The final single-tile
+`--shape-world --benchmark-level=0` path reports `generator=VoxelForgeCore`.
+
+The final no-hash, cold W8 direct full-payload benchmark was interleaved
+reference/core/core/reference with `--stream-benchmark --shape-world --force-generate --workers=8
+--clip-radius=3`. Process wall means were 1.041 s for the reference (1.035, 1.047) and 1.0245 s
+for Core (1.037, 1.012), 16.5 ms faster. Each run generated 2,226 tiles with the full 110.90 MiB
+payload and zero cache hits.
+
+The separate neighborhood-first `--stream-pipeline` benchmark was repeated in eight interleaved
+processes per build. Whole-view-ready means were 0.887 s for the reference (0.878–0.899 s) and
+0.894 s for Core (0.880–0.932 s). The 7 ms mean difference is smaller than the 54 ms combined
+run-to-run range; Core reached the 27-tile neighborhood and 960-tile full-detail milestones at
+comparable times. Direct benchmark, pipeline, audit, and capture logs are retained in
+`Core/verification`.