VoxelForge/Standalone/SHARED_CPP_LIBRARY_PLAN.md

# VoxelForgeCore shared-library plan

This plan describes the extracted shape-world pipeline and the next Unreal integration. The
standalone implementation is under `Standalone/Core`; no file in the plugin's `Source` tree was
changed.

## Current layout and build

- `Core/include/VoxelForgeCore/VoxelForgeCore.h` is the public C++20 API. It includes only standard
  C++ headers and value types.
- `Core/src/VoxelForgeCore.cpp` adapts the copied density evaluator and marching-cubes mesher to the
  public API. `ShapeWorldInternal.h` owns shape compilation, density/detail evaluation, edits and
  audits. `FarShellsInternal.h` owns far-parent repairs and strata/connector shells.
- `Core/core_sources.txt` is the translation-unit manifest. `Build.ps1` reads it for both
  `-Target Core` and `-Target All`; the latter links the viewer against
  `Build/VoxelForgeCore/VoxelForgeCore.lib`. Use `-OutputName` to keep another executable open.
  Builds use `/std:c++20 /O2 /arch:AVX2 /fp:precise /MP8`, BelowNormal priority and affinity
  `0xFF00`.
- `Source/StandaloneShapeWorld.h` and `Source/StandaloneResearchTunnels.h` are forwarding headers
  for old standalone include sites. `StreamingWorld.h` remains host code: it owns worker threads,
  queued jobs, caches, cancellation, D3D upload state and per-frame upload budgets. When a core
  world is supplied, it gets desired tiles and their priority from the library and submits each
  tile to `World::generateTile`. Shape-world `--stream-benchmark`, `--stream-pipeline`, edit,
  audit, and `--benchmark-level` runs use the Core world. Explicit evaluator and owner-comparison
  modes retain the compatibility generator only as the comparison side of those diagnostics.
- The build compiles the copied CPU generator and mesher with the private compatibility headers
  under `Compat`. Those headers and their Unreal-shaped stand-ins are implementation details. They
  are absent from the public include tree and public API.

## Implementation namespace and Unreal link safety

The public `VoxelForgeCore.h` surface contains only standard C++ declarations. The copied
generator, mesher, and compatibility implementation compile under `vfcore::detail`, so engine
symbols such as `FVector`, `TArray`, `FMath`, and `UObject`-shaped stand-ins do not acquire the same
linker names as Unreal's definitions. `Core/src/VoxelForgeCore.cpp` owns the public API boundary;
it opens `vfcore::detail` only around its private implementation includes, then defines the
`VoxelForgeCore` API outside that namespace.

For each non-boundary source in `Core/core_sources.txt`, `Build.ps1` writes a one-source shim under
`Build/VoxelForgeCore/wrappers`. The shim includes `CoreNamespacePreamble.h` before opening
`vfcore::detail`, then includes the canonical source path from the manifest. The preamble loads
the standard/platform headers and stateless JSON parser before namespace entry, preventing system
declarations from being accidentally nested in `vfcore::detail`. The `VoxelForgeCore.cpp` row is
compiled directly because it sets up its own public/private namespace boundary. Keep one
implementation source per shim.

`Compat` still defines Unreal-shaped types and a few Unreal-named macros for the copied code, but
those declarations live in private implementation translation units only. The UBT module must
disable unity builds and PCHs for these shims, and must not force-include Unreal headers into
them. This keeps compatibility macros local to one translation unit. Normal plugin adapter
translation units may include `CoreMinimal.h` and the public `VoxelForgeCore.h` together; the
public header does not include any compatibility header or Unreal declaration.

`Build.ps1` runs `Core/CheckCoreSymbols.ps1` on the completed static library. The check invokes
`dumpbin /SYMBOLS`, derives copied Unreal-style type names from `Compat`, `Source`, and `Core/src`,
and fails on an unqualified global occurrence. It also fails closed if `dumpbin` returns too little
symbol data to inspect. Rebuilding the `557f6aa` manifest directly and running the same checker
found 16,255 unqualified Unreal-style type symbols; the namespaced build checks 177,569
symbol-table lines against 370 type names with no violations. The baseline and current logs are
retained in `Core/verification/link_safety_557f6aa_final.log` and
`Core/verification/link_safety_final.log`. `Core/verification/UnrealHeaderCoexistence.cpp` was
also compiled against the installed Unreal 5.7 `CoreMinimal.h` and the public header; see
`Core/verification/unreal_header_smoke_final.log`.

## API and ownership

`ResolvedWorldSnapshot::FromJson` takes the normalized UTF-8 schema-v1 JSON produced by the
standalone owner export. It resolves the input into owned read-only data. `World` owns the compiled
shape field and edit state for that snapshot. See [Core/API.md](Core/API.md) for signatures, units,
and examples.

The host supplies a `JobScheduler`. The core calls its synchronous `parallel_for` during shape
compilation; the host chooses the pool, fibers, task graph or serial execution. Core code starts no
threads. After compilation, hosts may run independent tile generation and query calls on their own
workers. Serialize compilation and edit application against reads/generation of the same `World`;
there is no internal lock or cancellation policy.

Tile meshes own positions, normals, UVs, colors and indices. Positions are centimetres; query and
tile coordinates are voxels. Group counts expose the ground index run, ceiling index run, and skirt
triangle subset. Density/sign, point-solid, segment-solid, and tile-classification queries support
host gameplay and collision checks without returning engine objects.

Edit transactions contain an authority-supplied increasing sequence and an ordered brush list. The
core hashes those exact values, accepts an identical replay idempotently, rejects sequence reuse
with different data and out-of-order transactions, and returns affected desired tiles grouped by
LOD. The host schedules `generateTile` for those keys and drops results made stale by its own
cancellation/version policy. Every client must apply the same sequence and brush order.

## Unreal integration: UBT module

Add a `VoxelForgeCore` runtime module to the plugin and make the existing `VoxelForge` module depend
on it. Its `Build.cs` sets C++20, exports only `Standalone/Core/include`, and keeps the Standalone
root plus these private include roots available to the core implementation:

- `Standalone`
- `Standalone/Core/src`
- `Standalone/Compat`
- `Standalone/Source`
- `Standalone/Source/VoxelForge/Public`
- `Standalone/Source/VoxelForge/Private`

Set `PCHUsage = PCHUsageMode.NoPCHs` and `bUseUnity = false` for the Core module. For example:

```csharp
string PluginRoot = Path.GetFullPath(Path.Combine(ModuleDirectory, "../.."));
string Standalone = Path.Combine(PluginRoot, "Standalone");
PublicIncludePaths.Add(Path.Combine(Standalone, "Core", "include"));
PrivateIncludePaths.AddRange(new[] {
    Standalone,
    Path.Combine(Standalone, "Core", "src"),
    Path.Combine(Standalone, "Compat"),
    Path.Combine(Standalone, "Source"),
    Path.Combine(Standalone, "Source", "VoxelForge", "Public"),
    Path.Combine(Standalone, "Source", "VoxelForge", "Private")
});
CppStandard = CppStandardVersion.Cpp20;
PCHUsage = PCHUsageMode.NoPCHs;
bUseUnity = false;
```

UBT discovers translation units under a module's `Source` tree. Add one thin `.cpp` include shim per
row of `Core/core_sources.txt`, with each shim including exactly one canonical source file by a
path rooted at the private `Standalone` include path. For every implementation row except
`Core/src/VoxelForgeCore.cpp`, the shim includes `CoreNamespacePreamble.h`, opens
`namespace vfcore::detail`, then includes that one source. The special API row gets a shim that
includes `Core/src/VoxelForgeCore.cpp` directly; that source establishes its own namespace
boundary. Keeping one included source per shim preserves the translation-unit boundaries used by
`Build.ps1`. A manifest check in the UBT module should fail when a row lacks a shim, a shim names a
different file, or an extra shim exists. Do not copy implementation files into the module. The
standalone and UBT builds then compile the same canonical source bytes.

The existing `VoxelForge` module owns the adapter and calls the new module. Its adapter runs on the
game thread, resolves soft references and asset defaults, preserves array order, and serializes
settings, biome and terrain-op records, strata layouts, passages, seed, voxel size, clip limits and
viewer start into the exact schema-v1 document accepted by `FromJson`. It must not serialize
UObject pointers or depend on unordered map iteration. Snapshot creation returns a descriptive
error for unresolved or unsupported assets. Once created, workers only see the immutable core
snapshot and standard C++ results.

The plugin supplies a `JobScheduler` backed by its chosen task system. The adapter submits shape
build work and tile jobs to that scheduler, observes generation IDs for cancellation, and discards
stale results. It copies completed `TileMesh` data into `FVoxelMeshData` or the plugin's current
mesh payload on the host side, preserving centimetre positions, normals, UVs, colors, indices and
ground/ceiling/skirt group counts. UObject creation, collision cooking, render-resource changes,
game-thread application, residency and upload budgets remain in the plugin.

## What the plugin pipeline replaces

The current plugin's `AVoxelWorld::GenerateTileResult` worker path performs proofs and
`UVoxelGenerator::ClassifyTile`, calls `UVoxelMarchingCubesMesher::GenerateMesh`, builds
`RealtimeMesh` streams with `BuildTileStreamSet`, then publishes an epoch-tagged `FChunkResult` for
game-thread upload/collision handling. `AVoxelWorld::CarveAtPosition`, `CarveBox`, and
`CarveCapsule` route soft `FVoxelModification` brushes through `UVoxelDiffLayer::ApplyModification`
and remesh the affected chunks. The adapter replaces those CPU-field and edit operations as
follows:

| Current call or state | Core API replacement | Keep in the plugin |
| --- | --- | --- |
| Shape interval/detail-band build and any active shape-world dispatch | `World::compile` with explicit bounds/detail options | Asset resolution, build admission and cancellation |
| `GetDensityAt`, lattice sign tests, and point/segment checks in meshing/gameplay | `densityAt`, `densityAtLattice`, `isSolidAt`, `isSolidAlongSegment` | Gameplay policy and physics objects |
| `ClassifyTile` / tile proof | `classifyBox` | Clipmap residency and tile lifetime |
| `GenerateMesh` plus shape-world far-parent repair and shell stitching | `generateTile` | Convert owned result to `FVoxelMeshData`, call `BuildTileStreamSet`, and upload it |
| `VoxelClipmapDesiredTiles::Build` tile-set calculation | `desiredTiles` and `prioritizeTiles` | Decide when to request, cancel, retain or evict jobs |
| `UVoxelDiffLayer::ApplyModification` and affected-chunk scan | `applyEditTransaction` with `densityOffset` brushes carrying signed strength and falloff; regenerate returned LOD keys | Brush budgets, authority sequence, replication, and result acceptance |

Delete plugin-side active-world dispatch when routing these calls; the `World` context is an
explicit object passed to each operation. The plugin's epoch/cancellation checks still reject stale
results, but cannot interrupt an executing core call. Because edit application replaces field edit
state, the host must drain in-flight core calls before applying a transaction and then submit the
affected tiles under a new epoch. Do not move worker admission, collision cooking, actor or material
creation, or rendering into the library.

## Acceptance record

The `557f6aa` commit is the strict output baseline. The rebuilt namespaced Core matches all 11
full-detail, whole-view and LOD0–8 hash rows at W1 and W8, with W1/W8 parity. The fresh records are
`Core/verification/last_standard_hash_w1.stdout.log`,
`Core/verification/last_standard_hash_w8.stdout.log`, and
`Core/verification/final_rebuilt_hash_parity.log`.

The look and shape audits pass, and walkability remains 5,642/5,642. The dedicated passage-camera
detail audit passes at 13,271 passage samples with a detail ratio of 1.152. Far survival passes
with 7,200 surfaces, zero missing surfaces or holes, and 10,976 far connectors. The damaged-shell
control removes 512 triangles, reports 16 missing boundary surfaces and 16 holes, and exits 14.
The final rebuilt logs are `Core/verification/final_rebuilt_shape_audit.stdout.log`,
`Core/verification/final_rebuilt_passage_camera_audit.stdout.log`,
`Core/verification/final_rebuilt_far_survival.stdout.log`, and
`Core/verification/final_rebuilt_bad_shell_control.stdout.log`.

The six fixed 1280×800 views were recaptured from the final rebuilt viewer and compare byte-for-byte
with the prior Core captures. Their images are `Core/verification/core_final_rebuild_*.png`; the SHA-256
results are in `Core/verification/final_rebuilt_capture_parity.log`. The single-tile
`--shape-world --benchmark-level=0` path reports `generator=VoxelForgeCore`.

The link-safety check demonstrates the failure before the change and the pass after it. Running it
on the `557f6aa` library finds 16,255 unqualified Unreal-style symbols; the rebuilt Core library
passes after checking 177,569 `dumpbin /SYMBOLS` lines against 370 copied Unreal-style type names.
`Core/verification/link_safety_557f6aa_final.log` and
`Core/verification/link_safety_final.log` retain the outputs. A smoke translation unit that
includes the installed Unreal 5.7 `CoreMinimal.h` and the public `VoxelForgeCore.h` also compiles;
see `Core/verification/unreal_header_smoke_final.log`.

**Speed gate: not met.** The no-hash, cold W8 `--stream-pipeline --shape-world --force-generate`
check was run in four interleaved baseline/Core pairs, each generating 2,226 tiles and the full
110.90 MiB mesh payload with no cache hits. `whole_view_ready` averaged 0.89175 s for the exact
`557f6aa` executable and 0.92050 s for the namespaced build, a 28.8 ms (3.2%) regression. Stream
wall time was effectively unchanged at 0.46000 s and 0.46025 s; the difference was in shape
compilation, including `intervals+signs` averaging 95.7 ms on baseline and 114.9 ms on Core. The
raw cold runs are in `Core/verification/final_rebuilt_pipeline_*.stdout.log`. API-only `/Ob3` and
`/Ob1` trials also retained hash parity but were slower, so the build is restored to `/O2 /MP8`.
The required “no slower” acceptance condition remains outstanding.