VoxelForge/Standalone/SHARED_CPP_LIBRARY_PLAN.md

# VoxelForgeCore shared-library plan

This plan began with the `557f6aa` Core extraction and is updated with Unreal integration round 1.
Canonical implementation files remain under `Standalone/`; the plugin now has a `VoxelForgeCore`
runtime module and a headless schema-v1 asset adapter check. The `AVoxelWorld` runtime route is
still pending.

## Current layout and build

- `Core/include/VoxelForgeCore/VoxelForgeCore.h` is the public C++20 API. It includes only standard
  C++ headers and value types.
- `Core/src/VoxelForgeCore.cpp` adapts the copied density evaluator and marching-cubes mesher to the
  public API. `ShapeWorldInternal.h` owns shape compilation, density/detail evaluation, edits and
  audits. `FarShellsInternal.h` owns far-parent repairs and strata/connector shells.
- `Core/core_sources.txt` is the translation-unit manifest. `Build.ps1` reads it for both
  `-Target Core` and `-Target All`; the latter links the viewer against
  `Build/VoxelForgeCore/VoxelForgeCore.lib`. Use `-OutputName` to keep another executable open.
  Builds use `/std:c++20 /O2 /arch:AVX2 /fp:precise /MP8`, BelowNormal priority and affinity
  `0xFF00`.
- `Source/StandaloneShapeWorld.h` and `Source/StandaloneResearchTunnels.h` are forwarding headers
  for old standalone include sites. `StreamingWorld.h` remains host code: it owns worker threads,
  queued jobs, caches, cancellation, D3D upload state and per-frame upload budgets. When a core
  world is supplied, it gets desired tiles and their priority from the library and submits each
  tile to `World::generateTile`. Shape-world `--stream-benchmark`, `--stream-pipeline`, edit,
  audit, and `--benchmark-level` runs use the Core world. Explicit evaluator and owner-comparison
  modes retain the compatibility generator only as the comparison side of those diagnostics.
- The build compiles the copied CPU generator and mesher with the private compatibility headers
  under `Compat`. Those headers and their Unreal-shaped stand-ins are implementation details. They
  are absent from the public include tree and public API.

## Implementation namespace and Unreal link safety

The public `VoxelForgeCore.h` surface contains only standard C++ declarations. The copied
generator, mesher, and compatibility implementation compile under `vfcore::detail`, so engine
symbols such as `FVector`, `TArray`, `FMath`, and `UObject`-shaped stand-ins do not acquire the same
linker names as Unreal's definitions. `Core/src/VoxelForgeCore.cpp` owns the public API boundary;
it opens `vfcore::detail` only around its private implementation includes, then defines the
`VoxelForgeCore` API outside that namespace.

For each non-boundary source in `Core/core_sources.txt`, `Build.ps1` writes a one-source shim under
`Build/VoxelForgeCore/wrappers`. The shim includes `CoreNamespacePreamble.h` before opening
`vfcore::detail`, then includes the canonical source path from the manifest. The preamble loads
the standard/platform headers and stateless JSON parser before namespace entry, preventing system
declarations from being accidentally nested in `vfcore::detail`. The `VoxelForgeCore.cpp` row is
compiled directly because it sets up its own public/private namespace boundary. Keep one
implementation source per shim.

`Compat` still defines Unreal-shaped types and a few Unreal-named macros for the copied code, but
those declarations live in private implementation translation units only. The UBT module must
disable unity builds and PCHs for these shims, and must not force-include Unreal headers into
them. This keeps compatibility macros local to one translation unit. Normal plugin adapter
translation units may include `CoreMinimal.h` and the public `VoxelForgeCore.h` together; the
public header does not include any compatibility header or Unreal declaration.

`Build.ps1` runs `Core/CheckCoreSymbols.ps1` on the completed static library. The check invokes
`dumpbin /SYMBOLS`, derives copied Unreal-style type names from `Compat`, `Source`, and `Core/src`,
and fails on an unqualified global occurrence. It also fails closed if `dumpbin` returns too little
symbol data to inspect. Rebuilding the `557f6aa` manifest directly and running the same checker
found 16,255 unqualified Unreal-style type symbols; the namespaced build checks 177,569
symbol-table lines against 370 type names with no violations. The baseline and current logs are
retained in `Core/verification/link_safety_557f6aa_final.log` and
`Core/verification/link_safety_final.log`. `Core/verification/UnrealHeaderCoexistence.cpp` was
also compiled against the installed Unreal 5.7 `CoreMinimal.h` and the public header; see
`Core/verification/unreal_header_smoke_final.log`.

## API and ownership

`ResolvedWorldSnapshot::FromJson` takes the normalized UTF-8 schema-v1 JSON produced by the
standalone owner export. It resolves the input into owned read-only data. `World` owns the compiled
shape field and edit state for that snapshot. See [Core/API.md](Core/API.md) for signatures, units,
and examples.

The host supplies a `JobScheduler`. The core calls its synchronous `parallel_for` during shape
compilation; the host chooses the pool, fibers, task graph or serial execution. Core code starts no
threads. After compilation, hosts may run independent tile generation and query calls on their own
workers. Serialize compilation and edit application against reads/generation of the same `World`;
there is no internal lock or cancellation policy.

Tile meshes own positions, normals, UVs, colors and indices. Positions are centimetres; query and
tile coordinates are voxels. Group counts expose the ground index run, ceiling index run, and skirt
triangle subset. Density/sign, point-solid, segment-solid, and tile-classification queries support
host gameplay and collision checks without returning engine objects.

Edit transactions contain an authority-supplied increasing sequence and an ordered brush list. The
core hashes those exact values, accepts an identical replay idempotently, rejects sequence reuse
with different data and out-of-order transactions, and returns affected desired tiles grouped by
LOD. The host schedules `generateTile` for those keys and drops results made stale by its own
cancellation/version policy. Every client must apply the same sequence and brush order.

## Unreal integration: UBT module and adapter status

The plugin now declares `VoxelForgeCore` as a runtime module and links it from both `VoxelForge` and
`VoxelForgeEditor`. Its `Build.cs` sets C++20, exports only `Standalone/Core/include`, and keeps the
Standalone root plus these private include roots available to the core implementation:

- `Standalone`
- `Standalone/Core/src`
- `Standalone/Compat`
- `Standalone/Source`
- `Standalone/Source/VoxelForge/Public`
- `Standalone/Source/VoxelForge/Private`

The module uses `PCHUsage = PCHUsageMode.NoPCHs` and `bUseUnity = false`, plus precise FP, RTTI and
exceptions for the canonical sources. It also isolates an actual engine `CoreMinimal.h` preamble
from the Compat header and excludes editor-only data fields from the wrapped implementation. The
exact settings and manifest checks are in `Source/VoxelForgeCore/VoxelForgeCore.Build.cs`.

UBT discovers translation units under a module's `Source` tree. There is one thin `.cpp` include
shim per row of `Core/core_sources.txt`, and each includes exactly one canonical source file by a
path rooted at the private `Standalone` include path. For every implementation row except
`Core/src/VoxelForgeCore.cpp`, the shim includes `CoreNamespacePreamble.h`, opens
`namespace vfcore::detail`, then includes that one source. The special API row gets a shim that
includes `Core/src/VoxelForgeCore.cpp` directly; that source establishes its own namespace
boundary. Keeping one included source per shim preserves the translation-unit boundaries used by
`Build.ps1`. UBT validates every manifest row, checks the exact source name in each shim, and rejects
missing or extra shims. Do not copy implementation files into the module; standalone and UBT
compile the same canonical source bytes.

The implemented adapter currently lives in the editor-only `VoxelForgeExploreCommandlet`, not in
`AVoxelWorld`. It runs on the game thread, resolves owner asset data, and serializes settings,
strata, passages, seed, voxel scale, clip limits and viewer start into schema-v1 JSON accepted by
`FromJson`. It serializes values rather than UObject pointers and sorts map/set members and object
keys for determinism. The original asset-derived JSON parses as-is. This check does not yet create
the runtime world snapshot owned by `AVoxelWorld`.

The commandlet validation scheduler runs Core compile work at limits 1, 8, and 16; it is not the
plugin's UE task-system adapter. A runtime implementation still needs to schedule compile/tile work,
observe epochs/cancellation, and copy Core's owned mesh arrays/groups into `FVoxelMeshData` while
retaining the plugin's upload and collision path.

## Planned runtime route: not wired yet

The current plugin's `AVoxelWorld::GenerateTileResult` worker path remains active. It performs
proofs and `UVoxelGenerator::ClassifyTile`, calls `UVoxelMarchingCubesMesher::GenerateMesh`, builds
`RealtimeMesh` streams with `BuildTileStreamSet`, then publishes an epoch-tagged `FChunkResult` for
game-thread upload/collision handling. `AVoxelWorld::CarveAtPosition`, `CarveBox`, and
`CarveCapsule` route soft `FVoxelModification` brushes through `UVoxelDiffLayer::ApplyModification`
and remesh the affected chunks. The table below records the intended replacement; no row is wired
into `AVoxelWorld` yet:

| Current call or state | Core API replacement | Keep in the plugin |
| --- | --- | --- |
| Shape interval/detail-band build and any active shape-world dispatch | `World::compile` with explicit bounds/detail options | Asset resolution, build admission and cancellation |
| `GetDensityAt`, lattice sign tests, and point/segment checks in meshing/gameplay | `densityAt`, `densityAtLattice`, `isSolidAt`, `isSolidAlongSegment` | Gameplay policy and physics objects |
| `ClassifyTile` / tile proof | `classifyBox` | Clipmap residency and tile lifetime |
| `GenerateMesh` plus shape-world far-parent repair and shell stitching | `generateTile` | Convert owned result to `FVoxelMeshData`, call `BuildTileStreamSet`, and upload it |
| `VoxelClipmapDesiredTiles::Build` tile-set calculation | `desiredTiles` and `prioritizeTiles` | Decide when to request, cancel, retain or evict jobs |
| `UVoxelDiffLayer::ApplyModification` and affected-chunk scan | `applyEditTransaction` with `densityOffset` brushes carrying signed strength and falloff; regenerate returned LOD keys | Brush budgets, authority sequence, replication, and result acceptance |

The next round should add `bUseVoxelForgeCore=false` to `AVoxelWorld` and leave the existing path
unchanged when false. Pass the Core `World` context explicitly. Plugin epoch/cancellation checks
must reject stale results; they cannot interrupt an executing Core call. Drain in-flight Core calls
before applying a transaction, then submit affected tiles under a new epoch. Do not move worker
admission, collision cooking, actor or material creation, or rendering into the library.

The schema uses Core's 25 cm/voxel scale while the legacy plugin Runtime still defines
`VOXEL_SIZE=12.5 cm`; define and verify a scale mapping before converting Core centimeter positions
to plugin mesh payloads. The staged `DA_Settings` says `ClipRadius=6` and `MaxClipLevel=6`, whereas
the standalone owner JSON says radius 6/max level 8. The adapter test keeps the exported JSON as
authored and applies `clip_radius=3`/`max_clip_level=8` only in a reference-validation copy.

## Acceptance record

The `557f6aa` commit is the strict output baseline. The rebuilt namespaced Core matches all 11
full-detail, whole-view and LOD0–8 hash rows at W1 and W8, with W1/W8 parity. The step-29 CPU
reference also preserves its approved W16 hash set. The fresh W1/W8 records are
`Core/verification/last_standard_hash_w1.stdout.log`,
`Core/verification/last_standard_hash_w8.stdout.log`, and
`Core/verification/final_rebuilt_hash_parity.log`.

The look and shape audits pass, and walkability remains 5,642/5,642. The dedicated passage-camera
detail audit passes at 13,271 passage samples with a detail ratio of 1.152. Far survival passes
with 7,200 surfaces, zero missing surfaces or holes, and 10,976 far connectors. The damaged-shell
control removes 512 triangles, reports 16 missing boundary surfaces and 16 holes, and exits 14.
The final rebuilt logs are `Core/verification/final_rebuilt_shape_audit.stdout.log`,
`Core/verification/final_rebuilt_passage_camera_audit.stdout.log`,
`Core/verification/final_rebuilt_far_survival.stdout.log`, and
`Core/verification/final_rebuilt_bad_shell_control.stdout.log`.

The six fixed 1280×800 views were recaptured from the final rebuilt viewer and compare byte-for-byte
with the prior Core captures. Their images are `Core/verification/core_final_rebuild_*.png`; the SHA-256
results are in `Core/verification/final_rebuilt_capture_parity.log`. The single-tile
`--shape-world --benchmark-level=0` path reports `generator=VoxelForgeCore`.

The link-safety check demonstrates the failure before the change and the pass after it. Running it
on the `557f6aa` library finds 16,255 unqualified Unreal-style symbols; the rebuilt Core library
passes after checking 177,569 `dumpbin /SYMBOLS` lines against 370 copied Unreal-style type names.
`Core/verification/link_safety_557f6aa_final.log` and
`Core/verification/link_safety_final.log` retain the outputs. A smoke translation unit that
includes the installed Unreal 5.7 `CoreMinimal.h` and the public `VoxelForgeCore.h` also compiles;
see `Core/verification/unreal_header_smoke_final.log`.

**Speed gate: not met.** The no-hash, cold W8 `--stream-pipeline --shape-world --force-generate`
check was run in four interleaved baseline/Core pairs, each generating 2,226 tiles and the full
110.90 MiB mesh payload with no cache hits. `whole_view_ready` averaged 0.89175 s for the exact
`557f6aa` executable and 0.92050 s for the namespaced build, a 28.8 ms (3.2%) regression. Stream
wall time was effectively unchanged at 0.46000 s and 0.46025 s; the difference was in shape
compilation, including `intervals+signs` averaging 95.7 ms on baseline and 114.9 ms on Core. The
raw cold runs are in `Core/verification/final_rebuilt_pipeline_*.stdout.log`. API-only `/Ob3` and
`/Ob1` trials also retained hash parity but were slower, so the build is restored to `/O2 /MP8`.
The required “no slower” acceptance condition remains outstanding.

## Unreal integration round 1 record

BuildHost linked `VoxelForgeCore` and `VoxelForge`, compiled all 18 manifest shims, and passed the
real shim-object symbol audit (83,843 `dumpbin` lines / 399 copied Unreal-style type names). Five
staged owner asset copies matched their source SHA-256 values. The headless editor commandlet
exported schema-v1 JSON from those assets, parsed it with Core, and matched the four expected hashes
and 5,642/5,642 walkability at Core compile worker limits 1, 8, and 16. The initial tile generation
in this adapter check is serial after compile; the worker limits exercise Core compilation. Full
details and the run ledger are in `UNREAL_INTEGRATION_REPORT.md`.

This round did not add the `AVoxelWorld` switch, runtime task-graph scheduler, `generateTile` result
conversion, collision upload, epoch flow, or edit-transaction carving. The plugin continues to use
its existing generation path. Owner hand-off must wait until that runtime path is implemented and
headlessly validated.