VoxelForge/Standalone/SHARED_CPP_LIBRARY_PLAN.md

VoxelForgeCore shared-library plan

This plan describes the extracted shape-world pipeline and the next Unreal integration. The standalone implementation is under Standalone/Core; no file in the plugin's Source tree was changed.

Current layout and build

• Core/include/VoxelForgeCore/VoxelForgeCore.h is the public C++20 API. It includes only standard C++ headers and value types.
• Core/src/VoxelForgeCore.cpp adapts the copied density evaluator and marching-cubes mesher to the public API. ShapeWorldInternal.h owns shape compilation, density/detail evaluation, edits and audits. FarShellsInternal.h owns far-parent repairs and strata/connector shells.
• Core/core_sources.txt is the translation-unit manifest. Build.ps1 reads it for both -Target Core and -Target All; the latter links the viewer against Build/VoxelForgeCore/VoxelForgeCore.lib. Use -OutputName to keep another executable open. Builds use /std:c++20 /O2 /arch:AVX2 /fp:precise /MP8, BelowNormal priority and affinity 0xFF00.
• Source/StandaloneShapeWorld.h and Source/StandaloneResearchTunnels.h are forwarding headers for old standalone include sites. StreamingWorld.h remains host code: it owns worker threads, queued jobs, caches, cancellation, D3D upload state and per-frame upload budgets. When a core world is supplied, it gets desired tiles and their priority from the library and submits each tile to World::generateTile. Shape-world --stream-benchmark, --stream-pipeline, edit, audit, and --benchmark-level runs use the Core world. Explicit evaluator and owner-comparison modes retain the compatibility generator only as the comparison side of those diagnostics.
• The build manifest compiles the copied CPU generator and mesher with the private compatibility headers under Compat. Those headers and their Unreal-shaped stand-ins are implementation details. They are absent from the public include tree and public API.

API and ownership

ResolvedWorldSnapshot::FromJson takes the normalized UTF-8 schema-v1 JSON produced by the standalone owner export. It resolves the input into owned read-only data. World owns the compiled shape field and edit state for that snapshot. See Core/API.md for signatures, units, and examples.

The host supplies a JobScheduler. The core calls its synchronous parallel_for during shape compilation; the host chooses the pool, fibers, task graph or serial execution. Core code starts no threads. After compilation, hosts may run independent tile generation and query calls on their own workers. Serialize compilation and edit application against reads/generation of the same World; there is no internal lock or cancellation policy.

Tile meshes own positions, normals, UVs, colors and indices. Positions are centimetres; query and tile coordinates are voxels. Group counts expose the ground index run, ceiling index run, and skirt triangle subset. Density/sign, point-solid, segment-solid, and tile-classification queries support host gameplay and collision checks without returning engine objects.

Edit transactions contain an authority-supplied increasing sequence and an ordered brush list. The core hashes those exact values, accepts an identical replay idempotently, rejects sequence reuse with different data and out-of-order transactions, and returns affected desired tiles grouped by LOD. The host schedules generateTile for those keys and drops results made stale by its own cancellation/version policy. Every client must apply the same sequence and brush order.

Unreal integration: UBT module

Add a VoxelForgeCore runtime module to the plugin. Its Build.cs sets CppStandardVersion.Cpp20, exports only Standalone/Core/include, and keeps these private include roots available to the core implementation:

• Standalone/Core/src
• Standalone/Compat
• Standalone/Source
• Standalone/Source/VoxelForge/Public
• Standalone/Source/VoxelForge/Private

UBT discovers translation units under a module's Source tree. Add one thin .cpp include shim per row of Core/core_sources.txt, with each shim including exactly one canonical source file by a relative path into Standalone. Keeping one included source per shim preserves the translation unit boundaries used by Build.ps1. A manifest check in the UBT module should fail when a manifest row lacks a shim or a shim names a different file. Do not copy implementation files into the module. The standalone and UBT builds then compile the same source bytes.

The existing VoxelForge module owns the adapter and calls the new module. Its adapter runs on the game thread, resolves soft references and asset defaults, preserves array order, and serializes settings, biome and terrain-op records, strata layouts, passages, seed, voxel size, clip limits and viewer start into the exact schema-v1 document accepted by FromJson. It must not serialize UObject pointers or depend on unordered map iteration. Snapshot creation returns a descriptive error for unresolved or unsupported assets. Once created, workers only see the immutable core snapshot and standard C++ results.

The plugin supplies a JobScheduler backed by its chosen task system. The adapter submits shape build work and tile jobs to that scheduler, observes generation IDs for cancellation, and discards stale results. It copies completed TileMesh data into FVoxelMeshData or the plugin's current mesh payload on the host side, preserving centimetre positions, normals, UVs, colors, indices and ground/ceiling/skirt group counts. UObject creation, collision cooking, render-resource changes, game-thread application, residency and upload budgets remain in the plugin.

What the plugin pipeline replaces

The current plugin's AVoxelWorld::GenerateTileResult worker path performs proofs and UVoxelGenerator::ClassifyTile, calls UVoxelMarchingCubesMesher::GenerateMesh, builds RealtimeMesh streams with BuildTileStreamSet, then publishes an epoch-tagged FChunkResult for game-thread upload/collision handling. AVoxelWorld::CarveAtPosition, CarveBox, and CarveCapsule route soft FVoxelModification brushes through UVoxelDiffLayer::ApplyModification and remesh the affected chunks. The adapter replaces those CPU-field and edit operations as follows:

Current call or state	Core API replacement	Keep in the plugin
Shape interval/detail-band build and any active shape-world dispatch	World::compile with explicit bounds/detail options	Asset resolution, build admission and cancellation
GetDensityAt, lattice sign tests, and point/segment checks in meshing/gameplay	densityAt, densityAtLattice, isSolidAt, isSolidAlongSegment	Gameplay policy and physics objects
ClassifyTile / tile proof	classifyBox	Clipmap residency and tile lifetime
GenerateMesh plus shape-world far-parent repair and shell stitching	generateTile	Convert owned result to FVoxelMeshData, call BuildTileStreamSet, and upload it
VoxelClipmapDesiredTiles::Build tile-set calculation	desiredTiles and prioritizeTiles	Decide when to request, cancel, retain or evict jobs
UVoxelDiffLayer::ApplyModification and affected-chunk scan	applyEditTransaction with densityOffset brushes carrying signed strength and falloff; regenerate returned LOD keys	Brush budgets, authority sequence, replication, and result acceptance

Delete plugin-side active-world dispatch when routing these calls; the World context is an explicit object passed to each operation. The plugin's epoch/cancellation checks still reject stale results, but cannot interrupt an executing core call. Because edit application replaces field edit state, the host must drain in-flight core calls before applying a transaction and then submit the affected tiles under a new epoch. Do not move worker admission, collision cooking, actor or material creation, or rendering into the library.

Acceptance record

The retained Round 11 executable is the reference at checkout 631dd57. The final Core build matched its 11 hash rows (full detail, whole view and LOD0–8) at both W1 and W8. It retained the 5,642/5,642 route audit, passed the shape/look and far-survival audits, and the damaged-shell control still reports 512 removed triangles, 16 missing boundary surfaces, 16 holes, and exit 14. Six fixed 1280×800 captures compare byte-for-byte with the Round 11 references. The final single-tile --shape-world --benchmark-level=0 path reports generator=VoxelForgeCore.

The final no-hash, cold W8 direct full-payload benchmark was interleaved reference/core/core/reference with --stream-benchmark --shape-world --force-generate --workers=8 --clip-radius=3. Process wall means were 1.041 s for the reference (1.035, 1.047) and 1.0245 s for Core (1.037, 1.012), 16.5 ms faster. Each run generated 2,226 tiles with the full 110.90 MiB payload and zero cache hits.

The separate neighborhood-first --stream-pipeline benchmark was repeated in eight interleaved processes per build. Whole-view-ready means were 0.887 s for the reference (0.878–0.899 s) and 0.894 s for Core (0.880–0.932 s). The 7 ms mean difference is smaller than the 54 ms combined run-to-run range; Core reached the 27-tile neighborhood and 960-tile full-detail milestones at comparable times. Direct benchmark, pipeline, audit, and capture logs are retained in Core/verification.