VoxelForge/Standalone/SHARED_CPP_LIBRARY_PLAN.md

VoxelForgeCore shared-library plan

This plan began with the 557f6aa Core extraction and is updated with Unreal integration round 1. Canonical implementation files remain under Standalone/; the plugin now has a VoxelForgeCore runtime module and a headless schema-v1 asset adapter check. The AVoxelWorld runtime route is still pending.

Current layout and build

• Core/include/VoxelForgeCore/VoxelForgeCore.h is the public C++20 API. It includes only standard C++ headers and value types.
• Core/src/VoxelForgeCore.cpp adapts the copied density evaluator and marching-cubes mesher to the public API. ShapeWorldInternal.h owns shape compilation, density/detail evaluation, edits and audits. FarShellsInternal.h owns far-parent repairs and strata/connector shells.
• Core/core_sources.txt is the translation-unit manifest. Build.ps1 reads it for both -Target Core and -Target All; the latter links the viewer against Build/VoxelForgeCore/VoxelForgeCore.lib. Use -OutputName to keep another executable open. Builds use /std:c++20 /O2 /arch:AVX2 /fp:precise /MP8, BelowNormal priority and affinity 0xFF00.
• Source/StandaloneShapeWorld.h and Source/StandaloneResearchTunnels.h are forwarding headers for old standalone include sites. StreamingWorld.h remains host code: it owns worker threads, queued jobs, caches, cancellation, D3D upload state and per-frame upload budgets. When a core world is supplied, it gets desired tiles and their priority from the library and submits each tile to World::generateTile. Shape-world --stream-benchmark, --stream-pipeline, edit, audit, and --benchmark-level runs use the Core world. Explicit evaluator and owner-comparison modes retain the compatibility generator only as the comparison side of those diagnostics.
• The build compiles the copied CPU generator and mesher with the private compatibility headers under Compat. Those headers and their Unreal-shaped stand-ins are implementation details. They are absent from the public include tree and public API.

Implementation namespace and Unreal link safety

The public VoxelForgeCore.h surface contains only standard C++ declarations. The copied generator, mesher, and compatibility implementation compile under vfcore::detail, so engine symbols such as FVector, TArray, FMath, and UObject-shaped stand-ins do not acquire the same linker names as Unreal's definitions. Core/src/VoxelForgeCore.cpp owns the public API boundary; it opens vfcore::detail only around its private implementation includes, then defines the VoxelForgeCore API outside that namespace.

For each non-boundary source in Core/core_sources.txt, Build.ps1 writes a one-source shim under Build/VoxelForgeCore/wrappers. The shim includes CoreNamespacePreamble.h before opening vfcore::detail, then includes the canonical source path from the manifest. The preamble loads the standard/platform headers and stateless JSON parser before namespace entry, preventing system declarations from being accidentally nested in vfcore::detail. The VoxelForgeCore.cpp row is compiled directly because it sets up its own public/private namespace boundary. Keep one implementation source per shim.

Compat still defines Unreal-shaped types and a few Unreal-named macros for the copied code, but those declarations live in private implementation translation units only. The UBT module must disable unity builds and PCHs for these shims, and must not force-include Unreal headers into them. This keeps compatibility macros local to one translation unit. Normal plugin adapter translation units may include CoreMinimal.h and the public VoxelForgeCore.h together; the public header does not include any compatibility header or Unreal declaration.

Build.ps1 runs Core/CheckCoreSymbols.ps1 on the completed static library. The check invokes dumpbin /SYMBOLS, derives copied Unreal-style type names from Compat, Source, and Core/src, and fails on an unqualified global occurrence. It also fails closed if dumpbin returns too little symbol data to inspect. Rebuilding the 557f6aa manifest directly and running the same checker found 16,255 unqualified Unreal-style type symbols; the namespaced build checks 177,569 symbol-table lines against 370 type names with no violations. The baseline and current logs are retained in Core/verification/link_safety_557f6aa_final.log and Core/verification/link_safety_final.log. Core/verification/UnrealHeaderCoexistence.cpp was also compiled against the installed Unreal 5.7 CoreMinimal.h and the public header; see Core/verification/unreal_header_smoke_final.log.

API and ownership

ResolvedWorldSnapshot::FromJson takes the normalized UTF-8 schema-v1 JSON produced by the standalone owner export. It resolves the input into owned read-only data. World owns the compiled shape field and edit state for that snapshot. See Core/API.md for signatures, units, and examples.

The host supplies a JobScheduler. The core calls its synchronous parallel_for during shape compilation; the host chooses the pool, fibers, task graph or serial execution. Core code starts no threads. After compilation, hosts may run independent tile generation and query calls on their own workers. Serialize compilation and edit application against reads/generation of the same World; there is no internal lock or cancellation policy.

Tile meshes own positions, normals, UVs, colors and indices. Positions are centimetres; query and tile coordinates are voxels. Group counts expose the ground index run, ceiling index run, and skirt triangle subset. Density/sign, point-solid, segment-solid, and tile-classification queries support host gameplay and collision checks without returning engine objects.

Edit transactions contain an authority-supplied increasing sequence and an ordered brush list. The core hashes those exact values, accepts an identical replay idempotently, rejects sequence reuse with different data and out-of-order transactions, and returns affected desired tiles grouped by LOD. The host schedules generateTile for those keys and drops results made stale by its own cancellation/version policy. Every client must apply the same sequence and brush order.

Unreal integration: UBT module and adapter status

The plugin now declares VoxelForgeCore as a runtime module and links it from both VoxelForge and VoxelForgeEditor. Its Build.cs sets C++20, exports only Standalone/Core/include, and keeps the Standalone root plus these private include roots available to the core implementation:

• Standalone
• Standalone/Core/src
• Standalone/Compat
• Standalone/Source
• Standalone/Source/VoxelForge/Public
• Standalone/Source/VoxelForge/Private

The module uses PCHUsage = PCHUsageMode.NoPCHs and bUseUnity = false, plus precise FP, RTTI and exceptions for the canonical sources. It also isolates an actual engine CoreMinimal.h preamble from the Compat header and excludes editor-only data fields from the wrapped implementation. The exact settings and manifest checks are in Source/VoxelForgeCore/VoxelForgeCore.Build.cs.

UBT discovers translation units under a module's Source tree. There is one thin .cpp include shim per row of Core/core_sources.txt, and each includes exactly one canonical source file by a path rooted at the private Standalone include path. For every implementation row except Core/src/VoxelForgeCore.cpp, the shim includes CoreNamespacePreamble.h, opens namespace vfcore::detail, then includes that one source. The special API row gets a shim that includes Core/src/VoxelForgeCore.cpp directly; that source establishes its own namespace boundary. Keeping one included source per shim preserves the translation-unit boundaries used by Build.ps1. UBT validates every manifest row, checks the exact source name in each shim, and rejects missing or extra shims. Do not copy implementation files into the module; standalone and UBT compile the same canonical source bytes.

The implemented adapter currently lives in the editor-only VoxelForgeExploreCommandlet, not in AVoxelWorld. It runs on the game thread, resolves owner asset data, and serializes settings, strata, passages, seed, voxel scale, clip limits and viewer start into schema-v1 JSON accepted by FromJson. It serializes values rather than UObject pointers and sorts map/set members and object keys for determinism. The original asset-derived JSON parses as-is. This check does not yet create the runtime world snapshot owned by AVoxelWorld.

The commandlet validation scheduler runs Core compile work at limits 1, 8, and 16; it is not the plugin's UE task-system adapter. A runtime implementation still needs to schedule compile/tile work, observe epochs/cancellation, and copy Core's owned mesh arrays/groups into FVoxelMeshData while retaining the plugin's upload and collision path.

Planned runtime route: not wired yet

The current plugin's AVoxelWorld::GenerateTileResult worker path remains active. It performs proofs and UVoxelGenerator::ClassifyTile, calls UVoxelMarchingCubesMesher::GenerateMesh, builds RealtimeMesh streams with BuildTileStreamSet, then publishes an epoch-tagged FChunkResult for game-thread upload/collision handling. AVoxelWorld::CarveAtPosition, CarveBox, and CarveCapsule route soft FVoxelModification brushes through UVoxelDiffLayer::ApplyModification and remesh the affected chunks. The table below records the intended replacement; no row is wired into AVoxelWorld yet:

Current call or state	Core API replacement	Keep in the plugin
Shape interval/detail-band build and any active shape-world dispatch	World::compile with explicit bounds/detail options	Asset resolution, build admission and cancellation
GetDensityAt, lattice sign tests, and point/segment checks in meshing/gameplay	densityAt, densityAtLattice, isSolidAt, isSolidAlongSegment	Gameplay policy and physics objects
ClassifyTile / tile proof	classifyBox	Clipmap residency and tile lifetime
GenerateMesh plus shape-world far-parent repair and shell stitching	generateTile	Convert owned result to FVoxelMeshData, call BuildTileStreamSet, and upload it
VoxelClipmapDesiredTiles::Build tile-set calculation	desiredTiles and prioritizeTiles	Decide when to request, cancel, retain or evict jobs
UVoxelDiffLayer::ApplyModification and affected-chunk scan	applyEditTransaction with densityOffset brushes carrying signed strength and falloff; regenerate returned LOD keys	Brush budgets, authority sequence, replication, and result acceptance

The next round should add bUseVoxelForgeCore=false to AVoxelWorld and leave the existing path unchanged when false. Pass the Core World context explicitly. Plugin epoch/cancellation checks must reject stale results; they cannot interrupt an executing Core call. Drain in-flight Core calls before applying a transaction, then submit affected tiles under a new epoch. Do not move worker admission, collision cooking, actor or material creation, or rendering into the library.

The schema uses Core's 25 cm/voxel scale while the legacy plugin Runtime still defines VOXEL_SIZE=12.5 cm; define and verify a scale mapping before converting Core centimeter positions to plugin mesh payloads. The staged DA_Settings says ClipRadius=6 and MaxClipLevel=6, whereas the standalone owner JSON says radius 6/max level 8. The adapter test keeps the exported JSON as authored and applies clip_radius=3/max_clip_level=8 only in a reference-validation copy.

Acceptance record

The 557f6aa commit is the strict output baseline. The rebuilt namespaced Core matches all 11 full-detail, whole-view and LOD0–8 hash rows at W1 and W8, with W1/W8 parity. The step-29 CPU reference also preserves its approved W16 hash set. The fresh W1/W8 records are Core/verification/last_standard_hash_w1.stdout.log, Core/verification/last_standard_hash_w8.stdout.log, and Core/verification/final_rebuilt_hash_parity.log.

The look and shape audits pass, and walkability remains 5,642/5,642. The dedicated passage-camera detail audit passes at 13,271 passage samples with a detail ratio of 1.152. Far survival passes with 7,200 surfaces, zero missing surfaces or holes, and 10,976 far connectors. The damaged-shell control removes 512 triangles, reports 16 missing boundary surfaces and 16 holes, and exits 14. The final rebuilt logs are Core/verification/final_rebuilt_shape_audit.stdout.log, Core/verification/final_rebuilt_passage_camera_audit.stdout.log, Core/verification/final_rebuilt_far_survival.stdout.log, and Core/verification/final_rebuilt_bad_shell_control.stdout.log.

The six fixed 1280×800 views were recaptured from the final rebuilt viewer and compare byte-for-byte with the prior Core captures. Their images are Core/verification/core_final_rebuild_*.png; the SHA-256 results are in Core/verification/final_rebuilt_capture_parity.log. The single-tile --shape-world --benchmark-level=0 path reports generator=VoxelForgeCore.

The link-safety check demonstrates the failure before the change and the pass after it. Running it on the 557f6aa library finds 16,255 unqualified Unreal-style symbols; the rebuilt Core library passes after checking 177,569 dumpbin /SYMBOLS lines against 370 copied Unreal-style type names. Core/verification/link_safety_557f6aa_final.log and Core/verification/link_safety_final.log retain the outputs. A smoke translation unit that includes the installed Unreal 5.7 CoreMinimal.h and the public VoxelForgeCore.h also compiles; see Core/verification/unreal_header_smoke_final.log.

Speed gate: not met. The no-hash, cold W8 --stream-pipeline --shape-world --force-generate check was run in four interleaved baseline/Core pairs, each generating 2,226 tiles and the full 110.90 MiB mesh payload with no cache hits. whole_view_ready averaged 0.89175 s for the exact 557f6aa executable and 0.92050 s for the namespaced build, a 28.8 ms (3.2%) regression. Stream wall time was effectively unchanged at 0.46000 s and 0.46025 s; the difference was in shape compilation, including intervals+signs averaging 95.7 ms on baseline and 114.9 ms on Core. The raw cold runs are in Core/verification/final_rebuilt_pipeline_*.stdout.log. API-only /Ob3 and /Ob1 trials also retained hash parity but were slower, so the build is restored to /O2 /MP8. The required “no slower” acceptance condition remains outstanding.

Unreal integration round 1 record

BuildHost linked VoxelForgeCore and VoxelForge, compiled all 18 manifest shims, and passed the real shim-object symbol audit (83,843 dumpbin lines / 399 copied Unreal-style type names). Five staged owner asset copies matched their source SHA-256 values. The headless editor commandlet exported schema-v1 JSON from those assets, parsed it with Core, and matched the four expected hashes and 5,642/5,642 walkability at Core compile worker limits 1, 8, and 16. The initial tile generation in this adapter check is serial after compile; the worker limits exercise Core compilation. Full details and the run ledger are in UNREAL_INTEGRATION_REPORT.md.

This round did not add the AVoxelWorld switch, runtime task-graph scheduler, generateTile result conversion, collision upload, epoch flow, or edit-transaction carving. The plugin continues to use its existing generation path. Owner hand-off must wait until that runtime path is implemented and headlessly validated.