wip(unreal): VoxelForgeCore UBT module + asset adapter reproduce the standalone world
Luna, integration round 1 (BuildHost only; owner's project and assets untouched): - New runtime module VoxelForgeCore: one shim per Standalone/Core/core_sources.txt row (sources stay canonical under Standalone/), NoPCH/no unity, manifest check; VoxelForge links it (no runtime routing yet). UE link-safety audit: 18 shims, 83,843 symbol lines vs 399 Unreal-style names, 0 clashes. - Adapter (editor commandlet path) turns staged copies of the owner's DA_* assets into schema-v1 JSON; with clip_radius=3/max_clip_level=8 it reproduces all four Core hashes (0xdb2991c73215af4c, 0x8236c5edd7fc8e4f, 0x4c48fa75a543d58e, 0xbea22e76b34a4dfc) and walkability 5,642/5,642 at worker limits 1, 8 and 16. - Noted: the owner's settings asset says max clip level 6 (standalone used 8); Core uses 25 cm voxels vs the plugin runtime's 12.5 cm. Not done yet: AVoxelWorld switch and Core runtime route (round 2). Docs: CODEMAP, ARCHITECTURE, SHARED_CPP_LIBRARY_PLAN, UNREAL_INTEGRATION_REPORT. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MmSoF64BbRFmk9C3p1VsBJ
This commit is contained in:
Fr0zkaandClaude Opus 5.5 committed
1 parent
a1dfc0422e
commit
29ca99af7f
32 files changed
+919
-69
No files matched your search
@@ -1,8 +1,9 @@
|
||||
# VoxelForgeCore shared-library plan
|
||||
|
||||
This plan describes the extracted shape-world pipeline and the next Unreal integration. The
|
||||
standalone implementation is under `Standalone/Core`; no file in the plugin's `Source` tree was
|
||||
changed.
|
||||
This plan began with the `557f6aa` Core extraction and is updated with Unreal integration round 1.
|
||||
Canonical implementation files remain under `Standalone/`; the plugin now has a `VoxelForgeCore`
|
||||
runtime module and a headless schema-v1 asset adapter check. The `AVoxelWorld` runtime route is
|
||||
still pending.
|
||||
|
||||
## Current layout and build
|
||||
|
||||
@@ -86,11 +87,11 @@ with different data and out-of-order transactions, and returns affected desired
|
||||
LOD. The host schedules `generateTile` for those keys and drops results made stale by its own
|
||||
cancellation/version policy. Every client must apply the same sequence and brush order.
|
||||
|
||||
## Unreal integration: UBT module
|
||||
## Unreal integration: UBT module and adapter status
|
||||
|
||||
Add a `VoxelForgeCore` runtime module to the plugin and make the existing `VoxelForge` module depend
|
||||
on it. Its `Build.cs` sets C++20, exports only `Standalone/Core/include`, and keeps the Standalone
|
||||
root plus these private include roots available to the core implementation:
|
||||
The plugin now declares `VoxelForgeCore` as a runtime module and links it from both `VoxelForge` and
|
||||
`VoxelForgeEditor`. Its `Build.cs` sets C++20, exports only `Standalone/Core/include`, and keeps the
|
||||
Standalone root plus these private include roots available to the core implementation:
|
||||
|
||||
- `Standalone`
|
||||
- `Standalone/Core/src`
|
||||
@@ -99,60 +100,43 @@ root plus these private include roots available to the core implementation:
|
||||
- `Standalone/Source/VoxelForge/Public`
|
||||
- `Standalone/Source/VoxelForge/Private`
|
||||
|
||||
Set `PCHUsage = PCHUsageMode.NoPCHs` and `bUseUnity = false` for the Core module. For example:
|
||||
The module uses `PCHUsage = PCHUsageMode.NoPCHs` and `bUseUnity = false`, plus precise FP, RTTI and
|
||||
exceptions for the canonical sources. It also isolates an actual engine `CoreMinimal.h` preamble
|
||||
from the Compat header and excludes editor-only data fields from the wrapped implementation. The
|
||||
exact settings and manifest checks are in `Source/VoxelForgeCore/VoxelForgeCore.Build.cs`.
|
||||
|
||||
```csharp
|
||||
string PluginRoot = Path.GetFullPath(Path.Combine(ModuleDirectory, "../.."));
|
||||
string Standalone = Path.Combine(PluginRoot, "Standalone");
|
||||
PublicIncludePaths.Add(Path.Combine(Standalone, "Core", "include"));
|
||||
PrivateIncludePaths.AddRange(new[] {
|
||||
Standalone,
|
||||
Path.Combine(Standalone, "Core", "src"),
|
||||
Path.Combine(Standalone, "Compat"),
|
||||
Path.Combine(Standalone, "Source"),
|
||||
Path.Combine(Standalone, "Source", "VoxelForge", "Public"),
|
||||
Path.Combine(Standalone, "Source", "VoxelForge", "Private")
|
||||
});
|
||||
CppStandard = CppStandardVersion.Cpp20;
|
||||
PCHUsage = PCHUsageMode.NoPCHs;
|
||||
bUseUnity = false;
|
||||
```
|
||||
|
||||
UBT discovers translation units under a module's `Source` tree. Add one thin `.cpp` include shim per
|
||||
row of `Core/core_sources.txt`, with each shim including exactly one canonical source file by a
|
||||
UBT discovers translation units under a module's `Source` tree. There is one thin `.cpp` include
|
||||
shim per row of `Core/core_sources.txt`, and each includes exactly one canonical source file by a
|
||||
path rooted at the private `Standalone` include path. For every implementation row except
|
||||
`Core/src/VoxelForgeCore.cpp`, the shim includes `CoreNamespacePreamble.h`, opens
|
||||
`namespace vfcore::detail`, then includes that one source. The special API row gets a shim that
|
||||
includes `Core/src/VoxelForgeCore.cpp` directly; that source establishes its own namespace
|
||||
boundary. Keeping one included source per shim preserves the translation-unit boundaries used by
|
||||
`Build.ps1`. A manifest check in the UBT module should fail when a row lacks a shim, a shim names a
|
||||
different file, or an extra shim exists. Do not copy implementation files into the module. The
|
||||
standalone and UBT builds then compile the same canonical source bytes.
|
||||
`Build.ps1`. UBT validates every manifest row, checks the exact source name in each shim, and rejects
|
||||
missing or extra shims. Do not copy implementation files into the module; standalone and UBT
|
||||
compile the same canonical source bytes.
|
||||
|
||||
The existing `VoxelForge` module owns the adapter and calls the new module. Its adapter runs on the
|
||||
game thread, resolves soft references and asset defaults, preserves array order, and serializes
|
||||
settings, biome and terrain-op records, strata layouts, passages, seed, voxel size, clip limits and
|
||||
viewer start into the exact schema-v1 document accepted by `FromJson`. It must not serialize
|
||||
UObject pointers or depend on unordered map iteration. Snapshot creation returns a descriptive
|
||||
error for unresolved or unsupported assets. Once created, workers only see the immutable core
|
||||
snapshot and standard C++ results.
|
||||
The implemented adapter currently lives in the editor-only `VoxelForgeExploreCommandlet`, not in
|
||||
`AVoxelWorld`. It runs on the game thread, resolves owner asset data, and serializes settings,
|
||||
strata, passages, seed, voxel scale, clip limits and viewer start into schema-v1 JSON accepted by
|
||||
`FromJson`. It serializes values rather than UObject pointers and sorts map/set members and object
|
||||
keys for determinism. The original asset-derived JSON parses as-is. This check does not yet create
|
||||
the runtime world snapshot owned by `AVoxelWorld`.
|
||||
|
||||
The plugin supplies a `JobScheduler` backed by its chosen task system. The adapter submits shape
|
||||
build work and tile jobs to that scheduler, observes generation IDs for cancellation, and discards
|
||||
stale results. It copies completed `TileMesh` data into `FVoxelMeshData` or the plugin's current
|
||||
mesh payload on the host side, preserving centimetre positions, normals, UVs, colors, indices and
|
||||
ground/ceiling/skirt group counts. UObject creation, collision cooking, render-resource changes,
|
||||
game-thread application, residency and upload budgets remain in the plugin.
|
||||
The commandlet validation scheduler runs Core compile work at limits 1, 8, and 16; it is not the
|
||||
plugin's UE task-system adapter. A runtime implementation still needs to schedule compile/tile work,
|
||||
observe epochs/cancellation, and copy Core's owned mesh arrays/groups into `FVoxelMeshData` while
|
||||
retaining the plugin's upload and collision path.
|
||||
|
||||
## What the plugin pipeline replaces
|
||||
## Planned runtime route: not wired yet
|
||||
|
||||
The current plugin's `AVoxelWorld::GenerateTileResult` worker path performs proofs and
|
||||
`UVoxelGenerator::ClassifyTile`, calls `UVoxelMarchingCubesMesher::GenerateMesh`, builds
|
||||
The current plugin's `AVoxelWorld::GenerateTileResult` worker path remains active. It performs
|
||||
proofs and `UVoxelGenerator::ClassifyTile`, calls `UVoxelMarchingCubesMesher::GenerateMesh`, builds
|
||||
`RealtimeMesh` streams with `BuildTileStreamSet`, then publishes an epoch-tagged `FChunkResult` for
|
||||
game-thread upload/collision handling. `AVoxelWorld::CarveAtPosition`, `CarveBox`, and
|
||||
`CarveCapsule` route soft `FVoxelModification` brushes through `UVoxelDiffLayer::ApplyModification`
|
||||
and remesh the affected chunks. The adapter replaces those CPU-field and edit operations as
|
||||
follows:
|
||||
and remesh the affected chunks. The table below records the intended replacement; no row is wired
|
||||
into `AVoxelWorld` yet:
|
||||
|
||||
| Current call or state | Core API replacement | Keep in the plugin |
|
||||
| --- | --- | --- |
|
||||
@@ -163,17 +147,23 @@ follows:
|
||||
| `VoxelClipmapDesiredTiles::Build` tile-set calculation | `desiredTiles` and `prioritizeTiles` | Decide when to request, cancel, retain or evict jobs |
|
||||
| `UVoxelDiffLayer::ApplyModification` and affected-chunk scan | `applyEditTransaction` with `densityOffset` brushes carrying signed strength and falloff; regenerate returned LOD keys | Brush budgets, authority sequence, replication, and result acceptance |
|
||||
|
||||
Delete plugin-side active-world dispatch when routing these calls; the `World` context is an
|
||||
explicit object passed to each operation. The plugin's epoch/cancellation checks still reject stale
|
||||
results, but cannot interrupt an executing core call. Because edit application replaces field edit
|
||||
state, the host must drain in-flight core calls before applying a transaction and then submit the
|
||||
affected tiles under a new epoch. Do not move worker admission, collision cooking, actor or material
|
||||
creation, or rendering into the library.
|
||||
The next round should add `bUseVoxelForgeCore=false` to `AVoxelWorld` and leave the existing path
|
||||
unchanged when false. Pass the Core `World` context explicitly. Plugin epoch/cancellation checks
|
||||
must reject stale results; they cannot interrupt an executing Core call. Drain in-flight Core calls
|
||||
before applying a transaction, then submit affected tiles under a new epoch. Do not move worker
|
||||
admission, collision cooking, actor or material creation, or rendering into the library.
|
||||
|
||||
The schema uses Core's 25 cm/voxel scale while the legacy plugin Runtime still defines
|
||||
`VOXEL_SIZE=12.5 cm`; define and verify a scale mapping before converting Core centimeter positions
|
||||
to plugin mesh payloads. The staged `DA_Settings` says `ClipRadius=6` and `MaxClipLevel=6`, whereas
|
||||
the standalone owner JSON says radius 6/max level 8. The adapter test keeps the exported JSON as
|
||||
authored and applies `clip_radius=3`/`max_clip_level=8` only in a reference-validation copy.
|
||||
|
||||
## Acceptance record
|
||||
|
||||
The `557f6aa` commit is the strict output baseline. The rebuilt namespaced Core matches all 11
|
||||
full-detail, whole-view and LOD0–8 hash rows at W1 and W8, with W1/W8 parity. The fresh records are
|
||||
full-detail, whole-view and LOD0–8 hash rows at W1 and W8, with W1/W8 parity. The step-29 CPU
|
||||
reference also preserves its approved W16 hash set. The fresh W1/W8 records are
|
||||
`Core/verification/last_standard_hash_w1.stdout.log`,
|
||||
`Core/verification/last_standard_hash_w8.stdout.log`, and
|
||||
`Core/verification/final_rebuilt_hash_parity.log`.
|
||||
@@ -209,3 +199,18 @@ compilation, including `intervals+signs` averaging 95.7 ms on baseline and 114.9
|
||||
raw cold runs are in `Core/verification/final_rebuilt_pipeline_*.stdout.log`. API-only `/Ob3` and
|
||||
`/Ob1` trials also retained hash parity but were slower, so the build is restored to `/O2 /MP8`.
|
||||
The required “no slower” acceptance condition remains outstanding.
|
||||
|
||||
## Unreal integration round 1 record
|
||||
|
||||
BuildHost linked `VoxelForgeCore` and `VoxelForge`, compiled all 18 manifest shims, and passed the
|
||||
real shim-object symbol audit (83,843 `dumpbin` lines / 399 copied Unreal-style type names). Five
|
||||
staged owner asset copies matched their source SHA-256 values. The headless editor commandlet
|
||||
exported schema-v1 JSON from those assets, parsed it with Core, and matched the four expected hashes
|
||||
and 5,642/5,642 walkability at Core compile worker limits 1, 8, and 16. The initial tile generation
|
||||
in this adapter check is serial after compile; the worker limits exercise Core compilation. Full
|
||||
details and the run ledger are in `UNREAL_INTEGRATION_REPORT.md`.
|
||||
|
||||
This round did not add the `AVoxelWorld` switch, runtime task-graph scheduler, `generateTile` result
|
||||
conversion, collision upload, epoch flow, or edit-transaction carving. The plugin continues to use
|
||||
its existing generation path. Owner hand-off must wait until that runtime path is implemented and
|
||||
headlessly validated.
|
||||
Reference in new issue
Block a user