Files
VoxelForge/Standalone/Research/WorldAssemblerM3/Unreal1/REPORT.md
T
Fr0zkaandClaude Opus 5.5 fc9227f9e0 fix(unreal): U1 -- Unreal compiles again on L61; CoreGeometry asset section; versioned layout-unit upgrade
Windows MSVC catch-up after abyss L15-L61 (no Unreal there): 64-bit clipmap
shift (C4334), qualified VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel, UHT
metadata and duplicate-signature fixes. Optional CoreGeometry section on
UVoxelStrateDefinition (native caves, lattice, profiled heightfields, named
stamps, rooms/tunnels; off/empty by default), resolved after passage planning.
VoxelForgeAssetVersion 2 converts strate height/transition/gap to metres on
load (exact, 0 m change on the owner's 5 assets; packages never saved).
Gates pass (SurfaceWorld, FloatingIslands, Core-off, M2, W1/W8/W16).
KNOWN REGRESSION vs L14 (2026-10-20, same command, same PC): full set 3.9 ->
5.37 s, worker-seconds 15.8 -> 30, LOD0 1.11 -> 1.69 s, ring 0.50 -> 0.80 s;
not yet attributed (Core L15-L61 vs Unreal side).
Report: Standalone/Research/WorldAssemblerM3/Unreal1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 12:12:08 +01:00

16 KiB
Raw Blame History

VoxelForge U1 — Unreal catch-up report

Date: 2026-10-26. All builds, Unreal launches, logs, and asset reads used the isolated Saved/BuildHost host project. The owner VoxelM.uproject and editor were not opened or built. The RealtimeMeshComponent tree was read-only and unchanged. No .uasset was edited or saved.

Research reports and code/API decisions

  • Research14's first-build checklist was reviewed. BuildHost compiled Runtime, Editor, and Core; GetStrateForChunk continues to receive the layout-chunk Z coordinate. The signed hash and bounded shift changes are included in the clean MSVC build.
  • Research17 asks for the optional CoreGeometry recipe, named stamps and cavern records, landmark binding, and resolved-layout identity. These are reflected and mapped. The report refers to stamp resolver V2, while the checked-out API includes V1/V2/V3 and requires V3 for the new monolith/platform motif vocabulary. Unreal calls V1 for original stamps and V3 for motifs, then combines and validates them. Source was followed.
  • Research8 describes the full spatial-property inventory and notes a 4 m legacy chunk versus the separate layout chunk. The current code resolves the layout chunk at 8 m; the migration follows that saved setting. U1 migrates only strate height, transition width, and inter-strate gap. The remaining inventory is listed under “Unit upgrade limits”; item 3 is therefore partial.
  • Research27 water was read and deliberately not implemented this round.

Part 1 — compile and BuildHost gates

Command:

.\Tools\BuildVoxelForgeCoreHost.ps1 -MaxParallelActions 12 -TimeoutSeconds 900

Final result: BuildHost UBT attempt 1 passed; UBT reports Result: Succeeded. Runtime, Editor, and Core inside Unreal compiled with MSVC. The Core link audit passed: 18 manifest shims linked, 118,399 dumpbin lines checked against 521 Unreal-style copied type names. Five existing asset copy hashes matched the recorded BuildHost hashes. Log: Saved/BuildHost/UBT-CoreIntegration.log; link audit: Saved/BuildHost/VoxelForgeCore_ue_link_safety.log.

Compile errors fixed:

Error Cause and fix
VoxelClipmapDesiredTiles.h(82) C4334 The shift is intentionally 64-bit at the supported coordinate bound. The left operand is now int64{1} before shifting by the validated level, avoiding the old 32-bit shift followed by widening.
VoxelForgeCoreAdapter.cpp(114) C2065 Core moved the subvoxel constant into its namespace. The adapter now references VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel.
UHT: Units metadata on TArray<FVector> Unreal permits that metadata on numeric and struct properties, not this array. Removed the invalid array metadata; each authored point is still interpreted and converted as metres by the adapter.
Adapter C2143 with cascading undeclared identifiers An incomplete duplicate AttachValidatedWorldAssembly signature remained above the real definition after editing. Removed the duplicate line; the valid definition compiled.

The initial memory preflight used Get-CimInstance, which was denied, then Get-Counter, which could not find the requested counter. Neither launched Unreal. The scripts now use GlobalMemoryStatusEx; measured free memory was above 4 GiB before launches. The Floating Islands launch's separate process-list expression emitted a non-terminating PowerShell binding warning; a second process check confirmed exactly one normal-priority Unreal process, and the gate completed.

Gates

Gate Exact command Result
SurfaceWorld, 25 cm .\Tools\TestVoxelForgeCoreWorldHost.ps1 -TimeoutSeconds 900 -CoreVoxelSizeCm 25 PASS: native SurfaceWorld, collision geometry, anchor movement, and carving; peak private memory 3,216,474,112 bytes. W1/W8/W16 stable: density 0x25d62110da7f3651, mesh 0xd7184d5ad92b107f, payload 0xf7dde7d6ec40fd34, topology 0x60bbc7c8ef8bb309.
Floating Islands, 25 cm .\Tools\TestVoxelForgeCoreWorldHost.ps1 -TimeoutSeconds 900 -CoreVoxelSizeCm 25 -FloatingIslandsOnly PASS: standalone water-fixed audit, LOD0/1, W1/W8/W16; peak private memory 2,591,363,072 bytes. Coverage checked 573 local samples and 3,438 faces; 432 LOD transition faces. Standalone density/mesh/payload/topology hashes: 0xc2ba8f32b81dacec / 0x91fe8a21be55324d / 0x7a487c46247ce77c / 0x1eaee14ac02b0a80.
Core-off legacy .\Tools\TestVoxelForgeCoreWorldHost.ps1 -TimeoutSeconds 900 -CoreVoxelSizeCm 25 -CoreOffTest PASS: legacy voxel size 12.5 cm; 3 density samples; peak private memory 1,536,880,640 bytes.
M2 analytic fixtures .\Tools\TestVoxelForgeCoreWorldHost.ps1 -TimeoutSeconds 900 -CoreVoxelSizeCm 25 -CoreMesherRecipe WorldAssemblerHybridV1 -HybridFixturesOnly PASS: 5 analytic fixtures and 1 border check; peak private memory 1,988,993,024 bytes. This is the focused M2 fixture gate, not all 625 transitions.

Cold spawn and high-speed traces

Each trace used a fresh BuildHost Unreal process, the current owner plan, Core at 25 cm, and normal priority. Trace files and full command logs are under Saved/BuildHost/HostProject/Plugins/VoxelForge/Saved/CoreWorldValidation/Logs/.

Cold owner-plan command:

.\Tools\TestVoxelForgeCoreWorldHost.ps1 -TimeoutSeconds 900 -CoreVoxelSizeCm 25 -StreamingTraceOnly

The 2,578-tile initial set was fully ready in 5.779 s. The first LOD0 ring contained 512 tiles; its ready event arrived in 1.794 s with 1.017 s reported for that epoch. Summing worker_task_s from tile records through initial_set_ready gives 34.777 worker-seconds. The runtime summary reports spawn tick p95/max 5.136/21.230 ms and walk tick p95/max 4.467/42.794 ms at 6 m/s. Per-LOD0 tile worker generation was p95/max 32.835/52.929 ms. Peak private memory was 2,906,755,072 bytes.

High-speed command:

.\Tools\TestVoxelForgeCoreWorldHost.ps1 -TimeoutSeconds 900 -CoreVoxelSizeCm 25 -StreamingTraceOnly -TestMoveSpeedCmPerSecond 4000 -EnforceCollisionGate

PASS, peak private memory 3,172,507,648 bytes. The initial set was ready in 7.858 s (2,578 loaded at the initial milestone); the first ring had 512 tiles, arrived at 2.070 s, and reported 0.913 s. Initial tile-worker sum was 36.056 worker-seconds. End-of-run desired/loaded counts were 2,582/4,816. Spawn tick p95/max was 3.354/12.768 ms; walk tick p95/max was 39.577/50.165 ms at 40 m/s. The trace records 58 obsolete tile aborts during the high-speed walk.

The final timing processes used these BuildHost plugin binaries (SHA-256):

Module Binary SHA-256
Runtime UnrealEditor-VoxelForge.dll D1D889293F4AB94FD7B2B6DB376BC3395A070AE929B46387A23022F84F351CCE
Core UnrealEditor-VoxelForgeCore.dll B8801421D1757B00FF8106789DC3C368162E585DD0CD70A8F20959FF362D81B4
Editor UnrealEditor-VoxelForgeEditor.dll 1A5692D336A9D2A66D7CE699CC7F91BA77B35962BAA7FC2EC36E3F1F443F2DE0

Against the 2026-10-20 L14 reference (full set about 3.9 s, ring 0.50 s, LOD0 1.11 s, 15.8 worker-seconds, tick p95 about 2.7 ms), this cold trace is slower: full-set time +48%, ring epoch time about +103%, first-ring wall time +62% against the 1.11 s LOD0 reference, initial worker sum about +120%, and 6 m/s spawn/walk p95 are +90%/+66%. The measured first-ring wall event is 1.794 s. The 40 m/s trace also has a high walk p95, with rapid tile churn and 58 obsolete results visible in the trace. Quality gates pass, but this performance regression is not explained by a controlled A/B; the L15–L61 Core source changed since L14, so no single cause is claimed.

Part 2 — CoreGeometry wiring

UVoxelStrateDefinition now has an optional CoreGeometry section. None and empty arrays are the defaults. Recipe options are native caves, native lattice, and profiled heightfield (periodic hills, mirrored relief, or flat profile). Assets can author stable-named SDF stamps, rooms, and tunnel records in metres. The adapter plans passages first, resolves stamps and graph records next, validates, and then finalizes the layout. A stable landmark name binds to the resolved record. Core geometry anchors are read from the finalized JSON; UObject and RMC work stays on the game thread.

Core capability access from Unreal

Core capability from L15–L61 Selectable from an Unreal asset? Wired or remaining
Native caves Yes CoreGeometry.GeneratorRecipe=NativeCaves plus stable-named room/tunnel records; requires TunnelNetwork or Underwater.
Profiled heightfields Yes SurfaceWorld only; periodic hills, mirrored relief, or flat profile with metre fields.
Native lattice Yes NativeLattice plus room/tunnel/polyline records; requires Maze or VerticalShafts. Graph geometry is authored, not route-planned by the asset layer.
Cavern graph Yes Stable room and tunnel records reach ResolveResolvedCavernGraphV1.
SDF stamps Yes Stable-named carve ellipsoid/arch and fill pillar/dais stamps use V1 resolver.
Motifs Yes, for monolith/platform FillMonolith and FillPlatform use stamp vocabulary V3; Research17's V2 name differs from current source.
Core slab generator No Legacy FlatPlain and CrystalChamber remain available, but CoreGeometry has no Core slab recipe option or adapter mapping.
Ore veins No Core exposes an ore-vein request/resolver API, but CoreGeometry has no reflected request field or adapter mapping.

No water feature was added.

No-CoreGeometry and owner asset hashes

All five BuildHost copies retain their recorded hashes before and after the audit; the migration commandlet only loaded and inspected them. The old assets have no authored CoreGeometry, so the new recipe remains Off and all arrays remain default-empty. Current owner-plan W1/W8/W16 output hashes are recorded above. The current owner-plan assembly hash is 2d9e50e902e140e8. The L14 pre-round Core output hash is not byte-comparable: L15–L61 changed Core in the meantime. Thus the asset hashes and the current no-geometry hash matrix show the new fields are inactive on the owner's assets; they do not claim byte identity with L14.

Part 3 — unit upgrade status and asset audit

VoxelForgeAssetVersion::GUID is A76D5E31-4B2C-4F9A-B8C1-D2046E9F7738; current version is 2 (version 1 retains the tunnel-floor slope conversion). Loading assets with version below 2 converts old integer layout counts once. The version is carried through Unreal's custom-version archive. The five unmodified BuildHost copies loaded successfully in the clean audit, commandlet peak private memory 1,608,806,400 bytes. Saved/BuildHost/AssetMigrationAudit.final.log contains the values.

BuildHost asset Old serialized value In-memory metre value Physical delta
DA_Settings gap 0 layout chunks; layout chunk 8 m gap 0 m 0 m
DA_Strate1 height 64 chunks; transition 1 chunk height 512 m; transition 8 m 0 m
DA_Strate2 height 30 chunks; transition 3 chunks height 240 m; transition 24 m 0 m
DA_Strate3 height 16 chunks; transition 2 chunks height 128 m; transition 16 m 0 m
DA_Strate4 height 16 chunks; transition 2 chunks height 128 m; transition 16 m 0 m

Every conversion is exact at the saved 8 m layout width. The old and new numeric representations differ because the units differ; the resulting physical dimensions change by 0 m. The settings PostLoad also sees DA_Strate4 through its fixed-strata reference; the direct audit then loads all four definitions. No packages were saved.

BuildHost asset SHA-256 after audit (same values enforced by the BuildHost script before the audit):

Asset SHA-256
DA_Settings.uasset 8ABE8243104980260FECA0A6F4CF4927768E8E568221A6F6A2C2694F6D15D454
DA_Strate1.uasset 76063D691D5CE26D2014B587E1972A6724787BE1204E4652CA863BB8D5001BD0
DA_Strate2.uasset 02F56A18CED991BAF93DFB2F69438152275E8353B5E279C1CB7DD6E079E352E2
DA_Strate3.uasset F47437F8D156C608AA06E40C5B76716ED4A19FA0F057688E7FECC04DC976428C
DA_Strate4.uasset 7C0DE7D06876035AC7BC4F117B9166AFD77402021843BA4900B470F4B0D0185B

Unit upgrade limits

The Research8 inventory is broader than the U1 migration. Automatic conversion is not yet wired for legacy streaming/render/content chunk distances; decoration, brush, generator, biome, passage, shaft, island, disturbance, terrain-op and landmark owner-voxel lengths/frequencies; SurfaceWorld/Core-voxel lengths/frequencies; placement vectors, centimetre offsets, and other reflected spatial fields. The new CoreGeometry fields themselves are metre-valued. The legacy fields continue to work in their current units. Completing this inventory needs additional versioned fields/converters and consumer updates; U1 does not claim that those settings now author metres.

Async safety review

The diff leaves the existing safeguards intact:

  • Generation Epoch travels with worker results and stale epochs are discarded before apply.
  • All four result queues remain TQueue<..., EQueueMode::Mpsc>.
  • EndPlay signals shutdown, waits for active tasks, then drains each queue.
  • Workers check bShuttingDown before work and before publishing results.
  • UVoxelDiffLayer::ModsLock still guards all concurrent modification-map access.
  • Core layout resolution and content-anchor setup read UObjects on initialization; landmark updates and RMC mutation are game-thread work. No new worker-side UObject or RMC mutation was introduced.

Failed and retried runs

  • Earlier SurfaceWorld harness attempts exposed missing owner-to-Core length scale in imported JSON, using world origin instead of viewer_start, and a collision reference centered on the wrong anchor. The harness now supplies owner/Core scale and uses the manifest viewer and collision center; the final SurfaceWorld gate passes.
  • The first Floating Islands fixture anchor audit exceeded its 0.25 density residual because the runtime uses finite-step column refinement. The diagnostic-only bound is now 0.5 Core voxel and reports residual, Core-voxel and local-centimetre coordinates. Runtime generation was not changed; the final island audit passes.
  • An additional enforced-collision streaming run reached 900 seconds waiting on support tile (-5,0,-31) with 2,917/2,918 desired and 2,914/2,917 loaded. It was stopped at the time box and not repeated. The normal SurfaceWorld gate and the requested 40 m/s trace passed.
  • Standalone fast verification was attempted with py Standalone/Tools/verify/verify.py fast --out Saved/BuildHost/StandaloneVerifyFast --overall-timeout 900. It exited before the tier started because Windows Python has no signal.SIGALRM; no standalone tests ran, and no verify output was written outside BuildHost.
  • First asset-audit launch exited 3 because the DDC graph had no writable node. A retry loaded and reported all five assets, but Unreal returned exit 1 due the default user-profile DDC diagnostic. Those two logs show default-cache write probes failing with WriteError: 3 (path not found); no write succeeded. The clean third launch isolated LOCALAPPDATA under BuildHost and exited 0. No new UECC-* folder appeared; the only folder is the pre-existing 2026-09-17 crash folder.
  • No run exited with 0xC0000142; no such retry was needed.

Likely compile spots for the next Core/API merge

  • VoxelForgeCoreAdapter.cpp: Core ResolvedLayoutSnapshotV3, stamp V1/V3, cavern graph resolver, profile field names, stable-ID conversion, and passage/resolver/finalize ordering.
  • VoxelStrateTypes.h: UHT support for nested CoreGeometry structs, FVector metre metadata, and polyline array metadata (the array itself cannot carry Units).
  • VoxelSettings.cpp, VoxelStrateDefinition.cpp, and VoxelForgeAssetVersion.h: custom-version registration, ordering with the existing version-1 tunnel-floor migration, and serialization.
  • VoxelForgeExploreCommandlet.cpp: BuildHost-only asset audit command and editor-module access to the runtime types.
  • VoxelClipmapDesiredTiles.h: signed/64-bit coordinate shifts and clamped LOD bounds.
  • VoxelStrateManager.cpp: keep layout chunk Z as the lookup input and convert physical bounds only once at the layout boundary.

Large compiler, Unreal, and trace outputs remain under Saved/BuildHost.