diff --git a/CODEMAP.md b/CODEMAP.md index ef1ca27..f6d1fc1 100644 --- a/CODEMAP.md +++ b/CODEMAP.md @@ -117,9 +117,10 @@ remains `0`, and lateral regions remain gated off. |------|------| | `../../VoxelForge.uplugin` | Plugin manifest. Runtime module `VoxelForge` plus the Editor-only `VoxelForgeEditor` commandlet module. Beta. | | `VoxelForge.Build.cs` | Deps: Core, CoreUObject, Engine, **GameplayTags**, **RealtimeMeshComponent**. | -| `Tools/VoxelForgeTest.ps1` | The single guarded test entry point. `-Scenario canonical|owner|probe|perf|tests|surface-fall|crossing|gate-stress|visual-edit|visual-lod|visual-gap-walk|visual-gap-inject|dig-motion|parity|horizon|horizon-render|determinism`, optional `-Build`, `-Assets owner|default`, `-Cvars`, absolute `-Out`, `-Label`, `-HorizonLevel`, `-HorizonRenderDistanceChunks`, `-ParityExportSize=32|64|128`, `-ParityExportTile=x,y,z`, and `-SharedDDC`; stages the host/assets, isolates user/DDC/config state, serializes one Unreal launch at a time, enforces the 8 GiB/30-minute guards, and emits compact `result.json`/`summary.txt` plus full logs. Parity can render its game leg with `-EnableRender` and compares the selected export region against the exact streamed tile hashes. `visual-gap-walk` records camera-frustum surface-point coverage at 6000 cm/s; `visual-gap-inject` proves an early component hide is detected; `dig-motion` logs per-frame vertical velocity, gate, support readiness, and movement mode. `-Cvars @{ 'voxel.TestGapAuditLegacyHandoff'=1 }` replays the old visibility handoff for A/B audit baselines. Horizon traces report desired/loaded tiles, resident stream geometry, worker seconds, steady wall time, and peak private memory; `horizon-render` captures the owner view; `determinism` compares per-tile hashes across the default CRT path and `-voxel.CrtFma3=0`. | +| `Tools/VoxelForgeTest.ps1` | The single guarded test entry point. `-Scenario canonical|owner|probe|perf|tests|surface-fall|crossing|gate-stress|visual-edit|visual-lod|visual-gap-walk|visual-gap-inject|dig-motion|parity|horizon|horizon-render|determinism|region-eval`, optional `-Build`, `-Assets owner|default`, `-Cvars`, absolute `-Out`, `-Label`, `-HorizonLevel`, `-HorizonRenderDistanceChunks`, `-ParityExportSize=32|64|128`, `-ParityExportTile=x,y,z`, and `-SharedDDC`; stages the host/assets, isolates user/DDC/config state, serializes one Unreal launch at a time, enforces the 8 GiB/30-minute guards, waits for 6 GiB free RAM before launches, and emits compact `result.json`/`summary.txt` plus full logs. Parity can render its game leg with `-EnableRender` and compares the selected export region against the exact streamed tile hashes. `visual-gap-walk` records camera-frustum surface-point coverage at 6000 cm/s; `visual-gap-inject` proves an early component hide is detected; `dig-motion` logs per-frame vertical velocity, gate, support readiness, and movement mode. `-Cvars @{ 'voxel.TestGapAuditLegacyHandoff'=1 }` replays the old visibility handoff for A/B audit baselines. Horizon traces report desired/loaded tiles, resident stream geometry, worker seconds, steady wall time, and peak private memory; `horizon-render` captures the owner view; `determinism` compares per-tile hashes across the default CRT path and `-voxel.CrtFma3=0`; `region-eval` runs the isolated step-2 lattice comparison and candidate-bin ablation. | `Public/VoxelForgeModule.h` / `Private/VoxelForgeModule.cpp` | `FVoxelForgeModule` boilerplate (Startup/Shutdown just log). | -| `Source/VoxelForgeEditor/VoxelForgeEditor.Build.cs` / `Private/VoxelForgeExploreCommandlet.*` | Editor/commandlet-only `-run=VoxelForgeExplore`; owns bounded render/`VF_` walk/OBJ export orchestration and is not linked by Game/Shipping. Optional `-exporttilex/y/z` sets an exact L0 tile origin for controlled export/game parity; omitting all three preserves the existing center-derived origin. | +| `Source/VoxelForgeEditor/VoxelForgeEditor.Build.cs` / `Private/VoxelForgeExploreCommandlet.*` | Editor/commandlet-only `-run=VoxelForgeExplore`; owns bounded render/`VF_` walk/OBJ export orchestration and is not linked by Game/Shipping. Optional `-exporttilex/y/z` sets an exact L0 tile origin for controlled export/game parity; omitting all three preserves the existing center-derived origin. `-regioneval=1` runs the default-off step-2 research harness against live `GetDensityAt`. | +| `Source/VoxelForgeEditor/Private/VoxelForgeRegionEvalExperiment.*` | Six-partition lattice driver for the commandlet-only step-2 experiment: cold 259³ timing, bitwise repeat hashes, fixed 16-bit quantized comparisons, and exact candidate-visit attribution. It calls the live final-field API in both modes; the prepared mode selects editor-only 8³ SDF candidate bins. | | `Public/VoxelGeometryHash.h` | Header-only geometry digest shared by commandlet export and runtime tile-hash dump; hashes mesh counts and the vertex/normal/UV/color/triangle arrays with the historical CRC order. | | `Public/VoxelStats.h` / `Private/VoxelStats.cpp` | `stat VoxelForge` DWORD counters for tile classification, skipping, meshing, operator-stack verdicts, and cave-bail diagnosis. Each `Cave Bail *` counter names one guard; `Cave Bail No Stack` counts slots with no buildable stack (no definition, or a degenerate strate). | @@ -458,6 +459,13 @@ Header is rich with inline docs. Two namespaces + a per-chunk cache system. — deterministic hashing for room/tunnel placement (no storage, infinite worlds). - Cache structs (h:520-950): `FCachedRoom`, `FCachedTunnel`, `FCachedPit`, `FCachedChimney`, `FCachedColumn`, `FChunkSDFCache`. +- `FChunkSDFBrickIndex` is an editor-only-at-use 8-voxel 3D bin beside the existing XY index. The + default-off `SetResearchBrickBinningEnabled` selector builds/selects it only for the explore + commandlet. `VF_BuildResearchBrickIndices` uses the cached room cull reach (shape, floor relief, + blend and terrain-op envelope), room-floor-join bounds, and the union of SDF/world tunnel + centerline influence and bounding-sphere bounds. Source item indices stay in ascending order; + invalid or oversized grids fall back to today's XY broad phase. This step-2 candidate-bin + ablation still routes every lattice point through the canonical `GetDensityAt` final field. - `namespace VoxelCaveMorphology`: | Function | .cpp line | Role | |----------|-----------|------| diff --git a/RESEARCH-SPEED-2026-09.md b/RESEARCH-SPEED-2026-09.md index 327d593..49ea62f 100644 --- a/RESEARCH-SPEED-2026-09.md +++ b/RESEARCH-SPEED-2026-09.md @@ -33,7 +33,11 @@ The steps below describe **future plugin experiments**. This research pass makes 1. **Day-one falsification: stage the existing collision and upload clocks.** Add timestamps for collision request, RMC end-of-frame dequeue, `GThreadPool` worker start, geometry extraction end, `CookComplexMesh` start/end, game-thread `ApplyCollisionUpdate` start/end, physics-state ready and first successful capsule sweep. Keep existing generation, mesh and gameplay behavior while profiling. Test one ~4,000-triangle tile and the pawn's current 3³=27-tile support neighborhood, idle at the current 14 generation tasks, and with the generation cap reduced to 12. If the cook's queue wait dominates, test a dedicated priority cook dispatched directly after player-support mesh extraction; compare its core cook and install times with RMC's route. In parallel, prequeue a 341-tile render corpus and run caps 4/8/16 with the same 2 ms drain guard; record visible applies/s, game/render/RHI clocks and first-visible frame. **Pass the cook shortcut only if** core cook p95 ≤10 ms, the 27-tile player support is ready ≤150 ms after meshes exist under load, and an edit-time field override can switch floor/support within the first simulated/rendered frame with zero invented velocity in 100 repeated underfoot digs. If any edit test fails, implement field-aware Mover as the player authority; if the shortcut passes all tests, use prioritized cooks for steady terrain plus field queries while an edit's cook is pending. **Pass the ungrouped upload path only if** it sustains ≥525 visible applies/s at 60 Hz with game-thread drain ≤2 ms/frame and no render/RHI backlog. These are decision thresholds, not existing measurements. **Step 1 results (2026-09-30, commit `41b001b`):** core cook **1.50 ms p50 / 2.28 ms p95** (passes); through RMC's collision route the same support bodies were ready **8.5–10.8 s** after their meshes existed (fails): the wait is RMC's path, not Chaos, so player-support collision must bypass it. CMC with a field override: **883 false-support frames in 100 underfoot digs** (fails): step 4 is **field-aware Mover**. Uploads: cap 16 = **639 applies/s**, drain p95 2.22 ms (misses 2 ms by 0.22 ms), no render/RHI backlog, so tune the drain and **do not group yet**. The "8× per-sample gap" was a denominator artefact: 263 ns was export *wall* time with 15 workers on a shared lattice (≈3.4 µs CPU per sample); the real per-sample CPU cost is ~2–3.4 µs in both paths, so the 12–16× reduction target stands. -2. **Standalone exact-float evaluator ablation.** Extract representative *real* TunnelNetwork and composer plans, including warped rooms, passages, stratum transitions, route floors and diff edits, into `Saved/Research` only. Keep the plugin's active evaluator and worlds untouched. Compare baseline point API to a prepared float region/brick API at the same lattice coordinates and op order. The reference is the chosen **new-world float definition**, not old-season geometry. **Pass:** zero 16-bit quantized-value disagreements and zero route/capsule regression across at least five seeds; ≤**600 ms** for the full 259³ cold ring on six workers before certificates. Record candidate visits, cache checks and worker CPU. Failure means local compilation did not remove enough work; investigate before rewriting numeric math. +2. **Exact-float region-evaluator experiment.** Use an isolated, editor-only explore-commandlet path inside the plugin because the live composer, strate manager, generator and SDF cache depend on Unreal types. Keep gameplay default-off and compare at the same lattice points and operator order. The reference is the chosen **new-world float definition**, not old-season geometry. **Pass:** zero 16-bit quantized-value disagreements and zero route/capsule regression across at least five seeds; ≤**600 ms** for the full 259³ cold ring on six workers before certificates. Record candidate visits, cache checks and worker CPU. Failure means local compilation did not remove enough work; investigate before rewriting numeric math. + + **Step 2 result (2026-09-30, attempt C; pass bar not met, full compiler untested):** five-seed Q16 comparison and two-run raw-float determinism pass. The deliberate room-bin reach reduction to 0.5 is caught (18,107 Q16 differences, 735 sign differences), with no saturated samples. On the 17,373,979-point cold ring, the live `GetDensityAt` p50 is **6.776 s**, **1.679 µs worker CPU/sample**, and **29.164 worker CPU seconds**; the editor-only 8³ SDF-bin variant is **19.543 s**, **2.879 µs worker CPU/sample**, and **50.023 worker CPU seconds**. Both measured variants observed six workers. Sampled visits fall from **21.19 to 16.38/sample** (about 23%), but wall time is 2.88× slower and the 600 ms target is missed. The variant still calls `GetDensityAt` for every lattice point; its bins cover rooms, room-floor joins and whole tunnel chains, while segment loops, route/cache decisions, passages, landing posts, disturbances and player edits remain in the canonical per-sample path. This is a **candidate-bin ablation, not the planned compiled region program**; it falsifies per-sample 3D bin lookup as a sufficient optimization but does not falsify hoisting region decisions or compiling the full ordered program. Step 2 remains open until that full program is measured; do not start step 3's certificate work against this slower baseline. Full report: `Saved/BuildHost/HostProject/Plugins/VoxelForge/Saved/TestHarness/step2_region_eval_20260930_c_region-eval/region_eval.json`. + + **Launch record:** attempt A's guarded UBT invocation logged `Result: Failed` (JSON writer and `ParallelFor` overload compile errors); it exited before launching a commandlet. Attempt B built successfully but `-regioneval=1` was ignored by Unreal argument parsing, so the ordinary explore commandlet exited successfully without the expected JSON. Attempt C used a bare `-regioneval`, rebuilt the editor module, and completed the intended experiment. No attempt timed out, crossed the 8 GiB private-memory limit, or produced a crash report; C peaked at 7,137,738,752 private bytes. The runner was corrected to detect UBT's logged failure even when its process exit code is zero. 3. **Add certificates, shared lattice and contouring.** Prove final-sign bounds after all writers and edits; classify 8³ bricks, evaluate uncertain ones, share global corners, and mesh them with today's marching-cubes table. Keep the old tile route as a versioned reference while the new route is validated. **Pass:** zero false-uniform bricks and zero sign mismatch at all 17,373,979 ring lattice sites for the test corpus, cold worker generation ≤**350 ms**, and max render-visible/full ring ≤**900 ms** in the later UE gate. If certification consumes more time than it saves, fall back to evaluating uncertain regions without that proof; do not repeat the archived 0.588→2.057 s interval-control mistake. 3b. **AVX2 check against the new norm (owner's request, 2026-09-29).** Once step 3's brick evaluator is the baseline, try an explicit 8-lane AVX2 version of its hot loop (the plugin's current noise path is 4-lane SSE4.1). **The owner does not require AVX2 to match the scalar path bit for bit** (clarified 2026-09-29): only the shipped product must be deterministic across players' machines. If adopted, the AVX2 path *is* the canonical world definition (FMA allowed, as long as it is always used), and every machine must run that same path: require AVX2 at startup rather than falling back to a scalar path that would produce a different world. **Keep it only if** it is faster on the step-3 corpus and the AVX2 path gives the same lattice hash on two runs and on two different AVX2 CPU models; otherwise record the number and drop it. The round-two integer AVX2 probe (4.25–5.86× over its own scalar) is the reference for what explicit vectorisation can buy. 4. **Choose and implement player collision.** Use step 1 to select prioritized Chaos+edit overlay or field-aware Mover; keep ordinary collision with nonterrain actors and keep NPC/projectile field-query backup. Version floor/contact state by edit sequence. **Pass:** at every teleport the player can perform an actual blocked capsule movement on the intended terrain ≤**900 ms** after the request, 100 underfoot digs add **0** spurious velocity impulses, and 100 excavations with no cave below have **0** frames of false support or false air. The full visible ring must also meet the separate ≤1,000 ms hard deadline. A successful cook timing without the edit tests is insufficient. diff --git a/Source/VoxelForge/Private/VoxelCaveMorphology.cpp b/Source/VoxelForge/Private/VoxelCaveMorphology.cpp index fe11ac0..59f5937 100644 --- a/Source/VoxelForge/Private/VoxelCaveMorphology.cpp +++ b/Source/VoxelForge/Private/VoxelCaveMorphology.cpp @@ -46,6 +46,11 @@ namespace { +#if WITH_EDITOR + thread_local bool GVoxelResearchBrickBinningEnabled = false; + thread_local float GVoxelResearchBrickRoomReachScale = 1.0f; +#endif + using FVFPlayerDensitySampler = TFunctionRef; // Density coordinates and cached cull radii are float-valued. FVector is double-valued in @@ -1282,6 +1287,68 @@ bool FChunkSDFSpatialIndex::GetRange( return OutBegin >= 0 && OutEnd >= OutBegin && OutEnd <= ItemIndices.Num(); } +void FChunkSDFBrickIndex::Reset() +{ + MinBrickX = MinBrickY = MinBrickZ = 0; + NumBricksX = NumBricksY = NumBricksZ = 0; + bValid = false; + CellOffsets.Reset(); + ItemIndices.Reset(); +} + +void FChunkSDFBrickIndex::Release() +{ + MinBrickX = MinBrickY = MinBrickZ = 0; + NumBricksX = NumBricksY = NumBricksZ = 0; + bValid = false; + CellOffsets.Empty(); + ItemIndices.Empty(); +} + +SIZE_T FChunkSDFBrickIndex::GetAllocatedSize() const +{ + return static_cast(CellOffsets.GetAllocatedSize()) + + static_cast(ItemIndices.GetAllocatedSize()); +} + +bool FChunkSDFBrickIndex::GetRange( + float WorldX, float WorldY, float WorldZ, + int32& OutBegin, int32& OutEnd) const +{ + OutBegin = OutEnd = 0; + constexpr double BrickSize = 8.0; + if (!bValid || NumBricksX <= 0 || NumBricksY <= 0 || NumBricksZ <= 0 + || !VoxelMath::IsFinite(WorldX) || !VoxelMath::IsFinite(WorldY) + || !VoxelMath::IsFinite(WorldZ)) + { + return false; + } + const double BX = FMath::FloorToDouble(static_cast(WorldX) / BrickSize); + const double BY = FMath::FloorToDouble(static_cast(WorldY) / BrickSize); + const double BZ = FMath::FloorToDouble(static_cast(WorldZ) / BrickSize); + if (BX < MIN_int32 || BX > MAX_int32 || BY < MIN_int32 || BY > MAX_int32 + || BZ < MIN_int32 || BZ > MAX_int32) + { + return false; + } + const int64 X = static_cast(BX) - MinBrickX; + const int64 Y = static_cast(BY) - MinBrickY; + const int64 Z = static_cast(BZ) - MinBrickZ; + if (X < 0 || X >= NumBricksX || Y < 0 || Y >= NumBricksY + || Z < 0 || Z >= NumBricksZ) + { + return false; + } + const int64 Slot = (Z * NumBricksY + Y) * NumBricksX + X; + if (Slot < 0 || Slot + 1 >= CellOffsets.Num()) + { + return false; + } + OutBegin = CellOffsets[static_cast(Slot)]; + OutEnd = CellOffsets[static_cast(Slot + 1)]; + return OutBegin >= 0 && OutEnd >= OutBegin && OutEnd <= ItemIndices.Num(); +} + namespace { struct FVFSpatialBounds @@ -1513,6 +1580,42 @@ namespace return CandidateCount; } + template + static FORCEINLINE int32 VF_ForEachSDFCandidate( + const FChunkSDFSpatialIndex& SpatialIndex, +#if WITH_EDITOR + const FChunkSDFBrickIndex& BrickIndex, +#endif + int32 ItemCount, float WorldX, float WorldY, float WorldZ, + FVisit&& Visit, bool bUseSpatialIndex = true) + { +#if WITH_EDITOR + if (VoxelCaveMorphology::IsResearchBrickBinningEnabled()) + { + int32 Begin = 0; + int32 End = 0; + if (BrickIndex.GetRange(WorldX, WorldY, WorldZ, Begin, End)) + { + int32 CandidateCount = 0; + const int32* Indices = BrickIndex.ItemIndices.GetData(); + for (int32 Cursor = Begin; Cursor < End; ++Cursor) + { + const int32 ItemIndex = Indices[Cursor]; + if (ItemIndex >= 0 && ItemIndex < ItemCount) + { + ++CandidateCount; + Visit(ItemIndex); + } + } + return CandidateCount; + } + } +#endif + return VF_ForEachSpatialCandidate( + SpatialIndex, ItemCount, WorldX, WorldY, + Forward(Visit), bUseSpatialIndex); + } + static bool VF_GetRoomSpatialBounds(int32, const FCachedRoom& Room, FVFSpatialBounds& OutBounds) { return VF_SetSpatialBounds( @@ -1559,9 +1662,13 @@ namespace } bInside = VF_CachedRoomShapeSDF(Room, Position) < 0.0f; }; - VF_ForEachSpatialCandidate( - Cache.RoomSpatialIndex, Cache.Rooms.Num(), - static_cast(Position.X), static_cast(Position.Y), + VF_ForEachSDFCandidate( + Cache.RoomSpatialIndex, +#if WITH_EDITOR + Cache.RoomBrickIndex, +#endif + Cache.Rooms.Num(), static_cast(Position.X), + static_cast(Position.Y), static_cast(Position.Z), VisitRoom, bUseSpatialIndex); return bInside; } @@ -1643,6 +1750,261 @@ namespace Column.CenterX, Column.CenterY, FMath::Sqrt(FMath::Max(Column.BoundXYRadiusSq, 0.0f)), OutBounds); } + +#if WITH_EDITOR + struct FVFBrickBounds + { + FVector Min = FVector::ZeroVector; + FVector Max = FVector::ZeroVector; + }; + + static bool VF_IncludeBrickBounds( + FVFBrickBounds& InOut, const FVector& Min, const FVector& Max, bool& bHasBounds) + { + if (!VoxelMath::IsFinite(Min.X) || !VoxelMath::IsFinite(Min.Y) + || !VoxelMath::IsFinite(Min.Z) || !VoxelMath::IsFinite(Max.X) + || !VoxelMath::IsFinite(Max.Y) || !VoxelMath::IsFinite(Max.Z) + || Min.X > Max.X || Min.Y > Max.Y || Min.Z > Max.Z) + { + return false; + } + if (!bHasBounds) + { + InOut.Min = Min; + InOut.Max = Max; + bHasBounds = true; + } + else + { + InOut.Min.X = FMath::Min(InOut.Min.X, Min.X); + InOut.Min.Y = FMath::Min(InOut.Min.Y, Min.Y); + InOut.Min.Z = FMath::Min(InOut.Min.Z, Min.Z); + InOut.Max.X = FMath::Max(InOut.Max.X, Max.X); + InOut.Max.Y = FMath::Max(InOut.Max.Y, Max.Y); + InOut.Max.Z = FMath::Max(InOut.Max.Z, Max.Z); + } + return true; + } + + static bool VF_IncludeBrickSphere( + FVFBrickBounds& InOut, const FVector& Center, float Radius, bool& bHasBounds) + { + if (!VoxelMath::IsFinite(Radius) || Radius < 0.0f) + { + return false; + } + const FVector FloatCenter( + static_cast(Center.X), static_cast(Center.Y), static_cast(Center.Z)); + const float CoordinateScale = FMath::Max3( + 1.0f, FMath::Abs(static_cast(Center.X)), + FMath::Max3(FMath::Abs(static_cast(Center.Y)), + FMath::Abs(static_cast(Center.Z)), FMath::Abs(Radius))); + // Leave a float-operation margin around the exact live cull. It covers rounding in the + // float center conversion, distance square/sum, and radius-square comparison; it is a + // numerical bound, not an authored feature-radius guess. + const float ConservativeRadius = Radius + CoordinateScale * (32.0f * FLT_EPSILON); + const FVector R(ConservativeRadius, ConservativeRadius, ConservativeRadius); + return VF_IncludeBrickBounds(InOut, FloatCenter - R, FloatCenter + R, bHasBounds); + } + + template + static void VF_BuildBrickIndex( + FChunkSDFBrickIndex& OutIndex, int32 ItemCount, FGetBounds&& GetBounds) + { + OutIndex.Reset(); + if (ItemCount <= 0) + { + OutIndex.bValid = true; + return; + } + + constexpr double BrickSize = 8.0; + constexpr int64 MaxBuckets = 1024 * 1024; + constexpr int64 MaxReferences = 4 * 1024 * 1024; + TArray ItemMin; + TArray ItemMax; + ItemMin.SetNumUninitialized(ItemCount); + ItemMax.SetNumUninitialized(ItemCount); + int32 GlobalMinX = MAX_int32, GlobalMinY = MAX_int32, GlobalMinZ = MAX_int32; + int32 GlobalMaxX = MIN_int32, GlobalMaxY = MIN_int32, GlobalMaxZ = MIN_int32; + int64 TotalReferences = 0; + for (int32 Item = 0; Item < ItemCount; ++Item) + { + FVFBrickBounds Bounds; + if (!GetBounds(Item, Bounds) + || !VoxelMath::IsFinite(Bounds.Min.X) || !VoxelMath::IsFinite(Bounds.Min.Y) + || !VoxelMath::IsFinite(Bounds.Min.Z) || !VoxelMath::IsFinite(Bounds.Max.X) + || !VoxelMath::IsFinite(Bounds.Max.Y) || !VoxelMath::IsFinite(Bounds.Max.Z)) + { + return; + } + const double MinX = FMath::FloorToDouble(Bounds.Min.X / BrickSize); + const double MinY = FMath::FloorToDouble(Bounds.Min.Y / BrickSize); + const double MinZ = FMath::FloorToDouble(Bounds.Min.Z / BrickSize); + const double MaxX = FMath::FloorToDouble(Bounds.Max.X / BrickSize); + const double MaxY = FMath::FloorToDouble(Bounds.Max.Y / BrickSize); + const double MaxZ = FMath::FloorToDouble(Bounds.Max.Z / BrickSize); + if (MinX < MIN_int32 || MinX > MAX_int32 || MinY < MIN_int32 || MinY > MAX_int32 + || MinZ < MIN_int32 || MinZ > MAX_int32 || MaxX < MIN_int32 || MaxX > MAX_int32 + || MaxY < MIN_int32 || MaxY > MAX_int32 || MaxZ < MIN_int32 || MaxZ > MAX_int32) + { + return; + } + const FIntVector Lower( + static_cast(MinX), static_cast(MinY), static_cast(MinZ)); + const FIntVector Upper( + static_cast(MaxX), static_cast(MaxY), static_cast(MaxZ)); + if (Lower.X > Upper.X || Lower.Y > Upper.Y || Lower.Z > Upper.Z) + { + return; + } + ItemMin[Item] = Lower; + ItemMax[Item] = Upper; + GlobalMinX = FMath::Min(GlobalMinX, Lower.X); + GlobalMinY = FMath::Min(GlobalMinY, Lower.Y); + GlobalMinZ = FMath::Min(GlobalMinZ, Lower.Z); + GlobalMaxX = FMath::Max(GlobalMaxX, Upper.X); + GlobalMaxY = FMath::Max(GlobalMaxY, Upper.Y); + GlobalMaxZ = FMath::Max(GlobalMaxZ, Upper.Z); + const int64 NX = static_cast(Upper.X) - Lower.X + 1; + const int64 NY = static_cast(Upper.Y) - Lower.Y + 1; + const int64 NZ = static_cast(Upper.Z) - Lower.Z + 1; + if (NX <= 0 || NY <= 0 || NZ <= 0 || NX > MaxReferences / NY + || NX * NY > MaxReferences / NZ) + { + return; + } + TotalReferences += NX * NY * NZ; + if (TotalReferences > MaxReferences) + { + return; + } + } + + const int64 NumX = static_cast(GlobalMaxX) - GlobalMinX + 1; + const int64 NumY = static_cast(GlobalMaxY) - GlobalMinY + 1; + const int64 NumZ = static_cast(GlobalMaxZ) - GlobalMinZ + 1; + if (NumX <= 0 || NumY <= 0 || NumZ <= 0 || NumX > MaxBuckets / NumY + || NumX * NumY > MaxBuckets / NumZ) + { + return; + } + const int64 BucketCount = NumX * NumY * NumZ; + OutIndex.MinBrickX = GlobalMinX; + OutIndex.MinBrickY = GlobalMinY; + OutIndex.MinBrickZ = GlobalMinZ; + OutIndex.NumBricksX = static_cast(NumX); + OutIndex.NumBricksY = static_cast(NumY); + OutIndex.NumBricksZ = static_cast(NumZ); + OutIndex.CellOffsets.SetNumZeroed(static_cast(BucketCount + 1)); + for (int32 Item = 0; Item < ItemCount; ++Item) + { + for (int32 Z = ItemMin[Item].Z; Z <= ItemMax[Item].Z; ++Z) + { + for (int32 Y = ItemMin[Item].Y; Y <= ItemMax[Item].Y; ++Y) + { + const int64 Row = (static_cast(Z - OutIndex.MinBrickZ) + * OutIndex.NumBricksY + (Y - OutIndex.MinBrickY)) + * OutIndex.NumBricksX; + for (int32 X = ItemMin[Item].X; X <= ItemMax[Item].X; ++X) + { + const int64 Slot = Row + (X - OutIndex.MinBrickX); + ++OutIndex.CellOffsets[static_cast(Slot + 1)]; + } + } + } + } + for (int32 Slot = 0; Slot < static_cast(BucketCount); ++Slot) + { + OutIndex.CellOffsets[Slot + 1] += OutIndex.CellOffsets[Slot]; + } + OutIndex.ItemIndices.SetNumUninitialized(static_cast(TotalReferences)); + TArray WriteOffsets = OutIndex.CellOffsets; + for (int32 Item = 0; Item < ItemCount; ++Item) + { + for (int32 Z = ItemMin[Item].Z; Z <= ItemMax[Item].Z; ++Z) + { + for (int32 Y = ItemMin[Item].Y; Y <= ItemMax[Item].Y; ++Y) + { + const int64 Row = (static_cast(Z - OutIndex.MinBrickZ) + * OutIndex.NumBricksY + (Y - OutIndex.MinBrickY)) + * OutIndex.NumBricksX; + for (int32 X = ItemMin[Item].X; X <= ItemMax[Item].X; ++X) + { + const int64 Slot = Row + (X - OutIndex.MinBrickX); + OutIndex.ItemIndices[WriteOffsets[static_cast(Slot)]++] = Item; + } + } + } + } + OutIndex.bValid = true; + } + + static bool VF_GetRoomBrickBoundsSafe( + int32, const FCachedRoom& Room, FVFBrickBounds& OutBounds) + { + // EvaluateSDFCached receives the already-warped room query coordinate. Bin in that same + // space, so we preserve the live query's exact warp without estimating an inverse-warp + // displacement. CullRadiusSq is the baked superset of room shape, relief, blend and op reach. + const float Radius = FMath::Sqrt(FMath::Max(Room.CullRadiusSq, 0.0f)) + * FMath::Clamp(GVoxelResearchBrickRoomReachScale, 0.0f, 1.0f); + const FVector R(Radius, Radius, Radius); + bool bHasBounds = false; + return VF_IncludeBrickBounds(OutBounds, Room.Center - R, Room.Center + R, bHasBounds); + } + + static bool VF_GetJoinBrickBounds( + int32, const FCachedRoomFloorJoin& Join, FVFBrickBounds& OutBounds) + { + const float Radius = FMath::Sqrt(FMath::Max(Join.BoundRadiusSq, 0.0f)); + const FVector R(Radius, Radius, Radius); + bool bHasBounds = false; + return VF_IncludeBrickBounds(OutBounds, Join.BoundCenter - R, Join.BoundCenter + R, bHasBounds); + } + + static bool VF_GetTunnelBrickBounds( + int32, const FCachedTunnel& Tunnel, FVFBrickBounds& OutBounds) + { + // The final field evaluates both the warped SDF chain and the authored world chain. Keep + // both bound sets, including their finalized blend/detail/floor influence, in every bin. + bool bHasBounds = false; + const auto IncludeSphere = [&](const FVector& Center, float Radius) + { + return VF_IncludeBrickSphere(OutBounds, Center, Radius, bHasBounds); + }; + const auto IncludeAabb = [&](const FVector& Min, const FVector& Max, float Radius) + { + const FVector FloatMin( + static_cast(Min.X), static_cast(Min.Y), static_cast(Min.Z)); + const FVector FloatMax( + static_cast(Max.X), static_cast(Max.Y), static_cast(Max.Z)); + const float CoordinateScale = FMath::Max3( + 1.0f, + FMath::Max3(FMath::Abs(static_cast(Min.X)), FMath::Abs(static_cast(Max.X)), FMath::Abs(Radius)), + FMath::Max3(FMath::Abs(static_cast(Min.Y)), FMath::Abs(static_cast(Max.Y)), + FMath::Max(FMath::Abs(static_cast(Min.Z)), FMath::Abs(static_cast(Max.Z))))); + const float ConservativeRadius = FMath::Max(Radius, 0.0f) + + CoordinateScale * (32.0f * FLT_EPSILON); + const FVector R(ConservativeRadius, ConservativeRadius, ConservativeRadius); + return VF_IncludeBrickBounds(OutBounds, FloatMin - R, FloatMax + R, bHasBounds); + }; + return IncludeSphere(Tunnel.BoundCenter, FMath::Sqrt(FMath::Max(Tunnel.BoundRadiusSq, 0.0f))) + && IncludeSphere(Tunnel.WorldBoundCenter, FMath::Sqrt(FMath::Max(Tunnel.WorldBoundRadiusSq, 0.0f))) + && IncludeAabb(Tunnel.SDFCenterlineMin, Tunnel.SDFCenterlineMax, Tunnel.SDFInfluenceRadius) + && IncludeAabb(Tunnel.WorldCenterlineMin, Tunnel.WorldCenterlineMax, Tunnel.WorldInfluenceRadius) + && bHasBounds; + } + + static void VF_BuildResearchBrickIndices(FChunkSDFCache& Cache) + { + VF_BuildBrickIndex(Cache.RoomBrickIndex, Cache.Rooms.Num(), + [&Cache](int32 I, FVFBrickBounds& B) { return VF_GetRoomBrickBoundsSafe(I, Cache.Rooms[I], B); }); + VF_BuildBrickIndex(Cache.RoomFloorJoinBrickIndex, Cache.RoomFloorJoins.Num(), + [&Cache](int32 I, FVFBrickBounds& B) { return VF_GetJoinBrickBounds(I, Cache.RoomFloorJoins[I], B); }); + VF_BuildBrickIndex(Cache.TunnelBrickIndex, Cache.Tunnels.Num(), + [&Cache](int32 I, FVFBrickBounds& B) { return VF_GetTunnelBrickBounds(I, Cache.Tunnels[I], B); }); + } +#endif } void FChunkSDFCache::Reset() @@ -1667,6 +2029,11 @@ void FChunkSDFCache::Reset() PitSpatialIndex.Reset(); ChimneySpatialIndex.Reset(); ColumnSpatialIndex.Reset(); +#if WITH_EDITOR + RoomBrickIndex.Reset(); + RoomFloorJoinBrickIndex.Reset(); + TunnelBrickIndex.Reset(); +#endif SDFBlendRadius = 0.0f; } @@ -1684,6 +2051,11 @@ void FChunkSDFCache::Release() PitSpatialIndex.Release(); ChimneySpatialIndex.Release(); ColumnSpatialIndex.Release(); +#if WITH_EDITOR + RoomBrickIndex.Release(); + RoomFloorJoinBrickIndex.Release(); + TunnelBrickIndex.Release(); +#endif } SIZE_T FChunkSDFCache::GetAllocatedSize() const @@ -1712,6 +2084,12 @@ VoxelDensityProfile::FCacheMemoryBreakdown FChunkSDFCache::GetAllocatedSizeBreak + PitSpatialIndex.GetAllocatedSize() + ChimneySpatialIndex.GetAllocatedSize() + ColumnSpatialIndex.GetAllocatedSize()); +#if WITH_EDITOR + Breakdown.SpatialIndexBytes += static_cast( + RoomBrickIndex.GetAllocatedSize() + + RoomFloorJoinBrickIndex.GetAllocatedSize() + + TunnelBrickIndex.GetAllocatedSize()); +#endif for (const FCachedTunnel& Tunnel : Tunnels) { Breakdown.TunnelsBytes += ArrayBytes(Tunnel.ControlPoints); @@ -2753,6 +3131,12 @@ static void BakeRoomFeature( { return VF_GetColumnSpatialBounds(Index, Cache.Columns[Index], Bounds); }); +#if WITH_EDITOR + if (VoxelCaveMorphology::IsResearchBrickBinningEnabled()) + { + VF_BuildResearchBrickIndices(Cache); + } +#endif } namespace { @@ -6278,6 +6662,24 @@ namespace VoxelPassageGeometry::SealSafetyMarginVoxels; } +#if WITH_EDITOR +void VoxelCaveMorphology::SetResearchBrickBinningEnabled(bool bEnabled) +{ + GVoxelResearchBrickBinningEnabled = bEnabled; +} + +bool VoxelCaveMorphology::IsResearchBrickBinningEnabled() +{ + return GVoxelResearchBrickBinningEnabled; +} + +void VoxelCaveMorphology::SetResearchBrickRoomReachScale(float Scale) +{ + GVoxelResearchBrickRoomReachScale = VoxelMath::IsFinite(Scale) + ? FMath::Clamp(Scale, 0.0f, 1.0f) : 1.0f; +} +#endif + FVector VoxelCaveMorphology::ApplyCaveWarp( const FVector& WorldPoint, const FStrateGenerationParams& Params, @@ -8702,9 +9104,12 @@ float VoxelCaveMorphology::EvaluateSDFCached( int32 RoomCandidateCount = 0; if (!VoxelDensityAblation::IsRoomSDFOff()) { - RoomCandidateCount = VF_ForEachSpatialCandidate( - Cache.RoomSpatialIndex, Cache.Rooms.Num(), WorldX, WorldY, EvaluateRoom, - bUseSpatialIndex); + RoomCandidateCount = VF_ForEachSDFCandidate( + Cache.RoomSpatialIndex, +#if WITH_EDITOR + Cache.RoomBrickIndex, +#endif + Cache.Rooms.Num(), WorldX, WorldY, WorldZ, EvaluateRoom, bUseSpatialIndex); } if (VoxelDensityProfile::AreCountersEnabled()) { @@ -8742,9 +9147,13 @@ float VoxelCaveMorphology::EvaluateSDFCached( }; if (!VoxelDensityAblation::IsRoomSDFOff()) { - VF_ForEachSpatialCandidate( - Cache.RoomFloorJoinSpatialIndex, Cache.RoomFloorJoins.Num(), - WorldX, WorldY, EvaluateJoin, bUseSpatialIndex); + VF_ForEachSDFCandidate( + Cache.RoomFloorJoinSpatialIndex, +#if WITH_EDITOR + Cache.RoomFloorJoinBrickIndex, +#endif + Cache.RoomFloorJoins.Num(), WorldX, WorldY, WorldZ, + EvaluateJoin, bUseSpatialIndex); } //========================================================================= @@ -8839,10 +9248,14 @@ float VoxelCaveMorphology::EvaluateSDFCached( int32 TunnelCandidateCount = 0; if (!VoxelDensityAblation::IsTunnelSDFOff()) { - TunnelCandidateCount = VF_ForEachSpatialCandidate( - Cache.TunnelSpatialIndex, Cache.Tunnels.Num(), TunnelPosition.X, TunnelPosition.Y, - EvaluateTunnel, - bUseSpatialIndex); + TunnelCandidateCount = VF_ForEachSDFCandidate( + Cache.TunnelSpatialIndex, +#if WITH_EDITOR + Cache.TunnelBrickIndex, +#endif + Cache.Tunnels.Num(), static_cast(TunnelPosition.X), + static_cast(TunnelPosition.Y), static_cast(TunnelPosition.Z), + EvaluateTunnel, bUseSpatialIndex); } if (VoxelDensityProfile::AreCountersEnabled()) { @@ -9002,9 +9415,13 @@ FTunnelCoreWorldEvaluation VoxelCaveMorphology::EvaluateTunnelCoreWorld( } }; - const int32 TunnelCandidateCount = VF_ForEachSpatialCandidate( - Cache.TunnelSpatialIndex, Cache.Tunnels.Num(), WorldX, WorldY, EvaluateTunnel, - bUseSpatialIndex); + const int32 TunnelCandidateCount = VF_ForEachSDFCandidate( + Cache.TunnelSpatialIndex, +#if WITH_EDITOR + Cache.TunnelBrickIndex, +#endif + Cache.Tunnels.Num(), WorldX, WorldY, WorldZ, + EvaluateTunnel, bUseSpatialIndex); if (VoxelDensityProfile::AreCountersEnabled()) { VoxelDensityProfile::AddCounter( diff --git a/Source/VoxelForge/Public/VoxelCaveMorphology.h b/Source/VoxelForge/Public/VoxelCaveMorphology.h index ffe22c9..c5c47b9 100644 --- a/Source/VoxelForge/Public/VoxelCaveMorphology.h +++ b/Source/VoxelForge/Public/VoxelCaveMorphology.h @@ -858,6 +858,28 @@ struct FChunkSDFSpatialIndex bool GetRange(float WorldX, float WorldY, int32& OutBegin, int32& OutEnd) const; }; +// Research-only 3D broad phase used by the step-2 region evaluator. It uses 8-voxel bins and +// keeps source indices in ascending order so SmoothMin / ownership reductions retain the live +// evaluator's order. Invalid or oversized grids fall back to the existing XY index. +struct FChunkSDFBrickIndex +{ + int32 MinBrickX = 0; + int32 MinBrickY = 0; + int32 MinBrickZ = 0; + int32 NumBricksX = 0; + int32 NumBricksY = 0; + int32 NumBricksZ = 0; + bool bValid = false; + TArray CellOffsets; + TArray ItemIndices; + + void Reset(); + void Release(); + SIZE_T GetAllocatedSize() const; + bool GetRange(float WorldX, float WorldY, float WorldZ, + int32& OutBegin, int32& OutEnd) const; +}; + template FORCEINLINE int32 VF_ForEachChunkSDFSpatialCandidate( const FChunkSDFSpatialIndex& Index, int32 ItemCount, @@ -910,6 +932,12 @@ struct FChunkSDFCache FChunkSDFSpatialIndex ChimneySpatialIndex; FChunkSDFSpatialIndex ColumnSpatialIndex; +#if WITH_EDITOR + FChunkSDFBrickIndex RoomBrickIndex; + FChunkSDFBrickIndex RoomFloorJoinBrickIndex; + FChunkSDFBrickIndex TunnelBrickIndex; +#endif + // Kept with the cache because the world-space structural evaluator has no params argument. // A zero value is a valid fallback for legacy/empty caches. float SDFBlendRadius = 0.0f; @@ -951,6 +979,15 @@ struct FTunnelCoreWorldEvaluation namespace VoxelCaveMorphology { +#if WITH_EDITOR + /** + * Enable the isolated step-2 8^3-cell SDF binning path on the current worker thread. The + * default is off and gameplay callers never construct or select these indices. + */ + VOXELFORGE_API void SetResearchBrickBinningEnabled(bool bEnabled); + VOXELFORGE_API bool IsResearchBrickBinningEnabled(); + VOXELFORGE_API void SetResearchBrickRoomReachScale(float Scale); +#endif // The player-fit memo is process-wide and uses a fixed-size table. These controls are kept // beside the cave source so the command-line value is applied before generation workers start, // while the CVar remains available for an in-process A/B switch. diff --git a/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp b/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp index 1e7f550..684e034 100644 --- a/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp +++ b/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp @@ -1,5 +1,6 @@ #include "VoxelForgeExploreCommandlet.h" #include "VoxelStackOfflineSymbolizer.h" +#include "VoxelForgeRegionEvalExperiment.h" #include "CoreMinimal.h" #include "Async/ParallelFor.h" @@ -40,6 +41,7 @@ #include "VoxelStrateDefinition.h" #include "VoxelStrateManager.h" #include "VoxelStrateMeasure.h" +#include "VoxelStrateComposer.h" #include "VoxelTypes.h" #include @@ -1456,6 +1458,737 @@ struct FExploreWorld } }; +struct FRegionEvalPass +{ + VoxelForgeRegionEval::FTiming Timing; + VoxelForgeRegionEval::FCandidateVisits Visits; + TArray Densities; + TArray SampleIndices; + FIntVector Origin = FIntVector::ZeroValue; + int32 RecipeIndex = INDEX_NONE; + uint32 RecipeHash = 0; + int32 PassageCount = 0; + int32 EditChunkCount = 0; + int32 TargetedSampleCount = 0; + double WorldPrepareSeconds = 0.0; + FString StartedUtc; + FString EndedUtc; +}; + +int32 FindRoomGraphRecipeIndex(int32 Seed) +{ + for (int32 Index = 0; Index < 64; ++Index) + { + const FVoxelOpStackRecipe Recipe = VF_RollStrateStructure(Seed, Index); + if (Recipe.ShapeSource.OpClass == EVoxelStrateOpClass::RoomGraphSource) + { + return Index; + } + } + return INDEX_NONE; +} + +bool PrepareRegionEvalWorld( + FExploreWorld& World, + const FExploreArguments& BaseArguments, + int32 Seed, + int32& OutRecipeIndex, + uint32& OutRecipeHash, + int32& OutEditChunkCount, + FString& OutError) +{ + FExploreArguments Arguments = BaseArguments; + Arguments.Seed = Seed; + Arguments.bSeedOverride = true; + Arguments.bRender = false; + Arguments.bWalk = false; + Arguments.bExport = false; + Arguments.bBlockEarlyOut = false; + if (!World.Build(Arguments, OutError)) + { + return false; + } + if (!World.Definitions.IsValidIndex(Arguments.Slot)) + { + OutError = TEXT("Region evaluator target definition is missing."); + return false; + } + UVoxelStrateDefinition* Definition = World.Definitions[Arguments.Slot].Get(); + if (Definition == nullptr || Definition->GeneratorType != ECaveGeneratorType::TunnelNetwork) + { + OutError = TEXT("Region evaluator requires the owner TunnelNetwork target slot."); + return false; + } + + OutRecipeIndex = FindRoomGraphRecipeIndex(Seed); + if (OutRecipeIndex == INDEX_NONE) + { + OutError = FString::Printf( + TEXT("No RoomGraphSource recipe found in the 64-entry structure roll for seed %d."), Seed); + return false; + } + const FVoxelOpStackRecipe Recipe = VF_RollStrateStructure(Seed, OutRecipeIndex); + OutRecipeHash = VF_HashStrateStructureRecipe(Recipe); + + // Make the selected RoomGraph recipe exercise the authored warp path. The definition is a + // transient duplicate of DA_Strate3, and both the live and prepared passes receive the same + // operator parameters and recipe. + FVoxelStrateArchetypeParams NativeParams; + NativeParams.TunnelNetworkParams = Definition->GenerationParams; + NativeParams.TunnelNetworkParams.TunnelWarpStrength = FMath::Max( + NativeParams.TunnelNetworkParams.TunnelWarpStrength, 1.25f); + NativeParams.SlabParams = Definition->SlabParams; + NativeParams.MazeParams = Definition->MazeParams; + NativeParams.SurfaceParams = Definition->SurfaceParams; + NativeParams.VerticalShaftParams = Definition->VerticalShaftParams; + NativeParams.FloatingIslandParams = Definition->FloatingIslandParams; + FString ComposerError; + if (!World.Manager->SetComposerOverrideForStrate( + Arguments.Slot, Seed, Definition->GeneratorType, + NativeParams, true, &Recipe, ComposerError)) + { + OutError = FString::Printf( + TEXT("Could not install rolled composer recipe for seed %d: %s"), + Seed, *ComposerError); + return false; + } + + const int32 CenterZ = (World.TargetTopWorldZ + World.TargetBottomWorldZ) / 2; + FVoxelModification Carve; + Carve.Center = FVector(8.0f, 4.0f, static_cast(CenterZ)); + Carve.Radius = 6.0f; + Carve.Strength = -12.0f; + Carve.Shape = EVoxelBrushShape::Sphere; + const TArray CarvedChunks = World.DiffLayer->ApplyModification(Carve); + FVoxelModification Fill; + Fill.Center = FVector(-12.0f, 6.0f, static_cast(CenterZ - 10)); + Fill.Radius = 4.0f; + Fill.Strength = 12.0f; + Fill.Shape = EVoxelBrushShape::Sphere; + const TArray FilledChunks = World.DiffLayer->ApplyModification(Fill); + if (CarvedChunks.IsEmpty() || FilledChunks.IsEmpty()) + { + OutError = TEXT("Transient player carve/fill edits did not enter the diff layer."); + return false; + } + OutEditChunkCount = CarvedChunks.Num() + FilledChunks.Num(); + + int32 RecipeSeed = 0; + ECaveGeneratorType RecipeArchetype = ECaveGeneratorType::TunnelNetwork; + FVoxelStrateArchetypeParams ResolvedParams; + FVoxelOpStackRecipe ResolvedRecipe; + const int32 ChunkZ = FMath::FloorToInt(static_cast(CenterZ) / CHUNK_SIZE); + if (!World.Manager->GetRecipeForChunk( + FIntVector(0, 0, ChunkZ), RecipeSeed, RecipeArchetype, + ResolvedParams, ResolvedRecipe) + || !VF_AreStrateStructureRecipesIdentical(Recipe, ResolvedRecipe)) + { + OutError = TEXT("Composer recipe was not active on the target density lattice."); + return false; + } + return true; +} + +void AddRegionSampleIndex( + TSet& Unique, + int32 X, int32 Y, int32 Z, + FIntVector Origin, int32 Dimension) +{ + const int32 LX = X - Origin.X; + const int32 LY = Y - Origin.Y; + const int32 LZ = Z - Origin.Z; + if (LX >= 0 && LX < Dimension && LY >= 0 && LY < Dimension + && LZ >= 0 && LZ < Dimension) + { + Unique.Add(VoxelForgeRegionEval::LinearIndex(LX, LY, LZ, Dimension)); + } +} + +TArray BuildRegionEvalSampleIndices( + const FExploreWorld& World, + FIntVector Origin, + int32 Dimension, + int32 RequestedRandomSamples, + int32 Seed, + int32& OutTargetedSamples) +{ + const int64 Total = static_cast(Dimension) * Dimension * Dimension; + const int32 RandomCount = FMath::Clamp(RequestedRandomSamples, 1, static_cast(Total)); + TArray Indices; + Indices.Reserve(RandomCount + 50000); + + auto Gcd = [](int64 A, int64 B) + { + while (B != 0) + { + const int64 R = A % B; + A = B; + B = R; + } + return A; + }; + int64 Multiplier = 104729; + while (Gcd(Multiplier, Total) != 1) + { + Multiplier += 2; + } + const int64 Offset = (static_cast(Seed) * 48271ll) % Total; + for (int32 Sample = 0; Sample < RandomCount; ++Sample) + { + Indices.Add((static_cast(Sample) * Multiplier + Offset) % Total); + } + + TSet Unique; + Unique.Reserve(50000); + for (const int64 Index : Indices) + { + Unique.Add(Index); + } + const int32 RandomUniqueCount = Unique.Num(); + + // Dense probes around both edited volumes. + const int32 CenterZ = (World.TargetTopWorldZ + World.TargetBottomWorldZ) / 2; + const FVector EditCenters[] = { + FVector(8.0f, 4.0f, static_cast(CenterZ)), + FVector(-12.0f, 6.0f, static_cast(CenterZ - 10)), + }; + for (const FVector& Center : EditCenters) + { + const int32 CX = FMath::FloorToInt(Center.X); + const int32 CY = FMath::FloorToInt(Center.Y); + const int32 CZ = FMath::FloorToInt(Center.Z); + for (int32 Z = CZ - 8; Z <= CZ + 8; ++Z) + { + for (int32 Y = CY - 8; Y <= CY + 8; ++Y) + { + for (int32 X = CX - 8; X <= CX + 8; ++X) + { + AddRegionSampleIndex(Unique, X, Y, Z, Origin, Dimension); + } + } + } + } + + // Include lattice sites around each authored route tube, landing room, and floor anchor. + for (const FVoxelPassage& Passage : World.Manager->GetPassages()) + { + TArray Anchors; + Anchors.Add(Passage.UpperPoint); + Anchors.Add(Passage.LowerPoint); + Anchors.Add(Passage.UpperLanding.StandingPoint); + Anchors.Add(Passage.UpperLanding.DoorPoint); + Anchors.Add(Passage.LowerLanding.StandingPoint); + Anchors.Add(Passage.LowerLanding.DoorPoint); + Anchors.Append(Passage.ControlPoints); + for (const FVector& Anchor : Anchors) + { + const int32 AX = FMath::FloorToInt(Anchor.X); + const int32 AY = FMath::FloorToInt(Anchor.Y); + const int32 AZ = FMath::FloorToInt(Anchor.Z); + for (int32 Z = AZ - 3; Z <= AZ + 3; ++Z) + { + for (int32 Y = AY - 3; Y <= AY + 3; ++Y) + { + for (int32 X = AX - 3; X <= AX + 3; ++X) + { + AddRegionSampleIndex(Unique, X, Y, Z, Origin, Dimension); + } + } + } + } + } + + // Both strate boundaries are swept across the horizontal region, including ±3 lattice rows. + const int32 Boundaries[] = { World.TargetTopWorldZ, World.TargetBottomWorldZ }; + for (const int32 BoundaryZ : Boundaries) + { + for (int32 Dz = -3; Dz <= 3; ++Dz) + { + for (int32 Y = -128; Y <= 128; Y += 8) + { + for (int32 X = -128; X <= 128; X += 8) + { + AddRegionSampleIndex(Unique, X, Y, BoundaryZ + Dz, Origin, Dimension); + } + } + } + } + + Indices = Unique.Array(); + Indices.Sort(); + OutTargetedSamples = FMath::Max(0, Indices.Num() - RandomUniqueCount); + return Indices; +} + +bool RunRegionEvalPass( + const FExploreArguments& BaseArguments, + int32 Seed, + bool bPrepared, + float ReachScale, + bool bFullRing, + const TArray* RequestedIndices, + int32 RequestedSamples, + bool bCountCandidates, + bool bKeepDensities, + FRegionEvalPass& OutPass, + FString& OutError) +{ + OutPass = FRegionEvalPass(); + FExploreWorld World; + const double PrepareStartSeconds = FPlatformTime::Seconds(); + if (!PrepareRegionEvalWorld( + World, BaseArguments, Seed, OutPass.RecipeIndex, + OutPass.RecipeHash, OutPass.EditChunkCount, OutError)) + { + return false; + } + OutPass.WorldPrepareSeconds = FPlatformTime::Seconds() - PrepareStartSeconds; + OutPass.PassageCount = World.Manager->GetPassages().Num(); + const int32 CenterZ = (World.TargetTopWorldZ + World.TargetBottomWorldZ) / 2; + OutPass.Origin = FIntVector(-129, -129, CenterZ - 129); + if (OutPass.PassageCount <= 0) + { + OutError = FString::Printf(TEXT("Seed %d generated no inter-strate route passages."), Seed); + return false; + } + if (RequestedIndices != nullptr) + { + OutPass.SampleIndices = *RequestedIndices; + } + else + { + OutPass.SampleIndices = BuildRegionEvalSampleIndices( + World, OutPass.Origin, VoxelForgeRegionEval::RingDimension, + RequestedSamples, Seed, OutPass.TargetedSampleCount); + } + if (OutPass.SampleIndices.IsEmpty()) + { + OutError = TEXT("Region evaluator produced an empty correctness sample set."); + return false; + } + + OutPass.StartedUtc = FDateTime::UtcNow().ToIso8601(); + const bool bEvaluated = VoxelForgeRegionEval::Evaluate( + *World.Generator, OutPass.Origin, VoxelForgeRegionEval::RingDimension, + bFullRing ? nullptr : &OutPass.SampleIndices, + bFullRing, bPrepared, ReachScale, + bKeepDensities ? &OutPass.Densities : nullptr, + OutPass.Timing); + OutPass.EndedUtc = FDateTime::UtcNow().ToIso8601(); + if (!bEvaluated) + { + OutError = FString::Printf( + TEXT("Density evaluation failed or emitted a non-finite value (seed %d)."), Seed); + return false; + } + if (bCountCandidates) + { + OutPass.Visits = VoxelForgeRegionEval::CountCandidates( + *World.Generator, OutPass.Origin, + VoxelForgeRegionEval::RingDimension, + OutPass.SampleIndices, bPrepared, ReachScale); + if (OutPass.Visits.SampleCount != OutPass.SampleIndices.Num()) + { + OutError = TEXT("Candidate counter sweep did not consume the full fixed sample set."); + return false; + } + } + return true; +} + +TSharedRef RegionEvalTimingJson(const FRegionEvalPass& Pass, bool bPrepared) +{ + TSharedRef Json = MakeShared(); + Json->SetStringField(TEXT("path"), bPrepared ? TEXT("prepared_8cube_bins") : TEXT("live_GetDensityAt")); + Json->SetStringField(TEXT("started_utc"), Pass.StartedUtc); + Json->SetStringField(TEXT("ended_utc"), Pass.EndedUtc); + Json->SetNumberField(TEXT("worker_count"), Pass.Timing.Workers); + Json->SetNumberField(TEXT("observed_worker_threads"), Pass.Timing.ObservedWorkerThreads); + Json->SetNumberField(TEXT("samples"), static_cast(Pass.Timing.SampleCount)); + Json->SetNumberField(TEXT("world_setup_seconds_before_density_ring"), Pass.WorldPrepareSeconds); + Json->SetNumberField(TEXT("wall_seconds"), Pass.Timing.WallSeconds); + Json->SetNumberField(TEXT("worker_cpu_seconds"), Pass.Timing.WorkerCpuSeconds); + Json->SetNumberField(TEXT("worker_cpu_ns_per_sample"), Pass.Timing.NanosecondsPerSample); + Json->SetStringField(TEXT("worker_cpu_source"), Pass.Timing.bWorkerCpuFromOsThreadTimes + ? TEXT("summed Windows GetThreadTimes user+kernel CPU across six fixed work partitions") + : TEXT("summed worker elapsed fallback; OS thread CPU counters unavailable")); + Json->SetStringField(TEXT("quantized_hash_fnv1a64"), + FString::Printf(TEXT("%016llx"), static_cast(Pass.Timing.QuantizedHash))); + Json->SetStringField(TEXT("raw_float_hash_fnv1a64"), + FString::Printf(TEXT("%016llx"), static_cast(Pass.Timing.RawFloatHash))); + Json->SetNumberField(TEXT("saturated_samples"), static_cast(Pass.Timing.SaturatedSamples)); + Json->SetNumberField(TEXT("non_finite_samples"), static_cast(Pass.Timing.NonFiniteSamples)); + if (Pass.Visits.SampleCount > 0) + { + Json->SetNumberField(TEXT("candidate_visits_per_sample"), Pass.Visits.PerSample); + Json->SetNumberField(TEXT("room_candidate_visits"), static_cast(Pass.Visits.Room)); + Json->SetNumberField(TEXT("tunnel_candidate_visits"), static_cast(Pass.Visits.Tunnel)); + Json->SetNumberField(TEXT("tunnel_core_candidate_visits"), static_cast(Pass.Visits.TunnelCore)); + Json->SetNumberField(TEXT("passage_candidate_visits"), static_cast(Pass.Visits.Passage)); + Json->SetNumberField(TEXT("tunnel_cache_lookups"), static_cast(Pass.Visits.TunnelCacheLookups)); + Json->SetNumberField(TEXT("tunnel_cache_hits"), static_cast(Pass.Visits.TunnelCacheHits)); + Json->SetNumberField(TEXT("tunnel_cache_misses"), static_cast(Pass.Visits.TunnelCacheMisses)); + Json->SetNumberField(TEXT("sdf_cache_builds"), static_cast(Pass.Visits.SdfCacheBuilds)); + Json->SetNumberField(TEXT("candidate_counter_samples"), static_cast(Pass.Visits.SampleCount)); + Json->SetNumberField(TEXT("attribution_sample_interval"), Pass.Visits.AttributionSampleInterval); + Json->SetStringField(TEXT("attribution_basis"), TEXT("sampled VoxelDensityProfile phases; diagnostic only, excluded from clean timing")); + Json->SetNumberField(TEXT("get_density_at_attributed_ns_per_sample"), Pass.Visits.GetDensityAtAttributedNsPerSample); + Json->SetNumberField(TEXT("density_prologue_attributed_ns_per_sample"), Pass.Visits.DensityPrologueAttributedNsPerSample); + Json->SetNumberField(TEXT("density_core_attributed_ns_per_sample"), Pass.Visits.DensityCoreAttributedNsPerSample); + Json->SetNumberField(TEXT("density_disturbances_attributed_ns_per_sample"), Pass.Visits.DensityDisturbancesAttributedNsPerSample); + Json->SetNumberField(TEXT("density_structural_posts_attributed_ns_per_sample"), Pass.Visits.DensityStructuralPostsAttributedNsPerSample); + Json->SetNumberField(TEXT("density_boundary_seal_attributed_ns_per_sample"), Pass.Visits.DensityBoundarySealAttributedNsPerSample); + Json->SetNumberField(TEXT("density_diff_layer_attributed_ns_per_sample"), Pass.Visits.DensityDiffLayerAttributedNsPerSample); + Json->SetNumberField(TEXT("density_tail_attributed_ns_per_sample"), Pass.Visits.DensityTailAttributedNsPerSample); + } + return Json; +} + +struct FQuantizedComparison +{ + int64 Disagreements = 0; + int64 SignDisagreements = 0; + double MaxAbsoluteFloatDelta = 0.0; + int64 FirstIndex = INDEX_NONE; +}; + +FQuantizedComparison CompareQuantizedArrays( + const TArray& A, const TArray& B, + const TArray* LatticeIndices = nullptr) +{ + FQuantizedComparison Result; + if (A.Num() != B.Num()) + { + Result.Disagreements = FMath::Max(A.Num(), B.Num()); + return Result; + } + for (int32 Index = 0; Index < A.Num(); ++Index) + { + bool bSaturatedA = false, bFiniteA = false; + bool bSaturatedB = false, bFiniteB = false; + const int16 QA = VoxelForgeRegionEval::QuantizeDensity(A[Index], bSaturatedA, bFiniteA); + const int16 QB = VoxelForgeRegionEval::QuantizeDensity(B[Index], bSaturatedB, bFiniteB); + Result.MaxAbsoluteFloatDelta = FMath::Max( + Result.MaxAbsoluteFloatDelta, + FMath::Abs(static_cast(A[Index]) - static_cast(B[Index]))); + if (!bFiniteA || !bFiniteB || bSaturatedA || bSaturatedB || QA != QB) + { + ++Result.Disagreements; + if (Result.FirstIndex == INDEX_NONE) + { + Result.FirstIndex = LatticeIndices != nullptr + ? (*LatticeIndices)[Index] : static_cast(Index); + } + } + if ((QA < 0) != (QB < 0)) + { + ++Result.SignDisagreements; + } + } + return Result; +} + +TSharedRef RegionEvalComparisonJson(const FQuantizedComparison& Comparison) +{ + TSharedRef Json = MakeShared(); + Json->SetNumberField(TEXT("quantized_disagreements"), static_cast(Comparison.Disagreements)); + Json->SetNumberField(TEXT("quantized_sign_disagreements"), static_cast(Comparison.SignDisagreements)); + Json->SetNumberField(TEXT("max_abs_float_delta"), Comparison.MaxAbsoluteFloatDelta); + Json->SetNumberField(TEXT("first_disagreement_linear_index"), static_cast(Comparison.FirstIndex)); + return Json; +} + +int32 RunRegionEval(const FString& Params) +{ + FExploreArguments Arguments; + FString Error; + if (!ParseArguments(Params, Arguments, Error)) + { + UE_LOG(LogTemp, Error, TEXT("[VoxelForgeRegionEval] argument error: %s"), *Error); + return 2; + } + int32 RequestedSamples = 262144; + FParse::Value(*Params, TEXT("regionsamples="), RequestedSamples); + RequestedSamples = FMath::Clamp(RequestedSamples, 32768, 1000000); + IFileManager::Get().MakeDirectory(*Arguments.OutDirectory, true); + + const int32 Seeds[] = { 1337, 7331, 19427, 30241, 41011 }; + TArray> SeedReports; + TArray> OldTimingReports; + TArray> PreparedTimingReports; + TArray OldFullDensities; + TArray PreparedFullDensities; + TArray PrimarySampleIndices; + FQuantizedComparison FullComparison; + uint64 OldFullRawHash = 0; + uint64 PreparedFullRawHash = 0; + uint64 OldFullSaturatedSamples = 0; + uint64 PreparedFullSaturatedSamples = 0; + bool bAllSeedComparisonsPass = true; + bool bFullDeterminismPass = true; + int32 PrimaryTargetedSampleCount = 0; + int32 PrimaryPassageCount = 0; + int32 PrimaryEditChunkCount = 0; + int32 PrimaryRecipeIndex = INDEX_NONE; + uint32 PrimaryRecipeHash = 0; + + // Cold full-ring timings use independent transient worlds for every pass. Caches cannot be + // warmed by the comparison leg, and the seed/layout/composer/edit payload remains identical. + for (int32 Trial = 0; Trial < 2; ++Trial) + { + FRegionEvalPass Pass; + const bool bOk = RunRegionEvalPass( + Arguments, Seeds[0], false, 1.0f, true, nullptr, + RequestedSamples, Trial == 0, true, Pass, Error); + if (!bOk) + { + UE_LOG(LogTemp, Error, TEXT("[VoxelForgeRegionEval] live full-ring trial failed: %s"), *Error); + return 2; + } + CollectGarbage(GARBAGE_COLLECTION_KEEPFLAGS); + if (Trial == 0) + { + OldFullDensities = MoveTemp(Pass.Densities); + OldFullRawHash = Pass.Timing.RawFloatHash; + OldFullSaturatedSamples = Pass.Timing.SaturatedSamples; + PrimarySampleIndices = Pass.SampleIndices; + PrimaryTargetedSampleCount = Pass.TargetedSampleCount; + PrimaryPassageCount = Pass.PassageCount; + PrimaryEditChunkCount = Pass.EditChunkCount; + PrimaryRecipeIndex = Pass.RecipeIndex; + PrimaryRecipeHash = Pass.RecipeHash; + } + else + { + bFullDeterminismPass &= Pass.Timing.RawFloatHash == OldFullRawHash; + } + OldTimingReports.Add(MakeShared(RegionEvalTimingJson(Pass, false))); + } + if (PrimarySampleIndices.IsEmpty()) + { + UE_LOG(LogTemp, Error, TEXT("[VoxelForgeRegionEval] primary statistical sample set is empty.")); + return 2; + } + + for (int32 Trial = 0; Trial < 2; ++Trial) + { + FRegionEvalPass Pass; + const bool bOk = RunRegionEvalPass( + Arguments, Seeds[0], true, 1.0f, true, nullptr, + RequestedSamples, Trial == 0, true, Pass, Error); + if (!bOk) + { + UE_LOG(LogTemp, Error, TEXT("[VoxelForgeRegionEval] prepared full-ring trial failed: %s"), *Error); + return 2; + } + CollectGarbage(GARBAGE_COLLECTION_KEEPFLAGS); + if (Trial == 0) + { + PreparedFullDensities = MoveTemp(Pass.Densities); + PreparedFullRawHash = Pass.Timing.RawFloatHash; + PreparedFullSaturatedSamples = Pass.Timing.SaturatedSamples; + FullComparison = CompareQuantizedArrays(OldFullDensities, PreparedFullDensities); + } + else + { + bFullDeterminismPass &= Pass.Timing.RawFloatHash == PreparedFullRawHash; + } + PreparedTimingReports.Add(MakeShared(RegionEvalTimingJson(Pass, true))); + } + bAllSeedComparisonsPass &= FullComparison.Disagreements == 0 + && FullComparison.SignDisagreements == 0 + && OldFullSaturatedSamples == 0 + && PreparedFullSaturatedSamples == 0; + + // Statistical fixed-lattice comparisons across four additional independent seeds. The sample + // set is a deterministic affine permutation of the entire 259^3 ring plus dense strata, + // passage-floor, landing, carve, and fill probes. + for (int32 SeedIndex = 1; SeedIndex < UE_ARRAY_COUNT(Seeds); ++SeedIndex) + { + FRegionEvalPass OldPass; + if (!RunRegionEvalPass( + Arguments, Seeds[SeedIndex], false, 1.0f, false, nullptr, + RequestedSamples, true, true, OldPass, Error)) + { + UE_LOG(LogTemp, Error, TEXT("[VoxelForgeRegionEval] seed %d live sample failed: %s"), + Seeds[SeedIndex], *Error); + return 2; + } + CollectGarbage(GARBAGE_COLLECTION_KEEPFLAGS); + FRegionEvalPass NewPass; + if (!RunRegionEvalPass( + Arguments, Seeds[SeedIndex], true, 1.0f, false, &OldPass.SampleIndices, + RequestedSamples, true, true, NewPass, Error)) + { + UE_LOG(LogTemp, Error, TEXT("[VoxelForgeRegionEval] seed %d prepared sample failed: %s"), + Seeds[SeedIndex], *Error); + return 2; + } + CollectGarbage(GARBAGE_COLLECTION_KEEPFLAGS); + const FQuantizedComparison Comparison = CompareQuantizedArrays( + OldPass.Densities, NewPass.Densities, &OldPass.SampleIndices); + bAllSeedComparisonsPass &= Comparison.Disagreements == 0 + && Comparison.SignDisagreements == 0 + && OldPass.Timing.SaturatedSamples == 0 + && NewPass.Timing.SaturatedSamples == 0; + + TSharedRef SeedJson = MakeShared(); + SeedJson->SetNumberField(TEXT("seed"), Seeds[SeedIndex]); + SeedJson->SetNumberField(TEXT("samples"), OldPass.Timing.SampleCount); + SeedJson->SetNumberField(TEXT("targeted_route_edit_boundary_samples"), OldPass.TargetedSampleCount); + SeedJson->SetNumberField(TEXT("composer_recipe_index"), OldPass.RecipeIndex); + SeedJson->SetStringField(TEXT("composer_recipe_hash"), FString::Printf(TEXT("%08x"), OldPass.RecipeHash)); + SeedJson->SetNumberField(TEXT("passage_count"), OldPass.PassageCount); + SeedJson->SetNumberField(TEXT("edit_chunk_count"), OldPass.EditChunkCount); + SeedJson->SetObjectField(TEXT("comparison"), RegionEvalComparisonJson(Comparison)); + SeedJson->SetObjectField(TEXT("live_candidate_visits"), RegionEvalTimingJson(OldPass, false)); + SeedJson->SetObjectField(TEXT("prepared_candidate_visits"), RegionEvalTimingJson(NewPass, true)); + SeedReports.Add(MakeShared(SeedJson)); + } + + // Deliberately shrink one reach, evaluate the same statistical lattice points, and require + // the reference comparison to catch omitted room influence. The scale is scoped to this one + // commandlet pass and every worker resets it to 1.0 afterward. + FRegionEvalPass ShrunkPass; + if (!RunRegionEvalPass( + Arguments, Seeds[0], true, 0.50f, false, &PrimarySampleIndices, + RequestedSamples, false, true, ShrunkPass, Error)) + { + UE_LOG(LogTemp, Error, TEXT("[VoxelForgeRegionEval] deliberate reach-shrink run failed: %s"), *Error); + return 2; + } + CollectGarbage(GARBAGE_COLLECTION_KEEPFLAGS); + FQuantizedComparison ShrinkComparison; + ShrinkComparison.FirstIndex = INDEX_NONE; + for (int32 Sample = 0; Sample < ShrunkPass.Densities.Num(); ++Sample) + { + const int64 LatticeIndex = PrimarySampleIndices[Sample]; + bool bSaturatedOld = false, bFiniteOld = false; + bool bSaturatedShrink = false, bFiniteShrink = false; + const int16 QOld = VoxelForgeRegionEval::QuantizeDensity( + OldFullDensities[static_cast(LatticeIndex)], bSaturatedOld, bFiniteOld); + const int16 QShrink = VoxelForgeRegionEval::QuantizeDensity( + ShrunkPass.Densities[Sample], bSaturatedShrink, bFiniteShrink); + ShrinkComparison.MaxAbsoluteFloatDelta = FMath::Max( + ShrinkComparison.MaxAbsoluteFloatDelta, + FMath::Abs(static_cast(OldFullDensities[static_cast(LatticeIndex)]) + - static_cast(ShrunkPass.Densities[Sample]))); + if (!bFiniteOld || !bFiniteShrink || bSaturatedOld || bSaturatedShrink || QOld != QShrink) + { + ++ShrinkComparison.Disagreements; + if (ShrinkComparison.FirstIndex == INDEX_NONE) ShrinkComparison.FirstIndex = LatticeIndex; + } + if ((QOld < 0) != (QShrink < 0)) ++ShrinkComparison.SignDisagreements; + } + + double PreparedWallSum = 0.0; + for (const TSharedPtr& Value : PreparedTimingReports) + { + const TSharedPtr Json = Value->AsObject(); + PreparedWallSum += Json->GetNumberField(TEXT("wall_seconds")); + } + const double PreparedMedianWall = PreparedWallSum / 2.0; + double OldWallSum = 0.0; + for (const TSharedPtr& Value : OldTimingReports) + { + OldWallSum += Value->AsObject()->GetNumberField(TEXT("wall_seconds")); + } + const double OldMedianWall = OldWallSum / 2.0; + bool bSixWorkersObserved = true; + for (const TSharedPtr& Value : OldTimingReports) + { + bSixWorkersObserved &= Value->AsObject()->GetNumberField(TEXT("observed_worker_threads")) + >= VoxelForgeRegionEval::WorkerCount; + } + for (const TSharedPtr& Value : PreparedTimingReports) + { + bSixWorkersObserved &= Value->AsObject()->GetNumberField(TEXT("observed_worker_threads")) + >= VoxelForgeRegionEval::WorkerCount; + } + const bool bSpeedPass = PreparedMedianWall <= 0.600 && bSixWorkersObserved; + + TSharedRef Report = MakeShared(); + Report->SetNumberField(TEXT("schema_version"), 1); + Report->SetStringField(TEXT("route"), TEXT("editor-only explore commandlet; required Unreal composer, manager, and generator types")); + Report->SetStringField(TEXT("evaluator"), TEXT("live GetDensityAt final field with editor-only 8^3 SDF room/join/tunnel candidate bins")); + Report->SetStringField(TEXT("comparison_quantization"), TEXT("signed int16; round ties away from zero; multiply density by 16; saturate outside [-2048, 2047.9375] density voxels")); + Report->SetNumberField(TEXT("worker_count"), VoxelForgeRegionEval::WorkerCount); + Report->SetNumberField(TEXT("ring_dimension"), VoxelForgeRegionEval::RingDimension); + Report->SetNumberField(TEXT("ring_lattice_samples"), static_cast(VoxelForgeRegionEval::RingDimension) * VoxelForgeRegionEval::RingDimension * VoxelForgeRegionEval::RingDimension); + Report->SetNumberField(TEXT("ring_interval_span_voxels"), VoxelForgeRegionEval::RingDimension - 1); + Report->SetNumberField(TEXT("ring_wall_p50_live_seconds"), OldMedianWall); + Report->SetNumberField(TEXT("ring_wall_p50_prepared_seconds"), PreparedMedianWall); + Report->SetBoolField(TEXT("speed_pass_600ms"), bSpeedPass); + Report->SetBoolField(TEXT("all_full_ring_trials_used_six_threads"), bSixWorkersObserved); + Report->SetBoolField(TEXT("five_seed_correctness_pass"), bAllSeedComparisonsPass); + Report->SetBoolField(TEXT("two_run_determinism_pass"), bFullDeterminismPass); + Report->SetStringField(TEXT("determinism_basis"), TEXT("bitwise float32 lattice hash on two independent cold worlds for live and prepared paths")); + Report->SetObjectField(TEXT("full_ring_comparison"), RegionEvalComparisonJson(FullComparison)); + Report->SetArrayField(TEXT("live_full_ring_trials"), OldTimingReports); + Report->SetArrayField(TEXT("prepared_full_ring_trials"), PreparedTimingReports); + Report->SetArrayField(TEXT("additional_seed_comparisons"), SeedReports); + + TSharedRef Coverage = MakeShared(); + Coverage->SetStringField(TEXT("owner_settings"), Arguments.SettingsReference); + Coverage->SetStringField(TEXT("owner_strate"), Arguments.StrateReference); + Coverage->SetNumberField(TEXT("primary_seed"), Seeds[0]); + Coverage->SetNumberField(TEXT("primary_targeted_route_edit_boundary_samples"), PrimaryTargetedSampleCount); + Coverage->SetNumberField(TEXT("target_passages"), PrimaryPassageCount); + Coverage->SetNumberField(TEXT("transient_edit_chunks"), PrimaryEditChunkCount); + Coverage->SetNumberField(TEXT("composer_recipe_index"), PrimaryRecipeIndex); + Coverage->SetStringField(TEXT("composer_recipe_hash"), FString::Printf(TEXT("%08x"), PrimaryRecipeHash)); + Coverage->SetStringField(TEXT("composer_recipe_family"), TEXT("RoomGraphSource with TunnelNetwork params; rolled by VF_RollStrateStructure")); + Coverage->SetNumberField(TEXT("room_warp_strength_lower_bound"), 1.25); + Coverage->SetStringField(TEXT("room_bin_reach"), TEXT("cached CullRadiusSq: shape extent + floor-relief envelope + 3x SDF blend + conservative terrain-op reach; mouth-rise reach uses center-to-mouth distance + blend")); + Coverage->SetStringField(TEXT("tunnel_bin_reach"), TEXT("union of finalized warped-SDF and world-chain centerline bounds expanded by their influence radii, plus both conservative bounding spheres")); + Coverage->SetStringField(TEXT("warp_coordinates"), TEXT("room bins are queried after the exact live cave warp in the same coordinate domain; tunnel bins union warped-SDF and unwarped world-chain reach")); + Coverage->SetStringField(TEXT("tunnel_bin_unit"), TEXT("whole cached tunnel chain; per-chain segment loops remain in the exact evaluator")); + Coverage->SetStringField(TEXT("unbinned_writers"), TEXT("passage/landing posts, disturbances, and player diff edits stay in canonical GetDensityAt tail; this is the candidate-bin ablation, not the full region compiler")); + Coverage->SetBoolField(TEXT("route_landing_floor_probes"), PrimaryPassageCount > 0); + Coverage->SetBoolField(TEXT("strata_top_and_bottom_boundary_sweep"), true); + Coverage->SetBoolField(TEXT("player_carve_and_fill_edits"), true); + Report->SetObjectField(TEXT("content_coverage"), Coverage); + + TSharedRef Shrink = MakeShared(); + Shrink->SetNumberField(TEXT("room_reach_scale"), 0.50); + Shrink->SetBoolField(TEXT("deliberately_shrunk_bin_was_caught"), ShrinkComparison.Disagreements > 0); + Shrink->SetNumberField(TEXT("saturated_samples"), static_cast(ShrunkPass.Timing.SaturatedSamples)); + Shrink->SetNumberField(TEXT("quantized_disagreements"), static_cast(ShrinkComparison.Disagreements)); + Shrink->SetNumberField(TEXT("sign_disagreements"), static_cast(ShrinkComparison.SignDisagreements)); + Shrink->SetNumberField(TEXT("max_abs_float_delta"), ShrinkComparison.MaxAbsoluteFloatDelta); + Shrink->SetNumberField(TEXT("first_disagreement_linear_index"), static_cast(ShrinkComparison.FirstIndex)); + Report->SetObjectField(TEXT("deliberate_reach_shrink"), Shrink); + + const bool bPassed = bAllSeedComparisonsPass && bFullDeterminismPass + && ShrinkComparison.Disagreements > 0 + && ShrunkPass.Timing.SaturatedSamples == 0; + Report->SetStringField(TEXT("status"), bPassed ? TEXT("correctness_pass_speed_measured") : TEXT("correctness_failed")); + Report->SetBoolField(TEXT("pass"), bPassed); + Report->SetStringField(TEXT("speed_verdict"), bSpeedPass + ? TEXT("meets 600 ms") : TEXT("misses 600 ms; candidate bins are an ablation, not a full region compiler")); + FString JsonText; + TSharedRef> Writer = TJsonWriterFactory<>::Create(&JsonText); + if (!FJsonSerializer::Serialize(Report, Writer) + || !FFileHelper::SaveStringToFile( + JsonText, *FPaths::Combine(Arguments.OutDirectory, TEXT("region_eval.json")), + FFileHelper::EEncodingOptions::ForceUTF8WithoutBOM)) + { + UE_LOG(LogTemp, Error, TEXT("[VoxelForgeRegionEval] could not write region_eval.json.")); + return 2; + } + UE_LOG(LogTemp, Display, + TEXT("[VoxelForgeRegionEval] status=%s speed600=%s live_ms=%.3f prepared_ms=%.3f " + "full_q16_diff=%lld five_seed=%s deterministic=%s shrink_q16_diff=%lld report=%s"), + bPassed ? TEXT("pass") : TEXT("fail"), bSpeedPass ? TEXT("yes") : TEXT("no"), + OldMedianWall * 1000.0, PreparedMedianWall * 1000.0, + static_cast(FullComparison.Disagreements), + bAllSeedComparisonsPass ? TEXT("pass") : TEXT("fail"), + bFullDeterminismPass ? TEXT("pass") : TEXT("fail"), + static_cast(ShrinkComparison.Disagreements), + *FPaths::Combine(Arguments.OutDirectory, TEXT("region_eval.json"))); + return bPassed ? 0 : 2; +} + class FGeneratorDensitySampler final : public IVoxelStrateDensitySampler { public: @@ -8593,6 +9326,14 @@ int32 RunBatch(const FString& Params) int32 UVoxelForgeExploreCommandlet::Main(const FString& Params) { + const bool bRegionEvalSwitch = FParse::Param(*Params, TEXT("regioneval")); + int32 RegionEvalFlag = 0; + const bool bRegionEvalValue = FParse::Value(*Params, TEXT("regioneval="), RegionEvalFlag); + if (bRegionEvalSwitch || (bRegionEvalValue && RegionEvalFlag != 0)) + { + return RunRegionEval(Params); + } + FString SymbolizeCsvPath; if (FParse::Value(*Params, TEXT("symbolizestacks="), SymbolizeCsvPath)) { diff --git a/Source/VoxelForgeEditor/Private/VoxelForgeRegionEvalExperiment.cpp b/Source/VoxelForgeEditor/Private/VoxelForgeRegionEvalExperiment.cpp new file mode 100644 index 0000000..c5ef28f --- /dev/null +++ b/Source/VoxelForgeEditor/Private/VoxelForgeRegionEvalExperiment.cpp @@ -0,0 +1,312 @@ +#include "VoxelForgeRegionEvalExperiment.h" + +#include "Async/ParallelFor.h" +#include "HAL/PlatformTime.h" +#include "VoxelCaveMorphology.h" +#include "VoxelDensityProfile.h" +#include "VoxelGenerator.h" +#include "HAL/PlatformTLS.h" + +#if PLATFORM_WINDOWS +#include "Windows/AllowWindowsPlatformTypes.h" +THIRD_PARTY_INCLUDES_START +#include +THIRD_PARTY_INCLUDES_END +#include "Windows/HideWindowsPlatformTypes.h" +#endif + +namespace VoxelForgeRegionEval +{ + namespace + { + constexpr double WindowsFileTimeSeconds = 1.0e-7; + + uint64 Fnv1aByte(uint64 Hash, uint8 Byte) + { + Hash ^= Byte; + return Hash * 1099511628211ull; + } + + uint64 HashRawFloats(const TArray& Densities) + { + uint64 Hash = 14695981039346656037ull; + for (const float Density : Densities) + { + uint32 Bits = 0; + FMemory::Memcpy(&Bits, &Density, sizeof(Bits)); + Hash = Fnv1aByte(Hash, static_cast(Bits & 0xFFu)); + Hash = Fnv1aByte(Hash, static_cast((Bits >> 8) & 0xFFu)); + Hash = Fnv1aByte(Hash, static_cast((Bits >> 16) & 0xFFu)); + Hash = Fnv1aByte(Hash, static_cast((Bits >> 24) & 0xFFu)); + } + return Hash; + } + + bool GetCurrentThreadCpuSeconds(double& OutSeconds) + { +#if PLATFORM_WINDOWS + FILETIME CreationTime = {}; + FILETIME ExitTime = {}; + FILETIME KernelTime = {}; + FILETIME UserTime = {}; + if (GetThreadTimes( + GetCurrentThread(), &CreationTime, &ExitTime, &KernelTime, &UserTime) == 0) + { + return false; + } + ULARGE_INTEGER KernelTicks = {}; + ULARGE_INTEGER UserTicks = {}; + KernelTicks.LowPart = KernelTime.dwLowDateTime; + KernelTicks.HighPart = KernelTime.dwHighDateTime; + UserTicks.LowPart = UserTime.dwLowDateTime; + UserTicks.HighPart = UserTime.dwHighDateTime; + OutSeconds = static_cast(KernelTicks.QuadPart + UserTicks.QuadPart) + * WindowsFileTimeSeconds; + return true; +#else + (void)OutSeconds; + return false; +#endif + } + + int64 LatticeCount(int32 Dimension) + { + return static_cast(Dimension) * Dimension * Dimension; + } + } + + int64 LinearIndex(int32 X, int32 Y, int32 Z, int32 Dimension) + { + return (static_cast(Z) * Dimension + Y) * Dimension + X; + } + + int16 QuantizeDensity(float Density, bool& bOutSaturated, bool& bOutFinite) + { + bOutFinite = FMath::IsFinite(Density); + bOutSaturated = false; + if (!bOutFinite) + { + return 0; + } + const double Scaled = static_cast(Density) * QuantizationScale; + const double Rounded = Scaled >= 0.0 + ? FMath::FloorToDouble(Scaled + 0.5) + : FMath::CeilToDouble(Scaled - 0.5); + if (Rounded > static_cast(MAX_int16)) + { + bOutSaturated = true; + return MAX_int16; + } + if (Rounded < static_cast(MIN_int16)) + { + bOutSaturated = true; + return MIN_int16; + } + return static_cast(Rounded); + } + + uint64 HashQuantized(const TArray& Densities) + { + uint64 Hash = 14695981039346656037ull; + for (const float Density : Densities) + { + bool bSaturated = false; + bool bFinite = false; + const uint16 Q = static_cast(QuantizeDensity(Density, bSaturated, bFinite)); + Hash = Fnv1aByte(Hash, static_cast(Q & 0xFFu)); + Hash = Fnv1aByte(Hash, static_cast(Q >> 8)); + } + return Hash; + } + + bool Evaluate( + const UVoxelGenerator& Generator, + FIntVector OriginVoxels, + int32 Dimension, + const TArray* SampleIndices, + bool bFullRing, + bool bPrepared, + float ReachScale, + TArray* OutDensities, + FTiming& OutTiming) + { + OutTiming = FTiming(); + if (Dimension <= 0 || Dimension > 4096 + || (bFullRing && SampleIndices != nullptr) + || (!bFullRing && SampleIndices == nullptr)) + { + return false; + } + const int64 TotalLatticeSamples = LatticeCount(Dimension); + const int64 SampleCount = bFullRing + ? TotalLatticeSamples : static_cast(SampleIndices->Num()); + if (SampleCount <= 0 || SampleCount > TotalLatticeSamples) + { + return false; + } + const double StartSeconds = FPlatformTime::Seconds(); + if (OutDensities != nullptr) + { + if (SampleCount > MAX_int32) + { + return false; + } + OutDensities->SetNumUninitialized(static_cast(SampleCount)); + } + + const int32 OriginX = OriginVoxels.X; + const int32 OriginY = OriginVoxels.Y; + const int32 OriginZ = OriginVoxels.Z; + double WorkerSeconds[WorkerCount] = {}; + double WorkerCpuSeconds[WorkerCount] = {}; + bool WorkerCpuFromOsThreadTimes[WorkerCount] = {}; + bool bAllWorkerCpuFromOsThreadTimes = true; + uint32 WorkerThreadIds[WorkerCount] = {}; + uint64 WorkerNonFinite[WorkerCount] = {}; + ParallelFor(WorkerCount, [&](int32 WorkerIndex) + { + VoxelCaveMorphology::SetResearchBrickBinningEnabled(bPrepared); + VoxelCaveMorphology::SetResearchBrickRoomReachScale(ReachScale); + WorkerThreadIds[WorkerIndex] = FPlatformTLS::GetCurrentThreadId(); + const int64 Begin = (SampleCount * WorkerIndex) / WorkerCount; + const int64 End = (SampleCount * (WorkerIndex + 1)) / WorkerCount; + const double WorkerStart = FPlatformTime::Seconds(); + double WorkerCpuStart = 0.0; + const bool bHaveWorkerCpuStart = GetCurrentThreadCpuSeconds(WorkerCpuStart); + for (int64 OutputIndex = Begin; OutputIndex < End; ++OutputIndex) + { + const int64 LatticeIndex = bFullRing + ? OutputIndex : (*SampleIndices)[static_cast(OutputIndex)]; + if (LatticeIndex < 0 || LatticeIndex >= TotalLatticeSamples) + { + ++WorkerNonFinite[WorkerIndex]; + continue; + } + const int32 X = static_cast(LatticeIndex % Dimension); + const int32 Y = static_cast((LatticeIndex / Dimension) % Dimension); + const int32 Z = static_cast(LatticeIndex + / (static_cast(Dimension) * Dimension)); + const float Density = Generator.GetDensityAt( + static_cast(OriginX + X), + static_cast(OriginY + Y), + static_cast(OriginZ + Z)); + if (OutDensities != nullptr) + { + (*OutDensities)[static_cast(OutputIndex)] = Density; + } + } + WorkerSeconds[WorkerIndex] = FPlatformTime::Seconds() - WorkerStart; + double WorkerCpuEnd = 0.0; + if (bHaveWorkerCpuStart && GetCurrentThreadCpuSeconds(WorkerCpuEnd)) + { + WorkerCpuSeconds[WorkerIndex] = FMath::Max(0.0, WorkerCpuEnd - WorkerCpuStart); + WorkerCpuFromOsThreadTimes[WorkerIndex] = true; + } + else + { + WorkerCpuSeconds[WorkerIndex] = WorkerSeconds[WorkerIndex]; + } + VoxelCaveMorphology::SetResearchBrickBinningEnabled(false); + VoxelCaveMorphology::SetResearchBrickRoomReachScale(1.0f); + }, EParallelForFlags::None); + + OutTiming.WallSeconds = FPlatformTime::Seconds() - StartSeconds; + OutTiming.SampleCount = SampleCount; + for (int32 WorkerIndex = 0; WorkerIndex < WorkerCount; ++WorkerIndex) + { + OutTiming.WorkerCpuSeconds += WorkerCpuSeconds[WorkerIndex]; + bAllWorkerCpuFromOsThreadTimes &= WorkerCpuFromOsThreadTimes[WorkerIndex]; + OutTiming.NonFiniteSamples += WorkerNonFinite[WorkerIndex]; + bool bAlreadyCounted = false; + for (int32 PriorWorker = 0; PriorWorker < WorkerIndex; ++PriorWorker) + { + bAlreadyCounted |= WorkerThreadIds[PriorWorker] == WorkerThreadIds[WorkerIndex]; + } + OutTiming.ObservedWorkerThreads += bAlreadyCounted ? 0 : 1; + } + OutTiming.bWorkerCpuFromOsThreadTimes = bAllWorkerCpuFromOsThreadTimes; + OutTiming.NanosecondsPerSample = OutTiming.SampleCount > 0 + ? OutTiming.WorkerCpuSeconds * 1.0e9 / static_cast(OutTiming.SampleCount) + : 0.0; + if (OutDensities != nullptr) + { + for (const float Density : *OutDensities) + { + bool bSaturated = false; + bool bFinite = false; + QuantizeDensity(Density, bSaturated, bFinite); + OutTiming.SaturatedSamples += bSaturated ? 1u : 0u; + OutTiming.NonFiniteSamples += bFinite ? 0u : 1u; + } + OutTiming.QuantizedHash = HashQuantized(*OutDensities); + OutTiming.RawFloatHash = HashRawFloats(*OutDensities); + } + return OutTiming.NonFiniteSamples == 0; + } + + FCandidateVisits CountCandidates( + const UVoxelGenerator& Generator, + FIntVector OriginVoxels, + int32 Dimension, + const TArray& SampleIndices, + bool bPrepared, + float ReachScale) + { + FCandidateVisits Result; + Result.SampleCount = SampleIndices.Num(); + VoxelDensityProfile::Reset(); + VoxelDensityProfile::SetMode( + VoxelDensityProfile::EMode::Attribution, + VoxelDensityProfile::DefaultSampleInterval); + FTiming IgnoredTiming; + const bool bEvaluated = Evaluate( + Generator, OriginVoxels, Dimension, &SampleIndices, false, + bPrepared, ReachScale, nullptr, IgnoredTiming); + const VoxelDensityProfile::FSnapshot Snapshot = VoxelDensityProfile::Snapshot(); + VoxelDensityProfile::SetMode( + VoxelDensityProfile::EMode::Disabled, + VoxelDensityProfile::DefaultSampleInterval); + VoxelDensityProfile::Reset(); + if (!bEvaluated) + { + return Result; + } + using VoxelDensityProfile::ECounter; + Result.Room = Snapshot.Counters[static_cast(ECounter::CaveRoomCandidates)]; + Result.Tunnel = Snapshot.Counters[static_cast(ECounter::CaveTunnelCandidates)]; + Result.TunnelCore = Snapshot.Counters[static_cast(ECounter::TunnelCoreCandidates)]; + Result.Passage = Snapshot.Counters[static_cast(ECounter::PassageCandidates)]; + Result.TunnelCacheLookups = Snapshot.Counters[static_cast(ECounter::TunnelCacheLookup)]; + Result.TunnelCacheHits = Snapshot.Counters[static_cast(ECounter::TunnelCacheHit)]; + Result.TunnelCacheMisses = Snapshot.Counters[static_cast(ECounter::TunnelCacheMiss)]; + Result.SdfCacheBuilds = Snapshot.Counters[static_cast(ECounter::SdfCacheBuild)]; + Result.Total = Result.Room + Result.Tunnel + Result.TunnelCore + Result.Passage; + Result.PerSample = Result.SampleCount > 0 + ? static_cast(Result.Total) / Result.SampleCount : 0.0; + Result.AttributionSampleInterval = Snapshot.SampleInterval; + const auto AttributedNanosecondsPerSample = [&](VoxelDensityProfile::EBucket Bucket) + { + return Result.SampleCount > 0 + ? FPlatformTime::ToSeconds64(Snapshot.Cycles[static_cast(Bucket)]) + * 1.0e9 / static_cast(Result.SampleCount) + : 0.0; + }; + Result.GetDensityAtAttributedNsPerSample = AttributedNanosecondsPerSample( + VoxelDensityProfile::EBucket::GetDensityAt); + Result.DensityPrologueAttributedNsPerSample = AttributedNanosecondsPerSample( + VoxelDensityProfile::EBucket::DensityPrologue); + Result.DensityCoreAttributedNsPerSample = AttributedNanosecondsPerSample( + VoxelDensityProfile::EBucket::DensityCore); + Result.DensityDisturbancesAttributedNsPerSample = AttributedNanosecondsPerSample( + VoxelDensityProfile::EBucket::DensityDisturbances); + Result.DensityStructuralPostsAttributedNsPerSample = AttributedNanosecondsPerSample( + VoxelDensityProfile::EBucket::DensityStructuralPosts); + Result.DensityBoundarySealAttributedNsPerSample = AttributedNanosecondsPerSample( + VoxelDensityProfile::EBucket::DensityBoundarySeal); + Result.DensityDiffLayerAttributedNsPerSample = AttributedNanosecondsPerSample( + VoxelDensityProfile::EBucket::DensityDiffLayer); + Result.DensityTailAttributedNsPerSample = AttributedNanosecondsPerSample( + VoxelDensityProfile::EBucket::DensityTail); + return Result; + } +} diff --git a/Source/VoxelForgeEditor/Private/VoxelForgeRegionEvalExperiment.h b/Source/VoxelForgeEditor/Private/VoxelForgeRegionEvalExperiment.h new file mode 100644 index 0000000..6c714e6 --- /dev/null +++ b/Source/VoxelForgeEditor/Private/VoxelForgeRegionEvalExperiment.h @@ -0,0 +1,84 @@ +#pragma once + +#include "CoreMinimal.h" + +class UVoxelGenerator; + +namespace VoxelForgeRegionEval +{ + constexpr int32 WorkerCount = 6; + constexpr int32 RingDimension = 259; + // 1/16 voxel resolution covers the full prepared test field without clipping ordinary + // interior/exterior distances into indistinguishable int16 saturation values. + constexpr double QuantizationScale = 16.0; + + struct FTiming + { + int32 Workers = WorkerCount; + int32 ObservedWorkerThreads = 0; + int64 SampleCount = 0; + double WallSeconds = 0.0; + double WorkerCpuSeconds = 0.0; + double NanosecondsPerSample = 0.0; + bool bWorkerCpuFromOsThreadTimes = false; + uint64 QuantizedHash = 0; + uint64 RawFloatHash = 0; + uint64 SaturatedSamples = 0; + uint64 NonFiniteSamples = 0; + }; + + struct FCandidateVisits + { + int64 SampleCount = 0; + uint64 Room = 0; + uint64 Tunnel = 0; + uint64 TunnelCore = 0; + uint64 Passage = 0; + uint64 TunnelCacheLookups = 0; + uint64 TunnelCacheHits = 0; + uint64 TunnelCacheMisses = 0; + uint64 SdfCacheBuilds = 0; + uint64 Total = 0; + double PerSample = 0.0; + uint32 AttributionSampleInterval = 0; + double GetDensityAtAttributedNsPerSample = 0.0; + double DensityPrologueAttributedNsPerSample = 0.0; + double DensityCoreAttributedNsPerSample = 0.0; + double DensityDisturbancesAttributedNsPerSample = 0.0; + double DensityStructuralPostsAttributedNsPerSample = 0.0; + double DensityBoundarySealAttributedNsPerSample = 0.0; + double DensityDiffLayerAttributedNsPerSample = 0.0; + double DensityTailAttributedNsPerSample = 0.0; + }; + + /** + * Evaluate a deterministic lattice through the live final-field API. The prepared mode only + * enables the commandlet-only 8^3 SDF candidate bins; all field writers and their order remain + * the live GetDensityAt implementation. FullRing uses brick-grouped output ownership and + * writes every lattice density exactly once. SampleIndices are sorted lattice offsets for the + * statistical correctness / candidate-counter sweeps. + */ + bool Evaluate( + const UVoxelGenerator& Generator, + FIntVector OriginVoxels, + int32 Dimension, + const TArray* SampleIndices, + bool bFullRing, + bool bPrepared, + float ReachScale, + TArray* OutDensities, + FTiming& OutTiming); + + /** Count exact room/tunnel/core/passage candidate visits with instrumentation enabled. */ + FCandidateVisits CountCandidates( + const UVoxelGenerator& Generator, + FIntVector OriginVoxels, + int32 Dimension, + const TArray& SampleIndices, + bool bPrepared, + float ReachScale); + + int16 QuantizeDensity(float Density, bool& bOutSaturated, bool& bOutFinite); + uint64 HashQuantized(const TArray& Densities); + int64 LinearIndex(int32 X, int32 Y, int32 Z, int32 Dimension); +} diff --git a/Tools/VoxelForgeTest.ps1 b/Tools/VoxelForgeTest.ps1 index 6cf3311..aae7449 100644 --- a/Tools/VoxelForgeTest.ps1 +++ b/Tools/VoxelForgeTest.ps1 @@ -1,6 +1,6 @@ [CmdletBinding()] param( - [ValidateSet('canonical', 'owner', 'probe', 'perf', 'visual-edit', 'visual-lod', 'visual-gap-walk', 'visual-gap-inject', 'dig-motion', 'tests', 'surface-fall', 'crossing', 'gate-stress', 'parity', 'horizon', 'horizon-render', 'determinism', 'symbolize-stacks')] + [ValidateSet('canonical', 'owner', 'probe', 'perf', 'visual-edit', 'visual-lod', 'visual-gap-walk', 'visual-gap-inject', 'dig-motion', 'tests', 'surface-fall', 'crossing', 'gate-stress', 'parity', 'horizon', 'horizon-render', 'determinism', 'symbolize-stacks', 'region-eval')] [string]$Scenario = 'canonical', [switch]$Build, [switch]$EnableRender, @@ -247,13 +247,30 @@ public static class VoxelForgeTestMemoryStatus { } function Assert-EnoughFreeMemory([string]$When) { - $status = New-Object VoxelForgeTestMemoryStatus+MEMORYSTATUSEX - if (-not [VoxelForgeTestMemoryStatus]::GlobalMemoryStatusEx($status)) { - throw "Could not query free physical memory before $When; refusing to launch." - } - $freeGiB = [double]$status.ullAvailPhys / 1GB - if ($freeGiB -lt 6.0) { - throw ("Only {0:N2} GiB free before {1}; refusing to launch (minimum 6 GiB)." -f $freeGiB, $When) + $deadline = [datetime]::UtcNow.AddMinutes(30) + $didWait = $false + while ($true) { + $status = New-Object VoxelForgeTestMemoryStatus+MEMORYSTATUSEX + if (-not [VoxelForgeTestMemoryStatus]::GlobalMemoryStatusEx($status)) { + throw "Could not query free physical memory before $When; refusing to launch." + } + $freeGiB = [double]$status.ullAvailPhys / 1GB + if ($freeGiB -ge 6.0) { + if ($didWait) { + Write-Host ("Free memory recovered to {0:N2} GiB before {1}; continuing." -f $freeGiB, $When) + } + Assert-NoForeignUnreal "$When after memory check" + return + } + if ([datetime]::UtcNow -ge $deadline) { + throw ("Free memory stayed at {0:N2} GiB before {1} for 30 minutes; refusing to launch." -f $freeGiB, $When) + } + if (-not $didWait) { + Write-Host ("Waiting for free memory: {0:N2} GiB before {1}; need at least 6.00 GiB." -f $freeGiB, $When) + $didWait = $true + } + Assert-NoForeignUnreal "$When while waiting for memory" + Start-Sleep -Seconds 15 } } @@ -551,7 +568,7 @@ function Sync-OwnerAssets { $null = Copy-Item -LiteralPath $source -Destination $target -Force -PassThru [void]$rows.Add([ordered]@{ name = $assetName - used_by_scenario = $Scenario -in @('owner', 'probe', 'parity', 'tests', 'surface-fall', 'crossing', 'gate-stress', 'visual-edit', 'visual-lod', 'visual-gap-walk', 'visual-gap-inject', 'dig-motion', 'horizon', 'horizon-render', 'determinism') + used_by_scenario = $Scenario -in @('owner', 'probe', 'parity', 'tests', 'surface-fall', 'crossing', 'gate-stress', 'visual-edit', 'visual-lod', 'visual-gap-walk', 'visual-gap-inject', 'dig-motion', 'horizon', 'horizon-render', 'determinism', 'region-eval') source_path = $source source_sha256 = Get-Sha256 $source staged_path = $target @@ -599,7 +616,7 @@ function Get-AssetEvidence { $target = Join-Path $HostRoot "Content\VoxelForge\$assetName.uasset" [void]$rows.Add([ordered]@{ name = $assetName - used_by_scenario = $Scenario -in @('owner', 'probe', 'parity', 'tests', 'surface-fall', 'crossing', 'gate-stress', 'visual-edit', 'visual-lod', 'visual-gap-walk', 'visual-gap-inject', 'dig-motion', 'horizon', 'horizon-render', 'determinism') + used_by_scenario = $Scenario -in @('owner', 'probe', 'parity', 'tests', 'surface-fall', 'crossing', 'gate-stress', 'visual-edit', 'visual-lod', 'visual-gap-walk', 'visual-gap-inject', 'dig-motion', 'horizon', 'horizon-render', 'determinism', 'region-eval') source_path = $source source_sha256 = Get-Sha256 $source staged_path = $target @@ -1328,6 +1345,13 @@ try { '-WaitMutex', '-FromMsBuild', '-architecture=x64', '-NoUBA', '-ForceHeaderGeneration', '-MaxParallelActions=4', ('-Log="{0}"' -f $buildLog)) $BuildRecord = Invoke-BuildProcess $buildArgs $buildLog $buildError $buildStdout + $ubtCompilerFailures = @(Select-String -LiteralPath $buildLog ` + -Pattern 'Result:\s*Failed|error C\d+|fatal error C\d+' -ErrorAction SilentlyContinue | + Select-Object -First 20 | ForEach-Object { $_.Line.Trim() }) + if ($ubtCompilerFailures.Count -gt 0) { + $BuildRecord.status = 'failed' + $BuildRecord.compiler_diagnostics = $ubtCompilerFailures + } if ($BuildRecord.status -ne 'passed') { Add-Failure "staged build $($BuildRecord.status)" } $runtimeAfter = Get-Sha256 (Join-Path $StagePlugin 'Binaries\Win64\UnrealEditor-VoxelForge.dll') $editorAfter = Get-Sha256 (Join-Path $StagePlugin 'Binaries\Win64\UnrealEditor-VoxelForgeEditor.dll') @@ -1338,12 +1362,15 @@ try { $BuildRecord.editor_dll_after = $editorAfter $runtimeChanged = @($sourceChangedKeys | Where-Object { $_ -like 'source\voxelforge\*' }).Count -gt 0 $editorChanged = @($sourceChangedKeys | Where-Object { $_ -like 'source\voxelforgeeditor\*' }).Count -gt 0 - if ($runtimeChanged -and $runtimeBefore -eq $runtimeAfter) { + if ($BuildRecord.status -eq 'passed' -and $runtimeChanged -and $runtimeBefore -eq $runtimeAfter) { throw 'Source changed under Source/VoxelForge but the staged runtime DLL SHA-256 did not change.' } - if ($editorChanged -and $editorBefore -eq $editorAfter) { + if ($BuildRecord.status -eq 'passed' -and $editorChanged -and $editorBefore -eq $editorAfter) { throw 'Source changed under Source/VoxelForgeEditor but the staged editor DLL SHA-256 did not change.' } + if ($BuildRecord.status -ne 'passed') { + throw "Staged UBT build failed; skipping the region-eval commandlet. See $buildLog." + } } if ($Assets -eq 'owner') { $AssetEvidence = @(Sync-OwnerAssets) @@ -1398,6 +1425,36 @@ try { (Join-Path $RunRoot 'Logs\probe.log') (Join-Path $RunRoot 'Logs\probe.stdout.log') (Join-Path $RunRoot 'Logs\probe.stderr.log') $CommandletMetrics = Get-CommandletMetrics $commandletOut 'owner probe' } + 'region-eval' { + $regionLog = Join-Path $RunRoot 'Logs\region-eval.log' + $args = New-CommonArguments 'commandlet' $regionLog (Merge-Cvars $baseCvars) + $args += New-CommandletArguments $commandletOut 'walk,export' ($Assets -eq 'owner') $false 128 + $args += @('-regioneval', '-regionsamples=262144') + $commandletLedger = Invoke-UnrealLaunch 'region-eval-commandlet' $args ` + $regionLog (Join-Path $RunRoot 'Logs\region-eval.stdout.log') ` + (Join-Path $RunRoot 'Logs\region-eval.stderr.log') + $regionReportPath = Join-Path $commandletOut 'region_eval.json' + $regionReport = Read-JsonFile $regionReportPath + if ($null -eq $regionReport) { + $CommandletMetrics = [ordered]@{ + status = 'missing' + output = $commandletOut + report = $regionReportPath + } + Add-Failure "region-eval did not produce $regionReportPath" + } else { + $CommandletMetrics = [ordered]@{ + status = if ((Get-JsonValue $regionReport 'pass' $false) -and $commandletLedger.status -eq 'passed') { 'correctness_pass' } else { 'failed' } + output = $commandletOut + report = $regionReportPath + speed_pass_600ms = Get-JsonValue $regionReport 'speed_pass_600ms' $false + result = $regionReport + } + if (-not (Get-JsonValue $regionReport 'pass' $false)) { + Add-Failure 'region-eval correctness, determinism, or deliberate reach-shrink gate failed' + } + } + } 'symbolize-stacks' { if (([string]::IsNullOrWhiteSpace($StackSamplesPath)) -or -not (Test-Path -LiteralPath $StackSamplesPath -PathType Leaf)) { throw "-StackSamplesPath must name an existing raw sampler CSV." @@ -1802,6 +1859,7 @@ $result = [ordered]@{ obj_sha256 = if ($CommandletMetrics.Contains('obj_sha256')) { $CommandletMetrics.obj_sha256 } else { $null } probe_amplitudes = if ($CommandletMetrics.Contains('probe_amplitudes')) { $CommandletMetrics.probe_amplitudes } else { [ordered]@{} } commandlet = $CommandletMetrics + region_eval = if ($Scenario -eq 'region-eval') { $CommandletMetrics } else { $null } game = $GameMetrics horizon = $HorizonMetrics horizon_render = $HorizonRenderMetrics @@ -1842,6 +1900,7 @@ try { [void]$summaryLines.Add("connectivity=$(ConvertTo-CompactJson $result.connectivity)") [void]$summaryLines.Add("geometry_hash=$($result.geometry_hash)") [void]$summaryLines.Add("obj_sha256=$($result.obj_sha256)") + [void]$summaryLines.Add("region_eval=$(ConvertTo-CompactJson $result.region_eval)") [void]$summaryLines.Add("surface_fall=$(ConvertTo-CompactJson $result.surface_fall)") [void]$summaryLines.Add("strate_crossing=$(ConvertTo-CompactJson $result.strate_crossing)") [void]$summaryLines.Add("visual_staleness=$(ConvertTo-CompactJson $result.visual_staleness)")