diff --git a/CODEMAP.md b/CODEMAP.md index 5cbb2f1..a64ed72 100644 --- a/CODEMAP.md +++ b/CODEMAP.md @@ -114,7 +114,7 @@ remains `0`, and lateral regions remain gated off. |------|------| | `../../VoxelForge.uplugin` | Plugin manifest. Runtime module `VoxelForge` plus the Editor-only `VoxelForgeEditor` commandlet module. Beta. | | `VoxelForge.Build.cs` | Deps: Core, CoreUObject, Engine, **GameplayTags**, **RealtimeMeshComponent**. | -| `Tools/VoxelForgeTest.ps1` | The single guarded test entry point. `-Scenario canonical|owner|probe|perf|tests|parity`, optional `-Build`, `-Assets owner|default`, `-Cvars`, absolute `-Out`, and `-Label`; stages the host/assets, isolates user/DDC/config state, serializes Unreal launches, and emits compact `result.json`/`summary.txt` plus full logs. | +| `Tools/VoxelForgeTest.ps1` | The single guarded test entry point. `-Scenario canonical|owner|probe|perf|tests|parity|horizon|horizon-render|determinism`, optional `-Build`, `-Assets owner|default`, `-Cvars`, absolute `-Out`, `-Label`, `-HorizonLevel`, `-HorizonRenderDistanceChunks`, and `-SharedDDC`; stages the host/assets, isolates user/DDC/config state, serializes one Unreal launch at a time, enforces the 8 GiB/30-minute guards, and emits compact `result.json`/`summary.txt` plus full logs. Horizon traces report desired/loaded tiles, resident stream geometry, worker seconds, steady wall time, and peak private memory; `horizon-render` captures the owner view; `determinism` compares per-tile hashes across the default CRT path and `-voxel.CrtFma3=0`. | | `Public/VoxelForgeModule.h` / `Private/VoxelForgeModule.cpp` | `FVoxelForgeModule` boilerplate (Startup/Shutdown just log). | | `Source/VoxelForgeEditor/VoxelForgeEditor.Build.cs` / `Private/VoxelForgeExploreCommandlet.*` | Editor/commandlet-only `-run=VoxelForgeExplore`; owns bounded render/`VF_` walk/OBJ export orchestration and is not linked by Game/Shipping. | | `Public/VoxelGeometryHash.h` | Header-only geometry digest shared by commandlet export and runtime tile-hash dump; hashes mesh counts and the vertex/normal/UV/color/triangle arrays with the historical CRC order. | @@ -260,7 +260,7 @@ redesign; tile identity lives in `FVoxelTileKey` (VoxelWorld.h). | Group | Fields (line) | |-------|---------------| | Streaming | `ViewDistanceXY=16`, `ViewDistanceUp/Down=5`, `MaxConcurrentTasks=16`, `MaxMeshAppliesPerFrame=4` (defaults — actual values live on the data asset) | -| Clipmap | `ClipRadius`, `MaxClipLevel`, `FullResClipLevels`, `CoarseTileCells`, `RenderDistanceChunks` (custom horizontal reach: the outermost shell keeps generating until it covers this many chunks; 0 = off), `bFarSheetRing` + `FarSheetSpanLevels` (F18 — the render-distance ring streams per-surface SHEETS: level MaxClipLevel+span heightfield tiles instead of MC, see `GenerateSheetMesh`), skirts, `LODOctaveDrop` (T2.b octave drop on coarse tiles — 0 = off/byte-identical) (the old `LOD0/1Distance` + `ContentMaxLevel` were dead → removed) | +| Clipmap | `ClipRadius`, `MaxClipLevel=5`, `FullResClipLevels`, `CoarseTileCells`, `RenderDistanceChunks=768` (measured ship defaults; custom horizontal reach: the outermost shell keeps generating until it covers this many chunks; 0 = off), `bFarSheetRing=false` by default, plus `FarSheetSpanLevels` (F18 — the optional render-distance ring streams per-surface SHEETS: level MaxClipLevel+span heightfield tiles instead of MC, see `GenerateSheetMesh`), skirts, `LODOctaveDrop` (T2.b octave drop on coarse tiles — 0 = off/byte-identical). Existing serialized data assets retain their authored values until deliberately migrated. | | World bounds | `WorldRadiusVoxels=8192` and `EdgeSealThickness=64` (actor-space XY radial shell; radius 0 = true legacy no-op) | | Lighting | `bEnableDensityVolume` + DensityVolume* tunables (§3.11 density clipmap / mini-sun shadows) | | Rendering | `VoxelMaterial` (61) | @@ -270,7 +270,7 @@ redesign; tile identity lives in `FVoxelTileKey` (VoxelWorld.h). ### 3.5 World orchestrator — `Public/VoxelWorld.h` + `Private/VoxelWorld.cpp` `AVoxelWorld : AActor` — owns everything, drives streaming. Also `FChunkResult` struct (VoxelWorld.h:35) = async task payload (coord, chunk, meshdata, LOD, **Epoch**, request timing, -desired-set epoch and obsolete/cancellation state). +worker-built stream geometry counts/bytes, desired-set epoch and obsolete/cancellation state). **Owned objects (UPROPERTY):** `Settings`, `Generator`, `Mesher`, `StrateManager`, `DiffLayer` (VoxelWorld.h:55-78). **Storage:** `Chunks` map, `ChunkMeshes` map, @@ -307,7 +307,7 @@ casse cette arithmetique *structurellement* — `BeginPlay` loggue une **Error** | `ApplyTileResult` | — | **Shared game-thread apply** for one `FChunkResult` (async drain + sync carve): discards stale generation epochs and keys no longer desired, marks loaded, ingests capture, EMPTY releases the tile's existing component (a re-gen can flip content→empty on band change — old geometry must not linger), else `ApplyMeshToTile`. Returns true iff a visible mesh uploaded (counts the budget). Doesn't touch `PendingTiles` (caller's). | | `FVoxelStackSampler` / `FScopedVoxelStackRegistration` (`VoxelStackSampler.h/.cpp`) | — | **Diagnostic raw stack sampler, off by default.** A sampler thread captures registered generation workers' stacks (`CaptureThreadStackBackTrace`) every N µs and tags each with the tile's LOD. It writes bounded raw PCs to CSV plus a symbolization-free summary under `Saved/`; symbol resolution is deferred until after process exit so the measured process never initializes/calls DbgHelp for this diagnostic. Lifecycle teardown is serialized and registration TLS is session-tagged. Registration is per tile TASK in `LoadTile` (and the export's tasks), never per sample; disabled costs one relaxed atomic load. Game `-voxel.SampleStacks=`, commandlet `-samplestacks[=us]`. | | Room-cache performance (2026-09-13): `VF_FindPlayerFitPointForRoomMemoized` / `GPlayerFitMemo` (`VoxelCaveMorphology.cpp`), `voxel.TileCacheWindow` + `VoxelGenLOD::TileOriginVoxels/TileCellsPerAxis` (`VoxelGenerator.h`), `FChunkSDFSpatialIndex` | — | **Player-fit memo:** exact 97-word key, process-wide, `voxel.PlayerFitMemo`. **Tile cache window:** the room cache covers the requesting tile's footprint plus a fixed collect margin. It is set via `TGuardValue` in `GenerateMesh`/classify, and is window-invariant by construction (the margin does not grow with the tile). **LOD-gated:** fused path at LOD ≥ `voxel.TileCacheWindowMinLOD` (3), op-stack at LOD ≥ `voxel.TileCacheWindowOpStackMinLOD` (4); lower LODs keep the 4-chunk worker window, because the tile window slowed LOD0 evaluation ~6%. Strates whose authored terrain ops exceed the proven 256-voxel extent (or are malformed/unloaded) fall back to the legacy window (`VF_ComputeTerrainOpEnvelope`). `voxel.SpatialIndex` is -1 auto (off at fused LOD0, on elsewhere), 0 or 1. **Spatial index:** ordered XY buckets over cache primitives, exact rejection. The `static_assert` in `VoxelStrateTypes.h` ties `VF_STRATE_PARAM_FIELDS` to `FStrateGenerationParams` (83 fields + 8 B padding). | -| `GenerateTileResult` | — | **Shared worker-side gen** for one tile (async `LoadTile` task + sync `SyncRemeshTile`): [ClassifyTile (T1.d) + exact validation, only when `voxel.OuterClassifierMode=1`; **default 0 since 2026-09-13**] → `GenerateMesh`/`GenerateSheetMesh` → `BuildTileStreamSet`. Reads Generator/Mesher only → safe on a worker or the game thread; fills `FChunkResult`, no enqueue. | +| `GenerateTileResult` | — | **Shared worker-side gen** for one tile (async `LoadTile` task + sync `SyncRemeshTile`): [ClassifyTile (T1.d) + exact validation, only when `voxel.OuterClassifierMode=1`; **default 0 since 2026-09-13**] → `GenerateMesh`/`GenerateSheetMesh` → `BuildTileStreamSet`; records vertex count and resident stream allocation bytes for the horizon trace. Reads Generator/Mesher only → safe on a worker or the game thread; fills `FChunkResult`, no enqueue. | | `RecordTileHash` / `WriteTileHashDump` | — | Headless `-voxel.TileHashDump=` parity instrument. Records the exact `FVoxelMeshData` used by the streaming result, including empty/classified tiles, then writes sorted absolute tile keys, LOD step/cells, content-band limits, settings digest context, and the shared `VoxelGeometryHash` at `EndPlay`. | | `SyncRemeshTile` | — | **INSTANT DIG**: level-0 same-frame re-mesh on the game thread (`GenerateTileResult` + `ApplyTileResult` inline, Cells=CHUNK_SIZE/Step=1 + strate band, no capture). Used for the tile under the brush centre so a carve is visible THIS frame; one full-res gen on the game thread. | | `UpdateChunksAroundPosition` | 362 | Builds desired set, cancels pending keys that left it, sorts from the **true pawn position**, submits the occupied/support LOD0 tile and the next tile along the pawn heading as an absolute `BackgroundHigh` prefix, then loads/unloads and handles LOD changes. **Delta cull**: `BuildDesiredTiles` returns the LEAVERS (stamped `DesiredStamped` map, one sweep) — only those + `TransitionHold` are considered per crossing, not every loaded tile. `BuildDesiredTiles` also applies `RenderDistanceChunks`: the outermost shell widens to cover the distance — as level-MaxClipLevel MC tiles, or (F18 `bFarSheetRing`) as a SHEET ring at level MaxClipLevel+span (covered-check vs the MaxClipLevel box; `VF_OuterShell` shared with `IsTileInClipRange` so the cull sees the same horizon), dz pre-clamped to the vertical band. **§9.3 anchors:** also prunes dead `StreamingAnchors` + detects their chunk crossings → rebuilds the desired set when an anchor moves/(un)registers (`bAnchorsMoved`/`bForceDesiredRebuild`), same cadence as player movement. §8.10. | diff --git a/Source/VoxelForge/Private/VoxelStartupTrace.cpp b/Source/VoxelForge/Private/VoxelStartupTrace.cpp index 414685b..7c8b967 100644 --- a/Source/VoxelForge/Private/VoxelStartupTrace.cpp +++ b/Source/VoxelForge/Private/VoxelStartupTrace.cpp @@ -98,7 +98,9 @@ namespace int32 AllAir = 0; int32 AllSolid = 0; int32 Mixed = 0; + int64 Vertices = 0; int64 Triangles = 0; + uint64 GeometryBytes = 0; for (const FTileSample& Sample : Samples) { @@ -108,7 +110,9 @@ namespace if (Sample.Verdict == 0) ++Mixed; else if (Sample.Verdict == 1) ++AllSolid; else if (Sample.Verdict == 2) ++AllAir; + Vertices += Sample.Vertices; Triangles += Sample.Triangles; + GeometryBytes += Sample.GeometryBytes; Request.Add(Sample.RequestToApplySeconds); Queue.Add(Sample.QueueWaitSeconds); Generation.Add(Sample.GenerationSeconds); @@ -118,14 +122,16 @@ namespace } return FString::Printf( - TEXT("{\"count\":%d,\"empty\":%d,\"all_air\":%d,\"all_solid\":%d,\"mixed\":%d,\"triangles\":%lld," + TEXT("{\"count\":%d,\"empty\":%d,\"all_air\":%d,\"all_solid\":%d,\"mixed\":%d,\"vertices\":%lld,\"triangles\":%lld,\"geometry_bytes\":%llu," "\"request_apply_s\":{\"p50\":%s,\"p95\":%s,\"max\":%s}," "\"queue_s\":{\"p50\":%s,\"p95\":%s,\"max\":%s}," "\"generation_s\":{\"p50\":%s,\"p95\":%s,\"max\":%s}," "\"classify_s\":{\"p50\":%s,\"p95\":%s,\"max\":%s}," "\"mesh_s\":{\"p50\":%s,\"p95\":%s,\"max\":%s}," "\"apply_s\":{\"p50\":%s,\"p95\":%s,\"max\":%s}}"), - Count, Empty, AllAir, AllSolid, Mixed, static_cast(Triangles), + Count, Empty, AllAir, AllSolid, Mixed, + static_cast(Vertices), static_cast(Triangles), + static_cast(GeometryBytes), *Number(Percentile(Request, 0.50)), *Number(Percentile(Request, 0.95)), *Number(Maximum(Request)), *Number(Percentile(Queue, 0.50)), *Number(Percentile(Queue, 0.95)), *Number(Maximum(Queue)), *Number(Percentile(Generation, 0.50)), *Number(Percentile(Generation, 0.95)), *Number(Maximum(Generation)), @@ -260,12 +266,13 @@ void RecordTile(const FTileSample& Sample) GState->Samples.Add(Sample); ++GState->TileRecordsWritten; WriteEventLocked(*GState, TEXT("tile"), FString::Printf( - TEXT("\"tile\":[%d,%d,%d],\"level\":%d,\"verdict\":%d,\"proof\":%d,\"empty\":%d,\"triangles\":%d," + TEXT("\"tile\":[%d,%d,%d],\"level\":%d,\"verdict\":%d,\"proof\":%d,\"empty\":%d,\"vertices\":%d,\"triangles\":%d,\"geometry_bytes\":%llu," "\"request_to_apply_s\":%s,\"queue_wait_s\":%s,\"worker_queue_s\":%s,\"result_queue_s\":%s," "\"generation_s\":%s,\"classify_s\":%s,\"mesh_s\":%s,\"stream_s\":%s,\"apply_s\":%s"), Sample.TileX, Sample.TileY, Sample.TileZ, Sample.Level, Sample.Verdict, Sample.bSealedSolidProof ? 1 : 0, - Sample.bEmpty ? 1 : 0, Sample.Triangles, + Sample.bEmpty ? 1 : 0, Sample.Vertices, Sample.Triangles, + static_cast(Sample.GeometryBytes), *Number(Sample.RequestToApplySeconds), *Number(Sample.QueueWaitSeconds), *Number(Sample.WorkerQueueSeconds), *Number(Sample.ResultQueueSeconds), *Number(Sample.GenerationSeconds), *Number(Sample.ClassifySeconds), diff --git a/Source/VoxelForge/Private/VoxelWorld.cpp b/Source/VoxelForge/Private/VoxelWorld.cpp index 0a60633..94859f4 100644 --- a/Source/VoxelForge/Private/VoxelWorld.cpp +++ b/Source/VoxelForge/Private/VoxelWorld.cpp @@ -40,6 +40,8 @@ #include "Kismet/KismetMaterialLibrary.h" #include "Engine/VolumeTexture.h" #include "Engine/Engine.h" +#include "Engine/GameViewportClient.h" +#include "Slate/SceneViewport.h" #include "Camera/CameraActor.h" #include "VoxelAtmosphereManager.h" #include "DrawDebugHelpers.h" @@ -535,6 +537,7 @@ void AVoxelWorld::RegenerateAllChunks() for (auto& Pair : TileComponents) { if (Pair.Value) ReleaseTileComponent(Pair.Value); } TileComponents.Empty(); LoadedTiles.Empty(); + LoadedTileGeometry.Reset(); CollisionReadyTiles.Empty(); CollisionNotRequiredTiles.Empty(); CollisionSolidTiles.Empty(); @@ -1420,6 +1423,7 @@ void AVoxelWorld::BeginPlay() AppliedTileCount = 0; AppliedVisibleTileCount = 0; AppliedTriangleCount = 0; + LoadedTileGeometry.Reset(); for (int32 LOD = 0; LOD < TrackedClassifierLODCount; ++LOD) { AppliedVisibleTileCountByLevel[LOD] = 0; @@ -1536,18 +1540,43 @@ void AVoxelWorld::BeginPlay() } int32 TestRenderDistanceChunks = 0; - if (FParse::Value(FCommandLine::Get(), TEXT("voxel.TestRenderDistanceChunks="), + const bool bTestRenderDistance = + FParse::Value(FCommandLine::Get(), TEXT("voxel.TestRenderDistanceChunks="), TestRenderDistanceChunks) - && TestRenderDistanceChunks > 0) + && TestRenderDistanceChunks > 0; + int32 TestMaxClipLevel = -1; + const bool bTestMaxClipLevel = + FParse::Value(FCommandLine::Get(), TEXT("voxel.TestMaxClipLevel="), TestMaxClipLevel); + int32 TestFarSheetRing = -1; + const bool bTestFarSheetRing = + FParse::Value(FCommandLine::Get(), TEXT("voxel.TestFarSheetRing="), TestFarSheetRing); + if (bTestRenderDistance || bTestMaxClipLevel || bTestFarSheetRing) { if (UVoxelSettings* TestSettings = DuplicateObject(Settings, this)) { const int32 AuthoredRenderDistance = Settings->RenderDistanceChunks; - TestSettings->RenderDistanceChunks = TestRenderDistanceChunks; + const int32 AuthoredMaxClipLevel = Settings->MaxClipLevel; + const bool bAuthoredFarSheetRing = Settings->bFarSheetRing; + if (bTestRenderDistance) + { + TestSettings->RenderDistanceChunks = TestRenderDistanceChunks; + } + if (bTestMaxClipLevel) + { + TestSettings->MaxClipLevel = FMath::Clamp(TestMaxClipLevel, 0, 8); + } + if (bTestFarSheetRing) + { + TestSettings->bFarSheetRing = TestFarSheetRing != 0; + } Settings = TestSettings; UE_LOG(LogTemp, Display, - TEXT("[VoxelForgeTestCeiling] transient_render_distance_chunks=%d authored=%d"), - TestRenderDistanceChunks, AuthoredRenderDistance); + TEXT("[VoxelForgeHorizonMeasure] transient_settings=1 " + "render_distance_chunks=%d authored=%d max_clip_level=%d authored=%d " + "far_sheet_ring=%d authored=%d"), + Settings->RenderDistanceChunks, AuthoredRenderDistance, + Settings->MaxClipLevel, AuthoredMaxClipLevel, + Settings->bFarSheetRing ? 1 : 0, bAuthoredFarSheetRing ? 1 : 0); } } @@ -1801,6 +1830,7 @@ void AVoxelWorld::BeginPlay() void AVoxelWorld::ConfigureHeadlessStreamingTest() { bHeadlessStreamingTestMovement = false; + bHeadlessStreamingTestExitOnSteady = false; bHeadlessStreamingTestExitRequested = false; bStartupTraceThroughCrossing = false; bHeadlessStreamingTestCenterOverride = false; @@ -1917,9 +1947,14 @@ void AVoxelWorld::ConfigureHeadlessStreamingTest() float Speed = 800.0f; float StartSeconds = 20.0f; float ExitSeconds = 0.0f; + int32 ExitOnSteadyValue = 0; FParse::Value(CommandLine, TEXT("voxel.TestMoveSpeedCmPerSecond="), Speed); FParse::Value(CommandLine, TEXT("voxel.TestMoveStartSeconds="), StartSeconds); FParse::Value(CommandLine, TEXT("voxel.TestExitSeconds="), ExitSeconds); + if (FParse::Value(CommandLine, TEXT("voxel.TestExitOnSteady="), ExitOnSteadyValue)) + { + bHeadlessStreamingTestExitOnSteady = ExitOnSteadyValue != 0; + } int32 CrossingValue = 0; bHeadlessStrateCrossingTest = @@ -2181,6 +2216,9 @@ void AVoxelWorld::WriteTileHashDump() Writer->WriteValue(TEXT("lod_octave_drop"), Settings->LODOctaveDrop); Writer->WriteValue(TEXT("full_res_clip_levels"), Settings->FullResClipLevels); Writer->WriteValue(TEXT("coarse_tile_cells"), Settings->CoarseTileCells); + Writer->WriteValue(TEXT("max_clip_level"), Settings->MaxClipLevel); + Writer->WriteValue(TEXT("render_distance_chunks"), Settings->RenderDistanceChunks); + Writer->WriteValue(TEXT("far_sheet_ring"), Settings->bFarSheetRing); Writer->WriteValue(TEXT("strate_content_cut_min_level"), Settings->GetEffectiveStrateContentCutMinLevel()); Writer->WriteObjectEnd(); @@ -3592,6 +3630,10 @@ void AVoxelWorld::Tick(float DeltaTime) TMap SteadyLevelCounts; TMap SteadyLevelMins; TMap SteadyLevelMaxs; + TMap LoadedLevelCounts; + TMap LoadedLevelVertices; + TMap LoadedLevelTriangles; + TMap LoadedLevelGeometryBytes; for (const FVoxelTileKey& Key : DesiredSorted) { ++SteadyLevelCounts.FindOrAdd(Key.Level); @@ -3604,6 +3646,16 @@ void AVoxelWorld::Tick(float DeltaTime) Max.Y = FMath::Max(Max.Y, Key.Coord.Y); Max.Z = FMath::Max(Max.Z, Key.Coord.Z); } + for (const FVoxelTileKey& Key : LoadedTiles) + { + ++LoadedLevelCounts.FindOrAdd(Key.Level); + if (const FLoadedTileGeometryStats* Geometry = LoadedTileGeometry.Find(Key)) + { + LoadedLevelVertices.FindOrAdd(Key.Level) += Geometry->Vertices; + LoadedLevelTriangles.FindOrAdd(Key.Level) += Geometry->Triangles; + LoadedLevelGeometryBytes.FindOrAdd(Key.Level) += Geometry->GeometryBytes; + } + } VoxelForgeStartupTrace::RecordEvent(TEXT("steady_state"), FString::Printf( TEXT("\"desired_tiles\":%d,\"loaded_tiles\":%d,\"collision_ready_tiles\":%d," "\"collision_not_required_tiles\":%d,\"pending_collision_cooks\":%d," @@ -3635,8 +3687,22 @@ void AVoxelWorld::Tick(float DeltaTime) *FString::SanitizeFloat(TileExtentMetres), *FString::SanitizeFloat(SpanX), *FString::SanitizeFloat(SpanY), *FString::SanitizeFloat(SpanZ), Settings && Level > VF_MaxMarchingCubesLevel(Settings) ? 1 : 0)); + VoxelForgeStartupTrace::RecordEvent(TEXT("steady_loaded_level"), FString::Printf( + TEXT("\"level\":%d,\"tiles\":%d,\"vertices\":%llu,\"triangles\":%llu,\"geometry_bytes\":%llu"), + Level, LoadedLevelCounts.FindRef(Level), + static_cast(LoadedLevelVertices.FindRef(Level)), + static_cast(LoadedLevelTriangles.FindRef(Level)), + static_cast(LoadedLevelGeometryBytes.FindRef(Level)))); } + const bool bExitOnSteady = bHeadlessStreamingTestExitOnSteady; VoxelForgeStartupTrace::Finish(TEXT("desired_set_satisfied_and_queue_drained")); + if (bExitOnSteady && !bHeadlessStreamingTestExitRequested) + { + bHeadlessStreamingTestExitRequested = true; + UE_LOG(LogTemp, Display, + TEXT("[VoxelForgeStreamingTest] steady_exit_requested=1")); + FPlatformMisc::RequestExit(false, TEXT("VoxelForge steady streaming measurement complete")); + } } // Complete the one-shot diagnostic request only after the player's level-0 centre tile is @@ -3745,12 +3811,17 @@ void AVoxelWorld::Tick(float DeltaTime) >= static_cast(TestCeilingCaptureDelaySeconds)) { bTestCeilingCaptureRequested = true; - const bool bExecSucceeded = GEngine - && GEngine->Exec(World, TEXT("HighResShot 1")); + // UEngine::Exec is not routed through the game viewport in the command-line game + // target, so HighResShot returns false there even with a live render viewport. Call + // the viewport API directly; it schedules the same one-frame high-res capture. + const bool bCaptureAccepted = GEngine + && GEngine->GameViewport + && GEngine->GameViewport->GetGameViewport() + && GEngine->GameViewport->GetGameViewport()->TakeHighResScreenShot(); UE_LOG(LogTemp, Display, TEXT("[VoxelForgeTestCeiling] capture_requested=%d elapsed_s=%.3f " "visible_tiles=%llu triangles=%llu"), - bExecSucceeded ? 1 : 0, + bCaptureAccepted ? 1 : 0, FPlatformTime::Seconds() - TestCeilingViewBeginSeconds, static_cast(AppliedVisibleTileCount), static_cast(AppliedTriangleCount)); @@ -4680,6 +4751,7 @@ void AVoxelWorld::HandleTileCollisionCookComplete( // A failed/ignored current cook is not a loaded guarantee. Leaving the tile unloaded makes // the normal submit loop retry it while the pawn remains gated. LoadedTiles.Remove(Tile); + LoadedTileGeometry.Remove(Tile); bAllChunksLoaded = false; } @@ -4927,7 +4999,9 @@ bool AVoxelWorld::ApplyTileResult(FChunkResult& Result) Sample.Verdict = Result.ClassifyVerdict; Sample.bSealedSolidProof = Result.bSealedSolidProof; Sample.bEmpty = bResultEmpty; + Sample.Vertices = Result.NumVertices; Sample.Triangles = Result.NumTriangles; + Sample.GeometryBytes = Result.GeometryBytes; Sample.RequestToApplySeconds = RequestToApply; Sample.WorkerQueueSeconds = WorkerQueueSeconds; Sample.ResultQueueSeconds = ResultQueueSeconds; @@ -4992,6 +5066,7 @@ bool AVoxelWorld::ApplyTileResult(FChunkResult& Result) FPlatformTime::Cycles64() - EmptyReleaseStartCycles)); } LoadedTiles.Add(Result.Tile); + LoadedTileGeometry.Remove(Result.Tile); LogTileApplied(); if (Result.Tile.Level == 0 && Result.bRecordStreamingLatency) { @@ -5014,11 +5089,16 @@ bool AVoxelWorld::ApplyTileResult(FChunkResult& Result) CollisionSolidTiles.Remove(Result.Tile); PendingCollisionCooks.Remove(Result.Tile); LoadedTiles.Remove(Result.Tile); + LoadedTileGeometry.Remove(Result.Tile); bAllChunksLoaded = false; LogTileApplied(); return false; } LoadedTiles.Add(Result.Tile); + LoadedTileGeometry.Add(Result.Tile, FLoadedTileGeometryStats{ + static_cast(FMath::Max(0, Result.NumVertices)), + static_cast(FMath::Max(0, Result.NumTriangles)), + Result.GeometryBytes}); LogTileApplied(); return true; } @@ -6503,6 +6583,12 @@ void AVoxelWorld::GenerateTileResult(const FVoxelTileKey& Tile, const FIntVector VoxelDensityProfile::FScopedTimer RuntimeStreamTimer( VoxelDensityProfile::EBucket::RuntimeStreamBuilding); BuildTileStreamSet(*Result.Streams, MeshData); + Result.NumVertices = MeshData.Vertices.Num(); + Result.GeometryBytes = 0; + Result.Streams->ForEach([&](const RealtimeMesh::FRealtimeMeshStream& Stream) + { + Result.GeometryBytes += static_cast(Stream.GetAllocatedSize()); + }); StreamSeconds = bMeasureTile ? FPlatformTime::Seconds() - StreamStartSeconds : 0.0; if (ShouldAbortWork(ObsoleteFlag)) { @@ -6651,6 +6737,7 @@ void AVoxelWorld::UnloadTile(const FVoxelTileKey& Tile) TileComponents.Remove(Tile); } LoadedTiles.Remove(Tile); + LoadedTileGeometry.Remove(Tile); TransitionHold.Remove(Tile); // couvre aussi le settled cull (qui ne tient pas la hold à jour) } diff --git a/Source/VoxelForge/Public/VoxelSettings.h b/Source/VoxelForge/Public/VoxelSettings.h index c01e612..972887c 100644 --- a/Source/VoxelForge/Public/VoxelSettings.h +++ b/Source/VoxelForge/Public/VoxelSettings.h @@ -148,8 +148,9 @@ public: // Coarsest LOD level. 0 = full-res only (no clipmap). Each level up doubles tile size & // reach at ~constant cost — raise this to see much farther for cheap. (Level L step = 2^L.) + // Ship default: level 5, the measured horizon-distance balance at 768 chunks. UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Voxel|Clipmap", meta = (ClampMin = "0", ClampMax = "8")) - int32 MaxClipLevel = 4; + int32 MaxClipLevel = 5; // How many near levels mesh at FULL resolution (CHUNK_SIZE cells). Levels at or above this // use the cheaper CoarseTileCells → MUCH faster gen for the far field (it's far, so the @@ -171,9 +172,9 @@ public: // cell must fit inside a strate band — level ≥7 blanks via the too-coarse skip) and buy the // remaining horizon here. Cost: the extra ring is all same-level tiles — tile/draw/gen count // grows with (distance / 2^MaxClipLevel)², so each MaxClipLevel step down quadruples the ring. - // 0 = off (natural reach, byte-identical streaming). + // Ship default: 768 chunks (about 6.1 km at 8 m/chunk). 0 = off (natural reach). UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Voxel|Clipmap", meta = (ClampMin = "0")) - int32 RenderDistanceChunks = 0; + int32 RenderDistanceChunks = 768; // F18 — the render-distance ring streams per-surface SHEETS instead of MC tiles: in an open // strate the far field is exactly two heightfields (TerrainZ + sky-cap CeilSurf, both already @@ -182,10 +183,11 @@ public: // sheet spans 2^FarSheetSpanLevels MC-tile footprints per axis). Non-open strates produce // empty sheets (their far ring was enclosed rock anyway). Carved features (passages, chasms, // spine) don't show at sheet distance. Needs the strate band armed (StrateContentCutMinLevel - // active); in the inter-strate gap the sheet ring blanks until you land. Off = the ring stays - // MC tiles at level MaxClipLevel (pre-F18 behaviour). Only matters when RenderDistanceChunks > 0. + // active); in the inter-strate gap the sheet ring blanks until you land. The ship default is OFF: + // the ring stays real MC tiles at level MaxClipLevel (pre-F18 behaviour). Only matters when + // RenderDistanceChunks > 0. UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Voxel|Clipmap") - bool bFarSheetRing = true; + bool bFarSheetRing = false; // Sheet tile size = MaxClipLevel + this many levels (2 → one sheet covers 4×4 MC-tile // footprints → ~16× fewer far components). Sampling density stays that of the MaxClipLevel diff --git a/Source/VoxelForge/Public/VoxelStartupTrace.h b/Source/VoxelForge/Public/VoxelStartupTrace.h index ca6ba16..305e286 100644 --- a/Source/VoxelForge/Public/VoxelStartupTrace.h +++ b/Source/VoxelForge/Public/VoxelStartupTrace.h @@ -16,7 +16,9 @@ namespace VoxelForgeStartupTrace int32 Verdict = -1; // Mixed=0, AllSolid=1, AllAir=2, -1=not classified (sheet/early path) bool bSealedSolidProof = false; bool bEmpty = true; + int32 Vertices = 0; int32 Triangles = 0; + uint64 GeometryBytes = 0; // This is worker-result -> game-thread mesh submission. Collision readiness is a later // per-tile RMC completion event recorded separately as collision_ready. double RequestToApplySeconds = 0.0; diff --git a/Source/VoxelForge/Public/VoxelWorld.h b/Source/VoxelForge/Public/VoxelWorld.h index 6a7c0ff..3712034 100644 --- a/Source/VoxelForge/Public/VoxelWorld.h +++ b/Source/VoxelForge/Public/VoxelWorld.h @@ -109,6 +109,8 @@ struct FChunkResult uint64 GenerationEndCycles = 0; uint64 ApplyStartCycles = 0; int32 ClassifyVerdict = -1; double ClassifySeconds = 0.0; double MeshSeconds = 0.0; double StreamSeconds = 0.0; int32 NumTriangles = 0; int32 NumCeilingTriangles = 0; + int32 NumVertices = 0; + uint64 GeometryBytes = 0; bool bSealedSolidProof = false; // true when the cheap gap/seal proof skipped meshing bool bOutOfLayoutAirProof = false; // true when the exact constant-air guard skipped meshing // Clean, low-overhead work counters. These are carried with the result so the worker can @@ -154,6 +156,14 @@ struct FVoxelTileHashDumpRecord FString GeometryHash; }; +/** Resident CPU geometry payload associated with one currently loaded tile. */ +struct FLoadedTileGeometryStats +{ + uint64 Vertices = 0; + uint64 Triangles = 0; + uint64 GeometryBytes = 0; +}; + UCLASS() class VOXELFORGE_API AVoxelWorld : public AActor { @@ -985,6 +995,7 @@ public: uint64 AppliedVisibleTileCountByLevel[TrackedClassifierLODCount]{}; uint64 AppliedTriangleCountByLevel[TrackedClassifierLODCount]{}; uint64 AppliedBandTileCountByLevel[TrackedClassifierLODCount]{}; + TMap LoadedTileGeometry; struct FStreamingLatencySample { @@ -998,6 +1009,7 @@ public: // Headless validation hook. These are intentionally non-UPROPERTY state and are enabled only // by explicit command-line switches; they do not alter ordinary player movement or streaming. bool bHeadlessStreamingTestMovement = false; + bool bHeadlessStreamingTestExitOnSteady = false; bool bHeadlessStreamingTestExitRequested = false; bool bStartupTraceThroughCrossing = false; bool bHeadlessStreamingTestCenterOverride = false; diff --git a/Tools/VoxelForgeTest.ps1 b/Tools/VoxelForgeTest.ps1 index 07c5c0c..27f2d1d 100644 --- a/Tools/VoxelForgeTest.ps1 +++ b/Tools/VoxelForgeTest.ps1 @@ -1,6 +1,6 @@ [CmdletBinding()] param( - [ValidateSet('canonical', 'owner', 'probe', 'perf', 'tests', 'surface-fall', 'crossing', 'gate-stress', 'parity')] + [ValidateSet('canonical', 'owner', 'probe', 'perf', 'tests', 'surface-fall', 'crossing', 'gate-stress', 'parity', 'horizon', 'horizon-render', 'determinism')] [string]$Scenario = 'canonical', [switch]$Build, [ValidateSet('owner', 'default')] @@ -8,7 +8,12 @@ param( [hashtable]$Cvars = @{}, [string]$TestFilter, [string]$Out, - [string]$Label + [string]$Label, + [ValidateRange(0, 8)] + [int]$HorizonLevel = 5, + [ValidateRange(1, 100000)] + [int]$HorizonRenderDistanceChunks = 768, + [string]$SharedDDC ) # One guarded entry point for all VoxelForge measurements. It intentionally keeps the Unreal @@ -28,7 +33,7 @@ $EngineRoot = 'E:\Program Files\Epic Games\UE_5.7' $EditorExe = Join-Path $EngineRoot 'Engine\Binaries\Win64\UnrealEditor-Cmd.exe' $UbtDll = Join-Path $EngineRoot 'Engine\Binaries\DotNET\UnrealBuildTool\UnrealBuildTool.dll' $LaunchTimeoutSeconds = 1800 -$UnrealPrivateMemoryLimitBytes = 6GB +$UnrealPrivateMemoryLimitBytes = 8GB # The VoxelForge category includes two late, several-minute op-stack sweeps. They remain in # the suite; when the serialized aggregate launch reaches this 30-minute guard, validate those # named tests with -TestFilter so every test still runs under the standing launch limit. @@ -44,6 +49,15 @@ $BuildRecord = [ordered]@{ requested = [bool]$Build; status = 'not_requested' } $AssetEvidence = @() $CommandletMetrics = [ordered]@{} $GameMetrics = [ordered]@{} +$HorizonMetrics = [ordered]@{ + requested = $Scenario -in @('horizon', 'horizon-render') + status = 'not_run' + level = $HorizonLevel + render_distance_chunks = $HorizonRenderDistanceChunks + far_sheet_ring = $false +} +$HorizonRenderMetrics = [ordered]@{ requested = $Scenario -eq 'horizon-render'; status = 'not_run' } +$DeterminismEvidence = [ordered]@{ requested = $Scenario -eq 'determinism'; status = 'not_run' } $SurfaceFallMetrics = [ordered]@{ requested = $Scenario -eq 'surface-fall' -or ($Scenario -eq 'tests' -and [string]::IsNullOrWhiteSpace($TestFilter)) @@ -66,6 +80,7 @@ $LocalAppDataRoot = Join-Path ([Environment]::GetFolderPath('LocalApplicationDat $RunRoot = $null $ResultPath = $null $SummaryPath = $null +$DDCPath = $null function Add-Failure([string]$Reason) { if ([string]::IsNullOrWhiteSpace($Reason)) { return } @@ -485,7 +500,7 @@ function Sync-OwnerAssets { $null = Copy-Item -LiteralPath $source -Destination $target -Force -PassThru [void]$rows.Add([ordered]@{ name = $assetName - used_by_scenario = $Scenario -in @('owner', 'probe', 'parity', 'tests', 'surface-fall', 'crossing', 'gate-stress') + used_by_scenario = $Scenario -in @('owner', 'probe', 'parity', 'tests', 'surface-fall', 'crossing', 'gate-stress', 'horizon', 'horizon-render', 'determinism') source_path = $source source_sha256 = Get-Sha256 $source staged_path = $target @@ -533,7 +548,7 @@ function Get-AssetEvidence { $target = Join-Path $HostRoot "Content\VoxelForge\$assetName.uasset" [void]$rows.Add([ordered]@{ name = $assetName - used_by_scenario = $Scenario -in @('owner', 'probe', 'parity', 'tests', 'surface-fall', 'crossing', 'gate-stress') + used_by_scenario = $Scenario -in @('owner', 'probe', 'parity', 'tests', 'surface-fall', 'crossing', 'gate-stress', 'horizon', 'horizon-render', 'determinism') source_path = $source source_sha256 = Get-Sha256 $source staged_path = $target @@ -558,13 +573,19 @@ function Merge-Cvars([System.Collections.IDictionary]$Base) { return $merged } -function New-CommonArguments([string]$Kind, [string]$LogPath, $CvarMap) { +function New-CommonArguments([string]$Kind, [string]$LogPath, $CvarMap, [bool]$EnableRender = $false) { $safeLabel = ($Label -replace '[^A-Za-z0-9_.-]', '_') $args = [System.Collections.Generic.List[string]]::new() # Quoting the whole -project= token is intentional: this is the failure mode that used to # swallow every argument after "Projet Unreal". [void]$args.Add(('-project="{0}"' -f $HostProject)) - [void]$args.Add('-nullrhi') + if ($EnableRender) { + [void]$args.Add('-RenderOffscreen') + [void]$args.Add('-ResX=1280') + [void]$args.Add('-ResY=720') + } else { + [void]$args.Add('-nullrhi') + } [void]$args.Add('-DDC-ForceMemoryCache') [void]$args.Add('-unattended') [void]$args.Add('-nop4') @@ -580,7 +601,7 @@ function New-CommonArguments([string]$Kind, [string]$LogPath, $CvarMap) { [void]$args.Add(('-userdir="{0}"' -f (Join-Path $RunRoot 'User'))) [void]$args.Add(('-ZenDataPath="{0}"' -f (Join-Path $RunRoot 'ZenData'))) [void]$args.Add('-ddc=NoZenLocalFallback') - [void]$args.Add(('-LocalDataCachePath="{0}"' -f (Join-Path $RunRoot 'DDC'))) + [void]$args.Add(('-LocalDataCachePath="{0}"' -f $DDCPath)) [void]$args.Add(('-abslog="{0}"' -f $LogPath)) if ($Kind -eq 'commandlet') { [void]$args.Add('-run=VoxelForgeExplore') @@ -720,6 +741,212 @@ function Get-StreamingMetrics([string]$LogPath) { return $metrics } +function Get-HorizonMetrics([string]$TracePath, $Ledger, $LatencyMetrics) { + $metrics = [ordered]@{ + status = 'missing' + trace = $TracePath + desired_tiles_by_level = [ordered]@{} + loaded_tiles_by_level = [ordered]@{} + vertices_by_level = [ordered]@{} + triangles_by_level = [ordered]@{} + geometry_memory_bytes_by_level = [ordered]@{} + geometry_memory_mib_by_level = [ordered]@{} + desired_tiles_total = 0 + loaded_tiles_total = 0 + vertices_total = 0 + triangles_total = 0 + geometry_memory_bytes_total = 0 + geometry_memory_mib_total = 0.0 + worker_generation_seconds = if ($null -ne $LatencyMetrics) { + Get-JsonValue $LatencyMetrics 'worker_generation_sum_s' $null + } else { $null } + settled_trace_seconds = $null + trace_summary_wall_seconds = $null + process_wall_seconds = $null + peak_private_bytes = if ($null -ne $Ledger) { + [int64](Get-JsonValue $Ledger 'peak_private_bytes' 0) + } else { 0L } + peak_private_gib = 0.0 + trace_records_seen = $null + trace_records_written = $null + trace_records_suppressed = $null + } + $metrics.peak_private_gib = [Math]::Round($metrics.peak_private_bytes / 1GB, 3) + + if (-not (Test-Path -LiteralPath $TracePath -PathType Leaf)) { return $metrics } + $events = [System.Collections.Generic.List[object]]::new() + foreach ($line in @(Get-Content -LiteralPath $TracePath -ErrorAction SilentlyContinue)) { + if ([string]::IsNullOrWhiteSpace($line)) { continue } + try { [void]$events.Add(($line | ConvertFrom-Json -ErrorAction Stop)) } catch { } + } + $steady = @($events | Where-Object { [string](Get-JsonValue $_ 'type' '') -eq 'steady_state' } | Select-Object -Last 1) + $summary = @($events | Where-Object { [string](Get-JsonValue $_ 'type' '') -eq 'summary' } | Select-Object -Last 1) + $desiredEvents = @($events | Where-Object { [string](Get-JsonValue $_ 'type' '') -eq 'steady_desired_level' }) + $loadedEvents = @($events | Where-Object { [string](Get-JsonValue $_ 'type' '') -eq 'steady_loaded_level' }) + if ($steady.Count -eq 0 -or $summary.Count -eq 0 -or $desiredEvents.Count -eq 0) { + return $metrics + } + + foreach ($event in $desiredEvents) { + $level = [string][int](Get-JsonValue $event 'level' -1) + $metrics.desired_tiles_by_level[$level] = [int](Get-JsonValue $event 'tiles' 0) + } + foreach ($event in $loadedEvents) { + $level = [string][int](Get-JsonValue $event 'level' -1) + $metrics.loaded_tiles_by_level[$level] = [int](Get-JsonValue $event 'tiles' 0) + $metrics.vertices_by_level[$level] = [int64](Get-JsonValue $event 'vertices' 0) + $metrics.triangles_by_level[$level] = [int64](Get-JsonValue $event 'triangles' 0) + $bytes = [int64](Get-JsonValue $event 'geometry_bytes' 0) + $metrics.geometry_memory_bytes_by_level[$level] = $bytes + $metrics.geometry_memory_mib_by_level[$level] = [Math]::Round($bytes / 1MB, 3) + } + foreach ($value in $metrics.desired_tiles_by_level.Values) { $metrics.desired_tiles_total += [int64]$value } + foreach ($value in $metrics.loaded_tiles_by_level.Values) { $metrics.loaded_tiles_total += [int64]$value } + foreach ($value in $metrics.vertices_by_level.Values) { $metrics.vertices_total += [int64]$value } + foreach ($value in $metrics.triangles_by_level.Values) { $metrics.triangles_total += [int64]$value } + foreach ($value in $metrics.geometry_memory_bytes_by_level.Values) { $metrics.geometry_memory_bytes_total += [int64]$value } + $metrics.geometry_memory_mib_total = [Math]::Round($metrics.geometry_memory_bytes_total / 1MB, 3) + $metrics.settled_trace_seconds = [double](Get-JsonValue $steady[0] 't_s' 0.0) + $metrics.trace_summary_wall_seconds = [double](Get-JsonValue $summary[0] 'wall_s' 0.0) + $metrics.trace_records_seen = [int](Get-JsonValue $summary[0] 'tile_records_seen' 0) + $metrics.trace_records_written = [int](Get-JsonValue $summary[0] 'tile_records_written' 0) + $metrics.trace_records_suppressed = [int](Get-JsonValue $summary[0] 'tile_records_suppressed' 0) + if ($null -ne $Ledger) { + $started = [datetime]::Parse([string](Get-JsonValue $Ledger 'started_utc' '')) + $ended = [datetime]::Parse([string](Get-JsonValue $Ledger 'ended_utc' '')) + $metrics.process_wall_seconds = [Math]::Round(($ended.ToUniversalTime() - $started.ToUniversalTime()).TotalSeconds, 3) + } + $metrics.status = 'ok' + return $metrics +} + +function Get-HorizonRenderMetrics($Ledger, [string]$LogPath) { + $metrics = [ordered]@{ + status = 'failed' + log = $LogPath + capture_requested = $null + capture_elapsed_seconds = $null + capture_visible_tiles = $null + capture_triangles = $null + images = @() + } + $lines = @(Select-String -LiteralPath $LogPath -Pattern '\[VoxelForgeTestCeiling\] capture_requested=' -ErrorAction SilentlyContinue) + if ($lines.Count -gt 0) { + $line = [string]$lines[-1].Line + $metrics.raw = $line + foreach ($key in @('capture_requested', 'elapsed_s', 'visible_tiles', 'triangles')) { + $match = [regex]::Match($line, ('{0}=(?[^ ]+)' -f $key)) + if ($match.Success) { + if ($key -eq 'elapsed_s') { $metrics.capture_elapsed_seconds = [double]$match.Groups['v'].Value } + elseif ($key -eq 'visible_tiles') { $metrics.capture_visible_tiles = [int64]$match.Groups['v'].Value } + elseif ($key -eq 'triangles') { $metrics.capture_triangles = [int64]$match.Groups['v'].Value } + else { $metrics.capture_requested = [int]$match.Groups['v'].Value } + } + } + } + $started = if ($null -ne $Ledger) { + [datetime]::Parse([string](Get-JsonValue $Ledger 'started_utc' '')) + } else { [datetime]::MinValue } + $roots = @( + (Join-Path $HostRoot 'Saved'), + $RunRoot, + (Join-Path $RunRoot 'User')) + $images = [System.Collections.Generic.List[object]]::new() + foreach ($root in $roots) { + if (-not (Test-Path -LiteralPath $root -PathType Container)) { continue } + foreach ($image in @(Get-ChildItem -LiteralPath $root -Filter '*.png' -File -Recurse -ErrorAction SilentlyContinue)) { + if ($image.LastWriteTimeUtc -lt $started.ToUniversalTime().AddSeconds(-2)) { continue } + [void]$images.Add([ordered]@{ + path = $image.FullName + bytes = [int64]$image.Length + modified_utc = $image.LastWriteTimeUtc.ToString('o') + }) + } + } + $metrics.images = @($images | Sort-Object path -Unique) + if ($metrics.capture_requested -eq 1 -and $metrics.images.Count -gt 0) { + $metrics.status = 'passed' + } elseif ($null -eq $Ledger -or (Get-JsonValue $Ledger 'status' 'failed') -ne 'passed') { + $metrics.status = 'not_run' + } + return $metrics +} + +function Compare-DeterminismDumps([string]$DefaultPath, [string]$CrtFma3OffPath) { + $defaultJson = Read-JsonFile $DefaultPath + $fmaJson = Read-JsonFile $CrtFma3OffPath + $evidence = [ordered]@{ + status = 'failed' + default_dump = $DefaultPath + crt_fma3_off_dump = $CrtFma3OffPath + default_record_count = 0 + crt_fma3_off_record_count = 0 + compared_tile_count = 0 + first_differences = @() + } + if ($null -eq $defaultJson -or $null -eq $fmaJson) { + $evidence.reason = 'one or both tile hash dumps are missing' + return $evidence + } + $defaultRecords = @((Get-JsonValue $defaultJson 'records' @())) + $fmaRecords = @((Get-JsonValue $fmaJson 'records' @())) + $evidence.default_record_count = $defaultRecords.Count + $evidence.crt_fma3_off_record_count = $fmaRecords.Count + $defaultByKey = @{} + $fmaByKey = @{} + foreach ($record in $defaultRecords) { + $key = "$($record.tile_x):$($record.tile_y):$($record.tile_z):$($record.level)" + $defaultByKey[$key] = $record + } + foreach ($record in $fmaRecords) { + $key = "$($record.tile_x):$($record.tile_y):$($record.tile_z):$($record.level)" + $fmaByKey[$key] = $record + } + $differences = [System.Collections.Generic.List[object]]::new() + $keys = @($defaultByKey.Keys) + @($fmaByKey.Keys) | Sort-Object -Unique + foreach ($key in $keys) { + if (-not $defaultByKey.ContainsKey($key)) { + [void]$differences.Add([ordered]@{ tile = $key; reason = 'missing_from_default' }) + continue + } + if (-not $fmaByKey.ContainsKey($key)) { + [void]$differences.Add([ordered]@{ tile = $key; reason = 'missing_from_crt_fma3_off' }) + continue + } + $left = $defaultByKey[$key] + $right = $fmaByKey[$key] + foreach ($property in @('origin_x_voxels', 'origin_y_voxels', 'origin_z_voxels', + 'step', 'cells', 'band_chunk_lo', 'band_chunk_hi', + 'sheet_tile', 'empty', 'triangle_count', 'geometry_hash')) { + if ([string](Get-JsonValue $left $property '') -ne [string](Get-JsonValue $right $property '')) { + [void]$differences.Add([ordered]@{ + tile = $key + reason = "property_mismatch:$property" + default = Get-JsonValue $left $property $null + crt_fma3_off = Get-JsonValue $right $property $null + }) + break + } + } + } + $evidence.compared_tile_count = [Math]::Min($defaultByKey.Count, $fmaByKey.Count) + $evidence.first_differences = @($differences | Select-Object -First 10) + $leftConfig = Get-JsonValue $defaultJson 'runtime_config' $null + $rightConfig = Get-JsonValue $fmaJson 'runtime_config' $null + $evidence.runtime_config_match = (ConvertTo-CompactJson $leftConfig) -eq (ConvertTo-CompactJson $rightConfig) + $evidence.default_runtime_config = $leftConfig + $evidence.crt_fma3_off_runtime_config = $rightConfig + if ($differences.Count -eq 0 -and $evidence.runtime_config_match -and + $defaultByKey.Count -eq $fmaByKey.Count) { + $evidence.status = 'passed' + } else { + $evidence.reason = if ($differences.Count -gt 0) { 'per-tile geometry differs' } + elseif (-not $evidence.runtime_config_match) { 'runtime configuration differs' } + else { 'tile record counts differ' } + } + return $evidence +} + function Get-SurfaceFallMetrics([string]$LogPath) { $lineMatch = @(Select-String -LiteralPath $LogPath -Pattern '\[VoxelForgeSurfaceFallTest\] result=' -ErrorAction SilentlyContinue | Select-Object -Last 1) @@ -926,7 +1153,17 @@ try { Ensure-Directory (Join-Path $RunRoot 'Logs') Ensure-Directory (Join-Path $RunRoot 'User') Ensure-Directory (Join-Path $RunRoot 'ZenData') - Ensure-Directory (Join-Path $RunRoot 'DDC') + $DDCPath = Join-Path $RunRoot 'DDC' + if (-not [string]::IsNullOrWhiteSpace($SharedDDC)) { + if (-not [IO.Path]::IsPathRooted($SharedDDC)) { + throw "-SharedDDC must be an absolute path: $SharedDDC" + } + $DDCPath = Get-FullPath $SharedDDC + if (-not (Test-UnderPath $DDCPath (Join-Path $PluginRoot 'Saved'))) { + throw "-SharedDDC must stay under the plugin Saved directory: $DDCPath" + } + } + Ensure-Directory $DDCPath $localBefore = Get-StateSnapshot $LocalAppDataRoot foreach ($required in @($EditorExe, $UbtDll, $HostProject, $ProjectFile)) { @@ -944,7 +1181,7 @@ try { $buildStdout = Join-Path $RunRoot 'Logs\build.stdout.log' $buildError = Join-Path $RunRoot 'Logs\build.err.log' $buildArgs = @(('"{0}"' -f $UbtDll), 'UnrealEditor', 'Win64', 'Development', ('-Project="{0}"' -f $HostProject), - '-WaitMutex', '-FromMsBuild', '-architecture=x64', '-NoUBA', '-MaxParallelActions=2', + '-WaitMutex', '-FromMsBuild', '-architecture=x64', '-NoUBA', '-MaxParallelActions=4', ('-Log="{0}"' -f $buildLog)) $BuildRecord = Invoke-BuildProcess $buildArgs $buildLog $buildError $buildStdout if ($BuildRecord.status -ne 'passed') { Add-Failure "staged build $($BuildRecord.status)" } @@ -1017,6 +1254,94 @@ try { (Join-Path $RunRoot 'Logs\probe.log') (Join-Path $RunRoot 'Logs\probe.stdout.log') (Join-Path $RunRoot 'Logs\probe.stderr.log') $CommandletMetrics = Get-CommandletMetrics $commandletOut 'owner probe' } + 'horizon' { + $tracePath = Join-Path $RunRoot 'horizon_trace.jsonl' + $baseCvars['voxel.TestRenderDistanceChunks'] = $HorizonRenderDistanceChunks + $baseCvars['voxel.TestMaxClipLevel'] = $HorizonLevel + $baseCvars['voxel.TestFarSheetRing'] = 0 + $baseCvars['voxel.StartupTraceFile'] = $tracePath + $baseCvars['voxel.StartupTraceMaxTiles'] = 16384 + $baseCvars['voxel.TestExitOnSteady'] = 1 + $baseCvars['voxel.OuterClassifierMode'] = 0 + $args = New-CommonArguments 'game' (Join-Path $RunRoot 'Logs\horizon.log') (Merge-Cvars $baseCvars) + $gameLedger = Invoke-UnrealLaunch "horizon-l$($HorizonLevel)-$($HorizonRenderDistanceChunks)-game" $args ` + (Join-Path $RunRoot 'Logs\horizon.log') (Join-Path $RunRoot 'Logs\horizon.stdout.log') (Join-Path $RunRoot 'Logs\horizon.stderr.log') + $GameMetrics = Get-StreamingMetrics (Join-Path $RunRoot 'Logs\horizon.log') + $HorizonMetrics = Get-HorizonMetrics $tracePath $gameLedger $GameMetrics + if ($HorizonMetrics.status -ne 'ok') { + Add-Failure "horizon measurement trace did not settle: $($HorizonMetrics.status)" + } + } + 'horizon-render' { + $tracePath = Join-Path $RunRoot 'horizon_render_trace.jsonl' + $baseCvars['voxel.TestRenderDistanceChunks'] = $HorizonRenderDistanceChunks + $baseCvars['voxel.TestMaxClipLevel'] = $HorizonLevel + $baseCvars['voxel.TestFarSheetRing'] = 0 + $baseCvars['voxel.StartupTraceFile'] = $tracePath + $baseCvars['voxel.StartupTraceMaxTiles'] = 16384 + $baseCvars['voxel.TestCeilingView'] = 1 + $baseCvars['voxel.TestCeilingCaptureDelaySeconds'] = 90 + $baseCvars['voxel.TestExitSeconds'] = 180 + $baseCvars['voxel.OuterClassifierMode'] = 0 + $renderLog = Join-Path $RunRoot 'Logs\horizon-render.log' + $args = New-CommonArguments 'game' $renderLog (Merge-Cvars $baseCvars) $true + $gameLedger = Invoke-UnrealLaunch "horizon-l$($HorizonLevel)-$($HorizonRenderDistanceChunks)-render" $args ` + $renderLog (Join-Path $RunRoot 'Logs\horizon-render.stdout.log') (Join-Path $RunRoot 'Logs\horizon-render.stderr.log') + $GameMetrics = Get-StreamingMetrics $renderLog + $HorizonMetrics = Get-HorizonMetrics $tracePath $gameLedger $GameMetrics + $HorizonRenderMetrics = Get-HorizonRenderMetrics $gameLedger $renderLog + if ($HorizonMetrics.status -ne 'ok') { + Add-Failure "horizon render trace did not settle: $($HorizonMetrics.status)" + } + if ($HorizonRenderMetrics.status -ne 'passed') { + Add-Failure "horizon render did not produce a capture: $($HorizonRenderMetrics.status)" + } + } + 'determinism' { + $defaultDump = Join-Path $RunRoot 'tile_hashes_default.json' + $crtFma3OffDump = Join-Path $RunRoot 'tile_hashes_crt_fma3_off.json' + $defaultTrace = Join-Path $RunRoot 'determinism_default_trace.jsonl' + $crtFma3OffTrace = Join-Path $RunRoot 'determinism_crt_fma3_off_trace.jsonl' + $determinismBase = [ordered]@{ + 'voxel.TestRenderDistanceChunks' = $HorizonRenderDistanceChunks + 'voxel.TestMaxClipLevel' = $HorizonLevel + 'voxel.TestFarSheetRing' = 0 + 'voxel.TestExitOnSteady' = 1 + 'voxel.StartupTraceMaxTiles' = 16384 + 'voxel.OuterClassifierMode' = 0 + } + $defaultCvars = [ordered]@{} + foreach ($key in $determinismBase.Keys) { $defaultCvars[$key] = $determinismBase[$key] } + $defaultCvars['voxel.StartupTraceFile'] = $defaultTrace + $defaultCvars['voxel.TileHashDump'] = $defaultDump + $defaultLog = Join-Path $RunRoot 'Logs\determinism-default.log' + $defaultArgs = New-CommonArguments 'game' $defaultLog (Merge-Cvars $defaultCvars) + $defaultLedger = Invoke-UnrealLaunch "determinism-default-l$($HorizonLevel)-$($HorizonRenderDistanceChunks)" $defaultArgs ` + $defaultLog (Join-Path $RunRoot 'Logs\determinism-default.stdout.log') (Join-Path $RunRoot 'Logs\determinism-default.stderr.log') + $defaultMetrics = Get-HorizonMetrics $defaultTrace $defaultLedger (Get-StreamingMetrics $defaultLog) + + $crtFma3OffCvars = [ordered]@{} + foreach ($key in $determinismBase.Keys) { $crtFma3OffCvars[$key] = $determinismBase[$key] } + $crtFma3OffCvars['voxel.StartupTraceFile'] = $crtFma3OffTrace + $crtFma3OffCvars['voxel.TileHashDump'] = $crtFma3OffDump + $crtFma3OffCvars['voxel.CrtFma3'] = 0 + $crtFma3OffLog = Join-Path $RunRoot 'Logs\determinism-crt-fma3-off.log' + $crtFma3OffArgs = New-CommonArguments 'game' $crtFma3OffLog (Merge-Cvars $crtFma3OffCvars) + $crtFma3OffLedger = Invoke-UnrealLaunch "determinism-crt-fma3-off-l$($HorizonLevel)-$($HorizonRenderDistanceChunks)" $crtFma3OffArgs ` + $crtFma3OffLog (Join-Path $RunRoot 'Logs\determinism-crt-fma3-off.stdout.log') (Join-Path $RunRoot 'Logs\determinism-crt-fma3-off.stderr.log') + $crtFma3OffMetrics = Get-HorizonMetrics $crtFma3OffTrace $crtFma3OffLedger (Get-StreamingMetrics $crtFma3OffLog) + + $DeterminismEvidence = Compare-DeterminismDumps $defaultDump $crtFma3OffDump + $DeterminismEvidence.level = $HorizonLevel + $DeterminismEvidence.render_distance_chunks = $HorizonRenderDistanceChunks + $DeterminismEvidence.far_sheet_ring = $false + $DeterminismEvidence.default_measurement = $defaultMetrics + $DeterminismEvidence.crt_fma3_off_measurement = $crtFma3OffMetrics + $GameMetrics = $crtFma3OffMetrics + if ($DeterminismEvidence.status -ne 'passed') { + Add-Failure "determinism comparison failed: $(Get-JsonValue $DeterminismEvidence 'reason' 'unknown difference')" + } + } 'perf' { $baseCvars['voxel.TestExitSeconds'] = 15 $baseCvars['voxel.ProfileTileGeneration'] = 1 @@ -1201,6 +1526,7 @@ $result = [ordered]@{ scenario = $Scenario label = $Label run_directory = $RunRoot + ddc_path = $DDCPath project = $ProjectFile staged_host = $HostProject assets_mode = $Assets @@ -1220,6 +1546,9 @@ $result = [ordered]@{ probe_amplitudes = if ($CommandletMetrics.Contains('probe_amplitudes')) { $CommandletMetrics.probe_amplitudes } else { [ordered]@{} } commandlet = $CommandletMetrics game = $GameMetrics + horizon = $HorizonMetrics + horizon_render = $HorizonRenderMetrics + determinism = $DeterminismEvidence surface_fall = $SurfaceFallMetrics strate_crossing = $StrateCrossingMetrics collision_gate_stress = $CollisionGateStressMetrics @@ -1246,6 +1575,7 @@ try { [void]$summaryLines.Add("$(if ($pass) { 'PASS' } else { 'FAIL' }) $reason") [void]$summaryLines.Add("scenario=$Scenario label=$Label") [void]$summaryLines.Add("run_directory=$RunRoot") + [void]$summaryLines.Add("ddc_path=$DDCPath") [void]$summaryLines.Add("assets=$Assets") [void]$summaryLines.Add("canonical_baseline=$(ConvertTo-CompactJson $result.canonical_baseline)") [void]$summaryLines.Add("capability_counts=$(ConvertTo-CompactJson $result.capability_counts)") @@ -1257,6 +1587,9 @@ try { [void]$summaryLines.Add("collision_gate_stress=$(ConvertTo-CompactJson $result.collision_gate_stress)") [void]$summaryLines.Add("probe_amplitudes=$(ConvertTo-CompactJson $result.probe_amplitudes)") [void]$summaryLines.Add("worker_seconds=$(ConvertTo-CompactJson $result.worker_seconds)") + [void]$summaryLines.Add("horizon=$(ConvertTo-CompactJson $result.horizon)") + [void]$summaryLines.Add("horizon_render=$(ConvertTo-CompactJson $result.horizon_render)") + [void]$summaryLines.Add("determinism=$(ConvertTo-CompactJson $result.determinism)") [void]$summaryLines.Add("p50_p95=$(ConvertTo-CompactJson $result.p50_p95)") [void]$summaryLines.Add("tests=$(ConvertTo-CompactJson $result.tests)") [void]$summaryLines.Add("parity=$(ConvertTo-CompactJson $result.parity)")