diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index f453b62..c6dd4fe 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1648,6 +1648,37 @@ FloatingIslands checks passed; the S3/S4 ground, cap and coverage audit passed. negative control, hashes, stage ledger and the deferred LOD8 optimization plan are in [`Standalone/Research/WorldAssemblerM3/Round13/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round13/REPORT.md). +### 8.31 LOD8 planar far-horizon sampling — World Assembler S10 / M3 Round 14 + +The planar far horizon is already built from a `Heightfield2D` column profile, but LOD8 previously +refined ground and cap at a 128-voxel grid even though a 25 cm LOD8 marching-cubes cell spans 512 +voxels (128 m). `BuildNativeHorizonGridCoordinates` now keeps the globally aligned 128-voxel edge +strips and uses the marching-cubes step through the LOD8 interior. The boundary strips preserve +shared edge samples; the interior does not build detail smaller than the LOD8 cell spacing. LOD4–7 +retain their prior sample coordinates. No agreement tolerance was relaxed. + +This applies to native horizon plans that declare a planar heightfield. Ground and cap still run in +separate refinement passes, using the same deterministic coordinate lattice. `Heightfield2D` keeps +its thread-local column cache: S10 records remaining duplicate evaluations across LOD8 tiles and +workers but does not share cache pages between workers. The `--far-horizon-lod8-step=128` Standalone +override restores the old sampling density for an A/B control; it does not restore the old v2 hash +salt. The default sampling change bumps the far recipe hash version from 2 to 3. LOD0–3 hashes and +density are unchanged. + +`VoxelForgeSharedFeatureCost` now reports column evaluation, marching-cubes, ground-refine, +cap-refine and output-assembly timings separately. Column evaluation is subtracted from each +overlapping MC/refinement wall interval; output assembly includes the final payload flattening and +optional diagnostic hashing. Percentiles are computed independently and are not additive. The +optional LOD8 XY capture uses worker-local vectors and records only diagnostics; it does not affect +mesh generation or recipe identity. + +On the staged 25 cm SurfaceWorld plan, the warmed LOD8 full-tile p95 fell from 59.848 ms to +13.933 ms. Column evaluation fell from 31.319 ms to 7.478 ms; ground and cap refinement fell from +11.666/10.029 ms to 1.690/1.330 ms. The S3 ground agreement, S4 cap agreement and 100% cap-ray +coverage gates still pass at their existing tolerances. BuildHost actor startup, the W1/W8/W16 +matrix, W1/W8/W12 Standalone hashes, the 40 m/s support-ring trace and the S10/S9 captures are +recorded in [`Standalone/Research/WorldAssemblerM3/Round14/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round14/REPORT.md). + ## 9. Multiplayer model (listen-server first, dedicated-friendly) > **Status: transport is DESIGN ONLY — nothing is networked in-tree yet** (no `Replicated`/`HasAuthority`/RPCs; a diff --git a/CODEMAP.md b/CODEMAP.md index 13c5e3b..23b635d 100644 --- a/CODEMAP.md +++ b/CODEMAP.md @@ -1053,6 +1053,18 @@ missing desired keys. `VoxelForgeExploreCommandlet.cpp` and `Tools/TestVoxelForg accept a movement vector so the same trace covers horizontal and vertical travel. See ARCHITECTURE §8.30 and [`Standalone/Research/WorldAssemblerM3/Round13/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round13/REPORT.md). +### 3.17 LOD8 planar far-horizon sampling — World Assembler S10 / M3 Round 14 + +Native planar SurfaceWorld horizon refinement uses the LOD8 marching-cubes step for its interior +grid (512 voxels / 128 m at 25 cm), retaining globally aligned 128-voxel boundary strips for tile +and LOD joins. LOD4–LOD7 keep their previous grids. Far recipe version 3 salts changed LOD4–LOD8 +mesh outputs; density and LOD0–LOD3 output are unchanged. The ground and cap passes use the same +coordinate lattice but still sample separately. The per-thread column cache remains local to each +worker; no cross-worker page cache was added. Diagnostics report exclusive column, MC, ground, +cap and output-assembly costs, plus opt-in duplicated LOD8 XY evaluations. The Standalone-only +`--far-horizon-lod8-step=128` restores S9 sampling for the negative control. See ARCHITECTURE §8.31 +and [`Standalone/Research/WorldAssemblerM3/Round14/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round14/REPORT.md). + ## 4. The density pipeline (most-edited hot path) ### 4.1 `GetDensityWithParams` (TunnelNetwork) — VoxelGenerator.cpp:3749 diff --git a/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp b/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp index 63e0d5f..3a5aa9b 100644 --- a/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp +++ b/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp @@ -15945,6 +15945,8 @@ bool RunCoreSharedFeatureCostAudit(VoxelForgeCore::World& World, TArray FullTimes,CandidateTimes,FullShellTimes,CandidateShellTimes,ListTimes; TArray UnknownBoxes,FieldCalls,ColumnMisses,StratumShellTimes,ConnectorShellTimes; TArray ClassificationTimes,ColumnComputeTimes,MeshingTimes,HorizonTimes,RepairSkipped; + TArray ColumnEvalExclusiveTimes,MarchingExclusiveTimes,GroundRefineExclusiveTimes; + TArray CapRefineExclusiveTimes,MeshAssemblyTimes; TArray StratumFieldCalls,ConnectorFieldCalls,ShellTriangles,GroundTriangles,GroundRemoved,CapTriangles; for (const auto& Key : Keys) { @@ -15978,6 +15980,11 @@ bool RunCoreSharedFeatureCostAudit(VoxelForgeCore::World& World, ColumnMisses.Add(static_cast(Full->diagnostics.columnCacheMisses)); ClassificationTimes.Add(Full->diagnostics.classificationMilliseconds); ColumnComputeTimes.Add(Full->diagnostics.columnComputeMilliseconds); + ColumnEvalExclusiveTimes.Add(Full->diagnostics.columnEvaluationExclusiveMilliseconds); + MarchingExclusiveTimes.Add(Full->diagnostics.marchingCubesExclusiveMilliseconds); + GroundRefineExclusiveTimes.Add(Full->diagnostics.groundRefinementExclusiveMilliseconds); + CapRefineExclusiveTimes.Add(Full->diagnostics.capRefinementExclusiveMilliseconds); + MeshAssemblyTimes.Add(Full->diagnostics.meshAssemblyMilliseconds); MeshingTimes.Add(Full->diagnostics.meshingMilliseconds); HorizonTimes.Add(Full->diagnostics.farHorizonMilliseconds); RepairSkipped.Add(Full->diagnostics.farShellRepairSkipped?1.0:0.0); @@ -15994,9 +16001,11 @@ bool RunCoreSharedFeatureCostAudit(VoxelForgeCore::World& World, { Values.Sort(); return Values.IsEmpty() ? 0.0 : Values[FMath::Clamp( FMath::CeilToInt(Values.Num()*0.95f)-1,0,Values.Num()-1)]; }; UE_LOG(LogTemp,Display, - TEXT("[VoxelForgeSharedFeatureCost] label=%s lod=%d tiles=%d hashes_identical=1 full_p95_ms=%.5f candidate_p95_ms=%.5f list_p95_ms=%.5f classification_p95_ms=%.5f column_compute_p95_ms=%.5f meshing_p95_ms=%.5f horizon_p95_ms=%.5f full_shell_p95_ms=%.5f candidate_shell_p95_ms=%.5f repair_skipped_p95=%.0f unknown_boxes_p95=%.0f field_calls_p95=%.0f column_misses_p95=%.0f stratum_shell_p95_ms=%.5f stratum_field_calls_p95=%.0f connector_shell_p95_ms=%.5f connector_field_calls_p95=%.0f shell_triangles_p95=%.0f horizon_ground_triangles_p95=%.0f horizon_ground_removed_p95=%.0f horizon_cap_triangles_p95=%.0f"), + TEXT("[VoxelForgeSharedFeatureCost] label=%s lod=%d tiles=%d hashes_identical=1 full_p95_ms=%.5f candidate_p95_ms=%.5f list_p95_ms=%.5f classification_p95_ms=%.5f column_compute_p95_ms=%.5f column_eval_exclusive_p95_ms=%.5f mc_exclusive_p95_ms=%.5f ground_refine_exclusive_p95_ms=%.5f cap_refine_exclusive_p95_ms=%.5f mesh_assembly_p95_ms=%.5f meshing_p95_ms=%.5f horizon_p95_ms=%.5f full_shell_p95_ms=%.5f candidate_shell_p95_ms=%.5f repair_skipped_p95=%.0f unknown_boxes_p95=%.0f field_calls_p95=%.0f column_misses_p95=%.0f stratum_shell_p95_ms=%.5f stratum_field_calls_p95=%.0f connector_shell_p95_ms=%.5f connector_field_calls_p95=%.0f shell_triangles_p95=%.0f horizon_ground_triangles_p95=%.0f horizon_ground_removed_p95=%.0f horizon_cap_triangles_p95=%.0f"), Label,LOD,FullTimes.Num(),P95(FullTimes),P95(CandidateTimes),P95(ListTimes), - P95(ClassificationTimes),P95(ColumnComputeTimes),P95(MeshingTimes),P95(HorizonTimes), + P95(ClassificationTimes),P95(ColumnComputeTimes),P95(ColumnEvalExclusiveTimes), + P95(MarchingExclusiveTimes),P95(GroundRefineExclusiveTimes),P95(CapRefineExclusiveTimes), + P95(MeshAssemblyTimes),P95(MeshingTimes),P95(HorizonTimes), P95(FullShellTimes),P95(CandidateShellTimes),P95(RepairSkipped),P95(UnknownBoxes),P95(FieldCalls), P95(ColumnMisses),P95(StratumShellTimes),P95(StratumFieldCalls), P95(ConnectorShellTimes),P95(ConnectorFieldCalls),P95(ShellTriangles), diff --git a/Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h b/Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h index 9e1accf..3bc6f02 100644 --- a/Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h +++ b/Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h @@ -125,6 +125,11 @@ namespace VoxelForgeCore std::uint64_t columnCacheMisses = 0; double classificationMilliseconds = 0.0; double columnComputeMilliseconds = 0.0; + double columnEvaluationExclusiveMilliseconds = 0.0; + double marchingCubesExclusiveMilliseconds = 0.0; + double groundRefinementExclusiveMilliseconds = 0.0; + double capRefinementExclusiveMilliseconds = 0.0; + double meshAssemblyMilliseconds = 0.0; double meshingMilliseconds = 0.0; double featureListMilliseconds = 0.0; double farShellMilliseconds = 0.0; @@ -146,6 +151,9 @@ namespace VoxelForgeCore std::uint64_t capRefinedCells = 0; std::uint64_t capRefinedTriangles = 0; bool capRefined = false; + std::uint64_t columnEvaluationWorkerHash = 0; + bool columnCoordinateCaptureComplete = true; + std::vector> evaluatedColumnCoordinates; }; struct TileMesh @@ -628,6 +636,9 @@ namespace VoxelForgeCore std::uint64_t columnHits = 0; std::uint64_t columnMisses = 0; double columnComputeMilliseconds = 0.0; + bool captureCoordinates = false; + bool coordinateCaptureComplete = true; + std::vector> evaluatedCoordinates; }; struct FieldInfluenceBounds @@ -676,8 +687,8 @@ namespace VoxelForgeCore std::int64_t minimumYSubvoxels, std::int64_t minimumZSubvoxels, std::int64_t maximumXSubvoxels, std::int64_t maximumYSubvoxels, std::int64_t maximumZSubvoxels, std::int32_t stepVoxels) const noexcept; - static void resetThreadColumnCacheStats() noexcept; - static HeightfieldColumnCacheStats threadColumnCacheStats() noexcept; + static void resetThreadColumnCacheStats(bool captureCoordinates = false) noexcept; + static const HeightfieldColumnCacheStats& threadColumnCacheStats() noexcept; FieldInterval interval(std::int64_t minimumXSubvoxels, std::int64_t minimumYSubvoxels, std::int64_t minimumZSubvoxels, @@ -1002,11 +1013,17 @@ namespace VoxelForgeCore // Validation control only; never participates in recipe/world identity. // Controle de validation seulement, sans effet sur l'identite du monde. bool useAssemblyFeatureQueries = true; + // Opt-in diagnostic only; records XY columns evaluated by this LOD8 tile. + // Diagnostic seulement; enregistre les colonnes XY evaluees par cette tuile LOD8. + bool captureColumnCoordinates = false; + // LOD8-only planar horizon interior grid; zero follows the MC step, 128 restores S9. + // Grille interieure LOD8 seulement; zero suit la maille MC, 128 restaure S9. + std::int32_t lod8HorizonInteriorStepVoxels = 0; // Bump when the distant cap representation changes. LOD4-8 mesh hashes are salted; // density and LOD0-3 hashes remain unchanged. // Incremente quand le plafond lointain change. Les hashes LOD4-8 incluent cette version; // densite et hashes LOD0-3 restent inchanges. - std::uint32_t farHorizonVersion = 2; + std::uint32_t farHorizonVersion = 3; }; class VFCORE_API World diff --git a/Standalone/Core/src/VoxelForgeCore.cpp b/Standalone/Core/src/VoxelForgeCore.cpp index 60f9c51..505f032 100644 --- a/Standalone/Core/src/VoxelForgeCore.cpp +++ b/Standalone/Core/src/VoxelForgeCore.cpp @@ -11,6 +11,8 @@ #include #include #include +#include +#include namespace vfcore::detail { @@ -247,6 +249,45 @@ namespace VoxelForgeCore bool Refined = false; }; + std::vector BuildNativeHorizonGridCoordinates(int32 TileExtent,int32 InteriorStep) + { + constexpr int32 BoundaryStep=128; + std::vector Coordinates; + if(TileExtent<=0)return Coordinates; + if(InteriorStep<=BoundaryStep) + { + Coordinates.reserve(static_cast(TileExtent/InteriorStep)+1u); + for(int32 Position=0;Position(TileExtent/BoundaryStep)+1u); + for(int32 Position=0;Position(GridStepVoxels)*Scale; const FIntVector TileOrigin=Key.OriginVoxels(); const int32 TileExtent=Key.ExtentVoxels(); + const int32 RequestedStep=Key.Level==8 + ?(InteriorStepOverride>0?InteriorStepOverride:MeshStepVoxels) + :std::min(BoundaryStepVoxels,1< GridCoordinates= + BuildNativeHorizonGridCoordinates(TileExtent,GridStepVoxels); + const int32 GridSize=static_cast(GridCoordinates.size()); + if(GridSize<2)return Stats; const double TileMinZ=static_cast(TileOrigin.Z)*Scale; const double TileMaxZ=static_cast(TileOrigin.Z+TileExtent)*Scale; const int32 OldSurfaceVertices=std::clamp(Mesh.NumSurfaceVertices,0,Mesh.Vertices.Num()); @@ -324,8 +376,9 @@ namespace VoxelForgeCore struct FGroundProfile { - int32 Cells=0,GridStep=0; + int32 GridStep=0; double MinimumZ=0.0,MaximumZ=0.0; + std::vector Coordinates; std::vector Ground,Cap; }; std::vector GroundProfiles; @@ -344,14 +397,14 @@ namespace VoxelForgeCore FColor Color=FColor::White; int32 VertexIndex=INDEX_NONE; }; - std::vector Grid(static_cast(Cells+1)*(Cells+1)); - const auto GridIndex=[Cells](int32 X,int32 Y) - {return static_cast(Y)*(Cells+1)+X;}; - for(int32 Y=0;Y<=Cells;++Y) - for(int32 X=0;X<=Cells;++X) + std::vector Grid(static_cast(GridSize)*GridSize); + const auto GridIndex=[GridSize](int32 X,int32 Y) + {return static_cast(Y)*GridSize+X;}; + for(int32 Y=0;Y(TileOrigin.X)+static_cast(X)*GridStepVoxels)*Scale; - const int64 YSub=(static_cast(TileOrigin.Y)+static_cast(Y)*GridStepVoxels)*Scale; + const int64 XSub=(static_cast(TileOrigin.X)+GridCoordinates[X])*Scale; + const int64 YSub=(static_cast(TileOrigin.Y)+GridCoordinates[Y])*Scale; const auto Column=Block.point(XSub,YSub,0); auto& Sample=Grid[GridIndex(X,Y)]; Sample.Ground=static_cast(Column.groundHeightSubvoxels); @@ -364,13 +417,13 @@ namespace VoxelForgeCore static_cast(std::clamp(std::lround(Blend*255.0f),0l,255l)), static_cast(std::clamp(Neighbor,0,255))); } - for(int32 Y=0;Y<=Cells;++Y) - for(int32 X=0;X<=Cells;++X) + for(int32 Y=0;Y(Right-Left)*GridStepSubvoxels; - const double DY=static_cast(Up-Down)*GridStepSubvoxels; + const int32 Left=std::max(0,X-1),Right=std::min(GridSize-1,X+1); + const int32 Down=std::max(0,Y-1),Up=std::min(GridSize-1,Y+1); + const double DX=static_cast(GridCoordinates[Right]-GridCoordinates[Left])*Scale; + const double DY=static_cast(GridCoordinates[Up]-GridCoordinates[Down])*Scale; const double SlopeX=DX>0.0?(Grid[GridIndex(Right,Y)].Ground -Grid[GridIndex(Left,Y)].Ground)/DX:0.0; const double SlopeY=DY>0.0?(Grid[GridIndex(X,Up)].Ground @@ -382,7 +435,7 @@ namespace VoxelForgeCore } FGroundProfile Profile; - Profile.Cells=Cells;Profile.GridStep=GridStepVoxels; + Profile.GridStep=GridStepVoxels;Profile.Coordinates=GridCoordinates; Profile.MinimumZ=static_cast(std::min(Plan.bottomChunkZ,Plan.topChunkZ))*CHUNK_SIZE; Profile.MaximumZ=static_cast(std::max(Plan.bottomChunkZ,Plan.topChunkZ)+1)*CHUNK_SIZE; Profile.Ground.reserve(Grid.size());Profile.Cap.reserve(Grid.size()); @@ -392,25 +445,25 @@ namespace VoxelForgeCore // Partager les sommets du profil entre cellules reduit le payload d'horizon. // Share regular profile vertices across adjacent cells to bound horizon payload. - for(int32 Y=0;Y<=Cells;++Y) - for(int32 X=0;X<=Cells;++X) + for(int32 Y=0;Y=Sample.Cap||Sample.GroundTileMaxZ)continue; - const int64 XSub=(static_cast(TileOrigin.X)+static_cast(X)*GridStepVoxels)*Scale; - const int64 YSub=(static_cast(TileOrigin.Y)+static_cast(Y)*GridStepVoxels)*Scale; + const int64 XSub=(static_cast(TileOrigin.X)+GridCoordinates[X])*Scale; + const int64 YSub=(static_cast(TileOrigin.Y)+GridCoordinates[Y])*Scale; const double VoxelSizeCm=static_cast(Owner.CoreVoxelSizeMicrons)/10000.0; Sample.VertexIndex=static_cast(AddedVertices.size()); - AddedVertices.emplace_back(static_cast(TileOrigin.X+X*GridStepVoxels)*VoxelSizeCm, - static_cast(TileOrigin.Y+Y*GridStepVoxels)*VoxelSizeCm, + AddedVertices.emplace_back(static_cast(TileOrigin.X+GridCoordinates[X])*VoxelSizeCm, + static_cast(TileOrigin.Y+GridCoordinates[Y])*VoxelSizeCm, Sample.Ground/Scale*VoxelSizeCm); AddedNormals.push_back(Sample.Normal); if(bHasUVs)AddedUVs.emplace_back(static_cast(XSub/Scale),static_cast(YSub/Scale)); if(bHasColors)AddedColors.push_back(Sample.Color); } - for(int32 Y=0;Y(TileOrigin.X+GX*GridStepVoxels)*Scale; - PolygonA[Corner].YSub=static_cast(TileOrigin.Y+GY*GridStepVoxels)*Scale; + PolygonA[Corner].XSub=static_cast(TileOrigin.X+GridCoordinates[GX])*Scale; + PolygonA[Corner].YSub=static_cast(TileOrigin.Y+GridCoordinates[GY])*Scale; PolygonA[Corner].GroundSub=Sample.Ground; PolygonA[Corner].CapSub=Sample.Cap; PolygonA[Corner].ZSub=Sample.Ground; @@ -487,16 +540,29 @@ namespace VoxelForgeCore const auto InterpolateProfile=[](const std::vector& Values,const FGroundProfile& Profile, double LocalX,double LocalY) { - const double GridX=std::clamp(LocalX/Profile.GridStep,0.0,static_cast(Profile.Cells)); - const double GridY=std::clamp(LocalY/Profile.GridStep,0.0,static_cast(Profile.Cells)); - const int32 X=std::min(Profile.Cells-1,static_cast(std::floor(GridX))); - const int32 Y=std::min(Profile.Cells-1,static_cast(std::floor(GridY))); - const double TX=GridX-X,TY=GridY-Y; - const std::size_t Width=static_cast(Profile.Cells+1); + const std::size_t Width=Profile.Coordinates.size(); + const auto FindCell=[&](double Local) + { + const double Clamped=std::clamp(Local,static_cast(Profile.Coordinates.front()), + static_cast(Profile.Coordinates.back())); + const auto Upper=std::upper_bound(Profile.Coordinates.begin(),Profile.Coordinates.end(),Clamped); + if(Upper==Profile.Coordinates.begin())return std::pair{0u,0.0}; + const std::size_t Index=std::min(Width-2u, + static_cast(Upper-Profile.Coordinates.begin()-1)); + const double Start=Profile.Coordinates[Index]; + const double Span=Profile.Coordinates[Index+1]-Profile.Coordinates[Index]; + return std::pair{Index,Span>0.0?(Clamped-Start)/Span:0.0}; + }; + const auto [X,TX]=FindCell(LocalX); + const auto [Y,TY]=FindCell(LocalY); const auto At=[&](int32 PX,int32 PY){return Values[static_cast(PY)*Width+PX];}; if(TX>=TY) - return At(X,Y)*(1.0-TX)+At(X+1,Y)*(TX-TY)+At(X+1,Y+1)*TY; - return At(X,Y)*(1.0-TY)+At(X+1,Y+1)*TX+At(X,Y+1)*(TY-TX); + return At(static_cast(X),static_cast(Y))*(1.0-TX) + +At(static_cast(X+1u),static_cast(Y))*(TX-TY) + +At(static_cast(X+1u),static_cast(Y+1u))*TY; + return At(static_cast(X),static_cast(Y))*(1.0-TY) + +At(static_cast(X+1u),static_cast(Y+1u))*TX + +At(static_cast(X),static_cast(Y+1u))*(TY-TX); }; for(int32 Triangle=0;TriangleTileExtent||LocalY>TileExtent)continue; for(const auto& Profile:GroundProfiles) { - if(LocalX>Profile.Cells*Profile.GridStep||LocalY>Profile.Cells*Profile.GridStep)continue; + if(LocalX>Profile.Coordinates.back()||LocalY>Profile.Coordinates.back())continue; const double Ground=InterpolateProfile(Profile.Ground,Profile,LocalX,LocalY); const double Cap=InterpolateProfile(Profile.Cap,Profile,LocalX,LocalY); if(Cap-Ground<=1.0||GroundTileMaxZ/Scale ||GroundProfile.MaximumZ)continue; - const double ReplacementBand=std::max(32.0,2.0*Profile.GridStep); + const double ReplacementBand=std::max(32.0, + 2.0*std::min(Profile.GridStep,BoundaryStepVoxels)); if(Z>=Ground-ReplacementBand) { RemoveOldGround[static_cast(Triangle)]=1; @@ -574,17 +641,28 @@ namespace VoxelForgeCore } FNativeCapRefinementStats RefineFarNativeCap(const FStandaloneOwnerWorld& Owner, - const WorldAssemblyManifest& Manifest,const FVoxelTileKey& Key,FVoxelMeshData& Mesh) + const WorldAssemblyManifest& Manifest,const FVoxelTileKey& Key,int32 MeshStepVoxels, + int32 InteriorStepOverride,FVoxelMeshData& Mesh) { FNativeCapRefinementStats Stats; if(Key.Level<4||!Owner.Generator)return Stats; - const int32 GridStepVoxels=Key.Level<5?std::min(128,1<(GridStepVoxels)*Scale; + constexpr int32 BoundaryStepVoxels=128; + const int32 RequestedStep=Key.Level==8 + ?(InteriorStepOverride>0?InteriorStepOverride:MeshStepVoxels) + :(Key.Level<5?std::min(BoundaryStepVoxels,1< GridCoordinates= + BuildNativeHorizonGridCoordinates(TileExtent,EffectiveGridStep); + const int32 GridSize=static_cast(GridCoordinates.size()); + if(GridSize<2)return Stats; const int64 TileMinZ=static_cast(TileOrigin.Z)*Scale; const int64 TileMaxZ=static_cast(TileOrigin.Z+TileExtent)*Scale; const int32 OldSurfaceVertices=std::clamp(Mesh.NumSurfaceVertices,0,Mesh.Vertices.Num()); @@ -612,14 +690,14 @@ namespace VoxelForgeCore FColor Color=FColor::White; int32 VertexIndex=INDEX_NONE; }; - std::vector Grid(static_cast(Cells+1)*(Cells+1)); - const auto GridIndex=[Cells](int32 X,int32 Y) - {return static_cast(Y)*(Cells+1)+X;}; - for(int32 Y=0;Y<=Cells;++Y) - for(int32 X=0;X<=Cells;++X) + std::vector Grid(static_cast(GridSize)*GridSize); + const auto GridIndex=[GridSize](int32 X,int32 Y) + {return static_cast(Y)*GridSize+X;}; + for(int32 Y=0;Y(TileOrigin.X)+static_cast(X)*GridStepVoxels)*Scale; - const int64 YSub=(static_cast(TileOrigin.Y)+static_cast(Y)*GridStepVoxels)*Scale; + const int64 XSub=(static_cast(TileOrigin.X)+GridCoordinates[X])*Scale; + const int64 YSub=(static_cast(TileOrigin.Y)+GridCoordinates[Y])*Scale; const auto Column=Block.point(XSub,YSub,0); auto& Sample=Grid[GridIndex(X,Y)]; Sample.Ground=static_cast(Column.groundHeightSubvoxels); @@ -633,13 +711,13 @@ namespace VoxelForgeCore const uint8 B=static_cast(std::clamp(std::lround(Blend*255.0f),0l,255l)); Sample.Color=FColor(R,0,B,A); } - for(int32 Y=0;Y<=Cells;++Y) - for(int32 X=0;X<=Cells;++X) + for(int32 Y=0;Y(Right-Left)*GridStepSubvoxels; - const double DY=static_cast(Up-Down)*GridStepSubvoxels; + const int32 Left=std::max(0,X-1),Right=std::min(GridSize-1,X+1); + const int32 Down=std::max(0,Y-1),Up=std::min(GridSize-1,Y+1); + const double DX=static_cast(GridCoordinates[Right]-GridCoordinates[Left])*Scale; + const double DY=static_cast(GridCoordinates[Up]-GridCoordinates[Down])*Scale; const double SlopeX=DX>0.0?(Grid[GridIndex(Right,Y)].Cap-Grid[GridIndex(Left,Y)].Cap)/DX:0.0; const double SlopeY=DY>0.0?(Grid[GridIndex(X,Up)].Cap-Grid[GridIndex(X,Down)].Cap)/DY:0.0; auto& Sample=Grid[GridIndex(X,Y)]; @@ -655,10 +733,10 @@ namespace VoxelForgeCore if(Sample.CapTileMaxZ)continue; Sample.VertexIndex=static_cast(AddedVertices.size()); const std::size_t Index=static_cast(&Sample-Grid.data()); - const int32 GX=static_cast(Index%static_cast(Cells+1)); - const int32 GY=static_cast(Index/static_cast(Cells+1)); - const double XSub=static_cast(TileOrigin.X+GX*GridStepVoxels)*Scale; - const double YSub=static_cast(TileOrigin.Y+GY*GridStepVoxels)*Scale; + const int32 GX=static_cast(Index%static_cast(GridSize)); + const int32 GY=static_cast(Index/static_cast(GridSize)); + const double XSub=static_cast(TileOrigin.X+GridCoordinates[GX])*Scale; + const double YSub=static_cast(TileOrigin.Y+GridCoordinates[GY])*Scale; const double VoxelSizeCm=static_cast(Owner.CoreVoxelSizeMicrons)/10000.0; AddedVertices.emplace_back(XSub/Scale*VoxelSizeCm,YSub/Scale*VoxelSizeCm, Sample.Cap/Scale*VoxelSizeCm); @@ -668,8 +746,8 @@ namespace VoxelForgeCore if(bHasColors)AddedColors.push_back(Sample.Color); } - for(int32 Y=0;Y(TileOrigin.X+GX*GridStepVoxels)*Scale; - PolygonA[Corner].YSub=static_cast(TileOrigin.Y+GY*GridStepVoxels)*Scale; + PolygonA[Corner].XSub=static_cast(TileOrigin.X+GridCoordinates[GX])*Scale; + PolygonA[Corner].YSub=static_cast(TileOrigin.Y+GridCoordinates[GY])*Scale; PolygonA[Corner].ZSub=Sample.Cap;PolygonA[Corner].GroundSub=Sample.Ground; PolygonA[Corner].Normal=Sample.Normal;PolygonA[Corner].Color=Sample.Color; } @@ -1400,7 +1478,8 @@ namespace VoxelForgeCore &&bHasNativeSurfaceHorizonPlan&&!bHasIslandPlanInTile; const bool bSkipFarShellRepair=key.lod>=7&&bUseNativeSurfaceHorizon; FStandaloneShapeWorld::ResetThreadClassificationStats(); - VoxelForgeCore::Heightfield2D::resetThreadColumnCacheStats(); + VoxelForgeCore::Heightfield2D::resetThreadColumnCacheStats( + options.captureColumnCoordinates&&key.lod==8); bool bNativeSurfaceTile=false; bool bNativeSurfaceBandSafe=false; if(bHasWorldAssembly) @@ -1427,7 +1506,7 @@ namespace VoxelForgeCore { out.diagnostics.meshingSkipped=true; const auto Counts=FStandaloneShapeWorld::GetThreadClassificationStats(); - const auto Columns=VoxelForgeCore::Heightfield2D::threadColumnCacheStats(); + const auto& Columns=VoxelForgeCore::Heightfield2D::threadColumnCacheStats(); out.diagnostics.solidBoxes=Counts.SolidBoxes; out.diagnostics.airBoxes=Counts.AirBoxes; out.diagnostics.mixedBoxes=Counts.MixedBoxes; @@ -1571,9 +1650,18 @@ namespace VoxelForgeCore out.diagnostics.featureListMilliseconds=std::chrono::duration( std::chrono::steady_clock::now()-FeatureListStart).count(); const ScopedWorldAssemblyFeatureQueriesV1 featureScope(featureQueries); + const double ColumnMillisecondsBeforeMeshing= + VoxelForgeCore::Heightfield2D::threadColumnCacheStats().columnComputeMilliseconds; + const auto MarchingCubesStart=std::chrono::steady_clock::now(); FVoxelMeshData mesh=mesher.GenerateMesh(origin,step,cells,nullptr, MeshBandMinZVoxel,MeshBandMaxZVoxel,&sampleCount,nullptr,nullptr, (options.hashDensity||options.captureDensitySamples)?&densityValues:nullptr); + const double MarchingCubesWallMilliseconds=std::chrono::duration( + std::chrono::steady_clock::now()-MarchingCubesStart).count(); + const double ColumnMillisecondsAfterMeshing= + VoxelForgeCore::Heightfield2D::threadColumnCacheStats().columnComputeMilliseconds; + out.diagnostics.marchingCubesExclusiveMilliseconds=std::max(0.0, + MarchingCubesWallMilliseconds-(ColumnMillisecondsAfterMeshing-ColumnMillisecondsBeforeMeshing)); const double MeshingMilliseconds=std::chrono::duration( std::chrono::steady_clock::now()-MeshingStart).count(); out.diagnostics.farHorizonVersion=key.lod>=4?options.farHorizonVersion:0; @@ -1612,9 +1700,29 @@ namespace VoxelForgeCore { const auto HorizonStart=std::chrono::steady_clock::now(); FNativeCapRefinementStats GroundStats; + const double ColumnMillisecondsBeforeGround= + VoxelForgeCore::Heightfield2D::threadColumnCacheStats().columnComputeMilliseconds; + const auto GroundStart=std::chrono::steady_clock::now(); if(key.lod>=4) - GroundStats=RefineFarNativeGround(world,*impl_->worldAssembly,nativeKey,mesh); - const auto CapStats=RefineFarNativeCap(world,*impl_->worldAssembly,nativeKey,mesh); + GroundStats=RefineFarNativeGround(world,*impl_->worldAssembly,nativeKey,step, + options.lod8HorizonInteriorStepVoxels,mesh); + const double GroundWallMilliseconds=std::chrono::duration( + std::chrono::steady_clock::now()-GroundStart).count(); + const double ColumnMillisecondsAfterGround= + VoxelForgeCore::Heightfield2D::threadColumnCacheStats().columnComputeMilliseconds; + out.diagnostics.groundRefinementExclusiveMilliseconds=std::max(0.0, + GroundWallMilliseconds-(ColumnMillisecondsAfterGround-ColumnMillisecondsBeforeGround)); + const double ColumnMillisecondsBeforeCap= + VoxelForgeCore::Heightfield2D::threadColumnCacheStats().columnComputeMilliseconds; + const auto CapStart=std::chrono::steady_clock::now(); + const auto CapStats=RefineFarNativeCap(world,*impl_->worldAssembly,nativeKey,step, + options.lod8HorizonInteriorStepVoxels,mesh); + const double CapWallMilliseconds=std::chrono::duration( + std::chrono::steady_clock::now()-CapStart).count(); + const double ColumnMillisecondsAfterCap= + VoxelForgeCore::Heightfield2D::threadColumnCacheStats().columnComputeMilliseconds; + out.diagnostics.capRefinementExclusiveMilliseconds=std::max(0.0, + CapWallMilliseconds-(ColumnMillisecondsAfterCap-ColumnMillisecondsBeforeCap)); out.diagnostics.farHorizonMilliseconds=std::chrono::duration( std::chrono::steady_clock::now()-HorizonStart).count(); out.diagnostics.capRefined=CapStats.Refined; @@ -1629,7 +1737,7 @@ namespace VoxelForgeCore if(bHasWorldAssembly) { const auto Counts=FStandaloneShapeWorld::GetThreadClassificationStats(); - const auto Columns=VoxelForgeCore::Heightfield2D::threadColumnCacheStats(); + const auto& Columns=VoxelForgeCore::Heightfield2D::threadColumnCacheStats(); out.diagnostics.fieldCalls=Columns.pointCalls; out.diagnostics.floatFieldCalls=Counts.NativeFloatFieldCalls; out.diagnostics.latticeFieldCalls=Counts.NativeLatticeFieldCalls; @@ -1637,7 +1745,15 @@ namespace VoxelForgeCore out.diagnostics.columnCacheHits=Columns.columnHits; out.diagnostics.columnCacheMisses=Columns.columnMisses; out.diagnostics.columnComputeMilliseconds=Columns.columnComputeMilliseconds; + out.diagnostics.columnEvaluationExclusiveMilliseconds=Columns.columnComputeMilliseconds; out.diagnostics.classificationMilliseconds=Counts.Milliseconds; + if(options.captureColumnCoordinates&&key.lod==8) + { + out.diagnostics.columnEvaluationWorkerHash=static_cast( + std::hash{}(std::this_thread::get_id()))+1u; + out.diagnostics.columnCoordinateCaptureComplete=Columns.coordinateCaptureComplete; + out.diagnostics.evaluatedColumnCoordinates=Columns.evaluatedCoordinates; + } if(bNativeSurfaceTile) { out.diagnostics.solidBoxes=Counts.SolidBoxes; @@ -1653,6 +1769,7 @@ namespace VoxelForgeCore } out.diagnostics.heightfieldBandLimited=bNativeHeightfieldBandLimited; out.diagnostics.meshingMilliseconds=MeshingMilliseconds; + const auto MeshAssemblyStart=std::chrono::steady_clock::now(); out.meshPayloadAllocatedBytes=static_cast(mesh.Vertices.GetAllocatedSize()) +static_cast(mesh.Normals.GetAllocatedSize()) +static_cast(mesh.UVs.GetAllocatedSize()) @@ -1696,6 +1813,8 @@ namespace VoxelForgeCore } out.hasDensityHash=out.hasMeshHash=out.hasPayloadHash=out.hasTopologyHash=true; } + out.diagnostics.meshAssemblyMilliseconds=std::chrono::duration( + std::chrono::steady_clock::now()-MeshAssemblyStart).count(); return out; } diff --git a/Standalone/Core/src/WorldAssemblerInternal.h b/Standalone/Core/src/WorldAssemblerInternal.h index 26be4f7..d76eeae 100644 --- a/Standalone/Core/src/WorldAssemblerInternal.h +++ b/Standalone/Core/src/WorldAssemblerInternal.h @@ -1487,6 +1487,17 @@ namespace world_assembly_detail HeightfieldColumnStats.columnComputeMilliseconds+=std::chrono::duration( std::chrono::steady_clock::now()-start).count(); ++HeightfieldColumnStats.columnMisses; + if(HeightfieldColumnStats.captureCoordinates&&HeightfieldColumnStats.coordinateCaptureComplete) + { + try + { + HeightfieldColumnStats.evaluatedCoordinates.push_back({x,y}); + } + catch(...) + { + HeightfieldColumnStats.coordinateCaptureComplete=false; + } + } entry={&recipe,&plan,recipe.contentHash,plan.contentHash,x,y,column}; return column; } @@ -1954,10 +1965,19 @@ HeightfieldPointSample Heightfield2D::point(std::int64_t x, std::int64_t y, return out; } -void Heightfield2D::resetThreadColumnCacheStats() noexcept -{ world_assembly_detail::HeightfieldColumnStats={}; } +void Heightfield2D::resetThreadColumnCacheStats(bool captureCoordinates) noexcept +{ + auto& Stats=world_assembly_detail::HeightfieldColumnStats; + Stats.pointCalls=0; + Stats.columnHits=0; + Stats.columnMisses=0; + Stats.columnComputeMilliseconds=0.0; + Stats.captureCoordinates=captureCoordinates; + Stats.coordinateCaptureComplete=true; + Stats.evaluatedCoordinates.clear(); +} -HeightfieldColumnCacheStats Heightfield2D::threadColumnCacheStats() noexcept +const HeightfieldColumnCacheStats& Heightfield2D::threadColumnCacheStats() noexcept { return world_assembly_detail::HeightfieldColumnStats; } HeightfieldBoxClass Heightfield2D::classifyLatticeBox(std::int64_t minimumX, diff --git a/Standalone/Research/WorldAssemblerM3/Round14/REPORT.md b/Standalone/Research/WorldAssemblerM3/Round14/REPORT.md new file mode 100644 index 0000000..16a6c55 --- /dev/null +++ b/Standalone/Research/WorldAssemblerM3/Round14/REPORT.md @@ -0,0 +1,213 @@ +# World assembler S10 / M3 Round 14 — cheaper far ring + +Date: 2026-10-19. Base: S9 / `e7586a1`. No commit, push or stash. + +## Result + +The planar far path now samples the LOD8 interior at the spacing that LOD8 marching cubes can +represent. At 25 cm, that is 512 voxels / 128 m per MC cell. Globally aligned 128-voxel (32 m) +strips remain along tile boundaries. LOD4–LOD7 coordinates and the 2 cm agreement tolerances are +unchanged. Far recipe version is 3; far mesh hashes change, while density and LOD0–LOD3 remain +unchanged. + +On the owner SurfaceWorld plan, the warmed eight-tile BuildHost LOD8 full-tile p95 fell 76.7%, +from 59.848 ms to 13.933 ms. In the fresh 6 m/s actor spawn trace, the 2,578 initial tile jobs +used 26.870 worker-seconds; the LOD8 share was 4.288 seconds. Round13’s S8 spawn reference was +about 41 worker-seconds, with 18.798 seconds in its 306 LOD8 jobs. Initial readiness was 7.117 s, +below the 7.7 s target. The active 3×3 walkable ring was ready in 0.490 s, and all 512 LOD0 tiles +were ready at 1.428 s. + +## Count first — exclusive cost and duplicate columns + +The counters are measured per tile. Column evaluation is timed only around uncached column work. +MC, ground and cap counters subtract column time from their respective wall intervals. The mesh +assembly window covers final payload flattening and optional diagnostic hashing. Each p95 is +computed independently, so these p95 values are not additive. + +| LOD8 warmed p95, BuildHost 25 cm | S9 | S10 | Change | +|---|---:|---:|---:| +| Full tile | 59.848 ms | 13.933 ms | −76.7% | +| Column evaluation | 31.319 ms | 7.478 ms | −76.1% | +| Marching cubes | 2.667 ms | 2.359 ms | −11.6% | +| Ground refinement | 11.666 ms | 1.690 ms | −85.5% | +| Cap refinement | 10.029 ms | 1.330 ms | −86.7% | +| Mesh assembly and diagnostic hash | 5.445 ms | 1.095 ms | −79.9% | +| Column-cache misses | 9,679 | 2,287 | −76.4% | + +The high-cost S9 sample generated 8,192 ground triangles, removed 531 coarse ground triangles, +and generated 6,732 cap triangles. S10 generated 968 ground triangles, removed 534, and generated +840 cap triangles. The geometric checks below pass at the same tolerance. + +The fresh W12 Standalone overlap capture counted 153 LOD8 tiles. S10 evaluated 106,219 unique XY +columns and repeated 32,738 neighboring tile evaluations, including 30,446 cross-worker repeats. +The same-binary step-128 control counted 660,283 unique columns, 44,319 neighboring repeats and +41,665 cross-worker repeats. This confirms that cross-worker duplicates remain, while the coarser +interior removes most column work without sharing cache state between workers. + +| Fresh W12 Standalone pipeline | Default step 0 | Negative control step 128 | +|---|---:|---:| +| LOD8 worker sum | 3.206 s | 13.534 s | +| LOD8 unique XY columns | 106,219 | 660,283 | +| Repeated neighboring evaluations | 32,738 | 44,319 | +| Cross-worker neighboring repeats | 30,446 | 41,665 | + +Both runs used the current v3 binary. Step 128 restores S9 sampling density, but retains the v3 +hash salt; its far mesh hash is therefore expected to differ from S9’s v2 hash. + +## BuildHost actor spawn and ring + +The fresh 6 m/s trace used the S9 movement, collision and backlog settings: Legacy order, Dynamic +near reserve, collision lookahead on, 256 result cap, and a 12-CPU affinity mask. Per-LOD worker +generation sums for the 2,578 initial generation ordinals were: + +| LOD | Initial tiles | Worker seconds | +|---:|---:|---:| +| 0 | 512 | 3.840 | +| 1 | 448 | 3.781 | +| 2 | 320 | 0.811 | +| 3 | 272 | 2.093 | +| 4 | 272 | 6.688 | +| 5 | 208 | 1.535 | +| 6 | 144 | 1.988 | +| 7 | 96 | 1.846 | +| 8 | 306 | 4.288 | +| **Total** | **2,578** | **26.870** | + +At 6 m/s, initial readiness was 7.116525 s (2,578 desired, 2,617 loaded); spawn-tick p95/max was +1.824/4.993 ms. The support ring became ready at 0.490 s (9/9). The complete LOD0 set was ready at +1.427630 s, with a 0.990836 s measured build interval. Round13’s S9 6 m/s initial-ready value was +7.813133 s. + +At 40 m/s (4,000 cm/s), all 75 moving support-center arrivals from 30.205 s through 45.011 s had +the full 9/9 walkable ring ready: 75/75, 100%. A stationary startup gate event before movement is +excluded from that moving-center denominator. Initial readiness was 7.087272 s. Spawn-tick +p95/max was 1.833/5.365 ms; the trace passed. + +## Quality and determinism + +- S3 ground agreement: 72 shared route points; far LOD7 and LOD8 maximum error 0.0427 cm, cross-LOD + error 0 cm, within 2 cm. The 100 cm datum-offset control measured 100.0427 cm and was detected. +- S4 cap agreement: 128 shared points; maximum error 0.0504 cm, within 2 cm. The 100 cm datum-offset + control measured 100.0015 cm and was detected. Cap census emitted all 385/385 requested bands. +- Cap coverage: 2,520/2,520 upward rays hit the cap (100%); no profile-roof, mesh or clip-edge misses. +- Core-off legacy passed at 12.5 cm. The M2 Hybrid V1 and V2 fixture sets each passed five analytic + fixtures and border checks. SurfaceWorld and FloatingIslands passed BuildHost actor validation at + 25 cm. Water stayed fixed; carve and fill remained unrestricted. +- Standalone W1/W8/W12 produced identical hashes for the full 2,553-tile view: density + `c10697a8f48380c7`, mesh `e75c9119a1498cd5`, payload `49a9a8d1835a6c8c`, topology + `edcb23288b5e8724`. +- BuildHost SurfaceWorld W1/W8/W16 matrix was stable: density `0cd2a1a4152ee4aa`, mesh + `a06243aee16b65d5`, payload `c700162935c262d7`, topology `0af901aa42b452bc`. FloatingIslands + W1/W8/W16 was stable: density `4dcfb37585d7d986`, mesh `f14f76fbf065c6ed`, payload + `53035c45fd1a082e`, topology `23550100654a4ec3`. W16 was a logical partition with the process + still capped to 12 executing CPUs. + +Far mesh hashes differ from S9 as expected under recipe version 3; current-build determinism passes. + +## Matched captures + +All captures are 640×400 and under 22 KB. Each was produced at 60 fps with VSync on, then the +Standalone viewer was closed after its back-buffer capture. Raw 1280×800 PNGs and run logs are under +`Saved/BuildHost/Round14/CaptureRaw/`. + +| View | S9 sampling control, 128 voxels | S10 default, MC spacing | +|---|---|---| +| Far horizon, camera 0,-35,-487.94,0,-22 | [s9_far_horizon.jpg](captures/s9_far_horizon.jpg) (21.9 KB) | [s10_far_horizon.jpg](captures/s10_far_horizon.jpg) (18.0 KB) | +| Cap, camera 0,-35,-487.94,0,22.42 | [s9_cap.jpg](captures/s9_cap.jpg) (10.8 KB) | [s10_cap.jpg](captures/s10_cap.jpg) (10.8 KB) | + +The cap is low contrast in this view, as in Round8; the 2 cm agreement, triangle census and ray +coverage provide the cap acceptance evidence. + +## Builds, commands and hashes + +Builds on the final source: + +~~~powershell +.\Tools\BuildVoxelForgeCoreHost.ps1 -MaxParallelActions 12 -TimeoutSeconds 900 +.\Standalone\Build.ps1 -Target All -CompileWorkers 12 -BuildPriority Normal +~~~ + +The fresh Standalone determinism processes used the same command with workers 1, 8 and 12: + +~~~powershell +foreach ($workers in @(1, 8, 12)) { + .\Standalone\bin\VoxelForgeCoreStandalone.exe --workers=$workers --world-assembler-prototype --stream-benchmark --stream-hash --force-generate --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --far-horizon-lod8-step=0 --process-priority=Normal --process-affinity=4095 --frame-limit=60 +} +~~~ + +The current-build duplicate count and same-binary step-128 control used: + +~~~powershell +.\Standalone\bin\VoxelForgeCoreStandalone.exe --workers=12 --world-assembler-prototype --stream-pipeline --stream-hash --profile-far-column-overlap --force-generate --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --far-horizon-lod8-step=0 --process-priority=Normal --process-affinity=4095 --frame-limit=60 +.\Standalone\bin\VoxelForgeCoreStandalone.exe --workers=12 --world-assembler-prototype --stream-pipeline --stream-hash --profile-far-column-overlap --force-generate --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --far-horizon-lod8-step=128 --process-priority=Normal --process-affinity=4095 --frame-limit=60 +~~~ + +Quality and BuildHost actor commands: + +~~~powershell +.\Standalone\bin\VoxelForgeCoreStandalone.exe --world-assembler-audit --workers=12 --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --process-priority=Normal --process-affinity=4095 --frame-limit=60 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -CoreVoxelSizeCm 25 -TimeoutSeconds 900 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -FloatingIslandsOnly -CoreVoxelSizeCm 25 -TimeoutSeconds 900 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -CoreOffTest -CoreVoxelSizeCm 12.5 -TimeoutSeconds 900 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -HybridFixturesOnly -CoreMesherRecipe WorldAssemblerHybridV1 -CoreVoxelSizeCm 25 -TimeoutSeconds 900 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -HybridFixturesOnly -CoreMesherRecipe WorldAssemblerHybridV2 -CoreVoxelSizeCm 25 -TimeoutSeconds 900 +~~~ + +The two fresh actor traces differed only in speed, 600 or 4,000 cm/s: + +~~~powershell +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StartupApplyBoost 0 -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -StreamingCollisionLookahead 1 -TestMoveDirection '0,1,0' -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 600 -TimeoutSeconds 900 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StartupApplyBoost 0 -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -StreamingCollisionLookahead 1 -TestMoveDirection '0,1,0' -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 4000 -TimeoutSeconds 900 +~~~ + +Final binary SHA-256: + +| Binary | SHA-256 | +|---|---| +| BuildHost UnrealEditor-VoxelForgeCore.dll | `D87CC56BACCDE6559705EEAEA7DB73D1A6ABF79484D37A3887B94A9FD62BABD3` | +| BuildHost UnrealEditor-VoxelForge.dll | `9D0522ECC9A6AC52D9F3ACC2FD44399FE70F557C5F11D0203AB4C4B8EE75C198` | +| BuildHost UnrealEditor-VoxelForgeEditor.dll | `AA196DA5A6DB3C84B5EC4F002AEBC787386DBB8F2798F5D1EEC03010BF642BBA` | +| Standalone VoxelForgeCoreStandalone.exe | `E316D7E3E0B3D3639DAE61D6E2D1091F38C59A210B3B7DD8124928D21A8119A5` | + +## Run notes, files and safety review + +- BuildHost and Standalone ran at Normal priority with no more than 12 executing CPUs; free memory + was above 17 GB before the final launches. BuildHost peak private memory was 3.76 GiB for + SurfaceWorld, 3.27 GiB for FloatingIslands, 3.39 GiB for the 6 m/s trace and 3.67 GiB for the + 40 m/s trace. The Standalone overlap/control processes peaked below 0.72 GiB. No process exceeded + 8 GiB, and no launch overlapped another Unreal/Standalone process. +- Standalone’s first compile attempt failed in `StandaloneViewer.cpp` because one helper referred to + `App` outside its scope; that reference was corrected and the final build passed. The existing + `vswhere.exe` lookup warning and `VoxelStrateManager.cpp` C4838 warnings remained. +- Three BuildHost CVar attempts (direct argument and two `ExecCmds` forms) read back zero and did not + change the worker grid; that no-op CVar/harness path was removed. An early Standalone step-128 + attempt also did not reach the pipeline; CLI pass-through was corrected and the final same-binary + step-128 control above is effective. No CVar remains in the final source. +- One high-pitch cap capture was black and was discarded. A separate early capture shell lacked its + memory-probe type because the PowerShell session was fresh; that capture completed without a + pre-launch memory gate. The matched final captures were rerun with the gate active. No + `0xC0000142` occurred and no run needed retrying. +- The only UECC directory was the pre-existing + `UECC-Windows-F1A3EFD5469F5DD62E68EBBACAC84376_0000`, timestamped 2026-09-17; no new UECC folder + was created. +- No new source files. Changed source files and compile spots: + +| File | Compile spot | +|---|---| +| `Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h`, `Standalone/Core/src/VoxelForgeCore.cpp`, `Standalone/Core/src/WorldAssemblerInternal.h` | VoxelForgeCore in Standalone and BuildHost | +| `Standalone/Source/StreamingWorld.h`, `Standalone/Source/StandaloneViewer.cpp` | StandaloneViewer / VoxelForgeCoreStandalone.exe | +| `Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp` | BuildHost VoxelForgeEditor module | +| `Tools/TestVoxelForgeCoreWorldHost.ps1` | PowerShell harness only; recipe-specific fixture logs | + +Async-safety review: no actor async source changed. Epoch checks, MPSC queues, EndPlay draining, +worker `bShuttingDown` checks and `ModsLock` remain unchanged. Region-page publication, shared_ptr +lifetime and concurrent eviction remain unchanged. New duplicate capture stays thread-local and is +copied into the tile result; its worker-id hash is diagnostic only. UObject and RealtimeMeshComponent +mutation remain on the game thread. No owner `.uproject`, editor, `.uasset`, or RealtimeMeshComponent +file was modified. `git diff --check` passed. No commit, push or stash. + +### Dialogue + +**User:** Keep the 12-thread cap; count exclusive LOD8 costs first, optimize the planar far path, and verify quality and determinism. + +**Codex:** The LOD8 interior now follows the MC spacing, with 128-voxel boundary strips preserved. The owner-plan LOD8 p95 fell from 59.848 ms to 13.933 ms; cap and ground gates, both BuildHost actor matrices, current-build worker hashes, and the 40 m/s support ring passed. diff --git a/Standalone/Research/WorldAssemblerM3/Round14/captures/s10_cap.jpg b/Standalone/Research/WorldAssemblerM3/Round14/captures/s10_cap.jpg new file mode 100644 index 0000000..7d3c5be Binary files /dev/null and b/Standalone/Research/WorldAssemblerM3/Round14/captures/s10_cap.jpg differ diff --git a/Standalone/Research/WorldAssemblerM3/Round14/captures/s10_far_horizon.jpg b/Standalone/Research/WorldAssemblerM3/Round14/captures/s10_far_horizon.jpg new file mode 100644 index 0000000..6be9f82 Binary files /dev/null and b/Standalone/Research/WorldAssemblerM3/Round14/captures/s10_far_horizon.jpg differ diff --git a/Standalone/Research/WorldAssemblerM3/Round14/captures/s9_cap.jpg b/Standalone/Research/WorldAssemblerM3/Round14/captures/s9_cap.jpg new file mode 100644 index 0000000..7d3047c Binary files /dev/null and b/Standalone/Research/WorldAssemblerM3/Round14/captures/s9_cap.jpg differ diff --git a/Standalone/Research/WorldAssemblerM3/Round14/captures/s9_far_horizon.jpg b/Standalone/Research/WorldAssemblerM3/Round14/captures/s9_far_horizon.jpg new file mode 100644 index 0000000..d4e8e13 Binary files /dev/null and b/Standalone/Research/WorldAssemblerM3/Round14/captures/s9_far_horizon.jpg differ diff --git a/Standalone/Source/StandaloneViewer.cpp b/Standalone/Source/StandaloneViewer.cpp index 1dd6464..4617bf6 100644 --- a/Standalone/Source/StandaloneViewer.cpp +++ b/Standalone/Source/StandaloneViewer.cpp @@ -1720,6 +1720,7 @@ namespace bool Keys[256]{}; int32 Seed=0; int32 WorkerCount=8; + int32 FarHorizonLOD8InteriorStepVoxels=0; int32 ClipRadius=3; int32 MaxClipLevelOverride=-1; int32 CaptureTileLimit=0; @@ -2903,7 +2904,8 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75; Streamer=std::make_unique(*World,StreamWorkers,false, (!bShapeWorld&&MeshCache.IsLoaded())?&MeshCache:nullptr, bFarShapeStagePending?World->FullResClipLevels-1:-1, - false,false,false,false,ShapeWorld.get()); + false,false,false,false,ShapeWorld.get(),false, + FarHorizonLOD8InteriorStepVoxels); if(bGpuFarTimelineEnabled) Streamer->EnableColdTimelineTrace(ColdProcessStart,&ColdTimelineFrameOrdinal); if(bGpuFarEnabled&&ShapeWorld&&ShapeWorld->hasFarWorld()) @@ -6974,7 +6976,8 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75; const FVector& CameraPositionVoxels,int32 WorkerCount,int32 MaxLevel, VoxelForgeCore::DetailVariant DetailVariant,const VoxelForgeCore::JobScheduler& Scheduler, bool bHashDensity,bool bSkipUvAndColors,bool bSkipMaterialGroups,bool bSkipUV,bool bSkipColors, - std::chrono::steady_clock::time_point ColdStart) + std::chrono::steady_clock::time_point ColdStart,bool bProfileFarColumnOverlap, + int32 FarHorizonLOD8InteriorStepVoxels) { using Clock=std::chrono::steady_clock; FStandaloneOwnerWorld& World=*OwnerWorld; @@ -7042,7 +7045,8 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75; const auto ShapeReady=Clock::now(); const double ShapeSeconds=std::chrono::duration(ShapeReady-ShapeBuildStart).count(); FStandaloneStreamingGenerator Streamer(World,WorkerCount,bHashDensity,nullptr,MaxLevel, - bSkipUvAndColors,bSkipMaterialGroups,bSkipUV,bSkipColors,&Shape); + bSkipUvAndColors,bSkipMaterialGroups,bSkipUV,bSkipColors,&Shape,bProfileFarColumnOverlap, + FarHorizonLOD8InteriorStepVoxels); const auto StreamStart=Clock::now(); const FStandaloneStreamingUpdate Update=Streamer.StartWithDesired(Desired); if(Update.DesiredTiles!=static_cast(Desired.Tiles.size())) @@ -7055,6 +7059,43 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75; std::vector DensityRows,MeshRows,PayloadRows,TopologyRows; int32 Finished=0,NeighborhoodFinished=0,FullDetailFinished=0; int32 FullDetailExpected=0; + struct FColumnXY + { + std::int64_t X=0,Y=0; + bool operator==(const FColumnXY&)const=default; + }; + struct FColumnXYHash + { + std::size_t operator()(const FColumnXY& Value)const noexcept + { + const std::size_t X=std::hash{}(Value.X); + const std::size_t Y=std::hash{}(Value.Y); + return X^(Y+static_cast(0x9E3779B9u)+(X<<6)+(X>>2)); + } + }; + struct FFirstColumnUse + { + FVoxelTileKey Tile; + std::uint64_t WorkerHash=0; + }; + std::unordered_map LOD8ColumnFirstUse; + std::uint64_t LOD8ColumnDuplicateEvents=0,LOD8NeighbourDuplicateEvents=0; + std::uint64_t LOD8CrossWorkerNeighbourDuplicateEvents=0; + std::size_t LOD8ColumnCaptureTiles=0; + bool bLOD8ColumnCaptureComplete=true; + std::array WorkerSecondsByLOD{}; + std::array ColumnEvalMsByLOD{},MarchingExclusiveMsByLOD{}; + std::array GroundRefineExclusiveMsByLOD{},CapRefineExclusiveMsByLOD{}; + std::array MeshAssemblyMsByLOD{}; + std::array TileCountByLOD{}; + const auto AreLOD8Neighbours=[](const FVoxelTileKey& A,const FVoxelTileKey& B) + { + if(A.Level!=8||B.Level!=8)return false; + const std::int64_t DX=std::abs(static_cast(A.Coord.X)-B.Coord.X); + const std::int64_t DY=std::abs(static_cast(A.Coord.Y)-B.Coord.Y); + const std::int64_t DZ=std::abs(static_cast(A.Coord.Z)-B.Coord.Z); + return std::max({DX,DY,DZ})<=1&&(DX+DY+DZ)>0; + }; uint64 NativeSolidBoxes=0,NativeAirBoxes=0,NativeMixedBoxes=0,NativeUnknownBoxes=0; uint64 NativeFieldCalls=0,NativeColumnHits=0,NativeColumnMisses=0,NativeMeshingSkips=0; double NativeClassificationMilliseconds=0.0,NativeColumnComputeMilliseconds=0.0; @@ -7070,6 +7111,41 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75; for(const FStandaloneCpuTile& Tile:Ready) { ++Finished; + if(Tile.Key.Level>=0&&Tile.Key.Level(TileCountByLOD.size())) + { + const std::size_t LOD=static_cast(Tile.Key.Level); + ++TileCountByLOD[LOD]; + WorkerSecondsByLOD[LOD]+=Tile.WorkerSeconds; + ColumnEvalMsByLOD[LOD]+=Tile.CoreDiagnostics.columnEvaluationExclusiveMilliseconds; + MarchingExclusiveMsByLOD[LOD]+=Tile.CoreDiagnostics.marchingCubesExclusiveMilliseconds; + GroundRefineExclusiveMsByLOD[LOD]+=Tile.CoreDiagnostics.groundRefinementExclusiveMilliseconds; + CapRefineExclusiveMsByLOD[LOD]+=Tile.CoreDiagnostics.capRefinementExclusiveMilliseconds; + MeshAssemblyMsByLOD[LOD]+=Tile.CoreDiagnostics.meshAssemblyMilliseconds; + } + if(bProfileFarColumnOverlap&&Tile.Key.Level==8) + { + ++LOD8ColumnCaptureTiles; + bLOD8ColumnCaptureComplete=bLOD8ColumnCaptureComplete + &&Tile.CoreDiagnostics.columnCoordinateCaptureComplete; + std::vector> Coordinates= + Tile.CoreDiagnostics.evaluatedColumnCoordinates; + std::sort(Coordinates.begin(),Coordinates.end()); + Coordinates.erase(std::unique(Coordinates.begin(),Coordinates.end()),Coordinates.end()); + for(const auto& Coordinate:Coordinates) + { + const FColumnXY XY{Coordinate[0],Coordinate[1]}; + const auto [First,Inserted]=LOD8ColumnFirstUse.try_emplace(XY, + FFirstColumnUse{Tile.Key,Tile.CoreDiagnostics.columnEvaluationWorkerHash}); + if(Inserted)continue; + ++LOD8ColumnDuplicateEvents; + if(AreLOD8Neighbours(First->second.Tile,Tile.Key)) + { + ++LOD8NeighbourDuplicateEvents; + if(First->second.WorkerHash!=Tile.CoreDiagnostics.columnEvaluationWorkerHash) + ++LOD8CrossWorkerNeighbourDuplicateEvents; + } + } + } WholeViewReady=std::max(WholeViewReady,Tile.CompletedAt); if(Neighborhood.contains(Tile.Key)) { @@ -7115,6 +7191,22 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75; } } Streamer.Stop(); + if(bProfileFarColumnOverlap) + { + std::printf("[WorldAssemblerLOD8ColumnOverlap] tiles=%zu unique_xy_columns=%zu repeated_tile_evaluations=%llu neighbouring_duplicate_evaluations=%llu cross_worker_neighbouring_duplicates=%llu capture_complete=%d\n", + LOD8ColumnCaptureTiles,LOD8ColumnFirstUse.size(), + static_cast(LOD8ColumnDuplicateEvents), + static_cast(LOD8NeighbourDuplicateEvents), + static_cast(LOD8CrossWorkerNeighbourDuplicateEvents), + bLOD8ColumnCaptureComplete?1:0); + } + for(std::size_t LOD=0;LOD(TileCountByLOD[LOD]),WorkerSecondsByLOD[LOD], + ColumnEvalMsByLOD[LOD],MarchingExclusiveMsByLOD[LOD], + GroundRefineExclusiveMsByLOD[LOD],CapRefineExclusiveMsByLOD[LOD],MeshAssemblyMsByLOD[LOD]); + } const double NeighborhoodSeconds=std::chrono::duration(NeighborhoodReady-ColdStart).count(); const double NeighborhoodMeshSeconds=std::chrono::duration(NeighborhoodReady-ShapeReady).count(); const double FullDetailSeconds=std::chrono::duration(FullDetailReady-ColdStart).count(); @@ -7160,7 +7252,8 @@ float3 Lit=Rock*(Ambient+0.88*Diffuse)+float3(0.50,0.69,0.82)*Head*0.75; static_cast(HashTileRows(TopologyRows))); } return NeighborhoodFinished==static_cast(Neighborhood.size()) - &&FullDetailFinished==FullDetailExpected&&Finished==Update.DesiredTiles&&bHashesPassing?0:6; + &&FullDetailFinished==FullDetailExpected&&Finished==Update.DesiredTiles&&bHashesPassing + &&(!bProfileFarColumnOverlap||(bLOD8ColumnCaptureComplete&&LOD8ColumnCaptureTiles>0))?0:6; } int RunStreamingBenchmark(FStandaloneOwnerWorld& World,VoxelForgeCore::World* CoreWorld, const FVector& CameraPositionVoxels, @@ -9508,6 +9601,7 @@ int main(int Argc,char** Argv) std::string WorldAssemblyManifestOutputPath; bool bStreamBenchmark=false; bool bStreamPipeline=false; + bool bProfileFarColumnOverlap=false; bool bCoarseFirstStreamOrder=false; bool bProfileDetailBand=false; bool bPassageDetailAuditOnly=false; @@ -9530,6 +9624,9 @@ int main(int Argc,char** Argv) { const std::string_view Arg=Argv[I]; if(Arg.starts_with("--capture="))App.CapturePath=std::string(Arg.substr(10)); + else if(Arg.starts_with("--far-horizon-lod8-step=")) + App.FarHorizonLOD8InteriorStepVoxels=std::max(0,std::atoi( + std::string(Arg.substr(std::string_view("--far-horizon-lod8-step=").size())).c_str())); else if(Arg.starts_with("--capture-flight-sequence-prefix=")) App.FlightCaptureSequencePrefix=std::string( Arg.substr(std::string_view("--capture-flight-sequence-prefix=").size())); @@ -9642,6 +9739,8 @@ int main(int Argc,char** Argv) else if(Arg=="--coarse-first-stream-order")bCoarseFirstStreamOrder=true; else if(Arg=="--stream-pipeline") {bStreamPipeline=true;bStreamBenchmark=true;App.bShapeWorld=true;App.bForceGenerate=true;} + else if(Arg=="--profile-far-column-overlap") + {bProfileFarColumnOverlap=true;bStreamPipeline=true;bStreamBenchmark=true;App.bShapeWorld=true;App.bForceGenerate=true;} else if(Arg=="--edit-benchmark") {bEditBenchmark=true;bStreamBenchmark=true;App.bShapeWorld=true;} else if(Arg=="--stream-hash")bStreamHash=true; @@ -9858,7 +9957,8 @@ int main(int Argc,char** Argv) FVector(App.Position.X*4.0,App.Position.Y*4.0,App.Position.Z*4.0), App.WorkerCount,StreamBenchmarkMaxLevel,App.ShapeDetailVariant,App.CoreScheduler, bStreamHash||!BakeMeshCachePath.empty(),bSkipUvAndColors,bSkipMaterialGroups, - bSkipUV,bSkipColors,ColdProcessStart); + bSkipUV,bSkipColors,ColdProcessStart,bProfileFarColumnOverlap, + App.FarHorizonLOD8InteriorStepVoxels); if(bEditBenchmark) return RunEditTransactionBenchmark(*App.World,*App.ShapeWorld, FVector(App.Position.X*4.0,App.Position.Y*4.0,App.Position.Z*4.0),App.WorkerCount); diff --git a/Standalone/Source/StreamingWorld.h b/Standalone/Source/StreamingWorld.h index 7b9bcd9..764431e 100644 --- a/Standalone/Source/StreamingWorld.h +++ b/Standalone/Source/StreamingWorld.h @@ -423,11 +423,15 @@ public: bool bInSkipMaterialGroups=false, bool bInSkipUV=false, bool bInSkipColors=false, - VoxelForgeCore::World* InCoreWorld=nullptr) + VoxelForgeCore::World* InCoreWorld=nullptr, + bool bInCaptureFarColumnCoordinates=false, + int32 InLod8HorizonInteriorStepVoxels=0) :World(&InWorld),WorkerCount(FMath::Clamp(InWorkerCount,1,16)),bHashDensity(bInHashDensity), MeshCache(InMeshCache),MaxLevelFilter(InMaxLevelFilter), CoreWorld(InCoreWorld),bSkipUvAndColors(bInSkipUvAndColors),bSkipMaterialGroups(bInSkipMaterialGroups), - bSkipUV(bInSkipUV),bSkipColors(bInSkipColors) + bSkipUV(bInSkipUV),bSkipColors(bInSkipColors), + bCaptureFarColumnCoordinates(bInCaptureFarColumnCoordinates), + Lod8HorizonInteriorStepVoxels(std::max(0,InLod8HorizonInteriorStepVoxels)) { VoxelGenLOD::InitializeConsoleSwitches(); UVoxelMarchingCubesMesher::InitializeConsoleSwitches(); @@ -795,6 +799,8 @@ private: Options.bandMinZVoxel=Job.BandZMin;Options.bandMaxZVoxel=Job.BandZMax; Options.hashDensity=bHashDensity;Options.skipUvAndColors=bSkipUvAndColors; Options.skipMaterialGroups=bSkipMaterialGroups;Options.skipUv=bSkipUV;Options.skipColors=bSkipColors; + Options.captureColumnCoordinates=bCaptureFarColumnCoordinates&&Level==8; + Options.lod8HorizonInteriorStepVoxels=Lod8HorizonInteriorStepVoxels; std::string Error; auto Result=CoreWorld->generateTile( {{Job.Key.Coord.X,Job.Key.Coord.Y,Job.Key.Coord.Z},Job.Key.Level},Options,Error); @@ -965,6 +971,8 @@ private: bool bSkipMaterialGroups=false; bool bSkipUV=false; bool bSkipColors=false; + bool bCaptureFarColumnCoordinates=false; + int32 Lod8HorizonInteriorStepVoxels=0; std::atomic CacheHitCount{0}; mutable std::mutex Mutex; std::condition_variable Wake; diff --git a/Tools/TestVoxelForgeCoreWorldHost.ps1 b/Tools/TestVoxelForgeCoreWorldHost.ps1 index 8da0934..7b12030 100644 --- a/Tools/TestVoxelForgeCoreWorldHost.ps1 +++ b/Tools/TestVoxelForgeCoreWorldHost.ps1 @@ -179,6 +179,7 @@ $RunName += switch ($NearReserveMode) { 'Dynamic' { '_nearreserve_dynamic' } default { '_nearreserve_auto' } } +if ($HybridFixturesOnly) { $RunName += "_$($CoreMesherRecipe.ToLowerInvariant())" } if ($EnforceCollisionGate) { $RunName += '_collision_gate' } if ($StartupTrace -or $StreamingTraceOnly) { $RunName += '_trace' } if ($StreamingTraceOnly) {