diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 8cf4038..0e9eda6 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -694,6 +694,10 @@ driven by `EditorBrush*` props. position → seamless across chunk borders (both sides use identical pure samples). NO per-vertex `GetDensityAt` (was ~6/vertex, often as costly as the whole grid). `ComputeGradientNormal` is now unused. Only NORMALS changed vs the old path; geometry is identical. + Round 18 memoizes these exact central-difference `FVector`s in per-worker scratch arrays for the + step-1 ShapeWorld path. Validity is reset for each tile; cache hits return the same vector and do + not bypass the density grid, halo, or two-pass cell loop. Keep the cache worker-local and exact; + do not move gradient probes above the surface-cell test. - **Core hybrid sharp mesher (M2)** (`CoreMesherRecipe` / `MesherRecipe`): the property remains opt-in and defaults to `MarchingCubesV1`; `bUseVoxelForgeCore` remains separately default-off. V1 preserves the round-3 feature-aware dual-contouring path for repeatability. V2 reuses the @@ -1778,6 +1782,20 @@ shared-page A/B test, not proof that a shared cache will pay after synchronizati costs. Round 17 changed no density, mesh, collision, or recipe output; recipe versions remain unchanged. See [`Standalone/Research/WorldAssemblerM3/Round17/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round17/REPORT.md). +**Round 18 / S14:** the 65,536-entry worker-local XY column cache keeps the legacy direct-map +lookup for LOD0–2 (at most 33/28/4 conflict misses in the cold audit) and uses four-way sets for +LOD3–8. The deterministic coordinate/hash mixer and round-robin replacement cut LOD4 from 8,224 +misses / 7,407 conflict misses to 1,437 / 0, and LOD3 from 1,634 / 1,119 to 574 / 0. Exact key +validation keeps fill order from affecting samples. `voxel.CoreColumnCacheDirectMapped=1` restores +the S13 route as a recorded negative control. + +The step-1 ShapeWorld gradient memo reduced LOD0 fallback density probes from 11,474 to 3,018 at +the sampled p95 and gradient-normal cost from 65.10 to 24.16 ns/sample; the two-pass MC invariant +is unchanged. `voxel.CoreMCGradientMemoization=0` is its recorded negative control. No shared XY +page was added: cross-worker repeats alone do not establish a net page-cache win, and the S8 +immutable-page lifetime rules would add ownership and memory cost. Round 18 changed no output or +recipe version. See [`Standalone/Research/WorldAssemblerM3/Round18/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round18/REPORT.md). + ## 9. Multiplayer model (listen-server first, dedicated-friendly) > **Status: transport is DESIGN ONLY — nothing is networked in-tree yet** (no `Replicated`/`HasAuthority`/RPCs; a diff --git a/CODEMAP.md b/CODEMAP.md index 6acaec4..6c0939b 100644 --- a/CODEMAP.md +++ b/CODEMAP.md @@ -1095,6 +1095,20 @@ generation output. Measurement-only stride and octave probes are confined to the audit and are not runtime controls. The result and limits are recorded in [`Standalone/Research/WorldAssemblerM3/Round17/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round17/REPORT.md). +### 3.20 MC stage audit and coarse column-cache conflicts — World Assembler S14 / M3 Round 18 + +`Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp` records the sampled density +grid, cell classification, grid-gradient normals, edge interpolation, and vertex/dedup stream +stages for the BuildHost audit. Its exact per-worker gradient memo is controlled and trace-read +back through `voxel.CoreMCGradientMemoization`. `Standalone/Core/src/WorldAssemblerInternal.h` +keeps the 65,536-entry worker-local column cache, using the S13 direct-map for LOD0–2 and a +deterministic four-way set-associative route for LOD3–8; `voxel.CoreColumnCacheDirectMapped=1` +restores the S13 path for the negative control. Runtime CVar capture lives in +`Source/VoxelForge/Private/VoxelWorld.cpp`; stage and conflict counts are reported by +`Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp`. The measurements, controls, and +cold-path decision are in +[`Standalone/Research/WorldAssemblerM3/Round18/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round18/REPORT.md). + ## 4. The density pipeline (most-edited hot path) ### 4.1 `GetDensityWithParams` (TunnelNetwork) — VoxelGenerator.cpp:3749 diff --git a/Source/VoxelForge/Private/VoxelWorld.cpp b/Source/VoxelForge/Private/VoxelWorld.cpp index 483ddd2..f1dd455 100644 --- a/Source/VoxelForge/Private/VoxelWorld.cpp +++ b/Source/VoxelForge/Private/VoxelWorld.cpp @@ -235,6 +235,18 @@ namespace GVoxelForgeStreamingBackgroundWorkerCap, TEXT("Size tile admission slots to the TaskGraph background workers used by tile tasks; 0 restores the logical-core cap.")); + int32 GVoxelForgeCoreColumnCacheDirectMapped = 0; + FAutoConsoleVariableRef CVarVoxelForgeCoreColumnCacheDirectMapped( + TEXT("voxel.CoreColumnCacheDirectMapped"), + GVoxelForgeCoreColumnCacheDirectMapped, + TEXT("Negative control: 0 uses the four-way deterministic XY column cache; 1 restores S13 direct-map indexing.")); + + int32 GVoxelForgeCoreMCGradientMemoization = 1; + FAutoConsoleVariableRef CVarVoxelForgeCoreMCGradientMemoization( + TEXT("voxel.CoreMCGradientMemoization"), + GVoxelForgeCoreMCGradientMemoization, + TEXT("Negative control: 1 reuses exact LOD0 lattice gradients across adjacent MC cells; 0 restores per-cell gradient probes.")); + int32 GVoxelForgeProfileTileGeneration = 0; FAutoConsoleVariableRef CVarVoxelForgeProfileTileGeneration( TEXT("voxel.ProfileTileGeneration"), @@ -2080,6 +2092,8 @@ void AVoxelWorld::BeginPlay() int32 CommandLineStreamingResultBacklogCap = GVoxelForgeStreamingResultBacklogCap; int32 CommandLineStreamingReadyQueueDepth = GVoxelForgeStreamingReadyQueueDepth; int32 CommandLineStreamingBackgroundWorkerCap = GVoxelForgeStreamingBackgroundWorkerCap; + int32 CommandLineCoreColumnCacheDirectMapped = GVoxelForgeCoreColumnCacheDirectMapped; + int32 CommandLineCoreMCGradientMemoization = GVoxelForgeCoreMCGradientMemoization; int32 CommandLineStep1GenerationCap = GVoxelForgeStep1GenerationCap; int32 CommandLineStep1ExitAfterFrames = GVoxelForgeStep1ExitAfterFrames; int32 CommandLineStep1CsvCaptureOnBeginPlay = GVoxelForgeStep1CsvCaptureOnBeginPlay; @@ -2186,6 +2200,18 @@ void AVoxelWorld::BeginPlay() GVoxelForgeStreamingBackgroundWorkerCap = CommandLineStreamingBackgroundWorkerCap != 0 ? 1 : 0; } + if (FParse::Value(FCommandLine::Get(), TEXT("voxel.CoreColumnCacheDirectMapped="), + CommandLineCoreColumnCacheDirectMapped)) + { + GVoxelForgeCoreColumnCacheDirectMapped= + CommandLineCoreColumnCacheDirectMapped!=0?1:0; + } + if (FParse::Value(FCommandLine::Get(), TEXT("voxel.CoreMCGradientMemoization="), + CommandLineCoreMCGradientMemoization)) + { + GVoxelForgeCoreMCGradientMemoization= + CommandLineCoreMCGradientMemoization!=0?1:0; + } if (FParse::Value(FCommandLine::Get(), TEXT("voxel.Step1PrequeueVisibleLOD0="), CommandLineStep1PrequeueVisibleLOD0)) { @@ -2609,7 +2635,9 @@ void AVoxelWorld::BeginPlay() "\"streaming_legacy_pending_result_cap\":%d," "\"streaming_collision_lookahead\":%d,\"core_region_concurrent_admission\":%d," "\"collision_gate\":%d," - "\"streaming_result_backlog_cap\":%d,\"streaming_startup_apply_boost\":%d"), + "\"streaming_result_backlog_cap\":%d,\"streaming_startup_apply_boost\":%d," + "\"core_column_cache_direct_mapped\":%d," + "\"core_mc_gradient_memoization\":%d"), FPlatformMisc::NumberOfCoresIncludingHyperthreads(), GetMaxConcurrentStreamingTasks(), TaskScheduler.GetNumWorkers(), TaskScheduler.GetMaxNumWorkers(), static_cast(TaskScheduler.GetWorkerPriority()), @@ -2622,7 +2650,9 @@ void AVoxelWorld::BeginPlay() GVoxelForgeStreamingLegacyPendingResultCap, GVoxelForgeStreamingCollisionLookahead, GVoxelForgeCoreRegionConcurrentAdmission, GVoxelForgeCollisionGate, - GVoxelForgeStreamingResultBacklogCap, GVoxelForgeStreamingStartupApplyBoost)); + GVoxelForgeStreamingResultBacklogCap, GVoxelForgeStreamingStartupApplyBoost, + GVoxelForgeCoreColumnCacheDirectMapped, + GVoxelForgeCoreMCGradientMemoization)); VoxelForgeStartupTrace::RecordEvent(TEXT("streaming_worker_cap_readback"), FString::Printf( TEXT("\"streaming_background_worker_cap\":%d,\"background_worker_threads\":%d," "\"effective_tile_slots\":%d"), @@ -13043,13 +13073,15 @@ void AVoxelWorld::LoadCoreTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority VoxelForgeStartupTrace::RecordRequest(RequestSample, TaskEpoch); } const bool bCaptureHashes = bCaptureVoxelForgeCoreHashesForValidation; + const bool bLegacyDirectMappedColumnCache=GVoxelForgeCoreColumnCacheDirectMapped!=0; + const bool bMemoizeCoreMCGradients=GVoxelForgeCoreMCGradientMemoization!=0; const UE::Tasks::ETaskPriority TaskPriority = bNearPriority ? UE::Tasks::ETaskPriority::BackgroundHigh : UE::Tasks::ETaskPriority::BackgroundNormal; UE::Tasks::Launch(TEXT("VoxelForgeCoreTile"), [this, State, Cancellation, Tile, WorkPriority, TaskEpoch, TaskDesiredEpoch, TaskModificationEpoch, RequestStartCycles, RequestOrdinal, bResidentAtRequest, - bCaptureHashes]() + bCaptureHashes,bLegacyDirectMappedColumnCache,bMemoizeCoreMCGradients]() { struct FTaskGuard { @@ -13153,6 +13185,8 @@ void AVoxelWorld::LoadCoreTile(const FVoxelTileKey& Tile, EVoxelTileWorkPriority break; } Options.hashDensity = bCaptureHashes; + Options.legacyDirectMappedColumnCache=bLegacyDirectMappedColumnCache; + Options.memoizeMcGradients=bMemoizeCoreMCGradients; std::string CoreError; std::optional CoreMesh; { diff --git a/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp b/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp index 57e68d9..9de9dca 100644 --- a/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp +++ b/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp @@ -16204,10 +16204,10 @@ bool RunCoreSharedFeatureCostAudit(VoxelForgeCore::World& World, } VoxelForgeCore::Heightfield2D::resetThreadColumnCacheStats(false,true,0); } - for (const int32 LOD : {0,1,3,4,5,8}) + for (const int32 LOD : {0,1,2,3,4,5,6,7,8}) { TArray FullTimes,CandidateTimes,FullShellTimes,CandidateShellTimes,ListTimes; - TArray UnknownBoxes,FieldCalls,ColumnMisses,StratumShellTimes,ConnectorShellTimes; + TArray UnknownBoxes,FieldCalls,UnclampedFieldCalls,ColumnMisses,ColumnConflictMisses,StratumShellTimes,ConnectorShellTimes; TArray DensitySamples,ClassificationTimes,ClassificationExclusiveTimes; TArray ColumnComputeTimes,MeshingTimes,HorizonTimes,RepairSkipped; TArray ColumnEvalExclusiveTimes,MarchingExclusiveTimes,GroundRefineExclusiveTimes; @@ -16216,13 +16216,29 @@ bool RunCoreSharedFeatureCostAudit(VoxelForgeCore::World& World, TArray ClassificationNsPerSample,ColumnNsPerSample,MarchingNsPerSample; TArray GroundNsPerSample,CapNsPerSample,AssemblyNsPerSample,ListNsPerSample; TArray MarchingColumnNsPerSample,GroundColumnNsPerSample,CapColumnNsPerSample; + TArray McProfileTotalTimes,McProfileGridTimes,McProfileGridExclusiveTimes; + TArray McProfileCellTimes,McProfileNormalTimes,McProfileInterpolationTimes; + TArray McProfileStreamTimes,McProfileOtherTimes,McProfileResidualTimes; + TArray McProfileGradientFallbackCalls,McProfileEdgeMidpointCalls,McProfileGradientMemoHits; + TArray McProfileGridNsPerSample,McProfileGridExclusiveNsPerSample; + TArray McProfileCellNsPerSample,McProfileNormalNsPerSample; + TArray McProfileInterpolationNsPerSample,McProfileStreamNsPerSample; + TArray McProfileOtherNsPerSample,McProfileResidualNsPerSample; TArray StratumFieldCalls,ConnectorFieldCalls,ShellTriangles,GroundTriangles,GroundRemoved,CapTriangles; + const bool bMcStageAudit=FParse::Param(FCommandLine::Get(),TEXT("coremcstageaudit")); + int32 AuditDirectMappedColumnCache=0,AuditMemoizeMcGradients=1; + FParse::Value(FCommandLine::Get(),TEXT("voxel.CoreColumnCacheDirectMapped="), + AuditDirectMappedColumnCache); + FParse::Value(FCommandLine::Get(),TEXT("voxel.CoreMCGradientMemoization="), + AuditMemoizeMcGradients); for (const auto& Key : Keys) { if (Key.lod!=LOD || FullTimes.Num()>=8) continue; VoxelForgeCore::TileGenerationOptions Options; Options.hashDensity=true; Options.useAssemblyFeatureQueries=false; + Options.legacyDirectMappedColumnCache=AuditDirectMappedColumnCache!=0; + Options.memoizeMcGradients=AuditMemoizeMcGradients!=0; std::string Error; // Check density and mesh parity with hashes, then time uncached runtime generation. // Verifier la parite avec hashes, puis mesurer la generation sans cache de colonnes. @@ -16256,7 +16272,9 @@ bool RunCoreSharedFeatureCostAudit(VoxelForgeCore::World& World, DensitySamples.Add(static_cast(Candidate->densitySampleCount)); UnknownBoxes.Add(static_cast(Diagnostics.unknownBoxes)); FieldCalls.Add(static_cast(Diagnostics.fieldCalls)); + UnclampedFieldCalls.Add(static_cast(Diagnostics.unclampedFieldCalls)); ColumnMisses.Add(static_cast(Diagnostics.columnCacheMisses)); + ColumnConflictMisses.Add(static_cast(Diagnostics.columnCacheConflictMisses)); ClassificationTimes.Add(Diagnostics.classificationMilliseconds); ClassificationExclusiveTimes.Add(Diagnostics.classificationExclusiveMilliseconds); ColumnComputeTimes.Add(Diagnostics.columnComputeMilliseconds); @@ -16301,13 +16319,66 @@ bool RunCoreSharedFeatureCostAudit(VoxelForgeCore::World& World, *1000000.0/SampleCount); ListNsPerSample.Add(Diagnostics.featureListMilliseconds*1000000.0/SampleCount); } + if(bMcStageAudit) + { + // Profile a separate cold tile so the audit's ordinary timing rows keep the + // uninstrumented path and comparable clock scope. + // Profiler une tuile froide separee pour garder les mesures ordinaires sans instrumentation. + Options.profileMesherStages=true; + VoxelForgeCore::Heightfield2D::resetThreadColumnCacheStats(false,true); + auto Profiled=World.generateTile(Key,Options,Error); + Options.profileMesherStages=false; + if(!Profiled) + { OutError=TEXT("Cold MC stage profile tile generation failed."); return false; } + const auto& Mc=Profiled->diagnostics; + const double Total=Mc.marchingCubesExclusiveMilliseconds + +Mc.marchingCubesColumnComputeMilliseconds; + const double Grid=Mc.marchingCubesDensityGridMilliseconds; + const double GridExclusive=FMath::Max(0.0,Grid + -Mc.marchingCubesColumnComputeMilliseconds); + const double Cell=Mc.marchingCubesCellClassificationMilliseconds; + const double Normal=Mc.marchingCubesGradientNormalsMilliseconds; + const double Interpolation=Mc.marchingCubesEdgeInterpolationMilliseconds; + const double Stream=Mc.marchingCubesVertexDedupStreamMilliseconds; + const double Other=Mc.marchingCubesOtherMilliseconds; + const double Residual=FMath::Max(0.0,Mc.marchingCubesExclusiveMilliseconds + -GridExclusive-Cell-Normal-Interpolation-Stream); + McProfileTotalTimes.Add(Total); + McProfileGridTimes.Add(Grid); + McProfileGridExclusiveTimes.Add(GridExclusive); + McProfileCellTimes.Add(Cell); + McProfileNormalTimes.Add(Normal); + McProfileInterpolationTimes.Add(Interpolation); + McProfileStreamTimes.Add(Stream); + McProfileOtherTimes.Add(Other); + McProfileResidualTimes.Add(Residual); + McProfileGradientFallbackCalls.Add(static_cast( + Mc.marchingCubesGradientFallbackDensityCalls)); + McProfileEdgeMidpointCalls.Add(static_cast( + Mc.marchingCubesEdgeMidpointDensityCalls)); + McProfileGradientMemoHits.Add(static_cast( + Mc.marchingCubesGradientMemoHits)); + const double ProfileSamples=static_cast(Profiled->densitySampleCount); + if(ProfileSamples>0.0) + { + McProfileGridNsPerSample.Add(Grid*1000000.0/ProfileSamples); + McProfileGridExclusiveNsPerSample.Add(GridExclusive*1000000.0/ProfileSamples); + McProfileCellNsPerSample.Add(Cell*1000000.0/ProfileSamples); + McProfileNormalNsPerSample.Add(Normal*1000000.0/ProfileSamples); + McProfileInterpolationNsPerSample.Add(Interpolation*1000000.0/ProfileSamples); + McProfileStreamNsPerSample.Add(Stream*1000000.0/ProfileSamples); + McProfileOtherNsPerSample.Add(Other*1000000.0/ProfileSamples); + McProfileResidualNsPerSample.Add(Residual*1000000.0/ProfileSamples); + } + } } const auto P95=[](TArray Values) { Values.Sort(); return Values.IsEmpty() ? 0.0 : Values[FMath::Clamp( FMath::CeilToInt(Values.Num()*0.95f)-1,0,Values.Num()-1)]; }; UE_LOG(LogTemp,Display, - TEXT("[VoxelForgeSharedFeatureCost] label=%s lod=%d tiles=%d hashes_identical=1 full_p95_ms=%.5f candidate_p95_ms=%.5f list_p95_ms=%.5f classification_wall_p95_ms=%.5f classification_exclusive_p95_ms=%.5f column_compute_p95_ms=%.5f column_eval_exclusive_p95_ms=%.5f mc_column_compute_p95_ms=%.5f mc_exclusive_p95_ms=%.5f ground_column_compute_p95_ms=%.5f ground_refine_exclusive_p95_ms=%.5f cap_column_compute_p95_ms=%.5f cap_refine_exclusive_p95_ms=%.5f mesh_assembly_p95_ms=%.5f meshing_p95_ms=%.5f horizon_p95_ms=%.5f samples_p95=%.0f classification_ns_per_sample_p95=%.2f column_ns_per_sample_p95=%.2f mc_column_ns_per_sample_p95=%.2f mc_ns_per_sample_p95=%.2f ground_column_ns_per_sample_p95=%.2f ground_ns_per_sample_p95=%.2f cap_column_ns_per_sample_p95=%.2f cap_ns_per_sample_p95=%.2f assembly_ns_per_sample_p95=%.2f feature_list_ns_per_sample_p95=%.2f full_shell_p95_ms=%.5f candidate_shell_p95_ms=%.5f repair_skipped_p95=%.0f unknown_boxes_p95=%.0f field_calls_p95=%.0f column_misses_p95=%.0f stratum_shell_p95_ms=%.5f stratum_field_calls_p95=%.0f connector_shell_p95_ms=%.5f connector_field_calls_p95=%.0f shell_triangles_p95=%.0f horizon_ground_triangles_p95=%.0f horizon_ground_removed_p95=%.0f horizon_cap_triangles_p95=%.0f"), - Label,LOD,FullTimes.Num(),P95(FullTimes),P95(CandidateTimes),P95(ListTimes), + TEXT("[VoxelForgeSharedFeatureCost] label=%s lod=%d column_cache_direct_mapped=%d gradient_memoization=%d tiles=%d hashes_identical=1 full_p95_ms=%.5f candidate_p95_ms=%.5f list_p95_ms=%.5f classification_wall_p95_ms=%.5f classification_exclusive_p95_ms=%.5f column_compute_p95_ms=%.5f column_eval_exclusive_p95_ms=%.5f mc_column_compute_p95_ms=%.5f mc_exclusive_p95_ms=%.5f ground_column_compute_p95_ms=%.5f ground_refine_exclusive_p95_ms=%.5f cap_column_compute_p95_ms=%.5f cap_refine_exclusive_p95_ms=%.5f mesh_assembly_p95_ms=%.5f meshing_p95_ms=%.5f horizon_p95_ms=%.5f samples_p95=%.0f classification_ns_per_sample_p95=%.2f column_ns_per_sample_p95=%.2f mc_column_ns_per_sample_p95=%.2f mc_ns_per_sample_p95=%.2f ground_column_ns_per_sample_p95=%.2f ground_ns_per_sample_p95=%.2f cap_column_ns_per_sample_p95=%.2f cap_ns_per_sample_p95=%.2f assembly_ns_per_sample_p95=%.2f feature_list_ns_per_sample_p95=%.2f full_shell_p95_ms=%.5f candidate_shell_p95_ms=%.5f repair_skipped_p95=%.0f unknown_boxes_p95=%.0f field_calls_p95=%.0f unclamped_field_calls_p95=%.0f column_misses_p95=%.0f column_conflict_misses_p95=%.0f stratum_shell_p95_ms=%.5f stratum_field_calls_p95=%.0f connector_shell_p95_ms=%.5f connector_field_calls_p95=%.0f shell_triangles_p95=%.0f horizon_ground_triangles_p95=%.0f horizon_ground_removed_p95=%.0f horizon_cap_triangles_p95=%.0f"), + Label,LOD,AuditDirectMappedColumnCache!=0?1:0,AuditMemoizeMcGradients!=0?1:0, + FullTimes.Num(),P95(FullTimes),P95(CandidateTimes),P95(ListTimes), P95(ClassificationTimes),P95(ClassificationExclusiveTimes),P95(ColumnComputeTimes), P95(ColumnEvalExclusiveTimes),P95(MarchingColumnTimes),P95(MarchingExclusiveTimes), P95(GroundColumnTimes),P95(GroundRefineExclusiveTimes),P95(CapColumnTimes), @@ -16317,9 +16388,24 @@ bool RunCoreSharedFeatureCostAudit(VoxelForgeCore::World& World, P95(MarchingNsPerSample),P95(GroundColumnNsPerSample),P95(GroundNsPerSample), P95(CapColumnNsPerSample),P95(CapNsPerSample),P95(AssemblyNsPerSample),P95(ListNsPerSample), P95(FullShellTimes),P95(CandidateShellTimes),P95(RepairSkipped),P95(UnknownBoxes),P95(FieldCalls), - P95(ColumnMisses),P95(StratumShellTimes),P95(StratumFieldCalls), + P95(UnclampedFieldCalls), + P95(ColumnMisses),P95(ColumnConflictMisses),P95(StratumShellTimes),P95(StratumFieldCalls), P95(ConnectorShellTimes),P95(ConnectorFieldCalls),P95(ShellTriangles), P95(GroundTriangles),P95(GroundRemoved),P95(CapTriangles)); + if(bMcStageAudit) + { + UE_LOG(LogTemp,Display, + TEXT("[VoxelForgeMCStageCost] label=%s lod=%d tiles=%d sampling=row_total_and_1_of_8_cells total_p95_ms=%.5f density_grid_inclusive_p95_ms=%.5f density_grid_ex_column_p95_ms=%.5f cell_classification_p95_ms=%.5f gradient_normals_p95_ms=%.5f edge_interpolation_p95_ms=%.5f vertex_dedup_stream_p95_ms=%.5f setup_finalization_p95_ms=%.5f unassigned_residual_p95_ms=%.5f grid_ns_per_sample_p95=%.2f grid_ex_column_ns_per_sample_p95=%.2f cell_ns_per_sample_p95=%.2f normals_ns_per_sample_p95=%.2f interpolation_ns_per_sample_p95=%.2f dedup_stream_ns_per_sample_p95=%.2f setup_finalization_ns_per_sample_p95=%.2f residual_ns_per_sample_p95=%.2f gradient_fallback_density_calls_p95=%.0f edge_midpoint_density_probes_p95=%.0f gradient_memo_hits_p95=%.0f"), + Label,LOD,McProfileTotalTimes.Num(),P95(McProfileTotalTimes),P95(McProfileGridTimes), + P95(McProfileGridExclusiveTimes),P95(McProfileCellTimes),P95(McProfileNormalTimes), + P95(McProfileInterpolationTimes),P95(McProfileStreamTimes),P95(McProfileOtherTimes), + P95(McProfileResidualTimes),P95(McProfileGridNsPerSample), + P95(McProfileGridExclusiveNsPerSample),P95(McProfileCellNsPerSample), + P95(McProfileNormalNsPerSample),P95(McProfileInterpolationNsPerSample), + P95(McProfileStreamNsPerSample),P95(McProfileOtherNsPerSample), + P95(McProfileResidualNsPerSample),P95(McProfileGradientFallbackCalls), + P95(McProfileEdgeMidpointCalls),P95(McProfileGradientMemoHits)); + } } if(Assembly!=nullptr&&!Assembly->plan.strates.empty()) { diff --git a/Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h b/Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h index cc98e35..8466849 100644 --- a/Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h +++ b/Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h @@ -123,6 +123,7 @@ namespace VoxelForgeCore std::uint64_t unclampedFieldCalls = 0; std::uint64_t columnCacheHits = 0; std::uint64_t columnCacheMisses = 0; + std::uint64_t columnCacheConflictMisses = 0; double classificationMilliseconds = 0.0; double columnComputeMilliseconds = 0.0; double columnEvaluationExclusiveMilliseconds = 0.0; @@ -131,6 +132,15 @@ namespace VoxelForgeCore double capRefinementColumnComputeMilliseconds = 0.0; double classificationExclusiveMilliseconds = 0.0; double marchingCubesExclusiveMilliseconds = 0.0; + double marchingCubesDensityGridMilliseconds = 0.0; + double marchingCubesCellClassificationMilliseconds = 0.0; + double marchingCubesGradientNormalsMilliseconds = 0.0; + double marchingCubesEdgeInterpolationMilliseconds = 0.0; + double marchingCubesVertexDedupStreamMilliseconds = 0.0; + double marchingCubesOtherMilliseconds = 0.0; + std::uint64_t marchingCubesGradientFallbackDensityCalls = 0; + std::uint64_t marchingCubesEdgeMidpointDensityCalls = 0; + std::uint64_t marchingCubesGradientMemoHits = 0; double groundRefinementExclusiveMilliseconds = 0.0; double capRefinementExclusiveMilliseconds = 0.0; double meshAssemblyMilliseconds = 0.0; @@ -639,6 +649,7 @@ namespace VoxelForgeCore std::uint64_t pointCalls = 0; std::uint64_t columnHits = 0; std::uint64_t columnMisses = 0; + std::uint64_t columnConflictMisses = 0; double columnComputeMilliseconds = 0.0; bool captureCoordinates = false; bool coordinateCaptureComplete = true; @@ -693,6 +704,7 @@ namespace VoxelForgeCore std::int64_t maximumZSubvoxels, std::int32_t stepVoxels) const noexcept; static void resetThreadColumnCacheStats(bool captureCoordinates = false, bool clearCache = false, std::uint32_t octaveDrop = 0) noexcept; + static void setThreadColumnCacheLegacyDirectMapped(bool enabled) noexcept; static const HeightfieldColumnCacheStats& threadColumnCacheStats() noexcept; FieldInterval interval(std::int64_t minimumXSubvoxels, std::int64_t minimumYSubvoxels, @@ -1021,6 +1033,15 @@ namespace VoxelForgeCore // Opt-in diagnostic only; records XY columns evaluated by this LOD8 tile. // Diagnostic seulement; enregistre les colonnes XY evaluees par cette tuile LOD8. bool captureColumnCoordinates = false; + // Separate cold pass only; samples the MC inner stages without changing output. + // Passe froide separee seulement; mesure les etapes internes MC sans changer la sortie. + bool profileMesherStages = false; + // Negative control for the shared XY-column cache; 1 restores the old direct-map indexing. + // Controle negatif du cache XY partage; 1 restaure l'ancien indexage direct. + bool legacyDirectMappedColumnCache = false; + // Diagnostic/performance control; memoizes exact MC lattice gradients at LOD0. + // Controle de mesure/performance; memorise les gradients MC exacts au LOD0. + bool memoizeMcGradients = true; // LOD8-only planar horizon interior grid; zero follows the MC step, 128 restores S9. // Grille interieure LOD8 seulement; zero suit la maille MC, 128 restaure S9. std::int32_t lod8HorizonInteriorStepVoxels = 0; diff --git a/Standalone/Core/src/VoxelForgeCore.cpp b/Standalone/Core/src/VoxelForgeCore.cpp index 392129c..ab20440 100644 --- a/Standalone/Core/src/VoxelForgeCore.cpp +++ b/Standalone/Core/src/VoxelForgeCore.cpp @@ -1325,6 +1325,11 @@ namespace VoxelForgeCore std::optional World::generateTile(const TileKey& key,const TileGenerationOptions& options, std::string& error) const { + // Fine owner tiles had zero direct-map conflict evictions in the cold audit. Keep their + // single-probe path and use associativity for the coarse coordinate families that collide. + // Les tuiles proches n'avaient aucun conflit; garder leur acces simple et associer les LOD grossiers. + Heightfield2D::setThreadColumnCacheLegacyDirectMapped( + options.legacyDirectMappedColumnCache||key.lod<=2); if(!impl_||!impl_->owner||!impl_->shape||!impl_->shape->IsBuilt()) { error="world must be compiled before generating tiles";return std::nullopt; } const auto& world=*impl_->owner; @@ -1342,6 +1347,7 @@ namespace VoxelForgeCore mesher.bStandaloneSkipUvAndColors=options.skipUvAndColors; mesher.bStandaloneSkipUV=options.skipUv;mesher.bStandaloneSkipColors=options.skipColors; mesher.bStandaloneSkipMaterialGroups=options.skipMaterialGroups; + mesher.bStandaloneEnableGradientMemoization=options.memoizeMcGradients; mesher.bGenerateSkirts=world.bGenerateSkirts; mesher.SkirtCells=static_cast(world.SkirtCells) *static_cast(world.OwnerVoxelSizeMicrons) @@ -1436,6 +1442,7 @@ namespace VoxelForgeCore } } int64 sampleCount=0;std::vector densityValues; + FVoxelMarchingCubesStageTimings McStageTimings; TileMesh out;out.key=key; out.diagnostics.featureRefinedResolution=bFeatureRefinedResolution; const bool bHasWorldAssembly=impl_->worldAssembly!=nullptr; @@ -1523,6 +1530,7 @@ namespace VoxelForgeCore out.diagnostics.unclampedFieldCalls=Counts.NativeUnclampedFieldCalls; out.diagnostics.columnCacheHits=Columns.columnHits; out.diagnostics.columnCacheMisses=Columns.columnMisses; + out.diagnostics.columnCacheConflictMisses=Columns.columnConflictMisses; out.diagnostics.classificationMilliseconds=Counts.Milliseconds; out.diagnostics.classificationExclusiveMilliseconds=std::max(0.0, Counts.Milliseconds-classificationColumnComputeMilliseconds); @@ -1670,7 +1678,8 @@ namespace VoxelForgeCore const auto MarchingCubesStart=std::chrono::steady_clock::now(); FVoxelMeshData mesh=mesher.GenerateMesh(origin,step,cells,nullptr, MeshBandMinZVoxel,MeshBandMaxZVoxel,&sampleCount,nullptr,nullptr, - (options.hashDensity||options.captureDensitySamples)?&densityValues:nullptr); + (options.hashDensity||options.captureDensitySamples)?&densityValues:nullptr, + options.profileMesherStages?&McStageTimings:nullptr); const double MarchingCubesWallMilliseconds=std::chrono::duration( std::chrono::steady_clock::now()-MarchingCubesStart).count(); const double ColumnMillisecondsAfterMeshing= @@ -1679,6 +1688,24 @@ namespace VoxelForgeCore MarchingCubesWallMilliseconds-(ColumnMillisecondsAfterMeshing-ColumnMillisecondsBeforeMeshing)); out.diagnostics.marchingCubesColumnComputeMilliseconds=std::max(0.0, ColumnMillisecondsAfterMeshing-ColumnMillisecondsBeforeMeshing); + if(options.profileMesherStages) + { + out.diagnostics.marchingCubesDensityGridMilliseconds=McStageTimings.DensityGridMilliseconds; + out.diagnostics.marchingCubesCellClassificationMilliseconds= + McStageTimings.CellClassificationMilliseconds; + out.diagnostics.marchingCubesGradientNormalsMilliseconds= + McStageTimings.GradientNormalsMilliseconds; + out.diagnostics.marchingCubesEdgeInterpolationMilliseconds= + McStageTimings.EdgeInterpolationMilliseconds; + out.diagnostics.marchingCubesVertexDedupStreamMilliseconds= + McStageTimings.VertexDedupStreamMilliseconds; + out.diagnostics.marchingCubesOtherMilliseconds=McStageTimings.OtherMilliseconds; + out.diagnostics.marchingCubesGradientFallbackDensityCalls= + McStageTimings.GradientFallbackDensityCalls; + out.diagnostics.marchingCubesEdgeMidpointDensityCalls= + McStageTimings.EdgeMidpointDensityCalls; + out.diagnostics.marchingCubesGradientMemoHits=McStageTimings.GradientMemoHits; + } const double MeshingMilliseconds=std::chrono::duration( std::chrono::steady_clock::now()-MeshingStart).count(); out.diagnostics.farHorizonVersion=key.lod>=4?options.farHorizonVersion:0; @@ -1765,6 +1792,7 @@ namespace VoxelForgeCore out.diagnostics.unclampedFieldCalls=Counts.NativeUnclampedFieldCalls; out.diagnostics.columnCacheHits=Columns.columnHits; out.diagnostics.columnCacheMisses=Columns.columnMisses; + out.diagnostics.columnCacheConflictMisses=Columns.columnConflictMisses; out.diagnostics.columnComputeMilliseconds=Columns.columnComputeMilliseconds; out.diagnostics.columnEvaluationExclusiveMilliseconds=Columns.columnComputeMilliseconds; out.diagnostics.classificationMilliseconds=Counts.Milliseconds; diff --git a/Standalone/Core/src/WorldAssemblerInternal.h b/Standalone/Core/src/WorldAssemblerInternal.h index 7cc66b1..2152086 100644 --- a/Standalone/Core/src/WorldAssemblerInternal.h +++ b/Standalone/Core/src/WorldAssemblerInternal.h @@ -1361,11 +1361,35 @@ namespace world_assembly_detail std::int64_t x = 0, y = 0; HeightfieldPointSample column{}; }; - // Coarse LOD tiles revisit a wider set of gradient-sample columns than the full-resolution - // band. Keep those columns resident for a complete worker tile so the 3D mesher does not - // repeat the procedural biome evaluation for every Z sample. + // Keep 65,536 entries in each worker cache; four ways tolerate coordinate families that + // deliberately share low hash bits at coarse LODs. + // Chaque cache worker garde 65 536 entrees; quatre voies absorbent les coordonnees grossieres + // qui partagent les bits faibles du hash. + constexpr std::size_t HeightfieldColumnCacheWays=4; + constexpr std::size_t HeightfieldColumnCacheSetCount=65536/HeightfieldColumnCacheWays; inline thread_local std::array HeightfieldColumnCache{}; + inline thread_local std::array + HeightfieldColumnCacheNextWay{}; inline thread_local HeightfieldColumnCacheStats HeightfieldColumnStats{}; + inline thread_local bool HeightfieldColumnCacheLegacyDirectMapped=false; + + inline std::size_t HeightfieldColumnCacheSet(const WorldAssemblyRecipeV1& recipe, + const WorldAssemblyStratumPlanV1& plan,std::int64_t x,std::int64_t y) noexcept + { + // Avalanche every coordinate bit before masking. The old direct hash kept too little + // entropy from widely spaced coarse LOD coordinates in its low 16 bits. + // Melanger tous les bits de coordonnee avant le masque; l'ancien hash perdait de + // l'entropie sur les coordonnees espacees des LOD grossiers. + std::uint64_t mixed=static_cast(x)*0x9E3779B185EBCA87ull + ^std::rotl(static_cast(y)*0xC2B2AE3D27D4EB4Full,29) + ^recipe.contentHash^std::rotl(plan.contentHash,17); + mixed^=mixed>>30; + mixed*=0xBF58476D1CE4E5B9ull; + mixed^=mixed>>27; + mixed*=0x94D049BB133111EBull; + mixed^=mixed>>31; + return static_cast(mixed)&(HeightfieldColumnCacheSetCount-1); + } inline HeightfieldPointSample EvaluateColumn(const WorldAssemblyRecipeV1& recipe, const WorldAssemblyStratumPlanV1& plan,std::int64_t x,std::int64_t y) @@ -1480,15 +1504,48 @@ namespace world_assembly_detail inline HeightfieldPointSample CachedColumn(const WorldAssemblyRecipeV1& recipe, const WorldAssemblyStratumPlanV1& plan,std::int64_t x,std::int64_t y) { - const std::uint64_t mixed=(static_cast(x)*0x9E3779B185EBCA87ull) - ^std::rotl(static_cast(y)*0xC2B2AE3D27D4EB4Full,23) - ^static_cast(reinterpret_cast(&recipe)) - ^std::rotl(static_cast(reinterpret_cast(&plan)),41); - auto& entry=HeightfieldColumnCache[static_cast(mixed)& - (HeightfieldColumnCache.size()-1)]; - if(entry.recipe==&recipe&&entry.plan==&plan&&entry.recipeHash==recipe.contentHash - &&entry.planHash==plan.contentHash&&entry.x==x&&entry.y==y) - {++HeightfieldColumnStats.columnHits;return entry.column;} + HeightfieldColumnCacheEntry* replacement=nullptr; + if(HeightfieldColumnCacheLegacyDirectMapped) + { + // Preserve S13's exact direct-map index as an in-process negative control. + // Conserver l'index direct exact de S13 comme controle negatif en cours d'execution. + const std::uint64_t mixed=(static_cast(x)*0x9E3779B185EBCA87ull) + ^std::rotl(static_cast(y)*0xC2B2AE3D27D4EB4Full,23) + ^static_cast(reinterpret_cast(&recipe)) + ^std::rotl(static_cast(reinterpret_cast(&plan)),41); + replacement=&HeightfieldColumnCache[static_cast(mixed) + &(HeightfieldColumnCache.size()-1)]; + if(replacement->recipe==&recipe&&replacement->plan==&plan + &&replacement->recipeHash==recipe.contentHash + &&replacement->planHash==plan.contentHash&&replacement->x==x&&replacement->y==y) + {++HeightfieldColumnStats.columnHits;return replacement->column;} + if(replacement->recipe!=nullptr)++HeightfieldColumnStats.columnConflictMisses; + } + else + { + const std::size_t set=HeightfieldColumnCacheSet(recipe,plan,x,y); + const std::size_t first=set*HeightfieldColumnCacheWays; + std::size_t replacementWay=HeightfieldColumnCacheWays; + for(std::size_t way=0;way( + (replacementWay+1)%HeightfieldColumnCacheWays); + replacement=&HeightfieldColumnCache[first+replacementWay]; + } + auto& entry=*replacement; const auto start=std::chrono::steady_clock::now(); HeightfieldPointSample column=EvaluateColumn(recipe,plan,x,y); HeightfieldColumnStats.columnComputeMilliseconds+=std::chrono::duration( @@ -1977,19 +2034,27 @@ void Heightfield2D::resetThreadColumnCacheStats(bool captureCoordinates, bool cl { // Diagnostic probes may clear old tile entries before comparing isolated sample grids. // Les sondes de mesure peuvent vider les entrees d'une tuile avant de comparer deux grilles. - if(clearCache)world_assembly_detail::HeightfieldColumnCache.fill( - world_assembly_detail::HeightfieldColumnCacheEntry{}); + if(clearCache) + { + world_assembly_detail::HeightfieldColumnCache.fill( + world_assembly_detail::HeightfieldColumnCacheEntry{}); + world_assembly_detail::HeightfieldColumnCacheNextWay.fill(0); + } auto& Stats=world_assembly_detail::HeightfieldColumnStats; world_assembly_detail::HeightfieldColumnOctaveDrop=octaveDrop; Stats.pointCalls=0; Stats.columnHits=0; Stats.columnMisses=0; + Stats.columnConflictMisses=0; Stats.columnComputeMilliseconds=0.0; Stats.captureCoordinates=captureCoordinates; Stats.coordinateCaptureComplete=true; Stats.evaluatedCoordinates.clear(); } +void Heightfield2D::setThreadColumnCacheLegacyDirectMapped(bool enabled) noexcept +{ world_assembly_detail::HeightfieldColumnCacheLegacyDirectMapped=enabled; } + const HeightfieldColumnCacheStats& Heightfield2D::threadColumnCacheStats() noexcept { return world_assembly_detail::HeightfieldColumnStats; } diff --git a/Standalone/Research/WorldAssemblerM3/Round18/REPORT.md b/Standalone/Research/WorldAssemblerM3/Round18/REPORT.md new file mode 100644 index 0000000..be27a1e --- /dev/null +++ b/Standalone/Research/WorldAssemblerM3/Round18/REPORT.md @@ -0,0 +1,340 @@ +# World Assembler S14 — Round 18 + +## Decision + +Shipped two output-preserving changes justified by the measurements: + +1. Keep the 65,536-entry XY column cache worker-local. Use the old direct-map route at LOD0–2, + where conflict counts are low, and deterministic four-way sets at LOD3–8. The LOD4 miss spike + falls from 8,224 to 1,437, with conflict misses falling from 7,407 to zero. +2. Memoize exact step-1 ShapeWorld MC grid gradients per worker and tile. The MC two-pass loop, + density grid, halo, central differences, and output values stay intact. LOD0 MC-exclusive cost + falls from 241.87 to 201.48 ns/sample in the corrected-source audit; the original S13 figure was + 243.40 ns/sample. + +No shared XY page was built: cross-worker repeats show opportunity, but no measured page-cache A/B +shows a net cold-path win after the cache fix. The primary cold trace used 21.722 worker-seconds against +the 28.8 s target. Output hashes remain unchanged, so the recipe stays `MarchingCubesV1`; no recipe +bump or matched captures were required. + +## 1. What the 243 ns/sample contains + +The S13 “MC-exclusive” timer excludes MC column evaluation; it does **not** mean “triangle loop +only.” For LOD0, the corrected-source eight-tile audit measured 8.638 ms MC-exclusive plus 3.852 ms +MC column evaluation at the tuned setting. Each LOD0 tile has 35³ = 42,875 density samples. The +exclusive result is 201.48 ns/sample. The cell loop itself is only one part of it. + +In the sampled stage audit, tuned LOD0 p95 was: density grid 8.971 ms (7.071 ms excluding column +evaluation), cell classification 0.143 ms, grid-gradient normals 1.036 ms, edge interpolation +0.080 ms, vertex dedup/stream 0.545 ms, and about 2.168 ms of unassigned remainder. The stage +profiler samples one in eight cells; these are independently calculated p95 values and do not add +to a measured total. Its `Other`/setup-finalization and unassigned-residual fields are computed +remainder buckets from different inclusive/exclusive timers, not separately isolated setup or +allocator timers. + +The 164.93 ns/sample grid remainder is not a standalone `height - z` timer. Every lattice sample +still enters the exact `Heightfield2D::point` path: fixed-coordinate conversion, recipe/plan lookup, +the worker-local XY cache probe, the Z-dependent ground/cap composition, and any applicable exact +feature/edit checks. Column generation itself is counted separately: LOD0 records 44,100 field +calls, 2,244 cache misses (about 94.9% hits), and 89.89 ns/sample of column computation. The audit +does not split cache-hit lookup from Z composition and call overhead. Those Z/feature checks cannot +all move to one XY-column operation. The MC gradient work can be shared per lattice point and is +now memoized exactly; edge interpolation is already limited to crossed edges and the vertex map +deduplicates shared edge vertices. Material and allocation have no separate timers; the mesher's +thread-local grids and dedup map retain capacity between tiles. + +The MC route still uses the protected two-pass loop: pass one rejects empty cells from the eight +cached corners; pass two computes positions, gradients, and emitted triangles only for surface +cells. Classification is per cell; exact gradients are reused per lattice point; interpolation and +deduplication are limited to crossed edges and their shared vertex keys. The stage measurements +place classification in the first pass and interpolation plus dedup/stream in the second. +Material/ground-cap grouping and vertex attributes are folded into +vertex creation and streaming; material work has no separate timer. The 35 edge-midpoint density +probes are negligible beside the gradient fallback calls. + +| LOD | Density grid incl. | Grid excl. columns | Cell classification | Gradient normals | Edge interpolation | Dedup/stream | Unassigned residual | +|---:|---:|---:|---:|---:|---:|---:|---:| +| 0 | 210.11→209.22 | 168.04→164.93 | 3.28→3.32 | 65.10→24.16 | 1.91→1.86 | 12.96→12.72 | 50.40→50.56 | +| 1 | 163.74→164.91 | 163.74→164.91 | 3.82→3.55 | 4.20→4.01 | 2.14→2.10 | 13.47→13.56 | 54.77→55.06 | +| 2 | 390.55→395.47 | 0.00→0.00 | 2.84→2.89 | 11.62→11.51 | 6.14→5.47 | 68.70→69.15 | 277.12→306.20 | +| 3 | 777.21→437.15 | 120.00→131.99 | 3.15→2.97 | 15.37→15.37 | 6.77→6.25 | 604.40→407.99 | 46.97→110.89 | +| 4 | 4,603.98→425.25 | 210.50→178.79 | 2.92→3.32 | 10.15→10.96 | 5.60→5.48 | 565.21→309.90 | 95.44→109.50 | +| 5 | 385.90→434.09 | 135.75→155.07 | 3.02→3.48 | 5.13→7.00 | 1.78→3.50 | 155.02→38.26 | 96.38→137.65 | +| 6 | 366.38→371.92 | 88.76→101.46 | 2.97→3.09 | 18.66→18.11 | 9.72→8.39 | 431.20→185.07 | 124.84→171.70 | +| 7 | 361.29→447.14 | 115.03→102.64 | 3.13→2.99 | 20.47→17.75 | 9.69→8.84 | 443.10→239.22 | 99.92→254.21 | +| 8 | 367.98→377.58 | 95.55→96.05 | 4.05→3.05 | 14.90→19.08 | 7.48→7.92 | 513.15→213.89 | 159.02→234.19 | + +All stage values are p95 **ns/sample**, before→after. “Before” is this build with the direct-map +cache and gradient memo off (`1/0`); “after” is this build's tuned default (`0/1`). Each side measured eight +tiles per LOD, with `hashes_identical=1`. LOD0/1 use 42,875 samples; LOD2–8 use 6,859. Large +variation in dedup/stream and residual is content/triangle-count and sampling noise, not a new +approximation. The last column is the commandlet's unassigned-residual bucket; the separately +reported setup-finalization field is also computed as a remainder and is not an allocator timer. +These warmed-route stage probes explain the hot path; the cold production timings are the +fresh-process trace in section 4. + +| LOD | Candidate tile p95 ms | MC column p95 ms | MC-exclusive p95 ms / ns per sample | Column misses / conflicts | +|---:|---:|---:|---:|---:| +| 0 | 14.079→12.492 | 3.777→3.852 | 10.370→8.638 / 241.87→201.48 | 2,244/33→2,244/33 | +| 1 | 9.824→9.842 | 4.187→4.197 | 7.183→7.307 / 167.53→170.43 | 2,289/28→2,289/28 | +| 2 | 4.715→4.709 | 3.080→3.066 | 1.606→1.616 / 281.61→282.71 | 681/4→681/4 | +| 3 | 6.945→5.393 | 4.016→2.477 | 2.917→2.903 / 425.28→423.27 | 1,634/1,119→574/0 | +| 4 | 36.854→9.227 | 30.216→2.257 | 2.970→2.946 / 432.93→429.54 | 8,224/7,407→1,437/0 | +| 5 | 11.055→9.129 | 3.029→2.278 | 1.279→1.269 / 186.43→184.98 | 1,916/1,476→1,351/0 | +| 6 | 30.703→26.872 | 5.148→3.841 | 2.142→2.000 / 312.23→291.57 | 6,403/2,094→5,570/1 | +| 7 | 20.831→15.390 | 4.725→3.460 | 2.411→2.689 / 351.52→391.97 | 3,472/2,926→1,822/0 | +| 8 | 14.948→9.829 | 5.715→3.358 | 2.534→2.770 / 369.38→403.82 | 2,708/2,215→1,162/0 | + +For LOD0 specifically, the S13 audit row was 14.160 ms candidate, 3.708 ms column evaluation, +10.436 ms MC-exclusive, and 243.40 ns/sample. The tuned corrected-source 35³ route is 12.492 ms +candidate and 8.638 ms MC-exclusive. Subtracting the measured density-grid p95 from candidate p95 +leaves about 3.52 ms for the remaining work (about 82 ns/sample); the independently sampled stage +subtotal is about 3.97 ms (about 93 ns/sample). This is the measured in-world estimate for the +MC/classification/normal/interpolation/streaming work after density-grid evaluation, not a direct +pre-filled-grid benchmark. Treat the p95 subtraction as an estimate, not an additive percentile. + +Gradient memoization reduces LOD0 fallback density probes from 11,474 to 3,018 per sampled p95 +(−73.7%) and normal cost from 65.10 to 24.16 ns/sample. It records 5,610 memo hits. The memo stores +the exact central-difference vector at each grid lattice point, is worker-local, and resets its +validity bytes per tile. The existing grid and per-worker capacities are reused. No material +approximation or altered gradient arithmetic is involved. + +## 2. Why LOD4 had the cache spike + +The old cache already holds 65,536 `HeightfieldPointSample`s per worker, but its direct-map index +uses pointer identity and coordinate bits. Coarse LOD coordinates share sparse low-bit patterns; +the LOD4 16-voxel cap grid interleaves different XY probes with the MC grid and its halo/normal +lookups. The measured direct-map conflict count (7,407 of 8,224 misses) shows this is mostly +eviction, rather than 8,224 necessary unique columns. The audit did not tag every miss by probe +source, so the individual halo-versus-cap contribution is not claimed. + +The new default keeps the same total entries and exact key verification. Its avalanched coordinate +hash selects one of 16,384 four-way sets; deterministic round-robin replacement only changes +residency, never values. Exact recipe/plan identities, content hashes, and XY coordinates are +checked on every hit, so fill order cannot affect output. LOD0–2 keep the fast original path because +they have only 33, 28, and 4 direct-map conflict misses in the audit. LOD6 remains at 5,570 misses +but only one conflict, showing that its remaining cost comes largely from its larger real XY +footprint and connector-shell work, not direct-map collision. + +`voxel.CoreColumnCacheDirectMapped=1` is the direct-map negative control and was read back as `1` +in the trace. With gradient memo still on, the initial set took 6.624 s and 43.493 worker-seconds; +LOD4 used 11.071 worker-seconds. The tuned default took 4.238 s and 21.722 worker-seconds, with +LOD4 at 1.862 worker-seconds. The isolated audit also reports the per-LOD miss deltas above. + +The separate memo negative control was `voxel.CoreMCGradientMemoization=0` with the four-way +column cache still enabled; both values were read back as `0/0`. It took 4.908 s and 27.609 +worker-seconds, versus 4.238 s and 21.722 s at the `0/1` default. Its 3×3 support ring was 0.602 s +versus 0.447 s, and LOD0 last apply was 1.453 s versus 1.171 s. This cold-path A/B supports keeping +the exact gradient memo. The direct-map control is `1/1` and isolates the cache behavior; the memo +control is `0/0` and isolates the gradient behavior. + +## 3. Shared XY pages + +The corrected-source Standalone W8 overlap capture counted 281,910 cross-worker neighboring +duplicate columns at LOD0 and 123,803 at LOD4. Against Round17 (276,658 and 202,684), LOD0 is up +about 2%, while LOD4 is down about 39% after fixing intra-worker cache conflicts. The counters are +schedule-dependent work counts, not world output. Multiplying those counts by the current stage +audit's p95 column-time-per-miss ratios gives a rough estimate of 0.68 worker-seconds across the +LOD0/LOD4 repeats if each repeat avoided a recomputation and the shared lookup had zero cost. That +is about 3.1% of the measured 21.722 worker-seconds, before synchronization and memory costs. This +is an estimate, not a cold-path A/B win measurement. + +After the four-way change, the remaining upper bound is too small to justify adding an unmeasured +synchronized page in this round. It would also add ownership/lifetime and bounded-memory work under +the S8 rules. The page was therefore not built. Existing immutable region pages and their S8 +lifetime are unchanged; no new page lifetime needs proving. + +## 4. Cold 40 m/s trace + +Fresh BuildHost process, 25 cm SurfaceWorld, 40 m/s along +Y, background worker cap 1, ready queue +8, dynamic reserve, collision lookahead 1, `CoreColumnCacheDirectMapped=0`, and +`CoreMCGradientMemoization=1`. The `worker_backend_config` JSONL row reads back all these controls. +Round17 and Round18 use the same trace command and post-`BeginPlay` `elapsed_s` origin. In this run +`elapsed_s=4.238189`; raw trace `t_s` is 4.320372. Trace `t_s` begins 0.082022 s before the +end-of-BeginPlay origin. The 3×3 walkable support ring is measured at its first ready gate using +that same origin. S12's actor report uses a different clock origin; its comparisons are stated as +references, not clock-normalized samples. + +| Metric | Round17 S13 | S14 primary | S14 fresh repeat | +|---|---:|---:|---:| +| Initial set | 2,578 tiles / 4.825 s | 2,578 / 4.238 s | 2,578 / 5.151 s | +| Initial-set worker seconds | 28.760 | 21.722 | 29.969 | +| 3×3 walkable support ring | 0.510 s | 0.447 s | 0.971 s | +| LOD0 last worker end | 1.131 s | 1.156 s | 1.524 s | +| LOD0 last apply | 1.146 s | 1.171 s | 1.532 s | +| 40 m/s moving readiness | 75/75 ready | 75/75 ready | 57/57 logged samples ready | +| Spawn `UWorld::Tick` p95/max | 3.133 / 5.426 ms | 4.200 / 5.571 ms | 4.601 / 6.682 ms | +| Walk `UWorld::Tick` p95/max | 30.314 / 43.142 ms | 32.218 / 45.720 ms | 33.986 / 41.654 ms | + +| Initial-set LOD | Tiles | S14 primary worker s | S14 repeat worker s | +|---:|---:|---:|---:| +| 0 | 512 | 4.520 | 6.319 | +| 1 | 448 | 4.696 | 6.304 | +| 2 | 320 | 0.870 | 1.241 | +| 3 | 272 | 0.829 | 1.193 | +| 4 | 272 | 1.862 | 2.634 | +| 5 | 208 | 1.478 | 2.121 | +| 6 | 144 | 2.179 | 3.168 | +| 7 | 96 | 1.723 | 2.206 | +| 8 | 306 | 3.565 | 4.782 | +| **Total** | **2,578** | **21.722** | **29.969** | + +The same-build LOD0 candidate-tile p95 fell 11.3% (14.079 to 12.492 ms). In the cold trace, LOD0 +worker time fell 3.3% (4.672 to 4.520 s across 512 tiles, or 9.13 to 8.83 ms per tile). + +The primary cold trace meets the 4.8 s full-set, 28.8 worker-second, and 0.51 s support-ring +targets, with all 75 moving samples ready. LOD0 is materially cheaper per tile. LOD0 last apply is +81 ms over the ~1.09 s target. I made one more fresh default trace to check the tick regression; +the same binary and `0/1` CVar readback passed the commandlet but took 5.151 s and 29.969 +worker-seconds for all 2,578 tiles, 0.971 s for the support ring, and 1.532 s for the last LOD0 +apply. Peak private memory was 3.29 GiB. Its spawn tick was 4.601/6.682 ms and walking tick was +33.986/41.654 ms p95/max; all 57 logged 40 m/s streaming samples stayed walkable-ready. Thus the +full-set, worker-second, support-ring, and LOD0 apply targets pass in the primary run but fail on +this repeat. Spawn tick p95/max is worse than S13 and matched S12 in both runs; walking p95 is +worse than both in both runs. The first-run walking max is worse than both references; the repeat +is worse than S12 but below S13. The strict no-worse tick goal therefore fails. I kept the S13 +worker cap at 1; the measurements do not justify raising it. + +## 5. Quality and determinism + +- BuildHost W1/W8/W16 stable: density/mesh/payload/topology + `0cd2a1a4152ee4aa / a06243aee16b65d5 / c700162935c262d7 / 0af901aa42b452bc`. +- Standalone W1/W8/W12, fresh `--force-generate`: whole-view 2,553-tile hashes all match: + `c10697a8f48380c7 / e75c9119a1498cd5 / 49a9a8d1835a6c8c / edcb23288b5e8724`. + Full-detail 960-tile hashes also match: + `0ee0579bd15cbbe4 / 32e5d4964d610fee / 1932b96ee4b1d910 / 69743c6bad3447dc`. +- Core-off at 12.5 cm: `VOXELFORGE_CORE_OFF_LEGACY_PASS`. +- M2 Hybrid V1 and V2 at 25 cm: five analytic fixtures and borders passed. +- SurfaceWorld at 25 cm: BuildHost W1/W8/W16, geometry scale and height agreement, carve/fill, + native/bridge, and collision checks passed. +- S3 far-horizon agreement: max 0.0427 cm against 2 cm. +- S4 far-cap agreement: 128 shared points, max 0.0504 cm against 2 cm; negative control detected. + Cap rays: 1,584/1,584 (100%), no misses. +- FloatingIslands at 25 cm: W1/W8/W16 stable: + `4dcfb37585d7d986 / f14f76fbf065c6ed / 53035c45fd1a082e / 23550100654a4ec3`; + island and span contacts, anchors, carve/fill passed; water remained fixed. + +No density, mesh, topology, payload, or collision output changed. No recipe bump or paired near/far +captures are needed. + +## 6. Build, commands, and evidence + +Builds (Normal priority, maximum 12 parallel workers): + +```powershell +.\Tools\BuildVoxelForgeCoreHost.ps1 -MaxParallelActions 12 -TimeoutSeconds 900 +.\Standalone\Build.ps1 -Target All -CompileWorkers 12 -BuildPriority Normal +``` + +BuildHost feature-stage controls: + +```powershell +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -SharedFeatureAuditOnly -CoreMCStageAudit -CoreColumnCacheDirectMapped 1 -CoreMCGradientMemoization 0 -CoreVoxelSizeCm 25 -TimeoutSeconds 900 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -SharedFeatureAuditOnly -CoreMCStageAudit -CoreColumnCacheDirectMapped 0 -CoreMCGradientMemoization 1 -CoreVoxelSizeCm 25 -TimeoutSeconds 900 +``` + +Cold stream command: + +```powershell +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -StreamingCollisionLookahead 1 -TestMoveDirection '0,1,0' -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 4000 -StreamingReadyQueueDepth 8 -StreamingBackgroundWorkerCap 1 -CoreColumnCacheDirectMapped 0 -CoreMCGradientMemoization 1 -TimeoutSeconds 900 +``` + +The cold-path negative controls held every other argument fixed; the trace JSONL recorded the CVar +readbacks (`0/0` for memo-off, `1/1` for direct-map): + +```powershell +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -StreamingCollisionLookahead 1 -TestMoveDirection '0,1,0' -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 4000 -StreamingReadyQueueDepth 8 -StreamingBackgroundWorkerCap 1 -CoreColumnCacheDirectMapped 0 -CoreMCGradientMemoization 0 -TimeoutSeconds 900 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -StreamingTraceOnly -StreamingOrder Legacy -NearReserveMode Dynamic -CoreVoxelSizeCm 25 -StreamingCollisionLookahead 1 -TestMoveDirection '0,1,0' -CoreRegionConcurrentAdmission 1 -StreamingResultBacklogCap 256 -TestMoveSpeedCmPerSecond 4000 -StreamingReadyQueueDepth 8 -StreamingBackgroundWorkerCap 1 -CoreColumnCacheDirectMapped 1 -CoreMCGradientMemoization 1 -TimeoutSeconds 900 +``` + +Standalone processes were sequential, Normal priority, 12-CPU affinity, and used `--force-generate`: + +```powershell +.\Standalone\bin\VoxelForgeCoreStandalone.exe --workers=1 --world-assembler-prototype --stream-benchmark --stream-pipeline --stream-hash --force-generate --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --far-horizon-lod8-step=0 --process-priority=Normal --process-affinity=4095 --frame-limit=60 +.\Standalone\bin\VoxelForgeCoreStandalone.exe --workers=8 --world-assembler-prototype --stream-benchmark --stream-pipeline --stream-hash --profile-column-overlap --force-generate --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --far-horizon-lod8-step=0 --process-priority=Normal --process-affinity=4095 --frame-limit=60 +.\Standalone\bin\VoxelForgeCoreStandalone.exe --workers=12 --world-assembler-prototype --stream-benchmark --stream-pipeline --stream-hash --force-generate --clip-radius=3 --max-clip-level=8 --camera=0,-35,-487.94,0,-22 --far-horizon-lod8-step=0 --process-priority=Normal --process-affinity=4095 --frame-limit=60 +``` + +Other BuildHost gate commands: + +```powershell +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -CoreOffTest -CoreVoxelSizeCm 12.5 -TimeoutSeconds 900 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -HybridFixturesOnly -CoreMesherRecipe WorldAssemblerHybridV1 -CoreVoxelSizeCm 25 -TimeoutSeconds 900 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -HybridFixturesOnly -CoreMesherRecipe WorldAssemblerHybridV2 -CoreVoxelSizeCm 25 -TimeoutSeconds 900 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -CoreVoxelSizeCm 25 -TimeoutSeconds 900 +.\Tools\TestVoxelForgeCoreWorldHost.ps1 -FloatingIslandsOnly -CoreVoxelSizeCm 25 -TimeoutSeconds 900 +``` + +Measured binaries and SHA-256: + +| Binary | SHA-256 | +|---|---| +| UE `UnrealEditor-Cmd.exe` | `ECCE9996191F7F7E69B3D104EECDD0E678AE69176FDCCD79828C3E75A0654D6D` | +| BuildHost `UnrealEditor-VoxelForge.dll` | `4E55A38547B96221562AEC84D61B0FFC7E455E21420C51E14D2BEACA88217A07` | +| BuildHost `UnrealEditor-VoxelForgeCore.dll` | `1F34F96208FC0E0A72CE2CE80014EBFB110C29C8333053A90735DF9C7BEE7949` | +| BuildHost `UnrealEditor-VoxelForgeEditor.dll` | `9B35350BBA3E7CE4F8D8DA617C216B21CB4CF9869FE5903FD84AAC6906B8187B` | +| Standalone `VoxelForgeCoreStandalone.exe` | `91D29D8BA421304F085BFAB4CCE5EC9DDD32C40BE13F15E118E0DC9606D21B84` | + +Full logs, traces, and Standalone stdout/stderr are under `Saved/BuildHost/`; the current 40 m/s +trace files are in `Saved/BuildHost/HostProject/Plugins/VoxelForge/Saved/CoreWorldValidation/Logs/`. +The fresh repeat uses the same CVar-derived JSONL path as the primary trace, so its JSONL is the +current trace; the faster primary run's engine log summary is retained in that Logs directory as +`CoreWorldValidation-owner_surface_stream_trace_only_applyboost_default_bgcap_1_readyq_8_legacy_order_colcache_4way_gradientmemo_on_nearreserve_dynamic_trace_move_4000_lookahead_1_dir_0_1_0_region_1_backlog_256-backup-2026.10.19-15.44.14.log`. +Per-run Standalone command records and logs are in `Saved/BuildHost/WorldAssemblerM3/Round18/Standalone/`. + +## 7. Failed probes, retries, and safety review + +- `-coremcstageaudit=1` was ignored by Unreal’s boolean argument parser; the audit was repeated + with the bare `-coremcstageaudit` flag. +- An early direct-map control did not read back `1`, and an early audit binary did not log either + new CVar. Those measurements were rejected. I added the S11-style explicit BeginPlay parse and + trace readback, rebuilt BuildHost, then reran the direct/off baseline and tuned audit; the JSONL + readbacks are `1/0` and `0/1` respectively. +- The first corrected shared-audit command was rejected by a harness guard before launch because + it treated audit CVars as trace-only controls. The guard now allows them in shared-feature + audits; the rerun passed. +- The first W8 Standalone wrapper could not resolve its process-local memory helper type. That + run exited 0 and its process peak was 0.70 GiB, but its free-memory preflight was invalid. I + repeated W8 with the helper defined in that process: free-before 17.12 GiB, peak 0.71 GiB, + exit 0, and the whole-view hashes matched W1/W12. +- `Get-CimInstance` was denied and `Get-Counter` lacked the requested counter. I used + `GlobalMemoryStatusEx` instead; the initial available-memory reading was 15.85 GiB. BuildHost + scripts also enforce the 4 GiB preflight, Normal priority, 12-CPU affinity, 8 GiB process cap, + and 900 s timeout. +- Standalone build printed that `vswhere.exe` was not recognized and emitted existing + `VoxelStrateManager.cpp` C4838 narrowing warnings, but built both targets and exited successfully. +- BuildHost UBT passed on its first attempt; the 18 Core link shims passed link-safety inspection. + No `0xC0000142` occurred. No new `UECC-*` folder appeared (the only matching folder predates + this round, from 2026-09-17). +- Final diff review made the gradient memo's per-tile validity reset explicit by clearing every + validity byte after resizing the scratch array. Both targets were rebuilt and the corrected-source + stage, determinism, quality, and cold-trace gates were rerun; hashes stayed identical. + +Diff review: no edits to `Epoch`, either MPSC queue, `EndPlay` draining, worker shutdown checks, +`ModsLock`, immutable region-page lifetime, or game-thread UObject/RMC mutation. The new cache and +gradient scratch remain thread-local. The readable RealtimeMeshComponent source was not modified. +No owner project, editor, asset, generated header, `Binaries/`, or `Intermediate/` source was edited. +No commit, push, or stash. + +## 8. Files and compile spots + +No new source file was added. + +| Changed file | Likely compile/build spot | +|---|---| +| `Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h`, `Standalone/Core/src/VoxelForgeCore.cpp`, `Standalone/Core/src/WorldAssemblerInternal.h` | Standalone Core and BuildHost `VoxelForgeCore` | +| `Standalone/Source/VoxelForge/Public/VoxelMarchingCubesMesher.h`, `Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp` | Standalone executable and BuildHost `VoxelForge` runtime | +| `Source/VoxelForge/Private/VoxelWorld.cpp` | BuildHost `VoxelForge` runtime | +| `Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp` | BuildHost `VoxelForgeEditor` | +| `Tools/TestVoxelForgeCoreWorldHost.ps1` | BuildHost validation harness; no compile | +| `ARCHITECTURE.md`, `CODEMAP.md` | Documentation; no compile | + +### Dialogue + +**Owner:** What is inside the LOD0 MC-exclusive cost, why does LOD4 miss, and is a shared page worth it? + +**Codex:** LOD0’s dominant measured MC-exclusive stage is density-grid evaluation excluding XY +column work (about 165 ns/sample), followed by grid-gradient normals (24 ns/sample after exact +memoization). The cache counters show LOD4’s direct-map spike was chiefly conflicts; four-way sets +remove those while keeping exact samples. Cross-worker repeats remain, but no shared-page A/B +measured a net cold-path win, so I left the S8 page lifetime unchanged. diff --git a/Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp b/Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp index 502b466..936ef02 100644 --- a/Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp +++ b/Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp @@ -120,13 +120,16 @@ FVoxelMeshData UVoxelMarchingCubesMesher::GenerateMesh(FIntVector OriginVoxels, int64* OutDensitySampleCount, FVoxelSharedDensityGrid* OutPreEditDensityGrid, const FVoxelSharedDensityGrid* InSharedDensityGrid, - std::vector* OutDebugDensitySamples) + std::vector* OutDebugDensitySamples, + FVoxelMarchingCubesStageTimings* OutStageTimings) { TRACE_CPUPROFILER_EVENT_SCOPE(VoxelForge_MesherGenerateMesh); + const uint64 StageMesherStartCycles=OutStageTimings?FPlatformTime::Cycles64():0; FVoxelMeshData MeshData; if (OutCaptureGrid) { OutCaptureGrid->Reset(); } if (OutPreEditDensityGrid) { OutPreEditDensityGrid->Reset(); } if (OutDensitySampleCount) { *OutDensitySampleCount = 0; } + if (OutStageTimings) { *OutStageTimings = FVoxelMarchingCubesStageTimings{}; } if (OutDebugDensitySamples) { OutDebugDensitySamples->clear(); } if (!Generator) return MeshData; const bool bEmitUV = !bStandaloneSkipUvAndColors && !bStandaloneSkipUV; @@ -851,6 +854,7 @@ FVoxelMeshData UVoxelMarchingCubesMesher::GenerateMesh(FIntVector OriginVoxels, } // Bande de strate : seules les rangées Z réellement lues (cellules CzLo..CzHi + marges de // gradient) sont échantillonnées — le reste du tampon reste non initialisé et non lu. + const uint64 StageDensityGridStartCycles=OutStageTimings?FPlatformTime::Cycles64():0; VoxelDensityProfile::FScopedTimer MesherDensityGridTimer( VoxelDensityProfile::EBucket::MesherDensityGrid); #if WITH_EDITOR @@ -1250,6 +1254,10 @@ FVoxelMeshData UVoxelMarchingCubesMesher::GenerateMesh(FIntVector OriginVoxels, } } MesherDensityGridTimer.End(); + if(OutStageTimings) + OutStageTimings->DensityGridMilliseconds=static_cast( + FPlatformTime::Cycles64()-StageDensityGridStartCycles) + *FPlatformTime::GetSecondsPerCycle64()*1000.0; if (bUseProofBlocks) { @@ -1345,6 +1353,22 @@ FVoxelMeshData UVoxelMarchingCubesMesher::GenerateMesh(FIntVector OriginVoxels, { return DensityGrid[((gz + 1) * MDim + (gy + 1)) * MDim + (gx + 1)]; }; + // Reuse exact grid-corner gradients between incident cells within this tile. + // Reutiliser les gradients exacts des coins entre cellules voisines de cette tuile. + static thread_local TArray GradientMemo; + static thread_local TArray GradientMemoValid; + const bool bUseGradientMemo=bStandaloneEnableGradientMemoization + &&ShapeWorldContext!=nullptr&&Step==1; + if(bUseGradientMemo) + { + GradientMemo.SetNumUninitialized(DensityGrid.Num()); + GradientMemoValid.SetNumUninitialized(DensityGrid.Num()); + // Clear every tile even when the reused TArray keeps the same length. + // Effacer chaque tuile meme si le TArray reutilise conserve la meme taille. + if(GradientMemoValid.Num()>0) + FMemory::Memzero(GradientMemoValid.GetData(), + static_cast(GradientMemoValid.Num())*sizeof(uint8)); + } auto SampleGradientG = [&](int32 gx, int32 gy, int32 gz) -> float { const float Cached = SampleG(gx, gy, gz); @@ -1353,6 +1377,7 @@ FVoxelMeshData UVoxelMarchingCubesMesher::GenerateMesh(FIntVector OriginVoxels, { return Cached; } + if(OutStageTimings)++OutStageTimings->GradientFallbackDensityCalls; return ShapeWorldContext->DensityAtUnclamped( static_cast(OriginVoxels.X + gx * Step), static_cast(OriginVoxels.Y + gy * Step), @@ -1360,11 +1385,23 @@ FVoxelMeshData UVoxelMarchingCubesMesher::GenerateMesh(FIntVector OriginVoxels, }; auto GradAt = [&](int32 gx, int32 gy, int32 gz) -> FVector { + const int32 CacheIndex=((gz+1)*MDim+(gy+1))*MDim+(gx+1); + if(bUseGradientMemo&&GradientMemoValid[CacheIndex]!=0) + { + if(OutStageTimings)++OutStageTimings->GradientMemoHits; + return GradientMemo[CacheIndex]; + } // Densité négative=solide, positive=air → le gradient pointe vers l'air (sortant). - return FVector( + const FVector Gradient( SampleGradientG(gx + 1, gy, gz) - SampleGradientG(gx - 1, gy, gz), SampleGradientG(gx, gy + 1, gz) - SampleGradientG(gx, gy - 1, gz), SampleGradientG(gx, gy, gz + 1) - SampleGradientG(gx, gy, gz - 1)); + if(bUseGradientMemo) + { + GradientMemo[CacheIndex]=Gradient; + GradientMemoValid[CacheIndex]=1; + } + return Gradient; }; // DC is limited to tagged LOD0 patches. Coarse tiles and a connected patch that reaches a @@ -1401,8 +1438,12 @@ FVoxelMeshData UVoxelMarchingCubesMesher::GenerateMesh(FIntVector OriginVoxels, GroundTris.Reset(); CapTris.Reset(); const bool bProfileMesher = VoxelDensityProfile::IsCycleTimingEnabled(); + const bool bAuditMesherStages=OutStageTimings!=nullptr; uint64 CellClassificationCalls = 0; uint64 SurfaceCellCalls = 0; + uint64 AuditCellClassificationCycles=0,AuditGradientCycles=0; + uint64 AuditInterpolationCycles=0,AuditDedupStreamCycles=0; + uint64 AuditGradientSamples=0,AuditInterpolationSamples=0,AuditDedupStreamSamples=0; uint8 CaseIndexRow[CHUNK_SIZE]{}; uint8 SurfaceRow[CHUNK_SIZE]{}; uint8 SampleClassificationRow[CHUNK_SIZE]{}; @@ -1413,7 +1454,7 @@ FVoxelMeshData UVoxelMarchingCubesMesher::GenerateMesh(FIntVector OriginVoxels, { if (ShouldAbortWork()) return FVoxelMeshData(); FMemory::Memzero(SampleClassificationRow, sizeof(SampleClassificationRow)); - const uint64 ClassificationRowStart = bProfileMesher + const uint64 ClassificationRowStart = (bProfileMesher||bAuditMesherStages) ? FPlatformTime::Cycles64() : 0; int32 ClassificationRowCalls = 0; for (int32 GroupX = 0; GroupX < CellsPerAxis; GroupX += 8) @@ -1454,9 +1495,10 @@ FVoxelMeshData UVoxelMarchingCubesMesher::GenerateMesh(FIntVector OriginVoxels, } CaseIndexRow[CellX] = static_cast(CaseIndex); SurfaceRow[CellX] = MCEdgeTable[CaseIndex] != 0; - SampleClassificationRow[CellX] = bProfileMesher - && VoxelDensityProfile::ShouldSample( - VoxelDensityProfile::EBucket::MesherCellClassification); + SampleClassificationRow[CellX] = + (bProfileMesher&&VoxelDensityProfile::ShouldSample( + VoxelDensityProfile::EBucket::MesherCellClassification)) + ||(bAuditMesherStages&&(CellClassificationCalls%8u)==0u); ++ClassificationRowCalls; ++CellClassificationCalls; } @@ -1477,26 +1519,31 @@ FVoxelMeshData UVoxelMarchingCubesMesher::GenerateMesh(FIntVector OriginVoxels, } CaseIndexRow[CellX] = static_cast(CaseIndex); SurfaceRow[CellX] = MCEdgeTable[CaseIndex] != 0; - SampleClassificationRow[CellX] = bProfileMesher - && VoxelDensityProfile::ShouldSample( - VoxelDensityProfile::EBucket::MesherCellClassification); + SampleClassificationRow[CellX] = + (bProfileMesher&&VoxelDensityProfile::ShouldSample( + VoxelDensityProfile::EBucket::MesherCellClassification)) + ||(bAuditMesherStages&&(CellClassificationCalls%8u)==0u); ++ClassificationRowCalls; ++CellClassificationCalls; } } } - if (bProfileMesher && ClassificationRowCalls > 0) + if ((bProfileMesher||bAuditMesherStages) && ClassificationRowCalls > 0) { const uint64 RowCycles = FPlatformTime::Cycles64() - ClassificationRowStart; - const uint64 PerCellCycles = RowCycles - / static_cast(ClassificationRowCalls); - for (int32 cx = 0; cx < CellsPerAxis; ++cx) + if(bAuditMesherStages)AuditCellClassificationCycles+=RowCycles; + if(bProfileMesher) { - if (SampleClassificationRow[cx]) + const uint64 PerCellCycles = RowCycles + / static_cast(ClassificationRowCalls); + for (int32 cx = 0; cx < CellsPerAxis; ++cx) { - VoxelDensityProfile::AddSampledMeasurement( - VoxelDensityProfile::EBucket::MesherCellClassification, - PerCellCycles); + if (SampleClassificationRow[cx]) + { + VoxelDensityProfile::AddSampledMeasurement( + VoxelDensityProfile::EBucket::MesherCellClassification, + PerCellCycles); + } } } } @@ -1537,9 +1584,11 @@ FVoxelMeshData UVoxelMarchingCubesMesher::GenerateMesh(FIntVector OriginVoxels, } if (bSampleCell) { - VoxelDensityProfile::AddSampledMeasurement( - VoxelDensityProfile::EBucket::MesherGradientNormals, - FPlatformTime::Cycles64() - GradientStart); + const uint64 StageCycles=FPlatformTime::Cycles64()-GradientStart; + if(bProfileMesher)VoxelDensityProfile::AddSampledMeasurement( + VoxelDensityProfile::EBucket::MesherGradientNormals,StageCycles); + if(bAuditMesherStages) + {AuditGradientCycles+=StageCycles;++AuditGradientSamples;} } // Interpolation des positions + normales sur les arêtes traversées. t = point de @@ -1563,6 +1612,7 @@ FVoxelMeshData UVoxelMarchingCubesMesher::GenerateMesh(FIntVector OriginVoxels, if(ShapeWorldContext&&Step==1&&T>=0.375f&&T<=0.625f) { const FVector Midpoint=(Positions[A]+Positions[B])*0.5f; + if(OutStageTimings)++OutStageTimings->EdgeMidpointDensityCalls; const float MidDensity=ShapeWorldContext->DensityAtUnclamped( Midpoint.X/GeometryVoxelSize,Midpoint.Y/GeometryVoxelSize, Midpoint.Z/GeometryVoxelSize); @@ -1611,9 +1661,11 @@ FVoxelMeshData UVoxelMarchingCubesMesher::GenerateMesh(FIntVector OriginVoxels, } if (bSampleCell) { - VoxelDensityProfile::AddSampledMeasurement( - VoxelDensityProfile::EBucket::MesherVertexInterpolation, - FPlatformTime::Cycles64() - VertexInterpolationStart); + const uint64 StageCycles=FPlatformTime::Cycles64()-VertexInterpolationStart; + if(bProfileMesher)VoxelDensityProfile::AddSampledMeasurement( + VoxelDensityProfile::EBucket::MesherVertexInterpolation,StageCycles); + if(bAuditMesherStages) + {AuditInterpolationCycles+=StageCycles;++AuditInterpolationSamples;} } // Génère les triangles avec vertices dédupliqués. @@ -1651,9 +1703,11 @@ FVoxelMeshData UVoxelMarchingCubesMesher::GenerateMesh(FIntVector OriginVoxels, } if (bSampleCell) { - VoxelDensityProfile::AddSampledMeasurement( - VoxelDensityProfile::EBucket::MesherStreamBuilding, - FPlatformTime::Cycles64() - StreamBuildingStart); + const uint64 StageCycles=FPlatformTime::Cycles64()-StreamBuildingStart; + if(bProfileMesher)VoxelDensityProfile::AddSampledMeasurement( + VoxelDensityProfile::EBucket::MesherStreamBuilding,StageCycles); + if(bAuditMesherStages) + {AuditDedupStreamCycles+=StageCycles;++AuditDedupStreamSamples;} } } } @@ -1989,5 +2043,31 @@ FVoxelMeshData UVoxelMarchingCubesMesher::GenerateMesh(FIntVector OriginVoxels, OutputAllocatedBytes); } + if(OutStageTimings) + { + const double CycleToMs=FPlatformTime::GetSecondsPerCycle64()*1000.0; + const auto ScaleSampledCycles=[CycleToMs](uint64 Cycles,uint64 Samples,uint64 Calls) + { + return Samples>0?static_cast(Cycles)*static_cast(Calls) + /static_cast(Samples)*CycleToMs:0.0; + }; + OutStageTimings->CellClassificationMilliseconds= + static_cast(AuditCellClassificationCycles)*CycleToMs; + OutStageTimings->GradientNormalsMilliseconds=ScaleSampledCycles( + AuditGradientCycles,AuditGradientSamples,SurfaceCellCalls); + OutStageTimings->EdgeInterpolationMilliseconds=ScaleSampledCycles( + AuditInterpolationCycles,AuditInterpolationSamples,SurfaceCellCalls); + OutStageTimings->VertexDedupStreamMilliseconds=ScaleSampledCycles( + AuditDedupStreamCycles,AuditDedupStreamSamples,SurfaceCellCalls); + const double TotalMilliseconds=static_cast( + FPlatformTime::Cycles64()-StageMesherStartCycles)*CycleToMs; + OutStageTimings->OtherMilliseconds=FMath::Max(0.0,TotalMilliseconds + -OutStageTimings->DensityGridMilliseconds + -OutStageTimings->CellClassificationMilliseconds + -OutStageTimings->GradientNormalsMilliseconds + -OutStageTimings->EdgeInterpolationMilliseconds + -OutStageTimings->VertexDedupStreamMilliseconds); + } + return MeshData; } diff --git a/Standalone/Source/VoxelForge/Public/VoxelMarchingCubesMesher.h b/Standalone/Source/VoxelForge/Public/VoxelMarchingCubesMesher.h index 8270e50..056dcac 100644 --- a/Standalone/Source/VoxelForge/Public/VoxelMarchingCubesMesher.h +++ b/Standalone/Source/VoxelForge/Public/VoxelMarchingCubesMesher.h @@ -41,6 +41,21 @@ struct FVoxelSharedDensityGrid } }; +// Optional, sampled timing output for the research audit; null in production calls. +// Mesures echantillonnees facultatives pour l'audit; pointeur nul en production. +struct FVoxelMarchingCubesStageTimings +{ + double DensityGridMilliseconds = 0.0; + double CellClassificationMilliseconds = 0.0; + double GradientNormalsMilliseconds = 0.0; + double EdgeInterpolationMilliseconds = 0.0; + double VertexDedupStreamMilliseconds = 0.0; + double OtherMilliseconds = 0.0; + uint64 GradientFallbackDensityCalls = 0; + uint64 EdgeMidpointDensityCalls = 0; + uint64 GradientMemoHits = 0; +}; + // Stable plan data copied onto one worker-owned mesher for hybrid LOD0 generation. // Donnees stables du plan copiees dans un mailleur de worker pour le mode hybride LOD0. struct FVoxelHybridSharpFeatureBox @@ -91,7 +106,8 @@ public: int64* OutDensitySampleCount = nullptr, FVoxelSharedDensityGrid* OutPreEditDensityGrid = nullptr, const FVoxelSharedDensityGrid* InSharedDensityGrid = nullptr, - std::vector* OutDebugDensitySamples = nullptr); + std::vector* OutDebugDensitySamples = nullptr, + FVoxelMarchingCubesStageTimings* OutStageTimings = nullptr); //========================================================================= // SERVICES (injectés par AVoxelWorld) @@ -161,6 +177,7 @@ public: bool bStandaloneSkipUV = false; bool bStandaloneSkipColors = false; bool bStandaloneSkipMaterialGroups = false; + bool bStandaloneEnableGradientMemoization = true; // Seuil de l'isosurface dans le champ de densité. // Convention MC: densité < IsoLevel = solide, >= = air. diff --git a/Tools/TestVoxelForgeCoreWorldHost.ps1 b/Tools/TestVoxelForgeCoreWorldHost.ps1 index 7f4a3e0..c901890 100644 --- a/Tools/TestVoxelForgeCoreWorldHost.ps1 +++ b/Tools/TestVoxelForgeCoreWorldHost.ps1 @@ -15,6 +15,8 @@ param( [ValidateRange(0, 4096)][int]$StreamingResultBacklogCap = 256, [ValidateRange(0, 10)][int]$StreamingReadyQueueDepth = 8, [ValidateRange(0, 1)][int]$StreamingBackgroundWorkerCap = 1, + [ValidateRange(0, 1)][int]$CoreColumnCacheDirectMapped = 0, + [ValidateRange(0, 1)][int]$CoreMCGradientMemoization = 1, [ValidateRange(0, 900)][int]$Step1MaxMeshApplies = 0, [ValidateRange(0, 900)][int]$TestTeleportAfterSeconds = 0, [ValidateRange(0, 1000000)][int]$TestTeleportDistanceCm = 0, @@ -29,6 +31,7 @@ param( [switch]$CoreOffTest, [switch]$CorePlanOnly, [switch]$SharedFeatureAuditOnly, + [switch]$CoreMCStageAudit, [int]$CoreTestSeed = [int]::MinValue ) @@ -105,6 +108,10 @@ if (-not $StreamingTraceOnly -and ($TestMoveSpeedCmPerSecond -ne 600 -or $TestTeleportDistanceCm -ne 0)) { throw 'Streaming, backlog, apply and teleport controls require -StreamingTraceOnly.' } +if (-not $StreamingTraceOnly -and -not $SharedFeatureAuditOnly -and + ($CoreColumnCacheDirectMapped -ne 0 -or $CoreMCGradientMemoization -ne 1)) { + throw 'Column-cache and gradient-memoization controls require a streaming or shared-feature audit.' +} if (($TestTeleportAfterSeconds -eq 0) -ne ($TestTeleportDistanceCm -eq 0)) { throw 'Set both -TestTeleportAfterSeconds and -TestTeleportDistanceCm for the teleport probe.' } @@ -180,7 +187,13 @@ if ($SharedFeatureAuditOnly) { if ($SelectedModeCount -ne 0) { throw '-SharedFeatureAuditOnly uses the SurfaceWorld owner route.' } $RunName = 'shared_feature_audit' } +if ($CoreMCStageAudit -and -not $SharedFeatureAuditOnly) { + throw '-CoreMCStageAudit requires -SharedFeatureAuditOnly.' +} +if ($CoreMCStageAudit) { $RunName += '_mc_stages' } $RunName += if ($StreamingOrder -eq 'Legacy') { '_legacy_order' } else { '_coarse_first' } +$RunName += if ($CoreColumnCacheDirectMapped -eq 1) { '_colcache_direct' } else { '_colcache_4way' } +$RunName += if ($CoreMCGradientMemoization -eq 1) { '_gradientmemo_on' } else { '_gradientmemo_off' } $RunName += switch ($NearReserveMode) { 'Legacy' { '_nearreserve_legacy' } 'Dynamic' { '_nearreserve_dynamic' } @@ -328,6 +341,8 @@ $BaseArguments += "-voxel.CoreRegionConcurrentAdmission=$CoreRegionConcurrentAdm $BaseArguments += "-voxel.StreamingResultBacklogCap=$StreamingResultBacklogCap" $BaseArguments += "-voxel.StreamingReadyQueueDepth=$StreamingReadyQueueDepth" $BaseArguments += "-voxel.StreamingBackgroundWorkerCap=$StreamingBackgroundWorkerCap" +$BaseArguments += "-voxel.CoreColumnCacheDirectMapped=$CoreColumnCacheDirectMapped" +$BaseArguments += "-voxel.CoreMCGradientMemoization=$CoreMCGradientMemoization" if ($Step1MaxMeshApplies -gt 0) { $BaseArguments += "-voxel.Step1MaxMeshApplies=$Step1MaxMeshApplies" } @@ -350,6 +365,7 @@ if ($StartupTrace -or $StreamingTraceOnly) { $BaseArguments += '-voxel.StartupTraceMaxTiles=16384' } if ($SharedFeatureAuditOnly) { $BaseArguments += '-corefeatureauditonly' } +if ($CoreMCStageAudit) { $BaseArguments += '-coremcstageaudit' } if ($HasManifestOverride) { $BaseArguments += ('-coreworldassembly="{0}"' -f $StagedWorldAssembly) }