tools(profile): the instrument lied by 11.5x -- now it checks itself, and batching kills startup cost
Overnight task 1 of the owner's mandate: "better diagnostic, profiler and tools, so we can batch test way more quicker." WHY THIS CAME BEFORE ANY MORE OPTIMISATIONee89c4ddelivered a 5x candidate reduction and a 16x cheaper support floor for 0% end-to-end gain, because the profiler inflated by ~11.5x (14.74 us/sample profiled vs 1.28 us/voxel real). Two rounds of bucket rankings were artefacts of FPlatformTime::Cycles64() wrapped around ~100 ns operations -- the timer pair calibrates at ~0.1 us, so it was measuring itself. PROFILER Deterministic count sampling plus amortised block timing every 64 density calls. Coarse counters (calls, candidates) stay always-on -- they were never the problem. Timing-only scopes are opt-in behind -profiledensityfull. UE CPU trace scopes added. before profiled 14.74 us/sample vs 1.28 us/voxel real ~11.5x after profiled 0.463050 s vs 0.558470 s unprofiled -17.09% geometry 235AC2DC in both ⭐ Every profiled report now prints profiled total, unprofiled total, discrepancy, ratio and geometry equality, and EXITS CODE 2 if the comparison fails. An intermediate run hit -22.6% and failed loudly, which is the point: this class of bug must be impossible to miss rather than something noticed three rounds later. ⚠️ The -17% residual is run-to-run noise, so ~17% is our measurement floor. Improvements smaller than that cannot be claimed from a single pair of runs. BATCH EXPLORER One process, many cases, one JSON. Startup was ~25-30 s of engine boot for ~3 s of work, which is what made every iteration expensive. 3 cases in 6.59 s of case time standalone vs batched, same binary: hash 235AC2DC, 2,461 tris, 343,000 tunnel lookups, 554,381 passage evaluations -- identical on both -out= is enforced absolute under the plugin's own Saved/. CACHE BREAKDOWN (the 272 KB/entry question) The slot table is small. The weight is spatial-index bins and candidate arrays, room arrays, support-column arrays and tunnel-world indices -- i.e. much of the 411 MiB is the spatial indexee89c4dadded, which also bought 0% end-to-end. Worth revisiting on those terms. RESEARCH (recorded, not acted on) -- CODEX-TASK-1-PROFILER-BATCH-RESEARCH.md Ranked follow-ups: deterministic empty-space early-out, density-grid reuse, task granularity (ParallelForWithExistingTaskContext + MinBatchSize), SIMD MC (~2x over optimised scalar, but cross-machine determinism is the risk), sparse hierarchy, RMC partial updates, Transvoxel LOD seams. Sources cited in the doc. ⚠️ Upstream RMC is 5.3.2; the owner's copy identifies as 5.4 with custom modifications. No upgrade is safe. Left untouched. PUSHBACK, accepted: "buckets must sum to their parent" needs a parallel-time definition -- summing worker CPU is not wall time. Reports now separate worker CPU sum from wall-clock estimate. End-to-end unchanged, as expected for a tooling change: 2.746 / 2.691 s on the 128^3 seed 1337 region (was 2.678-2.705). Build clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
This commit is contained in:
1 parent
ee89c4d8c2
commit
e63f016835
10 files changed
+1494
-124
No files matched your search
@@ -3293,7 +3293,7 @@ namespace
|
||||
&& SharedCache->RegionSize == RegionSize;
|
||||
if (!bCacheMatches)
|
||||
{
|
||||
if (VoxelDensityProfile::IsEnabled())
|
||||
if (VoxelDensityProfile::AreCountersEnabled())
|
||||
{
|
||||
VoxelDensityProfile::AddCounter(
|
||||
VoxelDensityProfile::ECounter::SdfCacheBuild);
|
||||
@@ -3844,7 +3844,7 @@ namespace
|
||||
FState& S = State();
|
||||
const FVector& Position = S.LastWorldPosition;
|
||||
const FChunkSDFCache& Cache = GetCache();
|
||||
if (VoxelDensityProfile::IsEnabled())
|
||||
if (VoxelDensityProfile::AreCountersEnabled())
|
||||
{
|
||||
VoxelDensityProfile::AddCounter(
|
||||
VoxelDensityProfile::ECounter::TunnelSupportFloorQueries);
|
||||
@@ -5534,6 +5534,7 @@ void VoxelDensityOps::ReportWorkerRoomGraphCacheFootprint()
|
||||
uint64 EntryBytes = 0;
|
||||
uint64 LargestEntryBytes = 0;
|
||||
uint64 ValidEntries = 0;
|
||||
VoxelDensityProfile::FCacheMemoryBreakdown Breakdown;
|
||||
for (int32 Index = 0; Index < RoomGraphCacheSlotCount; ++Index)
|
||||
{
|
||||
const FRoomGraphCacheEntry& Entry = GRoomGraphCache[Index];
|
||||
@@ -5543,6 +5544,7 @@ void VoxelDensityOps::ReportWorkerRoomGraphCacheFootprint()
|
||||
}
|
||||
|
||||
++ValidEntries;
|
||||
Breakdown += Entry.Cache.GetAllocatedSizeBreakdown();
|
||||
const uint64 EntryDynamicBytes = static_cast<uint64>(Entry.Cache.GetAllocatedSize());
|
||||
const uint64 FullEntryBytes = static_cast<uint64>(sizeof(FRoomGraphCacheEntry))
|
||||
+ EntryDynamicBytes;
|
||||
@@ -5550,6 +5552,7 @@ void VoxelDensityOps::ReportWorkerRoomGraphCacheFootprint()
|
||||
EntryBytes += FullEntryBytes;
|
||||
LargestEntryBytes = FMath::Max(LargestEntryBytes, FullEntryBytes);
|
||||
}
|
||||
Breakdown.SlotStorageBytes = static_cast<uint64>(sizeof(GRoomGraphCache));
|
||||
|
||||
VoxelDensityProfile::SetWorkerRoomGraphCacheFootprint(
|
||||
RoomGraphCacheSlotCount,
|
||||
@@ -5558,7 +5561,8 @@ void VoxelDensityOps::ReportWorkerRoomGraphCacheFootprint()
|
||||
DynamicBytes,
|
||||
EntryBytes,
|
||||
LargestEntryBytes,
|
||||
ValidEntries);
|
||||
ValidEntries,
|
||||
Breakdown);
|
||||
}
|
||||
|
||||
//=============================================================================
|
||||
|
||||
Reference in new issue
Block a user