tools(profile): the instrument lied by 11.5x -- now it checks itself, and batching kills startup cost

0 Fork 0

Overnight task 1 of the owner's mandate: "better diagnostic, profiler and tools,
so we can batch test way more quicker."

WHY THIS CAME BEFORE ANY MORE OPTIMISATION
ee89c4d delivered a 5x candidate reduction and a 16x cheaper support floor for
0% end-to-end gain, because the profiler inflated by ~11.5x (14.74 us/sample
profiled vs 1.28 us/voxel real). Two rounds of bucket rankings were artefacts of
FPlatformTime::Cycles64() wrapped around ~100 ns operations -- the timer pair
calibrates at ~0.1 us, so it was measuring itself.

PROFILER
Deterministic count sampling plus amortised block timing every 64 density calls.
Coarse counters (calls, candidates) stay always-on -- they were never the problem.
Timing-only scopes are opt-in behind -profiledensityfull. UE CPU trace scopes added.

  before   profiled 14.74 us/sample  vs  1.28 us/voxel real   ~11.5x
  after    profiled 0.463050 s       vs  0.558470 s unprofiled   -17.09%
           geometry 235AC2DC in both

⭐ Every profiled report now prints profiled total, unprofiled total, discrepancy,
ratio and geometry equality, and EXITS CODE 2 if the comparison fails. An
intermediate run hit -22.6% and failed loudly, which is the point: this class of
bug must be impossible to miss rather than something noticed three rounds later.

⚠️ The -17% residual is run-to-run noise, so ~17% is our measurement floor.
Improvements smaller than that cannot be claimed from a single pair of runs.

BATCH EXPLORER
One process, many cases, one JSON. Startup was ~25-30 s of engine boot for ~3 s of
work, which is what made every iteration expensive.

  3 cases in 6.59 s of case time
  standalone vs batched, same binary: hash 235AC2DC, 2,461 tris, 343,000 tunnel
  lookups, 554,381 passage evaluations -- identical on both

-out= is enforced absolute under the plugin's own Saved/.

CACHE BREAKDOWN (the 272 KB/entry question)
The slot table is small. The weight is spatial-index bins and candidate arrays,
room arrays, support-column arrays and tunnel-world indices -- i.e. much of the
411 MiB is the spatial index ee89c4d added, which also bought 0% end-to-end.
Worth revisiting on those terms.

RESEARCH (recorded, not acted on) -- CODEX-TASK-1-PROFILER-BATCH-RESEARCH.md
Ranked follow-ups: deterministic empty-space early-out, density-grid reuse, task
granularity (ParallelForWithExistingTaskContext + MinBatchSize), SIMD MC (~2x over
optimised scalar, but cross-machine determinism is the risk), sparse hierarchy,
RMC partial updates, Transvoxel LOD seams. Sources cited in the doc.
⚠️ Upstream RMC is 5.3.2; the owner's copy identifies as 5.4 with custom
modifications. No upgrade is safe. Left untouched.

PUSHBACK, accepted: "buckets must sum to their parent" needs a parallel-time
definition -- summing worker CPU is not wall time. Reports now separate worker CPU
sum from wall-clock estimate.

End-to-end unchanged, as expected for a tooling change: 2.746 / 2.691 s on the
128^3 seed 1337 region (was 2.678-2.705). Build clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb

This commit is contained in:

Fr0zkaandClaude Opus 5 committed 2026-09-09 05:02:21 +02:00

1 parent ee89c4d8c2

commit e63f016835

10 files changed

+1494 -124

No files matched your search

Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp

+839 -28

View File

File diff suppressed because it is too large. Load diff