Fr0zka/VoxelForge
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

perf(gen): a thrashing one-slot cache cost 18.7x -- and the world is unchanged

Browse Source 
TunnelNetwork density evaluation was 45x more expensive than Maze for identical
work: 2,097,152 voxels, 64 tiles, same mesher, same machine. Profiling killed
every hypothesis I brought to it -- box classification skipping 0/64 tiles was
true but irrelevant (export bypasses classification), near-surface early-outs
fired at 49.7%, the room cache built 340 entries for 2.74M samples, and the
per-room overrides resolved once per sample rather than once per modifier.

The cause was the wrapper, not the stack. A mesher tile samples an EXPANDED grid
(-1..Cells+1), so one worker crosses several exact chunk coordinates while
meshing a single tile. The generator's cache held only the last coordinate, so
every boundary crossing rebuilt the resolved parameters, the whole 22-operator
stack and the tunnel-core cache. On one representative tile: 3,675 refetches,
38.0s. The entire operator stack was 18.5s of a 4,272s total -- 0.43%.

Replaced with a 128-slot direct-mapped thread_local cache keyed on owner, chunk,
layout version, params fingerprint and generator type. Collisions force a safe
rebuild, never a stale read. No shared mutable state, no order-dependent
reduction.

  TunnelNetwork  344.5s / 164.267 us/vox  ->  18.4s / 8.763 us/vox   (18.7x)
  Maze             7.4s /   3.535 us/vox  ->   1.08s / 0.513 us/vox  ( 6.9x)
  same tile        38.0s                  ->   0.51s ; refetches 3675 -> 27

Output is BYTE-IDENTICAL, before vs after and across repeats, both archetypes:
TunnelNetwork 50d6b4d0..f23126, Maze 364f7489..2fbe99. The permission to change
the field was not needed.

Also in this commit, from the two preceding tasks:

- Explorer renders from the MESH instead of raymarching. It meshes the region
  once through the canonical UVoxelMarchingCubesMesher, then CPU-rasterises via
  a BVH. A viewpoint went from ~16 minutes to 0.155s; 8 views now cost 1.25s of
  drawing. No render pixel samples the density field.
- One sampled grid and player-fit mask per run, shared by render/walk/export,
  instead of the same 368s walk computed twice.
- A wall-clock budget (25 min default, hard 30 max) that stops cleanly and
  records truncated/truncated_during/completed_modes rather than overshooting.
- ParallelFor across the measurement sampling; the suite was ~10% of a 16-thread
  machine.
- -surfaceroughness= on the commandlet, and a surface-relief metric in export
  mode (edge lengths, dihedral distribution, face-vs-smooth normal divergence)
  so "does this look like rock" is a number, not a screenshot argument.
- Smooth vertex normals in the rasteriser: FVoxelMeshData::Normals is the
  density-gradient normal the engine shades with, and the raster path was
  recomputing a face normal over it.
- ScaleDiagnosis: baked candidate indices are now derived from the current
  corpus, so a legitimate field change stops breaking the test.

NOT verified, and not claimed:
- TunnelNetworkSpineEquivalence exceeded the 30-minute cap and was stopped.
  Inconclusive, not passing. The identical hashes are strong evidence the
  op-stack/legacy equivalence holds, but the test has not confirmed it.
- CrossPlatformDigest passes but reports 2 samples within 1e-6 of the
  isosurface; FMath::Sin/Cos leaves Windows/Linux equivalence unproven.
  Pre-existing, not caused by this change.
- The cache's memory footprint (128 entries x ~16 workers) is unmeasured.

What the roughness sweep settled, for the record: SurfaceRoughness 0 -> 8 moves
cost by 0.3% (163.2 / 163.2 / 163.7 us), so roughness was never the cost. It
does reach the surface (dihedral p50 1.1 -> 2.2 -> 5.9), which kills my claim
that the density-channel operator contributed nothing. TunnelNetwork's hard
seams are present at roughness 0 (4.4% of edges past 40 deg vs Maze's 1.0%), so
they are structural -- in how primitives are unioned -- not a roughness
deficiency. More roughness makes them worse, not better.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
This commit is contained in:
Fr0zkaandClaude Opus 5 committed 2026-09-08 20:31:56 +02:00
1 parent d97373c5a8
commit e08b49a183
7 files changed
+2274 -450
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

Source/VoxelForge/Private/VoxelStrateMeasure.cpp
+127 -33
View File
@@ -455,6 +455,8 @@ namespace VoxelStrateMeasurePrivate
// only when it explicitly requested a capture.
Destination.Air = MoveTemp(Source.Air);
Destination.Density = MoveTemp(Source.Density);
Destination.PlayerFitMask.Reset();
Destination.PlayerFitCellCount = 0;
}
void DecodeIndex(const FSampleGrid& Grid, int32 Index, int32& OutX, int32& OutY, int32& OutZ)
@@ -1374,6 +1376,37 @@ namespace VoxelStrateMeasurePrivate
? 0.0f : static_cast<float>(MinimumDistance);
}
int64 CountAirCells(
const FSampleGrid& Grid,
const TArray<uint8>* EligibilityMask = nullptr)
{
TArray<int64> AirCountsByZ;
AirCountsByZ.Init(0, Grid.NumZ);
ParallelFor(Grid.NumZ, [&](int32 Z)
{
int64 Count = 0;
for (int32 Y = 0; Y < Grid.NumY; ++Y)
{
for (int32 X = 0; X < Grid.NumX; ++X)
{
const int32 Cell = Grid.Index(X, Y, Z);
Count += Grid.Air[Cell] != 0u
&& (EligibilityMask == nullptr || (*EligibilityMask)[Cell] != 0u)
? 1
: 0;
}
}
AirCountsByZ[Z] = Count;
});
int64 Total = 0;
for (int32 Z = 0; Z < Grid.NumZ; ++Z)
{
Total += AirCountsByZ[Z];
}
return Total;
}
void BuildPlayerFitMetrics(
const FSampleGrid& Grid,
const FVoxelStrateMeasureSettings& Settings,
@@ -1453,10 +1486,7 @@ namespace VoxelStrateMeasurePrivate
OutMetrics.SampledMinY = Grid.MinY;
OutMetrics.SampledMaxY = Grid.MaxY;
OutMetrics.NumSampled = Grid.CellCount;
for (const uint8 bAir : Grid.Air)
{
OutMetrics.NumAir += bAir != 0u ? 1 : 0;
}
OutMetrics.NumAir = CountAirCells(Grid);
OutMetrics.NumSolid = static_cast<int64>(Grid.CellCount) - OutMetrics.NumAir;
OutMetrics.AirFraction = OutMetrics.NumSampled > 0
? static_cast<float>(static_cast<double>(OutMetrics.NumAir)
@@ -1487,11 +1517,7 @@ namespace VoxelStrateMeasurePrivate
TArray<int32> Components;
int32 NumComponents = 0;
int64 NumAir = 0;
for (const uint8 bAir : Grid.Air)
{
NumAir += bAir != 0u ? 1 : 0;
}
const int64 NumAir = CountAirCells(Grid);
int64 LargestComponentCells = 0;
int32 LargestComponentLowestCell = INDEX_NONE;
@@ -1537,8 +1563,21 @@ namespace VoxelStrateMeasurePrivate
TArray<uint8> WalkableColumns;
WalkableColumns.Init(0u, static_cast<int32>(XYColumnCount64));
for (int32 Z = 0; Z < Grid.NumZ; ++Z)
// Candidate poses read only the immutable sampled field. Keep every worker's clearance
// values and compact column flags private, then merge by fixed Z order so the median and
// projected surface graph remain deterministic.
TArray<int64> WalkableCountsByZ;
WalkableCountsByZ.Init(0, Grid.NumZ);
TArray<TArray<int32>> ClearanceValuesByZ;
ClearanceValuesByZ.SetNum(Grid.NumZ);
TArray<TArray<uint8>> WalkableColumnsByZ;
WalkableColumnsByZ.SetNum(Grid.NumZ);
ParallelFor(Grid.NumZ, [&](int32 Z)
{
TArray<int32>& SliceClearanceValues = ClearanceValuesByZ[Z];
TArray<uint8>& SliceWalkableColumns = WalkableColumnsByZ[Z];
SliceWalkableColumns.Init(0u, static_cast<int32>(XYColumnCount64));
int64 SliceWalkableCount = 0;
for (int32 Y = 0; Y < Grid.NumY; ++Y)
{
for (int32 X = 0; X < Grid.NumX; ++X)
@@ -1563,8 +1602,8 @@ namespace VoxelStrateMeasurePrivate
}
if (!bHasHeadroom) continue;
++NumWalkable;
WalkableColumns[X + Grid.NumX * Y] = 1u;
++SliceWalkableCount;
SliceWalkableColumns[X + Grid.NumX * Y] = 1u;
int32 ClearanceCells = 0;
while (static_cast<int64>(Z) + ClearanceCells < Grid.NumZ
&& Grid.Air[Grid.Index(X, Y, Z + ClearanceCells)] != 0u)
@@ -1573,10 +1612,25 @@ namespace VoxelStrateMeasurePrivate
}
const int64 ClearanceVoxels = static_cast<int64>(ClearanceCells)
* Settings.SampleStep;
ClearanceValues.Add(ClearanceVoxels > INT32_MAX
SliceClearanceValues.Add(ClearanceVoxels > INT32_MAX
? INT32_MAX : static_cast<int32>(ClearanceVoxels));
}
}
WalkableCountsByZ[Z] = SliceWalkableCount;
});
for (int32 Z = 0; Z < Grid.NumZ; ++Z)
{
NumWalkable += WalkableCountsByZ[Z];
const TArray<uint8>& SliceWalkableColumns = WalkableColumnsByZ[Z];
for (int32 Column = 0; Column < SliceWalkableColumns.Num(); ++Column)
{
if (SliceWalkableColumns[Column] != 0u)
{
WalkableColumns[Column] = 1u;
}
}
ClearanceValues.Append(ClearanceValuesByZ[Z]);
}
InOutMetrics.WalkableFraction = NumAir > 0
@@ -1654,15 +1708,29 @@ namespace VoxelStrateMeasurePrivate
TArray<int32> Distances;
BuildManhattanDistanceToSolid(Grid, Distances);
TArray<TArray<float>> FeatureScalesByZ;
FeatureScalesByZ.SetNum(Grid.NumZ);
ParallelFor(Grid.NumZ, [&](int32 Z)
{
TArray<float>& SliceFeatureScales = FeatureScalesByZ[Z];
for (int32 Y = 0; Y < Grid.NumY; ++Y)
{
for (int32 X = 0; X < Grid.NumX; ++X)
{
const int32 Index = Grid.Index(X, Y, Z);
if (Grid.Air[Index] != 0u)
{
SliceFeatureScales.Add(static_cast<float>(Distances[Index])
* static_cast<float>(Settings.SampleStep));
}
}
}
});
TArray<float> FeatureScales;
FeatureScales.Reserve(static_cast<int32>(NumAir));
for (int32 Index = 0; Index < Grid.CellCount; ++Index)
for (int32 Z = 0; Z < Grid.NumZ; ++Z)
{
if (Grid.Air[Index] != 0u)
{
FeatureScales.Add(static_cast<float>(Distances[Index])
* static_cast<float>(Settings.SampleStep));
}
FeatureScales.Append(FeatureScalesByZ[Z]);
}
InOutMetrics.MedianFeatureScale = MedianFloat(FeatureScales);
}
@@ -2220,13 +2288,7 @@ namespace VoxelStrateMeasurePrivate
int64 LargestComponentCells = 0;
int32 LargestComponentLowestCell = INDEX_NONE;
int32 NumComponentsAtLeast1Pct = 0;
int64 NumAir = 0;
for (int32 Index = 0; Index < Grid.CellCount; ++Index)
{
NumAir += Grid.Air[Index] != 0u
&& (EligibilityMask == nullptr || (*EligibilityMask)[Index] != 0u)
? 1 : 0;
}
const int64 NumAir = CountAirCells(Grid, EligibilityMask);
FloodFillAir(
Grid,
Components,
@@ -2284,10 +2346,16 @@ namespace VoxelStrateMeasurePrivate
OutDiagnostics.StartComponentCells == LargestComponentCells;
OutDiagnostics.bGoalComponentIsLargest =
OutDiagnostics.GoalComponentCells == LargestComponentCells;
OutDiagnostics.StartToGoalComponentDistanceCells = DistanceToComponent(
Grid, Start, GoalComponent, Components);
OutDiagnostics.GoalToStartComponentDistanceCells = DistanceToComponent(
Grid, Goal, StartComponent, Components);
TArray<float> ComponentDistances;
ComponentDistances.Init(0.0f, 2);
ParallelFor(2, [&](int32 DistanceIndex)
{
ComponentDistances[DistanceIndex] = DistanceIndex == 0
? DistanceToComponent(Grid, Start, GoalComponent, Components)
: DistanceToComponent(Grid, Goal, StartComponent, Components);
});
OutDiagnostics.StartToGoalComponentDistanceCells = ComponentDistances[0];
OutDiagnostics.GoalToStartComponentDistanceCells = ComponentDistances[1];
}
EVoxelConnectivityResult QueryConnectivity(
@@ -2702,7 +2770,12 @@ FVoxelStrateMetrics VF_MeasureStrate(
Result.SampledMaxX = Grid.MaxX;
Result.SampledMinY = Grid.MinY;
Result.SampledMaxY = Grid.MaxY;
VoxelStrateMeasurePrivate::BuildMetricsFromGrid(Grid, Settings, Result);
TArray<uint8> PlayerFitMask;
VoxelStrateMeasurePrivate::BuildMetricsFromGrid(
Grid,
Settings,
Result,
OutSampleGrid != nullptr ? &PlayerFitMask : nullptr);
Result.bValid = Result.NumSampled > 0;
if (!Result.bValid)
{
@@ -2711,6 +2784,8 @@ FVoxelStrateMetrics VF_MeasureStrate(
if (OutSampleGrid != nullptr && Result.bValid)
{
VoxelStrateMeasurePrivate::ExportSampleGrid(Grid, *OutSampleGrid);
OutSampleGrid->PlayerFitMask = MoveTemp(PlayerFitMask);
OutSampleGrid->PlayerFitCellCount = Result.NumPlayerFitCells;
}
return Result;
}
@@ -2748,7 +2823,12 @@ FVoxelStrateMetrics VF_MeasureStrateWithSampler(
Result.SampledMaxX = Grid.MaxX;
Result.SampledMinY = Grid.MinY;
Result.SampledMaxY = Grid.MaxY;
VoxelStrateMeasurePrivate::BuildMetricsFromGrid(Grid, Settings, Result);
TArray<uint8> PlayerFitMask;
VoxelStrateMeasurePrivate::BuildMetricsFromGrid(
Grid,
Settings,
Result,
OutSampleGrid != nullptr ? &PlayerFitMask : nullptr);
Result.bValid = Result.NumSampled > 0;
if (!Result.bValid)
{
@@ -2757,6 +2837,8 @@ FVoxelStrateMetrics VF_MeasureStrateWithSampler(
if (OutSampleGrid != nullptr && Result.bValid)
{
VoxelStrateMeasurePrivate::ExportSampleGrid(Grid, *OutSampleGrid);
OutSampleGrid->PlayerFitMask = MoveTemp(PlayerFitMask);
OutSampleGrid->PlayerFitCellCount = Result.NumPlayerFitCells;
}
return Result;
}
@@ -2951,11 +3033,16 @@ bool VF_MeasurePlayerFitWalkWithSampler(
const FVector& BVoxel,
const FVoxelStrateMeasureSettings& Settings,
FVoxelPlayerFitWalkReport& OutReport,
const TArray<FVector>* ComponentProbePoints)
const TArray<FVector>* ComponentProbePoints,
FVoxelStrateSampleGrid* OutSampleGrid)
{
using namespace VoxelStrateMeasurePrivate;
OutReport = FVoxelPlayerFitWalkReport();
if (OutSampleGrid != nullptr)
{
*OutSampleGrid = FVoxelStrateSampleGrid();
}
OutReport.CapsuleRadiusMeters = Settings.PlayerCapsuleRadiusVoxels
* FVoxelPlayerCapsuleConstants::VoxelSizeMeters;
OutReport.CapsuleWidthMeters = 2.0f * OutReport.CapsuleRadiusMeters;
@@ -3271,6 +3358,13 @@ bool VF_MeasurePlayerFitWalkWithSampler(
{
OutReport.RefusalReason = Diagnostics.RefusalReason;
}
if (OutSampleGrid != nullptr)
{
ExportSampleGrid(Grid, *OutSampleGrid);
OutSampleGrid->PlayerFitMask = MoveTemp(PlayerFitMask);
OutSampleGrid->PlayerFitCellCount = NumPlayerFitCells;
}
return true;
}