Fr0zkaandClaude Opus 5
ee89c4d8c2
Owner, after playing 0576e80: "slightly faster, still not on par... I can outrun the
generation and fall." Also, unprompted: "the tunnels seem forced, with a forced
flat -- it technically looks good, but could be a reason." He was right that it
was a reason; it was not THE reason.
WHAT LANDED, all real and all measured
Conservative spatial index (immutable per-chunk uniform bins, baked conservative
bounds, squared rejects, deterministic ordering):
candidates scanned/sample 25.140 -> 5.192
candidates evaluated/sample 13.879 -> 2.992
cave tunnels 11.228/6.069 -> 1.525/0.723
tunnel core 11.228/5.831 -> 1.216/0.567
The earlier flat AABB attempt failed because loose bounds culled almost nothing
while adding a pass. Technique from GPU Gems 3 ch.32 broad-phase.
The forced-flat floor, which the owner spotted by eye: a floor decision is
inherently per-COLUMN and was being recomputed for every Z sample in that column.
Now resolved once per integer XY column and reused.
TunnelCorePosts 2.29 us/sample -> 0.138 us/sample (16x)
The floor is unchanged in behaviour: still applied only after a tunnel passes its
world bounds and walkable projection, still walkable, look untouched.
FIRST MEASUREMENT OF OUR MEMORY FOOTPRINT
The owner: "it shouldn't be pressured on such system, even less when not running
anything." He is right, and it is ours:
total resident worker cache 431,162,116 B = 411 MiB
15 workers 27.41 MiB each
average valid entry 278,704 B
largest entry 541,344 B
static slot storage 2.4 MiB (dynamic: 428.6 MB)
Cutting 4096 -> 128 slots only saved static storage. The weight is 272 KB PER
CACHED CHUNK STATE, ~102 live per worker. Nobody has yet asked why one cached
chunk state needs 272 KB. That is the open question, not the slot count.
⚠️ AND THE PART THAT MATTERS MOST: THIS BOUGHT NOTHING END-TO-END
128^3 TunnelNetwork, seed 1337 slot 1, same region, unprofiled:
before this commit 2.623 s
after, 3 runs 2.705 / 2.678 / 2.682 s
Flat. A 5x candidate reduction and a 16x cheaper floor moved real generation time
by zero.
The reason is visible in the numbers and I should have caught it earlier: the
PROFILED path reports 14.74 us/sample while the real unprofiled cost is
1.28 us/voxel. **The instrument inflates by ~11x.** Every bucket we have been
ranking and optimising for two rounds was measured under that inflation, so the
ranking does not reflect where real time goes. We optimised what the profiler
exaggerated.
⇒ Before any further optimisation: make the profiler cheap enough that profiled
and unprofiled totals agree within a small factor, or switch to sampling. An
alarming number is the instrument until proven otherwise -- three for three in
this project, and now four.
Still true: ~21x slower than the pre-op-stack build, and generation still loses
to a walking player. Geometry changed by 40 triangles (153,692 -> 153,732), which
is allowed; two full exports hash identically (37E00BEE..0A7DF). Build 42/42.
No CPU-feature dispatch. Nothing outside Plugins/VoxelForge was touched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
371 lines
17 KiB
C++
371 lines
17 KiB
C++
#include "VoxelDensityProfile.h"
|
|
|
|
namespace VoxelDensityProfile
|
|
{
|
|
namespace
|
|
{
|
|
std::atomic<bool> GEnabled(false);
|
|
|
|
// The old implementation performed a contended atomic fetch_add for every timer and
|
|
// counter. A TunnelNetwork sample can close twenty-plus ranges and add several
|
|
// candidate counters, so profiling changed the workload it was supposed to measure.
|
|
// Keep one accumulator per worker and only aggregate at Snapshot(). Commandlet resets
|
|
// happen between completed jobs, so the deliberately light-weight list is sufficient.
|
|
struct FThreadState
|
|
{
|
|
uint64 Calls[BucketCount]{};
|
|
uint64 Cycles[BucketCount]{};
|
|
uint64 Counters[CounterCount]{};
|
|
|
|
uint64 TunnelCacheCapacityEntries = 0;
|
|
uint64 TunnelCacheValidEntries = 0;
|
|
uint64 TunnelCacheStaticBytes = 0;
|
|
uint64 TunnelCacheDynamicBytes = 0;
|
|
uint64 TunnelCacheEntryBytes = 0;
|
|
uint64 TunnelCacheLargestEntryBytes = 0;
|
|
uint64 TunnelCacheLargestWorkerValidEntries = 0;
|
|
|
|
uint64 RoomGraphCacheCapacityEntries = 0;
|
|
uint64 RoomGraphCacheValidEntries = 0;
|
|
uint64 RoomGraphCacheStaticBytes = 0;
|
|
uint64 RoomGraphCacheDynamicBytes = 0;
|
|
uint64 RoomGraphCacheEntryBytes = 0;
|
|
uint64 RoomGraphCacheLargestEntryBytes = 0;
|
|
uint64 RoomGraphCacheLargestWorkerValidEntries = 0;
|
|
|
|
std::atomic<FThreadState*> Next{nullptr};
|
|
};
|
|
|
|
std::atomic<FThreadState*> GThreadStates{nullptr};
|
|
thread_local FThreadState* GThreadState = nullptr;
|
|
|
|
FThreadState& GetThreadState()
|
|
{
|
|
if (GThreadState == nullptr)
|
|
{
|
|
GThreadState = new FThreadState();
|
|
FThreadState* Head = GThreadStates.load(std::memory_order_relaxed);
|
|
do
|
|
{
|
|
GThreadState->Next.store(Head, std::memory_order_relaxed);
|
|
}
|
|
while (!GThreadStates.compare_exchange_weak(
|
|
Head, GThreadState,
|
|
std::memory_order_release,
|
|
std::memory_order_relaxed));
|
|
}
|
|
return *GThreadState;
|
|
}
|
|
|
|
int32 ToIndex(EBucket Bucket)
|
|
{
|
|
const int32 Index = static_cast<int32>(Bucket);
|
|
return (Index >= 0 && Index < BucketCount)
|
|
? Index
|
|
: static_cast<int32>(EBucket::OtherOp);
|
|
}
|
|
}
|
|
|
|
void SetEnabled(bool bEnabled)
|
|
{
|
|
GEnabled.store(bEnabled, std::memory_order_relaxed);
|
|
}
|
|
|
|
bool IsEnabled()
|
|
{
|
|
return GEnabled.load(std::memory_order_relaxed);
|
|
}
|
|
|
|
void Reset()
|
|
{
|
|
for (FThreadState* State = GThreadStates.load(std::memory_order_acquire);
|
|
State != nullptr;
|
|
State = State->Next.load(std::memory_order_acquire))
|
|
{
|
|
for (int32 Index = 0; Index < BucketCount; ++Index)
|
|
{
|
|
State->Calls[Index] = 0;
|
|
State->Cycles[Index] = 0;
|
|
}
|
|
for (int32 Index = 0; Index < CounterCount; ++Index)
|
|
{
|
|
State->Counters[Index] = 0;
|
|
}
|
|
}
|
|
}
|
|
|
|
FSnapshot Snapshot()
|
|
{
|
|
FSnapshot Result;
|
|
for (FThreadState* State = GThreadStates.load(std::memory_order_acquire);
|
|
State != nullptr;
|
|
State = State->Next.load(std::memory_order_acquire))
|
|
{
|
|
for (int32 Index = 0; Index < BucketCount; ++Index)
|
|
{
|
|
Result.Calls[Index] += State->Calls[Index];
|
|
Result.Cycles[Index] += State->Cycles[Index];
|
|
}
|
|
for (int32 Index = 0; Index < CounterCount; ++Index)
|
|
{
|
|
Result.Counters[Index] += State->Counters[Index];
|
|
}
|
|
|
|
const uint64 TunnelWorkerBytes = State->TunnelCacheStaticBytes
|
|
+ State->TunnelCacheDynamicBytes;
|
|
if (State->TunnelCacheCapacityEntries > 0)
|
|
{
|
|
++Result.TunnelCacheWorkers;
|
|
Result.TunnelCacheCapacityEntries += State->TunnelCacheCapacityEntries;
|
|
Result.TunnelCacheValidEntries += State->TunnelCacheValidEntries;
|
|
Result.TunnelCacheStaticBytes += State->TunnelCacheStaticBytes;
|
|
Result.TunnelCacheDynamicBytes += State->TunnelCacheDynamicBytes;
|
|
Result.TunnelCacheEntryBytes += State->TunnelCacheEntryBytes;
|
|
Result.TunnelCacheLargestEntryBytes = FMath::Max(
|
|
Result.TunnelCacheLargestEntryBytes,
|
|
State->TunnelCacheLargestEntryBytes);
|
|
Result.TunnelCacheLargestWorkerValidEntries = FMath::Max(
|
|
Result.TunnelCacheLargestWorkerValidEntries,
|
|
State->TunnelCacheValidEntries);
|
|
Result.TunnelCacheLargestWorkerBytes = FMath::Max(
|
|
Result.TunnelCacheLargestWorkerBytes, TunnelWorkerBytes);
|
|
}
|
|
|
|
const uint64 RoomGraphWorkerBytes = State->RoomGraphCacheStaticBytes
|
|
+ State->RoomGraphCacheDynamicBytes;
|
|
if (State->RoomGraphCacheCapacityEntries > 0)
|
|
{
|
|
++Result.RoomGraphCacheWorkers;
|
|
Result.RoomGraphCacheCapacityEntries += State->RoomGraphCacheCapacityEntries;
|
|
Result.RoomGraphCacheValidEntries += State->RoomGraphCacheValidEntries;
|
|
Result.RoomGraphCacheStaticBytes += State->RoomGraphCacheStaticBytes;
|
|
Result.RoomGraphCacheDynamicBytes += State->RoomGraphCacheDynamicBytes;
|
|
Result.RoomGraphCacheEntryBytes += State->RoomGraphCacheEntryBytes;
|
|
Result.RoomGraphCacheLargestEntryBytes = FMath::Max(
|
|
Result.RoomGraphCacheLargestEntryBytes,
|
|
State->RoomGraphCacheLargestEntryBytes);
|
|
Result.RoomGraphCacheLargestWorkerValidEntries = FMath::Max(
|
|
Result.RoomGraphCacheLargestWorkerValidEntries,
|
|
State->RoomGraphCacheValidEntries);
|
|
Result.RoomGraphCacheLargestWorkerBytes = FMath::Max(
|
|
Result.RoomGraphCacheLargestWorkerBytes, RoomGraphWorkerBytes);
|
|
}
|
|
}
|
|
return Result;
|
|
}
|
|
|
|
void AddCounter(ECounter Counter, uint64 Amount)
|
|
{
|
|
const int32 Index = static_cast<int32>(Counter);
|
|
if (Index >= 0 && Index < CounterCount)
|
|
{
|
|
GetThreadState().Counters[Index] += Amount;
|
|
}
|
|
}
|
|
|
|
void AddMeasurement(EBucket Bucket, uint64 Cycles, uint64 Calls)
|
|
{
|
|
const int32 Index = ToIndex(Bucket);
|
|
FThreadState& State = GetThreadState();
|
|
State.Calls[Index] += Calls;
|
|
State.Cycles[Index] += Cycles;
|
|
}
|
|
|
|
void SetWorkerTunnelCacheFootprint(
|
|
uint64 CapacityEntries, uint64 ValidEntries,
|
|
uint64 StaticBytes, uint64 DynamicBytes,
|
|
uint64 EntryBytes, uint64 LargestEntryBytes,
|
|
uint64 LargestWorkerValidEntries)
|
|
{
|
|
if (!IsEnabled()) { return; }
|
|
FThreadState& State = GetThreadState();
|
|
State.TunnelCacheCapacityEntries = CapacityEntries;
|
|
State.TunnelCacheValidEntries = ValidEntries;
|
|
State.TunnelCacheStaticBytes = StaticBytes;
|
|
State.TunnelCacheDynamicBytes = DynamicBytes;
|
|
State.TunnelCacheEntryBytes = EntryBytes;
|
|
State.TunnelCacheLargestEntryBytes = LargestEntryBytes;
|
|
State.TunnelCacheLargestWorkerValidEntries = LargestWorkerValidEntries;
|
|
}
|
|
|
|
void SetWorkerRoomGraphCacheFootprint(
|
|
uint64 CapacityEntries, uint64 ValidEntries,
|
|
uint64 StaticBytes, uint64 DynamicBytes,
|
|
uint64 EntryBytes, uint64 LargestEntryBytes,
|
|
uint64 LargestWorkerValidEntries)
|
|
{
|
|
if (!IsEnabled()) { return; }
|
|
FThreadState& State = GetThreadState();
|
|
State.RoomGraphCacheCapacityEntries = CapacityEntries;
|
|
State.RoomGraphCacheValidEntries = ValidEntries;
|
|
State.RoomGraphCacheStaticBytes = StaticBytes;
|
|
State.RoomGraphCacheDynamicBytes = DynamicBytes;
|
|
State.RoomGraphCacheEntryBytes = EntryBytes;
|
|
State.RoomGraphCacheLargestEntryBytes = LargestEntryBytes;
|
|
State.RoomGraphCacheLargestWorkerValidEntries = LargestWorkerValidEntries;
|
|
}
|
|
|
|
const TCHAR* CounterName(ECounter Counter)
|
|
{
|
|
switch (Counter)
|
|
{
|
|
case ECounter::TunnelCacheLookup: return TEXT("TunnelCacheLookup");
|
|
case ECounter::TunnelCacheHit: return TEXT("TunnelCacheHit");
|
|
case ECounter::TunnelCacheMiss: return TEXT("TunnelCacheMiss");
|
|
case ECounter::SdfCacheBuild: return TEXT("SdfCacheBuild");
|
|
case ECounter::CaveRoomCandidates:return TEXT("CaveRoomCandidates");
|
|
case ECounter::CaveRoomEvaluated: return TEXT("CaveRoomEvaluated");
|
|
case ECounter::CaveTunnelCandidates:return TEXT("CaveTunnelCandidates");
|
|
case ECounter::CaveTunnelEvaluated:return TEXT("CaveTunnelEvaluated");
|
|
case ECounter::TunnelCoreCandidates:return TEXT("TunnelCoreCandidates");
|
|
case ECounter::TunnelCoreEvaluated:return TEXT("TunnelCoreEvaluated");
|
|
case ECounter::TunnelSupportColumnBuilds:return TEXT("TunnelSupportColumnBuilds");
|
|
case ECounter::TunnelSupportColumnCandidates:return TEXT("TunnelSupportColumnCandidates");
|
|
case ECounter::TunnelSupportFloorQueries:return TEXT("TunnelSupportFloorQueries");
|
|
case ECounter::TunnelSupportFloorChecks:return TEXT("TunnelSupportFloorChecks");
|
|
case ECounter::PassageCandidates: return TEXT("PassageCandidates");
|
|
case ECounter::PassageEvaluated: return TEXT("PassageEvaluated");
|
|
case ECounter::Count: break;
|
|
}
|
|
return TEXT("Unknown");
|
|
}
|
|
|
|
EBucket BucketFromName(const TCHAR* Name)
|
|
{
|
|
if (Name == nullptr) { return EBucket::OtherOp; }
|
|
|
|
struct FNameBucket
|
|
{
|
|
const TCHAR* Name;
|
|
EBucket Bucket;
|
|
};
|
|
static const FNameBucket Names[] =
|
|
{
|
|
{ TEXT("ApplyDisturbances"), EBucket::ApplyDisturbances },
|
|
{ TEXT("PassageModifier"), EBucket::PassageModifier },
|
|
{ TEXT("PassageLandingAir"), EBucket::PassageLandingAir },
|
|
{ TEXT("PassageLandingFloor"), EBucket::PassageLandingFloor },
|
|
{ TEXT("PassageLandingRoomFloor"), EBucket::PassageLandingRoomFloor },
|
|
{ TEXT("PassageTunnelAir"), EBucket::PassageTunnelAir },
|
|
{ TEXT("PassageStructuralPosts"), EBucket::PassageStructuralPosts },
|
|
{ TEXT("TunnelCoreSupport"), EBucket::TunnelCoreSupport },
|
|
{ TEXT("TunnelCoreSdf"), EBucket::TunnelCoreSdf },
|
|
{ TEXT("TunnelCorePosts"), EBucket::TunnelCorePosts },
|
|
{ TEXT("StructuralTail"), EBucket::StructuralTail },
|
|
{ TEXT("RoomGraphSource"), EBucket::RoomGraphSource },
|
|
{ TEXT("SdfCarve"), EBucket::SdfCarve },
|
|
{ TEXT("SdfConvertOp"), EBucket::SdfCarve },
|
|
{ TEXT("CaveRoughnessMod"), EBucket::CaveRoughnessMod },
|
|
{ TEXT("CaveTerraceMod"), EBucket::CaveTerraceMod },
|
|
{ TEXT("LayerLineMod"), EBucket::LayerLineMod },
|
|
{ TEXT("RibbingMod"), EBucket::RibbingMod },
|
|
{ TEXT("CaveOverhangMod"), EBucket::CaveOverhangMod },
|
|
{ TEXT("CaveCliffMod"), EBucket::CaveCliffMod },
|
|
{ TEXT("ScallopMod"), EBucket::ScallopMod },
|
|
{ TEXT("CaveArchMod"), EBucket::CaveArchMod },
|
|
{ TEXT("RoomColumnMod"), EBucket::RoomColumnMod },
|
|
{ TEXT("DomeMod"), EBucket::DomeMod },
|
|
{ TEXT("PinchMod"), EBucket::PinchMod },
|
|
{ TEXT("FloorBiasMod"), EBucket::FloorBiasMod },
|
|
{ TEXT("WormFieldSource"), EBucket::WormFieldSource },
|
|
{ TEXT("OriginSpineOp"), EBucket::OriginSpineOp },
|
|
{ TEXT("BoundarySealOp"), EBucket::BoundarySealOp },
|
|
{ TEXT("PassageCarveOp"), EBucket::PassageCarveOp },
|
|
{ TEXT("CaveTunnelFloorOp"), EBucket::CaveTunnelFloorOp },
|
|
{ TEXT("CaveTunnelAirOp"), EBucket::CaveTunnelAirOp },
|
|
{ TEXT("XYEdgeSealOp"), EBucket::XYEdgeSealOp },
|
|
{ TEXT("DensityPrologue"), EBucket::DensityPrologue },
|
|
{ TEXT("DensityCore"), EBucket::DensityCore },
|
|
{ TEXT("DensityDisturbances"), EBucket::DensityDisturbances },
|
|
{ TEXT("DensityStructuralPosts"), EBucket::DensityStructuralPosts },
|
|
{ TEXT("DensityBoundarySeal"), EBucket::DensityBoundarySeal },
|
|
{ TEXT("DensityDiffLayer"), EBucket::DensityDiffLayer },
|
|
{ TEXT("DensityTail"), EBucket::DensityTail },
|
|
{ TEXT("MesherGenerateMesh"), EBucket::MesherGenerateMesh },
|
|
{ TEXT("MesherDensityGrid"), EBucket::MesherDensityGrid },
|
|
{ TEXT("MesherCellClassification"), EBucket::MesherCellClassification },
|
|
{ TEXT("MesherGradientNormals"), EBucket::MesherGradientNormals },
|
|
{ TEXT("MesherVertexInterpolation"), EBucket::MesherVertexInterpolation },
|
|
{ TEXT("MesherStreamBuilding"), EBucket::MesherStreamBuilding },
|
|
{ TEXT("MesherOther"), EBucket::MesherOther },
|
|
};
|
|
for (const FNameBucket& Entry : Names)
|
|
{
|
|
if (FCString::Strcmp(Name, Entry.Name) == 0)
|
|
{
|
|
return Entry.Bucket;
|
|
}
|
|
}
|
|
return EBucket::OtherOp;
|
|
}
|
|
|
|
const TCHAR* BucketName(EBucket Bucket)
|
|
{
|
|
switch (Bucket)
|
|
{
|
|
case EBucket::GetDensityAt: return TEXT("GetDensityAt");
|
|
case EBucket::ApplyDisturbances:return TEXT("ApplyDisturbances");
|
|
case EBucket::PassageModifier: return TEXT("PassageModifier");
|
|
case EBucket::PassageLandingAir:return TEXT("PassageLandingAir");
|
|
case EBucket::PassageLandingFloor:return TEXT("PassageLandingFloor");
|
|
case EBucket::PassageLandingRoomFloor:return TEXT("PassageLandingRoomFloor");
|
|
case EBucket::PassageTunnelAir:return TEXT("PassageTunnelAir");
|
|
case EBucket::PassageStructuralPosts:return TEXT("PassageStructuralPosts");
|
|
case EBucket::TunnelCoreSupport:return TEXT("TunnelCoreSupport");
|
|
case EBucket::TunnelCoreSdf: return TEXT("TunnelCoreSdf");
|
|
case EBucket::TunnelCorePosts: return TEXT("TunnelCorePosts");
|
|
case EBucket::StructuralTail: return TEXT("StructuralTail");
|
|
case EBucket::RoomGraphSource: return TEXT("RoomGraphSource");
|
|
case EBucket::SdfCarve: return TEXT("SdfCarve");
|
|
case EBucket::CaveRoughnessMod: return TEXT("CaveRoughnessMod");
|
|
case EBucket::CaveTerraceMod: return TEXT("CaveTerraceMod");
|
|
case EBucket::LayerLineMod: return TEXT("LayerLineMod");
|
|
case EBucket::RibbingMod: return TEXT("RibbingMod");
|
|
case EBucket::CaveOverhangMod: return TEXT("CaveOverhangMod");
|
|
case EBucket::CaveCliffMod: return TEXT("CaveCliffMod");
|
|
case EBucket::ScallopMod: return TEXT("ScallopMod");
|
|
case EBucket::CaveArchMod: return TEXT("CaveArchMod");
|
|
case EBucket::RoomColumnMod: return TEXT("RoomColumnMod");
|
|
case EBucket::DomeMod: return TEXT("DomeMod");
|
|
case EBucket::PinchMod: return TEXT("PinchMod");
|
|
case EBucket::FloorBiasMod: return TEXT("FloorBiasMod");
|
|
case EBucket::WormFieldSource: return TEXT("WormFieldSource");
|
|
case EBucket::OriginSpineOp: return TEXT("OriginSpineOp");
|
|
case EBucket::BoundarySealOp: return TEXT("BoundarySealOp");
|
|
case EBucket::PassageCarveOp: return TEXT("PassageCarveOp");
|
|
case EBucket::CaveTunnelFloorOp:return TEXT("CaveTunnelFloorOp");
|
|
case EBucket::CaveTunnelAirOp: return TEXT("CaveTunnelAirOp");
|
|
case EBucket::XYEdgeSealOp: return TEXT("XYEdgeSealOp");
|
|
case EBucket::OtherOp: return TEXT("OtherOp");
|
|
case EBucket::DensityPrologue: return TEXT("DensityPrologue");
|
|
case EBucket::DensityCore: return TEXT("DensityCore");
|
|
case EBucket::DensityDisturbances:return TEXT("DensityDisturbances");
|
|
case EBucket::DensityStructuralPosts:return TEXT("DensityStructuralPosts");
|
|
case EBucket::DensityBoundarySeal:return TEXT("DensityBoundarySeal");
|
|
case EBucket::DensityDiffLayer: return TEXT("DensityDiffLayer");
|
|
case EBucket::DensityTail: return TEXT("DensityTail");
|
|
case EBucket::MesherGenerateMesh:return TEXT("MesherGenerateMesh");
|
|
case EBucket::MesherDensityGrid:return TEXT("MesherDensityGrid");
|
|
case EBucket::MesherCellClassification:return TEXT("MesherCellClassification");
|
|
case EBucket::MesherGradientNormals:return TEXT("MesherGradientNormals");
|
|
case EBucket::MesherVertexInterpolation:return TEXT("MesherVertexInterpolation");
|
|
case EBucket::MesherStreamBuilding:return TEXT("MesherStreamBuilding");
|
|
case EBucket::MesherOther: return TEXT("MesherOther");
|
|
case EBucket::Count: break;
|
|
}
|
|
return TEXT("Unknown");
|
|
}
|
|
|
|
FScopedTimer::~FScopedTimer()
|
|
{
|
|
End();
|
|
}
|
|
|
|
void FScopedTimer::End()
|
|
{
|
|
if (!bActive) { return; }
|
|
AddMeasurement(Bucket, FPlatformTime::Cycles64() - StartCycles, 1);
|
|
bActive = false;
|
|
}
|
|
}
|