fix(crash): EndPlay tore the world down under live workers -- the cache made it lethal

PIE crashed with EXCEPTION_ACCESS_VIOLATION reading 0xffffffffffffffff:

  FRoomGraphSource::Eval()   VoxelDensityOpStack.cpp:3056  (Manager->GetLayoutVersion())
  UVoxelGenerator::GetDensityAt()  VoxelGenerator.cpp:1103 (ActiveOpStack->EvalMC)
  ...  AVoxelWorld::LoadTile lambda, on a task-graph worker

Two lifetime bugs, combined:

1. EndPlay gave up after a 3-second deadline and destroyed UObjects while workers
   were still inside GenerateTileResult. The crash log records 4, 3 and 7 tasks
   still running during teardown. ARCHITECTURE has always said EndPlay blocks on
   ActiveTaskCount -> 0; it did not.
2. e08b49a's worker-local cache keeps 128 built op stacks alive per thread, and
   those stacks hold a raw UVoxelStrateManager*. Before the cache, a stack was
   rebuilt at every chunk crossing and never outlived its owner for long. After
   it, a freed manager stayed reachable and got dereferenced.

The cache did not create the teardown race. It made an existing one lethal.

Fixes:
- EndPlay now waits unconditionally for ActiveTaskCount == 0. The 3s deadline is
  demoted to a log line: a slow drain is reported, teardown is never torn out
  from under a reader.
- UVoxelStrateManager carries a process-unique CacheLifetimeId, folded into the
  cache key. Deliberately not a UObject pointer: allocator address reuse must not
  let a stale worker-local stack look current across PIE sessions. PassagesVersion
  restarts per layout and cannot make that distinction.
- Cached passage/room manager references are weak pointers, fail-closed.
- Verified FStrateGenerationParams is copied BY VALUE into tunnel operators, and
  stack moves preserve owned operator allocations, so no operator references a
  previous cache entry's parameters.

  TunnelNetwork mesh  342.332 s -> 16.199 s   (21.1x)
  per tile, 64 tiles    5.349 s ->  0.253 s

NOT verified, and it matters here:
- No PIE measurement. The owner restricted this session to analysis and fixes, so
  the numbers above are the commandlet's direct-mesher path, NOT the streaming
  path the game uses. They are not game timings and are not presented as such.
- The 128-slot cache may churn at distance: an L0 tile crosses at most 27 exact
  chunks, but L2/L3/L4 tiles touch 216 / 1,000 / 6,859 chunk keys. Whether that
  helps or hurts in game is unmeasured.
- EndPlay waiting forever converts a crash into a hang if a task never observes
  bShuttingDown. Workers do check it, but this trade is deliberate and worth
  watching.

Named, not changed: the straight "roads" the owner still sees are inter-strate
passages. FStratePassageConfig defaults Connections=1 and Style=Straight
(VoxelStrateTypes.h:1781); GeneratePassages maps Straight -> SlopedTunnel and
builds a two-leg switchback that FPassageCarveOp carves
(VoxelStrateManager.cpp:819). d97373c removed the landing-to-hub radial roads; it
did not touch these. FLatticeCorridorSource also still emits axis-aligned capsule
edges for Maze (VoxelDensityOpStack.cpp:548). Awaiting the owner's call on what
should replace them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
This commit is contained in:
Fr0zkaandClaude Opus 5 committed 2026-09-08 21:17:57 +02:00
1 parent e08b49a183
commit 64f5d61e08
5 files changed
+105 -28

No files matched your search

@@ -1814,9 +1814,9 @@ namespace
void Eval(float X, float Y, float Z, FVoxelOpSample& InOut) const override
{
if (Manager)
if (const UVoxelStrateManager* LiveManager = Manager.Get())
{
Manager->ApplyPassageModifier(InOut.Density, X, Y, Z, Base, Seal);
LiveManager->ApplyPassageModifier(InOut.Density, X, Y, Z, Base, Seal);
}
// Keep the source and op-stack operation order identical: the origin room is carved
// before the boundary seal, then its support floor is reasserted after the passage
@@ -1829,23 +1829,24 @@ namespace
// simplement branchée au bon endroit au lieu d'être un cas particulier du classifieur.
EVoxelOpEffect EffectOverBox(const FBox& VoxelBox, const FVoxelOpContext&) const override
{
if (!Manager) { return EVoxelOpEffect::Identity; }
const UVoxelStrateManager* LiveManager = Manager.Get();
if (!LiveManager) { return EVoxelOpEffect::Identity; }
if (VoxelPassageGeometry::OriginLandingFloorTouchesBox(
VoxelBox, TopZ, BotZ, Seal, SpineRadius)
|| Manager->AnyPassageLandingFloorNearBox(VoxelBox.Min, VoxelBox.Max))
|| LiveManager->AnyPassageLandingFloorNearBox(VoxelBox.Min, VoxelBox.Max))
{
// The room carves air, while its support slab force-writes solid. Both
// hypotheses must therefore be killed for a box touching that slab.
return EVoxelOpEffect::Both;
}
return Manager->AnyPassageNearBox(VoxelBox.Min, VoxelBox.Max)
return LiveManager->AnyPassageNearBox(VoxelBox.Min, VoxelBox.Max)
? EVoxelOpEffect::CarveOnly : EVoxelOpEffect::Identity;
}
const TCHAR* DebugName() const override { return TEXT("PassageCarveOp"); }
private:
const UVoxelStrateManager* Manager;
TWeakObjectPtr<const UVoxelStrateManager> Manager;
float Base, Seal, TopZ, BotZ, SpineRadius;
};
@@ -2805,6 +2806,7 @@ namespace
FRoomGraphSource(const FStrateGenerationParams& InP, int32 InSeed,
const UVoxelStrateManager* InManager)
: P(InP), Seed(InSeed), SeedU((uint32)InSeed), Manager(InManager)
, ManagerLifetimeId(InManager ? InManager->GetCacheLifetimeId() : 0)
, ParamsFingerprint(FCrc::MemCrc32(&InP, sizeof(InP)))
{}
@@ -2853,6 +2855,7 @@ namespace
uint32 CachedSeed = 0;
uint32 CachedFingerprint = 0xFFFFFFFFu;
uint32 CachedLayout = 0xFFFFFFFFu;
uint64 CachedManagerLifetimeId = 0;
/** La salle de SDF minimal pour le dernier voxel évalué. -1 = aucune. */
int32 NearestRoom = -1;
@@ -2991,6 +2994,18 @@ namespace
// sur chaque voxel de roc profond, ce que l'original ne paie pas.)
S.bLocalParamsValid = false;
const UVoxelStrateManager* LiveManager = Manager.Get();
// A cached stack may outlive its UObject owner when an editor world is stopped. The
// generator's manager-lifetime key prevents selecting it normally; this guard is the
// final boundary for a task that is already inside an old stack. Keep commandlet stacks
// (which deliberately have no manager) functional, but never evaluate stale room data.
if (ManagerLifetimeId != 0 && LiveManager == nullptr)
{
S.Cache = FChunkSDFCache();
InOut.Sdf = FLT_MAX;
return;
}
if (!(P.RoomDensity > 0.0f && P.RoomSpacing > 0.0f)) { return; } // Sdf reste FLT_MAX
const float EffectiveZ = EffZ(WorldZ);
@@ -3046,18 +3061,19 @@ namespace
// Index de strate — mémo (chunk-Z, version de layout), transcrit tel quel. La requête
// vise le CENTRE de la bande, donc le résultat est une fonction pure de la clé.
int32 StrateIdx = 0;
if (Manager)
if (LiveManager)
{
thread_local int32 SI_ChunkZ = INT32_MAX;
thread_local uint32 SI_Version = 0xFFFFFFFFu;
thread_local int32 SI_Index = 0;
const int32 QZ = FMath::FloorToInt(WorldZ / (float)CHUNK_SIZE);
const uint32 LV = Manager->GetLayoutVersion();
const uint32 LV = LiveManager->GetLayoutVersion();
if (QZ != SI_ChunkZ || LV != SI_Version)
{
SI_ChunkZ = QZ;
SI_Version = LV;
SI_Index = Manager->GetStrateIndex(((float)QZ + 0.5f) * CHUNK_SIZE * VOXEL_SIZE);
SI_Index = LiveManager->GetStrateIndex(
((float)QZ + 0.5f) * CHUNK_SIZE * VOXEL_SIZE);
}
StrateIdx = SI_Index;
}
@@ -3065,6 +3081,7 @@ namespace
const bool bNeedRebuild =
StrateIdx != S.CachedStrate || SeedU != S.CachedSeed ||
ParamsFingerprint != S.CachedFingerprint || LayoutVersion != S.CachedLayout ||
ManagerLifetimeId != S.CachedManagerLifetimeId ||
WarpedX < S.CachedSMinX || WarpedX > S.CachedSMaxX ||
WarpedY < S.CachedSMinY || WarpedY > S.CachedSMaxY;
@@ -3084,10 +3101,10 @@ namespace
const float SMaxY = ChunkMaxY + Expansion;
const TArray<FStrateTerrainOpEntry>* TerrainOps = nullptr;
if (Manager)
if (LiveManager)
{
const int32 ChunkZ = FMath::FloorToInt(WorldZ / (float)CHUNK_SIZE);
UVoxelStrateDefinition* Def = Manager->GetStrateForChunk(
UVoxelStrateDefinition* Def = LiveManager->GetStrateForChunk(
FIntVector(CacheChunkX, CacheChunkY, ChunkZ));
if (Def) { TerrainOps = &Def->TerrainOperations; }
}
@@ -3101,6 +3118,7 @@ namespace
S.CachedSeed = SeedU;
S.CachedFingerprint = ParamsFingerprint;
S.CachedLayout = LayoutVersion;
S.CachedManagerLifetimeId = ManagerLifetimeId;
}
float CaveSDF = VoxelCaveMorphology::EvaluateSDFCached(
@@ -3198,6 +3216,7 @@ namespace
uint32 KeySeed = 0;
uint32 KeyFingerprint = 0xFFFFFFFFu;
uint32 KeyLayout = 0xFFFFFFFFu;
uint64 KeyManagerLifetimeId = 0;
bool bValid = false;
EVoxelOpEffect Verdict = EVoxelOpEffect::Both;
FVoxelBoxSdfInterval SdfInterval;
@@ -3281,6 +3300,7 @@ namespace
const FVoxelOpContext& Ctx) const override
{
FBoxState& B = BoxState();
const UVoxelStrateManager* LiveManager = Manager.Get();
auto Unknown = [&]()
{
@@ -3290,6 +3310,12 @@ namespace
};
auto Finite = [](float V) { return FMath::IsFinite(V); };
if (ManagerLifetimeId != 0 && LiveManager == nullptr)
{
Unknown();
return;
}
const float BoxMinX = (float)VoxelBox.Min.X;
const float BoxMinY = (float)VoxelBox.Min.Y;
const float BoxMinZ = (float)VoxelBox.Min.Z;
@@ -3362,13 +3388,13 @@ namespace
int32 StrateIdx = 0;
const TArray<FStrateTerrainOpEntry>* TerrainOps = nullptr;
if (Manager)
if (LiveManager)
{
StrateIdx = Manager->GetStrateIndex(
StrateIdx = LiveManager->GetStrateIndex(
((float)CZ0 + 0.5f) * CHUNK_SIZE * VOXEL_SIZE);
for (int32 CZ = CZ0 + 1; CZ <= CZ1; ++CZ)
{
if (Manager->GetStrateIndex(
if (LiveManager->GetStrateIndex(
((float)CZ + 0.5f) * CHUNK_SIZE * VOXEL_SIZE) != StrateIdx)
{
Unknown();
@@ -3380,12 +3406,12 @@ namespace
// columns from it. Every chunk touched by this query must therefore resolve to the
// same definition before one cache can represent the box.
UVoxelStrateDefinition* Def0 =
Manager->GetStrateForChunk(FIntVector(CX0, CY0, CZ0));
LiveManager->GetStrateForChunk(FIntVector(CX0, CY0, CZ0));
for (int32 CZ = CZ0; CZ <= CZ1; ++CZ)
for (int32 CY = CY0; CY <= CY1; ++CY)
for (int32 CX = CX0; CX <= CX1; ++CX)
{
if (Manager->GetStrateForChunk(FIntVector(CX, CY, CZ)) != Def0)
if (LiveManager->GetStrateForChunk(FIntVector(CX, CY, CZ)) != Def0)
{
Unknown();
return;
@@ -3396,7 +3422,8 @@ namespace
const uint32 LV = Ctx.LayoutVersion;
if (B.bValid && B.KeyBox == VoxelBox && B.KeyStrate == StrateIdx
&& B.KeySeed == SeedU && B.KeyFingerprint == ParamsFingerprint && B.KeyLayout == LV)
&& B.KeySeed == SeedU && B.KeyFingerprint == ParamsFingerprint && B.KeyLayout == LV
&& B.KeyManagerLifetimeId == ManagerLifetimeId)
{
InOut = B.SdfInterval;
return;
@@ -3603,6 +3630,7 @@ namespace
B.KeySeed = SeedU;
B.KeyFingerprint = ParamsFingerprint;
B.KeyLayout = LV;
B.KeyManagerLifetimeId = ManagerLifetimeId;
B.bValid = true;
InOut = Own;
}
@@ -3620,7 +3648,8 @@ namespace
FStrateGenerationParams P;
int32 Seed;
uint32 SeedU;
const UVoxelStrateManager* Manager; // NON possédant
TWeakObjectPtr<const UVoxelStrateManager> Manager;
uint64 ManagerLifetimeId = 0;
uint32 ParamsFingerprint;
uint32 LayoutVersion = 0;
};
+19 -3
View File
@@ -470,6 +470,7 @@ namespace
{
FChunkSDFCache Cache;
uint64 OwnerId = 0;
uint64 ManagerLifetimeId = 0;
FIntVector Chunk = FIntVector(INT32_MAX, INT32_MAX, INT32_MAX);
uint32 ParamsFingerprint = 0xFFFFFFFFu;
uint32 LayoutVersion = 0xFFFFFFFFu;
@@ -492,6 +493,7 @@ namespace
{
bool bValid = false;
uint64 OwnerId = 0;
uint64 ManagerLifetimeId = 0;
FIntVector Chunk = FIntVector(INT32_MAX, INT32_MAX, INT32_MAX);
uint32 LayoutVersion = 0xFFFFFFFFu;
ECaveGeneratorType GenType = ECaveGeneratorType::TunnelNetwork;
@@ -518,11 +520,13 @@ namespace
const FStrateGenerationParams& Params,
uint32 ParamsFingerprint,
uint32 LayoutVersion,
uint64 ManagerLifetimeId,
uint64 OwnerId,
FTunnelCoreCacheState& OutState)
{
if (OutState.bValid
&& OutState.OwnerId == OwnerId
&& OutState.ManagerLifetimeId == ManagerLifetimeId
&& OutState.Chunk == ChunkCoord
&& OutState.ParamsFingerprint == ParamsFingerprint
&& OutState.LayoutVersion == LayoutVersion)
@@ -546,6 +550,7 @@ namespace
Params, static_cast<uint32>(Manager.GetWorldSeed()),
StrateIndex, nullptr);
OutState.OwnerId = OwnerId;
OutState.ManagerLifetimeId = ManagerLifetimeId;
OutState.Chunk = ChunkCoord;
OutState.ParamsFingerprint = ParamsFingerprint;
OutState.LayoutVersion = LayoutVersion;
@@ -799,6 +804,7 @@ float UVoxelGenerator::GetDensityAt(float WorldX, float WorldY, float WorldZ) co
// la strate n'a pas coché `bUseOperatorStack` ET que son archétype n'est pas porté.
thread_local FVoxelOpStack CP_OpStack;
thread_local bool CP_UseOpStack = false;
thread_local uint64 CP_ManagerLifetimeId = 0;
thread_local FTunnelNetworkDensityCacheEntry CP_TunnelDensityCache[TunnelDensityCacheSlotCount];
thread_local FTunnelNetworkDensityCacheEntry* CP_ActiveTunnelDensityCache = nullptr;
// Cooked-season recipes and editor candidate recipes share this immutable worker-local
@@ -817,6 +823,7 @@ float UVoxelGenerator::GetDensityAt(float WorldX, float WorldY, float WorldZ) co
#endif
const uint32 LayoutVersion = StrateManager->GetLayoutVersion();
const uint64 ManagerLifetimeId = StrateManager->GetCacheLifetimeId();
FVoxelOpStack* ActiveOpStack = &CP_OpStack;
FTunnelCoreCacheState* ActiveTunnelCoreCache = &GTunnelCoreCache;
bool bLoadedTunnelDensityCache = false;
@@ -827,6 +834,7 @@ float UVoxelGenerator::GetDensityAt(float WorldX, float WorldY, float WorldZ) co
CP_TunnelDensityCache[TunnelDensityCacheSlot(ChunkCoord)];
const bool bCacheKeyMatches = CachedEntry.bValid
&& CachedEntry.OwnerId == DensityCacheOwnerId
&& CachedEntry.ManagerLifetimeId == ManagerLifetimeId
&& CachedEntry.Chunk == ChunkCoord
&& CachedEntry.LayoutVersion == LayoutVersion
&& (CachedEntry.GenType == ECaveGeneratorType::TunnelNetwork
@@ -836,6 +844,7 @@ float UVoxelGenerator::GetDensityAt(float WorldX, float WorldY, float WorldZ) co
if (CP_ActiveTunnelDensityCache != &CachedEntry)
{
CP_OwnerId = CachedEntry.OwnerId;
CP_ManagerLifetimeId = CachedEntry.ManagerLifetimeId;
CP_Version = CachedEntry.LayoutVersion;
CP_Chunk = CachedEntry.Chunk;
CP_GenType = CachedEntry.GenType;
@@ -856,8 +865,10 @@ float UVoxelGenerator::GetDensityAt(float WorldX, float WorldY, float WorldZ) co
}
const bool bOwnerChanged = DensityCacheOwnerId != CP_OwnerId;
const bool bManagerChanged = ManagerLifetimeId != CP_ManagerLifetimeId;
if (!bLoadedTunnelDensityCache
&& (bOwnerChanged || ChunkCoord != CP_Chunk || LayoutVersion != CP_Version))
&& (bOwnerChanged || bManagerChanged || ChunkCoord != CP_Chunk
|| LayoutVersion != CP_Version))
{
CP_ActiveTunnelDensityCache = nullptr;
@@ -867,8 +878,12 @@ float UVoxelGenerator::GetDensityAt(float WorldX, float WorldY, float WorldZ) co
// change : deux mondes peuvent partager version et coordonnées, jamais leur contexte.
// The biome grid's XY box says nothing about its context. Owner changes invalidate it
// too: two worlds may share version and coordinates, never cached params/context.
if (bOwnerChanged || LayoutVersion != CP_Version) { CP_BiomeCache.Invalidate(); }
if (bOwnerChanged || bManagerChanged || LayoutVersion != CP_Version)
{
CP_BiomeCache.Invalidate();
}
CP_OwnerId = DensityCacheOwnerId;
CP_ManagerLifetimeId = ManagerLifetimeId;
CP_Version = LayoutVersion;
CP_Chunk = ChunkCoord;
CP_GenType = StrateManager->GetGeneratorTypeForChunk(ChunkCoord);
@@ -944,7 +959,7 @@ float UVoxelGenerator::GetDensityAt(float WorldX, float WorldY, float WorldZ) co
{
PrepareTunnelCoreCache(
*StrateManager, ChunkCoord, CP_Tunnel, CP_TunnelFP,
LayoutVersion, DensityCacheOwnerId, GTunnelCoreCache);
LayoutVersion, ManagerLifetimeId, DensityCacheOwnerId, GTunnelCoreCache);
}
else
{
@@ -1072,6 +1087,7 @@ float UVoxelGenerator::GetDensityAt(float WorldX, float WorldY, float WorldZ) co
CP_TunnelDensityCache[TunnelDensityCacheSlot(ChunkCoord)];
CachedEntry.bValid = false;
CachedEntry.OwnerId = DensityCacheOwnerId;
CachedEntry.ManagerLifetimeId = ManagerLifetimeId;
CachedEntry.Chunk = ChunkCoord;
CachedEntry.LayoutVersion = LayoutVersion;
CachedEntry.GenType = CP_GenType;
@@ -12,6 +12,18 @@
#include "VoxelBiomeDefinition.h" // For UVoxelBiomeDefinition (biome context flatten)
#include "UObject/UObjectGlobals.h"
#include <atomic>
namespace
{
std::atomic<uint64> GNextStrateManagerLifetimeId { 0 };
}
UVoxelStrateManager::UVoxelStrateManager()
: CacheLifetimeId(GNextStrateManagerLifetimeId.fetch_add(1, std::memory_order_relaxed) + 1)
{
}
#if WITH_EDITOR
#include "VoxelStrateComposer.h"
@@ -73,6 +85,7 @@ namespace
struct FPassageEvaluationCache
{
const UVoxelStrateManager* Owner = nullptr;
uint64 OwnerLifetimeId = 0;
FIntVector Chunk = FIntVector(INT32_MAX, INT32_MAX, INT32_MAX);
uint32 Version = 0xFFFFFFFFu;
TArray<int32> Nearby;
@@ -90,12 +103,15 @@ namespace
{
FPassageEvaluationCache& Cache = VF_GetPassageEvaluationCache();
const uint32 Version = Manager ? Manager->GetLayoutVersion() : 0u;
if (Cache.Owner == Manager && Cache.Chunk == ChunkCoord && Cache.Version == Version)
const uint64 LifetimeId = Manager ? Manager->GetCacheLifetimeId() : 0;
if (Cache.Owner == Manager && Cache.OwnerLifetimeId == LifetimeId
&& Cache.Chunk == ChunkCoord && Cache.Version == Version)
{
return Cache.Nearby;
}
Cache.Owner = Manager;
Cache.OwnerLifetimeId = LifetimeId;
Cache.Chunk = ChunkCoord;
Cache.Version = Version;
Cache.Nearby.Reset();
+7 -6
View File
@@ -857,17 +857,18 @@ void AVoxelWorld::EndPlay(const EEndPlayReason::Type EndPlayReason)
// Signal all async tasks to bail out ASAP
bShuttingDown.store(true, std::memory_order_release);
// Wait for all running tasks to finish before destroying UObjects.
// Tasks check bShuttingDown and exit early, so this should be fast.
// Timeout after 3 seconds to avoid hanging the editor.
// Wait for every running task before destroying UObjects. A timeout here is unsafe: a task
// already inside GenerateTileResult may still be evaluating a cached op stack that points at
// this world's manager. Log a slow drain, but never tear the world down underneath a reader.
const double Deadline = FPlatformTime::Seconds() + 3.0;
bool bReportedSlowDrain = false;
while (ActiveTaskCount.load(std::memory_order_relaxed) > 0)
{
if (FPlatformTime::Seconds() > Deadline)
if (!bReportedSlowDrain && FPlatformTime::Seconds() > Deadline)
{
UE_LOG(LogTemp, Warning, TEXT("[VoxelWorld] EndPlay: %d tasks still running after 3s timeout"),
UE_LOG(LogTemp, Warning, TEXT("[VoxelWorld] EndPlay: waiting for %d tasks after 3s; UObject teardown is deferred"),
ActiveTaskCount.load(std::memory_order_relaxed));
break;
bReportedSlowDrain = true;
}
FPlatformProcess::Yield(); // Give CPU to other threads
}
@@ -145,6 +145,8 @@ class VOXELFORGE_API UVoxelStrateManager : public UObject
GENERATED_BODY()
public:
UVoxelStrateManager();
//=========================================================================
// INITIALIZATION
//=========================================================================
@@ -178,6 +180,15 @@ public:
const FString& GetSeasonContentHash() const { return ActiveSeasonContentHash; }
int32 GetWorldSeed() const { return CachedSeed; }
/**
* Monotonic identity of this manager UObject instance.
*
* PassagesVersion intentionally starts over for every layout, so it cannot distinguish a
* cached stack built by an older PIE manager that happened to have the same layout version.
* Worker-local generator caches use this identity as their manager-lifetime boundary.
*/
uint64 GetCacheLifetimeId() const { return CacheLifetimeId; }
#if WITH_EDITOR
/**
* Install an editor-only candidate over one already-built slot. The slot's definition,
@@ -430,6 +441,10 @@ protected:
// cached passage/strate data after either kind of live change.
uint32 PassagesVersion = 0;
// Never reused for the lifetime of the process. This is deliberately not a UObject pointer:
// allocator address reuse must not make a stale worker-local stack look current.
uint64 CacheLifetimeId = 0;
// How many chunks at strate boundaries are blended (transition zone)
int32 BlendChunks = 2;