Author SHA1 Message Date
Fr0zkaandClaude Opus 5.5 b5ed321b9e wip(L16, side branch): disk terrain cache design + headless Linux prototype
Exact lossless zlib tile records (keyed on world/season/recipe/plan/evaluator/options/tile),
pristine cache plus ordered edits, server never trusts client caches, corruption/stale rejection,
atomic writes, 2 GB cap. Loaded tiles match regenerated hashes (L1 view, 24 m crossing).
NOT for assembler/world yet: the shared StreamBenchmarkPipeline.h includes Linux-only headers
(unistd/fcntl/zlib), which would break the Windows viewer build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CSFYgsdW8ejZSyiMf4qRSX
2026-10-20 04:19:56 +02:00
35 changed files with 1502 additions and 1366 deletions

No files matched your search

-22
View File
@@ -1840,28 +1840,6 @@ fill and check page tiles against the evaluator they actually sample. This is th
the Unreal adapter and its asynchronous MPSC/Epoch flow are unchanged. See
[`Standalone/Research/WorldAssemblerM3/Linux13/REPORT.md`](Standalone/Research/WorldAssemblerM3/Linux13/REPORT.md).
### 8.37 Owner/Core stratum coordinates and FloatingIslands boundaries — World Assembler M3 L17
Manager chunk coordinates are in owner chunks, while Core density positions are Core voxels. Convert
between them with `32 * OwnerVoxelSizeMicrons / CoreVoxelSizeMicrons`; do not divide Core positions
by `CHUNK_SIZE` directly. The L17 owner snapshot uses 0.125 m owner voxels and 0.25 m Core voxels,
so one owner chunk is 16 Core voxels (4 m), not 32 (8 m). The old hard-coded divisor doubled the
manifest's vertical stratum bounds and misplaced native-only readiness queries.
FloatingIslands is the sparse `FloatingIslandFieldV1` manifest route. Its 0.5 m `boundarySeal`
limits feature influence; it is not a signed solid cap across the whole XY plane. Applying it as a
solid cap produced a false top at +3.5 m, while the doubled legacy layer bound also produced a cap at
+8 m. SurfaceWorld remains the planar `Heightfield2D` field with its independent ground and cap
surfaces. Adjacent native FI/SW strata now clip their signed distances to the same physical owner
boundary, preserving exact surface signs and height through the mesh sample lattice.
The output change increments `WorldAssemblyEvaluatorVersion` from 6 to 7; the plan schema and
recipe geometry version stay unchanged because their inputs did not change. The L17 probes found
zero cave/floor influence at the false square and at the sampled island ground. The shared Core
coordinate conversion also changes SurfaceWorld native readiness and hashes where the previous
mapping selected the wrong route. See
[`Standalone/Research/WorldAssemblerM3/Linux17/REPORT.md`](Standalone/Research/WorldAssemblerM3/Linux17/REPORT.md).
## 9. Multiplayer model (listen-server first, dedicated-friendly)
> **Status: transport is DESIGN ONLY — nothing is networked in-tree yet** (no `Replicated`/`HasAuthority`/RPCs; a
+1 -9
View File
@@ -1132,19 +1132,11 @@ Implementation and evidence: [`Standalone/Research/WorldAssemblerM3/Linux9/REPOR
| Symbol | Canonical file | Role |
|---|---|---|
| `FStandaloneShapeWorld::SetSharedDensitySource`, `SharedDensitySource` | `Standalone/Core/src/ShapeWorldInternal.h`, bound before publication in `Standalone/Core/src/VoxelForgeCore.cpp` | Makes compiled analytic pages delegate field, lattice, detail, classification, and solid queries to the Core world's base shape; `WorldAssemblyEvaluatorVersion` is 7. |
| `FStandaloneShapeWorld::SetSharedDensitySource`, `SharedDensitySource` | `Standalone/Core/src/ShapeWorldInternal.h`, bound before publication in `Standalone/Core/src/VoxelForgeCore.cpp` | Makes compiled analytic pages delegate field, lattice, detail, classification, and solid queries to the Core world's base shape; `WorldAssemblyEvaluatorVersion` is 6. |
The base manifest field is authoritative for tile generation and exact queries. See ARCHITECTURE
§8.36 and [`Standalone/Research/WorldAssemblerM3/Linux13/REPORT.md`](Standalone/Research/WorldAssemblerM3/Linux13/REPORT.md).
### 3.23 Owner/Core stratum coordinates — World Assembler M3 L17
`FStandaloneShapeWorld::GetCoreVoxelsPerOwnerChunk` is the shared conversion for manager chunk
lookups, manifest layer bounds, readiness proofs, and adjacent native strata. FloatingIslands uses
`FloatingIslandFieldV1`; its boundary seal bounds feature influence and does not fill an XY-wide
solid cap. SurfaceWorld remains the `Heightfield2D` path. See ARCHITECTURE §8.37 and
[`Standalone/Research/WorldAssemblerM3/Linux17/REPORT.md`](Standalone/Research/WorldAssemblerM3/Linux17/REPORT.md).
## 4. The density pipeline (most-edited hot path)
### 4.1 `GetDensityWithParams` (TunnelNetwork) — VoxelGenerator.cpp:3749
+1 -1
View File
@@ -118,7 +118,7 @@ PIDS=()
CORE_LIBRARY="$BUILD_DIR/libVoxelForgeCore.a"
"$(command -v ar)" rcs "$CORE_LIBRARY" "${CORE_OBJECTS[@]}"
OUTPUT="$BIN_DIR/VoxelForgeCoreHeadless-$VARIANT"
"$COMPILER" -pthread "${LEGACY_OBJECTS[@]}" "$CORE_LIBRARY" -o "$OUTPUT"
"$COMPILER" -pthread "${LEGACY_OBJECTS[@]}" "$CORE_LIBRARY" -lz -o "$OUTPUT"
CAPTURE_OUTPUT="$BIN_DIR/VoxelForgeOffscreenCapture-$VARIANT"
"$COMPILER" -std=c++20 -O2 -fno-fast-math -pthread \
"$ROOT/Source/LinuxOffscreenCapture.cpp" -o "$CAPTURE_OUTPUT" \
@@ -249,8 +249,8 @@ namespace VoxelForgeCore
// L'assembleur v1 encode coordonnees et scalaires en 1/256 de voxel, independamment des tuiles.
inline constexpr std::int64_t WorldAssemblySubvoxelsPerVoxel = 256;
inline constexpr std::uint32_t WorldAssemblySchemaVersion = 1;
// Version 7 fixes owner/Core chunk mapping and native adjacent-stratum boundaries.
inline constexpr std::uint32_t WorldAssemblyEvaluatorVersion = 7;
// Version 6 binds every compiled tile page to the world's base density evaluator.
inline constexpr std::uint32_t WorldAssemblyEvaluatorVersion = 6;
enum class FieldChannel : std::uint32_t
{
+51 -144
View File
@@ -47,18 +47,6 @@ public:
return static_cast<double>(CHUNK_SIZE) * OwnerVoxelSizeCm / GetCoreVoxelSizeCm();
}
int32 OwnerChunkLowerBoundaryCoreVoxel(int32 ChunkZ) const
{
return static_cast<int32>(std::floor(
static_cast<double>(ChunkZ) * GetCoreVoxelsPerOwnerChunk()));
}
int32 OwnerChunkUpperBoundaryCoreVoxel(int32 ChunkZ) const
{
return static_cast<int32>(std::ceil(
static_cast<double>(ChunkZ) * GetCoreVoxelsPerOwnerChunk()));
}
enum class EDetailVariant : uint8 { Coherent, Smooth, PerVoxel };
struct FBoxClassification
{
@@ -724,21 +712,14 @@ public:
if(Owner&&Owner->Manager)
{
const double TileZ0=Origin.Z,TileZ1=Origin.Z+static_cast<double>(Cells)*Step;
const double CoreVoxelsPerOwnerChunk=GetCoreVoxelsPerOwnerChunk();
for(const FStrateSlot& Slot:Owner->Manager->GetLayout())
{
const int32 MidChunkZ=(Slot.TopChunkZ+Slot.BottomChunkZ)/2;
if(bUseNativeManifestFields
&&Owner->Manager->GetGeneratorTypeForChunk(FIntVector(0,0,MidChunkZ))
==ECaveGeneratorType::FloatingIslands)
continue;
const FStrateGenerationParams Params=Owner->Manager->GetGenerationParams(
FIntVector(0,0,MidChunkZ));
FIntVector(0,0,(Slot.TopChunkZ+Slot.BottomChunkZ)/2));
const float HalfWidth=static_cast<float>(std::max(8.0,
static_cast<double>(Params.BoundarySealThickness)+8.0));
const float Boundaries[2]={
static_cast<float>(static_cast<double>(Slot.BottomChunkZ)*CoreVoxelsPerOwnerChunk),
static_cast<float>(static_cast<double>(Slot.TopChunkZ+1)*CoreVoxelsPerOwnerChunk)};
const float Boundaries[2]={static_cast<float>(Slot.BottomChunkZ)*CHUNK_SIZE,
static_cast<float>(Slot.TopChunkZ+1)*CHUNK_SIZE};
for(const float Z:Boundaries)
if(Z+HalfWidth>=TileZ0&&Z-HalfWidth<=TileZ1)
Out.stratumBoundaries.push_back({Z,HalfWidth});
@@ -1047,21 +1028,19 @@ public:
float NativeDensity=0.0f;
if(!NativeSurfaceDensityAt(X,Y,Z,NativeDensity))
{RecordNativeOnlyViolation();return 0.0f;}
return ApplyAdjacentStratumBoundary(X,Y,Z,NativeDensity,false);
return NativeDensity;
}
float IslandDensity=0.0f;
if(NativeFloatingIslandDensityAt(X,Y,Z,IslandDensity))
{
++ThreadClassificationStats().NativeFloatFieldCalls;
return ApplyEditBrushes(ApplyAdjacentStratumBoundary(
X,Y,Z,IslandDensity,true),X,Y,Z);
return ApplyEditBrushes(IslandDensity,X,Y,Z);
}
float NativeDensity=0.0f;
if(NativeSurfaceDensityAt(X,Y,Z,NativeDensity))
{
++ThreadClassificationStats().NativeFloatFieldCalls;
return ApplyEditBrushes(ApplyAdjacentStratumBoundary(
X,Y,Z,NativeDensity,false),X,Y,Z);
return ApplyEditBrushes(NativeDensity,X,Y,Z);
}
if(UsesLegacySurfaceField(X,Y,Z))
return LegacySurfaceDensityAt(X,Y,Z);
@@ -1085,27 +1064,21 @@ public:
if(!NativeSurfaceDensityAt(static_cast<float>(X),static_cast<float>(Y),
static_cast<float>(Z),NativeDensity))
{RecordNativeOnlyViolation();return 0.0f;}
return ApplyAdjacentStratumBoundary(
static_cast<float>(X),static_cast<float>(Y),static_cast<float>(Z),
NativeDensity,false);
return NativeDensity;
}
float IslandDensity=0.0f;
if(NativeFloatingIslandDensityAt(static_cast<float>(X),static_cast<float>(Y),
static_cast<float>(Z),IslandDensity))
{
++ThreadClassificationStats().NativeLatticeFieldCalls;
return ApplyEditBrushes(ApplyAdjacentStratumBoundary(
static_cast<float>(X),static_cast<float>(Y),static_cast<float>(Z),
IslandDensity,true),X,Y,Z);
return ApplyEditBrushes(IslandDensity,X,Y,Z);
}
float NativeDensity=0.0f;
if(NativeSurfaceDensityAt(static_cast<float>(X),static_cast<float>(Y),
static_cast<float>(Z),NativeDensity))
{
++ThreadClassificationStats().NativeLatticeFieldCalls;
return ApplyEditBrushes(ApplyAdjacentStratumBoundary(
static_cast<float>(X),static_cast<float>(Y),static_cast<float>(Z),
NativeDensity,false),X,Y,Z);
return ApplyEditBrushes(NativeDensity,X,Y,Z);
}
if(UsesLegacySurfaceField(static_cast<float>(X),static_cast<float>(Y),
static_cast<float>(Z)))
@@ -1129,21 +1102,19 @@ public:
float NativeDensity=0.0f;
if(!NativeSurfaceDensityAt(X,Y,Z,NativeDensity))
{RecordNativeOnlyViolation();return 0.0f;}
return ApplyAdjacentStratumBoundary(X,Y,Z,NativeDensity,false);
return NativeDensity;
}
float IslandDensity=0.0f;
if(NativeFloatingIslandDensityAt(X,Y,Z,IslandDensity))
{
++ThreadClassificationStats().NativeUnclampedFieldCalls;
return ApplyEditBrushes(ApplyAdjacentStratumBoundary(
X,Y,Z,IslandDensity,true),X,Y,Z);
return ApplyEditBrushes(IslandDensity,X,Y,Z);
}
float NativeDensity=0.0f;
if(NativeSurfaceDensityAt(X,Y,Z,NativeDensity))
{
++ThreadClassificationStats().NativeUnclampedFieldCalls;
return ApplyEditBrushes(ApplyAdjacentStratumBoundary(
X,Y,Z,NativeDensity,false),X,Y,Z);
return ApplyEditBrushes(NativeDensity,X,Y,Z);
}
if(UsesLegacySurfaceField(X,Y,Z))
return LegacySurfaceDensityAt(X,Y,Z);
@@ -1289,18 +1260,12 @@ private:
if(OutFloorInfluence)*OutFloorInfluence=0.0f;
float IslandDensity=0.0f;
if(NativeFloatingIslandDensityAt(X,Y,Z,IslandDensity))
{
IslandDensity=ApplyAdjacentStratumBoundary(X,Y,Z,IslandDensity,true);
return bApplyEdits?ApplyEditBrushes(IslandDensity,X,Y,Z):IslandDensity;
}
if(UsesLegacySurfaceField(X,Y,Z))
{
float NativeDensity=0.0f;
if(NativeSurfaceDensityAt(X,Y,Z,NativeDensity))
{
NativeDensity=ApplyAdjacentStratumBoundary(X,Y,Z,NativeDensity,false);
return bApplyEdits?ApplyEditBrushes(NativeDensity,X,Y,Z):NativeDensity;
}
const float Density=Owner->Generator->GetDensityAt(X,Y,Z);
return bApplyEdits?ApplyEditBrushes(Density,X,Y,Z):Density;
}
@@ -1356,11 +1321,8 @@ public:
return SharedDensitySource->GetNativeSurfaceHeightsAt(X,Y,ChunkZ,OutGroundZ,OutCeilingZ);
if(!WorldAssembly||!Owner||!Owner->Manager||!Owner->Generator
||!std::isfinite(X)||!std::isfinite(Y))return false;
const double CoreVoxelsPerOwnerChunk=GetCoreVoxelsPerOwnerChunk();
const std::int64_t chunkMinimum=static_cast<std::int64_t>(std::floor(
static_cast<double>(ChunkZ)*CoreVoxelsPerOwnerChunk));
const std::int64_t chunkMaximum=static_cast<std::int64_t>(std::ceil(
static_cast<double>(ChunkZ+1)*CoreVoxelsPerOwnerChunk));
const std::int64_t chunkMinimum=static_cast<std::int64_t>(ChunkZ)*CHUNK_SIZE;
const std::int64_t chunkMaximum=chunkMinimum+CHUNK_SIZE;
if(bNativeOnly&&(!NativeOnlyContainsColumn(X,Y)
||chunkMaximum<NativeOnlyMinimum.Z||chunkMinimum>NativeOnlyMaximum.Z
||Owner->Manager->GetGeneratorTypeForChunk(FIntVector(0,0,ChunkZ))
@@ -1382,8 +1344,7 @@ public:
const VoxelForgeCore::Heightfield2D Block(WorldAssembly->recipe,*Plan);
if(!Block.isValid())return false;
const auto Column=Block.point(ToFixed(X),ToFixed(Y),
static_cast<int64>(std::llround(static_cast<double>(ChunkZ)
*CoreVoxelsPerOwnerChunk*VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel)));
static_cast<int64>(ChunkZ)*CHUNK_SIZE*VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel);
OutGroundZ=static_cast<float>(Column.groundHeightSubvoxels/Scale);
OutCeilingZ=static_cast<float>(Column.capHeightSubvoxels/Scale);
return std::isfinite(OutGroundZ)&&std::isfinite(OutCeilingZ);
@@ -1399,11 +1360,10 @@ public:
const int32 Pad=std::max(Step,1);
const int32 BMin=Origin.Z-Pad;
const int32 BMax=Origin.Z+(Cells+1)*Step;
const double CoreVoxelsPerOwnerChunk=GetCoreVoxelsPerOwnerChunk();
const int32 FirstChunk=FMath::FloorToInt(static_cast<float>(BMin)
/static_cast<float>(CoreVoxelsPerOwnerChunk));
/static_cast<float>(CHUNK_SIZE));
const int32 LastChunk=FMath::FloorToInt(static_cast<float>(BMax)
/static_cast<float>(CoreVoxelsPerOwnerChunk));
/static_cast<float>(CHUNK_SIZE));
const VoxelForgeCore::WorldAssemblyStratumPlanV1* Plan=nullptr;
for(int32 ChunkZ=FirstChunk;ChunkZ<=LastChunk;++ChunkZ)
{
@@ -1447,13 +1407,12 @@ public:
bool bIslandOtherStrateOverlap=false,bBadPlan=false;
if(Owner&&Owner->Manager)
{
// All manifest strata use the same resolved owner-chunk to Core-voxel scale.
// Toutes les strates du manifeste partagent l'echelle owner/Core resolue.
const double CoreVoxelsPerOwnerChunk=GetCoreVoxelsPerOwnerChunk();
// Keep the delivered SurfaceWorld chunk mapping unchanged.
// Conserver le mapping de chunks SurfaceWorld deja livre.
const int32 FirstChunk=FMath::FloorToInt(static_cast<float>(BMin[2])
/static_cast<float>(CoreVoxelsPerOwnerChunk));
/static_cast<float>(CHUNK_SIZE));
const int32 LastChunk=FMath::FloorToInt(static_cast<float>(BMax[2])
/static_cast<float>(CoreVoxelsPerOwnerChunk));
/static_cast<float>(CHUNK_SIZE));
for(int32 ChunkZ=FirstChunk;ChunkZ<=LastChunk;++ChunkZ)
{
const ECaveGeneratorType Type=Owner->Manager->GetGeneratorTypeForChunk(
@@ -1585,6 +1544,19 @@ public:
{
if(!IslandPlan)return Finish(EVoxelTileClass::Mixed,true);
constexpr std::int64_t Scale=VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel;
const std::int64_t Bottom=IslandPlan->manifest.recipe.layoutMinimumZSubvoxels;
const std::int64_t Top=IslandPlan->manifest.recipe.layoutMaximumZSubvoxels;
const std::int64_t Seal=IslandPlan->manifest.recipe.boundarySealSubvoxels;
const std::int64_t BoxBottom=static_cast<std::int64_t>(BMin[2])*Scale;
const std::int64_t BoxTop=static_cast<std::int64_t>(BMax[2])*Scale;
const bool bBottomSealOverlap=BoxBottom<Bottom+Seal&&BoxTop>=Bottom;
const bool bTopSealOverlap=BoxBottom<=Top&&BoxTop>Top-Seal;
if(bBottomSealOverlap)
return Finish(BoxBottom>=Bottom&&BoxTop<=Bottom+Seal
?EVoxelTileClass::AllSolid:EVoxelTileClass::Mixed);
if(bTopSealOverlap)
return Finish(BoxBottom>=Top-Seal&&BoxTop<=Top
?EVoxelTileClass::AllSolid:EVoxelTileClass::Mixed);
const VoxelForgeCore::FloatingIslandFieldV1 Block(
IslandPlan->manifest.recipe,IslandPlan->manifest.plan);
if(!Block.isValid())return Finish(EVoxelTileClass::Mixed,true);
@@ -2107,8 +2079,7 @@ private:
{
if(!bUseNativeManifestFields||!WorldAssembly||!Owner||!Owner->Manager||!Owner->Generator
||!std::isfinite(X)||!std::isfinite(Y)||!std::isfinite(Z))return false;
const int32 ChunkZ=FMath::FloorToInt(Z
/static_cast<float>(GetCoreVoxelsPerOwnerChunk()));
const int32 ChunkZ=FMath::FloorToInt(Z/static_cast<float>(CHUNK_SIZE));
if(Owner->Manager->GetGeneratorTypeForChunk(FIntVector(0,0,ChunkZ))
!=ECaveGeneratorType::SurfaceWorld)return false;
const int32 StrateIndex=Owner->Manager->GetStrateIndex(
@@ -2154,79 +2125,23 @@ private:
const VoxelForgeCore::FloatingIslandFieldV1 Field(
Plan->manifest.recipe,Plan->manifest.plan);
if(!Field.isValid())return false;
// The recipe's boundary seal is a feature margin, not a horizontal solid cap.
// La marge de scellement du plan borne ses formes, sans remplir tout le plan XY.
const std::int64_t Density=Field.densitySubvoxels(ToFixed(X),ToFixed(Y),ToFixed(Z));
std::int64_t Density=Field.densitySubvoxels(ToFixed(X),ToFixed(Y),ToFixed(Z));
const std::int64_t Bottom=Plan->manifest.recipe.layoutMinimumZSubvoxels;
const std::int64_t Top=Plan->manifest.recipe.layoutMaximumZSubvoxels;
const std::int64_t Seal=Plan->manifest.recipe.boundarySealSubvoxels;
const std::int64_t Zq=ToFixed(Z);
if(Zq>=Bottom&&Zq<Bottom+Seal)Density=std::min(Density,-std::max<std::int64_t>(1,Zq-Bottom));
if(Zq<=Top&&Zq>Top-Seal)Density=std::min(Density,-std::max<std::int64_t>(1,Top-Zq));
OutDensity=static_cast<float>(static_cast<double>(Density)/Scale);
return std::isfinite(OutDensity);
}
float ApplyAdjacentStratumBoundary(float X,float Y,float Z,float Density,
bool bFloatingIsland) const
{
if(!bUseNativeManifestFields||!WorldAssembly||!Owner||!Owner->Manager
||!std::isfinite(X)||!std::isfinite(Y)||!std::isfinite(Z))return Density;
const double CoreVoxelsPerOwnerChunk=GetCoreVoxelsPerOwnerChunk();
const int32 ChunkZ=FMath::FloorToInt(Z/static_cast<float>(CoreVoxelsPerOwnerChunk));
const ECaveGeneratorType CurrentType=Owner->Manager->GetGeneratorTypeForChunk(
FIntVector(0,0,ChunkZ));
if(bFloatingIsland&&CurrentType==ECaveGeneratorType::FloatingIslands)
{
const int32 StrateIndex=Owner->Manager->GetStrateIndex(
static_cast<float>((static_cast<double>(ChunkZ)*CHUNK_SIZE+CHUNK_SIZE/2)*VOXEL_SIZE));
const auto* Plan=FindWorldAssemblyFloatingIslandStratum(StrateIndex,ChunkZ);
if(!Plan)return Density;
const int32 LowerChunk=Plan->bottomChunkZ-1;
if(Owner->Manager->GetGeneratorTypeForChunk(FIntVector(0,0,LowerChunk))
!=ECaveGeneratorType::SurfaceWorld)return Density;
const double Boundary=static_cast<double>(Plan->bottomChunkZ)
*CoreVoxelsPerOwnerChunk;
float LowerDensity=0.0f,UpperDensity=0.0f;
if(!NativeSurfaceDensityAt(X,Y,static_cast<float>(Boundary-0.5),LowerDensity)
||!NativeFloatingIslandDensityAt(X,Y,static_cast<float>(Boundary+0.5),UpperDensity)
||(LowerDensity<0.0f)==(UpperDensity<0.0f))return Density;
// Clip the two signed-distance fields to their shared owner-stratum plane.
// This keeps a solid-below/air-above seam continuous at every mesh sample step.
// Les deux champs sont limites au plan commun, quel que soit le pas de maillage.
const float Plane=static_cast<float>(static_cast<double>(Z)-Boundary);
return LowerDensity<0.0f?std::min(Density,Plane)
:std::max(Density,-Plane);
}
if(!bFloatingIsland&&CurrentType==ECaveGeneratorType::SurfaceWorld)
{
const int32 StrateIndex=Owner->Manager->GetStrateIndex(
static_cast<float>((static_cast<double>(ChunkZ)*CHUNK_SIZE+CHUNK_SIZE/2)*VOXEL_SIZE));
const auto* Plan=FindWorldAssemblyStratum(StrateIndex,ChunkZ);
if(!Plan)return Density;
const int32 UpperChunk=Plan->topChunkZ+1;
if(Owner->Manager->GetGeneratorTypeForChunk(FIntVector(0,0,UpperChunk))
!=ECaveGeneratorType::FloatingIslands)return Density;
const auto* UpperPlan=FindWorldAssemblyFloatingIslandStratum(
Owner->Manager->GetStrateIndex(static_cast<float>((
static_cast<double>(UpperChunk)*CHUNK_SIZE+CHUNK_SIZE/2)*VOXEL_SIZE)),UpperChunk);
if(!UpperPlan)return Density;
const double Boundary=static_cast<double>(UpperPlan->bottomChunkZ)
*CoreVoxelsPerOwnerChunk;
float LowerDensity=0.0f,UpperDensity=0.0f;
if(!NativeSurfaceDensityAt(X,Y,static_cast<float>(Boundary-0.5),LowerDensity)
||!NativeFloatingIslandDensityAt(X,Y,static_cast<float>(Boundary+0.5),UpperDensity)
||(LowerDensity<0.0f)==(UpperDensity<0.0f))return Density;
const float Plane=static_cast<float>(static_cast<double>(Z)-Boundary);
return LowerDensity<0.0f?std::max(Density,Plane)
:std::min(Density,-Plane);
}
return Density;
}
bool UsesLegacySurfaceField(float X,float Y,float Z) const
{
if(!Owner||!Owner->Manager||!Owner->Generator)return false;
const float CoreVoxelsPerOwnerChunk=static_cast<float>(GetCoreVoxelsPerOwnerChunk());
const FIntVector ChunkCoord(FMath::FloorToInt(X/CoreVoxelsPerOwnerChunk),
FMath::FloorToInt(Y/CoreVoxelsPerOwnerChunk),
FMath::FloorToInt(Z/CoreVoxelsPerOwnerChunk));
const FIntVector ChunkCoord(FMath::FloorToInt(X/static_cast<float>(CHUNK_SIZE)),
FMath::FloorToInt(Y/static_cast<float>(CHUNK_SIZE)),
FMath::FloorToInt(Z/static_cast<float>(CHUNK_SIZE)));
return Owner->Manager->GetGeneratorTypeForChunk(ChunkCoord)
==ECaveGeneratorType::SurfaceWorld;
}
@@ -2436,26 +2351,19 @@ private:
int32 Top=MIN_int32,Bottom=MAX_int32;
for(const FStrateSlot& Slot:Layout)
{
const int32 MidChunkZ=(Slot.TopChunkZ+Slot.BottomChunkZ)/2;
// A manifest-backed FloatingIslands stratum supplies its own sparse island
// field; it must not extend the legacy solid envelope across the open sky.
if(bUseNativeManifestFields
&&Owner->Manager->GetGeneratorTypeForChunk(FIntVector(0,0,MidChunkZ))
==ECaveGeneratorType::FloatingIslands)
continue;
Top=std::max(Top,Slot.TopChunkZ);Bottom=std::min(Bottom,Slot.BottomChunkZ);
if(!Slot.Definition)continue;
const int32 MidChunkZ=(Slot.TopChunkZ+Slot.BottomChunkZ)/2;
FLayerField Field;
Field.Lo=OwnerChunkLowerBoundaryCoreVoxel(Slot.BottomChunkZ);
Field.Hi=OwnerChunkUpperBoundaryCoreVoxel(Slot.TopChunkZ+1);
Field.Lo=Slot.BottomChunkZ*CHUNK_SIZE;
Field.Hi=(Slot.TopChunkZ+1)*CHUNK_SIZE;
Field.StrateIndex=Slot.StrateIndex;
Field.Params=Owner->Manager->GetGenerationParams(FIntVector(0,0,MidChunkZ));
LayerFields.push_back(std::move(Field));
}
if(Top>=Bottom)
{
LayerRanges.push_back({OwnerChunkLowerBoundaryCoreVoxel(Bottom),
OwnerChunkUpperBoundaryCoreVoxel(Top+1)});
LayerRanges.push_back({Bottom*CHUNK_SIZE,(Top+1)*CHUNK_SIZE});
}
}
@@ -3566,9 +3474,8 @@ private:
auto Unit=[&](uint32 H){return static_cast<double>(Hash(H)>>8)*(1.0/16777216.0);};
for(const FStrateSlot& Slot:Layout)
{
const double CoreVoxelsPerOwnerChunk=GetCoreVoxelsPerOwnerChunk();
const double ZBottom=static_cast<double>(Slot.BottomChunkZ)*CoreVoxelsPerOwnerChunk;
const double ZTop=static_cast<double>(Slot.TopChunkZ+1)*CoreVoxelsPerOwnerChunk;
const double ZBottom=static_cast<double>(Slot.BottomChunkZ)*CHUNK_SIZE;
const double ZTop=static_cast<double>(Slot.TopChunkZ+1)*CHUNK_SIZE;
for(int32 GY=GY0;GY<=GY1;++GY)for(int32 GX=GX0;GX<=GX1;++GX)
{
uint32 H=Hash(Seed^static_cast<uint32>(GX)*0x9E3779B9u
+5 -7
View File
@@ -1162,13 +1162,8 @@ namespace VoxelForgeCore
const std::int64_t tileMinimum[3]={boundsMin.X,boundsMin.Y,boundsMin.Z};
const std::int64_t tileMaximum[3]={boundsMax.X,boundsMax.Y,boundsMax.Z};
const double coreVoxelsPerOwnerChunk=static_cast<double>(CHUNK_SIZE)
*static_cast<double>(impl_->owner->OwnerVoxelSizeMicrons)
/static_cast<double>(std::max(1u,impl_->owner->CoreVoxelSizeMicrons));
const int32 firstSurfaceChunk=FMath::FloorToInt(
static_cast<double>(boundsMin.Z)/coreVoxelsPerOwnerChunk);
const int32 lastSurfaceChunk=FMath::FloorToInt(
static_cast<double>(boundsMax.Z)/coreVoxelsPerOwnerChunk);
const int32 firstSurfaceChunk=FloorDiv(boundsMin.Z,CHUNK_SIZE);
const int32 lastSurfaceChunk=FloorDiv(boundsMax.Z,CHUNK_SIZE);
const WorldAssemblyStratumPlanV1* selectedPlan=nullptr;
for(int32 chunkZ=firstSurfaceChunk;chunkZ<=lastSurfaceChunk;++chunkZ)
{
@@ -1188,6 +1183,9 @@ namespace VoxelForgeCore
if(!selectedPlan||!Heightfield2D(impl_->worldAssembly->recipe,*selectedPlan).isValid())
return false;
const double coreVoxelsPerOwnerChunk=static_cast<double>(CHUNK_SIZE)
*static_cast<double>(impl_->owner->OwnerVoxelSizeMicrons)
/static_cast<double>(std::max(1u,impl_->owner->CoreVoxelSizeMicrons));
const int32 firstIslandChunk=FMath::FloorToInt(
static_cast<double>(boundsMin.Z)/coreVoxelsPerOwnerChunk);
const int32 lastIslandChunk=FMath::FloorToInt(
@@ -0,0 +1,88 @@
#!/usr/bin/env python3
"""Per-tile zstd CLI comparison over the pristine L1 records from this round."""
from pathlib import Path
import shutil
import struct
import subprocess
import time
import zlib
REPO = Path.cwd().resolve()
CACHE = REPO / "Saved/Linux/cache/l16-prototype"
CAPTURE_LOG = REPO / "Saved/Linux/l16-cache-l1-q24-full-write.log"
TEMP = CACHE / "codec-zstd"
RESULT = REPO / "Saved/Linux/l16-zstd-codec.txt"
assert CACHE.is_dir() and not CACHE.is_symlink()
assert CACHE.resolve().parent == (REPO / "Saved/Linux/cache").resolve()
assert CAPTURE_LOG.is_file()
assert not TEMP.exists()
cutoff_ns = CAPTURE_LOG.stat().st_mtime_ns
records = [
path for world in CACHE.glob("world_*")
for path in world.glob("*.vftile")
if "_s0_d0000000000000000_" in path.name and path.stat().st_mtime_ns <= cutoff_ns
]
records.sort()
assert len(records) == 2553, f"expected 2553 L1 pristine records, found {len(records)}"
codec = shutil.which("zstd")
assert codec, "zstd CLI is not available"
TEMP.mkdir()
per_lod = {}
try:
for source in records:
record = source.read_bytes()
assert record[:8] == b"VFTILE16"
raw_size, compressed_size = struct.unpack_from("<QQ", record, 124)
raw = zlib.decompress(record[144:144 + compressed_size])
assert len(raw) == raw_size
lod = int(source.name.split("_", 1)[0][3:])
entry = per_lod.setdefault(lod, {"raw": 0, "files": [], "encode_ns": 0, "decode_ns": 0, "zstd": 0})
raw_file = TEMP / (source.stem + ".raw")
raw_file.write_bytes(raw)
entry["raw"] += len(raw)
entry["files"].append(raw_file)
for lod in sorted(per_lod):
entry = per_lod[lod]
start = time.perf_counter_ns()
subprocess.run([codec, "-q", "-1", "-f", "--", *map(str, entry["files"])], check=True)
entry["encode_ns"] = time.perf_counter_ns() - start
compressed_files = [Path(str(path) + ".zst") for path in entry["files"]]
entry["zstd"] = sum(path.stat().st_size for path in compressed_files)
start = time.perf_counter_ns()
subprocess.run([codec, "-q", "-d", "-f", "--", *map(str, compressed_files)], check=True)
entry["decode_ns"] = time.perf_counter_ns() - start
# Verify each decoded output byte-for-byte against the zlib payload again.
# The filename is the tile stem, so a deterministic source lookup is sufficient.
source_by_stem = {path.stem: path for path in records}
for entry in per_lod.values():
for raw_file in entry["files"]:
source = source_by_stem[raw_file.stem]
data = source.read_bytes()
_, compressed_size = struct.unpack_from("<QQ", data, 124)
assert raw_file.read_bytes() == zlib.decompress(data[144:144 + compressed_size])
lines = ["lod tiles raw_total_B zstd1_total_B zstd1_avg_B encode_us_per_tile decode_us_per_tile status"]
for lod in sorted(per_lod):
entry = per_lod[lod]
count = len(entry["files"])
lines.append(
f"{lod} {count} {entry['raw']} {entry['zstd']} {entry['zstd'] / count:.1f} "
f"{entry['encode_ns'] / count / 1000:.2f} {entry['decode_ns'] / count / 1000:.2f} PASS"
)
raw_total = sum(entry["raw"] for entry in per_lod.values())
zstd_total = sum(entry["zstd"] for entry in per_lod.values())
lines.append(f"total_tiles={len(records)} raw_bytes={raw_total} zstd1_bytes={zstd_total} ratio={zstd_total / raw_total:.6f}")
lines.append("decode=byte-exact; timing includes zstd CLI process startup once per LOD")
RESULT.write_text("\n".join(lines) + "\n")
print("\n".join(lines))
finally:
resolved = TEMP.resolve()
assert resolved.parent == CACHE.resolve()
shutil.rmtree(TEMP)
File diff suppressed because it is too large. Load diff
@@ -1,89 +0,0 @@
#!/usr/bin/env python3
"""Measure upward mesh faces around the FI/SW stratum interface."""
import array
import collections
import math
import struct
import sys
def read_exact(stream, size):
data = stream.read(size)
if len(data) != size:
raise EOFError(f"wanted {size} bytes, got {len(data)}")
return data
path = sys.argv[1]
low_z = float(sys.argv[2]) if len(sys.argv) > 2 else -80.0
high_z = float(sys.argv[3]) if len(sys.argv) > 3 else -45.0
with open(path, "rb") as stream:
if read_exact(stream, 8) != b"VFMSH001":
raise SystemExit("bad VFMSH001 header")
version, voxel_um, tile_count = struct.unpack("<3I", read_exact(stream, 12))
camera = struct.unpack("<3d", read_exact(stream, 24))
yaw, pitch = struct.unpack("<2d", read_exact(stream, 16))
per_lod = collections.defaultdict(lambda: {"tiles": 0, "faces": [], "spans": []})
near_tiles = collections.defaultdict(list)
for _ in range(tile_count):
key = struct.unpack("<4i", read_exact(stream, 16))
surf_vertices, skirt_count, vertex_count, index_count, mask_count = struct.unpack(
"<5I", read_exact(stream, 20))
p = array.array("f")
p.frombytes(read_exact(stream, vertex_count * 12))
n = array.array("f")
n.frombytes(read_exact(stream, vertex_count * 12))
if sys.byteorder != "little":
p.byteswap(); n.byteswap()
indices = array.array("I")
indices.frombytes(read_exact(stream, index_count * 4))
masks = read_exact(stream, mask_count)
if sys.byteorder != "little":
indices.byteswap()
lod = key[3]
z_values = []
for tri in range(index_count // 3):
if masks and masks[tri] != 0:
continue
a, b, c = indices[tri * 3:tri * 3 + 3]
if max(a, b, c) >= surf_vertices:
continue
normal_z = (n[a * 3 + 2] + n[b * 3 + 2] + n[c * 3 + 2]) / 3.0
if normal_z < 0.65:
continue
z = (p[a * 3 + 2] + p[b * 3 + 2] + p[c * 3 + 2]) / 3.0
if low_z <= z <= high_z:
z_values.append(z)
if not z_values:
continue
z_values.sort()
row = per_lod[lod]
row["tiles"] += 1
row["faces"].extend(z_values)
span = z_values[math.floor(.95 * (len(z_values) - 1))] - z_values[math.floor(.05 * (len(z_values) - 1))]
row["spans"].append(span)
extent = 32 * (1 << lod) * voxel_um / 1_000_000.0
cx = (key[0] + 0.5) * extent
cy = (key[1] + 0.5) * extent
near_tiles[lod].append((math.hypot(cx - camera[0], cy - camera[1]), key,
z_values[len(z_values) // 2], span, len(z_values)))
def q(values, fraction):
return values[math.floor(fraction * (len(values) - 1))] if values else math.nan
print(f"camera_m={camera} yaw={yaw} pitch={pitch} boundary_band_m={low_z}..{high_z}")
print("lod tiles_with_boundary_faces faces z_m_p05/median/p95 tile_span_m_p50/p95/max")
for lod in sorted(per_lod):
row = per_lod[lod]
spans = sorted(row["spans"])
print(f"{lod} {row['tiles']} {len(row['faces'])} "
f"{q(sorted(row['faces']), .05):.4f}/{q(sorted(row['faces']), .5):.4f}/"
f"{q(sorted(row['faces']), .95):.4f} "
f"{q(spans, .5):.4f}/{q(spans, .95):.4f}/{max(spans):.4f}")
print("nearest_xy_boundary_tiles_by_lod:")
for lod in sorted(near_tiles):
for distance, key, median, span, faces in sorted(near_tiles[lod])[:3]:
print(f"lod={lod} key={key} xy_distance_m={distance:.2f} "
f"boundary_median_z={median:.4f} within_tile_p05_p95_m={span:.4f} faces={faces}")
@@ -1,143 +0,0 @@
#include "Json.h"
#include "StandaloneCoreScheduler.h"
#include "StandaloneSnapshotAdapter.h"
#include "VoxelForgeCore/VoxelForgeCore.h"
#include <algorithm>
#include <cmath>
#include <cstdint>
#include <cstdio>
#include <fstream>
#include <iterator>
#include <string>
namespace
{
constexpr double CoreVoxelsPerMeter = 4.0;
constexpr std::int64_t QPerCoreVoxel = VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel;
std::int64_t ToQ(double Meters)
{
return static_cast<std::int64_t>(std::llround(Meters*CoreVoxelsPerMeter*QPerCoreVoxel));
}
void PrintLegacyBoundarySealTerms(const VoxelForgeCore::FloatingIslandManifestV1& Manifest,
double X, double Y, double Z)
{
using namespace VoxelForgeCore;
const auto Xq=ToQ(X),Yq=ToQ(Y),Zq=ToQ(Z);
const FloatingIslandFieldV1 field(Manifest.recipe,Manifest.plan);
const std::int64_t fieldQ=field.densitySubvoxels(Xq,Yq,Zq);
const std::int64_t bottom=Manifest.recipe.layoutMinimumZSubvoxels;
const std::int64_t top=Manifest.recipe.layoutMaximumZSubvoxels;
const std::int64_t seal=Manifest.recipe.boundarySealSubvoxels;
std::int64_t bridged=fieldQ;
bool applied=false;
if(Zq>=bottom&&Zq<bottom+seal)
{bridged=std::min(bridged,-std::max<std::int64_t>(1,Zq-bottom));applied=true;}
if(Zq<=top&&Zq>top-seal)
{bridged=std::min(bridged,-std::max<std::int64_t>(1,top-Zq));applied=true;}
std::printf("[PreFixIslandTerms] xyz_m=%.3f,%.3f,%.3f native_field_q=%lld "
"legacy_solid_seal_delta_q=%lld legacy_seal_active=%d pre_fix_wrapper_q=%lld\n",
X,Y,Z,static_cast<long long>(fieldQ),
static_cast<long long>(bridged-fieldQ),applied?1:0,static_cast<long long>(bridged));
for(const auto& island:Manifest.plan.islands)
{
const bool inBounds=Xq>=island.influenceMinimumSubvoxels[0]
&&Xq<=island.influenceMaximumSubvoxels[0]
&&Yq>=island.influenceMinimumSubvoxels[1]
&&Yq<=island.influenceMaximumSubvoxels[1]
&&Zq>=island.influenceMinimumSubvoxels[2]
&&Zq<=island.influenceMaximumSubvoxels[2];
std::printf("[IslandGroup] ordinal=%u center_m=%.3f,%.3f,%.3f "
"bounds_contains_sample=%d\n",
island.ordinal,island.centerSubvoxels[0]/1024.0,island.centerSubvoxels[1]/1024.0,
island.centerSubvoxels[2]/1024.0,inBounds?1:0);
}
for(const auto& edge:Manifest.plan.clusterEdges)
{
bool inBounds=true;
const std::int64_t q[3]={Xq,Yq,Zq};
for(std::size_t axis=0;axis<3;++axis)
inBounds=inBounds&&q[axis]>=edge.influenceMinimumSubvoxels[axis]
&&q[axis]<=edge.influenceMaximumSubvoxels[axis];
std::printf("[IslandConnector] id=%016llx bounds_contains_sample=%d\n",
static_cast<unsigned long long>(edge.stableId),inBounds?1:0);
}
}
}
void PrintDensityLine(VoxelForgeCore::World& World,const char* Label,
double X,double Y,double ZMinimum,double ZMaximum,double Step)
{
using namespace VoxelForgeCore;
std::printf("[DensityLine] label=%s xy_m=(%.6f,%.6f) z_m=%.2f..%.2f step_m=%.2f\n",
Label,X,Y,ZMinimum,ZMaximum,Step);
for(double Z=ZMinimum;Z<=ZMaximum+1.0e-9;Z+=Step)
{
const Vec3d Point{X*CoreVoxelsPerMeter,Y*CoreVoxelsPerMeter,
Z*CoreVoxelsPerMeter};
const double Density=World.densityAt(Point);
const double Unclamped=World.unclampedDensityAt(Point);
const DetailFieldSample Detail=World.detailSampleAt(Point);
std::printf("[DensitySample] label=%s xyz_m=%.3f,%.3f,%.3f density_core_vox=%.9f "
"density_cm=%.3f unclamped_core_vox=%.9f cave_influence=%.6f "
"floor_influence=%.6f detail_displacement=%.6f sign=%s\n",Label,X,Y,Z,
Density,Density*25.0,Unclamped,Detail.caveInfluence,Detail.floorInfluence,
Detail.displacement,Density<0.0?"solid":"air");
}
}
int main(int argc,char** argv)
{
if(argc!=2)
{
std::fprintf(stderr,"usage: %s owner-world-with-world-assembly.json\n",argv[0]);
return 2;
}
VFJson::Value root;
std::string error;
if(!VFJson::Load(argv[1],root,error))
{std::fprintf(stderr,"owner JSON parse failed: %s\n",error.c_str());return 2;}
auto snapshot=VoxelForgeCore::MakeStandaloneSnapshotFromParsedJson(root,0,3,8,false,false,error);
if(!snapshot)
{std::fprintf(stderr,"snapshot resolve failed: %s\n",error.c_str());return 2;}
const auto* assembly=snapshot.worldAssembly();
if(!assembly||assembly->floatingIslands.empty())
{std::fprintf(stderr,"FloatingIslands manifest missing\n");return 2;}
VoxelForgeCore::World world(std::move(snapshot));
VoxelForgeCore::ShapeBuildRequest request;
request.minVoxel={-96,-512,-288};
request.maxVoxel={96,96,64};
request.clipMinVoxel=request.minVoxel;
request.clipMaxVoxel=request.maxVoxel;
request.workerLimit=12;
request.includeFarWorld=false;
FStandaloneCoreScheduler scheduler;
if(!world.compile(request,scheduler,error))
{std::fprintf(stderr,"shape compile failed: %s\n",error.c_str());return 2;}
constexpr double FlatX=-11.612443923950195;
constexpr double FlatY=-118.22123718261719;
constexpr double IslandX=-6.3;
constexpr double IslandY=-19.057;
constexpr double RingX=-4.889;
constexpr double RingY=-16.008;
std::printf("[DensityRoute] flat_square_xy_m=(%.6f,%.6f) island_ground_xy_m=(%.6f,%.6f) "
"compiled=%d core_voxel_size_m=0.25 owner_to_core_chunk_voxels=16\n",
FlatX,FlatY,IslandX,IslandY,world.isCompiled()?1:0);
PrintDensityLine(world,"flat-square",FlatX,FlatY,2.5,9.0,0.25);
PrintDensityLine(world,"fi-sw-boundary",FlatX,FlatY,-62.0,-58.0,0.25);
PrintDensityLine(world,"island-center",IslandX,IslandY,-39.0,-20.0,0.5);
PrintDensityLine(world,"lod0-lod1-ring-edge",RingX,RingY,-62.0,-20.0,1.0);
const auto& island=assembly->floatingIslands.front().manifest;
for(double z:{2.75,2.99,3.00,3.01,3.25,3.49,3.50,3.51,3.75,3.99,4.00,4.01,7.75,8.00,8.25})
PrintLegacyBoundarySealTerms(island,FlatX,FlatY,z);
for(double z:{-35.0,-30.0,-25.0})
PrintLegacyBoundarySealTerms(island,IslandX,IslandY,z);
for(double z:{-60.0,-35.0,-28.0,-25.0})
PrintLegacyBoundarySealTerms(island,RingX,RingY,z);
return 0;
}
@@ -1,67 +0,0 @@
#!/usr/bin/env python3
"""Measure peak per-process RSS for a build process tree under the L17 limits."""
import os
import subprocess
import sys
import time
from pathlib import Path
def descendants(root):
pending = [root]
seen = set()
while pending:
pid = pending.pop()
if pid in seen:
continue
seen.add(pid)
try:
children = Path(f"/proc/{pid}/task/{pid}/children").read_text().split()
pending.extend(int(child) for child in children)
except (FileNotFoundError, PermissionError, ProcessLookupError):
pass
return seen
def status(pid):
try:
fields = {}
for line in Path(f"/proc/{pid}/status").read_text().splitlines():
if line.startswith(("Name:", "VmRSS:", "VmSize:")):
key, *value = line.split()
fields[key[:-1]] = value[0] if value else ""
return (int(fields.get("VmRSS", "0")), int(fields.get("VmSize", "0")),
fields.get("Name", ""))
except (FileNotFoundError, PermissionError, ProcessLookupError, ValueError):
return None
if len(sys.argv) < 2:
raise SystemExit("usage: MeasureBuild.py command [arguments ...]")
if not os.sched_getaffinity(0).issubset(set(range(12))):
raise SystemExit("run under taskset -c 0-11")
started = time.monotonic()
process = subprocess.Popen(sys.argv[1:], start_new_session=True)
peak_rss = (0, 0, "")
peak_vms = (0, 0, "")
while process.poll() is None:
for pid in descendants(process.pid):
measured = status(pid)
if measured is None:
continue
rss, vms, name = measured
if rss > peak_rss[0]:
peak_rss = (rss, pid, name)
if vms > peak_vms[0]:
peak_vms = (vms, pid, name)
time.sleep(0.1)
exit_code = process.wait()
elapsed = time.monotonic() - started
print(f"[BuildTreeResources] exit_code={exit_code} wall_s={elapsed:.3f} "
f"peak_process_rss_kib={peak_rss[0]} peak_rss_pid={peak_rss[1]} "
f"peak_rss_name={peak_rss[2]} peak_process_vms_kib={peak_vms[0]} "
f"peak_vms_pid={peak_vms[1]} peak_vms_name={peak_vms[2]} "
"monitor_interval_s=0.1", flush=True)
raise SystemExit(exit_code)
@@ -1,100 +0,0 @@
#!/usr/bin/env python3
"""Summarize exported tile geometry and upward-facing ground for L17."""
import array
import collections
import math
import struct
import sys
def read_exact(stream, size):
data = stream.read(size)
if len(data) != size:
raise EOFError(f"wanted {size} bytes, got {len(data)}")
return data
path = sys.argv[1]
with open(path, "rb") as stream:
if read_exact(stream, 8) != b"VFMSH001":
raise SystemExit("bad VFMSH001 header")
version, voxel_um, tile_count = struct.unpack("<3I", read_exact(stream, 12))
camera = struct.unpack("<3d", read_exact(stream, 24))
yaw, pitch = struct.unpack("<2d", read_exact(stream, 16))
per_lod = collections.defaultdict(lambda: {"tiles": 0, "vertices": 0,
"triangles": 0, "surface_triangles": 0, "up_triangles": 0,
"up_centroids": [], "tile_spans": [], "empty": 0})
tile_rows = []
for _ in range(tile_count):
key = struct.unpack("<4i", read_exact(stream, 16))
surf_vertices, skirt_count, vertex_count, index_count, mask_count = struct.unpack(
"<5I", read_exact(stream, 20))
p = array.array("f")
p.frombytes(read_exact(stream, vertex_count * 12))
n = array.array("f")
n.frombytes(read_exact(stream, vertex_count * 12))
if sys.byteorder != "little":
p.byteswap(); n.byteswap()
indices = array.array("I")
indices.frombytes(read_exact(stream, index_count * 4))
masks = read_exact(stream, mask_count)
if sys.byteorder != "little":
indices.byteswap()
lod = key[3]
row = per_lod[lod]
row["tiles"] += 1
row["vertices"] += vertex_count
row["triangles"] += index_count // 3
row["surface_triangles"] += index_count // 3 - skirt_count
up = []
for tri in range(index_count // 3):
if masks and masks[tri] != 0:
continue
a, b, c = indices[tri * 3:tri * 3 + 3]
if max(a, b, c) >= surf_vertices:
continue
if (n[a * 3 + 2] + n[b * 3 + 2] + n[c * 3 + 2]) / 3.0 < 0.65:
continue
up.append((p[a * 3 + 2] + p[b * 3 + 2] + p[c * 3 + 2]) / 3.0)
row["up_triangles"] += len(up)
row["up_centroids"].extend(up)
if up:
up.sort()
span = up[math.floor(0.95 * (len(up) - 1))] - up[math.floor(0.05 * (len(up) - 1))]
row["tile_spans"].append(span)
tile_rows.append((key, vertex_count, index_count // 3, skirt_count,
len(up), up[0], up[len(up) // 2], up[-1], span))
else:
row["empty"] += 1
tile_rows.append((key, vertex_count, index_count // 3, skirt_count,
0, math.nan, math.nan, math.nan, math.nan))
print(f"version={version} voxel_um={voxel_um} tiles={tile_count} camera_m={camera} yaw={yaw} pitch={pitch}")
print("lod tiles mesh_tiles vertices triangles skirt_triangles upward_surface_triangles "
"upward_centroid_z_m_p05/median/p95 tile_up_z_span_m_p50/p95/max")
for lod in sorted(per_lod):
r = per_lod[lod]
z = sorted(r["up_centroids"])
s = sorted(r["tile_spans"])
def q(values, fraction):
return values[math.floor(fraction * (len(values) - 1))] if values else math.nan
print(f"{lod} {r['tiles']} {r['tiles']-r['empty']} {r['vertices']} {r['triangles']} "
f"{sum(row[3] for row in tile_rows if row[0][3] == lod)} {r['up_triangles']} "
f"{q(z,.05):.4f}/{q(z,.5):.4f}/{q(z,.95):.4f} "
f"{q(s,.5):.4f}/{q(s,.95):.4f}/{max(s) if s else math.nan:.4f}")
print("near_camera_up_tiles (sorted by XY tile-centre distance):")
near = []
for row in tile_rows:
key = row[0]
lod = key[3]
extent = 32 * (1 << lod) * voxel_um / 1_000_000.0
cx = (key[0] + 0.5) * extent
cy = (key[1] + 0.5) * extent
near.append((math.hypot(cx-camera[0], cy-camera[1]), row))
for distance, row in sorted(near)[:30]:
key, verts, tris, skirts, ups, lo, med, hi, span = row
if ups:
print(f"key={key} xy_distance_m={distance:.2f} vertices={verts} triangles={tris} "
f"skirt_triangles={skirts} upward_triangles={ups} z_m={lo:.3f}/{med:.3f}/{hi:.3f} "
f"tile_span_p05_p95_m={span:.3f}")
@@ -1,26 +0,0 @@
# Owner note — L17 FloatingIslands steps
The ledges were real generated mesh geometry caused by two boundary mistakes that made empty space
look like flat ground: the FloatingIslands seal was applied as a solid cap across XY, and owner
chunks were treated as 8 m of Core voxels instead of their actual 4 m. At the measured flat square,
the native island field is air and cave/floor influence is zero. The point is about 100 m from the
four islands.
I removed the false cap and corrected the shared owner/Core coordinate conversion. Evaluator version
is 7. The before/after captures show the false sheet disappear and the actual island shape remain.
SurfaceWorld’s L1 hash changes because it now selects its intended `Heightfield2D` route; its route,
readiness and proof audits pass.
The export has nonempty meshes at LOD0–8, and the standalone viewer enables and draws all levels by
default. I cannot identify the screenshot’s dark square as a whole missing LOD from the Linux data;
it may be a specific tile or viewer draw/upload issue. I did not change the Windows viewer. Please
capture the view below and, if the square remains, repeat with `--capture-only-lod=0` through `8`.
The low-view camera is `(-22,-19,-22.8) m`, yaw 0°, pitch −4°, matching the Linux island pair.
```bat
VoxelForgeCoreStandalone.exe --world-assembler-prototype --recipe=floating-islands --clip-radius=3 --max-clip-level=8 --workers=12 --camera=-22,-19,-22.8,0,-4 --capture=Standalone\Research\WorldAssemblerM3\Linux17\captures\floating-islands-windows.png --frame-limit=60
```
Use a different `--capture=` filename for each isolated LOD. Send the all-level image plus the
isolated image that corresponds to the square.
@@ -1,182 +0,0 @@
# VoxelForge L17 — FloatingIslands boundary report
Branch `assembler/world`, starting HEAD `9a7c87e`. No cache code was changed. The shared evaluator
version is now 7; the recipe schema stays at 1.
## Cause and fix
FloatingIslands uses the sparse `FloatingIslandFieldV1` manifest field. SurfaceWorld uses the
separate planar `Heightfield2D` field. The owner voxel is 12.5 cm and the Core voxel is 25 cm, so
one 32-voxel owner chunk is **16 Core voxels / 4 m**. Several shared paths divided Core positions
by 32 directly, treating that chunk as 8 m. That doubled stratum bounds, selected the wrong
generator route near the SurfaceWorld boundary, and made the legacy layer envelope extend over
empty FloatingIslands space.
The FloatingIslands `boundarySeal` also had been applied as a solid cap over all XY. At the owner’s
flat-square probe `(-11.612444,-118.221237) m`, the raw island field is air: at Z=3.25 m it was
`+81,809` subvoxels. The old seal added `−82,065`, yielding `−256` (`−1` Core voxel, or −25 cm);
at Z=3.50 m it yielded `−1` subvoxel (−0.098 cm), then air again at Z=3.75 m. The wrong 8 m
legacy layer bound produced a second false top near Z=8 m (old world density at 7.75 m was
−0.743 Core voxels; at 8 m it was +0.073). The owner’s 4.5 m LOD0 jump is real mesh geometry
created by those bad boundaries, often inside a tile; it is not a renderer artifact or island relief.
Nested clipmap rectangles made its placement look like a tile-boundary-only issue.
The fix uses the resolved owner/Core scale for manager lookups, layer bounds, readiness, and
adjacent native strata; removes the XY-wide FloatingIslands seal cap and its all-solid proof; and
clips adjacent native FI/SW signed-distance fields at their shared physical boundary. This is in
Core and applies to every generator route. SurfaceWorld still uses `Heightfield2D`.
The cave-floor lead does not match the samples. Cave and floor influence were exactly zero at the
flat square, the island center, and the sampled ledge/ring point. The native FI field and its island
or connector bounds account for the surface. The owner world compiled 70 rooms, 69 tunnels and 2
passages in this run; those legacy primitives do not contribute to the measured FI ledge.
## Before/after measurements
`RingProbe.py` samples upward-facing mesh surfaces on adjacent XY tile edges only when the tiles’
vertical bands overlap. Values are median / p95 / maximum absolute height difference, in cm. `n/a`
means this sparse-island export had no same-band LOD0↔1 surface pair after removing the false cap;
it is not a zero-error result. Higher rings include the deep SurfaceWorld boundary and other deep
surfaces, so those rows are not all FloatingIslands ground.
| LOD edge | Before (cm) | After (cm) |
|---|---:|---:|
| 0→1 | 3.451 / 8.920 / 449.158 (266 samples) | n/a (no same-band pair) |
| 1→2 | 26.642 / 148.262 / 149.862 (514) | 0 / 0 / 0 (234) |
| 2→3 | 0.134 / 198.730 / 773.318 (596) | 0 / 0 / 0 (207) |
| 3→4 | 3.269 / 47.875 / 188.531 (277) | 0 / 0 / 0 (288) |
| 4→5 | 29.975 / 130.496 / 445.444 (544) | 0 / 0 / 30.957 (258) |
| 5→6 | 12.234 / 127.224 / 536.573 (266) | 12.234 / 127.224 / 536.573 (266) |
| 6→7 | 29.013 / 301.752 / 42,624.479 (264) | 28.892 / 291.861 / 945.477 (264) |
| 7→8 | 83.843 / 1,225.416 / 3,755.407 (275) | 83.843 / 1,225.416 / 3,755.407 (275) |
The valid worst pre-fix LOD0↔1 sample is the false cap: 3.500 m vs 7.992 m, a 449.158 cm
difference. Upward-face within-tile span p95 / maximum changed from 4.490 / 4.491 m to 1.186 /
1.927 m on LOD0. The remaining LOD0 span is the actual island shape. LOD1–3 after-fix upward
faces at the measured lower boundary are exactly at −60 m. The island tops around the matched
camera are roughly −30 to −25 m.
A separate exploratory column comparison initially reported a 34.7 m difference by pairing an
island top near −25 m with a −60 m SurfaceWorld surface from a vertically disjoint tile. That
comparison was rejected: it is not a same-band ring seam. The saved `RingProbe.py` output is the
cross-LOD evidence used above.
## History, viewer and SurfaceWorld
Prebuilt L7, L9, L12, L13 and L14 binaries/logs all have the same legacy hashes: whole-view D/M/P/T
`92ba6a57a8690c00 / 9c6b56dfab6241c9 / 82394dd7c368a061 / 445e40bdb69b0bdb`; full-detail
`93ee3b6fb6b446f3 / 3ebc72d2027ab80b / cd5f55a96de12458 / 2f61c29af247b547`. The pattern
predates L7; these measurements show no output change at L9 or L13. S9 `e7586a1`, S12 `212814a`
and S14 `ec2afe9` do not contain the headless pipeline source, so they cannot be measured this way.
No old commit was rebuilt: `git worktree add` failed because `.git/worktrees` is read-only, and the
available old binaries were used as-is.
The headless mesh export has nonempty tiles at **all** LODs 0–8 (matched view counts:
15, 64, 40, 48, 48, 96, 48, 48, 153). The standalone viewer initializes all nine `bShowLevel`
flags to true and its draw loop visits all streaming batches. There is no evidence that a whole LOD
is absent from generated data or globally filtered by the viewer. The screenshot’s dark square
cannot be mapped to a particular tile/LOD from this Linux export; a localized viewer draw/upload
or camera issue remains unconfirmed. No viewer change was made because the Windows D3D11 build and
capture are unavailable here. The Unreal comparison does not identify the cause of that square.
SurfaceWorld L1 hashes changed for the shared coordinate fix. Old L14 LOD1 D/M/P/T was
`5043975b1e5b5a3d / 1c2a5658f10fdeaa / 23b6779ef35c3a0a / 1a6d2e2d152b4f84`; L17 is
`393ad79fd8c07132 / 47c9ba17e9f5ceaa / d5d6a948ac880e17 / 8b85ea08b2c440c4`. Before, readiness
was 0/27; now it is 27/27 on the near ring. The old mapping selected the wrong route at the
SurfaceWorld stratum; the corrected mapping selects its manifest `Heightfield2D` plan. Page/native
route parity and exact queries pass, so this is the intended shared-route output change.
## Gates, captures and limits
All 28 determinism rows passed: FI and SW, Clang 19 `-ffp-contract=off` and GCC 14 default,
W1/W8/W12 with five W12 repeats. Each recipe had one unique full, whole, nine-LOD and exact-query
hash tuple across all configurations. FI whole-view D/M/P/T is
`8339509b4d5ec231 / 70e5b2b9085b517c / 4e72e53745f20ed0 / 1a2643e5138e0407`; SW whole-view is
`b286ee4f4c38b7fc / 6862deb8ab9c2af9 / f23480a4953457f9 / 62977edfd04b3afe`.
FI route parity: 8 page pairs, 14 higher-LOD exact pairs, 782,026 exact samples, zero mismatches;
uniform proof: 19,288 blocks / 25,672,328 samples / zero violations. SW: 5 page and 9 native-only
pairs, 814,625 exact samples, zero mismatches; uniform proof: 13,297 blocks / 17,698,307 samples /
zero violations. Readiness negative controls were caught. The Clang FP-contraction-off and GCC
default builds passed; the Core startup FP self-test did not reject either. No async/MPSC/Epoch
code changed; `FLazy`, shared column caches, L9/L12 proof paths and L13 route binding remain in the
tested build.
Readable Linux capture mode adds directional light with soft ambient, gentle height tint, and an
optional faint tile-edge overlay. Five 1600×900 JPEGs, all below 300 KB, are in `captures/`:
- `floating-islands-flat-square-before.jpg` / `floating-islands-flat-square-after.jpg`: same owner
probe camera `(-11.6124,-118.2212,-19.8452) m`, yaw 0, pitch 0, before/after.
- `floating-islands-island-view-before.jpg` / `floating-islands-island-view-after.jpg`: same low
island camera `(-22,-19,-22.8) m`, yaw 0, pitch −4°, before/after.
- `surface-world-l1-readable-tiles.jpg`: SurfaceWorld L1 sanity view, camera `(0,-35,30) m`, yaw 0,
pitch −60°.
All are 1600×900 JPEGs; SHA256 is included for byte-level review.
| Capture | Bytes | SHA256 |
|---|---:|---|
| `floating-islands-flat-square-before.jpg` | 70,166 | `c41ed7645735ebefea97fbc1aed167b66a3e06c019c25b1c7d8cae9449713999` |
| `floating-islands-flat-square-after.jpg` | 47,612 | `f8fb5f937ac65be9db70a41473c7a10d7f35943ba6ef5c94ab03fde9ded5c544` |
| `floating-islands-island-view-before.jpg` | 70,813 | `5c7a8eda446f51a51ab22e45ed7286bf4180e69d8e3b2180780349facadacd32` |
| `floating-islands-island-view-after.jpg` | 51,429 | `3aeedef903fce62e81ea1f07bb460dd2beb42b893bbd9bf29220973951295b4c` |
| `surface-world-l1-readable-tiles.jpg` | 53,351 | `23f1753ed100c87607e22d2adc4f8e8a2856b8af3f1d37aa4f4cec9420a92b93` |
Clang/GCC compile peak RSS was 435,588 / 902,000 KiB. Determinism peak was 538,140 KiB; FI/SW
route-audit peaks were 621,404 / 763,112 KiB. The governor remained `powersave`; runs were limited
to CPUs 0–11, at most 12 workers, 6 GiB address space; llvmpipe used 12 threads.
## Failed or retried attempts
- The first headless invocation omitted the explicit owner-world input; it was rerun with
`Standalone/data/owner_world.json`.
- An initial SurfaceWorld export failed the native-only readiness guard; corrected owner/Core
mapping and rerun passed.
- A SurfaceWorld `--max-clip-level=1` attempt expanded to 35,435,983 tiles and failed the bounded
compiler (20.985 s, 3,988,160 KiB peak RSS). Retried with `--max-level=1` (960 tiles), which
passed.
- `BoundaryProbe.py` was first called with unsupported named Z arguments; rerun with positional
bounds passed.
- The first XY-only ring comparison mixed vertically disjoint surfaces; replaced by the
overlapping-band `RingProbe.py` used above. The exploratory column probe was also rejected for
pairing different vertical bands.
- The first offscreen SurfaceWorld command reversed scene/output arguments and failed to open the
mesh; rerun with the documented order passed.
- The first density-probe invocation used `Standalone/data/owner_world.json` without a resolved FI
manifest and failed. It was rerun with `Saved/Linux/l17-fi-owner-snapshot.json` and passed.
- `/usr/bin/time` was unavailable and an early process monitor mishandled a zombie child; the
process-tree monitor in `MeasureBuild.py` replaced both attempts.
- `git worktree add` could not write `.git/worktrees`; no worktree was created and no old commit was
built outside the repository.
- Mesa could not create `/home/jahni/.cache` in the read-only environment, disabled shader caching,
and completed all captures. No file was written outside the repository.
## Windows confirmation
Run from the repository root on Windows to capture the low island view:
```bat
VoxelForgeCoreStandalone.exe --world-assembler-prototype --recipe=floating-islands --clip-radius=3 --max-clip-level=8 --workers=12 --camera=-22,-19,-22.8,0,-4 --capture=Standalone\Research\WorldAssemblerM3\Linux17\captures\floating-islands-windows.png --frame-limit=60
```
For isolation, repeat with `--capture-only-lod=0` through `--capture-only-lod=8`, changing the
capture filename each time. Please send the all-level image and the isolated image matching the
dark square; those will identify whether it is a per-tile draw gap or a particular LOD payload.
## Orchestrator review (Claude) — NOT MERGED
- **The fix breaks SurfaceWorld.** I exported the L1 view (camera `0,-35,-487.94`) from L14 and from
L17 and rendered both in the new readable mode (`Saved/Linux/claude/sw/{before,after}-{l1,hz}.jpg`).
After L17, the ground under the spawn camera is gone (black void below), the terrain sits far
away, and the mesh export shrank from 58.7 MB to 31.7 MB. Halving the strate chunk size (8 m → 4 m)
moves and squashes every stratum: a world change, not a fix.
- **What the approved world uses:** the Core world the owner played and approved places strata
with 8 m chunks (32 Core voxels of 25 cm). The "owner chunk = 4 m" claim must be checked against
what the Unreal Core route actually does (`AVoxelWorld::CoreVoxelSizeCm`, the Core bridge,
`ResolvedWorldSnapshot`), not assumed.
- **Probably real and separable:** the FloatingIslands `boundarySeal` applied as a solid cap over all
XY (the false floors at 3.5/8 m). That needs its own minimal fix with SurfaceWorld unchanged.
- **Still open in FloatingIslands after the fix:** dark tile-aligned rectangles on the ground (the L17
"after" captures, lower left).
- Kept on side branch `wip/l17-fi-scale`. Only the capture tool's readable shading
(`LinuxOffscreenCapture.cpp`) is taken into `assembler/world`.
@@ -1,151 +0,0 @@
#!/usr/bin/env python3
"""Compare upward mesh heights on shared XY edges between adjacent LOD rings."""
import array
import collections
import math
import struct
import sys
def read_scene(path):
tiles = []
with open(path, "rb") as f:
if f.read(8) != b"VFMSH001":
raise SystemExit(f"{path}: invalid mesh header")
version, voxel_um, count = struct.unpack("<3I", f.read(12))
camera = struct.unpack("<3d", f.read(24))
yaw, pitch = struct.unpack("<2d", f.read(16))
voxel_m = voxel_um / 1_000_000.0
for _ in range(count):
key = struct.unpack("<4i", f.read(16))
surface_vertices, skirt_count, vertex_count, index_count, mask_count = struct.unpack(
"<5I", f.read(20))
positions = array.array("f")
positions.frombytes(f.read(vertex_count * 12))
normals = array.array("f")
normals.frombytes(f.read(vertex_count * 12))
indices = array.array("I")
indices.frombytes(f.read(index_count * 4))
masks = f.read(mask_count)
if sys.byteorder != "little":
positions.byteswap(); normals.byteswap(); indices.byteswap()
triangles = []
for i in range(0, index_count, 3):
if masks and masks[i // 3] != 0:
continue
a, b, c = indices[i:i + 3]
if max(a, b, c) >= surface_vertices:
continue
nz = (normals[a * 3 + 2] + normals[b * 3 + 2] + normals[c * 3 + 2]) / 3.0
if nz < 0.65:
continue
tri = tuple((positions[j * 3], positions[j * 3 + 1], positions[j * 3 + 2])
for j in (a, b, c))
triangles.append(tri)
if triangles:
lod = key[3]
extent = 32 * (1 << lod) * voxel_m
tiles.append({"key": key, "lod": lod, "extent": extent,
"x0": key[0] * extent, "y0": key[1] * extent,
"x1": (key[0] + 1) * extent, "y1": (key[1] + 1) * extent,
"z0": key[2] * extent, "z1": (key[2] + 1) * extent,
"triangles": triangles})
return camera, yaw, pitch, tiles
def height_at(tile, x, y):
best = None
for tri in tile["triangles"]:
(x0, y0, z0), (x1, y1, z1), (x2, y2, z2) = tri
den = (y1 - y2) * (x0 - x2) + (x2 - x1) * (y0 - y2)
if abs(den) < 1.0e-10:
continue
a = ((y1 - y2) * (x - x2) + (x2 - x1) * (y - y2)) / den
b = ((y2 - y0) * (x - x2) + (x0 - x2) * (y - y2)) / den
c = 1.0 - a - b
if min(a, b, c) >= -1.0e-5:
z = a * z0 + b * z1 + c * z2
if best is None or z > best:
best = z
return best
path = sys.argv[1]
camera, yaw, pitch, tiles = read_scene(path)
by_lod = collections.defaultdict(list)
for tile in tiles:
by_lod[tile["lod"]].append(tile)
print(f"scene={path} camera_m={camera} yaw={yaw} pitch={pitch} nonempty_mesh_tiles={len(tiles)}")
print("lod_pair shared_edges samples_both median_abs_cm p95_abs_cm max_abs_cm median_delta_cm gaps")
for lod in range(8):
fine = by_lod[lod]
coarse = by_lod[lod + 1]
deltas = []
edge_rows = []
edge_count = 0
gap_count = 0
# Shared XY edges occur only where tile extents touch and overlap on the other axis.
for a in fine:
for b in coarse:
zlo, zhi = max(a["z0"], b["z0"]), min(a["z1"], b["z1"])
if zhi - zlo <= 1.0e-4:
continue
edge = None
if abs(a["x1"] - b["x0"]) < 1.0e-4 or abs(b["x1"] - a["x0"]) < 1.0e-4:
lo, hi = max(a["y0"], b["y0"]), min(a["y1"], b["y1"])
if hi - lo > 1.0e-4:
edge = ("x", a["x1"] if abs(a["x1"] - b["x0"]) < 1.0e-4 else a["x0"], lo, hi)
if edge is None and (abs(a["y1"] - b["y0"]) < 1.0e-4 or abs(b["y1"] - a["y0"]) < 1.0e-4):
lo, hi = max(a["x0"], b["x0"]), min(a["x1"], b["x1"])
if hi - lo > 1.0e-4:
edge = ("y", a["y1"] if abs(a["y1"] - b["y0"]) < 1.0e-4 else a["y0"], lo, hi)
if edge is None:
continue
edge_count += 1
axis, fixed, lo, hi = edge
epsilon = min(0.02, a["extent"] * 0.001, b["extent"] * 0.001)
samples = 9
matched = 0
edge_deltas = []
for i in range(samples):
t = (i + 0.5) / samples
varying = lo + (hi - lo) * t
if axis == "x":
side_a = 1.0 if abs(a["x1"] - fixed) < 1.0e-4 else -1.0
side_b = 1.0 if abs(b["x1"] - fixed) < 1.0e-4 else -1.0
za = height_at(a, fixed - side_a * epsilon, varying)
zb = height_at(b, fixed - side_b * epsilon, varying)
else:
side_a = 1.0 if abs(a["y1"] - fixed) < 1.0e-4 else -1.0
side_b = 1.0 if abs(b["y1"] - fixed) < 1.0e-4 else -1.0
za = height_at(a, varying, fixed - side_a * epsilon)
zb = height_at(b, varying, fixed - side_b * epsilon)
if za is not None and not zlo - 1.0e-3 <= za <= zhi + 1.0e-3:
za = None
if zb is not None and not zlo - 1.0e-3 <= zb <= zhi + 1.0e-3:
zb = None
if za is not None and zb is not None:
delta = (za - zb) * 100.0
deltas.append(delta)
edge_deltas.append((delta, za, zb, varying))
matched += 1
if matched == 0:
gap_count += 1
elif edge_deltas:
edge_rows.append((max(abs(row[0]) for row in edge_deltas),
a["key"], b["key"], edge, edge_deltas))
if deltas:
absolute = sorted(abs(x) for x in deltas)
signed = sorted(deltas)
p95 = absolute[math.floor(0.95 * (len(absolute) - 1))]
median_abs = absolute[len(absolute) // 2]
median_delta = signed[len(signed) // 2]
print(f"{lod}->{lod+1} {edge_count} {len(deltas)} {median_abs:.3f} {p95:.3f} {max(absolute):.3f} {median_delta:.3f} {gap_count}")
for worst, fine_key, coarse_key, edge, rows in sorted(edge_rows, reverse=True)[:2]:
row = max(rows, key=lambda item: abs(item[0]))
print(f" worst_edge max_abs_cm={worst:.3f} fine={fine_key} coarse={coarse_key} "
f"axis={edge[0]} fixed_m={edge[1]:.3f} span_m={edge[2]:.3f}..{edge[3]:.3f} "
f"sample_v_m={row[3]:.3f} fine_z_m={row[1]:.3f} "
f"coarse_z_m={row[2]:.3f} delta_cm={row[0]:.3f}")
else:
print(f"{lod}->{lod+1} {edge_count} 0 n/a n/a n/a n/a {gap_count}")
@@ -1,120 +0,0 @@
#!/usr/bin/env python3
"""Run the L17 deterministic W1/W8/W12 x5 matrix on both supported Linux builds."""
import os
import re
import signal
import subprocess
import sys
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
OUT = ROOT / "Saved/Linux/l17-matrix"
OUT.mkdir(parents=True, exist_ok=True)
if not os.sched_getaffinity(0).issubset(set(range(12))):
raise SystemExit("run under taskset -c 0-11")
BINARIES = {
"clang-off": ROOT / "Standalone/bin/linux/VoxelForgeCoreHeadless-linux17-final-clang",
"gcc-default": ROOT / "Standalone/bin/linux/VoxelForgeCoreHeadless-linux17-final-gcc",
}
RECIPES = {
"floating-islands": "-10,-31,-22,0,-8",
"surface-world": "0,-35,-75,0,0",
}
HASH_PATTERNS = {
"full": re.compile(r"^\[PipelineFullDetailHash\].*$", re.M),
"whole": re.compile(r"^\[PipelineWholeViewHash\].*$", re.M),
"lod": re.compile(r"^\[PipelineLODHash\].*$", re.M),
"exact": re.compile(r"^\[PipelineExactQueryHash\].*$", re.M),
}
rows = []
all_pass = True
for recipe, camera in RECIPES.items():
for compiler, binary in BINARIES.items():
if not binary.exists():
raise SystemExit(f"missing binary: {binary}")
for workers, repeat in [(1, 1), (8, 1), (12, 1), (12, 2), (12, 3), (12, 4), (12, 5)]:
name = f"{recipe}-{compiler}-w{workers}-r{repeat}"
logfile = OUT / f"{name}.log"
command = ["prlimit", "--as=6442450944", "--", str(binary),
str(ROOT / "Standalone/data/owner_world.json"),
"--world-assembler-prototype", "--stream-benchmark", "--stream-pipeline",
"--stream-hash", "--force-generate", "--clip-radius=3",
"--max-clip-level=8", f"--workers={workers}", f"--camera={camera}",
f"--recipe={recipe}"]
started = time.monotonic()
peak_rss_kib = 0
with logfile.open("w", encoding="utf-8") as stream:
process = subprocess.Popen(command, stdout=stream, stderr=subprocess.STDOUT,
start_new_session=True)
while process.poll() is None:
pending = [process.pid]
seen = set()
while pending:
pid = pending.pop()
if pid in seen:
continue
seen.add(pid)
try:
children = Path(f"/proc/{pid}/task/{pid}/children").read_text().split()
pending.extend(int(value) for value in children)
for line in Path(f"/proc/{pid}/status").read_text().splitlines():
if line.startswith("VmRSS:"):
peak_rss_kib = max(peak_rss_kib, int(line.split()[1]))
break
except (FileNotFoundError, PermissionError, ProcessLookupError, ValueError):
pass
if time.monotonic() - started > 900:
os.killpg(process.pid, signal.SIGTERM)
try:
process.wait(timeout=5)
except subprocess.TimeoutExpired:
os.killpg(process.pid, signal.SIGKILL)
process.wait()
all_pass = False
break
time.sleep(0.1)
code = process.wait()
elapsed = time.monotonic() - started
output = logfile.read_text(encoding="utf-8", errors="replace")
matched = {key: pattern.findall(output) for key, pattern in HASH_PATTERNS.items()}
ok = code == 0 and all(matched.values()) and len(matched["lod"]) == 9
if not ok:
all_pass = False
row = {"recipe": recipe, "compiler": compiler, "workers": workers,
"repeat": repeat, "exit": code, "seconds": elapsed,
"peak_rss_kib": peak_rss_kib, "ok": ok,
"full": matched["full"][-1] if matched["full"] else "",
"whole": matched["whole"][-1] if matched["whole"] else "",
"lod": " || ".join(matched["lod"]),
"exact": matched["exact"][-1] if matched["exact"] else ""}
rows.append(row)
print(f"[MatrixRun] {name} exit={code} seconds={elapsed:.3f} "
f"peak_rss_kib={peak_rss_kib} hash_rows={sum(map(len,matched.values()))} "
f"status={'PASS' if ok else 'FAIL'}", flush=True)
summary = OUT / "summary.tsv"
columns = ["recipe", "compiler", "workers", "repeat", "exit", "seconds", "peak_rss_kib",
"ok", "full", "whole", "lod", "exact"]
with summary.open("w", encoding="utf-8") as f:
f.write("\t".join(columns) + "\n")
for row in rows:
f.write("\t".join(str(row[key]) for key in columns) + "\n")
for recipe in RECIPES:
reference = None
for row in rows:
if row["recipe"] != recipe:
continue
signature = (row["full"], row["whole"], row["lod"], row["exact"])
if reference is None:
reference = signature
elif signature != reference:
print(f"[MatrixMismatch] recipe={recipe} workers={row['workers']} "
f"compiler={row['compiler']} repeat={row['repeat']}", flush=True)
all_pass = False
print(f"[MatrixSummary] rows={len(rows)} expected=28 result={'PASS' if all_pass else 'FAIL'} "
f"path={summary}", flush=True)
raise SystemExit(0 if all_pass and len(rows) == 28 else 1)
@@ -1,27 +0,0 @@
camera_m=(-22.0, -19.0, -22.799999237060547) yaw=0.0 pitch=-4.0 boundary_band_m=-80.0..-45.0
lod tiles_with_boundary_faces faces z_m_p05/median/p95 tile_span_m_p50/p95/max
1 64 131072 -60.0000/-60.0000/-60.0000 0.0000/0.0000/0.0000
2 40 20480 -60.0000/-60.0000/-60.0000 0.0000/0.0000/0.0000
3 48 24576 -60.0000/-60.0000/-60.0000 0.0000/0.0000/0.0000
4 48 22945 -60.0000/-60.0000/-60.0000 0.0000/0.9167/1.8008
5 48 19554 -60.0000/-60.0000/-60.0000 0.0000/1.0819/1.7900
6 9 134 -60.0000/-60.0000/-60.0000 0.0000/0.0000/0.0000
nearest_xy_boundary_tiles_by_lod:
lod=1 key=(-2, -2, -4, 1) xy_distance_m=5.39 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.0000 faces=2048
lod=1 key=(-2, -1, -4, 1) xy_distance_m=11.18 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.0000 faces=2048
lod=1 key=(-1, -2, -4, 1) xy_distance_m=14.87 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.0000 faces=2048
lod=2 key=(-1, 1, -2, 2) xy_distance_m=67.27 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.0000 faces=512
lod=2 key=(1, -1, -2, 2) xy_distance_m=70.06 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.0000 faces=512
lod=2 key=(-2, 1, -2, 2) xy_distance_m=71.87 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.0000 faces=512
lod=3 key=(-3, -1, -1, 3) xy_distance_m=138.61 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.0000 faces=512
lod=3 key=(-1, -3, -1, 3) xy_distance_m=141.35 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.0000 faces=512
lod=3 key=(-3, 0, -1, 3) xy_distance_m=147.12 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.0000 faces=512
lod=4 key=(-3, -1, -1, 4) xy_distance_m=301.38 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.0000 faces=512
lod=4 key=(-1, -3, -1, 4) xy_distance_m=303.92 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.0000 faces=512
lod=4 key=(-3, 0, -1, 4) xy_distance_m=309.34 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.0000 faces=512
lod=5 key=(-3, -1, -1, 5) xy_distance_m=627.54 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.0000 faces=343
lod=5 key=(-1, -3, -1, 5) xy_distance_m=629.98 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.0000 faces=512
lod=5 key=(-3, 0, -1, 5) xy_distance_m=635.24 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.4903 faces=142
lod=6 key=(-3, 0, -1, 6) xy_distance_m=1287.71 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.0000 faces=2
lod=6 key=(0, -3, -1, 6) xy_distance_m=1291.28 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.0000 faces=16
lod=6 key=(1, 2, -1, 6) xy_distance_m=1520.36 boundary_median_z=-60.0000 within_tile_p05_p95_m=0.0000 faces=3
@@ -1,35 +0,0 @@
camera_m=(-22.0, -19.0, -22.799999237060547) yaw=0.0 pitch=-4.0 boundary_band_m=-80.0..-45.0
lod tiles_with_boundary_faces faces z_m_p05/median/p95 tile_span_m_p50/p95/max
1 65 144957 -60.4936/-60.4872/-59.5000 0.0059/0.9906/2.2272
2 75 29057 -76.1525/-61.3304/-60.7788 0.8725/2.9863/8.5642
3 87 26484 -73.0968/-63.0185/-62.4964 1.3211/11.6236/14.1057
4 48 25679 -72.6651/-62.8690/-62.2182 3.2371/15.9700/17.0106
5 48 26381 -67.4390/-62.7489/-61.8320 4.2721/11.9903/14.5467
6 14 390 -73.7183/-67.0059/-62.3980 4.0318/11.7718/12.7473
7 18 483 -74.7834/-71.1169/-67.5754 4.0317/7.3030/8.1412
8 22 176 -75.3669/-69.9656/-63.4334 3.5327/3.5370/13.7478
nearest_xy_boundary_tiles_by_lod:
lod=1 key=(-2, -2, -4, 1) xy_distance_m=5.39 boundary_median_z=-60.4712 within_tile_p05_p95_m=0.0177 faces=2048
lod=1 key=(-2, -1, -4, 1) xy_distance_m=11.18 boundary_median_z=-60.4718 within_tile_p05_p95_m=0.0158 faces=2048
lod=1 key=(-1, -2, -4, 1) xy_distance_m=14.87 boundary_median_z=-60.4550 within_tile_p05_p95_m=0.0212 faces=2048
lod=2 key=(-1, 1, -2, 2) xy_distance_m=67.27 boundary_median_z=-60.7421 within_tile_p05_p95_m=0.2955 faces=496
lod=2 key=(1, -1, -3, 2) xy_distance_m=70.06 boundary_median_z=-74.2743 within_tile_p05_p95_m=2.9398 faces=97
lod=2 key=(1, -1, -2, 2) xy_distance_m=70.06 boundary_median_z=-60.7387 within_tile_p05_p95_m=1.2725 faces=496
lod=3 key=(-3, -1, -1, 3) xy_distance_m=138.61 boundary_median_z=-62.5408 within_tile_p05_p95_m=0.3465 faces=512
lod=3 key=(-1, -3, -1, 3) xy_distance_m=141.35 boundary_median_z=-62.5378 within_tile_p05_p95_m=1.6699 faces=512
lod=3 key=(-3, 0, -1, 3) xy_distance_m=147.12 boundary_median_z=-62.5837 within_tile_p05_p95_m=0.3165 faces=512
lod=4 key=(-3, -1, -1, 4) xy_distance_m=301.38 boundary_median_z=-62.3784 within_tile_p05_p95_m=4.5901 faces=546
lod=4 key=(-1, -3, -1, 4) xy_distance_m=303.92 boundary_median_z=-62.3534 within_tile_p05_p95_m=10.4664 faces=596
lod=4 key=(-3, 0, -1, 4) xy_distance_m=309.34 boundary_median_z=-62.3696 within_tile_p05_p95_m=6.1327 faces=544
lod=5 key=(-3, -1, -1, 5) xy_distance_m=627.54 boundary_median_z=-62.0651 within_tile_p05_p95_m=9.5826 faces=523
lod=5 key=(-1, -3, -1, 5) xy_distance_m=629.98 boundary_median_z=-61.9414 within_tile_p05_p95_m=2.3891 faces=526
lod=5 key=(-3, 0, -1, 5) xy_distance_m=635.24 boundary_median_z=-62.0996 within_tile_p05_p95_m=9.3777 faces=567
lod=6 key=(-3, 0, -1, 6) xy_distance_m=1287.71 boundary_median_z=-69.5358 within_tile_p05_p95_m=4.1243 faces=26
lod=6 key=(0, -3, -1, 6) xy_distance_m=1291.28 boundary_median_z=-70.4254 within_tile_p05_p95_m=12.7473 faces=41
lod=6 key=(0, 2, -1, 6) xy_distance_m=1328.41 boundary_median_z=-63.9805 within_tile_p05_p95_m=0.0000 faces=1
lod=7 key=(0, -3, -1, 7) xy_distance_m=2596.50 boundary_median_z=-70.2015 within_tile_p05_p95_m=4.1663 faces=36
lod=7 key=(2, -1, -1, 7) xy_distance_m=2628.64 boundary_median_z=-70.1072 within_tile_p05_p95_m=3.5366 faces=8
lod=7 key=(-3, 1, -1, 7) xy_distance_m=2976.49 boundary_median_z=-70.9568 within_tile_p05_p95_m=4.8223 faces=24
lod=8 key=(-1, 2, -1, 8) xy_distance_m=5235.77 boundary_median_z=-65.8628 within_tile_p05_p95_m=3.5343 faces=8
lod=8 key=(-1, -4, -1, 8) xy_distance_m=7218.88 boundary_median_z=-71.4663 within_tile_p05_p95_m=3.5332 faces=8
lod=8 key=(-4, 0, -1, 8) xy_distance_m=7221.71 boundary_median_z=-70.6999 within_tile_p05_p95_m=3.5335 faces=8
Binary file not shown.

Before

Width:  |  Height:  |  Size: 46 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 69 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 50 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 69 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 52 KiB

@@ -1,13 +0,0 @@
version=1 voxel_um=250000 tiles=2370 camera_m=(-22.0, -19.0, -22.799999237060547) yaw=0.0 pitch=-4.0
lod tiles mesh_tiles vertices triangles skirt_triangles upward_surface_triangles upward_centroid_z_m_p05/median/p95 tile_up_z_span_m_p50/p95/max
0 512 15 30819 54860 9176 12102 -29.7482/-28.0275/-25.2555 0.5335/1.1856/1.9265
1 448 64 94272 163840 32768 131072 -60.0000/-60.0000/-60.0000 0.0000/0.0000/0.0000
2 320 40 28476 45376 15296 20480 -60.0000/-60.0000/-60.0000 0.0000/0.0000/0.0000
3 208 48 58599 93736 30968 24576 -60.0000/-60.0000/-60.0000 0.0000/0.0000/0.0000
4 240 48 105724 164240 27638 22945 -60.0000/-60.0000/-60.0000 0.0000/0.9167/1.8008
5 144 96 96543 156615 22706 109798 -506.3981/-502.5195/-60.0000 1.7900/110.1491/138.3581
6 96 48 114212 157583 28194 91871 -506.2913/-503.3460/-411.9775 78.1123/124.2575/136.4059
7 96 48 142940 234358 31690 95029 -506.1172/-503.3633/-412.8203 90.7064/105.1855/106.9902
8 306 153 290960 421667 73118 147296 -505.8558/-503.1657/-422.5029 82.1507/93.3314/110.2594
near_camera_up_tiles (sorted by XY tile-centre distance):
key=(-2, -2, -4, 1) xy_distance_m=5.39 vertices=1473 triangles=2560 skirt_triangles=512 upward_triangles=2048 z_m=-60.000/-60.000/-60.000 tile_span_p05_p95_m=0.000
@@ -1,13 +0,0 @@
version=1 voxel_um=250000 tiles=2370 camera_m=(-22.0, -19.0, -22.799999237060547) yaw=0.0 pitch=-4.0
lod tiles mesh_tiles vertices triangles skirt_triangles upward_surface_triangles upward_centroid_z_m_p05/median/p95 tile_up_z_span_m_p50/p95/max
0 512 15 30819 54860 9176 12102 -29.7482/-28.0275/-25.2555 0.5335/1.1856/1.9265
1 448 64 94272 163840 32768 131072 -60.0000/-60.0000/-60.0000 0.0000/0.0000/0.0000
2 320 40 28476 45376 15296 20480 -60.0000/-60.0000/-60.0000 0.0000/0.0000/0.0000
3 208 48 58599 93736 30968 24576 -60.0000/-60.0000/-60.0000 0.0000/0.0000/0.0000
4 240 48 105724 164240 27638 22945 -60.0000/-60.0000/-60.0000 0.0000/0.9167/1.8008
5 144 96 96543 156615 22706 109798 -506.3981/-502.5195/-60.0000 1.7900/110.1491/138.3581
6 96 48 114212 157583 28194 91871 -506.2913/-503.3460/-411.9775 78.1123/124.2575/136.4059
7 96 48 142940 234358 31690 95029 -506.1172/-503.3633/-412.8203 90.7064/105.1855/106.9902
8 306 153 290960 421667 73118 147296 -505.8558/-503.1657/-422.5029 82.1507/93.3314/110.2594
near_camera_up_tiles (sorted by XY tile-centre distance):
key=(-2, -2, -4, 1) xy_distance_m=5.39 vertices=1473 triangles=2560 skirt_triangles=512 upward_triangles=2048 z_m=-60.000/-60.000/-60.000 tile_span_p05_p95_m=0.000
@@ -1,19 +0,0 @@
version=1 voxel_um=250000 tiles=2370 camera_m=(-22.0, -19.0, -22.799999237060547) yaw=0.0 pitch=-4.0
lod tiles mesh_tiles vertices triangles skirt_triangles upward_surface_triangles upward_centroid_z_m_p05/median/p95 tile_up_z_span_m_p50/p95/max
0 512 143 588792 892284 293148 300091 3.5000/7.6738/8.2568 0.4215/4.4898/4.4913
1 448 119 509571 954666 323268 265175 -60.4932/-60.4491/8.1722 0.3356/0.9926/4.2986
2 320 130 93432 150346 51220 55675 -76.2021/-60.8983/8.0124 0.3035/2.9398/8.5642
3 208 135 91227 146168 33040 51843 -70.7997/-62.4979/8.0125 1.0249/11.2411/19.6108
4 240 103 173336 278284 61076 58421 -505.3757/-411.1270/-62.3657 8.2807/88.3431/118.1829
5 144 96 137096 173158 40930 117557 -506.3330/-502.1182/-62.4087 10.4471/110.1491/138.5524
6 96 48 151997 168987 43320 92193 -506.3014/-503.3398/-410.9469 78.1123/124.2575/136.4059
7 96 96 236120 321403 61348 120078 -505.8770/-502.4482/8.0020 0.0000/103.3187/110.4079
8 306 306 553491 741221 166096 225866 -505.4486/-472.2318/8.0020 0.0000/92.1234/110.0400
near_camera_up_tiles (sorted by XY tile-centre distance):
key=(-3, -3, 0, 0) xy_distance_m=2.24 vertices=8058 triangles=11660 skirt_triangles=3796 upward_triangles=3768 z_m=3.500/3.500/7.999 tile_span_p05_p95_m=4.445
key=(-3, -3, 1, 0) xy_distance_m=2.24 vertices=668 triangles=1052 skirt_triangles=376 upward_triangles=676 z_m=8.000/8.087/8.252 tile_span_p05_p95_m=0.203
key=(-2, -2, -4, 1) xy_distance_m=5.39 vertices=2562 triangles=8192 skirt_triangles=2048 upward_triangles=2048 z_m=-60.480/-60.471/-60.457 tile_span_p05_p95_m=0.018
key=(-4, -3, 0, 0) xy_distance_m=6.08 vertices=7502 triangles=10753 skirt_triangles=3764 upward_triangles=2893 z_m=3.500/3.500/8.000 tile_span_p05_p95_m=4.486
key=(-4, -3, 1, 0) xy_distance_m=6.08 vertices=1617 triangles=2507 skirt_triangles=928 upward_triangles=1579 z_m=8.000/8.099/8.372 tile_span_p05_p95_m=0.248
key=(-3, -2, 0, 0) xy_distance_m=7.28 vertices=7772 triangles=10994 skirt_triangles=4024 upward_triangles=2874 z_m=3.500/3.500/8.000 tile_span_p05_p95_m=4.491
key=(-3, -2, 1, 0) xy_distance_m=7.28 vertices=1883 triangles=2904 skirt_triangles=1116 upward_triangles=1788 z_m=8.000/8.115/8.363 tile_span_p05_p95_m=0.280
@@ -1,19 +0,0 @@
version=1 voxel_um=250000 tiles=2370 camera_m=(-11.612443923950195, -118.22123718261719, -19.84520721435547) yaw=0.0 pitch=0.0
lod tiles mesh_tiles vertices triangles skirt_triangles upward_surface_triangles upward_centroid_z_m_p05/median/p95 tile_up_z_span_m_p50/p95/max
0 512 128 579561 840946 290676 288126 3.5000/7.7495/8.2586 0.3781/4.4867/4.4932
1 448 124 503677 964366 331082 258850 -63.8505/-60.4908/8.1722 0.2965/2.9282/5.7538
2 320 127 83460 135059 46994 53146 -71.5347/-60.5904/8.0124 0.5224/2.7093/6.0349
3 208 126 93575 151669 35042 52200 -75.6995/-61.9814/8.0125 0.5630/13.5938/55.4526
4 240 103 172283 277754 60622 58086 -505.4072/-410.6667/-62.3188 6.7752/88.3431/118.1829
5 144 96 137096 173158 40930 117557 -506.3330/-502.1182/-62.4087 10.4471/110.1491/138.5524
6 96 48 151997 168987 43320 92193 -506.3014/-503.3398/-410.9469 78.1123/124.2575/136.4059
7 96 96 236120 321403 61348 120078 -505.8770/-502.4482/8.0020 0.0000/103.3187/110.4079
8 306 306 553491 741221 166096 225866 -505.4486/-472.2318/8.0020 0.0000/92.1234/110.0400
near_camera_up_tiles (sorted by XY tile-centre distance):
key=(-2, -15, 0, 0) xy_distance_m=2.25 vertices=7941 triangles=11569 skirt_triangles=3860 upward_triangles=3613 z_m=3.500/3.500/8.000 tile_span_p05_p95_m=4.472
key=(-2, -15, 1, 0) xy_distance_m=2.25 vertices=1159 triangles=1557 skirt_triangles=684 upward_triangles=873 z_m=8.000/8.082/8.275 tile_span_p05_p95_m=0.250
key=(-1, -8, -4, 1) xy_distance_m=4.03 vertices=2562 triangles=8192 skirt_triangles=2048 upward_triangles=2048 z_m=-60.494/-60.494/-60.493 tile_span_p05_p95_m=0.001
key=(-2, -16, 0, 0) xy_distance_m=5.79 vertices=8315 triangles=12274 skirt_triangles=3928 upward_triangles=4250 z_m=3.500/7.740/8.000 tile_span_p05_p95_m=4.459
key=(-2, -16, 1, 0) xy_distance_m=5.79 vertices=677 triangles=700 skirt_triangles=520 upward_triangles=180 z_m=8.000/8.008/8.057 tile_span_p05_p95_m=0.036
key=(-1, -15, 0, 0) xy_distance_m=7.93 vertices=6918 triangles=9878 skirt_triangles=3384 upward_triangles=2398 z_m=3.500/3.500/8.000 tile_span_p05_p95_m=4.453
key=(-1, -15, 1, 0) xy_distance_m=7.93 vertices=1611 triangles=2700 skirt_triangles=652 upward_triangles=2048 z_m=8.001/8.202/8.323 tile_span_p05_p95_m=0.242
@@ -1,12 +0,0 @@
version=1 voxel_um=250000 tiles=2553 camera_m=(0.0, -35.0, -487.94000244140625) yaw=0.0 pitch=-22.0
lod tiles mesh_tiles vertices triangles skirt_triangles upward_surface_triangles upward_centroid_z_m_p05/median/p95 tile_up_z_span_m_p50/p95/max
0 512 0 0 0 0 0 nan/nan/nan nan/nan/nan
1 448 0 0 0 0 0 nan/nan/nan nan/nan/nan
2 448 0 0 0 0 0 nan/nan/nan nan/nan/nan
3 384 0 0 0 0 0 nan/nan/nan nan/nan/nan
4 320 52 139437 209316 34996 8129 -56.2002/-42.7865/-32.7998 6.2138/32.5482/53.6770
5 144 102 100254 158707 25586 96013 -506.9092/-503.1813/-378.0397 12.8110/109.5153/138.3581
6 96 61 110399 149031 17088 92803 -506.4873/-503.3776/-412.3926 53.2054/124.2575/136.4059
7 48 48 132222 219052 16356 95003 -506.1172/-503.3639/-412.8861 90.7064/105.1855/106.9902
8 153 153 229080 384768 82518 147226 -505.8568/-503.1676/-422.8789 82.0687/93.3180/110.0400
near_camera_up_tiles (sorted by XY tile-centre distance):
@@ -1,5 +0,0 @@
version=1 voxel_um=250000 tiles=960 camera_m=(-11.612443923950195, -118.22123718261719, -506.7861328125) yaw=0.0 pitch=0.0
lod tiles mesh_tiles vertices triangles skirt_triangles upward_surface_triangles upward_centroid_z_m_p05/median/p95 tile_up_z_span_m_p50/p95/max
0 512 0 0 0 0 0 nan/nan/nan nan/nan/nan
1 448 0 0 0 0 0 nan/nan/nan nan/nan/nan
near_camera_up_tiles (sorted by XY tile-centre distance):
@@ -1,5 +0,0 @@
version=1 voxel_um=250000 tiles=960 camera_m=(0.0, -35.0, -75.0) yaw=0.0 pitch=0.0
lod tiles mesh_tiles vertices triangles skirt_triangles upward_surface_triangles upward_centroid_z_m_p05/median/p95 tile_up_z_span_m_p50/p95/max
0 512 0 0 0 0 0 nan/nan/nan nan/nan/nan
1 448 76 153596 265964 51168 82938 -43.9418/-14.9892/-6.3079 0.8803/5.3022/10.8345
near_camera_up_tiles (sorted by XY tile-centre distance):
@@ -1,5 +0,0 @@
version=1 voxel_um=250000 tiles=960 camera_m=(0.0, -35.0, -42.0) yaw=0.0 pitch=-8.0
lod tiles mesh_tiles vertices triangles skirt_triangles upward_surface_triangles upward_centroid_z_m_p05/median/p95 tile_up_z_span_m_p50/p95/max
0 512 25 80260 139176 28088 34692 -43.9640/-33.5509/-32.8013 0.7072/4.9851/7.0117
1 448 131 295583 474122 103438 230953 -33.6783/3.9507/4.1428 0.3514/4.4247/10.8345
near_camera_up_tiles (sorted by XY tile-centre distance):
+11 -113
View File
@@ -25,8 +25,6 @@ namespace
float Position[3];
float Normal[3];
float Height;
float WorldPosition[3];
float TileData[4];
};
struct FScene
@@ -34,8 +32,6 @@ namespace
std::uint32_t VoxelSizeMicrons=0;
std::array<double,3> CameraMeters{};
double YawDegrees=0.0,PitchDegrees=0.0;
float MinimumHeight=std::numeric_limits<float>::infinity();
float MaximumHeight=-std::numeric_limits<float>::infinity();
std::vector<FVertex> Vertices;
std::vector<std::uint32_t> Indices;
std::uint64_t Triangles=0,SkirtTriangles=0;
@@ -74,8 +70,7 @@ namespace
if(!Read(File,SurfaceVertices)||!Read(File,SkirtCount)||!Read(File,VertexCount)
||!Read(File,IndexCount)||!Read(File,MaskCount)
||VertexCount>10000000||IndexCount>60000000||MaskCount>20000000
||IndexCount%3!=0||(MaskCount!=0&&MaskCount!=IndexCount/3)
||Key[3]<0||Key[3]>8)
||IndexCount%3!=0||(MaskCount!=0&&MaskCount!=IndexCount/3))
{Error="tile mesh header is malformed";return false;}
if(TotalVertices+VertexCount>std::numeric_limits<std::uint32_t>::max()
||TotalIndices+IndexCount>std::numeric_limits<std::size_t>::max())
@@ -91,25 +86,6 @@ namespace
for(float& Value:Scene.Vertices[I].Normal)if(!Read(File,Value))
{Error="tile normals are truncated";return false;}
Scene.Vertices[I].Height=Scene.Vertices[I].Position[2];
Scene.MinimumHeight=std::min(Scene.MinimumHeight,Scene.Vertices[I].Height);
Scene.MaximumHeight=std::max(Scene.MaximumHeight,Scene.Vertices[I].Height);
}
const double TileVoxels=static_cast<double>(32u<<Key[3]);
const float TileExtentMeters=static_cast<float>(TileVoxels
*static_cast<double>(Scene.VoxelSizeMicrons)/1000000.0);
const float TileOriginX=static_cast<float>(static_cast<double>(Key[0])
*TileExtentMeters);
const float TileOriginY=static_cast<float>(static_cast<double>(Key[1])
*TileExtentMeters);
for(std::size_t I=Base;I<Base+VertexCount;++I)
{
FVertex& Vertex=Scene.Vertices[I];
std::copy(std::begin(Vertex.Position),std::end(Vertex.Position),
std::begin(Vertex.WorldPosition));
Vertex.TileData[0]=TileOriginX;
Vertex.TileData[1]=TileOriginY;
Vertex.TileData[2]=TileExtentMeters;
Vertex.TileData[3]=static_cast<float>(Key[3]);
}
const std::size_t IndexOffset=Scene.Indices.size();
Scene.Indices.resize(IndexOffset+IndexCount);
@@ -175,12 +151,8 @@ precision highp float;
layout(location=0) in vec3 inCameraPosition;
layout(location=1) in vec3 inNormal;
layout(location=2) in float inHeight;
layout(location=3) in vec3 inWorldPosition;
layout(location=4) in vec4 inTileData;
out vec3 vNormal;
out float vHeight;
out vec3 vWorldPosition;
out vec4 vTileData;
uniform float uTanHalfHorizontal;
uniform float uTanHalfVertical;
uniform float uNear;
@@ -193,54 +165,19 @@ void main()
inCameraPosition.y/uTanHalfVertical,clipZ,z);
vNormal=normalize(inNormal);
vHeight=inHeight;
vWorldPosition=inWorldPosition;
vTileData=inTileData;
})GLSL";
static constexpr char FragmentSource[]=R"GLSL(#version 300 es
precision highp float;
in vec3 vNormal;
in float vHeight;
in vec3 vWorldPosition;
in vec4 vTileData;
out vec4 outColor;
uniform int uReadableMode;
uniform int uTileEdges;
uniform float uHeightCenter;
uniform float uHeightRange;
uniform float uTileEdgeWidth;
uniform vec3 uLightDirection;
void main()
{
vec3 n=normalize(vNormal);
vec3 color;
if(uReadableMode==0)
{
vec3 normalColor=abs(n)*0.78+vec3(0.12);
float band=mod(floor((vHeight+10000.0)/80.0),2.0);
vec3 bandColor=mix(vec3(0.24,0.31,0.37),vec3(0.39,0.42,0.35),band);
color=mix(bandColor,normalColor,0.68);
}
else
{
float heightT=clamp(0.5+(vHeight-uHeightCenter)/max(uHeightRange,0.001),0.0,1.0);
vec3 heightTint=mix(vec3(0.80,0.91,1.0),vec3(1.0,0.91,0.77),heightT);
float diffuse=max(dot(n,normalize(uLightDirection)),0.0);
float illumination=0.62+0.38*diffuse;
color=vec3(0.64,0.62,0.56)*heightTint*illumination;
}
if(uTileEdges!=0&&vTileData.z>0.0)
{
vec2 local=mod(vWorldPosition.xy-vTileData.xy,vTileData.z);
vec2 edge=min(local,vec2(vTileData.z)-local);
float distanceToTileEdge=min(edge.x,edge.y);
float antialias=max(fwidth(distanceToTileEdge),0.005);
float line=1.0-smoothstep(uTileEdgeWidth-antialias,
uTileEdgeWidth+antialias,distanceToTileEdge);
vec3 edgeColor=mix(vec3(0.16,0.22,0.25),vec3(0.30,0.23,0.14),
clamp(vTileData.w/8.0,0.0,1.0));
color=mix(color,edgeColor,0.32*line);
}
outColor=vec4(color,1.0);
vec3 normalColor=abs(n)*0.78+vec3(0.12);
float band=mod(floor((vHeight+10000.0)/80.0),2.0);
vec3 bandColor=mix(vec3(0.24,0.31,0.37),vec3(0.39,0.42,0.35),band);
outColor=vec4(mix(bandColor,normalColor,0.68),1.0);
})GLSL";
const GLuint VS=CompileShader(GL_VERTEX_SHADER,VertexSource);
const GLuint FS=CompileShader(GL_FRAGMENT_SHADER,FragmentSource);
@@ -287,8 +224,7 @@ void main()
}
bool Render(FScene& Scene,const std::filesystem::path& Output,int Width,int Height,
double HorizontalFovDegrees,bool ReadableMode,bool TileEdges,
double TileEdgeWidthMeters)
double HorizontalFovDegrees)
{
EGLDisplay Display=MakeDisplay();
EGLint Major=0,Minor=0;
@@ -368,10 +304,6 @@ void main()
reinterpret_cast<void*>(offsetof(FVertex,Normal)));
glEnableVertexAttribArray(2);glVertexAttribPointer(2,1,GL_FLOAT,GL_FALSE,sizeof(FVertex),
reinterpret_cast<void*>(offsetof(FVertex,Height)));
glEnableVertexAttribArray(3);glVertexAttribPointer(3,3,GL_FLOAT,GL_FALSE,sizeof(FVertex),
reinterpret_cast<void*>(offsetof(FVertex,WorldPosition)));
glEnableVertexAttribArray(4);glVertexAttribPointer(4,4,GL_FLOAT,GL_FALSE,sizeof(FVertex),
reinterpret_cast<void*>(offsetof(FVertex,TileData)));
glViewport(0,0,Width,Height);glDisable(GL_CULL_FACE);glEnable(GL_DEPTH_TEST);
glDepthFunc(GL_LEQUAL);glClearColor(0.035f,0.055f,0.075f,1.0f);
glClearDepthf(1.0f);glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT);
@@ -380,14 +312,6 @@ void main()
glUniform1f(glGetUniformLocation(Program,"uTanHalfVertical"),static_cast<float>(TanHalfVertical));
glUniform1f(glGetUniformLocation(Program,"uNear"),static_cast<float>(Near));
glUniform1f(glGetUniformLocation(Program,"uFar"),static_cast<float>(Far));
glUniform1i(glGetUniformLocation(Program,"uReadableMode"),ReadableMode?1:0);
glUniform1i(glGetUniformLocation(Program,"uTileEdges"),TileEdges?1:0);
glUniform1f(glGetUniformLocation(Program,"uHeightCenter"),
static_cast<float>(Scene.CameraMeters[2]));
glUniform1f(glGetUniformLocation(Program,"uHeightRange"),64.0f);
glUniform1f(glGetUniformLocation(Program,"uTileEdgeWidth"),
static_cast<float>(TileEdgeWidthMeters));
glUniform3f(glGetUniformLocation(Program,"uLightDirection"),-0.42f,-0.58f,0.70f);
glDrawElements(GL_TRIANGLES,static_cast<GLsizei>(Scene.Indices.size()),GL_UNSIGNED_INT,nullptr);
glFinish();
std::vector<std::uint8_t> Pixels(static_cast<std::size_t>(Width)*Height*4);
@@ -399,10 +323,9 @@ void main()
{
std::error_code SizeError;
const auto Bytes=std::filesystem::file_size(Output,SizeError);
std::printf("[OffscreenCapture] triangles=%llu tiles=%u skirts=%llu shading=%s tile_edges=%s edge_width_m=%.3f png_bytes=%llu status=PASS path=%s\n",
std::printf("[OffscreenCapture] triangles=%llu tiles=%u skirts=%llu png_bytes=%llu status=PASS path=%s\n",
static_cast<unsigned long long>(Scene.Triangles),Scene.TileCount,
static_cast<unsigned long long>(Scene.SkirtTriangles),
ReadableMode?"readable":"normal",TileEdges?"on":"off",TileEdgeWidthMeters,
static_cast<unsigned long long>(SizeError?0:Bytes),Output.string().c_str());
}
glDeleteProgram(Program);glDeleteBuffers(1,&Ebo);glDeleteBuffers(1,&Vbo);
@@ -418,13 +341,12 @@ int main(int argc,char** argv)
{
if(argc<3)
{
std::fprintf(stderr,"Usage: %s scene.vfmesh output.png [--size=WIDTHxHEIGHT] [--fov-deg=90] [--camera-m=X,Y,Z] [--yaw-deg=N] [--pitch-deg=N] [--shade=readable|normal] [--tile-edges=on|off] [--tile-edge-width-m=N]\n",argv[0]);
std::fprintf(stderr,"Usage: %s scene.vfmesh output.png [--size=WIDTHxHEIGHT] [--fov-deg=90] [--camera-m=X,Y,Z] [--yaw-deg=N] [--pitch-deg=N]\n",argv[0]);
return 2;
}
int Width=2560,Height=1440;double Fov=90.0;
bool HasYaw=false,HasPitch=false;double Yaw=0.0,Pitch=0.0;
bool HasCamera=false;std::array<double,3> Camera{};
bool ReadableMode=true,TileEdges=false;double TileEdgeWidthMeters=0.15;
for(int I=3;I<argc;++I)
{
const std::string_view Arg(argv[I]);
@@ -458,28 +380,6 @@ int main(int argc,char** argv)
{std::fprintf(stderr,"Invalid camera angle\n");return 2;}
if(IsYaw){Yaw=Parsed;HasYaw=true;}else{Pitch=Parsed;HasPitch=true;}
}
else if(Arg.starts_with("--shade="))
{
const std::string_view Mode=Arg.substr(8);
if(Mode=="readable")ReadableMode=true;
else if(Mode=="normal")ReadableMode=false;
else {std::fprintf(stderr,"Invalid shading mode\n");return 2;}
}
else if(Arg.starts_with("--tile-edges="))
{
const std::string_view Mode=Arg.substr(13);
if(Mode=="on")TileEdges=true;
else if(Mode=="off")TileEdges=false;
else {std::fprintf(stderr,"Invalid tile-edge mode\n");return 2;}
}
else if(Arg.starts_with("--tile-edge-width-m="))
{
char* End=nullptr;const std::string Value(Arg.substr(20));
TileEdgeWidthMeters=std::strtod(Value.c_str(),&End);
if(End==Value.c_str()||*End!='\0'||!std::isfinite(TileEdgeWidthMeters)
||TileEdgeWidthMeters<=0.0||TileEdgeWidthMeters>5.0)
{std::fprintf(stderr,"Invalid tile-edge width\n");return 2;}
}
else {std::fprintf(stderr,"Unknown option: %.*s\n",static_cast<int>(Arg.size()),Arg.data());return 2;}
}
FScene Scene;std::string Error;
@@ -488,12 +388,10 @@ int main(int argc,char** argv)
if(HasCamera)Scene.CameraMeters=Camera;
if(HasYaw)Scene.YawDegrees=Yaw;
if(HasPitch)Scene.PitchDegrees=Pitch;
std::printf("[OffscreenScene] voxels_per_meter=%.6f camera_m=(%.3f,%.3f,%.3f) yaw=%.2f pitch=%.2f height_m=%.3f..%.3f shade=%s tile_edges=%s vertices=%zu triangles=%llu\n",
std::printf("[OffscreenScene] voxels_per_meter=%.6f camera_m=(%.3f,%.3f,%.3f) yaw=%.2f pitch=%.2f vertices=%zu triangles=%llu\n",
1000000.0/Scene.VoxelSizeMicrons,Scene.CameraMeters[0],Scene.CameraMeters[1],Scene.CameraMeters[2],
Scene.YawDegrees,Scene.PitchDegrees,Scene.MinimumHeight,Scene.MaximumHeight,
ReadableMode?"readable":"normal",TileEdges?"on":"off",Scene.Vertices.size(),
Scene.YawDegrees,Scene.PitchDegrees,Scene.Vertices.size(),
static_cast<unsigned long long>(Scene.Triangles));
return Render(Scene,argv[2],Width,Height,Fov,ReadableMode,TileEdges,
TileEdgeWidthMeters)?0:3;
return Render(Scene,argv[2],Width,Height,Fov)?0:3;
}
+16 -26
View File
@@ -35,6 +35,7 @@ namespace
bool StreamBenchmark=false;
bool StreamPipeline=false;
bool StreamHash=false;
bool TileCachePrototype=false;
bool ProfileColumnOverlap=false;
bool UseSharedColumnCache=true;
bool ForceGenerate=false;
@@ -63,7 +64,6 @@ namespace
float CameraYawDegrees=0.0f;
float CameraPitchDegrees=0.0f;
std::filesystem::path MeshExportPath;
std::filesystem::path WorldAssemblyManifestOutputPath;
};
bool ParseInteger(std::string_view text,int& output)
@@ -116,11 +116,11 @@ namespace
" --region-crossing (compile entering Core regions at the camera step)\n"
" --column-cache=shared|thread-local (default: shared)\n"
" --mesh-export=FILE (write the exact generated tile meshes for offline rendering)\n"
" --world-assembler-manifest-output=FILE (write the resolved plan for diagnostics)\n"
" --profile-column-overlap --far-horizon-lod8-step=N --force-generate\n"
" --seed=N --recipe=surface-world|floating-islands\n"
" --auto-surface-camera (place the 27-tile cube on this seed's surface)\n"
" --mesh-coverage (report non-empty tiles and indexed triangles)\n"
" --tile-cache-prototype (opt-in exact disk-cache round trip; writes Saved/Linux/cache/l16-prototype)\n"
" --screen-space-gate (measure display error and historical negative controls)\n"
" --audit-uniform-block-proofs (fill and check every interval-certified block)\n"
" --profile-lod-work (report per-LOD density sample counts)\n"
@@ -139,6 +139,15 @@ namespace
else if(arg=="--stream-benchmark")options.StreamBenchmark=true;
else if(arg=="--stream-pipeline")options.StreamPipeline=true;
else if(arg=="--stream-hash")options.StreamHash=true;
else if(arg=="--tile-cache-prototype")
{
options.TileCachePrototype=true;
options.StreamPipeline=true;
options.StreamBenchmark=true;
options.WorldAssemblerPrototype=true;
options.ForceGenerate=true;
options.StreamHash=true;
}
else if(arg=="--profile-column-overlap")options.ProfileColumnOverlap=true;
else if(arg=="--force-generate")options.ForceGenerate=true;
else if(arg=="--mesh-coverage")options.MeshCoverage=true;
@@ -206,13 +215,6 @@ namespace
if(options.MeshExportPath.empty())
{std::fprintf(stderr,"--mesh-export needs a file path\n");return false;}
}
else if(arg.starts_with("--world-assembler-manifest-output="))
{
options.WorldAssemblyManifestOutputPath=std::filesystem::path(
arg.substr(std::string_view("--world-assembler-manifest-output=").size()));
if(options.WorldAssemblyManifestOutputPath.empty())
{std::fprintf(stderr,"--world-assembler-manifest-output needs a file path\n");return false;}
}
else if(arg.starts_with("--workers="))
{if(!readInt("--workers=",options.WorkerCount))return false;}
else if(arg.starts_with("--clip-radius="))
@@ -323,22 +325,6 @@ int main(int argc,char** argv)
&&!StandaloneWorldAssemblyPrototype::AttachFloatingIslandWorldAssembly(
*ownerWorld,ownerJsonRoot,worldAssembly,error))
{std::fprintf(stderr,"Floating-island assembly setup failed: %s\n",error.c_str());return 2;}
if(!options.WorldAssemblyManifestOutputPath.empty())
{
std::error_code directoryError;
const auto parent=options.WorldAssemblyManifestOutputPath.parent_path();
if(!parent.empty())std::filesystem::create_directories(parent,directoryError);
if(directoryError)
{std::fprintf(stderr,"Manifest output directory failed: %s\n",directoryError.message().c_str());return 2;}
std::ofstream manifestFile(options.WorldAssemblyManifestOutputPath,std::ios::binary|std::ios::trunc);
if(!manifestFile)
{std::fprintf(stderr,"Cannot write manifest output: %s\n",options.WorldAssemblyManifestOutputPath.string().c_str());return 2;}
manifestFile<<worldAssembly.ToJson()<<'\n';
if(!manifestFile)
{std::fprintf(stderr,"Manifest output write failed: %s\n",options.WorldAssemblyManifestOutputPath.string().c_str());return 2;}
std::printf("[WorldAssemblyManifest] path=%s status=PASS\n",
options.WorldAssemblyManifestOutputPath.string().c_str());
}
if(options.AutoSurfaceCamera)
{
constexpr std::int64_t UnitsPerVoxel=VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel;
@@ -402,5 +388,9 @@ int main(int argc,char** argv)
cameraStepVoxels,options.AuditUniformBlockProofs,options.ProfileLodWork,
options.RegionCrossing,options.AuditTileShapeReadiness,
options.DiagnosticSlowShapeCompileMs,options.AuditBasePageEquivalence,
options.AuditRouteParity);
options.AuditRouteParity,
options.TileCachePrototype
?std::filesystem::current_path()/"Saved/Linux/cache/l16-prototype"
:std::filesystem::path{},
options.DataPath);
}
+112 -7
View File
@@ -3,6 +3,7 @@
#include "OwnerWorldLoader.h"
#include "StreamingWorld.h"
#include "LinuxScreenSpaceGate.h"
#include "../Research/WorldAssemblerM3/Linux16/TileCachePrototype.h"
#include <algorithm>
#include <array>
@@ -253,7 +254,9 @@ namespace StandaloneStreamBenchmark
bool bAuditUniformBlockProofs=false,bool bReportLodWorkProfile=false,
bool bRegionCrossing=false,
bool bAuditTileShapeReadiness=false,int DiagnosticSlowShapeCompileMs=0,
bool bAuditBasePageEquivalence=false,bool bAuditRouteParity=false)
bool bAuditBasePageEquivalence=false,bool bAuditRouteParity=false,
const std::filesystem::path& TileCachePrototypeRoot={},
const std::filesystem::path& OwnerSnapshotPath={})
{
using Clock=std::chrono::steady_clock;
FStandaloneOwnerWorld& World=*OwnerWorld;
@@ -274,6 +277,42 @@ namespace StandaloneStreamBenchmark
const bool bRoundTunnels=World.Generator->bStandaloneResearchRoundTunnels;
const bool bEnhancedTunnelWalls=World.Generator->bStandaloneResearchEnhancedWalls;
VoxelForgeCore::World Shape(CoreSnapshot);
std::unique_ptr<StandaloneTileCachePrototype::TileDiskCache> TileCache;
FStandaloneCoreTileRoute CoreTileRoute;
VoxelForgeCore::TileGenerationOptions CacheOptions;
if(!TileCachePrototypeRoot.empty())
{
const auto RepoRoot=std::filesystem::current_path().lexically_normal();
const auto AllowedRoot=(RepoRoot/"Saved/Linux/cache").lexically_normal();
const auto RequestedRoot=std::filesystem::absolute(TileCachePrototypeRoot).lexically_normal();
const std::string AllowedString=AllowedRoot.string()+std::string(1,std::filesystem::path::preferred_separator);
if(RequestedRoot.string().rfind(AllowedString,0)!=0||OwnerSnapshotPath.empty())
{
std::fprintf(stderr,"[TileCachePrototype] path rejected: cache writes must stay under Saved/Linux/cache and owner snapshot identity must be provided.\n");
return 2;
}
if(ClosestDisplayDistances[0]==0.0)
ClosestDisplayDistances=StandaloneScreenSpaceGate::ClosestDisplayedDistancesMeters(
World,CameraPositionVoxels,World.CoreVoxelSizeMicrons);
const auto Identity=StandaloneTileCachePrototype::MakeIdentity(
World,CoreSnapshot,OwnerSnapshotPath);
if(Identity.ownerSnapshotHash==0||!Assembly)
{
std::fprintf(stderr,"[TileCachePrototype] missing owner snapshot or world assembly identity.\n");
return 2;
}
TileCache=std::make_unique<StandaloneTileCachePrototype::TileDiskCache>(
RequestedRoot,Identity,ClosestDisplayDistances);
CacheOptions.hashDensity=true;
StandaloneTileCachePrototype::TileDiskCache* Cache=TileCache.get();
CoreTileRoute=[Cache](const VoxelForgeCore::World& TileWorld,
const VoxelForgeCore::TileKey& Key,
const VoxelForgeCore::TileGenerationOptions& Options,bool& bHit,
std::string& Error)
{return Cache->GenerateOrLoad(TileWorld,Key,Options,bHit,Error);};
std::printf("[TileCachePrototype] enabled root=%s owner_snapshot=%s mode=worker_generate_store_read_or_warm_hit codec=zlib-6\n",
RequestedRoot.string().c_str(),OwnerSnapshotPath.string().c_str());
}
FStandaloneDesiredTileSet Desired=StandaloneWorldMesher::BuildDesiredSet(World,CameraPositionVoxels);
const auto CoreTiles=Shape.desiredTiles(
{CameraPositionVoxels.X,CameraPositionVoxels.Y,CameraPositionVoxels.Z});
@@ -411,7 +450,7 @@ namespace StandaloneStreamBenchmark
ShapeRequest.enhancedTunnelWalls=bEnhancedTunnelWalls;
FStandaloneStreamingGenerator Streamer(World,WorkerCount,bHashDensity,nullptr,MaxLevel,
bSkipUvAndColors,bSkipMaterialGroups,bSkipUV,bSkipColors,&Shape,bProfileColumnOverlap,
FarHorizonLOD8InteriorStepVoxels,bUseSharedColumnCache);
FarHorizonLOD8InteriorStepVoxels,bUseSharedColumnCache,std::move(CoreTileRoute));
Streamer.SetUniformBlockProofAuditEnabled(bAuditUniformBlockProofs);
if(bScreenSpaceGate||bReportLodWorkProfile)Streamer.SetMesherStageProfilingEnabled(true);
const auto StreamStart=Clock::now();
@@ -657,6 +696,7 @@ namespace StandaloneStreamBenchmark
VoxelForgeCore::TileGenerationOptions PageOptions=CaptureOptions;
PageOptions.auditUniformBlockProofs=true;
int32 CachePairs=0,CacheMismatches=0,EditCachePairs=0,EditCacheMismatches=0;
int32 BasePageMismatches=0,PageRoutesVerified=0;
std::uint64_t PageProofBlocks=0,PageProofSamples=0,PageProofFallbacks=0,
PageProofUnknown=0,PageProofViolations=0;
@@ -716,6 +756,28 @@ namespace StandaloneStreamBenchmark
PageProofUnknown+=Page->diagnostics.uniformBlockProofUnknownBlocks;
PageProofViolations+=Page->diagnostics.uniformBlockProofViolations;
}
if(TileCache)
{
VoxelForgeCore::TileMesh Cached;
std::string CacheError;
bool bCacheReady=TileCache->LoadExpected(Row.Candidate.Key,CacheOptions,Cached,CacheError);
if(!bCacheReady&&Row.Base)
bCacheReady=TileCache->StoreAndLoadExplicit(*Row.Base,CacheOptions,
TileCache->identity(),Cached,CacheError);
const bool bCacheSame=bCacheReady&&Row.Base
&&SameTileOutput(*Row.Base,Cached)
&&StandaloneTileCachePrototype::SameExactMesh(*Row.Base,Cached);
++CachePairs;
if(!bCacheSame)++CacheMismatches;
std::printf("[RouteParityPair] route=base/disk-cache key=(%d,%d,%d,lod%d) same_density=%d same_mesh=%d same_payload=%d same_topology=%d exact_arrays=%d status=%s error=%s\n",
Row.Candidate.Key.coordinate.x,Row.Candidate.Key.coordinate.y,
Row.Candidate.Key.coordinate.z,Row.Candidate.Key.lod,
Row.Base&&Cached.densityHash==Row.Base->densityHash?1:0,
Row.Base&&Cached.meshHash==Row.Base->meshHash?1:0,
Row.Base&&Cached.payloadHash==Row.Base->payloadHash?1:0,
Row.Base&&Cached.topologyHash==Row.Base->topologyHash?1:0,
bCacheSame?1:0,bCacheSame?"PASS":"FAIL",CacheError.c_str());
}
std::printf("[RouteParityPair] route=base/page key=(%d,%d,%d,lod%d) page_requested=%d page_verified=%d same_density=%d same_mesh=%d same_payload=%d same_topology=%d samples=%llu max_abs_density_delta=%.9g sign_mismatches=%llu edge_crossing_mismatches=%llu shared_surface_edges=%llu max_surface_delta_vox=%.9g exact_query_mismatches=%llu base_native_float=%llu page_native_float=%llu first_density=(%.9g,%.9g) status=%s error=%s\n",
Row.Candidate.Key.coordinate.x,Row.Candidate.Key.coordinate.y,
Row.Candidate.Key.coordinate.z,Row.Candidate.Key.lod,
@@ -899,6 +961,27 @@ namespace StandaloneStreamBenchmark
&&EditedPage->diagnostics.analyticRegionPageRoute;
const bool bEditSame=EditedBase&&EditedPage&&bEditPageVerified
&&SameTileOutput(*EditedBase,*EditedPage);
bool bEditCacheSame=false;
if(TileCache&&EditedBase&&Applied)
{
VoxelForgeCore::TileMesh CachedEdited;
std::string CacheError;
const auto EditedIdentity=TileCache->WithEdit(Transaction.sequence,Applied->digest);
bEditCacheSame=TileCache->StoreAndLoadExplicit(*EditedBase,CacheOptions,
EditedIdentity,CachedEdited,CacheError)
&&SameTileOutput(*EditedBase,CachedEdited);
++EditCachePairs;
if(!bEditCacheSame)++EditCacheMismatches;
std::printf("[RouteParityPair] route=edited-base/disk-cache key=(%d,%d,%d,lod%d) edit_sequence=%llu edit_digest=0x%016llx same_density=%d same_mesh=%d same_payload=%d same_topology=%d exact_arrays=%d status=%s error=%s\n",
Key.coordinate.x,Key.coordinate.y,Key.coordinate.z,Key.lod,
static_cast<unsigned long long>(Transaction.sequence),
static_cast<unsigned long long>(Applied->digest),
CachedEdited.densityHash==EditedBase->densityHash?1:0,
CachedEdited.meshHash==EditedBase->meshHash?1:0,
CachedEdited.payloadHash==EditedBase->payloadHash?1:0,
CachedEdited.topologyHash==EditedBase->topologyHash?1:0,
bEditCacheSame?1:0,bEditCacheSame?"PASS":"FAIL",CacheError.c_str());
}
const bool bEditPass=Before&&Applied&&EditedBase&&bEditPageBuilt
&&bEditPageVerified&&bEditChanged&&bEditSame&&EditComparison.valid
&&EditComparison.exactQueryMismatches==0;
@@ -996,11 +1079,13 @@ namespace StandaloneStreamBenchmark
&&HistoryPairs==(bHasCameraStep&&bRegionCrossing?1:0)&&HistoryMismatches==0
&&PageRoutesVerified==static_cast<int32>(Rows.size())
&&NativeRoutesVerified==NativePairs&&PageProofViolations==0
&&PageExactMismatches==0&&NativeExactMismatches==0;
std::printf("[RouteParityAudit] seed=%d page_pairs=%zu page_verified=%d page_mismatches=%d native_only_pairs=%d native_verified=%d native_mismatches=%d higher_lod_exact_pairs=%d higher_lod_exact_mismatches=%d edit_pairs=%d edit_mismatches=%d history_pairs=%d history_mismatches=%d exact_query_samples=%llu exact_query_mismatches=%llu page_density_samples=%llu max_abs_density_delta=%.9g mean_abs_density_delta=%.9g sign_mismatches=%llu edge_crossing_mismatches=%llu shared_surface_edges=%llu max_surface_delta_vox=%.9g mean_surface_delta_vox=%.9g page_proof_blocks=%llu page_proof_samples=%llu page_proof_fallbacks=%llu page_proof_unknown=%llu page_proof_violations=%llu status=%s\n",
&&PageExactMismatches==0&&NativeExactMismatches==0
&&(!TileCache||(CachePairs>0&&CacheMismatches==0&&EditCachePairs==1&&EditCacheMismatches==0));
std::printf("[RouteParityAudit] seed=%d page_pairs=%zu page_verified=%d page_mismatches=%d native_only_pairs=%d native_verified=%d native_mismatches=%d higher_lod_exact_pairs=%d higher_lod_exact_mismatches=%d edit_pairs=%d edit_mismatches=%d history_pairs=%d history_mismatches=%d cache_pairs=%d cache_mismatches=%d edit_cache_pairs=%d edit_cache_mismatches=%d exact_query_samples=%llu exact_query_mismatches=%llu page_density_samples=%llu max_abs_density_delta=%.9g mean_abs_density_delta=%.9g sign_mismatches=%llu edge_crossing_mismatches=%llu shared_surface_edges=%llu max_surface_delta_vox=%.9g mean_surface_delta_vox=%.9g page_proof_blocks=%llu page_proof_samples=%llu page_proof_fallbacks=%llu page_proof_unknown=%llu page_proof_violations=%llu status=%s\n",
World.Seed,Rows.size(),PageRoutesVerified,BasePageMismatches,
NativePairs,NativeRoutesVerified,NativeMismatches,ExactQueryPairs,
ExactQueryMismatches,EditPairs,EditMismatches,HistoryPairs,HistoryMismatches,
CachePairs,CacheMismatches,EditCachePairs,EditCacheMismatches,
static_cast<unsigned long long>(PageExactSamples
+NativeExactSamples+ExactQuerySamples),
static_cast<unsigned long long>(PageExactMismatches
@@ -1640,10 +1725,12 @@ namespace StandaloneStreamBenchmark
const double WholeViewSeconds=std::chrono::duration<double>(WholeViewReady-ColdStart).count();
const double StreamSeconds=std::chrono::duration<double>(WholeViewReady-StreamStart).count();
const double FirstTileSeconds=std::chrono::duration<double>(FirstTileReady-ColdStart).count();
std::printf("[StreamPipeline] workers=%d shape=%.3fs first_tile_ready=%.3fs neighborhood_mesh=%.3fs neighborhood_ready=%.3fs full_detail_ready=%.3fs whole_view_ready=%.3fs neighborhood_tiles=%d/27 full_detail_tiles=%d/%d whole_tiles=%d/%zu stream_wall=%.3fs cache_hits=0 full_payload=1\n",
const std::uint64_t TileCacheHits=TileCache?TileCache->hitCount():Streamer.GetCacheHits();
std::printf("[StreamPipeline] workers=%d shape=%.3fs first_tile_ready=%.3fs neighborhood_mesh=%.3fs neighborhood_ready=%.3fs full_detail_ready=%.3fs whole_view_ready=%.3fs neighborhood_tiles=%d/27 full_detail_tiles=%d/%d whole_tiles=%d/%zu stream_wall=%.3fs cache_hits=%llu full_payload=1\n",
WorkerCount,ShapeSeconds,FirstTileSeconds,NeighborhoodMeshSeconds,NeighborhoodSeconds,
FullDetailSeconds,WholeViewSeconds,NeighborhoodFinished,
FullDetailFinished,FullDetailExpected,Finished,Desired.Tiles.size(),StreamSeconds);
FullDetailFinished,FullDetailExpected,Finished,Desired.Tiles.size(),StreamSeconds,
static_cast<unsigned long long>(TileCacheHits));
if(bReportMeshCoverage)
std::printf("[PipelineMeshCoverage] tiles=%d nonempty_mesh_tiles=%llu indexed_triangles=%llu\n",
Finished,static_cast<unsigned long long>(WholeNonEmptyMeshTiles),
@@ -1739,10 +1826,28 @@ namespace StandaloneStreamBenchmark
static_cast<unsigned long long>(ExactDetailHash),
static_cast<unsigned long long>(ExactCollisionHash));
}
bool bTileCacheSafetyPassing=true;
if(TileCache)
{
std::string CacheSafetyError;
bTileCacheSafetyPassing=TileCache->RunSafetyControls(CacheOptions,CacheSafetyError);
if(!bTileCacheSafetyPassing)
std::fprintf(stderr,"[TileCacheSafety] failed: %s\n",CacheSafetyError.c_str());
TileCache->PrintSummary();
TileCache->MeasureDiskReads();
const std::uint64_t CacheBytes=TileCache->bytesOnDisk();
std::printf("[TileCacheBudget] bytes=%llu maximum=%llu status=%s\n",
static_cast<unsigned long long>(CacheBytes),
static_cast<unsigned long long>(StandaloneTileCachePrototype::MaximumDiskBytes),
CacheBytes<=StandaloneTileCachePrototype::MaximumDiskBytes?"PASS":"FAIL");
bTileCacheSafetyPassing=bTileCacheSafetyPassing
&&CacheBytes<=StandaloneTileCachePrototype::MaximumDiskBytes;
}
return NeighborhoodFinished==static_cast<int32>(Neighborhood.size())
&&FullDetailFinished==FullDetailExpected&&Finished==Update.DesiredTiles&&bHashesPassing
&&(!bProfileColumnOverlap||bColumnOverlapCapturePassing)
&&(!bExportRequested||bSceneExportPassing)&&bScreenSpaceGatePassing
&&(!bAuditUniformBlockProofs||BlockProofViolations==0)?0:6;
&&(!bAuditUniformBlockProofs||BlockProofViolations==0)
&&bTileCacheSafetyPassing?0:6;
}
} // namespace StandaloneStreamBenchmark
+14 -4
View File
@@ -67,6 +67,10 @@ struct FStandaloneCpuTile
};
using FStandaloneCoreTileRoute = std::function<std::optional<VoxelForgeCore::TileMesh>(
const VoxelForgeCore::World&,const VoxelForgeCore::TileKey&,
const VoxelForgeCore::TileGenerationOptions&,bool&,std::string&)>;
inline uint64 HashStandaloneMeshPayload(const FVoxelMeshData& Mesh)
{
uint64 Hash=14695981039346656037ull;
@@ -430,14 +434,16 @@ public:
VoxelForgeCore::World* InCoreWorld=nullptr,
bool bInCaptureColumnCoordinates=false,
int32 InLod8HorizonInteriorStepVoxels=0,
bool bInSharedColumnCache=true)
bool bInSharedColumnCache=true,
FStandaloneCoreTileRoute InCoreTileRoute={})
:World(&InWorld),WorkerCount(FMath::Clamp(InWorkerCount,1,16)),bHashDensity(bInHashDensity),
MeshCache(InMeshCache),MaxLevelFilter(InMaxLevelFilter),
CoreWorld(InCoreWorld),bSkipUvAndColors(bInSkipUvAndColors),bSkipMaterialGroups(bInSkipMaterialGroups),
bSkipUV(bInSkipUV),bSkipColors(bInSkipColors),
bCaptureColumnCoordinates(bInCaptureColumnCoordinates),
bSharedColumnCache(bInSharedColumnCache),
Lod8HorizonInteriorStepVoxels(std::max(0,InLod8HorizonInteriorStepVoxels))
Lod8HorizonInteriorStepVoxels(std::max(0,InLod8HorizonInteriorStepVoxels)),
CoreTileRoute(std::move(InCoreTileRoute))
{
VoxelGenLOD::InitializeConsoleSwitches();
UVoxelMarchingCubesMesher::InitializeConsoleSwitches();
@@ -830,8 +836,11 @@ private:
Options.diagnosticColumnInterpolationStride=
std::max<std::uint32_t>(1,DiagnosticCellStride);
std::string Error;
auto Result=CoreWorld->generateTile(
{{Job.Key.Coord.X,Job.Key.Coord.Y,Job.Key.Coord.Z},Job.Key.Level},Options,Error);
const VoxelForgeCore::TileKey CoreKey{{Job.Key.Coord.X,Job.Key.Coord.Y,Job.Key.Coord.Z},Job.Key.Level};
bool bDiskCacheHit=false;
auto Result=CoreTileRoute
?CoreTileRoute(*CoreWorld,CoreKey,Options,bDiskCacheHit,Error)
:CoreWorld->generateTile(CoreKey,Options,Error);
if(!Result)
{
std::fprintf(stderr,"[VoxelForgeCore] tile generation failed: %s\n",Error.c_str());
@@ -1003,6 +1012,7 @@ private:
bool bCaptureColumnCoordinates=false;
bool bSharedColumnCache=true;
int32 Lod8HorizonInteriorStepVoxels=0;
FStandaloneCoreTileRoute CoreTileRoute;
bool bProfileMesherStages=false;
bool bAuditUniformBlockProofs=false;
std::atomic<uint64> CacheHitCount{0};