Fr0zka/VoxelForge
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

merge into: Fr0zka/VoxelForge:wip/l16-tile-cache
Fr0zka/VoxelForge:main Fr0zka/VoxelForge:assembler/world Fr0zka/VoxelForge:wip/l62-far-lod-cuts Fr0zka/VoxelForge:wip/l59-open-shaft Fr0zka/VoxelForge:wip/l25-helix Fr0zka/VoxelForge:wip/l17-fi-scale Fr0zka/VoxelForge:wip/l16-tile-cache Fr0zka/VoxelForge:wip/l15-far-budget Fr0zka/VoxelForge:wip/l5-far-cutoff Fr0zka/VoxelForge:to-be-examined Fr0zka/VoxelForge:experimental
..
pull from: Fr0zka/VoxelForge:wip/l15-far-budget
Fr0zka/VoxelForge:assembler/world Fr0zka/VoxelForge:wip/l62-far-lod-cuts Fr0zka/VoxelForge:wip/l59-open-shaft Fr0zka/VoxelForge:wip/l25-helix Fr0zka/VoxelForge:wip/l17-fi-scale Fr0zka/VoxelForge:wip/l16-tile-cache Fr0zka/VoxelForge:wip/l15-far-budget Fr0zka/VoxelForge:wip/l5-far-cutoff Fr0zka/VoxelForge:to-be-examined Fr0zka/VoxelForge:experimental Fr0zka/VoxelForge:main
1 Commits
wip/l16-tile-cache .. wip/l15-far-budget
Author SHA1 Message Date
Fr0zkaandClaude Opus 5.5 a6185b1a3f wip(L15, side branch): extended far gate + rejected far-mesh candidates (null result) 
Refine x2 at LOD4-8 saves 13.6% but breaks the 1 px rule, shading and swaps, and opens holes;
station x2 and ray removal open connector gaps. Not for assembler/world. See Linux15/REPORT.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CSFYgsdW8ejZSyiMf4qRSX
 2026-10-20 03:29:13 +02:00
15 changed files with 756 additions and 1779 deletions
Expand all files Collapse all files

No files matched your search

ARCHITECTURE.md
+20
View File
@@ -570,6 +570,15 @@ ApplyModification` (BlueprintCallable) + `EditorCarveSphere/EditorFillSphere` (C
driven by `EditorBrush*` props. driven by `EditorBrush*` props.
### 8.10 Performance invariants (DON'T regress) ### 8.10 Performance invariants (DON'T regress)
- **Far display mesh budget (L15)** (`WorldAssemblyEvaluatorVersion` 7): recipes may explicitly
declare `far_display_mesh` for LOD3+ display generation. With no declaration, the L14 mesh path
remains byte-for-byte selected. This setting must never alter LOD0–2, exact field queries,
collision, edits, or server state. A declared reduction must pass the same-LOD screen gate at the
closest displayed distance: at most 1.0 px geometry displacement, at most 1/255 normalized
shading/material delta, no worse swap pop, and no new holes, cracks, or floating pieces. Connector
wall repairs are safety geometry: retain every station/ray needed to close the existing gaps. Keep
the setting in the shared Core path so all recipe routes use the same declaration and evaluator.
- **Streaming** (`UpdateChunksAroundPosition`): rebuild/cull the desired set only when the player - **Streaming** (`UpdateChunksAroundPosition`): rebuild/cull the desired set only when the player
crosses a level-0 chunk boundary, an anchor moves, or a rebuild is forced; use the stamped crosses a level-0 chunk boundary, an anchor moves, or a rebuild is forced; use the stamped
desired map for the cull and idle via `bAllChunksLoaded`. On every rebuild, pending requests whose desired map for the cull and idle via `bAllChunksLoaded`. On every rebuild, pending requests whose
@@ -1840,6 +1849,17 @@ fill and check page tiles against the evaluator they actually sample. This is th
the Unreal adapter and its asynchronous MPSC/Epoch flow are unchanged. See the Unreal adapter and its asynchronous MPSC/Epoch flow are unchanged. See
[`Standalone/Research/WorldAssemblerM3/Linux13/REPORT.md`](Standalone/Research/WorldAssemblerM3/Linux13/REPORT.md). [`Standalone/Research/WorldAssemblerM3/Linux13/REPORT.md`](Standalone/Research/WorldAssemblerM3/Linux13/REPORT.md).
### 8.37 Screen-gated far display budget — World Assembler M3 L15
Evaluator version 7 adds the optional, serialized `far_display_mesh` recipe declaration for
display-only LOD3+ refinement and connector repair. A missing declaration selects the existing L14
path; exact queries, LOD0–2, collision, edits, and route binding stay on their existing shared Core
evaluators. The acceptance gate compares against today's same-LOD display mesh, reports geometry and
fixed-light/material shading deltas, checks every displayed tile, and measures each adjacent LOD
swap before and after. It rejects any new hole, crack, or floating piece, including lost connector
panels. No tested L15 candidate met the full budget, so no declaration is enabled by default. See
[`Standalone/Research/WorldAssemblerM3/Linux15/REPORT.md`](Standalone/Research/WorldAssemblerM3/Linux15/REPORT.md).
## 9. Multiplayer model (listen-server first, dedicated-friendly) ## 9. Multiplayer model (listen-server first, dedicated-friendly)
> **Status: transport is DESIGN ONLY — nothing is networked in-tree yet** (no `Replicated`/`HasAuthority`/RPCs; a > **Status: transport is DESIGN ONLY — nothing is networked in-tree yet** (no `Replicated`/`HasAuthority`/RPCs; a
CODEMAP.md
+11 -1
View File
@@ -1132,11 +1132,21 @@ Implementation and evidence: [`Standalone/Research/WorldAssemblerM3/Linux9/REPOR
| Symbol | Canonical file | Role | | Symbol | Canonical file | Role |
|---|---|---| |---|---|---|
| `FStandaloneShapeWorld::SetSharedDensitySource`, `SharedDensitySource` | `Standalone/Core/src/ShapeWorldInternal.h`, bound before publication in `Standalone/Core/src/VoxelForgeCore.cpp` | Makes compiled analytic pages delegate field, lattice, detail, classification, and solid queries to the Core world's base shape; `WorldAssemblyEvaluatorVersion` is 6. | | `FStandaloneShapeWorld::SetSharedDensitySource`, `SharedDensitySource` | `Standalone/Core/src/ShapeWorldInternal.h`, bound before publication in `Standalone/Core/src/VoxelForgeCore.cpp` | Makes compiled analytic pages delegate field, lattice, detail, classification, and solid queries to the Core world's base shape; `WorldAssemblyEvaluatorVersion` is 7, including optional far display-mesh recipe settings. |
The base manifest field is authoritative for tile generation and exact queries. See ARCHITECTURE The base manifest field is authoritative for tile generation and exact queries. See ARCHITECTURE
§8.36 and [`Standalone/Research/WorldAssemblerM3/Linux13/REPORT.md`](Standalone/Research/WorldAssemblerM3/Linux13/REPORT.md). §8.36 and [`Standalone/Research/WorldAssemblerM3/Linux13/REPORT.md`](Standalone/Research/WorldAssemblerM3/Linux13/REPORT.md).
### 3.23 Screen-gated far display mesh — World Assembler M3 L15
`ApplyFarDisplayMeshDeclaration` in `Standalone/Source/WorldAssemblyPrototype.h` parses the optional
`far_display_mesh` recipe block; `WorldAssemblyEvaluatorVersion` 7 serializes and hashes it.
`VoxelForgeCore::World::generateTile` consumes its display-only refinement and connector settings.
No declaration keeps the L14 output path. `Standalone/Source/LinuxScreenSpaceGate.h` compares every
desired LOD3–8 tile against today's same-LOD mesh and audits geometry, shading, swaps, and connector
coverage. The measured candidates and null result are in
[`Standalone/Research/WorldAssemblerM3/Linux15/REPORT.md`](Standalone/Research/WorldAssemblerM3/Linux15/REPORT.md).
## 4. The density pipeline (most-edited hot path) ## 4. The density pipeline (most-edited hot path)
### 4.1 `GetDensityWithParams` (TunnelNetwork) — VoxelGenerator.cpp:3749 ### 4.1 `GetDensityWithParams` (TunnelNetwork) — VoxelGenerator.cpp:3749
Standalone/BuildLinux.sh
+1 -1
View File
@@ -118,7 +118,7 @@ PIDS=()
CORE_LIBRARY="$BUILD_DIR/libVoxelForgeCore.a" CORE_LIBRARY="$BUILD_DIR/libVoxelForgeCore.a"
"$(command -v ar)" rcs "$CORE_LIBRARY" "${CORE_OBJECTS[@]}" "$(command -v ar)" rcs "$CORE_LIBRARY" "${CORE_OBJECTS[@]}"
OUTPUT="$BIN_DIR/VoxelForgeCoreHeadless-$VARIANT" OUTPUT="$BIN_DIR/VoxelForgeCoreHeadless-$VARIANT"
"$COMPILER" -pthread "${LEGACY_OBJECTS[@]}" "$CORE_LIBRARY" -lz -o "$OUTPUT" "$COMPILER" -pthread "${LEGACY_OBJECTS[@]}" "$CORE_LIBRARY" -o "$OUTPUT"
CAPTURE_OUTPUT="$BIN_DIR/VoxelForgeOffscreenCapture-$VARIANT" CAPTURE_OUTPUT="$BIN_DIR/VoxelForgeOffscreenCapture-$VARIANT"
"$COMPILER" -std=c++20 -O2 -fno-fast-math -pthread \ "$COMPILER" -std=c++20 -O2 -fno-fast-math -pthread \
"$ROOT/Source/LinuxOffscreenCapture.cpp" -o "$CAPTURE_OUTPUT" \ "$ROOT/Source/LinuxOffscreenCapture.cpp" -o "$CAPTURE_OUTPUT" \
Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h
+29 -1
View File
@@ -184,6 +184,12 @@ namespace VoxelForgeCore
std::uint64_t farConnectorColumnEvaluations = 0; std::uint64_t farConnectorColumnEvaluations = 0;
double farStratumColumnComputeMilliseconds = 0.0; double farStratumColumnComputeMilliseconds = 0.0;
double farConnectorColumnComputeMilliseconds = 0.0; double farConnectorColumnComputeMilliseconds = 0.0;
// Opt-in quality audit at the L14 route-wall stations and four axial rays.
std::uint64_t farConnectorAuditRays = 0;
std::uint64_t farConnectorAuditHoles = 0;
std::uint64_t farConnectorAuditFloating = 0;
std::vector<std::uint64_t> farConnectorAuditHoleIds;
std::vector<std::uint64_t> farConnectorAuditFloatingIds;
std::uint64_t farShellTrianglesAdded = 0; std::uint64_t farShellTrianglesAdded = 0;
std::uint64_t horizonGroundTriangles = 0; std::uint64_t horizonGroundTriangles = 0;
std::uint64_t horizonGroundTrianglesRemoved = 0; std::uint64_t horizonGroundTrianglesRemoved = 0;
@@ -250,7 +256,8 @@ namespace VoxelForgeCore
inline constexpr std::int64_t WorldAssemblySubvoxelsPerVoxel = 256; inline constexpr std::int64_t WorldAssemblySubvoxelsPerVoxel = 256;
inline constexpr std::uint32_t WorldAssemblySchemaVersion = 1; inline constexpr std::uint32_t WorldAssemblySchemaVersion = 1;
// Version 6 binds every compiled tile page to the world's base density evaluator. // Version 6 binds every compiled tile page to the world's base density evaluator.
inline constexpr std::uint32_t WorldAssemblyEvaluatorVersion = 6; // Version 7 adds optional, display-only far-mesh quality declarations.
inline constexpr std::uint32_t WorldAssemblyEvaluatorVersion = 7;
enum class FieldChannel : std::uint32_t enum class FieldChannel : std::uint32_t
{ {
@@ -312,6 +319,17 @@ namespace VoxelForgeCore
std::int64_t biomeClimateReliefWavelengthSubvoxels = 256 * 3333; std::int64_t biomeClimateReliefWavelengthSubvoxels = 256 * 3333;
std::int64_t biomeClimateMoistureWavelengthSubvoxels = 256 * 2500; std::int64_t biomeClimateMoistureWavelengthSubvoxels = 256 * 2500;
std::int64_t biomeClimateReliefContrastQ16 = 2ll << 16; std::int64_t biomeClimateReliefContrastQ16 = 2ll << 16;
// Optional display-only mesh budget. Zero leaves the L14 far-mesh path unchanged.
// Budget is in 1/1000 pixel at the closest display distance.
std::uint32_t farDisplayGeometryBudgetMillipixels = 0;
// Scale/mask change only display refinement density at the selected LODs.
// A zero scale or mask leaves the current refinement grids intact.
std::uint32_t farRefinementGridScale = 0;
std::uint32_t farRefinementLodMask = 0;
// Connector scale 0 and ray mask 0 preserve the current station/ray pattern.
std::uint32_t farConnectorStationScale = 0;
std::uint32_t farConnectorRayMask = 0;
bool farConnectorReuseStationRadii = false;
std::vector<WorldAssemblyBiomeProfileV1> biomes; std::vector<WorldAssemblyBiomeProfileV1> biomes;
std::uint64_t contentHash = 0; std::uint64_t contentHash = 0;
}; };
@@ -1099,6 +1117,16 @@ namespace VoxelForgeCore
// Incremente quand le plafond lointain change. Les hashes LOD4-8 incluent cette version; // Incremente quand le plafond lointain change. Les hashes LOD4-8 incluent cette version;
// densite et hashes LOD0-3 restent inchanges. // densite et hashes LOD0-3 restent inchanges.
std::uint32_t farHorizonVersion = 3; std::uint32_t farHorizonVersion = 3;
// Research-only override of the recipe declaration; used by same-LOD controls.
bool overrideFarDisplayMesh = false;
std::uint32_t farDisplayGeometryBudgetMillipixels = 0;
std::uint32_t farRefinementGridScale = 0;
std::uint32_t farRefinementLodMask = 0;
std::uint32_t farConnectorStationScale = 0;
std::uint32_t farConnectorRayMask = 0;
bool farConnectorReuseStationRadii = false;
bool diagnosticRemoveFarConnectorPanels = false;
bool diagnosticAuditFarConnectorCoverage = false;
// Linux research controls for reproducing L5 negative controls. Defaults preserve output. // Linux research controls for reproducing L5 negative controls. Defaults preserve output.
std::uint32_t diagnosticGlobalOctaveDrop = 0; std::uint32_t diagnosticGlobalOctaveDrop = 0;
std::uint32_t diagnosticCellStride = 1; std::uint32_t diagnosticCellStride = 1;
Standalone/Core/src/FarShellsInternal.h
+120 -4
View File
@@ -242,7 +242,8 @@ inline bool FindFarConnectorRadiusNearNominal(const FStandaloneShapeWorld& Shape
} }
inline void AppendFarConnectorShells(const FStandaloneShapeWorld& Shape,const FVoxelTileKey& Key, inline void AppendFarConnectorShells(const FStandaloneShapeWorld& Shape,const FVoxelTileKey& Key,
int32 Step,int32 Cells,FVoxelMeshData& Mesh) int32 Step,int32 Cells,FVoxelMeshData& Mesh,int32 StationScale=1,
uint32 RayMask=0x0fu,bool bReuseStationRadii=false)
{ {
if(Key.Level<6||Step<=0)return; if(Key.Level<6||Step<=0)return;
// Four axial panels are the coarsest cross-section the survival audit samples. // Four axial panels are the coarsest cross-section the survival audit samples.
@@ -313,7 +314,8 @@ inline void AppendFarConnectorShells(const FStandaloneShapeWorld& Shape,const FV
if(!bIntersects||TMax-TMin<1.0e-6)continue; if(!bIntersects||TMax-TMin<1.0e-6)continue;
// LOD7 and LOD8 share a 256-voxel connector station grid; near LOD keeps 128-voxel checks. // LOD7 and LOD8 share a 256-voxel connector station grid; near LOD keeps 128-voxel checks.
// LOD7 et LOD8 partagent des stations aux 256 voxels; pres du joueur, garder les pas de 128. // LOD7 et LOD8 partagent des stations aux 256 voxels; pres du joueur, garder les pas de 128.
const double ConnectorStationStep=Key.Level>=7?std::max(256.0,static_cast<double>(Step)):128.0; const double ConnectorStationStep=(Key.Level>=7?std::max(256.0,static_cast<double>(Step)):128.0)
*std::max(1,StationScale);
const int32 AxialSampleCount=std::max(1,static_cast<int32>(std::ceil(Length/ConnectorStationStep))); const int32 AxialSampleCount=std::max(1,static_cast<int32>(std::ceil(Length/ConnectorStationStep)));
std::vector<uint8> MissingMasks(static_cast<size_t>(AxialSampleCount),0); std::vector<uint8> MissingMasks(static_cast<size_t>(AxialSampleCount),0);
std::vector<std::array<double,4>> SourceRadii(static_cast<size_t>(AxialSampleCount)); std::vector<std::array<double,4>> SourceRadii(static_cast<size_t>(AxialSampleCount));
@@ -332,6 +334,7 @@ inline void AppendFarConnectorShells(const FStandaloneShapeWorld& Shape,const FV
const double DetectionTolerance=MatchTolerance*0.98; const double DetectionTolerance=MatchTolerance*0.98;
for(int32 DirectionIndex=0;DirectionIndex<4;++DirectionIndex) for(int32 DirectionIndex=0;DirectionIndex<4;++DirectionIndex)
{ {
if((RayMask&(1u<<DirectionIndex))==0)continue;
const FVector& Direction=RadialDirections[DirectionIndex]; const FVector& Direction=RadialDirections[DirectionIndex];
if(!ContainsFarTilePoint(Key,Center+Direction*RayLength))continue; if(!ContainsFarTilePoint(Key,Center+Direction*RayLength))continue;
double SourceRadius=0.0,SourceUncertainty=0.0; double SourceRadius=0.0,SourceUncertainty=0.0;
@@ -388,7 +391,13 @@ inline void AppendFarConnectorShells(const FStandaloneShapeWorld& Shape,const FV
{Ring.Radii[DirectionIndex]=Connector.Radius;continue;} {Ring.Radii[DirectionIndex]=Connector.Radius;continue;}
double Radius=Connector.Radius; double Radius=Connector.Radius;
double RadiusUncertainty=0.0; double RadiusUncertainty=0.0;
if(!FindFarConnectorRadiusNearNominal(Shape,Center, const int32 NearestStation=std::clamp(static_cast<int32>(std::lround(
T*AxialSampleCount-0.5)),0,AxialSampleCount-1);
const double ReusedRadius=bReuseStationRadii
?SourceRadii[static_cast<size_t>(NearestStation)][DirectionIndex]:0.0;
if(ReusedRadius>0.0)
Radius=ReusedRadius;
else if(!FindFarConnectorRadiusNearNominal(Shape,Center,
RadialDirections[DirectionIndex],Connector.Radius,Radius,RadiusUncertainty) RadialDirections[DirectionIndex],Connector.Radius,Radius,RadiusUncertainty)
&&!FindFarConnectorSurfaceRadius(Shape,Center,RadialDirections[DirectionIndex], &&!FindFarConnectorSurfaceRadius(Shape,Center,RadialDirections[DirectionIndex],
Connector.Radius,Radius)) Connector.Radius,Radius))
@@ -432,7 +441,8 @@ inline void AppendFarConnectorShells(const FStandaloneShapeWorld& Shape,const FV
} }
} }
std::vector<FRing> Rings; std::vector<FRing> Rings;
const double MaxRingStep=Key.Level>=7?std::max(256.0,static_cast<double>(Step)):128.0; const double MaxRingStep=(Key.Level>=7?std::max(256.0,static_cast<double>(Step)):128.0)
*std::max(1,StationScale);
double CurrentT=RunMin; double CurrentT=RunMin;
Rings.push_back(MakeRing(CurrentT,RunStart,RunEnd,RunRadiusMask)); Rings.push_back(MakeRing(CurrentT,RunStart,RunEnd,RunRadiusMask));
while(CurrentT<RunMax-1.0e-7) while(CurrentT<RunMax-1.0e-7)
@@ -814,6 +824,112 @@ inline void AppendFarConnectorShells(const FStandaloneShapeWorld& Shape,const FV
Mesh.NumSurfaceVertices=OldSurfaceVertices+AddedVertices; Mesh.NumSurfaceVertices=OldSurfaceVertices+AddedVertices;
} }
struct FFarConnectorCoverageAudit
{
std::uint64_t Rays=0;
std::uint64_t Holes=0;
std::uint64_t Floating=0;
std::vector<std::uint64_t> HoleIds;
std::vector<std::uint64_t> FloatingIds;
};
inline std::uint64_t FarConnectorAuditRayId(const FVoxelTileKey& Key,uint32 Connector,
int32 Station,int32 Direction)
{
std::uint64_t Hash=14695981039346656037ull;
const auto Add=[&](std::uint64_t Value,int Bytes)
{
for(int Byte=0;Byte<Bytes;++Byte)
{Hash^=static_cast<uint8>(Value>>(Byte*8));Hash*=1099511628211ull;}
};
Add(static_cast<uint32>(Key.Coord.X),4);Add(static_cast<uint32>(Key.Coord.Y),4);
Add(static_cast<uint32>(Key.Coord.Z),4);Add(static_cast<uint32>(Key.Level),4);
Add(Connector,4);Add(static_cast<uint32>(Station),4);Add(static_cast<uint32>(Direction),4);
return Hash?Hash:1;
}
inline FFarConnectorCoverageAudit AuditFarConnectorCoverage(const FStandaloneShapeWorld& Shape,
const FVoxelTileKey& Key,int32 Step,int32 Cells,const FVoxelMeshData& Mesh)
{
FFarConnectorCoverageAudit Audit;
if(Key.Level<6||Step<=0||Mesh.Triangles.Num()<3)return Audit;
const FIntVector TileOrigin=Key.OriginVoxels();
const int32 TileExtent=Key.ExtentVoxels();
FFarMeshSegmentIndex MeshIndex(Mesh,TileOrigin,TileExtent,Cells,Step);
const auto& Connectors=Shape.GetFarConnectorSegments();
std::vector<uint32> CandidateIndices;
Shape.FindFarConnectorCandidates(
FIntVector(TileOrigin.X-64,TileOrigin.Y-64,TileOrigin.Z-64),
FIntVector(TileOrigin.X+TileExtent+64,TileOrigin.Y+TileExtent+64,TileOrigin.Z+TileExtent+64),
CandidateIndices);
for(const uint32 ConnectorIndex:CandidateIndices)
{
if(ConnectorIndex>=Connectors.size())continue;
const auto& Connector=Connectors[ConnectorIndex];
const FVector Delta=Connector.End-Connector.Start;
const double Length=std::sqrt(Delta.X*Delta.X+Delta.Y*Delta.Y+Delta.Z*Delta.Z);
if(Length<=1.0e-6)continue;
const FVector Axis=Delta/Length;
FVector U=FVector::CrossProduct(Axis,FVector::UpVector);
if(U.SizeSquared()<0.01)U=FVector::CrossProduct(Axis,FVector::RightVector);
U.Normalize();
FVector V=FVector::CrossProduct(Axis,U);V.Normalize();
const FVector Directions[4]={U,U*-1.0,V,V*-1.0};
const double Deltas[3]={Delta.X,Delta.Y,Delta.Z};
const double Starts[3]={Connector.Start.X,Connector.Start.Y,Connector.Start.Z};
const double Low[3]={TileOrigin.X-Connector.Radius,TileOrigin.Y-Connector.Radius,
TileOrigin.Z-Connector.Radius};
const double High[3]={TileOrigin.X+TileExtent+Connector.Radius,
TileOrigin.Y+TileExtent+Connector.Radius,TileOrigin.Z+TileExtent+Connector.Radius};
double TMin=0.0,TMax=1.0;
bool bIntersects=true;
for(int32 AxisIndex=0;AxisIndex<3;++AxisIndex)
{
if(std::abs(Deltas[AxisIndex])<1.0e-9)
{if(Starts[AxisIndex]<Low[AxisIndex]||Starts[AxisIndex]>High[AxisIndex])bIntersects=false;continue;}
double A=(Low[AxisIndex]-Starts[AxisIndex])/Deltas[AxisIndex];
double B=(High[AxisIndex]-Starts[AxisIndex])/Deltas[AxisIndex];
if(A>B)std::swap(A,B);
TMin=std::max(TMin,A);TMax=std::min(TMax,B);
if(TMin>TMax){bIntersects=false;break;}
}
if(!bIntersects||TMax-TMin<1.0e-6)continue;
const double StationStep=Key.Level>=7?std::max(256.0,static_cast<double>(Step)):128.0;
const int32 StationCount=std::max(1,static_cast<int32>(std::ceil(Length/StationStep)));
const double RayLength=Connector.Radius*2.0+32.0;
const double MatchTolerance=std::max(8.0,static_cast<double>(Connector.Radius)*0.35);
const double DetectionTolerance=MatchTolerance*0.98;
const double SourceRadiusBand=std::max(16.0,static_cast<double>(Connector.Radius)*0.75);
for(int32 Station=0;Station<StationCount;++Station)
{
const FVector Center=Connector.Start+Delta*((static_cast<double>(Station)+0.5)/StationCount);
if(!ContainsFarTilePoint(Key,Center))continue;
for(int32 DirectionIndex=0;DirectionIndex<4;++DirectionIndex)
{
if(!ContainsFarTilePoint(Key,Center+Directions[DirectionIndex]*RayLength))continue;
double SourceRadius=0.0,Uncertainty=0.0;
const bool bSource=FindFarConnectorRadiusNearNominal(Shape,Center,
Directions[DirectionIndex],Connector.Radius,SourceRadius,Uncertainty)
||FindFarConnectorSurfaceRadius(Shape,Center,Directions[DirectionIndex],
Connector.Radius,SourceRadius);
const double MeshDifference=MeshIndex.NearestHitDifference(Center,
Directions[DirectionIndex],RayLength,bSource?SourceRadius:Connector.Radius);
if(bSource&&std::abs(SourceRadius-Connector.Radius)<=SourceRadiusBand)
{
++Audit.Rays;
if(!std::isfinite(MeshDifference)||MeshDifference>DetectionTolerance)
Audit.HoleIds.push_back(FarConnectorAuditRayId(Key,ConnectorIndex,Station,DirectionIndex));
}
else if(std::isfinite(MeshDifference)&&MeshDifference<=DetectionTolerance)
Audit.FloatingIds.push_back(FarConnectorAuditRayId(Key,ConnectorIndex,Station,DirectionIndex));
}
}
}
Audit.Holes=Audit.HoleIds.size();
Audit.Floating=Audit.FloatingIds.size();
return Audit;
}
struct FFarMeshColumnIndex struct FFarMeshColumnIndex
{ {
FFarMeshColumnIndex(const FVoxelMeshData& InMesh,const FIntVector& InOrigin,int32 InExtent, FFarMeshColumnIndex(const FVoxelMeshData& InMesh,const FIntVector& InOrigin,int32 InExtent,
Standalone/Core/src/VoxelForgeCore.cpp
+61 -13
View File
@@ -343,7 +343,7 @@ namespace VoxelForgeCore
FNativeCapRefinementStats RefineFarNativeGround(const FStandaloneOwnerWorld& Owner, FNativeCapRefinementStats RefineFarNativeGround(const FStandaloneOwnerWorld& Owner,
const WorldAssemblyManifest& Manifest,const FVoxelTileKey& Key,int32 MeshStepVoxels, const WorldAssemblyManifest& Manifest,const FVoxelTileKey& Key,int32 MeshStepVoxels,
int32 InteriorStepOverride,FVoxelMeshData& Mesh) int32 InteriorStepOverride,int32 RefinementGridScale,FVoxelMeshData& Mesh)
{ {
FNativeCapRefinementStats Stats; FNativeCapRefinementStats Stats;
if(Key.Level<5||!Owner.Generator)return Stats; if(Key.Level<5||!Owner.Generator)return Stats;
@@ -354,11 +354,12 @@ namespace VoxelForgeCore
const int32 RequestedStep=Key.Level==8 const int32 RequestedStep=Key.Level==8
?(InteriorStepOverride>0?InteriorStepOverride:MeshStepVoxels) ?(InteriorStepOverride>0?InteriorStepOverride:MeshStepVoxels)
:std::min(BoundaryStepVoxels,1<<Key.Level); :std::min(BoundaryStepVoxels,1<<Key.Level);
const int32 ScaledRequestedStep=RequestedStep*std::max(1,RefinementGridScale);
const int32 GridStepVoxels=Key.Level==8 const int32 GridStepVoxels=Key.Level==8
?((RequestedStep<BoundaryStepVoxels||RequestedStep%BoundaryStepVoxels!=0 ?((ScaledRequestedStep<BoundaryStepVoxels||ScaledRequestedStep%BoundaryStepVoxels!=0
||TileExtent<=0||TileExtent%RequestedStep!=0) ||TileExtent<=0||TileExtent%ScaledRequestedStep!=0)
?BoundaryStepVoxels:std::max(BoundaryStepVoxels,RequestedStep)) ?BoundaryStepVoxels:std::max(BoundaryStepVoxels,ScaledRequestedStep))
:RequestedStep; :ScaledRequestedStep;
if(TileExtent<=0||TileExtent%GridStepVoxels!=0)return Stats; if(TileExtent<=0||TileExtent%GridStepVoxels!=0)return Stats;
const std::vector<int32> GridCoordinates= const std::vector<int32> GridCoordinates=
BuildNativeHorizonGridCoordinates(TileExtent,GridStepVoxels); BuildNativeHorizonGridCoordinates(TileExtent,GridStepVoxels);
@@ -644,7 +645,7 @@ namespace VoxelForgeCore
FNativeCapRefinementStats RefineFarNativeCap(const FStandaloneOwnerWorld& Owner, FNativeCapRefinementStats RefineFarNativeCap(const FStandaloneOwnerWorld& Owner,
const WorldAssemblyManifest& Manifest,const FVoxelTileKey& Key,int32 MeshStepVoxels, const WorldAssemblyManifest& Manifest,const FVoxelTileKey& Key,int32 MeshStepVoxels,
int32 InteriorStepOverride,FVoxelMeshData& Mesh) int32 InteriorStepOverride,int32 RefinementGridScale,FVoxelMeshData& Mesh)
{ {
FNativeCapRefinementStats Stats; FNativeCapRefinementStats Stats;
if(Key.Level<4||!Owner.Generator)return Stats; if(Key.Level<4||!Owner.Generator)return Stats;
@@ -654,11 +655,12 @@ namespace VoxelForgeCore
:(Key.Level<5?std::min(BoundaryStepVoxels,1<<Key.Level):BoundaryStepVoxels); :(Key.Level<5?std::min(BoundaryStepVoxels,1<<Key.Level):BoundaryStepVoxels);
const FIntVector TileOrigin=Key.OriginVoxels(); const FIntVector TileOrigin=Key.OriginVoxels();
const int32 TileExtent=Key.ExtentVoxels(); const int32 TileExtent=Key.ExtentVoxels();
const int32 ScaledRequestedStep=RequestedStep*std::max(1,RefinementGridScale);
const int32 EffectiveGridStep=Key.Level==8 const int32 EffectiveGridStep=Key.Level==8
?((RequestedStep<BoundaryStepVoxels||RequestedStep%BoundaryStepVoxels!=0 ?((ScaledRequestedStep<BoundaryStepVoxels||ScaledRequestedStep%BoundaryStepVoxels!=0
||TileExtent<=0||TileExtent%RequestedStep!=0) ||TileExtent<=0||TileExtent%ScaledRequestedStep!=0)
?BoundaryStepVoxels:std::max(BoundaryStepVoxels,RequestedStep)) ?BoundaryStepVoxels:std::max(BoundaryStepVoxels,ScaledRequestedStep))
:RequestedStep; :ScaledRequestedStep;
constexpr int64 Scale=WorldAssemblySubvoxelsPerVoxel; constexpr int64 Scale=WorldAssemblySubvoxelsPerVoxel;
if(TileExtent<=0||TileExtent%EffectiveGridStep!=0)return Stats; if(TileExtent<=0||TileExtent%EffectiveGridStep!=0)return Stats;
const std::vector<int32> GridCoordinates= const std::vector<int32> GridCoordinates=
@@ -1606,6 +1608,37 @@ namespace VoxelForgeCore
options.legacyDirectMappedColumnCache||key.lod<=2); options.legacyDirectMappedColumnCache||key.lod<=2);
if(!impl_||!impl_->owner) if(!impl_||!impl_->owner)
{ error="resolved world snapshot is empty";return std::nullopt; } { error="resolved world snapshot is empty";return std::nullopt; }
const WorldAssemblyRecipeV1* FarRecipe=impl_->worldAssembly?&impl_->worldAssembly->recipe:nullptr;
std::uint32_t FarDisplayBudget=FarRecipe?FarRecipe->farDisplayGeometryBudgetMillipixels:0;
std::uint32_t FarRefinementScale=FarRecipe?FarRecipe->farRefinementGridScale:0;
std::uint32_t FarRefinementLodMask=FarRecipe?FarRecipe->farRefinementLodMask:0;
std::uint32_t FarConnectorStationScale=FarRecipe?FarRecipe->farConnectorStationScale:0;
std::uint32_t FarConnectorRayMask=FarRecipe?FarRecipe->farConnectorRayMask:0;
bool bFarConnectorReuseStationRadii=FarRecipe&&FarRecipe->farConnectorReuseStationRadii;
if(options.overrideFarDisplayMesh)
{
FarDisplayBudget=options.farDisplayGeometryBudgetMillipixels;
FarRefinementScale=options.farRefinementGridScale;
FarRefinementLodMask=options.farRefinementLodMask;
FarConnectorStationScale=options.farConnectorStationScale;
FarConnectorRayMask=options.farConnectorRayMask;
bFarConnectorReuseStationRadii=options.farConnectorReuseStationRadii;
}
const auto ValidFarScale=[](std::uint32_t Value)
{return Value==0||Value==2||Value==4||Value==8;};
if((FarDisplayBudget!=0&&FarDisplayBudget!=1000)
||!ValidFarScale(FarRefinementScale)
||!ValidFarScale(FarConnectorStationScale)
||(FarRefinementLodMask&~0x1f0u)!=0||FarConnectorRayMask>0x0fu
||(FarRefinementScale!=0&&FarRefinementLodMask==0))
{error="far display mesh override is outside the recipe's supported 1 px settings";return std::nullopt;}
const std::uint32_t ActiveFarRefinementScale=FarDisplayBudget==1000
&&(FarRefinementLodMask&(1u<<std::clamp(key.lod,0,31)))!=0
?FarRefinementScale:0;
const int32 ActiveConnectorStationScale=FarDisplayBudget==1000
&&FarConnectorStationScale!=0?static_cast<int32>(FarConnectorStationScale):1;
const uint32 ActiveConnectorRayMask=FarDisplayBudget==1000
&&FarConnectorRayMask!=0?FarConnectorRayMask:0x0fu;
const bool bShapeReady=impl_->shapeReady.load(std::memory_order_acquire); const bool bShapeReady=impl_->shapeReady.load(std::memory_order_acquire);
const bool bNativeOnly=!bShapeReady||options.diagnosticForceNativeOnly; const bool bNativeOnly=!bShapeReady||options.diagnosticForceNativeOnly;
if(!bShapeReady&&!options.diagnosticForceNativeOnly&&!isTileShapeFree(key)) if(!bShapeReady&&!options.diagnosticForceNativeOnly&&!isTileShapeFree(key))
@@ -2135,7 +2168,10 @@ namespace VoxelForgeCore
BeforeConnectorColumnComputeMilliseconds=Columns.columnComputeMilliseconds; BeforeConnectorColumnComputeMilliseconds=Columns.columnComputeMilliseconds;
} }
const auto ConnectorStart=std::chrono::steady_clock::now(); const auto ConnectorStart=std::chrono::steady_clock::now();
AppendFarConnectorShells(farShape,nativeKey,step,world.CoarseTileCells,mesh); if(!options.diagnosticRemoveFarConnectorPanels)
AppendFarConnectorShells(farShape,nativeKey,step,world.CoarseTileCells,mesh,
ActiveConnectorStationScale,ActiveConnectorRayMask,
FarDisplayBudget==1000&&bFarConnectorReuseStationRadii);
const auto AfterConnector=FStandaloneShapeWorld::GetThreadClassificationStats(); const auto AfterConnector=FStandaloneShapeWorld::GetThreadClassificationStats();
out.diagnostics.farConnectorShellMilliseconds=std::chrono::duration<double,std::milli>( out.diagnostics.farConnectorShellMilliseconds=std::chrono::duration<double,std::milli>(
std::chrono::steady_clock::now()-ConnectorStart).count(); std::chrono::steady_clock::now()-ConnectorStart).count();
@@ -2161,6 +2197,16 @@ namespace VoxelForgeCore
} }
out.diagnostics.farShellTrianglesAdded+=static_cast<std::uint64_t>( out.diagnostics.farShellTrianglesAdded+=static_cast<std::uint64_t>(
std::max(0,mesh.Triangles.Num()/3-BeforeConnectorTriangles)); std::max(0,mesh.Triangles.Num()/3-BeforeConnectorTriangles));
if(options.diagnosticAuditFarConnectorCoverage)
{
const auto Audit=AuditFarConnectorCoverage(farShape,nativeKey,step,
world.CoarseTileCells,mesh);
out.diagnostics.farConnectorAuditRays=Audit.Rays;
out.diagnostics.farConnectorAuditHoles=Audit.Holes;
out.diagnostics.farConnectorAuditFloating=Audit.Floating;
out.diagnostics.farConnectorAuditHoleIds=Audit.HoleIds;
out.diagnostics.farConnectorAuditFloatingIds=Audit.FloatingIds;
}
} }
out.diagnostics.farShellMilliseconds=std::chrono::duration<double,std::milli>( out.diagnostics.farShellMilliseconds=std::chrono::duration<double,std::milli>(
std::chrono::steady_clock::now()-FarShellStart).count(); std::chrono::steady_clock::now()-FarShellStart).count();
@@ -2168,12 +2214,14 @@ namespace VoxelForgeCore
{ {
const auto HorizonStart=std::chrono::steady_clock::now(); const auto HorizonStart=std::chrono::steady_clock::now();
FNativeCapRefinementStats GroundStats; FNativeCapRefinementStats GroundStats;
const int32 ActiveRefinementScale=ActiveFarRefinementScale>1
?static_cast<int32>(ActiveFarRefinementScale):1;
const double ColumnMillisecondsBeforeGround= const double ColumnMillisecondsBeforeGround=
VoxelForgeCore::Heightfield2D::threadColumnCacheStats().columnComputeMilliseconds; VoxelForgeCore::Heightfield2D::threadColumnCacheStats().columnComputeMilliseconds;
const auto GroundStart=std::chrono::steady_clock::now(); const auto GroundStart=std::chrono::steady_clock::now();
if(key.lod>=4) if(key.lod>=4)
GroundStats=RefineFarNativeGround(world,*impl_->worldAssembly,nativeKey,step, GroundStats=RefineFarNativeGround(world,*impl_->worldAssembly,nativeKey,step,
options.lod8HorizonInteriorStepVoxels,mesh); options.lod8HorizonInteriorStepVoxels,ActiveRefinementScale,mesh);
const double GroundWallMilliseconds=std::chrono::duration<double,std::milli>( const double GroundWallMilliseconds=std::chrono::duration<double,std::milli>(
std::chrono::steady_clock::now()-GroundStart).count(); std::chrono::steady_clock::now()-GroundStart).count();
const double ColumnMillisecondsAfterGround= const double ColumnMillisecondsAfterGround=
@@ -2186,7 +2234,7 @@ namespace VoxelForgeCore
VoxelForgeCore::Heightfield2D::threadColumnCacheStats().columnComputeMilliseconds; VoxelForgeCore::Heightfield2D::threadColumnCacheStats().columnComputeMilliseconds;
const auto CapStart=std::chrono::steady_clock::now(); const auto CapStart=std::chrono::steady_clock::now();
const auto CapStats=RefineFarNativeCap(world,*impl_->worldAssembly,nativeKey,step, const auto CapStats=RefineFarNativeCap(world,*impl_->worldAssembly,nativeKey,step,
options.lod8HorizonInteriorStepVoxels,mesh); options.lod8HorizonInteriorStepVoxels,ActiveRefinementScale,mesh);
const double CapWallMilliseconds=std::chrono::duration<double,std::milli>( const double CapWallMilliseconds=std::chrono::duration<double,std::milli>(
std::chrono::steady_clock::now()-CapStart).count(); std::chrono::steady_clock::now()-CapStart).count();
const double ColumnMillisecondsAfterCap= const double ColumnMillisecondsAfterCap=
Standalone/Core/src/WorldAssemblerInternal.h
+46
View File
@@ -244,6 +244,24 @@ namespace world_assembly_detail
|| recipe.biomeBorderBlendSubvoxels < 0 || recipe.biomeBorderBlendSubvoxels < 0
|| recipe.biomeBorderBlendSubvoxels > recipe.biomeCellSizeSubvoxels) || recipe.biomeBorderBlendSubvoxels > recipe.biomeCellSizeSubvoxels)
{ error = "world assembly biome-map parameters are outside the supported fixed-scale range"; return false; } { error = "world assembly biome-map parameters are outside the supported fixed-scale range"; return false; }
if (recipe.evaluatorVersion >= 7)
{
constexpr std::uint32_t FarLodMask = 0x1f0u; // LOD4..8.
const bool anyFarSetting = recipe.farDisplayGeometryBudgetMillipixels != 0
|| recipe.farRefinementGridScale != 0 || recipe.farRefinementLodMask != 0
|| recipe.farConnectorStationScale != 0 || recipe.farConnectorRayMask != 0
|| recipe.farConnectorReuseStationRadii;
const auto validScale = [](std::uint32_t value)
{ return value == 0 || value == 2 || value == 4 || value == 8; };
if ((anyFarSetting && recipe.farDisplayGeometryBudgetMillipixels != 1000)
|| recipe.farDisplayGeometryBudgetMillipixels > 1000
|| !validScale(recipe.farRefinementGridScale)
|| (recipe.farRefinementLodMask & ~FarLodMask) != 0
|| !validScale(recipe.farConnectorStationScale)
|| recipe.farConnectorRayMask > 0x0fu
|| (recipe.farRefinementGridScale != 0 && recipe.farRefinementLodMask == 0))
{ error = "far display mesh declaration must use the 1 px budget and supported LOD, scale, and ray values"; return false; }
}
std::vector<std::uint32_t> ids; std::vector<std::uint32_t> ids;
ids.reserve(recipe.biomes.size()); ids.reserve(recipe.biomes.size());
for (const auto& biome : recipe.biomes) for (const auto& biome : recipe.biomes)
@@ -431,6 +449,16 @@ namespace world_assembly_detail
HashWord(hash, static_cast<std::uint64_t>(recipe.biomeClimateMoistureWavelengthSubvoxels), 8); HashWord(hash, static_cast<std::uint64_t>(recipe.biomeClimateMoistureWavelengthSubvoxels), 8);
HashWord(hash, static_cast<std::uint64_t>(recipe.biomeClimateReliefContrastQ16), 8); HashWord(hash, static_cast<std::uint64_t>(recipe.biomeClimateReliefContrastQ16), 8);
} }
if (recipe.evaluatorVersion >= 7)
{
HashWord(hash, 0x4641524449535037ull, 8);
HashWord(hash, recipe.farDisplayGeometryBudgetMillipixels, 4);
HashWord(hash, recipe.farRefinementGridScale, 4);
HashWord(hash, recipe.farRefinementLodMask, 4);
HashWord(hash, recipe.farConnectorStationScale, 4);
HashWord(hash, recipe.farConnectorRayMask, 4);
HashWord(hash, recipe.farConnectorReuseStationRadii ? 1u : 0u, 1);
}
HashWord(hash, recipe.biomes.size(), 4); HashWord(hash, recipe.biomes.size(), 4);
for (const auto& biome : recipe.biomes) for (const auto& biome : recipe.biomes)
{ {
@@ -641,6 +669,16 @@ namespace world_assembly_detail
|| !ReadInteger(value.Get("climate_moisture_wavelength_q"), recipe.biomeClimateMoistureWavelengthSubvoxels) || !ReadInteger(value.Get("climate_moisture_wavelength_q"), recipe.biomeClimateMoistureWavelengthSubvoxels)
|| !ReadInteger(value.Get("climate_relief_contrast_q16"), recipe.biomeClimateReliefContrastQ16))) || !ReadInteger(value.Get("climate_relief_contrast_q16"), recipe.biomeClimateReliefContrastQ16)))
{ error = "world assembly recipe is missing evaluator-2 climate parameters"; return false; } { error = "world assembly recipe is missing evaluator-2 climate parameters"; return false; }
if (recipe.evaluatorVersion >= 7
&& (!ReadU32(value.Get("far_display_budget_millipixels"), recipe.farDisplayGeometryBudgetMillipixels)
|| !ReadU32(value.Get("far_refinement_grid_scale"), recipe.farRefinementGridScale)
|| !ReadU32(value.Get("far_refinement_lod_mask"), recipe.farRefinementLodMask)
|| !ReadU32(value.Get("far_connector_station_scale"), recipe.farConnectorStationScale)
|| !ReadU32(value.Get("far_connector_ray_mask"), recipe.farConnectorRayMask)
|| !std::holds_alternative<bool>(value.Get("far_connector_reuse_station_radii").Data)))
{ error = "world assembly recipe is missing evaluator-7 far display mesh parameters"; return false; }
if (recipe.evaluatorVersion >= 7)
recipe.farConnectorReuseStationRadii = std::get<bool>(value.Get("far_connector_reuse_station_radii").Data);
for (const auto& item : value.Get("biomes").AsArray()) for (const auto& item : value.Get("biomes").AsArray())
{ {
WorldAssemblyBiomeProfileV1 biome; WorldAssemblyBiomeProfileV1 biome;
@@ -702,6 +740,14 @@ namespace world_assembly_detail
out << ",\"climate_relief_wavelength_q\":" << recipe.biomeClimateReliefWavelengthSubvoxels out << ",\"climate_relief_wavelength_q\":" << recipe.biomeClimateReliefWavelengthSubvoxels
<< ",\"climate_moisture_wavelength_q\":" << recipe.biomeClimateMoistureWavelengthSubvoxels << ",\"climate_moisture_wavelength_q\":" << recipe.biomeClimateMoistureWavelengthSubvoxels
<< ",\"climate_relief_contrast_q16\":" << recipe.biomeClimateReliefContrastQ16; << ",\"climate_relief_contrast_q16\":" << recipe.biomeClimateReliefContrastQ16;
if (recipe.evaluatorVersion >= 7)
out << ",\"far_display_budget_millipixels\":" << recipe.farDisplayGeometryBudgetMillipixels
<< ",\"far_refinement_grid_scale\":" << recipe.farRefinementGridScale
<< ",\"far_refinement_lod_mask\":" << recipe.farRefinementLodMask
<< ",\"far_connector_station_scale\":" << recipe.farConnectorStationScale
<< ",\"far_connector_ray_mask\":" << recipe.farConnectorRayMask
<< ",\"far_connector_reuse_station_radii\":"
<< (recipe.farConnectorReuseStationRadii ? "true" : "false");
out << ",\"content_hash\":\"" << HashText(recipe.contentHash) << "\",\"biomes\":["; out << ",\"content_hash\":\"" << HashText(recipe.contentHash) << "\",\"biomes\":[";
for (std::size_t i = 0; i < recipe.biomes.size(); ++i) for (std::size_t i = 0; i < recipe.biomes.size(); ++i)
{ {
Standalone/Research/WorldAssemblerM3/Linux16/MeasureZstd.py
-88
View File
@@ -1,88 +0,0 @@
#!/usr/bin/env python3
"""Per-tile zstd CLI comparison over the pristine L1 records from this round."""
from pathlib import Path
import shutil
import struct
import subprocess
import time
import zlib
REPO = Path.cwd().resolve()
CACHE = REPO / "Saved/Linux/cache/l16-prototype"
CAPTURE_LOG = REPO / "Saved/Linux/l16-cache-l1-q24-full-write.log"
TEMP = CACHE / "codec-zstd"
RESULT = REPO / "Saved/Linux/l16-zstd-codec.txt"
assert CACHE.is_dir() and not CACHE.is_symlink()
assert CACHE.resolve().parent == (REPO / "Saved/Linux/cache").resolve()
assert CAPTURE_LOG.is_file()
assert not TEMP.exists()
cutoff_ns = CAPTURE_LOG.stat().st_mtime_ns
records = [
path for world in CACHE.glob("world_*")
for path in world.glob("*.vftile")
if "_s0_d0000000000000000_" in path.name and path.stat().st_mtime_ns <= cutoff_ns
]
records.sort()
assert len(records) == 2553, f"expected 2553 L1 pristine records, found {len(records)}"
codec = shutil.which("zstd")
assert codec, "zstd CLI is not available"
TEMP.mkdir()
per_lod = {}
try:
for source in records:
record = source.read_bytes()
assert record[:8] == b"VFTILE16"
raw_size, compressed_size = struct.unpack_from("<QQ", record, 124)
raw = zlib.decompress(record[144:144 + compressed_size])
assert len(raw) == raw_size
lod = int(source.name.split("_", 1)[0][3:])
entry = per_lod.setdefault(lod, {"raw": 0, "files": [], "encode_ns": 0, "decode_ns": 0, "zstd": 0})
raw_file = TEMP / (source.stem + ".raw")
raw_file.write_bytes(raw)
entry["raw"] += len(raw)
entry["files"].append(raw_file)
for lod in sorted(per_lod):
entry = per_lod[lod]
start = time.perf_counter_ns()
subprocess.run([codec, "-q", "-1", "-f", "--", *map(str, entry["files"])], check=True)
entry["encode_ns"] = time.perf_counter_ns() - start
compressed_files = [Path(str(path) + ".zst") for path in entry["files"]]
entry["zstd"] = sum(path.stat().st_size for path in compressed_files)
start = time.perf_counter_ns()
subprocess.run([codec, "-q", "-d", "-f", "--", *map(str, compressed_files)], check=True)
entry["decode_ns"] = time.perf_counter_ns() - start
# Verify each decoded output byte-for-byte against the zlib payload again.
# The filename is the tile stem, so a deterministic source lookup is sufficient.
source_by_stem = {path.stem: path for path in records}
for entry in per_lod.values():
for raw_file in entry["files"]:
source = source_by_stem[raw_file.stem]
data = source.read_bytes()
_, compressed_size = struct.unpack_from("<QQ", data, 124)
assert raw_file.read_bytes() == zlib.decompress(data[144:144 + compressed_size])
lines = ["lod tiles raw_total_B zstd1_total_B zstd1_avg_B encode_us_per_tile decode_us_per_tile status"]
for lod in sorted(per_lod):
entry = per_lod[lod]
count = len(entry["files"])
lines.append(
f"{lod} {count} {entry['raw']} {entry['zstd']} {entry['zstd'] / count:.1f} "
f"{entry['encode_ns'] / count / 1000:.2f} {entry['decode_ns'] / count / 1000:.2f} PASS"
)
raw_total = sum(entry["raw"] for entry in per_lod.values())
zstd_total = sum(entry["zstd"] for entry in per_lod.values())
lines.append(f"total_tiles={len(records)} raw_bytes={raw_total} zstd1_bytes={zstd_total} ratio={zstd_total / raw_total:.6f}")
lines.append("decode=byte-exact; timing includes zstd CLI process startup once per LOD")
RESULT.write_text("\n".join(lines) + "\n")
print("\n".join(lines))
finally:
resolved = TEMP.resolve()
assert resolved.parent == CACHE.resolve()
shutil.rmtree(TEMP)
Standalone/Research/WorldAssemblerM3/Linux16/REPORT.md
-232
View File
@@ -1,232 +0,0 @@
# Linux16 — terrain cache design and Core prototype
This is a persistence feature. The measurements judge exactness, storage, and worker responsiveness;
first-time terrain generation remains a separate cost. The prototype is headless and opt-in. Unreal
integration remains for the owner's PC.
## Design
- **Cache identity.** Production keys should include a stable world ID; season ID and canonical
season-content hash; seed and RNG contract; recipe, plan, biome/manifest and owner-content hashes;
schema and `WorldAssemblyEvaluatorVersion` (6 here); tile coordinate and LOD; voxel size and tile
dimensions; generation options; material/payload-section schema; mesher contract; and a generator
build/FP-contract identity. Include a platform or compiler identity unless cross-build output has
been proven equivalent. Clang and GCC matched this round's full hash matrix, but that alone does
not make every future build interchangeable. Use canonical content hashes so whitespace or JSON
key order does not change the world identity.
- **Prototype identity.** The header and file name include the raw owner-snapshot hash, Core
configuration, recipe/plan/manifest hashes, seed, schema, evaluator, voxel size, tile size,
mesher/FP contract, generation-options hash, tile key and LOD. The edited-tile probe also includes
edit sequence and tile edit digest. The prototype uses 64-bit FNV for identity inputs and CRC32
for accidental file corruption. Production should use a 256-bit content hash and a stable world
ID; the prototype hashes are evidence machinery, not a security boundary.
- **Edits.** Keep the reusable cache pristine. On load, apply the authoritative versioned diff layer
in sequence, then build the edited render/collision result. A derived edited-tile cache, if useful,
must key on the last applied sequence and a digest of edits intersecting that tile. A newer edit
invalidates only derived entries for touched tiles; it leaves pristine entries valid. The edit
probe wrote and reread sequence 1 with digest `3cd5603ebf707d0c` and matched every array and hash.
- **Server.** Never accept a client's cached mesh or collision as authoritative. The server may use
its own locally generated cache for pristine meshes, then apply its ordered edit layer and build
collision from that result. Client cache contents never cross the network; only validated edits
do. A server restart can reuse its own cache after identity and payload validation.
- **Invalidation and verification.** Season/content, seed/RNG, recipe, plan, evaluator, generator,
mesher, FP, voxel-grid, LOD, options, or payload-schema changes select a different key. Evaluator
bump and edit-sequence bump were rejected by the prototype. A canonical uncompressed-payload hash
should be stored and verified at load; corruption or a mismatch is a cache miss, followed by
regeneration on a worker. The prototype verifies CRCs and structure on every load, compares the
complete mesh arrays and Core D/M/P/T hashes after every generated write/read, and regenerates a
deliberately corrupted cached tile. This proves the sampled write/read route; a file's own hash
cannot independently prove generator semantics on every later hit, so complete identity and
version discipline remain essential.
- **Stored data.** The prototype stores the final Core mesh: exact double positions/normals, float
UVs, RGBA, section/group counts, indices, skirt masks, density sample count, and D/M/P/T hashes.
This avoids remeshing on a hit and maps directly to future RMC sections. The indices and section
ranges can feed server-side collision construction after edits. Density/sign tiles offer more
freedom for future meshers but require meshing, materials, and collision work on load; density
sizes were not benchmarked here. Keep collision ownership server-side in either design.
- **Worker/smoothness path.** Cache reads, zlib decode, integrity checks, serialization, and writes
run in the existing headless Core tile worker/result route. The Unreal game thread was not
present on this machine; its later integration should submit cache work through the same worker
queue and return the ordinary tile result. The prototype uses that route for all recipes and
changes no recipe implementation. It exercises SurfaceWorld for cache round trips; the cache-off
matrix and route audits cover SurfaceWorld and FloatingIslands.
- **Read versus regenerate.** Track per-LOD generation and load/decode EWMA. If a demanded tile's
estimated read/decode time exceeds generation time, schedule generation directly; if an attempted
load fails, discard it and regenerate without blocking the result queue. Keep the same tile
priority and permit the previous coarser LOD to remain visible while work finishes. On this L1
set, warm read/decode averaged about 0.11 ms per LOD0 tile and 0.91–1.61 ms per LOD8/7 tile,
compared with 3.97 ms LOD0 and 12.89–24.24 ms LOD8/6 mesh generation in the instrumented fill.
- **Layout, crash safety, and eviction.** One file per tile under a world-content directory keeps
reads and eviction independent; the prototype has 2,553 L1 records and 2,986 touched records after
the 24 m crossing. It writes a same-directory `.tmp`, flushes and `fsync`s it, atomically renames
to `.vftile`, then syncs the directory. Readers only open final `.vftile` names and reject bad
lengths, versions, identities, CRCs, counts, indices, and padding. A crash before rename leaves a
non-loadable temp file. Production startup should remove old temp files. Use a global 2 GB cap
with an LRU index by last use, protect entries currently in use, reserve space under one lock,
evict oldest unpinned records before writes, and skip the write if space still cannot be made.
The prototype enforces the cap and skips writes when full; it does not implement LRU eviction.
Its 4 KiB padding enabled `O_DIRECT` measurements and can be removed in production buffered I/O.
## Compression measurements
Available system zlib headers/library (zlib 1.3.1) were linked with `-lz`. The existing zstd 1.5.7
CLI had no development headers, so it was tested as a per-tile CLI codec in LOD batches; its times
include one process start per LOD. No installs or system changes were made. All totals below cover
all 2,553 L1 tiles except Q16, whose first run had five route-audit tiles prepopulated before codec
sampling. Sizes are payload bytes; the lossless cache also has headers and per-file 4 KiB padding.
| Encoding | Total bytes | Relative to raw | Exact Core arrays? | L1 gate |
|---|---:|---:|---|---|
| Raw canonical mesh | 107,342,349 | 100% | Yes | Pass |
| zlib level 6 | 47,388,401 | 44.15% | Yes | Pass |
| zstd level 1 (CLI, per tile) | 46,513,479 | 43.33% | Yes | Pass; 1.85% smaller than zlib payload |
| Q16 positions + oct16 normals + delta-varint indices | 38,672,768 (2,548 tiles sampled) | 36.17% | No | Fail: 11.4482 px at LOD0; 6 degeneracy changes (2 at LOD4, 4 at LOD6) |
| Q24 positions + oct16 normals + delta-varint indices | 42,862,996 | 39.94% | No | Pass: 0.0458 px max, 0.000060 Lambert delta max, 0.00361° normal error max, no winding/degeneracy changes |
Q24 is about 9.5% smaller than zlib's payload, but it changes floating-point positions/normals and
cannot satisfy exact loaded-versus-regenerated hashes. The selected cache path is lossless zlib.
The zstd CLI decoded every tile byte-for-byte, and was modestly smaller/faster in this test. There
are no zstd development headers here, so it was not linked into the worker prototype. The L1 zlib
records occupied 55,685,120 bytes on disk, including record framing and page padding.
Per-tile size averages by LOD:
| LOD | Tiles | Raw B | zlib B | zstd B | Q16 B* | Q24 B |
|---:|---:|---:|---:|---:|---:|---:|
| 0 | 512 | 19,232.6 | 7,901.7 | 8,009.4 | 7,115.2 | 7,797.3 |
| 1 | 448 | 15,437.8 | 6,715.2 | 6,715.6 | 5,394.2 | 6,244.9 |
| 2 | 448 | 11,152.2 | 4,452.4 | 4,516.8 | 4,129.9 | 4,548.7 |
| 3 | 384 | 11,750.3 | 4,813.1 | 4,885.5 | 4,361.7 | 4,801.7 |
| 4 | 320 | 49,186.9 | 18,865.3 | 18,352.3 | 17,858.1 | 19,738.2 |
| 5 | 144 | 83,106.3 | 37,693.0 | 37,004.3 | 30,247.6 | 33,494.7 |
| 6 | 96 | 128,102.3 | 59,624.7 | 56,521.2 | 46,390.6 | 51,437.7 |
| 7 | 48 | 308,206.8 | 141,387.7 | 139,348.3 | 110,361.7 | 121,997.0 |
| 8 | 153 | 171,712.3 | 81,805.0 | 79,662.9 | 60,755.8 | 67,046.4 |
*Q16 averages are over the 2,548 newly generated tiles; five prepopulated tiles did not enter its
codec sample. Raw, zlib, and Q24 averages are from the clean 2,553-tile fill.
Average codec encode/decode time per tile, microseconds:
| LOD | Raw E/D | zlib E/D | zstd CLI E/D | Q24 mesh E/D |
|---:|---:|---:|---:|---:|
| 0 | 38.60 / 12.31 | 736.63 / 79.49 | 72.14 / 47.97 | 16.28 / 18.78 |
| 1 | 30.13 / 9.47 | 608.30 / 68.07 | 62.71 / 40.53 | 14.26 / 16.04 |
| 2 | 18.94 / 4.64 | 286.44 / 37.86 | 48.50 / 34.65 | 8.20 / 7.55 |
| 3 | 20.21 / 5.18 | 313.54 / 41.77 | 50.73 / 35.90 | 8.81 / 8.07 |
| 4 | 90.46 / 27.53 | 1,508.23 / 189.40 | 134.50 / 78.66 | 41.58 / 41.93 |
| 5 | 163.48 / 55.53 | 2,935.24 / 370.92 | 220.23 / 140.52 | 76.90 / 79.86 |
| 6 | 254.98 / 96.01 | 4,900.50 / 611.07 | 361.50 / 184.35 | 127.39 / 136.19 |
| 7 | 621.57 / 211.84 | 11,944.70 / 1,475.82 | 861.84 / 442.64 | 327.72 / 391.27 |
| 8 | 325.34 / 109.80 | 6,478.56 / 784.32 | 486.54 / 289.96 | 167.25 / 186.01 |
## Smoothness, prototype, and budgets
- A clean write generated, losslessly stored, reloaded, and exactly compared all 2,553 L1 tiles.
The whole-view D/M/P/T hash matched L14. The warm run hit 2,553/2,553 tiles, returned the same
hash, and reached whole-view-ready in 0.450 s including 0.193 s of shape setup; tile streaming
took 0.243 s.
- The 24 m crossing covered 432 entering tiles. Its D/M/P/T hash was
`337ca62131576728 / 7612f48e2ddbbaab / 69a67ed4add83b87 / f11ac1ebd0f7ddb8`, equal to L14.
The cache route audit passed 9 pristine pairs, one edited pair, and one movement-history pair,
all with exact arrays and zero D/M/P/T mismatches. The crossing fill generated and round-tripped
428 tiles; four had already been created by the route audit.
- Safety controls passed: evaluator bump rejected, edit-sequence bump rejected, a flipped compressed
payload byte rejected by record CRC, and a truncated file rejected by size validation. A separate
L1 run with one real tile corrupted regenerated that tile, hit the other 2,552, and restored the
exact L14 whole-view hash.
- `O_DIRECT` bypassed the kernel page cache for the 55,685,120-byte L1 record set: 0.269 s
(197.6 MiB/s). Buffered first touch was 0.0191 s and the repeat was 0.0126 s, but those passes
followed file writes/verification and were page-cache warm. No page cache was dropped; device
firmware may also cache blocks, so `O_DIRECT` is page-cache-cold evidence rather than proof of a
physical cold-device read. The ordinary worker hit path includes buffered read, zlib decode,
validation, and tile construction.
- The instrumentation measures raw, zlib, and mesh candidates on a miss, so its clean-fill
whole-view-ready time was 2.674 s versus 1.718 s for the hash-on no-cache L1 run. This includes
benchmark-only extra codec work. Production should encode only its selected lossless format on a
miss. Warm disk work and decode stay on workers; the first tile was ready in 0.226 s on fill and
0.208 s warm. These headless timings show worker completion; an Unreal frame-time check is still
for the integration round.
- The cache root peaked at 60,835,840 bytes after L1, the 24 m crossing, edit probe, and corruption
controls; the enforced limit is 2,000,000,000 bytes. No eviction was needed. The actual cache
implementation is a headless Core-worker prototype, not the Unreal plugin integration.
## Default-path cost and compatibility
Five interleaved W12 L1 pairs, hash off, 2,553 tiles each:
| Pair | L14 seconds | L16 cache-disabled seconds |
|---:|---:|---:|
| 1 | 1.450 | 1.446 |
| 2 | 1.464 | 1.457 |
| 3 | 1.458 | 1.453 |
| 4 | 1.525 | 1.472 |
| 5 | 1.459 | 1.483 |
Median: 1.459 s L14, 1.457 s L16, −0.14%; no measurable cache-off cost in this run.
All 28 L14 hash configurations passed (SurfaceWorld and FloatingIslands; Clang 19 FP off and GCC
14 default FP; W1/W8/W12 with five W12 repeats). Every one matched all 12 full-detail, whole-view,
nine LOD, and exact-query rows. The four default 24 m crossing checks (W1/W12 on both compilers)
matched all 13 rows, including the 432-tile crossing hash. Default route parity passed for both
compilers across SurfaceWorld seeds 0, 1, and 17 and FloatingIslands seed 0, using the existing
96 m history probe. The cache route passed its separate L1 and 24 m crossing audits.
Representative SurfaceWorld seed-0 hashes (D/M/P/T):
| View | Density / mesh / payload / topology |
|---|---|
| Full detail, 960 tiles | `0ee0579bd15cbbe4 / 32e5d4964d610fee / 1932b96ee4b1d910 / 69743c6bad3447dc` |
| Whole L1, 2,553 tiles | `c10697a8f48380c7 / e75c9119a1498cd5 / 49a9a8d1835a6c8c / edcb23288b5e8724` |
| LOD0, 512 tiles | `769f2a706d585a72 / 33eeeb39fa443816 / 645621a1d968983f / c2457ed905036c9d` |
| LOD1, 448 tiles | `69cc5a5c0a2c60af / b48335f5410c57d5 / d15c5480162ab8ba / feca1bd025823604` |
| LOD2, 448 tiles | `85deed5683a77202 / 53469895987aeea2 / e55f8ae7e1532033 / d5feb12a0e85c09b` |
| LOD3, 384 tiles | `6518caefb3be7e7f / ec513c90b853a614 / f62fa1137ff17f82 / ccf1dcf681965816` |
| LOD4, 320 tiles | `62d4be96021992f4 / f1e4c6efcc2615a9 / 9dc5f9c88be02f33 / fe6f55125a07c3bf` |
| LOD5, 144 tiles | `5e4a8c38b8cea6fc / 08ee56998aaf21e5 / 541eea8558f26cc8 / 6539374a8b3336d8` |
| LOD6, 96 tiles | `e036e95b9006945b / 624a33f941a99b02 / 81b73e14f68bdf86 / 4de4c63ce1f80a56` |
| LOD7, 48 tiles | `8ee358af56a30d8d / 46ab3c6184048b7c / 0a6164b7e68e80c5 / 0c26b85bb03bc831` |
| LOD8, 153 tiles | `c3458e1cbdb9c471 / 46c03d0994d75192 / 8f89fc5bde12d230 / d16dc0cf62031898` |
| Exact queries: density / unclamped / detail / collision | `e4d4a8eb95b35933 / e4d4a8eb95b35933 / efcba20c0c70c415 / e604823a249029bf` |
FloatingIslands full-detail was `6d842d2878b08ef6 / 4b868a3184f08035 / 2cedf2236788752d / eac17d5e0cb5044a`; whole-view was `5be4d8cde79b2a76 / 96f75041f6861b3a / 3700a516cc89d50c / 6b99948d6eab4f21`. Its exact-query hashes also matched L14: `61daded9c303becb / 61daded9c303becb / efcba20c0c70c415 / 84d37463be9b0573`.
## Runs and limits
- First Clang build was interrupted during a silent compile phase. The next build exposed a const
error in the prototype file index; making that index mutable fixed it. Final Clang and GCC builds
passed.
- The first L1 command used the binary's incorrect default data path (`Standalone/bin/data`) and
exited before writing. Supplying `Standalone/data/owner_world.json` passed.
- The first codec run sampled 2,548 tiles because route parity had prepopulated five; it was kept
for Q16 measurements. A clean fill then sampled all 2,553 Q24 tiles.
- A route wrapper first required a moving hash when hashing was disabled; the route audit itself
passed. Seed 17's 24 m history probe had no departed-tile candidate and failed on both L16 and
the L14 control. The same L14 96 m history setup passed on both L16 compilers. The required 24 m
crossing was separately checked and passed.
- The first crossing-hash comparator used an incorrect old-log filename; correcting the filename
let all four comparisons pass. No run exceeded 30 minutes or stalled for 15 minutes.
- Final review found that the cache-hit route audit's `exact_arrays` field initially reported hash
agreement without comparing the arrays themselves. The audit was changed to compare complete
arrays; after rebuilding, both L1 and 24 m crossing route audits passed with zero mismatches.
Runs used CPUs 0–11, at most 12 build jobs, a 6 GiB address-space cap, and normal priority. The
governor stayed `powersave`; peak sampled single-process RSS was 943,844 KiB (FloatingIslands route
audit, GCC). No installs, system changes, or page-cache drops were made. The L3 FP-contract preamble
and self-test, L4 `FLazy`, L7 shared cache, L9/L12 proofs, L11 readiness/crash fix, L13 route binding,
and async worker/result rules remain enabled. Cache logic is recipe-agnostic.
**Owner note:** Saving generated terrain works exactly and fits well under the round's disk cap.
Lossless zlib saved about 56% of the serialized payload. The smaller Q24 mesh encoding passed the
screen gate but changed exact hashes, so the feature should use the lossless record format.
## Orchestrator review (Claude)
- Design accepted: the key, the pristine cache plus ordered edits, a server that never trusts client
caches, corruption/stale rejection, lossless zlib, and atomic writes.
- **Not merged into `assembler/world`:** `Standalone/Source/StreamBenchmarkPipeline.h` (included by the
Windows viewer `StandaloneViewer.cpp`) now unconditionally includes `TileCachePrototype.h`, which
includes `<unistd.h>`, `<fcntl.h>` and `<zlib.h>`. That would break the MSVC build. Kept on side
branch `wip/l16-tile-cache`. Before merging, the prototype must be platform-guarded (or the cache
given a portable I/O and codec layer), and zlib needs a Windows source (UE ships zlib).
- Cache-off cost re-checked by Luna at −0.14% (within noise).
Standalone/Research/WorldAssemblerM3/Linux16/TileCachePrototype.h
-1201
View File
File diff suppressed because it is too large. Load diff
Standalone/Source/LinuxScreenSpaceGate.h
+393 -92
View File
@@ -42,6 +42,7 @@ namespace StandaloneScreenSpaceGate
double MaximumSlopeDegrees=0.0; double MaximumSlopeDegrees=0.0;
std::array<std::uint64_t,4> SlopeSamples{}; std::array<std::uint64_t,4> SlopeSamples{};
std::array<double,4> SlopeP99{},SlopeMaximum{}; std::array<double,4> SlopeP99{},SlopeMaximum{};
std::vector<double> SamplePixels;
bool Valid=false; bool Valid=false;
}; };
@@ -52,6 +53,14 @@ namespace StandaloneScreenSpaceGate
double SlopeY=0.0; double SlopeY=0.0;
}; };
struct FMeshSurfaceHit
{
double Z=0.0;
std::array<double,3> Normal{};
std::array<std::uint8_t,4> Color{};
bool Ceiling=false;
};
struct FViewBasis struct FViewBasis
{ {
std::array<double,3> Right{}; std::array<double,3> Right{};
@@ -162,7 +171,7 @@ namespace StandaloneScreenSpaceGate
{ {
int Side=65; int Side=65;
double MinimumX=0.0,MaximumX=0.0,MinimumY=0.0,MaximumY=0.0; double MinimumX=0.0,MaximumX=0.0,MinimumY=0.0,MaximumY=0.0;
std::vector<std::vector<double>> Hits; std::vector<std::vector<FMeshSurfaceHit>> Hits;
FHeightGrid()=default; FHeightGrid()=default;
FHeightGrid(int InSide,double MinX,double MaxX,double MinY,double MaxY) FHeightGrid(int InSide,double MinX,double MaxX,double MinY,double MaxY)
@@ -175,13 +184,13 @@ namespace StandaloneScreenSpaceGate
double Y(int Iy) const double Y(int Iy) const
{return MinimumY+(MaximumY-MinimumY)*Iy/(Side-1);} {return MinimumY+(MaximumY-MinimumY)*Iy/(Side-1);}
void Add(int Ix,int Iy,double Z) void Add(int Ix,int Iy,const FMeshSurfaceHit& Hit)
{ {
auto& Values=Hits[static_cast<std::size_t>(Iy)*Side+Ix]; auto& Values=Hits[static_cast<std::size_t>(Iy)*Side+Ix];
Values.push_back(Z); Values.push_back(Hit);
} }
const std::vector<double>& At(int Ix,int Iy) const const std::vector<FMeshSurfaceHit>& At(int Ix,int Iy) const
{return Hits[static_cast<std::size_t>(Iy)*Side+Ix];} {return Hits[static_cast<std::size_t>(Iy)*Side+Ix];}
}; };
@@ -246,6 +255,8 @@ namespace StandaloneScreenSpaceGate
{ {
if(!Tile||Tile->Indices.size()<3)continue; if(!Tile||Tile->Indices.size()<3)continue;
const std::size_t TriangleCount=Tile->Indices.size()/3; const std::size_t TriangleCount=Tile->Indices.size()/3;
const std::size_t CeilingStart=TriangleCount
-std::min<std::size_t>(TriangleCount,Tile->CeilingTriangleCount);
for(std::size_t Triangle=0;Triangle<TriangleCount;++Triangle) for(std::size_t Triangle=0;Triangle<TriangleCount;++Triangle)
{ {
if(Triangle<Tile->TriangleSkirtFaceMasks.size() if(Triangle<Tile->TriangleSkirtFaceMasks.size()
@@ -274,15 +285,33 @@ namespace StandaloneScreenSpaceGate
const double W1=((C[1]-A[1])*(X-C[0])+(A[0]-C[0])*(Y-C[1]))/Denominator; const double W1=((C[1]-A[1])*(X-C[0])+(A[0]-C[0])*(Y-C[1]))/Denominator;
const double W2=1.0-W0-W1; const double W2=1.0-W0-W1;
if(W0<-1e-6||W1<-1e-6||W2<-1e-6)continue; if(W0<-1e-6||W1<-1e-6||W2<-1e-6)continue;
Grid.Add(Ix,Iy,W0*A[2]+W1*B[2]+W2*C[2]); const auto& VA=Tile->Vertices[I0];
const auto& VB=Tile->Vertices[I1];
const auto& VC=Tile->Vertices[I2];
std::array<double,3> Normal{};
for(int Axis=0;Axis<3;++Axis)
Normal[Axis]=W0*VA.Normal[Axis]+W1*VB.Normal[Axis]+W2*VC.Normal[Axis];
const double NormalLength=std::sqrt(Normal[0]*Normal[0]
+Normal[1]*Normal[1]+Normal[2]*Normal[2]);
if(NormalLength>1.0e-12)
for(double& Component:Normal)Component/=NormalLength;
std::array<std::uint8_t,4> Color{};
if(Tile->VertexColors.size()==Tile->Vertices.size())
for(int Channel=0;Channel<4;++Channel)
Color[Channel]=static_cast<std::uint8_t>(std::clamp(std::lround(
W0*Tile->VertexColors[I0][Channel]
+W1*Tile->VertexColors[I1][Channel]
+W2*Tile->VertexColors[I2][Channel]),0l,255l));
Grid.Add(Ix,Iy,{W0*A[2]+W1*B[2]+W2*C[2],Normal,Color,Triangle>=CeilingStart});
} }
} }
} }
for(auto& Values:Grid.Hits) for(auto& Values:Grid.Hits)
{ {
std::sort(Values.begin(),Values.end()); std::stable_sort(Values.begin(),Values.end(),[](const FMeshSurfaceHit& A,
Values.erase(std::unique(Values.begin(),Values.end(),[](double A,double B) const FMeshSurfaceHit& B){return A.Z<B.Z;});
{return std::abs(A-B)<0.02;}),Values.end()); Values.erase(std::unique(Values.begin(),Values.end(),[](const FMeshSurfaceHit& A,
const FMeshSurfaceHit& B){return std::abs(A.Z-B.Z)<0.02;}),Values.end());
} }
return Grid; return Grid;
} }
@@ -335,14 +364,14 @@ namespace StandaloneScreenSpaceGate
return Roots; return Roots;
} }
inline bool MatchNearest(const std::vector<double>& Hits,const std::vector<bool>& Used, inline bool MatchNearest(const std::vector<FMeshSurfaceHit>& Hits,const std::vector<bool>& Used,
double Root,std::size_t& OutIndex,double& OutError) double Root,std::size_t& OutIndex,double& OutError)
{ {
double Best=std::numeric_limits<double>::infinity();std::size_t BestIndex=Hits.size(); double Best=std::numeric_limits<double>::infinity();std::size_t BestIndex=Hits.size();
for(std::size_t I=0;I<Hits.size();++I) for(std::size_t I=0;I<Hits.size();++I)
if(I>=Used.size()||!Used[I]) if(I>=Used.size()||!Used[I])
{ {
const double Error=std::abs(Hits[I]-Root); const double Error=std::abs(Hits[I].Z-Root);
if(Error<Best){Best=Error;BestIndex=I;} if(Error<Best){Best=Error;BestIndex=I;}
} }
if(BestIndex==Hits.size())return false; if(BestIndex==Hits.size())return false;
@@ -371,7 +400,7 @@ namespace StandaloneScreenSpaceGate
continue; continue;
} }
Used[HitIndex]=true; Used[HitIndex]=true;
AddError(Accumulator,MeshGrid.At(Ix,Iy)[HitIndex]-Root.ZMeters, AddError(Accumulator,MeshGrid.At(Ix,Iy)[HitIndex].Z-Root.ZMeters,
Root,PixelBudgetMeters,View,MeshGrid.X(Ix),MeshGrid.Y(Iy)); Root,PixelBudgetMeters,View,MeshGrid.X(Ix),MeshGrid.Y(Iy));
++Row.Samples; ++Row.Samples;
} }
@@ -399,7 +428,7 @@ namespace StandaloneScreenSpaceGate
std::vector<bool> UsedReference(Reference.size(),false),UsedMesh(Mesh.size(),false); std::vector<bool> UsedReference(Reference.size(),false),UsedMesh(Mesh.size(),false);
for(std::size_t ReferenceIndex=0;ReferenceIndex<Reference.size();++ReferenceIndex) for(std::size_t ReferenceIndex=0;ReferenceIndex<Reference.size();++ReferenceIndex)
{ {
const double ReferenceZ=Reference[ReferenceIndex]; const double ReferenceZ=Reference[ReferenceIndex].Z;
UsedReference[ReferenceIndex]=true; UsedReference[ReferenceIndex]=true;
std::size_t MeshIndex=0;double MeshError=0.0; std::size_t MeshIndex=0;double MeshError=0.0;
if(!MatchNearest(Mesh,UsedMesh,ReferenceZ,MeshIndex,MeshError)) if(!MatchNearest(Mesh,UsedMesh,ReferenceZ,MeshIndex,MeshError))
@@ -416,7 +445,7 @@ namespace StandaloneScreenSpaceGate
if(BestRoot!=Roots.end()&&std::abs(BestRoot->ZMeters-ReferenceZ)<1.0) if(BestRoot!=Roots.end()&&std::abs(BestRoot->ZMeters-ReferenceZ)<1.0)
Root=*BestRoot; Root=*BestRoot;
} }
AddError(Accumulator,Mesh[MeshIndex]-Reference[ReferenceIndex], AddError(Accumulator,Mesh[MeshIndex].Z-Reference[ReferenceIndex].Z,
Root,PixelBudgetMeters,View,ReferenceGrid.X(Ix),ReferenceGrid.Y(Iy)); Root,PixelBudgetMeters,View,ReferenceGrid.X(Ix),ReferenceGrid.Y(Iy));
} }
for(std::size_t I=0;I<Mesh.size();++I) for(std::size_t I=0;I<Mesh.size();++I)
@@ -426,6 +455,110 @@ namespace StandaloneScreenSpaceGate
return Row; return Row;
} }
struct FMeshComparisonRow
{
FErrorRow Geometry;
std::uint64_t Holes=0,Cracks=0,Floating=0,MaterialMismatches=0;
double ShadeP50=0.0,ShadeP99=0.0,ShadeMaximum=0.0;
double NormalDegreesP50=0.0,NormalDegreesP99=0.0,NormalDegreesMaximum=0.0;
std::vector<double> GeometryPixels,ShadeValues,NormalAngles;
bool Valid=false;
};
inline std::array<double,3> UnitNormal(const FMeshSurfaceHit& Hit)
{
std::array<double,3> N=Hit.Normal;
const double Length=std::sqrt(N[0]*N[0]+N[1]*N[1]+N[2]*N[2]);
if(Length<=1.0e-12)return {0.0,0.0,Hit.Ceiling?-1.0:1.0};
for(double& Component:N)Component/=Length;
return N;
}
inline FMeshComparisonRow CompareMeshes(const FHeightGrid& Reference,
const FHeightGrid& Candidate,const FVoxelTileKey& Patch,double VoxelMeters,
double PixelBudgetMeters,const FViewBasis& View)
{
FMeshComparisonRow Row;Row.Valid=Reference.Side==Candidate.Side;
if(!Row.Valid)return Row;
FErrorAccumulator Geometry;
std::vector<double> ShadeValues,NormalAngles;
const std::size_t CellCount=static_cast<std::size_t>(Reference.Side)*Reference.Side;
ShadeValues.reserve(CellCount);NormalAngles.reserve(CellCount);
const double LightLength=std::sqrt(0.3*0.3+(-0.4)*(-0.4)+0.85*0.85);
const std::array<double,3> Light={0.3/LightLength,-0.4/LightLength,0.85/LightLength};
for(int Iy=0;Iy<Reference.Side;++Iy)
for(int Ix=0;Ix<Reference.Side;++Ix)
{
const auto& Before=Reference.At(Ix,Iy);
const auto& After=Candidate.At(Ix,Iy);
std::vector<bool> Used(After.size(),false);
for(std::size_t I=0;I<Before.size();++I)
{
std::size_t Matched=0;double Difference=0.0;
if(!MatchNearest(After,Used,Before[I].Z,Matched,Difference))
{
++Row.Holes;AddMissing(Geometry);ShadeValues.push_back(1.0);
NormalAngles.push_back(180.0);continue;
}
Used[Matched]=true;
const FMeshSurfaceHit& A=Before[I];
const FMeshSurfaceHit& B=After[Matched];
const auto NA=UnitNormal(A),NB=UnitNormal(B);
const double Dot=std::clamp(NA[0]*NB[0]+NA[1]*NB[1]+NA[2]*NB[2],-1.0,1.0);
constexpr double Pi=3.14159265358979323846;
const double Angle=std::acos(Dot)*180.0/Pi;
NormalAngles.push_back(Angle);
const double DiffuseA=std::max(0.0,NA[0]*Light[0]+NA[1]*Light[1]+NA[2]*Light[2]);
const double DiffuseB=std::max(0.0,NB[0]*Light[0]+NB[1]*Light[1]+NB[2]*Light[2]);
const bool bMaterialChanged=A.Color[0]!=B.Color[0]||A.Color[3]!=B.Color[3]
||A.Ceiling!=B.Ceiling;
if(bMaterialChanged)++Row.MaterialMismatches;
const double MaterialDelta=bMaterialChanged?1.0:std::max(
std::abs(static_cast<double>(A.Color[1])-B.Color[1]),
std::abs(static_cast<double>(A.Color[2])-B.Color[2]))/255.0;
ShadeValues.push_back(std::max(std::abs(DiffuseA-DiffuseB),MaterialDelta));
const double NX=NA[2]<0.0?NA[0]/std::max(1.0e-12,-NA[2])
:-NA[0]/std::max(1.0e-12,NA[2]);
const double NY=NA[2]<0.0?NA[1]/std::max(1.0e-12,-NA[2])
:-NA[1]/std::max(1.0e-12,NA[2]);
AddError(Geometry,B.Z-A.Z,{A.Z,NX,NY},PixelBudgetMeters,View,
Reference.X(Ix),Reference.Y(Iy));
}
for(std::size_t I=0;I<After.size();++I)
if(!Used[I])
{++Row.Floating;AddMissing(Geometry);ShadeValues.push_back(1.0);NormalAngles.push_back(180.0);}
}
for(int Iy=0;Iy<Reference.Side;++Iy)
for(int Ix=0;Ix<Reference.Side;++Ix)
{
const std::size_t Index=static_cast<std::size_t>(Iy)*Reference.Side+Ix;
const auto CheckEdge=[&](int NX,int NY)
{
if(NX>=Reference.Side||NY>=Reference.Side)return;
const auto& A=Reference.Hits[Index];
const auto& B=Reference.At(NX,NY);
const auto& CA=Candidate.Hits[Index];
const auto& CB=Candidate.At(NX,NY);
if(!A.empty()&&!B.empty()&&(CA.empty()!=CB.empty()))++Row.Cracks;
};
CheckEdge(Ix+1,Iy);CheckEdge(Ix,Iy+1);
}
Row.Geometry.Lod=Patch.Level;Row.Geometry.Valid=Row.Valid;
Row.Geometry.Missing=Row.Holes;Row.Geometry.Floating=Row.Floating;
FinalizeError(Row.Geometry,Geometry);
Row.ShadeP50=Percentile(ShadeValues,0.50);
Row.ShadeP99=Percentile(ShadeValues,0.99);
Row.ShadeMaximum=ShadeValues.empty()?0.0:*std::max_element(ShadeValues.begin(),ShadeValues.end());
Row.NormalDegreesP50=Percentile(NormalAngles,0.50);
Row.NormalDegreesP99=Percentile(NormalAngles,0.99);
Row.NormalDegreesMaximum=NormalAngles.empty()?0.0
:*std::max_element(NormalAngles.begin(),NormalAngles.end());
Row.GeometryPixels=std::move(Geometry.Pixels);
Row.ShadeValues=std::move(ShadeValues);
Row.NormalAngles=std::move(NormalAngles);
return Row;
}
inline FPopRow ComparePop(const FHeightGrid& Coarse,const FHeightGrid& Fine, inline FPopRow ComparePop(const FHeightGrid& Coarse,const FHeightGrid& Fine,
const FVoxelTileKey& Patch,double VoxelMeters,double PixelBudgetMeters, const FVoxelTileKey& Patch,double VoxelMeters,double PixelBudgetMeters,
const VoxelForgeCore::WorldAssemblyManifest& Assembly,const FViewBasis& View) const VoxelForgeCore::WorldAssemblyManifest& Assembly,const FViewBasis& View)
@@ -452,11 +585,12 @@ namespace StandaloneScreenSpaceGate
} }
CoarseUsed[CoarseIndex]=true; CoarseUsed[CoarseIndex]=true;
FineUsed[FineIndex]=true; FineUsed[FineIndex]=true;
AddError(Accumulator,CoarseHits[CoarseIndex]-FineHits[FineIndex], AddError(Accumulator,CoarseHits[CoarseIndex].Z-FineHits[FineIndex].Z,
Root,PixelBudgetMeters,View,Coarse.X(Ix),Coarse.Y(Iy)); Root,PixelBudgetMeters,View,Coarse.X(Ix),Coarse.Y(Iy));
++Row.Samples; ++Row.Samples;
} }
} }
Row.SamplePixels=Accumulator.Pixels;
FErrorRow Stats;Stats.Lod=Patch.Level;Stats.Valid=true; FErrorRow Stats;Stats.Lod=Patch.Level;Stats.Valid=true;
FinalizeError(Stats,Accumulator); FinalizeError(Stats,Accumulator);
Row.Samples=Stats.Samples;Row.P50=Stats.P50;Row.P99=Stats.P99;Row.Maximum=Stats.Maximum; Row.Samples=Stats.Samples;Row.P50=Stats.P50;Row.P99=Stats.P99;Row.Maximum=Stats.Maximum;
@@ -472,6 +606,16 @@ namespace StandaloneScreenSpaceGate
return Row; return Row;
} }
inline bool PopNotWorseAtAnySample(const FPopRow& Before,const FPopRow& After)
{
if(!Before.Valid||!After.Valid||Before.Samples<16
||Before.SamplePixels.size()!=After.SamplePixels.size()
||After.Cracks>Before.Cracks||After.Floating>Before.Floating)return false;
for(std::size_t I=0;I<Before.SamplePixels.size();++I)
if(After.SamplePixels[I]>Before.SamplePixels[I]+1.0e-4)return false;
return true;
}
inline std::array<double,9> ClosestDisplayedDistancesMeters( inline std::array<double,9> ClosestDisplayedDistancesMeters(
const FStandaloneOwnerWorld& World,const FVector& CameraVoxels, const FStandaloneOwnerWorld& World,const FVector& CameraVoxels,
std::uint32_t VoxelSizeMicrons) std::uint32_t VoxelSizeMicrons)
@@ -544,6 +688,7 @@ namespace StandaloneScreenSpaceGate
inline bool Run(const std::array<std::optional<FStandaloneCpuTile>,9>& SampleTiles, inline bool Run(const std::array<std::optional<FStandaloneCpuTile>,9>& SampleTiles,
FStandaloneStreamingGenerator& Streamer, FStandaloneStreamingGenerator& Streamer,
const std::vector<FVoxelTileKey>& DesiredKeys,
const VoxelForgeCore::WorldAssemblyManifest& Assembly, const VoxelForgeCore::WorldAssemblyManifest& Assembly,
const std::array<double,9>& Distances,std::uint32_t VoxelSizeMicrons,int MaxLod, const std::array<double,9>& Distances,std::uint32_t VoxelSizeMicrons,int MaxLod,
const std::array<std::int64_t,9>& PixelBudgetsSubvoxels, const std::array<std::int64_t,9>& PixelBudgetsSubvoxels,
@@ -552,17 +697,26 @@ namespace StandaloneScreenSpaceGate
const double VoxelMeters=static_cast<double>(VoxelSizeMicrons)/1000000.0; const double VoxelMeters=static_cast<double>(VoxelSizeMicrons)/1000000.0;
const FViewBasis View=MakeViewBasis(CameraYawDegrees,CameraPitchDegrees); const FViewBasis View=MakeViewBasis(CameraYawDegrees,CameraPitchDegrees);
std::array<double,9> Budgets{}; std::array<double,9> Budgets{};
std::array<FErrorRow,9> ErrorRows{}; VoxelForgeCore::TileGenerationOptions BaselineOptions;
std::array<FPopRow,9> PopRows{}; BaselineOptions.overrideFarDisplayMesh=true;
VoxelForgeCore::TileGenerationOptions CandidateOptions;
CandidateOptions.overrideFarDisplayMesh=true;
CandidateOptions.farDisplayGeometryBudgetMillipixels=Assembly.recipe.farDisplayGeometryBudgetMillipixels;
CandidateOptions.farRefinementGridScale=Assembly.recipe.farRefinementGridScale;
CandidateOptions.farRefinementLodMask=Assembly.recipe.farRefinementLodMask;
CandidateOptions.farConnectorStationScale=Assembly.recipe.farConnectorStationScale;
CandidateOptions.farConnectorRayMask=Assembly.recipe.farConnectorRayMask;
CandidateOptions.farConnectorReuseStationRadii=Assembly.recipe.farConnectorReuseStationRadii;
bool GatePass=true; bool GatePass=true;
std::printf("[ScreenSpaceGate] angular_pixel=%.9f rad (horizontal_fov=90deg,width=2560) lod>=3_limit=1.0px sampling=65x65 skirts=excluded metric=local_surface_normal_projected field_reference=direct-Heightfield2D-roots normal_probe_vox=1 view_yaw_deg=%.3f view_pitch_deg=%.3f\n", constexpr double ShadeLimit=1.0/255.0;
std::printf("[ScreenSpaceGate] angular_pixel=%.9f rad (horizontal_fov=90deg,width=2560) lod>=3_geometry_limit=1.0px same_lod_reference=today mesh sampling=65x65 skirts=excluded shading=max(lambert-delta,material-selector-or-blend-delta) shade_limit=1/255 normal_probe_vox=1 view_yaw_deg=%.3f view_pitch_deg=%.3f\n",
OnePixelRadians,CameraYawDegrees,CameraPitchDegrees); OnePixelRadians,CameraYawDegrees,CameraPitchDegrees);
for(int Lod=0;Lod<=MaxLod&&Lod<9;++Lod) for(int Lod=0;Lod<=MaxLod&&Lod<9;++Lod)
{ {
Budgets[Lod]=Distances[Lod]*OnePixelRadians; Budgets[Lod]=Distances[Lod]*OnePixelRadians;
std::printf("[ScreenSpaceDistance] lod=%d closest_m=%.3f pixel_budget_m=%.6f pixel_budget_cm=%.3f pixel_budget_subvoxels=%lld rule=%s\n", std::printf("[ScreenSpaceDistance] lod=%d closest_m=%.3f pixel_budget_m=%.6f pixel_budget_cm=%.3f pixel_budget_subvoxels=%lld rule=%s\n",
Lod,Distances[Lod],Budgets[Lod],Budgets[Lod]*100.0, Lod,Distances[Lod],Budgets[Lod],Budgets[Lod]*100.0,
static_cast<long long>(PixelBudgetsSubvoxels[Lod]),Lod<3?"exact-2cm":"1px"); static_cast<long long>(PixelBudgetsSubvoxels[Lod]),Lod<3?"exact-2cm":"same_lod_1px");
} }
const double LOD3CellMeters=static_cast<double>(CHUNK_SIZE*(1<<3))*VoxelMeters/64.0; const double LOD3CellMeters=static_cast<double>(CHUNK_SIZE*(1<<3))*VoxelMeters/64.0;
PrintKnownFailureControl(LOD3CellMeters,Distances[3],View); PrintKnownFailureControl(LOD3CellMeters,Distances[3],View);
@@ -578,115 +732,117 @@ namespace StandaloneScreenSpaceGate
const FStandaloneCpuTile* CoarsePtr=&CoarseTile; const FStandaloneCpuTile* CoarsePtr=&CoarseTile;
const std::span<const FStandaloneCpuTile* const> CoarseSpan(&CoarsePtr,1); const std::span<const FStandaloneCpuTile* const> CoarseSpan(&CoarsePtr,1);
const FHeightGrid CoarseGrid=Rasterize(CoarseSpan,CoarseTile.Key,VoxelMeters); const FHeightGrid CoarseGrid=Rasterize(CoarseSpan,CoarseTile.Key,VoxelMeters);
const FStandaloneCpuTile ExactTile=Streamer.GenerateDiagnostic( FStandaloneCpuTile BaselineTile=Streamer.GenerateDiagnostic(
CoarseTile.Key,0,1,true); CoarseTile.Key,0,1,true,&BaselineOptions,true);
const FStandaloneCpuTile* ExactPointer=&ExactTile; const FStandaloneCpuTile* BaselinePointer=&BaselineTile;
const std::span<const FStandaloneCpuTile* const> ExactSpan(&ExactPointer,1); const std::span<const FStandaloneCpuTile* const> BaselineSpan(&BaselinePointer,1);
const FHeightGrid ExactGrid=Rasterize(ExactSpan,ExactTile.Key,VoxelMeters); const FHeightGrid BaselineGrid=Rasterize(BaselineSpan,BaselineTile.Key,VoxelMeters);
ErrorRows[Lod]=CompareToReference(CoarseGrid,ExactGrid,CoarseTile.Key,VoxelMeters, const FMeshComparisonRow MeshDelta=CompareMeshes(BaselineGrid,CoarseGrid,
Budgets[Lod],Assembly,View); CoarseTile.Key,VoxelMeters,Budgets[Lod],View);
const FErrorRow& Error=ErrorRows[Lod]; const FErrorRow& Error=MeshDelta.Geometry;
const bool LodPass=Error.Valid&&Error.Missing==0&&Error.Floating==0 const bool LodPass=Error.Valid&&Error.Missing==0&&Error.Floating==0
&&Error.Samples>=16&&Error.Maximum<=1.0; &&MeshDelta.Cracks==0&&Error.Samples>=16&&Error.Maximum<=1.0
&&MeshDelta.ShadeMaximum<=ShadeLimit;
GatePass=GatePass&&LodPass; GatePass=GatePass&&LodPass;
std::printf("[ScreenSpaceGateLOD] lod=%d key=(%d,%d,%d) samples=%llu normal_px_p50=%.4f p99=%.4f max=%.4f vertical_px_p50=%.4f p99=%.4f max=%.4f max_normal_m=%.6f max_xy_m=(%.3f,%.3f) max_slope_deg=%.2f missing=%llu floating=%llu status=%s\n", std::printf("[ScreenSpaceGateLOD] lod=%d key=(%d,%d,%d) samples=%llu geometry_px_p50=%.4f p99=%.4f max=%.4f shading_delta_p50=%.6f p99=%.6f max=%.6f normal_deg_p50=%.4f p99=%.4f max=%.4f material_mismatch=%llu holes=%llu cracks=%llu floating=%llu status=%s\n",
Lod,CoarseTile.Key.Coord.X,CoarseTile.Key.Coord.Y,CoarseTile.Key.Coord.Z, Lod,CoarseTile.Key.Coord.X,CoarseTile.Key.Coord.Y,CoarseTile.Key.Coord.Z,
static_cast<unsigned long long>(Error.Samples),Error.P50,Error.P99,Error.Maximum, static_cast<unsigned long long>(Error.Samples),Error.P50,Error.P99,Error.Maximum,
Error.VerticalP50,Error.VerticalP99,Error.VerticalMaximum, MeshDelta.ShadeP50,MeshDelta.ShadeP99,MeshDelta.ShadeMaximum,
Error.MaximumNormalDistanceMeters,Error.MaximumX,Error.MaximumY, MeshDelta.NormalDegreesP50,MeshDelta.NormalDegreesP99,MeshDelta.NormalDegreesMaximum,
Error.MaximumSlopeDegrees, static_cast<unsigned long long>(MeshDelta.MaterialMismatches),
static_cast<unsigned long long>(Error.Missing),static_cast<unsigned long long>(Error.Floating), static_cast<unsigned long long>(MeshDelta.Holes),
static_cast<unsigned long long>(MeshDelta.Cracks),
static_cast<unsigned long long>(MeshDelta.Floating),
LodPass?"PASS":"REJECT"); LodPass?"PASS":"REJECT");
PrintSlopeTail("same_lod",Lod,Error.SlopeSamples,Error.SlopeP99,Error.SlopeMaximum);
const FErrorRow FieldError=CompareToExact(CoarseGrid,CoarseTile.Key,VoxelMeters, const FErrorRow FieldError=CompareToExact(CoarseGrid,CoarseTile.Key,VoxelMeters,
Budgets[Lod],Assembly,View); Budgets[Lod],Assembly,View);
const bool FieldPass=FieldError.Valid&&FieldError.Missing==0&&FieldError.Floating==0 const bool FieldPass=FieldError.Valid&&FieldError.Missing==0&&FieldError.Floating==0
&&FieldError.Samples>=16&&FieldError.Maximum<=1.0; &&FieldError.Samples>=16&&FieldError.Maximum<=1.0;
GatePass=GatePass&&FieldPass; std::printf("[ScreenSpaceGateFieldLOD] lod=%d samples=%llu field_geometry_px_p50=%.4f p99=%.4f max=%.4f missing=%llu floating=%llu status=INFO_ONLY\n",
std::printf("[ScreenSpaceGateFieldLOD] lod=%d samples=%llu normal_px_p50=%.4f p99=%.4f max=%.4f vertical_px_p50=%.4f p99=%.4f max=%.4f max_normal_m=%.6f max_xyz_m=(%.3f,%.3f,%.3f) max_slope_deg=%.2f missing=%llu floating=%llu status=%s\n",
Lod,static_cast<unsigned long long>(FieldError.Samples),FieldError.P50, Lod,static_cast<unsigned long long>(FieldError.Samples),FieldError.P50,
FieldError.P99,FieldError.Maximum,FieldError.VerticalP50,FieldError.VerticalP99, FieldError.P99,FieldError.Maximum,
FieldError.VerticalMaximum,FieldError.MaximumNormalDistanceMeters,
FieldError.MaximumX,FieldError.MaximumY,FieldError.MaximumDisplayedZ,
FieldError.MaximumSlopeDegrees,
static_cast<unsigned long long>(FieldError.Missing), static_cast<unsigned long long>(FieldError.Missing),
static_cast<unsigned long long>(FieldError.Floating),FieldPass?"PASS":"REJECT"); static_cast<unsigned long long>(FieldError.Floating));
PrintSlopeTail("field",Lod,FieldError.SlopeSamples,FieldError.SlopeP99,
FieldError.SlopeMaximum);
bool FaultControlsPass=true; bool FaultControlsPass=true;
const FHeightGrid MissingGrid(ExactGrid.Side,ExactGrid.MinimumX,ExactGrid.MaximumX, const FHeightGrid MissingGrid(BaselineGrid.Side,BaselineGrid.MinimumX,BaselineGrid.MaximumX,
ExactGrid.MinimumY,ExactGrid.MaximumY); BaselineGrid.MinimumY,BaselineGrid.MaximumY);
const FErrorRow MissingError=CompareToReference(MissingGrid,ExactGrid, const FMeshComparisonRow MissingError=CompareMeshes(BaselineGrid,MissingGrid,
CoarseTile.Key,VoxelMeters,Budgets[Lod],Assembly,View); CoarseTile.Key,VoxelMeters,Budgets[Lod],View);
const bool MissingRejected=MissingError.Missing>0&&MissingError.Maximum>1.0; const bool MissingRejected=MissingError.Holes>0&&MissingError.Geometry.Maximum>1.0;
FaultControlsPass=FaultControlsPass&&MissingRejected; FaultControlsPass=FaultControlsPass&&MissingRejected;
std::printf("[ScreenSpaceFaultInjection] lod=%d case=missing-displayed-surface missing=%llu status=%s\n", std::printf("[ScreenSpaceFaultInjection] lod=%d case=missing-displayed-surface holes=%llu status=%s\n",
Lod,static_cast<unsigned long long>(MissingError.Missing), Lod,static_cast<unsigned long long>(MissingError.Holes),
MissingRejected?"REJECTED_AS_REQUIRED":"GATE_FALSE_NEGATIVE"); MissingRejected?"REJECTED_AS_REQUIRED":"GATE_FALSE_NEGATIVE");
FStandaloneCpuTile FloatingTile=CoarseTile; FStandaloneCpuTile FloatingTile=BaselineTile;
const std::uint32_t FloatingVertexBase=static_cast<std::uint32_t>(FloatingTile.Vertices.size()); const std::uint32_t FloatingVertexBase=static_cast<std::uint32_t>(FloatingTile.Vertices.size());
for(FStandaloneCpuVertex Vertex:CoarseTile.Vertices) for(FStandaloneCpuVertex Vertex:BaselineTile.Vertices)
{ {
Vertex.Position[2]+=2.0f; Vertex.Position[2]+=2.0f;
FloatingTile.Vertices.push_back(Vertex); FloatingTile.Vertices.push_back(Vertex);
} }
for(const std::uint32_t Index:CoarseTile.Indices) FloatingTile.VertexColors.insert(FloatingTile.VertexColors.end(),
BaselineTile.VertexColors.begin(),BaselineTile.VertexColors.end());
for(const std::uint32_t Index:BaselineTile.Indices)
FloatingTile.Indices.push_back(FloatingVertexBase+Index); FloatingTile.Indices.push_back(FloatingVertexBase+Index);
FloatingTile.TriangleSkirtFaceMasks.insert(FloatingTile.TriangleSkirtFaceMasks.end(), FloatingTile.TriangleSkirtFaceMasks.insert(FloatingTile.TriangleSkirtFaceMasks.end(),
CoarseTile.TriangleSkirtFaceMasks.begin(),CoarseTile.TriangleSkirtFaceMasks.end()); BaselineTile.TriangleSkirtFaceMasks.begin(),BaselineTile.TriangleSkirtFaceMasks.end());
const FStandaloneCpuTile* FloatingPointer=&FloatingTile; const FStandaloneCpuTile* FloatingPointer=&FloatingTile;
const std::span<const FStandaloneCpuTile* const> FloatingSpan(&FloatingPointer,1); const std::span<const FStandaloneCpuTile* const> FloatingSpan(&FloatingPointer,1);
const FHeightGrid FloatingGrid=Rasterize(FloatingSpan,CoarseTile.Key,VoxelMeters); const FHeightGrid FloatingGrid=Rasterize(FloatingSpan,BaselineTile.Key,VoxelMeters);
const FErrorRow FloatingError=CompareToReference(FloatingGrid,ExactGrid, const FMeshComparisonRow FloatingError=CompareMeshes(BaselineGrid,FloatingGrid,
CoarseTile.Key,VoxelMeters,Budgets[Lod],Assembly,View); CoarseTile.Key,VoxelMeters,Budgets[Lod],View);
const bool FloatingRejected=FloatingError.Floating>0||FloatingError.Maximum>1.0; const bool FloatingRejected=FloatingError.Floating>0||FloatingError.Geometry.Maximum>1.0;
FaultControlsPass=FaultControlsPass&&FloatingRejected; FaultControlsPass=FaultControlsPass&&FloatingRejected;
std::printf("[ScreenSpaceFaultInjection] lod=%d case=2m-floating-surface floating=%llu max_px=%.4f status=%s\n", std::printf("[ScreenSpaceFaultInjection] lod=%d case=2m-floating-surface floating=%llu max_px=%.4f status=%s\n",
Lod,static_cast<unsigned long long>(FloatingError.Floating),FloatingError.Maximum, Lod,static_cast<unsigned long long>(FloatingError.Floating),FloatingError.Geometry.Maximum,
FloatingRejected?"REJECTED_AS_REQUIRED":"GATE_FALSE_NEGATIVE"); FloatingRejected?"REJECTED_AS_REQUIRED":"GATE_FALSE_NEGATIVE");
std::vector<FStandaloneCpuTile> FineTiles; std::vector<FStandaloneCpuTile> FineBeforeTiles,FineAfterTiles;
FineTiles.reserve(8); FineBeforeTiles.reserve(8);FineAfterTiles.reserve(8);
for(int Dz=0;Dz<=1;++Dz) for(int Dz=0;Dz<=1;++Dz)
for(int Dy=0;Dy<=1;++Dy) for(int Dy=0;Dy<=1;++Dy)
for(int Dx=0;Dx<=1;++Dx) for(int Dx=0;Dx<=1;++Dx)
{ {
const FVoxelTileKey FineKey(FIntVector(2*CoarseTile.Key.Coord.X+Dx, const FVoxelTileKey FineKey(FIntVector(2*CoarseTile.Key.Coord.X+Dx,
2*CoarseTile.Key.Coord.Y+Dy,2*CoarseTile.Key.Coord.Z+Dz),Lod-1); 2*CoarseTile.Key.Coord.Y+Dy,2*CoarseTile.Key.Coord.Z+Dz),Lod-1);
FStandaloneCpuTile FineTile=Streamer.GenerateDiagnostic(FineKey); FStandaloneCpuTile FineBefore=Streamer.GenerateDiagnostic(
if(!FineTile.Vertices.empty())FineTiles.push_back(std::move(FineTile)); FineKey,0,1,true,&BaselineOptions,true);
FStandaloneCpuTile FineAfter=Streamer.GenerateDiagnostic(
FineKey,0,1,true,nullptr,true);
if(!FineBefore.Vertices.empty())FineBeforeTiles.push_back(std::move(FineBefore));
if(!FineAfter.Vertices.empty())FineAfterTiles.push_back(std::move(FineAfter));
} }
std::vector<const FStandaloneCpuTile*> FinePointers; std::vector<const FStandaloneCpuTile*> FineBeforePointers,FineAfterPointers;
FinePointers.reserve(FineTiles.size()); FineBeforePointers.reserve(FineBeforeTiles.size());FineAfterPointers.reserve(FineAfterTiles.size());
for(const auto& Tile:FineTiles)FinePointers.push_back(&Tile); for(const auto& Tile:FineBeforeTiles)FineBeforePointers.push_back(&Tile);
const auto FineSpan=std::span<const FStandaloneCpuTile* const>( for(const auto& Tile:FineAfterTiles)FineAfterPointers.push_back(&Tile);
FinePointers.data(),FinePointers.size()); const auto FineBeforeSpan=std::span<const FStandaloneCpuTile* const>(
const FHeightGrid FineGrid=Rasterize(FineSpan,CoarseTile.Key,VoxelMeters); FineBeforePointers.data(),FineBeforePointers.size());
PopRows[Lod]=ComparePop(CoarseGrid,FineGrid,CoarseTile.Key,VoxelMeters, const auto FineAfterSpan=std::span<const FStandaloneCpuTile* const>(
FineAfterPointers.data(),FineAfterPointers.size());
const FHeightGrid FineBeforeGrid=Rasterize(FineBeforeSpan,CoarseTile.Key,VoxelMeters);
const FHeightGrid FineAfterGrid=Rasterize(FineAfterSpan,CoarseTile.Key,VoxelMeters);
const FPopRow PopBefore=ComparePop(BaselineGrid,FineBeforeGrid,CoarseTile.Key,VoxelMeters,
Budgets[Lod],Assembly,View); Budgets[Lod],Assembly,View);
const FPopRow& Pop=PopRows[Lod]; const FPopRow PopAfter=ComparePop(CoarseGrid,FineAfterGrid,CoarseTile.Key,VoxelMeters,
const bool PopPass=Pop.Valid&&Pop.Cracks==0&&Pop.Floating==0&&Pop.Maximum<=1.0; Budgets[Lod],Assembly,View);
const bool PopPass=PopNotWorseAtAnySample(PopBefore,PopAfter);
GatePass=GatePass&&PopPass; GatePass=GatePass&&PopPass;
std::printf("[ScreenSpaceSwap] lod=%d_to_%d samples=%llu normal_px_p50=%.4f p99=%.4f max=%.4f vertical_px_p50=%.4f p99=%.4f max=%.4f max_normal_m=%.6f max_xyz_m=(%.3f,%.3f,%.3f) max_slope_deg=%.2f cracks=%llu floating=%llu status=%s\n", std::printf("[ScreenSpaceSwap] swap=%d_to_%d distance_m=%.3f samples_before=%llu pop_before_px_p50=%.4f p99=%.4f max=%.4f pop_after_px_p50=%.4f p99=%.4f max=%.4f missing_root_samples_before=%llu after=%llu status=%s\n",
Lod,Lod-1,static_cast<unsigned long long>(Pop.Samples), Lod,Lod-1,Distances[Lod],static_cast<unsigned long long>(PopBefore.Samples),
Pop.P50,Pop.P99,Pop.Maximum,Pop.VerticalP50,Pop.VerticalP99,Pop.VerticalMaximum, PopBefore.P50,PopBefore.P99,PopBefore.Maximum,PopAfter.P50,PopAfter.P99,PopAfter.Maximum,
Pop.MaximumNormalDistanceMeters,Pop.MaximumX,Pop.MaximumY, static_cast<unsigned long long>(PopBefore.Cracks),static_cast<unsigned long long>(PopAfter.Cracks),
Pop.MaximumDisplayedZ,
Pop.MaximumSlopeDegrees,static_cast<unsigned long long>(Pop.Cracks),
static_cast<unsigned long long>(Pop.Floating),
PopPass?"PASS":"REJECT"); PopPass?"PASS":"REJECT");
PrintSlopeTail("swap",Lod,Pop.SlopeSamples,Pop.SlopeP99,Pop.SlopeMaximum);
const FHeightGrid MissingFineGrid(CoarseGrid.Side,CoarseGrid.MinimumX, const FHeightGrid MissingFineGrid(BaselineGrid.Side,BaselineGrid.MinimumX,
CoarseGrid.MaximumX,CoarseGrid.MinimumY,CoarseGrid.MaximumY); BaselineGrid.MaximumX,BaselineGrid.MinimumY,BaselineGrid.MaximumY);
const FPopRow MissingFineError=ComparePop(CoarseGrid,MissingFineGrid, const FPopRow MissingFineError=ComparePop(BaselineGrid,MissingFineGrid,
CoarseTile.Key,VoxelMeters,Budgets[Lod],Assembly,View); CoarseTile.Key,VoxelMeters,Budgets[Lod],Assembly,View);
const bool CrackRejected=MissingFineError.Cracks>0&&MissingFineError.Maximum>1.0; const bool CrackRejected=MissingFineError.Cracks>0&&MissingFineError.Maximum>1.0;
FaultControlsPass=FaultControlsPass&&CrackRejected; FaultControlsPass=FaultControlsPass&&CrackRejected;
GatePass=GatePass&&FaultControlsPass;
std::printf("[ScreenSpaceFaultInjection] lod=%d case=missing-fine-ring-surface cracks=%llu max_px=%.4f status=%s\n", std::printf("[ScreenSpaceFaultInjection] lod=%d case=missing-fine-ring-surface cracks=%llu max_px=%.4f status=%s\n",
Lod,static_cast<unsigned long long>(MissingFineError.Cracks), Lod,static_cast<unsigned long long>(MissingFineError.Cracks),
MissingFineError.Maximum,CrackRejected?"REJECTED_AS_REQUIRED":"GATE_FALSE_NEGATIVE"); MissingFineError.Maximum,CrackRejected?"REJECTED_AS_REQUIRED":"GATE_FALSE_NEGATIVE");
@@ -695,25 +851,170 @@ namespace StandaloneScreenSpaceGate
const auto CheckNegative=[&](const char* Name,std::uint32_t Drop,std::uint32_t Stride) const auto CheckNegative=[&](const char* Name,std::uint32_t Drop,std::uint32_t Stride)
{ {
FStandaloneCpuTile Bad=Streamer.GenerateDiagnostic( FStandaloneCpuTile Bad=Streamer.GenerateDiagnostic(
CoarseTile.Key,Drop,Stride,true); CoarseTile.Key,Drop,Stride,true,nullptr,true);
const FStandaloneCpuTile* BadPointer=&Bad; const FStandaloneCpuTile* BadPointer=&Bad;
const std::span<const FStandaloneCpuTile* const> BadSpan(&BadPointer,1); const std::span<const FStandaloneCpuTile* const> BadSpan(&BadPointer,1);
const FHeightGrid BadGrid=Rasterize(BadSpan,Bad.Key,VoxelMeters); const FHeightGrid BadGrid=Rasterize(BadSpan,Bad.Key,VoxelMeters);
const FErrorRow BadError=CompareToReference(BadGrid,ExactGrid, const FMeshComparisonRow BadError=CompareMeshes(BaselineGrid,BadGrid,
CoarseTile.Key,VoxelMeters,Budgets[Lod],Assembly,View); CoarseTile.Key,VoxelMeters,Budgets[Lod],View);
const bool Rejected=BadError.Missing>0||BadError.Floating>0 const bool Rejected=BadError.Holes>0||BadError.Floating>0||BadError.Cracks>0
||BadError.Samples<16||BadError.Maximum>1.0; ||BadError.Geometry.Samples<16||BadError.Geometry.Maximum>1.0
||BadError.ShadeMaximum>ShadeLimit;
NegativeControlsPass=NegativeControlsPass&&Rejected; NegativeControlsPass=NegativeControlsPass&&Rejected;
std::printf("[ScreenSpaceNegative] lod=%d case=%s p50_px=%.4f p99_px=%.4f max_px=%.4f missing=%llu floating=%llu status=%s\n", std::printf("[ScreenSpaceNegative] lod=%d case=%s p50_px=%.4f p99_px=%.4f max_px=%.4f shade_max=%.6f holes=%llu cracks=%llu floating=%llu status=%s\n",
Lod,Name,BadError.P50,BadError.P99,BadError.Maximum, Lod,Name,BadError.Geometry.P50,BadError.Geometry.P99,BadError.Geometry.Maximum,
static_cast<unsigned long long>(BadError.Missing), BadError.ShadeMaximum,static_cast<unsigned long long>(BadError.Holes),
static_cast<unsigned long long>(BadError.Cracks),
static_cast<unsigned long long>(BadError.Floating), static_cast<unsigned long long>(BadError.Floating),
Rejected?"REJECTED_AS_REQUIRED":"GATE_FALSE_NEGATIVE"); Rejected?"REJECTED_AS_REQUIRED":"GATE_FALSE_NEGATIVE");
}; };
CheckNegative("s13-global-octave-drop",1,1); CheckNegative("s13-global-octave-drop",1,1);
CheckNegative("stride-2-column-interpolation",0,2); CheckNegative("stride-2-column-interpolation",0,2);
GatePass=GatePass&&NegativeControlsPass; GatePass=GatePass&&FaultControlsPass&&NegativeControlsPass;
} }
struct FCoverageRow
{
std::uint64_t Tiles=0,Holes=0,Cracks=0,Floating=0,MaterialMismatches=0;
std::vector<double> GeometryPixels,ShadeValues,NormalAngles;
};
std::array<FCoverageRow,9> Coverage;
for(const FVoxelTileKey& Key:DesiredKeys)
{
const int Lod=Key.Level;
if(Lod<3||Lod>MaxLod||Lod>=9)continue;
FStandaloneCpuTile BaselineTile=Streamer.GenerateDiagnostic(
Key,0,1,true,&BaselineOptions,true);
FStandaloneCpuTile CandidateTile=Streamer.GenerateDiagnostic(
Key,0,1,true,&CandidateOptions,true);
const FStandaloneCpuTile* BaselinePointer=&BaselineTile;
const FStandaloneCpuTile* CandidatePointer=&CandidateTile;
const std::span<const FStandaloneCpuTile* const> BaselineSpan(&BaselinePointer,1);
const std::span<const FStandaloneCpuTile* const> CandidateSpan(&CandidatePointer,1);
const FHeightGrid BaselineGrid=Rasterize(BaselineSpan,Key,VoxelMeters);
const FHeightGrid CandidateGrid=Rasterize(CandidateSpan,Key,VoxelMeters);
const FMeshComparisonRow Delta=CompareMeshes(BaselineGrid,CandidateGrid,
Key,VoxelMeters,Budgets[Lod],View);
auto& Row=Coverage[Lod];
++Row.Tiles;
Row.Holes+=Delta.Holes;Row.Cracks+=Delta.Cracks;Row.Floating+=Delta.Floating;
Row.MaterialMismatches+=Delta.MaterialMismatches;
Row.GeometryPixels.insert(Row.GeometryPixels.end(),
Delta.GeometryPixels.begin(),Delta.GeometryPixels.end());
Row.ShadeValues.insert(Row.ShadeValues.end(),Delta.ShadeValues.begin(),Delta.ShadeValues.end());
Row.NormalAngles.insert(Row.NormalAngles.end(),Delta.NormalAngles.begin(),Delta.NormalAngles.end());
}
for(int Lod=3;Lod<=MaxLod&&Lod<9;++Lod)
{
auto& Row=Coverage[Lod];
const double GeometryP50=Percentile(Row.GeometryPixels,0.50);
const double GeometryP99=Percentile(Row.GeometryPixels,0.99);
const double GeometryMaximum=Row.GeometryPixels.empty()?0.0
:*std::max_element(Row.GeometryPixels.begin(),Row.GeometryPixels.end());
const double ShadeP50=Percentile(Row.ShadeValues,0.50);
const double ShadeP99=Percentile(Row.ShadeValues,0.99);
const double ShadeMaximum=Row.ShadeValues.empty()?0.0
:*std::max_element(Row.ShadeValues.begin(),Row.ShadeValues.end());
const double NormalP50=Percentile(Row.NormalAngles,0.50);
const double NormalP99=Percentile(Row.NormalAngles,0.99);
const double NormalMaximum=Row.NormalAngles.empty()?0.0
:*std::max_element(Row.NormalAngles.begin(),Row.NormalAngles.end());
const bool LodPass=Row.Tiles>0&&Row.GeometryPixels.size()>=16
&&GeometryMaximum<=1.0&&ShadeMaximum<=ShadeLimit
&&Row.Holes==0&&Row.Cracks==0&&Row.Floating==0;
GatePass=GatePass&&LodPass;
std::printf("[ScreenSpaceGateCoverageLOD] lod=%d tiles=%llu geometry_samples=%llu geometry_px_p50=%.4f p99=%.4f max=%.4f shading_delta_p50=%.6f p99=%.6f max=%.6f normal_deg_p50=%.4f p99=%.4f max=%.4f material_mismatch=%llu holes=%llu cracks=%llu floating=%llu status=%s\n",
Lod,static_cast<unsigned long long>(Row.Tiles),
static_cast<unsigned long long>(Row.GeometryPixels.size()),
GeometryP50,GeometryP99,GeometryMaximum,ShadeP50,ShadeP99,ShadeMaximum,
NormalP50,NormalP99,NormalMaximum,
static_cast<unsigned long long>(Row.MaterialMismatches),
static_cast<unsigned long long>(Row.Holes),
static_cast<unsigned long long>(Row.Cracks),
static_cast<unsigned long long>(Row.Floating),LodPass?"PASS":"REJECT");
}
std::uint64_t RepairOpportunities=0,BaselineWallHoles=0,BaselineWallFloating=0;
std::uint64_t CandidateWallRays=0,CandidateWallHoles=0,CandidateWallFloating=0;
std::vector<std::uint64_t> BaselineHoleIds,CandidateHoleIds;
std::vector<std::uint64_t> BaselineFloatingIds,CandidateFloatingIds;
std::vector<std::uint64_t> CandidatePanelRemovalHoleIds;
bool bRanPanelRemovalControl=false;
for(const FVoxelTileKey& Key:DesiredKeys)
{
if(Key.Level<6||Key.Level>8)continue;
VoxelForgeCore::TileGenerationOptions NoPanelBaseline=BaselineOptions;
NoPanelBaseline.diagnosticRemoveFarConnectorPanels=true;
NoPanelBaseline.diagnosticAuditFarConnectorCoverage=true;
const FStandaloneCpuTile BaseNoPanels=Streamer.GenerateDiagnostic(Key,0,1,true,&NoPanelBaseline);
RepairOpportunities+=BaseNoPanels.CoreDiagnostics.farConnectorAuditHoles;
VoxelForgeCore::TileGenerationOptions AuditedBaseline=BaselineOptions;
AuditedBaseline.diagnosticAuditFarConnectorCoverage=true;
const FStandaloneCpuTile BaseRepaired=Streamer.GenerateDiagnostic(Key,0,1,true,&AuditedBaseline);
BaselineWallHoles+=BaseRepaired.CoreDiagnostics.farConnectorAuditHoles;
BaselineWallFloating+=BaseRepaired.CoreDiagnostics.farConnectorAuditFloating;
BaselineHoleIds.insert(BaselineHoleIds.end(),
BaseRepaired.CoreDiagnostics.farConnectorAuditHoleIds.begin(),
BaseRepaired.CoreDiagnostics.farConnectorAuditHoleIds.end());
BaselineFloatingIds.insert(BaselineFloatingIds.end(),
BaseRepaired.CoreDiagnostics.farConnectorAuditFloatingIds.begin(),
BaseRepaired.CoreDiagnostics.farConnectorAuditFloatingIds.end());
VoxelForgeCore::TileGenerationOptions AuditedCandidate=CandidateOptions;
AuditedCandidate.diagnosticAuditFarConnectorCoverage=true;
const FStandaloneCpuTile CandidateRepaired=Streamer.GenerateDiagnostic(Key,0,1,true,&AuditedCandidate);
CandidateWallRays+=CandidateRepaired.CoreDiagnostics.farConnectorAuditRays;
CandidateWallHoles+=CandidateRepaired.CoreDiagnostics.farConnectorAuditHoles;
CandidateWallFloating+=CandidateRepaired.CoreDiagnostics.farConnectorAuditFloating;
CandidateHoleIds.insert(CandidateHoleIds.end(),
CandidateRepaired.CoreDiagnostics.farConnectorAuditHoleIds.begin(),
CandidateRepaired.CoreDiagnostics.farConnectorAuditHoleIds.end());
CandidateFloatingIds.insert(CandidateFloatingIds.end(),
CandidateRepaired.CoreDiagnostics.farConnectorAuditFloatingIds.begin(),
CandidateRepaired.CoreDiagnostics.farConnectorAuditFloatingIds.end());
if(!bRanPanelRemovalControl&&BaseNoPanels.CoreDiagnostics.farConnectorAuditHoles>0)
{
AuditedCandidate.diagnosticRemoveFarConnectorPanels=true;
const FStandaloneCpuTile CandidateNoPanels=Streamer.GenerateDiagnostic(
Key,0,1,true,&AuditedCandidate);
CandidatePanelRemovalHoleIds=CandidateNoPanels.CoreDiagnostics.farConnectorAuditHoleIds;
bRanPanelRemovalControl=true;
}
}
std::sort(BaselineHoleIds.begin(),BaselineHoleIds.end());
std::sort(CandidateHoleIds.begin(),CandidateHoleIds.end());
std::sort(BaselineFloatingIds.begin(),BaselineFloatingIds.end());
std::sort(CandidateFloatingIds.begin(),CandidateFloatingIds.end());
const auto IsSubset=[](const std::vector<std::uint64_t>& A,const std::vector<std::uint64_t>& B)
{return std::includes(B.begin(),B.end(),A.begin(),A.end());};
std::sort(CandidatePanelRemovalHoleIds.begin(),CandidatePanelRemovalHoleIds.end());
const bool bNoNewHoles=IsSubset(CandidateHoleIds,BaselineHoleIds);
const bool bNoNewFloating=IsSubset(CandidateFloatingIds,BaselineFloatingIds);
const bool bPanelRemovalDetected=bRanPanelRemovalControl
&&!IsSubset(CandidatePanelRemovalHoleIds,CandidateHoleIds);
std::vector<std::uint64_t> NewHoleIds,NewFloatingIds,PanelRemovalNewHoleIds;
std::set_difference(CandidateHoleIds.begin(),CandidateHoleIds.end(),
BaselineHoleIds.begin(),BaselineHoleIds.end(),std::back_inserter(NewHoleIds));
std::set_difference(CandidateFloatingIds.begin(),CandidateFloatingIds.end(),
BaselineFloatingIds.begin(),BaselineFloatingIds.end(),std::back_inserter(NewFloatingIds));
std::set_difference(CandidatePanelRemovalHoleIds.begin(),CandidatePanelRemovalHoleIds.end(),
CandidateHoleIds.begin(),CandidateHoleIds.end(),std::back_inserter(PanelRemovalNewHoleIds));
const bool bRepairRowsPass=RepairOpportunities>0&&bNoNewHoles&&bNoNewFloating
&&bPanelRemovalDetected;
GatePass=GatePass&&bRepairRowsPass;
std::printf("[ScreenSpaceConnectorGate] lods=6..8 baseline_missing_ray_stations=%llu candidate_audited_rays=%llu baseline_holes=%llu candidate_holes=%llu new_holes=%llu baseline_floating=%llu candidate_floating=%llu new_floating=%llu panel_removal_new_holes=%llu status=%s\n",
static_cast<unsigned long long>(RepairOpportunities),
static_cast<unsigned long long>(CandidateWallRays),
static_cast<unsigned long long>(BaselineWallHoles),
static_cast<unsigned long long>(CandidateWallHoles),
static_cast<unsigned long long>(NewHoleIds.size()),
static_cast<unsigned long long>(BaselineWallFloating),
static_cast<unsigned long long>(CandidateWallFloating),
static_cast<unsigned long long>(NewFloatingIds.size()),
static_cast<unsigned long long>(PanelRemovalNewHoleIds.size()),
bRepairRowsPass?"PASS":"REJECT");
std::printf("[ScreenSpaceGateResult] status=%s\n",GatePass?"PASS":"REJECT"); std::printf("[ScreenSpaceGateResult] status=%s\n",GatePass?"PASS":"REJECT");
return GatePass; return GatePass;
} }
Standalone/Source/StreamBenchmarkHeadless.cpp
+1 -16
View File
@@ -35,7 +35,6 @@ namespace
bool StreamBenchmark=false; bool StreamBenchmark=false;
bool StreamPipeline=false; bool StreamPipeline=false;
bool StreamHash=false; bool StreamHash=false;
bool TileCachePrototype=false;
bool ProfileColumnOverlap=false; bool ProfileColumnOverlap=false;
bool UseSharedColumnCache=true; bool UseSharedColumnCache=true;
bool ForceGenerate=false; bool ForceGenerate=false;
@@ -120,7 +119,6 @@ namespace
" --seed=N --recipe=surface-world|floating-islands\n" " --seed=N --recipe=surface-world|floating-islands\n"
" --auto-surface-camera (place the 27-tile cube on this seed's surface)\n" " --auto-surface-camera (place the 27-tile cube on this seed's surface)\n"
" --mesh-coverage (report non-empty tiles and indexed triangles)\n" " --mesh-coverage (report non-empty tiles and indexed triangles)\n"
" --tile-cache-prototype (opt-in exact disk-cache round trip; writes Saved/Linux/cache/l16-prototype)\n"
" --screen-space-gate (measure display error and historical negative controls)\n" " --screen-space-gate (measure display error and historical negative controls)\n"
" --audit-uniform-block-proofs (fill and check every interval-certified block)\n" " --audit-uniform-block-proofs (fill and check every interval-certified block)\n"
" --profile-lod-work (report per-LOD density sample counts)\n" " --profile-lod-work (report per-LOD density sample counts)\n"
@@ -139,15 +137,6 @@ namespace
else if(arg=="--stream-benchmark")options.StreamBenchmark=true; else if(arg=="--stream-benchmark")options.StreamBenchmark=true;
else if(arg=="--stream-pipeline")options.StreamPipeline=true; else if(arg=="--stream-pipeline")options.StreamPipeline=true;
else if(arg=="--stream-hash")options.StreamHash=true; else if(arg=="--stream-hash")options.StreamHash=true;
else if(arg=="--tile-cache-prototype")
{
options.TileCachePrototype=true;
options.StreamPipeline=true;
options.StreamBenchmark=true;
options.WorldAssemblerPrototype=true;
options.ForceGenerate=true;
options.StreamHash=true;
}
else if(arg=="--profile-column-overlap")options.ProfileColumnOverlap=true; else if(arg=="--profile-column-overlap")options.ProfileColumnOverlap=true;
else if(arg=="--force-generate")options.ForceGenerate=true; else if(arg=="--force-generate")options.ForceGenerate=true;
else if(arg=="--mesh-coverage")options.MeshCoverage=true; else if(arg=="--mesh-coverage")options.MeshCoverage=true;
@@ -388,9 +377,5 @@ int main(int argc,char** argv)
cameraStepVoxels,options.AuditUniformBlockProofs,options.ProfileLodWork, cameraStepVoxels,options.AuditUniformBlockProofs,options.ProfileLodWork,
options.RegionCrossing,options.AuditTileShapeReadiness, options.RegionCrossing,options.AuditTileShapeReadiness,
options.DiagnosticSlowShapeCompileMs,options.AuditBasePageEquivalence, options.DiagnosticSlowShapeCompileMs,options.AuditBasePageEquivalence,
options.AuditRouteParity, options.AuditRouteParity);
options.TileCachePrototype
?std::filesystem::current_path()/"Saved/Linux/cache/l16-prototype"
:std::filesystem::path{},
options.DataPath);
} }
Standalone/Source/StreamBenchmarkPipeline.h
+9 -113
View File
@@ -3,7 +3,6 @@
#include "OwnerWorldLoader.h" #include "OwnerWorldLoader.h"
#include "StreamingWorld.h" #include "StreamingWorld.h"
#include "LinuxScreenSpaceGate.h" #include "LinuxScreenSpaceGate.h"
#include "../Research/WorldAssemblerM3/Linux16/TileCachePrototype.h"
#include <algorithm> #include <algorithm>
#include <array> #include <array>
@@ -254,9 +253,7 @@ namespace StandaloneStreamBenchmark
bool bAuditUniformBlockProofs=false,bool bReportLodWorkProfile=false, bool bAuditUniformBlockProofs=false,bool bReportLodWorkProfile=false,
bool bRegionCrossing=false, bool bRegionCrossing=false,
bool bAuditTileShapeReadiness=false,int DiagnosticSlowShapeCompileMs=0, bool bAuditTileShapeReadiness=false,int DiagnosticSlowShapeCompileMs=0,
bool bAuditBasePageEquivalence=false,bool bAuditRouteParity=false, bool bAuditBasePageEquivalence=false,bool bAuditRouteParity=false)
const std::filesystem::path& TileCachePrototypeRoot={},
const std::filesystem::path& OwnerSnapshotPath={})
{ {
using Clock=std::chrono::steady_clock; using Clock=std::chrono::steady_clock;
FStandaloneOwnerWorld& World=*OwnerWorld; FStandaloneOwnerWorld& World=*OwnerWorld;
@@ -277,42 +274,6 @@ namespace StandaloneStreamBenchmark
const bool bRoundTunnels=World.Generator->bStandaloneResearchRoundTunnels; const bool bRoundTunnels=World.Generator->bStandaloneResearchRoundTunnels;
const bool bEnhancedTunnelWalls=World.Generator->bStandaloneResearchEnhancedWalls; const bool bEnhancedTunnelWalls=World.Generator->bStandaloneResearchEnhancedWalls;
VoxelForgeCore::World Shape(CoreSnapshot); VoxelForgeCore::World Shape(CoreSnapshot);
std::unique_ptr<StandaloneTileCachePrototype::TileDiskCache> TileCache;
FStandaloneCoreTileRoute CoreTileRoute;
VoxelForgeCore::TileGenerationOptions CacheOptions;
if(!TileCachePrototypeRoot.empty())
{
const auto RepoRoot=std::filesystem::current_path().lexically_normal();
const auto AllowedRoot=(RepoRoot/"Saved/Linux/cache").lexically_normal();
const auto RequestedRoot=std::filesystem::absolute(TileCachePrototypeRoot).lexically_normal();
const std::string AllowedString=AllowedRoot.string()+std::string(1,std::filesystem::path::preferred_separator);
if(RequestedRoot.string().rfind(AllowedString,0)!=0||OwnerSnapshotPath.empty())
{
std::fprintf(stderr,"[TileCachePrototype] path rejected: cache writes must stay under Saved/Linux/cache and owner snapshot identity must be provided.\n");
return 2;
}
if(ClosestDisplayDistances[0]==0.0)
ClosestDisplayDistances=StandaloneScreenSpaceGate::ClosestDisplayedDistancesMeters(
World,CameraPositionVoxels,World.CoreVoxelSizeMicrons);
const auto Identity=StandaloneTileCachePrototype::MakeIdentity(
World,CoreSnapshot,OwnerSnapshotPath);
if(Identity.ownerSnapshotHash==0||!Assembly)
{
std::fprintf(stderr,"[TileCachePrototype] missing owner snapshot or world assembly identity.\n");
return 2;
}
TileCache=std::make_unique<StandaloneTileCachePrototype::TileDiskCache>(
RequestedRoot,Identity,ClosestDisplayDistances);
CacheOptions.hashDensity=true;
StandaloneTileCachePrototype::TileDiskCache* Cache=TileCache.get();
CoreTileRoute=[Cache](const VoxelForgeCore::World& TileWorld,
const VoxelForgeCore::TileKey& Key,
const VoxelForgeCore::TileGenerationOptions& Options,bool& bHit,
std::string& Error)
{return Cache->GenerateOrLoad(TileWorld,Key,Options,bHit,Error);};
std::printf("[TileCachePrototype] enabled root=%s owner_snapshot=%s mode=worker_generate_store_read_or_warm_hit codec=zlib-6\n",
RequestedRoot.string().c_str(),OwnerSnapshotPath.string().c_str());
}
FStandaloneDesiredTileSet Desired=StandaloneWorldMesher::BuildDesiredSet(World,CameraPositionVoxels); FStandaloneDesiredTileSet Desired=StandaloneWorldMesher::BuildDesiredSet(World,CameraPositionVoxels);
const auto CoreTiles=Shape.desiredTiles( const auto CoreTiles=Shape.desiredTiles(
{CameraPositionVoxels.X,CameraPositionVoxels.Y,CameraPositionVoxels.Z}); {CameraPositionVoxels.X,CameraPositionVoxels.Y,CameraPositionVoxels.Z});
@@ -450,7 +411,8 @@ namespace StandaloneStreamBenchmark
ShapeRequest.enhancedTunnelWalls=bEnhancedTunnelWalls; ShapeRequest.enhancedTunnelWalls=bEnhancedTunnelWalls;
FStandaloneStreamingGenerator Streamer(World,WorkerCount,bHashDensity,nullptr,MaxLevel, FStandaloneStreamingGenerator Streamer(World,WorkerCount,bHashDensity,nullptr,MaxLevel,
bSkipUvAndColors,bSkipMaterialGroups,bSkipUV,bSkipColors,&Shape,bProfileColumnOverlap, bSkipUvAndColors,bSkipMaterialGroups,bSkipUV,bSkipColors,&Shape,bProfileColumnOverlap,
FarHorizonLOD8InteriorStepVoxels,bUseSharedColumnCache,std::move(CoreTileRoute)); FarHorizonLOD8InteriorStepVoxels,bUseSharedColumnCache);
Streamer.SetDisplayAttributeCaptureEnabled(bScreenSpaceGate);
Streamer.SetUniformBlockProofAuditEnabled(bAuditUniformBlockProofs); Streamer.SetUniformBlockProofAuditEnabled(bAuditUniformBlockProofs);
if(bScreenSpaceGate||bReportLodWorkProfile)Streamer.SetMesherStageProfilingEnabled(true); if(bScreenSpaceGate||bReportLodWorkProfile)Streamer.SetMesherStageProfilingEnabled(true);
const auto StreamStart=Clock::now(); const auto StreamStart=Clock::now();
@@ -696,7 +658,6 @@ namespace StandaloneStreamBenchmark
VoxelForgeCore::TileGenerationOptions PageOptions=CaptureOptions; VoxelForgeCore::TileGenerationOptions PageOptions=CaptureOptions;
PageOptions.auditUniformBlockProofs=true; PageOptions.auditUniformBlockProofs=true;
int32 CachePairs=0,CacheMismatches=0,EditCachePairs=0,EditCacheMismatches=0;
int32 BasePageMismatches=0,PageRoutesVerified=0; int32 BasePageMismatches=0,PageRoutesVerified=0;
std::uint64_t PageProofBlocks=0,PageProofSamples=0,PageProofFallbacks=0, std::uint64_t PageProofBlocks=0,PageProofSamples=0,PageProofFallbacks=0,
PageProofUnknown=0,PageProofViolations=0; PageProofUnknown=0,PageProofViolations=0;
@@ -756,28 +717,6 @@ namespace StandaloneStreamBenchmark
PageProofUnknown+=Page->diagnostics.uniformBlockProofUnknownBlocks; PageProofUnknown+=Page->diagnostics.uniformBlockProofUnknownBlocks;
PageProofViolations+=Page->diagnostics.uniformBlockProofViolations; PageProofViolations+=Page->diagnostics.uniformBlockProofViolations;
} }
if(TileCache)
{
VoxelForgeCore::TileMesh Cached;
std::string CacheError;
bool bCacheReady=TileCache->LoadExpected(Row.Candidate.Key,CacheOptions,Cached,CacheError);
if(!bCacheReady&&Row.Base)
bCacheReady=TileCache->StoreAndLoadExplicit(*Row.Base,CacheOptions,
TileCache->identity(),Cached,CacheError);
const bool bCacheSame=bCacheReady&&Row.Base
&&SameTileOutput(*Row.Base,Cached)
&&StandaloneTileCachePrototype::SameExactMesh(*Row.Base,Cached);
++CachePairs;
if(!bCacheSame)++CacheMismatches;
std::printf("[RouteParityPair] route=base/disk-cache key=(%d,%d,%d,lod%d) same_density=%d same_mesh=%d same_payload=%d same_topology=%d exact_arrays=%d status=%s error=%s\n",
Row.Candidate.Key.coordinate.x,Row.Candidate.Key.coordinate.y,
Row.Candidate.Key.coordinate.z,Row.Candidate.Key.lod,
Row.Base&&Cached.densityHash==Row.Base->densityHash?1:0,
Row.Base&&Cached.meshHash==Row.Base->meshHash?1:0,
Row.Base&&Cached.payloadHash==Row.Base->payloadHash?1:0,
Row.Base&&Cached.topologyHash==Row.Base->topologyHash?1:0,
bCacheSame?1:0,bCacheSame?"PASS":"FAIL",CacheError.c_str());
}
std::printf("[RouteParityPair] route=base/page key=(%d,%d,%d,lod%d) page_requested=%d page_verified=%d same_density=%d same_mesh=%d same_payload=%d same_topology=%d samples=%llu max_abs_density_delta=%.9g sign_mismatches=%llu edge_crossing_mismatches=%llu shared_surface_edges=%llu max_surface_delta_vox=%.9g exact_query_mismatches=%llu base_native_float=%llu page_native_float=%llu first_density=(%.9g,%.9g) status=%s error=%s\n", std::printf("[RouteParityPair] route=base/page key=(%d,%d,%d,lod%d) page_requested=%d page_verified=%d same_density=%d same_mesh=%d same_payload=%d same_topology=%d samples=%llu max_abs_density_delta=%.9g sign_mismatches=%llu edge_crossing_mismatches=%llu shared_surface_edges=%llu max_surface_delta_vox=%.9g exact_query_mismatches=%llu base_native_float=%llu page_native_float=%llu first_density=(%.9g,%.9g) status=%s error=%s\n",
Row.Candidate.Key.coordinate.x,Row.Candidate.Key.coordinate.y, Row.Candidate.Key.coordinate.x,Row.Candidate.Key.coordinate.y,
Row.Candidate.Key.coordinate.z,Row.Candidate.Key.lod, Row.Candidate.Key.coordinate.z,Row.Candidate.Key.lod,
@@ -961,27 +900,6 @@ namespace StandaloneStreamBenchmark
&&EditedPage->diagnostics.analyticRegionPageRoute; &&EditedPage->diagnostics.analyticRegionPageRoute;
const bool bEditSame=EditedBase&&EditedPage&&bEditPageVerified const bool bEditSame=EditedBase&&EditedPage&&bEditPageVerified
&&SameTileOutput(*EditedBase,*EditedPage); &&SameTileOutput(*EditedBase,*EditedPage);
bool bEditCacheSame=false;
if(TileCache&&EditedBase&&Applied)
{
VoxelForgeCore::TileMesh CachedEdited;
std::string CacheError;
const auto EditedIdentity=TileCache->WithEdit(Transaction.sequence,Applied->digest);
bEditCacheSame=TileCache->StoreAndLoadExplicit(*EditedBase,CacheOptions,
EditedIdentity,CachedEdited,CacheError)
&&SameTileOutput(*EditedBase,CachedEdited);
++EditCachePairs;
if(!bEditCacheSame)++EditCacheMismatches;
std::printf("[RouteParityPair] route=edited-base/disk-cache key=(%d,%d,%d,lod%d) edit_sequence=%llu edit_digest=0x%016llx same_density=%d same_mesh=%d same_payload=%d same_topology=%d exact_arrays=%d status=%s error=%s\n",
Key.coordinate.x,Key.coordinate.y,Key.coordinate.z,Key.lod,
static_cast<unsigned long long>(Transaction.sequence),
static_cast<unsigned long long>(Applied->digest),
CachedEdited.densityHash==EditedBase->densityHash?1:0,
CachedEdited.meshHash==EditedBase->meshHash?1:0,
CachedEdited.payloadHash==EditedBase->payloadHash?1:0,
CachedEdited.topologyHash==EditedBase->topologyHash?1:0,
bEditCacheSame?1:0,bEditCacheSame?"PASS":"FAIL",CacheError.c_str());
}
const bool bEditPass=Before&&Applied&&EditedBase&&bEditPageBuilt const bool bEditPass=Before&&Applied&&EditedBase&&bEditPageBuilt
&&bEditPageVerified&&bEditChanged&&bEditSame&&EditComparison.valid &&bEditPageVerified&&bEditChanged&&bEditSame&&EditComparison.valid
&&EditComparison.exactQueryMismatches==0; &&EditComparison.exactQueryMismatches==0;
@@ -1079,13 +997,11 @@ namespace StandaloneStreamBenchmark
&&HistoryPairs==(bHasCameraStep&&bRegionCrossing?1:0)&&HistoryMismatches==0 &&HistoryPairs==(bHasCameraStep&&bRegionCrossing?1:0)&&HistoryMismatches==0
&&PageRoutesVerified==static_cast<int32>(Rows.size()) &&PageRoutesVerified==static_cast<int32>(Rows.size())
&&NativeRoutesVerified==NativePairs&&PageProofViolations==0 &&NativeRoutesVerified==NativePairs&&PageProofViolations==0
&&PageExactMismatches==0&&NativeExactMismatches==0 &&PageExactMismatches==0&&NativeExactMismatches==0;
&&(!TileCache||(CachePairs>0&&CacheMismatches==0&&EditCachePairs==1&&EditCacheMismatches==0)); std::printf("[RouteParityAudit] seed=%d page_pairs=%zu page_verified=%d page_mismatches=%d native_only_pairs=%d native_verified=%d native_mismatches=%d higher_lod_exact_pairs=%d higher_lod_exact_mismatches=%d edit_pairs=%d edit_mismatches=%d history_pairs=%d history_mismatches=%d exact_query_samples=%llu exact_query_mismatches=%llu page_density_samples=%llu max_abs_density_delta=%.9g mean_abs_density_delta=%.9g sign_mismatches=%llu edge_crossing_mismatches=%llu shared_surface_edges=%llu max_surface_delta_vox=%.9g mean_surface_delta_vox=%.9g page_proof_blocks=%llu page_proof_samples=%llu page_proof_fallbacks=%llu page_proof_unknown=%llu page_proof_violations=%llu status=%s\n",
std::printf("[RouteParityAudit] seed=%d page_pairs=%zu page_verified=%d page_mismatches=%d native_only_pairs=%d native_verified=%d native_mismatches=%d higher_lod_exact_pairs=%d higher_lod_exact_mismatches=%d edit_pairs=%d edit_mismatches=%d history_pairs=%d history_mismatches=%d cache_pairs=%d cache_mismatches=%d edit_cache_pairs=%d edit_cache_mismatches=%d exact_query_samples=%llu exact_query_mismatches=%llu page_density_samples=%llu max_abs_density_delta=%.9g mean_abs_density_delta=%.9g sign_mismatches=%llu edge_crossing_mismatches=%llu shared_surface_edges=%llu max_surface_delta_vox=%.9g mean_surface_delta_vox=%.9g page_proof_blocks=%llu page_proof_samples=%llu page_proof_fallbacks=%llu page_proof_unknown=%llu page_proof_violations=%llu status=%s\n",
World.Seed,Rows.size(),PageRoutesVerified,BasePageMismatches, World.Seed,Rows.size(),PageRoutesVerified,BasePageMismatches,
NativePairs,NativeRoutesVerified,NativeMismatches,ExactQueryPairs, NativePairs,NativeRoutesVerified,NativeMismatches,ExactQueryPairs,
ExactQueryMismatches,EditPairs,EditMismatches,HistoryPairs,HistoryMismatches, ExactQueryMismatches,EditPairs,EditMismatches,HistoryPairs,HistoryMismatches,
CachePairs,CacheMismatches,EditCachePairs,EditCacheMismatches,
static_cast<unsigned long long>(PageExactSamples static_cast<unsigned long long>(PageExactSamples
+NativeExactSamples+ExactQuerySamples), +NativeExactSamples+ExactQuerySamples),
static_cast<unsigned long long>(PageExactMismatches static_cast<unsigned long long>(PageExactMismatches
@@ -1565,7 +1481,7 @@ namespace StandaloneStreamBenchmark
{ {
if(Assembly) if(Assembly)
bScreenSpaceGatePassing=StandaloneScreenSpaceGate::Run(GateSampleTiles, bScreenSpaceGatePassing=StandaloneScreenSpaceGate::Run(GateSampleTiles,
Streamer,*Assembly,ClosestDisplayDistances,World.CoreVoxelSizeMicrons, Streamer,Desired.Tiles,*Assembly,ClosestDisplayDistances,World.CoreVoxelSizeMicrons,
World.MaxClipLevel,PixelBudgetsSubvoxels,CameraYawDegrees, World.MaxClipLevel,PixelBudgetsSubvoxels,CameraYawDegrees,
CameraPitchDegrees); CameraPitchDegrees);
else else
@@ -1725,12 +1641,10 @@ namespace StandaloneStreamBenchmark
const double WholeViewSeconds=std::chrono::duration<double>(WholeViewReady-ColdStart).count(); const double WholeViewSeconds=std::chrono::duration<double>(WholeViewReady-ColdStart).count();
const double StreamSeconds=std::chrono::duration<double>(WholeViewReady-StreamStart).count(); const double StreamSeconds=std::chrono::duration<double>(WholeViewReady-StreamStart).count();
const double FirstTileSeconds=std::chrono::duration<double>(FirstTileReady-ColdStart).count(); const double FirstTileSeconds=std::chrono::duration<double>(FirstTileReady-ColdStart).count();
const std::uint64_t TileCacheHits=TileCache?TileCache->hitCount():Streamer.GetCacheHits(); std::printf("[StreamPipeline] workers=%d shape=%.3fs first_tile_ready=%.3fs neighborhood_mesh=%.3fs neighborhood_ready=%.3fs full_detail_ready=%.3fs whole_view_ready=%.3fs neighborhood_tiles=%d/27 full_detail_tiles=%d/%d whole_tiles=%d/%zu stream_wall=%.3fs cache_hits=0 full_payload=1\n",
std::printf("[StreamPipeline] workers=%d shape=%.3fs first_tile_ready=%.3fs neighborhood_mesh=%.3fs neighborhood_ready=%.3fs full_detail_ready=%.3fs whole_view_ready=%.3fs neighborhood_tiles=%d/27 full_detail_tiles=%d/%d whole_tiles=%d/%zu stream_wall=%.3fs cache_hits=%llu full_payload=1\n",
WorkerCount,ShapeSeconds,FirstTileSeconds,NeighborhoodMeshSeconds,NeighborhoodSeconds, WorkerCount,ShapeSeconds,FirstTileSeconds,NeighborhoodMeshSeconds,NeighborhoodSeconds,
FullDetailSeconds,WholeViewSeconds,NeighborhoodFinished, FullDetailSeconds,WholeViewSeconds,NeighborhoodFinished,
FullDetailFinished,FullDetailExpected,Finished,Desired.Tiles.size(),StreamSeconds, FullDetailFinished,FullDetailExpected,Finished,Desired.Tiles.size(),StreamSeconds);
static_cast<unsigned long long>(TileCacheHits));
if(bReportMeshCoverage) if(bReportMeshCoverage)
std::printf("[PipelineMeshCoverage] tiles=%d nonempty_mesh_tiles=%llu indexed_triangles=%llu\n", std::printf("[PipelineMeshCoverage] tiles=%d nonempty_mesh_tiles=%llu indexed_triangles=%llu\n",
Finished,static_cast<unsigned long long>(WholeNonEmptyMeshTiles), Finished,static_cast<unsigned long long>(WholeNonEmptyMeshTiles),
@@ -1826,28 +1740,10 @@ namespace StandaloneStreamBenchmark
static_cast<unsigned long long>(ExactDetailHash), static_cast<unsigned long long>(ExactDetailHash),
static_cast<unsigned long long>(ExactCollisionHash)); static_cast<unsigned long long>(ExactCollisionHash));
} }
bool bTileCacheSafetyPassing=true;
if(TileCache)
{
std::string CacheSafetyError;
bTileCacheSafetyPassing=TileCache->RunSafetyControls(CacheOptions,CacheSafetyError);
if(!bTileCacheSafetyPassing)
std::fprintf(stderr,"[TileCacheSafety] failed: %s\n",CacheSafetyError.c_str());
TileCache->PrintSummary();
TileCache->MeasureDiskReads();
const std::uint64_t CacheBytes=TileCache->bytesOnDisk();
std::printf("[TileCacheBudget] bytes=%llu maximum=%llu status=%s\n",
static_cast<unsigned long long>(CacheBytes),
static_cast<unsigned long long>(StandaloneTileCachePrototype::MaximumDiskBytes),
CacheBytes<=StandaloneTileCachePrototype::MaximumDiskBytes?"PASS":"FAIL");
bTileCacheSafetyPassing=bTileCacheSafetyPassing
&&CacheBytes<=StandaloneTileCachePrototype::MaximumDiskBytes;
}
return NeighborhoodFinished==static_cast<int32>(Neighborhood.size()) return NeighborhoodFinished==static_cast<int32>(Neighborhood.size())
&&FullDetailFinished==FullDetailExpected&&Finished==Update.DesiredTiles&&bHashesPassing &&FullDetailFinished==FullDetailExpected&&Finished==Update.DesiredTiles&&bHashesPassing
&&(!bProfileColumnOverlap||bColumnOverlapCapturePassing) &&(!bProfileColumnOverlap||bColumnOverlapCapturePassing)
&&(!bExportRequested||bSceneExportPassing)&&bScreenSpaceGatePassing &&(!bExportRequested||bSceneExportPassing)&&bScreenSpaceGatePassing
&&(!bAuditUniformBlockProofs||BlockProofViolations==0) &&(!bAuditUniformBlockProofs||BlockProofViolations==0)?0:6;
&&bTileCacheSafetyPassing?0:6;
} }
} // namespace StandaloneStreamBenchmark } // namespace StandaloneStreamBenchmark
Standalone/Source/StreamingWorld.h
+36 -17
View File
@@ -43,9 +43,11 @@ struct FStandaloneCpuTile
FVoxelTileKey Key; FVoxelTileKey Key;
uint64 RequestedCrossing=0; uint64 RequestedCrossing=0;
std::vector<FStandaloneCpuVertex> Vertices; std::vector<FStandaloneCpuVertex> Vertices;
std::vector<std::array<uint8,4>> VertexColors;
std::vector<uint32> Indices; std::vector<uint32> Indices;
std::vector<uint8> TriangleSkirtFaceMasks; std::vector<uint8> TriangleSkirtFaceMasks;
uint32 SurfaceVertexCount=0; uint32 SurfaceVertexCount=0;
uint32 CeilingTriangleCount=0;
uint32 SkirtTriangleCount=0; uint32 SkirtTriangleCount=0;
uint64 DensitySamples=0; uint64 DensitySamples=0;
double WorkerSeconds=0.0; double WorkerSeconds=0.0;
@@ -67,10 +69,6 @@ struct FStandaloneCpuTile
}; };
using FStandaloneCoreTileRoute = std::function<std::optional<VoxelForgeCore::TileMesh>(
const VoxelForgeCore::World&,const VoxelForgeCore::TileKey&,
const VoxelForgeCore::TileGenerationOptions&,bool&,std::string&)>;
inline uint64 HashStandaloneMeshPayload(const FVoxelMeshData& Mesh) inline uint64 HashStandaloneMeshPayload(const FVoxelMeshData& Mesh)
{ {
uint64 Hash=14695981039346656037ull; uint64 Hash=14695981039346656037ull;
@@ -434,16 +432,14 @@ public:
VoxelForgeCore::World* InCoreWorld=nullptr, VoxelForgeCore::World* InCoreWorld=nullptr,
bool bInCaptureColumnCoordinates=false, bool bInCaptureColumnCoordinates=false,
int32 InLod8HorizonInteriorStepVoxels=0, int32 InLod8HorizonInteriorStepVoxels=0,
bool bInSharedColumnCache=true, bool bInSharedColumnCache=true)
FStandaloneCoreTileRoute InCoreTileRoute={})
:World(&InWorld),WorkerCount(FMath::Clamp(InWorkerCount,1,16)),bHashDensity(bInHashDensity), :World(&InWorld),WorkerCount(FMath::Clamp(InWorkerCount,1,16)),bHashDensity(bInHashDensity),
MeshCache(InMeshCache),MaxLevelFilter(InMaxLevelFilter), MeshCache(InMeshCache),MaxLevelFilter(InMaxLevelFilter),
CoreWorld(InCoreWorld),bSkipUvAndColors(bInSkipUvAndColors),bSkipMaterialGroups(bInSkipMaterialGroups), CoreWorld(InCoreWorld),bSkipUvAndColors(bInSkipUvAndColors),bSkipMaterialGroups(bInSkipMaterialGroups),
bSkipUV(bInSkipUV),bSkipColors(bInSkipColors), bSkipUV(bInSkipUV),bSkipColors(bInSkipColors),
bCaptureColumnCoordinates(bInCaptureColumnCoordinates), bCaptureColumnCoordinates(bInCaptureColumnCoordinates),
bSharedColumnCache(bInSharedColumnCache), bSharedColumnCache(bInSharedColumnCache),
Lod8HorizonInteriorStepVoxels(std::max(0,InLod8HorizonInteriorStepVoxels)), Lod8HorizonInteriorStepVoxels(std::max(0,InLod8HorizonInteriorStepVoxels))
CoreTileRoute(std::move(InCoreTileRoute))
{ {
VoxelGenLOD::InitializeConsoleSwitches(); VoxelGenLOD::InitializeConsoleSwitches();
UVoxelMarchingCubesMesher::InitializeConsoleSwitches(); UVoxelMarchingCubesMesher::InitializeConsoleSwitches();
@@ -659,12 +655,17 @@ public:
{bProfileMesherStages=Enabled;} {bProfileMesherStages=Enabled;}
void SetUniformBlockProofAuditEnabled(bool Enabled) void SetUniformBlockProofAuditEnabled(bool Enabled)
{bAuditUniformBlockProofs=Enabled;} {bAuditUniformBlockProofs=Enabled;}
void SetDisplayAttributeCaptureEnabled(bool Enabled)
{bCaptureDisplayAttributes=Enabled;}
FStandaloneCpuTile GenerateDiagnostic(const FVoxelTileKey& Key, FStandaloneCpuTile GenerateDiagnostic(const FVoxelTileKey& Key,
std::uint32_t GlobalOctaveDrop=0,std::uint32_t CellStride=1, std::uint32_t GlobalOctaveDrop=0,std::uint32_t CellStride=1,
bool ForceGenerate=false) bool ForceGenerate=false,
const VoxelForgeCore::TileGenerationOptions* FarDisplayOverride=nullptr,
bool CaptureDisplayAttributes=false)
{ {
FStandaloneTileJob Job;Job.Key=Key;Job.StartedAt=FClock::now();Job.QueuedAt=Job.StartedAt; FStandaloneTileJob Job;Job.Key=Key;Job.StartedAt=FClock::now();Job.QueuedAt=Job.StartedAt;
return Generate(Job,GlobalOctaveDrop,CellStride,ForceGenerate); return Generate(Job,GlobalOctaveDrop,CellStride,ForceGenerate,FarDisplayOverride,
CaptureDisplayAttributes);
} }
private: private:
@@ -784,7 +785,9 @@ private:
} }
FStandaloneCpuTile Generate(const FStandaloneTileJob& Job, FStandaloneCpuTile Generate(const FStandaloneTileJob& Job,
std::uint32_t DiagnosticGlobalOctaveDrop=0,std::uint32_t DiagnosticCellStride=1, std::uint32_t DiagnosticGlobalOctaveDrop=0,std::uint32_t DiagnosticCellStride=1,
bool ForceGenerate=false) bool ForceGenerate=false,
const VoxelForgeCore::TileGenerationOptions* FarDisplayOverride=nullptr,
bool CaptureDisplayAttributes=false)
{ {
const auto StartTime=Job.StartedAt; const auto StartTime=Job.StartedAt;
FStandaloneCpuTile Out;Out.Key=Job.Key;Out.RequestedCrossing=Job.RequestedCrossing; FStandaloneCpuTile Out;Out.Key=Job.Key;Out.RequestedCrossing=Job.RequestedCrossing;
@@ -835,12 +838,21 @@ private:
Options.diagnosticCellStride=std::max<std::uint32_t>(1,DiagnosticCellStride); Options.diagnosticCellStride=std::max<std::uint32_t>(1,DiagnosticCellStride);
Options.diagnosticColumnInterpolationStride= Options.diagnosticColumnInterpolationStride=
std::max<std::uint32_t>(1,DiagnosticCellStride); std::max<std::uint32_t>(1,DiagnosticCellStride);
if(FarDisplayOverride)
{
Options.overrideFarDisplayMesh=FarDisplayOverride->overrideFarDisplayMesh;
Options.farDisplayGeometryBudgetMillipixels=FarDisplayOverride->farDisplayGeometryBudgetMillipixels;
Options.farRefinementGridScale=FarDisplayOverride->farRefinementGridScale;
Options.farRefinementLodMask=FarDisplayOverride->farRefinementLodMask;
Options.farConnectorStationScale=FarDisplayOverride->farConnectorStationScale;
Options.farConnectorRayMask=FarDisplayOverride->farConnectorRayMask;
Options.farConnectorReuseStationRadii=FarDisplayOverride->farConnectorReuseStationRadii;
Options.diagnosticRemoveFarConnectorPanels=FarDisplayOverride->diagnosticRemoveFarConnectorPanels;
Options.diagnosticAuditFarConnectorCoverage=FarDisplayOverride->diagnosticAuditFarConnectorCoverage;
}
std::string Error; std::string Error;
const VoxelForgeCore::TileKey CoreKey{{Job.Key.Coord.X,Job.Key.Coord.Y,Job.Key.Coord.Z},Job.Key.Level}; auto Result=CoreWorld->generateTile(
bool bDiskCacheHit=false; {{Job.Key.Coord.X,Job.Key.Coord.Y,Job.Key.Coord.Z},Job.Key.Level},Options,Error);
auto Result=CoreTileRoute
?CoreTileRoute(*CoreWorld,CoreKey,Options,bDiskCacheHit,Error)
:CoreWorld->generateTile(CoreKey,Options,Error);
if(!Result) if(!Result)
{ {
std::fprintf(stderr,"[VoxelForgeCore] tile generation failed: %s\n",Error.c_str()); std::fprintf(stderr,"[VoxelForgeCore] tile generation failed: %s\n",Error.c_str());
@@ -851,6 +863,7 @@ private:
Out.DensitySamples=Result->densitySampleCount; Out.DensitySamples=Result->densitySampleCount;
Out.CoreDiagnostics=Result->diagnostics; Out.CoreDiagnostics=Result->diagnostics;
Out.SurfaceVertexCount=Result->groups.surfaceVertexCount; Out.SurfaceVertexCount=Result->groups.surfaceVertexCount;
Out.CeilingTriangleCount=Result->groups.ceilingTriangleCount;
Out.SkirtTriangleCount=Result->groups.skirtTriangleCount; Out.SkirtTriangleCount=Result->groups.skirtTriangleCount;
Out.MeshPayloadAllocatedBytes=Result->meshPayloadAllocatedBytes; Out.MeshPayloadAllocatedBytes=Result->meshPayloadAllocatedBytes;
Out.DensityHash=Result->densityHash;Out.MeshHash=Result->meshHash; Out.DensityHash=Result->densityHash;Out.MeshHash=Result->meshHash;
@@ -858,12 +871,18 @@ private:
Out.bHasDensityHash=Result->hasDensityHash;Out.bHasMeshHash=Result->hasMeshHash; Out.bHasDensityHash=Result->hasDensityHash;Out.bHasMeshHash=Result->hasMeshHash;
Out.bHasPayloadHash=Result->hasPayloadHash;Out.bHasTopologyHash=Result->hasTopologyHash; Out.bHasPayloadHash=Result->hasPayloadHash;Out.bHasTopologyHash=Result->hasTopologyHash;
Out.Vertices.reserve(Result->vertices.size()); Out.Vertices.reserve(Result->vertices.size());
const bool bCaptureAttributes=CaptureDisplayAttributes||bCaptureDisplayAttributes;
if(bCaptureAttributes)Out.VertexColors.reserve(Result->vertices.size());
for(const auto& Vertex:Result->vertices) for(const auto& Vertex:Result->vertices)
{
Out.Vertices.push_back({ Out.Vertices.push_back({
{static_cast<float>(Vertex.position.x*0.01),static_cast<float>(Vertex.position.y*0.01), {static_cast<float>(Vertex.position.x*0.01),static_cast<float>(Vertex.position.y*0.01),
static_cast<float>(Vertex.position.z*0.01)}, static_cast<float>(Vertex.position.z*0.01)},
{static_cast<float>(Vertex.normal.x),static_cast<float>(Vertex.normal.y), {static_cast<float>(Vertex.normal.x),static_cast<float>(Vertex.normal.y),
static_cast<float>(Vertex.normal.z)}}); static_cast<float>(Vertex.normal.z)}});
if(bCaptureAttributes)
Out.VertexColors.push_back({Vertex.color.r,Vertex.color.g,Vertex.color.b,Vertex.color.a});
}
Out.Indices=std::move(Result->indices); Out.Indices=std::move(Result->indices);
Out.TriangleSkirtFaceMasks=std::move(Result->skirtFaceMasks); Out.TriangleSkirtFaceMasks=std::move(Result->skirtFaceMasks);
if(bProfile) if(bProfile)
@@ -1010,9 +1029,9 @@ private:
bool bSkipUV=false; bool bSkipUV=false;
bool bSkipColors=false; bool bSkipColors=false;
bool bCaptureColumnCoordinates=false; bool bCaptureColumnCoordinates=false;
bool bCaptureDisplayAttributes=false;
bool bSharedColumnCache=true; bool bSharedColumnCache=true;
int32 Lod8HorizonInteriorStepVoxels=0; int32 Lod8HorizonInteriorStepVoxels=0;
FStandaloneCoreTileRoute CoreTileRoute;
bool bProfileMesherStages=false; bool bProfileMesherStages=false;
bool bAuditUniformBlockProofs=false; bool bAuditUniformBlockProofs=false;
std::atomic<uint64> CacheHitCount{0}; std::atomic<uint64> CacheHitCount{0};
Standalone/Source/WorldAssemblyPrototype.h
+29
View File
@@ -30,6 +30,34 @@ namespace StandaloneWorldAssemblyPrototype
return std::max<std::int64_t>(1,WorldAssemblyFixed(wavelength)); return std::max<std::int64_t>(1,WorldAssemblyFixed(wavelength));
} }
inline bool ApplyFarDisplayMeshDeclaration(const VFJson::Value& root,
VoxelForgeCore::WorldAssemblyRecipeV1& recipe,std::string& error)
{
const VFJson::Value& declaration=root.Get("far_display_mesh");
if(std::holds_alternative<std::nullptr_t>(declaration.Data))return true;
if(!declaration.IsObject())
{error="far_display_mesh must be an object when declared";return false;}
const auto ReadUnsigned=[&](const char* key,std::uint32_t& output)
{
const VFJson::Value& value=declaration.Get(key);
if(!value.IsNumber()||!std::isfinite(value.Number())||value.Number()<0.0
||value.Number()>static_cast<double>(UINT32_MAX)
||std::floor(value.Number())!=value.Number())return false;
output=static_cast<std::uint32_t>(value.Number());return true;
};
if(!ReadUnsigned("geometry_budget_millipixels",recipe.farDisplayGeometryBudgetMillipixels)
||!ReadUnsigned("refinement_grid_scale",recipe.farRefinementGridScale)
||!ReadUnsigned("refinement_lod_mask",recipe.farRefinementLodMask)
||!ReadUnsigned("connector_station_scale",recipe.farConnectorStationScale)
||!ReadUnsigned("connector_ray_mask",recipe.farConnectorRayMask))
{error="far_display_mesh declaration has a missing or non-integer parameter";return false;}
const VFJson::Value& reuse=declaration.Get("connector_reuse_station_radii");
if(!std::holds_alternative<bool>(reuse.Data))
{error="far_display_mesh connector_reuse_station_radii must be boolean";return false;}
recipe.farConnectorReuseStationRadii=std::get<bool>(reuse.Data);
return true;
}
inline uint64 WorldAssemblyFeatureId(uint32 seed,int64 x,int64 y,uint32 kind) inline uint64 WorldAssemblyFeatureId(uint32 seed,int64 x,int64 y,uint32 kind)
{ {
uint64 hash=14695981039346656037ull; uint64 hash=14695981039346656037ull;
@@ -223,6 +251,7 @@ namespace StandaloneWorldAssemblyPrototype
recipe.biomeClimateReliefWavelengthSubvoxels=WorldAssemblyWavelength(map.ReliefFrequency,3333.0); recipe.biomeClimateReliefWavelengthSubvoxels=WorldAssemblyWavelength(map.ReliefFrequency,3333.0);
recipe.biomeClimateMoistureWavelengthSubvoxels=WorldAssemblyWavelength(map.MoistureFrequency,2500.0); recipe.biomeClimateMoistureWavelengthSubvoxels=WorldAssemblyWavelength(map.MoistureFrequency,2500.0);
recipe.biomeClimateReliefContrastQ16=WorldAssemblyQ16(map.ReliefContrast); recipe.biomeClimateReliefContrastQ16=WorldAssemblyQ16(map.ReliefContrast);
if(!ApplyFarDisplayMeshDeclaration(root,recipe,error))return false;
const auto makeProfile=[&](uint32 stableId,const FSurfaceGenerationParams& params, const auto makeProfile=[&](uint32 stableId,const FSurfaceGenerationParams& params,
const FBiomeResolved* climate) const FBiomeResolved* climate)
{ {