Fr0zka/VoxelForge
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

perf(mesher): adjacent tiles were re-evaluating 18% of the field -- 2.69s to 1.49s

Browse Source 
Overnight task 2. Every claim here is end-to-end wall clock on a real 128^3 export,
not a profiled bucket, because the last two rounds shipped large bucket wins worth
0% in reality. Measurement floor is ~17% run-to-run; only Part 2 cleared it.

PART 2 -- KEPT. Density-grid reuse.
The mesher samples an expanded grid (-1..Cells+1), so adjacent tiles re-evaluated
their shared boundary planes. Measured: 2,744,000 requested samples contained
495,909 duplicates -- 18.07% of the field generated twice. A run-scoped read-only
lattice (FVoxelSharedDensityGrid) owns each world sample once and hands it to both
adjacent tile grids. Duplicates now 0.

  128^3 TunnelNetwork seed 1337 slot 1, freshly built binary:
    before  2.603 / 2.693 / 2.697 s
    after   1.488 / 1.499 / 1.486 s      -44.6%, 1.80x

  Geometry SHA-256 98eaad61..d3c4f IDENTICAL to the pre-change binary's output.
  153,740 triangles either way. Pure win, no field change.

The lattice is deliberately not a UObject and not a cache with eviction: ownership,
lifetime and iteration order stay explicit. A malformed or incomplete lattice is
ignored and falls back to the canonical generator path rather than altering geometry.

PART 1 -- REVERTED. Block-level empty-space early-out.
**0 of 4,096 blocks were safely classifiable**, and the attempt made it 85% slower
(3.632 -> 6.735 s). No unsafe bound was retained.
⇒ This is a finding, not just a failure: our conservative bounds are too loose to
prove ANY 8^3 block uniform. Hierarchical evaluation of any kind is blocked until
we know which operator has effectively unbounded reach. Next task.

PART 3 -- REVERTED. Task granularity.
15 workers, 64 tile jobs, mean job 0.594 s, launch fringe ~14.1 us. MinBatchSize=4
was 1.09% SLOWER end-to-end despite a better bucket time -- exactly the trap this
task's rule exists to catch. Default stays 1. ParallelForWithExistingTaskContext
not adopted.

PART 4 -- NO CHANGE. Renderer audited over 3 runs (8.736 s total, 1.483 s raster).
Mesh and BVH are already reused across viewpoints; no per-pixel heap allocation, no
redundant remeshing. Nothing gross to fix; it benefits from the above automatically.

STILL OPEN: the spatial index from ee89c4d has never earned its keep in wall clock
and accounts for ~214 MB of measured allocations. Codex proposes an on/off ablation
then removal if confirmed. Not done unilaterally.

⚠️ PROCESS NOTE: the agent reported "build passed" but the runtime DLL was still
from 03:07 while sources were edited at 05:22/05:42. My first three verification
runs measured the OLD binary and showed no change. Always check binary timestamps
against source timestamps before believing a measurement.

Cumulative on this region: 342.2 s -> 16.2 s -> 2.62 s -> 1.49 s (230x).
Determinism: identical hashes across all repeat runs. Build clean. Nothing outside
Plugins/VoxelForge touched; RealtimeMeshComponent untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
This commit is contained in:
Fr0zkaandClaude Opus 5 committed 2026-09-09 05:59:56 +02:00
1 parent e63f016835
commit 2f33870a84
6 files changed
+488 -8
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

CODEX-TASK-2-PERF-REPORT.md
+140
View File
@@ -0,0 +1,140 @@
# Task 2 performance report
Date: 2026-09-09
Repository: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge`
Work began at `experimental` / `e63f016`.
## Measurement rule
All performance numbers below are per-case `run.elapsed_seconds` from the batch report. This is
the wall clock for a real commandlet case through the export or render result; it includes the
shared-density-grid build when enabled. `summary.mesh_seconds` and profiler buckets are included
only as diagnostics, never as the performance claim.
The export comparison is the same `128^3` `TunnelNetwork`, seed `1337`, slot `1`, operator stack,
step `1`, with three cases in one commandlet process on each side. The 17% reporting floor is
therefore met by both run count and the retained density-grid result.
## Part 1 — block early-out
I tested an `8^3` block reject behind conservative fallback logic. On this cave/stack world, every
one of the 4,096 blocks was left uncertain and went through normal sampling:
| batch | runs | end-to-end wall seconds | blocks skipped | skip rate | result |
| --- | ---: | --- | ---: | ---: | --- |
| baseline | 3 | 3.805057, 3.522129, 3.569186 (avg 3.632124) | — | — | reference |
| block reject | 3 | 6.341454, 6.562621, 7.300234 (avg 6.734770) | 0 / 4,096 | 0.00% | **reverted** |
The candidate added classification work without skipping a block (+85.42% wall clock). It also
did not produce a useful supremum for the complete cave stack, so I did not promote a guessed
`max` parameter into a topology decision. `ClassifyTile` remains untouched: its stack, disturbance,
diff-layer, and cave guards are not a safe substitute for a new block proof. A false AllSolid
verdict here would delete a 3D cave, which is not an acceptable trade.
This did not clear the floor and was reverted.
## Part 2 — density-grid reuse
The canonical mesher requests 35^3 samples per 32-cell tile. For 64 tiles that is:
- tile requests: `64 * 35^3 = 2,744,000`;
- unique points in the expanded 128-cell lattice: `131^3 = 2,248,091`;
- duplicate evaluations: `495,909` (18.07%).
The first implementation evaluated the shared lattice as Z planes. It removed all duplicates but
was slower because it destroyed the generator's spatial locality:
| candidate | runs | end-to-end wall seconds | duplicates | result |
| --- | ---: | --- | --- | --- |
| matched control | 3 | 3.668954, 3.633445, 3.655658 (avg 3.652686) | 495,909 → 495,909 | reference |
| global-plane precompute | 3 | 5.349520, 5.352927, 5.510752 (avg 5.404400) | 495,909 → 0 | **reverted** |
| owner-grid precompute | 3 | 1.950207, 1.973673, 2.161305 (avg 2.028395) | 495,909 → 0 | retained candidate |
The retained implementation partitions the shared lattice into 64 disjoint rectangular owner
regions. Each global sample is generated exactly once, then each tile copies its local halo window
from the immutable run-scoped grid. The precompute is deliberately inside the wall-clock timer;
the final three-case repeat measured:
| batch | runs | end-to-end wall seconds | average |
| --- | ---: | --- | ---: |
| final control (`density_grid_reuse=false`) | 3 | 3.659587, 3.619013, 3.582257 | 3.620286 s |
| final reuse (`density_grid_reuse=true`) | 3 | 1.908017, 1.854344, 1.884170 | 1.882177 s |
That is a **48.01% wall-clock reduction / 1.92x speedup**, clearing the 17% floor. The final
reuse runs averaged 1.474271 s for the shared-grid build, and still won end to end. The 128^3
shared lattice is about 8.6 MiB of float storage. Direct explorer runs default to reuse; a batch
case can pass `density_grid_reuse=false` for a matched control.
The run-scoped handoff is confined to the explorer's finite canonical export. The streaming game
path has no one-shot set of neighboring tiles or invalidation boundary, so it was not given a
locking plane cache that could serialize workers or outlive a generation epoch.
## Part 3 — task granularity
The final control observed 15 worker thread IDs and 64 logical tile invocations. Across its three
runs, the measured tile-job averages were:
| metric | average |
| --- | ---: |
| fastest logical tile job | 0.188817 s |
| mean logical tile job | 0.593751 s |
| slowest logical tile job | 0.955788 s |
| sum of tile-job durations | 38.000045 s |
| tile parallel wall | 3.150925 s |
| launch-fringe estimate | 14.1 us |
| workers / logical jobs | 15 / 64 |
I tested `MinBatchSize=4` with three matched cases: wall seconds `3.590884, 3.634734,
3.753986` (avg `3.659868`) versus the final control average `3.620286`. The tile parallel bucket
was slightly lower (`3.125197` s vs `3.150925` s), but end-to-end wall clock was **1.09% slower**.
The observed worker count and logical job count stayed 15 / 64, and the launch-fringe estimate
remained about 13.1 us. This is the required bucket-win/no-wall-win case: the tuning was reverted;
the production/default setting remains `MinBatchSize=1`.
`ParallelForWithExistingTaskContext` was not adopted. The measured launch fringe is negligible
relative to 0.59 s tile work, and the mesher already uses thread-local scratch storage. No claim is
made here about the separate game workload of many small streaming submissions.
## Part 4 — renderer
No renderer rewrite or RMC change was justified. The source audit and three final render cases
showed:
- the canonical mesh is built once per explorer world;
- the triangle cache/BVH is built once and reused by all eight viewpoints;
- each viewpoint owns one `FColor` output allocation;
- the per-pixel traversal uses a fixed stack array, not a heap allocation;
- camera movement does not call the mesher again.
The three 512x288 render cases had total case wall seconds `8.704730, 8.985744, 8.517606`
(avg `8.736027`), including camera-seed walking. Raster-only diagnostics averaged 1.482921 s
for eight views, 0.286297 s for the first view, 0.170946 s per additional view, and 0.083190 s
for one-time acceleration construction. This is consistent with the owner's approximately
0.15 s/view observation; meshing benefits from Part 2 automatically. There was no renderer
change, so there is no before/after performance claim to report or keep.
## Memory/index follow-up
The existing profiler research measured 431,162,116 bytes (411.2 MiB) across 15 workers in the
large run. In the representative 128^3 breakdown, spatial-index bins and candidate arrays alone
accounted for 214,345,644 bytes, inside a 360,063,124-byte cache total. The index addition did not
show an end-to-end wall-clock gain in the prior measurements.
I did not remove it silently. The explicit follow-up proposal is an index-on/index-off batch
ablation before deletion: removing it should recover roughly that allocation, at the cost of
broader room/tunnel candidate scans and potentially slower density queries. That trade is not
proven by this task's export comparison.
## Determinism, invariants, and scope
All three final control hashes and all three final reuse hashes are `ED0F1F31`; every case reported
`json_deterministic=yes`. The three render cases also repeated the same geometry hash
`847BE4B9`. Owner regions are disjoint, tile results are merged in fixed Z/Y/X order, no RNG or
runtime CPU-feature dispatch was added, and the shared grid is read-only after its deterministic
fill. The only worker writes are to distinct cells of that run-owned output lattice; generator and
world state remain read-only.
The build succeeded with UnrealBuildTool after the final source change. Changes are limited to the
VoxelForge source and this report, with generated evidence under the plugin's `Saved` directory.
RealtimeMeshComponent was not modified. No VoxelM root, sibling plugin, `Content`, or `Config`
files were written; no links or historical worktrees were created.
Source/VoxelForge/Private/VoxelGenerator.cpp
+20
View File
@@ -178,6 +178,26 @@ static thread_local FSurfaceColumnCache GSurfColCache;
thread_local int32 VoxelGenLOD::OctaveBias = 0;
thread_local int32 VoxelGenLOD::SampleStep = 1;
int32 VoxelGenLOD::GetThreadOctaveBias()
{
return OctaveBias;
}
void VoxelGenLOD::SetThreadOctaveBias(int32 Value)
{
OctaveBias = Value;
}
int32 VoxelGenLOD::GetThreadSampleStep()
{
return SampleStep;
}
void VoxelGenLOD::SetThreadSampleStep(int32 Value)
{
SampleStep = Value;
}
// NOTE (T2.a): the fBm/Ridged bodies moved to VoxelNoise.h, where octaves are evaluated
// 4-wide via SSE (Perlin3D_x4). These thin wrappers keep every call site unchanged. They
// sample a DIFFERENT (float hash-gradient) noise field than the old FMath::PerlinNoise3D,
Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp
+34 -3
View File
@@ -231,6 +231,26 @@ FVoxelMeshData UVoxelMarchingCubesMesher::GenerateMesh(FIntVector OriginVoxels,
// capacité, donc plus de malloc/free de ~170 Ko (35³ floats) par tuile.
static thread_local TArray<float> DensityGrid;
DensityGrid.SetNumUninitialized(MDim * MDim * MDim);
const FVoxelSharedDensityGrid* ReuseGrid = SharedDensityGrid;
const int32 SharedDeltaX = ReuseGrid
? (OriginVoxels.X - ReuseGrid->OriginVoxels.X) / Step : 0;
const int32 SharedDeltaY = ReuseGrid
? (OriginVoxels.Y - ReuseGrid->OriginVoxels.Y) / Step : 0;
const int32 SharedDeltaZ = ReuseGrid
? (OriginVoxels.Z - ReuseGrid->OriginVoxels.Z) / Step : 0;
const bool bUseSharedDensityGrid = ReuseGrid
&& ReuseGrid->Step == Step
&& ReuseGrid->Dim > 0
&& ReuseGrid->Samples.Num() == ReuseGrid->Dim * ReuseGrid->Dim * ReuseGrid->Dim
&& (OriginVoxels.X - ReuseGrid->OriginVoxels.X) % Step == 0
&& (OriginVoxels.Y - ReuseGrid->OriginVoxels.Y) % Step == 0
&& (OriginVoxels.Z - ReuseGrid->OriginVoxels.Z) % Step == 0
&& SharedDeltaX - 1 >= 0
&& SharedDeltaY - 1 >= 0
&& SharedDeltaZ - 1 >= 0
&& SharedDeltaX + GridDim < ReuseGrid->Dim
&& SharedDeltaY + GridDim < ReuseGrid->Dim
&& SharedDeltaZ + GridDim < ReuseGrid->Dim;
// Bande de strate : seules les rangées Z réellement lues (cellules CzLo..CzHi + marges de
// gradient) sont échantillonnées — le reste du tampon reste non initialisé et non lu.
VoxelDensityProfile::FScopedTimer MesherDensityGridTimer(
@@ -243,12 +263,23 @@ FVoxelMeshData UVoxelMarchingCubesMesher::GenerateMesh(FIntVector OriginVoxels,
if (ShouldAbortWork()) return FVoxelMeshData();
for (int32 gx = -1; gx <= GridDim; gx++)
{
// World voxel = tile origin + grid offset scaled by the cell size (Step).
DensityGrid[((gz + 1) * MDim + (gy + 1)) * MDim + (gx + 1)] =
Generator->GetDensityAt(
const int32 LocalIndex = ((gz + 1) * MDim + (gy + 1)) * MDim + (gx + 1);
if (bUseSharedDensityGrid)
{
const int32 SharedX = SharedDeltaX + gx;
const int32 SharedY = SharedDeltaY + gy;
const int32 SharedZ = SharedDeltaZ + gz;
DensityGrid[LocalIndex] = ReuseGrid->Samples[
(SharedZ * ReuseGrid->Dim + SharedY) * ReuseGrid->Dim + SharedX];
}
else
{
// World voxel = tile origin + grid offset scaled by the cell size (Step).
DensityGrid[LocalIndex] = Generator->GetDensityAt(
OriginVoxels.X + gx * Step,
OriginVoxels.Y + gy * Step,
OriginVoxels.Z + gz * Step);
}
}
}
}
Source/VoxelForge/Public/VoxelGenerator.h
+8 -2
View File
@@ -41,8 +41,9 @@ class UVoxelBiomeDefinition;
// the same thread, and biome/climate must stay LOD-independent.
namespace VoxelGenLOD
{
// NOT VOXELFORGE_API: MSVC forbids dll-interface on thread_local (C2492). Both users
// (generator + mesher) are inside this module, so no export is needed anyway.
// The TLS variables themselves stay module-local because MSVC forbids exporting thread_local
// data (C2492). These tiny accessors let the editor explorer build a shared LOD grid without
// linking directly to the storage in the game module.
extern thread_local int32 OctaveBias;
// Sampling step of the tile currently being meshed on this worker. GetDensityAt itself has
@@ -50,6 +51,11 @@ namespace VoxelGenLOD
// read-only hint for caches that may safely widen their spatial window for coarse tiles.
extern thread_local int32 SampleStep;
VOXELFORGE_API int32 GetThreadOctaveBias();
VOXELFORGE_API void SetThreadOctaveBias(int32 Value);
VOXELFORGE_API int32 GetThreadSampleStep();
VOXELFORGE_API void SetThreadSampleStep(int32 Value);
// Effective octave count for a per-voxel noise call site.
// At least 1 octave always survives (the coarse base shape).
FORCEINLINE int32 Eff(int32 Octaves) { return FMath::Max(1, Octaves - OctaveBias); }
Source/VoxelForge/Public/VoxelMarchingCubesMesher.h
+27
View File
@@ -17,6 +17,27 @@
#include "VoxelGenerator.h"
#include "VoxelMarchingCubesMesher.generated.h"
// A run-scoped, read-only density lattice. The explorer owns the storage and installs the
// pointer only while its canonical tiles are being meshed. The lattice includes the one-point
// halo needed by every tile, so each world sample is generated once and copied into both adjacent
// tile scratch grids. It is deliberately not a UObject or a cache with eviction: ownership,
// lifetime, and iteration order remain explicit and deterministic.
struct FVoxelSharedDensityGrid
{
FIntVector OriginVoxels = FIntVector::ZeroValue;
int32 Step = 1;
int32 Dim = 0;
TArray<float> Samples;
void Reset()
{
OriginVoxels = FIntVector::ZeroValue;
Step = 1;
Dim = 0;
Samples.Reset();
}
};
UCLASS(BlueprintType)
class VOXELFORGE_API UVoxelMarchingCubesMesher : public UObject
{
@@ -93,6 +114,11 @@ public:
// This is intentionally read-only: workers may observe shutdown, but never mutate world state.
void SetShutdownFlag(const std::atomic<bool>* InShutdownFlag) { ShutdownFlag = InShutdownFlag; }
// Optional run-scoped shared lattice. The caller must keep it alive and immutable until all
// GenerateMesh calls finish. A malformed/incomplete lattice is ignored and falls back to
// the canonical generator path rather than changing geometry.
void SetSharedDensityGrid(const FVoxelSharedDensityGrid* InGrid) { SharedDensityGrid = InGrid; }
private:
FORCEINLINE bool ShouldAbortWork() const
{
@@ -100,6 +126,7 @@ private:
}
const std::atomic<bool>* ShutdownFlag = nullptr;
const FVoxelSharedDensityGrid* SharedDensityGrid = nullptr;
public:
Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp
+259 -3
View File
@@ -6,6 +6,7 @@
#include "Containers/StringConv.h"
#include "HAL/FileManager.h"
#include "HAL/PlatformTime.h"
#include "HAL/PlatformTLS.h"
#include "IImageWrapper.h"
#include "IImageWrapperModule.h"
#include "Misc/CommandLine.h"
@@ -210,6 +211,10 @@ struct FExploreArguments
float RenderMaxDistanceVoxels = DefaultRenderMaxDistanceVoxels;
int32 ExportSize = DefaultExportSize;
int32 ExportStep = DefaultExportStep;
// The canonical export owns one immutable lattice for the whole run. Batch cases can pass
// density_grid_reuse=false to produce a matched wall-clock control.
bool bReuseDensityGrid = true;
int32 MeshMinBatchSize = 1;
int32 MaxWalkCells = DefaultMaxWalkCells;
float BudgetMinutes = DefaultBudgetMinutes;
bool bSurfaceRoughnessOverride = false;
@@ -267,6 +272,11 @@ bool ParseArguments(const FString& Params, FExploreArguments& OutArguments, FStr
FParse::Value(*Params, TEXT("rendermaxdistance="), OutArguments.RenderMaxDistanceVoxels);
FParse::Value(*Params, TEXT("exportsize="), OutArguments.ExportSize);
FParse::Value(*Params, TEXT("exportstep="), OutArguments.ExportStep);
int32 DensityGridReuse = OutArguments.bReuseDensityGrid ? 1 : 0;
FParse::Value(*Params, TEXT("densitygridreuse="), DensityGridReuse);
OutArguments.bReuseDensityGrid = DensityGridReuse != 0;
FParse::Value(*Params, TEXT("meshminbatch="), OutArguments.MeshMinBatchSize);
OutArguments.MeshMinBatchSize = FMath::Clamp(OutArguments.MeshMinBatchSize, 1, 64);
int32 Lod = 0;
if (FParse::Value(*Params, TEXT("lod="), Lod))
{
@@ -548,11 +558,27 @@ struct FExploreWorld
// One canonical mesh cache for this commandlet run. Render and export consume these exact
// triangles; neither path is allowed to ask the density field once per pixel.
FVoxelMeshData ExploreMesh;
FVoxelSharedDensityGrid ExploreSharedDensityGrid;
FIntVector ExploreMeshOrigin = FIntVector::ZeroValue;
int32 ExploreMeshSize = 0;
int32 ExploreMeshTileCount = 0;
int32 ExploreMeshTilesCompleted = 0;
double ExploreMeshSeconds = 0.0;
double ExploreDensityGridSeconds = 0.0;
int64 ExploreDensityGridTotalSamples = 0;
int64 ExploreDensityGridUniqueSamples = 0;
int64 ExploreDensityGridDuplicateSamplesBefore = 0;
int64 ExploreDensityGridDuplicateSamplesAfter = 0;
bool bExploreDensityGridReused = false;
int32 ExploreMeshTaskWorkerCount = 0;
int32 ExploreMeshTaskJobCount = 0;
int32 ExploreMeshTaskMinBatchSize = 1;
double ExploreMeshTaskMinSeconds = 0.0;
double ExploreMeshTaskMeanSeconds = 0.0;
double ExploreMeshTaskMaxSeconds = 0.0;
double ExploreMeshTaskSumSeconds = 0.0;
double ExploreMeshTaskWallSeconds = 0.0;
double ExploreMeshTaskLaunchOverheadSeconds = 0.0;
bool bExploreMeshBuilt = false;
bool bExploreMeshComplete = false;
FString ExploreMeshError;
@@ -1253,6 +1279,23 @@ bool EnsureExploreMesh(
FMath::FloorToInt(0.5f * static_cast<float>(
World.TargetBottomWorldZ + World.TargetTopWorldZ)));
World.ExploreMeshSeconds = 0.0;
World.ExploreDensityGridSeconds = 0.0;
World.ExploreDensityGridTotalSamples = 0;
World.ExploreDensityGridUniqueSamples = 0;
World.ExploreDensityGridDuplicateSamplesBefore = 0;
World.ExploreDensityGridDuplicateSamplesAfter = 0;
World.bExploreDensityGridReused = false;
World.ExploreMeshTaskWorkerCount = 0;
World.ExploreMeshTaskJobCount = 0;
World.ExploreMeshTaskMinBatchSize = FMath::Clamp(Arguments.MeshMinBatchSize, 1, 64);
World.ExploreMeshTaskMinSeconds = 0.0;
World.ExploreMeshTaskMeanSeconds = 0.0;
World.ExploreMeshTaskMaxSeconds = 0.0;
World.ExploreMeshTaskSumSeconds = 0.0;
World.ExploreMeshTaskWallSeconds = 0.0;
World.ExploreMeshTaskLaunchOverheadSeconds = 0.0;
World.Mesher->SetSharedDensityGrid(nullptr);
World.ExploreSharedDensityGrid.Reset();
if (Budget.ShouldStop(TEXT("mesh")))
{
@@ -1275,6 +1318,114 @@ bool EnsureExploreMesh(
return false;
}
// Adjacent tiles ask for the same one-point-halo planes. Build one immutable lattice for
// the whole export when requested, then let each tile copy its local window from it. The
// precomputation is inside MeshStartSeconds on purpose: this is an end-to-end export number,
// not a profiled bucket hiding the cost of producing the cache.
const int32 CellsPerTile = CHUNK_SIZE / Arguments.ExportStep;
const int32 PointsPerTile = CellsPerTile + 3;
const int64 TileGridSamples = static_cast<int64>(PointsPerTile)
* static_cast<int64>(PointsPerTile)
* static_cast<int64>(PointsPerTile);
const int32 SharedGridDim = Arguments.ExportSize / Arguments.ExportStep + 3;
const int64 SharedGridSamples = static_cast<int64>(SharedGridDim)
* static_cast<int64>(SharedGridDim)
* static_cast<int64>(SharedGridDim);
World.ExploreDensityGridTotalSamples = static_cast<int64>(World.ExploreMeshTileCount)
* TileGridSamples;
World.ExploreDensityGridUniqueSamples = SharedGridSamples;
World.ExploreDensityGridDuplicateSamplesBefore = FMath::Max<int64>(
0, World.ExploreDensityGridTotalSamples - SharedGridSamples);
World.ExploreDensityGridDuplicateSamplesAfter =
Arguments.bReuseDensityGrid ? 0 : World.ExploreDensityGridDuplicateSamplesBefore;
if (Arguments.bReuseDensityGrid)
{
const double DensityGridStartSeconds = FPlatformTime::Seconds();
World.ExploreSharedDensityGrid.OriginVoxels = World.ExploreMeshOrigin
- FIntVector(Arguments.ExportStep);
World.ExploreSharedDensityGrid.Step = Arguments.ExportStep;
World.ExploreSharedDensityGrid.Dim = SharedGridDim;
World.ExploreSharedDensityGrid.Samples.SetNumUninitialized(
SharedGridDim * SharedGridDim * SharedGridDim);
std::atomic<bool> bGridWorkCancelled(false);
const double GridWorkDeadline = Budget.StartSeconds + Budget.LimitSeconds - 0.25;
const int32 GridTileCount = TilesPerAxis;
ParallelFor(
TEXT("VoxelForgeExploreDensityGrid"),
World.ExploreMeshTileCount,
World.ExploreMeshTaskMinBatchSize,
[&](int32 LinearOwnerIndex)
{
if (bGridWorkCancelled.load(std::memory_order_relaxed)
|| FPlatformTime::Seconds() >= GridWorkDeadline)
{
bGridWorkCancelled.store(true, std::memory_order_relaxed);
return;
}
const int32 GridOctaveBias = (World.Mesher->LODOctaveDrop > 0
&& Arguments.ExportStep > 1)
? World.Mesher->LODOctaveDrop
* (int32)FMath::FloorLog2((uint32)Arguments.ExportStep)
: 0;
const int32 PreviousOctaveBias = VoxelGenLOD::GetThreadOctaveBias();
const int32 PreviousSampleStep = VoxelGenLOD::GetThreadSampleStep();
VoxelGenLOD::SetThreadOctaveBias(GridOctaveBias);
VoxelGenLOD::SetThreadSampleStep(Arguments.ExportStep);
const int32 OwnerX = LinearOwnerIndex % GridTileCount;
const int32 OwnerY = (LinearOwnerIndex / GridTileCount) % GridTileCount;
const int32 OwnerZ = LinearOwnerIndex / (GridTileCount * GridTileCount);
// Partition the shared lattice into disjoint rectangular owner regions. A
// boundary sample belongs to exactly one owner; tile windows may read it from
// either side. This keeps the generator's spatial locality while guaranteeing
// one evaluation per global lattice point.
const auto OwnerMinGrid = [CellsPerTile](int32 Owner) -> int32
{
return Owner == 0 ? 0 : 1 + Owner * CellsPerTile;
};
const auto OwnerMaxGrid = [CellsPerTile, SharedGridDim, GridTileCount](int32 Owner) -> int32
{
return Owner == GridTileCount - 1
? SharedGridDim
: 1 + (Owner + 1) * CellsPerTile;
};
const int32 GxMin = OwnerMinGrid(OwnerX);
const int32 GxMax = OwnerMaxGrid(OwnerX);
const int32 GyMin = OwnerMinGrid(OwnerY);
const int32 GyMax = OwnerMaxGrid(OwnerY);
const int32 GzMin = OwnerMinGrid(OwnerZ);
const int32 GzMax = OwnerMaxGrid(OwnerZ);
for (int32 Gz = GzMin; Gz < GzMax; ++Gz)
{
for (int32 Gy = GyMin; Gy < GyMax; ++Gy)
{
for (int32 Gx = GxMin; Gx < GxMax; ++Gx)
{
const int32 Index = (Gz * SharedGridDim + Gy) * SharedGridDim + Gx;
World.ExploreSharedDensityGrid.Samples[Index] =
World.Generator->GetDensityAt(
World.ExploreSharedDensityGrid.OriginVoxels.X + Gx * Arguments.ExportStep,
World.ExploreSharedDensityGrid.OriginVoxels.Y + Gy * Arguments.ExportStep,
World.ExploreSharedDensityGrid.OriginVoxels.Z + Gz * Arguments.ExportStep);
}
}
}
VoxelGenLOD::SetThreadOctaveBias(PreviousOctaveBias);
VoxelGenLOD::SetThreadSampleStep(PreviousSampleStep);
},
EParallelForFlags::None);
World.ExploreDensityGridSeconds = FPlatformTime::Seconds() - DensityGridStartSeconds;
if (bGridWorkCancelled.load(std::memory_order_relaxed))
{
World.ExploreMeshError = TEXT("The wall-clock budget elapsed while building the shared density grid.");
OutError = World.ExploreMeshError;
World.ExploreMeshSeconds = FPlatformTime::Seconds() - MeshStartSeconds;
return false;
}
World.Mesher->SetSharedDensityGrid(&World.ExploreSharedDensityGrid);
World.bExploreDensityGridReused = true;
}
// Generate each tile through the canonical mesher on worker threads, then append strictly in
// Z/Y/X order. The mesher's scratch buffers and the generator's hot caches are thread-local;
// the fixed merge order keeps the aggregate mesh and all downstream raster/export bytes
@@ -1285,10 +1436,21 @@ bool EnsureExploreMesh(
TileErrors.SetNum(World.ExploreMeshTileCount);
TArray<uint8> TileCompleted;
TileCompleted.Init(0, World.ExploreMeshTileCount);
TArray<double> TileDurations;
TileDurations.Init(0.0, World.ExploreMeshTileCount);
TArray<double> TileStartOffsets;
TileStartOffsets.Init(-1.0, World.ExploreMeshTileCount);
TArray<double> TileEndOffsets;
TileEndOffsets.Init(-1.0, World.ExploreMeshTileCount);
TArray<uint32> TileThreadIds;
TileThreadIds.Init(0, World.ExploreMeshTileCount);
std::atomic<bool> bTileWorkCancelled(false);
const double TileWorkDeadline = Budget.StartSeconds + Budget.LimitSeconds - 0.25;
const double ParallelStartSeconds = FPlatformTime::Seconds();
ParallelFor(
TEXT("VoxelForgeExploreMeshTiles"),
World.ExploreMeshTileCount,
World.ExploreMeshTaskMinBatchSize,
[&](int32 LinearTileIndex)
{
if (bTileWorkCancelled.load(std::memory_order_relaxed)
@@ -1298,6 +1460,10 @@ bool EnsureExploreMesh(
return;
}
const double TileStartSeconds = FPlatformTime::Seconds();
TileStartOffsets[LinearTileIndex] = TileStartSeconds - ParallelStartSeconds;
TileThreadIds[LinearTileIndex] = FPlatformTLS::GetCurrentThreadId();
const int32 TileX = LinearTileIndex % TilesPerAxis;
const int32 TileY = (LinearTileIndex / TilesPerAxis) % TilesPerAxis;
const int32 TileZ = LinearTileIndex / (TilesPerAxis * TilesPerAxis);
@@ -1313,10 +1479,54 @@ bool EnsureExploreMesh(
{
TileErrors[LinearTileIndex] = MoveTemp(TileError);
bTileWorkCancelled.store(true, std::memory_order_relaxed);
const double TileEndSeconds = FPlatformTime::Seconds();
TileEndOffsets[LinearTileIndex] = TileEndSeconds - ParallelStartSeconds;
TileDurations[LinearTileIndex] = TileEndSeconds - TileStartSeconds;
return;
}
TileCompleted[LinearTileIndex] = 1;
});
const double TileEndSeconds = FPlatformTime::Seconds();
TileEndOffsets[LinearTileIndex] = TileEndSeconds - ParallelStartSeconds;
TileDurations[LinearTileIndex] = TileEndSeconds - TileStartSeconds;
},
EParallelForFlags::None);
const double ParallelEndSeconds = FPlatformTime::Seconds();
TArray<double> CompletedTileDurations;
TSet<uint32> WorkerIds;
double EarliestTileStart = 1.0e30;
double LatestTileEnd = 0.0;
for (int32 TileIndex = 0; TileIndex < World.ExploreMeshTileCount; ++TileIndex)
{
if (!TileCompleted[TileIndex])
{
continue;
}
CompletedTileDurations.Add(TileDurations[TileIndex]);
WorkerIds.Add(TileThreadIds[TileIndex]);
EarliestTileStart = FMath::Min(EarliestTileStart, TileStartOffsets[TileIndex]);
LatestTileEnd = FMath::Max(LatestTileEnd, TileEndOffsets[TileIndex]);
}
if (CompletedTileDurations.Num() > 0)
{
CompletedTileDurations.Sort();
World.ExploreMeshTaskJobCount = CompletedTileDurations.Num();
World.ExploreMeshTaskWorkerCount = WorkerIds.Num();
World.ExploreMeshTaskMinSeconds = CompletedTileDurations[0];
World.ExploreMeshTaskMaxSeconds = CompletedTileDurations.Last();
for (const double Duration : CompletedTileDurations)
{
World.ExploreMeshTaskSumSeconds += Duration;
}
World.ExploreMeshTaskMeanSeconds = World.ExploreMeshTaskSumSeconds
/ static_cast<double>(CompletedTileDurations.Num());
}
World.ExploreMeshTaskWallSeconds = ParallelEndSeconds - ParallelStartSeconds;
if (EarliestTileStart < 1.0e30)
{
World.ExploreMeshTaskLaunchOverheadSeconds = FMath::Max(0.0, EarliestTileStart)
+ FMath::Max(0.0, World.ExploreMeshTaskWallSeconds - LatestTileEnd);
}
int32 CompletedTiles = 0;
FString MeshError;
@@ -3656,6 +3866,8 @@ FString BuildExploreJson(
Writer->WriteValue(TEXT("render_step_voxels"), static_cast<double>(Arguments.RenderStepVoxels));
Writer->WriteValue(TEXT("export_size"), Arguments.ExportSize);
Writer->WriteValue(TEXT("export_step"), Arguments.ExportStep);
Writer->WriteValue(TEXT("density_grid_reuse"), Arguments.bReuseDensityGrid);
Writer->WriteValue(TEXT("mesh_min_batch_size"), Arguments.MeshMinBatchSize);
Writer->WriteValue(TEXT("profile_density"), Arguments.bProfileDensity);
Writer->WriteValue(TEXT("profile_density_full"), Arguments.bProfileDensityFull);
Writer->WriteValue(TEXT("profile_lod"), Arguments.bProfileLod);
@@ -3693,6 +3905,26 @@ FString BuildExploreJson(
Writer->WriteValue(TEXT("us_per_voxel"), SummaryUsPerVoxel);
Writer->WriteValue(TEXT("triangle_count"), SummaryTriangleCount);
Writer->WriteValue(TEXT("geometry_hash"), SummaryGeometryHash);
Writer->WriteObjectStart(TEXT("density_grid_reuse"));
Writer->WriteValue(TEXT("enabled"), World.bExploreDensityGridReused);
Writer->WriteValue(TEXT("total_tile_grid_samples"), World.ExploreDensityGridTotalSamples);
Writer->WriteValue(TEXT("unique_shared_grid_samples"), World.ExploreDensityGridUniqueSamples);
Writer->WriteValue(TEXT("duplicate_evaluations_before"), World.ExploreDensityGridDuplicateSamplesBefore);
Writer->WriteValue(TEXT("duplicate_evaluations_after"), World.ExploreDensityGridDuplicateSamplesAfter);
Writer->WriteValue(TEXT("precompute_seconds"), World.ExploreDensityGridSeconds);
Writer->WriteObjectEnd();
Writer->WriteObjectStart(TEXT("mesher_tasks"));
Writer->WriteValue(TEXT("min_batch_size"), World.ExploreMeshTaskMinBatchSize);
Writer->WriteValue(TEXT("worker_count"), World.ExploreMeshTaskWorkerCount);
Writer->WriteValue(TEXT("job_count"), World.ExploreMeshTaskJobCount);
Writer->WriteValue(TEXT("job_min_seconds"), World.ExploreMeshTaskMinSeconds);
Writer->WriteValue(TEXT("job_mean_seconds"), World.ExploreMeshTaskMeanSeconds);
Writer->WriteValue(TEXT("job_max_seconds"), World.ExploreMeshTaskMaxSeconds);
Writer->WriteValue(TEXT("job_sum_seconds"), World.ExploreMeshTaskSumSeconds);
Writer->WriteValue(TEXT("parallel_wall_seconds"), World.ExploreMeshTaskWallSeconds);
Writer->WriteValue(TEXT("estimated_launch_overhead_seconds"),
World.ExploreMeshTaskLaunchOverheadSeconds);
Writer->WriteObjectEnd();
Writer->WriteObjectStart(TEXT("counters"));
for (int32 Index = 0; Index < VoxelDensityProfile::CounterCount; ++Index)
{
@@ -4013,6 +4245,20 @@ FString BuildExploreJson(
Writer->WriteValue(TEXT("mesh_seconds"), Output.Export.MeshSeconds);
Writer->WriteValue(TEXT("mesh_us_per_voxel"), Output.Export.MeshUsPerVoxel);
Writer->WriteValue(TEXT("geometry_hash"), Output.Export.GeometryHash);
Writer->WriteValue(TEXT("density_grid_reuse"), World.bExploreDensityGridReused);
Writer->WriteValue(TEXT("density_grid_precompute_seconds"), World.ExploreDensityGridSeconds);
Writer->WriteValue(TEXT("density_grid_duplicate_evaluations_before"),
World.ExploreDensityGridDuplicateSamplesBefore);
Writer->WriteValue(TEXT("density_grid_duplicate_evaluations_after"),
World.ExploreDensityGridDuplicateSamplesAfter);
Writer->WriteValue(TEXT("mesh_task_min_batch_size"), World.ExploreMeshTaskMinBatchSize);
Writer->WriteValue(TEXT("mesh_task_worker_count"), World.ExploreMeshTaskWorkerCount);
Writer->WriteValue(TEXT("mesh_task_job_count"), World.ExploreMeshTaskJobCount);
Writer->WriteValue(TEXT("mesh_task_job_mean_seconds"), World.ExploreMeshTaskMeanSeconds);
Writer->WriteValue(TEXT("mesh_task_job_max_seconds"), World.ExploreMeshTaskMaxSeconds);
Writer->WriteValue(TEXT("mesh_task_parallel_wall_seconds"), World.ExploreMeshTaskWallSeconds);
Writer->WriteValue(TEXT("mesh_task_launch_overhead_seconds"),
World.ExploreMeshTaskLaunchOverheadSeconds);
Writer->WriteValue(TEXT("export_step"), Arguments.ExportStep);
Writer->WriteValue(TEXT("truncated"), Output.Export.bTruncated);
Writer->WriteValue(TEXT("canonical_mesher_tiles_completed"), Output.Export.MesherTilesCompleted);
@@ -4853,6 +5099,14 @@ bool BuildBatchCaseParams(
ExportStep = 1 << Lod;
}
bool bReuseDensityGrid = true;
JsonBool(Case, TEXT("density_grid_reuse"), bReuseDensityGrid);
int32 MeshMinBatchSize = 1;
if (JsonNumber(Case, TEXT("mesh_min_batch_size"), Number))
{
MeshMinBatchSize = FMath::Clamp(FMath::RoundToInt(Number), 1, 64);
}
FString Modes = TEXT("export");
JsonString(Case, TEXT("modes"), Modes);
bool bOperatorStack = true;
@@ -4869,9 +5123,11 @@ bool BuildBatchCaseParams(
OutParams = FString::Printf(
TEXT("-seed=%d -archetype=%s -slot=%d -modes=%s -opstack=%d "
"-exportsize=%d -exportstep=%d -failurefocus=%d -out=\"%s\""),
"-exportsize=%d -exportstep=%d -densitygridreuse=%d -meshminbatch=%d "
"-failurefocus=%d -out=\"%s\""),
Seed, *Archetype, Slot, *Modes, bOperatorStack ? 1 : 0,
ExportSize, ExportStep, bFailureFocus ? 1 : 0, *CaseOutDirectory);
ExportSize, ExportStep, bReuseDensityGrid ? 1 : 0, MeshMinBatchSize,
bFailureFocus ? 1 : 0, *CaseOutDirectory);
if (bProfileDensity)
{