wip(assembler): round L7 -- shared column cache across workers: whole view 16-19% faster, world unchanged
SharedColumnCache.h: a world-owned, bounded 4-way second-tier cache of exact column values (heightfield and floating islands), keyed by recipe/plan identity plus content hash, XY and island id. Lock-free seqlock reads; set-local spin lock on misses. The FLazy thread-local tier stays first. 83.6% of repeated column work was cross-worker; formula evaluations -60.9%. W12 L1 view: Luna 2.165 -> 1.754 s; reviewer interleaved 2.059 -> 1.722 s. 8 m crossing new tiles 82 -> 52 ms. +66 MiB peak RSS. All hashes (whole view, per-LOD, exact queries, islands seed 0) identical at W1/W8/W12, Clang off and GCC default. MSVC/UBT not compiled here. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CSFYgsdW8ejZSyiMf4qRSX
This commit is contained in:
1 parent
9898e057fd
commit
4b1108bf86
12 files changed
+712
-53
No files matched your search
@@ -732,6 +732,15 @@ driven by `EditorBrush*` props.
|
||||
its default member initialization was never observable; removing that generated TLS leaf avoids
|
||||
constructor work without changing a height or density. Integer queries still read the cached
|
||||
payload, and fractional queries still use the same direct locals.
|
||||
- **Core shared column cache (L7)**: `Heightfield2D` and FloatingIslands keep their `FLazy`
|
||||
worker-local caches as the first tier. A world-owned, fixed-size four-way table is the second
|
||||
tier, keyed by recipe and plan identity plus content hash, exact XY, and an optional exact
|
||||
sub-domain identity (FloatingIslands uses the island stable ID). Hits read versioned atomic
|
||||
entries without taking a lock; misses take only their set's spin lock, recheck, compute once, and
|
||||
publish. The tables are bounded and die with the world after its tile work has drained. This
|
||||
deliberately adds shared reads on column-cache hits while preserving the no-lock hot-path rule;
|
||||
S14's purely worker-local cache remains the first lookup. Keep large worker-local storage behind
|
||||
`FLazy`, and do not make cache values or keys depend on engine objects.
|
||||
- **Sampler unknown-source audit**: the earlier plugin-side `FSurfaceColumn` constructor PCs no longer
|
||||
occur after the trivial-payload change. The largest remaining LOD0 unknown cluster in the final
|
||||
sampler raw CSV is `UnrealEditor-Core.dll + RVA 0xC080` (runtime PCs around
|
||||
@@ -1797,6 +1806,23 @@ page was added: cross-worker repeats alone do not establish a net page-cache win
|
||||
immutable-page lifetime rules would add ownership and memory cost. Round 18 changed no output or
|
||||
recipe version. See [`Standalone/Research/WorldAssemblerM3/Round18/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round18/REPORT.md).
|
||||
|
||||
### 8.35 Shared Core column cache — World Assembler L7
|
||||
|
||||
L7 measured 1.34 million repeated formula evaluations on the W12 L1 view before sharing: 83.6%
|
||||
were another worker's XY, 4.0% were same-worker repeats after eviction, and 12.0% were halo/normal
|
||||
repeats. The world-owned shared second tier cut measured formula executions by 60.9% and improved
|
||||
median whole-view-ready time by 19.0%; moving 8 m and generating 144 entering tiles improved median
|
||||
wall time by 36.1%. Peak RSS rose by about 66 MiB. The cache is a bounded exact-value table rather
|
||||
than published pages; no scheduling affinity or page prefetch was needed to clear the measured
|
||||
whole-view threshold.
|
||||
|
||||
The thread-local first tier remains because it avoids shared atomics for local hits. The shared
|
||||
table has no lock on hits; only a miss takes a set-local spin lock. FloatingIslands' cache identity
|
||||
includes its island stable ID because distinct anchors can request the same XY with different
|
||||
column metrics. All W1/W12 L1 hashes, exact queries, and the seed-0 FloatingIslands fixture match
|
||||
the pre-L7 binary with Clang FP contraction off and GCC defaults. See
|
||||
[`Standalone/Research/WorldAssemblerM3/Linux7/REPORT.md`](Standalone/Research/WorldAssemblerM3/Linux7/REPORT.md).
|
||||
|
||||
## 9. Multiplayer model (listen-server first, dedicated-friendly)
|
||||
|
||||
> **Status: transport is DESIGN ONLY — nothing is networked in-tree yet** (no `Replicated`/`HasAuthority`/RPCs; a
|
||||
|
||||
+2
-1
@@ -158,7 +158,8 @@ remains `0`, and lateral regions remain gated off.
|
||||
| `VoxelForge.Build.cs` | Deps: Core, CoreUObject, Engine, **GameplayTags**, **RealtimeMeshComponent**, `VoxelForgeCore`. `AVoxelWorld::bUseVoxelForgeCore` opts into the Core runtime route; it defaults off. |
|
||||
| `../../VoxelForgeCore/VoxelForgeCore.Build.cs` | C++20 runtime module. Reads `Standalone/Core/core_sources.txt`, validates one include shim per canonical source, and fails UBT on missing, extra, or mismatched shims. No unity/PCH; private copied implementation stays under `vfcore::detail`. |
|
||||
| `../../VoxelForgeCore/Private/Manifest/VFCore_*.cpp` | Eighteen one-source UBT shims that include canonical files under `Standalone/`; they do not copy Core implementation. The API-boundary row includes `Core/src/VoxelForgeCore.cpp` directly. |
|
||||
| `../../Standalone/Core/include/VoxelForgeCore/ThreadLocalStorage.h` | `VoxelForgeCoreThreadLocal::FLazy<T>`: thread-local handle that heap-allocates a worker-private cache on first use. All large Core and legacy caches use it (HeightfieldColumnCache, IslandColumnCache, GFloorReliefCache, GRoomGraphCache, GSurfColCache, tunnel/diff caches). See ARCHITECTURE §8.10. |
|
||||
| `../../Standalone/Core/include/VoxelForgeCore/ThreadLocalStorage.h` | `VoxelForgeCoreThreadLocal::FLazy<T>`: thread-local handle that heap-allocates a worker-private cache on first use. Large worker-local caches use it (HeightfieldColumnCache, IslandColumnCache, GFloorReliefCache, GRoomGraphCache, GSurfColCache, tunnel/diff caches). See ARCHITECTURE §8.10. |
|
||||
| `../../Standalone/Core/include/VoxelForgeCore/SharedColumnCache.h` | Bounded four-way second-tier cache for exact per-column values. Readers use versioned atomic slots without a lock; misses use a set-local spin lock. World-owned lifetime, recipe/plan identity and content hashes, XY and optional sub-domain identity. SurfaceWorld and FloatingIslands share this mechanism; see ARCHITECTURE §8.35. |
|
||||
| `../../Standalone/Core/src/CoreNamespacePreamble.h` | First include of every Core TU. Forces FP contraction off per compiler (Clang/MSVC/GCC pragmas; unknown compiler = `#error`) and defines `vfcore::fp_contract::selfTest`, which `World::compile` runs before building: a fused multiply-add (e.g. Clang `-ffp-contract=fast`) refuses world generation. The world depends on contraction (Linux2/Linux3 reports), so never remove either. |
|
||||
| `../../VoxelForgeCore/Private/ModuleEntry/` | Unreal module entry and an isolated `CoreMinimal.h` shim for the actual engine header. |
|
||||
| `../../Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h` | Standard-library-only public Core API. `VOXELFORGECORE_API` exports the API from the Unreal module and is empty for standalone builds. |
|
||||
|
||||
@@ -0,0 +1,183 @@
|
||||
#pragma once
|
||||
|
||||
#include <array>
|
||||
#include <atomic>
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
#include <cstring>
|
||||
#include <memory>
|
||||
#include <thread>
|
||||
#include <type_traits>
|
||||
#include <utility>
|
||||
|
||||
namespace VoxelForgeCoreShared
|
||||
{
|
||||
// Exact input identity for a pure XY column value. LOD is deliberately not part of this key:
|
||||
// the same recipe/plan column has the same bits at every sampling step.
|
||||
struct FColumnCacheKey
|
||||
{
|
||||
std::uint64_t recipeIdentity = 0;
|
||||
std::uint64_t recipeContentHash = 0;
|
||||
std::uint64_t planIdentity = 0;
|
||||
std::uint64_t planContentHash = 0;
|
||||
std::int64_t xSubvoxels = 0;
|
||||
std::int64_t ySubvoxels = 0;
|
||||
// Optional exact sub-domain identity for recipes with several independent columns
|
||||
// at the same XY (for example, a floating-island anchor).
|
||||
std::uint64_t contextIdentity = 0;
|
||||
|
||||
std::array<std::uint64_t, 7> words() const noexcept
|
||||
{
|
||||
return {recipeIdentity, recipeContentHash, planIdentity, planContentHash,
|
||||
static_cast<std::uint64_t>(xSubvoxels), static_cast<std::uint64_t>(ySubvoxels),
|
||||
contextIdentity};
|
||||
}
|
||||
};
|
||||
|
||||
// A bounded, world-owned second-level cache. Existing thread-local caches stay first, so
|
||||
// their ordinary hit path remains private and lock-free. Shared hits read atomic words; only
|
||||
// a shared miss takes the set spin lock. Entries are four-way set associative, evict on
|
||||
// collision, and carry pointer identities plus content hashes to reject stale reuse.
|
||||
// Les caches TLS restent le premier niveau; seul un defaut partage prend le verrou du groupe.
|
||||
template <typename TValue, std::size_t SetCount, std::size_t Ways = 4>
|
||||
class TSharedColumnValueCache
|
||||
{
|
||||
static_assert(SetCount > 0 && (SetCount & (SetCount - 1)) == 0,
|
||||
"column cache set count must be a power of two");
|
||||
static_assert(Ways > 0 && Ways <= 16, "unsupported column cache associativity");
|
||||
static_assert(std::is_trivially_copyable_v<TValue>,
|
||||
"shared column values must be trivially copyable");
|
||||
|
||||
static constexpr std::size_t ValueWordCount = (sizeof(TValue) + 7) / 8;
|
||||
using FKeyWords = std::array<std::uint64_t, 7>;
|
||||
using FValueWords = std::array<std::uint64_t, ValueWordCount>;
|
||||
|
||||
struct FEntry
|
||||
{
|
||||
std::atomic<std::uint64_t> Version{0};
|
||||
std::atomic<std::uint64_t> Tag{0};
|
||||
std::array<std::atomic<std::uint64_t>, 7> Key{};
|
||||
std::array<std::atomic<std::uint64_t>, ValueWordCount> Value{};
|
||||
};
|
||||
|
||||
struct FSet
|
||||
{
|
||||
std::atomic_flag Writer = ATOMIC_FLAG_INIT;
|
||||
std::uint8_t NextWay = 0;
|
||||
std::array<FEntry, Ways> Entries{};
|
||||
};
|
||||
|
||||
struct FSpinGuard
|
||||
{
|
||||
explicit FSpinGuard(std::atomic_flag& value) : Flag(value)
|
||||
{
|
||||
while (Flag.test_and_set(std::memory_order_acquire))
|
||||
std::this_thread::yield();
|
||||
}
|
||||
~FSpinGuard() { Flag.clear(std::memory_order_release); }
|
||||
std::atomic_flag& Flag;
|
||||
};
|
||||
|
||||
public:
|
||||
TSharedColumnValueCache() : Sets(std::make_unique<FSet[]>(SetCount)) {}
|
||||
TSharedColumnValueCache(const TSharedColumnValueCache&) = delete;
|
||||
TSharedColumnValueCache& operator=(const TSharedColumnValueCache&) = delete;
|
||||
|
||||
static TSharedColumnValueCache* setThreadActive(TSharedColumnValueCache* value) noexcept
|
||||
{
|
||||
TSharedColumnValueCache* previous = Active;
|
||||
Active = value;
|
||||
return previous;
|
||||
}
|
||||
|
||||
static TSharedColumnValueCache* threadActive() noexcept { return Active; }
|
||||
|
||||
template <typename TFactory>
|
||||
TValue getOrCompute(const FColumnCacheKey& key, TFactory&& factory,
|
||||
bool& computed)
|
||||
{
|
||||
computed = false;
|
||||
const FKeyWords keyWords = key.words();
|
||||
const std::uint64_t tag = Hash(keyWords);
|
||||
const std::size_t setIndex = static_cast<std::size_t>(tag) & (SetCount - 1);
|
||||
FSet& set = Sets[setIndex];
|
||||
TValue value{};
|
||||
if (TryReadSet(set, keyWords, tag, value)) return value;
|
||||
|
||||
FSpinGuard lock(set.Writer);
|
||||
if (TryReadSet(set, keyWords, tag, value)) return value;
|
||||
|
||||
value = std::forward<TFactory>(factory)();
|
||||
computed = true;
|
||||
FValueWords valueWords{};
|
||||
std::memcpy(valueWords.data(), &value, sizeof(TValue));
|
||||
|
||||
std::size_t replacement = Ways;
|
||||
for (std::size_t way = 0; way < Ways; ++way)
|
||||
if (set.Entries[way].Version.load(std::memory_order_seq_cst) == 0)
|
||||
{ replacement = way; break; }
|
||||
if (replacement == Ways)
|
||||
{
|
||||
replacement = set.NextWay;
|
||||
set.NextWay = static_cast<std::uint8_t>((set.NextWay + 1) % Ways);
|
||||
}
|
||||
FEntry& entry = set.Entries[replacement];
|
||||
std::uint64_t version = entry.Version.load(std::memory_order_seq_cst);
|
||||
if (version & 1u) ++version;
|
||||
entry.Version.store(version + 1, std::memory_order_seq_cst);
|
||||
for (std::size_t word = 0; word < keyWords.size(); ++word)
|
||||
entry.Key[word].store(keyWords[word], std::memory_order_seq_cst);
|
||||
for (std::size_t word = 0; word < valueWords.size(); ++word)
|
||||
entry.Value[word].store(valueWords[word], std::memory_order_seq_cst);
|
||||
entry.Tag.store(tag, std::memory_order_seq_cst);
|
||||
entry.Version.store(version + 2, std::memory_order_seq_cst);
|
||||
return value;
|
||||
}
|
||||
|
||||
private:
|
||||
static std::uint64_t Hash(const FKeyWords& words) noexcept
|
||||
{
|
||||
std::uint64_t hash = 0x9E3779B185EBCA87ull;
|
||||
for (const std::uint64_t word : words)
|
||||
{
|
||||
hash ^= word + 0x9E3779B97F4A7C15ull + (hash << 6) + (hash >> 2);
|
||||
hash ^= hash >> 30;
|
||||
hash *= 0xBF58476D1CE4E5B9ull;
|
||||
hash ^= hash >> 27;
|
||||
hash *= 0x94D049BB133111EBull;
|
||||
hash ^= hash >> 31;
|
||||
}
|
||||
return hash == 0 ? 1 : hash;
|
||||
}
|
||||
|
||||
static bool TryRead(const FEntry& entry, const FKeyWords& keyWords,
|
||||
std::uint64_t tag, TValue& out)
|
||||
{
|
||||
const std::uint64_t before = entry.Version.load(std::memory_order_seq_cst);
|
||||
if (before == 0 || (before & 1u)
|
||||
|| entry.Tag.load(std::memory_order_seq_cst) != tag) return false;
|
||||
FKeyWords foundKey{};
|
||||
for (std::size_t word = 0; word < foundKey.size(); ++word)
|
||||
foundKey[word] = entry.Key[word].load(std::memory_order_seq_cst);
|
||||
if (foundKey != keyWords) return false;
|
||||
FValueWords foundValue{};
|
||||
for (std::size_t word = 0; word < foundValue.size(); ++word)
|
||||
foundValue[word] = entry.Value[word].load(std::memory_order_seq_cst);
|
||||
const std::uint64_t after = entry.Version.load(std::memory_order_seq_cst);
|
||||
if (before != after || (after & 1u)) return false;
|
||||
std::memcpy(&out, foundValue.data(), sizeof(TValue));
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool TryReadSet(const FSet& set, const FKeyWords& keyWords,
|
||||
std::uint64_t tag, TValue& out)
|
||||
{
|
||||
for (const FEntry& entry : set.Entries)
|
||||
if (TryRead(entry, keyWords, tag, out)) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
std::unique_ptr<FSet[]> Sets;
|
||||
inline static thread_local TSharedColumnValueCache* Active = nullptr;
|
||||
};
|
||||
}
|
||||
@@ -110,6 +110,13 @@ namespace VoxelForgeCore
|
||||
|
||||
enum class TileClass : std::uint8_t { allSolid, allAir, mixed, unknown };
|
||||
|
||||
struct HeightfieldColumnEvaluation
|
||||
{
|
||||
std::int64_t x = 0, y = 0;
|
||||
std::uint64_t sequence = 0;
|
||||
bool outsideTileFootprint = false;
|
||||
};
|
||||
|
||||
struct TileGenerationStats
|
||||
{
|
||||
TileClass tileClass = TileClass::unknown;
|
||||
@@ -124,6 +131,8 @@ namespace VoxelForgeCore
|
||||
std::uint64_t columnCacheHits = 0;
|
||||
std::uint64_t columnCacheMisses = 0;
|
||||
std::uint64_t columnCacheConflictMisses = 0;
|
||||
std::uint64_t sharedColumnCacheHits = 0;
|
||||
std::uint64_t columnEvaluations = 0;
|
||||
double classificationMilliseconds = 0.0;
|
||||
double columnComputeMilliseconds = 0.0;
|
||||
double columnEvaluationExclusiveMilliseconds = 0.0;
|
||||
@@ -168,7 +177,7 @@ namespace VoxelForgeCore
|
||||
bool capRefined = false;
|
||||
std::uint64_t columnEvaluationWorkerHash = 0;
|
||||
bool columnCoordinateCaptureComplete = true;
|
||||
std::vector<std::array<std::int64_t, 2>> evaluatedColumnCoordinates;
|
||||
std::vector<HeightfieldColumnEvaluation> evaluatedColumnCoordinates;
|
||||
};
|
||||
|
||||
struct TileMesh
|
||||
@@ -651,10 +660,12 @@ namespace VoxelForgeCore
|
||||
std::uint64_t columnHits = 0;
|
||||
std::uint64_t columnMisses = 0;
|
||||
std::uint64_t columnConflictMisses = 0;
|
||||
std::uint64_t sharedColumnHits = 0;
|
||||
std::uint64_t columnEvaluations = 0;
|
||||
double columnComputeMilliseconds = 0.0;
|
||||
bool captureCoordinates = false;
|
||||
bool coordinateCaptureComplete = true;
|
||||
std::vector<std::array<std::int64_t, 2>> evaluatedCoordinates;
|
||||
std::vector<HeightfieldColumnEvaluation> evaluatedCoordinates;
|
||||
};
|
||||
|
||||
struct FieldInfluenceBounds
|
||||
@@ -705,6 +716,9 @@ namespace VoxelForgeCore
|
||||
std::int64_t maximumZSubvoxels, std::int32_t stepVoxels) const noexcept;
|
||||
static void resetThreadColumnCacheStats(bool captureCoordinates = false,
|
||||
bool clearCache = false, std::uint32_t octaveDrop = 0) noexcept;
|
||||
static void setThreadColumnCaptureBounds(std::int64_t minimumXSubvoxels,
|
||||
std::int64_t minimumYSubvoxels, std::int64_t maximumXSubvoxels,
|
||||
std::int64_t maximumYSubvoxels) noexcept;
|
||||
static void setThreadColumnOctaveDrop(std::uint32_t octaveDrop) noexcept;
|
||||
static std::uint32_t threadColumnOctaveDrop() noexcept;
|
||||
static void setThreadColumnInterpolationStrideSubvoxels(std::int64_t stride) noexcept;
|
||||
@@ -1038,6 +1052,7 @@ namespace VoxelForgeCore
|
||||
// Opt-in diagnostic only; records XY columns evaluated by this LOD8 tile.
|
||||
// Diagnostic seulement; enregistre les colonnes XY evaluees par cette tuile LOD8.
|
||||
bool captureColumnCoordinates = false;
|
||||
bool useSharedColumnCache = true;
|
||||
// Separate cold pass only; samples the MC inner stages without changing output.
|
||||
// Passe froide separee seulement; mesure les etapes internes MC sans changer la sortie.
|
||||
bool profileMesherStages = false;
|
||||
|
||||
@@ -55,6 +55,7 @@
|
||||
#include "Json.h"
|
||||
#include "VoxelForgeCore/VoxelForgeCore.h"
|
||||
#include "VoxelForgeCore/ThreadLocalStorage.h"
|
||||
#include "VoxelForgeCore/SharedColumnCache.h"
|
||||
|
||||
// Keep Core's expression-level FMA behavior fixed regardless of the toolchain setting.
|
||||
// Clang's "fast" mode ignores the pragma, so Core's runtime self-test rejects that build.
|
||||
|
||||
@@ -425,6 +425,9 @@ namespace floating_island_detail
|
||||
struct FIslandColumnCacheEntry
|
||||
{
|
||||
bool bValid=false;
|
||||
const FloatingIslandRecipeV1* RecipeIdentity=nullptr;
|
||||
const FloatingIslandPlanV1* PlanIdentity=nullptr;
|
||||
std::uint64_t RecipeContentHash=0,PlanContentHash=0;
|
||||
std::uint32_t Seed=0;
|
||||
std::uint64_t StableId=0;
|
||||
std::int64_t SampleX=0,SampleY=0,CenterX=0,CenterY=0,TopZ=0,Radius=0,TopHalf=0;
|
||||
@@ -432,9 +435,12 @@ namespace floating_island_detail
|
||||
double DeltaX=0.0,DeltaY=0.0,Radial=0.0,Angle=0.0,LobePhase=0.0,TopSurface=0.0;
|
||||
};
|
||||
|
||||
using FSharedIslandColumnCache =
|
||||
VoxelForgeCoreShared::TSharedColumnValueCache<FIslandColumnCacheEntry,16384,4>;
|
||||
|
||||
inline const FIslandColumnCacheEntry& IslandColumnCache(
|
||||
const FloatingIslandRecipeV1& recipe,const FloatingIslandAnchorV1& island,
|
||||
double x,double y)
|
||||
const FloatingIslandRecipeV1& recipe,const FloatingIslandPlanV1& plan,
|
||||
const FloatingIslandAnchorV1& island,double x,double y)
|
||||
{
|
||||
// Density samples arrive on integer Core lattice columns; reuse the exact column metrics
|
||||
// across Z samples without changing the field value or touching shared world state.
|
||||
@@ -450,7 +456,10 @@ namespace floating_island_detail
|
||||
std::uint64_t IndexKey=island.stableId
|
||||
^(static_cast<std::uint64_t>(SampleX)*0x9e3779b97f4a7c15ull)
|
||||
^(static_cast<std::uint64_t>(SampleY)*0xc2b2ae3d27d4eb4full)
|
||||
^(static_cast<std::uint64_t>(recipe.seed)<<23);
|
||||
^(static_cast<std::uint64_t>(recipe.seed)<<23)
|
||||
^recipe.contentHash^std::rotl(plan.contentHash,17)
|
||||
^static_cast<std::uint64_t>(reinterpret_cast<std::uintptr_t>(&recipe))
|
||||
^std::rotl(static_cast<std::uint64_t>(reinterpret_cast<std::uintptr_t>(&plan)),31);
|
||||
IndexKey^=IndexKey>>33;
|
||||
IndexKey*=0xff51afd7ed558ccdull;
|
||||
IndexKey^=IndexKey>>33;
|
||||
@@ -459,7 +468,9 @@ namespace floating_island_detail
|
||||
for(std::size_t Way=0;Way<Ways;++Way)
|
||||
{
|
||||
FIslandColumnCacheEntry& Entry=cache[Base+Way];
|
||||
const bool bKeyMatches=Entry.bValid&&Entry.Seed==recipe.seed
|
||||
const bool bKeyMatches=Entry.bValid&&Entry.RecipeIdentity==&recipe
|
||||
&&Entry.PlanIdentity==&plan&&Entry.RecipeContentHash==recipe.contentHash
|
||||
&&Entry.PlanContentHash==plan.contentHash&&Entry.Seed==recipe.seed
|
||||
&&Entry.StableId==island.stableId
|
||||
&&Entry.SampleX==SampleX&&Entry.SampleY==SampleY
|
||||
&&Entry.CenterX==island.centerSubvoxels[0]
|
||||
@@ -473,8 +484,14 @@ namespace floating_island_detail
|
||||
if(!Entry.bValid)Replacement=&Entry;
|
||||
}
|
||||
|
||||
FIslandColumnCacheEntry& Entry=*Replacement;
|
||||
const auto build=[&]()
|
||||
{
|
||||
FIslandColumnCacheEntry Entry;
|
||||
Entry.bValid=true;
|
||||
Entry.RecipeIdentity=&recipe;
|
||||
Entry.PlanIdentity=&plan;
|
||||
Entry.RecipeContentHash=recipe.contentHash;
|
||||
Entry.PlanContentHash=plan.contentHash;
|
||||
Entry.Seed=recipe.seed;
|
||||
Entry.StableId=island.stableId;
|
||||
Entry.SampleX=SampleX;
|
||||
@@ -499,6 +516,21 @@ namespace floating_island_detail
|
||||
*static_cast<double>(island.topHalfSubvoxels)*2.0*Edge*Edge;
|
||||
Entry.TopSurface=static_cast<double>(island.topZSubvoxels)-Drop;
|
||||
return Entry;
|
||||
};
|
||||
FIslandColumnCacheEntry Entry{};
|
||||
if(auto* shared=FSharedIslandColumnCache::threadActive())
|
||||
{
|
||||
const VoxelForgeCoreShared::FColumnCacheKey key{
|
||||
static_cast<std::uint64_t>(reinterpret_cast<std::uintptr_t>(&recipe)),
|
||||
recipe.contentHash,
|
||||
static_cast<std::uint64_t>(reinterpret_cast<std::uintptr_t>(&plan)),
|
||||
plan.contentHash,SampleX,SampleY,island.stableId};
|
||||
bool computed=false;
|
||||
Entry=shared->getOrCompute(key,build,computed);
|
||||
}
|
||||
else Entry=build();
|
||||
*Replacement=Entry;
|
||||
return *Replacement;
|
||||
}
|
||||
|
||||
inline std::int64_t ScaledQ16(std::int64_t value, std::int64_t scaleQ16)
|
||||
@@ -641,6 +673,7 @@ namespace floating_island_detail
|
||||
}
|
||||
|
||||
inline double IslandDensity(const FloatingIslandRecipeV1& recipe,
|
||||
const FloatingIslandPlanV1& plan,
|
||||
const FloatingIslandAnchorV1& island,
|
||||
double x, double y, double z)
|
||||
{
|
||||
@@ -665,7 +698,7 @@ namespace floating_island_detail
|
||||
std::max({z - top, radial - envelope, bottomZ - z}), x, y, z);
|
||||
}
|
||||
|
||||
const FIslandColumnCacheEntry& Column=IslandColumnCache(recipe,island,x,y);
|
||||
const FIslandColumnCacheEntry& Column=IslandColumnCache(recipe,plan,island,x,y);
|
||||
const double dx=Column.DeltaX;
|
||||
const double dy=Column.DeltaY;
|
||||
const double radial=Column.Radial;
|
||||
@@ -733,7 +766,7 @@ namespace floating_island_detail
|
||||
double result = static_cast<double>(MaxDensity);
|
||||
for (const auto& island : plan.islands)
|
||||
{
|
||||
double group = IslandDensity(recipe, island,
|
||||
double group = IslandDensity(recipe, plan, island,
|
||||
static_cast<double>(x), static_cast<double>(y), static_cast<double>(z));
|
||||
for (const auto& edge : plan.clusterEdges)
|
||||
{
|
||||
@@ -769,7 +802,7 @@ namespace floating_island_detail
|
||||
const auto evaluateLocalGroup = [&](std::size_t islandIndex)
|
||||
{
|
||||
const auto& island = plan.islands[islandIndex];
|
||||
double group = IslandDensity(recipe, island,
|
||||
double group = IslandDensity(recipe, plan, island,
|
||||
static_cast<double>(x), static_cast<double>(y), static_cast<double>(z));
|
||||
for (const std::uint32_t edgeIndex : featureIndices)
|
||||
{
|
||||
@@ -795,7 +828,7 @@ namespace floating_island_detail
|
||||
const auto evaluateDistantGroup = [&](std::size_t islandIndex)
|
||||
{
|
||||
const auto& island = plan.islands[islandIndex];
|
||||
double group = IslandDensity(recipe, island,
|
||||
double group = IslandDensity(recipe, plan, island,
|
||||
static_cast<double>(x), static_cast<double>(y), static_cast<double>(z));
|
||||
for (const auto& edge : plan.clusterEdges)
|
||||
{
|
||||
|
||||
@@ -943,6 +943,10 @@ namespace VoxelForgeCore
|
||||
std::shared_ptr<const WorldAssemblyManifest> worldAssembly;
|
||||
WorldConfiguration configuration;
|
||||
std::unique_ptr<FStandaloneShapeWorld> shape;
|
||||
std::unique_ptr<world_assembly_detail::FSharedHeightfieldColumnCache>
|
||||
sharedHeightfieldColumns;
|
||||
std::unique_ptr<floating_island_detail::FSharedIslandColumnCache>
|
||||
sharedIslandColumns;
|
||||
ShapeBuildRequest buildRequest;
|
||||
mutable std::shared_mutex tileRegionMutex;
|
||||
std::unordered_map<TileKey,std::shared_ptr<FStandaloneShapeWorld>,TileKeyHash> tileRegions;
|
||||
@@ -953,6 +957,23 @@ namespace VoxelForgeCore
|
||||
std::uint64_t lastSequence=0;
|
||||
};
|
||||
|
||||
struct FScopedSharedColumnCaches
|
||||
{
|
||||
using FHeightfieldCache=world_assembly_detail::FSharedHeightfieldColumnCache;
|
||||
using FIslandCache=floating_island_detail::FSharedIslandColumnCache;
|
||||
FHeightfieldCache* PreviousHeightfield=nullptr;
|
||||
FIslandCache* PreviousIslands=nullptr;
|
||||
|
||||
FScopedSharedColumnCaches(FHeightfieldCache* Heightfield,FIslandCache* Islands)
|
||||
: PreviousHeightfield(FHeightfieldCache::setThreadActive(Heightfield)),
|
||||
PreviousIslands(FIslandCache::setThreadActive(Islands)) {}
|
||||
~FScopedSharedColumnCaches()
|
||||
{
|
||||
FHeightfieldCache::setThreadActive(PreviousHeightfield);
|
||||
FIslandCache::setThreadActive(PreviousIslands);
|
||||
}
|
||||
};
|
||||
|
||||
World::World(ResolvedWorldSnapshot snapshot):impl_(std::make_unique<Impl>())
|
||||
{
|
||||
if(snapshot.data_)
|
||||
@@ -960,6 +981,15 @@ namespace VoxelForgeCore
|
||||
impl_->owner=snapshot.data_->owner;
|
||||
impl_->worldAssembly=snapshot.data_->worldAssembly;
|
||||
impl_->configuration=snapshot.data_->configuration;
|
||||
if(impl_->worldAssembly)
|
||||
{
|
||||
if(!impl_->worldAssembly->plan.strates.empty())
|
||||
impl_->sharedHeightfieldColumns=
|
||||
std::make_unique<world_assembly_detail::FSharedHeightfieldColumnCache>();
|
||||
if(!impl_->worldAssembly->floatingIslands.empty())
|
||||
impl_->sharedIslandColumns=
|
||||
std::make_unique<floating_island_detail::FSharedIslandColumnCache>();
|
||||
}
|
||||
}
|
||||
}
|
||||
World::~World()=default;
|
||||
@@ -1337,6 +1367,9 @@ namespace VoxelForgeCore
|
||||
options.legacyDirectMappedColumnCache||key.lod<=2);
|
||||
if(!impl_||!impl_->owner||!impl_->shape||!impl_->shape->IsBuilt())
|
||||
{ error="world must be compiled before generating tiles";return std::nullopt; }
|
||||
FScopedSharedColumnCaches sharedColumnCaches(options.useSharedColumnCache
|
||||
?impl_->sharedHeightfieldColumns.get():nullptr,
|
||||
options.useSharedColumnCache?impl_->sharedIslandColumns.get():nullptr);
|
||||
const auto& world=*impl_->owner;
|
||||
std::shared_ptr<FStandaloneShapeWorld> page;
|
||||
if(impl_->tileRegionCount.load(std::memory_order_acquire)!=0)
|
||||
@@ -1500,6 +1533,15 @@ namespace VoxelForgeCore
|
||||
FStandaloneShapeWorld::ResetThreadClassificationStats();
|
||||
VoxelForgeCore::Heightfield2D::resetThreadColumnCacheStats(
|
||||
options.captureColumnCoordinates);
|
||||
if(options.captureColumnCoordinates)
|
||||
{
|
||||
constexpr std::int64_t SubvoxelsPerVoxel=WorldAssemblySubvoxelsPerVoxel;
|
||||
VoxelForgeCore::Heightfield2D::setThreadColumnCaptureBounds(
|
||||
static_cast<std::int64_t>(origin.X)*SubvoxelsPerVoxel,
|
||||
static_cast<std::int64_t>(origin.Y)*SubvoxelsPerVoxel,
|
||||
static_cast<std::int64_t>(origin.X+extent)*SubvoxelsPerVoxel,
|
||||
static_cast<std::int64_t>(origin.Y+extent)*SubvoxelsPerVoxel);
|
||||
}
|
||||
bool bNativeSurfaceTile=false;
|
||||
bool bNativeSurfaceBandSafe=false;
|
||||
double classificationColumnComputeMilliseconds=0.0;
|
||||
@@ -1544,6 +1586,8 @@ namespace VoxelForgeCore
|
||||
out.diagnostics.columnCacheHits=Columns.columnHits;
|
||||
out.diagnostics.columnCacheMisses=Columns.columnMisses;
|
||||
out.diagnostics.columnCacheConflictMisses=Columns.columnConflictMisses;
|
||||
out.diagnostics.sharedColumnCacheHits=Columns.sharedColumnHits;
|
||||
out.diagnostics.columnEvaluations=Columns.columnEvaluations;
|
||||
out.diagnostics.classificationMilliseconds=Counts.Milliseconds;
|
||||
out.diagnostics.classificationExclusiveMilliseconds=std::max(0.0,
|
||||
Counts.Milliseconds-classificationColumnComputeMilliseconds);
|
||||
@@ -1829,6 +1873,8 @@ namespace VoxelForgeCore
|
||||
out.diagnostics.columnCacheHits=Columns.columnHits;
|
||||
out.diagnostics.columnCacheMisses=Columns.columnMisses;
|
||||
out.diagnostics.columnCacheConflictMisses=Columns.columnConflictMisses;
|
||||
out.diagnostics.sharedColumnCacheHits=Columns.sharedColumnHits;
|
||||
out.diagnostics.columnEvaluations=Columns.columnEvaluations;
|
||||
out.diagnostics.columnComputeMilliseconds=Columns.columnComputeMilliseconds;
|
||||
out.diagnostics.columnEvaluationExclusiveMilliseconds=Columns.columnComputeMilliseconds;
|
||||
out.diagnostics.classificationMilliseconds=Counts.Milliseconds;
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
#include "VoxelForgeCore/VoxelForgeCore.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <atomic>
|
||||
#include <bit>
|
||||
#include <charconv>
|
||||
#include <chrono>
|
||||
@@ -121,6 +122,9 @@ WorldAssemblyFeatureColumnRangeV1 BuildWorldAssemblyFeatureColumnRangeV1(
|
||||
|
||||
namespace world_assembly_detail
|
||||
{
|
||||
using FSharedHeightfieldColumnCache =
|
||||
VoxelForgeCoreShared::TSharedColumnValueCache<HeightfieldPointSample,131072,4>;
|
||||
|
||||
// Preserve recipe order without allocating or looking up metadata in the point loop.
|
||||
// Garder l'ordre de recette sans allocation ni recherche de metadonnees par point.
|
||||
struct FeatureRange
|
||||
@@ -1375,6 +1379,9 @@ namespace world_assembly_detail
|
||||
std::array<std::uint8_t, HeightfieldColumnCacheSetCount>> HeightfieldColumnCacheNextWay;
|
||||
inline thread_local HeightfieldColumnCacheStats HeightfieldColumnStats{};
|
||||
inline thread_local bool HeightfieldColumnCacheLegacyDirectMapped=false;
|
||||
inline thread_local std::array<std::int64_t,4> HeightfieldColumnCaptureBounds{};
|
||||
inline thread_local bool HeightfieldColumnCaptureBoundsValid=false;
|
||||
inline std::atomic<std::uint64_t> HeightfieldColumnCaptureSequence{0};
|
||||
|
||||
inline std::size_t HeightfieldColumnCacheSet(const WorldAssemblyRecipeV1& recipe,
|
||||
const WorldAssemblyStratumPlanV1& plan,std::int64_t x,std::int64_t y) noexcept
|
||||
@@ -1587,22 +1594,46 @@ namespace world_assembly_detail
|
||||
replacement=&cache[first+replacementWay];
|
||||
}
|
||||
auto& entry=*replacement;
|
||||
++HeightfieldColumnStats.columnMisses;
|
||||
const auto compute=[&]()
|
||||
{
|
||||
const std::uint64_t sequence=HeightfieldColumnStats.captureCoordinates
|
||||
?HeightfieldColumnCaptureSequence.fetch_add(1,std::memory_order_relaxed):0;
|
||||
const auto start=std::chrono::steady_clock::now();
|
||||
HeightfieldPointSample column=EvaluateColumn(recipe,plan,x,y);
|
||||
HeightfieldPointSample result=EvaluateColumn(recipe,plan,x,y);
|
||||
HeightfieldColumnStats.columnComputeMilliseconds+=std::chrono::duration<double,std::milli>(
|
||||
std::chrono::steady_clock::now()-start).count();
|
||||
++HeightfieldColumnStats.columnMisses;
|
||||
if(HeightfieldColumnStats.captureCoordinates&&HeightfieldColumnStats.coordinateCaptureComplete)
|
||||
++HeightfieldColumnStats.columnEvaluations;
|
||||
if(HeightfieldColumnStats.captureCoordinates
|
||||
&&HeightfieldColumnStats.coordinateCaptureComplete)
|
||||
{
|
||||
try
|
||||
{
|
||||
HeightfieldColumnStats.evaluatedCoordinates.push_back({x,y});
|
||||
const bool outside=HeightfieldColumnCaptureBoundsValid
|
||||
&&(x<HeightfieldColumnCaptureBounds[0]||x>HeightfieldColumnCaptureBounds[2]
|
||||
||y<HeightfieldColumnCaptureBounds[1]||y>HeightfieldColumnCaptureBounds[3]);
|
||||
HeightfieldColumnStats.evaluatedCoordinates.push_back({x,y,sequence,outside});
|
||||
}
|
||||
catch(...)
|
||||
{
|
||||
HeightfieldColumnStats.coordinateCaptureComplete=false;
|
||||
}
|
||||
}
|
||||
return result;
|
||||
};
|
||||
HeightfieldPointSample column{};
|
||||
if(auto* shared=FSharedHeightfieldColumnCache::threadActive())
|
||||
{
|
||||
const VoxelForgeCoreShared::FColumnCacheKey key{
|
||||
static_cast<std::uint64_t>(reinterpret_cast<std::uintptr_t>(&recipe)),
|
||||
recipe.contentHash,
|
||||
static_cast<std::uint64_t>(reinterpret_cast<std::uintptr_t>(&plan)),
|
||||
plan.contentHash,x,y};
|
||||
bool computed=false;
|
||||
column=shared->getOrCompute(key,compute,computed);
|
||||
if(!computed)++HeightfieldColumnStats.sharedColumnHits;
|
||||
}
|
||||
else column=compute();
|
||||
entry={&recipe,&plan,recipe.contentHash,plan.contentHash,x,y,column};
|
||||
return column;
|
||||
}
|
||||
@@ -2087,10 +2118,22 @@ void Heightfield2D::resetThreadColumnCacheStats(bool captureCoordinates, bool cl
|
||||
Stats.columnHits=0;
|
||||
Stats.columnMisses=0;
|
||||
Stats.columnConflictMisses=0;
|
||||
Stats.sharedColumnHits=0;
|
||||
Stats.columnEvaluations=0;
|
||||
Stats.columnComputeMilliseconds=0.0;
|
||||
Stats.captureCoordinates=captureCoordinates;
|
||||
Stats.coordinateCaptureComplete=true;
|
||||
Stats.evaluatedCoordinates.clear();
|
||||
world_assembly_detail::HeightfieldColumnCaptureBoundsValid=false;
|
||||
}
|
||||
|
||||
void Heightfield2D::setThreadColumnCaptureBounds(std::int64_t minimumXSubvoxels,
|
||||
std::int64_t minimumYSubvoxels,std::int64_t maximumXSubvoxels,
|
||||
std::int64_t maximumYSubvoxels) noexcept
|
||||
{
|
||||
world_assembly_detail::HeightfieldColumnCaptureBounds={minimumXSubvoxels,
|
||||
minimumYSubvoxels,maximumXSubvoxels,maximumYSubvoxels};
|
||||
world_assembly_detail::HeightfieldColumnCaptureBoundsValid=true;
|
||||
}
|
||||
|
||||
void Heightfield2D::setThreadColumnCacheLegacyDirectMapped(bool enabled) noexcept
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
# Linux7 — shared terrain columns
|
||||
|
||||
**Decision: keep the shared second tier.** On the L1 view it cut median W12 whole-view time by
|
||||
19.0%, well above the 3% threshold, and the 8 m camera crossing was also faster. All final output
|
||||
hashes match the `9898e05` binary.
|
||||
|
||||
Runs used the `linux6` base, Clang 19 with FP contraction off, GCC 14 defaults, CPUs 0–11, normal
|
||||
priority, a 6 GiB address-space cap, and the unchanged `powersave` governor. Static timings below
|
||||
are three alternating base/candidate W12 runs of 2,553 tiles.
|
||||
|
||||
## 1. What was recomputed
|
||||
|
||||
Three complete W12 captures per mode on the L1 view. Counts below are medians of actual column
|
||||
formula executions after worker-local cache misses, grouped by LOD. “Same worker” means the worker
|
||||
had computed that XY before and later missed its local entry. Halo/normal repeats are separated
|
||||
before worker attribution; the parenthesized value is the total computed halo/normal probes,
|
||||
including unique ones. Each field's median is independent, so row medians can differ slightly from
|
||||
their sums.
|
||||
|
||||
| LOD | Base total / unique XY | Same worker | Different worker | Halo repeats / probes | Shared total / unique XY |
|
||||
|---:|---:|---:|---:|---:|---:|
|
||||
| 0 | 556,134 / 128,240 | 22,307 | 358,908 | 46,679 / 70,972 | 135,720 / 127,370 |
|
||||
| 1 | 452,626 / 104,284 | 16,393 | 287,791 | 39,245 / 59,082 | 85,658 / 85,479 |
|
||||
| 2 | 144,251 / 32,274 | 14,667 | 74,377 | 22,933 / 30,942 | 28,019 / 28,019 |
|
||||
| 3 | 91,158 / 33,256 | 0 | 46,387 | 11,564 / 20,339 | 16,213 / 16,213 |
|
||||
| 4 | 272,776 / 91,325 | 100 | 167,083 | 14,266 / 28,089 | 90,105 / 90,065 |
|
||||
| 5 | 178,511 / 71,957 | 1 | 98,687 | 7,892 / 20,042 | 70,932 / 70,930 |
|
||||
| 6 | 342,763 / 280,334 | 20 | 56,204 | 6,205 / 19,977 | 279,266 / 279,202 |
|
||||
| 7 | 79,400 / 71,468 | 1 | 4,951 | 2,980 / 14,329 | 70,492 / 70,492 |
|
||||
| 8 | 137,062 / 106,219 | 0 | 22,138 | 8,704 / 22,609 | 106,219 / 106,219 |
|
||||
|
||||
Across the per-LOD medians, the base made 2.255 million formula evaluations: 919 thousand unique
|
||||
XYs and 1.336 million repeats. Of the repeats, 83.6% were from another worker, 4.0% were same-worker
|
||||
evictions, and 12.0% were halo/normal probes. The shared candidate made 883 thousand evaluations,
|
||||
60.9% fewer; about 8.6 thousand repeats remained. This is mostly a worker-sharing problem, not a
|
||||
local-cache-capacity problem.
|
||||
|
||||
## 2. Design
|
||||
|
||||
`SharedColumnCache.h` is a bounded, four-way second-tier value cache used by both `Heightfield2D`
|
||||
and FloatingIslands. Its key includes recipe and plan pointer identity plus content hash, exact XY,
|
||||
and an optional exact sub-domain identity; FloatingIslands supplies the island stable ID because
|
||||
different anchors can use the same XY. LOD is omitted because the column value is LOD-independent.
|
||||
The heightfield table has 524,288 slots and the island table has 65,536. Both are allocated with
|
||||
the `World::Impl` and destroyed with it. The Standalone streamer stops and joins workers before
|
||||
world teardown; engine shutdown must keep the same drain-before-destroy ordering as the existing
|
||||
generator state.
|
||||
|
||||
The S14 `FLazy` thread-local caches stay first, so their hits remain worker-private. Shared hits read
|
||||
versioned atomic entries without taking a lock; a miss takes a set-local spin lock, rechecks, then
|
||||
computes and publishes. A hit racing with a slot replacement can fall through as a miss. Fixed
|
||||
capacity bounds memory and replacement bounds reuse as the player moves. No page ring, prefetch, or
|
||||
neighbor affinity was needed. The shared table is not an immutable page store; atomic fields and
|
||||
version checks give readers a consistent value while a slot can be replaced. Workers only read
|
||||
recipe/plan identity and content hashes; they do not mutate generator state or call UObject/engine
|
||||
APIs. Epoch, shutdown, and MPSC result handling were not changed. The L3 FP-contract preamble and
|
||||
self-test remain intact, as does the `FLazy` rule for large thread-local storage.
|
||||
|
||||
## 3. Speed and memory
|
||||
|
||||
| W12 static whole view | Base | Shared | Change |
|
||||
|---|---:|---:|---:|
|
||||
| `whole_view_ready` median | 2.165 s | 1.754 s | −19.0% |
|
||||
| Peak RSS median (max) | 401.5 MiB (402.6) | 467.8 MiB (468.0) | +66.3 MiB median |
|
||||
|
||||
Per-LOD worker sum is the sum of tile worker seconds, not wall time. Values are medians across the
|
||||
same three static runs.
|
||||
|
||||
| LOD | Base worker sum | Shared worker sum | Change |
|
||||
|---:|---:|---:|---:|
|
||||
| 0 | 3.830 s | 2.667 s | −30.4% |
|
||||
| 1 | 5.301 s | 4.085 s | −22.9% |
|
||||
| 2 | 1.296 s | 0.795 s | −38.7% |
|
||||
| 3 | 1.116 s | 0.725 s | −35.1% |
|
||||
| 4 | 2.692 s | 1.674 s | −37.8% |
|
||||
| 5 | 1.911 s | 1.459 s | −23.7% |
|
||||
| 6 | 2.903 s | 2.621 s | −9.7% |
|
||||
| 7 | 1.242 s | 1.227 s | −1.2% |
|
||||
| 8 | 2.318 s | 2.271 s | −2.0% |
|
||||
|
||||
The moving trial warmed the first view, moved the camera 8 m, then generated 144 entering tiles
|
||||
(128 LOD0, 16 LOD1) on the same world and worker pool. It ran three times per cache mode.
|
||||
|
||||
| Moving check | Shared | Thread-local | Change |
|
||||
|---|---:|---:|---:|
|
||||
| New-tile wall median | 52.39 ms | 82.00 ms | −36.1% |
|
||||
| LOD0 worker sum median | 403.44 ms | 693.67 ms | −41.8% |
|
||||
| LOD1 worker sum median | 138.34 ms | 182.29 ms | −24.1% |
|
||||
| Peak RSS median (max) | 466.1 MiB (466.9) | 466.1 MiB (467.6) | Same cache allocation in both modes |
|
||||
|
||||
## 4. Hashes
|
||||
|
||||
Every W1/W12 result matched the base for Clang FP contraction off and GCC default: full detail,
|
||||
whole view, each LOD, and exact queries. The L1 base hashes are:
|
||||
|
||||
| Scope | Density | Mesh | Payload | Topology |
|
||||
|---|---|---|---|---|
|
||||
| Full detail, 960 tiles | `0x0ee0579bd15cbbe4` | `0x32e5d4964d610fee` | `0x1932b96ee4b1d910` | `0x69743c6bad3447dc` |
|
||||
| Whole view, 2,553 tiles | `0xc10697a8f48380c7` | `0xe75c9119a1498cd5` | `0x49a9a8d1835a6c8c` | `0xedcb23288b5e8724` |
|
||||
|
||||
| LOD | Density | Mesh | Payload | Topology |
|
||||
|---:|---|---|---|---|
|
||||
| 0 | `0x769f2a706d585a72` | `0x33eeeb39fa443816` | `0x645621a1d968983f` | `0xc2457ed905036c9d` |
|
||||
| 1 | `0x69cc5a5c0a2c60af` | `0xb48335f5410c57d5` | `0xd15c5480162ab8ba` | `0xfeca1bd025823604` |
|
||||
| 2 | `0x85deed5683a77202` | `0x53469895987aeea2` | `0xe55f8ae7e1532033` | `0xd5feb12a0e85c09b` |
|
||||
| 3 | `0x6518caefb3be7e7f` | `0xec513c90b853a614` | `0xf62fa1137ff17f82` | `0xccf1dcf681965816` |
|
||||
| 4 | `0x62d4be96021992f4` | `0xf1e4c6efcc2615a9` | `0x9dc5f9c88be02f33` | `0xfe6f55125a07c3bf` |
|
||||
| 5 | `0x5e4a8c38b8cea6fc` | `0x08ee56998aaf21e5` | `0x541eea8558f26cc8` | `0x6539374a8b3336d8` |
|
||||
| 6 | `0xe036e95b9006945b` | `0x624a33f941a99b02` | `0x81b73e14f68bdf86` | `0x4de4c63ce1f80a56` |
|
||||
| 7 | `0x8ee358af56a30d8d` | `0x46ab3c6184048b7c` | `0x0a6164b7e68e80c5` | `0x0c26b85bb03bc831` |
|
||||
| 8 | `0xc3458e1cbdb9c471` | `0x46c03d0994d75192` | `0x8f89fc5bde12d230` | `0xd16dc0cf62031898` |
|
||||
|
||||
Exact queries (9 probes): density and unclamped `0xe4d4a8eb95b35933`, detail
|
||||
`0xefcba20c0c70c415`, collision `0xe604823a249029bf`. The FloatingIslands seed-0 fixture also
|
||||
passes at Clang/GCC W1/W12 with recipe hash `0x39a24198ad8cf88f`; its whole-view hash matches the
|
||||
base: density `0xa8fc5f8f4eaf77a3`, mesh `0xfff7d8ac8923eb9a`, payload `0xd730c51d959c1c59`,
|
||||
topology `0xbb33ddb76cd386be`.
|
||||
|
||||
## 5. Transfer estimate and Windows work
|
||||
|
||||
The majority of the opportunity is cross-worker, so it does not disappear just because the 5800X3D
|
||||
has a larger L3. Same-worker evictions are only 4% of baseline repeats; that capacity-driven part
|
||||
may shrink. The candidate adds about 66 MiB peak RSS, and a 96 MiB L3 could keep much of the shared
|
||||
table resident, though other game data will compete for it. I expect the win to transfer and clear
|
||||
3%, likely landing in the mid-single to low-double digits; that percentage is an estimate until the
|
||||
5800X3D run.
|
||||
|
||||
Windows to-do:
|
||||
|
||||
1. Build the Core route with MSVC `/fp:precise`; keep the FP-contract self-test and check the
|
||||
64-bit atomic slot path on the supported Windows target.
|
||||
2. Run the W1/W12 whole-view, full-detail, per-LOD, exact-query, and seed-0 FloatingIslands checks
|
||||
against the base binary.
|
||||
3. Interleave three static W12 A/B runs on the 5800X3D, then repeat the 8 m camera crossing; record
|
||||
peak RSS and the machine's unchanged governor.
|
||||
4. Exercise real streaming shutdown/profile changes and confirm workers drain before world/cache
|
||||
destruction, with Epoch checks and MPSC result queues still intact.
|
||||
|
||||
## 6. Failed and corrected runs
|
||||
|
||||
- The first Clang W1 hash invocation omitted the explicit data path; it exited 2 after resolving
|
||||
`Standalone/bin/data/owner_world.json`. The retry passed with
|
||||
`Standalone/data/owner_world.json` supplied.
|
||||
- The first moving step was 32 m and failed the shape interval 160-million-node safety bound. The
|
||||
benchmark was retried at 8 m, which completed three times in each cache mode.
|
||||
- The first shared FloatingIslands cache key omitted the island stable ID. The W12 fixture exposed
|
||||
an LOD4 hash mismatch. Adding the exact sub-domain ID fixed it; final hashes match at W1/W12 for
|
||||
both compilers.
|
||||
- An early fixture command passed `--seed=0`, which also overrode the surface-world seed. The final
|
||||
fixture comparison omits that override; FloatingIslands itself still runs its seed-0 check.
|
||||
- No run exceeded 30 minutes or remained stuck for 15 minutes. All final Clang and GCC builds
|
||||
succeeded with 12 jobs.
|
||||
|
||||
## Owner note
|
||||
|
||||
Linux measured a 19.0% W12 whole-view gain and a 36.1% gain on the 8 m crossing, at about 66 MiB
|
||||
more peak RSS. Please run the Windows hash matrix and static/moving A/B on the 5800X3D before using
|
||||
the laptop percentage as an owner-machine estimate.
|
||||
|
||||
## Orchestrator review (Claude)
|
||||
|
||||
- **Concurrency read:** a seqlock over all-atomic fields (seq_cst), with the version checked before
|
||||
and after the key/value loads; readers fall back to a miss on any mismatch. Writers hold the set
|
||||
spin lock and mark the version odd while writing. There is no non-atomic shared access, so a
|
||||
reader can't return a torn value. The per-tile scoped `setThreadActive`/restore binds each thread
|
||||
to its own world's table. Default on (`useSharedColumnCache = true`).
|
||||
- **Re-run by me** (W12, L1 view, interleaved): base `linux6` 2.014/2.059/2.092 s vs L7 `linux7shared`
|
||||
1.657/1.730/1.729 s, then 1.718/1.705/1.722 s (median 2.059 → 1.722 s, -16%). The whole-view hash was
|
||||
identical in all 8 L7 runs at W12, plus W1 (8.036 s) and W8 (1.799 s).
|
||||
- Luna's island-key bug was caught by the W12 hash fixture before review, so the determinism check
|
||||
works as intended.
|
||||
@@ -35,6 +35,7 @@ namespace
|
||||
bool StreamPipeline=false;
|
||||
bool StreamHash=false;
|
||||
bool ProfileColumnOverlap=false;
|
||||
bool UseSharedColumnCache=true;
|
||||
bool ForceGenerate=false;
|
||||
bool MeshCoverage=false;
|
||||
bool ScreenSpaceGate=false;
|
||||
@@ -47,8 +48,10 @@ namespace
|
||||
int TlsProbeWorkers=0;
|
||||
std::string TlsProbeTouch="none";
|
||||
bool HasCamera=false;
|
||||
bool HasCameraStep=false;
|
||||
bool AutoSurfaceCamera=false;
|
||||
FVector CameraMeters=FVector::ZeroVector;
|
||||
FVector CameraStepMeters=FVector::ZeroVector;
|
||||
float CameraYawDegrees=0.0f;
|
||||
float CameraPitchDegrees=0.0f;
|
||||
std::filesystem::path MeshExportPath;
|
||||
@@ -77,6 +80,15 @@ namespace
|
||||
return true;
|
||||
}
|
||||
|
||||
bool ParseCameraStep(std::string_view text,FVector& meters)
|
||||
{
|
||||
const std::string values(text);
|
||||
float x=0.0f,y=0.0f,z=0.0f;
|
||||
if(std::sscanf(values.c_str(),"%f,%f,%f",&x,&y,&z)!=3)return false;
|
||||
meters={x,y,z};
|
||||
return true;
|
||||
}
|
||||
|
||||
bool ParseOptions(int argc,char** argv,FOptions& options)
|
||||
{
|
||||
const std::filesystem::path executable=std::filesystem::absolute(argv[0]);
|
||||
@@ -91,6 +103,8 @@ namespace
|
||||
std::printf("Usage: %s [owner_world.json] --world-assembler-prototype --stream-benchmark --stream-pipeline [options]\n"
|
||||
" --workers=N --clip-radius=N --max-clip-level=N --max-level=N\n"
|
||||
" --camera=X,Y,Z,YAW,PITCH (position in metres) --stream-hash\n"
|
||||
" --camera-step=X,Y,Z (optional second view step in metres)\n"
|
||||
" --column-cache=shared|thread-local (default: shared)\n"
|
||||
" --mesh-export=FILE (write the exact generated tile meshes for offline rendering)\n"
|
||||
" --profile-column-overlap --far-horizon-lod8-step=N --force-generate\n"
|
||||
" --seed=N --recipe=surface-world|floating-islands\n"
|
||||
@@ -149,6 +163,20 @@ namespace
|
||||
{std::fprintf(stderr,"Invalid --camera value: %.*s\n",static_cast<int>(arg.size()),arg.data());return false;}
|
||||
options.HasCamera=true;
|
||||
}
|
||||
else if(arg.starts_with("--camera-step="))
|
||||
{
|
||||
if(!ParseCameraStep(arg.substr(std::string_view("--camera-step=").size()),
|
||||
options.CameraStepMeters))
|
||||
{std::fprintf(stderr,"Invalid --camera-step value: %.*s\n",static_cast<int>(arg.size()),arg.data());return false;}
|
||||
options.HasCameraStep=true;
|
||||
}
|
||||
else if(arg.starts_with("--column-cache="))
|
||||
{
|
||||
const std::string_view mode=arg.substr(std::string_view("--column-cache=").size());
|
||||
if(mode=="shared")options.UseSharedColumnCache=true;
|
||||
else if(mode=="thread-local")options.UseSharedColumnCache=false;
|
||||
else {std::fprintf(stderr,"Invalid --column-cache value: %.*s\n",static_cast<int>(arg.size()),arg.data());return false;}
|
||||
}
|
||||
else if(arg.starts_with("--mesh-export="))
|
||||
{
|
||||
options.MeshExportPath=std::filesystem::path(arg.substr(14));
|
||||
@@ -199,10 +227,12 @@ int main(int argc,char** argv)
|
||||
if(!ParseOptions(argc,argv,options))return 2;
|
||||
if(options.TlsProbeRequested)
|
||||
return StandaloneTlsProbe::Run(options.TlsProbeWorkers,options.TlsProbeTouch);
|
||||
std::printf("[HeadlessConfig] data=%s workers=%d clip_radius=%d max_clip_level=%d max_level=%d hash=%s column_overlap=%s force_generate=%s recipe=%s seed_override=%s render_distance_override=%s\n",
|
||||
std::printf("[HeadlessConfig] data=%s workers=%d clip_radius=%d max_clip_level=%d max_level=%d hash=%s column_overlap=%s force_generate=%s recipe=%s shared_column_cache=%s camera_step_m=(%.3f,%.3f,%.3f) seed_override=%s render_distance_override=%s\n",
|
||||
options.DataPath.string().c_str(),options.WorkerCount,options.ClipRadius,options.MaxClipLevel,
|
||||
options.MaxLevel,options.StreamHash?"on":"off",options.ProfileColumnOverlap?"on":"off",
|
||||
options.ForceGenerate?"on":"off",options.FloatingIslandsRecipe?"floating-islands":"surface-world",
|
||||
options.UseSharedColumnCache?"on":"off",options.CameraStepMeters.X,options.CameraStepMeters.Y,
|
||||
options.CameraStepMeters.Z,
|
||||
options.HasSeedOverride?"yes":"no",options.HasRenderDistanceOverride?"yes":"no");
|
||||
std::printf("[ColdTimeline] stage=owner_data_load begin_ms=%.3f\n",
|
||||
std::chrono::duration<double,std::milli>(Clock::now()-coldStart).count());
|
||||
@@ -309,10 +339,14 @@ int main(int argc,char** argv)
|
||||
{std::fprintf(stderr,"Shape interval build failed: %s\n",error.c_str());return 2;}
|
||||
|
||||
FStandaloneCoreScheduler scheduler;
|
||||
const FVector cameraStepVoxels=options.HasCameraStep
|
||||
?FVector(options.CameraStepMeters.X*4.0f,options.CameraStepMeters.Y*4.0f,
|
||||
options.CameraStepMeters.Z*4.0f):FVector::ZeroVector;
|
||||
return StandaloneStreamBenchmark::RunPipelinedStreamingBenchmark(ownerWorld,snapshot,
|
||||
cameraVoxels,options.WorkerCount,options.MaxLevel,VoxelForgeCore::DetailVariant::coherent,
|
||||
scheduler,options.StreamHash,false,false,false,false,coldStart,
|
||||
options.ProfileColumnOverlap,options.FarHorizonLOD8InteriorStepVoxels,
|
||||
options.MeshCoverage,options.MeshExportPath,options.CameraYawDegrees,
|
||||
options.CameraPitchDegrees,options.ScreenSpaceGate);
|
||||
options.CameraPitchDegrees,options.ScreenSpaceGate,options.UseSharedColumnCache,
|
||||
cameraStepVoxels);
|
||||
}
|
||||
@@ -137,7 +137,8 @@ namespace StandaloneStreamBenchmark
|
||||
std::chrono::steady_clock::time_point ColdStart,bool bProfileColumnOverlap,
|
||||
int32 FarHorizonLOD8InteriorStepVoxels,bool bReportMeshCoverage=false,
|
||||
const std::filesystem::path& MeshExportPath={},float CameraYawDegrees=0.0f,
|
||||
float CameraPitchDegrees=0.0f,bool bScreenSpaceGate=false)
|
||||
float CameraPitchDegrees=0.0f,bool bScreenSpaceGate=false,
|
||||
bool bUseSharedColumnCache=true,FVector CameraStepVoxels=FVector::ZeroVector)
|
||||
{
|
||||
using Clock=std::chrono::steady_clock;
|
||||
FStandaloneOwnerWorld& World=*OwnerWorld;
|
||||
@@ -167,6 +168,21 @@ namespace StandaloneStreamBenchmark
|
||||
if(MaxLevel>=0)
|
||||
Desired.Tiles.erase(std::remove_if(Desired.Tiles.begin(),Desired.Tiles.end(),
|
||||
[&](const FVoxelTileKey& Tile){return Tile.Level>MaxLevel;}),Desired.Tiles.end());
|
||||
const bool bHasCameraStep=std::abs(CameraStepVoxels.X)+std::abs(CameraStepVoxels.Y)
|
||||
+std::abs(CameraStepVoxels.Z)>1.0e-6f;
|
||||
FStandaloneDesiredTileSet MovedDesired;
|
||||
if(bHasCameraStep)
|
||||
{
|
||||
const FVector MovedCamera=CameraPositionVoxels+CameraStepVoxels;
|
||||
MovedDesired=StandaloneWorldMesher::BuildDesiredSet(World,MovedCamera);
|
||||
const auto MovedCoreTiles=Shape.desiredTiles({MovedCamera.X,MovedCamera.Y,MovedCamera.Z});
|
||||
MovedDesired.Tiles.clear();MovedDesired.Tiles.reserve(MovedCoreTiles.size());
|
||||
for(const auto& Key:MovedCoreTiles)
|
||||
MovedDesired.Tiles.emplace_back(FIntVector(Key.coordinate.x,Key.coordinate.y,Key.coordinate.z),Key.lod);
|
||||
if(MaxLevel>=0)
|
||||
MovedDesired.Tiles.erase(std::remove_if(MovedDesired.Tiles.begin(),MovedDesired.Tiles.end(),
|
||||
[&](const FVoxelTileKey& Tile){return Tile.Level>MaxLevel;}),MovedDesired.Tiles.end());
|
||||
}
|
||||
if(!Desired.Tiles.empty())
|
||||
{
|
||||
FIntVector volumeMin(INT32_MAX,INT32_MAX,INT32_MAX);
|
||||
@@ -191,7 +207,9 @@ namespace StandaloneStreamBenchmark
|
||||
}
|
||||
FIntVector FineMin(INT32_MAX,INT32_MAX,INT32_MAX),FineMax(INT32_MIN,INT32_MIN,INT32_MIN);
|
||||
FIntVector ClipMin(INT32_MAX,INT32_MAX,INT32_MAX),ClipMax(INT32_MIN,INT32_MIN,INT32_MIN);
|
||||
for(const FVoxelTileKey& Tile:Desired.Tiles)
|
||||
const auto AccumulateBounds=[&](const std::vector<FVoxelTileKey>& Tiles)
|
||||
{
|
||||
for(const FVoxelTileKey& Tile:Tiles)
|
||||
{
|
||||
const FIntVector Origin=Tile.OriginVoxels();const int32 Extent=Tile.ExtentVoxels();
|
||||
ClipMin.X=std::min(ClipMin.X,Origin.X);ClipMin.Y=std::min(ClipMin.Y,Origin.Y);ClipMin.Z=std::min(ClipMin.Z,Origin.Z);
|
||||
@@ -202,6 +220,9 @@ namespace StandaloneStreamBenchmark
|
||||
FineMax.X=std::max(FineMax.X,Origin.X+Extent);FineMax.Y=std::max(FineMax.Y,Origin.Y+Extent);FineMax.Z=std::max(FineMax.Z,Origin.Z+Extent);
|
||||
}
|
||||
}
|
||||
};
|
||||
AccumulateBounds(Desired.Tiles);
|
||||
if(bHasCameraStep)AccumulateBounds(MovedDesired.Tiles);
|
||||
if(FineMin.X==INT32_MAX||ClipMin.X==INT32_MAX)
|
||||
{std::fprintf(stderr,"[StreamPipeline] no desired detail or clipmap tiles.\n");return 7;}
|
||||
|
||||
@@ -242,7 +263,7 @@ namespace StandaloneStreamBenchmark
|
||||
const double ShapeSeconds=std::chrono::duration<double>(ShapeReady-ShapeBuildStart).count();
|
||||
FStandaloneStreamingGenerator Streamer(World,WorkerCount,bHashDensity,nullptr,MaxLevel,
|
||||
bSkipUvAndColors,bSkipMaterialGroups,bSkipUV,bSkipColors,&Shape,bProfileColumnOverlap,
|
||||
FarHorizonLOD8InteriorStepVoxels);
|
||||
FarHorizonLOD8InteriorStepVoxels,bUseSharedColumnCache);
|
||||
if(bScreenSpaceGate)Streamer.SetMesherStageProfilingEnabled(true);
|
||||
const auto StreamStart=Clock::now();
|
||||
const FStandaloneStreamingUpdate Update=Streamer.StartWithDesired(Desired);
|
||||
@@ -287,13 +308,19 @@ namespace StandaloneStreamBenchmark
|
||||
return X^(Y+static_cast<std::size_t>(0x9E3779B9u)+(X<<6)+(X>>2));
|
||||
}
|
||||
};
|
||||
struct FFirstColumnUse
|
||||
struct FColumnEvaluationEvent
|
||||
{
|
||||
FColumnXY XY;
|
||||
FVoxelTileKey Tile;
|
||||
std::uint64_t WorkerHash=0;
|
||||
std::uint64_t Sequence=0;
|
||||
bool bOutsideTileFootprint=false;
|
||||
};
|
||||
std::array<std::unordered_map<FColumnXY,std::vector<FFirstColumnUse>,FColumnXYHash>,9> ColumnUsesByLOD;
|
||||
std::array<std::uint64_t,9> ColumnDuplicateEventsByLOD{},NeighbourDuplicateEventsByLOD{};
|
||||
std::array<std::vector<FColumnEvaluationEvent>,9> ColumnEventsByLOD;
|
||||
std::array<std::uint64_t,9> ColumnEvaluationEventsByLOD{},UniqueColumnCoordinatesByLOD{};
|
||||
std::array<std::uint64_t,9> ColumnDuplicateEventsByLOD{},SameWorkerEvictedRepeatsByLOD{};
|
||||
std::array<std::uint64_t,9> DifferentWorkerRepeatsByLOD{},HaloNormalRepeatsByLOD{};
|
||||
std::array<std::uint64_t,9> HaloNormalEvaluationsByLOD{},NeighbourDuplicateEventsByLOD{};
|
||||
std::array<std::uint64_t,9> CrossWorkerNeighbourDuplicateEventsByLOD{};
|
||||
std::array<std::uint64_t,9> ColumnCaptureTilesByLOD{},ExpectedTilesByLOD{};
|
||||
std::array<bool,9> ColumnCaptureCompleteByLOD{};
|
||||
@@ -376,33 +403,12 @@ namespace StandaloneStreamBenchmark
|
||||
++ColumnCaptureTilesByLOD[LOD];
|
||||
ColumnCaptureCompleteByLOD[LOD]=ColumnCaptureCompleteByLOD[LOD]
|
||||
&&Tile.CoreDiagnostics.columnCoordinateCaptureComplete;
|
||||
std::vector<std::array<std::int64_t,2>> Coordinates=
|
||||
Tile.CoreDiagnostics.evaluatedColumnCoordinates;
|
||||
std::sort(Coordinates.begin(),Coordinates.end());
|
||||
Coordinates.erase(std::unique(Coordinates.begin(),Coordinates.end()),Coordinates.end());
|
||||
for(const auto& Coordinate:Coordinates)
|
||||
auto& Events=ColumnEventsByLOD[LOD];
|
||||
for(const auto& Coordinate:Tile.CoreDiagnostics.evaluatedColumnCoordinates)
|
||||
{
|
||||
const FColumnXY XY{Coordinate[0],Coordinate[1]};
|
||||
auto [Uses,Inserted]=ColumnUsesByLOD[LOD].try_emplace(XY);
|
||||
std::vector<FFirstColumnUse>& PriorUses=Uses->second;
|
||||
if(Inserted)
|
||||
{
|
||||
PriorUses.push_back({Tile.Key,Tile.CoreDiagnostics.columnEvaluationWorkerHash});
|
||||
continue;
|
||||
}
|
||||
++ColumnDuplicateEventsByLOD[LOD];
|
||||
bool bHasNeighbourDuplicate=false,bHasCrossWorkerNeighbourDuplicate=false;
|
||||
for(const FFirstColumnUse& Prior:PriorUses)
|
||||
{
|
||||
if(!AreSameLODNeighbours(Prior.Tile,Tile.Key))continue;
|
||||
bHasNeighbourDuplicate=true;
|
||||
bHasCrossWorkerNeighbourDuplicate=bHasCrossWorkerNeighbourDuplicate
|
||||
||Prior.WorkerHash!=Tile.CoreDiagnostics.columnEvaluationWorkerHash;
|
||||
}
|
||||
if(bHasNeighbourDuplicate)++NeighbourDuplicateEventsByLOD[LOD];
|
||||
if(bHasCrossWorkerNeighbourDuplicate)
|
||||
++CrossWorkerNeighbourDuplicateEventsByLOD[LOD];
|
||||
PriorUses.push_back({Tile.Key,Tile.CoreDiagnostics.columnEvaluationWorkerHash});
|
||||
Events.push_back({{Coordinate.x,Coordinate.y},Tile.Key,
|
||||
Tile.CoreDiagnostics.columnEvaluationWorkerHash,Coordinate.sequence,
|
||||
Coordinate.outsideTileFootprint});
|
||||
}
|
||||
}
|
||||
WholeViewReady=std::max(WholeViewReady,Tile.CompletedAt);
|
||||
@@ -449,6 +455,45 @@ namespace StandaloneStreamBenchmark
|
||||
bFullDetailReported=true;
|
||||
}
|
||||
}
|
||||
if(bHasCameraStep)
|
||||
{
|
||||
const auto MovingStart=Clock::now();
|
||||
const auto MovingUpdate=Streamer.UpdateDesiredWithSet(MovedDesired,Desired.Tiles,1);
|
||||
std::array<std::uint64_t,9> MovingTilesByLOD{};
|
||||
std::array<double,9> MovingWorkerSecondsByLOD{};
|
||||
int32 MovingFinished=0;
|
||||
while(MovingFinished<MovingUpdate.NewJobs)
|
||||
{
|
||||
std::vector<FStandaloneCpuTile> Ready=Streamer.TakeReady();
|
||||
if(Ready.empty())
|
||||
{std::this_thread::sleep_for(std::chrono::milliseconds(1));continue;}
|
||||
for(const FStandaloneCpuTile& Tile:Ready)
|
||||
{
|
||||
++MovingFinished;
|
||||
if(Tile.Key.Level>=0&&Tile.Key.Level<static_cast<int32>(MovingTilesByLOD.size()))
|
||||
{
|
||||
const std::size_t LOD=static_cast<std::size_t>(Tile.Key.Level);
|
||||
++MovingTilesByLOD[LOD];
|
||||
MovingWorkerSecondsByLOD[LOD]+=Tile.WorkerSeconds;
|
||||
}
|
||||
}
|
||||
}
|
||||
const double MovingWallSeconds=std::chrono::duration<double>(Clock::now()-MovingStart).count();
|
||||
const double StepDistanceMeters=std::sqrt(
|
||||
static_cast<double>(CameraStepVoxels.X)*CameraStepVoxels.X
|
||||
+static_cast<double>(CameraStepVoxels.Y)*CameraStepVoxels.Y
|
||||
+static_cast<double>(CameraStepVoxels.Z)*CameraStepVoxels.Z)
|
||||
*static_cast<double>(World.CoreVoxelSizeMicrons)/1000000.0;
|
||||
std::printf("[MovingCameraStep] step_voxels=(%.3f,%.3f,%.3f) distance_m=%.3f desired_tiles=%zu new_tiles=%d completed=%d wall_s=%.6f shared_column_cache=%s\n",
|
||||
CameraStepVoxels.X,CameraStepVoxels.Y,CameraStepVoxels.Z,StepDistanceMeters,
|
||||
MovedDesired.Tiles.size(),MovingUpdate.NewJobs,MovingFinished,MovingWallSeconds,
|
||||
bUseSharedColumnCache?"on":"off");
|
||||
for(std::size_t LOD=0;LOD<MovingTilesByLOD.size();++LOD)
|
||||
if(MovingTilesByLOD[LOD]!=0)
|
||||
std::printf("[MovingCameraStepLOD] lod=%zu tiles=%llu worker_sum_s=%.6f\n",
|
||||
LOD,static_cast<unsigned long long>(MovingTilesByLOD[LOD]),
|
||||
MovingWorkerSecondsByLOD[LOD]);
|
||||
}
|
||||
Streamer.Stop();
|
||||
bool bScreenSpaceGatePassing=true;
|
||||
if(bScreenSpaceGate)
|
||||
@@ -472,6 +517,58 @@ namespace StandaloneStreamBenchmark
|
||||
bSceneExportPassing?"PASS":"FAIL",MeshExportPath.string().c_str());
|
||||
bSceneExportPassing=bSceneExportPassing&&!FileSizeError;
|
||||
}
|
||||
if(bProfileColumnOverlap)
|
||||
{
|
||||
struct FColumnPriorUse
|
||||
{
|
||||
FVoxelTileKey Tile;
|
||||
std::uint64_t WorkerHash=0;
|
||||
};
|
||||
for(std::size_t LOD=0;LOD<ColumnEventsByLOD.size();++LOD)
|
||||
{
|
||||
auto& Events=ColumnEventsByLOD[LOD];
|
||||
std::sort(Events.begin(),Events.end(),[](const FColumnEvaluationEvent& A,
|
||||
const FColumnEvaluationEvent& B){return A.Sequence<B.Sequence;});
|
||||
std::unordered_map<FColumnXY,std::vector<FColumnPriorUse>,FColumnXYHash> UsesByXY;
|
||||
UsesByXY.reserve(Events.size());
|
||||
for(const FColumnEvaluationEvent& Event:Events)
|
||||
{
|
||||
++ColumnEvaluationEventsByLOD[LOD];
|
||||
if(Event.bOutsideTileFootprint)++HaloNormalEvaluationsByLOD[LOD];
|
||||
auto [Found,Inserted]=UsesByXY.try_emplace(Event.XY);
|
||||
auto& PriorUses=Found->second;
|
||||
if(Inserted)
|
||||
{
|
||||
++UniqueColumnCoordinatesByLOD[LOD];
|
||||
PriorUses.push_back({Event.Tile,Event.WorkerHash});
|
||||
continue;
|
||||
}
|
||||
++ColumnDuplicateEventsByLOD[LOD];
|
||||
if(Event.bOutsideTileFootprint)
|
||||
++HaloNormalRepeatsByLOD[LOD];
|
||||
else
|
||||
{
|
||||
const bool bSameWorkerSeen=std::any_of(PriorUses.begin(),PriorUses.end(),
|
||||
[&](const FColumnPriorUse& Prior)
|
||||
{return Prior.WorkerHash==Event.WorkerHash;});
|
||||
if(bSameWorkerSeen)++SameWorkerEvictedRepeatsByLOD[LOD];
|
||||
else ++DifferentWorkerRepeatsByLOD[LOD];
|
||||
}
|
||||
bool bHasNeighbourDuplicate=false,bHasCrossWorkerNeighbourDuplicate=false;
|
||||
for(const FColumnPriorUse& Prior:PriorUses)
|
||||
{
|
||||
if(!AreSameLODNeighbours(Prior.Tile,Event.Tile))continue;
|
||||
bHasNeighbourDuplicate=true;
|
||||
bHasCrossWorkerNeighbourDuplicate=bHasCrossWorkerNeighbourDuplicate
|
||||
||Prior.WorkerHash!=Event.WorkerHash;
|
||||
}
|
||||
if(bHasNeighbourDuplicate)++NeighbourDuplicateEventsByLOD[LOD];
|
||||
if(bHasCrossWorkerNeighbourDuplicate)
|
||||
++CrossWorkerNeighbourDuplicateEventsByLOD[LOD];
|
||||
PriorUses.push_back({Event.Tile,Event.WorkerHash});
|
||||
}
|
||||
}
|
||||
}
|
||||
bool bColumnOverlapCapturePassing=true;
|
||||
if(bProfileColumnOverlap)
|
||||
{
|
||||
@@ -480,11 +577,16 @@ namespace StandaloneStreamBenchmark
|
||||
const bool bLODComplete=ColumnCaptureCompleteByLOD[LOD]
|
||||
&&ColumnCaptureTilesByLOD[LOD]==ExpectedTilesByLOD[LOD];
|
||||
bColumnOverlapCapturePassing=bColumnOverlapCapturePassing&&bLODComplete;
|
||||
std::printf("[WorldAssemblerLODColumnOverlap] lod=%zu tiles=%llu expected_tiles=%llu unique_xy_columns=%zu repeated_tile_evaluations=%llu neighbouring_duplicate_evaluations=%llu cross_worker_neighbouring_duplicates=%llu capture_complete=%d\n",
|
||||
std::printf("[WorldAssemblerLODColumnOverlap] lod=%zu tiles=%llu expected_tiles=%llu total_column_evaluations=%llu unique_xy_columns=%llu repeated_column_evaluations=%llu same_worker_evicted_repeats=%llu different_worker_repeats=%llu halo_normal_repeats=%llu halo_normal_evaluations=%llu neighbouring_duplicate_evaluations=%llu cross_worker_neighbouring_duplicates=%llu capture_complete=%d\n",
|
||||
LOD,static_cast<unsigned long long>(ColumnCaptureTilesByLOD[LOD]),
|
||||
static_cast<unsigned long long>(ExpectedTilesByLOD[LOD]),
|
||||
ColumnUsesByLOD[LOD].size(),
|
||||
static_cast<unsigned long long>(ColumnEvaluationEventsByLOD[LOD]),
|
||||
static_cast<unsigned long long>(UniqueColumnCoordinatesByLOD[LOD]),
|
||||
static_cast<unsigned long long>(ColumnDuplicateEventsByLOD[LOD]),
|
||||
static_cast<unsigned long long>(SameWorkerEvictedRepeatsByLOD[LOD]),
|
||||
static_cast<unsigned long long>(DifferentWorkerRepeatsByLOD[LOD]),
|
||||
static_cast<unsigned long long>(HaloNormalRepeatsByLOD[LOD]),
|
||||
static_cast<unsigned long long>(HaloNormalEvaluationsByLOD[LOD]),
|
||||
static_cast<unsigned long long>(NeighbourDuplicateEventsByLOD[LOD]),
|
||||
static_cast<unsigned long long>(CrossWorkerNeighbourDuplicateEventsByLOD[LOD]),
|
||||
bLODComplete?1:0);
|
||||
|
||||
@@ -428,12 +428,14 @@ public:
|
||||
bool bInSkipColors=false,
|
||||
VoxelForgeCore::World* InCoreWorld=nullptr,
|
||||
bool bInCaptureColumnCoordinates=false,
|
||||
int32 InLod8HorizonInteriorStepVoxels=0)
|
||||
int32 InLod8HorizonInteriorStepVoxels=0,
|
||||
bool bInSharedColumnCache=true)
|
||||
:World(&InWorld),WorkerCount(FMath::Clamp(InWorkerCount,1,16)),bHashDensity(bInHashDensity),
|
||||
MeshCache(InMeshCache),MaxLevelFilter(InMaxLevelFilter),
|
||||
CoreWorld(InCoreWorld),bSkipUvAndColors(bInSkipUvAndColors),bSkipMaterialGroups(bInSkipMaterialGroups),
|
||||
bSkipUV(bInSkipUV),bSkipColors(bInSkipColors),
|
||||
bCaptureColumnCoordinates(bInCaptureColumnCoordinates),
|
||||
bSharedColumnCache(bInSharedColumnCache),
|
||||
Lod8HorizonInteriorStepVoxels(std::max(0,InLod8HorizonInteriorStepVoxels))
|
||||
{
|
||||
VoxelGenLOD::InitializeConsoleSwitches();
|
||||
@@ -816,6 +818,7 @@ private:
|
||||
Options.skipUvAndColors=bSkipUvAndColors;
|
||||
Options.skipMaterialGroups=bSkipMaterialGroups;Options.skipUv=bSkipUV;Options.skipColors=bSkipColors;
|
||||
Options.captureColumnCoordinates=bCaptureColumnCoordinates;
|
||||
Options.useSharedColumnCache=bSharedColumnCache;
|
||||
Options.lod8HorizonInteriorStepVoxels=Lod8HorizonInteriorStepVoxels;
|
||||
Options.profileMesherStages=bProfileMesherStages&&!ForceGenerate;
|
||||
Options.diagnosticGlobalOctaveDrop=DiagnosticGlobalOctaveDrop;
|
||||
@@ -993,6 +996,7 @@ private:
|
||||
bool bSkipUV=false;
|
||||
bool bSkipColors=false;
|
||||
bool bCaptureColumnCoordinates=false;
|
||||
bool bSharedColumnCache=true;
|
||||
int32 Lod8HorizonInteriorStepVoxels=0;
|
||||
bool bProfileMesherStages=false;
|
||||
std::atomic<uint64> CacheHitCount{0};
|
||||
|
||||
Reference in new issue
Block a user