From 4b1108bf8654700c2d9a489d39f66c8a6e1c696a Mon Sep 17 00:00:00 2001 From: Fr0zka Date: Mon, 19 Oct 2026 21:46:15 +0200 Subject: [PATCH] wip(assembler): round L7 -- shared column cache across workers: whole view 16-19% faster, world unchanged SharedColumnCache.h: a world-owned, bounded 4-way second-tier cache of exact column values (heightfield and floating islands), keyed by recipe/plan identity plus content hash, XY and island id. Lock-free seqlock reads; set-local spin lock on misses. The FLazy thread-local tier stays first. 83.6% of repeated column work was cross-worker; formula evaluations -60.9%. W12 L1 view: Luna 2.165 -> 1.754 s; reviewer interleaved 2.059 -> 1.722 s. 8 m crossing new tiles 82 -> 52 ms. +66 MiB peak RSS. All hashes (whole view, per-LOD, exact queries, islands seed 0) identical at W1/W8/W12, Clang off and GCC default. MSVC/UBT not compiled here. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01CSFYgsdW8ejZSyiMf4qRSX --- ARCHITECTURE.md | 26 +++ CODEMAP.md | 3 +- .../VoxelForgeCore/SharedColumnCache.h | 183 +++++++++++++++++ .../include/VoxelForgeCore/VoxelForgeCore.h | 19 +- Standalone/Core/src/CoreNamespacePreamble.h | 1 + Standalone/Core/src/FloatingIslandsInternal.h | 101 ++++++---- Standalone/Core/src/VoxelForgeCore.cpp | 46 +++++ Standalone/Core/src/WorldAssemblerInternal.h | 65 +++++-- .../WorldAssemblerM3/Linux7/REPORT.md | 171 ++++++++++++++++ Standalone/Source/StreamBenchmarkHeadless.cpp | 38 +++- Standalone/Source/StreamBenchmarkPipeline.h | 184 ++++++++++++++---- Standalone/Source/StreamingWorld.h | 6 +- 12 files changed, 751 insertions(+), 92 deletions(-) create mode 100644 Standalone/Core/include/VoxelForgeCore/SharedColumnCache.h create mode 100644 Standalone/Research/WorldAssemblerM3/Linux7/REPORT.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 9990369..52481c2 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -732,6 +732,15 @@ driven by `EditorBrush*` props. its default member initialization was never observable; removing that generated TLS leaf avoids constructor work without changing a height or density. Integer queries still read the cached payload, and fractional queries still use the same direct locals. +- **Core shared column cache (L7)**: `Heightfield2D` and FloatingIslands keep their `FLazy` + worker-local caches as the first tier. A world-owned, fixed-size four-way table is the second + tier, keyed by recipe and plan identity plus content hash, exact XY, and an optional exact + sub-domain identity (FloatingIslands uses the island stable ID). Hits read versioned atomic + entries without taking a lock; misses take only their set's spin lock, recheck, compute once, and + publish. The tables are bounded and die with the world after its tile work has drained. This + deliberately adds shared reads on column-cache hits while preserving the no-lock hot-path rule; + S14's purely worker-local cache remains the first lookup. Keep large worker-local storage behind + `FLazy`, and do not make cache values or keys depend on engine objects. - **Sampler unknown-source audit**: the earlier plugin-side `FSurfaceColumn` constructor PCs no longer occur after the trivial-payload change. The largest remaining LOD0 unknown cluster in the final sampler raw CSV is `UnrealEditor-Core.dll + RVA 0xC080` (runtime PCs around @@ -1797,6 +1806,23 @@ page was added: cross-worker repeats alone do not establish a net page-cache win immutable-page lifetime rules would add ownership and memory cost. Round 18 changed no output or recipe version. See [`Standalone/Research/WorldAssemblerM3/Round18/REPORT.md`](Standalone/Research/WorldAssemblerM3/Round18/REPORT.md). +### 8.35 Shared Core column cache — World Assembler L7 + +L7 measured 1.34 million repeated formula evaluations on the W12 L1 view before sharing: 83.6% +were another worker's XY, 4.0% were same-worker repeats after eviction, and 12.0% were halo/normal +repeats. The world-owned shared second tier cut measured formula executions by 60.9% and improved +median whole-view-ready time by 19.0%; moving 8 m and generating 144 entering tiles improved median +wall time by 36.1%. Peak RSS rose by about 66 MiB. The cache is a bounded exact-value table rather +than published pages; no scheduling affinity or page prefetch was needed to clear the measured +whole-view threshold. + +The thread-local first tier remains because it avoids shared atomics for local hits. The shared +table has no lock on hits; only a miss takes a set-local spin lock. FloatingIslands' cache identity +includes its island stable ID because distinct anchors can request the same XY with different +column metrics. All W1/W12 L1 hashes, exact queries, and the seed-0 FloatingIslands fixture match +the pre-L7 binary with Clang FP contraction off and GCC defaults. See +[`Standalone/Research/WorldAssemblerM3/Linux7/REPORT.md`](Standalone/Research/WorldAssemblerM3/Linux7/REPORT.md). + ## 9. Multiplayer model (listen-server first, dedicated-friendly) > **Status: transport is DESIGN ONLY — nothing is networked in-tree yet** (no `Replicated`/`HasAuthority`/RPCs; a diff --git a/CODEMAP.md b/CODEMAP.md index d924f54..8210927 100644 --- a/CODEMAP.md +++ b/CODEMAP.md @@ -158,7 +158,8 @@ remains `0`, and lateral regions remain gated off. | `VoxelForge.Build.cs` | Deps: Core, CoreUObject, Engine, **GameplayTags**, **RealtimeMeshComponent**, `VoxelForgeCore`. `AVoxelWorld::bUseVoxelForgeCore` opts into the Core runtime route; it defaults off. | | `../../VoxelForgeCore/VoxelForgeCore.Build.cs` | C++20 runtime module. Reads `Standalone/Core/core_sources.txt`, validates one include shim per canonical source, and fails UBT on missing, extra, or mismatched shims. No unity/PCH; private copied implementation stays under `vfcore::detail`. | | `../../VoxelForgeCore/Private/Manifest/VFCore_*.cpp` | Eighteen one-source UBT shims that include canonical files under `Standalone/`; they do not copy Core implementation. The API-boundary row includes `Core/src/VoxelForgeCore.cpp` directly. | -| `../../Standalone/Core/include/VoxelForgeCore/ThreadLocalStorage.h` | `VoxelForgeCoreThreadLocal::FLazy`: thread-local handle that heap-allocates a worker-private cache on first use. All large Core and legacy caches use it (HeightfieldColumnCache, IslandColumnCache, GFloorReliefCache, GRoomGraphCache, GSurfColCache, tunnel/diff caches). See ARCHITECTURE §8.10. | +| `../../Standalone/Core/include/VoxelForgeCore/ThreadLocalStorage.h` | `VoxelForgeCoreThreadLocal::FLazy`: thread-local handle that heap-allocates a worker-private cache on first use. Large worker-local caches use it (HeightfieldColumnCache, IslandColumnCache, GFloorReliefCache, GRoomGraphCache, GSurfColCache, tunnel/diff caches). See ARCHITECTURE §8.10. | +| `../../Standalone/Core/include/VoxelForgeCore/SharedColumnCache.h` | Bounded four-way second-tier cache for exact per-column values. Readers use versioned atomic slots without a lock; misses use a set-local spin lock. World-owned lifetime, recipe/plan identity and content hashes, XY and optional sub-domain identity. SurfaceWorld and FloatingIslands share this mechanism; see ARCHITECTURE §8.35. | | `../../Standalone/Core/src/CoreNamespacePreamble.h` | First include of every Core TU. Forces FP contraction off per compiler (Clang/MSVC/GCC pragmas; unknown compiler = `#error`) and defines `vfcore::fp_contract::selfTest`, which `World::compile` runs before building: a fused multiply-add (e.g. Clang `-ffp-contract=fast`) refuses world generation. The world depends on contraction (Linux2/Linux3 reports), so never remove either. | | `../../VoxelForgeCore/Private/ModuleEntry/` | Unreal module entry and an isolated `CoreMinimal.h` shim for the actual engine header. | | `../../Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h` | Standard-library-only public Core API. `VOXELFORGECORE_API` exports the API from the Unreal module and is empty for standalone builds. | diff --git a/Standalone/Core/include/VoxelForgeCore/SharedColumnCache.h b/Standalone/Core/include/VoxelForgeCore/SharedColumnCache.h new file mode 100644 index 0000000..13d25b0 --- /dev/null +++ b/Standalone/Core/include/VoxelForgeCore/SharedColumnCache.h @@ -0,0 +1,183 @@ +#pragma once + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace VoxelForgeCoreShared +{ + // Exact input identity for a pure XY column value. LOD is deliberately not part of this key: + // the same recipe/plan column has the same bits at every sampling step. + struct FColumnCacheKey + { + std::uint64_t recipeIdentity = 0; + std::uint64_t recipeContentHash = 0; + std::uint64_t planIdentity = 0; + std::uint64_t planContentHash = 0; + std::int64_t xSubvoxels = 0; + std::int64_t ySubvoxels = 0; + // Optional exact sub-domain identity for recipes with several independent columns + // at the same XY (for example, a floating-island anchor). + std::uint64_t contextIdentity = 0; + + std::array words() const noexcept + { + return {recipeIdentity, recipeContentHash, planIdentity, planContentHash, + static_cast(xSubvoxels), static_cast(ySubvoxels), + contextIdentity}; + } + }; + + // A bounded, world-owned second-level cache. Existing thread-local caches stay first, so + // their ordinary hit path remains private and lock-free. Shared hits read atomic words; only + // a shared miss takes the set spin lock. Entries are four-way set associative, evict on + // collision, and carry pointer identities plus content hashes to reject stale reuse. + // Les caches TLS restent le premier niveau; seul un defaut partage prend le verrou du groupe. + template + class TSharedColumnValueCache + { + static_assert(SetCount > 0 && (SetCount & (SetCount - 1)) == 0, + "column cache set count must be a power of two"); + static_assert(Ways > 0 && Ways <= 16, "unsupported column cache associativity"); + static_assert(std::is_trivially_copyable_v, + "shared column values must be trivially copyable"); + + static constexpr std::size_t ValueWordCount = (sizeof(TValue) + 7) / 8; + using FKeyWords = std::array; + using FValueWords = std::array; + + struct FEntry + { + std::atomic Version{0}; + std::atomic Tag{0}; + std::array, 7> Key{}; + std::array, ValueWordCount> Value{}; + }; + + struct FSet + { + std::atomic_flag Writer = ATOMIC_FLAG_INIT; + std::uint8_t NextWay = 0; + std::array Entries{}; + }; + + struct FSpinGuard + { + explicit FSpinGuard(std::atomic_flag& value) : Flag(value) + { + while (Flag.test_and_set(std::memory_order_acquire)) + std::this_thread::yield(); + } + ~FSpinGuard() { Flag.clear(std::memory_order_release); } + std::atomic_flag& Flag; + }; + + public: + TSharedColumnValueCache() : Sets(std::make_unique(SetCount)) {} + TSharedColumnValueCache(const TSharedColumnValueCache&) = delete; + TSharedColumnValueCache& operator=(const TSharedColumnValueCache&) = delete; + + static TSharedColumnValueCache* setThreadActive(TSharedColumnValueCache* value) noexcept + { + TSharedColumnValueCache* previous = Active; + Active = value; + return previous; + } + + static TSharedColumnValueCache* threadActive() noexcept { return Active; } + + template + TValue getOrCompute(const FColumnCacheKey& key, TFactory&& factory, + bool& computed) + { + computed = false; + const FKeyWords keyWords = key.words(); + const std::uint64_t tag = Hash(keyWords); + const std::size_t setIndex = static_cast(tag) & (SetCount - 1); + FSet& set = Sets[setIndex]; + TValue value{}; + if (TryReadSet(set, keyWords, tag, value)) return value; + + FSpinGuard lock(set.Writer); + if (TryReadSet(set, keyWords, tag, value)) return value; + + value = std::forward(factory)(); + computed = true; + FValueWords valueWords{}; + std::memcpy(valueWords.data(), &value, sizeof(TValue)); + + std::size_t replacement = Ways; + for (std::size_t way = 0; way < Ways; ++way) + if (set.Entries[way].Version.load(std::memory_order_seq_cst) == 0) + { replacement = way; break; } + if (replacement == Ways) + { + replacement = set.NextWay; + set.NextWay = static_cast((set.NextWay + 1) % Ways); + } + FEntry& entry = set.Entries[replacement]; + std::uint64_t version = entry.Version.load(std::memory_order_seq_cst); + if (version & 1u) ++version; + entry.Version.store(version + 1, std::memory_order_seq_cst); + for (std::size_t word = 0; word < keyWords.size(); ++word) + entry.Key[word].store(keyWords[word], std::memory_order_seq_cst); + for (std::size_t word = 0; word < valueWords.size(); ++word) + entry.Value[word].store(valueWords[word], std::memory_order_seq_cst); + entry.Tag.store(tag, std::memory_order_seq_cst); + entry.Version.store(version + 2, std::memory_order_seq_cst); + return value; + } + + private: + static std::uint64_t Hash(const FKeyWords& words) noexcept + { + std::uint64_t hash = 0x9E3779B185EBCA87ull; + for (const std::uint64_t word : words) + { + hash ^= word + 0x9E3779B97F4A7C15ull + (hash << 6) + (hash >> 2); + hash ^= hash >> 30; + hash *= 0xBF58476D1CE4E5B9ull; + hash ^= hash >> 27; + hash *= 0x94D049BB133111EBull; + hash ^= hash >> 31; + } + return hash == 0 ? 1 : hash; + } + + static bool TryRead(const FEntry& entry, const FKeyWords& keyWords, + std::uint64_t tag, TValue& out) + { + const std::uint64_t before = entry.Version.load(std::memory_order_seq_cst); + if (before == 0 || (before & 1u) + || entry.Tag.load(std::memory_order_seq_cst) != tag) return false; + FKeyWords foundKey{}; + for (std::size_t word = 0; word < foundKey.size(); ++word) + foundKey[word] = entry.Key[word].load(std::memory_order_seq_cst); + if (foundKey != keyWords) return false; + FValueWords foundValue{}; + for (std::size_t word = 0; word < foundValue.size(); ++word) + foundValue[word] = entry.Value[word].load(std::memory_order_seq_cst); + const std::uint64_t after = entry.Version.load(std::memory_order_seq_cst); + if (before != after || (after & 1u)) return false; + std::memcpy(&out, foundValue.data(), sizeof(TValue)); + return true; + } + + static bool TryReadSet(const FSet& set, const FKeyWords& keyWords, + std::uint64_t tag, TValue& out) + { + for (const FEntry& entry : set.Entries) + if (TryRead(entry, keyWords, tag, out)) return true; + return false; + } + + std::unique_ptr Sets; + inline static thread_local TSharedColumnValueCache* Active = nullptr; + }; +} diff --git a/Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h b/Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h index b6caf5f..46f1a66 100644 --- a/Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h +++ b/Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h @@ -110,6 +110,13 @@ namespace VoxelForgeCore enum class TileClass : std::uint8_t { allSolid, allAir, mixed, unknown }; + struct HeightfieldColumnEvaluation + { + std::int64_t x = 0, y = 0; + std::uint64_t sequence = 0; + bool outsideTileFootprint = false; + }; + struct TileGenerationStats { TileClass tileClass = TileClass::unknown; @@ -124,6 +131,8 @@ namespace VoxelForgeCore std::uint64_t columnCacheHits = 0; std::uint64_t columnCacheMisses = 0; std::uint64_t columnCacheConflictMisses = 0; + std::uint64_t sharedColumnCacheHits = 0; + std::uint64_t columnEvaluations = 0; double classificationMilliseconds = 0.0; double columnComputeMilliseconds = 0.0; double columnEvaluationExclusiveMilliseconds = 0.0; @@ -168,7 +177,7 @@ namespace VoxelForgeCore bool capRefined = false; std::uint64_t columnEvaluationWorkerHash = 0; bool columnCoordinateCaptureComplete = true; - std::vector> evaluatedColumnCoordinates; + std::vector evaluatedColumnCoordinates; }; struct TileMesh @@ -651,10 +660,12 @@ namespace VoxelForgeCore std::uint64_t columnHits = 0; std::uint64_t columnMisses = 0; std::uint64_t columnConflictMisses = 0; + std::uint64_t sharedColumnHits = 0; + std::uint64_t columnEvaluations = 0; double columnComputeMilliseconds = 0.0; bool captureCoordinates = false; bool coordinateCaptureComplete = true; - std::vector> evaluatedCoordinates; + std::vector evaluatedCoordinates; }; struct FieldInfluenceBounds @@ -705,6 +716,9 @@ namespace VoxelForgeCore std::int64_t maximumZSubvoxels, std::int32_t stepVoxels) const noexcept; static void resetThreadColumnCacheStats(bool captureCoordinates = false, bool clearCache = false, std::uint32_t octaveDrop = 0) noexcept; + static void setThreadColumnCaptureBounds(std::int64_t minimumXSubvoxels, + std::int64_t minimumYSubvoxels, std::int64_t maximumXSubvoxels, + std::int64_t maximumYSubvoxels) noexcept; static void setThreadColumnOctaveDrop(std::uint32_t octaveDrop) noexcept; static std::uint32_t threadColumnOctaveDrop() noexcept; static void setThreadColumnInterpolationStrideSubvoxels(std::int64_t stride) noexcept; @@ -1038,6 +1052,7 @@ namespace VoxelForgeCore // Opt-in diagnostic only; records XY columns evaluated by this LOD8 tile. // Diagnostic seulement; enregistre les colonnes XY evaluees par cette tuile LOD8. bool captureColumnCoordinates = false; + bool useSharedColumnCache = true; // Separate cold pass only; samples the MC inner stages without changing output. // Passe froide separee seulement; mesure les etapes internes MC sans changer la sortie. bool profileMesherStages = false; diff --git a/Standalone/Core/src/CoreNamespacePreamble.h b/Standalone/Core/src/CoreNamespacePreamble.h index 7dfdaff..78a6670 100644 --- a/Standalone/Core/src/CoreNamespacePreamble.h +++ b/Standalone/Core/src/CoreNamespacePreamble.h @@ -55,6 +55,7 @@ #include "Json.h" #include "VoxelForgeCore/VoxelForgeCore.h" #include "VoxelForgeCore/ThreadLocalStorage.h" +#include "VoxelForgeCore/SharedColumnCache.h" // Keep Core's expression-level FMA behavior fixed regardless of the toolchain setting. // Clang's "fast" mode ignores the pragma, so Core's runtime self-test rejects that build. diff --git a/Standalone/Core/src/FloatingIslandsInternal.h b/Standalone/Core/src/FloatingIslandsInternal.h index 3e54f35..fbb8555 100644 --- a/Standalone/Core/src/FloatingIslandsInternal.h +++ b/Standalone/Core/src/FloatingIslandsInternal.h @@ -425,6 +425,9 @@ namespace floating_island_detail struct FIslandColumnCacheEntry { bool bValid=false; + const FloatingIslandRecipeV1* RecipeIdentity=nullptr; + const FloatingIslandPlanV1* PlanIdentity=nullptr; + std::uint64_t RecipeContentHash=0,PlanContentHash=0; std::uint32_t Seed=0; std::uint64_t StableId=0; std::int64_t SampleX=0,SampleY=0,CenterX=0,CenterY=0,TopZ=0,Radius=0,TopHalf=0; @@ -432,9 +435,12 @@ namespace floating_island_detail double DeltaX=0.0,DeltaY=0.0,Radial=0.0,Angle=0.0,LobePhase=0.0,TopSurface=0.0; }; + using FSharedIslandColumnCache = + VoxelForgeCoreShared::TSharedColumnValueCache; + inline const FIslandColumnCacheEntry& IslandColumnCache( - const FloatingIslandRecipeV1& recipe,const FloatingIslandAnchorV1& island, - double x,double y) + const FloatingIslandRecipeV1& recipe,const FloatingIslandPlanV1& plan, + const FloatingIslandAnchorV1& island,double x,double y) { // Density samples arrive on integer Core lattice columns; reuse the exact column metrics // across Z samples without changing the field value or touching shared world state. @@ -450,7 +456,10 @@ namespace floating_island_detail std::uint64_t IndexKey=island.stableId ^(static_cast(SampleX)*0x9e3779b97f4a7c15ull) ^(static_cast(SampleY)*0xc2b2ae3d27d4eb4full) - ^(static_cast(recipe.seed)<<23); + ^(static_cast(recipe.seed)<<23) + ^recipe.contentHash^std::rotl(plan.contentHash,17) + ^static_cast(reinterpret_cast(&recipe)) + ^std::rotl(static_cast(reinterpret_cast(&plan)),31); IndexKey^=IndexKey>>33; IndexKey*=0xff51afd7ed558ccdull; IndexKey^=IndexKey>>33; @@ -459,7 +468,9 @@ namespace floating_island_detail for(std::size_t Way=0;Way(island.centerSubvoxels[0]); - Entry.DeltaY=y-static_cast(island.centerSubvoxels[1]); - Entry.Radial=std::hypot(Entry.DeltaX,Entry.DeltaY); - Entry.Angle=std::atan2(Entry.DeltaY,Entry.DeltaX); - Entry.LobePhase=LobePhase(recipe,island); - const double EffectiveRadius=std::max(1.0,static_cast(island.radiusSubvoxels) - +LobeOffsetWithPhase(recipe,Entry.Angle,static_cast(island.topZSubvoxels), - Entry.LobePhase)); - const double Edge=std::clamp(Entry.Radial/EffectiveRadius,0.0,1.0); - const double Drop=(1.0-Q16ToDouble(recipe.topFlattenQ16)) - *static_cast(island.topHalfSubvoxels)*2.0*Edge*Edge; - Entry.TopSurface=static_cast(island.topZSubvoxels)-Drop; - return Entry; + const auto build=[&]() + { + FIslandColumnCacheEntry Entry; + Entry.bValid=true; + Entry.RecipeIdentity=&recipe; + Entry.PlanIdentity=&plan; + Entry.RecipeContentHash=recipe.contentHash; + Entry.PlanContentHash=plan.contentHash; + Entry.Seed=recipe.seed; + Entry.StableId=island.stableId; + Entry.SampleX=SampleX; + Entry.SampleY=SampleY; + Entry.CenterX=island.centerSubvoxels[0]; + Entry.CenterY=island.centerSubvoxels[1]; + Entry.TopZ=island.topZSubvoxels; + Entry.Radius=island.radiusSubvoxels; + Entry.TopHalf=island.topHalfSubvoxels; + Entry.TopFlattenQ16=recipe.topFlattenQ16; + Entry.ContourWarp=recipe.contourWarpSubvoxels; + Entry.DeltaX=x-static_cast(island.centerSubvoxels[0]); + Entry.DeltaY=y-static_cast(island.centerSubvoxels[1]); + Entry.Radial=std::hypot(Entry.DeltaX,Entry.DeltaY); + Entry.Angle=std::atan2(Entry.DeltaY,Entry.DeltaX); + Entry.LobePhase=LobePhase(recipe,island); + const double EffectiveRadius=std::max(1.0,static_cast(island.radiusSubvoxels) + +LobeOffsetWithPhase(recipe,Entry.Angle,static_cast(island.topZSubvoxels), + Entry.LobePhase)); + const double Edge=std::clamp(Entry.Radial/EffectiveRadius,0.0,1.0); + const double Drop=(1.0-Q16ToDouble(recipe.topFlattenQ16)) + *static_cast(island.topHalfSubvoxels)*2.0*Edge*Edge; + Entry.TopSurface=static_cast(island.topZSubvoxels)-Drop; + return Entry; + }; + FIslandColumnCacheEntry Entry{}; + if(auto* shared=FSharedIslandColumnCache::threadActive()) + { + const VoxelForgeCoreShared::FColumnCacheKey key{ + static_cast(reinterpret_cast(&recipe)), + recipe.contentHash, + static_cast(reinterpret_cast(&plan)), + plan.contentHash,SampleX,SampleY,island.stableId}; + bool computed=false; + Entry=shared->getOrCompute(key,build,computed); + } + else Entry=build(); + *Replacement=Entry; + return *Replacement; } inline std::int64_t ScaledQ16(std::int64_t value, std::int64_t scaleQ16) @@ -641,6 +673,7 @@ namespace floating_island_detail } inline double IslandDensity(const FloatingIslandRecipeV1& recipe, + const FloatingIslandPlanV1& plan, const FloatingIslandAnchorV1& island, double x, double y, double z) { @@ -665,7 +698,7 @@ namespace floating_island_detail std::max({z - top, radial - envelope, bottomZ - z}), x, y, z); } - const FIslandColumnCacheEntry& Column=IslandColumnCache(recipe,island,x,y); + const FIslandColumnCacheEntry& Column=IslandColumnCache(recipe,plan,island,x,y); const double dx=Column.DeltaX; const double dy=Column.DeltaY; const double radial=Column.Radial; @@ -733,7 +766,7 @@ namespace floating_island_detail double result = static_cast(MaxDensity); for (const auto& island : plan.islands) { - double group = IslandDensity(recipe, island, + double group = IslandDensity(recipe, plan, island, static_cast(x), static_cast(y), static_cast(z)); for (const auto& edge : plan.clusterEdges) { @@ -769,7 +802,7 @@ namespace floating_island_detail const auto evaluateLocalGroup = [&](std::size_t islandIndex) { const auto& island = plan.islands[islandIndex]; - double group = IslandDensity(recipe, island, + double group = IslandDensity(recipe, plan, island, static_cast(x), static_cast(y), static_cast(z)); for (const std::uint32_t edgeIndex : featureIndices) { @@ -795,7 +828,7 @@ namespace floating_island_detail const auto evaluateDistantGroup = [&](std::size_t islandIndex) { const auto& island = plan.islands[islandIndex]; - double group = IslandDensity(recipe, island, + double group = IslandDensity(recipe, plan, island, static_cast(x), static_cast(y), static_cast(z)); for (const auto& edge : plan.clusterEdges) { diff --git a/Standalone/Core/src/VoxelForgeCore.cpp b/Standalone/Core/src/VoxelForgeCore.cpp index 58763cf..908825c 100644 --- a/Standalone/Core/src/VoxelForgeCore.cpp +++ b/Standalone/Core/src/VoxelForgeCore.cpp @@ -943,6 +943,10 @@ namespace VoxelForgeCore std::shared_ptr worldAssembly; WorldConfiguration configuration; std::unique_ptr shape; + std::unique_ptr + sharedHeightfieldColumns; + std::unique_ptr + sharedIslandColumns; ShapeBuildRequest buildRequest; mutable std::shared_mutex tileRegionMutex; std::unordered_map,TileKeyHash> tileRegions; @@ -953,6 +957,23 @@ namespace VoxelForgeCore std::uint64_t lastSequence=0; }; + struct FScopedSharedColumnCaches + { + using FHeightfieldCache=world_assembly_detail::FSharedHeightfieldColumnCache; + using FIslandCache=floating_island_detail::FSharedIslandColumnCache; + FHeightfieldCache* PreviousHeightfield=nullptr; + FIslandCache* PreviousIslands=nullptr; + + FScopedSharedColumnCaches(FHeightfieldCache* Heightfield,FIslandCache* Islands) + : PreviousHeightfield(FHeightfieldCache::setThreadActive(Heightfield)), + PreviousIslands(FIslandCache::setThreadActive(Islands)) {} + ~FScopedSharedColumnCaches() + { + FHeightfieldCache::setThreadActive(PreviousHeightfield); + FIslandCache::setThreadActive(PreviousIslands); + } + }; + World::World(ResolvedWorldSnapshot snapshot):impl_(std::make_unique()) { if(snapshot.data_) @@ -960,6 +981,15 @@ namespace VoxelForgeCore impl_->owner=snapshot.data_->owner; impl_->worldAssembly=snapshot.data_->worldAssembly; impl_->configuration=snapshot.data_->configuration; + if(impl_->worldAssembly) + { + if(!impl_->worldAssembly->plan.strates.empty()) + impl_->sharedHeightfieldColumns= + std::make_unique(); + if(!impl_->worldAssembly->floatingIslands.empty()) + impl_->sharedIslandColumns= + std::make_unique(); + } } } World::~World()=default; @@ -1337,6 +1367,9 @@ namespace VoxelForgeCore options.legacyDirectMappedColumnCache||key.lod<=2); if(!impl_||!impl_->owner||!impl_->shape||!impl_->shape->IsBuilt()) { error="world must be compiled before generating tiles";return std::nullopt; } + FScopedSharedColumnCaches sharedColumnCaches(options.useSharedColumnCache + ?impl_->sharedHeightfieldColumns.get():nullptr, + options.useSharedColumnCache?impl_->sharedIslandColumns.get():nullptr); const auto& world=*impl_->owner; std::shared_ptr page; if(impl_->tileRegionCount.load(std::memory_order_acquire)!=0) @@ -1500,6 +1533,15 @@ namespace VoxelForgeCore FStandaloneShapeWorld::ResetThreadClassificationStats(); VoxelForgeCore::Heightfield2D::resetThreadColumnCacheStats( options.captureColumnCoordinates); + if(options.captureColumnCoordinates) + { + constexpr std::int64_t SubvoxelsPerVoxel=WorldAssemblySubvoxelsPerVoxel; + VoxelForgeCore::Heightfield2D::setThreadColumnCaptureBounds( + static_cast(origin.X)*SubvoxelsPerVoxel, + static_cast(origin.Y)*SubvoxelsPerVoxel, + static_cast(origin.X+extent)*SubvoxelsPerVoxel, + static_cast(origin.Y+extent)*SubvoxelsPerVoxel); + } bool bNativeSurfaceTile=false; bool bNativeSurfaceBandSafe=false; double classificationColumnComputeMilliseconds=0.0; @@ -1544,6 +1586,8 @@ namespace VoxelForgeCore out.diagnostics.columnCacheHits=Columns.columnHits; out.diagnostics.columnCacheMisses=Columns.columnMisses; out.diagnostics.columnCacheConflictMisses=Columns.columnConflictMisses; + out.diagnostics.sharedColumnCacheHits=Columns.sharedColumnHits; + out.diagnostics.columnEvaluations=Columns.columnEvaluations; out.diagnostics.classificationMilliseconds=Counts.Milliseconds; out.diagnostics.classificationExclusiveMilliseconds=std::max(0.0, Counts.Milliseconds-classificationColumnComputeMilliseconds); @@ -1829,6 +1873,8 @@ namespace VoxelForgeCore out.diagnostics.columnCacheHits=Columns.columnHits; out.diagnostics.columnCacheMisses=Columns.columnMisses; out.diagnostics.columnCacheConflictMisses=Columns.columnConflictMisses; + out.diagnostics.sharedColumnCacheHits=Columns.sharedColumnHits; + out.diagnostics.columnEvaluations=Columns.columnEvaluations; out.diagnostics.columnComputeMilliseconds=Columns.columnComputeMilliseconds; out.diagnostics.columnEvaluationExclusiveMilliseconds=Columns.columnComputeMilliseconds; out.diagnostics.classificationMilliseconds=Counts.Milliseconds; diff --git a/Standalone/Core/src/WorldAssemblerInternal.h b/Standalone/Core/src/WorldAssemblerInternal.h index 5ce67d4..23175d0 100644 --- a/Standalone/Core/src/WorldAssemblerInternal.h +++ b/Standalone/Core/src/WorldAssemblerInternal.h @@ -5,6 +5,7 @@ #include "VoxelForgeCore/VoxelForgeCore.h" #include +#include #include #include #include @@ -121,6 +122,9 @@ WorldAssemblyFeatureColumnRangeV1 BuildWorldAssemblyFeatureColumnRangeV1( namespace world_assembly_detail { + using FSharedHeightfieldColumnCache = + VoxelForgeCoreShared::TSharedColumnValueCache; + // Preserve recipe order without allocating or looking up metadata in the point loop. // Garder l'ordre de recette sans allocation ni recherche de metadonnees par point. struct FeatureRange @@ -1375,6 +1379,9 @@ namespace world_assembly_detail std::array> HeightfieldColumnCacheNextWay; inline thread_local HeightfieldColumnCacheStats HeightfieldColumnStats{}; inline thread_local bool HeightfieldColumnCacheLegacyDirectMapped=false; + inline thread_local std::array HeightfieldColumnCaptureBounds{}; + inline thread_local bool HeightfieldColumnCaptureBoundsValid=false; + inline std::atomic HeightfieldColumnCaptureSequence{0}; inline std::size_t HeightfieldColumnCacheSet(const WorldAssemblyRecipeV1& recipe, const WorldAssemblyStratumPlanV1& plan,std::int64_t x,std::int64_t y) noexcept @@ -1587,22 +1594,46 @@ namespace world_assembly_detail replacement=&cache[first+replacementWay]; } auto& entry=*replacement; - const auto start=std::chrono::steady_clock::now(); - HeightfieldPointSample column=EvaluateColumn(recipe,plan,x,y); - HeightfieldColumnStats.columnComputeMilliseconds+=std::chrono::duration( - std::chrono::steady_clock::now()-start).count(); ++HeightfieldColumnStats.columnMisses; - if(HeightfieldColumnStats.captureCoordinates&&HeightfieldColumnStats.coordinateCaptureComplete) + const auto compute=[&]() { - try + const std::uint64_t sequence=HeightfieldColumnStats.captureCoordinates + ?HeightfieldColumnCaptureSequence.fetch_add(1,std::memory_order_relaxed):0; + const auto start=std::chrono::steady_clock::now(); + HeightfieldPointSample result=EvaluateColumn(recipe,plan,x,y); + HeightfieldColumnStats.columnComputeMilliseconds+=std::chrono::duration( + std::chrono::steady_clock::now()-start).count(); + ++HeightfieldColumnStats.columnEvaluations; + if(HeightfieldColumnStats.captureCoordinates + &&HeightfieldColumnStats.coordinateCaptureComplete) { - HeightfieldColumnStats.evaluatedCoordinates.push_back({x,y}); - } - catch(...) - { - HeightfieldColumnStats.coordinateCaptureComplete=false; + try + { + const bool outside=HeightfieldColumnCaptureBoundsValid + &&(xHeightfieldColumnCaptureBounds[2] + ||yHeightfieldColumnCaptureBounds[3]); + HeightfieldColumnStats.evaluatedCoordinates.push_back({x,y,sequence,outside}); + } + catch(...) + { + HeightfieldColumnStats.coordinateCaptureComplete=false; + } } + return result; + }; + HeightfieldPointSample column{}; + if(auto* shared=FSharedHeightfieldColumnCache::threadActive()) + { + const VoxelForgeCoreShared::FColumnCacheKey key{ + static_cast(reinterpret_cast(&recipe)), + recipe.contentHash, + static_cast(reinterpret_cast(&plan)), + plan.contentHash,x,y}; + bool computed=false; + column=shared->getOrCompute(key,compute,computed); + if(!computed)++HeightfieldColumnStats.sharedColumnHits; } + else column=compute(); entry={&recipe,&plan,recipe.contentHash,plan.contentHash,x,y,column}; return column; } @@ -2087,10 +2118,22 @@ void Heightfield2D::resetThreadColumnCacheStats(bool captureCoordinates, bool cl Stats.columnHits=0; Stats.columnMisses=0; Stats.columnConflictMisses=0; + Stats.sharedColumnHits=0; + Stats.columnEvaluations=0; Stats.columnComputeMilliseconds=0.0; Stats.captureCoordinates=captureCoordinates; Stats.coordinateCaptureComplete=true; Stats.evaluatedCoordinates.clear(); + world_assembly_detail::HeightfieldColumnCaptureBoundsValid=false; +} + +void Heightfield2D::setThreadColumnCaptureBounds(std::int64_t minimumXSubvoxels, + std::int64_t minimumYSubvoxels,std::int64_t maximumXSubvoxels, + std::int64_t maximumYSubvoxels) noexcept +{ + world_assembly_detail::HeightfieldColumnCaptureBounds={minimumXSubvoxels, + minimumYSubvoxels,maximumXSubvoxels,maximumYSubvoxels}; + world_assembly_detail::HeightfieldColumnCaptureBoundsValid=true; } void Heightfield2D::setThreadColumnCacheLegacyDirectMapped(bool enabled) noexcept diff --git a/Standalone/Research/WorldAssemblerM3/Linux7/REPORT.md b/Standalone/Research/WorldAssemblerM3/Linux7/REPORT.md new file mode 100644 index 0000000..5f5178f --- /dev/null +++ b/Standalone/Research/WorldAssemblerM3/Linux7/REPORT.md @@ -0,0 +1,171 @@ +# Linux7 — shared terrain columns + +**Decision: keep the shared second tier.** On the L1 view it cut median W12 whole-view time by +19.0%, well above the 3% threshold, and the 8 m camera crossing was also faster. All final output +hashes match the `9898e05` binary. + +Runs used the `linux6` base, Clang 19 with FP contraction off, GCC 14 defaults, CPUs 0–11, normal +priority, a 6 GiB address-space cap, and the unchanged `powersave` governor. Static timings below +are three alternating base/candidate W12 runs of 2,553 tiles. + +## 1. What was recomputed + +Three complete W12 captures per mode on the L1 view. Counts below are medians of actual column +formula executions after worker-local cache misses, grouped by LOD. “Same worker” means the worker +had computed that XY before and later missed its local entry. Halo/normal repeats are separated +before worker attribution; the parenthesized value is the total computed halo/normal probes, +including unique ones. Each field's median is independent, so row medians can differ slightly from +their sums. + +| LOD | Base total / unique XY | Same worker | Different worker | Halo repeats / probes | Shared total / unique XY | +|---:|---:|---:|---:|---:|---:| +| 0 | 556,134 / 128,240 | 22,307 | 358,908 | 46,679 / 70,972 | 135,720 / 127,370 | +| 1 | 452,626 / 104,284 | 16,393 | 287,791 | 39,245 / 59,082 | 85,658 / 85,479 | +| 2 | 144,251 / 32,274 | 14,667 | 74,377 | 22,933 / 30,942 | 28,019 / 28,019 | +| 3 | 91,158 / 33,256 | 0 | 46,387 | 11,564 / 20,339 | 16,213 / 16,213 | +| 4 | 272,776 / 91,325 | 100 | 167,083 | 14,266 / 28,089 | 90,105 / 90,065 | +| 5 | 178,511 / 71,957 | 1 | 98,687 | 7,892 / 20,042 | 70,932 / 70,930 | +| 6 | 342,763 / 280,334 | 20 | 56,204 | 6,205 / 19,977 | 279,266 / 279,202 | +| 7 | 79,400 / 71,468 | 1 | 4,951 | 2,980 / 14,329 | 70,492 / 70,492 | +| 8 | 137,062 / 106,219 | 0 | 22,138 | 8,704 / 22,609 | 106,219 / 106,219 | + +Across the per-LOD medians, the base made 2.255 million formula evaluations: 919 thousand unique +XYs and 1.336 million repeats. Of the repeats, 83.6% were from another worker, 4.0% were same-worker +evictions, and 12.0% were halo/normal probes. The shared candidate made 883 thousand evaluations, +60.9% fewer; about 8.6 thousand repeats remained. This is mostly a worker-sharing problem, not a +local-cache-capacity problem. + +## 2. Design + +`SharedColumnCache.h` is a bounded, four-way second-tier value cache used by both `Heightfield2D` +and FloatingIslands. Its key includes recipe and plan pointer identity plus content hash, exact XY, +and an optional exact sub-domain identity; FloatingIslands supplies the island stable ID because +different anchors can use the same XY. LOD is omitted because the column value is LOD-independent. +The heightfield table has 524,288 slots and the island table has 65,536. Both are allocated with +the `World::Impl` and destroyed with it. The Standalone streamer stops and joins workers before +world teardown; engine shutdown must keep the same drain-before-destroy ordering as the existing +generator state. + +The S14 `FLazy` thread-local caches stay first, so their hits remain worker-private. Shared hits read +versioned atomic entries without taking a lock; a miss takes a set-local spin lock, rechecks, then +computes and publishes. A hit racing with a slot replacement can fall through as a miss. Fixed +capacity bounds memory and replacement bounds reuse as the player moves. No page ring, prefetch, or +neighbor affinity was needed. The shared table is not an immutable page store; atomic fields and +version checks give readers a consistent value while a slot can be replaced. Workers only read +recipe/plan identity and content hashes; they do not mutate generator state or call UObject/engine +APIs. Epoch, shutdown, and MPSC result handling were not changed. The L3 FP-contract preamble and +self-test remain intact, as does the `FLazy` rule for large thread-local storage. + +## 3. Speed and memory + +| W12 static whole view | Base | Shared | Change | +|---|---:|---:|---:| +| `whole_view_ready` median | 2.165 s | 1.754 s | −19.0% | +| Peak RSS median (max) | 401.5 MiB (402.6) | 467.8 MiB (468.0) | +66.3 MiB median | + +Per-LOD worker sum is the sum of tile worker seconds, not wall time. Values are medians across the +same three static runs. + +| LOD | Base worker sum | Shared worker sum | Change | +|---:|---:|---:|---:| +| 0 | 3.830 s | 2.667 s | −30.4% | +| 1 | 5.301 s | 4.085 s | −22.9% | +| 2 | 1.296 s | 0.795 s | −38.7% | +| 3 | 1.116 s | 0.725 s | −35.1% | +| 4 | 2.692 s | 1.674 s | −37.8% | +| 5 | 1.911 s | 1.459 s | −23.7% | +| 6 | 2.903 s | 2.621 s | −9.7% | +| 7 | 1.242 s | 1.227 s | −1.2% | +| 8 | 2.318 s | 2.271 s | −2.0% | + +The moving trial warmed the first view, moved the camera 8 m, then generated 144 entering tiles +(128 LOD0, 16 LOD1) on the same world and worker pool. It ran three times per cache mode. + +| Moving check | Shared | Thread-local | Change | +|---|---:|---:|---:| +| New-tile wall median | 52.39 ms | 82.00 ms | −36.1% | +| LOD0 worker sum median | 403.44 ms | 693.67 ms | −41.8% | +| LOD1 worker sum median | 138.34 ms | 182.29 ms | −24.1% | +| Peak RSS median (max) | 466.1 MiB (466.9) | 466.1 MiB (467.6) | Same cache allocation in both modes | + +## 4. Hashes + +Every W1/W12 result matched the base for Clang FP contraction off and GCC default: full detail, +whole view, each LOD, and exact queries. The L1 base hashes are: + +| Scope | Density | Mesh | Payload | Topology | +|---|---|---|---|---| +| Full detail, 960 tiles | `0x0ee0579bd15cbbe4` | `0x32e5d4964d610fee` | `0x1932b96ee4b1d910` | `0x69743c6bad3447dc` | +| Whole view, 2,553 tiles | `0xc10697a8f48380c7` | `0xe75c9119a1498cd5` | `0x49a9a8d1835a6c8c` | `0xedcb23288b5e8724` | + +| LOD | Density | Mesh | Payload | Topology | +|---:|---|---|---|---| +| 0 | `0x769f2a706d585a72` | `0x33eeeb39fa443816` | `0x645621a1d968983f` | `0xc2457ed905036c9d` | +| 1 | `0x69cc5a5c0a2c60af` | `0xb48335f5410c57d5` | `0xd15c5480162ab8ba` | `0xfeca1bd025823604` | +| 2 | `0x85deed5683a77202` | `0x53469895987aeea2` | `0xe55f8ae7e1532033` | `0xd5feb12a0e85c09b` | +| 3 | `0x6518caefb3be7e7f` | `0xec513c90b853a614` | `0xf62fa1137ff17f82` | `0xccf1dcf681965816` | +| 4 | `0x62d4be96021992f4` | `0xf1e4c6efcc2615a9` | `0x9dc5f9c88be02f33` | `0xfe6f55125a07c3bf` | +| 5 | `0x5e4a8c38b8cea6fc` | `0x08ee56998aaf21e5` | `0x541eea8558f26cc8` | `0x6539374a8b3336d8` | +| 6 | `0xe036e95b9006945b` | `0x624a33f941a99b02` | `0x81b73e14f68bdf86` | `0x4de4c63ce1f80a56` | +| 7 | `0x8ee358af56a30d8d` | `0x46ab3c6184048b7c` | `0x0a6164b7e68e80c5` | `0x0c26b85bb03bc831` | +| 8 | `0xc3458e1cbdb9c471` | `0x46c03d0994d75192` | `0x8f89fc5bde12d230` | `0xd16dc0cf62031898` | + +Exact queries (9 probes): density and unclamped `0xe4d4a8eb95b35933`, detail +`0xefcba20c0c70c415`, collision `0xe604823a249029bf`. The FloatingIslands seed-0 fixture also +passes at Clang/GCC W1/W12 with recipe hash `0x39a24198ad8cf88f`; its whole-view hash matches the +base: density `0xa8fc5f8f4eaf77a3`, mesh `0xfff7d8ac8923eb9a`, payload `0xd730c51d959c1c59`, +topology `0xbb33ddb76cd386be`. + +## 5. Transfer estimate and Windows work + +The majority of the opportunity is cross-worker, so it does not disappear just because the 5800X3D +has a larger L3. Same-worker evictions are only 4% of baseline repeats; that capacity-driven part +may shrink. The candidate adds about 66 MiB peak RSS, and a 96 MiB L3 could keep much of the shared +table resident, though other game data will compete for it. I expect the win to transfer and clear +3%, likely landing in the mid-single to low-double digits; that percentage is an estimate until the +5800X3D run. + +Windows to-do: + +1. Build the Core route with MSVC `/fp:precise`; keep the FP-contract self-test and check the + 64-bit atomic slot path on the supported Windows target. +2. Run the W1/W12 whole-view, full-detail, per-LOD, exact-query, and seed-0 FloatingIslands checks + against the base binary. +3. Interleave three static W12 A/B runs on the 5800X3D, then repeat the 8 m camera crossing; record + peak RSS and the machine's unchanged governor. +4. Exercise real streaming shutdown/profile changes and confirm workers drain before world/cache + destruction, with Epoch checks and MPSC result queues still intact. + +## 6. Failed and corrected runs + +- The first Clang W1 hash invocation omitted the explicit data path; it exited 2 after resolving + `Standalone/bin/data/owner_world.json`. The retry passed with + `Standalone/data/owner_world.json` supplied. +- The first moving step was 32 m and failed the shape interval 160-million-node safety bound. The + benchmark was retried at 8 m, which completed three times in each cache mode. +- The first shared FloatingIslands cache key omitted the island stable ID. The W12 fixture exposed + an LOD4 hash mismatch. Adding the exact sub-domain ID fixed it; final hashes match at W1/W12 for + both compilers. +- An early fixture command passed `--seed=0`, which also overrode the surface-world seed. The final + fixture comparison omits that override; FloatingIslands itself still runs its seed-0 check. +- No run exceeded 30 minutes or remained stuck for 15 minutes. All final Clang and GCC builds + succeeded with 12 jobs. + +## Owner note + +Linux measured a 19.0% W12 whole-view gain and a 36.1% gain on the 8 m crossing, at about 66 MiB +more peak RSS. Please run the Windows hash matrix and static/moving A/B on the 5800X3D before using +the laptop percentage as an owner-machine estimate. + +## Orchestrator review (Claude) + +- **Concurrency read:** a seqlock over all-atomic fields (seq_cst), with the version checked before + and after the key/value loads; readers fall back to a miss on any mismatch. Writers hold the set + spin lock and mark the version odd while writing. There is no non-atomic shared access, so a + reader can't return a torn value. The per-tile scoped `setThreadActive`/restore binds each thread + to its own world's table. Default on (`useSharedColumnCache = true`). +- **Re-run by me** (W12, L1 view, interleaved): base `linux6` 2.014/2.059/2.092 s vs L7 `linux7shared` + 1.657/1.730/1.729 s, then 1.718/1.705/1.722 s (median 2.059 → 1.722 s, -16%). The whole-view hash was + identical in all 8 L7 runs at W12, plus W1 (8.036 s) and W8 (1.799 s). +- Luna's island-key bug was caught by the W12 hash fixture before review, so the determinism check + works as intended. diff --git a/Standalone/Source/StreamBenchmarkHeadless.cpp b/Standalone/Source/StreamBenchmarkHeadless.cpp index ad0ad16..59adbe7 100644 --- a/Standalone/Source/StreamBenchmarkHeadless.cpp +++ b/Standalone/Source/StreamBenchmarkHeadless.cpp @@ -35,6 +35,7 @@ namespace bool StreamPipeline=false; bool StreamHash=false; bool ProfileColumnOverlap=false; + bool UseSharedColumnCache=true; bool ForceGenerate=false; bool MeshCoverage=false; bool ScreenSpaceGate=false; @@ -47,8 +48,10 @@ namespace int TlsProbeWorkers=0; std::string TlsProbeTouch="none"; bool HasCamera=false; + bool HasCameraStep=false; bool AutoSurfaceCamera=false; FVector CameraMeters=FVector::ZeroVector; + FVector CameraStepMeters=FVector::ZeroVector; float CameraYawDegrees=0.0f; float CameraPitchDegrees=0.0f; std::filesystem::path MeshExportPath; @@ -77,6 +80,15 @@ namespace return true; } + bool ParseCameraStep(std::string_view text,FVector& meters) + { + const std::string values(text); + float x=0.0f,y=0.0f,z=0.0f; + if(std::sscanf(values.c_str(),"%f,%f,%f",&x,&y,&z)!=3)return false; + meters={x,y,z}; + return true; + } + bool ParseOptions(int argc,char** argv,FOptions& options) { const std::filesystem::path executable=std::filesystem::absolute(argv[0]); @@ -91,6 +103,8 @@ namespace std::printf("Usage: %s [owner_world.json] --world-assembler-prototype --stream-benchmark --stream-pipeline [options]\n" " --workers=N --clip-radius=N --max-clip-level=N --max-level=N\n" " --camera=X,Y,Z,YAW,PITCH (position in metres) --stream-hash\n" + " --camera-step=X,Y,Z (optional second view step in metres)\n" + " --column-cache=shared|thread-local (default: shared)\n" " --mesh-export=FILE (write the exact generated tile meshes for offline rendering)\n" " --profile-column-overlap --far-horizon-lod8-step=N --force-generate\n" " --seed=N --recipe=surface-world|floating-islands\n" @@ -149,6 +163,20 @@ namespace {std::fprintf(stderr,"Invalid --camera value: %.*s\n",static_cast(arg.size()),arg.data());return false;} options.HasCamera=true; } + else if(arg.starts_with("--camera-step=")) + { + if(!ParseCameraStep(arg.substr(std::string_view("--camera-step=").size()), + options.CameraStepMeters)) + {std::fprintf(stderr,"Invalid --camera-step value: %.*s\n",static_cast(arg.size()),arg.data());return false;} + options.HasCameraStep=true; + } + else if(arg.starts_with("--column-cache=")) + { + const std::string_view mode=arg.substr(std::string_view("--column-cache=").size()); + if(mode=="shared")options.UseSharedColumnCache=true; + else if(mode=="thread-local")options.UseSharedColumnCache=false; + else {std::fprintf(stderr,"Invalid --column-cache value: %.*s\n",static_cast(arg.size()),arg.data());return false;} + } else if(arg.starts_with("--mesh-export=")) { options.MeshExportPath=std::filesystem::path(arg.substr(14)); @@ -199,10 +227,12 @@ int main(int argc,char** argv) if(!ParseOptions(argc,argv,options))return 2; if(options.TlsProbeRequested) return StandaloneTlsProbe::Run(options.TlsProbeWorkers,options.TlsProbeTouch); - std::printf("[HeadlessConfig] data=%s workers=%d clip_radius=%d max_clip_level=%d max_level=%d hash=%s column_overlap=%s force_generate=%s recipe=%s seed_override=%s render_distance_override=%s\n", + std::printf("[HeadlessConfig] data=%s workers=%d clip_radius=%d max_clip_level=%d max_level=%d hash=%s column_overlap=%s force_generate=%s recipe=%s shared_column_cache=%s camera_step_m=(%.3f,%.3f,%.3f) seed_override=%s render_distance_override=%s\n", options.DataPath.string().c_str(),options.WorkerCount,options.ClipRadius,options.MaxClipLevel, options.MaxLevel,options.StreamHash?"on":"off",options.ProfileColumnOverlap?"on":"off", options.ForceGenerate?"on":"off",options.FloatingIslandsRecipe?"floating-islands":"surface-world", + options.UseSharedColumnCache?"on":"off",options.CameraStepMeters.X,options.CameraStepMeters.Y, + options.CameraStepMeters.Z, options.HasSeedOverride?"yes":"no",options.HasRenderDistanceOverride?"yes":"no"); std::printf("[ColdTimeline] stage=owner_data_load begin_ms=%.3f\n", std::chrono::duration(Clock::now()-coldStart).count()); @@ -309,10 +339,14 @@ int main(int argc,char** argv) {std::fprintf(stderr,"Shape interval build failed: %s\n",error.c_str());return 2;} FStandaloneCoreScheduler scheduler; + const FVector cameraStepVoxels=options.HasCameraStep + ?FVector(options.CameraStepMeters.X*4.0f,options.CameraStepMeters.Y*4.0f, + options.CameraStepMeters.Z*4.0f):FVector::ZeroVector; return StandaloneStreamBenchmark::RunPipelinedStreamingBenchmark(ownerWorld,snapshot, cameraVoxels,options.WorkerCount,options.MaxLevel,VoxelForgeCore::DetailVariant::coherent, scheduler,options.StreamHash,false,false,false,false,coldStart, options.ProfileColumnOverlap,options.FarHorizonLOD8InteriorStepVoxels, options.MeshCoverage,options.MeshExportPath,options.CameraYawDegrees, - options.CameraPitchDegrees,options.ScreenSpaceGate); + options.CameraPitchDegrees,options.ScreenSpaceGate,options.UseSharedColumnCache, + cameraStepVoxels); } diff --git a/Standalone/Source/StreamBenchmarkPipeline.h b/Standalone/Source/StreamBenchmarkPipeline.h index 8388ea8..0ab413e 100644 --- a/Standalone/Source/StreamBenchmarkPipeline.h +++ b/Standalone/Source/StreamBenchmarkPipeline.h @@ -137,7 +137,8 @@ namespace StandaloneStreamBenchmark std::chrono::steady_clock::time_point ColdStart,bool bProfileColumnOverlap, int32 FarHorizonLOD8InteriorStepVoxels,bool bReportMeshCoverage=false, const std::filesystem::path& MeshExportPath={},float CameraYawDegrees=0.0f, - float CameraPitchDegrees=0.0f,bool bScreenSpaceGate=false) + float CameraPitchDegrees=0.0f,bool bScreenSpaceGate=false, + bool bUseSharedColumnCache=true,FVector CameraStepVoxels=FVector::ZeroVector) { using Clock=std::chrono::steady_clock; FStandaloneOwnerWorld& World=*OwnerWorld; @@ -167,6 +168,21 @@ namespace StandaloneStreamBenchmark if(MaxLevel>=0) Desired.Tiles.erase(std::remove_if(Desired.Tiles.begin(),Desired.Tiles.end(), [&](const FVoxelTileKey& Tile){return Tile.Level>MaxLevel;}),Desired.Tiles.end()); + const bool bHasCameraStep=std::abs(CameraStepVoxels.X)+std::abs(CameraStepVoxels.Y) + +std::abs(CameraStepVoxels.Z)>1.0e-6f; + FStandaloneDesiredTileSet MovedDesired; + if(bHasCameraStep) + { + const FVector MovedCamera=CameraPositionVoxels+CameraStepVoxels; + MovedDesired=StandaloneWorldMesher::BuildDesiredSet(World,MovedCamera); + const auto MovedCoreTiles=Shape.desiredTiles({MovedCamera.X,MovedCamera.Y,MovedCamera.Z}); + MovedDesired.Tiles.clear();MovedDesired.Tiles.reserve(MovedCoreTiles.size()); + for(const auto& Key:MovedCoreTiles) + MovedDesired.Tiles.emplace_back(FIntVector(Key.coordinate.x,Key.coordinate.y,Key.coordinate.z),Key.lod); + if(MaxLevel>=0) + MovedDesired.Tiles.erase(std::remove_if(MovedDesired.Tiles.begin(),MovedDesired.Tiles.end(), + [&](const FVoxelTileKey& Tile){return Tile.Level>MaxLevel;}),MovedDesired.Tiles.end()); + } if(!Desired.Tiles.empty()) { FIntVector volumeMin(INT32_MAX,INT32_MAX,INT32_MAX); @@ -191,17 +207,22 @@ namespace StandaloneStreamBenchmark } FIntVector FineMin(INT32_MAX,INT32_MAX,INT32_MAX),FineMax(INT32_MIN,INT32_MIN,INT32_MIN); FIntVector ClipMin(INT32_MAX,INT32_MAX,INT32_MAX),ClipMax(INT32_MIN,INT32_MIN,INT32_MIN); - for(const FVoxelTileKey& Tile:Desired.Tiles) + const auto AccumulateBounds=[&](const std::vector& Tiles) { - const FIntVector Origin=Tile.OriginVoxels();const int32 Extent=Tile.ExtentVoxels(); - ClipMin.X=std::min(ClipMin.X,Origin.X);ClipMin.Y=std::min(ClipMin.Y,Origin.Y);ClipMin.Z=std::min(ClipMin.Z,Origin.Z); - ClipMax.X=std::max(ClipMax.X,Origin.X+Extent);ClipMax.Y=std::max(ClipMax.Y,Origin.Y+Extent);ClipMax.Z=std::max(ClipMax.Z,Origin.Z+Extent); - if(Tile.Level(ShapeReady-ShapeBuildStart).count(); FStandaloneStreamingGenerator Streamer(World,WorkerCount,bHashDensity,nullptr,MaxLevel, bSkipUvAndColors,bSkipMaterialGroups,bSkipUV,bSkipColors,&Shape,bProfileColumnOverlap, - FarHorizonLOD8InteriorStepVoxels); + FarHorizonLOD8InteriorStepVoxels,bUseSharedColumnCache); if(bScreenSpaceGate)Streamer.SetMesherStageProfilingEnabled(true); const auto StreamStart=Clock::now(); const FStandaloneStreamingUpdate Update=Streamer.StartWithDesired(Desired); @@ -287,13 +308,19 @@ namespace StandaloneStreamBenchmark return X^(Y+static_cast(0x9E3779B9u)+(X<<6)+(X>>2)); } }; - struct FFirstColumnUse + struct FColumnEvaluationEvent { + FColumnXY XY; FVoxelTileKey Tile; std::uint64_t WorkerHash=0; + std::uint64_t Sequence=0; + bool bOutsideTileFootprint=false; }; - std::array,FColumnXYHash>,9> ColumnUsesByLOD; - std::array ColumnDuplicateEventsByLOD{},NeighbourDuplicateEventsByLOD{}; + std::array,9> ColumnEventsByLOD; + std::array ColumnEvaluationEventsByLOD{},UniqueColumnCoordinatesByLOD{}; + std::array ColumnDuplicateEventsByLOD{},SameWorkerEvictedRepeatsByLOD{}; + std::array DifferentWorkerRepeatsByLOD{},HaloNormalRepeatsByLOD{}; + std::array HaloNormalEvaluationsByLOD{},NeighbourDuplicateEventsByLOD{}; std::array CrossWorkerNeighbourDuplicateEventsByLOD{}; std::array ColumnCaptureTilesByLOD{},ExpectedTilesByLOD{}; std::array ColumnCaptureCompleteByLOD{}; @@ -376,33 +403,12 @@ namespace StandaloneStreamBenchmark ++ColumnCaptureTilesByLOD[LOD]; ColumnCaptureCompleteByLOD[LOD]=ColumnCaptureCompleteByLOD[LOD] &&Tile.CoreDiagnostics.columnCoordinateCaptureComplete; - std::vector> Coordinates= - Tile.CoreDiagnostics.evaluatedColumnCoordinates; - std::sort(Coordinates.begin(),Coordinates.end()); - Coordinates.erase(std::unique(Coordinates.begin(),Coordinates.end()),Coordinates.end()); - for(const auto& Coordinate:Coordinates) + auto& Events=ColumnEventsByLOD[LOD]; + for(const auto& Coordinate:Tile.CoreDiagnostics.evaluatedColumnCoordinates) { - const FColumnXY XY{Coordinate[0],Coordinate[1]}; - auto [Uses,Inserted]=ColumnUsesByLOD[LOD].try_emplace(XY); - std::vector& PriorUses=Uses->second; - if(Inserted) - { - PriorUses.push_back({Tile.Key,Tile.CoreDiagnostics.columnEvaluationWorkerHash}); - continue; - } - ++ColumnDuplicateEventsByLOD[LOD]; - bool bHasNeighbourDuplicate=false,bHasCrossWorkerNeighbourDuplicate=false; - for(const FFirstColumnUse& Prior:PriorUses) - { - if(!AreSameLODNeighbours(Prior.Tile,Tile.Key))continue; - bHasNeighbourDuplicate=true; - bHasCrossWorkerNeighbourDuplicate=bHasCrossWorkerNeighbourDuplicate - ||Prior.WorkerHash!=Tile.CoreDiagnostics.columnEvaluationWorkerHash; - } - if(bHasNeighbourDuplicate)++NeighbourDuplicateEventsByLOD[LOD]; - if(bHasCrossWorkerNeighbourDuplicate) - ++CrossWorkerNeighbourDuplicateEventsByLOD[LOD]; - PriorUses.push_back({Tile.Key,Tile.CoreDiagnostics.columnEvaluationWorkerHash}); + Events.push_back({{Coordinate.x,Coordinate.y},Tile.Key, + Tile.CoreDiagnostics.columnEvaluationWorkerHash,Coordinate.sequence, + Coordinate.outsideTileFootprint}); } } WholeViewReady=std::max(WholeViewReady,Tile.CompletedAt); @@ -449,6 +455,45 @@ namespace StandaloneStreamBenchmark bFullDetailReported=true; } } + if(bHasCameraStep) + { + const auto MovingStart=Clock::now(); + const auto MovingUpdate=Streamer.UpdateDesiredWithSet(MovedDesired,Desired.Tiles,1); + std::array MovingTilesByLOD{}; + std::array MovingWorkerSecondsByLOD{}; + int32 MovingFinished=0; + while(MovingFinished Ready=Streamer.TakeReady(); + if(Ready.empty()) + {std::this_thread::sleep_for(std::chrono::milliseconds(1));continue;} + for(const FStandaloneCpuTile& Tile:Ready) + { + ++MovingFinished; + if(Tile.Key.Level>=0&&Tile.Key.Level(MovingTilesByLOD.size())) + { + const std::size_t LOD=static_cast(Tile.Key.Level); + ++MovingTilesByLOD[LOD]; + MovingWorkerSecondsByLOD[LOD]+=Tile.WorkerSeconds; + } + } + } + const double MovingWallSeconds=std::chrono::duration(Clock::now()-MovingStart).count(); + const double StepDistanceMeters=std::sqrt( + static_cast(CameraStepVoxels.X)*CameraStepVoxels.X + +static_cast(CameraStepVoxels.Y)*CameraStepVoxels.Y + +static_cast(CameraStepVoxels.Z)*CameraStepVoxels.Z) + *static_cast(World.CoreVoxelSizeMicrons)/1000000.0; + std::printf("[MovingCameraStep] step_voxels=(%.3f,%.3f,%.3f) distance_m=%.3f desired_tiles=%zu new_tiles=%d completed=%d wall_s=%.6f shared_column_cache=%s\n", + CameraStepVoxels.X,CameraStepVoxels.Y,CameraStepVoxels.Z,StepDistanceMeters, + MovedDesired.Tiles.size(),MovingUpdate.NewJobs,MovingFinished,MovingWallSeconds, + bUseSharedColumnCache?"on":"off"); + for(std::size_t LOD=0;LOD(MovingTilesByLOD[LOD]), + MovingWorkerSecondsByLOD[LOD]); + } Streamer.Stop(); bool bScreenSpaceGatePassing=true; if(bScreenSpaceGate) @@ -472,6 +517,58 @@ namespace StandaloneStreamBenchmark bSceneExportPassing?"PASS":"FAIL",MeshExportPath.string().c_str()); bSceneExportPassing=bSceneExportPassing&&!FileSizeError; } + if(bProfileColumnOverlap) + { + struct FColumnPriorUse + { + FVoxelTileKey Tile; + std::uint64_t WorkerHash=0; + }; + for(std::size_t LOD=0;LOD,FColumnXYHash> UsesByXY; + UsesByXY.reserve(Events.size()); + for(const FColumnEvaluationEvent& Event:Events) + { + ++ColumnEvaluationEventsByLOD[LOD]; + if(Event.bOutsideTileFootprint)++HaloNormalEvaluationsByLOD[LOD]; + auto [Found,Inserted]=UsesByXY.try_emplace(Event.XY); + auto& PriorUses=Found->second; + if(Inserted) + { + ++UniqueColumnCoordinatesByLOD[LOD]; + PriorUses.push_back({Event.Tile,Event.WorkerHash}); + continue; + } + ++ColumnDuplicateEventsByLOD[LOD]; + if(Event.bOutsideTileFootprint) + ++HaloNormalRepeatsByLOD[LOD]; + else + { + const bool bSameWorkerSeen=std::any_of(PriorUses.begin(),PriorUses.end(), + [&](const FColumnPriorUse& Prior) + {return Prior.WorkerHash==Event.WorkerHash;}); + if(bSameWorkerSeen)++SameWorkerEvictedRepeatsByLOD[LOD]; + else ++DifferentWorkerRepeatsByLOD[LOD]; + } + bool bHasNeighbourDuplicate=false,bHasCrossWorkerNeighbourDuplicate=false; + for(const FColumnPriorUse& Prior:PriorUses) + { + if(!AreSameLODNeighbours(Prior.Tile,Event.Tile))continue; + bHasNeighbourDuplicate=true; + bHasCrossWorkerNeighbourDuplicate=bHasCrossWorkerNeighbourDuplicate + ||Prior.WorkerHash!=Event.WorkerHash; + } + if(bHasNeighbourDuplicate)++NeighbourDuplicateEventsByLOD[LOD]; + if(bHasCrossWorkerNeighbourDuplicate) + ++CrossWorkerNeighbourDuplicateEventsByLOD[LOD]; + PriorUses.push_back({Event.Tile,Event.WorkerHash}); + } + } + } bool bColumnOverlapCapturePassing=true; if(bProfileColumnOverlap) { @@ -480,11 +577,16 @@ namespace StandaloneStreamBenchmark const bool bLODComplete=ColumnCaptureCompleteByLOD[LOD] &&ColumnCaptureTilesByLOD[LOD]==ExpectedTilesByLOD[LOD]; bColumnOverlapCapturePassing=bColumnOverlapCapturePassing&&bLODComplete; - std::printf("[WorldAssemblerLODColumnOverlap] lod=%zu tiles=%llu expected_tiles=%llu unique_xy_columns=%zu repeated_tile_evaluations=%llu neighbouring_duplicate_evaluations=%llu cross_worker_neighbouring_duplicates=%llu capture_complete=%d\n", + std::printf("[WorldAssemblerLODColumnOverlap] lod=%zu tiles=%llu expected_tiles=%llu total_column_evaluations=%llu unique_xy_columns=%llu repeated_column_evaluations=%llu same_worker_evicted_repeats=%llu different_worker_repeats=%llu halo_normal_repeats=%llu halo_normal_evaluations=%llu neighbouring_duplicate_evaluations=%llu cross_worker_neighbouring_duplicates=%llu capture_complete=%d\n", LOD,static_cast(ColumnCaptureTilesByLOD[LOD]), static_cast(ExpectedTilesByLOD[LOD]), - ColumnUsesByLOD[LOD].size(), + static_cast(ColumnEvaluationEventsByLOD[LOD]), + static_cast(UniqueColumnCoordinatesByLOD[LOD]), static_cast(ColumnDuplicateEventsByLOD[LOD]), + static_cast(SameWorkerEvictedRepeatsByLOD[LOD]), + static_cast(DifferentWorkerRepeatsByLOD[LOD]), + static_cast(HaloNormalRepeatsByLOD[LOD]), + static_cast(HaloNormalEvaluationsByLOD[LOD]), static_cast(NeighbourDuplicateEventsByLOD[LOD]), static_cast(CrossWorkerNeighbourDuplicateEventsByLOD[LOD]), bLODComplete?1:0); diff --git a/Standalone/Source/StreamingWorld.h b/Standalone/Source/StreamingWorld.h index 0391016..8059982 100644 --- a/Standalone/Source/StreamingWorld.h +++ b/Standalone/Source/StreamingWorld.h @@ -428,12 +428,14 @@ public: bool bInSkipColors=false, VoxelForgeCore::World* InCoreWorld=nullptr, bool bInCaptureColumnCoordinates=false, - int32 InLod8HorizonInteriorStepVoxels=0) + int32 InLod8HorizonInteriorStepVoxels=0, + bool bInSharedColumnCache=true) :World(&InWorld),WorkerCount(FMath::Clamp(InWorkerCount,1,16)),bHashDensity(bInHashDensity), MeshCache(InMeshCache),MaxLevelFilter(InMaxLevelFilter), CoreWorld(InCoreWorld),bSkipUvAndColors(bInSkipUvAndColors),bSkipMaterialGroups(bInSkipMaterialGroups), bSkipUV(bInSkipUV),bSkipColors(bInSkipColors), bCaptureColumnCoordinates(bInCaptureColumnCoordinates), + bSharedColumnCache(bInSharedColumnCache), Lod8HorizonInteriorStepVoxels(std::max(0,InLod8HorizonInteriorStepVoxels)) { VoxelGenLOD::InitializeConsoleSwitches(); @@ -816,6 +818,7 @@ private: Options.skipUvAndColors=bSkipUvAndColors; Options.skipMaterialGroups=bSkipMaterialGroups;Options.skipUv=bSkipUV;Options.skipColors=bSkipColors; Options.captureColumnCoordinates=bCaptureColumnCoordinates; + Options.useSharedColumnCache=bSharedColumnCache; Options.lod8HorizonInteriorStepVoxels=Lod8HorizonInteriorStepVoxels; Options.profileMesherStages=bProfileMesherStages&&!ForceGenerate; Options.diagnosticGlobalOctaveDrop=DiagnosticGlobalOctaveDrop; @@ -993,6 +996,7 @@ private: bool bSkipUV=false; bool bSkipColors=false; bool bCaptureColumnCoordinates=false; + bool bSharedColumnCache=true; int32 Lod8HorizonInteriorStepVoxels=0; bool bProfileMesherStages=false; std::atomic CacheHitCount{0};