chore(repo): archive historical docs under Docs/archive, drop tracked pycache
Move closed audits, task briefs, design rounds, perf reports, the OPSTACK plan/progress logs and WORK-NEXT.md into Docs/archive/ (git mv, history kept). Whitelist Docs/ in the strict .gitignore, ignore Tools/**/__pycache__/, and remove the tracked .pyc. Repoint references in the kept docs and in non-test source comments to the new paths.
This commit is contained in:
34 files changed
+20
-15
No files matched your search
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,195 @@
|
||||
> ## ⚠️ REVIEWER'S NOTE — Claude, 2026-08-16. Read before acting on anything below.
|
||||
>
|
||||
> This report was produced by a **read-only Codex pass (`gpt-5.6-sol`, high effort)**. It is a
|
||||
> *lead list*, not a verified defect list. Every item is labelled "Verified by reading" **by its own
|
||||
> author**; that label is the author's claim, not an independent check.
|
||||
>
|
||||
> **What I checked myself, and what came of it:**
|
||||
>
|
||||
> | finding | my verdict |
|
||||
> |---|---|
|
||||
> | **VF-05** (radius envelope in `BuildChunkCache`) | ✅ **CONFIRMED and FIXED** — `CODEX-TASK-006`. Real, and the worst of three instances of this class: it is `TunnelNetwork`, it is in code **both** density paths share, and it breaks window invariance (`ARCHITECTURE §8.4`). Genuinely good find. |
|
||||
> | **VF-02** (3-second shutdown timeout) | ✅ **premise confirmed** — `VoxelWorld.cpp:327` literally reads *"Timeout after 3 seconds to avoid hanging the editor."* Note `CLAUDE.md` states the invariant more strongly than the code implements it ("EndPlay blocks on `ActiveTaskCount → 0`" — it blocks *with a deadline*). Worth deciding deliberately. |
|
||||
> | **VF-03** (TLS caches omit the owning world) | ⛔ **MY EARLIER VERDICT HERE WAS WRONG — corrected 2026-08-16.** I wrote that its fixture citation was fabricated. **It is not.** `VoxelForgeTestFixture.h` lines ~134/146 explicitly document `CP_UseOpStack` contamination between worlds; I had read only the file's 30-line header comment and asserted a negative from a partial read. VF-03's core claim is **CONFIRMED**: `GetDensityAt` keys its `thread_local CP_*` state by `(ChunkCoord, LayoutVersion)` with **no generator/world identity**, and every manager's version starts at the same value — so a second world on the same worker can inherit the first's params, `CP_UseOpStack` and stack. The *breadth* of VF-03 (the `OC_*`/`BM_*`/passage/biome/diff caches) is still unproven and should be audited as one owner-identity task. Original note kept below for the record: ~~substance plausible, evidence overstated~~ |
|
||||
> | ~~VF-03 (superseded)~~ | ~~**substance plausible, EVIDENCE OVERSTATED.**~~ It claims *"the test fixture explicitly documents observed cross-world contamination."* It does not. `VoxelForgeTestFixture.h` documents that the `thread_local` caches exist and flags an unrelated `TSoftObjectPtr` risk. The underlying point (caches keyed on chunk/seed/layout but not on which generator owns them) may still hold — but it needs checking on its own merits, not on this citation. |
|
||||
> | **VF-01** (live rebuild races streaming workers) | ✅ **CONFIRMED — the most serious finding here.** `UVoxelStrateManager::Initialize` does `StrateLayout.Empty()` (:~36) **and** `Passages.Empty()` (:171) then `Passages.Add()`, i.e. it frees and reallocates both arrays. There is **no lock, no barrier, no drain** anywhere in that file. Worker-side readers of the same arrays: `AnyPassageNearBox` (:460, range-for over `Passages`), `EvaluateModifierSDF` (indexes `Passages[...]`), `FindSlotIndexForChunkZ` (iterates `StrateLayout`) — all reached from `GetDensityAt`/`ClassifyTile` on mesher workers. And `RegenerateAllChunks()` (which bumps the epoch) runs **after** `Initialize`, so previous-epoch workers are still live during the mutation. **This is the same class already fixed once in this codebase** — `DiffLayer.ChunkMods` got `ModsLock` after a carve-vs-stream access violation. Four call sites, incl. `OnObjectModifiedInEditor` (:309), which fires automatically when a strate asset is edited while the world streams. **NOT fixed — see the note below.** |
|
||||
> | **VF-10** (~74-field per-voxel params copy) | ✅ **confirmed real, but Sol missed the conclusion that matters.** The 74 fields are real and the copy is per near-surface sample. **However it is INHERITED from the original path — `GetDensityWithParams` does the same copy — so both paths pay it equally and it does NOT explain the op-stack perf regression.** The op stack actually *improved* it (memoised so eleven detail ops don't each repeat it), and the site says so in its own comment. Genuine future optimisation for both paths; **not** the answer to "why is the op path slower". |
|
||||
> | VF-04, VF-06, VF-07, VF-08, VF-09 | **NOT independently verified.** Read them as leads. |
|
||||
>
|
||||
> **Do not treat an unverified row as actionable.** The lesson this project keeps paying for is that a
|
||||
> confident chain resting on an unchecked premise reverses about half the time — and VF-03 is an
|
||||
> instance of exactly that, inside an audit written to find them.
|
||||
|
||||
# VoxelForge code quality and efficiency audit — August 2026
|
||||
|
||||
| Finding | File | Severity | Tier | Evidence status |
|
||||
|---|---|---:|---|---|
|
||||
| VF-01 — Live rebuilds mutate generation state while workers read it | `VoxelWorld.cpp`, `VoxelStrateManager.cpp` | Critical | Async lifecycle / per-tile workers | Verified by reading |
|
||||
| VF-02 — Shutdown timeouts allow tasks to outlive their owners | `VoxelWorld.cpp`, `VoxelContentManager.cpp` | Critical | Async lifecycle | Verified by reading |
|
||||
| VF-03 — Function-static TLS caches omit the owning world/generator | `VoxelGenerator.cpp`, `VoxelStrateManager.cpp`, `VoxelDensityOpStack.cpp` | Critical | Per-voxel caches; per-chunk refill | Verified by reading |
|
||||
| VF-04 — Box/capsule edits bypass the intended budget and use the wrong live-deco removal volume | `VoxelDiffLayer.cpp`, `VoxelDiffLayer.h`, `VoxelWorld.cpp` | High | Per modification | Verified by reading |
|
||||
| VF-05 — Cave collection bounds can be smaller than generated geometry when min/max fields are reversed | `VoxelCaveMorphology.cpp`, `VoxelStrateTypes.h` | High | Per-chunk cache construction / skip bound | Verified by reading |
|
||||
| VF-06 — A fixed-only strate configuration silently disables the strate system | `VoxelWorld.cpp`, `VoxelStrateManager.cpp` | High | Initialization | Verified by reading |
|
||||
| VF-07 — World origin is used as the “no player” sentinel | `VoxelWorld.cpp`, `VoxelWorld.h` | Medium | Per frame / streaming gate | Verified by reading |
|
||||
| VF-08 — Decoration palettes are rebuilt every tick and deep-copied into every cell task | `VoxelContentManager.cpp`, `VoxelContentManager.h`, `VoxelStrateTypes.h` | Medium | Per frame and per decoration cell | Verified by reading |
|
||||
| VF-09 — Clearing decoration builds forgets still-running tasks and defeats the concurrency cap | `VoxelContentManager.cpp` | Medium | Per rebuild / async scheduling | Verified by reading |
|
||||
| VF-10 — Per-room terrain params are reconstructed for every near-surface sample | `VoxelGenerator.cpp`, `VoxelDensityOpStack.cpp`, `VoxelCaveMorphology.cpp` | Medium | Per near-surface voxel | Verified by reading |
|
||||
|
||||
## Scope and evidence
|
||||
|
||||
This was a static, read-only review. I read `CODEMAP.md`, `ARCHITECTURE.md` including §8.10, `REVIEW_FINDINGS.md`, the relevant public contracts, implementations, and tests. I did not build, compile, or run the plugin. Every item below is therefore marked **Verified by reading**: the cited control flow or cache-key omission is present in the source. Runtime frequency and timing impact are reasoned from that source, not measured in this review. No “suspicious only” item is included.
|
||||
|
||||
The deliberate old/new density-path duplication and every settled decision listed in the review request are excluded.
|
||||
|
||||
## Findings
|
||||
|
||||
### VF-01 — Live rebuilds mutate generation state while workers read it
|
||||
|
||||
**Evidence status:** Verified by reading.
|
||||
|
||||
**Location:** `Source/VoxelForge/Private/VoxelWorld.cpp` — `AVoxelWorld::RebuildStrates` (lines 140–153), `AVoxelWorld::OnObjectModifiedInEditor` (238–316), `AVoxelWorld::ChangeSeed` (2064–2114), `AVoxelWorld::LoadTile` (1345–1478), and `AVoxelWorld::GenerateTileResult` (1481 onward); `Source/VoxelForge/Private/VoxelStrateManager.cpp` — `UVoxelStrateManager::Initialize` (28–163).
|
||||
|
||||
**What is wrong:** chunk tasks capture `this` and call `GenerateTileResult`, which reads `Generator`, `Mesher`, and through them `StrateManager`. Meanwhile, each live-rebuild path mutates the same objects on the game thread. `Initialize` empties and repopulates `StrateLayout`, empties and repopulates `Passages`, and changes cached seed/settings fields. `ChangeSeed` also writes the generator's plain `Seed`/`OriginSpineRadius`. There is no lock, immutable snapshot, or worker quiescence around those writes.
|
||||
|
||||
The order makes the race especially direct: `RebuildStrates` and `OnObjectModifiedInEditor` call `StrateManager->Initialize(...)` before `RegenerateAllChunks()` increments `GenerationEpoch`. `ChangeSeed` also changes generator and manager state before regeneration. The epoch only rejects a finished result; it does not make concurrent reads of reallocating `TArray`s safe and cannot repair undefined behavior that happened while producing the result. Decoration and density-volume workers also read the generator and need to be included in the same transition.
|
||||
|
||||
**Why it matters:** an edit or seed change during active streaming can race a worker iterating or indexing storage that `Initialize` has freed/reallocated. Outcomes range from a tile built from mixed old/new settings to an access violation. This is a correctness and lifetime issue, not merely stale-result work.
|
||||
|
||||
**Concrete change:** introduce an immutable generation snapshot containing the seed, layout, passages, resolved definitions/op data, and a unique generation ID. Atomically publish the new snapshot and have every task capture a strong reference to one snapshot. The smaller alternative is a rebuild barrier: stop new chunk/deco/density work, wait without timeout for all generator readers, mutate the state, bump the epoch, then resume. Incrementing the epoch before mutation is useful but is not sufficient without snapshotting or quiescence.
|
||||
|
||||
### VF-02 — Shutdown timeouts allow tasks to outlive their owners
|
||||
|
||||
**Evidence status:** Verified by reading.
|
||||
|
||||
**Location:** `Source/VoxelForge/Private/VoxelWorld.cpp` — `AVoxelWorld::EndPlay` (320–375) and the `[this, ...]` task in `AVoxelWorld::LoadTile` (1456–1478); `Source/VoxelForge/Private/VoxelContentManager.cpp` — `UVoxelContentManager::BeginDestroy` (58–63), `NotifyShutdown` (65–80), and the `[this, ...]` task in `LaunchDecoTasks` (384–404).
|
||||
|
||||
**What is wrong:** both shutdown drains stop waiting after three seconds and continue teardown while work may still be running. Chunk tasks retain a raw `this` and even their RAII guard holds a reference to `AVoxelWorld::ActiveTaskCount`. Decoration tasks retain a raw content-manager `this`, read its `bShuttingDown`, and may access its queue. `BeginDestroy` sets a flag but does not establish task completion before calling `Super::BeginDestroy`.
|
||||
|
||||
The decoration counter is also file-global (`GActiveDecoTasks`), so it is neither an ownership handle nor a per-manager proof that this manager's tasks are finished.
|
||||
|
||||
**Why it matters:** if the timeout is reached, later task reads, queue writes, or the guard decrement can target an object whose EndPlay/destruction has advanced. The timeout converts a slow task into a possible use-after-free. The shutdown flag reduces ordinary latency but does not cancel a task already inside generation or marching.
|
||||
|
||||
**Concrete change:** retain `UE::Tasks::FTask` handles per owner and make UObject destruction contingent on their completion. Stop submissions first, request cancellation, and either wait unconditionally in a safe shutdown phase or defer final destruction through `IsReadyForFinishDestroy` until the owner's task group is empty. Replace the global decoration count with per-instance task ownership. A watchdog may log a long wait, but it must not release the objects that unfinished tasks can still touch.
|
||||
|
||||
### VF-03 — Function-static TLS caches omit the owning world/generator
|
||||
|
||||
**Evidence status:** Verified by reading. The test fixture explicitly documents observed cross-world contamination.
|
||||
|
||||
**Location:**
|
||||
|
||||
- `Source/VoxelForge/Private/VoxelGenerator.cpp` — `UVoxelGenerator::GetDensityAt`: `CP_*` cache (583–622), `GSurfColCache` access (738), and `DiffSlots` (808–826); `ClassifyTile`: `TC_BiomeCache`/`TC_SeenVersion` (2720–2731); `GetBiomeMaterialAt`: `BM_*` cache (3470–3485).
|
||||
- `Source/VoxelForge/Private/VoxelStrateManager.cpp` — `GeneratePassages` (169–173, 352–353) and `EvaluateModifierSDF`: `SL_*` shortlist plus unchecked `Passages[PIdx]` (372–420).
|
||||
- `Source/VoxelForge/Private/VoxelDensityOpStack.cpp` — `FRoomGraphSource::Eval`: `SI_*` strate-index memo (2172–2188).
|
||||
- `Source/VoxelForge/Private/Tests/VoxelForgeTestFixture.h` — `FTestWorld` construction (126–152).
|
||||
|
||||
**What is wrong:** these are function/file-static `thread_local` caches, so one worker-thread cache is shared by every VoxelForge instance evaluated on that thread. Their keys use coordinates and per-instance counters such as `LayoutVersion` or `ModsVersion`, but omit the owning generator/manager/diff layer. Two freshly initialized worlds normally both report layout version 1; two diff layers also start with the same modification version. Equal coordinates and versions therefore make the second world reuse the first world's params, operator stack (including its manager pointer), biome context, surface columns, or modification snapshot.
|
||||
|
||||
This is not hypothetical test hygiene. `VoxelForgeTestFixture.h` states that two test worlds both reporting version 1 caused the second world to receive the first world's params and `CP_UseOpStack`; it works around the problem by repeatedly initializing each test manager until its version is process-unique. Production has no such workaround.
|
||||
|
||||
The passage cache has a more severe failure mode. `SL_Nearby` stores indices from manager A, then manager B with the same `(chunk, PassagesVersion)` can execute `Passages[PIdx]` without `IsValidIndex`. Also, `GeneratePassages` empties `Passages` and returns for an empty layout before incrementing `PassagesVersion`, so the same manager can retain stale indices after an empty rebuild.
|
||||
|
||||
**Why it matters:** multiple VoxelWorld actors, PIE worlds, tests, previews, or address-reused objects can produce density/materials/modifications from the wrong world. The passage case can read out of bounds. This affects the per-voxel tier—up to roughly 35³ = 42,875 base samples per full-resolution tile—although the bad selection occurs at cache-refill granularity.
|
||||
|
||||
**Concrete change:** give each immutable generation context a process-unique, monotonic cache ID and include it in every shared TLS key. Give each diff layer its own unique ID as well. Prefer a per-worker cache object scoped to that context over scattered function statics. Move the passage-version increment so every clear/rebuild, including the empty-layout exit, invalidates the cache; retain `Passages.IsValidIndex(PIdx)` as defense in depth. Remove the test fixture's serial-bump workaround once production keys express owner identity.
|
||||
|
||||
### VF-04 — Box/capsule edits bypass the intended budget and use the wrong live-deco removal volume
|
||||
|
||||
**Evidence status:** Verified by reading.
|
||||
|
||||
**Location:** `Source/VoxelForge/Private/VoxelDiffLayer.cpp` — `UVoxelDiffLayer::CanModify` (20–45) and `ApplyModification` (63–133); `Source/VoxelForge/Public/VoxelDiffLayer.h` — `FVoxelModification::GetWorldBounds` (97–121); `Source/VoxelForge/Private/VoxelWorld.cpp` — `AVoxelWorld::ApplyModification` (1839–1871), `CarveBox`/`FillBox` (1874–1893), and `CarveCapsule`/`FillCapsule` (1896–1915).
|
||||
|
||||
**What is wrong:** budget validation knows only a scalar radius and always charges `4/3*pi*r^3`. `ApplyModification` clamps `Mod.Radius`, but leaves `BoxExtent`, `CapsuleEnd`, and `Falloff` unchanged. The shape-aware AABB then uses those unchanged values. Consequently:
|
||||
|
||||
- a box with huge extents is stored across its full huge AABB even though only its proxy radius was clamped and sphere volume was charged;
|
||||
- a capsule of arbitrary length is charged only as a sphere of its tube radius;
|
||||
- an untrusted or accidental large shape can enumerate and allocate entries for an enormous number of chunks despite `MaxBrushRadius`/`MaxTotalVolume` being presented as safety limits.
|
||||
|
||||
The live decoration cleanup is inconsistent in the other direction. It always removes a sphere centered at `Modification.Center` with the original `Modification.Radius`. For a capsule this is only endpoint A, leaving decorations floating along most of the segment. For a box, `max(half extent)` does not cover the corners and ignores falloff. It also does not use the clamped modification that was actually stored.
|
||||
|
||||
**Why it matters:** the budget can be bypassed precisely by the shapes most able to create a large remesh/storage burst. Separately, box/capsule edits leave visibly invalid live content until a later decoration rebuild.
|
||||
|
||||
**Concrete change:** make validation and accounting accept the complete `FVoxelModification`. Validate finite, non-negative geometry; enforce extent/tube-radius and capsule-length limits; and charge a documented shape volume (or a deliberately conservative support-AABB volume including falloff). Return the normalized/applied modification or its actual bounds from `ApplyModification`. Use those applied bounds for decoration invalidation—prefer a shape-aware removal query, or at least a conservative sphere centered on the bounds center with the bounds half-diagonal. Keep `CanModify` and `ApplyModification` on the same normalization/accounting function so UI/server decisions cannot drift.
|
||||
|
||||
### VF-05 — Cave collection bounds can be smaller than generated geometry when min/max fields are reversed
|
||||
|
||||
**Evidence status:** Verified by reading.
|
||||
|
||||
**Location:** `Source/VoxelForge/Private/VoxelCaveMorphology.cpp` — `VoxelCaveMorphology::BuildChunkCache`: `MaxInfluence` (127–130), `CollectMargin` (152), `RoomZBuffer` (171), room radius generation (260–265), and tunnel radius generation (464–468); `EvaluateSDF` margin (871–874). Authoring fields are in `Source/VoxelForge/Public/VoxelStrateTypes.h` — `MinRoomRadius`/`MaxRoomRadius` (490/498) and `TunnelMinRadius`/`TunnelMaxRadius` (595/602).
|
||||
|
||||
**What is wrong:** `FMath::Lerp(Min, Max, t)` produces values up to `max(Min, Max)` even when an asset has the endpoints reversed. The collection/influence math assumes the field named `Max*` is numerically largest: it uses only `MaxRoomRadius` and `TunnelMaxRadius`. The properties have no cross-field validation enforcing `Min <= Max`.
|
||||
|
||||
If `MinRoomRadius > MaxRoomRadius`, actual generated rooms can be larger than `MaxInfluence`, `CollectMargin`, and `RoomZBuffer` assume. If `TunnelMinRadius > TunnelMaxRadius`, the same applies to tunnel reach. `RoomReachesSearchBox` uses the actual radius, but it cannot test a room whose anchor cell was never collected because the collect region was too small. The convenience wrapper repeats the underestimated margin.
|
||||
|
||||
**Why it matters:** this is an under-bound, not a conservative overestimate. A room/tunnel able to affect a chunk may not be created in that chunk's cache, producing window-dependent density, seams, missing mesh, or missing collision. Reversed ranges are authorable and can also arise transiently while live-editing the two fields.
|
||||
|
||||
**Concrete change:** derive bound-only envelopes as `Max(MinRoomRadius, MaxRoomRadius)` and `Max(TunnelMinRadius, TunnelMaxRadius)` and use them in `MaxInfluence`, collection margins, vertical room buffer, and the wrapper margin. Do not reorder the endpoints passed to `Lerp`, because that would change deterministic room/tunnel assignment; only make the bounds cover every value the existing interpolation can produce. Add asset validation that warns on reversed or non-positive ranges.
|
||||
|
||||
### VF-06 — A fixed-only strate configuration silently disables the strate system
|
||||
|
||||
**Evidence status:** Verified by reading.
|
||||
|
||||
**Location:** `Source/VoxelForge/Private/VoxelWorld.cpp` — `AVoxelWorld::BeginPlay` (413–421); `Source/VoxelForge/Private/VoxelStrateManager.cpp` — `UVoxelStrateManager::Initialize` (43–55, 76–105).
|
||||
|
||||
**What is wrong:** `BeginPlay` creates the strate manager only when `Settings->StratePool.Num() > 0`. The manager itself explicitly supports fixed entries independently: it loads `FixedStrates` and selects a fixed definition before consulting the shuffled pool. A valid setup in which every requested slot is fixed and `StratePool` is empty therefore never constructs the manager.
|
||||
|
||||
**Why it matters:** the generator silently falls back to generic TunnelNetwork terrain, while content and atmosphere receive a null manager. Authored fixed strata are ignored without an initialization error.
|
||||
|
||||
**Concrete change:** initialize the manager when either `StratePool` or `FixedStrates` is non-empty. Validate that every index in `[0, TotalStrates)` can resolve either a fixed definition or a pool fallback, and emit a clear error for uncovered slots rather than silently changing generation mode.
|
||||
|
||||
### VF-07 — World origin is used as the “no player” sentinel
|
||||
|
||||
**Evidence status:** Verified by reading.
|
||||
|
||||
**Location:** `Source/VoxelForge/Private/VoxelWorld.cpp` — `AVoxelWorld::Tick` (456–484) and `GetPlayerPosition` (529–537); declaration/comment in `Source/VoxelForge/Public/VoxelWorld.h` (634–635).
|
||||
|
||||
**What is wrong:** `GetPlayerPosition` returns `FVector::ZeroVector` when there is no pawn, but a pawn at the real world origin returns the same value. `Tick` tests `PlayerLastPos != FVector::ZeroVector` before all terrain streaming, atmosphere, decorations, landmarks, water, and density-volume updates.
|
||||
|
||||
**Why it matters:** origin is a common initial spawn. While the pawn is exactly there, no initial terrain/content streaming is submitted; behavior begins only after it moves away.
|
||||
|
||||
**Concrete change:** return success separately from the coordinate (`bool TryGetPlayerPosition(FVector& Out)` or an optional), or obtain the controller/pawn in `Tick` and gate on pointer validity. Treat every finite coordinate, including zero, as a valid position.
|
||||
|
||||
### VF-08 — Decoration palettes are rebuilt every tick and deep-copied into every cell task
|
||||
|
||||
**Evidence status:** Verified by reading.
|
||||
|
||||
**Location:** `Source/VoxelForge/Private/VoxelContentManager.cpp` — `UVoxelContentManager::UpdateDecorations` (147–256) and `LaunchDecoTasks` (366–398); `Source/VoxelForge/Public/VoxelContentManager.h` — `FDecoCellResult::Entries` (151–158); nested decoration arrays in `Source/VoxelForge/Public/VoxelStrateTypes.h` — `FDecoCompanion::SubCompanions` (2073–2076) and `FStrateDecoration::Companions` (2120–2123).
|
||||
|
||||
**What is wrong:** `UpdateDecorations` is called every tick. Before it checks whether the player changed cell or stratum, it resets both flattened palettes, walks every biome and decoration, and copies every `FStrateDecoration`. Those structs contain nested `TArray`s, so this is not a trivial POD copy. `LaunchDecoTasks` then deep-copies the same grid palette and biome-tag array once for every cell task and moves that copy through the result solely so `EntryIdx` can be decoded on the game thread.
|
||||
|
||||
At the default 4x4 region size, one new region is 16 cell tasks carrying 16 copies of the same immutable palette. The per-frame rebuild also contradicts the nearby “cheap no-op unless the player crosses a decoration cell boundary or changes strate” expectation.
|
||||
|
||||
**Why it matters:** this creates allocator traffic and memory bandwidth on both the steady game-thread path and every decoration-streaming burst. Large biome palettes with companion/sub-companion trees amplify the cost.
|
||||
|
||||
**Concrete change:** build an immutable resolved palette snapshot only when its inputs change (stratum/layout/asset revision, tier assignment, or relevant settings). Capture a thread-safe shared reference in cell tasks and carry that same reference in results, or resolve spawn commands to a compact immutable profile table once. Continue draining tasks/results each tick, but do not destroy and reconstruct unchanged nested arrays.
|
||||
|
||||
### VF-09 — Clearing decoration builds forgets still-running tasks and defeats the concurrency cap
|
||||
|
||||
**Evidence status:** Verified by reading.
|
||||
|
||||
**Location:** `Source/VoxelForge/Private/VoxelContentManager.cpp` — `ResetGridBuildState` (88–94), `LaunchDecoTasks` (344–404), `ProcessDecoResults` (763–774), and `ClearAllDecorations` (983–994).
|
||||
|
||||
**What is wrong:** `ClearAllDecorations` abandons builds and calls `ResetGridBuildState`, which clears `InFlightCells` even though the corresponding tasks are not cancelled or awaited. A new build can immediately launch another task for the same cell. When the old result arrives, `ProcessDecoResults` removes `R.Cell` from `InFlightCells` before checking its `BuildId`; this can remove the new task's marker. Payload merging is protected by `BuildId`, but scheduling ownership is not.
|
||||
|
||||
The throttle uses `NearGrid.InFlightCells.Num() + FarGrid.InFlightCells.Num()`, not `GActiveDecoTasks`, so forgotten/incorrectly removed markers allow actual worker count to exceed `MaxConcurrentDecorationTasks`. Repeated clear/rebuild cycles can compound the excess precisely when live editing or regeneration is already generating other work.
|
||||
|
||||
**Why it matters:** the configured cap is documented as preventing decoration marching from crowding mesh-generation workers. This bookkeeping path invalidates that guarantee and can create avoidable CPU/memory bursts. It can also cause redundant same-cell work, though `BuildId` prevents duplicate applied decorations.
|
||||
|
||||
**Concrete change:** track an in-flight token that includes grid, cell, and build ID (for example, `TMap<FIntPoint, uint32>`), and remove it only when the completing result owns that exact token. Do not erase live tokens when abandoning build payloads; retain them until completion/cancellation. Better, keep per-instance task handles/counts and throttle on the actual running count, with build identity used only for result relevance.
|
||||
|
||||
### VF-10 — Per-room terrain params are reconstructed for every near-surface sample
|
||||
|
||||
**Evidence status:** Verified by reading.
|
||||
|
||||
**Location:** `Source/VoxelForge/Private/VoxelGenerator.cpp` — `UVoxelGenerator::GetDensityWithParams`, per-room block (1294–1328); `Source/VoxelForge/Private/VoxelDensityOpStack.cpp` — `FRoomGraphSource::FState`/`LocalParams` (1993–2012, 2041–2073); `Source/VoxelForge/Private/VoxelCaveMorphology.cpp` — room-op selection and existing per-room feature pre-bake (653–678).
|
||||
|
||||
**What is wrong:** for every `bNearCaveSurface` sample, the original path copies the roughly 74-field `FStrateGenerationParams` and makes a virtual `RoomOp->ApplyTo` call for the nearest room. The operator stack preserves one such copy per sample through `LocalParams()`—correctly memoized so eleven detail operators do not each repeat it—but the work is still invariant for all samples whose nearest cached room is the same.
|
||||
|
||||
`BuildChunkCache` already selects `RoomOp`/weight per cached room and calls `ApplyTo` once per room to pre-bake pits, chimneys, and columns. The remaining detail-op parameters can be resolved at that same per-room tier.
|
||||
|
||||
**Why it matters:** this is inside the density hot path. A full-resolution tile has about 42,875 base grid samples, plus density calls used for surface normals; only near-surface samples pay this block, but those are exactly the samples concentrated around generated geometry. Copying a large struct and dispatching virtually per sample adds bandwidth and instruction cost that could be per-room/per-chunk.
|
||||
|
||||
**Concrete change:** add a compact `FResolvedRoomDetailParams` to `FCachedRoom`, containing only fields consumed by the eleven per-room detail stages, and populate it once during `BuildChunkCache` from the chunk's base params plus the selected op. Both the oracle path and operator-stack path should reference that shared resolved payload. Keep the existing operation order and verify bit-for-bit equivalence; this is a hoist of loop-invariant data, not a split or transcription of `BuildChunkCache`.
|
||||
|
||||
## Priority order
|
||||
|
||||
Fix VF-01 through VF-03 first: they are memory-model/lifetime/cache-identity problems and can produce crashes or cross-world corruption. VF-04 through VF-06 are deterministic correctness failures with bounded, local fixes. VF-07 is a small but user-visible initialization defect. VF-08 through VF-10 are worthwhile efficiency changes after the correctness hazards are closed; VF-08 and VF-09 should be addressed together because an immutable palette snapshot and explicit task ownership naturally simplify both paths.
|
||||
@@ -0,0 +1,234 @@
|
||||
# AUDIT-COLD-2026-09-14: read-only examination, with the orchestrator's verification notes
|
||||
|
||||
Produced by Codex (gpt-5.6-luna, max effort, read-only sandbox) on 2026-09-14 at HEAD `7a972f5`,
|
||||
briefed with no project background. It did read the plugin-root docs, so it is not context-free,
|
||||
but its findings are code-level. Findings are verbatim below. These notes record what the
|
||||
orchestrator checked against the code afterwards.
|
||||
|
||||
## Orchestrator verification notes
|
||||
- **#7 CONFIRMED, a determinism bug introduced by the tile-cache work (`2de603c`/`cfad3d1`).**
|
||||
`VoxelGenerator.cpp:2968` moves `CP_OpStack` into the cache entry and resets `GTunnelCoreCache`.
|
||||
A later request for the same `ChunkCoord` under the other `bTileCacheContext` misses the key
|
||||
(`:2660`), but the refetch guard (`:2723-2725`) skips because chunk, owner and version are
|
||||
unchanged, so evaluation uses the moved-from (empty) stack and core cache. The hit path sets
|
||||
`CP_Chunk` to the shared key `(0,0,Z)` (`:2698`), so the spawn column `X=Y=0` reaches this state
|
||||
after an ordinary hit, not only in a narrow timing window. Coarse tiles (LOD >= 3) covering the
|
||||
origin can be evaluated wrongly depending on the order work reaches a worker.
|
||||
- **#3 and #8 are real but narrower than stated.** In multiplayer, the server and client worlds share
|
||||
seed and layout by design, so the cached values are identical. Contamination needs worlds that
|
||||
genuinely differ under the same key: editor edits that do not bump the version, or two diff
|
||||
layers with different contents at the same `ModsVersion`. Cheap to fix (add owner/layer identity).
|
||||
- **#19 is not a defect under C++20** (UE 5.7), where left-shifting negative signed values is
|
||||
defined. Overflow for extreme coordinates remains theoretical.
|
||||
- **#21 matters again** if the gap/seal AllSolid proof (in progress) calls `HasAnyModInChunkRange`
|
||||
per LOD0 tile.
|
||||
- The others (#1, #2, #4-6, #9-18, #20, #22) were not independently re-verified. Verify each before
|
||||
fixing it.
|
||||
|
||||
---
|
||||
|
||||
# Read-only audit report
|
||||
|
||||
I made no file changes and did not build or run the plugin. “VERIFIED” means the failure follows directly from the source path; “SUSPECT” means a runtime/compiler-dependent hazard that was not executed.
|
||||
|
||||
## Highest impact
|
||||
|
||||
1. **The density-volume worker races live generator mutation — P1 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelWorld.cpp` — `FScopedGenerationPause` (203–222), `AVoxelWorld::RebuildStrates` (551–582), `AVoxelWorld::ChangeSeed` (5569–5623).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelDensityVolume.cpp` — `UVoxelDensityVolume::Reset` (101–120), `ProcessOneFill` (603–637).
|
||||
|
||||
`FScopedGenerationPause` waits for ordinary generation tasks and decoration tasks, but not the dedicated density-volume fill thread. That thread reads the raw `Generator` pointer and calls `GetDensityAt` while `RebuildStrates` or `ChangeSeed` mutates the generator and strata manager.
|
||||
|
||||
A fill in progress during a PIE edit or seed change can therefore read partially rebuilt arrays or mixed old/new settings. This is an actual data race with crash/garbled-density potential. `Reset` also clears `PendingFills` but not the already queued `FillQueue`, causing stale work to continue after reset.
|
||||
|
||||
2. **Decoration tasks can use a destroyed content manager — P1 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelContentManager.cpp` — `UVoxelContentManager::BeginDestroy` (58–62), `NotifyShutdown` (65–75), `WaitForDecorationTasks` (78–85), `LaunchDecoTasks` (327–415).
|
||||
|
||||
Tasks capture raw `this` and later access `bShuttingDown` and `DecoResults`. Shutdown waits only three seconds, ignores the timeout result, and `BeginDestroy` immediately calls `Super::BeginDestroy`.
|
||||
|
||||
Any decoration task exceeding the timeout can later enqueue into the destroyed manager. The global task counter also includes tasks belonging to unrelated worlds, so one world can time out because another world is busy.
|
||||
|
||||
3. **The diff snapshot TLS cache is not keyed by its owning diff layer — P1 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelGenerator.cpp` — `UVoxelGenerator::GetDensityAt` (3069–3097).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Public\VoxelDiffLayer.h` — `GetModsVersion` and `ModsVersion` (241, 289).
|
||||
|
||||
`thread_local FDiffSlot DiffSlots[64]` considers only chunk coordinates and `ModsVersion`. It does not include the `UVoxelDiffLayer*`.
|
||||
|
||||
Two PIE worlds commonly have the same chunk and the same per-layer version—for example, both reach version 2 after their first edit. If a worker samples world A first, world B can hit A’s cached modification snapshot and apply the wrong edits. This produces cross-world terrain contamination dependent on worker history.
|
||||
|
||||
4. **Fixed-only strata configurations are silently disabled — P1 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelWorld.cpp` — `AVoxelWorld::BeginPlay` (1294–1305).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelStrateManager.cpp` — `UVoxelStrateManager::Initialize` (1060–1244).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelGenerator.cpp` — fallback generation in `GetDensityAt` (3030–3039).
|
||||
|
||||
`BeginPlay` creates a strata manager only when a season is assigned or `StratePool.Num() > 0`. It does not test `FixedStrates.Num()`.
|
||||
|
||||
A settings asset with valid fixed strata but no season and no pool therefore receives no strata manager. The generator takes its generic fallback path, ignoring the authored fixed definitions.
|
||||
|
||||
5. **Settings rebuilds leave dependent systems with inconsistent snapshots — P1/P2 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelWorld.cpp` — `RebuildStrates` (551–582), `PostEditChangeProperty` (885–895), `OnObjectModifiedInEditor` (898–985).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelGenerator.cpp` — `InitializeSettings` (2079–2088).
|
||||
|
||||
`RebuildStrates` reinitializes the strata manager but does not reinitialize generator settings. `PostEditChangeProperty` only regenerates chunks. Asset-change handling recognizes strata and terrain-op definitions, but not general settings or biome assets.
|
||||
|
||||
Changing seed, world radius, edge sealing, the settings object reference, or related configuration can leave the manager using new values while the generator, content manager, or density volume still use old snapshots. `ChangeSeed` has a separate correct reinitialization path, but the other rebuild paths do not. This also contradicts the `RebuildStrates` documentation in `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Public\VoxelWorld.h` (572–575), which promises to re-read all settings.
|
||||
|
||||
6. **Invalid strata initialization destroys the previous valid layout — P1/P2 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelStrateManager.cpp` — `UVoxelStrateManager::Initialize` (1060–1205).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelWorld.cpp` — `RebuildStrates` (562–576), `OnObjectModifiedInEditor` (975–985).
|
||||
|
||||
`Initialize` empties `StrateLayout`, `SeasonStrates`, and related state before all season/layout validation has succeeded. If an assigned season is missing, malformed, or contains invalid bounds, initialization returns false after the old layout has already been discarded.
|
||||
|
||||
`RebuildStrates` then leaves existing meshes visible while future generation queries see an empty or partial manager. The editor asset-modification path ignores the initialization return value and regenerates anyway, making a transient invalid asset capable of replacing the working world with fallback/empty terrain.
|
||||
|
||||
## Material correctness and determinism issues
|
||||
|
||||
7. **The operator-stack cache is stale when tile-cache context changes — P2 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelGenerator.cpp` — `UVoxelGenerator::GetDensityAt` (2218–2230, 2255–2261, 2320–2324, 2393–2407, 2551–2573, 2600–2672).
|
||||
|
||||
The tunnel cache key includes whether tile-cache mode is active, but the rebuild guard does not. A fine-LOD request can build and move the operator stack into the cache, leaving the worker-local `CP_OpStack` empty. A later request for the same chunk under a different tile-cache context misses the cache key but does not rebuild the stack because chunk, manager, and layout version are unchanged.
|
||||
|
||||
The empty or mismatched stack is then used for density evaluation. LOD request order can therefore change terrain results.
|
||||
|
||||
8. **Several surface and biome TLS caches omit generator/context identity — P2 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelGenerator.cpp` — `FSurfaceColumnCache::Acquire` (274–328), `GetDensityAt` (2692–2705), `GetSurfaceHeightAt` (5149–5166), `ClassifyTile` (6014–6023), `GetBiomeMaterialAt` (7019–7034).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Public\VoxelBiomeTypes.h` — `FChunkBiomeCache::Contains` (240–245).
|
||||
|
||||
These worker-local caches use combinations of chunk, seed, strata key, and layout version, but not the owning generator, strata manager, or biome context. Two worlds can legitimately share all those scalar keys while having different terrain or biome parameters.
|
||||
|
||||
When a pooled worker handles both worlds, the second world can receive cached surface columns, biome classification, or material data from the first. Direct asset changes that do not advance the relevant version have the same stale-cache problem.
|
||||
|
||||
9. **Native shaft/island topology caches omit parameters used to build them — P2 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelGenerator.cpp` — `GetVerticalShaftDensity` (7090–7106, 7211–7219, 7392–7413), `GetFloatingIslandDensity` (7518–7554).
|
||||
|
||||
The shaft cache key omits `LedgeSpacing` and `LedgeDepth`, although both affect connector and ledge placement. The floating-island cache key omits `BoundarySealThickness`, although it affects cached vertical spread.
|
||||
|
||||
Changing these values—or using two contexts with identical keyed values but different omitted values—reuses old topology on the same worker.
|
||||
|
||||
10. **Completed pre-edit decoration results can resurrect removed objects — P2 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelWorld.cpp` — `ApplyModification` (5319–5375).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelContentManager.cpp` — `LaunchDecoTasks` (381–404), `ProcessDecoResults` (769–807), `MergeCellResult` (815–862), `RemoveDecorationsInSphere` (1249–1260).
|
||||
|
||||
Applying a live diff removes currently applied decorations but does not invalidate queued/in-flight decoration builds or associate them with a diff version. `BuildId` only identifies decoration-region rebuilds.
|
||||
|
||||
Sequence: a task computes old grass/props, the edit removes current instances, then the old result is drained and accepted. The old decorations reappear over the edited terrain until another rebuild occurs.
|
||||
|
||||
11. **Landmark conditions are evaluated without the actual biome context — P2 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelContentManager.cpp` — `UpdateDecorations` (153–263), `UpdateLandmarks` (1263–1310), `SpawnFromProfile` (1101–1137).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelGenerator.cpp` — `EvaluateTerrainConditions` (6682–6717).
|
||||
|
||||
Normal decoration updates populate `FDecoContext::BiomeCtx`; landmark updates construct a context but never set `BiomeCtx`. A landmark profile using biome, moisture, relief, or biome-border conditions therefore evaluates with the default/invalid biome map.
|
||||
|
||||
On a non-default biome map, landmarks can spawn or fail to spawn differently from ordinary decorations.
|
||||
|
||||
12. **Decoration cleanup uses local coordinates as world coordinates — P2 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelWorld.cpp` — `ApplyModification` (5369–5375); `WorldToLocalVoxel` and `LocalVoxelToWorld` (2314–2321).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelContentManager.cpp` — `RemoveInstancesInContent` (1231–1242).
|
||||
|
||||
`Modification.Center` is a local voxel coordinate, but cleanup computes `Center * VOXEL_SIZE` and passes it to a world-space overlap query. For a translated actor, the query is centered at the wrong location.
|
||||
|
||||
The same cleanup also approximates boxes and capsules with only `Modification.Radius`, which does not cover a long capsule or a box’s full extent. Existing decorations can remain over an edit or unrelated origin decorations can be removed.
|
||||
|
||||
13. **Decoration application order is timing-dependent — P2 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelContentManager.cpp` — `LaunchDecoTasks` (327–415), `ProcessDecoResults` (769–810), `ApplyRegion` (885–952), `RebuildDesiredCells` (281–324).
|
||||
|
||||
Tasks enqueue results in completion order. Results append actor spawns and HISM transforms in that order; mesh buckets are iterated through a `TMap`. Equal-distance pending regions also have no deterministic tie-breaker.
|
||||
|
||||
The same candidate set can therefore produce different actor spawn order, NetGUID/side-effect order, HISM instance indices, and internal clustering across runs or machines.
|
||||
|
||||
14. **The passage shortlist version is not advanced when passages are cleared — P2/P3 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelStrateManager.cpp` — `GeneratePassages` (1611–1620, 2477), `EvaluateModifierSDF` (2488–2524), `VF_GetNearbyPassages` (294–338).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Public\VoxelStrateManager.h` — `GetLayoutVersion` and `PassagesVersion` (250, 515, 519).
|
||||
|
||||
`GeneratePassages` empties `Passages` and returns early for an empty layout without incrementing `PassagesVersion`. Existing worker TLS nearby-passage caches still match the manager lifetime, chunk, and unchanged version.
|
||||
|
||||
After a previously valid layout becomes empty, `EvaluateModifierSDF` can reuse indices into the now-empty array and dereference invalid passage data. This is an edge configuration, but it is reachable through the invalid-layout paths above.
|
||||
|
||||
## Input-contract and performance issues
|
||||
|
||||
15. **Diff modifications accept nonfinite and invalid geometry — P2/P3 — VERIFIED for malformed input**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelDiffLayer.cpp` — `CanModify` (20–44), `ApplyModification` (63–96), `EvaluateMods` (191–248).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelWorld.cpp` — `CarveAtPosition` (5301–5307).
|
||||
|
||||
There is no finite-value validation for strength, radius, centers, endpoints, or extents. A NaN strength survives `FMath::Abs`, then propagates through `TotalOffset` and density evaluation. Infinite strength produces infinite density. Negative or zero radii can consume budget while producing no meaningful spherical modification.
|
||||
|
||||
This is reachable from C++ or malformed Blueprint/runtime data and permanently poisons the diff layer until cleared.
|
||||
|
||||
16. **The public diff API contradicts itself about coordinate units — P2/P3 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Public\VoxelDiffLayer.h` — `FVoxelModification::Center` (64), `CapsuleEnd` (86).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Public\VoxelWorld.h` — `ApplyModification` documentation (334–340).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelDiffLayer.cpp` — `GetWorldBounds` (88–96).
|
||||
|
||||
The modification struct comments describe the center as world-space, while the low-level world API and implementation interpret it as voxel-space. The helper functions explicitly convert world centimeters into voxel coordinates.
|
||||
|
||||
A caller following the struct comment and supplying centimeters gets an edit displaced by a factor of `VOXEL_SIZE`.
|
||||
|
||||
17. **Diff budget accounting underestimates boxes and long capsules — P3 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelWorld.cpp` — `CarveBox`/`FillBox` (5379–5398), capsule helpers (5401 onward).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelDiffLayer.cpp` — `CanModify` (34–39).
|
||||
|
||||
The budget charges every shape as a sphere with volume based on `Modification.Radius`. Boxes use the maximum half-extent as a proxy; capsules use the tube radius even when endpoints are very far apart.
|
||||
|
||||
A capsule with radius 3 and endpoints 1000 voxels apart is charged like a tiny sphere despite affecting a long volume and many chunks. Large edits can therefore bypass the intended modification budget and cause excessive remeshing.
|
||||
|
||||
18. **Decoration radius/spacing changes are ignored while the player is stationary — P3 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelContentManager.cpp` — `UpdateDecorations` (199–256), `RebuildDesiredCells` (265–325).
|
||||
|
||||
Radius and spacing are refreshed every update, but desired-cell rebuilding occurs only when `PlayerCell != LastDecoCell`.
|
||||
|
||||
Shrinking the radius leaves distant regions loaded; expanding it does not enqueue new regions; changing spacing leaves existing regions at the old grid layout. Movement eventually repairs the state, but a stationary player sees stale decoration coverage.
|
||||
|
||||
19. **Signed left shifts of negative chunk coordinates are undefined — P3 — SUSPECT**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelWorld.cpp` — `BuildDesiredTiles` (3498–3548).
|
||||
|
||||
Streaming computes coordinates with expressions such as `T.Z << L`. Ordinary underground tiles have negative `T.Z`, so this left-shifts a negative signed integer. Large positive coordinates can also overflow.
|
||||
|
||||
The exact manifestation is compiler/optimization-dependent, so I classify this as SUSPECT rather than a fully end-to-end verified runtime failure.
|
||||
|
||||
20. **Interleaved transitions can divide by zero — P3 — VERIFIED math path**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Public\VoxelStrateDefinition.h` — `TransitionBlendChunks` (99–101).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelStrateManager.cpp` — `GetGenerationParams` (5673, 5728–5753, 5826).
|
||||
|
||||
The property metadata clamps the value in the editor, but runtime/cooked data can still contain zero. The interleaved transition path divides warped distance by `EffectiveBlend` without a zero guard.
|
||||
|
||||
A zero-width transition can produce Inf/NaN blend values and invalid generation parameters.
|
||||
|
||||
21. **Tile classification scans the entire modification map — P3 — VERIFIED**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelGenerator.cpp` — `ClassifyTileUncached` (5785–5790).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelDiffLayer.cpp` — `HasAnyModInChunkRange` (170–188).
|
||||
|
||||
Classification calls a routine that walks every `ChunkMods` key under the read lock instead of using the existing per-chunk grouping to narrow the query.
|
||||
|
||||
After many edits, each tile classification becomes O(number of modified chunks), increasing lock hold time and making edit-heavy worlds disproportionately expensive.
|
||||
|
||||
22. **Operator documentation is internally stale — P3 — VERIFIED documentation contradiction**
|
||||
|
||||
Location: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Public\VoxelDensityOp.h` (1–3, 86–92).
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelGenerator.cpp` — `UVoxelGenerator::GetDensityAt` (2393–2600).
|
||||
|
||||
The header says the density-op system is not wired into `GetDensityAt`, while later comments say all eight operators now have the stack path. The generator does in fact execute the operator stack.
|
||||
|
||||
This is not itself a runtime defect, but it is misleading maintenance documentation around a central generation path.
|
||||
|
||||
The most urgent issues are the uncoordinated density-volume thread, decoration-task lifetime, cross-world TLS diff cache, and incomplete settings/layout rebuilds.
|
||||
@@ -0,0 +1,402 @@
|
||||
# AUDIT-SOL-2026-09-11.md — read-only audit of experimental @ aa62971
|
||||
|
||||
Performed by Codex **gpt-5.6-sol**, reasoning effort xhigh, **sandbox read-only** — no file was
|
||||
created, modified or deleted. Commissioned by Jahni after the experimental branch showed two
|
||||
regressions against main: generation much slower, and the player falling through the world.
|
||||
|
||||
Verbatim report follows.
|
||||
|
||||
---
|
||||
|
||||
# Audit conclusion
|
||||
|
||||
The two regressions share a cause, but they are not the same bug:
|
||||
|
||||
1. **The remaining generation slowdown is primarily in density evaluation.** The composable operator representation is still being interpreted in the sampling hot path, with rich per-sample state, block-wide operator passes, and several corrective passes afterward. The outer tile classifier is not where the original ~8× went.
|
||||
|
||||
2. **There is no “collision-ready before player-ready” invariant.** Neither branch explicitly had one. `main` was fast enough—and had simpler spawn geometry—so the race remained hidden. `experimental` made the race long enough to become visible, especially because the new origin room’s air and supporting floor can occupy different tiles.
|
||||
|
||||
I disagree with the inference that the old behavior proves there was a correctness barrier. The observation is valid, but the code shows a timing dependency masquerading as a guarantee.
|
||||
|
||||
## 1. Density evaluation is the dominant performance regression
|
||||
|
||||
**Status: Verified architecture; strong attribution. Highest expected performance impact.**
|
||||
|
||||
A streamed LOD0 tile asks for a 35³ haloed density grid: **42,875 sample positions** ([VoxelMarchingCubesMesher.cpp:227](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:227>)).
|
||||
|
||||
At `34f06df`, that hot path was approximately:
|
||||
|
||||
- Resolve one cached chunk parameter set.
|
||||
- Enter one direct archetype switch.
|
||||
- Keep most intermediate values in local variables.
|
||||
- Skip the detailed cave suite when `bNearCaveSurface` was false—historically around 70% of deep-solid samples.
|
||||
- Apply disturbances/differences and return.
|
||||
|
||||
The relevant historical path is `34f06df:Source/VoxelForge/Private/VoxelGenerator.cpp:438-631` and `:947-1621`.
|
||||
|
||||
Current `experimental` instead:
|
||||
|
||||
- Allocates/initializes an `FVoxelOpSample` for every position, containing density, SDF, floor metadata, tunnel metadata, pointers and indices ([VoxelDensityOp.h:327](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelDensityOp.h:327>)).
|
||||
- Determines a block-wide active-operator list.
|
||||
- Traverses the sample array once per active operator.
|
||||
- Uses scalar per-sample fallback implementations for most concrete operators ([VoxelDensityOpStack.cpp:51](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelDensityOpStack.cpp:51>)).
|
||||
- Copies block scratch into the session cache and later back into output.
|
||||
- Still calls the scalar `GetDensityAt` wrapper for every grid point, so routing, coordinate setup and the post-stack structural tail remain per-sample ([VoxelGenerator.cpp:1690](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:1690>), [VoxelGenerator.cpp:2165](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:2165>)).
|
||||
|
||||
Loop inversion removed much of the virtual dispatch cost, but it did not restore the old locality or short-circuiting. An operator whose interval intersects the surface anywhere in the block can be run across the whole block; the old path could decide per sample that expensive detail work was irrelevant.
|
||||
|
||||
That is the architectural mistake: **the composable authoring model was carried directly into execution instead of being compiled/lowered into a fused evaluator.** Composition is not itself the problem. Interpreting it through rich mutable sample records is.
|
||||
|
||||
The 128³ comparison strongly supports this conclusion: the editor export path owns a shared density grid and does not force the streaming outer-classifier path ([VoxelForgeExploreCommandlet.cpp:1700](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp:1700>)), yet it still takes approximately 1.04 seconds versus 0.130 seconds before the operator-stack wire-in.
|
||||
|
||||
The later features add cost, particularly the D-shaped floors, but the history indicates roughly a 17% net penalty there—not anything close to the remaining 8×.
|
||||
|
||||
## 2. The player can become active before any supporting collision exists
|
||||
|
||||
**Status: Verified. Highest correctness impact.**
|
||||
|
||||
`BeginPlay` initializes systems, but it does not synchronously request a spawn support tile, wait for a mesh, or gate pawn gravity/control ([VoxelWorld.cpp:988](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:988>)).
|
||||
|
||||
Streaming begins from `Tick`, and only when `GetPlayerPosition()` is not exactly `FVector::ZeroVector` ([VoxelWorld.cpp:1167](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:1167>), [VoxelWorld.cpp:1359](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:1359>)). Thus an exact world-origin pawn is indistinguishable from “no pawn.” Even when Z is nonzero and that specific sentinel is avoided, there is still no bootstrap barrier.
|
||||
|
||||
The supplied play trace shows:
|
||||
|
||||
- `BeginPlay` finished at about **50 ms**.
|
||||
- The first desired-tile pass did not begin until about **439 ms**.
|
||||
- The first LOD0 nonempty/mixed result was not ready until about **1.093 s**.
|
||||
- That result still represented mesh/collision submission, not collision-cook completion.
|
||||
|
||||
At ordinary gravity, one second is enough to fall roughly five metres.
|
||||
|
||||
There are three additional gaps:
|
||||
|
||||
- A tile is inserted into `LoadedTiles` before its empty/nonempty application path completes ([VoxelWorld.cpp:1532](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:1532>)).
|
||||
- Only LOD0 receives collision ([VoxelWorld.cpp:3109](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:3109>)). Coarser visible terrain can therefore be present while the corresponding collidable LOD0 tile is still queued.
|
||||
- Current architecture documentation says RMC collision is asynchronously cooked ([ARCHITECTURE.md:817](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/ARCHITECTURE.md:817>)), but `VoxelWorld` records no cook-complete state. Its `apply_s` measurement ends after submission and says nothing about when physics can actually collide.
|
||||
|
||||
I compared the equivalent paths on `main`. It also:
|
||||
|
||||
- Starts streaming from `Tick`.
|
||||
- Uses the zero-position guard.
|
||||
- Marks a tile loaded before handling its empty result.
|
||||
- Enables collision only for LOD0.
|
||||
- Has no spawn collision barrier or collision-cook acknowledgement.
|
||||
|
||||
So no explicit guarantee was removed inside `VoxelWorld`. What `experimental` broke was the **implicit timing budget** that had hidden this race.
|
||||
|
||||
The new origin-room geometry makes it worse. The first central LOD0 request in the trace is uniformly air and correctly receives no component. That is not itself an error: the player occupies air, while collision belongs to the neighboring floor/wall surface tile. With the current default origin geometry, the floor can lie near or below the lower tile boundary ([VoxelPassageGeometry.h:365](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelPassageGeometry.h:365>)). The supporting tile therefore has to arrive and finish cooking independently.
|
||||
|
||||
`CollisionOnlyTiles` does not close this gap. It controls presentation of already generated LOD0 components for non-player anchors; it does not synthesize or reserve collision underneath the local pawn. The supplied trace records zero collision-only tiles.
|
||||
|
||||
## 3. The mesher’s nested block classifier is probably self-inflicted overhead
|
||||
|
||||
**Status: Mechanism verified; net cost strongly suspected. High performance impact.**
|
||||
|
||||
The code contradicts its own guidance:
|
||||
|
||||
- The mesher header says the optional 8³ refinement should remain disabled during streaming because repeated proofs may cost more than they save ([VoxelMarchingCubesMesher.h:150](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelMarchingCubesMesher.h:150>)).
|
||||
- `VoxelWorld` unconditionally enables it ([VoxelWorld.cpp:1102](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:1102>)).
|
||||
|
||||
For every outer-Mixed tile, the mesher may run up to 64 additional `ClassifyTile` calls ([VoxelMarchingCubesMesher.cpp:254](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:254>)). On cache misses, each call builds another operator stack ([VoxelGenerator.cpp:5779](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:5779>)).
|
||||
|
||||
In `MyPlay`:
|
||||
|
||||
- 298 LOD0 tiles were outer-Mixed.
|
||||
- All 298 produced geometry.
|
||||
- Therefore every one eventually populated the full density grid.
|
||||
- The nested classifier could only avoid marching-cubes cell processing in uniform sub-blocks; it could not avoid the main density grid once any surface sub-block was found.
|
||||
|
||||
This is precisely the workload where the header warns it may lose.
|
||||
|
||||
There is also an instrumentation trap: the outer cache counters are snapped before meshing. Nested classifier calls occur inside the measured `mesh_s` interval ([VoxelWorld.cpp:2762](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:2762>)). Consequently, the reported “cache hits 0” does **not** describe classifier activity originating inside the mesher.
|
||||
|
||||
This deserves the first controlled A/B test: identical trace and seed with only `bUseBlockEarlyOut` changed, while separately counting outer and nested classifier calls.
|
||||
|
||||
## 4. The outer whole-tile classifier is probably earning its keep
|
||||
|
||||
**Status: Verified workload behavior; exact counterfactual unmeasured.**
|
||||
|
||||
I would not remove the outer classifier based on the 95 ms figure.
|
||||
|
||||
It rejected **547 of 845 LOD0 tiles**, or 64.7%, before meshing:
|
||||
|
||||
- 484 uniformly solid.
|
||||
- 63 uniformly air.
|
||||
- 298 mixed, all of which produced geometry.
|
||||
|
||||
A full 35³ grid for those rejected tiles would represent up to 23.45 million density positions. Previous history also shows that the outer classifier eliminated multi-second empty-tile fallbacks.
|
||||
|
||||
The 95 ms is not a clean CPU cost. It is elapsed wall time inside a `BackgroundNormal` worker task, so it includes preemption and contention. A second trace in the repository from the same current code has an LOD0 classifier median around **8.45 ms**, versus **94.7 ms** in the moving/rendering owner session. The real play number is important for latency, but it should not be interpreted as intrinsic classifier compute.
|
||||
|
||||
My assessment is:
|
||||
|
||||
- **Outer whole-tile classification:** valuable under the current expensive evaluator.
|
||||
- **Nested 8³ block classification:** likely negative in this workload.
|
||||
- **Long-term:** making density evaluation cheap again may change the outer classifier’s break-even point.
|
||||
|
||||
## 5. Movement does not cancel obsolete generation work
|
||||
|
||||
**Status: Verified mechanism; contribution to running falls strongly suspected.**
|
||||
|
||||
The abort predicate only notices shutdown or an explicit generation pause ([VoxelWorld.h:764](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelWorld.h:764>)). It does not abort a tile because the player has moved and the tile is no longer desired.
|
||||
|
||||
Desired work is globally distance-sorted using an artificial chunk-center position rather than the exact pawn position, with no strong collision-critical/L0 priority ([VoxelWorld.cpp:1864](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:1864>)). Up to 14 background tasks can therefore be occupied by old, coarse, or non-supporting work while the player’s new LOD0 floor waits.
|
||||
|
||||
By the steady point in the supplied session there were 1,481 loaded tiles for 899 currently desired tiles, with roughly 582 on the unload/transition side. Retaining old rendered tiles is not automatically wrong, but generation already launched for the old neighborhood is not preempted in favor of collision-critical tiles.
|
||||
|
||||
This is a credible explanation for falls while running. The trace cannot prove it because it omits tile coordinates, pawn position and collision-cook completion.
|
||||
|
||||
## 6. The verdict cache is mismatched to normal streaming
|
||||
|
||||
**Status: Implementation verified; negative startup ROI suspected. Medium impact.**
|
||||
|
||||
The cache:
|
||||
|
||||
- Is global and protected by one lock.
|
||||
- Advances a clock and linearly scans entries.
|
||||
- Holds only 512 proofs ([VoxelGenerator.cpp:1512](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:1512>)).
|
||||
- Keys include sample step, limiting reuse across LODs.
|
||||
- Does not store a useful outer proof for Mixed tiles.
|
||||
|
||||
Fresh streaming normally visits unique regions, which explains the zero outer hits in both supplied traces. Nested block classifiers can then insert many short-lived proofs, pay the lock/scan costs, and evict one another—without appearing in the trace’s cache telemetry.
|
||||
|
||||
It demonstrably helped a deliberately repeated commandlet case, but that access pattern is not representative of a moving player. I would treat it as unproven or likely negative during startup until nested calls are measured.
|
||||
|
||||
## 7. Field semantics are now implemented in several overlapping places
|
||||
|
||||
**Status: Verified duplication; cost not isolated. Medium impact and high maintenance risk.**
|
||||
|
||||
The effective field is no longer simply “the operator stack”:
|
||||
|
||||
1. The operator stack evaluates the composable field.
|
||||
2. The classifier maintains parallel interval/proof semantics.
|
||||
3. `GetDensityAt` performs a post-stack repair pipeline for origin floors, passage structure, passage floors and room floors.
|
||||
|
||||
For example, `FRoomGraphSource` evaluates the cached room graph SDF and then separately traverses tunnels again for metadata ([VoxelDensityOpStack.cpp:4234](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelDensityOpStack.cpp:4234>)). The common tail then reasserts origin and passage structure/floors ([VoxelGenerator.cpp:2244](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:2244>)).
|
||||
|
||||
This is the clearest sign of overengineering: later correctness fixes are compensating for where an operation sits in the pipeline. It creates duplicated traversal, makes interval proofs harder to trust, and prevents the stack from being the single semantic source of truth.
|
||||
|
||||
Runtime also does not use the commandlet’s shared density-grid ownership, so adjacent tiles repeat halo/boundary samples. Earlier measurement put this duplication around 18%. It existed in the old architecture too, so it did not cause the original regression, but every repeated sample is now much more expensive.
|
||||
|
||||
## 8. There are two lower-confidence ways to get a permanent hole
|
||||
|
||||
**Status: Verified risk paths; not demonstrated by this trace.**
|
||||
|
||||
First, whole-tile AllAir/AllSolid verdicts are trusted without exact validation. They become loaded `bEmpty` tiles with no component. Internal block verdicts are later checked against the full grid when meshing occurs, but an outer uniform verdict has no equivalent safety net.
|
||||
|
||||
The supplied aggregate argues against a widespread problem—every LOD0 Mixed tile produced geometry—but it cannot establish whether the floor tile beneath the pawn was incorrectly declared uniform because tile coordinates were not logged.
|
||||
|
||||
Second, `ApplyMeshToTile` has no success return. If runtime-mesh initialization fails, the function returns while the tile remains recorded as loaded, so it will not naturally retry ([VoxelWorld.cpp:3064](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:3064>)). I found no trace evidence that this occurred.
|
||||
|
||||
The normal `bAborted` result path is not the likely culprit: it removes the pending record without marking the tile loaded, allowing a later desired pass to reschedule it. There is, however, an editor pause corner case where a worker can exit without enqueuing any result and leave its pending entry behind if pause acquisition times out. That is a real bug, but it does not match this epoch-zero play session.
|
||||
|
||||
## What I would prove next
|
||||
|
||||
In order:
|
||||
|
||||
1. A/B the forced nested block classifier on the exact owner trace, recording outer versus nested calls, proof outcomes, stack builds and cache probes.
|
||||
2. Add collision-specific timestamps: requested, mesh submitted, collision cook started/completed, component registered, plus tile and pawn coordinates.
|
||||
3. Force exact meshing/validation for every supposedly uniform LOD0 tile in the spawn neighborhood to rule a false classifier proof in or out.
|
||||
4. Benchmark a compiled/fused evaluator that retains the authored operator graph but restores per-sample short-circuiting and avoids spilling `FVoxelOpSample` through memory for every operator.
|
||||
5. Prioritize a small LOD0 support neighborhood and require collision-cook completion before releasing the pawn. Faster generation alone cannot make the current lifecycle correct.
|
||||
|
||||
The blunt version: **the operator abstraction leaked into the innermost loop, and streaming treats “work submitted/applied” as “safe to stand on.”** The classifier work then became a collection of compensators for the expensive evaluator, some useful and some apparently duplicative.
|
||||
|
||||
This was a read-only audit. No files were created, modified or deleted; `git status` remains clean on `experimental` at `aa62971`.
|
||||
tokens used
|
||||
782 274
|
||||
# Audit conclusion
|
||||
|
||||
The two regressions share a cause, but they are not the same bug:
|
||||
|
||||
1. **The remaining generation slowdown is primarily in density evaluation.** The composable operator representation is still being interpreted in the sampling hot path, with rich per-sample state, block-wide operator passes, and several corrective passes afterward. The outer tile classifier is not where the original ~8× went.
|
||||
|
||||
2. **There is no “collision-ready before player-ready” invariant.** Neither branch explicitly had one. `main` was fast enough—and had simpler spawn geometry—so the race remained hidden. `experimental` made the race long enough to become visible, especially because the new origin room’s air and supporting floor can occupy different tiles.
|
||||
|
||||
I disagree with the inference that the old behavior proves there was a correctness barrier. The observation is valid, but the code shows a timing dependency masquerading as a guarantee.
|
||||
|
||||
## 1. Density evaluation is the dominant performance regression
|
||||
|
||||
**Status: Verified architecture; strong attribution. Highest expected performance impact.**
|
||||
|
||||
A streamed LOD0 tile asks for a 35³ haloed density grid: **42,875 sample positions** ([VoxelMarchingCubesMesher.cpp:227](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:227>)).
|
||||
|
||||
At `34f06df`, that hot path was approximately:
|
||||
|
||||
- Resolve one cached chunk parameter set.
|
||||
- Enter one direct archetype switch.
|
||||
- Keep most intermediate values in local variables.
|
||||
- Skip the detailed cave suite when `bNearCaveSurface` was false—historically around 70% of deep-solid samples.
|
||||
- Apply disturbances/differences and return.
|
||||
|
||||
The relevant historical path is `34f06df:Source/VoxelForge/Private/VoxelGenerator.cpp:438-631` and `:947-1621`.
|
||||
|
||||
Current `experimental` instead:
|
||||
|
||||
- Allocates/initializes an `FVoxelOpSample` for every position, containing density, SDF, floor metadata, tunnel metadata, pointers and indices ([VoxelDensityOp.h:327](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelDensityOp.h:327>)).
|
||||
- Determines a block-wide active-operator list.
|
||||
- Traverses the sample array once per active operator.
|
||||
- Uses scalar per-sample fallback implementations for most concrete operators ([VoxelDensityOpStack.cpp:51](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelDensityOpStack.cpp:51>)).
|
||||
- Copies block scratch into the session cache and later back into output.
|
||||
- Still calls the scalar `GetDensityAt` wrapper for every grid point, so routing, coordinate setup and the post-stack structural tail remain per-sample ([VoxelGenerator.cpp:1690](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:1690>), [VoxelGenerator.cpp:2165](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:2165>)).
|
||||
|
||||
Loop inversion removed much of the virtual dispatch cost, but it did not restore the old locality or short-circuiting. An operator whose interval intersects the surface anywhere in the block can be run across the whole block; the old path could decide per sample that expensive detail work was irrelevant.
|
||||
|
||||
That is the architectural mistake: **the composable authoring model was carried directly into execution instead of being compiled/lowered into a fused evaluator.** Composition is not itself the problem. Interpreting it through rich mutable sample records is.
|
||||
|
||||
The 128³ comparison strongly supports this conclusion: the editor export path owns a shared density grid and does not force the streaming outer-classifier path ([VoxelForgeExploreCommandlet.cpp:1700](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp:1700>)), yet it still takes approximately 1.04 seconds versus 0.130 seconds before the operator-stack wire-in.
|
||||
|
||||
The later features add cost, particularly the D-shaped floors, but the history indicates roughly a 17% net penalty there—not anything close to the remaining 8×.
|
||||
|
||||
## 2. The player can become active before any supporting collision exists
|
||||
|
||||
**Status: Verified. Highest correctness impact.**
|
||||
|
||||
`BeginPlay` initializes systems, but it does not synchronously request a spawn support tile, wait for a mesh, or gate pawn gravity/control ([VoxelWorld.cpp:988](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:988>)).
|
||||
|
||||
Streaming begins from `Tick`, and only when `GetPlayerPosition()` is not exactly `FVector::ZeroVector` ([VoxelWorld.cpp:1167](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:1167>), [VoxelWorld.cpp:1359](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:1359>)). Thus an exact world-origin pawn is indistinguishable from “no pawn.” Even when Z is nonzero and that specific sentinel is avoided, there is still no bootstrap barrier.
|
||||
|
||||
The supplied play trace shows:
|
||||
|
||||
- `BeginPlay` finished at about **50 ms**.
|
||||
- The first desired-tile pass did not begin until about **439 ms**.
|
||||
- The first LOD0 nonempty/mixed result was not ready until about **1.093 s**.
|
||||
- That result still represented mesh/collision submission, not collision-cook completion.
|
||||
|
||||
At ordinary gravity, one second is enough to fall roughly five metres.
|
||||
|
||||
There are three additional gaps:
|
||||
|
||||
- A tile is inserted into `LoadedTiles` before its empty/nonempty application path completes ([VoxelWorld.cpp:1532](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:1532>)).
|
||||
- Only LOD0 receives collision ([VoxelWorld.cpp:3109](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:3109>)). Coarser visible terrain can therefore be present while the corresponding collidable LOD0 tile is still queued.
|
||||
- Current architecture documentation says RMC collision is asynchronously cooked ([ARCHITECTURE.md:817](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/ARCHITECTURE.md:817>)), but `VoxelWorld` records no cook-complete state. Its `apply_s` measurement ends after submission and says nothing about when physics can actually collide.
|
||||
|
||||
I compared the equivalent paths on `main`. It also:
|
||||
|
||||
- Starts streaming from `Tick`.
|
||||
- Uses the zero-position guard.
|
||||
- Marks a tile loaded before handling its empty result.
|
||||
- Enables collision only for LOD0.
|
||||
- Has no spawn collision barrier or collision-cook acknowledgement.
|
||||
|
||||
So no explicit guarantee was removed inside `VoxelWorld`. What `experimental` broke was the **implicit timing budget** that had hidden this race.
|
||||
|
||||
The new origin-room geometry makes it worse. The first central LOD0 request in the trace is uniformly air and correctly receives no component. That is not itself an error: the player occupies air, while collision belongs to the neighboring floor/wall surface tile. With the current default origin geometry, the floor can lie near or below the lower tile boundary ([VoxelPassageGeometry.h:365](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelPassageGeometry.h:365>)). The supporting tile therefore has to arrive and finish cooking independently.
|
||||
|
||||
`CollisionOnlyTiles` does not close this gap. It controls presentation of already generated LOD0 components for non-player anchors; it does not synthesize or reserve collision underneath the local pawn. The supplied trace records zero collision-only tiles.
|
||||
|
||||
## 3. The mesher’s nested block classifier is probably self-inflicted overhead
|
||||
|
||||
**Status: Mechanism verified; net cost strongly suspected. High performance impact.**
|
||||
|
||||
The code contradicts its own guidance:
|
||||
|
||||
- The mesher header says the optional 8³ refinement should remain disabled during streaming because repeated proofs may cost more than they save ([VoxelMarchingCubesMesher.h:150](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelMarchingCubesMesher.h:150>)).
|
||||
- `VoxelWorld` unconditionally enables it ([VoxelWorld.cpp:1102](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:1102>)).
|
||||
|
||||
For every outer-Mixed tile, the mesher may run up to 64 additional `ClassifyTile` calls ([VoxelMarchingCubesMesher.cpp:254](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:254>)). On cache misses, each call builds another operator stack ([VoxelGenerator.cpp:5779](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:5779>)).
|
||||
|
||||
In `MyPlay`:
|
||||
|
||||
- 298 LOD0 tiles were outer-Mixed.
|
||||
- All 298 produced geometry.
|
||||
- Therefore every one eventually populated the full density grid.
|
||||
- The nested classifier could only avoid marching-cubes cell processing in uniform sub-blocks; it could not avoid the main density grid once any surface sub-block was found.
|
||||
|
||||
This is precisely the workload where the header warns it may lose.
|
||||
|
||||
There is also an instrumentation trap: the outer cache counters are snapped before meshing. Nested classifier calls occur inside the measured `mesh_s` interval ([VoxelWorld.cpp:2762](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:2762>)). Consequently, the reported “cache hits 0” does **not** describe classifier activity originating inside the mesher.
|
||||
|
||||
This deserves the first controlled A/B test: identical trace and seed with only `bUseBlockEarlyOut` changed, while separately counting outer and nested classifier calls.
|
||||
|
||||
## 4. The outer whole-tile classifier is probably earning its keep
|
||||
|
||||
**Status: Verified workload behavior; exact counterfactual unmeasured.**
|
||||
|
||||
I would not remove the outer classifier based on the 95 ms figure.
|
||||
|
||||
It rejected **547 of 845 LOD0 tiles**, or 64.7%, before meshing:
|
||||
|
||||
- 484 uniformly solid.
|
||||
- 63 uniformly air.
|
||||
- 298 mixed, all of which produced geometry.
|
||||
|
||||
A full 35³ grid for those rejected tiles would represent up to 23.45 million density positions. Previous history also shows that the outer classifier eliminated multi-second empty-tile fallbacks.
|
||||
|
||||
The 95 ms is not a clean CPU cost. It is elapsed wall time inside a `BackgroundNormal` worker task, so it includes preemption and contention. A second trace in the repository from the same current code has an LOD0 classifier median around **8.45 ms**, versus **94.7 ms** in the moving/rendering owner session. The real play number is important for latency, but it should not be interpreted as intrinsic classifier compute.
|
||||
|
||||
My assessment is:
|
||||
|
||||
- **Outer whole-tile classification:** valuable under the current expensive evaluator.
|
||||
- **Nested 8³ block classification:** likely negative in this workload.
|
||||
- **Long-term:** making density evaluation cheap again may change the outer classifier’s break-even point.
|
||||
|
||||
## 5. Movement does not cancel obsolete generation work
|
||||
|
||||
**Status: Verified mechanism; contribution to running falls strongly suspected.**
|
||||
|
||||
The abort predicate only notices shutdown or an explicit generation pause ([VoxelWorld.h:764](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelWorld.h:764>)). It does not abort a tile because the player has moved and the tile is no longer desired.
|
||||
|
||||
Desired work is globally distance-sorted using an artificial chunk-center position rather than the exact pawn position, with no strong collision-critical/L0 priority ([VoxelWorld.cpp:1864](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:1864>)). Up to 14 background tasks can therefore be occupied by old, coarse, or non-supporting work while the player’s new LOD0 floor waits.
|
||||
|
||||
By the steady point in the supplied session there were 1,481 loaded tiles for 899 currently desired tiles, with roughly 582 on the unload/transition side. Retaining old rendered tiles is not automatically wrong, but generation already launched for the old neighborhood is not preempted in favor of collision-critical tiles.
|
||||
|
||||
This is a credible explanation for falls while running. The trace cannot prove it because it omits tile coordinates, pawn position and collision-cook completion.
|
||||
|
||||
## 6. The verdict cache is mismatched to normal streaming
|
||||
|
||||
**Status: Implementation verified; negative startup ROI suspected. Medium impact.**
|
||||
|
||||
The cache:
|
||||
|
||||
- Is global and protected by one lock.
|
||||
- Advances a clock and linearly scans entries.
|
||||
- Holds only 512 proofs ([VoxelGenerator.cpp:1512](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:1512>)).
|
||||
- Keys include sample step, limiting reuse across LODs.
|
||||
- Does not store a useful outer proof for Mixed tiles.
|
||||
|
||||
Fresh streaming normally visits unique regions, which explains the zero outer hits in both supplied traces. Nested block classifiers can then insert many short-lived proofs, pay the lock/scan costs, and evict one another—without appearing in the trace’s cache telemetry.
|
||||
|
||||
It demonstrably helped a deliberately repeated commandlet case, but that access pattern is not representative of a moving player. I would treat it as unproven or likely negative during startup until nested calls are measured.
|
||||
|
||||
## 7. Field semantics are now implemented in several overlapping places
|
||||
|
||||
**Status: Verified duplication; cost not isolated. Medium impact and high maintenance risk.**
|
||||
|
||||
The effective field is no longer simply “the operator stack”:
|
||||
|
||||
1. The operator stack evaluates the composable field.
|
||||
2. The classifier maintains parallel interval/proof semantics.
|
||||
3. `GetDensityAt` performs a post-stack repair pipeline for origin floors, passage structure, passage floors and room floors.
|
||||
|
||||
For example, `FRoomGraphSource` evaluates the cached room graph SDF and then separately traverses tunnels again for metadata ([VoxelDensityOpStack.cpp:4234](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelDensityOpStack.cpp:4234>)). The common tail then reasserts origin and passage structure/floors ([VoxelGenerator.cpp:2244](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:2244>)).
|
||||
|
||||
This is the clearest sign of overengineering: later correctness fixes are compensating for where an operation sits in the pipeline. It creates duplicated traversal, makes interval proofs harder to trust, and prevents the stack from being the single semantic source of truth.
|
||||
|
||||
Runtime also does not use the commandlet’s shared density-grid ownership, so adjacent tiles repeat halo/boundary samples. Earlier measurement put this duplication around 18%. It existed in the old architecture too, so it did not cause the original regression, but every repeated sample is now much more expensive.
|
||||
|
||||
## 8. There are two lower-confidence ways to get a permanent hole
|
||||
|
||||
**Status: Verified risk paths; not demonstrated by this trace.**
|
||||
|
||||
First, whole-tile AllAir/AllSolid verdicts are trusted without exact validation. They become loaded `bEmpty` tiles with no component. Internal block verdicts are later checked against the full grid when meshing occurs, but an outer uniform verdict has no equivalent safety net.
|
||||
|
||||
The supplied aggregate argues against a widespread problem—every LOD0 Mixed tile produced geometry—but it cannot establish whether the floor tile beneath the pawn was incorrectly declared uniform because tile coordinates were not logged.
|
||||
|
||||
Second, `ApplyMeshToTile` has no success return. If runtime-mesh initialization fails, the function returns while the tile remains recorded as loaded, so it will not naturally retry ([VoxelWorld.cpp:3064](<E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:3064>)). I found no trace evidence that this occurred.
|
||||
|
||||
The normal `bAborted` result path is not the likely culprit: it removes the pending record without marking the tile loaded, allowing a later desired pass to reschedule it. There is, however, an editor pause corner case where a worker can exit without enqueuing any result and leave its pending entry behind if pause acquisition times out. That is a real bug, but it does not match this epoch-zero play session.
|
||||
|
||||
## What I would prove next
|
||||
|
||||
In order:
|
||||
|
||||
1. A/B the forced nested block classifier on the exact owner trace, recording outer versus nested calls, proof outcomes, stack builds and cache probes.
|
||||
2. Add collision-specific timestamps: requested, mesh submitted, collision cook started/completed, component registered, plus tile and pawn coordinates.
|
||||
3. Force exact meshing/validation for every supposedly uniform LOD0 tile in the spawn neighborhood to rule a false classifier proof in or out.
|
||||
4. Benchmark a compiled/fused evaluator that retains the authored operator graph but restores per-sample short-circuiting and avoids spilling `FVoxelOpSample` through memory for every operator.
|
||||
5. Prioritize a small LOD0 support neighborhood and require collision-cook completion before releasing the pawn. Faster generation alone cannot make the current lifecycle correct.
|
||||
|
||||
The blunt version: **the operator abstraction leaked into the innermost loop, and streaming treats “work submitted/applied” as “safe to stand on.”** The classifier work then became a collection of compensators for the expensive evaluator, some useful and some apparently duplicative.
|
||||
|
||||
This was a read-only audit. No files were created, modified or deleted; `git status` remains clean on `experimental` at `aa62971`.
|
||||
@@ -0,0 +1,161 @@
|
||||
# Codex task 001 — make tile-skipping observable in the running game
|
||||
|
||||
**Owner:** Codex (Model Luna, xHigh) · **Orchestrator:** Claude · **Branch:** `experimental`
|
||||
**Status:** specified, not started
|
||||
|
||||
---
|
||||
|
||||
## Why this exists
|
||||
|
||||
Jahni built the world, looked at it, and said: *"I don't know if it dropped any meshing? but it looks
|
||||
alright by the eye."*
|
||||
|
||||
He's right to be unsure — **there is no way to answer that question from inside the game.** The
|
||||
plugin has **zero** stat counters (`grep INC_DWORD_STAT` → nothing). Tile-skipping is the largest
|
||||
perf item in the whole plan and it is currently unobservable in production; it has only ever been
|
||||
measured in an automation harness, on 40 sampled tiles.
|
||||
|
||||
And a visual check cannot answer it: *skipped correctly* and *skipped nothing* render identically.
|
||||
This codebase has paid repeatedly for exactly that confusion — see the "coverage is a number, not a
|
||||
boolean" lessons in `OPSTACK-HANDOFF.md`.
|
||||
|
||||
**The real prize:** with no strate opted in, cave-archetype skips must read **0**. After ticking
|
||||
`bUseOperatorStack` on one `TunnelNetwork` strate and flying underground, they must become non-zero.
|
||||
That is the **production-side proof of T1.d**, which does not exist today.
|
||||
|
||||
## The sites — there are TWO, and the second one is the one that answers the question
|
||||
|
||||
### Site A — the skip itself
|
||||
|
||||
`Source/VoxelForge/Private/VoxelWorld.cpp`, in **`AVoxelWorld::GenerateTileResult`** (~line 1501).
|
||||
Trust the symbol, not the line number.
|
||||
|
||||
```cpp
|
||||
bool bTrivialEmpty = false;
|
||||
if (!bSheetTile && !bWantCapture && Generator && Mesher && Mesher->IsoLevel == 0.0f)
|
||||
{
|
||||
TRACE_CPUPROFILER_EVENT_SCOPE(VoxelForge_ClassifyTile);
|
||||
bTrivialEmpty = (Generator->ClassifyTile(OriginVoxels, Step, Cells) != EVoxelTileClass::Mixed);
|
||||
}
|
||||
|
||||
FVoxelMeshData MeshData;
|
||||
if (!bTrivialEmpty)
|
||||
{
|
||||
TRACE_CPUPROFILER_EVENT_SCOPE(VoxelForge_GenerateMesh);
|
||||
MeshData = bSheetTile ? Mesher->GenerateSheetMesh(...) : Mesher->GenerateMesh(...);
|
||||
}
|
||||
```
|
||||
|
||||
### Site B — where the OPERATOR STACK's verdict is produced
|
||||
|
||||
`Source/VoxelForge/Private/VoxelGenerator.cpp`, in **`UVoxelGenerator::ClassifyTile`**, at the **exit
|
||||
of the `if (bAnyCave)` block** (~line 3009) — the last two lines of that block:
|
||||
|
||||
```cpp
|
||||
if (bCanSolid == bCanAir) { return EVoxelTileClass::Mixed; }
|
||||
return bCanSolid ? EVoxelTileClass::AllSolid : EVoxelTileClass::AllAir;
|
||||
```
|
||||
|
||||
**Why site A alone cannot answer the question — this is the correction that makes the task
|
||||
meaningful.** `ClassifyTile` has *two* independent ways to reach a non-`Mixed` verdict:
|
||||
|
||||
- the **hand-written** path, which predates all of this work: a chunk in a **bedrock gap** sets
|
||||
`bCanAir = false` (VoxelGenerator.cpp ~2835) and, absent a passage or the origin spine, the tile
|
||||
resolves **`AllSolid`**. Likewise the SurfaceWorld column scan. This fires with **no strate opted
|
||||
in at all**;
|
||||
- the **operator-stack** path, the `if (bAnyCave)` block, which is the only thing T1.d added.
|
||||
|
||||
So `TilesSkippedAllSolid` at site A **will already be non-zero underground before any strate is
|
||||
ticked** — the bedrock between strates guarantees it. A single lumped counter would make the
|
||||
before/after unreadable, and that is exactly the "when a zero has several possible causes, give each
|
||||
one its own number" lesson this project already paid for.
|
||||
|
||||
Site B's counters have the opposite property, and it is a strong one: `ClassifyTile` returns `Mixed`
|
||||
outright at the cave branch when `UsesOperatorStackForChunk(CC)` is false (~line 2809, and again per
|
||||
chunk of the box at ~2914). **With no strate opted in, the site-B counters are zero by
|
||||
construction, not merely by observation** — so a non-zero reading after ticking the box cannot come
|
||||
from anywhere else.
|
||||
|
||||
## What to build
|
||||
|
||||
1. **A stat group.** New header `Source/VoxelForge/Public/VoxelStats.h`:
|
||||
`DECLARE_STATS_GROUP(TEXT("VoxelForge"), STATGROUP_VoxelForge, STATCAT_Advanced);` plus
|
||||
`DECLARE_DWORD_COUNTER_STAT_EXTERN` for each counter below. `DEFINE_STAT` for each goes in **one**
|
||||
`.cpp` — put them in a new `Source/VoxelForge/Private/VoxelStats.cpp`.
|
||||
|
||||
2. **Six per-frame counters** (`DWORD_COUNTER`, so `stat VoxelForge` shows a rate, not a total):
|
||||
|
||||
| counter | site | incremented when |
|
||||
|---|---|---|
|
||||
| `TilesClassified` | A | the classifier gate was entered (the `if` above ran `ClassifyTile`) |
|
||||
| `TilesSkippedAllSolid` | A | verdict was `AllSolid` |
|
||||
| `TilesSkippedAllAir` | A | verdict was `AllAir` |
|
||||
| `TilesMeshed` | A | `GenerateMesh` / `GenerateSheetMesh` actually ran |
|
||||
| `TilesOpStackSolid` | B | the `bAnyCave` block returned `AllSolid` |
|
||||
| `TilesOpStackAir` | B | the `bAnyCave` block returned `AllAir` |
|
||||
|
||||
Splitting solid from air is the point, not decoration: **cave archetypes prove `AllSolid`**.
|
||||
Splitting site B from site A is the whole deliverable — see "Why site A alone cannot answer the
|
||||
question" above. `TilesOpStackSolid ≤ TilesSkippedAllSolid` always, and the difference is the
|
||||
pre-existing bedrock/surface skipping.
|
||||
|
||||
At site B, increment on the `return` line only — **not** before the
|
||||
`if (bCanSolid == bCanAir) return Mixed;` guard, which is where the block bails out with no
|
||||
verdict.
|
||||
|
||||
3. To get the verdict you need it as a value, not a bool. Changing
|
||||
`bTrivialEmpty = (Classify(...) != Mixed)` into a stored `EVoxelTileClass Verdict = Classify(...)`
|
||||
followed by `bTrivialEmpty = (Verdict != Mixed)` is **fine and expected**.
|
||||
|
||||
## ⚠️ Invariants — a violation here is not a bug, it is a hole
|
||||
|
||||
1. **DO NOT change `bTrivialEmpty`'s value or the control flow.** That bool decides whether a tile
|
||||
gets geometry **and collision**. A wrong value is invisible until a player falls through the
|
||||
floor. Refactor the expression, never the condition.
|
||||
2. **DO NOT touch the gate `!bSheetTile && !bWantCapture && Generator && Mesher && Mesher->IsoLevel
|
||||
== 0.0f`.** Every clause is load-bearing and documented in the comment above it — sheet tiles have
|
||||
no marching cubes, capture tiles need the grid even when uniform, and the verdicts assume the MC
|
||||
iso is exactly zero.
|
||||
3. **Thread safety: this runs on WORKERS.** `GenerateTileResult` is called from the async ChunkGen
|
||||
task *and* the synchronous carve path. Use the `INC_DWORD_STAT` family, which is per-thread-packet
|
||||
safe. **A plain `static int32` counter, even `++` on an `int32`, is a data race — do not.**
|
||||
4. **Zero cost when stats are compiled out.** The `INC_DWORD_STAT` macros already vanish when
|
||||
`STATS == 0`. Do not wrap them in an `if` that survives, and do not compute anything solely to
|
||||
feed a counter outside the macro.
|
||||
5. **No new includes in a public header beyond `Stats/Stats.h`**; the plugin follows IWYU and the
|
||||
include debt was cleared deliberately (`AUDIT §C9` work).
|
||||
6. **At site B, do not touch `ClassifyTile`'s control flow either — and do not add an early
|
||||
`return`.** That function is a chain of conservative guards that all **fail to `Mixed`**; every
|
||||
`return` in it is load-bearing. Add the counter to the existing `return` expression's statement,
|
||||
nothing else. `ClassifyTile` is `const` and runs on the same workers as site A, so the same
|
||||
`INC_DWORD_STAT`-not-`static int32` rule applies.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- Editor, `stat VoxelForge` on screen, fly around: the numbers move.
|
||||
- **`TilesClassified == TilesSkippedAllSolid + TilesSkippedAllAir + TilesMeshed`** for tiles that
|
||||
entered the gate. (Tiles that fail the gate are meshed without being classified, so `TilesMeshed`
|
||||
is legitimately larger than the classified total — say so in a comment rather than "fixing" it.)
|
||||
- **`TilesOpStackSolid ≤ TilesSkippedAllSolid`** and **`TilesOpStackAir ≤ TilesSkippedAllAir`**,
|
||||
always. A violation means site B is counting a verdict that site A did not act on.
|
||||
- With **no strate opted in**, flying underground through a `TunnelNetwork` strate:
|
||||
- `TilesSkippedAllSolid` is expected to be **non-zero** — that is the pre-existing bedrock/surface
|
||||
skipping, not a bug, and it is why the lumped counter cannot be the deliverable;
|
||||
- `TilesOpStackSolid` and `TilesOpStackAir` are **0**. This is the baseline, and it must be
|
||||
*observed* before the next step even though it is guaranteed by the flag gate.
|
||||
- Tick `bUseOperatorStack` on **one** `TunnelNetwork` strate, fly the same route:
|
||||
**`TilesOpStackSolid` becomes non-zero.** ← this is the deliverable, and it is the first
|
||||
production-side evidence T1.d has ever had.
|
||||
|
||||
## Notes for the reviewer (Claude)
|
||||
|
||||
- Check the verdict refactor byte-for-byte against the original condition. `!= Mixed` is the whole
|
||||
contract.
|
||||
- Check the counters are `DWORD_COUNTER` (per-frame) and not `DWORD_ACCUMULATOR`.
|
||||
- Confirm no counter is incremented outside the gate in a way that double-counts the carve path,
|
||||
which calls `GenerateTileResult` synchronously from the game thread.
|
||||
- Site B: confirm the increment sits **after** the `bCanSolid == bCanAir` bail-out, and that the two
|
||||
counters follow `bCanSolid` the same way the returned enum does — a swapped pair reads as a
|
||||
plausible result and proves the wrong thing.
|
||||
- The automation tests call `ClassifyTile` directly; they will move the site-B counters. Harmless,
|
||||
but do not let a test-only path become the only thing that moves them.
|
||||
@@ -0,0 +1,129 @@
|
||||
# Codex task 002 — is the op-stack column memo thrashing? Count, don't guess.
|
||||
|
||||
**Owner:** Codex (Model Luna, xHigh) · **Orchestrator:** Claude · **Branch:** `experimental`
|
||||
**Status:** specified, not started
|
||||
**Depends on:** `CODEX-TASK-001` — this adds counters to the **same** `stat VoxelForge` group.
|
||||
Do 001 first; this task assumes `VoxelStats.h` already exists.
|
||||
|
||||
---
|
||||
|
||||
## Why this exists
|
||||
|
||||
The operator-stack density path is measurably slower than the `switch` it replaces, and the cause
|
||||
has never been attributed. There are three standing suspects. **This task measures the first one,
|
||||
and does not fix anything.** That order is deliberate: `AUDIT §C10` cost six builds and five refuted
|
||||
hypotheses by reasoning first, and the last session re-learned it.
|
||||
|
||||
### The hypothesis, derived from the code
|
||||
|
||||
`FSurfaceColumnSource::GetColumn` (`VoxelDensityOpStack.cpp` ~615) memoises a computed column in a
|
||||
**direct-mapped, 4096-entry `thread_local` table**, indexed by a hash of the two XY floats:
|
||||
|
||||
```cpp
|
||||
struct FSlot { uint64 Key; float X, Y; FColumn C; }; // 40 bytes
|
||||
thread_local FSlot Slots[4096] = {}; // 160 KB per worker
|
||||
const uint32 Idx = ((HX * 0x9E3779B9u) ^ (HY * 0x85EBCA6Bu)) >> 20; // [0,4095]
|
||||
```
|
||||
|
||||
The original path, `GSurfColCache` in `GetDensityAt`, is instead a **direct-indexed box**:
|
||||
`CI = (IY - Box.BaseY) * Dim + (IX - Box.BaseX)`, with a `Computed[CI]` flag. **No hash, therefore
|
||||
no collisions, therefore every column is computed exactly once.**
|
||||
|
||||
Now the sampling order, which is the premise that makes this bite. `FVoxelMarchingCubesMesher`
|
||||
pre-samples with **Z as the OUTERMOST loop** (`VoxelMarchingCubesMesher.cpp` ~226):
|
||||
|
||||
```cpp
|
||||
for (int32 gz = GzLo; gz <= GzHi; gz++)
|
||||
for (int32 gy = -1; gy <= GridDim; gy++)
|
||||
for (int32 gx = -1; gx <= GridDim; gx++)
|
||||
Generator->GetDensityAt(...);
|
||||
```
|
||||
|
||||
So the mesher sweeps a **whole XY plane at every Z level**. Every column in the tile is revisited
|
||||
once per Z plane — roughly 34 times.
|
||||
|
||||
The table's own comment sized it for this: *"A chunk is CHUNK_SIZE² columns (1024), so the first
|
||||
draft's 256 entries could not even hold one chunk and thrashed inside a single tile. 4096 covers
|
||||
four chunks."* **That reasoning has a gap.** A direct-mapped table does not need to be full to
|
||||
evict — it needs two live keys to collide. At ~1156 columns per plane in 4096 slots (load factor
|
||||
0.28), the expected number of columns sharing a slot with another is **~285, about 25 %**. Those
|
||||
columns evict each other, miss again on the next Z plane, and recompute the **entire height stack** —
|
||||
structural source, cliff (four structural resamples), terrace, layer-line, beach, and the ceiling
|
||||
stack — every single plane.
|
||||
|
||||
Order of magnitude if that is right: ~1156 column computations on the original path versus
|
||||
~1156 + 285 × 34 ≈ **10 000** on the op path. Roughly **9×** the column work, on the plugin's most
|
||||
expensive archetype.
|
||||
|
||||
**That is a derivation, not a measurement, and it is exactly the kind of confident chain this
|
||||
project has watched reverse six times.** Hence: count first.
|
||||
|
||||
## What to build — two counters, no behaviour change
|
||||
|
||||
Add to the existing `stat VoxelForge` group from task 001:
|
||||
|
||||
| counter | incremented when |
|
||||
|---|---|
|
||||
| `ColumnMemoHit` | `GetColumn` found a live entry (the `if` body did **not** run) |
|
||||
| `ColumnMemoMiss` | `GetColumn` recomputed (the `if` body ran) |
|
||||
|
||||
That is the entire change. Two `INC_DWORD_STAT` calls inside `FSurfaceColumnSource::GetColumn`,
|
||||
around the existing `if (S.Key != ColumnKey || S.X != WorldX || S.Y != WorldY)`.
|
||||
|
||||
## ⚠️ Invariants
|
||||
|
||||
1. **Change nothing else in `GetColumn`.** Not the table size, not the hash, not the key comparison.
|
||||
The point of this task is to produce a number that decides whether the fix is worth writing; a
|
||||
change bundled in makes the number unattributable. **Do not "obviously improve" the table while
|
||||
you are in there** — if the fix ships in the same build as the instrument, we learn nothing, and
|
||||
this project has a written lesson about exactly that.
|
||||
2. **`GetColumn` is `const` and runs on mesher workers.** `INC_DWORD_STAT`, never a `static int32++`
|
||||
— same rule as task 001, same reason.
|
||||
3. **The key comparison is load-bearing and must stay complete.** `S.Key != ColumnKey || S.X != X ||
|
||||
S.Y != Y` — the full key is compared on every touch precisely so a hash collision can only cost a
|
||||
recompute and never return **someone else's column**. Do not shorten it to feed a counter.
|
||||
4. **Zero cost when `STATS == 0`.** No value computed outside the macros.
|
||||
5. `FCaveCliffMod` and the overhang read this same memo through
|
||||
`Column->GetColumn(...)` (~line 864). They are legitimate traffic and must be counted, not
|
||||
excluded — they are part of why a miss is expensive.
|
||||
|
||||
## Acceptance — the prediction is now numeric (tightened 2026-08-16 from the real grid dimensions)
|
||||
|
||||
The hand-wavy "20–30 %" band this section used to carry has been replaced by an arithmetic
|
||||
prediction, because the inputs are all statically knowable and were read out of the source:
|
||||
|
||||
- `CHUNK_SIZE = 32`, `CellsPerAxis = 32`, `GridDim = 33`, and the pre-sample loops run
|
||||
`g ∈ [-1, GridDim]` per axis ⇒ **35 × 35 = 1225 distinct columns per tile**, over **35 Z planes**
|
||||
(the mesher's own buffer comment, "35³ floats", confirms the dimension).
|
||||
- 1225 keys in 4096 slots is a load factor of **0.299**. Expected slots holding exactly one key
|
||||
`= 4096 · np(1-p)^(n-1) ≈ 908`, so **~317 columns (25.9 %) share a slot with another** and evict
|
||||
each other on every plane.
|
||||
- ⇒ op path ≈ `1225 + 34 × 317` ≈ **12 000** column computations per tile.
|
||||
Original path (`GSurfColCache`, direct-indexed, `Computed[CI]` persists) = **1225**.
|
||||
**≈ 9.8×.**
|
||||
|
||||
### How to read the result
|
||||
|
||||
⚠️ **Compare the RATIO OF THE TWO HYPOTHESES, not an absolute percentage.** The overhang and cliff
|
||||
modifiers call `GetColumn` again at the same XY (~line 864); every extra consumer adds **hits** and
|
||||
no misses, so it inflates the denominator and drags the miss *rate* down without changing the
|
||||
verdict. What does not move is the ~10× gap between the two outcomes.
|
||||
|
||||
| observation | verdict |
|
||||
|---|---|
|
||||
| misses ≈ **8–10×** the hit-path baseline (single-consumer: ~28 % of lookups) | **CONFIRMED** — the table evicts on collision every Z plane. The fix gets its own task, with this run as its "before". |
|
||||
| misses ≈ **1 per distinct column** (single-consumer: ~3 %, hit rate ≥ 97 %) | **hypothesis WRONG.** The table behaves like the box, the ~9.8× does not exist, and the perf cost is suspect 2 (19 virtual calls per voxel). |
|
||||
|
||||
*A negative result here is a real result.* It retires the most-suspected cause and is worth the build
|
||||
either way; it must be written into `OPSTACK-PROGRESS.md`, not quietly dropped.
|
||||
|
||||
Report **both raw numbers**, never the ratio alone — a ratio cannot distinguish "few lookups" from
|
||||
"many", and the absolute miss count is what the fix would be reducing.
|
||||
|
||||
## Notes for the reviewer (Claude)
|
||||
|
||||
- Confirm the miss counter sits inside the `if`, and the hit counter in an `else` — not computed
|
||||
from a subtraction, which would silently agree with itself.
|
||||
- Confirm `GetColumn`'s early-out path (if any is added later) cannot skip both counters.
|
||||
- Confirm nothing else in `VoxelDensityOpStack.cpp` changed. `git diff --stat` should show one file
|
||||
and a handful of lines.
|
||||
@@ -0,0 +1,109 @@
|
||||
# Codex task 003 — three `ExtraReach` formulas use an FBM bound this file already proved wrong
|
||||
|
||||
**Owner:** Codex (Model Luna, xHigh) · **Orchestrator:** Claude · **Branch:** `experimental`
|
||||
**Status:** specified, not started
|
||||
**Kind:** ⚠️ **correctness of a box verdict** — the class of bug that deletes collision. Not a perf task.
|
||||
|
||||
---
|
||||
|
||||
## Why this exists
|
||||
|
||||
`VoxelDensityOpStack.cpp` contains a rigorous, written derivation that `|Perlin3D| ≤ 1.5`, exposes it
|
||||
as `PerlinAbsBound` (line ~2336), and uses it correctly for the tunnel warp dilation (~2452). The
|
||||
comment there is explicit that the loose "~[-1,1]" figure from the noise header is **not** to be
|
||||
relied on, and `OPSTACK-HANDOFF.md` records the standard: *a bound in a box verdict must be PROVED,
|
||||
not observed — over-estimating costs CPU, under-estimating deletes collision.*
|
||||
|
||||
**Three `ExtraReach` formulas in the same file silently assume `sup|FBM| ≤ 1.0`.** Each carries the
|
||||
comment "FBM ∈ [-1,1]", which is exactly the claim the file disproves 1800 lines earlier.
|
||||
|
||||
And `VoxelNoise::FBM` **is normalised** — it returns `Total / MaxValue` where `MaxValue = Σ Amp`
|
||||
(`VoxelNoise.h` ~272). So `sup|FBM| = sup|Perlin3D|` exactly: **1.5, not 1.0.** The octave sum
|
||||
neither amplifies nor attenuates the bound.
|
||||
|
||||
### What that costs, per archetype, at the shipped defaults
|
||||
|
||||
`Identity` from these sources means "no primitive within `ExtraReach` of the box", i.e. `Sdf ≥
|
||||
ExtraReach` throughout. Roughness then does `Sdf += FBM · VOXEL_NOISE_SCALE · Strength`
|
||||
(`FSdfRoughnessMod::Eval`), so worst case `Sdf' ≥ ExtraReach − B·1.25·|Roughness|`. Soundness
|
||||
requires `Sdf'` to stay at or above the downstream carve/fill threshold.
|
||||
|
||||
| archetype | `ExtraReach` at defaults | downstream threshold | needs `B ≤` | verdict at `B = 1.5` |
|
||||
|---|---|---|---|---|
|
||||
| **VerticalShafts** (`Rough 3.0`) | `1.25·3 + 2 + 1` = **6.75** | carve blend **2.0** | **1.27** | ⛔ **UNSOUND** (margin −0.875) |
|
||||
| **Maze** (`Rough 2.0`) | `1.25·2 + 2 + 1` = **5.5** | carve blend **2.0** | **1.40** | ⛔ **UNSOUND** (margin −0.25) |
|
||||
| **FloatingIslands** (`Rough 4.0`, `K 5.0`) | `1.25·4 + 2·5 + 1` = **16.0** | fill `K` **5.0** (+ `K/6` SmoothMin dip) | **2.03** | ✅ sound — but only because `K` is large. Sound by parameter luck, not by construction. |
|
||||
|
||||
Break-even roughness for the two carve archetypes is `|Rough| ≤ 1.6`; they ship at 3.0 and 2.0.
|
||||
|
||||
**How alarmed to be, stated honestly.** No strate has `bUseOperatorStack` ticked, so nothing in the
|
||||
running game is affected today. The brute-force tile scans report 0 violations — but they *sample*,
|
||||
and they were sampling against a shaft source that proved **zero** tiles until `e002bd4`, so the
|
||||
shaft path has never been exercised at all. The empirical sup of this Perlin is estimated at
|
||||
~1.0–1.1, which is *below* the 1.27 the shafts need — which is why nothing has been seen yet, and
|
||||
also why the margin is uncomfortably thin. The bug is that the verdict rests on an unproved bound,
|
||||
which is the thing this codebase has already decided it does not do.
|
||||
|
||||
## The fix
|
||||
|
||||
1. **Hoist `PerlinAbsBound` to file scope and rename it `VF_PerlinAbsBound`**, so there is **one**
|
||||
definition rather than a class-static plus three implicit `1.0`s. Keep the existing derivation
|
||||
comment with it — it is the justification, not decoration.
|
||||
|
||||
**Naming, resolved:** the file-scope helpers in this file are all `VF_`-prefixed
|
||||
(`VF_NearCaveSurface`, `VF_DistPointSegment`, `VF_NoCaveOverBox`), so a file-scope constant takes
|
||||
the same prefix. That means this is a **rename**, not just a move:
|
||||
- delete the `static constexpr float PerlinAbsBound = 1.5f;` class-static inside `FRoomGraphSource`
|
||||
(~2336), moving its whole derivation comment with it;
|
||||
- **update `FRoomGraphSource`'s own use at ~2452** (`P.CaveWarpStrength * VOXEL_NOISE_SCALE *
|
||||
PerlinAbsBound`) to the new name. This is the one place where the warp dilation is computed and
|
||||
it must keep computing the identical value — the rename must not change its arithmetic.
|
||||
- after the edit, `grep -n "PerlinAbsBound" ` must show **only** `VF_PerlinAbsBound` occurrences.
|
||||
2. **Multiply the roughness term by it in all three `ExtraReach` formulas** (~4106 VerticalShafts,
|
||||
~4215 FloatingIslands, ~4247 Maze):
|
||||
|
||||
```cpp
|
||||
// before
|
||||
FMath::Abs(P.SurfaceRoughness) * VOXEL_NOISE_SCALE
|
||||
// after
|
||||
FMath::Abs(P.SurfaceRoughness) * VOXEL_NOISE_SCALE * VF_PerlinAbsBound
|
||||
```
|
||||
3. **Fix the three comments.** Each says "FBM ∈ [-1,1]". Replace with the real statement: `FBM` is
|
||||
normalised (`Total / MaxValue`), so `sup|FBM| = sup|Perlin3D| =` the proved `PerlinAbsBound`.
|
||||
A comment that states a refuted bound is how this happened in the first place.
|
||||
|
||||
## ⚠️ Invariants
|
||||
|
||||
1. **This must not change density by one bit.** `ExtraReach` is read **only** inside
|
||||
`EffectOverBox` (verified: every other occurrence is a comment or the `float ExtraReach;` member
|
||||
declaration — no `Eval`, no `GetCells`). The eight equivalence tests compare `Eval` bit for bit
|
||||
and must stay green. **If you find yourself editing an `Eval`, stop — you have the wrong site.**
|
||||
2. **The change direction is strictly conservative**: larger `ExtraReach` ⇒ more `CarveOnly`, fewer
|
||||
`Identity` ⇒ *fewer* tiles proved uniform. It can only cost CPU, never open a hole. Do not
|
||||
"balance" it by tightening something else in the same edit.
|
||||
3. **Anonymous-namespace placement.** Put the hoisted constant **above the labelled end of the
|
||||
anonymous namespace**, not anchored on the FACTORIES banner — anchoring there puts it outside and
|
||||
the brace added with it closes nothing. This mistake has been made twice in this file and the
|
||||
file says so.
|
||||
4. **`FRoomGraphSource`'s warp dilation changes NAME ONLY.** It already uses the bound correctly and
|
||||
is the reference implementation for this fix; the value it computes must be bit-identical after
|
||||
the rename. Do not alter its formula, its `√2` factor, or anything else in that function.
|
||||
5. Comments are French + English; match the surrounding file.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- `git diff --stat` shows **one** file: `Source/VoxelForge/Private/VoxelDensityOpStack.cpp`.
|
||||
- All three `ExtraReach` definitions include the bound; no fourth site exists (`BuildTunnelNetworkStack`
|
||||
has no `ExtraReach` — it uses `PerlinAbsBound` directly for the warp).
|
||||
- No `Eval` body changed.
|
||||
- After the build: the eight equivalence tests stay green (density unchanged), and the box-verdict
|
||||
lines for **Maze** and **VerticalShafts** may report *fewer* proved tiles than before. **A drop
|
||||
there is the expected, correct outcome, not a regression** — it is the cost of a sound bound.
|
||||
Record the before/after in `OPSTACK-PROGRESS.md`.
|
||||
|
||||
## Notes for the reviewer (Claude)
|
||||
|
||||
- Confirm the constant is genuinely at file scope inside the anonymous namespace and that the
|
||||
class-static is gone, not shadowed — two definitions that can drift is the failure this fixes.
|
||||
- Confirm all three call sites got it. Two out of three is worse than none, because it looks done.
|
||||
- Confirm no `Eval`, `GetCells`, or `GetCellsAt` body appears in the diff.
|
||||
@@ -0,0 +1,101 @@
|
||||
# Codex task 004 — two box verdicts assume `MinRadius ≤ MaxRadius`; a third one already doesn't
|
||||
|
||||
**Owner:** Codex (Model Luna, xHigh) · **Orchestrator:** Claude · **Branch:** `experimental`
|
||||
**Status:** specified, not started
|
||||
**Kind:** ⚠️ **correctness of a box verdict.** Same class as task 003. Small fix, closes a class.
|
||||
|
||||
---
|
||||
|
||||
## Why this exists
|
||||
|
||||
Three operators roll a primitive radius from a hash between two designer-set params:
|
||||
|
||||
```cpp
|
||||
Out.R = FMath::Lerp(MinRadius, MaxRadius, hash01); // FGridColumnMod ~1129
|
||||
Out.R = FMath::Lerp(P.ShaftMinRadius, P.ShaftMaxRadius, hash01); // FShaftFieldSource ~1560
|
||||
Out.Rxy = FMath::Lerp(P.IslandMinRadius, P.IslandMaxRadius, hash01); // FIslandBlobSource ~1850
|
||||
```
|
||||
|
||||
`FMath::Lerp(A, B, t)` with `t ∈ [0,1]` lands anywhere in `[min(A,B), max(A,B)]` — it does **not**
|
||||
require `A ≤ B`.
|
||||
|
||||
Each op's `EffectOverBox` then sweeps a **range of lattice cells** around the query box, padded by
|
||||
the largest radius a cell could hold, and tests each rolled primitive exactly. The pad decides which
|
||||
cells are *looked at at all*, so a pad smaller than the true maximum radius means **cells are never
|
||||
examined**, their primitives are never tested, and the op reports `Identity` for a box that its own
|
||||
`Eval` will carve or fill.
|
||||
|
||||
| op | pad used for the cell sweep | correct? |
|
||||
|---|---|---|
|
||||
| `FGridColumnMod` ~1086 | `FMath::Max(MaxRadius, 0.0f) + ColBlend` | ⛔ **exposed** |
|
||||
| `FShaftFieldSource` ~1436 | `FMath::Max(P.ShaftMaxRadius, P.ConnectorRadius) + ExtraReach` | ⛔ **exposed** |
|
||||
| `FIslandBlobSource` ~1803 | `const float MaxR = FMath::Max(P.IslandMinRadius, P.IslandMaxRadius);` | ✅ **already correct** |
|
||||
|
||||
**The third one is the point.** Someone hit this exact concern while writing the island source and
|
||||
guarded it. The other two shipped without the guard. This task makes the three consistent.
|
||||
|
||||
## How exploitable, stated honestly
|
||||
|
||||
The shipped defaults are correctly ordered (`2/5`, `2/7`, `5/11`), so **nothing is broken out of the
|
||||
box.** It needs a mis-ordered asset value — `ColumnMinRadius = 8, ColumnMaxRadius = 4`.
|
||||
|
||||
Nothing prevents that. The `UPROPERTY` metas carry `ClampMin = "1.0"`, which is a per-property
|
||||
floor; Unreal has no declarative way to say "must be ≤ that other property". And `ColumnMinRadius`
|
||||
is *also* settable per-room through `UVoxelTerrainOpDefinition`, so it is not only the strate asset.
|
||||
|
||||
What makes it worth the five lines: when it does happen, the failure is **invisible and maddening**.
|
||||
`Eval` still draws the fat column perfectly, so every tile that gets meshed looks correct; only the
|
||||
tiles the classifier *skipped* are missing — no geometry, no collision, in a world that otherwise
|
||||
looks right.
|
||||
|
||||
## The fix — five lines, and one thing you must NOT do
|
||||
|
||||
Make each pad use the true envelope:
|
||||
|
||||
```cpp
|
||||
// FGridColumnMod ~1086
|
||||
const float Reach = FMath::Max3(MinRadius, MaxRadius, 0.0f) + ColBlend;
|
||||
|
||||
// FShaftFieldSource ~1436
|
||||
const float Pad = FMath::Max3(P.ShaftMinRadius, P.ShaftMaxRadius, P.ConnectorRadius) + ExtraReach;
|
||||
```
|
||||
|
||||
(Use whatever spelling is idiomatic here — check that `FMath::Max3` is already used in this codebase
|
||||
before reaching for it; nested `FMath::Max` is fine and matches `FIslandBlobSource`'s existing line.)
|
||||
|
||||
### ⛔ DO NOT "fix it properly" by normalising the params
|
||||
|
||||
The tempting larger fix — swap `Min`/`Max` at resolution time so `Min ≤ Max` always — is **wrong and
|
||||
will break the build's tests.** `Eval` computes `Lerp(Min, Max, t)`; swapping the endpoints maps the
|
||||
same hash `t` to a *different* radius for the same cell. That changes generated geometry and breaks
|
||||
the eight bit-for-bit equivalence tests against the `switch` path.
|
||||
|
||||
**Only the BOUND may become conservative. `Eval` stays byte-identical.** This is the same rule as
|
||||
task 003 and the same reason.
|
||||
|
||||
## ⚠️ Invariants
|
||||
|
||||
1. **No `Eval`, `RollColumn`, `RollShaft`, `GetCells`, or `GetCellsAt` body may change.** If your
|
||||
diff touches one, you have the wrong site — stop and say so.
|
||||
2. **Do not touch `FIslandBlobSource`.** It is already correct and is the reference for this fix.
|
||||
3. The change direction is strictly conservative: a wider sweep examines *more* cells, so a verdict
|
||||
can only move from `Identity` toward `CarveOnly`/`FillOnly`, never the reverse. Do not add any
|
||||
compensating tightening.
|
||||
4. Comments are French + English; match the surrounding file. Say **why** the envelope is
|
||||
`max(Min, Max)` and not `Max` — the next reader must not "simplify" it back.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- `git diff --stat` shows exactly one file: `Source/VoxelForge/Private/VoxelDensityOpStack.cpp`,
|
||||
and a handful of lines.
|
||||
- The three ops now agree on the pattern.
|
||||
- After the build: **every box-verdict line and every equivalence test is unchanged**, because the
|
||||
shipped defaults are correctly ordered and the envelope only differs when they are not. **A change
|
||||
in any of those numbers means the diff did something it should not have.** That is this task's
|
||||
whole acceptance signal — a *no-op at defaults* is the expected, correct result.
|
||||
|
||||
## Notes for the reviewer (Claude)
|
||||
|
||||
- Confirm both pads changed and `FIslandBlobSource` did not.
|
||||
- Confirm no `Lerp` argument order was touched anywhere — that is the failure mode that would look
|
||||
like a tidy-up and silently change the world.
|
||||
@@ -0,0 +1,99 @@
|
||||
# Codex task 005 — three box-verdict tile scans sample less than one lattice period
|
||||
|
||||
**Owner:** Codex (Model Luna, xHigh) · **Orchestrator:** Claude · **Branch:** `experimental`
|
||||
**Status:** specified, not started
|
||||
**Kind:** test coverage. **Zero risk to the game — no non-test file may change.**
|
||||
|
||||
---
|
||||
|
||||
## Why this exists
|
||||
|
||||
This project has already found and fixed this exact defect **twice**:
|
||||
|
||||
> *"A sampler must cover at least one period of what it samples. The tunnel test drew tile XY from
|
||||
> ±32 voxels with `RoomSpacing 80` — it measured the spine hub and called it the world. The shaft
|
||||
> test had the identical bug (±48 against `ShaftSpacing 55`)."* — `OPSTACK-HANDOFF.md`
|
||||
|
||||
Both were fixed to `SpanCells = 55` ⇒ **±440 voxels**, and both now print their own extent in units
|
||||
of the pattern's period so it cannot silently regress.
|
||||
|
||||
**Three box-verdict tile scans were never fixed**, because the fix was applied where the bug was
|
||||
noticed rather than to the class. All three still use the original `Rng.RandRange(-6, 6) * Extent`
|
||||
with `Step = 1, Cells = 8` ⇒ `Extent = 8` ⇒ **half-extent ±48 voxels**:
|
||||
|
||||
| test | half-extent | lattice period (default, **unchanged by the fixture**) | coverage |
|
||||
|---|---|---|---|
|
||||
| `VoxelForgeOpStackIslandTest.cpp` ~244 | ±48 | `IslandSpacing` **95** | **0.51 periods** ⛔ worse than either bug already fixed |
|
||||
| `VoxelForgeOpStackSlabTest.cpp` ~289 | ±48 | `ColumnSpacing` **60** | **0.80 periods** ⛔ |
|
||||
| `VoxelForgeOpStackMazeTest.cpp` ~276 | ±48 | `CellSize` **40** | 1.20 periods ⚠️ marginal |
|
||||
|
||||
Verified: none of `EnableIslandFeatures` / the slab tuning / the maze setup overrides the spacing, so
|
||||
the header defaults are what these tests actually run against.
|
||||
|
||||
**Why it matters right now, specifically.** Two commits just changed the box verdicts these very
|
||||
tests are supposed to guard — `7dbdf51` (the `ExtraReach` bound, which touches **islands**, maze and
|
||||
shafts) and `eaa44bf` (the radius envelope, which touches **slab columns** and shafts). The tests
|
||||
that would catch a mistake in those changes currently sample about half a lattice cell.
|
||||
|
||||
## The rule the two fixed tests already encode
|
||||
|
||||
`Extent = Step * Cells = 8` voxels, so `SpanVoxels = SpanCells * 8`. To get **8 periods** of
|
||||
half-extent you set:
|
||||
|
||||
> **`SpanCells` = the lattice spacing** (`55` for `ShaftSpacing 55` — that is where the shaft test's
|
||||
> `55` comes from, and it is not a coincidence).
|
||||
|
||||
Apply the same:
|
||||
|
||||
| test | `SpanCells` | resulting half-extent | periods |
|
||||
|---|---|---|---|
|
||||
| Island | `95` | ±760 | 8.0 |
|
||||
| Slab | `60` | ±480 | 8.0 |
|
||||
| Maze | `40` | ±320 | 8.0 |
|
||||
|
||||
## What to build
|
||||
|
||||
For each of the three tests, mirror **exactly** what `VoxelForgeOpStackShaftTest.cpp` (~212) does:
|
||||
|
||||
1. Hoist `const int32 SpanCells` and `const int32 SpanVoxels = SpanCells * 8;` **outside the tile
|
||||
loop** — the report needs them and `Extent` is loop-local. (That scoping slip has already happened
|
||||
once in this file family; the shaft test's comment records it.)
|
||||
2. Draw `Rng.RandRange(-SpanCells, SpanCells) * Extent` for X and Y. **Leave the Z draw exactly as
|
||||
it is** — it is clamped to the strate slot and is not part of this defect.
|
||||
3. Extend the existing report line to print `SpanVoxels`, the live ratio
|
||||
`(float)SpanVoxels / FMath::Max(<the spacing param>, 1.0f)`, and the spacing itself — so the
|
||||
extent is stated in units of the pattern's own period and a future narrowing is visible.
|
||||
|
||||
## ⚠️ Invariants
|
||||
|
||||
1. **This is NOT "widen until it passes."** The comment already in the tunnel test says it best and
|
||||
the same reasoning applies here: *every proved tile is still brute-forced voxel by voxel below, so
|
||||
a wider sampler that produced a FALSE verdict fails exactly as before. We are changing what the
|
||||
measurement **looks at**, not what it **demands**.* Do not touch the brute-force loop, its
|
||||
tolerance, or any `AddError`.
|
||||
2. **Do not adjust an assertion to accommodate a moved number.** Widening will change the proved /
|
||||
Mixed counts — that is the point. If an existing assertion would now fail, **report it and stop**;
|
||||
do not retune it. ("Don't assert a number you want to improve" is a written lesson here.)
|
||||
3. **No file outside `Source/VoxelForge/Private/Tests/` may change.** `git diff --stat` must list
|
||||
only those three test files.
|
||||
4. Do not change `Step`, `Cells`, the tile count (`60`), or the RNG seeds — a changed seed makes the
|
||||
before/after incomparable, and comparability is the whole point of touching this now.
|
||||
5. Comments are French + English; match the surrounding file.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- Three test files changed, nothing else.
|
||||
- Each of the three now prints its extent **and** that extent in periods of its own spacing param.
|
||||
- After the build, each ratio line reads **≥ 8 periods**.
|
||||
- The proved counts will move. **That is expected.** What must NOT move: `violations` / `NumUnsound`
|
||||
stays **0** in all three. If it becomes non-zero, the wider sampler has found a genuine hole that
|
||||
the narrow one was hiding — which would be this task paying for itself immediately, and must be
|
||||
reported loudly rather than tuned away.
|
||||
|
||||
## Notes for the reviewer (Claude)
|
||||
|
||||
- Confirm `SpanCells`/`SpanVoxels` are outside the tile loop in all three.
|
||||
- Confirm the Z draw is untouched.
|
||||
- Confirm the ratio is computed **live** from the params struct, not hardcoded — a hardcoded "8.0
|
||||
periods" in a format string would be a success message that asserts coverage while measuring
|
||||
nothing, which is a named failure mode in this project.
|
||||
@@ -0,0 +1,98 @@
|
||||
# Codex task 006 — the same `Min > Max` under-bound, in `BuildChunkCache` (both density paths)
|
||||
|
||||
**Owner:** Codex (Model Luna, xHigh) · **Orchestrator:** Claude · **Branch:** `experimental`
|
||||
**Status:** specified, not started
|
||||
**Kind:** ⚠️ **correctness of a collection bound.** Third instance of this class; the worst of the three.
|
||||
**Credit:** found by the Sol-High read-only audit (`AUDIT-2026-08-CODEX.md`, VF-05) and **verified
|
||||
against the code** before being specified.
|
||||
|
||||
---
|
||||
|
||||
## Why this exists
|
||||
|
||||
`CODEX-TASK-004` fixed two cell-sweep pads that assumed the field named `Max*` was numerically the
|
||||
larger one. **The same defect exists in `VoxelCaveMorphology.cpp`, and it matters more**, for three
|
||||
reasons:
|
||||
|
||||
1. It is **`TunnelNetwork`** — the largest and most-used archetype.
|
||||
2. `BuildChunkCache` is called by **both** density paths: the original `switch` *and*
|
||||
`FRoomGraphSource`, which deliberately calls it rather than transcribing it. **This is not an
|
||||
operator-stack bug — it is in the shipped original code and always has been.**
|
||||
3. Its failure mode is a **window-invariance break** (`ARCHITECTURE §8.4`), not just a missing room:
|
||||
whether a room exists depends on which chunk you queried from. In a multiplayer game that means
|
||||
two peers generate different geometry from the same seed.
|
||||
|
||||
### The mechanism
|
||||
|
||||
Radii are interpolated:
|
||||
|
||||
```cpp
|
||||
Room.RadiusXY = FMath::Lerp(Params.MinRoomRadius, Params.MaxRoomRadius, SizeFactor); // ~262
|
||||
const float RadA = FMath::Lerp(Params.TunnelMinRadius, Params.TunnelMaxRadius, FactorA); // ~467
|
||||
const float RadB = FMath::Lerp(Params.TunnelMinRadius, Params.TunnelMaxRadius, FactorB); // ~468
|
||||
```
|
||||
|
||||
`FMath::Lerp(A, B, t)` with `t ∈ [0,1]` yields anywhere in `[min(A,B), max(A,B)]` — it does **not**
|
||||
require `A ≤ B`. But every bound derived from those radii reads only the `Max*` field:
|
||||
|
||||
| site | line | expression |
|
||||
|---|---|---|
|
||||
| `MaxInfluence` | ~127–130 | `FMath::Max(Params.MaxRoomRadius, Params.TunnelWarpStrength + Params.TunnelMaxRadius) + Params.SDFBlendRadius` |
|
||||
| `CollectMargin` | ~152 | `2.0f * MaxTunnelLen + MaxInfluence` (inherits it — **no separate edit needed**) |
|
||||
| `RoomZBuffer` | ~171 | `Params.MaxRoomRadius * Params.RoomHeightRatio` |
|
||||
| `EvaluateSDF`'s `Margin` | ~871–874 | the same expression as `MaxInfluence`, duplicated |
|
||||
|
||||
With `MinRoomRadius > MaxRoomRadius`, rooms larger than `MaxInfluence` are generated, so a room that
|
||||
can reach a chunk may sit in a cell the collect region never visited. `RoomReachesSearchBox` uses the
|
||||
*actual* radius and is therefore correct — but it can only test rooms that were collected at all.
|
||||
|
||||
## The fix
|
||||
|
||||
Derive **bound-only envelopes** and use them at all four sites:
|
||||
|
||||
```cpp
|
||||
const float RoomRadiusEnvelope = FMath::Max(Params.MinRoomRadius, Params.MaxRoomRadius);
|
||||
const float TunnelRadiusEnvelope = FMath::Max(Params.TunnelMinRadius, Params.TunnelMaxRadius);
|
||||
```
|
||||
|
||||
- `MaxInfluence` → `FMath::Max(RoomRadiusEnvelope, Params.TunnelWarpStrength + TunnelRadiusEnvelope) + Params.SDFBlendRadius`
|
||||
- `RoomZBuffer` → `RoomRadiusEnvelope * Params.RoomHeightRatio`
|
||||
- `EvaluateSDF`'s `Margin` → the same corrected expression.
|
||||
|
||||
⚠️ `MaxInfluence` and `EvaluateSDF`'s `Margin` are the **same formula written twice**. They must stay
|
||||
identical. If a shared helper is natural here, use one — two copies of a rule that must agree is a
|
||||
bug factory, and this file already has the duplication. If you introduce a helper, keep it local to
|
||||
this translation unit and do not change either call site's semantics.
|
||||
|
||||
## ⛔ DO NOT reorder the `Lerp` endpoints
|
||||
|
||||
Swapping to `Lerp(min, max, t)` maps the same hash `t` to a **different radius** for the same room,
|
||||
which changes generated geometry and breaks the eight bit-for-bit equivalence tests. **Only the
|
||||
bounds may become conservative. The three `Lerp` calls must not be touched at all.**
|
||||
This is the same rule as tasks 003 and 004, and it is the third time it applies.
|
||||
|
||||
## ⚠️ Invariants
|
||||
|
||||
1. **No `Lerp` line changes. No room/tunnel placement, hashing, or `bStore` logic changes.**
|
||||
If your diff touches `Room.RadiusXY`, `RadA`, `RadB`, or `RoomReachesSearchBox`, stop and say so.
|
||||
2. The change is strictly conservative: a larger envelope collects **more** cells, never fewer.
|
||||
3. Exactly one file: `Source/VoxelForge/Private/VoxelCaveMorphology.cpp`.
|
||||
4. Comments are French + English; match the file. Say **why** the envelope is `max(Min, Max)` — the
|
||||
next reader must not "simplify" it back to `MaxRoomRadius`.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- One file changed, a handful of lines.
|
||||
- **With correctly ordered params, `max(Min, Max) == Max`, so every number is bit-identical and this
|
||||
is a NO-OP. That is the acceptance signal.** The eight equivalence tests, every box-verdict line
|
||||
and every `violations` count must be **unchanged** after the build. A moved number means the diff
|
||||
did something it should not have.
|
||||
- It only changes behaviour for an asset whose range is inverted — which is precisely the case that
|
||||
was silently producing window-dependent geometry.
|
||||
|
||||
## Notes for the reviewer (Claude)
|
||||
|
||||
- Confirm all four bound sites use the envelopes, and that `CollectMargin` inherits rather than being
|
||||
edited separately.
|
||||
- Confirm `MaxInfluence` and `EvaluateSDF`'s `Margin` still compute the identical expression.
|
||||
- Confirm no `Lerp` argument order changed anywhere in the file.
|
||||
@@ -0,0 +1,150 @@
|
||||
# Codex task 007 — VF-01: never mutate layout/passages while workers read them
|
||||
|
||||
**Owner:** Codex (Model Luna, xHigh) · **Orchestrator:** Claude · **Branch:** `experimental`
|
||||
**Status:** specified, not started
|
||||
**Kind:** ⚠️ **crash class (use-after-free).** Highest-severity item found on 2026-08-16.
|
||||
**Origin:** Sol-High audit VF-01, **independently confirmed by reading** before this spec was written.
|
||||
|
||||
---
|
||||
|
||||
## The defect
|
||||
|
||||
`UVoxelStrateManager::Initialize` does `StrateLayout.Empty()` and `Passages.Empty()` + `Passages.Add()`
|
||||
— it **frees and reallocates** both arrays. There is **no lock, no barrier, no drain** in that file.
|
||||
|
||||
Meanwhile those same arrays are read **on mesher worker threads**:
|
||||
|
||||
| reader | access |
|
||||
|---|---|
|
||||
| `AnyPassageNearBox` (`VoxelStrateManager.cpp:460`) | range-`for` over `Passages` |
|
||||
| `EvaluateModifierSDF` | indexes `Passages[...]` |
|
||||
| `FindSlotIndexForChunkZ` | iterates `StrateLayout` |
|
||||
|
||||
all reached from `GetDensityAt` / `ClassifyTile` inside chunk tasks.
|
||||
|
||||
`RegenerateAllChunks()` bumps the epoch **after** `Initialize`, so previous-epoch workers are live
|
||||
*during* the mutation. **The epoch rejects a finished result; it cannot make a read of a freed
|
||||
allocation safe.**
|
||||
|
||||
**Precedent in this very codebase:** `DiffLayer.ChunkMods` is read on mesher workers and written on
|
||||
the game thread, and all access now holds `ModsLock` — added after a real carve-vs-stream access
|
||||
violation. `StrateLayout` / `Passages` are the same shape with no guard.
|
||||
|
||||
Four `Initialize` call sites:
|
||||
|
||||
| line | function | dangerous? |
|
||||
|---|---|---|
|
||||
| 145 | `RebuildStrates` | **yes** |
|
||||
| 309 | `OnObjectModifiedInEditor` | **yes — fires automatically on a strate asset edit while streaming** |
|
||||
| 417 | `BeginPlay` | **no** — no tasks exist yet. **Leave it alone.** |
|
||||
| 2091 | `ChangeSeed` | **yes** (also writes `Generator`'s `Seed` / `OriginSpineRadius`) |
|
||||
|
||||
## Why THIS fix and not the other two
|
||||
|
||||
Rejected deliberately — do not "improve" the design into either of these:
|
||||
|
||||
- **An `FRWLock` around the two arrays** (the `ModsLock` shape) would put a **read lock on the
|
||||
per-voxel hot path** — `EvaluateModifierSDF` and `FindSlotIndexForChunkZ` run ~43k times per tile.
|
||||
There is an open, unmeasured perf regression under active investigation (`CODEX-TASK-001/002`);
|
||||
adding hot-path lock traffic now would **contaminate the very measurement those tasks exist to
|
||||
take.** Correct, but the worst possible timing.
|
||||
- **An immutable generation snapshot** (Sol's suggestion) is the right long-term architecture and a
|
||||
real refactor of `UVoxelStrateManager`'s whole API surface. Too large to improvise, and it belongs
|
||||
in a design conversation.
|
||||
|
||||
**The drain has zero hot-path cost**, reuses machinery already proven in `EndPlay`, and its only
|
||||
cost — a brief stall — lands exclusively on **human-initiated editor actions** (asset edit, rebuild,
|
||||
seed change). It never occurs during play.
|
||||
|
||||
## What to build
|
||||
|
||||
### 1. A pause flag distinct from shutdown
|
||||
|
||||
Add to `AVoxelWorld`: `std::atomic<bool> bGenerationPaused{false};`
|
||||
|
||||
⚠️ **Do NOT reuse `bShuttingDown` for this.** It would work mechanically, but it means "we are tearing
|
||||
down" and a future reader would be misled about lifetime. Introduce a small helper used at the
|
||||
existing gate points:
|
||||
|
||||
```cpp
|
||||
FORCEINLINE bool ShouldAbortWork() const
|
||||
{
|
||||
return bShuttingDown.load(std::memory_order_relaxed)
|
||||
|| bGenerationPaused.load(std::memory_order_relaxed);
|
||||
}
|
||||
```
|
||||
|
||||
Route the **existing** checks through it — the submission gate (`VoxelWorld.cpp:638`) and the
|
||||
in-task checks (`:1467`, `:1474`). **Do not add new check points**; do not change what those sites do
|
||||
when the check is true.
|
||||
|
||||
### 2. An RAII scoped pause, modelled on `EndPlay`'s drain
|
||||
|
||||
`EndPlay` (`:323–334`) already implements this exact pattern: raise the gate, then spin until
|
||||
`ActiveTaskCount` reaches 0. Mirror it.
|
||||
|
||||
```
|
||||
FScopedGenerationPause guard(this);
|
||||
if (!guard.Acquired()) { /* log error, DO NOT mutate, return */ }
|
||||
```
|
||||
|
||||
- **Ctor:** set `bGenerationPaused = true`, then wait for **both** `AVoxelWorld::ActiveTaskCount == 0`
|
||||
**and** the decoration tasks to finish. Decoration tasks are counted by the file-static
|
||||
`GActiveDecoTasks` in `VoxelContentManager.cpp` and already drained by `NotifyShutdown` (`:65–80`) —
|
||||
add a small public drain/wait accessor on `UVoxelContentManager` rather than exposing the counter.
|
||||
- **Dtor:** always clear `bGenerationPaused`, including on the failure path.
|
||||
|
||||
### 3. ⚠️ FAIL SAFE — this is the most important line in the spec
|
||||
|
||||
If the deadline expires with tasks still running: **DO NOT MUTATE.** Log an error naming the
|
||||
function, clear the flag, and return, leaving the world in its previous consistent state. The user
|
||||
can retry the edit.
|
||||
|
||||
**Mutating anyway is what the bug already does.** A timeout that proceeds is not a fix. The three
|
||||
dangerous call sites must each be structured so the `Initialize` call is *unreachable* unless the
|
||||
pause was acquired.
|
||||
|
||||
Use a generous deadline (≥ 5 s) and log at `Error` when it expires — a silent skip would look like
|
||||
the edit simply didn't apply.
|
||||
|
||||
### 4. Wrap the three call sites
|
||||
|
||||
`RebuildStrates`, `OnObjectModifiedInEditor`, `ChangeSeed`. The pause must cover **all** the mutation,
|
||||
including `ChangeSeed`'s writes to the generator's `Seed` / `OriginSpineRadius`, and it must be
|
||||
released **before** `RegenerateAllChunks()` so regeneration can submit work. **`BeginPlay` is not
|
||||
wrapped.**
|
||||
|
||||
## ⚠️ Invariants
|
||||
|
||||
1. **No density, mesher, or geometry code may change.** This must not move one bit of generated
|
||||
terrain. If your diff touches `VoxelGenerator.cpp`, `VoxelDensityOpStack.cpp`,
|
||||
`VoxelCaveMorphology.cpp` or `VoxelMarchingCubesMesher.cpp`, stop — wrong site.
|
||||
2. **No deadlock.** The pause is taken on the **game thread**. Verify by reading that chunk tasks
|
||||
never block on the game thread (they read the generator and `Enqueue` to an MPSC queue, which is
|
||||
non-blocking) — so a drain is bounded. **State in your report that you checked this**, and if you
|
||||
find any worker path that waits on the game thread, STOP and report it instead of proceeding.
|
||||
3. **`ProcessQueue` stays `EQueueMode::Mpsc`.** Do not touch it.
|
||||
4. **Do not change `EndPlay`.** Its 3-second timeout is a separate, deliberate decision
|
||||
(audit VF-02) and is Jahni's call, not part of this task.
|
||||
5. **Carry the `Epoch`** through anything you touch; do not reorder the existing epoch bump relative
|
||||
to `RegenerateAllChunks`.
|
||||
6. Comments are French + English; match the surrounding file.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- `stat`/gameplay unchanged; **generated terrain bit-identical** (the equivalence tests and every
|
||||
box-verdict number must be untouched — this change cannot reach them).
|
||||
- Editing a strate asset while the world streams: brief stall, then the edit applies. **No crash.**
|
||||
- The failure path is reachable and honest: if the drain times out, an `Error` log names the function
|
||||
and the world keeps its previous state.
|
||||
- `git diff --stat` should list `VoxelWorld.cpp`, `VoxelWorld.h`, and `VoxelContentManager.{h,cpp}`
|
||||
for the drain accessor. Nothing else.
|
||||
|
||||
## Notes for the reviewer (Claude)
|
||||
|
||||
- Confirm the three dangerous sites cannot reach `Initialize` when the pause was not acquired, and
|
||||
that `BeginPlay` is untouched.
|
||||
- Confirm the dtor clears the flag on **every** path including early return.
|
||||
- Confirm `ShouldAbortWork` replaced the existing checks rather than adding new ones, and that
|
||||
`bShuttingDown`'s own semantics are unchanged.
|
||||
- Confirm the deco drain is included — chunk tasks alone are not the whole reader set.
|
||||
@@ -0,0 +1,150 @@
|
||||
# Codex task 008 — (A) fix the measured column-memo thrash, (B) diagnose why T1.d never fires in game
|
||||
|
||||
**Owner:** Codex (Model Luna, xHigh) · **Orchestrator:** Claude · **Branch:** `experimental`
|
||||
**Status:** specified, not started
|
||||
|
||||
Two independent changes in different subsystems, deliberately bundled into one build because their
|
||||
signals cannot contaminate each other: (A) is SurfaceWorld column caching, (B) is a counter in the
|
||||
cave branch of `ClassifyTile`.
|
||||
|
||||
---
|
||||
|
||||
# PART A — replace the hashed column memo with a direct-indexed box
|
||||
|
||||
## This is now MEASURED, not suspected
|
||||
|
||||
`stat VoxelForge` in the running game, SurfaceWorld-dominated flight:
|
||||
|
||||
```
|
||||
Column Memo Hits avg 102,300.84
|
||||
Column Memo Misses avg 17,683.60 → miss rate 14.7%
|
||||
```
|
||||
|
||||
Predicted **14.0%** if the table thrashes, **1.4%** if it does not. It thrashes. Second confirmation
|
||||
from a different statistic: 17,683 misses ÷ 2.13 tiles meshed = **~8,300 column recomputes per
|
||||
tile**, where a healthy cache does ~1,225 — **6.8×**.
|
||||
|
||||
**The cause.** `FSurfaceColumnSource::GetColumn` (`VoxelDensityOpStack.cpp` ~615) uses a
|
||||
**direct-mapped, 4096-entry hashed** table. A direct-mapped table evicts on *collision*, not on
|
||||
fullness: 1225 columns per tile in 4096 slots is a load factor of 0.30, at which ~317 columns (26%)
|
||||
share a slot and evict each other — **on every one of the ~35 Z planes**, because the mesher
|
||||
pre-samples Z-outermost (`VoxelMarchingCubesMesher.cpp` ~226). Each miss recomputes the entire height
|
||||
stack: structural source, cliff (four structural resamples), terrace, layer-line, beach, ceiling.
|
||||
|
||||
## The fix — copy the scheme that already works, one file away
|
||||
|
||||
`GSurfColCache` / `FSurfaceColumnBox` in `VoxelGenerator.cpp` (~152, and its use at ~737) is the
|
||||
original path's solution to the identical problem: a **direct-indexed box** —
|
||||
`CI = (IY - Box.BaseY) * Dim + (IX - Box.BaseX)` with a `Computed[CI]` flag — centred on the first
|
||||
sample and rebuilt when a query leaves it. **No hash ⇒ no collisions ⇒ every column computed exactly
|
||||
once.** Read it before writing; you are porting a proven scheme, not inventing one.
|
||||
|
||||
Apply the same structure inside `FSurfaceColumnSource`, keeping it `thread_local`.
|
||||
|
||||
## ⚠️ The invariant that must not be lost in the port
|
||||
|
||||
The current memo's validity check is `S.Key != ColumnKey || S.X != WorldX || S.Y != WorldY`, and
|
||||
`ColumnKey` is built in `PrepareChunk` from **strate + layout version + seed + `ParamsFingerprint`**.
|
||||
|
||||
**`ParamsFingerprint` is load-bearing and its absence was a real shipped bug** — without it, two
|
||||
stacks of the same strate with different params shared columns, the overhang silently vanished, and
|
||||
only 69 of 20000 samples showed it. The comment at the site records this. **The new box's validity
|
||||
key must still contain all four**, or you reopen a fixed bug. (Note `GSurfColCache` itself keys on
|
||||
`(box XY, StrateKey, Seed, LayoutVersion)` **without** the fingerprint — do **not** copy that part;
|
||||
it is the weakness the op-stack memo deliberately closed.)
|
||||
|
||||
Also keep the full XY comparison semantics: a lookup must never return a column computed for a
|
||||
different XY. With a direct-indexed box that is structural (the index *is* the XY), but the box
|
||||
bounds check must be exact.
|
||||
|
||||
## Keep the counters
|
||||
|
||||
`ColumnMemoHit` / `ColumnMemoMiss` must keep working, incremented on the same meaning (miss = a
|
||||
column was recomputed). They are the before/after instrument.
|
||||
|
||||
## Invariants
|
||||
|
||||
1. **Density must not move by one bit.** This changes *caching*, never a computed value. The eight
|
||||
equivalence tests — especially `SurfaceHeightEquivalence` and its overhang section — must stay
|
||||
bit-identical. If your diff changes what `TerrainStack.EvalHeight` / `CeilingStack.EvalHeight`
|
||||
compute, or the overhang gate maths, you have the wrong site.
|
||||
2. Both consumers keep working: the source (`~858`) and the overhang (`~923`, via
|
||||
`Column->GetColumn`). The overhang **must** see the same column the source did — that is
|
||||
"by construction rather than by convention", and the current code says so.
|
||||
3. Sizing: state in a comment how many columns a tile needs (a 35×35 grid = 1225) and size the box so
|
||||
one tile fits without eviction, as `FSurfaceColumnBox` does.
|
||||
4. `thread_local` stays; no shared mutable state across workers.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- `ColumnMemoMiss` drops roughly **10×**; miss rate goes from **14.7% → ~1.5%**.
|
||||
- The eight equivalence tests stay green and bit-identical.
|
||||
- If the miss rate does **not** fall, say so plainly — a fix that does not move its own instrument is
|
||||
a failed fix, not a partial one.
|
||||
|
||||
---
|
||||
|
||||
# PART B — name which guard stops T1.d in the running game
|
||||
|
||||
## The problem
|
||||
|
||||
`Tiles Operator Stack Solid` / `Air` **never appeared** in `stat VoxelForge`, while the harness proves
|
||||
11 of 40 tiles at production defaults. Rows only render in frames where a counter fires, so site B is
|
||||
never reached in game. The cave branch of `UVoxelGenerator::ClassifyTile` has **13 `return
|
||||
EVoxelTileClass::Mixed` paths** and we cannot tell which one fires.
|
||||
|
||||
## What to build
|
||||
|
||||
Add DWORD counters to the existing `stat VoxelForge` group (`VoxelStats.h` / `.cpp`) that attribute
|
||||
the bail, grouped by *reason* rather than one per line:
|
||||
|
||||
| counter | fires when |
|
||||
|---|---|
|
||||
| `CaveBailNotOpStack` | `UsesOperatorStackForChunk` is false (either the initial check or the per-chunk sweep) |
|
||||
| `CaveBailMixedContent` | `bAnyNonCave` — the tile also touches a gap or SurfaceWorld chunk — or a second cave slot, or out-of-layout |
|
||||
| `CaveBailParams` | the params `Memcmp` disagreed across the box, the archetype differed, or `NumChunkCoords > 27` |
|
||||
| `CaveBailStackVerdict` | the stack built fine but `ClassifyBox` returned `Mixed` |
|
||||
| `CaveBailDisturbance` | the final `bCanSolid == bCanAir` after disturbances |
|
||||
| `CaveBailNoStack` | `VF_BuildOpStackForChunk` returned false |
|
||||
|
||||
Increment **exactly one** per bail, immediately before the `return`. Together with the existing
|
||||
`TilesOpStackSolid` / `TilesOpStackAir`, one underground flight then names the cause outright.
|
||||
|
||||
## Invariants
|
||||
|
||||
1. **DO NOT change any control flow, condition, or return value in `ClassifyTile`.** Every `return`
|
||||
there is a conservative guard that fails to `Mixed`; a wrong verdict leaves a tile with no
|
||||
geometry and no collision. Add counters beside the existing returns and nothing else.
|
||||
2. `ClassifyTile` is `const` and runs on **worker threads** — `INC_DWORD_STAT` only, never a
|
||||
`static int32++`. It routes through `FThreadStats::AddMessage` (per-thread packets), which is why
|
||||
it is safe.
|
||||
3. Zero cost when `STATS == 0`: compute nothing outside the macros.
|
||||
4. Do not touch the non-cave parts of `ClassifyTile` (gap / SurfaceWorld / column scan).
|
||||
|
||||
## Acceptance
|
||||
|
||||
Fly underground in a TunnelNetwork strate: exactly one bail counter should dominate, or
|
||||
`TilesOpStackSolid` should finally appear. Either outcome is a result.
|
||||
|
||||
---
|
||||
|
||||
# Shared rules
|
||||
|
||||
- **NEVER build, compile or run the editor or the tests.** Stop when the code is written.
|
||||
- **Do not `git commit`, `git push`, `git checkout`, `git stash`, `git restore`.** Uncommitted work
|
||||
in the tree must survive.
|
||||
- Comments are French + English; match the surrounding file.
|
||||
- Macro spelling: `KINDA_SMALL_NUMBER`, not the `UE_`-prefixed form.
|
||||
- When inserting anything into `VoxelDensityOpStack.cpp`, put it **above the labelled end of the
|
||||
anonymous namespace** — anchoring on the FACTORIES banner puts it outside and the brace closes
|
||||
nothing. This mistake has been made twice in that file.
|
||||
|
||||
## Report
|
||||
|
||||
1. The diff for Part A and Part B separately.
|
||||
2. `git diff --stat`.
|
||||
3. Explicit confirmation that: no height-stack or overhang maths changed; the new column key still
|
||||
contains strate + layout + seed + `ParamsFingerprint`; `ClassifyTile`'s control flow and return
|
||||
values are untouched; exactly one bail counter fires per bail path.
|
||||
4. Likely compile-error spots, specifically.
|
||||
5. Anything in this spec that contradicts the code — **stop and say so rather than guessing.**
|
||||
@@ -0,0 +1,129 @@
|
||||
# VoxelForge Task 1: profiler, batch explorer, and research handoff
|
||||
|
||||
Research date: 2026-09-09. This note records the implementation and the read-only web research for Task 1. No RealtimeMeshComponent files were changed.
|
||||
|
||||
## 1. Measurement result
|
||||
|
||||
The old result in the task brief was 14.74 us/sample on the profiled density path versus 1.28 us/voxel without instrumentation: about 11.5x apparent overhead. That is consistent with putting `FPlatformTime::Cycles64()` around a roughly 100 ns operation. The timer call is not free, and at that granularity the measurement is mostly the measurement.
|
||||
|
||||
The new default is sampled profiling:
|
||||
|
||||
- Coarse calls and counters stay cheap and are collected in thread-local state.
|
||||
- `GetDensityAt` uses a deterministic count interval of 64 calls and one amortized block timer, rather than one timer pair per density call.
|
||||
- Full cycle timing remains an explicit opt-in (`-profiledensityfull`) for a small investigation, not the normal batch mode.
|
||||
- The default sampled mode reports the parent density/mesher wall estimate. Fine per-operation scopes are disabled in that mode; use Full or Unreal Insights for a detailed phase breakdown.
|
||||
- Sampling and timers only write report counters. They do not choose cells, alter density values, alter task partitioning, or participate in any reduction, so they cannot affect generated geometry.
|
||||
|
||||
The timer calibration in the validation run measured one `Cycles64` pair as approximately one platform cycle, or 0.1 us at the reported clock conversion. That is the direct self-check for the suspected timer-dominates-work failure mode.
|
||||
|
||||
Validation on the same 64^3 `TunnelNetwork` export:
|
||||
|
||||
| measurement | seconds |
|
||||
| --- | ---: |
|
||||
| profiled sampled export | 0.4630500004 |
|
||||
| fresh unprofiled reference export | 0.5584703982 |
|
||||
| discrepancy | -17.09% |
|
||||
| profiled/reference ratio | 0.82914 |
|
||||
|
||||
Both exports produced geometry hash `235AC2DC`. The commandlet now puts all four values in every profiled report and fails with exit code 2 when the absolute discrepancy exceeds 20% or the reference hash differs. The report also exposes `cpu_us` (sum of worker CPU estimates) separately from `wall_us`/`total_us` (largest-worker wall estimate). Summing CPU time across 15 workers is not a valid wall-clock parent; collapsing those fields was one way to make a parallel profile lie.
|
||||
|
||||
The density report has an explicit ledger. In sampled mode, the parent estimate is the measured quantity, fine phase buckets are marked unavailable, and the unattributed remainder makes the ledger sum exactly to the parent. This is honest about what was measured instead of inventing a fine breakdown from disabled timers.
|
||||
|
||||
## 2. Batch explorer
|
||||
|
||||
The explorer commandlet accepts one JSON file and keeps one editor process alive for every case:
|
||||
|
||||
```text
|
||||
UnrealEditor-Cmd.exe "<HostProject>.uproject" -run=VoxelForgeExplore -batch="E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Saved\BatchValidation.json" -out="E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Saved\BatchValidationRun2" -trace=cpu,counters
|
||||
```
|
||||
|
||||
`-out` must be an absolute directory below this plugin's `Saved` directory. The commandlet writes one `batch.json` below that directory. The input supports, per case: `id`, `seed`, `archetype`, `slot`, `export_size`, `export_step`/`step`/`lod`, `modes`, `operator_stack`, `profile_density`, `profile_density_full`, `profile_lod`, render/failure-focus settings, and a per-case budget. The parser caps one batch at 256 cases and clamps the total budget to 30 minutes; each case also has a bounded budget.
|
||||
|
||||
Before every case, the commandlet resets the density profile and all per-case report state, then applies the case's profile settings. The output contains per case: mesh seconds, microseconds per voxel, triangle count, geometry hash, counters, profile mode, and the case exit code. Invalid or timed-out cases are represented in the same machine-readable output rather than silently disappearing.
|
||||
|
||||
The final three-case confirmation completed in 6.59 seconds of commandlet case time after the single process launch:
|
||||
|
||||
| case | triangles | mesh seconds | hash | result |
|
||||
| --- | ---: | ---: | --- | --- |
|
||||
| repeat_unprofiled | 2,461 | 0.391420 | `235AC2DC` | pass |
|
||||
| repeat_profiled | 2,461 | 0.515374 | `235AC2DC` | self-check pass |
|
||||
| maze_lod1 | 5,218 | 0.277830 | `784EBF92` | pass |
|
||||
|
||||
The standalone profiled 64^3 case and the `repeat_profiled` case inside the batch both produced `235AC2DC`, 2,461 triangles, 343,000 tunnel lookups, and 554,381 passage evaluations. That is the contamination check: same final build, seed, archetype, slot, export size, step, and modes produced the same geometry and coarse counters alone and in a reused process. The standalone profiled export also matched its fresh unprofiled reference hash.
|
||||
|
||||
## 3. Cache footprint answer
|
||||
|
||||
The historical 15-worker report that prompted the question was:
|
||||
|
||||
```text
|
||||
capacity_entries=3840 valid_entries=1542 static_bytes=2534400
|
||||
dynamic_bytes=428627716 total_bytes=431162116
|
||||
avg_valid_entry_bytes=278703.6 largest_entry_bytes=541344
|
||||
tunnel_valid=1054 tunnel_dynamic=345243636
|
||||
roomgraph_valid=488 roomgraph_dynamic=83384080
|
||||
```
|
||||
|
||||
That is 431,162,116 bytes (411.2 MiB) in aggregate and 278,703.6 bytes per valid entry on average (278.7 kB decimal, about 265.7 KiB). The number is not one monolithic 272 kB struct. Each worker owns cache slots, and valid entries contain variable-length arrays.
|
||||
|
||||
A representative 128^3 breakdown added by this task was 360,063,124 bytes total:
|
||||
|
||||
| member family | tunnel cache | room-graph cache |
|
||||
| --- | ---: | ---: |
|
||||
| slot storage | 1,658,880 | 875,520 |
|
||||
| op stack | 8,264,000 | 0 |
|
||||
| rooms | 47,876,912 | 31,822,512 |
|
||||
| room-floor joins | 4,936,512 | 3,278,976 |
|
||||
| tunnels and control arrays | 5,560,248 | 3,591,584 |
|
||||
| support-column entries | 7,153,328 | 0 |
|
||||
| support columns | 30,699,008 | 0 |
|
||||
| room index bins + candidates | 30,759,720 | 22,700,380 |
|
||||
| tunnel index bins + candidates | 60,972,480 | 41,583,988 |
|
||||
| tunnel-world index bins + candidates | 34,635,272 | 23,693,804 |
|
||||
| dynamic total | 230,857,480 | 126,671,244 |
|
||||
| static + dynamic total | 232,516,360 | 127,546,764 |
|
||||
|
||||
The dominant allocation is the spatial index bin/candidate storage, followed by rooms and support-column arrays. Slot storage is comparatively small. The exact per-entry average moves with the generated case and cache occupancy, which explains why the 128^3 confirmation is not numerically identical to the historical 411 MiB run.
|
||||
|
||||
## 4. Research findings
|
||||
|
||||
### RealtimeMeshComponent, read-only
|
||||
|
||||
At research time the upstream releases page lists v5.3.2 as the latest tagged release: [RMC releases](https://github.com/TriAxis-Games/RealtimeMeshComponent/releases). The disposable build host's copy identifies itself as `VersionName: 5.4`, so the upstream tag number is not a safe linear comparison; this project also carries custom changes. The visible upstream history includes a 5.1 game-thread/render-thread restructuring and a 5.0.5 release mentioning mesh-update and memory-overuse fixes. The public release page does not provide a complete API diff against this custom 5.4 copy, so I am treating those as compatibility signals rather than promising source compatibility. The local copy was not upgraded, diffed destructively, or otherwise modified.
|
||||
|
||||
The current upstream guidance is to build an `FRealtimeMeshStreamSet` as a CPU-side value on a worker, then cross to the game thread to create/update the component section group; use a weak object pointer for the handoff. The structure documentation describes a game/render proxy mirror and distinguishes in-place dynamic/compute updates from static proxy recreation. See [updating mesh data](https://triaxis.games/realtime-mesh/docs/component-core/updating-mesh-data/) and [RMC component structure](https://triaxis.games/realtime-mesh/docs/component-core/structure/). The practical candidate is therefore worker-side stream construction plus the smallest possible section/buffer update, not touching a `URealtimeMeshComponent` from a worker. Custom RMC changes make this a high-risk follow-up.
|
||||
|
||||
### UE 5.7 tasking and tracing
|
||||
|
||||
Epic's UE 5.7 API documentation says `ParallelForWithExistingTaskContext` gives each spawned task a task-local workspace, and `MinBatchSize` limits launches to `DivUp(Num, MinBatchSize)` workers. The pre-work variant runs caller work before that thread helps the parallel loop. Both pages explicitly warn against clogging the task graph with long-running or blocking work: [existing task context](https://dev.epicgames.com/documentation/en-us/unreal-engine/API/Runtime/Core/ParallelForWithExistingTaskConte-) and [pre-work task context](https://dev.epicgames.com/documentation/unreal-engine/API/Runtime/Core/ParallelForWithPreWorkWithTaskCo-). The next tuning target should be measured batch size and task-local scratch reuse for short density/cell jobs, not more tiny tasks.
|
||||
|
||||
UE's built-in tracing is the better fine-grained instrument. The documented channels include CPU, Counters, Task, and StackSampling; `-trace=cpu,counters` is a valid launch selection. `TRACE_CPUPROFILER_EVENT_SCOPE` with static scope names and `CountersTrace.h`/`TRACE_COUNTER_*` are the low-cost native hooks. See [Trace channels](https://dev.epicgames.com/documentation/en-us/unreal-engine/trace-in-unreal-engine-5), [the tracing guide](https://dev.epicgames.com/documentation/en-us/unreal-engine/developer-guide-to-tracing-in-unreal-engine?lang=en-US), and the [Unreal Insights reference](https://dev.epicgames.com/documentation/en-us/unreal-engine/unreal-insights-reference-in-unreal-engine-5). Timing Insights supplies timers, counters, callers/callees, and task execution views; Task Graph Insights exposes execution paths and critical paths: [Timing Insights](https://dev.epicgames.com/documentation/en-us/unreal-engine/timing-insights-in-unreal-engine?lang=en-US) and [Task Graph Insights](https://dev.epicgames.com/documentation/unreal-engine/task-graph-insights-in-unreal-engine-5?lang=en-US).
|
||||
|
||||
This task therefore keeps the lightweight sampled report for automated self-checks and adds native CPU scopes/counters for investigation. Insights should own future fine per-scope and task scheduling analysis; the hand-rolled profiler should not grow back into a timer around every density call.
|
||||
|
||||
### Marching cubes and density generation
|
||||
|
||||
The upstream [Transvoxel overview](https://transvoxel.org/) confirms that transition cells use local voxel data and a bounded lookup table to stitch different-resolution meshes without cracks. It is relevant to LOD boundary work and dynamic retriangulation, not a reason to change the current mesh in this task. A classic SIMD study reports nearly 4x over an unoptimized serial Marching Cubes implementation and about 2x over compiler-optimized C: [Newman et al., *High performance SIMD marching cubes*](https://www.sciencedirect.com/science/article/abs/pii/S0097849303002760). Those are historical results on older x86 and should be treated as an upper-bound signal, not a VoxelForge forecast.
|
||||
|
||||
## 5. Ranked next-task candidates (not implemented here)
|
||||
|
||||
The magnitude estimates are expected ranges for the affected stage, inferred from the sources and this run's counters; they are not promises for end-to-end time.
|
||||
|
||||
| rank | candidate | expected magnitude | risk |
|
||||
| ---: | --- | --- | --- |
|
||||
| 1 | Conservative macrocell/cell empty-space early-out using deterministic density bounds, with a fallback on boundary uncertainty | 2–10x on empty or mostly empty chunks; smaller on surface-heavy chunks | Medium-high: bounds must never skip a crossing cell or change traversal order |
|
||||
| 2 | Reuse one canonical density grid/field across meshing, LOD, export, and renderer preparation; invalidate by seed/slot/parameters | 20–60% less repeated density work and memory bandwidth where consumers overlap | Medium: cache lifetime and invalidation |
|
||||
| 3 | Tune `ParallelFor` granularity and task-local scratch for short jobs; measure `MinBatchSize`, worker count, and critical path in Insights | 5–25% less scheduling/coordination cost | Low-medium: load imbalance and contention can reverse the win |
|
||||
| 4 | SIMD/SoA batches for density and Marching Cubes table work | 1.5–4x for the vectorized stage in favorable workloads | High: exact cross-machine geometry and floating-point contraction; no runtime CPU dispatch is allowed |
|
||||
| 5 | Hierarchical sparse/occupancy metadata or distance bounds to skip whole regions before field evaluation | 2–20x in sparse worlds | High: build/update cost, conservative topology, and invalidation |
|
||||
| 6 | RMC partial section/buffer updates after worker-side stream construction | 10–50% less renderer update/bandwidth cost when only part of a section changes | High: custom RMC modifications and game/render-thread ownership |
|
||||
| 7 | Transvoxel transition cells for LOD boundaries | 10–30% less boundary patch/rebuild work in LOD-heavy scenes; also removes crack workarounds | Medium-high: transition topology and hash stability |
|
||||
|
||||
The first safe experiment should be the deterministic early-out or field reuse, after a baseline with the corrected profiler. SIMD is deliberately below those despite its attractive paper result because the project's cross-machine geometry requirement is absolute.
|
||||
|
||||
## 6. Pushback and constraints
|
||||
|
||||
The requirement that buckets sum to their parent needs a definition in a parallel program. Raw worker CPU time can exceed wall time by the worker count; the report now exposes both and makes the wall estimate the parent used for the self-check. A fine raw-bucket sum cannot be claimed when sampled mode intentionally does not time those scopes, so the report says that explicitly and accounts for the remainder.
|
||||
|
||||
The 20% self-check threshold is a practical alarm, not a proof of zero measurement cost: OS scheduling, cache state, DDC state, and editor startup can move two exports. The geometry hash check is exact; repeated cases or a median of several runs should be used when a performance regression is close to the threshold.
|
||||
|
||||
The sampling decision is non-deterministic only in its elapsed-time observation. It is isolated from the generation inputs and outputs. The batch commandlet resets profile/cache-report state per case, but it intentionally reuses the engine process to remove startup cost; the standalone-versus-batch geometry hash is the proof that this reuse did not contaminate the tested case.
|
||||
@@ -0,0 +1,140 @@
|
||||
# Task 2 performance report
|
||||
|
||||
Date: 2026-09-09
|
||||
Repository: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge`
|
||||
Work began at `experimental` / `e63f016`.
|
||||
|
||||
## Measurement rule
|
||||
|
||||
All performance numbers below are per-case `run.elapsed_seconds` from the batch report. This is
|
||||
the wall clock for a real commandlet case through the export or render result; it includes the
|
||||
shared-density-grid build when enabled. `summary.mesh_seconds` and profiler buckets are included
|
||||
only as diagnostics, never as the performance claim.
|
||||
|
||||
The export comparison is the same `128^3` `TunnelNetwork`, seed `1337`, slot `1`, operator stack,
|
||||
step `1`, with three cases in one commandlet process on each side. The 17% reporting floor is
|
||||
therefore met by both run count and the retained density-grid result.
|
||||
|
||||
## Part 1 — block early-out
|
||||
|
||||
I tested an `8^3` block reject behind conservative fallback logic. On this cave/stack world, every
|
||||
one of the 4,096 blocks was left uncertain and went through normal sampling:
|
||||
|
||||
| batch | runs | end-to-end wall seconds | blocks skipped | skip rate | result |
|
||||
| --- | ---: | --- | ---: | ---: | --- |
|
||||
| baseline | 3 | 3.805057, 3.522129, 3.569186 (avg 3.632124) | — | — | reference |
|
||||
| block reject | 3 | 6.341454, 6.562621, 7.300234 (avg 6.734770) | 0 / 4,096 | 0.00% | **reverted** |
|
||||
|
||||
The candidate added classification work without skipping a block (+85.42% wall clock). It also
|
||||
did not produce a useful supremum for the complete cave stack, so I did not promote a guessed
|
||||
`max` parameter into a topology decision. `ClassifyTile` remains untouched: its stack, disturbance,
|
||||
diff-layer, and cave guards are not a safe substitute for a new block proof. A false AllSolid
|
||||
verdict here would delete a 3D cave, which is not an acceptable trade.
|
||||
|
||||
This did not clear the floor and was reverted.
|
||||
|
||||
## Part 2 — density-grid reuse
|
||||
|
||||
The canonical mesher requests 35^3 samples per 32-cell tile. For 64 tiles that is:
|
||||
|
||||
- tile requests: `64 * 35^3 = 2,744,000`;
|
||||
- unique points in the expanded 128-cell lattice: `131^3 = 2,248,091`;
|
||||
- duplicate evaluations: `495,909` (18.07%).
|
||||
|
||||
The first implementation evaluated the shared lattice as Z planes. It removed all duplicates but
|
||||
was slower because it destroyed the generator's spatial locality:
|
||||
|
||||
| candidate | runs | end-to-end wall seconds | duplicates | result |
|
||||
| --- | ---: | --- | --- | --- |
|
||||
| matched control | 3 | 3.668954, 3.633445, 3.655658 (avg 3.652686) | 495,909 → 495,909 | reference |
|
||||
| global-plane precompute | 3 | 5.349520, 5.352927, 5.510752 (avg 5.404400) | 495,909 → 0 | **reverted** |
|
||||
| owner-grid precompute | 3 | 1.950207, 1.973673, 2.161305 (avg 2.028395) | 495,909 → 0 | retained candidate |
|
||||
|
||||
The retained implementation partitions the shared lattice into 64 disjoint rectangular owner
|
||||
regions. Each global sample is generated exactly once, then each tile copies its local halo window
|
||||
from the immutable run-scoped grid. The precompute is deliberately inside the wall-clock timer;
|
||||
the final three-case repeat measured:
|
||||
|
||||
| batch | runs | end-to-end wall seconds | average |
|
||||
| --- | ---: | --- | ---: |
|
||||
| final control (`density_grid_reuse=false`) | 3 | 3.659587, 3.619013, 3.582257 | 3.620286 s |
|
||||
| final reuse (`density_grid_reuse=true`) | 3 | 1.908017, 1.854344, 1.884170 | 1.882177 s |
|
||||
|
||||
That is a **48.01% wall-clock reduction / 1.92x speedup**, clearing the 17% floor. The final
|
||||
reuse runs averaged 1.474271 s for the shared-grid build, and still won end to end. The 128^3
|
||||
shared lattice is about 8.6 MiB of float storage. Direct explorer runs default to reuse; a batch
|
||||
case can pass `density_grid_reuse=false` for a matched control.
|
||||
|
||||
The run-scoped handoff is confined to the explorer's finite canonical export. The streaming game
|
||||
path has no one-shot set of neighboring tiles or invalidation boundary, so it was not given a
|
||||
locking plane cache that could serialize workers or outlive a generation epoch.
|
||||
|
||||
## Part 3 — task granularity
|
||||
|
||||
The final control observed 15 worker thread IDs and 64 logical tile invocations. Across its three
|
||||
runs, the measured tile-job averages were:
|
||||
|
||||
| metric | average |
|
||||
| --- | ---: |
|
||||
| fastest logical tile job | 0.188817 s |
|
||||
| mean logical tile job | 0.593751 s |
|
||||
| slowest logical tile job | 0.955788 s |
|
||||
| sum of tile-job durations | 38.000045 s |
|
||||
| tile parallel wall | 3.150925 s |
|
||||
| launch-fringe estimate | 14.1 us |
|
||||
| workers / logical jobs | 15 / 64 |
|
||||
|
||||
I tested `MinBatchSize=4` with three matched cases: wall seconds `3.590884, 3.634734,
|
||||
3.753986` (avg `3.659868`) versus the final control average `3.620286`. The tile parallel bucket
|
||||
was slightly lower (`3.125197` s vs `3.150925` s), but end-to-end wall clock was **1.09% slower**.
|
||||
The observed worker count and logical job count stayed 15 / 64, and the launch-fringe estimate
|
||||
remained about 13.1 us. This is the required bucket-win/no-wall-win case: the tuning was reverted;
|
||||
the production/default setting remains `MinBatchSize=1`.
|
||||
|
||||
`ParallelForWithExistingTaskContext` was not adopted. The measured launch fringe is negligible
|
||||
relative to 0.59 s tile work, and the mesher already uses thread-local scratch storage. No claim is
|
||||
made here about the separate game workload of many small streaming submissions.
|
||||
|
||||
## Part 4 — renderer
|
||||
|
||||
No renderer rewrite or RMC change was justified. The source audit and three final render cases
|
||||
showed:
|
||||
|
||||
- the canonical mesh is built once per explorer world;
|
||||
- the triangle cache/BVH is built once and reused by all eight viewpoints;
|
||||
- each viewpoint owns one `FColor` output allocation;
|
||||
- the per-pixel traversal uses a fixed stack array, not a heap allocation;
|
||||
- camera movement does not call the mesher again.
|
||||
|
||||
The three 512x288 render cases had total case wall seconds `8.704730, 8.985744, 8.517606`
|
||||
(avg `8.736027`), including camera-seed walking. Raster-only diagnostics averaged 1.482921 s
|
||||
for eight views, 0.286297 s for the first view, 0.170946 s per additional view, and 0.083190 s
|
||||
for one-time acceleration construction. This is consistent with the owner's approximately
|
||||
0.15 s/view observation; meshing benefits from Part 2 automatically. There was no renderer
|
||||
change, so there is no before/after performance claim to report or keep.
|
||||
|
||||
## Memory/index follow-up
|
||||
|
||||
The existing profiler research measured 431,162,116 bytes (411.2 MiB) across 15 workers in the
|
||||
large run. In the representative 128^3 breakdown, spatial-index bins and candidate arrays alone
|
||||
accounted for 214,345,644 bytes, inside a 360,063,124-byte cache total. The index addition did not
|
||||
show an end-to-end wall-clock gain in the prior measurements.
|
||||
|
||||
I did not remove it silently. The explicit follow-up proposal is an index-on/index-off batch
|
||||
ablation before deletion: removing it should recover roughly that allocation, at the cost of
|
||||
broader room/tunnel candidate scans and potentially slower density queries. That trade is not
|
||||
proven by this task's export comparison.
|
||||
|
||||
## Determinism, invariants, and scope
|
||||
|
||||
All three final control hashes and all three final reuse hashes are `ED0F1F31`; every case reported
|
||||
`json_deterministic=yes`. The three render cases also repeated the same geometry hash
|
||||
`847BE4B9`. Owner regions are disjoint, tile results are merged in fixed Z/Y/X order, no RNG or
|
||||
runtime CPU-feature dispatch was added, and the shared grid is read-only after its deterministic
|
||||
fill. The only worker writes are to distinct cells of that run-owned output lattice; generator and
|
||||
world state remain read-only.
|
||||
|
||||
The build succeeded with UnrealBuildTool after the final source change. Changes are limited to the
|
||||
VoxelForge source and this report, with generated evidence under the plugin's `Saved` directory.
|
||||
RealtimeMeshComponent was not modified. No VoxelM root, sibling plugin, `Content`, or `Config`
|
||||
files were written; no links or historical worktrees were created.
|
||||
@@ -0,0 +1,887 @@
|
||||
# DESIGN-SOL-2026-09-15-CODE: round two, Sol reads the code
|
||||
|
||||
Continues `DESIGN-SOL-2026-09-15.md`. At the owner's request, Sol's round-one session was forked
|
||||
(`codex exec fork`, `gpt-5.6-sol`, reasoning high, read-only sandbox, ~594k tokens). Sol was given
|
||||
the repository and asked **what could reasonably change and what it would take to prove each change
|
||||
worth it**. Advice only: no files were changed, nothing was built or run. The owner's stance: *"open
|
||||
to big changes, but only if they prove themselves worthy."*
|
||||
|
||||
## Orchestrator verification (2026-09-15, against HEAD `cf9904e` and the UE 5.7 engine source)
|
||||
Checked by reading, not by running anything.
|
||||
|
||||
| Sol's claim | Check |
|
||||
|---|---|
|
||||
| `FMath::Sin/Cos` reach the MSVC CRT on Win64 | **Verified.** `GenericPlatformMath.h:521/530` returns `sinf`/`cosf`; `FMicrosoftPlatformMathBase` (via `TUnrealPlatformMathSSE4Base`) and the SSE bases override neither. |
|
||||
| Per-sample lookup machinery is ~5% exclusive, not the 27% inclusive figure | **Verified**, `PERF-SAMPLED.md` lines 11-13: `GetRange` 3.8%, `VF_ForEachSpatialCandidate` 1.2%. My round-one note "the largest gap is the loop direction" overstated it. |
|
||||
| Parse-once switches are unsynchronised globals parsed on worker threads | **Verified**, `VoxelGenerator.cpp` (`VF_Parse*Switch` called in `GetDensityAt`). In practice every racer writes the same value; the real exposure is a live CVar change mid-generation. |
|
||||
| `MemCrc32` over raw param structs; the comment says no false positive is possible | **Verified** (`VoxelGenerator.cpp` ~3165). A CRC32 collision is possible in principle (about 2^-32 per comparison); padding can only cause a spurious rebuild. Low probability. |
|
||||
| Mesher dedups vertices by rounding position x100 | **Verified** (`VoxelMarchingCubesMesher.cpp` `GetOrCreateVertex`). Only amplifies an upstream numeric difference; not an independent source of divergence. |
|
||||
| ARCHITECTURE says every landing gets a connector to (0,0); the code says a source-fit landing stays local | **Verified contradiction**: `ARCHITECTURE.md` ~518 vs `VoxelStrateManager.cpp` ~2108. One of them is wrong. |
|
||||
| Lateral-region audit 4/16 | **Verified**, `ARCHITECTURE.md` ~122. That mechanism is explicitly "not ready to promote", so it is not live terrain. |
|
||||
| Layout shuffle uses a stateful `FRandomStream` | **Verified** (`VoxelStrateManager.cpp` ~1155), after a deterministic sort. Deterministic; only order-coupled when content is added. |
|
||||
| Coarse LOD derived from fine data would break the clipmap invariant | Agreed (`ARCHITECTURE.md` ~708). My round-one table wrongly listed it as a gap. |
|
||||
| Noise lattice float-to-int, MXCSR rounding mode, multiplayer edit ordering | Not re-checked in detail; recorded as Sol's claims. Edit ordering matters once networking exists. |
|
||||
|
||||
---
|
||||
|
||||
## The prompt, as sent
|
||||
|
||||
# Phase two: now you get the real code
|
||||
|
||||
Thank you for the first answer. It was the most useful design input this project has had in a
|
||||
while. The owner read it and his reaction was: **"what Sol proposed can be nice. I am open to big
|
||||
changes, but only if they prove themselves worthy."** That is the question now: **what could reasonably be changed in this code, and what would it take to
|
||||
prove each change worth it?**
|
||||
|
||||
**The earlier rule is lifted: you may now read and search files.** This remains a read-only
|
||||
consultation:
|
||||
- Reading commands only (`rg`, `Get-Content`, `git log/show/diff`). Do not write or edit any file.
|
||||
Do not build, compile, run tests, or launch Unreal. A single test run here takes up to an hour,
|
||||
and the machine is the owner's.
|
||||
- Use absolute paths; your shell's working directory can drift.
|
||||
- Your final message is the whole deliverable. Please list the files you actually read at the end.
|
||||
|
||||
## The code
|
||||
Plugin root: `E:\Projet Unreal\VoxelM\Plugins\VoxelForge` (git branch `experimental`, HEAD `cf9904e`).
|
||||
Start with these, reading targeted sections rather than whole files; some are over 100 KB:
|
||||
- `CLAUDE.md` (rules), then `CODEMAP.md` (§2 data flow, §3 symbol→file index, §6 gotchas).
|
||||
- `ARCHITECTURE.md` §8, especially **§8.10, the performance invariants** that exist for measured
|
||||
reasons.
|
||||
- `WORK-NEXT.md`, section "State after 2026-09-15" and "Open design questions for the owner".
|
||||
- `PERF-SAMPLED.md`, section "Current-tip cost re-rank — 2026-09-15 (1d824aa)" (from line ~1669):
|
||||
the measured cost map. Two later commits cut a further ~12% (`a668833`, reach gating) and ~4%
|
||||
(`410788f`, per-chunk routing cache).
|
||||
- `DESIGN-SOL-2026-09-15.md`: your first answer, plus **my** mapping of it onto this code (the
|
||||
"Orchestrator notes"). I wrote that mapping from memory of the code; check it and correct me.
|
||||
- `AUDIT-SOL-2026-09-11.md` and `AUDIT-COLD-2026-09-14.md`: earlier read-only audits, so you don't
|
||||
repeat them.
|
||||
- Hot code: `Source/VoxelForge/Private/VoxelGenerator.cpp` (`GetDensityAt`,
|
||||
`GetDensityWithParams`), `VoxelCaveMorphology.cpp` (tunnel SDF, world-space tunnel core, player-fit
|
||||
memo), `VoxelDensityOpStack.cpp` (operator stack, fused evaluator), `VoxelStrateManager.cpp`
|
||||
(strates, passages), the mesher / tile scheduling, and `VoxelForge.Build.cs` (`/fp:precise` is
|
||||
already set).
|
||||
|
||||
## What is measured (don't re-derive; do challenge it if the code says otherwise)
|
||||
- In-game worker CPU for the standard static scenario is ~27.6 s, down from ~184 s at the start of
|
||||
this push, with the same world and the same capability. Measured on the headless game path with a
|
||||
stack sampler plus remove-one ablation.
|
||||
- **Tried and rejected, please don't re-propose these without new evidence:**
|
||||
- Merging the warped tunnel SDF with the world-space tunnel core into one pass: the world core is
|
||||
load-bearing (its control points keep authored floor anchors before warp). Replacing it lost
|
||||
1,146 player-fit and 975 walk-reachable cells.
|
||||
- An always-on exact per-sample cull: +23% slower.
|
||||
- Cheaper value-noise worms: no gain. Worms off entirely saves only ~3%.
|
||||
- The acceptance rules, which any proposal must live with:
|
||||
- **Determinism:** same build + same seed = same world on every machine. Bit-identity with
|
||||
*older* builds is NOT required, so a change that alters the field is allowed. Classifiers and
|
||||
skip proofs must never change the field.
|
||||
- **Capability:** the existing gate (player-fit 20,830 / walk-reachable 10,909 cells, connected,
|
||||
on the canonical scenario) must not regress.
|
||||
- **Performance:** judged on the headless GAME path, static and moving, comparing the final
|
||||
default build against the committed-tip DLL. A switch-off A/B inside the new build does not
|
||||
count.
|
||||
- The operator stack and the composer stay as the authoring language: worlds are composed, not
|
||||
hand-built.
|
||||
|
||||
## What we would like from you
|
||||
1. **Correct my mapping.** For each row of the table in `DESIGN-SOL-2026-09-15.md`, say whether it
|
||||
is right, and give file:symbol evidence where it is wrong.
|
||||
2. **Your ideas against this code.** For each idea from your first answer that survives contact with
|
||||
the code:
|
||||
- where it plugs in (files, symbols) and what it replaces or keeps;
|
||||
- its rough size;
|
||||
- its risks to capability, determinism, the composer and the §8.10 invariants;
|
||||
- its expected gain, grounded in the measured cost map. Say what share of the ~27.6 s it could
|
||||
touch, and mark each number measured or inferred.
|
||||
3. **Rank them.** Then, for the top one or two big changes, design a **proof-of-worth ladder**:
|
||||
- Rung 1: the cheapest experiment that could kill the idea early. Ideally instrumentation only,
|
||||
e.g. how much time goes to per-sample candidate search, and how many candidates a sample sees.
|
||||
- Rung 2: a prototype scoped to one feature type, behind a console switch.
|
||||
- Rung 3: full adoption.
|
||||
Each rung needs an explicit, falsifiable kill criterion, and the numbers that would justify going
|
||||
on. The owner wants big changes to earn their place, so make the early rungs cheap and honest.
|
||||
4. **Determinism audit of numerics on the world-deciding path.** We build with `/fp:precise`.
|
||||
Enumerate the remaining cross-machine hazards with file:line:
|
||||
- CRT transcendentals (`sin/cos/pow/exp/log/atan2/...`, including through `FMath`);
|
||||
- any runtime CPU-feature dispatch (SIMD paths chosen at runtime, ISPC);
|
||||
- float-to-int conversions at boundaries;
|
||||
- order-dependent accumulation across threads, uninitialised data, timing dependence.
|
||||
Give severity and the minimal fix for each. In particular: on UE 5.7 Win64, do `FMath::Sin/Cos`
|
||||
end up in the MSVC CRT (whose FMA3 variant is chosen at runtime), or in engine code? If engine
|
||||
source is not reachable from here, say so and say what you would check.
|
||||
5. **What not to graft here**, and why.
|
||||
|
||||
This is advice only. Nothing will be changed as a result of this run until the owner has read your
|
||||
answer and decided, so don't write code or patches. Describe what could reasonably change, and what
|
||||
it would take.
|
||||
|
||||
Push back wherever the code shows my framing or my mapping is wrong. That is worth more than
|
||||
agreement. Say plainly where you are unsure.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Sol's answer, verbatim
|
||||
|
||||
# Executive conclusion
|
||||
|
||||
I would not replace VoxelForge’s representation, mesher, streaming model, or operator authoring language wholesale. The current design is already much closer to the “lazy compiled field” architecture from the first answer than the orchestrator notes imply, and the measured reduction from ~184 s to ~27.6 s shows that its major structural choices are viable.
|
||||
|
||||
Three changes remain worth serious consideration:
|
||||
|
||||
1. A tile-wide, feature-major lowering of the canonical tunnel evaluator. This is the only remaining architectural change with a plausible 8–18% total worker-CPU gain, but it has not yet earned implementation.
|
||||
2. Gameplay reuse of exact density samples across adjacent tiles. The existing mesher already has most of the consumer-side plumbing, and the theoretical duplicate-sample ceiling is surprisingly large.
|
||||
3. A canonical numeric ABI for world-deciding math. This is primarily correctness, not performance: `/fp:precise` does not currently close the determinism requirement.
|
||||
|
||||
I would also harden the traversal contract. The canonical capability gate passes, but the code and documentation do not support the stronger claim that every composed world has a structurally guaranteed entry-to-exit route.
|
||||
|
||||
All performance percentages below distinguish:
|
||||
|
||||
- **Measured:** directly present in the supplied profiles or commit evidence.
|
||||
- **Inferred:** derived from those measurements; not yet benchmarked.
|
||||
|
||||
---
|
||||
|
||||
# 1. Correction of the orchestrator mapping
|
||||
|
||||
There is no table in `DESIGN-SOL-2026-09-15.md` at this HEAD. There are three mapping bullets at [DESIGN-SOL-2026-09-15.md:9](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/DESIGN-SOL-2026-09-15.md:9>). I have treated those as the requested rows.
|
||||
|
||||
| Orchestrator note | Verdict | Correction |
|
||||
|---|---|---|
|
||||
| Determinism risk from `FMath::Sin/Cos` and UCRT FMA3 dispatch | **Substantively correct; count is stale/ambiguous** | The risk is real and `/fp:precise` does not fix library implementations. The current source contains 42 direct `Sin/Cos` invocations in terrain generation if legacy and fused implementations are both counted, plus eight in deterministic content placement—not simply 34. |
|
||||
| “The largest gap is the loop direction” | **Plausible, not proved** | Every tunnel sample does query a 2D spatial bucket and iterate candidates, but the exclusive lookup machinery was only about 5% in the newer profile. The old 27% figure was inclusive of the feature kernels. Feature-major batching may still win through SIMD, hoisting and locality, but candidate lookup alone cannot justify a rewrite. |
|
||||
| Already shares built-first traversal, hash RNG, collision gate, partial narrow-band skipping | **Mixed** | Collision gating and narrow-band/reach proofs are real. Hash-keyed randomness is dominant but not universal. Traversal is structurally constructed for major archetypes, but the universal guarantee is not established, especially for fallback passage landings and lateral-region composition. |
|
||||
|
||||
## 1.1 Determinism note
|
||||
|
||||
The direct terrain-generation `Sin/Cos` sites are:
|
||||
|
||||
- [VoxelGenerator.cpp:1129](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:1129>), `1178`, `1377`, `1456`, `1557`, `1612`, `4708`, `4738`, `5723`.
|
||||
- [VoxelDensityOpStack.cpp:7507](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelDensityOpStack.cpp:7507>) and `7609`.
|
||||
- [VoxelHeightOpStack.cpp:232](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelHeightOpStack.cpp:232>).
|
||||
- [VoxelCaveMorphology.cpp:2931](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelCaveMorphology.cpp:2931>), `3266`, `6826`, `6843`, `7367`, `7487–7488`, `7520–7521`.
|
||||
- [VoxelStrateManager.cpp:1830](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelStrateManager.cpp:1830>), `1993`, `2008`, `2021–2022`, `2032–2033`, `2045`, `2087`, `2142`.
|
||||
|
||||
The duplication matters: several calls belong to parallel legacy/fused implementations, so a raw count overstates unique runtime work while still accurately describing the audit surface.
|
||||
|
||||
The engine source resolves the uncertainty conclusively:
|
||||
|
||||
- `FMath` inherits `FPlatformMath` at [UnrealMathUtility.h:269](</E:/Program Files/Epic Games/UE_5.7/Engine/Source/Runtime/Core/Public/Math/UnrealMathUtility.h:269>).
|
||||
- Win64 aliases `FPlatformMath` to `FMicrosoftPlatformMathBase` at [WindowsPlatformMath.h:18](</E:/Program Files/Epic Games/UE_5.7/Engine/Source/Runtime/Core/Public/Windows/WindowsPlatformMath.h:18>).
|
||||
- The Microsoft class inherits the generic implementation and does not override `Sin` or `Cos` at [MicrosoftPlatformMath.h:12](</E:/Program Files/Epic Games/UE_5.7/Engine/Source/Runtime/Core/Public/Microsoft/MicrosoftPlatformMath.h:12>).
|
||||
- Generic `Sin` calls `sinf` and generic `Cos` calls `cosf` at [GenericPlatformMath.h:521](</E:/Program Files/Epic Games/UE_5.7/Engine/Source/Runtime/Core/Public/GenericPlatform/GenericPlatformMath.h:521>) and [GenericPlatformMath.h:530](</E:/Program Files/Epic Games/UE_5.7/Engine/Source/Runtime/Core/Public/GenericPlatform/GenericPlatformMath.h:530>).
|
||||
|
||||
So, on UE 5.7 Win64, `FMath::Sin/Cos` use the MSVC/UCRT functions. They do not use Unreal’s polynomial `FMath::SinCos(float)`, which is separate engine code at [UnrealMathUtility.h:752](</E:/Program Files/Epic Games/UE_5.7/Engine/Source/Runtime/Core/Public/Math/UnrealMathUtility.h:752>).
|
||||
|
||||
Microsoft documents that the x64 CRT selects FMA3 implementations at startup based on CPU support and that the alternate algorithms can produce different results. [`_set_FMA3_enable` documentation](https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/get-fma3-enable-set-fma3-enable?view=msvc-170).
|
||||
|
||||
The existing cross-platform test already recognizes the libm hole at [VoxelForgeCrossPlatformTest.cpp:158](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/Tests/VoxelForgeCrossPlatformTest.cpp:158>), but it reports risk rather than enforcing an identical field digest. Therefore constraint 1 is not currently proved.
|
||||
|
||||
## 1.2 “Largest gap is loop direction”
|
||||
|
||||
What the note gets right:
|
||||
|
||||
- A cache builds 2D chunk buckets at [VoxelCaveMorphology.cpp:1325](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelCaveMorphology.cpp:1325>).
|
||||
- Each sample calls `VF_ForEachSpatialCandidate`, including `GetRange`, at [VoxelCaveMorphology.cpp:1462](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelCaveMorphology.cpp:1462>).
|
||||
- Tunnel SDF and world-core evaluation perform separate candidate walks at [VoxelCaveMorphology.cpp:7847](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelCaveMorphology.cpp:7847>) and [VoxelCaveMorphology.cpp:8042](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelCaveMorphology.cpp:8042>).
|
||||
- The canonical fused path remains scalar per sample: `GetDensityAt` calls `GetDensityWithParams` at [VoxelGenerator.cpp:3386](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:3386>).
|
||||
- The mesher calls `GetDensityAt` once for every required grid sample at [VoxelMarchingCubesMesher.cpp:694](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:694>).
|
||||
|
||||
What is overstated:
|
||||
|
||||
- The `1d824aa` sampler’s `VF_ForEachSpatialCandidate` share was inclusive of work executed inside the visitor.
|
||||
- The later `410788f` exclusive attribution puts `GetRange` around 3.8% and `VF_ForEachSpatialCandidate` itself around 1.2%.
|
||||
- Thus the directly removable search overhead is approximately 5% of the old profile, not 27%.
|
||||
- The expensive swept-tunnel work remains inside those callbacks. A loop inversion must make that work cheaper—not merely remove the lookup—to justify itself.
|
||||
|
||||
“Coarse LOD from fine data is absent” is factually true but not evidence of a gap. Direct evaluation at every LOD is a deliberate clipmap invariant: it maintains roughly constant tile complexity and does not force far terrain to materialize at LOD0. See [ARCHITECTURE.md:708](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/ARCHITECTURE.md:708>).
|
||||
|
||||
“Shared halo samples are absent” needs qualification:
|
||||
|
||||
- Every tile already has a single local density grid plus a one-sample margin; cell corners and normals are not resampled. See [VoxelMarchingCubesMesher.cpp:656](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:656>) and [ARCHITECTURE.md:652](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/ARCHITECTURE.md:652>).
|
||||
- `FVoxelSharedDensityGrid` already exists and the mesher can consume it at [VoxelMarchingCubesMesher.cpp:660](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:660>).
|
||||
- It is populated only by the editor exploration commandlet, culminating in `SetSharedDensityGrid` at [VoxelForgeExploreCommandlet.cpp:1905](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForgeEditor/Private/VoxelForgeExploreCommandlet.cpp:1905>).
|
||||
- Gameplay does not share density samples across neighboring tiles.
|
||||
|
||||
So the missing opportunity is specifically **cross-tile gameplay reuse**, not basic grid or halo caching.
|
||||
|
||||
## 1.3 “Already shared”
|
||||
|
||||
### Traversal: partial, not universal
|
||||
|
||||
The current system has real construction-first features: walk-critical tunnel forests, D-floors, option-B winding and structural posts. That part of the mapping is fair.
|
||||
|
||||
But two facts prevent me from calling the hard guarantee closed:
|
||||
|
||||
1. `ARCHITECTURE.md` says every landing receives a deterministic connector to the origin landing at [ARCHITECTURE.md:518](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/ARCHITECTURE.md:518>). Current code explicitly says the opposite: a source-fit landing remains local and no radial connector is synthesized at [VoxelStrateManager.cpp:2108](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelStrateManager.cpp:2108>). The document is stale or the implementation regressed from its stated contract.
|
||||
|
||||
2. The lateral-region audit remains at 4/16 connected cases after landing, including region-zero controls, at [ARCHITECTURE.md:122](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/ARCHITECTURE.md:122>).
|
||||
|
||||
`VF_SuggestLandingPoint` can make a local landing safe when it succeeds, but the documentation says unsupported/no-footing cases fall back to historical random reach at [ARCHITECTURE.md:512](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/ARCHITECTURE.md:512>). That is not a construction proof.
|
||||
|
||||
The canonical 20,830 / 10,909 gate is valuable regression protection. It is not a proof across seeds, composer rolls and archetype mixtures.
|
||||
|
||||
### Randomness: mostly hash-keyed
|
||||
|
||||
Most feature decisions are keyed hashes. The layout shuffle still uses one stateful `FRandomStream` after sorting the asset pool at [VoxelStrateManager.cpp:1145](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelStrateManager.cpp:1145>) and [VoxelStrateManager.cpp:1157](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelStrateManager.cpp:1157>).
|
||||
|
||||
That is deterministic for the same build and pool. It is nevertheless order-coupled: adding one pool choice can shift every later choice. It therefore falls short of the first answer’s “stable ID plus counter-based randomness” ideal, though it does not presently violate the stated cross-machine rule.
|
||||
|
||||
### Collision gate: correct
|
||||
|
||||
The implementation now treats actual collision-cook completion as admission authority. This is stronger than speculative prefetch. It satisfies the hard safety part even under overload by stopping motion.
|
||||
|
||||
What remains from the first answer is UX improvement: predictive fall/shaft prefetch could make the gate engage less often. It is not required for correctness.
|
||||
|
||||
### Narrow bands: correct and substantial
|
||||
|
||||
The sealed-solid proof, tunnel-core reach, passage-carving reach, worm block proof, cached topology and lateral-region preparation all exist. They are not “partial” in the sense of being superficial; §8.10 records a considerable exact-skip system.
|
||||
|
||||
---
|
||||
|
||||
# 2. Ideas that survive contact with the code
|
||||
|
||||
## Ranked recommendations
|
||||
|
||||
| Rank | Change | Share of 27.6 s it could touch | Expected net gain | Rough size |
|
||||
|---|---|---:|---:|---:|
|
||||
| 1 | Tile-wide, feature-major lowering for tunnels | **35–50% inferred**: 9.7–13.8 s | **8–18% inferred**: 2.2–5.0 s | 0.8–1.5 KLOC prototype; 3–6 KLOC production |
|
||||
| 2 | Cross-tile exact density-grid reuse | Up to **22.9% theoretical/inferred**: 6.3 s; actual unknown | **5–12% inferred** if reuse is high | 0.6–1.2 KLOC prototype; 1.5–3 KLOC production |
|
||||
| 3 | Canonical numeric ABI | Probably under 5% runtime touched; correctness is the value | Approximately neutral, perhaps −1% to +1% inferred | 0.3–0.8 KLOC plus tests |
|
||||
| 4 | Traversal-plan contract hardening | 0% of static worker time | Correctness, not speed | 1–3 KLOC plus seed-matrix tests |
|
||||
| 5 | Exact base-density reuse and canonical edit compaction | **0% measured** in the static scenario | Potentially large only after repeated edits; unmeasured | 1.5–3 KLOC |
|
||||
| 6 | Deadline/fall-aware prefetch | Moving-streaming work only; unknown | Gate-engagement and latency improvement, not established CPU savings | 1–2 KLOC |
|
||||
| 7 | Topology-aware underground visibility | Far streaming only; unknown | Speculative | Large |
|
||||
|
||||
The `1d824aa` percentages are historical measured shares from a 29.30 s build, not additive and not exact for the current 27.6 s build. Scaling them onto 27.6 s is explicitly inferred.
|
||||
|
||||
## 2.1 Tile-wide, feature-major lowering
|
||||
|
||||
### Where it plugs in
|
||||
|
||||
Add a tile/grid evaluation entry point beside:
|
||||
|
||||
- `UVoxelGenerator::GetDensityAt` at [VoxelGenerator.cpp:2799](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:2799>).
|
||||
- `UVoxelGenerator::GetDensityWithParams` at [VoxelGenerator.cpp:3921](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:3921>).
|
||||
- The mesher grid-fill loop at [VoxelMarchingCubesMesher.cpp:694](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:694>).
|
||||
|
||||
Conceptually:
|
||||
|
||||
```text
|
||||
composer/operator stack
|
||||
↓ lowering
|
||||
canonical tile program
|
||||
↓
|
||||
tile-local SoA buffers
|
||||
↓
|
||||
existing density grid
|
||||
↓
|
||||
existing two-pass marching cubes
|
||||
```
|
||||
|
||||
It should replace only the repeated scalar execution of the canonical tunnel-network graph. It should keep:
|
||||
|
||||
- `FVoxelOpStack` as the authoring language.
|
||||
- `BuildTunnelNetworkStack` and the fused-evaluator designation.
|
||||
- `FChunkSDFCache` and its search-box validity.
|
||||
- The warped tunnel SDF and world-space core as separate semantic stages.
|
||||
- The common structural tail, boundary seal and edit layer.
|
||||
- Scalar fallback for custom recipes, fractional point queries and unsupported stacks.
|
||||
- The existing marching-cubes grid, margin and two-pass cell loop.
|
||||
|
||||
The production algorithm I would try is:
|
||||
|
||||
1. Prepare the same chunk/tile context once.
|
||||
2. Allocate thread-local SoA arrays for density, warped coordinates, nearest-room ownership and world-core handoff flags.
|
||||
3. Compute sample-invariant and column-invariant data in batches.
|
||||
4. Iterate rooms/tunnels in existing stable feature-index order.
|
||||
5. Clip each feature’s conservative 3D influence bounds to the tile lattice.
|
||||
6. Traverse X-contiguous spans and evaluate four or eight positions together where the formula permits.
|
||||
7. Apply smooth unions in the same per-sample feature order.
|
||||
8. Run later operators and the structural tail in their canonical order.
|
||||
9. Write the existing `DensityGrid`.
|
||||
|
||||
The index builder inserts item indices in ascending feature order at [VoxelCaveMorphology.cpp:1445](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelCaveMorphology.cpp:1445>). Maintaining that order matters because smooth-min and other floating operations are order-sensitive.
|
||||
|
||||
### Measured basis
|
||||
|
||||
Historical remove-one shares:
|
||||
|
||||
- Tunnel core: **20.84% measured**, corresponding to 5.75 s when scaled to 27.6 s.
|
||||
- Tunnel SDF: **12.64% measured**, corresponding to 3.49 s.
|
||||
- Room SDF: **4.24% measured**, corresponding to 1.17 s.
|
||||
- Cave warp: **6.68% measured**, corresponding to 1.84 s.
|
||||
- Passage carving/structure can be incorporated later.
|
||||
|
||||
The tunnel core plus tunnel SDF therefore account for **33.48% measured historically**, or 9.24 s scaled to the current total. A batch evaluator cannot eliminate that geometry; it can only reduce its loop, load, branch and arithmetic cost.
|
||||
|
||||
The newer exclusive profile says lookup machinery itself was approximately **5% measured historically**, about 1.38 s scaled. This is why “invert the loops” alone is not enough. The hoped-for gain must come from:
|
||||
|
||||
- Hoisting tunnel metadata.
|
||||
- Avoiding repeated lambdas/cache lookups and repeated world-coordinate construction.
|
||||
- SIMD across adjacent X samples.
|
||||
- Reusing centerline segment data across samples.
|
||||
- Better sequential reads and writes.
|
||||
- Clipping feature loops in 3D rather than retrieving a 2D bucket and rejecting on Z for every sample.
|
||||
|
||||
### Risks
|
||||
|
||||
- **Capability:** high. The core/SDF handoff and room-floor ownership are subtle. The rejected merge shows this is load-bearing.
|
||||
- **Determinism:** medium-high. SIMD must use the same fixed instruction path on every x64 machine. Any conservative span skip must be reference-checked and field-identical.
|
||||
- **Composer:** low if this remains a lowering backend. High if it becomes a separate hand-authored terrain definition.
|
||||
- **§8.10:** high implementation risk. It must preserve search-box validity, lateral per-chunk preparation, reach masks, the margin grid, cancellation points, TLS reuse and direct evaluation at each LOD.
|
||||
- **Maintenance:** high if the scalar and batched semantics fork. Shared primitive functions and generated/lowered stage descriptions are preferable to copying thousands of lines.
|
||||
|
||||
I would not generalize it to every archetype first. TunnelNetwork/Underwater already have a canonical fused designation and dominate the measured cost.
|
||||
|
||||
## 2.2 Exact density sharing across gameplay tiles
|
||||
|
||||
This is a more concrete surviving piece of the “shared bricks/halos” proposal than a full sparse-brick rewrite.
|
||||
|
||||
### Why it may be larger than it looks
|
||||
|
||||
A normal full-resolution tile has 32 cells but samples a 35³ grid because of the cell boundary and one-point normal margin.
|
||||
|
||||
For an ideal dense array of adjacent same-LOD tiles, the asymptotic duplicate fraction is:
|
||||
|
||||
```text
|
||||
1 − 32³ / 35³ = 23.6%
|
||||
```
|
||||
|
||||
This is an inferred theoretical ceiling. Actual reuse will be lower because of clipmap shell boundaries, Z bands, cancellation and tile completion order.
|
||||
|
||||
The old sampled profile attributed **96.9% measured, inclusive** to `GetDensityAt`. If duplicated coordinates had average cost, the absolute theoretical CPU ceiling would be approximately:
|
||||
|
||||
```text
|
||||
23.6% × 96.9% = 22.9% of total
|
||||
```
|
||||
|
||||
That is 6.3 s of the 27.6 s total. It is not an expected gain; it is the reason this deserves instrumentation.
|
||||
|
||||
### Where it plugs in
|
||||
|
||||
- Existing consumer support: `FVoxelSharedDensityGrid` in [VoxelMarchingCubesMesher.h:25](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelMarchingCubesMesher.h:25>).
|
||||
- Existing copy path: [VoxelMarchingCubesMesher.cpp:660](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:660>).
|
||||
- Scheduler and tile lifetime: `AVoxelWorld::LoadTile`, task completion and cancellation paths.
|
||||
- Edit revision: `UVoxelDiffLayer::ModsVersion` and the snapshot mechanism.
|
||||
|
||||
I would prototype an **opportunistic completed-neighbor slab cache**, not a global per-sample hash table:
|
||||
|
||||
- Store exact float boundary slabs from recently completed tiles.
|
||||
- Key by generator owner, layout/generation epoch, final-field edit revision, sample step and integer world-lattice bounds.
|
||||
- Before sampling a tile, copy all available overlap slabs with simple block loops.
|
||||
- Evaluate only missing rectangular regions.
|
||||
- Never wait for a neighboring tile.
|
||||
- Evict outside the desired/in-flight neighborhood.
|
||||
|
||||
The current `SetSharedDensityGrid` member pointer is suitable for the single-threaded editor commandlet, not for concurrent gameplay tasks. Production should pass immutable donor views with each mesh request, not mutate shared mesher state.
|
||||
|
||||
Do not reuse the quantized density-volume capture for meshing. Its bytes are sufficient for lighting volume capture, not exact marching-cubes interpolation.
|
||||
|
||||
### Risks
|
||||
|
||||
- **Streaming:** a support tile must never wait for a batch or neighbor.
|
||||
- **Revision correctness:** an edit or layout rebuild must prevent reuse of stale final densities.
|
||||
- **Memory:** retaining complete 35³ float grids is ~171 KB per tile. Store bounded slabs or short-lived grids, not every loaded tile indefinitely.
|
||||
- **Concurrency:** avoid global locks in the sample loop.
|
||||
- **Determinism:** exact copied floats are safe if the key is complete. A stale cache hit is a geometry error.
|
||||
- **§8.10:** preserve task cancellation and the thread-local destination grid; copying merely replaces a subset of its fill operations.
|
||||
|
||||
Expected net gain is **5–12% inferred**, conditional on actual ready-before-use reuse being high. It may also fail completely if most neighboring tasks execute concurrently and finish too late to donate samples.
|
||||
|
||||
## 2.3 Canonical numeric ABI
|
||||
|
||||
This is the surviving portion of the first answer’s WorldPlan recommendation with the highest correctness value.
|
||||
|
||||
The code already has a substantial plan/layout layer, cached topology, per-strate parameters and operator recipes. I would not introduce a second “WorldPlan” representation. I would add:
|
||||
|
||||
- A generation format/version ID.
|
||||
- Stable feature IDs for topology objects and composer choices.
|
||||
- Canonical deterministic `SinCos`, normalization and checked coordinate conversions.
|
||||
- Golden plan, density and mesh digests.
|
||||
- An explicit floating-environment contract for worker tasks.
|
||||
|
||||
This touches very little of the 27.6 s. The per-sample `Sin` calls may be measurable, but the topology and passage calls are cold. Expect approximately neutral performance.
|
||||
|
||||
## 2.4 Traversal contract hardening
|
||||
|
||||
The least invasive structural fix is to make passage creation return one of two outcomes:
|
||||
|
||||
- A landing with a certificate identifying the existing structural feature/component to which it belongs.
|
||||
- A newly constructed explicit connector included in the mandatory route graph.
|
||||
|
||||
A failed `VF_SuggestLandingPoint` should cause a deterministic retry/recomposition or explicit connector. It should not silently fall back to an unproved random reach.
|
||||
|
||||
For lateral compositions, generate and reserve the mandatory entry-to-exit spine at the parent level before regional creative stacks. Regional terrain may join or decorate it but not own its existence.
|
||||
|
||||
This preserves the composer and adds no material density-loop cost. The proof is a seed/archetype/composition matrix, not merely the canonical scenario.
|
||||
|
||||
## 2.5 Edit compaction and base-grid reuse
|
||||
|
||||
The current edit layer duplicates an edit into each affected chunk and evaluates the list serially at [VoxelDiffLayer.cpp:361](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelDiffLayer.cpp:361>). There is no compaction into canonical per-brick deltas.
|
||||
|
||||
This does not touch the standard static 27.6 s scenario: **0% measured**.
|
||||
|
||||
For mining-heavy sessions, I would eventually keep:
|
||||
|
||||
- Immutable generated base samples for a recently remeshed edited tile.
|
||||
- A server-sequenced edit stream.
|
||||
- Canonical per-tile delta compaction after a threshold.
|
||||
- Separate base-generation, edit, mesh and collision revision numbers.
|
||||
|
||||
Do this only after creating an edit-heavy benchmark. Depending on edit-list length, it might remove 30–80% of edit-layer work or much of repeated base evaluation, but that is wholly inferred and should not be sold using the static benchmark.
|
||||
|
||||
---
|
||||
|
||||
# 3. Proof-of-worth ladders
|
||||
|
||||
## Change A: tile-wide feature-major tunnel evaluation
|
||||
|
||||
### Rung 1 — Instrument the current evaluator
|
||||
|
||||
Use the existing counters:
|
||||
|
||||
- `CaveRoomCandidates/Evaluated`
|
||||
- `CaveTunnelCandidates/Evaluated`
|
||||
- `TunnelCoreCandidates/Evaluated`
|
||||
|
||||
They are recorded in [VoxelCaveMorphology.cpp:7765](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelCaveMorphology.cpp:7765>), [VoxelCaveMorphology.cpp:7844](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelCaveMorphology.cpp:7844>) and [VoxelCaveMorphology.cpp:8042](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelCaveMorphology.cpp:8042>), then logged per tile at [VoxelWorld.cpp:4961](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelWorld.cpp:4961>).
|
||||
|
||||
Add profile-only counters for:
|
||||
|
||||
- `GetRange` calls/hits/fallbacks and cycles.
|
||||
- Candidate visits rejected by 3D AABB.
|
||||
- Candidate visits rejected by bounding sphere.
|
||||
- Actual swept-tunnel evaluations.
|
||||
- Centerline segment iterations.
|
||||
- For each tile, the predicted feature-major lattice visits obtained by clipping every feature’s conservative 3D bounds to that tile.
|
||||
- Separate static/moving and LOD0/all-LOD totals.
|
||||
|
||||
Do not time every sample with a heavyweight timer; aggregate cycles around whole loops or rely on sampling.
|
||||
|
||||
**Kill immediately if:**
|
||||
|
||||
- Exclusive candidate/rejection machinery remains below 6–7% of worker time, and
|
||||
- Predicted feature-major visits are at least 80% of current expensive evaluations, and
|
||||
- There is no clear X-lane SIMD opportunity in the swept kernel.
|
||||
|
||||
That combination caps the likely gain below what a large rewrite deserves.
|
||||
|
||||
**Proceed if either:**
|
||||
|
||||
- Conservative feature spans reduce candidate/kernel visits by at least 2×, or
|
||||
- The combined measured loop/branch/kernel share with realistic SIMD opportunity is at least 12% of total worker time.
|
||||
|
||||
Also require projected scratch below roughly 2 MB per worker for a single-tile prototype.
|
||||
|
||||
### Rung 2 — Tunnel SDF only
|
||||
|
||||
Behind a diagnostic console switch:
|
||||
|
||||
- Batch the warped tunnel SDF.
|
||||
- Leave `EvaluateTunnelCoreWorld` entirely unchanged.
|
||||
- Preserve room-mouth ownership and feature order.
|
||||
- Keep scalar evaluation as the reference path.
|
||||
- Add a debug mode comparing every produced sample with the reference field.
|
||||
|
||||
The tunnel SDF had a **12.64% measured historical share**, so the prototype has enough budget to show a useful result without touching the load-bearing core.
|
||||
|
||||
**Kill immediately on:**
|
||||
|
||||
- Player-fit below 20,830.
|
||||
- Walk-reachable below 10,909.
|
||||
- Loss of connected status.
|
||||
- Any nondeterministic digest across repeated runs or machines.
|
||||
- A span/proof skip that changes a reference density bit.
|
||||
- A violation of the two-region cache window, margin grid or cancellation contracts.
|
||||
|
||||
**Performance kill criterion:**
|
||||
|
||||
- Less than 3% total worker-CPU improvement in either static or moving, or
|
||||
- More than 3% regression in generation/collision-ready p95.
|
||||
|
||||
**Justification to continue:**
|
||||
|
||||
- At least 5% total worker-CPU improvement in both static and moving, with no material p95 or memory regression.
|
||||
|
||||
The switch-on/off result is only an internal decision aid. It does not satisfy final acceptance.
|
||||
|
||||
### Rung 3 — Separate batch stages for world core and passage work
|
||||
|
||||
Add the world-space core as a second, semantically separate batch stage. Do not merge it with the warped SDF. Then consider passage carving/structure if the same infrastructure applies.
|
||||
|
||||
Full adoption requires:
|
||||
|
||||
- Default-on build compared against the untouched `cf9904e` DLL.
|
||||
- At least 10% worker-CPU improvement in both static and moving, or at least 8% plus a material collision-ready/p95 improvement.
|
||||
- Canonical capability counts and connection.
|
||||
- Skip/reference tests clean.
|
||||
- Memory and task-cancellation behavior within existing bounds.
|
||||
- No custom-recipe or lateral-composition semantic fork.
|
||||
|
||||
## Change B: cross-tile exact density reuse
|
||||
|
||||
### Rung 1 — Count exact duplicate opportunities
|
||||
|
||||
Record, for every generated tile:
|
||||
|
||||
- Origin, step, sampled Z band and generation/edit revision.
|
||||
- Task start and completion order.
|
||||
- Count of requested lattice coordinates.
|
||||
- Count of distinct lattice coordinates across the scenario.
|
||||
- Count whose donor tile had already completed before the consumer started.
|
||||
- Reuse grouped by LOD, same/cross-LOD, static/moving and density-cost bucket.
|
||||
|
||||
This can be computed offline from a compact tile trace; there is no need for a runtime hash set of every sample in the production path.
|
||||
|
||||
**Kill if:**
|
||||
|
||||
- Exact duplicate coordinates are below 12% of all density calls, or
|
||||
- Fewer than half of duplicates have a completed donor in time, or
|
||||
- Time-weighted eligible duplicates represent less than 5% of worker CPU.
|
||||
|
||||
**Proceed if:**
|
||||
|
||||
- At least 15% of calls are exact duplicates and at least 60% are available opportunistically, or
|
||||
- The conservative time-weighted saving ceiling exceeds 10% of total worker CPU.
|
||||
|
||||
### Rung 2 — LOD0 completed-neighbor slabs
|
||||
|
||||
Prototype only LOD0:
|
||||
|
||||
- Exact float slabs.
|
||||
- Immutable per-request donor views.
|
||||
- No waiting.
|
||||
- No cross-LOD reuse.
|
||||
- Strict owner/layout/edit revision key.
|
||||
- Bounded memory, preferably 32–64 MB.
|
||||
- Missing samples follow the current path.
|
||||
|
||||
**Kill immediately on any field/mesh digest difference.** This should be a pure reuse, not a field-changing implementation.
|
||||
|
||||
**Performance kill criterion:**
|
||||
|
||||
- Less than 4% worker-CPU improvement in either static or moving.
|
||||
- More than 3% collision-ready or generation p95 regression.
|
||||
- Memory above the agreed cap.
|
||||
- Significant loss of task concurrency.
|
||||
|
||||
**Proceed if:**
|
||||
|
||||
- At least 6% worker-CPU improvement in both scenarios.
|
||||
- No readiness regression.
|
||||
- No stale-hit failure under edit/rebuild/cancellation stress.
|
||||
|
||||
### Rung 3 — Full same-step gameplay reuse
|
||||
|
||||
Extend to same-step tiles at all useful clip levels, with revisioned invalidation and bounded eviction. Cross-LOD reuse is optional and should be separately justified.
|
||||
|
||||
Adopt only if the final default build beats `cf9904e` by at least 7–8% overall or produces a slightly smaller CPU gain plus a clear collision-readiness improvement. Remove the diagnostic switch only after the committed-DLL comparison.
|
||||
|
||||
---
|
||||
|
||||
# 4. Numeric determinism audit
|
||||
|
||||
## 4.1 Critical: CRT `sin/cos`
|
||||
|
||||
**Severity: critical.**
|
||||
|
||||
These functions affect:
|
||||
|
||||
- Passage control points and landing directions.
|
||||
- Tunnel winding and pinch orientation.
|
||||
- Room/feature placement.
|
||||
- Per-sample line/rib/surface detail.
|
||||
- Walkability thresholds.
|
||||
|
||||
The UCRT can select different implementations in the same Win64 binary based on CPU features. Windows and Linux libm implementations may also differ.
|
||||
|
||||
Minimal fix:
|
||||
|
||||
1. Introduce one canonical float `SinCos` implementation with explicitly bounded input, fixed range reduction and fixed operation grouping.
|
||||
2. The UE float `FMath::SinCos` polynomial is a reasonable prototype because it is engine arithmetic rather than CRT dispatch. I would wrap or vendor it so the numeric contract is explicit.
|
||||
3. Replace paired `Sin`/`Cos` calls together.
|
||||
4. Use the same canonical sine output for individual sine-only sites.
|
||||
5. Pin golden input/output bit patterns and world-plan/field digests.
|
||||
6. Retune if necessary; a new build may change the field.
|
||||
|
||||
Calling `_set_FMA3_enable(0)` is not a complete fix. It is process-global, Windows-specific and does not align UCRT with glibc.
|
||||
|
||||
## 4.2 Medium: `sqrt` and implicit vector normalization
|
||||
|
||||
`FMath::Sqrt` also reaches `sqrtf` at [GenericPlatformMath.h:552](</E:/Program Files/Epic Games/UE_5.7/Engine/Source/Runtime/Core/Public/GenericPlatform/GenericPlatformMath.h:552>). On x64 this will commonly become a hardware square-root instruction, which is much less alarming than `sin/cos`, but the source-level contract still delegates to the platform library/compiler.
|
||||
|
||||
World-deciding examples include:
|
||||
|
||||
- Morphology topology/walkability: [VoxelCaveMorphology.cpp:639](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelCaveMorphology.cpp:639>), `785`, `2171`, `2188`, `3019`, `3027`, `4919`, `5101`, `5355`.
|
||||
- Tunnel/room field evaluation: `7704`, `8356`, `8550`.
|
||||
- Passage/layout geometry: [VoxelStrateManager.cpp:666](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelStrateManager.cpp:666>), `2393`, `2406`, `3701`, `3804`.
|
||||
- Composer decisions: [VoxelStrateComposer.cpp:1572](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelStrateComposer.cpp:1572>), `1606–1608`, `4141`, `4172`, `4662–4665`, `4849`, `5016–5018`.
|
||||
- Numerous SDF sites in `VoxelGenerator.cpp` and `VoxelDensityOpStack.cpp`.
|
||||
- `FVector::Size`, `Dist` and `Normalize`, for example passage direction normalization at [VoxelStrateManager.cpp:2085](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelStrateManager.cpp:2085>).
|
||||
|
||||
Minimal fix:
|
||||
|
||||
- Replace square-root comparisons with squared comparisons where mathematically equivalent.
|
||||
- Canonicalize only the remaining square roots that affect topology, integer conversion or threshold decisions.
|
||||
- Verify emitted Win64 instructions and run golden vectors before attempting to replace every SDF square root. I am not claiming a demonstrated cross-CPU `sqrt` divergence here.
|
||||
|
||||
No direct runtime uses of `FMath::Pow`, `Exp`, `Log`, `Atan`, `Atan2` or `Tan` were found in the plugin generation sources. That does not rule out transitive engine/library use, but there are no direct call sites to fix.
|
||||
|
||||
## 4.3 High: runtime CPU state and dispatch
|
||||
|
||||
### Confirmed dispatch
|
||||
|
||||
- UCRT FMA3 `sin/cos`: critical, as above.
|
||||
|
||||
### Plugin SIMD
|
||||
|
||||
`VoxelNoise` selects scalar versus SSE at compile time through `VF_NOISE_USE_SIMD` at [VoxelNoise.h:35](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelNoise.h:35>). The hot implementation uses explicit SSE4.1 at [VoxelNoise.h:248](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelNoise.h:248>).
|
||||
|
||||
I found no plugin runtime CPUID dispatch and no ISPC path. The same Win64 build therefore takes the same plugin noise implementation on all supported x64 CPUs.
|
||||
|
||||
The “scalar and SIMD are bit-identical” claim at [VoxelNoise.h:20](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelNoise.h:20>) should still have a pinned exhaustive/random corpus. It is a claim worth continuously testing, not merely documenting.
|
||||
|
||||
### MXCSR rounding mode and FTZ/DAZ
|
||||
|
||||
UE’s integer conversions inherit SSE implementations such as `FloorToInt32` and `RoundToInt32` at [UnrealPlatformMathSSE.h:86](</E:/Program Files/Epic Games/UE_5.7/Engine/Source/Runtime/Core/Public/Math/UnrealPlatformMathSSE.h:86>). Several use rounding-mode-sensitive conversion instructions and assume the normal environment.
|
||||
|
||||
Another library changing a worker thread’s MXCSR rounding mode could change world-to-cell decisions. FTZ/DAZ can likewise alter very small intermediates.
|
||||
|
||||
Minimal fix:
|
||||
|
||||
- At generation-task entry, assert or install a canonical MXCSR configuration and restore the previous state at exit.
|
||||
- Prefer explicit rounding instructions or checked conversion helpers for canonical boundaries.
|
||||
- Test all supported task threads, not only the game thread.
|
||||
|
||||
## 4.4 Critical/high: float-to-integer boundaries
|
||||
|
||||
### Noise lattice conversion
|
||||
|
||||
[VoxelNoise.h:96](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelNoise.h:96>) floors a float and casts it directly to `int32`. The SIMD version uses `_mm_cvttps_epi32` at [VoxelNoise.h:255](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Public/VoxelNoise.h:255>).
|
||||
|
||||
For NaN or out-of-range values, scalar C++ conversion and SSE conversion do not provide a suitable canonical world rule.
|
||||
|
||||
**Severity: high**, especially with `WorldRadiusVoxels == 0` permitting an unbounded world.
|
||||
|
||||
Fix:
|
||||
|
||||
- Define a supported integer-coordinate envelope.
|
||||
- Reject or clamp malformed/out-of-range queries before noise evaluation.
|
||||
- Use one checked floor/wrap policy shared by scalar and SIMD paths.
|
||||
- Test the boundary values and their immediate float neighbors.
|
||||
|
||||
### Chunk routing
|
||||
|
||||
`GetDensityAt` converts public float coordinates to chunk coordinates without a finite/range guard at [VoxelGenerator.cpp:2836](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:2836>).
|
||||
|
||||
Internal mesher calls are bounded integer coordinates, but other callers need not be.
|
||||
|
||||
Fix:
|
||||
|
||||
- Give the mesher an integer-lattice entry point.
|
||||
- Keep the public float query checked.
|
||||
- Decide explicitly what happens outside representable chunk space.
|
||||
|
||||
The edit layer already demonstrates the correct pattern at [VoxelDiffLayer.cpp:220](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelDiffLayer.cpp:220>).
|
||||
|
||||
### Procedural cell selection
|
||||
|
||||
`VoxelCaveMorphology.cpp` contains many `FloorToInt` cell selections, including the placement/cache families around `3902–3903`, `3954–3956`, `4038–4039`, `4180–4181`, `4554–4555`, `4683–4684`, `4881–4882`, `5085–5086`, `5195–5196` and `6055–6058`.
|
||||
|
||||
These are safe only while inputs remain finite and representable. They should consume the same declared coordinate envelope.
|
||||
|
||||
### Conservative skip boundaries
|
||||
|
||||
Passage and box proofs use float-to-int lattice conversions, for example around `VoxelStrateManager.cpp:3333–3334`. These are lower severity if they are genuinely one-sided: disagreement must cause a missed skip, never omitted geometry.
|
||||
|
||||
The better formulation is integer/rational lattice `ceil`/`floor` from validated bounds, with explicit outward rounding.
|
||||
|
||||
### Mesh vertex identity
|
||||
|
||||
The mesher deduplicates a vertex by rounding its interpolated world position multiplied by 100 at [VoxelMarchingCubesMesher.cpp:154](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:154>).
|
||||
|
||||
A small numeric difference crossing a half-centimetre rounding boundary can change vertex sharing and triangle/index output even if the surface samples remain on the same side.
|
||||
|
||||
**Severity: medium-high.**
|
||||
|
||||
Minimal fix: identify marching-cubes vertices canonically by integer cell coordinate plus edge axis, not rounded interpolated position. That is both cheaper and independent of interpolation rounding.
|
||||
|
||||
## 4.5 High: worker-side global races
|
||||
|
||||
The following “parse once” flags and their backing values are ordinary globals accessed from worker generation:
|
||||
|
||||
- Generator flags at [VoxelGenerator.cpp:47](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:47>), `73`, `98`, `124`, `143`, `202`.
|
||||
- The parsers are invoked inside every `GetDensityAt` at [VoxelGenerator.cpp:2820](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:2820>).
|
||||
- Mesher flags at [VoxelMarchingCubesMesher.cpp:20](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:20>) and `42`, invoked at `97–98`.
|
||||
|
||||
Multiple first-time worker calls can read/write the booleans concurrently. `FAutoConsoleVariableRef` can also mutate backing integers while workers read them. That is C++ data-race undefined behavior.
|
||||
|
||||
Minimal fix:
|
||||
|
||||
- Parse command-line state once on the game/module initialization thread before generation begins.
|
||||
- Construct an immutable generation-policy snapshot.
|
||||
- Copy that snapshot into each task or generation epoch.
|
||||
- A live CVar change that affects the field must create a new epoch and rebuild; it must not alter active workers in place.
|
||||
- Diagnostics that are intentionally live should use atomics and must not influence the canonical field.
|
||||
|
||||
## 4.6 Critical for multiplayer edits: order-dependent accumulation
|
||||
|
||||
Edits are appended to per-chunk arrays at [VoxelDiffLayer.cpp:253](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelDiffLayer.cpp:253>) and accumulated serially in insertion order at [VoxelDiffLayer.cpp:374](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelDiffLayer.cpp:374>).
|
||||
|
||||
Floating addition is order-dependent. If clients apply replicated edits in packet-arrival order, their density fields can differ.
|
||||
|
||||
Minimal fix:
|
||||
|
||||
- Give every accepted edit a monotonic authoritative server sequence.
|
||||
- Buffer out-of-order edits and evaluate them in that sequence.
|
||||
- Include sequence ranges in edit/density cache revisions.
|
||||
- During compaction, use a canonical order or quantized/fixed-point delta representation.
|
||||
- Never parallel-reduce floating edit contributions without a deterministic reduction tree.
|
||||
|
||||
Current local serial evaluation is stable if every machine receives the exact same order. The network contract must guarantee that.
|
||||
|
||||
## 4.7 Medium: raw-memory fingerprints and padding
|
||||
|
||||
The code hashes raw parameter structs using `FCrc::MemCrc32` at:
|
||||
|
||||
- [VoxelGenerator.cpp:3170](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelGenerator.cpp:3170>) and `3842`.
|
||||
- [VoxelDensityOpStack.cpp:1116](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelDensityOpStack.cpp:1116>), `1119` and `3693`.
|
||||
|
||||
The accompanying comments say a raw-memory CRC cannot produce a false positive. That is incorrect:
|
||||
|
||||
- Padding may be indeterminate, yielding nondeterministic false misses.
|
||||
- CRC32 can collide, yielding a false hit.
|
||||
- If the fingerprint is the only cache identity, a false hit can reuse terrain computed from different parameters.
|
||||
|
||||
Minimal fix:
|
||||
|
||||
- Hash fields individually in declared canonical order.
|
||||
- Normalize bools, floats and zero.
|
||||
- Prefer 64- or 128-bit hashes.
|
||||
- On a hash match, compare the complete canonical key before reusing semantic data.
|
||||
|
||||
This is probably a low-frequency failure, but an absolute determinism/correctness contract should not use “CRC collisions cannot happen” as an invariant.
|
||||
|
||||
## 4.8 Stateful randomness and ordering
|
||||
|
||||
`FRandomStream` layout shuffle is serial and deterministic for the current sorted pool, so it is not presently a cross-thread race.
|
||||
|
||||
Its weakness is evolutionary stability: inserting a pool member changes subsequent random consumption. Since older-build identity is not required, this is not urgent. Stable per-slot keyed selection becomes useful if saved plans, live content packs or replicated compact layouts must survive content additions predictably.
|
||||
|
||||
Current spatial candidate iteration is serial and ordered. I found no current parallel reduction of SDFs or edit values. A feature-major rewrite must preserve per-sample feature order or deliberately become a field-changing new canonical evaluator.
|
||||
|
||||
## 4.9 Uninitialized data and timing
|
||||
|
||||
I inspected the important `SetNumUninitialized` sites and found no confirmed uninitialized read:
|
||||
|
||||
- Spatial-index bound arrays are filled for every valid item before use at [VoxelCaveMorphology.cpp:1343](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelCaveMorphology.cpp:1343>).
|
||||
- Index storage is completely filled from prefix offsets at `1443–1455`.
|
||||
- The mesher samples only the Z band subsequently read; this is documented at [VoxelMarchingCubesMesher.cpp:680](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp:680>).
|
||||
|
||||
The raw struct padding used by `MemCrc32` is the confirmed uninitialized-byte concern.
|
||||
|
||||
I found no density value chosen by wall-clock timing. Task timing controls which tiles become ready first, but epoch, cancellation and collision gates prevent obsolete work from becoming canonical terrain. Cache identity and edit sequencing are the places where timing could become semantic if keys/order are incomplete.
|
||||
|
||||
---
|
||||
|
||||
# 5. What I would not graft onto this code
|
||||
|
||||
- **No wholesale persistent sparse base-density brick store.** It would duplicate a pure procedural evaluator, add large revision/save/memory machinery and invalidate the current direct-LOD clipmap advantage. Transient tile buffers and edited-brick deltas capture most of the value.
|
||||
|
||||
- **No coarse LOD derived from mandatory fine voxels.** It would make far terrain require fine generation and undermine the constant-cell-count clipmap at [ARCHITECTURE.md:708](</E:/Projet Unreal/VoxelM/Plugins/VoxelForge/ARCHITECTURE.md:708>).
|
||||
|
||||
- **No min/max octree classifier as another always-on front end.** The outer classifier and exact per-sample cull already demonstrated that extra proof work can lose. Bounds should cheaply clip batch spans, not become a parallel evaluator.
|
||||
|
||||
- **No warped-SDF/world-core merge.** The rejected capability loss demonstrates that they express different geometry. Batch them separately.
|
||||
|
||||
- **No value-noise worm substitution or worm-focused rewrite.** Worm removal is only about 3%; that budget cannot support architectural work.
|
||||
|
||||
- **No general-purpose interpreted CSG evaluator.** The project has already paid to lower the canonical stack. General composition belongs in the authoring IR, not the sample loop.
|
||||
|
||||
- **No shell/B-rep base terrain or local shell-to-voxel collars.** Arbitrary noise, junctions, mixed archetypes and mining make the watertight seam problem much larger than the likely benefit.
|
||||
|
||||
- **No dual-contouring/transvoxel or mesh-extractor rewrite without an extractor-specific profile.** Density evaluation is the measured bottleneck. Changing topology would risk capability and seams while attacking an unmeasured target.
|
||||
|
||||
- **No GPU-authoritative field or meshing.** Cross-vendor bit identity remains unproved. GPU rendering/culling can be non-authoritative later.
|
||||
|
||||
- **No global concurrent hash map for individual density samples.** Per-sample lookup/locking could repeat the always-on-cull failure. Reuse should operate on rectangular slabs or immutable tile buffers.
|
||||
|
||||
- **No actor/UObject per brick.** Current plain worker data and game-thread publication are the right split.
|
||||
|
||||
- **No second WorldPlan/composer hierarchy.** `FStrateLayout`, recipes, cached topology and the operator stack already constitute one. Add stable IDs and contracts to those structures.
|
||||
|
||||
- **No conservative collision proxy unless gate stalls remain a real UX problem.** The current cook-complete admission gate already supplies correctness. A second proxy system needs a user-visible problem and measurements.
|
||||
|
||||
- **No claim that generated traversal remains guaranteed after arbitrary edits.** If that becomes a requirement, it needs an explicit server product rule: protect, reject or repair the final route.
|
||||
|
||||
---
|
||||
|
||||
# Bottom line
|
||||
|
||||
The owner’s “prove it worthy” stance is appropriate. The code has reached the point where another clever per-sample condition is more likely to lose than win.
|
||||
|
||||
I would authorize three preliminary efforts:
|
||||
|
||||
1. Instrument feature-major work potential using the existing candidate/evaluated counters.
|
||||
2. Trace exact duplicate density coordinates and ready-before-use donors across gameplay tiles.
|
||||
3. Close the numeric ABI hole, beginning with a FMA3-on/off and cross-platform golden corpus.
|
||||
|
||||
Only the first two are performance bets. Neither should become a production rewrite unless it clears its early kill criteria. The numeric work is different: the code currently documents, rather than satisfies, the hard field-determinism requirement.
|
||||
|
||||
# Files read
|
||||
|
||||
No files were changed. I did not build, run tests or launch Unreal.
|
||||
|
||||
Targeted sections read:
|
||||
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\CLAUDE.md`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\CODEMAP.md`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\ARCHITECTURE.md`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\WORK-NEXT.md`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\PERF-SAMPLED.md`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\DESIGN-SOL-2026-09-15.md`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\AUDIT-SOL-2026-09-11.md`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\AUDIT-COLD-2026-09-14.md`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\fable-idea.md`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\VoxelForge.Build.cs`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelGenerator.cpp`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelCaveMorphology.cpp`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelDensityOpStack.cpp`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelStrateManager.cpp`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelMarchingCubesMesher.cpp`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelWorld.cpp`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\VoxelDiffLayer.cpp`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Public\VoxelNoise.h`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Public\VoxelDensityProfile.h`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Public\VoxelMarchingCubesMesher.h`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForge\Private\Tests\VoxelForgeCrossPlatformTest.cpp`
|
||||
- `E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Source\VoxelForgeEditor\Private\VoxelForgeExploreCommandlet.cpp`
|
||||
|
||||
Files inspected through targeted symbol searches:
|
||||
|
||||
- `VoxelContentManager.cpp`
|
||||
- `VoxelHeightOpStack.cpp`
|
||||
- `VoxelStrateComposer.cpp`
|
||||
- `VoxelStrateMeasure.cpp`
|
||||
- `VoxelCaveMorphology.h`
|
||||
- `VoxelDensityPrimitives.h`
|
||||
- `VoxelHeightOp.h`
|
||||
- `VoxelPassageGeometry.h`
|
||||
- `VoxelStrateTypes.h`
|
||||
- `VoxelForgeComposerStructureRollTest.cpp`
|
||||
- `VoxelForgePassageOpenSpaceTest.cpp`
|
||||
- `VoxelForgeScaleDiagnosisTest.cpp`
|
||||
- `VoxelForgeStrateConnectivityTest.cpp`
|
||||
- `VoxelForgeWormBlockSkipTest.cpp`
|
||||
|
||||
UE 5.7 engine source read:
|
||||
|
||||
- `GenericPlatformMath.h`
|
||||
- `UnrealMathUtility.h`
|
||||
- `MicrosoftPlatformMath.h`
|
||||
- `WindowsPlatformMath.h`
|
||||
- `UnrealPlatformMathSSE.h`
|
||||
- `UnrealPlatformMathSSE4.h`
|
||||
|
||||
I also inspected git metadata/diffs for `1d824aa`, `a668833` and `410788f`.
|
||||
@@ -0,0 +1,580 @@
|
||||
# DESIGN-SOL-2026-09-15: "build it every better way", asked cold
|
||||
|
||||
On the owner's idea (see memory `reframe-when-returns-diminish`): after the exact-skip perf push
|
||||
reached diminishing returns, we asked Sol (`gpt-5.6-sol`, reasoning high) a first-principles
|
||||
design question. It was run from an empty folder, read-only, told not to read files, and it read
|
||||
none (0 commands). It got the game's goals and constraints, never the code. The prompt and the
|
||||
answer are verbatim below.
|
||||
|
||||
## Orchestrator notes (checked against the code, 2026-09-15)
|
||||
- **Determinism risk confirmed in our code.** Sol warns against platform transcendentals on the
|
||||
path that decides the world. VoxelForge has 34 `FMath::Sin/Cos` calls in generation code:
|
||||
per-sample strata line/rib detail (`VoxelGenerator.cpp:4708, 4738, 5723`;
|
||||
`VoxelDensityOpStack.cpp:7507, 7609`) and shape building (passage spirals/cascades
|
||||
`VoxelStrateManager.cpp:1830-2142`, option B winding waves `VoxelCaveMorphology.cpp:6826-6843`,
|
||||
tunnel pinches `7487-7521`). The MSVC C runtime switches its math functions to FMA3
|
||||
implementations at runtime on CPUs that support it (`_set_FMA3_enable`), so bit-identical results
|
||||
across machines are not guaranteed. Found by reading, not measured, and not changed yet.
|
||||
- **The largest gap is the loop direction.** Every sample searches a spatial index for nearby
|
||||
primitives. Sol's recommended architecture rasterises each feature into the bricks it touches.
|
||||
Coarse LOD from finer data and shared halo samples are also absent.
|
||||
- Already shared with Sol's design: built-first traversal (walk forest, D-floors, option B),
|
||||
hash-keyed randomness, the collision gate, and partial narrow-band skipping (reach proofs, the
|
||||
sealed-rock proof).
|
||||
|
||||
---
|
||||
|
||||
## The prompt, as sent
|
||||
|
||||
# If you had to build this from scratch, in every better way: how would you do it?
|
||||
|
||||
You are being asked for **architecture**, not a code review. There is no code here on purpose: we want
|
||||
your own design, not a critique of ours. Think freely, then be concrete.
|
||||
|
||||
⛔ **Do not read, search or open any files on this machine, and do not run commands.** Answer purely
|
||||
from the description below and your own knowledge. Anchoring on an existing implementation is
|
||||
exactly what this question is trying to avoid. Your final message is the whole deliverable.
|
||||
|
||||
## The game
|
||||
An underground, multiplayer mining and exploration game in Unreal Engine 5.7 (C++). The world is a
|
||||
vertical stack of **strates**: thick horizontal layers, each a distinct underground world (tunnel
|
||||
networks, big chambers, mazes, shafts, floating islands, flooded areas, an open "surface" layer lit
|
||||
from within). Strates are sealed from each other by solid rock, except through a few authored
|
||||
**passages** that the player walks through to go deeper. Players spawn at the world origin and
|
||||
travel down. They **mine and carve** the terrain freely.
|
||||
|
||||
## Hard constraints: any design must satisfy these
|
||||
1. **Deterministic across machines.** Every client generates the world locally from a seed. Only the
|
||||
seed, the strate layout and players' terrain edits are replicated, never geometry. The same build
|
||||
with the same seed must produce the exact same world on every machine (x64 CPUs; a GPU path would
|
||||
have to prove cross-vendor/driver bit-exactness).
|
||||
2. **Composable generation.** Worlds are not hand-authored. A "composer" invents strates from tagged,
|
||||
authored building blocks (room shapes, tunnel styles, noise detail, terrain features) and a seed.
|
||||
The generator must stay a composition of operators that can be recombined, and archetypes can mix
|
||||
inside one strate.
|
||||
3. **Traversal guarantees.** From the point where a player arrives in a strate to where they leave,
|
||||
there must always be a walkable route (player capsule ~1.8 m tall, ~0.34 m radius, 45 cm step,
|
||||
~44 degree max walkable slope). Tunnels have walkable D-shaped floors, wind when a direct route
|
||||
would be too steep, and may have dramatic climbable drops only where a walkable alternative
|
||||
exists. Rooms cut cleanly into tunnels.
|
||||
4. **Terrain is volumetric and editable.** Caves, overhangs and arbitrary mining, so not a
|
||||
heightfield. Edits must re-mesh quickly and deterministically.
|
||||
5. **Streaming must keep up with a running player** (~8 m/s, faster when falling), with collision
|
||||
ready under the player before they can stand there. Far terrain is visible at lower detail.
|
||||
6. Resolution: ~25 cm voxels near the player. Target PCs: 8-core/16-thread desktop CPUs and a decent
|
||||
GPU.
|
||||
|
||||
## What we would like from you
|
||||
1. **Two or three genuinely different architectures** for generation, representation, LOD,
|
||||
meshing, streaming and editing that satisfy ALL the constraints. For each: how the terrain is
|
||||
represented and evaluated, where the work happens, how LODs are produced, how edits work, how
|
||||
determinism is guaranteed, the expected performance character, and the biggest risks.
|
||||
2. **Where the real performance wins are**, compared with the obvious design ("for every voxel
|
||||
sample, evaluate every nearby shape and noise"). Be specific: algorithms and data structures.
|
||||
3. **What you would deliberately NOT build**, and why.
|
||||
4. **If you could only change three things** in a working implementation of the obvious design,
|
||||
what would they be?
|
||||
|
||||
Be concrete. "Use the GPU" or "use ECS" is not an answer unless it addresses constraint 1 and the
|
||||
composition requirement. Say plainly where you are unsure.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Sol's answer, verbatim
|
||||
|
||||
I would make the deterministic artifact a compact “world plan,” then compile that plan locally into terrain. The plan describes topology, portals, feature instances, archetype regions, and generation-version hashes—not voxels or meshes.
|
||||
|
||||
My preferred implementation would be a feature-rasterized sparse brick volume. It retains operator composition, but executes operators feature-by-feature into the bricks they touch instead of asking every voxel about every feature. That is usually the best balance of predictable performance, editability, and determinism.
|
||||
|
||||
Two constraints need precise interpretation:
|
||||
|
||||
- If traversal must remain guaranteed after arbitrary player edits, “mine freely” and “always walkable” conflict. Players can mine away every possible floor. You must either protect a route, reject edits that destroy the last route, or automatically create a replacement.
|
||||
- Collision readiness cannot be guaranteed under arbitrary stalls or unbounded falling using prefetch alone. A hard guarantee requires bounded terminal velocity plus admission control, an immediate conservative collision proxy, or both.
|
||||
|
||||
Everything below assumes generated terrain must guarantee traversal. For post-edit traversal, I would use a server-enforced “last-route covenant”: ordinary mining is unrestricted, but an edit that would destroy the final walkable route is rejected or replaced with an authored support/bridge.
|
||||
|
||||
## Common foundation
|
||||
|
||||
Before discussing terrain representations, all three architectures should share the same deterministic world-planning layer.
|
||||
|
||||
### 1. The composer produces a WorldPlan
|
||||
|
||||
For each strate, the composer creates:
|
||||
|
||||
- Bounds and interface rock thickness.
|
||||
- Entry and exit passage transforms.
|
||||
- A mandatory traversal graph.
|
||||
- Optional chambers, side routes, drops, shafts, islands, flooded regions, and rewards.
|
||||
- Spatial archetype regions and their tags.
|
||||
- Feature instances and their stable IDs.
|
||||
- Material and detail rules.
|
||||
- A generation-format version and content-pack hash.
|
||||
|
||||
The replicated “strate layout” could be the full WorldPlan or only the high-level choices needed to reconstruct it. I favor replicating the compact plan: it prevents small composer changes from silently changing an existing saved world while still avoiding geometry replication.
|
||||
|
||||
Adjacent strates share authored portal contracts. The interface is solid by default, with only explicitly reserved passage volumes allowed to cross it. Cave noise is bounded and cannot affect the interface exclusion zone.
|
||||
|
||||
### 2. Traversal is constructed, not discovered afterward
|
||||
|
||||
Generate the mandatory route before decorative terrain:
|
||||
|
||||
1. Place entry, exit, and required intermediate anchors.
|
||||
2. Route a centerline through a coarse 3D planning grid.
|
||||
3. Include elevation, heading, and accumulated grade in the search state.
|
||||
4. When a direct segment exceeds the design grade, generate switchbacks or a spiral.
|
||||
5. Sweep a guaranteed-clear tunnel profile along that centerline.
|
||||
6. Add rooms and optional routes.
|
||||
7. Apply destructive detail and noise only outside the protected clearance envelope.
|
||||
8. Run an independent capsule validator as an assertion.
|
||||
|
||||
I would design the mandatory route below the mechanical limit—perhaps 35–38 degrees rather than 44—to leave room for meshing, collision quantization, and network movement tolerances.
|
||||
|
||||
The protected route volume is the capsule’s configuration space: at least the capsule radius plus margin laterally, enough headroom vertically, and a conservative floor envelope. It is stronger than merely checking whether an air voxel exists.
|
||||
|
||||
D-shaped tunnels should come from a swept profile with:
|
||||
|
||||
- A controlled floor curve.
|
||||
- An arched roof.
|
||||
- Fixed-point parallel-transport frames along the centerline.
|
||||
- Junction transition pieces where tunnels meet rooms.
|
||||
- A “no detail displacement” band around the walking surface.
|
||||
|
||||
A dramatic drop is added only as a graph chord or optional branch. It never replaces a mandatory walkable graph edge. Graph bridges and articulation points make that easy to verify.
|
||||
|
||||
The final validator should perform capsule sweeps or search through capsule-eroded free space at collision resolution. If it fails, retry only the optional layout using a deterministic attempt number; do not rely on an unconstrained random retry loop.
|
||||
|
||||
### 3. Determinism is an explicit numeric ABI
|
||||
|
||||
I would not trust “same C++ code on x64” by itself.
|
||||
|
||||
The generation contract would require:
|
||||
|
||||
- Integer or fixed-point world coordinates, preferably `int64`.
|
||||
- Fixed-point density and interpolation with specified rounding and saturation.
|
||||
- Integer lattice noise with fixed lookup tables—no platform `sin`, `pow`, or floating-point noise libraries.
|
||||
- Counter-based random values keyed by tuples such as
|
||||
`(world seed, strate ID, feature ID, decision kind, attempt)`.
|
||||
- No shared sequential PRNG streams.
|
||||
- Canonical ordering by stable ID whenever operations are noncommutative.
|
||||
- No dependence on hash-map iteration order, thread scheduling, pointer values, or allocation order.
|
||||
- Canonical triangulation, vertex ordering, ambiguity resolution, and material selection.
|
||||
- Explicit overflow behavior and wide intermediate arithmetic.
|
||||
- A generation-version/content hash checked during multiplayer connection and save loading.
|
||||
|
||||
Floating point can be used after canonical geometry exists—for rendering transforms, shading, and other non-authoritative work. It should not determine topology, occupancy, route selection, or edit results.
|
||||
|
||||
GPU work is limited to rendering, culling, upload, and possibly non-authoritative visual effects. I would not make GPU field evaluation or meshing canonical unless an integer-only implementation had been proven bit-exact across vendors and drivers.
|
||||
|
||||
---
|
||||
|
||||
# Architecture A: Lazy compiled implicit field
|
||||
|
||||
This is the most flexible architecture for arbitrary procedural shapes.
|
||||
|
||||
## Representation and evaluation
|
||||
|
||||
The WorldPlan is compiled into an immutable field program. Operators include:
|
||||
|
||||
- Room primitives.
|
||||
- Swept tunnel primitives.
|
||||
- Terrain unions and subtractions.
|
||||
- Bounded displacement/noise.
|
||||
- Material masks.
|
||||
- Authored feature fields.
|
||||
- Player edit operators.
|
||||
|
||||
The field is a deterministic integer scalar field, not necessarily a mathematically exact signed-distance field. Only its sign, interpolation, bounds, and material semantics need to be defined precisely.
|
||||
|
||||
Every feature has:
|
||||
|
||||
- A conservative bounding box.
|
||||
- A bound on field contribution.
|
||||
- A stable ID and priority.
|
||||
- A reference to a specialized evaluation kernel.
|
||||
|
||||
A spatial index maps macro-cells and bricks to relevant operators. When a chunk is requested, its operator set is gathered once, canonically sorted, simplified, and compiled into a chunk-local program.
|
||||
|
||||
Examples of compilation:
|
||||
|
||||
- Collapse unions of compatible room primitives into one loop.
|
||||
- Separate terrain topology from material evaluation.
|
||||
- Group identical tunnel kernels.
|
||||
- Remove operators whose bounds cannot influence the chunk.
|
||||
- Create archetype masks so that a noise style is evaluated only in its own region.
|
||||
- Propagate maximum noise amplitude for later pruning.
|
||||
|
||||
## LOD and meshing
|
||||
|
||||
Use world-aligned sparse bricks, for example:
|
||||
|
||||
- 4 m brick: `16³` cells at 25 cm.
|
||||
- 8 m mesh page: `2³` bricks.
|
||||
- 32–64 m macro-cell for planning and indexing.
|
||||
|
||||
Sample only requested bricks and their one-cell halos. Cache corner samples so neighboring chunks do not recompute them.
|
||||
|
||||
LOD is generated by evaluating the same field on world-aligned grids at spacings of 25, 50, 100 cm, and so on. An interval hierarchy records minimum and maximum possible density over coarse nodes:
|
||||
|
||||
- If the interval is entirely solid or air, no finer samples are needed.
|
||||
- If it crosses zero, refine.
|
||||
- Mandatory passages can be marked as topology-critical so coarse visual LOD does not accidentally seal them.
|
||||
|
||||
For meshing, I would use either:
|
||||
|
||||
- Fixed-point surface nets with deterministic vertex placement, or
|
||||
- Deterministic marching cubes with an integer asymptotic decider.
|
||||
|
||||
Dual contouring can produce better sharp features, but its QEF solver must be fixed-point with completely specified pivoting and rounding. A conventional floating-point QEF is a determinism trap.
|
||||
|
||||
Neighboring LODs remain 2:1 balanced and use fixed transition-cell tables. Every page owns vertices according to a canonical boundary rule, avoiding duplicate and differently rounded seam vertices.
|
||||
|
||||
LOD0 supplies the canonical collision triangles. Far LOD is render-only.
|
||||
|
||||
## Edits
|
||||
|
||||
Edits are quantized CSG stamps:
|
||||
|
||||
- Shape and dimensions.
|
||||
- Fixed-point transform.
|
||||
- Operation: mine, add, replace material, and so forth.
|
||||
- Server sequence number.
|
||||
- Author and optional gameplay metadata.
|
||||
|
||||
The server assigns ordering for noncommutative operations. Pure subtraction can be treated as commutative, but I would still preserve a total order for debugging and save compatibility.
|
||||
|
||||
Edits are indexed spatially. A changed edit invalidates only intersected bricks plus the meshing halo. Long edit lists are deterministically folded into per-brick edit blocks or checkpoints; the checkpoint is still player-edit data, not base-world geometry.
|
||||
|
||||
## Performance character
|
||||
|
||||
Advantages:
|
||||
|
||||
- Excellent composability.
|
||||
- Low persistent memory.
|
||||
- Arbitrary procedural primitives remain natural.
|
||||
- Any LOD can be evaluated directly.
|
||||
- Untouched, distant regions never need materialization.
|
||||
|
||||
Costs:
|
||||
|
||||
- Worst-case chunks can still have expensive field evaluation.
|
||||
- Domain warping and heavily overlapping operators weaken spatial bounds.
|
||||
- Repeated edits require compaction or their query cost grows indefinitely.
|
||||
- Fixed-point vector math and meshing require careful engineering.
|
||||
|
||||
The main risk is believing the spatial index alone solves performance. If every surface sample still evaluates 40 candidate features and six noise octaves, this architecture can miss streaming deadlines.
|
||||
|
||||
---
|
||||
|
||||
# Architecture B: Feature-rasterized sparse brick volume
|
||||
|
||||
This is the architecture I would most likely ship.
|
||||
|
||||
Instead of the field being the runtime truth, the WorldPlan is compiled into canonical sparse voxel bricks. Operators remain composable, but execute in bulk.
|
||||
|
||||
## Representation and generation
|
||||
|
||||
Each brick is one of:
|
||||
|
||||
- Uniform solid.
|
||||
- Uniform air.
|
||||
- Compressed material-only block.
|
||||
- Dense or narrow-band fixed-point density samples.
|
||||
- Edited dense block.
|
||||
|
||||
The generator processes features rather than samples:
|
||||
|
||||
1. Determine which bricks a tunnel, room, or feature intersects.
|
||||
2. Rasterize that feature into those bricks using a specialized kernel.
|
||||
3. Apply phases in a canonical order: structural voids, rooms, optional features, surface detail, materials, edits.
|
||||
4. Never touch bricks outside the feature’s conservative bounds.
|
||||
|
||||
A swept tunnel rasterizer iterates its centerline segments and their affected scanlines. A room stamp writes spans or masks. Authored building blocks can contain canonical sampled volumes and be placed using resampling-free transforms such as 90-degree rotations and reflections. More general transforms use a precisely specified fixed-point resampler.
|
||||
|
||||
For boolean-only phases, contributions can be accumulated independently and reduced in any order. Noncommutative phases use sorted feature IDs and ping-pong channels.
|
||||
|
||||
Noise becomes a bulk brick kernel. Better still, it runs only on bricks already known to contain a possible surface. Bounded noise cannot turn a deeply solid or deeply empty brick into a surface brick.
|
||||
|
||||
A sparse radix tree or hashed macro-cell table owns bricks. The storage itself need not have deterministic iteration order, because all externally visible processing uses sorted integer coordinates.
|
||||
|
||||
## LOD and meshing
|
||||
|
||||
Every brick produces a small hierarchy containing:
|
||||
|
||||
- Minimum density.
|
||||
- Maximum density.
|
||||
- Material summary.
|
||||
- Surface-presence mask.
|
||||
- Optional conservative solid/air coverage.
|
||||
|
||||
Parent LOD bricks are derived using an exact integer reduction. A simple averaged density is insufficient because it can erase thin walls or close tunnels. Min/max information decides whether a coarse region is definitely homogeneous or must retain detail.
|
||||
|
||||
Mesh generation reads the brick and its halo, then uses the same deterministic extractor and transition cells as Architecture A.
|
||||
|
||||
A useful split is:
|
||||
|
||||
- 25 cm render and collision near players.
|
||||
- Potentially coarser conservative collision away from traversal-critical regions.
|
||||
- Far render pages assembled from brick hierarchy nodes.
|
||||
|
||||
## Edits
|
||||
|
||||
Edits operate directly on affected brick spans or samples. This makes mining latency extremely predictable:
|
||||
|
||||
- Copy-on-write the changed brick.
|
||||
- Apply the quantized edit kernel.
|
||||
- Recompute only its mip ancestry.
|
||||
- Remesh it and its boundary neighbors.
|
||||
- Recook only affected collision pages.
|
||||
|
||||
Keep the original generated brick reconstructible from the WorldPlan. A save stores the edit operations or a canonical edit-derived brick delta, never the generated base brick.
|
||||
|
||||
## Performance character
|
||||
|
||||
Advantages:
|
||||
|
||||
- Work is proportional to affected brick volume, not `samples × candidate operators`.
|
||||
- Excellent cache locality and straightforward SIMD.
|
||||
- Uniform regions are extremely cheap.
|
||||
- Edits are fast and bounded.
|
||||
- Collision and LOD data naturally derive from the same representation.
|
||||
- Performance is more predictable than lazy per-sample evaluation.
|
||||
|
||||
Costs:
|
||||
|
||||
- Higher memory use.
|
||||
- A requested area must be materialized before use.
|
||||
- Poorly bounded features can still touch too many bricks.
|
||||
- Large filled volumes are wasteful unless uniform-brick detection happens early.
|
||||
- Authored sampled blocks can become repetitive if the composer does not provide enough transforms and procedural variation.
|
||||
|
||||
The main risk is turning it into a dense voxel world. The architecture works because homogeneous regions remain implicit and only surface/edit bricks become dense.
|
||||
|
||||
---
|
||||
|
||||
# Architecture C: Deterministic shell-first base with local voxelization for edits
|
||||
|
||||
This is a genuinely different option and can be extremely fast when most terrain remains untouched. It is also the most difficult to engineer robustly.
|
||||
|
||||
## Representation and generation
|
||||
|
||||
The unedited world is represented as deterministic watertight boundary patches:
|
||||
|
||||
- Polygonal swept tunnels.
|
||||
- Parametric room shells.
|
||||
- Terrain surface patches.
|
||||
- Portal and junction patches.
|
||||
- Bounded displacement patches.
|
||||
|
||||
Solid rock is implicit outside the air shells. There is no full base density volume.
|
||||
|
||||
All topology decisions use fixed-point or exact rational predicates. Curves are canonical polygonal approximations generated from quantized tables. Intersections and room/tunnel cuts use a deliberately restricted exact boolean system—not a general floating-point mesh boolean library.
|
||||
|
||||
The WorldPlan spatially indexes these patches. Untouched chunks triangulate only the surface patches that intersect them.
|
||||
|
||||
## LOD and meshing
|
||||
|
||||
Base patches have deterministic subdivision hierarchies. Adjacent patches share boundary vertex schedules, so independently generated LODs meet exactly. Far LOD selects coarser patch subdivisions.
|
||||
|
||||
When a player edits terrain, the affected area switches representation:
|
||||
|
||||
1. Allocate sparse voxel bricks around the edit.
|
||||
2. Classify the base shell into those bricks.
|
||||
3. Apply the edit operations.
|
||||
4. Mesh the local volume.
|
||||
5. Use a collar region whose boundary samples match the untouched shell.
|
||||
|
||||
An edit completely inside solid rock begins from a uniform-solid brick and creates a new cavity. An edit touching a cave wall classifies the nearby base shell first.
|
||||
|
||||
Collision uses base shell triangles in untouched regions and voxel-derived triangles in edited regions.
|
||||
|
||||
## Determinism
|
||||
|
||||
The shell topology, clipping predicates, tessellation, and triangle order must all be integer/rational and canonical. Conventional runtime mesh booleans are not acceptable.
|
||||
|
||||
## Performance character
|
||||
|
||||
Advantages:
|
||||
|
||||
- Untouched terrain costs roughly proportional to visible surface area.
|
||||
- No field evaluation across solid interiors.
|
||||
- Very cheap far LOD.
|
||||
- Works especially well for huge chambers and relatively localized mining.
|
||||
- Traversal proxy geometry is available immediately from the route plan.
|
||||
|
||||
Costs and risks:
|
||||
|
||||
- Exact, watertight joins between arbitrary procedural shapes are hard.
|
||||
- Noisy displacement and multi-way junctions are particularly troublesome.
|
||||
- The boundary between shell terrain and voxelized edited terrain is complex.
|
||||
- Large-scale mining gradually converts the world into Architecture B anyway.
|
||||
- Tooling and debugging burden are substantially higher.
|
||||
|
||||
I would consider this only if profiling showed base-world generation dominating and player edits remained spatially sparse. I would not choose it as the first production architecture.
|
||||
|
||||
---
|
||||
|
||||
# Streaming and collision
|
||||
|
||||
The streaming scheduler should be shared across architectures and driven by deadlines, not simple distance.
|
||||
|
||||
Maintain separate interest sets:
|
||||
|
||||
- Immediate safety set: LOD0 collision required.
|
||||
- Near visual set: LOD0 or LOD1 meshes.
|
||||
- Predicted movement set: locations reachable within a time horizon.
|
||||
- Far visual set.
|
||||
- Edit-dirty set.
|
||||
- Passage prefetch set for the next strate.
|
||||
|
||||
Priority is approximately “time until the player could touch this page,” accounting for:
|
||||
|
||||
- Current velocity and acceleration.
|
||||
- Character braking.
|
||||
- Network prediction uncertainty.
|
||||
- Gravity and terminal fall velocity.
|
||||
- Nearby shaft connectivity.
|
||||
- Other players on the authoritative server.
|
||||
|
||||
A vertical fall requires a swept ballistic volume, not a horizontal streaming radius. Known shafts and authored passages should prefetch their entire reachable landing corridor as soon as a player approaches them.
|
||||
|
||||
The worker pipeline is:
|
||||
|
||||
`plan/index lookup → field sampling or brick rasterization → mesh → collision cook → game-thread publication`
|
||||
|
||||
Each stage has revision numbers and supports cancellation. Results are published only if their input revision is still current.
|
||||
|
||||
Collision receives the highest priority. For a hard guarantee, I would use both:
|
||||
|
||||
- Immediately available conservative collision proxies derived from the route plan.
|
||||
- Movement admission: a character cannot cross into a page whose required collision revision is not active.
|
||||
|
||||
The latter should almost never trigger, but without it “collision will always be ready” is only a performance hope. Under severe overload, slowing a character at a streaming boundary is better than letting them fall through the world.
|
||||
|
||||
On Unreal’s side, generation data should remain plain C++ memory off-thread. UObject/component creation and scene publication happen on the game thread. I would use a custom terrain scene proxy and a dedicated collision provider rather than one heavy actor/component per tiny brick.
|
||||
|
||||
The server generates collision for all relevant players and remains authoritative. Clients generate the same static terrain for prediction and rendering. Physics simulation itself does not need cross-machine determinism as long as the server is authoritative, but the collision input triangles must still be canonical.
|
||||
|
||||
---
|
||||
|
||||
# Where the real performance wins are
|
||||
|
||||
The dominant improvement is changing the work from:
|
||||
|
||||
`number of samples × number of nearby shapes × number of noise layers`
|
||||
|
||||
toward:
|
||||
|
||||
`features touching requested bricks + cells near actual surfaces + dirty edit cells`
|
||||
|
||||
Specifically:
|
||||
|
||||
## 1. Feature-to-brick compilation
|
||||
|
||||
Build a macro-cell index when the WorldPlan is created. A requested brick gets a small, already-filtered operator program.
|
||||
|
||||
Better still, Architecture B iterates each feature over its covered cell spans. A sphere or swept tunnel should rasterize scanlines; it should not be rediscovered independently by every voxel sample.
|
||||
|
||||
Use separate indices for immutable world features and mutable edits.
|
||||
|
||||
## 2. Interval bounds and narrow-band evaluation
|
||||
|
||||
Every primitive and noise stack should expose conservative bounds.
|
||||
|
||||
If the base field for a region is farther from zero than the maximum possible displacement, skip all detail noise. This is especially important for expensive domain warps.
|
||||
|
||||
For an octree node:
|
||||
|
||||
- Entirely solid: emit no samples.
|
||||
- Entirely air: emit no samples.
|
||||
- Possibly crosses the surface: subdivide or evaluate.
|
||||
|
||||
This often removes more work than SIMD optimization.
|
||||
|
||||
## 3. Stop evaluating every archetype everywhere
|
||||
|
||||
Give archetypes spatial masks determined at planning resolution. A crystal-cavern noise stack should not even enter the chunk program for a neighboring sedimentary tunnel.
|
||||
|
||||
At transitions, compile only the two or three contributing archetypes and a deterministic blend mask.
|
||||
|
||||
## 4. Cache lattice samples and halos
|
||||
|
||||
Adjacent cells and chunks share corners. Cache canonical world-grid samples by brick revision. Do not evaluate each corner up to eight times, and do not let two chunks compute a shared boundary using different local coordinate paths.
|
||||
|
||||
## 5. Use bulk data layouts
|
||||
|
||||
For dense bricks:
|
||||
|
||||
- Structure-of-arrays density and material channels.
|
||||
- Fixed-size aligned slabs.
|
||||
- SIMD integer kernels.
|
||||
- Bitmasks for active surface cells.
|
||||
- Scanline/span writes for primitives.
|
||||
- Small thread-local scratch arenas.
|
||||
- No allocation in inner loops.
|
||||
|
||||
Bitsets are particularly effective for “possibly contains surface,” material-presence, and dirty-cell masks.
|
||||
|
||||
## 6. Incremental edit dependency tracking
|
||||
|
||||
An edit should invalidate:
|
||||
|
||||
- Intersected bricks.
|
||||
- One-cell mesh halos.
|
||||
- Their LOD ancestors.
|
||||
- Their collision pages.
|
||||
|
||||
Nothing else.
|
||||
|
||||
Give every brick separate generation, edit, mesh, and collision revisions. This prevents a late worker result from publishing obsolete terrain.
|
||||
|
||||
An ever-growing edit list is not viable. Compact edits into canonical per-brick deltas after a threshold while preserving the original network/save semantics.
|
||||
|
||||
## 7. Separate collision urgency from visual quality
|
||||
|
||||
Generate collision first. It can initially use a conservative, lower-complexity surface as long as it preserves walkability and does not allow passage into solid rock. Refine the render mesh afterward.
|
||||
|
||||
Do not block collision readiness on material extraction, tangents, decoration, meshlets, or high-quality far-LOD work.
|
||||
|
||||
## 8. Exploit underground visibility
|
||||
|
||||
In an underground world, Euclidean distance is a poor visibility predictor. Build coarse visibility cells from the chamber/tunnel graph. Far terrain behind several sealed turns can be deprioritized even if spatially close.
|
||||
|
||||
GPU occlusion and meshlet culling help rendering, but avoiding CPU generation of invisible far surfaces is the larger win.
|
||||
|
||||
---
|
||||
|
||||
# What I would deliberately not build
|
||||
|
||||
- A dense 3D array for an entire strate. Solid rock and empty chambers must remain sparse or implicit.
|
||||
- A global sequential random stream. Adding one decoration would change the rest of the world.
|
||||
- Floating-point generation decisions, floating mesh booleans, or platform math functions in the canonical path.
|
||||
- GPU-authoritative generation or meshing without proven integer bit-exactness.
|
||||
- A generic interpreted CSG graph evaluated from its root for every sample.
|
||||
- Unlimited domain-warp noise. It destroys useful spatial and amplitude bounds.
|
||||
- A general-purpose runtime B-rep boolean engine. Architecture C should support a narrow, controlled set of exact operations.
|
||||
- Independent marching-cubes meshes at each LOD with skirts added later as a patch. LOD ownership and transition topology need to be designed from the start.
|
||||
- Runtime full-strate flood fills as the primary traversal guarantee. Connectivity should be structural; validation is a backstop.
|
||||
- A never-compacted replicated edit list.
|
||||
- One Unreal actor or UObject per voxel brick.
|
||||
- Dependence on dynamic Nanite behavior for core terrain correctness. It may be useful for rendering, but collision, edits, and determinism need an independent path.
|
||||
- Streaming based only on concentric distance rings.
|
||||
- A system where render mesh completion is the only thing preventing a player from falling through unloaded terrain.
|
||||
- A claim that arbitrary post-edit traversal is guaranteed without restricting, rejecting, or repairing edits.
|
||||
|
||||
---
|
||||
|
||||
# If I could change only three things
|
||||
|
||||
1. **Introduce a deterministic topology-first WorldPlan compiler.**
|
||||
This establishes stable IDs, counter-based randomness, portal contracts, protected walkable routes, bounded feature instances, archetype masks, and a versioned numeric ABI. It fixes correctness and gives later systems much better information.
|
||||
|
||||
2. **Replace per-sample feature iteration with feature-to-brick compilation and sparse surface bricks.**
|
||||
Build spatial indices, rasterize features in bulk where possible, apply noise only within bounded surface bands, cache shared samples, and maintain min/max LOD hierarchies. This is likely the largest CPU improvement.
|
||||
|
||||
3. **Replace distance-only chunk streaming with a revisioned, deadline-driven collision pipeline.**
|
||||
Predict reachable motion including falls, prefetch passages and shafts, prioritize collision independently, invalidate only edit dependencies, and retain a conservative collision/admission fallback. This turns streaming from “usually fast enough” into an enforceable gameplay contract.
|
||||
|
||||
My first production choice would be Architecture B, while retaining Architecture A’s immutable operator language as the authoring and compilation input. In other words: compose with procedural operators, but execute them into sparse bricks feature-by-feature. It gives up a little theoretical elegance in exchange for much more controllable frame time, edit latency, collision readiness, and debugging.
|
||||
|
||||
> Round two, with the code: see `DESIGN-SOL-2026-09-15-CODE.md`. Sol corrected two of the notes above: per-sample lookup is ~5% exclusive (not the main cost), and coarse LOD from fine data is excluded by the clipmap design, not a gap.
|
||||
@@ -0,0 +1,165 @@
|
||||
# VoxelForge investigation — T1.d and SurfaceWorld column memo
|
||||
|
||||
Date: 2026-08-16
|
||||
Base: `experimental` at `4ba53f2`
|
||||
Method: static source review only. **No build, compile, editor launch, or automation test was run.**
|
||||
|
||||
## Executive conclusion
|
||||
|
||||
The current explanation “one unticked neighbouring strate makes the exhaustive per-chunk sweep
|
||||
reject otherwise-safe boundary tiles” is not what the code does.
|
||||
|
||||
`UVoxelStrateManager::UsesOperatorStackForChunk` depends only on `ChunkCoord.Z`. Before the full XYZ
|
||||
sweep, `UVoxelGenerator::ClassifyTile` has already visited every sampled lattice Z, required the
|
||||
operator-stack predicate for every cave Z it sampled, and required all cave samples to belong to one
|
||||
layout slot. Within that one slot the flag cannot vary with X or Y. The later XYZ flag check is
|
||||
therefore redundant with the current layout implementation.
|
||||
|
||||
That does **not** make cross-strate boundary tiles safe to classify with one stack. They still fail
|
||||
the independent one-slot, generator-type, and bit-identical-params guards. Removing only the later
|
||||
flag check would not unlock those tiles. Requiring the density samples represented by one verdict to
|
||||
use one implementation is necessary in principle; the current exhaustive re-check is over-strict/
|
||||
redundant, but it is not the measured T1.d blocker.
|
||||
|
||||
The diagnostic is less precise than the log claims: the initial `UsesOperatorStackForChunk` check
|
||||
runs before the different-slot check, so a coarse/boundary tile that reaches an unticked adjacent
|
||||
cave slot can increment `Cave Bail Not Op Stack` even though it would subsequently have failed as
|
||||
mixed content. The measured 80% therefore does not distinguish “the flown slot itself is unticked”
|
||||
from “a boundary tile encountered an unticked slot first.” The asset state still has to be read in
|
||||
the editor.
|
||||
|
||||
For the column memo, the port is genuinely incomplete. `GSurfColCache` is a six-box spatial LRU;
|
||||
`FSurfaceColumnSource::GetColumn` currently owns one 81x81 direct-indexed box. A recenter or
|
||||
`ColumnKey` change clears all 6,561 computed flags in that one box. This is a verified structural
|
||||
difference, but the observed 10–30% location-dependent miss band does not prove its performance
|
||||
impact. Only the same seed and same route can do that.
|
||||
|
||||
## Findings
|
||||
|
||||
| Status | Finding | Evidence and consequence |
|
||||
|---|---|---|
|
||||
| **Verified by reading** | The late “every chunk opted in” sweep is redundant today. | `VoxelGenerator.cpp`, `UVoxelGenerator::ClassifyTile`; `VoxelStrateManager.cpp`, `UVoxelStrateManager::UsesOperatorStackForChunk`. The predicate ignores X/Y and is constant for a layout slot. The earlier Z loop and `CaveBotChunkZ` guard already establish one opted-in cave slot. |
|
||||
| **Verified by reading** | Boundary tiles remain unsafe for a one-stack verdict for reasons independent of the flag. | `ClassifyTile` rejects a second cave slot, a different generator type, and any non-bit-identical parameter struct. Gradient/Interleaved transitions can vary params per chunk. Removing the redundant flag re-check alone cannot change the safe result from `Mixed`. |
|
||||
| **Verified by reading** | `Cave Bail Not Op Stack` is an ambiguous attribution counter. | The first predicate check occurs before `GetStrateChunkZBounds`/different-slot attribution. A boundary tile can be counted as NotOpStack even though mixed content would also reject it. Terrain correctness is unaffected; diagnosis is affected. |
|
||||
| **Suspicious, needs checking** | Most of the measured 1.42/1.77 NotOpStack bails come from the primary cave asset itself being unticked. | This is the simplest explanation for interior tiles, but `.uasset` state is not readable from this source tree and the counter does not separate primary-slot from boundary-slot failures. Confirm in the editor or use Approach A's initialization log. |
|
||||
| **Verified by reading** | The op-stack column memo loses an entire 6,561-cell working set on any recenter/key change. | `VoxelDensityOpStack.cpp`, `FSurfaceColumnSource::GetColumn`, has one direct-indexed box. `VoxelGenerator.cpp`, `FSurfaceColumnCache`, has `NumBoxes = 6` and evicts only one LRU box. |
|
||||
| **Suspicious, needs checking** | The one-box design materially causes the observed 10–30% in-game miss rate. | Plausible and location-sensitive, but unmeasured. The retracted different-route comparison cannot support a before/after claim. |
|
||||
| **Verified by reading** | VF-03's core CP-cache contamination defect is present, and the current source now contradicts the audit reviewer's header. | `VoxelGenerator.cpp`, `UVoxelGenerator::GetDensityAt`, keys the function-static `thread_local CP_*` state by `(ChunkCoord, LayoutVersion)` with no generator/world identity. Each manager's version begins at the same value. `VoxelForgeTestFixture.h`, `FTestWorld::Build`, now explicitly documents the contamination and repeatedly calls `Initialize` to give test worlds process-unique versions. Production has no equivalent owner key. A second world on the same worker can reuse the first world's params, `CP_UseOpStack`, and stack. |
|
||||
| **Suspicious, needs checking** | Every other TLS cache named in audit VF-03 has the same cross-world exposure. | Several keys visibly omit an owner, but this pass proved the `CP_*` path only. The broader `OC_*`, `BM_*`, passage shortlist, biome, diff, and op-local cache set should be audited as one owner-identity task rather than assumed from the old audit row. Both approaches below leave this open. |
|
||||
|
||||
The audit header says VF-03's fixture citation was fabricated. That statement is stale relative to
|
||||
the checked-out source: the cited explanatory block exists now in `VoxelForgeTestFixture.h` and is
|
||||
specific about `CP_UseOpStack` contamination. This does not automatically validate every cache
|
||||
listed in VF-03; it does validate the CP-cache defect above.
|
||||
|
||||
## Two implemented approaches
|
||||
|
||||
| | Approach A — explicit opt-in + six-box spatial LRU | Approach B — code-enforced cutover + four-way associative memo |
|
||||
|---|---|---|
|
||||
| Strategy | Preserve the per-asset A/B contract. Add initialization-time warnings naming every disabled layout slot; the human fixes the `.uasset`. Port the reference six-box LRU to the op source. | Ignore the serialized flag at runtime and route all eight ported archetypes through the stack. Keep the UPROPERTY for asset compatibility. Replace the box with 4,096 exact coordinate entries arranged as 1,024 sets x 4 ways. |
|
||||
| T1.d effect | No silent behavior change. T1.d fires only after the relevant assets are enabled. Boundary tiles still conservatively fail for slot/params reasons. | `Cave Bail Not Op Stack` should disappear for every valid ported slot without asset edits. `GetDensityAt` and `ClassifyTile` still use the same manager predicate and same stack factory. |
|
||||
| Cache behavior | Six independent 81x81 boxes. An acquisition miss recenters/clears one LRU victim; five boxes remain warm. Closest match to the proven reference. | Hash routes by exact X/Y plus both halves of `ColumnKey`; exact X/Y/key comparison decides hits. A miss evicts one entry in one set. No bulk clear. Four-way conflicts and capacity eviction remain possible. |
|
||||
| Runtime/memory cost | Approximately 0.79 MiB TLS per worker for this memo, versus roughly 0.13 MiB for the current single box. Six linear box checks per integer query. | Approximately 160 KiB TLS per worker (compiler padding can change this). Four exact probes per integer query plus hashing/LRU-rank updates. |
|
||||
| Main risk | High per-worker memory multiplication. The diagnostic currently enumerates SurfaceWorld slots too, although SurfaceWorld's exact-lattice T1.d path does not depend on this flag; narrow that warning before adopting it. | Intentional behavior change for every false-flag asset; the switch can no longer be selected for production A/B. Existing false/true ClassifyTile fixtures now exercise the same runtime path, reducing the distinction between those two end-to-end tests. Associative conflicts may underperform the spatial LRU. |
|
||||
| What it does not solve | VF-03 owner identity; boundary params/slot conservatism; proof of memo benefit. | VF-03 owner identity; boundary params/slot conservatism; proof of memo benefit. |
|
||||
|
||||
### Recommendation
|
||||
|
||||
Start from **Approach A**, after narrowing its warning to cave archetypes or rewording the
|
||||
SurfaceWorld entry. The code does not justify weakening `ClassifyTile`, and the current flag is still
|
||||
valuable for a same-route A/B. Approach A preserves that measurement lever and ports the known
|
||||
reference cache design. Its memory cost is the reason not to merge it blindly: measure it against
|
||||
Approach B on the identical route.
|
||||
|
||||
Approach B is the cleaner long-term endpoint only if the project has deliberately decided to retire
|
||||
the switch as a runtime fallback. It removes configuration drift and is much smaller in TLS, but it
|
||||
spends the A/B lever and moves every false-flag asset at once. The eight equivalence suites make that
|
||||
a defensible experiment, not a zero-risk migration.
|
||||
|
||||
## Worktrees and files touched
|
||||
|
||||
### Approach A
|
||||
|
||||
Worktree: `E:\Projet Unreal\VoxelM\Plugins\VF-approach-A`
|
||||
|
||||
- `Source/VoxelForge/Private/VoxelDensityOpStack.cpp`
|
||||
- `Source/VoxelForge/Private/VoxelStrateManager.cpp`
|
||||
- `CODEMAP.md`
|
||||
|
||||
Likely compile-error/watch spots:
|
||||
|
||||
- The function-local `thread_local FColumnCache` containing six large aggregate boxes on MSVC/UE's
|
||||
TLS implementation.
|
||||
- Nested local cache types and `FMemory::Memzero` of each victim's `Computed` array.
|
||||
- The new `UE_LOG` format strings/arguments in `UVoxelStrateManager::Initialize`.
|
||||
|
||||
### Approach B
|
||||
|
||||
Worktree: `E:\Projet Unreal\VoxelM\Plugins\VF-approach-B`
|
||||
|
||||
- `Source/VoxelForge/Private/VoxelDensityOpStack.cpp`
|
||||
- `Source/VoxelForge/Private/VoxelGenerator.cpp` (comments only)
|
||||
- `Source/VoxelForge/Private/VoxelStrateManager.cpp`
|
||||
- `Source/VoxelForge/Public/VoxelDensityOpStack.h` (comments only)
|
||||
- `Source/VoxelForge/Public/VoxelStrateDefinition.h` (UPROPERTY retained; comments only)
|
||||
- `Source/VoxelForge/Public/VoxelStrateManager.h` (comments only)
|
||||
- `Source/VoxelForge/Private/Tests/VoxelForgeClassifyTileTest.cpp` (comments/messages only)
|
||||
- `Source/VoxelForge/Private/Tests/VoxelForgeTestFixture.h` (comments only)
|
||||
- `CODEMAP.md`
|
||||
|
||||
Likely compile-error/watch spots:
|
||||
|
||||
- The function-local `thread_local FColumnMemo` 4,096-entry aggregate.
|
||||
- `VoxelHash::Mix` calls and casts for X/Y plus low/high halves of the 64-bit key.
|
||||
- The `uint8` four-way LRU ranks and local-entry default initialization.
|
||||
- UHT should see no serialized layout change: `bUseOperatorStack` was not removed or renamed.
|
||||
|
||||
Both worktrees are detached from `experimental`, contain uncommitted changes, and passed
|
||||
`git diff --check`. Neither was built or tested.
|
||||
|
||||
## Human measurement protocol
|
||||
|
||||
Do not compare screenshots or stat averages from different flights. For every baseline/candidate:
|
||||
|
||||
1. Use the **same world seed, same asset values, same start point, same route, same speed, same LOD/
|
||||
streaming settings, same capture duration, and the same warm-up policy**.
|
||||
2. Record at least one repeat of the route; scheduler variation can move worker-local cache reuse even
|
||||
when the geographic route is identical.
|
||||
3. Capture `stat VoxelForge` and an Insights trace together. Normalize column misses and worker time by
|
||||
`Tiles Meshed`; totals alone conflate cheaper tiles with fewer tiles.
|
||||
4. Change only one worktree/approach at a time. Do not compare Approach A after an asset edit with
|
||||
Approach B before that edit and call it a cache result.
|
||||
|
||||
For T1.d, record:
|
||||
|
||||
- `Cave Bail Not Op Stack`, `Cave Bail Mixed Content`, `Cave Bail Params`,
|
||||
`Cave Bail Stack Verdict`, `Cave Bail Disturbance`, `Cave Bail No Stack`;
|
||||
- `Tiles Classified`, `Tiles Meshed`, `Tiles Operator Stack Solid/Air`, and total skipped Solid/Air;
|
||||
- the route segment's active strate and whether each neighboring asset's legacy flag is enabled.
|
||||
|
||||
Expected interpretation:
|
||||
|
||||
- Approach A: the initialization log must identify any false flag. After the human enables the
|
||||
relevant cave definitions, NotOpStack should approach zero in single-strate interiors and
|
||||
`Tiles Operator Stack Solid/Air` should become non-zero. Boundary tiles may move to MixedContent,
|
||||
Params, or StackVerdict; that is conservative and expected.
|
||||
- Approach B: NotOpStack should be zero for valid ported cave slots without asset edits. A non-zero
|
||||
value then points to out-of-layout/invalid-slot logic or a stale build, not the legacy flag.
|
||||
- In both: `Tiles Meshed < Tiles Classified` is the production prize. A zero `violations` result and
|
||||
all eight bit-equivalence suites green remain mandatory before trusting it.
|
||||
|
||||
For the memo, record:
|
||||
|
||||
- `Column Memo Hits` and `Column Memo Misses` as a miss percentage;
|
||||
- misses per `Tiles Meshed`;
|
||||
- Insights `VoxelForge_ClassifyTile` and `VoxelForge_GenerateMesh` count and time per meshed tile;
|
||||
- visible LOD-ring update time/throughput on the identical route;
|
||||
- process memory if comparing the six-box LRU against the associative table at the same worker count.
|
||||
|
||||
A lower miss percentage on a different route is not evidence. A valid claim is: same seed, same
|
||||
route, same settings, same denominator, with the candidate as the only change.
|
||||
|
||||
## Ready-to-build status
|
||||
|
||||
Both alternatives are ready for the human's review/build step. No result in this document is a
|
||||
compile or runtime claim; all implementation conclusions are from source and diff inspection.
|
||||
@@ -0,0 +1,629 @@
|
||||
# VoxelForge — the decomposition map
|
||||
|
||||
> **What this is:** every one of the 8 archetypes, read line by line, broken into the four roles of
|
||||
> `OPSTACK-PLAN.md §2.5`, with each existing param traced to the op that will own it. Written
|
||||
> 2026-07-27 so no later port has to re-derive it.
|
||||
>
|
||||
> **Read `OPSTACK-PLAN.md §2.5` first.** The whole point is that an archetype becomes
|
||||
> `source + combiners + modifiers`, not one opaque op. If a port here collapses into a single
|
||||
> `FMazeOp`, the refactor has failed its own test.
|
||||
>
|
||||
> **Status:** analysis only. No op exists. `Public/VoxelDensityOp.h` (the contract) is written; the
|
||||
> `switch` in `GetDensityAt` is untouched.
|
||||
|
||||
---
|
||||
|
||||
## 0. What fell out — read this before the per-archetype sections
|
||||
|
||||
Three findings changed how I'd sequence the work. They are the reason this document is worth its
|
||||
length.
|
||||
|
||||
### 0.1 ⚠️ The contract needs an SDF channel, not just a density channel
|
||||
|
||||
`IVoxelDensityOp::Eval` returns *density*. But look at what TunnelNetwork actually does:
|
||||
|
||||
```
|
||||
CaveSDF = EvaluateSDFCached(rooms + tunnels) // SDF space
|
||||
CaveSDF = SmoothMin(CaveSDF, PitSDF, Pit.BlendK) // SDF space
|
||||
CaveSDF = SmoothMin(CaveSDF, ChimneySDF, Chim.BlendK) // SDF space
|
||||
→ ONE carve at the end: Density -= CarveFactor · BaseDensity · 2
|
||||
```
|
||||
|
||||
Maze, VerticalShafts and FloatingIslands do the same shape: build an SDF from several primitives,
|
||||
perturb the SDF with roughness noise, then convert once.
|
||||
|
||||
**If each primitive becomes a density op with its own carve, the `SmoothMin` junctions are lost** —
|
||||
a pit would meet its room at a hard seam instead of the organic blend the code deliberately builds
|
||||
(the comment at the pit block says exactly this: *"SmoothMin at the pit-to-room junction creates the
|
||||
same organic transition as tunnel-to-room (no hard seam at PitTopZ)"*). Roughness is worse: three of
|
||||
the four archetypes add noise to the **SDF**, which displaces the surface; adding the same noise to
|
||||
**density** scales with the local gradient and is a different effect.
|
||||
|
||||
So ops need two channels: `float Density` and `float Sdf`, with an explicit `SdfToDensity` op that
|
||||
converts. Sketch:
|
||||
|
||||
```cpp
|
||||
struct FVoxelOpSample { float Density; float Sdf; }; // Sdf = FLT_MAX ⇒ "no surface nearby"
|
||||
virtual void Eval(float X, float Y, float Z, FVoxelOpSample& InOut) const = 0;
|
||||
```
|
||||
|
||||
**This is the one part of `OPSTACK-PLAN §3` I think is wrong as written, and it matters far beyond
|
||||
fidelity:** SDF-space `SmoothMin` between two *different* sources is precisely how "a room graph
|
||||
carved into a mountain" produces an organic junction rather than one field punching a hole in the
|
||||
other. The single-channel contract can only ever overwrite. **Recommend adopting the two-channel
|
||||
`Eval` before porting Maze** — it is cheaper now than after four ports.
|
||||
|
||||
*(Left as a recommendation, not a change: the header as committed is single-channel, and this is
|
||||
Jahni's call.)*
|
||||
|
||||
### 0.2 ⚠️ Worm tunnels are why TunnelNetwork can never skip a tile — and the fix is a number
|
||||
|
||||
`AUDIT §6.2` frames "placed vs fielded" as a question about *future* 3D caves. It is already a live
|
||||
cost. Worms are a pure 3D-noise threshold carve with no bounds:
|
||||
|
||||
```
|
||||
if (WormStrength > 0 && WormThreshold > 0) { ... Density -= t · WormStrength · NetworkMask; }
|
||||
```
|
||||
|
||||
Unbounded in space ⇒ `EffectOverBox` = `CarveOnly` **everywhere** ⇒ `AllSolid` is dead for every
|
||||
tile of every strate with worms enabled. Direction alone cannot recover it.
|
||||
|
||||
**But the amplitude is bounded and trivially known:** `t ∈ [0,1]`, `NetworkMask ∈ [0,1]`, so the worm
|
||||
op can move density toward air by at most `WormStrength`. If the stack so far is provably solid by
|
||||
more than the sum of every remaining op's max carve, the box is still `AllSolid`.
|
||||
|
||||
So the first numeric interval bound worth writing is not the SDF Lipschitz bound the plan reaches
|
||||
for — it is **a scalar amplitude cap on the fielded-noise carves**. Roughly ten lines, and it is
|
||||
what unlocks deep-rock skipping for the plugin's most-used archetype. `OPSTACK-PLAN §4` defers all
|
||||
numeric bounds to Phase 3; on this evidence one of them belongs in Phase 2.
|
||||
|
||||
### 0.3 Disturbances already have bounds that `ClassifyTile` throws away
|
||||
|
||||
Today: `if (D.ChasmDensity > 0) bCanSolid = false;` — strate-wide, for the whole tile.
|
||||
|
||||
But chasms/bridges/ridges are hash-placed on an XY lattice of known spacing and radius, and the
|
||||
per-voxel code already builds the 3×3 candidate list. A box that no candidate reaches is `Identity`.
|
||||
This is a **pure win available to SurfaceWorld today**, independent of everything else in this plan:
|
||||
any surface strate that enables chasms currently loses `AllSolid` on every tile in it.
|
||||
|
||||
---
|
||||
|
||||
## 1. The shared primitive library
|
||||
|
||||
The 8 archetypes are ~6 functions in costumes (`OPSTACK-PLAN §1b`). Decomposed, here is what
|
||||
actually repeats. **Reuse count is the payoff metric** — anything used once is suspicious.
|
||||
|
||||
### Role 1 — FIELD SOURCES
|
||||
|
||||
| Op | What it lays down | Used by | `IsXYPure` | `ClassifyBox` / `EffectOverBox` |
|
||||
|---|---|---|---|---|
|
||||
| `FConstantRockSource` | `Density = BaseDensity` (solid) | TunnelNetwork, Maze, VerticalShafts, **bedrock gaps** | ✅ trivially | `ClassifyBox` → **AllSolid**, always. Free, exact. |
|
||||
| `FConstantVoidSource` | `Density = −BaseDensity` (air) | FloatingIslands | ✅ | `ClassifyBox` → **AllAir**, always. |
|
||||
| `FSlabVoidSource` | floor surface + ceiling surface, `Density = −min(z−floor, ceil−z)` | FlatPlain, CrystalChamber | ❌ *see §3.1* | `ClassifyBox` by sampling floor/ceiling over the box's XY corners — **exact, cheap, and a new skip win** |
|
||||
| `FHeightfieldSource` | `max(TerrainZ − z, z − CeilSurf)` | SurfaceWorld | ✅ (the column) | `ClassifyBox` by sampling columns on the mesher's exact lattice — **this is today's `TestColumn`, moved verbatim** |
|
||||
| `FRoomGraphSource` | rooms + tunnels (+ pits + chimneys) SDF | TunnelNetwork | ❌ | `Identity` when no room/tunnel bound reaches the box — **`FCachedRoom`/`FCachedTunnel` already carry the bounds** |
|
||||
| `FLatticeCorridorSource` | 3D lattice capsule corridors | Maze | ❌ | `Identity` when no open edge's capsule bound reaches the box |
|
||||
| `FShaftFieldSource` | vertical cylinders + horizontal connectors | VerticalShafts | ❌ (cylinders are XY-pure; connectors are not) | `Identity` when no shaft cell reaches the box in XY |
|
||||
| `FIslandBlobSource` | tapered flat-topped blobs | FloatingIslands | ❌ | `Identity` when no island bound reaches the box |
|
||||
| `FWormFieldSource` | 3D-noise threshold carve | TunnelNetwork | ❌ | **`CarveOnly` always** — see §0.2. The one unbounded source. |
|
||||
|
||||
Note how much `Identity` is available and unused: **six of nine sources can prove themselves absent
|
||||
from most of the volume**, and today none of them do.
|
||||
|
||||
### Role 2 — COMBINERS
|
||||
|
||||
`Replace` · `Union`(min) · `Subtract`(max) · `SmoothUnion`/`SmoothSubtract` (reuse
|
||||
`VoxelSDF::SmoothMin/Max`) · `Add` · `Mask`. Plus, from §0.1, the conversion op:
|
||||
|
||||
| Op | Meaning |
|
||||
|---|---|
|
||||
| `FSdfCarve(blend)` | `SDF < blend` ⇒ `Density -= smoothstep(...)·BaseDensity·2`. **The exact same six lines appear in TunnelNetwork, Maze and VerticalShafts.** |
|
||||
| `FSdfFill(blend)` | the `+=` mirror. FloatingIslands. |
|
||||
|
||||
That one op deduplicates three copies of the carve formula and one of the fill.
|
||||
|
||||
### Role 3 — DETAIL MODIFIERS
|
||||
|
||||
Everything here is gated on being near a surface, and everything here already exists.
|
||||
|
||||
| Op | Space | Used by | Notes |
|
||||
|---|---|---|---|
|
||||
| `FSurfaceRoughnessMod` | **SDF** *or* **density** — two variants, see below | all 4 SDF archetypes | the single biggest reuse in the plugin |
|
||||
| `FTerraceMod` (cave) | density | TunnelNetwork | SDF-gradient orientation gate (2 extra SDF evals) |
|
||||
| `FLayerLineMod` | density | TunnelNetwork | sine along Z, cubed |
|
||||
| `FRibbingMod` | density | TunnelNetwork | sine along Z, squared, `+=` |
|
||||
| `FCaveOverhangMod` | density | TunnelNetwork | low-Z-frequency fBm, positive lobe only |
|
||||
| `FCaveCliffMod` | density | TunnelNetwork | noise-modulated vertical gradient |
|
||||
| `FScallopMod` | density | TunnelNetwork | cellular noise |
|
||||
| `FArchMod` | density | TunnelNetwork | room-relative |
|
||||
| `FRoomColumnMod` | density | TunnelNetwork | room-relative, pre-baked in `BuildChunkCache` |
|
||||
| `FDomeMod` | density | TunnelNetwork | room-relative |
|
||||
| `FPinchMod` | density | TunnelNetwork | room-relative |
|
||||
| `FFloorBiasMod` | density | TunnelNetwork | only inside cave air |
|
||||
| `FPitMod` / `FChimneyMod` | **SDF** | TunnelNetwork | `SmoothMin`'d into the cave SDF — §0.1's motivating case |
|
||||
| `FGridColumnMod` | density | FlatPlain, CrystalChamber | world-grid cylinders — **different op** from `FRoomColumnMod` |
|
||||
| `FShaftLedgeMod` | density | VerticalShafts | banded shelves, half-sided |
|
||||
| `FChasmMod` / `FBridgeMod` / `FRidgeMod` | density (MC) | **all archetypes** (disturbances) | see §0.3 |
|
||||
|
||||
**The two roughness variants, because this is the trap:**
|
||||
|
||||
- **SDF variant** (Maze, VerticalShafts, FloatingIslands): `Sdf += fBm(x·k, y·k, z·k)·SCALE·Rough`.
|
||||
Raw, no fade, no clamp, fixed frequency baked into the call site (`0.12`, `0.1`, `0.08`).
|
||||
- **Density variant** (TunnelNetwork): two octave sets (main + fine×3), optional domain warp,
|
||||
four noise types, a `min(…, 0)` clamp so roughness can never re-fill definite air, and a
|
||||
quadratic fade by distance from surface.
|
||||
|
||||
They are *not* the same op with different params. Port them as one op with a `Space` enum and let
|
||||
the SDF variant's fixed frequencies become real params — that alone is a small authoring win, and
|
||||
`OPSTACK-PLAN §2.6` explicitly permits the re-tune.
|
||||
|
||||
### Role 4 — STRUCTURAL POST (fixed order, appended by the compiler, never author-omittable)
|
||||
|
||||
| # | Op | Today | Effect over box |
|
||||
|---|---|---|---|
|
||||
| 1 | `FOriginSpineOp` | `ApplyOriginSpine` | `CarveOnly`; `Identity` when the XY circle (R + 3) misses the box, or Z is outside the interior |
|
||||
| 2 | `FBoundarySealOp` | `ApplyBoundarySeal` | **`ClassifyBox` → AllSolid inside its band** (forcing — see `VoxelDensityOp.h`); `Identity` when the box misses both bands |
|
||||
| 3 | `FPassageCarveOp` | `ApplyPassageCarving` | `CarveOnly`; `Identity` via `AnyPassageNearBox` — **already written** |
|
||||
| 4 | `FDiffLayerOp` | the diff block in `GetDensityAt` | `Both` when mods intersect; `Identity` via `HasAnyModInChunkRange` — **already written** |
|
||||
|
||||
The order is load-bearing and is the order the code already uses: spine carves the interior only,
|
||||
the seal then re-solidifies its bands (the spine deliberately never touches them), passages punch
|
||||
through everything including the seal, and the player wins last.
|
||||
|
||||
> ### ⛔ RETIRED 2026-07-28 — frames were never a fifth thing. Porting all three candidates killed it.
|
||||
>
|
||||
> The section below argues for a `FRAME` op family from three examples. All three are now ported,
|
||||
> and none of them turned out to need one:
|
||||
>
|
||||
> - **`CaveWarp` wraps exactly ONE operator.** Pits and chimneys explicitly read *unwarped* coords
|
||||
> while writing the same SDF channel — the thing this document called "the single fiddliest thing
|
||||
> in the whole decomposition". Inside one operator the difficulty evaporates: the warp is a local
|
||||
> variable, not an inherited context. A transform whose scope is one op is not a frame.
|
||||
> - **`VerticalScale` is `Z / Scale`** — a pure function of a scalar and a param, recomputed in one
|
||||
> line by each op that needs it. A frame would add a channel to avoid a division.
|
||||
> - **The island warp (§7)** was kept local for the same reason, before the other two were even read.
|
||||
>
|
||||
> **Zero frames out of three candidates.** It was not missing infrastructure; it was one idea seen
|
||||
> three times from a distance. Kept below as the reasoning that was superseded, not as a plan.
|
||||
|
||||
### A fifth thing the plan doesn't name: FRAME OPS
|
||||
|
||||
Two archetypes transform the *query coordinates* rather than the field:
|
||||
|
||||
- `VerticalScale` — `EffectiveZ = WorldZ / VerticalScale` (TunnelNetwork), stretches everything below it.
|
||||
- `CaveWarpStrength/Frequency` — domain-warps the coords the SDF is evaluated at, *but not* the
|
||||
coords roughness/terrain-ops/columns use. The comment is emphatic about why: *"terracing stays
|
||||
horizontal, columns stay vertical"*.
|
||||
|
||||
So a frame op has **scope** — it applies to some ops below it and not others. Modelling that as
|
||||
"push frame / pop frame" markers in the tape is straightforward; modelling it as a per-op flag is
|
||||
not, because the same op can appear inside and outside a frame. **Decide this before the tape
|
||||
format is fixed.** It also has a `EffectOverBox` consequence: any op under a warp frame must inflate
|
||||
its box by the warp amplitude before answering — the existing code already does exactly this
|
||||
(`Expansion = CaveWarpStrength + 2`).
|
||||
|
||||
---
|
||||
|
||||
## 2. TunnelNetwork *(and Underwater — identical rock, a water flag)*
|
||||
|
||||
`GetDensityWithParams`, ~1080 lines, the biggest single function in the plugin. **Port LAST** — it
|
||||
owns `BuildChunkCache`'s two-region window-invariance discipline (§8.4), the most delicate code here.
|
||||
|
||||
```
|
||||
FRAME VerticalScale (Z pre-divide, wraps everything below)
|
||||
├─ FConstantRockSource Replace BaseDensity
|
||||
├─ FRAME CaveWarp (SDF queries only — NOT the ops below the frame)
|
||||
│ ├─ FRoomGraphSource → Sdf rooms + tunnels, cached per chunk
|
||||
│ ├─ FPitMod → Sdf SmoothMin, unwarped coords ⚠️
|
||||
│ └─ FChimneyMod → Sdf SmoothMin, unwarped coords ⚠️
|
||||
├─ FSdfCarve(SDFBlendRadius) Subtract
|
||||
├─ [gate: bNearCaveSurface = Sdf < SDFBlendRadius·3]
|
||||
│ ├─ FSurfaceRoughnessMod Add density-space variant
|
||||
│ ├─ ── per-room op override ── ⚠️ see below
|
||||
│ ├─ FTerraceMod Add
|
||||
│ ├─ FLayerLineMod Subtract
|
||||
│ ├─ FRibbingMod Add
|
||||
│ ├─ FCaveOverhangMod Add
|
||||
│ ├─ FCaveCliffMod Add
|
||||
│ ├─ FScallopMod Subtract
|
||||
│ ├─ FArchMod Add
|
||||
│ ├─ FRoomColumnMod Add
|
||||
│ ├─ FDomeMod Subtract
|
||||
│ ├─ FPinchMod Add
|
||||
│ └─ FFloorBiasMod Add
|
||||
├─ FWormFieldSource Subtract ⚠️ ungated, unbounded — §0.2
|
||||
└─ [structural post ×4]
|
||||
```
|
||||
|
||||
⚠️ **Pits and chimneys are evaluated at UNWARPED coordinates while rooms are evaluated at warped
|
||||
ones**, and then `SmoothMin`'d together. The comment justifies it (pit anchors come from unwarped
|
||||
room centres). Under a frame model this means pits/chimneys must sit *outside* the warp frame while
|
||||
still writing the same SDF channel. That is expressible, but it is the single fiddliest thing in the
|
||||
whole decomposition — **budget for it and do not discover it during the port.**
|
||||
|
||||
⚠️ **The per-room terrain-op override has no clean home.** Today: `NearestRoomIdx` picks a room,
|
||||
that room's hash-rolled `UVoxelTerrainOpDefinition` is applied onto a *copy of the whole param
|
||||
struct*, and the copy shadows `Params` for the rest of the function. In an op stack there is no
|
||||
"whole param struct" to overwrite. Two options:
|
||||
|
||||
- **(a) Scope by room.** Each modifier gains an optional "only inside room N's influence" predicate,
|
||||
and the room-graph source publishes the nearest-room index per voxel as stack state. Faithful,
|
||||
and it generalises to "this op only inside this region", which is the `Mask` combiner already in
|
||||
the plan.
|
||||
- **(b) Drop per-room ops**, make modifiers strate-wide, and recover variety with `Mask` on a
|
||||
hash field. Much simpler, visibly different world.
|
||||
|
||||
**(a) is right** — per-room variety is a real feature and (b) would flatten it — but it is the piece
|
||||
that could blow the Phase-1 timebox if attempted early. It is also the *only* consumer of
|
||||
`NearestRoomIdx`, so it can be deferred: port TunnelNetwork's geometry first with strate-wide ops,
|
||||
add room scoping after.
|
||||
|
||||
**Tile-skipping prize:** currently zero. After the port, with §0.2's amplitude bound and the room
|
||||
bounds already in `FCachedRoom`, deep bedrock below a tunnel network becomes provably `AllSolid`.
|
||||
This is the largest single perf item in the whole plan.
|
||||
|
||||
---
|
||||
|
||||
## 3. FlatPlain and CrystalChamber
|
||||
|
||||
**These are one op with two default sets** — `OPSTACK-PLAN §4` calls this the first real win, and
|
||||
reading the code confirms it: `GetSlabDensity` is called for both types with no branch on which.
|
||||
CrystalChamber is FlatPlain with a bigger `CeilingRoughness`.
|
||||
|
||||
```
|
||||
FSlabVoidSource Replace floor surface + ceiling surface → void field
|
||||
FGridColumnMod Add world-grid jittered cylinders, infinite height
|
||||
[structural post ×4]
|
||||
```
|
||||
|
||||
That is the entire archetype. Two of the eight collapse into one, and the ceiling's `abs(noise)`
|
||||
(formations hang down only, never punch up) is a two-line flag on the source.
|
||||
|
||||
> **Observed in-editor 2026-07-27, and it settles the question:** Jahni reports FlatPlain and
|
||||
> CrystalChamber render **identical** in the live world. They should — they share
|
||||
> `FSlabGenerationParams`, and nothing in the content sets them apart. **The enum promised a
|
||||
> difference the data never delivered**, in the shipped world as well as in the test fixture.
|
||||
> So the merge does not lose a distinction; it *reveals* that there was none. Making CrystalChamber
|
||||
> look like a crystal chamber is a **params** job — raise `CeilingRoughness` (6 → ~20, what
|
||||
> `SlabEquivalence`'s tuned pass uses) and drop `CeilingRelativeHeight` a little. That is authoring,
|
||||
> which is exactly the outcome the whole refactor is aiming at.
|
||||
|
||||
### 3.1 ✅ RESOLVED 2026-07-27 — Jahni: the Z term can go. Removed.
|
||||
|
||||
**Decision:** the Z term was not intentional character. It is gone from `GetSlabDensity` (both
|
||||
surfaces), `FSlabVoidSource` is XY-pure, and FlatPlain + CrystalChamber are ported and wired.
|
||||
|
||||
**What that bought, and what it cost:**
|
||||
- `IsXYPure() == true` ⇒ the T1.a column-cache treatment becomes available generically.
|
||||
- An **exact `ClassifyBox` with no sampling**: `VoxelNoise::FBM`'s contract is `[-1,1]`, so both
|
||||
surfaces live in Z bands with known bounds — a tile entirely below the floor band is provably
|
||||
solid, a tile strictly between the bands is provably air. These two archetypes proved **zero**
|
||||
tiles before. `VoxelForge.OpStack.SlabEquivalence` reports the count.
|
||||
- **Cost: the world re-tunes once.** Dropping the term samples a different slice of the noise
|
||||
field, so floor and ceiling shapes change (they do not degrade). Covered by §2.6's explicit
|
||||
permission to re-tune.
|
||||
|
||||
The original finding, kept because it explains why the answer mattered:
|
||||
|
||||
`FSlabVoidSource` is **not XY-pure, and probably should be.** Both surfaces sample noise with a
|
||||
small Z term:
|
||||
|
||||
```cpp
|
||||
FloorNoise: FractalNoise3D(x·FF, y·FF, WorldZ·FF·0.05f) // ← Z
|
||||
CeilNoise: FractalNoise3D(x·CF, y·CF, WorldZ·CF·0.08f) // ← Z
|
||||
```
|
||||
|
||||
A "floor surface height" that depends on the altitude you sample it from is geometrically odd — the
|
||||
floor is at a different height depending on which voxel asks. In practice the coefficient is tiny so
|
||||
it reads as a subtle vertical smear rather than a bug, and it is deterministic, so nothing is broken.
|
||||
But it blocks the T1.a column-cache treatment and it makes an exact `ClassifyBox` more awkward
|
||||
(the surface must be sampled per Z rather than per column).
|
||||
|
||||
**Question for Jahni: was the `·0.05f` Z term intentional character, or a leftover from copying the
|
||||
3D-noise call signature?** If it can go, `FSlabVoidSource` becomes XY-pure, gets the column cache
|
||||
for free, and gets an exact box classification — which means FlatPlain and CrystalChamber start
|
||||
skipping trivial tiles, which they never have. That is a large win for a one-character change, so
|
||||
it is worth asking rather than assuming either way.
|
||||
|
||||
**Answered: it can go.** See the resolution above.
|
||||
|
||||
---
|
||||
|
||||
## 4. Maze — **the Phase 1 port**
|
||||
|
||||
The plan picks Maze, and the code justifies the pick completely. ~100 lines, and it decomposes
|
||||
without any of the awkwardness elsewhere.
|
||||
|
||||
```
|
||||
FConstantRockSource Replace BaseDensity
|
||||
FLatticeCorridorSource → Sdf capsules over open lattice edges
|
||||
FSurfaceRoughnessMod → Sdf SDF-space variant, frequency 0.12 (hardcoded today)
|
||||
FSdfCarve(blend = 2.0) Subtract
|
||||
[structural post ×4]
|
||||
```
|
||||
|
||||
**Why it is the right first port, beyond size:** edge identity is `hash(lower node, axis)`, so two
|
||||
adjacent chunks *cannot* disagree. No `BuildChunkCache`, no COLLECT/STORE region, no window-invariance
|
||||
risk at all (`AUDIT §6.4` names this the pattern to prefer). If the source/modifier split does not
|
||||
fall out here, it will not fall out anywhere, and that is exactly what the stop-trigger is for.
|
||||
|
||||
**`EffectOverBox`:** the open-edge set reachable from a box is the same `{-1,0}³` node sweep the
|
||||
per-voxel code already does, at cell granularity. Capsule bound = `CorridorRadius + roughness
|
||||
amplitude + blend`. `Identity` when no open edge's capsule reaches the box — which for a sparse
|
||||
`BranchProbability` is most of the volume. **Maze currently skips zero tiles; this is its first.**
|
||||
|
||||
**The proof `OPSTACK-PLAN §4` asks for:** once `FLatticeCorridorSource` exists, drop it into a
|
||||
`SurfaceWorld` strate under the terrain and confirm you get a maze inside a mountain with no C++.
|
||||
Note this requires §0.1's SDF channel to look *good* (smooth junctions where corridors meet rock);
|
||||
it works but reads harsh without it.
|
||||
|
||||
---
|
||||
|
||||
## 5. SurfaceWorld — biggest payoff, most care
|
||||
|
||||
Three XY-pure functions plus a cheap per-voxel combine. The column cache (T1.a) and the exact-lattice
|
||||
`ClassifyTile` bound must both survive the port — they are the two most valuable pieces of
|
||||
engineering in the file.
|
||||
|
||||
```
|
||||
FHeightfieldSource Replace ← the whole column pipeline, XY-pure
|
||||
├─ FStructuralHeightField continents + mountains + detail, under a warp frame
|
||||
├─ FCliffHeightMod slope-gated steepening (4 structural resamples)
|
||||
├─ FTerraceHeightMod relief-gated plateaus
|
||||
├─ FLayerLineHeightMod sine bands
|
||||
└─ FBeachHeightMod flatten toward the water line
|
||||
FSkyCapSource Subtract ceiling: warp + signed swell + downward-only hang
|
||||
FOverhangShelfMod Union ⚠️ per-voxel, NOT XY-pure — the one 3D op here
|
||||
[structural post ×4]
|
||||
```
|
||||
|
||||
> ### ⚠️ RESOLVED 2026-07-27 — the height ops needed a SECOND OP FAMILY, not a sub-list
|
||||
>
|
||||
> This section says the height ops *"operate on Z values in the column, not on density"* and then
|
||||
> lists them as children of `FHeightfieldSource`. Writing them made the consequence unavoidable:
|
||||
> **they do not fit `IVoxelDensityOp` at all.** Its signature is `Eval(x, y, z, FVoxelOpSample&)` —
|
||||
> per voxel, density + SDF. A height op has **no input Z** (it produces one), is XY-pure (once per
|
||||
> column), and writes neither channel.
|
||||
>
|
||||
> The two ways to force it were both bad: a per-voxel third channel for what is a **column**
|
||||
> property, or collapsing all five into one opaque op — `OPSTACK-PLAN §2.5`'s explicit failure mode.
|
||||
>
|
||||
> **So height space got its own contract: `VoxelHeightOp.h`** (`FVoxelHeightSample` with
|
||||
> `Height` + `Relief`, `IVoxelHeightOp`, `FVoxelHeightStack`). Same lesson as `§0.1`, one step
|
||||
> further: §0.1 found that density needed a second *channel*; this found that terrain needs a second
|
||||
> *space*. Verified by `VoxelForge.OpStack.SurfaceHeightEquivalence` before anything was built on
|
||||
> top of it — deliberately, so a wrong answer would have cost one test rather than a whole port.
|
||||
>
|
||||
> **Bonus the type system gives for free:** a height stack cannot contain Z-dependent data, because
|
||||
> there is no Z in the signature to put there. `AUDIT §6.3` warns that Z-dependent data smuggled into
|
||||
> `FSurfaceColumn` silently corrupts every chunk in the vertical stack and that `ValidateDeterminism`
|
||||
> would not catch it. Here the *type* forbids it rather than a convention.
|
||||
|
||||
**Critical distinction the port must preserve:** the height ops (`FCliffHeightMod` and friends)
|
||||
operate on **Z values in the column**, not on density. They are XY-pure and belong in
|
||||
`PrepareChunk`/the column cache. `FOverhangShelfMod` operates per voxel and re-samples the
|
||||
structural heightfield at a shifted XY. Mixing those two up puts Z-dependent data in `FSurfaceColumn`,
|
||||
which `AUDIT §6.3` warns silently corrupts every chunk in the vertical stack — **and
|
||||
`ValidateDeterminism`, which samples along an X boundary, would not catch it.**
|
||||
|
||||
This is the archetype where `IsXYPure()` earns its place in the contract: it turns an implicit
|
||||
convention that has to be remembered into a declaration the compiler routes on.
|
||||
|
||||
**Biome blending:** the heightfield is evaluated for the dominant biome and its nearest neighbour and
|
||||
the two *heights* are lerped. In stack terms that is the `Mask` combiner with a biome-weight field —
|
||||
which is exactly the mechanism `OPSTACK-PLAN §4 Phase 3` wants for unifying strates and biomes. So
|
||||
SurfaceWorld's existing biome blend is the prototype for the whole Phase 3 idea, and porting it is
|
||||
how that gets validated.
|
||||
|
||||
---
|
||||
|
||||
## 6. VerticalShafts
|
||||
|
||||
```
|
||||
FConstantRockSource Replace
|
||||
FShaftFieldSource → Sdf infinite cylinders (XY-only) + hash-gated connectors
|
||||
FSurfaceRoughnessMod → Sdf SDF variant, frequency 0.1
|
||||
FSdfCarve(blend = 2.0) Subtract
|
||||
FShaftLedgeMod Union banded shelves, +X/+Y half only so a climb path remains
|
||||
[structural post ×4]
|
||||
```
|
||||
|
||||
Nearly identical in shape to Maze — same `source → roughness → carve` spine, different primitive.
|
||||
That similarity is the evidence the abstraction is real: two archetypes that look unrelated in the
|
||||
`switch` are the same three ops with a different source.
|
||||
|
||||
**Split worth making:** the shafts are XY-pure infinite cylinders; the connectors are not. Two ops
|
||||
(`FShaftColumnSource` XY-pure + `FShaftConnectorSource`) let the cylinder half get the column-cache
|
||||
treatment and answer `ClassifyBox` exactly in XY. Keeping them as one op forfeits that.
|
||||
|
||||
---
|
||||
|
||||
## 7. FloatingIslands
|
||||
|
||||
The only archetype whose source is **air**, which is what makes it a good composition test.
|
||||
|
||||
```
|
||||
FConstantVoidSource Replace −BaseDensity (open void)
|
||||
FRAME IslandWarp XY domain warp (lobed outlines, amplitude ~0.35·meanR)
|
||||
└─ FIslandBlobSource → Sdf tapered flat-top blobs, SmoothMin'd together
|
||||
FSurfaceRoughnessMod → Sdf SDF variant, frequency 0.08, 4 octaves
|
||||
FSdfFill(SDFBlendRadius) Union
|
||||
[structural post ×4]
|
||||
```
|
||||
|
||||
Note the warp here is applied to the **query** (`WX`,`WY` computed once per voxel and shared by all
|
||||
nearby islands) exactly like TunnelNetwork's cave warp — same frame concept, third instance. Three
|
||||
uses is enough to make frames a first-class part of the model rather than a special case.
|
||||
|
||||
**`FIslandBlobSource` is the cleanest `Identity` opportunity in the plugin:** islands are hash-placed
|
||||
with a known XY radius and explicit `TopZ`/`BotZ`. A box that no island's AABB reaches is provably
|
||||
untouched — and since a floating-island strate is *mostly* empty void, that is most tiles. Combined
|
||||
with `FConstantVoidSource`'s `ClassifyBox → AllAir`, a FloatingIslands strate could go from skipping
|
||||
zero tiles to skipping the large majority of them.
|
||||
|
||||
#### ✅ PORTED 2026-07-28 — three deviations from the sketch above, all deliberate
|
||||
|
||||
1. **`FConstantVoidSource` and `FSdfFill` are not new classes.** Each is the class it mirrors, with
|
||||
the opposite **sign**: `FConstantFieldSource(±Base)` and `FSdfConvertOp(Sign = ±1)`, two factories
|
||||
each. The table above listed them as separate ops; writing them separately would have duplicated
|
||||
the classifier and the six-line formula for nothing. Multiplying by ±1 is exact in IEEE-754, so
|
||||
the three already-green ports are bit-for-bit untouched by the generalisation.
|
||||
**This is the port's actual result:** reuse **by inversion** rather than by identity — evidence
|
||||
that the abstract axis (the sign of the internal density) is the right one, not just that two
|
||||
archetypes happened to look alike.
|
||||
2. **The warp stays inside the source; no `FRAME` op was built.** Frames are worth building at the
|
||||
second real user, and two of the three (`TunnelNetwork`'s cave warp, its tunnel warp) are not
|
||||
ported yet. Designing the abstraction against a single example is what this refactor has avoided
|
||||
throughout — cf. `IVoxelBiomeField`, which was born from a concrete second need. Revisit with
|
||||
TunnelNetwork.
|
||||
3. **`AUDIT §C1`'s last surviving site was in this archetype** and was fixed in both paths in the
|
||||
same pass (the warp's `(float)S * 0.0007f`; the 2026-07-27 sweep matched `SeedF * K` and missed
|
||||
the `(float)S` spelling).
|
||||
|
||||
**The `Identity` bound is one-sided, and that matters:** `Sdf ≥ WorldZ − TopSurf` bounds an island
|
||||
from **above** only. Below `BotZ` the SDF degenerates to ≈ `DistXY`, so a hairline thread of matter
|
||||
hangs down each island's axis to the strate floor. Rejecting a box because it sits below an island
|
||||
would be a hole. Only the top rejects.
|
||||
|
||||
---
|
||||
|
||||
## 8. Underwater
|
||||
|
||||
`GetDensityAt` routes `Underwater` to `GetDensityWithParams` with a comment: *"Underwater shares
|
||||
tunnel rock (water table is a render-side overlay)."* There is **no density difference at all**.
|
||||
|
||||
So `Underwater` is not an archetype, it is TunnelNetwork plus `WaterLevelRelative` consumed by the
|
||||
water render system. When `ECaveGeneratorType` finally disappears, this one vanishes for free — it
|
||||
never needed to exist as a generator type.
|
||||
|
||||
---
|
||||
|
||||
## 9. Q2 — the param audit: every field, and who claims it
|
||||
|
||||
`FStrateGenerationParams` via the `VF_STRATE_PARAM_FIELDS` X-macro. **Every field is claimed except
|
||||
where flagged.**
|
||||
|
||||
| Field(s) | Destination op |
|
||||
|---|---|
|
||||
| `BaseDensity` | **stack-global** — read by every source and all four structural-post ops. Not any single op's param. |
|
||||
| `VerticalScale` | `FRAME VerticalScale` |
|
||||
| `WormFrequency`, `WormHorizontalBias`, `WormThreshold`, `WormStrength`, `WormNetworkRange` | `FWormFieldSource` |
|
||||
| `RoomSpacing`, `RoomDensity`, `MinRoomRadius`, `MaxRoomRadius`, `RoomHeightRatio`, `RoomShapeVariety`, `RoomFloorCutMin`, `RoomFloorCutMax`, `FloorReliefStrength`, `FloorReliefFrequency` | `FRoomGraphSource` |
|
||||
| `OriginRoomRadius`, `OriginRoomMaxConnections` | `FRoomGraphSource` — ⚠️ but see §10.3, they couple to the spine |
|
||||
| `TunnelMinRadius`, `TunnelMaxRadius`, `TunnelDensity`, `MaxTunnelLength`, `TunnelWarpStrength`, `TunnelHorizontalBias`, `bTunnelsFlowTowardOrigin`, `TunnelEndpointZOffset` | `FRoomGraphSource` |
|
||||
| `SDFBlendRadius` | `FSdfCarve` / `FSdfFill` — **shared**, also the `bNearCaveSurface` gate width |
|
||||
| `CaveWarpStrength`, `CaveWarpFrequency` | `FRAME CaveWarp` |
|
||||
| `SurfaceRoughness`, `RoughnessFrequency`, `RoughnessNoiseType`, `DomainWarpStrength`, `DomainWarpFrequency` | `FSurfaceRoughnessMod` (density variant) |
|
||||
| `BoundarySealThickness` | `FBoundarySealOp` — and read by the spine, disturbances, shaft connectors, island spread |
|
||||
| `StrateTopWorldZ`, `StrateBottomWorldZ` | **`FVoxelOpContext`, not op params.** Runtime-injected, never author-set. |
|
||||
| `FloorBias` | `FFloorBiasMod` |
|
||||
| `TerraceStepHeight`, `TerraceHardness`, `TerraceNoiseDisplacement` | `FTerraceMod` |
|
||||
| `LayerLineSpacing`, `LayerLineDepth` | `FLayerLineMod` |
|
||||
| `OverhangStrength`, `OverhangDepth`, `OverhangFrequency` | `FCaveOverhangMod` ⚠️ name collision, §9.1 |
|
||||
| `RibbingSpacing`, `RibbingDepth` | `FRibbingMod` |
|
||||
| `CliffStrength` | `FCaveCliffMod` ⚠️ name collision, §9.1 |
|
||||
| `ScallopStrength`, `ScallopFrequency` | `FScallopMod` |
|
||||
| `ArchDensity`, `ArchMinRadius`, `ArchMaxRadius` | `FArchMod` |
|
||||
| `ColumnDensity`, `ColumnMinRadius`, `ColumnMaxRadius` | `FRoomColumnMod` ⚠️ name collision, §9.1 |
|
||||
| `PitDensity`, `PitMinRadius`, `PitMaxRadius`, `PitDepth` | `FPitMod` |
|
||||
| `ChimneyDensity`, `ChimneyMinRadius`, `ChimneyMaxRadius`, `ChimneyHeight` | `FChimneyMod` |
|
||||
| `DomeDensity`, `DomeMinRadius`, `DomeMaxRadius`, `DomeHeightRatio` | `FDomeMod` |
|
||||
| `PinchDensity`, `PinchStrength`, `PinchLength` | `FPinchMod` |
|
||||
| **`WaterLevelRelative`** | ⚠️ **claimed by no density op.** See §9.2. |
|
||||
|
||||
### 9.1 Three names mean different things in different structs
|
||||
|
||||
`FStrateGenerationParams` and `FSurfaceGenerationParams` both define `CliffStrength`,
|
||||
`OverhangStrength`/`OverhangFrequency`, `TerraceHardness` and `ColumnDensity`-family fields — and
|
||||
they are **genuinely different operations**:
|
||||
|
||||
| Name | in `FStrateGenerationParams` (cave) | in `FSurfaceGenerationParams` (surface) |
|
||||
|---|---|---|
|
||||
| `CliffStrength` | noise-modulated vertical density gradient near a cave wall | slope-gated steepening of a **height value** |
|
||||
| `OverhangStrength` | fBm lobe adding rock into a cave | F20 warped-terrain union making a real 3D shelf |
|
||||
| `TerraceHardness` | staircase edge width on a cave wall | plateau/riser ratio of a **height** quantiser |
|
||||
| `ColumnDensity` | room-anchored columns | *(slab struct)* world-grid cylinders |
|
||||
|
||||
Today the type system keeps them apart. Once ops are data assets in one list, **nothing does** —
|
||||
`DA_Op_Cliff` would be ambiguous. Name them for what they operate on from day one:
|
||||
`FCaveWallCliffMod` vs `FHeightSlopeCliffMod`, `FCaveShelfMod` vs `FTerrainOverhangMod`. Cheap now,
|
||||
a rename with authored assets in the field later.
|
||||
|
||||
### 9.2 The one unclaimed field: `WaterLevelRelative`
|
||||
|
||||
It lives in `FStrateGenerationParams` (and is `Lerp`'d at every strate boundary along with the
|
||||
density params), but **nothing in the density path reads it.** Its consumers are
|
||||
`GetWaterLevelWorldZForChunk` (the water render system) and `ComputeSurfaceTerrainZ`'s beach
|
||||
flattening — which reads the *surface* struct's copy, not this one.
|
||||
|
||||
Not dead, but misfiled: it is a *content/render* property riding in a *density* struct, and it is
|
||||
being interpolated across strate boundaries where a water plane should probably be a hard property
|
||||
of one strate. **Report, don't delete** — but when ops become assets it should move to the strate
|
||||
itself rather than to any op.
|
||||
|
||||
### 9.3 Fields that are context, not params
|
||||
|
||||
`StrateTopWorldZ` / `StrateBottomWorldZ` are runtime-injected by
|
||||
`UVoxelStrateManager::Get*ParamsForChunk`, never author-set, and every archetype's first line is a
|
||||
degenerate check on them. They belong in `FVoxelOpContext` (where the header already puts them) and
|
||||
should be *removed* from the per-op param structs so an author cannot see or set them. That deletes
|
||||
a whole class of "I set the Z bounds and nothing happened".
|
||||
|
||||
---
|
||||
|
||||
## 10. Ordering rules the compiler must enforce
|
||||
|
||||
### 10.1 Structural post is appended, always, in order
|
||||
`FOriginSpineOp` → `FBoundarySealOp` → `FPassageCarveOp` → `FDiffLayerOp`. Verified identical in all
|
||||
six density functions. An author cannot omit, reorder or insert between them.
|
||||
|
||||
### 10.2 Disturbances run after the archetype, before the diff layer
|
||||
`ApplyDisturbances` is called in `GetDensityAt` *after* the archetype function returns (so after that
|
||||
function's own spine/seal/passages) and *before* the diff layer. So the true global order is:
|
||||
|
||||
```
|
||||
[archetype stack] → spine → seal → passages → disturbances → diff layer
|
||||
```
|
||||
|
||||
Disturbances self-limit to the seal interior (`if (Z <= InnerBot || Z >= InnerTop) return;`), which
|
||||
is why running them after the seal is safe. **Preserve this or bridges will punch through seals.**
|
||||
|
||||
### 10.3 The spine and the origin room are two systems aimed at the same place
|
||||
`ApplyOriginSpine` carves an unconditional column at XY (0,0) in every strate; `OriginRoomRadius`
|
||||
makes the room graph put a big room there too; `bOpenSurfaceEntry` opens a shaft from above. Three
|
||||
mechanisms, one location, and `AUDIT C8` already flags an unchecked invariant between
|
||||
`OriginRoomRadius` and the COLLECT margin. When these become ops, the coupling becomes visible and
|
||||
should be either unified or documented — right now it works by everyone independently agreeing that
|
||||
(0,0) is special.
|
||||
|
||||
### 10.4 Recommended port order (unchanged from the plan, now with reasons from the code)
|
||||
|
||||
1. **Maze** — cleanest split, no connectivity decision, cheapest mistake. §4.
|
||||
2. **FlatPlain + CrystalChamber** — two archetypes → one op. Ask §3.1 first.
|
||||
3. **FloatingIslands** — the biggest `Identity` win, and the first air-source stack.
|
||||
4. **VerticalShafts** — same spine as Maze, validates the source-swap claim.
|
||||
5. **SurfaceWorld** — biggest payoff; the column cache and exact-lattice bound must survive.
|
||||
6. **TunnelNetwork** — last. §8.4, the warp/pit coordinate split, and the per-room op override.
|
||||
7. **Underwater** — falls out of 6 for free.
|
||||
|
||||
---
|
||||
|
||||
## 11. Open questions for Jahni
|
||||
|
||||
Ranked by how much they change the work.
|
||||
|
||||
1. **Two-channel `Eval` (density + SDF)?** §0.1. Changes the contract. Cheapest to decide now.
|
||||
My recommendation: yes — without it, cross-source `SmoothMin` is impossible and "a maze inside a
|
||||
mountain" reads as a hole punched in rock rather than a cave that belongs there.
|
||||
2. **Is `FSlabVoidSource`'s Z-term intentional?** §3.1. One character; unlocks XY-purity, the column
|
||||
cache and exact tile classification for two archetypes.
|
||||
3. **Per-room terrain ops: scope-by-room (faithful) or strate-wide + `Mask` (simpler, flatter)?**
|
||||
§2. Recommend scope-by-room, but *after* the geometry port, not during it.
|
||||
4. **Amplitude bound on the worm carve in Phase 2 rather than Phase 3?** §0.2. It is the difference
|
||||
between TunnelNetwork skipping tiles and never skipping tiles.
|
||||
5. **Rename the colliding cave/surface op names before any asset is authored?** §9.1.
|
||||
|
||||
---
|
||||
|
||||
*Written 2026-07-27 by Opus 5, unattended, as build-free queue item Q1. Companion to
|
||||
`OPSTACK-PLAN.md` (the plan) and `OPSTACK-PROGRESS.md` (what is actually built).*
|
||||
@@ -0,0 +1,103 @@
|
||||
# VoxelForge handoff — the operator stack is DONE. Read this, not the old jargon.
|
||||
|
||||
> Paste this into a fresh session. Written 2026-08-16, deliberately in plain language: the previous
|
||||
> version of this file had become a private dialect that Jahni could not read, which is a failure of
|
||||
> the document, not of the reader.
|
||||
|
||||
---
|
||||
|
||||
## 1. What the operator stack was, in one paragraph
|
||||
|
||||
Cave generation used to be one big `switch`: each cave type (tunnels, maze, shafts, floating
|
||||
islands, surface…) was its own hardcoded ~200–1000 line C++ function. The refactor replaced that with
|
||||
small composable pieces ("operators") that stack up to produce the same terrain. **The promise was
|
||||
that you could eventually invent new world types by combining pieces in the editor instead of asking
|
||||
for another thousand-line function.**
|
||||
|
||||
**That promise — "Phase 3", ops as data assets — was never built, and is NOT being built now.**
|
||||
|
||||
## 2. Status: done. Stop refactoring.
|
||||
|
||||
- **8 of 8 archetypes ported**, running in production (`bUseOperatorStack` is ticked on the game's
|
||||
strate assets), and **bit-for-bit identical** to the old path. Verified by a 115 000-sample field
|
||||
digest plus eight per-archetype equivalence tests.
|
||||
- **14/14 automation tests green, 0 violations anywhere.**
|
||||
- **39 % of tiles are skipped in the running game** (`Tiles Meshed` 1.15 vs `Tiles Classified` 1.88).
|
||||
It was 0 % on the morning of 2026-08-16. Each skipped tile avoids ~43 000 density evaluations plus
|
||||
marching cubes.
|
||||
|
||||
**Decision taken 2026-08-16, with Jahni:** the op stack is finished. **Do not start Phase 3. Do not
|
||||
start another refactor.** The old `switch` stays in place as the correctness oracle — deleting it
|
||||
buys nothing today. Next work should be things Jahni can *see*: see `fable-idea.md` (F7 set-pieces,
|
||||
F9 audio were queued before this started).
|
||||
|
||||
### The honest ledger, so nobody re-litigates it
|
||||
|
||||
Three weeks, 120 commits, from 2026-07-27. Every `feat:` commit in that window is a *port* of
|
||||
something that already worked. **The world did not change by a single voxel — that was the
|
||||
acceptance criterion.** What Jahni actually got: the 39 % perf win, two genuine pre-existing bugs
|
||||
found (a use-after-free on every strate-asset edit while streaming, and an under-bounded room
|
||||
collection that could make two multiplayer peers generate different geometry), and a number of fixes
|
||||
to bugs the refactor itself introduced. That is a thin return for three weeks, and it is why the
|
||||
direction changed.
|
||||
|
||||
## 3. The jargon, translated
|
||||
|
||||
Almost all of it means one thing: **can we prove a chunk of world is entirely rock or entirely air
|
||||
without checking every point in it, so we can skip the expensive work?**
|
||||
|
||||
| term | plain meaning |
|
||||
|---|---|
|
||||
| **T1.d / tile skipping** | that idea. The single biggest perf item in the plan. |
|
||||
| **box verdict / `ClassifyBox`** | "is this whole box uniform?" → `AllSolid`, `AllAir`, or `Mixed` (don't know) |
|
||||
| **`Mixed`** | "can't prove it" — always safe, just means we do the work |
|
||||
| **`ClassifyTile`** | the function that decides, per tile, whether to skip meshing |
|
||||
| **operator / op stack** | one generation step (rock, carve, roughness…) and the list of them |
|
||||
| **equivalence test** | proof the new path produces byte-identical terrain to the old one |
|
||||
| **`violations`** | ⚠️ **the only number that means danger.** A tile wrongly proved uniform has *no geometry and no collision* — a player falls through the floor. Must always be 0. |
|
||||
|
||||
## 4. What is verified, and what is not
|
||||
|
||||
**Verified:** everything through commit `871ca19` — tests green, digests unchanged, 39 % measured
|
||||
in game.
|
||||
|
||||
⚠️ **Built but NOT re-verified:** `4d33321` (Sol's boundary fold — lets a tile that straddles cave and
|
||||
open air still resolve) and `91585ea` (a test-only warning demotion). Jahni built these and says the
|
||||
game *looks* fine, but **the test suite has not been re-run and the counters have not been re-read
|
||||
since.** Before trusting them:
|
||||
|
||||
1. run the `VoxelForge` automation filter — **`violations` must be 0 and all eight equivalences
|
||||
bit-identical**;
|
||||
2. `stat VoxelForge` in game — the accounting must close:
|
||||
`Tiles Meshed + Skipped All Air + Skipped All Solid = Tiles Classified`.
|
||||
|
||||
If either fails, `git revert 4d33321` — the 39 % win does not depend on it.
|
||||
|
||||
## 5. Rules that still prevent real bugs
|
||||
|
||||
- **Never build.** Jahni builds; he has the editor open and it costs him real time. Say "ready to
|
||||
build" and list likely compile-error spots.
|
||||
- **`violations` 0 and the eight equivalences bit-identical** — the only non-negotiable results.
|
||||
- **A bound used to skip work must be PROVED, not observed.** Use `VF_PerlinAbsBound` (= 1.5);
|
||||
`FMath::Lerp(A,B,t)` spans `[min(A,B), max(A,B)]`, so a radius envelope is `max(Min,Max)`.
|
||||
- **Never change `ClassifyTile`'s conditions or return values casually.** Every `return` there fails
|
||||
safe to `Mixed`.
|
||||
- **Never state a `.uasset` value from memory** (like `bUseOperatorStack`). Ask, or read it in the
|
||||
editor. This sent a full day sideways.
|
||||
- `FindSlotIndexForChunkZ` is **protected**; `GetStrateChunkZBounds` is the public equivalent.
|
||||
- Push `experimental` freely. **Never push `main`.**
|
||||
|
||||
## 6. Two lessons that generalise beyond this plugin
|
||||
|
||||
- **A signal that always says the same thing measures nothing.** A counter that can fire for two
|
||||
reasons is not a measurement — splitting one such counter is what finally cracked T1.d after a day
|
||||
of wrong inference. A warning that fires every run and always means "this is fine" is noise that
|
||||
trains the reader to ignore warnings; one of those quietly worried Jahni for several sessions.
|
||||
- **Verify the premise, and verify it completely.** Multiple confident chains reversed on checking
|
||||
this month. Twice the failure was a *partial* read — grepping a symbol and reporting it as checked
|
||||
for something else. A grep that finds a declaration has not checked its access specifier.
|
||||
|
||||
## 7. The open question, which matters more than any of the above
|
||||
|
||||
**What do you want the world to *do* that it doesn't?** Three weeks went into a pipeline instead of
|
||||
that question. Start there.
|
||||
@@ -0,0 +1,548 @@
|
||||
# VoxelForge — Density Operator Stack: the plan
|
||||
|
||||
> **What this is:** the agreed direction for turning VoxelForge from an *archetype dispatcher* into a
|
||||
> *composable density pipeline*, so new world ideas become authoring instead of C++. Written
|
||||
> 2026-07-26 as a handoff for a future context — read this instead of re-deriving it.
|
||||
>
|
||||
> **Status (2026-07-28):** **Phases 0.5, 1 and 2 CODE-COMPLETE — 8 of 8 archetypes ported**, each
|
||||
> bit-identical to its original in an equivalence test, all wired behind `bUseOperatorStack`:
|
||||
> **Maze · FlatPlain · CrystalChamber · SurfaceWorld (biomes included) · VerticalShafts ·
|
||||
> FloatingIslands · TunnelNetwork · Underwater.** The archetype `switch` now has a complete
|
||||
> operator-stack twin, opt-in per strate.
|
||||
>
|
||||
> ✅ **BUILT AND GREEN, 2026-07-28 — 14 tests.** TunnelNetwork A+B bit-identical over 6000 samples
|
||||
> with all twelve group-coverage probes non-zero, all four noise branches covered, 0 gate leaks, and
|
||||
> C1 proved by 10 Terrace-op rooms containing 1119 samples. One open warning: the `Underwater` check
|
||||
> landed 0 samples in open cave, so its bit-identity proves little — diagnosed, not guessed, in
|
||||
> `OPSTACK-PROGRESS.md`'s last entry.
|
||||
>
|
||||
> **Not done, and it is the next real prize:** `ClassifyTile` still uses hand-written guards and does
|
||||
> not consume `ClassifyBox`. That is where measured tile-skipping becomes frames.
|
||||
>
|
||||
> Two things came out of Phase 2 that were not in the original design: **height space**
|
||||
> (`VoxelHeightOp.h`, a second operator family — some things are not another channel but another
|
||||
> *space*) and **`IVoxelBiomeField`** (ops depend on a capability, never on the generator, which is
|
||||
> what lets them become assets in Phase 3). Both are described in `OPSTACK-DECOMPOSITION §5`.
|
||||
>
|
||||
> **Known open:** generation is measurably slower on the op path (one fix landed — the column memo
|
||||
> was discarding itself every chunk; virtual dispatch and the hashed lookup remain). Deferred by
|
||||
> Jahni until the transition is complete. Live state and the next action live in
|
||||
> [OPSTACK-PROGRESS.md](OPSTACK-PROGRESS.md) — read its last entry first. The per-archetype
|
||||
> breakdown is in [OPSTACK-DECOMPOSITION.md](OPSTACK-DECOMPOSITION.md).
|
||||
>
|
||||
> **Read first:** `CODEMAP.md` (navigation) · `ARCHITECTURE.md §8.10` (the perf invariants this must not
|
||||
> break) · `AUDIT-2026-07.md §6` (the 3D hazards this is designed to kill permanently).
|
||||
>
|
||||
> **Jahni's goal, verbatim (2026-07-26):** *"a world generator, of any kind, of any possibility, almost
|
||||
> — any combination of ideas you could guess, could be happening."*
|
||||
|
||||
---
|
||||
|
||||
## 0. The one-paragraph version
|
||||
|
||||
`UVoxelGenerator::GetDensityAt` is a `switch` over 8 hardcoded `ECaveGeneratorType` values, each owning
|
||||
a bespoke density function and param struct. That means (a) a new world idea costs ~6 edit sites, and
|
||||
(b) **ideas cannot combine** — one archetype owns the whole voxel. The fix is to make density a **stack
|
||||
of small operators**, each of which can `PrepareChunk` / `Eval` / declare **what it can do to a box**.
|
||||
That last part is the keystone: it makes `ClassifyTile` generic and correct *forever*, instead of a
|
||||
hand-written guard per feature (the thing that already caused one revert and is the top hazard for the
|
||||
current 3D work). Staged so the game never stops working, and so the first useful step is small.
|
||||
|
||||
**Non-goal, stated deliberately:** this must *serve* the descent-through-strates structure, not dissolve
|
||||
it. "The world is a vertical stack you dig down through, each layer its own place" is the idea of this
|
||||
project — the seals, the (0,0) spine and the passages only mean something because of it. The op stack
|
||||
should make each strate more surprising, not turn the world into undifferentiated composable soup.
|
||||
|
||||
---
|
||||
|
||||
## 1. Why this specific shape fits this specific codebase
|
||||
|
||||
Not a generic "use composition" argument — three concrete reasons:
|
||||
|
||||
**(a) It formalises what the code already does by hand.** Every archetype already: hoists chunk-constant
|
||||
work into a `thread_local` cache (`PrepareChunk`), evaluates cheaply per voxel (`Eval`), and — in
|
||||
`ClassifyTile` — has a hand-written statement of what it can do to a tile (`Bounds`). The operator model
|
||||
isn't a new discipline; it's the existing discipline, named.
|
||||
|
||||
**(b) The 8 archetypes are ~6 functions wearing costumes.** `FlatPlain` and `CrystalChamber` are literally
|
||||
the same function with different defaults; `Underwater` is `TunnelNetwork` + a water flag. Decomposed,
|
||||
they're roughly **15 orthogonal primitives** (floor surface, ceiling surface, hash cylinders, room SDF
|
||||
graph, tunnel capsules, worm noise, lattice corridors, heightfield stack, island blobs, boundary seal,
|
||||
spine carve, passage carve, disturbances, surface ops, diff layer). 15 primitives that combine covers
|
||||
vastly more than 8 that don't.
|
||||
|
||||
**(c) It kills the recurring hazard permanently.** T1.d guards are currently written per feature
|
||||
(`AnyPassageNearBox`, the spine circle test, chasm/bridge flags, `HasAnyModInChunkRange`, phase-2's
|
||||
`OverhangMargin`). Every new 3D feature needs one, forgetting one is a hole, and one was already
|
||||
forgotten badly enough to revert T1.d v1 on 2026-06-26. Under this model a new op **cannot ship without
|
||||
answering the question**, and a test catches it if the answer is wrong.
|
||||
|
||||
---
|
||||
|
||||
## 2. The keystone insight — start with DIRECTION, not intervals
|
||||
|
||||
The full version of `Bounds` returns a numeric interval. **Don't start there.** Almost every existing
|
||||
operator is *one-directional*: it only ever carves, or only ever fills.
|
||||
|
||||
```cpp
|
||||
enum class EVoxelOpEffect : uint8
|
||||
{
|
||||
CarveOnly, // can only move density toward AIR → kills the AllSolid hypothesis
|
||||
FillOnly, // can only move density toward SOLID → kills the AllAir hypothesis
|
||||
Both, // unconstrained
|
||||
Identity // provably no effect on this box (the early-out that makes it fast)
|
||||
};
|
||||
|
||||
virtual EVoxelOpEffect EffectOverBox(const FBox& VoxelBox, const FVoxelOpContext& Ctx) const;
|
||||
```
|
||||
|
||||
**This alone reproduces every hand-written guard in `ClassifyTile` today, generically.** Look at the
|
||||
current code: "passages ⇒ `bCanSolid = false`" *is* `CarveOnly`. "bridges/ridges ⇒ `bCanAir = false`"
|
||||
*is* `FillOnly`. "no passage near this box ⇒ skip" *is* `Identity`.
|
||||
|
||||
So **Phase 1 ships with no numeric bounds at all** and already gets the whole safety property. Numeric
|
||||
intervals are a later tightening for gen-cost, not a correctness prerequisite. This is what turns a scary
|
||||
refactor into a small first step — and it's the thing to remember if this plan ever feels too big.
|
||||
|
||||
### Every existing primitive already has an obvious answer
|
||||
|
||||
Filled in here so a future context doesn't have to re-derive it:
|
||||
|
||||
| Primitive | Effect | Identity test (cheap) | Numeric bound (later) |
|
||||
|---|---|---|---|
|
||||
| `ApplyBoundarySeal` | **FillOnly** (`FMath::Max`) | box misses both seal bands | `+[0, BaseDensity]` |
|
||||
| `ApplyOriginSpine` | **CarveOnly** | circle-vs-box XY, + Z outside interior | `−[0, Base*2 + Seal]` |
|
||||
| `ApplyPassageCarving` | **CarveOnly** | `AnyPassageNearBox` — **already written** | `−[0, …]` via `AirTarget` |
|
||||
| Disturbance chasms | **CarveOnly** | `ChasmDensity == 0`, or lattice miss | `+[0, Solid]` toward air |
|
||||
| Disturbance bridges/ridges | **FillOnly** | density == 0, or lattice miss | `−[0, Solid]` |
|
||||
| F20 overhang | **FillOnly**, banded | outside `(TerrainZ, TerrainZ+Height]` — **already written** | union ⇒ `max` only |
|
||||
| Room / tunnel / column SDF | Both | bounding sphere vs box — **already written** | **Lipschitz-1**: `SDF ∈ [SDF(c) − r, SDF(c) + r]`, `r` = box half-diagonal |
|
||||
| fBm / Ridged term `k·N(...)` | Both | `k == 0` | `±\|k\|` — `FBM` returns `[-1,1]` by construction (`Total/MaxValue`) |
|
||||
| Heightfield (`TerrainZ − Z`) | Both | — | `[minT − Zmax, maxT − Zmin]`; **or sample the lattice exactly** (see below) |
|
||||
| Diff layer | Both | `HasAnyModInChunkRange` — **already written** | `±max\|Strength\|` over mods in range |
|
||||
|
||||
Two things to notice:
|
||||
|
||||
- **Five of these already exist as code.** The work is mostly *moving* guards, not inventing them.
|
||||
- **`Bounds` is allowed to be exact-by-sampling, not just analytic.** Today's `ClassifyTile` gets the
|
||||
tightest possible SurfaceWorld verdict by evaluating `ComputeSurfaceColumn` **on the mesher's exact
|
||||
lattice** — same functions, same floats, so the verdict is exact rather than estimated. That must
|
||||
survive. The contract is "conservative", not "closed-form": an op may sample to answer.
|
||||
|
||||
### Prior art (30 minutes, worth it before Phase 1)
|
||||
|
||||
- [Keeter, *Massively Parallel Rendering of Complex Closed-Form Implicit Surfaces*](https://dl.acm.org/doi/10.1145/3386569.3392429) + [fidget](https://github.com/mkeeter/fidget) — interval arithmetic per expression node to prune empty regions. The industrial version of this idea.
|
||||
- [Barbier et al., *Lipschitz Pruning: Hierarchical Simplification of Primitive-Based SDFs* (CGF 2025)](https://onlinelibrary.wiley.com/doi/10.1111/cgf.70057) — **the better fit for us**: bounds each primitive's range of influence and treats primitives as **black boxes**, where full interval arithmetic needs interval semantics defined for every node. Our ops are SDFs (Lipschitz by construction) and bounded-amplitude fBm. Black-box bounds; don't build a node-level IA engine.
|
||||
|
||||
---
|
||||
|
||||
## 2.5 ⚠️ The op TAXONOMY — and why this is NOT the old room-ops system
|
||||
|
||||
**Jahni's objection, 2026-07-27, and it is the correct one:** *"ops structure — which, let's all be honest,
|
||||
was what I had before, 'room operations' which would modify stuff, so I sure hope your idea is not that.
|
||||
There's a world of difference between a grotto strate and an open world strata."*
|
||||
|
||||
He is right, and a fresh context **must** internalise this or it will build something useless.
|
||||
|
||||
`UVoxelTerrainOpDefinition` today can only **perturb density near a surface that already exists**, inside
|
||||
a fixed archetype (`Terrace`, `LayerLines`, `Ribbing`, `Cliff`, `Scallop`, `Overhang`, `Arch`, `Column`,
|
||||
`Pit`, `Chimney`, `Dome`, `Pinch` — all applied near cave walls where `bNearCaveSurface`). It cannot turn a
|
||||
grotto into an open world, because it never decides **what the field IS**. That decision lives in the
|
||||
`switch` in `GetDensityAt`, which is exactly the thing we are removing.
|
||||
|
||||
So the op stack has **four ROLES**, and the old system only had role 3:
|
||||
|
||||
### Role 1 — FIELD SOURCES (the new thing; this is the "world of difference")
|
||||
Produce a density field *from nothing*. **This is what makes a grotto a grotto and an open world an open
|
||||
world.** Each of today's archetypes is fundamentally one of these:
|
||||
|
||||
| Source | Today's archetype |
|
||||
|---|---|
|
||||
| Heightfield ground + sky-cap ceiling | `SurfaceWorld` |
|
||||
| Room-graph SDF (hash rooms + tunnel capsules) | `TunnelNetwork` |
|
||||
| Floor/ceiling slab void | `FlatPlain`, `CrystalChamber` |
|
||||
| 3D lattice corridors | `Maze` |
|
||||
| Full-height shafts + connectors | `VerticalShafts` |
|
||||
| Suspended island blobs in open void | `FloatingIslands` |
|
||||
|
||||
A source is a first-class op with the same three-method contract. **A "strate archetype" therefore stops
|
||||
being an enum and becomes `source + combiners + modifiers`.** Two strates differ in their SOURCE first,
|
||||
their modifiers second.
|
||||
|
||||
### Role 2 — COMBINERS (how sources merge; this is what makes ideas compose)
|
||||
`Replace` · `Union`(min) · `Subtract`(max) · `SmoothUnion`/`SmoothSubtract` (reuse `VoxelSDF::SmoothMin/Max`)
|
||||
· `Mask` (scale the next op by a field: biome weight, slope gate, relief, depth).
|
||||
|
||||
This is the role that buys the ambition. *Floating islands **inside** a grotto. A maze **beneath** an open
|
||||
world's ground. A room-graph carved **into** a mountain.* None of those are expressible today at any price.
|
||||
|
||||
### Role 3 — DETAIL MODIFIERS (the old system, demoted to one role of four)
|
||||
Roughness, terrace, layer lines, ribbing, scallop, cliff, overhang, domes, pinch. **All of today's
|
||||
`UVoxelTerrainOpDefinition` types land here, essentially unchanged.** They keep working; they stop being
|
||||
the whole story.
|
||||
|
||||
### Role 4 — STRUCTURAL POST (fixed order, non-negotiable, runs last)
|
||||
`ApplyOriginSpine` → `ApplyBoundarySeal` → `ApplyPassageCarving` → diff layer.
|
||||
|
||||
These are **world invariants**, not creative choices: descent must stay possible, seals must hold, passages
|
||||
must punch through anything, player edits win. They are ops for uniformity but the stack compiler must
|
||||
always append them, in this order, regardless of authoring. **An author must not be able to omit them.**
|
||||
|
||||
### Plus: SCOPING
|
||||
Any op may be gated by a region predicate — Z band, XY region, biome index, slope range, depth. Scoping is
|
||||
what lets one strate hold several sources without them fighting, and it's the mechanism that unifies
|
||||
"strate" and "biome" into one concept in Phase 3.
|
||||
|
||||
> **The test for whether this refactor was worth doing:** can you author *"an open-world surface strate
|
||||
> whose mountains contain a room-graph cave system, with floating islands in the upper void"* **without
|
||||
> writing C++?** If no, it collapsed back into the old system and something went wrong.
|
||||
|
||||
---
|
||||
|
||||
## 2.6 The acceptance bar — "very close", NOT byte-identical
|
||||
|
||||
**Jahni, 2026-07-27:** *"I want not a 1/1 replica of the current strates with the current system, but a
|
||||
possible very close result to what I have right now, else it won't really matter much."*
|
||||
|
||||
Two different properties get confused here. Keep them apart:
|
||||
|
||||
| Property | Required? | Meaning |
|
||||
|---|---|---|
|
||||
| **`ValidateDeterminism` = 0** | **YES, ALWAYS** | the same world point sampled twice, from different cache windows/threads, returns the identical float. This is window invariance (§8.4) — self-consistency. Non-negotiable, it's what prevents seams and MP divergence. |
|
||||
| **Byte-identical to the OLD system's output** | **NO** | matching the pre-refactor world float-for-float. |
|
||||
|
||||
**This is a deliberate relaxation of what an earlier draft of this plan demanded, and it matters
|
||||
strategically:** if bit-identity were required, the cheap path would be to wrap each old density function
|
||||
as one monolithic op — 8 opaque ops that don't compose, i.e. **the switch with extra steps and zero
|
||||
gain.** Releasing that constraint is what permits *real* decomposition into the primitives in §2.5.
|
||||
|
||||
> **✅ CONFIRMED THE HARD WAY, 2026-07-27.** The Maze port reproduces `GetMazeDensity`'s **SDF bit for
|
||||
> bit**, and its final density to within 1-2 ULP on ~2% of samples, with **zero isosurface
|
||||
> crossings** — geometrically identical, not one triangle moved. The exact origin of that last
|
||||
> rounding was chased through five measured-and-refuted hypotheses and then **parked by decision**;
|
||||
> the full evidence is in `AUDIT-2026-07.md §C10`. **Read C10 before ever reopening it.**
|
||||
>
|
||||
> **The operational bar for every remaining archetype port, encoded in
|
||||
> `VoxelForge.OpStack.MazeEquivalence`:** hard-fail on any isosurface crossing (that moves geometry);
|
||||
> tolerate ULP-scale deltas (the accepted floor); warn on anything larger (that is real port drift).
|
||||
>
|
||||
> **And the rule that came out of it:** never run the archetype `switch` and the operator stack in the
|
||||
> same world, and never compare their outputs for equality — a half-migrated strate would seam. Not a
|
||||
> client-desync risk (the field is proven bit-pure within a binary); the cross-platform concern is
|
||||
> `§C9`.
|
||||
|
||||
**The bar instead:** for each ported archetype, an authored op stack must reproduce the *character* of the
|
||||
old one — same scale, same navigability, same feel, recognisably the same kind of place. Judged by Jahni
|
||||
on a screenshot at a fixed seed, not by a diff. Expect and accept a one-time re-tune, exactly as the T2.a
|
||||
SIMD noise switch required.
|
||||
|
||||
---
|
||||
|
||||
### 2.6.1 ⚠️ RELAXED FURTHER, 2026-07-27 — resemblance to the old world is NOT a requirement at all
|
||||
|
||||
**Jahni, verbatim:** *"I do not need your work to be identical or near identical to what I had
|
||||
before, only having it 99.99% at worst reproducible if two people share the same seed, since
|
||||
everyone rebuilds it on multiplayer."*
|
||||
|
||||
**This replaces the "recognisably the same place" bar above.** The requirement is not fidelity to the
|
||||
past — it is **agreement between peers in the present**. Restated as the only two properties that
|
||||
now matter:
|
||||
|
||||
| Property | Required? | Enforced by |
|
||||
|---|---|---|
|
||||
| **Same seed ⇒ same world, on every peer** | **YES — this is the whole bar** | `DensityPurity` within a binary; **`§C9`** across binaries/platforms |
|
||||
| Resemblance to the pre-refactor world | **NO** | nothing; freely re-tunable |
|
||||
| Bit-identity with the archetype `switch` | **NO** | nothing; never compare them |
|
||||
|
||||
**What this changes, concretely:**
|
||||
|
||||
1. **`§C10` is closed, not parked.** It measures old-path vs new-path agreement, and the two paths
|
||||
will never both exist in a shipped world. The residue cannot affect anything Jahni requires.
|
||||
2. **The equivalence tests keep their value, but for a different reason.** They are no longer
|
||||
*fidelity* checks; they are **port-correctness** checks — a transcription slip is still a real
|
||||
bug, and comparing against the old function is the cheapest way to catch one. Read them that way.
|
||||
The hard-fail (isosurface crossing) stays; the ULP grading is now diagnostic only.
|
||||
3. **`§C9` is promoted from a footnote to THE risk.** "Two people share a seed" is exactly the
|
||||
guarantee `/fp:fast` weakens across toolchains, and a Linux dedicated server generating collision
|
||||
geometry against Windows clients is the concrete case.
|
||||
4. **Changes that re-roll the world's noise are no longer expensive.** `§C1` in particular was
|
||||
deferred *only* because it forces a re-tune. That objection is gone.
|
||||
|
||||
---
|
||||
|
||||
## 3. The contract
|
||||
|
||||
```cpp
|
||||
// Chunk-constant inputs. Mirrors what the thread_local CP_* block resolves today.
|
||||
struct FVoxelOpContext
|
||||
{
|
||||
FIntVector ChunkCoord;
|
||||
int32 Step; // LOD step — ops may cheapen themselves (see §7)
|
||||
uint32 Seed;
|
||||
uint32 LayoutVersion; // ⚠️ see AUDIT C2 — every cache key MUST include this
|
||||
float StrateTopWorldZ, StrateBottomWorldZ;
|
||||
const FBiomeContext* Biome; // null = strate has no biome field
|
||||
};
|
||||
|
||||
class IVoxelDensityOp
|
||||
{
|
||||
public:
|
||||
// Hoist all chunk-constant work here (room lists, biome grids, column caches, lattice bakes).
|
||||
// Called once per chunk per worker. This is where today's thread_local caches move to.
|
||||
virtual void PrepareChunk(const FVoxelOpContext& Ctx) = 0;
|
||||
|
||||
// Per-voxel. InDensity = what the stack produced so far, MC convention (negative = solid).
|
||||
virtual float Eval(float X, float Y, float Z, float InDensity) const = 0;
|
||||
|
||||
// CONSERVATIVE. Phase 1: direction only. Phase 3: add a numeric interval overload.
|
||||
// Returning Both is always SAFE (costs CPU); returning the wrong one is a HOLE.
|
||||
virtual EVoxelOpEffect EffectOverBox(const FBox& VoxelBox, const FVoxelOpContext& Ctx) const = 0;
|
||||
|
||||
// Declares whether Eval depends on Z. XY-pure ops get the T1.a column-cache treatment
|
||||
// generically instead of SurfaceWorld having a bespoke one.
|
||||
virtual bool IsXYPure() const { return false; }
|
||||
};
|
||||
```
|
||||
|
||||
**Composition semantics** — keep the vocabulary small, and reuse what exists (`VoxelSDF::SmoothMin` /
|
||||
`SmoothMax`):
|
||||
|
||||
| Mode | Meaning |
|
||||
|---|---|
|
||||
| `Replace` | ignore `InDensity` (stack roots: heightfield, base density) |
|
||||
| `Union` (min) | add solid — bridges, islands, columns |
|
||||
| `Subtract` (max) | carve air — rooms, tunnels, passages, spine |
|
||||
| `SmoothUnion/Subtract` | the same with `SmoothMin/Max(k)` — organic junctions |
|
||||
| `Add` | scalar accumulate — noise/roughness terms |
|
||||
| `Mask` | scale the *next* op by a field (biome weight, slope gate, relief) |
|
||||
|
||||
`Mask` is what buys most of the expressiveness: "this op, but only in high-relief regions / only on
|
||||
steep slopes / only in this biome" becomes composition rather than a bespoke gate inside each op.
|
||||
|
||||
---
|
||||
|
||||
## 4. Order of work
|
||||
|
||||
### Phase 0 — THIS WEEK. Ship the 3D caves. Refactor nothing.
|
||||
|
||||
The current feature (caves inside mountains, volumetric generation) ships as-is with a **hand-written
|
||||
`ClassifyTile` guard** — see `AUDIT-2026-07.md §6.1` for why it's mandatory (without it the caves are
|
||||
never meshed: no geometry, no collision, invisible until you fall through them).
|
||||
|
||||
**One constraint only:** write the guard as a standalone function in the shape of the future contract —
|
||||
|
||||
```cpp
|
||||
// Not a member of anything yet. Just the right shape.
|
||||
EVoxelOpEffect CaveSystemEffectOverBox(const FBox& VoxelBox, const FVoxelOpContext& Ctx);
|
||||
```
|
||||
|
||||
Same work, zero architectural commitment, and it establishes the pattern.
|
||||
|
||||
**Also decide here (see `AUDIT §6.2`): placed, not fielded.** Hash-located cave systems with bounds
|
||||
(like `FCachedRoom`/`FCachedTunnel`) let the guard prove most of the mountain still solid and keep
|
||||
T1.d's 44% worker-CPU win. A global 3D noise field makes every deep tile `Mixed` and hands that back.
|
||||
|
||||
> ✅ **Gate:** caves render and collide when approached through solid rock, and worker CPU hasn't
|
||||
> visibly regressed.
|
||||
|
||||
---
|
||||
|
||||
### Phase 0.5 — The safety net. ~1 day. Do this before Phase 1, not after.
|
||||
> **✅ WRITTEN 2026-07-27 — ⏳ NOT YET COMPILED OR RUN.** Four tests in `Private/Tests/` (the three
|
||||
> below, plus a live-edit regression test for `AUDIT C2`). The gate below is NOT met until Jahni
|
||||
> builds and runs them. See `OPSTACK-PROGRESS.md`.
|
||||
|
||||
Three automation tests (`Source/VoxelForge/Private/Tests/`, `IMPLEMENT_SIMPLE_AUTOMATION_TEST`).
|
||||
There are currently **zero tests**, and nothing machine-checks the dozens of "bit-identical" claims in
|
||||
the docs.
|
||||
|
||||
1. **Density purity** — sample 10k points, shuffle query order, re-sample, assert bit-equality. Catches
|
||||
every cache-key bug including `AUDIT C2`. Run it across **multiple worker threads**, because
|
||||
`ValidateDeterminism` runs on the game thread and would miss worker-cache divergence.
|
||||
2. **`ClassifyTile` soundness** — for random tiles, if the verdict is `AllSolid`/`AllAir`, brute-force
|
||||
the lattice and assert every sample agrees. **This is the highest-consequence function in the plugin
|
||||
and it is currently validated only by reasoning.**
|
||||
3. **`DiffLayer` under contention** — N readers + a writer; assert no crash, monotonic version.
|
||||
|
||||
> ✅ **Gate:** all three green on the current code. If #1 or #2 fails, you've found a live bug — fix it
|
||||
> before building on top.
|
||||
|
||||
---
|
||||
|
||||
### Phase 1 — The pivot. Port ONE archetype. Timebox it.
|
||||
|
||||
**Pick `Maze`.** ~100 lines, no cross-chunk connectivity decision, trivial bound (corridor SDF is
|
||||
Lipschitz-1 off a lattice), and it's the least-used archetype so a mistake is cheap.
|
||||
|
||||
1. ✅ **DONE 2026-07-27 (uncompiled):** `IVoxelDensityOp` + `EVoxelOpEffect` + `FVoxelOpContext` +
|
||||
the four role tags + the box-verdict fold, in `Public/VoxelDensityOp.h`. **One addition beyond
|
||||
this spec:** `ClassifyBox` is not source-only — forcing ops (the boundary seal inside its band)
|
||||
overwrite the input, which pure direction cannot express. Rationale in the header.
|
||||
**One open question the decomposition raised:** `Eval` probably needs an SDF channel as well as
|
||||
a density channel — see `OPSTACK-DECOMPOSITION.md §0.1`. Decide before porting Maze.
|
||||
2. **DECOMPOSE, don't wrap** (§2.5, §2.6). Maze becomes a stack, not one op:
|
||||
`FLatticeCorridorSource` (role 1 — the capsule field off the 3D lattice) → `Subtract` →
|
||||
`FSurfaceRoughnessMod` (role 3 — the existing `SurfaceRoughness` perturbation) → then the four
|
||||
structural-post ops appended automatically. **If it comes out as a single `FMazeOp`, the refactor
|
||||
has failed its own test** — that's the switch with extra steps.
|
||||
3. `GetDensityAt` gains **one** branch: strate has an op stack ⇒ run it; else fall through to today's
|
||||
switch. **Both systems coexist**, indefinitely if needed.
|
||||
4. `ClassifyTile` gains a generic path (`EffectOverBox` folded over the stack) used only by ported strates.
|
||||
|
||||
> ✅ **Gate:** `ValidateDeterminism` = 0 delta (§2.6 — always), Phase 0.5 tests green, and the Maze world
|
||||
> is **recognisably the same place** at a fixed seed — same corridor scale, same connectivity, same feel.
|
||||
> Judged on a screenshot, not a diff. A one-time re-tune of the params is expected and fine.
|
||||
>
|
||||
> ✅ **The real proof, and the one that answers Jahni's objection:** once `FLatticeCorridorSource` exists,
|
||||
> **drop it into a `SurfaceWorld` strate underneath the terrain** and confirm you get a maze inside a
|
||||
> mountain with no C++ written. If that works, the architecture is doing the thing it was built for.
|
||||
>
|
||||
> 🛑 **Stop-and-reconsider trigger:** if Phase 1 exceeds ~2 days, or the source/modifier split doesn't fall
|
||||
> out naturally from the existing code, the abstraction is wrong for this domain. **Say so plainly, revert,
|
||||
> and report** — do NOT push through and port a second archetype to prove a point.
|
||||
|
||||
---
|
||||
|
||||
### Phase 2 — Port opportunistically. No big bang.
|
||||
|
||||
Port each archetype **the next time a feature makes you open it anyway**. The switch shrinks on its own.
|
||||
Suggested order when there's a free choice — cheapest and least risky first:
|
||||
|
||||
✅ `Maze` (P1) → ✅ `FlatPlain`/`CrystalChamber` (one op, two default sets — the first real win: two
|
||||
archetypes collapse into one; **done**, `BuildSlabStack`, 8 archetypes → 7) → ✅ `SurfaceWorld`
|
||||
(**done**, incl. biomes — needed a whole second op family, `VoxelHeightOp.h`; biggest payoff, biggest
|
||||
care: the T1.a column cache and the exact-lattice `ClassifyTile` bound both survived) →
|
||||
✅ `VerticalShafts` (**done**, 3 ops reused from Maze unchanged) →
|
||||
✅ `FloatingIslands` (**done**, `BuildFloatingIslandStack` — the stack that runs **backwards**: void
|
||||
source + fill instead of rock source + carve, the *same* classes with the opposite sign; only the
|
||||
blob source is new) → ✅ `TunnelNetwork` + `Underwater` (**done**, one builder for both —
|
||||
`BuildTunnelNetworkStack`, 19 ops).
|
||||
|
||||
**8 of 8 ported.** The last two were really one: `Underwater` *is* TunnelNetwork plus
|
||||
`WaterLevelRelative` (§8, re-verified before relying on it), so the switch lost its last two cases in
|
||||
a single port.
|
||||
|
||||
TunnelNetwork was ~1080 lines and was taken in **three stages, each verifiable on its own** rather
|
||||
than as ~600 unverified lines on top of ~200 (the `AUDIT §P3` pattern):
|
||||
* **A** — SDF spine: vertical scale, base rock, cave warp, room graph (+ pits + chimneys), carve,
|
||||
worms, structural post. Verifiable *while incomplete* because every detail modifier is
|
||||
amplitude-gated and defaults to zero, so zeroing them sends the ORIGINAL down exactly stage A's path.
|
||||
* **B** — the twelve detail modifiers of `STEP 4b–4h`, one group per commit, each with a coverage
|
||||
probe that proves the group actually moved something (`B1` roughness, `B2` terrace/lines/ribs,
|
||||
`B3` overhang/cliff/scallop/arch, `B4` columns/domes/pinch/floor-bias, `B5` the gate itself).
|
||||
* **C** — the per-room op override (`§2`'s option (a), and it needed no scoping predicate: one op
|
||||
owns the state, eleven read it), `Underwater`, and the flag flip.
|
||||
|
||||
⚠️ **`FRoomGraphSource` CALLS `BuildChunkCache`/`EvaluateSDFCached`; it does not transcribe them.**
|
||||
That is where §8.4's two-region window-invariance discipline lives, and a copy would fork it — with
|
||||
the fork "validated" by a test that compares it to the original.
|
||||
|
||||
Along the way, `FStrateGenerationParams`' 74 fields decompose into per-op structs, which retires the
|
||||
`VF_STRATE_PARAM_FIELDS` X-macro drift problem for free.
|
||||
|
||||
---
|
||||
|
||||
### Phase 3 — The payoff. Ops become data.
|
||||
|
||||
1. `UVoxelDensityOpDefinition : UPrimaryDataAsset` — one asset per op, mirroring today's
|
||||
`UVoxelTerrainOpDefinition` (which is *already* the right authoring shape: an asset + weight +
|
||||
probability in an ordered list).
|
||||
2. **A strate becomes "a Z range + an ordered op list"** instead of "an enum + a param bag".
|
||||
3. **A biome becomes "an XY region predicate + an op list" — the same mechanism.** Today these are two
|
||||
unrelated systems (`ECaveGeneratorType` dispatch vs the biome field with its `bOverrideTerrain`
|
||||
special case that only works for `SurfaceWorld`). Unifying them is where the expressiveness comes
|
||||
from: any op, scoped by any region, in any combination.
|
||||
4. Numeric interval bounds where profiling shows `Both` is costing real gen time.
|
||||
5. **Only if needed:** compile the per-chunk stack into a flat opcode tape and interpret with a switch,
|
||||
killing per-voxel virtual dispatch (~43k samples/tile × N ops). Standard technique. **Don't do this
|
||||
pre-emptively** — measure first; the noise is likely still dominant.
|
||||
|
||||
---
|
||||
|
||||
## 5. Invariants this must not break
|
||||
|
||||
Non-negotiable. Each has a documented reason and, mostly, a scar.
|
||||
|
||||
- **Window invariance (§8.4).** Every op stays a pure function of world coords + seed. Any op with a
|
||||
*connectivity decision over a neighbourhood* inherits `BuildChunkCache`'s two-region COLLECT/STORE
|
||||
discipline. **Prefer the `Maze` pattern** (pure hash of `(lower node, axis)` — adjacent chunks cannot
|
||||
disagree, no cache, no COLLECT region) unless connectivity is a gameplay requirement.
|
||||
- **T1.a column cache Z-independence.** `FSurfaceColumn` data is keyed `(XY box, StrateKey, Seed)` with
|
||||
**no ChunkZ** and shared down the whole vertical stack. XY-pure data goes in the column cache;
|
||||
Z-dependent evaluation happens per voxel. `IsXYPure()` exists to make this explicit instead of implicit.
|
||||
- **Cache keys include `LayoutVersion`** (`AUDIT C2` — today's `CP_Chunk`/`OC_Chunk`/`BM_Chunk` don't,
|
||||
and serve stale params after a live edit). `FVoxelOpContext` carries it so a new op can't forget.
|
||||
- **`ProcessQueue` stays `EQueueMode::Mpsc`**; ops are read-only on workers; `Epoch` carries through
|
||||
every async path.
|
||||
- **The two-pass MC loop, margin ring, and `thread_local` grid reuse** (§8.10) are untouched by all of
|
||||
this — the op stack lives *below* `GetDensityAt`, the mesher never knows.
|
||||
|
||||
---
|
||||
|
||||
## 6. Risks, and how each one is detected
|
||||
|
||||
| Risk | Detection |
|
||||
|---|---|
|
||||
| A wrong `EffectOverBox` ⇒ **a hole** | Phase 0.5 test #2 (brute-force vs verdict) — the load-bearing one |
|
||||
| A cache key missing an input ⇒ **seams** | Phase 0.5 test #1 (shuffled order, multi-threaded) |
|
||||
| Per-voxel dispatch cost | Insights `VoxelForge_GenerateMesh` before/after each port; tape compile if it bites |
|
||||
| Abstraction is wrong for this domain | The Phase 1 stop-trigger — one archetype, timeboxed, revert cheaply |
|
||||
| Scope creep into a node-graph editor | See §7 |
|
||||
|
||||
---
|
||||
|
||||
## 7. Explicitly NOT doing
|
||||
|
||||
- **No node-graph editor.** An ordered `TArray` of op assets is 90% of the value. A graph UI is a
|
||||
separate project, years later, and chasing it is how generative systems die.
|
||||
- **No GPU density.** Same reasons as `fable-idea` Part I: readback latency, CPU collision, and
|
||||
cross-GPU float determinism is fatal for "replicate the seed, regenerate identically on every peer"
|
||||
(`ARCHITECTURE §9.1`).
|
||||
- **No node-level interval arithmetic engine.** Black-box Lipschitz/amplitude bounds per op (§2).
|
||||
- **No big-bang port.** If more than one archetype is mid-port at any time, stop.
|
||||
- **No dissolving the strate structure.** See §0.
|
||||
|
||||
---
|
||||
|
||||
## 8. Independent fixes — do these regardless, they get worse with time
|
||||
|
||||
From `AUDIT-2026-07.md §5`. None depend on this plan; all become harder inside it.
|
||||
|
||||
1. **Bound `SeedF`** (`AUDIT C1`) — `const float SeedF = (float)(VoxelHash::Mix((uint32)Seed) & 0x3FFF);`
|
||||
at all 6 definition sites. Large seeds currently collapse noise terms to constants. One world re-tune.
|
||||
**Do it before tuning 3D caves against a seed you might later randomise.**
|
||||
2. ✅ **DONE 2026-07-27 (uncompiled)** — `GetLayoutVersion()` added to `CP_Chunk` / `OC_Chunk` /
|
||||
`BM_Chunk`, **plus two the audit missed**: `TC_BiomeCache` in `ClassifyTile`, and the
|
||||
`GSurfColCache` box key (whose `StrateKey` is `round(StrateBottomWorldZ)`, so a live edit that
|
||||
changes terrain params without moving the strate served stale columns — the most visible form of
|
||||
the bug). `FChunkBiomeCache::Invalidate()` added, since a validity BOX says nothing about the
|
||||
`FBiomeContext` its cells were classified against. (`AUDIT C2`.)
|
||||
3. **`GetPlayerPosition` no-player flag** (`AUDIT C4`) — `(0,0,0)` is the designed spine landing and
|
||||
currently stalls all streaming.
|
||||
4. **Unbounded joins on shutdown** (`AUDIT C5`).
|
||||
5. ✅ **DONE 2026-07-27** — `!*.md` in `.gitignore`; all nine design docs are tracked (commit `3128852`).
|
||||
6. ✅ **DONE** — the work lives on branch `experimental`; `main` is the known-good fallback.
|
||||
|
||||
---
|
||||
|
||||
## 9. Resume here
|
||||
|
||||
**Next action (2026-07-27): BUILD.** Phase 0.5's four tests and the Phase 1 skeleton header are
|
||||
committed and unverified. Nothing else should be written until they compile and the tests are green
|
||||
— writing unverified code on top of unverified code is the exact pattern `AUDIT §P3` documents.
|
||||
|
||||
After the build, in order: (1) fix whatever the tests report, (2) answer the five questions in
|
||||
`OPSTACK-DECOMPOSITION.md §11` — especially the SDF channel, which changes the contract and is
|
||||
cheapest to decide before any port, (3) port `Maze` per `OPSTACK-DECOMPOSITION.md §4`.
|
||||
|
||||
`AUDIT C1` (unbounded `SeedF`) is deliberately still open — see the progress log for why it was held
|
||||
back rather than forgotten.
|
||||
|
||||
When picking this up cold: read §0, §2 (the direction-only insight — that's what makes step 1 small),
|
||||
and §4. The rest is reference. If the plan feels too big, re-read §2: **Phase 1 needs no numeric bounds
|
||||
at all**, and the first real win is two archetypes collapsing into one.
|
||||
|
||||
---
|
||||
|
||||
*Changelog — 2026-07-26: written by Opus 5 after a full-tree audit, at Jahni's request, as a durable
|
||||
handoff so a future context doesn't re-derive it. Companion to `AUDIT-2026-07.md`.*
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,260 @@
|
||||
# Kickoff prompt — density operator stack refactor
|
||||
|
||||
*Paste the block below into a fresh context. Everything above the line is for Jahni, not the new session.*
|
||||
|
||||
**Branch:** already created and checked out — `experimental` (from `69fa73e tmp`). `main` is untouched.
|
||||
|
||||
**Why this exists:** the refactor is too big for one context. This prompt makes any fresh session able to
|
||||
start, or resume at a phase boundary, without re-deriving the design.
|
||||
|
||||
**Working rhythm it enforces:** one large batch of code → stop → *"ready to build"* + likely compile-error
|
||||
spots → Jahni builds → he pastes errors/screenshots → fix → next batch. That is deliberate; see §Autonomy.
|
||||
|
||||
---
|
||||
|
||||
```
|
||||
You're picking up an agreed refactor of the VoxelForge UE5 voxel plugin, on branch `experimental`
|
||||
(already checked out — do not create another). I'm Jahni. This was designed with a previous context
|
||||
and written down so you don't have to re-derive it.
|
||||
|
||||
GOAL
|
||||
Replace the hardcoded archetype `switch` in `UVoxelGenerator::GetDensityAt` with a composable
|
||||
density OPERATOR STACK, so new world ideas become data-authoring instead of C++. Long-term ambition:
|
||||
"a world generator of any kind, of any possibility — any combination of ideas could be happening."
|
||||
|
||||
READ FIRST, IN THIS ORDER (do not skip, do not skim §2.5)
|
||||
1. CLAUDE.md — project rules. Rule #1 is absolute.
|
||||
2. OPSTACK-PLAN.md — THE PLAN. §0 the summary, §2 why step 1 is small, **§2.5 the op taxonomy**,
|
||||
§2.6 the acceptance bar, §4 the phases, §5 the invariants, §7 non-goals.
|
||||
3. AUDIT-2026-07.md — §6 (the 3D hazards) and §5 (priority list). §1 has real open bugs.
|
||||
4. CODEMAP.md — navigation. Trust symbol names over line numbers; the lines are stale.
|
||||
5. ARCHITECTURE.md §8.10 — the perf invariants. Read before touching any hot path.
|
||||
|
||||
THE THREE WAYS YOU WILL FAIL — internalise these before writing code
|
||||
|
||||
(1) BUILDING. Never run a build, compile, or the editor. Jahni builds everything himself; he has the
|
||||
editor open and running it yourself just burns cost and tells you nothing. When a batch of code is
|
||||
done: STOP, say "ready to build", and list the likely compile-error spots. Then WAIT. Same for
|
||||
in-editor verification — ask for a screenshot, don't try to produce one.
|
||||
|
||||
(2) WRAPPING INSTEAD OF DECOMPOSING. Read OPSTACK-PLAN §2.5. Jahni already had a "room operations"
|
||||
system — ops that perturb density near an existing surface. His words: "there's a world of
|
||||
difference between a grotto strate and an open world strata." If you turn each old density
|
||||
function into one monolithic op, you have rebuilt the switch with extra steps and wasted the
|
||||
effort. Archetypes must DECOMPOSE into: FIELD SOURCES (role 1 — what makes a grotto a grotto),
|
||||
COMBINERS (role 2 — how sources merge; this is what makes ideas compose), DETAIL MODIFIERS
|
||||
(role 3 — his old ops, now one role of four), STRUCTURAL POST (role 4 — spine/seal/passage/diff,
|
||||
always appended in that order, never author-omittable).
|
||||
The test: can you author "open-world surface strate whose mountains contain a room-graph cave
|
||||
system, with floating islands in the upper void" with NO C++? If no, you built the wrong thing.
|
||||
|
||||
(3) BREAKING AN INVARIANT. OPSTACK-PLAN §5. In particular: window invariance (§8.4 — every op is a
|
||||
pure function of world coords + seed); the T1.a column cache is keyed (XY box, StrateKey, Seed)
|
||||
with NO ChunkZ and is shared down the whole vertical stack, so XY-pure data only; every cache key
|
||||
must include LayoutVersion (see AUDIT C2 — three existing caches get this wrong today);
|
||||
ProcessQueue stays EQueueMode::Mpsc; Epoch carries through every async path.
|
||||
|
||||
ACCEPTANCE BAR — read OPSTACK-PLAN §2.6 carefully, two properties get confused
|
||||
- `ValidateDeterminism` = 0 delta: REQUIRED ALWAYS. Same point, different cache window/thread,
|
||||
identical float. This is self-consistency, not reproduction.
|
||||
- Byte-identical to the OLD system's output: NOT required. Jahni: "not a 1/1 replica, but a possible
|
||||
very close result to what I have right now, else it won't really matter much." Recognisably the same
|
||||
kind of place at a fixed seed, judged on a screenshot. A one-time param re-tune is expected and fine.
|
||||
This relaxation is WHY you are allowed to decompose properly instead of wrapping.
|
||||
|
||||
WHAT TO DO — first batch (Jahni has standing permission for multiple changes per build)
|
||||
|
||||
A. Phase 0.5 from the plan — the three automation tests. There are currently ZERO tests in this
|
||||
plugin and nothing machine-checks the many "bit-identical" claims in the docs.
|
||||
1. Density purity: sample ~10k points, shuffle query order, re-sample, assert bit-equality. Run it
|
||||
across MULTIPLE worker threads (the existing `ValidateDeterminism` button is game-thread only and
|
||||
would miss worker-cache divergence — that's how AUDIT C2 hid).
|
||||
2. ClassifyTile soundness: for random tiles, if the verdict is AllSolid/AllAir, brute-force the
|
||||
lattice and assert every sample agrees. This is the highest-consequence function in the plugin
|
||||
and is currently validated only by reasoning; a false verdict is an invisible, collisionless hole.
|
||||
3. DiffLayer under contention: N readers + a writer, assert no crash and monotonic version.
|
||||
|
||||
B. Phase 1 skeleton — `Public/VoxelDensityOp.h`: `IVoxelDensityOp` (PrepareChunk / Eval /
|
||||
EffectOverBox / IsXYPure), `EVoxelOpEffect { CarveOnly, FillOnly, Both, Identity }`, the four role
|
||||
tags, `FVoxelOpContext` (carrying LayoutVersion), and the combiner enum. Header + docs only, no
|
||||
ports yet. Start with DIRECTION-only effects — no numeric intervals (OPSTACK-PLAN §2 explains why
|
||||
this alone reproduces every hand-written ClassifyTile guard, and why it makes step 1 small).
|
||||
|
||||
C. Then STOP and hand off for a build. Do not start porting Maze in the same batch.
|
||||
|
||||
If (A) fails on the current code you have found a live bug — report it, fix it, don't build on top.
|
||||
|
||||
FOUR THINGS AGREED WITH JAHNI 2026-07-27 THAT THE PLAN UNDER-STATES — hold these as goals
|
||||
|
||||
(i) THE BIGGEST PERF PRIZE IS TILE-SKIPPING FOR CAVE STRATES, and it is currently zero. Read
|
||||
`ClassifyTile`: any chunk that is neither a bedrock gap nor SurfaceWorld hits
|
||||
`return EVoxelTileClass::Mixed; // archétype cave […] pas prouvable en v1`. So TunnelNetwork,
|
||||
Maze, VerticalShafts, FloatingIslands, FlatPlain, CrystalChamber and Underwater capture NONE of
|
||||
T1.d's win (which was 84% of gens empty, −44% worker CPU). `fable-idea` wanted this from the start
|
||||
("for cave strates, 'no room/tunnel/passage/spine/seal/diff-layer bounds intersect' — all bounding
|
||||
data already exists") and it never happened because a bespoke prover per archetype was too much.
|
||||
`EffectOverBox` IS the generic mechanism. A room-graph source returning `Identity` when no room or
|
||||
tunnel bound reaches the box makes deep bedrock skippable for the first time. Treat this as an
|
||||
explicit deliverable of each port, not a side effect.
|
||||
|
||||
(ii) COMPILE THE STACK TO A FLAT TAPE, and don't wait for Phase 3 if the profile says otherwise.
|
||||
Naive per-voxel virtual dispatch is ~6 ops × ~43k samples/tile ≈ 257k indirect calls ≈ ~1 ms/tile
|
||||
of pure overhead — material against current gen cost. In `PrepareChunk`, compile the stack into a
|
||||
flat `(opcode, params)` array and run a switch over a small dense opcode set in the inner loop: no
|
||||
vtables, params cache-hot, predictable branches. Offsetting win, worth stating: ops that are
|
||||
disabled or out-of-scope are ABSENT from the tape, so the ~15 per-voxel `if (Params.X > 0)` gates
|
||||
inside `GetDensityWithParams` today become zero cost instead of one always-false branch each.
|
||||
Take an Insights capture after Phase 1; if dispatch shows up, do the tape then.
|
||||
|
||||
(iii) THE PERF STORY IS "≈ NEUTRAL PLUS ONE REAL WIN", NOT "FASTER". Do not oversell it in docs or
|
||||
reports. The reason to do this refactor is composition. Honest expectation: neutral per-voxel after
|
||||
the tape, meaningful gain on tiles-never-generated. If a measurement contradicts that, say so.
|
||||
|
||||
(iv) SHIP PRESET STACKS so authoring doesn't regress. A simple world today is "pick an enum, fill one
|
||||
struct"; after, it's "assemble 3-5 assets in the right order", which is more clicks and a new class
|
||||
of mistake (ordering is now semantic). Provide `DA_Stack_ClassicGrotto`, `DA_Stack_OpenWorld` etc.
|
||||
as starting points a strate can diverge from. `ECaveGeneratorType` is expected to disappear
|
||||
eventually — but only after every archetype is ported; it stays as the fallback path until then.
|
||||
|
||||
HOW TO REPORT AT EVERY STOP
|
||||
- What changed, file by file.
|
||||
- "Ready to build" + the specific spots likely to error (signatures, UHT, includes, template/lambda
|
||||
capture) so Jahni knows where to look.
|
||||
- What he should LOOK AT in-editor afterwards, concretely, and what a pass vs fail looks like.
|
||||
- What the next batch will be.
|
||||
- Anything you became unsure about. Ask rather than assume — for UE API behaviour especially, ask him
|
||||
for the docs instead of guessing (DivideAndRoundDown truncating rather than flooring already cost
|
||||
build cycles once).
|
||||
|
||||
DISCIPLINE
|
||||
- Update CODEMAP §3 rows for any new/renamed symbol; ARCHITECTURE §8 for design changes; tick phases
|
||||
in OPSTACK-PLAN.md as they land. Comments are French + English — match the surrounding file.
|
||||
- Never edit Binaries/, Intermediate/, *.generated.h.
|
||||
- Density sign: NEGATIVE = solid, POSITIVE = air at the mesher. #1 source of confusion.
|
||||
- Commit per feature with a real message (this branch exists so you can commit freely; `main` is the
|
||||
known-good fallback). Do not push.
|
||||
- If the plan turns out to be wrong for this domain, SAY SO and stop. There is an explicit
|
||||
stop-trigger on Phase 1: if it exceeds ~2 days or the source/modifier split doesn't fall out
|
||||
naturally from the existing code, revert and report rather than pushing through.
|
||||
|
||||
UNATTENDED OPERATION — Jahni may start you and go to sleep. Read this before your first tool call.
|
||||
|
||||
YOU CANNOT CHECK YOUR REMAINING BUDGET. No tool reports usage or quota. So do not try, and do not
|
||||
claim to. Instead assume the harder thing: **this session can end at any moment, without warning,
|
||||
mid-edit, and nobody will be watching.** Everything below follows from that.
|
||||
|
||||
CRASH-SAFE DISCIPLINE (non-negotiable when unattended)
|
||||
1. `git commit` after every coherent unit — a file, a test, a header. Small and often. You are on
|
||||
branch `experimental`; `main` is the known-good fallback, so committing costs nothing and a
|
||||
half-finished commit is infinitely better than an uncommitted half-edit. Pushing `experimental`
|
||||
is fine and expected (it is tracked as `origin/experimental` since 2026-07-29); **never push
|
||||
`main`.**
|
||||
2. Maintain `OPSTACK-PROGRESS.md` at the plugin root. APPEND (never rewrite) a dated entry per
|
||||
milestone: what you did, what you believe is true, what is UNVERIFIED (i.e. everything not yet
|
||||
built), and the single next action. Write the entry BEFORE starting the work it describes, so an
|
||||
abrupt death still leaves an accurate marker. This file is how the next context resumes.
|
||||
3. Never leave a file mid-transformation across a stopping point. If you're partway through changing
|
||||
a signature and its call sites, finish all call sites or revert the change. A tree that doesn't
|
||||
compile *for a reason you documented* is fine; one that doesn't compile for an unknown reason is
|
||||
the thing that wastes Jahni's morning.
|
||||
4. Don't batch a risky change with a safe one in the same commit. If a build fails he needs to know
|
||||
which half did it.
|
||||
|
||||
WHEN YOU REACH THE BUILD GATE — this will happen quickly, and it is not a failure
|
||||
The first batch (A + B) is a few hours at most, and then you physically cannot verify anything. At
|
||||
that point: STOP writing plugin code. Do NOT invent more C++ to fill the night — writing unverified
|
||||
code on top of unverified code is the exact failure `AUDIT-2026-07.md §P3` documents, and doing it
|
||||
unattended would be the worst version of it.
|
||||
|
||||
Instead work the BUILD-FREE QUEUE, in this order. All of it is genuinely useful and none of it can
|
||||
break anything:
|
||||
|
||||
Q1. ★ THE DECOMPOSITION MAP — the highest-value unattended task by far. Create
|
||||
`OPSTACK-DECOMPOSITION.md`: for EACH of the 8 archetypes, read its density function carefully
|
||||
and write out its proposed op breakdown — which FIELD SOURCE, which COMBINERS, which DETAIL
|
||||
MODIFIERS, and exactly which existing params migrate to which op asset (field by field, so
|
||||
nothing is silently dropped). Note per op: `IsXYPure()`, its `EffectOverBox` strategy, and any
|
||||
shared primitive two archetypes could reuse. This is hours of careful reading, has zero build
|
||||
risk, and it de-risks and speeds up every later port. Do this before anything else in the queue.
|
||||
Q2. Audit the `FStrateGenerationParams` 74 fields against Q1 and list any that no op claims — those
|
||||
are either dead or a decomposition gap. Report, don't delete.
|
||||
Q3. Tick the stale "PENDING BUILD" markers in `fable-idea.md` / `ARCHITECTURE.md` (confirmed
|
||||
resolved 2026-07-26 — everything is built and working; see `AUDIT-2026-07.md §0`).
|
||||
Q4. Fix `.gitignore` to `!*.md` — the design docs are currently untracked (`AUDIT P1`).
|
||||
Q5. STOP. Write the final `OPSTACK-PROGRESS.md` entry and a clear "good morning" summary: what's
|
||||
ready to build, exact likely error spots, what to look at in the editor, what a pass looks like.
|
||||
Then idle. Do not start Phase 2. Do not port a second archetype. Do not refactor anything not
|
||||
in the plan.
|
||||
|
||||
There is no prize for burning the whole night. A small, committed, well-documented, build-ready
|
||||
increment plus a complete decomposition map is a genuinely good night's work.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Autonomy — what a fresh context can and cannot do alone
|
||||
|
||||
**Asked:** *"possibly even, you tell me if it's possible, for you to wait for the token to replenish? to
|
||||
continue it all automatically and on your own."* — and then: *"can you tell it to check for token remaining
|
||||
before continuing actions? I plan to let it run as I sleep."*
|
||||
|
||||
**On checking the budget: no. There is no tool that reports remaining usage or quota**, so an instruction to
|
||||
"check tokens first" would be unfollowable — a safety net that looks real and isn't. The prompt therefore
|
||||
instructs the opposite and stronger discipline: **assume the session can die at any moment, unattended**,
|
||||
and make every stopping point crash-safe (commit-per-unit, an append-only progress log written *before*
|
||||
the work it describes, never a half-applied edit). That achieves the actual goal — not waking up to a
|
||||
broken tree — without depending on information the session can't get.
|
||||
|
||||
**On resuming automatically across a limit reset: the mechanism exists, but don't use it for this, and
|
||||
tokens are not the real limit.**
|
||||
|
||||
- A self-pacing loop is available (`/loop` with no interval, which schedules its own wakeups). Whether it
|
||||
resumes cleanly across a usage-limit reset is not something to promise — treat it as unverified.
|
||||
- **The actual blocker is the build gate, not the budget.** Every meaningful step of a C++ refactor ends
|
||||
at a compile, and rule #1 is that Jahni compiles. A loop left running would therefore do exactly one
|
||||
thing: **write more unverified code on top of unverified code** — which is the precise failure pattern
|
||||
`AUDIT-2026-07.md §P3` was written about (three weeks of stacked "PENDING BUILD"). Automating it would
|
||||
deepen the problem it diagnosed.
|
||||
- **What genuinely scales instead: batch size.** A single context can produce a large, coherent,
|
||||
self-contained batch before stopping. Standing permission for multiple changes per build already exists.
|
||||
So the throughput lever is "fewer, bigger handoffs", not "unattended looping".
|
||||
- **What IS worth looping:** work with no build gate — documentation passes, analysis, or the separate
|
||||
web-based world-rating harness. Not this.
|
||||
|
||||
**So the honest shape of "handling it on its own":** a fresh context can own the *design decisions, the
|
||||
code, the doc updates and the sequencing* end to end, across many sessions, resuming from
|
||||
`OPSTACK-PLAN.md §9`. It cannot own *verification*. That stays with Jahni, and given that the failure
|
||||
modes here are invisible holes and silent seams, that's the correct place for it.
|
||||
|
||||
### What an overnight run realistically produces
|
||||
|
||||
Setting expectations honestly, because the first batch is deliberately small:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| ~1–3 h | Phase 0.5 tests + Phase 1 skeleton header. **Then it hits the build gate and cannot verify anything.** |
|
||||
| remaining night | The BUILD-FREE QUEUE — dominated by **Q1, the decomposition map**: all 8 archetypes read carefully and broken into source / combiners / modifiers, with every param field traced to its destination op. Genuinely hours of work, zero build risk, and it makes every later port faster. |
|
||||
| morning | A committed build-ready increment, `OPSTACK-PROGRESS.md`, `OPSTACK-DECOMPOSITION.md`, and a "good morning" summary with exact likely error spots. |
|
||||
|
||||
**What it must NOT do overnight:** port archetypes, start Phase 2, or write more plugin C++ once the gate
|
||||
is reached. The prompt says this explicitly, twice, because "fill the night with code" is the tempting
|
||||
wrong answer and it reproduces the stacked-unverified-work pattern the audit was written about.
|
||||
|
||||
---
|
||||
|
||||
## Feasibility, honestly
|
||||
|
||||
| Phase | Effort | Risk |
|
||||
|---|---|---|
|
||||
| 0.5 — three tests | 1 session + 1-2 build rounds | Low. May surface a real bug (that's a win). |
|
||||
| 1 — skeleton + Maze decomposed | 1-2 sessions + 2-3 build rounds | **Medium — this is the go/no-go.** |
|
||||
| 2 — port remaining archetypes | 1 session each, opportunistic | Medium; `TunnelNetwork` last, it owns §8.4. |
|
||||
| 3 — ops as assets, strate = op list | several sessions | Medium-high; big authoring-surface change. |
|
||||
|
||||
**The single genuine unknown is Phase 1**, and it's cheap to find out — one archetype, timeboxed, with an
|
||||
explicit abort. If `Maze` doesn't decompose cleanly into a source + a modifier, the abstraction is wrong
|
||||
for this domain and two days bought that knowledge.
|
||||
|
||||
**The largest hidden cost is not code, it's re-tuning.** Every ported archetype needs its params re-dialled
|
||||
to look right again (§2.6 accepts this). That's Jahni's time in the editor, not a context's time writing
|
||||
C++, and it is likely to dominate the schedule.
|
||||
@@ -0,0 +1,93 @@
|
||||
# PERF-ATTRIBUTION
|
||||
|
||||
Measured 2026-09-12 at `cf0319d` (`experimental`). This was measurement-only: the source changes add
|
||||
diagnostic counters and clocks; they do not change density or meshing decisions. The primary case is
|
||||
the 128^3 `TunnelNetwork`, seed 1337, slot 1, operator-stack setting 1, origin `(-64,-64,-224)`
|
||||
voxels, step 1, 64 canonical 32^3 tiles, and 15 workers.
|
||||
|
||||
`clean` means diagnostics off and `-blockearlyout=0`, with fused evaluation and shared density-grid
|
||||
reuse enabled. The three clean primary runs used an exact-HEAD DLL staged before instrumentation:
|
||||
603.599, 590.746, and 569.627 ms mesh time, mean **587.991 ms**. The requested 0.556 s is therefore
|
||||
within the observed run spread but was not reproduced exactly on this host. `attr` means the final
|
||||
instrumented DLL, `-perfattribution`, sample interval 128. Its wall-clock self-check compares the
|
||||
same export against an unprofiled run in the same process.
|
||||
|
||||
The prescribed build was run successfully with:
|
||||
|
||||
```text
|
||||
dotnet "E:\Program Files\Epic Games\UE_5.7\Engine\Binaries\DotNET\UnrealBuildTool\UnrealBuildTool.dll" UnrealEditor Win64 Development "-Project=E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Saved\BuildHost\HostProject\HostProject.uproject" -WaitMutex -FromMsBuild -architecture=x64 -NoUBA "-Log=E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Saved\PerfAttributionBuild.log"
|
||||
```
|
||||
|
||||
The measured process loaded the staged runtime DLL at
|
||||
`E:\Projet Unreal\VoxelM\Plugins\VoxelForge\Saved\BuildHost\HostProject\Plugins\VoxelForge\Binaries\Win64\UnrealEditor-VoxelForge.dll`, SHA-256
|
||||
`C0CB812BB25EFE8C836DC1C1718C408DA5FE9D9F258B4AE12A6CA846510CA0B2`, length 7,673,856 bytes. The
|
||||
staged editor DLL was `E64C9E80869EFA53FA1556947718DB8E64975ABF6B546657BCFF5D775A2B679A`, length
|
||||
502,784 bytes. The game harness recorded the same runtime path and hash. The primary attribution and
|
||||
final classifier exports returned `status=ok`, geometry CRC `07C14005`, and the required OBJ SHA-256
|
||||
`b3e5f4c398dd0547c6c55f415872058246d4a292c2f0d636ee1cd0829de9b377`.
|
||||
|
||||
The table is the attribution. Only rows marked **additive** are summed. Profiler component rows are
|
||||
sampled nested diagnostics; their CPU and worker-wall figures are evidence about composition, not
|
||||
additional time to add to the wall ledger.
|
||||
|
||||
| Bucket / question | Measurement and mode | Elapsed time, volume, or count | Whole-clock accounting / interpretation |
|
||||
|---|---|---:|---|
|
||||
| Historical comparison | `34f06df` value supplied in the request | 130.000 ms | Historical pre-op-stack reference; the exact old binary was not rebuilt because that commit predates the current editor commandlet and host harness. |
|
||||
| Current clean primary | 3 exports, exact `cf0319d` DLL, diagnostics **off**, fused=1, reuse=1, classifier early-out=0 | **587.991 ms mean**; 569.627–603.599 ms | Clean headline mesh clock. Mean is 4.523x the supplied 130 ms reference. |
|
||||
| Primary attribution self-check | Final DLL, `attr`; profiled 606.128 ms versus same-argument unprofiled 602.341 ms | 1.0063x; +0.629% | Passed. Geometry equal. The profiler did not reproduce the 11.5x inflation seen in the earlier diagnostic mode. |
|
||||
| Full commandlet export | Final DLL, `attr`; mesh plus output and bookkeeping | **1,004.557 ms** | Full export wall, not just mesh generation. The OBJ writer is a material part of this clock. |
|
||||
| Full export: mesh generation | Additive export ledger | 606.128 ms; 60.338% of export | Includes the complete additive mesh ledger below. |
|
||||
| Full export: OBJ/output interval | Additive; includes nested `MeasureExploreMesh` | 394.273 ms; 39.249% | `measure_seconds` is 82.964 ms inside this interval and is not added again. |
|
||||
| Full export: geometry hash | Additive | 2.946 ms; 0.293% | CRC/hash bookkeeping. |
|
||||
| Full export: manifest | Additive | 0.984 ms; 0.098% | Manifest write. |
|
||||
| Full export: remainder | Additive export remainder | 0.225 ms; 0.022% | Export ledger remainder. Mesh + OBJ interval + hash + manifest + remainder = 1,004.557 ms. |
|
||||
| Mesh ledger: preparation | **Additive** | 0.000101 ms; 0.0000% of mesh | No meaningful cost. |
|
||||
| Mesh ledger: shared density-grid fill | **Additive**; 2,248,091 unique shared samples | **590.923 ms; 97.4913% of mesh** | This is the dominant remaining cost. It is the caller-side density-grid stage and includes the canonical fused `GetDensityAt` work. |
|
||||
| Mesh ledger: pre-tile setup | **Additive** | 0.001602 ms; 0.0003% | No meaningful cost. |
|
||||
| Mesh ledger: tile parallel wall | **Additive**; 64 jobs, 15 workers | 10.972 ms; 1.8102% | Per-tile cell scan, interpolation, normals, stream assembly, and task scheduling after the shared grid exists. |
|
||||
| Mesh ledger: tile merge | **Additive** | 4.232 ms; 0.6982% | Cross-tile output merge. |
|
||||
| Mesh ledger: accounting remainder | **Additive** | 0.000097 ms; 0.0000% | Stage sum 606.128205 ms versus mesh clock 606.128301 ms; `sum_matches_mesh=true`. Nothing is hidden in this remainder. |
|
||||
| Mesh ledger total | Final DLL, `attr` | **606.128301 ms** | Complete mesh wall clock. |
|
||||
| `GetDensityAt` parent | Final primary, `attr`; 2,248,091 calls, 17,558 sampled parent intervals, 15 workers | 580.184 ms worker-wall estimate | Nested inside the 590.923 ms density-grid stage. The remaining 10.738 ms is grid-loop/task/read overhead outside the parent scope; it is accounted for by the additive density-grid row. |
|
||||
| Fused evaluator itself | Final primary, `attr`; 17,571 sampled component scopes; 2,248,091 fused tunnel samples | 107.183 ms sampled CPU; 12.301 ms sampled worker wall | Diagnostic sample only, not additive. Detail ran for 1,079,777 samples and was skipped for 1,168,314, about 52.0% of fused lattice samples. |
|
||||
| Fused density core | Final primary, `attr`; sampled nested scope | 108.371 ms sampled CPU; 12.377 ms sampled worker wall | Diagnostic sample only. It is not a 108 ms total bucket and must not be added to the mesh ledger. |
|
||||
| Room/tunnel graph traversal | Final primary counters | Rooms: 11,099,891 candidates / 1,730,920 evaluated; tunnels: 28,951,420 candidates / 3,462,565 evaluated; tunnel-core: 28,951,420 candidates / 1,881,812 evaluated | Room and tunnel candidate traversal is repeated inside the common density work. This is evidence of work volume, not a separately additive clock. |
|
||||
| Room graph build and SDF | Final primary, `attr` | `RoomGraphBuild`: 77 calls, 192.146 ms sampled CPU / 19.323 ms worker wall; `RoomGraphSdf`: 12.101 ms / 0.989 ms sampled CPU/wall | The build is cold/cache construction work; SDF figures are nested samples. The room graph is not a large independent wall bucket on the primary fused route. |
|
||||
| Tunnel-core world evaluation | Final primary, `attr` | 8.857 ms sampled CPU; 0.872 ms sampled worker wall | Nested diagnostic sample. The core traversal is present, but the measured evidence does not support calling it the whole 4.3x. |
|
||||
| Post-stack structural/passage tail | Final primary, `attr`; `GetDensityAt` tail | `StructuralTail`: 20.853 ms CPU / 1.896 ms wall; `DensityStructuralPosts`: 21.909 ms / 1.990 ms; `PassageStructuralPosts`: 10.476 ms / 0.997 ms sampled CPU/wall | This is real current-only post-stack work, but it is a small sampled component relative to the 590.923 ms density-grid wall. All rows are nested and non-additive. Passage modifier was 4.579 ms CPU / 0.404 ms wall; landing-floor and room-floor samples were 1.691 ms / 0.177 ms and 0.665 ms / 0.060 ms. |
|
||||
| Floor/profile work | Final primary counters, fused route; classifier figures from the separate classifier control | Primary support-column builds/queries/checks: 0 / 0 / 0; authored-floor samples: 0; floor-profile builds 2,636; segments 12,624; backstops 6,509 tunnel and 49 room. Classifier control: 211,289 support-column builds, 2,171,763 candidates, 1,370,314 floor queries, 2,296,243 checks, 9,710 authored-floor samples, 97,950 profiles and 442,278 profile segments. | The fused primary route does not execute the classifier’s support-column path. The classifier does, and that work is part of its separate 2.057 s route, not the clean 0.588 s headline. |
|
||||
| Software tunnel cache | Final primary, `attr` | 2,248,043 hits, 48 misses, 77 cache builds | Cache lookup is not the removed verdict cache. The current tunnel cache is overwhelmingly hit; no hardware cache-miss counter was available. |
|
||||
| Classifier absent from the clean headline | Clean exact-HEAD control, diagnostics **off**, classifier early-out=0 | 0 classifier calls | The stated 0.556 s case does not pay interval proofs. It is incorrect to charge classifier time to that baseline. |
|
||||
| Classifier control when enabled | 3 clean exact-HEAD controls, diagnostics **off**, classifier early-out=1 | **2,056.596 ms mean** | +1,468.605 ms versus the clean primary mean; 3.498x the primary. This is a separate, expensive route, not the source of the clean 0.556 s. |
|
||||
| Classifier total self-check | Final DLL, `attr`, classifier early-out=1 | 1,980.835 ms profiled versus 2,013.547 ms unprofiled; 0.9838x | Passed; geometry equal. Profile overhead was -1.62% in this run. |
|
||||
| Interval proofs | Final classifier attribution; 32 sampled calls out of 4,096 | 494,976 refine/box calls; 479,872 whole-mixed, 15,104 whole-solid, 0 whole-air; 63,488 splits; max depth 4 | Proof detail is present and conservative, but its route is costly. The classifier’s scaled diagnostic work was 14,113.190 ms interval-box, 62.221 ms exact-core, and 399.770 ms exact-final microseconds; these are diagnostic work estimates, not additive wall time. |
|
||||
| Classifier repeated room traversal | Final classifier attribution | 2,041,600 room-tail queries, 933,120 evaluated; 6,030,720 exact-core cache hits; 612,864 exact-core samples; 768 exact-final samples | The classifier does repeat room-graph work. Scaled diagnostic work was 11,423.590 ms room propagation, 4,665.818 ms exact primitive, 5,815.002 ms cache-window, and 1,227.366 ms room tail microseconds. |
|
||||
| Shared grid traffic | Final primary, `attr` | 2,744,000 tile-grid samples; 2,248,091 unique shared samples; 495,909 duplicate evaluations before reuse, 0 after | The shared lattice removes duplicate evaluations, but filling the remaining 2.248 M samples is still 590.923 ms. |
|
||||
| Mesher cell classification | Final primary, `attr`; nested sample | 2,097,152 cell calls; 1.344 ms sampled worker wall | The cell loop is not the dominant wall bucket. |
|
||||
| Mesher vertex interpolation | Final primary, `attr`; nested sample | 76,808 calls; 0.012 ms sampled worker wall | Negligible in the observed sample. |
|
||||
| Mesher gradient normals | Final primary, `attr`; nested sample | 76,808 calls; 1.056 ms sampled worker wall | Small nested cost; gradient normals come from the density grid. |
|
||||
| Mesher stream building | Final primary, `attr`; nested sample | 76,808 calls; 3.140 ms sampled worker wall | Small relative to density-grid fill. Runtime stream handoff is not executed by the export commandlet. |
|
||||
| Mesher output and memory | Final primary, `attr` | 82,273 vertices, 153,346 triangles; logical output arrays 7,434,716 bytes; allocated output 8,449,820 bytes; OBJ 17,587,937 bytes; estimated working set 14,851,564 bytes | Byte counters are logical allocator/output traffic, not physical DRAM traffic. |
|
||||
| Grid read/write bytes | Final primary, `attr` | Density-grid bytes 10,976,000; shared-grid read bytes 10,976,000 | The instrumentation sees logical grid traffic. It does not see hardware cache misses, NUMA traffic, or memory-controller events. |
|
||||
| Operator-block scratch/copy | Final primary, `attr` | 0 scratch bytes, 0 copy bytes, 0 builds, 0 operators | The old interpreted block machinery is not on the canonical primary route. |
|
||||
| Operator-block control | Clean diagnostics-off controls with reuse=0 and fused=0 | 2,332.186 ms; 1,566 block builds; 2,486,750 block samples; 31,320 operator visits; 99,470,000 scratch bytes; 101,956,750 copy bytes | This is a deliberately separate block-path control, not the clean fused baseline. It demonstrates the traffic that is absent from the primary route. |
|
||||
| Scalar and fused no-reuse controls | Clean diagnostics-off controls with reuse=0 | Scalar 2,733.080 ms; fused 2,218.132 ms | These controls show the cost of removing shared-grid reuse, but they are not attribution buckets for the primary run. |
|
||||
| What `34f06df` did not do | Read-only source comparison at the historical commit | No active TunnelNetwork operator-stack evaluation, no fused evaluator, no current post-stack tunnel-floor/structural tail, no current editor export commandlet | The old source already had a shared density grid, `ClassifyTile`, marching-cubes cell classification, interpolation, grid-gradient normals, and stream construction. Those are not new current-only work. The old source also had room/tunnel morphology and passage carving, so “rooms/tunnels exist” alone is not a delta. |
|
||||
| Primary operation-level attribution gap | Final primary profiler’s own ledger | Sampled phase sum 14.443 ms versus sampled `GetDensityAt` parent 580.184 ms; sampling gap 565.741 ms | This is the honest limit: component scopes were sampled to keep profiled/unprofiled totals within 0.63%, so their expanded operation-level wall shares are not identifiable from this run. The full mesh wall is nevertheless closed by the additive stage ledger, with a 0.000097 ms remainder. |
|
||||
| Headless game clean check | `PerfAttribGameCleanFinal_20260912`; diagnostics **off**, tile profiling off; 343 LOD0-ready samples | request-to-ready p95 **0.375470 s**; queue wait p95 0.082292 s; generation p95 0.309538 s; collision cook p95 0.052945 s | Secondary check agrees with the earlier conclusion that queue wait is not the main generation constraint. |
|
||||
| Headless game attribution check | `PerfAttribGameAttrFinal_20260912`; diagnostics **on**, tile profiling on; 834 profile lines, 822 applied | request-to-ready p95 **0.395715 s**; queue wait p95 0.083363 s; generation p95 0.315281 s; collision cook p95 0.061387 s | Attribution overhead was modest in the end-to-end game summary. Tile-profile p95 was 1.295847 s, of which classifier was 0.106626 s, mesh 1.284040 s, stream building 0.000361 s, room work 0.008842 s, and stack work 0.012019 s; these game tiles are not the same workload as the single 128^3 export. |
|
||||
|
||||
The surprise is that the clean primary is almost entirely the shared density-grid fill: the ordinary
|
||||
mesher stages after it total only about 15.204 ms. The fused evaluator is genuinely on the canonical
|
||||
path, and the room/tunnel candidate counters show substantial repeated traversal, but the sampled
|
||||
component timers cannot honestly turn that 590.923 ms into exact per-operation percentages without
|
||||
reintroducing the profiler inflation this round was designed to avoid. The classifier is an even
|
||||
larger cost when enabled, yet it is absent from the clean baseline. The old path already performed
|
||||
the grid and marching-cubes phases, so those phases should not be described as newly introduced
|
||||
work; the current-only items are the active fused route and the later structural/floor machinery.
|
||||
|
||||
What could not be attributed: physical allocation-event counts, physical memory/cache misses, and an
|
||||
exact numeric per-operation delta against the old `34f06df` binary. The old commit has no compatible
|
||||
current export commandlet, and rebuilding it would not be the same measurement harness. The complete
|
||||
current clocks, deterministic geometry, clean/profiled comparison, current counters, and all logical
|
||||
memory-byte counters are accounted for above. Existing logs and outputs under `Saved\` were retained.
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,371 @@
|
||||
# WORK-NEXT — the queue after the read-only audit (2026-09-11)
|
||||
|
||||
Ordered by importance, then by size. Everything here comes from Sol's read-only audit of
|
||||
`experimental` @ `aa62971`, checked against `main`.
|
||||
|
||||
The audit's two-sentence verdict:
|
||||
> *The operator abstraction leaked into the innermost loop, and streaming treats
|
||||
> "work submitted/applied" as "safe to stand on."*
|
||||
|
||||
**Standing rules for every item**: determinism is absolute (same build + same seed = same world on
|
||||
every machine); output may change between builds; nothing is deleted to go fast; capability before
|
||||
speed (player-fit 20,830 / walk-reachable 10,909 — do not drop below); scope is
|
||||
`Plugins\VoxelForge` only, no links, absolute `-out=` paths under the plugin's own `Saved\`.
|
||||
|
||||
---
|
||||
|
||||
## 1. Collision-ready before player-ready · CORRECTNESS · medium
|
||||
|
||||
The player falls through the world. There is **no invariant** that a tile you occupy has collision.
|
||||
`main` has no such barrier either — it was simply fast enough to hide the race.
|
||||
|
||||
- `apply_s` ends at mesh **submission**. RMC cooks collision **asynchronously** and we never record
|
||||
cook-complete. We have never measured when physics can actually collide.
|
||||
- A tile enters `LoadedTiles` before its apply path completes (`VoxelWorld.cpp:1532`).
|
||||
- Only LOD0 carries collision (`VoxelWorld.cpp:3109`), so visible coarse terrain can exist while the
|
||||
collidable tile is still queued.
|
||||
|
||||
**Do**: record cook-complete; gate the pawn (spawn and movement) on the supporting tile being
|
||||
collidable, not merely applied.
|
||||
|
||||
✅ **Owner playtest, 2026-09-13: he can no longer fall through the world.** The gate uses RMC's
|
||||
`SetCollisionConfig()` future as the authority (`HandleTileCollisionCookComplete`,
|
||||
`IsPlayerSupportCollisionReady`).
|
||||
|
||||
📖 **Reading `Plugins/RealtimeMeshComponent` is permitted for this item — READ ONLY.** Find whether a
|
||||
cook-complete delegate/flag exists. ⛔ No edits, no builds against it: the owner's copy carries custom
|
||||
modifications and was destroyed once by an over-broad delete.
|
||||
|
||||
## 2. Lower the operator graph into a fused evaluator · PERFORMANCE · large
|
||||
|
||||
This is where the ~8× lives (pre-op-stack `34f06df` did 128³ in 0.130 s; we are at ~1.04 s).
|
||||
|
||||
A LOD0 tile samples 42,875 positions. The old path short-circuited the whole cave suite per sample
|
||||
when `bNearCaveSurface` was false — roughly **70% of deep-solid samples**. We now allocate a rich
|
||||
`FVoxelOpSample` per position, run each active operator across the entire block, use scalar fallbacks
|
||||
for most operators, copy scratch in and out, and still call scalar `GetDensityAt` per grid point.
|
||||
|
||||
> *An operator whose interval intersects the surface anywhere in the block runs across the whole
|
||||
> block; the old path could decide per sample that expensive detail work was irrelevant.*
|
||||
|
||||
**Do**: keep the authored operator graph, but **compile/lower it** into a fused evaluator that restores
|
||||
per-sample short-circuiting and stops spilling `FVoxelOpSample` through memory per operator.
|
||||
⚠️ Byte-identical geometry is the acceptance test. Stage it so the tree never breaks.
|
||||
|
||||
Note: the D-floors/terracing features are only ~17% of the penalty — not the 8×.
|
||||
|
||||
## 3. Cancel obsolete work, and prioritise the floor · BOTH · medium
|
||||
|
||||
Nothing aborts a tile when the player moves and it stops being desired. Work is distance-sorted from
|
||||
an artificial chunk centre, not the pawn, with no collision-critical priority
|
||||
(`VoxelWorld.cpp:1864`). Up to 14 workers can be busy with stale coarse tiles while the player's LOD0
|
||||
floor waits.
|
||||
|
||||
**Do**: abort no-longer-desired tiles; sort from the true pawn position; give the occupied tile and the
|
||||
next tile along the heading absolute priority.
|
||||
|
||||
## 4. Collapse the duplicated field semantics · PERFORMANCE ARCHITECTURE · medium-large
|
||||
|
||||
The effective field now lives in three places: the operator stack, the classifier's parallel interval
|
||||
logic, and a post-stack repair tail in `GetDensityAt` (origin floors, passage structure, passage
|
||||
floors, room floors). `FRoomGraphSource` evaluates the room graph then traverses tunnels *again* for
|
||||
metadata (`VoxelDensityOpStack.cpp:4234`).
|
||||
|
||||
The audit calls this *"the clearest sign of overengineering"*: later fixes compensating for where an
|
||||
operation sits in the pipeline. It also makes interval proofs harder to trust.
|
||||
|
||||
**Do**: make the stack the single semantic source of truth. Likely overlaps heavily with item 2.
|
||||
|
||||
**Promoted 2026-09-12**: this is performance architecture, not maintenance. The classifier is a
|
||||
second evaluator (interval proofs, exact certificates, and final-field validation) beside the
|
||||
canonical lowered evaluator, and the runtime can validate the same tile again before meshing.
|
||||
Items 1 and 2 in the current round deliberately leave this lowering work untouched; any
|
||||
classifier retained later must consume the lowered representation and reusable density data.
|
||||
|
||||
## 5. Verdict cache removed after moving-session audit · complete
|
||||
|
||||
The clean moving-session audit found zero outer or nested hits: 2,109 outer probes and 19,904 nested
|
||||
probes produced 15,877 stores and 15,365 replacements in the full 512-entry table. Same-binary
|
||||
cache-on/cache-off request-to-ready was 0.367645/0.365978 s p95 and generation was 0.285559/0.285992
|
||||
s p95, so the cache was neutral within run noise while paying the global lock and linear scans.
|
||||
|
||||
The global verdict cache, lock, entry table, invalidation path, cache-only profiler fields, and
|
||||
deliberate cache probe are removed. The proof classifier remains; item 4 is still deliberately untouched.
|
||||
|
||||
## 6. Three small, sharp ones · tiny
|
||||
|
||||
- **The zero-position guard.** Streaming only starts when `GetPlayerPosition()` is not exactly
|
||||
`FVector::ZeroVector` (`VoxelWorld.cpp:1167`, `:1359`) — a pawn at the world origin is
|
||||
indistinguishable from "no pawn", and **the owner spawns at (0,0)**.
|
||||
- **`ApplyMeshToTile` has no success return** (`VoxelWorld.cpp:3064`). If runtime-mesh init fails the
|
||||
tile stays recorded as loaded and never retries — a permanent hole.
|
||||
- **Whole-tile uniform verdicts are trusted without exact validation.** Block verdicts get checked
|
||||
against the full grid when meshing; an outer AllAir/AllSolid verdict has no equivalent safety net.
|
||||
|
||||
## 7. Deferred, with reasons
|
||||
- **Coarse-first LOD refinement** — blocked until coarse tiles can carry collision, or item 1 lands.
|
||||
- **GPU evaluation** — right hardware, but bit-exactness across vendors/drivers is unproven and
|
||||
determinism is absolute. Item 2 is the honest prerequisite either way.
|
||||
- **The origin spawn room** — the owner considers it a design mistake that should not exist. Design
|
||||
conversation, not a fix.
|
||||
|
||||
---
|
||||
|
||||
## Outer-classifier A/B — measured 2026-09-12 (nested refinement OFF in every mode)
|
||||
|
||||
Clean, diagnostics off, headless game path. Static = 343 LOD0 samples, two runs per mode; moving =
|
||||
160 m at 8 m/s, 1,274 LOD0 samples. Density calls = validation + mesher.
|
||||
|
||||
| mode | ready p95 static | gen p95 static | worker CPU static | density calls static | ready p95 moving | gen p95 moving | density calls moving |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
| 0 no outer classifier | **182.4-182.7 ms** | **124.9-127.0 ms** | **184.9-193.5 s** | **18.05 M** | **198.9 ms** | **129.9 ms** | **57.7 M** |
|
||||
| 1 current (validate, discard) | 189.7-194.8 ms | 139.1-140.6 ms | 201.2-202.3 s | 21.89 M | 208.0 ms | 143.4 ms | 69.6 M |
|
||||
| 2 validation-grid reuse | 190.0-194.7 ms | 140.3-141.3 ms | 201.2-201.3 s | 21.89 M | 210.7 ms | 143.3 ms | 69.6 M |
|
||||
|
||||
**Verdict: the outer classifier is net-negative on the game path.** Mode 0 is better on every latency
|
||||
figure and does ~18% fewer density calls. It cannot remove geometry, since it skips nothing.
|
||||
|
||||
**Mode 2 cannot win as designed.** It reuses the validation grid only when a uniform verdict is
|
||||
*disproved* (`VoxelWorld.cpp` ~4012), and validation disproved none: 0 reuse tiles in every run. Mixed
|
||||
tiles, the ones that actually need a grid, never run validation, so there is nothing to reuse.
|
||||
|
||||
**Done 2026-09-13, owner approved:** `GVoxelForgeOuterClassifierMode` now defaults to 0, and mode 2
|
||||
is deleted along with its reuse grid, its mesher parameter and its counter. Mode 1 is still
|
||||
reachable with `voxel.OuterClassifierMode=1` as an A/B instrument. Verified:
|
||||
- export OBJ SHA-256 unchanged (`b3e5f4c3..9b377`)
|
||||
- a run without the override prints `outer_classifier_mode=0 validation_density_calls=0`
|
||||
- **841/841 tiles have identical triangle counts between mode 0 and mode 1**. The 394 tiles mode 1
|
||||
skipped as uniform produce 0 triangles in mode 0, so the strate band cut was not hiding geometry
|
||||
behind the classifier.
|
||||
|
||||
⚠️ **Correction to `f204413`.** Its message says the nested classifier's 3.5x "cannot be reproduced".
|
||||
That was my measurement error: I passed `-voxel.UseBlockEarlyOut=1` (the STREAMING console variable)
|
||||
to the export commandlet, which reads its own `-blockearlyout=` argument, so both of my runs had
|
||||
nested refinement off. Re-run with the right flag: `-blockearlyout=0` 0.553 / 0.572 s,
|
||||
`-blockearlyout=1` 2.036 / 2.060 s, which is **3.6x**, with byte-identical geometry. The 3.5x was real,
|
||||
and disabling nested refinement in streaming (`f204413`) was worth it.
|
||||
|
||||
## Room-cache work, 2026-09-13 (`2990009`, `2de603c`, `cfad3d1`)
|
||||
Player-fit memo, then the tile-sized room cache gated to LOD3+ (fused) / LOD4+ (op-stack). Against
|
||||
`2990009`, static worker CPU is 48 -> 39 s and moving 125 -> 114 s, with identical geometry.
|
||||
|
||||
⚠️ **Correction to `cfad3d1`.** Its message says absolute numbers drift because "the machine is in
|
||||
daytime use." That was my unverified guess, and it is false: the owner had not used the PC all day.
|
||||
The real signal is systematic. The switch-off baseline came out at 58 s in two rounds hours apart,
|
||||
against 48 s at `2990009`. So something left ON with `voxel.TileCacheWindow=0` costs ~10 s, and LOD0
|
||||
generation p50 is ~4% slower than at `2990009` (73 -> 77 ms static). Queued: attribute it with
|
||||
in-tree switches (first suspect: `voxel.SpatialIndex` auto-on at fused LOD1-2), never by running old
|
||||
commits.
|
||||
|
||||
**Resolved 2026-09-13 (`ab4bbe3` + the switch sweep):** not the spatial index; every `SpatialIndex`
|
||||
setting is within ~1%. `TileCacheWindow=0` at HEAD is a different configuration from `2990009`, not
|
||||
the old path. It still thrashes at L3/L4 (p95 ~250 ms), now at 54 s. The DEFAULT beats `2990009` on
|
||||
every metric: static worker 48 -> 34 s, moving 125 -> 101 s, LOD0 generation p50 73 -> 66 ms static
|
||||
and 70 -> 62 ms moving. The "LOD0 ~4% slower" worry is gone. `SpatialIndex` now defaults to 1
|
||||
(marginally best, within noise). Per-tile identity: 0/841 tiles differ against traces recorded
|
||||
before the always-on LOD0 changes.
|
||||
|
||||
**Next LOD0 target, found by the sampler's caller aggregation:** the "unknown" ~13.8% LOD0 leaf is
|
||||
`ucrtbase.dll!_finite`, the non-inlined CRT call behind `FMath::IsFinite`. Its callers are the tunnel
|
||||
chain, passage landing/floor checks and `GetDensityAt`. An inline bit test is exactly the same
|
||||
predicate, so it changes no field.
|
||||
|
||||
## State after 2026-09-14 (HEAD `7a972f5`)
|
||||
|
||||
**In-game generation since the start of 2026-09-13**, headless game path (static = 841 tiles;
|
||||
moving = 160 m at 8 m/s):
|
||||
```
|
||||
worker CPU static LOD0 generation p50 static
|
||||
c65b015 classifier off ~184 s ~75 ms
|
||||
2990009 player-fit memo 48 s 73 ms
|
||||
cfad3d1 tile cache window (LOD3+) 39 s 77 ms
|
||||
ab4bbe3 field-exact LOD0 overhead 33 s 66 ms
|
||||
5489b5e inline exact IsFinite ~31-34 s ~62-66 ms
|
||||
7a972f5 (current) ~34 s ~66 ms
|
||||
```
|
||||
The later rounds' absolute figures come from slightly different harness settings. Compare within a
|
||||
round.
|
||||
|
||||
**Measured, not guessed: what each component costs** (`6ccf371`, remove-one ablation, static worker
|
||||
saving): tunnel SDF 11.4% (and it owns the LOD0 p95 tail), cave warp 10.3%, passage carving 10.1%,
|
||||
landing posts 9.9%, structural posts 5.6%, worms 2.4%, everything else <= 2.2%. Ablations change
|
||||
the field, so these mix computation cost with the cost of the geometry each component creates.
|
||||
|
||||
**Tried and not kept:**
|
||||
- the exact worm block skip: 4% skip rate, neutral, off by default;
|
||||
- value-noise / lattice worms: no gain, removed;
|
||||
- exact culling (SmoothMin cutoff, tile candidates, post early-outs): the build was 23% slower even
|
||||
with every switch off, parked on local branch `wip/exact-cull-20260914`. Only the tunnel-core
|
||||
ablation gate fix was ported.
|
||||
|
||||
**Measurement lessons**, recorded because each cost a round:
|
||||
- line-level sampler attribution of inlined code is a hint; confirm it with an ablation;
|
||||
- a switch-off A/B inside a new build cannot see always-on cost; always compare the final default
|
||||
build against the committed tip's DLL, with a fixed command line;
|
||||
- launch Unreal serially (it is a GUI exe that returns at once), and reap crash reporters.
|
||||
|
||||
**Next candidates:** a cheaper cave warp (field-changing: coarse warp lattice); the tunnel SDF (field-
|
||||
changing merge of the warped SDF and the world-space core); a real per-tile candidate filter
|
||||
designed to cost nothing per sample; the cold read-only audit's findings.
|
||||
|
||||
## State after 2026-09-15 (HEAD `a668833`)
|
||||
|
||||
**Gameplay fixes, from the owner's playtest:**
|
||||
- `4c8d06f` the player can walk from one strate into the next. `StrateContentCutMinLevel`
|
||||
defaulted to 0, which content-cut the LOD0 collision tiles, so the collision gate blocked the
|
||||
passage: a deadlock. Regression test `-voxel.TestStrateCrossing=1`.
|
||||
- `39c306b` its +29% cost recovered by an exact sealed-rock proof (`TryProveSealedSolidTile`).
|
||||
- `cf2b2ad` tunnel floors, option B (the owner's choice): no staircases; steep tunnels WIND
|
||||
(walk-critical, a reserved spanning forest) or get a few big climbable DROPS (only leaf/redundant
|
||||
edges). Visual acceptance is the owner's in-game check; the renders did not show it.
|
||||
- `9260775` passage D-floors were missing inside gap chunks (a real bug since `adb34df`), plus
|
||||
two stale tests corrected.
|
||||
|
||||
**Correctness:** `8879045` fixed the cold audit's findings (`AUDIT-COLD-2026-09-14.md`), including
|
||||
a determinism bug introduced by the tile-cache work (a moved-from operator stack could be evaluated
|
||||
on a cache miss), plus cache world-identity, lifetimes and input guards.
|
||||
|
||||
**Performance since `7ece3ce`** (each measured against the committed tip's DLL, exact field):
|
||||
`ce4d878` landing/structural reach -12%; `1d824aa` cave warp SIMD batching -4%; `a668833`
|
||||
tunnel-core tile reach + passage-carving 8^3 reach -1.3 to -2.6%. Static worker time is now ~28 s,
|
||||
from ~184 s at the start of the push, with the same world and capability.
|
||||
|
||||
**Current cost map** (`0208e93`, remove-one, static): tunnel core 20.8%, tunnel SDF 12.6%, passage
|
||||
carving 9.1%, cave warp 6.7%, structural posts 6.6%, unattributed ~28%. The tunnel core is
|
||||
near-dominated (~99% of its cycles are in tiles near tunnels), so exact skips cannot remove it. The
|
||||
next lever is the field-changing SDF/core merge (in progress), which will also add an automation
|
||||
test for the reach proofs.
|
||||
|
||||
**Parked:** `wip/exact-cull-20260914` (per-sample culls, a 23% always-on regression; only its gate
|
||||
fix was ported).
|
||||
|
||||
**CLOSED 2026-09-15: the tunnel SDF/core merge. Do not re-propose it without a new idea.** The
|
||||
world-space core is load-bearing: `WorldControlPoints` keep the authored room-floor anchors and
|
||||
walkable routes before cave warp (`8e42d6d`, `cf2b2ad`). Replacing it cost 1,146 fit / 975
|
||||
walk-reachable cells; the exact shared-segment variant was 5.6-12.6% SLOWER. The remaining big
|
||||
costs (tunnel core ~20%, tunnel SDF ~13%) are the geometry itself, so the exact-skip perf push has
|
||||
reached diminishing returns. The reach proofs are now guarded by
|
||||
`VoxelForge.Correctness.TilePostReachProof`.
|
||||
|
||||
## State after 2026-09-16: Sol's design review, deterministic maths (`10fd0b6`)
|
||||
|
||||
**Reframe.** Local "find the hot spot" rounds had reached diminishing returns, so, on the owner's
|
||||
idea, Sol High was asked how it would build the project from scratch (`DESIGN-SOL-2026-09-15.md`),
|
||||
then read the code and said what could reasonably change (`DESIGN-SOL-2026-09-15-CODE.md`). Its
|
||||
verdict: don't rebuild. There are two performance bets, each with instrumentation-only first steps
|
||||
and kill criteria, plus one determinism hole. Owner's rule: each idea gets its own branch and is
|
||||
dropped if it doesn't hold capability and performance.
|
||||
|
||||
**Done: `sol/deterministic-math`, merged.** On Win64, `FMath::Sin/Cos` were CRT `sinf/cosf`, and the
|
||||
UCRT picks an FMA3 variant by CPU at start-up. Now `VoxelMath::DetSinCos` (fixed reduction and
|
||||
polynomial, max error 5.96e-8) is used on every world-deciding site; `-voxel.CrtFma3=0` forces the
|
||||
CRT's other path, and exports are byte-identical either way. Before the fix, the two paths also
|
||||
matched on this Ryzen, so the hole was real but not observed here. Capability is exact;
|
||||
static/moving worker time is -3.6%/-3.5% vs the tip DLL. Order-independent vertex distance is at
|
||||
most 8.7 micrometres (a line-by-line OBJ diff reported "11.4 m", a line-order artefact: compare
|
||||
meshes by nearest vertex, not by line).
|
||||
|
||||
**New canonical hashes (the world changed once, by design):**
|
||||
- export-only OBJ SHA-256 `CBB8AE185E949B33CF2656441BA4AB17398975A098967870DFD45B435A37906F`
|
||||
(geometry `B9EF84AC`), was `b3e5f4c3..9b377`;
|
||||
- worms/capability region `9242C093`, was `311CA42C` (walk OBJ `20CCF7BD..0100`);
|
||||
- field digest (cross-platform test grid) shape `0x767FB0650936435D`, field `0x2AE570A5A1A9C5CE`;
|
||||
- capability unchanged: 20,830 / 10,909, connected.
|
||||
|
||||
**Rung 1 done (`b003bee`, `ba7a083`, branch `sol/perf-probes`, probes off by default):**
|
||||
- ⛔ **Bet A (tile-wide feature-major tunnel evaluation) is KILLED. Do not re-propose it without a
|
||||
new idea.** The per-sample lookup it removes is only 5.1-6.1% exclusive, and feature-major
|
||||
visiting would touch **5.2-6.6x more lattice points** than the current code evaluates (static
|
||||
LOD0: 49.3M predicted vs 9.5M expensive evaluations), because the cheap AABB/sphere rejection
|
||||
(63M of 80M visits) is exactly what it throws away. The swept kernel is branch-divergent
|
||||
(per-sample segment choice, ownership and support branches), so no low-risk X-lane SIMD.
|
||||
- ✅ **Bet B (cross-tile density reuse) cleared every gate**, and the orchestrator re-derived it
|
||||
from the raw trace independently (`scratchpad/dupcheck.py`): static LOD0 20.04% of density calls
|
||||
are exact duplicates, 80.6% with a donor already finished (~2.96 s of 18.3 s); moving 20.70%,
|
||||
63.3% (~5.94 s of 45.3 s). Ceiling ~16% static / ~13% moving of worker CPU.
|
||||
|
||||
**Rung 2 (`sol/tile-sample-reuse`, WIP `9a67452`): INTERRUPTED by the Codex usage limit**
|
||||
(2026-09-16 02:07, resets 19 Sep 12:44). The slab cache, its key test, mesher donor views and
|
||||
`VoxelWorld` plumbing are committed **UNVALIDATED: never compiled, run or measured** (zero Unreal
|
||||
launches that round). Resume brief: `scratchpad/tile-sample-reuse-resume.md`. Acceptance unchanged:
|
||||
byte-identical canonical export, capability 20,830 / 10,909, >=6% worker CPU in both scenarios vs
|
||||
the tip; kill under 4%. Also still owed: a same-session switch-off A/B, because rung 1's probes-off
|
||||
static (28.18 s) sits above the maths round's (27.41 s) for the same code.
|
||||
|
||||
**Then:** landing gap (ARCHITECTURE, passage landings): replace the random-reach fallback with a
|
||||
deterministic retry.
|
||||
Smaller hardening noted by Sol, not scheduled: field-wise cache hashing instead of raw-memory
|
||||
CRC32; an integer-lattice mesher entry point; server-sequenced edit order once networking exists.
|
||||
|
||||
## State after 2026-09-17 night: tunnels from the owner's playtest (branch `fix/tunnel-shape`)
|
||||
|
||||
**Found in play, not in harness:**
|
||||
- His editor had been running a 14 Sep build (no crossing fix: `CutMinLevel=0`).
|
||||
- A gentle slope of 0.1 froze world loading.
|
||||
- Tunnels were ribbed on floor, walls and ceiling with every terrace setting at 0.
|
||||
- Tunnels poke into rooms.
|
||||
|
||||
The harness uses plugin defaults, not his `DA_Strate3`/`DA_Settings`. **Reproduce owner reports with
|
||||
his assets staged into `Saved\BuildHost`.**
|
||||
|
||||
**Cause of the ribs (`4fa3931`), proven in game by the owner.** `voxel.TunnelAblateTunnelCore=1`
|
||||
removes them. That switch is read ONCE per process, at the first Play: set it before pressing Play,
|
||||
and check `[VoxelForgeTunnelAblation] resolved mask`. The tunnel core's final reassertion replaced
|
||||
interior samples with a fixed +16 air density, so the field lost its distance gradient and marching
|
||||
cubes snapped the surface to the voxel grid. It now reasserts the continuous `-Core.SDF`. The
|
||||
periodic amplitude along a tunnel is now under 1.1 cm (was up to 16.9 cm on walls).
|
||||
- ⛔ The first attempt (`0231072`, a Catmull-Rom centreline) was wrong, cost 18-30% CPU, and is
|
||||
reverted. It was "verified" by small, unmatched renders. **Use a quantitative instrument**, e.g. the
|
||||
surface-position oscillation along the axis (`tunnelcoreprobe` mode in the explore commandlet).
|
||||
|
||||
**Owner decisions implemented:**
|
||||
- no authored tunnel floor: `bTunnelFloorEnabled` is ignored and the slab/support-floor paths are
|
||||
detached;
|
||||
- slope in degrees (0-90, migrated from tangents, 3840-voxel route cap);
|
||||
- winding and ledges independent of any floor switch.
|
||||
|
||||
Canonical walk: Connected, player-fit 41,184, walk-only 13,879 (was 20,830 / 10,909). One perf pair:
|
||||
+2.7% static / +4.2% moving vs `bf9384a`, to be re-measured. Not merged: waiting for the owner's
|
||||
in-game look.
|
||||
|
||||
## State after 2026-09-18 night: room wins over tunnels (branch `fix/tunnel-mouths`, `d01af46`)
|
||||
|
||||
The owner played the merged rib fix: *"the tunnels do look better now"*, performance on par. Then he
|
||||
reported a rock shelf spanning a room, ripple rings around tunnel mouths, and tunnels still running
|
||||
into rooms.
|
||||
|
||||
- **Cause of the shelf:** the tunnel core reasserted solid below its own floor **inside room
|
||||
volumes**, plus the landing-floor post. Measured as tunnel-attributed solid inside rooms:
|
||||
`469 → 0`. A matched re-render of his room shows the slab gone.
|
||||
- **Room ownership is now enforced in every tunnel path** (generic tunnel SDF, core evaluator, both
|
||||
native fallbacks): a room suppresses the tunnel bottom inside itself, while a short fade keeps the
|
||||
exterior mouth fillet.
|
||||
- **Mouth trim is now ON by default** (`voxel.TunnelMouthTrim`, dev-only cvar, shipping constant
|
||||
true). It broke walkability before; it no longer does, because trimmed endpoints take their height
|
||||
from the room's own vertical zero crossing (join offsets `-2.08/+0.40 → -1.13/-0.71` voxels).
|
||||
- **Mouth ripples:** periodic amplitude `1.49 cm` floor / `0 cm` wall (degree-six trend removal;
|
||||
raw local-median `6.19 cm` kept as audit). ⚠️ Instrument lesson: a broad room-floor bowl reads as a
|
||||
6 cm "period" unless the slow trend is removed first.
|
||||
- Canonical walk `Connected`, `42,870` fit / `17,413` walk-reachable (was 41,184 / 13,879); export
|
||||
identical with `-voxel.CrtFma3=0`; `TilePostReachProof` and `DetSinCos` pass.
|
||||
- Left when the owner reclaimed his PC: strate crossing and the perf pair.
|
||||
|
||||
**Harness notes (they cost real rounds):**
|
||||
- `-run=VoxelForgeExplore` belongs to explore/export/probe launches ONLY. With it, automation tests
|
||||
and `-game` runs silently run the commandlet instead.
|
||||
- The staged host's `Content` lost `/Game/Untitled`; the crossing test needs that level (it carries
|
||||
the `AVoxelWorld` actor), else Unreal fails the map load and crashes on the empty world.
|
||||
- Codex's headless runs deleted the owner's `%LOCALAPPDATA%\...\EditorSettings.ini` on 17 Sep (a
|
||||
failed cross-volume move, error 5). Runs now use isolated user/config dirs; verify that file's
|
||||
hash before and after a round.
|
||||
|
||||
**Queued, waiting for the owner's go:**
|
||||
1. Terracing he can target: rooms / tunnels / terrain, and floors / walls / ceilings (brief ready:
|
||||
scratchpad `terrace-targets.md`).
|
||||
2. Tunnel mouths trimmed at room walls (`voxel.TunnelMouthTrim`, default off: 36 m walk gap when on).
|
||||
3. The landing random-reach fallback.
|
||||
4. Bet B rung 2 (paused).
|
||||
|
||||
### Open design questions for the owner
|
||||
- All five strate slots resolve to `DA_Strate3`. Intended?
|
||||
- Should a composer roll be allowed to overwrite an explicitly authored value?
|
||||
Reference in new issue
Block a user