VoxelForge/CODEX-TASK-008-column-memo-and-t1d-diagnosis.md

# Codex task 008 — (A) fix the measured column-memo thrash, (B) diagnose why T1.d never fires in game

**Owner:** Codex (Model Luna, xHigh) · **Orchestrator:** Claude · **Branch:** `experimental`
**Status:** specified, not started

Two independent changes in different subsystems, deliberately bundled into one build because their
signals cannot contaminate each other: (A) is SurfaceWorld column caching, (B) is a counter in the
cave branch of `ClassifyTile`.

---

# PART A — replace the hashed column memo with a direct-indexed box

## This is now MEASURED, not suspected

`stat VoxelForge` in the running game, SurfaceWorld-dominated flight:

```
Column Memo Hits    avg 102,300.84
Column Memo Misses  avg  17,683.60      → miss rate 14.7%
```

Predicted **14.0%** if the table thrashes, **1.4%** if it does not. It thrashes. Second confirmation
from a different statistic: 17,683 misses ÷ 2.13 tiles meshed = **~8,300 column recomputes per
tile**, where a healthy cache does ~1,225 — **6.8×**.

**The cause.** `FSurfaceColumnSource::GetColumn` (`VoxelDensityOpStack.cpp` ~615) uses a
**direct-mapped, 4096-entry hashed** table. A direct-mapped table evicts on *collision*, not on
fullness: 1225 columns per tile in 4096 slots is a load factor of 0.30, at which ~317 columns (26%)
share a slot and evict each other — **on every one of the ~35 Z planes**, because the mesher
pre-samples Z-outermost (`VoxelMarchingCubesMesher.cpp` ~226). Each miss recomputes the entire height
stack: structural source, cliff (four structural resamples), terrace, layer-line, beach, ceiling.

## The fix — copy the scheme that already works, one file away

`GSurfColCache` / `FSurfaceColumnBox` in `VoxelGenerator.cpp` (~152, and its use at ~737) is the
original path's solution to the identical problem: a **direct-indexed box** —
`CI = (IY - Box.BaseY) * Dim + (IX - Box.BaseX)` with a `Computed[CI]` flag — centred on the first
sample and rebuilt when a query leaves it. **No hash ⇒ no collisions ⇒ every column computed exactly
once.** Read it before writing; you are porting a proven scheme, not inventing one.

Apply the same structure inside `FSurfaceColumnSource`, keeping it `thread_local`.

## ⚠️ The invariant that must not be lost in the port

The current memo's validity check is `S.Key != ColumnKey || S.X != WorldX || S.Y != WorldY`, and
`ColumnKey` is built in `PrepareChunk` from **strate + layout version + seed + `ParamsFingerprint`**.

**`ParamsFingerprint` is load-bearing and its absence was a real shipped bug** — without it, two
stacks of the same strate with different params shared columns, the overhang silently vanished, and
only 69 of 20000 samples showed it. The comment at the site records this. **The new box's validity
key must still contain all four**, or you reopen a fixed bug. (Note `GSurfColCache` itself keys on
`(box XY, StrateKey, Seed, LayoutVersion)` **without** the fingerprint — do **not** copy that part;
it is the weakness the op-stack memo deliberately closed.)

Also keep the full XY comparison semantics: a lookup must never return a column computed for a
different XY. With a direct-indexed box that is structural (the index *is* the XY), but the box
bounds check must be exact.

## Keep the counters

`ColumnMemoHit` / `ColumnMemoMiss` must keep working, incremented on the same meaning (miss = a
column was recomputed). They are the before/after instrument.

## Invariants

1. **Density must not move by one bit.** This changes *caching*, never a computed value. The eight
   equivalence tests — especially `SurfaceHeightEquivalence` and its overhang section — must stay
   bit-identical. If your diff changes what `TerrainStack.EvalHeight` / `CeilingStack.EvalHeight`
   compute, or the overhang gate maths, you have the wrong site.
2. Both consumers keep working: the source (`~858`) and the overhang (`~923`, via
   `Column->GetColumn`). The overhang **must** see the same column the source did — that is
   "by construction rather than by convention", and the current code says so.
3. Sizing: state in a comment how many columns a tile needs (a 35×35 grid = 1225) and size the box so
   one tile fits without eviction, as `FSurfaceColumnBox` does.
4. `thread_local` stays; no shared mutable state across workers.

## Acceptance

- `ColumnMemoMiss` drops roughly **10×**; miss rate goes from **14.7% → ~1.5%**.
- The eight equivalence tests stay green and bit-identical.
- If the miss rate does **not** fall, say so plainly — a fix that does not move its own instrument is
  a failed fix, not a partial one.

---

# PART B — name which guard stops T1.d in the running game

## The problem

`Tiles Operator Stack Solid` / `Air` **never appeared** in `stat VoxelForge`, while the harness proves
11 of 40 tiles at production defaults. Rows only render in frames where a counter fires, so site B is
never reached in game. The cave branch of `UVoxelGenerator::ClassifyTile` has **13 `return
EVoxelTileClass::Mixed` paths** and we cannot tell which one fires.

## What to build

Add DWORD counters to the existing `stat VoxelForge` group (`VoxelStats.h` / `.cpp`) that attribute
the bail, grouped by *reason* rather than one per line:

| counter | fires when |
|---|---|
| `CaveBailNotOpStack` | `UsesOperatorStackForChunk` is false (either the initial check or the per-chunk sweep) |
| `CaveBailMixedContent` | `bAnyNonCave` — the tile also touches a gap or SurfaceWorld chunk — or a second cave slot, or out-of-layout |
| `CaveBailParams` | the params `Memcmp` disagreed across the box, the archetype differed, or `NumChunkCoords > 27` |
| `CaveBailStackVerdict` | the stack built fine but `ClassifyBox` returned `Mixed` |
| `CaveBailDisturbance` | the final `bCanSolid == bCanAir` after disturbances |
| `CaveBailNoStack` | `VF_BuildOpStackForChunk` returned false |

Increment **exactly one** per bail, immediately before the `return`. Together with the existing
`TilesOpStackSolid` / `TilesOpStackAir`, one underground flight then names the cause outright.

## Invariants

1. **DO NOT change any control flow, condition, or return value in `ClassifyTile`.** Every `return`
   there is a conservative guard that fails to `Mixed`; a wrong verdict leaves a tile with no
   geometry and no collision. Add counters beside the existing returns and nothing else.
2. `ClassifyTile` is `const` and runs on **worker threads** — `INC_DWORD_STAT` only, never a
   `static int32++`. It routes through `FThreadStats::AddMessage` (per-thread packets), which is why
   it is safe.
3. Zero cost when `STATS == 0`: compute nothing outside the macros.
4. Do not touch the non-cave parts of `ClassifyTile` (gap / SurfaceWorld / column scan).

## Acceptance

Fly underground in a TunnelNetwork strate: exactly one bail counter should dominate, or
`TilesOpStackSolid` should finally appear. Either outcome is a result.

---

# Shared rules

- **NEVER build, compile or run the editor or the tests.** Stop when the code is written.
- **Do not `git commit`, `git push`, `git checkout`, `git stash`, `git restore`.** Uncommitted work
  in the tree must survive.
- Comments are French + English; match the surrounding file.
- Macro spelling: `KINDA_SMALL_NUMBER`, not the `UE_`-prefixed form.
- When inserting anything into `VoxelDensityOpStack.cpp`, put it **above the labelled end of the
  anonymous namespace** — anchoring on the FACTORIES banner puts it outside and the brace closes
  nothing. This mistake has been made twice in that file.

## Report

1. The diff for Part A and Part B separately.
2. `git diff --stat`.
3. Explicit confirmation that: no height-stack or overhang maths changed; the new column key still
   contains strate + layout + seed + `ParamsFingerprint`; `ClassifyTile`'s control flow and return
   values are untouched; exactly one bail counter fires per bail path.
4. Likely compile-error spots, specifically.
5. Anything in this spec that contradicts the code — **stop and say so rather than guessing.**