diff --git a/CODEX-TASK-001-tile-skip-stats.md b/CODEX-TASK-001-tile-skip-stats.md new file mode 100644 index 0000000..c007c45 --- /dev/null +++ b/CODEX-TASK-001-tile-skip-stats.md @@ -0,0 +1,106 @@ +# Codex task 001 — make tile-skipping observable in the running game + +**Owner:** Codex (Model Luna, xHigh) · **Orchestrator:** Claude · **Branch:** `experimental` +**Status:** specified, not started + +--- + +## Why this exists + +Jahni built the world, looked at it, and said: *"I don't know if it dropped any meshing? but it looks +alright by the eye."* + +He's right to be unsure — **there is no way to answer that question from inside the game.** The +plugin has **zero** stat counters (`grep INC_DWORD_STAT` → nothing). Tile-skipping is the largest +perf item in the whole plan and it is currently unobservable in production; it has only ever been +measured in an automation harness, on 40 sampled tiles. + +And a visual check cannot answer it: *skipped correctly* and *skipped nothing* render identically. +This codebase has paid repeatedly for exactly that confusion — see the "coverage is a number, not a +boolean" lessons in `OPSTACK-HANDOFF.md`. + +**The real prize:** with no strate opted in, cave-archetype skips must read **0**. After ticking +`bUseOperatorStack` on one `TunnelNetwork` strate and flying underground, they must become non-zero. +That is the **production-side proof of T1.d**, which does not exist today. + +## The site — do not go looking, it is one place + +`Source/VoxelForge/Private/VoxelWorld.cpp`, in **`AVoxelWorld::GenerateTileResult`** (~line 1501). +Trust the symbol, not the line number. + +```cpp +bool bTrivialEmpty = false; +if (!bSheetTile && !bWantCapture && Generator && Mesher && Mesher->IsoLevel == 0.0f) +{ + TRACE_CPUPROFILER_EVENT_SCOPE(VoxelForge_ClassifyTile); + bTrivialEmpty = (Generator->ClassifyTile(OriginVoxels, Step, Cells) != EVoxelTileClass::Mixed); +} + +FVoxelMeshData MeshData; +if (!bTrivialEmpty) +{ + TRACE_CPUPROFILER_EVENT_SCOPE(VoxelForge_GenerateMesh); + MeshData = bSheetTile ? Mesher->GenerateSheetMesh(...) : Mesher->GenerateMesh(...); +} +``` + +## What to build + +1. **A stat group.** New header `Source/VoxelForge/Public/VoxelStats.h`: + `DECLARE_STATS_GROUP(TEXT("VoxelForge"), STATGROUP_VoxelForge, STATCAT_Advanced);` plus + `DECLARE_DWORD_COUNTER_STAT_EXTERN` for each counter below. `DEFINE_STAT` for each goes in **one** + `.cpp` — put them in a new `Source/VoxelForge/Private/VoxelStats.cpp`. + +2. **Four per-frame counters** (`DWORD_COUNTER`, so `stat VoxelForge` shows a rate, not a total): + + | counter | incremented when | + |---|---| + | `TilesClassified` | the classifier gate was entered (the `if` above ran `ClassifyTile`) | + | `TilesSkippedAllSolid` | verdict was `AllSolid` | + | `TilesSkippedAllAir` | verdict was `AllAir` | + | `TilesMeshed` | `GenerateMesh` / `GenerateSheetMesh` actually ran | + + Splitting solid from air is the point, not decoration: **cave archetypes prove `AllSolid`**, so + that counter is the one that answers "did the op-stack work do anything in the real game". + +3. To get the verdict you need it as a value, not a bool. Changing + `bTrivialEmpty = (Classify(...) != Mixed)` into a stored `EVoxelTileClass Verdict = Classify(...)` + followed by `bTrivialEmpty = (Verdict != Mixed)` is **fine and expected**. + +## ⚠️ Invariants — a violation here is not a bug, it is a hole + +1. **DO NOT change `bTrivialEmpty`'s value or the control flow.** That bool decides whether a tile + gets geometry **and collision**. A wrong value is invisible until a player falls through the + floor. Refactor the expression, never the condition. +2. **DO NOT touch the gate `!bSheetTile && !bWantCapture && Generator && Mesher && Mesher->IsoLevel + == 0.0f`.** Every clause is load-bearing and documented in the comment above it — sheet tiles have + no marching cubes, capture tiles need the grid even when uniform, and the verdicts assume the MC + iso is exactly zero. +3. **Thread safety: this runs on WORKERS.** `GenerateTileResult` is called from the async ChunkGen + task *and* the synchronous carve path. Use the `INC_DWORD_STAT` family, which is per-thread-packet + safe. **A plain `static int32` counter, even `++` on an `int32`, is a data race — do not.** +4. **Zero cost when stats are compiled out.** The `INC_DWORD_STAT` macros already vanish when + `STATS == 0`. Do not wrap them in an `if` that survives, and do not compute anything solely to + feed a counter outside the macro. +5. **No new includes in a public header beyond `Stats/Stats.h`**; the plugin follows IWYU and the + include debt was cleared deliberately (`AUDIT §C9` work). + +## Acceptance + +- Editor, `stat VoxelForge` on screen, fly around: the numbers move. +- **`TilesClassified == TilesSkippedAllSolid + TilesSkippedAllAir + TilesMeshed`** for tiles that + entered the gate. (Tiles that fail the gate are meshed without being classified, so `TilesMeshed` + is legitimately larger than the classified total — say so in a comment rather than "fixing" it.) +- With **no strate opted in**: flying underground through a `TunnelNetwork` strate, + `TilesSkippedAllSolid` stays **0**. That is the baseline and it must be observed *before* the + next step, or the next step proves nothing. +- Tick `bUseOperatorStack` on **one** `TunnelNetwork` strate, fly the same route: + `TilesSkippedAllSolid` becomes **non-zero**. ← this is the deliverable. + +## Notes for the reviewer (Claude) + +- Check the verdict refactor byte-for-byte against the original condition. `!= Mixed` is the whole + contract. +- Check the counters are `DWORD_COUNTER` (per-frame) and not `DWORD_ACCUMULATOR`. +- Confirm no counter is incremented outside the gate in a way that double-counts the carve path, + which calls `GenerateTileResult` synchronously from the game thread. diff --git a/OPSTACK-HANDOFF.md b/OPSTACK-HANDOFF.md index 68a6c3b..becb549 100644 --- a/OPSTACK-HANDOFF.md +++ b/OPSTACK-HANDOFF.md @@ -1,10 +1,17 @@ -# Handoff — VoxelForge operator stack, 2026-07-29 (T1.d delivered and measured) +# Handoff — VoxelForge operator stack, 2026-07-29 (T1.d measured in the harness, unproven in the game) > Paste the block below into a fresh session. Everything it refers to is on disk and in git. > -> **State:** 8 of 8 archetypes ported and green. **Tile-skipping now actually works and is measured: -> 11 of 40 tiles proved `AllSolid` at production defaults, 14641 voxels brute-forced, 0 violations.** -> `AUDIT §C2` is fixed. One commit is written but **not yet built** — see "First action". +> **State:** 8 of 8 archetypes ported and green. **Tile-skipping works and is measured — in the +> automation harness:** 11 of 40 tiles proved `AllSolid` at production defaults, 14641 voxels +> brute-forced, 0 violations. `AUDIT §C2` is fixed. `experimental` is pushed (`origin/experimental`). +> +> **Two things are open, and the second is the more interesting one:** +> 1. commit `e002bd4` (VerticalShafts) is written and **not yet built**; +> 2. **none of this is observable in the running game.** Jahni built a world, looked at it, and said +> *"I don't know if it dropped any meshing? but it looks alright by the eye."* He is right to be +> unsure — the plugin has **zero stat counters**, and *skipped correctly* renders identically to +> *skipped nothing*. `CODEX-TASK-001-tile-skip-stats.md` is the spec that closes this. --- @@ -26,6 +33,24 @@ re-derive them. §C9's library half is the top open theoretical risk with 0 measured exposure. 6. **`CODEMAP.md`** — navigation. Trust symbol names over line numbers. +## How we work now — Codex writes, you orchestrate + +From 2026-07-29 this project runs **in tandem with Codex (Model Luna, xHigh)**. **Codex handles most +of the coding; you orchestrate.** Concretely: + +- You read the code and decide *what* to do; you write **precise specs** Codex executes; you **review + what comes back against the real code, not against its description**; you own the docs + (`OPSTACK-PROGRESS.md`, `CODEMAP §3`, this file) and the measurements. +- **Hand Codex the INVARIANT, not just the task.** This codebase's traps are invisible in a diff — + density sign, `Identity` meaning `Sdf ≥ T` (below), cache keys needing params + `LayoutVersion`, + inserting classes above the anonymous-namespace end marker. A spec that omits these gets code that + compiles and deletes collision. +- `CODEX-TASK-*.md` at the plugin root are the specs. Each carries a **Why**, the **exact site**, the + **invariants**, an **acceptance** section, and **notes for the reviewer**. Write the next one the + same way. +- Unchanged: **never build** (Jahni does), and a plausible patch is not a verified one until a + measurement says so. + ## Where things stand All 8 archetypes have an operator-stack twin, per-strate opt-in, each equivalence-tested **bit for @@ -37,7 +62,7 @@ Everything sits behind `UVoxelStrateDefinition::bUseOperatorStack`; the ported l that is my call and I still haven't made it. `GetDensityAt` and `ClassifyTile` build the stack through the **same** factory, `VF_BuildOpStackForChunk` — a second copy would be a hole, not a bug. -### ✅ T1.d — the tile-skipping prize — is real, and it is measured +### ✅ T1.d — the tile-skipping prize — is real and measured **in the harness** (not yet in the game) `FRoomGraphSource::EffectOverBox` answers **spatially**. The result, brute-forced voxel by voxel: @@ -70,10 +95,30 @@ collision**. The warning is written at the site you land on when you add one. `|A−B| ≥ K`, so the running minimum saturates at `K` below the smallest term. Without that observation the slack would scale with the ~88 tunnels in a cache and the criterion would be dead.) -## First action: build, then read ONE line +## First actions — one build to read, one task to hand Codex -**The last commit (`e002bd4`, VerticalShafts) is written and NOT built.** Everything before it is -built and green. +### (a) Hand Codex `CODEX-TASK-001-tile-skip-stats.md` — this is the priority + +Everything in this refactor has been proved in an automation harness on 40 sampled tiles, and +**nothing has ever been observed in the running game.** The task adds a `stat VoxelForge` group with +`TilesClassified / TilesSkippedAllSolid / TilesSkippedAllAir / TilesMeshed`. + +Its deliverable is a **before/after that constitutes the production proof of T1.d**: with no strate +opted in, `TilesSkippedAllSolid` underground must read **0**; after ticking `bUseOperatorStack` on +one `TunnelNetwork` strate and flying the same route, it must be **non-zero**. The spec carries the +invariants — most importantly that `bTrivialEmpty` decides whether a tile has **collision**, and that +`GenerateTileResult` runs on **worker threads** so a plain `static int32++` is a data race. + +Interim answer if Jahni wants it before that lands: **Unreal Insights already shows this.** The trace +scopes `VoxelForge_ClassifyTile` and `VoxelForge_GenerateMesh` exist at the site; a skipped tile is a +`ClassifyTile` with no `GenerateMesh` after it. ⚠️ But ~84 % of tiles were *already* being rejected by +the hand-written SurfaceWorld/bedrock paths long before this work, so surface skips will drown the +cave ones — you must be **underground in an opted-in `TunnelNetwork` strate** for the number to mean +anything. + +### (b) Build `e002bd4` (VerticalShafts) and read ONE line + +Everything before it is built and green. > Build, run the `VoxelForge` filter, and read > **`Box verdicts over 60 VerticalShafts tiles`**. diff --git a/OPSTACK-PROGRESS.md b/OPSTACK-PROGRESS.md index 6d2a3dd..7fc8699 100644 --- a/OPSTACK-PROGRESS.md +++ b/OPSTACK-PROGRESS.md @@ -3266,3 +3266,67 @@ existing assertion fails rather than a player falling through the floor. stay 0. If it is still 0 proved, the warning now says what to check *first*: `ExtraReach` inflates both remaining tests, so its value against `ShaftMaxRadius` is the thing to look at before touching either test — not a re-derivation from scratch. + +## 2026-07-29 — clean stopping point. Codex joins; the production-observability gap is named. + +### The finding that ends this session + +Jahni built a world, looked at it, and said: *"I don't know if it dropped any meshing? but it looks +alright by the eye."* + +That sentence is the honest state of this entire refactor. **Everything proved so far was proved in +an automation harness, on 40 sampled tiles.** In the running game, tile-skipping is **unobservable**: + +- `grep INC_DWORD_STAT Source/` → **nothing**. The plugin has zero stat counters. +- *Skipped correctly* and *skipped nothing* render identically, so no visual check can separate them. +- And `bUseOperatorStack` defaults **false** with no strate ticked, so unless Jahni ticked one, the + honest answer is "nothing new was dropped, by construction". + +This is the same failure this session spent six rounds learning to avoid — *coverage is a number, not +a boolean* — applied to production instead of to a test. The tests got that discipline; the game +never did. + +**Interim answer that needs no code:** the trace scopes `VoxelForge_ClassifyTile` and +`VoxelForge_GenerateMesh` already exist at the site (`AVoxelWorld::GenerateTileResult`, +VoxelWorld.cpp ~1501). In Unreal Insights, a skipped tile is a `ClassifyTile` with no `GenerateMesh` +after it. ⚠️ But the comment there records that **~84 % of tiles were already being rejected** by the +hand-written SurfaceWorld/bedrock paths long before the op stack existed, so surface skips drown the +cave ones — the number only means something **underground in an opted-in `TunnelNetwork` strate**. + +### `CODEX-TASK-001-tile-skip-stats.md` + +Written and committed. A `stat VoxelForge` group with `TilesClassified / TilesSkippedAllSolid / +TilesSkippedAllAir / TilesMeshed`. Solid and air are split deliberately: **cave archetypes prove +`AllSolid`**, so that counter is the one that answers whether the op-stack work did anything real. + +Its deliverable is a before/after that is **the production proof of T1.d**, which does not exist +today: no strate opted in ⇒ `TilesSkippedAllSolid` underground reads 0; tick `bUseOperatorStack` on +one `TunnelNetwork` strate, fly the same route ⇒ non-zero. + +The spec carries the invariants rather than just the task, because that is the whole point of a spec +here: `bTrivialEmpty` decides whether a tile has **collision**, the five-clause gate is load-bearing, +and `GenerateTileResult` runs on **worker threads** — so a plain `static int32++` is a data race and +`INC_DWORD_STAT` is not. + +### Working arrangement, from now on + +**Codex (Model Luna, xHigh) handles most of the coding; I orchestrate** — read the code, decide what +to do, write precise specs, review what comes back **against the code rather than its description**, +and own the docs and the measurements. Recorded in the handoff (`## How we work now`) and in memory. + +### Git + +`experimental` is now **pushed and tracked** (`origin/experimental`, Jahni's Gitea). `main` stays +pinned at the known-good commit. The old flat "never push" rule was corrected in both places that +stated it — with the caveat that matters: **a pushed commit is not a "verified green" marker**, since +this branch carries unbuilt work by design. `OPSTACK-PROGRESS.md` remains the only record of what was +actually built. + +### Still open, in priority order + +1. **`CODEX-TASK-001`** — the production observability gap. Priority, because every claim this + refactor makes is currently harness-only. +2. **Build `e002bd4`** (VerticalShafts connector capsules) and read + `Box verdicts over 60 VerticalShafts tiles`. 0 was the number for the project's whole life. +3. **PERF** — the op path is measurably slower; suspects listed in the handoff. Measure first. +4. The warp squeeze stays **parked** with its ceiling measured and a negative result recorded.