research: Astra low review of per-block bit-flag masks (read-only)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
2026-10-30 21:30:51 +01:00
2 changed files with 136 additions and 0 deletions
@@ -0,0 +1,53 @@
+# VoxelForge — read-only design review: per-block bit flags (masks) to skip work
+
+You are reviewing an idea against real code. **Do not edit, create or delete any file, and do not build or run
+anything.** Read code and reason. Be thorough; mark each claim as [CODE] (with file:line), [MEASURED] (from a report),
+or [INFERENCE].
+
+## Context
+
+VoxelForge Core (`Standalone/Core`) generates a density field: negative = solid, positive = air, 25 cm voxels, 32³ tiles,
+LOD0–8 clipmap, marching cubes. The production mesher for the standalone path is in `Standalone/Source/...`, and Core is
+also compiled into Unreal. Same seed must give the same world on every machine and worker count; byte-identity with
+older versions is NOT required.
+
+Branch `assembler/world` (HEAD `7022d02`'s parent line, currently `4041d4b`) has the smooth inter-strate tunnels.
+Readiness target: LOD0–2 near terrain in ~1 s on arrival. Measured today: SW ~0.7–0.8 s, VS ~0.9 s, FI/TN ~1.0 s
+headless. In Linux UE, start→LOD0–2 is ~2.2–2.5 s. At 30 m/s in UE, near coverage is ~40%.
+
+Reports to consult: `Standalone/Research/WorldAssemblerM3/Linux1*/REPORT.md`, especially:
+- L77/L87/L117, cost census;
+- L124, shape gate and per-strate timings;
+- L125/L126/L128, tunnel prepare: per-segment work, 67–73% pair overlap, support up to 7.9 m;
+- L128, where an exact one-owner slab scheme halved pairs but got slower because of per-sample ownership tests.
+
+Tile skipping already exists at tile level: box verdicts / classify, where all-solid or all-air tiles are skipped.
+
+## The idea (from the owner)
+
+Keep bit flags per block (1 bit per voxel, or per small brick of e.g. 4³ or 8³ voxels): for example "certainly deep
+solid", "certainly deep air", "near a surface", "touched by a tunnel", "already evaluated". Use them to skip work in
+generation and meshing, and possibly elsewhere (collision, streaming, edits, AI/nav, lighting).
+
+## Questions
+
+1. Where does time actually go per tile today, and which of that work a mask could skip? Cover density fill,
+   classification, mesher (marching cubes, normals/gradients, skirts, seam refinement), passage prepare, and composition.
+   Use measured census numbers where they exist.
+2. Which flags are sound? A flag must be a proof, never a guess. A mask "certainly solid" must be derived from a
+   conservative bound (the bound must be the supremum of what the evaluator produces), or the mesher will delete
+   surfaces. Which bounds already exist in the code (box verdicts, interval bounds) that a brick-level mask could reuse?
+   Name the existing hazards. For example, `Identity` means `Sdf >= T`, and `ClassifyTile` proving below-TerrainZ solid
+   would delete 3D caves.
+3. Granularity: per-voxel bits vs per-brick (4³/8³) vs per-column. What does each cost to build, versus what it
+   saves? Consider SIMD and cache behaviour. Marching cubes needs only cells whose corner signs differ, and gradients
+   need a halo.
+4. "Already evaluated" or "checked" flags: where would they be correct (memoisation of identical queries), and where
+   would they be wrong (nearest-of-many queries like the tunnel segments)? Is there repeated identical evaluation today,
+   for example halo samples shared between neighbouring tiles, or skirt/seam re-queries?
+5. Other uses: player edits (diff layer), collision-only server tiles, streaming priority, nav. Which would a mask
+   genuinely help, given the multiplayer rule (replicate seed + layout + diff, never geometry)?
+6. Recommendation: the one or two mask designs most likely to pay off, with the expected gain (a range, marked
+   inference), the measurement that would prove or kill each, and the risks. Keep it short.
+
+Write plain Markdown, at most ~1,200 words.
@@ -0,0 +1,83 @@
+[INFERENCE] **Improve the existing brick-proof system; don’t add a general “checked voxel” mask.** The promising savings are avoided field/composition work and passage preparation, rather than marching-cubes classification.
+
+References below abbreviate these files: [M = production mesher](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp), [S = ShapeWorldInternal.h](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h), [C = VoxelForgeCore.cpp](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/VoxelForgeCore.cpp), [O = VoxelDensityOpStack.cpp](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Source/VoxelForge/Private/VoxelDensityOpStack.cpp). L-numbers identify `Standalone/Research/WorldAssemblerM3/LinuxN/REPORT.md`.
+
+[INFERENCE] Evidence limitation: Linux128 is absent from this checkout. Its supplied 7.9 m support and slower one-owner result remain unverified here. The reports below span different revisions; their percentages must not be presented as today’s census.
+
+**1. Where time goes**
+
+[MEASURED] L77’s exclusive FI/TN LOD0–2 worker account:
+
+| Work | FI | TN |
+|---|---:|---:|
+| Total worker CPU/view | 23.02 s | 21.50 s |
+| Passage composition | 65.9% | 65.4% |
+| Support | 14.2% | 13.5% |
+| Base field | 3.2% | 3.3% |
+| Bands / dispatch | 2.3% / 3.7% | 2.6% / 3.6% |
+| MC / skirts / refinement, excluding field work | 1.6% / 0.1% / 7.5% | 1.2% / 0.2% / 8.6% |
+
+[MEASURED] Grid fill occupied 19.058/17.377 worker-seconds, **overlapping** those phase categories. L117’s later census found support zero; its smooth-tunnel regression concentrated about 95% of added tile CPU in passage-feature tiles. Cell classification stayed flat; gradients became cheaper. Thus L77 identifies historical leverage, not current percentages.
+
+[MEASURED] L125 preparation averaged 514/130/4.1 ms worker work at LOD0/1/2. LOD0 evaluated 3.11–8.49 million segment/sample pairs with 67–72% overlap, although the exact band occupied only 5–7% of grid samples. L126 reduced floor-profile calls 20–23% without useful readiness improvement.
+
+[CODE] Brick skipping already exists: configurable block verdicts, required-sample masks, and active-block gradient halos (M:646,1053). MC already rejects cells without sign crossings before interpolation/gradients (M:2186). Gradient memoisation exists (M:1956). Skirts issue separate density queries (M:2491); refinement has its own stage (C:2975).
+
+[INFERENCE] A pre-evaluation proof can save density, composition, and subsequent MC work. A sign mask constructed after density evaluation saves only downstream scanning. Classification itself costs work; no isolated current classification percentage is established by these reports.
+
+[MEASURED] L124’s shape gate alone took 179–226 ms, with no near-tile admissions before completion. L87 additionally measured a 107–153 ms far-parent gate and no headless collision wait. Tile masks cannot directly remove these gates or explain the supplied UE/headless gap.
+
+**2. Sound flags and hazards**
+
+[INFERENCE] For final density \(D\), solid requires an upper bound covering the evaluator’s supremum, \(U<0\); air requires a valid lower bound \(L\ge0\). “Deep” requires an explicit margin. Unknown must stay unknown. A crossing proves an MC-active cell; uniform sampled signs do not prove an entire continuous volume surface-free.
+
+[CODE] Reusable proofs include:
+
+- S:5428: halo-expanded box classification.
+- S:5974: exact surface-column extrema over the MC lattice.
+- S:6019: floating-island density intervals.
+- S:5197,6032: native cave intervals.
+- S:5949: stamp interval composition with outward numerical padding.
+- O:6998: operation-specific inactivity from SDF lower bounds.
+
+[CODE] These proofs have guards: passage overlap returns unknown (S:5687); unsupported stamp composition returns unknown (S:5655); stratum crossings are rejected (S:5900). The mesher distinguishes uniform candidates from `bSkipSafe` certificates (M:715,756), and otherwise validates candidates against actual samples (M:1838).
+
+[CODE] For gated modifiers, `Identity` can mean `Sdf >= T`, not `abs(Sdf) >= T` (O:1496,6998). Noise bounds cannot assume nominal `[-1,1]`: FBM and unclamped cellular bounds are explicitly discussed at O:848,301.
+
+[INFERENCE] Additional hazards: confusing internal SDF with final density; certifying base terrain before caves, tunnels, shells, stamps or edits; using lattice proofs for off-grid queries; ignoring rounding or gradient halos. A below-`TerrainZ` solid rule applied to 3D caves deletes caves. “Touched by tunnel” should mean conservative candidate overlap; only **proven untouched** authorizes omission.
+
+**3. Granularity**
+
+[INFERENCE] Storage per flag for 32³ cells:
+
+| Granularity | Storage | Construction/tradeoff |
+|---|---:|---|
+| Per cell | 4 KiB | 32,768 decisions; expensive unless derived during necessary work |
+| 4³ bricks | 64 B | 512 proofs; tighter boundaries, more proof/halo overhead |
+| 8³ bricks | 8 B | 64 proofs; cheaper, more mixed bricks |
+| XY columns | 128 B | 1,024 decisions; useful mainly for heightfield-specific proofs |
+
+[INFERENCE] These exclude corner/halo samples, metadata and additional flags. A column bit cannot represent multiple cave intervals. Brick runs preserve contiguous X sampling and SIMD better than scattered per-sample branches; bit packing alone does not establish a speedup.
+
+[CODE] The existing implementation uses byte arrays and preserves exact samples around active bricks (M:1053–1083). Replacing densities with arbitrary same-sign constants near active cells would corrupt interpolated positions or normals.
+
+**4. “Already evaluated”**
+
+[CODE] Passage preparation precedes meshing (C:2747). Its valid sentinel is initialization, not completion: subsequent segments update floor selection, shell minima and round-distance winners, with deterministic tie-breaking (S:11520,11615,11688).
+
+[INFERENCE] Therefore first-visitor wins is invalid. Memoisation is valid only for identical queries against identical seed/layout/evaluator/edit revision and query semantics; cache the value, not just a bit.
+
+[MEASURED] L77 counted 8.6–8.9% halo points, without measuring duplicate coordinates. L125 verified 4,900 shared samples across four neighbour pairs and still recorded substantial off-grid skirt/midpoint fallbacks.
+
+[CODE] Shared-grid reuse and gradient validity caches already exist (M:1031,1956). [INFERENCE] Skirt/seam queries are reuse candidates only when coordinates and semantics actually match; “same voxel” is insufficient.
+
+**5. Other uses**
+
+[INFERENCE] Edit-dirty bricks can localize regeneration, with invalidation expanded through normals, seams and collision dependencies. Collision-only servers can skip certified empty regions, but still need boundary geometry. Streaming can prioritize potentially occupied bricks; nav and lighting need clearance/connectivity or visibility data beyond occupancy. Keep masks derived locally from replicated seed + layout + diff, versioned with that state.
+
+**6. Recommendation**
+
+- [INFERENCE] **Tune existing 4³/8³ certificates**, caching proofs and emitting contiguous required-sample runs. Plausible incremental near-ready saving: **0–5%**, with regressions possible. Measure proof time, samples avoided, halo expansion, total worker CPU and UE readiness.
+- [INFERENCE] **Pass required-region information into passage preparation**, rejecting whole segment/brick regions conservatively. Plausible near-ready saving: **0–10%**, highly uncertain because passage overlap currently vetoes proofs. Kill it if pair reduction fails to reduce preparation and end-to-end time.
+
+[INFERENCE] Both require seam/off-grid validation, adversarial bound checks, edit invalidation tests, and cross-machine/worker determinism. Preserve canonical tie-breaking; old-version byte identity need not constrain a sound redesign.