Commit Graph
64 Commits
Author SHA1 Message Date
Fr0zkaandClaude Opus 5 4f8b4a9a27 fix(gen): the phantom terracing is the PASSAGE support writer -- plus a region verdict cache
The owner played the build and reported four things. Three are answered with evidence,
one is disproven, and my own leading hypothesis was wrong.

⭐ 1. THE PHANTOM TERRACING IS NOT MARCHING-CUBES ALIASING
He reported stairs surviving with bTunnelFloorTerracingEnabled = false. I suspected MC
lattice aliasing on a near-planar floor; I was wrong. An A/B over the same mesher,
counted in pixels:

  graph-tunnel floor removed     2,813 / 147,456 px changed   1.91%
  passage support removed       25,695 / 147,456 px changed  17.43%

⇒ The bands are the INTER-STRATE PASSAGE SUPPORT WRITER, which is independent of the
tunnel terracing flag. That is exactly why turning terracing off did nothing: he was
disabling the wrong system. FCaveTerraceMod is inactive (TerraceStepHeight=0) and was
never involved.

⚠️ It cannot simply be switched off. With it disabled:
  fit 20,804 -> 20,511, walk-reachable 10,884 -> 7,525, components 260 -> 272.
Route still connected, but a third of reachable space gone. It stays as a correctness
backstop. The native fix is the same pattern we just applied to tunnels: author the
inter-strate passage floor profile during passage CONSTRUCTION. Codex judged that
refactor too large to risk this round and said so rather than attempting it.

MC aliasing remains theoretically possible at shallow angles (Lorensen & Cline 1987),
but the same-mesher A/B proves our writer dominates. Dual Contouring (Ju et al.) was
considered and rejected as a disproportionate mesher/topology change.

2. THE ROUGHNESS COST DEFECT DOES NOT EXIST -- my hypothesis, cleanly disproven
  FBM 2.91s / 66.92% skips · Ridged 2.85s / 66.53% · Mixed 2.92s / 66.82%
  Cellular 2.99s / 66.43% · FBM frequency 0.025-0.5: 2.88-2.99s, 66.80-67.04%
I predicted a shared bound across noise types collapsing the classifier. The code
already carries separate conservative bounds per type. No roughness code was changed.
⇒ Whatever the owner saw is not steady-state export cost. Most likely the in-editor
regeneration a parameter change triggers, which is the full-rebuild cost, not the
per-sample cost. Worth confirming with him.

3. ROOM MOUTH RISE -- his idea, implemented, off by default
  RoomMouthRiseBlendVoxels, default 8.0
  blend 8   fit 20,804  reach 10,884  40,429 v / 75,416 t   capability-NEUTRAL
  blend 24  fit 20,792  reach 10,874  40,337 v / 75,232 t   REJECTED, loses capability
Relief preserved at blend 8 (dihedral p50 2.5584, p90 33.7191). The chosen camera did
not see a mouth bench, so no visual claim is made.

4. ⭐ VERDICT CACHE WITH REGION REUSE -- answers "why re-prove what we proved?"
  130 classification calls -> 1 store, 129 hits, of which 128 ENCLOSING-REGION hits.
The region level is the one that matters: deep rock is empty thousands of chunks at a
time, and one proof over a large box now serves all of them.

Soundness, which is the whole risk (a stale "empty" silently deletes geometry):
512-entry bounded LRU; stores ONLY AllAir/AllSolid, never Mixed; key includes seed,
layout/lifetime id, params fingerprint, diff version, density/seal settings, support
mode, lattice step, origin and extent; dynamic season/composer/terrain-op/biome
configs FAIL CLOSED and bypass reuse. Deliberately session-resident, not on disk --
cross-process persistence cannot serialise all asset/layout/edit state safely yet.

VERIFIED BY ME: build 0 errors; export 1.353 / 1.430 / 1.432 s (2260c4d was
1.320 / 1.424 / 1.439); geometry SHA-256 b3e5f4c3..9b377 IDENTICAL to 2260c4d.
Peak RSS ~1.75 GB. Determinism repeat equal.

⚠️ My composer hypothesis was also wrong: I suspected
`bTunnelFloorTerracingEnabled = CategoryRng.FRand() < 0.75f` in the composer was
overriding his setting. It is not the cause here. The design question it raises is
still open and worth the owner's view: should a composer roll be allowed to overwrite
an explicitly authored value? Otherwise "I turned it off" never reliably means anything.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
2026-09-11 13:13:28 +02:00
Fr0zkaandClaude Opus 5 8e42d6d807 feat(tunnels): floor authored at BUILD time, room clips tunnel, climb tier added
The round the owner asked for, with three corrections from Codex that are worth more
than the features.

WHAT LANDED
- The complete tunnel floor profile is authored into FCachedTunnel during
  BuildChunkCache, not derived per sample. This was his point: "it should be done at
  creation time, since it knows its destination."
- Room floor ownership blends/clips the tunnel floor at mouths -- his A > B boolean.
- The traversal validator gained a climb tier, structured exactly as agreed:
    bWalkOnlyCanReachDeparture / bClimbCanReachDeparture kept SEPARATE, with
    WalkOnly / Climb / MineRequired cell counts reported alongside.
  The comment in the header states the contract: "The primordial capability contract
  is walk-only... climb is reported as a separate tier and never makes the hard
  connectivity verdict pass." That is the distinction we wanted: reporting tiers, not
  permission.
- Floor controls exposed under Cave Morphology|Tunnels|Floor.

⚠️ CORRECTION 1 -- LARGE LEDGES ARE NOT CAPABILITY-SAFE YET, so they stay opt-in.
    default (preference 0)   fit 20,804   walk reach 10,884   0 ledges
    preference 2             fit 20,765   walk reach 10,712   360 ledges
  Ledges cost 172 walk-reachable cells. Codex kept preference 0 (the old staircase) as
  the default rather than shipping the owner's preferred look at the price of walkable
  space, and led its report with that. Correct call.

⚠️ CORRECTION 2 -- THE BACKSTOP STILL CANNOT GO. Support/room fallback counters were
  zero, but the structural protection clamp fired on 31,017 of 31,017 authored floor
  samples -- 100%. So the shape is still not correct by construction; the clamp is
  doing the work. Instrumenting it was the right move: we now have a number instead of
  an argument.

⚠️ CORRECTION 3 -- THE "X + 2A" CLEARANCE BUDGET IS NOT IMPLEMENTABLE AS I SPECIFIED.
  Interval propagation bounds FIELD VALUES, not SPATIAL DISPLACEMENT. Deriving A needs
  a proven Lipschitz/influence bound for every post-operation, which we do not have.
  Both the owner and I assumed the interval work already gave us this; it does not.
  The idea is still right, the prerequisite is a displacement bound.

MEASURED BY ME: build 0 errors; export 1.365 / 1.357 / 1.420 s (checkpoint was
1.321 / 1.357), flat within noise; 153,346 triangles.

ALSO MEASURED THIS ROUND
- Edit cost: 8 affected chunks, 195.5 ms total -- generation 195.471 ms, apply 0.059,
  diff 0.020, queue 0.001. It is full level-0 regeneration; a dirty-mesh path needs
  partial section support that is unavailable without modifying RMC.
- Streaming caps: effective generation cap 14, pending max 14, CAP HIT. Mesh applies
  per frame 4 (not hit), 2 ms apply budget (not hit), unloads 6 (not hit).
  ⇒ The generation core cap is what binds during startup fill.
- Peak export RSS 1.70 GB (was 19.5 GB before the memory work).
- LOD4 empty outliers gone. LOD0 not solved: 4,600 empty tiles at 3 us median, but 11
  mixed-classifier fallbacks over 100 ms, max 554 ms -- verdict=Mixed, then the full
  mesher, producing no triangles.

⚠️ UNVERIFIED AND CONCERNING: the reported in-game meshed times are far worse than
  412a7ad's -- p50 498.8 ms vs 115 ms, p95 1.227 s vs 410 ms, max 1.974 s vs 844 ms.
  The samples differ (9,336 ready events vs 841 requests, different traversal, possibly
  queue wait included), and the export path is flat, so I do not know whether this is a
  real regression or a different measurement. I have NOT reproduced it myself. It is
  the first thing the next round must settle.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
2026-09-11 04:25:45 +02:00
Fr0zkaandClaude Opus 5 e91f1c44bb wip(tunnels): checkpoint -- floor parameters exposed, round cut short by Codex credit limit
⚠️ THIS IS A CHECKPOINT, NOT A FINISHED CHANGE. The agent hit its usage limit
mid-round (locked out until 2026-09-15) after compacting its context, and never
filed a report or capability numbers. Committing so 1,344 lines are not left
uncommitted for five days. Verified by me only to the extent stated below.

WHAT LANDED
The tunnel floor controls are now real strate parameters under
"Cave Morphology|Tunnels|Floor", which is what the owner asked for -- hand-made
assets can tune them and the composer can roll them later:

  bTunnelFloorEnabled                true
  bTunnelFloorTerracingEnabled       true
  TunnelFloorTerraceStepHeight       1.71 voxels   (the existing 42.75 cm riser)
  TunnelFloorMaxLedgeHeight         12.0  voxels   (3 m -- climbable, not walkable)
  TunnelFloorGentleSlopeThreshold    tan(44 deg)
  TunnelFloorLedgeCountPreference    0             (0 = old per-segment staircase;
                                                    >0 = whole-chain ledge policy)
  TunnelFloorMaxLedges               4096

⭐ So the staircase-vs-ledges question the owner raised is now answerable IN THE
EDITOR without a rebuild: set LedgeCountPreference to 2-3 and the whole-chain
policy takes over from the per-segment one.

VERIFIED BY ME
  build            0 errors
  export, 2 runs   1.321 / 1.357 s   (ce53552 was 1.384 / 1.405 / 1.415)  ~-4%
  triangles        153,382           (was 153,338, +44)
  geometry hash    9815c9a2..ebeb9   -- CHANGED, so behaviour is live beyond the
                                       stated compatibility defaults

⚠️ CAPABILITY DIPPED SLIGHTLY, and I am not calling this an improvement:
  player-fit cells   20,835 -> 20,800   (-35,  -0.17%)
  reachable cells    10,901 -> 10,886   (-15,  -0.14%)
  largest component  10,886 = reachable, still one connected mass

Small enough to be a consequence of the 44-triangle geometry shift rather than a
real regression, but it is a drop, and my own rule for this work is that fit and
reachable must not fall. Treat it as unresolved until the round can be finished.

NOT DONE (the substance of the brief):
- terracing objective still minimises riser height, not step count
- the room does not yet clip the tunnel floor where they overlap (A > B)
- the traversal validator still cannot climb, so it cannot score real ledges
- the floor profile is still derived per sample, not authored at build time
- the support-column backstop is still present, i.e. the shape is still not
  correct by construction
- dig cost and the streaming-cap audit untouched

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
2026-09-10 22:01:32 +02:00
Fr0zkaandClaude Opus 5 412a7ad219 perf(game): the path he actually walks -- meshed p95 4.91s -> 0.41s, RSS 19.5GB -> 1.7GB
The owner: "I want real performance gain for the ingame path... not in deletion, but
in optimization and tricks." Three rounds of ours had improved the commandlet export
he never runs. This one is the streaming path.

MEMORY -- it was retention, not the new lattice
19.5 GB was worker FChunkSDFCache arrays keeping capacity when keys were replaced
(946 MB across 15 workers in diagnostics), shared graph caches with no byte or
lifetime budget, and batch commandlets holding transient UObject worlds until exit.
BlockSkip is bounded to 8^3 bytes per tile; interval scratch is worker-local and
capped. Fixes: release storage on key replacement, LRU with a 256 MiB resident
budget, limit in-flight graph builds, drop oversized entries, CollectGarbage between
batch cases.

  peak RSS   19.5 GB  ->  1.643 GB (their runs) / 1.71 GB (mine, 3x128^3)   -91%
  a 3-case stress batch showed no accumulation

WHY SOME EMPTY TILES COST 5.6s AND OTHERS 3us -- a 1.9 MILLION x spread
Fast ones returned straight from the layout/edge proof (verdict AllAir/AllSolid,
refine=0, mesh=0). Slow ones returned Mixed, so the caller ran the FULL mesher grid
and only discovered emptiness by brute force -- logs show empty=1 after ~5 s of mesh
work. The interval machinery now reaches a verdict before a mesh job is enqueued.

IN GAME, 841 requests at streaming radius:

                     count     p50        p95        max
  empty                394   6.087 ms   12.113 ms   25.049 ms
  meshed               447 114.748 ms  410.314 ms  844.043 ms
  LOD0 empty           221   7.126 ms   11.901 ms   25.049 ms
  LOD0 meshed          122 355.794 ms  600.945 ms  844.043 ms

  against d38d4ca:  meshed p50/p95/max 0.6085 / 4.9075 / 5.0199 s
                    LOD4 empty max 5.618 s
  => meshed p50 5.3x, p95 12.0x, and the multi-second empty outliers are gone.

REQUEST-TO-READY -- the number that answers "I can outrun the generation and fall"
  all tiles     p50  57.576 ms   p95 429.052 ms   max 905.633 ms
  empty         p50  30.943 ms   p95  98.302 ms   max 305.505 ms
  meshed        p50 128.521 ms   p95 462.235 ms   max 905.633 ms

At the harness walking speed of 6 m/s: p95 = 2.574 m, worst case = 5.434 m. A walking
player now outruns tile readiness by a couple of metres, not by a strate.

CAPABILITY PRESERVED, which is the point
The owner asked that speed never come from quietly simplifying the world. It did not:
geometry SHA-256 98eaad61..d3c4f is IDENTICAL to d38d4ca, 153,740 triangles, 82,516
vertices, across three runs. No resolution, sample-count, octave or seed reduction.
Unknown intervals still cost the skip. Prove and skip, never simplify and hope.

Export path, my own build, 3 runs: 1.088 / 1.102 / 1.100 s (was 1.171-1.258 s). ~9%,
below the 17% noise floor, so not claimed as a speed win -- the export was not the
target.

Honest remainder: LOD0 empties are ~7 ms p50 with a 25 ms tail, not the 2-5 ms of a
pure layout proof. Those are bounded, sound interval refinements that never invoke
meshing; going lower needs stronger proof bounds, not more skipping.

Verified by me: build 0 errors, binary timestamp checked against sources, peak RSS
measured, hashes compared. Nothing outside Plugins/VoxelForge; RMC untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
2026-09-10 11:05:09 +02:00
Fr0zkaandClaude Opus 5 0e9df9e5a7 perf(gen): the game path is NOTHING like the export path -- and empty tiles cost 5.6s
Overnight task 3. The headline is not a speedup: it is that we have been measuring
the wrong thing all along.

⭐ IN-GAME MEASUREMENT, FINALLY (headless -game -nullrhi, TunnelNetwork PIE map)

  46,078 profiled tile calls
  46,025 EMPTY  (99.885%)
      53 actually meshed
  meshed    p50 0.6085 s   p95 4.9075 s   max 5.0199 s   mean 1.3515 s
  LOD1-3 empty medians ~2-3 us
  LOD4 empty OUTLIERS up to 5.618 s

The pawn fell through successive strata during the run, reproducing the owner's
"I can outrun the generation and fall" exactly.

Two facts to sit with:
1. **Almost nothing is meshed. 99.9% of tile requests produce nothing** -- and some
   of those empty tiles take FIVE AND A HALF SECONDS. A tile that generates no
   geometry costing seconds is the bug, and it is classification cost, not meshing.
2. The export path we have optimised for three rounds does 64 tiles in ~1.5 s
   total. The game does 46,078 calls with multi-second tile latency. **They share
   the density function and nothing else about their cost profile.**

⇒ Every optimisation so far was aimed at the path the owner does not play.

WHY NO BLOCK CAN BE SKIPPED -- answered, and it is structural
Per-op bound-vs-brute-force table produced (op_bounds.json). Most ops bound fine
once given proven suprema, and those were tightened:
  FBM envelope 1.5, cellular 6.0, SdfConvertOp exact 2*|BaseDensity| = 16,
  Lerp radius envelope max(Min,Max), plus state-aware room/column/floor-bias/worm/
  terrace/cliff/arch/dome/pinch/roughness bounds. Structural ops now return exact
  zero outside their geometric bands.

But OriginSpineOp, PassageCarveOp and the active seals **cannot** take a finite
generic density bound: they Lerp/Min/Max against the CURRENT density, so with
arbitrary incoming density their supremum is mathematically unbounded. This is not
a missing EffectOverBox, it is the operator semantics.
⇒ Hierarchical evaluation requires propagating a density INTERVAL through the
stack, or changing those operators. That is the real prerequisite, now named.

Early-out with tightened bounds still skips 0 of 4,096 blocks and costs 96.6%
(3.305 -> 6.497 s). Left opt-in and DISABLED. ClassifyTile's TerrainZ/overhang
guard untouched.

SPATIAL INDEX REMOVED (ee89c4d's, which never earned its keep)
  wall clock   3.1407 s on -> 3.1894 s off   = 1.5%, below the 17% floor
  cache bytes  557,626,564 -> 246,049,920    = -297 MiB
  candidates   19.5M -> 81.6M scanned (4.17x), evaluated +25.6%
Memory the owner can feel beats counters he cannot. Per-primitive broadphase stays.

VERIFIED END-TO-END on a correctly built binary: 1.515 / 1.554 / 1.569 s, geometry
hash 98eaad61..d3c4f unchanged from 2f33870. Task 3 is memory, not speed.

⚠️ PROCESS, second time: the agent's reported DLL timestamp was from a staging copy
it built under the plugin's own Saved/, not Plugins/VoxelForge/Binaries. My first
three runs measured the 05:58 binary again. Verify the binary you are about to
measure, not the one someone says they built.

Determinism: all export hashes ED0F1F31 within the run; classifier vs generator
verdicts zero mismatches. Nothing outside Plugins/VoxelForge; RMC untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
2026-09-09 09:00:41 +02:00
Fr0zkaandClaude Opus 5 2f33870a84 perf(mesher): adjacent tiles were re-evaluating 18% of the field -- 2.69s to 1.49s
Overnight task 2. Every claim here is end-to-end wall clock on a real 128^3 export,
not a profiled bucket, because the last two rounds shipped large bucket wins worth
0% in reality. Measurement floor is ~17% run-to-run; only Part 2 cleared it.

PART 2 -- KEPT. Density-grid reuse.
The mesher samples an expanded grid (-1..Cells+1), so adjacent tiles re-evaluated
their shared boundary planes. Measured: 2,744,000 requested samples contained
495,909 duplicates -- 18.07% of the field generated twice. A run-scoped read-only
lattice (FVoxelSharedDensityGrid) owns each world sample once and hands it to both
adjacent tile grids. Duplicates now 0.

  128^3 TunnelNetwork seed 1337 slot 1, freshly built binary:
    before  2.603 / 2.693 / 2.697 s
    after   1.488 / 1.499 / 1.486 s      -44.6%, 1.80x

  Geometry SHA-256 98eaad61..d3c4f IDENTICAL to the pre-change binary's output.
  153,740 triangles either way. Pure win, no field change.

The lattice is deliberately not a UObject and not a cache with eviction: ownership,
lifetime and iteration order stay explicit. A malformed or incomplete lattice is
ignored and falls back to the canonical generator path rather than altering geometry.

PART 1 -- REVERTED. Block-level empty-space early-out.
**0 of 4,096 blocks were safely classifiable**, and the attempt made it 85% slower
(3.632 -> 6.735 s). No unsafe bound was retained.
⇒ This is a finding, not just a failure: our conservative bounds are too loose to
prove ANY 8^3 block uniform. Hierarchical evaluation of any kind is blocked until
we know which operator has effectively unbounded reach. Next task.

PART 3 -- REVERTED. Task granularity.
15 workers, 64 tile jobs, mean job 0.594 s, launch fringe ~14.1 us. MinBatchSize=4
was 1.09% SLOWER end-to-end despite a better bucket time -- exactly the trap this
task's rule exists to catch. Default stays 1. ParallelForWithExistingTaskContext
not adopted.

PART 4 -- NO CHANGE. Renderer audited over 3 runs (8.736 s total, 1.483 s raster).
Mesh and BVH are already reused across viewpoints; no per-pixel heap allocation, no
redundant remeshing. Nothing gross to fix; it benefits from the above automatically.

STILL OPEN: the spatial index from ee89c4d has never earned its keep in wall clock
and accounts for ~214 MB of measured allocations. Codex proposes an on/off ablation
then removal if confirmed. Not done unilaterally.

⚠️ PROCESS NOTE: the agent reported "build passed" but the runtime DLL was still
from 03:07 while sources were edited at 05:22/05:42. My first three verification
runs measured the OLD binary and showed no change. Always check binary timestamps
against source timestamps before believing a measurement.

Cumulative on this region: 342.2 s -> 16.2 s -> 2.62 s -> 1.49 s (230x).
Determinism: identical hashes across all repeat runs. Build clean. Nothing outside
Plugins/VoxelForge touched; RealtimeMeshComponent untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
2026-09-09 05:59:56 +02:00
Fr0zkaandClaude Opus 5 e63f016835 tools(profile): the instrument lied by 11.5x -- now it checks itself, and batching kills startup cost
Overnight task 1 of the owner's mandate: "better diagnostic, profiler and tools,
so we can batch test way more quicker."

WHY THIS CAME BEFORE ANY MORE OPTIMISATION
ee89c4d delivered a 5x candidate reduction and a 16x cheaper support floor for
0% end-to-end gain, because the profiler inflated by ~11.5x (14.74 us/sample
profiled vs 1.28 us/voxel real). Two rounds of bucket rankings were artefacts of
FPlatformTime::Cycles64() wrapped around ~100 ns operations -- the timer pair
calibrates at ~0.1 us, so it was measuring itself.

PROFILER
Deterministic count sampling plus amortised block timing every 64 density calls.
Coarse counters (calls, candidates) stay always-on -- they were never the problem.
Timing-only scopes are opt-in behind -profiledensityfull. UE CPU trace scopes added.

  before   profiled 14.74 us/sample  vs  1.28 us/voxel real   ~11.5x
  after    profiled 0.463050 s       vs  0.558470 s unprofiled   -17.09%
           geometry 235AC2DC in both

⭐ Every profiled report now prints profiled total, unprofiled total, discrepancy,
ratio and geometry equality, and EXITS CODE 2 if the comparison fails. An
intermediate run hit -22.6% and failed loudly, which is the point: this class of
bug must be impossible to miss rather than something noticed three rounds later.

⚠️ The -17% residual is run-to-run noise, so ~17% is our measurement floor.
Improvements smaller than that cannot be claimed from a single pair of runs.

BATCH EXPLORER
One process, many cases, one JSON. Startup was ~25-30 s of engine boot for ~3 s of
work, which is what made every iteration expensive.

  3 cases in 6.59 s of case time
  standalone vs batched, same binary: hash 235AC2DC, 2,461 tris, 343,000 tunnel
  lookups, 554,381 passage evaluations -- identical on both

-out= is enforced absolute under the plugin's own Saved/.

CACHE BREAKDOWN (the 272 KB/entry question)
The slot table is small. The weight is spatial-index bins and candidate arrays,
room arrays, support-column arrays and tunnel-world indices -- i.e. much of the
411 MiB is the spatial index ee89c4d added, which also bought 0% end-to-end.
Worth revisiting on those terms.

RESEARCH (recorded, not acted on) -- CODEX-TASK-1-PROFILER-BATCH-RESEARCH.md
Ranked follow-ups: deterministic empty-space early-out, density-grid reuse, task
granularity (ParallelForWithExistingTaskContext + MinBatchSize), SIMD MC (~2x over
optimised scalar, but cross-machine determinism is the risk), sparse hierarchy,
RMC partial updates, Transvoxel LOD seams. Sources cited in the doc.
⚠️ Upstream RMC is 5.3.2; the owner's copy identifies as 5.4 with custom
modifications. No upgrade is safe. Left untouched.

PUSHBACK, accepted: "buckets must sum to their parent" needs a parallel-time
definition -- summing worker CPU is not wall time. Reports now separate worker CPU
sum from wall-clock estimate.

End-to-end unchanged, as expected for a tooling change: 2.746 / 2.691 s on the
128^3 seed 1337 region (was 2.678-2.705). Build clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
2026-09-09 05:02:21 +02:00
Fr0zkaandClaude Opus 5 ee89c4d8c2 perf(gen): 5x fewer candidates, 16x cheaper floor -- and 0% end-to-end. Read the last section.
Owner, after playing 0576e80: "slightly faster, still not on par... I can outrun the
generation and fall." Also, unprompted: "the tunnels seem forced, with a forced
flat -- it technically looks good, but could be a reason." He was right that it
was a reason; it was not THE reason.

WHAT LANDED, all real and all measured

Conservative spatial index (immutable per-chunk uniform bins, baked conservative
bounds, squared rejects, deterministic ordering):

  candidates scanned/sample   25.140 -> 5.192
  candidates evaluated/sample 13.879 -> 2.992
    cave tunnels  11.228/6.069 -> 1.525/0.723
    tunnel core   11.228/5.831 -> 1.216/0.567

The earlier flat AABB attempt failed because loose bounds culled almost nothing
while adding a pass. Technique from GPU Gems 3 ch.32 broad-phase.

The forced-flat floor, which the owner spotted by eye: a floor decision is
inherently per-COLUMN and was being recomputed for every Z sample in that column.
Now resolved once per integer XY column and reused.

  TunnelCorePosts   2.29 us/sample -> 0.138 us/sample   (16x)

The floor is unchanged in behaviour: still applied only after a tunnel passes its
world bounds and walkable projection, still walkable, look untouched.

FIRST MEASUREMENT OF OUR MEMORY FOOTPRINT
The owner: "it shouldn't be pressured on such system, even less when not running
anything." He is right, and it is ours:

  total resident worker cache   431,162,116 B  =  411 MiB
  15 workers                    27.41 MiB each
  average valid entry           278,704 B
  largest entry                 541,344 B
  static slot storage             2.4 MiB   (dynamic: 428.6 MB)

Cutting 4096 -> 128 slots only saved static storage. The weight is 272 KB PER
CACHED CHUNK STATE, ~102 live per worker. Nobody has yet asked why one cached
chunk state needs 272 KB. That is the open question, not the slot count.

⚠️ AND THE PART THAT MATTERS MOST: THIS BOUGHT NOTHING END-TO-END

  128^3 TunnelNetwork, seed 1337 slot 1, same region, unprofiled:
    before this commit   2.623 s
    after, 3 runs        2.705 / 2.678 / 2.682 s

Flat. A 5x candidate reduction and a 16x cheaper floor moved real generation time
by zero.

The reason is visible in the numbers and I should have caught it earlier: the
PROFILED path reports 14.74 us/sample while the real unprofiled cost is
1.28 us/voxel. **The instrument inflates by ~11x.** Every bucket we have been
ranking and optimising for two rounds was measured under that inflation, so the
ranking does not reflect where real time goes. We optimised what the profiler
exaggerated.

⇒ Before any further optimisation: make the profiler cheap enough that profiled
and unprofiled totals agree within a small factor, or switch to sampling. An
alarming number is the instrument until proven otherwise -- three for three in
this project, and now four.

Still true: ~21x slower than the pre-op-stack build, and generation still loses
to a walking player. Geometry changed by 40 triangles (153,692 -> 153,732), which
is allowed; two full exports hash identically (37E00BEE..0A7DF). Build 42/42.
No CPU-feature dispatch. Nothing outside Plugins/VoxelForge was touched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
2026-09-09 03:05:40 +02:00
Fr0zkaandClaude Opus 5 0576e800c4 perf(gen): the landing work was per-voxel -- 342s to 2.6s, same world
The owner played 64f5d61 and reported the truth: "generation still takes ages...
TunnelNetwork doesn't even finish generating, as big as the surface world" while
SurfaceWorld completed in under 10s, and stopping PIE hung for 30+ seconds.

e08b49a's 21x was real and irrelevant. It fixed per-CHUNK state rebuilds, which
dominate the commandlet export path. The game is bound by per-VOXEL evaluation.
Same symptom, different bottleneck, and nobody had measured the second one.

WHY PIE HUNG
bShuttingDown was never checked inside VoxelMarchingCubesMesher.cpp -- zero
occurrences. A worker inside GenerateMesh could not abort, so 64f5d61's correct
unconditional EndPlay wait was waiting on tiles physically unable to stop. The
hang was not a teardown bug; it was evidence that one tile took tens of seconds.
Added shutdown polling per slab/row in the mesher and through GenerateTileResult;
aborted results carry Epoch + bAborted and ProcessQueue drops them. EndPlay still
waits unconditionally -- the 3s mark only logs now.

  PIE stop   3.409s / 44.759s  ->  4 ms teardown, 38 us worker drain

WHERE THE PER-VOXEL COST WAS
The owner's own comparison was the sharpest instrument available. Profiled both:

  GetDensityAt          TunnelNetwork 12.64 us/sample   SurfaceWorld 1.32
  tunnel candidates evaluated      7.088 per sample                 0
  tunnel-core candidates evaluated 7.201 per sample                 0
  room candidates evaluated        1.000 per sample                 0

  largest buckets: RoomGraphSource 2.57, TunnelCorePosts 0.76,
                   PassageCarveOp 0.22, PassageStructuralPosts 0.20

The three regressing commits map cleanly, and one is innocent:
  7a20eee drainage tree  -- cache-build work, NOT per-voxel
  413c15d landings       -- repeated full passage modification per sample
  55bf807 landing rooms  -- room SDF + tunnel/core evaluation per sample

Fixes: FPassageCarveOp carves only, landing/support moved to the structural MC
post; tunnel core support and SDF fused into one pass; room graph caches shared
per worker on complete deterministic keys; tunnel density cache uses open
addressing with 4096 slots. Room cache rebuilds fell to 9 per 42,875 samples, so
what remains is genuine O(local cached room/tunnel count) work.

  TunnelNetwork 128^3, seed 1337 slot 1, same region:
    342.2 s / 163.161 us/vox   ->   2.623 s / 1.251 us/vox     (130x)

  Geometry SHA-256 aaf95559..066b4, IDENTICAL to the pre-optimisation run at the
  same roughness. 153,692 triangles either way. Verified independently of the
  agent's own report.

NOT FIXED, and the first one is the next real target:
- L4 tiles still cost 16.93 s. Hit rate falls 99.9% (L0) -> 23.7% (L4), but
  1024 -> 4096 slots only moved 18.39s -> 16.93s and the 1,927 misses ARE the
  distinct key count. That is low reuse, not eviction. A bigger cache will not
  save it.
- TunnelNetwork is still ~10x SurfaceWorld (0.354s vs 0.028s on a matched 32^3).
- The live PIE tile table is SurfaceWorld: this checkout's PIE map is configured
  that way. TunnelNetwork went through the same mesher and density path via the
  commandlet, but no TunnelNetwork tile has been timed in an actual game session.
  Export numbers are not presented as game numbers.

Still outstanding, untouched here: the inter-strate passage has intersections it
should not have, routes through open passage when it should not, and should
descend from a low point of the current strate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
2026-09-08 23:47:53 +02:00
Fr0zkaandClaude Opus 5 e08b49a183 perf(gen): a thrashing one-slot cache cost 18.7x -- and the world is unchanged
TunnelNetwork density evaluation was 45x more expensive than Maze for identical
work: 2,097,152 voxels, 64 tiles, same mesher, same machine. Profiling killed
every hypothesis I brought to it -- box classification skipping 0/64 tiles was
true but irrelevant (export bypasses classification), near-surface early-outs
fired at 49.7%, the room cache built 340 entries for 2.74M samples, and the
per-room overrides resolved once per sample rather than once per modifier.

The cause was the wrapper, not the stack. A mesher tile samples an EXPANDED grid
(-1..Cells+1), so one worker crosses several exact chunk coordinates while
meshing a single tile. The generator's cache held only the last coordinate, so
every boundary crossing rebuilt the resolved parameters, the whole 22-operator
stack and the tunnel-core cache. On one representative tile: 3,675 refetches,
38.0s. The entire operator stack was 18.5s of a 4,272s total -- 0.43%.

Replaced with a 128-slot direct-mapped thread_local cache keyed on owner, chunk,
layout version, params fingerprint and generator type. Collisions force a safe
rebuild, never a stale read. No shared mutable state, no order-dependent
reduction.

  TunnelNetwork  344.5s / 164.267 us/vox  ->  18.4s / 8.763 us/vox   (18.7x)
  Maze             7.4s /   3.535 us/vox  ->   1.08s / 0.513 us/vox  ( 6.9x)
  same tile        38.0s                  ->   0.51s ; refetches 3675 -> 27

Output is BYTE-IDENTICAL, before vs after and across repeats, both archetypes:
TunnelNetwork 50d6b4d0..f23126, Maze 364f7489..2fbe99. The permission to change
the field was not needed.

Also in this commit, from the two preceding tasks:

- Explorer renders from the MESH instead of raymarching. It meshes the region
  once through the canonical UVoxelMarchingCubesMesher, then CPU-rasterises via
  a BVH. A viewpoint went from ~16 minutes to 0.155s; 8 views now cost 1.25s of
  drawing. No render pixel samples the density field.
- One sampled grid and player-fit mask per run, shared by render/walk/export,
  instead of the same 368s walk computed twice.
- A wall-clock budget (25 min default, hard 30 max) that stops cleanly and
  records truncated/truncated_during/completed_modes rather than overshooting.
- ParallelFor across the measurement sampling; the suite was ~10% of a 16-thread
  machine.
- -surfaceroughness= on the commandlet, and a surface-relief metric in export
  mode (edge lengths, dihedral distribution, face-vs-smooth normal divergence)
  so "does this look like rock" is a number, not a screenshot argument.
- Smooth vertex normals in the rasteriser: FVoxelMeshData::Normals is the
  density-gradient normal the engine shades with, and the raster path was
  recomputing a face normal over it.
- ScaleDiagnosis: baked candidate indices are now derived from the current
  corpus, so a legitimate field change stops breaking the test.

NOT verified, and not claimed:
- TunnelNetworkSpineEquivalence exceeded the 30-minute cap and was stopped.
  Inconclusive, not passing. The identical hashes are strong evidence the
  op-stack/legacy equivalence holds, but the test has not confirmed it.
- CrossPlatformDigest passes but reports 2 samples within 1e-6 of the
  isosurface; FMath::Sin/Cos leaves Windows/Linux equivalence unproven.
  Pre-existing, not caused by this change.
- The cache's memory footprint (128 entries x ~16 workers) is unmeasured.

What the roughness sweep settled, for the record: SurfaceRoughness 0 -> 8 moves
cost by 0.3% (163.2 / 163.2 / 163.7 us), so roughness was never the cost. It
does reach the surface (dihedral p50 1.1 -> 2.2 -> 5.9), which kills my claim
that the density-channel operator contributed nothing. TunnelNetwork's hard
seams are present at roughness 0 (4.4% of edges past 40 deg vs Maze's 1.0%), so
they are structural -- in how primitives are unioned -- not a roughness
deficiency. More roughness makes them worse, not better.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
2026-09-08 20:31:56 +02:00
Fr0zkaandClaude Opus 5 d97373c5a8 feat(gen): Jahni's playtest corrections -- rooms flattened, roads deleted
From walking a TunnelNetwork preset at (0,0). Four corrections, and two of
them were the same mistake.

2a. The (0,0) landing room is sized to host a FUTURE straight vertical
    shaft from the surface, with flooring around the origin itself. The
    room is the landing for a shaft the player digs later.
2b. The inter-strate descent no longer starts at the spine -- placed by
    deterministic hash within a radius around (0,0). The spine room is for
    the surface shaft; the descent is elsewhere.
2c. ⛔ DELETED the landing->hub connector roads from 413c15d. Jahni:
    "its straight up making other tunnels straightly through everything...
    they have nothing to do here." Zero references remain.
2d. ⭐ FLATTEN THE ROOM JOINS instead. Where two spherical rooms intersect
    and leave a lip above head height, the join is brought down to a
    walkable level.

⭐ 2c AND 2d ARE ONE CORRECTION. I solved "rooms do not connect" by
TUNNELLING BETWEEN THEM when the answer was MAKING THEIR FLOORS MEET. The
81-run sweep had already said so -- zero narrow-gap events, every failure a
vertical severance, renders showing "a sharp diagonal ledge descending into
black open space". A radial corridor to the origin could never have fixed
that; flattening the joins is aimed at exactly it.

2e. Descent tunnel 1.5x wider, per "1.5x bigger wouldn't be too much".
2f. Corridors WANDER. Jahni, from the renders: "its not a maze, its still a
    latice made straight paths." The spanning tree changed WHICH edges
    exist, not what an edge looks like -- every corridor was a straight
    axis-aligned capsule, every junction 90 degrees. Now hash-jittered
    chains, seam-safe like the tree.

RENDERER: 192x128 -> 512x288, and cameras are now seeded from the
player-fit set -- "a density-checked pose from the exact player-fit walk
mask". Previously they sat INSIDE ROCK looking at back-faces, which Jahni
spotted instantly and no metric could. First genuinely legible pictures of
this world.

ParallelFor landed at 6 sites in VoxelStrateMeasure before I stopped the
run; the rest of that work continues next.

⚠️ AND THE RENDERER IS ARCHITECTURALLY WRONG, per Jahni: "why, in engine,
does it take a minute to have clear view of thousands of angles, but in our
renderer it takes ages?" Because the engine MESHES ONCE and rasterises --
~32k density calls per chunk, then every angle is free. I RAYMARCH: tens of
millions of density calls PER IMAGE, nothing reused between frames. We
already have the mesher and an export mode. Next task renders from the
mesh.

Also from him: runs must stay under 30 minutes. This one hit 33 and
produced 2 images.

Build clean. Committed as a checkpoint -- the task was stopped mid-Part-3,
not finished.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
2026-09-08 14:14:29 +02:00
Fr0zkaandClaude Opus 5 55bf807f60 feat(descent): landing ROOMS and a walkable tunnel -- per Jahni's correction
Jahni corrected the design: "the connection between level is a tiny tunnel
connecting both strate, not vertical, it needs to be walkable" and "'the
spine' shouldn't be carved except optionally from the top first strate,
then it's used to have a landing room".

COMPOSER-NOTES §6.8 had said the spine "provides a clean landing space, not
a continuous open shaft" -- and the implementation carved a vertical
cylinder anyway. A vertical cylinder has no walkable floor except at its
bottom, so a carve guaranteeing open AIR guaranteed nothing STANDABLE.
That one omission is behind five of my dead hypotheses this week.

Now:
  origin spine   a finite rounded ROOM per strate, no vertical bore.
                 radius 14 -> 8 m wide, 7.5 m tall
  descent        a walkable tunnel, single-file by design: 3 m mouth,
                 2.5 m waist, gradient capped at a NAMED 15 deg
  exotic styles  VerticalShaft / SpiralDescent / CascadingDrops /
                 CrackCrevice preserved, no longer the default

⚠️ THE GEOMETRY BITES, and it is worth Jahni's eye: at 15 deg a 32 m drop
needs 119.4 m of horizontal run -- about 59.7 m per switchback leg. That is
a long walk between strates. It is the honest consequence of "walkable",
not a tuning artifact, and the alternative is a steeper number.

⭐ AND THE ROOT CAUSE GOT NAMED A THIRD TIME, by Codex mid-run: "the round
tube was hollow air but had no authored support strip, so 'open' still did
not mean 'standable'." Tunnels now carry a floor band, and the root join
terminates in the finite origin room rather than an annular shaft hub.

Measured, commandlet sweep, 8 archetypes x 8 seeds, 64/64 valid:
  origin column has player-fit cells   17/64 -> 63/64
  origin column reachable               0/64 ->  7/64
  arrival landing probes            315/320 fit AND in the arrival component
  tunnel audit    5,916/5,916 floor and capsule samples pass; zero gradient,
                  floor, capsule, geometry or seal failures
  narrow-gap events 0        box verdicts 14 checked, 0 violations

⚠️ arrival->departure is 6/64. The landings and tunnels are now provably
walkable; what remains is the INTRA-STRATE vertical connectivity problem
found in 6b91669 -- floors at different levels inside a strate with nothing
joining them. Separate and larger. Not tuned toward.

⚠️ PROCESS: this took 5.7 hours, of which Codex used 4 SECONDS of CPU. The
rest was polling the 73-minute automation suite at ~10% machine load. The
commandlet exists to avoid exactly that. New rule recorded: iterate with the
commandlet, full suite ONCE at the end.

Editor + Shipping builds clean. Automation 32/32, 0 failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
2026-09-08 02:14:37 +02:00
Fr0zkaandClaude Opus 5 6b916699af test(explore): 81 runs in an afternoon -- every failure is a DROP, not a squeeze
First real use of the commandlet, and it paid for itself immediately: 64
sweep runs + failure renders + origin probes, which at the old 73-minutes-
per-automation-run would have been days.

⭐ FINDING 1: THE WORLD IS FRAGMENTED VERTICALLY, NOT PINCHED.
Across all 64 runs: ZERO narrow-gap events. Every failed run has separate
arrival/departure fit components with a positive gap, 12.75 m to 72.98 m.
So corridors are not too tight anywhere -- the space is severed.

And for the first time in this project we LOOKED at a failure instead of
inferring from aggregates:
  FlatPlain/42   "a broad flat floor sheet with a sharp diagonal ledge
                  descending into black open space" -- a drop, gap 63.9 m
  SurfaceWorld/7 "terrain shelves and a large black void between them.
                  There is no connecting floor" -- gap 51.8 m
  Maze/42        "a massive solid rock wall/overhang fills the separation"
                  -- gap 27.7 m
I checked the images myself. FlatPlain shows two floor LEVELS 27 m apart
in Z with nothing joining them.

⇒ The missing thing is VERTICAL CONNECTION between floor levels. Ramps,
slopes, ledges, stairs. Every archetype builds floors; nothing builds a way
between them.

⭐ FINDING 2: THE ORIGIN HUB IS NOT REACHABLE, which contradicts 413c15d.
  agent's reachable set included the origin column:  0 / 64 runs
  origin column had ANY player-fit cell:            17 / 64 runs
  Maze slot 4, all 8 seeds:                          0 fit cells, unreachable
Even in the 17, the origin cells were not in the arrival component.

The reason is structural and obvious in hindsight: the origin spine is a
VERTICAL CYLINDER. A vertical shaft has no walkable floor except at its
bottom, so a scalar-density carve that guarantees open air guarantees
nothing standable. The landings join a hub you cannot stand in.

That is a finding about the §6.5 hub join as exposed to the player-fit
layer -- not walk tuning. Recorded rather than patched.

Also from Jahni, looking at the renders -- two things numbers missed:
- The render cameras sit INSIDE ROCK, viewing tunnel surfaces from the
  solid side. Cameras must be seeded from the player-fit set.
- ⚠️ THE MAZE IS STILL A LATTICE OF STRAIGHT PATHS. The spanning tree
  changed WHICH edges exist, not what an edge LOOKS like: every corridor is
  still a straight axis-aligned capsule between cell centres, every junction
  90 deg. So 15.9% dead ends and mean degree 2.504 measured maze-ness of the
  TOPOLOGY while the eye sees a sparser grid. Sixth time a metric answered a
  question adjacent to the one that mattered.

Two tool issues found: PowerShell needs -modes="render,walk" quoted or it
parses as render-only; and "showcase slot" is ambiguous (cards use 1/6, the
Maze diagnosis uses 4).

Determinism spot-checked: repeated arguments give identical SHA-256 JSON.
Measurement-only changes; no generation, parameter or threshold edits.
Editor build succeeded.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aheo2crrZmHuYtPpdY2Cb
2026-09-07 18:33:15 +02:00
Fr0zkaandClaude Opus 5 bd547c35c4 feat(explore): a commandlet -- 91x faster, and the first real PICTURE
Every experiment in this project cost a build + full-suite cycle. The
landing task took 52 of them over ~9 hours. And nobody could SEE the world:
every visual judgement all session was made on density SLICES -- diagrams,
not pictures. That is how a floor heaving 1.4 m every two metres got
described as "roughness visible but modest" while Jahni spotted it in
thirty seconds by walking it.

-run=VoxelForgeExplore, editor-only module, three modes, one build many runs:

  render  raymarched PNGs, fixed 0.25-voxel step + 10 bisection iterations
          (NOT sphere tracing -- the op stack yields density, not a signed
          distance, so a long step tunnels through walls). Three views, each
          with a 1.76 m human marker in frame, because every scale mistake
          here came from numbers with no body beside them.
  walk    an agent using the EXISTING VF player-fit stencil, never its own
          movement model -- two definitions of "can a player stand here"
          would drift and we could not tell which was lying.
  export  OBJ through the canonical UVoxelMarchingCubesMesher. No second
          mesher; a forked meshing invariant is the bug this project keeps
          paying for. 23.2 MB, 106,762 verts, 200,842 tris for 128^3.

⭐ THE NUMBER THAT JUSTIFIES IT: the full automation run takes 4,396 s
(73m16s). The commandlet does all three modes in 48.1 s wall, 5.9 s of
which is engine startup. ~91x faster. And I had guessed the suite cost
~90 s -- it is 73 MINUTES. The iteration loop was far more expensive than
I knew.

First real walk metrics on the Maze (seed 0, slot 4):
  distance 214 m · tortuosity 4.98 · dead ends 0.935/100 m
  narrow-gap events 0 · reachable fit fraction 3.01%
  result NotConnectedAtThisResolution
Tortuosity ~5 is a genuinely winding route, which is right for a maze.
3% reachable is not.

Editor Development build clean. SHIPPING build clean with the receipt
containing VoxelForge and NOT VoxelForgeEditor -- the instrument cannot
reach a packaged game. Automation 32/32, 0 failed.

⚠️ Codex initially shipped this UNBUILT, and correctly: I omitted the
## Validation section from that spec, so it fell back to CLAUDE.md's
"NEVER build or compile". 100+ compile errors from three causes -- bare
TCondensedJsonPrintPolicy and TJsonWriter needing their template
arguments, and a missing UObject/Package.h. That include is the SECOND
miss this session, so CODEMAP §6 now carries the gotcha.

⚠️ Unrelated, found in passing: VoxelForge.uplugin does not declare
RealtimeMeshComponent although the runtime module uses it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K8V9my1PL68pmQb2gutZZV
2026-09-07 04:07:07 +02:00