From 04cdd197fb859ecadaa6aa15873391a65c3271a2 Mon Sep 17 00:00:00 2001 From: Fr0zka Date: Wed, 28 Oct 2026 04:46:50 +0100 Subject: [PATCH] research: Astra pass 3, compiled spatial passage evaluator design (read-only) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1 --- .../WorldAssemblerM3/Astra1/BRIEF-pass3.md | 74 ++++++++ .../WorldAssemblerM3/Astra1/REPORT-pass3.md | 161 ++++++++++++++++++ 2 files changed, 235 insertions(+) create mode 100644 Standalone/Research/WorldAssemblerM3/Astra1/BRIEF-pass3.md create mode 100644 Standalone/Research/WorldAssemblerM3/Astra1/REPORT-pass3.md diff --git a/Standalone/Research/WorldAssemblerM3/Astra1/BRIEF-pass3.md b/Standalone/Research/WorldAssemblerM3/Astra1/BRIEF-pass3.md new file mode 100644 index 0000000..3a7c534 --- /dev/null +++ b/Standalone/Research/WorldAssemblerM3/Astra1/BRIEF-pass3.md @@ -0,0 +1,74 @@ +Thank you for both earlier passes. Your rec #1 (localize passages) turned out to be right, and here is what we've +learned since. This is a focused third pass: **passage evaluation design.** Read and reason only; no edits +anywhere, including temp files. Web allowed. Labels as before. + +## What we know now (all measured, ablations verified to remove the work) + +- **Owner four-strate world, cold LOD0–2-ready at W12:** FI 2.27 s, TN 2.25 s. Passages off: FI 0.81 s, + TN 0.85 s. Boundary bands cost only 0.1–0.2 s. Passage composition costs **~1.4–1.6 s** of near readiness, and + it alone separates FI/TN from the ~1 s goal. SW 1.02 s and VS 1.13 s are close already. +- **Inside the passage phase** (sampled, L83): carve distance ~40%, wall relief ~21–24% (3D value noise, 3 octaves + at 43/29/19 voxels), support restoration/columns ~14%, mouth shaping ~4%, other ~11%. Top symbols: + `ApplyResolvedStratePassageCarves`, `VF_FieldCachedValueNoise`, `VisitNearestPassageSpatialNode`, + `PassagePotentialWithinNode`, the carve lambda. +- **Exact trims done** (L81–L85): + - a density-aware outer reach bound (carve = `max(source, -roughSDF/Q)`, reach = `-d·Q + 0.24 m`); + - an exact relief skip when it can't change the max; + - per-tile BVH node masks and contiguous ranges; + - heavy-tile splitting; + - heavy-first order. + + Each gave 5–12%. Readiness is throughput-bound (workers 81–85% busy). +- **Two representation shortcuts failed:** + - L86: a lazy exact per-sample result cache. 24% hit rate, ~700 MB, +47% slower. + - L88: a tile-local 4-voxel trilinear cache of passage deltas, exact near the surface. Coarse-node + evaluations still paid full route cost, and carve-segment visits rose from 34.5M to 56.2M (FI), making it + +46% slower. + + **Lesson:** each route evaluation is expensive in itself, and reorganising *where* we evaluate doesn't help + unless the evaluation gets cheaper. +- **Composer semantics that must keep their meaning:** + - the carve is `max(source, -roughSDF/Q)`; + - support restoration, applied after later overlapping bores; + - segment-local capsule clearance; + - mouth shaping and fades; + - the walkable tread floor; + - Catmull–Rom joins on short segments; + - wall/floor relief of ±0.21/0.20 m. + + Reports: `Standalone/Research/WorldAssemblerM3/Linux81`–`Linux88/REPORT.md`; code in `ShapeWorldInternal.h` + (`ApplyResolvedStratePassageCarves` and its helpers). + +## The owner's rules (unchanged) + +- One seed gives the same world on every machine. Byte equality with older versions isn't required; a perf pass + may change the world slightly. +- Passages stay walkable arrival → departure (support, capsule, grade, ripple audits), and look natural. +- No visible quality regression. +- Target: **LOD0–2 ready in ~1 s** after cold arrival. The whole view is a bonus; continuous low-priority far + generation afterwards. + +## Questions + +1. **What makes one route evaluation expensive** (read the code path end to end)? Per sample: how many segments are + visited, what does each visit compute (closest point on a Catmull–Rom or linear segment, frames, radius + interpolation, relief noise, support capsule tests, later-bore checks)? Which parts are redundant per sample, per + segment, or per tile? +2. **Design a passage representation where per-sample cost is near-constant and small,** and the work is not merely + moved around. Candidates to evaluate (don't limit yourself): + - analytic primitives: polyline/capsule chains with precomputed per-segment frames and an arc-length table; + - relief defined in 2D (arc length, angle) with a cheap periodic or table noise; + - a precomputed narrow-band distance grid per passage, built once per world at a fixed resolution with + deterministic integer construction, plus its build cost; + - separating "is this sample within reach" (cheap) from "exact carve value" (expensive); + - vectorising across a z-run of samples sharing the same segment candidates. + + For each: expected per-sample cost, build cost, memory, how exact (displacement vs today), determinism risk, and + effort. +3. **Rank by expected gain toward ~1 s,** with arithmetic from the numbers above. For the top choice, describe the + smallest prototype and the cheapest falsifier, so the implementer can prove it before building it fully. + +## Output + +At most ~3,000 words, **ranked recommendation first**, file:line evidence, labels [CODE]/[MEASURED]/[PRIOR ART]/ +[INFERENCE]. diff --git a/Standalone/Research/WorldAssemblerM3/Astra1/REPORT-pass3.md b/Standalone/Research/WorldAssemblerM3/Astra1/REPORT-pass3.md new file mode 100644 index 0000000..ec989f3 --- /dev/null +++ b/Standalone/Research/WorldAssemblerM3/Astra1/REPORT-pass3.md @@ -0,0 +1,161 @@ +# Astra 1, pass 3 — passage evaluation design (gpt-6-astra, medium, read-only), 2026-10-28 + +**[INFERENCE] Ranked recommendation: replace recursive route queries with a compiled spatial passage evaluator—not another cache of evaluated densities.** + +1. **Compile short lists of prepared carve, tread and clearance primitives into spatial bricks.** Evaluate those lists directly, retaining ordered support restoration and explicit mouth exceptions. +2. **Add a compact relief representation**, preferably sampled from today’s relief on the passage surface. Treat this as part of recommendation 1’s performance budget, not a sufficient standalone fix. +3. **Consider a sparse distance grid only if built directly from prepared primitives**, without calling today’s composer at every grid node. Its cold construction cost makes it a less attractive first prototype. + +[MEASURED → INFERENCE] The target requires a substantial replacement: + +\[ +T_{\text{FI}}\approx0.81+1.46r+B,\qquad +T_{\text{TN}}\approx0.85+1.40r+B +\] + +Here \(r\) is the fraction of passage cost remaining and \(B\) is added cold critical-path construction time. This is an ablation-based planning model, not a guaranteed linear prediction. + +| Passage cost remaining | FI, before added construction | TN | +|---|---:|---:| +| 30% | 1.25 s | 1.27 s | +| 15% | 1.03 s | 1.06 s | +| 10% | 0.96 s | 0.99 s | +| 5% | 0.88 s | 0.92 s | + +[INFERENCE] **Tenfold passage acceleration barely reaches one second before construction.** With 30 ms additional construction, TN needs approximately **11.7×**. Another 5–12% trim cannot close this gap. + +**What one evaluation actually does** + +[CODE] The entry point is [`ApplyResolvedStratePassageCarves:1115`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:1115). Its cost is a sequence of different geometric questions: + +1. **Reach screening.** Check tile flags, incoming density, passage Z range and floor-profile ranges. Solid samples can invoke `PassageMayAffectDensity` before entering the main passage loop. Inside that loop, another reach test may occur. +2. **Column/support preparation.** Scan eligible void crossings and possibly evaluate column distance. +3. **Nearest carve feature.** Search four ranges: descending body, upper endpoint, lower-mouth neighborhood and departure tail. +4. **Relief and mouth shaping.** Compute wall orientation, route-relative distances, entrance projection, relief and fades. +5. **Tread selection.** Find the floor-owning segment separately from the nearest carve segment; evaluate up to three neighboring support segments. +6. **Clearance and composition.** Check the passage’s player capsules, later bores and other passages’ capsules; apply, cancel or defer support. Finally restore mouth support and deferred support. + +[CODE] **A carve segment visit is not an iterative closest-point solve on Catmull–Rom.** It computes endpoint differences, squared length, a clamped projection onto the straight chord, then optionally evaluates a Catmull–Rom position at that same parameter. Radius remains interpolated from the endpoints. Distance requires a deterministic square root, followed by stable tie-breaking. [`ShapeWorldInternal.h:1497`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:1497) + +[INFERENCE] Consequently, replacing this with a mathematically accurate spline-distance solver would likely increase cost and change the field. Compiling the existing formula—or approximating it deliberately—is the appropriate comparison. + +[CODE] **Traversal itself is substantial.** Each nearest-node bound computes distance to an AABB using a square root, subtracts maximum radius, checks range membership and visits children. Potential-within queries perform a separate recursive search. Tile masks and contiguous ranges already remove some overhead; they do not eliminate per-sample traversal. [`:5590`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:5590), [`:5676`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:5676) + +[MEASURED → INFERENCE] L88’s approximately 34.5M/33.1M carve visits divided by approximately 14.06M/12.43M passage iterations gives about **2.45/2.66 carve visits per iteration**, assuming those census populations align. This is not visits per density query or per successful route evaluation: iterations include rejected passages, and the visit count excludes internal BVH nodes and separate capsule searches. **The problem is not necessarily hundreds of evaluated segments; it is expensive repeated search around relatively few winners.** + +[CODE] **Support is a different geometry query.** It uses horizontal projection, extended tread segments, distinct smoothing rules, native-density-dependent mouth offsets and floor-depth gates. Its floor-profile search deliberately avoids choosing a neighboring helix coil merely because that coil supplies the nearest wall. [`:1833`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:1833), [`:2127`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:2127) + +[CODE] Capsule checks project horizontally and test a vertical capsule with 0.34 m radius and 0.54 m axial half-length. Later-bore checks already have precomputed overlap candidates, but can fall back to BVH searches. Deferred support is restored after the passage loop. [`:5708`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:5708), [`:5810`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:5810), [`:2163`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:2163) + +[CODE] **Relief still costs work on cache hits.** Each of three octaves computes lattice coordinates, a cache hash, fades and seven interpolations. A miss additionally generates eight corner values. The cache has 64 direct-mapped entries. Wall relief uses wavelengths 43/29/19 and weights 0.50/0.35/0.15. [`VoxelForgeFieldShared.h:260`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/VoxelForgeFieldShared.h:260), [`:394`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/VoxelForgeFieldShared.h:394) + +[CODE] Mouth work additionally evaluates noise at projected route positions. Some calculations happen before determining whether the sample belongs to the mouth’s small affected region. [`ShapeWorldInternal.h:1683`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:1683) + +[INFERENCE] The reusable information is: + +- **Per segment:** chord vectors, inverse squared lengths, lengths, radius slopes, cubic coefficients, support flags, overlap extension and prefix/suffix arc lengths. +- **Per spatial brick:** possible carve winners, floor owners, clearance capsules, column features and ordered passage dependencies. +- **Per sample:** projections, distances, source-dependent support blending and final min/max operations. +- **Across a Z-run:** candidate lists and XY projections; not generally the winning segment or final support decision. + +[CODE] `BuildPassageComposerConstants` currently prepares passage-level metadata, not this complete segment representation. [`:5147`](/home/jahni/VoxelM/Plugins/VoxelForge/Standalone/Core/src/ShapeWorldInternal.h:5147) + +**1. Compile a spatial passage program** + +[INFERENCE] Build immutable sparse bricks containing contiguous references to prepared features. Use separate lists for **carve, tread, clearance and mouth/column operations**. A single nearest-segment identifier cannot represent their different ownership rules. + +For each brick: + +- Eliminate segments that cannot win its carve query, using conservative lower/upper distance bounds. +- Retain every potentially relevant overlapping coil. +- Store floor candidates independently. +- Store later-bore and other-capsule dependencies in stable passage order. +- Subdivide crowded bricks or use an explicit slow fallback; never silently cap candidates. + +[INFERENCE] The common query becomes one brick lookup, typically a few direct projections/distances, one relief lookup and a small ordered composition sequence. No reach-BVH traversal followed by nearest-BVH traversal followed by clearance-BVH traversal. + +**Important limitation:** “two to four candidates” is a design target to falsify, not something the current route guarantees. Nearby coils and short, densely authored segments may produce much larger lists. + +[INFERENCE] Prepare the existing chord/Catmull formulas first. This avoids multiplying segment count through premature tessellation. Subsequently merge or approximate pieces only where a measured error budget permits. Precomputed coefficients and reciprocals may change floating-point rounding; version the evaluator and enforce the same arithmetic across peers. + +[PRIOR ART] If route-coordinate relief needs frames, compute stable frames once. Rotation-minimizing frames are established for swept surfaces; ordinary Frenet frames can become discontinuous at inflections. **Keep the walkable floor gravity-referenced**, regardless of the wall-texture frame. [Wang et al., rotation-minimizing frames](https://www.microsoft.com/en-us/research/publication/computation-rotation-minimizing-frames/) + +[INFERENCE] **Do not discard the density-dependent reach rule.** A narrow geometric envelope alone is insufficient when `max(source, -roughSDF/Q)` can change deeply negative source values outside the visible tube. Outside a brick’s detailed region, return identity only with a valid source-aware proof. Otherwise use a wider prepared candidate set or an explicit fallback. Count that fallback in the speed budget. + +[INFERENCE] Cost model: + +- **Query:** target roughly **0.1–0.3 μs** for an ordinary active query with 2–4 prepared candidates, excluding rare complex exceptions. This is an engineering target, not measured performance. +- **Build:** \(O(S+R)\) segment preparation/reference emission, plus candidate-certification work; \(S\) segments and \(R\) brick references. Target **under 30 ms added cold critical path** initially. +- **Memory:** approximately 128–256 bytes per prepared segment, plus roughly `16 + 4K` bytes per simple brick list, before multiple semantic lists and directory overhead. Thousands of segments should require megabytes, not L86’s hundreds of megabytes. +- **Exactness:** prepared original formulas can preserve geometry closely; segment simplification has no automatic displacement guarantee. Keep mouth/support equations source-dependent. +- **Effort:** medium-to-large; the candidate compiler and semantic separation are the substantial work. + +[INFERENCE] Leaving the entire old support phase untouched is unlikely to suffice: **14% of the 1.4-second passage contribution is approximately 0.20 seconds**, already exceeding TN’s available 0.15-second margin. Preserve support’s meaning, but accelerate its feature selection too. + +**2. Replace repeated 3D relief evaluation with a small surface field** + +[INFERENCE] Use coordinates along the route and around its wall, with periodic angular indexing and continuous frames. A table can combine all three octaves into one lookup. Avoid evaluating general-purpose `atan2` unnecessarily; a specified octant-based angular mapping is an option, but its distortion must be checked. + +Prefer initially **sampling today’s 3D relief at the reference wall surface** rather than inventing unrelated 2D noise. This retains much of its spatial appearance. Extend the value radially only over a controlled band; preserve the floor-alignment weights and separate mouth-floor treatment. + +[INFERENCE] Costs and risks: + +- **Query:** target **20–80 ns** for coordinate mapping plus a warm four-value lookup; cold memory or frame interpolation can exceed this. +- **Build/memory:** illustrative 1 km of route, 0.25 m longitudinal spacing, 64 angular samples and int16 values: **256k samples, approximately 0.5 MB**. Build requires those relief evaluations once, not full composer evaluations. +- **Exactness:** independently regenerated noise can differ by **0.42 m in wall offset** between opposite extrema despite retaining ±0.21 m amplitude. Equal amplitude does not establish equal quality. Surface sampling reduces this risk but does not prove a displacement bound. +- **Determinism:** fixed integer coordinates, specified interpolation/rounding, periodic endpoints and stable frame construction. +- **Effort:** medium; seam handling, frame continuity and floor/mouth validation dominate. + +[MEASURED → INFERENCE] Even removing **all** the measured 21–24% relief share saves only roughly **0.3 seconds**. An 80% relief reduction alone predicts roughly **2.0 seconds**, not one second. It belongs in the combined representation. + +[INFERENCE] Cheapest falsifier: evaluate table interpolation error and normal/ripple changes on one curved wall, one mouth and one self-overlapping coil. Reject a table that needs near-original per-sample 3D noise to hide seams or preserve appearance. + +**3. Sparse narrow-band distance grids** + +[INFERENCE] A grid gives genuinely near-constant queries, but only if its construction avoids L88’s failure: **do not fill it by invoking the existing full route evaluator per node**. + +Rasterize prepared local primitives into owned bricks, reduce candidates deterministically, and construct the band from those primitives. Store separate carve and support/clearance information where needed. A baked final delta cannot represent all source-density-dependent mouth restoration. + +[PRIOR ART] OpenVDB’s narrow-band representation stores active distances near the surface and constant signed values elsewhere. That is useful representation prior art, but its clamped exterior is not automatically compatible with your composer. [OpenVDB representation](https://github.com/AcademySoftwareFoundation/openvdb/blob/master/doc/doc.txt) + +[INFERENCE] For route length \(L\), radius \(r\), band half-width \(w\) and spacing \(h\), band sample count is approximately: + +\[ +N \approx 4\pi r w L/h^3. +\] + +Illustratively, \(L=1{,}000\) m, \(r=3\) m, \(w=0.75\) m: + +- **25 cm spacing:** approximately 1.8M samples, **3.6 MB per int16 channel**. +- **12.5 cm spacing:** approximately 14.5M samples, **29 MB per channel**. + +Overlapping passages, mouth volumes, brick padding and multiple channels add memory. + +[INFERENCE] **Query:** roughly **0.05–0.25 μs per interpolated channel** as a target, with locality decisive. **Build:** millions of local distance operations; even 0.1–0.3 μs per sample means 0.18–0.54 CPU-seconds for the coarser example, before indexing, extra channels or allocation. Parallel wall time must be measured; it is not free. + +[INFERENCE] Grid spacing is not a displacement bound. At 25 cm, narrow tread/clearance transitions need analytic treatment or finer data. Integer storage does not ensure deterministic construction: integer distance/projection arithmetic, tie-breaking, reduction order and interpolation must also be specified. + +[INFERENCE] Effort: large. This is attractive only if direct construction plus all arrival queries beats the compiled analytic representation. A tiny benchmark of grid lookups alone proves nothing. + +**Other candidates** + +- **[INFERENCE] Cheap reach versus exact value:** retain the distinction, but answer reach from brick metadata or squared primitive bounds. The current distinction already exists; repeating its recursive BVH search is the expense. +- **[INFERENCE] SIMD Z-runs:** useful after candidate compilation. XY projection is reusable for tread/capsule queries, while 3D carve projection varies affinely with Z before clamping. Expect amortized lookup and arithmetic benefits, not eightfold speedup. Memory adds only scratch arrays; construction is unchanged. Deterministic lane arithmetic and stable winner selection are required. Medium effort. +- **[INFERENCE] Pure capsule-chain replacement:** cheap per primitive, but tessellation can increase candidate count and introduce curvature/radius artifacts. Preserve short-join shape with bounded approximation; do not equate a visually smooth centerline with a walkable tread. + +**Smallest prototype and decisive falsifier** + +[INFERENCE] Implement one prepared passage evaluator for the **FI/TN junction passage**, including an overlapping coil and tread—not a straight isolated tube. + +1. Compile segment constants and brick candidate lists; retain existing geometry formulas and relief. +2. Run the complete carve/support/clearance sequence from those lists. +3. Measure construction, query time, candidate-count distribution and fallback time separately. +4. Add the relief table only after direct evaluation beats the old traversal. + +[INFERENCE] The cheapest first falsifier is the **candidate compiler census**: if commonly sampled bricks still need many segments or fallback retains more than approximately 10% of original passage time, the proposed common path cannot meet the target even if nearly free. + +[INFERENCE] Then replay representative samples with their incoming densities and original spatial order. Include misses, mouths, support and overlapping bores; exclude benchmark-only allocations. A promising kernel must finally survive cold arrival timing including construction, peer comparisons, capsule/support/grade/ripple audits and visual inspection. + +[INFERENCE] Budget acceptance around **90–95% total passage-cost removal**, not an attractive isolated kernel speedup. The measured ceiling makes approximately one second plausible, but only if the new representation replaces the search-heavy semantic pipeline broadly enough. + +Read-only review; no files changed or generation/tests run. \ No newline at end of file