diff --git a/RESEARCH-ASTRA-2026-10.md b/RESEARCH-ASTRA-2026-10.md index d3f2d4b..9d44857 100644 --- a/RESEARCH-ASTRA-2026-10.md +++ b/RESEARCH-ASTRA-2026-10.md @@ -494,3 +494,290 @@ For implementation research, **the shared shoreline curve is my strongest near-t For invention, **the invertible material-space wall is the experiment I would fund**. It offers a plausible way to combine motion, persistent digging, compact networking, and cached geometry. Its first success criterion should be modest and strict: one moving slab whose hole, visible surface, and contact queries continue to agree. I would revise the earlier surface-first recommendation accordingly: retain explicit knowledge of the surfaces, their important intersections, and the material coordinates that survive animation. Those are useful facts the generator can provide once and reuse throughout rendering, editing, and gameplay. + +--- + +# Round 3 (fork): performance only, unrealistic goal (whole view < 1 s) — 2026-10-02 + +Prompt: scratchpad astra-think-3.md. Answer verbatim below. + +**I would bet on a generator that emits boundaries and solid intervals directly.** The distant world would come from a hierarchy of surface pieces. Nearby Boolean operations—joining caves, digging, blasting—would operate on intervals and packed bits. A density function could remain available for queries, but generating a view would no longer mean filling its volume with density samples. + +I can construct a **900 ms engineering budget** for this design. That is a target to test, not a measured prediction. Its essential condition is that the world’s composition limits the amount of surface work required. An arbitrary field can contain arbitrarily much geometry; no representation can guarantee a second without some complexity contract. + +The proposed architecture combines established techniques. The particular composition rules, division of work, and budget below are my proposal. + +**There is a larger opportunity in your current far field than the word “clipmap” suggests.** + +Your source supports expanding the outermost ring at a fixed maximum LOD. The checked-in settings use `RenderDistanceChunks=1536` and `MaxClipLevel=5`: that gives an outer radius of 48 tiles, approximately **97 × 97 horizontal positions per vertical slice**, before exclusions. Those are defaults, not a census of your running world. Nevertheless, this can turn the horizon into a large fixed-resolution grid. The settings explain why: coarser volume sampling can miss thin strata. [Settings](), [outer-ring calculation](). + +That is a representation problem with a performance consequence. **A distant floor should survive because it is explicitly a floor, rather than because a coarse lattice happens to intersect it.** + +There is a similarly useful distinction nearby: your column evaluator already shares setup across heights, but still loops through every Z sample. The opportunity below is to remove that loop by solving where the column enters and leaves rock. [Column evaluator](). + +**First, make each world piece know how to produce its surface.** + +A room, tunnel segment, shelf, shaft, pillar or floating mass would carry a small description with: + +- Its bounds and connections. +- A way to emit its surface at several resolutions. +- A way to intersect an axis-aligned line with its solid. +- Shared boundary curves for joining neighbouring pieces. +- A limit on how much refinement and Boolean work it can require. + +An untouched tunnel would emit a swept surface. A chamber would emit patches with explicitly constructed openings. A floating mass would emit a closed surface. These operations evaluate positions **on the surface**, without searching a surrounding volume for it. + +The difficult part is junctions. Drawing overlapping closed room and tunnel meshes leaves unwanted internal walls. I would begin with a bounded family of junction constructions whose openings, trimming and transitions are already defined. More unusual intersections would use the local interval compiler described below. + +This is a substantial generator redesign. Its advantage is that it makes the cost of a feature predictable. + +The hierarchy must also be **constructed from coarse to fine**. A parent knows the major chamber boundaries, floor sheets and passages before its children exist. Children add bounded shape variation and smaller features. A distant parent can therefore be rendered without generating and simplifying all its descendants. + +That does **not** make a coarse mesh an exact copy of every fine surface. What can be exact is the parent’s enclosure, connection information and refinement contract. The visible surface remains an approximation with a specified error. + +For deterministic connections, neighbouring pieces can derive their shared opening from the same integer face key. Each side then receives identical opening geometry. Matching-boundary composition has a precedent in Wang tiles; extending it to a hierarchical cave grammar with guaranteed connectors is my proposed adaptation. It needs a constructive connector for every allowed boundary configuration, otherwise a search problem returns. [Cohen et al., *Wang Tiles for Image and Texture Generation*](https://graphics.uni-konstanz.de/publikationen/Cohen2003WangTilesImage/index.html). + +Build the initial walkable route into this hierarchy, with enforced clearance and floor slope. Its existence then follows from construction; discovering it does not require generating the whole cave and running navigation over it. + +**The far field would contain selected surface patches, including explicit thin strata. It would contain no density tiles.** + +All selected LODs must be ready by the deadline. Showing a coarse preview while the requested near surface is still missing would not count. + +**Second, replace point evaluation with interval algebra wherever Boolean composition is necessary.** + +Consider a spherical cavity. At a fixed horizontal position \((x,y)\), define + +\[ +d=r^2-(x-c_x)^2-(y-c_y)^2. +\] + +If \(d<0\), that column misses the cavity. Otherwise its entire air interval is + +\[ +[c_z-\sqrt d,\;c_z+\sqrt d]. +\] + +One calculation has answered every height in that column. + +Multiple rooms and tunnels produce lists of intervals. Joining cavities becomes interval union. Adding a pillar removes an interval from air. A crater adds an air interval. There is no need to evaluate the composed function at every Z coordinate. + +This is related to **dexel representations**, used in machining and virtual sculpting: solids are represented by entry and exit depths along parallel lines, and cutting becomes interval operations. This is established machinery, not a new claim about solid modelling. [Peng et al., *Surface Reconstruction from Dexel Data for Virtual Sculpting*](https://scholarsmine.mst.edu/mec_aereng_facwork/1226/). + +For your world, I would use: + +- Quadrics for ellipsoids, cones and suitable chamber shapes. +- Piecewise analytic segments for tunnels. +- Explicit floor/ceiling intervals for shelves and room interiors. +- Bounded local exceptions for shapes outside that family. + +Using line families along **all three axes** provides boundary intersections in each direction, including on vertical walls. A 256-cell cube has approximately **198,000 lattice lines across the three directions**, versus roughly 17 million lattice points. If each line encounters an average of 12 candidate primitives, that is about **2.4 million primitive-line tests**. + +That is an operation-count estimate, not a speedup measurement: root solving and interval merging cost more than a single comparison. However, each test can replace many expensive calls to your complete density evaluator. + +This method becomes unattractive if every line crosses hundreds of tiny features. The generator must keep that complexity bounded, or represent those features directly as surfaces. + +**There are two useful ways to exploit 64-bit words here.** + +The first is to represent which primitives are active while sweeping through sorted interval endpoints. With at most 64 local primitives, their membership fits in one word. At a crossing, update the corresponding bit. Between crossings, membership is constant, so the local Boolean expression has one answer for the entire span. + +For a restricted construction such as “rock minus rooms and tunnels, plus pillars, minus craters,” the answer needs only a few tests of those membership masks. Arbitrary CSG trees require more work; the simple expression is a deliberate composition rule. + +The second is to store **64 consecutive rock/air signs in one word**: + +- Union, intersection and subtraction become OR, AND and AND-NOT. +- A long uniform span writes whole words. +- XOR with a neighbouring column identifies changed signs along X or Y. +- XOR with a shifted word identifies changed signs along Z. +- Bit scans enumerate only the interesting locations. + +For marching cubes, align the eight corner-sign words for 64 cells. Then + +\[ +\text{active}=\operatorname{OR}(\text{eight words}) +\;\land\; +\neg\operatorname{AND}(\text{eight words}). +\] + +That identifies 64 potentially intersected cells using word operations. `std::countr_zero` and clearing the lowest set bit can enumerate the survivors. Binary greedy meshers demonstrate the usefulness of processing occupancy and face masks this way. Their block-shaped output is not what I propose using for cave surfaces. [Binary Greedy Meshing](https://github.com/cgerikj/binary-greedy-meshing). + +The smooth surface comes from the **actual boundary crossings**, with quantized positions and normals, not from placing vertices halfway between binary samples. + +Two qualifications matter: + +1. Packing an arbitrary field into bits does not make generating those bits cheap. The interval construction is what removes the evaluations. +2. Equal corner signs can hide a thin feature entirely inside a cell. Such features need explicit primitive-intersection tags, refinement, or direct surface geometry. Binary signs alone cannot preserve them. + +A near ring with 17.4 million signs needs approximately **2.2 MB**, compared with approximately **69.6 MB for float densities**. Boundary data and meshes are additional, but the interior representation becomes small. + +**There is a particularly interesting mathematical family to investigate: piecewise triquadratic fields.** + +This is an optional extension, not something the first prototype must depend on. + +A polynomial that is quadratic separately in X, Y and Z has 27 coefficients. It offers two useful properties: + +- Restrict it to any axis-aligned line, and it becomes a quadratic: at most two isolated crossings unless the polynomial degenerates. +- Express it in the Bernstein basis over a box, and its value lies between the smallest and largest coefficients. + +Consequently, one small coefficient block supplies both cheap line intersections and conservative whole-box sign bounds. All coefficients positive means the entire box is air; all negative means rock. Subdivision can tighten the bounds. + +Bernstein bounds and subdivision are established mathematics used in implicit-surface research. **My speculative proposal is to make this restricted polynomial family part of the cave’s native construction language**, rather than fit it to an already expensive noise field. [Bán and Valasek, *Generalized Lipschitz Tracing of Implicit Surfaces*](https://onlinelibrary.wiley.com/doi/full/10.1111/cgf.15202). + +This distinction matters: fitting an arbitrary existing field would reintroduce sampling and approximation problems. + +Bounded detail can be handled similarly. If a parent function has a known range \([L,U]\), and all omitted detail together is bounded by \(R\), then + +\[ +F_{\text{complete}}\in[L-R,\;U+R]. +\] + +Whenever that interval excludes zero, all descendants are irrelevant to occupancy there. The amplitude bound must come from the construction rules; estimating it from a few samples would not be a proof. + +This approach spends a small, predictable amount on bounds supplied by each primitive. Your report’s failed generic interval experiment is good evidence against wrapping the existing entire evaluator in a more expensive arithmetic system. + +**The most radical version would upload surface instructions instead of most vertices.** + +An untouched surface patch can be described by its type, coefficients, transform, boundary identifiers and tessellation level. The GPU can draw a shared index pattern and calculate vertex positions from that description. This is ordinary rasterized geometry; it does not require raymarching the whole world each frame. + +As an illustrative estimate: + +- 20,000 patch descriptions × 128 bytes = **2.56 MB**. +- A conventional mesh with 1.2 million vertices and 2.4 million triangles, using 24-byte vertices and 32-bit indices, is **57.6 MB**. + +Both still produce geometry for the GPU to process. The saving is CPU generation, storage, transfer and object setup. + +Edited regions would switch to explicit generated patches. Untouched regions could remain compact procedural instances indefinitely. + +The costs are a custom rendering path, restricted patch families, careful transitions, and GPU arithmetic repeated in depth and shadow passes. Some patches should therefore be expanded once into GPU buffers and retained. This is a credible research direction, but I would **not require it to pass the first CPU experiment**. + +**The rest of the pipeline needs to be designed around bulk work.** + +Your recorded **639 mesh applies per second** is sufficient to make thousands of separate publications a problem even if generation becomes free. Your report also separates fast collision cooking from multi-second queue delays. Those are measured reasons to change the publication and support paths for this larger objective. [Recorded results](). + +I would use pooled geometry buffers and a small number of spatial render pages. Tiles or patches retain their own bounds and update ranges inside those pages. A small edit must update its ranges rather than rebuild a large page. + +GPU culling and indirect drawing can reduce CPU submission work while retaining many independently culled pieces. Direct3D provides the necessary indirect command mechanism; integrating it into Unreal’s scene, material and shadow paths is real implementation work. [Microsoft, *Indirect Drawing*](https://learn.microsoft.com/en-us/windows/win32/direct3d12/indirect-drawing). + +Only the player’s immediate support neighbourhood belongs on the initial collision deadline. Give it a priority path through generation, cooking and installation. Treat “physics state installed and a capsule sweep succeeds” as completion, rather than “cook task submitted.” + +Boundary ownership must also be explicit. Adjacent patches use identical quantized border samples and defined transition strips. Edited replacement patches meet those shared borders. This work belongs in the meshing budget; overlapping skirts are not a general proof of a watertight result. + +**Here is where I would allocate the second.** + +These are **proposed acceptance budgets**, not benchmark results. The workload assumed is: + +- A 64 m near region at roughly **25 cm structural resolution**, with finer appearance handled without a finer volume lattice. +- All selected distant surfaces to the requested horizon, including stacked layers. +- At most about 50,000 hierarchy records, 2–4 million nearby primitive-line tests, and **2.4 million output triangles**. +- Six generation workers. +- No seed-dependent geometry or field cache. + +The structural resolution and triangle count are explicit concessions. This budget does not establish the same result for every geometric feature in the current 12.5 cm field. + +| Stage | Work assumption | Principal data | Wall-time budget | +|---|---|---:|---:| +| Generate/select hierarchy and connections | Up to 50,000 records; order of 10–20 M simple operations | 3–8 MB | 30 ms | +| Solve and compose nearby intervals | 2–4 M primitive-line tests; order of 100–300 M arithmetic/comparison operations, plus roots | Paged scratch, target ≤24 MB | 150 ms | +| Produce sign masks and active cells | Up to roughly 1 M word groups; tens of millions of bit operations | 2–8 MB | 30 ms | +| Emit all near/far meshes and transitions | 1.2 M vertices, 2.4 M triangles; order of 100–250 M geometry operations | 57.6 MB output | 220 ms | +| Cook and install immediate support collision | Approximately the current small support neighbourhood | A few MB | 65 ms | +| Bulk GPU upload and scene publication | At most roughly 64 page publications | Approximately 60 MB | 100 ms | +| Final fence and first completed frame | Includes frame scheduling | — | 25 ms | +| Allocation, imbalance and integration contingency | — | — | 280 ms | +| **Total** | | | **900 ms** | + +The budgets are added conservatively; I have not counted overlapping stages as an extra saving. + +At an **assumed effective upload rate of 10 GB/s**, 60 MB takes approximately 6 ms of transfer time. That illustrates why publication, allocation and synchronization could matter more than raw bytes. It is not a measurement of your Unreal path. + +Likewise, the operation counts do not automatically imply the listed milliseconds. Roots, unpredictable branches and cache misses differ enormously in cost. The experiment must measure them. + +The largest unknown is the **required output complexity**. If a representative complete view needs 20 million triangles under the chosen surface error, this particular budget fails. The remedy would have to be better surface construction, instancing or approximation—not a claim that packed occupancy removed the output cost. + +“Cold” here means the application is running, shaders are available, and the seed/layout/edit data have arrived, but no world-specific result exists. Starting Unreal, compiling shaders or downloading an unbounded edit history requires a separate budget. + +**The 5800X3D makes a compact implementation particularly attractive.** + +AMD specifies eight cores, sixteen threads, 96 MB of L3, AVX2 and dual-channel DDR4-3200 for this processor. Its theoretical dual-channel memory rate is 51.2 GB/s; actual application bandwidth must be measured. [AMD specifications](https://www.amd.com/en/products/processors/desktops/ryzen/5000-series/amd-ryzen-7-5800x3d.html). + +I would organize the implementation as follows: + +- **Keep repeatedly accessed data around 40–60 MB.** The 96 MB is shared with the rest of the application. Stream finished mesh output rather than keeping every intermediate alive. +- **Process local pages with worker-owned scratch.** A 64³ sign block is only 32 KiB before borders. Finish its interval, boundary and mesh work while the relevant data are close. +- **Use arrays of primitive coefficients grouped by type.** AVX2 should process similar intersections together, without virtual dispatch or scattered feature lookups inside the loop. +- **Use AVX2 for batches, ordinary words for bitsets.** One vector handles four 64-bit masks. This is useful after the representation has removed most work; it is not a claimed 256-fold generator acceleration. +- **Exploit forward differences.** Along an integer scanline, \((x+1)^2-x^2=2x+1\). Quadratic terms can advance with additions; hash and coefficient setup happen once per feature or row. +- **Use Morton order between pages and simple linear order inside them.** This preserves spatial locality without expensive addressing in every inner iteration. +- **Measure six versus eight physical workers before using extra SMT workers.** Leave capacity for the game thread, renderer and collision path. Sixteen hardware threads do not supply sixteen independent cores. + +Avoid per-column allocation, shared append locks and global vertex hash tables. Count locally, assign deterministic output ranges, and write disjoint buffers. + +**Multiplayer agreement would come from a small, explicit numerical contract.** + +Use integer world keys, a versioned hash, fixed parameter quantization and stable feature identifiers. Define: + +- Rounding and sign-at-zero rules. +- Endpoint inclusion and coincident-crossing rules. +- Boolean operation order and material priority. +- Shared edge ownership and vertex quantization. +- Arithmetic ranges, widening and overflow behaviour. + +For quadrics, an integer square root or an exact predicate can determine the canonical crossing bracket. A floating-point estimate may accelerate the search only if exact arithmetic verifies and corrects the result. + +GPU floating-point surface shading can vary without changing gameplay. If GPU-generated geometry itself makes authoritative decisions, its integer decisions must match the CPU reference. Merely changing the language’s numeric type does not establish determinism. + +Clients independently generate the same descriptions and process the same ordered edits. Physics simulation need not become deterministic lockstep merely because world geometry is deterministic. + +For a 10 m spherical crater at 25 cm spacing, its projected footprint covers approximately + +\[ +\pi(10/0.25)^2 \approx 5{,}000 +\] + +columns per axis. That is roughly 15,000 affected lines across three directions, rather than evaluating the whole 20 m bounding cube. + +That makes immediate editing much more promising. It does **not** prove same-frame publication. The replacement surfaces must still be generated and switched together across every affected page and LOD. CPU support queries must observe the new edit immediately, even while replacement collision bodies are pending. + +**What this design gives up is mathematical generality, rather than three-dimensionality.** + +Overhangs, stacked chambers, floating masses, shafts and craters remain possible. The restrictions are on how they are composed: + +| Restriction | Performance obtained | +|---|---| +| A bounded primitive and junction family | Predictable surface emission and intersection cost | +| Limited overlap and crossings per local region | Short interval lists and cheap Boolean composition | +| Coarse structure constrains later refinement | Far geometry exists without generating descendants | +| Fine appearance does not all become collision topology | Surface work avoids growing with microscopic detail | +| Edited regions become local exceptions | The untouched majority keeps its compact representation | +| Finite supported scene and edit complexity | A meaningful loading-time target | + +The long-term risk is that edits accumulate until most regions become expensive exceptions. Local edit histories therefore need deterministic pruning or compaction into a bounded replacement representation. An unlimited edit log cannot have a fixed cold-load guarantee. + +A full-view raymarcher would remove mesh construction but move substantial work into every stereo frame. Global FFTs and tensor decompositions can be excellent for suitable restricted functions, but do not automatically survive arbitrary local cuts cheaply. I would pursue those after the boundary-producing construction has been tested. + +**The first experiment should test the interval compiler and count the output—not attempt a new renderer.** + +A one-day standalone C++ spike could answer the central question: + +1. **Build a representative geometric corpus.** Use room and tunnel records resembling your actual parameter ranges, plus pillars, shelves, floating solids, thin strata, rotated shapes and intersecting craters. Include crowded junctions. A sparse collection of spheres is insufficient. + +2. **Implement two versions of the same restricted world.** One evaluates occupancy point by point as a reference. The other generates intervals, composes them, fills sign words and extracts boundaries. They must describe the same new geometry; matching every detail of the old noise field is not the experiment. + +3. **Produce real mesh buffers.** Include boundary positions, indices, borders and deterministic ownership. A fast occupancy-only result can conceal an expensive reconstruction stage. + +4. **Exercise the far-world rule separately.** Generate the complete selected hierarchy and its coarse surface buffers without constructing fine descendants. Record nodes, patches, vertices and bytes. If implementing the full hierarchy exceeds the day, this part remains explicitly unproved. + +5. **Measure fresh-process runs on the 5800X3D.** Include allocation and first-touch costs. Test one, six and eight workers; report median, p95 and worst across seeds. Validate outside the timed region against the pointwise reference, including negative coordinates, page borders, tangencies, thin features and coincident edits. + +I would use these initial decision gates: + +| Result | Decision | +|---|---| +| All CPU geometry fits approximately **430 ms**, with output around **60 MB** | Proceed to the Unreal publication and collision test | +| Interval work fits its **150 ms** allocation, but meshing exceeds the budget | Investigate direct patches, reconstruction and output size | +| Only the simple cases pass; representative junctions produce huge interval lists | Reject or restrict that composition family | +| Far geometry requires fine descendants or dense strata sampling | The proposed hierarchy has not removed the principal cost | +| Any missing thin feature or inconsistent shared boundary | Correctness fails regardless of speed | + +The standalone test can establish whether the computational core deserves investment. The final proof is an Unreal run timed through **the first complete visible frame and successful player-support collision**, followed by a separate check that steady rendering meets the VR frame budget. + +I would spend the first day on **analytic intervals plus direct far surfaces**. They attack both sources of unnecessary work: repeatedly discovering nearby boundaries, and reconstructing distant surfaces from a volume that the player will never interact with.