merge: smooth inter-strate tunnels (L109-L126 line, evaluator v225) into the main line

Owner-approved. Brings the swept-tube shell, half-pipe open air, continuous floor h(s), rounded crest and
L125 slab stamping onto assembler/world (L120-L124 streaming work). Pending L127 verification.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
This commit is contained in:
Fr0zkaandClaude Opus 5.5 committed 2026-10-30 15:16:09 +01:00
commit aeb60bbda3
526 files changed
+38470 -5136

No files matched your search

@@ -0,0 +1,60 @@
# VoxelForge — read-only design review: stamp tunnels along the curve instead of searching the curve per sample
You are reviewing a design idea against real code. **Do not edit, create or delete any file, and do not build or run
anything.** Read code and reason. Be thorough; mark each claim as [CODE] (with file:line), [MEASURED] (from a report),
or [INFERENCE].
## Context
VoxelForge Core (`Standalone/Core`) generates a density field: negative = solid, positive = air, 25 cm voxels, 32³ tiles,
LOD0–8 clipmap, marching cubes. Same seed must give the same world on every machine and worker count; byte-identity with
older versions is NOT required.
Inter-strate tunnels ("passages") on branch `wip/l111-halfpipe-spline` (HEAD `707bc61`) are a smooth spline swept tube:
- a D-shaped walkable floor with a continuous floor height h(s), max grade 15°;
- a rounded crest;
- an open-air half-pipe where there is no rock;
- a thin solid shell ("plain shell rule": interior always air, a thin shell always solid);
- smooth-min where tunnels meet;
- mouths that ramp into the ground.
Key code: `Standalone/Core/src/ShapeWorldInternal.h` (`FScopedPassageTileQueries`, `prepareSweptTubeBase`, around
lines 374 and 11436), `Standalone/Core/src/VoxelForgeCore.cpp:~2747`. Reports: `Standalone/Research/WorldAssemblerM3/Linux109`
… `Linux119` (REPORT.md each).
Cost today [MEASURED, L117–L119]: the smooth tunnel is 13–22% slower than the coarse L109 tunnel at LOD0–2 readiness.
About 95% of the extra cost is in passage-feature tiles. The causes: sweep pairs (sample × curve segment), exact
fallbacks that re-scan segments for off-grid queries, segment visits, per-tile prepare, and smooth-min.
## The idea to review (owner + orchestrator)
Today each sample "pulls": it searches for the nearest curve segment and computes a distance. Proposed: "push"/stamp.
For each tile, walk the curve in rings (frames: position, forward, up, as with an Unreal spline mesh) and write the
tube's signed distance into only the lattice samples near each ring. Only a narrow band of about one voxel around the
wall needs an exact value; deep rock only needs "rock" and the tunnel interior only needs "air". Off-grid queries
(normals/gradients at seams, gameplay point queries) would read the stamped lattice, interpolating, instead of
re-searching the curve. Also: accept curve accuracy at one voxel (25 cm) rather than 2.5–10 cm, and build the curve
frames once per route rather than per tile.
## Questions
1. Is the cost model right? From the code, what does a passage-feature tile do per sample today, and what would stamping
replace? Estimate the work ratio (samples touched × ops) for a typical LOD0 tile and for LOD1–2.
2. What breaks or gets harder, concretely in this code?
- box verdicts/tile skipping (`ClassifyTile`-style proofs that a tile is all solid/air);
- seams between tiles: both neighbours must stamp identical values at shared samples, deterministically;
- LOD levels (coarser lattices);
- off-grid `DensityAtUnclamped` queries and gradients;
- min/smooth-min with the rest of the field and with other tunnels;
- the floor h(s), half-pipe, shell, mouths;
- walk audits;
- thread safety.
3. Is "write into the lattice" compatible with how density is evaluated (lattice fill vs point evaluation)? If the
pipeline is pull-only, what is the smallest change that gets the push benefit? For example, a per-tile sparse
band list built by walking rings, then lookups during fill.
4. Is there a simpler or cheaper alternative we're missing? For example, an analytic local frame per sample from the
spline parameter, or a coarse per-tile segment window that is already almost free.
5. Recommendation: the design you would build, in steps, with the evidence bar (what measurement would prove or kill
it). Keep it short.
Write your answer as plain Markdown, at most ~1,200 words.
@@ -0,0 +1,53 @@
[INFERENCE] **Build a tighter passage cache first.** Ring stamping is promising, but the code already pushes segment contributions into a tile lattice. The main opportunity is reducing coverage, overlap, and off-grid fallback work—not reversing a purely pull-based architecture.
References below: `H` = `Standalone/Core/src/ShapeWorldInternal.h`; `C` = `Standalone/Core/src/VoxelForgeCore.cpp`; `M` = `Standalone/Source/VoxelForge/Private/VoxelMarchingCubesMesher.cpp`. Report references mean `Standalone/Research/WorldAssemblerM3/LinuxNNN/REPORT.md`.
**1. Cost model**
[CODE] `prepareSweptTubeBase` allocates per-passage `(cells+3)³` sample records, including a one-step halo. It walks candidate bounding boxes, lattice samples, then fine segments; individual segment bounds reject some pairs. Accepted pairs compute projection, deterministic square root, radius, floor profile, and potentially shell distance. A final pass flattens the selected tube and caches relief (`H:11436`, `H:11489`, `H:11521`, `H:11659`).
[CODE] During density evaluation, aligned samples retrieve that cache, then still perform relief weighting, floor shaping, mouth fades, shell composition, and host-field composition. Misses invoke two hierarchy searches: nearest body/floor and nearby shells—not necessarily a linear whole-route scan (`H:1247`, `H:1346`, `H:1427`, `H:1435`).
[MEASURED] L119 reports 29.40M preparation pairs and 24.46M segment visits across its four LOD0–2 profiles. Preparation fell 59.8% versus L118, while fallbacks increased 55–58%. L117 attributes about 95% of the tile-worker regression to passage-feature tiles. Thus the diagnosis is sound, but those counters do not establish equivalent costs per operation.
[INFERENCE] For an illustrative straight crossing, assume exposed surface perimeter \(P=16\) m, tile crossing length \(L=32\Delta\), and exact band ±one lattice step. Unique band samples are approximately \(2PL/\Delta^2\):
| [INFERENCE] Normal tile | LOD0 | LOD1 | LOD2 |
|---|---:|---:|---:|
| Spacing Δ | 0.25 m | 0.5 m | 1 m |
| Band samples | 4,096 | 2,048 | 1,024 |
| Fraction of 35³ prepared grid | 9.6% | 4.8% | 2.4% |
[INFERENCE] These are geometric estimates, not measured “typical tiles”; clipping, floor/outer-shell surfaces, bends, and crossings change them. If today fraction \(f\) of the grid receives \(k\) segment evaluations, and stamping averages \(m\) contributions per band sample, the expensive-work ratio is approximately `(table fraction × m)/(f × k)`, multiplied by relative operation cost. For `f=0.5, k=2, m=2`, that is 19%, 9.6%, 4.8%. Allocation, classification, host evaluation, and composition remain. Dense rings can make \(m\) large and erase this advantage.
**2. Concrete difficulties**
- [CODE] **Skipping:** passage-bound overlap currently vetoes a uniform proof and returns `Mixed` (`H:5684`). [INFERENCE] Retain conservative bounds, expanded for approximation and reconstruction support. “No band samples written” proves neither solid nor air: a thin shell can lie between samples.
- [CODE] **Seams:** cache coordinates derive from integer world coordinates; nearest ties use segment index (`H:11448`, `H:11564`, `H:11640`). [INFERENCE] Generate rings from route-global parameters, never tile-local stepping. Include every contributor affecting shared samples and halos; use stable ordering and arithmetic. Independently clipping ring generation at tile boundaries risks disagreement.
- [INFERENCE] **LOD:** distinguish 25 cm curve error from lattice spacing. A fixed 25 cm band is insufficient around 1 m cells; cover interpolation corners and gradient support. A 0.75 m shell can disappear at coarse resolution. Independently reconstructed LOD fields also disagree off-grid; define one canonical point-query field.
- [CODE] **Off-grid:** `sweptSample` rejects non-lattice coordinates; public point queries call `DensityAtUnclamped` without requiring a tile cache (`H:11721`; `C:1875`). Gradients normally use density-grid values, with unclamped fallback (`M:1972`). [INFERENCE] Interpolation needs all eight valid corners, including outside loaded tiles. It changes gradients and zero crossings; interpolating only passage channels also differs from interpolating the fully composed field.
- [CODE] **Composition:** passage bodies and shells use separate hard minima; final composition includes host/native-floor exceptions (`H:1496–1519`). The floor corner uses smooth-max, implemented through smooth-min (`H:3493`, `H:6940`). [INFERENCE] Do not assume the measured smooth-min cost is tunnel-to-tunnel joining. Preserve separate void/shell channels and composition order. Arbitrary “rock/air” magnitudes can move interpolated crossings and smooth blends.
- [CODE] **Shape:** floor selection is independent of minimum round distance; the code explicitly warns that minimizing flattened segment distances makes downhill floors jump (`H:1328`). Floor evaluation uses horizontal projection, authored `h(s)`, and XY mouth blending (`H:3448`). Shells have rounded crests and five-metre mouth fades (`H:1461`, `H:3503`). [INFERENCE] A conventional spline-normal D-section changes this geometry. Stamped isolated rings can scallop; rasterize bounded slabs between frames while preserving gravity-relative floors, shell unions, and interior priority.
- [MEASURED] **Walkability:** L119’s maximum sampled floor-root offset was 7.884 mm; L113 documented voxel-aligned floor ribs. [INFERENCE] A 25 cm curve tolerance does not authorize 25 cm floor error or guarantee ≤15° grade. Audit reconstructed density and meshes, including mouths, overlaps, support, and headroom—not only planner `h(s)`.
- [CODE] **Threads:** copied worker scopes share prepared data through a shared pointer; active scope is thread-local (`H:353`, `H:11411`; `C:2854`). [INFERENCE] Finish stamping before publication. Parallel ring writes require deterministic reductions; mutable shared lazy caches introduce synchronization and lifetime problems.
**3. Smallest useful change**
[CODE] Preparation already precedes `GenerateMesh`, whose fill calls lattice evaluators (`C:2747`, `C:2892`; `M:1488`). Route segments are already prepared globally with 10 cm chord tolerance and an 8 m spacing cap (`H:4601`).
[INFERENCE] Replace preparation’s broad pair enumeration with conservative slab-to-grid spans or small spatial bins. Keep indexed lookups during pull-based fill. Store body/floor ownership and shell contributions separately; use explicit outside/interior/band states. A dense index plus compact records may beat a sparse hash map. Additional frames can be route-global, but this does not eliminate existing per-tile rasterization.
**4. Alternatives and recommendation**
[MEASURED] L115 already reduced coarse candidates 74.1% without reducing fine pairs. L118’s restriction of fallbacks to prepared spans changed output; L116’s 20.8% pair reduction did not recover readiness.
[INFERENCE] An analytic local frame still needs a parameter lookup, especially near folds. Prefer globally indexed conservative segment bins, shared by preparation and exact point queries.
[INFERENCE] Build in three steps: instrument unique samples, overlap, misses by caller, memory, and per-LOD time; implement tighter bins/slabs while preserving geometry; then separately trial 25 cm curve tolerance and canonical interpolation. Accept only reproducible near-readiness gains against L119/L109 plus unchanged walk/seam requirements and cross-compiler/worker determinism. Kill the approach if saved distance work becomes rasterization, memory, or reconstruction cost.
@@ -0,0 +1,111 @@
#!/usr/bin/env python3
"""Measure the fixed-depth floor and derived centreline against the authored grade."""
from __future__ import annotations
import json
import math
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
LAYOUT = ROOT / "Saved/Linux/l109/audit-layout-final-stamp.json"
OUT = ROOT / "Saved/Linux/l109/floor-ramp-grade-final.json"
MOUTH_RAMP_METRES = 5.0
def route_m(layout: dict, passage: dict):
scale = layout["core_voxel_size_um"] / 256.0 / 1_000_000.0
controls = [(tuple(float(v) * scale for v in row["position_q"]),
float(row["radius_q"]) * scale) for row in passage["control_points"]]
lower_q = passage["lower_mouth_q"]
player_center_above_floor_q = round(
1.13 * 256.0 * 1_000_000.0 / layout["core_voxel_size_um"])
lower_index = next((index for index, row in enumerate(passage["control_points"])
if row["position_q"][0] == lower_q[0]
and row["position_q"][1] == lower_q[1]
and row["position_q"][2] - row["radius_q"]
+ player_center_above_floor_q == lower_q[2]), None)
if lower_index is None:
raise RuntimeError(f"{passage['stable_id']}: lower mouth control is unbound")
return controls[:lower_index + 1]
def maximum_grade(profile):
grades = []
for (p0, z0), (p1, z1) in zip(profile, profile[1:]):
horizontal = math.hypot(p1[0] - p0[0], p1[1] - p0[1])
if horizontal > 1.0e-8:
grades.append(math.degrees(math.atan2(abs(z1 - z0), horizontal)))
return max(grades, default=0.0)
def main() -> int:
layout = json.loads(LAYOUT.read_text(encoding="utf-8"))
rows = []
for passage in layout["passages"]:
route = route_m(layout, passage)
segment_lengths = [math.dist(a[0], b[0]) for a, b in zip(route, route[1:])]
prefix = [0.0]
for length in segment_lengths:
prefix.append(prefix[-1] + length)
interior_radii = [radius for distance, (_, radius) in zip(prefix, route)
if distance >= 10.0 and prefix[-1] - distance >= 10.0]
minimum_body_radius = min(interior_radii or [radius for _, radius in route])
body_depth = max(0.5, minimum_body_radius - 0.23)
scale = layout["core_voxel_size_um"] / 256.0 / 1_000_000.0
upper_target = max(0.0, route[0][0][2]
- passage["upper_mouth_q"][2] * scale + 1.13)
lower_target = max(0.0, route[-1][0][2]
- passage["lower_mouth_q"][2] * scale + 1.13)
floor_profile = []
raw_fixed_depth_profile = []
maximum_centerline_shift = 0.0
for segment_index, (a, b, length) in enumerate(
zip(route, route[1:], segment_lengths)):
steps = max(1, math.ceil(length / 0.05))
for step in range(steps + 1):
if floor_profile and step == 0:
continue
t = step / steps
position = tuple(a[0][axis] + (b[0][axis] - a[0][axis]) * t
for axis in range(3))
radius = a[1] + (b[1] - a[1]) * t
along = prefix[segment_index] + length * t
mouth_distance = max(0.0, min(along, prefix[-1] - along))
u = max(0.0, min(1.0, mouth_distance / MOUTH_RAMP_METRES))
ramp = u * u * (3.0 - 2.0 * u)
local_radius_depth = max(0.0, radius - 0.23)
mouth_target = upper_target if along <= prefix[-1] * 0.5 else lower_target
mouth_depth = max(0.0, min(radius, mouth_target))
route_depth = mouth_depth + (local_radius_depth - mouth_depth) * ramp
tube_depth = min(body_depth, local_radius_depth)
effective_center = position[2] - route_depth + tube_depth
floor_profile.append((position, effective_center - tube_depth))
raw_fixed_depth_profile.append((position, position[2] - tube_depth))
maximum_centerline_shift = max(maximum_centerline_shift,
abs(effective_center - position[2]))
rows.append({
"passage": passage["stable_id"],
"path_m": prefix[-1],
"body_floor_depth_m": body_depth,
"mouth_blend_m": MOUTH_RAMP_METRES,
"preserved_route_floor_max_grade_deg": maximum_grade(floor_profile),
"unshifted_fixed_depth_floor_max_grade_deg": maximum_grade(
raw_fixed_depth_profile),
"maximum_derived_centerline_shift_m": maximum_centerline_shift,
})
OUT.write_text(json.dumps({
"method": "0.05 m route samples; fixed body depth from the minimum interior radius; "
"cubic smoothstep mouth blend",
"passages": rows,
}, indent=2) + "\n", encoding="utf-8")
for row in rows:
print(f"[L109FloorRamp] {row['passage']} "
f"depth_m={row['body_floor_depth_m']:.3f} "
f"preserved_grade_deg={row['preserved_route_floor_max_grade_deg']:.3f} "
f"unshifted_fixed_depth_grade_deg={row['unshifted_fixed_depth_floor_max_grade_deg']:.3f} "
f"max_center_shift_m={row['maximum_derived_centerline_shift_m']:.3f}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,70 @@
# Linux 109 — tunnel floor and shell
Baseline: `assembler/world` HEAD `95cf8de10723f2dffe7b3dfc116ecf350c7114ae`. Candidate source stamp: `9457b9bed7d57a4a387828961f9dd53632ec602eb5ea8f247b087be6ca5cc312` (evaluator v217).
## Shape
The per-tile swept tube now uses a horizontal flat floor, a 0.50 m smooth floor-to-wall corner, and a 0.75 m shell. Body floor depth is fixed per passage at the minimum interior radius minus 0.23 m, capped at the local radius. Five-metre cubic blends ramp both mouths into their ground; the lower mouth follows the matching authored native-cave floor where one exists. The route centreline shifts through radius tapers to keep that floor walkable. A small wall relief remains for naturalness.
The passage interiors are unioned first and always win over every passage shell, including at coiled crossings. The shell writes solid only where the host field is air; rock remains unchanged. There is no rock classification. Open-air shell and interior samples pass in the owner audit.
## Removed and retained
Removed route support restoration, deferred support after later bores, support columns, mouth-support restoration, and their floor-specific capsule checks and profiling records. Replaced the old open-air/column audit gates with the shell audit and removed the no-long-bridge and rock-column gates, which measured the deleted support mechanism. Walkability, capsule, headroom, grade, mouth, naturalness, and ripple gates remain. Floor-ripple probes fully masked by another passage's interior air are skipped only when the overlap is explicit; capsule walkability still has to pass.
Kept the native Lattice field's explicitly authored below-floor substrate because its spine needs it for walkability; this is part of that native field, not route support restoration. The legacy landmark demo retains its compatibility floor-cap path, which is separate from the resolved route sweep. Native Caves and Lattice fast checks pass.
## Performance
Five interleaved L80 pairs per cell, after one warmup for each binary; seconds are median `LOD0–2 near-ready / whole-view-ready`. Baseline binary is from `assembler/world` HEAD. Tile counts matched within each cell.
| Strate | W8 near | W8 whole | W12 near | W12 whole |
|---|---:|---:|---:|---:|
| SW | 0.680 → 0.737 (+8.3%) | 1.686 → 1.674 (−0.7%) | 0.700 → 0.761 (+8.8%) | 1.643 → 1.644 (+0.1%) |
| FI | 1.455 → 0.911 (−37.4%) | 2.419 → 1.772 (−26.8%) | 1.394 → 0.863 (−38.0%) | 2.288 → 1.664 (−27.3%) |
| TN | 1.423 → 0.956 (−32.8%) | 2.310 → 1.778 (−23.0%) | 1.294 → 0.895 (−30.9%) | 2.132 → 1.661 (−22.1%) |
| VS | 1.008 → 0.789 (−21.8%) | 1.917 → 1.639 (−14.5%) | 0.854 → 0.738 (−13.5%) | 1.716 → 1.541 (−10.2%) |
There is no broad regression. SW near-ready is about 0.06 s slower; its whole-view time is flat. **Measured:** SW LOD0–2 skirt triangles rise from 133,894 to 196,602, and summed skirt work rises from 455 to 743 ms at W8 and 650 to 1,044 ms at W12. **Inference:** the added open-air shell surface and skirts explain the small SW near-ready cost. FI, TN, and VS improve at both readiness marks.
## L77 phase census
Two samples per case. Entries are `share / estimated worker-s`; estimates use weighted L77 samples times `WorldAssemblerLODWorker` CPU, so they are not direct phase timers.
| Case | Base passage | Base support | Candidate passage | Candidate support |
|---|---:|---:|---:|---:|
| FI near | 44.62% / 4.108 | 14.84% / 1.366 | 22.90% / 1.133 | 0% / 0 |
| FI whole | 29.31% / 4.874 | 8.08% / 1.344 | 10.59% / 1.206 | 0% / 0 |
| TN near | 38.57% / 3.318 | 14.74% / 1.267 | 19.43% / 1.019 | 0% / 0 |
| TN whole | 24.48% / 3.824 | 7.59% / 1.186 | 9.25% / 1.074 | 0% / 0 |
The sampled support share disappears. Baseline estimates were 1.19–1.37 worker-s across these cases, below the 1.6–1.9 s expectation.
## Audits and determinism
The full-view all-LOD audit passes: 21,425/21,425 canonical passage centreline samples are open. All three interfaces pass capsule and support checks with zero blocked or unsupported samples, zero grade failures, and both mouths bound (2/2). Capsule samples: 266,175 (0–1), 15,120 (1–2), 64,764 (2–3). The shell audit sampled 89,365 floor points, 8,423 wall points, and 10,527 interior points: zero floor, wall, or interior violations; 22,755 open-air shell samples. Naturalness, mouth profile, and ripple summaries pass 3/3, with negative controls detected. Derived floor grade peaks at 14.49°; fast native Caves headroom is 0.30 m and native Lattice headroom is 2.20 m.
Fast tier: PASS, 259 PASS checks plus 5 diagnostics across 80 runs. The Clang/GCC hash matrix matches at W1/W8/W12 for the owner four-strata layout and both two-strata canonical recipes. The v217 canonical pins were written for Surface World style 1 and Floating Islands style 2.
Because Core sources are Unreal-compiled, the Linux VFHostEditor build passed with `-NoUBA -MaxParallelActions=4` (27 actions, 67.4 s). Three FI and three SW `after` arrival traces also passed, each settling its full requested LOD0–2 set. Unreal processes had no `prlimit`.
## Change metrics
LOD0–2 camera mesh windows; “moved” means a shared global lattice crossing displaced by more than 0.01 cm. Coverage is the paired edge-classifier coverage.
| Strate | Added | Removed | Moved | Coverage |
|---|---:|---:|---:|---:|
| SW | 78,201 | 14,530 | 2,758 | 99.75% |
| FI | 37,810 | 42,398 | 78,215 | 98.31% |
| TN | 44,694 | 48,925 | 48,816 | 98.29% |
| VS | 7,055 | 7,096 | 8,790 | 97.38% |
## Captures
84 PNGs: before and after, each with both mouths and 25/50/75% stations for all three passages. Each of the nine open-air sections has three views in each set. The [capture index](captures/INDEX.md) lists every image and flags every **OPEN AIR** view. All 84 source meshes and rendered views contain geometry; minimum visible triangles are 55,420 before and 55,484 after. Six before and seven after images met the pixel floor but were flagged low contrast for review.
Open-air sections (route distance in metres): `0–1`: 7.226–10.217, 13.456–17.443, 23.174–24.918, 1036.913–1489.321; `1–2`: 101.820–104.499; `2–3`: 7.677–34.747, 368.909–376.660, 393.828–396.510, 444.932–447.654.
## Failed runs
Early fast runs found a 0.79 m lower-mouth mismatch at native Caves and loss of the native Lattice route substrate; the mouth now matches the authored cave edge floor, and the explicitly authored Lattice floor field is retained. A sentinel check also initially treated the “no native floor” value as support; this was fixed. The first Unreal build stopped before compilation because UnrealBuildTool could not rotate its log under read-only `~/.config`; redirecting `XDG_CONFIG_HOME` to `/tmp` fixed it. Final audits, timings, hash runs, crossings, captures, build, and six traces passed. No run exceeded 30 minutes.
@@ -0,0 +1,511 @@
#!/usr/bin/env python3
"""Capture before/after passage mouths, route stations, and exposed shell sections."""
from __future__ import annotations
import json
import math
import re
from array import array
import struct
import subprocess
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
RESEARCH = Path(__file__).resolve().parent
CAPTURES = RESEARCH / "captures"
SAVED = ROOT / "Saved/Linux/l109/captures"
DATA = ROOT / "Standalone/data/owner_world.json"
BINARIES = {
"before": ROOT / "Saved/Linux/l109/bin-baseline/VoxelForgeCoreHeadless-l109-baseline",
"after": ROOT / "Saved/Linux/l109/bin-final/VoxelForgeCoreHeadless-l109-clang",
}
LAYOUTS = {
"before": ROOT / "Saved/Linux/l109/audit-layout-baseline.json",
"after": ROOT / "Saved/Linux/l109/audit-layout-final-stamp.json",
}
AUDIT_LOG = ROOT / "Saved/Linux/l109/audit-owner4-final-stamp.log"
RENDERERS = {
"before": ROOT / "Saved/Linux/l109/bin-baseline/VoxelForgeOffscreenCapture-l109-baseline",
"after": ROOT / "Saved/Linux/l109/bin-final/VoxelForgeOffscreenCapture-l109-clang",
}
SECTION = re.compile(
r"^\[PassageOpenAirSection\] id=(\S+) section=(\d+) start_m=([\d.]+) "
r"end_m=([\d.]+) samples=(\d+) midpoint_xyz_m=\(([^)]*)\) "
r"midpoint_radius_m=([\d.]+)", re.M)
DRAWN = re.compile(r"drawn_triangles=(\d+)")
QUALITY_METRICS = re.compile(r"stddev_luma=([\d.]+).*coverage_luma_gt12=([\d.]+)")
def vector(text: str) -> tuple[float, float, float]:
return tuple(float(value) for value in text.split(","))
def add(a, b):
return tuple(x + y for x, y in zip(a, b))
def sub(a, b):
return tuple(x - y for x, y in zip(a, b))
def mul(a, scale):
return tuple(x * scale for x in a)
def norm(a):
length = math.sqrt(sum(x * x for x in a))
if length <= 1.0e-9:
return (0.0, 1.0, 0.0)
return tuple(value / length for value in a)
def angles(camera, target):
delta = sub(target, camera)
horizontal = math.hypot(delta[0], delta[1])
return (math.degrees(math.atan2(delta[0], delta[1])),
math.degrees(math.atan2(delta[2], horizontal)))
def fmt(values) -> str:
return ",".join(f"{value:.6f}" for value in values)
def mesh_triangle_count(mesh: Path) -> int:
"""Read only the VFMSH headers and count indexed triangles."""
with mesh.open("rb") as stream:
magic = stream.read(8)
materials = magic == b"VFMSH002"
if magic not in (b"VFMSH001", b"VFMSH002"):
raise RuntimeError(f"unsupported mesh scene header: {mesh}")
version, _voxel_um, tile_count = struct.unpack("<3I", stream.read(12))
if version != (2 if materials else 1):
raise RuntimeError(f"mesh scene version mismatch: {mesh}")
stream.seek(40, 1)
triangles = 0
for _ in range(tile_count):
stream.seek(16, 1)
surface_count, _skirts, vertex_count, index_count, mask_count = struct.unpack(
"<5I", stream.read(20))
triangles += index_count // 3
stream.seek(vertex_count * 24 + (10 * vertex_count if materials else 0)
+ index_count * 4 + mask_count, 1)
if stream.read(1):
raise RuntimeError(f"trailing data in mesh scene: {mesh}")
return triangles
def dot(a, b):
return sum(x * y for x, y in zip(a, b))
def nearest_surface_anchor(mesh: Path, point, margin_m=200.0) -> dict:
"""Return a nearby VFMSH surface vertex and its normal."""
best_distance_sq = float("inf")
best_position = best_normal = None
with mesh.open("rb") as stream:
magic = stream.read(8)
materials = magic == b"VFMSH002"
if magic not in (b"VFMSH001", b"VFMSH002"):
raise RuntimeError(f"unsupported mesh scene header: {mesh}")
version, voxel_um, tile_count = struct.unpack("<3I", stream.read(12))
if version != (2 if materials else 1):
raise RuntimeError(f"mesh scene version mismatch: {mesh}")
stream.seek(40, 1)
for _ in range(tile_count):
key = struct.unpack("<4i", stream.read(16))
surface_count, _skirts, vertex_count, index_count, mask_count = struct.unpack(
"<5I", stream.read(20))
extent = (32 << key[3]) * voxel_um / 1_000_000.0
origin = (key[0] * extent, key[1] * extent, key[2] * extent)
if any(point[axis] < origin[axis] - margin_m or
point[axis] > origin[axis] + extent + margin_m for axis in range(3)):
stream.seek(vertex_count * 24 + (10 * vertex_count if materials else 0)
+ index_count * 4 + mask_count, 1)
continue
positions = array("f")
positions.frombytes(stream.read(vertex_count * 12))
normals = array("f")
normals.frombytes(stream.read(vertex_count * 12))
if materials:
stream.seek(vertex_count * 10, 1)
stream.seek(index_count * 4 + mask_count, 1)
if struct.pack("=I", 1) != struct.pack("<I", 1):
positions.byteswap()
normals.byteswap()
for index in range(min(surface_count, vertex_count)):
offset = index * 3
position = tuple(float(positions[offset + axis]) for axis in range(3))
distance_sq = dot(sub(position, point), sub(position, point))
if distance_sq >= best_distance_sq:
continue
raw_normal = tuple(float(normals[offset + axis]) for axis in range(3))
if dot(raw_normal, raw_normal) < 1.0e-10:
continue
best_distance_sq = distance_sq
best_position = position
best_normal = norm(raw_normal)
if best_position is None:
raise RuntimeError(f"no nearby surface vertices in mesh scene: {mesh}")
return {"position_m": best_position, "normal": best_normal,
"distance_m": math.sqrt(best_distance_sq)}
def sanitize(value: str) -> str:
return re.sub(r"[^A-Za-z0-9_-]+", "-", value).strip("-").replace("/", "-")
def route_for(layout: dict, passage: dict) -> list[tuple[tuple[float, float, float], float]]:
scale = layout["core_voxel_size_um"] / 256.0 / 1_000_000.0
controls = [(tuple(value * scale for value in row["position_q"]),
row["radius_q"] * scale) for row in passage["control_points"]]
lower_q = passage["lower_mouth_q"]
player_center_above_floor_q = round(
1.13 * 256.0 * 1_000_000.0 / layout["core_voxel_size_um"])
lower_index = next((index for index, row in enumerate(passage["control_points"])
if row["position_q"][0] == lower_q[0]
and row["position_q"][1] == lower_q[1]
and row["position_q"][2] - row["radius_q"]
+ player_center_above_floor_q == lower_q[2]), None)
if lower_index is None:
raise RuntimeError(f"{passage['stable_id']}: lower mouth control is unbound")
result = controls[:lower_index + 1]
if len(result) < 2:
raise RuntimeError(f"{passage['stable_id']}: incomplete route controls")
return result
def route_lengths(route):
lengths = [math.dist(a[0], b[0]) for a, b in zip(route, route[1:])]
prefix = [0.0]
for length in lengths:
prefix.append(prefix[-1] + length)
return lengths, prefix
def at_distance(route, lengths, prefix, distance):
distance = max(0.0, min(distance, prefix[-1]))
for index, length in enumerate(lengths):
if distance <= prefix[index + 1] or index + 1 == len(lengths):
t = 0.0 if length <= 1.0e-9 else (distance - prefix[index]) / length
a, b = route[index], route[index + 1]
return (tuple(a[0][axis] + (b[0][axis] - a[0][axis]) * t
for axis in range(3)),
a[1] + (b[1] - a[1]) * t)
return route[-1]
def tangent_at(route, lengths, prefix, distance):
before = at_distance(route, lengths, prefix, max(0.0, distance - 0.25))[0]
after = at_distance(route, lengths, prefix, min(prefix[-1], distance + 0.25))[0]
return norm(sub(after, before))
def derived_route(route, passage, layout):
"""Apply L109's fixed-depth floor offset to raw layout controls."""
lengths, prefix = route_lengths(route)
total = prefix[-1]
body_radii = [radius for index, (_, radius) in enumerate(route)
if prefix[index] >= 10.0 and total - prefix[index] >= 10.0]
body_depth = max(0.5, min(body_radii or [radius for _, radius in route]) - 0.23)
scale = layout["core_voxel_size_um"] / 256.0 / 1_000_000.0
upper_z = passage["upper_mouth_q"][2] * scale
lower_z = passage["lower_mouth_q"][2] * scale
upper_target = max(0.0, route[0][0][2] - upper_z + 1.13)
lower_target = max(0.0, route[-1][0][2] - lower_z + 1.13)
transformed = []
for index, (point, radius) in enumerate(route):
local_radius_depth = max(0.0, radius - 0.23)
mouth_distance = max(0.0, min(prefix[index], total - prefix[index]))
u = max(0.0, min(1.0, mouth_distance / 5.0))
ramp = u * u * (3.0 - 2.0 * u)
target = upper_target if prefix[index] <= total * 0.5 else lower_target
mouth_depth = max(0.0, min(radius, target))
route_depth = mouth_depth + (local_radius_depth - mouth_depth) * ramp
tube_depth = min(body_depth, local_radius_depth)
transformed.append(((point[0], point[1], point[2] - route_depth + tube_depth),
radius))
return transformed, body_depth
def make_views(layout: dict, audit_sections: list[dict], *, use_derived_shape: bool,
include_open_air: bool = True) -> list[dict]:
views = []
passages = sorted(layout["passages"], key=lambda item: item["stable_id"])
for passage in passages:
raw_route = route_for(layout, passage)
if use_derived_shape:
route, body_depth = derived_route(raw_route, passage, layout)
else:
route = raw_route
body_depth = max(point[1] for point in route)
def floor_depth(radius: float) -> float:
if not use_derived_shape:
return radius
return min(body_depth, max(0.0, radius - 0.23))
lengths, prefix = route_lengths(route)
total = prefix[-1]
passage_id = passage["stable_id"]
safe_id = sanitize(passage_id)
for end_name, distance, outward_sign in (
("upper-mouth", 0.0, -1.0), ("lower-mouth", total, 1.0)):
center, radius = at_distance(route, lengths, prefix, distance)
tangent = tangent_at(route, lengths, prefix, distance)
outward = mul(tangent, outward_sign)
floor_z = center[2] - floor_depth(radius)
eye = (center[0] + outward[0] * 4.5,
center[1] + outward[1] * 4.5,
floor_z + 1.55)
if end_name == "lower-mouth":
eye = add(eye, add(mul(outward, 5.0), (0.0, 0.0, 2.0)))
target = (center[0] - outward[0] * 2.0,
center[1] - outward[1] * 2.0,
floor_z + 1.45)
yaw, pitch = angles(eye, target)
views.append({"name": f"{safe_id}-{end_name}", "passage": passage_id,
"kind": "mouth", "station": end_name,
"camera": eye, "target": target, "yaw": yaw, "pitch": pitch,
"focus": center, "center_z": center[2]})
for fraction in (0.25, 0.50, 0.75):
distance = total * fraction
center, radius = at_distance(route, lengths, prefix, distance)
tangent = tangent_at(route, lengths, prefix, distance)
depth = floor_depth(radius)
floor_z = center[2] - depth
horizontal = norm((-tangent[1], tangent[0], 0.0))
eye = (center[0] + horizontal[0] * 0.75,
center[1] + horizontal[1] * 0.75, floor_z + 1.60)
target_center, target_radius = at_distance(route, lengths, prefix,
min(total, distance + 14.0))
target_depth = floor_depth(target_radius)
target = (target_center[0] + horizontal[0] * 2.35,
target_center[1] + horizontal[1] * 2.35,
target_center[2] - target_depth + 1.80)
yaw, pitch = angles(eye, target)
views.append({"name": f"{safe_id}-station-{int(fraction * 100)}",
"passage": passage_id, "kind": "station",
"station": f"{int(fraction * 100)}%", "camera": eye,
"target": target, "yaw": yaw, "pitch": pitch,
"focus": center, "center_z": center[2],
"route_distance_m": distance,
"route_length_m": total,
"requested_route_distance_m": distance})
if not include_open_air:
continue
for section in (item for item in audit_sections if item["passage"] == passage_id):
distance = 0.5 * (section["start_m"] + section["end_m"])
center, radius = (section["midpoint"], section["radius_m"])
tangent = tangent_at(route, lengths, prefix, distance)
side = norm((-tangent[1], tangent[0], 0.0))
depth = floor_depth(radius)
section_id = section["section"]
interior_eye = (center[0], center[1], center[2] - depth + 1.25)
interior_target = add(interior_eye, mul(tangent, 7.0))
side_eye = add(center, add(mul(side, radius + 5.0), (0.0, 0.0, 1.2)))
oblique_eye = add(center, add(mul(side, -(radius + 3.5)), (0.0, 0.0, 4.0)))
targets = (add(center, (0.0, 0.0, 0.1)),
add(center, mul(tangent, 2.0)), interior_target)
for view_index, (eye, target) in enumerate(zip(
(side_eye, oblique_eye, interior_eye), targets), start=1):
yaw, pitch = angles(eye, target)
views.append({"name": f"{safe_id}-openair-{section_id}-view-{view_index}",
"passage": passage_id, "kind": "open-air",
"station": f"section-{section_id}/view-{view_index}",
"camera": eye, "target": target, "yaw": yaw,
"pitch": pitch, "focus": center,
"center_z": center[2], "open_air_section": section_id,
"route_distance_m": distance,
"requested_route_distance_m": distance,
"route_length_m": total})
return views
def run_process(name: str, command: list[str], log: Path, *, allow_failure=False) -> str:
wrapped = ["taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
"timeout", "1740s", *map(str, command)]
started = time.monotonic()
with log.open("w", encoding="utf-8") as stream:
result = subprocess.run(wrapped, cwd=ROOT, stdout=stream,
stderr=subprocess.STDOUT, check=False, timeout=1740)
if result.returncode and not allow_failure:
raise RuntimeError(f"{name}: exit={result.returncode} log={log}")
print(f"[L109CaptureProcess] name={name} elapsed_s={time.monotonic()-started:.2f} "
f"exit={result.returncode} log={log.relative_to(ROOT)}", flush=True)
return log.read_text(encoding="utf-8", errors="replace")
def capture(variant: str, view: dict, layout: dict) -> dict:
variant_saved = SAVED / variant
variant_saved.mkdir(parents=True, exist_ok=True)
variant_capture = CAPTURES / variant
variant_capture.mkdir(parents=True, exist_ok=True)
scene = variant_saved / f"{view['name']}.vfmesh"
image = variant_capture / f"{view['name']}.png"
candidates = [(view["camera"], view["target"], view["focus"],
view.get("route_distance_m"), 0.0, "requested")]
if variant == "before" and view["kind"] != "mouth":
passage = next(item for item in layout["passages"]
if item["stable_id"] == view["passage"])
route = route_for(layout, passage)
lengths, prefix = route_lengths(route)
total = prefix[-1]
eye_offset = sub(view["camera"], view["focus"])
target_offset = sub(view["target"], view["focus"])
requested = view["route_distance_m"]
for delta in (5, -5, 10, -10, 20, -20, 35, -35, 50, -50, 75, -75,
100, -100, 150, -150, 200, -200):
distance = max(0.0, min(total, requested + delta))
if abs(distance - requested) < 0.01:
continue
center, _radius = at_distance(route, lengths, prefix, distance)
candidates.append((add(center, eye_offset), add(center, target_offset), center,
distance, distance - requested, "route-offset"))
attempt_errors = []
for attempt_index, (probe_camera, probe_target, probe_focus, route_distance,
route_adjustment, attempt_kind) in enumerate(candidates):
suffix = "" if attempt_index == 0 else f"-attempt-{attempt_index:02d}"
headless_log = variant_saved / f"{view['name']}{suffix}-mesh.log"
render_log = variant_saved / f"{view['name']}{suffix}-render.log"
quality_log = variant_saved / f"{view['name']}{suffix}-quality.log"
yaw, pitch = angles(probe_camera, probe_target)
camera = (*probe_camera, yaw, pitch)
export_command = [
str(BINARIES[variant]), str(DATA), "--world-assembler-prototype",
"--stream-benchmark", "--stream-pipeline", "--workers=12",
"--clip-radius=3", "--max-clip-level=0", "--max-level=0", "--force-generate",
"--recipe=surface-world", f"--camera={fmt(camera)}",
"--render-distance-chunks=1", f"--mesh-export={scene}",
]
export_output = run_process(f"{variant}-{view['name']}{suffix}-mesh", export_command,
headless_log)
if "status=PASS" not in export_output or not scene.is_file():
raise RuntimeError(f"{variant}-{view['name']}: missing successful scene export")
source_triangles = mesh_triangle_count(scene)
if source_triangles <= 0:
attempt_errors.append(f"{attempt_kind}@{route_distance}: empty mesh")
if variant == "after" or view["kind"] == "mouth":
break
continue
actual_camera, actual_target = probe_camera, probe_target
actual_focus = probe_focus
surface_distance = None
if attempt_kind == "route-offset":
anchor = nearest_surface_anchor(scene, probe_focus)
surface_distance = anchor["distance_m"]
actual_camera = add(anchor["position_m"], mul(anchor["normal"], 12.0))
actual_target = anchor["position_m"]
actual_focus = anchor["position_m"]
actual_yaw, actual_pitch = angles(actual_camera, actual_target)
render_command = [
str(RENDERERS[variant]), str(scene), str(image), "--size=1920x1080",
f"--camera-m={fmt(actual_camera)}", f"--yaw-deg={actual_yaw:.6f}",
f"--pitch-deg={actual_pitch:.6f}", "--shade=readable", "--headlamp=on",
"--height-tint=on", f"--height-center-m={actual_focus[2]:.6f}",
"--height-range-m=32",
]
render_output = run_process(f"{variant}-{view['name']}{suffix}-render", render_command,
render_log)
drawn = DRAWN.search(render_output)
if drawn is None or int(drawn.group(1)) <= 0 or not image.is_file():
attempt_errors.append(f"{attempt_kind}@{route_distance}: no drawn geometry")
if variant == "after" or view["kind"] == "mouth":
break
continue
quality = run_process(f"{variant}-{view['name']}{suffix}-quality", [
"python3", "-B", str(ROOT / "Standalone/Tools/verify/capture_quality.py"),
str(image)], quality_log, allow_failure=True)
quality_status = "pass"
if "status=PASS" not in quality:
metrics = QUALITY_METRICS.search(quality)
coverage = float(metrics.group(2)) if metrics else 0.0
if coverage >= 0.001:
quality_status = "low_contrast_geometry_visible"
else:
attempt_errors.append(f"{attempt_kind}@{route_distance}: no visible pixel coverage")
if variant == "after" or view["kind"] == "mouth":
break
continue
row = {**view, "variant": variant, "capture": str(image.relative_to(RESEARCH)),
"scene": str(scene.relative_to(ROOT)), "drawn_triangles": int(drawn.group(1)),
"source_triangles": source_triangles, "verified": True,
"pixel_quality_status": quality_status,
"route_distance_actual_m": route_distance,
"route_station_adjustment_m": route_adjustment,
"fallback_kind": attempt_kind if attempt_kind != "requested" else None,
"fallback_surface_distance_m": surface_distance,
"camera_actual_m": actual_camera, "target_actual_m": actual_target,
"logs": [str(headless_log.relative_to(ROOT)),
str(render_log.relative_to(ROOT)),
str(quality_log.relative_to(ROOT))]}
print("[L109Capture] " + json.dumps(row, sort_keys=True), flush=True)
return row
raise RuntimeError(f"{variant}-{view['name']}: no verified image; "
+ "; ".join(attempt_errors))
def main() -> int:
if any(not path.is_file() for path in (*BINARIES.values(), *RENDERERS.values(),
*LAYOUTS.values(), AUDIT_LOG)):
raise SystemExit("missing L109 capture binary, layout, or audit log")
sections = []
for passage, section, start, end, samples, midpoint, radius in SECTION.findall(
AUDIT_LOG.read_text(encoding="utf-8", errors="replace")):
sections.append({"passage": passage, "section": int(section),
"start_m": float(start), "end_m": float(end),
"samples": int(samples), "midpoint": vector(midpoint),
"radius_m": float(radius)})
if not sections:
raise SystemExit("no open-air sections found in the final passage audit")
all_rows = []
for variant in ("before", "after"):
layout = json.loads(LAYOUTS[variant].read_text(encoding="utf-8"))
include_open_air = True
views = make_views(layout, sections, use_derived_shape=variant == "after",
include_open_air=include_open_air)
expected = 5 * len(layout["passages"])
if include_open_air:
expected += 3 * len(sections)
if len(views) != expected:
raise RuntimeError(f"unexpected camera count: {len(views)} != {expected}")
print(f"[L109CapturePlan] variant={variant} passage_views={5*len(layout['passages'])} "
f"open_air_sections={len(sections) if include_open_air else 0} "
f"open_air_views={3*len(sections) if include_open_air else 0} "
f"total={len(views)}", flush=True)
for view in views:
all_rows.append(capture(variant, view, layout))
CAPTURES.mkdir(parents=True, exist_ok=True)
(CAPTURES / "capture-index.json").write_text(
json.dumps({"open_air_sections": sections, "captures": all_rows}, indent=2) + "\n",
encoding="utf-8")
lines = ["# L109 passage captures", "",
"Each PNG was rendered from its own exported mesh. The renderer reported visible "
"triangles, and each decoded image met the pixel coverage floor. "
"`low_contrast_geometry_visible` marks images that met this floor below the "
"preferred contrast target.", ""]
for variant in ("before", "after"):
lines.extend([f"## {variant.title()}", ""])
for row in all_rows:
if row["variant"] != variant:
continue
flag = " **OPEN AIR**" if row["kind"] == "open-air" else ""
quality_note = (f"; {row['pixel_quality_status']}"
if row["pixel_quality_status"] != "pass" else "")
lines.append(f"- [{row['station']}]({variant}/{Path(row['capture']).name}) — "
f"{row['passage']}{flag}; {row['drawn_triangles']} visible triangles"
f"{quality_note}")
lines.append("")
(CAPTURES / "INDEX.md").write_text("\n".join(lines), encoding="utf-8")
failures = [row for row in all_rows if not row["verified"]]
print(f"[L109CaptureSummary] captures={len(all_rows)} verified={len(all_rows)-len(failures)} "
f"failed={len(failures)} open_air_sections={len(sections)}", flush=True)
return 1 if failures else 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,140 @@
#!/usr/bin/env python3
"""Compare LOD0-2 owner mesh edge crossings before and after L109."""
from __future__ import annotations
import json
import math
import statistics
import struct
import subprocess
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
OUT = ROOT / "Saved/Linux/l109/crossings"
DATA = ROOT / "Standalone/data/owner_world.json"
BINARIES = {
"before": ROOT / "Saved/Linux/l109/bin-baseline/VoxelForgeCoreHeadless-l109-baseline",
"after": ROOT / "Saved/Linux/l109/bin-final/VoxelForgeCoreHeadless-l109-clang",
}
CAMERAS = {
"SW": (8.0, 16.0, -372.2626953125),
"FI": (25.8369140625, 3.19921875, -737.0),
"TN": (19.166015625, -14.2001953125, -753.3740234375),
"VS": (-20.0625, -5.068359375, -900.353515625),
}
def read_exact(stream, size):
data = stream.read(size)
if len(data) != size:
raise RuntimeError("truncated VFMSH export")
return data
def crossing_map(path: Path) -> tuple[dict, dict]:
crossings = {}
meta = {"tiles": 0, "surface_vertices": 0, "classified": 0}
with path.open("rb") as stream:
magic = read_exact(stream, 8)
materials = magic == b"VFMSH002"
if magic not in (b"VFMSH001", b"VFMSH002"):
raise RuntimeError(f"invalid mesh export {path}")
version, voxel_um, tile_count = struct.unpack("<3I", read_exact(stream, 12))
if version != (2 if materials else 1):
raise RuntimeError("VFMSH version mismatch")
read_exact(stream, 40)
voxel_m = voxel_um / 1_000_000.0
for _ in range(tile_count):
key = struct.unpack("<4i", read_exact(stream, 16))
surface_count, _skirts, vertex_count, index_count, mask_count = struct.unpack(
"<5I", read_exact(stream, 20))
positions = struct.iter_unpack("<3f", read_exact(stream, vertex_count * 12))
values = list(positions)
read_exact(stream, vertex_count * 12) # normals
if materials:
read_exact(stream, vertex_count * 10)
read_exact(stream, index_count * 4 + mask_count)
meta["tiles"] += 1
meta["surface_vertices"] += min(surface_count, vertex_count)
unit = voxel_m * (1 << key[3])
for position in values[:min(surface_count, vertex_count)]:
grid = tuple(value / unit for value in position)
fractions = tuple(abs(value - round(value)) for value in grid)
axis = max(range(3), key=fractions.__getitem__)
if fractions[axis] < 1.0e-4:
continue
fixed = [index for index in range(3) if index != axis]
if any(fractions[index] > 0.01 for index in fixed):
continue
edge = [0, 0, 0]
for index in fixed:
edge[index] = int(round(grid[index]))
edge[axis] = math.floor(grid[axis])
identity = (key[3], axis, *edge)
crossings.setdefault(identity, tuple(float(v) for v in position))
meta["classified"] += 1
if stream.read(1):
raise RuntimeError("trailing data in mesh export")
meta["classified_pct"] = (100.0 * meta["classified"] / meta["surface_vertices"]
if meta["surface_vertices"] else 0.0)
return crossings, meta
def export(strate: str, variant: str) -> Path:
OUT.mkdir(parents=True, exist_ok=True)
mesh = OUT / f"{strate.lower()}-{variant}.vfmesh"
log = OUT / f"{strate.lower()}-{variant}.log"
camera = ",".join(f"{v:.12f}" for v in CAMERAS[strate]) + ",0,0"
command = [
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
"timeout", "1740s", str(BINARIES[variant]), str(DATA),
"--world-assembler-prototype", "--stream-benchmark", "--stream-pipeline",
"--workers=8", "--clip-radius=3", "--max-clip-level=2", "--max-level=2",
"--force-generate", "--recipe=surface-world", f"--camera={camera}",
"--render-distance-chunks=0",
f"--mesh-export={mesh}",
]
started = time.monotonic()
with log.open("w", encoding="utf-8") as stream:
result = subprocess.run(command, cwd=ROOT, stdout=stream,
stderr=subprocess.STDOUT, timeout=1740, check=False)
output = log.read_text(encoding="utf-8", errors="replace")
if result.returncode or not mesh.is_file() or "status=PASS" not in output:
raise RuntimeError(f"mesh export failed: {strate}/{variant} exit={result.returncode} {log}")
return mesh
def main() -> int:
summaries = []
for strate in CAMERAS:
paths = {variant: export(strate, variant) for variant in ("before", "after")}
maps = {variant: crossing_map(path) for variant, path in paths.items()}
old, old_meta = maps["before"]
new, new_meta = maps["after"]
old_keys, new_keys = set(old), set(new)
shared = old_keys & new_keys
moved_cm = [100.0 * math.dist(old[key], new[key]) for key in shared
if math.dist(old[key], new[key]) > 0.0001]
row = {
"strate": strate, "scope": "camera LOD0-2 mesh window; global lattice edge IDs",
"added_crossings": len(new_keys - old_keys),
"removed_crossings": len(old_keys - new_keys),
"shared_crossings": len(shared), "moved_crossings": len(moved_cm),
"mean_moved_cm": statistics.mean(moved_cm) if moved_cm else 0.0,
"max_moved_cm": max(moved_cm, default=0.0),
"before": old_meta, "after": new_meta,
"classified_pair_coverage_pct": min(old_meta["classified_pct"],
new_meta["classified_pct"]),
"exports": {key: str(value.relative_to(ROOT)) for key, value in paths.items()},
}
summaries.append(row)
print("[L109CrossingDelta] " + json.dumps(row, sort_keys=True), flush=True)
(OUT / "summary.json").write_text(
json.dumps({"measurement": "per-vertex marching-cubes edge classifier; compare shared global lattice edges; positions >0.01cm count as moved",
"strata": summaries}, indent=2) + "\n", encoding="utf-8")
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,103 @@
#!/usr/bin/env python3
"""Check Clang/GCC hash identity at W1/W8/W12 and repin v217 owner hashes."""
from __future__ import annotations
import json
import re
import subprocess
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
OUT = ROOT / "Saved/Linux/l109/hashes"
DATASETS = {
"owner4": ROOT / "Standalone/data/owner_world.json",
"owner2": ROOT / "Standalone/data/regression/owner_world_2strate_l61.json",
}
BINARIES = {
"clang": ROOT / "Saved/Linux/l109/bin-final/VoxelForgeCoreHeadless-l109-clang",
"gcc": ROOT / "Saved/Linux/l109/bin-final/VoxelForgeCoreHeadless-l109-gcc",
}
HASH = re.compile(r"^\[Pipeline(?:FullDetailHash|WholeViewHash|LODHash|ExactQueryHash)\].*$", re.M)
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=(\S+)$", re.M)
def run(dataset: str, recipe: str, workers: int, compiler: str) -> dict:
OUT.mkdir(parents=True, exist_ok=True)
label = f"{dataset}-{recipe}-w{workers}-{compiler}"
log = OUT / f"{label}.log"
layout = OUT / f"{label}.layout.json"
command = [
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
"timeout", "1740s", str(BINARIES[compiler]), str(DATASETS[dataset]),
f"--workers={workers}", "--world-assembler-prototype", "--stream-benchmark",
"--stream-pipeline", "--force-generate", "--clip-radius=3",
"--max-clip-level=8", "--max-level=8", f"--recipe={recipe}",
"--stream-hash", f"--resolved-layout-output={layout}",
]
started = time.monotonic()
with log.open("w", encoding="utf-8") as stream:
result = subprocess.run(command, cwd=ROOT, stdout=stream,
stderr=subprocess.STDOUT, check=False, timeout=1740)
output = log.read_text(encoding="utf-8", errors="replace")
hashes = HASH.findall(output)
if result.returncode or len(hashes) != 12:
raise SystemExit(f"hash run failed or incomplete: {label}; exit={result.returncode} "
f"hashes={len(hashes)} log={log}")
row = {"dataset": dataset, "recipe": recipe, "workers": workers,
"compiler": compiler, "exit_code": result.returncode,
"hashes": hashes, "source_stamp": (STAMP.search(output).group(1)
if STAMP.search(output) else None),
"elapsed_s": round(time.monotonic() - started, 3),
"log": str(log.relative_to(ROOT))}
print("[L109HashRun] " + json.dumps(row, sort_keys=True), flush=True)
return row
def main() -> int:
if any(not path.is_file() for path in (*DATASETS.values(), *BINARIES.values())):
raise SystemExit("missing L109 hash input")
rows = []
recipes = {"owner4": ("surface-world",),
"owner2": ("surface-world", "floating-islands")}
for dataset, dataset_recipes in recipes.items():
for recipe in dataset_recipes:
for workers in (1, 8, 12):
for compiler in ("clang", "gcc"):
rows.append(run(dataset, recipe, workers, compiler))
(OUT / "runs-partial.json").write_text(
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
groups = {}
for dataset, dataset_recipes in recipes.items():
for recipe in dataset_recipes:
key = f"{dataset}/{recipe}"
group = [row for row in rows if row["dataset"] == dataset
and row["recipe"] == recipe]
reference = group[0]["hashes"]
groups[key] = {"all_worker_compiler_hashes_match": all(
row["hashes"] == reference for row in group),
"source_stamps": sorted({row["source_stamp"] for row in group}),
"runs": [{"workers": row["workers"], "compiler": row["compiler"],
"log": row["log"]} for row in group]}
if not groups[key]["all_worker_compiler_hashes_match"]:
raise SystemExit(f"determinism mismatch: {key}")
if dataset == "owner2":
canonical = next(row for row in group if row["workers"] == 12
and row["compiler"] == "clang")
recipe_name = "floating-islands" if recipe == "floating-islands" else "surface-world"
style = 1 if recipe_name == "surface-world" else 2
baseline = (ROOT / "Standalone/Tools/verify/baselines/owner-hash/canonical"
/ f"owner-hash-{recipe_name}-style{style}-v217.baseline")
baseline.write_text("\n".join(canonical["hashes"]) + "\n",
encoding="utf-8")
summary = {"runs": rows, "groups": groups,
"owner4_all_four_strata": "owner4/surface-world",
"owner2_canonical_pins": ["owner2/surface-world",
"owner2/floating-islands"]}
(OUT / "summary.json").write_text(json.dumps(summary, indent=2) + "\n",
encoding="utf-8")
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,112 @@
#!/usr/bin/env python3
"""Estimate L77 passage/support worker shares before and after L109."""
from __future__ import annotations
import json
import re
import statistics
import subprocess
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
OUT = ROOT / "Saved/Linux/l109/l77-phase-census"
DATA = ROOT / "Standalone/data/owner_world.json"
BINARIES = {
"base": ROOT / "Saved/Linux/l109/bin-baseline-profile/VoxelForgeCoreHeadless-l109-base-profile",
"candidate": ROOT / "Saved/Linux/l109/bin-final-profile/VoxelForgeCoreHeadless-l109-candidate-profile",
}
CAMERAS = {
"FI": (25.8369140625, 3.19921875, -737.0),
"TN": (19.166015625, -14.2001953125, -753.3740234375),
}
WORKER = re.compile(r"^\[WorldAssemblerLODWorker\] (.*)$", re.M)
SAMPLE = re.compile(r"^\[L77Sample\] (.*)$", re.M)
SAMPLER = re.compile(r"^\[L77Sampler\] action=stop (.*)$", re.M)
FIELDS = re.compile(r"([a-zA-Z0-9_]+)=([^ ]+)")
def run(stratum: str, variant: str, replicate: int) -> dict:
OUT.mkdir(parents=True, exist_ok=True)
label = f"{stratum.lower()}-{variant}-r{replicate}"
log = OUT / f"{label}.log"
camera = ",".join(f"{value:.12f}" for value in CAMERAS[stratum]) + ",0,0"
command = [
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
"timeout", "1740s", str(BINARIES[variant]), str(DATA),
"--world-assembler-prototype", "--stream-benchmark", "--stream-pipeline",
"--workers=8", "--clip-radius=3", "--max-clip-level=8", "--max-level=8",
f"--camera={camera}", "--profile-lod-work", "--stream-scheduler-policy=3",
"--linux77-profile",
]
started = time.monotonic()
with log.open("w", encoding="utf-8") as stream:
result = subprocess.run(command, cwd=ROOT, stdout=stream,
stderr=subprocess.STDOUT, check=False, timeout=1740)
content = log.read_text(encoding="utf-8", errors="replace")
sampler_match = SAMPLER.search(content)
workers = [dict(FIELDS.findall(match.group(1))) for match in WORKER.finditer(content)]
samples = [dict(FIELDS.findall(match.group(1))) for match in SAMPLE.finditer(content)]
if result.returncode or sampler_match is None or len(workers) < 9 or not samples:
raise SystemExit(f"phase census failed or markers missing: {log}; exit={result.returncode}")
sampler = dict(FIELDS.findall(sampler_match.group(1)))
if sampler.get("status") != "PASS":
raise SystemExit(f"L77 sampler failed: {log}: {sampler}")
lod_seconds = {int(row["lod"]): float(row["worker_sum_s"]) for row in workers}
def estimate(scope: str) -> dict:
lods = set(range(3)) if scope == "lod0-2" else set(range(9))
eligible = [row for row in samples if int(row["lod"]) in lods]
weights = {phase: sum(int(row.get("weight", "1")) for row in eligible
if row.get("phase") == phase)
for phase in ("passages", "support")}
total_weight = sum(int(row.get("weight", "1")) for row in eligible)
cpu_seconds = sum(value for lod, value in lod_seconds.items() if lod in lods)
result_row = {"worker_cpu_s": round(cpu_seconds, 4),
"weighted_samples": total_weight}
for phase, weight in weights.items():
share = weight / total_weight if total_weight else 0.0
result_row[f"{phase}_share_pct"] = round(share * 100.0, 3)
result_row[f"{phase}_worker_s_estimate"] = round(cpu_seconds * share, 4)
return result_row
row = {
"stratum": stratum, "variant": variant, "replicate": replicate,
"exit_code": result.returncode,
"sampler": sampler,
"lod0_2": estimate("lod0-2"), "whole_view": estimate("whole"),
"log": str(log.relative_to(ROOT)),
"runner_wall_s": round(time.monotonic() - started, 3),
}
print("[L109PhaseCensusRun] " + json.dumps(row, sort_keys=True), flush=True)
return row
def main() -> int:
if any(not path.is_file() for path in BINARIES.values()):
raise SystemExit("missing L77 profile binary")
rows = [run(stratum, variant, replicate)
for stratum in ("FI", "TN")
for replicate in (1, 2)
for variant in ("base", "candidate")]
summary = []
for stratum in ("FI", "TN"):
for variant in ("base", "candidate"):
entry = {"stratum": stratum, "variant": variant, "replicates": 2}
group = [row for row in rows if row["stratum"] == stratum
and row["variant"] == variant]
for scope in ("lod0_2", "whole_view"):
for key in ("worker_cpu_s", "passages_share_pct", "passages_worker_s_estimate",
"support_share_pct", "support_worker_s_estimate"):
values = [row[scope][key] for row in group]
entry[f"{scope}_{key}_runs"] = values
entry[f"{scope}_{key}_median"] = statistics.median(values)
summary.append(entry)
(OUT / "summary.json").write_text(
json.dumps({"runs": rows, "summary": summary}, indent=2) + "\n",
encoding="utf-8")
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,105 @@
#!/usr/bin/env python3
"""Five interleaved L80 readiness pairs for L109, at W8 and W12."""
from __future__ import annotations
import json
import re
import statistics
import subprocess
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
OUT = ROOT / "Saved/Linux/l109/timing"
DATA = ROOT / "Standalone/data/owner_world.json"
BINARIES = {
"base": ROOT / "Saved/Linux/l109/bin-baseline/VoxelForgeCoreHeadless-l109-baseline",
"candidate": ROOT / "Saved/Linux/l109/bin-final/VoxelForgeCoreHeadless-l109-clang",
}
CAMERAS = {
0: (8.0, 16.0, -372.2626953125),
1: (25.8369140625, 3.19921875, -737.0),
2: (19.166015625, -14.2001953125, -753.3740234375),
3: (-20.0625, -5.068359375, -900.353515625),
}
PIPELINE = re.compile(r"^\[StreamPipeline\] (.*)$", re.M)
FIELDS = re.compile(r"([a-zA-Z0-9_]+)=([^ ]+)")
NEAR = re.compile(
r"^\[WorldAssemblerNearLODReady\] lods=0-2 tiles=(\d+)/(\d+) cold_ready_s=([0-9.]+)$",
re.M,
)
def run(strate: int, workers: int, variant: str, phase: str, replicate: int) -> dict:
OUT.mkdir(parents=True, exist_ok=True)
label = f"s{strate}-w{workers}-{variant}-{phase}{replicate}"
log = OUT / f"{label}.log"
camera = ",".join(f"{v:.12f}" for v in CAMERAS[strate]) + ",0,0"
command = [
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
"timeout", "1740s", str(BINARIES[variant]), str(DATA),
"--world-assembler-prototype", "--stream-benchmark", "--stream-pipeline",
f"--workers={workers}", "--clip-radius=3", "--max-clip-level=8", "--max-level=8",
f"--camera={camera}", "--profile-lod-work", "--stream-scheduler-policy=3",
]
start = time.monotonic()
with log.open("w", encoding="utf-8") as output:
result = subprocess.run(command, cwd=ROOT, stdout=output,
stderr=subprocess.STDOUT, check=False, timeout=1740)
content = log.read_text(encoding="utf-8", errors="replace")
pipeline_match = PIPELINE.search(content)
near_match = NEAR.search(content)
if result.returncode or pipeline_match is None or near_match is None:
raise SystemExit(f"readiness run failed or trace missing: {log}; exit={result.returncode}")
pipeline = dict(FIELDS.findall(pipeline_match.group(1)))
row = {
"strate": strate, "workers": workers, "variant": variant,
"phase": phase, "replicate": replicate, "exit_code": result.returncode,
"wall_s": round(time.monotonic() - start, 3),
"lod02_ready_s": float(near_match.group(3)),
"whole_view_ready_s": float(pipeline["whole_view_ready"].removesuffix("s")),
"whole_tiles": int(pipeline["whole_tiles"].split("/")[0]),
"log": str(log.relative_to(ROOT)),
}
print("[L109TimingRun] " + json.dumps(row, sort_keys=True), flush=True)
return row
def main() -> int:
if not DATA.is_file() or any(not binary.is_file() for binary in BINARIES.values()):
raise SystemExit("missing data or timing binary")
rows = []
for strate in range(4):
for workers in (8, 12):
run(strate, workers, "base", "warmup", 0)
run(strate, workers, "candidate", "warmup", 0)
for replicate in range(1, 6):
order = ("base", "candidate") if replicate % 2 else ("candidate", "base")
for variant in order:
rows.append(run(strate, workers, variant, "measured", replicate))
(OUT / "runs-partial.json").write_text(
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
summary = []
for strate in range(4):
for workers in (8, 12):
item = {"strate": strate, "workers": workers, "pairs": 5}
for variant in ("base", "candidate"):
group = [row for row in rows if row["strate"] == strate
and row["workers"] == workers and row["variant"] == variant]
for key in ("lod02_ready_s", "whole_view_ready_s"):
values = [row[key] for row in group]
item[f"{variant}_{key}"] = values
item[f"{variant}_{key}_median"] = statistics.median(values)
for key in ("lod02_ready_s", "whole_view_ready_s"):
base = item[f"base_{key}_median"]
candidate = item[f"candidate_{key}_median"]
item[f"{key}_delta_pct"] = round(100 * (candidate / base - 1), 2)
summary.append(item)
(OUT / "summary.json").write_text(
json.dumps({"runs": rows, "summary": summary}, indent=2) + "\n",
encoding="utf-8")
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,484 @@
#!/usr/bin/env python3
"""Run one pinned null-RHI SW/FI arrival for Linux 107."""
from __future__ import annotations
import argparse
from collections import Counter
from datetime import datetime
import json
import re
import subprocess
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
HOST = Path("/home/jahni/VFHost")
HOST_PLUGIN_SAVED = HOST / "Plugins/VoxelForge/Saved/Linux/l109"
HOST_SAVED = HOST / "Saved/Linux/l109"
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
OUT = ROOT / "Saved/Linux/l109/unreal-arrival"
SPAWNS_M = {
"FI": (25.8369140625, 3.19921875, -737.0),
"SW": (25.8369140625, 3.19921875, -200.0),
}
MOVE_START_S = 20.0
MOVE_DURATION_S = 12.0
EXIT_S = MOVE_START_S + MOVE_DURATION_S + 5.0
RUN_TIMEOUT_S = 1740
def read_events(path: Path) -> list[dict]:
events = []
if path.is_file():
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
if line.startswith("{"):
try:
events.append(json.loads(line))
except json.JSONDecodeError:
pass
return events
def percentile(values: list[float], fraction: float) -> float | None:
if not values:
return None
ordered = sorted(values)
return ordered[min(len(ordered) - 1, int(round((len(ordered) - 1) * fraction)))]
def sample_unreal_peak_rss() -> tuple[int, int] | None:
executable = str(EDITOR).encode()
project = str(HOST / "VFHost.uproject").encode()
peak: tuple[int, int] | None = None
for process in Path("/proc").glob("[0-9]*"):
try:
command_line = (process / "cmdline").read_bytes()
if executable not in command_line or project not in command_line:
continue
status = (process / "status").read_text(encoding="ascii", errors="ignore")
match = re.search(r"^VmHWM:\s+(\d+)\s+kB$", status, re.MULTILINE)
if match:
rss_kib = int(match.group(1))
if peak is None or rss_kib > peak[0]:
peak = (rss_kib, int(process.name))
except (FileNotFoundError, PermissionError, ProcessLookupError, OSError):
continue
return peak
def endphysics_gap(stdout: Path) -> tuple[float | None, int | None]:
release_times: dict[int, dict[int, datetime]] = {}
pattern = re.compile(
r"^\[(\d{4}\.\d{2}\.\d{2}-\d{2}\.\d{2}\.\d{2}:\d{3})\].*"
r"LogTick: tick\s+(\d+)\s+-+\s+Release tick group ([34])")
with stdout.open(encoding="utf-8", errors="replace") as stream:
for line in stream:
match = pattern.match(line)
if not match:
continue
timestamp = datetime.strptime(match.group(1), "%Y.%m.%d-%H.%M.%S:%f")
frame = int(match.group(2))
group = int(match.group(3))
release_times.setdefault(frame, {})[group] = timestamp
gaps = [(round((times[4] - times[3]).total_seconds() * 1000.0, 3), frame)
for frame, times in release_times.items() if 3 in times and 4 in times]
return max(gaps, default=(None, None))
def run(stratum: str, repeat: int, phase: str, ready_depth: int,
background_cap: int, worker_reserve: int, helper_background: int,
tile_background_low: int, tile_background_normal: int, startup_boost: int,
tick_logs: bool, insights: bool) -> dict:
label = (f"{phase}-{stratum.lower()}-rd{ready_depth}-bg{background_cap}"
f"-reserve{worker_reserve}-helperbg{helper_background}"
f"-tilelow{tile_background_low}-tilenormal{tile_background_normal}"
f"-ticklog{int(tick_logs)}"
f"-boost{startup_boost}-{repeat}")
trace = OUT / f"{label}.trace.jsonl"
utrace = OUT / f"{label}.utrace"
abslog = OUT / f"{label}.log"
stdout = OUT / f"{label}.stdout.log"
output = HOST_PLUGIN_SAVED / "output" / label
userdir = HOST_SAVED / "user" / label
zendata = HOST_SAVED / "zen" / label
ddc = HOST_SAVED / "DDC"
for directory in (OUT, output, userdir, zendata, ddc):
directory.mkdir(parents=True, exist_ok=True)
trace.unlink(missing_ok=True)
utrace.unlink(missing_ok=True)
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWNS_M[stratum])
command = [
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
"-notraceserver", "-nocrashreports",
"-DisablePlugins=AndroidFileServer",
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
"-archetype=TunnelNetwork", f"-slot={1 if stratum == 'FI' else 0}",
"-modes=export", "-gameconfig=1", "-coreworldtest=1",
"-ownerfourstrateplan=1", "-corevoxelsizecm=25", "-coreislandtest=0",
"-corefallbacktest=0", "-coreofftest=0", "-coreplantest=0",
"-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings", "-corehybridmesher=0",
f"-out={output}", "-voxel.StreamingCoarseFirst=1",
"-voxel.StreamingDynamicNearReserve=1",
"-voxel.StreamingLegacyPendingResultCap=0",
"-voxel.CoreRegionConcurrentAdmission=1",
"-voxel.StreamingResultBacklogCap=256",
f"-voxel.StreamingReadyQueueDepth={ready_depth}",
f"-voxel.StreamingBackgroundWorkerCap={background_cap}",
f"-voxel.StreamingBackgroundWorkerReserve={worker_reserve}",
f"-voxel.CoreParallelForBackgroundPriority={helper_background}",
f"-voxel.StreamingTileBackgroundLow={tile_background_low}",
f"-voxel.StreamingTileBackgroundNormal={tile_background_normal}",
f"-voxel.TestLogTickFunctions={int(tick_logs)}",
"-voxel.CoreColumnCacheDirectMapped=0", "-voxel.CoreMCGradientMemoization=1",
"-voxel.StreamingCollisionLookahead=1",
f"-voxel.StreamingStartupApplyBoost={startup_boost}",
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=140.0",
f"-voxel.TestMoveStartSeconds={MOVE_START_S:.1f}",
f"-voxel.TestMoveDurationSeconds={MOVE_DURATION_S:.1f}",
f"-voxel.TestExitSeconds={EXIT_S:.1f}", "-voxel.TestMoveDirection=1,0,0",
"-voxel.StreamingSchedulerPolicy=3", "-voxel.TestExitOnSteady=0",
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
f"-abslog={abslog}",
]
if insights:
command += ["-trace=cpu,task,frame", "-statnamedevents",
f"-tracefile={utrace}", "-tracefiletrunc"]
wrapped = ["taskset", "-c", "0-11", "timeout", f"{RUN_TIMEOUT_S}s", *command]
started = time.monotonic()
with stdout.open("w", encoding="utf-8") as stream:
process = subprocess.Popen(wrapped, cwd=ROOT, stdout=stream,
stderr=subprocess.STDOUT)
peak_rss = None
while process.poll() is None:
observed = sample_unreal_peak_rss()
if observed and (peak_rss is None or observed[0] > peak_rss[0]):
peak_rss = observed
time.sleep(0.25)
exit_code = process.wait()
observed = sample_unreal_peak_rss()
if observed and (peak_rss is None or observed[0] > peak_rss[0]):
peak_rss = observed
events = read_events(trace)
worker_backend_config = next(
(e for e in events if e.get("type") == "worker_backend_config"), {})
worker_cap_readback = next(
(e for e in events if e.get("type") == "streaming_worker_cap_readback"), {})
samples = [e for e in events if e.get("type") == "streaming_sample"]
arrival = [e for e in samples if float(e.get("t_s", 0.0)) < MOVE_START_S]
full = next((e for e in arrival
if sum(e.get("desired_by_lod", [0] * 9)[:3]) > 0
and sum(e.get("settled_by_lod", [0] * 9)[:3])
>= sum(e.get("desired_by_lod", [0] * 9)[:3])), None)
first_admission = next((e for e in events if e.get("type") == "scheduler_admission"
and e.get("desired_rebuilt") == 1), None)
request_events = [e for e in events if e.get("type") == "request"]
initial_request_events = [e for e in request_events if e.get("epoch") == 0]
tile_by_ordinal = {e.get("request_ordinal"): e for e in events
if e.get("type") == "tile"}
collision_count = sum(1 for e in events if e.get("type") == "collision_ready"
and e.get("level", 9) <= 2
and full and e.get("t_s", 1e30) <= full["t_s"])
initial_tiles = [tile_by_ordinal[r.get("request_ordinal")] for r in initial_request_events
if full and r.get("t_s", 1e30) <= full["t_s"]
and r.get("request_ordinal") in tile_by_ordinal
and tile_by_ordinal[r["request_ordinal"]].get("level", 9) <= 2]
burst_start_s = float(first_admission["t_s"]) if first_admission else -1.0
burst_end_s = float(full["t_s"]) if full else -1.0
work_by_frame = {int(e.get("frame", -1)): e for e in events
if e.get("type") == "game_thread_work_frame"}
work_values = [e for e in work_by_frame.values()
if burst_start_s <= float(e.get("t_s", -1.0)) <= burst_end_s]
arrival_frame_ids = {int(e.get("frame", -1)) for e in work_values}
scheduler_by_frame = {int(e.get("frame", -1)): e for e in events
if e.get("type") == "scheduler_admission"}
scheduler_values = [e for e in events if e.get("type") == "scheduler_admission"
and burst_start_s <= float(e.get("t_s", -1.0)) <= burst_end_s]
apply_by_frame = {int(e.get("frame", -1)): e for e in events
if e.get("type") == "apply_frame"}
commandlet_tick_by_frame = {int(e.get("frame", -1)): e for e in events
if e.get("type") == "commandlet_world_tick_frame"}
garbage_collect_by_frame = {int(e.get("frame", -1)): e for e in events
if e.get("type") == "garbage_collect"}
# apply_frame N+1 records the interval containing AVoxelWorld::Tick N. Its apply_ms
# belongs to Tick N+1, so use apply_by_frame[N] for the same-tick apply breakdown.
near_intervals = [apply_by_frame[frame + 1] for frame in arrival_frame_ids
if frame + 1 in apply_by_frame]
rmc_by_frame: dict[int, dict[str, float]] = {}
for event in events:
frame = int(event.get("frame", -1))
if event.get("type") != "rmc_submit" or frame not in arrival_frame_ids:
continue
values = rmc_by_frame.setdefault(frame, {
"component_acquire_ms": 0.0,
"create_section_group_ms": 0.0,
"create_sections_ms": 0.0,
"collision_handoff_ms": 0.0,
})
for key in values:
values[key] += float(event.get(key, 0.0))
collision_completion_by_frame: dict[int, float] = {}
for event in events:
frame = int(event.get("frame", -1))
if event.get("type") == "collision_completion" and frame in arrival_frame_ids:
collision_completion_by_frame[frame] = collision_completion_by_frame.get(frame, 0.0) \
+ float(event.get("handler_ms", 0.0))
near_requests = [r for r in request_events if full and r.get("t_s", 1e30) <= full["t_s"]
and r.get("level", 9) <= 2]
initial_lod0_counts = Counter(
tuple(r.get("tile", ())) for r in near_requests if r.get("level") == 0)
repeated_lod0_keys = [
{"tile": list(tile), "requests": count}
for tile, count in sorted(initial_lod0_counts.items()) if count > 1]
initial_scheduler_frames = [e for e in events
if e.get("type") == "scheduler_admission" and full
and e.get("t_s", 1e30) <= full["t_s"]]
worst_interval = max(near_intervals, key=lambda e: float(e.get("frame_ms", 0.0)), default=None)
worst_interval_frame_id = int(worst_interval.get("frame", -1)) if worst_interval else -1
worst_work_frame_id = worst_interval_frame_id - 1
worst_work = work_by_frame.get(worst_work_frame_id, {})
worst_apply = apply_by_frame.get(worst_work_frame_id, {})
worst_scheduler = scheduler_by_frame.get(worst_work_frame_id, {})
worst_rmc = rmc_by_frame.get(worst_work_frame_id, {})
worst_collision_completion_ms = collision_completion_by_frame.get(worst_work_frame_id, 0.0)
rmc_values = [rmc_by_frame.get(frame, {}) for frame in arrival_frame_ids]
apply_values = [apply_by_frame[frame] for frame in arrival_frame_ids
if frame in apply_by_frame]
commandlet_tick_values = [commandlet_tick_by_frame[frame] for frame in arrival_frame_ids
if frame in commandlet_tick_by_frame]
garbage_collect_values = [garbage_collect_by_frame[frame] for frame in arrival_frame_ids
if frame in garbage_collect_by_frame]
def work_stat(name: str, fraction: float | None = None) -> float | None:
values = [float(e.get(name, 0.0)) for e in work_values]
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
def scheduler_stat(name: str, fraction: float | None = None) -> float | None:
values = [float(e.get(name, 0.0)) for e in scheduler_values]
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
def rmc_stat(name: str, fraction: float | None = None) -> float | None:
values = [float(e.get(name, 0.0)) for e in rmc_values]
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
def commandlet_stat(name: str, fraction: float | None = None) -> float | None:
values = [float(e.get(name, 0.0)) for e in commandlet_tick_values]
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
row = {
"label": label, "stratum": stratum, "repeat": repeat, "phase": phase,
"ready_queue_depth": ready_depth,
"requested_background_worker_cap": background_cap,
"background_worker_cap": worker_cap_readback.get(
"streaming_background_worker_cap", background_cap),
"requested_background_worker_reserve": worker_reserve,
"background_worker_reserve": worker_backend_config.get(
"streaming_background_worker_reserve", worker_cap_readback.get(
"background_worker_reserve", worker_reserve)),
"requested_core_parallel_for_background_priority": helper_background,
"core_parallel_for_background_priority": worker_backend_config.get(
"core_parallel_for_background_priority", helper_background),
"requested_streaming_tile_background_low": tile_background_low,
"streaming_tile_background_low": worker_backend_config.get(
"streaming_tile_background_low", tile_background_low),
"requested_streaming_tile_background_normal": tile_background_normal,
"streaming_tile_background_normal": worker_backend_config.get(
"streaming_tile_background_normal", tile_background_normal),
"effective_tile_slots": worker_backend_config.get("effective_tile_slots"),
"background_worker_threads": worker_backend_config.get("taskgraph_background_threads"),
"startup_apply_boost": startup_boost, "insights": insights,
"engine_tick_logs": tick_logs,
"exit_code": exit_code,
"launch_wrapper": "taskset -c 0-11 timeout 1740s (no address-space limit)",
"peak_rss_kib": peak_rss[0] if peak_rss else None,
"peak_rss_mib": round(peak_rss[0] / 1024.0, 1) if peak_rss else None,
"peak_rss_pid": peak_rss[1] if peak_rss else None,
"endphysics_to_postphysics_gap_ms_max": None,
"endphysics_gap_frame": None,
"wall_s": round(time.monotonic() - started, 3),
"desired_t_s": next((e.get("t_s") for e in events if e.get("type") == "desired_set"), None),
"first_admission_t_s": first_admission.get("t_s") if first_admission else None,
"lod02_full_settled_t_s": full.get("t_s") if full else None,
"near_settle_from_first_admission_s": round(
float(full["t_s"]) - float(first_admission["t_s"]), 4)
if full and first_admission else None,
"desired_by_lod": full.get("desired_by_lod", [])[:3] if full else None,
"settled_by_lod": full.get("settled_by_lod", [])[:3] if full else None,
"near_admitted_tiles": len(near_requests),
"initial_lod0_admissions": sum(initial_lod0_counts.values()),
"repeated_lod0_request_keys_before_settle": repeated_lod0_keys,
"peak_result_backlog_before_settle": max(
(int(e.get("result_backlog", 0)) for e in initial_scheduler_frames), default=0),
"last_admission_t_s": max((e.get("t_s", 0.0) for e in near_requests), default=None),
"near_tile_events": len(initial_tiles),
"near_worker_seconds": round(sum(e.get("worker_task_s", 0.0) for e in initial_tiles), 4),
"near_core_seconds": round(sum(e.get("core_generate_s", 0.0) for e in initial_tiles), 4),
"collision_ready_by_settle": collision_count,
"burst_frame_count": len(near_intervals),
"burst_frame_ms_p95": round(sorted(e.get("frame_ms", 0.0) for e in near_intervals)[
min(len(near_intervals) - 1, int(.95 * (len(near_intervals) - 1)))], 3)
if near_intervals else None,
"burst_frame_ms_max": round(max((e.get("frame_ms", 0.0) for e in near_intervals), default=0.0), 3),
"burst_full_world_tick_ms_p99": round(commandlet_stat("world_tick_ms", .99) or 0.0, 3),
"burst_full_world_tick_ms_max": round(commandlet_stat("world_tick_ms") or 0.0, 3),
"burst_world_pre_actor_ms_p99": round(commandlet_stat("world_pre_actor_ms", .99) or 0.0, 3),
"burst_world_pre_actor_ms_max": round(commandlet_stat("world_pre_actor_ms") or 0.0, 3),
"burst_world_actor_phase_ms_p99": round(commandlet_stat("world_actor_phase_ms", .99) or 0.0, 3),
"burst_world_actor_phase_ms_max": round(commandlet_stat("world_actor_phase_ms") or 0.0, 3),
"burst_world_post_actor_ms_p99": round(commandlet_stat("world_post_actor_ms", .99) or 0.0, 3),
"burst_world_post_actor_ms_max": round(commandlet_stat("world_post_actor_ms") or 0.0, 3),
"burst_commandlet_readiness_check_ms_max": round(
commandlet_stat("readiness_check_ms") or 0.0, 3),
"burst_commandlet_sleep_ms_p99": round(commandlet_stat("sleep_ms", .99) or 0.0, 3),
"burst_commandlet_sleep_ms_max": round(commandlet_stat("sleep_ms") or 0.0, 3),
"burst_commandlet_loop_ms_max": round(commandlet_stat("loop_ms") or 0.0, 3),
"burst_gc_count": len(garbage_collect_values),
"burst_gc_duration_ms_max": round(max(
(float(e.get("duration_ms", 0.0)) for e in garbage_collect_values), default=0.0), 3),
"burst_max_meshes_per_frame": max((e.get("meshes_applied", 0) for e in apply_values), default=0),
"burst_max_apply_ms": round(max((e.get("apply_ms", 0.0) for e in apply_values), default=0.0), 3),
"burst_max_admit_per_frame": max((sum(e.get("submitted_by_lod", [0] * 9)[:3])
for e in scheduler_values), default=0),
"burst_frame_timing_count": len(work_values),
"burst_active_generation_workers_p95": round(work_stat("active_generation_workers", .95) or 0.0, 3),
"burst_active_generation_workers_max": round(work_stat("active_generation_workers") or 0.0, 3),
"burst_active_tile_tasks_max": round(work_stat("active_tile_tasks") or 0.0, 3),
"burst_world_tick_ms_p99": round(work_stat("tick_ms", .99) or 0.0, 3),
"burst_world_tick_ms_max": round(work_stat("tick_ms") or 0.0, 3),
"burst_update_chunks_ms_p99": round(work_stat("update_chunks_ms", .99) or 0.0, 3),
"burst_update_chunks_ms_max": round(work_stat("update_chunks_ms") or 0.0, 3),
"burst_pending_drain_ms_p99": round(work_stat("pending_drain_ms", .99) or 0.0, 3),
"burst_pending_drain_ms_max": round(work_stat("pending_drain_ms") or 0.0, 3),
"burst_post_apply_callbacks_ms_p99": round(work_stat("post_apply_callbacks_ms", .99) or 0.0, 3),
"burst_post_apply_callbacks_ms_max": round(work_stat("post_apply_callbacks_ms") or 0.0, 3),
"burst_visibility_reconcile_ms_p99": round(work_stat("visibility_reconcile_ms", .99) or 0.0, 3),
"burst_visibility_reconcile_ms_max": round(work_stat("visibility_reconcile_ms") or 0.0, 3),
"burst_collision_probe_ms_max": round(work_stat("collision_probe_ms") or 0.0, 3),
"burst_visible_apply_probe_ms_max": round(work_stat("visible_apply_probe_ms") or 0.0, 3),
"burst_lod0_ready_ms_max": round(work_stat("lod0_ready_ms") or 0.0, 3),
"burst_steady_trace_ms_max": round(work_stat("steady_trace_ms") or 0.0, 3),
"burst_modification_ms_max": round(work_stat("modification_ms") or 0.0, 3),
"burst_lifecycle_audit_ms_p99": round(work_stat("lifecycle_audit_ms", .99) or 0.0, 3),
"burst_lifecycle_audit_ms_max": round(work_stat("lifecycle_audit_ms") or 0.0, 3),
"burst_collision_gate_ms_p99": round(work_stat("collision_gate_ms", .99) or 0.0, 3),
"burst_collision_gate_ms_max": round(work_stat("collision_gate_ms") or 0.0, 3),
"burst_unload_ms_p99": round(work_stat("unload_ms", .99) or 0.0, 3),
"burst_unload_ms_max": round(work_stat("unload_ms") or 0.0, 3),
"burst_trace_json_ms_p99": round(work_stat("trace_json_ms", .99) or 0.0, 3),
"burst_trace_json_ms_max": round(work_stat("trace_json_ms") or 0.0, 3),
"burst_desired_rebuild_ms_max": round(scheduler_stat("desired_rebuild_ms") or 0.0, 3),
"burst_rmc_create_group_ms_p99": round(rmc_stat("create_section_group_ms", .99) or 0.0, 3),
"burst_rmc_create_group_ms_max": round(rmc_stat("create_section_group_ms") or 0.0, 3),
"burst_rmc_collision_handoff_ms_max": round(rmc_stat("collision_handoff_ms") or 0.0, 3),
"burst_collision_completion_ms_max": round(max(
collision_completion_by_frame.values(), default=0.0), 3),
"worst_frame_breakdown": {
"frame": worst_work_frame_id,
"world_tick_frame": worst_work_frame_id,
"frame_interval_end_frame": worst_interval_frame_id,
"frame_interval_ms": round(float(worst_interval.get("frame_ms", 0.0)), 3)
if worst_interval else None,
"apply_ms": round(float(worst_apply.get("apply_ms", 0.0)), 3)
if worst_apply else None,
"result_drain_ms": round(float(worst_apply.get("drain_ms", 0.0)), 3)
if worst_apply else None,
"outside_world_tick_ms": round(max(0.0, float(worst_interval.get("frame_ms", 0.0))
- float(worst_work.get("tick_ms", 0.0))), 3)
if worst_interval else None,
"outside_full_world_tick_ms": round(max(0.0, float(worst_interval.get("frame_ms", 0.0))
- float(commandlet_tick_by_frame.get(worst_work_frame_id, {}).get("world_tick_ms", 0.0))), 3)
if worst_interval else None,
"full_world_tick_ms": round(float(commandlet_tick_by_frame.get(
worst_work_frame_id, {}).get("world_tick_ms", 0.0)), 3),
"world_pre_actor_ms": round(float(commandlet_tick_by_frame.get(
worst_work_frame_id, {}).get("world_pre_actor_ms", 0.0)), 3),
"world_actor_phase_ms": round(float(commandlet_tick_by_frame.get(
worst_work_frame_id, {}).get("world_actor_phase_ms", 0.0)), 3),
"world_post_actor_ms": round(float(commandlet_tick_by_frame.get(
worst_work_frame_id, {}).get("world_post_actor_ms", 0.0)), 3),
"commandlet_readiness_check_ms": round(float(commandlet_tick_by_frame.get(
worst_work_frame_id, {}).get("readiness_check_ms", 0.0)), 3),
"commandlet_sleep_ms": round(float(commandlet_tick_by_frame.get(
worst_work_frame_id, {}).get("sleep_ms", 0.0)), 3),
"commandlet_loop_ms": round(float(commandlet_tick_by_frame.get(
worst_work_frame_id, {}).get("loop_ms", 0.0)), 3),
"gc_duration_ms": round(float(garbage_collect_by_frame.get(
worst_work_frame_id, {}).get("duration_ms", 0.0)), 3),
"world_tick_ms": round(float(worst_work.get("tick_ms", 0.0)), 3),
"active_generation_workers": int(worst_work.get("active_generation_workers", 0)),
"active_tile_tasks": int(worst_work.get("active_tile_tasks", 0)),
"collision_gate_ms": round(float(worst_work.get("collision_gate_ms", 0.0)), 3),
"update_chunks_ms": round(float(worst_work.get("update_chunks_ms", 0.0)), 3),
"desired_rebuild_ms": round(float(worst_scheduler.get("desired_rebuild_ms", 0.0)), 3),
"pending_drain_ms": round(float(worst_work.get("pending_drain_ms", 0.0)), 3),
"post_apply_callbacks_ms": round(float(worst_work.get("post_apply_callbacks_ms", 0.0)), 3),
"collision_probe_ms": round(float(worst_work.get("collision_probe_ms", 0.0)), 3),
"visibility_reconcile_ms": round(float(worst_work.get("visibility_reconcile_ms", 0.0)), 3),
"visible_apply_probe_ms": round(float(worst_work.get("visible_apply_probe_ms", 0.0)), 3),
"lod0_ready_ms": round(float(worst_work.get("lod0_ready_ms", 0.0)), 3),
"unload_ms": round(float(worst_work.get("unload_ms", 0.0)), 3),
"steady_trace_ms": round(float(worst_work.get("steady_trace_ms", 0.0)), 3),
"modification_ms": round(float(worst_work.get("modification_ms", 0.0)), 3),
"lifecycle_audit_ms": round(float(worst_work.get("lifecycle_audit_ms", 0.0)), 3),
"trace_json_ms": round(float(worst_work.get("trace_json_ms", 0.0)), 3),
"trace_archive_serialize_ms": round(float(
worst_work.get("trace_archive_serialize_ms", 0.0)), 3),
"collision_completion_ms": round(worst_collision_completion_ms, 3),
**{key: round(float(worst_rmc.get(key, 0.0)), 3)
for key in ("component_acquire_ms", "create_section_group_ms",
"create_sections_ms", "collision_handoff_ms")},
},
"trace": str(trace.relative_to(ROOT)), "utrace": str(utrace.relative_to(ROOT)),
"abslog": str(abslog.relative_to(ROOT)), "stdout": str(stdout.relative_to(ROOT)),
}
stdout_text = stdout.read_text(encoding="utf-8", errors="replace")
gap_ms, gap_frame = endphysics_gap(stdout)
row["endphysics_to_postphysics_gap_ms_max"] = gap_ms
row["endphysics_gap_frame"] = gap_frame
failure = re.search(r"\[VoxelForgeCoreWorldTest\] failed: ([^\r\n]+)", stdout_text)
if failure:
row["world_test_failure"] = failure.group(1)
with (OUT / "runs.jsonl").open("a", encoding="utf-8") as stream:
stream.write(json.dumps(row, sort_keys=True) + "\n")
(OUT / f"{label}.summary.json").write_text(json.dumps(row, indent=2) + "\n",
encoding="utf-8")
print("[L109UnrealArrival] " + json.dumps(row, sort_keys=True), flush=True)
if exit_code:
raise SystemExit(f"Unreal commandlet failed: {stdout}; exit={exit_code}")
if not full or len(arrival) < 8:
raise SystemExit(f"Unreal arrival trace incomplete: {stdout}; exit={exit_code}")
if insights and (not utrace.is_file() or utrace.stat().st_size < 1024):
raise SystemExit(f"Unreal Insights trace missing or empty: {utrace}")
return row
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--stratum", choices=("FI", "SW"), required=True)
parser.add_argument("--repeat", type=int, default=1)
parser.add_argument("--phase", choices=("probe", "before", "after", "snapshot"), default="probe")
parser.add_argument("--ready-depth", type=int, default=64)
parser.add_argument("--background-cap", type=int, choices=(0, 1), default=1)
parser.add_argument("--worker-reserve", type=int, choices=(0, 1, 2, 3), default=0)
parser.add_argument("--helper-background", type=int, choices=(0, 1), default=1)
parser.add_argument("--tile-background-low", type=int, choices=(0, 1), default=1)
parser.add_argument("--tile-background-normal", type=int, choices=(0, 1), default=0)
parser.add_argument("--startup-boost", type=int, choices=(0, 1), default=1)
parser.add_argument("--tick-logs", action="store_true")
parser.add_argument("--insights", action="store_true")
args = parser.parse_args()
if not EDITOR.is_file():
raise SystemExit(f"missing Unreal commandlet executable: {EDITOR}")
run(args.stratum, args.repeat, args.phase, args.ready_depth,
args.background_cap, args.worker_reserve, args.helper_background,
args.tile_background_low, args.tile_background_normal, args.startup_boost,
args.tick_logs, args.insights)
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,93 @@
# L109 passage captures
Each PNG was rendered from its own exported mesh. The renderer reported visible triangles, and each decoded image met the pixel coverage floor. `low_contrast_geometry_visible` marks images that met this floor below the preferred contrast target.
## Before
- [upper-mouth](before/passage-0-1-0-upper-mouth.png) — passage/0-1/0; 188440 visible triangles
- [lower-mouth](before/passage-0-1-0-lower-mouth.png) — passage/0-1/0; 564520 visible triangles
- [25%](before/passage-0-1-0-station-25.png) — passage/0-1/0; 234197 visible triangles
- [50%](before/passage-0-1-0-station-50.png) — passage/0-1/0; 315900 visible triangles
- [75%](before/passage-0-1-0-station-75.png) — passage/0-1/0; 256359 visible triangles
- [section-1/view-1](before/passage-0-1-0-openair-1-view-1.png) — passage/0-1/0 **OPEN AIR**; 211462 visible triangles
- [section-1/view-2](before/passage-0-1-0-openair-1-view-2.png) — passage/0-1/0 **OPEN AIR**; 208255 visible triangles
- [section-1/view-3](before/passage-0-1-0-openair-1-view-3.png) — passage/0-1/0 **OPEN AIR**; 221239 visible triangles
- [section-2/view-1](before/passage-0-1-0-openair-2-view-1.png) — passage/0-1/0 **OPEN AIR**; 218664 visible triangles
- [section-2/view-2](before/passage-0-1-0-openair-2-view-2.png) — passage/0-1/0 **OPEN AIR**; 223817 visible triangles
- [section-2/view-3](before/passage-0-1-0-openair-2-view-3.png) — passage/0-1/0 **OPEN AIR**; 221239 visible triangles
- [section-3/view-1](before/passage-0-1-0-openair-3-view-1.png) — passage/0-1/0 **OPEN AIR**; 224005 visible triangles
- [section-3/view-2](before/passage-0-1-0-openair-3-view-2.png) — passage/0-1/0 **OPEN AIR**; 230599 visible triangles; low_contrast_geometry_visible
- [section-3/view-3](before/passage-0-1-0-openair-3-view-3.png) — passage/0-1/0 **OPEN AIR**; 229291 visible triangles
- [section-4/view-1](before/passage-0-1-0-openair-4-view-1.png) — passage/0-1/0 **OPEN AIR**; 407353 visible triangles
- [section-4/view-2](before/passage-0-1-0-openair-4-view-2.png) — passage/0-1/0 **OPEN AIR**; 406732 visible triangles
- [section-4/view-3](before/passage-0-1-0-openair-4-view-3.png) — passage/0-1/0 **OPEN AIR**; 363878 visible triangles
- [upper-mouth](before/passage-1-2-0-upper-mouth.png) — passage/1-2/0; 576595 visible triangles
- [lower-mouth](before/passage-1-2-0-lower-mouth.png) — passage/1-2/0; 555286 visible triangles
- [25%](before/passage-1-2-0-station-25.png) — passage/1-2/0; 667602 visible triangles
- [50%](before/passage-1-2-0-station-50.png) — passage/1-2/0; 602756 visible triangles
- [75%](before/passage-1-2-0-station-75.png) — passage/1-2/0; 620502 visible triangles
- [section-1/view-1](before/passage-1-2-0-openair-1-view-1.png) — passage/1-2/0 **OPEN AIR**; 636438 visible triangles
- [section-1/view-2](before/passage-1-2-0-openair-1-view-2.png) — passage/1-2/0 **OPEN AIR**; 593960 visible triangles
- [section-1/view-3](before/passage-1-2-0-openair-1-view-3.png) — passage/1-2/0 **OPEN AIR**; 622040 visible triangles
- [upper-mouth](before/passage-2-3-0-upper-mouth.png) — passage/2-3/0; 59568 visible triangles
- [lower-mouth](before/passage-2-3-0-lower-mouth.png) — passage/2-3/0; 294430 visible triangles; low_contrast_geometry_visible
- [25%](before/passage-2-3-0-station-25.png) — passage/2-3/0; 55420 visible triangles; low_contrast_geometry_visible
- [50%](before/passage-2-3-0-station-50.png) — passage/2-3/0; 458134 visible triangles
- [75%](before/passage-2-3-0-station-75.png) — passage/2-3/0; 475978 visible triangles
- [section-1/view-1](before/passage-2-3-0-openair-1-view-1.png) — passage/2-3/0 **OPEN AIR**; 104612 visible triangles
- [section-1/view-2](before/passage-2-3-0-openair-1-view-2.png) — passage/2-3/0 **OPEN AIR**; 81078 visible triangles
- [section-1/view-3](before/passage-2-3-0-openair-1-view-3.png) — passage/2-3/0 **OPEN AIR**; 94962 visible triangles; low_contrast_geometry_visible
- [section-2/view-1](before/passage-2-3-0-openair-2-view-1.png) — passage/2-3/0 **OPEN AIR**; 476052 visible triangles
- [section-2/view-2](before/passage-2-3-0-openair-2-view-2.png) — passage/2-3/0 **OPEN AIR**; 479198 visible triangles
- [section-2/view-3](before/passage-2-3-0-openair-2-view-3.png) — passage/2-3/0 **OPEN AIR**; 479198 visible triangles
- [section-3/view-1](before/passage-2-3-0-openair-3-view-1.png) — passage/2-3/0 **OPEN AIR**; 485580 visible triangles
- [section-3/view-2](before/passage-2-3-0-openair-3-view-2.png) — passage/2-3/0 **OPEN AIR**; 478866 visible triangles
- [section-3/view-3](before/passage-2-3-0-openair-3-view-3.png) — passage/2-3/0 **OPEN AIR**; 490304 visible triangles; low_contrast_geometry_visible
- [section-4/view-1](before/passage-2-3-0-openair-4-view-1.png) — passage/2-3/0 **OPEN AIR**; 475662 visible triangles
- [section-4/view-2](before/passage-2-3-0-openair-4-view-2.png) — passage/2-3/0 **OPEN AIR**; 496128 visible triangles
- [section-4/view-3](before/passage-2-3-0-openair-4-view-3.png) — passage/2-3/0 **OPEN AIR**; 496508 visible triangles; low_contrast_geometry_visible
## After
- [upper-mouth](after/passage-0-1-0-upper-mouth.png) — passage/0-1/0; 253282 visible triangles
- [lower-mouth](after/passage-0-1-0-lower-mouth.png) — passage/0-1/0; 559326 visible triangles
- [25%](after/passage-0-1-0-station-25.png) — passage/0-1/0; 145630 visible triangles
- [50%](after/passage-0-1-0-station-50.png) — passage/0-1/0; 430386 visible triangles
- [75%](after/passage-0-1-0-station-75.png) — passage/0-1/0; 294374 visible triangles
- [section-1/view-1](after/passage-0-1-0-openair-1-view-1.png) — passage/0-1/0 **OPEN AIR**; 289074 visible triangles
- [section-1/view-2](after/passage-0-1-0-openair-1-view-2.png) — passage/0-1/0 **OPEN AIR**; 298057 visible triangles
- [section-1/view-3](after/passage-0-1-0-openair-1-view-3.png) — passage/0-1/0 **OPEN AIR**; 322309 visible triangles
- [section-2/view-1](after/passage-0-1-0-openair-2-view-1.png) — passage/0-1/0 **OPEN AIR**; 283642 visible triangles
- [section-2/view-2](after/passage-0-1-0-openair-2-view-2.png) — passage/0-1/0 **OPEN AIR**; 316771 visible triangles
- [section-2/view-3](after/passage-0-1-0-openair-2-view-3.png) — passage/0-1/0 **OPEN AIR**; 322309 visible triangles
- [section-3/view-1](after/passage-0-1-0-openair-3-view-1.png) — passage/0-1/0 **OPEN AIR**; 262299 visible triangles
- [section-3/view-2](after/passage-0-1-0-openair-3-view-2.png) — passage/0-1/0 **OPEN AIR**; 310919 visible triangles; low_contrast_geometry_visible
- [section-3/view-3](after/passage-0-1-0-openair-3-view-3.png) — passage/0-1/0 **OPEN AIR**; 363071 visible triangles
- [section-4/view-1](after/passage-0-1-0-openair-4-view-1.png) — passage/0-1/0 **OPEN AIR**; 378007 visible triangles
- [section-4/view-2](after/passage-0-1-0-openair-4-view-2.png) — passage/0-1/0 **OPEN AIR**; 471132 visible triangles
- [section-4/view-3](after/passage-0-1-0-openair-4-view-3.png) — passage/0-1/0 **OPEN AIR**; 382187 visible triangles
- [upper-mouth](after/passage-1-2-0-upper-mouth.png) — passage/1-2/0; 570417 visible triangles; low_contrast_geometry_visible
- [lower-mouth](after/passage-1-2-0-lower-mouth.png) — passage/1-2/0; 511318 visible triangles
- [25%](after/passage-1-2-0-station-25.png) — passage/1-2/0; 661032 visible triangles
- [50%](after/passage-1-2-0-station-50.png) — passage/1-2/0; 602996 visible triangles
- [75%](after/passage-1-2-0-station-75.png) — passage/1-2/0; 615520 visible triangles
- [section-1/view-1](after/passage-1-2-0-openair-1-view-1.png) — passage/1-2/0 **OPEN AIR**; 629052 visible triangles
- [section-1/view-2](after/passage-1-2-0-openair-1-view-2.png) — passage/1-2/0 **OPEN AIR**; 588414 visible triangles
- [section-1/view-3](after/passage-1-2-0-openair-1-view-3.png) — passage/1-2/0 **OPEN AIR**; 616234 visible triangles
- [upper-mouth](after/passage-2-3-0-upper-mouth.png) — passage/2-3/0; 59562 visible triangles
- [lower-mouth](after/passage-2-3-0-lower-mouth.png) — passage/2-3/0; 294522 visible triangles; low_contrast_geometry_visible
- [25%](after/passage-2-3-0-station-25.png) — passage/2-3/0; 55484 visible triangles; low_contrast_geometry_visible
- [50%](after/passage-2-3-0-station-50.png) — passage/2-3/0; 457754 visible triangles
- [75%](after/passage-2-3-0-station-75.png) — passage/2-3/0; 475706 visible triangles
- [section-1/view-1](after/passage-2-3-0-openair-1-view-1.png) — passage/2-3/0 **OPEN AIR**; 104612 visible triangles
- [section-1/view-2](after/passage-2-3-0-openair-1-view-2.png) — passage/2-3/0 **OPEN AIR**; 81078 visible triangles
- [section-1/view-3](after/passage-2-3-0-openair-1-view-3.png) — passage/2-3/0 **OPEN AIR**; 94962 visible triangles; low_contrast_geometry_visible
- [section-2/view-1](after/passage-2-3-0-openair-2-view-1.png) — passage/2-3/0 **OPEN AIR**; 475688 visible triangles
- [section-2/view-2](after/passage-2-3-0-openair-2-view-2.png) — passage/2-3/0 **OPEN AIR**; 479134 visible triangles
- [section-2/view-3](after/passage-2-3-0-openair-2-view-3.png) — passage/2-3/0 **OPEN AIR**; 479134 visible triangles
- [section-3/view-1](after/passage-2-3-0-openair-3-view-1.png) — passage/2-3/0 **OPEN AIR**; 485216 visible triangles
- [section-3/view-2](after/passage-2-3-0-openair-3-view-2.png) — passage/2-3/0 **OPEN AIR**; 479198 visible triangles
- [section-3/view-3](after/passage-2-3-0-openair-3-view-3.png) — passage/2-3/0 **OPEN AIR**; 490436 visible triangles; low_contrast_geometry_visible
- [section-4/view-1](after/passage-2-3-0-openair-4-view-1.png) — passage/2-3/0 **OPEN AIR**; 475298 visible triangles
- [section-4/view-2](after/passage-2-3-0-openair-4-view-2.png) — passage/2-3/0 **OPEN AIR**; 495856 visible triangles
- [section-4/view-3](after/passage-2-3-0-openair-4-view-3.png) — passage/2-3/0 **OPEN AIR**; 496236 visible triangles; low_contrast_geometry_visible
Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 100 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 321 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 250 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 181 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 254 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 326 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 280 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 286 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 448 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 493 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 287 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 523 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 505 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 528 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 266 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 224 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 49 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 492 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 513 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 312 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 41 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 160 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 41 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 188 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 87 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 61 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 138 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 49 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 146 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 43 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 324 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 129 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 48 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 239 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 64 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 37 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 134 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 141 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 131 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 224 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 41 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 222 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 43 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 143 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 178 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 112 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 159 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 189 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 86 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 49 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 144 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 43 KiB

File diff suppressed because it is too large. Load diff
@@ -0,0 +1,42 @@
# Linux111 report
## Design
Version 218 resamples passage controls with a centripetal Catmull–Rom curve at about 1 m spacing (310 control spans became 2,790 prepared segments). Open-air shells use a 0.75 m floor slab and side walls up to 1.0 m above the floor; there is no roof. Passage interior remains higher priority than shell. The second design constrained the floor route to a 14.5° grade ceiling.
The per-tile candidate lists were not measured. The 310→2,790 figures are prepared route segment counts, not tile candidate list sizes.
## Speed
L109 medians below are the candidate column from the saved five-pair timing matrix. L111 timing was not run after the audits failed.
| Stratum | L109 W8 near-ready (s) | L109 W12 near-ready (s) | L111 W8/W12 |
|---|---:|---:|---|
| SurfaceWorld | 0.737 | 0.761 | Not run |
| FloatingIslands | 0.911 | 0.863 | Not run |
| TunnelNetwork | 0.956 | 0.895 | Not run |
| VerticalShafts | 0.789 | 0.738 | Not run |
## Audits
- Clang standalone build completed. The second design removed all grade failures. Passage 0–1 still had 2 unsupported walk samples; passages 1–2 and 2–3 passed. The combined resolved-passage audit failed.
- Mouth-profile audits passed 3/3. Naturalness and ripple audits passed 2/3.
- The half-pipe shell audit passed 2/3; passage 0–1 reported 11,544 floor and 6,945 wall violations. A targeted trace showed at least one reported floor sample was 0.101 m inside the passage after relief, where interior correctly overrides the shell. The audit's smooth-route overlap model still disagreed at that point, so the remaining shell counts are not conclusive.
- Fast tier, Clang/GCC W1/W8/W12 hash matrix, readiness timing, and Unreal Linux build were not run. Canonical v218 hashes were not generated.
## Captures
No Linux111 capture images were generated. Mouths, 25/50/75% stations, open-air sections, and programmatically verified 4 m outside/interior close-ups remain outstanding.
The candidate audit found these open-air sections:
- passage 0–1: 6.408–16.782 m; 19.500–24.439 m; 1,044.046–1,498.708 m
- passage 1–2: 81.654–85.646 m; 104.092–104.829 m
- passage 2–3: 7.701–26.008 m; 372.697–377.363 m
## Failed attempts
1. Dense spline plus half-pipe: walk grades exceeded 15° and passage 0–1 failed shell, naturalness, and ripple checks.
2. Dense spline plus a grade-limited floor: grade failures cleared, but passage 0–1 retained 2 unsupported walk samples and its shell/naturalness/ripple audits still failed. Work stopped after this second design as requested.
Logs: `Saved/Linux/l111/build-clang.log` and `Saved/Linux/l111/audit-owner.log`.
@@ -0,0 +1,44 @@
# Linux 112 — horizontal spline, planned floor profile
## Shape and audits
The passage now follows a dense centripetal spline in XY. Height stays piecewise linear on the planner's horizontal arc-length profile; radius and floor depth blend smoothly. Open-air sections have a floor slab and 1 m side walls, with no roof, and passage interiors take priority over shells. Version is v219; source stamp is `c31c8baca8f8308dca62edc6d286c8e84a8214ede7ca444fdaf337ae743ccee1`.
The audit samples the same generated centreline. No walkability thresholds were loosened: the grade limit remains 15°, headroom remains 0.30 m for Caves and 2.20 m for Lattice, and the shell, mouth, naturalness, and ripple gates remain in place. The owner route has zero blocked, unsupported, or grade-failure samples. The native Caves route passes at 15.000° maximum grade and 0.300 m minimum headroom; native Lattice passes with zero support failures at 10.028° maximum grade and 2.200 m headroom. All three passage interfaces pass shell, mouth, naturalness, and ripple audits. The FI blocked control at passage/1-2 remains detected (16,380 blocked capsule samples); all three blocked controls were detected.
The native Caves field owns its corridor so the transition shell cannot clip it; the passage interior still opens within its own stratum. Lattice's authored support below the floor remains solid. These preserve the same walkability and support requirements.
## Captures
[Capture index](captures/INDEX.md): 110 images verified (60 L109 before, 50 L112 after), covering mouths, 25/50/75% stations, and every open-air section. This includes 32 verified 4 m close-ups (18 before, 14 after) of the outside and interior walls. For the close-ups, the nominal route target snaps to a nearby mesh triangle; the camera remains 4.00 m from the mesh within 0.02 m, with wall-normal alignment at least 0.25.
## Readiness
Five interleaved L80 pairs per cell, all four strata, W8 and W12. Values are median seconds, L109 binary → L112 binary; near is LOD0–2 ready and whole is whole-view ready.
| Stratum | W8 near | W8 whole | W12 near | W12 whole |
|---|---:|---:|---:|---:|
| SW | 0.779 → 0.822 | 1.837 → 1.826 | 0.802 → 0.845 | 1.768 → 1.835 |
| FI | 0.972 → 1.063 | 1.935 → 2.022 | 0.924 → 0.992 | 1.792 → 1.884 |
| TN | 1.021 → 1.092 | 1.869 → 1.970 | 0.931 → 1.023 | 1.758 → 1.866 |
| VS | 0.832 → 0.921 | 1.764 → 1.827 | 0.783 → 0.861 | 1.666 → 1.741 |
L109's prior report recorded W12 near-ready at 0.863 s for FI and 0.895 s for TN. In this matrix the L109 binary remeasured at 0.924 s and 0.931 s; L112 measured 0.992 s and 1.023 s. That is +7.4% and +9.9% against the matched rerun, or +15.0% and +14.3% against the prior report values.
## Candidate lists
Feature and route-segment lists have the same counts. Each cell shows L109 → L112; totals are candidates across profiled tiles.
| Stratum | Profiled tiles | Total candidates | p95 per tile | Maximum per tile |
|---|---:|---:|---:|---:|
| SW | 1,763 → 1,742 | 583 → 4,216 | 3 → 18 | 21 → 185 |
| FI | 1,590 → 1,573 | 1,036 → 7,068 | 4 → 22 | 47 → 420 |
| TN | 1,492 → 1,480 | 869 → 4,784 | 4 → 22 | 23 → 116 |
| VS | 1,660 → 1,660 | 644 → 3,133 | 4 → 18 | 16 → 82 |
## Verification
- Fast tier: PASS, 264 checks.
- Clang/GCC W1/W8/W12 determinism: all 18 runs passed; hashes match within each configuration.
- Linux VFHostEditor build: succeeded with `-NoUBA -MaxParallelActions=4` after timing.
- No commit was created.
@@ -0,0 +1,83 @@
#!/usr/bin/env python3
"""Compare per-tile passage candidate-list sizes for L109 and L112."""
from __future__ import annotations
import json
import math
import re
import statistics
import subprocess
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
OUT = ROOT / "Saved/Linux/l112/candidate-lists"
DATA = ROOT / "Standalone/data/owner_world.json"
BINARIES = {
"L109": ROOT / "Saved/Linux/l109/bin-final/VoxelForgeCoreHeadless-l109-clang",
"L112": ROOT / "Saved/Linux/l112/bin-clang/VoxelForgeCoreHeadless-l112-clang",
}
CAMERAS = {
0: (8.0, 16.0, -372.2626953125),
1: (25.8369140625, 3.19921875, -737.0),
2: (19.166015625, -14.2001953125, -753.3740234375),
3: (-20.0625, -5.068359375, -900.353515625),
}
LIST = re.compile(r"^\[PassageComposerTile\].*?features=(\d+) route_candidates=(\d+)", re.M)
def percentile(values: list[int], fraction: float) -> int:
ordered = sorted(values)
if not ordered:
return 0
return ordered[min(len(ordered) - 1,
max(0, math.ceil(fraction * len(ordered)) - 1))]
def run(strate: int, version: str) -> dict:
OUT.mkdir(parents=True, exist_ok=True)
log = OUT / f"{version}-s{strate}.log"
camera = ",".join(f"{v:.12f}" for v in CAMERAS[strate]) + ",0,0"
command = [
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
"timeout", "1740s", str(BINARIES[version]), str(DATA),
"--world-assembler-prototype", "--stream-benchmark", "--stream-pipeline",
"--workers=12", "--clip-radius=3", "--max-clip-level=8", "--max-level=8",
f"--camera={camera}", "--profile-passage-composer", "--stream-scheduler-policy=3",
]
started = time.monotonic()
with log.open("w", encoding="utf-8") as stream:
result = subprocess.run(command, cwd=ROOT, stdout=stream,
stderr=subprocess.STDOUT, check=False, timeout=1740)
output = log.read_text(encoding="utf-8", errors="replace")
features = [int(row[0]) for row in LIST.findall(output)]
routes = [int(row[1]) for row in LIST.findall(output)]
if result.returncode or not features:
raise SystemExit(f"candidate-list profile missing: {log}; exit={result.returncode}")
row = {
"strate": strate, "version": version, "profiled_tiles": len(features),
"feature_candidates": {"sum": sum(features), "min": min(features),
"median": statistics.median(features), "p95": percentile(features, .95),
"max": max(features)},
"route_segment_candidates": {"sum": sum(routes), "min": min(routes),
"median": statistics.median(routes), "p95": percentile(routes, .95),
"max": max(routes)},
"elapsed_s": round(time.monotonic() - started, 3),
"log": str(log.relative_to(ROOT)),
}
print("[L112CandidateLists] " + json.dumps(row, sort_keys=True), flush=True)
return row
def main() -> int:
if any(not path.is_file() for path in (*BINARIES.values(), DATA)):
raise SystemExit("missing L109/L112 candidate-list input")
rows = [run(strate, version) for strate in range(4)
for version in ("L109", "L112")]
(OUT / "summary.json").write_text(json.dumps(rows, indent=2) + "\n",
encoding="utf-8")
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,839 @@
#!/usr/bin/env python3
"""Capture L109 and L112 routes, every open-air section, and verified 4 m wall views."""
from __future__ import annotations
import json
import math
import re
from array import array
import struct
import subprocess
import sys
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[4]
RESEARCH = Path(__file__).resolve().parent
CAPTURES = RESEARCH / "captures"
SAVED = ROOT / "Saved/Linux/l112/captures"
DATA = ROOT / "Standalone/data/owner_world.json"
BINARIES = {
"before": ROOT / "Saved/Linux/l109/bin-final/VoxelForgeCoreHeadless-l109-clang",
"after": ROOT / "Saved/Linux/l112/bin-clang/VoxelForgeCoreHeadless-l112-clang",
}
LAYOUTS = {
"before": ROOT / "Saved/Linux/l109/audit-layout-final-stamp.json",
"after": ROOT / "Saved/Linux/l112/audit-layout-candidate.json",
}
AUDIT_LOGS = {
"before": ROOT / "Saved/Linux/l109/audit-owner4-final-stamp.log",
"after": ROOT / "Saved/Linux/l112/audit-owner-clang.log",
}
RENDERERS = {
"before": ROOT / "Saved/Linux/l109/bin-final/VoxelForgeOffscreenCapture-l109-clang",
"after": ROOT / "Saved/Linux/l112/bin-clang/VoxelForgeOffscreenCapture-l112-clang",
}
SECTION = re.compile(
r"^\[PassageOpenAirSection\] id=(\S+) section=(\d+) start_m=([\d.]+) "
r"end_m=([\d.]+) samples=(\d+) midpoint_xyz_m=\(([^)]*)\) "
r"midpoint_radius_m=([\d.]+)", re.M)
DRAWN = re.compile(r"drawn_triangles=(\d+)")
QUALITY_METRICS = re.compile(r"stddev_luma=([\d.]+).*coverage_luma_gt12=([\d.]+)")
def vector(text: str) -> tuple[float, float, float]:
return tuple(float(value) for value in text.split(","))
def add(a, b):
return tuple(x + y for x, y in zip(a, b))
def sub(a, b):
return tuple(x - y for x, y in zip(a, b))
def mul(a, scale):
return tuple(x * scale for x in a)
def norm(a):
length = math.sqrt(sum(x * x for x in a))
if length <= 1.0e-9:
return (0.0, 1.0, 0.0)
return tuple(value / length for value in a)
def angles(camera, target):
delta = sub(target, camera)
horizontal = math.hypot(delta[0], delta[1])
return (math.degrees(math.atan2(delta[0], delta[1])),
math.degrees(math.atan2(delta[2], horizontal)))
def fmt(values) -> str:
return ",".join(f"{value:.6f}" for value in values)
def mesh_triangle_count(mesh: Path) -> int:
"""Read only the VFMSH headers and count indexed triangles."""
with mesh.open("rb") as stream:
magic = stream.read(8)
materials = magic == b"VFMSH002"
if magic not in (b"VFMSH001", b"VFMSH002"):
raise RuntimeError(f"unsupported mesh scene header: {mesh}")
version, _voxel_um, tile_count = struct.unpack("<3I", stream.read(12))
if version != (2 if materials else 1):
raise RuntimeError(f"mesh scene version mismatch: {mesh}")
stream.seek(40, 1)
triangles = 0
for _ in range(tile_count):
stream.seek(16, 1)
surface_count, _skirts, vertex_count, index_count, mask_count = struct.unpack(
"<5I", stream.read(20))
triangles += index_count // 3
stream.seek(vertex_count * 24 + (10 * vertex_count if materials else 0)
+ index_count * 4 + mask_count, 1)
if stream.read(1):
raise RuntimeError(f"trailing data in mesh scene: {mesh}")
return triangles
def dot(a, b):
return sum(x * y for x, y in zip(a, b))
def closest_point_triangle(point, a, b, c):
def closest_segment(start, end, first_weight, second_weight):
edge = sub(end, start)
length_squared = dot(edge, edge)
t = (min(1.0, max(0.0, dot(sub(point, start), edge) / length_squared))
if length_squared > 1.0e-16 else 0.0)
position = add(start, mul(edge, t))
weights = [0.0, 0.0, 0.0]
weights[first_weight], weights[second_weight] = 1.0 - t, t
return position, tuple(weights)
ab, ac = sub(b, a), sub(c, a)
cross = ((ab[1] * ac[2] - ab[2] * ac[1]),
(ab[2] * ac[0] - ab[0] * ac[2]),
(ab[0] * ac[1] - ab[1] * ac[0]))
if dot(cross, cross) <= 1.0e-16:
candidates = [closest_segment(a, b, 0, 1),
closest_segment(b, c, 1, 2),
closest_segment(c, a, 2, 0)]
return min(candidates, key=lambda candidate:
dot(sub(candidate[0], point), sub(candidate[0], point)))
ap = sub(point, a)
d1, d2 = dot(ab, ap), dot(ac, ap)
if d1 <= 0.0 and d2 <= 0.0:
return a, (1.0, 0.0, 0.0)
bp = sub(point, b)
d3, d4 = dot(ab, bp), dot(ac, bp)
if d3 >= 0.0 and d4 <= d3:
return b, (0.0, 1.0, 0.0)
vc = d1 * d4 - d3 * d2
if vc <= 0.0 and d1 >= 0.0 and d3 <= 0.0:
denominator = d1 - d3
if denominator <= 1.0e-16:
return a, (1.0, 0.0, 0.0)
v = d1 / denominator
return add(a, mul(ab, v)), (1.0 - v, v, 0.0)
cp = sub(point, c)
d5, d6 = dot(ab, cp), dot(ac, cp)
if d6 >= 0.0 and d5 <= d6:
return c, (0.0, 0.0, 1.0)
vb = d5 * d2 - d1 * d6
if vb <= 0.0 and d2 >= 0.0 and d6 <= 0.0:
denominator = d2 - d6
if denominator <= 1.0e-16:
return a, (1.0, 0.0, 0.0)
w = d2 / denominator
return add(a, mul(ac, w)), (1.0 - w, 0.0, w)
va = d3 * d6 - d5 * d4
if va <= 0.0 and d4 - d3 >= 0.0 and d5 - d6 >= 0.0:
denominator = (d4 - d3) + (d5 - d6)
if denominator <= 1.0e-16:
return b, (0.0, 1.0, 0.0)
w = (d4 - d3) / denominator
edge = sub(c, b)
return add(b, mul(edge, w)), (0.0, 1.0 - w, w)
face_denominator = va + vb + vc
if face_denominator <= 1.0e-16:
candidates = [closest_segment(a, b, 0, 1),
closest_segment(b, c, 1, 2),
closest_segment(c, a, 2, 0)]
return min(candidates, key=lambda candidate:
dot(sub(candidate[0], point), sub(candidate[0], point)))
denominator = 1.0 / face_denominator
v, w = vb * denominator, vc * denominator
weights = (1.0 - v - w, v, w)
closest = tuple(weights[0] * a[axis] + weights[1] * b[axis]
+ weights[2] * c[axis] for axis in range(3))
return closest, weights
def nearest_surface_anchor(mesh: Path, point, margin_m=200.0, *,
preferred_normal=None, minimum_alignment=-1.0) -> dict:
"""Return the nearest point on a VFMSH surface triangle and its normal."""
best_distance_sq = float("inf")
best_position = best_normal = None
with mesh.open("rb") as stream:
magic = stream.read(8)
materials = magic == b"VFMSH002"
if magic not in (b"VFMSH001", b"VFMSH002"):
raise RuntimeError(f"unsupported mesh scene header: {mesh}")
version, voxel_um, tile_count = struct.unpack("<3I", stream.read(12))
if version != (2 if materials else 1):
raise RuntimeError(f"mesh scene version mismatch: {mesh}")
stream.seek(40, 1)
for _ in range(tile_count):
key = struct.unpack("<4i", stream.read(16))
surface_count, _skirts, vertex_count, index_count, mask_count = struct.unpack(
"<5I", stream.read(20))
extent = (32 << key[3]) * voxel_um / 1_000_000.0
origin = (key[0] * extent, key[1] * extent, key[2] * extent)
if any(point[axis] < origin[axis] - margin_m or
point[axis] > origin[axis] + extent + margin_m for axis in range(3)):
stream.seek(vertex_count * 24 + (10 * vertex_count if materials else 0)
+ index_count * 4 + mask_count, 1)
continue
positions = array("f")
positions.frombytes(stream.read(vertex_count * 12))
normals = array("f")
normals.frombytes(stream.read(vertex_count * 12))
if materials:
stream.seek(vertex_count * 10, 1)
indices = array("I")
indices.frombytes(stream.read(index_count * 4))
stream.seek(mask_count, 1)
if struct.pack("=I", 1) != struct.pack("<I", 1):
positions.byteswap()
normals.byteswap()
indices.byteswap()
def vertex(index):
offset = index * 3
return tuple(float(positions[offset + axis]) for axis in range(3))
def vertex_normal(index):
offset = index * 3
return tuple(float(normals[offset + axis]) for axis in range(3))
for index in range(0, index_count - 2, 3):
triangle = (indices[index], indices[index + 1], indices[index + 2])
if any(vertex_index >= surface_count or vertex_index >= vertex_count
for vertex_index in triangle):
continue
a, b, c = (vertex(vertex_index) for vertex_index in triangle)
position, weights = closest_point_triangle(point, a, b, c)
delta = sub(position, point)
distance_sq = dot(delta, delta)
if distance_sq >= best_distance_sq:
continue
raw_normals = [vertex_normal(vertex_index) for vertex_index in triangle]
raw_normal = tuple(sum(weights[corner] * raw_normals[corner][axis]
for corner in range(3)) for axis in range(3))
if dot(raw_normal, raw_normal) < 1.0e-10:
raw_normal = (
(b[1] - a[1]) * (c[2] - a[2]) - (b[2] - a[2]) * (c[1] - a[1]),
(b[2] - a[2]) * (c[0] - a[0]) - (b[0] - a[0]) * (c[2] - a[2]),
(b[0] - a[0]) * (c[1] - a[1]) - (b[1] - a[1]) * (c[0] - a[0]))
if dot(raw_normal, raw_normal) < 1.0e-10:
continue
candidate_normal = norm(raw_normal)
if preferred_normal is not None:
alignment = dot(candidate_normal, preferred_normal)
if abs(alignment) < minimum_alignment:
continue
if alignment < 0.0:
candidate_normal = mul(candidate_normal, -1.0)
best_distance_sq = distance_sq
best_position = position
best_normal = candidate_normal
if best_position is None:
raise RuntimeError(f"no nearby surface vertices in mesh scene: {mesh}")
return {"position_m": best_position, "normal": best_normal,
"distance_m": math.sqrt(best_distance_sq)}
def sanitize(value: str) -> str:
return re.sub(r"[^A-Za-z0-9_-]+", "-", value).strip("-").replace("/", "-")
def route_height_at_xy(route, x, y):
if not route:
return None
if len(route) == 1:
point = route[0]
return point[2], (x - point[0]) ** 2 + (y - point[1]) ** 2
best = (float("inf"), None)
for a, b in zip(route, route[1:]):
dx, dy = b[0] - a[0], b[1] - a[1]
length_sq = dx * dx + dy * dy
t = min(1.0, max(0.0, ((x - a[0]) * dx + (y - a[1]) * dy) / length_sq)) \
if length_sq > 1.0e-12 else 0.0
px, py = a[0] + t * dx, a[1] + t * dy
distance_sq = (x - px) ** 2 + (y - py) ** 2
if distance_sq < best[0]:
best = (distance_sq, a[2] + (b[2] - a[2]) * t)
return best[1], best[0]
def upper_mouth_floor_depth(layout: dict, passage: dict, controls) -> float:
scale = layout["core_voxel_size_um"] / 256.0 / 1_000_000.0
authored = max(0.0, controls[0][0][2] - passage["upper_mouth_q"][2] * scale + 1.13)
stratum = next((row for row in layout["strates"]
if row["strate_index"] == passage["upper_strate_index"]), None)
if stratum is None or stratum["generator_kind"] != "FloatingIslands":
return authored
route = [tuple(value * scale for value in anchor["values"][:3])
for anchor in stratum.get("anchors", [])
if anchor.get("kind") == "route_point" and len(anchor.get("values", [])) >= 3]
projected = route_height_at_xy(route, controls[0][0][0], controls[0][0][1])
if projected is None or projected[1] > (controls[0][1] + 2.0) ** 2:
return authored
route_depth = max(0.0, controls[0][0][2] - projected[0] + 1.13)
return max(authored, route_depth)
def route_controls(layout: dict, passage: dict, *, adjust_upper_mouth=False):
scale = layout["core_voxel_size_um"] / 256.0 / 1_000_000.0
controls = [(tuple(value * scale for value in row["position_q"]),
row["radius_q"] * scale) for row in passage["control_points"]]
lower_q = passage["lower_mouth_q"]
player_center_above_floor_q = round(
1.13 * 256.0 * 1_000_000.0 / layout["core_voxel_size_um"])
lower_index = next((index for index, row in enumerate(passage["control_points"])
if row["position_q"][0] == lower_q[0]
and row["position_q"][1] == lower_q[1]
and row["position_q"][2] - row["radius_q"]
+ player_center_above_floor_q == lower_q[2]), None)
if lower_index is None:
raise RuntimeError(f"{passage['stable_id']}: lower mouth control is unbound")
if lower_index < 1:
raise RuntimeError(f"{passage['stable_id']}: incomplete route controls")
if adjust_upper_mouth:
controls[0] = (controls[0][0], max(controls[0][1],
upper_mouth_floor_depth(layout, passage, controls)))
return controls, lower_index
def route_for(layout: dict, passage: dict) -> list[tuple[tuple[float, float, float], float]]:
controls, lower_index = route_controls(layout, passage)
return controls[:lower_index + 1]
def spline_knot_span(a, b):
return max(1.0e-6, math.sqrt(math.sqrt((b[0] - a[0]) ** 2
+ (b[1] - a[1]) ** 2)))
def spline_point(controls, span, t):
as_point = lambda index: (*controls[index][0], controls[index][1])
p0, p1, p2, p3 = (as_point(max(0, span - 1)), as_point(span),
as_point(span + 1), as_point(min(span + 2, len(controls) - 1)))
dt0, dt1, dt2 = (spline_knot_span(p0, p1), spline_knot_span(p1, p2),
spline_knot_span(p2, p3))
t2, t3 = t * t, t * t * t
result = [0.0] * 4
for axis in (0, 1, 3):
m1 = dt1 * ((p1[axis] - p0[axis]) / dt0
- (p2[axis] - p0[axis]) / (dt0 + dt1)
+ (p2[axis] - p1[axis]) / dt1)
m2 = dt1 * ((p2[axis] - p1[axis]) / dt1
- (p3[axis] - p1[axis]) / (dt1 + dt2)
+ (p3[axis] - p2[axis]) / dt2)
result[axis] = ((2 * t3 - 3 * t2 + 1) * p1[axis]
+ (-2 * t3 + 3 * t2) * p2[axis]
+ (t3 - 2 * t2 + t) * m1 + (t3 - t2) * m2)
result[2] = p1[2] + (p2[2] - p1[2]) * t
result[3] = min(max(result[3], min(p1[3], p2[3])), max(p1[3], p2[3]))
return result
def smooth_route(layout: dict, passage: dict):
"""Mirror PassageSpline::BuildSegments from the runtime shared sampler."""
scale = layout["core_voxel_size_um"] / 256.0 / 1_000_000.0
planned_floor_profile = [
(row["position_q"][2] - row["radius_q"]) * scale
for row in passage["control_points"]]
controls, route_end = route_controls(layout, passage, adjust_upper_mouth=True)
result = []
along = 0.0
arc_steps = 32
for span in range(route_end):
arc = [0.0]
previous = spline_point(controls, span, 0.0)
for step in range(1, arc_steps + 1):
current = spline_point(controls, span, step / arc_steps)
arc.append(arc[-1] + math.hypot(current[0] - previous[0],
current[1] - previous[1]))
previous = current
segment_count = max(1, math.ceil(arc[-1]))
points = [spline_point(controls, span, 0.0)]
parameters = [0.0]
for segment in range(1, segment_count + 1):
target = min(arc[-1], arc[-1] * segment / segment_count)
upper = next(index for index in range(1, len(arc))
if arc[index] >= target)
lower = upper - 1
width = arc[upper] - arc[lower]
fraction = min(1.0, max(0.0, (target - arc[lower]) / width)) if width > 1e-12 else 0.0
t = (lower + fraction) / arc_steps
points.append(spline_point(controls, span, t))
parameters.append(t)
sampled_arc = [0.0]
for a, b in zip(points, points[1:]):
sampled_arc.append(sampled_arc[-1] + math.hypot(b[0] - a[0], b[1] - a[1]))
horizontal_length = sampled_arc[-1]
for index, point in enumerate(points):
progress = sampled_arc[index] / horizontal_length if horizontal_length > 1e-12 else parameters[index]
point[2] = controls[span][0][2] + (controls[span + 1][0][2]
- controls[span][0][2]) * progress
point.append(planned_floor_profile[span]
+ (planned_floor_profile[span + 1]
- planned_floor_profile[span]) * progress)
if not result:
result.append((tuple(points[0][:3]), points[0][3], 0.0, points[0][4]))
for a, b in zip(points, points[1:]):
length = math.dist(a[:3], b[:3])
along += length
result.append((tuple(b[:3]), b[3], along, b[4]))
return result
def derived_smooth_route(route, passage, layout):
total = route[-1][2]
interior_radii = [radius for _, radius, distance, _ in route
if distance >= 10.0 and total - distance >= 10.0]
body_depth = max(0.5, min(interior_radii or [radius for _, radius, _, _ in route]) - 0.23)
result = []
for point, radius, distance, planned_floor in route:
local_radius_depth = max(0.0, radius - 0.23)
tube_depth = min(body_depth, local_radius_depth)
center = (point[0], point[1], planned_floor + tube_depth)
result.append((center, radius, distance, planned_floor))
return result, body_depth
def route_lengths(route):
lengths = [((b[2] - a[2]) if len(a) > 2 and len(b) > 2
else math.dist(a[0], b[0])) for a, b in zip(route, route[1:])]
prefix = [0.0]
for length in lengths:
prefix.append(prefix[-1] + length)
return lengths, prefix
def at_distance(route, lengths, prefix, distance):
distance = max(0.0, min(distance, prefix[-1]))
for index, length in enumerate(lengths):
if distance <= prefix[index + 1] or index + 1 == len(lengths):
t = 0.0 if length <= 1.0e-9 else (distance - prefix[index]) / length
a, b = route[index], route[index + 1]
return (tuple(a[0][axis] + (b[0][axis] - a[0][axis]) * t
for axis in range(3)),
a[1] + (b[1] - a[1]) * t)
return route[-1]
def tangent_at(route, lengths, prefix, distance):
before = at_distance(route, lengths, prefix, max(0.0, distance - 0.25))[0]
after = at_distance(route, lengths, prefix, min(prefix[-1], distance + 0.25))[0]
return norm(sub(after, before))
def derived_route(route, passage, layout):
"""Apply L109's fixed-depth floor offset to raw layout controls."""
lengths, prefix = route_lengths(route)
total = prefix[-1]
body_radii = [radius for index, (_, radius) in enumerate(route)
if prefix[index] >= 10.0 and total - prefix[index] >= 10.0]
body_depth = max(0.5, min(body_radii or [radius for _, radius in route]) - 0.23)
scale = layout["core_voxel_size_um"] / 256.0 / 1_000_000.0
upper_z = passage["upper_mouth_q"][2] * scale
lower_z = passage["lower_mouth_q"][2] * scale
upper_target = max(0.0, route[0][0][2] - upper_z + 1.13)
lower_target = max(0.0, route[-1][0][2] - lower_z + 1.13)
transformed = []
for index, (point, radius) in enumerate(route):
local_radius_depth = max(0.0, radius - 0.23)
mouth_distance = max(0.0, min(prefix[index], total - prefix[index]))
u = max(0.0, min(1.0, mouth_distance / 5.0))
ramp = u * u * (3.0 - 2.0 * u)
target = upper_target if prefix[index] <= total * 0.5 else lower_target
mouth_depth = max(0.0, min(radius, target))
route_depth = mouth_depth + (local_radius_depth - mouth_depth) * ramp
tube_depth = min(body_depth, local_radius_depth)
transformed.append(((point[0], point[1], point[2] - route_depth + tube_depth),
radius))
return transformed, body_depth
def make_views(layout: dict, audit_sections: list[dict], *, use_derived_shape: bool,
use_spline: bool,
include_open_air: bool = True) -> list[dict]:
views = []
passages = sorted(layout["passages"], key=lambda item: item["stable_id"])
for passage in passages:
raw_route = smooth_route(layout, passage) if use_spline else route_for(layout, passage)
if use_derived_shape:
route, body_depth = (derived_smooth_route(raw_route, passage, layout)
if use_spline else derived_route(raw_route, passage, layout))
else:
route = raw_route
body_depth = max(point[1] for point in route)
def floor_depth(radius: float) -> float:
if not use_derived_shape:
return radius
return min(body_depth, max(0.0, radius - 0.23))
lengths, prefix = route_lengths(route)
total = prefix[-1]
passage_id = passage["stable_id"]
safe_id = sanitize(passage_id)
for end_name, distance, outward_sign in (
("upper-mouth", 0.0, -1.0), ("lower-mouth", total, 1.0)):
center, radius = at_distance(route, lengths, prefix, distance)
tangent = tangent_at(route, lengths, prefix, distance)
outward = mul(tangent, outward_sign)
floor_z = center[2] - floor_depth(radius)
eye = (center[0] + outward[0] * 4.5,
center[1] + outward[1] * 4.5,
floor_z + 1.55)
if end_name == "lower-mouth":
eye = add(eye, add(mul(outward, 5.0), (0.0, 0.0, 2.0)))
target = (center[0] - outward[0] * 2.0,
center[1] - outward[1] * 2.0,
floor_z + 1.45)
yaw, pitch = angles(eye, target)
views.append({"name": f"{safe_id}-{end_name}", "passage": passage_id,
"kind": "mouth", "station": end_name,
"camera": eye, "target": target, "yaw": yaw, "pitch": pitch,
"focus": center, "center_z": center[2]})
for fraction in (0.25, 0.50, 0.75):
distance = total * fraction
center, radius = at_distance(route, lengths, prefix, distance)
tangent = tangent_at(route, lengths, prefix, distance)
depth = floor_depth(radius)
floor_z = center[2] - depth
horizontal = norm((-tangent[1], tangent[0], 0.0))
eye = (center[0] + horizontal[0] * 0.75,
center[1] + horizontal[1] * 0.75, floor_z + 1.60)
target_center, target_radius = at_distance(route, lengths, prefix,
min(total, distance + 14.0))
target_depth = floor_depth(target_radius)
target = (target_center[0] + horizontal[0] * 2.35,
target_center[1] + horizontal[1] * 2.35,
target_center[2] - target_depth + 1.80)
yaw, pitch = angles(eye, target)
views.append({"name": f"{safe_id}-station-{int(fraction * 100)}",
"passage": passage_id, "kind": "station",
"station": f"{int(fraction * 100)}%", "camera": eye,
"target": target, "yaw": yaw, "pitch": pitch,
"focus": center, "center_z": center[2],
"route_distance_m": distance,
"route_length_m": total,
"requested_route_distance_m": distance})
if not include_open_air:
continue
for section in (item for item in audit_sections if item["passage"] == passage_id):
distance = 0.5 * (section["start_m"] + section["end_m"])
center, radius = (section["midpoint"], section["radius_m"])
tangent = tangent_at(route, lengths, prefix, distance)
side = norm((-tangent[1], tangent[0], 0.0))
depth = floor_depth(radius)
section_id = section["section"]
interior_eye = (center[0], center[1], center[2] - depth + 1.25)
interior_target = add(interior_eye, mul(tangent, 7.0))
side_eye = add(center, add(mul(side, radius + 5.0), (0.0, 0.0, 1.2)))
oblique_eye = add(center, add(mul(side, -(radius + 3.5)), (0.0, 0.0, 4.0)))
targets = (add(center, (0.0, 0.0, 0.1)),
add(center, mul(tangent, 2.0)), interior_target)
for view_index, (eye, target) in enumerate(zip(
(side_eye, oblique_eye, interior_eye), targets), start=1):
yaw, pitch = angles(eye, target)
views.append({"name": f"{safe_id}-openair-{section_id}-view-{view_index}",
"passage": passage_id, "kind": "open-air",
"station": f"section-{section_id}/view-{view_index}",
"camera": eye, "target": target, "yaw": yaw,
"pitch": pitch, "focus": center,
"center_z": center[2], "open_air_section": section_id,
"route_distance_m": distance,
"requested_route_distance_m": distance,
"route_length_m": total})
floor_z = center[2] - depth
outside_target = add(center, add(mul(side, radius), (0.0, 0.0, -depth + 0.5)))
outside_eye = add(outside_target, mul(side, 4.0))
interior_radius = math.sqrt(max(0.0, radius * radius
- max(0.0, depth - 0.5) ** 2))
inside_target = add(center, add(mul(side, interior_radius),
(0.0, 0.0, -depth + 0.5)))
inside_eye = add(inside_target,
add(mul(side, -1.0), mul(tangent, math.sqrt(15.0))))
for closeup, eye, target in (("outside", outside_eye, outside_target),
("interior-wall", inside_eye, inside_target)):
yaw, pitch = angles(eye, target)
views.append({"name": f"{safe_id}-openair-{section_id}-4m-{closeup}",
"passage": passage_id, "kind": "open-air-close",
"station": f"section-{section_id}/4m-{closeup}",
"camera": eye, "target": target, "yaw": yaw,
"pitch": pitch, "focus": center, "center_z": center[2],
"open_air_section": section_id, "closeup_side": closeup,
"closeup_preferred_normal": side if closeup == "outside"
else mul(side, -1.0),
"camera_target_distance_m": math.dist(eye, target),
"route_distance_m": distance,
"requested_route_distance_m": distance,
"route_length_m": total})
return views
def run_process(name: str, command: list[str], log: Path, *, allow_failure=False) -> str:
wrapped = ["taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
"timeout", "1740s", *map(str, command)]
started = time.monotonic()
with log.open("w", encoding="utf-8") as stream:
result = subprocess.run(wrapped, cwd=ROOT, stdout=stream,
stderr=subprocess.STDOUT, check=False, timeout=1740)
if result.returncode and not allow_failure:
raise RuntimeError(f"{name}: exit={result.returncode} log={log}")
print(f"[L112CaptureProcess] name={name} elapsed_s={time.monotonic()-started:.2f} "
f"exit={result.returncode} log={log.relative_to(ROOT)}", flush=True)
return log.read_text(encoding="utf-8", errors="replace")
def capture(variant: str, view: dict, layout: dict) -> dict:
variant_saved = SAVED / variant
variant_saved.mkdir(parents=True, exist_ok=True)
variant_capture = CAPTURES / variant
variant_capture.mkdir(parents=True, exist_ok=True)
scene = variant_saved / f"{view['name']}.vfmesh"
image = variant_capture / f"{view['name']}.png"
candidates = [(view["camera"], view["target"], view["focus"],
view.get("route_distance_m"), 0.0, "requested")]
if variant == "before" and view["kind"] not in ("mouth", "open-air-close"):
passage = next(item for item in layout["passages"]
if item["stable_id"] == view["passage"])
route = route_for(layout, passage)
lengths, prefix = route_lengths(route)
total = prefix[-1]
eye_offset = sub(view["camera"], view["focus"])
target_offset = sub(view["target"], view["focus"])
requested = view["route_distance_m"]
for delta in (5, -5, 10, -10, 20, -20, 35, -35, 50, -50, 75, -75,
100, -100, 150, -150, 200, -200):
distance = max(0.0, min(total, requested + delta))
if abs(distance - requested) < 0.01:
continue
center, _radius = at_distance(route, lengths, prefix, distance)
candidates.append((add(center, eye_offset), add(center, target_offset), center,
distance, distance - requested, "route-offset"))
attempt_errors = []
for attempt_index, (probe_camera, probe_target, probe_focus, route_distance,
route_adjustment, attempt_kind) in enumerate(candidates):
suffix = "" if attempt_index == 0 else f"-attempt-{attempt_index:02d}"
headless_log = variant_saved / f"{view['name']}{suffix}-mesh.log"
render_log = variant_saved / f"{view['name']}{suffix}-render.log"
quality_log = variant_saved / f"{view['name']}{suffix}-quality.log"
yaw, pitch = angles(probe_camera, probe_target)
camera = (*probe_camera, yaw, pitch)
export_command = [
str(BINARIES[variant]), str(DATA), "--world-assembler-prototype",
"--stream-benchmark", "--stream-pipeline", "--workers=12",
"--clip-radius=3", "--max-clip-level=0", "--max-level=0", "--force-generate",
"--recipe=surface-world", f"--camera={fmt(camera)}",
"--render-distance-chunks=1", f"--mesh-export={scene}",
]
export_output = run_process(f"{variant}-{view['name']}{suffix}-mesh", export_command,
headless_log)
if "status=PASS" not in export_output or not scene.is_file():
raise RuntimeError(f"{variant}-{view['name']}: missing successful scene export")
source_triangles = mesh_triangle_count(scene)
if source_triangles <= 0:
attempt_errors.append(f"{attempt_kind}@{route_distance}: empty mesh")
if variant == "after" or view["kind"] == "mouth":
break
continue
actual_camera, actual_target = probe_camera, probe_target
actual_focus = probe_focus
surface_distance = None
closeup_surface_distance = None
closeup_anchor_offset = None
closeup_normal_alignment = None
closeup_distance = None
if view["kind"] == "open-air-close":
try:
wall_anchor = nearest_surface_anchor(scene, probe_target, margin_m=5.0,
preferred_normal=view["closeup_preferred_normal"],
minimum_alignment=0.25)
closeup_surface_distance = wall_anchor["distance_m"]
closeup_anchor_offset = closeup_surface_distance
closeup_normal_alignment = dot(wall_anchor["normal"],
view["closeup_preferred_normal"])
if closeup_surface_distance > 5.0:
raise RuntimeError(f"{variant}-{view['name']}: nearest wall surface is "
f"{closeup_surface_distance:.3f} m from the route-local target")
actual_target = wall_anchor["position_m"]
actual_camera = add(actual_target, mul(wall_anchor["normal"], 4.0))
closeup_distance = math.dist(actual_camera, actual_target)
if abs(closeup_distance - 4.0) > 0.02:
raise RuntimeError(f"{variant}-{view['name']}: camera-to-wall distance is "
f"{closeup_distance:.3f} m, expected 4 m")
except RuntimeError as error:
raise RuntimeError(f"{variant}-{view['name']}: could not verify the 4 m wall "
f"view: {error}") from error
if attempt_kind == "route-offset":
anchor = nearest_surface_anchor(scene, probe_focus)
surface_distance = anchor["distance_m"]
actual_camera = add(anchor["position_m"], mul(anchor["normal"], 12.0))
actual_target = anchor["position_m"]
actual_focus = anchor["position_m"]
actual_yaw, actual_pitch = angles(actual_camera, actual_target)
render_command = [
str(RENDERERS[variant]), str(scene), str(image), "--size=1920x1080",
f"--camera-m={fmt(actual_camera)}", f"--yaw-deg={actual_yaw:.6f}",
f"--pitch-deg={actual_pitch:.6f}", "--shade=readable", "--headlamp=on",
"--height-tint=on", f"--height-center-m={actual_focus[2]:.6f}",
"--height-range-m=32",
]
render_output = run_process(f"{variant}-{view['name']}{suffix}-render", render_command,
render_log)
drawn = DRAWN.search(render_output)
if drawn is None or int(drawn.group(1)) <= 0 or not image.is_file():
attempt_errors.append(f"{attempt_kind}@{route_distance}: no drawn geometry")
if variant == "after" or view["kind"] == "mouth":
break
continue
quality = run_process(f"{variant}-{view['name']}{suffix}-quality", [
"python3", "-B", str(ROOT / "Standalone/Tools/verify/capture_quality.py"),
str(image)], quality_log, allow_failure=True)
quality_status = "pass"
if "status=PASS" not in quality:
metrics = QUALITY_METRICS.search(quality)
coverage = float(metrics.group(2)) if metrics else 0.0
if coverage >= 0.001:
quality_status = "low_contrast_geometry_visible"
else:
attempt_errors.append(f"{attempt_kind}@{route_distance}: no visible pixel coverage")
if variant == "after" or view["kind"] == "mouth":
break
continue
row = {**view, "variant": variant, "capture": str(image.relative_to(RESEARCH)),
"scene": str(scene.relative_to(ROOT)), "drawn_triangles": int(drawn.group(1)),
"source_triangles": source_triangles, "verified": True,
"pixel_quality_status": quality_status,
"route_distance_actual_m": route_distance,
"route_station_adjustment_m": route_adjustment,
"closeup_camera_target_distance_m": closeup_distance,
"closeup_target_surface_distance_m": closeup_surface_distance,
"closeup_predicted_target_offset_m": closeup_anchor_offset,
"closeup_wall_normal_alignment": closeup_normal_alignment,
"fallback_kind": attempt_kind if attempt_kind != "requested" else None,
"fallback_surface_distance_m": surface_distance,
"camera_actual_m": actual_camera, "target_actual_m": actual_target,
"logs": [str(headless_log.relative_to(ROOT)),
str(render_log.relative_to(ROOT)),
str(quality_log.relative_to(ROOT))]}
print("[L112Capture] " + json.dumps(row, sort_keys=True), flush=True)
return row
raise RuntimeError(f"{variant}-{view['name']}: no verified image; "
+ "; ".join(attempt_errors))
def main() -> int:
if any(not path.is_file() for path in (*BINARIES.values(), *RENDERERS.values(),
*LAYOUTS.values(), *AUDIT_LOGS.values())):
raise SystemExit("missing L109 capture binary, layout, or audit log")
all_rows = []
variants = ("before", "after")
if "--after-only" in sys.argv[1:]:
prior_log = SAVED.parent / "captures-before-first.log"
if not prior_log.is_file():
raise SystemExit(f"missing reusable L109 capture rows: {prior_log}")
for line in prior_log.read_text(encoding="utf-8", errors="replace").splitlines():
if line.startswith("[L112Capture] ") or line.startswith("[L109Capture] "):
all_rows.append(json.loads(line.split(" ", 1)[1]))
if not all_rows or any(row["variant"] != "before" for row in all_rows):
raise SystemExit("saved L109 capture rows are incomplete")
variants = ("after",)
sections_by_variant = {}
for variant in ("before", "after"):
sections = []
for passage, section, start, end, samples, midpoint, radius in SECTION.findall(
AUDIT_LOGS[variant].read_text(encoding="utf-8", errors="replace")):
sections.append({"passage": passage, "section": int(section),
"start_m": float(start), "end_m": float(end),
"samples": int(samples), "midpoint": vector(midpoint),
"radius_m": float(radius)})
if not sections:
raise SystemExit(f"no open-air sections found in the {variant} passage audit")
sections_by_variant[variant] = sections
if variant not in variants:
continue
if variant == "after" and variants == ("after",):
expected_before = 15 + 5 * len(sections_by_variant["before"])
if len(all_rows) != expected_before or any(
row.get("variant") != "before" or not row.get("verified")
for row in all_rows):
raise RuntimeError("saved L109 capture rows do not cover every required view")
layout = json.loads(LAYOUTS[variant].read_text(encoding="utf-8"))
include_open_air = True
views = make_views(layout, sections, use_derived_shape=True,
use_spline=variant == "after",
include_open_air=include_open_air)
expected = 5 * len(layout["passages"])
if include_open_air:
expected += 5 * len(sections)
if len(views) != expected:
raise RuntimeError(f"unexpected camera count: {len(views)} != {expected}")
print(f"[L112CapturePlan] variant={variant} passage_views={5*len(layout['passages'])} "
f"open_air_sections={len(sections) if include_open_air else 0} "
f"open_air_views={5*len(sections) if include_open_air else 0} "
f"total={len(views)}", flush=True)
for view in views:
all_rows.append(capture(variant, view, layout))
CAPTURES.mkdir(parents=True, exist_ok=True)
(CAPTURES / "capture-index.json").write_text(
json.dumps({"open_air_sections_by_variant": sections_by_variant,
"captures": all_rows}, indent=2) + "\n",
encoding="utf-8")
lines = ["# L112 passage captures", "",
"Each PNG was rendered from its own exported mesh. The renderer reported visible "
"triangles, and each decoded image met the pixel coverage floor. "
"`low_contrast_geometry_visible` marks images that met this floor below the "
"preferred contrast target.", ""]
for variant in ("before", "after"):
lines.extend([f"## {variant.title()}", ""])
for row in all_rows:
if row["variant"] != variant:
continue
flag = " **OPEN AIR**" if row["kind"] == "open-air" else ""
quality_note = (f"; {row['pixel_quality_status']}"
if row["pixel_quality_status"] != "pass" else "")
lines.append(f"- [{row['station']}]({variant}/{Path(row['capture']).name}) — "
f"{row['passage']}{flag}; {row['drawn_triangles']} visible triangles"
f"{quality_note}")
lines.append("")
(CAPTURES / "INDEX.md").write_text("\n".join(lines), encoding="utf-8")
failures = [row for row in all_rows if not row["verified"]]
closeups = [row for row in all_rows if row["kind"] == "open-air-close"]
closeups_ok = (len(closeups) == 2 * sum(1 for row in all_rows
if row["kind"] == "open-air" and row["station"].endswith("view-1"))
and all(row["verified"] and abs(row["closeup_camera_target_distance_m"] - 4.0) <= 0.02
and row["closeup_target_surface_distance_m"] <= 5.0
and row["closeup_wall_normal_alignment"] >= 0.25
and row["drawn_triangles"] > 0
and row["pixel_quality_status"] in
("pass", "low_contrast_geometry_visible") for row in closeups))
print(f"[L112CaptureSummary] captures={len(all_rows)} verified={len(all_rows)-len(failures)} "
f"failed={len(failures)} 4m_closeups={len(closeups)} closeups_verified="
f"{str(closeups_ok).lower()}", flush=True)
return 1 if failures or not closeups_ok else 0
if __name__ == "__main__":
raise SystemExit(main())
Loaded 100 of 526 files, more files were not shown because too many files have changed in this diff. Show more