fix: signed-overflow UB in VoxelHash made cave placement compiler-dependent; all generators LOD0-8; speed matrix (L60)

VoxelHash::Cell/Pair, legacy Cellular3D and the shared Core cellular hash added 0x7FFFFFFF to a signed
int32 before casting to uint32; overflow is undefined, so Clang and GCC placed cave passage 1 differently
(first differing sample at LOD1 tile (0,64,-512)). The bias is now done in unsigned arithmetic, in both the
standalone copy and the Unreal plugin header (Source/VoxelForge/Public/VoxelCaveMorphology.h). Native
caves, motifs and lattice now stream LOD0-8 with skirts and cross-level transitions. New verify.py
--speed-matrix runs every generator on one asserted-identical view (5 interleaved runs): motifs 1.33,
profiles 1.30-1.32, caves 1.41, natural SW 1.67, lattice 1.70, FI 4.93 s whole view. fast/medium/full PASS,
Clang/GCC peer hashes equal. Research R29 (open-edge ledge) and R30 (compiler-divergence suspects).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1

Standalone/Research/WorldAssemblerM3/Linux60/CaptureFarLODsL60.py

@@ -0,0 +1,219 @@
+#!/usr/bin/env python3
+"""Render cave, motif, and lattice features from meshes streamed at LOD4-8."""
+from __future__ import annotations
+
+import argparse
+import hashlib
+import importlib.util
+import json
+import math
+import os
+from pathlib import Path
+import re
+import struct
+import sys
+
+
+ROOT = Path(__file__).resolve().parents[4]
+CAPTURE54 = ROOT / "Standalone/Research/WorldAssemblerM3/Linux54/CaptureNativeCavesL54.py"
+spec = importlib.util.spec_from_file_location("capture_native_caves_l54", CAPTURE54)
+capture54 = importlib.util.module_from_spec(spec)
+assert spec and spec.loader
+spec.loader.exec_module(capture54)
+run_step = capture54.run_step
+camera_angles = capture54.camera_angles
+fmt = capture54.fmt
+QUALITY = ROOT / "Standalone/Tools/verify/capture_quality.py"
+Q = 1_000_000.0
+
+
+def sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as stream:
+        for block in iter(lambda: stream.read(1024 * 1024), b""):
+            digest.update(block)
+    return digest.hexdigest()
+
+
+def midpoint(points: list[tuple[float, float, float]]) -> tuple[float, float, float]:
+    return tuple(sum(point[axis] for point in points) / len(points) for axis in range(3))
+
+
+def read_feature(name: str, layout_path: Path):
+    layout = json.loads(layout_path.read_text(encoding="utf-8"))
+    if name == "caves":
+        edges = sorted(layout.get("cavern_edges", []), key=lambda row: row["stable_id"])
+        if len(edges) != 2:
+            raise ValueError("cave layout must have two resolved spine edges")
+        points = [[value / Q for value in point]
+                  for point in edges[0]["control_points_um"]]
+        target = tuple(points[-1][axis] for axis in range(3))
+        return target, target[2] + 2.0, 28.0, "cave mouth"
+    if name == "motifs":
+        monolith = next(row for row in layout.get("stamps", []) if row.get("kind") == 5)
+        target = tuple(value / Q for value in monolith["position_um"])
+        floor = monolith["floor_um"] / Q
+        return (target[0], target[1], floor + 12.0), floor + 12.0, 32.0, "monolith"
+    edges = layout.get("cavern_edges", [])
+    helix = next((edge for edge in edges
+        if edge.get("stable_id") == "lattice/spine/01-helix-ramp-v1"), None)
+    if helix is None:
+        raise ValueError("lattice layout has no resolved helix")
+    points = [tuple(value / Q for value in point) for point in helix["polyline_um"]]
+    target = midpoint(points)
+    return target, target[2] + 4.0, 36.0, "helix"
+
+
+def audit_roi_lods(mesh: Path, center: tuple[float, float, float], radius: float):
+    roi_counts: dict[int, int] = {}
+    roi_skirts: dict[int, int] = {}
+    with mesh.open("rb") as stream:
+        prefix = stream.read(60)
+        if len(prefix) != 60 or prefix[:8] != b"VFMSH001":
+            raise ValueError("far-LOD audit requires a VFMSH001 Core mesh")
+        version, voxel_um, tile_count = struct.unpack_from("<3I", prefix, 8)
+        if version != 1 or voxel_um <= 0:
+            raise ValueError("mesh version or voxel size is invalid")
+        for _ in range(tile_count):
+            header = stream.read(36)
+            if len(header) != 36:
+                raise ValueError("truncated far-LOD tile header")
+            key = struct.unpack_from("<4i", header, 0)
+            surface_count, skirt_count, vertex_count, index_count, mask_count = \
+                struct.unpack_from("<5I", header, 16)
+            positions = stream.read(vertex_count * 12)
+            normals = stream.read(vertex_count * 12)
+            index_bytes = stream.read(index_count * 4)
+            masks = stream.read(mask_count)
+            if (len(positions) != vertex_count * 12
+                    or len(normals) != vertex_count * 12
+                    or len(index_bytes) != index_count * 4
+                    or len(masks) != mask_count):
+                raise ValueError("truncated far-LOD tile payload")
+            lod = key[3]
+            coords = list(struct.iter_unpack("<3f", positions))
+            indices = struct.unpack(f"<{index_count}I", index_bytes) if index_count else ()
+            for triangle in range(index_count // 3):
+                ids = indices[triangle * 3:triangle * 3 + 3]
+                if max(ids) >= vertex_count:
+                    raise ValueError("far-LOD triangle references an invalid vertex")
+                is_skirt = bool(masks and triangle < len(masks) and masks[triangle])
+                if is_skirt:
+                    continue
+                centroid = tuple(sum(coords[index][axis] for index in ids) / 3.0
+                                 for axis in range(3))
+                distance_squared = sum((centroid[axis] - center[axis]) ** 2
+                                        for axis in range(3))
+                if distance_squared <= radius * radius:
+                    roi_counts[lod] = roi_counts.get(lod, 0) + 1
+            if skirt_count:
+                roi_skirts[lod] = roi_skirts.get(lod, 0) + skirt_count
+        if stream.read(1):
+            raise ValueError("far-LOD mesh has trailing bytes")
+    return roi_counts, roi_skirts
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--out", type=Path, required=True)
+    parser.add_argument("--fixture-dir", type=Path, required=True)
+    parser.add_argument("--layout-dir", type=Path, required=True)
+    parser.add_argument("--headless", type=Path, required=True)
+    parser.add_argument("--renderer", type=Path, required=True)
+    parser.add_argument("--artifact-dir", type=Path, required=True)
+    args = parser.parse_args()
+    out = args.out.resolve()
+    fixture_dir, layout_dir = args.fixture_dir.resolve(), args.layout_dir.resolve()
+    artifact_dir = args.artifact_dir.resolve()
+    mesh_dir, png_dir = out / "far-lod-meshes", out / "far-lod-png"
+    for directory in (mesh_dir, png_dir, artifact_dir):
+        directory.mkdir(parents=True, exist_ok=True)
+    env = os.environ.copy()
+    env["LP_NUM_THREADS"] = "12"
+    env["XDG_CACHE_HOME"] = str(out / "xdg-cache")
+
+    cases = (
+        ("caves", fixture_dir / "caves.json",
+         layout_dir / "native-caves-field-clang.json", "caves", []),
+        ("motifs", fixture_dir / "style-override-motifs-slice1.json",
+         layout_dir / "motif-slice1-capsule-reachability-clang.json", "surface-world", []),
+        ("lattice", fixture_dir / "lattice-l58.json",
+         layout_dir / "native-lattice-field-clang.json", "lattice", ["--lattice-strate=1"]),
+    )
+    quality_rows = []
+    for name, fixture, layout, recipe, extra_flags in cases:
+        if not fixture.is_file() or not layout.is_file():
+            raise RuntimeError(f"missing {name} fixture or layout")
+        target, target_height, radius, label = read_feature(name, layout)
+        stream_camera = (target[0] + 1024.0, target[1], target[2], 0.0, 0.0)
+        stream_yaw, stream_pitch = camera_angles(stream_camera[:3], target)
+        stream_camera = (*stream_camera[:3], stream_yaw, stream_pitch)
+        mesh = mesh_dir / f"{name}-lod4-8.vfmesh"
+        mesh_layout = layout_dir / f"{name}-far-lod-layout.json"
+        command = [args.headless, fixture, "--workers=12", "--world-assembler-prototype",
+            "--stream-benchmark", "--stream-pipeline", "--force-generate",
+            "--clip-radius=3", "--max-clip-level=8", "--max-level=-1",
+            "--render-distance-chunks=0", f"--recipe={recipe}",
+            f"--camera={fmt(stream_camera)}", *extra_flags,
+            f"--resolved-layout-output={mesh_layout}", f"--mesh-export={mesh}"]
+        output = run_step(f"{name}-far-lod-export", command,
+            out / f"capture-{name}-far-lod-export.log", env=env, stamped=True)
+        worker_rows = {int(match.group(1)): (int(match.group(2)), int(match.group(3)))
+            for match in re.finditer(
+                r"^\[WorldAssemblerLODWorker\] lod=(\d+) tiles=(\d+) .*?skirt_triangles=(\d+)",
+                output, re.M)}
+        if (set(worker_rows) != set(range(9))
+                or any(worker_rows[level][0] <= 0 for level in range(9))
+                or sum(worker_rows[level][1] for level in range(4, 9)) <= 0):
+            raise RuntimeError(f"{name} far view lacks LOD0-8 tile/skirt work: {worker_rows}")
+        if not mesh.is_file() or mesh.stat().st_size == 0:
+            raise RuntimeError(f"{name} far LOD export produced no mesh")
+        roi, roi_skirts = audit_roi_lods(mesh, target, radius)
+        feature_lods = sorted(level for level, count in roi.items() if count > 0)
+        far_feature_lods = [level for level in feature_lods if level >= 4]
+        if not far_feature_lods:
+            raise RuntimeError(f"{name} {label} has no non-skirt surface triangles in LOD4-8 ROI: {roi}")
+
+        camera = (target[0] + 18.0, target[1] - 28.0, target_height + 10.0)
+        yaw, pitch = camera_angles(camera, target)
+        png = png_dir / f"{name}-far-lod.png"
+        image = artifact_dir / f"{name}-far-lod.jpg"
+        render = [args.renderer, mesh, png, "--size=1280x720", "--fov-deg=76",
+            f"--camera-m={fmt(camera)}", f"--yaw-deg={yaw:.6f}",
+            f"--pitch-deg={pitch:.6f}", "--tile-skirts=on", "--shade=readable",
+            "--headlamp=on", "--height-tint=on", f"--height-center-m={target_height:.6f}",
+            "--height-range-m=42", "--tile-edges=off",
+            f"--focus-region={fmt((*target, radius))}"]
+        run_step(f"{name}-far-lod-render", render,
+            out / f"capture-{name}-far-lod-render.log", env=env, stamped=True)
+        run_step(f"{name}-far-lod-jpeg", ["ffmpeg", "-nostdin", "-hide_banner",
+            "-loglevel", "error", "-y", "-i", png, "-frames:v", "1", "-q:v", "5", image],
+            out / f"capture-{name}-far-lod-jpeg.log", env=env)
+        quality_output = run_step(f"{name}-far-lod-quality", [sys.executable, QUALITY, image],
+            out / f"capture-{name}-far-lod-quality.log", env=env)
+        quality = [line for line in quality_output.splitlines()
+                   if line.startswith("[CaptureQuality]")]
+        if not quality or not quality[-1].endswith("status=PASS"):
+            raise RuntimeError(f"{name} far LOD capture failed its image quality gate")
+        size = image.stat().st_size
+        if not 0 < size <= 300_000:
+            raise RuntimeError(f"{name} far LOD capture exceeds 300 KB: {size}")
+        print(f"[L60FarLODCapture] generator={name} feature={label} file={image} "
+            f"bytes={size} sha256={sha256(image)} feature_lods={feature_lods} "
+            f"feature_triangles={sum(roi.values())} far_skirt_triangles="
+            f"{sum(value for lod, value in roi_skirts.items() if lod >= 4)} status=PASS",
+            flush=True)
+        quality_rows.append({"generator": name, "feature": label, "file": str(image),
+            "bytes": size, "sha256": sha256(image), "feature_triangles_by_lod": roi,
+            "feature_skirts_by_lod": roi_skirts})
+    (out / "far-lod-captures.json").write_text(json.dumps(quality_rows, indent=2) + "\n",
+                                              encoding="utf-8")
+    return 0
+
+
+if __name__ == "__main__":
+    try:
+        raise SystemExit(main())
+    except Exception as error:
+        print(f"[L60FarLODCapture] status=FAIL error={error}", file=sys.stderr, flush=True)
+        raise SystemExit(1)

Standalone/Research/WorldAssemblerM3/Linux60/REPORT.md

@@ -0,0 +1,77 @@
+# Linux 60 report
+
+## Result and root cause
+
+The continuation is green. The full tier and speed matrix passed after fixing signed-overflow undefined behavior in the shared voxel hash.
+
+The first differing density was LOD1 tile `(0,64,-512)`, step 2, sample order 23,658 at `(64,82,-476)`. Before the fix, Clang returned `-4.0078125` (`0xc0804000`) and GCC returned `-4.125` (`0xc0840000`). The sample follows the mesher's LOD X8/boundary scalar path into `DensityAtLattice` and the compiled analytic `TunnelNetwork` field. Its upstream cave passage placement comes from `UVoxelStrateManager::GeneratePassages` → `PassageRandom01` → `VoxelHash::Cell(i,c,seed)`.
+
+`VoxelHash::Cell` added `0x7fffffff` to a signed cell coordinate before casting to `uint32`; connection `c=1` overflowed `int32`, so the compiler-dependent hash changed the passage coordinates and cave geometry. The trace showed passage pair `(0,1)` diverging, followed by all 21 control points of passage 1. The fix performs the bias in unsigned arithmetic in `Cell`, `Pair`, legacy `Cellular3D`, and the shared Core cellular hash. It does not quantize density. After the fix both compilers produced the same pair `(0,1)` coordinates, the same 44 passage points, and `0xc0804000` at the first sample. The evaluator version is now 211.
+
+Before/after traces are in `Saved/Linux/l60/diag/caves-{clang,gcc}-probe.log` and `Saved/Linux/l60/diag/cache/fullcache-{clang,gcc}.log`.
+
+## Far LOD support and seam checks
+
+All full-view fixtures use max level `-1`, clip radius 3, and max clip level 8. All three generators streamed LOD0–8 with skirts.
+
+- **Caves:** X8 and Z-run gates detect caves in manager and resolved stratum mappings, then use scalar lattice sampling for cave samples. Clang/GCC cave tile hashes match at W1/W8/W12, including random views at LOD1–2. The cave-mouth transition crossed levels 0–8 (432 entering tiles).
+- **Motifs:** The fixed motif field streams through the existing rings, caps, and skirts. The monolith transition crossed levels 0–8 (295 entering tiles); whole-surface foot, cap, and platform-edge audits passed.
+- **Lattice:** The fixture selects native lattice stratum 1 and uses the existing ring and skirt path. The helix transition crossed levels 0–8 (380 entering tiles); field, interval, reachability, and shape-free audits passed.
+
+Hard, gradient, and interleaved stratum transitions passed with zero hole columns and zero air samples. Far-LOD cave, motif, and lattice JPEG captures passed their audits and are under 300 KB.
+
+## Comparable speed matrix
+
+`verify.py full --speed-matrix` ran five interleaved repetitions per generator, rotating generator order. Every run used camera `(-12,-118,-20 m)`, rotation `(0°,0°)`, clip radius 3, max clip level 8, max level `-1`, camera step 24 m, 12 workers, and shared column cache. The stage asserted the shared HeadlessConfig fields, whole-view and per-LOD tile-count ranges, and stable counts across repetitions. It passed for all eight generators. Every listed tile count occurred in all five runs; timings are medians from the final full run.
+
+| Generator | Tiles | Whole view (s) | First tile (s) | Near ring (s) | Moving p50 (ms) | Moving p95 (ms) |
+|---|---:|---:|---:|---:|---:|---:|
+| Natural SW | 2,130 | 1.669 | 0.502 | 0.519 | 63.971 | 119.891 |
+| periodicHills | 1,986 | 1.311 | 0.454 | 0.479 | 60.267 | 115.648 |
+| mirroredRelief | 1,986 | 1.317 | 0.471 | 0.490 | 60.568 | 115.474 |
+| flatField | 1,986 | 1.303 | 0.465 | 0.485 | 60.627 | 117.144 |
+| Floating Islands | 2,130 | 4.926 | 0.817 | 0.977 | 528.939 | 871.290 |
+| Caves | 2,130 | 1.408 | 0.489 | 0.507 | 61.099 | 118.508 |
+| Motifs | 1,986 | 1.326 | 0.476 | 0.493 | 60.107 | 116.104 |
+| Lattice | 2,130 | 1.701 | 0.539 | 0.561 | 69.572 | 137.585 |
+
+Full run records and common config are in `Saved/Linux/l60/speed-matrix.json`.
+
+## Captures
+
+- `Saved/Linux/l60/captures/caves-far-lod.jpg` — 16,642 bytes; SHA-256 `55f8d3d2de76541e9f1c1d09b771af40bfbb37b1ce5afc0b2f70419b84ed71a3`
+- `Saved/Linux/l60/captures/motifs-far-lod.jpg` — 24,926 bytes; SHA-256 `bfc5d8ddebc3706fedd6108653b4777fac4bac44e71f60c8e58b3508c14c3aa6`
+- `Saved/Linux/l60/captures/lattice-far-lod.jpg` — 30,714 bytes; SHA-256 `c261728e48975f01c712e31594b4284eede289b73e793e6204b78cb8ff2a2947`
+
+## Hashes and tiers
+
+Clang 19 and GCC 14 builds used FP contraction off and 12 jobs. Final source stamp and source-manifest SHA-256: `902a0d8e45f895d1b53c0a86999adee5598ff77e1752aa7ea90397520cf7b82d`.
+
+- Clang headless: `dcf6286bcecac382f9cce2458e7c327485e9b8b993c3d544e627a8fc964947fa`
+- GCC headless: `be86f90f993ba27bfde235e0850ea783298396dc03456360cf4bb3fd4815988b`
+- Offscreen capture: `b8f7a441c0a41be6ebd61721ff903832a51291df7a886e669b9084d84441d4c5`
+
+| Tier | Status | Checks / runs | Elapsed |
+|---|---|---:|---:|
+| Fast | PASS before hash fix | 453 / 113 | 335.7 s |
+| Medium | PASS before hash fix | 968 / 289 | 1,559.1 s |
+| Full continuation trial 1 | FAIL: stale binary stamp; no checks ran | — | 0.0 s |
+| Full continuation trial 2 | PASS on source stamp `914af3...` | 2,304 / 660 | 3,256.9 s |
+| Full continuation trial 3 | PASS on final source stamp, including speed matrix | 2,304 / 660 | 3,260.6 s |
+
+Fast and medium were not rerun as standalone tiers after the hash change; the passing full run executed their checks at the current source stamp.
+
+## Trials and failed runs
+
+The prior L60 work used 15 trials. This continuation used 3 of its 10-trial budget. Trial 1 stopped before checks because it found stale Clang/GCC executables. Trial 2 passed, then the final source review restored the mesher's existing `<bit>` include and changed the source stamp, so trial 3 rebuilt and reran full against the exact final source. The pre-fix trial 15 had failed peer determinism on the same six LOD1 and eight LOD2 tile keys across four seeded views. Trial 3 had no failed checks; negative controls reported expected failures.
+
+## Orchestrator review (Claude)
+
+Accepted. The stamp matches the final Clang and GCC builds (`902a0d8e…`); full PASS, including the speed matrix.
+**The root cause is a real cross-platform determinism bug:** `VoxelHash::Cell`/`Pair`/`Cellular3D` added
+`0x7FFFFFFF` to a signed `int32` (signed overflow is undefined behaviour), so passage placement became
+compiler-dependent. **The same code lives in the Unreal plugin** (`Source/VoxelForge/Public/VoxelCaveMorphology.h`,
+identical to the standalone copy before the fix), and only the standalone copy had been fixed. I applied the identical
+fix to the plugin header and confirmed the two copies match. MSVC typically wraps there, so the owner's existing
+worlds are most likely unchanged, but this needs his Windows build. No other signed-bias pattern remains in `Source/`
+or Core. The comparable speed matrix is now a standing `verify.py full --speed-matrix` stage.