tools(verify): fast/medium/full verification tiers with a compiled-in source stamp (L39); research R6
BuildLinux.sh hashes a sorted manifest of compiled inputs into VOXELFORGE_SOURCE_STAMP, printed first by every executable; Standalone/Tools/verify/verify.py checks it before and after each tier and records binary/input hashes and the resolved recipe, style map and layout. Fast (163 s) runs on the canonical owner world only and asserts FI Style 2 / SW Style 1; medium (8 min) and full (26 min) add the style matrix, GCC, determinism and cost. Controls: stale source, mislabelled style-0 owner, all-black capture, L30/L31, L35/L36 and the L38 step are all rejected. No geometry change; owner hashes identical to L37. Research R6: leak table, generator contract and a 10-step metre-native migration plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
This commit is contained in:
Fr0zkaandClaude Opus 5.5 committed
1 parent
88c54eab44
commit
753cc9c259
22 files changed
+1831
-163
No files matched your search
@@ -0,0 +1,19 @@
|
||||
# VoxelForge verification
|
||||
|
||||
Build stamped Linux executables and run one of the three verification tiers from the repository root. Artifacts and logs go under `Saved/Linux/l39` by default.
|
||||
|
||||
```sh
|
||||
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s env VOXELFORGE_BUILD_ROOT=Saved/Linux/l39/build VOXELFORGE_BIN_DIR=Saved/Linux/l39/bin Standalone/BuildLinux.sh --compiler=clang++-19 --variant=l39-clang --jobs=12
|
||||
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s env VOXELFORGE_BUILD_ROOT=Saved/Linux/l39/build VOXELFORGE_BIN_DIR=Saved/Linux/l39/bin Standalone/BuildLinux.sh --compiler=g++ --variant=l39-gcc --jobs=12
|
||||
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s python3 -B Standalone/Tools/verify/verify.py fast
|
||||
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s python3 -B Standalone/Tools/verify/verify.py medium
|
||||
taskset -c 0-11 prlimit --as=6442450944 -- timeout 1740s python3 -B Standalone/Tools/verify/verify.py full
|
||||
```
|
||||
|
||||
`fast` checks only canonical `Standalone/data/owner_world.json`, asserts FI Style 2 and SW Style 1 from resolved snapshots, runs passage wall/floor ripple and coverage gates, the adjacent-profile step limit, W1/W12 hash comparisons, and LOD0 ring profiles with the L30/L31 controls. It also exercises stale-source, style-mislabel, black-capture, historical ripple, and L38 step controls.
|
||||
|
||||
`medium` adds the Clang style-override matrix, transitions, route/proof audits, LOD0–8 rings, and canonical-owner captures, shape, near/overview crease, and fragment checks. Run `medium` before `full`. Full reuses that evidence only when the source stamp, verification-tool hashes, binaries, inputs, logs, and audit-script hashes still match; otherwise it reruns the Clang medium work. It adds the owner/style/transition/proof matrix with GCC, the 30-cell determinism matrix, and five interleaved L36/L39 cost pairs per recipe. The final captures stay in the L39 output directory.
|
||||
|
||||
`BuildLinux.sh` hashes a sorted manifest of Core inputs, standalone sources and generated wrappers, build files, and optionally the Unreal adapter when `VOXELFORGE_INCLUDE_UNREAL_ADAPTER=1`. The manifest hash is compiled into the headless and offscreen binaries and printed before any other runtime log line. Each tier probes every available built executable at startup and re-probes it at completion, rejecting stale stamps or binary changes. Summaries store tool hashes, binary and JSON SHA-256 values, resolved recipe/style/layout identity, commands, elapsed time, and audit-script hashes.
|
||||
|
||||
Each tier stops at its first failed gate. Every process is limited to CPUs 0–11 and 6 GiB of address space; individual build and audit processes time out at 30 minutes. Summary files are `fast-summary.json`, `medium-summary.json`, and `full-summary.json`.
|
||||
@@ -0,0 +1,54 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Reject empty and nearly black captures by decoding pixels and measuring coverage."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import math
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
|
||||
LIMIT = ["taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--"]
|
||||
|
||||
|
||||
def inspect(path: Path) -> tuple[bool, str]:
|
||||
probe = subprocess.run([*LIMIT, "timeout", "30s", "ffmpeg", "-nostdin",
|
||||
"-hide_banner", "-loglevel", "error", "-i", str(path), "-frames:v", "1",
|
||||
"-f", "rawvideo", "-pix_fmt", "rgb24", "pipe:1"],
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=30, check=False)
|
||||
if probe.returncode != 0 or not probe.stdout or len(probe.stdout) % 3:
|
||||
return False, f"decode_failed exit={probe.returncode} bytes={len(probe.stdout)}"
|
||||
raw = probe.stdout
|
||||
count = len(raw) // 3
|
||||
total = 0
|
||||
square_total = 0
|
||||
covered = 0
|
||||
for index in range(0, len(raw), 3):
|
||||
luminance = (77 * raw[index] + 150 * raw[index + 1] + 29 * raw[index + 2]) >> 8
|
||||
total += luminance
|
||||
square_total += luminance * luminance
|
||||
covered += luminance > 12
|
||||
mean = total / count
|
||||
variance = max(0.0, square_total / count - mean * mean)
|
||||
deviation = math.sqrt(variance)
|
||||
coverage = covered / count
|
||||
passed = deviation >= 2.0 and coverage >= 0.001
|
||||
return passed, (f"pixels={count} mean_luma={mean:.3f} stddev_luma={deviation:.3f} "
|
||||
f"coverage_luma_gt12={coverage:.5f} thresholds=(stddev>=2.0,coverage>=0.001)")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("image", type=Path)
|
||||
args = parser.parse_args()
|
||||
if not args.image.is_file():
|
||||
print(f"[CaptureQuality] image={args.image} status=FAIL reason=missing")
|
||||
return 1
|
||||
passed, detail = inspect(args.image)
|
||||
print(f"[CaptureQuality] image={args.image} {detail} status={'PASS' if passed else 'FAIL'}")
|
||||
return 0 if passed else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,63 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Apply current ripple limits to retained L35 wall-rib and L36 washboard logs."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
import re
|
||||
|
||||
|
||||
def records(path: Path) -> list[dict[str, float | str]]:
|
||||
result = []
|
||||
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||
if "[PassageRippleAudit]" not in line:
|
||||
continue
|
||||
values = dict(re.findall(r"(?:^| )([a-z_]+)=([-0-9.]+)", line))
|
||||
def ratio(key: str) -> str:
|
||||
match = re.search(rf"(?:^| ){key}=([0-9]+)/([0-9]+)", line)
|
||||
return (str(int(match.group(1)) / max(1, int(match.group(2))))
|
||||
if match else "nan")
|
||||
try:
|
||||
item: dict[str, float | str] = {
|
||||
"local_peak_amplitude_m": float(values.get(
|
||||
"local_peak_amplitude_m", values.get("amplitude_m", "nan"))),
|
||||
"floor_amplitude_m": float(values.get("floor_amplitude_m", "nan")),
|
||||
"wall_coverage": float(ratio("coverage")),
|
||||
"floor_coverage": float(ratio("floor_coverage")),
|
||||
"status": "FAIL" if "status=FAIL" in line else "PASS",
|
||||
}
|
||||
except ValueError:
|
||||
continue
|
||||
result.append(item)
|
||||
return result
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("kind", choices=("wall-ribs", "floor-washboard"))
|
||||
parser.add_argument("log", type=Path)
|
||||
args = parser.parse_args()
|
||||
entries = records(args.log)
|
||||
if args.kind == "wall-ribs":
|
||||
candidates = [entry for entry in entries if
|
||||
float(entry["local_peak_amplitude_m"]) > 0.035
|
||||
or float(entry["wall_coverage"]) < 0.75]
|
||||
metric = max((float(entry["local_peak_amplitude_m"]) for entry in candidates),
|
||||
default=0.0)
|
||||
detail = f"max_wall_amplitude_m={metric:.4f} threshold_m=0.035"
|
||||
else:
|
||||
candidates = [entry for entry in entries if
|
||||
float(entry["floor_amplitude_m"]) > 0.015
|
||||
or float(entry["floor_coverage"]) < 0.75]
|
||||
metric = max((float(entry["floor_amplitude_m"]) for entry in candidates),
|
||||
default=0.0)
|
||||
detail = f"max_floor_amplitude_m={metric:.4f} threshold_m=0.015"
|
||||
detected = bool(candidates)
|
||||
print(f"[HistoricalRippleControl] kind={args.kind} log={args.log} "
|
||||
f"records={len(entries)} {detail} detected={str(detected).lower()} "
|
||||
f"status={'PASS' if detected else 'FAIL'}")
|
||||
return 0 if detected else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,68 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Measure adjacent route-relative mouth-profile steps; period fits are diagnostic."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
import re
|
||||
import sys
|
||||
|
||||
|
||||
PROFILE = re.compile(
|
||||
r"\[PassageFloorRippleSample\] id=(\S+) distance_m=([-0-9.]+).*?"
|
||||
r"residual_m=([-0-9.]+)")
|
||||
RIPPLE = re.compile(r"\[PassageRippleAudit\] id=(\S+).*?period_m=([-0-9.]+)")
|
||||
STEP_LIMIT_M = 0.100
|
||||
MAX_ADJACENT_DISTANCE_M = 0.750
|
||||
|
||||
|
||||
def measure(text: str) -> dict[str, object]:
|
||||
profiles: dict[str, list[tuple[float, float]]] = defaultdict(list)
|
||||
for match in PROFILE.finditer(text):
|
||||
passage, distance, residual = match.groups()
|
||||
profiles[passage].append((float(distance), float(residual)))
|
||||
|
||||
adjacent: list[tuple[float, str, float, float]] = []
|
||||
group_counts: dict[str, int] = {}
|
||||
for key, samples in profiles.items():
|
||||
samples.sort()
|
||||
group_counts[key] = len(samples)
|
||||
for before, after in zip(samples, samples[1:]):
|
||||
delta_d = abs(after[0] - before[0])
|
||||
if delta_d <= MAX_ADJACENT_DISTANCE_M:
|
||||
adjacent.append((abs(after[1] - before[1]), key, before[0], after[0]))
|
||||
|
||||
periods = {passage: float(period) for passage, period in RIPPLE.findall(text)}
|
||||
maximum = max((item[0] for item in adjacent), default=0.0)
|
||||
return {
|
||||
"maximum_adjacent_step_m": maximum,
|
||||
"adjacent_pairs": len(adjacent),
|
||||
"profile_groups": len(profiles),
|
||||
"profile_group_counts": group_counts,
|
||||
"fitted_periods_m": periods,
|
||||
"passed": bool(adjacent) and maximum <= STEP_LIMIT_M,
|
||||
"worst": max(adjacent, default=None),
|
||||
}
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("log", type=Path)
|
||||
parser.add_argument("--expect-fail", action="store_true",
|
||||
help="pass only when the adjacent-step limit is exceeded")
|
||||
args = parser.parse_args()
|
||||
result = measure(args.log.read_text(encoding="utf-8", errors="replace"))
|
||||
detected = not bool(result["passed"])
|
||||
ok = detected if args.expect_fail else not detected
|
||||
print(f"[OwnerAdjacentProfileStep] log={args.log} pairs={result['adjacent_pairs']} "
|
||||
f"groups={result['profile_groups']} max_step_m={result['maximum_adjacent_step_m']:.6f} "
|
||||
f"limit_m={STEP_LIMIT_M:.3f} max_adjacent_distance_m={MAX_ADJACENT_DISTANCE_M:.3f} "
|
||||
f"fitted_periods_m={result['fitted_periods_m']} "
|
||||
"period_role=diagnostic "
|
||||
f"detected={str(detected).lower()} status={'PASS' if ok else 'FAIL'}")
|
||||
return 0 if ok else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,396 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Shared source-stamp, process, and per-run evidence helpers for verify.py."""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
from source_stamp import source_stamp
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[3]
|
||||
OUT = Path(os.environ.get("VOXELFORGE_VERIFY_OUT", ROOT / "Saved/Linux/l39"))
|
||||
LIMIT = ["taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--"]
|
||||
RECIPE_RE = re.compile(
|
||||
r"^\[WorldAssemblerRecipeAudit\] requested=([^ ]+) actual=([^ ]+).*status=([^ ]+)$",
|
||||
re.M)
|
||||
STAMP_RE = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$")
|
||||
|
||||
|
||||
def sha256_file(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as stream:
|
||||
for block in iter(lambda: stream.read(1024 * 1024), b""):
|
||||
digest.update(block)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def script_hash(path: Path) -> str:
|
||||
return sha256_file(path) if path.is_file() else "MISSING"
|
||||
|
||||
|
||||
def verification_tool_hashes() -> dict[str, str]:
|
||||
return {str(path): sha256_file(path) for path in sorted(
|
||||
Path(__file__).resolve().parent.glob("*.py"))}
|
||||
|
||||
|
||||
def current_stamp() -> str:
|
||||
include_unreal = os.environ.get("VOXELFORGE_INCLUDE_UNREAL_ADAPTER", "0") == "1"
|
||||
return source_stamp(ROOT, include_unreal)
|
||||
|
||||
|
||||
def first_stamp(text: str) -> str | None:
|
||||
first = text.splitlines()[0] if text.splitlines() else ""
|
||||
match = STAMP_RE.fullmatch(first)
|
||||
return match.group(1) if match else None
|
||||
|
||||
|
||||
def bounded_run(command: list[str], *, timeout: int = 1800,
|
||||
cwd: Path = ROOT, env: dict[str, str] | None = None,
|
||||
stdout: Any = subprocess.PIPE) -> subprocess.CompletedProcess[str] | None:
|
||||
try:
|
||||
return subprocess.run([*LIMIT, "timeout", f"{timeout}s", *map(str, command)],
|
||||
cwd=cwd, env=env, stdout=stdout, stderr=subprocess.STDOUT,
|
||||
text=True, encoding="utf-8", errors="replace", timeout=timeout,
|
||||
check=False)
|
||||
except subprocess.TimeoutExpired:
|
||||
return None
|
||||
|
||||
|
||||
class VerificationRunner:
|
||||
def __init__(self, tier: str):
|
||||
self.tier = tier
|
||||
self.started = time.monotonic()
|
||||
self.expected = current_stamp()
|
||||
self.tool_hashes = verification_tool_hashes()
|
||||
self.runs: list[dict[str, Any]] = []
|
||||
self.steps: list[dict[str, Any]] = []
|
||||
self.binary_hashes: dict[str, str] = {}
|
||||
self.binary_stamp_overrides: dict[str, str] = {}
|
||||
self.reused_tiers: list[dict[str, Any]] = []
|
||||
OUT.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
def assert_current_tree(self, point: str) -> None:
|
||||
now = current_stamp()
|
||||
if now != self.expected:
|
||||
raise RuntimeError(
|
||||
f"source changed {point}: expected {self.expected}, recomputed {now}")
|
||||
tools_now = verification_tool_hashes()
|
||||
if tools_now != self.tool_hashes:
|
||||
raise RuntimeError(f"verification tool changed {point}: "
|
||||
f"expected {self.tool_hashes}, recomputed {tools_now}")
|
||||
|
||||
def reuse_passed_tier(self, path: Path, tier: str) -> bool:
|
||||
"""Reuse a completed tier only while all its source and evidence inputs match."""
|
||||
self.assert_current_tree(f"before reusing {tier}")
|
||||
if not path.is_file():
|
||||
return False
|
||||
try:
|
||||
summary = json.loads(path.read_text(encoding="utf-8"))
|
||||
except (OSError, json.JSONDecodeError):
|
||||
return False
|
||||
if (summary.get("tier") != tier or summary.get("status") != "PASS"
|
||||
or summary.get("source_stamp") != self.expected
|
||||
or summary.get("verification_tools") != self.tool_hashes):
|
||||
return False
|
||||
if any(row.get("status") not in ("PASS", "EXPECTED_FAIL")
|
||||
or row.get("source_stamp") != self.expected
|
||||
for row in summary.get("runs", [])):
|
||||
return False
|
||||
if any(row.get("status") != "PASS" for row in summary.get("checks", [])):
|
||||
return False
|
||||
for binary, digest in summary.get("binary_sha256", {}).items():
|
||||
binary_path = Path(binary)
|
||||
if not binary_path.is_file() or sha256_file(binary_path) != digest:
|
||||
return False
|
||||
for row in summary.get("runs", []):
|
||||
input_path = row.get("input_json")
|
||||
if input_path:
|
||||
path_input = Path(input_path)
|
||||
if (not path_input.is_file()
|
||||
or sha256_file(path_input) != row.get("input_json_sha256")):
|
||||
return False
|
||||
log_path = row.get("log")
|
||||
if log_path and not Path(log_path).is_file():
|
||||
return False
|
||||
for script, digest in row.get("audit_scripts", {}).items():
|
||||
script_path = Path(script)
|
||||
if not script_path.is_file() or sha256_file(script_path) != digest:
|
||||
return False
|
||||
digest = sha256_file(path)
|
||||
self.runs.extend(summary.get("runs", []))
|
||||
self.steps.extend(summary.get("checks", []))
|
||||
self.binary_hashes.update(summary.get("binary_sha256", {}))
|
||||
self.reused_tiers.append({
|
||||
"tier": tier, "summary": str(path), "summary_sha256": digest,
|
||||
"elapsed_seconds": summary.get("elapsed_seconds"),
|
||||
"source_stamp": self.expected,
|
||||
})
|
||||
self.assert_current_tree(f"after reusing {tier}")
|
||||
print(f"[{self.tier}] reuse-{tier} PASS "
|
||||
f"prior_elapsed={summary.get('elapsed_seconds', 0):.1f}s "
|
||||
f"source_stamp={self.expected}", flush=True)
|
||||
return True
|
||||
|
||||
def probe_binary(self, binary: Path, expected_stamp: str | None = None) -> tuple[str, str]:
|
||||
self.assert_current_tree(f"before probing {binary.name}")
|
||||
if not binary.is_file():
|
||||
raise RuntimeError(f"missing executable: {binary}")
|
||||
digest = sha256_file(binary)
|
||||
expected_stamp = expected_stamp or self.binary_stamp_overrides.get(str(binary), self.expected)
|
||||
proc = bounded_run([str(binary), "--print-source-stamp"], timeout=15)
|
||||
output = proc.stdout or "" if proc is not None else ""
|
||||
embedded = first_stamp(output)
|
||||
if proc is None or proc.returncode != 0 or embedded is None:
|
||||
raise RuntimeError(f"{binary} has no usable source stamp; probe output: {output[:300]}")
|
||||
if embedded != expected_stamp:
|
||||
raise RuntimeError(
|
||||
f"stale executable refused: {binary.name} has {embedded}, expected {expected_stamp}")
|
||||
previous = self.binary_hashes.get(str(binary))
|
||||
if previous is not None and previous != digest:
|
||||
raise RuntimeError(f"executable changed during verification: {binary.name}; "
|
||||
f"was {previous}, now {digest}")
|
||||
self.binary_hashes[str(binary)] = digest
|
||||
self.assert_current_tree(f"after probing {binary.name}")
|
||||
return embedded, digest
|
||||
|
||||
def assert_binaries_current(self) -> None:
|
||||
for binary in tuple(self.binary_hashes):
|
||||
self.probe_binary(Path(binary))
|
||||
|
||||
@staticmethod
|
||||
def layout_metadata(layout_path: Path) -> tuple[str | None, tuple[int, ...], dict[str, Any]]:
|
||||
if not layout_path.is_file():
|
||||
return None, (), {}
|
||||
data = json.loads(layout_path.read_text(encoding="utf-8"))
|
||||
styles = tuple(sorted({int(passage["style"]) for passage in data.get("passages", [])
|
||||
if "style" in passage}))
|
||||
resolved_strate_styles = {str(index): int(value) for index, value in
|
||||
data.get("resolved_strate_passage_styles", {}).items()}
|
||||
strata = {int(stratum.get("strate_index", stratum.get("index", -1))): stratum
|
||||
for stratum in data.get("strates", [])}
|
||||
details = {
|
||||
"layout_id": data.get("stable_identity"),
|
||||
"layout_schema": data.get("schema_version"),
|
||||
"passage_styles": styles,
|
||||
"strata": {str(index): value.get("generator_kind")
|
||||
for index, value in strata.items()},
|
||||
"resolved_strate_styles": resolved_strate_styles,
|
||||
"generator_passage_styles": {
|
||||
value.get("generator_kind"): resolved_strate_styles[str(index)]
|
||||
for index, value in strata.items()
|
||||
if str(index) in resolved_strate_styles},
|
||||
"passages": [{"id": value.get("stable_id"),
|
||||
"upper": value.get("upper_strate_index"),
|
||||
"lower": value.get("lower_strate_index"),
|
||||
"style": value.get("style")}
|
||||
for value in data.get("passages", [])],
|
||||
}
|
||||
return details["layout_id"], styles, details
|
||||
|
||||
def run_cpp(self, name: str, binary: Path, data: Path, recipe: str,
|
||||
flags: list[str], *, expected_style: int | None = None,
|
||||
workers: int = 12, timeout: int = 1800,
|
||||
layout_path: Path | None = None,
|
||||
max_level: int = -1,
|
||||
expected_strate_styles: dict[str, int] | None = None,
|
||||
script_inputs: list[Path] | None = None,
|
||||
allow_nonzero: bool = False,
|
||||
expected_stamp: str | None = None) -> tuple[dict[str, Any], str]:
|
||||
stamp, binary_digest = self.probe_binary(binary, expected_stamp)
|
||||
if not data.is_file():
|
||||
raise RuntimeError(f"missing input JSON: {data}")
|
||||
data_digest = sha256_file(data)
|
||||
layout_path = layout_path or (OUT / "layouts" / f"{name}.json")
|
||||
layout_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
clean_flags = [flag for flag in flags if not flag.startswith("--resolved-layout-output=")]
|
||||
command = [str(binary), str(data), f"--workers={workers}",
|
||||
"--world-assembler-prototype", "--stream-benchmark", "--stream-pipeline",
|
||||
"--force-generate", "--clip-radius=3", "--max-clip-level=8",
|
||||
f"--max-level={max_level}",
|
||||
f"--recipe={recipe}", *clean_flags, f"--resolved-layout-output={layout_path}"]
|
||||
log = OUT / f"{name}.log"
|
||||
log.parent.mkdir(parents=True, exist_ok=True)
|
||||
started = time.monotonic()
|
||||
with log.open("w", encoding="utf-8") as stream:
|
||||
proc = bounded_run(command, timeout=timeout, stdout=stream)
|
||||
elapsed = time.monotonic() - started
|
||||
output = log.read_text(encoding="utf-8", errors="replace") if log.exists() else ""
|
||||
actual_stamp = first_stamp(output)
|
||||
recipe_matches = RECIPE_RE.findall(output)
|
||||
resolved_recipe = recipe_matches[-1][1] if recipe_matches else None
|
||||
layout_id, styles, layout_details = self.layout_metadata(layout_path)
|
||||
code: int | str = proc.returncode if proc is not None else "TIMEOUT"
|
||||
stamp_ok = actual_stamp == stamp
|
||||
recipe_ok = recipe_matches == [(recipe, recipe, "PASS")]
|
||||
style_ok = expected_style is None or styles == (expected_style,)
|
||||
actual_strate_styles = layout_details.get("resolved_strate_styles", {})
|
||||
if expected_strate_styles is not None:
|
||||
style_ok = style_ok and all(actual_strate_styles.get(str(index)) == value
|
||||
for index, value in expected_strate_styles.items())
|
||||
run_ok = (stamp_ok and recipe_ok and style_ok and proc is not None
|
||||
and (allow_nonzero or proc.returncode == 0))
|
||||
row: dict[str, Any] = {
|
||||
"name": name, "tier": self.tier, "status": "PASS" if run_ok else "FAIL",
|
||||
"elapsed_seconds": round(elapsed, 3), "exit_code": code,
|
||||
"source_stamp": stamp, "binary": str(binary),
|
||||
"compiler": "gcc" if "gcc" in binary.name else "clang",
|
||||
"binary_sha256": binary_digest, "input_json": str(data),
|
||||
"input_json_sha256": data_digest, "requested_recipe": recipe,
|
||||
"resolved_recipe": resolved_recipe,
|
||||
"requested_style": expected_style, "resolved_style": list(styles),
|
||||
"style_id": ",".join(f"passage-style-{value}" for value in styles) or None,
|
||||
"requested_strate_styles": expected_strate_styles,
|
||||
"resolved_strate_styles": actual_strate_styles,
|
||||
"resolved_generator_styles": layout_details.get("generator_passage_styles", {}),
|
||||
"layout_id": layout_id, "layout": layout_details,
|
||||
"log": str(log),
|
||||
"audit_scripts": {str(path): script_hash(path) for path in (script_inputs or [])},
|
||||
"command": command,
|
||||
}
|
||||
self.runs.append(row)
|
||||
self.assert_current_tree(f"after run {name}")
|
||||
self.print_step(row)
|
||||
if not run_ok:
|
||||
raise VerificationFailure(f"{name} failed (exit={code}, stamp={stamp_ok}, "
|
||||
f"recipe={recipe_ok}, style={style_ok}); see {log}")
|
||||
return row, output
|
||||
|
||||
def run_program(self, name: str, binary: Path, command: list[str], *,
|
||||
data: Path | None, recipe: str | None, style: int | None,
|
||||
layout_path: Path | None = None, timeout: int = 1800,
|
||||
require_stamp_line: bool = True,
|
||||
script_inputs: list[Path] | None = None) -> tuple[dict[str, Any], str]:
|
||||
stamp, digest = self.probe_binary(binary)
|
||||
data_digest = sha256_file(data) if data is not None and data.is_file() else None
|
||||
log = OUT / f"{name}.log"
|
||||
log.parent.mkdir(parents=True, exist_ok=True)
|
||||
started = time.monotonic()
|
||||
with log.open("w", encoding="utf-8") as stream:
|
||||
proc = bounded_run(command, timeout=timeout, stdout=stream)
|
||||
elapsed = time.monotonic() - started
|
||||
output = log.read_text(encoding="utf-8", errors="replace")
|
||||
layout_id, styles, details = self.layout_metadata(layout_path) if layout_path else (None, (), {})
|
||||
code: int | str = proc.returncode if proc is not None else "TIMEOUT"
|
||||
stamp_ok = not require_stamp_line or first_stamp(output) == stamp
|
||||
ok = proc is not None and proc.returncode == 0 and stamp_ok
|
||||
row = {
|
||||
"name": name, "tier": self.tier, "status": "PASS" if ok else "FAIL",
|
||||
"elapsed_seconds": round(elapsed, 3), "exit_code": code,
|
||||
"source_stamp": stamp, "binary": str(binary), "binary_sha256": digest,
|
||||
"compiler": "gcc" if "gcc" in binary.name else "clang",
|
||||
"input_json": str(data) if data else None, "input_json_sha256": data_digest,
|
||||
"requested_recipe": recipe, "resolved_recipe": recipe,
|
||||
"requested_style": style, "resolved_style": list(styles),
|
||||
"style_id": ",".join(f"passage-style-{value}" for value in styles) or
|
||||
(f"passage-style-{style}" if style is not None else None),
|
||||
"resolved_strate_styles": details.get("resolved_strate_styles", {}),
|
||||
"resolved_generator_styles": details.get("generator_passage_styles", {}),
|
||||
"layout_id": layout_id, "layout": details, "log": str(log),
|
||||
"audit_scripts": {str(path): script_hash(path) for path in (script_inputs or [])},
|
||||
"command": command,
|
||||
}
|
||||
self.runs.append(row)
|
||||
self.assert_current_tree(f"after run {name}")
|
||||
self.print_step(row)
|
||||
if not ok:
|
||||
raise VerificationFailure(f"{name} failed (exit={code}, stamp={stamp_ok}); see {log}")
|
||||
return row, output
|
||||
|
||||
def run_stage(self, name: str, script: Path, args: list[str], *,
|
||||
binary: Path, data: Path | None, recipe: str | None,
|
||||
style: int | None, layout_path: Path | None = None,
|
||||
env: dict[str, str] | None = None, timeout: int = 1800,
|
||||
script_inputs: list[Path] | None = None) -> tuple[dict[str, Any], str]:
|
||||
merged_env = os.environ.copy()
|
||||
if env:
|
||||
merged_env.update(env)
|
||||
return self.run_program(name, binary,
|
||||
[sys.executable, str(script), *args], data=data, recipe=recipe,
|
||||
style=style, layout_path=layout_path, timeout=timeout,
|
||||
require_stamp_line=False,
|
||||
script_inputs=[script, *(script_inputs or [])]) if env is None else \
|
||||
self._run_stage_with_env(name, script, args, binary, data, recipe, style,
|
||||
layout_path, merged_env, timeout, script_inputs)
|
||||
|
||||
def _run_stage_with_env(self, name: str, script: Path, args: list[str],
|
||||
binary: Path, data: Path | None, recipe: str | None,
|
||||
style: int | None, layout_path: Path | None,
|
||||
env: dict[str, str], timeout: int,
|
||||
script_inputs: list[Path] | None) -> tuple[dict[str, Any], str]:
|
||||
stamp, digest = self.probe_binary(binary)
|
||||
data_digest = sha256_file(data) if data is not None and data.is_file() else None
|
||||
log = OUT / f"{name}.log"
|
||||
log.parent.mkdir(parents=True, exist_ok=True)
|
||||
started = time.monotonic()
|
||||
with log.open("w", encoding="utf-8") as stream:
|
||||
proc = bounded_run([sys.executable, str(script), *args], timeout=timeout,
|
||||
env=env, stdout=stream)
|
||||
elapsed = time.monotonic() - started
|
||||
output = log.read_text(encoding="utf-8", errors="replace")
|
||||
layout_id, styles, details = self.layout_metadata(layout_path) if layout_path else (None, (), {})
|
||||
code: int | str = proc.returncode if proc is not None else "TIMEOUT"
|
||||
ok = proc is not None and proc.returncode == 0
|
||||
row = {
|
||||
"name": name, "tier": self.tier, "status": "PASS" if ok else "FAIL",
|
||||
"elapsed_seconds": round(elapsed, 3), "exit_code": code,
|
||||
"source_stamp": stamp, "binary": str(binary), "binary_sha256": digest,
|
||||
"compiler": "gcc" if "gcc" in binary.name else "clang",
|
||||
"input_json": str(data) if data else None, "input_json_sha256": data_digest,
|
||||
"requested_recipe": recipe, "resolved_recipe": recipe,
|
||||
"requested_style": style, "resolved_style": list(styles),
|
||||
"style_id": ",".join(f"passage-style-{value}" for value in styles) or
|
||||
(f"passage-style-{style}" if style is not None else None),
|
||||
"resolved_strate_styles": details.get("resolved_strate_styles", {}),
|
||||
"resolved_generator_styles": details.get("generator_passage_styles", {}),
|
||||
"layout_id": layout_id, "layout": details, "log": str(log),
|
||||
"audit_scripts": {str(path): script_hash(path)
|
||||
for path in [script, *(script_inputs or [])]},
|
||||
"command": [sys.executable, str(script), *args],
|
||||
}
|
||||
self.runs.append(row)
|
||||
self.assert_current_tree(f"after stage {name}")
|
||||
self.print_step(row)
|
||||
if not ok:
|
||||
raise VerificationFailure(f"{name} failed (exit={code}); see {log}")
|
||||
return row, output
|
||||
|
||||
def print_step(self, row: dict[str, Any]) -> None:
|
||||
print(f"[{self.tier}] {row['name']} {row['status']} "
|
||||
f"{row.get('elapsed_seconds', 0):.1f}s "
|
||||
f"recipe={row.get('resolved_recipe') or '-'} "
|
||||
f"style={row.get('resolved_style') or '-'} "
|
||||
f"layout={row.get('layout_id') or '-'}", flush=True)
|
||||
|
||||
def require(self, name: str, passed: bool, detail: str = "") -> None:
|
||||
step = {"name": name, "status": "PASS" if passed else "FAIL", "detail": detail}
|
||||
self.steps.append(step)
|
||||
print(f"[{self.tier}] {name} {'PASS' if passed else 'FAIL'} {detail}", flush=True)
|
||||
if not passed:
|
||||
raise VerificationFailure(f"{name} failed: {detail}")
|
||||
|
||||
def save_summary(self, status: str, error: str | None = None) -> Path:
|
||||
elapsed = time.monotonic() - self.started
|
||||
summary = {
|
||||
"tier": self.tier, "status": status, "elapsed_seconds": round(elapsed, 3),
|
||||
"source_stamp": self.expected, "source_manifest_sha256": self.expected,
|
||||
"verification_tools": self.tool_hashes,
|
||||
"binary_sha256": self.binary_hashes, "runs": self.runs,
|
||||
"checks": self.steps, "reused_tiers": self.reused_tiers, "error": error,
|
||||
}
|
||||
path = OUT / f"{self.tier}-summary.json"
|
||||
path.write_text(json.dumps(summary, indent=2, sort_keys=True) + "\n",
|
||||
encoding="utf-8")
|
||||
print(f"[{self.tier}] SUMMARY {status} elapsed={elapsed:.1f}s file={path}", flush=True)
|
||||
return path
|
||||
|
||||
|
||||
class VerificationFailure(RuntimeError):
|
||||
pass
|
||||
@@ -0,0 +1,98 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build and verify the final-source manifest used by Linux executables."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
|
||||
def sha256(data: bytes) -> str:
|
||||
return hashlib.sha256(data).hexdigest()
|
||||
|
||||
|
||||
def manifest_entries(root: Path, include_unreal: bool = False) -> list[tuple[str, bytes]]:
|
||||
standalone = root / "Standalone"
|
||||
files: set[Path] = set()
|
||||
|
||||
core = standalone / "Core"
|
||||
for suffix in ("*.h", "*.hpp", "*.cpp", "*.txt"):
|
||||
files.update(core.rglob(suffix))
|
||||
|
||||
source = standalone / "Source"
|
||||
for suffix in ("*.h", "*.hpp", "*.cpp"):
|
||||
files.update(source.glob(suffix))
|
||||
|
||||
for path in standalone.glob("Build*"):
|
||||
if path.is_file():
|
||||
files.add(path)
|
||||
for suffix in ("*.cmake", "*.mk", "Makefile", "CMakeLists.txt"):
|
||||
files.update(standalone.glob(suffix))
|
||||
core_build = standalone / "Core"
|
||||
for suffix in ("*.cmake", "*.mk", "Makefile", "CMakeLists.txt"):
|
||||
files.update(core_build.glob(suffix))
|
||||
helper = standalone / "Tools/verify/source_stamp.py"
|
||||
if helper.is_file():
|
||||
files.add(helper)
|
||||
|
||||
if include_unreal:
|
||||
unreal = source / "VoxelForge"
|
||||
for suffix in ("*.h", "*.hpp", "*.cpp", "*.cs"):
|
||||
files.update(unreal.rglob(suffix))
|
||||
|
||||
entries = [(path.relative_to(root).as_posix(), path.read_bytes())
|
||||
for path in files if path.is_file()]
|
||||
|
||||
# BuildLinux creates one namespace wrapper per listed Core implementation.
|
||||
# Include each exact generated wrapper as a virtual source input.
|
||||
source_list = core / "core_sources.txt"
|
||||
for raw in source_list.read_text(encoding="utf-8").splitlines():
|
||||
name = raw.split("#", 1)[0].strip()
|
||||
if not name or name == "Core/src/VoxelForgeCore.cpp":
|
||||
continue
|
||||
source_path = standalone / name
|
||||
if not source_path.is_file():
|
||||
raise SystemExit(f"missing source listed in Core/core_sources.txt: {name}")
|
||||
wrapper_name = "vfcore_" + "".join(c if c.isalnum() else "_" for c in name)
|
||||
wrapper = ('#include "CoreNamespacePreamble.h"\n'
|
||||
"namespace vfcore::detail\n{\n"
|
||||
f'#include "{source_path}"\n'
|
||||
"}\n").encode("utf-8")
|
||||
entries.append((f"generated/wrappers/{wrapper_name}.cpp", wrapper))
|
||||
|
||||
return sorted(entries, key=lambda item: item[0])
|
||||
|
||||
|
||||
def rendered_manifest(root: Path, include_unreal: bool = False) -> bytes:
|
||||
lines = [f"{sha256(data)} {name}" for name, data in
|
||||
manifest_entries(root, include_unreal)]
|
||||
return ("\n".join(lines) + "\n").encode("utf-8")
|
||||
|
||||
|
||||
def source_stamp(root: Path, include_unreal: bool = False) -> str:
|
||||
return sha256(rendered_manifest(root, include_unreal))
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("command", choices=("stamp", "write-manifest"))
|
||||
parser.add_argument("--root", type=Path,
|
||||
default=Path(__file__).resolve().parents[3])
|
||||
parser.add_argument("--output", type=Path)
|
||||
parser.add_argument("--include-unreal-adapter", action="store_true")
|
||||
args = parser.parse_args()
|
||||
root = args.root.resolve()
|
||||
manifest = rendered_manifest(root, args.include_unreal_adapter)
|
||||
stamp = sha256(manifest)
|
||||
if args.command == "write-manifest":
|
||||
if args.output is None:
|
||||
parser.error("write-manifest requires --output")
|
||||
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||
args.output.write_bytes(manifest)
|
||||
print(stamp)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,640 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run the reusable VoxelForge verification tiers: fast, medium, or full."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shutil
|
||||
import statistics
|
||||
import struct
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import zlib
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[3]
|
||||
VERIFY_DIR = ROOT / "Standalone/Tools/verify"
|
||||
LINUX37 = ROOT / "Standalone/Research/WorldAssemblerM3/Linux37"
|
||||
OWNER = ROOT / "Standalone/data/owner_world.json"
|
||||
OUT_DEFAULT = ROOT / "Saved/Linux/l39"
|
||||
|
||||
PARSER = argparse.ArgumentParser(description=__doc__)
|
||||
PARSER.add_argument("tier", choices=("fast", "medium", "full"))
|
||||
PARSER.add_argument("--out", type=Path, default=OUT_DEFAULT,
|
||||
help="artifact directory (default: Saved/Linux/l39)")
|
||||
ARGS = PARSER.parse_args()
|
||||
OUT = ARGS.out.resolve()
|
||||
os.environ["VOXELFORGE_VERIFY_OUT"] = str(OUT)
|
||||
os.environ["PYTHONDONTWRITEBYTECODE"] = "1"
|
||||
sys.dont_write_bytecode = True
|
||||
|
||||
sys.path.insert(0, str(VERIFY_DIR))
|
||||
from owner_step import measure as measure_owner_step
|
||||
from runner import VerificationFailure, VerificationRunner, bounded_run
|
||||
from source_stamp import source_stamp
|
||||
|
||||
|
||||
CLANG = OUT / "bin/VoxelForgeCoreHeadless-l39-clang"
|
||||
GCC = OUT / "bin/VoxelForgeCoreHeadless-l39-gcc"
|
||||
RENDERER = OUT / "bin/VoxelForgeOffscreenCapture-l39-clang"
|
||||
HASH_RE = re.compile(r"^\[Pipeline(?:FullDetailHash|WholeViewHash|LODHash|ExactQueryHash)\].*$", re.M)
|
||||
RECIPE_RE = re.compile(
|
||||
r"^\[WorldAssemblerRecipeAudit\] requested=([^ ]+) actual=([^ ]+).*status=([^ ]+)$", re.M)
|
||||
|
||||
|
||||
def compiler_name(binary: Path) -> str:
|
||||
return "gcc" if "gcc" in binary.name else "clang"
|
||||
|
||||
|
||||
def env_for(*, capture: bool = False) -> dict[str, str]:
|
||||
result = os.environ.copy()
|
||||
result["VOXELFORGE_VERIFY_OUT"] = str(OUT)
|
||||
result["VOXELFORGE_L37_OUT"] = str(OUT)
|
||||
result["VOXELFORGE_L37_CAPTURE_DIR"] = str(OUT / "captures")
|
||||
result["VOXELFORGE_L37_HEADLESS"] = str(CLANG)
|
||||
result["VOXELFORGE_L37_RENDERER"] = str(RENDERER)
|
||||
result["VOXELFORGE_VERIFY_HEADLESS"] = str(CLANG)
|
||||
if capture:
|
||||
result["LP_NUM_THREADS"] = "12"
|
||||
result["XDG_CACHE_HOME"] = str(OUT / "xdg-cache")
|
||||
return result
|
||||
|
||||
|
||||
def all_match(text: str, pattern: str) -> bool:
|
||||
return re.search(pattern, text, re.M) is not None
|
||||
|
||||
|
||||
def parse_ratio(text: str, name: str) -> float | None:
|
||||
match = re.search(rf"(?:^| ){re.escape(name)}=(\d+)/(\d+)", text)
|
||||
return int(match.group(1)) / max(1, int(match.group(2))) if match else None
|
||||
|
||||
|
||||
def validate_passage(runner: VerificationRunner, name: str, output: str) -> None:
|
||||
required = (
|
||||
("strict-layout", r"^\[ResolvedLayoutStrict\] accepted=true issues=0 error=$"),
|
||||
("layout-controls", r"^\[ResolvedLayoutValidatorControls\].*status=PASS$"),
|
||||
("walkability", r"^\[Walkability\] PASS"),
|
||||
("floor-profile", r"^\[FloorProfileBlend\].*negative_controls=step_detected/slope_detected status=PASS$"),
|
||||
("mouth-profile", r"^\[PassageMouthProfileAudit\].*status=PASS$"),
|
||||
("route-binding", r"^\[ResolvedStrateRouteAudit\].*mouth_binding=PASS status=PASS$"),
|
||||
("naturalness", r"^\[PassageNaturalnessSummary\].*status=PASS$"),
|
||||
("open-air", r"^\[PassageOpenAirAuditSummary\].*status=PASS$"),
|
||||
("no-long-bridge", r"^\[PassageNoLongBridgeAudit\].*status=PASS$"),
|
||||
("rock-columns", r"^\[PassageRockColumnAuditSummary\].*status=PASS$"),
|
||||
("ripple-controls", r"^\[PassageRippleAuditSummary\].*status=PASS$"),
|
||||
("passage-summary", r"^\[ResolvedPassageAuditSummary\].*route_binding_failures=0 .*status=PASS$"),
|
||||
)
|
||||
for gate, pattern in required:
|
||||
runner.require(f"{name}:{gate}", all_match(output, pattern),
|
||||
"required resolved-passage gate")
|
||||
|
||||
rows = re.findall(r"^\[PassageRippleAudit\].*$", output, re.M)
|
||||
ripple_ok = bool(rows)
|
||||
for line in rows:
|
||||
wall = re.search(r"local_peak_amplitude_m=([-0-9.]+)", line)
|
||||
if wall is None:
|
||||
wall = re.search(r"amplitude_m=([-0-9.]+)", line)
|
||||
wall_threshold = re.search(r"threshold_m=([-0-9.]+)", line)
|
||||
floor = re.search(r"floor_amplitude_m=([-0-9.]+)", line)
|
||||
floor_threshold = re.search(r"floor_threshold_m=([-0-9.]+)", line)
|
||||
wall_coverage = parse_ratio(line, "coverage")
|
||||
floor_coverage = parse_ratio(line, "floor_coverage")
|
||||
ripple_ok &= bool(
|
||||
"status=PASS" in line and wall and wall_threshold and floor and floor_threshold
|
||||
and wall_coverage is not None and floor_coverage is not None
|
||||
and float(wall.group(1)) <= float(wall_threshold.group(1))
|
||||
and float(floor.group(1)) <= float(floor_threshold.group(1))
|
||||
and wall_coverage >= 0.75 and floor_coverage >= 0.75)
|
||||
runner.require(f"{name}:wall-floor-ripple-and-coverage", ripple_ok,
|
||||
f"profiles={len(rows)} wall/floor amplitudes under their thresholds; coverage>=0.75")
|
||||
|
||||
|
||||
def run_passage(runner: VerificationRunner, binary: Path, name: str,
|
||||
data: Path, recipe: str, style: int, *, workers: int = 12,
|
||||
flags: list[str] | None = None, max_level: int = -1,
|
||||
layout_path: Path | None = None, passage_style: int | None = None,
|
||||
expected_strate_styles: dict[str, int] | None = None) -> tuple[dict, str]:
|
||||
layout_path = layout_path or OUT / "layouts" / f"{name}.json"
|
||||
row, output = runner.run_cpp(name, binary, data, recipe,
|
||||
["--audit-resolved-layout-passages", *(flags or [])],
|
||||
expected_style=style if passage_style is None else passage_style,
|
||||
expected_strate_styles=expected_strate_styles,
|
||||
workers=workers, max_level=max_level,
|
||||
layout_path=layout_path)
|
||||
row["requested_style"] = style
|
||||
row["expected_passage_style"] = style if passage_style is None else passage_style
|
||||
row["requested_strate_styles"] = expected_strate_styles
|
||||
validate_passage(runner, name, output)
|
||||
return row, output
|
||||
|
||||
|
||||
def run_owner_passages(runner: VerificationRunner, binaries: tuple[Path, ...]) -> dict[str, tuple[dict, str]]:
|
||||
result: dict[str, tuple[dict, str]] = {}
|
||||
expected_strate_styles = {"0": 2, "1": 1}
|
||||
for binary in binaries:
|
||||
compiler = compiler_name(binary)
|
||||
for recipe, style in (("floating-islands", 2), ("surface-world", 1)):
|
||||
name = f"owner-{recipe}-style{style}-{compiler}"
|
||||
row, output = run_passage(runner, binary, name, OWNER, recipe, style,
|
||||
passage_style=2, expected_strate_styles=expected_strate_styles)
|
||||
result[f"{compiler}:{recipe}"] = (row, output)
|
||||
actual_styles = row.get("resolved_strate_styles", {})
|
||||
runner.require(f"{name}:owner-style-map",
|
||||
actual_styles == expected_strate_styles,
|
||||
f"resolved_strate_styles={actual_styles} expected={expected_strate_styles}")
|
||||
if compiler == "clang":
|
||||
measured = measure_owner_step(output)
|
||||
worst = measured.get("worst")
|
||||
runner.require(f"{name}:adjacent-owner-step",
|
||||
bool(measured["passed"]),
|
||||
f"max_step_m={measured['maximum_adjacent_step_m']:.6f} "
|
||||
f"limit_m=0.100 pairs={measured['adjacent_pairs']} "
|
||||
f"fitted_periods_m={measured['fitted_periods_m']} "
|
||||
f"period_role=diagnostic worst={worst}")
|
||||
return result
|
||||
|
||||
|
||||
def expected_control_row(runner: VerificationRunner, name: str) -> None:
|
||||
if runner.runs:
|
||||
row = runner.runs[-1]
|
||||
row["observed_status"] = row["status"]
|
||||
row["status"] = "EXPECTED_FAIL"
|
||||
row["negative_control"] = name
|
||||
|
||||
|
||||
def run_stamp_and_input_controls(runner: VerificationRunner) -> None:
|
||||
# Mutate one compiled Core input only while checking that the runner refuses it.
|
||||
core_file = ROOT / "Standalone/Core/src/VoxelForgeCore.cpp"
|
||||
original = core_file.read_bytes()
|
||||
rejected = False
|
||||
try:
|
||||
core_file.write_bytes(original + b"\n// temporary L39 stale-build negative control\n")
|
||||
try:
|
||||
runner.probe_binary(CLANG)
|
||||
except RuntimeError as error:
|
||||
rejected = "source changed" in str(error)
|
||||
finally:
|
||||
core_file.write_bytes(original)
|
||||
runner.assert_current_tree("after restoring stale-build control")
|
||||
runner.require("stale-build-source-edit-control", rejected,
|
||||
"runner refused the binary after a Core source edit; original bytes restored")
|
||||
|
||||
fixture = ROOT / "Saved/Linux/l33/fixtures/sw-upper-tunnel-network-style0.json"
|
||||
mislabelled = OUT / "controls/mislabelled-owner/owner_world.json"
|
||||
mislabelled.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copyfile(fixture, mislabelled)
|
||||
try:
|
||||
runner.run_cpp("control-mislabelled-owner-style0", CLANG, mislabelled,
|
||||
"floating-islands", ["--resolved-layout-only"], expected_style=2,
|
||||
layout_path=OUT / "controls/mislabelled-owner/layout-owner.json")
|
||||
detected = False
|
||||
except VerificationFailure:
|
||||
detected = bool(runner.runs and 0 in runner.runs[-1]["resolved_style"]
|
||||
and runner.runs[-1]["status"] == "FAIL")
|
||||
if detected:
|
||||
expected_control_row(runner, "mislabelled-style0-input")
|
||||
runner.require("mislabelled-owner-style-control", detected,
|
||||
"style-0 fixture named owner_world.json rejected for expected FI Style 2")
|
||||
|
||||
black = OUT / "controls/all-black.png"
|
||||
black.parent.mkdir(parents=True, exist_ok=True)
|
||||
write_black_png(black)
|
||||
script = VERIFY_DIR / "capture_quality.py"
|
||||
try:
|
||||
runner.run_stage("control-all-black-capture", script, [str(black)],
|
||||
binary=CLANG, data=None, recipe=None, style=None,
|
||||
env=env_for(), script_inputs=[black])
|
||||
black_rejected = False
|
||||
except VerificationFailure:
|
||||
black_rejected = "status=FAIL" in (OUT / "control-all-black-capture.log").read_text(
|
||||
encoding="utf-8", errors="replace")
|
||||
if black_rejected:
|
||||
expected_control_row(runner, "all-black-capture")
|
||||
runner.require("all-black-capture-control", black_rejected,
|
||||
"pixel variance and coverage floor reject a valid all-black PNG")
|
||||
|
||||
|
||||
def write_black_png(path: Path) -> None:
|
||||
def chunk(tag: bytes, payload: bytes) -> bytes:
|
||||
return (struct.pack(">I", len(payload)) + tag + payload
|
||||
+ struct.pack(">I", zlib.crc32(tag + payload) & 0xffffffff))
|
||||
pixels = b"".join(b"\0" + b"\0\0\0" * 2 for _ in range(2))
|
||||
body = (b"\x89PNG\r\n\x1a\n"
|
||||
+ chunk(b"IHDR", struct.pack(">IIBBBBB", 2, 2, 8, 2, 0, 0, 0))
|
||||
+ chunk(b"IDAT", zlib.compress(pixels)) + chunk(b"IEND", b""))
|
||||
path.write_bytes(body)
|
||||
|
||||
|
||||
def run_regression_controls(runner: VerificationRunner) -> None:
|
||||
run_stamp_and_input_controls(runner)
|
||||
controls = (
|
||||
("l35-wall-ribs", "wall-ribs",
|
||||
ROOT / "Saved/Linux/l35/failed/passage-audit-cascade-cosine-owner-fi-style3-clang.log"),
|
||||
("l36-floor-washboard", "floor-washboard",
|
||||
ROOT / "Saved/Linux/l36/audit-nearest-fi-style0.log"),
|
||||
)
|
||||
script = VERIFY_DIR / "historical_controls.py"
|
||||
for name, kind, log in controls:
|
||||
runner.run_stage(f"control-{name}", script, [kind, str(log)],
|
||||
binary=CLANG, data=None, recipe=None, style=None,
|
||||
env=env_for(), script_inputs=[log])
|
||||
|
||||
l38 = ROOT / "Saved/Linux/l38/owner-fi-real-final.log"
|
||||
step_script = VERIFY_DIR / "owner_step.py"
|
||||
runner.run_stage("control-l38-owner-step", step_script,
|
||||
[str(l38), "--expect-fail"], binary=CLANG, data=None,
|
||||
recipe=None, style=None, env=env_for(), script_inputs=[l38])
|
||||
|
||||
|
||||
def run_hash_pairs(runner: VerificationRunner,
|
||||
owner_results: dict[str, tuple[dict, str]]) -> None:
|
||||
for recipe, style in (("floating-islands", 2), ("surface-world", 1)):
|
||||
hashes: dict[int, tuple[str, ...]] = {}
|
||||
for workers in (1, 12):
|
||||
name = f"owner-hash-{recipe}-style{style}-w{workers}"
|
||||
row, output = runner.run_cpp(name, CLANG, OWNER, recipe,
|
||||
["--stream-hash"], expected_style=2,
|
||||
expected_strate_styles={"0": 2, "1": 1}, workers=workers,
|
||||
layout_path=OUT / "layouts" / f"{name}.json")
|
||||
row["requested_style"] = style
|
||||
row["expected_passage_style"] = 2
|
||||
values = tuple(HASH_RE.findall(output))
|
||||
hashes[workers] = values
|
||||
runner.require(f"{name}:hash-count", len(values) == 12,
|
||||
f"hashes={len(values)}/12")
|
||||
same = hashes[1] == hashes[12]
|
||||
baseline = ROOT / "Saved/Linux/l37" / f"determinism-l25-clang-{recipe}-w12.log"
|
||||
old_hashes = tuple(HASH_RE.findall(baseline.read_text(encoding="utf-8", errors="replace")))
|
||||
same_l37 = hashes[12] == old_hashes and len(old_hashes) == 12
|
||||
runner.require(f"owner-hash-{recipe}-w1-w12", same,
|
||||
"W1 and W12 each produced identical 12-hash signatures")
|
||||
runner.require(f"owner-hash-{recipe}-matches-l37", same_l37,
|
||||
f"current owner hashes match {baseline.name}; hashes={len(old_hashes)}")
|
||||
|
||||
|
||||
def run_ring(runner: VerificationRunner, *, full: bool) -> None:
|
||||
script = LINUX37 / "RunRingAuditsL37.py"
|
||||
audit_inputs = [script,
|
||||
ROOT / "Standalone/Research/WorldAssemblerM3/Linux32/AuditRingTerraceL32.py",
|
||||
ROOT / "Standalone/Research/WorldAssemblerM3/Linux32/AuditRingSetL32.py"]
|
||||
for label, recipe, style in (("fi-floor", "floating-islands", 2),
|
||||
("sw-ceiling", "surface-world", 1)):
|
||||
args = ["--scene", label]
|
||||
layout = OUT / "layouts" / f"ring-layout-{label}-lod0.json"
|
||||
if not full:
|
||||
args.append("--lod0-only")
|
||||
if label == "sw-ceiling":
|
||||
args.append("--skip-controls")
|
||||
runner.run_stage(f"ring-{label}-{'lod0-8' if full else 'lod0'}", script, args,
|
||||
binary=CLANG, data=OWNER, recipe=recipe, style=style,
|
||||
layout_path=layout, env=env_for(), script_inputs=audit_inputs)
|
||||
|
||||
|
||||
def style_override_matrix(runner: VerificationRunner, binary: Path) -> None:
|
||||
sources: list[tuple[str, Path, str]] = []
|
||||
for recipe in ("floating-islands", "surface-world"):
|
||||
for style in range(4):
|
||||
source = ROOT / f"Saved/Linux/linux25-owner-{recipe}-style{style}.json"
|
||||
sources.append((f"{recipe}-style{style}", source, recipe))
|
||||
for style in range(4):
|
||||
source = ROOT / f"Saved/Linux/l33/fixtures/sw-upper-tunnel-network-style{style}.json"
|
||||
sources.append((f"upper-sw-fixture-style{style}", source, "floating-islands"))
|
||||
|
||||
for fixture_name, source, recipe in sources:
|
||||
label = f"style-override-{fixture_name}"
|
||||
destination = OUT / "fixtures" / f"{label}.json"
|
||||
destination.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copyfile(source, destination)
|
||||
style = int(re.search(r"style([0-3])$", fixture_name).group(1))
|
||||
name = f"{label}-{compiler_name(binary)}"
|
||||
if fixture_name.startswith("upper-sw-fixture-"):
|
||||
expected_strate_styles = {"1": style}
|
||||
else:
|
||||
expected_strate_styles = {"0": style, "1": 1}
|
||||
row, output = runner.run_cpp(name, binary, destination, recipe,
|
||||
["--audit-resolved-layout-passages"], expected_style=None,
|
||||
expected_strate_styles=expected_strate_styles,
|
||||
layout_path=OUT / "layouts" / f"{name}.json")
|
||||
row["requested_style"] = style
|
||||
row["requested_strate_styles"] = expected_strate_styles
|
||||
validate_passage(runner, name, output)
|
||||
|
||||
|
||||
def run_transitions_and_invariants(runner: VerificationRunner,
|
||||
binaries: tuple[Path, ...]) -> None:
|
||||
for binary in binaries:
|
||||
compiler = compiler_name(binary)
|
||||
for recipe, style in (("floating-islands", 2), ("surface-world", 1)):
|
||||
for mode in ("hard", "gradient", "interleaved"):
|
||||
name = f"transition-{recipe}-{mode}-{compiler}"
|
||||
row, output = runner.run_cpp(name, binary, OWNER, recipe,
|
||||
[f"--transition-mode={mode}", "--audit-strate-transitions"],
|
||||
expected_style=2, expected_strate_styles={"0": 2, "1": 1},
|
||||
layout_path=OUT / "layouts" / f"{name}.json")
|
||||
row["requested_style"] = style
|
||||
row["expected_passage_style"] = 2
|
||||
gates = (r"^\[StrateTransitionAuditSummary\].*hole_columns=0 air_samples=0 .*status=PASS$",
|
||||
r"^\[StrateTransitionNegativeControls\].*status=PASS$")
|
||||
for index, pattern in enumerate(gates):
|
||||
runner.require(f"{name}:gate-{index + 1}", all_match(output, pattern), pattern)
|
||||
|
||||
name = f"invariant-route-proof-{recipe}-{compiler}"
|
||||
row, output = runner.run_cpp(name, binary, OWNER, recipe,
|
||||
["--audit-uniform-block-proofs", "--audit-route-parity"],
|
||||
expected_style=2, expected_strate_styles={"0": 2, "1": 1},
|
||||
layout_path=OUT / "layouts" / f"{name}.json")
|
||||
row["requested_style"] = style
|
||||
row["expected_passage_style"] = 2
|
||||
runner.require(f"{name}:route-parity",
|
||||
all_match(output, r"^\[RouteParityAudit\].*status=PASS$"),
|
||||
"route parity L13")
|
||||
runner.require(f"{name}:uniform-block-proof",
|
||||
all_match(output, r"^\[UniformBlockProofAudit\].*violations=0(?:\s|$)"),
|
||||
"uniform-block proof L9")
|
||||
|
||||
|
||||
def run_captures_and_mesh_audits(runner: VerificationRunner) -> None:
|
||||
capture_script = LINUX37 / "CapturePassageStandardSetL37.py"
|
||||
capture_inputs = [capture_script, VERIFY_DIR / "capture_quality.py",
|
||||
ROOT / "Standalone/Research/WorldAssemblerM3/Linux32/AuditRingTerraceL32.py"]
|
||||
capture_rows: dict[str, tuple[dict, str]] = {}
|
||||
for scene, recipe, style in (("owner-fi", "floating-islands", 2),
|
||||
("owner-sw", "surface-world", 1)):
|
||||
layout_path = OUT / f"layout-{scene}.json"
|
||||
name = f"capture-standard-{scene}-style{style}"
|
||||
row, output = runner.run_stage(name, capture_script,
|
||||
["--scene", scene], binary=CLANG, data=OWNER, recipe=recipe,
|
||||
style=style, layout_path=layout_path, env=env_for(capture=True),
|
||||
script_inputs=capture_inputs)
|
||||
resolved_strate_styles = row.get("resolved_strate_styles", {})
|
||||
target_strate = "0" if scene == "owner-fi" else "1"
|
||||
style_ok = (resolved_strate_styles == {"0": 2, "1": 1}
|
||||
and resolved_strate_styles.get(target_strate) == style)
|
||||
runner.require(f"{name}:resolved-style", style_ok,
|
||||
f"resolved_strate_styles={resolved_strate_styles} "
|
||||
f"target_strate={target_strate} expected_style={style}")
|
||||
capture_rows[scene] = (row, output)
|
||||
|
||||
shape_script = LINUX37 / "AuditMouthShapeL37.py"
|
||||
baseline = ROOT / "Saved/Linux/l37/no-tread-control"
|
||||
controls = [ROOT / "Saved/Linux/l36/owner-fi-ground-mouth.vfmesh"]
|
||||
for scene, recipe, style in (("owner-fi", "floating-islands", 2),
|
||||
("owner-sw", "surface-world", 1)):
|
||||
name = f"mouth-shape-{scene}-style{style}"
|
||||
runner.run_stage(name, shape_script,
|
||||
[f"--out={OUT}", f"--baseline-out={baseline}",
|
||||
"--control-out=Saved/Linux/l36", "--scene", scene],
|
||||
binary=CLANG, data=OWNER, recipe=recipe, style=style,
|
||||
layout_path=OUT / f"layout-{scene}.json", env=env_for(),
|
||||
script_inputs=[shape_script, *controls])
|
||||
|
||||
crease_script = LINUX37 / "RunMouthCreaseAuditL37.py"
|
||||
sw_capture_log = OUT / "capture-standard-owner-sw-style1.log"
|
||||
crease_args = [
|
||||
f"--candidate={OUT / 'owner-fi-ground-mouth.vfmesh'}",
|
||||
f"--candidate-sw={OUT / 'owner-sw-ground-mouth.vfmesh'}",
|
||||
f"--layout-sw={OUT / 'layout-owner-sw.json'}",
|
||||
f"--capture-log-sw={sw_capture_log}"]
|
||||
control_meshes = [ROOT / f"Saved/Linux/{round_name}/owner-fi-ground-mouth.vfmesh"
|
||||
for round_name in ("l35", "l36")]
|
||||
runner.run_stage("mouth-crease-fragments-owner-fi2-sw1", crease_script,
|
||||
crease_args, binary=CLANG, data=OWNER, recipe="floating-islands", style=2,
|
||||
layout_path=OUT / "layout-owner-fi.json", env=env_for(),
|
||||
script_inputs=[crease_script,
|
||||
LINUX37 / "RunMouthFragmentAuditL37.py", *control_meshes,
|
||||
OUT / "owner-fi-ground-mouth.vfmesh", OUT / "owner-sw-ground-mouth.vfmesh",
|
||||
OUT / "layout-owner-sw.json", sw_capture_log])
|
||||
|
||||
overview_script = LINUX37 / "CaptureMouthOverviewL37.py"
|
||||
runner.run_stage("capture-fi-overview-style2", overview_script, [],
|
||||
binary=CLANG, data=OWNER, recipe="floating-islands", style=2,
|
||||
layout_path=OUT / "layout-owner-fi.json", env=env_for(capture=True),
|
||||
script_inputs=[overview_script, VERIFY_DIR / "capture_quality.py",
|
||||
OUT / "owner-fi-ground-mouth.vfmesh"])
|
||||
|
||||
|
||||
def verify_determinism_cell(runner: VerificationRunner, binary: Path,
|
||||
kind: str, recipe: str, style: int, workers: int,
|
||||
reference: dict[tuple[str, str], tuple[str, ...]]) -> None:
|
||||
name = f"determinism-{kind}-{recipe}-{compiler_name(binary)}-w{workers}"
|
||||
flags = ["--stream-hash"]
|
||||
if kind == "crossing24":
|
||||
flags.extend(("--camera-step=24,0,0", "--region-crossing"))
|
||||
row, output = runner.run_cpp(name, binary, OWNER, recipe, flags,
|
||||
expected_style=2, expected_strate_styles={"0": 2, "1": 1},
|
||||
workers=workers,
|
||||
layout_path=OUT / "layouts" / f"{name}.json")
|
||||
row["requested_style"] = style
|
||||
row["expected_passage_style"] = 2
|
||||
values = tuple(HASH_RE.findall(output))
|
||||
key = (kind, recipe)
|
||||
expected = reference.get(key)
|
||||
stable = expected is None or values == expected
|
||||
if expected is None and values:
|
||||
reference[key] = values
|
||||
runner.require(f"{name}:stable-hashes", len(values) == 12 and stable,
|
||||
f"hashes={len(values)}/12 stable={stable}")
|
||||
|
||||
|
||||
def run_30_cell_determinism(runner: VerificationRunner) -> None:
|
||||
references: dict[tuple[str, str], tuple[str, ...]] = {}
|
||||
count = 0
|
||||
for kind in ("l25", "crossing24"):
|
||||
for recipe, style in (("surface-world", 1), ("floating-islands", 2)):
|
||||
for binary in (CLANG, GCC):
|
||||
for workers in (1, 8, 12):
|
||||
verify_determinism_cell(runner, binary, kind, recipe, style,
|
||||
workers, references)
|
||||
count += 1
|
||||
|
||||
fixture_source = ROOT / "Saved/Linux/l33/fixtures/sw-upper-tunnel-network-style0.json"
|
||||
fixture = OUT / "fixtures/style-override-upper-sw-fixture-style0.json"
|
||||
fixture.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copyfile(fixture_source, fixture)
|
||||
signatures = None
|
||||
for binary in (CLANG, GCC):
|
||||
for workers in (1, 8, 12):
|
||||
name = f"determinism-style-override-upper-sw-fixture-style0-{compiler_name(binary)}-w{workers}"
|
||||
row, output = runner.run_cpp(name, binary, fixture, "floating-islands",
|
||||
["--audit-resolved-layout-passages"], expected_style=None,
|
||||
expected_strate_styles={"1": 0}, workers=workers,
|
||||
layout_path=OUT / "layouts" / f"{name}.json")
|
||||
row["requested_style"] = 0
|
||||
row["requested_strate_styles"] = {"1": 0}
|
||||
validate_passage(runner, name, output)
|
||||
current = tuple(re.findall(
|
||||
r"^\[ResolvedLayoutSnapshot\].*stable_identity=([^ ]+)", output, re.M)
|
||||
+ re.findall(r"^\[WorldAssemblyPlanSerialization\].*plan_hash=([0-9a-f]+)", output, re.M)
|
||||
+ re.findall(r"^\[WorldAssemblerRouteAccepted\].*$", output, re.M))
|
||||
stable = signatures is None or current == signatures
|
||||
if signatures is None and current:
|
||||
signatures = current
|
||||
runner.require(f"{name}:stable-plan-route", bool(current) and stable,
|
||||
f"snapshot-plan-route_records={len(current)} stable={stable}")
|
||||
count += 1
|
||||
runner.require("determinism-30-cell-count", count == 30, f"cells={count}/30")
|
||||
|
||||
|
||||
def prepare_l36_cost_baseline(runner: VerificationRunner) -> tuple[Path, str]:
|
||||
temp = Path("/tmp/voxelforge-l36-stamped")
|
||||
if temp.exists():
|
||||
shutil.rmtree(temp)
|
||||
temp.mkdir(parents=True)
|
||||
tarball = temp / "l36.tar"
|
||||
archive = bounded_run(["git", "archive", "-o", str(tarball),
|
||||
"8be1c4d", "Standalone"], timeout=120)
|
||||
if archive is None or archive.returncode != 0 or not tarball.is_file():
|
||||
raise VerificationFailure("could not export historical L36 source commit 8be1c4d")
|
||||
extract = bounded_run(["tar", "-xf", str(tarball), "-C", str(temp)], timeout=120)
|
||||
if extract is None or extract.returncode != 0:
|
||||
raise VerificationFailure("historical L36 source extraction failed")
|
||||
tarball.unlink(missing_ok=True)
|
||||
|
||||
base = temp / "Standalone"
|
||||
(base / "Tools/verify").mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(ROOT / "Standalone/BuildLinux.sh", base / "BuildLinux.sh")
|
||||
shutil.copy2(VERIFY_DIR / "source_stamp.py", base / "Tools/verify/source_stamp.py")
|
||||
shutil.copy2(ROOT / "Standalone/Source/SourceStamp.h", base / "Source/SourceStamp.h")
|
||||
legacy_main = base / "Source/StreamBenchmarkHeadless.cpp"
|
||||
text = legacy_main.read_text(encoding="utf-8")
|
||||
if "VoxelForgePrintSourceStamp" not in text:
|
||||
text = text.replace("#include \"OwnerWorldLoader.h\"",
|
||||
"#include \"SourceStamp.h\"\n#include <cstring>\n#include \"OwnerWorldLoader.h\"", 1)
|
||||
anchor = re.search(r"int main\(int argc,\s*char\*\*\s*argv\)\s*\{", text)
|
||||
if anchor is None:
|
||||
raise VerificationFailure("cannot instrument historical L36 headless entry point")
|
||||
injected = ("\n VoxelForgePrintSourceStamp();\n"
|
||||
" if(argc==2&&std::strcmp(argv[1],\"--print-source-stamp\")==0)return 0;")
|
||||
text = text[:anchor.end()] + injected + text[anchor.end():]
|
||||
legacy_main.write_text(text, encoding="utf-8")
|
||||
|
||||
baseline_stamp = source_stamp(temp)
|
||||
binary = OUT / "historical-bin/VoxelForgeCoreHeadless-l36-stamped"
|
||||
env = os.environ.copy()
|
||||
env["VOXELFORGE_BUILD_ROOT"] = str(temp / "build")
|
||||
env["VOXELFORGE_BIN_DIR"] = str(binary.parent)
|
||||
command = ["bash", str(base / "BuildLinux.sh"), "--compiler=clang++-19",
|
||||
"--variant=l36-stamped", "--fp-contract=off", "--jobs=12"]
|
||||
log = OUT / "cost-baseline-build-l36.log"
|
||||
log.parent.mkdir(parents=True, exist_ok=True)
|
||||
started = time.monotonic()
|
||||
with log.open("w", encoding="utf-8") as stream:
|
||||
proc = bounded_run(command, timeout=1800, env=env, stdout=stream)
|
||||
elapsed = time.monotonic() - started
|
||||
runner.require("cost-baseline-l36-build",
|
||||
proc is not None and proc.returncode == 0 and binary.is_file(),
|
||||
f"source_stamp={baseline_stamp} elapsed_seconds={elapsed:.1f} log={log}")
|
||||
runner.binary_stamp_overrides[str(binary)] = baseline_stamp
|
||||
return binary, baseline_stamp
|
||||
|
||||
|
||||
def run_cost_pairs(runner: VerificationRunner) -> None:
|
||||
baseline, baseline_stamp = prepare_l36_cost_baseline(runner)
|
||||
candidate = CLANG
|
||||
orders = (("l36", "l39"), ("l39", "l36"), ("l36", "l39"),
|
||||
("l39", "l36"), ("l36", "l39"))
|
||||
by_recipe: dict[str, dict[str, list[float]]] = {
|
||||
"floating-islands": {"l36": [], "l39": []},
|
||||
"surface-world": {"l36": [], "l39": []}}
|
||||
by_pair: dict[tuple[str, int], dict[str, float]] = {}
|
||||
for recipe, style in (("floating-islands", 2), ("surface-world", 1)):
|
||||
for pair, order in enumerate(orders, 1):
|
||||
for build in order:
|
||||
binary = baseline if build == "l36" else candidate
|
||||
expected_style = None if build == "l36" else style
|
||||
name = f"cost-{recipe}-pair{pair}-{build}"
|
||||
expected_stamp = baseline_stamp if build == "l36" else None
|
||||
row, output = runner.run_cpp(name, binary, OWNER, recipe, [],
|
||||
expected_style=2 if build == "l39" else None,
|
||||
expected_strate_styles={"0": 2, "1": 1} if build == "l39" else None,
|
||||
layout_path=OUT / "layouts" / f"{name}.json",
|
||||
expected_stamp=expected_stamp)
|
||||
row["requested_style"] = style
|
||||
row["expected_passage_style"] = 2 if build == "l39" else None
|
||||
ready = re.findall(r"\bwhole_view_ready=([0-9.]+)s", output)
|
||||
ok = (len(ready) == 1 and "hash=off" in output
|
||||
and "cache_hits=0" in output
|
||||
and RECIPE_RE.findall(output) == [(recipe, recipe, "PASS")])
|
||||
runner.require(f"{name}:cost-record", ok,
|
||||
f"whole_view_ready={ready[0] if ready else 'MISSING'}s "
|
||||
f"source_stamp={row['source_stamp']}")
|
||||
by_recipe[recipe][build].append(float(ready[0]))
|
||||
by_pair.setdefault((recipe, pair), {})[build] = float(ready[0])
|
||||
for recipe, values in by_recipe.items():
|
||||
old = statistics.median(values["l36"])
|
||||
new = statistics.median(values["l39"])
|
||||
paired = [((by_pair[(recipe, pair)]["l39"] - by_pair[(recipe, pair)]["l36"])
|
||||
/ by_pair[(recipe, pair)]["l36"] * 100.0)
|
||||
for pair in range(1, 6)]
|
||||
runner.require(f"cost-{recipe}-five-pairs",
|
||||
len(values["l36"]) == len(values["l39"]) == 5,
|
||||
f"median_l36_s={old:.3f} median_l39_s={new:.3f} "
|
||||
f"median_change_pct={(new / old - 1.0) * 100.0:+.2f} "
|
||||
f"paired_change_pct={','.join(f'{value:+.2f}' for value in paired)}")
|
||||
|
||||
|
||||
def run_medium_suite(runner: VerificationRunner, *, both_compilers: bool = False) -> None:
|
||||
binaries = (CLANG, GCC) if both_compilers else (CLANG,)
|
||||
run_owner_passages(runner, binaries)
|
||||
run_regression_controls(runner)
|
||||
style_override_matrix(runner, CLANG)
|
||||
if both_compilers:
|
||||
style_override_matrix(runner, GCC)
|
||||
run_transitions_and_invariants(runner, binaries)
|
||||
run_ring(runner, full=True)
|
||||
run_captures_and_mesh_audits(runner)
|
||||
|
||||
|
||||
def run_gcc_supplement(runner: VerificationRunner) -> None:
|
||||
"""Add GCC evidence to a stamped, already-passed Clang medium tier."""
|
||||
run_owner_passages(runner, (GCC,))
|
||||
style_override_matrix(runner, GCC)
|
||||
run_transitions_and_invariants(runner, (GCC,))
|
||||
|
||||
|
||||
def main() -> int:
|
||||
runner = VerificationRunner(ARGS.tier)
|
||||
started = time.monotonic()
|
||||
try:
|
||||
if not OWNER.is_file():
|
||||
raise VerificationFailure(f"canonical owner input is missing: {OWNER}")
|
||||
for binary in (CLANG, GCC, RENDERER):
|
||||
if binary.is_file():
|
||||
runner.probe_binary(binary)
|
||||
if ARGS.tier == "fast":
|
||||
owners = run_owner_passages(runner, (CLANG,))
|
||||
run_regression_controls(runner)
|
||||
run_hash_pairs(runner, owners)
|
||||
run_ring(runner, full=False)
|
||||
elif ARGS.tier == "medium":
|
||||
run_medium_suite(runner, both_compilers=False)
|
||||
else:
|
||||
medium_summary = OUT / "medium-summary.json"
|
||||
if not runner.reuse_passed_tier(medium_summary, "medium"):
|
||||
run_medium_suite(runner, both_compilers=False)
|
||||
run_gcc_supplement(runner)
|
||||
run_30_cell_determinism(runner)
|
||||
run_cost_pairs(runner)
|
||||
# The validated medium summary carries the final canonical capture set.
|
||||
|
||||
runner.assert_binaries_current()
|
||||
runner.assert_current_tree("at tier completion")
|
||||
elapsed = time.monotonic() - started
|
||||
runner.save_summary("PASS")
|
||||
print(f"[{ARGS.tier}] DONE PASS elapsed={elapsed:.1f}s", flush=True)
|
||||
return 0
|
||||
except Exception as error:
|
||||
elapsed = time.monotonic() - started
|
||||
runner.save_summary("FAIL", str(error))
|
||||
print(f"[{ARGS.tier}] DONE FAIL elapsed={elapsed:.1f}s error={error}", flush=True)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in new issue
Block a user