diff --git a/Standalone/Research/WorldAssemblerM3/Linux124/REPORT.md b/Standalone/Research/WorldAssemblerM3/Linux124/REPORT.md new file mode 100644 index 0000000..651b7ae --- /dev/null +++ b/Standalone/Research/WorldAssemblerM3/Linux124/REPORT.md @@ -0,0 +1,45 @@ +# L124 — early arrival tiles + +## Decision + +Measured **not worth shipping** the design that independently builds pages for the nearest 27 LOD0 tiles. The existing shape gate is real, but this page set can remove only a small amount of work from the LOD0–2 completion path. Its estimated best case is below 100 ms for every stratum/worker pair except the unadjusted FI W8 tile-work ceiling; after page work, the largest estimate is 62 ms. VS is negative. No second design or runtime change was pursued. + +## Current L80 timings + +Five interleaved W8/W12 runs per stratum, Clang 19, policy 3, L80 protocol, source stamp `3f48d5ef6dfd7e735afcd824b742f997179810acd3ca76601a99c1131efb482e`. Values below are medians. `Near ready` is the complete LOD0–2 set. + +The host load was 0.5–1.7 before baseline timing. The unchanged headless binary was built first; no build overlapped a timing run. + +| Stratum | LOD0–2 tiles | Shape wall W8/W12 | First tile ready W8/W12 | Near ready W8/W12 | +|---|---:|---:|---:|---:| +| SW | 1,408 | 208/204 ms | 280/281 ms | 709/690 ms | +| FI | 1,216 | 182/179 ms | 261/261 ms | 1,441/1,329 ms | +| TN | 1,344 | 205/210 ms | 284/275 ms | 1,372/1,243 ms | +| VS | 1,344 | 226/221 ms | 291/283 ms | 992/835 ms | + +A separate cold timeline trace put shape-build start at 58–60 ms after process start and shape ready at 233–279 ms. The first LOD0–2 admission followed 0.65–0.70 ms later; there were zero LOD0–2 admissions before shape ready. The exact per-run records are in `Saved/Linux/l124/timing/` and `Saved/Linux/l124/workers/`. + +## What a tile reads + +`World::compile` publishes `shapeReady` only after `FStandaloneShapeWorld::Build` finishes. Shape-dependent `generateTile` calls wait for that publication. A tile’s mesher uses `ClassifyBoxDetailed`, lattice density queries, and off-grid `DensityAtUnclamped` queries for gradients and seam normals. Density dispatch can read native surface/island/lattice fields, or packed `RockWords`, `CandidateWords`, and `SurfaceDistanceQ`; fallback queries use analytic density. Composition also reads stratum transitions, cavern fields, stamps/edits, and passage carve data. The tile query scope includes a one-step normal halo and selects passage features, while passage void-crossing plans and primitive/field indices must already be valid. + +An exact local page is feasible: the existing full page builder constructs those dependencies. A temporary probe built a 128-voxel LOD0 page independently after base compilation and compared the center tile’s density, mesh, payload, and topology hashes. All four matched for SW, FI, TN, and VS at W8 and W12. This covered one tile per fixture, not the full 27-tile ring. The normal `compileTileRegion` path still requires the base shape and forwards page queries to it, so it cannot currently publish that page early. + +## Cost count + +An exact 128-voxel page took a median 7.5–12.6 ms wall per page (three interleaved W8/W12 probes); reported parallel-stage worker time was about 27–63 ms. The nearest 27 LOD0 tiles span 1 SW page, 8 FI pages, 1 TN page, and 4 VS pages. Their measured tile-service CPU and an optimistic cost-adjusted budget are: + +| Stratum | Ring tile CPU W8/W12 | Estimated net budget W8/W12 | +|---|---:|---:| +| SW | 439/770 ms | 51/62 ms | +| FI | 818/1,134 ms | 41/57 ms | +| TN | 402/577 ms | 43/43 ms | +| VS | 96/175 ms | −20/−5 ms | + +The budget subtracts reported page worker time for the required page count, then divides by stream workers. That counter excludes local primitive-build CPU. It also assumes all 27 tile jobs leave the post-shape queue and ignores contention, so it is an optimistic estimate, not an observed A/B speedup. + +During the base build, stream workers were idle for 8.00/8 or 12.00/12 worker-equivalents. Shape helper stages averaged 7.1–7.97 active workers of the configured 8; detail work alone occupied 52–87 ms wall. Whole-build process use averaged 4.6–5.3 of the 12 pinned CPUs. Early page and tile work would overlap the periods where the shape helper pool is full. Given the sub-100 ms estimate and VS’s negative budget, I stopped before adding publication and scheduling code. + +## Validation and artifacts + +The temporary page-cost/equivalence hooks were removed and the headless binary rebuilt against the unchanged L123 source stamp. No 24/24 hash matrix, unpublished-page poison assertion, 30 m/s trace, fast tier, or Linux UE candidate timing was run because no runtime change was retained. The independent-page hash probe was only a viability check after full shape publication. Baseline logs are in `Saved/Linux/l124/{timing,workers,page-cost,page-equivalence}/`. diff --git a/Standalone/Research/WorldAssemblerM3/Linux124/RunL124HeadlessBaseline.py b/Standalone/Research/WorldAssemblerM3/Linux124/RunL124HeadlessBaseline.py new file mode 100644 index 0000000..45e2d8e --- /dev/null +++ b/Standalone/Research/WorldAssemblerM3/Linux124/RunL124HeadlessBaseline.py @@ -0,0 +1,148 @@ +#!/usr/bin/env python3 +"""Five interleaved L80-protocol W8/W12 cold-readiness pairs per owner stratum.""" +from __future__ import annotations + +import json +import re +import statistics +import subprocess +import time +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[4] +DATA = ROOT / "Standalone/data/owner_world.json" +BIN = ROOT / "Saved/Linux/l124/bin/VoxelForgeCoreHeadless-l124-clang" +OUT = ROOT / "Saved/Linux/l124/timing" +STAMP_TOOL = ROOT / "Standalone/Tools/verify/source_stamp.py" +CAMERAS = { + "sw": (8.0, 16.0, -372.2626953125), + "fi": (25.8369140625, 3.19921875, -737.0), + "tn": (19.166015625, -14.2001953125, -753.3740234375), + "vs": (-20.0625, -5.068359375, -900.353515625), +} +FIELDS = re.compile(r"([a-zA-Z0-9_+.-]+)=([^ ]+)") +STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M) +SHAPE_FREE = re.compile( + r"^\[ShapeReadiness\] shape_free_desired=(\d+)/(\d+) near_ring=(\d+)/27", + re.M, +) +PROFILE = re.compile(r"^\[ShapeProfile\] (.*)$", re.M) +STAGE = re.compile(r"^\[ShapeStage\] (.*)$", re.M) +MILESTONE = re.compile(r"^\[PipelineMilestone\] (.*)$", re.M) +NEAR = re.compile( + r"^\[WorldAssemblerNearLODReady\] lods=0-2 tiles=(\d+)/(\d+) cold_ready_s=([0-9.]+)$", + re.M, +) +PIPELINE = re.compile(r"^\[StreamPipeline\] (.*)$", re.M) + + +def run(stratum: str, workers: int, phase: str, replicate: int) -> dict: + OUT.mkdir(parents=True, exist_ok=True) + label = f"{stratum}-w{workers}-{phase}{replicate}" + log = OUT / f"{label}.log" + camera = ",".join(f"{value:.12f}" for value in CAMERAS[stratum]) + ",0,0" + command = [ + "taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--", + "timeout", "1740s", str(BIN), str(DATA), "--world-assembler-prototype", + "--stream-benchmark", "--stream-pipeline", f"--workers={workers}", + "--clip-radius=3", "--max-clip-level=8", "--max-level=8", + f"--camera={camera}", "--profile-lod-work", "--stream-scheduler-policy=3", + ] + started = time.monotonic() + with log.open("w", encoding="utf-8") as output: + result = subprocess.run(command, cwd=ROOT, stdout=output, + stderr=subprocess.STDOUT, check=False, timeout=1740) + content = log.read_text(encoding="utf-8", errors="replace") + near, pipeline_match = NEAR.search(content), PIPELINE.search(content) + profile, stage, free = PROFILE.search(content), STAGE.search(content), SHAPE_FREE.search(content) + stamp = STAMP.search(content) + milestones = dict(FIELDS.findall(MILESTONE.search(content).group(1))) if MILESTONE.search(content) else {} + if result.returncode or near is None or pipeline_match is None or not all((profile, stage, free, stamp)): + raise RuntimeError(f"baseline run incomplete: {log}; exit={result.returncode}") + if stamp.group(1) != EXPECTED_STAMP: + raise RuntimeError(f"source stamp mismatch: {log}; got={stamp.group(1)} expected={EXPECTED_STAMP}") + pipeline = dict(FIELDS.findall(pipeline_match.group(1))) + profile_values = dict(FIELDS.findall(profile.group(1))) + stage_values = dict(FIELDS.findall(stage.group(1))) + row = { + "stratum": stratum, + "workers": workers, + "phase": phase, + "replicate": replicate, + "exit_code": result.returncode, + "wall_s": round(time.monotonic() - started, 3), + "source_stamp": stamp.group(1), + "shape_free_tiles": int(free.group(1)), + "desired_tiles": int(free.group(2)), + "shape_free_near_ring": int(free.group(3)), + "shape_compile_s": float(pipeline["shape"].removesuffix("s")), + "first_tile_ready_s": float(pipeline["first_tile_ready"].removesuffix("s")), + "near_ready_s": float(near.group(3)), + "near_ready_tiles": int(near.group(1)), + "near_desired_tiles": int(near.group(2)), + "worker_shape_stages_ms": { + key: float(value.removesuffix("ms")) + for key, value in profile_values.items() if key.endswith("_worker") + }, + "shape_stage_wall_ms": { + key: float(value) + for key, value in stage_values.items() + if key in {"local-shapes", "intervals+signs", "surface-band", "far-parents", "total"} + }, + "milestones": milestones, + "log": str(log.relative_to(ROOT)), + } + print("[L124HeadlessRun] " + json.dumps(row, sort_keys=True), flush=True) + return row + + +def main() -> int: + global EXPECTED_STAMP + expected = subprocess.run( + ["python3", "-B", str(STAMP_TOOL), "stamp", "--root", str(ROOT)], + cwd=ROOT, text=True, stdout=subprocess.PIPE, check=True, + ).stdout.strip() + EXPECTED_STAMP = expected + if not DATA.is_file() or not BIN.is_file(): + raise SystemExit(f"missing input: {DATA} or {BIN}") + rows: list[dict] = [] + for stratum in CAMERAS: + for workers in (8, 12): + run(stratum, workers, "warmup", 0) + orders = ((8, 12), (12, 8), (8, 12), (12, 8), (8, 12)) + for stratum in CAMERAS: + for replicate, order in enumerate(orders, 1): + paired = {} + for workers in order: + row = run(stratum, workers, "measured", replicate) + rows.append(row) + paired[workers] = row + (OUT / "runs-partial.json").write_text( + json.dumps(rows, indent=2) + "\n", encoding="utf-8") + print("[L124HeadlessPair] " + json.dumps({ + "stratum": stratum, + "replicate": replicate, + "w8_near_ready_s": paired[8]["near_ready_s"], + "w12_near_ready_s": paired[12]["near_ready_s"], + }, sort_keys=True), flush=True) + summary = [] + for stratum in CAMERAS: + entry = {"stratum": stratum, "interleaved_pairs": 5} + for workers in (8, 12): + subset = [r for r in rows if r["stratum"] == stratum and r["workers"] == workers] + entry[f"w{workers}_near_ready_s"] = [r["near_ready_s"] for r in subset] + entry[f"w{workers}_near_ready_median_s"] = statistics.median( + r["near_ready_s"] for r in subset) + entry[f"w{workers}_shape_s"] = [r["shape_compile_s"] for r in subset] + entry[f"w{workers}_shape_median_s"] = statistics.median( + r["shape_compile_s"] for r in subset) + summary.append(entry) + (OUT / "summary.json").write_text( + json.dumps({"source_stamp": EXPECTED_STAMP, "runs": rows, "summary": summary}, indent=2) + "\n", + encoding="utf-8", + ) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/Standalone/Research/WorldAssemblerM3/Linux124/RunL124WorkerDiagnostics.py b/Standalone/Research/WorldAssemblerM3/Linux124/RunL124WorkerDiagnostics.py new file mode 100644 index 0000000..775848b --- /dev/null +++ b/Standalone/Research/WorldAssemblerM3/Linux124/RunL124WorkerDiagnostics.py @@ -0,0 +1,151 @@ +#!/usr/bin/env python3 +"""Collect the separate worker-idle and admission timeline for L124's unchanged headless path.""" +from __future__ import annotations + +import json +import re +import subprocess +import time +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[4] +DATA = ROOT / "Standalone/data/owner_world.json" +BIN = ROOT / "Saved/Linux/l124/bin/VoxelForgeCoreHeadless-l124-clang" +OUT = ROOT / "Saved/Linux/l124/workers" +STAMP_TOOL = ROOT / "Standalone/Tools/verify/source_stamp.py" +CAMERAS = { + "sw": (8.0, 16.0, -372.2626953125), + "fi": (25.8369140625, 3.19921875, -737.0), + "tn": (19.166015625, -14.2001953125, -753.3740234375), + "vs": (-20.0625, -5.068359375, -900.353515625), +} +STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M) +COLD = re.compile( + r"^\[ColdTimeline\] event=(\S+) edge=(\S+) at_ms=([0-9.]+) cpu_ms=([0-9.]+)$", + re.M, +) +TIMELINE = re.compile( + r"^\[CpuTileTimeline\] event=(\S+) key=\(([-0-9]+),([-0-9]+),([-0-9]+),lod(\d+)\) " + r"at_ms=([0-9.-]+) frame=\d+ worker_id=(-?\d+) queue_depth=(\d+) " + r"active_workers=(\d+) priority_rank=(\d+)", re.M, +) +PROFILE = re.compile(r"^\[ShapeProfile\] (.*)$", re.M) +BREAKDOWN = re.compile(r"^\[ShapeBreakdown\] (.*)$", re.M) +NEAR = re.compile( + r"^\[WorldAssemblerNearLODReady\] lods=0-2 tiles=(\d+)/(\d+) cold_ready_s=([0-9.]+)$", + re.M, +) +PAIR = re.compile(r"([a-zA-Z0-9_+.-]+)=([0-9.]+)(?:ms)?") + + +def intervals(events: list[tuple[float, float]], start: float, end: float) -> float: + return sum(max(0.0, min(finish, end) - max(begin, start)) + for begin, finish in events) + + +def run(stratum: str, workers: int, expected_stamp: str) -> dict: + OUT.mkdir(parents=True, exist_ok=True) + log = OUT / f"{stratum}-w{workers}.log" + camera = ",".join(f"{value:.12f}" for value in CAMERAS[stratum]) + ",0,0" + command = [ + "taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--", + "timeout", "1740s", str(BIN), str(DATA), "--world-assembler-prototype", + "--stream-benchmark", "--stream-pipeline", f"--workers={workers}", + "--clip-radius=3", "--max-clip-level=8", "--max-level=8", + f"--camera={camera}", "--profile-lod-work", "--stream-scheduler-policy=3", + "--cold-worker-timeline", + ] + started = time.monotonic() + with log.open("w", encoding="utf-8") as output: + result = subprocess.run(command, cwd=ROOT, stdout=output, + stderr=subprocess.STDOUT, check=False, timeout=1740) + content = log.read_text(encoding="utf-8", errors="replace") + stamp = STAMP.search(content) + cold = {(event, edge): (float(at), float(cpu)) + for event, edge, at, cpu in COLD.findall(content)} + timeline = [ + {"event": event, "key": (int(x), int(y), int(z), int(lod)), + "at_ms": float(at), "worker_id": int(worker), + "queue_depth": int(depth), "active_workers": int(active), + "priority_rank": int(rank)} + for event, x, y, z, lod, at, worker, depth, active, rank in TIMELINE.findall(content) + ] + base_begin, base_end = cold.get(("base_compile", "begin")), cold.get(("base_compile", "end")) + near = NEAR.search(content) + profile_match, breakdown_match = PROFILE.search(content), BREAKDOWN.search(content) + if (result.returncode or stamp is None or stamp.group(1) != expected_stamp + or base_begin is None or base_end is None or near is None + or profile_match is None or breakdown_match is None): + raise RuntimeError(f"worker diagnostic incomplete: {log}; exit={result.returncode}") + begin_ms, begin_cpu = base_begin + end_ms, end_cpu = base_end + admitted = [event for event in timeline if event["event"] == "admitted" and event["key"][3] <= 2] + post_compile = [event for event in admitted if event["at_ms"] >= end_ms] + idle_by_worker: dict[int, list[tuple[float, float]]] = {} + idle_begin: dict[int, float] = {} + for event in timeline: + worker = event["worker_id"] + if worker < 0: + continue + if event["event"] == "idle_begin": + idle_begin[worker] = event["at_ms"] + elif event["event"] == "idle_end" and worker in idle_begin: + idle_by_worker.setdefault(worker, []).append((idle_begin.pop(worker), event["at_ms"])) + for worker, idle_start in idle_begin.items(): + idle_by_worker.setdefault(worker, []).append((idle_start, end_ms)) + idle_worker_ms = sum(intervals(spans, begin_ms, end_ms) + for spans in idle_by_worker.values()) + profile = {key: float(value) for key, value in PAIR.findall(profile_match.group(1))} + breakdown = {key: float(value) for key, value in PAIR.findall(breakdown_match.group(1))} + parallel_stages = {} + for name in ("rock_lines", "candidate_axes", "candidate_merge", "detail_lines"): + wall, worker = profile.get(f"{name}_wall"), profile.get(f"{name}_worker") + if wall is not None and worker is not None and wall > 0.0: + parallel_stages[name] = { + "wall_ms": wall, + "worker_ms": worker, + "average_busy_workers": worker / wall, + } + row = { + "stratum": stratum, + "workers": workers, + "exit_code": result.returncode, + "wall_s": round(time.monotonic() - started, 3), + "source_stamp": stamp.group(1), + "base_compile_start_ms": begin_ms, + "base_shape_ready_ms": end_ms, + "base_compile_wall_ms": end_ms - begin_ms, + "process_cpu_ms_during_compile": end_cpu - begin_cpu, + "average_process_cores_during_compile": (end_cpu - begin_cpu) / max(1e-9, end_ms - begin_ms), + "stream_workers_idle_ms_during_compile": idle_worker_ms, + "stream_idle_worker_equivalents": idle_worker_ms / max(1e-9, end_ms - begin_ms), + "first_lod0_2_admission_ms": min((event["at_ms"] for event in admitted), default=None), + "first_lod0_2_shape_dependent_admission_ms": min( + (event["at_ms"] for event in post_compile), default=None), + "near_ready_s": float(near.group(3)), + "lod0_2_admissions": len(admitted), + "lod0_2_admissions_before_shape_ready": sum(event["at_ms"] < end_ms for event in admitted), + "shape_parallel_stages": parallel_stages, + "owner_shape_worker_ms": breakdown.get("owner_shape_worker"), + "log": str(log.relative_to(ROOT)), + } + print("[L124WorkerRun] " + json.dumps(row, sort_keys=True), flush=True) + return row + + +def main() -> int: + expected_stamp = subprocess.run( + ["python3", "-B", str(STAMP_TOOL), "stamp", "--root", str(ROOT)], + cwd=ROOT, text=True, stdout=subprocess.PIPE, check=True, + ).stdout.strip() + rows = [run(stratum, workers, expected_stamp) + for stratum in CAMERAS for workers in (8, 12)] + (OUT / "summary.json").write_text( + json.dumps({"source_stamp": expected_stamp, "runs": rows}, indent=2) + "\n", + encoding="utf-8", + ) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main())