streaming: speed-aware scheduler with hysteresis (L80); arrival near-first, fast travel keeps the horizon
Policy 3 (headless + Unreal): walkable ring always first; at rest/arrival LOD0-2 before far; walking interleaves near:far ~4:1 toward heading; fast travel admits far coarse tiles right after the walkable ring. Hysteresis: walking 0.75/0.35 m/s, fast 12/8 m/s. Headless, far holes while moving 30 m/s: L77 9.0% / L79 77.2% / L80 8.9%. Cold LOD0-2 ready (one clean protocol, medians): SW 1.30, FI 3.61, TN 3.07, VS 1.43 s. Unreal 30 m/s traces end 2,233/2,237 loaded (commandlet readiness exit 1) for all policies. Passage zero-padding cut: ~50% fewer passage iterations on active tiles, max 6.6 cm surface displacement, no sign flips; not adopted yet (padding sweep next). World unchanged (v213). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
This commit is contained in:
Fr0zkaandClaude Opus 5.5 committed
1 parent
2c8a26c0d0
commit
b06397df9c
10 files changed
+830
-25
No files matched your search
@@ -0,0 +1,73 @@
|
||||
# Linux80
|
||||
|
||||
## Scheduler policy
|
||||
|
||||
Policy 3 is speed aware. The walkable LOD0 ring always leads. On arrival and at rest, LOD0–2 run before far work; far generation continues after the near queue drains. While walking, near and far tiles interleave at about 4:1, with near tiles ordered toward travel. At fast speed, far coarse tiles follow the walkable ring. Headless and Unreal use matching hysteresis: walking enters at 0.75 m/s and exits at 0.35 m/s; fast travel enters at 12 m/s and exits at 8 m/s.
|
||||
|
||||
Headless moving runs used 12 s of motion, 5 s of dwell, 69 samples per run, and had zero generation failures. Values are moving means; dwell is final far holes / near readiness.
|
||||
|
||||
| Speed | Policy | Far holes | Near ready | Dwell end |
|
||||
|---|---|---:|---:|---:|
|
||||
| 1.4 m/s | L77 | 7.70% | 75.74% | 0% / 100% |
|
||||
| 1.4 m/s | L79 | 32.84% | 82.08% | 0% / 100% |
|
||||
| 1.4 m/s | L80 | 26.62% | 81.54% | 0% / 100% |
|
||||
| 30 m/s | L77 | 8.96% | 49.40% | 0% / 100% |
|
||||
| 30 m/s | L79 | 77.21% | 56.96% | 0% / 100% |
|
||||
| 30 m/s | L80 | 8.94% | 46.50% | 0% / 100% |
|
||||
|
||||
Unreal null-RHI runs used the same 12 s + 5 s route, with no address-space cap. Values are moving means; each dwell ended with 0% far holes and 99.7–99.9% near readiness.
|
||||
|
||||
| Speed | Policy | Far holes | Near ready | Commandlet |
|
||||
|---|---|---:|---:|---|
|
||||
| 1.4 m/s | L77 | 5.46% | 99.12% | pass |
|
||||
| 1.4 m/s | L79 | 7.49% | 99.08% | pass |
|
||||
| 1.4 m/s | L80 | 9.23% | 99.13% | pass |
|
||||
| 30 m/s | L77 | 24.89% | 20.79% | exit 1* |
|
||||
| 30 m/s | L79 | 13.42% | 20.41% | exit 1* |
|
||||
| 30 m/s | L80 | 10.51% | 20.52% | exit 1* |
|
||||
|
||||
*All fast traces completed their movement and dwell samples, but the commandlet readiness check exited 1: desired=2,237, loaded=2,233, pending/cooks/active=0. Their trace data is usable for the table; the initial-readiness check remains a failed run. L80’s first walking attempt similarly exited with 2,235/2,237 loaded; its complete rerun passed.
|
||||
|
||||
## Cold arrival and timing
|
||||
|
||||
Seconds to LOD0–2 ready / whole view ready; three measured runs per policy and stratum, reported as medians.
|
||||
|
||||
| Stratum | L77 | L79 | L80 (HEAD) |
|
||||
|---|---:|---:|---:|
|
||||
| S0 | 2.798 / 2.798 | 1.488 / 2.730 | 1.298 / 2.917 |
|
||||
| S1 | 4.577 / 4.577 | 3.672 / 4.904 | 3.607 / 5.119 |
|
||||
| S2 | 4.501 / 4.501 | 3.691 / 4.720 | 3.068 / 4.318 |
|
||||
| S3 | 2.624 / 2.624 | 1.579 / 2.451 | 1.429 / 2.476 |
|
||||
|
||||
Protocol: rebuilt clean clang headless binary, 8 workers, identical data/flags, no external sampler; one unrecorded warm-up per stratum and policy, then three interleaved triplets in rotating policy order. Every headless process used `taskset -c 0-11`, `prlimit --as=6442450944`, and a 1,740 s timeout. L77/L79 are control policy IDs in the same final binary. L78’s 5.26 s and its L79 re-measure at 3.39 s are not comparable to these timings; neither used this binary and protocol. The older moving-camera values are likewise context only; this report reruns all three policies under one binary.
|
||||
|
||||
## Passage-cut audit
|
||||
|
||||
The L77 census profiled 1,602 tile visits. I selected the highest measured LOD0/1 segment-activity tiles across all four strates: 17 stratum records (14 unique tile keys), covering mouths, route body and support columns. L77’s census attributes 63% of near-LOD FI/TN CPU to passage work. On these active records, a zero-padding bound projected skips of 50.76% of passage iterations, 61.50% of carve visits, 49.76% of support visits and 51.72% of relief calls. That justified testing it.
|
||||
|
||||
The zero-padding candidate changed these samples:
|
||||
|
||||
| Stratum | Changed samples | Sign flips | Max absolute density delta location | Max surface displacement location |
|
||||
|---|---:|---:|---|---|
|
||||
| S0 | 19,434 | 0 | 54.329 at tile (-1,-2,-24), LOD1, sample (-34,-98,-1538) | 0 cm; no displaced crossing edge |
|
||||
| S1 | 63 | 0 | 2.198 at tile (-1,1,-43), LOD1, sample (-2,118,-2748) | 1.014 cm at edge start (-2,116,-2748), axis 1 |
|
||||
| S2 | 86 | 0 | 2.619 at tile (-1,1,-45), LOD1, sample (0,114,-2854) | 6.641 cm at edge start (0,112,-2854), axis 1 |
|
||||
| S3 | 21 | 0 | 0.455 at tile (1,-1,-55), LOD1, sample (80,-2,-3484) | 0 cm; no displaced crossing edge |
|
||||
|
||||
Per-sample base/candidate density, delta and sign-flip rows are in `Saved/Linux/l80/passage-census/deltas/strate{0..3}-candidate.csv`; the per-tile logs give individual counts and surface-edge details. The matching `*-candidate-pad48.csv` files contain headers only because there were no changed samples.
|
||||
|
||||
Padding to 48 voxels made all selected output samples identical: zero changes, zero sign flips and 0 cm displacement in every stratum. It also reduced projected skips to 0.226% of iterations, 0.098% of carve visits, 0.196% of support visits and 0.138% of relief calls; paired tile timings were mixed, with no consistent speedup. The useful cut therefore loses passage influence; the safe cut does not save meaningful CPU. The candidate was removed, with no field/version change, captures or peer-determinism run.
|
||||
|
||||
## Build, failures and files
|
||||
|
||||
The final Linux host build succeeded after timing with `-MaxParallelActions=4` using UBT’s local executor. An earlier UBA attempt made no action progress and was stopped before 30 minutes. The first local build caught an out-of-scope `MaxDesiredLevel` reference in the new Unreal coverage trace; that was fixed before the successful build. A first UBT attempt also failed before compilation when it tried to rotate a log in the read-only user config directory; the project config was temporarily redirected and restored.
|
||||
|
||||
Changed `Source/` files:
|
||||
|
||||
- `Source/VoxelForge/Private/VoxelWorld.cpp` — Unreal scheduler, hysteresis, movement test and coverage trace.
|
||||
- `Source/VoxelForge/Public/VoxelWorld.h` — movement-test duration state.
|
||||
- `Standalone/Source/StreamingWorld.h` — headless speed-aware order and matching hysteresis.
|
||||
- `Standalone/Source/StreamBenchmarkHeadless.cpp` — forwards route speed to the scheduler.
|
||||
- `Standalone/Source/StreamBenchmarkPipeline.h` — surface-delta location reporting.
|
||||
|
||||
Logs and runner scripts are under `Saved/Linux/l80/` and this directory. No commit, push or stash was made.
|
||||
@@ -0,0 +1,115 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run the L77/L79/L80 headless moving-camera and dwell protocol."""
|
||||
from __future__ import annotations
|
||||
import csv
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[4]
|
||||
OUT = ROOT / "Saved/Linux/l80/movement/headless"
|
||||
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||
BINARY = ROOT / "Saved/Linux/l80/bin/VoxelForgeCoreHeadless-l80-clean-clang"
|
||||
CAMERA = (19.166015625, -14.2001953125, -753.3740234375)
|
||||
POLICIES = {0: "L77", 2: "L79", 3: "L80"}
|
||||
SPEEDS = {"walk": 1.4, "fast": 30.0}
|
||||
SAMPLE = re.compile(r"^\[L79MovingSample\] (.*)$", re.M)
|
||||
SUMMARY = re.compile(r"^\[L79MovingSummary\] (.*)$", re.M)
|
||||
FIELDS = re.compile(r"([a-zA-Z0-9_]+)=([^ ]+)")
|
||||
|
||||
|
||||
def run(policy: int, speed_name: str) -> tuple[dict, list[dict]]:
|
||||
label = f"{speed_name}-{POLICIES[policy]}"
|
||||
log = OUT / f"{label}.log"
|
||||
OUT.mkdir(parents=True, exist_ok=True)
|
||||
camera = ",".join(f"{v:.12f}" for v in CAMERA) + ",0,0"
|
||||
command = [
|
||||
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||
"timeout", "1740s", str(BINARY), str(DATA), "--world-assembler-prototype",
|
||||
"--stream-benchmark", "--stream-pipeline", "--workers=8", "--clip-radius=3",
|
||||
"--max-clip-level=8", "--max-level=8", f"--camera={camera}",
|
||||
"--profile-lod-work", f"--stream-scheduler-policy={policy}",
|
||||
f"--moving-camera-speed-mps={SPEEDS[speed_name]:.1f}",
|
||||
"--moving-camera-duration-s=12", "--moving-camera-dwell-s=5",
|
||||
]
|
||||
start = time.monotonic()
|
||||
with log.open("w", encoding="utf-8") as output:
|
||||
result = subprocess.run(command, cwd=ROOT, stdout=output,
|
||||
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||
content = log.read_text(encoding="utf-8", errors="replace")
|
||||
samples = [dict(FIELDS.findall(match.group(1))) for match in SAMPLE.finditer(content)]
|
||||
summary_match = SUMMARY.search(content)
|
||||
engine = dict(FIELDS.findall(summary_match.group(1))) if summary_match else {}
|
||||
if result.returncode or not samples or not engine:
|
||||
raise SystemExit(f"moving-camera run failed or trace missing: {log}; exit={result.returncode}")
|
||||
for row in samples:
|
||||
for name in ("policy", "speed_mps", "t_s", "near_ready_pct", "far_missing_pct",
|
||||
"far_hole_pct", "far_stale_max_s", "generation_failures"):
|
||||
row[name] = float(row[name])
|
||||
row["label"] = label
|
||||
if int(float(engine.get("generation_failures", "-1"))) != 0:
|
||||
raise SystemExit(f"tile generation failure: {log}")
|
||||
run_info = {
|
||||
"label": label, "policy": POLICIES[policy], "policy_id": policy,
|
||||
"speed": speed_name, "speed_mps": SPEEDS[speed_name],
|
||||
"exit_code": result.returncode, "wall_s": round(time.monotonic() - start, 3),
|
||||
"samples": len(samples), "engine_summary": engine,
|
||||
"log": str(log.relative_to(ROOT)),
|
||||
}
|
||||
print("[L80HeadlessMoveRun] " + json.dumps(run_info, sort_keys=True), flush=True)
|
||||
return run_info, samples
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if not DATA.is_file() or not BINARY.is_file():
|
||||
raise SystemExit(f"missing input: {DATA} or {BINARY}")
|
||||
runs, all_samples = [], []
|
||||
schedule = [(policy, speed) for speed in ("walk", "fast")
|
||||
for policy in (0, 2, 3)]
|
||||
for policy, speed in schedule:
|
||||
row, samples = run(policy, speed)
|
||||
runs.append(row)
|
||||
all_samples.extend(samples)
|
||||
(OUT / "runs-partial.json").write_text(
|
||||
json.dumps(runs, indent=2) + "\n", encoding="utf-8")
|
||||
summary = []
|
||||
for speed_name, speed in SPEEDS.items():
|
||||
for policy in POLICIES.values():
|
||||
rows = [row for row in all_samples
|
||||
if row["speed_mps"] == speed and int(row["policy"]) ==
|
||||
next(pid for pid, name in POLICIES.items() if name == policy)]
|
||||
moving = [row for row in rows if row.get("phase") == "move"]
|
||||
dwell = [row for row in rows if row.get("phase") == "dwell"]
|
||||
summary.append({
|
||||
"speed": speed_name, "speed_mps": speed, "policy": policy,
|
||||
"moving_samples": len(moving), "dwell_samples": len(dwell),
|
||||
"far_holes_moving_mean_pct": round(sum(r["far_hole_pct"] for r in moving)
|
||||
/ max(1, len(moving)), 3),
|
||||
"far_holes_moving_max_pct": max((r["far_hole_pct"] for r in moving), default=None),
|
||||
"near_ready_moving_mean_pct": round(sum(r["near_ready_pct"] for r in moving)
|
||||
/ max(1, len(moving)), 3),
|
||||
"near_ready_moving_min_pct": min((r["near_ready_pct"] for r in moving), default=None),
|
||||
"far_holes_dwell_last_pct": dwell[-1]["far_hole_pct"] if dwell else None,
|
||||
"near_ready_dwell_last_pct": dwell[-1]["near_ready_pct"] if dwell else None,
|
||||
"first_all_far_ready_s": next(
|
||||
(r["engine_summary"].get("first_all_far_ready_s") for r in runs
|
||||
if r["speed"] == speed_name and r["policy"] == policy), None),
|
||||
"generation_failures": next(
|
||||
(r["engine_summary"].get("generation_failures") for r in runs
|
||||
if r["speed"] == speed_name and r["policy"] == policy), None),
|
||||
})
|
||||
(OUT / "summary.json").write_text(
|
||||
json.dumps({"runs": runs, "summary": summary}, indent=2) + "\n",
|
||||
encoding="utf-8")
|
||||
sample_path = OUT / "samples.csv"
|
||||
if all_samples:
|
||||
with sample_path.open("w", newline="", encoding="utf-8") as stream:
|
||||
writer = csv.DictWriter(stream, fieldnames=list(all_samples[0]))
|
||||
writer.writeheader()
|
||||
writer.writerows(all_samples)
|
||||
return 0
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,92 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Find active LOD0/1 passage tiles in each owner stratum."""
|
||||
from __future__ import annotations
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[4]
|
||||
OUT = ROOT / "Saved/Linux/l80/passage-census"
|
||||
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||
BINARY = ROOT / "Saved/Linux/l79/bin/VoxelForgeCoreHeadless-l79-final"
|
||||
CAMERAS = {
|
||||
0: (8.0, 16.0, -372.2626953125),
|
||||
1: (25.8369140625, 3.19921875, -737.0),
|
||||
2: (19.166015625, -14.2001953125, -753.3740234375),
|
||||
3: (-20.0625, -5.068359375, -900.353515625),
|
||||
}
|
||||
TILE = re.compile(r"^\[PassageComposerTile\] (.*)$", re.M)
|
||||
FIELD = re.compile(r"([a-zA-Z0-9_]+)=([^ ]+)")
|
||||
KEY = re.compile(r"key=\((-?\d+),(-?\d+),(-?\d+),lod(\d+)\)")
|
||||
|
||||
|
||||
def run(strate: int, camera: tuple[float, float, float]) -> dict:
|
||||
log = OUT / f"strate{strate}.log"
|
||||
camera_arg = ",".join(f"{v:.12f}" for v in camera) + ",0,0"
|
||||
command = [
|
||||
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||
"timeout", "1740s", str(BINARY), str(DATA), "--world-assembler-prototype",
|
||||
"--stream-benchmark", "--stream-pipeline", "--workers=8", "--clip-radius=3",
|
||||
"--max-clip-level=8", "--max-level=1", f"--camera={camera_arg}",
|
||||
"--profile-lod-work", "--profile-passage-composer",
|
||||
]
|
||||
start = time.monotonic()
|
||||
with log.open("w", encoding="utf-8") as out:
|
||||
proc = subprocess.run(command, cwd=ROOT, stdout=out, stderr=subprocess.STDOUT,
|
||||
check=False, timeout=1740)
|
||||
text = log.read_text(encoding="utf-8", errors="replace")
|
||||
rows = []
|
||||
for match in TILE.finditer(text):
|
||||
row = dict(FIELD.findall(match.group(1)))
|
||||
for name, value in tuple(row.items()):
|
||||
try:
|
||||
row[name] = int(value) if name not in {"worker_ms"} else float(value)
|
||||
except ValueError:
|
||||
pass
|
||||
key = KEY.search(match.group(1))
|
||||
if key:
|
||||
row["coord"] = [int(key.group(i)) for i in range(1, 4)]
|
||||
row["lod"] = int(key.group(4))
|
||||
row["strate"] = strate
|
||||
rows.append(row)
|
||||
summary = next((line for line in text.splitlines()
|
||||
if line.startswith("[StreamPipeline] ")), None)
|
||||
result = {"strate": strate, "exit_code": proc.returncode,
|
||||
"wall_s": round(time.monotonic() - start, 3),
|
||||
"profiled_tiles": len(rows), "log": str(log.relative_to(ROOT)),
|
||||
"summary": summary, "tiles": rows}
|
||||
print("[L80PassageCensus] " + json.dumps(
|
||||
{k: v for k, v in result.items() if k != "tiles"}, sort_keys=True), flush=True)
|
||||
if proc.returncode or not rows:
|
||||
raise SystemExit(f"census failed or emitted no tile rows: {log} exit={proc.returncode}")
|
||||
return result
|
||||
|
||||
|
||||
def main() -> int:
|
||||
OUT.mkdir(parents=True, exist_ok=True)
|
||||
runs = [run(i, CAMERAS[i]) for i in range(4)]
|
||||
tiles = [tile for result in runs for tile in result["tiles"]]
|
||||
categories = {
|
||||
"mouth": "mouth_route_segment_checks",
|
||||
"route_body": "carve_segment_visits",
|
||||
"support": "support_segment_visits",
|
||||
}
|
||||
leaders = {}
|
||||
for strate in range(4):
|
||||
for lod in (0, 1):
|
||||
for category, metric in categories.items():
|
||||
candidates = [tile for tile in tiles
|
||||
if tile["strate"] == strate and tile["lod"] == lod
|
||||
and tile.get(metric, 0) > 0]
|
||||
candidates.sort(key=lambda x: (x.get(metric, 0), x.get("worker_ms", 0)),
|
||||
reverse=True)
|
||||
leaders[f"s{strate}-l{lod}-{category}"] = candidates[:2]
|
||||
payload = {"runs": [{k: v for k, v in r.items() if k != "tiles"} for r in runs],
|
||||
"profiled_tile_count": len(tiles), "tiles": tiles, "leaders": leaders}
|
||||
(OUT / "summary.json").write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8")
|
||||
return 0
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,98 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Clean, sampler-free, interleaved cold-readiness comparison for L77/L79/L80."""
|
||||
from __future__ import annotations
|
||||
import json
|
||||
import re
|
||||
import statistics
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[4]
|
||||
OUT = ROOT / "Saved/Linux/l80/timing"
|
||||
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||
BINARY = ROOT / "Saved/Linux/l80/bin/VoxelForgeCoreHeadless-l80-clean-clang"
|
||||
POLICIES = {0: "L77", 2: "L79", 3: "L80"}
|
||||
CAMERAS = {
|
||||
0: (8.0, 16.0, -372.2626953125),
|
||||
1: (25.8369140625, 3.19921875, -737.0),
|
||||
2: (19.166015625, -14.2001953125, -753.3740234375),
|
||||
3: (-20.0625, -5.068359375, -900.353515625),
|
||||
}
|
||||
PIPELINE = re.compile(r"^\[StreamPipeline\] (.*)$", re.M)
|
||||
FIELDS = re.compile(r"([a-zA-Z0-9_]+)=([^ ]+)")
|
||||
NEAR = re.compile(r"^\[WorldAssemblerNearLODReady\] lods=0-2 tiles=(\d+)/(\d+) cold_ready_s=([0-9.]+)$", re.M)
|
||||
|
||||
|
||||
def run(strate: int, policy: int, phase: str, replicate: int) -> dict:
|
||||
OUT.mkdir(parents=True, exist_ok=True)
|
||||
label = f"s{strate}-{POLICIES[policy]}-{phase}{replicate}"
|
||||
log = OUT / f"{label}.log"
|
||||
camera = ",".join(f"{v:.12f}" for v in CAMERAS[strate]) + ",0,0"
|
||||
command = [
|
||||
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||
"timeout", "1740s", str(BINARY), str(DATA), "--world-assembler-prototype",
|
||||
"--stream-benchmark", "--stream-pipeline", "--workers=8", "--clip-radius=3",
|
||||
"--max-clip-level=8", "--max-level=8", f"--camera={camera}",
|
||||
"--profile-lod-work", f"--stream-scheduler-policy={policy}",
|
||||
]
|
||||
start = time.monotonic()
|
||||
with log.open("w", encoding="utf-8") as output:
|
||||
result = subprocess.run(command, cwd=ROOT, stdout=output,
|
||||
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||
content = log.read_text(encoding="utf-8", errors="replace")
|
||||
pipeline_match = PIPELINE.search(content)
|
||||
near_match = NEAR.search(content)
|
||||
if result.returncode or pipeline_match is None or near_match is None:
|
||||
raise SystemExit(f"readiness run failed or trace missing: {log}; exit={result.returncode}")
|
||||
pipeline = dict(FIELDS.findall(pipeline_match.group(1)))
|
||||
row = {
|
||||
"strate": strate, "policy": POLICIES[policy], "policy_id": policy,
|
||||
"phase": phase, "replicate": replicate, "exit_code": result.returncode,
|
||||
"wall_s": round(time.monotonic() - start, 3),
|
||||
"lod02_ready_s": float(near_match.group(3)),
|
||||
"whole_view_ready_s": float(pipeline["whole_view_ready"].removesuffix("s")),
|
||||
"whole_tiles": int(pipeline["whole_tiles"].split("/")[0]),
|
||||
"log": str(log.relative_to(ROOT)),
|
||||
}
|
||||
print("[L80TimingRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||
return row
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if not DATA.is_file() or not BINARY.is_file():
|
||||
raise SystemExit(f"missing input: {DATA} or {BINARY}")
|
||||
rows = []
|
||||
# One unrecorded warm-up per stratum and policy. It warms binary/data pages only.
|
||||
for strate in range(4):
|
||||
for policy in (0, 2, 3):
|
||||
run(strate, policy, "warmup", 0)
|
||||
# Three interleaved triplets; rotate order to keep any drift from favoring one policy.
|
||||
orders = ((0, 2, 3), (3, 2, 0), (2, 0, 3))
|
||||
for strate in range(4):
|
||||
for replicate, order in enumerate(orders, 1):
|
||||
for policy in order:
|
||||
row = run(strate, policy, "measured", replicate)
|
||||
rows.append(row)
|
||||
(OUT / "runs-partial.json").write_text(
|
||||
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||
summary = []
|
||||
for strate in range(4):
|
||||
group = [row for row in rows if row["strate"] == strate]
|
||||
entry = {"strate": strate, "measured_pairs_per_policy": 3}
|
||||
for policy in ("L77", "L79", "L80"):
|
||||
subset = [row for row in group if row["policy"] == policy]
|
||||
entry[f"{policy}_lod02_s"] = [row["lod02_ready_s"] for row in subset]
|
||||
entry[f"{policy}_lod02_median_s"] = statistics.median(
|
||||
row["lod02_ready_s"] for row in subset)
|
||||
entry[f"{policy}_whole_s"] = [row["whole_view_ready_s"] for row in subset]
|
||||
entry[f"{policy}_whole_median_s"] = statistics.median(
|
||||
row["whole_view_ready_s"] for row in subset)
|
||||
summary.append(entry)
|
||||
(OUT / "summary.json").write_text(
|
||||
json.dumps({"runs": rows, "summary": summary}, indent=2) + "\n",
|
||||
encoding="utf-8")
|
||||
return 0
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,164 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run the Unreal null-RHI moving and dwell trace for L77/L79/L80."""
|
||||
from __future__ import annotations
|
||||
import csv
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[4]
|
||||
HOST = Path("/home/jahni/VFHost")
|
||||
HOST_PLUGIN_SAVED = HOST / "Plugins/VoxelForge/Saved/Linux/l80"
|
||||
HOST_SAVED = HOST / "Saved/Linux/l80"
|
||||
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
|
||||
OUT = ROOT / "Saved/Linux/l80/movement/unreal"
|
||||
SPAWN_M = (19.166015625, -14.2001953125, -753.3740234375)
|
||||
SPEEDS = {"walk": 140.0, "fast": 3000.0}
|
||||
POLICIES = {0: "L77", 2: "L79", 3: "L80"}
|
||||
SAMPLE_SECONDS = 0.25
|
||||
MOVE_START_S = 6.0
|
||||
MOVE_DURATION_S = 12.0
|
||||
DWELL_S = 5.0
|
||||
EXIT_S = MOVE_START_S + MOVE_DURATION_S + DWELL_S
|
||||
EVENT = re.compile(r"^\{" )
|
||||
|
||||
|
||||
def parse_trace(path: Path, run: dict) -> tuple[list[dict], dict]:
|
||||
events = []
|
||||
if path.is_file():
|
||||
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||
if EVENT.match(line):
|
||||
try:
|
||||
events.append(json.loads(line))
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
samples = [e for e in events if e.get("type") == "streaming_sample"]
|
||||
move = [e for e in samples if float(e.get("speed_cm_s", 0.0)) >= 50.0]
|
||||
stop = next((e for e in events if e.get("type") == "streaming_test_move_stopped"), None)
|
||||
stop_t = float(stop.get("t_s", 0.0)) if stop else (
|
||||
float(move[0].get("t_s", 0.0)) + MOVE_DURATION_S if move else float("inf"))
|
||||
dwell = [e for e in samples if float(e.get("t_s", 0.0)) >= stop_t
|
||||
and float(e.get("speed_cm_s", 0.0)) < 50.0]
|
||||
rows = []
|
||||
for e in samples:
|
||||
desired = e.get("desired_by_lod", [0] * 9)
|
||||
resident = e.get("resident_by_lod", [0] * 9)
|
||||
covered = e.get("covered_by_lod", resident)
|
||||
far_desired = sum(desired[3:])
|
||||
far_covered = sum(covered[3:])
|
||||
rows.append({
|
||||
"run": run["label"], "policy": run["policy"], "speed": run["speed"],
|
||||
"t_s": e.get("t_s"), "speed_cm_s": e.get("speed_cm_s"),
|
||||
"far_desired": far_desired, "far_covered": far_covered,
|
||||
"far_hole_pct": round(100.0 * (far_desired - far_covered)
|
||||
/ max(1, far_desired), 3),
|
||||
"near_desired": sum(desired[:3]), "near_resident": sum(resident[:3]),
|
||||
"near_ready_pct": round(100.0 * sum(resident[:3])
|
||||
/ max(1, sum(desired[:3])), 3),
|
||||
"phase": "move" if float(e.get("speed_cm_s", 0.0)) >= 50.0 else
|
||||
"dwell" if float(e.get("t_s", 0.0)) >= stop_t else "arrival",
|
||||
})
|
||||
moving_rows = [r for r in rows if r["phase"] == "move"]
|
||||
dwell_rows = [r for r in rows if r["phase"] == "dwell"]
|
||||
summary = {
|
||||
"moving_samples": len(moving_rows), "dwell_samples": len(dwell_rows),
|
||||
"far_holes_moving_mean_pct": round(sum(r["far_hole_pct"] for r in moving_rows)
|
||||
/ max(1, len(moving_rows)), 3),
|
||||
"near_ready_moving_mean_pct": round(sum(r["near_ready_pct"] for r in moving_rows)
|
||||
/ max(1, len(moving_rows)), 3),
|
||||
"far_holes_dwell_last_pct": dwell_rows[-1]["far_hole_pct"] if dwell_rows else None,
|
||||
"near_ready_dwell_last_pct": dwell_rows[-1]["near_ready_pct"] if dwell_rows else None,
|
||||
"first_all_far_ready_s": next((r["t_s"] for r in moving_rows
|
||||
if r["far_hole_pct"] == 0.0), None),
|
||||
"move_stopped_t_s": stop_t if stop else None,
|
||||
}
|
||||
return rows, summary
|
||||
|
||||
|
||||
def run(policy: int, speed_name: str) -> tuple[dict, list[dict]]:
|
||||
label = f"{speed_name}-{POLICIES[policy]}"
|
||||
trace, abslog, stdout = (OUT / f"{label}.trace.jsonl", OUT / f"{label}.log",
|
||||
OUT / f"{label}.stdout.log")
|
||||
output = HOST_PLUGIN_SAVED / "output" / label
|
||||
userdir = HOST_SAVED / "user" / label
|
||||
zendata = HOST_SAVED / "zen" / label
|
||||
ddc = HOST_SAVED / "DDC"
|
||||
for directory in (OUT, output, userdir, zendata, ddc):
|
||||
directory.mkdir(parents=True, exist_ok=True)
|
||||
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWN_M)
|
||||
command = [
|
||||
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
|
||||
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
|
||||
"-notraceserver", "-nocrashreports", "-DisablePlugins=AndroidFileServer",
|
||||
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
|
||||
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
|
||||
"-archetype=TunnelNetwork", "-slot=0", "-modes=export", "-gameconfig=1",
|
||||
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
|
||||
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0", "-coreplantest=0",
|
||||
"-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings", "-corehybridmesher=0",
|
||||
f"-out={output}", "-voxel.StreamingCoarseFirst=1",
|
||||
"-voxel.StreamingDynamicNearReserve=1", "-voxel.StreamingLegacyPendingResultCap=0",
|
||||
"-voxel.CoreRegionConcurrentAdmission=1", "-voxel.StreamingResultBacklogCap=256",
|
||||
"-voxel.StreamingReadyQueueDepth=8", "-voxel.StreamingBackgroundWorkerCap=1",
|
||||
"-voxel.CoreColumnCacheDirectMapped=0", "-voxel.CoreMCGradientMemoization=1",
|
||||
"-voxel.StreamingCollisionLookahead=1", "-voxel.TestMove=1",
|
||||
f"-voxel.TestMoveSpeedCmPerSecond={SPEEDS[speed_name]:.1f}",
|
||||
f"-voxel.TestMoveStartSeconds={MOVE_START_S:.1f}",
|
||||
f"-voxel.TestMoveDurationSeconds={MOVE_DURATION_S:.1f}",
|
||||
f"-voxel.TestExitSeconds={EXIT_S:.1f}", "-voxel.TestMoveDirection=1,0,0",
|
||||
f"-voxel.StreamingSchedulerPolicy={policy}", "-voxel.TestExitOnSteady=0",
|
||||
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
|
||||
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
|
||||
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
|
||||
f"-abslog={abslog}",
|
||||
]
|
||||
wrapped = ["taskset", "-c", "0-11", "timeout", "1740s", *command]
|
||||
start = time.monotonic()
|
||||
with stdout.open("w", encoding="utf-8") as stream:
|
||||
result = subprocess.run(wrapped, cwd=ROOT, stdout=stream,
|
||||
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||
run_info = {"label": label, "policy": POLICIES[policy], "policy_id": policy,
|
||||
"speed": speed_name, "speed_cm_s": SPEEDS[speed_name],
|
||||
"exit_code": result.returncode, "wall_s": round(time.monotonic()-start, 3),
|
||||
"trace": str(trace.relative_to(ROOT)), "abslog": str(abslog.relative_to(ROOT)),
|
||||
"stdout": str(stdout.relative_to(ROOT))}
|
||||
rows, summary = parse_trace(trace, run_info)
|
||||
run_info.update(summary)
|
||||
stdout_text = stdout.read_text(encoding="utf-8", errors="replace")
|
||||
failure_match = re.search(
|
||||
r"\[VoxelForgeCoreWorldTest\] failed: ([^\r\n]+)", stdout_text)
|
||||
if result.returncode:
|
||||
run_info["run_failure"] = (failure_match.group(1) if failure_match
|
||||
else f"process exit {result.returncode}")
|
||||
if (not rows or summary["moving_samples"] < 40 or summary["dwell_samples"] < 16):
|
||||
raise SystemExit(f"Unreal trace incomplete: {stdout}; exit={result.returncode}; "
|
||||
f"moving={summary['moving_samples']} dwell={summary['dwell_samples']}")
|
||||
print("[L80UnrealMoveRun] " + json.dumps(run_info, sort_keys=True), flush=True)
|
||||
return run_info, rows
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if not EDITOR.is_file():
|
||||
raise SystemExit(f"missing Unreal commandlet executable: {EDITOR}")
|
||||
results, samples = [], []
|
||||
for speed in ("walk", "fast"):
|
||||
for policy in (0, 2, 3):
|
||||
run_info, rows = run(policy, speed)
|
||||
results.append(run_info)
|
||||
samples.extend(rows)
|
||||
(OUT / "runs-partial.json").write_text(json.dumps(results, indent=2) + "\n",
|
||||
encoding="utf-8")
|
||||
(OUT / "summary.json").write_text(json.dumps({"runs": results}, indent=2) + "\n",
|
||||
encoding="utf-8")
|
||||
if samples:
|
||||
with (OUT / "samples.csv").open("w", newline="", encoding="utf-8") as stream:
|
||||
writer = csv.DictWriter(stream, fieldnames=list(samples[0]))
|
||||
writer.writeheader()
|
||||
writer.writerows(samples)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in new issue
Block a user