research: floor-ribbon falsifier and band column cache (L106, no code merged)
A route-only floor profile cannot reproduce current support semantics: support is gated per sample by native density < 0, player-capsule clearance, later-bore deferral and neighbouring- passage capsule checks (the overlapping-coil tile exercises all of them). The exact per-tile boundary-band XY column cache matched all 24 hashes but showed no consistent readiness gain, so it is kept as a patch only (patches/band-column-cache-not-merged.patch), not merged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
No files matched your search
@@ -0,0 +1,32 @@
|
|||||||
|
# Linux 106
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
**Floor/support ribbon: stopped at the falsifier.** A route-only floor profile cannot replace the current support decisions. Support is gated by pointwise native density (`density < 0`), player-capsule clearance, later-bore deferral, and neighboring-passage capsule checks. Mouth support also blends back the native field and its relief under another capsule check. The overlapping-coil tile exercises the later-bore and neighboring-capsule paths.
|
||||||
|
|
||||||
|
At W1, the ordinary FI tile `(2,-2,-93,lod0)` had 81,816 support-segment visits and 33,422 capsule checks, with no later-bore or other-capsule checks. The overlapping-coil tile `(1,-2,-92,lod0)` had 181,956 visits, 93,034 capsule checks, 3,219 later-bore checks, and 2,442 other-capsule checks. These counts matched L105. Both tiles also had identical L105/L106 density, mesh, payload, and topology hashes.
|
||||||
|
|
||||||
|
No local floor candidate was emitted, so candidate-specific crossing and capsule-clearance deltas are **not measured**. For a minimal hybrid, precompute route-floor geometry in the tile field while retaining the native-density gate, both capsule checks, later-bore deferral, and mouth restoration. This can remove route projection work without changing those support rules.
|
||||||
|
|
||||||
|
**Boundary bands:** added a per-tile XY-column cache for active transition pairs. It serves exact lattice XY only; off-grid queries still call the original exact calculation. The FI profile measured 681,651 column/pair evaluations, 4,530,549 lattice cache hits, and 75,758 exact off-grid fallbacks. A seam-centered profile measured 623,891 evaluations, 2,578,260 hits, and 21,328 fallbacks.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
- Exact hashes: **24/24** Clang/GCC × W1/W8/W12 × SW/FI/TN/VS runs matched L105 tile and whole-view hashes; zero mismatches. Source stamp: `a033e6b53537f3550549a7a77f93f5a5d6594263c208cbbffc5d403f6d7ad7b5`.
|
||||||
|
- Fast tier: **PASS**, 274 checks, 269 PASS, 5 historical diagnostics, 0 failures. Walkability, capsule clearance, support, floor/mouth profiles, route binding, and headroom gates passed.
|
||||||
|
- L80 readiness: the required 3-pair medians are below. Near and whole sets completed in every cell.
|
||||||
|
|
||||||
|
| Stratum | Near W8 | Near W12 | Whole W8 | Whole W12 |
|
||||||
|
|---|---:|---:|---:|---:|
|
||||||
|
| SW | +4.8% | −0.6% | +8.7% | −1.1% |
|
||||||
|
| FI | −4.2% | +0.1% | −1.5% | +0.6% |
|
||||||
|
| TN | −3.4% | −2.4% | −2.0% | −1.5% |
|
||||||
|
| VS | −2.3% | −2.4% | −1.2% | −1.1% |
|
||||||
|
|
||||||
|
Readiness was noisy. A second full 3-pair set produced FI W12 deltas of +11.9% near and +7.6% whole; a focused 3-pair FI W12 retest measured −0.9% and −0.5%. All three result sets are under `Saved/Linux/l106/readiness-paired*`; I do not claim a consistent readiness speedup.
|
||||||
|
|
||||||
|
- Captures: visually checked all nine L105/L106 pairs; every pair had 0 changed pixels. Listed views: `passage/0-1/0`, `passage/1-2/0`, and `passage/2-3/0`, each at upper mouth, lower mouth, and mid-50. The [contact sheet](captures/contact-sheet.png) shows every pair; exact PNG paths are in `Saved/Linux/l106/captures/summary.json`.
|
||||||
|
- Unreal Linux Development build: **PASS**, 27/27 actions, `-NoUBA -MaxParallelActions=4`, after timing and without `prlimit`.
|
||||||
|
- FI Unreal arrivals: **3/3 PASS**, each settled 1,216 near tiles in 1.543–1.550 s from first admission. They ran with `taskset`, without `prlimit`.
|
||||||
|
|
||||||
|
No world behavior changed, so the evaluator version remains 216. No commit, push, or stash was made.
|
||||||
@@ -0,0 +1,602 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Capture matched L105/L106 passage mouth and mid-route views."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import struct
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
import zlib
|
||||||
|
from array import array
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
OUT = ROOT / "Saved/Linux/l106/captures"
|
||||||
|
IMAGE_DIR = Path(__file__).resolve().parent / "captures"
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
LAYOUT_PATH = ROOT / "Saved/Linux/l99/resolved-sw.json"
|
||||||
|
BINARIES = {
|
||||||
|
"before-l105": ROOT / "Saved/Linux/l105/bin/VoxelForgeCoreHeadless-l105-clang",
|
||||||
|
"after-l106": ROOT / "Saved/Linux/l106/bin/VoxelForgeCoreHeadless-l106-clang",
|
||||||
|
}
|
||||||
|
RENDERER = ROOT / "Saved/Linux/l106/bin/VoxelForgeOffscreenCapture-l106-clang"
|
||||||
|
VOXEL_Q = 256.0
|
||||||
|
CORE_UM_PER_VOXEL = 250_000.0
|
||||||
|
ROUTE_POINTS = {}
|
||||||
|
ROUTE_LENGTHS = {}
|
||||||
|
|
||||||
|
|
||||||
|
def add(a, b):
|
||||||
|
return tuple(x + y for x, y in zip(a, b))
|
||||||
|
|
||||||
|
|
||||||
|
def sub(a, b):
|
||||||
|
return tuple(x - y for x, y in zip(a, b))
|
||||||
|
|
||||||
|
|
||||||
|
def mul(a, s):
|
||||||
|
return tuple(x * s for x in a)
|
||||||
|
|
||||||
|
|
||||||
|
def dot(a, b):
|
||||||
|
return sum(x * y for x, y in zip(a, b))
|
||||||
|
|
||||||
|
|
||||||
|
def norm(a):
|
||||||
|
length = math.sqrt(dot(a, a))
|
||||||
|
if length < 1.0e-8:
|
||||||
|
raise ValueError("degenerate capture direction")
|
||||||
|
return tuple(x / length for x in a)
|
||||||
|
|
||||||
|
|
||||||
|
def fmt(values):
|
||||||
|
return ",".join(f"{value:.9f}" for value in values)
|
||||||
|
|
||||||
|
|
||||||
|
def angles(camera, target):
|
||||||
|
delta = sub(target, camera)
|
||||||
|
return (math.degrees(math.atan2(delta[0], delta[1])),
|
||||||
|
math.degrees(math.atan2(delta[2], math.hypot(delta[0], delta[1]))))
|
||||||
|
|
||||||
|
|
||||||
|
def interpolate(points, distance):
|
||||||
|
remaining = max(0.0, distance)
|
||||||
|
for a, b in zip(points, points[1:]):
|
||||||
|
delta = sub(b[0], a[0])
|
||||||
|
length = math.sqrt(dot(delta, delta))
|
||||||
|
if length < 1.0e-8:
|
||||||
|
continue
|
||||||
|
if remaining <= length:
|
||||||
|
t = remaining / length
|
||||||
|
return (tuple(a[0][i] + delta[i] * t for i in range(3)),
|
||||||
|
a[1] + (b[1] - a[1]) * t)
|
||||||
|
remaining -= length
|
||||||
|
return points[-1]
|
||||||
|
|
||||||
|
|
||||||
|
def route_frame(points, distance):
|
||||||
|
center, radius = interpolate(points, distance)
|
||||||
|
before, _ = interpolate(points, max(0.0, distance - 0.25))
|
||||||
|
after, _ = interpolate(points, min(sum(
|
||||||
|
math.sqrt(dot(sub(b[0], a[0]), sub(b[0], a[0])))
|
||||||
|
for a, b in zip(points, points[1:])), distance + 0.25))
|
||||||
|
tangent = norm(sub(after, before))
|
||||||
|
horizontal = norm((tangent[0], tangent[1], 0.0))
|
||||||
|
side = (-horizontal[1], horizontal[0], 0.0)
|
||||||
|
return center, radius, tangent, side
|
||||||
|
|
||||||
|
|
||||||
|
def route_length(points):
|
||||||
|
return sum(math.sqrt(dot(sub(b[0], a[0]), sub(b[0], a[0])))
|
||||||
|
for a, b in zip(points, points[1:]))
|
||||||
|
|
||||||
|
|
||||||
|
def nearest_route(points, point):
|
||||||
|
best = None
|
||||||
|
distance_along = 0.0
|
||||||
|
for a, b in zip(points, points[1:]):
|
||||||
|
delta = sub(b[0], a[0])
|
||||||
|
length_sq = dot(delta, delta)
|
||||||
|
if length_sq < 1.0e-10:
|
||||||
|
continue
|
||||||
|
segment_length = math.sqrt(length_sq)
|
||||||
|
t = max(0.0, min(1.0, dot(sub(point, a[0]), delta) / length_sq))
|
||||||
|
center = add(a[0], mul(delta, t))
|
||||||
|
tangent = norm(delta)
|
||||||
|
radial = sub(point, center)
|
||||||
|
radial = sub(radial, mul(tangent, dot(radial, tangent)))
|
||||||
|
distance = math.sqrt(dot(sub(point, center), sub(point, center)))
|
||||||
|
if best is None or distance < best[0]:
|
||||||
|
radius = a[1] + (b[1] - a[1]) * t
|
||||||
|
best = (distance, center, tangent, radial, radius,
|
||||||
|
distance_along + t * segment_length)
|
||||||
|
distance_along += segment_length
|
||||||
|
if best is None:
|
||||||
|
raise ValueError("passage route has no non-degenerate segment")
|
||||||
|
return best
|
||||||
|
|
||||||
|
|
||||||
|
def run(name, command, headless):
|
||||||
|
log = OUT / f"{name}.log"
|
||||||
|
wrapped = ["taskset", "-c", "0-11"]
|
||||||
|
if headless:
|
||||||
|
wrapped += ["prlimit", "--as=6442450944", "--"]
|
||||||
|
wrapped += ["timeout", "1740s", *map(str, command)]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
result = subprocess.run(wrapped, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1745)
|
||||||
|
row = {"name": name, "exit_code": result.returncode,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"log": str(log.relative_to(ROOT))}
|
||||||
|
print("[L106CaptureRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
if result.returncode != 0:
|
||||||
|
raise RuntimeError(f"capture step failed: {name}; see {log}")
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def nearest_surface_anchor(mesh, point, tile_search_margin_m=20.0):
|
||||||
|
"""Return the nearest surface vertex and outward mesh normal from a VFMSH scene."""
|
||||||
|
best_distance_sq = float("inf")
|
||||||
|
best_position = best_normal = None
|
||||||
|
with mesh.open("rb") as stream:
|
||||||
|
magic = stream.read(8)
|
||||||
|
has_materials = magic == b"VFMSH002"
|
||||||
|
if magic not in (b"VFMSH001", b"VFMSH002"):
|
||||||
|
raise RuntimeError(f"unsupported mesh scene header: {mesh}")
|
||||||
|
version, voxel_um, tile_count = struct.unpack("<3I", stream.read(12))
|
||||||
|
if version != (2 if has_materials else 1):
|
||||||
|
raise RuntimeError(f"mesh scene version mismatch: {mesh}")
|
||||||
|
stream.seek(40, 1) # stored camera position, yaw, and pitch
|
||||||
|
for _ in range(tile_count):
|
||||||
|
key = struct.unpack("<4i", stream.read(16))
|
||||||
|
surface_count, _, vertex_count, index_count, mask_count = struct.unpack(
|
||||||
|
"<5I", stream.read(20))
|
||||||
|
extent = (32 << key[3]) * voxel_um / 1_000_000.0
|
||||||
|
origin = (key[0] * extent, key[1] * extent, key[2] * extent)
|
||||||
|
if any(point[axis] < origin[axis] - tile_search_margin_m or
|
||||||
|
point[axis] > origin[axis] + extent + tile_search_margin_m
|
||||||
|
for axis in range(3)):
|
||||||
|
stream.seek(vertex_count * 24 + (10 * vertex_count if has_materials else 0)
|
||||||
|
+ index_count * 4 + mask_count, 1)
|
||||||
|
continue
|
||||||
|
positions = array("f")
|
||||||
|
positions.frombytes(stream.read(vertex_count * 12))
|
||||||
|
normals = array("f")
|
||||||
|
normals.frombytes(stream.read(vertex_count * 12))
|
||||||
|
if has_materials:
|
||||||
|
stream.seek(vertex_count * 10, 1)
|
||||||
|
stream.seek(index_count * 4 + mask_count, 1)
|
||||||
|
if struct.pack("=I", 1) != struct.pack("<I", 1):
|
||||||
|
positions.byteswap()
|
||||||
|
normals.byteswap()
|
||||||
|
for index in range(min(surface_count, vertex_count)):
|
||||||
|
offset = index * 3
|
||||||
|
# VFMSH stores vertex coordinates in world space; tile origin
|
||||||
|
# is metadata for edge rendering and must not be added again.
|
||||||
|
position = tuple(float(positions[offset + axis]) for axis in range(3))
|
||||||
|
delta = sub(position, point)
|
||||||
|
distance_sq = dot(delta, delta)
|
||||||
|
if distance_sq >= best_distance_sq:
|
||||||
|
continue
|
||||||
|
raw_normal = tuple(float(normals[offset + axis]) for axis in range(3))
|
||||||
|
if dot(raw_normal, raw_normal) < 1.0e-10:
|
||||||
|
continue
|
||||||
|
best_distance_sq = distance_sq
|
||||||
|
best_position = position
|
||||||
|
best_normal = norm(raw_normal)
|
||||||
|
if best_position is None:
|
||||||
|
raise RuntimeError(f"no nearby surface vertices in mesh scene: {mesh}")
|
||||||
|
return {"position_m": best_position, "normal": best_normal,
|
||||||
|
"distance_m": math.sqrt(best_distance_sq)}
|
||||||
|
|
||||||
|
|
||||||
|
def make_passages(layout):
|
||||||
|
q_to_m = layout["core_voxel_size_um"] / (VOXEL_Q * 1_000_000.0)
|
||||||
|
passages = []
|
||||||
|
all_routes = []
|
||||||
|
for passage in layout["passages"]:
|
||||||
|
controls = [(tuple(float(q) * q_to_m for q in p["position_q"]),
|
||||||
|
float(p["radius_q"]) * q_to_m)
|
||||||
|
for p in passage["control_points"]]
|
||||||
|
upper = tuple(float(q) * q_to_m for q in passage["upper_mouth_q"])
|
||||||
|
lower = tuple(float(q) * q_to_m for q in passage["lower_mouth_q"])
|
||||||
|
ui = min(range(len(controls)), key=lambda i: dot(sub(controls[i][0], upper),
|
||||||
|
sub(controls[i][0], upper)))
|
||||||
|
li = min(range(len(controls)), key=lambda i: dot(sub(controls[i][0], lower),
|
||||||
|
sub(controls[i][0], lower)))
|
||||||
|
points = controls[ui:li + 1]
|
||||||
|
if len(points) < 2:
|
||||||
|
raise RuntimeError(f"invalid mouth path: {passage['stable_id']}")
|
||||||
|
upper_offset = math.sqrt(dot(sub(points[0][0], upper), sub(points[0][0], upper)))
|
||||||
|
lower_offset = math.sqrt(dot(sub(points[-1][0], lower), sub(points[-1][0], lower)))
|
||||||
|
length = route_length(points)
|
||||||
|
passage_views = []
|
||||||
|
for endpoint, distance, camera_distance, outward_sign in (
|
||||||
|
("upper-mouth", 0.0, min(length, 7.4), -1.0),
|
||||||
|
("lower-mouth", length, max(0.0, length - 5.5), 1.0)):
|
||||||
|
center, _, tangent, side = route_frame(points, distance)
|
||||||
|
camera_center, _, _, camera_side = route_frame(points, camera_distance)
|
||||||
|
camera = (camera_center[0] + camera_side[0] * 2.5,
|
||||||
|
camera_center[1] + camera_side[1] * 2.5, camera_center[2])
|
||||||
|
outward = mul(tangent, outward_sign)
|
||||||
|
target = (center[0] + outward[0] * 8.0 + side[0] * 2.5,
|
||||||
|
center[1] + outward[1] * 8.0 + side[1] * 2.5, center[2])
|
||||||
|
yaw, pitch = angles(camera, target)
|
||||||
|
passage_views.append({"name": f"{passage['stable_id']}-{endpoint}",
|
||||||
|
"kind": "mouth", "focus": center, "camera": camera, "target": target,
|
||||||
|
"yaw": yaw, "pitch": pitch, "clip": None, "fov": 80.0,
|
||||||
|
"focus_radius": 20.0, "passage": passage["stable_id"],
|
||||||
|
"camera_location": "passage-air"})
|
||||||
|
for fraction in (0.50,):
|
||||||
|
distance = length * fraction
|
||||||
|
center, radius, _, side = route_frame(points, distance)
|
||||||
|
camera = (center[0] + side[0] * 0.75,
|
||||||
|
center[1] + side[1] * 0.75,
|
||||||
|
center[2] - radius + 1.6)
|
||||||
|
target_pos, target_radius = interpolate(
|
||||||
|
points, min(length, distance + 14.0))
|
||||||
|
target = (target_pos[0] + side[0] * 2.35,
|
||||||
|
target_pos[1] + side[1] * 2.35,
|
||||||
|
target_pos[2] - target_radius + 1.8)
|
||||||
|
yaw, pitch = angles(camera, target)
|
||||||
|
pct = int(round(fraction * 100))
|
||||||
|
passage_views.append({"name": f"{passage['stable_id']}-mid-{pct:02d}",
|
||||||
|
"kind": "mid-tunnel", "route_fraction": fraction,
|
||||||
|
"focus": center, "camera": camera, "target": target,
|
||||||
|
"yaw": yaw, "pitch": pitch, "clip": None,
|
||||||
|
"fov": 78.0, "focus_radius": 24.0,
|
||||||
|
"passage": passage["stable_id"]})
|
||||||
|
passages.append({"stable_id": passage["stable_id"], "route_length_m": length,
|
||||||
|
"control_points": len(points), "upper_mouth_anchor_offset_m": upper_offset,
|
||||||
|
"lower_mouth_anchor_offset_m": lower_offset, "views": passage_views})
|
||||||
|
all_routes.append((passage["stable_id"], points))
|
||||||
|
return passages, all_routes
|
||||||
|
|
||||||
|
|
||||||
|
def make_hotspot_views(routes, reach_summary):
|
||||||
|
views = []
|
||||||
|
for series, rows, value_key, location_key, region_key in (
|
||||||
|
("displacement", reach_summary["top_displacement_tiles"],
|
||||||
|
"max_displacement_cm", "displacement_location_metres", "displacement_region"),
|
||||||
|
("normal", reach_summary["top_normal_tiles"],
|
||||||
|
"max_normal_degrees", "normal_location_metres", "normal_region"),
|
||||||
|
):
|
||||||
|
for index, row in enumerate(rows, 1):
|
||||||
|
point = tuple(row[location_key])
|
||||||
|
candidates = [(nearest_route(route, point), name, route)
|
||||||
|
for name, route in routes]
|
||||||
|
(distance, center, tangent, radial, radius, along), nearest_id, route = min(
|
||||||
|
candidates, key=lambda item: item[0][0])
|
||||||
|
if math.sqrt(dot(radial, radial)) < 0.25:
|
||||||
|
horizontal = norm((tangent[0], tangent[1], 0.0))
|
||||||
|
radial = (-horizontal[1], horizontal[0], 0.0)
|
||||||
|
outward = norm(radial)
|
||||||
|
camera = add(point, mul(outward, 20.0))
|
||||||
|
target = point
|
||||||
|
plane = None
|
||||||
|
yaw, pitch = angles(camera, target)
|
||||||
|
views.append({"name": f"top-{series}-{index:02d}", "kind": f"top-{series}",
|
||||||
|
"focus": point, "camera": camera, "target": target,
|
||||||
|
"yaw": yaw, "pitch": pitch, "clip": plane, "fov": 68.0,
|
||||||
|
"focus_radius": 18.0, "strate": row["strate"],
|
||||||
|
"value": row[value_key], "region": row[region_key],
|
||||||
|
"camera_anchor": "nearest-surface-l100-v1", "nearest_passage": nearest_id,
|
||||||
|
"route_distance_m": distance})
|
||||||
|
return views
|
||||||
|
|
||||||
|
|
||||||
|
def decode_png(path):
|
||||||
|
"""Decode 8-bit RGB/RGBA PNG pixels using only the Python standard library."""
|
||||||
|
data = path.read_bytes()
|
||||||
|
if data[:8] != b"\x89PNG\r\n\x1a\n":
|
||||||
|
raise RuntimeError(f"not a PNG: {path}")
|
||||||
|
width = height = bit_depth = color_type = interlace = None
|
||||||
|
compressed = bytearray()
|
||||||
|
offset = 8
|
||||||
|
while offset < len(data):
|
||||||
|
length = struct.unpack_from(">I", data, offset)[0]
|
||||||
|
kind = data[offset + 4:offset + 8]
|
||||||
|
chunk = data[offset + 8:offset + 8 + length]
|
||||||
|
if kind == b"IHDR":
|
||||||
|
width, height, bit_depth, color_type, compression, filtering, interlace = \
|
||||||
|
struct.unpack(">IIBBBBB", chunk)
|
||||||
|
if compression or filtering or interlace:
|
||||||
|
raise RuntimeError(f"unsupported PNG coding in {path}")
|
||||||
|
elif kind == b"IDAT":
|
||||||
|
compressed.extend(chunk)
|
||||||
|
elif kind == b"IEND":
|
||||||
|
break
|
||||||
|
offset += 12 + length
|
||||||
|
if bit_depth != 8 or color_type not in (2, 6):
|
||||||
|
raise RuntimeError(f"expected 8-bit RGB/RGBA PNG: {path}")
|
||||||
|
channels = 3 if color_type == 2 else 4
|
||||||
|
stride = width * channels
|
||||||
|
encoded = zlib.decompress(compressed)
|
||||||
|
pixels = bytearray(height * stride)
|
||||||
|
source_offset = 0
|
||||||
|
for y in range(height):
|
||||||
|
filter_type = encoded[source_offset]
|
||||||
|
source_offset += 1
|
||||||
|
row = bytearray(encoded[source_offset:source_offset + stride])
|
||||||
|
source_offset += stride
|
||||||
|
prior_offset = (y - 1) * stride
|
||||||
|
for index in range(stride):
|
||||||
|
left = row[index - channels] if index >= channels else 0
|
||||||
|
above = pixels[prior_offset + index] if y else 0
|
||||||
|
upper_left = pixels[prior_offset + index - channels] \
|
||||||
|
if y and index >= channels else 0
|
||||||
|
if filter_type == 1:
|
||||||
|
row[index] = (row[index] + left) & 255
|
||||||
|
elif filter_type == 2:
|
||||||
|
row[index] = (row[index] + above) & 255
|
||||||
|
elif filter_type == 3:
|
||||||
|
row[index] = (row[index] + ((left + above) >> 1)) & 255
|
||||||
|
elif filter_type == 4:
|
||||||
|
estimate = left + above - upper_left
|
||||||
|
left_distance = abs(estimate - left)
|
||||||
|
above_distance = abs(estimate - above)
|
||||||
|
upper_left_distance = abs(estimate - upper_left)
|
||||||
|
if left_distance <= above_distance and left_distance <= upper_left_distance:
|
||||||
|
predictor = left
|
||||||
|
elif above_distance <= upper_left_distance:
|
||||||
|
predictor = above
|
||||||
|
else:
|
||||||
|
predictor = upper_left
|
||||||
|
row[index] = (row[index] + predictor) & 255
|
||||||
|
elif filter_type != 0:
|
||||||
|
raise RuntimeError(f"unsupported PNG filter {filter_type}: {path}")
|
||||||
|
start = y * stride
|
||||||
|
pixels[start:start + stride] = row
|
||||||
|
if channels == 3:
|
||||||
|
return width, height, pixels
|
||||||
|
rgb = bytearray(width * height * 3)
|
||||||
|
rgb[0::3] = pixels[0::4]
|
||||||
|
rgb[1::3] = pixels[1::4]
|
||||||
|
rgb[2::3] = pixels[2::4]
|
||||||
|
return width, height, rgb
|
||||||
|
|
||||||
|
|
||||||
|
def image_geometry_metrics(path):
|
||||||
|
width, height, rgb = decode_png(path)
|
||||||
|
color_counts = {}
|
||||||
|
for offset in range(0, len(rgb), 3):
|
||||||
|
color = (rgb[offset], rgb[offset + 1], rgb[offset + 2])
|
||||||
|
color_counts[color] = color_counts.get(color, 0) + 1
|
||||||
|
total = width * height
|
||||||
|
visible = total - max(color_counts.values())
|
||||||
|
return {"width": width, "height": height, "unique_rgb": len(color_counts),
|
||||||
|
"visible_geometry_pixels": visible,
|
||||||
|
"visible_geometry_fraction": visible / total}
|
||||||
|
|
||||||
|
|
||||||
|
def changed_pixel_count(before, after):
|
||||||
|
before_width, before_height, before_rgb = decode_png(before)
|
||||||
|
after_width, after_height, after_rgb = decode_png(after)
|
||||||
|
if (before_width, before_height) != (after_width, after_height):
|
||||||
|
raise RuntimeError(f"capture dimensions differ: {before} / {after}")
|
||||||
|
return sum(before_rgb[offset] != after_rgb[offset]
|
||||||
|
or before_rgb[offset + 1] != after_rgb[offset + 1]
|
||||||
|
or before_rgb[offset + 2] != after_rgb[offset + 2]
|
||||||
|
for offset in range(0, len(before_rgb), 3))
|
||||||
|
|
||||||
|
|
||||||
|
def mesh_triangle_count(mesh):
|
||||||
|
with mesh.open("rb") as stream:
|
||||||
|
magic = stream.read(8)
|
||||||
|
has_materials = magic == b"VFMSH002"
|
||||||
|
if magic not in (b"VFMSH001", b"VFMSH002"):
|
||||||
|
raise RuntimeError(f"unsupported mesh scene header: {mesh}")
|
||||||
|
version, _, tile_count = struct.unpack("<3I", stream.read(12))
|
||||||
|
if version != (2 if has_materials else 1):
|
||||||
|
raise RuntimeError(f"mesh scene version mismatch: {mesh}")
|
||||||
|
stream.seek(40, 1)
|
||||||
|
triangles = 0
|
||||||
|
for _ in range(tile_count):
|
||||||
|
stream.seek(16, 1)
|
||||||
|
_, _, vertex_count, index_count, mask_count = struct.unpack(
|
||||||
|
"<5I", stream.read(20))
|
||||||
|
triangles += index_count // 3
|
||||||
|
stream.seek(vertex_count * 24 + (10 * vertex_count if has_materials else 0)
|
||||||
|
+ index_count * 4 + mask_count, 1)
|
||||||
|
return triangles
|
||||||
|
|
||||||
|
|
||||||
|
def set_route_station(view, distance):
|
||||||
|
points = ROUTE_POINTS[view["passage"]]
|
||||||
|
length = ROUTE_LENGTHS[view["passage"]]
|
||||||
|
center, radius, _, side = route_frame(points, distance)
|
||||||
|
camera = (center[0] + side[0] * 0.75,
|
||||||
|
center[1] + side[1] * 0.75,
|
||||||
|
center[2] - radius + 1.6)
|
||||||
|
target_pos, target_radius = interpolate(points, min(length, distance + 14.0))
|
||||||
|
target = (target_pos[0] + side[0] * 2.35,
|
||||||
|
target_pos[1] + side[1] * 2.35,
|
||||||
|
target_pos[2] - target_radius + 1.8)
|
||||||
|
yaw, pitch = angles(camera, target)
|
||||||
|
if "route_fraction_requested" not in view:
|
||||||
|
view["route_fraction_requested"] = view["route_fraction"]
|
||||||
|
view["route_requested_focus"] = view["focus"]
|
||||||
|
view["route_fraction"] = distance / length
|
||||||
|
view["route_station_adjustment_m"] = distance - length * view["route_fraction_requested"]
|
||||||
|
view["focus"] = center
|
||||||
|
view["camera"] = camera
|
||||||
|
view["target"] = target
|
||||||
|
view["yaw"], view["pitch"] = yaw, pitch
|
||||||
|
view["clip"] = None
|
||||||
|
|
||||||
|
|
||||||
|
def render_command(view, mesh, image, camera, yaw, pitch):
|
||||||
|
args = [RENDERER, mesh, image,
|
||||||
|
"--size=1600x900", f"--fov-deg={view['fov']:.1f}",
|
||||||
|
f"--camera-m={fmt(camera)}", f"--yaw-deg={yaw:.6f}",
|
||||||
|
f"--pitch-deg={pitch:.6f}", "--shade=readable", "--headlamp=on",
|
||||||
|
"--height-tint=on", "--tile-edges=off", "--tile-skirts=off",
|
||||||
|
f"--height-center-m={view['focus'][2]:.6f}", "--height-range-m=10",
|
||||||
|
f"--focus-region={fmt((*view['focus'], view['focus_radius']))}"]
|
||||||
|
if view["clip"] is not None:
|
||||||
|
args.append(f"--clip-plane={fmt(view['clip'])}")
|
||||||
|
return args
|
||||||
|
|
||||||
|
|
||||||
|
def export_scene(view, binary, mesh, stem):
|
||||||
|
camera = view["camera"]
|
||||||
|
yaw, pitch = angles(camera, view["target"])
|
||||||
|
export_args = [binary, DATA, "--world-assembler-prototype", "--stream-benchmark",
|
||||||
|
"--stream-pipeline", "--workers=8", "--force-generate", "--clip-radius=3",
|
||||||
|
"--max-clip-level=0", "--max-level=0", "--render-distance-chunks=0",
|
||||||
|
f"--camera={fmt((*camera, yaw, pitch))}", f"--mesh-export={mesh}"]
|
||||||
|
row = run(f"{stem}-export", export_args, headless=True)
|
||||||
|
row["scene_bytes"] = mesh.stat().st_size if mesh.exists() else 0
|
||||||
|
row["scene_triangles"] = mesh_triangle_count(mesh) if row["scene_bytes"] else 0
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def export_and_render(view, label, binary):
|
||||||
|
safe = view["name"].replace("/", "-")
|
||||||
|
stem = f"{safe}-{label}"
|
||||||
|
mesh = OUT / f"{stem}.vfmesh"
|
||||||
|
image = IMAGE_DIR / f"{safe}-{label}.png"
|
||||||
|
camera = view["camera"]
|
||||||
|
yaw, pitch = angles(camera, view["target"])
|
||||||
|
export_row = export_scene(view, binary, mesh, stem)
|
||||||
|
if export_row["scene_bytes"] <= 0:
|
||||||
|
raise RuntimeError(f"empty mesh scene: {mesh}")
|
||||||
|
if export_row["scene_triangles"] == 0:
|
||||||
|
if label != "before-l105" or view.get("kind") != "mid-tunnel":
|
||||||
|
raise RuntimeError(f"route station exports no surface triangles: {mesh}")
|
||||||
|
points = ROUTE_POINTS[view["passage"]]
|
||||||
|
length = ROUTE_LENGTHS[view["passage"]]
|
||||||
|
requested_fraction = view["route_fraction"]
|
||||||
|
requested_distance = length * requested_fraction
|
||||||
|
found_surface = False
|
||||||
|
for offset in (5.0, -5.0, 10.0, -10.0, 20.0, -20.0, 35.0, -35.0,
|
||||||
|
50.0, -50.0, 75.0, -75.0, 100.0, -100.0, 150.0,
|
||||||
|
-150.0, 200.0, -200.0):
|
||||||
|
distance = max(0.0, min(length, requested_distance + offset))
|
||||||
|
if distance == requested_distance:
|
||||||
|
continue
|
||||||
|
set_route_station(view, distance)
|
||||||
|
adjusted_stem = f"{stem}-station-{view['route_fraction']:.5f}"
|
||||||
|
export_row = export_scene(view, binary, mesh, adjusted_stem)
|
||||||
|
if export_row["scene_triangles"] > 0:
|
||||||
|
view["camera_anchor"] = "route-station-shifted-to-nearby-geometry"
|
||||||
|
found_surface = True
|
||||||
|
break
|
||||||
|
if not found_surface:
|
||||||
|
raise RuntimeError(f"no geometry along nearby route stations: {mesh}")
|
||||||
|
camera = view["camera"]
|
||||||
|
yaw, pitch = angles(camera, view["target"])
|
||||||
|
if label == "before-l105" and view.get("camera_anchor") == "nearest-surface-l100-v1":
|
||||||
|
anchor = nearest_surface_anchor(mesh, view["focus"], tile_search_margin_m=256.0)
|
||||||
|
camera = add(anchor["position_m"], mul(anchor["normal"], 8.0))
|
||||||
|
view["camera"] = camera
|
||||||
|
view["target"] = anchor["position_m"]
|
||||||
|
view["yaw"], view["pitch"] = angles(camera, view["target"])
|
||||||
|
view["clip"] = None
|
||||||
|
view["surface_anchor"] = anchor
|
||||||
|
yaw, pitch = view["yaw"], view["pitch"]
|
||||||
|
render_args = render_command(view, mesh, image, camera, yaw, pitch)
|
||||||
|
try:
|
||||||
|
render_row = run(f"{stem}-render", render_args, headless=True)
|
||||||
|
except RuntimeError:
|
||||||
|
if view["clip"] is None:
|
||||||
|
raise
|
||||||
|
retry_args = [arg for arg in render_args
|
||||||
|
if not (isinstance(arg, str) and arg.startswith("--clip-plane="))]
|
||||||
|
render_row = run(f"{stem}-render-no-clip-retry", retry_args, headless=True)
|
||||||
|
render_row["clip_fallback"] = True
|
||||||
|
render_row["image"] = str(image.relative_to(ROOT))
|
||||||
|
render_row["image_bytes"] = image.stat().st_size if image.exists() else 0
|
||||||
|
if render_row["image_bytes"] <= 0:
|
||||||
|
raise RuntimeError(f"empty image: {image}")
|
||||||
|
geometry = image_geometry_metrics(image)
|
||||||
|
if label == "before-l105" and (
|
||||||
|
geometry["visible_geometry_pixels"] < 3000 or geometry["unique_rgb"] < 2):
|
||||||
|
# At a few deep route stations the default forward view looks into empty
|
||||||
|
# sky. Keep the camera on the route station and aim at the nearest L98
|
||||||
|
# surface in the exported scene; retain that matched target for L101.
|
||||||
|
anchor = nearest_surface_anchor(mesh, view["focus"], tile_search_margin_m=256.0)
|
||||||
|
if anchor["distance_m"] > 60.0:
|
||||||
|
raise RuntimeError(f"no nearby surface for fallback camera: {image} {anchor}")
|
||||||
|
camera = view["camera"]
|
||||||
|
target = anchor["position_m"]
|
||||||
|
yaw, pitch = angles(camera, target)
|
||||||
|
view["target"] = target
|
||||||
|
view["yaw"], view["pitch"] = yaw, pitch
|
||||||
|
view["clip"] = None
|
||||||
|
view["camera_anchor"] = "route-station-nearest-surface-target"
|
||||||
|
view["surface_anchor"] = anchor
|
||||||
|
retry_args = render_command(view, mesh, image, camera, yaw, pitch)
|
||||||
|
render_row = run(f"{stem}-render-surface-fallback", retry_args, headless=True)
|
||||||
|
render_row["image"] = str(image.relative_to(ROOT))
|
||||||
|
render_row["image_bytes"] = image.stat().st_size if image.exists() else 0
|
||||||
|
geometry = image_geometry_metrics(image)
|
||||||
|
render_row["geometry"] = geometry
|
||||||
|
if geometry["visible_geometry_pixels"] < 3000 or geometry["unique_rgb"] < 2:
|
||||||
|
raise RuntimeError(f"camera sees sky only or too little geometry: {image} {geometry}")
|
||||||
|
mesh.unlink()
|
||||||
|
return export_row, render_row
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
IMAGE_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
for path in (*BINARIES.values(), RENDERER, DATA, LAYOUT_PATH):
|
||||||
|
if not path.is_file():
|
||||||
|
raise SystemExit(f"missing input: {path}")
|
||||||
|
layout = json.loads(LAYOUT_PATH.read_text(encoding="utf-8"))
|
||||||
|
passages, routes = make_passages(layout)
|
||||||
|
ROUTE_POINTS.clear()
|
||||||
|
ROUTE_LENGTHS.clear()
|
||||||
|
for stable_id, points in routes:
|
||||||
|
ROUTE_POINTS[stable_id] = points
|
||||||
|
ROUTE_LENGTHS[stable_id] = route_length(points)
|
||||||
|
views = [view for passage in passages for view in passage["views"]]
|
||||||
|
partial_path = OUT / "runs-partial.json"
|
||||||
|
existing_records = {}
|
||||||
|
if partial_path.is_file():
|
||||||
|
for item in json.loads(partial_path.read_text(encoding="utf-8")):
|
||||||
|
existing_records[item["view"]["name"]] = item
|
||||||
|
records = []
|
||||||
|
for view in views:
|
||||||
|
prior = existing_records.get(view["name"])
|
||||||
|
camera_matches = (prior is not None and
|
||||||
|
prior["view"].get("camera_anchor") == view.get("camera_anchor"))
|
||||||
|
if camera_matches:
|
||||||
|
for key in ("camera", "target", "yaw", "pitch", "clip", "surface_anchor"):
|
||||||
|
if key in prior["view"]:
|
||||||
|
view[key] = prior["view"][key]
|
||||||
|
if camera_matches and all(version in prior and
|
||||||
|
(ROOT / prior[version]["render"]["image"]).is_file()
|
||||||
|
for version in BINARIES):
|
||||||
|
records.append(prior)
|
||||||
|
print(f"[L106CaptureSkip] complete_pair={view['name']}", flush=True)
|
||||||
|
continue
|
||||||
|
pair = {"view": view}
|
||||||
|
for label, binary in BINARIES.items():
|
||||||
|
export_row, render_row = export_and_render(view, label, binary)
|
||||||
|
pair[label] = {"export": export_row, "render": render_row}
|
||||||
|
(OUT / "runs-partial.json").write_text(json.dumps(records + [pair], indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
pair["comparison"] = {"changed_pixels": changed_pixel_count(
|
||||||
|
ROOT / pair["before-l105"]["render"]["image"],
|
||||||
|
ROOT / pair["after-l106"]["render"]["image"])}
|
||||||
|
(OUT / "runs-partial.json").write_text(json.dumps(records + [pair], indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
records.append(pair)
|
||||||
|
summary = {"source_layout_identity": layout["stable_identity"],
|
||||||
|
"view_count": len(views), "paired_png_count": len(views) * 2,
|
||||||
|
"passages": passages, "views": records}
|
||||||
|
(OUT / "summary.json").write_text(json.dumps(summary, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
print("[L106CaptureSummary] " + json.dumps({
|
||||||
|
"views": len(views), "paired_png_count": len(views) * 2,
|
||||||
|
"images": [row[version]["render"]["image"] for row in records
|
||||||
|
for version in BINARIES]}, sort_keys=True), flush=True)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,117 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Compare L106 exact tile hashes with L105 across all strata, compilers and workers."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
OUT = ROOT / "Saved/Linux/l106/hashes"
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
OLD = ROOT / "Saved/Linux/l105/hashes"
|
||||||
|
BINARIES = {
|
||||||
|
"clang": ROOT / "Saved/Linux/l106/bin/VoxelForgeCoreHeadless-l106-clang",
|
||||||
|
"gcc": ROOT / "Saved/Linux/l106/bin/VoxelForgeCoreHeadless-l106-gcc",
|
||||||
|
}
|
||||||
|
CAMERAS = {
|
||||||
|
"sw": ((8.0, 16.0, -372.2626953125), 2850),
|
||||||
|
"fi": ((25.8369140625, 3.19921875, -737.0), 2249),
|
||||||
|
"tn": ((19.166015625, -14.2001953125, -753.3740234375), 2233),
|
||||||
|
"vs": ((-20.0625, -5.068359375, -900.353515625), 2329),
|
||||||
|
}
|
||||||
|
TILE = re.compile(
|
||||||
|
r"^\[PipelineTileHash\] lod=(-?\d+) x=(-?\d+) y=(-?\d+) z=(-?\d+) "
|
||||||
|
r"density=(0x[0-9a-f]+) mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) "
|
||||||
|
r"topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
WHOLE = re.compile(
|
||||||
|
r"^\[PipelineWholeViewHash\] tiles=(\d+) density=(0x[0-9a-f]+) "
|
||||||
|
r"mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) topology=(0x[0-9a-f]+)$", re.M)
|
||||||
|
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M)
|
||||||
|
|
||||||
|
|
||||||
|
def parse(path: Path) -> tuple[dict, tuple[str, ...], str]:
|
||||||
|
content = path.read_text(encoding="utf-8", errors="replace")
|
||||||
|
tiles = {(int(m.group(1)), int(m.group(2)), int(m.group(3)), int(m.group(4))):
|
||||||
|
m.group(5, 6, 7, 8) for m in TILE.finditer(content)}
|
||||||
|
whole, stamp = WHOLE.search(content), STAMP.search(content)
|
||||||
|
if whole is None or stamp is None or len(tiles) != int(whole.group(1)):
|
||||||
|
raise RuntimeError(f"incomplete hash output: {path}")
|
||||||
|
return tiles, whole.groups(), stamp.group(1)
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, compiler: str, workers: int) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = OUT / f"{stratum}-{compiler}-w{workers}.log"
|
||||||
|
camera, expected_tiles = CAMERAS[stratum]
|
||||||
|
camera_arg = ",".join(f"{v:.12f}" for v in camera) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--", "timeout", "1740s",
|
||||||
|
str(BINARIES[compiler]), str(DATA), "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", f"--workers={workers}",
|
||||||
|
"--clip-radius=3", "--max-clip-level=8", "--max-level=8",
|
||||||
|
f"--camera={camera_arg}", "--stream-hash-tiles",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
if process.returncode:
|
||||||
|
raise RuntimeError(f"hash process exited {process.returncode}: {log}")
|
||||||
|
tiles, whole, stamp = parse(log)
|
||||||
|
if len(tiles) != expected_tiles:
|
||||||
|
raise RuntimeError(f"expected {expected_tiles} tiles, got {len(tiles)}: {log}")
|
||||||
|
old_log = OLD / f"{stratum}-{compiler}-w{workers}.log"
|
||||||
|
old_tiles, old_whole, _old_stamp = parse(old_log)
|
||||||
|
clang_reference, clang_whole, _ = parse(OLD / f"{stratum}-clang-w1.log")
|
||||||
|
|
||||||
|
def difference(reference: dict, candidate: dict) -> tuple[int, int]:
|
||||||
|
keys = set(reference) | set(candidate)
|
||||||
|
mismatches = sum(reference.get(key) != candidate.get(key) for key in keys)
|
||||||
|
return mismatches, len(keys)
|
||||||
|
|
||||||
|
old_mismatches, old_keys = difference(old_tiles, tiles)
|
||||||
|
cross_mismatches, cross_keys = difference(clang_reference, tiles)
|
||||||
|
row = {
|
||||||
|
"stratum": stratum, "compiler": compiler, "workers": workers,
|
||||||
|
"source_stamp": stamp, "tiles": len(tiles),
|
||||||
|
"l105_same_cell_tile_mismatches": old_mismatches,
|
||||||
|
"l105_same_cell_key_count": old_keys,
|
||||||
|
"l105_same_cell_whole_hash_match": whole == old_whole,
|
||||||
|
"clang_w1_tile_mismatches": cross_mismatches,
|
||||||
|
"clang_w1_key_count": cross_keys,
|
||||||
|
"clang_w1_whole_hash_match": whole == clang_whole,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L106HashRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
row["tile_hashes"] = {"|".join(map(str, key)): value for key, value in tiles.items()}
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
for path in (*BINARIES.values(), DATA):
|
||||||
|
if not path.is_file():
|
||||||
|
raise SystemExit(f"missing input: {path}")
|
||||||
|
rows = []
|
||||||
|
for stratum in CAMERAS:
|
||||||
|
for compiler in BINARIES:
|
||||||
|
for workers in (1, 8, 12):
|
||||||
|
rows.append(run(stratum, compiler, workers))
|
||||||
|
(OUT / "runs-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
failures = [row for row in rows if row["l105_same_cell_tile_mismatches"]
|
||||||
|
or not row["l105_same_cell_whole_hash_match"]
|
||||||
|
or row["clang_w1_tile_mismatches"]
|
||||||
|
or not row["clang_w1_whole_hash_match"]]
|
||||||
|
summary = {"runs": [{key: value for key, value in row.items() if key != "tile_hashes"}
|
||||||
|
for row in rows], "failed_runs": len(failures)}
|
||||||
|
(OUT / "summary.json").write_text(json.dumps(summary, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 1 if failures else 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,118 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""L80-protocol L105/L106 near and whole-view readiness pairs."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import statistics
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
OUT = ROOT / "Saved/Linux/l106/readiness-paired-extra"
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
BINARIES = {
|
||||||
|
"before": ROOT / "Saved/Linux/l105/bin/VoxelForgeCoreHeadless-l105-clang",
|
||||||
|
"after": ROOT / "Saved/Linux/l106/bin/VoxelForgeCoreHeadless-l106-clang",
|
||||||
|
}
|
||||||
|
STRATA = {
|
||||||
|
"sw": {"camera": (8.0, 16.0, -372.2626953125), "near": 1408, "whole": 2850},
|
||||||
|
"fi": {"camera": (25.8369140625, 3.19921875, -737.0), "near": 1216, "whole": 2249},
|
||||||
|
"tn": {"camera": (19.166015625, -14.2001953125, -753.3740234375), "near": 1344, "whole": 2233},
|
||||||
|
"vs": {"camera": (-20.0625, -5.068359375, -900.353515625), "near": 1344, "whole": 2329},
|
||||||
|
}
|
||||||
|
NEAR = re.compile(r"^\[WorldAssemblerNearLODReady\] lods=0-2 tiles=(\d+)/(\d+) cold_ready_s=([0-9.]+)$", re.M)
|
||||||
|
PIPELINE = re.compile(r"^\[StreamPipeline\] (.*)$", re.M)
|
||||||
|
CONFIG = re.compile(r"^\[HeadlessConfig\] (.*)$", re.M)
|
||||||
|
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M)
|
||||||
|
FIELDS = re.compile(r"([a-zA-Z0-9_]+)=([^ ]+)")
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, workers: int, build: str, phase: str, replicate: int) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
suffix = f"-r{replicate}" if phase == "measured" else ""
|
||||||
|
log = OUT / f"{stratum}-w{workers}-{build}-{phase}{suffix}.log"
|
||||||
|
camera = ",".join(f"{value:.12f}" for value in STRATA[stratum]["camera"]) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--", "timeout", "1740s",
|
||||||
|
str(BINARIES[build]), str(DATA), "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", f"--workers={workers}",
|
||||||
|
"--clip-radius=3", "--max-clip-level=8", "--max-level=8",
|
||||||
|
f"--camera={camera}", "--profile-lod-work", "--stream-scheduler-policy=3",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
content = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
near, pipeline, config, stamp = (NEAR.search(content), PIPELINE.search(content),
|
||||||
|
CONFIG.search(content), STAMP.search(content))
|
||||||
|
if process.returncode or not all((near, pipeline, config, stamp)):
|
||||||
|
raise RuntimeError(f"readiness run failed or missing marker: {log}; exit={process.returncode}")
|
||||||
|
expected = STRATA[stratum]
|
||||||
|
config_fields = dict(FIELDS.findall(config.group(1)))
|
||||||
|
pipeline_fields = dict(FIELDS.findall(pipeline.group(1)))
|
||||||
|
if int(config_fields["workers"]) != workers:
|
||||||
|
raise RuntimeError(f"worker mismatch: {log}")
|
||||||
|
if (int(near.group(1)), int(near.group(2))) != (expected["near"], expected["near"]):
|
||||||
|
raise RuntimeError(f"incomplete LOD0-2 set: {log}: {near.group(1)}/{near.group(2)}")
|
||||||
|
whole_done, whole_total = map(int, pipeline_fields["whole_tiles"].split("/"))
|
||||||
|
if (whole_done, whole_total) != (expected["whole"], expected["whole"]):
|
||||||
|
raise RuntimeError(f"incomplete whole view: {log}: {whole_done}/{whole_total}")
|
||||||
|
row = {
|
||||||
|
"stratum": stratum, "workers": workers, "build": build,
|
||||||
|
"phase": phase, "replicate": replicate, "source_stamp": stamp.group(1),
|
||||||
|
"near_tiles": expected["near"], "whole_tiles": whole_done,
|
||||||
|
"lod02_ready_s": float(near.group(3)),
|
||||||
|
"whole_view_ready_s": float(pipeline_fields["whole_view_ready"].removesuffix("s")),
|
||||||
|
"shape_s": float(pipeline_fields["shape"].removesuffix("s")),
|
||||||
|
"far_shape_s": float(pipeline_fields["far_shape"].removesuffix("s")),
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L106ReadinessRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
for path in (*BINARIES.values(), DATA):
|
||||||
|
if not path.is_file():
|
||||||
|
raise SystemExit(f"missing input: {path}")
|
||||||
|
cells = [(stratum, workers) for stratum in ("vs", "tn", "fi", "sw")
|
||||||
|
for workers in (12, 8)]
|
||||||
|
for stratum, workers in cells:
|
||||||
|
for build in ("before", "after"):
|
||||||
|
run(stratum, workers, build, "warmup", 0)
|
||||||
|
orders = (("before", "after"), ("after", "before"), ("before", "after"))
|
||||||
|
rows = []
|
||||||
|
for stratum, workers in cells:
|
||||||
|
for replicate, order in enumerate(orders, 1):
|
||||||
|
for build in order:
|
||||||
|
rows.append(run(stratum, workers, build, "measured", replicate))
|
||||||
|
(OUT / "runs-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
summary = []
|
||||||
|
for stratum in STRATA:
|
||||||
|
for workers in (8, 12):
|
||||||
|
group = [row for row in rows if row["stratum"] == stratum
|
||||||
|
and row["workers"] == workers]
|
||||||
|
item = {"stratum": stratum, "workers": workers, "measured_runs_per_build": 3}
|
||||||
|
for build in ("before", "after"):
|
||||||
|
subset = [row for row in group if row["build"] == build]
|
||||||
|
for metric in ("lod02_ready_s", "whole_view_ready_s"):
|
||||||
|
values = [row[metric] for row in subset]
|
||||||
|
item[f"{build}_{metric}"] = values
|
||||||
|
item[f"{build}_{metric}_median"] = statistics.median(values)
|
||||||
|
for metric in ("lod02_ready_s", "whole_view_ready_s"):
|
||||||
|
item[f"{metric}_delta_pct"] = 100.0 * (
|
||||||
|
item[f"after_{metric}_median"] / item[f"before_{metric}_median"] - 1.0)
|
||||||
|
summary.append(item)
|
||||||
|
(OUT / "summary.json").write_text(
|
||||||
|
json.dumps({"runs": rows, "summary": summary}, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,117 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""L80-protocol L105/L106 near and whole-view readiness pairs."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import statistics
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
OUT = ROOT / "Saved/Linux/l106/readiness-paired-fi-w12-extra"
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
BINARIES = {
|
||||||
|
"before": ROOT / "Saved/Linux/l105/bin/VoxelForgeCoreHeadless-l105-clang",
|
||||||
|
"after": ROOT / "Saved/Linux/l106/bin/VoxelForgeCoreHeadless-l106-clang",
|
||||||
|
}
|
||||||
|
STRATA = {
|
||||||
|
"sw": {"camera": (8.0, 16.0, -372.2626953125), "near": 1408, "whole": 2850},
|
||||||
|
"fi": {"camera": (25.8369140625, 3.19921875, -737.0), "near": 1216, "whole": 2249},
|
||||||
|
"tn": {"camera": (19.166015625, -14.2001953125, -753.3740234375), "near": 1344, "whole": 2233},
|
||||||
|
"vs": {"camera": (-20.0625, -5.068359375, -900.353515625), "near": 1344, "whole": 2329},
|
||||||
|
}
|
||||||
|
NEAR = re.compile(r"^\[WorldAssemblerNearLODReady\] lods=0-2 tiles=(\d+)/(\d+) cold_ready_s=([0-9.]+)$", re.M)
|
||||||
|
PIPELINE = re.compile(r"^\[StreamPipeline\] (.*)$", re.M)
|
||||||
|
CONFIG = re.compile(r"^\[HeadlessConfig\] (.*)$", re.M)
|
||||||
|
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M)
|
||||||
|
FIELDS = re.compile(r"([a-zA-Z0-9_]+)=([^ ]+)")
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, workers: int, build: str, phase: str, replicate: int) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
suffix = f"-r{replicate}" if phase == "measured" else ""
|
||||||
|
log = OUT / f"{stratum}-w{workers}-{build}-{phase}{suffix}.log"
|
||||||
|
camera = ",".join(f"{value:.12f}" for value in STRATA[stratum]["camera"]) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--", "timeout", "1740s",
|
||||||
|
str(BINARIES[build]), str(DATA), "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", f"--workers={workers}",
|
||||||
|
"--clip-radius=3", "--max-clip-level=8", "--max-level=8",
|
||||||
|
f"--camera={camera}", "--profile-lod-work", "--stream-scheduler-policy=3",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
content = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
near, pipeline, config, stamp = (NEAR.search(content), PIPELINE.search(content),
|
||||||
|
CONFIG.search(content), STAMP.search(content))
|
||||||
|
if process.returncode or not all((near, pipeline, config, stamp)):
|
||||||
|
raise RuntimeError(f"readiness run failed or missing marker: {log}; exit={process.returncode}")
|
||||||
|
expected = STRATA[stratum]
|
||||||
|
config_fields = dict(FIELDS.findall(config.group(1)))
|
||||||
|
pipeline_fields = dict(FIELDS.findall(pipeline.group(1)))
|
||||||
|
if int(config_fields["workers"]) != workers:
|
||||||
|
raise RuntimeError(f"worker mismatch: {log}")
|
||||||
|
if (int(near.group(1)), int(near.group(2))) != (expected["near"], expected["near"]):
|
||||||
|
raise RuntimeError(f"incomplete LOD0-2 set: {log}: {near.group(1)}/{near.group(2)}")
|
||||||
|
whole_done, whole_total = map(int, pipeline_fields["whole_tiles"].split("/"))
|
||||||
|
if (whole_done, whole_total) != (expected["whole"], expected["whole"]):
|
||||||
|
raise RuntimeError(f"incomplete whole view: {log}: {whole_done}/{whole_total}")
|
||||||
|
row = {
|
||||||
|
"stratum": stratum, "workers": workers, "build": build,
|
||||||
|
"phase": phase, "replicate": replicate, "source_stamp": stamp.group(1),
|
||||||
|
"near_tiles": expected["near"], "whole_tiles": whole_done,
|
||||||
|
"lod02_ready_s": float(near.group(3)),
|
||||||
|
"whole_view_ready_s": float(pipeline_fields["whole_view_ready"].removesuffix("s")),
|
||||||
|
"shape_s": float(pipeline_fields["shape"].removesuffix("s")),
|
||||||
|
"far_shape_s": float(pipeline_fields["far_shape"].removesuffix("s")),
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L106ReadinessRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
for path in (*BINARIES.values(), DATA):
|
||||||
|
if not path.is_file():
|
||||||
|
raise SystemExit(f"missing input: {path}")
|
||||||
|
cells = [("fi", 12)]
|
||||||
|
for stratum, workers in cells:
|
||||||
|
for build in ("before", "after"):
|
||||||
|
run(stratum, workers, build, "warmup", 0)
|
||||||
|
orders = (("before", "after"), ("after", "before"), ("before", "after"))
|
||||||
|
rows = []
|
||||||
|
for stratum, workers in cells:
|
||||||
|
for replicate, order in enumerate(orders, 1):
|
||||||
|
for build in order:
|
||||||
|
rows.append(run(stratum, workers, build, "measured", replicate))
|
||||||
|
(OUT / "runs-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
summary = []
|
||||||
|
for stratum in ("fi",):
|
||||||
|
for workers in (12,):
|
||||||
|
group = [row for row in rows if row["stratum"] == stratum
|
||||||
|
and row["workers"] == workers]
|
||||||
|
item = {"stratum": stratum, "workers": workers, "measured_runs_per_build": 3}
|
||||||
|
for build in ("before", "after"):
|
||||||
|
subset = [row for row in group if row["build"] == build]
|
||||||
|
for metric in ("lod02_ready_s", "whole_view_ready_s"):
|
||||||
|
values = [row[metric] for row in subset]
|
||||||
|
item[f"{build}_{metric}"] = values
|
||||||
|
item[f"{build}_{metric}_median"] = statistics.median(values)
|
||||||
|
for metric in ("lod02_ready_s", "whole_view_ready_s"):
|
||||||
|
item[f"{metric}_delta_pct"] = 100.0 * (
|
||||||
|
item[f"after_{metric}_median"] / item[f"before_{metric}_median"] - 1.0)
|
||||||
|
summary.append(item)
|
||||||
|
(OUT / "summary.json").write_text(
|
||||||
|
json.dumps({"runs": rows, "summary": summary}, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,118 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""L80-protocol L105/L106 near and whole-view readiness pairs."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import statistics
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
OUT = ROOT / "Saved/Linux/l106/readiness-paired"
|
||||||
|
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||||
|
BINARIES = {
|
||||||
|
"before": ROOT / "Saved/Linux/l105/bin/VoxelForgeCoreHeadless-l105-clang",
|
||||||
|
"after": ROOT / "Saved/Linux/l106/bin/VoxelForgeCoreHeadless-l106-clang",
|
||||||
|
}
|
||||||
|
STRATA = {
|
||||||
|
"sw": {"camera": (8.0, 16.0, -372.2626953125), "near": 1408, "whole": 2850},
|
||||||
|
"fi": {"camera": (25.8369140625, 3.19921875, -737.0), "near": 1216, "whole": 2249},
|
||||||
|
"tn": {"camera": (19.166015625, -14.2001953125, -753.3740234375), "near": 1344, "whole": 2233},
|
||||||
|
"vs": {"camera": (-20.0625, -5.068359375, -900.353515625), "near": 1344, "whole": 2329},
|
||||||
|
}
|
||||||
|
NEAR = re.compile(r"^\[WorldAssemblerNearLODReady\] lods=0-2 tiles=(\d+)/(\d+) cold_ready_s=([0-9.]+)$", re.M)
|
||||||
|
PIPELINE = re.compile(r"^\[StreamPipeline\] (.*)$", re.M)
|
||||||
|
CONFIG = re.compile(r"^\[HeadlessConfig\] (.*)$", re.M)
|
||||||
|
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M)
|
||||||
|
FIELDS = re.compile(r"([a-zA-Z0-9_]+)=([^ ]+)")
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, workers: int, build: str, phase: str, replicate: int) -> dict:
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
suffix = f"-r{replicate}" if phase == "measured" else ""
|
||||||
|
log = OUT / f"{stratum}-w{workers}-{build}-{phase}{suffix}.log"
|
||||||
|
camera = ",".join(f"{value:.12f}" for value in STRATA[stratum]["camera"]) + ",0,0"
|
||||||
|
command = [
|
||||||
|
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--", "timeout", "1740s",
|
||||||
|
str(BINARIES[build]), str(DATA), "--world-assembler-prototype",
|
||||||
|
"--stream-benchmark", "--stream-pipeline", f"--workers={workers}",
|
||||||
|
"--clip-radius=3", "--max-clip-level=8", "--max-level=8",
|
||||||
|
f"--camera={camera}", "--profile-lod-work", "--stream-scheduler-policy=3",
|
||||||
|
]
|
||||||
|
started = time.monotonic()
|
||||||
|
with log.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||||
|
content = log.read_text(encoding="utf-8", errors="replace")
|
||||||
|
near, pipeline, config, stamp = (NEAR.search(content), PIPELINE.search(content),
|
||||||
|
CONFIG.search(content), STAMP.search(content))
|
||||||
|
if process.returncode or not all((near, pipeline, config, stamp)):
|
||||||
|
raise RuntimeError(f"readiness run failed or missing marker: {log}; exit={process.returncode}")
|
||||||
|
expected = STRATA[stratum]
|
||||||
|
config_fields = dict(FIELDS.findall(config.group(1)))
|
||||||
|
pipeline_fields = dict(FIELDS.findall(pipeline.group(1)))
|
||||||
|
if int(config_fields["workers"]) != workers:
|
||||||
|
raise RuntimeError(f"worker mismatch: {log}")
|
||||||
|
if (int(near.group(1)), int(near.group(2))) != (expected["near"], expected["near"]):
|
||||||
|
raise RuntimeError(f"incomplete LOD0-2 set: {log}: {near.group(1)}/{near.group(2)}")
|
||||||
|
whole_done, whole_total = map(int, pipeline_fields["whole_tiles"].split("/"))
|
||||||
|
if (whole_done, whole_total) != (expected["whole"], expected["whole"]):
|
||||||
|
raise RuntimeError(f"incomplete whole view: {log}: {whole_done}/{whole_total}")
|
||||||
|
row = {
|
||||||
|
"stratum": stratum, "workers": workers, "build": build,
|
||||||
|
"phase": phase, "replicate": replicate, "source_stamp": stamp.group(1),
|
||||||
|
"near_tiles": expected["near"], "whole_tiles": whole_done,
|
||||||
|
"lod02_ready_s": float(near.group(3)),
|
||||||
|
"whole_view_ready_s": float(pipeline_fields["whole_view_ready"].removesuffix("s")),
|
||||||
|
"shape_s": float(pipeline_fields["shape"].removesuffix("s")),
|
||||||
|
"far_shape_s": float(pipeline_fields["far_shape"].removesuffix("s")),
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"log": str(log.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
print("[L106ReadinessRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
for path in (*BINARIES.values(), DATA):
|
||||||
|
if not path.is_file():
|
||||||
|
raise SystemExit(f"missing input: {path}")
|
||||||
|
cells = [(stratum, workers) for stratum in ("vs", "tn", "fi", "sw")
|
||||||
|
for workers in (12, 8)]
|
||||||
|
for stratum, workers in cells:
|
||||||
|
for build in ("before", "after"):
|
||||||
|
run(stratum, workers, build, "warmup", 0)
|
||||||
|
orders = (("before", "after"), ("after", "before"), ("before", "after"))
|
||||||
|
rows = []
|
||||||
|
for stratum, workers in cells:
|
||||||
|
for replicate, order in enumerate(orders, 1):
|
||||||
|
for build in order:
|
||||||
|
rows.append(run(stratum, workers, build, "measured", replicate))
|
||||||
|
(OUT / "runs-partial.json").write_text(
|
||||||
|
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||||
|
summary = []
|
||||||
|
for stratum in STRATA:
|
||||||
|
for workers in (8, 12):
|
||||||
|
group = [row for row in rows if row["stratum"] == stratum
|
||||||
|
and row["workers"] == workers]
|
||||||
|
item = {"stratum": stratum, "workers": workers, "measured_runs_per_build": 3}
|
||||||
|
for build in ("before", "after"):
|
||||||
|
subset = [row for row in group if row["build"] == build]
|
||||||
|
for metric in ("lod02_ready_s", "whole_view_ready_s"):
|
||||||
|
values = [row[metric] for row in subset]
|
||||||
|
item[f"{build}_{metric}"] = values
|
||||||
|
item[f"{build}_{metric}_median"] = statistics.median(values)
|
||||||
|
for metric in ("lod02_ready_s", "whole_view_ready_s"):
|
||||||
|
item[f"{metric}_delta_pct"] = 100.0 * (
|
||||||
|
item[f"after_{metric}_median"] / item[f"before_{metric}_median"] - 1.0)
|
||||||
|
summary.append(item)
|
||||||
|
(OUT / "summary.json").write_text(
|
||||||
|
json.dumps({"runs": rows, "summary": summary}, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,484 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Run one pinned null-RHI SW/FI arrival for Linux 106."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from collections import Counter
|
||||||
|
from datetime import datetime
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[4]
|
||||||
|
HOST = Path("/home/jahni/VFHost")
|
||||||
|
HOST_PLUGIN_SAVED = HOST / "Plugins/VoxelForge/Saved/Linux/l106"
|
||||||
|
HOST_SAVED = HOST / "Saved/Linux/l106"
|
||||||
|
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
|
||||||
|
OUT = ROOT / "Saved/Linux/l106/unreal-arrival"
|
||||||
|
SPAWNS_M = {
|
||||||
|
"FI": (25.8369140625, 3.19921875, -737.0),
|
||||||
|
"SW": (25.8369140625, 3.19921875, -200.0),
|
||||||
|
}
|
||||||
|
MOVE_START_S = 20.0
|
||||||
|
MOVE_DURATION_S = 12.0
|
||||||
|
EXIT_S = MOVE_START_S + MOVE_DURATION_S + 5.0
|
||||||
|
RUN_TIMEOUT_S = 1740
|
||||||
|
|
||||||
|
|
||||||
|
def read_events(path: Path) -> list[dict]:
|
||||||
|
events = []
|
||||||
|
if path.is_file():
|
||||||
|
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||||
|
if line.startswith("{"):
|
||||||
|
try:
|
||||||
|
events.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
return events
|
||||||
|
|
||||||
|
|
||||||
|
def percentile(values: list[float], fraction: float) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
ordered = sorted(values)
|
||||||
|
return ordered[min(len(ordered) - 1, int(round((len(ordered) - 1) * fraction)))]
|
||||||
|
|
||||||
|
|
||||||
|
def sample_unreal_peak_rss() -> tuple[int, int] | None:
|
||||||
|
executable = str(EDITOR).encode()
|
||||||
|
project = str(HOST / "VFHost.uproject").encode()
|
||||||
|
peak: tuple[int, int] | None = None
|
||||||
|
for process in Path("/proc").glob("[0-9]*"):
|
||||||
|
try:
|
||||||
|
command_line = (process / "cmdline").read_bytes()
|
||||||
|
if executable not in command_line or project not in command_line:
|
||||||
|
continue
|
||||||
|
status = (process / "status").read_text(encoding="ascii", errors="ignore")
|
||||||
|
match = re.search(r"^VmHWM:\s+(\d+)\s+kB$", status, re.MULTILINE)
|
||||||
|
if match:
|
||||||
|
rss_kib = int(match.group(1))
|
||||||
|
if peak is None or rss_kib > peak[0]:
|
||||||
|
peak = (rss_kib, int(process.name))
|
||||||
|
except (FileNotFoundError, PermissionError, ProcessLookupError, OSError):
|
||||||
|
continue
|
||||||
|
return peak
|
||||||
|
|
||||||
|
|
||||||
|
def endphysics_gap(stdout: Path) -> tuple[float | None, int | None]:
|
||||||
|
release_times: dict[int, dict[int, datetime]] = {}
|
||||||
|
pattern = re.compile(
|
||||||
|
r"^\[(\d{4}\.\d{2}\.\d{2}-\d{2}\.\d{2}\.\d{2}:\d{3})\].*"
|
||||||
|
r"LogTick: tick\s+(\d+)\s+-+\s+Release tick group ([34])")
|
||||||
|
with stdout.open(encoding="utf-8", errors="replace") as stream:
|
||||||
|
for line in stream:
|
||||||
|
match = pattern.match(line)
|
||||||
|
if not match:
|
||||||
|
continue
|
||||||
|
timestamp = datetime.strptime(match.group(1), "%Y.%m.%d-%H.%M.%S:%f")
|
||||||
|
frame = int(match.group(2))
|
||||||
|
group = int(match.group(3))
|
||||||
|
release_times.setdefault(frame, {})[group] = timestamp
|
||||||
|
gaps = [(round((times[4] - times[3]).total_seconds() * 1000.0, 3), frame)
|
||||||
|
for frame, times in release_times.items() if 3 in times and 4 in times]
|
||||||
|
return max(gaps, default=(None, None))
|
||||||
|
|
||||||
|
|
||||||
|
def run(stratum: str, repeat: int, phase: str, ready_depth: int,
|
||||||
|
background_cap: int, worker_reserve: int, helper_background: int,
|
||||||
|
tile_background_low: int, tile_background_normal: int, startup_boost: int,
|
||||||
|
tick_logs: bool, insights: bool) -> dict:
|
||||||
|
label = (f"{phase}-{stratum.lower()}-rd{ready_depth}-bg{background_cap}"
|
||||||
|
f"-reserve{worker_reserve}-helperbg{helper_background}"
|
||||||
|
f"-tilelow{tile_background_low}-tilenormal{tile_background_normal}"
|
||||||
|
f"-ticklog{int(tick_logs)}"
|
||||||
|
f"-boost{startup_boost}-{repeat}")
|
||||||
|
trace = OUT / f"{label}.trace.jsonl"
|
||||||
|
utrace = OUT / f"{label}.utrace"
|
||||||
|
abslog = OUT / f"{label}.log"
|
||||||
|
stdout = OUT / f"{label}.stdout.log"
|
||||||
|
output = HOST_PLUGIN_SAVED / "output" / label
|
||||||
|
userdir = HOST_SAVED / "user" / label
|
||||||
|
zendata = HOST_SAVED / "zen" / label
|
||||||
|
ddc = HOST_SAVED / "DDC"
|
||||||
|
for directory in (OUT, output, userdir, zendata, ddc):
|
||||||
|
directory.mkdir(parents=True, exist_ok=True)
|
||||||
|
trace.unlink(missing_ok=True)
|
||||||
|
utrace.unlink(missing_ok=True)
|
||||||
|
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in SPAWNS_M[stratum])
|
||||||
|
command = [
|
||||||
|
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
|
||||||
|
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
|
||||||
|
"-notraceserver", "-nocrashreports",
|
||||||
|
"-DisablePlugins=AndroidFileServer",
|
||||||
|
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
|
||||||
|
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
|
||||||
|
"-archetype=TunnelNetwork", f"-slot={1 if stratum == 'FI' else 0}",
|
||||||
|
"-modes=export", "-gameconfig=1", "-coreworldtest=1",
|
||||||
|
"-ownerfourstrateplan=1", "-corevoxelsizecm=25", "-coreislandtest=0",
|
||||||
|
"-corefallbacktest=0", "-coreofftest=0", "-coreplantest=0",
|
||||||
|
"-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings", "-corehybridmesher=0",
|
||||||
|
f"-out={output}", "-voxel.StreamingCoarseFirst=1",
|
||||||
|
"-voxel.StreamingDynamicNearReserve=1",
|
||||||
|
"-voxel.StreamingLegacyPendingResultCap=0",
|
||||||
|
"-voxel.CoreRegionConcurrentAdmission=1",
|
||||||
|
"-voxel.StreamingResultBacklogCap=256",
|
||||||
|
f"-voxel.StreamingReadyQueueDepth={ready_depth}",
|
||||||
|
f"-voxel.StreamingBackgroundWorkerCap={background_cap}",
|
||||||
|
f"-voxel.StreamingBackgroundWorkerReserve={worker_reserve}",
|
||||||
|
f"-voxel.CoreParallelForBackgroundPriority={helper_background}",
|
||||||
|
f"-voxel.StreamingTileBackgroundLow={tile_background_low}",
|
||||||
|
f"-voxel.StreamingTileBackgroundNormal={tile_background_normal}",
|
||||||
|
f"-voxel.TestLogTickFunctions={int(tick_logs)}",
|
||||||
|
"-voxel.CoreColumnCacheDirectMapped=0", "-voxel.CoreMCGradientMemoization=1",
|
||||||
|
"-voxel.StreamingCollisionLookahead=1",
|
||||||
|
f"-voxel.StreamingStartupApplyBoost={startup_boost}",
|
||||||
|
"-voxel.TestMove=1", "-voxel.TestMoveSpeedCmPerSecond=140.0",
|
||||||
|
f"-voxel.TestMoveStartSeconds={MOVE_START_S:.1f}",
|
||||||
|
f"-voxel.TestMoveDurationSeconds={MOVE_DURATION_S:.1f}",
|
||||||
|
f"-voxel.TestExitSeconds={EXIT_S:.1f}", "-voxel.TestMoveDirection=1,0,0",
|
||||||
|
"-voxel.StreamingSchedulerPolicy=3", "-voxel.TestExitOnSteady=0",
|
||||||
|
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
|
||||||
|
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
|
||||||
|
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
|
||||||
|
f"-abslog={abslog}",
|
||||||
|
]
|
||||||
|
if insights:
|
||||||
|
command += ["-trace=cpu,task,frame", "-statnamedevents",
|
||||||
|
f"-tracefile={utrace}", "-tracefiletrunc"]
|
||||||
|
wrapped = ["taskset", "-c", "0-11", "timeout", f"{RUN_TIMEOUT_S}s", *command]
|
||||||
|
started = time.monotonic()
|
||||||
|
with stdout.open("w", encoding="utf-8") as stream:
|
||||||
|
process = subprocess.Popen(wrapped, cwd=ROOT, stdout=stream,
|
||||||
|
stderr=subprocess.STDOUT)
|
||||||
|
peak_rss = None
|
||||||
|
while process.poll() is None:
|
||||||
|
observed = sample_unreal_peak_rss()
|
||||||
|
if observed and (peak_rss is None or observed[0] > peak_rss[0]):
|
||||||
|
peak_rss = observed
|
||||||
|
time.sleep(0.25)
|
||||||
|
exit_code = process.wait()
|
||||||
|
observed = sample_unreal_peak_rss()
|
||||||
|
if observed and (peak_rss is None or observed[0] > peak_rss[0]):
|
||||||
|
peak_rss = observed
|
||||||
|
|
||||||
|
events = read_events(trace)
|
||||||
|
worker_backend_config = next(
|
||||||
|
(e for e in events if e.get("type") == "worker_backend_config"), {})
|
||||||
|
worker_cap_readback = next(
|
||||||
|
(e for e in events if e.get("type") == "streaming_worker_cap_readback"), {})
|
||||||
|
samples = [e for e in events if e.get("type") == "streaming_sample"]
|
||||||
|
arrival = [e for e in samples if float(e.get("t_s", 0.0)) < MOVE_START_S]
|
||||||
|
full = next((e for e in arrival
|
||||||
|
if sum(e.get("desired_by_lod", [0] * 9)[:3]) > 0
|
||||||
|
and sum(e.get("settled_by_lod", [0] * 9)[:3])
|
||||||
|
>= sum(e.get("desired_by_lod", [0] * 9)[:3])), None)
|
||||||
|
first_admission = next((e for e in events if e.get("type") == "scheduler_admission"
|
||||||
|
and e.get("desired_rebuilt") == 1), None)
|
||||||
|
request_events = [e for e in events if e.get("type") == "request"]
|
||||||
|
initial_request_events = [e for e in request_events if e.get("epoch") == 0]
|
||||||
|
tile_by_ordinal = {e.get("request_ordinal"): e for e in events
|
||||||
|
if e.get("type") == "tile"}
|
||||||
|
collision_count = sum(1 for e in events if e.get("type") == "collision_ready"
|
||||||
|
and e.get("level", 9) <= 2
|
||||||
|
and full and e.get("t_s", 1e30) <= full["t_s"])
|
||||||
|
initial_tiles = [tile_by_ordinal[r.get("request_ordinal")] for r in initial_request_events
|
||||||
|
if full and r.get("t_s", 1e30) <= full["t_s"]
|
||||||
|
and r.get("request_ordinal") in tile_by_ordinal
|
||||||
|
and tile_by_ordinal[r["request_ordinal"]].get("level", 9) <= 2]
|
||||||
|
burst_start_s = float(first_admission["t_s"]) if first_admission else -1.0
|
||||||
|
burst_end_s = float(full["t_s"]) if full else -1.0
|
||||||
|
work_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "game_thread_work_frame"}
|
||||||
|
work_values = [e for e in work_by_frame.values()
|
||||||
|
if burst_start_s <= float(e.get("t_s", -1.0)) <= burst_end_s]
|
||||||
|
arrival_frame_ids = {int(e.get("frame", -1)) for e in work_values}
|
||||||
|
scheduler_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "scheduler_admission"}
|
||||||
|
scheduler_values = [e for e in events if e.get("type") == "scheduler_admission"
|
||||||
|
and burst_start_s <= float(e.get("t_s", -1.0)) <= burst_end_s]
|
||||||
|
apply_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "apply_frame"}
|
||||||
|
commandlet_tick_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "commandlet_world_tick_frame"}
|
||||||
|
garbage_collect_by_frame = {int(e.get("frame", -1)): e for e in events
|
||||||
|
if e.get("type") == "garbage_collect"}
|
||||||
|
# apply_frame N+1 records the interval containing AVoxelWorld::Tick N. Its apply_ms
|
||||||
|
# belongs to Tick N+1, so use apply_by_frame[N] for the same-tick apply breakdown.
|
||||||
|
near_intervals = [apply_by_frame[frame + 1] for frame in arrival_frame_ids
|
||||||
|
if frame + 1 in apply_by_frame]
|
||||||
|
rmc_by_frame: dict[int, dict[str, float]] = {}
|
||||||
|
for event in events:
|
||||||
|
frame = int(event.get("frame", -1))
|
||||||
|
if event.get("type") != "rmc_submit" or frame not in arrival_frame_ids:
|
||||||
|
continue
|
||||||
|
values = rmc_by_frame.setdefault(frame, {
|
||||||
|
"component_acquire_ms": 0.0,
|
||||||
|
"create_section_group_ms": 0.0,
|
||||||
|
"create_sections_ms": 0.0,
|
||||||
|
"collision_handoff_ms": 0.0,
|
||||||
|
})
|
||||||
|
for key in values:
|
||||||
|
values[key] += float(event.get(key, 0.0))
|
||||||
|
collision_completion_by_frame: dict[int, float] = {}
|
||||||
|
for event in events:
|
||||||
|
frame = int(event.get("frame", -1))
|
||||||
|
if event.get("type") == "collision_completion" and frame in arrival_frame_ids:
|
||||||
|
collision_completion_by_frame[frame] = collision_completion_by_frame.get(frame, 0.0) \
|
||||||
|
+ float(event.get("handler_ms", 0.0))
|
||||||
|
near_requests = [r for r in request_events if full and r.get("t_s", 1e30) <= full["t_s"]
|
||||||
|
and r.get("level", 9) <= 2]
|
||||||
|
initial_lod0_counts = Counter(
|
||||||
|
tuple(r.get("tile", ())) for r in near_requests if r.get("level") == 0)
|
||||||
|
repeated_lod0_keys = [
|
||||||
|
{"tile": list(tile), "requests": count}
|
||||||
|
for tile, count in sorted(initial_lod0_counts.items()) if count > 1]
|
||||||
|
initial_scheduler_frames = [e for e in events
|
||||||
|
if e.get("type") == "scheduler_admission" and full
|
||||||
|
and e.get("t_s", 1e30) <= full["t_s"]]
|
||||||
|
worst_interval = max(near_intervals, key=lambda e: float(e.get("frame_ms", 0.0)), default=None)
|
||||||
|
worst_interval_frame_id = int(worst_interval.get("frame", -1)) if worst_interval else -1
|
||||||
|
worst_work_frame_id = worst_interval_frame_id - 1
|
||||||
|
worst_work = work_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_apply = apply_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_scheduler = scheduler_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_rmc = rmc_by_frame.get(worst_work_frame_id, {})
|
||||||
|
worst_collision_completion_ms = collision_completion_by_frame.get(worst_work_frame_id, 0.0)
|
||||||
|
rmc_values = [rmc_by_frame.get(frame, {}) for frame in arrival_frame_ids]
|
||||||
|
apply_values = [apply_by_frame[frame] for frame in arrival_frame_ids
|
||||||
|
if frame in apply_by_frame]
|
||||||
|
commandlet_tick_values = [commandlet_tick_by_frame[frame] for frame in arrival_frame_ids
|
||||||
|
if frame in commandlet_tick_by_frame]
|
||||||
|
garbage_collect_values = [garbage_collect_by_frame[frame] for frame in arrival_frame_ids
|
||||||
|
if frame in garbage_collect_by_frame]
|
||||||
|
|
||||||
|
def work_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in work_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
def scheduler_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in scheduler_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
def rmc_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in rmc_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
def commandlet_stat(name: str, fraction: float | None = None) -> float | None:
|
||||||
|
values = [float(e.get(name, 0.0)) for e in commandlet_tick_values]
|
||||||
|
return percentile(values, fraction) if fraction is not None else max(values, default=0.0)
|
||||||
|
|
||||||
|
row = {
|
||||||
|
"label": label, "stratum": stratum, "repeat": repeat, "phase": phase,
|
||||||
|
"ready_queue_depth": ready_depth,
|
||||||
|
"requested_background_worker_cap": background_cap,
|
||||||
|
"background_worker_cap": worker_cap_readback.get(
|
||||||
|
"streaming_background_worker_cap", background_cap),
|
||||||
|
"requested_background_worker_reserve": worker_reserve,
|
||||||
|
"background_worker_reserve": worker_backend_config.get(
|
||||||
|
"streaming_background_worker_reserve", worker_cap_readback.get(
|
||||||
|
"background_worker_reserve", worker_reserve)),
|
||||||
|
"requested_core_parallel_for_background_priority": helper_background,
|
||||||
|
"core_parallel_for_background_priority": worker_backend_config.get(
|
||||||
|
"core_parallel_for_background_priority", helper_background),
|
||||||
|
"requested_streaming_tile_background_low": tile_background_low,
|
||||||
|
"streaming_tile_background_low": worker_backend_config.get(
|
||||||
|
"streaming_tile_background_low", tile_background_low),
|
||||||
|
"requested_streaming_tile_background_normal": tile_background_normal,
|
||||||
|
"streaming_tile_background_normal": worker_backend_config.get(
|
||||||
|
"streaming_tile_background_normal", tile_background_normal),
|
||||||
|
"effective_tile_slots": worker_backend_config.get("effective_tile_slots"),
|
||||||
|
"background_worker_threads": worker_backend_config.get("taskgraph_background_threads"),
|
||||||
|
"startup_apply_boost": startup_boost, "insights": insights,
|
||||||
|
"engine_tick_logs": tick_logs,
|
||||||
|
"exit_code": exit_code,
|
||||||
|
"launch_wrapper": "taskset -c 0-11 timeout 1740s (no address-space limit)",
|
||||||
|
"peak_rss_kib": peak_rss[0] if peak_rss else None,
|
||||||
|
"peak_rss_mib": round(peak_rss[0] / 1024.0, 1) if peak_rss else None,
|
||||||
|
"peak_rss_pid": peak_rss[1] if peak_rss else None,
|
||||||
|
"endphysics_to_postphysics_gap_ms_max": None,
|
||||||
|
"endphysics_gap_frame": None,
|
||||||
|
"wall_s": round(time.monotonic() - started, 3),
|
||||||
|
"desired_t_s": next((e.get("t_s") for e in events if e.get("type") == "desired_set"), None),
|
||||||
|
"first_admission_t_s": first_admission.get("t_s") if first_admission else None,
|
||||||
|
"lod02_full_settled_t_s": full.get("t_s") if full else None,
|
||||||
|
"near_settle_from_first_admission_s": round(
|
||||||
|
float(full["t_s"]) - float(first_admission["t_s"]), 4)
|
||||||
|
if full and first_admission else None,
|
||||||
|
"desired_by_lod": full.get("desired_by_lod", [])[:3] if full else None,
|
||||||
|
"settled_by_lod": full.get("settled_by_lod", [])[:3] if full else None,
|
||||||
|
"near_admitted_tiles": len(near_requests),
|
||||||
|
"initial_lod0_admissions": sum(initial_lod0_counts.values()),
|
||||||
|
"repeated_lod0_request_keys_before_settle": repeated_lod0_keys,
|
||||||
|
"peak_result_backlog_before_settle": max(
|
||||||
|
(int(e.get("result_backlog", 0)) for e in initial_scheduler_frames), default=0),
|
||||||
|
"last_admission_t_s": max((e.get("t_s", 0.0) for e in near_requests), default=None),
|
||||||
|
"near_tile_events": len(initial_tiles),
|
||||||
|
"near_worker_seconds": round(sum(e.get("worker_task_s", 0.0) for e in initial_tiles), 4),
|
||||||
|
"near_core_seconds": round(sum(e.get("core_generate_s", 0.0) for e in initial_tiles), 4),
|
||||||
|
"collision_ready_by_settle": collision_count,
|
||||||
|
"burst_frame_count": len(near_intervals),
|
||||||
|
"burst_frame_ms_p95": round(sorted(e.get("frame_ms", 0.0) for e in near_intervals)[
|
||||||
|
min(len(near_intervals) - 1, int(.95 * (len(near_intervals) - 1)))], 3)
|
||||||
|
if near_intervals else None,
|
||||||
|
"burst_frame_ms_max": round(max((e.get("frame_ms", 0.0) for e in near_intervals), default=0.0), 3),
|
||||||
|
"burst_full_world_tick_ms_p99": round(commandlet_stat("world_tick_ms", .99) or 0.0, 3),
|
||||||
|
"burst_full_world_tick_ms_max": round(commandlet_stat("world_tick_ms") or 0.0, 3),
|
||||||
|
"burst_world_pre_actor_ms_p99": round(commandlet_stat("world_pre_actor_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_pre_actor_ms_max": round(commandlet_stat("world_pre_actor_ms") or 0.0, 3),
|
||||||
|
"burst_world_actor_phase_ms_p99": round(commandlet_stat("world_actor_phase_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_actor_phase_ms_max": round(commandlet_stat("world_actor_phase_ms") or 0.0, 3),
|
||||||
|
"burst_world_post_actor_ms_p99": round(commandlet_stat("world_post_actor_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_post_actor_ms_max": round(commandlet_stat("world_post_actor_ms") or 0.0, 3),
|
||||||
|
"burst_commandlet_readiness_check_ms_max": round(
|
||||||
|
commandlet_stat("readiness_check_ms") or 0.0, 3),
|
||||||
|
"burst_commandlet_sleep_ms_p99": round(commandlet_stat("sleep_ms", .99) or 0.0, 3),
|
||||||
|
"burst_commandlet_sleep_ms_max": round(commandlet_stat("sleep_ms") or 0.0, 3),
|
||||||
|
"burst_commandlet_loop_ms_max": round(commandlet_stat("loop_ms") or 0.0, 3),
|
||||||
|
"burst_gc_count": len(garbage_collect_values),
|
||||||
|
"burst_gc_duration_ms_max": round(max(
|
||||||
|
(float(e.get("duration_ms", 0.0)) for e in garbage_collect_values), default=0.0), 3),
|
||||||
|
"burst_max_meshes_per_frame": max((e.get("meshes_applied", 0) for e in apply_values), default=0),
|
||||||
|
"burst_max_apply_ms": round(max((e.get("apply_ms", 0.0) for e in apply_values), default=0.0), 3),
|
||||||
|
"burst_max_admit_per_frame": max((sum(e.get("submitted_by_lod", [0] * 9)[:3])
|
||||||
|
for e in scheduler_values), default=0),
|
||||||
|
"burst_frame_timing_count": len(work_values),
|
||||||
|
"burst_active_generation_workers_p95": round(work_stat("active_generation_workers", .95) or 0.0, 3),
|
||||||
|
"burst_active_generation_workers_max": round(work_stat("active_generation_workers") or 0.0, 3),
|
||||||
|
"burst_active_tile_tasks_max": round(work_stat("active_tile_tasks") or 0.0, 3),
|
||||||
|
"burst_world_tick_ms_p99": round(work_stat("tick_ms", .99) or 0.0, 3),
|
||||||
|
"burst_world_tick_ms_max": round(work_stat("tick_ms") or 0.0, 3),
|
||||||
|
"burst_update_chunks_ms_p99": round(work_stat("update_chunks_ms", .99) or 0.0, 3),
|
||||||
|
"burst_update_chunks_ms_max": round(work_stat("update_chunks_ms") or 0.0, 3),
|
||||||
|
"burst_pending_drain_ms_p99": round(work_stat("pending_drain_ms", .99) or 0.0, 3),
|
||||||
|
"burst_pending_drain_ms_max": round(work_stat("pending_drain_ms") or 0.0, 3),
|
||||||
|
"burst_post_apply_callbacks_ms_p99": round(work_stat("post_apply_callbacks_ms", .99) or 0.0, 3),
|
||||||
|
"burst_post_apply_callbacks_ms_max": round(work_stat("post_apply_callbacks_ms") or 0.0, 3),
|
||||||
|
"burst_visibility_reconcile_ms_p99": round(work_stat("visibility_reconcile_ms", .99) or 0.0, 3),
|
||||||
|
"burst_visibility_reconcile_ms_max": round(work_stat("visibility_reconcile_ms") or 0.0, 3),
|
||||||
|
"burst_collision_probe_ms_max": round(work_stat("collision_probe_ms") or 0.0, 3),
|
||||||
|
"burst_visible_apply_probe_ms_max": round(work_stat("visible_apply_probe_ms") or 0.0, 3),
|
||||||
|
"burst_lod0_ready_ms_max": round(work_stat("lod0_ready_ms") or 0.0, 3),
|
||||||
|
"burst_steady_trace_ms_max": round(work_stat("steady_trace_ms") or 0.0, 3),
|
||||||
|
"burst_modification_ms_max": round(work_stat("modification_ms") or 0.0, 3),
|
||||||
|
"burst_lifecycle_audit_ms_p99": round(work_stat("lifecycle_audit_ms", .99) or 0.0, 3),
|
||||||
|
"burst_lifecycle_audit_ms_max": round(work_stat("lifecycle_audit_ms") or 0.0, 3),
|
||||||
|
"burst_collision_gate_ms_p99": round(work_stat("collision_gate_ms", .99) or 0.0, 3),
|
||||||
|
"burst_collision_gate_ms_max": round(work_stat("collision_gate_ms") or 0.0, 3),
|
||||||
|
"burst_unload_ms_p99": round(work_stat("unload_ms", .99) or 0.0, 3),
|
||||||
|
"burst_unload_ms_max": round(work_stat("unload_ms") or 0.0, 3),
|
||||||
|
"burst_trace_json_ms_p99": round(work_stat("trace_json_ms", .99) or 0.0, 3),
|
||||||
|
"burst_trace_json_ms_max": round(work_stat("trace_json_ms") or 0.0, 3),
|
||||||
|
"burst_desired_rebuild_ms_max": round(scheduler_stat("desired_rebuild_ms") or 0.0, 3),
|
||||||
|
"burst_rmc_create_group_ms_p99": round(rmc_stat("create_section_group_ms", .99) or 0.0, 3),
|
||||||
|
"burst_rmc_create_group_ms_max": round(rmc_stat("create_section_group_ms") or 0.0, 3),
|
||||||
|
"burst_rmc_collision_handoff_ms_max": round(rmc_stat("collision_handoff_ms") or 0.0, 3),
|
||||||
|
"burst_collision_completion_ms_max": round(max(
|
||||||
|
collision_completion_by_frame.values(), default=0.0), 3),
|
||||||
|
"worst_frame_breakdown": {
|
||||||
|
"frame": worst_work_frame_id,
|
||||||
|
"world_tick_frame": worst_work_frame_id,
|
||||||
|
"frame_interval_end_frame": worst_interval_frame_id,
|
||||||
|
"frame_interval_ms": round(float(worst_interval.get("frame_ms", 0.0)), 3)
|
||||||
|
if worst_interval else None,
|
||||||
|
"apply_ms": round(float(worst_apply.get("apply_ms", 0.0)), 3)
|
||||||
|
if worst_apply else None,
|
||||||
|
"result_drain_ms": round(float(worst_apply.get("drain_ms", 0.0)), 3)
|
||||||
|
if worst_apply else None,
|
||||||
|
"outside_world_tick_ms": round(max(0.0, float(worst_interval.get("frame_ms", 0.0))
|
||||||
|
- float(worst_work.get("tick_ms", 0.0))), 3)
|
||||||
|
if worst_interval else None,
|
||||||
|
"outside_full_world_tick_ms": round(max(0.0, float(worst_interval.get("frame_ms", 0.0))
|
||||||
|
- float(commandlet_tick_by_frame.get(worst_work_frame_id, {}).get("world_tick_ms", 0.0))), 3)
|
||||||
|
if worst_interval else None,
|
||||||
|
"full_world_tick_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_tick_ms", 0.0)), 3),
|
||||||
|
"world_pre_actor_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_pre_actor_ms", 0.0)), 3),
|
||||||
|
"world_actor_phase_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_actor_phase_ms", 0.0)), 3),
|
||||||
|
"world_post_actor_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("world_post_actor_ms", 0.0)), 3),
|
||||||
|
"commandlet_readiness_check_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("readiness_check_ms", 0.0)), 3),
|
||||||
|
"commandlet_sleep_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("sleep_ms", 0.0)), 3),
|
||||||
|
"commandlet_loop_ms": round(float(commandlet_tick_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("loop_ms", 0.0)), 3),
|
||||||
|
"gc_duration_ms": round(float(garbage_collect_by_frame.get(
|
||||||
|
worst_work_frame_id, {}).get("duration_ms", 0.0)), 3),
|
||||||
|
"world_tick_ms": round(float(worst_work.get("tick_ms", 0.0)), 3),
|
||||||
|
"active_generation_workers": int(worst_work.get("active_generation_workers", 0)),
|
||||||
|
"active_tile_tasks": int(worst_work.get("active_tile_tasks", 0)),
|
||||||
|
"collision_gate_ms": round(float(worst_work.get("collision_gate_ms", 0.0)), 3),
|
||||||
|
"update_chunks_ms": round(float(worst_work.get("update_chunks_ms", 0.0)), 3),
|
||||||
|
"desired_rebuild_ms": round(float(worst_scheduler.get("desired_rebuild_ms", 0.0)), 3),
|
||||||
|
"pending_drain_ms": round(float(worst_work.get("pending_drain_ms", 0.0)), 3),
|
||||||
|
"post_apply_callbacks_ms": round(float(worst_work.get("post_apply_callbacks_ms", 0.0)), 3),
|
||||||
|
"collision_probe_ms": round(float(worst_work.get("collision_probe_ms", 0.0)), 3),
|
||||||
|
"visibility_reconcile_ms": round(float(worst_work.get("visibility_reconcile_ms", 0.0)), 3),
|
||||||
|
"visible_apply_probe_ms": round(float(worst_work.get("visible_apply_probe_ms", 0.0)), 3),
|
||||||
|
"lod0_ready_ms": round(float(worst_work.get("lod0_ready_ms", 0.0)), 3),
|
||||||
|
"unload_ms": round(float(worst_work.get("unload_ms", 0.0)), 3),
|
||||||
|
"steady_trace_ms": round(float(worst_work.get("steady_trace_ms", 0.0)), 3),
|
||||||
|
"modification_ms": round(float(worst_work.get("modification_ms", 0.0)), 3),
|
||||||
|
"lifecycle_audit_ms": round(float(worst_work.get("lifecycle_audit_ms", 0.0)), 3),
|
||||||
|
"trace_json_ms": round(float(worst_work.get("trace_json_ms", 0.0)), 3),
|
||||||
|
"trace_archive_serialize_ms": round(float(
|
||||||
|
worst_work.get("trace_archive_serialize_ms", 0.0)), 3),
|
||||||
|
"collision_completion_ms": round(worst_collision_completion_ms, 3),
|
||||||
|
**{key: round(float(worst_rmc.get(key, 0.0)), 3)
|
||||||
|
for key in ("component_acquire_ms", "create_section_group_ms",
|
||||||
|
"create_sections_ms", "collision_handoff_ms")},
|
||||||
|
},
|
||||||
|
"trace": str(trace.relative_to(ROOT)), "utrace": str(utrace.relative_to(ROOT)),
|
||||||
|
"abslog": str(abslog.relative_to(ROOT)), "stdout": str(stdout.relative_to(ROOT)),
|
||||||
|
}
|
||||||
|
stdout_text = stdout.read_text(encoding="utf-8", errors="replace")
|
||||||
|
gap_ms, gap_frame = endphysics_gap(stdout)
|
||||||
|
row["endphysics_to_postphysics_gap_ms_max"] = gap_ms
|
||||||
|
row["endphysics_gap_frame"] = gap_frame
|
||||||
|
failure = re.search(r"\[VoxelForgeCoreWorldTest\] failed: ([^\r\n]+)", stdout_text)
|
||||||
|
if failure:
|
||||||
|
row["world_test_failure"] = failure.group(1)
|
||||||
|
with (OUT / "runs.jsonl").open("a", encoding="utf-8") as stream:
|
||||||
|
stream.write(json.dumps(row, sort_keys=True) + "\n")
|
||||||
|
(OUT / f"{label}.summary.json").write_text(json.dumps(row, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
print("[L106UnrealArrival] " + json.dumps(row, sort_keys=True), flush=True)
|
||||||
|
if exit_code:
|
||||||
|
raise SystemExit(f"Unreal commandlet failed: {stdout}; exit={exit_code}")
|
||||||
|
if not full or len(arrival) < 8:
|
||||||
|
raise SystemExit(f"Unreal arrival trace incomplete: {stdout}; exit={exit_code}")
|
||||||
|
if insights and (not utrace.is_file() or utrace.stat().st_size < 1024):
|
||||||
|
raise SystemExit(f"Unreal Insights trace missing or empty: {utrace}")
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--stratum", choices=("FI", "SW"), required=True)
|
||||||
|
parser.add_argument("--repeat", type=int, default=1)
|
||||||
|
parser.add_argument("--phase", choices=("probe", "before", "after", "snapshot"), default="probe")
|
||||||
|
parser.add_argument("--ready-depth", type=int, default=64)
|
||||||
|
parser.add_argument("--background-cap", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--worker-reserve", type=int, choices=(0, 1, 2, 3), default=0)
|
||||||
|
parser.add_argument("--helper-background", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--tile-background-low", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--tile-background-normal", type=int, choices=(0, 1), default=0)
|
||||||
|
parser.add_argument("--startup-boost", type=int, choices=(0, 1), default=1)
|
||||||
|
parser.add_argument("--tick-logs", action="store_true")
|
||||||
|
parser.add_argument("--insights", action="store_true")
|
||||||
|
args = parser.parse_args()
|
||||||
|
if not EDITOR.is_file():
|
||||||
|
raise SystemExit(f"missing Unreal commandlet executable: {EDITOR}")
|
||||||
|
run(args.stratum, args.repeat, args.phase, args.ready_depth,
|
||||||
|
args.background_cap, args.worker_reserve, args.helper_background,
|
||||||
|
args.tile_background_low, args.tile_background_normal, args.startup_boost,
|
||||||
|
args.tick_logs, args.insights)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
|
After Width: | Height: | Size: 178 KiB |
|
After Width: | Height: | Size: 193 KiB |
|
After Width: | Height: | Size: 193 KiB |
|
After Width: | Height: | Size: 135 KiB |
|
After Width: | Height: | Size: 135 KiB |
|
After Width: | Height: | Size: 83 KiB |
|
After Width: | Height: | Size: 83 KiB |
|
After Width: | Height: | Size: 26 KiB |
|
After Width: | Height: | Size: 26 KiB |
|
After Width: | Height: | Size: 173 KiB |
|
After Width: | Height: | Size: 173 KiB |
|
After Width: | Height: | Size: 266 KiB |
|
After Width: | Height: | Size: 266 KiB |
|
After Width: | Height: | Size: 11 KiB |
|
After Width: | Height: | Size: 11 KiB |
|
After Width: | Height: | Size: 16 KiB |
|
After Width: | Height: | Size: 16 KiB |
|
After Width: | Height: | Size: 8.3 KiB |
|
After Width: | Height: | Size: 8.3 KiB |
@@ -0,0 +1,272 @@
|
|||||||
|
diff --git a/Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h b/Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h
|
||||||
|
index 972aaf8..94ec19b 100644
|
||||||
|
--- a/Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h
|
||||||
|
+++ b/Standalone/Core/include/VoxelForgeCore/VoxelForgeCore.h
|
||||||
|
@@ -139,6 +139,9 @@ namespace VoxelForgeCore
|
||||||
|
std::uint64_t passageTileRouteSegmentCandidates = 0;
|
||||||
|
std::uint64_t passageTileMouthSegmentCandidates = 0;
|
||||||
|
std::uint64_t resolvedBoundaryBandQueries = 0;
|
||||||
|
+ std::uint64_t resolvedBoundaryBandColumnEvaluations = 0;
|
||||||
|
+ std::uint64_t resolvedBoundaryBandColumnCacheHits = 0;
|
||||||
|
+ std::uint64_t resolvedBoundaryBandColumnOffGridFallbacks = 0;
|
||||||
|
std::uint64_t passageComposerQueries = 0;
|
||||||
|
std::uint64_t passageIterations = 0;
|
||||||
|
std::uint64_t passageTileBoundChecks = 0;
|
||||||
|
diff --git a/Standalone/Core/src/ShapeWorldInternal.h b/Standalone/Core/src/ShapeWorldInternal.h
|
||||||
|
index 8626d7e..2c53c35 100644
|
||||||
|
--- a/Standalone/Core/src/ShapeWorldInternal.h
|
||||||
|
+++ b/Standalone/Core/src/ShapeWorldInternal.h
|
||||||
|
@@ -339,6 +339,15 @@ public:
|
||||||
|
verticalBoundSkipsByCaller{},negativeReachSkipsByCaller{},
|
||||||
|
positiveReachSkipsByCaller{};
|
||||||
|
};
|
||||||
|
+ struct FResolvedBoundaryColumnData
|
||||||
|
+ {
|
||||||
|
+ std::int32_t minimumXVoxel=0,minimumYVoxel=0,stepVoxel=0,dimension=0;
|
||||||
|
+ std::size_t pairCount=0;
|
||||||
|
+ std::vector<std::uint8_t> pairReady;
|
||||||
|
+ std::vector<VoxelForgeCore::ResolvedLayoutTransitionBandsV1> bands;
|
||||||
|
+ std::uint64_t evaluations=0;
|
||||||
|
+ mutable std::atomic<std::uint64_t> cacheHits{0},offGridFallbacks{0};
|
||||||
|
+ };
|
||||||
|
FScopedPassageTileQueries(const FStandaloneShapeWorld& shape,
|
||||||
|
const std::array<std::int64_t,3>& minimumSubvoxels,
|
||||||
|
const std::array<std::int64_t,3>& maximumSubvoxels,
|
||||||
|
@@ -502,6 +511,8 @@ public:
|
||||||
|
passageCarveReachMetres_(source.passageCarveReachMetres_),
|
||||||
|
passageSweepData_(source.passageSweepData_),
|
||||||
|
ownsPassageSweepDiagnostics_(false),
|
||||||
|
+ resolvedBoundaryColumnData_(source.resolvedBoundaryColumnData_),
|
||||||
|
+ ownsResolvedBoundaryColumnDiagnostics_(false),
|
||||||
|
clampCounters_(source.clampCounters_),ownsClampCounters_(false),
|
||||||
|
projectionCacheSize_(source.projectionCacheSize_),
|
||||||
|
reuseCounters_(source.reuseCounters_),ownsReuseCounters_(false)
|
||||||
|
@@ -546,6 +557,7 @@ public:
|
||||||
|
std::memory_order_relaxed);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
+ copyResolvedBoundaryColumnDiagnostics();
|
||||||
|
if(ownsReuseCounters_&&reuseCounters_)
|
||||||
|
std::printf("[L98PassageReuseTile] lod=%d projection_columns=%llu projection_computes=%llu projection_hits=%llu floor_profile_reads=%llu carve_prepared=%llu support_prepared=%llu capsule_prepared=%llu mouth_prepared=%llu\n",
|
||||||
|
profileLod_,
|
||||||
|
@@ -626,6 +638,23 @@ public:
|
||||||
|
bool capturesCallerDiagnostics() const noexcept{return profileStats_!=nullptr;}
|
||||||
|
void prepareSweptTubeBase(const FIntVector& origin,std::int32_t step,
|
||||||
|
std::int32_t cells,std::uint32_t seed);
|
||||||
|
+ void prepareResolvedBoundaryColumns(const FIntVector& origin,
|
||||||
|
+ std::int32_t step,std::int32_t cells);
|
||||||
|
+ const VoxelForgeCore::ResolvedLayoutTransitionBandsV1* cachedTransitionBands(
|
||||||
|
+ float X,float Y,const VoxelForgeCore::ResolvedLayoutStratumV2& upper,
|
||||||
|
+ const VoxelForgeCore::ResolvedLayoutStratumV2& lower) const noexcept;
|
||||||
|
+ void copyResolvedBoundaryColumnDiagnostics() const noexcept
|
||||||
|
+ {
|
||||||
|
+ if(!ownsResolvedBoundaryColumnDiagnostics_||!profileStats_
|
||||||
|
+ ||!resolvedBoundaryColumnData_)return;
|
||||||
|
+ profileStats_->resolvedBoundaryBandColumnEvaluations=
|
||||||
|
+ resolvedBoundaryColumnData_->evaluations;
|
||||||
|
+ profileStats_->resolvedBoundaryBandColumnCacheHits=
|
||||||
|
+ resolvedBoundaryColumnData_->cacheHits.load(std::memory_order_relaxed);
|
||||||
|
+ profileStats_->resolvedBoundaryBandColumnOffGridFallbacks=
|
||||||
|
+ resolvedBoundaryColumnData_->offGridFallbacks.load(
|
||||||
|
+ std::memory_order_relaxed);
|
||||||
|
+ }
|
||||||
|
const FPassageSweepSample* sweptSample(std::size_t passageIndex,
|
||||||
|
double qx,double qy,double qz) const;
|
||||||
|
bool hasSweptTubeBase() const noexcept{return passageSweepData_!=nullptr;}
|
||||||
|
@@ -917,6 +946,8 @@ public:
|
||||||
|
double passageCarveReachMetres_=0.0;
|
||||||
|
std::shared_ptr<FPassageTileSweepData> passageSweepData_;
|
||||||
|
bool ownsPassageSweepDiagnostics_=true;
|
||||||
|
+ std::shared_ptr<FResolvedBoundaryColumnData> resolvedBoundaryColumnData_;
|
||||||
|
+ bool ownsResolvedBoundaryColumnDiagnostics_=true;
|
||||||
|
mutable std::shared_ptr<FPassageSourceClampCounters> clampCounters_;
|
||||||
|
bool ownsClampCounters_=true;
|
||||||
|
std::size_t projectionCacheSize_=0;
|
||||||
|
@@ -1546,8 +1577,12 @@ public:
|
||||||
|
static_cast<double>(X)*Owner->CoreVoxelSizeMicrons));
|
||||||
|
const auto yMicrometres=static_cast<std::int64_t>(std::llround(
|
||||||
|
static_cast<double>(Y)*Owner->CoreVoxelSizeMicrons));
|
||||||
|
- const auto bands=ResolvedTransitionBandsAtXY(*transitionUpper,
|
||||||
|
- *transitionLower,xMicrometres,yMicrometres,skipPassageFade);
|
||||||
|
+ const auto* cachedBands=tileQueries&&tileQueries->matches(*this)
|
||||||
|
+ ?tileQueries->cachedTransitionBands(X,Y,*transitionUpper,
|
||||||
|
+ *transitionLower):nullptr;
|
||||||
|
+ const auto bands=cachedBands?*cachedBands:ResolvedTransitionBandsAtXY(
|
||||||
|
+ *transitionUpper,*transitionLower,xMicrometres,yMicrometres,
|
||||||
|
+ skipPassageFade);
|
||||||
|
const std::int64_t solidBottom=std::min(
|
||||||
|
bands.upperBedrockBottomMicrometres,bands.lowerCeilingBottomMicrometres);
|
||||||
|
const std::int64_t solidTop=std::max(
|
||||||
|
@@ -1609,8 +1644,11 @@ public:
|
||||||
|
#if defined(VF_L30_ENABLE_COST_COUNTERS)
|
||||||
|
if(skipPassageFade)++ThreadClassificationStats().TransitionPassageTileSkips;
|
||||||
|
#endif
|
||||||
|
- const auto resolvedBands=ResolvedTransitionBandsAtXY(*upper,*lower,
|
||||||
|
- xMicrometres,yMicrometres,skipPassageFade);
|
||||||
|
+ const auto* cachedBands=tileQueries&&tileQueries->matches(*this)
|
||||||
|
+ ?tileQueries->cachedTransitionBands(X,Y,*upper,*lower):nullptr;
|
||||||
|
+ const auto resolvedBands=cachedBands?*cachedBands:
|
||||||
|
+ ResolvedTransitionBandsAtXY(*upper,*lower,xMicrometres,yMicrometres,
|
||||||
|
+ skipPassageFade);
|
||||||
|
// Treat the two authored bands as one continuous solid interval. Its
|
||||||
|
// exterior faces remain the lower ceiling bottom and upper bedrock top;
|
||||||
|
// measuring signed distance from those faces also clears native generator
|
||||||
|
@@ -14424,6 +14462,115 @@ private:
|
||||||
|
inline static thread_local const FPassageCandidateBrick* ActivePassageCandidateBrick=nullptr;
|
||||||
|
};
|
||||||
|
|
||||||
|
+inline void FStandaloneShapeWorld::FScopedPassageTileQueries::prepareResolvedBoundaryColumns(
|
||||||
|
+ const FIntVector& origin,std::int32_t step,std::int32_t cells)
|
||||||
|
+{
|
||||||
|
+ if(!shape_||!mayContainResolvedBoundaryBand_||step<=0||cells<=0
|
||||||
|
+ ||!shape_->ResolvedLayout||!shape_->Owner
|
||||||
|
+ ||shape_->Owner->CoreVoxelSizeMicrons==0)return;
|
||||||
|
+
|
||||||
|
+ constexpr std::int64_t Q=VoxelForgeCore::WorldAssemblySubvoxelsPerVoxel;
|
||||||
|
+ const auto& layout=*shape_->ResolvedLayout;
|
||||||
|
+ auto data=std::make_shared<FResolvedBoundaryColumnData>();
|
||||||
|
+ data->minimumXVoxel=origin.X-step;
|
||||||
|
+ data->minimumYVoxel=origin.Y-step;
|
||||||
|
+ data->stepVoxel=step;
|
||||||
|
+ data->dimension=cells+3;
|
||||||
|
+ for(const auto& stratum:layout.strates)
|
||||||
|
+ if(stratum.order>=0)
|
||||||
|
+ data->pairCount=std::max(data->pairCount,
|
||||||
|
+ static_cast<std::size_t>(stratum.order)+1);
|
||||||
|
+ if(data->pairCount==0)return;
|
||||||
|
+ data->pairReady.assign(data->pairCount,0);
|
||||||
|
+ const double minimumZMicrometres=static_cast<double>(minimumSubvoxels_[2])
|
||||||
|
+ *shape_->Owner->CoreVoxelSizeMicrons/Q;
|
||||||
|
+ const double maximumZMicrometres=static_cast<double>(maximumSubvoxels_[2])
|
||||||
|
+ *shape_->Owner->CoreVoxelSizeMicrons/Q;
|
||||||
|
+ std::size_t activePairCount=0;
|
||||||
|
+ for(const auto& upper:layout.strates)
|
||||||
|
+ {
|
||||||
|
+ if(upper.order<0||static_cast<std::size_t>(upper.order)>=data->pairCount
|
||||||
|
+ ||!shape_->IsResolvedLayoutTransitionActive(upper))continue;
|
||||||
|
+ const auto* lower=shape_->FindResolvedLayoutStratumByOrder(upper.order+1);
|
||||||
|
+ if(!lower)continue;
|
||||||
|
+ const auto maximumDisplacement=
|
||||||
|
+ shape_->ResolvedTransitionMaximumFaceDisplacementMicrometres(upper,*lower);
|
||||||
|
+ const double transitionMinimum=std::min(upper.bedrock.bottomMicrometres,
|
||||||
|
+ lower->ceiling.bottomMicrometres)-maximumDisplacement;
|
||||||
|
+ const double transitionMaximum=std::max(upper.bedrock.topMicrometres,
|
||||||
|
+ lower->ceiling.topMicrometres)+maximumDisplacement;
|
||||||
|
+ if(maximumZMicrometres<transitionMinimum
|
||||||
|
+ ||minimumZMicrometres>=transitionMaximum)continue;
|
||||||
|
+ data->pairReady[static_cast<std::size_t>(upper.order)]=1;
|
||||||
|
+ ++activePairCount;
|
||||||
|
+ }
|
||||||
|
+ if(activePairCount==0)return;
|
||||||
|
+
|
||||||
|
+ const std::size_t columnCount=static_cast<std::size_t>(data->dimension)
|
||||||
|
+ *static_cast<std::size_t>(data->dimension);
|
||||||
|
+ if(columnCount>std::numeric_limits<std::size_t>::max()/data->pairCount)return;
|
||||||
|
+ data->bands.resize(columnCount*data->pairCount);
|
||||||
|
+ data->evaluations=static_cast<std::uint64_t>(columnCount)*activePairCount;
|
||||||
|
+ const bool skipPassageFade=!mayContainTransitionPassageFade_;
|
||||||
|
+ for(std::int32_t gy=0;gy<data->dimension;++gy)
|
||||||
|
+ for(std::int32_t gx=0;gx<data->dimension;++gx)
|
||||||
|
+ {
|
||||||
|
+ const float x=static_cast<float>(static_cast<std::int64_t>(data->minimumXVoxel)
|
||||||
|
+ +static_cast<std::int64_t>(gx)*data->stepVoxel);
|
||||||
|
+ const float y=static_cast<float>(static_cast<std::int64_t>(data->minimumYVoxel)
|
||||||
|
+ +static_cast<std::int64_t>(gy)*data->stepVoxel);
|
||||||
|
+ const std::int64_t xMicrometres=static_cast<std::int64_t>(std::llround(
|
||||||
|
+ static_cast<double>(x)*shape_->Owner->CoreVoxelSizeMicrons));
|
||||||
|
+ const std::int64_t yMicrometres=static_cast<std::int64_t>(std::llround(
|
||||||
|
+ static_cast<double>(y)*shape_->Owner->CoreVoxelSizeMicrons));
|
||||||
|
+ const std::size_t column=static_cast<std::size_t>(gy)
|
||||||
|
+ *static_cast<std::size_t>(data->dimension)+static_cast<std::size_t>(gx);
|
||||||
|
+ for(const auto& upper:layout.strates)
|
||||||
|
+ {
|
||||||
|
+ if(upper.order<0||static_cast<std::size_t>(upper.order)>=data->pairCount
|
||||||
|
+ ||!data->pairReady[static_cast<std::size_t>(upper.order)])continue;
|
||||||
|
+ const auto* lower=shape_->FindResolvedLayoutStratumByOrder(upper.order+1);
|
||||||
|
+ if(!lower)continue;
|
||||||
|
+ data->bands[column*data->pairCount+static_cast<std::size_t>(upper.order)]=
|
||||||
|
+ shape_->ResolvedTransitionBandsAtXY(upper,*lower,xMicrometres,
|
||||||
|
+ yMicrometres,skipPassageFade);
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
+ resolvedBoundaryColumnData_=std::move(data);
|
||||||
|
+}
|
||||||
|
+
|
||||||
|
+inline const VoxelForgeCore::ResolvedLayoutTransitionBandsV1*
|
||||||
|
+FStandaloneShapeWorld::FScopedPassageTileQueries::cachedTransitionBands(
|
||||||
|
+ float X,float Y,const VoxelForgeCore::ResolvedLayoutStratumV2& upper,
|
||||||
|
+ const VoxelForgeCore::ResolvedLayoutStratumV2& lower) const noexcept
|
||||||
|
+{
|
||||||
|
+ const auto* data=resolvedBoundaryColumnData_.get();
|
||||||
|
+ if(!data||upper.order<0||lower.order!=upper.order+1
|
||||||
|
+ ||static_cast<std::size_t>(upper.order)>=data->pairCount
|
||||||
|
+ ||!data->pairReady[static_cast<std::size_t>(upper.order)])return nullptr;
|
||||||
|
+ const double xCoordinate=(static_cast<double>(X)-data->minimumXVoxel)/data->stepVoxel;
|
||||||
|
+ const double yCoordinate=(static_cast<double>(Y)-data->minimumYVoxel)/data->stepVoxel;
|
||||||
|
+ const auto gx=static_cast<std::int64_t>(std::llround(xCoordinate));
|
||||||
|
+ const auto gy=static_cast<std::int64_t>(std::llround(yCoordinate));
|
||||||
|
+ if(gx<0||gy<0||gx>=data->dimension||gy>=data->dimension
|
||||||
|
+ ||static_cast<float>(static_cast<std::int64_t>(data->minimumXVoxel)
|
||||||
|
+ +gx*data->stepVoxel)!=X
|
||||||
|
+ ||static_cast<float>(static_cast<std::int64_t>(data->minimumYVoxel)
|
||||||
|
+ +gy*data->stepVoxel)!=Y)
|
||||||
|
+ {
|
||||||
|
+ if(profileStats_)
|
||||||
|
+ data->offGridFallbacks.fetch_add(1,std::memory_order_relaxed);
|
||||||
|
+ return nullptr;
|
||||||
|
+ }
|
||||||
|
+ const std::size_t column=static_cast<std::size_t>(gy)
|
||||||
|
+ *static_cast<std::size_t>(data->dimension)+static_cast<std::size_t>(gx);
|
||||||
|
+ const std::size_t index=column*data->pairCount
|
||||||
|
+ +static_cast<std::size_t>(upper.order);
|
||||||
|
+ if(index>=data->bands.size())return nullptr;
|
||||||
|
+ if(profileStats_)data->cacheHits.fetch_add(1,std::memory_order_relaxed);
|
||||||
|
+ return &data->bands[index];
|
||||||
|
+}
|
||||||
|
+
|
||||||
|
inline void FStandaloneShapeWorld::FScopedPassageTileQueries::prepareSweptTubeBase(
|
||||||
|
const FIntVector& origin,std::int32_t step,std::int32_t cells,std::uint32_t seed)
|
||||||
|
{
|
||||||
|
diff --git a/Standalone/Core/src/VoxelForgeCore.cpp b/Standalone/Core/src/VoxelForgeCore.cpp
|
||||||
|
index 0a5d451..81a04b1 100644
|
||||||
|
--- a/Standalone/Core/src/VoxelForgeCore.cpp
|
||||||
|
+++ b/Standalone/Core/src/VoxelForgeCore.cpp
|
||||||
|
@@ -2806,6 +2806,7 @@ namespace VoxelForgeCore
|
||||||
|
options.profilePassageComposer?&out.diagnostics:nullptr,key.lod,
|
||||||
|
options.diagnosticPassageSourceFloorVoxels,
|
||||||
|
options.diagnosticPassageCarveReachMetres);
|
||||||
|
+ passageFeatureScope.prepareResolvedBoundaryColumns(origin,MeshStep,MeshCells);
|
||||||
|
passageFeatureScope.prepareSweptTubeBase(origin,MeshStep,MeshCells,
|
||||||
|
static_cast<std::uint32_t>(world.Seed));
|
||||||
|
static std::atomic<bool> l98ProjectionProbeStarted{false};
|
||||||
|
@@ -3367,6 +3368,7 @@ namespace VoxelForgeCore
|
||||||
|
out.diagnostics.meshAssemblyMilliseconds=std::chrono::duration<double,std::milli>(
|
||||||
|
std::chrono::steady_clock::now()-MeshAssemblyStart).count();
|
||||||
|
CopyIslandCacheMetrics();
|
||||||
|
+ passageFeatureScope.copyResolvedBoundaryColumnDiagnostics();
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
diff --git a/Standalone/Source/StreamingWorld.h b/Standalone/Source/StreamingWorld.h
|
||||||
|
index 3019e39..3f706f9 100644
|
||||||
|
--- a/Standalone/Source/StreamingWorld.h
|
||||||
|
+++ b/Standalone/Source/StreamingWorld.h
|
||||||
|
@@ -1538,7 +1538,7 @@ private:
|
||||||
|
static_cast<unsigned long long>(D.islandSharedCacheConflictReplacements),
|
||||||
|
static_cast<double>(D.islandSharedCacheLockWaitNanoseconds)/1000.0);
|
||||||
|
if(D.passageComposerProfiled)
|
||||||
|
- std::printf("[PassageComposerTile] key=(%d,%d,%d,lod%d) worker_ms=%.3f samples=%llu native_field_calls=%llu features=%llu route_candidates=%llu mouth_candidates=%llu boundary_band_queries=%llu queries=%llu passage_iterations=%llu tile_bound_checks=%llu tile_bound_rejects=%llu tile_bound_skipped_iterations=%llu upper_mouth_checks=%llu column_candidates=%llu column_distance_calls=%llu negative_bounds=%llu negative_rejects=%llu positive_bounds=%llu positive_rejects=%llu carve_segment_visits=%llu smooth_carve_joins=%llu support_segment_visits=%llu smooth_support_joins=%llu support_capsule_checks=%llu later_bore_checks=%llu other_capsule_checks=%llu mouth_support_calls=%llu mouth_route_segment_checks=%llu relief_field_calls=%llu smoothstep_evaluations=%llu sweep_segments=%llu sweep_pairs=%llu sweep_outside_pairs=%llu sweep_min_updates=%llu sweep_base_samples=%llu sweep_detail_samples=%llu sweep_grid_samples=%llu sweep_no_influence_samples=%llu sweep_lookups=%llu sweep_cache_hits=%llu sweep_base_hits=%llu sweep_outside_hits=%llu sweep_missing=%llu\n",
|
||||||
|
+ std::printf("[PassageComposerTile] key=(%d,%d,%d,lod%d) worker_ms=%.3f samples=%llu native_field_calls=%llu features=%llu route_candidates=%llu mouth_candidates=%llu boundary_band_queries=%llu boundary_column_evaluations=%llu boundary_column_hits=%llu boundary_column_offgrid=%llu queries=%llu passage_iterations=%llu tile_bound_checks=%llu tile_bound_rejects=%llu tile_bound_skipped_iterations=%llu upper_mouth_checks=%llu column_candidates=%llu column_distance_calls=%llu negative_bounds=%llu negative_rejects=%llu positive_bounds=%llu positive_rejects=%llu carve_segment_visits=%llu smooth_carve_joins=%llu support_segment_visits=%llu smooth_support_joins=%llu support_capsule_checks=%llu later_bore_checks=%llu other_capsule_checks=%llu mouth_support_calls=%llu mouth_route_segment_checks=%llu relief_field_calls=%llu smoothstep_evaluations=%llu sweep_segments=%llu sweep_pairs=%llu sweep_outside_pairs=%llu sweep_min_updates=%llu sweep_base_samples=%llu sweep_detail_samples=%llu sweep_grid_samples=%llu sweep_no_influence_samples=%llu sweep_lookups=%llu sweep_cache_hits=%llu sweep_base_hits=%llu sweep_outside_hits=%llu sweep_missing=%llu\n",
|
||||||
|
Event.Key.Coord.X,Event.Key.Coord.Y,Event.Key.Coord.Z,Event.Key.Level,
|
||||||
|
Event.WorkerMilliseconds,static_cast<unsigned long long>(Event.DensitySamples),
|
||||||
|
static_cast<unsigned long long>(D.fieldCalls),
|
||||||
|
@@ -1546,6 +1546,9 @@ private:
|
||||||
|
static_cast<unsigned long long>(D.passageTileRouteSegmentCandidates),
|
||||||
|
static_cast<unsigned long long>(D.passageTileMouthSegmentCandidates),
|
||||||
|
static_cast<unsigned long long>(D.resolvedBoundaryBandQueries),
|
||||||
|
+ static_cast<unsigned long long>(D.resolvedBoundaryBandColumnEvaluations),
|
||||||
|
+ static_cast<unsigned long long>(D.resolvedBoundaryBandColumnCacheHits),
|
||||||
|
+ static_cast<unsigned long long>(D.resolvedBoundaryBandColumnOffGridFallbacks),
|
||||||
|
static_cast<unsigned long long>(D.passageComposerQueries),
|
||||||
|
static_cast<unsigned long long>(D.passageIterations),
|
||||||
|
static_cast<unsigned long long>(D.passageTileBoundChecks),
|
||||||