research: region completion gate (L123, no production change)
The 18 ms gate is one game-thread tick, not compile cost (0.25 ms). MPSC handoff keeps ownership but still polls per tick; velocity look-ahead didn't help (near -0.8 pt, p99 11.77 > 11.00). Both kept as a patch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
This commit is contained in:
Fr0zkaandClaude Opus 5.5 committed
1 parent
bae1144ee4
commit
5e7e0eb1c1
5 files changed
+1288
No files matched your search
@@ -0,0 +1,9 @@
|
||||
# L123: region completion gate
|
||||
|
||||
Five interleaved 30 m/s UE pairs show the gate is a game-thread poll delay. Core compile wall time is p50 0.251 ms (worker queue plus task p50 0.475 ms); request-to-observe is p50 18.89 ms, p95 21.68 ms. In 95% of traced completions, the next scheduler pass drains the MPSC result and resumes admission in that same pass; a small tail waits 2–4 scheduler frames. This is one tick interval, not separate compile, observe, and admit ticks.
|
||||
|
||||
The worker can safely hand off an immutable result through MPSC with its runtime state and `GenerationEpoch`. The game thread validates both before updating state and admitting tiles. Direct worker-side admission would touch game-thread state unsafely. This queue handoff preserves ownership but still polls once per tick; a game-thread wakeup/continuation is the safe way to try removing that delay. A wakeup was not timed in this round.
|
||||
|
||||
One-step velocity lookahead did not remove the wait (gate p50 19.0 ms). Paired near readiness fell 0.79 percentage points; paired far holes rose 0.14 points. Its worst-run p99 was 11.77 ms (limit 11.00) and worst tick 15.11 ms (limit 14.74). After a 90° turn, 252 of 960 retained ahead keys became stale. At 1.4 m/s it compiled 128 ahead tiles per run; paired near arrival was 2.6 ms faster, while far-clear arrival was 14 ms slower. Keep lookahead off.
|
||||
|
||||
The queue-only runs also missed the p99 limit (worst 11.46 ms), though their worst tick was 14.19 ms. Neither measured option meets the full bar. Headless 30 m/s completed with zero generation failures (82.71% near readiness, 14.21% far holes); this path has no UE region-completion gate. Geometry checks passed 24/24 for Clang/GCC W1/W8/W12, and the fast tier passed.
|
||||
@@ -0,0 +1,115 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Compare L123 exact tile hashes with L104 across strata, compilers, and worker counts."""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[4]
|
||||
OUT = ROOT / "Saved/Linux/l123/hashes"
|
||||
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||
OLD = ROOT / "Saved/Linux/l104/hashes"
|
||||
BINARIES = {
|
||||
"clang": ROOT / "Saved/Linux/l123/bin/VoxelForgeCoreHeadless-l123-clang",
|
||||
"gcc": ROOT / "Saved/Linux/l123/bin/VoxelForgeCoreHeadless-l123-gcc",
|
||||
}
|
||||
CAMERAS = {
|
||||
"sw": ((8.0, 16.0, -372.2626953125), 2850),
|
||||
"fi": ((25.8369140625, 3.19921875, -737.0), 2249),
|
||||
"tn": ((19.166015625, -14.2001953125, -753.3740234375), 2233),
|
||||
"vs": ((-20.0625, -5.068359375, -900.353515625), 2329),
|
||||
}
|
||||
TILE = re.compile(
|
||||
r"^\[PipelineTileHash\] lod=(-?\d+) x=(-?\d+) y=(-?\d+) z=(-?\d+) "
|
||||
r"density=(0x[0-9a-f]+) mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) "
|
||||
r"topology=(0x[0-9a-f]+)$", re.M)
|
||||
WHOLE = re.compile(
|
||||
r"^\[PipelineWholeViewHash\] tiles=(\d+) density=(0x[0-9a-f]+) "
|
||||
r"mesh=(0x[0-9a-f]+) payload=(0x[0-9a-f]+) topology=(0x[0-9a-f]+)$", re.M)
|
||||
STAMP = re.compile(r"^\[VoxelForgeSourceStamp\] sha256=([0-9a-f]{64})$", re.M)
|
||||
|
||||
|
||||
def parse(path: Path) -> tuple[dict, tuple[str, ...], str]:
|
||||
content = path.read_text(encoding="utf-8", errors="replace")
|
||||
tiles = {(int(m.group(1)), int(m.group(2)), int(m.group(3)), int(m.group(4))):
|
||||
m.group(5, 6, 7, 8) for m in TILE.finditer(content)}
|
||||
whole, stamp = WHOLE.search(content), STAMP.search(content)
|
||||
if whole is None or stamp is None or len(tiles) != int(whole.group(1)):
|
||||
raise RuntimeError(f"incomplete hash output: {path}")
|
||||
return tiles, whole.groups(), stamp.group(1)
|
||||
|
||||
|
||||
def run(stratum: str, compiler: str, workers: int) -> dict:
|
||||
OUT.mkdir(parents=True, exist_ok=True)
|
||||
log = OUT / f"{stratum}-{compiler}-w{workers}.log"
|
||||
camera, expected_tiles = CAMERAS[stratum]
|
||||
camera_arg = ",".join(f"{v:.12f}" for v in camera) + ",0,0"
|
||||
command = [
|
||||
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||
"timeout", "1740s", str(BINARIES[compiler]), str(DATA),
|
||||
"--world-assembler-prototype", "--stream-benchmark", "--stream-pipeline",
|
||||
f"--workers={workers}", "--clip-radius=3", "--max-clip-level=8",
|
||||
"--max-level=8", f"--camera={camera_arg}", "--stream-hash-tiles",
|
||||
]
|
||||
started = time.monotonic()
|
||||
with log.open("w", encoding="utf-8") as stream:
|
||||
process = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||
if process.returncode:
|
||||
raise RuntimeError(f"hash process exited {process.returncode}: {log}")
|
||||
tiles, whole, stamp = parse(log)
|
||||
if len(tiles) != expected_tiles:
|
||||
raise RuntimeError(f"expected {expected_tiles} tiles, got {len(tiles)}: {log}")
|
||||
old_tiles, old_whole, _ = parse(OLD / f"{stratum}-{compiler}-w{workers}.log")
|
||||
clang_tiles, clang_whole, _ = parse(OLD / f"{stratum}-clang-w1.log")
|
||||
|
||||
def difference(reference: dict, candidate: dict) -> tuple[int, int]:
|
||||
keys = set(reference) | set(candidate)
|
||||
return sum(reference.get(key) != candidate.get(key) for key in keys), len(keys)
|
||||
|
||||
old_mismatches, old_keys = difference(old_tiles, tiles)
|
||||
cross_mismatches, cross_keys = difference(clang_tiles, tiles)
|
||||
row = {
|
||||
"stratum": stratum, "compiler": compiler, "workers": workers,
|
||||
"source_stamp": stamp, "tiles": len(tiles),
|
||||
"l104_same_cell_tile_mismatches": old_mismatches,
|
||||
"l104_same_cell_key_count": old_keys,
|
||||
"l104_same_cell_whole_hash_match": whole == old_whole,
|
||||
"clang_w1_tile_mismatches": cross_mismatches,
|
||||
"clang_w1_key_count": cross_keys,
|
||||
"clang_w1_whole_hash_match": whole == clang_whole,
|
||||
"wall_s": round(time.monotonic() - started, 3),
|
||||
"log": str(log.relative_to(ROOT)),
|
||||
}
|
||||
print("[L123HashRun] " + json.dumps(row, sort_keys=True), flush=True)
|
||||
row["tile_hashes"] = {"|".join(map(str, key)): value for key, value in tiles.items()}
|
||||
return row
|
||||
|
||||
|
||||
def main() -> int:
|
||||
for path in (*BINARIES.values(), DATA):
|
||||
if not path.is_file():
|
||||
raise SystemExit(f"missing input: {path}")
|
||||
rows = []
|
||||
for stratum in CAMERAS:
|
||||
for compiler in BINARIES:
|
||||
for workers in (1, 8, 12):
|
||||
rows.append(run(stratum, compiler, workers))
|
||||
(OUT / "runs-partial.json").write_text(
|
||||
json.dumps(rows, indent=2) + "\n", encoding="utf-8")
|
||||
failures = [row for row in rows if row["l104_same_cell_tile_mismatches"]
|
||||
or not row["l104_same_cell_whole_hash_match"]
|
||||
or row["clang_w1_tile_mismatches"]
|
||||
or not row["clang_w1_whole_hash_match"]]
|
||||
summary = {"runs": [{key: value for key, value in row.items() if key != "tile_hashes"}
|
||||
for row in rows], "failed_runs": len(failures)}
|
||||
(OUT / "summary.json").write_text(json.dumps(summary, indent=2) + "\n",
|
||||
encoding="utf-8")
|
||||
return 1 if failures else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,77 @@
|
||||
#!/usr/bin/env python3
|
||||
"""One 30 m/s headless movement run for the UE-only region gate experiment."""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import math
|
||||
import re
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[4]
|
||||
DATA = ROOT / "Standalone/data/owner_world.json"
|
||||
BIN = ROOT / "Saved/Linux/l108/bin/VoxelForgeCoreHeadless-l108-after-clang"
|
||||
OUT = ROOT / "Saved/Linux/l123/headless"
|
||||
SAMPLE = re.compile(r"^\[L79MovingSample\] (.*)$", re.M)
|
||||
SUMMARY = re.compile(r"^\[L79MovingSummary\] (.*)$", re.M)
|
||||
FIELDS = re.compile(r"([a-zA-Z0-9_]+)=([^ ]+)")
|
||||
|
||||
|
||||
def p99(values: list[float]) -> float | None:
|
||||
if not values:
|
||||
return None
|
||||
values.sort()
|
||||
return values[max(0, math.ceil(0.99 * len(values)) - 1)]
|
||||
|
||||
|
||||
def main() -> int:
|
||||
OUT.mkdir(parents=True, exist_ok=True)
|
||||
log = OUT / "movement-30mps-r1.log"
|
||||
camera = "19.166015625000,-14.200195312500,-753.374023437500,0,0"
|
||||
command = [
|
||||
"taskset", "-c", "0-11", "prlimit", "--as=6442450944", "--",
|
||||
"timeout", "1740s", str(BIN), str(DATA), "--world-assembler-prototype",
|
||||
"--stream-benchmark", "--stream-pipeline", "--workers=8", "--clip-radius=3",
|
||||
"--max-clip-level=8", "--max-level=8", f"--camera={camera}",
|
||||
"--profile-lod-work", "--stream-scheduler-policy=3",
|
||||
"--moving-camera-speed-mps=30.0", "--moving-camera-duration-s=12",
|
||||
"--moving-camera-dwell-s=5",
|
||||
]
|
||||
started = time.monotonic()
|
||||
with log.open("w", encoding="utf-8") as stream:
|
||||
process = subprocess.run(command, cwd=ROOT, stdout=stream,
|
||||
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||
content = log.read_text(encoding="utf-8", errors="replace")
|
||||
rows = []
|
||||
for match in SAMPLE.finditer(content):
|
||||
row = dict(FIELDS.findall(match.group(1)))
|
||||
for name in ("near_ready_pct", "far_hole_pct", "queue_update_ms"):
|
||||
if name in row:
|
||||
row[name] = float(row[name])
|
||||
row["phase"] = row.get("phase", "move")
|
||||
rows.append(row)
|
||||
summary = dict(FIELDS.findall(SUMMARY.search(content).group(1))) if SUMMARY.search(content) else {}
|
||||
moving = [row for row in rows if row["phase"] == "move"]
|
||||
updates = [row["queue_update_ms"] for row in moving if "queue_update_ms" in row]
|
||||
result = {
|
||||
"exit_code": process.returncode,
|
||||
"wall_s": round(time.monotonic() - started, 3),
|
||||
"moving_samples": len(moving),
|
||||
"near_ready_moving_mean_pct": sum(r["near_ready_pct"] for r in moving) / max(1, len(moving)),
|
||||
"far_holes_moving_mean_pct": sum(r["far_hole_pct"] for r in moving) / max(1, len(moving)),
|
||||
"update_work_p99_ms": p99(updates),
|
||||
"update_work_max_ms": max(updates, default=None),
|
||||
"generation_failures": int(summary.get("generation_failures", -1)),
|
||||
"log": str(log.relative_to(ROOT)),
|
||||
}
|
||||
(OUT / "summary.json").write_text(json.dumps(result, indent=2) + "\n",
|
||||
encoding="utf-8")
|
||||
print("[L123Headless] " + json.dumps(result, sort_keys=True), flush=True)
|
||||
if process.returncode or not moving or result["generation_failures"] != 0:
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,262 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Five interleaved 30 m/s UE pairs for the Core region lookahead experiment."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
import statistics
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[4]
|
||||
HOST = Path("/home/jahni/VFHost")
|
||||
EDITOR = Path("/home/jahni/UnrealEngine/Engine/Binaries/Linux/UnrealEditor-Cmd")
|
||||
SPAWN_TN_M = (19.166015625, -14.2001953125, -753.3740234375)
|
||||
SPAWN_FI_M = (25.8369140625, 3.19921875, -737.0)
|
||||
|
||||
|
||||
def percentile(values: list[float], fraction: float) -> float | None:
|
||||
if not values:
|
||||
return None
|
||||
ordered = sorted(values)
|
||||
return ordered[max(0, min(len(ordered) - 1, math.ceil(len(ordered) * fraction) - 1))]
|
||||
|
||||
|
||||
def read_events(path: Path) -> list[dict]:
|
||||
rows = []
|
||||
if path.is_file():
|
||||
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||
if not line.startswith("{"):
|
||||
continue
|
||||
try:
|
||||
rows.append(json.loads(line))
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
return rows
|
||||
|
||||
|
||||
def run(scenario: str, condition: str, repetition: int, turn: bool = False) -> dict:
|
||||
lookahead = int(condition in ("lookahead", "turn_lookahead"))
|
||||
if scenario == "movement":
|
||||
speed, start, duration, exit_s, slot, spawn = (
|
||||
3000.0, 6.0, 12.0, 23.0, 0, SPAWN_TN_M)
|
||||
else:
|
||||
speed, start, duration, exit_s, slot, spawn = (
|
||||
140.0, 20.0, 12.0, 37.0, 1, SPAWN_FI_M)
|
||||
|
||||
label = f"{scenario}-{condition}-r{repetition}"
|
||||
output_group = "turn" if turn else scenario
|
||||
out = ROOT / "Saved/Linux/l123" / output_group / condition / "unreal"
|
||||
host_out = HOST / "Plugins/VoxelForge/Saved/Linux/l123" / output_group / condition
|
||||
host_saved = HOST / "Saved/Linux/l123"
|
||||
trace = out / f"{label}.trace.jsonl"
|
||||
abslog = out / f"{label}.log"
|
||||
stdout = out / f"{label}.stdout.log"
|
||||
export = host_out / "output" / label
|
||||
userdir = host_saved / "user" / label
|
||||
zendata = host_saved / "zen" / label
|
||||
ddc = host_saved / "DDC"
|
||||
for path in (out, export, userdir, zendata, ddc):
|
||||
path.mkdir(parents=True, exist_ok=True)
|
||||
trace.unlink(missing_ok=True)
|
||||
spawn_cm = ",".join(f"{value * 100.0:.3f}" for value in spawn)
|
||||
command = [
|
||||
str(EDITOR), str(HOST / "VFHost.uproject"), "-nullrhi", "-unattended",
|
||||
"-nop4", "-nosplash", "-NoSound", "-corelimit=12", "-NoZenAutoLaunch",
|
||||
"-notraceserver", "-nocrashreports", "-DisablePlugins=AndroidFileServer",
|
||||
"-ddc=NoZenLocalFallback", f"-userdir={userdir}", f"-ZenDataPath={zendata}",
|
||||
f"-LocalDataCachePath={ddc}", "-run=VoxelForgeExplore", "-seed=0",
|
||||
"-archetype=TunnelNetwork", f"-slot={slot}", "-modes=export", "-gameconfig=1",
|
||||
"-coreworldtest=1", "-ownerfourstrateplan=1", "-corevoxelsizecm=25",
|
||||
"-coreislandtest=0", "-corefallbacktest=0", "-coreofftest=0", "-coreplantest=0",
|
||||
"-settingsref=/Game/VoxelForge/DA_Settings.DA_Settings", "-corehybridmesher=0",
|
||||
f"-out={export}", "-voxel.StreamingCoarseFirst=1",
|
||||
"-voxel.StreamingDynamicNearReserve=1", "-voxel.StreamingLegacyPendingResultCap=0",
|
||||
"-voxel.CoreRegionConcurrentAdmission=1", "-voxel.StreamingResultBacklogCap=256",
|
||||
"-voxel.StreamingReadyQueueDepth=8", "-voxel.StreamingBackgroundWorkerCap=1",
|
||||
"-voxel.CoreColumnCacheDirectMapped=0", "-voxel.StreamingFarApplySupplement=1",
|
||||
"-voxel.CoreMCGradientMemoization=1", "-voxel.StreamingCollisionLookahead=1",
|
||||
f"-voxel.CoreRegionLookahead={lookahead}", "-voxel.TestMove=1",
|
||||
f"-voxel.TestMoveSpeedCmPerSecond={speed:.1f}",
|
||||
f"-voxel.TestMoveStartSeconds={start:.1f}",
|
||||
f"-voxel.TestMoveDurationSeconds={duration:.1f}",
|
||||
f"-voxel.TestExitSeconds={exit_s:.1f}", "-voxel.TestMoveDirection=1,0,0",
|
||||
"-voxel.StreamingSchedulerPolicy=3", "-voxel.TestExitOnSteady=0",
|
||||
"-coreworldstreamtrace=1", f"-voxel.StartupTraceSpawnCm={spawn_cm}",
|
||||
"-voxel.StartupTraceUntilEndPlay=1", "-voxel.StartupTraceThroughCrossing=1",
|
||||
"-voxel.StartupTraceMaxTiles=16384", f"-voxel.StartupTraceFile={trace}",
|
||||
f"-abslog={abslog}",
|
||||
]
|
||||
if turn:
|
||||
command.extend(["-voxel.TestMoveTurnAfterSeconds=6.0",
|
||||
"-voxel.TestMoveTurnDirection=0,1,0"])
|
||||
started = time.monotonic()
|
||||
wrapped = ["taskset", "-c", "0-11", "timeout", "1740s", *command]
|
||||
with stdout.open("w", encoding="utf-8") as stream:
|
||||
process = subprocess.run(wrapped, cwd=ROOT, stdout=stream,
|
||||
stderr=subprocess.STDOUT, check=False, timeout=1740)
|
||||
events = read_events(trace)
|
||||
samples = [event for event in events if event.get("type") == "streaming_sample"]
|
||||
moving = [event for event in samples if float(event.get("speed_cm_s", 0.0)) >= 50.0]
|
||||
stop_event = next((event for event in events
|
||||
if event.get("type") == "streaming_test_move_stopped"), {})
|
||||
stop_s = float(stop_event.get("t_s", 0.0))
|
||||
move_start = min((float(event.get("t_s", 0.0)) for event in moving), default=float("inf"))
|
||||
dwell = [event for event in samples if float(event.get("t_s", 0.0)) >= stop_s
|
||||
and float(event.get("speed_cm_s", 0.0)) < 50.0]
|
||||
backend = next((event for event in events if event.get("type") == "worker_backend_config"), {})
|
||||
cap = next((event for event in events if event.get("type") == "streaming_worker_cap_readback"), {})
|
||||
if process.returncode or len(moving) < 40 or len(dwell) < 16:
|
||||
raise RuntimeError(f"incomplete UE run {label}: exit={process.returncode}, "
|
||||
f"move={len(moving)}, dwell={len(dwell)}, trace={trace}")
|
||||
if (int(backend.get("core_region_lookahead", -1)) != lookahead
|
||||
or int(backend.get("streaming_far_apply_supplement", -1)) != 1
|
||||
or int(cap.get("streaming_background_worker_cap", -1)) != 1
|
||||
or int(backend.get("effective_tile_slots", -1)) != 8):
|
||||
raise RuntimeError(f"configuration readback mismatch: {backend}; {cap}")
|
||||
|
||||
moving_rows = []
|
||||
for event in moving:
|
||||
desired = event.get("desired_by_lod", [0] * 9)
|
||||
resident = event.get("resident_by_lod", [0] * 9)
|
||||
covered = event.get("covered_by_lod", resident)
|
||||
moving_rows.append({
|
||||
"near_ready_pct": 100.0 * sum(resident[:3]) / max(1, sum(desired[:3])),
|
||||
"lod0_ready_pct": 100.0 * resident[0] / max(1, desired[0]),
|
||||
"lod1_ready_pct": 100.0 * resident[1] / max(1, desired[1]),
|
||||
"lod2_ready_pct": 100.0 * resident[2] / max(1, desired[2]),
|
||||
"far_hole_pct": 100.0 * (sum(desired[3:]) - sum(covered[3:]))
|
||||
/ max(1, sum(desired[3:])),
|
||||
})
|
||||
work = [event for event in events if event.get("type") == "game_thread_work_frame"
|
||||
and move_start <= float(event.get("t_s", 0.0)) < stop_s]
|
||||
ticks = [float(event.get("tick_ms", 0.0)) for event in work]
|
||||
updates = [float(event.get("update_chunks_ms", 0.0)) for event in work]
|
||||
arrival = [event for event in samples if float(event.get("t_s", 0.0)) < start
|
||||
and float(event.get("speed_cm_s", 0.0)) < 50.0]
|
||||
|
||||
def settled(event: dict, end: int) -> bool:
|
||||
desired = event.get("desired_by_lod", [0] * 9)
|
||||
actual = event.get("settled_by_lod", [0] * 9)
|
||||
return sum(desired[:end]) > 0 and sum(actual[:end]) >= sum(desired[:end])
|
||||
|
||||
def far_holes(event: dict) -> float | None:
|
||||
desired = event.get("desired_by_lod", [0] * 9)[3:]
|
||||
covered = event.get("covered_by_lod", [0] * 9)[3:]
|
||||
return 100.0 * (sum(desired) - sum(covered)) / sum(desired) if sum(desired) else None
|
||||
|
||||
near_settled = next((event for event in arrival if settled(event, 3)), None)
|
||||
whole_settled = next((event for event in arrival if settled(event, 9)), None)
|
||||
far_clear = next((event for event in arrival if far_holes(event) == 0.0), None)
|
||||
compile_done = [event for event in events if event.get("type") == "core_region_compile_complete"
|
||||
and move_start <= float(event.get("t_s", 0.0)) < stop_s]
|
||||
compile_requests = [event for event in events
|
||||
if event.get("type") == "core_region_compile_request"
|
||||
and move_start <= float(event.get("t_s", 0.0)) < stop_s]
|
||||
lookahead_turns = [event for event in events
|
||||
if event.get("type") == "core_region_lookahead_turn"]
|
||||
row = {
|
||||
"scenario": scenario, "condition": condition, "repetition": repetition,
|
||||
"speed_mps": speed / 100.0, "core_region_lookahead": lookahead,
|
||||
"exit_code": process.returncode, "wall_s": round(time.monotonic() - started, 3),
|
||||
"moving_samples": len(moving), "dwell_samples": len(dwell),
|
||||
"near_ready_moving_mean_pct": statistics.mean(r["near_ready_pct"] for r in moving_rows),
|
||||
"lod0_ready_moving_mean_pct": statistics.mean(r["lod0_ready_pct"] for r in moving_rows),
|
||||
"lod1_ready_moving_mean_pct": statistics.mean(r["lod1_ready_pct"] for r in moving_rows),
|
||||
"lod2_ready_moving_mean_pct": statistics.mean(r["lod2_ready_pct"] for r in moving_rows),
|
||||
"far_holes_moving_mean_pct": statistics.mean(r["far_hole_pct"] for r in moving_rows),
|
||||
"game_thread_tick_p99_ms": percentile(ticks, 0.99),
|
||||
"game_thread_tick_max_ms": max(ticks, default=None),
|
||||
"stream_update_p99_ms": percentile(updates, 0.99),
|
||||
"stream_update_max_ms": max(updates, default=None),
|
||||
"lod02_settled_t_s": float(near_settled["t_s"]) if near_settled else None,
|
||||
"whole_view_settled_t_s": float(whole_settled["t_s"]) if whole_settled else None,
|
||||
"far_clear_t_s": float(far_clear["t_s"]) if far_clear else None,
|
||||
"far_holes_at_pre_move_pct": far_holes(arrival[-1]) if arrival else None,
|
||||
"pending_gate_p50_ms": percentile(
|
||||
[float(e.get("pending_gate_elapsed_ms", 0.0)) for e in compile_done], 0.50),
|
||||
"compile_queue_p95_ms": percentile(
|
||||
[float(e.get("compile_queue_ms", 0.0)) for e in compile_done], 0.95),
|
||||
"compile_task_p95_ms": percentile(
|
||||
[float(e.get("compile_task_ms", 0.0)) for e in compile_done], 0.95),
|
||||
"compiled_tiles_total": sum(int(e.get("compiled_tiles", 0)) for e in compile_done),
|
||||
"lookahead_tiles_peak": max((int(e.get("lookahead_region_tiles", 0))
|
||||
for e in compile_requests), default=0),
|
||||
"wasted_prefetch_tiles": sum(int(e.get("wasted_prefetch_tiles", 0))
|
||||
for e in compile_done),
|
||||
"lookahead_turn_events": lookahead_turns,
|
||||
"compile_completion_count": len(compile_done),
|
||||
"trace": str(trace.relative_to(ROOT)), "log": str(abslog.relative_to(ROOT)),
|
||||
}
|
||||
with (ROOT / "Saved/Linux/l123" / "runs.jsonl").open("a", encoding="utf-8") as stream:
|
||||
stream.write(json.dumps(row, sort_keys=True) + "\n")
|
||||
print("[L123Pair] " + json.dumps(row, sort_keys=True), flush=True)
|
||||
return row
|
||||
|
||||
|
||||
def pairs_for(runs: list[dict], scenario: str) -> list[dict]:
|
||||
pairs = []
|
||||
for repetition in range(1, 6):
|
||||
baseline = next(row for row in runs if row["scenario"] == scenario
|
||||
and row["condition"] == "baseline" and row["repetition"] == repetition)
|
||||
candidate = next(row for row in runs if row["scenario"] == scenario
|
||||
and row["condition"] == "lookahead" and row["repetition"] == repetition)
|
||||
pairs.append({"repetition": repetition,
|
||||
"near_ready_delta_pct": candidate["near_ready_moving_mean_pct"]
|
||||
- baseline["near_ready_moving_mean_pct"],
|
||||
"far_holes_delta_pct": candidate["far_holes_moving_mean_pct"]
|
||||
- baseline["far_holes_moving_mean_pct"],
|
||||
"tick_p99_delta_ms": candidate["game_thread_tick_p99_ms"]
|
||||
- baseline["game_thread_tick_p99_ms"],
|
||||
"tick_max_delta_ms": candidate["game_thread_tick_max_ms"]
|
||||
- baseline["game_thread_tick_max_ms"]})
|
||||
return pairs
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--scenario", choices=("movement", "arrival"), required=True)
|
||||
parser.add_argument("--turn-diagnostic", action="store_true")
|
||||
args = parser.parse_args()
|
||||
if not EDITOR.is_file():
|
||||
raise SystemExit(f"missing Unreal editor: {EDITOR}")
|
||||
root_out = ROOT / "Saved/Linux/l123" / args.scenario
|
||||
root_out.mkdir(parents=True, exist_ok=True)
|
||||
(ROOT / "Saved/Linux/l123").mkdir(parents=True, exist_ok=True)
|
||||
if args.turn_diagnostic:
|
||||
row = run("movement", "turn_lookahead", 1, turn=True)
|
||||
turn_out = ROOT / "Saved/Linux/l123/turn"
|
||||
turn_out.mkdir(parents=True, exist_ok=True)
|
||||
(turn_out / "summary.json").write_text(
|
||||
json.dumps({"run": row}, indent=2) + "\n", encoding="utf-8")
|
||||
return 0
|
||||
runs = []
|
||||
for repetition in range(1, 6):
|
||||
order = ("baseline", "lookahead") if repetition % 2 else ("lookahead", "baseline")
|
||||
for condition in order:
|
||||
runs.append(run(args.scenario, condition, repetition))
|
||||
(root_out / "runs-partial.json").write_text(
|
||||
json.dumps(runs, indent=2) + "\n", encoding="utf-8")
|
||||
pairs = pairs_for(runs, args.scenario)
|
||||
summary = {
|
||||
"design": "Compile/retain the current near region set plus one step along velocity; "
|
||||
"return completion through an Epoch-tagged MPSC queue.",
|
||||
"scenario": args.scenario, "runs": runs, "pairs": pairs,
|
||||
"paired_median_near_ready_delta_pct": statistics.median(
|
||||
pair["near_ready_delta_pct"] for pair in pairs),
|
||||
"paired_median_far_holes_delta_pct": statistics.median(
|
||||
pair["far_holes_delta_pct"] for pair in pairs),
|
||||
"paired_median_tick_p99_delta_ms": statistics.median(
|
||||
pair["tick_p99_delta_ms"] for pair in pairs),
|
||||
"paired_median_tick_max_delta_ms": statistics.median(
|
||||
pair["tick_max_delta_ms"] for pair in pairs),
|
||||
}
|
||||
(root_out / "summary.json").write_text(json.dumps(summary, indent=2) + "\n",
|
||||
encoding="utf-8")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,825 @@
|
||||
diff --git a/Source/VoxelForge/Private/VoxelWorld.cpp b/Source/VoxelForge/Private/VoxelWorld.cpp
|
||||
index 42ac6f5..2c7046b 100644
|
||||
--- a/Source/VoxelForge/Private/VoxelWorld.cpp
|
||||
+++ b/Source/VoxelForge/Private/VoxelWorld.cpp
|
||||
@@ -195,10 +195,13 @@ public:
|
||||
std::atomic<bool> bShuttingDown{false};
|
||||
std::atomic<uint64> NextEditSequence{1};
|
||||
std::atomic<uint64> RegionCompileStartCycles{0};
|
||||
+ std::atomic<uint64> RegionCompileWorkerStartCycles{0};
|
||||
std::atomic<uint64> RegionCompileEndCycles{0};
|
||||
std::atomic<int32> RegionCompileTileCount{0};
|
||||
std::atomic<int32> RegionCompileSerial{0};
|
||||
+ std::atomic<uint32> RegionCompileWorkerThreadId{0};
|
||||
std::atomic<double> RegionCompileWorkerSeconds{0.0};
|
||||
+ std::atomic<double> RegionCompileWallSeconds{0.0};
|
||||
int32 FullResolutionLodCount = 0;
|
||||
FString InitialFailureReason;
|
||||
};
|
||||
@@ -262,6 +265,12 @@ namespace
|
||||
GVoxelForgeCoreRegionConcurrentAdmission,
|
||||
TEXT("Allow tiles independent of a pending Core region compile to proceed; 0 restores the global streaming stop."));
|
||||
|
||||
+ int32 GVoxelForgeCoreRegionLookahead = 0;
|
||||
+ FAutoConsoleVariableRef CVarVoxelForgeCoreRegionLookahead(
|
||||
+ TEXT("voxel.CoreRegionLookahead"),
|
||||
+ GVoxelForgeCoreRegionLookahead,
|
||||
+ TEXT("Retain and compile the next near region set along velocity; 0 disables region lookahead."));
|
||||
+
|
||||
int32 GVoxelForgeStreamingResultBacklogCap = 256;
|
||||
FAutoConsoleVariableRef CVarVoxelForgeStreamingResultBacklogCap(
|
||||
TEXT("voxel.StreamingResultBacklogCap"),
|
||||
@@ -1745,6 +1754,10 @@ void AVoxelWorld::EndPlay(const EEndPlayReason::Type EndPlayReason)
|
||||
{
|
||||
if (Discard.bObsolete) { ++ObsoleteTileAbortCount; }
|
||||
}
|
||||
+ FCoreRegionCompileResult DiscardedRegionCompile;
|
||||
+ while (CoreRegionCompileResults.Dequeue(DiscardedRegionCompile))
|
||||
+ {
|
||||
+ }
|
||||
UE_LOG(LogTemp, Display, TEXT("[VoxelWorld] EndPlay: workers drained in %.6fs; obsolete_tile_aborts=%d"),
|
||||
FPlatformTime::Seconds() - EndPlayStartSeconds, ObsoleteTileAbortCount);
|
||||
VoxelForgeCoreState.Reset();
|
||||
@@ -2093,6 +2106,11 @@ void AVoxelWorld::BeginPlay()
|
||||
PendingCoreRegionTiles.Reset();
|
||||
PendingCoreRegionTileSet.Reset();
|
||||
PendingCoreRetainedTiles.Reset();
|
||||
+ CoreRegionRetainedDesiredTiles.Reset();
|
||||
+ CoreRegionLookaheadTiles.Reset();
|
||||
+ LastCoreRegionLookaheadCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
|
||||
+ LastCoreRegionLookaheadStep = FIntVector::ZeroValue;
|
||||
+ LastCoreRegionLookaheadGenerationEpoch = GenerationEpoch;
|
||||
CoreRegionCompileTilesInFlight.Reset();
|
||||
LastCoreRegionRequestCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
|
||||
PendingCoreRegionRequestCenter = FIntVector::ZeroValue;
|
||||
@@ -2149,6 +2167,7 @@ void AVoxelWorld::BeginPlay()
|
||||
int32 CommandLineSampleStacks = GVoxelForgeSampleStacks;
|
||||
int32 CommandLineCollisionGate = GVoxelForgeCollisionGate;
|
||||
int32 CommandLineCoreRegionConcurrentAdmission = GVoxelForgeCoreRegionConcurrentAdmission;
|
||||
+ int32 CommandLineCoreRegionLookahead = GVoxelForgeCoreRegionLookahead;
|
||||
int32 CommandLineStreamingCollisionLookahead = GVoxelForgeStreamingCollisionLookahead;
|
||||
int32 CommandLineStreamingResultBacklogCap = GVoxelForgeStreamingResultBacklogCap;
|
||||
int32 CommandLineStreamingReadyQueueDepth = GVoxelForgeStreamingReadyQueueDepth;
|
||||
@@ -2257,6 +2276,11 @@ void AVoxelWorld::BeginPlay()
|
||||
GVoxelForgeCoreRegionConcurrentAdmission =
|
||||
CommandLineCoreRegionConcurrentAdmission != 0 ? 1 : 0;
|
||||
}
|
||||
+ if (FParse::Value(FCommandLine::Get(), TEXT("voxel.CoreRegionLookahead="),
|
||||
+ CommandLineCoreRegionLookahead))
|
||||
+ {
|
||||
+ GVoxelForgeCoreRegionLookahead = CommandLineCoreRegionLookahead != 0 ? 1 : 0;
|
||||
+ }
|
||||
if (FParse::Value(FCommandLine::Get(), TEXT("voxel.StreamingResultBacklogCap="),
|
||||
CommandLineStreamingResultBacklogCap))
|
||||
{
|
||||
@@ -2747,6 +2771,7 @@ void AVoxelWorld::BeginPlay()
|
||||
"\"streaming_coarse_first\":%d,\"streaming_dynamic_near_reserve\":%d,"
|
||||
"\"streaming_legacy_pending_result_cap\":%d,"
|
||||
"\"streaming_collision_lookahead\":%d,\"core_region_concurrent_admission\":%d,"
|
||||
+ "\"core_region_lookahead\":%d,"
|
||||
"\"collision_gate\":%d,"
|
||||
"\"streaming_result_backlog_cap\":%d,\"streaming_startup_apply_boost\":%d,"
|
||||
"\"streaming_far_apply_supplement\":%d,"
|
||||
@@ -2769,6 +2794,7 @@ void AVoxelWorld::BeginPlay()
|
||||
GVoxelForgeStreamingCoarseFirst, GVoxelForgeStreamingDynamicNearReserve,
|
||||
GVoxelForgeStreamingLegacyPendingResultCap,
|
||||
GVoxelForgeStreamingCollisionLookahead, GVoxelForgeCoreRegionConcurrentAdmission,
|
||||
+ GVoxelForgeCoreRegionLookahead,
|
||||
GVoxelForgeCollisionGate,
|
||||
GVoxelForgeStreamingResultBacklogCap, GVoxelForgeStreamingStartupApplyBoost,
|
||||
GVoxelForgeStreamingFarApplySupplement,
|
||||
@@ -3246,11 +3272,13 @@ void AVoxelWorld::ConfigureHeadlessStreamingTest()
|
||||
float Speed = 800.0f;
|
||||
float StartSeconds = 20.0f;
|
||||
float MoveDurationSeconds = 0.0f;
|
||||
+ float MoveTurnAfterSeconds = 0.0f;
|
||||
float ExitSeconds = 0.0f;
|
||||
int32 ExitOnSteadyValue = 0;
|
||||
FParse::Value(CommandLine, TEXT("voxel.TestMoveSpeedCmPerSecond="), Speed);
|
||||
FParse::Value(CommandLine, TEXT("voxel.TestMoveStartSeconds="), StartSeconds);
|
||||
FParse::Value(CommandLine, TEXT("voxel.TestMoveDurationSeconds="), MoveDurationSeconds);
|
||||
+ FParse::Value(CommandLine, TEXT("voxel.TestMoveTurnAfterSeconds="), MoveTurnAfterSeconds);
|
||||
FParse::Value(CommandLine, TEXT("voxel.TestExitSeconds="), ExitSeconds);
|
||||
FParse::Value(CommandLine, TEXT("voxel.TestTeleportAfterSeconds="),
|
||||
HeadlessStreamingTestTeleportAtSeconds);
|
||||
@@ -3418,9 +3446,44 @@ void AVoxelWorld::ConfigureHeadlessStreamingTest()
|
||||
{
|
||||
HeadlessStreamingTestDirection = FVector::XAxisVector;
|
||||
}
|
||||
+ FString TurnDirectionText;
|
||||
+ const FString TurnDirectionKey(TEXT("voxel.TestMoveTurnDirection="));
|
||||
+ const int32 TurnDirectionKeyIndex = CommandLineText.Find(
|
||||
+ TurnDirectionKey, ESearchCase::IgnoreCase, ESearchDir::FromStart);
|
||||
+ if (TurnDirectionKeyIndex != INDEX_NONE)
|
||||
+ {
|
||||
+ TurnDirectionText = CommandLineText.Mid(
|
||||
+ TurnDirectionKeyIndex + TurnDirectionKey.Len());
|
||||
+ const int32 DirectionEnd = TurnDirectionText.Find(
|
||||
+ TEXT(" "), ESearchCase::CaseSensitive, ESearchDir::FromStart);
|
||||
+ if (DirectionEnd != INDEX_NONE)
|
||||
+ {
|
||||
+ TurnDirectionText.LeftInline(DirectionEnd, EAllowShrinking::No);
|
||||
+ }
|
||||
+ TurnDirectionText.TrimStartAndEndInline();
|
||||
+ TurnDirectionText.RemoveFromStart(TEXT("\""));
|
||||
+ TurnDirectionText.RemoveFromEnd(TEXT("\""));
|
||||
+ TArray<FString> Components;
|
||||
+ TurnDirectionText.ParseIntoArray(Components, TEXT(","), true);
|
||||
+ if (Components.Num() == 3)
|
||||
+ {
|
||||
+ HeadlessStreamingTestTurnDirection = FVector(
|
||||
+ FCString::Atof(*Components[0]),
|
||||
+ FCString::Atof(*Components[1]),
|
||||
+ FCString::Atof(*Components[2]));
|
||||
+ }
|
||||
+ }
|
||||
+ HeadlessStreamingTestTurnDirection =
|
||||
+ HeadlessStreamingTestTurnDirection.GetSafeNormal();
|
||||
+ if (HeadlessStreamingTestTurnDirection.IsNearlyZero())
|
||||
+ {
|
||||
+ HeadlessStreamingTestTurnDirection = FVector::YAxisVector;
|
||||
+ }
|
||||
HeadlessStreamingTestSpeedCmPerSecond = FMath::Max(Speed, 0.0f);
|
||||
HeadlessStreamingTestStartSeconds = FMath::Max(StartSeconds, 0.0f);
|
||||
HeadlessStreamingTestMoveDurationSeconds = FMath::Max(MoveDurationSeconds, 0.0f);
|
||||
+ HeadlessStreamingTestTurnAfterSeconds = FMath::Max(MoveTurnAfterSeconds, 0.0f);
|
||||
+ bHeadlessStreamingTestTurnDone = false;
|
||||
bHeadlessStreamingTestMovement = HeadlessStreamingTestSpeedCmPerSecond > 0.0f;
|
||||
}
|
||||
|
||||
@@ -3431,15 +3494,20 @@ void AVoxelWorld::ConfigureHeadlessStreamingTest()
|
||||
HeadlessStreamingTestBeginSeconds = FPlatformTime::Seconds();
|
||||
UE_LOG(LogTemp, Display,
|
||||
TEXT("[VoxelForgeStreamingTest] clean=1 movement=%d speed_cm_s=%.3f start_s=%.3f "
|
||||
- "move_duration_s=%.3f exit_s=%.3f direction=(%.3f,%.3f,%.3f)"),
|
||||
+ "move_duration_s=%.3f turn_after_s=%.3f exit_s=%.3f "
|
||||
+ "direction=(%.3f,%.3f,%.3f) turn_direction=(%.3f,%.3f,%.3f)"),
|
||||
bHeadlessStreamingTestMovement ? 1 : 0,
|
||||
HeadlessStreamingTestSpeedCmPerSecond,
|
||||
HeadlessStreamingTestStartSeconds,
|
||||
HeadlessStreamingTestMoveDurationSeconds,
|
||||
+ HeadlessStreamingTestTurnAfterSeconds,
|
||||
HeadlessStreamingTestExitSeconds,
|
||||
HeadlessStreamingTestDirection.X,
|
||||
HeadlessStreamingTestDirection.Y,
|
||||
- HeadlessStreamingTestDirection.Z);
|
||||
+ HeadlessStreamingTestDirection.Z,
|
||||
+ HeadlessStreamingTestTurnDirection.X,
|
||||
+ HeadlessStreamingTestTurnDirection.Y,
|
||||
+ HeadlessStreamingTestTurnDirection.Z);
|
||||
}
|
||||
|
||||
int32 CollisionGateStressValue = 0;
|
||||
@@ -4503,6 +4571,32 @@ void AVoxelWorld::AdvanceHeadlessStreamingTest(
|
||||
return;
|
||||
}
|
||||
|
||||
+ if (!bHeadlessStreamingTestTurnDone && HeadlessStreamingTestTurnAfterSeconds > 0.0f
|
||||
+ && ElapsedSeconds - static_cast<double>(HeadlessStreamingTestStartSeconds)
|
||||
+ >= static_cast<double>(HeadlessStreamingTestTurnAfterSeconds))
|
||||
+ {
|
||||
+ bHeadlessStreamingTestTurnDone = true;
|
||||
+ const FVector PreviousDirection = HeadlessStreamingTestDirection;
|
||||
+ HeadlessStreamingTestDirection = HeadlessStreamingTestTurnDirection;
|
||||
+ if (VoxelForgeStartupTrace::IsActive())
|
||||
+ {
|
||||
+ VoxelForgeStartupTrace::RecordEvent(TEXT("streaming_test_move_turned"),
|
||||
+ FString::Printf(
|
||||
+ TEXT("\"elapsed_s\":%s,\"from\":[%.0f,%.0f,%.0f],"
|
||||
+ "\"to\":[%.0f,%.0f,%.0f]"),
|
||||
+ *FString::SanitizeFloat(ElapsedSeconds),
|
||||
+ PreviousDirection.X, PreviousDirection.Y, PreviousDirection.Z,
|
||||
+ HeadlessStreamingTestDirection.X, HeadlessStreamingTestDirection.Y,
|
||||
+ HeadlessStreamingTestDirection.Z));
|
||||
+ }
|
||||
+ UE_LOG(LogTemp, Display,
|
||||
+ TEXT("[VoxelForgeStreamingTest] turn=1 elapsed_s=%.3f from=(%.0f,%.0f,%.0f) "
|
||||
+ "to=(%.0f,%.0f,%.0f)"),
|
||||
+ ElapsedSeconds, PreviousDirection.X, PreviousDirection.Y, PreviousDirection.Z,
|
||||
+ HeadlessStreamingTestDirection.X, HeadlessStreamingTestDirection.Y,
|
||||
+ HeadlessStreamingTestDirection.Z);
|
||||
+ }
|
||||
+
|
||||
if (HeadlessStreamingTestMoveDurationSeconds > 0.0f
|
||||
&& ElapsedSeconds - static_cast<double>(HeadlessStreamingTestStartSeconds)
|
||||
>= static_cast<double>(HeadlessStreamingTestMoveDurationSeconds))
|
||||
@@ -11258,16 +11352,113 @@ void AVoxelWorld::BuildCoreDesiredTiles(const FIntVector& Center, const FVector&
|
||||
}
|
||||
BuildDesiredTransitionIndex();
|
||||
|
||||
- bool bRetainedSetChanged = false;
|
||||
- int32 RetainedTileCount = 0;
|
||||
+ FIntVector RegionLookaheadStep = FIntVector::ZeroValue;
|
||||
+ if (GVoxelForgeCoreRegionLookahead != 0 && PlayerHeading.Size() >= 50.0f)
|
||||
+ {
|
||||
+ RegionLookaheadStep = GetStreamingHeadingStep(PlayerHeading, PlayerPawn);
|
||||
+ }
|
||||
+ const bool bRegionLookaheadTurn = RegionLookaheadStep != FIntVector::ZeroValue
|
||||
+ && LastCoreRegionLookaheadStep != FIntVector::ZeroValue
|
||||
+ && LastCoreRegionLookaheadStep != RegionLookaheadStep
|
||||
+ && LastCoreRegionLookaheadGenerationEpoch == GenerationEpoch;
|
||||
+ const FIntVector PreviousRegionLookaheadStep = LastCoreRegionLookaheadStep;
|
||||
+ const TArray<FVoxelTileKey> PreviousRegionLookaheadTiles = bRegionLookaheadTurn
|
||||
+ ? CoreRegionLookaheadTiles : TArray<FVoxelTileKey>();
|
||||
+ if (RegionLookaheadStep != FIntVector::ZeroValue
|
||||
+ && (LastCoreRegionLookaheadCenter != Center
|
||||
+ || LastCoreRegionLookaheadStep != RegionLookaheadStep
|
||||
+ || LastCoreRegionLookaheadGenerationEpoch != GenerationEpoch))
|
||||
+ {
|
||||
+ const FVector AheadCameraVoxels = CameraVoxels + FVector(
|
||||
+ static_cast<float>(RegionLookaheadStep.X),
|
||||
+ static_cast<float>(RegionLookaheadStep.Y),
|
||||
+ static_cast<float>(RegionLookaheadStep.Z)) * static_cast<float>(CHUNK_SIZE);
|
||||
+ std::vector<VoxelForgeCore::TileKey> AheadDesired;
|
||||
+ {
|
||||
+ FReadScopeLock Lock(State->WorldLock);
|
||||
+ if (State->World)
|
||||
+ {
|
||||
+ AheadDesired = State->World->desiredTiles(
|
||||
+ {AheadCameraVoxels.X, AheadCameraVoxels.Y, AheadCameraVoxels.Z});
|
||||
+ }
|
||||
+ }
|
||||
+ CoreRegionLookaheadTiles.Reset();
|
||||
+ TSet<FVoxelTileKey> AheadSet;
|
||||
+ for (const VoxelForgeCore::TileKey& Key : AheadDesired)
|
||||
+ {
|
||||
+ if (Key.lod >= State->FullResolutionLodCount) continue;
|
||||
+ const FVoxelTileKey RuntimeKey(
|
||||
+ FIntVector(Key.coordinate.x, Key.coordinate.y, Key.coordinate.z), Key.lod);
|
||||
+ if (!AheadSet.Contains(RuntimeKey))
|
||||
+ {
|
||||
+ AheadSet.Add(RuntimeKey);
|
||||
+ CoreRegionLookaheadTiles.Add(RuntimeKey);
|
||||
+ }
|
||||
+ }
|
||||
+ if (bRegionLookaheadTurn)
|
||||
+ {
|
||||
+ TSet<FVoxelTileKey> ActualSet;
|
||||
+ for (const FVoxelTileKey& Key : DesiredSorted) ActualSet.Add(Key);
|
||||
+ int32 StaleRetainedLookaheadTiles = 0;
|
||||
+ {
|
||||
+ FScopeLock RetainedLock(&State->RetainedTilesLock);
|
||||
+ for (const FVoxelTileKey& Key : PreviousRegionLookaheadTiles)
|
||||
+ {
|
||||
+ if (ActualSet.Contains(Key) || AheadSet.Contains(Key)) continue;
|
||||
+ const VoxelForgeCore::TileKey CoreKey{
|
||||
+ {Key.Coord.X, Key.Coord.Y, Key.Coord.Z}, Key.Level};
|
||||
+ if (State->RetainedTiles.contains(CoreKey))
|
||||
+ {
|
||||
+ ++StaleRetainedLookaheadTiles;
|
||||
+ }
|
||||
+ }
|
||||
+ }
|
||||
+ if (VoxelForgeStartupTrace::IsActive())
|
||||
+ {
|
||||
+ VoxelForgeStartupTrace::RecordEvent(TEXT("core_region_lookahead_turn"),
|
||||
+ FString::Printf(
|
||||
+ TEXT("\"previous_step\":[%d,%d,%d],\"new_step\":[%d,%d,%d],"
|
||||
+ "\"previous_prefetch_tiles\":%d,"
|
||||
+ "\"stale_retained_lookahead_tiles\":%d"),
|
||||
+ PreviousRegionLookaheadStep.X, PreviousRegionLookaheadStep.Y,
|
||||
+ PreviousRegionLookaheadStep.Z, RegionLookaheadStep.X,
|
||||
+ RegionLookaheadStep.Y, RegionLookaheadStep.Z,
|
||||
+ PreviousRegionLookaheadTiles.Num(), StaleRetainedLookaheadTiles));
|
||||
+ }
|
||||
+ }
|
||||
+ LastCoreRegionLookaheadCenter = Center;
|
||||
+ LastCoreRegionLookaheadStep = RegionLookaheadStep;
|
||||
+ LastCoreRegionLookaheadGenerationEpoch = GenerationEpoch;
|
||||
+ }
|
||||
+ else if (GVoxelForgeCoreRegionLookahead == 0)
|
||||
+ {
|
||||
+ CoreRegionLookaheadTiles.Reset();
|
||||
+ LastCoreRegionLookaheadCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
|
||||
+ LastCoreRegionLookaheadStep = FIntVector::ZeroValue;
|
||||
+ LastCoreRegionLookaheadGenerationEpoch = GenerationEpoch;
|
||||
+ }
|
||||
+
|
||||
+ CoreRegionRetainedDesiredTiles = DesiredSorted;
|
||||
TSet<FVoxelTileKey> DesiredSet;
|
||||
+ DesiredSet.Reserve(DesiredSorted.Num() + CoreRegionLookaheadTiles.Num());
|
||||
for (const FVoxelTileKey& Key : DesiredSorted) DesiredSet.Add(Key);
|
||||
+ for (const FVoxelTileKey& Key : CoreRegionLookaheadTiles)
|
||||
+ {
|
||||
+ if (!DesiredSet.Contains(Key))
|
||||
+ {
|
||||
+ DesiredSet.Add(Key);
|
||||
+ CoreRegionRetainedDesiredTiles.Add(Key);
|
||||
+ }
|
||||
+ }
|
||||
+
|
||||
+ bool bRetainedSetChanged = false;
|
||||
+ int32 RetainedTileCount = 0;
|
||||
TArray<FVoxelTileKey> MissingRetainedFineTiles;
|
||||
{
|
||||
FScopeLock Lock(&State->RetainedTilesLock);
|
||||
RetainedTileCount = static_cast<int32>(State->RetainedTiles.size());
|
||||
bRetainedSetChanged = DesiredSet.Num() != RetainedTileCount;
|
||||
- for (const FVoxelTileKey& Key : DesiredSorted)
|
||||
+ for (const FVoxelTileKey& Key : CoreRegionRetainedDesiredTiles)
|
||||
{
|
||||
const VoxelForgeCore::TileKey CoreKey{{Key.Coord.X, Key.Coord.Y, Key.Coord.Z}, Key.Level};
|
||||
const bool bAlreadyRetained = State->RetainedTiles.contains(CoreKey);
|
||||
@@ -11313,7 +11504,7 @@ void AVoxelWorld::BuildCoreDesiredTiles(const FIntVector& Center, const FVector&
|
||||
{
|
||||
PendingCoreRegionTileSet.Add(Key);
|
||||
}
|
||||
- PendingCoreRetainedTiles = DesiredSorted;
|
||||
+ PendingCoreRetainedTiles = CoreRegionRetainedDesiredTiles;
|
||||
bCoreRegionCompilePending = true;
|
||||
++CoreRegionCompileRequestSerial;
|
||||
if (VoxelForgeStartupTrace::IsActive())
|
||||
@@ -11322,6 +11513,7 @@ void AVoxelWorld::BuildCoreDesiredTiles(const FIntVector& Center, const FVector&
|
||||
TEXT("\"request_serial\":%d,\"reason\":\"%s\",\"center\":[%d,%d,%d],"
|
||||
"\"previous_center\":[%d,%d,%d],\"center_delta\":[%d,%d,%d],"
|
||||
"\"desired_tiles\":%d,\"retained_tiles\":%d,\"missing_region_tiles\":%d,"
|
||||
+ "\"lookahead_region_tiles\":%d,\"lookahead_step\":[%d,%d,%d],"
|
||||
"\"desired_epoch\":%u,\"active_tile_tasks\":%d"),
|
||||
CoreRegionCompileRequestSerial,
|
||||
!bHadPreviousCenter ? TEXT("initial_desired_region_set")
|
||||
@@ -11332,7 +11524,9 @@ void AVoxelWorld::BuildCoreDesiredTiles(const FIntVector& Center, const FVector&
|
||||
bCenterMoved ? Center.X - PreviousCenter.X : 0,
|
||||
bCenterMoved ? Center.Y - PreviousCenter.Y : 0,
|
||||
bCenterMoved ? Center.Z - PreviousCenter.Z : 0,
|
||||
- DesiredSorted.Num(), RetainedTileCount, PendingCoreRegionTiles.Num(), DesiredEpoch,
|
||||
+ DesiredSorted.Num(), RetainedTileCount, PendingCoreRegionTiles.Num(),
|
||||
+ CoreRegionLookaheadTiles.Num(), RegionLookaheadStep.X, RegionLookaheadStep.Y,
|
||||
+ RegionLookaheadStep.Z, DesiredEpoch,
|
||||
ActiveTileTaskCount.load(std::memory_order_relaxed)));
|
||||
}
|
||||
StartPendingCoreRegionCompile();
|
||||
@@ -11363,13 +11557,27 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||
State->bRegionCompileSucceeded.store(false, std::memory_order_release);
|
||||
const TArray<FVoxelTileKey> MissingTiles = PendingCoreRegionTiles;
|
||||
const TArray<FVoxelTileKey> RetainedTiles = PendingCoreRetainedTiles;
|
||||
+ TSet<FVoxelTileKey> ActualDesiredSet;
|
||||
+ TSet<FVoxelTileKey> PrefetchSet;
|
||||
+ for (const FVoxelTileKey& Key : DesiredSorted) ActualDesiredSet.Add(Key);
|
||||
+ for (const FVoxelTileKey& Key : CoreRegionLookaheadTiles) PrefetchSet.Add(Key);
|
||||
+ TArray<FVoxelTileKey> PrefetchTiles;
|
||||
+ for (const FVoxelTileKey& Key : MissingTiles)
|
||||
+ {
|
||||
+ if (PrefetchSet.Contains(Key) && !ActualDesiredSet.Contains(Key))
|
||||
+ {
|
||||
+ PrefetchTiles.Add(Key);
|
||||
+ }
|
||||
+ }
|
||||
const FIntVector CompileCenter = PendingCoreRegionRequestCenter;
|
||||
const bool bAllowConcurrentGeneration = GVoxelForgeCoreRegionConcurrentAdmission != 0;
|
||||
const uint32 TaskEpoch = GenerationEpoch;
|
||||
const uint32 TaskDesiredEpoch = DesiredEpoch;
|
||||
+ const int32 RequestSerial = CoreRegionCompileRequestSerial;
|
||||
const uint32 WorkerLimit = static_cast<uint32>(FMath::Clamp(GetMaxConcurrentTasks(), 1, 16));
|
||||
ActiveCoreRegionCompileCenter = CompileCenter;
|
||||
ActiveCoreRegionCompileRequestSerial = CoreRegionCompileRequestSerial;
|
||||
+ ActiveCoreRegionWastedPrefetchTiles = 0;
|
||||
ActiveCoreRegionCompileStartSeconds = FPlatformTime::Seconds();
|
||||
ActiveCoreRegionCompileStartCycles = FPlatformTime::Cycles64();
|
||||
CoreRegionCompileTilesInFlight.Reset();
|
||||
@@ -11381,25 +11589,32 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||
1, std::memory_order_relaxed) + 1;
|
||||
State->RegionCompileStartCycles.store(ActiveCoreRegionCompileStartCycles,
|
||||
std::memory_order_release);
|
||||
+ State->RegionCompileWorkerStartCycles.store(0, std::memory_order_release);
|
||||
State->RegionCompileEndCycles.store(0, std::memory_order_release);
|
||||
State->RegionCompileTileCount.store(0, std::memory_order_release);
|
||||
+ State->RegionCompileWorkerThreadId.store(0, std::memory_order_release);
|
||||
State->RegionCompileWorkerSeconds.store(0.0, std::memory_order_release);
|
||||
+ State->RegionCompileWallSeconds.store(0.0, std::memory_order_release);
|
||||
if (VoxelForgeStartupTrace::IsActive())
|
||||
{
|
||||
VoxelForgeStartupTrace::RecordEvent(TEXT("core_region_compile_start"), FString::Printf(
|
||||
TEXT("\"compile_serial\":%d,\"request_serial\":%d,\"center\":[%d,%d,%d],"
|
||||
"\"missing_region_tiles\":%d,\"retained_tiles\":%d,\"concurrent_admission\":%d,"
|
||||
- "\"active_tile_tasks\":%d,\"active_task_guard\":%d"),
|
||||
+ "\"prefetch_compile_tiles\":%d,"
|
||||
+ "\"active_tile_tasks\":%d,\"active_generation_workers\":%d,"
|
||||
+ "\"active_task_guard\":%d,\"game_thread_id\":%u"),
|
||||
CompileSerial, CoreRegionCompileRequestSerial,
|
||||
CompileCenter.X, CompileCenter.Y, CompileCenter.Z,
|
||||
MissingTiles.Num(), RetainedTiles.Num(), bAllowConcurrentGeneration ? 1 : 0,
|
||||
+ PrefetchTiles.Num(),
|
||||
ActiveTileTaskCount.load(std::memory_order_relaxed),
|
||||
- ActiveTaskCount.load(std::memory_order_relaxed)));
|
||||
+ ActiveGenerationWorkerCount.load(std::memory_order_relaxed),
|
||||
+ ActiveTaskCount.load(std::memory_order_relaxed), FPlatformTLS::GetCurrentThreadId()));
|
||||
}
|
||||
ActiveTaskCount.fetch_add(1, std::memory_order_relaxed);
|
||||
UE::Tasks::Launch(TEXT("VoxelForgeCoreTileRegion"),
|
||||
- [this, State, MissingTiles, RetainedTiles, TaskEpoch, TaskDesiredEpoch, WorkerLimit,
|
||||
- bAllowConcurrentGeneration, CompileSerial]()
|
||||
+ [this, State, MissingTiles, RetainedTiles, PrefetchTiles, CompileCenter, TaskEpoch,
|
||||
+ TaskDesiredEpoch, RequestSerial, WorkerLimit, bAllowConcurrentGeneration, CompileSerial]()
|
||||
{
|
||||
struct FTaskGuard
|
||||
{
|
||||
@@ -11407,12 +11622,23 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||
~FTaskGuard() { Counter.fetch_sub(1, std::memory_order_release); }
|
||||
} Guard{ActiveTaskCount};
|
||||
|
||||
+ FCoreRegionCompileResult Result;
|
||||
+ Result.RuntimeState = State;
|
||||
+ Result.GenerationEpoch = TaskEpoch;
|
||||
+ Result.DesiredEpoch = TaskDesiredEpoch;
|
||||
+ Result.Center = CompileCenter;
|
||||
+ Result.RequestSerial = RequestSerial;
|
||||
+ Result.CompileSerial = CompileSerial;
|
||||
+ Result.PrefetchTiles = PrefetchTiles;
|
||||
+ Result.WorkerStartCycles = FPlatformTime::Cycles64();
|
||||
+ Result.WorkerThreadId = FPlatformTLS::GetCurrentThreadId();
|
||||
+
|
||||
if (bShuttingDown.load(std::memory_order_acquire)
|
||||
|| State->bShuttingDown.load(std::memory_order_acquire))
|
||||
{
|
||||
- State->RegionCompileEndCycles.store(FPlatformTime::Cycles64(), std::memory_order_release);
|
||||
- State->bRegionCompileInFlight.store(false, std::memory_order_release);
|
||||
- State->bRegionCompileFinished.store(true, std::memory_order_release);
|
||||
+ Result.bAborted = true;
|
||||
+ Result.WorkerEndCycles = FPlatformTime::Cycles64();
|
||||
+ CoreRegionCompileResults.Enqueue(MoveTemp(Result));
|
||||
return;
|
||||
}
|
||||
std::vector<VoxelForgeCore::TileKey> Missing;
|
||||
@@ -11456,38 +11682,14 @@ void AVoxelWorld::StartPendingCoreRegionCompile()
|
||||
FWriteScopeLock Lock(State->WorldLock);
|
||||
CompileAndRetain();
|
||||
}
|
||||
- State->RegionCompileWorkerSeconds.store(CompileStats.workerSeconds,
|
||||
- std::memory_order_release);
|
||||
- State->bRegionCompileSucceeded.store(bCompiled, std::memory_order_release);
|
||||
- if (!bCompiled)
|
||||
- {
|
||||
- State->bRegionCompileFailed.store(true, std::memory_order_release);
|
||||
- UE_LOG(LogTemp, Error,
|
||||
- TEXT("[VoxelForgeCoreRuntime] tile-region compile failed epoch=%u desired_epoch=%u: %s"),
|
||||
- TaskEpoch, TaskDesiredEpoch, UTF8_TO_TCHAR(Error.c_str()));
|
||||
- }
|
||||
- else
|
||||
- {
|
||||
- {
|
||||
- FScopeLock RetainedLock(&State->RetainedTilesLock);
|
||||
- State->RetainedTiles.clear();
|
||||
- for (const VoxelForgeCore::TileKey& Key : Retained)
|
||||
- {
|
||||
- State->RetainedTiles.insert(Key);
|
||||
- }
|
||||
- }
|
||||
- State->RegionCompileTileCount.store(
|
||||
- static_cast<int32>(CompiledTileCount), std::memory_order_release);
|
||||
- State->bRebuildDesired.store(true, std::memory_order_release);
|
||||
- UE_LOG(LogTemp, Display,
|
||||
- TEXT("[VoxelForgeCoreRuntime] tile_regions_compiled=%d retained=%d epoch=%u desired_epoch=%u "
|
||||
- "wall_ms=%.3f worker_s=%.3f serial=%d"),
|
||||
- static_cast<int32>(CompiledTileCount), RetainedTiles.Num(), TaskEpoch, TaskDesiredEpoch,
|
||||
- CompileStats.wallSeconds * 1000.0, CompileStats.workerSeconds, CompileSerial);
|
||||
- }
|
||||
- State->RegionCompileEndCycles.store(FPlatformTime::Cycles64(), std::memory_order_release);
|
||||
- State->bRegionCompileInFlight.store(false, std::memory_order_release);
|
||||
- State->bRegionCompileFinished.store(true, std::memory_order_release);
|
||||
+ Result.bSucceeded = bCompiled;
|
||||
+ Result.RetainedTiles = RetainedTiles;
|
||||
+ Result.Error = UTF8_TO_TCHAR(Error.c_str());
|
||||
+ Result.CompiledTileCount = static_cast<int32>(CompiledTileCount);
|
||||
+ Result.WorkerSeconds = CompileStats.workerSeconds;
|
||||
+ Result.CoreWallSeconds = CompileStats.wallSeconds;
|
||||
+ Result.WorkerEndCycles = FPlatformTime::Cycles64();
|
||||
+ CoreRegionCompileResults.Enqueue(MoveTemp(Result));
|
||||
}, UE::Tasks::ETaskPriority::BackgroundNormal);
|
||||
}
|
||||
|
||||
@@ -12331,6 +12533,88 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||
RecordSchedulerSkip(TEXT("core_not_ready"));
|
||||
return; // Core mode never falls through to the legacy generator.
|
||||
}
|
||||
+ FCoreRegionCompileResult RegionResult;
|
||||
+ while (CoreRegionCompileResults.Dequeue(RegionResult))
|
||||
+ {
|
||||
+ if (RegionResult.RuntimeState.IsValid())
|
||||
+ {
|
||||
+ RegionResult.RuntimeState->bRegionCompileInFlight.store(
|
||||
+ false, std::memory_order_release);
|
||||
+ }
|
||||
+ if (RegionResult.bAborted
|
||||
+ || RegionResult.RuntimeState != State
|
||||
+ || RegionResult.GenerationEpoch != GenerationEpoch)
|
||||
+ {
|
||||
+ if (VoxelForgeStartupTrace::IsActive())
|
||||
+ {
|
||||
+ VoxelForgeStartupTrace::RecordEvent(TEXT("core_region_compile_discard"),
|
||||
+ FString::Printf(
|
||||
+ TEXT("\"request_serial\":%d,\"compile_serial\":%d,"
|
||||
+ "\"result_epoch\":%u,\"current_epoch\":%u,\"aborted\":%d"),
|
||||
+ RegionResult.RequestSerial, RegionResult.CompileSerial,
|
||||
+ RegionResult.GenerationEpoch, GenerationEpoch,
|
||||
+ RegionResult.bAborted ? 1 : 0));
|
||||
+ }
|
||||
+ continue;
|
||||
+ }
|
||||
+
|
||||
+ State->RegionCompileWorkerStartCycles.store(
|
||||
+ RegionResult.WorkerStartCycles, std::memory_order_release);
|
||||
+ State->RegionCompileEndCycles.store(
|
||||
+ RegionResult.WorkerEndCycles, std::memory_order_release);
|
||||
+ State->RegionCompileTileCount.store(
|
||||
+ RegionResult.CompiledTileCount, std::memory_order_release);
|
||||
+ State->RegionCompileWorkerThreadId.store(
|
||||
+ RegionResult.WorkerThreadId, std::memory_order_release);
|
||||
+ State->RegionCompileWorkerSeconds.store(
|
||||
+ RegionResult.WorkerSeconds, std::memory_order_release);
|
||||
+ State->RegionCompileWallSeconds.store(
|
||||
+ RegionResult.CoreWallSeconds, std::memory_order_release);
|
||||
+ State->bRegionCompileSucceeded.store(
|
||||
+ RegionResult.bSucceeded, std::memory_order_release);
|
||||
+ ActiveCoreRegionCompileCenter = RegionResult.Center;
|
||||
+ ActiveCoreRegionCompileRequestSerial = RegionResult.RequestSerial;
|
||||
+ TSet<FVoxelTileKey> CurrentDesiredSet;
|
||||
+ for (const FVoxelTileKey& Key : DesiredSorted) CurrentDesiredSet.Add(Key);
|
||||
+ ActiveCoreRegionWastedPrefetchTiles = 0;
|
||||
+ for (const FVoxelTileKey& Key : RegionResult.PrefetchTiles)
|
||||
+ {
|
||||
+ if (!CurrentDesiredSet.Contains(Key) && !CoreRegionLookaheadTiles.Contains(Key))
|
||||
+ {
|
||||
+ ++ActiveCoreRegionWastedPrefetchTiles;
|
||||
+ }
|
||||
+ }
|
||||
+ if (RegionResult.bSucceeded)
|
||||
+ {
|
||||
+ {
|
||||
+ FScopeLock RetainedLock(&State->RetainedTilesLock);
|
||||
+ State->RetainedTiles.clear();
|
||||
+ for (const FVoxelTileKey& Key : RegionResult.RetainedTiles)
|
||||
+ {
|
||||
+ State->RetainedTiles.insert({
|
||||
+ {Key.Coord.X, Key.Coord.Y, Key.Coord.Z}, Key.Level});
|
||||
+ }
|
||||
+ }
|
||||
+ State->bRebuildDesired.store(true, std::memory_order_release);
|
||||
+ UE_LOG(LogTemp, Display,
|
||||
+ TEXT("[VoxelForgeCoreRuntime] tile_regions_compiled=%d retained=%d "
|
||||
+ "epoch=%u desired_epoch=%u wall_ms=%.3f worker_s=%.3f serial=%d"),
|
||||
+ RegionResult.CompiledTileCount, RegionResult.RetainedTiles.Num(),
|
||||
+ RegionResult.GenerationEpoch, RegionResult.DesiredEpoch,
|
||||
+ RegionResult.CoreWallSeconds * 1000.0, RegionResult.WorkerSeconds,
|
||||
+ RegionResult.CompileSerial);
|
||||
+ }
|
||||
+ else
|
||||
+ {
|
||||
+ State->bRegionCompileFailed.store(true, std::memory_order_release);
|
||||
+ UE_LOG(LogTemp, Error,
|
||||
+ TEXT("[VoxelForgeCoreRuntime] tile-region compile failed epoch=%u "
|
||||
+ "desired_epoch=%u: %s"),
|
||||
+ RegionResult.GenerationEpoch, RegionResult.DesiredEpoch,
|
||||
+ *RegionResult.Error);
|
||||
+ }
|
||||
+ State->bRegionCompileFinished.store(true, std::memory_order_release);
|
||||
+ }
|
||||
if (State->bRebuildDesired.exchange(false, std::memory_order_acq_rel))
|
||||
{
|
||||
bForceDesiredRebuild = true;
|
||||
@@ -12345,9 +12629,17 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||
State->RegionCompileStartCycles.load(std::memory_order_acquire);
|
||||
const uint64 CompileEndCycles =
|
||||
State->RegionCompileEndCycles.load(std::memory_order_acquire);
|
||||
+ const uint64 CompileWorkerStartCycles =
|
||||
+ State->RegionCompileWorkerStartCycles.load(std::memory_order_acquire);
|
||||
const double CompileElapsedMilliseconds = CompileEndCycles >= CompileStartCycles
|
||||
? FPlatformTime::ToSeconds64(CompileEndCycles - CompileStartCycles) * 1000.0
|
||||
: 0.0;
|
||||
+ const double CompileQueueMilliseconds = CompileWorkerStartCycles >= CompileStartCycles
|
||||
+ ? FPlatformTime::ToSeconds64(CompileWorkerStartCycles - CompileStartCycles) * 1000.0
|
||||
+ : 0.0;
|
||||
+ const double CompileTaskMilliseconds = CompileEndCycles >= CompileWorkerStartCycles
|
||||
+ ? FPlatformTime::ToSeconds64(CompileEndCycles - CompileWorkerStartCycles) * 1000.0
|
||||
+ : 0.0;
|
||||
const double GateElapsedMilliseconds = CoreRegionPendingSinceSeconds > 0.0
|
||||
? FMath::Max(0.0, FPlatformTime::Seconds() - CoreRegionPendingSinceSeconds) * 1000.0
|
||||
: 0.0;
|
||||
@@ -12355,6 +12647,10 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||
State->RegionCompileTileCount.load(std::memory_order_acquire);
|
||||
const double CompileWorkerSeconds =
|
||||
State->RegionCompileWorkerSeconds.load(std::memory_order_acquire);
|
||||
+ const double CompileCoreWallSeconds =
|
||||
+ State->RegionCompileWallSeconds.load(std::memory_order_acquire);
|
||||
+ const uint32 CompileWorkerThreadId =
|
||||
+ State->RegionCompileWorkerThreadId.load(std::memory_order_acquire);
|
||||
const int32 CompileSerial =
|
||||
State->RegionCompileSerial.load(std::memory_order_acquire);
|
||||
const uint64 IndependentSubmissions = CoreRegionIndependentSubmissions;
|
||||
@@ -12364,16 +12660,23 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||
VoxelForgeStartupTrace::RecordEvent(TEXT("core_region_compile_complete"), FString::Printf(
|
||||
TEXT("\"compile_serial\":%d,\"request_serial\":%d,\"center\":[%d,%d,%d],"
|
||||
"\"success\":%d,\"compiled_tiles\":%d,\"compile_elapsed_ms\":%s,"
|
||||
- "\"compile_worker_s\":%s,\"pending_gate_elapsed_ms\":%s,"
|
||||
- "\"independent_submissions\":%llu,"
|
||||
+ "\"compile_queue_ms\":%s,\"compile_task_ms\":%s,"
|
||||
+ "\"compile_core_wall_ms\":%s,\"compile_worker_s\":%s,"
|
||||
+ "\"compile_worker_thread_id\":%u,\"pending_gate_elapsed_ms\":%s,"
|
||||
+ "\"independent_submissions\":%llu,\"wasted_prefetch_tiles\":%d,"
|
||||
"\"dependent_declines\":%llu,\"concurrent_admission\":%d"),
|
||||
CompileSerial, ActiveCoreRegionCompileRequestSerial,
|
||||
ActiveCoreRegionCompileCenter.X, ActiveCoreRegionCompileCenter.Y,
|
||||
ActiveCoreRegionCompileCenter.Z, bRegionCompileSucceeded ? 1 : 0,
|
||||
CompiledTileCount, *FString::SanitizeFloat(CompileElapsedMilliseconds),
|
||||
+ *FString::SanitizeFloat(CompileQueueMilliseconds),
|
||||
+ *FString::SanitizeFloat(CompileTaskMilliseconds),
|
||||
+ *FString::SanitizeFloat(CompileCoreWallSeconds * 1000.0),
|
||||
*FString::SanitizeFloat(CompileWorkerSeconds),
|
||||
+ CompileWorkerThreadId,
|
||||
*FString::SanitizeFloat(GateElapsedMilliseconds),
|
||||
static_cast<unsigned long long>(IndependentSubmissions),
|
||||
+ ActiveCoreRegionWastedPrefetchTiles,
|
||||
static_cast<unsigned long long>(DependentDeclines),
|
||||
GVoxelForgeCoreRegionConcurrentAdmission != 0 ? 1 : 0));
|
||||
}
|
||||
@@ -12388,8 +12691,9 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||
{
|
||||
FScopeLock RetainedLock(&State->RetainedTilesLock);
|
||||
bRetainedSetMatchesDesired =
|
||||
- DesiredSorted.Num() == static_cast<int32>(State->RetainedTiles.size());
|
||||
- for (const FVoxelTileKey& Key : DesiredSorted)
|
||||
+ CoreRegionRetainedDesiredTiles.Num()
|
||||
+ == static_cast<int32>(State->RetainedTiles.size());
|
||||
+ for (const FVoxelTileKey& Key : CoreRegionRetainedDesiredTiles)
|
||||
{
|
||||
const VoxelForgeCore::TileKey CoreKey{
|
||||
{Key.Coord.X, Key.Coord.Y, Key.Coord.Z}, Key.Level};
|
||||
@@ -12431,7 +12735,7 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||
{
|
||||
PendingCoreRegionTileSet.Add(Key);
|
||||
}
|
||||
- PendingCoreRetainedTiles = DesiredSorted;
|
||||
+ PendingCoreRetainedTiles = CoreRegionRetainedDesiredTiles;
|
||||
PendingCoreRegionRequestCenter = DesiredEpochCenterChunk;
|
||||
bCoreRegionCompilePending = true;
|
||||
StartPendingCoreRegionCompile();
|
||||
@@ -12584,6 +12888,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||
&& bWalkableRingReadyForReserve;
|
||||
int32 SchedulerSubmittedByLevel[9] = {};
|
||||
int32 SchedulerLoadAttemptsByLevel[9] = {};
|
||||
+ int32 SchedulerRegionCompileDeclinesByLevel[9] = {};
|
||||
+ int32 SchedulerFarParentDeclinesByLevel[9] = {};
|
||||
int32 SchedulerLevelCapReturns[9] = {};
|
||||
int32 SchedulerLevelCapBreaks[9] = {};
|
||||
int32 SchedulerCriticalFullCapBreaks = 0;
|
||||
@@ -12951,7 +13257,21 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||
if(CoreState.IsValid()
|
||||
&&CoreState->bFarWorldRequired.load(std::memory_order_acquire)
|
||||
&&!CoreState->bFarWorldReady.load(std::memory_order_acquire))
|
||||
+ {
|
||||
+ if (bTraceStreamingScheduler && Tile.Level >= 0 && Tile.Level < 9)
|
||||
+ {
|
||||
+ ++SchedulerFarParentDeclinesByLevel[Tile.Level];
|
||||
+ }
|
||||
return;
|
||||
+ }
|
||||
+ }
|
||||
+ if (bTraceStreamingScheduler && bUseVoxelForgeCore && bCoreRegionCompilePending
|
||||
+ && GVoxelForgeCoreRegionConcurrentAdmission != 0
|
||||
+ && (PendingCoreRegionTileSet.Contains(Tile)
|
||||
+ || CoreRegionCompileTilesInFlight.Contains(Tile))
|
||||
+ && Tile.Level >= 0 && Tile.Level < 9)
|
||||
+ {
|
||||
+ ++SchedulerRegionCompileDeclinesByLevel[Tile.Level];
|
||||
}
|
||||
if (Tile.Level >= 0 && Tile.Level < 9)
|
||||
{
|
||||
@@ -13661,10 +13981,37 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||
const double BusySeconds = FPlatformTime::ToSeconds64(WorkerCyclesDelta);
|
||||
const double SchedulerUpdateMilliseconds = (SchedulerTraceNow
|
||||
- StreamingUpdateStartSeconds) * 1000.0;
|
||||
+ int32 CriticalMissingTiles = 0;
|
||||
+ for (const FVoxelTileKey& Tile : CriticalDesiredTiles)
|
||||
+ {
|
||||
+ if (!LoadedTiles.Contains(Tile) && !CollisionNotRequiredTiles.Contains(Tile))
|
||||
+ {
|
||||
+ ++CriticalMissingTiles;
|
||||
+ }
|
||||
+ }
|
||||
+ int32 CollisionGateMissingTiles = 0;
|
||||
+ for (const FVoxelTileKey& Tile : CollisionGateDesiredTiles)
|
||||
+ {
|
||||
+ if (!LoadedTiles.Contains(Tile) && !CollisionNotRequiredTiles.Contains(Tile))
|
||||
+ {
|
||||
+ ++CollisionGateMissingTiles;
|
||||
+ }
|
||||
+ }
|
||||
+ const FIntVector DesiredCenterLag = DesiredEpochCenterChunk - CenterChunk;
|
||||
+ const TSharedPtr<FVoxelForgeCoreRuntimeState, ESPMode::ThreadSafe> TraceCoreState =
|
||||
+ VoxelForgeCoreState;
|
||||
+ const bool bFarWorldRequiredForTrace = TraceCoreState.IsValid()
|
||||
+ && TraceCoreState->bFarWorldRequired.load(std::memory_order_relaxed);
|
||||
+ const bool bFarWorldReadyForTrace = TraceCoreState.IsValid()
|
||||
+ && TraceCoreState->bFarWorldReady.load(std::memory_order_relaxed);
|
||||
VoxelForgeStartupTrace::RecordEvent(TEXT("scheduler_admission"), FString::Printf(
|
||||
TEXT("\"frame\":%llu,\"center\":[%d,%d,%d],\"desired_rebuilt\":%d,"
|
||||
"\"submission_loop_ran\":%d,\"all_chunks_loaded\":%d,"
|
||||
"\"walkable_ready\":%d,\"moving\":%d,"
|
||||
+ "\"pawn_collision_gate_engaged\":%d,\"critical_tiles_ready\":%d,"
|
||||
+ "\"critical_missing\":%d,\"collision_gate_missing\":%d,"
|
||||
+ "\"desired_center_lag\":[%d,%d,%d],\"desired_center_current\":%d,"
|
||||
+ "\"far_world_required\":%d,\"far_world_ready\":%d,"
|
||||
"\"core_region_compile_pending\":%d,\"core_ready\":%d,\"coarse_first\":%d,"
|
||||
"\"dynamic_near_reserve\":%d,\"legacy_pending_result_cap\":%d,"
|
||||
"\"core_region_concurrent_admission\":%d,"
|
||||
@@ -13685,6 +14032,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||
"\"level_candidate_scan_ms\":%s,\"passage_score_ms\":%s,"
|
||||
"\"level_sort_ms\":%s,\"level_admit_ms\":%s,"
|
||||
"\"desired_missing_by_lod\":%s,\"loaded_by_lod\":%s,\"pending_by_lod\":%s,"
|
||||
+ "\"region_compile_declines_by_lod\":%s,"
|
||||
+ "\"far_parent_declines_by_lod\":%s,"
|
||||
"\"submitted_by_lod\":%s,\"load_attempts_by_lod\":%s,"
|
||||
"\"admission_cap_by_lod\":%s,\"declined_rule_by_lod\":%s,"
|
||||
"\"level_cap_returns_by_lod\":%s,\"level_cap_breaks_by_lod\":%s,"
|
||||
@@ -13695,6 +14044,11 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||
static_cast<unsigned long long>(GFrameCounter), CenterChunk.X, CenterChunk.Y, CenterChunk.Z,
|
||||
bDesiredRebuiltThisUpdate ? 1 : 0, bSubmissionLoopRan ? 1 : 0, bAllChunksLoaded ? 1 : 0,
|
||||
bWalkableRingReadyForReserve ? 1 : 0, bPawnMoving ? 1 : 0,
|
||||
+ bPawnCollisionGateEngaged ? 1 : 0, CriticalMissingTiles == 0 ? 1 : 0,
|
||||
+ CriticalMissingTiles, CollisionGateMissingTiles,
|
||||
+ DesiredCenterLag.X, DesiredCenterLag.Y, DesiredCenterLag.Z,
|
||||
+ DesiredCenterLag == FIntVector::ZeroValue ? 1 : 0,
|
||||
+ bFarWorldRequiredForTrace ? 1 : 0, bFarWorldReadyForTrace ? 1 : 0,
|
||||
bCoreRegionCompilePending ? 1 : 0, IsVoxelForgeCoreReady() ? 1 : 0,
|
||||
GVoxelForgeStreamingCoarseFirst, GVoxelForgeStreamingDynamicNearReserve,
|
||||
GVoxelForgeStreamingLegacyPendingResultCap,
|
||||
@@ -13721,7 +14075,8 @@ void AVoxelWorld::UpdateChunksAroundPosition(const FVector& CenterPosition, APaw
|
||||
*FString::SanitizeFloat(LevelSortMilliseconds),
|
||||
*FString::SanitizeFloat(LevelAdmissionMilliseconds),
|
||||
*IntArrayJson(DesiredMissingByLevel), *IntArrayJson(LoadedByLevel),
|
||||
- *IntArrayJson(PendingByLevel), *IntArrayJson(SchedulerSubmittedByLevel),
|
||||
+ *IntArrayJson(PendingByLevel), *IntArrayJson(SchedulerRegionCompileDeclinesByLevel),
|
||||
+ *IntArrayJson(SchedulerFarParentDeclinesByLevel), *IntArrayJson(SchedulerSubmittedByLevel),
|
||||
*IntArrayJson(SchedulerLoadAttemptsByLevel), *IntArrayJson(AdmissionCapByLevel),
|
||||
*IntArrayJson(DeclinedRuleByLevel), *IntArrayJson(SchedulerLevelCapReturns),
|
||||
*IntArrayJson(SchedulerLevelCapBreaks), FullCapDeclinedTiles, ReserveDeclinedTiles,
|
||||
diff --git a/Source/VoxelForge/Public/VoxelWorld.h b/Source/VoxelForge/Public/VoxelWorld.h
|
||||
index b9fa34c..5979f29 100644
|
||||
--- a/Source/VoxelForge/Public/VoxelWorld.h
|
||||
+++ b/Source/VoxelForge/Public/VoxelWorld.h
|
||||
@@ -207,6 +207,28 @@ struct FChunkResult
|
||||
TArray<FVector> SurfaceWitnesses;
|
||||
};
|
||||
|
||||
+/** Worker-to-game-thread completion for immutable Core tile-region publication. */
|
||||
+struct FCoreRegionCompileResult
|
||||
+{
|
||||
+ TSharedPtr<FVoxelForgeCoreRuntimeState, ESPMode::ThreadSafe> RuntimeState;
|
||||
+ TArray<FVoxelTileKey> RetainedTiles;
|
||||
+ TArray<FVoxelTileKey> PrefetchTiles;
|
||||
+ FString Error;
|
||||
+ FIntVector Center = FIntVector::ZeroValue;
|
||||
+ uint32 GenerationEpoch = 0;
|
||||
+ uint32 DesiredEpoch = 0;
|
||||
+ uint64 WorkerStartCycles = 0;
|
||||
+ uint64 WorkerEndCycles = 0;
|
||||
+ uint32 WorkerThreadId = 0;
|
||||
+ int32 CompileSerial = 0;
|
||||
+ int32 RequestSerial = 0;
|
||||
+ int32 CompiledTileCount = 0;
|
||||
+ double WorkerSeconds = 0.0;
|
||||
+ double CoreWallSeconds = 0.0;
|
||||
+ bool bSucceeded = false;
|
||||
+ bool bAborted = false;
|
||||
+};
|
||||
+
|
||||
/** One worker-produced mesh identity emitted only by the explicit parity harness. */
|
||||
struct FVoxelTileHashDumpRecord
|
||||
{
|
||||
@@ -798,6 +820,11 @@ private:
|
||||
TArray<FVoxelTileKey> PendingCoreRegionTiles;
|
||||
TSet<FVoxelTileKey> PendingCoreRegionTileSet;
|
||||
TArray<FVoxelTileKey> PendingCoreRetainedTiles;
|
||||
+ TArray<FVoxelTileKey> CoreRegionRetainedDesiredTiles;
|
||||
+ TArray<FVoxelTileKey> CoreRegionLookaheadTiles;
|
||||
+ FIntVector LastCoreRegionLookaheadCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
|
||||
+ FIntVector LastCoreRegionLookaheadStep = FIntVector::ZeroValue;
|
||||
+ uint32 LastCoreRegionLookaheadGenerationEpoch = 0;
|
||||
TSet<FVoxelTileKey> CoreRegionCompileTilesInFlight;
|
||||
FIntVector LastCoreRegionRequestCenter = FIntVector(MAX_int32, MAX_int32, MAX_int32);
|
||||
FIntVector PendingCoreRegionRequestCenter = FIntVector::ZeroValue;
|
||||
@@ -810,6 +837,7 @@ private:
|
||||
uint64 CoreRegionDependentDeclines = 0;
|
||||
int32 CoreRegionCompileRequestSerial = 0;
|
||||
int32 ActiveCoreRegionCompileRequestSerial = 0;
|
||||
+ int32 ActiveCoreRegionWastedPrefetchTiles = 0;
|
||||
|
||||
public:
|
||||
|
||||
@@ -1139,6 +1167,7 @@ public:
|
||||
TQueue<FChunkResult, EQueueMode::Mpsc> EditedProcessQueue;
|
||||
TQueue<FChunkResult, EQueueMode::Mpsc> NearProcessQueue;
|
||||
TQueue<FChunkResult, EQueueMode::Mpsc> ProcessQueue;
|
||||
+ TQueue<FCoreRegionCompileResult, EQueueMode::Mpsc> CoreRegionCompileResults;
|
||||
// The cap counts completed MPSC results plus admitted tile tasks, so a slow game-thread apply
|
||||
// cannot accumulate an unbounded payload queue while completed work never consumes worker slots.
|
||||
// Le plafond compte les resultats MPSC et les taches admises; un apply lent ne peut donc pas
|
||||
@@ -1362,6 +1391,9 @@ public:
|
||||
float HeadlessStreamingTestSpeedCmPerSecond = 0.0f;
|
||||
float HeadlessStreamingTestStartSeconds = 0.0f;
|
||||
float HeadlessStreamingTestMoveDurationSeconds = 0.0f;
|
||||
+ float HeadlessStreamingTestTurnAfterSeconds = 0.0f;
|
||||
+ bool bHeadlessStreamingTestTurnDone = false;
|
||||
+ FVector HeadlessStreamingTestTurnDirection = FVector::YAxisVector;
|
||||
float HeadlessStreamingTestExitSeconds = 0.0f;
|
||||
float HeadlessStreamingTestTeleportAtSeconds = 0.0f;
|
||||
float HeadlessStreamingTestTeleportDistanceCm = 0.0f;
|
||||
Reference in new issue
Block a user