research(R5): tiered verification (fast/medium/full) and a final-source stamp
Inventory of every audit gate with controls and wall times; a fast tier (~3-4 min, real owner styles) per trial, medium (~8-10 min) per candidate, full (~32-34 min) once. A compiled-in source manifest stamp that every runner checks, so evidence provably comes from the committed tree. Lists duplicate runners. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
This commit is contained in:
1 parent
cf57c87daf
commit
88c54eab44
1 file changed
+52
@@ -0,0 +1,52 @@
|
||||
# R5 — faster verification, same failure coverage
|
||||
|
||||
Read-only review of StreamBenchmarkPipeline.h, Linux3 and Linux30–37 runners/reports, and saved logs. L38 was treated as in progress; this report does not depend on its concurrently changing outputs. Times come from runner elapsed_seconds summaries and matching logs where available; build logs have no elapsed timestamps.
|
||||
|
||||
## Audit inventory
|
||||
|
||||
| Gate / current runner | What it proves; negative control | Typical wall time | Overlap and value |
|
||||
|---|---|---:|---|
|
||||
| Seed/compiler/worker sweep — Linux3/RunSeedSweep.py (576 cells) | Across 48 seeds, 2 recipes, Clang off/on and GCC, W1/W12: nonempty 27-tile meshes and equal hashes across compilers/workers. No injected negative control. | 433 s / 7.2 min | Useful for seed/planner changes; unrelated to most passage geometry trials. |
|
||||
| Passage matrix — RunAuditsL31/32.py, RunUpperFixtureAuditsL33.py, RunPassageAuditsL34–37.py (24 cases) | Capsule clearance, support/fallback, grade, content/radius, route-mouth binding, open air, wall/floor ripple, mouth profile/naturalness, bridge and column checks, strict layout. Controls: blocked/no-passage, L23 terrace and smooth wall, L24 ungated support, synthetic spheres, beaded floor, malformed layouts. | 579 s / 9.7 min; 13–34 s per case | High value and falsifiable. Repeats static controls and full field work across 24 style/recipe/compiler cases. The L37 owner-named JSONs are style overrides, not canonical owner config. |
|
||||
| L13 route parity + L9 uniform-block proof — RunInvariantAuditsL37.py and RunLegacyAuditsL37.py | Cross-path/page and exact-query parity; no false uniform-block certificates. Transition sub-suite checks hard/gradient/interleaved interfaces for holes/air and rejects too-wide/too-thin bands. Transition controls are real; route/proof have no injected negative control. | Invariant suite 98 s / 1.6 min; transition part is ~19 s. RunLegacy adds 80 s / 1.3 min of duplicate route/proof work. | RunLegacyAuditsL37.py repeats the first four commands in RunInvariantAuditsL37.py with the same inputs and flags. Keep one. |
|
||||
| Ring / terrace / flat — MeasureRingL30/31.py, AuditRing*L32.py, RunRingAuditsL37.py | FI floor and SW ceiling surface profiles at LOD0–8: coverage, relief, flats, following steps and normals. L30 ring mesh and L31 flat mesh are retained failing controls. | 31 s for 18 scans + ~33 s for six exports: about 1 min | Strong, specific controls. Little overlap with tunnel mouth profiles; the full LOD sweep is final/medium work, not every trial. |
|
||||
| Determinism / crossing — RunDeterminismL31–37.py, RunCrossing30.py | Hashes for detail, whole view, LOD0–8 and exact queries across Clang/GCC, W1/8/12; a 24 m moving-camera crossing and upper-SW fixture. No injected bad-output control; equality across the matrix is the check. | 935 s / 15.6 min for 30 L37 cells | Valuable at the end. “30/30” means 30 distinct matrix cells, not 30 repeats. Reuses full world generation done by passage and cost gates. It cannot prove source freshness. |
|
||||
| L35/36 wall and floor ripple — nested in passage matrix; older AuditTunnelMeshNormalsL35.py | Density-field wall and floor profiles detect L35 wall ribs and L36 floor washboard. Controls: synthetic spheres and beaded floor; the old mesh-normal scan was supplemental. | Included in 9.7 min passage total | High value. L35 shows why wall-only sampling is insufficient; retain separate floor threshold/coverage. |
|
||||
| Built mouth shape / bridge — AuditMouthShapeL37.py; nested PassageNoLongBridgeAudit | Differential mesh metrics for planar patch, straight run and corners; L36 mouth mesh and synthetic box are controls. | Analyzer time not recorded; consumes already exported meshes | Weak bridge proxy: the report records a 1 m FI straight-run alert although the reviewed image had no straight raised bridge edge; planar area is informational. The raised-crossing fix was style-gated, so require an actual-owner route check and image review. |
|
||||
| Mouth crease / fragments — RunMouthCreaseAuditL37.py (imports RunMouthFragmentAuditL37.py) | 0.125 m mesh-normal scan and detached-component scan. L35 42° and L36 27° meshes are crease controls; L35 fragment is also a control. | Analyzer time not recorded; same exported mesh can serve shape and capture checks | Falsifiable, but current summary misses the 17 m overview: L37 measured 25.54° there against the frozen 20° gate while the near scan passed. Gate both views. The imported fragment scan already runs here, so a separate invocation decodes/scans the mesh again. |
|
||||
| Captures — CapturePassageStandardSetL37.py, CaptureMouthOverviewL37.py | Provides visual evidence; exit code and JPEG size are the only automatic checks. No semantic negative control. | 34–57 s for the 12-view standard set; overview is separate | Human review found bridges, slabs, ribs and shards, but L37's owner-fi/owner-sw views came from style-0 override JSONs. Recipe stamps do not prove style or input identity. |
|
||||
| Cost pairs — RunCostL30–37.py | Five interleaved Clang baseline/candidate pairs per recipe, zero cache hits; estimates performance deltas and noise. No injected slow-path control. | 194 s / 3.2 min for 20 L37 processes | Useful for performance claims, not functional safety. Shares full-view generation with hash tests; keep for medium/final only. |
|
||||
|
||||
The header also offers tile-readiness (with a deliberately mis-marked-tile control), resolved-layout-boundary, base-page-equivalence and profiling/screen-space diagnostics. They are not called by the L37 evidence runners, so they are not current gates. Historical L30–36 runner copies are predecessor implementations, not additional independent evidence.
|
||||
|
||||
### Pulling weight
|
||||
|
||||
Keep passage/ripple, transition/hole, ring controls, worker/compiler hash comparison, and mouth fragment checks. They have concrete thresholds or controls and have caught actual defects. Keep L13/L9 once per recipe/compiler in the final bar; it protects separate invariants but is duplicated in the current scripts. Keep cost pairs only when promoting a candidate or making a speed claim.
|
||||
|
||||
The weak gates are capture file-size checks, the informational planar metric, and the near-only crease summary. They can pass while the picture is wrong or the far view fails. The passage wall fitter also reports a best-fit “period” for an owner step: R4 established that L38's 7.812 m value fits a step, not a repeated wave. Continue to fail on local wall amplitude/coverage and add an owner-switch/adjacent-profile step limit; report fitted period as diagnostic only.
|
||||
|
||||
A clean L37 bar sums to about 32–34 minutes before build time; the hours reported for a round are accumulated across trials, rebuilds and retries. The saved build logs do not let us separate compile time.
|
||||
|
||||
## Tiered plan
|
||||
|
||||
Times are audit runtime on the L37 machine, excluding source compilation because the saved build logs do not time it. Every tier starts with the stamp check below; any missing/mismatched stamp or failed assertion stops the tier.
|
||||
|
||||
| Tier | Work | Estimate |
|
||||
|---|---|---:|
|
||||
| **Fast — every trial** | On the canonical Standalone/data/owner_world.json, run Clang W12 passage audits for actual FI Style 2 and SW Style 1. Require actual recipe and style from the resolved snapshot, mouth binding, full capsule/support/open-air checks, wall **and** floor ripple/coverage, and the owner-step metric. Run W1/W12 hash pairs on these same two owner cases. Run targeted FI-floor/SW-ceiling LOD0 ring profiles plus both L30/L31 negative controls. Export/review one real-owner FI crossing view and one SW mid-tunnel view when the changed code can affect shape. | About **3–4 min**, hard timeout at 5 min. Measured basis: L37 actual-style proxy cases are ~40 s total; W1/W12 owner hashes ~111 s; two LOD0 exports ~5 s plus scans/controls; two focused exports/renders fit the remaining margin. A rebuild is required first, but its time must be instrumented separately. |
|
||||
| **Medium — before calling a candidate good** | Clang all 12 style-override cases (4 styles × 2 owner recipes + 4 upper-SW fixture), plus canonical owner FI2/SW1 assertions; all three transition modes and L13/L9 once; complete LOD0–8 ring suite; actual-owner FI2/SW1 shape, crease (near and overview) and fragment checks with captures. Run one interleaved cost pair per recipe if performance-sensitive. Stop at first failure. | About **8–10 min** additional. L37 Clang passage half is ~5.3 min; rings ~1 min; transitions/L13/L9 <1 min; focused images/mesh checks and optional cost pairs ~1–2 min. |
|
||||
| **Full — once at final source** | 24-case passage matrix on both compilers; 30-cell cross-compiler/worker/crossing matrix; transitions and L13/L9 (no duplicate runner); LOD0–8 rings; exact-owner visuals and shape/crease/fragment checks; five cost pairs per recipe; final captures. Require all owner labels to map to canonical FI2/SW1 inputs and assert style in the resolved layout. | About **32–34 min** from measured L37 sequential process totals, excluding builds. Existing duplicate L13/L9 work accounts for ~80 s. |
|
||||
|
||||
This keeps every named regression covered: L23 terraces by the passage profile and its terrace control; L30 rings/L31 flattening by ring profiles and both saved controls; L35 ribs/L36 washboard by wall/floor ripple; the L37 bridge by actual-owner route/shape checks plus real-style captures; stale L37 binaries by the stamp; the L38 owner step by the actual FI2 local wall/owner-step gate; and the style-0 mislabel by canonical input and resolved-style assertions. The style sweep remains before promotion and the full compiler matrix remains final-only.
|
||||
|
||||
## Final-source stamp
|
||||
|
||||
At build start, make a sorted manifest of compiled source inputs and their SHA-256 values (Core, standalone pipeline/wrappers, Unreal adapter, build files), plus HEAD and the staged/unstaged patch and untracked source hashes. Exclude generated Saved/, binaries, captures and reports; record test-script and input/fixture hashes separately. Hash the manifest and inject VOXELFORGE_SOURCE_STAMP as a compile definition into every executable. The binary prints it as its first log record. Each runner computes the expected manifest before launching and refuses a missing/mismatched stamp; it records the stamp, executable SHA-256, compiler/flags, input JSON SHA-256, resolved recipe/style/layout ID, audit-script hashes and elapsed time in every log and summary. Recompute after build and after the final run to catch source changes during the bar. For release, compare the stamped source manifest with the committed tree and commit that exact tree; any source edit invalidates the build and all evidence.
|
||||
|
||||
## Delete from the active gate path
|
||||
|
||||
- Drop RunLegacyAuditsL37.py as a separate stage; its four L13/L9 invocations duplicate RunInvariantAuditsL37.py exactly.
|
||||
- Combine crease and fragment into one mesh read; RunMouthCreaseAuditL37.py already imports and repeats the fragment analyzer.
|
||||
- Retire AuditPassageMouthMeshProfilesL34/35.py from gating: the L34 report records 3 covered / 6 no-coverage results, and the passage density profile is the covered check.
|
||||
- Remove superseded Linux30–36 runner copies and exploratory ring/window-search probes from the **active gate list**. Keep their source, logs and failed meshes as historical controls/evidence.
|
||||
- Remove owner- labels from style-override artifacts; rename them style-override-*. Reserve “owner” for owner_world.json with asserted FI Style 2 / SW Style 1.
|
||||
Reference in new issue
Block a user