From af5f2103b36f8e7935586d2d3cb4cef44066df75 Mon Sep 17 00:00:00 2001 From: Fr0zka Date: Mon, 27 Jul 2026 14:15:43 +0200 Subject: [PATCH] =?UTF-8?q?test:=20the=20Maze=20residue=20is=20/fp:fast,?= =?UTF-8?q?=20not=20port=20drift=20=E2=80=94=20encode=20the=20real=20bar?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The bisect settled it. The difference survives every stage removal down to "corridors + carve ONLY", which is character-for-character transcribed code, so it is not in anything the decomposition added. Cause, read out of the engine rather than assumed (VCToolChain.cs): case FPSemanticsMode.Default: // Default is imprecise FP semantics. case FPSemanticsMode.Imprecise: Arguments.Add("/fp:fast"); break; with UBT's own doc: "the compiler is allowed to transform math expressions in ways that might result in differently rounded results". Identical source in two translation units may reassociate differently, worth ~1 ULP. It shows up on exactly the ~2% of samples inside the SDF blend shell, where Blend - Sdf catastrophically cancels; outside it Carve is exactly 0 or 1 and both agree. So MazeEquivalence now grades what it can actually assert: - hard fail : any isosurface crossing (geometry moves) - info : differences at ULP scale (the unavoidable floor) - warn : anything larger, which IS port drift, and runs the bisect A test that warns on every port would get ignored by the port that matters. Recorded in OPSTACK-PLAN 2.6, and as AUDIT C9 for the part that outlives this refactor: ARCHITECTURE 9.1's "every peer regenerates identically" holds only between bit-identical binaries under /fp:fast. Fine for one build on one platform; a real desync source for a Linux server plus Windows clients both regenerating authoritative geometry. The FPSemantics::Precise knob exists but must not be turned speculatively -- it blocks the vectorisation T2.a was chasing, on the hot loop, for an unmeasured cost. Co-Authored-By: Claude Opus 5 --- AUDIT-2026-07.md | 53 +++++++++++++++ OPSTACK-PLAN.md | 14 ++++ OPSTACK-PROGRESS.md | 59 +++++++++++++++++ .../Tests/VoxelForgeOpStackMazeTest.cpp | 66 ++++++++++++++++--- 4 files changed, 184 insertions(+), 8 deletions(-) diff --git a/AUDIT-2026-07.md b/AUDIT-2026-07.md index a906a51..e09259f 100644 --- a/AUDIT-2026-07.md +++ b/AUDIT-2026-07.md @@ -247,6 +247,59 @@ reachable only with a very large origin room *and* short tunnels. Flagging it be --- +### C9 — The build uses `/fp:fast`, and the MP plan assumes bit-reproducible terrain ⚠️ **added 2026-07-27, measured not assumed** + +Found while chasing a 1-ULP difference between `GetMazeDensity` and its operator-stack port. The +difference survived a full bisect down to code that is character-for-character identical, which +pointed at the toolchain rather than the port. It is: + +```csharp +// UnrealBuildTool/Platform/Windows/VCToolChain.cs +case FPSemanticsMode.Default: // Default is imprecise FP semantics. +case FPSemanticsMode.Imprecise: Arguments.Add("/fp:fast"); break; +``` + +and UBT's own doc for that mode: *"FP math isn't IEEE-754 compliant: the compiler is allowed to +transform math expressions in ways that might result in differently rounded results from what +IEEE-754 requires."* The plugin sets no `FPSemantics` override, so it gets the default. + +**Two consequences, one benign and one not:** + +**Benign — refactors cannot be bit-identical.** The same expression compiled into two translation +units may reassociate differently, worth ~1 ULP. So "the port reproduces the original exactly" is +not an achievable bar for the op-stack work, and `OPSTACK-PLAN §2.6`'s bar (recognisably the same +*place*, judged on a screenshot) was the right call for reasons beyond the ones it gave. The +acceptance criterion is now encoded in `VoxelForge.OpStack.MazeEquivalence`: **hard-fail on any +isosurface crossing, tolerate ULP-scale deltas, warn on anything larger.** + +**Not benign — `ARCHITECTURE §9.1`'s multiplayer model rests on this.** The plan is "replicate the +seed + layout + diff, never the geometry; every peer regenerates identically." That guarantee is +only as strong as the floating-point reproducibility of the density path, and under `/fp:fast` it +holds **only between bit-identical binaries**. Same build, same platform: fine — `DensityPurity` +proves the field is pure across threads and query order. But a different compiler version, a +different optimisation level (Debug vs Shipping), or a different platform's toolchain may produce a +subtly different world from the same seed. + +The docs already reject GPU density partly because *"cross-GPU float determinism is fatal for +replicate-the-seed"*. The same argument applies to the CPU across build variants, and had not been +stated. + +**How much it matters depends on a design question that is Jahni's, not the audit's:** + +- **If listen-server only, one binary per platform, and clients never regenerate authoritative + geometry** — this is a non-issue. Ship as is. +- **If a dedicated server on Linux and Windows clients both regenerate terrain and compare** — this + is a real desync source, and it will present as rare, unreproducible, geometry-only divergence: + approximately the worst bug class to diagnose. + +**The knob, if it turns out to matter:** `ModuleRules.FPSemantics = FPSemanticsMode.Precise` in +`VoxelForge.Build.cs` restores IEEE semantics for this module only. **Do not do this speculatively** — +the density path is the plugin's hot loop, `/fp:precise` blocks exactly the vectorisation and +contraction that T2.a's SIMD noise work was chasing, and the cost is unmeasured. It is a decision to +take with a profile in hand and a confirmed cross-platform requirement, not a tidy-up. + +--- + ### Threading — what's *right*, for the record Worth stating plainly, because it's the part that's easy to get wrong and this doesn't: diff --git a/OPSTACK-PLAN.md b/OPSTACK-PLAN.md index 545c7bb..6358b23 100644 --- a/OPSTACK-PLAN.md +++ b/OPSTACK-PLAN.md @@ -196,6 +196,20 @@ strategically:** if bit-identity were required, the cheap path would be to wrap as one monolithic op — 8 opaque ops that don't compose, i.e. **the switch with extra steps and zero gain.** Releasing that constraint is what permits *real* decomposition into the primitives in §2.5. +> **✅ CONFIRMED THE HARD WAY, 2026-07-27 — and it turns out bit-identity was never available anyway.** +> The Maze port reproduced `GetMazeDensity` to within 1 ULP on 2.3% of samples, with **zero +> isosurface crossings**. A four-stage bisect showed the residue surviving into code that is +> character-for-character transcribed, which pointed at the toolchain: the plugin compiles with +> **`/fp:fast`** (UnrealBuildTool's Windows default), which explicitly licenses the compiler to +> reassociate the same expression differently per translation unit. So **no port of this kind can be +> bit-identical, at any level of care.** See `AUDIT-2026-07.md §C9` — which also flags the part that +> matters more than this plan does: the multiplayer model's "every peer regenerates identically" +> holds only between bit-identical binaries. +> +> **The operational bar for every remaining archetype port, encoded in +> `VoxelForge.OpStack.MazeEquivalence`:** hard-fail on any isosurface crossing (that moves geometry); +> tolerate ULP-scale deltas (unavoidable); warn on anything larger (that is real port drift). + **The bar instead:** for each ported archetype, an authored op stack must reproduce the *character* of the old one — same scale, same navigability, same feel, recognisably the same kind of place. Judged by Jahni on a screenshot at a fixed seed, not by a diff. Expect and accept a one-time re-tune, exactly as the T2.a diff --git a/OPSTACK-PROGRESS.md b/OPSTACK-PROGRESS.md index b294d37..f5e88da 100644 --- a/OPSTACK-PROGRESS.md +++ b/OPSTACK-PROGRESS.md @@ -352,3 +352,62 @@ conclusion worth having explicitly rather than re-deriving per port. wire the stack into `GetDensityAt` behind a per-strate opt-in. --- + +## 2026-07-27 (afternoon) — the residue is the COMPILER. Measured, not guessed. + +**Bisect result:** + +``` +roughness off -> 151 / 5000 differ (max |delta| 1.907e-06) +roughness + seal off -> 155 / 5000 differ (max |delta| 9.537e-07) +roughness + seal + spine off -> 126 / 5000 differ (max |delta| 9.537e-07) +corridors + carve ONLY -> 126 / 5000 differ (max |delta| 9.537e-07) +``` + +The residue survives every stage removal, down to **constant rock + capsule SDF + carve** — code +that is a character-for-character transcription. So it is not in anything the decomposition added. + +**Cause, from the engine source rather than from memory** (`VCToolChain.cs`): + +```csharp +case FPSemanticsMode.Default: // Default is imprecise FP semantics. +case FPSemanticsMode.Imprecise: Arguments.Add("/fp:fast"); break; +``` + +UBT's own doc for that mode: *"FP math isn't IEEE-754 compliant: the compiler is allowed to transform +math expressions in ways that might result in differently rounded results."* The plugin sets no +override, so identical source in `VoxelGenerator.cpp` and `VoxelDensityOpStack.cpp` may legitimately +reassociate differently — worth about 1 ULP. + +Two prior hypotheses were wrong (the `FVector` round-trip, then "check the roughness window / carve +blend"). The bisect cost one build and settled it. **Noted as a working lesson: on a numeric +discrepancy, bisect before hypothesising a third time.** + +**Why exactly ~2.3% of samples:** `Blend - Sdf` catastrophically cancels at the edge of the blend +shell, amplifying a 1-ULP SDF difference into a 1-ULP density difference. Outside that thin shell +`Carve` is exactly 0 or exactly 1 and both paths agree bit for bit. + +### Consequences recorded + +1. **`OPSTACK-PLAN §2.6`** — bit-identity is not achievable in principle for these ports, at any + level of care. The operational bar for every remaining archetype, now encoded in the test: + **hard-fail on isosurface crossings · tolerate ULP-scale deltas · warn on anything larger** + (that last one is real port drift, and the test no longer cries wolf about the floor). +2. **`AUDIT-2026-07.md §C9` (new)** — the part that matters more than the port: `ARCHITECTURE §9.1`'s + multiplayer model is "replicate the seed, every peer regenerates identically", and under + `/fp:fast` that holds **only between bit-identical binaries**. Same build, same platform: fine + (`DensityPurity` proves it). Windows client + Linux dedicated server both regenerating + authoritative geometry: a real desync source, presenting as rare unreproducible geometry-only + divergence. The knob is `FPSemantics = FPSemanticsMode.Precise` in `VoxelForge.Build.cs`, and + **it should not be turned speculatively** — it blocks the vectorisation T2.a was chasing, on the + plugin's hot loop, for an unmeasured cost. Decision needs a profile and a confirmed + cross-platform requirement. + +**UNVERIFIED:** the test's new ULP-tolerance branch (expect `MazeEquivalence` to report the same 454 +samples as INFO rather than WARNING next run). + +**Next single action:** Phase 1 step 3 — wire the stack into `GetDensityAt` behind a per-strate +opt-in. Phase 1's question is fully answered: Maze decomposes cleanly, the stack is +window-invariant, and it proves 23/60 tiles uniform where `ClassifyTile` proves zero. + +--- diff --git a/Source/VoxelForge/Private/Tests/VoxelForgeOpStackMazeTest.cpp b/Source/VoxelForge/Private/Tests/VoxelForgeOpStackMazeTest.cpp index 4eeef6d..a1dd6c8 100644 --- a/Source/VoxelForge/Private/Tests/VoxelForgeOpStackMazeTest.cpp +++ b/Source/VoxelForge/Private/Tests/VoxelForgeOpStackMazeTest.cpp @@ -128,8 +128,34 @@ bool FVoxelForgeOpStackMazeTest::RunTest(const FString& Parameters) } // ── L'ÉQUIVALENCE. ── + // + // ⚠️ CE QUE « ÉQUIVALENT » PEUT VOULOIR DIRE ICI — conclusion mesurée, 2026-07-27. + // Le plugin est compilé en **/fp:fast** : c'est le défaut d'UnrealBuildTool sur Windows + // (`VCToolChain.cs` : `case FPSemanticsMode.Default: // Default is imprecise FP semantics`), + // et la doc de UBT le dit noir sur blanc : « FP math isn't IEEE-754 compliant: the compiler is + // allowed to transform math expressions in ways that might result in differently rounded + // results ». Le compilateur a donc le DROIT de réassocier/contracter la MÊME expression + // différemment selon l'unité de compilation et le contexte d'inlining. + // + // Donc : deux transcriptions littérales du même calcul, l'une dans VoxelGenerator.cpp et + // l'autre dans VoxelDensityOpStack.cpp, peuvent légitimement différer de ~1 ULP. + // **L'identité binaire n'est PAS atteignable en principe pour ces portages**, et ce n'est pas + // un défaut de la décomposition. C'est mesuré, pas supposé : le bisect ci-dessous a montré + // l'écart survivant jusqu'à « corridors + carve ONLY », c'est-à-dire du code identique + // caractère pour caractère. + // + // Le critère d'acceptation est donc celui que OPSTACK-PLAN §2.6 demandait déjà : + // • DUR : aucun échantillon ne change de CÔTÉ de l'isosurface (sinon la géométrie bouge) ; + // • SOUPLE: les écarts restent à l'échelle de l'ULP. Un écart plus grand n'est PAS du bruit + // de compilateur — c'est une vraie dérive de portage, et là il faut chercher. + // + // The plugin builds with /fp:fast (UBT's Windows default), which explicitly licenses the + // compiler to reassociate identical source differently per translation unit. Bit-identity is + // therefore NOT achievable in principle for these ports. Hard gate: no isosurface crossings. + // Soft gate: differences stay at ULP scale — anything larger is real drift, not compiler noise. int32 NumDiff = 0, WorstIdx = -1; float WorstDelta = 0.0f; + int32 NumBeyondUlpNoise = 0; // écarts TROP GRANDS pour être du bruit de compilateur int32 NumSolidDisagreements = 0; // le seul écart qui compte VRAIMENT : un côté d'iso différent for (int32 i = 0; i < NumMazeSamples; ++i) { @@ -143,6 +169,12 @@ bool FVoxelForgeOpStackMazeTest::RunTest(const FString& Parameters) ++NumDiff; const float Delta = FMath::Abs(Old - New); if (Delta > WorstDelta) { WorstDelta = Delta; WorstIdx = i; } + + // Tolérance : quelques ULP à la magnitude locale. `Blend - Sdf` amplifie fortement un + // écart d'ULP sur le SDF quand on est au bord de la zone de blend (annulation + // catastrophique), d'où une marge généreuse — mais bornée. + const float UlpNoise = 16.0f * FMath::Max(FMath::Abs(Old), 1.0f) * FLT_EPSILON; + if (Delta > UlpNoise) { ++NumBeyondUlpNoise; } } // Le mesher ne lit que le SIGNE (D >= IsoLevel ⇒ air). Deux valeurs peuvent différer d'un // ULP sans changer un seul triangle ; un désaccord de CÔTÉ change la géométrie. @@ -157,17 +189,35 @@ bool FVoxelForgeOpStackMazeTest::RunTest(const FString& Parameters) TEXT("GetMazeDensity exactly, which is as strong a signal as Phase 1 can get that the ") TEXT("source/modifier split is real and not imposed."), NumMazeSamples)); } + else if (NumBeyondUlpNoise == 0) + { + // Attendu, et compris. Pas un avertissement : crier au loup à chaque portage ferait + // ignorer le jour où l'écart est réel. + AddInfo(FString::Printf( + TEXT("%d of %d samples differ, ALL at ULP scale (largest |delta| %.9g at (%.0f, %.0f, ") + TEXT("%.0f)), and 0 cross the isosurface -- so not one triangle would move. This is the ") + TEXT("expected floor: the plugin builds with /fp:fast (UnrealBuildTool's Windows ") + TEXT("default -- VCToolChain.cs, \"Default is imprecise FP semantics\"), which lets the ") + TEXT("compiler reassociate identical source differently per translation unit. The ") + TEXT("bisect below confirmed it empirically: the residue survives into \"corridors + ") + TEXT("carve ONLY\", which is character-for-character transcribed code. Bit-identity is ") + TEXT("not achievable in principle here; OPSTACK-PLAN 2.6's bar (same PLACE, not same ") + TEXT("bits) is the right one and it is met."), + NumDiff, NumMazeSamples, WorstDelta, + WorstIdx >= 0 ? Points[WorstIdx].X : 0.0f, + WorstIdx >= 0 ? Points[WorstIdx].Y : 0.0f, + WorstIdx >= 0 ? Points[WorstIdx].Z : 0.0f)); + } else { AddWarning(FString::Printf( - TEXT("%d of %d samples differ (largest |delta| %.9g at (%.0f, %.0f, %.0f)); %d of them ") - TEXT("land on the OPPOSITE side of the isosurface. OPSTACK-PLAN section 2.6 does not ") - TEXT("require bit-identity, so this is a warning, not a failure -- but Maze SHOULD be ") - TEXT("reproducible exactly, so a nonzero count means the port drifted somewhere. Check, ") - TEXT("in order: the roughness apply-window (R + SurfaceRoughness + 2), the carve blend ") - TEXT("(2.0), the noise frequency (0.12) and octave count (3), and the order of the ") - TEXT("structural post ops."), - NumDiff, NumMazeSamples, WorstDelta, + TEXT("%d of %d samples differ and %d of them are TOO LARGE to be /fp:fast rounding ") + TEXT("noise (largest |delta| %.9g at (%.0f, %.0f, %.0f)); %d cross the isosurface. ") + TEXT("Unlike the ULP-scale floor, this IS port drift. Check, in order: the roughness ") + TEXT("apply-window (R + SurfaceRoughness + 2), the carve blend (2.0), the noise ") + TEXT("frequency (0.12) and octave count (3), and the order of the structural post ops. ") + TEXT("The bisect below narrows it to a stage."), + NumDiff, NumMazeSamples, NumBeyondUlpNoise, WorstDelta, WorstIdx >= 0 ? Points[WorstIdx].X : 0.0f, WorstIdx >= 0 ? Points[WorstIdx].Y : 0.0f, WorstIdx >= 0 ? Points[WorstIdx].Z : 0.0f,