refactor(core): bind generator fields once per resolved strate (L40, migration step 2); research R7
SurfaceWorld, FloatingIslands and legacy evaluators sit behind one immutable densityAt adapter held in a variant (no virtual call per sample); BindResolvedStratumFields builds a sorted strate table at world setup, so DensityAt, native SW/FI sampling and ClassifyBoxDetailed no longer ask the strate manager per sample. No arithmetic change: owner hashes identical to L37/L39 at W1/W12. Cost vs L39: FI -0.66%, SW -1.31% (noise). verify fast/medium/full PASS (full re-run alone: 407 gates). Research R7: disk cache design for UE5/Windows/multiplayer and a zlib probe (about 1/3 size, 0.1 ms/tile). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016dhxM43dnWvUu2B5Ph1iG1
This commit is contained in:
1 parent
753cc9c259
commit
efa9e89468
8 files changed
+859
-118
No files matched your search
@@ -0,0 +1,45 @@
|
||||
# Research 7 — terrain disk cache: Windows and multiplayer
|
||||
|
||||
Inspection date: 2026-10-23. Read-only review of `Standalone/Source/StreamingWorld.h`, `Standalone/Source/StandaloneViewer.cpp`, Core version declarations, and the verification README.
|
||||
|
||||
## What exists and Windows blockers
|
||||
|
||||
`FStandaloneMeshCache` is a whole-world archive, not an on-demand disk cache. Its filename is derived from seed, clip radius, and tunnel mode; the header also checks source JSON FNV-1a-64, generation settings, cache/algorithm version, tunnel mode, and voxel size. A tile lookup uses `(LOD, X, Y, Z)`. The `VFMESH1\0` v1 archive has an 80-byte raw C++ header, 40-byte raw tile records, float32 position/normal pairs, and uint32 indices. It is uncompressed and host-layout/endian dependent; the endian marker detects mismatch but does not convert fields. Save/load also omit generated `SurfaceVertexCount`, `SkirtTriangleCount`, and `TriangleSkirtFaceMasks`, so a cache hit cannot preserve skirt culling.
|
||||
|
||||
Load reads and validates the entire archive synchronously in `ConfigureProfile` before workers start. Streaming workers then copy hits from an immutable in-memory map. Bake save writes synchronously to the final path with truncation. There is no disk eviction or file lock; leaving tiles evicts resident meshes only. The cache has no `mmap`, `fsync`, rename, or zlib use today. Existing checks reject mismatched metadata, invalid LOD/counts, truncated data, duplicate keys, non-finite positions/normals, and out-of-range indices, but there is no payload checksum, total allocation budget, or trailing-data check. Stored density hashes are not checked against the mesh.
|
||||
|
||||
Windows concerns are the narrow `.string()` path conversion and a fixed `MAX_PATH` buffer in `DefaultDataPath`; the cache is written next to source data, which may be read-only in a packaged install. Windows long-path support needs application opt-in, and the cache path should stay in TCHAR/UE paths. Direct-to-final truncation can leave a partial archive and destroy the previous one; `ofstream::flush` is not a durable or atomic replacement. Concurrent bakes can collide. The standalone viewer itself has unguarded Win32/D3D11 headers and should not be pulled into the UE plugin; keep any platform adapter gated and use UE file APIs.
|
||||
|
||||
## UE5 design
|
||||
|
||||
- Store a versioned, endian-neutral per-tile payload under `FPaths::ProjectSavedDir()/VoxelForge/TileCache/<project-id>/<user-scope>/`. Use `ProjectPersistentDownloadDir()` when the packaged install’s Saved directory is not writable; append a project identity either way. [FPaths API](https://dev.epicgames.com/documentation/en-us/unreal-engine/API/Runtime/Core/FPaths)
|
||||
- Use `FPlatformFileManager::Get().GetPlatformFile().OpenAsyncRead()` and `IAsyncReadFileHandle::ReadRequest()` for tile/block reads. Parse and decompress on a worker; never wait for file I/O or decompression on the game thread. UE documents async open as non-blocking and read requests as caller-owned; keep buffers/requests alive through completion and cancellation. [FPlatformFileManager](https://dev.epicgames.com/documentation/unreal-engine/API/Runtime/Core/FPlatformFileManager), [IPlatformFile](https://dev.epicgames.com/documentation/unreal-engine/API/Runtime/Core/IPlatformFile), [ReadRequest](https://dev.epicgames.com/documentation/unreal-engine/API/Runtime/Core/Async/IAsyncReadFileHandle/ReadRequest?application_version=5.5)
|
||||
- Use UE `FCompression` with registered format names and `IsFormatValid`; its compression/decompression routines are thread-safe. This avoids a separately linked Windows zlib. Keep codec ID/version in each tile record and check availability in the packaged client/server. Oodle Data is supplied as a UE plugin and documents Kraken, Mermaid, and Selkie as distinct size/decode-speed choices; do not assume a packaging setting alone enables a runtime cache codec. [FCompression](https://dev.epicgames.com/documentation/en-us/unreal-engine/API/Runtime/Core/FCompression), [Oodle Data (UE 5.7)](https://dev.epicgames.com/documentation/en-us/unreal-engine/oodle-data?application_version=5.7)
|
||||
- Treat the generated terrain as persisted feature data, not as a speed claim. Record schema, world key, tile key, codec, compressed/uncompressed sizes, and payload digest. Include all mesh groups, masks, and collision inputs required to rebuild identical tiles. Write a unique temp beside the target, flush and close it, then replace/rename on the same volume. Do not allow a cross-volume copy fallback. Use `ReplaceFileW`/same-volume `MoveFileExW` through a Windows adapter; `FlushFileBuffers` is the file-data durability step. [ReplaceFileW](https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-replacefilew), [MoveFileExW](https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-movefileexw), [FlushFileBuffers](https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-flushfilebuffers), [Windows long paths](https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation)
|
||||
|
||||
Cache misses, corrupt entries, and unavailable codecs fall back to cold generation on workers. Keep eviction as a background size-bounded LRU; cache eviction only causes regeneration. Continue judging the no-cache cold path separately.
|
||||
|
||||
## Key, multiplayer, invalidation, validation
|
||||
|
||||
Key a base tile by authoritative season/world-plan identity (include Core recipe and plan content hashes), recipe/schema version, evaluator version, selected mesher plus an explicit mesher implementation revision, LOD and absolute tile coordinate. Add the development source stamp and cache-format version. Core currently declares WorldAssembly schema 2/evaluator 208; FloatingIsland recipe schema 1/default geometry version 4; mesher choices are MarchingCubesV1, HybridV1, HybridV2, with far-horizon version 4 affecting LOD4–8. Read actual resolved recipe/plan versions and hashes from the active snapshot. The verification README defines a SHA-256 source stamp over a sorted source manifest; it is dynamic and can optionally include the Unreal adapter. The current cache uses only the source-data FNV hash and does not carry that stamp.
|
||||
|
||||
Persist immutable base terrain only. Keep player edits in the replicated diff layer and overlay them at runtime. Do not write edited composite tiles into the base cache; if a separate composite cache is ever added, key it by the affected tile’s edit revision/content hash. A client cache is visual convenience only: the server uses its own matching cache or regenerates and builds its own collision; never accept client cache bytes as collision or gameplay authority.
|
||||
|
||||
On load, bound both per-record and total tile/byte allocations before allocating; require exact file/block lengths, known codec/schema/key, finite and bounded geometry, valid normals, triangle/index/mask counts, and index ranges. Verify a per-payload digest and regenerate on failure. A digest detects accidental corruption, not a malicious rewrite that also changes the digest; only importable/shared caches need a server signature or MAC. Never trust a cache supplied by a client.
|
||||
|
||||
## Compression probe and next build round
|
||||
|
||||
A small serial Python probe used `VFMSH001` exports `linux27-fi-anchor-pillar.vfmesh` and `linux27-sw-column-wide.vfmesh`, all non-empty tiles LOD0–8, CPython zlib level 1. Payload excludes scene/tile headers. These are Linux in-memory decode numbers, not UE/MSVC or disk-I/O results; exports lack recipe/source stamps and density samples.
|
||||
|
||||
| Export | Non-empty tiles | Mean raw → zlib bytes/tile | Payload ratio | Decode ms/tile p50 / p95 |
|
||||
|---|---:|---:|---:|---:|
|
||||
| FI | 1,380 | 75.6 KB → 24.2 KB | 32.0% | 0.109 / 0.523 |
|
||||
| SW | 1,416 | 60.0 KB → 21.5 KB | 35.8% | 0.114 / 0.459 |
|
||||
|
||||
The serial probe peak RSS was 15.4 MiB. Per-LOD details are in `Saved/Linux/research/R7_python_zlib_probe.json`; script: `Saved/Linux/research/R7_python_zlib_probe.py`.
|
||||
|
||||
Later compare none, zlib, Oodle Kraken/Mermaid/Selkie, and LZ4; quantized vertices (uint16 positions, compact normals) with delta indices; and density/sign-bit packing with RLE. Use the same owner FI and SW tile sets at LOD0–8, keyed to the exact active recipe/plan. Record bytes/tile (median/p95/total), single-core decode ms/tile (median/p95), peak working memory, and cold disk-read-plus-decode latency separately. Validate geometry error, seams, and server collision against uncompressed output. Sign-bit inputs are absent from these mesh exports, so that candidate needs a later density export and must include remesh time in its comparison.
|
||||
|
||||
## Owner question
|
||||
|
||||
Will cache files remain local to each user/install, or be copied/shared between machines?
|
||||
Reference in new issue
Block a user