diff --git a/RENDER.md b/RENDER.md index 4821efd..5ab2d43 100644 --- a/RENDER.md +++ b/RENDER.md @@ -1060,7 +1060,7 @@ $exe = Join-Path $env:TEMP 'godot-4.6.1-openwc\Godot_v4.6.1-stable_win64.exe' - Invalid batch variants and combo/transform indices remain skipped. The first decision for a normalized path remains cached until the same two historical map/shutdown clear sites. -- `StreamingWorldLoader` still loads raw M2 data and owns ResourceLoader, +- `StreamingWorldLoader` still requests raw M2 data and owns ResourceLoader, returned-Mesh adoption and finalization budgets. Refresh-version checks, M2Builder rebuild and fallback now belong to `M2RuntimeMeshFinalizer`. - No cache version, shader, material, geometry or visible rule changed. Synthetic @@ -1102,7 +1102,7 @@ $exe = Join-Path $env:TEMP 'godot-4.6.1-openwc\Godot_v4.6.1-stable_win64.exe' Node subtrees using the prior depth-first child order. - PackedScene extraction still instantiates exactly once and synchronously frees the temporary root before returning the borrowed Mesh reference. -- `StreamingWorldLoader` still owns ResourceLoader I/O, raw M2 loading, +- `StreamingWorldLoader` still owns ResourceLoader I/O and raw M2 consumers, material refresh/rebuild, cache/missing adoption, permits and MultiMesh builds. - Cache formats, traversal rules, profiles and visible output are unchanged. Synthetic traversal timing is not asset-backed leak or p95/p99 evidence. @@ -1111,15 +1111,29 @@ $exe = Join-Path $env:TEMP 'godot-4.6.1-openwc\Godot_v4.6.1-stable_win64.exe' - `M2RuntimeMeshFinalizer` now owns material refresh version `2`, stale-Mesh rebuild classification, M2Builder rebuild and original-Mesh fallback. -- Current Meshes still skip raw `.m2` loading. The loader retains FileAccess/ - ClassDB M2Loader I/O and supplies raw data only when the finalizer reports a - stale Mesh; both historical decision-cache clear sites are preserved. +- Current Meshes still skip raw `.m2` loading. `M2RawModelRepository` now owns + FileAccess/ClassDB M2Loader I/O and supplies raw data through the loader only + when the finalizer reports a stale Mesh; both historical clear sites persist. - Billboard/UV-rotation predicates, rebuild extraction, metadata key and failure fallback are unchanged. Cache adoption, missing state, permits and MultiMesh materialization remain loader-owned. - Synthetic triangle rebuild/fallback timing is not asset-backed material, descriptor-pressure/leak or p95/p99 evidence. +## 2026-07-17 M2 Raw Model Repository + +- `M2RawModelRepository` now owns the repeated extracted-file and optional + native `M2Loader` boundary for static `load_m2` and animated + `load_m2_animated` raw Dictionaries. +- `StreamingWorldLoader` delegates the stale-Mesh refresh, static prototype and + native animated prototype reads. It retains normalization, `.tscn/.glb` + fallback order, missing/prototype caches, builders, permits and Node/Mesh use. +- The repository retains no path, native object or parsed data. Empty paths, + absent files/classes/methods and invalid results produce the same empty-value + fallback contract; path join/globalization and native method names are exact. +- No cache format, parser, material, geometry, animation or visible rule changed. + Synthetic missing-input timing is not asset-backed I/O/p95/p99 or fidelity evidence. + ## 2026-07-17 WMO Placement Resolver Extraction - `WmoPlacementResolver` now owns lowercase/slash cache-key normalization, diff --git a/docs/modules/README.md b/docs/modules/README.md index e28919e..2bf662a 100644 --- a/docs/modules/README.md +++ b/docs/modules/README.md @@ -26,6 +26,7 @@ | M2 mesh resource cache state | Implemented extraction | [`m2-mesh-resource-cache-state.md`](m2-mesh-resource-cache-state.md) | | M2 mesh resource extractor | Implemented extraction | [`m2-mesh-resource-extractor.md`](m2-mesh-resource-extractor.md) | | M2 runtime mesh finalizer | Implemented extraction | [`m2-runtime-mesh-finalizer.md`](m2-runtime-mesh-finalizer.md) | +| M2 raw model repository | Implemented extraction | [`m2-raw-model-repository.md`](m2-raw-model-repository.md) | | WMO placement resolver | Implemented extraction | [`wmo-placement-resolver.md`](wmo-placement-resolver.md) | | WMO placement registry | Implemented extraction | [`wmo-placement-registry.md`](wmo-placement-registry.md) | | WMO render build step planner | Implemented extraction | [`wmo-render-build-step-planner.md`](wmo-render-build-step-planner.md) | diff --git a/docs/modules/m2-raw-model-repository.md b/docs/modules/m2-raw-model-repository.md new file mode 100644 index 0000000..9da0df0 --- /dev/null +++ b/docs/modules/m2-raw-model-repository.md @@ -0,0 +1,224 @@ +# M2 Raw Model Repository + +## Metadata + +| Field | Value | +|---|---| +| Status | Implemented extraction | +| Target/work package | M03 / `M03-RND-M2-RAW-MODEL-REPOSITORY-001` | +| Owners | Native M2Loader availability, file existence and raw Dictionary loading | +| Last verified | Worktree `work/sindo-main-codex/m03-m2-raw-model-repository`, 2026-07-17 | +| Profiles/capabilities | Static and native-animated raw M2 reads | + +## Purpose + +Provide one stateless boundary between renderer orchestration and the optional +native `M2Loader`. The repository resolves an extracted path and returns the raw +static or animated Dictionary consumed by existing builders and classifiers. + +## Non-goals + +- Normalize relative paths or choose `.tscn`/`.glb` cache fallbacks. +- Cache successful data, missing paths, Meshes, Nodes or animation decisions. +- Build geometry/materials, own queues or attach renderer objects. +- Change native parsing, cache formats, retry policy or visible behavior. + +## Context and boundaries + +```mermaid +flowchart LR + Loader[StreamingWorldLoader] --> Repository[M2RawModelRepository] + Repository --> File[Extracted M2 file] + Repository --> Native[ClassDB M2Loader] + Native --> Raw[Raw Dictionary] + Raw --> Loader + Loader --> Builder[Existing M2 builders/finalizer] +``` + +Allowed dependencies are `ProjectSettings`, `FileAccess` and `ClassDB` for the +native file boundary. Resource caches, builders, SceneTree types, worker pools, +renderer policy and other application layers are forbidden. + +## Public API + +| Symbol | Kind | Purpose | Thread/lifetime | Errors | +|---|---|---|---|---| +| `load_static_model_data(extracted_directory, normalized_relative_path)` | Query | Invoke native `load_m2` for one existing extracted file | Synchronous; stateless | Returns `{}` for rejected/unavailable/invalid result | +| `load_animated_model_data(extracted_directory, normalized_relative_path)` | Query | Invoke native `load_m2_animated` for one existing extracted file | Synchronous; stateless | Returns `{}` for rejected/unavailable/invalid result | + +## Inputs and outputs + +| Direction | Contract/data | Producer | Consumer | Ownership | Thread/lifetime | +|---|---|---|---|---|---| +| Input | Extracted directory String | Loader configuration | Repository path resolution | Copied value | One call | +| Input | Already-normalized relative M2 path String | Loader normalization | Repository | Copied value | One call | +| Input | Extracted `.m2` bytes | Local legal extraction | Native M2Loader | File-owned | Native call | +| Output | Static raw M2 Dictionary | Native `load_m2` | Loader/finalizer/M2Builder | Fresh native result | One caller operation | +| Output | Animated raw M2 Dictionary | Native `load_m2_animated` | Loader/animated builder | Fresh native result | One caller operation | +| Output | Empty Dictionary | Repository guards | Loader fallback/missing state | Fresh value | One failed call | + +Side effects are limited to file-existence inspection, synchronous native file +parsing and creation of one temporary native loader instance per accepted call. + +## Data flow + +```mermaid +flowchart TD + Request[Directory plus normalized path] --> Empty{Path empty?} + Empty -->|yes| Reject[Return empty Dictionary] + Empty -->|no| Class{M2Loader registered?} + Class -->|no| Reject + Class -->|yes| Resolve[globalize directory/path join] + Resolve --> Exists{File exists?} + Exists -->|no| Reject + Exists -->|yes| Instance[Instantiate M2Loader] + Instance --> Method{Instance and method available?} + Method -->|no| Reject + Method -->|yes| Parse[Call static or animated method] + Parse --> Type{Dictionary result?} + Type -->|no| Reject + Type -->|yes| Return[Return raw Dictionary] +``` + +## Lifecycle/state + +```mermaid +stateDiagram-v2 + [*] --> Ready: construct repository + Ready --> Resolving: load request + Resolving --> Ready: rejected input/unavailable dependency + Resolving --> Parsing: file and native method available + Parsing --> Ready: Dictionary or empty result returned + Ready --> [*]: caller releases reference +``` + +No request, result or failure state survives a call. + +## Main sequence + +```mermaid +sequenceDiagram + participant Loader as StreamingWorldLoader + participant Repo as M2RawModelRepository + participant File as FileAccess + participant Native as M2Loader + Loader->>Repo: load static/animated(directory, path) + Repo->>File: file_exists(globalized joined path) + alt dependency or file unavailable + Repo-->>Loader: empty Dictionary + else available + Repo->>Native: instantiate and call exact native method + Native-->>Repo: Variant + Repo-->>Loader: Dictionary or empty Dictionary + end +``` + +## Dependency diagram + +```mermaid +flowchart TB + Loader[StreamingWorldLoader] --> Repository[M2RawModelRepository] + Repository --> ProjectSettings + Repository --> FileAccess + Repository --> ClassDB + ClassDB --> Native[M2Loader extension] + Loader --> Finalizer[M2RuntimeMeshFinalizer] + Loader --> StaticBuilder[M2Builder] + Loader --> AnimatedBuilder[M2NativeAnimatedBuilder] + Repository -. no dependency .-> Finalizer + Repository -. no dependency .-> StaticBuilder + Repository -. no dependency .-> Cache[Renderer caches/queues] +``` + +## Ownership, threading and resources + +- The repository owns only call-local path, native instance and result values. +- The loader owns path normalization, cache/fallback selection and adoption. +- Native `M2Loader` owns parsing behavior and returns a new Dictionary value. +- Calls are synchronous on the caller's thread; current renderer callers use the + main thread. The module starts no tasks and touches no SceneTree objects. +- No file handle, native instance, Dictionary or engine Resource is retained. + +## Errors, cancellation and recovery + +| Failure | Detection | Behavior | Diagnostic | Recovery | +|---|---|---|---|---| +| Empty normalized path | String guard | Return `{}` | Dedicated verifier | Correct caller normalization | +| Native extension absent | `ClassDB.class_exists` | Return `{}` | Existing renderer fallback | Install/load extension and retry | +| Extracted file absent | `FileAccess.file_exists` | Return `{}` | Loader adopts historical missing state | Restore extraction and clear caller state | +| Instance/method absent | Null/`has_method` guard | Return `{}` | Source contract verifier | Repair extension registration | +| Parse/non-Dictionary result | Result type check | Return `{}` | Existing loader fallback | Repair asset/parser and retry | +| Cancellation requested | Not supported inside synchronous native call | Call completes | None | Caller controls whether to start call | + +## Configuration and capabilities + +| Setting/capability | Default | Profile | Runtime mutable | Effect | +|---|---|---|---|---| +| Native class name | `M2Loader` | All | No | Exact GDExtension registration | +| Static method | `load_m2` | All | No | Existing static parser path | +| Animated method | `load_m2_animated` | Experimental animated path | No | Existing native animation parser | +| Extracted directory | Loader `extracted_dir` | Scene configuration | Yes, caller-owned | Root used for path join | + +## Persistence, cache and migration + +The module adds no persistence or cache. Native methods, normalized relative +paths and `.tscn/.glb`/material versions are unchanged, so no migration or rebake +is required. + +## Diagnostics and observability + +- Logs: none; loader retains existing fallback and debug output. +- Metrics: none; rejected-input timing is verifier-only. +- Debug views: none. +- Correlation ID: normalized relative M2 path in the caller. + +## Verification + +- `verify_m2_raw_model_repository.gd` covers empty/missing/repeated failures, + exact static/animated method ownership, three loader delegations, forbidden + dependencies and 10,000 empty-path rejects under one second. +- Adjacent finalizer/extractor/cache/pipeline/animation/material/shutdown tests + protect caller behavior and ownership. +- Fidelity evidence is structural: exact path join/globalization and native + method names are preserved. No proprietary M2 or original-client visual claim + is made by this extraction. + +## Extension points + +Future asset-backed verification may add a legal fixture around these value +contracts. Caching, asynchronous I/O and parser replacement require separate +measured work packages rather than expansion of this repository. + +## Capability status + +| Capability | Status | Evidence | Gap/next step | +|---|---|---|---| +| Static raw M2 read | Implemented extraction | Source/invalid/missing verifier | Legal success fixture pending | +| Native animated raw M2 read | Implemented extraction | Exact method/source contract | Legal animated fixture pending | +| Stateless failure behavior | Verified synthetic | Repeated missing call and source boundary | Asset-backed corrupt input pending | + +## Known gaps and risks + +- Native parsing remains synchronous and has no mid-call cancellation. +- The repository deliberately does not distinguish missing dependency, missing + file and parser failure; loader fallback behavior historically treats them alike. +- Proprietary asset traversal, descriptor pressure, p95/p99 and visual comparison + remain outside this structural extraction. + +## Source map + +| Path | Responsibility | +|---|---| +| `src/render/m2/m2_raw_model_repository.gd` | Stateless native class/file/method boundary | +| `src/scenes/streaming/streaming_world_loader.gd` | Path normalization, fallback/cache state and result consumers | +| `src/native/src/m2_loader.cpp` | Native static/animated parsing implementation | +| `src/tools/verify_m2_raw_model_repository.gd` | Rejected-input/source/dependency/timing regression | + +## Related decisions and references + +- [`m2-runtime-mesh-finalizer.md`](m2-runtime-mesh-finalizer.md) +- [`m2-mesh-resource-cache-state.md`](m2-mesh-resource-cache-state.md) +- [`m2-mesh-load-pipeline-state.md`](m2-mesh-load-pipeline-state.md) +- [`world-renderer.md`](world-renderer.md) +- [`../../RENDER.md`](../../RENDER.md) +- [`../../targets/roadmap/02-rendering-and-graphics.md`](../../targets/roadmap/02-rendering-and-graphics.md) diff --git a/docs/modules/m2-runtime-mesh-finalizer.md b/docs/modules/m2-runtime-mesh-finalizer.md index 5307d54..a7484c9 100644 --- a/docs/modules/m2-runtime-mesh-finalizer.md +++ b/docs/modules/m2-runtime-mesh-finalizer.md @@ -29,7 +29,7 @@ UV-rotation cases and returning the historical original-Mesh fallback. flowchart LR Extractor[M2MeshResourceExtractor] --> Loader[StreamingWorldLoader] Loader -->|is raw data required?| Finalizer[M2RuntimeMeshFinalizer] - Loader --> Raw[M2Loader raw Dictionary] + Loader --> Raw[M2RawModelRepository] Raw --> Finalizer Finalizer --> Classifier[M2RuntimeMeshRebuildClassifier] Finalizer --> Builder[M2Builder] @@ -55,7 +55,7 @@ Nodes outside temporary rebuild roots and other application layers are forbidden | Direction | Contract/data | Producer | Consumer | Ownership | Thread/lifetime | |---|---|---|---|---|---| | Input | Extracted Mesh and normalized M2 path | Loader/extractor adapter | Finalizer | Borrowed Mesh reference/String copy | One call | -| Input | Already-loaded raw M2 Dictionary | Loader raw-file boundary | Classifier/M2Builder | Caller-owned value container | One call | +| Input | Already-loaded raw M2 Dictionary | `M2RawModelRepository` through loader adapter | Classifier/M2Builder | Caller-owned value container | One call | | Input | Extracted directory path | Loader configuration | M2Builder texture resolution | Copied String | One rebuild | | Output | Original or rebuilt current Mesh | Finalizer | Cache/prototype adapter | Borrowed/new Resource reference | Cache may retain | @@ -68,7 +68,7 @@ M2Builder prototype creation/destruction during required rebuilds. flowchart TD Mesh[Extracted Mesh] --> Current{Refresh version >= 2?} Current -->|yes| ReturnOriginal[Return original unchanged] - Current -->|no| NeedRaw[Loader supplies raw M2 data] + Current -->|no| NeedRaw[Repository supplies raw M2 data through loader] NeedRaw --> HasData{Raw data available?} HasData -->|no| Mark[Mark original version 2] HasData -->|yes| Classify{Billboard or UV rotation?} @@ -111,7 +111,7 @@ sequenceDiagram Finalizer-->>Loader: false; reuse Mesh else stale Mesh Finalizer-->>Loader: true - Loader->>Raw: load raw Dictionary + Loader->>Raw: load static raw Dictionary Loader->>Finalizer: finalize_mesh(path, mesh, raw, extracted dir) Finalizer->>Classifier: needs_runtime_mesh_rebuild(path, raw) opt rebuild required @@ -130,7 +130,7 @@ flowchart TB Finalizer --> Classifier[M2RuntimeMeshRebuildClassifier] Finalizer --> Extractor[M2MeshResourceExtractor] Finalizer --> Builder[M2Builder] - Loader --> Raw[FileAccess / ClassDB M2Loader] + Loader --> Raw[M2RawModelRepository] Loader --> Cache[M2MeshResourceCacheState] Finalizer -. no dependency .-> Raw Finalizer -. no dependency .-> Cache @@ -139,7 +139,7 @@ flowchart TB ## Ownership, threading and resources - Renderer main thread executes metadata changes and M2Builder work. -- Loader owns raw M2 file existence/load calls and supplies value data. +- `M2RawModelRepository` owns raw M2 file/native calls; loader supplies its value result. - Finalizer owns classifier memoization and temporary rebuild prototype lifetime. - M2Builder owns construction rules; extractor selects the first rebuilt Mesh. - Cache/prototype adapters decide where the returned Mesh reference is retained. @@ -183,8 +183,8 @@ logs/queue metrics are unchanged. Normalized M2 path remains the correlation key ## Extension points -- Raw M2 file loading may later move behind a dedicated repository without - changing the finalizer's value-data contract. +- Asset-backed fixture coverage can be added without changing the finalizer's + value-data contract. - Refresh versions above `2` require explicit cache/fidelity evidence and docs. ## Capability status @@ -193,7 +193,7 @@ logs/queue metrics are unchanged. Normalized M2 path remains the correlation key |---|---|---|---| | Runtime Mesh refresh/fallback | Implemented extraction | Synthetic transition/rebuild verifier | Asset-backed corrupt/stale cache pending | | Rebuild predicate | Implemented extraction dependency | Classifier fixtures | Original-client material comparison pending | -| Raw M2 refresh I/O | Existing loader-owned | Source boundary | Separate repository extraction optional | +| Raw M2 refresh I/O | Implemented repository dependency | Repository source/invalid verifier | Legal success fixture pending | ## Known gaps and risks @@ -207,7 +207,8 @@ logs/queue metrics are unchanged. Normalized M2 path remains the correlation key | Path | Responsibility | |---|---| | `src/render/m2/m2_runtime_mesh_finalizer.gd` | Refresh version, classification, rebuild and fallback | -| `src/scenes/streaming/streaming_world_loader.gd` | Raw-file I/O and returned-Mesh adoption | +| `src/render/m2/m2_raw_model_repository.gd` | Raw-file/native I/O and Dictionary result | +| `src/scenes/streaming/streaming_world_loader.gd` | Repository call and returned-Mesh adoption | | `src/render/m2/m2_runtime_mesh_rebuild_classifier.gd` | Memoized billboard/UV-rotation predicate | | `src/render/m2/m2_mesh_resource_extractor.gd` | First rebuilt-Mesh selection | | `src/tools/verify_m2_runtime_mesh_finalizer.gd` | Transition/rebuild/boundary/timing regression | @@ -217,6 +218,7 @@ logs/queue metrics are unchanged. Normalized M2 path remains the correlation key - [`m2-runtime-mesh-rebuild-classifier.md`](m2-runtime-mesh-rebuild-classifier.md) - [`m2-mesh-resource-extractor.md`](m2-mesh-resource-extractor.md) - [`m2-mesh-resource-cache-state.md`](m2-mesh-resource-cache-state.md) +- [`m2-raw-model-repository.md`](m2-raw-model-repository.md) - [`world-renderer.md`](world-renderer.md) - [`../../RENDER.md`](../../RENDER.md) - [`../../targets/roadmap/02-rendering-and-graphics.md`](../../targets/roadmap/02-rendering-and-graphics.md) diff --git a/docs/modules/m2-runtime-mesh-rebuild-classifier.md b/docs/modules/m2-runtime-mesh-rebuild-classifier.md index db1f0ca..517c050 100644 --- a/docs/modules/m2-runtime-mesh-rebuild-classifier.md +++ b/docs/modules/m2-runtime-mesh-rebuild-classifier.md @@ -56,7 +56,7 @@ WorkerThreadPool, Mesh/Node/RID ownership and other layers are forbidden. | Direction | Contract/data | Producer | Consumer | Ownership | Thread/lifetime | |---|---|---|---|---|---| | Input | Normalized M2 relative path | Loader cache lookup | Classifier cache key | String value copied | Until clear | -| Input | Raw M2 batches/transforms/combos Dictionary | Native `M2Loader` adapter | Classifier | Borrowed; not mutated | First classification call per path | +| Input | Raw M2 batches/transforms/combos Dictionary | `M2RawModelRepository` through loader adapter | Classifier | Borrowed; not mutated | First classification call per path | | Output | Rebuild-required boolean | Classifier | Runtime Mesh finalizer | Scalar | Memoized until clear | | Output | Detached decision Dictionary | Classifier | Verifier/future diagnostics | Caller-owned copy | Snapshot lifetime | diff --git a/docs/modules/world-renderer.md b/docs/modules/world-renderer.md index d08aaf9..c943828 100644 --- a/docs/modules/world-renderer.md +++ b/docs/modules/world-renderer.md @@ -7,7 +7,7 @@ | Status | Partial | | Target/work package | M00 baseline; `M01-RND-STREAMING-FOCUS-001`; `M01-QAR-SERVER-SPAWN-RENDERER-001`; M03 facade/planner/scheduler/internal-access/ground/environment/entity packages; M03 terrain packages; M03 M2 packages; M03 WMO placement package | | Owners | Renderer workstream / milestone integrator | -| Last verified | Worktree `work/sindo-main-codex/m03-m2-runtime-mesh-finalizer`, 2026-07-17 | +| Last verified | Worktree `work/sindo-main-codex/m03-m2-raw-model-repository`, 2026-07-17 | | Profiles/capabilities | `Performance`, `Balanced`, `High`, `Custom`; Blizzlike fidelity incomplete | ## Purpose @@ -137,6 +137,7 @@ from externally reading/writing loader-private queue, task, cache and tile-state | `M2MeshResourceCacheState` | Internal M2 Resource cache | Owns prepared static Mesh references by normalized path | Renderer main thread; final shutdown | Empty path/null Mesh rejected; same path replaces | | `M2MeshResourceExtractor` | Internal M2 scene/resource service | Selects first Mesh from direct Resource, PackedScene or Node subtree | Renderer main thread; stateless except temporary instance | Invalid/no Mesh returns null; temporary PackedScene root freed | | `M2RuntimeMeshFinalizer` | Internal M2 preparation service | Owns refresh version, rebuild classification, M2Builder rebuild and fallback | Renderer main thread; decisions cached until reset | Null returns null; missing/failed rebuild marks and reuses original Mesh | +| `M2RawModelRepository` | Internal M2 native repository | Reads static/animated raw Dictionaries through exact M2Loader methods | Synchronous; stateless | Invalid/unavailable/non-Dictionary result returns empty Dictionary | | `WmoPlacementResolver.normalize_relative_path/resolve_unique_key/resolve_world_transform` | Internal pure WMO service | Resolves cache key, registry identity and world transform | Main/any thread; stateless | Missing UID uses tile/index fallback; transform fields use historical defaults | | `WmoPlacementRegistry.add_reference/release_reference/contains/active_count/diagnostic_snapshot/clear` | Internal WMO service | Owns placement-key to tile/global reference sets | Renderer main thread; map session | Empty/unknown/non-owner input is rejected without mutation | | `WmoRenderBuildStepPlanner.plan_step` | Internal pure WMO service | Selects one mesh-first lightweight render-group operation and next cursors | Main/any thread; stateless | Raw integer comparisons are preserved without clamping | @@ -180,6 +181,7 @@ loader configuration remains transitional composition data, not a caller API. | Internal WMO render cache | Normalized path, cache path and validated Resource | Loader / `WmoRenderResourceCacheState` | Loader lookup, ResourceLoader poll and build queue | State-owned Resource/path references; detached request snapshots | Until transient/full clear | | Internal WMO scene cache | Normalized path, `.tscn` path and validated PackedScene | Loader / `WmoSceneResourceCacheState` | Loader lookup, request poll and scene instantiation | State-owned PackedScene/path references; detached request snapshots | Until transient/full clear | | Internal ADT water load | Tile key, ADT path, task ID and parsed Dictionary | Loader/worker / `AdtWaterLoadPipelineState` | Loader task start, budgeted drain and finalization | State-owned records; mutex result mailbox | Request through result completion/reset | +| Internal raw M2 read | Extracted directory and normalized relative path | Loader / `M2RawModelRepository` | Finalizer, static or animated builder adapters | Fresh Dictionary; repository retains nothing | One synchronous native call | | Output | Desired tile/detail operations | Streamer plan application | Finalize queues | Loader-owned | Cross-frame | | Output | Terrain/M2/WMO/liquid instances | Loader/builders | Godot world/renderer | Loader/world owner | Main-thread attach | | Output | Detached renderer metrics | `WorldRenderFacade` | Capture/baseline tools | Caller-owned deep copy | On demand | @@ -372,7 +374,7 @@ sequenceDiagram Editor ownership on the main thread without retaining the generated subtree. - `M2RuntimeMeshRebuildClassifier` owns only normalized-path boolean decisions for billboard/UV-rotation material refresh and is composed by the runtime Mesh - finalizer. The loader retains raw M2 loading and returned-Mesh adoption. + finalizer. The raw repository loads value data; loader retains Mesh adoption. - `M2MeshLoadPipelineState` owns static M2 pending Resource paths, opaque terminal statuses and completion-order finalize FIFO. The loader retains cache path selection, ResourceLoader calls, permits, shared missing state and adoption. @@ -385,6 +387,9 @@ sequenceDiagram - `M2RuntimeMeshFinalizer` owns refresh version `2`, classifier lifetime, M2Builder rebuild and original-Mesh fallback. The loader loads raw data only after the finalizer reports that a cached Mesh is stale. +- `M2RawModelRepository` owns FileAccess/ClassDB availability and the exact + `load_m2`/`load_m2_animated` calls. The loader retains normalization, + fallback order, missing/prototype state and every result consumer. - Rendered-ground query results and diagnostic snapshots are caller-owned values; the facade never returns Mesh, Node, tile-state or queue references. - Loaded-mesh ground sampling is renderer diagnostics, not authoritative gameplay @@ -590,6 +595,7 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m | `src/render/m2/m2_mesh_resource_cache_state.gd` | Prepared static M2 Mesh references and final-shutdown release | | `src/render/m2/m2_mesh_resource_extractor.gd` | First-Mesh selection and temporary PackedScene instance lifetime | | `src/render/m2/m2_runtime_mesh_finalizer.gd` | Runtime refresh version, rebuild classification/build and fallback | +| `src/render/m2/m2_raw_model_repository.gd` | Stateless static/animated native M2 file boundary | | `src/render/streaming/streaming_target_planner.gd` | Scene-free wanted/retained ADT target calculation | | `src/render/streaming/streaming_target_policy.gd` | Immutable renderer radius/prefetch policy | | `src/render/streaming/streaming_target_plan.gd` | Immutable planner result with read-only tile-key sets | @@ -609,6 +615,7 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m | `src/tools/verify_m2_mesh_resource_cache_state.gd` | Static M2 Mesh cache ownership/lifetime/boundary/timing regression | | `src/tools/verify_m2_mesh_resource_extractor.gd` | M2 direct/PackedScene/subtree order/lifetime/boundary/timing regression | | `src/tools/verify_m2_runtime_mesh_finalizer.gd` | M2 current/stale/rebuild/fallback/boundary/timing regression | +| `src/tools/verify_m2_raw_model_repository.gd` | M2 invalid/missing/native-boundary/dependency/timing regression | | `src/tools/verify_streaming_target_planner.gd` | Planner behavior, dependency and bounded timing regression | | `src/tools/verify_render_budget_scheduler.gd` | Scheduler bounds, shared-lane priority, cancellation and timing regression | | `src/tools/verify_renderer_internal_access.gd` | Gameplay/EditorPlugin/registered renderer-tool boundary gate derived from private streamer fields | diff --git a/src/render/m2/m2_raw_model_repository.gd b/src/render/m2/m2_raw_model_repository.gd new file mode 100644 index 0000000..711064f --- /dev/null +++ b/src/render/m2/m2_raw_model_repository.gd @@ -0,0 +1,51 @@ +class_name M2RawModelRepository +extends RefCounted + +## Reads raw static and animated M2 Dictionaries through the optional native +## M2Loader boundary. The repository is stateless and does not cache failures. + +const NATIVE_LOADER_CLASS_NAME := "M2Loader" + + +## Loads one normalized static M2 path with the native `load_m2` method. +## Invalid input, unavailable native support and parse failures return `{}`. +func load_static_model_data( + extracted_directory: String, + normalized_relative_path: String) -> Dictionary: + return _load_model_data( + extracted_directory, + normalized_relative_path, + "load_m2" + ) + + +## Loads one normalized animated M2 path with the native `load_m2_animated` +## method. Invalid input, unavailable support and parse failures return `{}`. +func load_animated_model_data( + extracted_directory: String, + normalized_relative_path: String) -> Dictionary: + return _load_model_data( + extracted_directory, + normalized_relative_path, + "load_m2_animated" + ) + + +func _load_model_data( + extracted_directory: String, + normalized_relative_path: String, + native_method_name: StringName) -> Dictionary: + if normalized_relative_path.is_empty(): + return {} + if not ClassDB.class_exists(NATIVE_LOADER_CLASS_NAME): + return {} + var absolute_path := ProjectSettings.globalize_path( + extracted_directory.path_join(normalized_relative_path) + ) + if not FileAccess.file_exists(absolute_path): + return {} + var native_loader: Object = ClassDB.instantiate(NATIVE_LOADER_CLASS_NAME) + if native_loader == null or not native_loader.has_method(native_method_name): + return {} + var raw_model_data: Variant = native_loader.call(native_method_name, absolute_path) + return raw_model_data if raw_model_data is Dictionary else {} diff --git a/src/render/m2/m2_raw_model_repository.gd.uid b/src/render/m2/m2_raw_model_repository.gd.uid new file mode 100644 index 0000000..f3180db --- /dev/null +++ b/src/render/m2/m2_raw_model_repository.gd.uid @@ -0,0 +1 @@ +uid://bwqtuln7vpcj4 diff --git a/src/scenes/streaming/streaming_world_loader.gd b/src/scenes/streaming/streaming_world_loader.gd index 64a1340..5796c9d 100644 --- a/src/scenes/streaming/streaming_world_loader.gd +++ b/src/scenes/streaming/streaming_world_loader.gd @@ -63,6 +63,9 @@ const M2_BUILD_BATCH_PLANNER_SCRIPT := preload( const M2_RUNTIME_MESH_FINALIZER_SCRIPT := preload( "res://src/render/m2/m2_runtime_mesh_finalizer.gd" ) +const M2_RAW_MODEL_REPOSITORY_SCRIPT := preload( + "res://src/render/m2/m2_raw_model_repository.gd" +) const M2_MESH_LOAD_PIPELINE_STATE_SCRIPT := preload( "res://src/render/m2/m2_mesh_load_pipeline_state.gd" ) @@ -259,6 +262,7 @@ var _m2_placement_transform_resolver := M2_PLACEMENT_TRANSFORM_RESOLVER_SCRIPT.n var _m2_placement_grouper := M2_PLACEMENT_GROUPER_SCRIPT.new() var _m2_build_batch_planner := M2_BUILD_BATCH_PLANNER_SCRIPT.new() var _m2_runtime_mesh_finalizer := M2_RUNTIME_MESH_FINALIZER_SCRIPT.new() +var _m2_raw_model_repository := M2_RAW_MODEL_REPOSITORY_SCRIPT.new() var _m2_mesh_resource_cache_state := M2_MESH_RESOURCE_CACHE_STATE_SCRIPT.new() var _m2_mesh_resource_extractor := M2_MESH_RESOURCE_EXTRACTOR_SCRIPT.new() var _m2_mesh_load_pipeline_state := M2_MESH_LOAD_PIPELINE_STATE_SCRIPT.new() @@ -4653,7 +4657,10 @@ func _prepare_m2_mesh_for_runtime(normalized_rel: String, mesh: Mesh) -> Mesh: return null if not _m2_runtime_mesh_finalizer.requires_raw_data_for_refresh(mesh): return mesh - var data := _load_m2_raw_data_for_refresh(normalized_rel) + var data := _m2_raw_model_repository.load_static_model_data( + extracted_dir, + normalized_rel + ) return _m2_runtime_mesh_finalizer.finalize_mesh( normalized_rel, mesh, @@ -4673,19 +4680,6 @@ func _prepare_m2_node_for_runtime(normalized_rel: String, root: Node3D) -> void: mesh_instance.mesh = prepared -func _load_m2_raw_data_for_refresh(normalized_rel: String) -> Dictionary: - if normalized_rel.is_empty() or not ClassDB.class_exists("M2Loader"): - return {} - var abs_path := ProjectSettings.globalize_path(extracted_dir.path_join(normalized_rel)) - if not FileAccess.file_exists(abs_path): - return {} - var loader = ClassDB.instantiate("M2Loader") - if loader == null: - return {} - var data: Variant = loader.call("load_m2", abs_path) - return data if data is Dictionary else {} - - func _get_or_load_m2_mesh(rel_path: String) -> Mesh: var normalized_rel := _normalize_m2_rel_path(rel_path) if _m2_mesh_resource_cache_state.has_mesh(normalized_rel): @@ -4868,18 +4862,11 @@ func _get_or_load_m2_prototype(rel_path: String) -> Node3D: return node as Node3D break - # Fall back to raw M2Loader - if not ClassDB.class_exists("M2Loader"): - _m2_missing_cache[normalized_rel] = true - return null - - var abs_path := ProjectSettings.globalize_path(extracted_dir.path_join(normalized_rel)) - if not FileAccess.file_exists(abs_path): - _m2_missing_cache[normalized_rel] = true - return null - - var loader = ClassDB.instantiate("M2Loader") - var data: Dictionary = loader.call("load_m2", abs_path) + # Fall back to the raw native M2 repository. + var data := _m2_raw_model_repository.load_static_model_data( + extracted_dir, + normalized_rel + ) if data.is_empty(): _m2_missing_cache[normalized_rel] = true return null @@ -4901,24 +4888,10 @@ func _get_or_load_m2_native_animated_prototype(rel_path: String) -> Node3D: return _m2_animated_scene_cache[normalized_rel] if _m2_static_animation_cache.has(normalized_rel): return null - if not ClassDB.class_exists("M2Loader"): - _m2_static_animation_cache[normalized_rel] = true - return null - - var abs_path := ProjectSettings.globalize_path(extracted_dir.path_join(normalized_rel)) - if not FileAccess.file_exists(abs_path): - _m2_static_animation_cache[normalized_rel] = true - return null - - var loader = ClassDB.instantiate("M2Loader") - if loader == null: - _m2_static_animation_cache[normalized_rel] = true - return null - if not loader.has_method("load_m2_animated"): - _m2_static_animation_cache[normalized_rel] = true - return null - - var data: Dictionary = loader.call("load_m2_animated", abs_path) + var data := _m2_raw_model_repository.load_animated_model_data( + extracted_dir, + normalized_rel + ) var animated_surfaces: Array = data.get("animated_surfaces", []) if data.is_empty() or animated_surfaces.is_empty(): _m2_static_animation_cache[normalized_rel] = true diff --git a/src/tools/verify_m2_raw_model_repository.gd b/src/tools/verify_m2_raw_model_repository.gd new file mode 100644 index 0000000..ae0c7d4 --- /dev/null +++ b/src/tools/verify_m2_raw_model_repository.gd @@ -0,0 +1,133 @@ +extends SceneTree + +## Invalid/missing/native-boundary/source/performance contract for raw M2 I/O. + +const REPOSITORY_SCRIPT := preload("res://src/render/m2/m2_raw_model_repository.gd") +const REPOSITORY_PATH := "res://src/render/m2/m2_raw_model_repository.gd" +const LOADER_PATH := "res://src/scenes/streaming/streaming_world_loader.gd" + + +func _initialize() -> void: + var failures: Array[String] = [] + _verify_rejected_inputs(failures) + _verify_ownership_boundaries(failures) + var elapsed_milliseconds := _verify_bounded_timing(failures) + if not failures.is_empty(): + for failure in failures: + push_error("M2_RAW_MODEL_REPOSITORY: %s" % failure) + quit(1) + return + print( + "M2_RAW_MODEL_REPOSITORY PASS cases=10 iterations=10000 elapsed_ms=%.3f" + % elapsed_milliseconds + ) + quit(0) + + +func _verify_rejected_inputs(failures: Array[String]) -> void: + var repository: RefCounted = REPOSITORY_SCRIPT.new() + _expect_empty( + repository.call("load_static_model_data", "res://data/extracted", ""), + "empty static path", + failures + ) + _expect_empty( + repository.call("load_animated_model_data", "res://data/extracted", ""), + "empty animated path", + failures + ) + var missing_path := "world/__openwc_missing__/repository_fixture.m2" + _expect_empty( + repository.call("load_static_model_data", "res://data/extracted", missing_path), + "missing static file", + failures + ) + _expect_empty( + repository.call("load_animated_model_data", "res://data/extracted", missing_path), + "missing animated file", + failures + ) + _expect_empty( + repository.call("load_static_model_data", "res://data/extracted", missing_path), + "missing failure is not retained", + failures + ) + + +func _verify_ownership_boundaries(failures: Array[String]) -> void: + var repository_source := FileAccess.get_file_as_string(REPOSITORY_PATH) + var loader_source := FileAccess.get_file_as_string(LOADER_PATH) + _expect_true( + loader_source.contains("M2_RAW_MODEL_REPOSITORY_SCRIPT.new()"), + "loader composes raw model repository", + failures + ) + _expect_equal( + loader_source.count("_m2_raw_model_repository.load_static_model_data("), + 2, + "two static callers delegate", + failures + ) + _expect_equal( + loader_source.count("_m2_raw_model_repository.load_animated_model_data("), + 1, + "animated caller delegates", + failures + ) + for removed_loader_rule in [ + "func _load_m2_raw_data_for_refresh(", + "ClassDB.class_exists(\"M2Loader\")", + "ClassDB.instantiate(\"M2Loader\")", + "loader.call(\"load_m2\"", + "loader.call(\"load_m2_animated\"", + ]: + _expect_false(loader_source.contains(removed_loader_rule), "loader omits %s" % removed_loader_rule, failures) + for required_repository_rule in [ + "ClassDB.class_exists(NATIVE_LOADER_CLASS_NAME)", + "ProjectSettings.globalize_path(", + "extracted_directory.path_join(normalized_relative_path)", + "FileAccess.file_exists(absolute_path)", + "ClassDB.instantiate(NATIVE_LOADER_CLASS_NAME)", + "native_loader.has_method(native_method_name)", + "\"load_m2\"", + "\"load_m2_animated\"", + ]: + _expect_true(repository_source.contains(required_repository_rule), "repository owns %s" % required_repository_rule, failures) + for forbidden_dependency in [ + "ResourceLoader.", + "M2_BUILDER", + "_m2_missing_cache", + "Mesh", + "Node", + "WorkerThreadPool", + ]: + _expect_false(repository_source.contains(forbidden_dependency), "repository omits %s" % forbidden_dependency, failures) + + +func _verify_bounded_timing(failures: Array[String]) -> float: + var repository: RefCounted = REPOSITORY_SCRIPT.new() + var started_microseconds := Time.get_ticks_usec() + for _iteration in range(10000): + repository.call("load_static_model_data", "res://data/extracted", "") + var elapsed_milliseconds := float(Time.get_ticks_usec() - started_microseconds) / 1000.0 + _expect_true(elapsed_milliseconds < 1000.0, "10000 empty-path rejects under 1 second", failures) + return elapsed_milliseconds + + +func _expect_empty(actual: Variant, label: String, failures: Array[String]) -> void: + if not (actual is Dictionary) or not (actual as Dictionary).is_empty(): + failures.append(label) + + +func _expect_true(condition: bool, label: String, failures: Array[String]) -> void: + if not condition: + failures.append(label) + + +func _expect_false(condition: bool, label: String, failures: Array[String]) -> void: + _expect_true(not condition, label, failures) + + +func _expect_equal(actual: int, expected: int, label: String, failures: Array[String]) -> void: + if actual != expected: + failures.append("%s expected=%d actual=%d" % [label, expected, actual]) diff --git a/src/tools/verify_m2_raw_model_repository.gd.uid b/src/tools/verify_m2_raw_model_repository.gd.uid new file mode 100644 index 0000000..8bb5a40 --- /dev/null +++ b/src/tools/verify_m2_raw_model_repository.gd.uid @@ -0,0 +1 @@ +uid://byteanym3al5l diff --git a/src/tools/verify_m2_runtime_mesh_finalizer.gd b/src/tools/verify_m2_runtime_mesh_finalizer.gd index 7ec18bf..a23cd2d 100644 --- a/src/tools/verify_m2_runtime_mesh_finalizer.gd +++ b/src/tools/verify_m2_runtime_mesh_finalizer.gd @@ -5,6 +5,7 @@ extends SceneTree const FINALIZER_SCRIPT := preload("res://src/render/m2/m2_runtime_mesh_finalizer.gd") const FINALIZER_PATH := "res://src/render/m2/m2_runtime_mesh_finalizer.gd" +const RAW_REPOSITORY_PATH := "res://src/render/m2/m2_raw_model_repository.gd" const LOADER_PATH := "res://src/scenes/streaming/streaming_world_loader.gd" const REFRESH_META := "wow_m2_material_refresh_version" @@ -109,6 +110,7 @@ func _verify_rebuild_failure_fallback_and_clear(failures: Array[String]) -> void func _verify_ownership_boundaries(failures: Array[String]) -> void: var finalizer_source := FileAccess.get_file_as_string(FINALIZER_PATH) + var raw_repository_source := FileAccess.get_file_as_string(RAW_REPOSITORY_PATH) var loader_source := FileAccess.get_file_as_string(LOADER_PATH) _expect_true( loader_source.contains("M2_RUNTIME_MESH_FINALIZER_SCRIPT.new()"), @@ -124,14 +126,18 @@ func _verify_ownership_boundaries(failures: Array[String]) -> void: _expect_false(loader_source.contains("const M2_MATERIAL_REFRESH_VERSION"), "refresh version leaves loader", failures) _expect_false(loader_source.contains("func _rebuild_m2_mesh_from_data("), "legacy rebuild adapter removed", failures) for retained_loader_rule in [ - "func _load_m2_raw_data_for_refresh(", - "FileAccess.file_exists(abs_path)", - "ClassDB.instantiate(\"M2Loader\")", - "loader.call(\"load_m2\", abs_path)", + "M2_RAW_MODEL_REPOSITORY_SCRIPT.new()", + "_m2_raw_model_repository.load_static_model_data(", "_m2_mesh_resource_cache_state.store_mesh(", "_m2_missing_cache[normalized_rel]", ]: _expect_true(loader_source.contains(retained_loader_rule), "loader retains %s" % retained_loader_rule, failures) + for raw_repository_rule in [ + "FileAccess.file_exists(absolute_path)", + "ClassDB.instantiate(NATIVE_LOADER_CLASS_NAME)", + "\"load_m2\"", + ]: + _expect_true(raw_repository_source.contains(raw_repository_rule), "raw repository owns %s" % raw_repository_rule, failures) for required_finalizer_rule in [ "const MATERIAL_REFRESH_VERSION := 2", "M2_RUNTIME_MESH_REBUILD_CLASSIFIER_SCRIPT.new()",