From 7cb3e3412f9f47c3e84eba5f2f5b583e7437a390 Mon Sep 17 00:00:00 2001 From: sindoring Date: Sat, 18 Jul 2026 03:01:43 +0400 Subject: [PATCH] render: extract static M2 build resource observer --- RENDER.md | 10 + docs/modules/README.md | 1 + docs/modules/m2-build-resource-snapshot.md | 12 +- docs/modules/m2-mesh-load-pipeline-state.md | 3 +- docs/modules/m2-mesh-resource-cache-state.md | 2 +- .../m2-static-build-resource-observer.md | 209 ++++++++++++++++++ docs/modules/world-renderer.md | 23 +- .../m2/m2_static_build_resource_observer.gd | 126 +++++++++++ .../m2_static_build_resource_observer.gd.uid | 1 + .../streaming/streaming_world_loader.gd | 84 ++----- src/tools/verify_m2_build_dispatch_planner.gd | 2 +- .../verify_m2_build_resource_snapshot.gd | 8 +- .../verify_m2_mesh_resource_cache_state.gd | 5 +- ...erify_m2_static_build_resource_observer.gd | 127 +++++++++++ ...y_m2_static_build_resource_observer.gd.uid | 1 + targets/03-renderer-facade.md | 10 +- 16 files changed, 540 insertions(+), 84 deletions(-) create mode 100644 docs/modules/m2-static-build-resource-observer.md create mode 100644 src/render/m2/m2_static_build_resource_observer.gd create mode 100644 src/render/m2/m2_static_build_resource_observer.gd.uid create mode 100644 src/tools/verify_m2_static_build_resource_observer.gd create mode 100644 src/tools/verify_m2_static_build_resource_observer.gd.uid diff --git a/RENDER.md b/RENDER.md index 6e3c752..174ec7e 100644 --- a/RENDER.md +++ b/RENDER.md @@ -43,6 +43,7 @@ Paired run 2026-07-11 подтвердил крупный coordinate/placement g - `src/render/m2/m2_build_queue.gd` / `m2_build_job.gd` - typed pending M2 jobs, FIFO/stale tile keys, grouped-transform references and progress cursors without engine destruction. - `src/render/m2/m2_build_dispatch_planner.gd` - pure animation/static wait, materialization and missing-model advance decision. - `src/render/m2/m2_build_resource_snapshot.gd` - typed per-step animated/static resource observations without engine destruction. +- `src/render/m2/m2_static_build_resource_observer.gd` - static Mesh cache lookup, threaded request selection and missing transition. - `src/render/m2/m2_animation_load_pipeline_state.gd` - animated M2 threaded-load request records and completion-order finalize FIFO without I/O or Node ownership. - `src/render/m2/m2_mesh_load_pipeline_state.gd` - static M2 threaded-load request records, terminal statuses and completion-order finalize FIFO without I/O or Mesh ownership. - `src/render/m2/m2_mesh_resource_cache_state.gd` - normalized-path prepared static M2 Mesh references with final-shutdown lifetime. @@ -1084,6 +1085,15 @@ $exe = Join-Path $env:TEMP 'godot-4.6.1-openwc\Godot_v4.6.1-stable_win64.exe' - ResourceLoader/cache requests, permit/cursor transitions, engine lifetime, cache formats, profiles and visible rules remain unchanged. +## 2026-07-18 M2 Static Build Resource Observer Extraction + +- `M2StaticBuildResourceObserver` now owns prepared-Mesh lookup, existing-request + detection, `.tscn`-before-`.glb` candidate selection, pivot-prefix GLB rejection, + threaded request admission and terminal missing transition for build jobs. +- The observer fills `M2BuildResourceSnapshot`; the loader retains animation + observation, finalize drains, materialization, permits and engine lifetime. +- Candidate order, request errors, cache formats, profiles and visuals are unchanged. + ## 2026-07-17 M2 Runtime Mesh Rebuild Classifier Extraction - `M2RuntimeMeshRebuildClassifier` now owns the memoized decision used when a diff --git a/docs/modules/README.md b/docs/modules/README.md index 375f0a7..d96ab56 100644 --- a/docs/modules/README.md +++ b/docs/modules/README.md @@ -23,6 +23,7 @@ | M2 build batch planner | Implemented extraction | [`m2-build-batch-planner.md`](m2-build-batch-planner.md) | | M2 build dispatch planner | Implemented extraction | [`m2-build-dispatch-planner.md`](m2-build-dispatch-planner.md) | | M2 build resource snapshot | Implemented extraction | [`m2-build-resource-snapshot.md`](m2-build-resource-snapshot.md) | +| M2 static build resource observer | Implemented extraction | [`m2-static-build-resource-observer.md`](m2-static-build-resource-observer.md) | | M2 build queue | Implemented extraction | [`m2-build-queue.md`](m2-build-queue.md) | | M2 static batch materializer | Implemented extraction | [`m2-static-batch-materializer.md`](m2-static-batch-materializer.md) | | M2 runtime mesh rebuild classifier | Implemented extraction | [`m2-runtime-mesh-rebuild-classifier.md`](m2-runtime-mesh-rebuild-classifier.md) | diff --git a/docs/modules/m2-build-resource-snapshot.md b/docs/modules/m2-build-resource-snapshot.md index e27bd1d..41f0652 100644 --- a/docs/modules/m2-build-resource-snapshot.md +++ b/docs/modules/m2-build-resource-snapshot.md @@ -187,9 +187,9 @@ queue/hitch metrics remain unchanged. ## Extension points -A later resource-observation service may produce this snapshot while moving -ResourceLoader/cache request ownership out of the loader. The value contract -must remain independent of that service and of generic callback frameworks. +`M2StaticBuildResourceObserver` now produces the static phase. A later animated +observer may produce the first phase. The value contract remains independent of +both producers and of generic callback frameworks. ## Capability status @@ -197,7 +197,8 @@ must remain independent of that service and of generic callback frameworks. |---|---|---|---| | Typed per-step M2 resource observation | Implemented extraction | Identity/adoption/lifetime/source/timing verifier | Asset-backed traversal pending | | Dispatch decision | Implemented separately | Dispatch planner verifier | Asset-backed traversal pending | -| Resource lookup/request execution | Loader-owned | Cache/pipeline regressions | Service extraction pending | +| Static lookup/request execution | Implemented separately | Static observer verifier | Asset-backed traversal pending | +| Animated lookup/request execution | Loader-owned | Animation regressions | Service extraction pending | | Engine lifetime/materialization | Loader/materializer-owned | Lifetime/materializer regressions | GPU/p95/p99 evidence pending | ## Known gaps and risks @@ -213,7 +214,8 @@ must remain independent of that service and of generic callback frameworks. |---|---| | `src/render/m2/m2_build_resource_snapshot.gd` | Typed per-step resource observations and diagnostics | | `src/render/m2/m2_build_dispatch_planner.gd` | Snapshot-to-action planning | -| `src/scenes/streaming/streaming_world_loader.gd` | Resource observation/request execution and materializer borrowing | +| `src/render/m2/m2_static_build_resource_observer.gd` | Static snapshot production and requests | +| `src/scenes/streaming/streaming_world_loader.gd` | Animated observation/request execution and materializer borrowing | | `src/tools/verify_m2_build_resource_snapshot.gd` | Identity/adoption/lifetime/boundary/timing regression | ## Related decisions and references diff --git a/docs/modules/m2-mesh-load-pipeline-state.md b/docs/modules/m2-mesh-load-pipeline-state.md index 956b56d..acd6ea7 100644 --- a/docs/modules/m2-mesh-load-pipeline-state.md +++ b/docs/modules/m2-mesh-load-pipeline-state.md @@ -206,7 +206,8 @@ rebuild policy are unchanged; no migration or rebake is required. | Capability | Status | Evidence | Gap/next step | |---|---|---|---| | Static M2 request/finalize state | Implemented extraction | Contract/source/timing verifier | Asset-backed long traversal pending | -| ResourceLoader I/O | Existing loader-owned | Shutdown/material regressions | I/O adapter extraction optional | +| Static request admission I/O | Implemented in observer | Observer/shutdown regressions | Asset-backed traversal pending | +| Static request polling/finalize I/O | Existing loader-owned | Shutdown/material regressions | Finalize extraction optional | | Mesh cache | Implemented extraction | Mesh resource cache state verifier | Asset-backed memory/leak run pending | | First-Mesh extraction | Implemented extraction | Resource/order/lifetime verifier | Asset-backed corrupt-scene fixture pending | | Mesh preparation | Implemented extraction | Runtime finalizer transition/rebuild verifier | Asset-backed material comparison pending | diff --git a/docs/modules/m2-mesh-resource-cache-state.md b/docs/modules/m2-mesh-resource-cache-state.md index c89b659..660163a 100644 --- a/docs/modules/m2-mesh-resource-cache-state.md +++ b/docs/modules/m2-mesh-resource-cache-state.md @@ -186,7 +186,7 @@ the historical Mesh cache had no queue contribution or log site. | Capability | Status | Evidence | Gap/next step | |---|---|---|---| | Prepared static M2 Mesh cache | Implemented extraction | Contract/source/timing verifier | Asset-backed memory/leak run pending | -| M2 Mesh request lifecycle | Implemented extraction | Pipeline state verifier | ResourceLoader I/O remains loader-owned | +| M2 Mesh request lifecycle | Implemented extraction | Pipeline state verifier | Finalize polling remains loader-owned | | M2 Mesh extraction | Implemented extraction | Resource/order/lifetime verifier | Asset-backed corrupt-scene fixture pending | | M2 Mesh preparation | Implemented extraction | Runtime finalizer transition/rebuild verifier | Asset-backed material comparison pending | diff --git a/docs/modules/m2-static-build-resource-observer.md b/docs/modules/m2-static-build-resource-observer.md new file mode 100644 index 0000000..5df0d0d --- /dev/null +++ b/docs/modules/m2-static-build-resource-observer.md @@ -0,0 +1,209 @@ +# M2 Static Build Resource Observer + +## Metadata + +| Field | Value | +|---|---| +| Status | Implemented extraction | +| Target/work package | M03 / `M03-RND-M2-STATIC-BUILD-RESOURCE-OBSERVER-001` | +| Owners | Static build Mesh lookup, request selection and missing transition | +| Last verified | Worktree `work/sindo-main-codex/m03-m2-static-build-resource-observer`, 2026-07-18 | +| Profiles/capabilities | Existing static MultiMesh M2 build path | + +## Purpose + +Produce the static phase of `M2BuildResourceSnapshot`: reuse a prepared Mesh, +wait for an existing request, start the first eligible cache request, or record +a terminal missing model while preserving historical path and GLB rules. + +## Non-goals + +- Observe animated/native prototypes or finalize threaded loads. +- Own/free Meshes, Nodes, cache entries or snapshot lifetime. +- Materialize instances, plan batches, rotate queues or consume permits. +- Change cache format, candidate order, profiles or visible output. + +## Context and boundaries + +```mermaid +flowchart LR + Loader[StreamingWorldLoader] --> Observer[M2StaticBuildResourceObserver] + MeshCache[M2MeshResourceCacheState] --> Observer + Missing[M2PrototypeCacheState] --> Observer + Pipeline[M2MeshLoadPipelineState] --> Observer + Observer --> Snapshot[M2BuildResourceSnapshot] + Snapshot --> Dispatch[M2BuildDispatchPlanner] +``` + +## Public API + +| Symbol | Kind | Purpose | Errors | +|---|---|---|---| +| `observe(snapshot, path, cache_dir, mesh_cache, prototype_cache, pipeline)` | Command/query | Adopt cached/pending/requested/missing static result | Invalid inputs return `rejected` | +| `cache_resource_paths(cache_dir, path, extensions)` | Pure query | Return unique historical nested/lowercase/basename candidates | Empty values yield fewer/empty candidates | +| `glb_animation_schema(path)` | Read query | Read OpenWC schema marker from GLB JSON | Invalid/missing GLB returns empty String | +| `OUTCOME_*` | Constants | Stable diagnostic transition identifiers | None | + +## Inputs and outputs + +| Direction | Data | Producer | Consumer | Ownership/lifetime | +|---|---|---|---|---| +| Input | Normalized M2 path/cache directory | Loader | Observer | Copied Strings; one observation | +| Input | Mesh/missing/request states | M2 state services | Observer | Borrowed services; map/session | +| Input/output | Resource snapshot | Loader | Observer/dispatch/materializer | Borrowed; one build operation | +| Output | Cached Mesh or missing/pending values | Observer | Snapshot | Exact borrowed Mesh/scalars | +| Output | Threaded request record | Observer | Mesh pipeline | Pipeline-owned path record | +| Output | Outcome StringName | Observer | Tests/future diagnostics | Immutable scalar | + +## Data flow + +```mermaid +flowchart TD + Start[Observe static path] --> Cached{Mesh cached?} + Cached -->|yes| AdoptMesh[Adopt exact Mesh; cached] + Cached -->|no| Missing{Already missing?} + Missing -->|yes| AdoptMissing[Adopt missing] + Missing -->|no| Pending{Request pending?} + Pending -->|yes| AdoptPending[Adopt unresolved; pending] + Pending -->|no| Candidates[Generate tscn then glb candidates] + Candidates --> Eligible{Exists and not pivot-prefix GLB?} + Eligible -->|yes| Request[Threaded request] + Request -->|OK or busy| Remember[Remember request; requested] + Eligible -->|none/error| MarkMissing[Mark/adopt missing] +``` + +## Lifecycle/state + +```mermaid +stateDiagram-v2 + [*] --> Observing + Observing --> Cached + Observing --> Pending + Observing --> Requested + Observing --> Missing + Observing --> Rejected + Cached --> [*] + Pending --> [*] + Requested --> [*] + Missing --> [*] + Rejected --> [*] +``` + +## Main sequence + +```mermaid +sequenceDiagram + participant L as StreamingWorldLoader + participant O as StaticBuildResourceObserver + participant C as Mesh/prototype cache state + participant P as MeshLoadPipelineState + participant R as ResourceLoader + participant S as M2BuildResourceSnapshot + L->>O: observe(snapshot, path, cache_dir, states) + O->>C: cached/missing lookup + alt cache hit or missing + O->>S: adopt static observation + else pending + O->>P: has_request + O->>S: adopt unresolved observation + else request candidate + O->>R: exists + load_threaded_request + O->>P: remember_request + O->>S: adopt unresolved observation + end +``` + +## Dependency diagram + +```mermaid +flowchart TB + Observer[M2StaticBuildResourceObserver] --> ResourceLoader + Observer --> FileAccess[GLB header/JSON read] + Observer --> MeshCache[M2MeshResourceCacheState] + Observer --> PrototypeCache[M2PrototypeCacheState] + Observer --> Pipeline[M2MeshLoadPipelineState] + Observer --> Snapshot[M2BuildResourceSnapshot] + Observer -. no dependency .-> SceneTree + Observer -. no dependency .-> Materializers + Observer -. no dependency .-> Scheduler +``` + +## Ownership, threading and resources + +- Observer retains no request, cache, snapshot or engine reference. +- Cache/pipeline services retain their existing state and resource ownership. +- Snapshot borrows the exact cached Mesh; observer never frees it. +- Resource existence/request and GLB inspection run on renderer main thread. +- Finalization and Mesh preparation remain in the loader drain path. + +## Errors, cancellation and recovery + +| State | Behavior | Recovery | +|---|---|---| +| Invalid dependency/path | Return `rejected`; no mutation | Correct composition | +| Existing request | Return `pending`; no duplicate request | Retry when queue rotates | +| Pivot-prefix GLB | Skip as static candidate | Continue next candidate | +| Request error | Stop historical search and mark missing | Later reset/new session | +| No candidate | Mark/adopt terminal missing | Cache content becomes available in new session | +| Tile cancellation | Observer retains nothing | Loader/queue cancellation remains authoritative | + +## Configuration and capabilities + +The observer consumes existing `m2_cache_dir`. It adds no profile, cache version, +batch, visibility, shadow or scheduler setting. + +## Persistence, cache and migration + +No new persistence is introduced. Existing `.tscn/.glb` layout, pipeline records +and missing-cache lifetime are unchanged; no rebake is required. + +## Diagnostics and observability + +Returned outcomes distinguish rejected/cached/pending/requested/missing without +logging. Existing queue and hitch metrics remain unchanged. + +## Verification + +- Dedicated verifier covers invalid, cached identity, missing, pending, path + order/deduplication, absent GLB schema, source ownership and bounded timing. +- Snapshot/dispatch/pipeline/cache/shutdown/facade/internal-access/checkpoint + regressions protect adjacent behavior. +- No original-client visual claim is made; this is bookkeeping/I/O extraction. + +## Extension points + +Animated/native observation may become a separate producer of the animation +phase. Shared cache candidate/GLB inspection can later be reused without adding +a generic resource framework. + +## Capability status + +| Capability | Status | Evidence | Gap | +|---|---|---|---| +| Static build lookup/request production | Implemented extraction | Lifecycle/path/source/timing verifier | Asset-backed traversal pending | +| Static finalize/preparation | Existing loader-owned | Pipeline/finalizer regressions | Later extraction | +| Animated observation | Existing loader-owned | Animation regressions | Separate producer pending | + +## Known gaps and risks + +- Successful request behavior is protected by source plus existing pipeline + tests; starting an undrained asynchronous request in the unit fixture would leak. +- GLB schema parser reads synchronously, matching the previous loader behavior. +- Private traversal, p95/p99, leak and visual evidence remain unavailable. + +## Source map + +| Path | Responsibility | +|---|---| +| `src/render/m2/m2_static_build_resource_observer.gd` | Static cache/request/missing observation | +| `src/render/m2/m2_build_resource_snapshot.gd` | Typed observation result | +| `src/scenes/streaming/streaming_world_loader.gd` | Composition, finalize drain and action execution | +| `src/tools/verify_m2_static_build_resource_observer.gd` | Lifecycle/path/boundary/timing regression | + +## Related decisions and references + +- [`m2-build-resource-snapshot.md`](m2-build-resource-snapshot.md) +- [`m2-mesh-load-pipeline-state.md`](m2-mesh-load-pipeline-state.md) +- [`m2-mesh-resource-cache-state.md`](m2-mesh-resource-cache-state.md) +- [`world-renderer.md`](world-renderer.md) +- [`../../RENDER.md`](../../RENDER.md) diff --git a/docs/modules/world-renderer.md b/docs/modules/world-renderer.md index cca6440..890c5f5 100644 --- a/docs/modules/world-renderer.md +++ b/docs/modules/world-renderer.md @@ -142,6 +142,7 @@ from externally reading/writing loader-private queue, task, cache and tile-state | `M2BuildBatchPlanner.plan_batch` | Internal pure M2 service | Selects static/animated batch count and next group cursor | Main/any thread; stateless | Non-positive selected limit clamps to one; empty range completes | | `M2BuildDispatchPlanner.plan_step` | Internal pure M2 service | Selects wait, animated/static materialization or no-Node advancement from observed resource state | Main/any thread; stateless | Pending animation has priority; unresolved static Mesh waits | | `M2BuildResourceSnapshot` | Internal M2 value contract | Carries per-step normalized path, animated/static references and pending/missing observations | Renderer main thread; one build operation | Values retained exactly; release never frees resources | +| `M2StaticBuildResourceObserver.observe` | Internal M2 I/O service | Produces cached/pending/requested/missing static snapshot phase | Renderer main thread; stateless | Invalid composition rejected; no candidate marks missing | | `M2BuildQueue` / `M2BuildJob` | Internal M2 pending-state service | Own typed root/groups/cursor jobs and FIFO/stale tile keys | Renderer main thread; map session | Invalid enqueue rejected; stale keys drain independently of jobs | | `M2StaticBatchMaterializer.materialize_batch` | Internal M2 scene-materialization service | Builds and attaches one prepared-Mesh MultiMesh transform slice | Renderer main thread; stateless after each call | Invalid/empty input returns null; bounds are caller precondition | | `M2RuntimeMeshRebuildClassifier` | Internal memoized M2 service | Detects billboard/UV-rotation metadata requiring stale cached-mesh rebuild | Renderer main thread; cached until reset | Invalid variants/indices skipped; first path decision wins | @@ -393,9 +394,10 @@ sequenceDiagram without controlling engine lifetime. `M2BuildQueue` owns typed pending jobs, FIFO/stale keys, grouped-transform references and group/offset/serial cursors without freeing engine objects. - `M2StaticBatchMaterializer` owns static MultiMesh construction/attachment; - the loader retains resource observation/requests, action execution, - cursor adoption, root cleanup, budgets and Editor ownership. + `M2StaticBuildResourceObserver` owns static Mesh lookup/request admission and + snapshot adoption. `M2StaticBatchMaterializer` owns static MultiMesh + construction/attachment; the loader retains animated observation/requests, + action execution, cursor adoption, root cleanup, budgets and Editor ownership. - `WmoPlacementResolver` is stateless and owns only call-local cache-key, identity and transform values. `WmoPlacementRegistry` owns only placement-key reference sets. `WmoRenderBuildStepPlanner` owns only a call-local operation @@ -416,15 +418,17 @@ sequenceDiagram for billboard/UV-rotation material refresh and is composed by the runtime Mesh finalizer. The raw repository loads value data; loader retains Mesh adoption. - `M2MeshLoadPipelineState` owns static M2 pending Resource paths, opaque - terminal statuses and completion-order finalize FIFO. The loader retains cache - path selection, ResourceLoader calls, permits and adoption decisions; prototype - cache state owns shared missing outcomes. + terminal statuses and completion-order finalize FIFO. The static observer owns + cache path selection, request admission and initial snapshot adoption. The + loader retains ResourceLoader polling/finalize, permits and terminal adoption; + prototype cache state owns shared missing outcomes. - `M2MeshResourceCacheState` owns prepared static Mesh references and releases them at the existing final-shutdown site. Prototype state and materialization belong to the sibling cache service and loader respectively. - `M2MeshResourceExtractor` owns depth-first first-Mesh selection and temporary - PackedScene instance destruction. The loader retains ResourceLoader I/O, - cache/missing adoption and materialization. + PackedScene instance destruction. The static observer admits ResourceLoader + requests and initial cache/missing adoption; the loader retains polling, + terminal adoption and materialization. - `M2RuntimeMeshFinalizer` owns refresh version `2`, classifier lifetime, M2Builder rebuild and original-Mesh fallback. The loader loads raw data only after the finalizer reports that a cached Mesh is stale. @@ -579,6 +583,7 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m | M2 build batch planner | Implemented extraction | Scene-free limit/count/cursor/source/timing contract | Spatial cells and asset-backed p95/p99 remain pending | | M2 build dispatch planner | Implemented extraction | Scene-free priority/action/transition/source/timing contract | Resource observation/orchestration and asset-backed traversal remain pending | | M2 build resource snapshot | Implemented extraction | Typed identity/adoption/lifetime/source/timing contract | Resource observation service and asset-backed traversal remain pending | +| M2 static build resource observer | Implemented extraction | Cache/pending/path/request/source/timing contract | Asset-backed traversal and finalize extraction pending | | M2 build queue | Implemented extraction | Typed lifecycle/FIFO/progress/lifetime/source/timing contract | Resource dispatch and asset-backed traversal remain pending | | M2 animation load pipeline state | Implemented extraction | Synthetic lifecycle/FIFO/source/timing contract | Asset-backed traversal/leak/animation-fidelity/p95/p99 pending | | M2 animated scene finalizer | Implemented extraction | Synthetic type/lifetime/material/player/source/timing contract | Asset-backed GLB traversal/material/animation comparison pending | @@ -652,6 +657,7 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m | `src/render/m2/m2_static_batch_materializer.gd` | Static MultiMesh construction, render setup and attachment | | `src/render/m2/m2_build_dispatch_planner.gd` | M2 wait/materializer/advance action priority and transition plan | | `src/render/m2/m2_build_resource_snapshot.gd` | Typed per-step animated/static observations and borrowed references | +| `src/render/m2/m2_static_build_resource_observer.gd` | Static Mesh lookup/request/missing snapshot production | | `src/render/m2/m2_build_job.gd` | M2 root/groups references and group/offset/serial progress | | `src/render/m2/m2_build_queue.gd` | Keyed pending jobs and FIFO/stale tile-key lifecycle | | `src/render/m2/m2_mesh_load_pipeline_state.gd` | Static M2 pending Resource paths, terminal statuses and finalize FIFO | @@ -682,6 +688,7 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m | `src/tools/verify_m2_static_batch_materializer.gd` | Static M2 node/Mesh/render/attachment/boundary/timing regression | | `src/tools/verify_m2_build_dispatch_planner.gd` | M2 dispatch priority/action/transition/boundary/timing regression | | `src/tools/verify_m2_build_resource_snapshot.gd` | M2 observation identity/adoption/lifetime/boundary/timing regression | +| `src/tools/verify_m2_static_build_resource_observer.gd` | Static lookup/request/path/boundary/timing regression | | `src/tools/verify_m2_build_queue.gd` | M2 job/FIFO/progress/lifetime/boundary/timing regression | | `src/tools/verify_m2_mesh_load_pipeline_state.gd` | Static M2 request/terminal/FIFO/boundary/timing regression | | `src/tools/verify_m2_mesh_resource_cache_state.gd` | Static M2 Mesh cache ownership/lifetime/boundary/timing regression | diff --git a/src/render/m2/m2_static_build_resource_observer.gd b/src/render/m2/m2_static_build_resource_observer.gd new file mode 100644 index 0000000..41b280d --- /dev/null +++ b/src/render/m2/m2_static_build_resource_observer.gd @@ -0,0 +1,126 @@ +class_name M2StaticBuildResourceObserver +extends RefCounted + +## Resolves or requests one static M2 build Mesh and fills a resource snapshot. + +const OUTCOME_REJECTED := &"rejected" +const OUTCOME_CACHED := &"cached" +const OUTCOME_PENDING := &"pending" +const OUTCOME_REQUESTED := &"requested" +const OUTCOME_MISSING := &"missing" + + +## Observes cache/request state and adopts the exact static result into snapshot. +func observe( + resource_snapshot: RefCounted, + normalized_relative_path: String, + cache_directory: String, + mesh_cache_state: RefCounted, + prototype_cache_state: RefCounted, + load_pipeline_state: RefCounted +) -> StringName: + if ( + resource_snapshot == null + or normalized_relative_path.is_empty() + or mesh_cache_state == null + or prototype_cache_state == null + or load_pipeline_state == null + ): + return OUTCOME_REJECTED + if bool(mesh_cache_state.call("has_mesh", normalized_relative_path)): + resource_snapshot.call( + "adopt_static_observation", + mesh_cache_state.call("find_mesh", normalized_relative_path) as Mesh, + false + ) + return OUTCOME_CACHED + if bool(prototype_cache_state.call("is_model_missing", normalized_relative_path)): + resource_snapshot.call("adopt_static_observation", null, true) + return OUTCOME_MISSING + if bool(load_pipeline_state.call("has_request", normalized_relative_path)): + resource_snapshot.call("adopt_static_observation", null, false) + return OUTCOME_PENDING + + for cache_resource_path in cache_resource_paths( + cache_directory, + normalized_relative_path, + [".tscn", ".glb"] + ): + if not ResourceLoader.exists(cache_resource_path): + continue + if ( + cache_resource_path.get_extension().to_lower() == "glb" + and glb_animation_schema(cache_resource_path) == "pivot_prefix_v1" + ): + continue + var request_error := ResourceLoader.load_threaded_request( + cache_resource_path, + "", + false, + ResourceLoader.CACHE_MODE_REUSE + ) + if request_error == OK or request_error == ERR_BUSY: + load_pipeline_state.call( + "remember_request", + normalized_relative_path, + cache_resource_path + ) + resource_snapshot.call("adopt_static_observation", null, false) + return OUTCOME_REQUESTED + break + + prototype_cache_state.call("mark_model_missing", normalized_relative_path) + resource_snapshot.call("adopt_static_observation", null, true) + return OUTCOME_MISSING + + +## Returns historical nested/lowercase/basename cache candidates without I/O. +func cache_resource_paths( + cache_directory: String, + relative_path: String, + extensions: Array[String] +) -> PackedStringArray: + var normalized := relative_path.replace("\\", "/") + var lower := normalized.to_lower() + var stems := [ + normalized.get_basename(), + lower.get_basename(), + normalized.get_file().get_basename(), + lower.get_file().get_basename(), + ] + var result := PackedStringArray() + for extension in extensions: + for stem in stems: + if stem.is_empty(): + continue + var path := cache_directory.path_join(stem + extension) + if not result.has(path): + result.append(path) + return result + + +## Reads only the OpenWC animation schema marker from a GLB JSON chunk. +func glb_animation_schema(cache_resource_path: String) -> String: + var absolute_path := ProjectSettings.globalize_path(cache_resource_path) + if not FileAccess.file_exists(absolute_path): + return "" + var file := FileAccess.open(absolute_path, FileAccess.READ) + if file == null or file.get_length() < 20: + return "" + var magic := file.get_32() + var version := file.get_32() + file.get_32() + if magic != 0x46546c67 or version != 2: + return "" + var json_length := int(file.get_32()) + var chunk_type := file.get_32() + if json_length <= 0 or chunk_type != 0x4e4f534a: + return "" + var parsed: Variant = JSON.parse_string( + file.get_buffer(json_length).get_string_from_utf8() + ) + if not (parsed is Dictionary): + return "" + var asset: Dictionary = (parsed as Dictionary).get("asset", {}) + var extras: Dictionary = asset.get("extras", {}) + return String(extras.get("openwc_m2_anim_schema", "")) diff --git a/src/render/m2/m2_static_build_resource_observer.gd.uid b/src/render/m2/m2_static_build_resource_observer.gd.uid new file mode 100644 index 0000000..07fd72a --- /dev/null +++ b/src/render/m2/m2_static_build_resource_observer.gd.uid @@ -0,0 +1 @@ +uid://c0e3p80ld1dfb diff --git a/src/scenes/streaming/streaming_world_loader.gd b/src/scenes/streaming/streaming_world_loader.gd index 961a09a..d19af51 100644 --- a/src/scenes/streaming/streaming_world_loader.gd +++ b/src/scenes/streaming/streaming_world_loader.gd @@ -70,6 +70,9 @@ const M2_BUILD_QUEUE_SCRIPT := preload("res://src/render/m2/m2_build_queue.gd") const M2_STATIC_BATCH_MATERIALIZER_SCRIPT := preload( "res://src/render/m2/m2_static_batch_materializer.gd" ) +const M2_STATIC_BUILD_RESOURCE_OBSERVER_SCRIPT := preload( + "res://src/render/m2/m2_static_build_resource_observer.gd" +) const M2_RUNTIME_MESH_FINALIZER_SCRIPT := preload( "res://src/render/m2/m2_runtime_mesh_finalizer.gd" ) @@ -277,6 +280,9 @@ var _m2_group_result_mutex := Mutex.new() var _m2_group_result_queue: Array = [] var _m2_build_queue_state := M2_BUILD_QUEUE_SCRIPT.new() var _m2_build_dispatch_planner := M2_BUILD_DISPATCH_PLANNER_SCRIPT.new() +var _m2_static_build_resource_observer := ( + M2_STATIC_BUILD_RESOURCE_OBSERVER_SCRIPT.new() +) var _m2_unique_placement_registry := ( M2_UNIQUE_PLACEMENT_REGISTRY_SCRIPT.new() ) @@ -4373,14 +4379,13 @@ func _process_m2_build_jobs() -> void: batch_count > 0 and not bool(resource_snapshot.call("has_animated_prototype")) ): - var static_mesh := _get_m2_mesh_or_request(rel_path) - resource_snapshot.call( - "adopt_static_observation", - static_mesh, - ( - static_mesh == null - and _m2_prototype_cache_state.is_model_missing(normalized_rel) - ) + _m2_static_build_resource_observer.observe( + resource_snapshot, + normalized_rel, + m2_cache_dir, + _m2_mesh_resource_cache_state, + _m2_prototype_cache_state, + _m2_mesh_load_pipeline_state ) var dispatch_plan: Dictionary = _m2_build_dispatch_planner.plan_step( batch_count, @@ -4587,40 +4592,6 @@ func _normalize_m2_rel_path(rel_path: String) -> String: return normalized_rel -func _get_m2_mesh_or_request(rel_path: String) -> Mesh: - var normalized_rel := _normalize_m2_rel_path(rel_path) - if normalized_rel.is_empty(): - return null - if _m2_mesh_resource_cache_state.has_mesh(normalized_rel): - return _m2_mesh_resource_cache_state.find_mesh(normalized_rel) - if _m2_prototype_cache_state.is_model_missing(normalized_rel): - return null - _request_m2_mesh_load(normalized_rel) - return null - - -func _request_m2_mesh_load(normalized_rel: String) -> void: - if normalized_rel.is_empty() or _m2_mesh_load_pipeline_state.has_request(normalized_rel): - return - - for cache_res_path in _get_m2_cache_resource_paths(normalized_rel, [".tscn", ".glb"]): - if not ResourceLoader.exists(cache_res_path): - continue - if cache_res_path.get_extension().to_lower() == "glb" and _glb_cache_animation_schema(cache_res_path) == "pivot_prefix_v1": - continue - var err := ResourceLoader.load_threaded_request( - cache_res_path, - "", - false, - ResourceLoader.CACHE_MODE_REUSE) - if err == OK or err == ERR_BUSY: - _m2_mesh_load_pipeline_state.remember_request(normalized_rel, cache_res_path) - return - break - - _m2_prototype_cache_state.mark_model_missing(normalized_rel) - - func _prepare_m2_mesh_for_runtime(normalized_rel: String, mesh: Mesh) -> Mesh: if mesh == null: return null @@ -4820,7 +4791,12 @@ func _get_or_load_m2_prototype(rel_path: String) -> Node3D: for cache_res_path in _get_m2_cache_resource_paths(normalized_rel, [".tscn", ".glb"]): if not ResourceLoader.exists(cache_res_path): continue - if cache_res_path.get_extension().to_lower() == "glb" and _glb_cache_animation_schema(cache_res_path) == "pivot_prefix_v1": + if ( + cache_res_path.get_extension().to_lower() == "glb" + and _m2_static_build_resource_observer.glb_animation_schema( + cache_res_path + ) == "pivot_prefix_v1" + ): continue var resource: Resource = load(cache_res_path) if resource is PackedScene: @@ -5041,28 +5017,6 @@ func _glb_primitive_count(gltf: Dictionary) -> int: return count -func _glb_cache_animation_schema(cache_res_path: String) -> String: - var abs_path := ProjectSettings.globalize_path(cache_res_path) - if not FileAccess.file_exists(abs_path): - return "" - var file := FileAccess.open(abs_path, FileAccess.READ) - if file == null or file.get_length() < 20: - return "" - var magic := file.get_32() - var version := file.get_32() - file.get_32() - if magic != 0x46546c67 or version != 2: - return "" - var json_len := int(file.get_32()) - var chunk_type := file.get_32() - if json_len <= 0 or chunk_type != 0x4e4f534a: - return "" - var parsed = JSON.parse_string(file.get_buffer(json_len).get_string_from_utf8()) - if parsed is Dictionary: - return _glb_animation_schema(parsed as Dictionary) - return "" - - func _glb_animation_schema(gltf: Dictionary) -> String: var asset: Dictionary = gltf.get("asset", {}) var extras: Dictionary = asset.get("extras", {}) diff --git a/src/tools/verify_m2_build_dispatch_planner.gd b/src/tools/verify_m2_build_dispatch_planner.gd index a2d8513..ee052b7 100644 --- a/src/tools/verify_m2_build_dispatch_planner.gd +++ b/src/tools/verify_m2_build_dispatch_planner.gd @@ -160,7 +160,7 @@ func _verify_loader_and_dependency_boundaries(failures: Array[String]) -> void: failures ) _expect_true( - loader_source.contains("_get_m2_mesh_or_request(rel_path)") + loader_source.contains("_m2_static_build_resource_observer.observe(") and loader_source.contains("_materialize_m2_animated_batch(") and loader_source.contains("_materialize_m2_group_batch(") and loader_source.contains("try_consume_permit("), diff --git a/src/tools/verify_m2_build_resource_snapshot.gd b/src/tools/verify_m2_build_resource_snapshot.gd index b68e005..a35352c 100644 --- a/src/tools/verify_m2_build_resource_snapshot.gd +++ b/src/tools/verify_m2_build_resource_snapshot.gd @@ -8,6 +8,7 @@ const SNAPSHOT_SCRIPT := preload( ) const SNAPSHOT_PATH := "res://src/render/m2/m2_build_resource_snapshot.gd" const DISPATCH_PATH := "res://src/render/m2/m2_build_dispatch_planner.gd" +const STATIC_OBSERVER_PATH := "res://src/render/m2/m2_static_build_resource_observer.gd" const LOADER_PATH := "res://src/scenes/streaming/streaming_world_loader.gd" @@ -134,10 +135,10 @@ func _verify_engine_lifetime(failures: Array[String]) -> void: func _verify_loader_and_dependency_boundaries(failures: Array[String]) -> void: var loader_source := FileAccess.get_file_as_string(LOADER_PATH) var dispatch_source := FileAccess.get_file_as_string(DISPATCH_PATH) + var static_observer_source := FileAccess.get_file_as_string(STATIC_OBSERVER_PATH) var snapshot_source := FileAccess.get_file_as_string(SNAPSHOT_PATH) for delegated_token in [ "M2_BUILD_RESOURCE_SNAPSHOT_SCRIPT.new(", - "\"adopt_static_observation\"", "resource_snapshot.call(\"animated_prototype\")", "resource_snapshot.call(\"static_mesh\")", ]: @@ -146,6 +147,11 @@ func _verify_loader_and_dependency_boundaries(failures: Array[String]) -> void: "loader uses %s" % delegated_token, failures ) + _expect_true( + static_observer_source.contains("\"adopt_static_observation\""), + "static observer adopts snapshot", + failures + ) _expect_true( dispatch_source.contains("resource_snapshot.call(\"animation_request_pending\")") and dispatch_source.contains("resource_snapshot.call(\"has_static_mesh\")"), diff --git a/src/tools/verify_m2_mesh_resource_cache_state.gd b/src/tools/verify_m2_mesh_resource_cache_state.gd index ecb8d94..0887e2a 100644 --- a/src/tools/verify_m2_mesh_resource_cache_state.gd +++ b/src/tools/verify_m2_mesh_resource_cache_state.gd @@ -6,6 +6,7 @@ extends SceneTree const CACHE_SCRIPT := preload("res://src/render/m2/m2_mesh_resource_cache_state.gd") const CACHE_PATH := "res://src/render/m2/m2_mesh_resource_cache_state.gd" const LOADER_PATH := "res://src/scenes/streaming/streaming_world_loader.gd" +const STATIC_OBSERVER_PATH := "res://src/render/m2/m2_static_build_resource_observer.gd" func _initialize() -> void: @@ -68,6 +69,7 @@ func _verify_clear_and_diagnostics(failures: Array[String]) -> void: func _verify_ownership_boundaries(failures: Array[String]) -> void: var cache_source := FileAccess.get_file_as_string(CACHE_PATH) var loader_source := FileAccess.get_file_as_string(LOADER_PATH) + var observer_source := FileAccess.get_file_as_string(STATIC_OBSERVER_PATH) _expect_true( loader_source.contains("M2_MESH_RESOURCE_CACHE_STATE_SCRIPT.new()"), "loader composes Mesh cache state", @@ -81,7 +83,8 @@ func _verify_ownership_boundaries(failures: Array[String]) -> void: failures ) _expect_equal( - loader_source.count("_m2_mesh_resource_cache_state.find_mesh("), + loader_source.count("_m2_mesh_resource_cache_state.find_mesh(") + + observer_source.count("\"find_mesh\""), 2, "two existing lookups delegate", failures diff --git a/src/tools/verify_m2_static_build_resource_observer.gd b/src/tools/verify_m2_static_build_resource_observer.gd new file mode 100644 index 0000000..b0e01f8 --- /dev/null +++ b/src/tools/verify_m2_static_build_resource_observer.gd @@ -0,0 +1,127 @@ +extends SceneTree + +const OBSERVER_SCRIPT := preload("res://src/render/m2/m2_static_build_resource_observer.gd") +const SNAPSHOT_SCRIPT := preload("res://src/render/m2/m2_build_resource_snapshot.gd") +const MESH_CACHE_SCRIPT := preload("res://src/render/m2/m2_mesh_resource_cache_state.gd") +const PROTOTYPE_CACHE_SCRIPT := preload("res://src/render/m2/m2_prototype_cache_state.gd") +const PIPELINE_SCRIPT := preload("res://src/render/m2/m2_mesh_load_pipeline_state.gd") +const OBSERVER_PATH := "res://src/render/m2/m2_static_build_resource_observer.gd" +const LOADER_PATH := "res://src/scenes/streaming/streaming_world_loader.gd" + + +func _initialize() -> void: + var failures: Array[String] = [] + _verify_invalid_cached_missing_pending(failures) + _verify_request_and_path_order(failures) + _verify_source_boundaries(failures) + var elapsed := _verify_timing(failures) + if not failures.is_empty(): + for failure in failures: + push_error("M2_STATIC_BUILD_RESOURCE_OBSERVER: %s" % failure) + quit(1) + return + print("M2_STATIC_BUILD_RESOURCE_OBSERVER PASS cases=11 iterations=20000 elapsed_ms=%.3f" % elapsed) + quit(0) + + +func _verify_invalid_cached_missing_pending(failures: Array[String]) -> void: + var observer: RefCounted = OBSERVER_SCRIPT.new() + var mesh_cache: RefCounted = MESH_CACHE_SCRIPT.new() + var prototype_cache: RefCounted = PROTOTYPE_CACHE_SCRIPT.new() + var pipeline: RefCounted = PIPELINE_SCRIPT.new() + var snapshot: RefCounted = SNAPSHOT_SCRIPT.new("world/a.m2", null, false) + _expect( + observer.call( + "observe", null, "world/a.m2", "res://none", + mesh_cache, prototype_cache, pipeline + ) + == OBSERVER_SCRIPT.OUTCOME_REJECTED, + "null snapshot rejected", failures + ) + var mesh := ArrayMesh.new() + mesh_cache.call("store_mesh", "world/a.m2", mesh) + _expect( + observer.call( + "observe", snapshot, "world/a.m2", "res://none", + mesh_cache, prototype_cache, pipeline + ) + == OBSERVER_SCRIPT.OUTCOME_CACHED, "cached outcome", failures + ) + _expect(is_same(snapshot.call("static_mesh"), mesh), "cached Mesh identity", failures) + mesh_cache.call("clear") + prototype_cache.call("mark_model_missing", "world/a.m2") + _expect( + observer.call( + "observe", snapshot, "world/a.m2", "res://none", + mesh_cache, prototype_cache, pipeline + ) + == OBSERVER_SCRIPT.OUTCOME_MISSING, "missing outcome", failures + ) + _expect(bool(snapshot.call("static_model_missing")), "missing adopted", failures) + prototype_cache = PROTOTYPE_CACHE_SCRIPT.new() + pipeline.call("remember_request", "world/a.m2", "res://pending.tscn") + _expect( + observer.call( + "observe", snapshot, "world/a.m2", "res://none", + mesh_cache, prototype_cache, pipeline + ) + == OBSERVER_SCRIPT.OUTCOME_PENDING, "pending outcome", failures + ) + _expect(not bool(snapshot.call("static_model_missing")), "pending is not missing", failures) + pipeline.call("clear") + prototype_cache.call("clear_and_release") + + +func _verify_request_and_path_order(failures: Array[String]) -> void: + var observer: RefCounted = OBSERVER_SCRIPT.new() + var extensions: Array[String] = [".tscn", ".glb"] + var paths: PackedStringArray = observer.call( + "cache_resource_paths", "res://cache", "World/Foo/Bar.M2", extensions + ) + var expected := PackedStringArray([ + "res://cache/World/Foo/Bar.tscn", "res://cache/world/foo/bar.tscn", + "res://cache/Bar.tscn", "res://cache/bar.tscn", + "res://cache/World/Foo/Bar.glb", "res://cache/world/foo/bar.glb", + "res://cache/Bar.glb", "res://cache/bar.glb", + ]) + _expect(paths == expected, "candidate order", failures) + _expect( + observer.call("glb_animation_schema", "res://missing.glb") == "", + "missing GLB schema empty", failures + ) + + +func _verify_source_boundaries(failures: Array[String]) -> void: + var loader := FileAccess.get_file_as_string(LOADER_PATH) + var observer := FileAccess.get_file_as_string(OBSERVER_PATH) + _expect( + loader.contains("_m2_static_build_resource_observer.observe("), + "loader delegates observer", failures + ) + _expect(not loader.contains("func _get_m2_mesh_or_request"), "legacy lookup removed", failures) + _expect(not loader.contains("func _request_m2_mesh_load"), "legacy request removed", failures) + _expect( + observer.contains("ResourceLoader.load_threaded_request("), + "observer owns request", failures + ) + for forbidden in ["queue_free", "RenderingServer", "WorkerThreadPool", "Mutex"]: + _expect(not observer.contains(forbidden), "observer omits %s" % forbidden, failures) + + +func _verify_timing(failures: Array[String]) -> float: + var observer: RefCounted = OBSERVER_SCRIPT.new() + var extensions: Array[String] = [".tscn", ".glb"] + var started := Time.get_ticks_usec() + for index in range(20000): + observer.call( + "cache_resource_paths", "res://cache", + "world/%d/model.m2" % (index % 64), extensions + ) + var elapsed := float(Time.get_ticks_usec() - started) / 1000.0 + _expect(elapsed < 1000.0, "path planning bounded", failures) + return elapsed + + +func _expect(condition: bool, label: String, failures: Array[String]) -> void: + if not condition: + failures.append(label) diff --git a/src/tools/verify_m2_static_build_resource_observer.gd.uid b/src/tools/verify_m2_static_build_resource_observer.gd.uid new file mode 100644 index 0000000..8da3810 --- /dev/null +++ b/src/tools/verify_m2_static_build_resource_observer.gd.uid @@ -0,0 +1 @@ +uid://b8bxs4swxsntd diff --git a/targets/03-renderer-facade.md b/targets/03-renderer-facade.md index 711021e..4a41d15 100644 --- a/targets/03-renderer-facade.md +++ b/targets/03-renderer-facade.md @@ -58,7 +58,8 @@ Runtime и Editor используют facade; planner/scheduler тестиру `M03-RND-M2-STATIC-BATCH-MATERIALIZER-001`, `M03-RND-M2-BUILD-QUEUE-001`, `M03-RND-M2-BUILD-DISPATCH-PLANNER-001`, - `M03-RND-M2-BUILD-RESOURCE-SNAPSHOT-001` + `M03-RND-M2-BUILD-RESOURCE-SNAPSHOT-001`, + `M03-RND-M2-STATIC-BUILD-RESOURCE-OBSERVER-001` - Commands: dedicated scheduler/planner/facade/focus/coordinate/manifest/shutdown headless verifiers; Godot cold editor parse; coordination, documentation and diff gates. - Results: scheduler `cases=6 iterations=20000 elapsed_ms=10.216`; planner @@ -275,6 +276,10 @@ Runtime и Editor используют facade; planner/scheduler тестиру materializers, animation/Mesh pipelines, Mesh/prototype state, shutdown, facade, internal-access `30`, manifest and checkpoint dry-run `7/7`, documentation and coordination remained green. + M2 static build resource observer passed `cases=11 iterations=20000 + elapsed_ms=64.158` with invalid/cached/Mesh-identity/missing/pending outcomes, + exact candidate order, GLB schema fallback and source ownership. All 55 + autonomous headless regressions passed; internal-access remains `30`. M02 terrain-query regression remained green (13 pre-existing expired M00 claim warnings). - Fidelity comparison: all 16 historical operation limits and drain sites were @@ -429,6 +434,7 @@ Runtime и Editor используют facade; planner/scheduler тестиру typed M2 build job/queue and loader enqueue/drain/progress/cancel/clear adapters, stateless M2 build dispatch planner and loader resource-observation/action adapter, typed M2 build resource snapshot and loader/dispatch/materializer adapters, + static M2 build resource observer and loader/cache/pipeline/snapshot adapters, expanded facade/internal-access/coordinate verifiers; work-package claims and this evidence. - Remaining risks: worker concurrency and stale-result validation remain streamer-owned; cancellation stops new permits but does not interrupt @@ -455,6 +461,8 @@ Runtime и Editor используют facade; planner/scheduler тестиру separated and dispatch decisions are explicit, while resource observation/ requests, action execution and root cleanup remain in loader; resource state is typed per operation but its producing service is not yet extracted; + static lookup/request production is extracted, while animated observation and + both ResourceLoader finalize drains remain loader-owned; WMO ResourceLoader/FileAccess I/O, live fallback and group materialization remain in the loader; asset-backed WMO placement/portal/material/leak/p95/p99 evidence remains pending;