sindoring/open-wc
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

render: extract WMO scene resource finalizer

Browse Source
This commit is contained in:
sindoring
2026-07-18 14:16:49 +04:00
parent 9ba7ca3263
commit 7779e9e75f
12 changed files with 766 additions and 60 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/modules/README.md
+1
View File
@@ -48,6 +48,7 @@
| WMO render Resource cache state | Implemented extraction | [`wmo-render-resource-cache-state.md`](wmo-render-resource-cache-state.md) |
| WMO render Resource finalizer | Implemented extraction | [`wmo-render-resource-finalizer.md`](wmo-render-resource-finalizer.md) |
| WMO scene Resource cache state | Implemented extraction | [`wmo-scene-resource-cache-state.md`](wmo-scene-resource-cache-state.md) |
| WMO scene Resource finalizer | Implemented extraction | [`wmo-scene-resource-finalizer.md`](wmo-scene-resource-finalizer.md) |
| Third-person camera | Implemented | [`third-person-camera.md`](third-person-camera.md) |
| Character presentation | Implemented boundary / Partial fidelity | [`character-presentation.md`](character-presentation.md) |
| Renderer | Partial | [`world-renderer.md`](world-renderer.md), [`../../RENDER.md`](../../RENDER.md) |
docs/modules/wmo-scene-resource-cache-state.md
+21 -15
View File
@@ -14,8 +14,9 @@
Own mutually exclusive cached, missing and pending states for normalized WMO
scene paths. The state accepts only caller-validated `PackedScene` references and
records `.tscn` paths for threaded requests whose I/O, size limit and cache
metadata validation remain in `StreamingWorldLoader`.
records `.tscn` paths for threaded requests. Size admission remains in
`StreamingWorldLoader`; terminal I/O/probe validation belongs to
`WmoSceneResourceFinalizer`.
## Non-goals
@@ -33,8 +34,8 @@ flowchart LR
Loader --> State[WmoSceneResourceCacheState]
Loader --> Size[File existence and size limit]
Size --> ResourceLoader[Godot ResourceLoader]
ResourceLoader --> Loader
Loader --> Probe[Instantiate, metadata/version check, free]
ResourceLoader --> Finalizer[WmoSceneResourceFinalizer]
Finalizer --> Probe[Instantiate, metadata/version check, free]
Probe -->|accepted PackedScene or missing| State
Loader --> Live[Live-prototype fallback]
```
@@ -82,7 +83,7 @@ flowchart TD
Check -->|allowed| Request[Start threaded request]
Request -->|error| Mark
Request -->|accepted| Remember[remember_request]
Remember --> Poll[Loader polls snapshot]
Remember --> Poll[Scene Resource finalizer polls snapshot]
Poll -->|failure| CompleteMissing[complete as missing]
Poll -->|loaded| Validate[Instantiate and validate cache metadata]
Validate -->|valid| CompleteScene[complete with scene]
@@ -114,24 +115,27 @@ stateDiagram-v2
sequenceDiagram
participant Loader as StreamingWorldLoader
participant State as WmoSceneResourceCacheState
participant Finalizer as WmoSceneResourceFinalizer
participant RL as ResourceLoader
Loader->>State: cached/missing/pending queries
Loader->>Loader: exists and wmo_max_runtime_scene_mb check
Loader->>RL: load_threaded_request(.tscn)
Loader->>State: remember_request(normalized, path)
loop renderer tick
Loader->>State: request_paths_snapshot()
Loader->>RL: load_threaded_get_status(path)
Loader->>Finalizer: poll_terminal_requests(State)
Finalizer->>State: request_paths_snapshot()
Finalizer->>RL: load_threaded_get_status(path)
end
Loader->>RL: load_threaded_get(path)
Loader->>Loader: instantiate, validate WMO metadata, free probe
Loader->>State: complete with scene or as missing
Finalizer->>RL: load_threaded_get(path)
Finalizer->>Finalizer: instantiate, validate WMO metadata, free probe
Finalizer->>State: complete with scene or as missing
```
## Ownership, threading and resources
- State owns three Dictionaries and strong references to accepted PackedScenes.
- Loader owns paths, file measurement, requests, validation and all Node lifetime.
- Loader owns paths, file measurement, request admission and placed Nodes;
finalizer owns terminal I/O and call-local validation-probe lifetime.
- All mutation is serialized on the renderer main thread; no mutex is required.
- Detached request snapshots allow terminal removal while polling.
- Scene instantiation borrows the PackedScene and does not transfer cache ownership.
@@ -143,7 +147,7 @@ sequenceDiagram
| Missing `.tscn` | Loader ResourceLoader existence check | Direct missing state | Transient reset permits later retry |
| Oversize `.tscn` | Loader byte limit | Missing plus unchanged debug log | Raise limit/rebuild, then reset |
| Request-start/load failure | Loader error/status | Direct or terminal missing | Reset and retry later |
| Wrong type/stale metadata | Loader PackedScene/probe validation | Terminal missing; probe freed | Rebuild cache and reset |
| Wrong type/stale metadata | Finalizer PackedScene/probe validation | Terminal missing; probe freed | Rebuild cache and reset |
| Shutdown while pending | Loader drains snapshot | Clear transient, then full cache release | New loader starts absent |
## Configuration and capabilities
@@ -184,8 +188,9 @@ in the loader; the state emits no logs.
| Capability | Status | Evidence | Gap/next step |
|---|---|---|---|
| Cached WMO PackedScene state | Implemented extraction | Lifecycle/source/timing and shutdown verifiers | Asset-backed traversal/leak evidence pending |
| Size and cache metadata validation | Preserved in loader | Source boundary and WMO regressions | Oversize/stale asset fixtures could follow |
| ResourceLoader I/O and live fallback | Partial/loader-owned | Existing runtime behavior | Separate extraction if justified |
| Size admission | Preserved in loader | Source boundary and WMO regressions | Oversize asset fixture could follow |
| Terminal I/O and cache validation | Implemented extraction | Finalizer type/lifetime/source/timing verifier | Serialized stale fixture pending |
| Live fallback | Loader-owned | Existing runtime behavior | Separate extraction if justified |
## Known gaps and risks
@@ -199,7 +204,8 @@ in the loader; the state emits no logs.
| Path | Responsibility |
|---|---|
| `src/render/wmo/wmo_scene_resource_cache_state.gd` | Scene/missing/request state and resets |
| `src/scenes/streaming/streaming_world_loader.gd` | Paths, file limit, requests, validation, fallback and Node lifetime |
| `src/render/wmo/wmo_scene_resource_finalizer.gd` | Terminal I/O, probe validation/lifetime and publication |
| `src/scenes/streaming/streaming_world_loader.gd` | Paths, file limit, request admission, fallback and placed Nodes |
| `src/tools/verify_wmo_scene_resource_cache_state.gd` | State, boundary and timing regression |
| `src/tools/verify_render_runtime_cache_shutdown.gd` | Loader final cache ownership regression |
docs/modules/wmo-scene-resource-finalizer.md
+233
View File
@@ -0,0 +1,233 @@
# WMO Scene Resource Finalizer
## Metadata
| Field | Value |
|---|---|
| Status | Implemented extraction |
| Target/work package | M03 / `M03-RND-WMO-SCENE-RESOURCE-FINALIZER-001` |
| Owners | Cached WMO PackedScene terminal polling, validation-probe lifetime and scene/missing publication |
| Last verified | Worktree `work/sindo-main-codex/m03-wmo-scene-resource-finalizer`, 2026-07-18 |
| Profiles/capabilities | Profile-independent cached WMO `.tscn` finalization |
## Purpose
Finalize pending cached WMO `.tscn` requests outside `StreamingWorldLoader`:
poll terminal status, retrieve a PackedScene, instantiate one validation probe,
apply existing WMOBuilder cache metadata rules, release the probe and publish
either the exact scene or the historical missing outcome.
## Non-goals
- Select paths, measure file size or start ResourceLoader requests.
- Instantiate placed WMO scenes or execute live WMOBuilder fallback.
- Own cache lifetime, placements, build queues, permits or attached Nodes.
- Change cache metadata/version rules, fallback order or visible behavior.
## Context and boundaries
```mermaid
flowchart LR
Loader[StreamingWorldLoader] -->|compose and tick| Finalizer[WmoSceneResourceFinalizer]
Cache[WmoSceneResourceCacheState] -->|pending snapshot| Finalizer
Finalizer --> ResourceLoader
Finalizer --> Probe[Temporary Node3D probe]
Probe --> Validator[WMOBuilder cache validator]
Finalizer -->|PackedScene or missing| Cache
Cache --> Fallback[Placed cached scene or live fallback]
```
The service may depend on ResourceLoader, PackedScene/Node3D lifetime and the
injected cache validator. FileAccess, size policy, placement, attached SceneTree,
scheduler and application layers remain outside it.
## Public API
| Symbol | Kind | Purpose | Thread/lifetime | Errors |
|---|---|---|---|---|
| `poll_terminal_requests(cache_state)` | Command/query | Poll every pending scene path once and publish terminal outcomes | Renderer main thread; stateless between calls | Null state returns zero; non-terminal retained |
| `is_scene_cache_current(scene)` | Command/query | Instantiate, validate and release one probe | Renderer main thread; call-local Node | Null/no validator/wrong root/stale false |
| `load_threaded_get_status(path)` | Boundary query | Read threaded status | Renderer main thread; injectable tests | ResourceLoader semantics |
| `load_threaded_get(path)` | Boundary query | Retrieve terminal Resource | Renderer main thread; injectable tests | Null/wrong type rejected |
## Inputs and outputs
| Direction | Contract/data | Producer | Consumer | Ownership | Thread/lifetime |
|---|---|---|---|---|---|
| Input | Detached normalized-path to `.tscn` path Dictionary | Scene cache state | Finalizer polling | Caller-owned copy | One pass |
| Input | Opaque load status/terminal Resource | ResourceLoader | Finalizer | Value/borrowed Resource | One path |
| Input | Cached WMO PackedScene | ResourceLoader | Probe validator | Borrowed reference | One completion |
| Input | Metadata validator | Loader composition | Finalizer | Borrowed Object | Finalizer lifetime |
| Output | Exact validated PackedScene | Finalizer | Scene cache state | Cache adopts strong reference | Until full clear |
| Output | Missing transition | Finalizer | Scene cache state | Path-only state | Until transient clear |
| Side effect | Temporary validation Node | PackedScene/finalizer | Validator | Finalizer-owned | Freed before return |
## Data flow
```mermaid
flowchart TD
Snapshot[Detached pending snapshot] --> Poll[Poll next status in insertion order]
Poll --> Terminal{Loaded or failed?}
Terminal -->|no| Retain[Retain pending]
Terminal -->|failed| Missing[Complete missing]
Terminal -->|loaded| Get[Get Resource]
Get --> Type{PackedScene?}
Type -->|no| Missing
Type -->|yes| Instantiate[Instantiate one probe]
Instantiate --> Root{Node3D root?}
Root -->|no| FreeRejected[Free rejected Node] --> Missing
Root -->|yes| Validate[WMOBuilder metadata validation]
Validate --> Free[Free Node3D probe]
Free --> Current{Current?}
Current -->|yes| Adopt[Adopt exact PackedScene]
Current -->|no| Missing
```
## Lifecycle/state
```mermaid
stateDiagram-v2
[*] --> Polling
Polling --> Pending: non-terminal
Polling --> Probing: loaded PackedScene
Polling --> Missing: failed/null/wrong type
Probing --> Missing: wrong root or stale metadata
Probing --> Cached: current metadata
Probing --> Released: probe freed before outcome
Released --> Missing
Released --> Cached
Pending --> Polling: later tick
```
The finalizer retains no request or Node. Cache state owns Pending/Cached/Missing
lifetime; every instantiated probe is synchronously released before return.
## Main sequence
```mermaid
sequenceDiagram
participant Loader as StreamingWorldLoader
participant Finalizer as WmoSceneResourceFinalizer
participant Cache as WmoSceneResourceCacheState
participant RL as ResourceLoader
participant Validator as WMOBuilder validator
Loader->>Finalizer: poll_terminal_requests(Cache)
Finalizer->>Cache: request_paths_snapshot()
loop insertion-ordered paths
Finalizer->>RL: load_threaded_get_status(path)
alt failed
Finalizer->>Cache: complete_request_as_missing(path)
else loaded
Finalizer->>RL: load_threaded_get(path)
Finalizer->>Finalizer: instantiate Node3D probe
Finalizer->>Validator: is_scene_cache_current(probe)
Finalizer->>Finalizer: free probe
Finalizer->>Cache: complete with exact scene or missing
else non-terminal
Finalizer->>Finalizer: retain request
end
end
```
## Dependency diagram
```mermaid
flowchart TB
Loader[StreamingWorldLoader] --> Finalizer[WmoSceneResourceFinalizer]
Loader --> Cache[WmoSceneResourceCacheState]
Finalizer --> Cache
Finalizer --> ResourceLoader
Finalizer --> PackedScene
Finalizer --> Node3D
Finalizer --> Validator[WMOBuilder cache validator]
Finalizer -. no dependency .-> FileAccess
Finalizer -. no dependency .-> Placement
Finalizer -. no dependency .-> Scheduler
```
## Ownership, threading and resources
- Renderer main thread serializes polling, instantiation and cache publication.
- Finalizer owns every call-local validation probe and frees it before return.
- Non-Node3D rejected roots are also freed, closing the prior leak edge.
- Cache state adopts accepted exact PackedScene references until full clear.
- Loader owns admission/size policy, live fallback, placed Nodes and shutdown order.
## Errors, cancellation and recovery
| Failure/state | Detection | Behavior | Diagnostic | Recovery |
|---|---|---|---|---|
| Null cache state | Guard | Return zero | Contract verifier | Correct composition |
| Non-terminal | Status | Retain pending | Pending count | Poll later |
| Failed/null/wrong Resource | Status/type | Complete missing | Synthetic fixture | Reset/rebuild cache |
| Non-Node3D root | Probe type | Free probe; complete missing | Type/lifetime fixture | Rebuild cache |
| Stale metadata | Injected validator | Free probe; complete missing | Validation fixture | Rebuild/reset |
| Shutdown | Loader lifecycle | Drain then clear cache | Shutdown verifier | New loader starts absent |
## Configuration and capabilities
The loader injects the existing WMOBuilder cache validator. `wmo_cache_dir`,
`wmo_max_runtime_scene_mb`, debug logging, request timing and WMO budgets remain
loader-owned. No setting, profile or cache version is added.
## Persistence, cache and migration
No data is serialized. Existing WMOBuilder metadata/version acceptance is called
unchanged before adoption. The rejected-root lifetime fix needs no migration or
rebake and does not alter accepted cache output.
## Diagnostics and observability
- Poll returns terminal completion count for tests/future metrics.
- Existing pending count and renderer `wmobuild` metrics remain unchanged.
- No logs are emitted; oversize debug logging remains at loader admission.
- Normalized WMO relative path remains the correlation key.
## Verification
- `verify_wmo_scene_resource_finalizer.gd`: null/non-terminal, insertion order,
failed/load boundary, null/wrong/non-Node3D/stale rejection, current exact
identity, accepted/rejected probe release, source ownership and timing.
- Scene cache, render finalizer/cache, WMO queue/planner/registry/resolver,
shutdown, facade, internal-access and baseline regressions cover neighbors.
- Fidelity evidence is exact orchestration extraction plus rejected-root leak
repair; no asset-backed visual or original-client parity claim is made.
## Extension points
- Legal current/stale serialized `.tscn` fixtures can extend evidence unchanged.
- Placed-scene preparation stays separate because it mutates an attached WMO
instance rather than a call-local validation probe.
## Capability status
| Capability | Status | Evidence | Gap/next step |
|---|---|---|---|
| Cached WMO scene terminal polling | Implemented extraction | Status/order/source/timing verifier | Asset-backed traversal pending |
| Probe validation/lifetime | Implemented extraction | Current/stale/type/free fixtures | Serialized fixture pending |
| Scene/missing publication | Implemented extraction | Exact identity/negative fixtures | Asset-backed lifetime/leak run pending |
| File-size admission/live fallback | Loader-owned | Existing scene-cache regressions | Separate extraction if justified |
## Known gaps and risks
- PackedScene instantiation/metadata validation remains synchronous main-thread work.
- No proprietary WMO corpus, serialized stale/oversize fixture, long leak run,
traversal p95/p99 or paired original-client capture is included.
## Source map
| Path | Responsibility |
|---|---|
| `src/render/wmo/wmo_scene_resource_finalizer.gd` | Terminal polling, probe validation/lifetime and publication |
| `src/render/wmo/wmo_scene_resource_cache_state.gd` | Scene/missing/request ownership and resets |
| `src/scenes/streaming/streaming_world_loader.gd` | Admission/size policy, live fallback and placed Node ownership |
| `addons/mpq_extractor/loaders/wmo_builder.gd` | Existing cache metadata/version validation |
| `src/tools/verify_wmo_scene_resource_finalizer.gd` | Terminal/probe/lifetime/source/timing regression |
## Related decisions and references
- [`wmo-scene-resource-cache-state.md`](wmo-scene-resource-cache-state.md)
- [`wmo-render-resource-finalizer.md`](wmo-render-resource-finalizer.md)
- [`world-renderer.md`](world-renderer.md)
- [`../../RENDER.md`](../../RENDER.md)
- [`../../targets/roadmap/02-rendering-and-graphics.md`](../../targets/roadmap/02-rendering-and-graphics.md)
docs/modules/world-renderer.md
+9 -4
View File
@@ -167,6 +167,7 @@ from externally reading/writing loader-private queue, task, cache and tile-state
| `WmoRenderResourceCacheState` | Internal WMO cache-state service | Owns validated Resources, negative entries and pending cache paths | Renderer main thread; map/cache session | Invalid/occupied request and unknown completion are rejected |
| `WmoRenderResourceFinalizer` | Internal WMO terminal-I/O service | Polls lightweight render requests, validates script/format and publishes Resource/missing outcomes | Renderer main thread; stateless across calls | Non-terminal retained; failed/null/wrong/stale complete missing |
| `WmoSceneResourceCacheState` | Internal WMO cache-state service | Owns validated PackedScenes, negative entries and pending `.tscn` paths | Renderer main thread; map/cache session | Direct missing and terminal request transitions remain distinct |
| `WmoSceneResourceFinalizer` | Internal WMO terminal-I/O service | Polls cached scene requests, validates/frees probes and publishes scene/missing outcomes | Renderer main thread; stateless across calls | Failed/wrong/stale scenes complete missing; probes always released |
| `AdtWaterLoadPipelineState` | Internal liquid async-state service | Owns ADT water FIFO/dedupe, active task IDs and mutex result mailbox | Main-thread state; worker result publication | Invalid/duplicate requests rejected; clear does not interrupt workers |
| `AdtWaterSceneFinalizer.attach_water_scene` | Internal liquid main-thread service | Builds and attaches one existing-format ADT Water subtree | Main thread; stateless, returned Node tile-owned | Empty/invalid/dry input returns null without attachment |
@@ -416,9 +417,10 @@ sequenceDiagram
and pending cache paths; `WmoRenderResourceFinalizer` owns its terminal
ResourceLoader polling and script/format validation. `WmoSceneResourceCacheState`
similarly owns validated PackedScenes, negative entries and pending `.tscn`
paths. The loader retains request admission, PackedScene/FileAccess I/O, size
and scene-cache validation, live fallback, materialization, permits, validity
reactions and every Node lifecycle action.
paths; `WmoSceneResourceFinalizer` owns terminal ResourceLoader I/O and
validation-probe lifetime. The loader retains request admission, FileAccess
size checks, live fallback, materialization, permits, validity reactions and
every placed-Node lifecycle action.
- `AdtWaterLoadPipelineState` owns pending request order/deduplication, opaque
active task IDs and the worker-safe parsed-result mailbox. The loader retains
WorkerThreadPool start/wait, ADTLoader parsing, concurrency/finalize permits,
@@ -618,7 +620,8 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
| WMO render build queue | Implemented extraction | Typed lifecycle/order/ownership/source/timing contract | Materialization and asset-backed traversal/leak evidence pending |
| WMO render Resource cache state | Implemented extraction | Scene-free lifecycle/exclusivity/source/timing plus shutdown contract | Asset-backed traversal/leak evidence pending |
| WMO render Resource finalizer | Implemented extraction | Status/order/script/format/adoption/source/timing contract | Serialized/asset-backed corrupt-cache and leak evidence pending |
| WMO scene Resource cache state | Implemented extraction | Scene-free lifecycle/direct-missing/source/timing plus shutdown contract | ResourceLoader/live-fallback extraction and asset-backed traversal/leak evidence pending |
| WMO scene Resource cache state | Implemented extraction | Scene-free lifecycle/direct-missing/source/timing plus shutdown contract | Asset-backed traversal/leak evidence pending |
| WMO scene Resource finalizer | Implemented extraction | Status/order/type/probe/lifetime/adoption/source/timing contract | Serialized stale/oversize and asset-backed evidence pending |
| WMO rendering | Partial | Cached group rendering | Portals/rooms/material parity |
| ADT water load pipeline state | Implemented extraction | Scene-free FIFO/task/thread/source/timing contract | Parse/finalization and asset-backed traversal/leak evidence pending |
| Liquids | Partial | MH2O/MLIQ paths | LiquidType/depth/shore fidelity |
@@ -694,6 +697,7 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
| `src/render/m2/m2_native_animation_resource_observer.gd` | Native candidate/read/build/cache observation |
| `src/render/m2/m2_prototype_cache_state.gd` | Detached prototype ownership and negative lookup outcomes |
| `src/render/wmo/wmo_render_resource_finalizer.gd` | Lightweight WMO terminal polling, validation and publication |
| `src/render/wmo/wmo_scene_resource_finalizer.gd` | Cached WMO terminal polling, probe validation/lifetime and publication |
| `src/render/streaming/streaming_target_planner.gd` | Scene-free wanted/retained ADT target calculation |
| `src/render/streaming/streaming_target_policy.gd` | Immutable renderer radius/prefetch policy |
| `src/render/streaming/streaming_target_plan.gd` | Immutable planner result with read-only tile-key sets |
@@ -729,6 +733,7 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
| `src/tools/verify_m2_native_animation_resource_observer.gd` | Native candidate/cache/raw/build/adoption/boundary/timing regression |
| `src/tools/verify_m2_prototype_cache_state.gd` | M2 prototype identity/negative/lifecycle/boundary/timing regression |
| `src/tools/verify_wmo_render_resource_finalizer.gd` | WMO render status/order/validation/adoption/boundary/timing regression |
| `src/tools/verify_wmo_scene_resource_finalizer.gd` | WMO scene status/order/probe/lifetime/adoption/boundary/timing regression |
| `src/tools/verify_streaming_target_planner.gd` | Planner behavior, dependency and bounded timing regression |
| `src/tools/verify_render_budget_scheduler.gd` | Scheduler bounds, shared-lane priority, cancellation and timing regression |
| `src/tools/verify_renderer_internal_access.gd` | Gameplay/EditorPlugin/registered renderer-tool boundary gate derived from private streamer fields |