sindoring/open-wc
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

refactor(M03): extract WMO render build queue

Browse Source
This commit is contained in:
sindoring
2026-07-17 00:56:55 +04:00
parent 040b635ec6
commit e1bb105fff
12 changed files with 643 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/modules/README.md
+1
View File
@@ -22,6 +22,7 @@
| WMO placement resolver | Implemented extraction | [`wmo-placement-resolver.md`](wmo-placement-resolver.md) |
| WMO placement registry | Implemented extraction | [`wmo-placement-registry.md`](wmo-placement-registry.md) |
| WMO render build step planner | Implemented extraction | [`wmo-render-build-step-planner.md`](wmo-render-build-step-planner.md) |
| WMO render build queue | Implemented extraction | [`wmo-render-build-queue.md`](wmo-render-build-queue.md) |
| Third-person camera | Implemented | [`third-person-camera.md`](third-person-camera.md) |
| Character presentation | Implemented boundary / Partial fidelity | [`character-presentation.md`](character-presentation.md) |
| Renderer | Partial | [`world-renderer.md`](world-renderer.md), [`../../RENDER.md`](../../RENDER.md) |
docs/modules/wmo-render-build-queue.md
+238
View File
@@ -0,0 +1,238 @@
# WMO Render Build Queue
## Metadata
| Field | Value |
|---|---|
| Status | Implemented |
| Target/work package | M03 / `M03-RND-WMO-RENDER-BUILD-QUEUE-001` |
| Owners | Typed lightweight-WMO build jobs and FIFO placement-key queue |
| Last verified | Worktree `work/sindo-main-codex/m03-wmo-render-build-queue`, 2026-07-17 |
| Profiles/capabilities | Lightweight WMO render-cache path; profile-independent |
## Purpose
Own pending lightweight-WMO render build records and their FIFO placement-key
order outside the monolithic loader. Each typed job retains its placement key,
scene root, render resource and mesh/MultiMesh cursors while the loader performs
all validation, planning, materialization, permit consumption and destruction.
## Non-goals
- Free or queue-free Nodes, Resources or RIDs.
- Create MeshInstance3D/MultiMeshInstance3D or refresh materials.
- Own frame permits, placement references, caches or load requests.
- Change duplicate enqueue, stale-key cleanup, cancellation or ordering.
- Change cache versions, WMO transforms, shadows, visibility or fidelity rules.
## Context and boundaries
```mermaid
flowchart LR
Placement[Successful WMO placement] --> Loader[StreamingWorldLoader]
Root[Node3D root] --> Loader
Cache[WMOStreamingResource] --> Loader
Loader --> Queue[WmoRenderBuildQueue]
Queue --> Job[WmoRenderBuildJob]
Job --> Loader
Loader --> Planner[WmoRenderBuildStepPlanner]
Planner --> Step[Operation and next cursors]
Step --> Loader
Loader --> Scene[Main-thread group Node attach]
Loader --> Budget[RenderBudgetScheduler permit]
```
The queue may strongly reference Node3D and Resource through typed jobs, but it
must not destroy, mutate or inspect their render content. RenderingServer, RID,
Mesh/MultiMesh instances, files, worker threads, mutexes, gameplay, network and
editor UI are forbidden dependencies.
## Public API
### `WmoRenderBuildJob`
| Symbol | Kind | Purpose | Thread/lifetime | Errors |
|---|---|---|---|---|
| constructor `(placement_key, root, render_resource)` | Constructor | Create cursor-zero job with strong references | Renderer main thread; until cancel/clear/replacement | Queue validates inputs before construction |
| `placement_key()` | Query | Return immutable queue identity | Any serialized caller; job lifetime | None |
| `root()` | Query | Borrow retained Node3D reference | Main thread; no ownership transfer | May later be invalid; loader checks |
| `render_resource()` | Query | Borrow retained Resource reference | Main thread; no ownership transfer | Queue does not validate cache contents |
| `mesh_index()` / `multimesh_index()` | Query | Return current group cursors | Serialized job drain | None |
| `adopt_cursors(next_mesh_index, next_multimesh_index)` | Command | Atomically adopt planner cursors | Renderer main thread | Raw integers are accepted |
| `diagnostic_snapshot()` | Query | Return detached scalar state and presence flags | Caller-owned result | Never exposes engine objects |
### `WmoRenderBuildQueue`
| Symbol | Kind | Purpose | Thread/lifetime | Errors |
|---|---|---|---|---|
| `enqueue(placement_key, root, render_resource)` | Command | Replace keyed job and append FIFO key | Renderer main thread; map session | Empty/null input returns false |
| `has_pending()` / `pending_count()` | Query | Observe FIFO entries including duplicate/stale keys | Renderer main thread | None |
| `active_job_count()` | Query | Count current keyed jobs | Renderer main thread | None |
| `front_key()` / `pop_front()` | Queue operation | Inspect/remove one front FIFO key | Renderer main thread | Empty queue returns empty String |
| `job_for(placement_key)` | Query | Borrow current typed job or null for stale key | Renderer main thread | No ownership transfer |
| `cancel(placement_key)` | Command/query | Remove keyed job and first matching FIFO key | Renderer main thread | Unknown key returns false; stale duplicates remain |
| `clear()` | Command | Release all job/key references idempotently | Reset/shutdown | Does not free engine objects |
| `diagnostic_snapshot()` | Query | Return detached FIFO order and key-sorted scalar jobs | Caller-owned result | None |
## Inputs and outputs
| Direction | Contract/data | Producer | Consumer | Ownership | Thread/lifetime |
|---|---|---|---|---|---|
| Input | Resolved placement key | Placement/build adapter | Queue/job | Copied String | Until cancel/clear/replacement |
| Input | WMO root Node3D | Loader instance adapter | Job | Strong reference; loader owns engine lifetime | Until job release |
| Input | Validated WMOStreamingResource | Loader cache adapter | Job | Strong Resource reference; cache/loader owns semantics | Until job release |
| Input | Next mesh/MultiMesh cursors | Build step planner through loader | Job | Integer values | Until next adoption |
| Output | FIFO front key and typed job | Queue | Loader drain | Borrowed references | One drain iteration |
| Output | Detached scalar diagnostics | Job/queue | Verifier/future facade adapter | Fresh values | Caller lifetime |
Side effects are limited to reference retention/release, cursor mutation and
Dictionary/Array ordering. Scene tree, material, RID, file and permit effects
remain outside the module.
## Data flow
```mermaid
flowchart TD
Enqueue[enqueue key/root/resource] --> Valid{Inputs valid?}
Valid -->|no| Reject[false; unchanged]
Valid -->|yes| Replace[Replace keyed typed job]
Replace --> Append[Append key even when duplicate]
Drain[front key] --> Lookup{Current keyed job exists?}
Lookup -->|no stale| Pop[pop front and continue]
Lookup -->|yes| Borrow[Borrow job to loader]
Borrow --> Plan[Loader invokes step planner]
Plan --> Adopt[Job adopts both next cursors]
Cancel[cancel key] --> EraseJob[Erase keyed job]
EraseJob --> EraseFirst[Erase first matching FIFO key]
```
## Lifecycle/state
The queue can be `Empty`, `Pending` or `StaleFront`. Duplicate enqueue followed
by cancel can intentionally produce `StaleFront`, matching the previous separate
Dictionary/Array behavior. The loader pops a stale front without consuming a
permit. `clear` returns every state to `Empty` without freeing engine objects.
```mermaid
stateDiagram-v2
[*] --> Empty
Empty --> Pending: valid enqueue
Pending --> Pending: enqueue / cursor adoption
Pending --> Empty: final cancel or clear
Pending --> StaleFront: duplicate key then cancel first occurrence
StaleFront --> Pending: pop stale, more live entries remain
StaleFront --> Empty: pop stale or clear
```
## Main sequence
```mermaid
sequenceDiagram
participant Loader as StreamingWorldLoader
participant Queue as WmoRenderBuildQueue
participant Job as WmoRenderBuildJob
participant Planner as WmoRenderBuildStepPlanner
Loader->>Queue: enqueue(key, root, resource)
loop while permit and pending key
Loader->>Queue: front_key + job_for
alt stale key
Loader->>Queue: pop_front
else live job
Queue-->>Loader: borrowed Job
Loader->>Planner: plan_step(counts, job cursors)
Planner-->>Loader: operation and next cursors
Loader->>Loader: materialize one group on main thread
Loader->>Job: adopt_cursors(next mesh, next MultiMesh)
Loader->>Loader: consume one permit
end
end
Loader->>Queue: cancel(key) or clear()
Note over Queue: references released; Nodes/Resources not freed
```
## Ownership, threading and resources
- Queue owns its job Dictionary and FIFO key Array.
- Job owns strong references, but loader owns Node/Resource semantic lifetime.
- Loader checks `is_instance_valid`, creates/frees Nodes and controls caches.
- Loader invokes planner and adopts cursor output on the main thread.
- Queue/job contain no mutex; all mutation is serialized by the renderer drain.
- `clear` may release the last Resource reference but never calls engine free APIs.
## Errors, cancellation and recovery
| Failure/state | Detection | Behavior | Diagnostic | Recovery |
|---|---|---|---|---|
| Empty key/null root/resource | Enqueue guards | Reject without mutation | Contract verifier | Repair caller input |
| Duplicate key | Existing keyed job | Replace job and append key | Queue diagnostics | Historical stale handling remains |
| Stale FIFO key | `job_for` returns null | Loader pops front without permit | Contract/source verifier | Continue drain |
| Root invalidated externally | Loader validity check | Loader cancels keyed job | Shutdown/runtime diagnostics | Placement can rebuild later |
| Resource absent | Loader check | Loader cancels keyed job | Runtime diagnostics | Reload/requeue placement |
| Final placement release | Placement registry returns final | Loader cancels build then frees Node | Registry/shutdown regressions | Later placement starts fresh |
| Reset/shutdown | Loader lifecycle | Queue clear releases references | Shutdown verifier | Next map starts empty |
## Configuration and capabilities
The module introduces no settings or profile branches. WMO group permit count,
quality presets, cache limits, visibility and shadows remain loader-owned.
## Persistence, cache and migration
No data is serialized. WMO cache format/version/path and load request behavior
are unchanged. No migration or cache rebuild is required.
## Diagnostics and observability
`pending_count` preserves existing queue metrics, including stale/duplicate keys.
The detached snapshot exposes FIFO order, cursor values and root/resource presence
without exposing mutable jobs or engine objects. No runtime logs are emitted.
## Verification
- `verify_wmo_render_build_queue.gd`: invalid inputs, FIFO, typed references,
cursor adoption, duplicate replacement, stale pop, first-key cancel, clear,
engine lifetime, detached sorted diagnostics, source boundaries and timing.
- WMO planner/registry/resolver/material/shutdown and renderer regressions verify
unchanged ordering, permit use, destruction and surrounding lifecycle.
- Fidelity evidence is exact preservation of current queue/Dictionary semantics;
no original-client visual or asset-backed performance claim is made.
## Extension points
- Cache/load-request ownership can be extracted separately without changing jobs.
- A later typed materialization command may consume the borrowed job and planner
output while this queue remains the pending-state owner.
## Capability status
| Capability | Status | Evidence | Gap/next step |
|---|---|---|---|
| WMO render build pending-state ownership | Implemented | Lifecycle/source/timing verifier | Asset-backed traversal/leak evidence pending |
| WMO group planning | Implemented | Build-step planner verifier | Asset-backed p95/p99 pending |
| WMO materialization/cache loading | Partial | Existing runtime regressions | Separate loader seams remain |
## Known gaps and risks
- Jobs intentionally retain strong Node/Resource references until cancel/clear.
- Duplicate enqueue preserves a stale-key edge case rather than normalizing it.
- Resource array shape remains a WMO cache-builder invariant.
- No proprietary WMO corpus, traversal profile or original-client paired capture
is included in this package.
## Source map
| Path | Responsibility |
|---|---|
| `src/render/wmo/wmo_render_build_job.gd` | Typed references and cursors |
| `src/render/wmo/wmo_render_build_queue.gd` | Keyed jobs, FIFO order, cancel/clear and diagnostics |
| `src/render/wmo/wmo_render_build_step_planner.gd` | Pure operation/cursor planning |
| `src/scenes/streaming/streaming_world_loader.gd` | Validation, materialization, permits and engine destruction |
| `src/tools/verify_wmo_render_build_queue.gd` | Lifecycle, ownership, boundary and timing regression |
## Related decisions and references
- [`wmo-render-build-step-planner.md`](wmo-render-build-step-planner.md)
- [`wmo-placement-registry.md`](wmo-placement-registry.md)
- [`world-renderer.md`](world-renderer.md)
- [`../../RENDER.md`](../../RENDER.md)
- [`../../targets/roadmap/02-rendering-and-graphics.md`](../../targets/roadmap/02-rendering-and-graphics.md)
docs/modules/wmo-render-build-step-planner.md
+9 -5
View File
@@ -104,7 +104,8 @@ sequenceDiagram
## Ownership, threading and resources
- The planner owns only call-local scalar comparisons and the returned value map.
- The loader owns Resources, mesh arrays, build jobs/queue and cursor adoption.
- `WmoRenderBuildQueue` owns typed jobs, FIFO keys and strong Node3D/Resource
references. The loader owns mesh-array interpretation and engine lifetime.
- The loader creates and owns every MeshInstance3D/MultiMeshInstance3D and
performs material refresh and editor-owner assignment on the main thread.
- `RenderBudgetScheduler` owns permits; the planner neither reads nor consumes one.
@@ -149,8 +150,8 @@ existing loader/facade diagnostics.
## Extension points
- WMO render build job/queue ownership may be extracted separately after a typed
cancellation and Resource lifetime contract exists.
- `WmoRenderBuildQueue` now owns typed pending state; cache/load requests and
materialization remain separate extraction seams.
- Materialization can later receive typed commands without changing this planner.
## Capability status
@@ -158,7 +159,7 @@ existing loader/facade diagnostics.
| Capability | Status | Evidence | Gap/next step |
|---|---|---|---|
| WMO render build step planning | Implemented | Pure contract/source/timing verifier | Asset-backed traversal p95/p99 pending |
| WMO build job/resource ownership | Partial | Existing loader/runtime regressions | Separate state/cancellation package |
| WMO build job/queue ownership | Implemented | Typed lifecycle/source/timing verifier | Asset-backed traversal/leak evidence pending |
| WMO visual/portal/material fidelity | Partial | Renderer baseline/material checks | Original-client paired evidence pending |
## Known gaps and risks
@@ -173,12 +174,15 @@ existing loader/facade diagnostics.
| Path | Responsibility |
|---|---|
| `src/render/wmo/wmo_render_build_step_planner.gd` | Pure operation and cursor selection |
| `src/scenes/streaming/streaming_world_loader.gd` | Resource validation, jobs/queues, materialization, permits and cancellation |
| `src/render/wmo/wmo_render_build_job.gd` | Typed engine references and cursors |
| `src/render/wmo/wmo_render_build_queue.gd` | Pending jobs, FIFO keys and cancellation state |
| `src/scenes/streaming/streaming_world_loader.gd` | Resource validation, materialization, permits and engine lifetime |
| `src/tools/verify_wmo_render_build_step_planner.gd` | Contract, boundary and timing regression |
## Related decisions and references
- [`wmo-placement-registry.md`](wmo-placement-registry.md)
- [`wmo-render-build-queue.md`](wmo-render-build-queue.md)
- [`wmo-placement-resolver.md`](wmo-placement-resolver.md)
- [`world-renderer.md`](world-renderer.md)
- [`../../RENDER.md`](../../RENDER.md)
docs/modules/world-renderer.md
+15 -5
View File
@@ -7,7 +7,7 @@
| Status | Partial |
| Target/work package | M00 baseline; `M01-RND-STREAMING-FOCUS-001`; `M01-QAR-SERVER-SPAWN-RENDERER-001`; M03 facade/planner/scheduler/internal-access/ground/environment/entity packages; M03 terrain packages; M03 M2 packages; M03 WMO placement package |
| Owners | Renderer workstream / milestone integrator |
| Last verified | Worktree `work/sindo-main-codex/m03-wmo-render-build-planner`, 2026-07-17 |
| Last verified | Worktree `work/sindo-main-codex/m03-wmo-render-build-queue`, 2026-07-17 |
| Profiles/capabilities | `Performance`, `Balanced`, `High`, `Custom`; Blizzlike fidelity incomplete |
## Purpose
@@ -60,6 +60,8 @@ flowchart LR
WmoRegistry --> Loader
Loader --> WmoBuildStep[WmoRenderBuildStepPlanner]
WmoBuildStep --> Loader
Loader --> WmoBuildQueue[WmoRenderBuildQueue]
WmoBuildQueue --> Loader
Native --> Parsed[Parsed tile/model data]
Parsed --> Loader
Loader --> Scene[SceneTree nodes]
@@ -133,6 +135,7 @@ from externally reading/writing loader-private queue, task, cache and tile-state
| `WmoPlacementResolver.normalize_relative_path/resolve_unique_key/resolve_world_transform` | Internal pure WMO service | Resolves cache key, registry identity and world transform | Main/any thread; stateless | Missing UID uses tile/index fallback; transform fields use historical defaults |
| `WmoPlacementRegistry.add_reference/release_reference/contains/active_count/diagnostic_snapshot/clear` | Internal WMO service | Owns placement-key to tile/global reference sets | Renderer main thread; map session | Empty/unknown/non-owner input is rejected without mutation |
| `WmoRenderBuildStepPlanner.plan_step` | Internal pure WMO service | Selects one mesh-first lightweight render-group operation and next cursors | Main/any thread; stateless | Raw integer comparisons are preserved without clamping |
| `WmoRenderBuildQueue` / `WmoRenderBuildJob` | Internal WMO pending-state service | Owns typed root/resource/cursor jobs and FIFO placement keys | Renderer main thread; map session | Invalid enqueue rejected; duplicate/stale behavior preserved |
Публичным contract не считаются `StreamingWorldLoader` methods/properties,
внутренние dictionaries, queues, job records и generated node names. Scene-owned
@@ -164,6 +167,7 @@ loader configuration remains transitional composition data, not a caller API.
| Internal WMO placement | Path, MODF placement, tile/index | Loader / `WmoPlacementResolver` | WMO caches, registry and three instance adapters | Value-only String/Transform3D | Lookup/placement lifetime |
| Internal WMO ownership | Resolved placement key and tile/global reference key | Loader / `WmoPlacementRegistry` | Loader create/retain/final-free decisions | Registry-owned String sets; detached diagnostics | Map session or final release |
| Internal WMO build step | Mesh/MultiMesh counts and job cursors | Loader / `WmoRenderBuildStepPlanner` | Loader materialization/cursor adapter | Fresh scalar Dictionary | One group operation |
| Internal WMO pending build | Placement key, Node3D root, WMO Resource and cursors | Loader / `WmoRenderBuildQueue` | Loader drain and step planner adapter | Queue-owned job and strong references | Until cancel/clear/replacement |
| Output | Desired tile/detail operations | Streamer plan application | Finalize queues | Loader-owned | Cross-frame |
| Output | Terrain/M2/WMO/liquid instances | Loader/builders | Godot world/renderer | Loader/world owner | Main-thread attach |
| Output | Detached renderer metrics | `WorldRenderFacade` | Capture/baseline tools | Caller-owned deep copy | On demand |
@@ -214,7 +218,8 @@ flowchart TD
M2Batch --> M2
R --> WmoPlacement[WmoPlacementResolver]
WmoPlacement --> WmoRegistry[WmoPlacementRegistry]
WmoRegistry --> WmoBuildStep[WmoRenderBuildStepPlanner]
WmoRegistry --> WmoBuildQueue[WmoRenderBuildQueue]
WmoBuildQueue --> WmoBuildStep[WmoRenderBuildStepPlanner]
WmoBuildStep --> WMO
R --> B[RenderBudgetScheduler permits]
B --> Terrain[Terrain attach/upgrade]
@@ -340,8 +345,10 @@ sequenceDiagram
- `WmoPlacementResolver` is stateless and owns only call-local cache-key,
identity and transform values. `WmoPlacementRegistry` owns only placement-key
reference sets. `WmoRenderBuildStepPlanner` owns only a call-local operation
and cursor plan. The loader retains the key-to-Node map, caches, jobs, queues,
resources, cursor adoption, permits, cancellation and every Node lifecycle action.
and cursor plan. `WmoRenderBuildQueue` owns typed pending jobs, FIFO keys and
strong root/resource references without freeing engine objects. The loader
retains the placement key-to-Node map, caches/loads, materialization, permits,
validity reactions and every Node lifecycle action.
- Rendered-ground query results and diagnostic snapshots are caller-owned values;
the facade never returns Mesh, Node, tile-state or queue references.
- Loaded-mesh ground sampling is renderer diagnostics, not authoritative gameplay
@@ -439,6 +446,8 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
lifecycle, detached sorted diagnostics, source ownership and bounded timing.
- WMO render build step contract: mesh-before-MultiMesh order, one-index cursor
transition, completion/raw integer behavior, source ownership and bounded timing.
- WMO render build queue contract: typed references/cursors, FIFO, duplicate
replacement, stale-front cleanup, cancel/clear engine lifetime and bounded timing.
- Budget scheduler contract: exact lane exhaustion, shared chunk removal/create
priority, independent lanes, frame reset, invalid limits, terminal cancellation,
dependency boundaries and bounded permit timing without loading a world scene.
@@ -474,7 +483,8 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
| M2 build batch planner | Implemented extraction | Scene-free limit/count/cursor/source/timing contract | Queue/resource state, spatial cells and asset-backed p95/p99 remain pending |
| WMO placement resolver | Implemented extraction | Scene-free path/identity/transform/source/timing contract | Asset-backed comparison pending |
| WMO placement registry | Implemented extraction | Scene-free ownership/lifecycle/source/timing contract | Build/resource state and asset-backed cross-tile corpus pending |
| WMO render build step planner | Implemented extraction | Scene-free order/cursor/source/timing contract | Job/resource ownership and asset-backed traversal p95/p99 pending |
| WMO render build step planner | Implemented extraction | Scene-free order/cursor/source/timing contract | Asset-backed traversal p95/p99 pending |
| WMO render build queue | Implemented extraction | Typed lifecycle/order/ownership/source/timing contract | Cache/load state and asset-backed traversal/leak evidence pending |
| WMO rendering | Partial | Cached group rendering | Portals/rooms/material parity |
| Liquids | Partial | MH2O/MLIQ paths | LiquidType/depth/shore fidelity |
| Sky/light | Partial | DBC controller/logs | Zone transition paired validation |