239 lines
12 KiB
Markdown
239 lines
12 KiB
Markdown
# WMO Render Build Queue
|
|
|
|
## Metadata
|
|
|
|
| Field | Value |
|
|
|---|---|
|
|
| Status | Implemented |
|
|
| Target/work package | M03 / `M03-RND-WMO-RENDER-BUILD-QUEUE-001` |
|
|
| Owners | Typed lightweight-WMO build jobs and FIFO placement-key queue |
|
|
| Last verified | Worktree `work/sindo-main-codex/m03-wmo-render-build-queue`, 2026-07-17 |
|
|
| Profiles/capabilities | Lightweight WMO render-cache path; profile-independent |
|
|
|
|
## Purpose
|
|
|
|
Own pending lightweight-WMO render build records and their FIFO placement-key
|
|
order outside the monolithic loader. Each typed job retains its placement key,
|
|
scene root, render resource and mesh/MultiMesh cursors while the loader performs
|
|
all validation, planning, materialization, permit consumption and destruction.
|
|
|
|
## Non-goals
|
|
|
|
- Free or queue-free Nodes, Resources or RIDs.
|
|
- Create MeshInstance3D/MultiMeshInstance3D or refresh materials.
|
|
- Own frame permits, placement references, caches or load requests.
|
|
- Change duplicate enqueue, stale-key cleanup, cancellation or ordering.
|
|
- Change cache versions, WMO transforms, shadows, visibility or fidelity rules.
|
|
|
|
## Context and boundaries
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
Placement[Successful WMO placement] --> Loader[StreamingWorldLoader]
|
|
Root[Node3D root] --> Loader
|
|
Cache[WMOStreamingResource] --> Loader
|
|
Loader --> Queue[WmoRenderBuildQueue]
|
|
Queue --> Job[WmoRenderBuildJob]
|
|
Job --> Loader
|
|
Loader --> Planner[WmoRenderBuildStepPlanner]
|
|
Planner --> Step[Operation and next cursors]
|
|
Step --> Loader
|
|
Loader --> Scene[Main-thread group Node attach]
|
|
Loader --> Budget[RenderBudgetScheduler permit]
|
|
```
|
|
|
|
The queue may strongly reference Node3D and Resource through typed jobs, but it
|
|
must not destroy, mutate or inspect their render content. RenderingServer, RID,
|
|
Mesh/MultiMesh instances, files, worker threads, mutexes, gameplay, network and
|
|
editor UI are forbidden dependencies.
|
|
|
|
## Public API
|
|
|
|
### `WmoRenderBuildJob`
|
|
|
|
| Symbol | Kind | Purpose | Thread/lifetime | Errors |
|
|
|---|---|---|---|---|
|
|
| constructor `(placement_key, root, render_resource)` | Constructor | Create cursor-zero job with strong references | Renderer main thread; until cancel/clear/replacement | Queue validates inputs before construction |
|
|
| `placement_key()` | Query | Return immutable queue identity | Any serialized caller; job lifetime | None |
|
|
| `root()` | Query | Borrow retained Node3D reference | Main thread; no ownership transfer | May later be invalid; loader checks |
|
|
| `render_resource()` | Query | Borrow retained Resource reference | Main thread; no ownership transfer | Queue does not validate cache contents |
|
|
| `mesh_index()` / `multimesh_index()` | Query | Return current group cursors | Serialized job drain | None |
|
|
| `adopt_cursors(next_mesh_index, next_multimesh_index)` | Command | Atomically adopt planner cursors | Renderer main thread | Raw integers are accepted |
|
|
| `diagnostic_snapshot()` | Query | Return detached scalar state and presence flags | Caller-owned result | Never exposes engine objects |
|
|
|
|
### `WmoRenderBuildQueue`
|
|
|
|
| Symbol | Kind | Purpose | Thread/lifetime | Errors |
|
|
|---|---|---|---|---|
|
|
| `enqueue(placement_key, root, render_resource)` | Command | Replace keyed job and append FIFO key | Renderer main thread; map session | Empty/null input returns false |
|
|
| `has_pending()` / `pending_count()` | Query | Observe FIFO entries including duplicate/stale keys | Renderer main thread | None |
|
|
| `active_job_count()` | Query | Count current keyed jobs | Renderer main thread | None |
|
|
| `front_key()` / `pop_front()` | Queue operation | Inspect/remove one front FIFO key | Renderer main thread | Empty queue returns empty String |
|
|
| `job_for(placement_key)` | Query | Borrow current typed job or null for stale key | Renderer main thread | No ownership transfer |
|
|
| `cancel(placement_key)` | Command/query | Remove keyed job and first matching FIFO key | Renderer main thread | Unknown key returns false; stale duplicates remain |
|
|
| `clear()` | Command | Release all job/key references idempotently | Reset/shutdown | Does not free engine objects |
|
|
| `diagnostic_snapshot()` | Query | Return detached FIFO order and key-sorted scalar jobs | Caller-owned result | None |
|
|
|
|
## Inputs and outputs
|
|
|
|
| Direction | Contract/data | Producer | Consumer | Ownership | Thread/lifetime |
|
|
|---|---|---|---|---|---|
|
|
| Input | Resolved placement key | Placement/build adapter | Queue/job | Copied String | Until cancel/clear/replacement |
|
|
| Input | WMO root Node3D | Loader instance adapter | Job | Strong reference; loader owns engine lifetime | Until job release |
|
|
| Input | Validated WMOStreamingResource | Loader cache adapter | Job | Strong Resource reference; cache/loader owns semantics | Until job release |
|
|
| Input | Next mesh/MultiMesh cursors | Build step planner through loader | Job | Integer values | Until next adoption |
|
|
| Output | FIFO front key and typed job | Queue | Loader drain | Borrowed references | One drain iteration |
|
|
| Output | Detached scalar diagnostics | Job/queue | Verifier/future facade adapter | Fresh values | Caller lifetime |
|
|
|
|
Side effects are limited to reference retention/release, cursor mutation and
|
|
Dictionary/Array ordering. Scene tree, material, RID, file and permit effects
|
|
remain outside the module.
|
|
|
|
## Data flow
|
|
|
|
```mermaid
|
|
flowchart TD
|
|
Enqueue[enqueue key/root/resource] --> Valid{Inputs valid?}
|
|
Valid -->|no| Reject[false; unchanged]
|
|
Valid -->|yes| Replace[Replace keyed typed job]
|
|
Replace --> Append[Append key even when duplicate]
|
|
Drain[front key] --> Lookup{Current keyed job exists?}
|
|
Lookup -->|no stale| Pop[pop front and continue]
|
|
Lookup -->|yes| Borrow[Borrow job to loader]
|
|
Borrow --> Plan[Loader invokes step planner]
|
|
Plan --> Adopt[Job adopts both next cursors]
|
|
Cancel[cancel key] --> EraseJob[Erase keyed job]
|
|
EraseJob --> EraseFirst[Erase first matching FIFO key]
|
|
```
|
|
|
|
## Lifecycle/state
|
|
|
|
The queue can be `Empty`, `Pending` or `StaleFront`. Duplicate enqueue followed
|
|
by cancel can intentionally produce `StaleFront`, matching the previous separate
|
|
Dictionary/Array behavior. The loader pops a stale front without consuming a
|
|
permit. `clear` returns every state to `Empty` without freeing engine objects.
|
|
|
|
```mermaid
|
|
stateDiagram-v2
|
|
[*] --> Empty
|
|
Empty --> Pending: valid enqueue
|
|
Pending --> Pending: enqueue / cursor adoption
|
|
Pending --> Empty: final cancel or clear
|
|
Pending --> StaleFront: duplicate key then cancel first occurrence
|
|
StaleFront --> Pending: pop stale, more live entries remain
|
|
StaleFront --> Empty: pop stale or clear
|
|
```
|
|
|
|
## Main sequence
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Loader as StreamingWorldLoader
|
|
participant Queue as WmoRenderBuildQueue
|
|
participant Job as WmoRenderBuildJob
|
|
participant Planner as WmoRenderBuildStepPlanner
|
|
Loader->>Queue: enqueue(key, root, resource)
|
|
loop while permit and pending key
|
|
Loader->>Queue: front_key + job_for
|
|
alt stale key
|
|
Loader->>Queue: pop_front
|
|
else live job
|
|
Queue-->>Loader: borrowed Job
|
|
Loader->>Planner: plan_step(counts, job cursors)
|
|
Planner-->>Loader: operation and next cursors
|
|
Loader->>Loader: materialize one group on main thread
|
|
Loader->>Job: adopt_cursors(next mesh, next MultiMesh)
|
|
Loader->>Loader: consume one permit
|
|
end
|
|
end
|
|
Loader->>Queue: cancel(key) or clear()
|
|
Note over Queue: references released; Nodes/Resources not freed
|
|
```
|
|
|
|
## Ownership, threading and resources
|
|
|
|
- Queue owns its job Dictionary and FIFO key Array.
|
|
- Job owns strong references, but loader owns Node/Resource semantic lifetime.
|
|
- Loader checks `is_instance_valid`, creates/frees Nodes and controls caches.
|
|
- Loader invokes planner and adopts cursor output on the main thread.
|
|
- Queue/job contain no mutex; all mutation is serialized by the renderer drain.
|
|
- `clear` may release the last Resource reference but never calls engine free APIs.
|
|
|
|
## Errors, cancellation and recovery
|
|
|
|
| Failure/state | Detection | Behavior | Diagnostic | Recovery |
|
|
|---|---|---|---|---|
|
|
| Empty key/null root/resource | Enqueue guards | Reject without mutation | Contract verifier | Repair caller input |
|
|
| Duplicate key | Existing keyed job | Replace job and append key | Queue diagnostics | Historical stale handling remains |
|
|
| Stale FIFO key | `job_for` returns null | Loader pops front without permit | Contract/source verifier | Continue drain |
|
|
| Root invalidated externally | Loader validity check | Loader cancels keyed job | Shutdown/runtime diagnostics | Placement can rebuild later |
|
|
| Resource absent | Loader check | Loader cancels keyed job | Runtime diagnostics | Reload/requeue placement |
|
|
| Final placement release | Placement registry returns final | Loader cancels build then frees Node | Registry/shutdown regressions | Later placement starts fresh |
|
|
| Reset/shutdown | Loader lifecycle | Queue clear releases references | Shutdown verifier | Next map starts empty |
|
|
|
|
## Configuration and capabilities
|
|
|
|
The module introduces no settings or profile branches. WMO group permit count,
|
|
quality presets, cache limits, visibility and shadows remain loader-owned.
|
|
|
|
## Persistence, cache and migration
|
|
|
|
No data is serialized. WMO cache format/version/path and load request behavior
|
|
are unchanged. No migration or cache rebuild is required.
|
|
|
|
## Diagnostics and observability
|
|
|
|
`pending_count` preserves existing queue metrics, including stale/duplicate keys.
|
|
The detached snapshot exposes FIFO order, cursor values and root/resource presence
|
|
without exposing mutable jobs or engine objects. No runtime logs are emitted.
|
|
|
|
## Verification
|
|
|
|
- `verify_wmo_render_build_queue.gd`: invalid inputs, FIFO, typed references,
|
|
cursor adoption, duplicate replacement, stale pop, first-key cancel, clear,
|
|
engine lifetime, detached sorted diagnostics, source boundaries and timing.
|
|
- WMO planner/registry/resolver/material/shutdown and renderer regressions verify
|
|
unchanged ordering, permit use, destruction and surrounding lifecycle.
|
|
- Fidelity evidence is exact preservation of current queue/Dictionary semantics;
|
|
no original-client visual or asset-backed performance claim is made.
|
|
|
|
## Extension points
|
|
|
|
- Cache/load-request ownership can be extracted separately without changing jobs.
|
|
- A later typed materialization command may consume the borrowed job and planner
|
|
output while this queue remains the pending-state owner.
|
|
|
|
## Capability status
|
|
|
|
| Capability | Status | Evidence | Gap/next step |
|
|
|---|---|---|---|
|
|
| WMO render build pending-state ownership | Implemented | Lifecycle/source/timing verifier | Asset-backed traversal/leak evidence pending |
|
|
| WMO group planning | Implemented | Build-step planner verifier | Asset-backed p95/p99 pending |
|
|
| WMO materialization/cache loading | Partial | Existing runtime regressions | Separate loader seams remain |
|
|
|
|
## Known gaps and risks
|
|
|
|
- Jobs intentionally retain strong Node/Resource references until cancel/clear.
|
|
- Duplicate enqueue preserves a stale-key edge case rather than normalizing it.
|
|
- Resource array shape remains a WMO cache-builder invariant.
|
|
- No proprietary WMO corpus, traversal profile or original-client paired capture
|
|
is included in this package.
|
|
|
|
## Source map
|
|
|
|
| Path | Responsibility |
|
|
|---|---|
|
|
| `src/render/wmo/wmo_render_build_job.gd` | Typed references and cursors |
|
|
| `src/render/wmo/wmo_render_build_queue.gd` | Keyed jobs, FIFO order, cancel/clear and diagnostics |
|
|
| `src/render/wmo/wmo_render_build_step_planner.gd` | Pure operation/cursor planning |
|
|
| `src/scenes/streaming/streaming_world_loader.gd` | Validation, materialization, permits and engine destruction |
|
|
| `src/tools/verify_wmo_render_build_queue.gd` | Lifecycle, ownership, boundary and timing regression |
|
|
|
|
## Related decisions and references
|
|
|
|
- [`wmo-render-build-step-planner.md`](wmo-render-build-step-planner.md)
|
|
- [`wmo-placement-registry.md`](wmo-placement-registry.md)
|
|
- [`world-renderer.md`](world-renderer.md)
|
|
- [`../../RENDER.md`](../../RENDER.md)
|
|
- [`../../targets/roadmap/02-rendering-and-graphics.md`](../../targets/roadmap/02-rendering-and-graphics.md)
|