sindoring/open-wc
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

rnd(M03): extract render budget scheduler

Browse Source
This commit is contained in:
sindoring
2026-07-16 00:31:01 +04:00
parent 91f0724ce2
commit e52f703da5
9 changed files with 594 additions and 74 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/modules/README.md
+1
View File
@@ -16,6 +16,7 @@
| Character presentation | Implemented boundary / Partial fidelity | [`character-presentation.md`](character-presentation.md) |
| Renderer | Partial | [`world-renderer.md`](world-renderer.md), [`../../RENDER.md`](../../RENDER.md) |
| Streaming target planner | Implemented | [`streaming-target-planner.md`](streaming-target-planner.md) |
| Render budget scheduler | Implemented | [`render-budget-scheduler.md`](render-budget-scheduler.md) |
| Network/session | Planned | [`../../targets/roadmap/03-network-and-session.md`](../../targets/roadmap/03-network-and-session.md) |
| Gameplay | Prototype/Planned | [`../../targets/roadmap/04-gameplay-systems.md`](../../targets/roadmap/04-gameplay-systems.md) |
| UI/Lua/audio | Prototype/Planned | [`../../targets/roadmap/05-ui-lua-audio.md`](../../targets/roadmap/05-ui-lua-audio.md) |
docs/modules/render-budget-scheduler.md
+216
View File
@@ -0,0 +1,216 @@
# Render Budget Scheduler
## Metadata
| Field | Value |
|---|---|
| Status | Implemented |
| Target/work package | `M03-RND-SCHEDULER-001` |
| Owners | Renderer workstream / M03 integrator |
| Last verified | Worktree `work/sindo-main-codex/m03-render-budget-scheduler`, 2026-07-16 |
| Profiles/capabilities | Runtime and Editor main-thread operation quotas; profile values supplied by streamer composition |
## Purpose
Issue a bounded number of permits for each renderer operation lane during one
process frame. The scheduler centralizes quota accounting and teardown
cancellation without taking ownership of queues or render work.
## Non-goals
- Own, sort, drain or cap the stored length of a renderer queue.
- Submit, cancel or await worker and `ResourceLoader` tasks.
- Execute callbacks or mutate the SceneTree, resources, caches or RIDs.
- Adapt limits from elapsed time or create a general job/dependency framework.
- Select renderer quality or compatibility profiles.
## Context and boundaries
```mermaid
flowchart LR
Config[Streamer exported per-frame limits] --> Frame[operation limit snapshot]
Frame --> Scheduler[RenderBudgetScheduler]
Queues[StreamingWorldLoader owned queues] --> Drains[Ordered queue drains]
Scheduler -->|boolean permits| Drains
Drains --> Render[Main-thread finalize/attach/evict]
Shutdown[StreamingWorldLoader exit] -->|cancel| Scheduler
Scheduler -. does not own .-> Queues
```
Allowed dependencies are Godot value containers and `RefCounted`. The module
must not depend on `Node`, `WorkerThreadPool`, `ResourceLoader`, `RenderingServer`,
mutexes, renderer assets or streamer queue fields.
## Public API
| Symbol | Kind | Purpose | Thread/lifetime | Failure behavior |
|---|---|---|---|---|
| Lane constants | `StringName` constants | Stable operation categories used in the frame limit map | Process lifetime | Unknown lanes have zero permits |
| `begin_frame` | Method | Replaces all limits and clears consumption diagnostics for one frame | Main thread; once before drains | Negative limits clamp to zero; cannot revive a cancelled instance |
| `has_remaining_permit` | Query | Tests a lane without consuming | Main thread; current frame | Unknown/exhausted/cancelled returns `false` |
| `try_consume_permit` | Mutation | Atomically checks and consumes one caller operation permit | Main thread; current frame | Unknown/exhausted/cancelled returns `false` |
| `remaining_permits` | Query | Returns a lane's current unconsumed count | Main thread; current frame | Unknown/cancelled returns zero |
| `consumed_permits_snapshot` | Query | Returns detached per-lane diagnostics | Caller owns returned dictionary | Mutation cannot affect scheduler state |
| `cancel` | Lifecycle method | Permanently stops new permits and clears remaining limits | Main thread; renderer teardown | Idempotent; does not interrupt in-flight caller work |
| `is_cancelled` | Query | Reports terminal lifecycle state | Scheduler lifetime | No failure |
## Inputs and outputs
| Direction | Contract/data | Producer | Consumer | Ownership | Thread/lifetime |
|---|---|---|---|---|---|
| Input | `Dictionary[StringName, int]` operation limits | `StreamingWorldLoader` configuration snapshot | Scheduler | Copied by scheduler | One process frame |
| Input | Lane ID | Ordered streamer drain | Scheduler | Constant value | One permit check |
| Input | Cancellation | Streamer `_exit_tree` | Scheduler | Terminal flag | Renderer instance lifetime |
| Output | Permit boolean | Scheduler | Streamer drain loop | Scalar | One attempted operation |
| Output | Remaining count | Scheduler | Streamer/tests | Scalar | Current frame |
| Output | Consumed snapshot | Scheduler | Diagnostics/tests | Detached caller-owned copy | Current frame snapshot |
The scheduler has no I/O, logging, SceneTree, task, cache or GPU side effects.
## Data flow
```mermaid
flowchart TD
Exports[Exported operation limits] --> Limits[_render_operation_limits_for_frame]
Limits --> Begin[begin_frame]
Begin --> Remaining[remaining permits by lane]
Queue[Non-empty caller-owned queue] --> Check[try_consume_permit]
Remaining --> Check
Check -->|true| Execute[Caller performs one operation]
Check -->|false| Defer[Operation remains queued]
Execute --> Consumed[consumed permits diagnostics]
```
## Lifecycle and state
```mermaid
stateDiagram-v2
[*] --> Ready
Ready --> FrameActive: begin_frame
FrameActive --> FrameActive: consume/check
FrameActive --> FrameActive: begin_frame resets limits
Ready --> Cancelled: cancel
FrameActive --> Cancelled: cancel
Cancelled --> Cancelled: begin_frame/check/cancel
Cancelled --> [*]
```
Cancellation is intentionally terminal. A new renderer scene creates a new
scheduler instance; an old instance cannot accidentally resume after teardown.
## Main sequence
```mermaid
sequenceDiagram
participant Stream as StreamingWorldLoader
participant Budget as RenderBudgetScheduler
participant Queue as Loader-owned queues
participant Render as SceneTree/RenderingServer
Stream->>Budget: begin_frame(operation limits)
loop existing fixed drain order
Stream->>Queue: inspect next operation
Stream->>Budget: try_consume_permit(lane)
Budget-->>Stream: true or false
Stream->>Render: execute one operation when true
end
Stream->>Budget: cancel() during _exit_tree
Budget-->>Stream: later permits rejected
Stream->>Stream: await tasks and release owned resources
```
## Ownership, threading and resources
- `StreamingWorldLoader` owns every queue, task registry, cache, Node, Resource and RID.
- The scheduler owns two small per-frame counter dictionaries and one terminal flag.
- The streamer preserves operation order. In particular, chunk removals and
creations share `CHUNK_GEOMETRY`, with removals drained first.
- All current scheduler calls occur on the Godot main thread. The service has no
mutex and is not safe for concurrent mutation.
- A scheduler instance lives exactly as long as its loader instance.
## Errors, cancellation and recovery
| Condition | Detection | Behavior | Diagnostic | Recovery |
|---|---|---|---|---|
| Negative/zero limit | `begin_frame` clamp | Lane issues no permits | Contract verifier | Correct configuration or accept disabled lane |
| Unknown lane | Permit lookup miss | Returns `false`/zero | Contract verifier | Add the lane to the frame snapshot |
| Exhausted lane | Remaining count is zero | Work remains in caller queue for a later frame | Queue-depth/hitch metrics | Next `begin_frame` replenishes permits |
| Renderer teardown | Streamer calls `cancel` before task waits | New permits stop immediately | Shutdown ownership regression | New scene creates a new scheduler |
| In-flight operation at cancel | Caller already consumed permit | Scheduler does not interrupt it | Ownership docs | Caller completes/cleans it through existing shutdown path |
This cancellation scope is permit issuance, not worker cancellation. Existing
task/result staleness and shutdown waits remain streamer responsibilities.
## Configuration and capabilities
The streamer's existing exported values populate 16 lanes: tile finalization,
terrain upgrade/control-splat/splat-cache/splat-build, water finalization, shared
chunk geometry, tile-load starts, tile LOD create/remove, M2 animation/mesh
finalization and build, WMO instance/group build, and detail synchronization.
Defaults and quality-preset assignments are unchanged. `tile_finalize` retains
its historical minimum of one; other negative values behave as zero.
## Persistence, cache and migration
The module is runtime-only and serializes nothing. It changes no cache, asset,
scene, protocol or database format and requires no version bump or migration.
## Diagnostics and observability
- `consumed_permits_snapshot` exposes detached per-frame counts for tests and
future facade metrics without exposing mutable scheduler state.
- Existing queue depths, named hitch sections and `HITCH`/`PERF` logs remain in
`StreamingWorldLoader`.
- No scheduler logging occurs on the hot permit path.
## Verification
- `src/tools/verify_render_budget_scheduler.gd` covers exact exhaustion,
independent lanes, shared removal/create priority, frame reset, invalid/unknown
lanes, detached diagnostics, terminal cancellation and source boundaries.
- The verifier performs 20,000 permit checks with a generous 150 ms regression
ceiling on the headless runner; this is a guard, not a renderer p95 claim.
- Integration regressions must retain the seven renderer checkpoint plans and
existing focus/planner/facade contracts.
- Fidelity evidence is behavior-preserving: exported values, process order,
cache versions and visual operations are unchanged. No build-12340 parity
claim follows from this extraction.
## Extension points
- Add facade consumption metrics using detached snapshots.
- Move a service's queue drain behind its own boundary while retaining these lane IDs.
- Add measured time-aware policy only after asset-backed p95/p99 evidence proves
fixed operation counts insufficient; do not embed callbacks in this scheduler.
## Capability status
| Capability | Status | Evidence | Gap/next step |
|---|---|---|---|
| Per-lane frame bounds | Implemented | Exact exhaustion and independent-lane contracts | Asset-backed p95/p99 baseline remains |
| Shared removal/create priority | Implemented | Synthetic shared `CHUNK_GEOMETRY` contract | Other service priorities remain streamer-owned |
| Teardown permit cancellation | Implemented | Terminal cancellation contract and loader source gate | Worker stale-result cancellation remains streamer-owned |
| Detached consumption diagnostics | Implemented | Snapshot isolation contract | Not yet exposed through facade metrics |
## Known gaps and risks
- Queue storage length is not capped; only per-frame draining is bounded.
- Worker concurrency and stale-result cancellation remain existing streamer logic.
- Exact p95/p99 impact requires local extracted assets and paired baseline runs.
- Lane IDs are dynamic dictionary keys; the verifier guards the current contract.
## Source map
| Path | Responsibility |
|---|---|
| `src/render/streaming/render_budget_scheduler.gd` | Scene-free lane counters, permits, diagnostics and cancellation |
| `src/scenes/streaming/streaming_world_loader.gd` | Limit composition, fixed drain order, queue/resource ownership and operation execution |
| `src/tools/verify_render_budget_scheduler.gd` | Asset-free behavior, dependency and bounded timing regression |
## Related decisions and references
- [`world-renderer.md`](world-renderer.md)
- [`streaming-target-planner.md`](streaming-target-planner.md)
- [`../../targets/03-renderer-facade.md`](../../targets/03-renderer-facade.md)
- [`../../targets/roadmap/02-rendering-and-graphics.md`](../../targets/roadmap/02-rendering-and-graphics.md)
- [`../CODING_STANDARD.md`](../CODING_STANDARD.md)
docs/modules/world-renderer.md
+19 -6
View File
@@ -5,9 +5,9 @@
| Field | Value |
|---|---|
| Status | Partial |
| Target/work package | M00 baseline; `M01-RND-STREAMING-FOCUS-001`; `M01-QAR-SERVER-SPAWN-RENDERER-001`; `M03-RND-FACADE-001` |
| Target/work package | M00 baseline; `M01-RND-STREAMING-FOCUS-001`; `M01-QAR-SERVER-SPAWN-RENDERER-001`; `M03-RND-FACADE-001`; `M03-RND-STREAMING-PLANNER-001`; `M03-RND-SCHEDULER-001` |
| Owners | Renderer workstream / milestone integrator |
| Last verified | Worktree `work/sindo-main-codex/m03-renderer-facade`, 2026-07-15 |
| Last verified | Worktree `work/sindo-main-codex/m03-render-budget-scheduler`, 2026-07-16 |
| Profiles/capabilities | `Performance`, `Balanced`, `High`, `Custom`; Blizzlike fidelity incomplete |
## Purpose
@@ -37,6 +37,7 @@ flowchart LR
Loader --> Planner[StreamingTargetPlanner]
Planner --> TargetPlan[StreamingTargetPlan]
TargetPlan --> Loader
Loader --> Budget[RenderBudgetScheduler]
Assets[Extracted WDT/ADT/M2/WMO/BLP] --> Native[Native loaders]
Cache[Baked terrain/M2/WMO caches] --> Loader
Native --> Parsed[Parsed tile/model data]
@@ -98,6 +99,7 @@ loader configuration remains transitional composition data, not a caller API.
| Input | Baked resources | Cache build tools | Streamer/builders | Shared immutable resource/cache entry | Session/cache lifetime |
| Test input | Server-spawn render manifest | Coordinate fixture/verifier | Checkpoint capture tool | Repository-owned diagnostic data | Test-process lifetime |
| Output | `StreamingTargetPlan` | `StreamingTargetPlanner` | Streamer queue/state apply | Immutable ephemeral value | One refresh |
| Internal control | Per-frame lane limits and boolean permits | Streamer configuration / `RenderBudgetScheduler` | Ordered streamer drains | Scheduler-owned counters | Main thread/frame |
| Output | Desired tile/detail operations | Streamer plan application | Finalize queues | Loader-owned | Cross-frame |
| Output | Terrain/M2/WMO/liquid instances | Loader/builders | Godot world/renderer | Loader/world owner | Main-thread attach |
| Output | Detached renderer metrics | `WorldRenderFacade` | Capture/baseline tools | Caller-owned deep copy | On demand |
@@ -131,7 +133,7 @@ flowchart TD
T --> Q[Tile load queue]
Q --> Parse[Worker parse/cache load]
Parse --> R[Result queues]
R --> B[Per-frame budget scheduler]
R --> B[RenderBudgetScheduler permits]
B --> Terrain[Terrain attach/upgrade]
B --> M2[M2 group/MultiMesh attach]
B --> WMO[WMO instance/group attach]
@@ -173,7 +175,7 @@ sequenceDiagram
participant Stream as StreamingWorldLoader internal
participant Planner as StreamingTargetPlanner
participant Worker as Worker task
participant Budget as Main-thread budget
participant Budget as RenderBudgetScheduler
participant Render as SceneTree/RenderingServer
Source->>Focus: sample explicit Godot position
Focus->>Facade: set/refresh typed focus
@@ -183,9 +185,12 @@ sequenceDiagram
Stream->>Stream: queue priority and loaded-tile LOD/detail state
Stream->>Worker: parse/load tile and detail data
Worker-->>Stream: immutable result
Stream->>Budget: enqueue finalize operations
Budget->>Render: attach bounded terrain/M2/WMO/liquid work
Stream->>Budget: begin_frame(operation limits)
Stream->>Budget: try_consume_permit(lane)
Budget-->>Stream: true while lane remains bounded
Stream->>Render: attach permitted terrain/M2/WMO/liquid work
Stream->>Render: evict outside retention range
Stream->>Budget: shutdown: cancel permit issuance
Stream->>Worker: shutdown: wait for WorkerThreadPool tasks
Stream->>Stream: shutdown: finish registered ResourceLoader requests
Stream->>Render: clear queues and owned tree nodes/RIDs
@@ -199,6 +204,8 @@ sequenceDiagram
streamer, focus source, returned metrics, queues, caches, nodes or RIDs.
- `StreamingTargetPlanner` is stateless and owns only call-local calculations;
its immutable plan is consumed synchronously by the streamer.
- `RenderBudgetScheduler` owns only per-frame lane counters and a terminal
cancellation flag. The streamer retains queue ordering and all operation side effects.
- Focus producers own the immutable `StreamingFocus` reference; the loader retains
the latest reference and converts it to `Vector3` only inside the render boundary.
- Worker tasks не должны менять SceneTree и shared Resource concurrently.
@@ -261,6 +268,9 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
- Unit/contract tests: streaming-focus contract/wiring, material mapping, unique-ID dedupe, placement probes, baseline manifest, five-point coordinate calibration, synthetic perceptual checkpoint diff, camera-pose grid plan.
- Pure planner contract: center/corner/catalog/clamp/editor cases plus bounded
High-like iteration timing without loading a world scene.
- Budget scheduler contract: exact lane exhaustion, shared chunk removal/create
priority, independent lanes, frame reset, invalid limits, terminal cancellation,
dependency boundaries and bounded permit timing without loading a world scene.
- Integration/E2E: Eastern Kingdoms/Kalimdor streaming scenes, seven M00
cold/warm checkpoints and one dedicated mapped server-spawn checkpoint.
- Fidelity evidence: семь локальных build 12340 reference JPG покрывают terrain/ADT/M2/WMO/liquid/sky и реальный `GryphonRoost01`; automated cold/warm metrics имеют `compared=14`, `missing=0`. Sky использует приблизительно paired camera, а animated-M2 остаётся asset-paired/placement-unpaired из-за synthetic probe.
@@ -288,6 +298,7 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
| Golden server-spawn visualization | Implemented | Pinned AzerothCore fixture, validated renderer manifest and diagnostic origin-marker capture at ADT `(32,48)`/MCNK `(3,12)` | No TrinityCore row or original-client visual-parity claim |
| Stable renderer facade | Partial | `M03-RND-FACADE-001` contract/runtime/tool regressions | Focus and metrics migrated; environment/entity visuals/ground query remain |
| Pure streaming target planner | Implemented | `M03-RND-STREAMING-PLANNER-001` asset-free contract/performance regression | Queue scheduling and loaded-tile LOD application remain in streamer |
| Bounded render budget scheduler | Implemented | `M03-RND-SCHEDULER-001` asset-free permit/cancellation/performance regression | Queue storage, worker concurrency and operation execution remain in streamer/services |
## Known gaps and risks
@@ -317,11 +328,13 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
| `src/render/streaming/streaming_target_planner.gd` | Scene-free wanted/retained ADT target calculation |
| `src/render/streaming/streaming_target_policy.gd` | Immutable renderer radius/prefetch policy |
| `src/render/streaming/streaming_target_plan.gd` | Immutable planner result with read-only tile-key sets |
| `src/render/streaming/render_budget_scheduler.gd` | Scene-free per-frame operation permits and terminal cancellation |
| `src/scenes/streaming/streaming_world_loader.gd` | Current runtime orchestration, queues, caches and attachment |
| `src/domain/streaming/streaming_focus.gd` | Immutable camera-independent streaming position contract |
| `src/tools/verify_streaming_focus.gd` | Headless contract, dependency and runtime/tool wiring regression |
| `src/tools/verify_world_render_facade.gd` | Facade delegation, snapshot isolation and consumer wiring regression |
| `src/tools/verify_streaming_target_planner.gd` | Planner behavior, dependency and bounded timing regression |
| `src/tools/verify_render_budget_scheduler.gd` | Scheduler bounds, shared-lane priority, cancellation and timing regression |
| `addons/mpq_extractor/loaders/adt_builder.gd` | Terrain/material/liquid construction |
| `addons/mpq_extractor/loaders/m2_builder.gd` | Static M2 mesh/material construction |
| `addons/mpq_extractor/loaders/m2_native_animated_builder.gd` | Native animated M2 experiment |