315 lines
21 KiB
Markdown
315 lines
21 KiB
Markdown
# World Renderer
|
||
|
||
## Metadata
|
||
|
||
| Field | Value |
|
||
|---|---|
|
||
| Status | Partial |
|
||
| Target/work package | M00 baseline; `M01-RND-STREAMING-FOCUS-001`; `M01-QAR-SERVER-SPAWN-RENDERER-001`; декомпозиция M02–M03 |
|
||
| Owners | Renderer workstream / milestone integrator |
|
||
| Last verified | Worktree `work/sindo-main-codex/m01-server-spawn-renderer`, 2026-07-13 |
|
||
| Profiles/capabilities | `Performance`, `Balanced`, `High`, `Custom`; Blizzlike fidelity incomplete |
|
||
|
||
## Purpose
|
||
|
||
Загрузить и визуализировать мир WoW 3.3.5a вокруг streaming focus: terrain, ADT placements, M2, WMO, liquids, sky/light и character/world presentation experiments. Система должна скрывать I/O/parsing/finalization latency за prewarm и per-frame budgets.
|
||
|
||
## Non-goals
|
||
|
||
- Не владеет gameplay/world authoritative state.
|
||
- Не читает network packets или server DB.
|
||
- Не определяет player input/movement rules.
|
||
- Не является Content Project или editor source of truth.
|
||
- Текущий module не заявляет визуальное соответствие `1:1`; gaps ведутся в [`../../RENDER.md`](../../RENDER.md).
|
||
|
||
## Context and boundaries
|
||
|
||
```mermaid
|
||
flowchart LR
|
||
Player[Player or spectator Node3D] --> Adapter[Explicit source adapter]
|
||
Editor[Editor viewport adapter] --> Focus[StreamingFocus]
|
||
Capture[Capture tool] --> Adapter
|
||
SpawnFixture[Pinned server spawn fixture] --> SpawnManifest[Spawn render manifest]
|
||
SpawnManifest --> Capture
|
||
Adapter --> Focus
|
||
Focus --> Loader[StreamingWorldLoader]
|
||
Assets[Extracted WDT/ADT/M2/WMO/BLP] --> Native[Native loaders]
|
||
Cache[Baked terrain/M2/WMO caches] --> Loader
|
||
Native --> Parsed[Parsed tile/model data]
|
||
Parsed --> Loader
|
||
Loader --> Scene[SceneTree nodes]
|
||
Loader --> RS[RenderingServer RIDs/MultiMesh]
|
||
Sky[DBC sky/light data] --> SkyController[WowSkyController]
|
||
SkyController --> Scene
|
||
```
|
||
|
||
Allowed dependencies:
|
||
|
||
- native format loaders;
|
||
- render builders/material helpers;
|
||
- immutable/cached render resources;
|
||
- renderer configuration and focus;
|
||
- Godot rendering/scene APIs.
|
||
|
||
Forbidden dependencies:
|
||
|
||
- packet codecs/world session;
|
||
- gameplay reducers/quest/combat state;
|
||
- runtime UI Controls;
|
||
- direct TrinityCore/AzerothCore database access.
|
||
|
||
## Public API
|
||
|
||
Текущая система ещё не имеет стабильного facade. Фактический integration surface — `StreamingWorldLoader` Node3D и exported properties. M03 должен заменить этот surface на `WorldRenderFacade`.
|
||
|
||
| Symbol | Kind | Purpose | Thread/lifetime | Errors |
|
||
|---|---|---|---|---|
|
||
| `map_name` | Exported property | WDT/map directory name | Set before world load | Missing WDT produces diagnostic/empty world |
|
||
| `extracted_dir` | Exported property | Root of legally extracted client data | Session lifetime | Missing files reported per loader path |
|
||
| `StreamingFocus` | Immutable value | Typed Godot-basis position without Node/Camera dependency | Any thread; caller-owned reference | Null position is a caller contract violation |
|
||
| `streaming_focus_source_path` | Exported adapter property | Samples any explicit `Node3D` source into `StreamingFocus` | Main thread, scene lifetime | Missing/wrong node warns once and retains prior focus |
|
||
| `set_streaming_focus` | Public method | Accepts focus from app/editor/tool composition | Main thread/session | Null focus is ignored by refresh until replaced |
|
||
| `refresh_streaming_focus` | Public method | Samples configured source and applies refresh thresholds | Main thread/session | Returns `false` when no valid focus exists |
|
||
| `camera_path` | Exported property | Camera used only by optional automatic overview positioning | Main thread, scene lifetime | Missing camera skips automatic positioning |
|
||
| `quality_preset` | Exported property | Applies renderer budgets/radii | Before/at runtime depending property | Invalid combination should be diagnosed |
|
||
| `runtime_stats_enabled` | Exported property | Enables periodic performance snapshot | Runtime mutable | Logging overhead only |
|
||
| `hitch_profiler_enabled` | Exported property | Enables named hitch sections | Runtime mutable | Logging overhead only |
|
||
|
||
Публичным contract не считаются внутренние dictionaries, queues, job records и generated node names.
|
||
|
||
## Inputs and outputs
|
||
|
||
| Direction | Contract/data | Producer | Consumer | Ownership | Thread/lifetime |
|
||
|---|---|---|---|---|---|
|
||
| Input | `StreamingFocus` | Player/spectator/editor/capture adapter | `StreamingWorldLoader` | Immutable caller value | Main thread/session |
|
||
| Input | Focus-source `NodePath` | Runtime/test scene composition | Loader source adapter | Scene-owned node, sampled position | Main thread/scene lifetime |
|
||
| Input | Map/configuration | Scene/app/editor | `StreamingWorldLoader` | Caller config, copied/read | Main thread/session |
|
||
| Input | WDT/ADT/M2/WMO/BLP bytes | Extracted asset repository | Native loaders/builders | Loader result owns parsed data | Worker or controlled load |
|
||
| Input | Baked resources | Cache build tools | Streamer/builders | Shared immutable resource/cache entry | Session/cache lifetime |
|
||
| Test input | Server-spawn render manifest | Coordinate fixture/verifier | Checkpoint capture tool | Repository-owned diagnostic data | Test-process lifetime |
|
||
| Output | Desired tile/detail operations | Streaming planner inside loader | Finalize queues | Loader-owned | Cross-frame |
|
||
| Output | Terrain/M2/WMO/liquid instances | Loader/builders | Godot world/renderer | Loader/world owner | Main-thread attach |
|
||
| Output | Metrics/diagnostics | Hitch/stat instrumentation | Logs/baseline tools | Immutable records | Periodic/frame |
|
||
| Test output | Diagnostic spawn marker and cold/warm PNGs | Checkpoint capture tool | Human/integrator review | SceneTree marker; `user://` files | One capture process |
|
||
|
||
Side effects:
|
||
|
||
- reads extracted files and generated caches;
|
||
- creates/frees scene nodes, meshes, materials, MultiMeshes and RIDs;
|
||
- submits worker tasks;
|
||
- mutates shader globals/environment through render/sky paths;
|
||
- writes logs and baseline captures through tools.
|
||
|
||
## Data flow
|
||
|
||
```mermaid
|
||
flowchart TD
|
||
P[Player/spectator Node3D] --> A[Explicit scene adapter]
|
||
C[Capture camera] --> A
|
||
SF[Pinned server spawn] --> SM[Validated spawn manifest]
|
||
SM --> C
|
||
SM --> Marker[Renderer-native origin marker]
|
||
E[Editor viewport] --> EA[Editor adapter]
|
||
A --> F[StreamingFocus]
|
||
EA --> F
|
||
F --> T[Compute desired ADT tiles]
|
||
WDT[WDT tile catalog] --> T
|
||
T --> Q[Tile load queue]
|
||
Q --> P[Worker parse/cache load]
|
||
P --> R[Result queues]
|
||
R --> B[Per-frame budget scheduler]
|
||
B --> Terrain[Terrain attach/upgrade]
|
||
B --> M2[M2 group/MultiMesh attach]
|
||
B --> WMO[WMO instance/group attach]
|
||
B --> Liquid[Liquid attach]
|
||
Terrain --> World[Godot world]
|
||
M2 --> World
|
||
WMO --> World
|
||
Liquid --> World
|
||
Marker --> World
|
||
F --> E[Eviction/retention decisions]
|
||
E --> World
|
||
```
|
||
|
||
## Lifecycle/state
|
||
|
||
```mermaid
|
||
stateDiagram-v2
|
||
[*] --> Initializing
|
||
Initializing --> WaitingForFocus: WDT/catalog/config ready
|
||
Initializing --> Degraded: missing loaders/data/cache
|
||
WaitingForFocus --> Streaming: valid StreamingFocus
|
||
Streaming --> Streaming: load/finalize/evict ticks
|
||
Streaming --> WaitingForFocus: world reset without replacement focus
|
||
Streaming --> SwitchingMap: map/focus reset
|
||
SwitchingMap --> Initializing
|
||
WaitingForFocus --> ShuttingDown
|
||
Degraded --> ShuttingDown
|
||
Streaming --> ShuttingDown
|
||
ShuttingDown --> [*]
|
||
```
|
||
|
||
## Main sequence
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant Source as Player/Editor/Capture source
|
||
participant Focus as StreamingFocus adapter
|
||
participant Stream as StreamingWorldLoader
|
||
participant Worker as Worker task
|
||
participant Budget as Main-thread budget
|
||
participant Render as SceneTree/RenderingServer
|
||
Source->>Focus: sample explicit Godot position
|
||
Focus->>Stream: set/refresh typed focus
|
||
Stream->>Stream: desired tiles and priorities
|
||
Stream->>Worker: parse/load tile and detail data
|
||
Worker-->>Stream: immutable result
|
||
Stream->>Budget: enqueue finalize operations
|
||
Budget->>Render: attach bounded terrain/M2/WMO/liquid work
|
||
Stream->>Render: evict outside retention range
|
||
Stream->>Worker: shutdown: wait for WorkerThreadPool tasks
|
||
Stream->>Stream: shutdown: finish registered ResourceLoader requests
|
||
Stream->>Render: clear queues and owned tree nodes/RIDs
|
||
Stream->>Stream: free detached prototypes and release resource caches
|
||
```
|
||
|
||
## Ownership, threading and resources
|
||
|
||
- `StreamingWorldLoader` владеет tile states, queues, task registries, active world instances и render caches session scope.
|
||
- Focus producers own the immutable `StreamingFocus` reference; the loader retains
|
||
the latest reference and converts it to `Vector3` only inside the render boundary.
|
||
- Worker tasks не должны менять SceneTree и shared Resource concurrently.
|
||
- Parsed/grouped results передаются обратно через guarded result queues.
|
||
- Mesh/material/node/RID finalization выполняется main thread и ограничивается exported budgets.
|
||
- Shutdown/map switch обязан дождаться WorkerThreadPool jobs и зарегистрированных threaded ResourceLoader requests до очистки очередей, nodes, caches и RIDs.
|
||
- Parented streamed nodes остаются owned by SceneTree during shutdown; detached M2/WMO prototype caches are explicitly `free()`d after asynchronous work drains.
|
||
- A temporary liquid root with no generated child geometry is explicitly freed by `ADTBuilder` instead of being returned or abandoned.
|
||
- Cache resources считаются immutable после публикации.
|
||
|
||
## Errors, cancellation and recovery
|
||
|
||
| Failure | Detection | Behavior | Diagnostic | Recovery |
|
||
|---|---|---|---|---|
|
||
| Missing WDT/ADT | File/catalog check | Tile/map unavailable | Loader warning/error | Fix extraction/profile, reload |
|
||
| Missing focus source | Explicit NodePath resolution | Retain current streamed content and prior valid focus | One warning until source recovers | Fix composition path or call `set_streaming_focus` |
|
||
| No focus supplied | Refresh contract | No target recomputation; queues continue draining | `refresh_streaming_focus()` returns `false` | Supply player/editor/capture focus |
|
||
| Cache version mismatch | Required format constant | Reject stale cache/fallback | Version diagnostic | Rebuild matching cache |
|
||
| Worker failure | Task result/empty data | Skip/degrade affected asset | Queue/task warning | Retry/rebuild source |
|
||
| Main-thread hitch | Named section timing | Frame spike, work remains queued | `HITCH` log | Lower budget/fix finalize path |
|
||
| D3D12 descriptor exhaustion | Rendering backend error | Render failure/fallback backend | Godot error + baseline report | Dedup resources/fix settings |
|
||
| Teleport/map change | Focus/session transition | Old jobs become stale | Target/session generation | Cancel/drop stale results |
|
||
| Shutdown leak | Godot leak/RID diagnostics | Resource retained | Verbose shutdown report + cache shutdown verifier | Drain requests, preserve tree ownership, then free detached prototypes and clear resource caches |
|
||
|
||
## Configuration and capabilities
|
||
|
||
| Setting/capability | Default | Profile | Runtime mutable | Effect |
|
||
|---|---|---|---|---|
|
||
| Quality preset | Scene-specific | All | Limited | Applies radii and finalize budgets |
|
||
| Terrain control splat | Enabled in current High paths | Render | Limited | Near terrain texture quality |
|
||
| M2/WMO/liquids | Scene/profile-specific | Render | Debug/config | Enables asset categories |
|
||
| Runtime stats/hitch profiler | Usually debug | Dev/test | Yes | Observability cost |
|
||
| Shadow flags | Mostly disabled/limited | Blizzlike/Enhanced | Profile | CPU/GPU cost and visuals |
|
||
|
||
Exact exported settings and cache versions remain documented in [`../../RENDER.md`](../../RENDER.md) until extracted services receive individual specs.
|
||
|
||
## Persistence, cache and migration
|
||
|
||
- `StreamingFocus` is runtime-only and is not serialized. Scene migration adds
|
||
`streaming_focus_source_path`; existing `camera_path` remains valid only for
|
||
optional automatic overview positioning.
|
||
- Renderer reads caches under `data/cache`; caches are not committed.
|
||
- Format changes require `FORMAT_VERSION`/required-version bump and rebuild instructions.
|
||
- Atomic cache publication and unified manifest are target-state work from FND/M03.
|
||
- Material-only changes may refresh without geometry rebake only when documented safe.
|
||
|
||
## Diagnostics and observability
|
||
|
||
- Logs: `HITCH`, `PERF`, `TERRAIN_QUALITY`, `SKY_LIGHT`, `SKYBOX_MODEL`.
|
||
- Metrics: queue depths, active tiles/assets, named finalize times, frame percentiles and max hitch.
|
||
- Debug views: render checkpoint scenes/captures; future streaming/LOD/portal overlays.
|
||
- Correlation IDs: currently mostly tile/path keys; target architecture adds session/job IDs.
|
||
|
||
## Verification
|
||
|
||
- Unit/contract tests: streaming-focus contract/wiring, material mapping, unique-ID dedupe, placement probes, baseline manifest, five-point coordinate calibration, synthetic perceptual checkpoint diff, camera-pose grid plan.
|
||
- Integration/E2E: Eastern Kingdoms/Kalimdor streaming scenes, seven M00
|
||
cold/warm checkpoints and one dedicated mapped server-spawn checkpoint.
|
||
- Fidelity evidence: семь локальных build 12340 reference JPG покрывают terrain/ADT/M2/WMO/liquid/sky и реальный `GryphonRoost01`; automated cold/warm metrics имеют `compared=14`, `missing=0`. Sky использует приблизительно paired camera, а animated-M2 остаётся asset-paired/placement-unpaired из-за synthetic probe.
|
||
- Performance budgets: M00 report records cold/warm p95 and max hitch; no final acceptance threshold yet.
|
||
- Manual diagnostics: [`../RENDER_BASELINE.md`](../RENDER_BASELINE.md) and [`../../RENDER.md`](../../RENDER.md).
|
||
|
||
## Extension points
|
||
|
||
- Target `WorldRenderFacade` for runtime/editor/capture callers.
|
||
- Extend `StreamingFocus` with measured direction/velocity needs when the pure planner is extracted.
|
||
- Extracted planner, budget scheduler, terrain, M2, WMO, liquid, sky and character services.
|
||
- Separate Blizzlike/Enhanced material and quality profiles.
|
||
|
||
## Capability status
|
||
|
||
| Capability | Status | Evidence | Gap/next step |
|
||
|---|---|---|---|
|
||
| ADT streaming/terrain | Partial | M00 checkpoints and current scenes | Fidelity/performance gaps remain |
|
||
| Static M2 placement | Partial | MultiMesh/dedupe probes | Full materials/animation/effects |
|
||
| WMO rendering | Partial | Cached group rendering | Portals/rooms/material parity |
|
||
| Liquids | Partial | MH2O/MLIQ paths | LiquidType/depth/shore fidelity |
|
||
| Sky/light | Partial | DBC controller/logs | Zone transition paired validation |
|
||
| Character presentation | Prototype | Character creator/player experiment | Full customization/equipment states |
|
||
| Camera-independent streaming focus | Implemented | Typed contract, two player-focused runtime scenes and three explicitly camera-focused diagnostic tools | Multi-focus/direction/velocity belong to planner extraction |
|
||
| Golden server-spawn visualization | Implemented | Pinned AzerothCore fixture, validated renderer manifest and diagnostic origin-marker capture at ADT `(32,48)`/MCNK `(3,12)` | No TrinityCore row or original-client visual-parity claim |
|
||
| Stable renderer facade | Planned | M03 | Current caller uses monolithic Node API |
|
||
|
||
## Known gaps and risks
|
||
|
||
- Monolithic streamer mixes planning, jobs, caches and presentation.
|
||
- `camera_path` remains only for optional automatic overview positioning; it no
|
||
longer drives runtime streaming or discovers the active viewport camera.
|
||
- Original-client paired fidelity evidence incomplete.
|
||
- Первый paired run выявил coordinate/placement mismatch: несколько совпадающих server-derived camera positions оказываются под terrain или внутри WMO/rocks OpenWC.
|
||
- Terrain-height probe исключил under-terrain состояние для всех пяти точек; waterfall exact-XZ miss классифицирован как TriangleMesh seam/edge и подтверждён nearby sample в 2 units.
|
||
- Camera-occluder probe не нашёл camera containment в пяти точках; paired mismatch локализован прежде всего в manual look direction/target/FOV calibration, с явным ограничением по RID-only geometry.
|
||
- Empirical FOV sweep выявил, что checkpoint camera должна явно вызывать `make_current()`; после исправления projection ranking остаётся inconclusive из-за неизвестного manual yaw/pitch/framing reference.
|
||
- Camera-pose sweep can now rank bounded yaw/pitch grids without changing manifest defaults; perceptual ranking remains diagnostic and requires human framing approval.
|
||
- The first Goldshire pose grid was invalidated by WMO readiness: the inn is absent after 2 seconds but visible after the manifest-standard 8-second wait. Pose comparisons must stabilize asynchronous checkpoint content first.
|
||
- A ready-scene Goldshire 3x3 grid improved mean error from `0.101402` at zero offsets to `0.087952` at yaw `10`/pitch `-10`; pitch/FOV refinement remains required before manifest calibration.
|
||
- Joint Goldshire pitch/FOV refinement had no human-approved optimum: full-frame error preferred grass-heavy FOV 62/pitch -25, while FOV 38/pitch -10 better matched building scale. Landmark/region scoring is required before calibration.
|
||
- Common-region scoring `[300,100,1400,650]` moved Goldshire's numeric optimum to FOV 38/yaw 10/pitch -15, but human inspection still found an oversized/cropped facade; semantic landmarks or masks remain required.
|
||
- Final paired audit compared fourteen cold/warm images for seven private build 12340 references with zero missing pairs; mean error ranges `0.066893..0.176955`. Animated-M2 native motion is observed across two original-client phases, but its real Lakeshire placement differs from the synthetic Goldshire probe. Sky-transition evidence uses the checkpoint map position/time with manually approved approximate framing.
|
||
- D3D12 descriptor issues remain. Explicit shutdown ownership removes the GUI capture's Node/resource/RID leaks in focused and seven-checkpoint runs; a timing-dependent ObjectDB warning for anonymous zero-reference `RefCounted` objects remains as a separate engine-teardown diagnostic.
|
||
- M2/WMO/material/particle/ribbon/portal parity incomplete.
|
||
- Public API is mostly exported configuration rather than stable contracts.
|
||
|
||
## Source map
|
||
|
||
| Path | Responsibility |
|
||
|---|---|
|
||
| `src/scenes/streaming/streaming_world_loader.gd` | Current runtime orchestration, queues, caches and attachment |
|
||
| `src/domain/streaming/streaming_focus.gd` | Immutable camera-independent streaming position contract |
|
||
| `src/tools/verify_streaming_focus.gd` | Headless contract, dependency and runtime/tool wiring regression |
|
||
| `addons/mpq_extractor/loaders/adt_builder.gd` | Terrain/material/liquid construction |
|
||
| `addons/mpq_extractor/loaders/m2_builder.gd` | Static M2 mesh/material construction |
|
||
| `addons/mpq_extractor/loaders/m2_native_animated_builder.gd` | Native animated M2 experiment |
|
||
| `addons/mpq_extractor/loaders/wmo_builder.gd` | WMO groups/doodads/liquids construction |
|
||
| `src/scenes/sky/wow_sky_controller.gd` | DBC-driven sky/light/environment |
|
||
| `src/native/src/*_loader.cpp` | Native binary parsing |
|
||
| `src/tools/build_*cache.gd`, `src/tools/bake_*cache.gd` | Offline cache generation |
|
||
| `tools/run_render_baseline.ps1` | Unified M00 baseline runner |
|
||
| `src/tools/compare_render_checkpoints.gd` | Offline JPG/PNG paired-image perceptual metrics and JSON pass/fail report |
|
||
| `src/tools/verify_render_runtime_cache_shutdown.gd` | Headless ownership regression for detached runtime prototypes, resource caches and empty liquid roots |
|
||
| `src/tools/capture_render_checkpoints.gd` | Deterministic no-roll checkpoint camera, performance and visual capture |
|
||
| `src/tools/server_spawn_render_manifest.json` | Dedicated mapped AzerothCore spawn checkpoint and diagnostic marker configuration |
|
||
| `src/tools/verify_server_spawn_renderer.gd` | Fixture-to-renderer checkpoint contract verification |
|
||
| `tools/sweep_render_checkpoint_camera_pose.ps1` | Offline yaw/pitch capture grid and paired-error ranking |
|
||
| `src/tools/verify_render_coordinate_calibration.gd` | Build 12340 camera-coordinate golden point round-trip diagnostic |
|
||
| `src/tools/probe_render_terrain_height.gd` | Offline active-mesh terrain height and camera-clearance report |
|
||
| `src/tools/probe_render_camera_occluders.gd` | Scene-tree placement containment and camera-to-target AABB intersection report |
|
||
|
||
## Related decisions and references
|
||
|
||
- [`../../RENDER.md`](../../RENDER.md)
|
||
- [`../RENDER_BASELINE.md`](../RENDER_BASELINE.md)
|
||
- [`../CAMERA_POSE_SWEEP.md`](../CAMERA_POSE_SWEEP.md)
|
||
- [`../M00_FINAL_PAIRED_EVIDENCE.md`](../M00_FINAL_PAIRED_EVIDENCE.md)
|
||
- [`../../targets/roadmap/02-rendering-and-graphics.md`](../../targets/roadmap/02-rendering-and-graphics.md)
|
||
- [`../GODOT_BEST_PRACTICES.md`](../GODOT_BEST_PRACTICES.md)
|