Compare commits

...

5 Commits

Author SHA1 Message Date
sindoring ef782d1f2b docs(M03): accept M2 mesh load pipeline state
Work-Package: M03-RND-M2-MESH-LOAD-PIPELINE-001
Agent: sindo-main-codex
Tests: post-merge pipeline, classifier, materials, shutdown, facade, internal-access, manifest, documentation and coordination gates pass
Fidelity: preserves request polling/completion order, statuses, metrics and shutdown drain; asset-backed visual parity remains unclaimed
2026-07-17 11:09:20 +04:00
sindoring 83b3618c50 merge M03 M2 mesh load pipeline state 2026-07-17 10:24:50 +04:00
sindoring ccbb27a9fd docs(M03): hand off M2 mesh load pipeline state
Work-Package: M03-RND-M2-MESH-LOAD-PIPELINE-001
Agent: sindo-main-codex
Tests: 34 headless regressions, documentation/coordination gates and checkpoint dry-run pass
Fidelity: lifecycle extraction only; asset-backed visual parity remains unclaimed
2026-07-17 10:24:41 +04:00
sindoring 82df7a555d refactor(M03): extract M2 mesh load pipeline state
Work-Package: M03-RND-M2-MESH-LOAD-PIPELINE-001
Agent: sindo-main-codex
Tests: 34 headless contract regressions pass; documentation and coordination gates pass; checkpoint dry-run 7/7
Fidelity: preserves request insertion polling order, terminal completion FIFO/statuses, metrics and clear/shutdown behavior; no visual parity claim
2026-07-17 10:23:52 +04:00
sindoring 5f6287905c coord(M03): claim M2 mesh load pipeline state 2026-07-17 10:15:17 +04:00
11 changed files with 671 additions and 30 deletions
+17
View File
@@ -36,6 +36,7 @@ Paired run 2026-07-11 подтвердил крупный coordinate/placement g
- `src/render/liquid/adt_water_load_pipeline_state.gd` - ADT water pending FIFO/dedupe, active task IDs and worker-safe parsed-result mailbox without parser or Node ownership.
- `src/render/liquid/adt_water_scene_finalizer.gd` - stateless main-thread ADT water build/attach and optional persisted Editor ownership through the existing ADTBuilder.
- `src/render/m2/m2_runtime_mesh_rebuild_classifier.gd` - memoized billboard/UV-rotation decision for stale cached M2 runtime mesh refresh.
- `src/render/m2/m2_mesh_load_pipeline_state.gd` - static M2 threaded-load request records, terminal statuses and completion-order finalize FIFO without I/O or Mesh ownership.
- `src/scenes/streaming/eastern_kingdoms_streaming.tscn` - текущая сцена для проверки Azeroth/Eastern Kingdoms.
- `addons/mpq_extractor/loaders/adt_builder.gd` - сборка ADT terrain, control splat материалов и MH2O liquids.
- `addons/mpq_extractor/loaders/m2_builder.gd` - сборка статического M2 mesh/material.
@@ -1056,6 +1057,22 @@ $exe = Join-Path $env:TEMP 'godot-4.6.1-openwc\Godot_v4.6.1-stable_win64.exe'
- No cache version, shader, material, geometry or visible rule changed. Synthetic
predicate timing is not asset-backed descriptor-pressure/p95/p99 evidence.
## 2026-07-17 M2 Mesh Load Pipeline State Extraction
- `M2MeshLoadPipelineState` now owns static M2 successful threaded-load request
records and the FIFO populated when ResourceLoader polling reaches loaded or
failed terminal status.
- Pending polling snapshots preserve Dictionary insertion order; terminal FIFO
order follows completion transition order. Existing `m2_mesh` metrics still
count pending plus finalize records at all three sites.
- Shutdown still drains every pending ResourceLoader path before state clear;
map reset and shutdown retain the same two clear sites.
- `StreamingWorldLoader` still selects `.tscn`/`.glb` paths, excludes animated
pivot-prefix GLBs, performs all ResourceLoader calls, consumes
`M2_MESH_FINALIZE`, extracts/prepares Meshes and updates Mesh/missing caches.
- Cache formats, material refresh, profiles and visible output are unchanged.
Synthetic state timing is not asset-backed I/O, leak or p95/p99 evidence.
## 2026-07-17 WMO Placement Resolver Extraction
- `WmoPlacementResolver` now owns lowercase/slash cache-key normalization,
@@ -0,0 +1,109 @@
# M03-RND-M2-MESH-LOAD-PIPELINE-001 — M2 mesh load pipeline state
<!-- OPENWC_CLAIM:M03-RND-M2-MESH-LOAD-PIPELINE-001:sindo-main-codex:2026-07-20 -->
## Ownership
- Target: M03
- Program: RND
- Owner/Agent ID: sindo-main-codex
- Branch: `work/sindo-main-codex/m03-m2-mesh-load-pipeline`
- Lease expires UTC: 2026-07-20
- Integrator: M03 milestone integrator
## Outcome
Extract static M2 threaded ResourceLoader request records and their terminal
finalize FIFO from `StreamingWorldLoader`, preserving request order, status
transition, metrics and reset/shutdown behavior.
## Non-goals
- Call ResourceLoader or choose `.tscn`/`.glb` cache paths.
- Own M2 Mesh, scene, missing or material-refresh caches.
- Extract Mesh from Resources or rebuild/adopt runtime Meshes.
- Consume `M2_MESH_FINALIZE` permits.
- Change cache formats, animation loading or visible M2 behavior.
## Paths
- Exclusive: `src/render/m2/m2_mesh_load_pipeline_state.gd`,
`src/tools/verify_m2_mesh_load_pipeline_state.gd`,
`docs/modules/m2-mesh-load-pipeline-state.md`, this claim
- Shared/hotspots: `src/scenes/streaming/streaming_world_loader.gd`,
`docs/modules/world-renderer.md`, `docs/modules/README.md`, `RENDER.md`,
`targets/03-renderer-facade.md`
- Generated/ignored: `.godot`, native DLL, generated M2 resources, caches and
proprietary renderer corpus
## Contracts and data
- One normalized path owns at most one pending request.
- Pending records preserve Dictionary insertion order for polling snapshots.
- Terminal completion removes the request and appends its record/status to FIFO.
- Finalize pop remains oldest-first; duplicate/unknown transitions are rejected.
- Total work count remains pending plus finalize records.
- Clear drops bookkeeping only; loader drains ResourceLoader requests first.
## Dependencies
- Requires: accepted classifier package on current master `8ec13d6`
- Blocks: further M2 Mesh cache/finalization service extraction
- External state: none; verification uses synthetic paths and opaque statuses
## Verification
- Commands: dedicated validation/order/transition/FIFO/clear/source/timing
verifier; M2 classifier/builder/grouper/transform/registry, materials, shutdown,
facade, internal-access, manifest, documentation and coordination gates
- Fixtures: synthetic normalized paths, resource paths and integer terminal statuses
- Fidelity evidence: exact request/finalize transitions and metric/reset sites preserved
- Performance budget: 100 cycles over 256 request/completion/pop records under 1 second
## Documentation deliverables
- Inline public API docs
- Module specification with inputs/outputs, ownership and source map
- Data-flow, lifecycle/state, sequence and dependency diagrams
- World-renderer, module registry and RENDER status updates
## Simplicity and naming
- Important names introduced: `M2MeshLoadPipelineState`
- Simplest considered solution: one RefCounted with request map and finalize FIFO
- Rejected complexity/abstractions: generic ResourceLoader queue, signals,
callbacks, Mesh cache ownership or animation/static pipeline merger
- Unavoidable complexity and justification: polling and budgeted finalization are
distinct lifecycle stages across frames
- Measured optimization evidence: bounded synthetic transition loop
## Status
- State: accepted
- Done: state holder, loader adapters, contract verifier and required documentation
- Next: further M2 Mesh cache/finalizer extraction
- Blocked by:
## Handoff
- Commit: `82df7a5`
- Results: 34 headless contract regressions pass; dedicated pipeline state
verifier passes 100-by-256 transitions in 100.273 ms; documentation and
coordination gates pass; checkpoint dry-run passes 7/7.
- Remaining risks: no private asset traversal, leak/descriptor-pressure or
original-client visual comparison; ResourceLoader I/O and Mesh cache/finalizer
ownership remain in `StreamingWorldLoader` by design.
- Documentation updated: inline API; `m2-mesh-load-pipeline-state.md` with
data-flow, state, sequence and dependency diagrams; module registry,
`world-renderer.md` and `RENDER.md`.
<!-- OPENWC_HANDOFF:READY:M03-RND-M2-MESH-LOAD-PIPELINE-001:82df7a5 -->
## Integration
- Merge commit: `83b3618`
- Post-merge results: pipeline state `cases=11 iterations=100
elapsed_ms=69.196`; classifier, materials, shutdown, facade, internal-access
`34`, manifest `7/7`, documentation and coordination gates passed.
<!-- OPENWC_INTEGRATION:ACCEPTED:M03-RND-M2-MESH-LOAD-PIPELINE-001:83b3618 -->
+1
View File
@@ -22,6 +22,7 @@
| M2 placement grouper | Implemented extraction | [`m2-placement-grouper.md`](m2-placement-grouper.md) |
| M2 build batch planner | Implemented extraction | [`m2-build-batch-planner.md`](m2-build-batch-planner.md) |
| M2 runtime mesh rebuild classifier | Implemented extraction | [`m2-runtime-mesh-rebuild-classifier.md`](m2-runtime-mesh-rebuild-classifier.md) |
| M2 mesh load pipeline state | Implemented extraction | [`m2-mesh-load-pipeline-state.md`](m2-mesh-load-pipeline-state.md) |
| WMO placement resolver | Implemented extraction | [`wmo-placement-resolver.md`](wmo-placement-resolver.md) |
| WMO placement registry | Implemented extraction | [`wmo-placement-registry.md`](wmo-placement-registry.md) |
| WMO render build step planner | Implemented extraction | [`wmo-render-build-step-planner.md`](wmo-render-build-step-planner.md) |
+229
View File
@@ -0,0 +1,229 @@
# M2 Mesh Load Pipeline State
## Metadata
| Field | Value |
|---|---|
| Status | Implemented extraction |
| Target/work package | M03 / `M03-RND-M2-MESH-LOAD-PIPELINE-001` |
| Owners | Static M2 threaded request records and terminal finalize FIFO |
| Last verified | Worktree `work/sindo-main-codex/m03-m2-mesh-load-pipeline`, 2026-07-17 |
| Profiles/capabilities | Existing static M2 cached `.tscn`/`.glb` load path |
## Purpose
Own cross-frame bookkeeping between successful static M2 ResourceLoader request
start, terminal loaded/failed polling and budgeted main-thread finalization. The
loader retains every I/O and engine-resource operation.
## Non-goals
- Call ResourceLoader or select cache paths/formats.
- Own M2 Mesh, scene, missing or material-refresh caches.
- Extract Meshes from PackedScene/GLB Resources.
- Consume finalize permits or rebuild/adopt runtime Meshes.
- Merge static and animated M2 pipelines.
## Context and boundaries
```mermaid
flowchart LR
Need[Static M2 mesh needed] --> Loader[StreamingWorldLoader]
Loader --> IO[ResourceLoader request]
Loader --> State[M2MeshLoadPipelineState]
State --> Poll[Detached pending records]
Poll --> Loader
Loader --> IO
Loader -->|terminal status| State
State --> FIFO[Finalize FIFO]
FIFO --> Loader
Loader --> Budget[M2_MESH_FINALIZE permit]
Loader --> Cache[Mesh or missing cache]
```
Allowed dependencies are value containers, Strings and opaque integer statuses.
ResourceLoader, filesystem, WorkerThreadPool, Mesh/Resource/Node/RID ownership,
scheduler and other application layers are forbidden.
## Public API
| Symbol | Kind | Purpose | Thread/lifetime | Errors |
|---|---|---|---|---|
| `remember_request(normalized_relative_path, resource_path)` | Command/query | Insert one pending request | Renderer main thread; until terminal/discard/clear | Empty/duplicate rejected |
| `has_request(path)` | Query | Test pending dedupe | Main thread | Empty/unknown false |
| `request_records_snapshot()` | Query | Copy pending records in insertion order | Main thread; caller-owned | None |
| `complete_request(path, terminal_status)` | Command/query | Move request to terminal FIFO | Main thread after poll | Unknown rejected |
| `discard_request(path)` | Command/query | Remove request without finalization | Main thread | Unknown false |
| `has_finalize_record()` / `pop_finalize_record()` | Query/command | Observe/pop completion-order FIFO | Main-thread budget drain | Empty pop returns `{}` |
| `total_work_count()` | Query | Preserve pending-plus-finalize metric | Main thread | None |
| `pending_request_count()` / `finalize_record_count()` | Query | Stage diagnostics | Main thread | None |
| `clear()` | Command | Drop bookkeeping after I/O drain/reset | Main thread | Idempotent; does not drain I/O |
| `diagnostic_snapshot()` | Query | Detached request/finalize records | Main thread | No Resources exposed |
## Inputs and outputs
| Direction | Contract/data | Producer | Consumer | Ownership | Thread/lifetime |
|---|---|---|---|---|---|
| Input | Normalized M2 path and cache Resource path | Loader request adapter | Pending map | Copied Strings | Until terminal/discard/clear |
| Input | Opaque terminal ResourceLoader status | Loader poll adapter | Finalize record | Integer value | Until pop/clear |
| Output | Detached pending records | Pipeline state | Loader poll/shutdown adapter | Caller-owned copies | One poll/wait pass |
| Output | Oldest terminal record | Pipeline state | Loader finalizer | Record ownership transferred | One finalize attempt |
| Output | Detached diagnostics | Pipeline state | Verifier/future metrics | Caller-owned copies | Snapshot lifetime |
Side effects are limited to collection mutation and retaining String/integer values.
## Data flow
```mermaid
flowchart TD
Start[Successful ResourceLoader request] --> Remember[remember request]
Remember --> Snapshot[request records snapshot]
Snapshot --> Poll[Loader polls status]
Poll --> Active{In progress?}
Active -->|yes| Snapshot
Active -->|no| Complete[complete request with terminal status]
Complete --> FIFO[Append finalize FIFO]
FIFO --> Permit{Loader permit available?}
Permit -->|no| FIFO
Permit -->|yes| Pop[Pop oldest terminal record]
Pop --> Finalize[Loader gets Resource and adopts Mesh/missing state]
```
## Lifecycle/state
```mermaid
stateDiagram-v2
[*] --> Absent
Absent --> Pending: remember request
Pending --> Pending: non-terminal poll
Pending --> TerminalQueued: complete request
Pending --> Absent: discard
TerminalQueued --> Absent: pop finalize
Pending --> Absent: clear
TerminalQueued --> Absent: clear
```
One path has at most one pending request. After completion it may be requested
again only if loader cache/missing rules permit it.
## Main sequence
```mermaid
sequenceDiagram
participant Loader as StreamingWorldLoader
participant Resource as ResourceLoader
participant State as M2MeshLoadPipelineState
participant Budget as RenderBudgetScheduler
Loader->>Resource: load_threaded_request(cache path)
Resource-->>Loader: OK or ERR_BUSY
Loader->>State: remember_request(normalized, cache path)
loop frames
Loader->>State: request_records_snapshot()
Loader->>Resource: load_threaded_get_status(path)
end
Loader->>State: complete_request(normalized, terminal status)
Loader->>Budget: try_consume_permit(M2_MESH_FINALIZE)
Loader->>State: pop_finalize_record()
Loader->>Resource: load_threaded_get(path)
Loader->>Loader: extract/refresh/adopt Mesh or mark missing
```
## Dependency diagram
```mermaid
flowchart TB
Loader[StreamingWorldLoader] --> State[M2MeshLoadPipelineState]
Loader --> Resource[ResourceLoader]
Loader --> Budget[RenderBudgetScheduler]
Loader --> MeshCache[M2 Mesh/missing caches]
Loader --> Classifier[M2RuntimeMeshRebuildClassifier]
State -. no dependency .-> Resource
State -. no dependency .-> Budget
State -. no dependency .-> MeshCache
```
## Ownership, threading and resources
- Main thread serializes all state mutation and snapshots.
- State owns only request/finalize Dictionaries containing Strings and status integers.
- Loader owns ResourceLoader request lifetime and drains active paths before shutdown clear.
- Loader owns Mesh/missing caches, material refresh, M2Builder and engine resources.
- Snapshot and diagnostic records are detached; caller mutation cannot affect state.
## Errors, cancellation and recovery
| Failure | Detection | Behavior | Diagnostic | Recovery |
|---|---|---|---|---|
| Empty/duplicate request | State guard | Reject unchanged | Contract verifier | Correct caller/request later |
| Request start failure | Loader return code | No state insert; mark missing | Existing loader behavior | Cache/source correction |
| Non-terminal status | Loader poll | Keep pending | Existing queue metric | Poll next frame |
| Terminal load failure | Status in popped record | Loader marks missing | Existing missing behavior | World/cache reload |
| Empty defensive path | Loader before poll | Discard and mark missing | Source regression | Correct request producer |
| Shutdown | Loader drains pending Resource paths | Clear state | Shutdown verifier | New loader starts empty |
## Configuration and capabilities
| Setting/capability | Default | Profile | Runtime mutable | Effect |
|---|---|---|---|---|
| `m2_mesh_finalize_ops_per_tick` | `1` | Quality/custom | Yes | Bounds terminal FIFO pops outside state |
| Cache extension order | `.tscn`, `.glb` | Existing M2 path | No | Loader selects request path |
| Animated pivot-prefix GLB exclusion | Existing rule | All | No | Loader filters before request |
## Persistence, cache and migration
No pipeline state is serialized. M2 cache formats, material refresh version and
rebuild policy are unchanged; no migration or rebake is required.
## Diagnostics and observability
- `total_work_count` preserves all three historical `m2_mesh` metrics.
- Diagnostic snapshots expose only paths/statuses, never loaded Resources or Meshes.
- Existing loader logs and hitch sections remain unchanged.
- Normalized M2 path is the correlation key.
## Verification
- `verify_m2_mesh_load_pipeline_state.gd`: validation, dedupe, insertion-order
polling, terminal transition, completion FIFO, opaque status, discard, metrics,
detached diagnostics, loader ownership and 100-by-256 timing.
- Classifier/material/shutdown/M2 placement/build regressions cover adjacent behavior.
- Fidelity evidence is exact state/lifecycle extraction; no asset-backed or
original-client visual parity claim is made.
- Performance budget: 25,600 request/complete/pop transitions under one second.
## Extension points
- Mesh/missing cache adoption can move to a separate finalizer without changing this state.
- Animated M2 has distinct candidate/static fallback semantics and remains separate.
## Capability status
| Capability | Status | Evidence | Gap/next step |
|---|---|---|---|
| Static M2 request/finalize state | Implemented extraction | Contract/source/timing verifier | Asset-backed long traversal pending |
| ResourceLoader I/O | Existing loader-owned | Shutdown/material regressions | I/O adapter extraction optional |
| Mesh cache/materialization | Existing loader-owned | Renderer/material tests | Separate state/finalizer extraction pending |
## Known gaps and risks
- Records remain Dictionaries because ResourceLoader polling is a dynamic Godot boundary.
- Clear does not drain ResourceLoader; caller ordering is mandatory.
- Status integers are intentionally opaque inside state.
- No private asset traversal, leak/descriptor-pressure or p95/p99 run is included.
## Source map
| Path | Responsibility |
|---|---|
| `src/render/m2/m2_mesh_load_pipeline_state.gd` | Pending records, terminal FIFO and metrics |
| `src/scenes/streaming/streaming_world_loader.gd` | Cache path choice, I/O polling, permits and Mesh/missing adoption |
| `src/render/m2/m2_runtime_mesh_rebuild_classifier.gd` | Downstream stale Mesh rebuild decision |
| `src/tools/verify_m2_mesh_load_pipeline_state.gd` | Lifecycle/boundary/timing regression |
## Related decisions and references
- [`m2-runtime-mesh-rebuild-classifier.md`](m2-runtime-mesh-rebuild-classifier.md)
- [`m2-build-batch-planner.md`](m2-build-batch-planner.md)
- [`world-renderer.md`](world-renderer.md)
- [`../../RENDER.md`](../../RENDER.md)
- [`../../targets/roadmap/02-rendering-and-graphics.md`](../../targets/roadmap/02-rendering-and-graphics.md)
+7 -1
View File
@@ -7,7 +7,7 @@
| Status | Partial |
| Target/work package | M00 baseline; `M01-RND-STREAMING-FOCUS-001`; `M01-QAR-SERVER-SPAWN-RENDERER-001`; M03 facade/planner/scheduler/internal-access/ground/environment/entity packages; M03 terrain packages; M03 M2 packages; M03 WMO placement package |
| Owners | Renderer workstream / milestone integrator |
| Last verified | Worktree `work/sindo-main-codex/m03-m2-runtime-rebuild-classifier`, 2026-07-17 |
| Last verified | Worktree `work/sindo-main-codex/m03-m2-mesh-load-pipeline`, 2026-07-17 |
| Profiles/capabilities | `Performance`, `Balanced`, `High`, `Custom`; Blizzlike fidelity incomplete |
## Purpose
@@ -133,6 +133,7 @@ from externally reading/writing loader-private queue, task, cache and tile-state
| `M2PlacementGrouper.group_placements` | Internal pure M2 service | Validates and groups ordered tile-local placement transforms by normalized path | Worker/main thread; stateless | Invalid variants/name IDs/empty paths are skipped |
| `M2BuildBatchPlanner.plan_batch` | Internal pure M2 service | Selects static/animated batch count and next group cursor | Main/any thread; stateless | Non-positive selected limit clamps to one; empty range completes |
| `M2RuntimeMeshRebuildClassifier` | Internal memoized M2 service | Detects billboard/UV-rotation metadata requiring stale cached-mesh rebuild | Renderer main thread; cached until reset | Invalid variants/indices skipped; first path decision wins |
| `M2MeshLoadPipelineState` | Internal M2 async-state service | Owns static Mesh pending request records and terminal finalize FIFO | Renderer main thread; map/session | Empty/duplicate/unknown transitions rejected |
| `WmoPlacementResolver.normalize_relative_path/resolve_unique_key/resolve_world_transform` | Internal pure WMO service | Resolves cache key, registry identity and world transform | Main/any thread; stateless | Missing UID uses tile/index fallback; transform fields use historical defaults |
| `WmoPlacementRegistry.add_reference/release_reference/contains/active_count/diagnostic_snapshot/clear` | Internal WMO service | Owns placement-key to tile/global reference sets | Renderer main thread; map session | Empty/unknown/non-owner input is rejected without mutation |
| `WmoRenderBuildStepPlanner.plan_step` | Internal pure WMO service | Selects one mesh-first lightweight render-group operation and next cursors | Main/any thread; stateless | Raw integer comparisons are preserved without clamping |
@@ -369,6 +370,9 @@ sequenceDiagram
- `M2RuntimeMeshRebuildClassifier` owns only normalized-path boolean decisions
for billboard/UV-rotation material refresh. The loader retains raw M2 loading,
refresh-version checks, Mesh rebuild/adoption and both historical clear sites.
- `M2MeshLoadPipelineState` owns static M2 pending Resource paths, opaque
terminal statuses and completion-order finalize FIFO. The loader retains cache
path selection, ResourceLoader calls, permits, Mesh/missing caches and adoption.
- Rendered-ground query results and diagnostic snapshots are caller-owned values;
the facade never returns Mesh, Node, tile-state or queue references.
- Loaded-mesh ground sampling is renderer diagnostics, not authoritative gameplay
@@ -570,6 +574,7 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
| `src/render/liquid/adt_water_load_pipeline_state.gd` | ADT water request/task/result bookkeeping and worker-safe mailbox |
| `src/render/liquid/adt_water_scene_finalizer.gd` | Main-thread ADT water build, tile attachment and optional editor ownership |
| `src/render/m2/m2_runtime_mesh_rebuild_classifier.gd` | Billboard/UV-rotation stale cached-mesh rebuild decision and memoization |
| `src/render/m2/m2_mesh_load_pipeline_state.gd` | Static M2 pending Resource paths, terminal statuses and finalize FIFO |
| `src/render/streaming/streaming_target_planner.gd` | Scene-free wanted/retained ADT target calculation |
| `src/render/streaming/streaming_target_policy.gd` | Immutable renderer radius/prefetch policy |
| `src/render/streaming/streaming_target_plan.gd` | Immutable planner result with read-only tile-key sets |
@@ -585,6 +590,7 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
| `src/tools/verify_adt_water_load_pipeline_state.gd` | ADT water FIFO/task/thread/boundary/timing regression |
| `src/tools/verify_adt_water_scene_finalizer.gd` | Synthetic ADT water scene/ownership/boundary/timing regression |
| `src/tools/verify_m2_runtime_mesh_rebuild_classifier.gd` | M2 rebuild predicate/cache/boundary/timing regression |
| `src/tools/verify_m2_mesh_load_pipeline_state.gd` | Static M2 request/terminal/FIFO/boundary/timing regression |
| `src/tools/verify_streaming_target_planner.gd` | Planner behavior, dependency and bounded timing regression |
| `src/tools/verify_render_budget_scheduler.gd` | Scheduler bounds, shared-lane priority, cancellation and timing regression |
| `src/tools/verify_renderer_internal_access.gd` | Gameplay/EditorPlugin/registered renderer-tool boundary gate derived from private streamer fields |
@@ -0,0 +1,103 @@
class_name M2MeshLoadPipelineState
extends RefCounted
## Owns static M2 threaded-load request records and the terminal finalize FIFO.
## The caller owns ResourceLoader I/O, permits, Mesh caches and materialization.
var _request_by_normalized_path: Dictionary = {}
var _finalize_records: Array[Dictionary] = []
## Records one normalized M2 path and ResourceLoader path. Empty or duplicate
## values are rejected without mutation.
func remember_request(normalized_relative_path: String, resource_path: String) -> bool:
if normalized_relative_path.is_empty() or resource_path.is_empty():
return false
if _request_by_normalized_path.has(normalized_relative_path):
return false
_request_by_normalized_path[normalized_relative_path] = {
"normalized": normalized_relative_path,
"path": resource_path,
}
return true
## Returns whether one normalized path currently has a pending threaded request.
func has_request(normalized_relative_path: String) -> bool:
return (
not normalized_relative_path.is_empty()
and _request_by_normalized_path.has(normalized_relative_path)
)
## Returns detached pending records in their existing Dictionary insertion order.
func request_records_snapshot() -> Array[Dictionary]:
var records: Array[Dictionary] = []
for request_variant in _request_by_normalized_path.values():
var request: Dictionary = request_variant
records.append(request.duplicate())
return records
## Removes one pending request and appends its terminal status to the finalize
## FIFO. Unknown paths are rejected without mutation.
func complete_request(normalized_relative_path: String, terminal_status: int) -> bool:
if not _request_by_normalized_path.has(normalized_relative_path):
return false
var request: Dictionary = _request_by_normalized_path[normalized_relative_path]
_request_by_normalized_path.erase(normalized_relative_path)
var finalize_record := request.duplicate()
finalize_record["status"] = terminal_status
_finalize_records.append(finalize_record)
return true
## Removes one pending request without enqueuing finalization. Returns whether a
## record was removed.
func discard_request(normalized_relative_path: String) -> bool:
return _request_by_normalized_path.erase(normalized_relative_path)
## Returns whether a terminal record is waiting for budgeted finalization.
func has_finalize_record() -> bool:
return not _finalize_records.is_empty()
## Removes and returns the oldest terminal record. Empty means none is ready.
func pop_finalize_record() -> Dictionary:
if _finalize_records.is_empty():
return {}
return _finalize_records.pop_front()
## Returns pending plus terminal-finalize work for existing renderer metrics.
func total_work_count() -> int:
return _request_by_normalized_path.size() + _finalize_records.size()
## Returns the pending request count.
func pending_request_count() -> int:
return _request_by_normalized_path.size()
## Returns the terminal finalize FIFO count.
func finalize_record_count() -> int:
return _finalize_records.size()
## Clears request/finalize bookkeeping. The caller must drain ResourceLoader
## requests before orderly shutdown clear.
func clear() -> void:
_request_by_normalized_path.clear()
_finalize_records.clear()
## Returns detached request and finalize records without loaded Resources/Meshes.
func diagnostic_snapshot() -> Dictionary:
var finalize_records: Array[Dictionary] = []
for record in _finalize_records:
finalize_records.append(record.duplicate())
return {
"requests": request_records_snapshot(),
"finalize_records": finalize_records,
}
@@ -0,0 +1 @@
uid://dadsxuswq0tnd
+18 -24
View File
@@ -63,6 +63,9 @@ const M2_BUILD_BATCH_PLANNER_SCRIPT := preload(
const M2_RUNTIME_MESH_REBUILD_CLASSIFIER_SCRIPT := preload(
"res://src/render/m2/m2_runtime_mesh_rebuild_classifier.gd"
)
const M2_MESH_LOAD_PIPELINE_STATE_SCRIPT := preload(
"res://src/render/m2/m2_mesh_load_pipeline_state.gd"
)
const STREAMING_TARGET_PLANNER_SCRIPT := preload("res://src/render/streaming/streaming_target_planner.gd")
const STREAMING_TARGET_POLICY_SCRIPT := preload("res://src/render/streaming/streaming_target_policy.gd")
const RENDER_BUDGET_SCHEDULER_SCRIPT := preload("res://src/render/streaming/render_budget_scheduler.gd")
@@ -254,8 +257,7 @@ var _m2_runtime_mesh_rebuild_classifier := (
M2_RUNTIME_MESH_REBUILD_CLASSIFIER_SCRIPT.new()
)
var _m2_mesh_cache: Dictionary = {}
var _m2_mesh_load_requests: Dictionary = {}
var _m2_mesh_finalize_queue: Array = []
var _m2_mesh_load_pipeline_state := M2_MESH_LOAD_PIPELINE_STATE_SCRIPT.new()
var _m2_animation_load_requests: Dictionary = {}
var _m2_animation_finalize_queue: Array = []
var _wmo_build_jobs: Dictionary = {}
@@ -1011,7 +1013,7 @@ func _log_hitch_profile(start_usec: int, timings: Array[String], did_refresh: bo
_detail_asset_queue.size(),
_m2_group_tasks.size(),
_m2_animation_finalize_queue.size() + _m2_animation_load_requests.size(),
_m2_mesh_finalize_queue.size() + _m2_mesh_load_requests.size(),
_m2_mesh_load_pipeline_state.total_work_count(),
_m2_build_queue.size(),
(
_wmo_build_queue.size()
@@ -1042,7 +1044,7 @@ func render_baseline_snapshot() -> Dictionary:
"detail": _detail_asset_queue.size(),
"m2_task": _m2_group_tasks.size(),
"m2_animation": _m2_animation_finalize_queue.size() + _m2_animation_load_requests.size(),
"m2_mesh": _m2_mesh_finalize_queue.size() + _m2_mesh_load_requests.size(),
"m2_mesh": _m2_mesh_load_pipeline_state.total_work_count(),
"m2_build": _m2_build_queue.size(),
"wmo_build": (
_wmo_build_queue.size()
@@ -1328,7 +1330,7 @@ func _tick_runtime_stats(delta: float) -> void:
_detail_asset_queue.size(),
_m2_group_tasks.size(),
_m2_animation_finalize_queue.size() + _m2_animation_load_requests.size(),
_m2_mesh_finalize_queue.size() + _m2_mesh_load_requests.size(),
_m2_mesh_load_pipeline_state.total_work_count(),
_m2_build_queue.size(),
(
_wmo_build_queue.size()
@@ -2338,15 +2340,14 @@ func _wait_for_tile_tasks() -> void:
_m2_group_result_queue.clear()
_m2_group_result_mutex.unlock()
for pending in _m2_mesh_load_requests.values():
for pending in _m2_mesh_load_pipeline_state.request_records_snapshot():
var path: String = String(pending.get("path", ""))
if path.is_empty():
continue
var status := ResourceLoader.load_threaded_get_status(path)
if status == ResourceLoader.THREAD_LOAD_IN_PROGRESS or status == ResourceLoader.THREAD_LOAD_LOADED:
ResourceLoader.load_threaded_get(path)
_m2_mesh_load_requests.clear()
_m2_mesh_finalize_queue.clear()
_m2_mesh_load_pipeline_state.clear()
_m2_runtime_mesh_rebuild_classifier.clear()
for pending in _m2_animation_load_requests.values():
@@ -2996,8 +2997,7 @@ func _clear_streamed_world() -> void:
_cancel_m2_build_job(String(key))
_m2_build_queue.clear()
_m2_unique_placement_registry.clear()
_m2_mesh_load_requests.clear()
_m2_mesh_finalize_queue.clear()
_m2_mesh_load_pipeline_state.clear()
_m2_runtime_mesh_rebuild_classifier.clear()
_m2_animation_load_requests.clear()
_m2_animation_finalize_queue.clear()
@@ -4247,24 +4247,21 @@ func _drain_m2_animation_loads() -> void:
func _drain_m2_mesh_loads() -> void:
for normalized_rel_variant in _m2_mesh_load_requests.keys():
var normalized_rel := String(normalized_rel_variant)
var pending: Dictionary = _m2_mesh_load_requests[normalized_rel]
for pending in _m2_mesh_load_pipeline_state.request_records_snapshot():
var normalized_rel := String(pending.get("normalized", ""))
var path: String = String(pending.get("path", ""))
if path.is_empty():
_m2_mesh_load_requests.erase(normalized_rel)
_m2_mesh_load_pipeline_state.discard_request(normalized_rel)
_m2_missing_cache[normalized_rel] = true
continue
var status := ResourceLoader.load_threaded_get_status(path)
if status != ResourceLoader.THREAD_LOAD_LOADED and status != ResourceLoader.THREAD_LOAD_FAILED:
continue
pending["status"] = status
_m2_mesh_load_requests.erase(normalized_rel)
_m2_mesh_finalize_queue.append(pending)
_m2_mesh_load_pipeline_state.complete_request(normalized_rel, status)
while not _m2_mesh_finalize_queue.is_empty() and _render_budget_scheduler.try_consume_permit(
while _m2_mesh_load_pipeline_state.has_finalize_record() and _render_budget_scheduler.try_consume_permit(
RENDER_BUDGET_SCHEDULER_SCRIPT.M2_MESH_FINALIZE):
var pending: Dictionary = _m2_mesh_finalize_queue.pop_front()
var pending: Dictionary = _m2_mesh_load_pipeline_state.pop_finalize_record()
var normalized_rel := String(pending.get("normalized", ""))
if normalized_rel.is_empty() or _m2_mesh_cache.has(normalized_rel):
continue
@@ -4620,7 +4617,7 @@ func _get_m2_mesh_or_request(rel_path: String) -> Mesh:
func _request_m2_mesh_load(normalized_rel: String) -> void:
if normalized_rel.is_empty() or _m2_mesh_load_requests.has(normalized_rel):
if normalized_rel.is_empty() or _m2_mesh_load_pipeline_state.has_request(normalized_rel):
return
for cache_res_path in _get_m2_cache_resource_paths(normalized_rel, [".tscn", ".glb"]):
@@ -4634,10 +4631,7 @@ func _request_m2_mesh_load(normalized_rel: String) -> void:
false,
ResourceLoader.CACHE_MODE_REUSE)
if err == OK or err == ERR_BUSY:
_m2_mesh_load_requests[normalized_rel] = {
"normalized": normalized_rel,
"path": cache_res_path,
}
_m2_mesh_load_pipeline_state.remember_request(normalized_rel, cache_res_path)
return
break
@@ -0,0 +1,166 @@
extends SceneTree
## Synthetic request/terminal/FIFO/lifecycle/boundary/timing regression for
## static M2 threaded mesh loading.
const PIPELINE_SCRIPT := preload("res://src/render/m2/m2_mesh_load_pipeline_state.gd")
const PIPELINE_PATH := "res://src/render/m2/m2_mesh_load_pipeline_state.gd"
const LOADER_PATH := "res://src/scenes/streaming/streaming_world_loader.gd"
func _initialize() -> void:
var failures: Array[String] = []
_verify_validation_dedupe_and_order(failures)
_verify_terminal_transition_and_fifo(failures)
_verify_discard_metrics_clear_and_diagnostics(failures)
_verify_ownership_boundaries(failures)
var elapsed_milliseconds := _verify_bounded_timing(failures)
if not failures.is_empty():
for failure in failures:
push_error("M2_MESH_LOAD_PIPELINE_STATE: %s" % failure)
quit(1)
return
print(
"M2_MESH_LOAD_PIPELINE_STATE PASS cases=11 iterations=100 elapsed_ms=%.3f"
% elapsed_milliseconds
)
quit(0)
func _verify_validation_dedupe_and_order(failures: Array[String]) -> void:
var pipeline: RefCounted = PIPELINE_SCRIPT.new()
_expect_false(pipeline.call("remember_request", "", "res://a.tscn"), "empty normalized path rejected", failures)
_expect_false(pipeline.call("remember_request", "world/a.m2", ""), "empty resource path rejected", failures)
_expect_true(pipeline.call("remember_request", "world/a.m2", "res://a.tscn"), "first request accepted", failures)
_expect_false(pipeline.call("remember_request", "world/a.m2", "res://duplicate.glb"), "duplicate request rejected", failures)
_expect_true(pipeline.call("remember_request", "world/b.m2", "res://b.glb"), "second request accepted", failures)
_expect_true(pipeline.call("has_request", "world/a.m2"), "request observable", failures)
var records: Array[Dictionary] = pipeline.call("request_records_snapshot")
_expect_equal(records.size(), 2, "request record count", failures)
_expect_string_equal(String(records[0]["normalized"]), "world/a.m2", "request insertion order first", failures)
_expect_string_equal(String(records[1]["normalized"]), "world/b.m2", "request insertion order second", failures)
func _verify_terminal_transition_and_fifo(failures: Array[String]) -> void:
var pipeline: RefCounted = PIPELINE_SCRIPT.new()
pipeline.call("remember_request", "world/a.m2", "res://a.tscn")
pipeline.call("remember_request", "world/b.m2", "res://b.glb")
_expect_false(pipeline.call("complete_request", "world/missing.m2", 3), "unknown completion rejected", failures)
_expect_true(pipeline.call("complete_request", "world/b.m2", 4), "second request completes first", failures)
_expect_true(pipeline.call("complete_request", "world/a.m2", 3), "first request completes second", failures)
_expect_false(pipeline.call("has_request", "world/a.m2"), "completed request removed", failures)
_expect_equal(int(pipeline.call("pending_request_count")), 0, "all requests completed", failures)
_expect_equal(int(pipeline.call("finalize_record_count")), 2, "two terminal records", failures)
var first: Dictionary = pipeline.call("pop_finalize_record")
var second: Dictionary = pipeline.call("pop_finalize_record")
_expect_string_equal(String(first["normalized"]), "world/b.m2", "completion FIFO first", failures)
_expect_equal(int(first["status"]), 4, "first opaque status retained", failures)
_expect_string_equal(String(second["normalized"]), "world/a.m2", "completion FIFO second", failures)
_expect_equal(int(second["status"]), 3, "second opaque status retained", failures)
_expect_false(pipeline.call("has_finalize_record"), "finalize FIFO drained", failures)
_expect_true((pipeline.call("pop_finalize_record") as Dictionary).is_empty(), "empty pop returns Dictionary", failures)
func _verify_discard_metrics_clear_and_diagnostics(failures: Array[String]) -> void:
var pipeline: RefCounted = PIPELINE_SCRIPT.new()
pipeline.call("remember_request", "world/a.m2", "res://a.tscn")
pipeline.call("remember_request", "world/b.m2", "res://b.glb")
_expect_true(pipeline.call("discard_request", "world/a.m2"), "pending request discarded", failures)
_expect_false(pipeline.call("discard_request", "world/a.m2"), "discard is idempotent", failures)
pipeline.call("complete_request", "world/b.m2", 3)
_expect_equal(int(pipeline.call("total_work_count")), 1, "total includes finalize only", failures)
var snapshot: Dictionary = pipeline.call("diagnostic_snapshot")
(snapshot["finalize_records"] as Array)[0]["path"] = "mutated"
var fresh_snapshot: Dictionary = pipeline.call("diagnostic_snapshot")
_expect_string_equal(
String((fresh_snapshot["finalize_records"] as Array)[0]["path"]),
"res://b.glb",
"diagnostics detached",
failures
)
pipeline.call("clear")
pipeline.call("clear")
_expect_equal(int(pipeline.call("total_work_count")), 0, "clear removes all work", failures)
func _verify_ownership_boundaries(failures: Array[String]) -> void:
var pipeline_source := FileAccess.get_file_as_string(PIPELINE_PATH)
var loader_source := FileAccess.get_file_as_string(LOADER_PATH)
_expect_true(
loader_source.contains("M2_MESH_LOAD_PIPELINE_STATE_SCRIPT.new()"),
"loader composes pipeline state",
failures
)
_expect_false(loader_source.contains("var _m2_mesh_load_requests:"), "legacy request field removed", failures)
_expect_false(loader_source.contains("var _m2_mesh_finalize_queue:"), "legacy finalize field removed", failures)
_expect_equal(
loader_source.count("_m2_mesh_load_pipeline_state.total_work_count()"),
3,
"three existing metrics delegate",
failures
)
_expect_equal(
loader_source.count("_m2_mesh_load_pipeline_state.clear()"),
2,
"two existing clear sites delegate",
failures
)
for retained_loader_rule in [
"ResourceLoader.load_threaded_request(",
"ResourceLoader.load_threaded_get_status(path)",
"ResourceLoader.load_threaded_get(path)",
"RENDER_BUDGET_SCHEDULER_SCRIPT.M2_MESH_FINALIZE",
"_m2_mesh_cache[normalized_rel]",
"_m2_missing_cache[normalized_rel]",
]:
_expect_true(loader_source.contains(retained_loader_rule), "loader retains %s" % retained_loader_rule, failures)
for forbidden_dependency in [
"ResourceLoader.",
"WorkerThreadPool.",
": Mesh",
"Node3D",
"RID(",
"_m2_mesh_cache",
"_m2_missing_cache",
]:
_expect_false(
pipeline_source.contains(forbidden_dependency),
"pipeline omits %s ownership" % forbidden_dependency,
failures
)
func _verify_bounded_timing(failures: Array[String]) -> float:
var pipeline: RefCounted = PIPELINE_SCRIPT.new()
var started_microseconds := Time.get_ticks_usec()
for _iteration in range(100):
for path_index in range(256):
var normalized_path := "world/model_%d.m2" % path_index
pipeline.call("remember_request", normalized_path, "res://model_%d.tscn" % path_index)
for path_index in range(256):
pipeline.call("complete_request", "world/model_%d.m2" % path_index, 3)
while pipeline.call("has_finalize_record"):
pipeline.call("pop_finalize_record")
pipeline.call("clear")
var elapsed_milliseconds := float(Time.get_ticks_usec() - started_microseconds) / 1000.0
_expect_true(elapsed_milliseconds < 1000.0, "100 by 256 transitions under 1 second", failures)
return elapsed_milliseconds
func _expect_true(condition: bool, label: String, failures: Array[String]) -> void:
if not condition:
failures.append(label)
func _expect_false(condition: bool, label: String, failures: Array[String]) -> void:
_expect_true(not condition, label, failures)
func _expect_equal(actual: int, expected: int, label: String, failures: Array[String]) -> void:
if actual != expected:
failures.append("%s expected=%d actual=%d" % [label, expected, actual])
func _expect_string_equal(actual: String, expected: String, label: String, failures: Array[String]) -> void:
if actual != expected:
failures.append("%s expected=%s actual=%s" % [label, expected, actual])
@@ -0,0 +1 @@
uid://bdn3pdtg62ou1
+19 -5
View File
@@ -27,7 +27,7 @@ Runtime и Editor используют facade; planner/scheduler тестиру
## Evidence
- Date: 2026-07-17
- Revision/worktree: master merges `7ece2ab`, `7e35de7`, `80cb084`, `d6e5b53`, `c69abd6`, `606770c`, `630a0c1`, `f36fabb`, `24aef13`, `2342430`, `396be5e`, `f88bf97`, `3f84717`, `d2cb52f`, `f9e5a4f`, `c06aed5`, `a878e7c`, `60603e1`, `b4955d6`, `ada0fd9`, `f470775`, `874fd0f`, `38f831e`;
- Revision/worktree: master merges `7ece2ab`, `7e35de7`, `80cb084`, `d6e5b53`, `c69abd6`, `606770c`, `630a0c1`, `f36fabb`, `24aef13`, `2342430`, `396be5e`, `f88bf97`, `3f84717`, `d2cb52f`, `f9e5a4f`, `c06aed5`, `a878e7c`, `60603e1`, `b4955d6`, `ada0fd9`, `f470775`, `874fd0f`, `38f831e`, `83b3618`;
packages `M03-RND-SCHEDULER-001`, `M03-RND-INTERNAL-ACCESS-GATE-001`,
`M03-RND-FACADE-GROUND-QUERY-001`, `M03-RND-FACADE-ENVIRONMENT-001`,
`M03-RND-FACADE-ENTITY-001`, `M03-RND-TERRAIN-CACHE-SERVICE-001`,
@@ -44,7 +44,8 @@ Runtime и Editor используют facade; planner/scheduler тестиру
`M03-RND-WMO-SCENE-RESOURCE-CACHE-001`,
`M03-RND-ADT-WATER-LOAD-PIPELINE-001`,
`M03-RND-ADT-WATER-SCENE-FINALIZER-001`,
`M03-RND-M2-RUNTIME-REBUILD-CLASSIFIER-001`
`M03-RND-M2-RUNTIME-REBUILD-CLASSIFIER-001`,
`M03-RND-M2-MESH-LOAD-PIPELINE-001`
- Commands: dedicated scheduler/planner/facade/focus/coordinate/manifest/shutdown
headless verifiers; Godot cold editor parse; coordination, documentation and diff gates.
- Results: scheduler `cases=6 iterations=20000 elapsed_ms=10.216`; planner
@@ -135,6 +136,13 @@ Runtime и Editor используют facade; planner/scheduler тестиру
regressions and checkpoint dry-run `7/7` passed. Post-merge classifier
(`45.439ms`), materials, shutdown, M2 batch planner, facade, internal-access
`35`, `7/7` manifest, documentation and coordination gates remained green.
M2 mesh load pipeline state passed `cases=11 iterations=100
elapsed_ms=100.273` with request validation/dedupe, insertion-order polling,
terminal transition/completion FIFO, opaque statuses, discard, detached
diagnostics, metrics, clear and 100-by-256 timing. Thirty-four adjacent
headless regressions and checkpoint dry-run `7/7` passed. Post-merge pipeline
state (`69.196ms`), classifier, materials, shutdown, facade, internal-access
`34`, `7/7` manifest, documentation and coordination gates remained green.
M02 terrain-query regression
remained green (13 pre-existing expired M00 claim warnings).
- Fidelity comparison: all 16 historical operation limits and drain sites were
@@ -200,6 +208,11 @@ Runtime и Editor используют facade; planner/scheduler тестиру
`0.000001` speed threshold, four-stage cap and first-decision-per-path cache
lifetime. Raw loading, material refresh version, M2Builder rebuild/fallback and
Mesh adoption remain loader-owned.
Static M2 mesh pending requests preserve normalized-path dedupe and Dictionary
insertion polling order; loaded/failed transitions preserve completion-order
finalization FIFO and opaque statuses. All three pending-plus-finalize metrics,
both clear sites and shutdown ResourceLoader drain order are unchanged; cache
path selection, I/O, permits, Mesh/missing caches and adoption remain loader-owned.
No new build-12340 parity claim is made; asset-backed visual/p95/p99 comparison remains pending.
- Changed files: scene-free scheduler and verifier; streamer budget composition/
permit delegation; scheduler module spec; world renderer/module registry/RENDER notes;
@@ -225,6 +238,7 @@ Runtime и Editor используют facade; planner/scheduler тестиру
drain/release/reset/shutdown/metrics adapters,
stateless ADT water scene finalizer and both loader build/attach adapters,
memoized M2 runtime mesh rebuild classifier and loader query/clear adapters,
M2 mesh load pipeline state and loader request/poll/finalize/metric/clear adapters,
expanded facade/internal-access/coordinate verifiers; work-package claims and this evidence.
- Remaining risks: worker concurrency and stale-result validation remain
streamer-owned; cancellation stops new permits but does not interrupt
@@ -238,9 +252,9 @@ Runtime и Editor используют facade; planner/scheduler тестиру
private Azeroth data was absent for the asset-backed ADT placement probe; M2
asset-backed visual transform recheck, grouping/build/tasks/caches/finalization
and boundary traversal p95/p99 remain pending; path grouping is not yet
culling-driven spatial-cell grouping; M2 rebuild classification is separated,
but raw loading, resource/queue transitions and mesh finalization remain in
the loader; raw
culling-driven spatial-cell grouping; M2 rebuild classification and static
request/finalize bookkeeping are separated, but ResourceLoader I/O, Mesh/
missing caches and mesh finalization remain in the loader; raw
WMO ResourceLoader/FileAccess I/O, live fallback and group materialization
remain in the loader;
asset-backed WMO placement/portal/material/leak/p95/p99 evidence remains pending;