242 lines
12 KiB
Markdown
242 lines
12 KiB
Markdown
# M2 Mesh Load Pipeline State
|
|
|
|
## Metadata
|
|
|
|
| Field | Value |
|
|
|---|---|
|
|
| Status | Implemented extraction |
|
|
| Target/work package | M03 / `M03-RND-M2-MESH-LOAD-PIPELINE-001` |
|
|
| Owners | Static M2 threaded request records and terminal finalize FIFO |
|
|
| Last verified | Worktree `work/sindo-main-codex/m03-m2-mesh-load-pipeline`, 2026-07-17 |
|
|
| Profiles/capabilities | Existing static M2 cached `.tscn`/`.glb` load path |
|
|
|
|
## Purpose
|
|
|
|
Own cross-frame bookkeeping between successful static M2 ResourceLoader request
|
|
start, terminal loaded/failed polling and budgeted main-thread finalization. The
|
|
loader retains every I/O and engine-resource operation.
|
|
|
|
## Non-goals
|
|
|
|
- Call ResourceLoader or select cache paths/formats.
|
|
- Own M2 scene, missing or material-refresh caches, or prepared Mesh references.
|
|
- Extract Meshes from PackedScene/GLB Resources.
|
|
- Consume finalize permits or rebuild/adopt runtime Meshes.
|
|
- Merge static and animated M2 pipelines.
|
|
|
|
## Context and boundaries
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
Need[Static M2 mesh needed] --> Loader[StreamingWorldLoader]
|
|
Loader --> IO[ResourceLoader request]
|
|
Loader --> State[M2MeshLoadPipelineState]
|
|
State --> Poll[Detached pending records]
|
|
Poll --> Loader
|
|
Loader --> IO
|
|
Loader -->|terminal status| State
|
|
State --> FIFO[Finalize FIFO]
|
|
FIFO --> Loader
|
|
Loader --> Budget[M2_MESH_FINALIZE permit]
|
|
Loader --> Cache[M2 Mesh resource cache or M2 prototype outcome state]
|
|
```
|
|
|
|
Allowed dependencies are value containers, Strings and opaque integer statuses.
|
|
ResourceLoader, filesystem, WorkerThreadPool, Mesh/Resource/Node/RID ownership,
|
|
scheduler and other application layers are forbidden.
|
|
|
|
## Public API
|
|
|
|
| Symbol | Kind | Purpose | Thread/lifetime | Errors |
|
|
|---|---|---|---|---|
|
|
| `remember_request(normalized_relative_path, resource_path)` | Command/query | Insert one pending request | Renderer main thread; until terminal/discard/clear | Empty/duplicate rejected |
|
|
| `has_request(path)` | Query | Test pending dedupe | Main thread | Empty/unknown false |
|
|
| `request_records_snapshot()` | Query | Copy pending records in insertion order | Main thread; caller-owned | None |
|
|
| `complete_request(path, terminal_status)` | Command/query | Move request to terminal FIFO | Main thread after poll | Unknown rejected |
|
|
| `discard_request(path)` | Command/query | Remove request without finalization | Main thread | Unknown false |
|
|
| `has_finalize_record()` / `pop_finalize_record()` | Query/command | Observe/pop completion-order FIFO | Main-thread budget drain | Empty pop returns `{}` |
|
|
| `total_work_count()` | Query | Preserve pending-plus-finalize metric | Main thread | None |
|
|
| `pending_request_count()` / `finalize_record_count()` | Query | Stage diagnostics | Main thread | None |
|
|
| `clear()` | Command | Drop bookkeeping after I/O drain/reset | Main thread | Idempotent; does not drain I/O |
|
|
| `diagnostic_snapshot()` | Query | Detached request/finalize records | Main thread | No Resources exposed |
|
|
|
|
## Inputs and outputs
|
|
|
|
| Direction | Contract/data | Producer | Consumer | Ownership | Thread/lifetime |
|
|
|---|---|---|---|---|---|
|
|
| Input | Normalized M2 path and cache Resource path | Loader request adapter | Pending map | Copied Strings | Until terminal/discard/clear |
|
|
| Input | Opaque terminal ResourceLoader status | Loader poll adapter | Finalize record | Integer value | Until pop/clear |
|
|
| Output | Detached pending records | Pipeline state | Loader poll/shutdown adapter | Caller-owned copies | One poll/wait pass |
|
|
| Output | Oldest terminal record | Pipeline state | Loader finalizer | Record ownership transferred | One finalize attempt |
|
|
| Output | Detached diagnostics | Pipeline state | Verifier/future metrics | Caller-owned copies | Snapshot lifetime |
|
|
|
|
Side effects are limited to collection mutation and retaining String/integer values.
|
|
|
|
## Data flow
|
|
|
|
```mermaid
|
|
flowchart TD
|
|
Start[Successful ResourceLoader request] --> Remember[remember request]
|
|
Remember --> Snapshot[request records snapshot]
|
|
Snapshot --> Poll[Loader polls status]
|
|
Poll --> Active{In progress?}
|
|
Active -->|yes| Snapshot
|
|
Active -->|no| Complete[complete request with terminal status]
|
|
Complete --> FIFO[Append finalize FIFO]
|
|
FIFO --> Permit{Loader permit available?}
|
|
Permit -->|no| FIFO
|
|
Permit -->|yes| Pop[Pop oldest terminal record]
|
|
Pop --> Finalize[Loader gets Resource and delegates first-Mesh extraction]
|
|
Finalize --> Prepare[M2RuntimeMeshFinalizer prepares Mesh]
|
|
Prepare --> Adopt[Loader adopts Mesh or marks prototype outcome state]
|
|
```
|
|
|
|
## Lifecycle/state
|
|
|
|
```mermaid
|
|
stateDiagram-v2
|
|
[*] --> Absent
|
|
Absent --> Pending: remember request
|
|
Pending --> Pending: non-terminal poll
|
|
Pending --> TerminalQueued: complete request
|
|
Pending --> Absent: discard
|
|
TerminalQueued --> Absent: pop finalize
|
|
Pending --> Absent: clear
|
|
TerminalQueued --> Absent: clear
|
|
```
|
|
|
|
One path has at most one pending request. After completion it may be requested
|
|
again only if loader cache/missing rules permit it.
|
|
|
|
## Main sequence
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Loader as StreamingWorldLoader
|
|
participant Resource as ResourceLoader
|
|
participant State as M2MeshLoadPipelineState
|
|
participant Budget as RenderBudgetScheduler
|
|
Loader->>Resource: load_threaded_request(cache path)
|
|
Resource-->>Loader: OK or ERR_BUSY
|
|
Loader->>State: remember_request(normalized, cache path)
|
|
loop frames
|
|
Loader->>State: request_records_snapshot()
|
|
Loader->>Resource: load_threaded_get_status(path)
|
|
end
|
|
Loader->>State: complete_request(normalized, terminal status)
|
|
Loader->>Budget: try_consume_permit(M2_MESH_FINALIZE)
|
|
Loader->>State: pop_finalize_record()
|
|
Loader->>Resource: load_threaded_get(path)
|
|
Loader->>Loader: extract/refresh/adopt Mesh or mark missing
|
|
```
|
|
|
|
## Dependency diagram
|
|
|
|
```mermaid
|
|
flowchart TB
|
|
Loader[StreamingWorldLoader] --> State[M2MeshLoadPipelineState]
|
|
Loader --> Resource[ResourceLoader]
|
|
Loader --> Budget[RenderBudgetScheduler]
|
|
Loader --> MeshCache[M2 Mesh resource and prototype outcome cache states]
|
|
Loader --> Finalizer[M2RuntimeMeshFinalizer]
|
|
State -. no dependency .-> Resource
|
|
State -. no dependency .-> Budget
|
|
State -. no dependency .-> MeshCache
|
|
```
|
|
|
|
## Ownership, threading and resources
|
|
|
|
- Main thread serializes all state mutation and snapshots.
|
|
- State owns only request/finalize Dictionaries containing Strings and status integers.
|
|
- Loader owns ResourceLoader request lifetime and drains active paths before shutdown clear.
|
|
- `M2MeshResourceCacheState` owns prepared static Mesh references and
|
|
`M2RuntimeMeshFinalizer` owns refresh/rebuild/fallback. The loader owns shared
|
|
adoption decisions, raw I/O and remaining engine resources; the prototype
|
|
cache service owns missing outcomes.
|
|
- Snapshot and diagnostic records are detached; caller mutation cannot affect state.
|
|
|
|
## Errors, cancellation and recovery
|
|
|
|
| Failure | Detection | Behavior | Diagnostic | Recovery |
|
|
|---|---|---|---|---|
|
|
| Empty/duplicate request | State guard | Reject unchanged | Contract verifier | Correct caller/request later |
|
|
| Request start failure | Loader return code | No state insert; mark missing | Existing loader behavior | Cache/source correction |
|
|
| Non-terminal status | Loader poll | Keep pending | Existing queue metric | Poll next frame |
|
|
| Terminal load failure | Status in popped record | Loader marks missing | Existing missing behavior | World/cache reload |
|
|
| Empty defensive path | Loader before poll | Discard and mark missing | Source regression | Correct request producer |
|
|
| Shutdown | Loader drains pending Resource paths | Clear state | Shutdown verifier | New loader starts empty |
|
|
|
|
## Configuration and capabilities
|
|
|
|
| Setting/capability | Default | Profile | Runtime mutable | Effect |
|
|
|---|---|---|---|---|
|
|
| `m2_mesh_finalize_ops_per_tick` | `1` | Quality/custom | Yes | Bounds terminal FIFO pops outside state |
|
|
| Cache extension order | `.tscn`, `.glb` | Existing M2 path | No | Loader selects request path |
|
|
| Animated pivot-prefix GLB exclusion | Existing rule | All | No | Loader filters before request |
|
|
|
|
## Persistence, cache and migration
|
|
|
|
No pipeline state is serialized. M2 cache formats, material refresh version and
|
|
rebuild policy are unchanged; no migration or rebake is required.
|
|
|
|
## Diagnostics and observability
|
|
|
|
- `total_work_count` preserves all three historical `m2_mesh` metrics.
|
|
- Diagnostic snapshots expose only paths/statuses, never loaded Resources or Meshes.
|
|
- Existing loader logs and hitch sections remain unchanged.
|
|
- Normalized M2 path is the correlation key.
|
|
|
|
## Verification
|
|
|
|
- `verify_m2_mesh_load_pipeline_state.gd`: validation, dedupe, insertion-order
|
|
polling, terminal transition, completion FIFO, opaque status, discard, metrics,
|
|
detached diagnostics, loader ownership and 100-by-256 timing.
|
|
- Classifier/material/shutdown/M2 placement/build regressions cover adjacent behavior.
|
|
- Fidelity evidence is exact state/lifecycle extraction; no asset-backed or
|
|
original-client visual parity claim is made.
|
|
- Performance budget: 25,600 request/complete/pop transitions under one second.
|
|
|
|
## Extension points
|
|
|
|
- Mesh/missing cache adoption can move to a separate finalizer without changing this state.
|
|
- Animated M2 has distinct candidate/static fallback semantics and remains separate.
|
|
|
|
## Capability status
|
|
|
|
| Capability | Status | Evidence | Gap/next step |
|
|
|---|---|---|---|
|
|
| Static M2 request/finalize state | Implemented extraction | Contract/source/timing verifier | Asset-backed long traversal pending |
|
|
| ResourceLoader I/O | Existing loader-owned | Shutdown/material regressions | I/O adapter extraction optional |
|
|
| Mesh cache | Implemented extraction | Mesh resource cache state verifier | Asset-backed memory/leak run pending |
|
|
| First-Mesh extraction | Implemented extraction | Resource/order/lifetime verifier | Asset-backed corrupt-scene fixture pending |
|
|
| Mesh preparation | Implemented extraction | Runtime finalizer transition/rebuild verifier | Asset-backed material comparison pending |
|
|
| Mesh materialization | Existing loader-owned | Renderer/material tests | Further safe extraction pending |
|
|
|
|
## Known gaps and risks
|
|
|
|
- Records remain Dictionaries because ResourceLoader polling is a dynamic Godot boundary.
|
|
- Clear does not drain ResourceLoader; caller ordering is mandatory.
|
|
- Status integers are intentionally opaque inside state.
|
|
- No private asset traversal, leak/descriptor-pressure or p95/p99 run is included.
|
|
|
|
## Source map
|
|
|
|
| Path | Responsibility |
|
|
|---|---|
|
|
| `src/render/m2/m2_mesh_load_pipeline_state.gd` | Pending records, terminal FIFO and metrics |
|
|
| `src/render/m2/m2_mesh_resource_cache_state.gd` | Prepared static Mesh references and final clear |
|
|
| `src/render/m2/m2_mesh_resource_extractor.gd` | First-Mesh selection and temporary PackedScene lifetime |
|
|
| `src/render/m2/m2_runtime_mesh_finalizer.gd` | Refresh/rebuild/fallback preparation |
|
|
| `src/render/m2/m2_prototype_cache_state.gd` | Missing-model outcome retention |
|
|
| `src/scenes/streaming/streaming_world_loader.gd` | Cache path choice, I/O polling, permits and adoption decisions |
|
|
| `src/render/m2/m2_runtime_mesh_rebuild_classifier.gd` | Downstream stale Mesh rebuild decision |
|
|
| `src/tools/verify_m2_mesh_load_pipeline_state.gd` | Lifecycle/boundary/timing regression |
|
|
|
|
## Related decisions and references
|
|
|
|
- [`m2-runtime-mesh-rebuild-classifier.md`](m2-runtime-mesh-rebuild-classifier.md)
|
|
- [`m2-build-batch-planner.md`](m2-build-batch-planner.md)
|
|
- [`world-renderer.md`](world-renderer.md)
|
|
- [`../../RENDER.md`](../../RENDER.md)
|
|
- [`../../targets/roadmap/02-rendering-and-graphics.md`](../../targets/roadmap/02-rendering-and-graphics.md)
|