# ADT Water Load Pipeline State

## Metadata

| Field | Value |
|---|---|
| Status | Implemented |
| Target/work package | M03 / `M03-RND-ADT-WATER-LOAD-PIPELINE-001` |
| Owners | ADT water pending FIFO/dedupe, active task IDs and worker-safe result mailbox |
| Last verified | Worktree `work/sindo-main-codex/m03-adt-water-load-pipeline`, 2026-07-17 |
| Profiles/capabilities | ADT MH2O/MCLQ water loading; profile-independent state |

## Purpose

Own the cross-frame and cross-thread bookkeeping for asynchronous ADT water
loads outside `StreamingWorldLoader`: FIFO requests, queued-key deduplication,
opaque active task IDs and parsed-result delivery. The loader retains task
execution, parsing, budgets, stale-result decisions and scene finalization.

## Non-goals

- Start, cancel or wait WorkerThreadPool tasks.
- Instantiate `ADTLoader` or interpret MH2O/MCLQ data.
- Select concurrency limits or consume render permits.
- Create, attach or free Nodes, Meshes, materials or RIDs.
- Change liquid visuals, coordinates, tile acceptance or loaded-state timing.

## Context and boundaries

```mermaid
flowchart LR
    Tile[Loaded tile needing water] --> Loader[StreamingWorldLoader]
    Loader --> State[AdtWaterLoadPipelineState]
    State --> Request[Oldest tile/path request]
    Request --> Loader
    Loader --> Worker[WorkerThreadPool ADTLoader task]
    Worker --> State
    State --> Result[Parsed ADT result]
    Result --> Loader
    Loader --> Budget[WATER_FINALIZE permit]
    Loader --> Finalizer[AdtWaterSceneFinalizer]
    Finalizer --> Builder[ADTBuilder water scene]
    Finalizer --> Scene[Main-thread tile Node attach]
```

Allowed dependencies are RefCounted, Dictionary/Array/String and one Godot
`Mutex` around results. WorkerThreadPool, ADTLoader, scheduler, tile state,
scene-tree/render resources and gameplay/editor/network are forbidden.

## Public API

| Symbol | Kind | Purpose | Thread/lifetime | Errors |
|---|---|---|---|---|
| `enqueue_request(tile_key, adt_path)` | Command/query | Append deduplicated FIFO request | Renderer main thread; until dequeue/cancel/clear | Empty/queued/active returns false |
| `take_next_request()` | Command/query | Pop oldest request and release queued marker | Renderer main thread | Empty returns `{}` |
| `cancel_pending_requests(tile_key)` | Command/query | Remove all matching pending entries | Tile release/main thread | Empty/unknown removes zero; active unaffected |
| `remember_active_task(tile_key, task_id)` | Command/query | Retain loader-created opaque task ID | Main thread; until completion/clear | Empty/duplicate returns false |
| `has_active_task` / `active_task_id_for` | Query | Test/borrow active task identity | Main thread | Unknown ID is `-1` |
| `complete_active_task(tile_key)` | Command/query | Remove and return completed task ID | Main thread after wait | Unknown returns `-1` |
| `active_task_ids_snapshot()` | Query | Copy IDs for orderly shutdown waiting | Main thread | Detached Array |
| `publish_result(tile_key, path, data)` | Command/query | Append parsed result under mutex | Worker or main thread | Empty key/path returns false |
| `has_result()` / `pop_result()` | Query/command | Observe/pop oldest result under mutex | Main-thread drain | Empty pop returns `{}` |
| `total_load_count()` | Query | Pending plus active metric | Main thread | Results intentionally excluded |
| `pending_request_count()` / `active_task_count()` | Query | Support loader concurrency loop | Main thread | None |
| `clear()` | Command | Release all bookkeeping and results | Main thread after wait or map reset | Does not interrupt workers |
| `diagnostic_snapshot()` | Query | Return detached scalar requests/tasks/results | Main thread; result mailbox locked | Water data omitted |

## Inputs and outputs

| Direction | Contract/data | Producer | Consumer | Ownership | Thread/lifetime |
|---|---|---|---|---|---|
| Input | Tile key and ADT path | Loader tile state | Pending FIFO | Copied Strings | Until dequeue/cancel/clear |
| Input | Opaque task ID | Loader WorkerThreadPool adapter | Active task map | Integer value | Until wait/completion/clear |
| Input | Tile/path/parsed ADT Dictionary | Worker parser adapter | Result mailbox | Dictionary reference retained | Until pop/clear |
| Output | Oldest request Dictionary | Pipeline state | Loader task-start adapter | Fresh record from FIFO | One start attempt |
| Output | Active task ID snapshot | Pipeline state | Loader shutdown wait | Detached Array | Shutdown loop |
| Output | Oldest result Dictionary | Pipeline state | Loader finalize adapter | Mailbox record transferred | One finalize attempt |
| Output | Scalar diagnostics | Pipeline state | Verifier/future metrics | Detached records | Snapshot lifetime |

Side effects are limited to collection mutation, reference retention and result
mutex locking. Parsing, permits and scene mutation remain outside the module.

## Data flow

```mermaid
flowchart TD
    Enqueue[enqueue tile/path] --> Duplicate{Queued or active?}
    Duplicate -->|yes| Reject[false; unchanged]
    Duplicate -->|no| FIFO[Append FIFO and queued key]
    FIFO --> Take[take next; release queued marker]
    Take --> Validate[Loader checks tile/path]
    Validate --> Start[Loader starts worker and remembers task ID]
    Start --> Parse[Worker parses ADT]
    Parse --> Publish[Mutex-protected publish result]
    Publish --> Permit[Loader obtains WATER_FINALIZE permit]
    Permit --> Pop[Pop oldest result]
    Pop --> Wait[Loader waits task and completes active ID]
    Wait --> Stale{Tile/path/root current?}
    Stale -->|no| Discard[Discard result]
    Stale -->|yes| Attach[Build and attach water scene]
```

## Lifecycle/state

A tile request progresses through `Absent`, `Queued`, `Active` and
`ResultReady`; the loader wait/completion returns bookkeeping to `Absent` before
validating/attaching the result. Queued tile release cancels only `Queued`.
Active work is not interrupted; a later stale result is drained and discarded.

```mermaid
stateDiagram-v2
    [*] --> Absent
    Absent --> Queued: accepted enqueue
    Queued --> Absent: pending cancellation or invalid dequeue
    Queued --> Active: loader starts task
    Active --> ResultReady: worker publishes result
    ResultReady --> Absent: loader waits and completes task
    Queued --> Absent: clear
    Active --> Absent: clear bookkeeping only
    ResultReady --> Absent: clear
```

## Main sequence

```mermaid
sequenceDiagram
    participant Loader as StreamingWorldLoader main thread
    participant State as AdtWaterLoadPipelineState
    participant Worker as WorkerThreadPool task
    participant Budget as RenderBudgetScheduler
    participant Finalizer as AdtWaterSceneFinalizer
    Loader->>State: enqueue_request(tile key, ADT path)
    Loader->>State: take_next_request()
    Loader->>Worker: add_task(parse binding)
    Loader->>State: remember_active_task(key, task ID)
    Worker->>Worker: ADTLoader.load_adt(path)
    Worker->>State: publish_result(key, path, data)
    Note over State: result mutex protects publication/pop
    Loader->>Budget: try_consume_permit(WATER_FINALIZE)
    Loader->>State: pop_result()
    Loader->>Worker: wait_for_task_completion(task ID)
    Loader->>State: complete_active_task(key)
    Loader->>Loader: reject stale tile/path/root
    Loader->>Finalizer: attach_water_scene(data, origin, tile root, editor owner)
    Loader->>Loader: mark water_loaded
```

## Dependency diagram

```mermaid
flowchart TB
    Loader[StreamingWorldLoader] --> State[AdtWaterLoadPipelineState]
    Loader --> Scheduler[RenderBudgetScheduler]
    Loader --> Worker[WorkerThreadPool / ADTLoader]
    Loader --> Finalizer[AdtWaterSceneFinalizer]
    Finalizer --> Builder[ADTBuilder / SceneTree]
    Worker -->|publish only| State
    State -. no dependency .-> Scheduler
    State -. no dependency .-> Finalizer
```

## Ownership, threading and resources

- Main thread owns pending and active-task collection mutation.
- Worker threads may call only `publish_result`; result observe/pop/clear also lock.
- Parsed water Dictionary identity is retained until main-thread pop.
- Loader owns WorkerThreadPool task lifetime and must wait before orderly shutdown clear.
- Loader owns tile states and the tile subtree; `AdtWaterSceneFinalizer` performs
  the main-thread ADTBuilder call and attachment without retaining Nodes.
- `clear` cannot cancel a worker; post-reset publication becomes a stale result
  and is discarded because no active task/tile ownership remains.

## Errors, cancellation and recovery

| Failure | Detection | Behavior | Diagnostic | Recovery |
|---|---|---|---|---|
| Empty/duplicate request | Enqueue guards | Reject unchanged | Contract verifier | Correct caller state |
| Tile removed while queued | Loader release | Remove matching pending requests | Source/lifecycle verifier | Reload can enqueue later |
| Tile removed while active | Loader stale check after result | Wait task, remove active ID, discard result | Existing metrics | Reload can enqueue later |
| Parser unavailable/empty | Worker publishes empty Dictionary | Loader marks `water_loaded` without Node | Existing behavior | Future tile reload |
| Path changed | Loader compares result path | Discard after task completion | Source regression | Current tile can request again |
| Shutdown | Loader snapshots/waits active IDs | Clear all bookkeeping/results | Shutdown/source verifier | New loader starts empty |

## Configuration and capabilities

| Setting/capability | Default | Profile | Runtime mutable | Effect |
|---|---|---|---|---|
| `enable_water` | Loader setting | Quality/custom | Yes | Gates enqueue/process, unchanged |
| `max_concurrent_water_tasks` | Loader setting | Quality/custom | Yes | Bounds active tasks outside state |
| `water_finalize_ops_per_tick` | Loader setting | Quality/custom | Yes | Bounds result pops via scheduler |

## Persistence, cache and migration

No state is serialized or cached. ADT source/cache formats and liquid material
versions are unchanged; no migration or rebuild is required.

## Diagnostics and observability

- `total_load_count` preserves the prior pending-plus-active `water` metric.
- Snapshot exposes FIFO tile/path, sorted active keys and result tile/path only.
- Parsed liquid payloads, Nodes and task internals are not exposed.
- No logs or correlation IDs are introduced.

## Verification

- `verify_adt_water_load_pipeline_state.gd`: invalid/dedupe/FIFO behavior,
  active IDs, payload identity, real Thread publication, pending cancellation,
  clear, detached diagnostics, loader boundaries and bounded timing.
- Existing material/shutdown/facade/streaming regressions cover adjacent behavior.
- Fidelity evidence is preservation of current scheduling/finalize transitions;
  no MH2O/MCLQ visual or original-client parity claim is made.
- Performance budget: 100 cycles of 256 request/task/result transitions under 1 second.

## Extension points

- ADT parse execution can later move behind a parser/job adapter without changing state.
- Water scene materialization is now delegated to `AdtWaterSceneFinalizer`,
  independently of this async bookkeeping service.

## Capability status

| Capability | Status | Evidence | Gap/next step |
|---|---|---|---|
| ADT water load pipeline state | Implemented extraction | FIFO/task/thread/source/timing verifier | Asset-backed traversal/leak evidence pending |
| ADT liquid parsing | Existing/loader-owned adapter | Native ADTLoader and dry-run paths | MH2O/MCLQ fidelity fixtures pending |
| Water scene finalization | Implemented extraction | Dedicated synthetic scene/ownership verifier | Asset-backed traversal/hitch evidence pending |

## Known gaps and risks

- `clear` intentionally does not interrupt active WorkerThreadPool work.
- Result payload Dictionaries are retained by reference and must not be mutated after publication.
- Result availability and pop use separate mutex acquisitions; one serialized
  main-thread consumer preserves ordering, but the API is not a multi-consumer queue.
- No proprietary liquid corpus, traversal/leak run, p95/p99 or paired capture is included.

## Source map

| Path | Responsibility |
|---|---|
| `src/render/liquid/adt_water_load_pipeline_state.gd` | Request/task/result bookkeeping and result mutex |
| `src/render/liquid/adt_water_scene_finalizer.gd` | Main-thread ADTBuilder call, attach and optional editor ownership |
| `src/scenes/streaming/streaming_world_loader.gd` | Parse task execution, permits, stale checks and tile loaded timing |
| `addons/mpq_extractor/loaders/adt_builder.gd` | ADT liquid geometry/material construction |
| `src/tools/verify_adt_water_load_pipeline_state.gd` | Lifecycle/thread/boundary/timing regression |

## Related decisions and references

- [`world-renderer.md`](world-renderer.md)
- [`adt-water-scene-finalizer.md`](adt-water-scene-finalizer.md)
- [`render-budget-scheduler.md`](render-budget-scheduler.md)
- [`../../RENDER.md`](../../RENDER.md)
- [`../../targets/roadmap/02-rendering-and-graphics.md`](../../targets/roadmap/02-rendering-and-graphics.md)