open-wc/docs/modules/adt-water-load-pipeline-state.md

ADT Water Load Pipeline State

Metadata

Field	Value
Status	Implemented
Target/work package	M03 / M03-RND-ADT-WATER-LOAD-PIPELINE-001
Owners	ADT water pending FIFO/dedupe, active task IDs and worker-safe result mailbox
Last verified	Worktree work/sindo-main-codex/m03-adt-water-load-pipeline, 2026-07-17
Profiles/capabilities	ADT MH2O/MCLQ water loading; profile-independent state

Purpose

Own the cross-frame and cross-thread bookkeeping for asynchronous ADT water loads outside StreamingWorldLoader: FIFO requests, queued-key deduplication, opaque active task IDs and parsed-result delivery. The loader retains task execution, parsing, budgets, stale-result decisions and scene finalization.

Non-goals

• Start, cancel or wait WorkerThreadPool tasks.
• Instantiate ADTLoader or interpret MH2O/MCLQ data.
• Select concurrency limits or consume render permits.
• Create, attach or free Nodes, Meshes, materials or RIDs.
• Change liquid visuals, coordinates, tile acceptance or loaded-state timing.

Context and boundaries

flowchart LR
    Tile[Loaded tile needing water] --> Loader[StreamingWorldLoader]
    Loader --> State[AdtWaterLoadPipelineState]
    State --> Request[Oldest tile/path request]
    Request --> Loader
    Loader --> Worker[WorkerThreadPool ADTLoader task]
    Worker --> State
    State --> Result[Parsed ADT result]
    Result --> Loader
    Loader --> Budget[WATER_FINALIZE permit]
    Loader --> Builder[ADTBuilder water scene]
    Builder --> Scene[Main-thread tile Node attach]

Allowed dependencies are RefCounted, Dictionary/Array/String and one Godot Mutex around results. WorkerThreadPool, ADTLoader, scheduler, tile state, scene-tree/render resources and gameplay/editor/network are forbidden.

Public API

Symbol	Kind	Purpose	Thread/lifetime	Errors
enqueue_request(tile_key, adt_path)	Command/query	Append deduplicated FIFO request	Renderer main thread; until dequeue/cancel/clear	Empty/queued/active returns false
take_next_request()	Command/query	Pop oldest request and release queued marker	Renderer main thread	Empty returns {}
cancel_pending_requests(tile_key)	Command/query	Remove all matching pending entries	Tile release/main thread	Empty/unknown removes zero; active unaffected
remember_active_task(tile_key, task_id)	Command/query	Retain loader-created opaque task ID	Main thread; until completion/clear	Empty/duplicate returns false
has_active_task / active_task_id_for	Query	Test/borrow active task identity	Main thread	Unknown ID is -1
complete_active_task(tile_key)	Command/query	Remove and return completed task ID	Main thread after wait	Unknown returns -1
active_task_ids_snapshot()	Query	Copy IDs for orderly shutdown waiting	Main thread	Detached Array
publish_result(tile_key, path, data)	Command/query	Append parsed result under mutex	Worker or main thread	Empty key/path returns false
has_result() / pop_result()	Query/command	Observe/pop oldest result under mutex	Main-thread drain	Empty pop returns {}
total_load_count()	Query	Pending plus active metric	Main thread	Results intentionally excluded
pending_request_count() / active_task_count()	Query	Support loader concurrency loop	Main thread	None
clear()	Command	Release all bookkeeping and results	Main thread after wait or map reset	Does not interrupt workers
diagnostic_snapshot()	Query	Return detached scalar requests/tasks/results	Main thread; result mailbox locked	Water data omitted

Inputs and outputs

Direction	Contract/data	Producer	Consumer	Ownership	Thread/lifetime
Input	Tile key and ADT path	Loader tile state	Pending FIFO	Copied Strings	Until dequeue/cancel/clear
Input	Opaque task ID	Loader WorkerThreadPool adapter	Active task map	Integer value	Until wait/completion/clear
Input	Tile/path/parsed ADT Dictionary	Worker parser adapter	Result mailbox	Dictionary reference retained	Until pop/clear
Output	Oldest request Dictionary	Pipeline state	Loader task-start adapter	Fresh record from FIFO	One start attempt
Output	Active task ID snapshot	Pipeline state	Loader shutdown wait	Detached Array	Shutdown loop
Output	Oldest result Dictionary	Pipeline state	Loader finalize adapter	Mailbox record transferred	One finalize attempt
Output	Scalar diagnostics	Pipeline state	Verifier/future metrics	Detached records	Snapshot lifetime

Side effects are limited to collection mutation, reference retention and result mutex locking. Parsing, permits and scene mutation remain outside the module.

Data flow

flowchart TD
    Enqueue[enqueue tile/path] --> Duplicate{Queued or active?}
    Duplicate -->|yes| Reject[false; unchanged]
    Duplicate -->|no| FIFO[Append FIFO and queued key]
    FIFO --> Take[take next; release queued marker]
    Take --> Validate[Loader checks tile/path]
    Validate --> Start[Loader starts worker and remembers task ID]
    Start --> Parse[Worker parses ADT]
    Parse --> Publish[Mutex-protected publish result]
    Publish --> Permit[Loader obtains WATER_FINALIZE permit]
    Permit --> Pop[Pop oldest result]
    Pop --> Wait[Loader waits task and completes active ID]
    Wait --> Stale{Tile/path/root current?}
    Stale -->|no| Discard[Discard result]
    Stale -->|yes| Attach[Build and attach water scene]

Lifecycle/state

A tile request progresses through Absent, Queued, Active and ResultReady; the loader wait/completion returns bookkeeping to Absent before validating/attaching the result. Queued tile release cancels only Queued. Active work is not interrupted; a later stale result is drained and discarded.

stateDiagram-v2
    [*] --> Absent
    Absent --> Queued: accepted enqueue
    Queued --> Absent: pending cancellation or invalid dequeue
    Queued --> Active: loader starts task
    Active --> ResultReady: worker publishes result
    ResultReady --> Absent: loader waits and completes task
    Queued --> Absent: clear
    Active --> Absent: clear bookkeeping only
    ResultReady --> Absent: clear

Main sequence

sequenceDiagram
    participant Loader as StreamingWorldLoader main thread
    participant State as AdtWaterLoadPipelineState
    participant Worker as WorkerThreadPool task
    participant Budget as RenderBudgetScheduler
    participant Builder as ADTBuilder
    Loader->>State: enqueue_request(tile key, ADT path)
    Loader->>State: take_next_request()
    Loader->>Worker: add_task(parse binding)
    Loader->>State: remember_active_task(key, task ID)
    Worker->>Worker: ADTLoader.load_adt(path)
    Worker->>State: publish_result(key, path, data)
    Note over State: result mutex protects publication/pop
    Loader->>Budget: try_consume_permit(WATER_FINALIZE)
    Loader->>State: pop_result()
    Loader->>Worker: wait_for_task_completion(task ID)
    Loader->>State: complete_active_task(key)
    Loader->>Loader: reject stale tile/path/root
    Loader->>Builder: build_tile_water_scene(data, origin)
    Loader->>Loader: attach Node and mark water_loaded

Dependency diagram

flowchart TB
    Loader[StreamingWorldLoader] --> State[AdtWaterLoadPipelineState]
    Loader --> Scheduler[RenderBudgetScheduler]
    Loader --> Worker[WorkerThreadPool / ADTLoader]
    Loader --> Builder[ADTBuilder / SceneTree]
    Worker -->|publish only| State
    State -. no dependency .-> Scheduler
    State -. no dependency .-> Builder

Ownership, threading and resources

• Main thread owns pending and active-task collection mutation.
• Worker threads may call only publish_result; result observe/pop/clear also lock.
• Parsed water Dictionary identity is retained until main-thread pop.
• Loader owns WorkerThreadPool task lifetime and must wait before orderly shutdown clear.
• Loader owns tile states, ADTBuilder, water Nodes and every render resource.
• clear cannot cancel a worker; post-reset publication becomes a stale result and is discarded because no active task/tile ownership remains.

Errors, cancellation and recovery

Failure	Detection	Behavior	Diagnostic	Recovery
Empty/duplicate request	Enqueue guards	Reject unchanged	Contract verifier	Correct caller state
Tile removed while queued	Loader release	Remove matching pending requests	Source/lifecycle verifier	Reload can enqueue later
Tile removed while active	Loader stale check after result	Wait task, remove active ID, discard result	Existing metrics	Reload can enqueue later
Parser unavailable/empty	Worker publishes empty Dictionary	Loader marks water_loaded without Node	Existing behavior	Future tile reload
Path changed	Loader compares result path	Discard after task completion	Source regression	Current tile can request again
Shutdown	Loader snapshots/waits active IDs	Clear all bookkeeping/results	Shutdown/source verifier	New loader starts empty

Configuration and capabilities

Setting/capability	Default	Profile	Runtime mutable	Effect
enable_water	Loader setting	Quality/custom	Yes	Gates enqueue/process, unchanged
max_concurrent_water_tasks	Loader setting	Quality/custom	Yes	Bounds active tasks outside state
water_finalize_ops_per_tick	Loader setting	Quality/custom	Yes	Bounds result pops via scheduler

Persistence, cache and migration

No state is serialized or cached. ADT source/cache formats and liquid material versions are unchanged; no migration or rebuild is required.

Diagnostics and observability

• total_load_count preserves the prior pending-plus-active water metric.
• Snapshot exposes FIFO tile/path, sorted active keys and result tile/path only.
• Parsed liquid payloads, Nodes and task internals are not exposed.
• No logs or correlation IDs are introduced.

Verification

• verify_adt_water_load_pipeline_state.gd: invalid/dedupe/FIFO behavior, active IDs, payload identity, real Thread publication, pending cancellation, clear, detached diagnostics, loader boundaries and bounded timing.
• Existing material/shutdown/facade/streaming regressions cover adjacent behavior.
• Fidelity evidence is preservation of current scheduling/finalize transitions; no MH2O/MCLQ visual or original-client parity claim is made.
• Performance budget: 100 cycles of 256 request/task/result transitions under 1 second.

Extension points

• ADT parse execution can later move behind a parser/job adapter without changing state.
• Water scene materialization can be extracted separately because it owns Nodes and main-thread effects rather than async bookkeeping.

Capability status

Capability	Status	Evidence	Gap/next step
ADT water load pipeline state	Implemented extraction	FIFO/task/thread/source/timing verifier	Asset-backed traversal/leak evidence pending
ADT liquid parsing	Existing/loader-owned adapter	Native ADTLoader and dry-run paths	MH2O/MCLQ fidelity fixtures pending
Water scene finalization	Existing/loader-owned	Scheduler/material/shutdown regressions	Separate materialization extraction possible

Known gaps and risks

• clear intentionally does not interrupt active WorkerThreadPool work.
• Result payload Dictionaries are retained by reference and must not be mutated after publication.
• Result availability and pop use separate mutex acquisitions; one serialized main-thread consumer preserves ordering, but the API is not a multi-consumer queue.
• No proprietary liquid corpus, traversal/leak run, p95/p99 or paired capture is included.

Source map

Path	Responsibility
src/render/liquid/adt_water_load_pipeline_state.gd	Request/task/result bookkeeping and result mutex
src/scenes/streaming/streaming_world_loader.gd	Parse task execution, permits, stale checks and Node finalization
addons/mpq_extractor/loaders/adt_builder.gd	ADT liquid geometry/material construction
src/tools/verify_adt_water_load_pipeline_state.gd	Lifecycle/thread/boundary/timing regression

Related decisions and references

• world-renderer.md
• render-budget-scheduler.md
• ../../RENDER.md
• ../../targets/roadmap/02-rendering-and-graphics.md