open-wc/docs/modules/renderer-closeout-verification.md

Renderer Closeout Verification

Metadata

Field	Value
Status	Implemented
Target/work package	M03 / M03-QAR-INTEGRATOR-CLOSEOUT-001
Owner	Renderer structural, cache and performance acceptance gates
Last verified	Worktree work/sindo-main-codex-m03-integrator/m03-closeout, 2026-08-02
Profile	Blizzlike335; quality preset High for checkpoint evidence

Purpose

Provide reproducible closeout checks for the M03 renderer decomposition. The checks prove that worker boundaries remain CPU-only, main-thread frame steps are budgeted, cache versions remain explicit, and paired M00/M03 reports stay within the agreed 10% performance budgets.

Non-goals

• Claim pixel-level parity with the original build-12340 client.
• Replace subsystem unit verifiers or long-traversal release tests.
• Generate, mutate or migrate production cache payloads.
• Hide incompatible environments or cache inventories by normalizing reports.

Context and boundaries

flowchart LR
    M00[M00 commit capture] --> Reports[Render checkpoint reports]
    M03[M03 commit capture] --> Reports
    Reports --> Comparator[compare_render_performance.ps1]
    Sources[Renderer source and cache versions] --> Contracts[verify_renderer_closeout_contracts.gd]
    Comparator --> Stability[repeatability gate across short and long windows]
    Stability --> Evidence[JSON comparison and exit code]
    Contracts --> Evidence
    Evidence --> Target[M03 Evidence / DONE decision]

The capture command owns SceneTree execution and PNG/report writes. The PowerShell comparator is read-only except for its requested JSON output. The GDScript contract verifier reads source files and creates no renderer resources.

Public API

Symbol	Kind	Purpose	Preconditions	Failure
compare_render_performance.ps1 -BaselineReport <paths> -CandidateReport <paths> [-OutputReport <path>]	CLI	Compare one report or median of repeated reports	Same schema, profile, environment, cache contract/inventory and result keys	Exit 1 and enumerate incompatible fields or budget regressions
verify_render_performance_stability.ps1 -RepeatedSampleComparison <path> -LongWindowComparison <path>	CLI	Reject only a metric regression reproduced by both independent protocols	Both comparator reports contain the same 84 metrics	Exit 1 with every repeatable result/metric key
verify_renderer_closeout_contracts.gd	Godot CLI	Check worker/main-thread/cache/converter source contracts	Project parses and referenced sources exist	Exit 1 with named contract failure

Inputs and outputs

Direction	Contract/data	Producer	Consumer	Ownership	Lifetime
Input	M00/M03 report.json paths	capture_render_checkpoints.gd	Comparator	Filesystem-owned immutable evidence	One comparison
Input	Renderer/tool GDScript sources	Repository	Contract verifier	Read-only	One verifier run
Output	84 metric comparisons and failures	Comparator	Integrator/CI	Optional JSON plus process output	Evidence retention
Output	Repeatable and protocol-local regression inventories	Stability gate	Integrator/CI	Optional JSON plus process output	Evidence retention
Output	Structural pass/fail summary	Contract verifier	Integrator/CI	Process output	One run

Data flow

flowchart TD
    Read[Read every supplied report] --> Compatible{Metadata and inventory match?}
    Compatible -->|no| Fail[Exit 1 with exact mismatch]
    Compatible -->|yes| Median[Median each metric per checkpoint/pass]
    Median --> Pair[Pair 14 result keys]
    Pair --> Budget[Compare load, p95, p99, hitch and memory]
    Budget --> Json[Optional comparison JSON]
    Budget --> Cohorts[Repeated-sample and long-window comparisons]
    Cohorts --> Repeated{Same metric fails both?}
    Repeated -->|no| Pass[Exit 0 with protocol-local diagnostics]
    Repeated -->|yes| Fail

Lifecycle and sequence

sequenceDiagram
    participant I as Integrator
    participant B as M00 worktree
    participant C as M03 worktree
    participant R as Checkpoint capture
    participant G as Performance comparator
    I->>B: capture repeated baseline samples
    B->>R: same viewport, driver and cache inventory
    I->>C: capture repeated candidate samples
    C->>R: same viewport, driver and cache inventory
    I->>G: baseline paths plus candidate paths
    G->>G: compatibility checks and per-metric medians
    G-->>I: short-window and long-window JSON evidence
    I->>G: verify stability across both protocols
    G-->>I: repeatable-regression pass/fail

There is no persistent state machine. Each invocation is read, validate, aggregate, compare and terminate.

Ownership, threading and resources

• Reports and source files are borrowed read-only for one process.
• Median aggregation deep-copies the first report and never rewrites inputs.
• Rendering remains owned by the GUI capture process on Godot's main thread.
• The contract verifier is headless and does not instantiate the streaming world.
• The optional comparison report is wholly owned by the caller-selected path.

Errors, cancellation and recovery

Failure	Behavior	Recovery
Missing/malformed report	Terminate with path/shape error	Regenerate that capture
Environment/cache mismatch	Fail before accepting metrics	Recapture both commits on the same machine and cache
Missing/duplicate checkpoint	Fail result-key validation	Repair manifest/capture completeness
Metric over budget	Record baseline, candidate, limit and percentage	Diagnose named checkpoint/lane; rerun only after a code or evidence correction
Interrupted GUI capture	No complete report is accepted	Remove/ignore partial output and rerun

Configuration and capabilities

The comparator reads thresholds from the baseline report. M03 uses 10% maximum regression for load time, frame p95, frame p99, maximum hitch, static memory and video memory. Repeated input paths are optional; when supplied, each side is reduced independently to the median for every metric.

Closeout uses two independent protocols: repeated three-second captures and a ten-second measurement window. A regression is accepted as real only when the same checkpoint/pass/metric exceeds its unchanged 10% budget in both protocols. Protocol-local failures remain in the JSON as noise diagnostics rather than being discarded.

Persistence, cache and migrations

The comparison JSON uses schema version 1 and contains source paths, sample counts, revisions, all metric pairs and failures. It is evidence, not a runtime cache. Renderer cache versions are read from the manifest/source contracts; this module performs no migration or invalidation.

Diagnostics and observability

• Success reports result pairs, comparison count and budget percentage.
• Failure output names every checkpoint/pass, metric, values and limit.
• The structural verifier reports worker count, frame-step count and cache-version count.
• Capture reports retain PNG hashes, queue snapshots, environment and cache inventory.

Verification, fidelity and performance

• verify_renderer_closeout_contracts.gd covers four worker boundaries, fifteen frame steps, seven cache versions and the nested M2 GLB output contract.
• compare_render_performance.ps1 compares 14 cold/warm result pairs and 84 metrics.
• verify_render_performance_stability.ps1 requires metric-key agreement and rejects any budget regression reproduced by both sampling protocols.
• M00 and M03 must be captured from their exact commits against the same cache inventory; old reports with a different inventory are rejected.
• PNG hashes and asset-backed coverage prove that terrain, ADT boundaries, dense M2, large WMO, liquid, animated M2 and sky were rendered. They do not prove original-client pixel parity without human/reference-image approval.

Extension points

CI may retain reports and comparison JSON as artifacts. A future release gate may add driver-version metadata, long-traversal samples or approved visual-diff thresholds without changing runtime renderer contracts.

Known gaps and risks

• Godot reports the rendering API but not the installed NVIDIA driver version.
• A 0.5-second historical M00 measurement window requires repeated median samples.
• Original-client screenshots are not part of the repository evidence set.
• Long-traversal descriptor pressure remains a later quality/release gate.

Source map

Path	Responsibility
tools/compare_render_performance.ps1	Compatibility, median aggregation and metric budgets
tools/verify_render_performance_stability.ps1	Cross-protocol repeatability acceptance
src/tools/verify_renderer_closeout_contracts.gd	Structural source/cache/converter contracts
src/tools/capture_render_checkpoints.gd	Asset-backed GUI capture and report generation
src/tools/render_baseline_manifest.json	Coverage, viewport, cache contract and budgets
targets/00-render-baseline.md	Accepted M00 measurement protocol
targets/03-renderer-facade.md	M03 acceptance and Evidence