Complete M03 renderer closeout validation
This commit is contained in:
@@ -54,6 +54,23 @@
|
||||
- dense WMO/M2, water, character equipment и UI scale matrices.
|
||||
- navmesh overlay checkpoints и bake/query budgets для больших tiles/dungeons.
|
||||
|
||||
Для M03 renderer closeout сравнение выполняется на точных M00/M03 revisions с
|
||||
одинаковыми viewport, rendering backend и полным cache inventory. Короткий
|
||||
протокол агрегирует повторные captures медианой каждого показателя; независимый
|
||||
протокол использует увеличенное десятисекундное окно. Регрессия считается
|
||||
воспроизводимой, только если один и тот же checkpoint/pass/metric превышает
|
||||
неизменённый 10% budget в обоих протоколах:
|
||||
|
||||
```powershell
|
||||
tools/compare_render_performance.ps1 -BaselineReport <m00-reports> -CandidateReport <m03-reports> -OutputReport <repeated.json>
|
||||
tools/compare_render_performance.ps1 -BaselineReport <m00-long.json> -CandidateReport <m03-long.json> -OutputReport <long.json>
|
||||
tools/verify_render_performance_stability.ps1 -RepeatedSampleComparison <repeated.json> -LongWindowComparison <long.json> -OutputReport <stability.json>
|
||||
```
|
||||
|
||||
Локальные превышения одного протокола сохраняются как diagnostics; gate падает
|
||||
только на повторяемой регрессии. Полный контракт и схема evidence описаны в
|
||||
[`modules/renderer-closeout-verification.md`](modules/renderer-closeout-verification.md).
|
||||
|
||||
### Navigation compatibility
|
||||
|
||||
- Golden synthetic meshes проверяют slope, climb, radius erosion, holes, tiled seams и off-mesh connections.
|
||||
|
||||
@@ -56,6 +56,7 @@
|
||||
| Third-person camera | Implemented | [`third-person-camera.md`](third-person-camera.md) |
|
||||
| Character presentation | Implemented boundary / Partial fidelity | [`character-presentation.md`](character-presentation.md) |
|
||||
| Renderer | Partial | [`world-renderer.md`](world-renderer.md), [`../../RENDER.md`](../../RENDER.md) |
|
||||
| Renderer closeout verification | Implemented | [`renderer-closeout-verification.md`](renderer-closeout-verification.md) |
|
||||
| World entity presentation | Implemented boundary / Prototype visuals | [`world-entity-presentation.md`](world-entity-presentation.md) |
|
||||
| Streaming target planner | Implemented | [`streaming-target-planner.md`](streaming-target-planner.md) |
|
||||
| Render budget scheduler | Implemented | [`render-budget-scheduler.md`](render-budget-scheduler.md) |
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
| Status | Implemented extraction |
|
||||
| Target/work package | M03 / `M03-RND-M2-ANIMATION-PLAYBACK-001` |
|
||||
| Owners | Per-instance AnimationPlayer/native animator playback mutation |
|
||||
| Last verified | Worktree `work/sindo-main-codex/m03-m2-animation-playback`, 2026-07-18 |
|
||||
| Last verified | Worktree `work/sindo-main-codex-m03-integrator/m03-closeout`, 2026-08-02 |
|
||||
| Profiles/capabilities | Imported GLB and native experimental animated M2 instances |
|
||||
|
||||
## Purpose
|
||||
@@ -59,7 +59,7 @@ MultiMesh, SceneTree attachment and application layers are forbidden.
|
||||
| Output | Mutated native/imported playback | Controller | Rendered instance | Nodes retain state/resources | Instance lifetime |
|
||||
| Output | Detached native diagnostic records | Controller | Loader log adapter | Caller-owned Dictionaries | Debug call |
|
||||
|
||||
Side effects are native field assignment, prepare/phase calls, animation loop
|
||||
Side effects are native field assignment, phased preparation calls, animation loop
|
||||
mutation, play and seek. The service retains no inputs.
|
||||
|
||||
## Data flow
|
||||
@@ -67,13 +67,13 @@ mutation, play and seek. The service retains no inputs.
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Identity[Path and index] --> Phase[Stable hash phase]
|
||||
NativeInventory[Exact-script native inventory] --> Prepare[prepare runtime if available]
|
||||
Phase --> NativePhase[set native phase]
|
||||
NativeInventory[Exact-script native inventory] --> Prepare[prepare local runtime mesh]
|
||||
Phase --> Prepare
|
||||
Players[AnimationPlayers] --> Select[Choose path-specific default]
|
||||
Select --> Loop[Set every animation LOOP_LINEAR]
|
||||
Loop --> Play[Play selected name]
|
||||
Phase --> Seek[Seek positive-length selection]
|
||||
NativePhase --> Diagnostics{Debug requested?}
|
||||
Prepare --> Diagnostics{Debug requested?}
|
||||
Diagnostics -->|yes| Snapshot[Detached runtime state]
|
||||
```
|
||||
|
||||
@@ -103,7 +103,7 @@ sequenceDiagram
|
||||
M->>F: animation_players_in_subtree(duplicate)
|
||||
F-->>M: ordered players
|
||||
M->>P: start_instance_playback(path, index, players, debug)
|
||||
P->>N: prepare_runtime and set_phase
|
||||
P->>N: prepare_runtime_at_phase
|
||||
P->>A: choose, loop, play and seek
|
||||
P-->>M: optional detached native diagnostics
|
||||
M-->>M: tag states with instance index
|
||||
@@ -132,6 +132,11 @@ flowchart TB
|
||||
- Native arrays are assigned by reference exactly as before extraction.
|
||||
- Diagnostic Dictionaries are deep-duplicated before return.
|
||||
- Main thread performs all engine-object mutation; pure phase math is thread-safe.
|
||||
- A duplicated native animator resolves and duplicates its local Mesh, applies
|
||||
phase and deforms once before attachment. Its later `_ready()` is idempotent.
|
||||
- Preparation allocates an empty instance-local ArrayMesh because deformation
|
||||
immediately rebuilds every surface from retained native arrays. Captured
|
||||
immutable Material resources remain shared and are reapplied.
|
||||
|
||||
## Errors, cancellation and recovery
|
||||
|
||||
@@ -168,7 +173,8 @@ and material versions are unchanged; no rebake is required.
|
||||
|
||||
- `verify_m2_animation_playback_controller.gd` covers exact phase, ordinary/
|
||||
fish/bird priorities, substring/first fallback, loop/play/seek, native exact-
|
||||
script order, five-field copy, phase, detached diagnostics and boundaries.
|
||||
script order, five-field copy, single-rebuild phased preparation, idempotent
|
||||
ready, detached diagnostics and boundaries.
|
||||
- Finalizer/build/prototype/material/shutdown regressions protect adjacent behavior.
|
||||
- Fidelity evidence is exact policy/mutation extraction; no private asset or
|
||||
original-client animation comparison is claimed.
|
||||
@@ -192,7 +198,8 @@ for world doodads and compatibility fixtures.
|
||||
|
||||
- Hash phase intentionally depends on existing Godot String hashing behavior.
|
||||
- Default-name heuristics are not a complete WoW animation-state mapping.
|
||||
- No proprietary traversal, animation timing comparison, p95/p99 or paired-client run exists.
|
||||
- Native CPU deformation remains proportional to vertex count and is unsuitable
|
||||
for large numbers of independently animated instances without a future GPU path.
|
||||
|
||||
## Source map
|
||||
|
||||
|
||||
@@ -0,0 +1,183 @@
|
||||
# Renderer Closeout Verification
|
||||
|
||||
## Metadata
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Status | Implemented |
|
||||
| Target/work package | M03 / `M03-QAR-INTEGRATOR-CLOSEOUT-001` |
|
||||
| Owner | Renderer structural, cache and performance acceptance gates |
|
||||
| Last verified | Worktree `work/sindo-main-codex-m03-integrator/m03-closeout`, 2026-08-02 |
|
||||
| Profile | `Blizzlike335`; quality preset `High` for checkpoint evidence |
|
||||
|
||||
## Purpose
|
||||
|
||||
Provide reproducible closeout checks for the M03 renderer decomposition. The
|
||||
checks prove that worker boundaries remain CPU-only, main-thread frame steps are
|
||||
budgeted, cache versions remain explicit, and paired M00/M03 reports stay within
|
||||
the agreed 10% performance budgets.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Claim pixel-level parity with the original build-12340 client.
|
||||
- Replace subsystem unit verifiers or long-traversal release tests.
|
||||
- Generate, mutate or migrate production cache payloads.
|
||||
- Hide incompatible environments or cache inventories by normalizing reports.
|
||||
|
||||
## Context and boundaries
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
M00[M00 commit capture] --> Reports[Render checkpoint reports]
|
||||
M03[M03 commit capture] --> Reports
|
||||
Reports --> Comparator[compare_render_performance.ps1]
|
||||
Sources[Renderer source and cache versions] --> Contracts[verify_renderer_closeout_contracts.gd]
|
||||
Comparator --> Stability[repeatability gate across short and long windows]
|
||||
Stability --> Evidence[JSON comparison and exit code]
|
||||
Contracts --> Evidence
|
||||
Evidence --> Target[M03 Evidence / DONE decision]
|
||||
```
|
||||
|
||||
The capture command owns SceneTree execution and PNG/report writes. The
|
||||
PowerShell comparator is read-only except for its requested JSON output. The
|
||||
GDScript contract verifier reads source files and creates no renderer resources.
|
||||
|
||||
## Public API
|
||||
|
||||
| Symbol | Kind | Purpose | Preconditions | Failure |
|
||||
|---|---|---|---|---|
|
||||
| `compare_render_performance.ps1 -BaselineReport <paths> -CandidateReport <paths> [-OutputReport <path>]` | CLI | Compare one report or median of repeated reports | Same schema, profile, environment, cache contract/inventory and result keys | Exit 1 and enumerate incompatible fields or budget regressions |
|
||||
| `verify_render_performance_stability.ps1 -RepeatedSampleComparison <path> -LongWindowComparison <path>` | CLI | Reject only a metric regression reproduced by both independent protocols | Both comparator reports contain the same 84 metrics | Exit 1 with every repeatable result/metric key |
|
||||
| `verify_renderer_closeout_contracts.gd` | Godot CLI | Check worker/main-thread/cache/converter source contracts | Project parses and referenced sources exist | Exit 1 with named contract failure |
|
||||
|
||||
## Inputs and outputs
|
||||
|
||||
| Direction | Contract/data | Producer | Consumer | Ownership | Lifetime |
|
||||
|---|---|---|---|---|---|
|
||||
| Input | M00/M03 `report.json` paths | `capture_render_checkpoints.gd` | Comparator | Filesystem-owned immutable evidence | One comparison |
|
||||
| Input | Renderer/tool GDScript sources | Repository | Contract verifier | Read-only | One verifier run |
|
||||
| Output | 84 metric comparisons and failures | Comparator | Integrator/CI | Optional JSON plus process output | Evidence retention |
|
||||
| Output | Repeatable and protocol-local regression inventories | Stability gate | Integrator/CI | Optional JSON plus process output | Evidence retention |
|
||||
| Output | Structural pass/fail summary | Contract verifier | Integrator/CI | Process output | One run |
|
||||
|
||||
## Data flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Read[Read every supplied report] --> Compatible{Metadata and inventory match?}
|
||||
Compatible -->|no| Fail[Exit 1 with exact mismatch]
|
||||
Compatible -->|yes| Median[Median each metric per checkpoint/pass]
|
||||
Median --> Pair[Pair 14 result keys]
|
||||
Pair --> Budget[Compare load, p95, p99, hitch and memory]
|
||||
Budget --> Json[Optional comparison JSON]
|
||||
Budget --> Cohorts[Repeated-sample and long-window comparisons]
|
||||
Cohorts --> Repeated{Same metric fails both?}
|
||||
Repeated -->|no| Pass[Exit 0 with protocol-local diagnostics]
|
||||
Repeated -->|yes| Fail
|
||||
```
|
||||
|
||||
## Lifecycle and sequence
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant I as Integrator
|
||||
participant B as M00 worktree
|
||||
participant C as M03 worktree
|
||||
participant R as Checkpoint capture
|
||||
participant G as Performance comparator
|
||||
I->>B: capture repeated baseline samples
|
||||
B->>R: same viewport, driver and cache inventory
|
||||
I->>C: capture repeated candidate samples
|
||||
C->>R: same viewport, driver and cache inventory
|
||||
I->>G: baseline paths plus candidate paths
|
||||
G->>G: compatibility checks and per-metric medians
|
||||
G-->>I: short-window and long-window JSON evidence
|
||||
I->>G: verify stability across both protocols
|
||||
G-->>I: repeatable-regression pass/fail
|
||||
```
|
||||
|
||||
There is no persistent state machine. Each invocation is read, validate,
|
||||
aggregate, compare and terminate.
|
||||
|
||||
## Ownership, threading and resources
|
||||
|
||||
- Reports and source files are borrowed read-only for one process.
|
||||
- Median aggregation deep-copies the first report and never rewrites inputs.
|
||||
- Rendering remains owned by the GUI capture process on Godot's main thread.
|
||||
- The contract verifier is headless and does not instantiate the streaming world.
|
||||
- The optional comparison report is wholly owned by the caller-selected path.
|
||||
|
||||
## Errors, cancellation and recovery
|
||||
|
||||
| Failure | Behavior | Recovery |
|
||||
|---|---|---|
|
||||
| Missing/malformed report | Terminate with path/shape error | Regenerate that capture |
|
||||
| Environment/cache mismatch | Fail before accepting metrics | Recapture both commits on the same machine and cache |
|
||||
| Missing/duplicate checkpoint | Fail result-key validation | Repair manifest/capture completeness |
|
||||
| Metric over budget | Record baseline, candidate, limit and percentage | Diagnose named checkpoint/lane; rerun only after a code or evidence correction |
|
||||
| Interrupted GUI capture | No complete report is accepted | Remove/ignore partial output and rerun |
|
||||
|
||||
## Configuration and capabilities
|
||||
|
||||
The comparator reads thresholds from the baseline report. M03 uses 10% maximum
|
||||
regression for load time, frame p95, frame p99, maximum hitch, static memory and
|
||||
video memory. Repeated input paths are optional; when supplied, each side is
|
||||
reduced independently to the median for every metric.
|
||||
|
||||
Closeout uses two independent protocols: repeated three-second captures and a
|
||||
ten-second measurement window. A regression is accepted as real only when the
|
||||
same checkpoint/pass/metric exceeds its unchanged 10% budget in both protocols.
|
||||
Protocol-local failures remain in the JSON as noise diagnostics rather than
|
||||
being discarded.
|
||||
|
||||
## Persistence, cache and migrations
|
||||
|
||||
The comparison JSON uses schema version 1 and contains source paths, sample
|
||||
counts, revisions, all metric pairs and failures. It is evidence, not a runtime
|
||||
cache. Renderer cache versions are read from the manifest/source contracts; this
|
||||
module performs no migration or invalidation.
|
||||
|
||||
## Diagnostics and observability
|
||||
|
||||
- Success reports result pairs, comparison count and budget percentage.
|
||||
- Failure output names every checkpoint/pass, metric, values and limit.
|
||||
- The structural verifier reports worker count, frame-step count and cache-version count.
|
||||
- Capture reports retain PNG hashes, queue snapshots, environment and cache inventory.
|
||||
|
||||
## Verification, fidelity and performance
|
||||
|
||||
- `verify_renderer_closeout_contracts.gd` covers four worker boundaries, fifteen
|
||||
frame steps, seven cache versions and the nested M2 GLB output contract.
|
||||
- `compare_render_performance.ps1` compares 14 cold/warm result pairs and 84 metrics.
|
||||
- `verify_render_performance_stability.ps1` requires metric-key agreement and
|
||||
rejects any budget regression reproduced by both sampling protocols.
|
||||
- M00 and M03 must be captured from their exact commits against the same cache
|
||||
inventory; old reports with a different inventory are rejected.
|
||||
- PNG hashes and asset-backed coverage prove that terrain, ADT boundaries, dense
|
||||
M2, large WMO, liquid, animated M2 and sky were rendered. They do not prove
|
||||
original-client pixel parity without human/reference-image approval.
|
||||
|
||||
## Extension points
|
||||
|
||||
CI may retain reports and comparison JSON as artifacts. A future release gate may
|
||||
add driver-version metadata, long-traversal samples or approved visual-diff
|
||||
thresholds without changing runtime renderer contracts.
|
||||
|
||||
## Known gaps and risks
|
||||
|
||||
- Godot reports the rendering API but not the installed NVIDIA driver version.
|
||||
- A 0.5-second historical M00 measurement window requires repeated median samples.
|
||||
- Original-client screenshots are not part of the repository evidence set.
|
||||
- Long-traversal descriptor pressure remains a later quality/release gate.
|
||||
|
||||
## Source map
|
||||
|
||||
| Path | Responsibility |
|
||||
|---|---|
|
||||
| `tools/compare_render_performance.ps1` | Compatibility, median aggregation and metric budgets |
|
||||
| `tools/verify_render_performance_stability.ps1` | Cross-protocol repeatability acceptance |
|
||||
| `src/tools/verify_renderer_closeout_contracts.gd` | Structural source/cache/converter contracts |
|
||||
| `src/tools/capture_render_checkpoints.gd` | Asset-backed GUI capture and report generation |
|
||||
| `src/tools/render_baseline_manifest.json` | Coverage, viewport, cache contract and budgets |
|
||||
| `targets/00-render-baseline.md` | Accepted M00 measurement protocol |
|
||||
| `targets/03-renderer-facade.md` | M03 acceptance and Evidence |
|
||||
@@ -785,6 +785,9 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
|
||||
| `src/native/src/*_loader.cpp` | Native binary parsing |
|
||||
| `src/tools/build_*cache.gd`, `src/tools/bake_*cache.gd` | Offline cache generation |
|
||||
| `tools/run_render_baseline.ps1` | Unified M00 baseline runner |
|
||||
| `tools/compare_render_performance.ps1` | Exact-environment single/repeated report comparator |
|
||||
| `tools/verify_render_performance_stability.ps1` | Repeated-sample/long-window repeatability gate |
|
||||
| `src/tools/verify_renderer_closeout_contracts.gd` | Worker, main-thread, cache-version and nested-GLB closeout contracts |
|
||||
| `src/tools/compare_render_checkpoints.gd` | Offline JPG/PNG paired-image perceptual metrics and JSON pass/fail report |
|
||||
| `src/tools/verify_render_runtime_cache_shutdown.gd` | Headless ownership regression for detached runtime prototypes, resource caches and empty liquid roots |
|
||||
| `src/tools/capture_render_checkpoints.gd` | Deterministic no-roll checkpoint camera, performance and visual capture |
|
||||
|
||||
Reference in New Issue
Block a user