sindoring/open-wc
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

test(M00): complete animation and sky evidence

Browse Source 
Work-Package: M00-QAR-ANIMATION-SKY-EVIDENCE-001
Agent: sindo-main-codex
Tests: 14-pair visual audit; manifest/self-test; coordination/documentation gates
Fidelity: sky evidence paired approximately; Gryphon animation observed but placement-unpaired
This commit is contained in:
sindoring
2026-07-13 10:01:19 +04:00
parent 239cb2432e
commit 0327e4862d
3 changed files with 121 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files
coordination/claims/M00-QAR-ANIMATION-SKY-EVIDENCE-001.md
+78
View File
@@ -0,0 +1,78 @@
# M00-QAR-ANIMATION-SKY-EVIDENCE-001 — Animation and sky evidence
<!-- OPENWC_CLAIM:M00-QAR-ANIMATION-SKY-EVIDENCE-001:sindo-main-codex:2026-07-15 -->
## Ownership
- Target: M00
- Program: QAR
- Owner/Agent ID: sindo-main-codex
- Branch: `work/sindo-main-codex/m00-animation-sky-evidence`
- Lease expires UTC: 2026-07-15
- Integrator: milestone integrator
## Outcome
Add private build 12340 evidence for the remaining sky-transition and native animated-M2 coverage, rerun all paired comparisons, and hand the resulting audit to the milestone integrator.
## Non-goals
- Committing proprietary screenshots or extracted assets.
- Claiming renderer parity from mismatched world placement or animation phase.
- Marking M00 DONE; only the milestone integrator may change target status or Evidence.
## Paths
- Exclusive: this claim and animation/sky evidence documentation
- Shared/hotspots: `docs/M00_FINAL_PAIRED_EVIDENCE.md`, `docs/modules/world-renderer.md`
- Generated/ignored: normalized private references under `sources/OpenWCReferenceCheckpoints`; captures and reports under `user://`
## Contracts and data
- Public API/events: unchanged
- Schema/format version: unchanged
- Migration/compatibility: existing five-reference comparison remains valid
- Consumers: M00 milestone integrator
## Dependencies
- Requires: user-captured build 12340 screenshots at 19:00 and two real `GryphonRoost01` animation phases
- Blocks: final integrator decision on M00 completion
- External state: private screenshots remain outside Git
## Verification
- Commands: private-reference preparation, full baseline runner, comparator self-test, coordination and documentation gates
- Fixtures: seven private build 12340 checkpoint references plus a second animation-phase provenance frame
- Fidelity evidence: sky-transition screenshot and observed wing pose change over three seconds
- Performance budget: existing M00 capture budgets
## Documentation deliverables
- Inline public API docs: unchanged
- Module specification: paired coverage and remaining limitations
- Data-flow diagram: unchanged
- Sequence/state/dependency diagrams: unchanged
- Source map/status updates: final paired evidence audit
## Simplicity and naming
- Important names introduced: none
- Simplest considered solution: normalize private captures outside Git and reuse the existing comparator
- Rejected complexity/abstractions: adding a video comparator or committing an asset corpus
- Unavoidable complexity and justification: animation phase and synthetic placement require human evidence classification alongside pixel metrics
- Measured optimization evidence: not applicable
## Status
- State: ready-for-review
- Done: exact-size private references registered; real ADT placement verified; canonical and visual captures completed; fourteen comparisons audited
- Next: milestone integrator reviews synthetic/real placement classification and GUI RID risk
- Blocked by:
## Handoff
- Commit: this work-package commit
- Results: `compared=14`, `failed=14`, `missing=0`; mean error `0.066893..0.176955`; original-client Gryphon wing phase changed over three seconds
- Remaining risks: synthetic probe placement differs from the original-client world placement; GUI RID/resource diagnostics remain separate
- Documentation updated: `docs/M00_FINAL_PAIRED_EVIDENCE.md`, `docs/modules/world-renderer.md`
docs/M00_FINAL_PAIRED_EVIDENCE.md
+41 -27
View File
@@ -1,60 +1,74 @@
# M00 Final Paired Evidence — 2026-07-12 # M00 Final Paired Evidence — 2026-07-13
## Scope and result ## Scope and result
The final local M00 audit used Godot 4.6.1, the existing cache state, the manifest-standard 8-second readiness wait, and `2560x1440` visual candidates. Five private WoW 3.3.5a build 12340 references produced ten cold/warm comparisons with no missing pair. Private images and generated reports remain outside Git. The final local M00 audit used Godot 4.6.1, the existing cache state, the manifest-standard 8-second readiness wait, and `2560x1440` visual candidates. Seven private WoW 3.3.5a build 12340 references produced fourteen cold/warm comparisons with no missing pair. Private images and generated reports remain outside Git.
The command completed even though every pair exceeded the strict regression tolerance. Comparator exit code `1` means a valid report with visual gaps; only codes outside `0/1` are infrastructure failures. Every pair exceeded the strict regression tolerance. Comparator exit code `1` means a valid report with measured visual gaps; only codes outside `0/1` are infrastructure failures. This evidence establishes coverage and current differences, not renderer parity.
```powershell ```powershell
.\tools\run_render_baseline.ps1 ` .\tools\run_render_baseline.ps1 `
-Output user://render_baseline_m00_final_2026-07-12 ` -Output user://render_baseline_m00_complete_2026-07-13 `
-CacheState existing ` -CacheState existing `
-WaitSeconds 8 ` -WaitSeconds 8 `
-MeasureSeconds 0.5 ` -MeasureSeconds 0.5 `
-ReferenceCheckpointDirectory .\sources\OpenWCReferenceCheckpoints ` -ReferenceCheckpointDirectory .\sources\OpenWCReferenceCheckpoints `
-VisualComparisonReport user://render_baseline_m00_final_2026-07-12/visual_comparison.json ` -VisualComparisonReport user://render_baseline_m00_complete_2026-07-13/visual_comparison.json `
-VisualComparisonWidth 2560 ` -VisualComparisonWidth 2560 `
-VisualComparisonHeight 1440 -VisualComparisonHeight 1440
``` ```
The initial shell timeout detached the canonical capture process after it started. The process completed all fourteen canonical captures and wrote `report.json`; the documented visual-capture and comparator stages were then invoked separately with the same arguments. The final output directory contains both complete reports.
## Paired metrics ## Paired metrics
| Checkpoint | Pass | Mean perceptual error | Changed-pixel ratio | Classification | | Checkpoint | Pass | Mean perceptual error | Changed-pixel ratio | Classification |
|---|---|---:|---:|---| |---|---|---:|---:|---|
| `elwynn_adt_boundary` | cold | 0.087347 | 0.634984 | Composition plus terrain/vegetation/material gaps | | `elwynn_adt_boundary` | cold | 0.087258 | 0.634580 | Composition plus terrain/vegetation/material gaps |
| `elwynn_adt_boundary` | warm | 0.087255 | 0.634598 | Stable against revisit | | `elwynn_adt_boundary` | warm | 0.087255 | 0.634598 | Stable against revisit |
| `elwynn_terrain_overview` | cold | 0.174136 | 0.851806 | Largest composition/framing gap | | `elwynn_sky_dusk` | cold | 0.094778 | 0.575956 | Approximate paired camera; sky color, lighting and scene-composition gaps |
| `elwynn_terrain_overview` | warm | 0.176952 | 0.861275 | Stable gap, not a cold-load artifact | | `elwynn_sky_dusk` | warm | 0.094757 | 0.575898 | Stable against revisit |
| `elwynn_waterfall_liquid` | cold | 0.088526 | 0.706897 | Framing, terrain, liquid and lighting gaps | | `elwynn_terrain_overview` | cold | 0.174137 | 0.851791 | Largest composition/framing gap |
| `elwynn_waterfall_liquid` | warm | 0.087490 | 0.695281 | Stable against revisit | | `elwynn_terrain_overview` | warm | 0.176955 | 0.861271 | Stable gap, not a cold-load artifact |
| `goldshire_dense_m2` | cold | 0.066892 | 0.575205 | Smallest available paired gap; vegetation/material differences remain | | `elwynn_waterfall_liquid` | cold | 0.088629 | 0.706522 | Framing, terrain, liquid and lighting gaps |
| `goldshire_dense_m2` | warm | 0.066927 | 0.575547 | Stable against revisit | | `elwynn_waterfall_liquid` | warm | 0.087398 | 0.693690 | Stable against revisit |
| `goldshire_inn_large_wmo` | cold | 0.101230 | 0.701836 | WMO is ready; manual framing and presentation gaps remain | | `goldshire_dense_m2` | cold | 0.066893 | 0.575257 | Smallest available paired gap; vegetation/material differences remain |
| `goldshire_inn_large_wmo` | warm | 0.101203 | 0.701855 | Stable against revisit | | `goldshire_dense_m2` | warm | 0.066927 | 0.575592 | Stable against revisit |
| `goldshire_inn_large_wmo` | cold | 0.101224 | 0.701835 | WMO is ready; manual framing and presentation gaps remain |
| `goldshire_inn_large_wmo` | warm | 0.101206 | 0.701855 | Stable against revisit |
| `gryphon_roost_native_animation` | cold | 0.119467 | 0.882073 | Same build-12340 asset, different real/synthetic placement and uncontrolled animation phase |
| `gryphon_roost_native_animation` | warm | 0.119471 | 0.882076 | Full-frame metric is diagnostic only; animation observation is human evidence |
Cold and warm values are close for all five references. The paired gaps are therefore not explained by first-visit cache readiness. Goldshire timing evidence separately demonstrated that the 8-second wait is necessary for WMO publication. Cold and warm values are close for all seven references. The measured gaps are therefore not explained by first-visit cache readiness. Goldshire timing evidence separately demonstrated that the 8-second wait is necessary for WMO publication.
## Animation and sky provenance
The private source corpus preserves exact screenshot bytes; no resize, crop or generated content was used:
| Evidence | SHA-256 | Interpretation |
|---|---|---|
| `elwynn_sky_dusk.jpg` | `1fb06836cacae714c76eaf9b8bf7c292fd5e2f2df33eef821bfe4c4a6bd97072` | Build 12340, Elwynn camera position, server time approximately 19:00, UI hidden |
| `gryphon_roost_native_animation.jpg` | `d3488c741cb8dfe7212198e48332842631a951862a4388b67045338f37357447` | Build 12340 real-world `GryphonRoost01` placement, first visible wing phase |
| diagnostic animation phase B | `1025572c7ab9fa0d2e500ff026be46eb9c1e6e5447116dbdc3675dd5f44574ab` | Same camera three seconds later; visibly raised wing phase |
| diagnostic placement context | `b12c0b713a7d75123ed30f5a316b33537331442cd56615aea69214c7cc4e7a42` | Two real Lakeshire placements and surrounding build-12340 scene |
An ADT scan found no `GryphonRoost01` placement in Goldshire. The M00 checkpoint deliberately injects the model there as a synthetic probe. The original-client observation therefore uses two real Lakeshire placements. The three-second pose change establishes `Observed` native animation behavior, but the full-frame comparison cannot establish spatial or animation-phase parity.
## Fidelity interpretation ## Fidelity interpretation
M00 now has reproducible quantitative evidence for terrain, ADT boundary, dense M2, large WMO and liquid checkpoints. The reports demonstrate current differences; they do not claim renderer parity. Camera calibration experiments also show that unknown manual yaw/pitch/FOV cannot be recovered reliably from full-frame color error alone. Region scoring reduces terrain bias but still requires semantic landmark approval. M00 now has private build 12340 evidence for every required coverage category: terrain, ADT boundary, dense M2, large WMO, liquid, animated M2 and sky transition. The sky reference is approximately paired by map position/time and manually approved composition. Animated-M2 evidence is asset-paired but placement-unpaired because the OpenWC probe is synthetic.
Two manifest checkpoints still lack original-client paired references: The reports demonstrate current differences and do not claim `1:1`. Camera calibration experiments also show that unknown manual yaw/pitch/FOV cannot be recovered reliably from full-frame color error alone. Region scoring reduces terrain bias but still requires semantic landmark approval.
- `gryphon_roost_native_animation` is a synthetic native-animation probe and has no matching build 12340 fixture.
- `elwynn_sky_dusk` has no approved build 12340 sky-transition fixture at the same time/weather state.
These must either receive private reference evidence or be explicitly accepted by the milestone integrator as unpaired diagnostic coverage. Until that decision, the target remains `ACTIVE`.
## Operational risks ## Operational risks
The full GUI capture completed and wrote both reports, but Godot still emitted renderer RID/resource leak diagnostics at exit. The earlier anonymous headless `RefCounted` leak is fixed; GUI renderer allocations are a distinct known risk. They do not invalidate the written images or metrics, but must remain recorded rather than hidden. The full GUI visual capture completed and wrote its report, but Godot still emitted renderer RID/resource leak diagnostics at exit, including leaked material, shader, texture, instance and rendering-device RIDs. The earlier anonymous headless `RefCounted` leak is fixed; these GUI renderer allocations are a distinct known risk. They do not invalidate the written images or metrics, but must remain recorded rather than hidden.
## Integrator decision checklist ## Integrator decision checklist
- Five available reference checkpoints: paired cold/warm report complete, missing pairs zero. - Seven required reference checkpoints: cold/warm report complete, `compared=14`, `missing=0`.
- Animated M2 fixture: missing or explicitly accepted as synthetic-only. - Animated M2: build-12340 native motion observed in two phases; synthetic/real placement mismatch explicitly classified.
- Sky-transition fixture: missing or explicitly accepted as unpaired. - Sky transition: build-12340 19:00 evidence present with approximate paired camera and human-approved composition.
- Renderer visual differences: registered as gaps, not parity claims. - Renderer visual differences: registered as gaps, not parity claims.
- GUI renderer RID/resource diagnostics: accepted as a tracked risk or assigned a follow-up blocker. - GUI renderer RID/resource diagnostics: accepted as a tracked risk or assigned a follow-up blocker.
- Target status and `OPENWC_TARGET_DONE` marker: integrator-only action. - Target status, target Evidence and `OPENWC_TARGET_DONE` marker: integrator-only actions.
docs/modules/world-renderer.md
+2 -2
View File
@@ -197,7 +197,7 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
- Unit/contract tests: material mapping, unique-ID dedupe, placement probes, baseline manifest, five-point coordinate calibration, synthetic perceptual checkpoint diff, camera-pose grid plan. - Unit/contract tests: material mapping, unique-ID dedupe, placement probes, baseline manifest, five-point coordinate calibration, synthetic perceptual checkpoint diff, camera-pose grid plan.
- Integration/E2E: Eastern Kingdoms/Kalimdor streaming scenes and seven cold/warm checkpoints. - Integration/E2E: Eastern Kingdoms/Kalimdor streaming scenes and seven cold/warm checkpoints.
- Fidelity evidence: пять локальных build 12340 reference JPG откалибровали terrain/ADT/M2/WMO/liquid viewpoints; automated paired-image metrics exist, но synthetic animation/dusk и полный human approval ещё не закрыты. - Fidelity evidence: семь локальных build 12340 reference JPG покрывают terrain/ADT/M2/WMO/liquid/sky и реальный `GryphonRoost01`; automated cold/warm metrics имеют `compared=14`, `missing=0`. Sky использует приблизительно paired camera, а animated-M2 остаётся asset-paired/placement-unpaired из-за synthetic probe.
- Performance budgets: M00 report records cold/warm p95 and max hitch; no final acceptance threshold yet. - Performance budgets: M00 report records cold/warm p95 and max hitch; no final acceptance threshold yet.
- Manual diagnostics: [`../RENDER_BASELINE.md`](../RENDER_BASELINE.md) and [`../../RENDER.md`](../../RENDER.md). - Manual diagnostics: [`../RENDER_BASELINE.md`](../RENDER_BASELINE.md) and [`../../RENDER.md`](../../RENDER.md).
@@ -234,7 +234,7 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
- A ready-scene Goldshire 3x3 grid improved mean error from `0.101402` at zero offsets to `0.087952` at yaw `10`/pitch `-10`; pitch/FOV refinement remains required before manifest calibration. - A ready-scene Goldshire 3x3 grid improved mean error from `0.101402` at zero offsets to `0.087952` at yaw `10`/pitch `-10`; pitch/FOV refinement remains required before manifest calibration.
- Joint Goldshire pitch/FOV refinement had no human-approved optimum: full-frame error preferred grass-heavy FOV 62/pitch -25, while FOV 38/pitch -10 better matched building scale. Landmark/region scoring is required before calibration. - Joint Goldshire pitch/FOV refinement had no human-approved optimum: full-frame error preferred grass-heavy FOV 62/pitch -25, while FOV 38/pitch -10 better matched building scale. Landmark/region scoring is required before calibration.
- Common-region scoring `[300,100,1400,650]` moved Goldshire's numeric optimum to FOV 38/yaw 10/pitch -15, but human inspection still found an oversized/cropped facade; semantic landmarks or masks remain required. - Common-region scoring `[300,100,1400,650]` moved Goldshire's numeric optimum to FOV 38/yaw 10/pitch -15, but human inspection still found an oversized/cropped facade; semantic landmarks or masks remain required.
- Final paired audit compared ten cold/warm images for five private build 12340 references with zero missing pairs; mean error ranges `0.066892..0.176952`. Animated-M2 and sky-transition references remain an integrator acceptance decision. - Final paired audit compared fourteen cold/warm images for seven private build 12340 references with zero missing pairs; mean error ranges `0.066893..0.176955`. Animated-M2 native motion is observed across two original-client phases, but its real Lakeshire placement differs from the synthetic Goldshire probe. Sky-transition evidence uses the checkpoint map position/time with manually approved approximate framing.
- D3D12 descriptor issues remain; the capture-path anonymous `RefCounted` shutdown leak is regression-covered by a clean verbose dry-run, while other RID/resource diagnostics still require independent evidence. - D3D12 descriptor issues remain; the capture-path anonymous `RefCounted` shutdown leak is regression-covered by a clean verbose dry-run, while other RID/resource diagnostics still require independent evidence.
- M2/WMO/material/particle/ribbon/portal parity incomplete. - M2/WMO/material/particle/ribbon/portal parity incomplete.
- Public API is mostly exported configuration rather than stable contracts. - Public API is mostly exported configuration rather than stable contracts.