Compare commits

..

4 Commits

Author SHA1 Message Date
sindoring 0327e4862d test(M00): complete animation and sky evidence
Work-Package: M00-QAR-ANIMATION-SKY-EVIDENCE-001
Agent: sindo-main-codex
Tests: 14-pair visual audit; manifest/self-test; coordination/documentation gates
Fidelity: sky evidence paired approximately; Gryphon animation observed but placement-unpaired
2026-07-13 10:01:19 +04:00
sindoring 239cb2432e Merge pull request 'test(M00): record final paired evidence' (#13) from work/sindo-main-codex/m00-final-paired-evidence into master
Reviewed-on: #13
2026-07-12 02:28:00 +03:00
sindoring b5ddc86c92 test(M00): record final paired evidence 2026-07-12 03:21:39 +04:00
sindoring 3d90c6d79a Merge pull request 'test(M00): add landmark-region visual diff' (#12) from work/sindo-main-codex/m00-landmark-region-diff into master
Reviewed-on: #12
2026-07-12 02:07:30 +03:00
5 changed files with 243 additions and 2 deletions
@@ -0,0 +1,78 @@
# M00-QAR-ANIMATION-SKY-EVIDENCE-001 — Animation and sky evidence
<!-- OPENWC_CLAIM:M00-QAR-ANIMATION-SKY-EVIDENCE-001:sindo-main-codex:2026-07-15 -->
## Ownership
- Target: M00
- Program: QAR
- Owner/Agent ID: sindo-main-codex
- Branch: `work/sindo-main-codex/m00-animation-sky-evidence`
- Lease expires UTC: 2026-07-15
- Integrator: milestone integrator
## Outcome
Add private build 12340 evidence for the remaining sky-transition and native animated-M2 coverage, rerun all paired comparisons, and hand the resulting audit to the milestone integrator.
## Non-goals
- Committing proprietary screenshots or extracted assets.
- Claiming renderer parity from mismatched world placement or animation phase.
- Marking M00 DONE; only the milestone integrator may change target status or Evidence.
## Paths
- Exclusive: this claim and animation/sky evidence documentation
- Shared/hotspots: `docs/M00_FINAL_PAIRED_EVIDENCE.md`, `docs/modules/world-renderer.md`
- Generated/ignored: normalized private references under `sources/OpenWCReferenceCheckpoints`; captures and reports under `user://`
## Contracts and data
- Public API/events: unchanged
- Schema/format version: unchanged
- Migration/compatibility: existing five-reference comparison remains valid
- Consumers: M00 milestone integrator
## Dependencies
- Requires: user-captured build 12340 screenshots at 19:00 and two real `GryphonRoost01` animation phases
- Blocks: final integrator decision on M00 completion
- External state: private screenshots remain outside Git
## Verification
- Commands: private-reference preparation, full baseline runner, comparator self-test, coordination and documentation gates
- Fixtures: seven private build 12340 checkpoint references plus a second animation-phase provenance frame
- Fidelity evidence: sky-transition screenshot and observed wing pose change over three seconds
- Performance budget: existing M00 capture budgets
## Documentation deliverables
- Inline public API docs: unchanged
- Module specification: paired coverage and remaining limitations
- Data-flow diagram: unchanged
- Sequence/state/dependency diagrams: unchanged
- Source map/status updates: final paired evidence audit
## Simplicity and naming
- Important names introduced: none
- Simplest considered solution: normalize private captures outside Git and reuse the existing comparator
- Rejected complexity/abstractions: adding a video comparator or committing an asset corpus
- Unavoidable complexity and justification: animation phase and synthetic placement require human evidence classification alongside pixel metrics
- Measured optimization evidence: not applicable
## Status
- State: ready-for-review
- Done: exact-size private references registered; real ADT placement verified; canonical and visual captures completed; fourteen comparisons audited
- Next: milestone integrator reviews synthetic/real placement classification and GUI RID risk
- Blocked by:
## Handoff
- Commit: this work-package commit
- Results: `compared=14`, `failed=14`, `missing=0`; mean error `0.066893..0.176955`; original-client Gryphon wing phase changed over three seconds
- Remaining risks: synthetic probe placement differs from the original-client world placement; GUI RID/resource diagnostics remain separate
- Documentation updated: `docs/M00_FINAL_PAIRED_EVIDENCE.md`, `docs/modules/world-renderer.md`
@@ -0,0 +1,78 @@
# M00-QAR-FINAL-PAIRED-EVIDENCE-001 — Final paired evidence audit
<!-- OPENWC_CLAIM:M00-QAR-FINAL-PAIRED-EVIDENCE-001:sindo-main-codex:2026-07-14 -->
## Ownership
- Target: M00
- Program: QAR
- Owner/Agent ID: sindo-main-codex
- Branch: `work/sindo-main-codex/m00-final-paired-evidence`
- Lease expires UTC: 2026-07-14
- Integrator: milestone integrator
## Outcome
Produce one final WMO-ready paired report for all available build 12340 references and reconcile the remaining M00 acceptance gaps.
## Non-goals
- Achieving renderer parity inside M00.
- Calibrating unknown manual camera parameters.
- Marking the target DONE; only the milestone integrator may do so.
## Paths
- Exclusive: paired-run exit handling and final evidence record
- Shared/hotspots: renderer baseline/module findings
- Generated/ignored: final captures and reports under `user://`
## Contracts and data
- Public API/events: visual comparison exit 0/1 both count as completed report generation
- Schema/format version: unchanged
- Migration/compatibility: comparator infrastructure errors remain fatal
- Consumers: M00 integrator
## Dependencies
- Requires: merged region diagnostic and five private reference images
- Blocks: integrator decision on M00 completion
- External state: private images remain outside Git
## Verification
- Commands: complete paired baseline run, report audit, repository gates
- Fixtures: five build 12340 references
- Fidelity evidence: cold/warm comparison metrics and human gap classification
- Performance budget: existing M00 capture budgets
## Documentation deliverables
- Inline public API docs: PowerShell comparison step semantics
- Module specification: final evidence status
- Data-flow diagram: unchanged
- Sequence/state/dependency diagrams: unchanged
- Source map/status updates: baseline acceptance summary
## Simplicity and naming
- Important names introduced: `Invoke-GodotComparisonStep`
- Simplest considered solution: allow only comparator exit codes 0 and 1
- Rejected complexity/abstractions: suppressing all process failures
- Unavoidable complexity and justification: threshold exceedance is evidence, not infrastructure failure
- Measured optimization evidence: not applicable
## Status
- State: ready-for-review
- Done: paired-run exit contract fixed; full baseline and visual capture completed; ten pairs audited; final acceptance document written
- Next: integrator decides animated/sky reference policy and GUI RID-risk disposition
- Blocked by:
## Handoff
- Commit: this work-package commit
- Results: compared=10, failed=10, missing=0; mean error 0.066892..0.176952; cold/warm gaps stable; full workflow returned success with threshold evidence preserved
- Remaining risks: no build 12340 animated-M2 or sky-transition reference; GUI renderer RID/resources still reported at exit
- Documentation updated: `docs/M00_FINAL_PAIRED_EVIDENCE.md`, `docs/modules/world-renderer.md`
+74
View File
@@ -0,0 +1,74 @@
# M00 Final Paired Evidence — 2026-07-13
## Scope and result
The final local M00 audit used Godot 4.6.1, the existing cache state, the manifest-standard 8-second readiness wait, and `2560x1440` visual candidates. Seven private WoW 3.3.5a build 12340 references produced fourteen cold/warm comparisons with no missing pair. Private images and generated reports remain outside Git.
Every pair exceeded the strict regression tolerance. Comparator exit code `1` means a valid report with measured visual gaps; only codes outside `0/1` are infrastructure failures. This evidence establishes coverage and current differences, not renderer parity.
```powershell
.\tools\run_render_baseline.ps1 `
-Output user://render_baseline_m00_complete_2026-07-13 `
-CacheState existing `
-WaitSeconds 8 `
-MeasureSeconds 0.5 `
-ReferenceCheckpointDirectory .\sources\OpenWCReferenceCheckpoints `
-VisualComparisonReport user://render_baseline_m00_complete_2026-07-13/visual_comparison.json `
-VisualComparisonWidth 2560 `
-VisualComparisonHeight 1440
```
The initial shell timeout detached the canonical capture process after it started. The process completed all fourteen canonical captures and wrote `report.json`; the documented visual-capture and comparator stages were then invoked separately with the same arguments. The final output directory contains both complete reports.
## Paired metrics
| Checkpoint | Pass | Mean perceptual error | Changed-pixel ratio | Classification |
|---|---|---:|---:|---|
| `elwynn_adt_boundary` | cold | 0.087258 | 0.634580 | Composition plus terrain/vegetation/material gaps |
| `elwynn_adt_boundary` | warm | 0.087255 | 0.634598 | Stable against revisit |
| `elwynn_sky_dusk` | cold | 0.094778 | 0.575956 | Approximate paired camera; sky color, lighting and scene-composition gaps |
| `elwynn_sky_dusk` | warm | 0.094757 | 0.575898 | Stable against revisit |
| `elwynn_terrain_overview` | cold | 0.174137 | 0.851791 | Largest composition/framing gap |
| `elwynn_terrain_overview` | warm | 0.176955 | 0.861271 | Stable gap, not a cold-load artifact |
| `elwynn_waterfall_liquid` | cold | 0.088629 | 0.706522 | Framing, terrain, liquid and lighting gaps |
| `elwynn_waterfall_liquid` | warm | 0.087398 | 0.693690 | Stable against revisit |
| `goldshire_dense_m2` | cold | 0.066893 | 0.575257 | Smallest available paired gap; vegetation/material differences remain |
| `goldshire_dense_m2` | warm | 0.066927 | 0.575592 | Stable against revisit |
| `goldshire_inn_large_wmo` | cold | 0.101224 | 0.701835 | WMO is ready; manual framing and presentation gaps remain |
| `goldshire_inn_large_wmo` | warm | 0.101206 | 0.701855 | Stable against revisit |
| `gryphon_roost_native_animation` | cold | 0.119467 | 0.882073 | Same build-12340 asset, different real/synthetic placement and uncontrolled animation phase |
| `gryphon_roost_native_animation` | warm | 0.119471 | 0.882076 | Full-frame metric is diagnostic only; animation observation is human evidence |
Cold and warm values are close for all seven references. The measured gaps are therefore not explained by first-visit cache readiness. Goldshire timing evidence separately demonstrated that the 8-second wait is necessary for WMO publication.
## Animation and sky provenance
The private source corpus preserves exact screenshot bytes; no resize, crop or generated content was used:
| Evidence | SHA-256 | Interpretation |
|---|---|---|
| `elwynn_sky_dusk.jpg` | `1fb06836cacae714c76eaf9b8bf7c292fd5e2f2df33eef821bfe4c4a6bd97072` | Build 12340, Elwynn camera position, server time approximately 19:00, UI hidden |
| `gryphon_roost_native_animation.jpg` | `d3488c741cb8dfe7212198e48332842631a951862a4388b67045338f37357447` | Build 12340 real-world `GryphonRoost01` placement, first visible wing phase |
| diagnostic animation phase B | `1025572c7ab9fa0d2e500ff026be46eb9c1e6e5447116dbdc3675dd5f44574ab` | Same camera three seconds later; visibly raised wing phase |
| diagnostic placement context | `b12c0b713a7d75123ed30f5a316b33537331442cd56615aea69214c7cc4e7a42` | Two real Lakeshire placements and surrounding build-12340 scene |
An ADT scan found no `GryphonRoost01` placement in Goldshire. The M00 checkpoint deliberately injects the model there as a synthetic probe. The original-client observation therefore uses two real Lakeshire placements. The three-second pose change establishes `Observed` native animation behavior, but the full-frame comparison cannot establish spatial or animation-phase parity.
## Fidelity interpretation
M00 now has private build 12340 evidence for every required coverage category: terrain, ADT boundary, dense M2, large WMO, liquid, animated M2 and sky transition. The sky reference is approximately paired by map position/time and manually approved composition. Animated-M2 evidence is asset-paired but placement-unpaired because the OpenWC probe is synthetic.
The reports demonstrate current differences and do not claim `1:1`. Camera calibration experiments also show that unknown manual yaw/pitch/FOV cannot be recovered reliably from full-frame color error alone. Region scoring reduces terrain bias but still requires semantic landmark approval.
## Operational risks
The full GUI visual capture completed and wrote its report, but Godot still emitted renderer RID/resource leak diagnostics at exit, including leaked material, shader, texture, instance and rendering-device RIDs. The earlier anonymous headless `RefCounted` leak is fixed; these GUI renderer allocations are a distinct known risk. They do not invalidate the written images or metrics, but must remain recorded rather than hidden.
## Integrator decision checklist
- Seven required reference checkpoints: cold/warm report complete, `compared=14`, `missing=0`.
- Animated M2: build-12340 native motion observed in two phases; synthetic/real placement mismatch explicitly classified.
- Sky transition: build-12340 19:00 evidence present with approximate paired camera and human-approved composition.
- Renderer visual differences: registered as gaps, not parity claims.
- GUI renderer RID/resource diagnostics: accepted as a tracked risk or assigned a follow-up blocker.
- Target status, target Evidence and `OPENWC_TARGET_DONE` marker: integrator-only actions.
+3 -1
View File
@@ -197,7 +197,7 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
- Unit/contract tests: material mapping, unique-ID dedupe, placement probes, baseline manifest, five-point coordinate calibration, synthetic perceptual checkpoint diff, camera-pose grid plan.
- Integration/E2E: Eastern Kingdoms/Kalimdor streaming scenes and seven cold/warm checkpoints.
- Fidelity evidence: пять локальных build 12340 reference JPG откалибровали terrain/ADT/M2/WMO/liquid viewpoints; automated paired-image metrics exist, но synthetic animation/dusk и полный human approval ещё не закрыты.
- Fidelity evidence: семь локальных build 12340 reference JPG покрывают terrain/ADT/M2/WMO/liquid/sky и реальный `GryphonRoost01`; automated cold/warm metrics имеют `compared=14`, `missing=0`. Sky использует приблизительно paired camera, а animated-M2 остаётся asset-paired/placement-unpaired из-за synthetic probe.
- Performance budgets: M00 report records cold/warm p95 and max hitch; no final acceptance threshold yet.
- Manual diagnostics: [`../RENDER_BASELINE.md`](../RENDER_BASELINE.md) and [`../../RENDER.md`](../../RENDER.md).
@@ -234,6 +234,7 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
- A ready-scene Goldshire 3x3 grid improved mean error from `0.101402` at zero offsets to `0.087952` at yaw `10`/pitch `-10`; pitch/FOV refinement remains required before manifest calibration.
- Joint Goldshire pitch/FOV refinement had no human-approved optimum: full-frame error preferred grass-heavy FOV 62/pitch -25, while FOV 38/pitch -10 better matched building scale. Landmark/region scoring is required before calibration.
- Common-region scoring `[300,100,1400,650]` moved Goldshire's numeric optimum to FOV 38/yaw 10/pitch -15, but human inspection still found an oversized/cropped facade; semantic landmarks or masks remain required.
- Final paired audit compared fourteen cold/warm images for seven private build 12340 references with zero missing pairs; mean error ranges `0.066893..0.176955`. Animated-M2 native motion is observed across two original-client phases, but its real Lakeshire placement differs from the synthetic Goldshire probe. Sky-transition evidence uses the checkpoint map position/time with manually approved approximate framing.
- D3D12 descriptor issues remain; the capture-path anonymous `RefCounted` shutdown leak is regression-covered by a clean verbose dry-run, while other RID/resource diagnostics still require independent evidence.
- M2/WMO/material/particle/ribbon/portal parity incomplete.
- Public API is mostly exported configuration rather than stable contracts.
@@ -263,5 +264,6 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
- [`../../RENDER.md`](../../RENDER.md)
- [`../RENDER_BASELINE.md`](../RENDER_BASELINE.md)
- [`../CAMERA_POSE_SWEEP.md`](../CAMERA_POSE_SWEEP.md)
- [`../M00_FINAL_PAIRED_EVIDENCE.md`](../M00_FINAL_PAIRED_EVIDENCE.md)
- [`../../targets/roadmap/02-rendering-and-graphics.md`](../../targets/roadmap/02-rendering-and-graphics.md)
- [`../GODOT_BEST_PRACTICES.md`](../GODOT_BEST_PRACTICES.md)
+10 -1
View File
@@ -40,6 +40,15 @@ function Invoke-GodotStep {
}
}
function Invoke-GodotComparisonStep {
param([string]$Name, [string[]]$Arguments)
Write-Host "[$Name] $GodotPath $($Arguments -join ' ')"
$process = Start-Process -FilePath $GodotPath -ArgumentList $Arguments -NoNewWindow -Wait -PassThru
if ($process.ExitCode -notin @(0, 1)) {
throw "$Name failed with exit code $($process.ExitCode)"
}
}
Push-Location $repoRoot
try {
Invoke-GodotStep "project-load" @("--headless", "--path", ".", "--quit")
@@ -80,7 +89,7 @@ try {
"--viewport-height", $VisualComparisonHeight.ToString()
)
Invoke-GodotStep "visual-capture" $visualCaptureArgs
Invoke-GodotStep "visual-comparison" @(
Invoke-GodotComparisonStep "visual-comparison" @(
"--headless", "--path", ".",
"--script", "res://src/tools/compare_render_checkpoints.gd",
"--",