Compare commits
7 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 0327e4862d | |||
| 239cb2432e | |||
| b5ddc86c92 | |||
| 3d90c6d79a | |||
| a10206b797 | |||
| 3172f41ee5 | |||
| 5a957d2ebb |
@@ -0,0 +1,78 @@
|
||||
# M00-QAR-ANIMATION-SKY-EVIDENCE-001 — Animation and sky evidence
|
||||
|
||||
<!-- OPENWC_CLAIM:M00-QAR-ANIMATION-SKY-EVIDENCE-001:sindo-main-codex:2026-07-15 -->
|
||||
|
||||
## Ownership
|
||||
|
||||
- Target: M00
|
||||
- Program: QAR
|
||||
- Owner/Agent ID: sindo-main-codex
|
||||
- Branch: `work/sindo-main-codex/m00-animation-sky-evidence`
|
||||
- Lease expires UTC: 2026-07-15
|
||||
- Integrator: milestone integrator
|
||||
|
||||
## Outcome
|
||||
|
||||
Add private build 12340 evidence for the remaining sky-transition and native animated-M2 coverage, rerun all paired comparisons, and hand the resulting audit to the milestone integrator.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Committing proprietary screenshots or extracted assets.
|
||||
- Claiming renderer parity from mismatched world placement or animation phase.
|
||||
- Marking M00 DONE; only the milestone integrator may change target status or Evidence.
|
||||
|
||||
## Paths
|
||||
|
||||
- Exclusive: this claim and animation/sky evidence documentation
|
||||
- Shared/hotspots: `docs/M00_FINAL_PAIRED_EVIDENCE.md`, `docs/modules/world-renderer.md`
|
||||
- Generated/ignored: normalized private references under `sources/OpenWCReferenceCheckpoints`; captures and reports under `user://`
|
||||
|
||||
## Contracts and data
|
||||
|
||||
- Public API/events: unchanged
|
||||
- Schema/format version: unchanged
|
||||
- Migration/compatibility: existing five-reference comparison remains valid
|
||||
- Consumers: M00 milestone integrator
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Requires: user-captured build 12340 screenshots at 19:00 and two real `GryphonRoost01` animation phases
|
||||
- Blocks: final integrator decision on M00 completion
|
||||
- External state: private screenshots remain outside Git
|
||||
|
||||
## Verification
|
||||
|
||||
- Commands: private-reference preparation, full baseline runner, comparator self-test, coordination and documentation gates
|
||||
- Fixtures: seven private build 12340 checkpoint references plus a second animation-phase provenance frame
|
||||
- Fidelity evidence: sky-transition screenshot and observed wing pose change over three seconds
|
||||
- Performance budget: existing M00 capture budgets
|
||||
|
||||
## Documentation deliverables
|
||||
|
||||
- Inline public API docs: unchanged
|
||||
- Module specification: paired coverage and remaining limitations
|
||||
- Data-flow diagram: unchanged
|
||||
- Sequence/state/dependency diagrams: unchanged
|
||||
- Source map/status updates: final paired evidence audit
|
||||
|
||||
## Simplicity and naming
|
||||
|
||||
- Important names introduced: none
|
||||
- Simplest considered solution: normalize private captures outside Git and reuse the existing comparator
|
||||
- Rejected complexity/abstractions: adding a video comparator or committing an asset corpus
|
||||
- Unavoidable complexity and justification: animation phase and synthetic placement require human evidence classification alongside pixel metrics
|
||||
- Measured optimization evidence: not applicable
|
||||
|
||||
## Status
|
||||
|
||||
- State: ready-for-review
|
||||
- Done: exact-size private references registered; real ADT placement verified; canonical and visual captures completed; fourteen comparisons audited
|
||||
- Next: milestone integrator reviews synthetic/real placement classification and GUI RID risk
|
||||
- Blocked by:
|
||||
|
||||
## Handoff
|
||||
|
||||
- Commit: this work-package commit
|
||||
- Results: `compared=14`, `failed=14`, `missing=0`; mean error `0.066893..0.176955`; original-client Gryphon wing phase changed over three seconds
|
||||
- Remaining risks: synthetic probe placement differs from the original-client world placement; GUI RID/resource diagnostics remain separate
|
||||
- Documentation updated: `docs/M00_FINAL_PAIRED_EVIDENCE.md`, `docs/modules/world-renderer.md`
|
||||
@@ -0,0 +1,78 @@
|
||||
# M00-QAR-CAMERA-FRAMING-REFINEMENT-001 — Goldshire pitch/FOV refinement
|
||||
|
||||
<!-- OPENWC_CLAIM:M00-QAR-CAMERA-FRAMING-REFINEMENT-001:sindo-main-codex:2026-07-14 -->
|
||||
|
||||
## Ownership
|
||||
|
||||
- Target: M00
|
||||
- Program: QAR
|
||||
- Owner/Agent ID: sindo-main-codex
|
||||
- Branch: `work/sindo-main-codex/m00-camera-framing-refinement`
|
||||
- Lease expires UTC: 2026-07-14
|
||||
- Integrator: milestone integrator
|
||||
|
||||
## Outcome
|
||||
|
||||
Jointly refine Goldshire checkpoint pitch and FOV against a WMO-ready build 12340 reference.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Changing runtime player camera defaults.
|
||||
- Changing manifest values without an interior optimum and human approval.
|
||||
- Optimizing yaw beyond the ready-scene coarse result.
|
||||
|
||||
## Paths
|
||||
|
||||
- Exclusive: joint FOV option in camera-pose sweep and framing evidence
|
||||
- Shared/hotspots: camera-pose operational documentation and renderer module findings
|
||||
- Generated/ignored: candidate PNGs and ranking JSON under `user://`
|
||||
|
||||
## Contracts and data
|
||||
|
||||
- Public API/events: additive `-CameraFovValues`; scalar `-CameraFov` remains compatible
|
||||
- Schema/format version: ranking schema remains version 1 with existing per-candidate FOV field
|
||||
- Migration/compatibility: scalar sweeps retain existing candidate paths
|
||||
- Consumers: M00 fidelity workflow
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Requires: WMO-ready 8-second capture contract
|
||||
- Blocks: Goldshire manifest framing decision
|
||||
- External state: private build 12340 reference remains outside Git
|
||||
|
||||
## Verification
|
||||
|
||||
- Commands: joint-grid plan regression, real ready-scene sweep, comparator self-test, repository gates
|
||||
- Fixtures: private Goldshire Inn reference
|
||||
- Fidelity evidence: joint pitch/FOV ranking and human inspection
|
||||
- Performance budget: offline diagnostic
|
||||
|
||||
## Documentation deliverables
|
||||
|
||||
- Inline public API docs: PowerShell parameter
|
||||
- Module specification: framing finding
|
||||
- Data-flow diagram: FOV joins pose grid
|
||||
- Sequence/state/dependency diagrams: unchanged
|
||||
- Source map/status updates: operational guide
|
||||
|
||||
## Simplicity and naming
|
||||
|
||||
- Important names introduced: `CameraFovValues`, `effectiveCameraFovValues`
|
||||
- Simplest considered solution: one additional loop around the existing grid
|
||||
- Rejected complexity/abstractions: generic optimizer
|
||||
- Unavoidable complexity and justification: projection and pitch jointly affect framing
|
||||
- Measured optimization evidence: bounded 1x5x3 grid
|
||||
|
||||
## Status
|
||||
|
||||
- State: ready-for-review
|
||||
- Done: additive joint FOV grid, compatibility-preserving paths, plan regression, ready-scene 15-candidate ranking and human inspection
|
||||
- Next: add approved landmark/region scoring before any manifest camera calibration
|
||||
- Blocked by:
|
||||
|
||||
## Handoff
|
||||
|
||||
- Commit: this work-package commit
|
||||
- Results: numeric best FOV 62/yaw 10/pitch -25 at mean 0.078843 and ratio 0.667721 was visually rejected as grass-dominated; FOV 38/yaw 10/pitch -10 better matched building scale at mean 0.084220
|
||||
- Remaining risks: full-frame metric does not preserve landmark framing; no manifest values were changed
|
||||
- Documentation updated: `docs/CAMERA_POSE_SWEEP.md`, `docs/modules/world-renderer.md`
|
||||
@@ -0,0 +1,78 @@
|
||||
# M00-QAR-FINAL-PAIRED-EVIDENCE-001 — Final paired evidence audit
|
||||
|
||||
<!-- OPENWC_CLAIM:M00-QAR-FINAL-PAIRED-EVIDENCE-001:sindo-main-codex:2026-07-14 -->
|
||||
|
||||
## Ownership
|
||||
|
||||
- Target: M00
|
||||
- Program: QAR
|
||||
- Owner/Agent ID: sindo-main-codex
|
||||
- Branch: `work/sindo-main-codex/m00-final-paired-evidence`
|
||||
- Lease expires UTC: 2026-07-14
|
||||
- Integrator: milestone integrator
|
||||
|
||||
## Outcome
|
||||
|
||||
Produce one final WMO-ready paired report for all available build 12340 references and reconcile the remaining M00 acceptance gaps.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Achieving renderer parity inside M00.
|
||||
- Calibrating unknown manual camera parameters.
|
||||
- Marking the target DONE; only the milestone integrator may do so.
|
||||
|
||||
## Paths
|
||||
|
||||
- Exclusive: paired-run exit handling and final evidence record
|
||||
- Shared/hotspots: renderer baseline/module findings
|
||||
- Generated/ignored: final captures and reports under `user://`
|
||||
|
||||
## Contracts and data
|
||||
|
||||
- Public API/events: visual comparison exit 0/1 both count as completed report generation
|
||||
- Schema/format version: unchanged
|
||||
- Migration/compatibility: comparator infrastructure errors remain fatal
|
||||
- Consumers: M00 integrator
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Requires: merged region diagnostic and five private reference images
|
||||
- Blocks: integrator decision on M00 completion
|
||||
- External state: private images remain outside Git
|
||||
|
||||
## Verification
|
||||
|
||||
- Commands: complete paired baseline run, report audit, repository gates
|
||||
- Fixtures: five build 12340 references
|
||||
- Fidelity evidence: cold/warm comparison metrics and human gap classification
|
||||
- Performance budget: existing M00 capture budgets
|
||||
|
||||
## Documentation deliverables
|
||||
|
||||
- Inline public API docs: PowerShell comparison step semantics
|
||||
- Module specification: final evidence status
|
||||
- Data-flow diagram: unchanged
|
||||
- Sequence/state/dependency diagrams: unchanged
|
||||
- Source map/status updates: baseline acceptance summary
|
||||
|
||||
## Simplicity and naming
|
||||
|
||||
- Important names introduced: `Invoke-GodotComparisonStep`
|
||||
- Simplest considered solution: allow only comparator exit codes 0 and 1
|
||||
- Rejected complexity/abstractions: suppressing all process failures
|
||||
- Unavoidable complexity and justification: threshold exceedance is evidence, not infrastructure failure
|
||||
- Measured optimization evidence: not applicable
|
||||
|
||||
## Status
|
||||
|
||||
- State: ready-for-review
|
||||
- Done: paired-run exit contract fixed; full baseline and visual capture completed; ten pairs audited; final acceptance document written
|
||||
- Next: integrator decides animated/sky reference policy and GUI RID-risk disposition
|
||||
- Blocked by:
|
||||
|
||||
## Handoff
|
||||
|
||||
- Commit: this work-package commit
|
||||
- Results: compared=10, failed=10, missing=0; mean error 0.066892..0.176952; cold/warm gaps stable; full workflow returned success with threshold evidence preserved
|
||||
- Remaining risks: no build 12340 animated-M2 or sky-transition reference; GUI renderer RID/resources still reported at exit
|
||||
- Documentation updated: `docs/M00_FINAL_PAIRED_EVIDENCE.md`, `docs/modules/world-renderer.md`
|
||||
@@ -0,0 +1,78 @@
|
||||
# M00-QAR-LANDMARK-REGION-DIFF-001 — Landmark-region visual diff
|
||||
|
||||
<!-- OPENWC_CLAIM:M00-QAR-LANDMARK-REGION-DIFF-001:sindo-main-codex:2026-07-14 -->
|
||||
|
||||
## Ownership
|
||||
|
||||
- Target: M00
|
||||
- Program: QAR
|
||||
- Owner/Agent ID: sindo-main-codex
|
||||
- Branch: `work/sindo-main-codex/m00-landmark-region-diff`
|
||||
- Lease expires UTC: 2026-07-14
|
||||
- Integrator: milestone integrator
|
||||
|
||||
## Outcome
|
||||
|
||||
Rank camera candidates inside an approved common screen region so unrelated grass/sky coverage cannot dominate landmark framing.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Automatic feature detection or image warping.
|
||||
- Different crops for reference and candidate.
|
||||
- Changing manifest camera values without human approval.
|
||||
|
||||
## Paths
|
||||
|
||||
- Exclusive: comparator common-region option and sweep pass-through
|
||||
- Shared/hotspots: camera-pose documentation and renderer fidelity findings
|
||||
- Generated/ignored: local ROI sweep images/reports
|
||||
|
||||
## Contracts and data
|
||||
|
||||
- Public API/events: additive comparator `--region x,y,width,height` and sweep `-ComparisonRegion`
|
||||
- Schema/format version: report/ranking schema remains version 1 with additive region metadata
|
||||
- Migration/compatibility: omitted region preserves full-frame behavior
|
||||
- Consumers: M00 paired camera calibration
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Requires: ready-scene joint pitch/FOV sweep
|
||||
- Blocks: landmark-weighted Goldshire framing decision
|
||||
- External state: private build 12340 reference
|
||||
|
||||
## Verification
|
||||
|
||||
- Commands: comparator synthetic ROI regression, invalid-bound regression, sweep plan, bounded real ROI sweep, repository gates
|
||||
- Fixtures: synthetic 2x2 images and private Goldshire reference
|
||||
- Fidelity evidence: facade-region ranking plus human inspection
|
||||
- Performance budget: region comparison visits only selected pixels
|
||||
|
||||
## Documentation deliverables
|
||||
|
||||
- Inline public API docs: comparator usage and PowerShell parameter
|
||||
- Module specification: ROI finding
|
||||
- Data-flow diagram: approved region into comparator
|
||||
- Sequence/state/dependency diagrams: unchanged
|
||||
- Source map/status updates: camera-pose guide
|
||||
|
||||
## Simplicity and naming
|
||||
|
||||
- Important names introduced: `ComparisonRegion`, `comparison_region`
|
||||
- Simplest considered solution: one common integer rectangle
|
||||
- Rejected complexity/abstractions: masks, feature detectors, separate movable crops
|
||||
- Unavoidable complexity and justification: full-frame color error is biased by large terrain regions
|
||||
- Measured optimization evidence: bounded pixel loop
|
||||
|
||||
## Status
|
||||
|
||||
- State: ready-for-review
|
||||
- Done: common ROI parser/validation/reporting, synthetic inclusion/exclusion regression, sweep pass-through, 15-candidate Goldshire rescoring and human inspection
|
||||
- Next: define explicit approved landmark coordinates or mask; do not calibrate manifest from the common rectangle alone
|
||||
- Blocked by:
|
||||
|
||||
## Handoff
|
||||
|
||||
- Commit: this work-package commit
|
||||
- Results: ROI [300,100,1400,650] ranked FOV 38/yaw 10/pitch -15 first at mean 0.081777 and ratio 0.697185; visually the facade remains oversized and cropped
|
||||
- Remaining risks: rectangle includes non-semantic road/terrain pixels and cannot prove landmark alignment
|
||||
- Documentation updated: `docs/CAMERA_POSE_SWEEP.md`, `docs/modules/world-renderer.md`
|
||||
@@ -14,11 +14,12 @@ This offline M00 diagnostic recovers reproducible checkpoint framing when an ori
|
||||
-Checkpoint elwynn_adt_boundary `
|
||||
-YawOffsets -15,-10,-5,0,5,10,15 `
|
||||
-PitchOffsets -10,-5,0,5,10 `
|
||||
-CameraFovValues 38,50,62 `
|
||||
-ViewportWidth 1280 -ViewportHeight 960 `
|
||||
-WaitSeconds 8
|
||||
```
|
||||
|
||||
Each candidate receives its own output directory and comparison report. The viewport must exactly match the reference image dimensions; the runner fails on `size_mismatch` instead of inventing a score. The default 8-second wait matches the M00 manifest and is a readiness precondition: lowering it requires evidence that the checkpoint's WMO/M2 content has already appeared. `ranking.json` sorts candidates by mean perceptual error and then changed-pixel ratio. Use `-PlanOnly` to validate the Cartesian grid and output paths without rendering. Run a coarse grid first, then a finer grid around the best candidate.
|
||||
Each candidate receives its own output directory and comparison report. `-CameraFovValues` adds FOV as another bounded grid dimension; omit it to retain the scalar `-CameraFov` workflow and legacy candidate paths. The viewport must exactly match the reference image dimensions; the runner fails on `size_mismatch` instead of inventing a score. The default 8-second wait matches the M00 manifest and is a readiness precondition: lowering it requires evidence that the checkpoint's WMO/M2 content has already appeared. `ranking.json` sorts candidates by mean perceptual error and then changed-pixel ratio. Use `-PlanOnly` to validate the Cartesian grid and output paths without rendering. Run a coarse grid first, then a finer grid around the best candidate.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
@@ -47,4 +48,16 @@ These values are invalid camera-pose evidence because the sweep used a 2-second
|
||||
|
||||
The ready-scene 3x3 rerun at FOV 62 degrees ranked `(yaw=10, pitch=-10)` first with mean error `0.087952` and changed-pixel ratio `0.665527`, compared with `0.101402`/`0.702489` at zero offsets. Human inspection confirms that the inn is present and the direction is plausible. Pitch remains on the tested boundary and the building scale differs from the reference, so these offsets are coarse evidence only; a joint pitch/FOV refinement is required before changing manifest camera values.
|
||||
|
||||
## Goldshire joint pitch/FOV refinement — 2026-07-12
|
||||
|
||||
A WMO-ready joint grid fixed yaw at 10 degrees and evaluated pitch `[-25, -20, -15, -10, -5]` across FOV `[38, 50, 62]`. Full-frame error ranked FOV 62/pitch -25 first at `0.078843` with changed-pixel ratio `0.667721`. Human inspection rejects it as a manifest calibration: the view is dominated by grass and crops the inn. FOV 38/pitch -10 is closer to the reference building scale but ranks only `0.084220`, and horizontal landmark alignment remains different.
|
||||
|
||||
The grid therefore has no approved joint optimum. Full-frame color error continues to trade landmark framing against large terrain regions. The manifest remains unchanged. The next registration method must score an approved inn/road/tree landmark region or explicit landmark coordinates rather than the entire image.
|
||||
|
||||
## Common landmark region — 2026-07-12
|
||||
|
||||
The comparator accepts an optional common screen rectangle through `--region x,y,width,height`; the sweep runner exposes it as `-ComparisonRegion x,y,width,height`. Coordinates are integer pixels in the shared reference/candidate image space. Both images must have identical dimensions and the rectangle must be fully inside them. Omitting the option preserves full-frame behavior.
|
||||
|
||||
The existing 15 WMO-ready Goldshire candidates were rescored inside `[300,100,1400,650]`, covering the reference facade/chimney and nearby trees. ROI error ranked FOV 38/yaw 10/pitch -15 first at `0.081777` with changed-pixel ratio `0.697185`. This is an interior pitch candidate and improves on the full-frame grass preference, but human inspection still rejects manifest calibration: the facade is too large, its upper part is cropped, and the rectangle retains substantial road/terrain pixels. A common rectangle is useful diagnostic weighting, not semantic landmark registration. Explicit approved landmark coordinates or a mask remain necessary.
|
||||
|
||||
During this run the sweep orchestration was hardened in two ways: expected comparator exit code `1` is now collected through an explicit child process, and viewport dimensions are explicit. A reference/candidate `size_mismatch` now fails before ranking instead of producing an empty metric.
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
# M00 Final Paired Evidence — 2026-07-13
|
||||
|
||||
## Scope and result
|
||||
|
||||
The final local M00 audit used Godot 4.6.1, the existing cache state, the manifest-standard 8-second readiness wait, and `2560x1440` visual candidates. Seven private WoW 3.3.5a build 12340 references produced fourteen cold/warm comparisons with no missing pair. Private images and generated reports remain outside Git.
|
||||
|
||||
Every pair exceeded the strict regression tolerance. Comparator exit code `1` means a valid report with measured visual gaps; only codes outside `0/1` are infrastructure failures. This evidence establishes coverage and current differences, not renderer parity.
|
||||
|
||||
```powershell
|
||||
.\tools\run_render_baseline.ps1 `
|
||||
-Output user://render_baseline_m00_complete_2026-07-13 `
|
||||
-CacheState existing `
|
||||
-WaitSeconds 8 `
|
||||
-MeasureSeconds 0.5 `
|
||||
-ReferenceCheckpointDirectory .\sources\OpenWCReferenceCheckpoints `
|
||||
-VisualComparisonReport user://render_baseline_m00_complete_2026-07-13/visual_comparison.json `
|
||||
-VisualComparisonWidth 2560 `
|
||||
-VisualComparisonHeight 1440
|
||||
```
|
||||
|
||||
The initial shell timeout detached the canonical capture process after it started. The process completed all fourteen canonical captures and wrote `report.json`; the documented visual-capture and comparator stages were then invoked separately with the same arguments. The final output directory contains both complete reports.
|
||||
|
||||
## Paired metrics
|
||||
|
||||
| Checkpoint | Pass | Mean perceptual error | Changed-pixel ratio | Classification |
|
||||
|---|---|---:|---:|---|
|
||||
| `elwynn_adt_boundary` | cold | 0.087258 | 0.634580 | Composition plus terrain/vegetation/material gaps |
|
||||
| `elwynn_adt_boundary` | warm | 0.087255 | 0.634598 | Stable against revisit |
|
||||
| `elwynn_sky_dusk` | cold | 0.094778 | 0.575956 | Approximate paired camera; sky color, lighting and scene-composition gaps |
|
||||
| `elwynn_sky_dusk` | warm | 0.094757 | 0.575898 | Stable against revisit |
|
||||
| `elwynn_terrain_overview` | cold | 0.174137 | 0.851791 | Largest composition/framing gap |
|
||||
| `elwynn_terrain_overview` | warm | 0.176955 | 0.861271 | Stable gap, not a cold-load artifact |
|
||||
| `elwynn_waterfall_liquid` | cold | 0.088629 | 0.706522 | Framing, terrain, liquid and lighting gaps |
|
||||
| `elwynn_waterfall_liquid` | warm | 0.087398 | 0.693690 | Stable against revisit |
|
||||
| `goldshire_dense_m2` | cold | 0.066893 | 0.575257 | Smallest available paired gap; vegetation/material differences remain |
|
||||
| `goldshire_dense_m2` | warm | 0.066927 | 0.575592 | Stable against revisit |
|
||||
| `goldshire_inn_large_wmo` | cold | 0.101224 | 0.701835 | WMO is ready; manual framing and presentation gaps remain |
|
||||
| `goldshire_inn_large_wmo` | warm | 0.101206 | 0.701855 | Stable against revisit |
|
||||
| `gryphon_roost_native_animation` | cold | 0.119467 | 0.882073 | Same build-12340 asset, different real/synthetic placement and uncontrolled animation phase |
|
||||
| `gryphon_roost_native_animation` | warm | 0.119471 | 0.882076 | Full-frame metric is diagnostic only; animation observation is human evidence |
|
||||
|
||||
Cold and warm values are close for all seven references. The measured gaps are therefore not explained by first-visit cache readiness. Goldshire timing evidence separately demonstrated that the 8-second wait is necessary for WMO publication.
|
||||
|
||||
## Animation and sky provenance
|
||||
|
||||
The private source corpus preserves exact screenshot bytes; no resize, crop or generated content was used:
|
||||
|
||||
| Evidence | SHA-256 | Interpretation |
|
||||
|---|---|---|
|
||||
| `elwynn_sky_dusk.jpg` | `1fb06836cacae714c76eaf9b8bf7c292fd5e2f2df33eef821bfe4c4a6bd97072` | Build 12340, Elwynn camera position, server time approximately 19:00, UI hidden |
|
||||
| `gryphon_roost_native_animation.jpg` | `d3488c741cb8dfe7212198e48332842631a951862a4388b67045338f37357447` | Build 12340 real-world `GryphonRoost01` placement, first visible wing phase |
|
||||
| diagnostic animation phase B | `1025572c7ab9fa0d2e500ff026be46eb9c1e6e5447116dbdc3675dd5f44574ab` | Same camera three seconds later; visibly raised wing phase |
|
||||
| diagnostic placement context | `b12c0b713a7d75123ed30f5a316b33537331442cd56615aea69214c7cc4e7a42` | Two real Lakeshire placements and surrounding build-12340 scene |
|
||||
|
||||
An ADT scan found no `GryphonRoost01` placement in Goldshire. The M00 checkpoint deliberately injects the model there as a synthetic probe. The original-client observation therefore uses two real Lakeshire placements. The three-second pose change establishes `Observed` native animation behavior, but the full-frame comparison cannot establish spatial or animation-phase parity.
|
||||
|
||||
## Fidelity interpretation
|
||||
|
||||
M00 now has private build 12340 evidence for every required coverage category: terrain, ADT boundary, dense M2, large WMO, liquid, animated M2 and sky transition. The sky reference is approximately paired by map position/time and manually approved composition. Animated-M2 evidence is asset-paired but placement-unpaired because the OpenWC probe is synthetic.
|
||||
|
||||
The reports demonstrate current differences and do not claim `1:1`. Camera calibration experiments also show that unknown manual yaw/pitch/FOV cannot be recovered reliably from full-frame color error alone. Region scoring reduces terrain bias but still requires semantic landmark approval.
|
||||
|
||||
## Operational risks
|
||||
|
||||
The full GUI visual capture completed and wrote its report, but Godot still emitted renderer RID/resource leak diagnostics at exit, including leaked material, shader, texture, instance and rendering-device RIDs. The earlier anonymous headless `RefCounted` leak is fixed; these GUI renderer allocations are a distinct known risk. They do not invalidate the written images or metrics, but must remain recorded rather than hidden.
|
||||
|
||||
## Integrator decision checklist
|
||||
|
||||
- Seven required reference checkpoints: cold/warm report complete, `compared=14`, `missing=0`.
|
||||
- Animated M2: build-12340 native motion observed in two phases; synthetic/real placement mismatch explicitly classified.
|
||||
- Sky transition: build-12340 19:00 evidence present with approximate paired camera and human-approved composition.
|
||||
- Renderer visual differences: registered as gaps, not parity claims.
|
||||
- GUI renderer RID/resource diagnostics: accepted as a tracked risk or assigned a follow-up blocker.
|
||||
- Target status, target Evidence and `OPENWC_TARGET_DONE` marker: integrator-only actions.
|
||||
@@ -197,7 +197,7 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
|
||||
|
||||
- Unit/contract tests: material mapping, unique-ID dedupe, placement probes, baseline manifest, five-point coordinate calibration, synthetic perceptual checkpoint diff, camera-pose grid plan.
|
||||
- Integration/E2E: Eastern Kingdoms/Kalimdor streaming scenes and seven cold/warm checkpoints.
|
||||
- Fidelity evidence: пять локальных build 12340 reference JPG откалибровали terrain/ADT/M2/WMO/liquid viewpoints; automated paired-image metrics exist, но synthetic animation/dusk и полный human approval ещё не закрыты.
|
||||
- Fidelity evidence: семь локальных build 12340 reference JPG покрывают terrain/ADT/M2/WMO/liquid/sky и реальный `GryphonRoost01`; automated cold/warm metrics имеют `compared=14`, `missing=0`. Sky использует приблизительно paired camera, а animated-M2 остаётся asset-paired/placement-unpaired из-за synthetic probe.
|
||||
- Performance budgets: M00 report records cold/warm p95 and max hitch; no final acceptance threshold yet.
|
||||
- Manual diagnostics: [`../RENDER_BASELINE.md`](../RENDER_BASELINE.md) and [`../../RENDER.md`](../../RENDER.md).
|
||||
|
||||
@@ -232,6 +232,9 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
|
||||
- Camera-pose sweep can now rank bounded yaw/pitch grids without changing manifest defaults; perceptual ranking remains diagnostic and requires human framing approval.
|
||||
- The first Goldshire pose grid was invalidated by WMO readiness: the inn is absent after 2 seconds but visible after the manifest-standard 8-second wait. Pose comparisons must stabilize asynchronous checkpoint content first.
|
||||
- A ready-scene Goldshire 3x3 grid improved mean error from `0.101402` at zero offsets to `0.087952` at yaw `10`/pitch `-10`; pitch/FOV refinement remains required before manifest calibration.
|
||||
- Joint Goldshire pitch/FOV refinement had no human-approved optimum: full-frame error preferred grass-heavy FOV 62/pitch -25, while FOV 38/pitch -10 better matched building scale. Landmark/region scoring is required before calibration.
|
||||
- Common-region scoring `[300,100,1400,650]` moved Goldshire's numeric optimum to FOV 38/yaw 10/pitch -15, but human inspection still found an oversized/cropped facade; semantic landmarks or masks remain required.
|
||||
- Final paired audit compared fourteen cold/warm images for seven private build 12340 references with zero missing pairs; mean error ranges `0.066893..0.176955`. Animated-M2 native motion is observed across two original-client phases, but its real Lakeshire placement differs from the synthetic Goldshire probe. Sky-transition evidence uses the checkpoint map position/time with manually approved approximate framing.
|
||||
- D3D12 descriptor issues remain; the capture-path anonymous `RefCounted` shutdown leak is regression-covered by a clean verbose dry-run, while other RID/resource diagnostics still require independent evidence.
|
||||
- M2/WMO/material/particle/ribbon/portal parity incomplete.
|
||||
- Public API is mostly exported configuration rather than stable contracts.
|
||||
@@ -261,5 +264,6 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
|
||||
- [`../../RENDER.md`](../../RENDER.md)
|
||||
- [`../RENDER_BASELINE.md`](../RENDER_BASELINE.md)
|
||||
- [`../CAMERA_POSE_SWEEP.md`](../CAMERA_POSE_SWEEP.md)
|
||||
- [`../M00_FINAL_PAIRED_EVIDENCE.md`](../M00_FINAL_PAIRED_EVIDENCE.md)
|
||||
- [`../../targets/roadmap/02-rendering-and-graphics.md`](../../targets/roadmap/02-rendering-and-graphics.md)
|
||||
- [`../GODOT_BEST_PRACTICES.md`](../GODOT_BEST_PRACTICES.md)
|
||||
|
||||
@@ -3,6 +3,7 @@ extends SceneTree
|
||||
## Compares original-client JPG/PNG references with OpenWC checkpoint PNGs.
|
||||
## Usage: godot --headless --path . --script res://src/tools/compare_render_checkpoints.gd --
|
||||
## --reference <directory> --candidate <directory> [--output <report.json>]
|
||||
## [--region x,y,width,height]
|
||||
## [--pixel-threshold 0.04] [--mean-threshold 0.015] [--changed-ratio-threshold 0.01]
|
||||
## Use --self-test for a synthetic regression check that writes only under user://.
|
||||
|
||||
@@ -56,13 +57,18 @@ func _parse_arguments(raw_arguments: PackedStringArray) -> Dictionary:
|
||||
parsed.self_test = true
|
||||
index += 1
|
||||
continue
|
||||
if argument not in ["--reference", "--candidate", "--output", "--only", "--pixel-threshold", "--mean-threshold", "--changed-ratio-threshold"]:
|
||||
if argument not in ["--reference", "--candidate", "--output", "--only", "--region", "--pixel-threshold", "--mean-threshold", "--changed-ratio-threshold"]:
|
||||
return {"error": "unknown argument: %s" % argument}
|
||||
if index + 1 >= raw_arguments.size():
|
||||
return {"error": "missing value for %s" % argument}
|
||||
var key := argument.trim_prefix("--").replace("-", "_")
|
||||
var value := raw_arguments[index + 1]
|
||||
if key.ends_with("threshold"):
|
||||
if key == "region":
|
||||
var region_parse_result := _parse_comparison_region(value)
|
||||
if region_parse_result.has("error"):
|
||||
return region_parse_result
|
||||
parsed[key] = region_parse_result.region
|
||||
elif key.ends_with("threshold"):
|
||||
if not value.is_valid_float():
|
||||
return {"error": "%s requires a number" % argument}
|
||||
parsed[key] = float(value)
|
||||
@@ -72,6 +78,24 @@ func _parse_arguments(raw_arguments: PackedStringArray) -> Dictionary:
|
||||
return parsed
|
||||
|
||||
|
||||
func _parse_comparison_region(value: String) -> Dictionary:
|
||||
var components := value.split(",", false)
|
||||
if components.size() != 4:
|
||||
return {"error": "--region requires x,y,width,height"}
|
||||
for component in components:
|
||||
if not component.strip_edges().is_valid_int():
|
||||
return {"error": "--region components must be integers"}
|
||||
var region := Rect2i(
|
||||
int(components[0]),
|
||||
int(components[1]),
|
||||
int(components[2]),
|
||||
int(components[3])
|
||||
)
|
||||
if region.position.x < 0 or region.position.y < 0 or region.size.x <= 0 or region.size.y <= 0:
|
||||
return {"error": "--region requires non-negative x/y and positive width/height"}
|
||||
return {"region": region}
|
||||
|
||||
|
||||
func _compare_directories(reference_directory: String, candidate_directory: String, options: Dictionary) -> Dictionary:
|
||||
reference_directory = ProjectSettings.globalize_path(reference_directory)
|
||||
candidate_directory = ProjectSettings.globalize_path(candidate_directory)
|
||||
@@ -108,6 +132,7 @@ func _compare_directories(reference_directory: String, candidate_directory: Stri
|
||||
"pixel_threshold": options.pixel_threshold,
|
||||
"mean_threshold": options.mean_threshold,
|
||||
"changed_ratio_threshold": options.changed_ratio_threshold,
|
||||
"comparison_region": _region_array(options.get("region", Rect2i())),
|
||||
"compared_count": results.size() - missing_count,
|
||||
"failed_count": failed_count,
|
||||
"missing_count": missing_count,
|
||||
@@ -127,12 +152,22 @@ func _compare_images(reference_path: String, candidate_path: String, options: Di
|
||||
"reference_size": [reference_image.get_width(), reference_image.get_height()],
|
||||
"candidate_size": [candidate_image.get_width(), candidate_image.get_height()],
|
||||
}
|
||||
var comparison_region: Rect2i = options.get("region", Rect2i(Vector2i.ZERO, reference_image.get_size()))
|
||||
if comparison_region == Rect2i():
|
||||
comparison_region = Rect2i(Vector2i.ZERO, reference_image.get_size())
|
||||
var image_region := Rect2i(Vector2i.ZERO, reference_image.get_size())
|
||||
if not image_region.encloses(comparison_region):
|
||||
return {
|
||||
"status": "region_out_of_bounds",
|
||||
"region": _region_array(comparison_region),
|
||||
"image_size": [reference_image.get_width(), reference_image.get_height()],
|
||||
}
|
||||
|
||||
var error_sum := 0.0
|
||||
var changed_pixel_count := 0
|
||||
var pixel_count := reference_image.get_width() * reference_image.get_height()
|
||||
for y in reference_image.get_height():
|
||||
for x in reference_image.get_width():
|
||||
var pixel_count := comparison_region.size.x * comparison_region.size.y
|
||||
for y in range(comparison_region.position.y, comparison_region.end.y):
|
||||
for x in range(comparison_region.position.x, comparison_region.end.x):
|
||||
var perceptual_error := _perceptual_color_error(reference_image.get_pixel(x, y), candidate_image.get_pixel(x, y))
|
||||
error_sum += perceptual_error
|
||||
if perceptual_error > float(options.pixel_threshold):
|
||||
@@ -144,11 +179,18 @@ func _compare_images(reference_path: String, candidate_path: String, options: Di
|
||||
"status": "passed" if passed else "threshold_exceeded",
|
||||
"width": reference_image.get_width(),
|
||||
"height": reference_image.get_height(),
|
||||
"region": _region_array(comparison_region),
|
||||
"mean_perceptual_error": mean_perceptual_error,
|
||||
"changed_pixel_ratio": changed_pixel_ratio,
|
||||
}
|
||||
|
||||
|
||||
func _region_array(region: Rect2i) -> Array[int]:
|
||||
if region == Rect2i():
|
||||
return []
|
||||
return [region.position.x, region.position.y, region.size.x, region.size.y]
|
||||
|
||||
|
||||
func _perceptual_color_error(reference_color: Color, candidate_color: Color) -> float:
|
||||
var reference_linear := reference_color.srgb_to_linear()
|
||||
var candidate_linear := candidate_color.srgb_to_linear()
|
||||
@@ -229,6 +271,22 @@ func _run_self_test() -> int:
|
||||
if not identical_report.passed or changed_report.passed:
|
||||
push_error("RENDER_CHECKPOINT_DIFF SELF_TEST: expected identical pass and changed failure")
|
||||
return 1
|
||||
var region_options := options.duplicate()
|
||||
region_options["region"] = Rect2i(1, 1, 1, 1)
|
||||
var excluded_change_report := _compare_directories(reference_directory, changed_directory, region_options)
|
||||
if not excluded_change_report.passed or excluded_change_report.comparison_region != [1, 1, 1, 1]:
|
||||
push_error("RENDER_CHECKPOINT_DIFF SELF_TEST: region must exclude pixels outside the rectangle")
|
||||
return 1
|
||||
var out_of_bounds_options := options.duplicate()
|
||||
out_of_bounds_options["region"] = Rect2i(1, 1, 2, 2)
|
||||
var out_of_bounds_result := _compare_images(
|
||||
reference_directory.path_join("synthetic.jpg"),
|
||||
changed_directory.path_join("synthetic__cold_process.png"),
|
||||
out_of_bounds_options
|
||||
)
|
||||
if out_of_bounds_result.status != "region_out_of_bounds":
|
||||
push_error("RENDER_CHECKPOINT_DIFF SELF_TEST: out-of-bounds region must fail")
|
||||
return 1
|
||||
if reference_image.save_jpg(reference_directory.path_join("ignored.jpg"), 1.0) != OK:
|
||||
return 1
|
||||
var filtered_options := options.duplicate()
|
||||
|
||||
@@ -40,6 +40,15 @@ function Invoke-GodotStep {
|
||||
}
|
||||
}
|
||||
|
||||
function Invoke-GodotComparisonStep {
|
||||
param([string]$Name, [string[]]$Arguments)
|
||||
Write-Host "[$Name] $GodotPath $($Arguments -join ' ')"
|
||||
$process = Start-Process -FilePath $GodotPath -ArgumentList $Arguments -NoNewWindow -Wait -PassThru
|
||||
if ($process.ExitCode -notin @(0, 1)) {
|
||||
throw "$Name failed with exit code $($process.ExitCode)"
|
||||
}
|
||||
}
|
||||
|
||||
Push-Location $repoRoot
|
||||
try {
|
||||
Invoke-GodotStep "project-load" @("--headless", "--path", ".", "--quit")
|
||||
@@ -80,7 +89,7 @@ try {
|
||||
"--viewport-height", $VisualComparisonHeight.ToString()
|
||||
)
|
||||
Invoke-GodotStep "visual-capture" $visualCaptureArgs
|
||||
Invoke-GodotStep "visual-comparison" @(
|
||||
Invoke-GodotComparisonStep "visual-comparison" @(
|
||||
"--headless", "--path", ".",
|
||||
"--script", "res://src/tools/compare_render_checkpoints.gd",
|
||||
"--",
|
||||
|
||||
@@ -9,8 +9,10 @@ param(
|
||||
[double[]]$YawOffsets = @(-10.0, 0.0, 10.0),
|
||||
[double[]]$PitchOffsets = @(-10.0, 0.0, 10.0),
|
||||
[double]$CameraFov = 62.0,
|
||||
[double[]]$CameraFovValues = @(),
|
||||
[int]$ViewportWidth = 1280,
|
||||
[int]$ViewportHeight = 900,
|
||||
[int[]]$ComparisonRegion = @(),
|
||||
[double]$WaitSeconds = 8.0,
|
||||
[double]$MeasureSeconds = 0.1,
|
||||
[switch]$PlanOnly
|
||||
@@ -32,9 +34,13 @@ if (-not (Test-Path -LiteralPath $GodotPath)) {
|
||||
if (-not (Test-Path -LiteralPath $ReferenceDirectory)) {
|
||||
throw "Reference directory not found: $ReferenceDirectory"
|
||||
}
|
||||
if ($ComparisonRegion.Count -notin @(0, 4)) {
|
||||
throw "ComparisonRegion requires x,y,width,height"
|
||||
}
|
||||
|
||||
$invariantCulture = [Globalization.CultureInfo]::InvariantCulture
|
||||
$cameraFovText = $CameraFov.ToString($invariantCulture)
|
||||
$isJointFovSweep = $CameraFovValues.Count -gt 0
|
||||
$effectiveCameraFovValues = if ($isJointFovSweep) { $CameraFovValues } else { @($CameraFov) }
|
||||
$waitSecondsText = $WaitSeconds.ToString($invariantCulture)
|
||||
$measureSecondsText = $MeasureSeconds.ToString($invariantCulture)
|
||||
$absoluteOutput = if ($Output.StartsWith("user://")) {
|
||||
@@ -46,11 +52,16 @@ $ranking = [Collections.Generic.List[object]]::new()
|
||||
|
||||
Push-Location $repoRoot
|
||||
try {
|
||||
foreach ($yawOffset in $YawOffsets) {
|
||||
foreach ($pitchOffset in $PitchOffsets) {
|
||||
foreach ($cameraFovValue in $effectiveCameraFovValues) {
|
||||
foreach ($yawOffset in $YawOffsets) {
|
||||
foreach ($pitchOffset in $PitchOffsets) {
|
||||
$cameraFovText = $cameraFovValue.ToString("0.###", $invariantCulture)
|
||||
$yawText = $yawOffset.ToString("0.###", $invariantCulture)
|
||||
$pitchText = $pitchOffset.ToString("0.###", $invariantCulture)
|
||||
$candidateName = "yaw_$($yawText.Replace('-', 'n').Replace('.', '_'))__pitch_$($pitchText.Replace('-', 'n').Replace('.', '_'))"
|
||||
if ($isJointFovSweep) {
|
||||
$candidateName = "fov_$($cameraFovText.Replace('.', '_'))__$candidateName"
|
||||
}
|
||||
$candidateOutput = "$Output/$candidateName"
|
||||
$comparisonReport = Join-Path $absoluteOutput "$candidateName.json"
|
||||
|
||||
@@ -58,7 +69,7 @@ try {
|
||||
$ranking.Add([pscustomobject]@{
|
||||
yaw_offset_degrees = $yawOffset
|
||||
pitch_offset_degrees = $pitchOffset
|
||||
camera_fov_degrees = $CameraFov
|
||||
camera_fov_degrees = $cameraFovValue
|
||||
candidate_directory = $candidateOutput
|
||||
})
|
||||
continue
|
||||
@@ -82,6 +93,9 @@ try {
|
||||
"--reference", $ReferenceDirectory, "--candidate", $candidateOutput,
|
||||
"--only", $Checkpoint, "--output", $comparisonReport
|
||||
)
|
||||
if ($ComparisonRegion.Count -eq 4) {
|
||||
$comparisonArguments += @("--region", ($ComparisonRegion -join ","))
|
||||
}
|
||||
$comparisonProcess = Start-Process -FilePath $GodotPath -ArgumentList $comparisonArguments -Wait -PassThru -NoNewWindow
|
||||
if ($comparisonProcess.ExitCode -notin @(0, 1)) {
|
||||
throw "Comparison failed for yaw=$yawText pitch=$pitchText"
|
||||
@@ -101,12 +115,13 @@ try {
|
||||
$ranking.Add([pscustomobject]@{
|
||||
yaw_offset_degrees = $yawOffset
|
||||
pitch_offset_degrees = $pitchOffset
|
||||
camera_fov_degrees = $CameraFov
|
||||
camera_fov_degrees = $cameraFovValue
|
||||
mean_perceptual_error = $meanError
|
||||
changed_pixel_ratio = $changedRatio
|
||||
candidate_directory = $candidateOutput
|
||||
comparison_report = $comparisonReport
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -123,11 +138,12 @@ try {
|
||||
viewport = @($ViewportWidth, $ViewportHeight)
|
||||
wait_seconds = $WaitSeconds
|
||||
measure_seconds = $MeasureSeconds
|
||||
comparison_region = @($ComparisonRegion)
|
||||
candidates = $rankedCandidates
|
||||
}
|
||||
New-Item -ItemType Directory -Force -Path $absoluteOutput | Out-Null
|
||||
$rankingDocument | ConvertTo-Json -Depth 6 | Set-Content -Encoding UTF8 -LiteralPath $rankingPath
|
||||
$rankedCandidates | Format-Table yaw_offset_degrees, pitch_offset_degrees, mean_perceptual_error, changed_pixel_ratio
|
||||
$rankedCandidates | Format-Table camera_fov_degrees, yaw_offset_degrees, pitch_offset_degrees, mean_perceptual_error, changed_pixel_ratio
|
||||
$completionKind = if ($PlanOnly) { "plan" } else { "ranking" }
|
||||
Write-Output "Camera pose sweep $completionKind completed: $rankingPath"
|
||||
} finally {
|
||||
|
||||
Reference in New Issue
Block a user