Compare commits
5 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 0327e4862d | |||
| 239cb2432e | |||
| b5ddc86c92 | |||
| 3d90c6d79a | |||
| a10206b797 |
@@ -0,0 +1,78 @@
|
|||||||
|
# M00-QAR-ANIMATION-SKY-EVIDENCE-001 — Animation and sky evidence
|
||||||
|
|
||||||
|
<!-- OPENWC_CLAIM:M00-QAR-ANIMATION-SKY-EVIDENCE-001:sindo-main-codex:2026-07-15 -->
|
||||||
|
|
||||||
|
## Ownership
|
||||||
|
|
||||||
|
- Target: M00
|
||||||
|
- Program: QAR
|
||||||
|
- Owner/Agent ID: sindo-main-codex
|
||||||
|
- Branch: `work/sindo-main-codex/m00-animation-sky-evidence`
|
||||||
|
- Lease expires UTC: 2026-07-15
|
||||||
|
- Integrator: milestone integrator
|
||||||
|
|
||||||
|
## Outcome
|
||||||
|
|
||||||
|
Add private build 12340 evidence for the remaining sky-transition and native animated-M2 coverage, rerun all paired comparisons, and hand the resulting audit to the milestone integrator.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- Committing proprietary screenshots or extracted assets.
|
||||||
|
- Claiming renderer parity from mismatched world placement or animation phase.
|
||||||
|
- Marking M00 DONE; only the milestone integrator may change target status or Evidence.
|
||||||
|
|
||||||
|
## Paths
|
||||||
|
|
||||||
|
- Exclusive: this claim and animation/sky evidence documentation
|
||||||
|
- Shared/hotspots: `docs/M00_FINAL_PAIRED_EVIDENCE.md`, `docs/modules/world-renderer.md`
|
||||||
|
- Generated/ignored: normalized private references under `sources/OpenWCReferenceCheckpoints`; captures and reports under `user://`
|
||||||
|
|
||||||
|
## Contracts and data
|
||||||
|
|
||||||
|
- Public API/events: unchanged
|
||||||
|
- Schema/format version: unchanged
|
||||||
|
- Migration/compatibility: existing five-reference comparison remains valid
|
||||||
|
- Consumers: M00 milestone integrator
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
- Requires: user-captured build 12340 screenshots at 19:00 and two real `GryphonRoost01` animation phases
|
||||||
|
- Blocks: final integrator decision on M00 completion
|
||||||
|
- External state: private screenshots remain outside Git
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
- Commands: private-reference preparation, full baseline runner, comparator self-test, coordination and documentation gates
|
||||||
|
- Fixtures: seven private build 12340 checkpoint references plus a second animation-phase provenance frame
|
||||||
|
- Fidelity evidence: sky-transition screenshot and observed wing pose change over three seconds
|
||||||
|
- Performance budget: existing M00 capture budgets
|
||||||
|
|
||||||
|
## Documentation deliverables
|
||||||
|
|
||||||
|
- Inline public API docs: unchanged
|
||||||
|
- Module specification: paired coverage and remaining limitations
|
||||||
|
- Data-flow diagram: unchanged
|
||||||
|
- Sequence/state/dependency diagrams: unchanged
|
||||||
|
- Source map/status updates: final paired evidence audit
|
||||||
|
|
||||||
|
## Simplicity and naming
|
||||||
|
|
||||||
|
- Important names introduced: none
|
||||||
|
- Simplest considered solution: normalize private captures outside Git and reuse the existing comparator
|
||||||
|
- Rejected complexity/abstractions: adding a video comparator or committing an asset corpus
|
||||||
|
- Unavoidable complexity and justification: animation phase and synthetic placement require human evidence classification alongside pixel metrics
|
||||||
|
- Measured optimization evidence: not applicable
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
- State: ready-for-review
|
||||||
|
- Done: exact-size private references registered; real ADT placement verified; canonical and visual captures completed; fourteen comparisons audited
|
||||||
|
- Next: milestone integrator reviews synthetic/real placement classification and GUI RID risk
|
||||||
|
- Blocked by:
|
||||||
|
|
||||||
|
## Handoff
|
||||||
|
|
||||||
|
- Commit: this work-package commit
|
||||||
|
- Results: `compared=14`, `failed=14`, `missing=0`; mean error `0.066893..0.176955`; original-client Gryphon wing phase changed over three seconds
|
||||||
|
- Remaining risks: synthetic probe placement differs from the original-client world placement; GUI RID/resource diagnostics remain separate
|
||||||
|
- Documentation updated: `docs/M00_FINAL_PAIRED_EVIDENCE.md`, `docs/modules/world-renderer.md`
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
# M00-QAR-FINAL-PAIRED-EVIDENCE-001 — Final paired evidence audit
|
||||||
|
|
||||||
|
<!-- OPENWC_CLAIM:M00-QAR-FINAL-PAIRED-EVIDENCE-001:sindo-main-codex:2026-07-14 -->
|
||||||
|
|
||||||
|
## Ownership
|
||||||
|
|
||||||
|
- Target: M00
|
||||||
|
- Program: QAR
|
||||||
|
- Owner/Agent ID: sindo-main-codex
|
||||||
|
- Branch: `work/sindo-main-codex/m00-final-paired-evidence`
|
||||||
|
- Lease expires UTC: 2026-07-14
|
||||||
|
- Integrator: milestone integrator
|
||||||
|
|
||||||
|
## Outcome
|
||||||
|
|
||||||
|
Produce one final WMO-ready paired report for all available build 12340 references and reconcile the remaining M00 acceptance gaps.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- Achieving renderer parity inside M00.
|
||||||
|
- Calibrating unknown manual camera parameters.
|
||||||
|
- Marking the target DONE; only the milestone integrator may do so.
|
||||||
|
|
||||||
|
## Paths
|
||||||
|
|
||||||
|
- Exclusive: paired-run exit handling and final evidence record
|
||||||
|
- Shared/hotspots: renderer baseline/module findings
|
||||||
|
- Generated/ignored: final captures and reports under `user://`
|
||||||
|
|
||||||
|
## Contracts and data
|
||||||
|
|
||||||
|
- Public API/events: visual comparison exit 0/1 both count as completed report generation
|
||||||
|
- Schema/format version: unchanged
|
||||||
|
- Migration/compatibility: comparator infrastructure errors remain fatal
|
||||||
|
- Consumers: M00 integrator
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
- Requires: merged region diagnostic and five private reference images
|
||||||
|
- Blocks: integrator decision on M00 completion
|
||||||
|
- External state: private images remain outside Git
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
- Commands: complete paired baseline run, report audit, repository gates
|
||||||
|
- Fixtures: five build 12340 references
|
||||||
|
- Fidelity evidence: cold/warm comparison metrics and human gap classification
|
||||||
|
- Performance budget: existing M00 capture budgets
|
||||||
|
|
||||||
|
## Documentation deliverables
|
||||||
|
|
||||||
|
- Inline public API docs: PowerShell comparison step semantics
|
||||||
|
- Module specification: final evidence status
|
||||||
|
- Data-flow diagram: unchanged
|
||||||
|
- Sequence/state/dependency diagrams: unchanged
|
||||||
|
- Source map/status updates: baseline acceptance summary
|
||||||
|
|
||||||
|
## Simplicity and naming
|
||||||
|
|
||||||
|
- Important names introduced: `Invoke-GodotComparisonStep`
|
||||||
|
- Simplest considered solution: allow only comparator exit codes 0 and 1
|
||||||
|
- Rejected complexity/abstractions: suppressing all process failures
|
||||||
|
- Unavoidable complexity and justification: threshold exceedance is evidence, not infrastructure failure
|
||||||
|
- Measured optimization evidence: not applicable
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
- State: ready-for-review
|
||||||
|
- Done: paired-run exit contract fixed; full baseline and visual capture completed; ten pairs audited; final acceptance document written
|
||||||
|
- Next: integrator decides animated/sky reference policy and GUI RID-risk disposition
|
||||||
|
- Blocked by:
|
||||||
|
|
||||||
|
## Handoff
|
||||||
|
|
||||||
|
- Commit: this work-package commit
|
||||||
|
- Results: compared=10, failed=10, missing=0; mean error 0.066892..0.176952; cold/warm gaps stable; full workflow returned success with threshold evidence preserved
|
||||||
|
- Remaining risks: no build 12340 animated-M2 or sky-transition reference; GUI renderer RID/resources still reported at exit
|
||||||
|
- Documentation updated: `docs/M00_FINAL_PAIRED_EVIDENCE.md`, `docs/modules/world-renderer.md`
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
# M00-QAR-LANDMARK-REGION-DIFF-001 — Landmark-region visual diff
|
||||||
|
|
||||||
|
<!-- OPENWC_CLAIM:M00-QAR-LANDMARK-REGION-DIFF-001:sindo-main-codex:2026-07-14 -->
|
||||||
|
|
||||||
|
## Ownership
|
||||||
|
|
||||||
|
- Target: M00
|
||||||
|
- Program: QAR
|
||||||
|
- Owner/Agent ID: sindo-main-codex
|
||||||
|
- Branch: `work/sindo-main-codex/m00-landmark-region-diff`
|
||||||
|
- Lease expires UTC: 2026-07-14
|
||||||
|
- Integrator: milestone integrator
|
||||||
|
|
||||||
|
## Outcome
|
||||||
|
|
||||||
|
Rank camera candidates inside an approved common screen region so unrelated grass/sky coverage cannot dominate landmark framing.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- Automatic feature detection or image warping.
|
||||||
|
- Different crops for reference and candidate.
|
||||||
|
- Changing manifest camera values without human approval.
|
||||||
|
|
||||||
|
## Paths
|
||||||
|
|
||||||
|
- Exclusive: comparator common-region option and sweep pass-through
|
||||||
|
- Shared/hotspots: camera-pose documentation and renderer fidelity findings
|
||||||
|
- Generated/ignored: local ROI sweep images/reports
|
||||||
|
|
||||||
|
## Contracts and data
|
||||||
|
|
||||||
|
- Public API/events: additive comparator `--region x,y,width,height` and sweep `-ComparisonRegion`
|
||||||
|
- Schema/format version: report/ranking schema remains version 1 with additive region metadata
|
||||||
|
- Migration/compatibility: omitted region preserves full-frame behavior
|
||||||
|
- Consumers: M00 paired camera calibration
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
- Requires: ready-scene joint pitch/FOV sweep
|
||||||
|
- Blocks: landmark-weighted Goldshire framing decision
|
||||||
|
- External state: private build 12340 reference
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
- Commands: comparator synthetic ROI regression, invalid-bound regression, sweep plan, bounded real ROI sweep, repository gates
|
||||||
|
- Fixtures: synthetic 2x2 images and private Goldshire reference
|
||||||
|
- Fidelity evidence: facade-region ranking plus human inspection
|
||||||
|
- Performance budget: region comparison visits only selected pixels
|
||||||
|
|
||||||
|
## Documentation deliverables
|
||||||
|
|
||||||
|
- Inline public API docs: comparator usage and PowerShell parameter
|
||||||
|
- Module specification: ROI finding
|
||||||
|
- Data-flow diagram: approved region into comparator
|
||||||
|
- Sequence/state/dependency diagrams: unchanged
|
||||||
|
- Source map/status updates: camera-pose guide
|
||||||
|
|
||||||
|
## Simplicity and naming
|
||||||
|
|
||||||
|
- Important names introduced: `ComparisonRegion`, `comparison_region`
|
||||||
|
- Simplest considered solution: one common integer rectangle
|
||||||
|
- Rejected complexity/abstractions: masks, feature detectors, separate movable crops
|
||||||
|
- Unavoidable complexity and justification: full-frame color error is biased by large terrain regions
|
||||||
|
- Measured optimization evidence: bounded pixel loop
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
- State: ready-for-review
|
||||||
|
- Done: common ROI parser/validation/reporting, synthetic inclusion/exclusion regression, sweep pass-through, 15-candidate Goldshire rescoring and human inspection
|
||||||
|
- Next: define explicit approved landmark coordinates or mask; do not calibrate manifest from the common rectangle alone
|
||||||
|
- Blocked by:
|
||||||
|
|
||||||
|
## Handoff
|
||||||
|
|
||||||
|
- Commit: this work-package commit
|
||||||
|
- Results: ROI [300,100,1400,650] ranked FOV 38/yaw 10/pitch -15 first at mean 0.081777 and ratio 0.697185; visually the facade remains oversized and cropped
|
||||||
|
- Remaining risks: rectangle includes non-semantic road/terrain pixels and cannot prove landmark alignment
|
||||||
|
- Documentation updated: `docs/CAMERA_POSE_SWEEP.md`, `docs/modules/world-renderer.md`
|
||||||
@@ -54,4 +54,10 @@ A WMO-ready joint grid fixed yaw at 10 degrees and evaluated pitch `[-25, -20, -
|
|||||||
|
|
||||||
The grid therefore has no approved joint optimum. Full-frame color error continues to trade landmark framing against large terrain regions. The manifest remains unchanged. The next registration method must score an approved inn/road/tree landmark region or explicit landmark coordinates rather than the entire image.
|
The grid therefore has no approved joint optimum. Full-frame color error continues to trade landmark framing against large terrain regions. The manifest remains unchanged. The next registration method must score an approved inn/road/tree landmark region or explicit landmark coordinates rather than the entire image.
|
||||||
|
|
||||||
|
## Common landmark region — 2026-07-12
|
||||||
|
|
||||||
|
The comparator accepts an optional common screen rectangle through `--region x,y,width,height`; the sweep runner exposes it as `-ComparisonRegion x,y,width,height`. Coordinates are integer pixels in the shared reference/candidate image space. Both images must have identical dimensions and the rectangle must be fully inside them. Omitting the option preserves full-frame behavior.
|
||||||
|
|
||||||
|
The existing 15 WMO-ready Goldshire candidates were rescored inside `[300,100,1400,650]`, covering the reference facade/chimney and nearby trees. ROI error ranked FOV 38/yaw 10/pitch -15 first at `0.081777` with changed-pixel ratio `0.697185`. This is an interior pitch candidate and improves on the full-frame grass preference, but human inspection still rejects manifest calibration: the facade is too large, its upper part is cropped, and the rectangle retains substantial road/terrain pixels. A common rectangle is useful diagnostic weighting, not semantic landmark registration. Explicit approved landmark coordinates or a mask remain necessary.
|
||||||
|
|
||||||
During this run the sweep orchestration was hardened in two ways: expected comparator exit code `1` is now collected through an explicit child process, and viewport dimensions are explicit. A reference/candidate `size_mismatch` now fails before ranking instead of producing an empty metric.
|
During this run the sweep orchestration was hardened in two ways: expected comparator exit code `1` is now collected through an explicit child process, and viewport dimensions are explicit. A reference/candidate `size_mismatch` now fails before ranking instead of producing an empty metric.
|
||||||
|
|||||||
@@ -0,0 +1,74 @@
|
|||||||
|
# M00 Final Paired Evidence — 2026-07-13
|
||||||
|
|
||||||
|
## Scope and result
|
||||||
|
|
||||||
|
The final local M00 audit used Godot 4.6.1, the existing cache state, the manifest-standard 8-second readiness wait, and `2560x1440` visual candidates. Seven private WoW 3.3.5a build 12340 references produced fourteen cold/warm comparisons with no missing pair. Private images and generated reports remain outside Git.
|
||||||
|
|
||||||
|
Every pair exceeded the strict regression tolerance. Comparator exit code `1` means a valid report with measured visual gaps; only codes outside `0/1` are infrastructure failures. This evidence establishes coverage and current differences, not renderer parity.
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
.\tools\run_render_baseline.ps1 `
|
||||||
|
-Output user://render_baseline_m00_complete_2026-07-13 `
|
||||||
|
-CacheState existing `
|
||||||
|
-WaitSeconds 8 `
|
||||||
|
-MeasureSeconds 0.5 `
|
||||||
|
-ReferenceCheckpointDirectory .\sources\OpenWCReferenceCheckpoints `
|
||||||
|
-VisualComparisonReport user://render_baseline_m00_complete_2026-07-13/visual_comparison.json `
|
||||||
|
-VisualComparisonWidth 2560 `
|
||||||
|
-VisualComparisonHeight 1440
|
||||||
|
```
|
||||||
|
|
||||||
|
The initial shell timeout detached the canonical capture process after it started. The process completed all fourteen canonical captures and wrote `report.json`; the documented visual-capture and comparator stages were then invoked separately with the same arguments. The final output directory contains both complete reports.
|
||||||
|
|
||||||
|
## Paired metrics
|
||||||
|
|
||||||
|
| Checkpoint | Pass | Mean perceptual error | Changed-pixel ratio | Classification |
|
||||||
|
|---|---|---:|---:|---|
|
||||||
|
| `elwynn_adt_boundary` | cold | 0.087258 | 0.634580 | Composition plus terrain/vegetation/material gaps |
|
||||||
|
| `elwynn_adt_boundary` | warm | 0.087255 | 0.634598 | Stable against revisit |
|
||||||
|
| `elwynn_sky_dusk` | cold | 0.094778 | 0.575956 | Approximate paired camera; sky color, lighting and scene-composition gaps |
|
||||||
|
| `elwynn_sky_dusk` | warm | 0.094757 | 0.575898 | Stable against revisit |
|
||||||
|
| `elwynn_terrain_overview` | cold | 0.174137 | 0.851791 | Largest composition/framing gap |
|
||||||
|
| `elwynn_terrain_overview` | warm | 0.176955 | 0.861271 | Stable gap, not a cold-load artifact |
|
||||||
|
| `elwynn_waterfall_liquid` | cold | 0.088629 | 0.706522 | Framing, terrain, liquid and lighting gaps |
|
||||||
|
| `elwynn_waterfall_liquid` | warm | 0.087398 | 0.693690 | Stable against revisit |
|
||||||
|
| `goldshire_dense_m2` | cold | 0.066893 | 0.575257 | Smallest available paired gap; vegetation/material differences remain |
|
||||||
|
| `goldshire_dense_m2` | warm | 0.066927 | 0.575592 | Stable against revisit |
|
||||||
|
| `goldshire_inn_large_wmo` | cold | 0.101224 | 0.701835 | WMO is ready; manual framing and presentation gaps remain |
|
||||||
|
| `goldshire_inn_large_wmo` | warm | 0.101206 | 0.701855 | Stable against revisit |
|
||||||
|
| `gryphon_roost_native_animation` | cold | 0.119467 | 0.882073 | Same build-12340 asset, different real/synthetic placement and uncontrolled animation phase |
|
||||||
|
| `gryphon_roost_native_animation` | warm | 0.119471 | 0.882076 | Full-frame metric is diagnostic only; animation observation is human evidence |
|
||||||
|
|
||||||
|
Cold and warm values are close for all seven references. The measured gaps are therefore not explained by first-visit cache readiness. Goldshire timing evidence separately demonstrated that the 8-second wait is necessary for WMO publication.
|
||||||
|
|
||||||
|
## Animation and sky provenance
|
||||||
|
|
||||||
|
The private source corpus preserves exact screenshot bytes; no resize, crop or generated content was used:
|
||||||
|
|
||||||
|
| Evidence | SHA-256 | Interpretation |
|
||||||
|
|---|---|---|
|
||||||
|
| `elwynn_sky_dusk.jpg` | `1fb06836cacae714c76eaf9b8bf7c292fd5e2f2df33eef821bfe4c4a6bd97072` | Build 12340, Elwynn camera position, server time approximately 19:00, UI hidden |
|
||||||
|
| `gryphon_roost_native_animation.jpg` | `d3488c741cb8dfe7212198e48332842631a951862a4388b67045338f37357447` | Build 12340 real-world `GryphonRoost01` placement, first visible wing phase |
|
||||||
|
| diagnostic animation phase B | `1025572c7ab9fa0d2e500ff026be46eb9c1e6e5447116dbdc3675dd5f44574ab` | Same camera three seconds later; visibly raised wing phase |
|
||||||
|
| diagnostic placement context | `b12c0b713a7d75123ed30f5a316b33537331442cd56615aea69214c7cc4e7a42` | Two real Lakeshire placements and surrounding build-12340 scene |
|
||||||
|
|
||||||
|
An ADT scan found no `GryphonRoost01` placement in Goldshire. The M00 checkpoint deliberately injects the model there as a synthetic probe. The original-client observation therefore uses two real Lakeshire placements. The three-second pose change establishes `Observed` native animation behavior, but the full-frame comparison cannot establish spatial or animation-phase parity.
|
||||||
|
|
||||||
|
## Fidelity interpretation
|
||||||
|
|
||||||
|
M00 now has private build 12340 evidence for every required coverage category: terrain, ADT boundary, dense M2, large WMO, liquid, animated M2 and sky transition. The sky reference is approximately paired by map position/time and manually approved composition. Animated-M2 evidence is asset-paired but placement-unpaired because the OpenWC probe is synthetic.
|
||||||
|
|
||||||
|
The reports demonstrate current differences and do not claim `1:1`. Camera calibration experiments also show that unknown manual yaw/pitch/FOV cannot be recovered reliably from full-frame color error alone. Region scoring reduces terrain bias but still requires semantic landmark approval.
|
||||||
|
|
||||||
|
## Operational risks
|
||||||
|
|
||||||
|
The full GUI visual capture completed and wrote its report, but Godot still emitted renderer RID/resource leak diagnostics at exit, including leaked material, shader, texture, instance and rendering-device RIDs. The earlier anonymous headless `RefCounted` leak is fixed; these GUI renderer allocations are a distinct known risk. They do not invalidate the written images or metrics, but must remain recorded rather than hidden.
|
||||||
|
|
||||||
|
## Integrator decision checklist
|
||||||
|
|
||||||
|
- Seven required reference checkpoints: cold/warm report complete, `compared=14`, `missing=0`.
|
||||||
|
- Animated M2: build-12340 native motion observed in two phases; synthetic/real placement mismatch explicitly classified.
|
||||||
|
- Sky transition: build-12340 19:00 evidence present with approximate paired camera and human-approved composition.
|
||||||
|
- Renderer visual differences: registered as gaps, not parity claims.
|
||||||
|
- GUI renderer RID/resource diagnostics: accepted as a tracked risk or assigned a follow-up blocker.
|
||||||
|
- Target status, target Evidence and `OPENWC_TARGET_DONE` marker: integrator-only actions.
|
||||||
@@ -197,7 +197,7 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
|
|||||||
|
|
||||||
- Unit/contract tests: material mapping, unique-ID dedupe, placement probes, baseline manifest, five-point coordinate calibration, synthetic perceptual checkpoint diff, camera-pose grid plan.
|
- Unit/contract tests: material mapping, unique-ID dedupe, placement probes, baseline manifest, five-point coordinate calibration, synthetic perceptual checkpoint diff, camera-pose grid plan.
|
||||||
- Integration/E2E: Eastern Kingdoms/Kalimdor streaming scenes and seven cold/warm checkpoints.
|
- Integration/E2E: Eastern Kingdoms/Kalimdor streaming scenes and seven cold/warm checkpoints.
|
||||||
- Fidelity evidence: пять локальных build 12340 reference JPG откалибровали terrain/ADT/M2/WMO/liquid viewpoints; automated paired-image metrics exist, но synthetic animation/dusk и полный human approval ещё не закрыты.
|
- Fidelity evidence: семь локальных build 12340 reference JPG покрывают terrain/ADT/M2/WMO/liquid/sky и реальный `GryphonRoost01`; automated cold/warm metrics имеют `compared=14`, `missing=0`. Sky использует приблизительно paired camera, а animated-M2 остаётся asset-paired/placement-unpaired из-за synthetic probe.
|
||||||
- Performance budgets: M00 report records cold/warm p95 and max hitch; no final acceptance threshold yet.
|
- Performance budgets: M00 report records cold/warm p95 and max hitch; no final acceptance threshold yet.
|
||||||
- Manual diagnostics: [`../RENDER_BASELINE.md`](../RENDER_BASELINE.md) and [`../../RENDER.md`](../../RENDER.md).
|
- Manual diagnostics: [`../RENDER_BASELINE.md`](../RENDER_BASELINE.md) and [`../../RENDER.md`](../../RENDER.md).
|
||||||
|
|
||||||
@@ -233,6 +233,8 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
|
|||||||
- The first Goldshire pose grid was invalidated by WMO readiness: the inn is absent after 2 seconds but visible after the manifest-standard 8-second wait. Pose comparisons must stabilize asynchronous checkpoint content first.
|
- The first Goldshire pose grid was invalidated by WMO readiness: the inn is absent after 2 seconds but visible after the manifest-standard 8-second wait. Pose comparisons must stabilize asynchronous checkpoint content first.
|
||||||
- A ready-scene Goldshire 3x3 grid improved mean error from `0.101402` at zero offsets to `0.087952` at yaw `10`/pitch `-10`; pitch/FOV refinement remains required before manifest calibration.
|
- A ready-scene Goldshire 3x3 grid improved mean error from `0.101402` at zero offsets to `0.087952` at yaw `10`/pitch `-10`; pitch/FOV refinement remains required before manifest calibration.
|
||||||
- Joint Goldshire pitch/FOV refinement had no human-approved optimum: full-frame error preferred grass-heavy FOV 62/pitch -25, while FOV 38/pitch -10 better matched building scale. Landmark/region scoring is required before calibration.
|
- Joint Goldshire pitch/FOV refinement had no human-approved optimum: full-frame error preferred grass-heavy FOV 62/pitch -25, while FOV 38/pitch -10 better matched building scale. Landmark/region scoring is required before calibration.
|
||||||
|
- Common-region scoring `[300,100,1400,650]` moved Goldshire's numeric optimum to FOV 38/yaw 10/pitch -15, but human inspection still found an oversized/cropped facade; semantic landmarks or masks remain required.
|
||||||
|
- Final paired audit compared fourteen cold/warm images for seven private build 12340 references with zero missing pairs; mean error ranges `0.066893..0.176955`. Animated-M2 native motion is observed across two original-client phases, but its real Lakeshire placement differs from the synthetic Goldshire probe. Sky-transition evidence uses the checkpoint map position/time with manually approved approximate framing.
|
||||||
- D3D12 descriptor issues remain; the capture-path anonymous `RefCounted` shutdown leak is regression-covered by a clean verbose dry-run, while other RID/resource diagnostics still require independent evidence.
|
- D3D12 descriptor issues remain; the capture-path anonymous `RefCounted` shutdown leak is regression-covered by a clean verbose dry-run, while other RID/resource diagnostics still require independent evidence.
|
||||||
- M2/WMO/material/particle/ribbon/portal parity incomplete.
|
- M2/WMO/material/particle/ribbon/portal parity incomplete.
|
||||||
- Public API is mostly exported configuration rather than stable contracts.
|
- Public API is mostly exported configuration rather than stable contracts.
|
||||||
@@ -262,5 +264,6 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
|
|||||||
- [`../../RENDER.md`](../../RENDER.md)
|
- [`../../RENDER.md`](../../RENDER.md)
|
||||||
- [`../RENDER_BASELINE.md`](../RENDER_BASELINE.md)
|
- [`../RENDER_BASELINE.md`](../RENDER_BASELINE.md)
|
||||||
- [`../CAMERA_POSE_SWEEP.md`](../CAMERA_POSE_SWEEP.md)
|
- [`../CAMERA_POSE_SWEEP.md`](../CAMERA_POSE_SWEEP.md)
|
||||||
|
- [`../M00_FINAL_PAIRED_EVIDENCE.md`](../M00_FINAL_PAIRED_EVIDENCE.md)
|
||||||
- [`../../targets/roadmap/02-rendering-and-graphics.md`](../../targets/roadmap/02-rendering-and-graphics.md)
|
- [`../../targets/roadmap/02-rendering-and-graphics.md`](../../targets/roadmap/02-rendering-and-graphics.md)
|
||||||
- [`../GODOT_BEST_PRACTICES.md`](../GODOT_BEST_PRACTICES.md)
|
- [`../GODOT_BEST_PRACTICES.md`](../GODOT_BEST_PRACTICES.md)
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ extends SceneTree
|
|||||||
## Compares original-client JPG/PNG references with OpenWC checkpoint PNGs.
|
## Compares original-client JPG/PNG references with OpenWC checkpoint PNGs.
|
||||||
## Usage: godot --headless --path . --script res://src/tools/compare_render_checkpoints.gd --
|
## Usage: godot --headless --path . --script res://src/tools/compare_render_checkpoints.gd --
|
||||||
## --reference <directory> --candidate <directory> [--output <report.json>]
|
## --reference <directory> --candidate <directory> [--output <report.json>]
|
||||||
|
## [--region x,y,width,height]
|
||||||
## [--pixel-threshold 0.04] [--mean-threshold 0.015] [--changed-ratio-threshold 0.01]
|
## [--pixel-threshold 0.04] [--mean-threshold 0.015] [--changed-ratio-threshold 0.01]
|
||||||
## Use --self-test for a synthetic regression check that writes only under user://.
|
## Use --self-test for a synthetic regression check that writes only under user://.
|
||||||
|
|
||||||
@@ -56,13 +57,18 @@ func _parse_arguments(raw_arguments: PackedStringArray) -> Dictionary:
|
|||||||
parsed.self_test = true
|
parsed.self_test = true
|
||||||
index += 1
|
index += 1
|
||||||
continue
|
continue
|
||||||
if argument not in ["--reference", "--candidate", "--output", "--only", "--pixel-threshold", "--mean-threshold", "--changed-ratio-threshold"]:
|
if argument not in ["--reference", "--candidate", "--output", "--only", "--region", "--pixel-threshold", "--mean-threshold", "--changed-ratio-threshold"]:
|
||||||
return {"error": "unknown argument: %s" % argument}
|
return {"error": "unknown argument: %s" % argument}
|
||||||
if index + 1 >= raw_arguments.size():
|
if index + 1 >= raw_arguments.size():
|
||||||
return {"error": "missing value for %s" % argument}
|
return {"error": "missing value for %s" % argument}
|
||||||
var key := argument.trim_prefix("--").replace("-", "_")
|
var key := argument.trim_prefix("--").replace("-", "_")
|
||||||
var value := raw_arguments[index + 1]
|
var value := raw_arguments[index + 1]
|
||||||
if key.ends_with("threshold"):
|
if key == "region":
|
||||||
|
var region_parse_result := _parse_comparison_region(value)
|
||||||
|
if region_parse_result.has("error"):
|
||||||
|
return region_parse_result
|
||||||
|
parsed[key] = region_parse_result.region
|
||||||
|
elif key.ends_with("threshold"):
|
||||||
if not value.is_valid_float():
|
if not value.is_valid_float():
|
||||||
return {"error": "%s requires a number" % argument}
|
return {"error": "%s requires a number" % argument}
|
||||||
parsed[key] = float(value)
|
parsed[key] = float(value)
|
||||||
@@ -72,6 +78,24 @@ func _parse_arguments(raw_arguments: PackedStringArray) -> Dictionary:
|
|||||||
return parsed
|
return parsed
|
||||||
|
|
||||||
|
|
||||||
|
func _parse_comparison_region(value: String) -> Dictionary:
|
||||||
|
var components := value.split(",", false)
|
||||||
|
if components.size() != 4:
|
||||||
|
return {"error": "--region requires x,y,width,height"}
|
||||||
|
for component in components:
|
||||||
|
if not component.strip_edges().is_valid_int():
|
||||||
|
return {"error": "--region components must be integers"}
|
||||||
|
var region := Rect2i(
|
||||||
|
int(components[0]),
|
||||||
|
int(components[1]),
|
||||||
|
int(components[2]),
|
||||||
|
int(components[3])
|
||||||
|
)
|
||||||
|
if region.position.x < 0 or region.position.y < 0 or region.size.x <= 0 or region.size.y <= 0:
|
||||||
|
return {"error": "--region requires non-negative x/y and positive width/height"}
|
||||||
|
return {"region": region}
|
||||||
|
|
||||||
|
|
||||||
func _compare_directories(reference_directory: String, candidate_directory: String, options: Dictionary) -> Dictionary:
|
func _compare_directories(reference_directory: String, candidate_directory: String, options: Dictionary) -> Dictionary:
|
||||||
reference_directory = ProjectSettings.globalize_path(reference_directory)
|
reference_directory = ProjectSettings.globalize_path(reference_directory)
|
||||||
candidate_directory = ProjectSettings.globalize_path(candidate_directory)
|
candidate_directory = ProjectSettings.globalize_path(candidate_directory)
|
||||||
@@ -108,6 +132,7 @@ func _compare_directories(reference_directory: String, candidate_directory: Stri
|
|||||||
"pixel_threshold": options.pixel_threshold,
|
"pixel_threshold": options.pixel_threshold,
|
||||||
"mean_threshold": options.mean_threshold,
|
"mean_threshold": options.mean_threshold,
|
||||||
"changed_ratio_threshold": options.changed_ratio_threshold,
|
"changed_ratio_threshold": options.changed_ratio_threshold,
|
||||||
|
"comparison_region": _region_array(options.get("region", Rect2i())),
|
||||||
"compared_count": results.size() - missing_count,
|
"compared_count": results.size() - missing_count,
|
||||||
"failed_count": failed_count,
|
"failed_count": failed_count,
|
||||||
"missing_count": missing_count,
|
"missing_count": missing_count,
|
||||||
@@ -127,12 +152,22 @@ func _compare_images(reference_path: String, candidate_path: String, options: Di
|
|||||||
"reference_size": [reference_image.get_width(), reference_image.get_height()],
|
"reference_size": [reference_image.get_width(), reference_image.get_height()],
|
||||||
"candidate_size": [candidate_image.get_width(), candidate_image.get_height()],
|
"candidate_size": [candidate_image.get_width(), candidate_image.get_height()],
|
||||||
}
|
}
|
||||||
|
var comparison_region: Rect2i = options.get("region", Rect2i(Vector2i.ZERO, reference_image.get_size()))
|
||||||
|
if comparison_region == Rect2i():
|
||||||
|
comparison_region = Rect2i(Vector2i.ZERO, reference_image.get_size())
|
||||||
|
var image_region := Rect2i(Vector2i.ZERO, reference_image.get_size())
|
||||||
|
if not image_region.encloses(comparison_region):
|
||||||
|
return {
|
||||||
|
"status": "region_out_of_bounds",
|
||||||
|
"region": _region_array(comparison_region),
|
||||||
|
"image_size": [reference_image.get_width(), reference_image.get_height()],
|
||||||
|
}
|
||||||
|
|
||||||
var error_sum := 0.0
|
var error_sum := 0.0
|
||||||
var changed_pixel_count := 0
|
var changed_pixel_count := 0
|
||||||
var pixel_count := reference_image.get_width() * reference_image.get_height()
|
var pixel_count := comparison_region.size.x * comparison_region.size.y
|
||||||
for y in reference_image.get_height():
|
for y in range(comparison_region.position.y, comparison_region.end.y):
|
||||||
for x in reference_image.get_width():
|
for x in range(comparison_region.position.x, comparison_region.end.x):
|
||||||
var perceptual_error := _perceptual_color_error(reference_image.get_pixel(x, y), candidate_image.get_pixel(x, y))
|
var perceptual_error := _perceptual_color_error(reference_image.get_pixel(x, y), candidate_image.get_pixel(x, y))
|
||||||
error_sum += perceptual_error
|
error_sum += perceptual_error
|
||||||
if perceptual_error > float(options.pixel_threshold):
|
if perceptual_error > float(options.pixel_threshold):
|
||||||
@@ -144,11 +179,18 @@ func _compare_images(reference_path: String, candidate_path: String, options: Di
|
|||||||
"status": "passed" if passed else "threshold_exceeded",
|
"status": "passed" if passed else "threshold_exceeded",
|
||||||
"width": reference_image.get_width(),
|
"width": reference_image.get_width(),
|
||||||
"height": reference_image.get_height(),
|
"height": reference_image.get_height(),
|
||||||
|
"region": _region_array(comparison_region),
|
||||||
"mean_perceptual_error": mean_perceptual_error,
|
"mean_perceptual_error": mean_perceptual_error,
|
||||||
"changed_pixel_ratio": changed_pixel_ratio,
|
"changed_pixel_ratio": changed_pixel_ratio,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
func _region_array(region: Rect2i) -> Array[int]:
|
||||||
|
if region == Rect2i():
|
||||||
|
return []
|
||||||
|
return [region.position.x, region.position.y, region.size.x, region.size.y]
|
||||||
|
|
||||||
|
|
||||||
func _perceptual_color_error(reference_color: Color, candidate_color: Color) -> float:
|
func _perceptual_color_error(reference_color: Color, candidate_color: Color) -> float:
|
||||||
var reference_linear := reference_color.srgb_to_linear()
|
var reference_linear := reference_color.srgb_to_linear()
|
||||||
var candidate_linear := candidate_color.srgb_to_linear()
|
var candidate_linear := candidate_color.srgb_to_linear()
|
||||||
@@ -229,6 +271,22 @@ func _run_self_test() -> int:
|
|||||||
if not identical_report.passed or changed_report.passed:
|
if not identical_report.passed or changed_report.passed:
|
||||||
push_error("RENDER_CHECKPOINT_DIFF SELF_TEST: expected identical pass and changed failure")
|
push_error("RENDER_CHECKPOINT_DIFF SELF_TEST: expected identical pass and changed failure")
|
||||||
return 1
|
return 1
|
||||||
|
var region_options := options.duplicate()
|
||||||
|
region_options["region"] = Rect2i(1, 1, 1, 1)
|
||||||
|
var excluded_change_report := _compare_directories(reference_directory, changed_directory, region_options)
|
||||||
|
if not excluded_change_report.passed or excluded_change_report.comparison_region != [1, 1, 1, 1]:
|
||||||
|
push_error("RENDER_CHECKPOINT_DIFF SELF_TEST: region must exclude pixels outside the rectangle")
|
||||||
|
return 1
|
||||||
|
var out_of_bounds_options := options.duplicate()
|
||||||
|
out_of_bounds_options["region"] = Rect2i(1, 1, 2, 2)
|
||||||
|
var out_of_bounds_result := _compare_images(
|
||||||
|
reference_directory.path_join("synthetic.jpg"),
|
||||||
|
changed_directory.path_join("synthetic__cold_process.png"),
|
||||||
|
out_of_bounds_options
|
||||||
|
)
|
||||||
|
if out_of_bounds_result.status != "region_out_of_bounds":
|
||||||
|
push_error("RENDER_CHECKPOINT_DIFF SELF_TEST: out-of-bounds region must fail")
|
||||||
|
return 1
|
||||||
if reference_image.save_jpg(reference_directory.path_join("ignored.jpg"), 1.0) != OK:
|
if reference_image.save_jpg(reference_directory.path_join("ignored.jpg"), 1.0) != OK:
|
||||||
return 1
|
return 1
|
||||||
var filtered_options := options.duplicate()
|
var filtered_options := options.duplicate()
|
||||||
|
|||||||
@@ -40,6 +40,15 @@ function Invoke-GodotStep {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function Invoke-GodotComparisonStep {
|
||||||
|
param([string]$Name, [string[]]$Arguments)
|
||||||
|
Write-Host "[$Name] $GodotPath $($Arguments -join ' ')"
|
||||||
|
$process = Start-Process -FilePath $GodotPath -ArgumentList $Arguments -NoNewWindow -Wait -PassThru
|
||||||
|
if ($process.ExitCode -notin @(0, 1)) {
|
||||||
|
throw "$Name failed with exit code $($process.ExitCode)"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
Push-Location $repoRoot
|
Push-Location $repoRoot
|
||||||
try {
|
try {
|
||||||
Invoke-GodotStep "project-load" @("--headless", "--path", ".", "--quit")
|
Invoke-GodotStep "project-load" @("--headless", "--path", ".", "--quit")
|
||||||
@@ -80,7 +89,7 @@ try {
|
|||||||
"--viewport-height", $VisualComparisonHeight.ToString()
|
"--viewport-height", $VisualComparisonHeight.ToString()
|
||||||
)
|
)
|
||||||
Invoke-GodotStep "visual-capture" $visualCaptureArgs
|
Invoke-GodotStep "visual-capture" $visualCaptureArgs
|
||||||
Invoke-GodotStep "visual-comparison" @(
|
Invoke-GodotComparisonStep "visual-comparison" @(
|
||||||
"--headless", "--path", ".",
|
"--headless", "--path", ".",
|
||||||
"--script", "res://src/tools/compare_render_checkpoints.gd",
|
"--script", "res://src/tools/compare_render_checkpoints.gd",
|
||||||
"--",
|
"--",
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ param(
|
|||||||
[double[]]$CameraFovValues = @(),
|
[double[]]$CameraFovValues = @(),
|
||||||
[int]$ViewportWidth = 1280,
|
[int]$ViewportWidth = 1280,
|
||||||
[int]$ViewportHeight = 900,
|
[int]$ViewportHeight = 900,
|
||||||
|
[int[]]$ComparisonRegion = @(),
|
||||||
[double]$WaitSeconds = 8.0,
|
[double]$WaitSeconds = 8.0,
|
||||||
[double]$MeasureSeconds = 0.1,
|
[double]$MeasureSeconds = 0.1,
|
||||||
[switch]$PlanOnly
|
[switch]$PlanOnly
|
||||||
@@ -33,6 +34,9 @@ if (-not (Test-Path -LiteralPath $GodotPath)) {
|
|||||||
if (-not (Test-Path -LiteralPath $ReferenceDirectory)) {
|
if (-not (Test-Path -LiteralPath $ReferenceDirectory)) {
|
||||||
throw "Reference directory not found: $ReferenceDirectory"
|
throw "Reference directory not found: $ReferenceDirectory"
|
||||||
}
|
}
|
||||||
|
if ($ComparisonRegion.Count -notin @(0, 4)) {
|
||||||
|
throw "ComparisonRegion requires x,y,width,height"
|
||||||
|
}
|
||||||
|
|
||||||
$invariantCulture = [Globalization.CultureInfo]::InvariantCulture
|
$invariantCulture = [Globalization.CultureInfo]::InvariantCulture
|
||||||
$isJointFovSweep = $CameraFovValues.Count -gt 0
|
$isJointFovSweep = $CameraFovValues.Count -gt 0
|
||||||
@@ -89,6 +93,9 @@ try {
|
|||||||
"--reference", $ReferenceDirectory, "--candidate", $candidateOutput,
|
"--reference", $ReferenceDirectory, "--candidate", $candidateOutput,
|
||||||
"--only", $Checkpoint, "--output", $comparisonReport
|
"--only", $Checkpoint, "--output", $comparisonReport
|
||||||
)
|
)
|
||||||
|
if ($ComparisonRegion.Count -eq 4) {
|
||||||
|
$comparisonArguments += @("--region", ($ComparisonRegion -join ","))
|
||||||
|
}
|
||||||
$comparisonProcess = Start-Process -FilePath $GodotPath -ArgumentList $comparisonArguments -Wait -PassThru -NoNewWindow
|
$comparisonProcess = Start-Process -FilePath $GodotPath -ArgumentList $comparisonArguments -Wait -PassThru -NoNewWindow
|
||||||
if ($comparisonProcess.ExitCode -notin @(0, 1)) {
|
if ($comparisonProcess.ExitCode -notin @(0, 1)) {
|
||||||
throw "Comparison failed for yaw=$yawText pitch=$pitchText"
|
throw "Comparison failed for yaw=$yawText pitch=$pitchText"
|
||||||
@@ -131,6 +138,7 @@ try {
|
|||||||
viewport = @($ViewportWidth, $ViewportHeight)
|
viewport = @($ViewportWidth, $ViewportHeight)
|
||||||
wait_seconds = $WaitSeconds
|
wait_seconds = $WaitSeconds
|
||||||
measure_seconds = $MeasureSeconds
|
measure_seconds = $MeasureSeconds
|
||||||
|
comparison_region = @($ComparisonRegion)
|
||||||
candidates = $rankedCandidates
|
candidates = $rankedCandidates
|
||||||
}
|
}
|
||||||
New-Item -ItemType Directory -Force -Path $absoluteOutput | Out-Null
|
New-Item -ItemType Directory -Force -Path $absoluteOutput | Out-Null
|
||||||
|
|||||||
Reference in New Issue
Block a user