test(M00): record final paired evidence

Reviewed-on: #12

coordination/claims/M00-QAR-FINAL-PAIRED-EVIDENCE-001.md

@@ -0,0 +1,78 @@
+# M00-QAR-FINAL-PAIRED-EVIDENCE-001 — Final paired evidence audit
+
+<!-- OPENWC_CLAIM:M00-QAR-FINAL-PAIRED-EVIDENCE-001:sindo-main-codex:2026-07-14 -->
+
+## Ownership
+
+- Target: M00
+- Program: QAR
+- Owner/Agent ID: sindo-main-codex
+- Branch: `work/sindo-main-codex/m00-final-paired-evidence`
+- Lease expires UTC: 2026-07-14
+- Integrator: milestone integrator
+
+## Outcome
+
+Produce one final WMO-ready paired report for all available build 12340 references and reconcile the remaining M00 acceptance gaps.
+
+## Non-goals
+
+- Achieving renderer parity inside M00.
+- Calibrating unknown manual camera parameters.
+- Marking the target DONE; only the milestone integrator may do so.
+
+## Paths
+
+- Exclusive: paired-run exit handling and final evidence record
+- Shared/hotspots: renderer baseline/module findings
+- Generated/ignored: final captures and reports under `user://`
+
+## Contracts and data
+
+- Public API/events: visual comparison exit 0/1 both count as completed report generation
+- Schema/format version: unchanged
+- Migration/compatibility: comparator infrastructure errors remain fatal
+- Consumers: M00 integrator
+
+## Dependencies
+
+- Requires: merged region diagnostic and five private reference images
+- Blocks: integrator decision on M00 completion
+- External state: private images remain outside Git
+
+## Verification
+
+- Commands: complete paired baseline run, report audit, repository gates
+- Fixtures: five build 12340 references
+- Fidelity evidence: cold/warm comparison metrics and human gap classification
+- Performance budget: existing M00 capture budgets
+
+## Documentation deliverables
+
+- Inline public API docs: PowerShell comparison step semantics
+- Module specification: final evidence status
+- Data-flow diagram: unchanged
+- Sequence/state/dependency diagrams: unchanged
+- Source map/status updates: baseline acceptance summary
+
+## Simplicity and naming
+
+- Important names introduced: `Invoke-GodotComparisonStep`
+- Simplest considered solution: allow only comparator exit codes 0 and 1
+- Rejected complexity/abstractions: suppressing all process failures
+- Unavoidable complexity and justification: threshold exceedance is evidence, not infrastructure failure
+- Measured optimization evidence: not applicable
+
+## Status
+
+- State: ready-for-review
+- Done: paired-run exit contract fixed; full baseline and visual capture completed; ten pairs audited; final acceptance document written
+- Next: integrator decides animated/sky reference policy and GUI RID-risk disposition
+- Blocked by:
+
+## Handoff
+
+- Commit: this work-package commit
+- Results: compared=10, failed=10, missing=0; mean error 0.066892..0.176952; cold/warm gaps stable; full workflow returned success with threshold evidence preserved
+- Remaining risks: no build 12340 animated-M2 or sky-transition reference; GUI renderer RID/resources still reported at exit
+- Documentation updated: `docs/M00_FINAL_PAIRED_EVIDENCE.md`, `docs/modules/world-renderer.md`

coordination/claims/M00-QAR-LANDMARK-REGION-DIFF-001.md

@@ -0,0 +1,78 @@
+# M00-QAR-LANDMARK-REGION-DIFF-001 — Landmark-region visual diff
+
+<!-- OPENWC_CLAIM:M00-QAR-LANDMARK-REGION-DIFF-001:sindo-main-codex:2026-07-14 -->
+
+## Ownership
+
+- Target: M00
+- Program: QAR
+- Owner/Agent ID: sindo-main-codex
+- Branch: `work/sindo-main-codex/m00-landmark-region-diff`
+- Lease expires UTC: 2026-07-14
+- Integrator: milestone integrator
+
+## Outcome
+
+Rank camera candidates inside an approved common screen region so unrelated grass/sky coverage cannot dominate landmark framing.
+
+## Non-goals
+
+- Automatic feature detection or image warping.
+- Different crops for reference and candidate.
+- Changing manifest camera values without human approval.
+
+## Paths
+
+- Exclusive: comparator common-region option and sweep pass-through
+- Shared/hotspots: camera-pose documentation and renderer fidelity findings
+- Generated/ignored: local ROI sweep images/reports
+
+## Contracts and data
+
+- Public API/events: additive comparator `--region x,y,width,height` and sweep `-ComparisonRegion`
+- Schema/format version: report/ranking schema remains version 1 with additive region metadata
+- Migration/compatibility: omitted region preserves full-frame behavior
+- Consumers: M00 paired camera calibration
+
+## Dependencies
+
+- Requires: ready-scene joint pitch/FOV sweep
+- Blocks: landmark-weighted Goldshire framing decision
+- External state: private build 12340 reference
+
+## Verification
+
+- Commands: comparator synthetic ROI regression, invalid-bound regression, sweep plan, bounded real ROI sweep, repository gates
+- Fixtures: synthetic 2x2 images and private Goldshire reference
+- Fidelity evidence: facade-region ranking plus human inspection
+- Performance budget: region comparison visits only selected pixels
+
+## Documentation deliverables
+
+- Inline public API docs: comparator usage and PowerShell parameter
+- Module specification: ROI finding
+- Data-flow diagram: approved region into comparator
+- Sequence/state/dependency diagrams: unchanged
+- Source map/status updates: camera-pose guide
+
+## Simplicity and naming
+
+- Important names introduced: `ComparisonRegion`, `comparison_region`
+- Simplest considered solution: one common integer rectangle
+- Rejected complexity/abstractions: masks, feature detectors, separate movable crops
+- Unavoidable complexity and justification: full-frame color error is biased by large terrain regions
+- Measured optimization evidence: bounded pixel loop
+
+## Status
+
+- State: ready-for-review
+- Done: common ROI parser/validation/reporting, synthetic inclusion/exclusion regression, sweep pass-through, 15-candidate Goldshire rescoring and human inspection
+- Next: define explicit approved landmark coordinates or mask; do not calibrate manifest from the common rectangle alone
+- Blocked by:
+
+## Handoff
+
+- Commit: this work-package commit
+- Results: ROI [300,100,1400,650] ranked FOV 38/yaw 10/pitch -15 first at mean 0.081777 and ratio 0.697185; visually the facade remains oversized and cropped
+- Remaining risks: rectangle includes non-semantic road/terrain pixels and cannot prove landmark alignment
+- Documentation updated: `docs/CAMERA_POSE_SWEEP.md`, `docs/modules/world-renderer.md`

docs/CAMERA_POSE_SWEEP.md

@@ -54,4 +54,10 @@ A WMO-ready joint grid fixed yaw at 10 degrees and evaluated pitch `[-25, -20, -
 
 The grid therefore has no approved joint optimum. Full-frame color error continues to trade landmark framing against large terrain regions. The manifest remains unchanged. The next registration method must score an approved inn/road/tree landmark region or explicit landmark coordinates rather than the entire image.
 
+## Common landmark region — 2026-07-12
+
+The comparator accepts an optional common screen rectangle through `--region x,y,width,height`; the sweep runner exposes it as `-ComparisonRegion x,y,width,height`. Coordinates are integer pixels in the shared reference/candidate image space. Both images must have identical dimensions and the rectangle must be fully inside them. Omitting the option preserves full-frame behavior.
+
+The existing 15 WMO-ready Goldshire candidates were rescored inside `[300,100,1400,650]`, covering the reference facade/chimney and nearby trees. ROI error ranked FOV 38/yaw 10/pitch -15 first at `0.081777` with changed-pixel ratio `0.697185`. This is an interior pitch candidate and improves on the full-frame grass preference, but human inspection still rejects manifest calibration: the facade is too large, its upper part is cropped, and the rectangle retains substantial road/terrain pixels. A common rectangle is useful diagnostic weighting, not semantic landmark registration. Explicit approved landmark coordinates or a mask remain necessary.
+
 During this run the sweep orchestration was hardened in two ways: expected comparator exit code `1` is now collected through an explicit child process, and viewport dimensions are explicit. A reference/candidate `size_mismatch` now fails before ranking instead of producing an empty metric.

docs/M00_FINAL_PAIRED_EVIDENCE.md

@@ -0,0 +1,60 @@
+# M00 Final Paired Evidence — 2026-07-12
+
+## Scope and result
+
+The final local M00 audit used Godot 4.6.1, the existing cache state, the manifest-standard 8-second readiness wait, and `2560x1440` visual candidates. Five private WoW 3.3.5a build 12340 references produced ten cold/warm comparisons with no missing pair. Private images and generated reports remain outside Git.
+
+The command completed even though every pair exceeded the strict regression tolerance. Comparator exit code `1` means a valid report with visual gaps; only codes outside `0/1` are infrastructure failures.
+
+```powershell
+.\tools\run_render_baseline.ps1 `
+  -Output user://render_baseline_m00_final_2026-07-12 `
+  -CacheState existing `
+  -WaitSeconds 8 `
+  -MeasureSeconds 0.5 `
+  -ReferenceCheckpointDirectory .\sources\OpenWCReferenceCheckpoints `
+  -VisualComparisonReport user://render_baseline_m00_final_2026-07-12/visual_comparison.json `
+  -VisualComparisonWidth 2560 `
+  -VisualComparisonHeight 1440
+```
+
+## Paired metrics
+
+| Checkpoint | Pass | Mean perceptual error | Changed-pixel ratio | Classification |
+|---|---|---:|---:|---|
+| `elwynn_adt_boundary` | cold | 0.087347 | 0.634984 | Composition plus terrain/vegetation/material gaps |
+| `elwynn_adt_boundary` | warm | 0.087255 | 0.634598 | Stable against revisit |
+| `elwynn_terrain_overview` | cold | 0.174136 | 0.851806 | Largest composition/framing gap |
+| `elwynn_terrain_overview` | warm | 0.176952 | 0.861275 | Stable gap, not a cold-load artifact |
+| `elwynn_waterfall_liquid` | cold | 0.088526 | 0.706897 | Framing, terrain, liquid and lighting gaps |
+| `elwynn_waterfall_liquid` | warm | 0.087490 | 0.695281 | Stable against revisit |
+| `goldshire_dense_m2` | cold | 0.066892 | 0.575205 | Smallest available paired gap; vegetation/material differences remain |
+| `goldshire_dense_m2` | warm | 0.066927 | 0.575547 | Stable against revisit |
+| `goldshire_inn_large_wmo` | cold | 0.101230 | 0.701836 | WMO is ready; manual framing and presentation gaps remain |
+| `goldshire_inn_large_wmo` | warm | 0.101203 | 0.701855 | Stable against revisit |
+
+Cold and warm values are close for all five references. The paired gaps are therefore not explained by first-visit cache readiness. Goldshire timing evidence separately demonstrated that the 8-second wait is necessary for WMO publication.
+
+## Fidelity interpretation
+
+M00 now has reproducible quantitative evidence for terrain, ADT boundary, dense M2, large WMO and liquid checkpoints. The reports demonstrate current differences; they do not claim renderer parity. Camera calibration experiments also show that unknown manual yaw/pitch/FOV cannot be recovered reliably from full-frame color error alone. Region scoring reduces terrain bias but still requires semantic landmark approval.
+
+Two manifest checkpoints still lack original-client paired references:
+
+- `gryphon_roost_native_animation` is a synthetic native-animation probe and has no matching build 12340 fixture.
+- `elwynn_sky_dusk` has no approved build 12340 sky-transition fixture at the same time/weather state.
+
+These must either receive private reference evidence or be explicitly accepted by the milestone integrator as unpaired diagnostic coverage. Until that decision, the target remains `ACTIVE`.
+
+## Operational risks
+
+The full GUI capture completed and wrote both reports, but Godot still emitted renderer RID/resource leak diagnostics at exit. The earlier anonymous headless `RefCounted` leak is fixed; GUI renderer allocations are a distinct known risk. They do not invalidate the written images or metrics, but must remain recorded rather than hidden.
+
+## Integrator decision checklist
+
+- Five available reference checkpoints: paired cold/warm report complete, missing pairs zero.
+- Animated M2 fixture: missing or explicitly accepted as synthetic-only.
+- Sky-transition fixture: missing or explicitly accepted as unpaired.
+- Renderer visual differences: registered as gaps, not parity claims.
+- GUI renderer RID/resource diagnostics: accepted as a tracked risk or assigned a follow-up blocker.
+- Target status and `OPENWC_TARGET_DONE` marker: integrator-only action.

docs/modules/world-renderer.md

@@ -233,6 +233,8 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
 - The first Goldshire pose grid was invalidated by WMO readiness: the inn is absent after 2 seconds but visible after the manifest-standard 8-second wait. Pose comparisons must stabilize asynchronous checkpoint content first.
 - A ready-scene Goldshire 3x3 grid improved mean error from `0.101402` at zero offsets to `0.087952` at yaw `10`/pitch `-10`; pitch/FOV refinement remains required before manifest calibration.
 - Joint Goldshire pitch/FOV refinement had no human-approved optimum: full-frame error preferred grass-heavy FOV 62/pitch -25, while FOV 38/pitch -10 better matched building scale. Landmark/region scoring is required before calibration.
+- Common-region scoring `[300,100,1400,650]` moved Goldshire's numeric optimum to FOV 38/yaw 10/pitch -15, but human inspection still found an oversized/cropped facade; semantic landmarks or masks remain required.
+- Final paired audit compared ten cold/warm images for five private build 12340 references with zero missing pairs; mean error ranges `0.066892..0.176952`. Animated-M2 and sky-transition references remain an integrator acceptance decision.
 - D3D12 descriptor issues remain; the capture-path anonymous `RefCounted` shutdown leak is regression-covered by a clean verbose dry-run, while other RID/resource diagnostics still require independent evidence.
 - M2/WMO/material/particle/ribbon/portal parity incomplete.
 - Public API is mostly exported configuration rather than stable contracts.
@@ -262,5 +264,6 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
 - [`../../RENDER.md`](../../RENDER.md)
 - [`../RENDER_BASELINE.md`](../RENDER_BASELINE.md)
 - [`../CAMERA_POSE_SWEEP.md`](../CAMERA_POSE_SWEEP.md)
+- [`../M00_FINAL_PAIRED_EVIDENCE.md`](../M00_FINAL_PAIRED_EVIDENCE.md)
 - [`../../targets/roadmap/02-rendering-and-graphics.md`](../../targets/roadmap/02-rendering-and-graphics.md)
 - [`../GODOT_BEST_PRACTICES.md`](../GODOT_BEST_PRACTICES.md)

src/tools/compare_render_checkpoints.gd

@@ -3,6 +3,7 @@ extends SceneTree
 ## Compares original-client JPG/PNG references with OpenWC checkpoint PNGs.
 ## Usage: godot --headless --path . --script res://src/tools/compare_render_checkpoints.gd --
 ##   --reference <directory> --candidate <directory> [--output <report.json>]
+##   [--region x,y,width,height]
 ##   [--pixel-threshold 0.04] [--mean-threshold 0.015] [--changed-ratio-threshold 0.01]
 ## Use --self-test for a synthetic regression check that writes only under user://.
 
@@ -56,13 +57,18 @@ func _parse_arguments(raw_arguments: PackedStringArray) -> Dictionary:
 			parsed.self_test = true
 			index += 1
 			continue
-		if argument not in ["--reference", "--candidate", "--output", "--only", "--pixel-threshold", "--mean-threshold", "--changed-ratio-threshold"]:
+		if argument not in ["--reference", "--candidate", "--output", "--only", "--region", "--pixel-threshold", "--mean-threshold", "--changed-ratio-threshold"]:
 			return {"error": "unknown argument: %s" % argument}
 		if index + 1 >= raw_arguments.size():
 			return {"error": "missing value for %s" % argument}
 		var key := argument.trim_prefix("--").replace("-", "_")
 		var value := raw_arguments[index + 1]
-		if key.ends_with("threshold"):
+		if key == "region":
+			var region_parse_result := _parse_comparison_region(value)
+			if region_parse_result.has("error"):
+				return region_parse_result
+			parsed[key] = region_parse_result.region
+		elif key.ends_with("threshold"):
 			if not value.is_valid_float():
 				return {"error": "%s requires a number" % argument}
 			parsed[key] = float(value)
@@ -72,6 +78,24 @@ func _parse_arguments(raw_arguments: PackedStringArray) -> Dictionary:
 	return parsed
 
 
+func _parse_comparison_region(value: String) -> Dictionary:
+	var components := value.split(",", false)
+	if components.size() != 4:
+		return {"error": "--region requires x,y,width,height"}
+	for component in components:
+		if not component.strip_edges().is_valid_int():
+			return {"error": "--region components must be integers"}
+	var region := Rect2i(
+		int(components[0]),
+		int(components[1]),
+		int(components[2]),
+		int(components[3])
+	)
+	if region.position.x < 0 or region.position.y < 0 or region.size.x <= 0 or region.size.y <= 0:
+		return {"error": "--region requires non-negative x/y and positive width/height"}
+	return {"region": region}
+
+
 func _compare_directories(reference_directory: String, candidate_directory: String, options: Dictionary) -> Dictionary:
 	reference_directory = ProjectSettings.globalize_path(reference_directory)
 	candidate_directory = ProjectSettings.globalize_path(candidate_directory)
@@ -108,6 +132,7 @@ func _compare_directories(reference_directory: String, candidate_directory: Stri
 		"pixel_threshold": options.pixel_threshold,
 		"mean_threshold": options.mean_threshold,
 		"changed_ratio_threshold": options.changed_ratio_threshold,
+		"comparison_region": _region_array(options.get("region", Rect2i())),
 		"compared_count": results.size() - missing_count,
 		"failed_count": failed_count,
 		"missing_count": missing_count,
@@ -127,12 +152,22 @@ func _compare_images(reference_path: String, candidate_path: String, options: Di
 			"reference_size": [reference_image.get_width(), reference_image.get_height()],
 			"candidate_size": [candidate_image.get_width(), candidate_image.get_height()],
 		}
+	var comparison_region: Rect2i = options.get("region", Rect2i(Vector2i.ZERO, reference_image.get_size()))
+	if comparison_region == Rect2i():
+		comparison_region = Rect2i(Vector2i.ZERO, reference_image.get_size())
+	var image_region := Rect2i(Vector2i.ZERO, reference_image.get_size())
+	if not image_region.encloses(comparison_region):
+		return {
+			"status": "region_out_of_bounds",
+			"region": _region_array(comparison_region),
+			"image_size": [reference_image.get_width(), reference_image.get_height()],
+		}
 
 	var error_sum := 0.0
 	var changed_pixel_count := 0
-	var pixel_count := reference_image.get_width() * reference_image.get_height()
-	for y in reference_image.get_height():
-		for x in reference_image.get_width():
+	var pixel_count := comparison_region.size.x * comparison_region.size.y
+	for y in range(comparison_region.position.y, comparison_region.end.y):
+		for x in range(comparison_region.position.x, comparison_region.end.x):
 			var perceptual_error := _perceptual_color_error(reference_image.get_pixel(x, y), candidate_image.get_pixel(x, y))
 			error_sum += perceptual_error
 			if perceptual_error > float(options.pixel_threshold):
@@ -144,11 +179,18 @@ func _compare_images(reference_path: String, candidate_path: String, options: Di
 		"status": "passed" if passed else "threshold_exceeded",
 		"width": reference_image.get_width(),
 		"height": reference_image.get_height(),
+		"region": _region_array(comparison_region),
 		"mean_perceptual_error": mean_perceptual_error,
 		"changed_pixel_ratio": changed_pixel_ratio,
 	}
 
 
+func _region_array(region: Rect2i) -> Array[int]:
+	if region == Rect2i():
+		return []
+	return [region.position.x, region.position.y, region.size.x, region.size.y]
+
+
 func _perceptual_color_error(reference_color: Color, candidate_color: Color) -> float:
 	var reference_linear := reference_color.srgb_to_linear()
 	var candidate_linear := candidate_color.srgb_to_linear()
@@ -229,6 +271,22 @@ func _run_self_test() -> int:
 	if not identical_report.passed or changed_report.passed:
 		push_error("RENDER_CHECKPOINT_DIFF SELF_TEST: expected identical pass and changed failure")
 		return 1
+	var region_options := options.duplicate()
+	region_options["region"] = Rect2i(1, 1, 1, 1)
+	var excluded_change_report := _compare_directories(reference_directory, changed_directory, region_options)
+	if not excluded_change_report.passed or excluded_change_report.comparison_region != [1, 1, 1, 1]:
+		push_error("RENDER_CHECKPOINT_DIFF SELF_TEST: region must exclude pixels outside the rectangle")
+		return 1
+	var out_of_bounds_options := options.duplicate()
+	out_of_bounds_options["region"] = Rect2i(1, 1, 2, 2)
+	var out_of_bounds_result := _compare_images(
+		reference_directory.path_join("synthetic.jpg"),
+		changed_directory.path_join("synthetic__cold_process.png"),
+		out_of_bounds_options
+	)
+	if out_of_bounds_result.status != "region_out_of_bounds":
+		push_error("RENDER_CHECKPOINT_DIFF SELF_TEST: out-of-bounds region must fail")
+		return 1
 	if reference_image.save_jpg(reference_directory.path_join("ignored.jpg"), 1.0) != OK:
 		return 1
 	var filtered_options := options.duplicate()

tools/run_render_baseline.ps1

@@ -40,6 +40,15 @@ function Invoke-GodotStep {
     }
 }
 
+function Invoke-GodotComparisonStep {
+    param([string]$Name, [string[]]$Arguments)
+    Write-Host "[$Name] $GodotPath $($Arguments -join ' ')"
+    $process = Start-Process -FilePath $GodotPath -ArgumentList $Arguments -NoNewWindow -Wait -PassThru
+    if ($process.ExitCode -notin @(0, 1)) {
+        throw "$Name failed with exit code $($process.ExitCode)"
+    }
+}
+
 Push-Location $repoRoot
 try {
     Invoke-GodotStep "project-load" @("--headless", "--path", ".", "--quit")
@@ -80,7 +89,7 @@ try {
 			"--viewport-height", $VisualComparisonHeight.ToString()
 		)
 		Invoke-GodotStep "visual-capture" $visualCaptureArgs
-        Invoke-GodotStep "visual-comparison" @(
+        Invoke-GodotComparisonStep "visual-comparison" @(
             "--headless", "--path", ".",
             "--script", "res://src/tools/compare_render_checkpoints.gd",
             "--",

tools/sweep_render_checkpoint_camera_pose.ps1

@@ -12,6 +12,7 @@ param(
     [double[]]$CameraFovValues = @(),
     [int]$ViewportWidth = 1280,
     [int]$ViewportHeight = 900,
+    [int[]]$ComparisonRegion = @(),
     [double]$WaitSeconds = 8.0,
     [double]$MeasureSeconds = 0.1,
     [switch]$PlanOnly
@@ -33,6 +34,9 @@ if (-not (Test-Path -LiteralPath $GodotPath)) {
 if (-not (Test-Path -LiteralPath $ReferenceDirectory)) {
     throw "Reference directory not found: $ReferenceDirectory"
 }
+if ($ComparisonRegion.Count -notin @(0, 4)) {
+    throw "ComparisonRegion requires x,y,width,height"
+}
 
 $invariantCulture = [Globalization.CultureInfo]::InvariantCulture
 $isJointFovSweep = $CameraFovValues.Count -gt 0
@@ -89,6 +93,9 @@ try {
                 "--reference", $ReferenceDirectory, "--candidate", $candidateOutput,
                 "--only", $Checkpoint, "--output", $comparisonReport
             )
+            if ($ComparisonRegion.Count -eq 4) {
+                $comparisonArguments += @("--region", ($ComparisonRegion -join ","))
+            }
             $comparisonProcess = Start-Process -FilePath $GodotPath -ArgumentList $comparisonArguments -Wait -PassThru -NoNewWindow
             if ($comparisonProcess.ExitCode -notin @(0, 1)) {
                 throw "Comparison failed for yaw=$yawText pitch=$pitchText"
@@ -131,6 +138,7 @@ try {
         viewport = @($ViewportWidth, $ViewportHeight)
         wait_seconds = $WaitSeconds
         measure_seconds = $MeasureSeconds
+        comparison_region = @($ComparisonRegion)
         candidates = $rankedCandidates
     }
     New-Item -ItemType Directory -Force -Path $absoluteOutput | Out-Null