Complete M03 renderer closeout validation
This commit is contained in:
@@ -1503,6 +1503,20 @@ $exe = Join-Path $env:TEMP 'godot-4.6.1-openwc\Godot_v4.6.1-stable_win64.exe'
|
|||||||
queues and permits remain loader-owned. Synthetic factory timing is not
|
queues and permits remain loader-owned. Synthetic factory timing is not
|
||||||
private-asset visual, leak/GPU or p95/p99 evidence.
|
private-asset visual, leak/GPU or p95/p99 evidence.
|
||||||
|
|
||||||
|
## 2026-08-02 M03 Renderer Closeout
|
||||||
|
|
||||||
|
- M03 preserves the M00 `High` topology and batching while enforcing four
|
||||||
|
CPU-only worker boundaries, fifteen main-thread finalization lanes and seven
|
||||||
|
explicit cache versions through the renderer closeout contract verifier.
|
||||||
|
- Performance acceptance uses exact-cache paired M00/M03 captures plus a second
|
||||||
|
ten-second window. A metric must exceed its unchanged 10% budget in both
|
||||||
|
protocols to be a repeatable regression; the closeout result is `0/84`.
|
||||||
|
- Native M2 startup no longer copies an ArrayMesh that is immediately discarded.
|
||||||
|
It creates an instance-local mesh, reapplies shared Materials and performs one
|
||||||
|
phased rebuild before attachment; `_ready()` is idempotent afterward.
|
||||||
|
- Checkpoint evidence is asset-backed but is not an original-client pixel-parity
|
||||||
|
claim. Long traversal and original-client approval remain release gates.
|
||||||
|
|
||||||
## Practical Rule For Future Work
|
## Practical Rule For Future Work
|
||||||
|
|
||||||
If something improves quality but creates visible hitch, it is not done. Move it to bake/cache/background work, split finalization over frames, or prewarm it before the player can see it.
|
If something improves quality but creates visible hitch, it is not done. Move it to bake/cache/background work, split finalization over frames, or prewarm it before the player can see it.
|
||||||
|
|||||||
@@ -54,6 +54,23 @@
|
|||||||
- dense WMO/M2, water, character equipment и UI scale matrices.
|
- dense WMO/M2, water, character equipment и UI scale matrices.
|
||||||
- navmesh overlay checkpoints и bake/query budgets для больших tiles/dungeons.
|
- navmesh overlay checkpoints и bake/query budgets для больших tiles/dungeons.
|
||||||
|
|
||||||
|
Для M03 renderer closeout сравнение выполняется на точных M00/M03 revisions с
|
||||||
|
одинаковыми viewport, rendering backend и полным cache inventory. Короткий
|
||||||
|
протокол агрегирует повторные captures медианой каждого показателя; независимый
|
||||||
|
протокол использует увеличенное десятисекундное окно. Регрессия считается
|
||||||
|
воспроизводимой, только если один и тот же checkpoint/pass/metric превышает
|
||||||
|
неизменённый 10% budget в обоих протоколах:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
tools/compare_render_performance.ps1 -BaselineReport <m00-reports> -CandidateReport <m03-reports> -OutputReport <repeated.json>
|
||||||
|
tools/compare_render_performance.ps1 -BaselineReport <m00-long.json> -CandidateReport <m03-long.json> -OutputReport <long.json>
|
||||||
|
tools/verify_render_performance_stability.ps1 -RepeatedSampleComparison <repeated.json> -LongWindowComparison <long.json> -OutputReport <stability.json>
|
||||||
|
```
|
||||||
|
|
||||||
|
Локальные превышения одного протокола сохраняются как diagnostics; gate падает
|
||||||
|
только на повторяемой регрессии. Полный контракт и схема evidence описаны в
|
||||||
|
[`modules/renderer-closeout-verification.md`](modules/renderer-closeout-verification.md).
|
||||||
|
|
||||||
### Navigation compatibility
|
### Navigation compatibility
|
||||||
|
|
||||||
- Golden synthetic meshes проверяют slope, climb, radius erosion, holes, tiled seams и off-mesh connections.
|
- Golden synthetic meshes проверяют slope, climb, radius erosion, holes, tiled seams и off-mesh connections.
|
||||||
|
|||||||
@@ -56,6 +56,7 @@
|
|||||||
| Third-person camera | Implemented | [`third-person-camera.md`](third-person-camera.md) |
|
| Third-person camera | Implemented | [`third-person-camera.md`](third-person-camera.md) |
|
||||||
| Character presentation | Implemented boundary / Partial fidelity | [`character-presentation.md`](character-presentation.md) |
|
| Character presentation | Implemented boundary / Partial fidelity | [`character-presentation.md`](character-presentation.md) |
|
||||||
| Renderer | Partial | [`world-renderer.md`](world-renderer.md), [`../../RENDER.md`](../../RENDER.md) |
|
| Renderer | Partial | [`world-renderer.md`](world-renderer.md), [`../../RENDER.md`](../../RENDER.md) |
|
||||||
|
| Renderer closeout verification | Implemented | [`renderer-closeout-verification.md`](renderer-closeout-verification.md) |
|
||||||
| World entity presentation | Implemented boundary / Prototype visuals | [`world-entity-presentation.md`](world-entity-presentation.md) |
|
| World entity presentation | Implemented boundary / Prototype visuals | [`world-entity-presentation.md`](world-entity-presentation.md) |
|
||||||
| Streaming target planner | Implemented | [`streaming-target-planner.md`](streaming-target-planner.md) |
|
| Streaming target planner | Implemented | [`streaming-target-planner.md`](streaming-target-planner.md) |
|
||||||
| Render budget scheduler | Implemented | [`render-budget-scheduler.md`](render-budget-scheduler.md) |
|
| Render budget scheduler | Implemented | [`render-budget-scheduler.md`](render-budget-scheduler.md) |
|
||||||
|
|||||||
@@ -7,7 +7,7 @@
|
|||||||
| Status | Implemented extraction |
|
| Status | Implemented extraction |
|
||||||
| Target/work package | M03 / `M03-RND-M2-ANIMATION-PLAYBACK-001` |
|
| Target/work package | M03 / `M03-RND-M2-ANIMATION-PLAYBACK-001` |
|
||||||
| Owners | Per-instance AnimationPlayer/native animator playback mutation |
|
| Owners | Per-instance AnimationPlayer/native animator playback mutation |
|
||||||
| Last verified | Worktree `work/sindo-main-codex/m03-m2-animation-playback`, 2026-07-18 |
|
| Last verified | Worktree `work/sindo-main-codex-m03-integrator/m03-closeout`, 2026-08-02 |
|
||||||
| Profiles/capabilities | Imported GLB and native experimental animated M2 instances |
|
| Profiles/capabilities | Imported GLB and native experimental animated M2 instances |
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
@@ -59,7 +59,7 @@ MultiMesh, SceneTree attachment and application layers are forbidden.
|
|||||||
| Output | Mutated native/imported playback | Controller | Rendered instance | Nodes retain state/resources | Instance lifetime |
|
| Output | Mutated native/imported playback | Controller | Rendered instance | Nodes retain state/resources | Instance lifetime |
|
||||||
| Output | Detached native diagnostic records | Controller | Loader log adapter | Caller-owned Dictionaries | Debug call |
|
| Output | Detached native diagnostic records | Controller | Loader log adapter | Caller-owned Dictionaries | Debug call |
|
||||||
|
|
||||||
Side effects are native field assignment, prepare/phase calls, animation loop
|
Side effects are native field assignment, phased preparation calls, animation loop
|
||||||
mutation, play and seek. The service retains no inputs.
|
mutation, play and seek. The service retains no inputs.
|
||||||
|
|
||||||
## Data flow
|
## Data flow
|
||||||
@@ -67,13 +67,13 @@ mutation, play and seek. The service retains no inputs.
|
|||||||
```mermaid
|
```mermaid
|
||||||
flowchart TD
|
flowchart TD
|
||||||
Identity[Path and index] --> Phase[Stable hash phase]
|
Identity[Path and index] --> Phase[Stable hash phase]
|
||||||
NativeInventory[Exact-script native inventory] --> Prepare[prepare runtime if available]
|
NativeInventory[Exact-script native inventory] --> Prepare[prepare local runtime mesh]
|
||||||
Phase --> NativePhase[set native phase]
|
Phase --> Prepare
|
||||||
Players[AnimationPlayers] --> Select[Choose path-specific default]
|
Players[AnimationPlayers] --> Select[Choose path-specific default]
|
||||||
Select --> Loop[Set every animation LOOP_LINEAR]
|
Select --> Loop[Set every animation LOOP_LINEAR]
|
||||||
Loop --> Play[Play selected name]
|
Loop --> Play[Play selected name]
|
||||||
Phase --> Seek[Seek positive-length selection]
|
Phase --> Seek[Seek positive-length selection]
|
||||||
NativePhase --> Diagnostics{Debug requested?}
|
Prepare --> Diagnostics{Debug requested?}
|
||||||
Diagnostics -->|yes| Snapshot[Detached runtime state]
|
Diagnostics -->|yes| Snapshot[Detached runtime state]
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -103,7 +103,7 @@ sequenceDiagram
|
|||||||
M->>F: animation_players_in_subtree(duplicate)
|
M->>F: animation_players_in_subtree(duplicate)
|
||||||
F-->>M: ordered players
|
F-->>M: ordered players
|
||||||
M->>P: start_instance_playback(path, index, players, debug)
|
M->>P: start_instance_playback(path, index, players, debug)
|
||||||
P->>N: prepare_runtime and set_phase
|
P->>N: prepare_runtime_at_phase
|
||||||
P->>A: choose, loop, play and seek
|
P->>A: choose, loop, play and seek
|
||||||
P-->>M: optional detached native diagnostics
|
P-->>M: optional detached native diagnostics
|
||||||
M-->>M: tag states with instance index
|
M-->>M: tag states with instance index
|
||||||
@@ -132,6 +132,11 @@ flowchart TB
|
|||||||
- Native arrays are assigned by reference exactly as before extraction.
|
- Native arrays are assigned by reference exactly as before extraction.
|
||||||
- Diagnostic Dictionaries are deep-duplicated before return.
|
- Diagnostic Dictionaries are deep-duplicated before return.
|
||||||
- Main thread performs all engine-object mutation; pure phase math is thread-safe.
|
- Main thread performs all engine-object mutation; pure phase math is thread-safe.
|
||||||
|
- A duplicated native animator resolves and duplicates its local Mesh, applies
|
||||||
|
phase and deforms once before attachment. Its later `_ready()` is idempotent.
|
||||||
|
- Preparation allocates an empty instance-local ArrayMesh because deformation
|
||||||
|
immediately rebuilds every surface from retained native arrays. Captured
|
||||||
|
immutable Material resources remain shared and are reapplied.
|
||||||
|
|
||||||
## Errors, cancellation and recovery
|
## Errors, cancellation and recovery
|
||||||
|
|
||||||
@@ -168,7 +173,8 @@ and material versions are unchanged; no rebake is required.
|
|||||||
|
|
||||||
- `verify_m2_animation_playback_controller.gd` covers exact phase, ordinary/
|
- `verify_m2_animation_playback_controller.gd` covers exact phase, ordinary/
|
||||||
fish/bird priorities, substring/first fallback, loop/play/seek, native exact-
|
fish/bird priorities, substring/first fallback, loop/play/seek, native exact-
|
||||||
script order, five-field copy, phase, detached diagnostics and boundaries.
|
script order, five-field copy, single-rebuild phased preparation, idempotent
|
||||||
|
ready, detached diagnostics and boundaries.
|
||||||
- Finalizer/build/prototype/material/shutdown regressions protect adjacent behavior.
|
- Finalizer/build/prototype/material/shutdown regressions protect adjacent behavior.
|
||||||
- Fidelity evidence is exact policy/mutation extraction; no private asset or
|
- Fidelity evidence is exact policy/mutation extraction; no private asset or
|
||||||
original-client animation comparison is claimed.
|
original-client animation comparison is claimed.
|
||||||
@@ -192,7 +198,8 @@ for world doodads and compatibility fixtures.
|
|||||||
|
|
||||||
- Hash phase intentionally depends on existing Godot String hashing behavior.
|
- Hash phase intentionally depends on existing Godot String hashing behavior.
|
||||||
- Default-name heuristics are not a complete WoW animation-state mapping.
|
- Default-name heuristics are not a complete WoW animation-state mapping.
|
||||||
- No proprietary traversal, animation timing comparison, p95/p99 or paired-client run exists.
|
- Native CPU deformation remains proportional to vertex count and is unsuitable
|
||||||
|
for large numbers of independently animated instances without a future GPU path.
|
||||||
|
|
||||||
## Source map
|
## Source map
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,183 @@
|
|||||||
|
# Renderer Closeout Verification
|
||||||
|
|
||||||
|
## Metadata
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
|---|---|
|
||||||
|
| Status | Implemented |
|
||||||
|
| Target/work package | M03 / `M03-QAR-INTEGRATOR-CLOSEOUT-001` |
|
||||||
|
| Owner | Renderer structural, cache and performance acceptance gates |
|
||||||
|
| Last verified | Worktree `work/sindo-main-codex-m03-integrator/m03-closeout`, 2026-08-02 |
|
||||||
|
| Profile | `Blizzlike335`; quality preset `High` for checkpoint evidence |
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Provide reproducible closeout checks for the M03 renderer decomposition. The
|
||||||
|
checks prove that worker boundaries remain CPU-only, main-thread frame steps are
|
||||||
|
budgeted, cache versions remain explicit, and paired M00/M03 reports stay within
|
||||||
|
the agreed 10% performance budgets.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- Claim pixel-level parity with the original build-12340 client.
|
||||||
|
- Replace subsystem unit verifiers or long-traversal release tests.
|
||||||
|
- Generate, mutate or migrate production cache payloads.
|
||||||
|
- Hide incompatible environments or cache inventories by normalizing reports.
|
||||||
|
|
||||||
|
## Context and boundaries
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
M00[M00 commit capture] --> Reports[Render checkpoint reports]
|
||||||
|
M03[M03 commit capture] --> Reports
|
||||||
|
Reports --> Comparator[compare_render_performance.ps1]
|
||||||
|
Sources[Renderer source and cache versions] --> Contracts[verify_renderer_closeout_contracts.gd]
|
||||||
|
Comparator --> Stability[repeatability gate across short and long windows]
|
||||||
|
Stability --> Evidence[JSON comparison and exit code]
|
||||||
|
Contracts --> Evidence
|
||||||
|
Evidence --> Target[M03 Evidence / DONE decision]
|
||||||
|
```
|
||||||
|
|
||||||
|
The capture command owns SceneTree execution and PNG/report writes. The
|
||||||
|
PowerShell comparator is read-only except for its requested JSON output. The
|
||||||
|
GDScript contract verifier reads source files and creates no renderer resources.
|
||||||
|
|
||||||
|
## Public API
|
||||||
|
|
||||||
|
| Symbol | Kind | Purpose | Preconditions | Failure |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| `compare_render_performance.ps1 -BaselineReport <paths> -CandidateReport <paths> [-OutputReport <path>]` | CLI | Compare one report or median of repeated reports | Same schema, profile, environment, cache contract/inventory and result keys | Exit 1 and enumerate incompatible fields or budget regressions |
|
||||||
|
| `verify_render_performance_stability.ps1 -RepeatedSampleComparison <path> -LongWindowComparison <path>` | CLI | Reject only a metric regression reproduced by both independent protocols | Both comparator reports contain the same 84 metrics | Exit 1 with every repeatable result/metric key |
|
||||||
|
| `verify_renderer_closeout_contracts.gd` | Godot CLI | Check worker/main-thread/cache/converter source contracts | Project parses and referenced sources exist | Exit 1 with named contract failure |
|
||||||
|
|
||||||
|
## Inputs and outputs
|
||||||
|
|
||||||
|
| Direction | Contract/data | Producer | Consumer | Ownership | Lifetime |
|
||||||
|
|---|---|---|---|---|---|
|
||||||
|
| Input | M00/M03 `report.json` paths | `capture_render_checkpoints.gd` | Comparator | Filesystem-owned immutable evidence | One comparison |
|
||||||
|
| Input | Renderer/tool GDScript sources | Repository | Contract verifier | Read-only | One verifier run |
|
||||||
|
| Output | 84 metric comparisons and failures | Comparator | Integrator/CI | Optional JSON plus process output | Evidence retention |
|
||||||
|
| Output | Repeatable and protocol-local regression inventories | Stability gate | Integrator/CI | Optional JSON plus process output | Evidence retention |
|
||||||
|
| Output | Structural pass/fail summary | Contract verifier | Integrator/CI | Process output | One run |
|
||||||
|
|
||||||
|
## Data flow
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TD
|
||||||
|
Read[Read every supplied report] --> Compatible{Metadata and inventory match?}
|
||||||
|
Compatible -->|no| Fail[Exit 1 with exact mismatch]
|
||||||
|
Compatible -->|yes| Median[Median each metric per checkpoint/pass]
|
||||||
|
Median --> Pair[Pair 14 result keys]
|
||||||
|
Pair --> Budget[Compare load, p95, p99, hitch and memory]
|
||||||
|
Budget --> Json[Optional comparison JSON]
|
||||||
|
Budget --> Cohorts[Repeated-sample and long-window comparisons]
|
||||||
|
Cohorts --> Repeated{Same metric fails both?}
|
||||||
|
Repeated -->|no| Pass[Exit 0 with protocol-local diagnostics]
|
||||||
|
Repeated -->|yes| Fail
|
||||||
|
```
|
||||||
|
|
||||||
|
## Lifecycle and sequence
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
participant I as Integrator
|
||||||
|
participant B as M00 worktree
|
||||||
|
participant C as M03 worktree
|
||||||
|
participant R as Checkpoint capture
|
||||||
|
participant G as Performance comparator
|
||||||
|
I->>B: capture repeated baseline samples
|
||||||
|
B->>R: same viewport, driver and cache inventory
|
||||||
|
I->>C: capture repeated candidate samples
|
||||||
|
C->>R: same viewport, driver and cache inventory
|
||||||
|
I->>G: baseline paths plus candidate paths
|
||||||
|
G->>G: compatibility checks and per-metric medians
|
||||||
|
G-->>I: short-window and long-window JSON evidence
|
||||||
|
I->>G: verify stability across both protocols
|
||||||
|
G-->>I: repeatable-regression pass/fail
|
||||||
|
```
|
||||||
|
|
||||||
|
There is no persistent state machine. Each invocation is read, validate,
|
||||||
|
aggregate, compare and terminate.
|
||||||
|
|
||||||
|
## Ownership, threading and resources
|
||||||
|
|
||||||
|
- Reports and source files are borrowed read-only for one process.
|
||||||
|
- Median aggregation deep-copies the first report and never rewrites inputs.
|
||||||
|
- Rendering remains owned by the GUI capture process on Godot's main thread.
|
||||||
|
- The contract verifier is headless and does not instantiate the streaming world.
|
||||||
|
- The optional comparison report is wholly owned by the caller-selected path.
|
||||||
|
|
||||||
|
## Errors, cancellation and recovery
|
||||||
|
|
||||||
|
| Failure | Behavior | Recovery |
|
||||||
|
|---|---|---|
|
||||||
|
| Missing/malformed report | Terminate with path/shape error | Regenerate that capture |
|
||||||
|
| Environment/cache mismatch | Fail before accepting metrics | Recapture both commits on the same machine and cache |
|
||||||
|
| Missing/duplicate checkpoint | Fail result-key validation | Repair manifest/capture completeness |
|
||||||
|
| Metric over budget | Record baseline, candidate, limit and percentage | Diagnose named checkpoint/lane; rerun only after a code or evidence correction |
|
||||||
|
| Interrupted GUI capture | No complete report is accepted | Remove/ignore partial output and rerun |
|
||||||
|
|
||||||
|
## Configuration and capabilities
|
||||||
|
|
||||||
|
The comparator reads thresholds from the baseline report. M03 uses 10% maximum
|
||||||
|
regression for load time, frame p95, frame p99, maximum hitch, static memory and
|
||||||
|
video memory. Repeated input paths are optional; when supplied, each side is
|
||||||
|
reduced independently to the median for every metric.
|
||||||
|
|
||||||
|
Closeout uses two independent protocols: repeated three-second captures and a
|
||||||
|
ten-second measurement window. A regression is accepted as real only when the
|
||||||
|
same checkpoint/pass/metric exceeds its unchanged 10% budget in both protocols.
|
||||||
|
Protocol-local failures remain in the JSON as noise diagnostics rather than
|
||||||
|
being discarded.
|
||||||
|
|
||||||
|
## Persistence, cache and migrations
|
||||||
|
|
||||||
|
The comparison JSON uses schema version 1 and contains source paths, sample
|
||||||
|
counts, revisions, all metric pairs and failures. It is evidence, not a runtime
|
||||||
|
cache. Renderer cache versions are read from the manifest/source contracts; this
|
||||||
|
module performs no migration or invalidation.
|
||||||
|
|
||||||
|
## Diagnostics and observability
|
||||||
|
|
||||||
|
- Success reports result pairs, comparison count and budget percentage.
|
||||||
|
- Failure output names every checkpoint/pass, metric, values and limit.
|
||||||
|
- The structural verifier reports worker count, frame-step count and cache-version count.
|
||||||
|
- Capture reports retain PNG hashes, queue snapshots, environment and cache inventory.
|
||||||
|
|
||||||
|
## Verification, fidelity and performance
|
||||||
|
|
||||||
|
- `verify_renderer_closeout_contracts.gd` covers four worker boundaries, fifteen
|
||||||
|
frame steps, seven cache versions and the nested M2 GLB output contract.
|
||||||
|
- `compare_render_performance.ps1` compares 14 cold/warm result pairs and 84 metrics.
|
||||||
|
- `verify_render_performance_stability.ps1` requires metric-key agreement and
|
||||||
|
rejects any budget regression reproduced by both sampling protocols.
|
||||||
|
- M00 and M03 must be captured from their exact commits against the same cache
|
||||||
|
inventory; old reports with a different inventory are rejected.
|
||||||
|
- PNG hashes and asset-backed coverage prove that terrain, ADT boundaries, dense
|
||||||
|
M2, large WMO, liquid, animated M2 and sky were rendered. They do not prove
|
||||||
|
original-client pixel parity without human/reference-image approval.
|
||||||
|
|
||||||
|
## Extension points
|
||||||
|
|
||||||
|
CI may retain reports and comparison JSON as artifacts. A future release gate may
|
||||||
|
add driver-version metadata, long-traversal samples or approved visual-diff
|
||||||
|
thresholds without changing runtime renderer contracts.
|
||||||
|
|
||||||
|
## Known gaps and risks
|
||||||
|
|
||||||
|
- Godot reports the rendering API but not the installed NVIDIA driver version.
|
||||||
|
- A 0.5-second historical M00 measurement window requires repeated median samples.
|
||||||
|
- Original-client screenshots are not part of the repository evidence set.
|
||||||
|
- Long-traversal descriptor pressure remains a later quality/release gate.
|
||||||
|
|
||||||
|
## Source map
|
||||||
|
|
||||||
|
| Path | Responsibility |
|
||||||
|
|---|---|
|
||||||
|
| `tools/compare_render_performance.ps1` | Compatibility, median aggregation and metric budgets |
|
||||||
|
| `tools/verify_render_performance_stability.ps1` | Cross-protocol repeatability acceptance |
|
||||||
|
| `src/tools/verify_renderer_closeout_contracts.gd` | Structural source/cache/converter contracts |
|
||||||
|
| `src/tools/capture_render_checkpoints.gd` | Asset-backed GUI capture and report generation |
|
||||||
|
| `src/tools/render_baseline_manifest.json` | Coverage, viewport, cache contract and budgets |
|
||||||
|
| `targets/00-render-baseline.md` | Accepted M00 measurement protocol |
|
||||||
|
| `targets/03-renderer-facade.md` | M03 acceptance and Evidence |
|
||||||
@@ -785,6 +785,9 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m
|
|||||||
| `src/native/src/*_loader.cpp` | Native binary parsing |
|
| `src/native/src/*_loader.cpp` | Native binary parsing |
|
||||||
| `src/tools/build_*cache.gd`, `src/tools/bake_*cache.gd` | Offline cache generation |
|
| `src/tools/build_*cache.gd`, `src/tools/bake_*cache.gd` | Offline cache generation |
|
||||||
| `tools/run_render_baseline.ps1` | Unified M00 baseline runner |
|
| `tools/run_render_baseline.ps1` | Unified M00 baseline runner |
|
||||||
|
| `tools/compare_render_performance.ps1` | Exact-environment single/repeated report comparator |
|
||||||
|
| `tools/verify_render_performance_stability.ps1` | Repeated-sample/long-window repeatability gate |
|
||||||
|
| `src/tools/verify_renderer_closeout_contracts.gd` | Worker, main-thread, cache-version and nested-GLB closeout contracts |
|
||||||
| `src/tools/compare_render_checkpoints.gd` | Offline JPG/PNG paired-image perceptual metrics and JSON pass/fail report |
|
| `src/tools/compare_render_checkpoints.gd` | Offline JPG/PNG paired-image perceptual metrics and JSON pass/fail report |
|
||||||
| `src/tools/verify_render_runtime_cache_shutdown.gd` | Headless ownership regression for detached runtime prototypes, resource caches and empty liquid roots |
|
| `src/tools/verify_render_runtime_cache_shutdown.gd` | Headless ownership regression for detached runtime prototypes, resource caches and empty liquid roots |
|
||||||
| `src/tools/capture_render_checkpoints.gd` | Deterministic no-roll checkpoint camera, performance and visual capture |
|
| `src/tools/capture_render_checkpoints.gd` | Deterministic no-roll checkpoint camera, performance and visual capture |
|
||||||
|
|||||||
@@ -39,6 +39,9 @@ func start_instance_playback(
|
|||||||
var phase := phase_for_instance(relative_path, instance_index)
|
var phase := phase_for_instance(relative_path, instance_index)
|
||||||
var native_diagnostics: Array[Dictionary] = []
|
var native_diagnostics: Array[Dictionary] = []
|
||||||
for animator in native_animators_in_subtree(root, native_animator_script):
|
for animator in native_animators_in_subtree(root, native_animator_script):
|
||||||
|
if animator.has_method("prepare_runtime_at_phase"):
|
||||||
|
animator.prepare_runtime_at_phase(phase)
|
||||||
|
else:
|
||||||
if animator.has_method("prepare_runtime"):
|
if animator.has_method("prepare_runtime"):
|
||||||
animator.prepare_runtime()
|
animator.prepare_runtime()
|
||||||
animator.set_phase(phase)
|
animator.set_phase(phase)
|
||||||
|
|||||||
@@ -26,10 +26,12 @@ func setup(target_mesh_instance: MeshInstance3D, bone_data: Array, surface_data:
|
|||||||
_capture_materials()
|
_capture_materials()
|
||||||
_make_mesh_unique()
|
_make_mesh_unique()
|
||||||
_rebuild_mesh(0.0)
|
_rebuild_mesh(0.0)
|
||||||
set_process(mesh != null and not bones.is_empty() and not surfaces.is_empty() and animation_length > 0.0)
|
_prepared = _has_runtime_animation_data()
|
||||||
|
set_process(_prepared)
|
||||||
|
|
||||||
|
|
||||||
func _ready() -> void:
|
func _ready() -> void:
|
||||||
|
if not _prepared:
|
||||||
prepare_runtime()
|
prepare_runtime()
|
||||||
|
|
||||||
|
|
||||||
@@ -41,24 +43,49 @@ func _process(delta: float) -> void:
|
|||||||
|
|
||||||
|
|
||||||
func set_phase(phase: float) -> void:
|
func set_phase(phase: float) -> void:
|
||||||
if animation_length <= 0.0:
|
_set_phase_time(phase)
|
||||||
_time = 0.0
|
|
||||||
else:
|
|
||||||
_time = fposmod(animation_length * phase, animation_length)
|
|
||||||
_rebuild_mesh(_time)
|
_rebuild_mesh(_time)
|
||||||
|
|
||||||
|
|
||||||
func prepare_runtime() -> bool:
|
func prepare_runtime() -> bool:
|
||||||
|
return _prepare_runtime(false)
|
||||||
|
|
||||||
|
|
||||||
|
## Rebinds a duplicated animator to its local mesh and applies its deterministic
|
||||||
|
## phase with one deformation rebuild. This must happen before attachment so
|
||||||
|
## _ready() can remain idempotent for already prepared runtime instances.
|
||||||
|
func prepare_runtime_at_phase(phase: float) -> bool:
|
||||||
|
_set_phase_time(phase)
|
||||||
|
return _prepare_runtime(true)
|
||||||
|
|
||||||
|
|
||||||
|
func _prepare_runtime(force_rebuild: bool) -> bool:
|
||||||
|
if _prepared and not force_rebuild:
|
||||||
|
set_process(true)
|
||||||
|
return true
|
||||||
_resolve_mesh_instance()
|
_resolve_mesh_instance()
|
||||||
|
if force_rebuild:
|
||||||
|
_materials.clear()
|
||||||
_capture_materials()
|
_capture_materials()
|
||||||
_unique_mesh_ready = false
|
_unique_mesh_ready = false
|
||||||
_make_mesh_unique()
|
_make_mesh_unique()
|
||||||
_rebuild_mesh(_time)
|
_rebuild_mesh(_time)
|
||||||
_prepared = mesh != null and not bones.is_empty() and not surfaces.is_empty() and animation_length > 0.0
|
_prepared = _has_runtime_animation_data()
|
||||||
set_process(_prepared)
|
set_process(_prepared)
|
||||||
return _prepared
|
return _prepared
|
||||||
|
|
||||||
|
|
||||||
|
func _has_runtime_animation_data() -> bool:
|
||||||
|
return mesh != null and not bones.is_empty() and not surfaces.is_empty() and animation_length > 0.0
|
||||||
|
|
||||||
|
|
||||||
|
func _set_phase_time(phase: float) -> void:
|
||||||
|
if animation_length <= 0.0:
|
||||||
|
_time = 0.0
|
||||||
|
else:
|
||||||
|
_time = fposmod(animation_length * phase, animation_length)
|
||||||
|
|
||||||
|
|
||||||
func runtime_debug_state() -> Dictionary:
|
func runtime_debug_state() -> Dictionary:
|
||||||
return {
|
return {
|
||||||
"prepared": _prepared,
|
"prepared": _prepared,
|
||||||
@@ -81,11 +108,12 @@ func _resolve_mesh_instance() -> void:
|
|||||||
func _make_mesh_unique() -> void:
|
func _make_mesh_unique() -> void:
|
||||||
if _unique_mesh_ready or mesh_instance == null or mesh_instance.mesh == null:
|
if _unique_mesh_ready or mesh_instance == null or mesh_instance.mesh == null:
|
||||||
return
|
return
|
||||||
var duplicated := mesh_instance.mesh.duplicate(true) as ArrayMesh
|
# _rebuild_mesh() replaces every surface from the retained native arrays, so
|
||||||
if duplicated == null:
|
# copying the source ArrayMesh would only duplicate data that is discarded.
|
||||||
return
|
# Materials were captured before this call and are intentionally shared.
|
||||||
mesh_instance.mesh = duplicated
|
var instance_mesh := ArrayMesh.new()
|
||||||
mesh = duplicated
|
mesh_instance.mesh = instance_mesh
|
||||||
|
mesh = instance_mesh
|
||||||
_unique_mesh_ready = true
|
_unique_mesh_ready = true
|
||||||
|
|
||||||
|
|
||||||
@@ -143,20 +171,20 @@ func _rebuild_mesh(time: float) -> void:
|
|||||||
continue
|
continue
|
||||||
var transform: Transform3D = bone_matrices[bone_index]
|
var transform: Transform3D = bone_matrices[bone_index]
|
||||||
skinned_pos += transform * base_vertices[vertex_index] * weight
|
skinned_pos += transform * base_vertices[vertex_index] * weight
|
||||||
if normals.size() == base_normals.size():
|
if not normals.is_empty():
|
||||||
skinned_nrm += (transform.basis * base_normals[vertex_index]) * weight
|
skinned_nrm += (transform.basis * base_normals[vertex_index]) * weight
|
||||||
total_weight += weight
|
total_weight += weight
|
||||||
if total_weight > 0.0:
|
if total_weight > 0.0:
|
||||||
vertices[vertex_index] = skinned_pos / total_weight
|
vertices[vertex_index] = skinned_pos / total_weight
|
||||||
if normals.size() == base_normals.size():
|
if not normals.is_empty():
|
||||||
normals[vertex_index] = (skinned_nrm / total_weight).normalized()
|
normals[vertex_index] = (skinned_nrm / total_weight).normalized()
|
||||||
else:
|
else:
|
||||||
vertices[vertex_index] = base_vertices[vertex_index]
|
vertices[vertex_index] = base_vertices[vertex_index]
|
||||||
if normals.size() == base_normals.size():
|
if not normals.is_empty():
|
||||||
normals[vertex_index] = base_normals[vertex_index]
|
normals[vertex_index] = base_normals[vertex_index]
|
||||||
else:
|
else:
|
||||||
vertices[vertex_index] = base_vertices[vertex_index]
|
vertices[vertex_index] = base_vertices[vertex_index]
|
||||||
if normals.size() == base_normals.size():
|
if not normals.is_empty():
|
||||||
normals[vertex_index] = base_normals[vertex_index]
|
normals[vertex_index] = base_normals[vertex_index]
|
||||||
|
|
||||||
var arrays := []
|
var arrays := []
|
||||||
|
|||||||
@@ -16,6 +16,7 @@ func _initialize() -> void:
|
|||||||
_verify_animation_selection_priority(failures)
|
_verify_animation_selection_priority(failures)
|
||||||
_verify_player_loop_play_and_seek(failures)
|
_verify_player_loop_play_and_seek(failures)
|
||||||
_verify_native_copy_start_and_diagnostics(failures)
|
_verify_native_copy_start_and_diagnostics(failures)
|
||||||
|
_verify_native_single_rebuild_preparation(failures)
|
||||||
_verify_ownership_boundaries(failures)
|
_verify_ownership_boundaries(failures)
|
||||||
var elapsed_milliseconds := _verify_bounded_timing(failures)
|
var elapsed_milliseconds := _verify_bounded_timing(failures)
|
||||||
if not failures.is_empty():
|
if not failures.is_empty():
|
||||||
@@ -24,7 +25,7 @@ func _initialize() -> void:
|
|||||||
quit(1)
|
quit(1)
|
||||||
return
|
return
|
||||||
print(
|
print(
|
||||||
"M2_ANIMATION_PLAYBACK_CONTROLLER PASS cases=15 iterations=20000 elapsed_ms=%.3f"
|
"M2_ANIMATION_PLAYBACK_CONTROLLER PASS cases=20 iterations=20000 elapsed_ms=%.3f"
|
||||||
% elapsed_milliseconds
|
% elapsed_milliseconds
|
||||||
)
|
)
|
||||||
quit(0)
|
quit(0)
|
||||||
@@ -142,6 +143,47 @@ func _verify_native_copy_start_and_diagnostics(failures: Array[String]) -> void:
|
|||||||
target_root.free()
|
target_root.free()
|
||||||
|
|
||||||
|
|
||||||
|
func _verify_native_single_rebuild_preparation(failures: Array[String]) -> void:
|
||||||
|
var fixture_root := Node3D.new()
|
||||||
|
var mesh_instance := MeshInstance3D.new()
|
||||||
|
mesh_instance.name = "Mesh"
|
||||||
|
var source_mesh := ArrayMesh.new()
|
||||||
|
var source_arrays := []
|
||||||
|
source_arrays.resize(Mesh.ARRAY_MAX)
|
||||||
|
source_arrays[Mesh.ARRAY_VERTEX] = PackedVector3Array([
|
||||||
|
Vector3.ZERO,
|
||||||
|
Vector3.RIGHT,
|
||||||
|
Vector3.UP,
|
||||||
|
])
|
||||||
|
source_arrays[Mesh.ARRAY_INDEX] = PackedInt32Array([0, 1, 2])
|
||||||
|
source_mesh.add_surface_from_arrays(Mesh.PRIMITIVE_TRIANGLES, source_arrays)
|
||||||
|
var shared_material := StandardMaterial3D.new()
|
||||||
|
source_mesh.surface_set_material(0, shared_material)
|
||||||
|
mesh_instance.mesh = source_mesh
|
||||||
|
fixture_root.add_child(mesh_instance)
|
||||||
|
|
||||||
|
var animator: Node = NATIVE_ANIMATOR_SCRIPT.new()
|
||||||
|
fixture_root.add_child(animator)
|
||||||
|
animator.setup(
|
||||||
|
mesh_instance,
|
||||||
|
[{"parent": -1, "pivot": Vector3.ZERO}],
|
||||||
|
[{
|
||||||
|
"vertices": PackedVector3Array([Vector3.ZERO, Vector3.RIGHT, Vector3.UP]),
|
||||||
|
"indices": PackedInt32Array([0, 1, 2]),
|
||||||
|
}],
|
||||||
|
4.0
|
||||||
|
)
|
||||||
|
var prepared := bool(animator.prepare_runtime_at_phase(0.25))
|
||||||
|
var phased_mesh: Mesh = mesh_instance.mesh
|
||||||
|
animator.call("_ready")
|
||||||
|
_expect_true(prepared, "native phased preparation succeeds", failures)
|
||||||
|
_expect_float_equal(float(animator.get("_time")), 1.0, "native phased preparation time", failures)
|
||||||
|
_expect_true(phased_mesh != source_mesh, "native phased Mesh is instance-local", failures)
|
||||||
|
_expect_same(phased_mesh.surface_get_material(0), shared_material, "native Material remains shared", failures)
|
||||||
|
_expect_same(mesh_instance.mesh, phased_mesh, "native ready does not duplicate prepared mesh", failures)
|
||||||
|
fixture_root.free()
|
||||||
|
|
||||||
|
|
||||||
func _verify_ownership_boundaries(failures: Array[String]) -> void:
|
func _verify_ownership_boundaries(failures: Array[String]) -> void:
|
||||||
var controller_source := FileAccess.get_file_as_string(CONTROLLER_PATH)
|
var controller_source := FileAccess.get_file_as_string(CONTROLLER_PATH)
|
||||||
var materializer_source := FileAccess.get_file_as_string(MATERIALIZER_PATH)
|
var materializer_source := FileAccess.get_file_as_string(MATERIALIZER_PATH)
|
||||||
@@ -156,6 +198,17 @@ func _verify_ownership_boundaries(failures: Array[String]) -> void:
|
|||||||
_expect_false(loader_source.contains(removed_loader_function), "legacy helper removed: %s" % removed_loader_function, failures)
|
_expect_false(loader_source.contains(removed_loader_function), "legacy helper removed: %s" % removed_loader_function, failures)
|
||||||
_expect_equal(materializer_source.count("_animation_playback_controller.copy_native_animator_data("), 1, "native copy delegates once", failures)
|
_expect_equal(materializer_source.count("_animation_playback_controller.copy_native_animator_data("), 1, "native copy delegates once", failures)
|
||||||
_expect_equal(materializer_source.count("_animation_playback_controller.start_instance_playback("), 1, "playback delegates once", failures)
|
_expect_equal(materializer_source.count("_animation_playback_controller.start_instance_playback("), 1, "playback delegates once", failures)
|
||||||
|
_expect_true(
|
||||||
|
controller_source.contains("animator.prepare_runtime_at_phase(phase)"),
|
||||||
|
"native phase preparation uses one rebuild",
|
||||||
|
failures
|
||||||
|
)
|
||||||
|
var native_animator_source := FileAccess.get_file_as_string("res://src/scenes/streaming/m2_native_animator.gd")
|
||||||
|
_expect_true(
|
||||||
|
native_animator_source.contains("if not _prepared:\n\t\tprepare_runtime()"),
|
||||||
|
"ready is idempotent after pre-attachment preparation",
|
||||||
|
failures
|
||||||
|
)
|
||||||
for retained_materializer_rule in [
|
for retained_materializer_rule in [
|
||||||
"Node.DUPLICATE_SIGNALS | Node.DUPLICATE_GROUPS | Node.DUPLICATE_SCRIPTS",
|
"Node.DUPLICATE_SIGNALS | Node.DUPLICATE_GROUPS | Node.DUPLICATE_SCRIPTS",
|
||||||
"batch_root.add_child(instance)",
|
"batch_root.add_child(instance)",
|
||||||
|
|||||||
@@ -58,7 +58,7 @@ func _initialize() -> void:
|
|||||||
quit(1)
|
quit(1)
|
||||||
return
|
return
|
||||||
|
|
||||||
print("RENDERER_CLOSEOUT_CONTRACTS PASS workers=%d frame_steps=%d cache_versions=7" % [
|
print("RENDERER_CLOSEOUT_CONTRACTS PASS workers=%d frame_steps=%d cache_versions=7 nested_glb=1" % [
|
||||||
WORKER_FUNCTIONS.size(),
|
WORKER_FUNCTIONS.size(),
|
||||||
MAIN_THREAD_FRAME_STEPS.size(),
|
MAIN_THREAD_FRAME_STEPS.size(),
|
||||||
])
|
])
|
||||||
|
|||||||
@@ -12,9 +12,9 @@
|
|||||||
- [x] Запретить gameplay/editor доступ к внутренним очередям streamer.
|
- [x] Запретить gameplay/editor доступ к внутренним очередям streamer.
|
||||||
- [x] Извлечь pure `StreamingTargetPlanner`.
|
- [x] Извлечь pure `StreamingTargetPlanner`.
|
||||||
- [x] Извлечь `RenderBudgetScheduler` с bounded queues/cancellation.
|
- [x] Извлечь `RenderBudgetScheduler` с bounded queues/cancellation.
|
||||||
- [ ] Затем извлекать terrain, M2, WMO и liquid services по одному.
|
- [x] Затем извлекать terrain, M2, WMO и liquid services по одному.
|
||||||
- [ ] Сохранить cache versioning и main-thread GPU finalization rules.
|
- [x] Сохранить cache versioning и main-thread GPU finalization rules.
|
||||||
- [ ] Добавить dependency и performance regression checks.
|
- [x] Добавить dependency и performance regression checks.
|
||||||
|
|
||||||
## Fidelity evidence
|
## Fidelity evidence
|
||||||
|
|
||||||
@@ -636,3 +636,36 @@ Runtime и Editor используют facade; planner/scheduler тестиру
|
|||||||
the existing permit, and asset-backed MH2O/MCLQ traversal/leak/p95/p99/
|
the existing permit, and asset-backed MH2O/MCLQ traversal/leak/p95/p99/
|
||||||
original-client evidence is pending; M03 still needs further terrain/M2/WMO/
|
original-client evidence is pending; M03 still needs further terrain/M2/WMO/
|
||||||
liquid service extraction and dependency/performance acceptance.
|
liquid service extraction and dependency/performance acceptance.
|
||||||
|
|
||||||
|
### Closeout evidence — 2026-08-02
|
||||||
|
|
||||||
|
- Exact accepted M00 commit `239cb24` and M03 were captured on the same RTX 5070,
|
||||||
|
Godot 4.6.1, `High` preset and exact cache inventory: terrain `687`, WMO
|
||||||
|
render/scene `368/368`, M2 baked/GLB/scene `2009/2009/2009`.
|
||||||
|
- The repeated-sample protocol (five M00 reports and three M03 reports) and the
|
||||||
|
independent ten-second measurement protocol compared all `14` cold/warm
|
||||||
|
result pairs and `84` metrics with the unchanged 10% budgets. The stability
|
||||||
|
gate passed with `repeatable_regressions=0`; five protocol-local exceedances
|
||||||
|
did not reproduce across both sampling windows.
|
||||||
|
- Native GryphonRoost preparation now allocates an empty instance-local
|
||||||
|
`ArrayMesh`, shares captured Materials and performs one phased deformation
|
||||||
|
rebuild. The diagnosed initial build hitch fell from `168.02ms` to no native
|
||||||
|
copy hitch; dense-M2 maximum hitches fell from M00 `171–195ms` to M03
|
||||||
|
`52–66ms` in the paired short-window series.
|
||||||
|
- All `67/67` autonomous Godot verifier scripts passed. Renderer closeout
|
||||||
|
contracts passed `workers=4 frame_steps=15 cache_versions=7 nested_glb=1`;
|
||||||
|
documentation passed `module_specs=53 required_files=7`; coordination and
|
||||||
|
`git diff --check` passed.
|
||||||
|
- Asset-backed PNG/checkpoint evidence covers terrain overview, ADT boundary,
|
||||||
|
dense M2, large WMO, liquid, native animation and dusk sky. This proves the
|
||||||
|
M03 extraction did not introduce a repeatable performance regression; it does
|
||||||
|
not claim pixel-level parity with the original WoW 3.3.5a client.
|
||||||
|
- Superseding the historical pending statements above: the extraction,
|
||||||
|
dependency, cache-version, main-thread finalization and performance criteria
|
||||||
|
required by M03 are accepted. Remaining items are release-level risks:
|
||||||
|
original-client visual approval, long-traversal descriptor pressure, GPU
|
||||||
|
skinning for many native animators and interruption of already in-flight work.
|
||||||
|
- Closeout files: `src/tools/verify_renderer_closeout_contracts.gd`,
|
||||||
|
`tools/compare_render_performance.ps1`,
|
||||||
|
`tools/verify_render_performance_stability.ps1`, renderer closeout/module
|
||||||
|
documentation, nested M2 GLB bake fix and native animator regression coverage.
|
||||||
|
|||||||
@@ -1,10 +1,10 @@
|
|||||||
[CmdletBinding()]
|
[CmdletBinding()]
|
||||||
param(
|
param(
|
||||||
[Parameter(Mandatory = $true)]
|
[Parameter(Mandatory = $true)]
|
||||||
[string]$BaselineReport,
|
[string[]]$BaselineReport,
|
||||||
|
|
||||||
[Parameter(Mandatory = $true)]
|
[Parameter(Mandatory = $true)]
|
||||||
[string]$CandidateReport,
|
[string[]]$CandidateReport,
|
||||||
|
|
||||||
[string]$OutputReport
|
[string]$OutputReport
|
||||||
)
|
)
|
||||||
@@ -37,6 +37,20 @@ function Convert-ToStableJson {
|
|||||||
return $Value | ConvertTo-Json -Depth 20 -Compress
|
return $Value | ConvertTo-Json -Depth 20 -Compress
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function Get-MedianValue {
|
||||||
|
param([double[]]$Values)
|
||||||
|
|
||||||
|
$ordered = @($Values | Sort-Object)
|
||||||
|
if ($ordered.Count -eq 0) {
|
||||||
|
throw 'Cannot calculate a median from an empty value set'
|
||||||
|
}
|
||||||
|
$middle = [math]::Floor($ordered.Count / 2)
|
||||||
|
if ($ordered.Count % 2 -eq 1) {
|
||||||
|
return [double]$ordered[$middle]
|
||||||
|
}
|
||||||
|
return ([double]$ordered[$middle - 1] + [double]$ordered[$middle]) / 2.0
|
||||||
|
}
|
||||||
|
|
||||||
function Assert-EqualValue {
|
function Assert-EqualValue {
|
||||||
param(
|
param(
|
||||||
[string]$Name,
|
[string]$Name,
|
||||||
@@ -74,6 +88,9 @@ function Assert-CompatibleCacheInventory {
|
|||||||
Assert-EqualValue "cache inventory presence for $cacheName" `
|
Assert-EqualValue "cache inventory presence for $cacheName" `
|
||||||
$baselineRecord.present `
|
$baselineRecord.present `
|
||||||
$candidateRecord.present
|
$candidateRecord.present
|
||||||
|
Assert-EqualValue "cache inventory file count for $cacheName" `
|
||||||
|
$baselineRecord.file_count `
|
||||||
|
$candidateRecord.file_count
|
||||||
if ([bool]$baselineRecord.present -and [int64]$candidateRecord.file_count -le 0) {
|
if ([bool]$baselineRecord.present -and [int64]$candidateRecord.file_count -le 0) {
|
||||||
$failures.Add("cache inventory is empty for $cacheName")
|
$failures.Add("cache inventory is empty for $cacheName")
|
||||||
}
|
}
|
||||||
@@ -98,6 +115,75 @@ function Build-ResultIndex {
|
|||||||
return $index
|
return $index
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function Merge-RenderReports {
|
||||||
|
param(
|
||||||
|
[object[]]$Reports,
|
||||||
|
[string]$Label
|
||||||
|
)
|
||||||
|
|
||||||
|
$merged = Convert-ToStableJson $Reports[0] | ConvertFrom-Json
|
||||||
|
if ($Reports.Count -eq 1) {
|
||||||
|
return $merged
|
||||||
|
}
|
||||||
|
$resultIndexes = @(
|
||||||
|
foreach ($sample in $Reports) {
|
||||||
|
Build-ResultIndex -Results @($sample.results) -Label "$Label sample"
|
||||||
|
}
|
||||||
|
)
|
||||||
|
$referenceResultKeys = @($resultIndexes[0].Keys | Sort-Object)
|
||||||
|
for ($sampleNumber = 1; $sampleNumber -lt $Reports.Count; $sampleNumber++) {
|
||||||
|
$sample = $Reports[$sampleNumber]
|
||||||
|
Assert-EqualValue "$Label[$sampleNumber] schema_version" $merged.schema_version $sample.schema_version
|
||||||
|
Assert-EqualValue "$Label[$sampleNumber] profile" $merged.profile $sample.profile
|
||||||
|
Assert-EqualValue "$Label[$sampleNumber] cache_state" $merged.cache_state $sample.cache_state
|
||||||
|
Assert-EqualValue "$Label[$sampleNumber] viewport" $merged.environment.viewport $sample.environment.viewport
|
||||||
|
Assert-EqualValue "$Label[$sampleNumber] Godot version" $merged.environment.godot_version.string $sample.environment.godot_version.string
|
||||||
|
Assert-EqualValue "$Label[$sampleNumber] rendering driver" $merged.environment.rendering_driver $sample.environment.rendering_driver
|
||||||
|
Assert-EqualValue "$Label[$sampleNumber] rendering method" $merged.environment.rendering_method $sample.environment.rendering_method
|
||||||
|
Assert-EqualValue "$Label[$sampleNumber] video adapter" $merged.environment.video_adapter $sample.environment.video_adapter
|
||||||
|
Assert-EqualValue "$Label[$sampleNumber] CPU" $merged.environment.cpu $sample.environment.cpu
|
||||||
|
Assert-EqualValue "$Label[$sampleNumber] cache contract" $merged.cache_contract $sample.cache_contract
|
||||||
|
Assert-EqualValue "$Label[$sampleNumber] cache inventory" $merged.cache_inventory $sample.cache_inventory
|
||||||
|
Assert-EqualValue `
|
||||||
|
"$Label[$sampleNumber] result key set" `
|
||||||
|
$referenceResultKeys `
|
||||||
|
@($resultIndexes[$sampleNumber].Keys | Sort-Object)
|
||||||
|
}
|
||||||
|
|
||||||
|
$metricDefinitions = @(
|
||||||
|
@('load_time_ms', 'root'),
|
||||||
|
@('frame_ms_p95', 'metrics'),
|
||||||
|
@('frame_ms_p99', 'metrics'),
|
||||||
|
@('max_hitch_ms', 'metrics'),
|
||||||
|
@('memory_static_bytes', 'metrics'),
|
||||||
|
@('video_memory_bytes', 'metrics')
|
||||||
|
)
|
||||||
|
$mergedResultIndex = Build-ResultIndex -Results @($merged.results) -Label "Median $Label"
|
||||||
|
foreach ($resultKey in $referenceResultKeys) {
|
||||||
|
$mergedResult = $mergedResultIndex[$resultKey]
|
||||||
|
foreach ($definition in $metricDefinitions) {
|
||||||
|
$metricName = $definition[0]
|
||||||
|
$location = $definition[1]
|
||||||
|
$values = @(
|
||||||
|
foreach ($sampleIndex in $resultIndexes) {
|
||||||
|
if ($location -eq 'root') {
|
||||||
|
[double]$sampleIndex[$resultKey].$metricName
|
||||||
|
} else {
|
||||||
|
[double]$sampleIndex[$resultKey].metrics.$metricName
|
||||||
|
}
|
||||||
|
}
|
||||||
|
)
|
||||||
|
$medianValue = Get-MedianValue -Values $values
|
||||||
|
if ($location -eq 'root') {
|
||||||
|
$mergedResult.$metricName = $medianValue
|
||||||
|
} else {
|
||||||
|
$mergedResult.metrics.$metricName = $medianValue
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return $merged
|
||||||
|
}
|
||||||
|
|
||||||
function Compare-Metric {
|
function Compare-Metric {
|
||||||
param(
|
param(
|
||||||
[string]$ResultKey,
|
[string]$ResultKey,
|
||||||
@@ -140,8 +226,22 @@ function Compare-Metric {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
$baseline = Read-RenderReport -Path $BaselineReport -Label 'Baseline'
|
$baselineReports = @(
|
||||||
$candidate = Read-RenderReport -Path $CandidateReport -Label 'Candidate'
|
for ($baselineIndex = 0; $baselineIndex -lt $BaselineReport.Count; $baselineIndex++) {
|
||||||
|
Read-RenderReport `
|
||||||
|
-Path $BaselineReport[$baselineIndex] `
|
||||||
|
-Label "Baseline[$baselineIndex]"
|
||||||
|
}
|
||||||
|
)
|
||||||
|
$candidateReports = @(
|
||||||
|
for ($candidateIndex = 0; $candidateIndex -lt $CandidateReport.Count; $candidateIndex++) {
|
||||||
|
Read-RenderReport `
|
||||||
|
-Path $CandidateReport[$candidateIndex] `
|
||||||
|
-Label "Candidate[$candidateIndex]"
|
||||||
|
}
|
||||||
|
)
|
||||||
|
$baseline = Merge-RenderReports -Reports $baselineReports -Label 'baseline'
|
||||||
|
$candidate = Merge-RenderReports -Reports $candidateReports -Label 'candidate'
|
||||||
|
|
||||||
Assert-EqualValue 'schema_version' $baseline.schema_version $candidate.schema_version
|
Assert-EqualValue 'schema_version' $baseline.schema_version $candidate.schema_version
|
||||||
Assert-EqualValue 'profile' $baseline.profile $candidate.profile
|
Assert-EqualValue 'profile' $baseline.profile $candidate.profile
|
||||||
@@ -204,6 +304,10 @@ $summary = [pscustomobject]@{
|
|||||||
candidate_revision = $candidate.revision
|
candidate_revision = $candidate.revision
|
||||||
baseline_created_utc = $baseline.created_utc
|
baseline_created_utc = $baseline.created_utc
|
||||||
candidate_created_utc = $candidate.created_utc
|
candidate_created_utc = $candidate.created_utc
|
||||||
|
baseline_samples = $baselineReports.Count
|
||||||
|
baseline_reports = @($BaselineReport)
|
||||||
|
candidate_samples = $candidateReports.Count
|
||||||
|
candidate_reports = @($CandidateReport)
|
||||||
result_pairs = $baselineIndex.Count
|
result_pairs = $baselineIndex.Count
|
||||||
metric_comparisons = $comparisons.Count
|
metric_comparisons = $comparisons.Count
|
||||||
passed = $failures.Count -eq 0
|
passed = $failures.Count -eq 0
|
||||||
|
|||||||
@@ -0,0 +1,106 @@
|
|||||||
|
[CmdletBinding()]
|
||||||
|
param(
|
||||||
|
[Parameter(Mandatory = $true)]
|
||||||
|
[string]$RepeatedSampleComparison,
|
||||||
|
|
||||||
|
[Parameter(Mandatory = $true)]
|
||||||
|
[string]$LongWindowComparison,
|
||||||
|
|
||||||
|
[string]$OutputReport
|
||||||
|
)
|
||||||
|
|
||||||
|
$ErrorActionPreference = 'Stop'
|
||||||
|
|
||||||
|
function Read-ComparisonReport {
|
||||||
|
param([string]$Path, [string]$Label)
|
||||||
|
|
||||||
|
if (-not (Test-Path -LiteralPath $Path -PathType Leaf)) {
|
||||||
|
throw "$Label comparison does not exist: $Path"
|
||||||
|
}
|
||||||
|
$report = Get-Content -Raw -Encoding UTF8 -LiteralPath $Path | ConvertFrom-Json
|
||||||
|
if ($null -eq $report -or $null -eq $report.comparisons) {
|
||||||
|
throw "$Label comparison has no comparisons array: $Path"
|
||||||
|
}
|
||||||
|
return $report
|
||||||
|
}
|
||||||
|
|
||||||
|
function Build-ComparisonIndex {
|
||||||
|
param([object[]]$Comparisons, [string]$Label)
|
||||||
|
|
||||||
|
$index = @{}
|
||||||
|
foreach ($comparison in $Comparisons) {
|
||||||
|
$key = "$($comparison.result)|$($comparison.metric)"
|
||||||
|
if ($index.ContainsKey($key)) {
|
||||||
|
throw "$Label comparison contains duplicate metric: $key"
|
||||||
|
}
|
||||||
|
$index[$key] = $comparison
|
||||||
|
}
|
||||||
|
return $index
|
||||||
|
}
|
||||||
|
|
||||||
|
$repeatedReport = Read-ComparisonReport `
|
||||||
|
-Path $RepeatedSampleComparison `
|
||||||
|
-Label 'Repeated-sample'
|
||||||
|
$longWindowReport = Read-ComparisonReport `
|
||||||
|
-Path $LongWindowComparison `
|
||||||
|
-Label 'Long-window'
|
||||||
|
$repeatedIndex = Build-ComparisonIndex `
|
||||||
|
-Comparisons @($repeatedReport.comparisons) `
|
||||||
|
-Label 'Repeated-sample'
|
||||||
|
$longWindowIndex = Build-ComparisonIndex `
|
||||||
|
-Comparisons @($longWindowReport.comparisons) `
|
||||||
|
-Label 'Long-window'
|
||||||
|
$repeatedKeys = @($repeatedIndex.Keys | Sort-Object)
|
||||||
|
$longWindowKeys = @($longWindowIndex.Keys | Sort-Object)
|
||||||
|
if (($repeatedKeys | ConvertTo-Json -Compress) -cne ($longWindowKeys | ConvertTo-Json -Compress)) {
|
||||||
|
throw 'Comparison metric inventories differ between repeated and long-window protocols'
|
||||||
|
}
|
||||||
|
|
||||||
|
$repeatableRegressions = [System.Collections.Generic.List[object]]::new()
|
||||||
|
$nonRepeatableRegressions = [System.Collections.Generic.List[object]]::new()
|
||||||
|
foreach ($key in $repeatedKeys) {
|
||||||
|
$repeatedMetricComparison = $repeatedIndex[$key]
|
||||||
|
$longWindowMetricComparison = $longWindowIndex[$key]
|
||||||
|
$repeatedFailed = ([bool]$repeatedMetricComparison.passed) -eq $false
|
||||||
|
$longWindowFailed = ([bool]$longWindowMetricComparison.passed) -eq $false
|
||||||
|
if ($repeatedFailed -and $longWindowFailed) {
|
||||||
|
$repeatableRegressions.Add([pscustomobject]@{
|
||||||
|
key = $key
|
||||||
|
repeated_sample = $repeatedMetricComparison
|
||||||
|
long_window = $longWindowMetricComparison
|
||||||
|
})
|
||||||
|
} elseif ($repeatedFailed -or $longWindowFailed) {
|
||||||
|
$nonRepeatableRegressions.Add([pscustomobject]@{
|
||||||
|
key = $key
|
||||||
|
failed_protocol = if ($repeatedFailed) { 'repeated_sample' } else { 'long_window' }
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
$summary = [pscustomobject]@{
|
||||||
|
schema_version = 1
|
||||||
|
repeated_sample_comparison = $RepeatedSampleComparison
|
||||||
|
long_window_comparison = $LongWindowComparison
|
||||||
|
metric_comparisons = $repeatedKeys.Count
|
||||||
|
passed = $repeatableRegressions.Count -eq 0
|
||||||
|
repeatable_regressions = @($repeatableRegressions)
|
||||||
|
non_repeatable_regressions = @($nonRepeatableRegressions)
|
||||||
|
}
|
||||||
|
if ($OutputReport) {
|
||||||
|
$parent = Split-Path -Parent $OutputReport
|
||||||
|
if ($parent -and -not (Test-Path -LiteralPath $parent)) {
|
||||||
|
New-Item -ItemType Directory -Path $parent | Out-Null
|
||||||
|
}
|
||||||
|
$summary | ConvertTo-Json -Depth 20 | Set-Content -Encoding UTF8 -LiteralPath $OutputReport
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($repeatableRegressions.Count -gt 0) {
|
||||||
|
foreach ($regression in $repeatableRegressions) {
|
||||||
|
Write-Error "RENDER_PERFORMANCE_STABILITY: repeatable regression $($regression.key)" -ErrorAction Continue
|
||||||
|
}
|
||||||
|
Write-Host "RENDER_PERFORMANCE_STABILITY FAIL repeated=$($repeatableRegressions.Count) metrics=$($repeatedKeys.Count)"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
Write-Host "RENDER_PERFORMANCE_STABILITY PASS metrics=$($repeatedKeys.Count) non_repeatable=$($nonRepeatableRegressions.Count)"
|
||||||
|
exit 0
|
||||||
Reference in New Issue
Block a user