From 9df32d38ef092a1bc940274f310e0fe87c3a0a37 Mon Sep 17 00:00:00 2001 From: sindoring Date: Fri, 17 Jul 2026 00:05:49 +0400 Subject: [PATCH 1/2] refactor(M03): extract M2 placement grouper --- RENDER.md | 11 + .../M03-RND-M2-PLACEMENT-GROUPER-001.md | 1 + docs/modules/README.md | 1 + docs/modules/m2-placement-grouper.md | 191 +++++++++++ .../m2-placement-transform-resolver.md | 39 +-- docs/modules/world-renderer.md | 25 +- src/render/m2/m2_placement_grouper.gd | 65 ++++ src/render/m2/m2_placement_grouper.gd.uid | 1 + .../streaming/streaming_world_loader.gd | 39 +-- src/tools/verify_m2_placement_grouper.gd | 296 ++++++++++++++++++ src/tools/verify_m2_placement_grouper.gd.uid | 1 + .../verify_m2_placement_transform_resolver.gd | 22 +- 12 files changed, 633 insertions(+), 59 deletions(-) create mode 100644 docs/modules/m2-placement-grouper.md create mode 100644 src/render/m2/m2_placement_grouper.gd create mode 100644 src/render/m2/m2_placement_grouper.gd.uid create mode 100644 src/tools/verify_m2_placement_grouper.gd create mode 100644 src/tools/verify_m2_placement_grouper.gd.uid diff --git a/RENDER.md b/RENDER.md index ae5ad8e..6aa1b46 100644 --- a/RENDER.md +++ b/RENDER.md @@ -1003,6 +1003,17 @@ $exe = Join-Path $env:TEMP 'godot-4.6.1-openwc\Godot_v4.6.1-stable_win64.exe' - Cache formats, placement positions and visible rules are unchanged by the extraction. Asset-backed visual recheck and general placement parity remain gaps. +## 2026-07-16 M2 Placement Grouper Extraction + +- `M2PlacementGrouper` now owns pure worker-side validation, historical path + normalization and ordered tile-local transform grouping of ADT M2 placements. +- Invalid records, backslash replacement, lowercase `.mdx/.mdl` conversion, + first-seen ordering and the `0.0001` scale clamp are unchanged. +- The loader retains tasks, mutex/result queues, stale-result checks, tile/build + state, caches and all Node/MultiMesh/Mesh/material/RID finalization. +- Cache formats, batching, profiles and visible rules are unchanged. Asset-backed + traversal p95/p99, visual recheck and spatial-cell grouping remain pending. + ## 2026-07-16 World Environment Snapshot Facade - `WorldEnvironmentSnapshot` carries one immutable finite time-of-day value, diff --git a/coordination/claims/M03-RND-M2-PLACEMENT-GROUPER-001.md b/coordination/claims/M03-RND-M2-PLACEMENT-GROUPER-001.md index 3686f92..e863fcf 100644 --- a/coordination/claims/M03-RND-M2-PLACEMENT-GROUPER-001.md +++ b/coordination/claims/M03-RND-M2-PLACEMENT-GROUPER-001.md @@ -30,6 +30,7 @@ grouping from the worker callback into a scene-free service. `src/tools/verify_m2_placement_grouper.gd`, `docs/modules/m2-placement-grouper.md`, this claim - Shared/hotspots: `src/scenes/streaming/streaming_world_loader.gd`, + `src/tools/verify_m2_placement_transform_resolver.gd`, `docs/modules/m2-placement-transform-resolver.md`, `docs/modules/world-renderer.md`, `docs/modules/README.md`, `RENDER.md`, `targets/03-renderer-facade.md` diff --git a/docs/modules/README.md b/docs/modules/README.md index 69ff605..41340d0 100644 --- a/docs/modules/README.md +++ b/docs/modules/README.md @@ -17,6 +17,7 @@ | Terrain chunk geometry queue planner | Implemented extraction | [`terrain-chunk-geometry-queue-planner.md`](terrain-chunk-geometry-queue-planner.md) | | M2 unique placement registry | Implemented extraction | [`m2-unique-placement-registry.md`](m2-unique-placement-registry.md) | | M2 placement transform resolver | Implemented | [`m2-placement-transform-resolver.md`](m2-placement-transform-resolver.md) | +| M2 placement grouper | Implemented extraction | [`m2-placement-grouper.md`](m2-placement-grouper.md) | | Third-person camera | Implemented | [`third-person-camera.md`](third-person-camera.md) | | Character presentation | Implemented boundary / Partial fidelity | [`character-presentation.md`](character-presentation.md) | | Renderer | Partial | [`world-renderer.md`](world-renderer.md), [`../../RENDER.md`](../../RENDER.md) | diff --git a/docs/modules/m2-placement-grouper.md b/docs/modules/m2-placement-grouper.md new file mode 100644 index 0000000..f91d4ac --- /dev/null +++ b/docs/modules/m2-placement-grouper.md @@ -0,0 +1,191 @@ +# M2 Placement Grouper + +## Metadata + +| Field | Value | +|---|---| +| Status | Implemented extraction | +| Target/work package | M03 / `M03-RND-M2-PLACEMENT-GROUPER-001` | +| Owners | Pure ADT M2 placement validation, normalization and transform grouping | +| Last verified | Worktree `work/sindo-main-codex/m03-m2-placement-grouper`, 2026-07-16 | +| Profiles/capabilities | Existing static M2 worker-grouping path | + +## Purpose + +Convert valid raw ADT M2 placement records into ordered tile-local transforms +grouped by the historically normalized relative model path. + +## Non-goals + +- Reserve cross-tile unique IDs or choose placement ownership. +- Own WorkerThreadPool tasks, mutexes, result queues or tile lifecycle. +- Load models, choose animated/static rendering or build MultiMesh batches. +- Create Node, Mesh, material, Resource or RID objects. +- Change path case, transform rules, batching or visual fidelity. + +## Context and boundaries + +```mermaid +flowchart LR + ADT[ADT names + placements] --> Registry[M2UniquePlacementRegistry] + Registry --> Grouper[M2PlacementGrouper] + Grouper --> Resolver[M2PlacementTransformResolver] + Resolver --> Grouper + Grouper --> Groups[Path to ordered Transform3D arrays] + Groups --> Worker[Loader worker result adapter] + Worker --> Build[Loader-owned build queue] +``` + +Allowed dependencies are value containers, `Vector3`, `Basis`, `Transform3D` +and `M2PlacementTransformResolver`. SceneTree, Node, RenderingServer, +WorkerThreadPool, mutexes, ResourceLoader, files, gameplay, network and editor UI +are forbidden. + +## Public API + +| Symbol | Kind | Purpose | Thread/lifetime | Errors | +|---|---|---|---|---| +| `group_placements(tile_origin, m2_names, m2_placements)` | Pure query | Return valid placements grouped as ordered tile-local transforms | Worker/main thread; call-local output | Invalid variants/name IDs/empty paths are skipped | + +## Inputs and outputs + +| Direction | Contract/data | Producer | Consumer | Ownership | Thread/lifetime | +|---|---|---|---|---|---| +| Input | Godot-space tile origin | Loader tile state | Local-position calculation | Value copy | One call | +| Input | ADT MMDX name table | Parsed ADT tile | Path lookup | Read-only PackedStringArray | One call | +| Input | Raw placement dictionaries | Unique registry result | Validation/transforms | Read-only shallow values | One call | +| Output | Relative path to ordered `Transform3D` arrays | Grouper | Loader worker/build adapter | Fresh Dictionary and arrays | One call/build job | + +Output retains first-seen group-key order through Godot Dictionary insertion +order and placement order inside each array. Inputs are not retained or returned. + +## Data flow + +```mermaid +flowchart TD + Call[group_placements] --> Next[Read placement] + Next --> Dict{Dictionary?} + Dict -->|no| Next + Dict -->|yes| Name{Valid name_id?} + Name -->|no| Next + Name -->|yes| Normalize[Backslash to slash; lowercase MDX/MDL to M2] + Normalize --> Empty{Path empty?} + Empty -->|yes| Next + Empty -->|no| Local[world position - tile origin] + Local --> Resolve[Resolve basis and origin offset] + Resolve --> Scale[Apply max scale 0.0001] + Scale --> Append[Append Transform3D to path group] + Append --> Next + Next -->|complete| Return[Return fresh groups] +``` + +## Lifecycle/state + +The grouper retains no state. Every call constructs one stateless transform +resolver and fresh output containers. Reset, cancellation and shutdown require +no grouper action. + +## Main sequence + +```mermaid +sequenceDiagram + participant Loader as Loader worker callback + participant Grouper as M2PlacementGrouper + participant Resolver as M2PlacementTransformResolver + Loader->>Grouper: group_placements(origin, names, placements) + loop valid placement + Grouper->>Resolver: resolve basis and origin offset + Resolver-->>Grouper: value-only basis and offset + Grouper->>Grouper: append local Transform3D + end + Grouper-->>Loader: fresh path groups + Loader->>Loader: mutex-protected result enqueue +``` + +## Ownership, threading and resources + +- The grouper owns only call-local containers and transforms. +- The loader owns tasks, mutex/result queue, tile/build state, caches, model + loading, Nodes, MultiMesh, Mesh, materials and RIDs. +- Worker use is safe because the grouper and resolver mutate no shared state. +- The caller owns returned containers; mutation cannot affect later calls. + +## Errors, cancellation and recovery + +| Failure | Detection | Behavior | Diagnostic | Recovery | +|---|---|---|---|---| +| Non-Dictionary placement | Type guard | Skip | Contract fixture | Repair parser input | +| Invalid name ID | Bounds guard | Skip | Contract fixture | Repair name table/placement | +| Empty path | Empty guard | Skip | Contract fixture | Supply valid MMDX entry | +| Missing position/rotation/scale | Defaults | Zero position/rotation and scale 1 | Contract fixture | Expected fallback | +| Zero/negative scale | Historical clamp | Basis scale uses `0.0001` | Contract fixture | Supply valid ADT scale | +| Cancellation after grouping | Loader state check | Discard stale result | Loader regressions | Re-request eligible detail | + +Cancellation stays loader-owned because grouping has no task or partial state. + +## Configuration and capabilities + +No setting or profile branch is introduced. Preserved behavior includes +backslash replacement without lowercasing, case-sensitive lowercase `.mdx/.mdl` +conversion, first-seen ordering, tile-local subtraction and scale clamp `0.0001`. + +## Persistence, cache and migration + +No persistence or cache format participates. ADT and M2 cache versions are +unchanged, so no rebuild or migration is required. + +## Diagnostics and observability + +The service emits no logs. Its verifier records contract results, source +boundaries and bounded synthetic 100-by-256 placement timing. Loader metrics +continue to own task/build/cache observability. + +## Verification + +- `verify_m2_placement_grouper.gd`: invalid input, path behavior, ordering, + defaults, local origin, regular/waterfall transforms, scale clamp, fresh output, + loader/source boundary and timing. +- `verify_m2_placement_transform_resolver.gd`: three transform consumers remain + covered as two direct loader adapters plus the grouper adapter. +- Registry/dedupe, facade, internal-access, manifest, shutdown, documentation and + coordination regressions remain required. +- Fidelity evidence is exact extraction; no new asset-backed parity claim is made. + +## Extension points + +- A later package may extract the M2 build-job state machine behind explicit + budgets while retaining this pure grouping input. +- Spatial cells require measured culling evidence and are outside this contract. + +## Capability status + +| Capability | Status | Evidence | Gap/next step | +|---|---|---|---| +| Pure path/transform grouping | Implemented extraction | Contract/source/timing verifier | Asset-backed traversal timing pending | +| Worker/build orchestration | Remains in loader | Existing regressions | Separate stateful extraction required | +| Spatial MultiMesh cells | Partial | Current path batches | Culling-driven design/evidence pending | + +## Known gaps and risks + +- Raw placement dictionaries remain the ADT parser boundary. +- Uppercase extensions remain unchanged even though transform exception matching + is case-insensitive. +- Grouping is by model path, not spatial culling cell. +- Asset-backed p95/p99 and visual comparison remain unavailable. + +## Source map + +| Path | Responsibility | +|---|---| +| `src/render/m2/m2_placement_grouper.gd` | Validation, normalization and grouping | +| `src/render/m2/m2_placement_transform_resolver.gd` | Basis and origin rules | +| `src/scenes/streaming/streaming_world_loader.gd` | Worker/result/build lifecycle and resources | +| `src/tools/verify_m2_placement_grouper.gd` | Contract, dependency and timing regression | + +## Related decisions and references + +- [`m2-placement-transform-resolver.md`](m2-placement-transform-resolver.md) +- [`m2-unique-placement-registry.md`](m2-unique-placement-registry.md) +- [`world-renderer.md`](world-renderer.md) +- [`../../RENDER.md`](../../RENDER.md) +- [`../../targets/roadmap/02-rendering-and-graphics.md`](../../targets/roadmap/02-rendering-and-graphics.md) diff --git a/docs/modules/m2-placement-transform-resolver.md b/docs/modules/m2-placement-transform-resolver.md index 6e4f5e5..2c5645e 100644 --- a/docs/modules/m2-placement-transform-resolver.md +++ b/docs/modules/m2-placement-transform-resolver.md @@ -13,7 +13,7 @@ ## Purpose Resolve the unscaled Godot-space orientation and optional local origin offset -used by ADT M2 worker grouping, placeholder rendering and instance creation. +used by the pure placement grouper, placeholder rendering and instance creation. ## Non-goals @@ -30,7 +30,7 @@ flowchart LR ADT[ADT placement rotation/path/scale] --> Resolver[M2PlacementTransformResolver] Resolver --> Basis[Unscaled Basis] Resolver --> Offset[Local origin offset] - Basis --> Group[Worker grouping] + Basis --> Group[M2PlacementGrouper] Basis --> Placeholder[Placeholder MultiMesh path] Basis --> Instance[Instance path] Offset --> Group @@ -57,8 +57,8 @@ files, gameplay, network and editor UI are forbidden. | Input | Godot Euler rotation in radians | ADT loader placement record | Resolver | Scalar copy | One call | | Input | Relative M2 path | ADT MMDX/name table | Resolver allowlist | Caller string | One call | | Input | Raw placement scale | ADT placement record | Offset compensation | Scalar copy | One call | -| Output | Unscaled `Basis` | Resolver | Group/placeholder/instance adapter | Value copy | One call | -| Output | Local `Vector3` offset | Resolver | Transform origin adapter | Value copy | One call | +| Output | Unscaled `Basis` | Resolver | Grouper/placeholder/instance adapter | Value copy | One call | +| Output | Local `Vector3` offset | Resolver | Grouper/placeholder/instance adapter | Value copy | One call | Callers preserve their historical basis scale minima: grouping/instance use `0.0001`, while placeholders use `0.01`. @@ -88,22 +88,22 @@ world reset and shutdown require no clear, cancellation or resource release. ```mermaid sequenceDiagram - participant Loader as StreamingWorldLoader + participant Caller as Grouper or StreamingWorldLoader participant Resolver as M2PlacementTransformResolver - participant Consumer as Group/placeholder/instance path - Loader->>Resolver: resolve_basis(rotation, path) - Resolver-->>Loader: unscaled Basis - Loader->>Resolver: resolve_origin_offset(rotation, path, scale) - Resolver-->>Loader: local offset - Loader->>Consumer: compose historical scaled Transform3D + participant Consumer as Grouper/placeholder/instance path + Caller->>Resolver: resolve_basis(rotation, path) + Resolver-->>Caller: unscaled Basis + Caller->>Resolver: resolve_origin_offset(rotation, path, scale) + Resolver-->>Caller: local offset + Caller->>Consumer: compose historical scaled Transform3D ``` ## Ownership, threading and resources - The resolver retains no mutable state or engine resources. -- The loader owns placement records, final transforms, tasks, queues, caches, - MultiMesh/Node/Mesh/material/RID creation and cleanup. -- Pure methods are safe for current worker grouping and main-thread consumers. +- The grouper owns worker-path final transforms; the loader owns placement records, + tasks, queues, caches and MultiMesh/Node/Mesh/material/RID creation and cleanup. +- Pure methods are safe for grouper worker use and main-thread consumers. - Returned `Basis`/`Vector3` values have no lifetime coupling to the resolver. ## Errors, cancellation and recovery @@ -134,14 +134,15 @@ versions and rebuild instructions remain unchanged. ## Diagnostics and observability The resolver emits no logs. The contract verifier compares formulas/constants, -checks all three loader consumers and records bounded synthetic timing. Visual +checks two direct loader consumers plus the grouper consumer and records bounded +synthetic timing. Visual and asset-backed evidence remains in the renderer checkpoint workflow. ## Verification - `verify_m2_placement_transform_resolver.gd`: regular fallback, path normalization, both cliff rocks, both waterfalls, twist anchor, negative-scale - compensation, three loader adapters, source boundary and 10,000-pair timing. + compensation, three consumers, source boundary and 10,000-pair timing. - M2 unique/dedupe, terrain, facade, internal-access, manifest, shutdown, scheduler, streaming and coordinate regressions remain required. - Fidelity evidence is exact extraction of existing calibrated formulas. The @@ -149,7 +150,7 @@ and asset-backed evidence remains in the renderer checkpoint workflow. ## Extension points -- The next package may compose this resolver into a pure placement grouper. +- `M2PlacementGrouper` composes this resolver for the worker grouping path. - New model corrections require separate measured fidelity evidence; this module is not a generic override registry. @@ -173,11 +174,13 @@ and asset-backed evidence remains in the renderer checkpoint workflow. | Path | Responsibility | |---|---| | `src/render/m2/m2_placement_transform_resolver.gd` | Pure basis/offset rules and calibrated allowlists | -| `src/scenes/streaming/streaming_world_loader.gd` | Three adapters and final Transform3D/resource ownership | +| `src/render/m2/m2_placement_grouper.gd` | Grouping adapter and local Transform3D ownership | +| `src/scenes/streaming/streaming_world_loader.gd` | Two direct adapters and renderer resource ownership | | `src/tools/verify_m2_placement_transform_resolver.gd` | Formula, source and performance regression | ## Related decisions and references +- [`m2-placement-grouper.md`](m2-placement-grouper.md) - [`m2-unique-placement-registry.md`](m2-unique-placement-registry.md) - [`world-renderer.md`](world-renderer.md) - [`../../RENDER.md`](../../RENDER.md) diff --git a/docs/modules/world-renderer.md b/docs/modules/world-renderer.md index de18a6e..9bd6a28 100644 --- a/docs/modules/world-renderer.md +++ b/docs/modules/world-renderer.md @@ -5,9 +5,9 @@ | Field | Value | |---|---| | Status | Partial | -| Target/work package | M00 baseline; `M01-RND-STREAMING-FOCUS-001`; `M01-QAR-SERVER-SPAWN-RENDERER-001`; M03 facade/planner/scheduler/internal-access/ground/environment/entity packages; M03 terrain packages; M03 M2 unique/transform packages | +| Target/work package | M00 baseline; `M01-RND-STREAMING-FOCUS-001`; `M01-QAR-SERVER-SPAWN-RENDERER-001`; M03 facade/planner/scheduler/internal-access/ground/environment/entity packages; M03 terrain packages; M03 M2 unique/transform/grouping packages | | Owners | Renderer workstream / milestone integrator | -| Last verified | Worktree `work/sindo-main-codex/m03-m2-placement-transform`, 2026-07-16 | +| Last verified | Worktree `work/sindo-main-codex/m03-m2-placement-grouper`, 2026-07-16 | | Profiles/capabilities | `Performance`, `Balanced`, `High`, `Custom`; Blizzlike fidelity incomplete | ## Purpose @@ -48,7 +48,9 @@ flowchart LR ChunkQueue --> Loader Loader --> M2Registry[M2UniquePlacementRegistry] M2Registry --> Loader - Loader --> M2Transform[M2PlacementTransformResolver] + Loader --> M2Grouper[M2PlacementGrouper] + M2Grouper --> M2Transform[M2PlacementTransformResolver] + M2Transform --> M2Grouper M2Transform --> Loader Native --> Parsed[Parsed tile/model data] Parsed --> Loader @@ -118,6 +120,7 @@ from externally reading/writing loader-private queue, task, cache and tile-state | `TerrainChunkGeometryQueuePlanner.plan` | Internal pure terrain query | Produces chunk create/remove requests from current and desired state | Synchronous per queue rebuild | Invalid focus returns empty arrays | | `M2UniquePlacementRegistry.reserve/release/clear` | Internal M2 service | Owns positive cross-tile ADT placement-ID reservations | Renderer thread/map session | Invalid/unkeyed/non-owner inputs preserve documented fallback | | `M2PlacementTransformResolver.resolve_basis/resolve_origin_offset` | Internal pure M2 service | Resolves regular and calibrated model-specific ADT placement transforms | Worker/main thread; stateless | Unknown paths use regular basis and zero offset | +| `M2PlacementGrouper.group_placements` | Internal pure M2 service | Validates and groups ordered tile-local placement transforms by normalized path | Worker/main thread; stateless | Invalid variants/name IDs/empty paths are skipped | Публичным contract не считаются `StreamingWorldLoader` methods/properties, внутренние dictionaries, queues, job records и generated node names. Scene-owned @@ -143,7 +146,8 @@ loader configuration remains transitional composition data, not a caller API. | Internal plan | Parsed chunks, typed focus and immutable chunk-LOD policy | Loader / `TerrainChunkLodPlanner` | Loader desired state | Fresh dictionary; no retained resources | One target refresh | | Internal plan | Current/desired chunk state and typed focus | Loader / `TerrainChunkGeometryQueuePlanner` | Loader chunk queues | Fresh request arrays; no retained resources | One queue rebuild | | Internal registry | Tile key and M2 placement unique IDs | Loader / `M2UniquePlacementRegistry` | Filtered grouping input and tile retry state | Registry-owned strings; fresh result arrays | Map session | -| Internal transform | Rotation/path/scale | Loader / `M2PlacementTransformResolver` | Group/placeholder/instance transforms | Value-only Basis/Vector3 | One placement | +| Internal transform | Rotation/path/scale | Loader or grouper / `M2PlacementTransformResolver` | Group/placeholder/instance transforms | Value-only Basis/Vector3 | One placement | +| Internal grouping | Tile origin, M2 names and placements | Loader / `M2PlacementGrouper` | Loader worker result/build job | Fresh Dictionary/Transform3D arrays | One grouping task | | Output | Desired tile/detail operations | Streamer plan application | Finalize queues | Loader-owned | Cross-frame | | Output | Terrain/M2/WMO/liquid instances | Loader/builders | Godot world/renderer | Loader/world owner | Main-thread attach | | Output | Detached renderer metrics | `WorldRenderFacade` | Capture/baseline tools | Caller-owned deep copy | On demand | @@ -188,8 +192,9 @@ flowchart TD DesiredChunkLod --> ChunkQueue[TerrainChunkGeometryQueuePlanner] ChunkQueue --> B R --> M2Registry[M2UniquePlacementRegistry reserve] - M2Registry --> M2 - M2Transform[M2PlacementTransformResolver] --> M2 + M2Registry --> M2Grouper[M2PlacementGrouper] + M2Transform[M2PlacementTransformResolver] --> M2Grouper + M2Grouper --> M2 R --> B[RenderBudgetScheduler permits] B --> Terrain[Terrain attach/upgrade] B --> M2[M2 group/MultiMesh attach] @@ -305,7 +310,10 @@ sequenceDiagram The loader retains owned/skipped tile arrays, candidate retry, grouping/build tasks, caches and all MultiMesh/Node/Mesh/material/RID side effects. - `M2PlacementTransformResolver` is stateless and retains no resources. The - loader owns final transforms and every grouping/build/render side effect. + grouper owns worker-path final transforms; direct placeholder/instance + transforms and every build/render side effect remain loader-owned. +- `M2PlacementGrouper` is stateless and owns only call-local grouped transforms. + The loader retains tasks, mutex/result queues, stale checks and build state. - Rendered-ground query results and diagnostic snapshots are caller-owned values; the facade never returns Mesh, Node, tile-state or queue references. - Loaded-mesh ground sampling is renderer diagnostics, not authoritative gameplay @@ -393,6 +401,8 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m bounded reserve/release timing without a world scene. - M2 placement transform contract: regular/cliffrock/waterfall formulas, path normalization, scale compensation, all three consumers and bounded timing. +- M2 placement grouper contract: invalid/default records, historical path rules, + group order, local transforms, fresh output, source boundary and bounded timing. - Budget scheduler contract: exact lane exhaustion, shared chunk removal/create priority, independent lanes, frame reset, invalid limits, terminal cancellation, dependency boundaries and bounded permit timing without loading a world scene. @@ -424,6 +434,7 @@ Exact exported settings and cache versions remain documented in [`../../RENDER.m | Static M2 placement | Partial | MultiMesh/dedupe probes | Full materials/animation/effects | | M2 unique placement registry | Implemented extraction | Scene-free ownership/lifecycle/timing contract and historical `uid:11785` smoke | Group/build/tasks/finalization and asset-backed p95/p99 remain pending | | M2 placement transform resolver | Implemented extraction | Scene-free formula/source/timing contract across three consumers | Asset-backed visual recheck and general placement parity pending | +| M2 placement grouper | Implemented extraction | Scene-free validation/order/transform/source/timing contract | Worker/build state, spatial cells and asset-backed p95/p99 remain pending | | WMO rendering | Partial | Cached group rendering | Portals/rooms/material parity | | Liquids | Partial | MH2O/MLIQ paths | LiquidType/depth/shore fidelity | | Sky/light | Partial | DBC controller/logs | Zone transition paired validation | diff --git a/src/render/m2/m2_placement_grouper.gd b/src/render/m2/m2_placement_grouper.gd new file mode 100644 index 0000000..53b338e --- /dev/null +++ b/src/render/m2/m2_placement_grouper.gd @@ -0,0 +1,65 @@ +class_name M2PlacementGrouper +extends RefCounted + +## Stateless ADT M2 placement validation, normalization and transform grouping. + +const M2_PLACEMENT_TRANSFORM_RESOLVER_SCRIPT := preload( + "res://src/render/m2/m2_placement_transform_resolver.gd" +) + + +## Groups valid placements by normalized relative M2 path while preserving +## placement order inside each group. Returned containers are caller-owned. +func group_placements( + tile_origin: Vector3, + m2_names: PackedStringArray, + m2_placements: Array +) -> Dictionary: + var groups: Dictionary = {} + var placement_transform_resolver := ( + M2_PLACEMENT_TRANSFORM_RESOLVER_SCRIPT.new() + ) + for placement_variant in m2_placements: + if not (placement_variant is Dictionary): + continue + var placement: Dictionary = placement_variant + var name_id: int = int(placement.get("name_id", -1)) + if name_id < 0 or name_id >= m2_names.size(): + continue + + var normalized_relative_path := _normalize_relative_path(m2_names[name_id]) + if normalized_relative_path.is_empty(): + continue + + var world_position: Vector3 = placement.get("pos", Vector3.ZERO) + var local_position := world_position - tile_origin + var rotation_radians: Vector3 = placement.get("rot", Vector3.ZERO) + var scale_value := float(placement.get("scale", 1.0)) + var placement_basis: Basis = placement_transform_resolver.resolve_basis( + rotation_radians, + normalized_relative_path + ) + var origin_offset: Vector3 = placement_transform_resolver.resolve_origin_offset( + rotation_radians, + normalized_relative_path, + scale_value + ) + var local_transform := Transform3D( + placement_basis.scaled(Vector3.ONE * maxf(scale_value, 0.0001)), + local_position + origin_offset + ) + + if not groups.has(normalized_relative_path): + groups[normalized_relative_path] = [] + (groups[normalized_relative_path] as Array).append(local_transform) + return groups + + +func _normalize_relative_path(relative_path: String) -> String: + var normalized_relative_path := relative_path.replace("\\", "/") + if ( + normalized_relative_path.ends_with(".mdx") + or normalized_relative_path.ends_with(".mdl") + ): + normalized_relative_path = normalized_relative_path.get_basename() + ".m2" + return normalized_relative_path diff --git a/src/render/m2/m2_placement_grouper.gd.uid b/src/render/m2/m2_placement_grouper.gd.uid new file mode 100644 index 0000000..02a274c --- /dev/null +++ b/src/render/m2/m2_placement_grouper.gd.uid @@ -0,0 +1 @@ +uid://dkktxa5essfn diff --git a/src/scenes/streaming/streaming_world_loader.gd b/src/scenes/streaming/streaming_world_loader.gd index 2fa0ed1..3ff1971 100644 --- a/src/scenes/streaming/streaming_world_loader.gd +++ b/src/scenes/streaming/streaming_world_loader.gd @@ -30,6 +30,9 @@ const M2_UNIQUE_PLACEMENT_REGISTRY_SCRIPT := preload( const M2_PLACEMENT_TRANSFORM_RESOLVER_SCRIPT := preload( "res://src/render/m2/m2_placement_transform_resolver.gd" ) +const M2_PLACEMENT_GROUPER_SCRIPT := preload( + "res://src/render/m2/m2_placement_grouper.gd" +) const STREAMING_TARGET_PLANNER_SCRIPT := preload("res://src/render/streaming/streaming_target_planner.gd") const STREAMING_TARGET_POLICY_SCRIPT := preload("res://src/render/streaming/streaming_target_policy.gd") const RENDER_BUDGET_SCHEDULER_SCRIPT := preload("res://src/render/streaming/render_budget_scheduler.gd") @@ -215,6 +218,7 @@ var _m2_unique_placement_registry := ( M2_UNIQUE_PLACEMENT_REGISTRY_SCRIPT.new() ) var _m2_placement_transform_resolver := M2_PLACEMENT_TRANSFORM_RESOLVER_SCRIPT.new() +var _m2_placement_grouper := M2_PLACEMENT_GROUPER_SCRIPT.new() var _m2_mesh_cache: Dictionary = {} var _m2_mesh_load_requests: Dictionary = {} var _m2_mesh_finalize_queue: Array = [] @@ -4119,36 +4123,11 @@ func _group_tile_m2_task( tile_origin: Vector3, m2_names: PackedStringArray, m2_placements: Array) -> void: - var groups: Dictionary = {} - for placement_variant in m2_placements: - if not (placement_variant is Dictionary): - continue - var placement: Dictionary = placement_variant - var name_id: int = int(placement.get("name_id", -1)) - if name_id < 0 or name_id >= m2_names.size(): - continue - - var normalized: String = str(m2_names[name_id]).replace("\\", "/") - if normalized.ends_with(".mdx") or normalized.ends_with(".mdl"): - normalized = normalized.get_basename() + ".m2" - if normalized.is_empty(): - continue - - var world_pos: Vector3 = placement.get("pos", Vector3.ZERO) - var pos: Vector3 = world_pos - tile_origin - var rot: Vector3 = placement.get("rot", Vector3.ZERO) - var scale_value: float = float(placement.get("scale", 1.0)) - var basis: Basis = _m2_placement_transform_resolver.resolve_basis(rot, normalized) - var offset: Vector3 = _m2_placement_transform_resolver.resolve_origin_offset( - rot, - normalized, - scale_value - ) - var xform := Transform3D(basis.scaled(Vector3.ONE * maxf(scale_value, 0.0001)), pos + offset) - - if not groups.has(normalized): - groups[normalized] = [] - (groups[normalized] as Array).append(xform) + var groups: Dictionary = _m2_placement_grouper.group_placements( + tile_origin, + m2_names, + m2_placements + ) _m2_group_result_mutex.lock() _m2_group_result_queue.append({ diff --git a/src/tools/verify_m2_placement_grouper.gd b/src/tools/verify_m2_placement_grouper.gd new file mode 100644 index 0000000..a47c331 --- /dev/null +++ b/src/tools/verify_m2_placement_grouper.gd @@ -0,0 +1,296 @@ +extends SceneTree + +## Asset-free contract, dependency and timing regression for ADT M2 grouping. + +const GROUPER_SCRIPT := preload("res://src/render/m2/m2_placement_grouper.gd") +const TRANSFORM_RESOLVER_SCRIPT := preload( + "res://src/render/m2/m2_placement_transform_resolver.gd" +) +const GROUPER_PATH := "res://src/render/m2/m2_placement_grouper.gd" +const LOADER_PATH := "res://src/scenes/streaming/streaming_world_loader.gd" + + +func _initialize() -> void: + var failures: Array[String] = [] + _verify_empty_and_invalid_inputs(failures) + _verify_historical_path_normalization(failures) + _verify_group_order(failures) + _verify_tile_local_defaults(failures) + _verify_regular_transform(failures) + _verify_waterfall_transform(failures) + _verify_scale_clamp(failures) + _verify_fresh_output(failures) + _verify_source_boundaries(failures) + var elapsed_milliseconds := _verify_bounded_timing(failures) + if not failures.is_empty(): + for failure in failures: + push_error("M2_PLACEMENT_GROUPER: %s" % failure) + quit(1) + return + print( + "M2_PLACEMENT_GROUPER PASS cases=9 iterations=100 placements=256 elapsed_ms=%.3f" + % elapsed_milliseconds + ) + quit(0) + + +func _verify_empty_and_invalid_inputs(failures: Array[String]) -> void: + var grouper := GROUPER_SCRIPT.new() + _expect_equal( + grouper.group_placements(Vector3.ZERO, PackedStringArray(), []).size(), + 0, + "empty", + failures + ) + var placements: Array = [ + "invalid", + {"name_id": -1}, + {"name_id": 2}, + {"name_id": 0}, + ] + var groups: Dictionary = grouper.group_placements( + Vector3.ZERO, + PackedStringArray([""]), + placements + ) + _expect_equal(groups.size(), 0, "invalid placements", failures) + + +func _verify_historical_path_normalization(failures: Array[String]) -> void: + var grouper := GROUPER_SCRIPT.new() + var names := PackedStringArray([ + "World\\Generic\\Tree.mdx", + "World/Generic/Rock.mdl", + "World/Generic/Upper.MDX", + ]) + var placements: Array = [{"name_id": 0}, {"name_id": 1}, {"name_id": 2}] + var groups: Dictionary = grouper.group_placements(Vector3.ZERO, names, placements) + _expect_true(groups.has("World/Generic/Tree.m2"), "backslash and mdx normalization", failures) + _expect_true(groups.has("World/Generic/Rock.m2"), "mdl normalization", failures) + _expect_true(groups.has("World/Generic/Upper.MDX"), "uppercase suffix preservation", failures) + + +func _verify_group_order(failures: Array[String]) -> void: + var grouper := GROUPER_SCRIPT.new() + var placements: Array = [ + {"name_id": 0, "pos": Vector3(3.0, 0.0, 0.0)}, + {"name_id": 1, "pos": Vector3(8.0, 0.0, 0.0)}, + {"name_id": 0, "pos": Vector3(5.0, 0.0, 0.0)}, + ] + var groups: Dictionary = grouper.group_placements( + Vector3.ZERO, + PackedStringArray(["Tree.m2", "Rock.m2"]), + placements + ) + _expect_equal(groups.size(), 2, "group count", failures) + var tree_transforms: Array = groups.get("Tree.m2", []) + _expect_equal(tree_transforms.size(), 2, "tree transform count", failures) + if tree_transforms.size() == 2: + _expect_vector_near(tree_transforms[0].origin, Vector3(3.0, 0.0, 0.0), "first tree", failures) + _expect_vector_near(tree_transforms[1].origin, Vector3(5.0, 0.0, 0.0), "second tree", failures) + + +func _verify_tile_local_defaults(failures: Array[String]) -> void: + var grouper := GROUPER_SCRIPT.new() + var groups: Dictionary = grouper.group_placements( + Vector3(10.0, 20.0, 30.0), + PackedStringArray(["Tree.m2"]), + [{"name_id": 0}] + ) + var transform: Transform3D = (groups["Tree.m2"] as Array)[0] + _expect_vector_near(transform.origin, Vector3(-10.0, -20.0, -30.0), "default local position", failures) + _expect_basis_near(transform.basis, Basis.IDENTITY, "default basis and scale", failures) + + +func _verify_regular_transform(failures: Array[String]) -> void: + var grouper := GROUPER_SCRIPT.new() + var rotation := Vector3(0.2, -0.4, 0.6) + var groups: Dictionary = grouper.group_placements( + Vector3(1.0, 2.0, 3.0), + PackedStringArray(["Tree.m2"]), + [{ + "name_id": 0, + "pos": Vector3(5.0, 8.0, 13.0), + "rot": rotation, + "scale": 2.0, + }] + ) + var transform: Transform3D = (groups["Tree.m2"] as Array)[0] + _expect_basis_near(transform.basis, Basis.from_euler(rotation).scaled(Vector3.ONE * 2.0), "regular basis", failures) + _expect_vector_near(transform.origin, Vector3(4.0, 6.0, 10.0), "regular origin", failures) + + +func _verify_waterfall_transform(failures: Array[String]) -> void: + var grouper := GROUPER_SCRIPT.new() + var resolver := TRANSFORM_RESOLVER_SCRIPT.new() + var path := "World/X/ElwynnTallWaterfall01.m2" + var rotation := Vector3(0.05, -0.2, 0.15) + var scale_value := 1.5 + var position := Vector3(20.0, 30.0, 40.0) + var tile_origin := Vector3(2.0, 3.0, 4.0) + var groups: Dictionary = grouper.group_placements( + tile_origin, + PackedStringArray([path]), + [{ + "name_id": 0, + "pos": position, + "rot": rotation, + "scale": scale_value, + }] + ) + var transform: Transform3D = (groups[path] as Array)[0] + var expected_basis: Basis = resolver.resolve_basis(rotation, path).scaled( + Vector3.ONE * scale_value + ) + var expected_origin: Vector3 = ( + position + - tile_origin + + resolver.resolve_origin_offset(rotation, path, scale_value) + ) + _expect_basis_near(transform.basis, expected_basis, "waterfall basis", failures) + _expect_vector_near(transform.origin, expected_origin, "waterfall origin", failures) + + +func _verify_scale_clamp(failures: Array[String]) -> void: + var grouper := GROUPER_SCRIPT.new() + var groups: Dictionary = grouper.group_placements( + Vector3.ZERO, + PackedStringArray(["Tree.m2"]), + [{"name_id": 0, "scale": -5.0}] + ) + var transform: Transform3D = (groups["Tree.m2"] as Array)[0] + _expect_basis_near( + transform.basis, + Basis.IDENTITY.scaled(Vector3.ONE * 0.0001), + "negative scale clamp", + failures + ) + + +func _verify_fresh_output(failures: Array[String]) -> void: + var grouper := GROUPER_SCRIPT.new() + var names := PackedStringArray(["Tree.m2"]) + var placements: Array = [{"name_id": 0}] + var first: Dictionary = grouper.group_placements(Vector3.ZERO, names, placements) + (first["Tree.m2"] as Array).clear() + var second: Dictionary = grouper.group_placements(Vector3.ZERO, names, placements) + _expect_equal((second["Tree.m2"] as Array).size(), 1, "fresh output", failures) + + +func _verify_source_boundaries(failures: Array[String]) -> void: + var loader_source := _read_text(LOADER_PATH, failures) + var grouper_source := _read_text(GROUPER_PATH, failures) + var grouping_task_source := _source_between( + loader_source, + "func _group_tile_m2_task(", + "func _drain_m2_group_results()", + failures + ) + _expect_equal( + loader_source.count("_m2_placement_grouper.group_placements("), + 1, + "one loader grouping adapter", + failures + ) + _expect_true( + not grouping_task_source.contains("for placement_variant in m2_placements"), + "loader omits grouping loop", + failures + ) + _expect_equal( + grouper_source.count("placement_transform_resolver.resolve_basis("), + 1, + "one grouper basis adapter", + failures + ) + _expect_equal( + grouper_source.count("placement_transform_resolver.resolve_origin_offset("), + 1, + "one grouper offset adapter", + failures + ) + for forbidden_dependency in ["WorkerThreadPool", "Mutex", "RenderingServer", "ResourceLoader"]: + _expect_true( + not grouper_source.contains(forbidden_dependency), + "grouper omits %s" % forbidden_dependency, + failures + ) + + +func _verify_bounded_timing(failures: Array[String]) -> float: + var grouper := GROUPER_SCRIPT.new() + var names := PackedStringArray(["Tree.m2", "Rock.mdx"]) + var placements: Array = [] + for placement_index in range(256): + placements.append({ + "name_id": placement_index % 2, + "pos": Vector3(float(placement_index), 0.0, float(-placement_index)), + "rot": Vector3(0.1, 0.2, 0.3), + "scale": 1.0, + }) + var started_microseconds := Time.get_ticks_usec() + for iteration in range(100): + var groups: Dictionary = grouper.group_placements(Vector3.ZERO, names, placements) + if groups.size() != 2: + failures.append("timing iteration %d produced wrong group count" % iteration) + break + var elapsed_milliseconds := float(Time.get_ticks_usec() - started_microseconds) / 1000.0 + _expect_true(elapsed_milliseconds < 1000.0, "100 x 256 grouping bounded", failures) + return elapsed_milliseconds + + +func _read_text(path: String, failures: Array[String]) -> String: + var file := FileAccess.open(path, FileAccess.READ) + if file == null: + failures.append("cannot open %s" % path) + return "" + return file.get_as_text() + + +func _source_between( + source: String, + start_marker: String, + end_marker: String, + failures: Array[String] +) -> String: + var start_offset := source.find(start_marker) + var end_offset := source.find(end_marker, start_offset + start_marker.length()) + if start_offset < 0 or end_offset <= start_offset: + failures.append("cannot isolate source between %s and %s" % [start_marker, end_marker]) + return "" + return source.substr(start_offset, end_offset - start_offset) + + +func _expect_basis_near( + actual_basis: Basis, + expected_basis: Basis, + label: String, + failures: Array[String] +) -> void: + if not actual_basis.is_equal_approx(expected_basis): + failures.append("%s basis mismatch" % label) + + +func _expect_vector_near( + actual_value: Vector3, + expected_value: Vector3, + label: String, + failures: Array[String] +) -> void: + if not actual_value.is_equal_approx(expected_value): + failures.append("%s expected %s, got %s" % [label, expected_value, actual_value]) + + +func _expect_equal( + actual_value: int, + expected_value: int, + label: String, + failures: Array[String] +) -> void: + if actual_value != expected_value: + failures.append("%s expected %d, got %d" % [label, expected_value, actual_value]) + + +func _expect_true(actual_value: bool, label: String, failures: Array[String]) -> void: + if not actual_value: + failures.append("%s expected true" % label) diff --git a/src/tools/verify_m2_placement_grouper.gd.uid b/src/tools/verify_m2_placement_grouper.gd.uid new file mode 100644 index 0000000..3f6c7f5 --- /dev/null +++ b/src/tools/verify_m2_placement_grouper.gd.uid @@ -0,0 +1 @@ +uid://bbx57qy0lp11w diff --git a/src/tools/verify_m2_placement_transform_resolver.gd b/src/tools/verify_m2_placement_transform_resolver.gd index 6fff9cb..d55a383 100644 --- a/src/tools/verify_m2_placement_transform_resolver.gd +++ b/src/tools/verify_m2_placement_transform_resolver.gd @@ -4,6 +4,7 @@ extends SceneTree const RESOLVER_SCRIPT := preload("res://src/render/m2/m2_placement_transform_resolver.gd") const LOADER_PATH := "res://src/scenes/streaming/streaming_world_loader.gd" +const GROUPER_PATH := "res://src/render/m2/m2_placement_grouper.gd" const WATERFALL_WORLD_YAW_OFFSET := PI * 0.5 const CLIFFROCK_WORLD_YAW_OFFSET := PI @@ -130,6 +131,7 @@ func _verify_path_normalization_and_scale_clamp(failures: Array[String]) -> void func _verify_loader_boundary(failures: Array[String]) -> void: var source := _read_text(LOADER_PATH, failures) + var grouper_source := _read_text(GROUPER_PATH, failures) _expect_true( source.contains("M2_PLACEMENT_TRANSFORM_RESOLVER_SCRIPT.new()"), "loader composes resolver", @@ -137,14 +139,26 @@ func _verify_loader_boundary(failures: Array[String]) -> void: ) _expect_equal( source.count("_m2_placement_transform_resolver.resolve_basis("), - 3, - "three basis adapters", + 2, + "two direct loader basis adapters", failures ) _expect_equal( source.count("_m2_placement_transform_resolver.resolve_origin_offset("), - 3, - "three offset adapters", + 2, + "two direct loader offset adapters", + failures + ) + _expect_equal( + grouper_source.count("placement_transform_resolver.resolve_basis("), + 1, + "one grouped basis adapter", + failures + ) + _expect_equal( + grouper_source.count("placement_transform_resolver.resolve_origin_offset("), + 1, + "one grouped offset adapter", failures ) _expect_true( From 7587d3bd3c6ae9431822712bc4ac6090417b2e2a Mon Sep 17 00:00:00 2001 From: sindoring Date: Fri, 17 Jul 2026 00:06:10 +0400 Subject: [PATCH 2/2] docs(M03): hand off M2 placement grouper --- .../M03-RND-M2-PLACEMENT-GROUPER-001.md | 29 ++++++++++++++----- 1 file changed, 22 insertions(+), 7 deletions(-) diff --git a/coordination/claims/M03-RND-M2-PLACEMENT-GROUPER-001.md b/coordination/claims/M03-RND-M2-PLACEMENT-GROUPER-001.md index e863fcf..41a7584 100644 --- a/coordination/claims/M03-RND-M2-PLACEMENT-GROUPER-001.md +++ b/coordination/claims/M03-RND-M2-PLACEMENT-GROUPER-001.md @@ -81,14 +81,29 @@ grouping from the worker callback into a scene-free service. ## Status -- State: claimed -- Done: boundary, exact preservation rules and ownership published -- Next: implement grouper, migrate worker callback, tests and docs +- State: handoff +- Done: pure grouper, worker adapter, regressions and documentation verified +- Next: M03 integrator acceptance and merge to master - Blocked by: + + ## Handoff -- Commit: -- Results: -- Remaining risks: -- Documentation updated: +- Commit: `9df32d3` (`refactor(M03): extract M2 placement grouper`) +- Results: dedicated grouper PASS (`cases=9`, `iterations=100`, + `placements=256`, `elapsed_ms=67.736`); transform resolver PASS (`cases=9`, + `iterations=10000`, `elapsed_ms=28.571`); M2 registry/dedupe, terrain + cache/LOD/queue, facade, internal-access, manifest, shutdown, scheduler, + streaming planner/focus, terrain-query and coordinate-boundary suites PASS; + editor import parsed the project with only expected missing private corpus + diagnostics; documentation PASS (`module_specs=17`, `required_files=7`); + coordination PASS with 23 pre-existing expired M00/M01 warnings; checkpoint + dry-run planned `7/7`. +- Remaining risks: no private asset-backed traversal/visual comparison; + uppercase extension preservation is historical but inconsistent with the + resolver's case-insensitive exception matching; grouping remains model-path + based rather than spatial-cell based; worker/build state and p95/p99 remain. +- Documentation updated: inline public API; new + `docs/modules/m2-placement-grouper.md`; transform resolver module spec, + renderer module registry, `docs/modules/world-renderer.md` and `RENDER.md`.