# Transformation & fusion blocks

Category dirs: `inference/core/workflows/core_steps/transformations/**` and `.../fusion/**`. See the universal invariants section in `SKILL.md` first.

**What / when.** `transformations/**` blocks reshape geometry or spatial data in-place (crop images, offset/filter/reproject boxes, slice, stabilize); `fusion/**` blocks combine or reduce multiple predictions/batch elements into one (consensus voting, class replacement, stitching crops back, collapsing a batch dimension). Reach here whenever you're moving `sv.Detections` between coordinate spaces or merging/reducing batches — not producing predictions from a model.

**Mental model.** Almost every block subclasses the plain `WorkflowBlock`/`WorkflowBlockManifest` (from `inference.core.workflows.prototypes.block`), sets `block_type` to `"transformation"` or `"fusion"`, consumes/produces the detection kinds (`OBJECT_DETECTION_PREDICTION_KIND`, `INSTANCE_SEGMENTATION_PREDICTION_KIND`, `KEYPOINT_DETECTION_PREDICTION_KIND`) and/or `IMAGE_KIND`, and threads work over `sv.Detections`. The distinguishing axis is **dimensionality**, declared via manifest classmethods:
- **1:1, same dim** — process each batch element independently, return a list aligned to the input `Batch`. Declare batch inputs with `get_parameters_accepting_batches()`. See `detection_offset/v1.py`, `detections_filter/v1.py`.
- **+1 (fan-out)** — one input element → many outputs. `get_output_dimensionality_offset() -> 1`; `run` returns a list-of-lists. See `dynamic_crop/v1.py`, `image_slicer/v2.py`.
- **-1 (collapse)** — nested batch → flat. `get_output_dimensionality_offset() -> -1`. See `dimension_collapse/v1.py`.
- **mixed-dim fusion** — inputs live at different nesting levels; align with `get_input_dimensionality_offsets()` + `get_dimensionality_reference_property()`. See `detections_stitch/v1.py` (predictions +1 vs reference_image) and `detections_classes_replacement/v1.py` (detections at 0, classifications at +1).

Most blocks are **stateless** (pure function of inputs). The video family is **stateful** — instance attributes hold per-`video_identifier` caches (e.g. `stabilize_detections/v1.py`).

**Nuances an author MUST know.**
- **Output-key names vary — bind carefully.** Downstream blocks bind to your exact output name (`crops`, `predictions`, `slices`, `tracked_detections`). `dimension_collapse/v1.py` outputs `output`, not `predictions`. Keep `describe_outputs()` names and the `run` dict keys identical and stable across versions.
- **Coordinate systems must move together.** When you crop/offset/stitch, every geometry field has to be re-based in lockstep: `xyxy`, `mask`, keypoints (`KEYPOINTS_XY_KEY_IN_SV_DETECTIONS`), polygons (`POLYGON_KEY_IN_SV_DETECTIONS`), and OBB corners (`ORIENTED_BOX_COORDINATES`). `crop_image` in `dynamic_crop/v1.py` and `move_detections` in `detections_stitch/v1.py` both explicitly translate all five — forgetting OBB corners leaves them in the parent frame next to crop-local `xyxy`. Use `WorkflowImageData.create_crop(...)` so parent-coordinate metadata is attached for the inverse stitch.
- **`sv.Detections` data-density / metadata keys.** Blocks depend on keys other blocks planted: `dynamic_crop` requires `DETECTION_ID_KEY` and raises if absent; `detections_stitch` requires `PARENT_COORDINATES_KEY` (and rejects scaled detections via `SCALING_RELATIVE_TO_PARENT_KEY`); `detection_offset` rewrites `PARENT_ID_KEY`/`DETECTION_ID_KEY` (and carries the known-bug TODO of breaking parent coords — issue #380). When you mutate boxes, regenerate `DETECTION_ID_KEY` (fresh `uuid4`) and set `PARENT_ID_KEY` to the source id, exactly as `detections_classes_replacement/v1.py` does.
- **Statefulness + REMOTE execution.** Stateful video blocks must key state by `image.video_metadata.video_identifier` (a `Dict[str, ...]` per video, see `__init__` in `stabilize_detections/v1.py`) and evict entries when tracks age out (window-based deletion of `cached_detections`). Declare `get_restrictions()` returning `STATEFUL_VIDEO_HTTP_SOFT_RESTRICTION` + `STILL_IMAGE_INPUT_SOFT_RESTRICTION` so the engine warns on stateless HTTP / single-image use where the cache never accumulates.
- **Fusion aggregation semantics are explicit knobs, not magic.** `detections_stitch` exposes `overlap_filtering_strategy` (none/nms/nmm) + `iou_threshold`; `detections_consensus/v1.py` takes a `List[Selector(...)]` (`predictions_batches`) — a variadic list of prediction sources — plus separate aggregation modes for presence-confidence, merge-confidence, coordinates, and masks. If your fusion merges overlapping boxes, surface the strategy rather than hard-coding NMS.
- **Empty / None handling.** Fusion inputs are frequently absent; set `accepts_empty_values() -> True` and branch (`detections_classes_replacement` returns `sv.Detections.empty()` / `{"predictions": None}`). Fan-out blocks emit `{"crops": None, "predictions": None}` placeholders for zero-area crops (`dynamic_crop`) to keep batch alignment.

**Example blocks to pattern-match (simplest → most representative).**
- `transformations/detection_offset/v1.py` — simplest 1:1 same-dim detections transform; the `Batch[sv.Detections]` → list return shape, box clipping to `image_dimensions`, and parent/detection-id rewiring.
- `fusion/dimension_collapse/v1.py` — the minimal -1 block; `get_output_dimensionality_offset() -> -1` + a two-line `run`.
- `transformations/dynamic_crop/v1.py` — canonical +1 fan-out; `create_crop`, translating all geometry into crop space, and per-detection None placeholders. Pairs with `image_slicer/v2.py` (SAHI tiling, same +1 pattern).
- `fusion/detections_stitch/v1.py` — the inverse of crop/slice; mixed-dim fusion (`get_input_dimensionality_offsets` + `get_dimensionality_reference_property`), reading `PARENT_COORDINATES_KEY` to move boxes/masks/OBB back to the original frame, and nms/nmm merge. The other half of the `dynamic_crop`/`image_slicer` → model → `detections_stitch` loop.
- `fusion/detections_classes_replacement/v1.py` — canonical two-stage fusion joining detections (dim 0) with per-crop classifications (dim +1) via `PARENT_ID_KEY`, with positional fallback for raw string/OCR inputs; cross-dim matching and class/confidence/id rewrite.
- `transformations/stabilize_detections/v1.py` — the reference stateful video block; per-`video_identifier` caches, Kalman/EMA smoothing, gap-fill eviction, and `get_restrictions()`.
- `fusion/detections_consensus/v1.py` — multi-source voting; the `List[Selector(...)]` variadic input and multiple independent aggregation-mode fields plus extra outputs (`object_present`, `presence_confidence`).
