# Visualization blocks

Category dir: `inference/core/workflows/core_steps/visualizations/**`. See the universal invariants section in `SKILL.md` first.

Blocks that render annotations onto an image (boxes, masks, labels, keypoints, traces, blur/pixelate, background fills) and return a new image. Reach for this category whenever the block's job is "take an image (+ usually predictions) and produce an annotated image" — almost always a thin wrapper over a `supervision` annotator.

**Mental model.** Every block subclasses a shared hierarchy in `visualizations/common/base.py`:
- `VisualizationManifest` / `VisualizationBlock` — the root. The manifest always declares an `image: Selector(kind=[IMAGE_KIND])` input (aliased `image`/`images`) plus a `copy_image: bool` toggle, and hard-codes a single output named `"image"` (the `OUTPUT_IMAGE_KEY` constant) of `IMAGE_KIND`. The block must implement `get_manifest`, `getAnnotator`, and `run`.
- `PredictionsVisualizationManifest` / `PredictionsVisualizationBlock` — adds a `predictions` input accepting the four detection kinds (object detection, instance segmentation, keypoint, RLE instance segmentation). Use for anything that annotates detections but doesn't need color config (`blur/v1.py`, `background_color/v1.py`).
- `ColorableVisualizationManifest` / `ColorableVisualizationBlock` in `visualizations/common/base_colorable.py` — the most common base. Adds `color_palette` / `palette_size` / `custom_colors` / `color_axis` and a `getPalette(...)` helper resolving names → `sv.ColorPalette`. Most annotators (bounding box, label, mask, halo, trace, corner, dot, ...) build on this.

Shared shape: operate per-image (batch handled by the engine — no `get_parameters_accepting_batches`), stateless w.r.t. workflow data, but each block instance keeps an in-memory `self.annotatorCache` dict keyed on the styling params so the `sv` annotator object is reused across frames (`getAnnotator` lazily fills it). `run` almost always ends `return {OUTPUT_IMAGE_KEY: WorkflowImageData.copy_and_replace(origin_image_data=image, numpy_image=annotated)}`. Not all blocks in the dir are colorable/prediction-driven: `model_comparison/v1.py` and `keypoint/v1.py` extend `VisualizationManifest` directly (keypoint redeclares its own `predictions` restricted to `KEYPOINT_DETECTION_PREDICTION_KIND`).

**Nuances an author MUST know.**
- **Fixed output key + no custom kinds.** The output is always `"image"`/`IMAGE_KIND` via `describe_outputs` in the base — don't rename it, and downstream stacking depends on it. These blocks consume existing kinds and produce `IMAGE_KIND`, so you generally never introduce/serialize a new data kind here.
- **`copy_image` mutation semantics.** When `copy_image=False` the annotator writes into `image.numpy_image` in place (see `run` in `bounding_box/v1.py`). Every block sets a UI `warnings` entry in `json_schema_extra` flagging that this mutates the shared input if other blocks read it — replicate that manifest warning.
- **Statefulness is the annotator cache, and it bites under REMOTE.** `trace/v1.py` stores per-track position history *inside* the cached `sv.TraceAnnotator`. It declares a SOFT `RuntimeRestriction` via `get_restrictions()` warning that on stateless/multi-replica REMOTE HTTP runtimes successive frames hit different workers, so traces reset/split — use local `InferencePipeline` execution. Any block relying on cross-frame annotator state must do the same.
- **Precondition checks on prediction contents.** Detection-driven blocks must guard for missing data density: `trace/v1.py` `run` raises if `predictions.tracker_id is None`; `label/v1.py` short-circuits on `len(predictions) == 0`; label pulls optional `sv.Detections.data` keys (e.g. `time_in_zone`, `AREA_KEY_IN_SV_DETECTIONS`) with N/A fallbacks rather than assuming presence.
- **Coordinate system = pixel space of the passed image.** `sv` annotators draw in `predictions.xyxy` pixel coords; label derives dimensions/centroids straight from `predictions.xyxy[i]` (`label/v1.py`). Predictions must already be in the coordinate frame of the `image` input (crops carry their own frame), so don't feed parent-image detections onto a crop image without re-anchoring.
- **`getAnnotator` cache key hygiene.** The cache key is a `_`-join of every styling param that changes the annotator (see `getAnnotator` in `bounding_box/v1.py`). Omitting a param from the key means stale annotators are reused when that param changes — include every config field that affects the annotator.
- **Compatibility pin.** Colorable manifests pin `get_execution_engine_compatibility() -> ">=1.3.0,<2.0.0"`; keep the same pin on new blocks in this category.

**Example blocks to pattern-match (simplest → most representative).**
- `visualizations/blur/v1.py` — simplest prediction-driven, non-colorable block (`PredictionsVisualizationManifest`); the minimal `getAnnotator`-cache + `copy_and_replace` skeleton.
- `visualizations/background_color/v1.py` — same base as blur but with a color string + opacity; consumes masks/detections without the full colorable palette machinery.
- `visualizations/bounding_box/v1.py` — the canonical `ColorableVisualizationBlock`; the full palette/`color_axis` wiring, cache key construction, and standard `run` shape. Pattern-match this first for a new colorable annotator.
- `visualizations/label/v1.py` — most feature-rich colorable block; branching label content off `sv.Detections` fields/`.data` keys, empty-predictions guard, pixel-space dimension math.
- `visualizations/trace/v1.py` — the stateful outlier; `RuntimeRestriction`/`get_restrictions()` and how per-frame annotator state interacts with REMOTE execution (contrast with the stateless `bounding_box` sibling).
- `visualizations/keypoint/v1.py` — extends `VisualizationManifest` directly (not colorable/`PredictionsVisualization`) with its own keypoint-only `predictions` selector; read when your block needs a bespoke prediction kind or annotator-mode switch (`edge`/`vertex`/`vertex_label`).
