# Analytics, tracker & sampling blocks

Category dirs: `inference/core/workflows/core_steps/analytics/**`, `.../trackers/**`, `.../sampling/**`. See the universal invariants section in `SKILL.md` first.

**What / when.** The *stateful, per-video* blocks: object trackers (assign persistent `tracker_id`), analytics that consume tracked detections to derive temporal metrics (time-in-zone, velocity, line crossings, path deviation), aggregators that fold a data stream into periodic summaries, and embedding-based sampling/anomaly gates. If a block needs to remember something from a previous frame, it lives here.

**Mental model.** Almost every block is a plain `WorkflowBlock` whose `run(...)` is called *once per frame* and which stashes cross-frame state in instance attributes keyed by `image.video_metadata.video_identifier` — see the `Dict[str, ...]` fields in `VelocityBlockV1.__init__` and `TimeInZoneBlockV3.__init__`. Two shared patterns dominate:
- **Trackers** inherit `TrackerBlockBase` (`trackers/_base.py`), which owns the per-`video_identifier` tracker dict + `InstanceCache`, filters immature tracks (`tracker_id == -1`), and emits the fixed 3-output contract `tracked_detections` / `new_instances` / `already_seen_instances` via `tracker_describe_outputs()`. A concrete tracker only implements `_create_tracker` (and optionally `_tracker_update`).
- **Analytics** blocks take `image` (for `video_metadata`) + tracked `sv.Detections`, mutate/annotate the detections, and hand them back. They accept `OBJECT_DETECTION` / `INSTANCE_SEGMENTATION` (velocity/trackers also `KEYPOINT_DETECTION`) prediction kinds and return the same kinds — data flows through, enriched.

Batch/dimensionality: these are single-image (dimensionality 1), no batch fan-out — they operate frame-at-a-time so state stays coherent. The exception is `DataAggregator` (`analytics/data_aggregator/v1.py`), a *reducer*: it returns empty outputs most frames and emits only at interval boundaries.

**Nuances an author MUST know.**
- **Statefulness + REMOTE execution is the #1 gotcha.** Per-video state lives in process memory, so every block here returns `[STATEFUL_VIDEO_HTTP_SOFT_RESTRICTION, STILL_IMAGE_INPUT_SOFT_RESTRICTION]` from `get_restrictions()`. On stateless/multi-replica HTTP with `StepExecutionMode.REMOTE`, successive frames may hit different workers, state resets, and tracking/counting/aggregation output is meaningless. Declare these restrictions on any new stateful block.
- **Tracked input is a hard precondition.** Analytics blocks require `detections.tracker_id is not None` and raise `ValueError` otherwise (`TimeInZoneBlockV3.run`, `VelocityBlockV1.run`). Fail loud, and expect the block to sit *downstream of a tracker*.
- **fps / timestamp duality.** Time math must branch on source: `frame_number / fps` for `metadata.comes_from_video_file`, else `metadata.frame_timestamp.timestamp()` (identical branch in time_in_zone v3 and velocity). Trackers default fps to 30 with a warning when absent (`TrackerBlockBase._run_tracker`).
- **New detection data-kinds must be serialized.** Analytics blocks write metrics into `detections.data[...]` under named constants (`TIME_IN_ZONE_KEY_IN_SV_DETECTIONS`, `VELOCITY_KEY_IN_SV_DETECTIONS`, `SPEED_KEY_IN_SV_DETECTIONS` in `execution_engine/constants.py`). Each such key needs a matching branch in `core_steps/common/serializers.py` (see the `if TIME_IN_ZONE_KEY...` / `if VELOCITY_KEY...` blocks) or it will not survive JSON serialization on the sink side. Adding a metric = constant + write + serializer entry.
- **Per-video eviction / cache bounds.** Unbounded per-`video_identifier` dicts leak across long-lived streams. Follow the FIFO patterns: `InstanceCache` (deque `maxlen`) in the tracker base, and `time_in_zone` v3's `OrderedDict` zone cache with `popitem(last=False)` at `ZONE_CACHE_SIZE`.
- **Coordinate system / anchors.** Zone and line blocks work in *pixel space* of the frame and use `sv.Position` triggering anchors (e.g. `CENTER`, `BOTTOM_CENTER`) to decide membership; velocity/perspective is uncalibrated pixels unless `pixels_per_meter` is set. Zone inputs are validated for nesting depth (`ensure_zone_is_list_of_polygons` / `calculate_nesting_depth`).
- **Tracker params are init-only.** `_create_tracker` runs once per video; changing thresholds on later frames has no effect (documented in `_run_tracker`). Trackers only associate on bboxes but index back into the full `sv.Detections`, preserving masks/keypoints/custom data.
- **Dynamic outputs.** `DataAggregator` declares `OutputDefinition(name="*")` and computes real names/kinds at runtime via `get_actual_outputs()` (`{variable}_{mode}`) — the pattern to copy when output shape depends on config.

**Example blocks to pattern-match (simplest → most representative).**
- `trackers/_base.py` — read first: the canonical stateful, per-`video_identifier`, 3-output tracker contract + `InstanceCache` eviction.
- `trackers/bytetrack/v1.py` — minimal concrete tracker: manifest + `_create_tracker` only. Contrast with `trackers/botsort/v1.py`, which overrides `_tracker_update` to feed frame pixels for camera-motion compensation (the reason `_tracker_update` exists).
- `analytics/velocity/v1.py` — cleanest "annotate tracked detections with a new metric": tracker_id guard, fps/timestamp branch, writes keys into `detections.data`.
- `analytics/time_in_zone/v3.py` — the fullest analytics reference: multi-zone polygon logic, `sv.Position` anchors, per-track entry-time state, and a bounded `OrderedDict` zone cache with FIFO eviction.
- `analytics/data_aggregator/v1.py` — the *reducer* exception: interval-gated emission, empty outputs between intervals, dynamic `get_actual_outputs()`, UQL `data_operations`.
- `sampling/identify_outliers/v1.py` — non-detection state: consumes `EMBEDDING_KIND`, keeps a sliding window + vMF stats, emits `is_outlier` / `percentile` / `warming_up`; sibling `sampling/identify_changes/v1.py` mirrors the shape.
