static USAGE: &str = r#"
Generate charts/maps from CSV data using the plotly charting library.

Produces a self-contained, interactive HTML chart (the plotly.js runtime is embedded,
so charts work offline; map basemaps fetch their tiles over the network at view time
unless the `white-bg` style is used). Set the QSV_VIZ_CDN environment variable to load
plotly.js from its CDN instead, shrinking the page by ~1.9MB at the cost of needing
network access to view it. By default the embedded plotly.js is gzip-compressed
(~1.9MB vs ~4.8MB) and inflated in-browser, which requires a browser with
DecompressionStream support (Chrome/Edge 80+, Firefox 113+, or Safari 16.4+);
set QSV_VIZ_NO_COMPRESS for plain-text, uncompressed HTML that also works on
older browsers.
Titles and labels are rendered as plain text - LaTeX (e.g. `$\alpha$`) is not typeset.
With a qsv build that includes the `viz_static` feature, charts can also be exported as
static PNG/SVG/PDF/JPEG/WebP images (this requires a Chromium/Firefox browser at
runtime - a webdriver is auto-managed by plotly).

The output format is inferred from the --output file extension (.html is the default).
Interactive HTML is written to stdout when --output is not given; image formats always
require --output. Use --open to view the result in your default browser/viewer.

Progress is shown on stderr by default: a spinner with per-phase status messages (loading
statistics, inferring the data dictionary, computing correlations, rendering, etc.). It is
auto-hidden when stderr is not a terminal (e.g. piped or redirected). Set the QSV_PROGRESSBAR
environment variable to a falsy value (0/false/off) to disable it.

Chart types (subcommands):
    smart       Auto-dashboard (Data Schematic). Picks an appropriate chart per
                column from the dataset's statistics & frequency distribution
                (no --x/--y needed).
    bar         Bar chart.        --x = category column, --y = value column.
    line        Line chart.       --x = x column, --y = y column.
    scatter     Scatter plot.     --x = x column, --y = y column.
    scatter3d   3D scatter plot.  --x, --y, --z = three numeric columns.
    histogram   Distribution.     --x = numeric column to bin.
    box         Box plot.         --y = value column, optional --x = group column.
    violin      Violin plot: a box plot plus a KDE density curve revealing the
                distribution's shape (modes, shoulders). Same inputs as box
                (--y = value column, optional --x = group column).
    pie         Proportions.      --x = label column, optional --y = value column.
    funnel      Stage-by-stage drop-off. --x = stage column, optional --y = value
                column (counts stage occurrences when omitted). Stages keep the
                order they first appear in the file, so the rows define the
                pipeline; plotly labels each band with its conversion from the
                previous stage.
    heatmap     Color grid. Correlation matrix of numeric columns (default; an
                optional column subset via --cols), or a category x category pivot
                with --x/--y/--z.
    contour     2D density contour of two numeric columns (--x and --y), binned
                into a grid (--bins controls the grid resolution).
    candlestick Financial OHLC.   --x = date column, plus --ohlc-open/--high/--low/--close.
    ohlc        Financial OHLC bars (same inputs as candlestick).
    sankey      Flow diagram.     --source, --target, optional --value column.
    radar       Polar/radar chart of numeric --cols, optional --series per trace.
    treemap     Part-to-whole hierarchy as nested tiles. --cols = 2+ dimension
                columns (levels), optional --value and --agg.
    sunburst    Part-to-whole hierarchy as concentric rings (same inputs as
                treemap). Better for deeper hierarchies.
    icicle      Part-to-whole hierarchy as stacked bars per level (same inputs as
                treemap). Level-aligned; good for deep, wide hierarchies.
    splom       Scatter-plot matrix: every pair of numeric --cols plotted against
                each other in a grid, with each column's distribution on the
                diagonal. Good for spotting correlations across many numeric
                columns at once (default: all numeric columns).
    parcats     Parallel categories: ribbons showing how rows flow across the
                categorical --cols (best with 3-4). Complements sankey (which
                takes 2 columns) for higher-dimensional categorical relationships.
    map         Geographic point map (or --density heatmap) on tile basemaps.
                Pick the coordinate columns with the lat/lon options below.
    geo         Geographic point map on a projection basemap (coastlines/land/
                countries; no tiles, no token). Uses the same lat/lon options
                as `map`, plus --projection. Good for global/country-scale data.
    choropleth  Filled-region map: color whole regions (countries, US states, or
                custom GeoJSON areas) by a value. --locations names the region-code
                column, --value/--agg the measure (row counts if omitted). Defaults
                to a token-free projection basemap; --map switches to MapLibre tiles.

`qsv viz smart` builds a DATA SCHEMATIC - a single, self-contained rendering of a
dataset's schema and statistics in which every claim shown is checkable against the
data it describes. The format is defined in docs/DATA_SCHEMATIC.md and is
tool-neutral; what `viz smart` produces is a "qsv Schematic".

A schematic is the drawing form of a SCHEMA - and `viz smart` already emits a
`.schema.json` - so the name is descriptive rather than decorative. Where a data
dictionary lists fields one at a time, a schematic shows components AND HOW THEY
CONNECT: correlation, process order, hierarchy, temporal pacing, spatial pairing.
It alloys two ways of knowing a dataset. The DETERMINISTIC half (statistics,
heuristics and algorithms - reproducible, offline, no tokens) picks the panels and
scales the axes; the SEMANTIC half (an optional LLM-inferred Data Dictionary, via
the --dictionary option) supplies what the fields MEAN - labels, roles and units,
all correctable by a human Data Steward. Both halves show their work.

The schematic is a one-page grid of subplots, reusing qsv's stats and frequency
caches (the first run computes & caches stats; later runs are fast). It auto-picks
panels, so no --x/--y is needed:

  Per-column panels (flow in the grid below the overview rows, see --grid-cols):
    - continuous numeric -> box plot (quartiles from the stats cache; sample
      points overlaid by a size heuristic, see --box-points)
    - low-cardinality / boolean -> frequency bar chart
    - ID-like (near-unique) and all-empty columns are skipped

  Overview panels (each leads the Data Schematic on its own full-width row):
    - KPI overview row (leads the Data Schematic when the dataset has headline numeric
      measures): a strip of "big number" tiles, one per headline measure (summed
      for extensive quantities, averaged for intensive ones). A measure tile
      becomes a GAUGE when the dictionary supplies a validated `x-qsv.gauge_range`
      that contains the value, and gains a "vs target" DELTA when it supplies an
      `x-qsv.target` (see --dictionary). Omitted for image exports and for datasets
      with no headline measure. (Overall dataset completeness - the share of
      non-empty cells - is a quiet "Completeness:" line in the header metadata
      table, not a KPI tile.)
    - pipeline panel, when the dictionary DECLARES one (see --dictionary). Which
      columns are process stages, and in which direction, is semantics rather than
      a statistic - no column-name vocabulary settles it and no statistic does
      either - so the panel is drawn ONLY from an explicit declaration, never
      guessed. Both encodings are supported: stages held in separate measure
      columns, and stages held as values of one category column. Costs one extra
      data pass over the declared stages only.
      The declaration fixes WHICH columns and in WHAT order; the numbers decide the
      FORM. A funnel's band widths are a containment claim, so one is drawn only
      while the stage totals never grow. If any stage outruns the one before it, the
      same declaration is drawn as a BRIDGE instead: the signed difference between
      consecutive totals, each step labelled as the arithmetic difference it is
      rather than as a flow. A funnel there would render a band wider than the one
      above it, asserting the opposite of the data. The subtitle says which form was
      used and why.
      Stage order is the declared order and is never re-sorted by size. For the
      column encoding, row-wise containment (does each stage nest inside the one
      before it?) is MEASURED and disclosed in the subtitle - separately from the
      form, since rows can overrun while the totals still shrink, or nest while the
      totals grow. Totals sum over the rows complete across every declared stage,
      so they do NOT match `stats.sum`; the subtitle always discloses that
      denominator. See also the standalone `qsv viz funnel` chart type, which takes
      its stage order from the file and needs no dictionary.
    - correlation heatmap, when 2+ continuous numeric columns exist (one extra
      data pass for Pearson correlations). If the strongest pair is at least
      moderately correlated, a drill-down is added beside it: a scatter (or a 2D
      density contour for large, overplotting datasets); with 3+ numeric columns,
      a 3D scatter of the strongest triple is added too.
    - time-series line, when an auto-detected date/datetime column and a
      continuous numeric column both exist.
    - geographic map, when a latitude/longitude pair is detected:
        - HTML uses a MapLibre tile map for a local extent, or an offline
          ScatterGeo projection world-overview for continental/global data.
        - static image export uses an offline ScatterGeo fit to the data extent
          (US-spanning data uses albers-usa); tile maps and 3D panels stay
          HTML-only, as tile maps need network tiles.
        - geographic outliers (points beyond the Tukey far-out fence of
          distances from the cluster centroid) get a distinct marker and are
          excluded from the spatial extent; the map zooms to the core, with a
          dotted no-fill box marking the full extent and (in HTML) Core/Full
          extent buttons. Outliers within the core's jurisdiction don't trigger
          the extent call-out.
        - with the `geocode` feature, the core extent (4 corners + center) is
          reverse-geocoded against the local Geonames index and drawn as a
          labeled bounding box with a location summary (e.g. "New York & New
          Jersey, United States"); outliers are called out with their count and
          jurisdiction. HTML points reveal city/state/country on hover (static
          exports omit it). The first run may download the index (~13MB, cached
          in ~/.qsv-cache); offline, the map renders without the overlay.
        - extents spanning the antimeridian (>180 degrees of longitude) are
          skipped.

Examples:
  # Data Schematic for a dataset, opened in the browser
  qsv viz smart data.csv --open

  # Data Schematic, at most 6 panels in a 3-column grid, top-5 categories per bar
  qsv viz smart data.csv --max-charts 6 --grid-cols 3 --limit 5 -o dashboard.html

  # Bar chart of fruit prices, opened in the browser
  qsv viz bar fruits.csv --x Fruit --y Price --title "Fruit prices" --open

  # Aggregate (sum) sales by region into a bar chart
  qsv viz bar sales.csv --x region --y amount --agg sum -o sales.html

  # Scatter plot with a separate series (trace) per category
  qsv viz scatter data.csv --x age --y income --series gender -o scatter.html

  # Bubble scatter: marker size by population, marker color by a numeric score
  qsv viz scatter data.csv --x gdp --y life_exp --size population --color score -o bubble.html

  # Gapminder bubble animation: one bubble per region moving over time
  qsv viz scatter regions.csv --x gdp_index --y wellbeing_index --size population_m --series region --slider month_date -o gapminder.html

  # Histogram of a numeric column with 30 bins
  qsv viz histogram data.csv --x value --bins 30 -o hist.html

  # Box plot of a value column grouped by a category, exported to PNG (needs viz_static)
  qsv viz box data.csv --y measurement --x group -o box.png

  # Box plot with every sample point overlaid (jittered) instead of just the outliers
  qsv viz box data.csv --y measurement --box-points all -o box.html

  # Violin plot (KDE density curve + inner box) of a value column grouped by a category
  qsv viz violin data.csv --y measurement --x group -o violin.html

  # Pie chart of category proportions (counts), as a donut
  qsv viz pie data.csv --x category --donut -o pie.html

  # Funnel of a pipeline whose stages are ROWS (stage column + amount column)
  qsv viz funnel pipeline.csv --x stage --y amount -o funnel.html

  # Correlation heatmap over all numeric columns
  qsv viz heatmap data.csv -o corr.html

  # Heatmap pivot: average value per (region x product)
  qsv viz heatmap sales.csv --x region --y product --z amount -o pivot.html

  # Candlestick chart from a date column and OHLC price columns
  qsv viz candlestick prices.csv --x date --ohlc-open open --high high --low low --close close -o ohlc.html

  # Sankey flow diagram of source -> target weighted by value
  qsv viz sankey flows.csv --source from --target to --value weight -o sankey.html

  # Radar chart comparing numeric metrics, one trace per team
  qsv viz radar teams.csv --cols speed,power,range,accuracy --series team -o radar.html

  # Point map of earthquakes, marker color by magnitude and size by depth
  qsv viz map quakes.csv --lat lat --lon lon --color magnitude --size depth -o map.html

  # Density heatmap of the same points, on a light Carto basemap
  qsv viz map quakes.csv --lat lat --lon lon --density --style carto-positron -o heat.html

  # 3D scatter of three numeric columns, colored by a fourth
  qsv viz scatter3d data.csv --x length --y width --z height --color weight -o scatter3d.html

  # 2D density contour of two numeric columns with a 40x40 grid
  qsv viz contour data.csv --x height --y weight --bins 40 -o contour.html

  # Projection map of earthquakes (token-free), marker color by magnitude
  qsv viz geo quakes.csv --lat lat --lon lon --color magnitude --projection natural-earth -o geo.html

  # Treemap of part-to-whole sales by region then category, sized by amount
  qsv viz treemap sales.csv --cols region,category --value amount --agg sum -o treemap.html

  # Sunburst of a deep 3-level web-traffic hierarchy, sized by row count
  qsv viz sunburst web.csv --cols source,campaign,landing_page -o sunburst.html

  # Choropleth coloring countries (ISO-3 codes) by a summed measure
  qsv viz choropleth gdp.csv --locations iso3 --value gdp --agg sum -o choropleth.html

  # US-state choropleth of row counts per state (2-letter state codes)
  qsv viz choropleth orders.csv --locations state --location-mode usa-states -o states.html

  # Custom GeoJSON regions on a MapLibre basemap, matched by a feature id
  qsv viz choropleth counties.csv --locations fips --value pop --map --geojson counties.json --feature-id-key id -o counties.html

  # Reverse-geocode lat/lon points to ISO-3 codes, then count per country (needs geocode feature)
  qsv viz choropleth stops.csv --geocode --lat lat --lon lon -o by_country.html

  # Point-in-polygon: bin lat/lon points into custom GeoJSON regions by count (no geocode)
  qsv viz choropleth quakes.csv --lat lat --lon lon --geojson prefectures.geojson --feature-id-key properties.id -o by_pref.html

For more examples, see https://github.com/dathere/qsv/blob/master/tests/test_viz.rs.
See also https://github.com/dathere/qsv/wiki/Visualization

Usage:
    qsv viz smart       [options] <input>
    qsv viz bar         [options] <input>
    qsv viz line        [options] <input>
    qsv viz scatter     [options] <input>
    qsv viz scatter3d   [options] <input>
    qsv viz histogram   [options] <input>
    qsv viz box         [options] <input>
    qsv viz violin      [options] <input>
    qsv viz pie         [options] <input>
    qsv viz funnel      [options] <input>
    qsv viz heatmap     [options] <input>
    qsv viz contour     [options] <input>
    qsv viz candlestick [options] <input>
    qsv viz ohlc        [options] <input>
    qsv viz sankey      [options] <input>
    qsv viz radar       [options] <input>
    qsv viz map         [options] <input>
    qsv viz geo         [options] <input>
    qsv viz choropleth  [options] <input>
    qsv viz treemap     [options] <input>
    qsv viz sunburst    [options] <input>
    qsv viz icicle      [options] <input>
    qsv viz splom       [options] <input>
    qsv viz parcats     [options] <input>
    qsv viz --help

viz options:
    -x, --x <col>          Column for the x-axis / category / bin / group.
    -y, --y <col>          Column for the y-axis / value.
    -z, --z <col>          The z column: a heatmap pivot value (with --x and --y), or
                           the third numeric axis for scatter3d.
    --cols <cols>          Columns to use. For heatmap: numeric columns for the
                           correlation matrix (default: all numeric). For radar:
                           the numeric axes to plot. For treemap/sunburst/icicle: the
                           categorical dimensions that form the hierarchy levels,
                           outermost first (e.g. region,category,subcategory).
                           For splom: the numeric columns to cross-plot (default:
                           all numeric). For parcats: the categorical dimensions
                           to flow across (best with 3-4).
    --series <col>         Column to split into multiple series (one trace per
                           distinct value). Applies to bar, line, scatter, scatter3d,
                           radar, map and geo.
    --color <col>          For scatter/scatter3d/map/geo: a numeric column to encode as
                           marker color (a continuous colorscale with a colorbar). For
                           categorical coloring, use the --series option instead. Cannot
                           be combined with --series. In map density mode, this column is
                           the heatmap weight.
    --size <col>           For scatter/scatter3d/map/geo: a numeric column to encode as
                           marker size, producing a bubble chart (values are rescaled to
                           a readable pixel range). Cannot be combined with --series,
                           EXCEPT on an animated scatter (see the --slider option), where
                           the --series column names the entity and this column is the
                           third data variable. In map density mode, this column is the
                           heatmap weight.
    --donut                Render a pie chart as a donut (with a center hole).
    --ohlc-open <col>      Open-price column for candlestick/ohlc charts.
    --high <col>           High-price column for candlestick/ohlc charts.
    --low <col>            Low-price column for candlestick/ohlc charts.
    --close <col>          Close-price column for candlestick/ohlc charts.
    --source <col>         Source node column for a sankey diagram.
    --target <col>         Target node column for a sankey diagram.
    --value <col>          Flow value column for a sankey diagram. When omitted,
                           each row counts as a flow of 1. For treemap/sunburst/icicle:
                           a numeric measure summed per sector (when omitted, each
                           row counts as 1).
    --sankey-value-order   Order sankey nodes by total flow (largest at the top of each
                           column). By DEFAULT nodes use plotly's crossing-minimizing
                           "snap" order, which packs the diagram more compactly; this
                           opts into the flow-ranked initial ordering instead. Either way
                           the rendered chart carries an on-screen "node order" button to
                           flip between the two layouts, and link ribbons are always
                           colored by their source node. Applies to the `sankey` chart
                           and the `smart` Data Schematic flow panel.
    --bins <n>             Number of bins. For histogram: bins along the x-axis
                           (default: auto). For contour: the per-axis resolution of
                           the density grid (default: 20).
    --agg <fn>             For bar/line, aggregate the y values when the x value
                           repeats. One of: sum, mean, count, min, max.
                           For treemap/sunburst/icicle, only additive aggregations
                           apply: count (default) or sum (requires --value).
                           For a bubble animation (see --slider), collapses each
                           entity x frame cell to a single bubble; defaults to mean
                           (the cell centroid). count is rejected there, since a row
                           count is not a position.
    --box-points <mode>    Which sample points to draw alongside a box. Reading the
                           raw values lets plotly render true Tukey whiskers (1.5*IQR)
                           with the points beyond the fences as outliers. One of:
                           outliers (only the outliers), all (every point, jittered),
                           suspected (mark suspected outliers), none (no points, but
                           still real Tukey whiskers). For `viz box` the default is
                           outliers. For `viz smart` this flag OVERRIDES the default
                           size-based heuristic, which overlays all points on a column
                           with few non-null values (<=1,000) — unless the Data Schematic has
                           more than 8 panels, where all-points cells turn to noise, so
                           the overlay drops to outliers-only — and only the outliers
                           for a medium one (<=10,000 values). Above that, a
                           column that HAS outliers shows
                           them as points on a precomputed quartile box (a single pass
                           collects only the out-of-fence values, capped); a column with
                           no outliers stays a fast cache-only quartile summary with no
                           data re-scan. An explicit mode is applied to every box panel
                           (one batched pass to read the values), except `none`, which
                           keeps the cache-only box for BOX panels. A smart violin
                           panel still renders under `none`, just with no sample
                           points — its KDE needs the values anyway; combine with the
                           violin option's `off` mode for the cache-only escape hatch.
                           The same modes also pick the sample points drawn beside a
                           violin, for `viz violin` and for the violin panels of
                           `viz smart` — though a smart violin panel skips the
                           size-based `all` upgrade (its KDE silhouette already shows
                           the distribution) and overlays only the outliers unless an
                           explicit mode is given.

map options:
    --lat <col>            Latitude column for a map (decimal degrees, -90 to 90).
    --lon <col>            Longitude column for a map (decimal degrees, -180 to 180).
    --text <col>           Column whose value labels each point on hover.
    --density              Render a density heatmap (DensityMap) instead of points.
                           Weighted by the --color or --size column when given, else by
                           a uniform weight. Cannot be combined with --series.
    --style <name>         MapLibre basemap style (all render without an access token):
                           open-street-map (the default), carto-positron,
                           carto-darkmatter, carto-voyager, white-bg, basic, streets,
                           outdoors, light, dark, satellite, satellite-streets. The three
                           carto styles also have label-free "-nolabels" variants (e.g.
                           carto-positron-nolabels) that keep basemap place names from
                           competing with a dense data overlay.
                           [default: open-street-map]

geo options:
    --projection <name>    Map projection for `viz geo`. One of: natural-earth (the
                           default), mercator, orthographic, equirectangular,
                           albers-usa, robinson, winkel-tripel, mollweide, hammer,
                           azimuthal-equal-area. `viz geo` also reuses the lat, lon,
                           text, color, size and series options from `map`.
                           [default: natural-earth]

choropleth options:
    --locations <col>      Column holding the region key for each row (an ISO-3 country
                           code, a 2-letter US state code, a country name, or a GeoJSON
                           feature id, per --location-mode). With --geocode, this instead
                           names a place-name column to forward-geocode into region codes.
    --location-mode <m>    How --locations values are matched to regions. One of: iso3
                           (the default, ISO-3166-1 alpha-3 country codes), usa-states
                           (2-letter US state codes), country-names (full country names),
                           geojson-id (match a --geojson feature id). [default: iso3]
    --color-scale <name>   Colorscale for the region fill. One of: viridis (the default),
                           cividis, greys, greens, blues, reds, ylgnbu, ylorrd, bluered,
                           rdbu, portland, electric, jet, hot, blackbody, earth, picnic,
                           rainbow. [default: viridis]
    --map                  Render on a token-free MapLibre tile basemap (a ChoroplethMap)
                           instead of the default projection basemap. Requires --geojson;
                           feature-id-key defaults to id. Reuses --style for the basemap.
    --geojson <src>        Custom region polygons as a local file path or an http(s) URL
                           to a GeoJSON FeatureCollection. May also be a shortcut name
                           defined in the QSV_GEOJSON_SHORTCUTS env var (a JSON map of
                           name to {path, id}); the shortcut's id sets --feature-id-key
                           when you don't pass one. The source is validated up front.
                           Required for --map, and for the geojson-id location mode. Also
                           enables point-in-polygon
                           binning: with --lat/--lon (and without --geocode), each row's
                           point is binned into the region whose polygon contains it
                           (exact, no geocoding) and colored by --value/--agg or counts.
                           In `viz smart`, it also overlays the region boundaries on the
                           map, labelling each with its --feature-name-key value (falling
                           back to its id) — as on-map text on the projection/static map,
                           or as a centroid hover marker on the interactive tile map (which
                           culls on-map text). Labels are omitted above 60 regions.
    --feature-id-key <k>   Property path in each GeoJSON feature whose value matches an
                           entry in the locations column, or that labels each binned
                           region (e.g. id, properties.fips). [default: id]
    --feature-name-key <k>  GeoJSON property path whose value is shown as the
                           human-readable region label in choropleth hover (e.g.
                           properties.name). When omitted, common name keys are
                           auto-detected; falls back to the feature id when absent.
    --geocode              Derive the region codes by reusing qsv's geocode engine
                           (needs a build with the geocode feature). Either reverse-geocode
                           the lat/lon points, or forward-geocode the locations name
                           column. Only valid with location modes iso3 or usa-states.
                           `viz choropleth` also reuses --value, --agg, --style and the
                           lat/lon options.
    --no-snap              For point-in-polygon binning (lat/lon points binned into a
                           custom GeoJSON without geocoding): do not snap at all — drop
                           every point that falls outside every region. By default an
                           outside point instead snaps to its nearest region when within
                           the snap-distance limit (see --snap-max-dist). Applies to both
                           the `viz choropleth` command and the `viz smart` GeoJSON
                           choropleth panel. A stderr note reports coverage either way;
                           each snapping region's hover tallies the points it absorbed
                           from outside, and snapped or dropped points are reported in a
                           note beneath the map (or in the smart panel's title).
    --snap-max-dist <km>   For point-in-polygon binning: the farthest (in km) an outside
                           point may snap to a region's boundary; points with no region
                           within this distance are dropped. Distance is an equirectangular
                           km approximation. When omitted, the cap is auto-derived from
                           the GeoJSON's median region size (10% of the median bbox
                           diagonal) and the lat/lon coordinate precision (coarse,
                           few-decimal coordinates raise the cap to cover their
                           quantization error), clamped to 0.1-100 km; a fixed 10 km is
                           used only when neither signal is available. Applies to both the
                           `viz choropleth` command and the `viz smart` GeoJSON choropleth
                           panel; the coverage notes state the cap applied and how it was
                           derived. Pass a large value for effectively unbounded snapping;
                           cannot be combined with --no-snap.

smart options:
    --max-charts <n>       Maximum number of panels in the Data Schematic. 0 (the default)
                           means auto: draw every eligible column (up to 64), for both
                           HTML and static image export (png/svg/pdf/...). Up to 8
                           cartesian panels render as one typed subplot grid; beyond 8,
                           HTML switches to an inline-div grid of independent plots, and
                           static image export uses domain-positioned axes to fit them in
                           one image. Set a positive <n> to cap the panel count instead.
                           Eligible columns beyond the cap are reported but not drawn.
                           [default: 0]
    --grid-cols <n>        Number of columns in the Data Schematic grid for the per-column
                           distribution panels. Overview panels (map/geo, correlation,
                           time-series) always span the full width. [default: 2]
    --preview-threshold <n>  Row threshold for the interactive data viewer drawer of
                           HTML dashboards. Next to the row count in the Data Schematic
                           metadata, an "(Explore)" link opens the underlying table
                           in a bottom drawer (with global, per-column and builder
                           search) when the dataset has at most <n> rows and ALL rows
                           are embedded in the page; above <n>, only the first <n>
                           rows are embedded and the link reads "(Preview)". Note that
                           embedded rows grow the HTML file (and browser memory) in
                           proportion to rows x columns. Set to 0 to disable the data
                           viewer entirely (no link, no embedded data). Only affects
                           `smart` HTML output. [default: 50000]
    --heatmap-density <n>  For the `viz smart` map panel: at or above <n> mappable
                           points, draw the core as a density heatmap (DensityMap)
                           instead of markers. The heatmap keeps per-point hover
                           (coordinates, plus the point's label). Defaults to 0
                           (disabled): dense point maps instead open as individual
                           points with an in-map "Clusters/Points" toggle (collapse the
                           dense areas into interactive count bubbles on demand)
                           rather than aggregating into a static heat surface.
                           Set a positive <n> to opt back into the heatmap above
                           that point count. [default: 0]
    --cluster <mode>       For the `viz smart` map panel: whether to offer an
                           in-map "Clusters/Points" toggle that collapses dense points
                           into native MapLibre count bubbles (which expand back
                           into individual, hoverable points on zoom-in). Hovering
                           a bubble shows its point count; clicking it zooms to
                           where that cluster breaks apart. The map always OPENS
                           as individual points; the toggle switches clustering on. One of: auto, on, off. "auto" (the
                           default) offers the toggle for a plain-marker core (no
                           density heatmap, no bubble-size measure) once it
                           reaches 1,000 points. "on" offers it whenever the core
                           draws as markers, regardless of point count or a
                           bubble-size measure. "off" omits the toggle, so points
                           are always drawn plainly. Only affects `smart`.
                           [default: auto]
    --photos               For the `viz smart` map panel: when a column holds image URLs
                           (detected from the stats cache - an http(s) value ending in a
                           common image extension), embed each point's image URL so that
                           dwelling on a map point for 2 seconds opens that row's photo in a
                           small preview card beside the pointer (arrow keys page through a
                           row with several photos; Escape dismisses the card). Points that
                           have photos say so in their hover label. When the image host allows
                           cross-origin reads, fetched images are cached in the browser's
                           IndexedDB, so re-dwelling a point or reloading the page serves them
                           locally; otherwise the preview falls back to a normal image load
                           served by the ordinary browser cache.
                           OFF by default, and deliberately so: images load from whatever
                           third-party host the DATA names, so enabling this makes everyone
                           who opens the Data Schematic request those URLs directly (revealing
                           their IP to that host). Nothing is fetched until a viewer dwells.
                           Embedding every mapped point's URLs also grows the HTML. Applies to
                           HTML output only - static image exports (PNG/SVG/PDF) are
                           unaffected - and only on the tile-basemap map panel. Only affects
                           `smart`.
    --limit <n>            Top-N categories per frequency bar chart. [default: 10]
    --no-nulls             Omit the "(NULL)" bar (empty cells) from frequency bar charts.
                           By default `viz smart` shows a "(NULL)" bar, like `qsv frequency`.
    --no-other             Omit the "Other (N)" aggregate bar from frequency bar charts. It
                           collects the categories beyond --limit (N = how many distinct
                           categories were rolled up) and is shown by default. When just
                           ONE category is left over, it is charted under its own name
                           instead of as an opaque "Other (1)" bar.
    --smarter              Before building the Data Schematic, run `qsv moarstats --advanced`
                           to enrich the stats cache with distribution-shape statistics
                           (bimodality, entropy, skewness, outlier share, Gini). This unlocks
                           histograms for bimodal columns, frequency bars for concentrated
                           high-cardinality columns, skew/outlier hints on box panels, and
                           Lorenz curves for the most unequal additive measures (high Gini).
                           Costs one extra pass over the data and writes <stem>.stats.csv,
                           its sidecars, and an .idx index (like running `qsv moarstats`
                           manually). On geocode-enabled builds it also enriches map point
                           hovers with the US FIPS code and annotates the spatial-extent
                           summary with the country's continent; the county is always shown
                           in map hovers, with or without --smarter. Only affects `smart`.
                           Applied only with default
                           parsing; inputs using --no-headers or a custom --delimiter
                           fall back to the standard Data Schematic.
    --hierarchy-style <k>  For `smart`, the chart used for the categorical part-to-whole
                           hierarchy panel (built when 2+ low-cardinality dimensions exist).
                           One of: auto (default), treemap, sunburst, icicle. auto follows
                           best practice — a treemap for a shallow 2-level hierarchy (accurate
                           size comparison) and a sunburst for a deep 3-level one (parent
                           child structure); icicle is an opt-in level-aligned alternative.
                           Only affects `smart`.
    --dictionary <src>     Use a describegpt Data Dictionary to guide panel selection from
                           each field's semantic role/concept (falling back to its content type)
                           instead of relying on column statistics alone: dimensions and numeric
                           codes (ward, census_tract, zone) become bars, measures get
                           box/correlation/trend panels, date/datetime columns feed the
                           time-series panel (not noisy frequency bars), identifiers / PII /
                           free-text are skipped, and lat/lon feed the map. Field labels are shown
                           as panel subtitles beneath the field-name titles. Columns the dictionary
                           cannot classify still use the statistical heuristic. <src> is one of:
                           "infer" to run describegpt on the input now (with description,
                           infer-content-type, two-pass and jsonschema output; requires an LLM
                           configured) and use its output; or a path to an existing describegpt
                           dictionary file
                           (jsonschema or json). With "infer", the generated dictionary is saved
                           beside the input as <stem>.schema.json so you can fine-tune it; if that
                           file already exists, it is reused as-is (skipping the LLM) - edit it to
                           fine-tune, or delete it to force a fresh re-infer.
                           Set QSV_VIZ_DICT_FRESH=1 to ignore an existing sidecar and bypass
                           describegpt's completion cache, forcing a genuinely fresh inference
                           that overwrites the sidecar on success.
                           Generation/read failures soft-fall back to the stats-only Data Schematic.
                           The dictionary also drives the KPI overview row via two optional
                           per-field hints in a property's "x-qsv" object (edit them in the saved
                           schema to fine-tune). A "gauge_range" of [min, max] on a continuous
                           numeric measure renders its KPI tile as a GAUGE on that canonical scale
                           (e.g. [0,1] for a ratio, [0,100] for a percent); qsv keeps it only when
                           the observed data lies within the range, so a mis-scaled range can't draw
                           a misleading dial, and "infer" emits it for canonical-scale measures.
                           A "target" number on a measure renders a "vs target" DELTA against that
                           goal (value minus target) - a GOAL you supply, never a fabricated
                           prior-period baseline, so "infer" never emits it; hand-author it.
                           The dictionary is also the ONLY source of the pipeline panel (drawn as
                           a funnel while the stage TOTALS never increase, otherwise as a bridge;
                           row-wise containment is measured & disclosed in the subtitle, but does
                           not decide the form), declared in
                           the dataset-level "x-qsv" object as a "relationships" entry
                           with "kind": "pipeline". Two encodings, both hand-editable:
                             stages as COLUMNS - "members" lists the stage columns in process
                             order, WIDEST/UPSTREAM FIRST (note this is the opposite direction
                             from a "kind":"ordered" group, which ascends), e.g.
                               {"kind":"pipeline",
                                "members":["planned_amt","committed_amt","spent_amt"]}
                             stages as ROW VALUES - "stage_column" names the category column,
                             "stages" lists its values in process order, and an optional
                             "value_column" names the measure to sum per stage (omit it to count
                             rows), e.g.
                               {"kind":"pipeline","members":["stage","revenue"],
                                "stage_column":"stage",
                                "stages":["Impression","Click","Lead","Conversion"],
                                "value_column":"revenue"}
                           Declared order is authoritative and is never re-sorted by size, so a
                           stage that outruns its predecessor stays visible instead of being
                           quietly reordered away - it switches the panel to a bridge rather than
                           drawing a funnel that widens. A declaration naming a missing column, or a
                           stage that is an average/rate rather than a summable amount, is skipped
                           with a note rather than erroring.
                           Only affects `smart`.
    --dictionary-context <file>  Path to a file with extra context about the dataset
                           (a glossary, README, data dictionary, PDF, etc.) forwarded to
                           describegpt as --context-file when `--dictionary infer` generates the
                           dictionary. Better context yields better role/concept/label/grain
                           tags, hence a better Data Schematic. Ignored unless `--dictionary infer`
                           is used (it does not apply when reading an existing dictionary file).
                           Only affects `smart`.
    --dict-info            When a usable Data Dictionary is available (per --dictionary), add
                           a "Data Dictionary" link beneath the Data Schematic title and an info
                           icon on each panel title: hovering shows that column's dictionary
                           description; clicking opens a human-friendly rendering of the
                           dictionary in a side drawer NEXT TO the plots (embedded in the
                           Data Schematic file - no extra file is written), scrolled to and
                           highlighting that column's entry. The drawer is open by default
                           on load and can be dismissed with its close button or Esc. The
                           dictionary page carries a
                           role-tinted table of contents and per-column "View chart" links
                           back to the panels, plus a row of download buttons: the
                           dictionary itself as JSON Schema, the frequency counts the
                           Data Schematic actually charted, and every generated sidecar this run
                           read (the stats cache and its metadata, the frequency cache when
                           it was reused, and the bivariate stats CSV when freshly written).
                           A sidecar qsv wrote but viz never read - the human-readable
                           <stem>.stats.csv - is NOT offered, since nothing can show it
                           describes the same computation the Data Schematic used.
                           Every file is BUNDLED into the HTML, so anyone you send the
                           Data Schematic to can download them with no access to your machine;
                           absolute local paths are stripped from the embedded metadata so
                           sharing a Data Schematic doesn't disclose your directory layout.
                           Sidecars over 4 MB are skipped with a note.
                           The drawer's popout button opens the same
                           document in its own browser tab instead (needs a browser that
                           allows user-initiated pop-ups). HTML output only; ignored with a
                           note when no dictionary is available or when exporting an image.
                           Only affects `smart`.
    --dataset-pid <value>  A persistent identifier (PID) for the dataset - typically a full
                           URL such as a DOI (https://doi.org/10.1234/abc) or other citable
                           link. When set, a "PID" row is added to the metadata table at the
                           top of the Data Schematic. http(s) and mailto values become a clickable
                           link (opened in a new tab); any other scheme is shown as plain text
                           rather than linked. HTML output only. Only affects `smart`.
    --bivariate            Add two pairwise-association overview panels driven by
                           `qsv moarstats --bivariate`: a normalized mutual information (NMI)
                           heatmap over every column pair (works for numeric AND categorical
                           columns, unlike the Pearson-only correlation heatmap), plus a ranked
                           "top relationships" bar of the strongest pairs when there are more than
                           8 chartable columns. The ranked bar (not the heatmap, which still shows
                           every pair) requires a pair's co-occurring row count to be at least 10%
                           of the best-supported pair's, so a technically-perfect NMI from two
                           sparsely-populated columns that only ever co-occur in a narrow row
                           slice can't crowd out a more broadly meaningful association; each bar's
                           hover shows its co-occurring row count. A numeric pair whose Spearman
                           rank correlation diverges sharply from its Pearson correlation is
                           flagged "nonlinear" in the hover. Identifier / PII / free-text columns
                           (per --dictionary) and map lat/lon columns are excluded. This
                           automatically turns on --smarter, and when --dictionary isn't already
                           set, also turns on --dictionary infer (so the PII/identifier exclusion
                           has semantic signal to work with). Capped at 50 columns (1,225 pairs);
                           wider datasets skip these panels with a warning (run
                           `qsv moarstats --bivariate` directly for the full pairwise output).
                           Only affects `smart`.
    --log-scale <mode>     Use a logarithmic y-axis for panels with a high dynamic
                           range: frequency bar panels whose tallest bar dwarfs the
                           rest (e.g. a large "(NULL)" or "Other (N)" bucket), so the
                           small categories stay visible; and box panels of an
                           all-positive column whose observed max/min ratio is huge
                           (a box otherwise squashed against its extremes).
                           One of: auto, on, off. "auto" (the default) switches a panel
                           to a log y-axis only when its dynamic range is high; "on"
                           forces a log y-axis on every frequency panel and every
                           all-positive box panel; "off" keeps the linear axes.
                           Only affects `smart`. [default: auto]
    --violin <mode>        Draw distribution panels as violins (a box plot
                           wrapped in a KDE density silhouette — a strict
                           superset of a plain box) instead of boxes.
                           One of: auto, on, off. "auto" (the default) draws a
                           violin for every continuous column EXCEPT those on
                           a log value axis (the KDE is estimated in linear
                           space, which a log axis would geometrically distort)
                           and those with fewer than 20 distinct values or
                           fewer than 50 non-null observations (too few for a
                           meaningful KDE — the curve would be smoothing
                           artifact) — all keep an honest box; "on" also
                           violins those; "off" never draws violins. Violin
                           KDEs are clamped to each column's observed range
                           (Plotly spanmode "hard") so a bounded or discrete
                           measure can't show density past a natural limit.
                           A column with at most 10,000 non-null values gets an
                           exact violin (with outlier points, like a raw box);
                           up to 30,000 values (one-fifth of the smart point
                           budget, so it scales with QSV_VIZ_MAX_POINTS) still
                           draws an exact violin but drops the point overlay;
                           a larger column draws its silhouette and inner box
                           from a deterministic sample of at most that many
                           evenly strided values, carries no point overlay (a
                           sample misses the true extremes), and is titled
                           "(sampled)".
                           An explicit all, outliers or suspected mode given
                           via the box-points option instead collects every
                           value for every panel, violins included; that
                           option's `none` mode just hides a violin's points.
                           Only affects `smart`.
                           [default: auto]

    --title <s>            Chart title.
    --x-title <s>          X-axis title. (defaults to the x column name)
    --y-title <s>          Y-axis title. (defaults to the y column name)
    --y-range <min:max>    Fix the y-axis to an explicit min:max range (two colon-
                           separated numbers, e.g. -10:55) instead of autoscaling;
                           points outside are clipped from view. Handy when one
                           extreme outlier squashes the rest. Applies to cartesian
                           charts (bar/line/scatter/histogram/box/violin/heatmap/
                           contour). Pass a negative min with `=`: --y-range=-10:55.
    --rangeslider          Add a draggable range-slider (a navigator strip) under the
                           x-axis for panning and zooming. Best for an ordered x-axis
                           (time series, line, candlestick/ohlc). Applies to cartesian
                           charts (bar/line/scatter/histogram/box/violin/candlestick/
                           ohlc); off by default.
    --slider <arg>         Animate the chart over a column: each distinct value of the
                           given column becomes an animation frame, with a Play/Pause
                           button and a slider to scrub through the frames (Gapminder-
                           style). Supported for bar/line/scatter and geo, and may be
                           split into animated traces with the --series option. (Not
                           supported for `viz map` — the MapLibre basemap can't animate;
                           use `viz geo` for an animated point map.)
                           On `viz scatter`, combining a slider with --series AND --size
                           builds a Gapminder BUBBLE ANIMATION: --series is the entity
                           (one colored bubble each), --x/--y the measure pair and --size
                           the third data variable (bubble area). Each entity x frame cell
                           collapses to one bubble via --agg (mean by default); bubble
                           sizes are scaled once across all frames so they stay comparable.
                           A slider column that parses as dates is auto-bucketed to a
                           readable number of frames; any other column frames on its
                           distinct values. For `smart`, this
                           controls whether one animated relationship-over-time panel is
                           added: the numeric pair whose per-time-bucket centroid path
                           bends the most (a trailing-window scatter colored by time
                           bucket), or an animated geo map of global-extent dated points,
                           or a Gapminder entity-bubble chart (at most one is shown; when
                           several qualify the bubble wins, then the geo map, then the
                           pair). auto (default) adds one only on a strong signal (a
                           date column and enough time buckets); on lowers the bar (as few
                           as two buckets) but still requires a genuinely drifting or
                           animatable relationship; off never does. Axis ranges are pinned
                           across frames so nothing jumps.
    --slider-speed <ms>    Milliseconds each animation frame is shown while playing.
                           [default: 800]
    --slider-cumulative    Accumulate rows across frames: frame N includes every row
                           whose --slider value is at or below the Nth value, so the
                           animation builds up cumulatively instead of replacing each
                           step. Applies to bar/line/scatter and geo.
                           On a bubble animation there is only ever one bubble per entity
                           per frame, so nothing piles up on screen; instead each bubble
                           shows a RUNNING aggregate over all frames up to that point, and
                           so drifts less and less as the animation plays.
    --annotation <s>       Caption note drawn at the bottom of the plot (e.g. to note
                           a clipped outlier). Cartesian charts only.
    --theme <name>         Plotly theme that drives the chart's overall look
                           (background, fonts, axis styling). One of: default,
                           plotly_white, plotly_dark, seaborn, seaborn_whitegrid,
                           seaborn_dark, matplotlib, plotnine (case-insensitive;
                           hyphens accepted). When omitted, qsv's built-in look
                           is used. Applies to all chart types, including `smart`.
    --language <lang>      Render the Data Schematic UI in this language. Accepts a
                           BCP-47/ISO 639 code (es, spa) or an English language
                           name (Spanish). Overrides the language detected in a
                           data dictionary, and is also forwarded to describegpt
                           when inferring one. Curated: en, es, fr, de, it,
                           pt-BR, ja, zh-CN. [default: auto]
    --width <n>            Image width in pixels for static export. Default 1000;
                           for `smart`, auto-scaled to the grid's column count.
    --height <n>           Image height in pixels for static export. Default 600;
                           for `smart`, auto-scaled to the number of panel rows.
    --scale <f>            Image scale factor (static export). [default: 1.0]
    --open                 Open the generated chart in the default browser/viewer.

Common options:
    -h, --help             Display this message
    -o, --output <file>    Write output to <file> instead of stdout. The chart
                           format is inferred from the extension: .html (default),
                           .png, .svg, .pdf, .jpeg, .webp.
    -d, --delimiter <arg>  The field delimiter for reading CSV data.
                           Must be a single character. (default: ,)
    -n, --no-headers       When set, the first row will not be interpreted
                           as headers. Columns can then only be selected by index.
"#;

use std::{
    collections::{BTreeMap, HashMap, HashSet},
    io::{IsTerminal, Write},
    time::{Duration, Instant},
};

use indicatif::{HumanCount, HumanDuration, ProgressBar, ProgressDrawTarget, ProgressStyle};
// the sankey node-order toggle, the map cluster/basemap toggles and the slider Play/Pause menu
// are all viz-general, so these are imported unconditionally; UpdateMenuDirection lays out the
// horizontal Play/Pause button row (and the geocode-gated extent menu).
use plotly::layout::update_menu::{
    Button, ButtonBuilder, ButtonMethod, UpdateMenu, UpdateMenuDirection, UpdateMenuType,
};
use plotly::{
    Bar, BoxPlot, Candlestick, Choropleth, ChoroplethMap, Configuration, Contour, DensityMap,
    Funnel, HeatMap, Histogram, Icicle, Indicator, Ohlc, Parcats, Pie, Plot, Sankey, Scatter,
    Scatter3D, ScatterGeo, ScatterMap, ScatterPolar, Splom, Sunburst, Trace, Traces, Treemap,
    Violin, Waterfall,
    box_plot::{BoxPoints, QuartileMethod},
    choropleth::{LocationMode, Marker as ChoroplethMarker},
    color::NamedColor,
    common::{
        Anchor, ColorBar, ColorScale, ColorScalePalette, Domain, ErrorData, ErrorType,
        ExponentFormat, Fill, Font, HoverInfo, Line, Marker, Mode, Orientation, Pattern,
        PatternShape, TextPosition, TickMode, Title,
    },
    funnel::Connector as FunnelConnector,
    indicator::{Delta, Gauge, GaugeAxis, Mode as IndicatorMode, Number},
    layout::{
        Animation, AnimationMode, AnimationOptions, Annotation, Axis, AxisType, CategoryOrder,
        Center, Frame, FrameSettings, GeoFitBounds, GeoResolution, HoverMode, Layout, LayoutGeo,
        LayoutMap, LayoutScene, MapBounds, MapStyle, Margin, ModeBar, Projection, ProjectionType,
        RangeSlider, Slider, SliderCurrentValue, SliderStepBuilder, TransitionSettings,
        themes::BuiltinTheme,
    },
    parcats::{ParcatsArrangement, ParcatsDimension, ParcatsLine},
    sankey::{Arrangement, Link, Node},
    splom::SplomDimension,
    sunburst::InsideTextOrientation,
    traces::{
        icicle::BranchValues as IcicleBranchValues,
        scatter_geo::Selection as GeoSelection,
        scatter_map::{Cluster, Selection as MapSelection},
    },
    treemap::{BranchValues, Marker as TreemapMarker, Pad},
    violin::{MeanLine, SpanMode, ViolinBox, ViolinPoints},
    waterfall::{Marker as WaterfallMarker, Measure, MeasureStyle},
};
use rayon::prelude::*;
use rust_i18n::t;
use serde::Deserialize;

use crate::{
    CliResult,
    cmd::viz_i18n,
    config::{Config, Delimiter},
    select::SelectColumns,
    util,
};

/// Plotly's `Layout` exposes typed axis fields only up to `x_axis8`/`y_axis8`, which caps a
/// single-`Plot` typed subplot grid at 8 panels. HTML dashboards that need more panels are
/// rendered as an inline-div grid of independent plots instead (see `render_smart_inline`).
const MAX_SUBPLOTS: usize = 8;

/// Hard ceiling on panels for the inline-div HTML Data Schematic (used when `--max-charts` exceeds
/// `MAX_SUBPLOTS` and the output is HTML). Bounds the page size for very wide datasets.
const MAX_PANELS_INLINE: usize = 64;

/// A column whose cardinality is at or below this is treated as categorical (frequency
/// bar) rather than continuous (box plot).
const CATEGORICAL_MAX_CARDINALITY: u64 = 30;

/// A categorical column whose single most-common category (a value, or empty/null) covers more than
/// this fraction of rows is treated as near-constant: too degenerate to be a meaningful hierarchy
/// or parallel-categories dimension (it renders as one dominant ribbon/block that conveys nothing),
/// yet its low cardinality would otherwise make it a preferred (coarsest) level. 0.90 keeps
/// genuinely spread columns — a 63%-dominant facility type, a 50/50 channel split — while excluding
/// near-empty log columns (a 311 log's `Taxi Company Borough`, blank on 99.9% of rows) and lopsided
/// flags (a 95%-"Closed" status).
const CATEGORICAL_DIM_MAX_DOMINANCE: f64 = 0.90;

/// A discrete integer scale with at most this many distinct levels — a 1..N ordinal rating
/// (satisfaction 1-5, 1-7 Likert, 1-10 / 0-10 NPS) — is treated as categorical even when a
/// dictionary tagged it an explicit `measure.*` concept. Much tighter than
/// `CATEGORICAL_MAX_CARDINALITY` so a genuine continuous measure (which spans far more than a
/// handful of integer values) is never demoted. 12 comfortably covers 0-10 / 1-10 scales.
const RATING_MAX_CARDINALITY: u64 = 12;

/// `viz smart` categorical part-to-whole (treemap/sunburst) panel tuning.
/// Maximum nesting levels (root excluded): with 3+ eligible low-cardinality dimensions a
/// 3-level hierarchy is built (rendered as a sunburst per best practice for deep paths),
/// otherwise a 2-level hierarchy (rendered as a treemap for shallow size comparison).
const HIER_MAX_DEPTH: usize = 3;
/// Per parent, only the top-N children (by value desc, label asc) are kept; the remainder
/// collapse into a single "Other (k)" sibling (mirrors `finalize_freq_bars`).
const HIER_TOP_N_PER_LEVEL: usize = 12;
/// Global safety cap on emitted hierarchy nodes; once reached, deeper expansion stops
/// (capped nodes render as leaves). Keeps a pathological column combo from exploding.
const HIER_MAX_NODES: usize = 200;
/// Minimum eligible categorical dimensions required to build a hierarchy panel at all.
const HIER_MIN_DIMS: usize = 2;
/// A dimension needs at least this many distinct values to be worth a hierarchy level: a 2-value
/// (boolean-like) column is a trivial split better shown as a single bar, and nesting by it would
/// needlessly force the whole Data Schematic onto the inline (non-typed-grid) render path.
const HIER_MIN_DIM_CARDINALITY: u64 = 3;
/// Path separator used to build collision-free plotly node `ids` (US control char, which
/// cannot occur in trimmed cell text).
const HIER_PATH_SEP: char = '\u{001F}';
/// Synthetic root node id for the hierarchy (parent of the level-1 categories).
const HIER_ROOT_ID: &str = "\u{001F}root";
/// Inline render height (px) for a hierarchy panel — taller than a normal overview cell so
/// nested treemap rectangles / sunburst rings stay legible.
const HIER_ROW_HEIGHT_PX: usize = 520;
/// Minimum (bias-corrected) Cramér's V — across every pair of the candidate hierarchy dimensions —
/// for `viz smart` to AUTO-build a treemap/sunburst. Nesting statistically independent categoricals
/// just replicates each level's marginal at every branch, conveying no structure that the separate
/// per-column frequency bars don't already show more legibly; below this association the panel is
/// skipped (a note is printed). 0.10 is Cohen's "small effect" floor for V. An explicit
/// `--hierarchy-style treemap|sunburst` bypasses the screen (the user asked for it deliberately).
const HIER_MIN_ASSOCIATION_CRAMERS_V: f64 = 0.10;

/// `viz smart --bivariate` directed-flow (Sankey) summary panel tuning.
/// A Sankey dimension needs at least this many distinct values to be worth a flow diagram — a
/// 2-value (boolean-like) column is a trivial split better shown as a bar. Matches
/// `HIER_MIN_DIM_CARDINALITY`; the upper bound is `CATEGORICAL_MAX_CARDINALITY` (the same
/// categorical pool the hierarchy panel draws from, so the two panels stay mutually exclusive on
/// a shared set of dimensions).
const SANKEY_MIN_DIM_CARDINALITY: u64 = HIER_MIN_DIM_CARDINALITY;
/// Per side, only the top-N categories (by total flow desc, label asc — the `finalize_freq_bars`
/// comparator) are kept; the remainder collapse into a single "Other (k)" node so the diagram
/// stays legible. Matches `HIER_TOP_N_PER_LEVEL`.
const SANKEY_TOP_N_PER_SIDE: usize = HIER_TOP_N_PER_LEVEL;
/// The many-to-many band on Theil's directed uncertainty coefficient U = MI / H(target) (from
/// moarstats' `--bivariate-stats u`). Below `SANKEY_U_MIN` the two dimensions are ~independent (no
/// flow structure the per-column bars don't already show); at/above `SANKEY_U_MAX` in EITHER
/// direction the mapping is near-functional (part-to-whole → the treemap/sunburst hierarchy panel
/// conveys it better). A Sankey earns its place only when the flow genuinely fans out both ways,
/// i.e. min over both directions ≥ MIN and max over both directions ≤ MAX. These are fixed
/// absolute cutoffs (like the sibling `|r|`/η²/Cramér's V gates) — U is self-normalizing, so the
/// same cutoff means the same thing on every dataset; do NOT auto-tune them per file.
const SANKEY_U_MIN: f64 = 0.10;
const SANKEY_U_MAX: f64 = 0.85;
/// If either side's collapsed "Other (k)" node exceeds this fraction of that side's total flow,
/// the top-N view would misrepresent the distribution, so the panel is skipped (a note is printed).
const SANKEY_MAX_OTHER_FRACTION: f64 = 0.50;

/// Opacity for a Sankey link ribbon, tinted from its SOURCE node's `PALETTE` color so each
/// flow is traceable back to where it originates (translucent enough to read overlaps and to
/// hold contrast against both the light and dark Data Schematic themes).
const SANKEY_LINK_ALPHA: f64 = 0.45;

/// Plotly `maxdepth` for sunburst panels: the number of levels rendered at once from the current
/// root, counting the center as one. 3 = center + 2 data rings. A 3-level sunburst fanned out to
/// ~100 tiny outer sectors whose labels were unreadable; capping the initial view to two rings
/// keeps every visible label legible, and clicking a sector rescales its subtree to the full ring
/// so the deeper level (with its now-large labels) is revealed on drill-down. For 2-level
/// hierarchies this is a no-op (both rings already fit within the cap).
const SUNBURST_MAXDEPTH: i32 = 3;

/// `viz smart --log-scale auto`: a frequency bar panel switches to a logarithmic y-axis when
/// its tallest bar is at least this many times the shortest positive bar. A dominating
/// "(NULL)"/"Other (N)" bucket flattens the real categories to invisible slivers on a linear
/// axis; a log scale keeps them legible. Requires at least 3 positive bars.
const LOG_SCALE_MIN_RATIO: f64 = 50.0;

/// Minimum absolute Pearson correlation for `viz smart` to add a scatter panel of the most
/// strongly correlated numeric pair. Below this (a weak relationship) the scatter is just a
/// noise cloud, so it's skipped — the correlation heatmap already conveys weak correlations.
const SCATTER_PAIR_MIN_ABS_R: f64 = 0.5;

/// Minimum centroid-path CURVATURE (in standardized units) for `viz smart` to ANIMATE a numeric
/// pair over time. Score = the perpendicular spread of the per-time-bucket centroids about their
/// OWN best-fit line (the smaller eigenvalue's √ of the standardized centroid-path covariance) —
/// how much the relationship's centroid path BENDS as time advances. It cleanly separates the only
/// case worth animating (a genuinely evolving 2-D relationship — a curved path) from the cases that
/// are NOT: a tautological near-line (centroids slide along the thin line → ~0), and two
/// independent 1-D trends (a straight centroid path, already told by the per-axis trend panels →
/// low). No |r| bound is needed — a high-|r| pair is a thin line whose centroids can't leave it, so
/// its curvature is intrinsically ~0. Empirically both tautological showcase pairs score ≤ 0.16 and
/// every boring dual-trend pair ≤ 0.26, while a real time-arc scores ~0.94; this bar sits
/// comfortably between.
const PAIR_CURVATURE_MIN: f64 = 0.5;

/// Trailing-window width (in time buckets) for the animated pair reveal: each frame shows the
/// current bucket plus the `ANIM_TRAILING_WINDOW - 1` preceding ones, so the cloud is seen to
/// MIGRATE rather than merely accumulate (a cumulative reveal hides motion — the whole point of a
/// drift-selected pair is the motion). Geo panels keep a cumulative reveal (events accumulating
/// across a map reads naturally).
const ANIM_TRAILING_WINDOW: usize = 3;

/// `viz smart` parallel-categories (parcats) panel bounds. parcats is reserved for
/// `PARCATS_MIN_DIMS` to `PARCATS_MAX_DIMS` categorical dimensions: 2-dimension relationships are
/// shown as a directed Sankey, and beyond 4 dimensions the ribbon flow becomes an unreadable
/// tangle.
const PARCATS_MIN_DIMS: usize = 3;
const PARCATS_MAX_DIMS: usize = 4;

/// Max distinct category tuples (ribbons) the parcats panel emits. Each eligible dimension can have
/// up to `CATEGORICAL_MAX_CARDINALITY` (30) values, so 4 dimensions could yield up to 30⁴ ≈ 810k
/// distinct paths — an enormous, unreadable trace. The panel keeps only the `PARCATS_MAX_PATHS`
/// heaviest tuples (by row count) and drops the long tail (logged), matching the bounded-node
/// approach of the hierarchy (`HIER_MAX_NODES`) and Sankey panels.
const PARCATS_MAX_PATHS: usize = 200;

/// Minimum correlation ratio η² (ANOVA effect size: between-group variance / total variance) for
/// `viz smart` to add a "measure by dimension" bar. η² ranges 0..1; 0.06 is Cohen's conventional
/// "medium" effect. Below this the dimension barely explains the measure, so the grouped bar would
/// be misleading noise — matching the strength-gate convention of the scatter (`|r|`) and hierarchy
/// (Cramér's V) overview panels.
const MEASURE_BY_DIM_MIN_ETA_SQUARED: f64 = 0.06;

/// `viz smart` skips the 3D scatter drill-down when the strongest numeric pair is at or above this
/// absolute correlation. The 3D is built ON that pair plus a third axis; when the pair is
/// near-collinear (|r| ~ 1) the two axes are effectively the same variable, so the cloud collapses
/// onto a plane and the "3D" adds nothing the 2D pair drill-down doesn't already show. The third
/// axis is otherwise chosen to be the LEAST redundant with the pair so the panel uses all three
/// dimensions.
const SMART_3D_COLLINEAR_MAX_ABS_R: f64 = 0.97;

/// `viz smart` flags the correlation drill-down pair as nonlinear when its Spearman |rho| exceeds
/// its Pearson |r| by at least this much. A large gap means the relationship is monotonic but
/// curved, so the single Pearson number understates it — the panel title notes the rho so the
/// reader doesn't read the cloud as merely linear. Pearson alone can't see this.
const SMART_NONLINEAR_MIN_GAP: f64 = 0.15;

/// Pearson skewness at or above which a column reads as pronouncedly RIGHT-skewed — a long high
/// tail over a dense low bulk. Matches `box_shape_hint`'s `SKEW_MIN_ABS`, so a column the box
/// panel titles "right-skewed" is the same column that gets a log value axis
/// (`box_log_skew_fallback`).
///
/// This detects MODERATE skew. It deliberately does NOT gate the robust-statistics choices
/// (`mean_is_outlier_driven`), which target a far more extreme regime this coefficient cannot
/// see — see that function for why.
const RIGHT_SKEW_MIN: f64 = 0.5;

/// How many times the median a column's mean must reach before the mean is treated as
/// outlier-driven rather than typical (`mean_is_outlier_driven`, issue #4220). At 2x the "average"
/// is already double the middle value, so half the rows sit below half the reported average — the
/// point past which ranking groups by their mean says more about who holds the biggest item than
/// about the groups.
const MEAN_MEDIAN_RATIO_MIN: f64 = 2.0;

/// Share of a correlation matrix's columns whose mean must be outlier-driven
/// (`mean_is_outlier_driven`) before `viz smart` computes that matrix with SPEARMAN's rank
/// correlation
/// instead of Pearson's (issue #4220). On heavy-tailed columns a handful of extreme rows dominate
/// Pearson's covariance, so the reported r describes the outliers rather than the bulk; ranks are
/// immune to that. A simple majority is the trigger: the matrix is ONE panel with ONE coefficient
/// label, so the choice has to describe the table as a whole, and a single skewed column among
/// many well-behaved ones is not reason enough to re-express every cell.
const SMART_CORR_SPEARMAN_MIN_SKEWED_SHARE: f64 = 0.5;

/// `viz pie` prints a non-fatal advisory (suggesting a bar chart) when it has at least
/// `PIE_NEAR_EQUAL_MIN_SLICES` slices whose coefficient of variation is below this: near-equal
/// slices are the worst case for a pie (the eye can't compare similar angles/areas), and a bar
/// chart is strictly easier to read. The explicit `viz pie` request is still honored.
const PIE_NEAR_EQUAL_MAX_CV: f64 = 0.35;
const PIE_NEAR_EQUAL_MIN_SLICES: usize = 4;

/// At or above this many complete rows, `viz smart` draws the strongest-correlated numeric pair as
/// a 2D density contour instead of a scatter: past this point a scatter overplots into a solid
/// mass, while a binned contour embeds only a fixed grid (so it's both more readable and far
/// smaller).
const SMART_CONTOUR_MIN_POINTS: usize = 5_000;

/// Bin resolution (per axis) for `viz smart`'s correlated-pair density contour panel.
const SMART_CONTOUR_BINS: usize = 30;

/// Legibility gate for `viz smart`'s density contour panel (issue #4223): when a single bin of the
/// `SMART_CONTOUR_BINS` x `SMART_CONTOUR_BINS` grid holds more than this share of ALL points, the
/// contour has collapsed into one dark cell against an empty field — every other bin is either
/// empty or indistinguishable, so the panel shows no distribution at all. This is the contour
/// counterpart of `scatter_fill_stats`'s `degenerate` verdict, which already drops an equivalently
/// unreadable scatter (the two branches of the same drill-down slot should hold to one standard).
/// Heavily right-skewed, zero-inflated money columns are the canonical trigger.
const SMART_CONTOUR_MAX_BIN_SHARE: f64 = 0.5;

/// Minimum surviving points for the positive-subset log density retry (issue #4223). When linear
/// binning fails `SMART_CONTOUR_MAX_BIN_SHARE`, the panel is rebuilt from only the rows positive on
/// BOTH axes, binned in log space — but that subset has to be big enough to still describe a
/// distribution rather than a handful of survivors.
const SMART_CONTOUR_LOG_MIN_POINTS: usize = 500;

/// Companion floor to `SMART_CONTOUR_LOG_MIN_POINTS`: the positive subset must also retain at least
/// this share of the rows. An absolute count alone would admit a 500-of-500,000 sliver, whose
/// density is a statement about a rounding error, not about the dataset.
const SMART_CONTOUR_LOG_MIN_SHARE: f64 = 0.1;

/// `viz smart` point-count thresholds for the default (no explicit `--box-points`) box-overlay
/// heuristic, measured against each column's NON-NULL value count (only those values are
/// collected, embedded and rendered — a mostly-null column shouldn't be denied an overlay tier
/// by rows it doesn't have): at or below ALL, every sample point is overlaid on a box (via
/// `BoxRaw`); at or below OUTLIERS, only the Tukey outliers (via `BoxRaw`). ABOVE OUTLIERS,
/// embedding the full column is an unreadable smear and a large payload, so the box stays a
/// precomputed quartile box — but if the column actually HAS outliers (cached min/max fall
/// outside the Tukey fences), they're overlaid as native box points via a single fence-filtered
/// pass (`BoxOutliers`); a column with no outliers stays a pure cache-only `BoxStats` with no
/// pass at all.
const SMART_BOX_ALL_MAX: u64 = 1_000;
const SMART_BOX_OUTLIERS_MAX: u64 = 10_000;

/// Above this many surviving Data Schematic panels, the size-based all-points box overlay
/// (`SMART_BOX_ALL_MAX`) is demoted to outliers-only: a thousand jittered points per panel reads
/// fine at 2-panel size but turns to noise at postage-stamp cell size. Matches `MAX_SUBPLOTS` —
/// past the typed-grid cap the Data Schematic switches to the denser inline grid. An explicit
/// `--box-points` mode is the user's call and is never demoted.
const SMART_ALL_POINTS_MAX_PANELS: usize = MAX_SUBPLOTS;

/// Max number of outlier points embedded per `BoxOutliers` panel, keeping the HTML bounded for
/// heavy-tailed columns. When a column has more outliers than this, the rest are dropped (the
/// overflow is logged); far below `MAX_SMART_POINTS` since these all render as discrete dots.
const SMART_BOX_OUTLIERS_CAP: usize = 5_000;

/// `viz smart` renders its geographic panel as a `ScatterGeo` projection world-overview (offline,
/// no tiles) instead of a zoomed MapLibre tile map when the coordinates span at least this many
/// degrees of longitude OR latitude — i.e. continental/global data, where a whole-world projection
/// gives better context than tiles framed to a wide bounding box. Local extents keep the tile map.
const SMART_GEO_MIN_LON_SPAN_DEG: f64 = 90.0;
const SMART_GEO_MIN_LAT_SPAN_DEG: f64 = 45.0;

/// Minimum spatial extent (in degrees of longitude AND latitude) before `viz smart` attempts the
/// per-country choropleth overview. A per-country breakdown is only meaningful at country scale, so
/// when the data's extent is smaller than this in BOTH dimensions it is treated as a single
/// metro/region cluster and skipped — which avoids the expensive all-row reverse-geocode pass (and
/// its index load) entirely for the common case of a high-row-count city/metro dataset. Anything
/// wider in either axis runs the pass, and `build_smart_choropleth_panel`'s own 2-country check has
/// the final say (a wide single-country dataset still resolves to one country → no panel). This is
/// a cheap min/max-extent gate that, unlike the geocoded bounding-box corners, also works for
/// global extents that wrap the antimeridian (where the corner metadata is suppressed).
const SMART_CHOROPLETH_MIN_SPAN_DEG: f64 = 8.0;

/// Fill opacity for a tile-basemap `ChoroplethMap` (the `viz smart` metro choropleth panel and
/// `viz choropleth --map`). Paired with `below("")`, which lifts the fill above the basemap's road
/// layers so they don't render on top and muddy the read; a near-opaque fill then keeps the region
/// colors legible while the small gaps between polygons (rivers, harbor) still reveal enough
/// basemap to orient the map.
const CHOROPLETH_MAP_FILL_OPACITY: f64 = 0.9;

/// Bimodality-coefficient threshold (Sarle's BC). A continuous numeric column whose
/// `bimodality_coefficient` reaches this AND is platykurtic (see `classify_measure`) is treated as
/// bimodal/multimodal, so `viz smart` draws a histogram (which shows the separate peaks) instead of
/// a box plot (which hides them). The coefficient is supplied by moarstats under `--smarter`, or
/// computed in one streaming pass by `enrich_bimodality` for plain `viz smart`.
///
/// Sarle's textbook cutoff is 5/9 (~0.5556) — the value for a UNIFORM distribution. But finite
/// samples of a uniform (or near-uniform) column scatter just ABOVE 5/9, so a strict cutoff
/// over-flags flat-but-unimodal data as "bimodal". A small margin (0.60) keeps genuinely two-peaked
/// columns (BC typically 0.7-1.0) while letting near-uniform columns stay box plots.
const BIMODALITY_COEFFICIENT_THRESHOLD: f64 = 0.60;

/// A violin's KDE over fewer distinct values than this reads as a lumpy comb, not a shape —
/// such columns keep their box panel under `--violin auto`.
const VIOLIN_MIN_CARDINALITY: u64 = 20;

/// A KDE needs enough *observations* — not just distinct values — before its silhouette reflects
/// the data rather than the smoothing bandwidth. `VIOLIN_MIN_CARDINALITY` guards distinct-value
/// count, but a near-unique column can clear it with only a few dozen rows, where the resulting
/// curve is bandwidth artifact rather than shape. Under `--violin auto` a column with fewer than
/// this many non-null values keeps its honest box (five numbers that degrade gracefully); `on`
/// still violins it at the user's explicit request.
const VIOLIN_MIN_POINTS: u64 = 50;

/// Target sample size for a violin (single-column or grouped) drawn from a column with more than
/// `SMART_BOX_OUTLIERS_MAX` non-null values: the values are deterministically strided down to at
/// most this many during collection (columns between the two thresholds keep all their values —
/// stride 1 — but already drop the point overlay). A KDE silhouette (and sample quartiles for
/// the inner box) from tens of thousands of evenly spaced values is visually indistinguishable
/// from one drawn from millions, at a bounded embed cost — so violins have NO dataset-size
/// ceiling. Sampled violins drop their point overlay: a strided sample misses the extremes, so
/// sampled "outliers" would mislead (boxes keep exact outliers via the separate fence-filtered
/// `BoxOutliers` pass).
///
/// Derived as one-fifth of the `MAX_SMART_POINTS` budget (30k at the 150k default), so
/// `QSV_VIZ_MAX_POINTS` scales the violin sample proportionally with the scatter/map budget. The
/// point-overlay cutoff (`SMART_BOX_OUTLIERS_MAX`) stays fixed, so the env var changes sample
/// density and the "(sampled)" boundary, not chart-type or overlay decisions.
static VIOLIN_SAMPLE_MAX: std::sync::LazyLock<u64> =
    std::sync::LazyLock::new(|| ((*MAX_SMART_POINTS as u64) / 5).max(1));

/// `viz smart` charts a high-cardinality categorical column (one that would otherwise be skipped)
/// only when moarstats says its distribution is concentrated rather than near-uniform: a
/// `normalized_entropy` at/above this is treated as noise (every value about equally frequent),
/// while a lower value means a few dominant categories worth a top-N bar.
const HIGH_CARD_ENTROPY_NOISE_THRESHOLD: f64 = 0.95;

/// Max bars in a single-series `viz bar` chart that still get value labels; beyond this the
/// labels would overlap, so they're omitted.
const LABEL_MAX_BARS: usize = 40;

/// `viz smart` caps how many data points it embeds in a single panel (map, time-series, and
/// correlated-pair scatter). Beyond this, the points are uniformly downsampled — stride sampling
/// preserves the overall shape/distribution of the data. The motivating case: a 1M-row dataset's
/// map once embedded 745K opaque markers — an unreadable blob, a ~25 MB payload, and a browser
/// that froze on the first pan/zoom. Originally 50K back when every embedded point cost ~200
/// plaintext bytes; raised to 150K after the compressed-payload work (trace-level hovertemplates
/// + float32 typed arrays + gzip'd figure payloads) cut the marginal cost to ~16 bytes/point (a
///   150K-point map Data Schematic is ~4.4 MB all-in) and MapLibre GL measured a steady 61 fps
///   pan/zoom at 150K-250K markers.
const DEFAULT_MAX_SMART_POINTS: usize = 150_000;

/// The resolved per-panel point budget, overridable with the `QSV_VIZ_MAX_POINTS` env var (an
/// advanced knob for denser/leaner output — larger values embed more points at the cost of a
/// heavier, slower-to-render HTML). A non-numeric or zero value falls back to
/// `DEFAULT_MAX_SMART_POINTS`. The violin sample budget (`VIOLIN_SAMPLE_MAX`) derives from this.
static MAX_SMART_POINTS: std::sync::LazyLock<usize> = std::sync::LazyLock::new(|| {
    std::env::var("QSV_VIZ_MAX_POINTS")
        .ok()
        .and_then(|v| v.trim().parse::<usize>().ok())
        .filter(|&n| n > 0)
        .unwrap_or(DEFAULT_MAX_SMART_POINTS)
});

/// Max geographic outlier points embedded per `viz smart` map panel. Outliers are the whole point
/// of the call-out, so this is set generously, but still bounds the embedded payload; if there are
/// more, they're uniformly stride-sampled (like the core points) rather than dropped wholesale.
const SMART_GEO_OUTLIER_CAP: usize = 5_000;

/// Marker opacity for the `viz smart` map panel when it's drawn as discrete points (i.e. fewer
/// than the `--heatmap-density` threshold rows). Mild transparency reveals overlapping points
/// instead of a flat blob.
const MAP_POINT_OPACITY: f64 = 0.4;

/// Styling for map points whose data-viewer row is selected (see `PanelKind::Map::row_ids`).
///
/// Size and color do the work, not opacity: the core markers already draw at `MAP_POINT_OPACITY`
/// (0.4), so there is little headroom to make a point stand out by becoming *more* opaque. The
/// selected fill is a magenta chosen to sit outside both the categorical palette (blues/greens)
/// and `GEO_OUTLIER_COLOR` (amber), so a selected outlier still reads as selected.
const MAP_SELECTED_POINT_SIZE: usize = 14;
const MAP_SELECTED_POINT_COLOR: &str = "#ff2d95";
/// Points *not* in the selection dim to this while a selection is active, so the selected few
/// stay findable in a dense cloud. Restores to `MAP_POINT_OPACITY` when the selection clears.
const MAP_UNSELECTED_POINT_OPACITY: f64 = 0.12;

/// Trace name of the "row pin" — the marker that stands in for a selected data-viewer row whose
/// point was never plotted (the map draws at most `MAX_SMART_POINTS`, so on a downsampled Data
/// Schematic most rows have no point of their own). Emitted EMPTY and filled in by the bridge.
///
/// Deliberately NOT localized: this is the handle the JS finds the trace by, since its index shifts
/// with the optional extent/`--geojson` overlay traces that may precede it. It never reaches a
/// reader — the trace and the layout both set `show_legend(false)`, and `MAP_HOVER_TEMPLATE` ends
/// in `<extra></extra>`, which suppresses the trace-name box. (Same bare-sentinel convention as the
/// `--geojson` boundary trace's `name("regions")`.)
const MAP_ROW_PIN_NAME: &str = "qsv-row-pin";

/// The halo drawn UNDER the pin, as its own trace.
///
/// A ring is the whole point of the pin's identity — it reuses the selection magenta (the pin IS
/// the selection) so only the silhouette can say "this is not one of the plotted points". It cannot
/// come from `marker.line`: plotly SILENTLY DROPS `marker.line` on a `scattermap`, and the
/// resulting MapLibre circle layer carries only `circle-color`/`circle-radius`/`circle-opacity`
/// with no `circle-stroke-*` at all (verified in-browser against the emitted style). So the ring is
/// a second, larger white marker beneath — which also means the two traces must be filled in and
/// cleared TOGETHER, and the halo has to be pushed FIRST so it renders below.
const MAP_ROW_PIN_HALO_NAME: &str = "qsv-row-pin-halo";

/// Pin sizes: the magenta core, and the white halo behind it. The gap is the ring's thickness, and
/// the core is deliberately larger than `MAP_SELECTED_POINT_SIZE` so a pin never reads as a
/// selected plotted point when a mixed selection puts both on screen.
const MAP_ROW_PIN_SIZE: usize = 18;
const MAP_ROW_PIN_HALO_SIZE: usize = 26;

/// At or above this many rendered core points, the `viz smart` map panel enables native MapLibre
/// point clustering on the core marker trace (unless a density heatmap or bubble-size encoding is
/// active). Below it, a handful of markers reads fine as-is and clustering would needlessly merge
/// well-separated points into a count bubble. Clustering is client-side, so it declutters overplot
/// without changing the embedded payload — the point cap (`MAX_SMART_POINTS`) still bounds size.
const SMART_CLUSTER_MIN_POINTS: usize = 1_000;

/// Zoom level at/above which point clusters fully disengage into individual markers. Set as high as
/// the basemap allows so clusters keep subdividing into finer bubbles as you zoom in, rather than
/// dumping a dense region's raw (overplotting) points all at once. Kept STRICTLY below the raster
/// basemap tile source's max zoom (18 for Carto/OSM) — MapLibre warns when `clusterMaxZoom` reaches
/// the source maxzoom (and plotly.js otherwise defaults `cluster.maxzoom` to 24). 17 is that
/// ceiling: one level deeper than the previous 16, so a dense metro keeps resolving finer.
const SMART_CLUSTER_MAX_ZOOM: f64 = 17.0;

/// Fraction trimmed from each end of the latitude/longitude distributions when framing a map, so a
/// few outlier coordinates (bad geocodes, sentinel values) can't blow up the center/zoom and push
/// the bulk of the data off-screen. 2.5% off each end — only the initial view is trimmed; every
/// point is still plotted, so a slightly tight default frame just means panning out to see the
/// long tail.
const MAP_FRAME_TRIM_FRAC: f64 = 0.025;

/// Padding added to each side of a map's data extent when deriving the MapLibre pan constraint
/// (`layout.map.bounds`), as a fraction of the extent's span. 1.0 lets the viewer zoom out to
/// roughly 3x the data's footprint for surrounding context, while still stopping a pan that would
/// slide the data off-screen into empty ocean and lose it entirely.
const MAP_BOUNDS_PAD_FRAC: f64 = 1.0;

/// Floor (in degrees) on the pan-constraint padding, so a tight city/metro extent still gets some
/// breathing room instead of a box clamped hard against the outermost points.
const MAP_BOUNDS_MIN_PAD_DEG: f64 = 0.5;

/// A pan constraint is skipped when the data spans more than this many degrees of longitude:
/// near-global or antimeridian-crossing extents can't be expressed as a simple west < east box,
/// and constraining a world-scale map adds nothing.
const MAP_BOUNDS_MAX_LON_SPAN_DEG: f64 = 180.0;

/// Marker diameter (pixels) the smallest and largest `--size` values map to in a bubble
/// scatter. Raw size values are linearly rescaled into this range so the plot stays
/// readable regardless of the column's magnitude.
const BUBBLE_MIN_PX: f64 = 6.0;
const BUBBLE_MAX_PX: f64 = 40.0;

/// Vertical space (in pixels) allotted to each row of subplots in the `viz smart`
/// Data Schematic. The total plot height scales with the number of rows so panels stay
/// readable instead of being crammed into plotly's ~450px default.
const ROW_HEIGHT_PX: usize = 320;

/// Taller height (in pixels) for full-width "overview" panels (map/geo, correlation heatmap and
/// its drill-downs, time-series) in the inline HTML Data Schematic, so they get more room than the
/// per-column box/bar/histogram panels — and, for a map, so the spatial-extent caption below it
/// has space.
const OVERVIEW_ROW_HEIGHT_PX: usize = 420;

/// Taller height (in pixels) for geographic overview panels (`Map`/`Geo`/`Choropleth`/
/// `ChoroplethMap`) in the inline HTML Data Schematic. A world/continental map at the default
/// overview height renders its points cramped into a short band; maps get their own taller base so
/// the markers are legible and the projection fills the cell. Fixed (not extent-aspect-driven) so
/// `MAP_PANEL_USABLE_HEIGHT_PX` — which `map_center_zoom` uses to frame the map — stays a constant.
const MAP_ROW_HEIGHT_PX: usize = 540;

/// Extra vertical band (in pixels) reserved BELOW an animated map (`AnimatedGeo`) for the scrub
/// slider + Play/Pause menu, so the controls sit under the map instead of overlaying and shrinking
/// it. Added to the panel height AND to the render-block bottom margin, so the map's usable draw
/// height matches a static map (`MAP_ROW_HEIGHT_PX - 48 - 20`) and the controls occupy this band.
const SLIDER_CONTROL_ALLOWANCE_PX: usize = 96;

/// Upward shift (px, above the plot area) of an inline panel's `--dict-info` title annotation, so
/// it lands in the top-margin band where a plain layout title would render.
const DEFAULT_TITLE_Y_SHIFT: f64 = 8.0;
/// The same shift for a cyclic-seasonality (polar) panel. Its angular tick labels are painted
/// ~22px ABOVE the plot area — this plotly version's `LayoutPolar` has no `domain` to rein the ring
/// in — so the title is lifted clear of them (paired with that panel's taller top margin).
const POLAR_TITLE_Y_SHIFT: f64 = 30.0;

/// Per-distinct-category vertical step (px) for a `Parcats` panel, and its clamp. Parcats height
/// scales with the tallest dimension's distinct-category count so many categories aren't crammed
/// into a short panel; clamped so a pathological high-cardinality axis can't produce a giant panel.
const PARCATS_PX_PER_CATEGORY: usize = 44;
const PARCATS_HEIGHT_MIN_PX: usize = 560;
const PARCATS_HEIGHT_MAX_PX: usize = 980;

/// Per-node vertical step (px) for a `Sankey` panel (scaled by the tallest column's node count),
/// and its clamp — same rationale as the parcats scaling.
const SANKEY_PX_PER_NODE: usize = 40;
const SANKEY_HEIGHT_MIN_PX: usize = 520;
const SANKEY_HEIGHT_MAX_PX: usize = 960;

/// Shorter height (in pixels) for a `ScatterPair` whose points are sparse / piled at one corner
/// (e.g. a heavily zero-skewed measure): the full overview height would be mostly an empty box, so
/// a low-fill scatter gets a compact panel. A DEGENERATE scatter (nearly all points identical) is
/// dropped entirely at build time rather than shrunk (see `scatter_fill_stats`).
const SCATTER_SPARSE_HEIGHT_PX: usize = 240;

/// Shorter height (in pixels) for the leading KPI overview row when its tiles are plain number
/// tiles: a big number plus a one-to-two-line label under it is much shorter than a chart, so the
/// full `OVERVIEW_ROW_HEIGHT_PX` leaves a large dead band below the labels. Sized to the tiles'
/// actual content so the row sits snug above the charts below it.
const KPI_ROW_HEIGHT_PX: usize = 230;

/// Height (in pixels) for a KPI row that contains at least one gauge tile. Gauge mode renders the
/// number lower (at the dial pivot), ~30px below where a plain number tile centers it, so the row
/// needs more vertical room than a pure-number row for the shared label baseline to clear the
/// gauge number without overlapping it.
const KPI_ROW_GAUGE_HEIGHT_PX: usize = 250;

/// Horizontal space (in pixels) per Data Schematic grid column, used to auto-size the `viz
/// smart` static image export width.
const SMART_COL_WIDTH_PX: usize = 500;

/// Fallback static-export image dimensions (pixels) when --width/--height are not given
/// and no auto-scaling applies (i.e. the non-`smart` chart types).
const DEFAULT_IMG_WIDTH: usize = 1000;
const DEFAULT_IMG_HEIGHT: usize = 600;

/// Web-Mercator tile size (px): MapLibre zoom z fits 360° of longitude across `512 * 2^z` px.
/// The unit the slippy-tile zoom math in `fitbounds_zoom` is defined against.
const MAP_TILE_SIZE_PX: f64 = 512.0;

/// Anti-clip margin for the `fitbounds_zoom` framing (a 1.15× span pad ≈ a 0.20 zoom reduction).
const MAP_FIT_PAD: f64 = 1.15;

/// Usable map draw height (px) for a `viz smart` MAP overview panel: `MAP_ROW_HEIGHT_PX` (540)
/// minus the map layout's top (48) + bottom (20) margins = 472. This is the *reliable* dimension at
/// HTML-generation time (width is `responsive`/unknown) and, for the wide-short map panel, the
/// binding constraint — so the latitude fit is computed against it. An animated map (`AnimatedGeo`)
/// reserves an extra `SLIDER_CONTROL_ALLOWANCE_PX` band in BOTH its height and its bottom margin,
/// so its usable draw height equals this same value.
const MAP_PANEL_USABLE_HEIGHT_PX: f64 = MAP_ROW_HEIGHT_PX as f64 - 48.0 - 20.0;

/// Assumed map draw width (px) for the smart MAP panel longitude fit. The panel spans the full
/// grid (`grid-column: 1 / -1`) and its real width is responsive/unknown when the HTML is
/// generated. The reliable `MAP_PANEL_USABLE_HEIGHT_PX` term binds the common case (a ~square or
/// tall extent is latitude-bound, so this width is irrelevant to it); the width term only matters
/// for a genuinely WIDE extent (a far east/west outlier stretching longitude). For those, over-
/// assuming the width would *over*-zoom and clip the extent on a narrower viewport — and silently
/// hiding points (especially the strays the Full extent exists to reveal) is worse than leaving
/// horizontal map context. So this is tied to a conservative minimum desktop content width (~960px,
/// safe for windows down to ~1024px): no over-zoom/clipping on supported viewports, at the cost of
/// some horizontal margin for wide extents on larger screens. A truly exact fit would recompute
/// from the live plot-div width in the browser (a possible follow-up).
const MAP_PANEL_ASSUMED_WIDTH_PX: f64 = 960.0;

/// Soft qualitative palette (Vega/Tableau-10) for coloring Data Schematic panels — distinct
/// but harmonious, and friendlier than plotly's saturated defaults.
const PALETTE: [&str; 8] = [
    "#4C78A8", "#F58518", "#54A24B", "#E45756", "#72B7B2", "#EECA3B", "#B279A2", "#FF9DA6",
];

/// Shared UI font stack and "ink" color for all Data Schematic text.
const FONT_FAMILY: &str = "Helvetica Neue, Helvetica, Arial, sans-serif";
const INK: &str = "#2A3F5F";
const PAPER_BG: &str = "#FFFFFF";
/// Stable `layout.uirevision` token. Plotly preserves user UI state (zoom, pan, box/lasso
/// selection, legend visibility) across `Plotly.react`/`Plotly.newPlot` calls as long as this
/// value is unchanged. qsv re-renders every graph div via `newPlot` to inject the fullscreen
/// modebar button and to recolor on theme toggle; without a constant `uirevision` those
/// re-renders reset the user's view. Applied centrally in `apply_theme`.
const VIZ_UIREVISION: &str = "qsv-viz";
const GRID_COLOR: &str = "#ECECEC";
const AXIS_LINE: &str = "#BCC4CE";

/// `geo` subplot fill palette (land / water / background), shared by the Rust initial render and
/// the `viz smart` light/dark toggle JS so choropleth & geo maps match the active theme instead of
/// always rendering light land/ocean. Light mirrors qsv's built-in look (`lightgray`/`lightblue`);
/// dark harmonizes with the dark toggle palette (paper `#111111`, line `#506784`).
const GEO_LAND_LIGHT: &str = "#d3d3d3";
const GEO_WATER_LIGHT: &str = "#add8e6";
const GEO_LAND_DARK: &str = "#2a3138";
const GEO_WATER_DARK: &str = "#16202b";
const GEO_BG_DARK: &str = "#111111";

/// Data Schematic plot margins (pixels). Kept as named constants because the title-band math
/// below needs the plot-area height (total height minus these margins).
const TOP_MARGIN_PX: usize = 80;
const BOTTOM_MARGIN_PX: usize = 60;

/// Pixels reserved above the top row of panels for their titles, and the gap (in pixels)
/// between a cell's top edge and its title. Both are kept in *pixels* (converted to paper
/// fractions via the plot-area height) so neither the band nor the offset scales with the
/// Data Schematic height — otherwise a tall Data Schematic's title would drift past `y=1` and
/// overlap the Data Schematic title. The band must comfortably exceed `TITLE_OFFSET_PX` plus the
/// title's rendered glyph height (~17px for the 13px font).
const TITLE_BAND_PX: usize = 32;
const TITLE_OFFSET_PX: usize = 6;
/// Extra band pixels reserved when panels carry a subtitle (dictionary label under the title),
/// so the second, smaller line clears the chart above without overlap.
const TITLE_SUBTITLE_EXTRA_PX: usize = 16;

/// Default Data Schematic left margin (pixels), widened when a correlation-heatmap panel is present
/// so its (long) numeric-column tick labels aren't clipped.
const DEFAULT_LEFT_MARGIN_PX: usize = 60;

/// Y-axis title shown ONLY on frequency panels rendered with a logarithmic y-axis (see
/// `--log-scale`). It's the visual cue that the axis is log, not linear; linear panels stay
/// title-less to keep the cells compact. The rotated title needs a little extra left margin
/// (`LOG_AXIS_TITLE_MARGIN_PX`) so it isn't clipped against the page edge.
///
/// A function rather than a `const` because the text is localized and the catalog resolves at
/// runtime.
fn log_axis_title() -> String {
    t!("viz.chart.log_axis_count").into_owned()
}
/// The log-cue y-axis title for a VALUE axis (a log box panel), where "count (log)" would be
/// wrong — the axis carries the column's values, not frequencies.
fn value_log_axis_title() -> String {
    t!("viz.chart.log_axis_value").into_owned()
}
const LOG_AXIS_TITLE_MARGIN_PX: usize = 20;

/// `viz smart` correlation-heatmap panel tuning. Long numeric-column names would clip against
/// the Data Schematic's left margin, so its axis tick labels are truncated to this many characters
/// and the left margin is widened (≈ this many pixels per character) to fit them. In-cell `r`
/// value labels are only drawn when the matrix is small enough to stay legible in one cell.
const CORR_LABEL_MAX_CHARS: usize = 16;
const CORR_LABEL_PX_PER_CHAR: usize = 7;
const CORR_INCELL_MAX_N: usize = 8;

/// `viz heatmap --x/--y/--z` (pivot) cell-count safety cap. The pivot's `z` is a DENSE
/// `y_cats × x_cats` matrix, so its cost is quadratic in the cardinality of the two key columns —
/// two 50k-cardinality columns would be 2.5e9 cells (~20 GB). Every other multi-category builder
/// here is bounded (`HIER_MAX_NODES`, `PARCATS_MAX_PATHS`, `CATEGORICAL_MAX_CARDINALITY`); this is
/// the equivalent guard, checked before the matrix is allocated. 1M cells is ~8 MB of `f64` and
/// already a very large heatmap to render.
const HEATMAP_PIVOT_MAX_CELLS: usize = 1_000_000;

/// `viz smart --bivariate` column-count safety cap. The association heatmap/top-relationships
/// panels are built from moarstats' ALL-PAIRS bivariate sidecar (50 choose 2 = 1,225 pairs at the
/// cap), so a wide dataset is skipped (with a warning suggesting `qsv moarstats --bivariate`
/// directly) rather than handing moarstats a combinatorially explosive pair count.
const BIVARIATE_MAX_COLUMNS: usize = 50;

/// `viz smart --bivariate`: a numeric pair's drill-down is flagged "nonlinear" in the association
/// heatmap/top-relationships hover when its Spearman |rho| exceeds its Pearson |r| by at least
/// this much — mirroring `SMART_NONLINEAR_MIN_GAP`'s correlation-pair nonlinearity note, applied
/// here per-cell across every bivariate pair instead of just the single strongest one.
const NONLINEAR_DIVERGENCE_THRESHOLD: f64 = 0.15;

/// `viz smart --bivariate`'s "Top Relationships" ranking requires a pair's co-occurring row count
/// (moarstats' `n_pairs`) to be at least this fraction of the best-supported pair's `n_pairs`
/// among the chartable columns. NMI has no notion of sample size: two columns that are each
/// populated for only a narrow, mostly-disjoint-from-everything-else slice of rows (e.g. taxi- or
/// bridge-specific fields in a 311 log) can co-occur perfectly in that sliver and score NMI=1.0,
/// outranking a moderate-NMI pair backed by the whole dataset. This ratio only gates the RANKED
/// list, not the association heatmap itself, which still shows every survivor pair for a full
/// overview.
const BIVARIATE_MIN_SUPPORT_RATIO: f64 = 0.10;

/// `viz smart --bivariate`'s "Top Relationships" panel is a horizontal multivariate lollipop
/// (dot position = NMI, dot size = co-occurrence support, dot color = nonlinear flag). The value
/// (NMI) axis is ZOOMED to fit the shown pairs rather than anchored at 0, because the strongest
/// associations cluster tightly near 1.0 where a zero-based axis crushes them into visually
/// identical positions (the very complaint this panel had). A dot plot — unlike a bar — carries no
/// area, so a non-zero baseline is legitimate. The zoom pads each end by `TOPREL_PAD_FRAC` of the
/// data span (but at least `TOPREL_MIN_PAD` in absolute terms, so a near-tied cluster like
/// 0.995..1.0 still gets breathing room and its dots separate), caps the top at `TOPREL_CEIL_CAP`
/// (NMI's ceiling is 1.0; the small overshoot leaves marker/label room), and enforces only a tiny
/// `TOPREL_MIN_SPAN` so an all-EXACTLY-tied top-N can't produce a zero-width axis (identical values
/// legitimately overlap). Category (pair) labels are truncated to `TOPREL_LABEL_MAX_CHARS` on the
/// y-axis (the full `"FieldA × FieldB"` rides in the hover) to bound the shared page left-margin.
const TOPREL_PAD_FRAC: f64 = 0.25;
const TOPREL_MIN_PAD: f64 = 0.01;
const TOPREL_MIN_SPAN: f64 = 0.02;
const TOPREL_CEIL_CAP: f64 = 1.02;
const TOPREL_LABEL_MAX_CHARS: usize = 56;

/// Marker colors for the "Top Relationships" lollipop dots. Because color here ENCODES DATA (the
/// nonlinear flag), the regular color is FIXED (the palette's blue) rather than the panel's
/// rotating slot color — otherwise the amber nonlinear flag could land next to a near-identical
/// yellow/orange palette slot and vanish. Blue-vs-amber is also a strongly colorblind-safe pairing.
const REGULAR_MARKER_COLOR: &str = PALETTE[0];
const NONLINEAR_MARKER_COLOR: &str = "#f9a825";

/// `viz smart` frequency-bar panel tuning. Very long category names (e.g. full agency names or
/// datetime strings) rotate into tall x-axis tick labels that squeeze the plot area, clipping
/// the outside value labels at the top of the cell. Displayed tick labels are truncated to this
/// many characters (the full value is preserved on hover) to keep the plot area tall enough.
const BAR_LABEL_MAX_CHARS: usize = 20;

/// Default labels for the aggregate frequency-bar buckets in `viz smart`, mirroring `qsv
/// frequency`'s defaults: the `(NULL)` bar collects empty cells and `Other (N)` collects the
/// categories beyond `--limit` (N = the count of those distinct categories). Suppressed by
/// `--no-nulls` / `--no-other` respectively.
///
/// Functions rather than `const`s because both are localized. For the frequency bars these are
/// purely DISPLAY text: `FreqBar::x_key` carries axis identity (via `AGG_KEY_SENTINEL`, extended
/// until provably unique) and `FreqBar::kind` carries the aggregate/category distinction, so
/// translating cannot change how bars collide or how they are colored.
///
/// `null_text` is ALSO used as a real grouping key in the hierarchy and parcats paths, where it
/// gets no sentinel. There, a category whose literal value equals the translated text merges into
/// the null bucket -- exactly as a category literally named "(NULL)" does in English today. The
/// trigger string changes with the locale; the behavior does not.
fn null_text() -> String {
    t!("viz.chart.null_bucket").into_owned()
}

/// The `Other (N)` bucket label. The count is part of the localized string rather than appended
/// to it, so a translation can put it wherever its grammar wants.
fn other_text(distinct: u64) -> String {
    t!(
        "viz.chart.other_bucket",
        q_count = HumanCount(distinct).to_string()
    )
    .into_owned()
}

/// Muted grey for the aggregate `(NULL)` / `Other (N)` frequency bars so they read as summary
/// buckets, visually distinct from the palette-colored real categories.
const MUTED_COLOR: &str = "#999999";
/// Bridge delta colours (issue #4222). A bridge's step bars are the only place in the Data
/// Schematic where a bar's SIGN is the message, so they get their own semantics rather than the
/// panel hue: a shortfall against the previous stage and an overrun beyond it must not look alike.
const BRIDGE_DOWN_COLOR: &str = "#c0504d";
const BRIDGE_UP_COLOR: &str = "#4f8a5b";

/// Faint grey wash used as the fill of a box/violin distribution panel when it renders on a
/// log value axis (see `log_distribution_body_cue`).
const LOG_DISTRIB_FILL: &str = "rgba(153, 153, 153, 0.15)";

/// Zero-width space suffixed onto the plotly category-axis key of the synthetic aggregate
/// frequency bars. It keeps their `x_key` distinct from a real category that happens to share
/// their display label (so the two never collapse onto the same plotly bar) while staying
/// invisible in the hovered key.
const AGG_KEY_SENTINEL: char = '\u{200B}';

/// One bar in a `viz smart` frequency panel.
#[derive(Clone)]
struct FreqBar {
    /// The plotly category-axis key — distinct within the panel so bars never collapse onto
    /// the same category slot. Real categories use their (trimmed) value; the synthetic
    /// aggregate buckets append `AGG_KEY_SENTINEL` so they can't collide with a real category
    /// that shares their display label.
    x_key: String,
    /// Friendly display label, used for the x-axis tick text (and the hovered category for
    /// real categories).
    label: String,
    count: u64,
    /// Whether this is a real category or a synthetic aggregate (NULL / Other) bucket. This —
    /// not the label — drives the bar color, so a real category literally named "(NULL)" or
    /// "Other (5)" is never mis-colored as an aggregate.
    kind:  FreqBarKind,
}

#[derive(Clone, Copy, PartialEq, Eq, Debug)]
enum FreqBarKind {
    Category,
    Aggregate,
}

/// Per-column frequency bars for the `viz smart` panels, keyed by column index.
type FreqMap = HashMap<usize, Vec<FreqBar>>;

#[derive(Deserialize, Clone)]
struct Args {
    cmd_smart:               bool,
    cmd_bar:                 bool,
    cmd_line:                bool,
    cmd_scatter:             bool,
    cmd_scatter3d:           bool,
    cmd_histogram:           bool,
    cmd_box:                 bool,
    cmd_violin:              bool,
    cmd_pie:                 bool,
    cmd_funnel:              bool,
    cmd_heatmap:             bool,
    cmd_contour:             bool,
    cmd_candlestick:         bool,
    cmd_ohlc:                bool,
    cmd_sankey:              bool,
    cmd_radar:               bool,
    cmd_map:                 bool,
    cmd_geo:                 bool,
    cmd_treemap:             bool,
    cmd_sunburst:            bool,
    cmd_icicle:              bool,
    cmd_choropleth:          bool,
    cmd_splom:               bool,
    cmd_parcats:             bool,
    arg_input:               Option<String>,
    flag_x:                  Option<SelectColumns>,
    flag_y:                  Option<SelectColumns>,
    flag_z:                  Option<SelectColumns>,
    flag_cols:               Option<SelectColumns>,
    flag_series:             Option<SelectColumns>,
    flag_donut:              bool,
    // scatter encodings: map a numeric column to per-point marker color (continuous
    // colorscale) and/or marker size (bubble chart). Mutually exclusive with --series.
    flag_color:              Option<SelectColumns>,
    flag_size:               Option<SelectColumns>,
    // candlestick / ohlc columns (--open is already taken by the browser-open flag below,
    // so the open-price column is selected with --ohlc-open)
    flag_ohlc_open:          Option<SelectColumns>,
    flag_high:               Option<SelectColumns>,
    flag_low:                Option<SelectColumns>,
    flag_close:              Option<SelectColumns>,
    // sankey columns
    flag_source:             Option<SelectColumns>,
    flag_target:             Option<SelectColumns>,
    flag_value:              Option<SelectColumns>,
    flag_sankey_value_order: bool,
    // map columns/options
    flag_lat:                Option<SelectColumns>,
    flag_lon:                Option<SelectColumns>,
    flag_text:               Option<SelectColumns>,
    flag_density:            bool,
    flag_style:              Option<String>,
    flag_projection:         Option<String>,
    // choropleth columns/options
    flag_locations:          Option<SelectColumns>,
    flag_location_mode:      Option<String>,
    flag_color_scale:        Option<String>,
    flag_map:                bool,
    flag_geojson:            Option<String>,
    flag_feature_id_key:     Option<String>,
    flag_feature_name_key:   Option<String>,
    flag_geocode:            bool,
    flag_no_snap:            bool,
    flag_snap_max_dist:      Option<f64>,
    flag_bins:               Option<usize>,
    flag_agg:                Option<String>,
    flag_box_points:         Option<String>,
    flag_max_charts:         usize,
    flag_grid_cols:          usize,
    flag_preview_threshold:  usize,
    flag_heatmap_density:    usize,
    flag_cluster:            String,
    flag_photos:             bool,
    flag_limit:              usize,
    flag_no_nulls:           bool,
    flag_no_other:           bool,
    flag_smarter:            bool,
    flag_bivariate:          bool,
    flag_hierarchy_style:    Option<String>,
    flag_dictionary:         Option<String>,
    flag_dictionary_context: Option<String>,
    flag_dict_info:          bool,
    flag_dataset_pid:        Option<String>,
    flag_log_scale:          String,
    flag_violin:             String,
    flag_title:              Option<String>,
    flag_x_title:            Option<String>,
    flag_y_title:            Option<String>,
    flag_y_range:            Option<String>,
    flag_rangeslider:        bool,
    flag_slider:             Option<String>,
    flag_slider_speed:       u64,
    flag_slider_cumulative:  bool,
    flag_annotation:         Option<String>,
    flag_theme:              Option<String>,
    // Data Schematic UI language. "auto" (the default) means "follow the data dictionary's
    // detected language, else English"; an explicit value always wins. Resolved once in
    // run() via viz_i18n::resolve.
    flag_language:           Option<String>,
    // width/height/scale only affect static image export (the viz_static feature). width
    // and height are optional: when unset, `viz smart` derives them from its grid shape and
    // other charts fall back to the defaults below.
    flag_width:              Option<usize>,
    flag_height:             Option<usize>,
    #[cfg_attr(not(feature = "viz_static"), allow(dead_code))]
    flag_scale:              f64,
    flag_open:               bool,
    flag_output:             Option<String>,
    flag_delimiter:          Option<Delimiter>,
    flag_no_headers:         bool,
}

/// The chart image format, derived from the --output extension.
#[derive(Clone, Copy, PartialEq, Eq)]
enum OutFormat {
    Html,
    Png,
    Jpeg,
    Webp,
    Svg,
    Pdf,
}

impl OutFormat {
    fn from_output(path: &str) -> Option<Self> {
        let ext = std::path::Path::new(path)
            .extension()
            .and_then(|e| e.to_str())
            .map(str::to_ascii_lowercase)
            .unwrap_or_default();
        match ext.as_str() {
            "html" | "htm" => Some(Self::Html),
            "png" => Some(Self::Png),
            "jpeg" | "jpg" => Some(Self::Jpeg),
            "webp" => Some(Self::Webp),
            "svg" => Some(Self::Svg),
            "pdf" => Some(Self::Pdf),
            _ => None,
        }
    }

    const fn is_image(self) -> bool {
        !matches!(self, Self::Html)
    }
}

pub fn run(argv: &[&str]) -> CliResult<()> {
    let mut args: Args = util::get_args(USAGE, argv)?;

    if let Some(name) = &args.flag_theme
        && parse_theme(name).is_none()
    {
        return fail_incorrectusage_clierror!(
            "Unknown --theme '{name}'. Valid themes: {VALID_THEMES}."
        );
    }

    // Validate --language up front, before any work: an explicit request we cannot honor
    // is a usage error, not something to silently anglicize. The dictionary-detected
    // language is resolved later (in the smart path, once the dictionary is parsed) since
    // it can only lower the precedence, never raise it.
    match viz_i18n::resolve(args.flag_language.as_deref(), None) {
        viz_i18n::Resolution::UnknownRequested(requested) => {
            return fail_incorrectusage_clierror!(
                "Unknown --language '{requested}'. Curated languages: {}.",
                viz_i18n::curated_list()
            );
        },
        viz_i18n::Resolution::Curated(row) => viz_i18n::set_active(row),
        // Unreachable with `detected: None`, but resolving rather than ignoring keeps the
        // match exhaustive if the dictionary language is ever folded in here.
        viz_i18n::Resolution::UncuratedDetected(_) => {},
    }

    // --snap-max-dist semantics are shared by the `viz choropleth` command and the `viz smart`
    // GeoJSON choropleth panel, so the universal constraints are validated here — before dispatch —
    // rather than only on the command path. Otherwise the smart path would silently clamp
    // negatives, accept a non-finite NaN/inf, or ignore the --no-snap conflict. The "only
    // --y-range is purely syntactic, so validate it HERE rather than in build_layout, which every
    // cartesian chart reaches only after its trace builder has already streamed the whole input:
    // `--y-range 5:1` on a large file otherwise aggregates for minutes before rejecting the flag.
    // The later call sites keep parsing it (cheap, and infallible once this has passed).
    if let Some(spec) = args.flag_y_range.as_deref() {
        parse_y_range(spec)?;
    }

    // applies to point-in-polygon binning" guard stays path-specific (build_choropleth_plot).
    if let Some(km) = args.flag_snap_max_dist {
        if args.flag_no_snap {
            return fail_incorrectusage_clierror!(
                "--snap-max-dist and --no-snap are mutually exclusive: --no-snap drops every \
                 point outside a region, while --snap-max-dist caps how far an outside point may \
                 snap."
            );
        }
        if !(km.is_finite() && km >= 0.0) {
            return fail_incorrectusage_clierror!(
                "--snap-max-dist must be a non-negative number of kilometers."
            );
        }
    }

    // The parsed+sanitized --geojson document, loaded ONCE here and threaded into whichever
    // plotting path consumes it (see `resolve_and_validate_geojson`).
    let mut loaded_geojson: Option<serde_json::Value> = None;
    // Resolve --geojson (a direct path/URL, or a QSV_GEOJSON_SHORTCUTS alias) and validate it
    // up front — fail fast on a bad source, unknown shortcut, or unusable feature-id-key before
    // any plotting work. Only the choropleth & smart subcommands consume --geojson.
    if args.flag_geojson.is_some() && (args.cmd_choropleth || args.cmd_smart) {
        // Detect whether --feature-id-key was explicitly passed (docopt fills its `id` default, so
        // the parsed value alone can't tell an explicit `--feature-id-key id` from the default).
        // Scan argv so an explicit key — including `--feature-id-key id` — always wins over a
        // shortcut's id.
        let feature_id_key_explicit = argv
            .iter()
            .any(|a| *a == "--feature-id-key" || a.starts_with("--feature-id-key="));
        loaded_geojson = resolve_and_validate_geojson(&mut args, feature_id_key_explicit)?;
    }

    let out_format = match args.flag_output.as_deref() {
        Some(path) => match OutFormat::from_output(path) {
            Some(fmt) => fmt,
            None => {
                return fail_incorrectusage_clierror!(
                    "Unsupported output extension for '{path}'. Use one of: html, png, jpeg, \
                     webp, svg, pdf."
                );
            },
        },
        None => OutFormat::Html,
    };

    #[cfg(not(feature = "viz_static"))]
    if out_format.is_image() {
        return fail_clierror!(
            "Static image export requires a qsv build with the `viz_static` feature (or a \
             prebuilt qsv binary). Output an .html file instead, or rebuild with `--features \
             viz_static`."
        );
    }

    // viz shows a progress spinner on stderr by default; QSV_PROGRESSBAR can disable it. A
    // recognized-falsy value turns it off; anything else (incl. unset) leaves it on. (Can't reuse
    // util::get_envvar_flag — that treats unset as false, i.e. default-OFF.) indicatif itself
    // hides the spinner when stderr isn't a terminal, so piped/redirected runs stay clean.
    let show_progress = match std::env::var("QSV_PROGRESSBAR") {
        Ok(v) => !matches!(
            v.trim().to_ascii_lowercase().as_str(),
            "" | "false" | "f" | "0" | "no" | "n" | "off"
        ),
        Err(_) => true,
    };
    let progress = viz_progress(show_progress);
    // register the bar so viz_note() can suspend it from anywhere in the call tree (ProgressBar is
    // Arc-backed, so the clone is cheap and shares the same underlying bar).
    let _ = VIZ_PROGRESS.set(progress.clone());
    // clear the spinner on EVERY exit path, incl. the `?` error returns below, so an error message
    // is never printed over a still-animating bar (the explicit finish_and_clear()s before the
    // success outputs stay — clearing twice is harmless).
    let _progress_guard = ProgressGuard(progress.clone());

    let (mut plot, smart_dims) = if args.cmd_smart {
        match build_smart(
            &args,
            out_format,
            &progress,
            show_progress,
            loaded_geojson.as_ref(),
        )? {
            // >8-panel HTML dashboards are assembled as an inline-div grid that bypasses the
            // single-`Plot` output path entirely.
            SmartRender::Inline(html) => {
                progress.finish_and_clear();
                return output_inline_html(&html, &args);
            },
            // >8-panel static image export is a raw Plotly JSON value (with xaxis9+/yaxis9+ that
            // the typed `Plot` can't hold), rendered directly via the static exporter.
            SmartRender::GridJson { value, dims } => {
                let img_width = args.flag_width.unwrap_or(dims.0);
                let img_height = args.flag_height.unwrap_or(dims.1);
                progress.set_message("Exporting image…");
                progress.finish_and_clear();
                return output_image_json(
                    &value,
                    &args,
                    out_format,
                    img_width,
                    img_height,
                    args.flag_scale,
                );
            },
            SmartRender::Grid {
                plot,
                dims,
                title,
                theme,
                dict_page,
                metadata,
                data_chrome,
            } => {
                // HTML smart-grid is wrapped in qsv's own page so it gets the light/dark toggle;
                // plotly's `to_html()` (used by the generic single-`Plot` path) has no injection
                // point. Static image export keeps the typed `Plot` path below.
                if matches!(out_format, OutFormat::Html) {
                    let html = render_smart_grid_page(
                        *plot,
                        theme,
                        &title,
                        dict_page.as_deref(),
                        metadata.as_deref(),
                        data_chrome.as_deref(),
                    );
                    progress.finish_and_clear();
                    return output_inline_html(&html, &args);
                }
                (plot, Some(dims))
            },
        }
    } else {
        (
            Box::new(build_plot(&args, out_format, &progress, loaded_geojson)?),
            None,
        )
    };

    // make the interactive HTML re-fit its width to the window/container on resize; this
    // is ignored by static image export (which sizes via --width/--height).
    plot.set_configuration(Configuration::new().responsive(true));

    // resolve static-export dimensions: an explicit --width/--height always wins, else the
    // `smart` auto-scaled size, else the fixed fallback.
    let (smart_w, smart_h) = smart_dims.unzip();
    let img_width = args.flag_width.or(smart_w).unwrap_or(DEFAULT_IMG_WIDTH);
    let img_height = args.flag_height.or(smart_h).unwrap_or(DEFAULT_IMG_HEIGHT);

    progress.finish_and_clear();
    output_plot(
        &plot,
        &args,
        out_format,
        img_width,
        img_height,
        args.flag_scale,
    )
}

/// The ways `viz smart` can render: a single-`Plot` typed subplot grid (up to `MAX_SUBPLOTS`
/// panels; supports static image export), a raw-JSON subplot grid with domain-positioned axes
/// (for static image export of >`MAX_SUBPLOTS` panels, where plotly's typed `Layout` runs out
/// of axis fields), or a self-contained inline-div HTML page (for >8-panel HTML dashboards).
enum SmartRender {
    // `Plot` is large; box it so the enum isn't bloated by the rarely-larger variant.
    Grid {
        plot:        Box<Plot>,
        dims:        (usize, usize),
        // carried out so the HTML path can wrap the plot in qsv's own toggle-enabled page
        // (`render_smart_grid_page`); the image-export path ignores all three.
        title:       String,
        theme:       Option<BuiltinTheme>,
        /// The embedded `--dict-info` Data Dictionary document (HTML output only).
        dict_page:   Option<String>,
        /// The top-of-Data Schematic metadata table (HTML output only).
        metadata:    Option<String>,
        /// The data viewer drawer chrome (issue #4283; HTML output only).
        data_chrome: Option<String>,
    },
    /// A fully-assembled Plotly JSON value (data + layout). Used only for static image export of
    /// more than `MAX_SUBPLOTS` panels: the layout carries `xaxis9+`/`yaxis9+`, which the typed
    /// `Layout` can't express, so it's rendered via `StaticExporter::write_fig` (raw JSON).
    GridJson {
        value: Box<serde_json::Value>,
        dims:  (usize, usize),
    },
    Inline(String),
}

/// Write a pre-assembled inline-div Data Schematic HTML string to `--output` (or stdout), honoring
/// `--open`. When `--open` is set without `--output`, the HTML is also written to a securely
/// created temporary file which is then opened — mirroring plotly's own `Plot::show()` for the
/// single-`Plot` path.
fn output_inline_html(html: &str, args: &Args) -> CliResult<()> {
    match args.flag_output.as_deref() {
        Some(path) => {
            std::fs::write(path, html)?;
            if args.flag_open {
                open_path(path)?;
            }
        },
        None => {
            std::io::stdout().write_all(html.as_bytes())?;
            if args.flag_open {
                // Create the temp file via tempfile (random name, O_EXCL) to avoid a symlink
                // attack on a predictable path, then persist it so the browser can read it
                // after qsv exits (a NamedTempFile would otherwise delete on drop).
                let mut tmp = tempfile::Builder::new()
                    .prefix("qsv-viz-smart-")
                    .suffix(".html")
                    .tempfile()?;
                tmp.write_all(html.as_bytes())?;
                let (_file, path) = tmp.keep().map_err(|e| {
                    crate::CliError::Other(format!(
                        "Could not persist temp Data Schematic file: {e}"
                    ))
                })?;
                open_path(&path.to_string_lossy())?;
            }
        },
    }
    Ok(())
}

/// Build a `Plot` for the requested chart subcommand.
fn build_plot(
    args: &Args,
    out_format: OutFormat,
    progress: &ProgressBar,
    geojson: Option<serde_json::Value>,
) -> CliResult<Plot> {
    progress.set_message("Building chart…");
    let kind = chart_kind(args);
    let slider_col = resolve_slider_col(args)?;
    // `viz scatter --slider <t> --series <entity> --size <measure>` is the Gapminder bubble
    // animation: --series names the entity (one bubble, one color, one trace per value) and --size
    // the third data variable. It is the one case where a marker encoding and --series coexist,
    // because each entity's trace carries exactly one point per frame.
    //
    // The `flag_color.is_none()` term is load-bearing: `encoded_scatter` is `color || size`, so
    // relaxing on `encoded_scatter` alone would also wave `--color` + `--series` past the conflict
    // check below and into the slider dispatch, which only screens for a lone `--color`.
    let bubble_anim = slider_col.is_some()
        && matches!(kind, Chart::Scatter)
        && args.flag_size.is_some()
        && args.flag_color.is_none()
        && args.flag_series.is_some();
    // --color/--size are per-point marker encodings that apply to scatter and map only, and
    // need a single trace, so they can't be combined with --series (which splits into traces).
    if encoded_scatter(args) {
        if !matches!(
            kind,
            Chart::Scatter | Chart::Scatter3D | Chart::Map | Chart::Geo
        ) {
            return fail_incorrectusage_clierror!(
                "--color/--size only apply to `viz scatter`, `viz scatter3d`, `viz map` and `viz \
                 geo`."
            );
        }
        if args.flag_series.is_some() && !bubble_anim {
            return fail_incorrectusage_clierror!(
                "--color/--size cannot be combined with --series. Use --series to split into \
                 colored traces by category, or --color/--size to encode numeric columns onto a \
                 single series.\n(The one exception is the bubble animation: `viz scatter \
                 --slider <col> --series <entity> --size <measure>`.)"
            );
        }
    }

    // --rangeslider adds an x-axis navigator strip; it only makes sense on cartesian charts with a
    // real x-axis (ordered x especially: line, candlestick/ohlc), so reject it on non-cartesian
    // kinds (map/geo/choropleth/pie/sankey/radar/hierarchy/splom/parcats/scatter3d/heatmap/
    // contour).
    if args.flag_rangeslider
        && !matches!(
            chart_kind(args),
            Chart::Bar
                | Chart::Line
                | Chart::Scatter
                | Chart::Histogram
                | Chart::Box
                | Chart::Violin
                | Chart::Candlestick
                | Chart::Ohlc
        )
    {
        return fail_incorrectusage_clierror!(
            "--rangeslider only applies to cartesian charts (bar, line, scatter, histogram, box, \
             violin, candlestick, ohlc)."
        );
    }

    // --slider animates the chart over a column's distinct values (one animation frame per value,
    // with a Play/Pause button and a scrub slider). It partitions the rows and builds its own
    // frames + slider/menu layout, so it's dispatched here before the per-kind builders below.
    // Supported for bar/line/scatter and geo; histogram/box are follow-ups, and `viz map` can't
    // animate (the MapLibre backend never settles its animation frames — use `viz geo` instead).
    if let Some(slider_col) = slider_col {
        match kind {
            Chart::Bar | Chart::Line | Chart::Scatter => {
                if bubble_anim {
                    return build_slider_bubble_plot(args, &slider_col);
                }
                if args.flag_color.is_some() {
                    return fail_incorrectusage_clierror!(
                        "--slider cannot yet be combined with --color (animating a continuous \
                         colorscale is a follow-up). Use --series to split into animated traces, \
                         or drop --color."
                    );
                }
                if args.flag_size.is_some() {
                    return fail_incorrectusage_clierror!(
                        "--slider with --size builds a Gapminder bubble animation, which needs an \
                         entity to put a bubble on. Add --series <col> (one bubble per distinct \
                         value), or drop --size."
                    );
                }
                return build_slider_xy_plot(args, kind, &slider_col);
            },
            Chart::Geo => {
                if encoded_scatter(args) {
                    return fail_incorrectusage_clierror!(
                        "--slider cannot yet be combined with --color/--size on `viz geo` (a \
                         follow-up). Use --series to split into animated traces, or drop \
                         --color/--size."
                    );
                }
                return build_slider_geo_plot(args, &slider_col);
            },
            Chart::Map => {
                return fail_incorrectusage_clierror!(
                    "--slider isn't supported for `viz map` — the MapLibre tile basemap can't \
                     animate reliably. Use `viz geo` for an animated point map."
                );
            },
            _ => {
                return fail_incorrectusage_clierror!(
                    "--slider currently supports `viz bar`, `viz line`, `viz scatter` and `viz \
                     geo`."
                );
            },
        }
    }

    // maps use a `map` layout (tile basemap, center, zoom) rather than cartesian x/y axes,
    // so they own their whole `Plot` and bypass the cartesian `build_layout` below.
    if matches!(chart_kind(args), Chart::Map) {
        return build_map_plot(args, out_format);
    }

    // `geo` uses a `geo` layout (projection basemap) and `scatter3d` a `scene` layout (3D
    // axes); neither is cartesian, so each owns its whole `Plot` like the map above.
    if matches!(chart_kind(args), Chart::Geo) {
        return build_geo_plot(args);
    }

    // choropleth fills whole regions on a `geo` (projection) or `map` (MapLibre) subplot — never
    // cartesian — so it owns its whole `Plot` like the maps above.
    if matches!(chart_kind(args), Chart::Choropleth) {
        return build_choropleth_plot(args, out_format, geojson);
    }
    if matches!(chart_kind(args), Chart::Scatter3D) {
        return build_scatter3d_plot(args);
    }

    // treemap / sunburst are domain-based (no cartesian x/y axes), like pie — they own their whole
    // `Plot` as well.
    if matches!(
        chart_kind(args),
        Chart::Treemap | Chart::Sunburst | Chart::Icicle
    ) {
        return build_hierarchy_plot(args);
    }

    // splom (scatter-plot matrix) self-generates its own N×N axis grid and parcats (parallel
    // categories) is domain-based — neither is cartesian, so each owns its whole `Plot`.
    if matches!(chart_kind(args), Chart::Splom) {
        return build_splom_plot(args);
    }
    if matches!(chart_kind(args), Chart::Parcats) {
        return build_parcats_plot(args);
    }

    let mut plot = Plot::new();

    // the sankey chart carries an on-screen "node order" toggle (a layout updatemenu), stashed
    // here from the trace builder and folded into the layout after it's constructed below.
    let mut sankey_order_menu: Option<UpdateMenu> = None;

    // default axis titles, derived from the selected column names
    let (default_x, default_y): (Option<String>, Option<String>) = match chart_kind(args) {
        Chart::Histogram => {
            let (trace, x_label) = build_histogram(args)?;
            plot.add_trace(trace);
            (Some(x_label), Some(t!("viz.chart.count").into_owned()))
        },
        Chart::Box => {
            let (trace, y_label, x_label) = build_box(args)?;
            plot.add_trace(trace);
            (x_label, Some(y_label))
        },
        Chart::Violin => {
            let (trace, y_label, x_label) = build_violin(args)?;
            plot.add_trace(trace);
            (x_label, Some(y_label))
        },
        // pie / sankey / radar are not cartesian, so they get no x/y axis titles
        Chart::Pie => {
            plot.add_trace(build_pie(args)?);
            (None, None)
        },
        // A funnel is cartesian, but plotly draws no visible value axis for it -- each band
        // carries its own value and conversion label instead. An axis title here would be dead
        // config that never renders, so it gets none, like the domain-based traces above.
        Chart::Funnel => {
            plot.add_trace(build_funnel_chart(args)?);
            (None, None)
        },
        Chart::Sankey => {
            let (trace, menu) = build_sankey(args)?;
            plot.add_trace(trace);
            sankey_order_menu = menu;
            (None, None)
        },
        Chart::Radar => {
            for trace in build_radar(args)? {
                plot.add_trace(trace);
            }
            (None, None)
        },
        Chart::Heatmap => {
            let (trace, x_label, y_label) = build_heatmap(args)?;
            plot.add_trace(trace);
            (x_label, y_label)
        },
        Chart::Contour => {
            let (trace, x_label, y_label) = build_contour(args)?;
            plot.add_trace(trace);
            (Some(x_label), Some(y_label))
        },
        kind @ (Chart::Candlestick | Chart::Ohlc) => {
            let (trace, x_label, y_label) = build_candlestick(args, matches!(kind, Chart::Ohlc))?;
            plot.add_trace(trace);
            (Some(x_label), Some(y_label))
        },
        // bar / line / scatter all consume (x, y) pairs, optionally split by --series
        kind => {
            let (traces, x_label, y_label) =
                if matches!(kind, Chart::Scatter) && encoded_scatter(args) {
                    build_scatter_encoded(args)?
                } else {
                    build_xy_traces(args, kind)?
                };
            for trace in traces {
                plot.add_trace(trace);
            }
            (Some(x_label), Some(y_label))
        },
    };

    let mut layout = build_layout(args, default_x, default_y)?;
    if let Some(menu) = sankey_order_menu {
        layout = layout.update_menus(vec![menu]);
    }
    // `x unified` (plotly.js) shows one tooltip per x across every trace — ideal for an ordered
    // x-axis (line trends, OHLC sessions) but wrong for unordered clouds, so it's scoped to those
    // chart kinds. The smart Data Schematic builds its own layout and never calls build_layout, so
    // this can't leak onto its heterogeneous panels.
    if matches!(
        chart_kind(args),
        Chart::Line | Chart::Candlestick | Chart::Ohlc
    ) {
        layout = layout.hover_mode(HoverMode::XUnified);
    }
    plot.set_layout(layout);
    Ok(plot)
}

/// The requested chart subcommand.
#[derive(Clone, Copy)]
enum Chart {
    Bar,
    Line,
    Scatter,
    Scatter3D,
    Histogram,
    Box,
    Violin,
    Pie,
    Funnel,
    Heatmap,
    Contour,
    Candlestick,
    Ohlc,
    Sankey,
    Radar,
    Map,
    Geo,
    Choropleth,
    Treemap,
    Sunburst,
    Icicle,
    Splom,
    Parcats,
}

fn chart_kind(args: &Args) -> Chart {
    if args.cmd_bar {
        Chart::Bar
    } else if args.cmd_line {
        Chart::Line
    } else if args.cmd_scatter {
        Chart::Scatter
    } else if args.cmd_scatter3d {
        Chart::Scatter3D
    } else if args.cmd_histogram {
        Chart::Histogram
    } else if args.cmd_box {
        Chart::Box
    } else if args.cmd_violin {
        Chart::Violin
    } else if args.cmd_pie {
        Chart::Pie
    } else if args.cmd_funnel {
        Chart::Funnel
    } else if args.cmd_heatmap {
        Chart::Heatmap
    } else if args.cmd_contour {
        Chart::Contour
    } else if args.cmd_candlestick {
        Chart::Candlestick
    } else if args.cmd_ohlc {
        Chart::Ohlc
    } else if args.cmd_sankey {
        Chart::Sankey
    } else if args.cmd_radar {
        Chart::Radar
    } else if args.cmd_map {
        Chart::Map
    } else if args.cmd_geo {
        Chart::Geo
    } else if args.cmd_choropleth {
        Chart::Choropleth
    } else if args.cmd_treemap {
        Chart::Treemap
    } else if args.cmd_sunburst {
        Chart::Sunburst
    } else if args.cmd_icicle {
        Chart::Icicle
    } else if args.cmd_splom {
        Chart::Splom
    } else if args.cmd_parcats {
        Chart::Parcats
    } else {
        unreachable!("docopt guarantees exactly one chart subcommand")
    }
}

/// A reader bundle: the configured reader, the resolved header record, and the *effective*
/// no-headers flag (which also honors the `QSV_NO_HEADERS` / `QSV_TOGGLE_HEADERS` env vars,
/// so selection and labeling stay consistent with how the reader treats the first row).
fn reader_and_headers(
    args: &Args,
) -> CliResult<(
    csv::Reader<Box<dyn std::io::Read + Send>>,
    csv::ByteRecord,
    bool,
)> {
    let rconfig = Config::new(args.arg_input.as_ref())
        .delimiter(args.flag_delimiter)
        .no_headers_flag(args.flag_no_headers);
    let no_headers = rconfig.no_headers;
    let mut rdr = rconfig.reader()?;
    let headers = rdr.byte_headers()?.clone();
    Ok((rdr, headers, no_headers))
}

/// Resolve a required single-column selector to its column index.
fn resolve_one(
    sel: Option<&SelectColumns>,
    headers: &csv::ByteRecord,
    no_headers: bool,
    flag: &str,
) -> CliResult<usize> {
    let Some(sel) = sel else {
        return fail_incorrectusage_clierror!("--{flag} is required for this chart type.");
    };
    let selection = sel
        .selection(headers, !no_headers)
        .map_err(crate::CliError::Other)?;
    if selection.len() != 1 {
        return fail_incorrectusage_clierror!(
            "--{flag} must select exactly one column (it selected {}).",
            selection.len()
        );
    }
    Ok(selection[0])
}

/// The label to use for a column index (its header name, or 1-based index when --no-headers).
fn col_label(headers: &csv::ByteRecord, idx: usize, no_headers: bool) -> String {
    if no_headers {
        format!("col {}", idx + 1)
    } else {
        String::from_utf8_lossy(&headers[idx]).into_owned()
    }
}

/// Read (series, x, y) rows. `y` is parsed as f64; rows whose y is empty or non-numeric
/// are skipped. Returns the rows grouped into series traces (one unnamed group when
/// `series_idx` is None), preserving column labels for axis titles.
struct XyData {
    x_label: String,
    y_label: String,
    /// (series name, xs, ys) — series name is empty for the no-series case.
    groups:  Vec<(String, Vec<String>, Vec<f64>)>,
}

fn read_xy(args: &Args) -> CliResult<XyData> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let x_idx = resolve_one(args.flag_x.as_ref(), &headers, nh, "x")?;
    let y_idx = resolve_one(args.flag_y.as_ref(), &headers, nh, "y")?;
    let series_idx = match args.flag_series.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "series")?),
        None => None,
    };

    // preserve series insertion order
    let mut order: Vec<String> = Vec::new();
    let mut grouped: BTreeMap<String, (Vec<String>, Vec<f64>)> = BTreeMap::new();

    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let Some(y) = parse_f64(record.get(y_idx)) else {
            continue;
        };
        let x = cell_to_string(record.get(x_idx));
        let series = match series_idx {
            Some(i) => cell_to_string(record.get(i)),
            None => String::new(),
        };
        // probe before cloning: `entry` takes an owned key, so the plain form allocates a second
        // String on EVERY row, not just when the group is new.
        let entry = if let Some(e) = grouped.get_mut(&series) {
            e
        } else {
            order.push(series.clone());
            grouped.entry(series).or_insert((Vec::new(), Vec::new()))
        };
        entry.0.push(x);
        entry.1.push(y);
    }

    let groups = order
        .into_iter()
        .map(|name| {
            let (xs, ys) = grouped.remove(&name).unwrap_or_default();
            (name, xs, ys)
        })
        .collect();

    Ok(XyData {
        x_label: col_label(&headers, x_idx, nh),
        y_label: col_label(&headers, y_idx, nh),
        groups,
    })
}

fn build_xy_traces(args: &Args, kind: Chart) -> CliResult<(Vec<Box<dyn Trace>>, String, String)> {
    let data = read_xy(args)?;
    let agg = parse_agg(args.flag_agg.as_deref())?;
    let traces = xy_groups_to_traces(data.groups, kind, agg, true, None);
    if traces.is_empty() {
        return fail_clierror!("No plottable rows found (is --y numeric?).");
    }
    Ok((traces, data.x_label, data.y_label))
}

/// Apply the per-group transform shared by the trace builder and the slider's range computation:
/// `--agg` (bar/line) collapses duplicate x, and line charts sort numeric x ascending. Keeping this
/// in one place ensures the pinned animation ranges are computed over the SAME values that are
/// actually plotted (e.g. summed bar heights, not raw rows).
fn apply_xy_transform(
    xs: Vec<String>,
    ys: Vec<f64>,
    kind: Chart,
    agg: Option<Agg>,
) -> (Vec<String>, Vec<f64>) {
    let (xs, ys) = match agg {
        Some(agg) if matches!(kind, Chart::Bar | Chart::Line) => aggregate(xs, ys, agg),
        _ => (xs, ys),
    };
    if matches!(kind, Chart::Line) {
        sort_line_xy(xs, ys)
    } else {
        (xs, ys)
    }
}

/// Turn `(series, xs, ys)` groups into bar/line/scatter traces, applying per-group `--agg`
/// (bar/line) and line x-sorting exactly like the non-animated path. `allow_bar_labels` enables the
/// SI value labels on single-series bar charts; animation frames pass `false` (per-frame labels are
/// noise, and the label-eligibility test depends on a single series which frames can't guarantee).
fn xy_groups_to_traces(
    groups: Vec<(String, Vec<String>, Vec<f64>)>,
    kind: Chart,
    agg: Option<Agg>,
    allow_bar_labels: bool,
    bar_colors: Option<&HashMap<String, &'static str>>,
) -> Vec<Box<dyn Trace>> {
    // value labels only make sense for a single-series bar chart with a modest bar count
    let single_series = groups.len() == 1;

    let mut traces: Vec<Box<dyn Trace>> = Vec::with_capacity(groups.len());
    for (name, xs, ys) in groups {
        let (xs, ys) = apply_xy_transform(xs, ys, kind, agg);
        match kind {
            Chart::Bar => {
                let show_labels = allow_bar_labels && single_series && ys.len() <= LABEL_MAX_BARS;
                // per-category bar colors (single-series only): each bar gets a distinct,
                // category-keyed color so an animated (--slider) bar chart reads as bars
                // growing/shrinking in place rather than one monochrome block whose bars
                // appear to slide between columns between frames
                let bar_marker = single_series.then_some(bar_colors).flatten().map(|cmap| {
                    let colors: Vec<&'static str> = xs
                        .iter()
                        .map(|x| cmap.get(x).copied().unwrap_or(PALETTE[0]))
                        .collect();
                    Marker::new().color_array(colors)
                });
                // stable per-bar ids (the category value) for object constancy across animation
                // frames — plotly's `ids` is the `animation_group` mechanism. WITHOUT it, plotly
                // matches bars between frames by ARRAY INDEX, so if a frame's rows arrive in a
                // different order the bar at index 0 tweens from one category's slot to another's
                // and visibly slides between columns. Keyed on the category, each bar transitions
                // in place (its category-axis slot is pinned), which is the actual fix for the
                // "bars moving between columns" illusion (color alone only made them identifiable).
                let bar_ids = bar_marker.is_some().then(|| xs.clone());
                let mut t = Bar::new(xs, ys);
                if !name.is_empty() {
                    t = t.name(escape_hover(&name));
                }
                if let Some(m) = bar_marker {
                    t = t.marker(m);
                }
                if let Some(ids) = bar_ids {
                    t = t.ids(ids);
                }
                if show_labels {
                    // SI-formatted value labels above each bar ("258k", "1.05M");
                    // clip_on_axis(false) keeps the tallest bar's label from being clipped
                    t = t
                        .text_template("%{y:.3s}")
                        .text_position(TextPosition::Outside)
                        .text_font(Font::new().size(10))
                        .clip_on_axis(false);
                }
                traces.push(t);
            },
            _ => {
                let mode = if matches!(kind, Chart::Line) {
                    Mode::Lines
                } else {
                    Mode::Markers
                };
                traces.push(scatter_trace(&name, xs, ys, mode));
            },
        }
    }
    traces
}

/// Resolve the `--slider` value for a standalone (non-`smart`) chart into an optional frame column.
/// Keyword-first, mirroring `parse_cluster_mode`: absent or `off` → no slider; `auto`/`on` are
/// `smart`-only and error here; anything else is parsed as a column selector.
fn resolve_slider_col(args: &Args) -> CliResult<Option<SelectColumns>> {
    let Some(spec) = args.flag_slider.as_deref() else {
        return Ok(None);
    };
    match spec.to_ascii_lowercase().as_str() {
        "off" => Ok(None),
        "auto" | "on" => fail_incorrectusage_clierror!(
            "--slider needs a column here (e.g. --slider year); auto/on/off only apply to `viz \
             smart`."
        ),
        _ => match SelectColumns::parse(spec) {
            Ok(sc) => Ok(Some(sc)),
            Err(e) => fail_incorrectusage_clierror!("invalid --slider column `{spec}`: {e}"),
        },
    }
}

/// Row data partitioned into ordered animation frames for the `--slider` path.
struct XySliderData {
    x_label:      String,
    y_label:      String,
    /// the `--slider` column's label, shown as the slider's current-value prefix
    frame_label:  String,
    /// distinct slider values in animation order (numeric if all parse, else lexical)
    frame_values: Vec<String>,
    /// per frame: one `(series, xs, ys)` group per `series_union` member (empty arrays for a
    /// series absent from that frame) so every frame has the SAME trace count and order. Stored
    /// RAW (untransformed) — the trace builder applies `--agg` exactly once.
    frames:       Vec<Vec<(String, Vec<String>, Vec<f64>)>>,
    /// pinned global x-range (only when every x parses as finite — a numeric axis)
    x_range:      Option<(f64, f64)>,
    /// fixed category order for a non-numeric x-axis, so categorical bars/points don't reorder or
    /// drop between frames (mutually exclusive with `x_range`)
    x_categories: Option<Vec<String>>,
    /// pinned global y-range
    y_range:      (f64, f64),
}

/// Read the rows and partition them into ordered animation frames by the `--slider` column,
/// grouping within each frame by `--series` (a single empty-named group when `--series` is unset).
/// Frame groups are stored RAW (untransformed) so the trace builder applies `--agg` exactly once;
/// the global x/y ranges (and the categorical x union) are computed from a transformed copy so they
/// reflect the actual PLOTTED values and stay pinned across every frame.
fn read_xy_slider(
    args: &Args,
    slider_col: &SelectColumns,
    kind: Chart,
    agg: Option<Agg>,
) -> CliResult<XySliderData> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let x_idx = resolve_one(args.flag_x.as_ref(), &headers, nh, "x")?;
    let y_idx = resolve_one(args.flag_y.as_ref(), &headers, nh, "y")?;
    let series_idx = match args.flag_series.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "series")?),
        None => None,
    };
    let frame_idx = resolve_one(Some(slider_col), &headers, nh, "slider")?;

    // frame value -> series -> (xs, ys); insertion order tracked separately
    let mut frame_order: Vec<String> = Vec::new();
    let mut series_order: Vec<String> = Vec::new();
    // membership index for `series_order`: a Vec::contains probe here is O(distinct series) on
    // EVERY row, which degenerates badly on a high-cardinality --series column.
    let mut series_seen: HashSet<String> = HashSet::new();
    let mut map: HashMap<String, HashMap<String, (Vec<String>, Vec<f64>)>> = HashMap::new();

    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let Some(y) = parse_f64(record.get(y_idx)) else {
            continue;
        };
        let x = cell_to_string(record.get(x_idx));
        let series = match series_idx {
            Some(i) => cell_to_string(record.get(i)),
            None => String::new(),
        };
        let frame = cell_to_string(record.get(frame_idx));

        if !map.contains_key(&frame) {
            frame_order.push(frame.clone());
        }
        let fmap = map.entry(frame).or_default();
        if !fmap.contains_key(&series) && series_seen.insert(series.clone()) {
            series_order.push(series.clone());
        }
        let entry = fmap
            .entry(series)
            .or_insert_with(|| (Vec::new(), Vec::new()));
        entry.0.push(x);
        entry.1.push(y);
    }
    if frame_order.is_empty() {
        return fail_clierror!("No plottable rows found (is --y numeric?).");
    }
    // a single frame is not an animation
    if frame_order.len() < 2 {
        return fail_incorrectusage_clierror!(
            "--slider needs at least 2 distinct values in the chosen column to animate; found 1."
        );
    }
    if frame_order.len() > SLIDER_MAX_FRAMES {
        return fail_incorrectusage_clierror!(
            "--slider column has {} distinct values; the maximum is {SLIDER_MAX_FRAMES}.\nEvery \
             frame embeds its own copy of the plotted data, so this would produce an unusably \
             large file. Bucket the column first (e.g. `qsv datefmt` to a coarser grain, or `qsv \
             apply`), or pick a lower-cardinality --slider column.",
            frame_order.len()
        );
    }

    // animation order: numeric when every frame value parses as finite, else lexical
    if frame_order.iter().all(|f| parse_finite_f64(f).is_some()) {
        frame_order.sort_by(|a, b| {
            parse_finite_f64(a)
                .unwrap_or(f64::NAN)
                .partial_cmp(&parse_finite_f64(b).unwrap_or(f64::NAN))
                .unwrap_or(std::cmp::Ordering::Equal)
        });
    } else {
        frame_order.sort();
    }

    let cumulative = args.flag_slider_cumulative;
    let mut frames: Vec<Vec<(String, Vec<String>, Vec<f64>)>> =
        Vec::with_capacity(frame_order.len());
    // ranges + the categorical x union are computed over the transformed (plotted) values, while
    // the frame groups themselves are stored RAW (the trace builder transforms them once)
    let mut ymin = f64::INFINITY;
    let mut ymax = f64::NEG_INFINITY;
    let mut all_x_numeric = true;
    let mut xmin = f64::INFINITY;
    let mut xmax = f64::NEG_INFINITY;
    let mut saw_numeric_x = false;
    let mut x_union: Vec<String> = Vec::new();
    let mut x_seen: HashSet<String> = HashSet::new();

    for fi in 0..frame_order.len() {
        let mut groups: Vec<(String, Vec<String>, Vec<f64>)> =
            Vec::with_capacity(series_order.len());
        for s in &series_order {
            let (mut xs, mut ys): (Vec<String>, Vec<f64>) = (Vec::new(), Vec::new());
            // cumulative frames accumulate every value at or below this one
            let lo = if cumulative { 0 } else { fi };
            for prev in &frame_order[lo..=fi] {
                if let Some(fm) = map.get(prev)
                    && let Some((fx, fy)) = fm.get(s)
                {
                    xs.extend_from_slice(fx);
                    ys.extend_from_slice(fy);
                }
            }
            // transform a COPY only to measure the plotted values (ranges + categorical union);
            // the RAW xs/ys are stored so the trace builder aggregates exactly once
            let (txs, tys) = apply_xy_transform(xs.clone(), ys.clone(), kind, agg);
            for &y in &tys {
                ymin = ymin.min(y);
                ymax = ymax.max(y);
            }
            for x in &txs {
                // categorical x union in first-seen order (used to pin a non-numeric axis)
                // contains-then-insert: `HashSet::insert` takes an owned key, so cloning
                // unconditionally allocates on every hit as well as every miss.
                if !x_seen.contains(x) {
                    x_seen.insert(x.clone());
                    x_union.push(x.clone());
                }
                if all_x_numeric {
                    if let Some(xn) = parse_finite_f64(x) {
                        saw_numeric_x = true;
                        xmin = xmin.min(xn);
                        xmax = xmax.max(xn);
                    } else {
                        all_x_numeric = false;
                    }
                }
            }
            groups.push((s.clone(), xs, ys));
        }
        frames.push(groups);
    }

    // pinned y-range over the plotted values (bars start at 0); a small pad keeps points off the
    // edge
    if matches!(kind, Chart::Bar) {
        ymin = ymin.min(0.0);
        ymax = ymax.max(0.0);
    }
    let yspan = (ymax - ymin).abs();
    let ypad = if yspan > f64::EPSILON {
        yspan * 0.05
    } else {
        ymax.abs().max(1.0) * 0.05
    };
    // don't pad a bar's baseline below 0
    let ylo = if matches!(kind, Chart::Bar) {
        ymin
    } else {
        ymin - ypad
    };
    let y_range = (ylo, ymax + ypad);

    // pin the x-axis one of two ways: a numeric range when every x parses, else a fixed category
    // array (order + membership) so categorical bars/points don't reorder or drop between frames
    let (x_range, x_categories) = if all_x_numeric && saw_numeric_x {
        let xspan = (xmax - xmin).abs();
        let xpad = if xspan > f64::EPSILON {
            xspan * 0.05
        } else {
            xmax.abs().max(1.0) * 0.05
        };
        (Some((xmin - xpad, xmax + xpad)), None)
    } else if !x_union.is_empty() {
        (None, Some(x_union))
    } else {
        (None, None)
    };

    Ok(XySliderData {
        x_label: col_label(&headers, x_idx, nh),
        y_label: col_label(&headers, y_idx, nh),
        frame_label: col_label(&headers, frame_idx, nh),
        frame_values: frame_order,
        frames,
        x_range,
        x_categories,
        y_range,
    })
}

/// Build an animated bar/line/scatter `Plot`: frame-0 traces as the base, one `Frame` per slider
/// value, and a layout carrying pinned axis ranges, a scrub `Slider` and a Play/Pause menu.
fn build_slider_xy_plot(args: &Args, kind: Chart, slider_col: &SelectColumns) -> CliResult<Plot> {
    let agg = parse_agg(args.flag_agg.as_deref())?;
    let mut data = read_xy_slider(args, slider_col, kind, agg)?;

    let mut plot = Plot::new();

    // For a single-series categorical bar animation, give each bar a distinct, category-keyed
    // color so the same category keeps its color across every frame. Without this, a monochrome
    // bar chart animating over --slider looks like the bars are sliding between columns instead
    // of growing/shrinking in place. Only single-series bars (no --series) with a pinned
    // categorical x-axis qualify; --series bars are already colored per series.
    let bar_colors: Option<HashMap<String, &'static str>> =
        if matches!(kind, Chart::Bar) && args.flag_series.is_none() {
            data.x_categories.as_ref().map(|cats| {
                cats.iter()
                    .enumerate()
                    .map(|(i, c)| (c.clone(), PALETTE[i % PALETTE.len()]))
                    .collect()
            })
        } else {
            None
        };

    // base = frame 0's traces; every frame carries the same trace count/order
    let base = xy_groups_to_traces(
        data.frames[0].clone(),
        kind,
        agg,
        false,
        bar_colors.as_ref(),
    );
    let n_traces = base.len();
    for t in base {
        plot.add_trace(t);
    }
    if n_traces == 0 {
        return fail_clierror!("No plottable rows found (is --y numeric?).");
    }

    // one animation frame per slider value
    // consume the frames rather than cloning each one: `xy_groups_to_traces` takes them by value,
    // and `data.frames` is dead after this loop. Under --slider-cumulative each frame already
    // holds a copy of every row from frames 0..=k, so cloning them all again doubled the peak.
    // (Frame 0 was cloned once above for the base traces; that copy is genuinely needed.)
    let frames = std::mem::take(&mut data.frames);
    for (fkey, groups) in data.frame_values.iter().zip(frames) {
        let traces = xy_groups_to_traces(groups, kind, agg, false, bar_colors.as_ref());
        let mut td = Traces::new();
        for t in traces {
            td.push(t);
        }
        let frame = Frame::new()
            .name(fkey.clone())
            .data(td)
            .traces((0..n_traces).collect());
        plot.add_frame(frame);
    }

    // layout: titles + pinned axis ranges + optional rangeslider, then the slider + Play/Pause menu
    let mut layout = Layout::new();
    if let Some(title) = &args.flag_title {
        layout = layout.title(Title::with_text(title));
    }
    // same sink as `build_layout`: the fallback is a raw CSV header, so it is escaped; an explicit
    // --x-title is operator-supplied and stays raw.
    let mut x = Axis::new().title(Title::with_text(
        args.flag_x_title
            .clone()
            .unwrap_or_else(|| escape_hover(&data.x_label)),
    ));
    if let Some((lo, hi)) = data.x_range {
        x = x.range(vec![lo, hi]);
    } else if let Some(cats) = data.x_categories {
        // pin a categorical x-axis to a fixed category array so bars/points keep the same order
        // and set of categories across every animation frame
        x = x.category_order(CategoryOrder::Array).category_array(cats);
    }
    if args.flag_rangeslider {
        x = x.range_slider(RangeSlider::new().visible(true));
    }
    layout = layout.x_axis(x);

    let mut y = Axis::new().title(Title::with_text(
        args.flag_y_title
            .clone()
            .unwrap_or_else(|| escape_hover(&data.y_label)),
    ));
    // an explicit --y-range wins over the pinned global range
    let (ylo, yhi) = if let Some(r) = &args.flag_y_range {
        parse_y_range(r)?
    } else {
        data.y_range
    };
    y = y.range(vec![ylo, yhi]);
    layout = layout.y_axis(y);

    if let Some(note) = &args.flag_annotation {
        layout = with_chart_note(layout, note);
    }

    layout = layout.sliders(vec![slider_control(
        &data.frame_label,
        &data.frame_values,
        args.flag_slider_speed,
    )?]);
    layout = layout.update_menus(vec![play_pause_menu(args.flag_slider_speed)?]);

    plot.set_layout(apply_theme(layout, args.theme()));
    Ok(plot)
}

/// Cap on the number of `--series` entities a bubble animation will draw. Every entity is its own
/// trace IN EVERY FRAME, so the embedded payload grows as entities x frames — an unbounded
/// high-cardinality `--series` would produce an unusably large file rather than a readable chart.
const BUBBLE_MAX_ENTITIES: usize = 60;

/// Build a Gapminder-style animated bubble chart: `--series` is the entity (one colored bubble
/// each), `--x`/`--y` the measure pair, `--size` the third data variable, and `--slider` the time
/// (or ordinal) axis. Each `(entity, frame)` cell collapses to ONE bubble via `--agg` (default
/// mean).
///
/// This is the standalone twin of the `viz smart` bubble panel and shares its reader
/// ([`read_entity_bucket_agg`]) and its trace/frame builder ([`add_bubble_traces_and_frames`]);
/// only the gates and the layout differ. Because the user asked for this chart by name, the
/// quality gates that make sense for an AUTO-selected panel are relaxed here and reported as
/// stderr notes instead — see [`EntityBucketOpts`].
fn build_slider_bubble_plot(args: &Args, slider_col: &SelectColumns) -> CliResult<Plot> {
    let (_rdr, headers, nh) = reader_and_headers(args)?;
    let x_idx = resolve_one(args.flag_x.as_ref(), &headers, nh, "x")?;
    let y_idx = resolve_one(args.flag_y.as_ref(), &headers, nh, "y")?;
    let entity_idx = resolve_one(args.flag_series.as_ref(), &headers, nh, "series")?;
    // `--size` is REQUIRED here, guaranteed by the `bubble_anim` dispatch predicate in
    // `build_plot`. (The reader's size-less fallback to a per-cell row count exists for `viz
    // smart`, which reaches this reader on a 2-numeric dataset with no third measure; it is not
    // reachable from the standalone path, so this must not advertise it.)
    let size_idx = resolve_one(args.flag_size.as_ref(), &headers, nh, "size")?;
    let frame_idx = resolve_one(Some(slider_col), &headers, nh, "slider")?;

    // `--agg count` would make the bubble's POSITION a row count, which is meaningless
    let agg = match parse_agg(args.flag_agg.as_deref())? {
        Some(Agg::Count) => {
            return fail_incorrectusage_clierror!(
                "--agg count has no meaning for a bubble's x/y position. Use sum, mean, min or \
                 max."
            );
        },
        Some(a) => a,
        // mean = the centroid the `viz smart` bubble panel has always drawn
        None => Agg::Mean,
    };

    let opts = EntityBucketOpts {
        size_idx: Some(size_idx),
        agg,
        cumulative: args.flag_slider_cumulative,
        prefer_dmy: util::get_envvar_flag("QSV_PREFER_DMY"),
        min_frames: 2,
        // already enforced upstream by `resolve_frame_axis`, which errors with guidance rather
        // than silently declining — the user picked this column
        max_frames: SLIDER_MAX_FRAMES,
        // draw what was asked for: keep every cell that has any data, keep an entity that shows
        // up in even one frame, and report the gaps rather than silently dropping them
        min_dense_floor: 1,
        min_cell_rows: 1,
        min_entities: 1,
        min_dense_pct: 0,
        warn: true,
    };
    let Some(data) = read_entity_bucket_agg(args, entity_idx, x_idx, y_idx, frame_idx, &opts)?
    else {
        return fail_clierror!(
            "No plottable bubbles found. A bubble animation needs numeric --x/--y (and --size) \
             values, a non-empty --series entity, and at least 2 distinct --slider values."
        );
    };
    if data.entities.len() > BUBBLE_MAX_ENTITIES {
        return fail_incorrectusage_clierror!(
            "--series has {} distinct values; the bubble animation maximum is \
             {BUBBLE_MAX_ENTITIES}.\nEvery entity is a separate trace in every frame, so this \
             would produce an unusably large file. Pick a lower-cardinality --series column.",
            data.entities.len()
        );
    }

    let x_label = col_label(&headers, x_idx, nh);
    let y_label = col_label(&headers, y_idx, nh);
    let size_label = col_label(&headers, size_idx, nh);
    let frame_label = col_label(&headers, frame_idx, nh);

    let mut plot = Plot::new();
    add_bubble_traces_and_frames(&mut plot, &data, &x_label, &y_label, &size_label);

    let mut layout = Layout::new().show_legend(true);
    if let Some(title) = &args.flag_title {
        layout = layout.title(Title::with_text(title));
    }
    // same sink as `build_layout`: the fallback is a raw CSV header, so it is escaped; an explicit
    // --x-title is operator-supplied and stays raw.
    let mut x = Axis::new().title(Title::with_text(
        args.flag_x_title
            .clone()
            .unwrap_or_else(|| escape_hover(&x_label)),
    ));
    // pin both axes globally so the bubbles move within a fixed frame instead of the axes
    // rescaling under them every frame
    x = x.range(vec![data.x_range.0, data.x_range.1]);
    if args.flag_rangeslider {
        x = x.range_slider(RangeSlider::new().visible(true));
    }
    layout = layout.x_axis(x);

    let mut y = Axis::new().title(Title::with_text(
        args.flag_y_title
            .clone()
            .unwrap_or_else(|| escape_hover(&y_label)),
    ));
    // an explicit --y-range wins over the pinned global range
    let (ylo, yhi) = if let Some(r) = &args.flag_y_range {
        parse_y_range(r)?
    } else {
        data.y_range
    };
    y = y.range(vec![ylo, yhi]);
    layout = layout.y_axis(y);

    if let Some(note) = &args.flag_annotation {
        layout = with_chart_note(layout, note);
    }

    layout = layout.sliders(vec![slider_control(
        &frame_label,
        &data.bucket_labels,
        args.flag_slider_speed,
    )?]);
    layout = layout.update_menus(vec![play_pause_menu(args.flag_slider_speed)?]);

    plot.set_layout(apply_theme(layout, args.theme()));
    Ok(plot)
}

/// Animation options for a slider step / Play button: immediate mode, `speed_ms` per frame, a
/// half-speed tween, and a redraw so trace-data changes actually repaint.
fn slider_anim_options(speed_ms: u64) -> AnimationOptions {
    AnimationOptions::new()
        .mode(AnimationMode::Immediate)
        .frame(
            FrameSettings::new()
                .duration(speed_ms as usize)
                .redraw(true),
        )
        .transition(TransitionSettings::new().duration((speed_ms / 2) as usize))
}

/// A scrub `Slider` with one animate step per frame value; dragging a step animates to that frame.
///
/// ESCAPING INVARIANT (see #4333): plotly renders its markup subset (`<b>`, `<i>`, `<a href>`, ...)
/// in BOTH slider surfaces — `drawLabel` (`sliders/draw.js:365`) and `drawCurrentValue`
/// (`:322`) each pipe their text through `svgTextUtils.convertToTspans`. The `data-notex` attr on
/// those nodes only suppresses MathJax, not the HTML subset. So the two RENDERED strings — the step
/// `label` and the current-value `prefix` — are escaped, since both carry untrusted text (raw CSV
/// cells / column headers / dictionary labels).
///
/// The other two fields must stay RAW: `.value()` and `Animation::frames()` are matched against
/// `Frame::name` by exact string, so escaping them desynchronizes the slider from the frames (the
/// #4247 decision recorded in CHANGELOG.md). `.value()` in particular must keep being set
/// EXPLICITLY: `sliders/defaults.js:101-102` does `coerce('value', label)`, so dropping it would
/// silently make `value` inherit the now-escaped label and break frame matching.
fn slider_control(col_label: &str, frame_values: &[String], speed_ms: u64) -> CliResult<Slider> {
    let mut steps = Vec::with_capacity(frame_values.len());
    for label in frame_values {
        let step = SliderStepBuilder::new()
            .label(escape_hover(label))
            .value(label.clone())
            .animation(
                Animation::frames(vec![label.clone()]).options(slider_anim_options(speed_ms)),
            )
            .build();
        match step {
            Ok(s) => steps.push(s),
            Err(e) => return fail_clierror!("failed to build slider step: {e}"),
        }
    }
    Ok(Slider::new()
        .active(0)
        .current_value(
            SliderCurrentValue::new()
                .visible(true)
                .prefix(format!("{}: ", escape_hover(col_label))),
        )
        // top padding drops the slider (and its current-value readout) clear of the x-axis title
        // so the two don't collide (the `common::Pad(t, b, l)` type — NOT the unqualified
        // `treemap::Pad` imported in this file)
        .pad(plotly::common::Pad::new(60, 10, 0))
        .steps(steps))
}

/// A top-left Play/Pause button row that runs / halts the frame animation.
fn play_pause_menu(speed_ms: u64) -> CliResult<UpdateMenu> {
    let play = ButtonBuilder::new()
        .label(&t!("viz.menu.play"))
        .animation(Animation::all_frames().options(slider_anim_options(speed_ms).fromcurrent(true)))
        .build();
    let play = match play {
        Ok(b) => b,
        Err(e) => return fail_clierror!("failed to build play button: {e}"),
    };
    let pause = ButtonBuilder::new()
        .label(&t!("viz.menu.pause"))
        .animation(Animation::pause())
        .build();
    let pause = match pause {
        Ok(b) => b,
        Err(e) => return fail_clierror!("failed to build pause button: {e}"),
    };
    Ok(UpdateMenu::new()
        .ty(UpdateMenuType::Buttons)
        .direction(UpdateMenuDirection::Left)
        .buttons(vec![play, pause])
        // top-left, clear of the centered title and the top-right modebar
        .x(0.0)
        .x_anchor(Anchor::Left)
        .y(1.0)
        .y_anchor(Anchor::Bottom)
        .show_active(false)
        .active(-1)
        .background_color(NamedColor::White)
        .border_color(NamedColor::Gray)
        .border_width(1)
        .font(Font::new().family(FONT_FAMILY).color(INK).size(11)))
}

/// Build a Scatter trace, using a numeric x-axis when every x value parses as a finite number
/// (`parse_finite_f64` semantics — "inf"/"nan" literals force a categorical axis instead of
/// serializing as `null` and silently dropping points), otherwise a categorical (string) x-axis.
fn scatter_trace(name: &str, xs: Vec<String>, ys: Vec<f64>, mode: Mode) -> Box<dyn Trace> {
    let xn = (!xs.is_empty())
        .then(|| {
            xs.iter()
                .map(|s| parse_finite_f64(s))
                .collect::<Option<Vec<f64>>>()
        })
        .flatten();
    if let Some(xn) = xn {
        let mut t = Scatter::new(xn, ys).mode(mode);
        if !name.is_empty() {
            t = t.name(escape_hover(name));
        }
        t
    } else {
        let mut t = Scatter::new(xs, ys).mode(mode);
        if !name.is_empty() {
            t = t.name(escape_hover(name));
        }
        t
    }
}

/// True when the user requested a per-point marker encoding (--color and/or --size).
fn encoded_scatter(args: &Args) -> bool {
    args.flag_color.is_some() || args.flag_size.is_some()
}

/// Build a single scatter trace that encodes numeric columns onto marker color (continuous
/// colorscale + colorbar) and/or marker size (bubble chart). Rows missing a numeric value
/// for y or any requested encoding are skipped.
fn build_scatter_encoded(args: &Args) -> CliResult<(Vec<Box<dyn Trace>>, String, String)> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let x_idx = resolve_one(args.flag_x.as_ref(), &headers, nh, "x")?;
    let y_idx = resolve_one(args.flag_y.as_ref(), &headers, nh, "y")?;
    let color_idx = match args.flag_color.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "color")?),
        None => None,
    };
    let size_idx = match args.flag_size.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "size")?),
        None => None,
    };

    let mut xs: Vec<String> = Vec::new();
    let mut ys: Vec<f64> = Vec::new();
    let mut colors: Vec<f64> = Vec::new();
    let mut sizes: Vec<f64> = Vec::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let Some(y) = parse_f64(record.get(y_idx)) else {
            continue;
        };
        // a point is only plottable if every requested encoding has a numeric value
        let color = match color_idx {
            Some(i) => match parse_f64(record.get(i)) {
                Some(v) => Some(v),
                None => continue,
            },
            None => None,
        };
        let size = match size_idx {
            Some(i) => match parse_f64(record.get(i)) {
                Some(v) => Some(v),
                None => continue,
            },
            None => None,
        };
        xs.push(cell_to_string(record.get(x_idx)));
        ys.push(y);
        if let Some(v) = color {
            colors.push(v);
        }
        if let Some(v) = size {
            sizes.push(v);
        }
    }
    if ys.is_empty() {
        return fail_clierror!(
            "No plottable rows found (are --y and the --color/--size columns numeric?)."
        );
    }

    let x_label = col_label(&headers, x_idx, nh);
    let y_label = col_label(&headers, y_idx, nh);
    let color_label = color_idx.map(|i| col_label(&headers, i, nh));
    let size_label = size_idx.map(|i| col_label(&headers, i, nh));

    // Pre-render per-point hover naming x/y plus any color/size encoding — plotly's default shows
    // only the bare (x, y) pair, hiding the encoded dimensions. Numeric values are comma-grouped.
    let xl = escape_hover(&x_label);
    let yl = escape_hover(&y_label);
    let cl = color_label.as_deref().map(escape_hover);
    let sl = size_label.as_deref().map(escape_hover);
    let hover: Vec<String> = (0..ys.len())
        .map(|p| {
            let xdisp = xs[p]
                .trim()
                .parse::<f64>()
                .ok()
                .map_or_else(|| escape_hover(&xs[p]), fmt_measure);
            let mut label = format!("{xl}: {xdisp}<br>{yl}: {}", fmt_measure(ys[p]));
            if let (Some(cl), Some(c)) = (cl.as_ref(), colors.get(p)) {
                label.push_str(&format!("<br>{cl}: {}", fmt_measure(*c)));
            }
            if let (Some(sl), Some(s)) = (sl.as_ref(), sizes.get(p)) {
                label.push_str(&format!("<br>{sl}: {}", fmt_measure(*s)));
            }
            label
        })
        .collect();

    let mut marker = Marker::new();
    if !sizes.is_empty() {
        marker = marker.size_array(scale_bubble_sizes(&sizes));
    }
    if !colors.is_empty() {
        marker = marker
            .color_array(colors)
            .color_scale(ColorScale::Palette(ColorScalePalette::Viridis))
            .show_scale(true)
            .color_bar(
                ColorBar::new().title(color_label.as_deref().map(escape_hover).unwrap_or_default()),
            );
    }

    let trace = scatter_with_marker(xs, ys, marker, hover);
    Ok((vec![trace], x_label, y_label))
}

/// Build a markers-mode Scatter with the given marker, using a numeric x-axis when every x
/// value parses as a finite number (otherwise a categorical x-axis), mirroring `scatter_trace`.
/// `hover`, when non-empty, is the row-aligned pre-rendered hover text (naming x/y and any
/// size/color encoding) attached via `HoverInfo::Text`.
fn scatter_with_marker(
    xs: Vec<String>,
    ys: Vec<f64>,
    marker: Marker,
    hover: Vec<String>,
) -> Box<dyn Trace> {
    // numeric vs categorical x produce differently-typed Scatter traces, so each branch owns the
    // (identical) hover attachment before coercing to `Box<dyn Trace>`.
    let xn = (!xs.is_empty())
        .then(|| {
            xs.iter()
                .map(|s| parse_finite_f64(s))
                .collect::<Option<Vec<f64>>>()
        })
        .flatten();
    if let Some(xn) = xn {
        let mut t = Scatter::new(xn, ys).mode(Mode::Markers).marker(marker);
        if !hover.is_empty() {
            t = t.hover_text_array(hover).hover_info(HoverInfo::Text);
        }
        t
    } else {
        let mut t = Scatter::new(xs, ys).mode(Mode::Markers).marker(marker);
        if !hover.is_empty() {
            t = t.hover_text_array(hover).hover_info(HoverInfo::Text);
        }
        t
    }
}

/// Linearly rescale raw `--size` values into the [`BUBBLE_MIN_PX`, `BUBBLE_MAX_PX`] pixel
/// range so bubbles stay readable regardless of the column's magnitude. When all values are
/// equal, every bubble gets the midpoint size. Non-finite values (a missing/NaN measure on a
/// sized map point) are excluded from the min/max and drawn at the smallest size, so they stay
/// visible rather than collapsing to a zero-pixel (invisible) marker.
fn scale_bubble_sizes(values: &[f64]) -> Vec<usize> {
    let (min, max) = values
        .iter()
        .filter(|v| v.is_finite())
        .fold((f64::INFINITY, f64::NEG_INFINITY), |(lo, hi), &v| {
            (lo.min(v), hi.max(v))
        });
    let span = max - min;
    values
        .iter()
        .map(|&v| {
            let t = if !v.is_finite() {
                0.0
            } else if span > 0.0 {
                (v - min) / span
            } else {
                0.5
            };
            (BUBBLE_MIN_PX + t * (BUBBLE_MAX_PX - BUBBLE_MIN_PX)).round() as usize
        })
        .collect()
}

/// Radius (pixels) of each point's influence in a `--density` heatmap. A moderate default that
/// reads as a smooth surface for city-to-continent scales without over-blurring dense clusters.
const MAP_DENSITY_RADIUS_PX: u8 = 20;

/// Density radius for the `viz smart` auto map panel. Smaller than the standalone `viz map`
/// default because the Data Schematic panel is small: a large radius saturates the whole built-up
/// area into one flat blob, hiding the internal hotspots a smaller radius reveals.
const MAP_SMART_DENSITY_RADIUS_PX: u8 = 8;

/// Resolve a `--style` name to its plotly MapLibre `MapStyle`. Every MapLibre bundled style
/// renders without an access token (unlike the old Mapbox basemaps, no token is ever needed).
/// (The Stamen basemaps have no MapLibre equivalent and were dropped.)
fn parse_map_style(name: &str) -> CliResult<MapStyle> {
    let resolved = match name.to_ascii_lowercase().as_str() {
        "open-street-map" | "osm" => MapStyle::OpenStreetMap,
        "carto-positron" => MapStyle::CartoPositron,
        "carto-positron-nolabels" => MapStyle::CartoPositronNoLabels,
        "carto-darkmatter" | "carto-dark-matter" => MapStyle::CartoDarkMatter,
        "carto-darkmatter-nolabels" | "carto-dark-matter-nolabels" => {
            MapStyle::CartoDarkMatterNoLabels
        },
        "carto-voyager" => MapStyle::CartoVoyager,
        "carto-voyager-nolabels" => MapStyle::CartoVoyagerNoLabels,
        "white-bg" => MapStyle::WhiteBg,
        "basic" => MapStyle::Basic,
        "streets" => MapStyle::Streets,
        "outdoors" => MapStyle::Outdoors,
        "light" => MapStyle::Light,
        "dark" => MapStyle::Dark,
        "satellite" => MapStyle::Satellite,
        "satellite-streets" => MapStyle::SatelliteStreets,
        other => {
            return fail_incorrectusage_clierror!(
                "Unknown --style '{other}'. Supported MapLibre styles (all token-free): \
                 open-street-map, carto-positron, carto-darkmatter, carto-voyager (each also as a \
                 label-free '-nolabels' variant, e.g. carto-positron-nolabels), white-bg, basic, \
                 streets, outdoors, light, dark, satellite, satellite-streets."
            );
        },
    };
    Ok(resolved)
}

/// Value at quantile `q` (0.0..=1.0) of an already-sorted slice, via nearest-rank. Empty → 0.0.
fn sorted_quantile(sorted: &[f64], q: f64) -> f64 {
    if sorted.is_empty() {
        return 0.0;
    }
    let idx = (((sorted.len() - 1) as f64) * q).round() as usize;
    sorted[idx.min(sorted.len() - 1)]
}

/// Uniformly downsample two row-aligned vectors to at most `cap` elements via evenly spaced stride
/// sampling, preserving the overall distribution/shape. Returns clones unchanged when already
/// within `cap` (or when `cap == 0`).
fn downsample_pair<X: Clone, Y: Clone>(xs: &[X], ys: &[Y], cap: usize) -> (Vec<X>, Vec<Y>) {
    let n = xs.len();
    if cap == 0 || n <= cap {
        return (xs.to_vec(), ys.to_vec());
    }
    let mut out_x = Vec::with_capacity(cap);
    let mut out_y = Vec::with_capacity(cap);
    for i in 0..cap {
        // endpoint-inclusive: first sample is index 0, last is index n-1, so a chronologically
        // sorted series (e.g. a time-series panel) keeps both its earliest and latest observation
        let idx = if cap == 1 { 0 } else { i * (n - 1) / (cap - 1) };
        out_x.push(xs[idx].clone());
        out_y.push(ys[idx].clone());
    }
    (out_x, out_y)
}

/// Largest-Triangle-Three-Buckets downsampling. Returns the indices (into `xs`/`ys`) of at most
/// `cap` points that best preserve the visual shape of a `y = f(x)` series (all `n` indices
/// unchanged when `n <= cap` or `cap == 0`). Unlike uniform stride sampling (`downsample_pair`),
/// this selects by triangle area in (x, y) space, so spikes/peaks between strides survive instead
/// of being stepped over. Requires `xs` sorted ascending (monotonic) and finite `ys`. First and
/// last points are always retained.
fn lttb_indices(xs: &[f64], ys: &[f64], cap: usize) -> Vec<usize> {
    let n = xs.len();
    if cap == 0 || n <= cap {
        return (0..n).collect();
    }
    if cap < 3 {
        return if cap == 1 { vec![0] } else { vec![0, n - 1] };
    }
    let mut out = Vec::with_capacity(cap);
    out.push(0); // always keep the first point
    let every = (n - 2) as f64 / (cap - 2) as f64; // bucket width over the interior points
    let mut a = 0usize; // last selected point (the triangle's first vertex)
    for i in 0..cap - 2 {
        // average of the *next* bucket is the triangle's third vertex
        let avg_start = ((i + 1) as f64 * every) as usize + 1;
        let avg_end = (((i + 2) as f64 * every) as usize + 1).min(n);
        let avg_len = (avg_end - avg_start).max(1) as f64;
        let (mut avg_x, mut avg_y) = (0.0_f64, 0.0_f64);
        for j in avg_start..avg_end {
            avg_x += xs[j];
            avg_y += ys[j];
        }
        avg_x /= avg_len;
        avg_y /= avg_len;
        // pick the point in the current bucket forming the largest triangle with (a, next-avg)
        let range_start = (i as f64 * every) as usize + 1;
        let range_end = (((i + 1) as f64 * every) as usize + 1).min(n);
        let (ax, ay) = (xs[a], ys[a]);
        let (mut max_area, mut chosen) = (-1.0_f64, range_start);
        for j in range_start..range_end {
            // FMA version of ((ax - avg_x) * (ys[j] - ay) - (ax - xs[j]) * (avg_y - ay)).abs();
            let area = (ax - xs[j])
                .mul_add(-(avg_y - ay), (ax - avg_x) * (ys[j] - ay))
                .abs();
            if area > max_area {
                max_area = area;
                chosen = j;
            }
        }
        out.push(chosen);
        a = chosen;
    }
    out.push(n - 1); // always keep the last point
    out
}

/// Web-Mercator normalized Y for a latitude (0 at +85.05°N, 1 at 85.05°S) — the vertical analogue
/// of `lon / 360`. Latitude is clamped to the Web-Mercator limit (±85.05°) so `tan`/`ln` can't blow
/// up. Used by `fitbounds_zoom` to size a latitude span in the same projected units MapLibre zoom
/// is defined in.
fn mercator_y(lat: f64) -> f64 {
    const MERCATOR_MAX_LAT: f64 = 85.05;
    let lat_rad = lat.clamp(-MERCATOR_MAX_LAT, MERCATOR_MAX_LAT).to_radians();
    (1.0 - (std::f64::consts::FRAC_PI_4 + lat_rad / 2.0).tan().ln() / std::f64::consts::PI) / 2.0
}

/// Inverse of `mercator_y`: normalized Web-Mercator y (0 at the north edge, 1 at the south) back
/// to a latitude in degrees.
fn inverse_mercator_y(y: f64) -> f64 {
    let t = (std::f64::consts::PI * (1.0 - 2.0 * y)).exp();
    (2.0 * (t.atan() - std::f64::consts::FRAC_PI_4)).to_degrees()
}

/// The latitude that sits VISUALLY halfway between two latitudes on a Web-Mercator map. Mercator
/// stretches poleward, so the geographic mean `(min + max) / 2` is not the middle of the drawn
/// box: pairing it with a Mercator-fitted zoom leaves the box off-centre and pushes the poleward
/// edge out of view. Both centre computations here fit their zoom with `fitbounds_zoom` (which
/// measures the latitude span in Mercator y), so both must take their centre in the same space.
fn mercator_lat_center(min_lat: f64, max_lat: f64) -> f64 {
    inverse_mercator_y((mercator_y(min_lat) + mercator_y(max_lat)) / 2.0)
}

/// Aspect-aware Web-Mercator `fitBounds` zoom: fits the longitude and latitude spans SEPARATELY
/// against the panel's pixel width/height and takes the tighter (smaller) zoom, so a wide-short map
/// panel frames a square core box by its (binding) height and a wide full-extent box by its width —
/// instead of the naive `floor(log2(360 / max(latSpan, lonSpan)))` that compared the larger span to
/// the 360°-wide world and ignored the panel aspect (over-zooming square boxes, under-zooming wide
/// ones).
/// ```text
///   zoom_lon = log2( (width_px  / 512) / (lon_span / 360) )
///   zoom_lat = log2( (height_px / 512) / (mercator_y(min_lat) - mercator_y(max_lat)) )
///   zoom     = floor( min(zoom_lon, zoom_lat) - log2(MAP_FIT_PAD) ).clamp(1, 16)
/// ```
/// A degenerate (single-point) box returns a street-level zoom of 10. `lon_span` is the already-
/// resolved (antimeridian-aware) span from `lon_center_and_span`, so for a global or dateline-
/// crossing extent it can approach 360; the math handles that (it just yields a lower zoom, then
/// `clamp(1, 16)`).
fn fitbounds_zoom(min_lat: f64, max_lat: f64, lon_span: f64, width_px: f64, height_px: f64) -> u8 {
    let lon_frac = lon_span / 360.0;
    let lat_frac = mercator_y(min_lat) - mercator_y(max_lat);
    if lon_frac <= 0.0 && lat_frac <= 0.0 {
        // degenerate (single point) box: sensible street-level default
        return 10;
    }
    let zoom_lon = if lon_frac > 0.0 {
        ((width_px / MAP_TILE_SIZE_PX) / lon_frac).log2()
    } else {
        f64::INFINITY
    };
    let zoom_lat = if lat_frac > 0.0 {
        ((height_px / MAP_TILE_SIZE_PX) / lat_frac).log2()
    } else {
        f64::INFINITY
    };
    (zoom_lon.min(zoom_lat) - MAP_FIT_PAD.log2())
        .floor()
        .clamp(1.0, 16.0) as u8
}

/// Compute a map center and a zoom level that frames the data, so the basemap doesn't default to
/// plotly's whole-world view centered at (0, 0). Longitude is handled with antimeridian wrap so a
/// cluster straddling the 180° line (e.g. 179 and -179) frames as the small arc across the date
/// line rather than spanning almost the whole globe and centering near 0.
///
/// `trim_frac` is the fraction trimmed off each end of the lat/lon distributions before framing,
/// so a few outlier coordinates can't dominate the center/zoom. `viz smart`'s auto panel passes
/// `MAP_FRAME_TRIM_FRAC`; the standalone `viz map` command passes `0.0` to frame the full extent
/// of every valid coordinate (its edge points are intentional, not noise).
///
/// `width_px`/`height_px` are the panel's pixel dimensions, used by the aspect-aware
/// `fitbounds_zoom`: the smart inline panel is wide and short (`MAP_PANEL_ASSUMED_WIDTH_PX` ×
/// `MAP_PANEL_USABLE_HEIGHT_PX`), the standalone `viz map` export is `DEFAULT_IMG_WIDTH` ×
/// `DEFAULT_IMG_HEIGHT`.
fn map_center_zoom(
    lats: &[f64],
    lons: &[f64],
    trim_frac: f64,
    width_px: f64,
    height_px: f64,
) -> (Center, u8) {
    let mut sorted_lats = lats.to_vec();
    sorted_lats.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
    let min_lat = sorted_quantile(&sorted_lats, trim_frac);
    let max_lat = sorted_quantile(&sorted_lats, 1.0 - trim_frac);
    let lat_center = mercator_lat_center(min_lat, max_lat);

    let (lon_center, lon_span) = lon_center_and_span(lons, trim_frac);
    let center = Center::new(lat_center, lon_center);

    let zoom = fitbounds_zoom(min_lat, max_lat, lon_span, width_px, height_px);
    (center, zoom)
}

/// The theme-appropriate Carto basemap for a smart-Data Schematic map panel. Data-overlay panels
/// (density heatmaps, choropleth fills) use the label-free Carto variant so basemap place names
/// don't compete with the data; ordinary point maps keep labels for street/city context. The
/// theme-toggle JS preserves this labeled-vs-nolabels distinction per subplot (see
/// `THEME_TOGGLE_JS`), so a panel's choice survives a live light/dark switch.
const fn smart_map_basemap(is_dark: bool, data_overlay: bool) -> MapStyle {
    match (is_dark, data_overlay) {
        (true, true) => MapStyle::CartoDarkMatterNoLabels,
        (true, false) => MapStyle::CartoDarkMatter,
        (false, true) => MapStyle::CartoPositronNoLabels,
        (false, false) => MapStyle::CartoPositron,
    }
}

/// Build a MapLibre pan constraint (`layout.map.bounds`) from one or more coordinate clouds: the
/// combined bounding box padded by `MAP_BOUNDS_PAD_FRAC` of its span (at least
/// `MAP_BOUNDS_MIN_PAD_DEG`) and clamped to valid lat/lon. Pass every cloud that any map view can
/// reach (core points AND geographic outliers) so a "Full extent" zoom stays inside the box.
/// Returns `None` for an empty/degenerate cloud or one spanning more than
/// `MAP_BOUNDS_MAX_LON_SPAN_DEG` of longitude (near-global / antimeridian-crossing), where a
/// west < east box is meaningless and a constraint isn't wanted.
fn map_pan_bounds(coord_groups: &[(&[f64], &[f64])]) -> Option<MapBounds> {
    let (mut min_lat, mut max_lat) = (f64::INFINITY, f64::NEG_INFINITY);
    let (mut min_lon, mut max_lon) = (f64::INFINITY, f64::NEG_INFINITY);
    for (lats, lons) in coord_groups {
        for &v in *lats {
            if v.is_finite() {
                min_lat = min_lat.min(v);
                max_lat = max_lat.max(v);
            }
        }
        for &v in *lons {
            if v.is_finite() {
                min_lon = min_lon.min(v);
                max_lon = max_lon.max(v);
            }
        }
    }
    if !(min_lat.is_finite() && max_lat.is_finite() && min_lon.is_finite() && max_lon.is_finite()) {
        return None;
    }
    let lon_span = max_lon - min_lon;
    if lon_span > MAP_BOUNDS_MAX_LON_SPAN_DEG {
        return None;
    }
    let lat_pad = ((max_lat - min_lat) * MAP_BOUNDS_PAD_FRAC).max(MAP_BOUNDS_MIN_PAD_DEG);
    let lon_pad = (lon_span * MAP_BOUNDS_PAD_FRAC).max(MAP_BOUNDS_MIN_PAD_DEG);
    Some(
        MapBounds::new()
            .west((min_lon - lon_pad).max(-180.0))
            .east((max_lon + lon_pad).min(180.0))
            .south((min_lat - lat_pad).max(-90.0))
            .north((max_lat + lat_pad).min(90.0)),
    )
}

/// Antimeridian-aware longitude center + span. The data occupies all of the 360° circle except
/// its single largest empty gap; the cluster is the complementary arc, so the span is
/// `360 - largest_gap` and the center is that arc's midpoint, normalized to [-180, 180]. When the
/// largest gap is the wrap between max and min (the common, non-crossing case), this reduces to a
/// `trim_frac`-trimmed midpoint and span (the same robust framing the latitude axis uses; pass
/// `0.0` for the plain full-extent `(min+max)/2` midpoint and `max-min` span).
fn lon_center_and_span(lons: &[f64], trim_frac: f64) -> (f64, f64) {
    let mut sorted: Vec<f64> = lons.to_vec();
    sorted.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
    let n = sorted.len();
    if n == 0 {
        // no points -> degenerate center/span (guards the sorted[0]/sorted[n - 1] indexing below)
        return (0.0, 0.0);
    }
    // largest gap between adjacent longitudes (the empty wedge the data does NOT cover)
    let mut max_gap = 0.0_f64;
    let mut gap_idx = 0; // gap lies between sorted[gap_idx] and sorted[gap_idx + 1]
    for i in 0..n.saturating_sub(1) {
        let gap = sorted[i + 1] - sorted[i];
        if gap > max_gap {
            max_gap = gap;
            gap_idx = i;
        }
    }
    // the wrap gap closes the circle from the easternmost point back to the westernmost
    let wrap_gap = (sorted[0] + 360.0) - sorted[n - 1];
    if wrap_gap >= max_gap {
        // data doesn't cross the antimeridian: trimmed (or full-extent when trim_frac == 0)
        // bounding-box midpoint and span
        let lo = sorted_quantile(&sorted, trim_frac);
        let hi = sorted_quantile(&sorted, 1.0 - trim_frac);
        ((lo + hi) / 2.0, hi - lo)
    } else {
        // data crosses the antimeridian: unwrap the cluster into a contiguous ascending longitude
        // range — the arc east of the gap (sorted[gap_idx + 1..]), then the points west of it
        // shifted +360 — and apply the same quantile trimming as the non-crossing branch, so a lone
        // far in-range outlier can't inflate the span. The cluster runs from sorted[gap_idx + 1]
        // eastward, wrapping past 180, to sorted[gap_idx] (+360 keeps the arc contiguous). With
        // trim_frac == 0 this reduces exactly to the full `360 - max_gap` span and the arc-midpoint
        // center.
        let mut unwrapped: Vec<f64> = Vec::with_capacity(n);
        unwrapped.extend_from_slice(&sorted[gap_idx + 1..]);
        unwrapped.extend(sorted[..=gap_idx].iter().map(|v| v + 360.0));
        let lo = sorted_quantile(&unwrapped, trim_frac);
        let hi = sorted_quantile(&unwrapped, 1.0 - trim_frac);
        let mut center = (lo + hi) / 2.0;
        if center > 180.0 {
            center -= 360.0;
        }
        (center, hi - lo)
    }
}

/// Build per-point hover text for a `viz geo` / `viz map` point trace: the labeled `--color` and
/// `--size` values (and any `--text` label) shown alongside the coordinates, so hovering a point
/// surfaces the values that drive it instead of only its bare lat/lon. Returns `None` when there
/// is nothing to add beyond the coordinates (no `--color`/`--size`/`--text`), leaving the trace on
/// plotly's default lon/lat hover. `color`/`size` are `(column label, row-aligned values)`; the
/// values arrays are the same length as `lats`/`lons` (a row is kept only when every requested
/// encoding is present). `texts` are already `escape_hover`-ed at read time.
fn geo_point_hover(
    lats: &[f64],
    lons: &[f64],
    color: Option<(&str, &[f64])>,
    size: Option<(&str, &[f64])>,
    texts: &[String],
) -> Option<Vec<String>> {
    if color.is_none() && size.is_none() && texts.is_empty() {
        return None;
    }
    let cl = color.map(|(l, _)| escape_hover(l));
    let sl = size.map(|(l, _)| escape_hover(l));
    let hover = (0..lats.len())
        .map(|p| {
            let mut h = String::new();
            // the --text label heads the tooltip as the point's name
            if let Some(t) = texts.get(p) {
                h.push_str(t);
                h.push_str("<br>");
            }
            // coordinates, matching the density path's `%{lat:.5f}, %{lon:.5f}` precision
            h.push_str(&format!("{:.5}, {:.5}", lats[p], lons[p]));
            if let (Some(cl), Some((_, cv))) = (cl.as_ref(), color) {
                h.push_str(&format!("<br>{cl}: {}", fmt_measure(cv[p])));
            }
            if let (Some(sl), Some((_, sv))) = (sl.as_ref(), size) {
                h.push_str(&format!("<br>{sl}: {}", fmt_measure(sv[p])));
            }
            h
        })
        .collect();
    Some(hover)
}

/// Per-point hover for a `--series` geo/map trace (and its animated `--slider` frames): the
/// series/category value (the trace `name`) plus the coordinates — and any `--text` label — so a
/// point's tooltip identifies which category it belongs to instead of showing only its bare
/// lat/lon. `texts` is honored only when it is row-aligned with `lats` (i.e. `--text` was given).
fn geo_series_hover(name: &str, lats: &[f64], lons: &[f64], texts: &[String]) -> Vec<String> {
    let has_text = texts.len() == lats.len();
    let ename = escape_hover(name);
    (0..lats.len())
        .map(|p| {
            let mut h = String::new();
            if has_text {
                h.push_str(&texts[p]);
                h.push_str("<br>");
            }
            if !ename.is_empty() {
                h.push_str(&ename);
                h.push_str("<br>");
            }
            h.push_str(&format!("{:.5}, {:.5}", lats[p], lons[p]));
            h
        })
        .collect()
}

fn scatter_map_with_marker(
    lats: Vec<f64>,
    lons: Vec<f64>,
    marker: Marker,
    hover: Option<Vec<String>>,
) -> Box<dyn Trace> {
    let mut t = ScatterMap::new(lats, lons)
        .mode(Mode::Markers)
        .marker(marker);
    if let Some(hover) = hover {
        t = t.hover_text_array(hover).hover_info(HoverInfo::Text);
    }
    t
}

fn map_series_traces(
    lats: Vec<f64>,
    lons: Vec<f64>,
    series: Vec<String>,
    texts: Vec<String>,
) -> Vec<Box<dyn Trace>> {
    let has_text = texts.len() == lats.len();
    let mut order: Vec<String> = Vec::new();
    let mut groups: BTreeMap<String, (Vec<f64>, Vec<f64>, Vec<String>)> = BTreeMap::new();
    for i in 0..lats.len() {
        let name = series[i].clone();
        let entry = groups.entry(name.clone()).or_insert_with(|| {
            order.push(name);
            (Vec::new(), Vec::new(), Vec::new())
        });
        entry.0.push(lats[i]);
        entry.1.push(lons[i]);
        if has_text {
            entry.2.push(texts[i].clone());
        }
    }
    order
        .into_iter()
        .map(|name| {
            let (la, lo, tx) = groups.remove(&name).unwrap_or_default();
            // hover names the series/category (and any --text label) beside the coordinates
            let hover = geo_series_hover(&name, &la, &lo, &tx);
            let mut t = ScatterMap::new(la, lo)
                .mode(Mode::Markers)
                .name(escape_hover(&name));
            t = t.hover_text_array(hover).hover_info(HoverInfo::Text);
            let trace: Box<dyn Trace> = t;
            trace
        })
        .collect()
}

/// Pixel dimensions to frame a map / choropleth basemap against (fed to `map_center_zoom`'s
/// aspect-aware fit). `--width`/`--height` size the static image export but are NOT applied to the
/// responsive HTML layout, so honor them only when exporting an image — so a narrow/tall or
/// wide/short export frames its own aspect instead of cropping the extent — while HTML frames for
/// the representative default aspect, matching how `run` sizes each output.
fn fit_dims(
    flag_width: Option<usize>,
    flag_height: Option<usize>,
    out_format: OutFormat,
) -> (usize, usize) {
    if out_format.is_image() {
        (
            flag_width.unwrap_or(DEFAULT_IMG_WIDTH),
            flag_height.unwrap_or(DEFAULT_IMG_HEIGHT),
        )
    } else {
        (DEFAULT_IMG_WIDTH, DEFAULT_IMG_HEIGHT)
    }
}

/// Build the complete `Plot` for `viz map`: a `ScatterMap` point map (optionally with
/// `--color`/`--size` marker encodings or `--series` per-category traces) or a `--density`
/// `DensityMap` heatmap, on a MapLibre tile basemap framed to the data's bounding box.
fn build_map_plot(args: &Args, out_format: OutFormat) -> CliResult<Plot> {
    let style_name = args.flag_style.as_deref().unwrap_or("open-street-map");
    let style = parse_map_style(style_name)?;

    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let lat_idx = resolve_one(args.flag_lat.as_ref(), &headers, nh, "lat")?;
    let lon_idx = resolve_one(args.flag_lon.as_ref(), &headers, nh, "lon")?;
    let color_idx = match args.flag_color.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "color")?),
        None => None,
    };
    let size_idx = match args.flag_size.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "size")?),
        None => None,
    };
    let series_idx = match args.flag_series.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "series")?),
        None => None,
    };
    let text_idx = match args.flag_text.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "text")?),
        None => None,
    };

    if args.flag_density && series_idx.is_some() {
        return fail_incorrectusage_clierror!(
            "--density renders a single heatmap layer and cannot be combined with --series."
        );
    }

    // accumulate row-aligned columns. A row is kept only when lat/lon are valid coordinates AND
    // every requested encoding column has a numeric value, so all per-point arrays stay aligned.
    let mut lats: Vec<f64> = Vec::new();
    let mut lons: Vec<f64> = Vec::new();
    let mut colors: Vec<f64> = Vec::new();
    let mut sizes: Vec<f64> = Vec::new();
    let mut series: Vec<String> = Vec::new();
    let mut texts: Vec<String> = Vec::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let (Some(lat), Some(lon)) = (
            parse_f64(record.get(lat_idx)),
            parse_f64(record.get(lon_idx)),
        ) else {
            continue;
        };
        if !(-90.0..=90.0).contains(&lat) || !(-180.0..=180.0).contains(&lon) {
            continue;
        }
        let color = match color_idx {
            Some(i) => match parse_f64(record.get(i)) {
                Some(v) => Some(v),
                None => continue,
            },
            None => None,
        };
        let size = match size_idx {
            Some(i) => match parse_f64(record.get(i)) {
                Some(v) => Some(v),
                None => continue,
            },
            None => None,
        };
        lats.push(lat);
        lons.push(lon);
        if let Some(v) = color {
            colors.push(v);
        }
        if let Some(v) = size {
            sizes.push(v);
        }
        if let Some(i) = series_idx {
            series.push(cell_to_string(record.get(i)));
        }
        if let Some(i) = text_idx {
            // escape: plotly renders point text/hover as pseudo-HTML, so a cell containing
            // `<b>`/`&` would otherwise display as markup instead of the literal value
            texts.push(escape_hover(&cell_to_string(record.get(i))));
        }
    }
    if lats.is_empty() {
        return fail_clierror!(
            "No mappable rows found (are --lat/--lon numeric and within valid coordinate ranges?)."
        );
    }

    // standalone `viz map`: frame the full extent — its edge coordinates are intentional.
    let (fit_w, fit_h) = fit_dims(args.flag_width, args.flag_height, out_format);
    let (center, zoom) = map_center_zoom(&lats, &lons, 0.0, fit_w as f64, fit_h as f64);
    // derive the pan constraint now, before the coordinate vecs are moved into the trace(s) below
    let pan_bounds = map_pan_bounds(&[(lats.as_slice(), lons.as_slice())]);

    let mut plot = Plot::new();
    if args.flag_density {
        // the weight column's label for the hover readout — None when z is the uniform 1.0
        // fallback (a bare "1" per point is noise, so it's omitted)
        let weight_label = if !colors.is_empty() {
            Some(col_label(&headers, color_idx.unwrap(), nh))
        } else if !sizes.is_empty() {
            Some(col_label(&headers, size_idx.unwrap(), nh))
        } else {
            None
        };
        // density weight: the --color or --size column when given, else a uniform 1.0
        let z = if !colors.is_empty() {
            colors
        } else if !sizes.is_empty() {
            sizes
        } else {
            vec![1.0_f64; lats.len()]
        };
        // per-point hover via ONE trace-level template: the optional --text label rides in the
        // `text` array, the coordinates are formatted by plotly from the lat/lon data arrays, and
        // the labeled weight value rides in `customdata` (pre-rendered — `fmt_measure`'s grouped/
        // trimmed output has no d3-format equivalent). Density traces honor only hovertemplate
        // (not hovertext), and a single template embeds the markup once instead of per point.
        let mut template = String::new();
        if !texts.is_empty() {
            template.push_str("%{text}<br>");
        }
        template.push_str("%{lat:.5f}, %{lon:.5f}");
        let weight_values: Option<Vec<String>> = weight_label.as_ref().map(|label| {
            template.push_str(&format!(
                "<br>{}: %{{customdata}}",
                escape_template_pct(&escape_hover(label))
            ));
            z.iter().map(|&v| fmt_measure(v)).collect()
        });
        template.push_str("<extra></extra>");
        let mut dens = DensityMap::new(lats, lons, z)
            .radius(MAP_DENSITY_RADIUS_PX)
            .hover_template(template);
        if !texts.is_empty() {
            dens = dens.text_array(texts);
        }
        if let Some(weight_values) = weight_values {
            dens = dens.custom_data(weight_values);
        }
        plot.add_trace(dens);
    } else if series_idx.is_some() {
        for trace in map_series_traces(lats, lons, series, texts) {
            plot.add_trace(trace);
        }
    } else {
        let color_label = color_idx.map(|i| col_label(&headers, i, nh));
        let size_label = size_idx.map(|i| col_label(&headers, i, nh));
        // build the enriched hover BEFORE the encoding arrays are moved into the marker
        let hover = geo_point_hover(
            &lats,
            &lons,
            color_label.as_deref().map(|l| (l, colors.as_slice())),
            size_label.as_deref().map(|l| (l, sizes.as_slice())),
            &texts,
        );
        let mut marker = Marker::new();
        if !sizes.is_empty() {
            marker = marker.size_array(scale_bubble_sizes(&sizes));
        }
        if !colors.is_empty() {
            marker = marker
                .color_array(colors)
                .color_scale(ColorScale::Palette(ColorScalePalette::Viridis))
                .show_scale(true)
                .color_bar(
                    ColorBar::new()
                        .title(color_label.as_deref().map(escape_hover).unwrap_or_default()),
                );
        }
        plot.add_trace(scatter_map_with_marker(lats, lons, marker, hover));
    }

    // MapLibre `map` subplot: all bundled styles render token-free, so no access token is embedded.
    let mut layout_map = LayoutMap::new()
        .style(style)
        .center(center)
        .zoom(f64::from(zoom));
    // constrain panning to the data's footprint so the map can't be dragged off into empty ocean.
    if let Some(bounds) = pan_bounds {
        layout_map = layout_map.bounds(bounds);
    }
    let mut layout = Layout::new()
        .map(layout_map)
        .show_legend(series_idx.is_some());
    if let Some(title) = &args.flag_title {
        layout = layout.title(Title::with_text(title));
    }
    plot.set_layout(apply_theme(layout, args.theme()));
    Ok(plot)
}

/// Parse a `--projection` name into a plotly geo `ProjectionType`. A curated, token-free subset
/// of plotly's projections; names are case-insensitive and accept spaces or underscores.
fn parse_projection(name: &str) -> CliResult<ProjectionType> {
    let normalized = name.to_ascii_lowercase().replace([' ', '_'], "-");
    let projection = match normalized.as_str() {
        "natural-earth" => ProjectionType::NaturalEarth,
        "mercator" => ProjectionType::Mercator,
        "orthographic" => ProjectionType::Orthographic,
        "equirectangular" => ProjectionType::Equirectangular,
        "albers-usa" => ProjectionType::AlbersUsa,
        "robinson" => ProjectionType::Robinson,
        "winkel-tripel" => ProjectionType::WinkelTripel,
        "mollweide" => ProjectionType::Mollweide,
        "hammer" => ProjectionType::Hammer,
        "azimuthal-equal-area" => ProjectionType::AzimuthalEqualArea,
        _ => {
            return fail_incorrectusage_clierror!(
                "Unknown --projection '{name}'. One of: natural-earth, mercator, orthographic, \
                 equirectangular, albers-usa, robinson, winkel-tripel, mollweide, hammer, \
                 azimuthal-equal-area."
            );
        },
    };
    Ok(projection)
}

fn scatter_geo_with_marker(
    lats: Vec<f64>,
    lons: Vec<f64>,
    marker: Marker,
    hover: Option<Vec<String>>,
) -> Box<dyn Trace> {
    let mut t = ScatterGeo::new(lats, lons)
        .mode(Mode::Markers)
        .marker(marker);
    if let Some(hover) = hover {
        t = t.hover_text_array(hover).hover_info(HoverInfo::Text);
    }
    t
}

fn geo_series_traces(
    lats: Vec<f64>,
    lons: Vec<f64>,
    series: Vec<String>,
    texts: Vec<String>,
) -> Vec<Box<dyn Trace>> {
    let has_text = texts.len() == lats.len();
    let mut order: Vec<String> = Vec::new();
    let mut groups: BTreeMap<String, (Vec<f64>, Vec<f64>, Vec<String>)> = BTreeMap::new();
    for i in 0..lats.len() {
        let name = series[i].clone();
        let entry = groups.entry(name.clone()).or_insert_with(|| {
            order.push(name);
            (Vec::new(), Vec::new(), Vec::new())
        });
        entry.0.push(lats[i]);
        entry.1.push(lons[i]);
        if has_text {
            entry.2.push(texts[i].clone());
        }
    }
    order
        .into_iter()
        .map(|name| {
            let (la, lo, tx) = groups.remove(&name).unwrap_or_default();
            // hover names the series/category (and any --text label) beside the coordinates
            let hover = geo_series_hover(&name, &la, &lo, &tx);
            let mut t = ScatterGeo::new(la, lo)
                .mode(Mode::Markers)
                .name(escape_hover(&name));
            t = t.hover_text_array(hover).hover_info(HoverInfo::Text);
            let trace: Box<dyn Trace> = t;
            trace
        })
        .collect()
}

/// The shared `geo` projection layout for `viz geo` (static and animated): a coastline/land/ocean
/// projection basemap (no tiles or token), the chart title, and legend visibility. The theme and
/// any slider/menu are applied by the caller.
fn geo_base_layout(args: &Args, show_legend: bool) -> CliResult<Layout> {
    let projection = parse_projection(args.flag_projection.as_deref().unwrap_or("natural-earth"))?;
    let (geo_land, geo_water, geo_bg) = geo_palette(args.theme());
    let geo = LayoutGeo::new()
        .projection(Projection::new().projection_type(projection))
        .resolution(GeoResolution::OneOverFiftyMillion)
        .showland(true)
        .landcolor(geo_land)
        .showocean(true)
        .oceancolor(geo_water)
        .showlakes(true)
        .lakecolor(geo_water)
        .showcountries(true)
        .bgcolor(geo_bg);
    let mut layout = Layout::new().geo(geo).show_legend(show_legend);
    if let Some(title) = &args.flag_title {
        layout = layout.title(Title::with_text(title));
    }
    Ok(layout)
}

/// Build the complete `Plot` for `viz geo`: a `ScatterGeo` point map on a projection basemap
/// (coastlines/land/countries, no tiles or token), with optional `--color`/`--size` marker
/// encodings or `--series` per-category traces. Mirrors `build_map_plot` minus the tile basemap.
fn build_geo_plot(args: &Args) -> CliResult<Plot> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let lat_idx = resolve_one(args.flag_lat.as_ref(), &headers, nh, "lat")?;
    let lon_idx = resolve_one(args.flag_lon.as_ref(), &headers, nh, "lon")?;
    let color_idx = match args.flag_color.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "color")?),
        None => None,
    };
    let size_idx = match args.flag_size.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "size")?),
        None => None,
    };
    let series_idx = match args.flag_series.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "series")?),
        None => None,
    };
    let text_idx = match args.flag_text.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "text")?),
        None => None,
    };

    // accumulate row-aligned columns; a row is kept only when lat/lon are valid coordinates AND
    // every requested encoding column has a numeric value, so all per-point arrays stay aligned.
    let mut lats: Vec<f64> = Vec::new();
    let mut lons: Vec<f64> = Vec::new();
    let mut colors: Vec<f64> = Vec::new();
    let mut sizes: Vec<f64> = Vec::new();
    let mut series: Vec<String> = Vec::new();
    let mut texts: Vec<String> = Vec::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let (Some(lat), Some(lon)) = (
            parse_f64(record.get(lat_idx)),
            parse_f64(record.get(lon_idx)),
        ) else {
            continue;
        };
        if !(-90.0..=90.0).contains(&lat) || !(-180.0..=180.0).contains(&lon) {
            continue;
        }
        let color = match color_idx {
            Some(i) => match parse_f64(record.get(i)) {
                Some(v) => Some(v),
                None => continue,
            },
            None => None,
        };
        let size = match size_idx {
            Some(i) => match parse_f64(record.get(i)) {
                Some(v) => Some(v),
                None => continue,
            },
            None => None,
        };
        lats.push(lat);
        lons.push(lon);
        if let Some(v) = color {
            colors.push(v);
        }
        if let Some(v) = size {
            sizes.push(v);
        }
        if let Some(i) = series_idx {
            series.push(cell_to_string(record.get(i)));
        }
        if let Some(i) = text_idx {
            // escape: plotly renders point text/hover as pseudo-HTML, so a cell containing
            // `<b>`/`&` would otherwise display as markup instead of the literal value
            texts.push(escape_hover(&cell_to_string(record.get(i))));
        }
    }
    if lats.is_empty() {
        return fail_clierror!(
            "No mappable rows found (are --lat/--lon numeric and within valid coordinate ranges?)."
        );
    }

    let mut plot = Plot::new();
    if series_idx.is_some() {
        for trace in geo_series_traces(lats, lons, series, texts) {
            plot.add_trace(trace);
        }
    } else {
        let color_label = color_idx.map(|i| col_label(&headers, i, nh));
        let size_label = size_idx.map(|i| col_label(&headers, i, nh));
        // build the enriched hover BEFORE the encoding arrays are moved into the marker
        let hover = geo_point_hover(
            &lats,
            &lons,
            color_label.as_deref().map(|l| (l, colors.as_slice())),
            size_label.as_deref().map(|l| (l, sizes.as_slice())),
            &texts,
        );
        let mut marker = Marker::new();
        if !sizes.is_empty() {
            marker = marker.size_array(scale_bubble_sizes(&sizes));
        }
        if !colors.is_empty() {
            marker = marker
                .color_array(colors)
                .color_scale(ColorScale::Palette(ColorScalePalette::Viridis))
                .show_scale(true)
                .color_bar(
                    ColorBar::new()
                        .title(color_label.as_deref().map(escape_hover).unwrap_or_default()),
                );
        }
        plot.add_trace(scatter_geo_with_marker(lats, lons, marker, hover));
    }

    let layout = geo_base_layout(args, series_idx.is_some())?;
    plot.set_layout(apply_theme(layout, args.theme()));
    Ok(plot)
}

/// Row coordinates partitioned into ordered animation frames for `viz geo --slider`.
struct GeoSliderData {
    /// the `--slider` column's label, shown as the slider's current-value prefix
    frame_label:  String,
    /// distinct slider values in animation order (numeric if all parse, else lexical)
    frame_values: Vec<String>,
    /// per frame: one `(series, lats, lons, texts)` group per `--series` category union member
    /// (empty arrays for a series absent from that frame) so every frame has the SAME trace count
    /// and order
    frames:       Vec<Vec<(String, Vec<f64>, Vec<f64>, Vec<String>)>>,
}

/// Read lat/lon rows and partition them into ordered animation frames by the `--slider` column,
/// grouping within each frame by `--series` (a single empty-named group when `--series` is unset).
/// The `geo` projection auto-frames the full extent, so no per-frame view pinning is needed.
fn read_geo_slider(args: &Args, slider_col: &SelectColumns) -> CliResult<GeoSliderData> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let lat_idx = resolve_one(args.flag_lat.as_ref(), &headers, nh, "lat")?;
    let lon_idx = resolve_one(args.flag_lon.as_ref(), &headers, nh, "lon")?;
    let series_idx = match args.flag_series.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "series")?),
        None => None,
    };
    let text_idx = match args.flag_text.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "text")?),
        None => None,
    };
    let frame_idx = resolve_one(Some(slider_col), &headers, nh, "slider")?;

    let mut frame_order: Vec<String> = Vec::new();
    let mut series_order: Vec<String> = Vec::new();
    // membership index for `series_order`: a Vec::contains probe here is O(distinct series) on
    // EVERY row, which degenerates badly on a high-cardinality --series column.
    let mut series_seen: HashSet<String> = HashSet::new();
    let mut map: HashMap<String, HashMap<String, (Vec<f64>, Vec<f64>, Vec<String>)>> =
        HashMap::new();

    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let (Some(lat), Some(lon)) = (
            parse_f64(record.get(lat_idx)),
            parse_f64(record.get(lon_idx)),
        ) else {
            continue;
        };
        if !(-90.0..=90.0).contains(&lat) || !(-180.0..=180.0).contains(&lon) {
            continue;
        }
        let series = match series_idx {
            Some(i) => cell_to_string(record.get(i)),
            None => String::new(),
        };
        let frame = cell_to_string(record.get(frame_idx));

        if !map.contains_key(&frame) {
            frame_order.push(frame.clone());
        }
        let fmap = map.entry(frame).or_default();
        if !fmap.contains_key(&series) && series_seen.insert(series.clone()) {
            series_order.push(series.clone());
        }
        let entry = fmap
            .entry(series)
            .or_insert_with(|| (Vec::new(), Vec::new(), Vec::new()));
        entry.0.push(lat);
        entry.1.push(lon);
        if let Some(i) = text_idx {
            // plotly renders hover text as pseudo-HTML, so escape `<`/`&` to show the literal value
            entry.2.push(escape_hover(&cell_to_string(record.get(i))));
        }
    }
    if frame_order.is_empty() {
        return fail_clierror!(
            "No mappable rows found (are --lat/--lon numeric and within valid coordinate ranges?)."
        );
    }
    if frame_order.len() < 2 {
        return fail_incorrectusage_clierror!(
            "--slider needs at least 2 distinct values in the chosen column to animate; found 1."
        );
    }
    if frame_order.len() > SLIDER_MAX_FRAMES {
        return fail_incorrectusage_clierror!(
            "--slider column has {} distinct values; the maximum is {SLIDER_MAX_FRAMES}.\nEvery \
             frame embeds its own copy of the plotted data, so this would produce an unusably \
             large file. Bucket the column first (e.g. `qsv datefmt` to a coarser grain, or `qsv \
             apply`), or pick a lower-cardinality --slider column.",
            frame_order.len()
        );
    }

    // animation order: numeric when every frame value parses as finite, else lexical
    if frame_order.iter().all(|f| parse_finite_f64(f).is_some()) {
        frame_order.sort_by(|a, b| {
            parse_finite_f64(a)
                .unwrap_or(f64::NAN)
                .partial_cmp(&parse_finite_f64(b).unwrap_or(f64::NAN))
                .unwrap_or(std::cmp::Ordering::Equal)
        });
    } else {
        frame_order.sort();
    }

    let cumulative = args.flag_slider_cumulative;
    let mut frames: Vec<Vec<(String, Vec<f64>, Vec<f64>, Vec<String>)>> =
        Vec::with_capacity(frame_order.len());
    for fi in 0..frame_order.len() {
        let mut groups: Vec<(String, Vec<f64>, Vec<f64>, Vec<String>)> =
            Vec::with_capacity(series_order.len());
        for s in &series_order {
            let (mut la, mut lo, mut tx): (Vec<f64>, Vec<f64>, Vec<String>) =
                (Vec::new(), Vec::new(), Vec::new());
            let start = if cumulative { 0 } else { fi };
            for prev in &frame_order[start..=fi] {
                if let Some(fm) = map.get(prev)
                    && let Some((pla, plo, ptx)) = fm.get(s)
                {
                    la.extend_from_slice(pla);
                    lo.extend_from_slice(plo);
                    tx.extend_from_slice(ptx);
                }
            }
            groups.push((s.clone(), la, lo, tx));
        }
        frames.push(groups);
    }

    Ok(GeoSliderData {
        frame_label: col_label(&headers, frame_idx, nh),
        frame_values: frame_order,
        frames,
    })
}

fn geo_frame_traces(groups: &[(String, Vec<f64>, Vec<f64>, Vec<String>)]) -> Vec<Box<dyn Trace>> {
    groups
        .iter()
        .map(|(name, la, lo, tx)| {
            // hover names the series/category (and any --text label) beside the coordinates, so an
            // animated point identifies its region/category instead of showing only lat/lon
            let hover = geo_series_hover(name, la, lo, tx);
            let mut t = ScatterGeo::new(la.clone(), lo.clone())
                .mode(Mode::Markers)
                .hover_text_array(hover)
                .hover_info(HoverInfo::Text);
            if !name.is_empty() {
                t = t.name(escape_hover(name));
            }
            let trace: Box<dyn Trace> = t;
            trace
        })
        .collect()
}

/// Build an animated `viz geo` `Plot`: frame-0 `ScatterGeo` traces as the base, one `Frame` per
/// slider value, and the geo layout carrying a scrub `Slider` and a Play/Pause menu. `scattergeo`
/// animates natively (unlike MapLibre `scattermap`), so this reuses the same slider/frame core as
/// the cartesian path.
fn build_slider_geo_plot(args: &Args, slider_col: &SelectColumns) -> CliResult<Plot> {
    let data = read_geo_slider(args, slider_col)?;

    let mut plot = Plot::new();
    let base = geo_frame_traces(&data.frames[0]);
    let n_traces = base.len();
    for t in base {
        plot.add_trace(t);
    }
    if n_traces == 0 {
        return fail_clierror!(
            "No mappable rows found (are --lat/--lon numeric and within valid coordinate ranges?)."
        );
    }

    for (fkey, groups) in data.frame_values.iter().zip(data.frames.iter()) {
        let traces = geo_frame_traces(groups);
        let mut td = Traces::new();
        for t in traces {
            td.push(t);
        }
        plot.add_frame(
            Frame::new()
                .name(fkey.clone())
                .data(td)
                .traces((0..n_traces).collect()),
        );
    }

    let mut layout = geo_base_layout(args, args.flag_series.is_some())?;
    layout = layout.sliders(vec![slider_control(
        &data.frame_label,
        &data.frame_values,
        args.flag_slider_speed,
    )?]);
    layout = layout.update_menus(vec![play_pause_menu(args.flag_slider_speed)?]);
    plot.set_layout(apply_theme(layout, args.theme()));
    Ok(plot)
}

/// Map a `--location-mode` name to a plotly [`LocationMode`].
fn parse_location_mode(name: &str) -> CliResult<LocationMode> {
    match name.to_ascii_lowercase().as_str() {
        "iso3" | "iso-3" => Ok(LocationMode::Iso3),
        "usa-states" | "usa_states" | "us-states" => Ok(LocationMode::UsaStates),
        "country-names" | "country_names" | "names" => Ok(LocationMode::CountryNames),
        "geojson-id" | "geojson_id" | "geojson" => Ok(LocationMode::GeoJsonId),
        other => fail_incorrectusage_clierror!(
            "Unknown --location-mode '{other}'. Use iso3, usa-states, country-names, or \
             geojson-id."
        ),
    }
}

/// Map a `--color-scale` name to a plotly [`ColorScalePalette`].
fn parse_color_scale(name: &str) -> CliResult<ColorScalePalette> {
    match name.to_ascii_lowercase().as_str() {
        "viridis" => Ok(ColorScalePalette::Viridis),
        "cividis" => Ok(ColorScalePalette::Cividis),
        "greys" | "grays" => Ok(ColorScalePalette::Greys),
        "greens" => Ok(ColorScalePalette::Greens),
        "blues" => Ok(ColorScalePalette::Blues),
        "reds" => Ok(ColorScalePalette::Reds),
        "ylgnbu" => Ok(ColorScalePalette::YlGnBu),
        "ylorrd" => Ok(ColorScalePalette::YlOrRd),
        "bluered" => Ok(ColorScalePalette::Bluered),
        "rdbu" => Ok(ColorScalePalette::RdBu),
        "portland" => Ok(ColorScalePalette::Portland),
        "electric" => Ok(ColorScalePalette::Electric),
        "jet" => Ok(ColorScalePalette::Jet),
        "hot" => Ok(ColorScalePalette::Hot),
        "blackbody" => Ok(ColorScalePalette::Blackbody),
        "earth" => Ok(ColorScalePalette::Earth),
        "picnic" => Ok(ColorScalePalette::Picnic),
        "rainbow" => Ok(ColorScalePalette::Rainbow),
        other => fail_incorrectusage_clierror!(
            "Unknown --color-scale '{other}'. Use viridis, cividis, greys, greens, blues, reds, \
             ylgnbu, ylorrd, bluered, rdbu, portland, electric, jet, hot, blackbody, earth, \
             picnic, or rainbow."
        ),
    }
}

/// Map a `--style` name to a token-free MapLibre [`MapStyle`] for `viz choropleth --map`.
fn parse_choropleth_map_style(name: &str) -> CliResult<MapStyle> {
    match name.to_ascii_lowercase().as_str() {
        "carto-positron" => Ok(MapStyle::CartoPositron),
        "carto-positron-nolabels" => Ok(MapStyle::CartoPositronNoLabels),
        "carto-darkmatter" | "carto-dark-matter" => Ok(MapStyle::CartoDarkMatter),
        "carto-darkmatter-nolabels" | "carto-dark-matter-nolabels" => {
            Ok(MapStyle::CartoDarkMatterNoLabels)
        },
        "carto-voyager" => Ok(MapStyle::CartoVoyager),
        "carto-voyager-nolabels" => Ok(MapStyle::CartoVoyagerNoLabels),
        "open-street-map" | "osm" => Ok(MapStyle::OpenStreetMap),
        "dark" => Ok(MapStyle::Dark),
        "light" => Ok(MapStyle::Light),
        "white-bg" => Ok(MapStyle::WhiteBg),
        "satellite" => Ok(MapStyle::Satellite),
        "basic" => Ok(MapStyle::Basic),
        other => fail_incorrectusage_clierror!(
            "Unknown --style '{other}' for `viz choropleth --map`. Use carto-positron, \
             carto-darkmatter, carto-voyager (each also as a label-free '-nolabels' variant, e.g. \
             carto-positron-nolabels), open-street-map, dark, light, white-bg, satellite, or \
             basic."
        ),
    }
}

/// Count of distinct `[x, y]` vertices in a GeoJSON ring (`serde_json` positions), ignoring the
/// closing duplicate and any non-numeric entries. A valid polygon ring needs at least 3.
fn ring_distinct_vertices(ring: &serde_json::Value) -> usize {
    let Some(arr) = ring.as_array() else {
        return 0;
    };
    let mut seen = std::collections::HashSet::with_capacity(arr.len());
    for pos in arr {
        if let Some(p) = pos.as_array()
            && p.len() >= 2
            && let (Some(x), Some(y)) = (p[0].as_f64(), p[1].as_f64())
        {
            seen.insert((x.to_bits(), y.to_bits()));
        }
    }
    seen.len()
}

/// SIGNED planar (shoelace) area of a GeoJSON ring in raw lon/lat degree space: positive when the
/// ring is wound counter-clockwise, negative when clockwise. Never a real geographic area — its
/// magnitude only ranks rings (largest = exterior) and its sign only drives the winding
/// normalization in [`repair_polygon_rings`].
fn ring_signed_area(ring: &serde_json::Value) -> f64 {
    let Some(arr) = ring.as_array() else {
        return 0.0;
    };
    let pts: Vec<(f64, f64)> = arr
        .iter()
        .filter_map(|pos| {
            let p = pos.as_array()?;
            Some((p.first()?.as_f64()?, p.get(1)?.as_f64()?))
        })
        .collect();
    if pts.len() < 3 {
        return 0.0;
    }
    // `windows(2)` covers the closing edge only when the ring is already closed (last == first).
    // GeoJSON rings SHOULD be closed but this codebase explicitly tolerates unclosed ones
    // (geojson_rings_to_closed appends the first vertex when missing), and dropping the closing
    // term makes the shoelace sum arbitrarily wrong for a ring offset from the origin — which can
    // rank a hole above its exterior in repair_polygon_rings and hand plotly the inverted ring
    // order that sanitize_geojson_for_render exists to prevent.
    let mut closed;
    let pts = if pts.first() == pts.last() {
        &pts[..]
    } else {
        closed = Vec::with_capacity(pts.len() + 1);
        closed.extend_from_slice(&pts);
        closed.push(pts[0]);
        &closed[..]
    };
    let mut a = 0.0;
    for w in pts.windows(2) {
        let [(x0, y0), (x1, y1)] = w else {
            continue;
        };
        a += x0 * y1 - x1 * y0;
    }
    a / 2.0
}

/// Repair one polygon's rings (a `Vec` of ring `Value`s) for plotly rendering: drop degenerate
/// rings — fewer than 3 distinct vertices, OR zero planar area (a collinear ring whose vertices are
/// distinct but lie on a line) — order the survivors largest-area-first so ring[0] is the exterior,
/// then normalize each ring's WINDING (see below). A zero-area ring is non-renderable either way:
/// as an exterior it leaves plotly's `MultiPolygon` centroid with no positive-area sub-polygon (the
/// crash this fix prevents), and as a hole it encloses nothing. Returns `None` when nothing
/// renderable remains. Ring `Value`s are moved and reversed in place, never rebuilt, so coordinate
/// precision is preserved exactly.
///
/// WINDING: plotly's `choropleth` trace projects with d3-geo, whose spherical polygon fill takes
/// the interior to be the side left of the ring direction — so a sub-hemisphere exterior ring must
/// be CLOCKWISE. This is the OPPOSITE of RFC 7946 §3.1.6, which mandates counter-clockwise
/// exteriors, so *spec-conforming* GeoJSON renders inverted: the fill floods the whole map and the
/// real regions punch out of it. Measured on a donut fixture: CW exterior + CCW hole draws the ring
/// and its hole correctly, while the RFC form fills the globe's complement. We therefore force
/// ring[0] clockwise and every hole counter-clockwise. The tile-based `choroplethmap` renderer is
/// winding-insensitive (it derives exterior-vs-hole from the area sign), so this is safe for both
/// paths, and the even-odd point-in-polygon binning in [`build_pip_features`] is winding- and
/// ring-order-independent, so region counts are unaffected.
fn repair_polygon_rings(rings: Vec<serde_json::Value>) -> Option<Vec<serde_json::Value>> {
    let mut kept: Vec<(f64, serde_json::Value)> = rings
        .into_iter()
        .filter(|r| ring_distinct_vertices(r) >= 3)
        .map(|r| (ring_signed_area(&r), r))
        .filter(|(area, _)| *area != 0.0)
        .collect();
    if kept.is_empty() {
        return None;
    }
    // largest area first = exterior, holes after. Stable: equal-area rings keep input order, so a
    // conforming polygon (exterior already first and largest, holes strictly smaller) is untouched.
    kept.sort_by(|a, b| {
        b.0.abs()
            .partial_cmp(&a.0.abs())
            .unwrap_or(std::cmp::Ordering::Equal)
    });
    // exterior (index 0) must wind clockwise (signed area < 0), holes counter-clockwise (> 0).
    Some(
        kept.into_iter()
            .enumerate()
            .map(|(i, (signed_area, mut ring))| {
                let want_clockwise = i == 0;
                let is_clockwise = signed_area < 0.0;
                if is_clockwise != want_clockwise
                    && let Some(arr) = ring.as_array_mut()
                {
                    arr.reverse();
                }
                ring
            })
            .collect(),
    )
}

/// Repair a single GeoJSON geometry in place for plotly rendering, returning the cleaned geometry
/// or `None` when no renderable polygon survives. Recurses into `GeometryCollection`; non-areal
/// geometries (Point/LineString/…) pass through unchanged.
fn repair_render_geometry(geom: &mut serde_json::Value) -> Option<serde_json::Value> {
    let gtype = geom.get("type")?.as_str()?.to_owned();
    match gtype.as_str() {
        "Polygon" => {
            let rings = geom.get_mut("coordinates")?.as_array_mut()?;
            let fixed = repair_polygon_rings(std::mem::take(rings))?;
            Some(serde_json::json!({ "type": "Polygon", "coordinates": fixed }))
        },
        "MultiPolygon" => {
            let polys = geom.get_mut("coordinates")?.as_array_mut()?;
            let mut out = Vec::with_capacity(polys.len());
            for poly in std::mem::take(polys) {
                if let serde_json::Value::Array(rings) = poly
                    && let Some(fixed) = repair_polygon_rings(rings)
                {
                    out.push(serde_json::Value::Array(fixed));
                }
            }
            (!out.is_empty())
                .then(|| serde_json::json!({ "type": "MultiPolygon", "coordinates": out }))
        },
        "GeometryCollection" => {
            let geoms = geom.get_mut("geometries")?.as_array_mut()?;
            let mut out = Vec::with_capacity(geoms.len());
            for mut g in std::mem::take(geoms) {
                if let Some(r) = repair_render_geometry(&mut g) {
                    out.push(r);
                }
            }
            (!out.is_empty())
                .then(|| serde_json::json!({ "type": "GeometryCollection", "geometries": out }))
        },
        _ => Some(geom.clone()),
    }
}

/// Sanitize a GeoJSON `FeatureCollection` in place so its polygons render in plotly's choropleth /
/// choroplethmap geometry pipeline. That pipeline assumes well-formed polygons — ring[0] is the
/// exterior and every hole lies inside it — and *throws* (blanking the WHOLE panel, not just the
/// one region) when a feature violates this. Real-world GeoJSON does: e.g. a zero-area sliver
/// listed first (becoming the "exterior") with the true boundary listed as a later "hole". For each
/// Polygon/MultiPolygon we drop degenerate rings, reorder so the largest ring is the exterior, and
/// normalize ring winding to what d3-geo needs (see [`repair_polygon_rings`]); features left with
/// no renderable polygon are dropped. The drop/reorder steps are no-ops for conforming GeoJSON (a
/// hole is always strictly smaller than its exterior); the winding step rewrites RFC
/// 7946-conforming rings, which d3-geo would otherwise render inverted. The even-odd
/// point-in-polygon binning in [`build_pip_features`] is winding- and ring-order-independent, so
/// region counts are unaffected.
fn sanitize_geojson_for_render(geojson: &mut serde_json::Value) {
    let Some(features) = geojson.get_mut("features").and_then(|f| f.as_array_mut()) else {
        return;
    };
    features.retain_mut(|feature| {
        let Some(geom) = feature.get_mut("geometry") else {
            return true;
        };
        if geom.is_null() {
            return true;
        }
        match repair_render_geometry(geom) {
            Some(repaired) => {
                *geom = repaired;
                true
            },
            None => false,
        }
    });
}

/// A `QSV_GEOJSON_SHORTCUTS` entry: a convenience alias mapping a short name to a GeoJSON
/// `path` (local file or http(s) URL) and an optional `id` (the feature-id-key to use).
#[derive(serde::Deserialize)]
struct GeojsonShortcut {
    path: String,
    #[serde(default)]
    id:   Option<String>,
}

/// Resolve a `--geojson` value as a `QSV_GEOJSON_SHORTCUTS` entry name.
/// Returns the resolved path/URL and the optional feature-id-key carried by the shortcut.
fn lookup_geojson_shortcut(name: &str) -> CliResult<(String, Option<String>)> {
    let raw = std::env::var("QSV_GEOJSON_SHORTCUTS").map_err(|_| {
        crate::CliError::Other(format!(
            "--geojson '{name}' is not an existing file or http(s) URL, and no \
             QSV_GEOJSON_SHORTCUTS are defined."
        ))
    })?;
    let map: std::collections::HashMap<String, GeojsonShortcut> = serde_json::from_str(&raw)
        .map_err(|e| {
            crate::CliError::Other(format!("QSV_GEOJSON_SHORTCUTS is not valid JSON: {e}"))
        })?;
    match map.get(name) {
        Some(s) => Ok((s.path.clone(), s.id.clone())),
        None => {
            let mut keys: Vec<&str> = map.keys().map(String::as_str).collect();
            keys.sort_unstable();
            fail_incorrectusage_clierror!(
                "Unknown --geojson shortcut '{name}'. Available QSV_GEOJSON_SHORTCUTS: {}.",
                keys.join(", ")
            )
        },
    }
}

/// Resolve a shortcut name and validate the GeoJSON up front (fail-fast, before any plotting).
/// Rewrites `args.flag_geojson` to the concrete path/URL and, when the value resolved via a
/// shortcut that carries an `id`, adopts it as the feature-id-key unless the user explicitly
/// passed `--feature-id-key` (see `feature_id_key_explicit`).
fn resolve_and_validate_geojson(
    args: &mut Args,
    feature_id_key_explicit: bool,
) -> CliResult<Option<serde_json::Value>> {
    let Some(spec) = args.flag_geojson.clone() else {
        return Ok(None);
    };

    // Disambiguate: a direct source is an http(s) URL or an existing local file; anything else is
    // treated as a shortcut NAME. (More robust than "try to load, fall back on failure": a 404'd
    // URL or an existing-but-broken file must NOT be silently reinterpreted as a shortcut name.)
    let is_url = spec.starts_with("http://") || spec.starts_with("https://");
    let (resolved, shortcut_id) = if is_url || std::path::Path::new(&spec).is_file() {
        (spec, None)
    } else {
        lookup_geojson_shortcut(&spec)?
    };

    // A shortcut's id fills in only when the user didn't explicitly pass --feature-id-key; an
    // explicit flag (including `--feature-id-key id`) always wins.
    if let Some(id) = shortcut_id
        && !feature_id_key_explicit
    {
        args.flag_feature_id_key = Some(id);
    }

    // Validate: fetch/read + FeatureCollection + the feature-id-key resolves on >=1 feature.
    let geojson = load_geojson(&resolved)?;
    let fc = geojson::FeatureCollection::deserialize(&geojson).map_err(|e| {
        crate::CliError::Other(format!(
            "--geojson '{resolved}' is not a valid GeoJSON FeatureCollection: {e}"
        ))
    })?;
    let key = args.flag_feature_id_key.as_deref().unwrap_or("id");
    if !fc
        .features
        .iter()
        .any(|f| feature_id_by_path(f, key).is_some())
    {
        return fail_incorrectusage_clierror!(
            "--feature-id-key '{key}' resolves on no feature in --geojson '{resolved}'. Use e.g. \
             'id' or 'properties.<name>'."
        );
    }
    // The key resolves somewhere, but every --geojson consumer (point-in-polygon binning in
    // build_pip_features and the native plotly choropleth map) needs at least one
    // Polygon/MultiPolygon feature that ALSO carries the id — reusing geojson_value_to_polygons
    // here keeps this fail-fast check in exact parity with build_pip_features' late check, so a
    // geojson whose id only lands on non-polygon features is rejected up front rather than after
    // all the expensive stats/dictionary work.
    if !fc.features.iter().any(|f| {
        feature_id_by_path(f, key).is_some()
            && f.geometry
                .as_ref()
                .is_some_and(|g| !geojson_value_to_polygons(&g.value).is_empty())
    }) {
        return fail_incorrectusage_clierror!(
            "--geojson '{resolved}' has no usable Polygon/MultiPolygon features with a '{key}' \
             id. Check --feature-id-key (e.g. 'id' or 'properties.<name>')."
        );
    }

    args.flag_geojson = Some(resolved);
    // hand the parsed document back so the plotting paths reuse it instead of re-fetching and
    // re-parsing: it was loaded 3x per standalone choropleth run (validation, binning, trace
    // geometry) and 2-3x again under `viz smart`. Beyond the wasted HTTP GETs and parse time,
    // a non-deterministic URL made those loads disagree — points binned against one fetch while
    // the map drew another's geometry.
    Ok(Some(geojson))
}

/// Load a GeoJSON `FeatureCollection` from a local file path or an http(s) URL into a JSON value.
fn load_geojson(spec: &str) -> CliResult<serde_json::Value> {
    let bytes = if spec.starts_with("http://") || spec.starts_with("https://") {
        use std::io::Read as _;

        // reqwest's default client has NO connect or read timeout, so a server that accepts the
        // connection and then stalls hangs qsv indefinitely with no way out. Honors QSV_TIMEOUT.
        let timeout = std::time::Duration::from_secs(
            util::timeout_secs(GEOJSON_FETCH_TIMEOUT_SECS).map_err(crate::CliError::Other)?,
        );
        let client = reqwest::blocking::Client::builder()
            .timeout(timeout)
            .connect_timeout(timeout)
            .build()
            .map_err(|e| crate::CliError::Other(format!("Could not build HTTP client: {e}")))?;
        let mut resp = client
            .get(spec)
            .send()
            .and_then(reqwest::blocking::Response::error_for_status)
            .map_err(|e| {
                crate::CliError::Other(format!("Failed to fetch --geojson URL '{spec}': {e}"))
            })?;
        // read through a bounded `take` rather than `bytes()`: an endless (or merely enormous)
        // body would otherwise be buffered into memory unchecked. Read one byte past the cap so
        // exceeding it is distinguishable from exactly reaching it.
        let mut buf: Vec<u8> = Vec::new();
        resp.by_ref()
            .take(GEOJSON_MAX_BYTES as u64 + 1)
            .read_to_end(&mut buf)
            .map_err(|e| {
                crate::CliError::Other(format!("Failed to read --geojson body from '{spec}': {e}"))
            })?;
        if buf.len() > GEOJSON_MAX_BYTES {
            return Err(crate::CliError::Other(format!(
                "--geojson '{spec}' exceeds the {} MB download limit.",
                GEOJSON_MAX_BYTES / 1_000_000
            )));
        }
        buf
    } else {
        std::fs::read(spec).map_err(|e| {
            crate::CliError::Other(format!("Failed to read --geojson '{spec}': {e}"))
        })?
    };
    let mut geojson: serde_json::Value = serde_json::from_slice(&bytes).map_err(|e| {
        crate::CliError::Other(format!("--geojson '{spec}' is not valid JSON: {e}"))
    })?;
    // repair malformed polygon ring ordering before any consumer embeds the geometry into a plotly
    // trace — a single feature whose "exterior" is a degenerate/inner ring otherwise throws in
    // plotly's geometry pipeline and blanks the entire choropleth panel.
    sanitize_geojson_for_render(&mut geojson);
    Ok(geojson)
}

/// One GeoJSON feature reduced to its polygon rings for point-in-polygon binning. `polygons` holds
/// one entry per polygon (a `MultiPolygon` yields several); each polygon is a list of linear rings
/// (ring 0 is the exterior, the rest are holes); each ring is a closed list of `[lon, lat]`
/// vertices. `bbox` is `[min_lon, min_lat, max_lon, max_lat]` over all vertices, for cheap
/// candidate prefiltering.
struct PipFeature {
    id:                 String,
    /// Human-readable region label (e.g. `properties.name` → "Kagoshima") for hover, when a name
    /// key is given or auto-detected; HTML-escaped at construction. `None` when no name resolves.
    name:               Option<String>,
    polygons:           Vec<Vec<Vec<[f64; 2]>>>,
    bbox:               [f64; 4],
    /// True when at least one of this feature's polygons genuinely crosses the ±180° antimeridian
    /// and was continuity-unwrapped into a contiguous extended longitude frame (vertices may lie
    /// beyond ±180°, within `(-180, 540)`), so the planar ray-cast and bbox prefilter stay
    /// contiguous across the dateline seam. Queries against such a feature must be probed at both
    /// 360°-equivalent longitudes via [`PipFeature::candidate_lons`]. Pole-enclosing rings pre-cut
    /// along the seam (e.g. Natural Earth's Antarctica) are deliberately KEPT RAW — their planar
    /// form is already ray-cast-correct — and don't set this flag by themselves.
    wraps_antimeridian: bool,
}

impl PipFeature {
    /// The query longitudes to test against this feature. An antimeridian-crossing feature stores
    /// continuity-unwrapped vertices extending past ±180°, so the query is probed at both of its
    /// 360°-equivalent representations — at most one can fall inside any single planar polygon
    /// (per-polygon spans are < 360°), so OR-ing the probes can't double-count. Non-wrapping
    /// features are tested at the raw longitude only.
    fn candidate_lons(&self, lon: f64) -> [Option<f64>; 2] {
        if self.wraps_antimeridian {
            return [Some(lon), Some(lon + 360.0)];
        }
        // A feature can sit ADJACENT to the antimeridian without crossing it (e.g. spanning
        // 179.5..180.0), in which case it carries no wrap flag — but a query just the other side
        // of the seam is still only a fraction of a degree away going east, while the raw
        // subtraction measures it as ~359.5 degrees the long way round. Distances are computed in
        // raw degree space, so without a second probe such a point measures ~40,000 km, exceeds
        // any snap cap, and is silently dropped as "outside every region".
        //
        // Only offer the alternate when the raw longitude gap already exceeds 180 degrees — i.e.
        // the short way round is genuinely the other direction. That is exactly the seam case, so
        // ordinary features pay one comparison and containment cannot gain a false positive (the
        // shifted query only enters the feature's span when the true angular distance is small).
        let (west, east) = (self.bbox[0], self.bbox[2]);
        let alt = if lon < west - 180.0 {
            Some(lon + 360.0)
        } else if lon > east + 180.0 {
            Some(lon - 360.0)
        } else {
            None
        };
        [Some(lon), alt]
    }
}

/// Resolve a GeoJSON feature's id by a dotted `--feature-id-key` path. Supports the top-level
/// `"id"`, `"properties.<...>"` paths (mirroring plotly's `featureidkey` convention), and any
/// other key as a top-level foreign member (e.g. a bare `"name"` on each feature — TopoJSON-style
/// exports carry names there rather than under `properties`). Strings and numbers both coerce to
/// `String` (CSV cells and plotly match feature ids as strings). Returns `None` when the path is
/// absent or the value isn't a string/number.
fn feature_id_by_path(feature: &geojson::Feature, key: &str) -> Option<String> {
    let coerce = |v: &serde_json::Value| -> Option<String> {
        match v {
            serde_json::Value::String(s) => Some(s.clone()),
            serde_json::Value::Number(n) => Some(n.to_string()),
            _ => None,
        }
    };
    if key == "id" {
        return feature.id.as_ref().map(|id| match id {
            geojson::feature::Id::String(s) => s.clone(),
            geojson::feature::Id::Number(n) => n.to_string(),
        });
    }
    let (root, segs) = match key.strip_prefix("properties.") {
        Some(rest) => {
            let mut segs = rest.split('.');
            (feature.properties.as_ref()?.get(segs.next()?)?, segs)
        },
        // no "properties." prefix: a top-level foreign member
        None => {
            let mut segs = key.split('.');
            (feature.foreign_members.as_ref()?.get(segs.next()?)?, segs)
        },
    };
    let mut cur = root;
    for seg in segs {
        cur = cur.get(seg)?;
    }
    coerce(cur)
}

/// HTML-escape a string for use inside a plotly hover label (which renders `<b>`/`<br>` as markup).
/// Escapes `&` first so region names/measure labels with `&`, `<`, `>` show as literal text instead
/// of being parsed as tags. Note: this does not neutralize a pathological `%{...}` in a label (out
/// of scope — choropleth hover here is pre-rendered text, not a plotly template).
fn escape_hover(s: &str) -> String {
    s.replace('&', "&amp;")
        .replace('<', "&lt;")
        .replace('>', "&gt;")
}

/// Escape a literal `%` for embedding inside a plotly `hovertemplate` (a bare `%` would be
/// misparsed as the start of a `%{...}` template token, so it's doubled). Feed already
/// `escape_hover`-ed text; per-point values referenced via `%{text}`/`%{customdata}` are data,
/// not template, and need no such escaping.
fn escape_template_pct(text: &str) -> String {
    text.replace('%', "%%")
}

/// Trace-level hovertemplate for smart map/geo point traces: the per-point `text` payload (see
/// `map_hover_text_values`) followed by the point's coordinates, which plotly formats from the
/// lat/lon data arrays themselves. Factoring the markup and coordinate line into ONE trace-level
/// template — instead of baking them into every embedded point's hover string — trims ~25 bytes
/// per point from the emitted HTML. `<extra></extra>` suppresses the trace-name box.
const MAP_HOVER_TEMPLATE: &str = "%{text}%{lat:.4f}, %{lon:.4f}<extra></extra>";

/// Per-point `text` values for `MAP_HOVER_TEMPLATE`: each non-empty hover string gets a trailing
/// `<br>` so the template's coordinate line lands on its own row, while a point with no
/// contributed content collapses to a bare coordinate hover with no leading blank line —
/// hovertemplate has no conditionals, so the separator must live in the data.
fn map_hover_text_values(hover: &[String]) -> Vec<String> {
    hover
        .iter()
        .map(|t| {
            if t.is_empty() {
                String::new()
            } else {
                format!("{t}<br>")
            }
        })
        .collect()
}

/// Format a measure value for a hover label: a whole number prints without a decimal point
/// (`65`), otherwise up to 3 decimals with trailing zeros trimmed (`12.5`, `0.333`). The integer
/// part is grouped with thousands separators (`1,234,567`).
fn fmt_measure(v: f64) -> String {
    let s = if v.is_finite() && v.fract() == 0.0 && v.abs() < 1e15 {
        format!("{v:.0}")
    } else {
        let s = format!("{v:.3}");
        s.trim_end_matches('0').trim_end_matches('.').to_string()
    };
    group_thousands(&s)
}

/// Insert `,` thousands separators into the integer part of a formatted number string, preserving
/// an optional leading sign and any `.fraction` suffix (`-1234.5` -> `-1,234.5`). Used by every
/// pre-rendered hover label so large magnitudes stay readable.
fn group_thousands(s: &str) -> String {
    let (sign, rest) = match s.strip_prefix('-') {
        Some(r) => ("-", r),
        None => ("", s),
    };
    let (int_part, frac_part) = match rest.split_once('.') {
        Some((i, f)) => (i, Some(f)),
        None => (rest, None),
    };
    let len = int_part.len();
    let mut grouped = String::with_capacity(len + len / 3);
    for (i, ch) in int_part.chars().enumerate() {
        if i > 0 && (len - i) % 3 == 0 {
            grouped.push(',');
        }
        grouped.push(ch);
    }
    match frac_part {
        Some(f) => format!("{sign}{grouped}.{f}"),
        None => format!("{sign}{grouped}"),
    }
}

/// Build the per-region hover text for a choropleth trace, aligned 1:1 to `locs`. Each label has:
/// a bold region name + id (`<b>Kagoshima</b> (JP46)`) when `names[i]` is non-empty, else the bold
/// id alone; the value labeled with its measure (`count: 65`); the share of the total
/// (`15.6% of total`) when `include_pct` (count/sum aggregations only); and the rank by descending
/// value (`rank 1 of 47`). `names` (when given) and `measure_label` are HTML-escaped by the caller
/// / here respectively; ids are escaped here. Attached via `.hover_text_array(..)` +
/// `HoverInfo::Text`, so the whole string is pre-rendered (no plotly template tokens).
fn choropleth_hover_text(
    locs: &[String],
    z: &[f64],
    names: Option<&[String]>,
    measure_label: &str,
    include_pct: bool,
) -> Vec<String> {
    let n = locs.len();
    let label = escape_hover(measure_label);
    let total: f64 = if include_pct {
        z.iter().copied().filter(|v| v.is_finite()).sum()
    } else {
        0.0
    };
    // rank by descending z (positional, 1-based; ties break by position).
    let mut order: Vec<usize> = (0..n).collect();
    order.sort_by(|&a, &b| z[b].partial_cmp(&z[a]).unwrap_or(std::cmp::Ordering::Equal));
    let mut rank = vec![0_usize; n];
    for (pos, &i) in order.iter().enumerate() {
        rank[i] = pos + 1;
    }
    (0..n)
        .map(|i| {
            let id = escape_hover(&locs[i]);
            let name = names
                .and_then(|ns| ns.get(i))
                .map(String::as_str)
                .filter(|s| !s.is_empty());
            let mut lines: Vec<String> = Vec::with_capacity(4);
            match name {
                Some(name) => lines.push(format!("<b>{name}</b> ({id})")),
                None => lines.push(format!("<b>{id}</b>")),
            }
            lines.push(format!("{label}: {}", fmt_measure(z[i])));
            if include_pct && total > 0.0 {
                lines.push(
                    t!(
                        "viz.hover.pct_of_total",
                        q_pct = format!("{:.1}", z[i] / total * 100.0)
                    )
                    .into_owned(),
                );
            }
            lines.push(t!("viz.hover.rank_of", q_rank = rank[i], q_total = n).into_owned());
            lines.join("<br>")
        })
        .collect()
}

/// Build the dataset portion of a map point's hover (no coordinates, no geocoded place): a bold
/// identifier line (when `id_value` is non-empty), then one `label: value` line per non-empty
/// `extra` field. Every value/label is HTML-escaped. Any reverse-geocoded place is added by
/// `assemble_map_hover`; the coordinate line comes from `MAP_HOVER_TEMPLATE` at render time.
fn format_map_dataset_line(id_value: Option<&str>, extra: &[(String, String)]) -> String {
    let mut lines: Vec<String> = Vec::with_capacity(extra.len() + 1);
    if let Some(id) = id_value.filter(|s| !s.is_empty()) {
        lines.push(format!("<b>{}</b>", escape_hover(id)));
    }
    for (label, value) in extra {
        if !value.is_empty() {
            lines.push(format!("{}: {}", escape_hover(label), escape_hover(value)));
        }
    }
    lines.join("<br>")
}

/// Assemble a map point's hover string from its dataset line (see `format_map_dataset_line`) and
/// the reverse-geocoded place (`geo_place`, already gap-filtered + escaped; empty when none).
/// When the dataset has no identifier column (`has_id` is false) the geocoded place serves as the
/// bold identifier; otherwise it's an additional location-context line. No coordinate line —
/// `MAP_HOVER_TEMPLATE` appends the coordinates at render time from the lat/lon data arrays, so
/// they aren't duplicated into every embedded hover string. Attached via `.text_array(..)` +
/// `.hover_template(MAP_HOVER_TEMPLATE)` (through `map_hover_text_values`).
///
/// `photo_count` (0 unless `--photos` embedded URLs for this point) adds the affordance line that
/// makes the hover-dwell preview discoverable: without it the 2-second dwell is an invisible
/// feature nobody would think to try. Points with no photo get NO line, so the hover stays clean
/// for the ~69% of Boston 311 cases that were closed without one.
fn assemble_map_hover(
    dataset_line: &str,
    has_id: bool,
    geo_place: &str,
    photo_count: usize,
) -> String {
    let mut lines: Vec<String> = Vec::with_capacity(3);
    if !has_id && !geo_place.is_empty() {
        // no identifier column: the reverse-geocoded place becomes the (bold) identifier
        lines.push(format!("<b>{geo_place}</b>"));
        if !dataset_line.is_empty() {
            lines.push(dataset_line.to_string());
        }
    } else {
        if !dataset_line.is_empty() {
            lines.push(dataset_line.to_string());
        }
        if !geo_place.is_empty() {
            lines.push(geo_place.to_string());
        }
    }
    if photo_count > 0 {
        let plural = if photo_count == 1 { "" } else { "s" };
        lines.push(format!(
            "<i>{photo_count} photo{plural} - keep hovering to view</i>"
        ));
    }
    lines.join("<br>")
}

/// Format a reverse-geocoded `GeoLabel` into a "City, County, Admin1, Country" hover line for
/// gap-filling — dataset fields win, so a component is dropped when (a) a chosen dataset column
/// already covers it by concept (`sup_*`), or (b) its value already appears in this point's
/// `dataset_line` (so e.g. a city-name identifier isn't echoed by the geocoded city). When
/// `verbose` (driven by `--smarter`) and the place resolved to a US location, the combined 5-digit
/// county FIPS (else the 2-digit state FIPS) is appended as "(FIPS …)". Components are
/// HTML-escaped.
#[cfg(feature = "geocode")]
fn geocode_hover_place(
    label: &crate::cmd::geocode::GeoLabel,
    sup_city: bool,
    sup_admin1: bool,
    sup_country: bool,
    verbose: bool,
    dataset_line: &str,
) -> String {
    let dataset_lc = dataset_line.to_lowercase();
    let shown = |s: &str| !s.trim().is_empty() && dataset_lc.contains(&s.to_lowercase());
    // county (admin2) sits between city and admin1 (state) — the natural City, County, State,
    // Country hierarchy. It's always-on — there's no `geo.county` concept to gap-fill against (only
    // city/state/country exist in the dictionary vocab) — but the `shown` value-match still drops
    // it when the county literally appears in this point's dataset line. Geonames US county
    // names already include "County", so no suffix is appended here.
    let place = [
        (!sup_city && !shown(&label.city)).then_some(label.city.as_str()),
        (!shown(&label.admin2)).then_some(label.admin2.as_str()),
        (!sup_admin1 && !shown(&label.admin1)).then_some(label.admin1.as_str()),
        (!sup_country && !shown(&label.country)).then_some(label.country.as_str()),
    ]
    .into_iter()
    .flatten()
    .filter(|s| !s.trim().is_empty())
    .map(escape_hover)
    .collect::<Vec<_>>()
    .join(", ");

    // FIPS is a verbose detail (US only): prefer the full county FIPS, else the state FIPS. Only
    // appended to an already-resolved place so it never appears as a bare "(FIPS …)".
    if verbose && !place.is_empty() {
        let fips = if !label.us_county_fips.is_empty() {
            Some(label.us_county_fips.as_str())
        } else if !label.us_state_fips.is_empty() {
            Some(label.us_state_fips.as_str())
        } else {
            None
        };
        if let Some(fips) = fips {
            return format!("{place} (FIPS {fips})");
        }
    }
    place
}

/// Build per-location region names aligned 1:1 to `locs` from binned/matched GeoJSON features: the
/// feature `name` for each location id, or `""` when that region has no name (so
/// [`choropleth_hover_text`] falls back to the bold id). Returns `None` when NO feature carries a
/// name, so the caller uses the universal id-only hover.
fn aligned_region_names(features: &[PipFeature], locs: &[String]) -> Option<Vec<String>> {
    if !features.iter().any(|f| f.name.is_some()) {
        return None;
    }
    let map: std::collections::HashMap<&str, &str> = features
        .iter()
        .filter_map(|f| f.name.as_deref().map(|n| (f.id.as_str(), n)))
        .collect();
    Some(
        locs.iter()
            .map(|id| {
                map.get(id.as_str())
                    .map_or(String::new(), |n| (*n).to_string())
            })
            .collect(),
    )
}

/// Convert a `geojson::GeometryValue::Polygon` ring set to closed `[lon, lat]` rings (appending the
/// first vertex when a ring isn't already closed, so even-odd ray-casting via `windows(2)` covers
/// every edge). Rings with fewer than 3 distinct vertices are dropped (degenerate point/line
/// rings).
fn geojson_rings_to_closed(poly: &[Vec<geojson::Position>]) -> Vec<Vec<[f64; 2]>> {
    poly.iter()
        .filter_map(|ring| {
            let mut pts: Vec<[f64; 2]> = ring
                .iter()
                .filter(|p| p.len() >= 2)
                .map(|p| [p[0], p[1]])
                .collect();
            // require at least 3 DISTINCT vertices: a ring with fewer is degenerate (a point or a
            // line segment, zero area) — it contributes no even-odd crossings and would only serve
            // as a phantom nearest-feature snap target. Short-circuits once 3 distinct points seen.
            let mut distinct: Vec<[f64; 2]> = Vec::with_capacity(3);
            for &p in &pts {
                if !distinct.contains(&p) {
                    distinct.push(p);
                    if distinct.len() >= 3 {
                        break;
                    }
                }
            }
            if distinct.len() < 3 {
                return None;
            }
            if pts.first() != pts.last() {
                pts.push(pts[0]);
            }
            Some(pts)
        })
        .collect()
}

/// Flatten a `geojson::GeometryValue` into a list of polygons (each = exterior + hole rings).
/// Handles Polygon, `MultiPolygon`, and nested `GeometryCollection`; other geometry types yield
/// nothing.
fn geojson_value_to_polygons(value: &geojson::GeometryValue) -> Vec<Vec<Vec<[f64; 2]>>> {
    match value {
        geojson::GeometryValue::Polygon { coordinates } => {
            let rings = geojson_rings_to_closed(coordinates);
            if rings.is_empty() {
                vec![]
            } else {
                vec![rings]
            }
        },
        geojson::GeometryValue::MultiPolygon { coordinates } => coordinates
            .iter()
            .map(|poly| geojson_rings_to_closed(poly))
            .filter(|rings| !rings.is_empty())
            .collect(),
        geojson::GeometryValue::GeometryCollection { geometries } => geometries
            .iter()
            .flat_map(|g| geojson_value_to_polygons(&g.value))
            .collect(),
        _ => vec![],
    }
}

/// Does any edge of this polygon's rings make a genuine ±180° antimeridian jump? True when an
/// edge's endpoints are far apart in raw longitude but CLOSE the short way across the seam (within
/// `MAX_SEAM_GAP_DEG`) — the signature of a dateline crossing (e.g. 179° -> -179°, or a wider
/// Pacific box 160° -> -160°). A genuinely wide but non-crossing polygon (e.g. a -100°..100° span
/// centered on the prime meridian) has a large short-way gap and is deliberately NOT flagged, so
/// its planar interior is binned correctly. When true, the polygon is continuity-unwrapped into a
/// contiguous extended longitude frame before planar point-in-polygon tests, which can't reason
/// across the seam.
fn polygon_crosses_antimeridian(polygon: &[Vec<[f64; 2]>]) -> bool {
    // A dateline crossing connects two vertices that are far apart in raw longitude (>180°) but
    // CLOSE the short way across the ±180° seam. We key off that short-way gap: `360 - |Δlon|`. A
    // real crossing like 179° -> -179° has a 2° seam gap; a wide Pacific box 160° -> -160° has a
    // 40° gap (still a crossing). A prime-meridian-centered wide polygon like -100° -> 100° spans
    // 200° but its short-way gap is 160° — it is a genuinely wide non-crossing region and must NOT
    // be normalized (that would push its prime-meridian interior outside the bbox). The
    // MAX_SEAM_GAP_DEG cutoff (90°) separates the two: normalize when the edge's endpoints are
    // within 90° of each other across the seam (inclusive), i.e. |Δlon| >= 360 - 90 = 270°. The
    // boundary is inclusive so an exactly-90°-gap box (135° -> -135°) is still treated as crossing.
    const MAX_SEAM_GAP_DEG: f64 = 90.0;
    polygon.iter().any(|ring| {
        ring.windows(2).any(|edge| {
            let &[[lon_a, _], [lon_b, _]] = edge else {
                return false;
            };
            (lon_a - lon_b).abs() >= 360.0 - MAX_SEAM_GAP_DEG
        })
    })
}

/// Continuity-unwrap a pre-closed ring across the ±180° antimeridian: each vertex longitude is
/// shifted by the multiple of 360° that puts it within 180° of its predecessor, so a seam-jumping
/// vertex walk (179° -> -179°) becomes contiguous (179° -> 181°) while genuine prime-meridian
/// crossings (0.4° -> -0.2°) stay short — a blanket "shift all negative lons by +360" would turn
/// the latter into a fake ~359°-long edge, scrambling ray-cast parity near 0° for any feature that
/// crosses BOTH meridians. Returns `None` when the unwrapped ring does not arrive back at its
/// starting longitude (a ±360° residue): that's the signature of a pole-enclosing ring pre-cut
/// along the seam (e.g. Natural Earth's Antarctica), whose RAW planar form is already correct for
/// even-odd ray casting and must not be reframed.
fn unwrap_ring_across_seam(ring: &[[f64; 2]]) -> Option<Vec<[f64; 2]>> {
    let mut out = Vec::with_capacity(ring.len());
    let mut prev = ring.first()?[0];
    for &[lon, lat] in ring {
        // FMA version of let lon = lon + ((prev - lon) / 360.0).round() * 360.0;
        let lon = ((prev - lon) / 360.0).round().mul_add(360.0, lon);
        out.push([lon, lat]);
        prev = lon;
    }
    if (out.last()?[0] - out.first()?[0]).abs() > 180.0 {
        return None;
    }
    Some(out)
}

/// Parse a GeoJSON `FeatureCollection` into [`PipFeature`]s keyed by `feature_id_key`. Features
/// missing the id key or lacking a Polygon/MultiPolygon geometry are skipped (and counted in a
/// stderr note). Errors when the input isn't a `FeatureCollection` or yields no usable features.
fn build_pip_features(
    geojson: &serde_json::Value,
    feature_id_key: &str,
    feature_name_key: Option<&str>,
) -> CliResult<Vec<PipFeature>> {
    // deserialize from a REFERENCE: `from_value` needs an owned `Value`, so it would deep-clone
    // every feature, property map and coordinate array first — doubling peak memory on a large
    // boundary file for no benefit. `&Value` implements `Deserializer`, so this borrows instead.
    let fc = geojson::FeatureCollection::deserialize(geojson).map_err(|e| {
        crate::CliError::Other(format!(
            "--geojson is not a valid GeoJSON FeatureCollection: {e}"
        ))
    })?;
    // Resolve the name key once: an explicit --feature-name-key, else auto-detect by probing common
    // name properties on the FIRST feature (a heterogeneous collection whose first feature lacks
    // the property won't auto-detect — use --feature-name-key to force it).
    let name_key: Option<String> = feature_name_key.map(str::to_string).or_else(|| {
        let first = fc.features.first()?;
        [
            "properties.name",
            "properties.NAME",
            "properties.Name",
            "properties.NAME_1",
            "name",
        ]
        .into_iter()
        .find(|k| feature_id_by_path(first, k).is_some())
        .map(str::to_string)
    });
    let mut out: Vec<PipFeature> = Vec::with_capacity(fc.features.len());
    let mut skipped = 0_usize;
    for feature in &fc.features {
        let Some(id) = feature_id_by_path(feature, feature_id_key) else {
            skipped += 1;
            continue;
        };
        let name = name_key
            .as_deref()
            .and_then(|k| feature_id_by_path(feature, k))
            .map(|n| escape_hover(&n));
        let mut polygons = match &feature.geometry {
            Some(g) => geojson_value_to_polygons(&g.value),
            None => Vec::new(),
        };
        if polygons.is_empty() {
            skipped += 1;
            continue;
        }
        // Normalize each antimeridian-crossing polygon into a contiguous extended longitude frame
        // (continuity unwrapping) so its bbox and the planar ray-cast stay contiguous across the
        // ±180° seam. Query longitudes are probed at both 360°-equivalent representations via
        // PipFeature::candidate_lons. Per-POLYGON (not per-feature) so a multipolygon's
        // non-crossing parts keep their raw frame, and per-polygon rings stay in ONE frame so
        // hole subtraction (even-odd parity) still works.
        let mut wraps_antimeridian = false;
        for polygon in &mut polygons {
            if !polygon_crosses_antimeridian(polygon) {
                continue;
            }
            // exterior + holes unwrapped together; a pole-cut ring (unwrap doesn't close) reverts
            // the whole polygon to its raw — already ray-cast-correct — planar form
            let Some(mut rings) = polygon
                .iter()
                .map(|ring| unwrap_ring_across_seam(ring))
                .collect::<Option<Vec<_>>>()
            else {
                continue;
            };
            // continuity anchors each ring at its own first vertex, so a hole can land a full
            // 360° away from its exterior; re-align each hole's frame to the exterior's midpoint
            let mid_lon = |ring: &[[f64; 2]]| -> f64 {
                let (min, max) = ring
                    .iter()
                    .fold((f64::INFINITY, f64::NEG_INFINITY), |(lo, hi), v| {
                        (lo.min(v[0]), hi.max(v[0]))
                    });
                f64::midpoint(min, max)
            };
            if let Some((exterior, holes)) = rings.split_first_mut() {
                let ext_mid = mid_lon(exterior);
                for hole in holes {
                    let k = ((ext_mid - mid_lon(hole)) / 360.0).round();
                    if k != 0.0 {
                        for v in hole.iter_mut() {
                            v[0] += k * 360.0;
                        }
                    }
                }
            }
            // keep every vertex within (-180, 540) so the {lon, lon+360} query candidates can
            // reach it (a ring anchored near -180 that walks west unwraps below -180)
            let min = rings
                .iter()
                .flatten()
                .map(|v| v[0])
                .fold(f64::INFINITY, f64::min);
            if min < -180.0 {
                for v in rings.iter_mut().flatten() {
                    v[0] += 360.0;
                }
            }
            *polygon = rings;
            wraps_antimeridian = true;
        }
        let mut bbox = [
            f64::INFINITY,
            f64::INFINITY,
            f64::NEG_INFINITY,
            f64::NEG_INFINITY,
        ];
        for ring in polygons.iter().flatten() {
            for &[lon, lat] in ring {
                bbox[0] = bbox[0].min(lon);
                bbox[1] = bbox[1].min(lat);
                bbox[2] = bbox[2].max(lon);
                bbox[3] = bbox[3].max(lat);
            }
        }
        out.push(PipFeature {
            id,
            name,
            polygons,
            bbox,
            wraps_antimeridian,
        });
    }
    if out.is_empty() {
        return fail_clierror!(
            "--geojson has no usable Polygon/MultiPolygon features with a '{feature_id_key}' id. \
             Check --feature-id-key (e.g. 'id' or 'properties.<name>')."
        );
    }
    if skipped > 0 {
        viz_note(&format!(
            "viz: skipped {skipped} GeoJSON feature(s) lacking a '{feature_id_key}' id or polygon \
             geometry."
        ));
    }
    Ok(out)
}

/// Even-odd ray-casting test: is `(lon, lat)` inside this single polygon (exterior + holes)? A
/// point in a hole crosses an even number of edges and is correctly reported as outside. Rings are
/// pre-closed by [`geojson_rings_to_closed`], so `windows(2)` enumerates every edge.
fn point_in_polygon(polygon: &[Vec<[f64; 2]>], lon: f64, lat: f64) -> bool {
    let mut inside = false;
    for ring in polygon {
        for edge in ring.windows(2) {
            let &[[xi, yi], [xj, yj]] = edge else {
                continue;
            };
            if ((yi > lat) != (yj > lat)) && (lon < (xj - xi) * (lat - yi) / (yj - yi) + xi) {
                inside = !inside;
            }
        }
    }
    inside
}

/// Is `(lon, lat)` inside this feature (bbox prefilter, then any constituent polygon)? The query
/// is probed at each of the feature's candidate longitudes (both 360°-equivalent representations
/// for antimeridian-crossing features, whose unwrapped vertices may extend past ±180°); a
/// wrong-frame probe lands planar-outside every ring, so OR-ing the probes is safe.
fn feature_contains(feature: &PipFeature, lon: f64, lat: f64) -> bool {
    if lat < feature.bbox[1] || lat > feature.bbox[3] {
        return false;
    }
    feature
        .candidate_lons(lon)
        .into_iter()
        .flatten()
        .any(|lon| {
            lon >= feature.bbox[0]
                && lon <= feature.bbox[2]
                && feature
                    .polygons
                    .iter()
                    .any(|poly| point_in_polygon(poly, lon, lat))
        })
}

/// Mean km per degree of latitude (and of longitude at the equator). Used by the point-in-polygon
/// snap-distance cap to turn lon/lat degrees into an isotropic km metric.
const KM_PER_DEG: f64 = 111.32;

/// Default `--snap-max-dist` (km): outside points snap to the nearest region only within this many
/// km of its boundary; farther strays are dropped. Caps the unbounded nearest-region snap so an
/// ocean GPS error or out-of-extent point can't be attributed to an arbitrary distant region.
const DEFAULT_SNAP_MAX_KM: f64 = 10.0;

/// Fraction of the median region size (bbox diagonal, km) used as the auto snap cap's context
/// term: an outside point may snap across at most this fraction of a typical region. 0.10
/// reproduces the old fixed 10 km cap at US-county scale (median bbox diagonal ~100 km) while
/// scaling down to sub-km caps for city-ward GeoJSONs and up (clamped) for country-scale polygons
/// whose coarse coastlines strand legitimate points offshore.
const SNAP_REGION_FRACTION: f64 = 0.10;

/// Multiplier applied to the coordinate-quantization radius when deriving the auto snap cap's
/// precision floor. 2.0 covers truncated (not just rounded) coordinates plus the equirectangular
/// approximation error, so a point whose printed coordinates land quantized on the wrong side of a
/// boundary still snaps back to its region.
const SNAP_PRECISION_SAFETY: f64 = 2.0;

/// Absolute floor for the auto-derived snap cap (km): never snap tighter than consumer GPS noise,
/// regardless of how small the GeoJSON regions are.
const SNAP_CAP_MIN_KM: f64 = 0.1;

/// Absolute ceiling for the auto-derived snap cap (km): even for continent-sized regions or
/// integer-degree coordinates, don't attribute a point to a region more than ~a metropolitan
/// diameter away.
const SNAP_CAP_MAX_KM: f64 = 100.0;

/// How the effective snap-distance cap was determined, for coverage-note provenance.
#[derive(Debug, Clone, Copy, PartialEq)]
enum SnapCapBasis {
    /// User passed `--snap-max-dist`; used verbatim.
    Explicit,
    /// Derived from the GeoJSON's median region size and/or the lat/lon coordinate precision
    /// (each term `None` when its signal was unavailable/degenerate).
    Auto {
        region_km:    Option<f64>,
        precision_km: Option<f64>,
    },
    /// Neither signal available; fixed [`DEFAULT_SNAP_MAX_KM`].
    Fallback,
}

/// The effective snap-distance cap (km) plus its provenance.
#[derive(Debug, Clone, Copy)]
struct SnapCap {
    km:    f64,
    basis: SnapCapBasis,
}

impl SnapCap {
    /// Short provenance phrase for the **stderr** coverage notes ("cap 3.2 km, {phrase}").
    ///
    /// Deliberately NOT localized: every one of this command's stderr diagnostics is English, so
    /// substituting a translated phrase here would emit a half-Spanish English sentence. The
    /// Data Schematic's own below-map note uses [`SnapCap::basis_phrase_localized`] instead; the
    /// two are pinned byte-identical under English by
    /// `snap_basis_catalog_matches_english_phrase`.
    const fn basis_phrase_en(&self) -> &'static str {
        match self.basis {
            SnapCapBasis::Explicit => "--snap-max-dist",
            SnapCapBasis::Auto {
                region_km: Some(_),
                precision_km: Some(_),
            } => "auto-derived from region size and coordinate precision",
            SnapCapBasis::Auto {
                region_km: Some(_),
                precision_km: None,
            } => "auto-derived from region size",
            SnapCapBasis::Auto {
                region_km: None,
                precision_km: Some(_),
            } => "auto-derived from coordinate precision",
            // Auto with both terms None never leaves resolve_snap_cap (it returns Fallback);
            // the arm only keeps the match total.
            SnapCapBasis::Auto {
                region_km: None,
                precision_km: None,
            }
            | SnapCapBasis::Fallback => "default",
        }
    }

    /// Same provenance phrase, in the Data Schematic's language, for the below-map coverage note.
    ///
    /// The keys are spelled out per arm rather than mapped over a runtime key, because
    /// `every_t_key_used_in_this_file_exists_in_the_catalog` only verifies literal keys and hard-
    /// fails on a computed one -- a typo would otherwise render as raw key text under the map.
    fn basis_phrase_localized(&self) -> String {
        match self.basis {
            SnapCapBasis::Explicit => t!("viz.notes.snap_basis_explicit"),
            SnapCapBasis::Auto {
                region_km: Some(_),
                precision_km: Some(_),
            } => t!("viz.notes.snap_basis_region_precision"),
            SnapCapBasis::Auto {
                region_km: Some(_),
                precision_km: None,
            } => t!("viz.notes.snap_basis_region"),
            SnapCapBasis::Auto {
                region_km: None,
                precision_km: Some(_),
            } => t!("viz.notes.snap_basis_precision"),
            SnapCapBasis::Auto {
                region_km: None,
                precision_km: None,
            }
            | SnapCapBasis::Fallback => t!("viz.notes.snap_basis_default"),
        }
        .into_owned()
    }
}

/// Median bbox diagonal of the features in equirectangular km (longitude degrees scaled by
/// `cos(mid-lat)`), the "typical region size" driving the auto snap cap's context term. Zero-size
/// bboxes (point/degenerate features — the same shapes the snap-target search would treat as
/// phantom targets) are skipped; `None` when no feature has a usable bbox. Antimeridian-crossing
/// features store continuity-unwrapped bboxes, so `bbox[2] - bbox[0]` is already the true angular
/// width, and the median absorbs pole-cut outliers (e.g. an Antarctica ring spanning ~360°).
fn median_bbox_diag_km(features: &[PipFeature]) -> Option<f64> {
    let mut diags: Vec<f64> = features
        .iter()
        .filter_map(|f| {
            let dlon = f.bbox[2] - f.bbox[0];
            let dlat = f.bbox[3] - f.bbox[1];
            if dlon <= 0.0 && dlat <= 0.0 {
                return None;
            }
            let mid_lat = f64::midpoint(f.bbox[1], f.bbox[3]);
            let dx = dlon * mid_lat.to_radians().cos() * KM_PER_DEG;
            let dy = dlat * KM_PER_DEG;
            Some(dx.hypot(dy))
        })
        .collect();
    if diags.is_empty() {
        return None;
    }
    diags.sort_unstable_by(f64::total_cmp);
    let n = diags.len();
    Some(if n % 2 == 1 {
        diags[n / 2]
    } else {
        f64::midpoint(diags[n / 2 - 1], diags[n / 2])
    })
}

/// The auto snap cap's stats-aware precision floor (km) for coordinates carrying `decimals`
/// decimal places: the worst-case quantization offset (half a cell diagonal,
/// `10^-d * KM_PER_DEG / sqrt(2)`) times [`SNAP_PRECISION_SAFETY`]. E.g. 1 decimal ≈ 15.7 km,
/// 2 decimals ≈ 1.6 km, 4+ decimals ≈ negligible.
fn precision_floor_km(decimals: u32) -> f64 {
    SNAP_PRECISION_SAFETY
        * KM_PER_DEG
        * std::f64::consts::FRAC_1_SQRT_2
        * 10_f64.powi(-i32::try_from(decimals).unwrap_or(i32::MAX))
}

/// Round to 2 significant digits so the cap we *display* in coverage notes (via `fmt_measure`,
/// ≤3 decimals) is exactly the cap we *apply*.
fn round_2_sig(v: f64) -> f64 {
    if v <= 0.0 || !v.is_finite() {
        return v;
    }
    let scale = 10_f64.powf(1.0 - v.abs().log10().floor());
    (v * scale).round() / scale
}

/// Resolve the effective snap-distance cap. An explicit `--snap-max-dist` wins verbatim.
/// Otherwise the cap adapts to context and stats: the larger of
/// [`SNAP_REGION_FRACTION`] × median region size and the coordinate-precision floor, clamped to
/// [`SNAP_CAP_MIN_KM`]..[`SNAP_CAP_MAX_KM`] and rounded to 2 significant digits. With neither
/// signal available, falls back to the fixed [`DEFAULT_SNAP_MAX_KM`].
fn resolve_snap_cap(
    explicit: Option<f64>,
    features: &[PipFeature],
    coord_decimals: Option<u32>,
) -> SnapCap {
    if let Some(km) = explicit {
        return SnapCap {
            km,
            basis: SnapCapBasis::Explicit,
        };
    }
    let region_km = median_bbox_diag_km(features).map(|d| d * SNAP_REGION_FRACTION);
    let precision_km = coord_decimals.map(precision_floor_km);
    let raw = match (region_km, precision_km) {
        (Some(r), Some(p)) => r.max(p),
        (Some(v), None) | (None, Some(v)) => v,
        (None, None) => {
            return SnapCap {
                km:    DEFAULT_SNAP_MAX_KM,
                basis: SnapCapBasis::Fallback,
            };
        },
    };
    SnapCap {
        km:    round_2_sig(raw.clamp(SNAP_CAP_MIN_KM, SNAP_CAP_MAX_KM)),
        basis: SnapCapBasis::Auto {
            region_km,
            precision_km,
        },
    }
}

/// Decimal places a raw CSV field carries in its shortest decimal form, matching the stats
/// cache's `max_precision` semantics: exponent-aware ("1.5e-3" → 4, "1.5e2" → 0) with trailing
/// zeros trimmed AFTER the exponent is applied — so "40.7100" → 2 and "4070e-2" (= 40.70) → 1,
/// not 2. Counting the mantissa's digit-sequence trailing zeros (rather than only trailing
/// fractional zeros) handles both forms uniformly: each trailing zero cancels one fractional
/// position of the exponent-shifted value. Feeds the auto snap cap's precision floor on the
/// command path, where no stats cache is loaded.
fn byte_decimal_places(field: &[u8]) -> u32 {
    let s = std::str::from_utf8(field).unwrap_or("").trim();
    let (mantissa, exp) = match s.find(['e', 'E']) {
        Some(i) => (&s[..i], s[i + 1..].parse::<i32>().unwrap_or(0)),
        None => (s, 0_i32),
    };
    // zero has no decimal places in shortest form, whatever the notation ("0e-3", "0.0e-3");
    // without this guard the exponent shift below would count phantom fractional positions.
    if !mantissa.bytes().any(|b| b.is_ascii_digit() && b != b'0') {
        return 0;
    }
    let frac_digits = match mantissa.find('.') {
        Some(i) => (mantissa.len() - i - 1) as i32,
        None => 0,
    };
    let trailing_zeros = mantissa
        .bytes()
        .rev()
        .filter(u8::is_ascii_digit)
        .take_while(|&b| b == b'0')
        .count() as i32;
    u32::try_from(
        frac_digits
            .saturating_sub(exp)
            .saturating_sub(trailing_zeros),
    )
    .unwrap_or(0)
}

/// Coordinate precision (decimal places) of the lat/lon columns from the stats cache: each
/// column's `max_precision` (Float; Integer counts as 0 decimals), combined with `min` — the
/// coarser axis drives the quantization error. `None` when either column's precision is unknown
/// (non-numeric type, or a stale cache without the field), which disables the precision floor.
fn stats_coord_decimals(
    stats: &[crate::cmd::stats::StatsData],
    lat_idx: usize,
    lon_idx: usize,
) -> Option<u32> {
    let decimals = |i: usize| -> Option<u32> {
        let s = stats.get(i)?;
        match s.r#type.as_str() {
            "Float" => s.max_precision,
            "Integer" => Some(0),
            _ => None,
        }
    };
    Some(decimals(lat_idx)?.min(decimals(lon_idx)?))
}

/// Squared Euclidean (degree-space) distance from a point to a line segment.
fn point_seg_dist2(px: f64, py: f64, ax: f64, ay: f64, bx: f64, by: f64) -> f64 {
    let (dx, dy) = (bx - ax, by - ay);
    // FMA version of let len2 = dx * dx + dy * dy;
    let len2 = dy.mul_add(dy, dx * dx);
    let t = if len2 <= f64::EPSILON {
        0.0
    } else {
        // (((px - ax) * dx + (py - ay) * dy) / len2).clamp(0.0, 1.0)
        ((py - ay).mul_add(dy, (px - ax) * dx) / len2).clamp(0.0, 1.0)
    };
    let (cx, cy) = (ax + t * dx, ay + t * dy);
    let (ex, ey) = (px - cx, py - cy);
    // ex * ex + ey * ey
    ey.mul_add(ey, ex * ex)
}

/// Squared distance from `(lon, lat)` to the nearest edge of any of the feature's rings, in the
/// equirectangular km space defined by the `(sx, sy)` per-degree scale factors (see
/// [`pip_assign`]). Passing `sx = sy = 1.0` recovers raw degree-space distance.
fn feature_dist2(feature: &PipFeature, lon: f64, lat: f64, sx: f64, sy: f64) -> f64 {
    let (px, py) = (lon * sx, lat * sy);
    let mut best = f64::INFINITY;
    for ring in feature.polygons.iter().flatten() {
        for edge in ring.windows(2) {
            let &[[ax, ay], [bx, by]] = edge else {
                continue;
            };
            let d = point_seg_dist2(px, py, ax * sx, ay * sy, bx * sx, by * sy);
            if d < best {
                best = d;
            }
        }
    }
    best
}

/// Lower-bound squared distance from `(lon, lat)` to a feature's bbox (0 when inside the bbox), in
/// the same `(sx, sy)`-scaled space as [`feature_dist2`] so it remains a valid prune bound.
fn bbox_dist2(bbox: &[f64; 4], lon: f64, lat: f64, sx: f64, sy: f64) -> f64 {
    let dx = (bbox[0] - lon).max(0.0).max(lon - bbox[2]) * sx;
    let dy = (bbox[1] - lat).max(0.0).max(lat - bbox[3]) * sy;
    dx * dx + dy * dy
}

/// Result of binning one point into a GeoJSON feature set.
#[derive(Debug, PartialEq, Eq)]
enum PipOutcome {
    /// Contained by feature at this index.
    Inside(usize),
    /// Outside every polygon; snapped to the nearest feature at this index.
    Snapped(usize),
    /// Outside every polygon and not snapped (dropped).
    Outside,
}

/// Assign a point to a feature: exact containment first; if none and `snap`, the nearest feature by
/// edge distance (bbox lower-bound pruned) provided it lies within `snap_max_km` (`f64::INFINITY`
/// = unlimited); otherwise [`PipOutcome::Outside`]. Distances use an equirectangular km
/// approximation centered on the query latitude (longitude degrees scaled by `cos(lat)`), so the
/// cap means the same east-west and north-south. The `Inside`/`Snapped`/`Outside` distinction lets
/// callers report how many points missed every polygon and how many were too far to snap.
fn pip_assign(
    features: &[PipFeature],
    lat: f64,
    lon: f64,
    snap: bool,
    snap_max_km: f64,
) -> PipOutcome {
    if let Some(i) = features.iter().position(|f| feature_contains(f, lon, lat)) {
        return PipOutcome::Inside(i);
    }
    if !snap {
        return PipOutcome::Outside;
    }
    // equirectangular km scale at the query latitude: km per degree of lat is ~constant; km per
    // degree of lon shrinks by cos(lat). Applying both to every distance keeps the cap isotropic.
    let sx = lat.to_radians().cos() * KM_PER_DEG;
    let sy = KM_PER_DEG;
    let mut best_i = None;
    let mut best_d2 = f64::INFINITY;
    // Seed the search with the feature whose BBOX is nearest, then let the loop below prune
    // against that. `best_d2` otherwise starts at infinity, so the first feature in FILE ORDER is
    // always fully edge-scanned no matter how far away it is, and pruning quality depends
    // entirely on how the boundary file happens to be ordered. This pass is bbox arithmetic only
    // (no vertices), so it costs O(features) cheap comparisons and typically reduces the full
    // ring scans below from "every unpruned feature" to one or two. The result is unchanged:
    // bbox_dist2 is a true lower bound, so pruning against a real distance can never discard a
    // nearer feature.
    let mut seed: Option<(usize, f64)> = None;
    let mut seed_bbox_d2 = f64::INFINITY;
    for (i, f) in features.iter().enumerate() {
        for flon in f.candidate_lons(lon).into_iter().flatten() {
            let b = bbox_dist2(&f.bbox, flon, lat, sx, sy);
            if b < seed_bbox_d2 {
                seed_bbox_d2 = b;
                seed = Some((i, flon));
            }
        }
    }
    if let Some((i, flon)) = seed {
        best_d2 = feature_dist2(&features[i], flon, lat, sx, sy);
        best_i = Some(i);
    }
    for (i, f) in features.iter().enumerate() {
        // probe the query at each of the feature's candidate longitudes so antimeridian-crossing
        // features (stored continuity-unwrapped past ±180°) measure distance consistently with
        // their vertices/bbox; the nearer representation wins.
        for flon in f.candidate_lons(lon).into_iter().flatten() {
            if bbox_dist2(&f.bbox, flon, lat, sx, sy) >= best_d2 {
                continue;
            }
            let d2 = feature_dist2(f, flon, lat, sx, sy);
            if d2 < best_d2 {
                best_d2 = d2;
                best_i = Some(i);
            }
        }
    }
    match best_i {
        Some(i) if best_d2 <= snap_max_km * snap_max_km => PipOutcome::Snapped(i),
        _ => PipOutcome::Outside,
    }
}

/// Collect every `[lon, lat]` vertex from a GeoJSON value into parallel `(lats, lons)` vectors, so
/// a `--map` choropleth can frame the MapLibre basemap to its regions instead of opening at
/// plotly's whole-world default (where county/city polygons are effectively invisible). Descends
/// only through GeoJSON's geometry-bearing keys (`features`/`geometry`/`geometries`/`coordinates`)
/// so numeric arrays inside feature `properties` (e.g. a stray `bbox` or data array) can't be
/// mistaken for coordinates. Out-of-range pairs are dropped. GeoJSON positions are `[lon, lat]`.
fn geojson_lat_lons(geojson: &serde_json::Value) -> (Vec<f64>, Vec<f64>) {
    fn walk(v: &serde_json::Value, lats: &mut Vec<f64>, lons: &mut Vec<f64>) {
        match v {
            serde_json::Value::Array(arr) => {
                // a leaf position is `[lon, lat, ...]`: the first two entries are bare numbers
                // (nested coordinate arrays have arrays, not numbers, in those slots).
                if arr.len() >= 2 && arr[0].is_number() && arr[1].is_number() {
                    if let (Some(lon), Some(lat)) = (arr[0].as_f64(), arr[1].as_f64())
                        && (-180.0..=180.0).contains(&lon)
                        && (-90.0..=90.0).contains(&lat)
                    {
                        lons.push(lon);
                        lats.push(lat);
                    }
                } else {
                    for item in arr {
                        walk(item, lats, lons);
                    }
                }
            },
            serde_json::Value::Object(map) => {
                for (k, val) in map {
                    if matches!(
                        k.as_str(),
                        "features" | "geometry" | "geometries" | "coordinates"
                    ) {
                        walk(val, lats, lons);
                    }
                }
            },
            _ => {},
        }
    }
    let mut lats = Vec::new();
    let mut lons = Vec::new();
    walk(geojson, &mut lats, &mut lons);
    (lats, lons)
}

/// Build the complete `Plot` for `viz choropleth`: fill whole geographic regions colored by an
/// aggregated value. Defaults to a token-free `Choropleth` on the projection `geo` subplot; `--map`
/// switches to a MapLibre `ChoroplethMap` (GeoJSON-only). Region keys come from `--locations`, or
/// — with `--geocode` — are derived from `--lat`/`--lon` (reverse) or a `--locations` name column
/// (forward) by reusing qsv's geocode engine.
fn build_choropleth_plot(
    args: &Args,
    out_format: OutFormat,
    geojson: Option<serde_json::Value>,
) -> CliResult<Plot> {
    // loaded once by `resolve_and_validate_geojson`; every consumer below borrows it and the
    // trace takes ownership last, so the document is fetched and parsed exactly once per run.
    let mut loaded_geojson = geojson;
    // Point-in-polygon binning: lat/lon points + a custom --geojson, without --geocode. It colors
    // regions by the GeoJSON feature that CONTAINS each point, so it always uses the geojson-id
    // location mode regardless of any --location-mode default. --geocode (when also present) takes
    // precedence — it is an explicit request for the geocode engine.
    //
    // lat/lon (point-in-polygon) and --locations (a pre-keyed region column) are two different
    // ways to identify regions; supplying both (without --geocode) is ambiguous, so fail rather
    // than silently ignore one. With --geocode, the geocode resolver enforces its own
    // exactly-one-source rule (lat/lon reverse OR --locations forward).
    if args.flag_geojson.is_some()
        && args.flag_lat.is_some()
        && args.flag_lon.is_some()
        && args.flag_locations.is_some()
        && !args.flag_geocode
    {
        return fail_incorrectusage_clierror!(
            "Ambiguous choropleth inputs: --lat/--lon bin each point into the --geojson region \
             that contains it (point-in-polygon), while --locations names a pre-keyed region \
             column. Supply one or the other (or add --geocode to reverse-geocode the points)."
        );
    }
    let pip = args.flag_geojson.is_some()
        && args.flag_lat.is_some()
        && args.flag_lon.is_some()
        && !args.flag_geocode;
    let snap = !args.flag_no_snap;
    let mode = if pip {
        LocationMode::GeoJsonId
    } else {
        parse_location_mode(args.flag_location_mode.as_deref().unwrap_or("iso3"))?
    };
    let palette = parse_color_scale(args.flag_color_scale.as_deref().unwrap_or("viridis"))?;

    // --value drives the colored measure; aggregation defaults to sum when a --value is given,
    // else per-region row counts. Non-count aggs require a --value column.
    let agg = match (
        parse_agg(args.flag_agg.as_deref())?,
        args.flag_value.is_some(),
    ) {
        (Some(a), _) => a,
        (None, true) => Agg::Sum,
        (None, false) => Agg::Count,
    };
    if agg != Agg::Count && args.flag_value.is_none() {
        return fail_incorrectusage_clierror!("--agg sum/mean/min/max requires a --value column.");
    }

    // --map (ChoroplethMap) is MapLibre + GeoJSON-only; the default geo Choropleth has built-in
    // country/state geometries and needs a GeoJSON only for the geojson-id location mode.
    if args.flag_map && args.flag_geojson.is_none() {
        return fail_incorrectusage_clierror!("--map (ChoroplethMap) requires --geojson.");
    }
    if matches!(mode, LocationMode::GeoJsonId) && args.flag_geojson.is_none() {
        return fail_incorrectusage_clierror!(
            "--location-mode geojson-id requires a --geojson source."
        );
    }
    if args.flag_map && args.flag_geocode {
        return fail_incorrectusage_clierror!(
            "--map cannot be combined with --geocode: geocode yields ISO-3/US-state codes, which \
             won't match a GeoJSON feature id. Use the default geo basemap with --geocode."
        );
    }
    if args.flag_no_snap && !pip {
        return fail_incorrectusage_clierror!(
            "--no-snap only applies to point-in-polygon binning (--lat/--lon + --geojson without \
             --geocode)."
        );
    }
    // --snap-max-dist caps the nearest-region snap distance (km). Its universal constraints
    // (non-negative finite, not combined with --no-snap) are validated up front in run(); here we
    // only reject it on a non-point-in-polygon path. When omitted, the cap is auto-derived from
    // the GeoJSON's median region size and the coordinates' decimal precision (see
    // resolve_snap_cap) so far strays drop instead of snapping to an arbitrary distant region;
    // pass a large value to restore unbounded snapping.
    if args.flag_snap_max_dist.is_some() && !pip {
        return fail_incorrectusage_clierror!(
            "--snap-max-dist only applies to point-in-polygon binning (--lat/--lon + --geojson \
             without --geocode)."
        );
    }

    // only the point-in-polygon path can drop points (no-snap, or no region within the cap), so
    // only it yields a below-map coverage note; the literal/geocoded paths return the plain
    // 4-tuple.
    let mut below_note: Option<String> = None;
    let (locations, z, measure_label, hover_text) = if pip {
        let (locs, z, label, hover, note) = choropleth_pip_locations(
            args,
            agg,
            snap,
            args.flag_snap_max_dist,
            loaded_geojson.as_ref(),
        )?;
        below_note = note;
        (locs, z, label, hover)
    } else if args.flag_geocode {
        choropleth_geocoded_locations(args, mode.clone(), agg)?
    } else {
        choropleth_literal_locations(args, agg, loaded_geojson.as_ref())?
    };

    if locations.is_empty() {
        return fail_clierror!(
            "No choropleth regions resolved (check --locations / --geocode inputs and \
             --location-mode)."
        );
    }

    let mut plot = Plot::new();
    if args.flag_map {
        // ChoroplethMap on the MapLibre `map` subplot (GeoJSON-only).
        let geojson = match loaded_geojson.take() {
            Some(v) => v,
            None => load_geojson(args.flag_geojson.as_deref().unwrap())?,
        };
        // collect the GeoJSON extent BEFORE the value is moved into the trace below.
        let (g_lats, g_lons) = geojson_lat_lons(&geojson);
        let trace = ChoroplethMap::new(locations, z)
            .geojson(geojson)
            .feature_id_key(args.flag_feature_id_key.as_deref().unwrap_or("id"))
            .color_scale(ColorScale::Palette(palette))
            .show_scale(true)
            .color_bar(ColorBar::new().title(escape_hover(&measure_label)))
            .marker(
                ChoroplethMarker::new()
                    .line(Line::new().width(0.5))
                    .opacity(CHOROPLETH_MAP_FILL_OPACITY),
            )
            // lift the fill above the basemap's road layers (plotly's default drops it just above
            // water, so roads render on top and muddy the read); near-opaque fill then reads clean.
            .below("")
            .hover_text_array(hover_text)
            .hover_info(HoverInfo::Text);
        plot.add_trace(trace);
        // --style carries a global docopt default of open-street-map (a token-free MapLibre style).
        let style =
            parse_choropleth_map_style(args.flag_style.as_deref().unwrap_or("open-street-map"))?;
        let mut layout_map = LayoutMap::new().style(style);
        // frame the basemap to the GeoJSON extent so local/custom regions (counties, cities) are
        // visible instead of being lost at plotly's default whole-world view centered at (0, 0).
        // Falls back to that default only when the GeoJSON has no usable coordinates. `fit_dims`
        // honors --width/--height for image exports (so a non-default static aspect frames its
        // extent instead of cropping) and the default aspect for HTML — matching `build_map_plot`.
        if !g_lats.is_empty() {
            let (fit_w, fit_h) = fit_dims(args.flag_width, args.flag_height, out_format);
            let (center, zoom) = map_center_zoom(&g_lats, &g_lons, 0.0, fit_w as f64, fit_h as f64);
            layout_map = layout_map.center(center).zoom(f64::from(zoom));
        }
        let mut layout = Layout::new().map(layout_map);
        if let Some(title) = &args.flag_title {
            layout = layout.title(Title::with_text(title));
        }
        if let Some(note) = &below_note {
            layout = with_below_note(layout, note);
        }
        plot.set_layout(apply_theme(layout, args.theme()));
    } else {
        // Choropleth on the projection `geo` subplot (token-free built-in geometries).
        let usa_states = matches!(mode, LocationMode::UsaStates);
        let mut trace = Choropleth::new(locations, z)
            .location_mode(mode)
            .color_scale(ColorScale::Palette(palette))
            .show_scale(true)
            .color_bar(ColorBar::new().title(escape_hover(&measure_label)))
            .marker(ChoroplethMarker::new().line(Line::new().width(0.5)))
            .hover_text_array(hover_text)
            .hover_info(HoverInfo::Text);
        let (geo_land, _geo_water, geo_bg) = geo_palette(args.theme());
        let mut geo = LayoutGeo::new()
            .resolution(GeoResolution::OneOverFiftyMillion)
            .showland(true)
            .landcolor(geo_land)
            .showcountries(true)
            // a choropleth paints no ocean, so geo.bgcolor IS the sea (and the area outside the
            // projection); it carries the theme so a dark map has a dark sea, not the default
            // white.
            .bgcolor(geo_bg);
        // usa-states built-in geometries need the albers-usa projection to frame the US (CONUS +
        // AK/HI insets); without it plotly draws the states tiny on the default whole-world view. A
        // --geojson source (handled below) frames itself, so let that override.
        if usa_states {
            // scope:"usa" restricts the basemap (showland, showcountries) to the US extent,
            // preventing Canadian land (e.g. British Columbia) from bleeding into the
            // albers-usa composite canvas above Washington State.
            geo = geo
                .projection(Projection::new().projection_type(ProjectionType::AlbersUsa))
                .scope("usa");
        }
        if let Some(spec) = args.flag_geojson.as_deref() {
            let geojson = match loaded_geojson.take() {
                Some(v) => v,
                None => load_geojson(spec)?,
            };
            // plotly only auto-scopes its BUILT-IN location modes (iso3 world / usa-states); a
            // custom GeoJSON has no built-in scope, so without framing its polygons sit tiny on the
            // default whole-world projection. Frame the projection to the GeoJSON extent, reusing
            // the smart panel's geo_framing (albers-usa / natural-earth / mercator fit, padded and
            // antimeridian-aware). Collect the extent BEFORE the value is moved into the trace.
            let (g_lats, g_lons) = geojson_lat_lons(&geojson);
            if !g_lats.is_empty() {
                // trim_frac 0.0: every GeoJSON vertex is intentional, so frame the FULL extent —
                // trimming would crop edge/island polygons.
                let (proj, lonaxis, lataxis) = geo_framing(&g_lats, &g_lons, 0.0);
                geo = geo.projection(Projection::new().projection_type(proj));
                if let Some(lonaxis) = lonaxis {
                    geo = geo.lonaxis(lonaxis);
                }
                if let Some(lataxis) = lataxis {
                    geo = geo.lataxis(lataxis);
                }
            }
            trace = trace
                .geojson(geojson)
                .feature_id_key(args.flag_feature_id_key.as_deref().unwrap_or("id"));
        }
        plot.add_trace(trace);
        let mut layout = Layout::new().geo(geo);
        if let Some(title) = &args.flag_title {
            layout = layout.title(Title::with_text(title));
        }
        if let Some(note) = &below_note {
            layout = with_below_note(layout, note);
        }
        plot.set_layout(apply_theme(layout, args.theme()));
    }
    Ok(plot)
}

/// Append a paper-anchored note centered just beneath the plot area, reserving bottom margin so it
/// isn't clipped. Used for the `viz choropleth --no-snap` coverage note (points dropped because
/// they fell outside every GeoJSON region) — a map has no bottom axis, so `y=0` sits in open space.
/// The font color is left to inherit so the smart light/dark toggle (and themes) flip it instead of
/// baking in a fixed ink.
fn with_below_note(layout: Layout, note: &str) -> Layout {
    layout.margin(Margin::new().bottom(60)).annotations(vec![
        Annotation::new()
            .text(note)
            .x(0.5)
            .y(0.0)
            .x_ref("paper")
            .y_ref("paper")
            .x_anchor(Anchor::Center)
            .y_anchor(Anchor::Top)
            .show_arrow(false)
            .font(Font::new().size(12)),
    ])
}

/// Append the generic `--annotation` caption, centered below the plot. Unlike `with_below_note`
/// (used for maps), a cartesian chart's x-axis ticks and title occupy the space just under the plot
/// area, so the note is pushed further down with `y_shift` and a larger bottom margin to clear
/// them.
fn with_chart_note(layout: Layout, note: &str) -> Layout {
    layout.margin(Margin::new().bottom(110)).annotations(vec![
        Annotation::new()
            .text(note)
            .x(0.5)
            .y(0.0)
            .x_ref("paper")
            .y_ref("paper")
            .x_anchor(Anchor::Center)
            .y_anchor(Anchor::Top)
            .y_shift(-58.0)
            .show_arrow(false)
            .font(Font::new().size(12)),
    ])
}

/// Resolve choropleth `(locations, z, measure_label, hover_text)` from a literal `--locations`
/// region-key column, aggregating the `--value` measure (or row counts) per region.
fn choropleth_literal_locations(
    args: &Args,
    agg: Agg,
    loaded_geojson: Option<&serde_json::Value>,
) -> CliResult<(Vec<String>, Vec<f64>, String, Vec<String>)> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let loc_idx = resolve_one(args.flag_locations.as_ref(), &headers, nh, "locations")?;
    let value_idx = match args.flag_value.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "value")?),
        None => None,
    };
    let measure_label = match value_idx {
        Some(i) => col_label(&headers, i, nh),
        None => t!("viz.chart.count").into_owned(),
    };

    let mut raw_locs: Vec<String> = Vec::new();
    let mut values: Vec<f64> = Vec::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let loc = cell_to_string(record.get(loc_idx));
        if loc.is_empty() {
            continue;
        }
        let value = match value_idx {
            Some(i) => match parse_f64(record.get(i)) {
                Some(v) => v,
                None => continue,
            },
            None => 1.0,
        };
        raw_locs.push(loc);
        values.push(value);
    }
    let (locs, z) = aggregate(raw_locs, values, agg);
    // when a custom --geojson is supplied (geojson-id / --map paths), resolve region names from its
    // features so hover shows human-readable labels — matching the point-in-polygon path. Without a
    // --geojson (ISO-3 / US-state built-in geometries) there are no feature names, so hover uses
    // the location id.
    let names = match args.flag_geojson.as_deref() {
        Some(spec) => {
            let owned;
            let geojson = match loaded_geojson {
                Some(v) => v,
                None => {
                    owned = load_geojson(spec)?;
                    &owned
                },
            };
            let key = args.flag_feature_id_key.as_deref().unwrap_or("id");
            let features = build_pip_features(geojson, key, args.flag_feature_name_key.as_deref())?;
            aligned_region_names(&features, &locs)
        },
        None => None,
    };
    let include_pct = matches!(agg, Agg::Count | Agg::Sum);
    let hover_text =
        choropleth_hover_text(&locs, &z, names.as_deref(), &measure_label, include_pct);
    Ok((locs, z, measure_label, hover_text))
}

/// Resolve choropleth `(locations, z, measure_label, hover_text, below_note)` by point-in-polygon
/// binning: each row's `--lat`/`--lon` point is assigned to the GeoJSON region whose polygon
/// contains it (or, when `snap`, to the nearest region within the snap-distance cap — an explicit
/// `--snap-max-dist` verbatim, else auto-derived from the GeoJSON's median region size and the
/// coordinates' as-written decimal precision; see [`resolve_snap_cap`]), and the `--value` measure
/// (or row counts) is aggregated per region id. The per-region hover gains an `includes N snapped
/// from outside` line for regions that absorbed snapped points (the region's count already
/// includes them). Emits stderr coverage notes, and — whenever points snapped or were dropped —
/// returns a `below_note` rendered beneath the map with the snap metadata.
fn choropleth_pip_locations(
    args: &Args,
    agg: Agg,
    snap: bool,
    snap_max_km: Option<f64>,
    loaded_geojson: Option<&serde_json::Value>,
) -> CliResult<(Vec<String>, Vec<f64>, String, Vec<String>, Option<String>)> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let lat_idx = resolve_one(args.flag_lat.as_ref(), &headers, nh, "lat")?;
    let lon_idx = resolve_one(args.flag_lon.as_ref(), &headers, nh, "lon")?;
    let value_idx = match args.flag_value.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "value")?),
        None => None,
    };
    let measure_label = match value_idx {
        Some(i) => col_label(&headers, i, nh),
        None => t!("viz.chart.count").into_owned(),
    };

    let feature_id_key = args.flag_feature_id_key.as_deref().unwrap_or("id");
    let owned;
    let geojson = match loaded_geojson {
        Some(v) => v,
        None => {
            owned = load_geojson(args.flag_geojson.as_deref().unwrap())?;
            &owned
        },
    };
    let features = build_pip_features(
        geojson,
        feature_id_key,
        args.flag_feature_name_key.as_deref(),
    )?;

    // pass 1: collect the points, tracking each coordinate column's as-written decimal precision
    // (a signal for the auto snap cap). Collect-then-assign rather than assigning per row: the
    // cap derivation needs the whole column's precision, which an interleaved loop couldn't know
    // before the first `pip_assign` call.
    let mut points: Vec<(f64, f64, f64)> = Vec::new();
    let mut lat_dec = 0_u32;
    let mut lon_dec = 0_u32;
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let (Some(lat), Some(lon)) = (
            parse_f64(record.get(lat_idx)),
            parse_f64(record.get(lon_idx)),
        ) else {
            continue;
        };
        if !((-90.0..=90.0).contains(&lat) && (-180.0..=180.0).contains(&lon)) {
            continue;
        }
        let value = match value_idx {
            Some(i) => match parse_f64(record.get(i)) {
                Some(v) => v,
                None => continue,
            },
            None => 1.0,
        };
        lat_dec = lat_dec.max(byte_decimal_places(record.get(lat_idx).unwrap_or_default()));
        lon_dec = lon_dec.max(byte_decimal_places(record.get(lon_idx).unwrap_or_default()));
        points.push((lat, lon, value));
    }
    // per column: max decimals across rows; across columns: min — the coarser axis drives the
    // quantization error.
    let coord_decimals = (!points.is_empty()).then(|| lat_dec.min(lon_dec));
    let cap = resolve_snap_cap(snap_max_km, &features, coord_decimals);

    // pass 2: assign each point to a region.
    let mut raw_locs: Vec<String> = Vec::new();
    let mut values: Vec<f64> = Vec::new();
    // per-region tally of points that landed outside every polygon but were snapped into this
    // region (keyed by feature id), surfaced in the region's hover.
    let mut snapped_by_id: HashMap<String, usize> = HashMap::new();
    let total = points.len();
    let mut snapped = 0_usize;
    let mut dropped = 0_usize;
    for (lat, lon, value) in points {
        match pip_assign(&features, lat, lon, snap, cap.km) {
            PipOutcome::Inside(fi) => {
                raw_locs.push(features[fi].id.clone());
                values.push(value);
            },
            PipOutcome::Snapped(fi) => {
                raw_locs.push(features[fi].id.clone());
                values.push(value);
                *snapped_by_id.entry(features[fi].id.clone()).or_insert(0) += 1;
                snapped += 1;
            },
            PipOutcome::Outside => dropped += 1,
        }
    }

    // describe where dropped points went: --no-snap drops everything outside; otherwise only points
    // with no region within the cap are dropped (finite cap; an unlimited cap never drops).
    let drop_reason = if snap {
        format!("no region within {} km", fmt_measure(cap.km))
    } else {
        "--no-snap".to_string()
    };
    if snapped > 0 {
        viz_note(&format!(
            "viz choropleth: {snapped} of {total} points were snapped to the nearest region (cap \
             {} km, {}).",
            fmt_measure(cap.km),
            cap.basis_phrase_en()
        ));
    }
    if dropped > 0 {
        viz_note(&format!(
            "viz choropleth: {dropped} of {total} points fell outside every region and were \
             dropped ({drop_reason})."
        ));
    }

    let (locs, z) = aggregate(raw_locs, values, agg);
    let names = aligned_region_names(&features, &locs);
    let include_pct = matches!(agg, Agg::Count | Agg::Sum);
    let mut hover_text =
        choropleth_hover_text(&locs, &z, names.as_deref(), &measure_label, include_pct);
    // annotate each region's hover with how many of its points were snapped in from outside. The
    // region's count already includes these, so word it as a subset, not an addition.
    for (h, loc) in hover_text.iter_mut().zip(&locs) {
        if let Some(&c) = snapped_by_id.get(loc)
            && c > 0
        {
            h.push_str(&format!("<br>{}", t!("viz.hover.snap_includes", q_n = c)));
        }
    }

    // surface snap/drop coverage beneath the map — a saved HTML has no stderr to fall back on.
    // Whenever anything snapped, the note states the snap metadata (count, cap, cap provenance).
    let mut note_parts: Vec<String> = Vec::new();
    if snapped > 0 {
        note_parts.push(
            t!(
                "viz.notes.snap_coverage",
                q_n = snapped,
                q_total = total,
                q_km = fmt_measure(cap.km),
                q_basis = cap.basis_phrase_localized()
            )
            .into_owned(),
        );
    }
    if dropped > 0 {
        note_parts.push(
            if snap {
                t!(
                    "viz.notes.drop_coverage",
                    q_n = dropped,
                    q_total = total,
                    q_km = fmt_measure(cap.km)
                )
            } else {
                t!(
                    "viz.notes.drop_coverage_no_snap",
                    q_n = dropped,
                    q_total = total
                )
            }
            .into_owned(),
        );
    }
    let below_note = (!note_parts.is_empty()).then(|| note_parts.join(" "));

    Ok((locs, z, measure_label, hover_text, below_note))
}

/// Resolve choropleth `(locations, z, measure_label, hover_text)` via qsv's geocode engine:
/// reverse-geocode `--lat`/`--lon` points, or forward-geocode a `--locations` name column, into
/// ISO-3 / US-state codes per `--location-mode`, then aggregate the `--value` measure (or row
/// counts) per region.
#[cfg(feature = "geocode")]
fn choropleth_geocoded_locations(
    args: &Args,
    mode: LocationMode,
    agg: Agg,
) -> CliResult<(Vec<String>, Vec<f64>, String, Vec<String>)> {
    if !matches!(mode, LocationMode::Iso3 | LocationMode::UsaStates) {
        return fail_incorrectusage_clierror!(
            "--geocode only resolves --location-mode iso3 or usa-states (the codes geocode can \
             produce)."
        );
    }
    let has_latlon = args.flag_lat.is_some() && args.flag_lon.is_some();
    let has_names = args.flag_locations.is_some();
    if has_latlon == has_names {
        return fail_incorrectusage_clierror!(
            "--geocode needs exactly one source: --lat/--lon points (reverse) OR a --locations \
             name column (forward)."
        );
    }

    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let value_idx = match args.flag_value.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "value")?),
        None => None,
    };
    let measure_label = match value_idx {
        Some(i) => col_label(&headers, i, nh),
        None => t!("viz.chart.count").into_owned(),
    };

    // Collect the per-row geocode query + aligned measure, skipping rows missing inputs.
    let mut values: Vec<f64> = Vec::new();
    let mut record = csv::ByteRecord::new();
    let regions = if has_latlon {
        let lat_idx = resolve_one(args.flag_lat.as_ref(), &headers, nh, "lat")?;
        let lon_idx = resolve_one(args.flag_lon.as_ref(), &headers, nh, "lon")?;
        let mut points: Vec<(f64, f64)> = Vec::new();
        while rdr.read_byte_record(&mut record)? {
            let (Some(lat), Some(lon)) = (
                parse_f64(record.get(lat_idx)),
                parse_f64(record.get(lon_idx)),
            ) else {
                continue;
            };
            let value = match value_idx {
                Some(i) => match parse_f64(record.get(i)) {
                    Some(v) => v,
                    None => continue,
                },
                None => 1.0,
            };
            points.push((lat, lon));
            values.push(value);
        }
        crate::cmd::geocode::reverse_geocode_regions(&points, None)?
    } else {
        let name_idx = resolve_one(args.flag_locations.as_ref(), &headers, nh, "locations")?;
        let mut names: Vec<String> = Vec::new();
        while rdr.read_byte_record(&mut record)? {
            let name = cell_to_string(record.get(name_idx));
            if name.is_empty() {
                continue;
            }
            let value = match value_idx {
                Some(i) => match parse_f64(record.get(i)) {
                    Some(v) => v,
                    None => continue,
                },
                None => 1.0,
            };
            names.push(name);
            values.push(value);
        }
        let name_refs: Vec<&str> = names.iter().map(String::as_str).collect();
        crate::cmd::geocode::forward_geocode_regions(&name_refs, None)?
    };

    // Map each resolved region to the code for the requested mode; drop rows that didn't resolve.
    let mut locations: Vec<String> = Vec::with_capacity(regions.len());
    let mut kept_values: Vec<f64> = Vec::with_capacity(regions.len());
    for (region, value) in regions.into_iter().zip(values) {
        let code = region.and_then(|r| match mode {
            LocationMode::Iso3 => (!r.iso3.is_empty()).then_some(r.iso3),
            LocationMode::UsaStates => r.us_state_code,
            _ => None,
        });
        if let Some(code) = code {
            locations.push(code);
            kept_values.push(value);
        }
    }
    let (locs, z) = aggregate(locations, kept_values, agg);
    let include_pct = matches!(agg, Agg::Count | Agg::Sum);
    let hover_text = choropleth_hover_text(&locs, &z, None, &measure_label, include_pct);
    Ok((locs, z, measure_label, hover_text))
}

/// Non-geocode build: `--geocode` is unsupported, so reject it with an actionable message.
#[cfg(not(feature = "geocode"))]
fn choropleth_geocoded_locations(
    _args: &Args,
    _mode: LocationMode,
    _agg: Agg,
) -> CliResult<(Vec<String>, Vec<f64>, String, Vec<String>)> {
    fail_incorrectusage_clierror!(
        "--geocode requires a qsv build with the `geocode` feature (or a prebuilt qsv binary). \
         Supply ready-made region codes via --locations instead."
    )
}

/// Split row-aligned (x, y, z) numeric triples into one `Scatter3D` trace per `--series` category,
/// preserving first-seen category order. `hover` is the shared per-point hover template (naming the
/// x/y/z columns) applied to every trace so the tooltip isn't plotly's bare `x/y/z`.
fn scatter3d_series_traces(
    xs: Vec<f64>,
    ys: Vec<f64>,
    zs: Vec<f64>,
    series: Vec<String>,
    hover: &str,
) -> Vec<Box<dyn Trace>> {
    let mut order: Vec<String> = Vec::new();
    let mut groups: BTreeMap<String, (Vec<f64>, Vec<f64>, Vec<f64>)> = BTreeMap::new();
    for i in 0..xs.len() {
        let name = series[i].clone();
        let entry = groups.entry(name.clone()).or_insert_with(|| {
            order.push(name);
            (Vec::new(), Vec::new(), Vec::new())
        });
        entry.0.push(xs[i]);
        entry.1.push(ys[i]);
        entry.2.push(zs[i]);
    }
    order
        .into_iter()
        .map(|name| {
            let (a, b, c) = groups.remove(&name).unwrap_or_default();
            let trace: Box<dyn Trace> = Scatter3D::new(a, b, c)
                .mode(Mode::Markers)
                .name(escape_hover(&name))
                .hover_template(hover);
            trace
        })
        .collect()
}

/// Build the complete `Plot` for `viz scatter3d`: a `Scatter3D` markers trace over three numeric
/// columns (--x/--y/--z), with optional `--color`/`--size` marker encodings or `--series` traces.
/// Uses a 3D `scene` layout (not cartesian axes), so it owns its whole `Plot`.
fn build_scatter3d_plot(args: &Args) -> CliResult<Plot> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let x_idx = resolve_one(args.flag_x.as_ref(), &headers, nh, "x")?;
    let y_idx = resolve_one(args.flag_y.as_ref(), &headers, nh, "y")?;
    let z_idx = resolve_one(args.flag_z.as_ref(), &headers, nh, "z")?;
    let color_idx = match args.flag_color.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "color")?),
        None => None,
    };
    let size_idx = match args.flag_size.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "size")?),
        None => None,
    };
    let series_idx = match args.flag_series.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "series")?),
        None => None,
    };

    // a row is kept only when x/y/z and every requested encoding parse as numbers, so all
    // per-point arrays stay aligned.
    let mut xs: Vec<f64> = Vec::new();
    let mut ys: Vec<f64> = Vec::new();
    let mut zs: Vec<f64> = Vec::new();
    let mut colors: Vec<f64> = Vec::new();
    let mut sizes: Vec<f64> = Vec::new();
    let mut series: Vec<String> = Vec::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let (Some(x), Some(y), Some(z)) = (
            parse_f64(record.get(x_idx)),
            parse_f64(record.get(y_idx)),
            parse_f64(record.get(z_idx)),
        ) else {
            continue;
        };
        let color = match color_idx {
            Some(i) => match parse_f64(record.get(i)) {
                Some(v) => Some(v),
                None => continue,
            },
            None => None,
        };
        let size = match size_idx {
            Some(i) => match parse_f64(record.get(i)) {
                Some(v) => Some(v),
                None => continue,
            },
            None => None,
        };
        xs.push(x);
        ys.push(y);
        zs.push(z);
        if let Some(v) = color {
            colors.push(v);
        }
        if let Some(v) = size {
            sizes.push(v);
        }
        if let Some(i) = series_idx {
            series.push(cell_to_string(record.get(i)));
        }
    }
    if xs.is_empty() {
        return fail_clierror!(
            "No plottable rows found (are --x/--y/--z and the --color/--size columns numeric?)."
        );
    }

    let x_title = args
        .flag_x_title
        .clone()
        .unwrap_or_else(|| col_label(&headers, x_idx, nh));
    let y_title = args
        .flag_y_title
        .clone()
        .unwrap_or_else(|| col_label(&headers, y_idx, nh));
    let z_title = col_label(&headers, z_idx, nh);
    // plotly's default 3D hover labels the coordinates with the bare axis letters x/y/z (it does
    // NOT read the scene axis titles), so build an explicit template naming each dimension with
    // comma-grouped values. escape: this is a hoverTEMPLATE — a raw header containing
    // `<extra>`/`<b>` would otherwise terminate/format the template instead of displaying
    // literally.
    let hover = format!(
        "{}: %{{x:,.3f}}<br>{}: %{{y:,.3f}}<br>{}: %{{z:,.3f}}<extra></extra>",
        escape_hover(&x_title),
        escape_hover(&y_title),
        escape_hover(&z_title)
    );

    let mut plot = Plot::new();
    if series_idx.is_some() {
        for trace in scatter3d_series_traces(xs, ys, zs, series, &hover) {
            plot.add_trace(trace);
        }
    } else {
        let mut marker = Marker::new();
        if !sizes.is_empty() {
            marker = marker.size_array(scale_bubble_sizes(&sizes));
        }
        if !colors.is_empty() {
            let color_label = col_label(&headers, color_idx.unwrap(), nh);
            marker = marker
                .color_array(colors)
                .color_scale(ColorScale::Palette(ColorScalePalette::Viridis))
                .show_scale(true)
                .color_bar(ColorBar::new().title(escape_hover(&color_label)));
        }
        plot.add_trace(
            Scatter3D::new(xs, ys, zs)
                .mode(Mode::Markers)
                .marker(marker)
                .hover_template(hover),
        );
    }

    let scene = LayoutScene::new()
        // axis titles render plotly pseudo-HTML and these are raw CSV headers, same sink as the
        // smart 3D panel
        .x_axis(Axis::new().title(Title::with_text(escape_hover(&x_title))))
        .y_axis(Axis::new().title(Title::with_text(escape_hover(&y_title))))
        .z_axis(Axis::new().title(Title::with_text(escape_hover(&z_title))));
    let mut layout = Layout::new().scene(scene).show_legend(series_idx.is_some());
    if let Some(title) = &args.flag_title {
        layout = layout.title(Title::with_text(title));
    }
    plot.set_layout(apply_theme(layout, args.theme()));
    Ok(plot)
}

/// Build the `Plot` for `viz treemap` / `viz sunburst`: a domain-based hierarchical part-to-whole
/// chart over the `--cols` dimensions (outer level first), sized by row count (default) or by a
/// summed `--value` measure. Owns its whole `Plot` (no cartesian axes), like `pie`.
fn build_hierarchy_plot(args: &Args) -> CliResult<Plot> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let dims = resolve_many(args.flag_cols.as_ref(), &headers, nh, "cols")?;
    if dims.len() < HIER_MIN_DIMS {
        return fail_incorrectusage_clierror!(
            "treemap/sunburst/icicle needs at least {HIER_MIN_DIMS} --cols (hierarchy levels, \
             outer first)."
        );
    }

    // hierarchy areas must be ADDITIVE so a parent equals the sum of its children: only `count`
    // (default; no --value) and `sum` (with --value) are valid. mean/min/max don't roll up a tree.
    let value_idx = match args
        .flag_agg
        .as_deref()
        .map(str::to_ascii_lowercase)
        .as_deref()
    {
        // a --value with no explicit --agg sums it (matches `viz bar`'s --value semantics)
        None => match args.flag_value.as_ref() {
            Some(s) => Some(resolve_one(Some(s), &headers, nh, "value")?),
            None => None,
        },
        Some("count") => None,
        Some("sum") => {
            let Some(s) = args.flag_value.as_ref() else {
                return fail_incorrectusage_clierror!("--agg sum requires a --value column.");
            };
            Some(resolve_one(Some(s), &headers, nh, "value")?)
        },
        Some(other) => {
            return fail_incorrectusage_clierror!(
                "treemap/sunburst/icicle only supports additive --agg (count or sum); got \
                 '{other}'."
            );
        },
    };

    let leaves = accumulate_hierarchy_counts(&mut rdr, &dims, value_idx)?;
    let depth = dims.len();
    let top_n = args.flag_limit.max(1);
    let style = if args.cmd_sunburst {
        HierStyle::Sunburst
    } else if args.cmd_icicle {
        HierStyle::Icicle
    } else {
        HierStyle::Treemap
    };
    let Some((labels, parents, values, ids)) =
        hierarchy_arrays(&leaves, depth, top_n, HIER_MAX_NODES, "All")
    else {
        return fail_clierror!(
            "No hierarchy to chart from --cols (need at least one level that splits into 2+ \
             groups)."
        );
    };

    let mut plot = Plot::new();
    plot.add_trace(hierarchy_trace(style, &labels, &parents, &values, &ids));
    let mut layout = Layout::new().show_legend(false);
    if let Some(title) = &args.flag_title {
        layout = layout.title(Title::with_text(title));
    }
    plot.set_layout(apply_theme(layout, args.theme()));
    Ok(plot)
}

/// Build the `Plot` for `viz splom`: a scatter-plot matrix (SPLOM) over the numeric `--cols`
/// (default: all numeric columns). Every pair of dimensions is cross-plotted in a grid, with each
/// dimension's own axis and its distribution on the diagonal. Splom self-generates its N×N axis
/// grid, so it owns its whole `Plot` (no shared cartesian axes), like the hierarchy charts above.
fn build_splom_plot(args: &Args) -> CliResult<Plot> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    // default to all columns when --cols is omitted; read_numeric_columns keeps the numeric ones
    let candidates: Vec<usize> = match args.flag_cols.as_ref() {
        Some(_) => resolve_many(args.flag_cols.as_ref(), &headers, nh, "cols")?,
        None => (0..headers.len()).collect(),
    };
    let (labels, columns, _) = read_numeric_columns(&mut rdr, &headers, nh, &candidates, false)?;
    if labels.len() < 2 {
        return fail_incorrectusage_clierror!(
            "splom needs at least 2 numeric --cols (a scatter-plot matrix cross-plots numeric \
             column pairs)."
        );
    }
    // read_numeric_columns is listwise: a row is dropped if ANY kept column is non-numeric/empty.
    // Columns that are each majority-numeric but never jointly complete leave empty value vectors,
    // which would emit an empty SPLOM — error out instead of charting nothing.
    if columns.first().map_or(0, Vec::len) == 0 {
        return fail_clierror!(
            "splom found no rows where all selected numeric --cols parse together (nothing to \
             plot)."
        );
    }

    let dimensions: Vec<SplomDimension<f64>> = labels
        .into_iter()
        .zip(columns)
        .map(|(label, values)| SplomDimension::new().label(label).values(values))
        .collect();

    let trace = Splom::new()
        .dimensions(dimensions)
        .marker(Marker::new().size(4).opacity(0.6));

    let mut plot = Plot::new();
    plot.add_trace(trace);
    let mut layout = Layout::new();
    if let Some(title) = &args.flag_title {
        layout = layout.title(Title::with_text(title));
    }
    plot.set_layout(apply_theme(layout, args.theme()));
    Ok(plot)
}

/// Build the `Plot` for `viz parcats`: a parallel-categories diagram over the categorical `--cols`
/// (best with 3-4). Rows are aggregated into distinct category tuples (weighted by `counts`), each
/// column becoming a dimension. Domain-based (no cartesian axes), so it owns its whole `Plot`.
fn build_parcats_plot(args: &Args) -> CliResult<Plot> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let dims = resolve_many(args.flag_cols.as_ref(), &headers, nh, "cols")?;
    if dims.len() < 2 {
        return fail_incorrectusage_clierror!(
            "parcats needs at least 2 categorical --cols (best with 3-4)."
        );
    }

    // aggregate identical category tuples into weighted paths so the trace stays compact
    let mut counts: HashMap<Vec<String>, f64> = HashMap::new();
    let mut order: Vec<Vec<String>> = Vec::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let tuple: Vec<String> = dims
            .iter()
            .map(|&i| cell_to_string(record.get(i)))
            .collect();
        counts
            .entry(tuple.clone())
            .and_modify(|c| *c += 1.0)
            .or_insert_with(|| {
                order.push(tuple);
                1.0
            });
    }
    if order.is_empty() {
        return fail_clierror!("No rows to chart for the parcats diagram.");
    }

    let weights: Vec<f64> = order.iter().map(|t| counts[t]).collect();
    let labels: Vec<String> = dims
        .iter()
        .map(|&idx| col_label(&headers, idx, nh))
        .collect();
    let (dimensions, line, ordered) = parcats_dimensions_and_line(&labels, &order, &weights);

    let mut trace = Parcats::new()
        .dimensions(dimensions)
        .counts_array(weights)
        .arrangement(ParcatsArrangement::Perpendicular)
        .bundle_colors(true);
    if let Some(line) = line {
        trace = trace.line(line);
    }

    let mut plot = Plot::new();
    plot.add_trace(trace);
    let mut layout = Layout::new().update_menus(vec![parcats_order_toggle_menu(&ordered)]);
    if let Some(title) = &args.flag_title {
        layout = layout.title(Title::with_text(title));
    }
    plot.set_layout(apply_theme(layout, args.theme()));
    Ok(plot)
}

/// Build the parcats dimensions with a count-ordered `categoryarray` baked into each axis (so the
/// heaviest categories cluster together — the view the chart opens in), plus a `ParcatsLine` that
/// colors each ribbon by its FIRST dimension's category (color-by-source, like a Sankey) and
/// bundles like colors. `tuples` are the distinct category combinations, `weights` their row
/// counts. Returns the dimensions, the colored line (`None` when there are no dimensions), and the
/// per-dimension count-ordered category arrays (fed to the order toggle so it can re-supply
/// `categoryarray` — plotly drops it from the trace whenever `categoryorder` leaves "array").
fn parcats_dimensions_and_line(
    labels: &[String],
    tuples: &[Vec<String>],
    weights: &[f64],
) -> (
    Vec<ParcatsDimension<String>>,
    Option<ParcatsLine>,
    Vec<Vec<String>>,
) {
    let ndims = labels.len();
    // per-dimension categories ordered by descending total weight (tie-break by name) — the baked
    // `categoryarray` the "category order" toggle switches on and off.
    let ordered: Vec<Vec<String>> = (0..ndims)
        .map(|pos| {
            let mut totals: HashMap<&str, f64> = HashMap::new();
            for (t, &w) in tuples.iter().zip(weights) {
                *totals.entry(t[pos].as_str()).or_insert(0.0) += w;
            }
            let mut cats: Vec<(&str, f64)> = totals.into_iter().collect();
            cats.sort_by(|a, b| b.1.total_cmp(&a.1).then_with(|| a.0.cmp(b.0)));
            cats.into_iter().map(|(c, _)| c.to_string()).collect()
        })
        .collect();

    let dimensions: Vec<ParcatsDimension<String>> = labels
        .iter()
        .enumerate()
        .map(|(pos, label)| {
            let values: Vec<String> = tuples.iter().map(|t| t[pos].clone()).collect();
            ParcatsDimension::new()
                .label(label.clone())
                .values(values)
                .category_order(CategoryOrder::Array)
                .category_array(ordered[pos].clone())
        })
        .collect();

    // color each ribbon by the rank of its first-dimension category (0..ncats-1), so ribbons
    // sharing an origin share a color; a sequential colorscale over the (usually few, since the
    // first axis is the coarsest) origin categories. No colorbar — the coloring is categorical.
    let line = ordered.first().map(|first| {
        let rank: HashMap<&str, usize> = first
            .iter()
            .enumerate()
            .map(|(i, c)| (c.as_str(), i))
            .collect();
        let colors: Vec<f64> = tuples.iter().map(|t| rank[t[0].as_str()] as f64).collect();
        let mut l = ParcatsLine::new()
            .color_array(colors)
            .color_scale(ColorScale::Palette(ColorScalePalette::Viridis))
            .show_scale(false);
        if first.len() > 1 {
            l = l.cmin(0.0).cmax((first.len() - 1) as f64);
        }
        l
    });
    (dimensions, line, ordered)
}

/// On-graph "category order" toggle for a parcats chart (standalone `viz parcats` and the `viz
/// smart` flow panel), mirroring the Sankey node-order toggle. Every dimension opens count-ordered
/// (its `categoryarray` — passed here as `ordered` — is baked into the trace and honored only while
/// `categoryorder == "array"`); the button flips every dimension's `categoryorder` to alphabetical
/// ("category ascending") and back. `args` fires on the FIRST click, so — since the chart opens in
/// array/count order — it is the alphabetical state (click 1 flips, click 2 restores), matching
/// `sankey_order_toggle_menu`.
///
/// The "back to array" state (`args2`) re-supplies each dimension's `categoryarray` alongside
/// `categoryorder: "array"`, because plotly DROPS `categoryarray` from the trace the moment
/// `categoryorder` leaves "array" — without re-supplying it the count order would be lost after the
/// first toggle and "array" would fall back to first-appearance order.
///
/// This stays a fully self-contained NATIVE toggle — `PARCATS_ORDER_SCRIPT` upgrades the flip to
/// an animated one by intercepting the click and running this same payload itself. Plotly's sankey
/// renderer transitions its node positions on every replot (hence the node-order toggle animates
/// for free), but the parcats renderer transitions only inside its own drag handler — its plot
/// entry point drops the `transitionOpts` plotly hands it, so a plain restyle always snaps.
///
/// The button is deliberately NOT marked `execute: false`: the figure JSON emitted here is also
/// lifted out of the page on its own (`examples/viz/gen_gallery.py` extracts the `Plotly.newPlot`
/// object and reassembles it under its own scaffold). A button that delegated its restyle to a
/// script would be DEAD wherever the script didn't follow it. Keeping the payload native means the
/// worst case is the un-animated snap this always did.
fn parcats_order_toggle_menu(ordered: &[Vec<String>]) -> UpdateMenu {
    let ndims = ordered.len();
    // alphabetical state: just flip categoryorder (categoryarray is ignored while != "array").
    let alpha: serde_json::Map<String, serde_json::Value> = (0..ndims)
        .map(|i| {
            (
                format!("dimensions[{i}].categoryorder"),
                serde_json::json!(["category ascending"]),
            )
        })
        .collect();
    // count/array state: restore BOTH categoryorder="array" AND the baked categoryarray.
    let mut array_state = serde_json::Map::new();
    for (i, cats) in ordered.iter().enumerate() {
        array_state.insert(
            format!("dimensions[{i}].categoryorder"),
            serde_json::json!(["array"]),
        );
        array_state.insert(
            format!("dimensions[{i}].categoryarray"),
            serde_json::json!([cats]),
        );
    }
    let alpha_args = serde_json::json!([serde_json::Value::Object(alpha), [0]]);
    let array_args = serde_json::json!([serde_json::Value::Object(array_state), [0]]);
    let toggle = Button::new()
        .label(&t!("viz.menu.category_order"))
        .method(ButtonMethod::Restyle)
        .args(alpha_args)
        .args2(array_args);
    UpdateMenu::new()
        .ty(UpdateMenuType::Buttons)
        .buttons(vec![toggle])
        // top-left of the plot, clear of the centered title and the top-right modebar.
        .x(0.0)
        .x_anchor(Anchor::Left)
        .y(1.0)
        .y_anchor(Anchor::Bottom)
        // Lift the pill clear of the FIRST dimension's axis label, which parcats draws just inside
        // the top-left of the plot area — exactly where a y=1/bottom-anchored button lands, so the
        // two overlapped. Bottom padding in PIXELS (not a fractional `y` bump, which would resolve
        // to a different gap on a tall standalone chart than on a short Data Schematic panel), and
        // plotly's automargin grows the top margin to keep the lifted pill inside the panel.
        // Sankey needs none of this: its node labels sit beside the nodes, not above them.
        .pad(plotly::common::Pad::new(0, PARCATS_TOGGLE_LABEL_GAP, 0))
        // a stateless flip: no persistent "pressed" highlight (both orders are valid states).
        .show_active(false)
        .active(-1)
        .background_color(NamedColor::White)
        .border_color(NamedColor::Gray)
        .border_width(1)
        .font(Font::new().family(FONT_FAMILY).color(INK).size(11))
}

/// Pixels of bottom padding under the parcats "category order" pill, lifting it off the first
/// dimension's label. The label is ~14px tall and starts at the plot-area top, so this clears it
/// with ~8px to spare on both a standalone chart and a `viz smart` panel.
const PARCATS_TOGGLE_LABEL_GAP: usize = 22;

/// True when a single category — a value, or empty/null — covers more than
/// `CATEGORICAL_DIM_MAX_DOMINANCE` of the `n` rows, making the column near-constant. qsv stats'
/// `mode_occurrences` (the most-frequent-value count) already counts empty/null when that is the
/// dominant "value", so it captures both null-dominated and value-dominated near-constants; it
/// falls back to the raw null count when the mode stat is absent from the cache (still catches the
/// common mostly-empty case). `n == 0` (unknown row count) disables the screen so behavior degrades
/// to the pre-existing "keep everything".
fn is_near_constant(s: &crate::cmd::stats::StatsData, n: u64) -> bool {
    if n == 0 {
        return false;
    }
    let dominant = s.mode_occurrences.unwrap_or(0).max(s.nullcount);
    #[allow(clippy::cast_precision_loss)]
    let share = dominant as f64 / n as f64;
    share > CATEGORICAL_DIM_MAX_DOMINANCE
}

/// Collect the eligible categorical dimensions shared by the hierarchy (treemap/sunburst) and
/// parallel-categories (parcats) overview panels: the `String` frequency-bar columns whose
/// cardinality is in `[HIER_MIN_DIM_CARDINALITY, CATEGORICAL_MAX_CARDINALITY]`, EXCLUDING
/// near-constant columns (see `is_near_constant`). Restricting to `String` excludes numeric codes
/// and booleans, which make poor — and surprising — flow/nesting levels. Returns `(idx,
/// cardinality, label)` tuples in panel order; callers sort by their own preference (both sort
/// coarsest-first). Both panels draw from this one pool so they stay mutually exclusive on the same
/// dimensions.
fn eligible_categorical_dims(
    panels: &[Panel],
    stats: &[crate::cmd::stats::StatsData],
    col_sems: &[ColSemantics],
    n: u64,
) -> Vec<(usize, u64, String)> {
    panels
        .iter()
        .filter_map(|p| match p.kind {
            PanelKind::FreqBar { idx } => {
                let s = &stats[idx];
                let card = s.cardinality;
                let eligible = s.r#type == "String"
                    && (HIER_MIN_DIM_CARDINALITY..=CATEGORICAL_MAX_CARDINALITY).contains(&card)
                    && !is_near_constant(s, n);
                eligible.then(|| {
                    // composite panels keep the dictionary's human label (like measure-by-dim /
                    // bivariate) rather than the raw field name per-column titles use; fall back to
                    // the header, or a positional name when headerless.
                    let sem = &col_sems[idx];
                    let label = if !sem.label.is_empty() {
                        sem.label.clone()
                    } else if s.field.is_empty() {
                        format!("col {}", idx + 1)
                    } else {
                        s.field.clone()
                    };
                    (idx, card, label)
                })
            },
            _ => None,
        })
        .collect()
}

/// Build the `viz smart` parallel-categories (parcats) overview panel over 3-4 associated
/// low-cardinality categorical dimensions. Returns the panel plus the dimension indices it claimed
/// (so the hierarchy panel can stay mutually exclusive on the same columns). Returns `None` when
/// there are fewer than `PARCATS_MIN_DIMS` eligible dimensions, when they are statistically
/// independent (max pairwise Cramér's V below `HIER_MIN_ASSOCIATION_CRAMERS_V`) — flowing
/// independent categoricals just replicates each marginal — or when they form a strict rollup tree
/// (a genuine part-to-whole nesting the treemap/sunburst panel visualizes better). HTML-only
/// (domain-based, like Sankey).
fn parcats_panel(
    args: &Args,
    mut dims: Vec<(usize, u64, String)>,
) -> CliResult<Option<(Panel, Vec<usize>)>> {
    if dims.len() < PARCATS_MIN_DIMS {
        return Ok(None);
    }
    // coarsest (lowest-cardinality) dimensions first, capped so the ribbon flow stays legible.
    dims.sort_by_key(|&(idx, card, _)| (card, idx));
    dims.truncate(PARCATS_MAX_DIMS);
    let depth = dims.len();
    let dim_idxs: Vec<usize> = dims.iter().map(|&(idx, ..)| idx).collect();
    // independence screen (same as the hierarchy panel; reads the joint counts, no extra pass here
    // beyond the one aggregation).
    let leaves = collect_hierarchy_counts(args, &dim_idxs, None)?;
    if max_pairwise_cramers_v(&leaves, depth) < HIER_MIN_ASSOCIATION_CRAMERS_V {
        return Ok(None);
    }
    // a strict rollup tree (each finer category has a single parent) is a genuine part-to-whole
    // hierarchy — leave it to the treemap/sunburst panel, which sizes each ring by rolled-up
    // totals. parcats is for many-to-many (co-occurrence) categorical relationships, where a
    // category legitimately appears under several parents.
    if is_strict_nesting(&leaves, depth) {
        return Ok(None);
    }
    // one ribbon per distinct category tuple, weighted by count; sorted (count desc, then tuple)
    // for deterministic output.
    let mut paths: Vec<(Vec<String>, f64)> = leaves.into_iter().collect();
    paths.sort_by(|a, b| b.1.total_cmp(&a.1).then_with(|| a.0.cmp(&b.0)));
    // bound the emitted trace: up to PARCATS_MAX_DIMS dims of up to CATEGORICAL_MAX_CARDINALITY
    // values each could yield hundreds of thousands of distinct tuples. Keep only the heaviest
    // ribbons (a parcats with thousands of near-zero-count paths is an unreadable tangle and bloats
    // the HTML) and log the dropped tail.
    if paths.len() > PARCATS_MAX_PATHS {
        let dropped = paths.len() - PARCATS_MAX_PATHS;
        let dropped_rows: f64 = paths[PARCATS_MAX_PATHS..].iter().map(|(_, c)| *c).sum();
        viz_note(&format!(
            "viz smart: parcats panel capped to the {PARCATS_MAX_PATHS} most common category \
             combinations ({dropped} rarer combinations covering {dropped_rows:.0} rows omitted)."
        ));
        paths.truncate(PARCATS_MAX_PATHS);
    }
    let counts: Vec<f64> = paths.iter().map(|(_, c)| *c).collect();
    let tuples: Vec<Vec<String>> = paths.into_iter().map(|(t, _)| t).collect();
    let dim_labels: Vec<String> = dims.into_iter().map(|(.., name)| name).collect();
    let title = t!(
        "viz.title.category_flow",
        q_dims = dim_labels.join(" \u{b7} ")
    )
    .into_owned();
    let panel = Panel::new(
        title,
        PanelKind::Parcats {
            dim_labels,
            tuples,
            counts,
        },
    );
    Ok(Some((panel, dim_idxs)))
}

/// Returns true when the joint category tuples form a strict rollup tree: across every adjacent
/// dimension pair, each finer-level category maps to exactly one coarser-level category (a
/// functional dependency finer → coarser). Such a set is a genuine part-to-whole hierarchy that
/// the treemap/sunburst panel sizes by rolled-up totals, so parcats abstains and leaves it to the
/// hierarchy panel; a many-to-many (co-occurrence) set — where a category appears under several
/// parents — is parcats' domain. `tuples` are keyed coarsest-first (index 0 = coarsest level),
/// matching `collect_hierarchy_counts`.
fn is_strict_nesting(tuples: &HashMap<Vec<String>, f64>, depth: usize) -> bool {
    if depth < 2 {
        return false;
    }
    for level in 0..depth - 1 {
        // child (finer, level+1) → parent (coarser, level) must be a function: a single parent.
        let mut child_to_parent: HashMap<&str, &str> = HashMap::new();
        for key in tuples.keys() {
            let parent = key[level].as_str();
            let child = key[level + 1].as_str();
            match child_to_parent.get(child) {
                Some(&existing) if existing != parent => return false,
                Some(_) => {},
                None => {
                    child_to_parent.insert(child, parent);
                },
            }
        }
    }
    true
}

fn build_histogram(args: &Args) -> CliResult<(Box<dyn Trace>, String)> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let x_idx = resolve_one(args.flag_x.as_ref(), &headers, nh, "x")?;

    let mut values: Vec<f64> = Vec::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        if let Some(v) = parse_f64(record.get(x_idx)) {
            values.push(v);
        }
    }
    if values.is_empty() {
        return fail_clierror!("No numeric values found in the --x column for the histogram.");
    }

    let label = col_label(&headers, x_idx, nh);
    let mut hist = Histogram::new(values).name(escape_hover(&label));
    if let Some(bins) = args.flag_bins {
        hist = hist.n_bins_x(bins);
    }
    Ok((hist, label))
}

fn build_box(args: &Args) -> CliResult<(Box<dyn Trace>, String, Option<String>)> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let y_idx = resolve_one(args.flag_y.as_ref(), &headers, nh, "y")?;
    let group_idx = match args.flag_x.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "x")?),
        None => None,
    };

    let mut ys: Vec<f64> = Vec::new();
    let mut groups: Vec<String> = Vec::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let Some(y) = parse_f64(record.get(y_idx)) else {
            continue;
        };
        ys.push(y);
        if let Some(i) = group_idx {
            groups.push(cell_to_string(record.get(i)));
        }
    }
    if ys.is_empty() {
        return fail_clierror!("No numeric values found in the --y column for the box plot.");
    }

    // Unlike `viz smart` (which draws boxes from precomputed quartiles with observed
    // min/max whiskers, since the stats cache has no per-row data), explicit `viz box`
    // reads the raw values, so plotly can render true Tukey whiskers (extending to the
    // most extreme point within 1.5*IQR of the quartiles) and plot the points beyond
    // those fences as individual outliers. `QuartileMethod::Linear` is the standard
    // (linear-interpolation) quartile definition Tukey fences are built on.
    let box_points = parse_box_points(args.flag_box_points.as_deref())?;

    let y_label = col_label(&headers, y_idx, nh);
    let x_label = group_idx.map(|i| col_label(&headers, i, nh));
    let trace: Box<dyn Trace> = if group_idx.is_some() {
        BoxPlot::new_xy(groups, ys)
            .quartile_method(QuartileMethod::Linear)
            .box_points(box_points)
    } else {
        BoxPlot::new(ys)
            .name(escape_hover(y_label.as_str()))
            .quartile_method(QuartileMethod::Linear)
            .box_points(box_points)
    };
    Ok((trace, y_label, x_label))
}

/// Build a violin trace: a KDE density silhouette around an inner quartile box and mean
/// line. Same inputs as `build_box` (--y values, optional --x group); like `viz box`, it
/// reads the raw values — a violin's KDE cannot be drawn from precomputed quartiles.
fn build_violin(args: &Args) -> CliResult<(Box<dyn Trace>, String, Option<String>)> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let y_idx = resolve_one(args.flag_y.as_ref(), &headers, nh, "y")?;
    let group_idx = match args.flag_x.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "x")?),
        None => None,
    };

    let mut ys: Vec<f64> = Vec::new();
    let mut groups: Vec<String> = Vec::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let Some(y) = parse_f64(record.get(y_idx)) else {
            continue;
        };
        ys.push(y);
        if let Some(i) = group_idx {
            groups.push(cell_to_string(record.get(i)));
        }
    }
    if ys.is_empty() {
        return fail_clierror!("No numeric values found in the --y column for the violin plot.");
    }

    let points = violin_points(&parse_box_points(args.flag_box_points.as_deref())?);

    let y_label = col_label(&headers, y_idx, nh);
    let x_label = group_idx.map(|i| col_label(&headers, i, nh));
    let trace: Box<dyn Trace> = if group_idx.is_some() {
        Violin::new_xy(groups, ys)
            .quartile_method(QuartileMethod::Linear)
            .box_plot(ViolinBox::new().visible(true))
            .mean_line(MeanLine::new().visible(true))
            .points(points)
    } else {
        Violin::new(ys)
            .name(escape_hover(y_label.as_str()))
            .quartile_method(QuartileMethod::Linear)
            .box_plot(ViolinBox::new().visible(true))
            .mean_line(MeanLine::new().visible(true))
            .points(points)
    };
    Ok((trace, y_label, x_label))
}

/// Resolve a required multi-column selector to its column indices (one or more).
fn resolve_many(
    sel: Option<&SelectColumns>,
    headers: &csv::ByteRecord,
    no_headers: bool,
    flag: &str,
) -> CliResult<Vec<usize>> {
    let Some(sel) = sel else {
        return fail_incorrectusage_clierror!("--{flag} is required for this chart type.");
    };
    let selection = sel
        .selection(headers, !no_headers)
        .map_err(crate::CliError::Other)?;
    if selection.is_empty() {
        return fail_incorrectusage_clierror!("--{flag} selected no columns.");
    }
    Ok(selection.iter().copied().collect())
}

/// From an already-open reader, keep the candidate columns that are numeric (a majority of
/// non-empty cells parse as f64), and return their labels plus listwise-complete value vectors
/// (rows where any kept column is non-numeric/empty are dropped). Takes the reader + headers
/// from the caller (rather than opening its own) so the input is read exactly once — opening a
/// second time would consume/corrupt a streamed stdin. Shared by the standalone correlation
/// heatmap and the `viz smart` correlation panel.
///
/// When `drop_near_unique` is set, near-unique identifier columns (a distinct value in > 95% of
/// numeric rows — a serial id, a monotonic order key) are dropped from the kept set *before*
/// the listwise row-drop, so a sparse or partially-non-numeric identifier can't discard rows
/// from the surviving measures (mirrors `viz smart`, which pre-filters identifiers via the
/// stats cache before calling this). Distinctness is tracked as a `u64` hash of the raw cell
/// bytes (not the parsed f64), so large integer ids beyond f64's 53-bit mantissa don't collapse
/// and evade the filter, while never duplicating the cell strings themselves. Only numeric cells
/// are tracked, so wide inputs with large free-form text columns (discarded as non-numeric) cost
/// nothing. Guard: columns are stripped only when >= 2 non-near-unique columns remain as a
/// correlation core; on tiny or all-continuous inputs every column is all-distinct and nothing
/// is stripped.
fn read_numeric_columns(
    rdr: &mut csv::Reader<Box<dyn std::io::Read + Send>>,
    headers: &csv::ByteRecord,
    nh: bool,
    candidates: &[usize],
    drop_near_unique: bool,
) -> CliResult<(Vec<String>, Vec<Vec<f64>>, Vec<usize>)> {
    use std::hash::{Hash, Hasher};

    // NaN is the missing sentinel rather than `Option<f64>`: `parse_f64` accepts only FINITE
    // values, so NaN can never be a legitimate reading, and this halves the buffer from 16 to 8
    // bytes per row per candidate — the dominant memory cost when `--cols` is omitted and every
    // column is a candidate.
    //
    // Deliberately NOT sampled: deciding the majority-numeric keep-set from the first N rows
    // would let us skip buffering the rest, but a column that reads as text early and numeric
    // later would be dropped on evidence the full pass contradicts. The verdict has to see every
    // row, and the reader is a single-pass stream (stdin included), so the values must be held.
    let mut raw: Vec<Vec<f64>> = vec![Vec::new(); candidates.len()];
    let mut nonempty = vec![0_usize; candidates.len()];
    let mut parsed = vec![0_usize; candidates.len()];
    // distinct hashes of the raw bytes of numeric cells per candidate — populated only when we
    // must detect near-unique identifiers. Storing an 8-byte hash (not the cell bytes) bounds
    // memory and avoids duplicating strings; tracking only numeric cells means large free-form
    // text columns (dropped below as non-numeric) contribute nothing.
    let mut distinct: Vec<std::collections::HashSet<u64>> = if drop_near_unique {
        vec![std::collections::HashSet::new(); candidates.len()]
    } else {
        Vec::new()
    };
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        for (k, &idx) in candidates.iter().enumerate() {
            let cell = record.get(idx);
            let is_nonempty = cell.is_some_and(|c| !c.iter().all(u8::is_ascii_whitespace));
            let v = parse_f64(cell);
            if is_nonempty {
                nonempty[k] += 1;
                if v.is_some() {
                    parsed[k] += 1;
                    if drop_near_unique {
                        let mut hasher = std::collections::hash_map::DefaultHasher::new();
                        cell.expect("nonempty implies Some").hash(&mut hasher);
                        distinct[k].insert(hasher.finish());
                    }
                }
            }
            raw[k].push(v.unwrap_or(f64::NAN));
        }
    }
    // keep majority-numeric columns (drops text/ID/date columns from an all-columns default)
    let mut keep: Vec<usize> = (0..candidates.len())
        .filter(|&k| nonempty[k] > 0 && parsed[k] * 2 >= nonempty[k])
        .collect();
    // strip near-unique identifiers from the kept set BEFORE the listwise transpose below, so a
    // sparse identifier's blank rows don't drop rows from the surviving measures. A kept column
    // has parsed[k] >= 1 (majority-numeric with nonempty > 0), so the ratio is well-defined.
    // Guarded on a >= 2 non-near-unique correlation core.
    if drop_near_unique {
        let near_unique: Vec<bool> = keep
            .iter()
            .map(|&k| {
                #[allow(clippy::cast_precision_loss)]
                let ratio = distinct[k].len() as f64 / parsed[k] as f64;
                ratio > 0.95
            })
            .collect();
        if near_unique.iter().filter(|&&nu| !nu).count() >= 2 {
            keep = keep
                .into_iter()
                .zip(near_unique)
                .filter_map(|(k, nu)| (!nu).then_some(k))
                .collect();
        }
    }
    if keep.is_empty() {
        return Ok((Vec::new(), Vec::new(), Vec::new()));
    }
    // original source-column index for each kept column (kept columns may be a subset of
    // `candidates` after the majority-numeric / near-unique filters), so callers can map a
    // by-position pick (e.g. the strongest correlation pair) back to its source column.
    let kept_indices: Vec<usize> = keep.iter().map(|&k| candidates[k]).collect();
    let labels: Vec<String> = keep
        .iter()
        .map(|&k| col_label(headers, candidates[k], nh))
        .collect();
    let n_rows = raw[0].len();
    let kept: Vec<&Vec<f64>> = keep.iter().map(|&k| &raw[k]).collect();
    let mut columns: Vec<Vec<f64>> = vec![Vec::new(); kept.len()];
    // transpose the kept columns, dropping any row where a kept column is non-numeric/empty
    for r in 0..n_rows {
        if kept.iter().all(|col| !col[r].is_nan()) {
            for (out, col) in columns.iter_mut().zip(&kept) {
                out.push(col[r]);
            }
        }
    }
    Ok((labels, columns, kept_indices))
}

/// Pearson correlation of two equal-length numeric slices via a numerically stable, centered
/// two-pass algorithm (raw-sums formulas suffer catastrophic cancellation for large values
/// with small variance). The denominator is `var_x.sqrt() * var_y.sqrt()` rather than
/// `(var_x * var_y).sqrt()` so it stays finite for large-but-valid variances (the product
/// would overflow to infinity and spuriously yield `NaN`). Returns `NaN` when the correlation
/// is undefined — fewer than two points, or zero variance in either input — so callers can
/// render it as a gap rather than a fabricated 0.0. The result is clamped to [-1, 1] to absorb
/// floating-point overshoot.
fn pearson(x: &[f64], y: &[f64]) -> f64 {
    let len = x.len().min(y.len());
    if len < 2 {
        return f64::NAN;
    }
    let n = len as f64;
    let mean_x = x[..len].iter().sum::<f64>() / n;
    let mean_y = y[..len].iter().sum::<f64>() / n;
    let (mut cov, mut var_x, mut var_y) = (0.0, 0.0, 0.0);
    for k in 0..len {
        let dx = x[k] - mean_x;
        let dy = y[k] - mean_y;
        cov = dx.mul_add(dy, cov);
        var_x = dx.mul_add(dx, var_x);
        var_y = dy.mul_add(dy, var_y);
    }
    let den = var_x.sqrt() * var_y.sqrt();
    if den == 0.0 || !den.is_finite() {
        f64::NAN
    } else {
        (cov / den).clamp(-1.0, 1.0)
    }
}

/// Tie-averaged (fractional) ranks of `v`, 1-based, for Spearman's rho. Equal values share the
/// mean of the ranks they span, so tied data isn't biased by an arbitrary ordering.
fn average_ranks(v: &[f64]) -> Vec<f64> {
    let n = v.len();
    let mut idx: Vec<usize> = (0..n).collect();
    idx.sort_by(|&a, &b| v[a].partial_cmp(&v[b]).unwrap_or(std::cmp::Ordering::Equal));
    let mut ranks = vec![0.0_f64; n];
    let mut i = 0;
    while i < n {
        let mut j = i + 1;
        while j < n && v[idx[j]] == v[idx[i]] {
            j += 1;
        }
        // mean of the 1-based ranks (i+1)..=j spanned by this tie group.
        #[allow(clippy::cast_precision_loss)]
        let avg = (i + 1 + j) as f64 / 2.0;
        for &k in &idx[i..j] {
            ranks[k] = avg;
        }
        i = j;
    }
    ranks
}

/// Spearman's rank correlation: Pearson on the (tie-averaged) ranks of each column. It measures
/// MONOTONIC association, so a large `|spearman| - |pearson|` gap signals a monotonic-but-curved
/// (nonlinear) relationship — one a single Pearson r understates. Returns `NaN` when undefined
/// (propagated from `pearson`).
fn spearman_rho(x: &[f64], y: &[f64]) -> f64 {
    pearson(&average_ranks(x), &average_ranks(y))
}

/// Blank out the upper triangle AND the diagonal of a symmetric correlation matrix (set to `NaN`,
/// which serializes to JSON `null` so plotly draws no cell and `corr_incell_annotations` skips it),
/// leaving only the lower triangle. A correlation matrix mirrors across the diagonal and the
/// diagonal is a trivial 1.0, so the full square wastes half the panel on redundant cells; the
/// lower triangle is the standard, less-cluttered presentation.
fn mask_to_lower_triangle(mut matrix: Vec<Vec<f64>>) -> Vec<Vec<f64>> {
    for (r, row) in matrix.iter_mut().enumerate() {
        for (c, cell) in row.iter_mut().enumerate() {
            if c >= r {
                *cell = f64::NAN;
            }
        }
    }
    matrix
}

/// Pick the least-redundant third axis for a 3D scatter, given the two columns `i`/`j` already
/// chosen as the strongest-correlated pair. "Least redundant" = the candidate `k` whose strongest
/// correlation to either chosen axis (`max(|r_ik|, |r_jk|)`) is smallest, so the third dimension
/// adds the most new information.
///
/// Candidates are restricted to those with DEFINED correlations to both chosen axes. A numeric
/// column can become constant (zero-variance) on the listwise-complete rows even though column
/// stats admitted it, leaving NaN correlations; `NaN.partial_cmp(..)` falls back to `Equal`, so an
/// unfiltered `min_by` could select such a degenerate column as "least redundant" and render a
/// flat, meaningless 3D axis. Returns `None` when no finite candidate remains (caller skips 3D).
fn least_redundant_third(matrix: &[Vec<f64>], i: usize, j: usize) -> Option<usize> {
    (0..matrix.len())
        .filter(|&k| k != i && k != j && matrix[i][k].is_finite() && matrix[j][k].is_finite())
        .min_by(|&a, &b| {
            let ra = matrix[i][a].abs().max(matrix[j][a].abs());
            let rb = matrix[i][b].abs().max(matrix[j][b].abs());
            ra.partial_cmp(&rb).unwrap_or(std::cmp::Ordering::Equal)
        })
}

/// Pick the third axis whose values best reveal structure as marker SIZE on the
/// strongest-correlated scatter pair `i`/`j`: the candidate `k` MOST associated with either chosen
/// axis (`max(|r_ik|, |r_jk|)` largest), so bubble size shows the steepest gradient across the
/// cloud. This is the complement of `least_redundant_third` (which `Scatter3D` uses), so the bubble
/// and 3D drill-downs encode different, each-meaningful third variables rather than the same one
/// twice. Same NaN-guarding as `least_redundant_third`: candidates need DEFINED correlations to
/// both chosen axes, else a zero-variance column's NaN could win. Returns `None` when no finite
/// candidate remains (caller renders a plain 2-D scatter).
fn most_associated_third(matrix: &[Vec<f64>], i: usize, j: usize) -> Option<usize> {
    (0..matrix.len())
        .filter(|&k| k != i && k != j && matrix[i][k].is_finite() && matrix[j][k].is_finite())
        .max_by(|&a, &b| {
            let ra = matrix[i][a].abs().max(matrix[j][a].abs());
            let rb = matrix[i][b].abs().max(matrix[j][b].abs());
            ra.partial_cmp(&rb).unwrap_or(std::cmp::Ordering::Equal)
        })
}

/// Pearson correlation matrix for equal-length numeric columns: NxN symmetric. The diagonal
/// is computed from `pearson` (rather than hard-coded to 1.0) so a degenerate column — zero
/// variance or too few observations — surfaces as `NaN` instead of a fabricated 1.0. Cells for
/// undefined correlations are likewise `NaN` (serialized to `null` → rendered as a heatmap gap).
fn pearson_matrix(columns: &[Vec<f64>]) -> Vec<Vec<f64>> {
    let n = columns.len();
    let mut m = vec![vec![f64::NAN; n]; n];
    if n == 0 {
        return m;
    }
    // Calling `pearson` per pair re-derived BOTH column means and both variances every time —
    // ~5 passes per pair, with each column's statistics recomputed N times. Centre each column
    // once (O(N * rows)), after which a pair is a single dot product. The centred values and the
    // mul_add accumulation order match `pearson` exactly, so results are unchanged.
    let len = columns.iter().map(Vec::len).min().unwrap_or(0);
    if len < 2 {
        return m;
    }
    let centered: Vec<(Vec<f64>, f64)> = columns
        .iter()
        .map(|c| {
            let mean = c[..len].iter().sum::<f64>() / len as f64;
            let cv: Vec<f64> = c[..len].iter().map(|v| v - mean).collect();
            let var = cv.iter().fold(0.0, |acc, d| d.mul_add(*d, acc));
            (cv, var.sqrt())
        })
        .collect();
    let dot = |a: &[f64], b: &[f64]| -> f64 {
        let mut cov = 0.0;
        for k in 0..len {
            cov = a[k].mul_add(b[k], cov);
        }
        cov
    };
    let corr = |i: usize, j: usize| -> f64 {
        let den = centered[i].1 * centered[j].1;
        if den == 0.0 || !den.is_finite() {
            f64::NAN
        } else {
            (dot(&centered[i].0, &centered[j].0) / den).clamp(-1.0, 1.0)
        }
    };
    // one row of the upper triangle per task; the matrix is filled symmetrically afterwards
    let rows: Vec<(usize, Vec<(usize, f64)>)> = (0..n)
        .into_par_iter()
        .map(|i| (i, (i..n).map(|j| (j, corr(i, j))).collect()))
        .collect();
    for (i, cells) in rows {
        for (j, r) in cells {
            m[i][j] = r;
            m[j][i] = r;
        }
    }
    m
}

/// Spearman rank-correlation matrix: `pearson_matrix` over each column's tie-averaged ranks
/// (issue #4220). Ranks are taken ONCE per column here rather than inside a per-pair
/// `spearman_rho`, which would re-sort every column N times for an N x N matrix.
///
/// Each column is ranked over exactly the prefix `pearson_matrix` will use (the shortest column's
/// length), so a rank is the value's position among the rows that actually enter the correlation.
fn spearman_matrix(columns: &[Vec<f64>]) -> Vec<Vec<f64>> {
    let len = columns.iter().map(Vec::len).min().unwrap_or(0);
    let ranked: Vec<Vec<f64>> = columns.iter().map(|c| average_ranks(&c[..len])).collect();
    pearson_matrix(&ranked)
}

/// Whether `viz smart`'s correlation matrix should be computed with Spearman's rho instead of
/// Pearson's r: true when at least `SMART_CORR_SPEARMAN_MIN_SKEWED_SHARE` of the matrix's columns
/// have an outlier-driven mean (`mean_is_outlier_driven`, issue #4220). Pearson's covariance is
/// built from deviations about those means, so on such columns the reported r describes the tail
/// rows; ranks are immune.
///
/// `kept_indices` maps a matrix position back to its ORIGINAL column index — `read_numeric_columns`
/// drops candidates that turn out to be unreadable, so the matrix is not necessarily indexed like
/// `stats`. Looking the skew up by the wrong index would silently gate on another column.
fn corr_prefers_spearman(stats: &[crate::cmd::stats::StatsData], kept_indices: &[usize]) -> bool {
    if kept_indices.is_empty() {
        return false;
    }
    let skewed = kept_indices
        .iter()
        .filter(|&&i| stats.get(i).is_some_and(mean_is_outlier_driven))
        .count();
    #[allow(clippy::cast_precision_loss)]
    let share = skewed as f64 / kept_indices.len() as f64;
    share >= SMART_CORR_SPEARMAN_MIN_SKEWED_SHARE
}

/// Index pair (i, j) with the largest absolute Pearson correlation in the off-diagonal of a
/// symmetric matrix, returned with its signed r. Skips NaN cells (undefined correlations, e.g.
/// a constant column). Returns None when the matrix has fewer than two columns or every
/// off-diagonal cell is NaN.
fn strongest_pair(matrix: &[Vec<f64>]) -> Option<(usize, usize, f64)> {
    let mut best: Option<(usize, usize, f64)> = None;
    for (i, row) in matrix.iter().enumerate() {
        for (j, &r) in row.iter().enumerate().skip(i + 1) {
            if r.is_nan() {
                continue;
            }
            if best.is_none_or(|(_, _, b)| r.abs() > b.abs()) {
                best = Some((i, j, r));
            }
        }
    }
    best
}

/// A diverging (`RdBu`) correlation heatmap trace fixed to the [-1, 1] scale. `axes` assigns
/// the subplot axis refs when used as a `viz smart` panel (None for the standalone chart).
/// A `hovertemplate` (and trace name) gives a clean `y vs x: r` tooltip instead of plotly's
/// default "trace 0". `spearman` swaps that hover symbol to rho, so a rank-correlation matrix
/// (`corr_prefers_spearman`, issue #4220) never labels its cells with Pearson's r.
fn corr_heatmap_trace(
    labels: Vec<String>,
    matrix: Vec<Vec<f64>>,
    axes: Option<(String, String)>,
    show_scale: bool,
    spearman: bool,
) -> Box<dyn Trace> {
    let (symbol, name) = if spearman {
        ("\u{3c1}", t!("viz.chart.trace_rank_correlation"))
    } else {
        ("r", t!("viz.chart.trace_correlation"))
    };
    heatmap_trace_with_range(
        labels,
        matrix,
        axes,
        show_scale,
        -1.0,
        1.0,
        Some(0.0),
        ColorScalePalette::RdBu,
        symbol,
        &name,
        None,
    )
}

/// `viz smart --bivariate`: a sequential (Viridis) [0, 1] heatmap trace for the NMI association
/// matrix. `hover_suffix`, when present, is rendered per-cell via plotly's `text` + `%{text}` so a
/// flagged-nonlinear pair's hover carries its Pearson/Spearman divergence note.
fn assoc_heatmap_trace(
    labels: Vec<String>,
    matrix: Vec<Vec<f64>>,
    hover_suffix: Vec<Vec<String>>,
    axes: Option<(String, String)>,
    show_scale: bool,
) -> Box<dyn Trace> {
    heatmap_trace_with_range(
        labels,
        matrix,
        axes,
        show_scale,
        0.0,
        1.0,
        None,
        ColorScalePalette::Viridis,
        "NMI",
        &t!("viz.chart.trace_association"),
        Some(hover_suffix),
    )
}

/// Shared heatmap-trace builder behind `corr_heatmap_trace` (diverging, [-1, 1]) and
/// `assoc_heatmap_trace` (sequential, [0, 1]): a fixed-range, named heatmap with a clean
/// `y ↔ x: <label> = value` hover (instead of plotly's default "trace 0"). `axes` assigns the
/// subplot axis refs when used as a `viz smart` panel (`None` for a standalone chart).
/// `text_matrix`, when present (same shape as `matrix`), appends a per-cell hover suffix via
/// `%{text}` — used to flag nonlinear numeric pairs in the association heatmap.
#[allow(clippy::too_many_arguments)]
fn heatmap_trace_with_range(
    labels: Vec<String>,
    matrix: Vec<Vec<f64>>,
    axes: Option<(String, String)>,
    show_scale: bool,
    zmin: f64,
    zmax: f64,
    zmid: Option<f64>,
    palette: ColorScalePalette,
    hover_label: &str,
    name: &str,
    text_matrix: Option<Vec<Vec<String>>>,
) -> Box<dyn Trace> {
    let mut hover_template = format!("%{{y}} \u{2194} %{{x}}<br>{hover_label} = %{{z:.3f}}");
    if text_matrix.is_some() {
        hover_template.push_str("%{text}");
    }
    hover_template.push_str("<extra></extra>");

    let mut h = HeatMap::new(labels.clone(), labels, matrix)
        .color_scale(ColorScale::Palette(palette))
        .zmin(zmin)
        .zmax(zmax)
        .show_scale(show_scale)
        .name(name)
        .hover_template(hover_template);
    if let Some(zmid) = zmid {
        h = h.zmid(zmid);
    }
    if let Some(tm) = text_matrix {
        h = h.text_matrix(tm);
    }
    if let Some((x, y)) = axes {
        h = h.x_axis(x).y_axis(y);
    }
    h
}

/// Truncate a label to at most `max` characters (Unicode-aware), appending an ellipsis when
/// shortened. Used for chart axis ticks so long column names don't clip against the page edge.
/// NOTE: truncation can collide distinct names — where the labels serve as plotly *category
/// coordinates* (heatmaps), use `truncate_labels_unique` instead.
fn truncate_label(s: &str, max: usize) -> String {
    if s.chars().count() <= max {
        return s.to_string();
    }
    let keep = max.saturating_sub(1);
    let mut out: String = s.chars().take(keep).collect();
    out.push('\u{2026}');
    out
}

/// `truncate_label` over a label set, disambiguating collisions with a `(2)`, `(3)`, ... suffix.
/// Plotly keys heatmap rows/columns by the category *string*, so two columns whose names share a
/// long prefix (e.g. `total_precipitation_mm` / `total_precipitation_in`) would otherwise
/// truncate identically and silently collapse onto one axis position, misattributing matrix
/// cells. The suffix may exceed `max` by a few chars — cosmetic only (margin sizing clamps).
fn truncate_labels_unique(labels: &[String], max: usize) -> Vec<String> {
    let mut used = std::collections::HashSet::with_capacity(labels.len());
    let mut out = Vec::with_capacity(labels.len());
    for l in labels {
        let base = truncate_label(l, max);
        let mut candidate = base.clone();
        let mut n = 1_usize;
        while !used.insert(candidate.clone()) {
            n += 1;
            candidate = format!("{base}({n})");
        }
        out.push(candidate);
    }
    out
}

/// Pie chart of label proportions: sums --y per --x label, or counts label occurrences when
/// --y is omitted.
fn build_pie(args: &Args) -> CliResult<Box<dyn Trace>> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let label_idx = resolve_one(args.flag_x.as_ref(), &headers, nh, "x")?;
    let value_idx = match args.flag_y.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "y")?),
        None => None,
    };

    let mut order: Vec<String> = Vec::new();
    let mut acc: HashMap<String, f64> = HashMap::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let label = cell_to_string(record.get(label_idx));
        if label.is_empty() {
            continue;
        }
        let inc = match value_idx {
            Some(i) => match parse_f64(record.get(i)) {
                Some(v) => v,
                None => continue,
            },
            None => 1.0,
        };
        if let Some(v) = acc.get_mut(&label) {
            *v += inc;
        } else {
            order.push(label.clone());
            acc.insert(label, inc);
        }
    }
    if order.is_empty() {
        return fail_clierror!("No data found for the pie chart.");
    }
    let values: Vec<f64> = order.iter().map(|l| acc[l]).collect();
    advise_if_pie_hard_to_read(&values);
    let mut pie = Pie::new(values).labels(order).text_info("label+percent");
    if args.flag_donut {
        pie = pie.hole(0.4);
    }
    Ok(pie)
}

/// `viz funnel`: a stage-by-stage drop-off chart. Sums `--y` per `--x` stage, or counts stage
/// occurrences when `--y` is omitted — the same input shape as `viz pie`, and read the same way.
///
/// The two funnels in qsv are complements, not duplicates, and the difference is WHERE the stage
/// order comes from. `viz smart`'s panel takes it from a dictionary declaration
/// (`x-qsv.relationships`, `kind: "pipeline"`) and supports both encodings, so it can chart a
/// row-shaped pipeline too — but only when one was declared. This command takes the order from
/// the FILE: **stage order is first-appearance order**, so it needs no dictionary, no LLM and no
/// declaration. An explicit `qsv viz funnel` is a request, not a detection.
///
/// Neither applies a containment check. Here there is nothing to check against — the user named
/// the stage column — and in `viz smart` containment is measured for disclosure only.
///
/// Stages are NOT sorted by value. A stage that outruns its predecessor is a finding the reader
/// needs to see, and sorting would quietly hide it.
fn build_funnel_chart(args: &Args) -> CliResult<Box<dyn Trace>> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let stage_idx = resolve_one(args.flag_x.as_ref(), &headers, nh, "x")?;
    let value_idx = match args.flag_y.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "y")?),
        None => None,
    };

    let mut order: Vec<String> = Vec::new();
    let mut acc: HashMap<String, f64> = HashMap::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let stage = cell_to_string(record.get(stage_idx));
        if stage.is_empty() {
            continue;
        }
        let inc = match value_idx {
            Some(i) => match parse_f64(record.get(i)) {
                Some(v) => v,
                None => continue,
            },
            None => 1.0,
        };
        if let Some(v) = acc.get_mut(&stage) {
            *v += inc;
        } else {
            order.push(stage.clone());
            acc.insert(stage, inc);
        }
    }
    if order.is_empty() {
        return fail_clierror!("No data found for the funnel chart.");
    }
    let values: Vec<f64> = order.iter().map(|s| acc[s]).collect();
    if values.iter().any(|v| *v < 0.0) {
        return fail_clierror!(
            "A funnel cannot represent a negative stage total; check the --y column."
        );
    }

    // plotly draws index 0 at the TOP and works downward, so first-appearance order is fed as-is.
    Ok(Funnel::new(values, order)
        .orientation(Orientation::Horizontal)
        .text_info("value+percent previous")
        .connector(FunnelConnector::new().visible(true)))
}

/// Non-fatal advisory: a pie of many NEAR-EQUAL slices is the worst case for a pie chart — humans
/// compare angles/areas poorly, so similar slices are nearly indistinguishable and a bar chart is
/// strictly easier to read. Measured by the coefficient of variation (stddev / mean) of the slice
/// values: low CV across enough slices means no slice dominates. Only nudges (prints to stderr);
/// the explicit `viz pie` request is still rendered.
fn advise_if_pie_hard_to_read(values: &[f64]) {
    if values.len() < PIE_NEAR_EQUAL_MIN_SLICES {
        return;
    }
    #[allow(clippy::cast_precision_loss)]
    let n = values.len() as f64;
    let mean = values.iter().sum::<f64>() / n;
    if mean <= 0.0 {
        return;
    }
    let variance = values.iter().map(|v| (v - mean) * (v - mean)).sum::<f64>() / n;
    let cv = variance.sqrt() / mean;
    if cv < PIE_NEAR_EQUAL_MAX_CV {
        viz_note(&format!(
            "viz pie: {} slices are near-equal (coefficient of variation {cv:.2} < \
             {PIE_NEAR_EQUAL_MAX_CV:.2}); a pie makes near-equal slices hard to compare. Consider \
             `viz bar` for an easier read.",
            values.len()
        ));
    }
}

/// Heatmap: a correlation matrix of numeric columns (default), or a category x category
/// pivot when --x, --y and --z are all given.
fn build_heatmap(args: &Args) -> CliResult<(Box<dyn Trace>, Option<String>, Option<String>)> {
    if args.flag_x.is_some() && args.flag_y.is_some() && args.flag_z.is_some() {
        return build_heatmap_pivot(args);
    }
    if args.flag_z.is_some() {
        return fail_incorrectusage_clierror!(
            "heatmap pivot mode needs --x, --y and --z together. Omit --z for a correlation \
             heatmap."
        );
    }
    build_heatmap_correlation(args)
}

fn build_heatmap_correlation(
    args: &Args,
) -> CliResult<(Box<dyn Trace>, Option<String>, Option<String>)> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let explicit_cols = args.flag_cols.is_some();
    let candidates: Vec<usize> = match args.flag_cols.as_ref() {
        Some(s) => resolve_many(Some(s), &headers, nh, "cols")?,
        None => (0..headers.len()).collect(),
    };
    // In auto mode (no explicit --cols), drop near-unique identifier columns (a monotonic order
    // key, a serial id): they hold a distinct value in nearly every row, carry no meaningful linear
    // relationship, and only add a noise row/column to the matrix. `read_numeric_columns` applies
    // this BEFORE its listwise row-drop, so a sparse identifier can't discard rows from the
    // surviving measures. Mirrors `viz smart`'s correlation filter (uniqueness_ratio > 0.95). An
    // explicit --cols is the user's deliberate selection and is charted as-is.
    let (labels, columns, _) =
        read_numeric_columns(&mut rdr, &headers, nh, &candidates, !explicit_cols)?;
    if labels.len() < 2 {
        return fail_clierror!(
            "A correlation heatmap needs at least 2 numeric columns (found {}). Use --cols to \
             select them.",
            labels.len()
        );
    }
    let n_obs = columns.first().map_or(0, Vec::len);
    if n_obs < 2 {
        return fail_clierror!(
            "A correlation heatmap needs at least 2 rows where all selected numeric columns are \
             present (found {n_obs})."
        );
    }
    let matrix = pearson_matrix(&columns);
    // uniquify: plotly keys heatmap axes by CATEGORY, so two identically-named columns (CSV
    // headers are not required to be unique) collapse onto one axis slot and one set of cells is
    // silently hidden. The `viz smart` correlation path already routes through this helper.
    let labels = truncate_labels_unique(&labels, CORR_LABEL_MAX_CHARS);
    // the standalone `viz corr` chart is always Pearson: it has no skew gate, and its --help
    // documents r
    Ok((
        corr_heatmap_trace(labels, matrix, None, true, false),
        None,
        None,
    ))
}

fn build_heatmap_pivot(args: &Args) -> CliResult<(Box<dyn Trace>, Option<String>, Option<String>)> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let x_idx = resolve_one(args.flag_x.as_ref(), &headers, nh, "x")?;
    let y_idx = resolve_one(args.flag_y.as_ref(), &headers, nh, "y")?;
    let z_idx = resolve_one(args.flag_z.as_ref(), &headers, nh, "z")?;

    let mut x_cats: Vec<String> = Vec::new();
    let mut x_pos: HashMap<String, usize> = HashMap::new();
    let mut y_cats: Vec<String> = Vec::new();
    let mut y_pos: HashMap<String, usize> = HashMap::new();
    // (y, x) -> (sum, count) so duplicate cells are averaged
    let mut cells: HashMap<(usize, usize), (f64, f64)> = HashMap::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let Some(z) = parse_f64(record.get(z_idx)) else {
            continue;
        };
        let xv = cell_to_string(record.get(x_idx));
        let yv = cell_to_string(record.get(y_idx));
        let xi = *x_pos.entry(xv.clone()).or_insert_with(|| {
            x_cats.push(xv);
            x_cats.len() - 1
        });
        let yi = *y_pos.entry(yv.clone()).or_insert_with(|| {
            y_cats.push(yv);
            y_cats.len() - 1
        });
        let e = cells.entry((yi, xi)).or_insert((0.0, 0.0));
        e.0 += z;
        e.1 += 1.0;
    }
    if x_cats.is_empty() || y_cats.is_empty() {
        return fail_clierror!("No data found for the heatmap pivot (is --z numeric?).");
    }
    // the `z` matrix below is DENSE, so its size is the PRODUCT of the two cardinalities: refuse
    // before allocating rather than OOM-ing on two high-cardinality key columns.
    let cell_count = x_cats.len().saturating_mul(y_cats.len());
    if cell_count > HEATMAP_PIVOT_MAX_CELLS {
        return fail_clierror!(
            "Heatmap pivot too large: --x has {} distinct values and --y has {}, which is {} \
             cells (max {HEATMAP_PIVOT_MAX_CELLS}).\nUse lower-cardinality --x/--y columns, or \
             pre-filter/bin the data first.",
            x_cats.len(),
            y_cats.len(),
            cell_count
        );
    }
    // z matrix indexed [y][x]; missing cells are NaN (serialized as null -> rendered as a gap)
    let mut z: Vec<Vec<f64>> = vec![vec![f64::NAN; x_cats.len()]; y_cats.len()];
    for ((yi, xi), (sum, cnt)) in cells {
        if cnt > 0.0 {
            z[yi][xi] = sum / cnt;
        }
    }
    let x_label = col_label(&headers, x_idx, nh);
    let y_label = col_label(&headers, y_idx, nh);
    let trace = HeatMap::new(x_cats, y_cats, z)
        .color_scale(ColorScale::Palette(ColorScalePalette::Viridis));
    Ok((trace, Some(x_label), Some(y_label)))
}

/// Bin two row-aligned numeric vectors into a `bins` x `bins` grid of point counts, returning the
/// per-axis bin-center coordinates and the `z[yi][xi]` count matrix. Shared by `viz contour` and
/// `viz smart`'s correlated-pair density panel. `log_axes` selects geometric (log-spaced) bin edges
/// per axis: `viz contour` always passes `(false, false)`, while `viz smart` sets an axis when its
/// positive-subset density retry engaged (issue #4223).
fn bin_2d(
    xs: &[f64],
    ys: &[f64],
    bins: usize,
    log_axes: (bool, bool),
) -> (Vec<f64>, Vec<f64>, Vec<Vec<f64>>) {
    // Bin edges are laid out in the axis's OWN space: linearly for a linear axis, geometrically
    // (evenly spaced in log10) for a log axis — so a heavy right tail gets bins that widen with
    // the values instead of one bin swallowing the whole low mass (issue #4223). log10 of a
    // non-positive value is -inf/NaN, so the `is_finite` guards below double as the "unplottable
    // on a log axis" filter; on a linear axis they simply drop non-finite input.
    let (log_x, log_y) = log_axes;
    let to_axis = |v: f64, log: bool| if log { v.log10() } else { v };
    let from_axis = |v: f64, log: bool| if log { 10_f64.powf(v) } else { v };

    let (mut x_min, mut x_max) = (f64::INFINITY, f64::NEG_INFINITY);
    let (mut y_min, mut y_max) = (f64::INFINITY, f64::NEG_INFINITY);
    for (&x, &y) in xs.iter().zip(ys.iter()) {
        let (ax, ay) = (to_axis(x, log_x), to_axis(y, log_y));
        if !ax.is_finite() || !ay.is_finite() {
            continue;
        }
        x_min = x_min.min(ax);
        x_max = x_max.max(ax);
        y_min = y_min.min(ay);
        y_max = y_max.max(ay);
    }
    // guard against a degenerate (single-value) axis so the bin math never divides by zero
    let x_span = if x_max > x_min { x_max - x_min } else { 1.0 };
    let y_span = if y_max > y_min { y_max - y_min } else { 1.0 };

    // z[yi][xi] = number of points falling in that cell
    let mut z = vec![vec![0.0_f64; bins]; bins];
    for (&x, &y) in xs.iter().zip(ys.iter()) {
        let (ax, ay) = (to_axis(x, log_x), to_axis(y, log_y));
        if !ax.is_finite() || !ay.is_finite() {
            continue;
        }
        let xi = (((ax - x_min) / x_span) * bins as f64) as usize;
        let yi = (((ay - y_min) / y_span) * bins as f64) as usize;
        z[yi.min(bins - 1)][xi.min(bins - 1)] += 1.0;
    }

    // bin-center coordinates for the axes, mapped back to RAW values: a plotly log axis is fed
    // raw coordinates and does the log transform itself, so geometric centers must be un-logged
    // here or the axis would log them twice.
    let x_centers: Vec<f64> = (0..bins)
        .map(|i| from_axis(x_min + (i as f64 + 0.5) * x_span / bins as f64, log_x))
        .collect();
    let y_centers: Vec<f64> = (0..bins)
        .map(|i| from_axis(y_min + (i as f64 + 0.5) * y_span / bins as f64, log_y))
        .collect();
    (x_centers, y_centers, z)
}

/// The share of all binned points that landed in the single busiest cell of a `bin_2d` grid — the
/// legibility measure behind `SMART_CONTOUR_MAX_BIN_SHARE` (issue #4223). A value near 1.0 means
/// the contour is one dark cell against an empty field: no distribution is visible, so the panel
/// is worthless no matter how the color scale is tuned. Returns 0.0 for an empty grid, which reads
/// as "not concentrated" and so never trips the gate on its own (an empty grid is caught earlier,
/// by the panel's own point-count guards).
fn density_concentration(z: &[Vec<f64>]) -> f64 {
    let mut total = 0.0_f64;
    let mut peak = 0.0_f64;
    for row in z {
        for &c in row {
            if c.is_finite() {
                total += c;
                peak = peak.max(c);
            }
        }
    }
    if total > 0.0 { peak / total } else { 0.0 }
}

/// Rows strictly positive on BOTH axes, for the log-space density retry (issue #4223). A log axis
/// cannot place a zero or a negative, so a zero-inflated pair (money columns are routinely 45-60%
/// zeros) can only be shown logarithmically by dropping that mass — which is honest ONLY if the
/// caller says so in the panel title, and only worth doing if enough points survive.
///
/// Returns the aligned positive rows plus the DROPPED share, or `None` when the survivors fall
/// below `SMART_CONTOUR_LOG_MIN_POINTS` / `SMART_CONTOUR_LOG_MIN_SHARE` — at which point the
/// caller drops the panel rather than describing a sliver as the distribution.
fn positive_subset_pair(xs: &[f64], ys: &[f64]) -> Option<(Vec<f64>, Vec<f64>, f64)> {
    let n = xs.len().min(ys.len());
    if n == 0 {
        return None;
    }
    let mut px: Vec<f64> = Vec::new();
    let mut py: Vec<f64> = Vec::new();
    for k in 0..n {
        let (x, y) = (xs[k], ys[k]);
        if x > 0.0 && y > 0.0 && x.is_finite() && y.is_finite() {
            px.push(x);
            py.push(y);
        }
    }
    #[allow(clippy::cast_precision_loss)]
    let kept_share = px.len() as f64 / n as f64;
    if px.len() < SMART_CONTOUR_LOG_MIN_POINTS || kept_share < SMART_CONTOUR_LOG_MIN_SHARE {
        return None;
    }
    Some((px, py, 1.0 - kept_share))
}

/// Build `viz smart`'s correlated-pair density panel, or decline (issue #4223), honoring the
/// resolved `--log-scale` mode:
///
/// - `Auto` (default): bin linearly and keep it only if LEGIBLE — a grid whose busiest cell holds
///   more than `SMART_CONTOUR_MAX_BIN_SHARE` of the points renders as a single dark square on an
///   empty field (the zero-inflated, heavily right-skewed money-column case). When it collapses,
///   retry the one honest logarithmic view: the rows positive on both axes, binned geometrically,
///   with the dropped share stated in the title.
/// - `Off`: never emit a log axis. Bin linearly; if that collapses there is no legible view left to
///   offer under the user's explicit no-log choice, so drop the panel rather than show an
///   unreadable single cell (the correlation heatmap above still reports the relationship).
/// - `On`: force the log view directly (skip the linear attempt), matching how the box/bar panels
///   treat `On` — a log axis can't place zeros, so this goes straight to the positive subset.
///
/// A too-small or still-concentrated positive subset yields no panel — an absent panel beats an
/// unreadable one. `name` is the caller's already-composed title (field pair + r, plus ρ when
/// nonlinear); `labels` is the same pair of field labels UNDECORATED, kept for the cell hover.
fn smart_contour_panel(
    xs: &[f64],
    ys: &[f64],
    name: &str,
    mode: LogScale,
    labels: (&str, &str),
) -> Option<Panel> {
    // `On` forces log without consulting the linear grid, just as `measure_by_dim_logs` / the box
    // paths force a log value axis under `On`.
    if mode == LogScale::On {
        return log_contour_panel(xs, ys, name, labels);
    }

    let (x, y, z) = bin_2d(xs, ys, SMART_CONTOUR_BINS, (false, false));
    if density_concentration(&z) <= SMART_CONTOUR_MAX_BIN_SHARE {
        return Some(Panel::new(
            name.to_string(),
            PanelKind::ContourPair {
                x,
                y,
                z,
                x_label: labels.0.to_string(),
                y_label: labels.1.to_string(),
            },
        ));
    }

    // the linear grid collapsed. Under `Off` the log retry is forbidden, so there is nothing
    // legible to draw; under `Auto` fall back to the log-space positive subset.
    if mode == LogScale::Off {
        return None;
    }
    log_contour_panel(xs, ys, name, labels)
}

/// The log-space density retry for `smart_contour_panel` (issue #4223): bin the strictly-positive
/// rows geometrically and keep the panel only if that view is itself legible. The title records the
/// omitted-zeros share so a subset is never shown dressed up as the whole; when NOTHING was dropped
/// (an all-positive pair reached here under `--log-scale on`) the title carries the plain log cue.
/// A non-zero omission that rounds below 1% is still disclosed (`<1%`) rather than silently hidden.
fn log_contour_panel(xs: &[f64], ys: &[f64], name: &str, labels: (&str, &str)) -> Option<Panel> {
    let (px, py, dropped) = positive_subset_pair(xs, ys)?;
    let (x, y, z) = bin_2d(&px, &py, SMART_CONTOUR_BINS, (true, true));
    if density_concentration(&z) > SMART_CONTOUR_MAX_BIN_SHARE {
        // even in log space the mass sits in one cell: there is no distribution to draw
        return None;
    }
    // Branch on the RAW dropped share, not its rounded percent: any omission at all must be
    // disclosed, so a handful of zeros in thousands of rows reads as `<1% of rows omitted`, never
    // the bare "log scale" cue reserved for a genuinely all-positive pair.
    let title = if dropped <= 0.0 {
        t!("viz.title.pair_log_scale", q_name = name).into_owned()
    } else {
        #[allow(clippy::cast_possible_truncation, clippy::cast_sign_loss)]
        let dropped_pct = (dropped * 100.0).round() as u32;
        let share = if dropped_pct == 0 {
            "<1%".to_string()
        } else {
            format!("{dropped_pct}%")
        };
        t!(
            "viz.title.pair_log_scale_positive",
            q_name = name,
            q_share = share
        )
        .into_owned()
    };
    Some(
        Panel::new(
            title,
            PanelKind::ContourPair {
                x,
                y,
                z,
                x_label: labels.0.to_string(),
                y_label: labels.1.to_string(),
            },
        )
        .with_axis_log((true, true)),
    )
}

fn contour_hover_template(x_label: &str, y_label: &str) -> String {
    t!(
        "viz.hover.contour",
        q_x = escape_template_pct(&escape_hover(x_label)),
        q_y = escape_template_pct(&escape_hover(y_label))
    )
    .into_owned()
}

fn lorenz_hover_template(label: &str, gini: f64) -> String {
    t!(
        "viz.hover.lorenz",
        q_label = escape_template_pct(&escape_hover(label)),
        q_gini = format!("{gini:.2}")
    )
    .into_owned()
}

/// Build a `Contour` trace: the 2D density of two numeric columns (--x and --y), binned into a
/// `--bins` x `--bins` grid of point counts. Unlike `heatmap` (categorical/correlation), this
/// shows the smooth joint distribution of two continuous variables.
fn build_contour(args: &Args) -> CliResult<(Box<dyn Trace>, String, String)> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let x_idx = resolve_one(args.flag_x.as_ref(), &headers, nh, "x")?;
    let y_idx = resolve_one(args.flag_y.as_ref(), &headers, nh, "y")?;

    let mut xs: Vec<f64> = Vec::new();
    let mut ys: Vec<f64> = Vec::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let (Some(x), Some(y)) = (parse_f64(record.get(x_idx)), parse_f64(record.get(y_idx)))
        else {
            continue;
        };
        xs.push(x);
        ys.push(y);
    }
    if xs.is_empty() {
        return fail_clierror!("No rows with numeric --x and --y values found for the contour.");
    }

    // grid resolution: --bins per axis (clamped to a sane range), else a default
    let bins = args.flag_bins.unwrap_or(20).clamp(2, 200);
    // `viz contour` is an explicit user request over user-named columns: bin linearly and honor it.
    let (x_centers, y_centers, z) = bin_2d(&xs, &ys, bins, (false, false));

    let (x_label, y_label) = (
        col_label(&headers, x_idx, nh),
        col_label(&headers, y_idx, nh),
    );
    let hover = contour_hover_template(&x_label, &y_label);
    let trace = Contour::new(x_centers, y_centers, z)
        .color_scale(ColorScale::Palette(ColorScalePalette::Viridis))
        .hover_template(&hover);
    Ok((trace, x_label, y_label))
}

/// Candlestick (or OHLC bar) chart from a date/x column and four numeric price columns.
fn build_candlestick(args: &Args, ohlc: bool) -> CliResult<(Box<dyn Trace>, String, String)> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let x_idx = resolve_one(args.flag_x.as_ref(), &headers, nh, "x")?;
    let o_idx = resolve_one(args.flag_ohlc_open.as_ref(), &headers, nh, "ohlc-open")?;
    let h_idx = resolve_one(args.flag_high.as_ref(), &headers, nh, "high")?;
    let l_idx = resolve_one(args.flag_low.as_ref(), &headers, nh, "low")?;
    let c_idx = resolve_one(args.flag_close.as_ref(), &headers, nh, "close")?;

    let (mut xs, mut open, mut high, mut low, mut close) =
        (Vec::new(), Vec::new(), Vec::new(), Vec::new(), Vec::new());
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let (Some(ov), Some(hv), Some(lv), Some(cv)) = (
            parse_f64(record.get(o_idx)),
            parse_f64(record.get(h_idx)),
            parse_f64(record.get(l_idx)),
            parse_f64(record.get(c_idx)),
        ) else {
            continue;
        };
        // x is passed through as a string; plotly auto-detects ISO dates client-side and
        // renders anything else as a category (no Rust-side date parsing).
        xs.push(cell_to_string(record.get(x_idx)));
        open.push(ov);
        high.push(hv);
        low.push(lv);
        close.push(cv);
    }
    if xs.is_empty() {
        return fail_clierror!(
            "No complete rows found (need numeric --ohlc-open/--high/--low/--close)."
        );
    }
    let x_label = col_label(&headers, x_idx, nh);
    // Plotly's default OHLC/candlestick hover label leads with the trace name; a custom
    // template gives a clean O/H/L/C readout. The x (date) is omitted on purpose — the
    // `x unified` hover mode (build_layout) already renders x as the tooltip header.
    // `hover_template_fallback` is defensive: financial traces have known gaps resolving
    // the per-point %{open|high|low|close} variables.
    let hover = t!("viz.hover.ohlc").into_owned();
    let trace: Box<dyn Trace> = if ohlc {
        Box::new(
            Ohlc::new(xs, open, high, low, close)
                .hover_template(hover)
                .hover_template_fallback("-"),
        )
    } else {
        Box::new(
            Candlestick::new(xs, open, high, low, close)
                .hover_template(hover)
                .hover_template_fallback("-"),
        )
    };
    Ok((trace, x_label, "price".to_string()))
}

/// Sankey flow diagram. Builds a unified node index across source & target labels and
/// aggregates duplicate source->target pairs into a single weighted link.
fn build_sankey(args: &Args) -> CliResult<(Box<dyn Trace>, Option<UpdateMenu>)> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let s_idx = resolve_one(args.flag_source.as_ref(), &headers, nh, "source")?;
    let t_idx = resolve_one(args.flag_target.as_ref(), &headers, nh, "target")?;
    let v_idx = match args.flag_value.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "value")?),
        None => None,
    };

    let mut node_labels: Vec<String> = Vec::new();
    let mut node_pos: HashMap<String, usize> = HashMap::new();
    let mut links: HashMap<(usize, usize), f64> = HashMap::new();
    let mut link_order: Vec<(usize, usize)> = Vec::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let s = cell_to_string(record.get(s_idx));
        let t = cell_to_string(record.get(t_idx));
        if s.is_empty() || t.is_empty() {
            continue;
        }
        // skip rows whose --value doesn't parse (matching pie/bar) rather than emitting
        // zero-weight ghost links/nodes
        let val = match v_idx {
            Some(i) => match parse_f64(record.get(i)) {
                Some(v) => v,
                None => continue,
            },
            None => 1.0,
        };
        let si = *node_pos.entry(s.clone()).or_insert_with(|| {
            node_labels.push(s);
            node_labels.len() - 1
        });
        let ti = *node_pos.entry(t.clone()).or_insert_with(|| {
            node_labels.push(t);
            node_labels.len() - 1
        });
        links
            .entry((si, ti))
            .and_modify(|w| *w += val)
            .or_insert_with(|| {
                link_order.push((si, ti));
                val
            });
    }
    if link_order.is_empty() {
        return fail_clierror!(
            "No flows found for the sankey diagram (need --source and --target)."
        );
    }

    let (mut sources, mut targets, mut values) = (Vec::new(), Vec::new(), Vec::new());
    for &(si, ti) in &link_order {
        sources.push(si);
        targets.push(ti);
        values.push(links[&(si, ti)]);
    }
    // snap (plotly's crossing-min layout) is the DEFAULT; `--sankey-value-order` opts into the
    // flow-ranked node ordering.
    let value_order = args.flag_sankey_value_order;
    let positions = sankey_value_positions(node_labels.len(), &sources, &targets, &values);
    let trace = sankey_trace(
        &node_labels,
        &sources,
        &targets,
        &values,
        value_order.then_some(()).and(positions.as_ref()),
    );
    // the on-screen order toggle is always offered (whichever order the chart opened in).
    let menu = positions
        .as_ref()
        .map(|(xs, ys)| sankey_order_toggle_menu(xs, ys, value_order));
    Ok((trace, menu))
}

/// Radar (polar) chart of numeric --cols. Each axis is min-max normalized to 0..1 across all
/// rows (axes typically differ in scale); --series produces one polygon per distinct value,
/// each the per-axis mean of its rows.
fn build_radar(args: &Args) -> CliResult<Vec<Box<dyn Trace>>> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let axis_idx = resolve_many(args.flag_cols.as_ref(), &headers, nh, "cols")?;
    if axis_idx.len() < 2 {
        return fail_incorrectusage_clierror!(
            "radar needs at least 2 columns via --cols (got {}).",
            axis_idx.len()
        );
    }
    let series_idx = match args.flag_series.as_ref() {
        Some(s) => Some(resolve_one(Some(s), &headers, nh, "series")?),
        None => None,
    };
    let axis_labels: Vec<String> = axis_idx
        .iter()
        .map(|&i| col_label(&headers, i, nh))
        .collect();

    let mut order: Vec<String> = Vec::new();
    let mut grouped: HashMap<String, Vec<Vec<f64>>> = HashMap::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let mut vals = Vec::with_capacity(axis_idx.len());
        let mut complete = true;
        for &i in &axis_idx {
            match parse_f64(record.get(i)) {
                Some(v) => vals.push(v),
                None => {
                    complete = false;
                    break;
                },
            }
        }
        if !complete {
            continue;
        }
        let series = match series_idx {
            Some(i) => cell_to_string(record.get(i)),
            None => String::new(),
        };
        grouped
            .entry(series.clone())
            .or_insert_with(|| {
                order.push(series);
                Vec::new()
            })
            .push(vals);
    }
    if order.is_empty() {
        return fail_clierror!("No numeric rows found for the radar chart.");
    }

    let n_axes = axis_idx.len();
    let mut mins = vec![f64::INFINITY; n_axes];
    let mut maxs = vec![f64::NEG_INFINITY; n_axes];
    for rows in grouped.values() {
        for row in rows {
            for a in 0..n_axes {
                mins[a] = mins[a].min(row[a]);
                maxs[a] = maxs[a].max(row[a]);
            }
        }
    }
    let normalize = |a: usize, v: f64| -> f64 {
        let (lo, hi) = (mins[a], maxs[a]);
        if (hi - lo).abs() < f64::EPSILON {
            0.5
        } else {
            (v - lo) / (hi - lo)
        }
    };

    // close the polygon by repeating the first axis at the end
    let mut theta_closed = axis_labels.clone();
    theta_closed.push(axis_labels[0].clone());

    let mut traces: Vec<Box<dyn Trace>> = Vec::with_capacity(order.len());
    for series in order {
        let rows = &grouped[&series];
        let mut means = vec![0.0; n_axes];
        for row in rows {
            for a in 0..n_axes {
                means[a] += row[a];
            }
        }
        for m in &mut means {
            *m /= rows.len() as f64;
        }
        let mut r: Vec<f64> = (0..n_axes).map(|a| normalize(a, means[a])).collect();
        r.push(r[0]);
        // Hover surfaces each axis with its ACTUAL mean (not the 0..1 plotted radius): the
        // normalized geometry stays comparable across differently-scaled axes while the tooltip
        // carries the real, comma-grouped value. Without this plotly shows only "trace 0".
        let mut hover: Vec<String> = (0..n_axes)
            .map(|a| {
                let line = format!(
                    "{}: {}",
                    escape_hover(&axis_labels[a]),
                    fmt_measure(means[a])
                );
                if series.is_empty() {
                    line
                } else {
                    format!("{}<br>{line}", escape_hover(&series))
                }
            })
            .collect();
        hover.push(hover[0].clone());
        // lines+markers so each axis intersection is its own hoverable vertex (the per-vertex
        // hover text below is keyed to those points), not just a single hover on the ring.
        let mut t = ScatterPolar::new(theta_closed.clone(), r)
            .mode(Mode::LinesMarkers)
            .fill(Fill::ToSelf)
            .hover_text_array(hover)
            .hover_info(HoverInfo::Text);
        if !series.is_empty() {
            t = t.name(escape_hover(&series));
        }
        traces.push(t);
    }
    Ok(traces)
}

/// Map a `--theme` name to a plotly built-in theme. Case-insensitive; hyphens are
/// accepted as separators. Returns `None` for an unrecognized name so callers can
/// surface a usage error.
fn parse_theme(name: &str) -> Option<BuiltinTheme> {
    match name.trim().to_ascii_lowercase().replace('-', "_").as_str() {
        "default" => Some(BuiltinTheme::Default),
        "plotly_white" | "white" => Some(BuiltinTheme::PlotlyWhite),
        "plotly_dark" | "dark" => Some(BuiltinTheme::PlotlyDark),
        "seaborn" => Some(BuiltinTheme::Seaborn),
        "seaborn_whitegrid" => Some(BuiltinTheme::SeabornWhitegrid),
        "seaborn_dark" => Some(BuiltinTheme::SeabornDark),
        "matplotlib" => Some(BuiltinTheme::Matplotlib),
        "plotnine" => Some(BuiltinTheme::Plotnine),
        _ => None,
    }
}

/// The comma-separated list of accepted `--theme` names, for usage errors.
const VALID_THEMES: &str = "default, plotly_white, plotly_dark, seaborn, seaborn_whitegrid, \
                            seaborn_dark, matplotlib, plotnine";

impl Args {
    /// The resolved plotly theme for this invocation. `flag_theme` is validated in
    /// `run()`, so an unrecognized name never reaches here (returns `None`).
    fn theme(&self) -> Option<BuiltinTheme> {
        self.flag_theme.as_deref().and_then(parse_theme)
    }
}

/// Apply the chosen built-in theme to a layout as a plotly template. When no theme
/// is set the layout is returned unchanged (qsv's built-in look). Explicit color
/// overrides at the call sites are gated on `theme.is_none()` so the template can
/// drive backgrounds, fonts and axis chrome.
fn apply_theme(layout: Layout, theme: Option<BuiltinTheme>) -> Layout {
    // Preserve user UI state (zoom/pan/selection/legend) across the `Plotly.newPlot`
    // re-renders qsv triggers for the fullscreen-modebar button and theme toggle.
    let layout = layout.ui_revision(VIZ_UIREVISION);
    match theme {
        Some(t) => layout.template(t.build()),
        None => layout,
    }
}

/// Whether the given theme renders on a dark page (so `geo` maps should use dark land/water fills).
fn is_dark_theme(theme: Option<BuiltinTheme>) -> bool {
    matches!(
        theme,
        Some(BuiltinTheme::PlotlyDark | BuiltinTheme::SeabornDark)
    )
}

/// The `(land, water, background)` colors for a `geo` subplot under the given theme. Light returns
/// qsv's built-in look (so unthemed/light renders are visually unchanged); dark returns the dark
/// fills so choropleth/geo maps are legible on a dark page. `background` is the subplot's
/// `geo.bgcolor` — it fills the sea on a choropleth (which paints no ocean) and the area outside
/// the projection (e.g. the corners around a non-rectangular globe).
fn geo_palette(theme: Option<BuiltinTheme>) -> (&'static str, &'static str, &'static str) {
    if is_dark_theme(theme) {
        (GEO_LAND_DARK, GEO_WATER_DARK, GEO_BG_DARK)
    } else {
        (GEO_LAND_LIGHT, GEO_WATER_LIGHT, PAPER_BG)
    }
}

/// The `(page background, text color, geo-meta caption color)` to use for the inline `viz smart`
/// HTML page chrome (the `<body>` around the panel grid), matching each built-in theme's
/// paper/font colors so a dark theme gets a dark page rather than dark plots floating on white.
/// Mirrors the values in plotly's `layout::themes` templates. `None` keeps qsv's built-in look.
/// The geo-meta color is the caption color for "Spatial extent:" shown below map panels. It must
/// contrast with the light-mode page background: light for dark-bg themes (`PlotlyDark`,
/// `SeabornDark`), dark for light-bg themes.
fn theme_page_chrome(theme: Option<BuiltinTheme>) -> (&'static str, &'static str, &'static str) {
    match theme {
        None => (PAPER_BG, INK, "#4b5563"),
        Some(BuiltinTheme::Default | BuiltinTheme::PlotlyWhite) => {
            ("#FFFFFF", "#2a3f5f", "#4b5563")
        },
        Some(BuiltinTheme::PlotlyDark) => ("#111111", "#f2f5fa", "#9aa4b2"),
        Some(BuiltinTheme::Seaborn) => ("#EAEAF2", "#333333", "#4b5563"),
        Some(BuiltinTheme::SeabornWhitegrid) => ("#FFFFFF", "#333333", "#4b5563"),
        Some(BuiltinTheme::SeabornDark) => ("#222222", "#eaeaf2", "#9aa4b2"),
        Some(BuiltinTheme::Matplotlib) => ("#FFFFFF", "black", "#4b5563"),
        Some(BuiltinTheme::Plotnine) => ("#EBEBEB", "#525252", "#4b5563"),
    }
}

/// The chart colors `(paper, font, grid, line, zero)` the toggle's runtime LIGHT palette applies
/// on load. For a light *non-default* theme (`seaborn`, `seaborn_whitegrid`, `matplotlib`,
/// `plotnine`) this mirrors that theme's own template colors, so the on-load `Plotly.relayout`
/// keeps the panel
/// backgrounds consistent with the themed page chrome (`theme_page_chrome`) instead of resetting
/// them to qsv's generic light look — which left e.g. a `#EAEAF2` seaborn page wrapping `#FFFFFF`
/// charts. Every other case (no theme, the white `default`/`plotly_white`, or a dark theme whose
/// LIGHT palette is only the toggle's light *complement*) keeps qsv's built-in light look, so
/// unthemed and dark-theme dashboards render byte-identically. Mirrors the fork's
/// `layout::themes` values. Trace colors come from the baked template (the relayout never touches
/// `colorway`/marker colors), so they are already theme-correct and out of scope here.
fn light_chart_palette(
    theme: Option<BuiltinTheme>,
) -> (
    &'static str,
    &'static str,
    &'static str,
    &'static str,
    &'static str,
) {
    match theme {
        Some(BuiltinTheme::Seaborn) => ("#EAEAF2", "#333333", "#D3D3D3", "#CCCCCC", "#D3D3D3"),
        Some(BuiltinTheme::SeabornWhitegrid) => {
            ("#FFFFFF", "#333333", "#E5E5E5", "#CCCCCC", "#E5E5E5")
        },
        Some(BuiltinTheme::Matplotlib) => ("#FFFFFF", "black", "#e5e5e5", "black", "#e5e5e5"),
        Some(BuiltinTheme::Plotnine) => ("#EBEBEB", "#525252", "#FFFFFF", "#CCCCCC", "#FFFFFF"),
        // No theme, the white default themes, and dark themes (whose light palette is just the
        // toggle's light complement) all keep qsv's built-in light look.
        _ => (PAPER_BG, INK, GRID_COLOR, AXIS_LINE, GRID_COLOR),
    }
}

/// The chart colors `(paper, font, grid, line, zero)` the toggle's runtime DARK palette applies.
/// Symmetric to `light_chart_palette`: a dark theme drives its own template colors so the dark
/// view (and the dark toggle target) honors that theme's actual shade — e.g. `seaborn_dark`'s
/// `#222222` rather than collapsing to the generic `#111111`. `plotly_dark`, the light themes,
/// and no theme keep the generic fixed-dark set (which equals `plotly_dark`'s own colors), so
/// those dashboards render byte-identically. Mirrors the fork's `layout::themes` values.
fn dark_chart_palette(
    theme: Option<BuiltinTheme>,
) -> (
    &'static str,
    &'static str,
    &'static str,
    &'static str,
    &'static str,
) {
    match theme {
        Some(BuiltinTheme::SeabornDark) => ("#222222", "#eaeaf2", "#444444", "#888888", "#444444"),
        // plotly_dark, the light themes, and no theme use the generic fixed-dark set.
        _ => ("#111111", "#f2f5fa", "#283442", "#506784", "#283442"),
    }
}

/// The CSS, button markup, and re-theming `<script>` for the `viz smart` light/dark toggle,
/// shared by both HTML render paths (the inline-div grid and the single typed-`Plot` grid).
/// Page chrome is driven by CSS variables (`--qsv-page-bg`/`--qsv-page-ink`/`--qsv-geo-meta`)
/// so flipping `body.qsv-dark` recolors the page instantly, while the script calls
/// `Plotly.relayout` on every live graph div to recolor the plots themselves. An explicit
/// `theme` is authoritative — it sets the initial mode and wins over any saved/OS preference
/// (so `--theme plotly_dark` always opens dark); only an unthemed Data Schematic (`theme == None`)
/// defers to the viewer's saved choice then OS preference. Known limitation: geo/scene/polar/pie
/// panels flip their background, font, and
/// container color, but basemap fills, map tiles, and trace marker colors do NOT re-theme
/// (that would need a trace-type-aware `Plotly.restyle`, which is out of scope here).
struct ToggleChrome {
    /// CSS for the `:root`/`body.qsv-dark` variables and the toggle button; goes in `<style>`.
    style:  String,
    /// The fixed-position toggle button; placed right after `<body>`.
    button: String,
    /// The toggle `<script>`; placed just before `</body>` (after all `Plotly.newPlot` calls).
    script: String,
}

fn toggle_chrome(theme: Option<BuiltinTheme>) -> ToggleChrome {
    let (light_bg, light_ink, light_geo_meta) = theme_page_chrome(theme);
    // Three-state initial mode. An EXPLICIT `--theme` is authoritative: it opens in that theme's
    // implied mode — dark for the dark built-ins, light for every other (light) built-in — and is
    // NOT overridden by a saved localStorage choice or a dark-mode OS (see `isDark()`). Only with
    // no `--theme` at all (`"system"`) do we defer to the saved choice, then
    // `prefers-color-scheme`.
    let default_mode = match theme {
        None => "system",
        Some(BuiltinTheme::PlotlyDark | BuiltinTheme::SeabornDark) => "dark",
        Some(_) => "light",
    };

    // The light/dark chart palettes the runtime toggle relayouts to. Each mirrors the selected
    // theme's own template colors for that mode (so the page chrome and the charts agree), else
    // qsv's built-in light look / the generic fixed-dark set. The dark page chrome reuses the dark
    // chart paper/font (geo-meta stays a fixed light gray, readable on any dark page).
    let (light_paper, light_font, light_grid, light_line, light_zero) = light_chart_palette(theme);
    let (dark_paper, dark_font, dark_grid, dark_line, dark_zero) = dark_chart_palette(theme);

    // Raw-string templates with token placeholders, so the brace-heavy JS needs no `{{`/`}}`
    // escaping and rustfmt's `format_strings` (regular-string-only) won't reflow/mangle them.
    // The link accent is keyed on the PAGE PAPER, not on the light/dark mode: the dark built-ins
    // (PlotlyDark/SeabornDark) put a dark page under `:root` too, so they need the light-on-dark
    // accent there as well. That is the same rule `--qsv-geo-meta` already follows.
    let (light_link, light_link_hover) = match theme {
        Some(BuiltinTheme::PlotlyDark | BuiltinTheme::SeabornDark) => {
            (LINK_ON_DARK, LINK_ON_DARK_HOVER)
        },
        _ => (LINK_ON_LIGHT, LINK_ON_LIGHT_HOVER),
    };
    let style = STYLE_TEMPLATE
        .replace("__LIGHT_BG__", light_bg)
        .replace("__LIGHT_INK__", light_ink)
        .replace("__LIGHT_GEO_META__", light_geo_meta)
        .replace("__LIGHT_LINK__", light_link)
        .replace("__LIGHT_LINK_HOVER__", light_link_hover)
        .replace("__DARK_BG__", dark_paper)
        .replace("__DARK_INK__", dark_font)
        .replace("__DARK_GEO_META__", "#9aa4b2")
        .replace("__DARK_LINK__", LINK_ON_DARK)
        .replace("__DARK_LINK_HOVER__", LINK_ON_DARK_HOVER)
        .replace("__FONT_FAMILY__", FONT_FAMILY);

    let button = format!(
        "<button id=\"qsv-theme-toggle\" type=\"button\" aria-label=\"{aria}\">{label}</button>",
        aria = t!("viz.chrome.theme_toggle_aria"),
        label = t!("viz.chrome.theme_toggle"),
    );

    // `buildUpdate` only sets keys present on a given graph's layout, so axis-less plots (pie) and
    // arbitrary subplot counts both work without knowing div ids.
    let script = SCRIPT_TEMPLATE
        .replace("__DEFAULT_MODE__", default_mode)
        .replace("__LIGHT_PAPER__", light_paper)
        .replace("__LIGHT_FONT__", light_font)
        .replace("__LIGHT_GRID__", light_grid)
        .replace("__LIGHT_LINE__", light_line)
        .replace("__LIGHT_ZERO__", light_zero)
        .replace("__DARK_PAPER__", dark_paper)
        .replace("__DARK_FONT__", dark_font)
        .replace("__DARK_GRID__", dark_grid)
        .replace("__DARK_LINE__", dark_line)
        .replace("__DARK_ZERO__", dark_zero)
        .replace("__GEO_LAND_LIGHT__", GEO_LAND_LIGHT)
        .replace("__GEO_WATER_LIGHT__", GEO_WATER_LIGHT)
        .replace("__GEO_LAND_DARK__", GEO_LAND_DARK)
        .replace("__GEO_WATER_DARK__", GEO_WATER_DARK);

    ToggleChrome {
        style,
        button,
        script,
    }
}

/// The fixed-position qsv/datHere logo (bottom-right) for `viz smart` HTML pages, linking to
/// the qsv site. Two theme variants are embedded as base64 PNG data URIs (so the page stays
/// self-contained) and CSS-swapped via the `body.qsv-dark` class the toggle already manages: the
/// dark-text logo shows in light mode, the light-text (dark-background) logo in dark mode. The
/// accompanying CSS lives inline in `smart_html_page`'s `<style>`.
fn logo_markup() -> String {
    use base64_simd::STANDARD as BASE64;
    let light = BASE64.encode_to_string(include_bytes!("assets/qsv_logo_light.png"));
    let dark = BASE64.encode_to_string(include_bytes!("assets/qsv_logo_dark.png"));
    // One key drives the aria-label and both `alt`s -- they are the same sentence, and
    // letting them drift would give a screen reader two different names for one link.
    let brand = t!("viz.chrome.logo_alt");
    format!(
        r#"<a id="qsv-logo" href="https://qsv.dathere.com/" target="_blank" rel="noopener" aria-label="{brand}"><img class="qsv-logo-light" alt="{brand}" src="data:image/png;base64,{light}" /><img class="qsv-logo-dark" alt="{brand}" src="data:image/png;base64,{dark}" /></a>"#
    )
}

/// Canonical location of the full third-party license texts, referenced from every generated page.
const THIRD_PARTY_NOTICES_URL: &str =
    "https://github.com/dathere/qsv/blob/master/THIRD_PARTY_NOTICES.md";

/// Machine-readable attribution for the third-party software embedded in a generated page, emitted
/// immediately after `<!doctype html>`.
///
/// plotly.js (MIT) always ships — embedded as a gzip+base64 payload, or referenced from the
/// version-pinned CDN URL — and it bundles MapLibre GL JS (BSD-3-Clause), so both are named
/// unconditionally. The vendored DataTables bundle (MIT) is only present when the data viewer
/// drawer is rendered, and OpenStreetMap/CARTO basemap TILES are only fetched when a MapLibre
/// panel is actually drawn; both clauses are therefore conditional, so the notice always describes
/// what is genuinely in the file.
///
/// This matters because the embedded plotly.js and DataTables payloads are gzipped: their own
/// copyright banners survive inside the payload but are not human-readable in the delivered
/// artifact. This comment (and `third_party_footer`) restore that visibility.
fn third_party_comment(datatables: bool, basemap: bool) -> String {
    let datatables_line = if datatables {
        "\n     DataTables 3.0.1 + Buttons/ColumnControl/DateTime/SearchBuilder\n       (c) \
         SpryMedia Ltd - MIT"
    } else {
        ""
    };
    let basemap_line = if basemap {
        "\n     Basemap tiles (c) OpenStreetMap contributors (ODbL 1.0), (c) CARTO"
    } else {
        ""
    };
    format!(
        "<!-- Third-party software used by this page:\n     plotly.js {PLOTLY_CDN_VERSION} - (c) \
         2012-2026 Plotly, Inc. - MIT\n       (bundles MapLibre GL JS - \
         BSD-3-Clause){datatables_line}{basemap_line}\n     Full texts: {THIRD_PARTY_NOTICES_URL} \
         -->"
    )
}

/// The human-visible credits line, rendered as a page footer in the normal document flow.
///
/// Deliberately does NOT credit the basemap tiles: MapLibre draws its own always-visible
/// attribution control ("(c) CARTO, (c) OpenStreetMap contributors") in the corner of every tile
/// panel, so the ODbL/CARTO obligation is already discharged *inside the map*, where it belongs.
/// Repeating it down in the footer was pure duplication. The machine-readable
/// `third_party_comment` still names the tile sources unconditionally.
fn third_party_footer(datatables: bool) -> String {
    // Only the label words are translatable; the anchors stay in Rust so no markup (and no
    // escaping question) enters the locale files. "(MIT)" is a licence identifier, not prose.
    let mut parts = vec![format!(
        r#"{} <a href="https://plotly.com/javascript/" target="_blank" rel="noopener">plotly.js</a> (MIT)"#,
        t!("viz.chrome.credits_charts")
    )];
    if datatables {
        parts.push(format!(
            r#"{} <a href="https://datatables.net/" target="_blank" rel="noopener">DataTables</a> (MIT)"#,
            t!("viz.chrome.credits_table")
        ));
    }
    format!(
        r#"<footer id="qsv-credits">{} &middot; <a href="{THIRD_PARTY_NOTICES_URL}" target="_blank" rel="noopener">{}</a></footer>"#,
        parts.join(" &middot; "),
        t!("viz.chrome.credits_notices")
    )
}

/// Styling for `third_party_footer`.
///
/// Every color is `var(--qsv-*, <fallback>)`. Inside a `viz smart` Data Schematic those variables
/// are defined by `STYLE_TEMPLATE` and re-bound under `body.qsv-dark`, so the footer re-themes with
/// the rest of the page for free — in both directions, and through the runtime Theme toggle.
///
/// The fallbacks are LIGHT-only, deliberately. They apply on exactly one page: the standalone
/// single-chart output, which reuses plotly's own template — and that template gives `<body>` no
/// background rule at all, so its paper is always the browser default (white). Even `--theme dark`
/// only darkens the *plot's* `paper_bgcolor`, never the page. An earlier revision keyed the
/// fallbacks on `@media (prefers-color-scheme: dark)`, which rendered `#6cb6ff` links and
/// `#97a2b0` text on that white paper for anyone whose OS is in dark mode — the exact contrast
/// failure `LINK_ON_LIGHT`/`LINK_ON_DARK` were introduced to fix. Media queries report the
/// viewer's preference, not the paper actually under the text, so there is nothing to key on here.
///
/// Padding is deliberately tight. An earlier revision reserved 58px below the last line to clear
/// a fixed `#qsv-logo` / `#qsv-theme-toggle` stack pinned to the bottom-right; both now sit in
/// this same footer row (see `.qsv-page-foot`), so that reserved band was dead space on every
/// page — and on the single-chart page, which never had the fixed stack, it always was.
fn third_party_footer_style() -> String {
    format!(
        "  #qsv-credits {{ font-size: 11px; line-height: 1.6; text-align: center; padding: 10px \
         16px 12px; color: var(--qsv-geo-meta, #5b6673); }}\n  #qsv-credits a, #qsv-credits \
         a:visited {{ color: var(--qsv-link, {LINK_ON_LIGHT}); text-decoration: none; }}\n  \
         #qsv-credits a:hover, #qsv-credits a:focus-visible {{ color: var(--qsv-link-hover, \
         {LINK_ON_LIGHT_HOVER}); text-decoration: underline; }}"
    )
}

/// Splice the attribution comment + credits footer into an already-rendered document, for the
/// single-chart path (which reuses plotly's own HTML template rather than `smart_html_page`).
///
/// Idempotent: a document that already carries the footer is returned untouched, so re-running
/// this over generated output can never double up the notice.
fn inject_third_party_notice(doc: &str, datatables: bool, basemap: bool) -> String {
    if doc.contains(r#"<footer id="qsv-credits">"#) {
        return doc.to_string();
    }
    let mut out = doc.to_string();
    if let Some(pos) = out.to_ascii_lowercase().find("<!doctype html>") {
        let at = pos + "<!doctype html>".len();
        out.insert_str(
            at,
            &format!("\n{}", third_party_comment(datatables, basemap)),
        );
    }
    if let Some(pos) = out.rfind("</body>") {
        let block = format!(
            "<style>\n{}\n</style>\n{}\n",
            third_party_footer_style(),
            third_party_footer(datatables)
        );
        out.insert_str(pos, &block);
    }
    out
}

/// Accent for actionable page-chrome links (the "Data Dictionary" and "(Explore)"/"(Preview)"
/// links) on LIGHT page paper. Left at the UA default these render `#0000EE`/`#551A8B`, which is
/// a contrast failure on dark paper and clashes with every built-in theme on light paper.
/// AA-contrasting against the whole light range (`#FFFFFF` … `#EAEAF2`).
const LINK_ON_LIGHT: &str = "#0a5fb4";
const LINK_ON_LIGHT_HOVER: &str = "#084b8f";
/// The same accent for DARK page paper (`#111111`/`#222222`, and the runtime `body.qsv-dark`).
const LINK_ON_DARK: &str = "#6cb6ff";
const LINK_ON_DARK_HOVER: &str = "#9ecbff";

/// Decorative glyph suffixes that give the two chrome links a visual cue. Both are hand-drawn
/// (no third-party icon asset, so the generated dashboards carry no attribution obligation),
/// stroked in `currentColor` so they re-theme with the link they sit in, and `aria-hidden` +
/// `focusable="false"` so assistive tech announces just the link text.
///
/// An open book — the "Data Dictionary" link.
const DICT_LINK_ICON: &str = r#"<svg class="qsv-link-icon" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.25" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true" focusable="false"><path d="M8 4.3C6.9 3.3 5.4 2.8 3.6 2.8c-.6 0-1.1.05-1.5.15v9.6c.4-.1.9-.15 1.5-.15 1.8 0 3.3.5 4.4 1.5 1.1-1 2.6-1.5 4.4-1.5.6 0 1.1.05 1.5.15v-9.6c-.4-.1-.9-.15-1.5-.15-1.8 0-3.3.5-4.4 1.5Z"/><path d="M8 4.3v9.6"/></svg>"#;
/// A spreadsheet grid — the "(Explore)"/"(Preview)" data-viewer link.
const DATA_LINK_ICON: &str = r#"<svg class="qsv-link-icon" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.25" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true" focusable="false"><rect x="1.7" y="2.6" width="12.6" height="10.8" rx="1.4"/><path d="M1.7 6.2h12.6M1.7 9.8h12.6M6.2 6.2v7.2M10.2 6.2v7.2"/></svg>"#;

/// CSS variables + toggle-button rule for `toggle_chrome`. Token placeholders are substituted at
/// runtime; kept as a raw string so the literal CSS braces stay intact.
const STYLE_TEMPLATE: &str = r"  :root { --qsv-page-bg: __LIGHT_BG__; --qsv-page-ink: __LIGHT_INK__; --qsv-geo-meta: __LIGHT_GEO_META__; --qsv-link: __LIGHT_LINK__; --qsv-link-hover: __LIGHT_LINK_HOVER__; }
  body.qsv-dark { --qsv-page-bg: __DARK_BG__; --qsv-page-ink: __DARK_INK__; --qsv-geo-meta: __DARK_GEO_META__; --qsv-link: __DARK_LINK__; --qsv-link-hover: __DARK_LINK_HOVER__; }
  /* In the normal document flow, inside the footer row (.qsv-page-foot) beside the credits and
     the logo. It used to be pinned bottom-right, which cost a 58px reserved band under the
     credits plus per-drawer offset rules; riding in the footer needs neither, and it can no
     longer collide with the Plotly modebar (top-right) or the geo extent menus (top-left). */
  #qsv-theme-toggle { font: 13px __FONT_FAMILY__; padding: 6px 12px; border-radius: 6px; border: 1px solid var(--qsv-page-ink); background: var(--qsv-page-bg); color: var(--qsv-page-ink); cursor: pointer; opacity: 0.85; }
  #qsv-theme-toggle:hover { opacity: 1; }";

/// The light/dark toggle `<script>` for `toggle_chrome`. Token placeholders are substituted at
/// runtime; kept as a raw string so the JS braces and regex backslashes need no escaping.
const SCRIPT_TEMPLATE: &str = r#"<script>
(function () {
  var themeDefaultMode = "__DEFAULT_MODE__";
  var DARK = { paper: "__DARK_PAPER__", plot: "__DARK_PAPER__", font: "__DARK_FONT__", grid: "__DARK_GRID__", line: "__DARK_LINE__", zero: "__DARK_ZERO__", bg: "__DARK_PAPER__", land: "__GEO_LAND_DARK__", water: "__GEO_WATER_DARK__", mapStyle: "carto-darkmatter", mapStyleNoLabels: "carto-darkmatter-nolabels" };
  var LIGHT = { paper: "__LIGHT_PAPER__", plot: "__LIGHT_PAPER__", font: "__LIGHT_FONT__", grid: "__LIGHT_GRID__", line: "__LIGHT_LINE__", zero: "__LIGHT_ZERO__", bg: "__LIGHT_PAPER__", land: "__GEO_LAND_LIGHT__", water: "__GEO_WATER_LIGHT__", mapStyle: "carto-positron", mapStyleNoLabels: "carto-positron-nolabels" };
  function isDark() {
    // An explicit --theme is authoritative: it wins over a stale cross-Data Schematic
    // saved preference (the localStorage key is shared across all qsv viz pages,
    // so a "light" toggle on one Data Schematic must not override --theme plotly_dark
    // on another).
    if (themeDefaultMode === "dark") return true;
    if (themeDefaultMode === "light") return false;
    // No --theme: defer to the viewer's saved choice, then OS preference.
    try {
      var saved = localStorage.getItem("qsv-viz-theme");
      if (saved === "dark") return true;
      if (saved === "light") return false;
    } catch (e) {}
    return window.matchMedia && window.matchMedia("(prefers-color-scheme: dark)").matches;
  }
  function buildUpdate(gd, p) {
    var u = { "paper_bgcolor": p.paper, "font.color": p.font };
    var lay = gd.layout || {};
    var hasAxis = false;
    Object.keys(lay).forEach(function (k) {
      if (/^xaxis\d*$/.test(k) || /^yaxis\d*$/.test(k)) {
        hasAxis = true;
        u[k + ".gridcolor"] = p.grid;
        u[k + ".linecolor"] = p.line;
        u[k + ".zerolinecolor"] = p.zero;
        u[k + ".tickcolor"] = p.line;
      }
      if (/^geo\d*$/.test(k)) {
        u[k + ".bgcolor"] = p.bg;
        u[k + ".landcolor"] = p.land;
        u[k + ".oceancolor"] = p.water;
        u[k + ".lakecolor"] = p.water;
      } else if (/^map\d*$/.test(k)) {
        // preserve a panel's labeled-vs-nolabels choice: density/choropleth overlays use the
        // label-free basemap, point maps keep labels. Swap only the theme, not the labeling.
        var curStyle = lay[k] && lay[k].style;
        var noLabels = typeof curStyle === "string" && /-nolabels$/.test(curStyle);
        u[k + ".style"] = noLabels ? p.mapStyleNoLabels : p.mapStyle;
      } else if (/^polar\d*$/.test(k) || /^scene\d*$/.test(k)) {
        u[k + ".bgcolor"] = p.bg;
      }
    });
    if (hasAxis) u["plot_bgcolor"] = p.plot;
    return u;
  }
  function hasMapLibre(gd) {
    // Any MapLibre `map`-subplot trace (choroplethmap/densitymap/scattermap) throws on a
    // relayout/react in our pinned plotly fork, so these must take the Plotly.newPlot path below.
    return (gd.data || []).some(function (t) {
      return t.type === "choroplethmap" || t.type === "densitymap" || t.type === "scattermap";
    });
  }
  // Apply a flat dotted-key update (buildUpdate's shape) onto a nested layout, in place — so the
  // same update can feed Plotly.newPlot (which wants nested layout) instead of relayout.
  function setPath(obj, path, val) {
    var parts = path.split("."), o = obj;
    for (var i = 0; i < parts.length - 1; i++) {
      if (o[parts[i]] == null || typeof o[parts[i]] !== "object") o[parts[i]] = {};
      o = o[parts[i]];
    }
    o[parts[parts.length - 1]] = val;
  }
  function apply(dark) {
    document.body.classList.toggle("qsv-dark", dark);
    var p = dark ? DARK : LIGHT;
    document.querySelectorAll(".js-plotly-plot").forEach(function (gd) {
      var u = buildUpdate(gd, p);
      try {
        if (hasMapLibre(gd)) {
          // A MapLibre map trace (choroplethmap/densitymap/scattermap) throws "setData of
          // undefined" on a relayout/react in our pinned plotly fork, so fold the theme update
          // (incl. the basemap `.style`) into a full Plotly.newPlot — the only primitive that
          // safely restyles it without blanking the layer. Preserve the injected fullscreen
          // modebar button via the div's existing config.
          Object.keys(u).forEach(function (k) { setPath(gd.layout, k, u[k]); });
          // Re-theme any "Basemap labels" toggle: its baked `map.style` args point at the
          // render-time theme's basemap, so rewrite each to the target theme's variant, preserving
          // the arg's own `-nolabels` suffix (labeled arg -> mapStyle, no-labels arg ->
          // mapStyleNoLabels). Guarded on the `map.style` key so non-map menus are untouched.
          (gd.layout.updatemenus || []).forEach(function (menu) {
            (menu.buttons || []).forEach(function (b) {
              ["args", "args2"].forEach(function (ak) {
                var a = b[ak];
                if (a && a[0] && typeof a[0]["map.style"] === "string") {
                  a[0]["map.style"] = /-nolabels$/.test(a[0]["map.style"]) ? p.mapStyleNoLabels : p.mapStyle;
                }
              });
            });
          });
          // preserve the page-scroll-friendly scrollZoom state across a theme flip (a MapLibre
          // theme change re-runs newPlot, which would otherwise reset scrollZoom to its map default).
          var cfg = { responsive: true, scrollZoom: document.fullscreenElement === gd };
          var add = gd._context && gd._context.modeBarButtonsToAdd;
          if (add && add.length) cfg.modeBarButtonsToAdd = add;
          var np = Plotly.newPlot(gd, gd.data, gd.layout, cfg);
          // plotly re-enables the MapLibre GL wheel-zoom handler on this newPlot, so re-assert the
          // inline/fullscreen scrollZoom state via the fullscreen script's published helper.
          if (window.__qsvRefitScrollZoom) window.__qsvRefitScrollZoom(gd);
          // that newPlot also builds a NEW MapLibre instance, dropping the cluster label repaint
          // and hover handlers with the old one — re-install them on the new map.
          if (window.__qsvRefitClusterUi) window.__qsvRefitClusterUi(gd);
          // this newPlot re-render silently drops gd.on listeners — rebind the Data Dictionary
          // annotation hook (when that feature is on the page) once the re-render settles,
          // fulfilled OR rejected: the listeners are already gone either way.
          if (gd.__qsvDictHooked && window.__qsvDictRehook && np && np.then) {
            var rehook = function () { gd.__qsvDictHooked = false; window.__qsvDictRehook(); };
            np.then(rehook, rehook);
          }
          // same for the --photos hover-dwell lightbox (see PHOTO_CHROME): its plotly_hover
          // listener went with the old plot, so rebind it once the re-render settles.
          if (gd.__qsvPhotoHooked && window.__qsvPhotoRehook && np && np.then) {
            var prehook = function () { gd.__qsvPhotoHooked = false; window.__qsvPhotoRehook.tries = 0; window.__qsvPhotoRehook(); };
            np.then(prehook, prehook);
          }
          // and for the map<->rows selection bridge (see MAP_SELECT_CHROME): this newPlot drops
          // its plotly_click listener AND repaints the points unselected, so the rehook also
          // re-applies the current selection.
          if (gd.__qsvSelHooked && window.__qsvSelRehook && np && np.then) {
            var srehook = function () { gd.__qsvSelHooked = false; window.__qsvSelRehook.tries = 0; window.__qsvSelRehook(); };
            np.then(srehook, srehook);
          }
        } else {
          Plotly.relayout(gd, u);
        }
      } catch (e) {}
    });
  }
  function init() {
    var dark = isDark();
    apply(dark);
    var btn = document.getElementById("qsv-theme-toggle");
    if (btn) btn.addEventListener("click", function () {
      var nowDark = !document.body.classList.contains("qsv-dark");
      try { localStorage.setItem("qsv-viz-theme", nowDark ? "dark" : "light"); } catch (e) {}
      apply(nowDark);
    });
    // on a compressed page, panels render only after the async plotly.js/figure inflation —
    // AFTER the apply(dark) above ran (it only reaches already-rendered .js-plotly-plot divs).
    // Re-apply on the bootstrap's ready event, debounced since every completed panel fires it.
    var reT;
    document.addEventListener("qsv:plotly-ready", function () {
      clearTimeout(reT);
      reT = setTimeout(function () { apply(document.body.classList.contains("qsv-dark")); }, 60);
    });
  }
  if (document.readyState === "loading")
    document.addEventListener("DOMContentLoaded", function () { setTimeout(init, 0); });
  else setTimeout(init, 0);
})();
</script>"#;

/// `<style>` body for the fullscreen modebar button. A single `:fullscreen` rule gives the
/// fullscreened graph div an opaque, theme-aware background so the desktop doesn't bleed through.
/// `var(--qsv-page-bg, #ffffff)` resolves to the page background on `viz smart` pages (which
/// define the var) and falls back to white on the plain single-chart document (no qsv CSS vars).
const FULLSCREEN_STYLE: &str =
    "  .js-plotly-plot:fullscreen { background: var(--qsv-page-bg, #ffffff); }";

/// A self-contained `<script>` that adds a custom "Fullscreen" button to every plot's modebar
/// and auto-fits map zoom to the rendered container size.
///
/// The plotly-rs `Configuration` we use can't carry a JS `click` handler, and a custom modebar
/// button can only be added to an already-rendered plot reliably by re-running
/// `Plotly.newPlot(gd, gd.data, gd.layout, cfg)` with `modeBarButtonsToAdd` (`Plotly.react`
/// historically ignores `config` changes, and a DOM-appended `.modebar-btn` gets wiped whenever
/// plotly rebuilds the modebar — e.g. on the theme toggle's `relayout`). So we re-render every
/// graph div once with the button added. The button toggles the native Fullscreen API on the
/// graph div.
///
/// Maps (MapLibre) bake an ABSOLUTE `zoom` computed in Rust for a fixed assumed pixel
/// size (`window.__qsvMapAssumedW`/`__qsvMapAssumedH`, published by a per-page prelude). Since
/// MapLibre zoom is logarithmic, the optimal zoom for any real container size is
/// `bakedZoom + log2(min(curW/assumedW, curH/assumedH))` with the baked center unchanged
/// (`fitTarget`, using each subplot's DOMAIN-scaled px, so it is correct whether the map is
/// full-bleed or one subplot in a grid). The baked zoom/center are captured ONCE before any
/// mutation (`gd.__qsvBaked`) so repeated re-fits never drift.
///
/// We must NEVER apply a map fit via `Plotly.relayout`/`react`: in our pinned plotly-rs fork those
/// throw "Cannot read properties of undefined (reading 'setData')" on a MapLibre `choroplethmap`
/// and blank the layer (the live regression this replaces). So the fit is applied two ways:
///   - INITIAL display: `applyFitLayout` mutates `gd.layout` BEFORE the button-injection
///     `Plotly.newPlot`, so the first paint already fills the real container (newPlot is the only
///     primitive that safely sets a map camera at render time — no extra paint, no relayout).
///   - FULLSCREEN enter/exit: `applyFitCamera` moves the GL map camera directly via the MapLibre GL
///     instance (`gd._fullLayout[k]._subplot.map.jumpTo`), which sits below plotly's
///     relayout/setData path — no tile reload, works even before the style loads.
///
/// Non-map charts just get `Plotly.Plots.resize`.
///
/// plotly's own render runs in a deferred `<script type="module">` with a top-level
/// `await Plotly.newPlot(...)`, which `DOMContentLoaded` does NOT wait for — so we can't assume the
/// plots are rendered when we first run. We therefore POLL: each `.plotly-graph-div` is enhanced
/// only once its `gd.data` is populated (marked with a `__qsvFs` flag so it's done exactly once),
/// retrying briefly until every div is ready (capped so a render error can't loop forever). Raw
/// string (like `SCRIPT_TEMPLATE`) so the brace-heavy JS needs no escaping.
/// Localize a `const` JS block's `__QSVI18N_*__` placeholders.
///
/// Placeholders sit in BARE (unquoted) JS positions: `js_string_literal` supplies the
/// surrounding quotes along with the escaping, so a translation containing an apostrophe
/// (fr "l'ensemble"), a quote, or a `<` cannot break out of its string literal or smuggle a
/// `</script>` into the page. Substitution is UNCONDITIONAL -- English takes exactly the same
/// path as every other locale, so there is no English-only branch that could drift.
fn fullscreen_script() -> String {
    FULLSCREEN_SCRIPT
        .replace(
            "__QSVI18N_FULLSCREEN__",
            &js_string_literal(&t!("viz.chrome.toggle_fullscreen")),
        )
        .replace(
            "__QSVI18N_LEGEND__",
            &js_string_literal(&t!("viz.chrome.toggle_legend")),
        )
        // Singular/plural are two explicit keys because the JS already branches on `n === 1`;
        // that deliberately does NOT generalize to languages with more than two plural forms,
        // which would need the count formatted in Rust or an Intl.PluralRules rewrite.
        .replace(
            "__QSVI18N_POINT_ONE__",
            &js_string_literal(&t!("viz.map.cluster_point_one")),
        )
        .replace(
            "__QSVI18N_POINT_MANY__",
            &js_string_literal(&t!("viz.map.cluster_point_many")),
        )
        .replace(
            "__QSVI18N_CLICK_ZOOM__",
            &js_string_literal(&t!("viz.map.cluster_click_to_zoom")),
        )
}

const FULLSCREEN_SCRIPT: &str = r#"<script>
(function () {
  var icon = { width: 512, height: 512, path: "M512 512v-208l-80 80-96-96-48 48 96 96-80 80z M512 0h-208l80 80-96 96 48 48 96-96 80 80z M0 512h208l-80-80 96-96-48-48-96 96-80-80z M0 0v208l80-80 96 96 48-48-96-96 80-80z" };
  var button = {
    name: "qsv-fullscreen",
    title: __QSVI18N_FULLSCREEN__,
    icon: icon,
    click: function (gd) {
      try {
        // requestFullscreen/exitFullscreen return a promise that rejects without transient user
        // activation (or when a Permissions-Policy blocks fullscreen); swallow it so it doesn't
        // surface as an unhandled rejection.
        var p = document.fullscreenElement ? document.exitFullscreen() : gd.requestFullscreen();
        if (p && p.catch) p.catch(function () {});
      } catch (e) {}
    }
  };
  // A "Toggle legend" button: rows of a color swatch + a label bar so it reads as a legend.
  var legendIcon = { width: 512, height: 512, path: "M0 96h96v64H0z M160 112h352v32H160z M0 224h96v64H0z M160 240h352v32H160z M0 352h96v64H0z M160 368h352v32H160z" };
  var legendButton = {
    name: "qsv-legend",
    title: __QSVI18N_LEGEND__,
    icon: legendIcon,
    click: function (gd) {
      try {
        // no-op on maps: Plotly.relayout throws on a MapLibre choroplethmap in our pinned
        // plotly fork, and a choropleth's "legend" is a colorbar (not governed by showlegend).
        if (mapKeys(gd).length) return;
        // relayout is the lightweight toggle: it preserves config-added modebar buttons
        // (gd._context.modeBarButtonsToAdd) and the plotly_buttonclicked handler that a
        // newPlot re-render would drop.
        // Read the EFFECTIVE state from _fullLayout (plotly's computed default is showlegend:true
        // for multi-trace charts that never set it), not gd.layout — else the first click on a
        // default-visible legend is a visual no-op.
        var fl = gd._fullLayout || gd.layout || {};
        var p = Plotly.relayout(gd, { showlegend: !fl.showlegend });
        if (p && p.catch) p.catch(function () {});
      } catch (e) {}
    }
  };
  // The assumed pixel size the baked MapLibre zoom was computed against (published by a
  // per-page prelude). Fallback to the standalone HTML default (1000x600) if absent.
  function assumedDims() {
    var w = (typeof window.__qsvMapAssumedW === "number" && window.__qsvMapAssumedW > 0) ? window.__qsvMapAssumedW : 1000;
    var h = (typeof window.__qsvMapAssumedH === "number" && window.__qsvMapAssumedH > 0) ? window.__qsvMapAssumedH : 600;
    return { w: w, h: h };
  }
  function mapKeys(gd) {
    var lay = (gd && gd.layout) || {};
    return Object.keys(lay).filter(function (k) {
      return /^map\d*$/.test(k) && lay[k] && typeof lay[k].zoom === "number";
    });
  }
  // Snapshot the baked (assumed-px) zoom + center ONCE, before any mutation, as the fixed
  // reference every later re-fit recomputes from (so repeated toggles can't drift).
  function captureBaked(gd) {
    if (gd.__qsvBaked) return;
    gd.__qsvBaked = {};
    var lay = gd.layout || {};
    mapKeys(gd).forEach(function (k) { gd.__qsvBaked[k] = { z: lay[k].zoom, c: lay[k].center }; });
  }
  function plotPx(gd) {
    var fl = gd._fullLayout || {};
    return { w: fl.width || gd.clientWidth || 0, h: fl.height || gd.clientHeight || 0 };
  }
  // Compute the optimal {zoom, center} for map subplot `k` at the current plot pixel size.
  // MapLibre zoom is logarithmic, so to keep the SAME bounds framed when the container
  // differs from the assumed px the baked zoom was computed for:
  //   zoom = bakedZoom + log2(min(curW/assumedW, curH/assumedH)); center = baked center.
  // curW/curH are DOMAIN-scaled (the subplot's own px), so this is correct for a full-bleed map AND
  // a map that is one subplot in a combined grid. Always recomputed from the fixed baked reference,
  // so repeated re-fits never drift.
  function fitTarget(gd, k, plotW, plotH) {
    var baked = gd.__qsvBaked && gd.__qsvBaked[k];
    if (!baked || typeof baked.z !== "number" || !(plotW > 0) || !(plotH > 0)) return null;
    var a = assumedDims(), lay = gd.layout || {};
    var dom = (lay[k] && lay[k].domain) || {};
    var dx = (dom.x && dom.x.length === 2) ? (dom.x[1] - dom.x[0]) : 1;
    var dy = (dom.y && dom.y.length === 2) ? (dom.y[1] - dom.y[0]) : 1;
    var ratio = Math.min((dx * plotW) / a.w, (dy * plotH) / a.h);
    if (!isFinite(ratio) || ratio <= 0) return null;
    return { zoom: baked.z + Math.log2(ratio), center: baked.c };
  }
  // Pre-render fit: mutate gd.layout so the NEXT Plotly.newPlot renders at the fitted zoom. Used for
  // the initial-display fit, folded into the button-injection newPlot (no extra paint). Crucially
  // NO Plotly.relayout/react — those throw on a MapLibre choroplethmap in our pinned plotly fork
  // ("setData of undefined") and blank the layer; only newPlot safely sets a map camera at render.
  function applyFitLayout(gd, plotW, plotH) {
    if (!gd || !gd.__qsvBaked) return;
    var lay = gd.layout || {};
    Object.keys(gd.__qsvBaked).forEach(function (k) {
      var t = fitTarget(gd, k, plotW, plotH);
      if (!t || !lay[k]) return;
      lay[k].zoom = t.zoom;
      if (t.center) lay[k].center = t.center;
    });
  }
  // Post-render fit: move the GL map camera directly via the MapLibre GL instance
  // (gd._fullLayout[k]._subplot.map). This sits BELOW plotly's relayout/react/setData path, so it
  // re-aims a choroplethmap's camera without the "setData of undefined" throw, needs no tile
  // reload, and works even before the GL style finishes loading. Used for fullscreen enter/exit.
  function applyFitCamera(gd, plotW, plotH) {
    if (!gd || !gd.__qsvBaked) return;
    var fl = gd._fullLayout || {}, lay = gd.layout || {};
    Object.keys(gd.__qsvBaked).forEach(function (k) {
      var t = fitTarget(gd, k, plotW, plotH);
      if (!t) return;
      var sp = fl[k] && fl[k]._subplot, m = sp && sp.map;
      if (!m || typeof m.jumpTo !== "function") return;
      var dest = { zoom: t.zoom };
      if (t.center && typeof t.center.lon === "number" && typeof t.center.lat === "number")
        dest.center = [t.center.lon, t.center.lat];
      try { m.jumpTo(dest); } catch (e) {}
      // keep plotly's stored layout coherent with the camera (plain property set, NOT relayout).
      if (lay[k]) { lay[k].zoom = t.zoom; if (t.center) lay[k].center = t.center; }
    });
  }
  // The GL map camera may not be attached for a frame or two after a resize; retry briefly
  // (bounded) until every captured map subplot has its `_subplot.map`, then aim the camera.
  function fitCameraNow(gd, tries) {
    if (tries === undefined) tries = 20;
    var fl = gd._fullLayout || {};
    var ready = Object.keys(gd.__qsvBaked || {}).every(function (k) {
      var sp = fl[k] && fl[k]._subplot; return sp && sp.map;
    });
    if (!ready && tries > 0) { setTimeout(function () { fitCameraNow(gd, tries - 1); }, 100); return; }
    var px = plotPx(gd);
    applyFitCamera(gd, px.w, px.h);
  }
  // plotly turns scrollZoom ON by default for geo/map/gl3d subplots, and for a MapLibre `map`
  // subplot it does NOT honor a `scrollZoom:false` render config at the GL-handler level (the layer
  // stays wheel-zoomable), so we disable the GL handler DIRECTLY after each render. Off inline → a
  // wheel/two-finger scroll over the map scrolls the PAGE; on in fullscreen → wheel zooms the map
  // (no page to scroll). Drag-pan is left untouched. Also set _context so geo/3D subplots (which
  // expose no post-render wheel handle) follow via plotly's own live read; toggling here needs no
  // re-render, so the map camera is preserved across a fullscreen flip.
  function setScrollZoom(gd, on) {
    try { if (gd._context) gd._context.scrollZoom = on; } catch (e) {}
    var fl = gd._fullLayout || {};
    mapKeys(gd).forEach(function (k) {
      var sp = fl[k] && fl[k]._subplot, m = sp && sp.map;
      if (m && m.scrollZoom && typeof m.scrollZoom.enable === "function") {
        try { if (on) m.scrollZoom.enable(); else m.scrollZoom.disable(); } catch (e) {}
      }
    });
  }
  // The MapLibre GL map attaches a frame or two after a render, so wait (bounded) until every map
  // subplot has its `_subplot.map`, then apply the scrollZoom state for the current fullscreen mode.
  // Published as a global so the theme-toggle script (a separate IIFE that re-runs newPlot on a
  // MapLibre panel, which re-enables the GL handler) can re-assert the state after its own render.
  function applyScrollZoom(gd, tries) {
    if (tries === undefined) tries = 20;
    var fl = gd._fullLayout || {};
    var ready = mapKeys(gd).every(function (k) { var sp = fl[k] && fl[k]._subplot; return sp && sp.map; });
    if (!ready && tries > 0) { setTimeout(function () { applyScrollZoom(gd, tries - 1); }, 100); return; }
    setScrollZoom(gd, document.fullscreenElement === gd);
  }
  window.__qsvRefitScrollZoom = applyScrollZoom;
  // ---- cluster bubbles: legible counts + hover ------------------------------------------------
  // Two plotly gaps to close. (1) It builds the cluster COUNT layer with an EMPTY paint
  // ({type:"symbol", paint:{}, layout:{"text-field":"{point_count_abbreviated}"}}), so MapLibre's
  // default text-color (#000000) applies and no trace attribute exposes it. (2) `cluster.color`
  // defaults to the trace's MARKER color — an identity color doing a magnitude job — so every
  // bubble reads the same regardless of size, and black digits on it can land anywhere from
  // 2.5:1 (white on the orange marker) to unreadable.
  //
  // So the bubbles are repainted on the GL instance as a proper SEQUENTIAL ramp keyed to
  // point_count (light = few, dark = many), with the label ink picked per step by measured WCAG
  // contrast — every pairing below clears 4.5:1 for the 12px count, worst case 6.59:1 light /
  // 7.86:1 dark. The ramps are single-hue (blue), lightness-monotone, and each validates against
  // its own basemap; the dark-basemap ramp is deliberately paler, since a step dark enough to
  // vanish into a dark map fails the mark-vs-surface floor. A surface-colored ring keeps a bubble
  // legible over a busy patch of basemap (park, water, motorway) where fill-vs-map contrast alone
  // would not hold.
  // NOTE: hex colors are single-quoted on purpose — a double-quote immediately followed by a hash
  // would close this script's Rust raw-string delimiter.
  var CLUSTER_INK = '#0b0b0b', CLUSTER_WHITE = '#ffffff';
  var CLUSTER_STEPS = [10, 100, 1000];
  var CLUSTER_REPAINT_MS = 500;
  var CLUSTER_SCHEME = {
    light: {
      fills: ['#86b6ef', '#5598e7', '#1c5cab', '#0d366b'],
      texts: [CLUSTER_INK, CLUSTER_INK, CLUSTER_WHITE, CLUSTER_WHITE],
      ring:  '#fafafa'
    },
    dark: {
      fills: ['#cde2fb', '#9ec5f4', '#6da7ec', '#184f95'],
      texts: [CLUSTER_INK, CLUSTER_INK, CLUSTER_INK, CLUSTER_WHITE],
      ring:  '#121212'
    }
  };
  // a MapLibre "step" expression over the cluster's point_count, so the ramp is evaluated per
  // bubble by the GL layer itself (no per-feature bookkeeping on our side).
  function clusterStepExpr(values) {
    var e = ["step", ["get", "point_count"], values[0]];
    for (var i = 0; i < CLUSTER_STEPS.length; i++) e.push(CLUSTER_STEPS[i], values[i + 1]);
    return e;
  }
  // Layers are (re)created by plotly whenever `cluster.enabled` flips, so ids are looked up by
  // suffix each time rather than cached. "-cluster" never matches "-cluster-count" (different
  // trailing 8 chars), so the circle and count layers stay distinct.
  function clusterLayerIds(map, suffix) {
    var style;
    try { style = map.getStyle(); } catch (e) { return []; }
    var layers = (style && style.layers) || [];
    return layers.filter(function (l) {
      return l && typeof l.id === "string" && l.id.slice(-suffix.length) === suffix;
    }).map(function (l) { return l.id; });
  }
  function paintClusters(map, isDark) {
    var scheme = isDark ? CLUSTER_SCHEME.dark : CLUSTER_SCHEME.light;
    clusterLayerIds(map, "-cluster").forEach(function (id) {
      try {
        map.setPaintProperty(id, "circle-color", clusterStepExpr(scheme.fills));
        map.setPaintProperty(id, "circle-stroke-color", scheme.ring);
        map.setPaintProperty(id, "circle-stroke-width", 1.5);
      } catch (e) {}
    });
    clusterLayerIds(map, "-cluster-count").forEach(function (id) {
      try {
        map.setPaintProperty(id, "text-color", clusterStepExpr(scheme.texts));
        // MapLibre drops a symbol that collides with another label, and plotly sets no overlap
        // policy — so a count silently disappeared whenever its bubble sat under a basemap place
        // label, leaving an unlabelled blob (the number is the entire point of the bubble). Pin it:
        // always draw the count, and don't let it suppress the basemap's own labels either.
        map.setLayoutProperty(id, "text-allow-overlap", true);
        map.setLayoutProperty(id, "text-ignore-placement", true);
      } catch (e) {}
    });
  }
  // WHEN to paint is the hard part: plotly creates the cluster layers only when `cluster.enabled`
  // flips on, and there is no event that reliably lands after that. `styledata` and `plotly_restyle`
  // both fire well BEFORE the layers exist (on a heavy Data Schematic the gap outran a bounded retry, and
  // the ramp silently didn't take), and MapLibre's `idle` never fires at all under plotly's usage
  // (measured: 0 firings on a 28-panel Data Schematic, even after a pan). So don't guess — poll a cheap
  // predicate and repaint whenever the layers are found unpainted. Our ramp is a step EXPRESSION (an
  // array) while plotly's default is a plain color string, so "needs paint" is a type check, and the
  // scan short-circuits to false on the common case of a map with no cluster layers at all.
  function clustersNeedPaint(map) {
    var ids = clusterLayerIds(map, "-cluster");
    if (!ids.length) return false;
    var v;
    try { v = map.getPaintProperty(ids[0], "circle-color"); } catch (e) { return false; }
    return !Array.isArray(v);
  }
  // plotly gives cluster bubbles NO hover: it filters clustered features out of the hover-bearing
  // circle layer (["!", ["has", "point_count"]]) and never registers the -cluster layers with its
  // hover system. So supply our own tooltip — a plain positioned div in the map container (the
  // maplibregl namespace is bundled but not published as a global, so Popup isn't reachable).
  function clusterTip(map) {
    if (map.__qsvClusterTip) return map.__qsvClusterTip;
    var el = document.createElement("div");
    el.style.cssText = "position:absolute;z-index:5;display:none;pointer-events:none;" +
      "padding:3px 7px;border-radius:4px;white-space:nowrap;" +
      "font:12px/1.35 Helvetica Neue, Helvetica, Arial, sans-serif;" +
      "color:#fff;background:rgba(0,0,0,.78);box-shadow:0 1px 3px rgba(0,0,0,.3)";
    try { map.getContainer().appendChild(el); } catch (e) { return null; }
    map.__qsvClusterTip = el;
    return el;
  }
  function installClusterUi(map, isDark) {
    // keyed on the GL instance: a theme flip / fullscreen re-render builds a NEW map, which
    // re-installs (picking up that render's basemap mode), while repeated calls against the same
    // instance are no-ops.
    if (map.__qsvClusterUi) return;
    map.__qsvClusterUi = true;
    paintClusters(map, isDark);
    // the cluster layers only exist while clustering is ON, so re-assert on every render settle
    // (see clustersNeedPaint) rather than trying to catch the moment they are added.
    // Repaint on a slow tick rather than on an event (see clustersNeedPaint): this covers every path
    // that rebuilds the layers — the "Clusters/Points" toggle, toggling it back off and on, the
    // extent buttons — with one mechanism. The tick clears itself once plotly discards this map
    // (a theme flip builds a new one, which gets its own install and its own tick), so no interval
    // outlives its map. Painting is a no-op unless the layers are actually found unpainted.
    var tick = setInterval(function () {
      var alive = true;
      try { if (!map.getStyle()) alive = false; } catch (e) { alive = false; }
      if (!alive) { clearInterval(tick); return; }
      if (clustersNeedPaint(map)) paintClusters(map, isDark);
    }, CLUSTER_REPAINT_MS);
    var tip = clusterTip(map);
    if (!tip) return;
    function hide() {
      tip.style.display = "none";
      try { map.getCanvas().style.cursor = ""; } catch (e) {}
    }
    function clusterAt(pt) {
      var ids = clusterLayerIds(map, "-cluster");
      if (!ids.length) return null;
      var hits;
      try { hits = map.queryRenderedFeatures(pt, { layers: ids }); } catch (e) { return null; }
      return (hits && hits[0]) || null;
    }
    map.on("mousemove", function (e) {
      var f = clusterAt(e.point);
      var n = f && f.properties && f.properties.point_count;
      if (typeof n !== "number") { hide(); return; }
      tip.textContent = n.toLocaleString() + (n === 1 ? __QSVI18N_POINT_ONE__ : __QSVI18N_POINT_MANY__) + __QSVI18N_CLICK_ZOOM__;
      tip.style.display = "block";
      tip.style.left = (e.point.x + 12) + "px";
      tip.style.top = (e.point.y + 12) + "px";
      try { map.getCanvas().style.cursor = "pointer"; } catch (err) {}
    });
    map.on("mouseout", hide);
    map.on("click", function (e) {
      var f = clusterAt(e.point);
      if (!f || !f.properties || typeof f.properties.cluster_id !== "number") return;
      var src;
      try { src = map.getSource(f.source); } catch (err) { return; }
      if (!src || typeof src.getClusterExpansionZoom !== "function") return;
      var center = f.geometry && f.geometry.coordinates;
      var fly = function (z) {
        if (typeof z !== "number" || !center) return;
        hide();
        try { map.easeTo({ center: center, zoom: z }); } catch (err) {}
      };
      // maplibre-gl returns a Promise in 3+ and takes a node-style callback in 2.x — support both.
      try {
        var p = src.getClusterExpansionZoom(f.properties.cluster_id, function (err, z) {
          if (!err) fly(z);
        });
        if (p && p.then) p.then(fly, function () {});
      } catch (err) {}
    });
  }
  // Same bounded wait as applyScrollZoom: the GL map attaches a frame or two after the render.
  function applyClusterUi(gd, tries) {
    if (tries === undefined) tries = 20;
    var fl = gd._fullLayout || {};
    var keys = mapKeys(gd);
    if (!keys.length) return;
    var ready = keys.every(function (k) { var sp = fl[k] && fl[k]._subplot; return sp && sp.map; });
    if (!ready && tries > 0) { setTimeout(function () { applyClusterUi(gd, tries - 1); }, 100); return; }
    keys.forEach(function (k) {
      var sp = fl[k] && fl[k]._subplot, m = sp && sp.map;
      // read the basemap mode from the style this render actually used, so a runtime theme flip
      // (which re-runs newPlot with the dark/light basemap) re-installs with the matching ramp.
      var style = (gd.layout && gd.layout[k] && gd.layout[k].style) || (fl[k] && fl[k].style) || "";
      if (m) installClusterUi(m, /dark/i.test(String(style)));
    });
  }
  window.__qsvRefitClusterUi = applyClusterUi;
  // gl3d (3D scene) panels need a DIFFERENT fix than geo/map. plotly's WebGL canvas swallows the
  // wheel (preventDefault) on EVERY wheel event — even when the scene is created with
  // scrollZoom:false, which suppresses the zoom but NOT the preventDefault. So a scroll over a 3D
  // panel neither zooms NOR scrolls the page (the wheel is eaten). Unlike geo (honors scrollZoom
  // live) and MapLibre (setScrollZoom toggles the GL handler), the robust fix is a capture-phase
  // wheel interceptor: inline we stopPropagation so the canvas handler never runs (never
  // preventDefaults) and the wheel scrolls the PAGE; in fullscreen we let it through. The listener
  // sits on gd (which persists across theme-toggle re-renders) so it survives a panel re-render.
  // Drag-rotate uses pointer events, so it is untouched.
  function sceneKeys(gd) {
    var fl = gd._fullLayout || {};
    return Object.keys(fl).filter(function (k) { return /^scene\d*$/.test(k); });
  }
  function installGl3dScrollFix(gd) {
    if (gd.__qsvGl3dFix || !sceneKeys(gd).length) return;
    gd.__qsvGl3dFix = true;
    gd.addEventListener("wheel", function (e) {
      if (document.fullscreenElement !== gd) e.stopPropagation();
    }, { capture: true, passive: true });
  }
  function enhance(gd) {
    gd.__qsvFs = true;
    try {
      // animation frames live on gd._transitionData, NOT gd.data/gd.layout, so the button-injection
      // newPlot below would drop them (killing the slider/Play animation). Capture them first and
      // re-add via Plotly.addFrames once newPlot resolves. .filter(Boolean) skips plotly's leading
      // hole; panels without frames leave this null (no-op).
      var qsvFrames = (gd._transitionData && gd._transitionData._frames)
        ? gd._transitionData._frames.filter(Boolean) : null;
      captureBaked(gd);
      // initial-display fit: bake the fitted zoom into the layout the button-injection newPlot
      // renders, so the first paint already fills the real container (no relayout, no reflow).
      var px = plotPx(gd);
      applyFitLayout(gd, px.w, px.h);
      // scrollZoom:false so a wheel/two-finger scroll over a geo/3D panel scrolls the PAGE instead
      // of zooming the subplot (plotly turns scrollZoom ON by default for geo/map/gl3d). This
      // newPlot governs the final render — it overrides the Rust-side Configuration — so scrollZoom
      // is set HERE. A MapLibre `map` panel ignores it at the GL level, so applyScrollZoom() below
      // disables that handler directly once the map attaches; `fullscreenchange` re-enables it.
      var qsvNp = Plotly.newPlot(gd, gd.data, gd.layout, { responsive: true, scrollZoom: document.fullscreenElement === gd, modeBarButtonsToAdd: [button, legendButton] });
      if (qsvFrames && qsvFrames.length) {
        if (qsvNp && qsvNp.then) qsvNp.then(function () { Plotly.addFrames(gd, qsvFrames); });
        else Plotly.addFrames(gd, qsvFrames);
      }
      // disable the MapLibre GL wheel-zoom handler inline (see applyScrollZoom): plotly leaves it on
      // regardless of the render config, so a scroll over the map would otherwise pan/zoom it and
      // swallow the page scroll.
      applyScrollZoom(gd);
      // cluster bubbles: repaint plotly's black-by-default count labels and add hover/click-to-zoom
      // (see installClusterUi). No-op for non-map panels and for maps that never cluster.
      applyClusterUi(gd);
      // gl3d panels: stop the WebGL canvas from eating the wheel inline so the page can scroll
      // (scrollZoom:false suppresses the zoom but not the canvas preventDefault; see above).
      installGl3dScrollFix(gd);
      // Core/Full extent buttons (viz smart outlier maps) relayout the map to a zoom baked for the
      // ASSUMED px — so they ignore the real/fullscreen/resized container. Adopt the clicked
      // extent's baked zoom/center as the new fit reference, then re-aim the camera for the CURRENT
      // size (the button's own relayout fired the assumed-px zoom; this corrects it dimension-aware).
      if (gd.on) gd.on("plotly_buttonclicked", function (ed) {
        try {
          var args = ed && ed.button && ed.button.args && ed.button.args[0];
          if (!args || !gd.__qsvBaked) return;
          var hit = false;
          Object.keys(gd.__qsvBaked).forEach(function (k) {
            var z = args[k + ".zoom"];
            if (typeof z !== "number") return;
            var c = args[k + ".center"];
            gd.__qsvBaked[k] = { z: z, c: (c !== undefined ? c : gd.__qsvBaked[k].c) };
            hit = true;
          });
          // defer so plotly's button relayout settles first, else it would overwrite the camera.
          if (hit) setTimeout(function () { fitCameraNow(gd); }, 0);
        } catch (e) {}
      });
      gd.addEventListener("fullscreenchange", function () {
        // Plotly.Plots.resize is async; aim the camera only AFTER it resolves so we read the
        // post-resize _fullLayout dims (else enter/exit would refit from stale dims and stick).
        // Same timing for scrollZoom: enable wheel-zoom entering fullscreen (no page to scroll),
        // disable on exit — applied post-resize so the map subplot's GL handle is attached.
        var fs = document.fullscreenElement === gd;
        var done = function () { fitCameraNow(gd); setScrollZoom(gd, fs); };
        var rp;
        try { rp = Plotly.Plots.resize(gd); } catch (e) {}
        if (rp && rp.then) rp.then(done).catch(done);
        else done();
      });
    } catch (e) {}
  }
  var tries = 0;
  function init() {
    if (typeof Plotly === "undefined") { if (tries++ < 100) setTimeout(init, 50); return; }
    var pending = false;
    document.querySelectorAll(".plotly-graph-div").forEach(function (gd) {
      if (gd.__qsvFs) return;
      if (gd.data) enhance(gd);
      else pending = true;
    });
    if (pending && tries++ < 100) setTimeout(init, 50);
  }
  if (document.readyState === "loading")
    document.addEventListener("DOMContentLoaded", function () { setTimeout(init, 0); });
  else setTimeout(init, 0);
})();
</script>"#;

/// A tiny prelude `<script>` publishing the assumed map pixel size the baked MapLibre
/// `zoom` was computed against, so `FULLSCREEN_SCRIPT`'s `qsvRefitMaps` can rescale the zoom to the
/// real rendered size (initial display + fullscreen). Sourced from the Rust constants so it can't
/// drift: standalone `viz map` HTML frames against `fit_dims` (`DEFAULT_IMG_WIDTH`×
/// `DEFAULT_IMG_HEIGHT`); `viz smart` panels against `MAP_PANEL_ASSUMED_WIDTH_PX`×
/// `MAP_PANEL_USABLE_HEIGHT_PX`.
fn assumed_map_dims_prelude(width_px: f64, height_px: f64) -> String {
    format!(
        "<script>window.__qsvMapAssumedW={width_px};window.__qsvMapAssumedH={height_px};</script>"
    )
}

/// Re-wrap `viz smart` KPI tile labels against their MEASURED pixel width, on every resize.
///
/// `wrap_kpi_label` wraps against a character budget at generation time, but the tile's pixel width
/// is not known then and is not even fixed afterwards: the KPI row spans anywhere from ~580px
/// (narrow window, or the `--dict-info` drawer open) to ~1400px on the same page, so no character
/// count is right at both ends — too few over-wraps a wide row, too many lets neighbouring labels
/// collide. This measures the tile, re-wraps, and `relayout`s only the labels that actually change.
///
/// Uses a `ResizeObserver` on the plot div rather than `window.onresize` because the common trigger
/// is the dictionary drawer opening, which shrinks the plot WITHOUT a window resize event. Falls
/// back to `window.onresize` where `ResizeObserver` is unavailable.
///
/// Runs after plotly's own `Plotly.newPlot` script tags but polls anyway (capped), since plotly
/// renders from a deferred module script — the same reason `FULLSCREEN_SCRIPT` retries.
const KPI_REWRAP_SCRIPT: &str = r#"<script>
(function () {
  var MARK = "qsv-kpi-label";
  // A single reused canvas: text measurement must match the annotation's own font, and creating a
  // canvas per call would churn.
  function measurer(font) {
    var c = measurer._c || (measurer._c = document.createElement("canvas"));
    var ctx = c.getContext("2d");
    ctx.font = font;
    return function (s) { return ctx.measureText(s).width; };
  }
  // Greedy word wrap, mirroring wrap_kpi_label: a single word wider than the tile is left intact
  // (never hard-split) and simply overflows.
  function wrap(text, widthPx, measure) {
    var words = text.split(/\s+/).filter(Boolean), lines = [], cur = "";
    for (var i = 0; i < words.length; i++) {
      var next = cur ? cur + " " + words[i] : words[i];
      if (cur && measure(next) > widthPx) { lines.push(cur); cur = words[i]; } else { cur = next; }
    }
    if (cur) lines.push(cur);
    return lines.join("<br>");
  }
  function rewrap(gd) {
    var anns = (gd.layout && gd.layout.annotations) || [], idx = [];
    for (var i = 0; i < anns.length; i++) { if (anns[i] && anns[i].name === MARK) idx.push(i); }
    if (!idx.length) return;
    // Annotation x is in PAPER coords, which span the plot area inside the margins -- so measure
    // against that same box, not the div's full width.
    var fl = gd._fullLayout, plotW = 0;
    if (fl && fl.width && fl.margin) { plotW = fl.width - fl.margin.l - fl.margin.r; }
    if (!plotW) { plotW = gd.clientWidth || gd.getBoundingClientRect().width; }
    if (!plotW) return;
    // Consecutive KPI labels sit at evenly spaced tile centers, so the gap between the first two IS
    // the tile width in paper units. A lone tile gets the whole row.
    var pitch = idx.length > 1 ? Math.abs(anns[idx[1]].x - anns[idx[0]].x) : 1;
    var avail = Math.max(40, pitch * plotW * 0.94);
    var upd = {}, changed = false;
    for (var k = 0; k < idx.length; k++) {
      var a = anns[idx[k]];
      var full = String(a.text || "").replace(/<br\s*\/?>/gi, " ").replace(/\s+/g, " ").trim();
      if (!full) continue;
      var f = a.font || {};
      var font = ((f.size || 13) + "px ") + (f.family || getComputedStyle(document.body).fontFamily || "sans-serif");
      var next = wrap(full, avail, measurer(font));
      if (next !== a.text) { upd["annotations[" + idx[k] + "].text"] = next; changed = true; }
    }
    // The `next !== a.text` guard also stops the relayout from re-triggering the observer forever.
    if (changed && window.Plotly && Plotly.relayout) { Plotly.relayout(gd, upd); }
  }
  function debounce(fn) {
    var t = null;
    return function () { clearTimeout(t); t = setTimeout(fn, 120); };
  }
  function attach() {
    var divs = document.querySelectorAll(".js-plotly-plot"), found = false;
    for (var i = 0; i < divs.length; i++) {
      var gd = divs[i];
      if (gd.__qsvKpiRw) { found = true; continue; }
      if (!gd.layout) { continue; }
      gd.__qsvKpiRw = true; found = true;
      var run = (function (el) { return debounce(function () { try { rewrap(el); } catch (e) {} }); })(gd);
      try { new ResizeObserver(run).observe(gd); } catch (e) { window.addEventListener("resize", run); }
      run();
    }
    return found;
  }
  var tries = 0;
  (function poll() { if (attach() || ++tries > 40) { return; } setTimeout(poll, 150); })();
})();
</script>"#;

/// How long the animated parcats category-order flip runs. Matched to the Sankey node-order
/// toggle's 500 ms (plotly's `traces/sankey/constants.js`) rather than the 300 ms of plotly's own
/// parcats drag transition: 300 ms is right for a drag, where the pointer has already told you
/// what moved, but reads as a flicker on a button press you may not even be looking at. The easing
/// stays cubic-in-out, as in `parcats.js` `dragEnd`.
const PARCATS_ORDER_DURATION_MS: u32 = 500;

/// Animates the "category order" flip on every parcats chart (standalone `viz parcats` and the
/// `viz smart` flow panel), giving it the moving transition the Sankey node-order toggle gets for
/// free.
///
/// WHY THIS IS NEEDED AT ALL: plotly's sankey renderer re-applies `transition().duration(500)` to
/// its node/link selections on EVERY replot, so a plain `Plotly.restyle` from an updatemenu button
/// animates. The parcats renderer does not — its module entry (`traces/parcats/plot.js`) accepts
/// plotly's `transitionOpts` and never forwards it, and the only `d3.transition` in `parcats.js`
/// lives in the drag handler. A restyled parcats therefore always snaps. There is no trace or
/// layout attribute that changes this, so the animation has to be driven here.
///
/// HOW: a capture-phase listener sees the click before plotly's own (which is bound to the button
/// element itself, further down the propagation path). Once it has confirmed the click IS a parcats
/// order toggle and that it can service it, it claims the event with `stopPropagation` — plotly's
/// handler never runs — and then
///   1. snapshots the geometry plotly animates on a drag — `g.dimension` / `g.category` `transform`
///      and `path.path` `d`,
///   2. runs the button's own restyle payload (picked by reading the trace's live `categoryorder`,
///      so no toggle state is stored anywhere and a theme flip or `Plotly.newPlot` can't desync
///      it),
///   3. rewinds each element to its snapshot value SYNCHRONOUSLY inside the restyle's `then` —
///      before the browser can paint the new state, which would flash — and tweens to the value
///      plotly just computed.
///
/// Claiming the click rather than disabling the button (`execute: false`) is what keeps this a pure
/// ENHANCEMENT: every bail-out below lets the event travel on to plotly, which applies the very
/// same restyle un-animated. That matters because the figure JSON can travel WITHOUT this script —
/// anything that lifts the bare `Plotly.newPlot` object out of an emitted page — and a button that
/// had delegated its restyle would be dead there. (`examples/viz/gen_gallery.py` does exactly that
/// lift, and re-splices this block via `PARCATS_ORDER_MARKER` so the gallery animates too, but the
/// button must not DEPEND on a consumer doing so.)
///
/// Interpolating attribute strings (rather than recomputing plotly's layout math) is exactly what
/// d3 — and so plotly's own drag transition — does. `tween` only interpolates when the two strings
/// differ solely in their numbers; anything else snaps, so a structural change can never emit a
/// malformed `d`. Bands and labels snap, matching plotly's drag transition, which also leaves them
/// out.
///
/// The listener is a single delegated capture-phase handler on `document`: it covers every panel
/// of a Data Schematic at once and, unlike a `gd.on(...)` plotly listener, survives the fullscreen
/// chrome's `Plotly.newPlot` re-render without any rehook. Charts with no parcats trace never
/// match, so the script is inert on every other page.
fn parcats_order_script() -> String {
    format!(
        "{PARCATS_ORDER_MARKER}\n{}",
        PARCATS_ORDER_SCRIPT.replace("__DURATION__", &PARCATS_ORDER_DURATION_MS.to_string())
    )
}

/// Marks the start of `PARCATS_ORDER_SCRIPT` in emitted HTML. `examples/viz/gen_gallery.py` lifts
/// the block from `<marker>` through `</script>` out of a generated page and re-splices it into the
/// gallery scaffold, so the gallery's reassembled parcats figure animates too; the marker is what
/// makes both the lift and the strip-and-re-add idempotent across regens.
const PARCATS_ORDER_MARKER: &str = "<!--qsv-parcats-order-->";

const PARCATS_ORDER_SCRIPT: &str = r#"<script>
(function () {
  var DURATION = __DURATION__;
  // one shared numeric-token pattern, recompiled per use so no /g lastIndex can leak between calls
  var NUM = "[-+]?(?:\\d*\\.\\d+|\\d+)(?:[eE][-+]?\\d+)?";
  function rx() { return new RegExp(NUM, "g"); }
  // interpolator over two attribute strings that differ ONLY in their numbers; null when the
  // non-numeric skeletons differ (different command count/order), so the caller snaps instead.
  function tween(from, to) {
    var a = from.match(rx()), b = to.match(rx());
    if (!a || !b || a.length !== b.length) { return null; }
    if (from.replace(rx(), "") !== to.replace(rx(), "")) { return null; }
    var lits = from.split(rx()), f = a.map(Number), t = b.map(Number);
    return function (u) {
      var s = lits[0];
      for (var i = 0; i < f.length; i++) { s += (f[i] + (t[i] - f[i]) * u) + lits[i + 1]; }
      return s;
    };
  }
  // cubic-in-out, d3's "cubic-in-out" — the ease plotly's own parcats drag transition uses
  function ease(u) { return u < 0.5 ? 4 * u * u * u : 1 - Math.pow(-2 * u + 2, 3) / 2; }
  var GEOMETRY = [
    ["g.trace.parcats g.dimension", "transform"],
    ["g.trace.parcats g.category", "transform"],
    ["g.trace.parcats path.path", "d"]
  ];
  function snapshot(gd) {
    var out = [];
    for (var g = 0; g < GEOMETRY.length; g++) {
      var els = gd.querySelectorAll(GEOMETRY[g][0]);
      for (var i = 0; i < els.length; i++) {
        out.push([els[i], GEOMETRY[g][1], els[i].getAttribute(GEOMETRY[g][1])]);
      }
    }
    return out;
  }
  // rewind to `before` and tween back to whatever plotly just rendered. Elements survive the
  // restyle with their identity intact (plotly keys the parcats joins on data indices, not display
  // position), so element N holds the same category before and after and the flip reads as a swap.
  function animate(gd, before) {
    var frames = [];
    for (var i = 0; i < before.length; i++) {
      var el = before[i][0], attr = before[i][1], from = before[i][2];
      if (!el.isConnected || from == null) { continue; }
      var to = el.getAttribute(attr);
      if (to == null || to === from) { continue; }
      var fn = tween(from, to);
      if (!fn) { continue; }
      el.setAttribute(attr, from);
      frames.push([el, attr, fn, to]);
    }
    if (!frames.length) { return; }
    // a second click mid-flight retargets from wherever the first one got to
    if (gd.__qsvParcatsAnim) { cancelAnimationFrame(gd.__qsvParcatsAnim); }
    var t0 = null;
    function step(ts) {
      if (t0 === null) { t0 = ts; }
      var u = Math.min(1, (ts - t0) / DURATION), e = ease(u);
      for (var i = 0; i < frames.length; i++) {
        // land on plotly's exact strings, never an interpolated approximation of them
        frames[i][0].setAttribute(frames[i][1], u === 1 ? frames[i][3] : frames[i][2](e));
      }
      gd.__qsvParcatsAnim = u < 1 ? requestAnimationFrame(step) : null;
    }
    gd.__qsvParcatsAnim = requestAnimationFrame(step);
  }
  // EVERY early return here hands the click back to plotly, which applies the same restyle without
  // the animation — so a failure to enhance is never a broken button.
  document.addEventListener("click", function (ev) {
    var target = ev.target;
    if (!target || !target.closest) { return; }
    var btn = target.closest("g.updatemenu-button");
    if (!btn) { return; }
    // the d3 datum plotly binds to the button group
    var opts = btn.__data__;
    if (!opts || opts.method !== "restyle" || !opts.args || !opts.args2) { return; }
    var gd = btn.closest(".js-plotly-plot");
    if (!gd || !gd.data || !window.Plotly) { return; }
    var idx = (opts.args[1] && opts.args[1][0]) || 0;
    var trace = gd.data[idx];
    if (!trace || trace.type !== "parcats") { return; }
    // ...and it must be THE category-order toggle, not some other restyle button on the same chart
    if (!opts.args[0] || !("dimensions[0].categoryorder" in opts.args[0])) { return; }
    // read the direction off the trace, not a stored flag: `args` is the alphabetical state, so it
    // applies exactly while the chart is still in its baked count ("array") order
    var dims = trace.dimensions || [];
    var payload = (!dims.length || dims[0].categoryorder === "array") ? opts.args : opts.args2;
    if (!payload[0]) { return; }
    // past this point the click is ours: plotly's own handler (bound to the button, i.e. later in
    // the propagation path) must not also apply the restyle
    ev.stopPropagation();
    var before = snapshot(gd);
    Plotly.restyle(gd, payload[0], payload[1]).then(function () {
      // MUST rewind synchronously here: deferring past this callback lets the new state paint
      try { animate(gd, before); } catch (e) {}
    });
  }, true);
})();
</script>"#;

/// Splice the fullscreen `<style>` + assumed-dims prelude + `<script>` into a full HTML document
/// immediately before the closing `</body>` (so the script runs after plotly's own
/// `Plotly.newPlot` script tags). Used for the plain single-chart document produced by
/// `Plot::to_html` (standalone HTML always frames against `DEFAULT_IMG_WIDTH`×`DEFAULT_IMG_HEIGHT`,
/// per `fit_dims`); the `viz smart` pages embed the same constants directly in `smart_html_page`.
/// Falls back to appending at the end if no `</body>` is present (defensive against future plotly
/// template changes).
fn inject_fullscreen_chrome(doc: &str) -> String {
    let prelude = assumed_map_dims_prelude(DEFAULT_IMG_WIDTH as f64, DEFAULT_IMG_HEIGHT as f64);
    let fullscreen_script = fullscreen_script();
    let parcats_script = parcats_order_script();
    let chrome = format!(
        "<style>\n{FULLSCREEN_STYLE}\n</style>\n{prelude}\n{fullscreen_script}\n{parcats_script}\n"
    );
    match doc.rfind("</body>") {
        Some(idx) => {
            let mut out = String::with_capacity(doc.len() + chrome.len());
            out.push_str(&doc[..idx]);
            out.push_str(&chrome);
            out.push_str(&doc[idx..]);
            out
        },
        None => format!("{doc}{chrome}"),
    }
}

/// How long the pointer must rest on a map point before its photo opens, in milliseconds. Long
/// enough that sweeping the cursor across a dense point cloud never fires a request — the dwell
/// IS the consent gesture for the third-party image fetch, so it must be deliberate.
const PHOTO_DWELL_MS: u32 = 2000;

/// `--photos` chrome: the hover-dwell photo preview `<style>` + `<script>`, injected into a `viz
/// smart` HTML page ONLY when an image-URL column was detected and the flag was given (mirroring
/// `dict_chrome`). Every other Data Schematic omits it entirely and so never references an
/// off-origin URL.
///
/// The preview is a SMALL card anchored beside the pointer (offset by `GAP`, flipped at the
/// viewport edges) rather than a full-screen overlay, so the marker it describes and the points
/// around it stay visible while it is open — the photo is context for the map, not a replacement
/// for it.
///
/// The script reads each point's `|`-joined image URLs from the trace `customdata` that
/// `build_map_panel` embedded. Nothing is fetched on load: the `<img>` `src` is assigned only
/// after the pointer rests on a point for `PHOTO_DWELL_MS`, so opening the Data Schematic makes no
/// third-party request at all and a viewer who never dwells makes none either.
///
/// To spare the image host, a dwelled image is fetched at most ONCE and then served locally:
/// `resolveSrc` layers an in-memory object-URL cache over an IndexedDB blob store
/// (`qsv-viz-photos`), so re-dwelling a point, reopening the card, and reloading the page all serve
/// from the cache with no new request. Reading bytes cross-origin needs CORS, so this is
/// progressive enhancement: when the host sends CORS the blob is cached (and persists across
/// reloads via IndexedDB); when it does not, `resolveSrc` returns null and `show` falls back to a
/// plain `<img>` load that the browser HTTP cache still serves. A per-origin `corsBlocked` memo
/// stops retrying `fetch` on a no-CORS host, so the cache path never issues more steady-state
/// requests than the un-cached one. Failed loads (expired/404 links) are never cached.
///
/// The polling `hook()` mirrors `DICT_SCRIPT_TEMPLATE`'s: plotly renders in a deferred module
/// script, so `gd.on` exists only after render, and the fullscreen enhancer's one-time
/// `Plotly.newPlot` would drop a listener attached before it (hence the `__qsvFs` wait, with a
/// ~10s fallback so a render error doesn't disable photos permanently). `__qsvPhotoRehook` is
/// published for the theme toggle, whose MapLibre re-render silently drops `gd.on` listeners.
/// Localize `PHOTO_CHROME`.
///
/// This block mixes BOTH escaping conventions, so read the placeholder's position before
/// changing one: `__QSVI18N_PHOTO_{ALT,CLOSE,ENLARGE*,PREV,NEXT}__` sit inside HTML attributes
/// written within single-quoted JS strings and use [`html_in_js_text`] (no added quotes), while
/// the `*_JS`/`*_UNAVAIL*`/`*_SHRINK*` ones are plain JS values passed to `setAttribute`/`.title`
/// and use [`js_string_literal`] (quotes included). The `_ARIA`/`_ARIA_JS` and
/// `_ENLARGE`/`_ENLARGE_JS` pairs are the SAME message in the two different positions -- they
/// share one locale key so the markup and the DOM update can never disagree.
fn photo_chrome() -> String {
    PHOTO_CHROME
        .replace("__QSVDWELLMS__", &PHOTO_DWELL_MS.to_string())
        // HTML-attribute positions
        .replace(
            "__QSVI18N_PHOTO_ALT__",
            &html_in_js_text(&t!("viz.photo.alt")),
        )
        .replace(
            "__QSVI18N_PHOTO_CLOSE__",
            &html_in_js_text(&t!("viz.photo.close")),
        )
        .replace(
            "__QSVI18N_PHOTO_ENLARGE_ARIA__",
            &html_in_js_text(&t!("viz.photo.enlarge_aria")),
        )
        .replace(
            "__QSVI18N_PHOTO_ENLARGE__",
            &html_in_js_text(&t!("viz.photo.enlarge")),
        )
        .replace(
            "__QSVI18N_PHOTO_PREV__",
            &html_in_js_text(&t!("viz.photo.previous")),
        )
        .replace(
            "__QSVI18N_PHOTO_NEXT__",
            &html_in_js_text(&t!("viz.photo.next")),
        )
        // plain JS string positions
        .replace(
            "__QSVI18N_PHOTO_UNAVAIL_SUFFIX__",
            &js_string_literal(&t!("viz.photo.unavailable_suffix")),
        )
        .replace(
            "__QSVI18N_PHOTO_UNAVAIL__",
            &js_string_literal(&t!("viz.photo.unavailable")),
        )
        .replace(
            "__QSVI18N_PHOTO_SHRINK_ARIA__",
            &js_string_literal(&t!("viz.photo.shrink_aria")),
        )
        .replace(
            "__QSVI18N_PHOTO_ENLARGE_ARIA_JS__",
            &js_string_literal(&t!("viz.photo.enlarge_aria")),
        )
        .replace(
            "__QSVI18N_PHOTO_SHRINK__",
            &js_string_literal(&t!("viz.photo.shrink")),
        )
        .replace(
            "__QSVI18N_PHOTO_ENLARGE_JS__",
            &js_string_literal(&t!("viz.photo.enlarge")),
        )
}

const PHOTO_CHROME: &str = r##"<style>
#qsv-photo-box { position: fixed; z-index: 1000; display: none; left: 0; top: 0; }
#qsv-photo-box.open { display: block; }
#qsv-photo-box .qsv-photo-inner { position: relative; line-height: 0; background: #ffffff; border: 1px solid rgba(0, 0, 0, 0.25); border-radius: 5px; box-shadow: 0 3px 14px rgba(0, 0, 0, 0.4); padding: 3px; }
body.qsv-dark #qsv-photo-box .qsv-photo-inner { background: #1b1b1f; border-color: rgba(255, 255, 255, 0.28); }
#qsv-photo-box img { display: block; max-width: 260px; max-height: 260px; border-radius: 3px; }
/* enlarge toggle: grow the card in place (still beside the marker), clamped to the viewport so a
   big photo near an edge can't overflow it (place() also keeps the box on-screen) */
#qsv-photo-box.qsv-photo-big img { max-width: min(520px, 92vw); max-height: min(520px, 85vh); }
/* a broken source collapses the <img> to nothing, so give the card a floor to caption */
#qsv-photo-box.qsv-photo-err img { min-width: 170px; min-height: 96px; }
/* while a photo resolves through the cache/fetch, hold a neutral box (never a stale image) */
#qsv-photo-box.qsv-photo-loading img { min-width: 170px; min-height: 96px; }
/* keep the loading box near the enlarged size so the card doesn't jump small->big on load, but
   clamp to the viewport so it can't overflow on a narrow screen. min == max fixes the box: it
   also CAPS the image once it decodes, so the img can't briefly grow to full enlarged height in
   the frame between `img.src` being set and the load handler (which lifts this cap and re-places
   in the same tick) - otherwise a tall enlarged image near a viewport edge flashes off-screen. */
#qsv-photo-box.qsv-photo-big.qsv-photo-loading img { min-width: min(320px, 80vw); min-height: min(220px, 60vh); max-width: min(320px, 80vw); max-height: min(220px, 60vh); object-fit: contain; }
#qsv-photo-box.qsv-photo-loading .qsv-photo-inner::after { content: "Loading\2026"; position: absolute; inset: 0; display: flex; align-items: center; justify-content: center; color: #888; font: 12px/1 sans-serif; pointer-events: none; }
#qsv-photo-box button { position: absolute; background: rgba(0, 0, 0, 0.6); color: #ffffff; border: none; cursor: pointer; font: 600 13px/1 sans-serif; border-radius: 3px; padding: 4px 6px; }
#qsv-photo-box button:hover { background: rgba(0, 0, 0, 0.85); }
#qsv-photo-box .qsv-photo-close { top: 5px; right: 5px; }
#qsv-photo-box .qsv-photo-zoom { top: 5px; left: 5px; font-size: 14px; }
#qsv-photo-box .qsv-photo-prev, #qsv-photo-box .qsv-photo-next { top: 50%; transform: translateY(-50%); display: none; }
#qsv-photo-box .qsv-photo-prev { left: 5px; }
#qsv-photo-box .qsv-photo-next { right: 5px; }
#qsv-photo-box.qsv-photo-multi .qsv-photo-prev, #qsv-photo-box.qsv-photo-multi .qsv-photo-next { display: block; }
#qsv-photo-box .qsv-photo-cap { position: absolute; left: 50%; bottom: 6px; transform: translateX(-50%); color: #ffffff; font: 11px/1.4 sans-serif; background: rgba(0, 0, 0, 0.65); padding: 2px 7px; border-radius: 3px; white-space: nowrap; }
</style>
<script>
(function () {
  var DWELL_MS = __QSVDWELLMS__;
  // gap from the cursor: large enough that the card never sits under the pointer or the map
  // marker it describes, small enough to read as attached to it.
  var GAP = 18, EDGE = 10;
  // Grace period before auto-closing once the pointer leaves the map. The card is anchored
  // BESIDE the marker, so reaching its arrows means leaving the map first — closing instantly
  // would make them unreachable. Any re-entry (map or card) within this window cancels the close.
  var LEAVE_GRACE_MS = 300;
  var timer = null, closeTimer = null, box = null, urls = [], at = 0, anchor = { x: 0, y: 0 };
  // Bumped by every show()/step()/new dwell so a slow cache/fetch that resolves after the user
  // has navigated to a different photo or point can detect it is stale and not clobber the img.
  var showToken = 0;
  // Enlarge toggle: grows the card in place (still beside the marker, no backdrop). Persists
  // across dwells so it reads as a mode — expand once and later photos stay big until shrunk.
  var enlarged = false;
  function scheduleClose() {
    clearTimeout(closeTimer);
    closeTimer = setTimeout(close, LEAVE_GRACE_MS);
  }
  function cancelClose() { clearTimeout(closeTimer); }

  // ---- image cache (progressive enhancement over the browser HTTP cache) ------------------
  // A dwelled image is fetched from its third-party host at most once, then served locally on
  // re-dwell, card-reopen, and (via IndexedDB) across page reloads. The 2s dwell already gates
  // every request; this keeps a viewer from re-hitting the origin for an image they've seen.
  //
  // Reading bytes from a cross-origin host needs CORS, so this ENHANCES rather than replaces the
  // plain <img> load: resolveSrc() yields a local blob URL when the host cooperates, else null,
  // and show() falls back to the raw URL (which the browser HTTP cache still serves). It never
  // issues more steady-state requests than the un-cached path — see corsBlocked below.
  var DB_NAME = "qsv-viz-photos", DB_STORE = "img", MEM_MAX = 60, IDB_MAX = 300;
  var mem = new Map();          // url -> object URL (insertion order ~= LRU)
  var inflight = new Map();     // url -> Promise<objectURL|null> (dedups concurrent requests)
  var corsBlocked = new Set();  // origins whose CORS fetch failed once: skip fetching thereafter
  var dbp = null;               // memoized Promise<IDBDatabase|null>
  function openDB() {
    if (dbp) return dbp;
    dbp = new Promise(function (res) {
      try {
        if (!window.indexedDB) return res(null);
        var rq = indexedDB.open(DB_NAME, 1);
        rq.onupgradeneeded = function () {
          var db = rq.result;
          if (!db.objectStoreNames.contains(DB_STORE)) db.createObjectStore(DB_STORE);
        };
        rq.onsuccess = function () { res(rq.result); };
        rq.onerror = function () { res(null); };
      } catch (e) { res(null); }
    });
    return dbp;
  }
  function idbGet(db, url) {
    return new Promise(function (res) {
      if (!db) return res(null);
      try {
        var rq = db.transaction(DB_STORE, "readonly").objectStore(DB_STORE).get(url);
        rq.onsuccess = function () { res(rq.result || null); };
        rq.onerror = function () { res(null); };
      } catch (e) { res(null); }
    });
  }
  function idbPut(db, url, blob) {
    if (!db) return;
    try {
      var store = db.transaction(DB_STORE, "readwrite").objectStore(DB_STORE);
      store.put(blob, url);
      // approximate eviction: content is immutable (content-addressed URLs), so exact LRU
      // isn't needed — just keep the store bounded by trimming the oldest keys on overflow.
      var cnt = store.count();
      cnt.onsuccess = function () {
        var over = cnt.result - IDB_MAX;
        if (over <= 0) return;
        var cur = store.openKeyCursor();
        cur.onsuccess = function () {
          var c = cur.result;
          if (c && over-- > 0) { store.delete(c.key); c.continue(); }
        };
      };
    } catch (e) {}
  }
  function memPut(url, objUrl) {
    mem.set(url, objUrl);
    if (mem.size > MEM_MAX) {
      var oldest = mem.keys().next().value, u = mem.get(oldest);
      mem.delete(oldest);
      try { URL.revokeObjectURL(u); } catch (e) {}
    }
  }
  function originOf(url) { try { return new URL(url).origin; } catch (e) { return url; } }
  // Resolve a displayable src for `url`: memory -> IndexedDB -> one CORS fetch -> null. A null
  // result means "no readable bytes" (no CORS, a fetch error, or a 4xx/5xx) and tells show() to
  // fall back to the raw URL. Never caches a failed load (transient errors / expired links stay
  // retryable). Concurrent callers for the same url share one promise.
  function resolveSrc(url) {
    if (mem.has(url)) return Promise.resolve(mem.get(url));
    if (inflight.has(url)) return inflight.get(url);
    var p = openDB().then(function (db) {
      return idbGet(db, url).then(function (blob) {
        if (blob) return { blob: blob, store: false };
        if (corsBlocked.has(originOf(url))) return null;
        // credentials:"same-origin" (the fetch default) sends cookies only for a same-origin
        // image — so a cookie-protected same-origin photo caches instead of failing and
        // re-fetching every dwell — while cross-origin stays credential-free (no cookies sent to
        // a third-party host, and compatible with an `Access-Control-Allow-Origin: *` response).
        return fetch(url, { mode: "cors", credentials: "same-origin", cache: "force-cache" }).then(
          function (r) { return r.ok ? r.blob().then(function (b) { return { blob: b, store: true, db: db }; }) : null; },
          function () { corsBlocked.add(originOf(url)); return null; }
        );
      });
    }).then(function (hit) {
      if (!hit) return null;
      if (hit.store) idbPut(hit.db, url, hit.blob);
      var objUrl = URL.createObjectURL(hit.blob);
      memPut(url, objUrl);
      return objUrl;
    }).catch(function () { return null; })
      .then(function (objUrl) { inflight.delete(url); return objUrl; });
    inflight.set(url, p);
    return p;
  }
  // -----------------------------------------------------------------------------------------
  function build() {
    if (box) return box;
    box = document.createElement("div");
    box.id = "qsv-photo-box";
    box.innerHTML = '<div class="qsv-photo-inner">' +
      '<img alt="__QSVI18N_PHOTO_ALT__" />' +
      '<button type="button" class="qsv-photo-close" aria-label="__QSVI18N_PHOTO_CLOSE__">&#215;</button>' +
      '<button type="button" class="qsv-photo-zoom" aria-label="__QSVI18N_PHOTO_ENLARGE_ARIA__" title="__QSVI18N_PHOTO_ENLARGE__">&#10530;</button>' +
      '<button type="button" class="qsv-photo-prev" aria-label="__QSVI18N_PHOTO_PREV__">&#8249;</button>' +
      '<button type="button" class="qsv-photo-next" aria-label="__QSVI18N_PHOTO_NEXT__">&#8250;</button>' +
      '<div class="qsv-photo-cap"></div></div>';
    document.body.appendChild(box);
    box.querySelector(".qsv-photo-close").addEventListener("click", close);
    // enlarge/shrink toggle: flip the persistent mode, restyle, and re-place so the resized card
    // stays beside the marker and on-screen.
    box.querySelector(".qsv-photo-zoom").addEventListener("click", function (e) {
      e.stopPropagation();
      enlarged = !enlarged;
      applyBig();
      place();
    });
    box.querySelector(".qsv-photo-prev").addEventListener("click", function (e) { e.stopPropagation(); step(-1); });
    box.querySelector(".qsv-photo-next").addEventListener("click", function (e) { e.stopPropagation(); step(1); });
    // the natural size is unknown until the bytes arrive, so re-place once it is: an unplaced
    // card measured at its empty size would otherwise land wrong and could cover the point.
    box.querySelector("img").addEventListener("load", function () {
      box.classList.remove("qsv-photo-loading");
      place();
    });
    // A data-supplied URL can 404 or expire (Boston's cloudinary links do), and a broken <img>
    // renders as an empty card that reads like a bug. Say so instead.
    box.querySelector("img").addEventListener("error", function () {
      box.classList.remove("qsv-photo-loading");
      box.classList.add("qsv-photo-err");
      box.querySelector(".qsv-photo-cap").textContent =
        urls.length > 1 ? (at + 1) + " / " + urls.length + __QSVI18N_PHOTO_UNAVAIL_SUFFIX__ : __QSVI18N_PHOTO_UNAVAIL__;
      place();
    });
    // moving onto the card keeps it up (its arrows are the reason to go there); moving off it
    // to anywhere that isn't the map dismisses it.
    box.addEventListener("mouseenter", cancelClose);
    box.addEventListener("mouseleave", scheduleClose);
    return box;
  }
  function close() {
    clearTimeout(closeTimer);
    // invalidate any in-flight resolveSrc so a late blob can't paint a dismissed card
    showToken++;
    if (!box) return;
    box.classList.remove("open");
    box.classList.remove("qsv-photo-loading");
    // drop the src so a dismissed card holds no decoded image (and a re-open re-requests rather
    // than flashing the previous point's photo while the new one loads)
    box.querySelector("img").removeAttribute("src");
    urls = [];
  }
  // Sync the enlarge state onto the card: toggle the size class and swap the button glyph/label
  // between enlarge (&#10530;) and shrink (&#10529;). Called from show() (so a new photo respects
  // the persistent mode) and from the toggle handler.
  function applyBig() {
    if (!box) return;
    box.classList.toggle("qsv-photo-big", enlarged);
    var z = box.querySelector(".qsv-photo-zoom");
    if (z) {
      z.innerHTML = enlarged ? "&#10529;" : "&#10530;";
      z.setAttribute("aria-label", enlarged ? __QSVI18N_PHOTO_SHRINK_ARIA__ : __QSVI18N_PHOTO_ENLARGE_ARIA_JS__);
      z.title = enlarged ? __QSVI18N_PHOTO_SHRINK__ : __QSVI18N_PHOTO_ENLARGE_JS__;
    }
  }
  // Anchor beside the MARKER without covering it: vertically centered on the point and offset
  // horizontally by GAP. Preferring the LEFT side keeps the card clear of plotly's own hover
  // tooltip, which opens to the right of the point; it flips right only when there isn't room,
  // and both axes are clamped so an edge-of-map point still shows its photo in full.
  function place() {
    if (!box || !box.classList.contains("open")) return;
    var r = box.getBoundingClientRect();
    var w = r.width || 0, h = r.height || 0;
    var left = anchor.x - GAP - w;
    if (left < EDGE) left = anchor.x + GAP;
    if (left + w > window.innerWidth - EDGE) left = Math.max(EDGE, window.innerWidth - EDGE - w);
    var top = anchor.y - h / 2;
    if (top + h > window.innerHeight - EDGE) top = window.innerHeight - EDGE - h;
    if (top < EDGE) top = EDGE;
    box.style.left = Math.round(left) + "px";
    box.style.top = Math.round(top) + "px";
  }
  function step(d) {
    if (urls.length < 2) return;
    at = (at + d + urls.length) % urls.length;
    show();
  }
  function show() {
    var b = build(), img = b.querySelector("img");
    b.classList.remove("qsv-photo-err");
    b.querySelector(".qsv-photo-cap").textContent = urls.length > 1 ? (at + 1) + " / " + urls.length : "";
    b.classList.toggle("qsv-photo-multi", urls.length > 1);
    b.classList.add("open");
    applyBig();
    // Bump the token so any in-flight resolve for a previous photo/point can detect it is stale.
    var url = urls[at], token = ++showToken;
    // A cached image paints instantly. Otherwise DROP the previous point's photo up front and
    // show a neutral "Loading…" box, so the card never displays a stale image at the new anchor
    // while the async cache/fetch runs. The load/error handlers clear the loading state.
    // `place()` is called AFTER the size-determining state (loading box / cached src) is applied,
    // so the card is clamped to the viewport at its actual size — otherwise an enlarged loading
    // box could overflow the edge or cover the marker until the image loaded. The load handler
    // re-places once the real image dimensions are known.
    var cached = mem.get(url);
    if (cached) {
      b.classList.remove("qsv-photo-loading");
      if (img.getAttribute("src") !== cached) img.src = cached;
      place();
      return;
    }
    img.removeAttribute("src");
    b.classList.add("qsv-photo-loading");
    place();
    resolveSrc(url).then(function (src) {
      if (token !== showToken) return;
      // `src || url` falls back to the raw URL when the host has no CORS or the fetch failed —
      // the browser HTTP cache still applies there.
      img.src = src || url;
    });
  }
  document.addEventListener("keydown", function (e) {
    if (!box || !box.classList.contains("open")) return;
    if (e.key === "Escape") close();
    else if (e.key === "ArrowLeft") step(-1);
    else if (e.key === "ArrowRight") step(1);
  });
  // a click anywhere outside the card dismisses it (it has no backdrop of its own)
  document.addEventListener("click", function (e) {
    if (box && box.classList.contains("open") && !box.contains(e.target)) close();
  });
  // Whether a rendered panel div carries the photo payload — a trace whose customdata is a
  // (non-empty) array of strings, i.e. the `|`-joined per-point URLs. Only such a panel gets the
  // hover + map-leave handlers; attaching the close-cancel handler to OTHER panels would let the
  // pointer moving from the map into an adjacent panel cancel the map-leave timer and strand a
  // stale card.
  function hasPhotoTrace(gd) {
    var d = gd.data;
    if (!d) return false;
    for (var i = 0; i < d.length; i++) {
      var cd = d[i] && d[i].customdata;
      if (cd && cd.length && typeof cd[0] === "string") return true;
    }
    return false;
  }
  function hook() {
    var tries = hook.tries || 0;
    var gds = document.querySelectorAll('#qsv-viz-smart-grid, [id^="qsv-viz-panel-"]');
    var pending = false;
    gds.forEach(function (gd) {
      if (gd.__qsvPhotoHooked || gd.__qsvPhotoSkip) return;
      if (!gd.on || (!gd.__qsvFs && tries < 100)) { pending = true; return; }
      // Decide ONCE, now that the panel has rendered: a non-photo panel is marked skipped so it
      // is never hooked and never keeps the poll alive. The map panel (photos present) proceeds.
      if (!hasPhotoTrace(gd)) { gd.__qsvPhotoSkip = true; return; }
      gd.__qsvPhotoHooked = true;
      // DOM listeners on the panel div itself, so leaving the map auto-dismisses the card.
      // Attached ONCE and flagged separately from __qsvPhotoHooked: the div survives plotly's
      // re-renders (only its `gd.on` listeners are dropped), so re-adding them on every rehook
      // would stack duplicate handlers.
      if (!gd.__qsvPhotoDom) {
        gd.__qsvPhotoDom = true;
        gd.addEventListener("mouseenter", cancelClose);
        gd.addEventListener("mouseleave", function () {
          // a pending dwell is abandoned too, so the card can't pop up after the pointer left
          clearTimeout(timer);
          scheduleClose();
        });
      }
      gd.on("plotly_hover", function (ev) {
        var pt = ev && ev.points && ev.points[0];
        var cd = pt && pt.customdata;
        if (typeof cd !== "string" || !cd) return;
        // Anchor on the MARKER, not the pointer: plotly's map hover event carries no
        // clientX/clientY (only a synthetic mousemove), and the marker is what must stay
        // uncovered anyway. `bbox` is relative to the plot div, so add the div's viewport
        // offset. Resolved at hover time — the event is not retained across the dwell.
        var r = gd.getBoundingClientRect(), bb = pt.bbox, x, y;
        if (bb) {
          x = r.left + (bb.x0 + bb.x1) / 2;
          y = r.top + (bb.y0 + bb.y1) / 2;
        } else if (typeof pt.xPixel === "number" && typeof pt.yPixel === "number") {
          x = r.left + pt.xPixel;
          y = r.top + pt.yPixel;
        } else {
          return;
        }
        clearTimeout(timer);
        timer = setTimeout(function () {
          anchor = { x: x, y: y };
          urls = cd.split("|");
          at = 0;
          show();
        }, DWELL_MS);
      });
      gd.on("plotly_unhover", function () { clearTimeout(timer); });
    });
    hook.tries = tries + 1;
    if (pending && hook.tries < 200) setTimeout(hook, 100);
  }
  window.__qsvPhotoRehook = hook;
  hook();
})();
</script>
"##;

/// Cross-link between the data viewer drawer's rows and the map panel's points, emitted only when
/// BOTH exist (a `Map`/`Geo` panel carrying `row_ids`, and the drawer chrome).
///
/// Both directions travel over the trace's plotly `ids` — the 0-based data-row ordinal that
/// `build_map_panel` attaches to the core and outlier point traces ONLY. The extent and
/// `--geojson` overlay traces carry no `ids`, so a click on a boundary can never select a row.
/// An id of `""` means the row exists but sits past the drawer's preview prefix.
///
/// The drawer side is reached exclusively through the small public seam that
/// `DATA_DRAWER_SCRIPT` publishes (`__qsvDataSelect` / `__qsvDataPageTo` / `__qsvDataNote` and the
/// `qsv-data-selection` event) rather than by touching DataTables directly — the drawer is built
/// lazily on first open, so there is nothing to talk to at page load.
///
/// Hooking follows the `PHOTO_CHROME` idiom: poll until the panel has rendered, wait for the
/// fullscreen script's one-time `Plotly.newPlot` (`__qsvFs`) so the click listener is not dropped
/// by it, and publish `__qsvSelRehook` for the theme toggle to call after ITS re-render.
/// Point-count ceiling above which a row selection is painted with the pin ALONE, skipping
/// plotly's `selectedpoints` dimming.
///
/// `selectedpoints` is declared `editType: "calc"` (verified in the vendored bundle), so assigning
/// it recalculates the ENTIRE trace -- every point and its hover string -- and rebuilds the GeoJSON
/// that MapLibre holds, structured-cloning it into the map worker. That cost is paid on every
/// single row click, and it is charged against the whole trace no matter how few rows are selected.
///
/// Measured against the Boston 311 map trace (264,951 points, each carrying a ~150-char hover
/// string) in an isolated harness: ~44 MB of JS heap per click, NOT reclaimed between clicks, so
/// eight row clicks walked the heap 264 MB -> 559 MB monotonically and froze the renderer on the
/// real page (which already carries ~300-500 MB of figure JSON and drawer rows). The same probe
/// reads +8 MB/click at 50k points and +13 MB/click at 150k.
///
/// 50k is where that cost stops being material. It is also where the dimming stops paying for
/// itself: fading 265k points to pick out one reads as a blank map, whereas the pin marks the row
/// unambiguously. Above this line the pin is therefore the whole answer -- see `pinsFor`, which
/// switches to pinning EVERY selected row rather than only the ones with no plotted point.
///
/// Note the default `MAX_SMART_POINTS` is 150k, so most large smart dashboards land above this
/// threshold: pin-mode is the common case for big maps, not an edge case.
const SEL_DIM_MAX_POINTS: usize = 50_000;

/// Localize `MAP_SELECT_CHROME`. Same contract as [`fullscreen_script`]: bare placeholders,
/// unconditional substitution, `js_string_literal` for quoting + escaping.
fn map_select_chrome(lat_col: usize, lon_col: usize) -> String {
    MAP_SELECT_CHROME
        .replace("__QSV_LAT_COL__", &lat_col.to_string())
        .replace("__QSV_LON_COL__", &lon_col.to_string())
        .replace("__QSV_SEL_DIM_MAX__", &SEL_DIM_MAX_POINTS.to_string())
        .replace(
            "__QSVI18N_SELECTED_ROW__",
            &js_string_literal(&t!("viz.map.selected_row")),
        )
        .replace(
            "__QSVI18N_ROW_NOT_PLOTTED__",
            &js_string_literal(&t!("viz.map.row_not_plotted")),
        )
        .replace(
            "__QSVI18N_ROW_HAS_NO_COORDINATES__",
            &js_string_literal(&t!("viz.map.row_has_no_coordinates")),
        )
        .replace(
            "__QSVI18N_ROW_NOT_IN_PREVIEW__",
            &js_string_literal(&t!("viz.map.row_not_in_preview")),
        )
        .replace(
            "__QSVI18N_ROW_FILTERED_OUT__",
            &js_string_literal(&t!("viz.map.row_is_filtered_out")),
        )
}

const MAP_SELECT_CHROME: &str = r##"<script>
(function () {
  function isPointTrace(t) {
    return !!(t && (t.type === "scattermap" || t.type === "scattergeo") && t.ids && t.ids.length);
  }
  // Type-gated, NOT matched on the name alone: trace names elsewhere in a Data Schematic come from the
  // data (series values, category labels), so a row whose category happened to read "qsv-row-pin"
  // could otherwise make a bar panel look pin-bearing. Only a `Map`/`Geo` panel ever emits these
  // names on these types, and none of the trace names on those panels are data-derived.
  function isPinTrace(t) {
    return !!(t && (t.type === "scattermap" || t.type === "scattergeo")
      && (t.name === "qsv-row-pin" || t.name === "qsv-row-pin-halo"));
  }
  // Both halves or neither: the pin is drawn as a core plus the halo beneath it and `applyPins`
  // restyles the pair together, so a lone half can never actually show anything.
  function hasPinPair(gd) {
    var pin = false, halo = false;
    (gd.data || []).forEach(function (t) {
      if (!isPinTrace(t)) return;
      if (t.name === "qsv-row-pin") pin = true;
      else halo = true;
    });
    return pin && halo;
  }
  // A panel is worth hooking if it can show a selection AT ALL — which is not the same as having
  // selectable points. The second case is a DENSITY map: its heatmap core carries no `ids` (and it
  // may have no outlier markers either), so nothing on it can be clicked back to a row, yet every
  // drawer row is by definition "not among plotted points" and can still be pinned at its own
  // coordinates. Without this the pin traces would be emitted and then never reachable.
  function hasPointTrace(gd) {
    return !!(gd.data && (gd.data.some(isPointTrace) || hasPinPair(gd)));
  }
  // id string -> {t: traceIndex, p: pointIndex}. Built once per hook so applying a selection is
  // O(selected) rather than O(points) — the core trace can carry 150k of them.
  // Above this many plotted points a selection is marked by the PIN ALONE and the dimming restyle
  // is skipped entirely -- see SEL_DIM_MAX_POINTS on the Rust side for the measurements behind the
  // number and why dimming is the thing worth dropping.
  var SEL_DIM_MAX = __QSV_SEL_DIM_MAX__;
  function buildIndex(gd) {
    var index = new Map(), traces = [], points = 0;
    gd.data.forEach(function (t, ti) {
      if (!isPointTrace(t)) return;
      traces.push(ti);
      // every point costs on a `calc` restyle, including the ones with no id, so the mode test
      // counts the trace's full length rather than the index's
      points += t.ids.length;
      for (var p = 0; p < t.ids.length; p++) {
        // "" marks a point whose row is past the preview prefix: nothing to select
        if (t.ids[p]) index.set(t.ids[p], { t: ti, p: p });
      }
    });
    gd.__qsvSelIndex = index;
    gd.__qsvSelTraces = traces;
    // Fixed for the life of the panel: the trace lengths never change after render, so the mode
    // cannot flip mid-session and no code path has to undo the other mode's painting.
    gd.__qsvSelBig = points > SEL_DIM_MAX;
    // The row pin and its halo, located by their sentinel names -- their indexes are not fixed,
    // since the optional extent and --geojson overlay traces sit between them and the points.
    gd.__qsvSelPin = gd.data.findIndex(function (t) { return t && t.name === "qsv-row-pin"; });
    gd.__qsvSelPinHalo = gd.data.findIndex(function (t) { return t && t.name === "qsv-row-pin-halo"; });
  }
  // ---- the row pin -----------------------------------------------------------------------------
  //
  // The map draws at most `MAX_SMART_POINTS`, so on a large dataset most drawer rows have no point
  // of their own -- on the Pittsburgh 311 Data Schematic only ~16% do. Such a row still HAS a location:
  // its coordinates sit in the drawer, in the same columns the map was built from. The pin marks
  // that location explicitly rather than panning to bare ground, which would invite reading a
  // neighbouring point as the selected record.
  //
  // Two causes are NOT the same and must not be conflated:
  //   * coordinates parse  -> the point was merely downsampled away  -> pin it
  //   * coordinates do not -> the row has no location at all         -> say so, pin nothing
  var LAT_COL = __QSV_LAT_COL__, LON_COL = __QSV_LON_COL__;
  // Rust decimal float syntax, which is what the server's `parse_f64` accepts. `Number()` alone is
  // BROADER — it also takes `0x`/`0o`/`0b` literals, which the server rejects — and that gap breaks
  // the one invariant this parse exists to hold: a cell the server refused must be refused here
  // too, or the row gets pinned at a bogus location instead of being reported as having none.
  // (`inf`/`nan`, which Rust does accept, are excluded here and by the isFinite check either way.)
  var DECIMAL_FLOAT = /^[+-]?(\d+\.?\d*|\.\d+)([eE][+-]?\d+)?$/;
  function cellNumber(cell) {
    // a DATE column ships as [display, sortKey]; coordinate columns never do, but be tolerant
    if (Array.isArray(cell)) cell = cell[0];
    if (cell === null || cell === undefined) return NaN;
    var s = String(cell).trim();
    return DECIMAL_FLOAT.test(s) ? Number(s) : NaN;
  }
  // Mirrors the server's own test exactly (`parse_f64` plus the lat/lon range check in
  // `build_map_panel`): a row rejected THERE never became a point, so rejecting it here too is what
  // separates "no location" from "downsampled away".
  function rowCoord(id) {
    if (!window.__qsvDataRowCells) return null;
    var cells = window.__qsvDataRowCells(id);
    if (!cells) return null;
    var lat = cellNumber(cells[LAT_COL]), lon = cellNumber(cells[LON_COL]);
    if (!isFinite(lat) || !isFinite(lon)) return null;
    if (lat < -90 || lat > 90 || lon < -180 || lon > 180) return null;
    return { lat: lat, lon: lon };
  }
  function pinsFor(gd, ids) {
    var pts = [], missing = 0, notPlotted = 0;
    var big = !!gd.__qsvSelBig;
    ids.forEach(function (id) {
      var loc = gd.__qsvSelIndex.get(String(id));
      if (loc) {
        // Normally a plotted row highlights its own point, so only the others need a pin. In
        // pin-only mode nothing dims that point, so the pin has to carry the plotted rows too.
        if (!big) return;
        // Its OWN coordinates, read off the trace -- NOT rowCoord, which goes through the drawer
        // and therefore only covers the preview prefix. On a 265k-point map against a 50k preview
        // that would leave every point past row 50,000 with no pin at all.
        var c = coordArrays(gd, loc.t);
        if (!c) return;
        var lat = c.lat[loc.p], lon = c.lon[loc.p];
        if (!isFinite(lat) || !isFinite(lon)) return;
        pts.push({ lat: lat, lon: lon, plotted: true });
        return;
      }
      var rc = rowCoord(id);
      // `notPlotted` is what the note keys off, NOT points.length: in pin-only mode a pin is the
      // ordinary way a SELECTED row is shown, so "row not among plotted points" would be a lie
      // for most pins. Only a row with no point of its own earns that note.
      if (rc) { pts.push({ lat: rc.lat, lon: rc.lon, plotted: false }); notPlotted++; }
      else missing++;
    });
    return { points: pts, missing: missing, notPlotted: notPlotted };
  }
  function applyPins(gd, pts) {
    if (!(gd.__qsvSelPin >= 0) || !(gd.__qsvSelPinHalo >= 0)) return;
    var lats = pts.map(function (p) { return p.lat; });
    var lons = pts.map(function (p) { return p.lon; });
    // A pin means two different things now, so it must not claim the wrong one: in pin-only mode
    // most pins sit ON a plotted point and are simply where the selected row is.
    var text = pts.map(function (p) {
      return (p.plotted ? __QSVI18N_SELECTED_ROW__ : __QSVI18N_ROW_NOT_PLOTTED__) + "<br>";
    });
    try {
      // halo and core in ONE restyle: they are a single mark and must never disagree, not even
      // for a frame. Values distribute positionally over the trace list.
      Plotly.restyle(
        gd,
        { lat: [lats, lats], lon: [lons, lons], text: [text, text] },
        [gd.__qsvSelPinHalo, gd.__qsvSelPin]
      );
    } catch (e) {}
  }
  function applySelection(gd, ids, pinned) {
    if (!gd.__qsvSelTraces || !gd.__qsvSelTraces.length) return;
    // Pin-only mode: the pin IS the selection, and `selectedpoints` is `editType: "calc"`, so
    // setting it here would recalculate every point and its hover string and rebuild the GeoJSON
    // MapLibre holds -- ~44 MB per click on a 265k-point trace, unreclaimed, which is what froze
    // the renderer. Returning before the restyle is the whole fix; nothing needs unwinding because
    // `selectedpoints` is never emitted in the figure and this mode is fixed at buildIndex time,
    // so no path can have painted dimming that is now left stale.
    if (gd.__qsvSelBig) return;
    var per = {}, hits = 0;
    gd.__qsvSelTraces.forEach(function (ti) { per[ti] = []; });
    ids.forEach(function (id) {
      var loc = gd.__qsvSelIndex.get(String(id));
      if (loc) { per[loc.t].push(loc.p); hits++; }
    });
    // `[]` and `null` mean different things to plotly and BOTH are needed here. `[]` is "a
    // selection is active but nothing in THIS trace matched", which dims the trace's points as
    // unselected — the right treatment for the outlier call-out trace while a core point is
    // selected, since leaving its markers at full amber would read as though they were selected
    // too. `null` is "no selection at all" and restores normal styling.
    //
    // The rule that `null` is also right when NOTHING matched (dimming the whole map with nothing
    // highlighted just looks broken) now carries a precondition rather than being unconditional:
    // a selected row with no plotted point may still be PINNED, and then there IS something
    // highlighted, so the plotted traces should dim around it exactly as they do for a real hit.
    // `null` survives only when the selection produces neither a point nor a pin.
    var sel = gd.__qsvSelTraces.map(function (ti) { return (hits || pinned) ? per[ti] : null; });
    try {
      Plotly.restyle(gd, { selectedpoints: sel }, gd.__qsvSelTraces);
    } catch (e) {}
  }
  // ---- camera: bring the selected point(s) into view ------------------------------------------
  //
  // MapLibre `scattermap` panels only. A `scattergeo` panel is deliberately left alone: it is
  // chosen only for a world/continent extent, and the continent scope is narrowed only when EVERY
  // point falls inside it, so its points are already in frame by construction. It also has no GL
  // instance — its camera is `geo.center`, a second mechanism that would buy nothing here.
  //
  // px of breathing room around a multi-point fit
  var MAP_REVEAL_FIT_PADDING = 40;
  function glMapFor(gd, key) {
    var fl = gd._fullLayout || {}, sp = fl[key] && fl[key]._subplot, m = sp && sp.map;
    return (m && typeof m.getBounds === "function" && typeof m.easeTo === "function") ? m : null;
  }
  function mapLayoutKeys(gd) {
    var lay = (gd && gd.layout) || {};
    return Object.keys(lay).filter(function (k) {
      return /^map\d*$/.test(k) && lay[k] && typeof lay[k].zoom === "number";
    });
  }
  // Keep `gd.layout`'s camera in step with the GL map.
  //
  // `Plotly.restyle` — which paints EVERY selection change — re-aims a MapLibre subplot at
  // `gd.layout[k]`, so a camera the layout does not know about is discarded: a reader who pans by
  // hand and then clicks a row would be snapped back, and worse, the "is this point already on
  // screen?" test below would have been answered against a view that no longer exists once the
  // restyle settles. Mirroring every settled move keeps the layout the single source of truth for
  // the camera, which is also what makes a revealed position survive the next selection change.
  //
  // Plain property sets, never a relayout: on a MapLibre subplot in the pinned plotly fork
  // relayout/react throw "setData of undefined" and blank the layer. The fullscreen re-fit is
  // unaffected — it recomputes from its own `__qsvBaked` snapshot, taken before any of this, and
  // so are the baked extent buttons, whose relayout args are absolute.
  //
  // Deliberately scoped to drawer-bearing pages: this whole script is only emitted when there is
  // a data viewer to cross-link to, so a map Data Schematic without one keeps the older behavior of
  // snapping back to the baked camera on a re-render. Do NOT "fix" that by hoisting this into
  // `fullscreen_script` — its per-panel bodies are wrapped in swallowing `try {} catch {}`, so an
  // unrelated helper throwing there would take the mirror down with it.
  function mirrorCamera(gd, tries) {
    var keys = mapLayoutKeys(gd);
    if (!keys.length) return;
    if (tries === undefined) tries = 20;
    if (!keys.every(function (k) { return glMapFor(gd, k); })) {
      if (tries > 0) setTimeout(function () { mirrorCamera(gd, tries - 1); }, 100);
      return;
    }
    keys.forEach(function (k) {
      var m = glMapFor(gd, k);
      // a re-render hands out a NEW GL instance, so the guard belongs on the instance
      if (!m || m.__qsvCamMirror) return;
      m.__qsvCamMirror = true;
      m.on("moveend", function () {
        try {
          var lay = gd.layout && gd.layout[k];
          if (!lay) return;
          var c = m.getCenter();
          lay.center = { lon: c.lng, lat: c.lat };
          lay.zoom = m.getZoom();
        } catch (e) {}
      });
    });
  }
  // The DECODED coordinate arrays for a trace.
  //
  // Not `gd.data[ti].lat` — under the float32 `bdata` encoding that is still the wire object
  // `{dtype, bdata}`, which has no indices, so reading a coordinate off it yields `undefined` and
  // every camera move would silently be skipped for exactly the dense maps that need it most.
  // plotly keeps the decoded `Float32Array` on `_fullData`, the same internal view this page
  // already reads for `_fullLayout`; it indexes just like the plain array an uncompressed page
  // ships, so one path covers both. The length check is what rejects a still-encoded array.
  function coordArrays(gd, ti) {
    var full = (gd._fullData && gd._fullData[ti]) || null, raw = gd.data[ti] || {};
    var lat = (full && full.lat) || raw.lat, lon = (full && full.lon) || raw.lon;
    if (!lat || !lon || typeof lat.length !== "number" || typeof lon.length !== "number") return null;
    return { lat: lat, lon: lon };
  }
  // selected points grouped by map subplot key: {"map": [{lat, lon}, ...], "map2": [...]}
  // Pinned rows count as selected points for framing purposes — the pin is what the reader is
  // being sent to look at — but only on a MapLibre panel; a `scattergeo` pin is shown and never
  // chased, which is the same rule its plotted points follow.
  function selectedMapPoints(gd, ids, pins) {
    var byKey = {};
    if (pins && pins.length && gd.__qsvSelPin >= 0) {
      var pinTrace = gd.data[gd.__qsvSelPin];
      if (pinTrace && pinTrace.type === "scattermap") {
        byKey[pinTrace.subplot || "map"] = pins.slice();
      }
    }
    // In pin-only mode every selected row is ALREADY pinned, so walking the ids again would add
    // each plotted row a second time at the same coordinates. One selected row would then look
    // like two points and take the fitBounds branch below with a degenerate zero-area box, instead
    // of the pan-only easeTo that a single point is supposed to get.
    if (gd.__qsvSelBig) return byKey;
    ids.forEach(function (id) {
      var loc = gd.__qsvSelIndex.get(String(id));
      if (!loc) return;
      var t = gd.data[loc.t];
      if (!t || t.type !== "scattermap") return;
      var c = coordArrays(gd, loc.t);
      if (!c) return;
      var lat = c.lat[loc.p], lon = c.lon[loc.p];
      if (!isFinite(lat) || !isFinite(lon)) return;
      var k = t.subplot || "map";
      (byKey[k] = byKey[k] || []).push({ lat: lat, lon: lon });
    });
    return byKey;
  }
  // Aim one map subplot at the selected points, unless they are all on screen already.
  //
  // MUST NOT run in the same task as the `Plotly.restyle` that paints the selection: restyle
  // CANCELS a camera animation started in that tick. easeTo/fitBounds return normally, nothing
  // throws, and the camera simply never leaves its starting point — while the identical call one
  // task later always works (reproduced repeatedly, both directions). Callers therefore defer,
  // and the attempt is verified afterwards in case the same swallow ever reappears at a longer
  // delay: a cancelled animation leaves the center bit-identical, whereas a live one has moved
  // well before the check.
  function aimAt(gd, k, list, attempts) {
    var m = glMapFor(gd, k);
    if (!m) return;
    try {
      // Nothing moves while every selected point is already on screen: the highlight alone is the
      // answer there, and a map that re-aims on every click is worse than one that holds still.
      // This is also why a point inside a collapsed cluster bubble is left alone — its coordinates
      // ARE in view; it is the bubble that hides it, and zooming to break the cluster open would
      // let a click in the table change the reader's zoom.
      var b = m.getBounds();
      if (list.every(function (p) { return b.contains([p.lon, p.lat]); })) return;
      var before = m.getCenter();
      gd.__qsvSelCamera = k;
      if (list.length === 1) {
        // a single point: pan only. The zoom is the reader's, and a click in the table has no
        // business changing it.
        m.easeTo({ center: [list[0].lon, list[0].lat] });
      } else {
        var lons = list.map(function (p) { return p.lon; });
        var lats = list.map(function (p) { return p.lat; });
        // `maxZoom` pinned to the CURRENT zoom keeps this a pure "bring into view": the fit can
        // only zoom OUT to take in scattered points, never surprise-zoom into two adjacent ones.
        // Known limitation: a min/max box is wrong across the antimeridian, so a selection that
        // straddles it fits the long way round.
        m.fitBounds([[Math.min.apply(null, lons), Math.min.apply(null, lats)],
                     [Math.max.apply(null, lons), Math.max.apply(null, lats)]],
                    { padding: MAP_REVEAL_FIT_PADDING, maxZoom: m.getZoom() });
      }
      if (attempts > 0) setTimeout(function () {
        // Never re-issue while the reader has hold of the map. A swallowed move plus a drag that
        // happens to be passing through the starting center would otherwise yank the camera out
        // from under their hand. Our own move is not caught by this: if it took, the center has
        // already changed and the check below does not fire anyway.
        if (typeof m.isMoving === "function" && m.isMoving()) return;
        var now = m.getCenter();
        if (now.lng === before.lng && now.lat === before.lat) aimAt(gd, k, list, attempts - 1);
      }, 350);
    } catch (e) {}
  }
  function revealOnMap(gd, ids, pins, tries) {
    if (!gd.__qsvSelIndex || !ids.length) return;
    var pts = selectedMapPoints(gd, ids, pins), keys = Object.keys(pts);
    if (!keys.length) return;
    // The GL map attaches a frame or two after a render — wait for it, bounded, the same way the
    // fullscreen script's camera re-fit does.
    if (tries === undefined) tries = 20;
    if (!keys.every(function (k) { return glMapFor(gd, k); })) {
      if (tries > 0) setTimeout(function () { revealOnMap(gd, ids, pins, tries - 1); }, 100);
      return;
    }
    keys.forEach(function (k) { aimAt(gd, k, pts[k], 2); });
  }
  // `Plotly.newPlot` detaches every listener on the graph div, and for MapLibre map traces it
  // does so AGAIN on a deferred pass that runs after its own promise has already resolved — so a
  // listener attached from that promise's callback is silently dropped (observed directly: after
  // a theme flip the panel still reported itself hooked while its click handler was gone, and
  // map clicks stopped selecting rows).
  //
  // Rather than guess a safe delay, attach at several and let a token retire every superseded
  // binding, so exactly one fires no matter which pass ends up owning the emitter.
  function bindClick(gd) {
    var token = (gd.__qsvSelToken || 0) + 1;
    gd.__qsvSelToken = token;
    gd.on("plotly_click", function (ev) {
      if (gd.__qsvSelToken !== token) return;
      onClick(gd, ev);
    });
  }
  function bindClickSettled(gd) {
    bindClick(gd);
    setTimeout(function () { bindClick(gd); }, 250);
    setTimeout(function () { bindClick(gd); }, 1200);
  }
  // last selection seen from the drawer, so a re-render (theme flip, fullscreen) can restore it
  var current = [];
  // repaint everything a re-render drops: the selection styling AND the pin's coordinates
  function restore(gd) {
    var pins = pinsFor(gd, current);
    applySelection(gd, current, pins.points.length);
    applyPins(gd, pins.points);
  }
  function eachHooked(fn) {
    document.querySelectorAll('#qsv-viz-smart-grid, [id^="qsv-viz-panel-"]').forEach(function (gd) {
      if (gd.__qsvSelHooked) fn(gd);
    });
  }
  // rows -> map
  document.addEventListener("qsv-data-selection", function (ev) {
    current = (ev.detail && ev.detail.indexes) || [];
    // Only a selection that ORIGINATED in the table moves the camera. Without this gate a map
    // click echoes straight back — `__qsvDataSelect` dispatches the same event — and the map
    // re-aims away from the very point the reader just clicked.
    var fromRows = !!(ev.detail && ev.detail.source === "rows");
    var reported = null;
    eachHooked(function (gd) {
      var pins = pinsFor(gd, current);
      if (!reported) reported = pins;
      // BOTH restyles run here, synchronously, before the camera is scheduled. Order is
      // load-bearing: a restyle cancels a camera move started in the same task, so the deferral
      // below must come after the last of them, not between them.
      applySelection(gd, current, pins.points.length);
      applyPins(gd, pins.points);
      if (fromRows) {
        setTimeout(function () { revealOnMap(gd, current, pins.points); }, 0);
      }
    });
    if (reported) notePins(reported);
  });
  // Why a pinned row looks different from a plotted one. The pin's own hover label carries the
  // explanation, so the note only has to make the reader look once — on a downsampled Data Schematic
  // most selections produce a pin, and a note every time would be pure noise.
  var notedNotPlotted = false;
  function notePins(pins) {
    if (!window.__qsvDataNote) return;
    // Keyed on `notPlotted`, not on how many pins were drawn: in pin-only mode a pin is the normal
    // way ANY selected row is shown, so counting pins here would fire "row not among plotted
    // points" on every single click, about rows that are plotted perfectly well.
    if (pins.notPlotted) {
      if (notedNotPlotted) return;
      notedNotPlotted = true;
      window.__qsvDataNote(__QSVI18N_ROW_NOT_PLOTTED__);
    } else if (pins.missing && !pins.points.length) {
      // nothing was drawn for this selection at all, so this note is its only feedback
      window.__qsvDataNote(__QSVI18N_ROW_HAS_NO_COORDINATES__);
    }
  }
  // map -> rows
  function onClick(gd, ev) {
    // with the drawer closed there is no visible selection, so a click keeps its old meaning
    if (!document.body.classList.contains("qsv-data-open")) return;
    var p = ev && ev.points && ev.points[0];
    if (!p) return;
    var t = gd.data[p.curveNumber];
    if (!isPointTrace(t)) return;
    var id = t.ids[p.pointNumber];
    // guard BEFORE any numeric coercion: +"" is 0, which would silently select the first row
    if (!id || !window.__qsvDataSelect) {
      if (id === "" && window.__qsvDataNote) window.__qsvDataNote(__QSVI18N_ROW_NOT_IN_PREVIEW__);
      return;
    }
    if (!window.__qsvDataSelect(id, false)) {
      if (window.__qsvDataNote) window.__qsvDataNote(__QSVI18N_ROW_NOT_IN_PREVIEW__);
      return;
    }
    // Page to the row, but only if it survives the drawer's current filters. When it does not,
    // the selection still shows in the "N rows selected" count — the user's filters are NEVER
    // cleared to reveal it.
    if (window.__qsvDataPageTo && !window.__qsvDataPageTo(id) && window.__qsvDataNote) {
      window.__qsvDataNote(__QSVI18N_ROW_FILTERED_OUT__);
    }
  }
  function hook() {
    var tries = hook.tries || 0;
    var gds = document.querySelectorAll('#qsv-viz-smart-grid, [id^="qsv-viz-panel-"]');
    var pending = false;
    gds.forEach(function (gd) {
      if (gd.__qsvSelHooked || gd.__qsvSelSkip) return;
      if (!gd.on || (!gd.__qsvFs && tries < 100)) { pending = true; return; }
      // decided once the panel has rendered: a panel with no id-bearing point trace is marked
      // skipped so it never keeps the poll alive
      if (!hasPointTrace(gd)) { gd.__qsvSelSkip = true; return; }
      gd.__qsvSelHooked = true;
      buildIndex(gd);
      bindClickSettled(gd);
      mirrorCamera(gd);
      // Re-arm on our OWN triggers rather than trusting the theme toggle to call the rehook it
      // publishes. Two observed failure modes make that dependency unreliable: the toggle wraps
      // its whole per-panel body in `try {} catch {}`, so an earlier helper throwing (seen on a
      // map with no cluster toggle) silently skips every rehook AFTER the newPlot has already
      // detached the listeners; and even when the rehook does run, the newPlot promise can
      // resolve before the emitter is final. These two DOM listeners live on the toggle button
      // and on `document`, which plotly never re-renders, so they always survive.
      if (!gd.__qsvSelWatch) {
        gd.__qsvSelWatch = true;
        var rearm = function () {
          bindClickSettled(gd);
          // a re-render hands out a fresh GL map, so the camera mirror has to be re-attached too
          mirrorCamera(gd);
          // The re-render repaints from gd.data, which resets BOTH the selection styling and the
          // pin's coordinates — the pin ships empty, so a re-render empties it again. Re-assert
          // the pair once it has settled.
          if (current.length) setTimeout(function () { restore(gd); }, 1300);
        };
        var tb = document.getElementById("qsv-theme-toggle");
        if (tb) tb.addEventListener("click", rearm);
        document.addEventListener("fullscreenchange", rearm);
      }
      // a re-render drops both the listener above and the painted selection; restoring it here
      // is what makes a theme flip mid-selection look seamless
      if (current.length) restore(gd);
    });
    hook.tries = tries + 1;
    if (pending && hook.tries < 200) setTimeout(hook, 100);
  }
  window.__qsvSelRehook = hook;
  hook();
})();
</script>
"##;

/// Localize + parameterize `CHORO_FILTER_CHROME`. Same contract as [`map_select_chrome`]: bare
/// placeholders, unconditional substitution, `js_string_literal` for quoting + escaping. The
/// applied-filter toast keeps a `__QSVREGION__` sentinel INSIDE the translated string; the JS
/// side substitutes the clicked region id into it at toast time. The feature-id -> raw-spellings
/// map is serialized as a JS object literal with the same `&`/`<`/`>` escape trio the other
/// inline JSON payloads use, so no cell value can smuggle a `</script>` into the page.
fn choro_filter_chrome(region_col: usize, region_raws: &HashMap<String, Vec<String>>) -> String {
    let raws_json = serde_json::to_string(region_raws)
        .unwrap_or_else(|_| "{}".to_string())
        .replace('&', "\\u0026")
        .replace('<', "\\u003c")
        .replace('>', "\\u003e");
    CHORO_FILTER_CHROME
        .replace("__QSV_REGION_COL__", &region_col.to_string())
        .replace("__QSV_REGION_MAP__", &raws_json)
        .replace(
            "__QSVI18N_REGION_FILTER_APPLIED__",
            &js_string_literal(&t!(
                "viz.map.region_filter_applied",
                q_region = "__QSVREGION__"
            )),
        )
        .replace(
            "__QSVI18N_REGION_FILTER_CLEARED__",
            &js_string_literal(&t!("viz.map.region_filter_cleared")),
        )
}

/// Region-click -> data-viewer filter bridge, emitted only when BOTH exist (a summary choropleth
/// panel carrying a real CSV region column, and the drawer chrome).
///
/// A click on a region while the drawer is open filters the table to that region's rows through
/// SearchBuilder's public `rebuild()` API — NOT `ColumnControl`, which exposes no way to SET a
/// selection (only `searchClear()`), and not a bare `column().search()`, which would filter
/// invisibly. Routing through SearchBuilder makes the filter *visible, editable state*: the
/// criterion shows in the "Advanced Filter (n)" popover where the reader can inspect, edit or
/// remove it, its count flows into Clear Filters via the drawer's `filterChanged` hook, and the
/// drawer's existing clear path (`searchBuilder.rebuild({})`) resets it with everything else.
///
/// The clicked `location` is the MATCHED GeoJSON feature id, not necessarily the raw cell value
/// (`match_region_code` zero-pads short numeric codes and case-folds), so the criterion values
/// are the embedded feature-id -> raw-spellings map entries UNIONED with the id itself (a
/// column can hold both spellings of the same region). A multi-spelling region becomes an OR
/// sub-group.
///
/// The summary choropleth panel is found by its trace `meta` carrying the region column index
/// (set in `inline_panel_plot_choropleth{,_map}`); hooking follows the `MAP_SELECT_CHROME`
/// idiom (poll until rendered, wait for `__qsvFs`, token-retired click bindings, re-arm on the
/// theme toggle and fullscreen), and the drawer is reached only through its public seam.
const CHORO_FILTER_CHROME: &str = r##"<script>
(function () {
  var COL = __QSV_REGION_COL__;
  var RAWS = __QSV_REGION_MAP__;
  var MSG_APPLIED = __QSVI18N_REGION_FILTER_APPLIED__;
  var MSG_CLEARED = __QSVI18N_REGION_FILTER_CLEARED__;
  function isRegionTrace(t) {
    return !!(t && (t.type === "choropleth" || t.type === "choroplethmap") && t.meta === COL);
  }
  // The criterion we inserted last, in NORMALIZED form — its identity for the toggle and
  // replace logic. getDetails() hands criteria back enriched with extra properties (observed:
  // `type: "num"`), so raw JSON equality on the inserted object never matches; the comparison
  // strips down to the shape we authored. A reader hand-editing the criterion in the Advanced
  // Filter popover changes that shape, so an edited criterion stops being ours and is left alone.
  var appliedLoc = null;
  var appliedKey = null;
  function normCrit(c) {
    if (!c) return null;
    if (c.logic) return { logic: c.logic, criteria: (c.criteria || []).map(normCrit) };
    return { condition: c.condition, data: c.data, value: c.value };
  }
  function keyOf(c) { return JSON.stringify(normCrit(c)); }
  function critFor(title, loc) {
    // UNION the canonical feature id with the variant spellings, never one or the other: a
    // column can hold BOTH "03103" and "3103" for the same region, and the map only records
    // the non-canonical spellings, so RAWS[loc] alone would drop the canonically-spelled rows
    var raws = RAWS[loc] ? [loc].concat(RAWS[loc]) : [loc];
    var mk = function (v) { return { condition: "=", data: title, value: [v] }; };
    return raws.length === 1 ? mk(raws[0]) : { logic: "OR", criteria: raws.map(mk) };
  }
  function apply(dt, loc) {
    if (!dt.searchBuilder) return;
    var title = dt.column(COL).title();
    var d = dt.searchBuilder.getDetails() || {};
    var before = d.criteria || [];
    var crit = before.filter(function (c) { return keyOf(c) !== appliedKey; });
    // same region, and our criterion was actually still standing -> this click toggles it OFF
    var toggledOff = appliedLoc === loc && crit.length !== before.length;
    if (toggledOff) {
      appliedLoc = null;
      appliedKey = null;
    } else {
      // a top-level OR group would change meaning when our criterion is ANDed beside it;
      // demote the reader's group one level so both filters keep their semantics
      if (crit.length > 1 && d.logic === "OR") crit = [{ logic: "OR", criteria: crit }];
      var ours = critFor(title, loc);
      appliedKey = keyOf(ours);
      appliedLoc = loc;
      crit.push(ours);
    }
    dt.searchBuilder.rebuild(crit.length ? { logic: "AND", criteria: crit } : {});
    // a programmatic rebuild() does NOT fire SearchBuilder's filterChanged hook (observed), so
    // the drawer's Clear Filters count would go stale; hand it the new count over the seam
    document.dispatchEvent(new CustomEvent("qsv-data-sb-changed", { detail: { count: crit.length } }));
    if (window.__qsvDataNote) {
      window.__qsvDataNote(toggledOff ? MSG_CLEARED : MSG_APPLIED.replace("__QSVREGION__", loc));
    }
  }
  // the drawer builds its DataTable lazily on first open; with the drawer open the instance
  // normally exists already, but the ready event covers the first-open race
  function withDT(fn) {
    if (window.__qsvDataDT) { fn(window.__qsvDataDT); return; }
    document.addEventListener("qsv-data-dt-ready", function () {
      if (window.__qsvDataDT) fn(window.__qsvDataDT);
    }, { once: true });
  }
  function onClick(gd, ev) {
    // with the drawer closed there is no table in sight to filter; the click keeps its old
    // meaning (hover/zoom only)
    if (!document.body.classList.contains("qsv-data-open")) return;
    var p = ev && ev.points && ev.points[0];
    if (!p) return;
    var t = gd.data[p.curveNumber];
    if (!isRegionTrace(t)) return;
    // `location` is the matched feature id; fall back through pointNumber for a trace variant
    // that reports the index only
    var loc = p.location != null ? String(p.location)
      : (t.locations && p.pointNumber != null && t.locations[p.pointNumber] != null
        ? String(t.locations[p.pointNumber]) : "");
    if (!loc) return;
    withDT(function (dt) { apply(dt, loc); });
  }
  // `Plotly.newPlot` detaches listeners on a deferred pass (see `bindClick` in the rows<->map
  // bridge); same token idiom so exactly one binding fires.
  function bindClick(gd) {
    var token = (gd.__qsvChoroToken || 0) + 1;
    gd.__qsvChoroToken = token;
    gd.on("plotly_click", function (ev) {
      if (gd.__qsvChoroToken !== token) return;
      onClick(gd, ev);
    });
  }
  function bindClickSettled(gd) {
    bindClick(gd);
    setTimeout(function () { bindClick(gd); }, 250);
    setTimeout(function () { bindClick(gd); }, 1200);
  }
  function hook() {
    var tries = hook.tries || 0;
    var gds = document.querySelectorAll('[id^="qsv-viz-panel-"]');
    var pending = false;
    gds.forEach(function (gd) {
      if (gd.__qsvChoroHooked || gd.__qsvChoroSkip) return;
      if (!gd.on || (!gd.__qsvFs && tries < 100)) { pending = true; return; }
      // decided once the panel has rendered: a panel with no region-column choropleth trace is
      // marked skipped so it never keeps the poll alive
      if (!(gd.data && gd.data.some(isRegionTrace))) { gd.__qsvChoroSkip = true; return; }
      gd.__qsvChoroHooked = true;
      bindClickSettled(gd);
      // re-arm on our OWN triggers rather than trusting the theme toggle to call the rehook
      // (same two failure modes the rows<->map bridge documents)
      if (!gd.__qsvChoroWatch) {
        gd.__qsvChoroWatch = true;
        var rearm = function () { bindClickSettled(gd); };
        var tb = document.getElementById("qsv-theme-toggle");
        if (tb) tb.addEventListener("click", rearm);
        document.addEventListener("fullscreenchange", rearm);
      }
    });
    hook.tries = tries + 1;
    if (pending && hook.tries < 200) setTimeout(hook, 100);
  }
  window.__qsvChoroRehook = hook;
  hook();
})();
</script>
"##;

fn smart_html_page(
    title_text: &str,
    theme: Option<BuiltinTheme>,
    extra_style: &str,
    body: &str,
    show_heading: bool,
    dict_page: Option<&str>,
    meta_table: Option<&str>,
    photos: bool,
    data_chrome: Option<&str>,
    has_basemap: bool,
    // `Some((lat_col, lon_col))` when a `Map`/`Geo` panel carries per-point row ordinals, so the
    // rows<->points bridge is worth emitting. The columns are the resolved coordinate pair, which
    // the bridge uses to pin a selected row that has no plotted point. Still gated on the drawer
    // actually being present (see `map_select_chrome`).
    map_select: Option<(usize, usize)>,
    // `Some((region_col, feature_id -> raw spellings))` when a summary choropleth carries a real
    // CSV region column, so the region-click -> data-viewer SearchBuilder filter is worth
    // emitting. Still gated on the drawer actually being present (see `choro_filter_chrome`).
    choro_filter: Option<(usize, &HashMap<String, Vec<String>>)>,
) -> String {
    let js = plotly_js_block();
    let meta_table = meta_table.unwrap_or("");
    // data viewer drawer chrome (issue #4283): empty unless the metadata Rows link is rendered,
    // so dashboards without the viewer stay byte-identical.
    let data_chrome = data_chrome.unwrap_or("");
    // Attribution for the embedded plotly.js / DataTables / basemap tiles. `data_chrome` being
    // non-empty is exactly the condition under which the DataTables bundle is embedded, so it
    // doubles as the "is DataTables in this file" test.
    let tp_comment = third_party_comment(!data_chrome.is_empty(), has_basemap);
    let tp_style = third_party_footer_style();
    let tp_footer = third_party_footer(!data_chrome.is_empty());
    let title = html_escape(title_text);
    let ToggleChrome {
        style: toggle_style,
        button,
        script,
    } = toggle_chrome(theme);
    let logo = logo_markup();
    // Publish the assumed map pixel size the smart-panel MapLibre zoom was baked against, so
    // the client can re-fit map zoom to the real rendered size (see `assumed_map_dims_prelude`).
    let fs_prelude =
        assumed_map_dims_prelude(MAP_PANEL_ASSUMED_WIDTH_PX, MAP_PANEL_USABLE_HEIGHT_PX);
    // The inline-div grid has no overall plot title (panels carry only their own), so it shows the
    // Data Schematic title as a page `<h1>`. The typed-`Plot` grid already bakes the Data Schematic
    // title into its layout (needed for static image export), so its wrapper suppresses the
    // `<h1>` to avoid rendering the title twice. The `<title>` (browser tab) is always set
    // either way.
    let heading = if show_heading {
        format!("<h1 class=\"qsv-viz-title\">{title}</h1>")
    } else {
        String::new()
    };
    // --dict-info chrome: the "Data Dictionary" link under the title (rendered whether or not the
    // page shows an <h1>, since the typed-grid page bakes its title into the plot) plus the
    // embedded dictionary document + `qsvOpenDict` script. All empty when --dict-info is off, so
    // those dashboards stay byte-identical.
    let dict_link_text = t!("viz.dict.title");
    let (dict_link, dict_chrome) = match dict_page {
        Some(page) => (
            format!(
                "<div class=\"qsv-viz-dict-link\"><a href=\"#\" onclick=\"return \
                 qsvOpenDict('')\">{dict_link_text}{DICT_LINK_ICON}</a></div>"
            ),
            format!(
                "<script type=\"text/html\" id=\"qsv-dict-src\">{page}</script>\n{}",
                // key on title AND dictionary content: two dashboards can share a title (e.g.
                // same file basename in different directories), and qsvOpenDict skips
                // rewriting an existing dict tab — a title-only key would then show the
                // FIRST Data Schematic's dictionary for both. Hashing the page in keeps the tab
                // per-dictionary while a same-content reload still reuses it.
                dict_script_template().replace(
                    "__QSVDICTKEY__",
                    &dict_short_hash(&format!("{title_text}\n{page}"))
                )
            ),
        ),
        None => (String::new(), String::new()),
    };
    // --photos chrome: empty (so no off-origin URL is ever referenced) unless the flag was given
    // AND a map panel actually embedded per-point image URLs.
    let photo_chrome = if photos {
        photo_chrome()
    } else {
        String::new()
    };
    // rows<->points bridge: needs BOTH ends to exist. `map_select` says the map carries row
    // ordinals; a non-empty `data_chrome` is exactly the "drawer is on this page" test (same
    // condition the DataTables bundle is embedded under).
    let map_select_chrome = if let Some((lat_col, lon_col)) = map_select
        && !data_chrome.is_empty()
    {
        map_select_chrome(lat_col, lon_col)
    } else {
        String::new()
    };
    // region-click -> drawer filter bridge: needs BOTH ends to exist. `choro_filter` says a
    // summary choropleth panel carries a real CSV region column; a non-empty `data_chrome` is
    // the same "drawer is on this page" test the rows<->points bridge uses.
    let choro_filter_chrome = if let Some((region_col, region_raws)) = choro_filter
        && !data_chrome.is_empty()
    {
        choro_filter_chrome(region_col, region_raws)
    } else {
        String::new()
    };
    // A RAW-string template (actual newlines, not `\n` escapes) so rustfmt's `format_strings` can't
    // split an escape across a line wrap and corrupt the output — it once mangled `\n{script}` into
    // a stray `\` + `n` in every page. `format!` still doubles the literal CSS braces (`{{`/`}}`)
    // and substitutes each placeholder in a single pass. The `#qsv-logo` rules CSS-swap the two
    // logo variants off `body.qsv-dark`; the logo itself rides in the `.qsv-page-foot` flow row.
    let ui_lang = viz_i18n::active_locale().bcp47;
    let fullscreen_script = fullscreen_script();
    let parcats_script = parcats_order_script();
    format!(
        r#"<!doctype html>
{tp_comment}
<html lang="{ui_lang}">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>{title}</title>
{js}
<style>
{tp_style}
  body {{ font-family: {FONT_FAMILY}; color: var(--qsv-page-ink); background: var(--qsv-page-bg); margin: 0; padding: 16px; }}
  h1.qsv-viz-title {{ font-size: 20px; font-weight: 600; text-align: center; margin: 8px 0 20px; }}
  .qsv-viz-geo-meta {{ font-size: 13px; color: var(--qsv-geo-meta); text-align: center; padding: 8px 4px 4px; }}
  .qsv-viz-dict-link {{ font-size: 13px; text-align: center; margin: -12px 0 16px; }}
  /* A bordered pill rather than the muted caption color the geo meta line uses: this is the only
     way into the dictionary, and as plain --qsv-geo-meta text it read as a caption, not a link. */
  .qsv-viz-dict-link a, .qsv-viz-dict-link a:visited {{ display: inline-flex; align-items: center; gap: 6px; padding: 3px 12px; border: 1px solid var(--qsv-link); border-radius: 999px; color: var(--qsv-link); font-weight: 600; text-decoration: none; }}
  .qsv-viz-dict-link a:hover, .qsv-viz-dict-link a:focus-visible {{ color: var(--qsv-link-hover); border-color: var(--qsv-link-hover); background: color-mix(in srgb, var(--qsv-link) 14%, transparent); }}
  /* sized in em so each glyph tracks the font-size of the link it suffixes */
  .qsv-link-icon {{ width: 1.05em; height: 1.05em; flex: none; vertical-align: -0.15em; }}
  table.qsv-viz-meta {{ margin: 0 auto 20px; border-collapse: collapse; font-size: 13px; }}
  table.qsv-viz-meta td {{ padding: 2px 10px; vertical-align: top; }}
  table.qsv-viz-meta td.qsv-viz-meta-k {{ color: var(--qsv-geo-meta); text-align: right; white-space: nowrap; }}
  /* One flow row at the foot of the page: credits centered, Theme toggle + logo right-aligned.
     Keeping all three in the footer (rather than pinning the toggle + logo bottom-right) is what
     lets the page end right after the credits, and it drops the drawer-offset rules both fixed
     widgets used to need.
     A 1fr/auto/1fr GRID, not a flex row: the credits must be centered on the PAGE, and in a flex
     row they would instead center in the space the right-hand group leaves over — visibly
     off-center. The empty first column mirrors the third, so column 2 lands on the true midpoint
     regardless of how wide the toggle + logo are. Below 700px the columns stack and center. */
  .qsv-page-foot {{ display: grid; grid-template-columns: 1fr auto 1fr; align-items: center; gap: 8px 16px; padding: 4px 16px 12px; }}
  .qsv-page-foot #qsv-credits {{ grid-column: 2; padding: 0; }}
  .qsv-page-foot-right {{ grid-column: 3; display: flex; align-items: center; justify-content: flex-end; gap: 12px; }}
  @media (max-width: 700px) {{ .qsv-page-foot {{ grid-template-columns: 1fr; }} .qsv-page-foot #qsv-credits, .qsv-page-foot-right {{ grid-column: 1; }} .qsv-page-foot-right {{ justify-content: center; }} }}
  #qsv-logo {{ opacity: 0.95; line-height: 0; }}
  #qsv-logo:hover {{ opacity: 1; }}
  /* Theme-aware halo so the logo stays legible on any paper color: a faint dark
     outline on light themes, a light outline under body.qsv-dark so the navy
     shield reads against near-black dark-theme paper. Keyed on body.qsv-dark, so
     it follows the runtime Theme toggle, not just the initial --theme. */
  #qsv-logo img {{ height: 28px; width: auto; display: block; filter: drop-shadow(0 0 2px rgba(0, 0, 0, 0.45)); }}
  #qsv-logo .qsv-logo-dark {{ display: none; }}
  body.qsv-dark #qsv-logo .qsv-logo-light {{ display: none; }}
  body.qsv-dark #qsv-logo .qsv-logo-dark {{ display: block; }}
  body.qsv-dark #qsv-logo img {{ filter: drop-shadow(0 0 2px rgba(255, 255, 255, 0.6)) drop-shadow(0 0 1px rgba(255, 255, 255, 0.45)); }}
{toggle_style}
{FULLSCREEN_STYLE}
{extra_style}
</style>
</head>
<body>
{heading}
{dict_link}
{meta_table}
{body}
{fs_prelude}
{fullscreen_script}
{parcats_script}
{KPI_REWRAP_SCRIPT}
{dict_chrome}
{data_chrome}
{photo_chrome}
{map_select_chrome}
{choro_filter_chrome}
{script}
<div class="qsv-page-foot">{tp_footer}<div class="qsv-page-foot-right">{button}{logo}</div></div>
</body>
</html>
"#
    )
}

/// The embedded plotly.js `<script>` tag for `viz smart` HTML pages, WITHOUT the MathJax
/// (tex-svg) bundle that `Plot::offline_js_sources` also embeds.
///
/// `offline_js_sources` emits two consecutive `<script>` tags — plotly.js (~4.4MB) first, then
/// the tex-svg MathJax bundle (~2.0MB). Smart dashboards only ever render plain-text titles and
/// labels (column names + stat-derived strings), never LaTeX `$...$`, and plotly guards its only
/// MathJax use behind `typeof MathJax != "undefined"`, so the bundle is dead weight here —
/// dropping it shrinks each self-contained Data Schematic by ~31% with no visual change.
///
/// Neither minified payload contains the literal `</script>`, so cutting after the first close
/// tag cleanly isolates the plotly.js tag. Guarded by a `Plotly` marker check (present only in
/// plotly.js, never in tex-svg): if a future plotly release reorders the two tags, we fall back
/// to the full bundle rather than risk stripping plotly.js itself.
fn plotly_js_only() -> String {
    const CLOSE: &str = "</script>";
    let full = Plot::offline_js_sources();
    if let Some(idx) = full.find(CLOSE) {
        let first = &full[..idx + CLOSE.len()];
        if first.contains("Plotly") {
            return first.to_string();
        }
    }
    // layout changed unexpectedly — keep the full (correct, if heavier) bundle.
    full
}

/// Fallback plotly.js CDN tag, used only if `Plot::online_cdn_js` stops emitting a recognizable
/// `cdn.plot.ly` script tag. Keep the version in sync with the crate's vendored
/// `resource/plotly.min.js` (the bundle `plotly_js_only` would otherwise inline).
const PLOTLY_CDN_FALLBACK: &str = concat!(
    r#"<script src="https://cdn.plot.ly/plotly-"#,
    "3.7.0",
    r#".min.js" charset="utf-8" integrity=""#,
    "sha384-l4G4qURPwALv583BKlynU/4LiUx0rk9jcTX58Aq+Jc+jgWb52zeH2geZkSS3UPPs",
    r#"" crossorigin="anonymous"></script>"#
);

/// The plotly.js version this file's CDN integrity hash was computed for, and that hash.
///
/// Under `QSV_VIZ_CDN=1` the emitted page executes plotly.js straight from `cdn.plot.ly`, so
/// without Subresource Integrity a compromised or tampered CDN response is arbitrary script
/// execution in every viewer of a published Data Schematic. The URL is version-pinned and therefore
/// immutable, which is what makes a baked hash safe.
///
/// Verified: the digest below is the sha384 of `https://cdn.plot.ly/plotly-3.7.0.min.js`, which is
/// both the file `Plot::online_cdn_js` points at AND byte-identical to the `resource/plotly.min.js`
/// this build inlines — so the CDN and embedded paths load the same plotly.js, and the embedded
/// bundle is a local oracle for this digest (see `plotly_cdn_sri_matches_the_embedded_bundle`).
///
/// `plotly_cdn_only` applies the hash ONLY when the tag it slices out of `Plot::online_cdn_js`
/// actually points at this version — after a plotly.rs bump the URL changes and an inherited
/// stale hash would block the script instead of protecting it.
/// To regenerate after a plotly.rs bump changes the CDN URL (there is no offline oracle for this
/// — the vendored bundle is a different version, so the digest can only come from the CDN):
///
/// ```text
/// curl -sSL -o /tmp/p.js https://cdn.plot.ly/plotly-<VERSION>.min.js
/// echo "sha384-$(openssl dgst -sha384 -binary /tmp/p.js | openssl base64 -A)"
/// ```
const PLOTLY_CDN_VERSION: &str = "3.7.0";
const PLOTLY_CDN_SRI: &str =
    "sha384-l4G4qURPwALv583BKlynU/4LiUx0rk9jcTX58Aq+Jc+jgWb52zeH2geZkSS3UPPs";

/// The plotly.js CDN `<script src>` tag, sliced out of `Plot::online_cdn_js` so the pinned
/// version can never drift from the bundle we would otherwise embed.
///
/// `online_cdn_js` emits two tags — MathJax (tex-svg, from jsDelivr) first, then plotly.js. We
/// want only the latter: `viz` never renders LaTeX (see `plotly_js_only`), and plotly guards its
/// only MathJax use behind a `typeof MathJax != "undefined"` probe.
fn plotly_cdn_only() -> String {
    const OPEN: &str = "<script src=\"https://cdn.plot.ly/";
    const CLOSE: &str = "</script>";
    let full = Plot::online_cdn_js();
    if let Some(start) = full.find(OPEN)
        && let Some(end) = full[start..].find(CLOSE)
    {
        let tag = &full[start..start + end + CLOSE.len()];
        // add integrity only when this really is the version the hash was computed for; a
        // plotly.rs bump changes the URL, and a stale hash would block the script outright
        let versioned = format!("plotly-{PLOTLY_CDN_VERSION}.min.js");
        if tag.contains(&versioned)
            && !tag.contains("integrity=")
            && let Some(close_angle) = tag.rfind("></script>")
        {
            return format!(
                "{} integrity=\"{PLOTLY_CDN_SRI}\" crossorigin=\"anonymous\"{}",
                &tag[..close_angle],
                &tag[close_angle..]
            );
        }
        return tag.to_string();
    }
    // shape changed unexpectedly — fall back to our own pinned tag rather than emit the
    // MathJax tag (or nothing) and leave `Plotly` undefined.
    PLOTLY_CDN_FALLBACK.to_string()
}

/// Whether generated HTML embeds compressed payloads: the plotly.js bundle and large map-panel
/// figure JSON as gzip+base64 (inflated in-browser via the native `DecompressionStream` API),
/// and map/geo per-point arrays as base64 float32 typed arrays. On by default — it cuts the
/// fixed plotly.js cost from ~4.8 MB to ~1.9 MB per file and shrinks map payloads 3-4x.
/// `QSV_VIZ_NO_COMPRESS=1` restores fully readable plain-text HTML: needed for pre-2023 browsers
/// (no `DecompressionStream`) and for strict-CSP embeds (the bootstrap installs plotly.js via
/// `eval`, which requires `unsafe-eval`), and handy when grepping/debugging the emitted HTML.
fn viz_compress() -> bool {
    !util::get_envvar_flag("QSV_VIZ_NO_COMPRESS")
}

/// Whether generated HTML loads plotly.js from the CDN (`<script src>`) instead of embedding it.
/// Off by default: self-contained, offline-capable output is the documented behavior. Setting
/// `QSV_VIZ_CDN=1` drops the ~1.9 MB embedded bundle to a ~100-byte tag, at the cost of needing
/// network access when the page is *viewed* — the right trade for pages published to the web.
///
/// Governs ONLY the plotly.js bundle. `QSV_VIZ_NO_COMPRESS` independently governs figure-JSON
/// gzip and the `bdata` float32 typed arrays, and the two compose freely (CDN plotly.js is 3.x,
/// which decodes `bdata` natively).
fn viz_cdn() -> bool {
    util::get_envvar_flag("QSV_VIZ_CDN")
}

/// Whether `viz smart --dictionary infer` forces a genuinely FRESH Data Dictionary inference.
/// Off by default: reuse is the cheap, documented behavior.
///
/// Two independent layers cache an inferred dictionary, and this bypasses BOTH — defeating only
/// one leaves the other silently serving the old dictionary:
///   1. the `<stem>.schema.json` sidecar beside the input, normally reused as-is (skipping the LLM
///      entirely), and
///   2. describegpt's own completion cache (`~/.qsv-cache/describegpt`, 28-day TTL), bypassed by
///      passing it `--fresh`.
///
/// Intended for regenerating dictionary-driven artifacts (e.g. the `examples/viz` gallery's
/// `--dictionary infer` dashboards) after a describegpt or prompt-file change. Note a successful
/// re-infer OVERWRITES the sidecar, including a hand-edited one; a failed infer leaves it untouched
/// and soft-falls back to the stats-only Data Schematic.
fn viz_dict_fresh() -> bool {
    util::get_envvar_flag("QSV_VIZ_DICT_FRESH")
}

/// gzip (at `level`) + standard-base64 encode, for `<script type="application/gzip-b64">`
/// payloads. Base64's alphabet has no `<`, so the payload can never contain `</script>` and
/// embeds into HTML with no further escaping. Returns an empty string on failure (callers fall
/// back to the uncompressed form).
fn gzip_b64(data: &[u8], level: flate2::Compression) -> String {
    use std::io::Write;

    let mut enc = flate2::write::GzEncoder::new(Vec::with_capacity(data.len() / 3), level);
    if enc.write_all(data).is_err() {
        return String::new();
    }
    match enc.finish() {
        Ok(bytes) => base64_simd::STANDARD.encode_to_string(&bytes),
        Err(_) => String::new(),
    }
}

/// The inner source of a single `<script ...>...</script>` tag (`None` if the shape is off).
fn script_tag_inner(tag: &str) -> Option<&str> {
    let start = tag.find('>')? + 1;
    let end = tag.rfind("</script>")?;
    (start <= end).then(|| &tag[start..end])
}

/// The plotly.js bundle gzipped at max compression + base64 (~4.84 MB -> ~1.95 MB), computed
/// once per process (gzip -9 of the bundle costs a few hundred ms; every emitted page reuses
/// it). Empty if the bundle tag had an unexpected shape — callers then emit the plain bundle.
static PLOTLY_GZ_B64: std::sync::LazyLock<String> = std::sync::LazyLock::new(|| {
    let tag = plotly_js_only();
    match script_tag_inner(&tag) {
        Some(src) => gzip_b64(src.as_bytes(), flate2::Compression::best()),
        None => String::new(),
    }
});

/// Queue-stub for `window.Plotly`, emitted (in `<head>`) BEFORE the panels' inline
/// `Plotly.newPlot(..)` script tags when the bundle is embedded compressed: the real bundle only
/// exists after an async `DecompressionStream` pass, so until then `newPlot` calls queue verbatim
/// and `PLOTLY_GZ_BOOTSTRAP` replays them in order — the panel scripts themselves need no
/// changes, and once the real global is installed later calls go straight through. The
/// fullscreen-modebar chrome already polls for rendered divs, and the theme toggle re-applies on
/// the `qsv:plotly-ready` event the bootstrap dispatches.
const PLOTLY_STUB_SCRIPT: &str = r#"<script>window.__qsvPlotQ=[];window.Plotly={newPlot:function(){window.__qsvPlotQ.push(arguments)}};</script>"#;

/// Shared inflate helpers for the gzip-embedded payloads. `__qsvGunzip` pumps a payload tag
/// through `DecompressionStream` with an explicit reader loop (`new Response(stream).text()` is
/// unreliable on older Safari). `qsvNewPlotGz` renders one gzip-embedded figure — via the stub's
/// queue when the bundle itself is still inflating. Each completed render dispatches
/// `qsv:plotly-ready` so the theme toggle can re-theme late-rendered panels.
const GZ_PRELUDE_SCRIPT: &str = r#"<script>
(function () {
  // Shared, once-only failure banner. Both the bundle bootstrap and `qsvNewPlotGz` can discover a
  // missing DecompressionStream; under QSV_VIZ_CDN there IS no bootstrap, so a gzipped figure must
  // be able to raise the banner itself rather than leave a silently blank panel.
  window.__qsvFail = function (msg) {
    if (window.__qsvFailed) return;
    window.__qsvFailed = true;
    var show = function () {
      document.body.insertAdjacentHTML("afterbegin",
        '<div style="margin:16px;padding:12px 16px;border:1px solid #c62828;border-radius:6px;background:#fff3f3;color:#b71c1c;font-family:sans-serif">' + msg + "</div>");
    };
    if (document.body) show(); else document.addEventListener("DOMContentLoaded", show);
  };
  window.__qsvNoDecompress = function () {
    window.__qsvFail("This page has gzip-compressed payloads and needs a browser with DecompressionStream support (Chrome/Edge 80+, Firefox 113+, Safari 16.4+). Alternatively, regenerate the file with the QSV_VIZ_NO_COMPRESS=1 environment variable set for a larger, uncompressed version.");
  };
  window.__qsvGunzip = function (el) {
    var bin = atob(el.textContent.trim()), n = bin.length, bytes = new Uint8Array(n);
    for (var i = 0; i < n; i++) bytes[i] = bin.charCodeAt(i);
    var reader = new Blob([bytes]).stream().pipeThrough(new DecompressionStream("gzip")).getReader();
    var dec = new TextDecoder(), out = "";
    return reader.read().then(function pump(r) {
      if (r.done) return out + dec.decode();
      out += dec.decode(r.value, { stream: true });
      return reader.read().then(pump);
    });
  };
  window.__qsvReady = function () { document.dispatchEvent(new Event("qsv:plotly-ready")); };
  window.qsvNewPlotGz = function (id) {
    var el = document.getElementById(id + "-fig");
    if (!el) return;
    // the banner is once-only, so this is a no-op when the bundle bootstrap already raised it
    if (!window.DecompressionStream) { window.__qsvNoDecompress(); return; }
    window.__qsvGunzip(el).then(function (s) {
      el.textContent = "";
      var p = Plotly.newPlot(id, JSON.parse(s));
      // return the render promise so a rejected newPlot lands in the catch below
      if (p && p.then) return p.then(window.__qsvReady);
      window.__qsvReady();
    }).catch(function (e) {
      // still announce readiness so the other panels' theme re-apply isn't held hostage
      console.error("qsv viz: figure payload render failed:", e);
      window.__qsvReady();
    });
  };
})();
</script>"#;

/// Bootstrap that inflates the gzip+base64 plotly.js payload, installs the real `Plotly` global
/// via indirect eval (global scope), replays the stub's queued `newPlot` calls in queue order,
/// and announces readiness. Emitted right after the payload tag in `<head>`, so inflation
/// overlaps the browser's parse of the multi-MB body. A browser without `DecompressionStream`
/// (pre-2023) gets an explanatory banner instead of a blank page.
///
/// The replay deliberately does NOT chain each panel behind the previous panel's render promise:
/// a MapLibre panel's `newPlot` promise can stay pending forever (map style/tile load never
/// resolves it), and a sequential `then` chain would stall every later panel behind it (#4155).
/// Firing the calls independently mirrors the uncompressed page, where each panel's inline
/// script invokes `newPlot` as it is parsed.
const PLOTLY_GZ_BOOTSTRAP: &str = r##"<script>
(function () {
  // `__qsvFail`/`__qsvNoDecompress` come from GZ_PRELUDE_SCRIPT, always emitted before this tag
  var fail = window.__qsvFail;
  if (!window.DecompressionStream) {
    window.__qsvNoDecompress();
    return;
  }
  window.__qsvGunzip(document.getElementById("qsv-plotly-gz")).then(function (src) {
    var q = window.__qsvPlotQ || [];
    window.__qsvPlotQ = null;
    delete window.Plotly;
    (0, eval)(src);__QSVI18N_PLOTLY_SETLOCALE__
    q.forEach(function (args) {
      // per-panel isolation: one failed (or forever-pending) render must not block the rest
      try {
        var p = Plotly.newPlot.apply(Plotly, args);
        // per-panel settle -> ready, so a stuck panel can't hold the theme re-apply hostage
        if (p && p.then) p.then(window.__qsvReady, function (e) {
          console.error("qsv viz: queued panel render failed:", e);
          window.__qsvReady();
        });
      } catch (e) {
        console.error("qsv viz: queued panel render failed:", e);
      }
    });
    // readiness floor: fires even if every queued panel's promise stays pending
    window.__qsvReady();
  }).catch(function (e) { fail("Failed to inflate the embedded plotly.js bundle: " + e); });
})();
</script>"##;

/// The plotly.js embed block for generated HTML pages: under `QSV_VIZ_CDN` a bare CDN
/// `<script src>` tag; under `QSV_VIZ_NO_COMPRESS` (or if compression failed) the plain inline
/// bundle from `plotly_js_only`; otherwise the queue stub + inflate helpers + gzip payload +
/// bootstrap, ~2.9 MB smaller per page.
///
/// Never emits the MathJax (tex-svg) bundle in any mode — see `plotly_js_only`.
/// The vendored plotly.js locale bundle for the active language, or empty for English.
///
/// The file self-registers: `typeof Plotly === "undefined" ? window.PlotlyLocales.push(locale) :
/// Plotly.register(locale)`. That makes it safe to emit BEFORE the bundle (the deferred
/// gzip path) and after it (CDN / plain inline) alike.
///
/// Registering a locale does not select it — `Plotly.setPlotConfig({locale})` does, and it only
/// affects renders that come after. See [`plotly_setlocale_js`] for the two placements.
fn plotly_locale_block() -> String {
    let row = viz_i18n::active_locale();
    match row.bcp47 {
        // English is plotly's built-in default: nothing to register, nothing to select, so an
        // English page is byte-identical to one built before viz learned to localize.
        "en" => String::new(),
        "es" => format!(
            "<script>{}</script>",
            include_str!("assets/i18n/plotly/plotly-locale-es.js")
        ),
        "fr" => format!(
            "<script>{}</script>",
            include_str!("assets/i18n/plotly/plotly-locale-fr.js")
        ),
        "de" => format!(
            "<script>{}</script>",
            include_str!("assets/i18n/plotly/plotly-locale-de.js")
        ),
        "it" => format!(
            "<script>{}</script>",
            include_str!("assets/i18n/plotly/plotly-locale-it.js")
        ),
        // the vendored file's own id is "pt-BR", which is why the LOCALES row uses that tag
        // rather than "pt" -- `plotly_setlocale_js` must ask for exactly what was registered.
        "pt-BR" => format!(
            "<script>{}</script>",
            include_str!("assets/i18n/plotly/plotly-locale-pt-BR.js")
        ),
        "ja" => format!(
            "<script>{}</script>",
            include_str!("assets/i18n/plotly/plotly-locale-ja.js")
        ),
        // as with pt-BR, the vendored file's own id is "zh-CN" -- the tag must match it exactly.
        "zh-CN" => format!(
            "<script>{}</script>",
            include_str!("assets/i18n/plotly/plotly-locale-zh-CN.js")
        ),
        // A curated language whose plotly locale has not been vendored yet: the chrome is still
        // translated, only the modebar tooltips stay English.
        _ => String::new(),
    }
}

/// `Plotly.setPlotConfig({locale})` for the active language, or empty for English.
///
/// Emitted in TWO places because plotly arrives on two different schedules, and the call must
/// land after `Plotly` exists but before any `newPlot`:
///  * CDN / plain-inline: `Plotly` is defined by the time the next `<script>` runs, so the call
///    goes straight after the library block;
///  * gzip-inline: the real global only appears when `PLOTLY_GZ_BOOTSTRAP` evals the inflated
///    bundle, and the queued `newPlot` calls are replayed immediately afterwards — so the call is
///    substituted into that bootstrap between the eval and the replay. Anywhere later (e.g. the
///    `qsv:plotly-ready` event) would be too late: the panels would already have rendered with the
///    English modebar.
fn plotly_setlocale_js() -> String {
    let row = viz_i18n::active_locale();
    if row.bcp47 == "en" || plotly_locale_block().is_empty() {
        return String::new();
    }
    // Leading newline + indent are part of the VALUE, not the template: the placeholder sits
    // flush against `(0, eval)(src);` so that substituting an empty string for English restores
    // the bootstrap byte-for-byte. A placeholder on its own line would leave a stray indented
    // blank line in every English page -- which is exactly what the golden check caught.
    format!(
        "\n    if (window.Plotly && Plotly.setPlotConfig) Plotly.setPlotConfig({{locale: {}}});",
        js_string_literal(row.bcp47)
    )
}

fn plotly_js_block() -> String {
    if viz_cdn() {
        // A classic (non-deferred, non-module) `<script src>` runs to completion before any
        // panel's `Plotly.newPlot` script, so `Plotly` is already defined: no `PLOTLY_STUB_SCRIPT`
        // queue and no `PLOTLY_GZ_BOOTSTRAP` are needed. `GZ_PRELUDE_SCRIPT` is still required
        // whenever figure payloads are gzipped, since `map_panel_inline_html` emits
        // `qsvNewPlotGz(..)` calls that need `__qsvGunzip`/`__qsvReady`.
        let cdn = plotly_cdn_only();
        let loc = plotly_locale_suffix();
        return if viz_compress() {
            format!("{cdn}\n{GZ_PRELUDE_SCRIPT}{loc}")
        } else {
            format!("{cdn}{loc}")
        };
    }
    let plain = plotly_js_only();
    let loc = plotly_locale_suffix();
    if !viz_compress() {
        return format!("{plain}{loc}");
    }
    let b64 = PLOTLY_GZ_B64.as_str();
    if b64.is_empty() {
        return format!("{plain}{loc}");
    }
    // Deferred path: the locale bundle self-registers into `window.PlotlyLocales` before the real
    // Plotly exists, and the SELECTION is spliced into the bootstrap (between eval and replay).
    let bootstrap =
        PLOTLY_GZ_BOOTSTRAP.replace("__QSVI18N_PLOTLY_SETLOCALE__", &plotly_setlocale_js());
    // The separator is part of the VALUE so that English (empty block) reproduces the original
    // byte sequence exactly -- a literal `\n{}` here would add a stray blank line to every
    // English page. Same trap as the bootstrap placeholder above; the golden check caught both.
    // The locale bundle MUST precede `PLOTLY_STUB_SCRIPT`. Its self-registration branches on
    // `typeof Plotly === "undefined"`: emitted after the stub it sees the stub's `window.Plotly`,
    // calls `register()` on a queue object that discards it, and the dictionary is lost when the
    // bootstrap deletes the stub — `setPlotConfig` still reports locale "es" while every built-in
    // modebar tooltip stays English. Emitted first, `Plotly` really is undefined, so it queues
    // into `window.PlotlyLocales`, which the real bundle drains on load. (Found only by opening
    // the page: every static check — file embedded, setPlotConfig present, correct ordering —
    // passed while the feature did nothing.)
    let locale_block = match plotly_locale_block() {
        b if b.is_empty() => String::new(),
        b => format!("{b}\n"),
    };
    format!(
        "{locale_block}{PLOTLY_STUB_SCRIPT}\n{GZ_PRELUDE_SCRIPT}\n<script id=\"qsv-plotly-gz\" \
         type=\"application/gzip-b64\">{b64}</script>\n{bootstrap}"
    )
}

/// Locale bundle + immediate selection, for the paths where `Plotly` is already defined.
/// Empty for English, so those pages keep their exact previous bytes.
fn plotly_locale_suffix() -> String {
    let block = plotly_locale_block();
    if block.is_empty() {
        return String::new();
    }
    format!("\n{block}\n<script>{}</script>", plotly_setlocale_js())
}

/// Vendored DataTables v3 bundle (core + `ColumnControl` + `DateTime` + SearchBuilder, "DataTables"
/// default styling,
/// minified + concatenated by the DataTables download builder). Powers the `viz smart` data
/// viewer drawer. MIT licensed (Copyright (C) 2008-2026, `SpryMedia` Ltd.) — the full license text
/// and the component inventory live in `assets/LICENSE-DataTables.txt`, and the builder header
/// comment in each file records the exact component versions. The bundle's `/*! ... */` copyright
/// banners MUST be preserved when re-fetching: the MIT license is conditioned on keeping them
/// intact. To rebuild or bump, fetch the concatenated combination the version consts below pin:
///
/// ```text
/// curl -sSL -o src/cmd/assets/datatables.min.js \
///   https://cdn.datatables.net/v/dt/<COMBO>/datatables.min.js
/// curl -sSL -o src/cmd/assets/datatables.min.css \
///   https://cdn.datatables.net/v/dt/<COMBO>/datatables.min.css
/// echo "sha384-$(openssl dgst -sha384 -binary src/cmd/assets/datatables.min.js | openssl base64 -A)"
/// echo "sha384-$(openssl dgst -sha384 -binary src/cmd/assets/datatables.min.css | openssl base64 -A)"
/// ```
///
/// where `<COMBO>` is `DATATABLES_CDN_COMBO`. The CDN serves exactly these vendored bytes at the
/// version-pinned (immutable) URL, so the vendored files are a local oracle for the SRI digests
/// (see `datatables_cdn_sri_matches_the_vendored_bundle`).
const DATATABLES_JS: &str = include_str!("assets/datatables.min.js");
const DATATABLES_CSS: &str = include_str!("assets/datatables.min.css");

/// The download-builder combination the vendored bundle was built from: DataTables core 3.0.1 +
/// Buttons 4.0.1 (for the popover SearchBuilder "Filter" button) + `ColumnControl` 2.0.0 (the
/// in-header per-column search widgets) + `DateTime` 2.0.0 + SearchBuilder 2.0.0, default
/// DataTables styling. Also the path segment of the version-pinned CDN URLs.
///
/// Two components are deliberately ABSENT and must not be added back when re-fetching:
/// - **Responsive**, dropped when horizontal scrolling (`scrollX`) replaced it.
/// - **Select** (`sl-4.0.0`), which would be the obvious way to implement the data viewer's row
///   selection. It is left out on cost/benefit, NOT because it cannot run here. Through the 3.x
///   line it hard-required a global `jQuery` (checked 2.1.0 through 3.1.0, plus the standalone and
///   ESM builds) and silently never registered on this page, which runs DataTables in vanilla mode.
///   **That is fixed in 4.0.0**: its UMD wrapper depends only on `datatables.net`, it registers
///   with `jQuery === undefined`, and `rows({selected: true})` then correctly reports 0 with
///   nothing selected instead of returning every row. Adopting it would still leave the hard parts
///   hand-rolled — the `source` tag that keeps the map bridge from echoing its own selection back,
///   the out-of-range index guard, and `__qsvDataPageTo` — while adding ~21 KB to the bundle and
///   re-testing the rows <-> map cross-link. Row selection therefore stays hand-rolled in
///   `DATA_DRAWER_SCRIPT`; revisit if Select grows a feature that seam needs.
const DATATABLES_CDN_COMBO: &str = "dt-3.0.1/b-4.0.1/cc-2.0.0/date-2.0.0/sb-2.0.0";
const DATATABLES_CDN_JS_SRI: &str =
    "sha384-y0MCk57Zxg695kxqXU1ZYUZvF2Raqy0JvVTqeDkInunnCFj5dvai0y1Fus2IMIJo";
const DATATABLES_CDN_CSS_SRI: &str =
    "sha384-lRGUvwnJvQ6dz9UARbiiTi6QUhuxCGaHkPD7sCILKXraFQOR9P67vWU5zVStDO2R";

/// The DataTables bundle gzipped at max compression + base64 (~300 KB -> ~112 KB b64), computed
/// once per process like `PLOTLY_GZ_B64`. Empty on (never-expected) gzip failure — callers then
/// emit the plain bundle.
static DATATABLES_GZ_B64: std::sync::LazyLock<String> =
    std::sync::LazyLock::new(|| gzip_b64(DATATABLES_JS.as_bytes(), flate2::Compression::best()));

/// Like `DATATABLES_GZ_B64` but for the stylesheet (~74 KB -> ~14 KB b64).
static DATATABLES_CSS_GZ_B64: std::sync::LazyLock<String> =
    std::sync::LazyLock::new(|| gzip_b64(DATATABLES_CSS.as_bytes(), flate2::Compression::best()));

/// The DataTables library block for the data viewer drawer: under `QSV_VIZ_CDN` version-pinned
/// `<link>`/`<script src>` tags with Subresource Integrity; under `QSV_VIZ_NO_COMPRESS` (or on
/// gzip failure) the plain inline bundle; otherwise gzip+base64 payload tags that
/// `DATA_DRAWER_SCRIPT` inflates lazily on first drawer open (zero page-load cost, and the
/// library never executes unless the drawer is actually used).
///
/// The payload elements are dispatched on by id + shape in `DATA_DRAWER_SCRIPT`:
/// `#qsv-data-lib` is a `<script>` holding either gzip-b64 or plain JS source (by `type`), and
/// `#qsv-data-css` is either a real `<style>` (already applied, nothing to do) or a gzip-b64
/// `<script>` to inflate into one. In CDN mode neither payload element exists and
/// `window.DataTable` is already installed by the `<script src>` tag.
fn datatables_lib_block() -> String {
    if viz_cdn() {
        return format!(
            "<link rel=\"stylesheet\" \
             href=\"https://cdn.datatables.net/v/dt/{DATATABLES_CDN_COMBO}/datatables.min.css\" \
             integrity=\"{DATATABLES_CDN_CSS_SRI}\" crossorigin=\"anonymous\">\n\
             <script \
             src=\"https://cdn.datatables.net/v/dt/{DATATABLES_CDN_COMBO}/datatables.min.js\" \
             integrity=\"{DATATABLES_CDN_JS_SRI}\" crossorigin=\"anonymous\"></script>"
        );
    }
    if viz_compress() {
        let js_b64 = DATATABLES_GZ_B64.as_str();
        let css_b64 = DATATABLES_CSS_GZ_B64.as_str();
        if !js_b64.is_empty() && !css_b64.is_empty() {
            return format!(
                "<script id=\"qsv-data-lib\" \
                 type=\"application/gzip-b64\">{js_b64}</script>\n<script id=\"qsv-data-css\" \
                 type=\"application/gzip-b64\">{css_b64}</script>"
            );
        }
    }
    // vendored bundle contains no `</script>`/`</style>` (asserted by unit tests), so plain
    // inline embedding needs no escaping
    format!(
        "<script id=\"qsv-data-lib\" type=\"text/javascript\">{DATATABLES_JS}</script>\n<style \
         id=\"qsv-data-css\">{DATATABLES_CSS}</style>"
    )
}

fn build_layout(
    args: &Args,
    default_x: Option<String>,
    default_y: Option<String>,
) -> CliResult<plotly::Layout> {
    let mut layout = plotly::Layout::new();
    if let Some(title) = &args.flag_title {
        layout = layout.title(Title::with_text(title));
    }
    // x-axis: title (explicit flag, else resolved column label) and/or an optional range-slider
    // navigator strip. Emit the axis when either is set so --rangeslider works even without a
    // title.
    // The FALLBACK is a raw CSV header and axis titles render plotly pseudo-HTML, so it is escaped
    // like every other data-derived axis title (scatter3d 6119, animated bubble 21943). An explicit
    // --x-title/--y-title is operator-supplied and stays raw, matching --title above.
    let x_title = args
        .flag_x_title
        .clone()
        .or_else(|| default_x.as_deref().map(escape_hover));
    if x_title.is_some() || args.flag_rangeslider {
        let mut x = Axis::new();
        if let Some(t) = x_title {
            x = x.title(Title::with_text(t));
        }
        if args.flag_rangeslider {
            x = x.range_slider(RangeSlider::new().visible(true));
        }
        layout = layout.x_axis(x);
    }
    // y-axis: title (flag or column label) and/or an explicit --y-range. Only emit the axis
    // when at least one is set, so the default (autoranging, untitled) case is untouched.
    let y_title = args
        .flag_y_title
        .clone()
        .or_else(|| default_y.as_deref().map(escape_hover));
    if y_title.is_some() || args.flag_y_range.is_some() {
        let mut y = Axis::new();
        if let Some(t) = y_title {
            y = y.title(Title::with_text(t));
        }
        if let Some(r) = &args.flag_y_range {
            let (lo, hi) = parse_y_range(r)?;
            // an explicit range turns autorange off, so out-of-range points clip from view
            y = y.range(vec![lo, hi]);
        }
        layout = layout.y_axis(y);
    }
    if let Some(note) = &args.flag_annotation {
        layout = with_chart_note(layout, note);
    }
    Ok(apply_theme(layout, args.theme()))
}

fn parse_y_range(spec: &str) -> CliResult<(f64, f64)> {
    let parts: Vec<&str> = spec.split(':').collect();
    if parts.len() != 2 {
        return fail_incorrectusage_clierror!(
            "--y-range must be two colon-separated numbers (min:max), e.g. -10:55; got `{spec}`"
        );
    }
    let Ok(lo) = parts[0].trim().parse::<f64>() else {
        return fail_incorrectusage_clierror!(
            "--y-range min `{}` is not a number",
            parts[0].trim()
        );
    };
    let Ok(hi) = parts[1].trim().parse::<f64>() else {
        return fail_incorrectusage_clierror!(
            "--y-range max `{}` is not a number",
            parts[1].trim()
        );
    };
    if !lo.is_finite() || !hi.is_finite() {
        return fail_incorrectusage_clierror!("--y-range values must be finite; got `{spec}`");
    }
    if lo >= hi {
        return fail_incorrectusage_clierror!("--y-range min ({lo}) must be less than max ({hi})");
    }
    Ok((lo, hi))
}

fn output_plot(
    plot: &Plot,
    args: &Args,
    fmt: OutFormat,
    width: usize,
    height: usize,
    scale: f64,
) -> CliResult<()> {
    if matches!(fmt, OutFormat::Html) {
        output_html(plot, args)
    } else {
        output_image(plot, args, fmt, width, height, scale)
    }
}

fn output_html(plot: &Plot, args: &Args) -> CliResult<()> {
    // `Plot::write_html`/`Plot::show` would bypass any post-processing (write straight to disk /
    // open plotly's own temp file), so render to a string, splice in the fullscreen modebar
    // button, and reuse `output_inline_html` for the `--output` / stdout / `--open`-temp-file
    // handling — the same path the `viz smart` dashboards already use.
    let mut html = plot.to_html();
    // Swap plotly's own `<head>` scripts for our block, so single charts get exactly what the
    // `viz smart` dashboards get. The needle is the FULL `offline_js_sources` string (both script
    // tags, verbatim what `Plot::to_html` interpolates), which drops the ~2.1 MB MathJax
    // (tex-svg) bundle along with the swap: `viz` renders only plain-text titles and labels, and
    // plotly guards its sole MathJax use behind a `typeof MathJax != "undefined"` probe. A LaTeX
    // `--title` therefore no longer typesets — which has always been true of smart dashboards.
    // If the fork ever retemplates and the needle misses, leave the (correct, heavier) page be.
    let needle = Plot::offline_js_sources();
    let block = plotly_js_block();
    if block != needle && html.contains(&needle) {
        html = html.replacen(&needle, &block, 1);
    }
    let html = inject_fullscreen_chrome(&html);
    // Single charts never carry the DataTables drawer (a `viz smart` feature); they fetch basemap
    // tiles only for `viz map` and for `viz choropleth --map` (the MapLibre choropleth variant).
    let html = inject_third_party_notice(
        &html,
        false,
        args.cmd_map || (args.cmd_choropleth && args.flag_map),
    );
    output_inline_html(&html, args)
}

// image formats (only reachable with the viz_static feature; guarded in run())
#[cfg(feature = "viz_static")]
fn output_image(
    plot: &Plot,
    args: &Args,
    fmt: OutFormat,
    width: usize,
    height: usize,
    scale: f64,
) -> CliResult<()> {
    let path = args
        .flag_output
        .as_deref()
        .expect("image format without --output should have been rejected");
    plot.write_image(path, image_format(fmt), width, height, scale)
        .map_err(image_export_err)?;
    if args.flag_open {
        open_path(path)?;
    }
    Ok(())
}

#[cfg(not(feature = "viz_static"))]
#[allow(clippy::unnecessary_wraps)]
fn output_image(
    _plot: &Plot,
    _args: &Args,
    _fmt: OutFormat,
    _width: usize,
    _height: usize,
    _scale: f64,
) -> CliResult<()> {
    unreachable!("image export is rejected in run() without the viz_static feature");
}

/// Render a pre-assembled Plotly JSON value (data + layout) to a static image, for `viz smart`
/// dashboards with > `MAX_SUBPLOTS` panels — whose layout carries `xaxis9+`/`yaxis9+` that the
/// typed `Plot` can't express. `StaticExporter::write_fig` accepts an arbitrary JSON value and
/// renders it through the same headless-browser backend as `Plot::write_image`.
#[cfg(feature = "viz_static")]
fn output_image_json(
    value: &serde_json::Value,
    args: &Args,
    fmt: OutFormat,
    width: usize,
    height: usize,
    scale: f64,
) -> CliResult<()> {
    use std::path::Path;

    use plotly::plotly_static::StaticExporterBuilder;

    let path = args
        .flag_output
        .as_deref()
        .expect("image format without --output should have been rejected");
    let mut exporter = StaticExporterBuilder::default()
        .build()
        .map_err(image_export_err)?;
    let result = exporter.write_fig(
        Path::new(path),
        value,
        image_format(fmt),
        width,
        height,
        scale,
    );
    // release the webdriver even if the render failed (a leaked session worsens later exports),
    // mirroring plotly's own `Plot::write_image`.
    exporter.close();
    result.map_err(image_export_err)?;
    if args.flag_open {
        open_path(path)?;
    }
    Ok(())
}

#[cfg(not(feature = "viz_static"))]
#[allow(clippy::unnecessary_wraps)]
fn output_image_json(
    _value: &serde_json::Value,
    _args: &Args,
    _fmt: OutFormat,
    _width: usize,
    _height: usize,
    _scale: f64,
) -> CliResult<()> {
    unreachable!("image export is rejected in run() without the viz_static feature");
}

/// Map a `viz` output format to plotly's `ImageFormat`. `Html` never reaches here (it's routed to
/// the HTML writers before any image export).
#[cfg(feature = "viz_static")]
fn image_format(fmt: OutFormat) -> plotly::ImageFormat {
    use plotly::ImageFormat;
    match fmt {
        OutFormat::Png => ImageFormat::PNG,
        OutFormat::Jpeg => ImageFormat::JPEG,
        OutFormat::Webp => ImageFormat::WEBP,
        OutFormat::Svg => ImageFormat::SVG,
        OutFormat::Pdf => ImageFormat::PDF,
        OutFormat::Html => unreachable!(),
    }
}

/// Shared error mapper for static image export failures, pointing users at the browser/webdriver
/// requirement (the most common cause).
#[cfg(feature = "viz_static")]
fn image_export_err(e: impl std::fmt::Display) -> crate::CliError {
    crate::CliError::Other(format!(
        "Static image export failed: {e}. A Chromium or Firefox browser must be installed and \
         available for plotly's webdriver-based export."
    ))
}

/// Open `path` in the user's default application, honoring the `BROWSER` environment variable
/// when set (via `opener::open_browser`).
fn open_path(path: &str) -> CliResult<()> {
    opener::open_browser(path)
        .map_err(|e| crate::CliError::Other(format!("Could not open '{path}': {e}")))
}

// ----- small helpers -----

fn cell_to_string(cell: Option<&[u8]>) -> String {
    cell.map(|b| String::from_utf8_lossy(b).into_owned())
        .unwrap_or_default()
}

fn parse_f64(cell: Option<&[u8]>) -> Option<f64> {
    parse_finite_f64(std::str::from_utf8(cell?).ok()?)
}

/// `parse_f64` for an already-decoded string: finite-only numeric parse.
///
/// Rejects non-finite literals: Rust's `f64::from_str` accepts "inf"/"-inf"/"infinity"/"nan", which
/// would poison min/max folds (e.g. an infinite axis span collapses a contour grid into one bin),
/// emit bogus ±inf aggregate bars, serialize as `null` (silently dropped points), or make sort
/// comparators non-total. Treat them as unparseable.
fn parse_finite_f64(s: &str) -> Option<f64> {
    let s = s.trim();
    if s.is_empty() {
        return None;
    }
    s.parse::<f64>().ok().filter(|v| v.is_finite())
}

#[derive(Clone, Copy, PartialEq, Eq, Debug)]
enum Agg {
    Sum,
    Mean,
    Count,
    Min,
    Max,
}

fn parse_agg(agg: Option<&str>) -> CliResult<Option<Agg>> {
    match agg {
        None => Ok(None),
        Some(s) => match s.to_ascii_lowercase().as_str() {
            "sum" => Ok(Some(Agg::Sum)),
            "mean" | "avg" | "average" => Ok(Some(Agg::Mean)),
            "count" => Ok(Some(Agg::Count)),
            "min" => Ok(Some(Agg::Min)),
            "max" => Ok(Some(Agg::Max)),
            other => {
                fail_incorrectusage_clierror!(
                    "Unknown --agg '{other}'. Use sum, mean, count, min, or max."
                )
            },
        },
    }
}

/// Best-practice auto-selection for a categorical hierarchy of `depth` levels: a **treemap** for
/// shallow hierarchies (≤2 levels), where area encodes value for accurate size comparison; a
/// **sunburst** for deeper ones (3+ levels), which emphasizes parent-child structure / path
/// tracing and keeps many small leaves legible as ring segments.
fn auto_hierarchy_style(depth: usize) -> HierStyle {
    if depth >= 3 {
        HierStyle::Sunburst
    } else {
        HierStyle::Treemap
    }
}

/// Resolve the `--hierarchy-style` flag (`auto` | `treemap` | `sunburst`; default `auto`) to a
/// concrete chart for a `depth`-level hierarchy. `auto` applies `auto_hierarchy_style`.
fn resolve_hierarchy_style(flag: Option<&str>, depth: usize) -> CliResult<HierStyle> {
    match flag.map(|s| s.to_ascii_lowercase()).as_deref() {
        None | Some("auto") => Ok(auto_hierarchy_style(depth)),
        Some("treemap") => Ok(HierStyle::Treemap),
        Some("sunburst") => Ok(HierStyle::Sunburst),
        Some("icicle") => Ok(HierStyle::Icicle),
        Some(other) => fail_incorrectusage_clierror!(
            "Unknown --hierarchy-style '{other}'. Use auto, treemap, sunburst, or icicle."
        ),
    }
}

/// Resolved `--log-scale` mode for `viz smart` frequency bar panels.
#[derive(Clone, Copy, PartialEq, Eq)]
enum LogScale {
    /// Per-panel: log y-axis only when the bar distribution's dynamic range is high.
    Auto,
    /// Force a log y-axis on every frequency bar panel.
    On,
    /// Never use a log y-axis (linear axes only).
    Off,
}

/// Parse the `--log-scale` flag (defaults to `auto`).
fn parse_log_scale(mode: &str) -> CliResult<LogScale> {
    match mode.to_ascii_lowercase().as_str() {
        "auto" => Ok(LogScale::Auto),
        "on" | "true" => Ok(LogScale::On),
        "off" | "false" => Ok(LogScale::Off),
        other => {
            fail_incorrectusage_clierror!("Unknown --log-scale '{other}'. Use auto, on, or off.")
        },
    }
}

/// Resolved `--violin` mode for `viz smart` raw box panels.
#[derive(Clone, Copy, PartialEq, Eq)]
enum ViolinMode {
    /// Per-panel: violin only when the cached shape statistics suggest a quartile box
    /// would hide structure (see `wants_violin`).
    Auto,
    /// Draw every raw-value box panel as a violin.
    On,
    /// Never draw violins (box panels only).
    Off,
}

/// Parse the `--violin` flag (defaults to `auto`).
fn parse_violin_mode(mode: &str) -> CliResult<ViolinMode> {
    match mode.to_ascii_lowercase().as_str() {
        "auto" => Ok(ViolinMode::Auto),
        "on" | "true" => Ok(ViolinMode::On),
        "off" | "false" => Ok(ViolinMode::Off),
        other => {
            fail_incorrectusage_clierror!("Unknown --violin '{other}'. Use auto, on, or off.")
        },
    }
}

/// Resolved `--cluster` mode for `viz smart` map marker panels.
#[derive(Clone, Copy, PartialEq, Eq)]
enum ClusterMode {
    /// Cluster only when the core renders as plain markers (no density heatmap, no bubble-size
    /// measure) AND the rendered core reaches `SMART_CLUSTER_MIN_POINTS`.
    Auto,
    /// Cluster whenever the core renders as markers, regardless of point count or a bubble-size
    /// measure. A density heatmap is a different trace type and still can't be clustered.
    On,
    /// Never cluster; dense marker maps draw every point plainly.
    Off,
}

/// Parse the `--cluster` flag (defaults to `auto`).
fn parse_cluster_mode(mode: &str) -> CliResult<ClusterMode> {
    match mode.to_ascii_lowercase().as_str() {
        "auto" => Ok(ClusterMode::Auto),
        "on" | "true" => Ok(ClusterMode::On),
        "off" | "false" => Ok(ClusterMode::Off),
        other => {
            fail_incorrectusage_clierror!("Unknown --cluster '{other}'. Use auto, on, or off.")
        },
    }
}

/// Decide whether a frequency bar panel should use a logarithmic y-axis, given its resolved
/// `--log-scale` mode and the panel's bar counts. `On` always logs (when there are 2+ positive
/// bars to compare), `Off` never does, and `Auto` logs only when the tallest positive bar is at
/// least `LOG_SCALE_MIN_RATIO`x the shortest positive bar (the high-dynamic-range case where a
/// dominating "(NULL)"/"Other" bucket would otherwise flatten the real categories).
fn freq_panel_logs(mode: LogScale, counts: &[u64]) -> bool {
    if mode == LogScale::Off {
        return false;
    }
    let mut min_pos = u64::MAX;
    let mut max_pos = 0_u64;
    let mut n_pos = 0_usize;
    for &c in counts {
        if c > 0 {
            n_pos += 1;
            min_pos = min_pos.min(c);
            max_pos = max_pos.max(c);
        }
    }
    match mode {
        // forcing log still needs 2+ positive bars to be meaningful (and a positive max)
        LogScale::On => n_pos >= 2 && max_pos > 0,
        LogScale::Auto => n_pos >= 3 && (max_pos as f64) >= (min_pos as f64) * LOG_SCALE_MIN_RATIO,
        LogScale::Off => false,
    }
}

/// Whether a smart box panel's VALUE axis should be logarithmic under the resolved `--log-scale`
/// mode, from the column's observed min/max. The strictly-positive fast path takes log when every
/// value is placeable on a log axis (`min > 0`, which also implies `n_zero == n_negative == 0`);
/// under `Auto` it additionally requires the observed max/min ratio to span `LOG_SCALE_MIN_RATIO`
/// — the high-dynamic-range case where a linear box squashes into a sliver against its extremes.
/// When `min <= 0`, `box_log_skew_fallback` still allows log for a heavily right-skewed,
/// overwhelmingly-positive column (with only a negligible non-positive tail) that would otherwise
/// needle on a linear axis — see issue #4219.
fn box_panel_logs(
    mode: LogScale,
    s: &crate::cmd::stats::StatsData,
    min: Option<f64>,
    max: Option<f64>,
) -> bool {
    if mode == LogScale::Off {
        return false;
    }
    let (Some(lo), Some(hi)) = (min, max) else {
        return false;
    };
    // strictly-positive fast path: every value is placeable on a log axis.
    if lo > 0.0 && hi >= lo {
        return match mode {
            LogScale::On => true,
            LogScale::Auto => hi >= lo * LOG_SCALE_MIN_RATIO,
            LogScale::Off => false,
        };
    }
    // skew-aware fallback (issue #4219): `min` is not strictly positive, so a strict log axis is
    // impossible — but a heavily right-skewed, overwhelmingly-positive column would collapse a
    // LINEAR box/violin into a needle against a lone far outlier. Prefer a log axis (dropping the
    // negligible non-positive tail, which plotly does on a log axis) when it stays legible. Fully
    // cache-driven and independent of which min/max the caller passed.
    box_log_skew_fallback(mode, s)
}

/// Skew-aware log-axis fallback for `box_panel_logs`, consulted only when a column's minimum is
/// not strictly positive (so the strictly-positive fast path declined). Enables a log value axis
/// for a heavily right-skewed, overwhelmingly-positive column whose linear box/violin would
/// degenerate into a needle against a lone far outlier (issue #4219). All signals come from the
/// stats cache, so plain `viz smart` benefits too (skew has a base-cache derivation; the sign
/// counts and quartiles are always cached). Guards, ALL required:
///   - the non-positive share (zeros + negatives) is negligible (`<= BOX_LOG_NONPOS_MAX`), so the
///     dropped tail is a rounding error, not the story — this is what keeps a zero-inflated column
///     (e.g. 45–60% zeros) LINEAR;
///   - the interquartile box sits in positive space (`q1 > 0`), so the median and quartiles render
///     on a log axis even though the tiny non-positive tail cannot;
///   - the skew is pronounced and to the RIGHT (`is_right_skewed`, the same threshold that labels
///     the title "right-skewed") — a left tail toward zero gains nothing;
///   - under `Auto`, the positive dynamic range (`max / q1`) spans `LOG_SCALE_MIN_RATIO`, the
///     high-dynamic-range case a linear axis squashes. `On` forces log without the range guard.
fn box_log_skew_fallback(mode: LogScale, s: &crate::cmd::stats::StatsData) -> bool {
    // at most this share of values may be non-positive (dropped on the log axis)
    const BOX_LOG_NONPOS_MAX: f64 = 0.05;

    if mode == LogScale::Off {
        return false;
    }
    let (Some(neg), Some(zero), Some(pos)) = (s.n_negative, s.n_zero, s.n_positive) else {
        return false;
    };
    let total = neg + zero + pos;
    if total == 0 {
        return false;
    }
    #[allow(clippy::cast_precision_loss)]
    let nonpos_share = (neg + zero) as f64 / total as f64;
    if nonpos_share > BOX_LOG_NONPOS_MAX {
        return false;
    }
    let (Some(q1), Some(hi)) = (s.q1, parse_stat_f64(s.max.as_deref())) else {
        return false;
    };
    if q1 <= 0.0 || !hi.is_finite() || hi < q1 {
        return false;
    }
    // right-skew only: the needle is a high-tail phenomenon, so require positive skew
    if !is_right_skewed(s) {
        return false;
    }
    match mode {
        LogScale::On => true,
        LogScale::Auto => hi >= q1 * LOG_SCALE_MIN_RATIO,
        LogScale::Off => false,
    }
}

/// On a log value axis, a precomputed box whose lower whisker is `<= 0` (the observed min can be 0
/// for a heavily right-skewed column — issue #4219) is log-undefined and breaks the ENTIRE box
/// trace: plotly drops the box and only stray outlier points survive, which is worse than the
/// linear needle the log axis set out to fix. Clamp such a whisker up to `q1` so the box body,
/// median and upper whisker render; the sub-zero tail is unshowable on a log axis anyway. On a
/// linear axis (or when the whisker is already strictly positive) the whisker is returned
/// unchanged.
fn log_safe_lower_fence(log_y: bool, fence: f64, q1: f64) -> f64 {
    if log_y && fence <= 0.0 { q1 } else { fence }
}

/// The raw-value counterpart of `log_safe_lower_fence` (issue #4219). `box_log_skew_fallback`
/// grants a log value axis to a heavily right-skewed column that still holds a small share
/// (<= 5%) of non-positive values, on the strength of `q1 > 0`. Where the box is PRECOMPUTED that
/// is handled by clamping the lower whisker, but `BoxRaw`/`Violin` hand plotly the values
/// themselves and let it derive the whisker/KDE — so a single `0` or negative value re-creates the
/// exact log-undefined fence #4219 set out to eliminate, breaking the whole trace. Drop the
/// unplottable values on a log axis (they are unshowable there regardless); on a linear axis the
/// values are returned untouched.
fn log_safe_values(log_y: bool, values: Vec<f64>) -> Vec<f64> {
    if log_y {
        values.into_iter().filter(|v| *v > 0.0).collect()
    } else {
        values
    }
}

/// Whether a `MeasureByDim` bar panel's value (y) axis should be logarithmic under the resolved
/// `--log-scale` mode, from its already-aggregated top-N bar values. A log axis can't place 0 or
/// negative values, so EVERY shown bar must be strictly positive — a single dominant bar over a
/// pile of zeros (the common degenerate case) therefore stays LINEAR, since log couldn't render
/// the zeros anyway. Under `Auto`, additionally requires the max/min ratio to span
/// `LOG_SCALE_MIN_RATIO` — the high-dynamic-range case a linear axis squashes into one tall bar
/// with the rest flattened against the baseline.
fn measure_by_dim_logs(mode: LogScale, values: &[f64]) -> bool {
    if mode == LogScale::Off {
        return false;
    }
    let (mut min_pos, mut max_pos) = (f64::INFINITY, f64::NEG_INFINITY);
    let mut n = 0_usize;
    for &v in values {
        if !v.is_finite() || v <= 0.0 {
            // any non-positive (or non-finite) bar rules out a log value axis outright
            return false;
        }
        n += 1;
        min_pos = min_pos.min(v);
        max_pos = max_pos.max(v);
    }
    match mode {
        // forcing log still needs 2+ positive bars to be meaningful
        LogScale::On => n >= 2,
        LogScale::Auto => n >= 3 && max_pos >= min_pos * LOG_SCALE_MIN_RATIO,
        LogScale::Off => false,
    }
}

/// Whether ONE axis of a relationship panel (scatter pair, density contour) should render
/// logarithmic, judged from the values actually plotted on it (issue #4223). The distribution
/// panels resolve their single value axis from the stats cache via `box_log_skew_fallback`; a
/// relationship panel instead holds both of its vectors in hand — already downsampled or
/// positive-subsetted — so the verdict is read off exactly what the reader will see, and the two
/// axes decide independently (a skewed measure against a well-behaved one gets one log axis, not
/// two).
///
/// A log axis cannot place a zero or a negative, so — as in `measure_by_dim_logs` — a single
/// non-positive value rules it out outright. That is deliberate: it keeps a zero-inflated column
/// LINEAR here, and routes the heavy-tail case through the explicit, title-annotated
/// positive-subset retry (`positive_subset_pair`) rather than silently dropping half the rows.
fn relationship_axis_log(mode: LogScale, vs: &[f64]) -> bool {
    if mode == LogScale::Off {
        return false;
    }
    let (mut min_pos, mut max_pos) = (f64::INFINITY, f64::NEG_INFINITY);
    let mut n = 0_usize;
    for &v in vs {
        if !v.is_finite() || v <= 0.0 {
            return false;
        }
        n += 1;
        min_pos = min_pos.min(v);
        max_pos = max_pos.max(v);
    }
    match mode {
        LogScale::On => n >= 2,
        // `Auto` additionally requires a dynamic range wide enough that a linear axis would
        // squash the low end against the origin — the exact failure #4223 reports.
        LogScale::Auto => n >= 3 && max_pos >= min_pos * LOG_SCALE_MIN_RATIO,
        LogScale::Off => false,
    }
}

/// Whether a `viz smart` distribution panel should render as a violin instead of a box, under
/// the resolved `--violin` mode. A violin (KDE silhouette + inner quartile box + mean line) is a
/// strict superset of a box plot, so under `auto` it's the DEFAULT representation whenever it can
/// be drawn faithfully — with two guards: `value_log` panels keep the box (plotly estimates the
/// KDE in linear space, so a log value axis geometrically distorts the density silhouette, while
/// a box's five-number summary survives log scaling honestly), and columns with fewer than
/// `VIOLIN_MIN_CARDINALITY` distinct values keep the box (their KDE reads as a lumpy comb, not a
/// shape). `on` is the user's explicit choice and overrides both guards; `off` always boxes.
/// Confirmed-bimodal columns never reach this — `classify_measure` routes them to a histogram
/// first.
fn wants_violin(
    mode: ViolinMode,
    s: &crate::cmd::stats::StatsData,
    value_log: bool,
    n_points: u64,
) -> bool {
    match mode {
        ViolinMode::Off => false,
        ViolinMode::On => true,
        ViolinMode::Auto => {
            !value_log && s.cardinality >= VIOLIN_MIN_CARDINALITY && n_points >= VIOLIN_MIN_POINTS
        },
    }
}

/// Whether a panel will render with a logarithmic y-axis under the resolved `--log-scale` mode.
/// Frequency bars and measure-by-dimension bars decide from their values (high dynamic range); box
/// panels carry the verdict resolved at classification time (`Panel::value_log`, from the cached
/// min/max — see `box_panel_logs`); every other panel kind is always linear. Used both to gate the
/// panel's y-axis title cue and to size the Data Schematic's left margin to fit it.
fn panel_is_log(panel: &Panel, freq: &FreqMap, log_scale: LogScale) -> bool {
    match &panel.kind {
        PanelKind::FreqBar { idx } => {
            let counts: Vec<u64> = freq
                .get(idx)
                .map(|bars| bars.iter().map(|b| b.count).collect())
                .unwrap_or_default();
            freq_panel_logs(log_scale, &counts)
        },
        PanelKind::MeasureByDim { values, .. } => measure_by_dim_logs(log_scale, values),
        PanelKind::BoxStats { .. }
        | PanelKind::BoxRaw { .. }
        | PanelKind::BoxOutliers { .. }
        | PanelKind::Violin { .. } => panel.value_log,
        _ => false,
    }
}

/// Parse the `--box-points` flag controlling which sample points are drawn alongside a
/// box plot. Defaults to `outliers` (only the points beyond the Tukey 1.5*IQR fences).
fn parse_box_points(points: Option<&str>) -> CliResult<BoxPoints> {
    match points {
        None => Ok(BoxPoints::Outliers),
        Some(s) => match s.to_ascii_lowercase().as_str() {
            "outliers" => Ok(BoxPoints::Outliers),
            "all" => Ok(BoxPoints::All),
            "suspected" | "suspectedoutliers" => Ok(BoxPoints::SuspectedOutliers),
            "none" | "false" => Ok(BoxPoints::False),
            other => {
                fail_incorrectusage_clierror!(
                    "Unknown --box-points '{other}'. Use outliers, all, suspected, or none."
                )
            },
        },
    }
}

/// Map a resolved `--box-points` mode onto the violin trace's identical points enum, so
/// the box and violin paths share one flag and one parser.
const fn violin_points(points: &BoxPoints) -> ViolinPoints {
    match points {
        BoxPoints::All => ViolinPoints::All,
        BoxPoints::Outliers => ViolinPoints::Outliers,
        BoxPoints::SuspectedOutliers => ViolinPoints::SuspectedOutliers,
        BoxPoints::False => ViolinPoints::False,
    }
}

/// Resolve the box-overlay mode for a `viz smart` box panel. An explicit user `--box-points` mode
/// always wins (an explicit `none` means "no overlay" — keep the cheap cache-only quartile box, so
/// this returns `None`). Without an explicit mode, a size-based heuristic on the column's
/// non-null value count (see `SMART_BOX_ALL_MAX`) picks the mode: all points for small data, just
/// outliers for medium, and none for large data (returning `None` so no raw-values pass is done
/// here — the caller may still overlay just the Tukey outliers on the cache-only box via a
/// fence-filtered pass; see `SMART_BOX_OUTLIERS_CAP`).
/// A `None` return means "don't build a raw box for this column".
fn smart_box_points(explicit: Option<&BoxPoints>, n_points: u64) -> Option<BoxPoints> {
    if let Some(mode) = explicit {
        return if matches!(mode, BoxPoints::False) {
            None
        } else {
            Some(mode.clone())
        };
    }
    if n_points <= SMART_BOX_ALL_MAX {
        Some(BoxPoints::All)
    } else if n_points <= SMART_BOX_OUTLIERS_MAX {
        Some(BoxPoints::Outliers)
    } else {
        None
    }
}

/// Aggregate y values by x category, preserving first-seen (input) order so that
/// downstream numeric ordering is not lost to lexicographic sorting (e.g. 1, 10, 2).
fn aggregate(xs: Vec<String>, ys: Vec<f64>, agg: Agg) -> (Vec<String>, Vec<f64>) {
    let mut order: Vec<String> = Vec::new();
    let mut acc: HashMap<String, (f64, f64)> = HashMap::new(); // x -> (running, count)
    for (x, y) in xs.into_iter().zip(ys) {
        // probe first: the owned key is only needed when the category is new (see read_xy)
        if !acc.contains_key(&x) {
            order.push(x.clone());
        }
        let e = acc.entry(x).or_insert_with(|| match agg {
            Agg::Min => (f64::INFINITY, 0.0),
            Agg::Max => (f64::NEG_INFINITY, 0.0),
            _ => (0.0, 0.0),
        });
        e.1 += 1.0;
        match agg {
            Agg::Sum | Agg::Mean => e.0 += y,
            Agg::Count => {},
            Agg::Min => e.0 = e.0.min(y),
            Agg::Max => e.0 = e.0.max(y),
        }
    }
    let mut out_x = Vec::with_capacity(order.len());
    let mut out_y = Vec::with_capacity(order.len());
    for x in order {
        let (running, count) = acc[&x];
        out_x.push(x);
        out_y.push(match agg {
            Agg::Sum | Agg::Min | Agg::Max => running,
            Agg::Mean => {
                if count == 0.0 {
                    0.0
                } else {
                    running / count
                }
            },
            Agg::Count => count,
        });
    }
    (out_x, out_y)
}

/// Cap on the number of candidate dimensions and measures the `MeasureByDim` panel considers, so a
/// very wide table doesn't build a D×M group accumulator that blows up memory. Generous — real
/// dashboards have far fewer low-cardinality dimensions than this.
const MEASURE_BY_DIM_MAX_CANDIDATES: usize = 16;

/// Build the "measure by dimension" overview panel: a numeric measure aggregated by a
/// low-cardinality categorical dimension, chosen as the (dimension, measure) pair whose association
/// — the correlation ratio η² (ANOVA effect size: between-group variance / total variance) — is
/// strongest, provided it clears `MEASURE_BY_DIM_MIN_ETA_SQUARED`. Returns `Ok(None)` when there
/// are no candidate pairs or none is associated strongly enough (a weak grouping would be
/// misleading).
///
/// η² is computed from the read values themselves in a single pass (not the stats cache), so the
/// between- and total-variance are over exactly the same rows. For each measure-present row the
/// value contributes to that measure's grand (sum, sum-of-squares, n) and, partitioned by each
/// dimension's category, to that (measure, dimension) group's (sum, count). Then
/// ```text
/// η² = SS_between / SS_total where SS_total = Σy² − (Σy)²/n and
/// SS_between = Σ_g (Σy_g)²/n_g − (Σy)²/n.
/// ```
fn measure_by_dim_panel(
    args: &Args,
    stats: &[crate::cmd::stats::StatsData],
    col_sems: &[ColSemantics],
    measure_indices: &[usize],
    dim_indices: &[usize],
    top_n: usize,
) -> CliResult<Option<Panel>> {
    let measures: Vec<usize> = measure_indices
        .iter()
        .take(MEASURE_BY_DIM_MAX_CANDIDATES)
        .copied()
        .collect();
    let dims: Vec<usize> = dim_indices
        .iter()
        .take(MEASURE_BY_DIM_MAX_CANDIDATES)
        .copied()
        .collect();
    if measures.is_empty() || dims.is_empty() {
        return Ok(None);
    }
    let (n_m, n_d) = (measures.len(), dims.len());

    // grand[mi] = (Σy, Σy², n) over rows where measure `mi` parses
    let mut grand: Vec<(f64, f64, u64)> = vec![(0.0, 0.0, 0); n_m];
    // groups[mi * n_d + di] : category -> (Σy, count), over the same measure-present rows
    let mut groups: Vec<HashMap<String, (f64, u64)>> = vec![HashMap::new(); n_m * n_d];

    let (mut rdr, _headers, _nh) = reader_and_headers(args)?;
    let mut record = csv::ByteRecord::new();
    // the dimension keys depend only on the ROW, not on the measure, but the dimension loop is
    // nested inside the measure loop — recomputing them per measure did up to
    // MEASURE_BY_DIM_MAX_CANDIDATES^2 (16x16 = 256) key builds per row, two String allocations
    // each, where n_d (16) suffice. Build them once per row and index by `di`.
    let mut row_keys: Vec<String> = Vec::with_capacity(n_d);
    while rdr.read_byte_record(&mut record)? {
        // only worth building when at least one measure parses on this row
        if !measures.iter().any(|&i| parse_f64(record.get(i)).is_some()) {
            continue;
        }
        row_keys.clear();
        for &d_idx in &dims {
            let raw = cell_to_string(record.get(d_idx));
            let trimmed = raw.trim();
            row_keys.push(if trimmed.is_empty() {
                null_text()
            } else {
                trimmed.to_string()
            });
        }
        for (mi, &m_idx) in measures.iter().enumerate() {
            let Some(y) = parse_f64(record.get(m_idx)) else {
                continue;
            };
            let g = &mut grand[mi];
            g.0 += y;
            g.1 += y * y;
            g.2 += 1;
            for (di, key) in row_keys.iter().enumerate() {
                let g = &mut groups[mi * n_d + di];
                if let Some(e) = g.get_mut(key) {
                    e.0 += y;
                    e.1 += 1;
                } else {
                    g.insert(key.clone(), (y, 1));
                }
            }
        }
    }

    // pick the (measure, dimension) pair with the strongest η²
    let mut best: Option<(f64, usize, usize)> = None; // (eta2, mi, di)
    for mi in 0..n_m {
        let (sum, sumsq, n) = grand[mi];
        if n < 2 {
            continue;
        }
        let nf = n as f64;
        let correction = sum * sum / nf;
        let ss_total = sumsq - correction;
        if ss_total <= 0.0 {
            continue; // constant measure — nothing to explain
        }
        for di in 0..n_d {
            let gmap = &groups[mi * n_d + di];
            if gmap.len() < 2 {
                continue; // need ≥2 groups to compare
            }
            let ss_between: f64 = gmap
                .values()
                .map(|&(gs, gc)| if gc > 0 { gs * gs / gc as f64 } else { 0.0 })
                .sum::<f64>()
                - correction;
            let eta2 = (ss_between / ss_total).clamp(0.0, 1.0);
            // Require a MEANINGFUL improvement, not just a larger float. `ss_between` sums over a
            // HashMap's values, so its rounding depends on iteration order: two dimensions that
            // are mathematically tied (1:1 columns like an agency code and its name explain
            // exactly the same variance) land within a few ULPs of each other and a bare `>`
            // picked a different winner run to run — the panel title flipped between reruns of
            // the same binary on the same file. An epsilon makes the earlier (lowest-index)
            // candidate win every tie, so the choice is reproducible.
            const ETA2_TIE_EPS: f64 = 1e-9;
            if best.is_none_or(|(b, ..)| eta2 > b + ETA2_TIE_EPS) {
                best = Some((eta2, mi, di));
            }
        }
    }

    let Some((eta2, mi, di)) = best.filter(|&(e, ..)| e >= MEASURE_BY_DIM_MIN_ETA_SQUARED) else {
        return Ok(None);
    };

    // resolve the human label for a column: dictionary label, else header, else positional
    let label = |idx: usize| {
        let sem = &col_sems[idx];
        if !sem.label.is_empty() {
            sem.label.clone()
        } else if stats[idx].field.is_empty() {
            format!("col {}", idx + 1)
        } else {
            stats[idx].field.clone()
        }
    };

    // Additive measures (counts/amounts) sum; rates/ratios average. An UN-TAGGED measure — which
    // is EVERY measure when no data dictionary is supplied, since `derive_semantics` returns the
    // default verdict without a dictionary row — used to fall through to the mean. When the mean is
    // set by the tail rather than a typical row (`mean_is_outlier_driven`), "top N by mean" ranks
    // whoever holds the single biggest item rather than the biggest total (issue #4220). Sum such a
    // column instead: for an additive measure the total is both the outlier-robust ranking and the
    // quantity a reader actually wants.
    //
    // The escape hatch is the name: an intensive quantity (a rate, index, temperature, per-capita
    // figure) must never be summed, and `is_intensive_measure` recognizes those from the header
    // alone — no dictionary required — so an un-tagged skewed RATE stays on the mean.
    let m_idx = measures[mi];
    let agg = match col_sems[m_idx].agg {
        Some(Agg::Sum) => Agg::Sum,
        Some(_) => Agg::Mean,
        None if mean_is_outlier_driven(&stats[m_idx])
            && !is_intensive_measure(&col_sems[m_idx].label, &stats[m_idx].field) =>
        {
            Agg::Sum
        },
        None => Agg::Mean,
    };
    let agg_word = if agg == Agg::Sum {
        t!("viz.title.agg_sum")
    } else {
        t!("viz.title.agg_mean")
    };

    // aggregate each category, then sort by value descending and cap to the top-N
    let gmap = &groups[mi * n_d + di];
    let mut rows: Vec<(String, f64)> = gmap
        .iter()
        .map(|(k, &(gs, gc))| {
            let v = if agg == Agg::Sum || gc == 0 {
                gs
            } else {
                gs / gc as f64
            };
            (k.clone(), v)
        })
        .collect();
    rows.sort_by(|a, b| b.1.partial_cmp(&a.1).unwrap_or(std::cmp::Ordering::Equal));
    let total_groups = rows.len();
    rows.truncate(top_n.max(1));
    let truncated = total_groups > rows.len();

    // η² IS the share of the measure's variance the grouping explains, so state it that way
    // (issue #4220). A bare "η²=0.17" reads as a headline effect size to anyone who doesn't hold
    // Cohen's conventions in their head; "explains 17% of variance" says the same thing and lets
    // the reader weigh the ranking for themselves.
    let mut title = t!(
        "viz.title.measure_by_dim",
        q_measure = label(measures[mi]),
        q_dim = label(dims[di]),
        q_agg = agg_word,
        q_pct = format!("{:.0}", eta2 * 100.0)
    )
    .into_owned();
    if truncated {
        title.push_str(&t!("viz.title.top_n", q_n = rows.len()));
    }

    let (labels, values): (Vec<String>, Vec<f64>) = rows.into_iter().unzip();
    Ok(Some(Panel::new(
        title,
        PanelKind::MeasureByDim { labels, values },
    )))
}

fn grouped_violin_panel(
    args: &Args,
    stats: &[crate::cmd::stats::StatsData],
    col_sems: &[ColSemantics],
    measure_indices: &[usize],
    dim_indices: &[usize],
    top_n: usize,
    violin_mode: ViolinMode,
    measure_nonnull: u64,
    explicit_points: Option<&BoxPoints>,
) -> CliResult<Option<Panel>> {
    // `--violin off` suppresses violins everywhere, including this supplementary
    // distribution-by-category overview (there is no grouped-box fallback panel), so bail before
    // the data pass rather than drawing a violin the flag forbids.
    if matches!(violin_mode, ViolinMode::Off) {
        return Ok(None);
    }
    let Some(&m_idx) = measure_indices.first() else {
        return Ok(None);
    };
    // group by the coarsest dimension that would also anchor the sunburst/treemap's outer ring
    // (cardinality >= HIER_MIN_DIM_CARDINALITY), so the violins split on the same categories the
    // hierarchy shows — fewest, most-readable violins. Fall back to the coarsest dimension overall
    // (e.g. a two-value flag) only when no dimension is rich enough for a hierarchy.
    let coarsest = |min_card: u64| {
        dim_indices
            .iter()
            .copied()
            .filter(|&i| stats[i].cardinality >= min_card)
            .min_by_key(|&i| (stats[i].cardinality, i))
    };
    let Some(d_idx) = coarsest(HIER_MIN_DIM_CARDINALITY).or_else(|| coarsest(0)) else {
        return Ok(None);
    };

    // Bound the buffer: keep only every stride-th non-null measure value so a large input can't
    // allocate an O(rows) Vec before sampling. Full per-category counts (cheap, O(cardinality))
    // still rank the top-N categories; a global stride samples proportionally across them.
    let stride = if measure_nonnull > *VIOLIN_SAMPLE_MAX {
        measure_nonnull.div_ceil(*VIOLIN_SAMPLE_MAX).max(1)
    } else {
        1
    };
    let sampled_flag = stride > 1;

    let mut counts: HashMap<String, u64> = HashMap::new();
    let mut sampled: HashMap<String, Vec<f64>> = HashMap::new();
    let (mut rdr, _headers, _nh) = reader_and_headers(args)?;
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let Some(y) = parse_f64(record.get(m_idx)) else {
            continue;
        };
        let raw = cell_to_string(record.get(d_idx));
        let key = if raw.trim().is_empty() {
            null_text()
        } else {
            raw.trim().to_string()
        };
        // stride PER CATEGORY (on each category's own 0-based value position), not on a global
        // counter: a global `seen % stride` aliases with row/category ordering — e.g. two
        // categories alternating by row with stride 2 would sample only one of them — whereas a
        // per-category counter samples every category's 1st, (stride+1)-th, … value independently,
        // so each keeps ~count/stride points and no populous category is starved. The retained
        // total stays ~VIOLIN_SAMPLE_MAX (Σ count/stride), plus at most one extra value per
        // category.
        let pos = {
            let n = counts.entry(key.clone()).or_insert(0);
            *n += 1;
            *n - 1
        };
        if pos.is_multiple_of(stride) {
            sampled.entry(key).or_default().push(y);
        }
    }

    // rank by full count, keep the top-N most-populous categories (descending count, then label
    // for a stable order), then emit their sampled values as parallel (group, value) arrays.
    let mut ranked: Vec<(String, u64)> = counts.into_iter().collect();
    ranked.sort_by(|a, b| b.1.cmp(&a.1).then_with(|| a.0.cmp(&b.0)));
    let total_cats = ranked.len();
    ranked.truncate(top_n.max(1));

    let mut groups: Vec<String> = Vec::new();
    let mut values: Vec<f64> = Vec::new();
    let mut kept_cats = 0_usize;
    for (cat, full_count) in &ranked {
        // under `--violin auto`, drop a category with too few observations for an honest KDE — its
        // silhouette would be bandwidth artifact, the very thing VIOLIN_MIN_POINTS guards against
        // for single-column violins. `--violin on` forces every category (guards bypassed). The
        // gate is on the FULL per-category count, not the strided-down sample.
        if matches!(violin_mode, ViolinMode::Auto) && *full_count < VIOLIN_MIN_POINTS {
            continue;
        }
        // a top category striped to zero samples (only possible for a tiny category under a large
        // stride) would draw an empty violin — skip it rather than render a blank slot.
        match sampled.get(cat) {
            Some(vals) if !vals.is_empty() => {
                kept_cats += 1;
                for &v in vals {
                    groups.push(cat.clone());
                    values.push(v);
                }
            },
            _ => {},
        }
    }
    // fewer than 2 eligible categories (a single violin conveys no grouping); under `auto` this
    // also fires when the sample-size gate leaves 0 or 1 well-populated category.
    if kept_cats < 2 {
        return Ok(None);
    }

    // an explicit --box-points is the user's call; otherwise overlay outliers, but draw NO points
    // when the values were stride-sampled (a sample misses the true extremes, so an "outlier"
    // overlay would be misleading — the KDE silhouette still shows the shape).
    let points = match explicit_points {
        Some(p) => p.clone(),
        None if sampled_flag => BoxPoints::False,
        None => BoxPoints::Outliers,
    };

    // resolve the human label for a column: dictionary label, else header, else positional
    let label = |idx: usize| {
        let sem = &col_sems[idx];
        if !sem.label.is_empty() {
            sem.label.clone()
        } else if stats[idx].field.is_empty() {
            format!("col {}", idx + 1)
        } else {
            stats[idx].field.clone()
        }
    };
    let mut title = t!(
        "viz.title.distribution_by",
        q_measure = label(m_idx),
        q_dim = label(d_idx)
    )
    .into_owned();
    if total_cats > kept_cats {
        title.push_str(&t!("viz.title.top_n", q_n = kept_cats));
    }

    Ok(Some(Panel::new(
        title,
        PanelKind::GroupedViolin {
            groups,
            values,
            points,
        },
    )))
}

/// Path to moarstats' `--bivariate` sidecar CSV for a given `viz smart` input, mirroring
/// `moarstats::get_bivariate_csv_path`'s non-joined form (`viz smart` never asks moarstats to
/// `--join-inputs`): `<parent>/<stem>.stats.bivariate.csv`.
fn bivariate_csv_path(input: &str) -> std::path::PathBuf {
    let input_path = std::path::Path::new(input);
    let parent = input_path
        .parent()
        .unwrap_or_else(|| std::path::Path::new(""));
    let fstem = input_path
        .file_stem()
        .map(|s| s.to_string_lossy().into_owned())
        .unwrap_or_else(|| input.to_string());
    parent.join(format!("{fstem}.stats.bivariate.csv"))
}

/// Path to the inferred Data Dictionary sidecar for a `viz smart --dictionary infer`
/// input: `<parent>/<stem>.schema.json`. Written after a successful infer so users can
/// fine-tune it, and reused (skipping the LLM) on later runs when present.
fn dictionary_sidecar_path(input: &str) -> std::path::PathBuf {
    let input_path = std::path::Path::new(input);
    let parent = input_path
        .parent()
        .unwrap_or_else(|| std::path::Path::new(""));
    let fstem = input_path
        .file_stem()
        .map(|s| s.to_string_lossy().into_owned())
        .unwrap_or_else(|| input.to_string());
    parent.join(format!("{fstem}.schema.json"))
}

/// Write the inferred Data Dictionary to its sidecar ATOMICALLY: fill a temp file in the same
/// directory, then rename it over the destination.
///
/// `fs::write` truncates the destination up front, so an I/O failure partway through would leave a
/// previously saved - possibly hand-edited - dictionary empty or half-written. That was harmless
/// while a fresh infer only ever ran when no sidecar existed, but `QSV_VIZ_DICT_FRESH` re-infers
/// over an existing one, so the destination must never be observable in a partial state: after
/// this, it is either the old dictionary or the complete new one.
///
/// The temp file is created in the destination's own directory so the rename stays within one
/// filesystem (a cross-device rename is not atomic and would fail), and is removed automatically if
/// anything fails before the rename.
///
/// On unix the resulting mode matches what `fs::write` would have produced: an existing sidecar
/// keeps its own mode, and a new one gets 0666 minus the process umask. Forcing a fixed mode here
/// would either widen a deliberately private dictionary or ignore a restrictive umask. The temp
/// file is created with that same target mode rather than a permissive default, so a private
/// dictionary's contents are never briefly readable through the temp file mid-write.
fn write_dictionary_sidecar(path: &std::path::Path, contents: &str) -> std::io::Result<()> {
    #[cfg(unix)]
    use std::os::unix::fs::PermissionsExt;

    let dir = path
        .parent()
        .filter(|p| !p.as_os_str().is_empty())
        .unwrap_or_else(|| std::path::Path::new("."));

    // Mode of an existing sidecar. `fs::write` wrote in place and never altered the mode of a file
    // that already existed, so an atomic replace has to restore it explicitly - otherwise a
    // deliberately private (e.g. 0600) dictionary would be silently widened. None when creating a
    // new sidecar.
    #[cfg(unix)]
    let existing_perms = std::fs::metadata(path).ok().map(|m| m.permissions());

    let mut builder = tempfile::Builder::new();
    builder.prefix(".qsv-dict-").suffix(".json.tmp");
    // Create the temp file with the mode the FINAL sidecar should have, not a permissive default:
    // it holds the complete dictionary from the first write, so anything wider would expose the
    // contents of a private sidecar for the length of the write. Replacing an existing sidecar
    // reuses its mode; a new one asks for 0666 and lets the kernel subtract the process umask,
    // exactly as `fs::write`'s create did (tempfile would otherwise use 0600, narrower than the
    // umask calls for). Creation is umask-filtered, so this can only ever be NARROWER than the
    // target - never wider - and the exact mode is restored after the write below.
    #[cfg(unix)]
    builder.permissions(
        existing_perms
            .clone()
            .unwrap_or_else(|| std::fs::Permissions::from_mode(0o666)),
    );

    let mut tmpfile = builder.tempfile_in(dir)?;
    tmpfile.write_all(contents.as_bytes())?;
    // flush to disk before the rename, so a crash right after it cannot leave the sidecar
    // pointing at unwritten data
    tmpfile.as_file_mut().sync_all()?;
    // re-apply the existing mode exactly: creation above was umask-filtered, so bits the umask
    // stripped (e.g. group-write on a 0660 sidecar) still have to be restored. `set_permissions`
    // is not umask-filtered.
    #[cfg(unix)]
    if let Some(perms) = existing_perms {
        tmpfile.as_file().set_permissions(perms)?;
    }
    tmpfile.persist(path).map_err(|e| e.error)?;
    Ok(())
}

/// Largest sidecar `collect_sidecar_downloads` will embed in the Data Dictionary page. The bytes
/// ride as a base64 `data:` URI (4/3 inflation) inside the Data Schematic HTML, so an unbounded
/// embed would balloon the file. Most sidecars are bounded by COLUMN count and stay tiny, but the
/// `frequency` cache stores complete per-value data for every column and can reach hundreds of MB
/// on a wide, high-cardinality dataset — hence the cap (and the size check BEFORE the read).
const MAX_SIDECAR_EMBED_BYTES: u64 = 4 * 1024 * 1024;

/// One generated sidecar offered as a download on the `--dict-info` Data Dictionary page.
struct SidecarDownload {
    /// The sidecar's own basename, used verbatim as the `download` attribute so several
    /// downloads from one page never collide.
    fname: String,
    /// A one-word tag ("stats", "freq", …) shown INSTEAD of the filename in the drawer's fixed
    /// button bar, where the full names don't fit. Without it every link there would collapse to
    /// an identical ⤓ glyph and the reader couldn't tell them apart without hovering.
    short: &'static str,
    /// MIME type for the `data:` URI, by extension.
    mime:  &'static str,
    /// Verbatim file bytes.
    bytes: Vec<u8>,
}

/// MIME type for a sidecar, by extension. Only the extensions qsv's own sidecars use.
fn sidecar_mime(path: &std::path::Path) -> &'static str {
    match path.extension().and_then(|e| e.to_str()) {
        Some("csv") => "text/csv",
        Some("jsonl") => "application/x-ndjson",
        _ => "application/json",
    }
}

/// Sidecar metadata keys that hold an absolute local filesystem path. A Data Schematic is made to
/// be SHARED, so embedding these verbatim would hand every recipient the author's directory layout
/// (`/Users/jane/clients/acme/…`). `redact_sidecar_paths` rewrites each to its bare file name.
const SIDECAR_PATH_KEYS: [&str; 3] = ["canonical_input_path", "canonical_stats_path", "arg_input"];

/// Where a sidecar keeps the metadata object carrying `SIDECAR_PATH_KEYS`.
#[derive(Clone, Copy, PartialEq, Eq)]
enum SidecarRedaction {
    /// No metadata object — the file is pure data (the stats CSV/JSONL, the bivariate CSV).
    None,
    /// The whole file is one JSON object (`.stats.csv.json`).
    JsonObject,
    /// JSONL whose FIRST line is the metadata object (`.freq.csv.data.jsonl`); the data lines
    /// that follow are passed through untouched.
    JsonlFirstLine,
}

/// Replace each `SIDECAR_PATH_KEYS` value in `obj` with its bare file name. Only string values are
/// touched, and only the last path component is kept, so the embedded metadata still identifies
/// WHICH file it describes without disclosing WHERE it lived.
fn redact_path_keys(obj: &mut serde_json::Value) {
    let Some(map) = obj.as_object_mut() else {
        return;
    };
    for key in SIDECAR_PATH_KEYS {
        if let Some(v) = map.get_mut(key)
            && let Some(s) = v.as_str()
            && let Some(base) = std::path::Path::new(s).file_name()
        {
            *v = serde_json::Value::String(base.to_string_lossy().into_owned());
        }
    }
}

/// Strip absolute local paths out of a sidecar's metadata before it is embedded in the shareable
/// Data Schematic. Returns the bytes unchanged for `SidecarRedaction::None`, and also whenever the
/// metadata can't be parsed — a sidecar we don't understand is left alone rather than mangled,
/// since the download is a convenience and corrupting it would be worse than the leak it avoids
/// (qsv writes these files, so an unparseable one means a hand-edit or a format change, not
/// untrusted input).
fn redact_sidecar_paths(bytes: Vec<u8>, kind: SidecarRedaction) -> Vec<u8> {
    match kind {
        SidecarRedaction::None => bytes,
        SidecarRedaction::JsonObject => {
            let Ok(mut v) = serde_json::from_slice::<serde_json::Value>(&bytes) else {
                return bytes;
            };
            redact_path_keys(&mut v);
            serde_json::to_vec_pretty(&v).unwrap_or(bytes)
        },
        SidecarRedaction::JsonlFirstLine => {
            let Ok(text) = String::from_utf8(bytes.clone()) else {
                return bytes;
            };
            let mut lines = text.splitn(2, '\n');
            let Some(first) = lines.next() else {
                return bytes;
            };
            let Ok(mut v) = serde_json::from_str::<serde_json::Value>(first) else {
                return bytes;
            };
            redact_path_keys(&mut v);
            let Ok(redacted) = serde_json::to_string(&v) else {
                return bytes;
            };
            let mut out = redacted.into_bytes();
            if let Some(rest) = lines.next() {
                out.push(b'\n');
                out.extend_from_slice(rest.as_bytes());
            }
            out
        },
    }
}

/// Read one sidecar for embedding, or `None` when it is absent, unreadable, or over
/// `MAX_SIDECAR_EMBED_BYTES`. Every failure is a SOFT skip (matching the dictionary loader's
/// fail-to-stats philosophy) — a missing download is never worth failing a Data Schematic over —
/// but an over-cap skip emits a `viz_note` so the omission is never silent.
fn read_sidecar(
    path: &std::path::Path,
    short: &'static str,
    redact: SidecarRedaction,
) -> Option<SidecarDownload> {
    let len = std::fs::metadata(path).ok()?.len();
    if len > MAX_SIDECAR_EMBED_BYTES {
        viz_note(&format!(
            "viz smart --dict-info: {} is {:.1} MB, over the {} MB embed limit; omitting its \
             download link.",
            path.display(),
            len as f64 / (1024.0 * 1024.0),
            MAX_SIDECAR_EMBED_BYTES / (1024 * 1024)
        ));
        return None;
    }
    let bytes = redact_sidecar_paths(std::fs::read(path).ok()?, redact);
    Some(SidecarDownload {
        fname: path.file_name()?.to_string_lossy().into_owned(),
        short,
        mime: sidecar_mime(path),
        bytes,
    })
}

/// The generated sidecars this `viz smart` run ACTUALLY consumed, for the `--dict-info` Data
/// Dictionary page's download row. Never speculative: a sidecar that merely exists on disk but was
/// not used (a stale bivariate CSV from a prior run, a frequency cache rejected as
/// stale/incompatible) is deliberately NOT offered, since handing the reader a file the Data
/// Schematic wasn't built from is worse than offering nothing.
///
/// Each path is derived exactly the way its CONSUMER derives it — do not "simplify" this by
/// canonicalizing once up front:
/// - the stats pair is keyed off `stats_anchor`, the path the run ALREADY resolved (see below),
///   canonicalized like `util::get_stats_records` does;
/// - the frequency cache goes through `frequency::frequency_cache_path`, which canonicalizes
///   internally the same way `read_frequency_cache_view` does;
/// - the bivariate CSV is keyed off the RAW input's file stem (`bivariate_csv_path`), which is also
///   what `bivariate_sidecar_fresh` was computed against.
///
/// `stats_anchor` MUST be the very path the stats were loaded from — in practice
/// `SmartPrep::stats_input`, the string handed to `get_stats_records`, whose `dc:` handle was
/// resolved once in `smart_prepare`. Do NOT resolve a `dc:` handle here, and do not substitute
/// another config's path: `diskcache::resolve_dc_path` auto-refreshes a stale entry (a network
/// fetch that can materialize a DIFFERENT CSV), so any second resolution can disagree with the
/// one the stats came from — the exact divergence this function exists to prevent, on top of
/// doing surprise network I/O while writing out a page.
///
/// For stdin the on-disk sidecars don't exist at all, but the charted-frequency CSV is generated in
/// memory and is still offered.
fn collect_sidecar_downloads(
    input: Option<&str>,
    stats_anchor: Option<&std::path::Path>,
    stats: &[crate::cmd::stats::StatsData],
    freq: &FreqMap,
    freq_cache_used: bool,
    bivariate_fresh: bool,
) -> Vec<SidecarDownload> {
    let mut out = Vec::new();

    // stats pair: `.stats.csv.data.jsonl` is what viz reads; `.stats.csv.json` records the parsing
    // options it was built with.
    //
    // The human-readable `.stats.csv` is deliberately NOT offered. viz never reads it, so it can't
    // be shown to be the stats behind this Data Schematic: an explicit `qsv stats --select … -o
    // <stem>.stats.csv` leaves a file that is newer than the source yet describes a different
    // computation than the cache viz actually used, and no mtime test can tell the two apart. The
    // same numbers ride in `.stats.csv.data.jsonl`, which viz demonstrably did read.
    if let Some(anchor) = stats_anchor
        && let Ok(canonical) = anchor.canonicalize()
    {
        for (ext, short, redact) in [
            ("stats.csv.data.jsonl", "stats", SidecarRedaction::None),
            ("stats.csv.json", "stats meta", SidecarRedaction::JsonObject),
        ] {
            if let Some(sc) = read_sidecar(&canonical.with_extension(ext), short, redact) {
                out.push(sc);
            }
        }
    }

    // frequency cache: offered ONLY when freq_from_cache actually hit. It is opportunistic — an
    // absent/stale/option-incompatible cache silently falls back to a full recompute pass — so
    // existence alone is not proof of use.
    if freq_cache_used
        && let Some(input) = input.filter(|i| *i != "-")
        && let Some(sc) = read_sidecar(
            &crate::cmd::frequency::frequency_cache_path(std::path::Path::new(input)),
            "freq",
            SidecarRedaction::JsonlFirstLine,
        )
    {
        out.push(sc);
    }

    // the frequency counts the Data Schematic actually CHARTED. Generated in memory (nothing is
    // written to disk), so unlike the cache above it is always available — whether the bars
    // came from a cache hit or a full recompute — and it is exactly what the reader sees in the
    // panels, aggregate `(NULL)`/`Other (N)` buckets included, already capped at --limit. Named
    // off the stats anchor, so a `dc:` handle yields `<cached name>.viz-frequency.csv` rather
    // than a download named after the literal `dc:` string.
    if let Some(sc) = charted_frequency_csv(
        stats_anchor.unwrap_or_else(|| std::path::Path::new("data")),
        stats,
        freq,
    ) {
        out.push(sc);
    }

    // bivariate: only when THIS run's moarstats subprocess provably wrote it.
    if bivariate_fresh
        && let Some(input) = input.filter(|i| *i != "-")
        && let Some(sc) = read_sidecar(
            &bivariate_csv_path(input),
            "bivariate",
            SidecarRedaction::None,
        )
    {
        out.push(sc);
    }

    out
}

/// The frequency-bar counts this Data Schematic charted, as a CSV download
/// (`<stem>.viz-frequency.csv`) built in memory — no file is written.
///
/// This is deliberately NOT the same thing as the `frequency` cache sidecar: the cache is the full
/// per-value distribution of every column and only exists when `qsv frequency --frequency-jsonl`
/// was run, whereas this is precisely what the panels show — only the charted columns, already
/// truncated to `--limit`, with the synthetic `(NULL)` / `Other (N)` aggregate buckets included and
/// flagged as such. Columns follow `stats` (the dataset's header order) and bars keep their
/// plotted order, so the file reads in the same sequence as the Data Schematic.
///
/// Returns `None` when no frequency panel was drawn.
fn charted_frequency_csv(
    input_path: &std::path::Path,
    stats: &[crate::cmd::stats::StatsData],
    freq: &FreqMap,
) -> Option<SidecarDownload> {
    if freq.is_empty() {
        return None;
    }
    let mut wtr = csv::Writer::from_writer(Vec::new());
    wtr.write_record(["column", "value", "count", "bucket"])
        .ok()?;
    for (idx, s) in stats.iter().enumerate() {
        let Some(bars) = freq.get(&idx) else {
            continue;
        };
        for bar in bars {
            wtr.write_record([
                s.field.as_str(),
                bar.label.as_str(),
                &bar.count.to_string(),
                if bar.kind == FreqBarKind::Aggregate {
                    "aggregate"
                } else {
                    "category"
                },
            ])
            .ok()?;
        }
    }
    let bytes = wtr.into_inner().ok()?;
    let stem = input_path
        .file_stem()
        .map_or_else(|| "data".to_string(), |s| s.to_string_lossy().into_owned());
    Some(SidecarDownload {
        fname: format!("{stem}.viz-frequency.csv"),
        short: "charted freq",
        mime: "text/csv",
        bytes,
    })
}

/// Build viz's phase-progress spinner. When `show`, it draws to stderr; indicatif hides it
/// automatically when stderr isn't a terminal (piped/redirected/CI), so no manual TTY check is
/// needed. When not shown, a hidden no-op bar keeps call sites branch-free.
///
/// The spinner steady-ticks so it stays visible AND animated throughout the long phases
/// (stats/dictionary/moarstats/map). Those phases' subprocesses run via `util::run_qsv_cmd`,
/// which *captures* their output rather than letting it hit the terminal, so they don't clash
/// with the live bar. viz's own stderr writes go through `viz_note()`, which suspends the
/// registered `VIZ_PROGRESS` bar around each write so they print cleanly around the tick.
fn viz_progress(show: bool) -> ProgressBar {
    if show {
        let pb = ProgressBar::with_draw_target(None, ProgressDrawTarget::stderr_with_hz(5));
        pb.set_style(
            ProgressStyle::with_template("{spinner:.green} [{elapsed_precise}] {msg}").unwrap(),
        );
        pb.enable_steady_tick(Duration::from_millis(120));
        pb
    } else {
        ProgressBar::with_draw_target(None, ProgressDrawTarget::hidden())
    }
}

/// RAII guard that clears viz's progress spinner on drop, so a live steady-ticking bar never
/// lingers past `run()` — including the `?` error paths that return after the bar is registered
/// but before an explicit `finish_and_clear()`. Without this, a validation/build error would print
/// the top-level error message on top of an animating spinner on a TTY. `finish_and_clear()` is
/// safe to call more than once, so the explicit clears on the success paths stay harmless.
struct ProgressGuard(ProgressBar);

impl Drop for ProgressGuard {
    fn drop(&mut self) {
        self.0.finish_and_clear();
    }
}

/// The active `viz` progress spinner, registered once by `run()` so notices emitted deep in the
/// call tree (map/choropleth/bivariate helpers, dictionary messages) can suspend it without every
/// helper having to thread a `&ProgressBar`. `viz` runs once per process, so a process-global is
/// safe here.
static VIZ_PROGRESS: std::sync::OnceLock<ProgressBar> = std::sync::OnceLock::new();

/// Emit a `viz` notice to stderr without garbling a live progress spinner. When a spinner is
/// registered (see `VIZ_PROGRESS`) it is briefly suspended around the write so indicatif clears and
/// redraws the bar cleanly; otherwise the message is written directly. A suspended write still
/// reaches stderr even when the bar is hidden (non-TTY), so notices are never swallowed.
fn viz_note(msg: &str) {
    match VIZ_PROGRESS.get() {
        Some(pb) => pb.suspend(|| eprintln!("{msg}")),
        None => eprintln!("{msg}"),
    }
}

/// Extract the model name from a describegpt JSON Schema dictionary. The model is baked into
/// the `x-qsv.generated_by` provenance string as a `Model: <model>` line. Returns `None` when
/// the text isn't parseable JSON or carries no such line (e.g. a hand-written dictionary).
fn parse_dict_model(json_text: &str) -> Option<String> {
    let value: serde_json::Value = serde_json::from_str(json_text).ok()?;
    let generated_by = value.get("x-qsv")?.get("generated_by")?.as_str()?;
    for line in generated_by.lines() {
        if let Some(rest) = line.trim().strip_prefix("Model:") {
            let model = rest.trim();
            if !model.is_empty() {
                return Some(model.to_string());
            }
        }
    }
    None
}

/// Sum the total token counts describegpt reports on its (captured) stderr. Each inference
/// pass prints a `TokenUsage { prompt: N, completion: N, total: N, elapsed: N, completion_tps:
/// N.NN }` line; `--two-pass` yields two. Returns `None` when no such line is present (e.g.
/// describegpt ran quiet), else the summed `total:` across passes.
fn sum_dict_tokens(stderr: &str) -> Option<u64> {
    let mut sum = 0_u64;
    let mut found = false;
    for line in stderr.lines() {
        let Some(start) = line.find("TokenUsage {") else {
            continue;
        };
        let Some(total_pos) = line[start..].find("total:") else {
            continue;
        };
        let digits: String = line[start + total_pos + "total:".len()..]
            .trim_start()
            .chars()
            .take_while(char::is_ascii_digit)
            .collect();
        if let Ok(n) = digits.parse::<u64>() {
            sum += n;
            found = true;
        }
    }
    found.then_some(sum)
}

/// Whether a bivariate sidecar on disk was written by the CURRENT `moarstats --bivariate` run,
/// used to guard against reading a stale one a prior run left behind. Given the pre-run mtime
/// snapshot (`before`, `None` if the file was absent), the post-run snapshot (`after`, `None` if
/// absent now), and whether the pre-run delete cleared the path (`cleared`): fresh iff a file
/// exists now AND either the path was cleared before the run (so any file present was written just
/// now) or its mtime advanced past the snapshot (the delete FAILED — e.g. a locked file — but
/// moarstats still rewrote it). A present-but-unadvanced file with a failed delete is stale and
/// must not be trusted.
fn bivariate_sidecar_is_fresh(
    before: Option<std::time::SystemTime>,
    after: Option<std::time::SystemTime>,
    cleared: bool,
) -> bool {
    match after {
        Some(after) => cleared || before.is_none_or(|before| after > before),
        None => false,
    }
}

/// One parsed row of moarstats' `--bivariate` sidecar CSV.
struct BivariateRow {
    field1:      String,
    field2:      String,
    nmi:         f64,
    pearson:     Option<f64>,
    spearman:    Option<f64>,
    /// Theil's directed uncertainty coefficients (from `--bivariate-stats u`), when present:
    /// `u_2_given_1` = U(field2|field1) = MI/H(field2), `u_1_given_2` = U(field1|field2) =
    /// MI/H(field1). `None` when the `u` columns are absent (older moarstats) or the cell is
    /// empty (a cardinality-skipped or degenerate pair) — the Sankey gate then simply doesn't
    /// fire.
    u_2_given_1: Option<f64>,
    u_1_given_2: Option<f64>,
    /// Co-occurring (both non-empty) row count moarstats used to compute this pair's stats.
    /// Always written by `moarstats --bivariate` regardless of `--bivariate-stats`; a missing or
    /// unparseable value degrades to 0 (fails the support-ratio gate in `bivariate_panels`
    /// rather than letting an un-vetted pair through).
    n_pairs:     u64,
}

/// Parse a moarstats `--bivariate` sidecar CSV, looking up every column BY HEADER NAME (the
/// emitted column set varies with `--bivariate-stats`) rather than a fixed index. A row missing
/// `field1`/`field2`/`normalized_mutual_information` (always requested by `viz smart
/// --bivariate`) — or with an unparseable NMI — is skipped; `pearson`/`spearman` are `None` when
/// their columns are absent or the cell is empty/unparseable. Returns `Err` only on a file-open
/// or CSV-structure read error; a missing/empty `pearson`/`spearman` column, or a CSV with a
/// header row but no data rows, both yield `Ok` (the latter an empty `Vec`).
fn parse_bivariate_csv(path: &std::path::Path) -> CliResult<Vec<BivariateRow>> {
    let mut rdr = csv::ReaderBuilder::new()
        .has_headers(true)
        .from_path(path)?;
    let headers = rdr.headers()?.clone();
    let find = |name: &str| headers.iter().position(|h| h == name);
    let (Some(f1i), Some(f2i), Some(nmi_i)) = (
        find("field1"),
        find("field2"),
        find("normalized_mutual_information"),
    ) else {
        return Ok(Vec::new());
    };
    let pearson_i = find("pearson_correlation");
    let spearman_i = find("spearman_correlation");
    let u_2_given_1_i = find("u_field2_given_field1");
    let u_1_given_2_i = find("u_field1_given_field2");
    let n_pairs_i = find("n_pairs");

    let mut rows = Vec::new();
    for result in rdr.records() {
        let record = result?;
        let (Some(field1), Some(field2)) = (record.get(f1i), record.get(f2i)) else {
            continue;
        };
        let Some(nmi) = record.get(nmi_i).and_then(|s| s.parse::<f64>().ok()) else {
            continue;
        };
        let pearson = pearson_i
            .and_then(|i| record.get(i))
            .and_then(|s| s.parse::<f64>().ok());
        let spearman = spearman_i
            .and_then(|i| record.get(i))
            .and_then(|s| s.parse::<f64>().ok());
        let u_2_given_1 = u_2_given_1_i
            .and_then(|i| record.get(i))
            .and_then(|s| s.parse::<f64>().ok());
        let u_1_given_2 = u_1_given_2_i
            .and_then(|i| record.get(i))
            .and_then(|s| s.parse::<f64>().ok());
        let n_pairs = n_pairs_i
            .and_then(|i| record.get(i))
            .and_then(|s| s.parse::<u64>().ok())
            .unwrap_or(0);
        rows.push(BivariateRow {
            field1: field1.to_string(),
            field2: field2.to_string(),
            nmi,
            pearson,
            spearman,
            u_2_given_1,
            u_1_given_2,
            n_pairs,
        });
    }
    Ok(rows)
}

/// Build the `viz smart --bivariate` overview panels — a normalized mutual information (NMI)
/// association heatmap and, for wider datasets, a ranked "top relationships" bar — from moarstats'
/// `--bivariate` sidecar CSV (written by the `--smarter` moarstats subprocess above when
/// `--bivariate` is enabled). Soft-fails (warns, returns `(None, None)`) when the sidecar is
/// missing/unreadable or carries no usable pairs, matching `--smarter`'s own degrade-gracefully
/// convention — a `--bivariate` run never aborts the Data Schematic over this. Pairs touching a
/// `Route::Skip` (identifier/PII/free-text) column or a map lat/lon column are dropped, mirroring
/// every other per-column panel builder's exclusions. The ranked bar additionally requires
/// co-occurrence support (`BIVARIATE_MIN_SUPPORT_RATIO`) — see its comment — so it can drop to
/// `None` even when the heatmap still renders.
fn bivariate_panels(
    args: &Args,
    input: &str,
    stats: &[crate::cmd::stats::StatsData],
    col_sems: &[ColSemantics],
    is_map_col: impl Fn(usize) -> bool,
) -> (Option<Panel>, Option<Panel>) {
    let path = bivariate_csv_path(input);
    let rows = match parse_bivariate_csv(&path) {
        Ok(rows) => rows,
        Err(e) => {
            viz_note(&format!(
                "viz smart --bivariate: could not read moarstats bivariate output ({e}); skipping \
                 the association panels."
            ));
            return (None, None);
        },
    };
    if rows.is_empty() {
        viz_note(
            "viz smart --bivariate: moarstats produced no bivariate pairs; skipping the \
             association panels.",
        );
        return (None, None);
    }

    // resolve field name -> column index once
    let field_idx: HashMap<&str, usize> = stats
        .iter()
        .enumerate()
        .map(|(i, s)| (s.field.as_str(), i))
        .collect();

    struct Survivor {
        i:       usize,
        j:       usize,
        nmi:     f64,
        n_pairs: u64,
        suffix:  Option<String>,
    }
    let mut survivors: Vec<Survivor> = Vec::new();
    for row in &rows {
        let (Some(&i), Some(&j)) = (
            field_idx.get(row.field1.as_str()),
            field_idx.get(row.field2.as_str()),
        ) else {
            continue;
        };
        // identifier/PII/free-text carry no association signal; a near-unique projected coordinate
        // yields only a high-cardinality NMI artifact (e.g. `Incident Address × Y Coordinate`).
        let excluded = |r: Route| matches!(r, Route::Skip | Route::ProjectedCoord);
        if i == j || excluded(col_sems[i].route) || excluded(col_sems[j].route) {
            continue;
        }
        if is_map_col(i) || is_map_col(j) {
            continue;
        }
        // a numeric pair whose Spearman rank correlation diverges sharply from its Pearson
        // correlation is monotonic but curved (nonlinear): a single Pearson number understates
        // it, so flag it in the hover (mirrors the strongest-pair drill-down's own nonlinearity
        // note, here applied per-cell across every bivariate pair).
        let suffix = match (row.pearson, row.spearman) {
            (Some(p), Some(s)) if (s.abs() - p.abs()) > NONLINEAR_DIVERGENCE_THRESHOLD => Some(
                format!("<br>\u{26a0} Nonlinear: pearson r={p:.2}, spearman \u{3c1}={s:.2}"),
            ),
            _ => None,
        };
        survivors.push(Survivor {
            i,
            j,
            nmi: row.nmi,
            n_pairs: row.n_pairs,
            suffix,
        });
    }
    if survivors.is_empty() {
        return (None, None);
    }

    // unique labels across surviving pairs, in column order (matches CorrHeatmap's convention)
    let mut idxs: Vec<usize> = Vec::new();
    for s in &survivors {
        if !idxs.contains(&s.i) {
            idxs.push(s.i);
        }
        if !idxs.contains(&s.j) {
            idxs.push(s.j);
        }
    }
    idxs.sort_unstable();

    // label a column by its raw field name (else positional), matching CorrHeatmap's axis
    // convention. The dictionary label, when present, still shows as the muted per-column panel
    // subtitle elsewhere — it just doesn't drive the NMI axis/pair labels.
    let label_of = |idx: usize| -> String {
        if stats[idx].field.is_empty() {
            format!("col {}", idx + 1)
        } else {
            stats[idx].field.clone()
        }
    };

    let assoc_panel = if idxs.len() >= 2 {
        let n = idxs.len();
        let pos: HashMap<usize, usize> =
            idxs.iter().enumerate().map(|(p, &idx)| (idx, p)).collect();
        let mut matrix = vec![vec![f64::NAN; n]; n];
        let mut hover_suffix = vec![vec![String::new(); n]; n];
        for s in &survivors {
            let (pi, pj) = (pos[&s.i], pos[&s.j]);
            matrix[pi][pj] = s.nmi;
            matrix[pj][pi] = s.nmi;
            if let Some(suf) = &s.suffix {
                hover_suffix[pi][pj].clone_from(suf);
                hover_suffix[pj][pi].clone_from(suf);
            }
        }
        let labels: Vec<String> = idxs.iter().map(|&idx| label_of(idx)).collect();
        Some(Panel::new(
            t!("viz.title.assoc_heatmap").into_owned(),
            PanelKind::AssocHeatmap {
                labels,
                matrix: mask_to_lower_triangle(matrix),
                hover_suffix,
            },
        ))
    } else {
        None
    };

    // ranked top-N relationships, only when the heatmap has grown past the point where its
    // in-cell numeric labels turn off (CORR_INCELL_MAX_N) — below that the heatmap alone is
    // already legible, so a separate ranked view would be redundant.
    let top_relationships_panel = if idxs.len() > CORR_INCELL_MAX_N {
        // NMI alone rewards perfect co-occurrence in a narrow, sparsely-populated slice of rows
        // (e.g. two columns only ever filled in together for one complaint sub-type) just as
        // readily as it rewards a pair genuinely associated across the whole dataset. Gate the
        // RANKING (not the heatmap, which still shows every survivor pair) on co-occurrence
        // support relative to the best-populated pair among these columns, so a technically-
        // perfect but thinly-supported association can't crowd out more broadly meaningful ones.
        let max_n_pairs = survivors.iter().map(|s| s.n_pairs).max().unwrap_or(0);
        let min_support = (max_n_pairs as f64 * BIVARIATE_MIN_SUPPORT_RATIO).round() as u64;
        let top_n = args.flag_limit.max(1);
        let mut ranked: Vec<&Survivor> = survivors
            .iter()
            .filter(|s| s.n_pairs >= min_support)
            .collect();
        ranked.sort_by(|a, b| {
            b.nmi
                .partial_cmp(&a.nmi)
                .unwrap_or(std::cmp::Ordering::Equal)
        });
        ranked.truncate(top_n);
        if ranked.is_empty() {
            None
        } else {
            let labels: Vec<String> = ranked
                .iter()
                .map(|s| format!("{} \u{d7} {}", label_of(s.i), label_of(s.j)))
                .collect();
            let values: Vec<f64> = ranked.iter().map(|s| s.nmi).collect();
            // dot SIZE encodes co-occurrence support (raw n_pairs); dot COLOR flags a nonlinear
            // numeric pair (present-suffix == Pearson/Spearman diverged past the threshold).
            let supports: Vec<f64> = ranked.iter().map(|s| s.n_pairs as f64).collect();
            let nonlinear: Vec<bool> = ranked.iter().map(|s| s.suffix.is_some()).collect();
            let hover_suffix: Vec<String> = ranked
                .iter()
                .map(|s| {
                    format!(
                        "<br>n={}{}",
                        s.n_pairs,
                        s.suffix.clone().unwrap_or_default()
                    )
                })
                .collect();
            Some(Panel::new(
                t!("viz.title.top_relationships").into_owned(),
                PanelKind::TopRelationships {
                    labels,
                    values,
                    supports,
                    nonlinear,
                    hover_suffix,
                },
            ))
        }
    } else {
        None
    };

    (assoc_panel, top_relationships_panel)
}

/// Reduce one side of a Sankey to its top-`SANKEY_TOP_N_PER_SIDE` categories (by total flow
/// descending, then label ascending — the `finalize_freq_bars` comparator). Returns the kept
/// labels in that order, the kept-label set, an optional `Other (k)` label for the `k` collapsed
/// categories, and the fraction of this side's total flow that landed in the collapsed remainder.
fn sankey_side_topn(
    tot: &HashMap<String, f64>,
) -> (
    Vec<String>,
    std::collections::HashSet<String>,
    Option<String>,
    f64,
) {
    let grand: f64 = tot.values().sum();
    let mut entries: Vec<(&String, f64)> = tot.iter().map(|(k, &v)| (k, v)).collect();
    entries.sort_by(|a, b| {
        b.1.partial_cmp(&a.1)
            .unwrap_or(std::cmp::Ordering::Equal)
            .then_with(|| a.0.cmp(b.0))
    });
    // when exactly one category would be collapsed, keep it instead: it occupies the same node
    // slot as the `Other (1)` bucket would, and naming it is strictly more informative (mirrors
    // `finalize_freq_bars`). `other_count` then falls to 0, i.e. the no-remainder path.
    let keep_n = if entries.len() == SANKEY_TOP_N_PER_SIDE + 1 {
        entries.len()
    } else {
        SANKEY_TOP_N_PER_SIDE
    };
    let kept: Vec<String> = entries
        .iter()
        .take(keep_n)
        .map(|(k, _)| (*k).clone())
        .collect();
    let kept_set: std::collections::HashSet<String> = kept.iter().cloned().collect();
    let other_count = entries.len().saturating_sub(kept.len());
    let other_flow: f64 = entries.iter().skip(keep_n).map(|(_, v)| *v).sum();
    let other_label = (other_count > 0).then(|| format!("Other ({other_count})"));
    let other_frac = if grand > 0.0 { other_flow / grand } else { 0.0 };
    (kept, kept_set, other_label, other_frac)
}

/// Aggregate co-occurrence counts between two categorical columns (`s_idx` → `t_idx`) in a single
/// data pass, reduce each side to its top-N categories (remainder collapsed into an `Other (k)`
/// node), and emit the bipartite Sankey arrays: `node_labels` lists every source node first, then
/// every target node, so a value appearing in both columns still yields two distinct nodes (a
/// proper left→right flow, never a self-loop). `link_source`/`link_target` are 0-based indices into
/// `node_labels`; `link_value` is the aggregated flow. Also returns each side's `Other` flow
/// fraction for the caller's readability guard. Rows with either cell empty are skipped; returns
/// `None` when no flow remains.
#[allow(clippy::type_complexity)]
fn build_smart_sankey_arrays(
    args: &Args,
    s_idx: usize,
    t_idx: usize,
) -> CliResult<Option<(Vec<String>, Vec<usize>, Vec<usize>, Vec<f64>, f64, f64)>> {
    let (mut rdr, ..) = reader_and_headers(args)?;
    let mut pair_counts: HashMap<(String, String), f64> = HashMap::new();
    let mut src_tot: HashMap<String, f64> = HashMap::new();
    let mut tgt_tot: HashMap<String, f64> = HashMap::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let s = cell_to_string(record.get(s_idx));
        let t = cell_to_string(record.get(t_idx));
        if s.is_empty() || t.is_empty() {
            continue;
        }
        *src_tot.entry(s.clone()).or_insert(0.0) += 1.0;
        *tgt_tot.entry(t.clone()).or_insert(0.0) += 1.0;
        *pair_counts.entry((s, t)).or_insert(0.0) += 1.0;
    }
    if pair_counts.is_empty() {
        return Ok(None);
    }

    let (kept_src, kept_src_set, other_src, other_src_frac) = sankey_side_topn(&src_tot);
    let (kept_tgt, kept_tgt_set, other_tgt, other_tgt_frac) = sankey_side_topn(&tgt_tot);

    // build the bipartite node index: source nodes 0..S, target nodes S..S+T.
    let mut source_nodes = kept_src.clone();
    if let Some(o) = &other_src {
        source_nodes.push(o.clone());
    }
    let mut target_nodes = kept_tgt.clone();
    if let Some(o) = &other_tgt {
        target_nodes.push(o.clone());
    }
    let src_offset = 0usize;
    let tgt_offset = source_nodes.len();
    // Position maps are keyed by KEPT labels only (distinct real categories). The collapsed
    // "Other (k)" bucket is addressed by its fixed slot — the last node on its side — NOT by its
    // display label, so a real category literally named "Other (k)" can't collide with the
    // synthetic aggregate and silently merge their flows (leaving the real node dangling).
    let src_pos: HashMap<&str, usize> = kept_src
        .iter()
        .enumerate()
        .map(|(i, l)| (l.as_str(), src_offset + i))
        .collect();
    let tgt_pos: HashMap<&str, usize> = kept_tgt
        .iter()
        .enumerate()
        .map(|(i, l)| (l.as_str(), tgt_offset + i))
        .collect();
    let other_src_idx = other_src.as_ref().map(|_| src_offset + kept_src.len());
    let other_tgt_idx = other_tgt.as_ref().map(|_| tgt_offset + kept_tgt.len());

    // remap each observed pair onto its kept node or the collapsed Other bucket, and re-aggregate.
    let mut links: HashMap<(usize, usize), f64> = HashMap::new();
    for ((s, t), w) in &pair_counts {
        let si = if kept_src_set.contains(s) {
            src_pos.get(s.as_str()).copied()
        } else {
            other_src_idx
        };
        let ti = if kept_tgt_set.contains(t) {
            tgt_pos.get(t.as_str()).copied()
        } else {
            other_tgt_idx
        };
        let (Some(si), Some(ti)) = (si, ti) else {
            continue;
        };
        *links.entry((si, ti)).or_insert(0.0) += *w;
    }
    if links.is_empty() {
        return Ok(None);
    }

    let node_labels: Vec<String> = source_nodes.into_iter().chain(target_nodes).collect();
    // deterministic link order (sorted by source then target index) for reproducible output.
    let mut link_keys: Vec<(usize, usize)> = links.keys().copied().collect();
    link_keys.sort_unstable();
    let (mut link_source, mut link_target, mut link_value) = (Vec::new(), Vec::new(), Vec::new());
    for key in link_keys {
        link_source.push(key.0);
        link_target.push(key.1);
        link_value.push(links[&key]);
    }
    Ok(Some((
        node_labels,
        link_source,
        link_target,
        link_value,
        other_src_frac,
        other_tgt_frac,
    )))
}

/// The concept FAMILY of a dictionary concept token — the segment before the first `.`
/// (`geo.city` → `geo`, `category.status` → `category`, `nyc.complaint_type` → `nyc`). Returns
/// `None` for an empty or `unknown` concept (an un-tagged column), so the caller treats it as
/// having no semantic signal and falls back to the statistical ranking.
fn sankey_concept_family(concept: &str) -> Option<&str> {
    let base = concept.trim();
    if base.is_empty() {
        return None;
    }
    let fam = base.split_once('.').map_or(base, |(f, _)| f);
    (fam != "unknown").then_some(fam)
}

/// Semantic ranking tier for a candidate Sankey pair given both columns' dictionary concepts
/// (lower = preferred). Additive over the statistical gate: when either concept is absent this
/// returns the neutral tier, so a dictionary-less Data Schematic ranks purely on the statistics.
/// - Tier 0 (preferred): a CROSS-FAMILY bridge (e.g. `nyc.complaint_type` → `category.status`,
///   `org.agency` → `category.status`) — the pair spans two different concept families, which is
///   what makes a flow tell a story.
/// - Tier 2 (deprioritized): BOTH sides are geo-context (`geo.*`) — a Sankey between two spatial
///   partitions shows administrative overlap, not a meaningful flow, so it only wins when nothing
///   better exists.
/// - Tier 1 (neutral): same-family, or one/both sides un-tagged.
fn sankey_semantic_tier(concept_a: &str, concept_b: &str) -> u8 {
    match (
        sankey_concept_family(concept_a),
        sankey_concept_family(concept_b),
    ) {
        (Some("geo"), Some("geo")) => 2,
        (Some(a), Some(b)) if a != b => 0,
        _ => 1,
    }
}

/// `viz smart --bivariate` directed-flow (Sankey) panel selection. From moarstats' bivariate
/// sidecar, pick the best pair of categorical dimensions to draw as a flow diagram: both columns
/// must be in `eligible` (the same String freq-bar dimension pool the hierarchy panel uses), and
/// the pair's directed uncertainty coefficients (Theil's U) must sit in the many-to-many band in
/// BOTH directions — `min(U) ≥ SANKEY_U_MIN` (genuinely associated, not independent) and
/// `max(U) ≤ SANKEY_U_MAX` (neither side near-functionally determines the other, which would be a
/// part-to-whole treemap). Thinly-supported pairs are dropped via the same relative
/// `BIVARIATE_MIN_SUPPORT_RATIO` guard the top-relationships bar uses. The winner (strongest
/// two-way association) is oriented source = higher-cardinality dimension → target =
/// lower-cardinality (many types flowing into fewer buckets), then a single aggregation pass builds
/// the ribbons. Returns the panel plus the chosen `(source_idx, target_idx)` so the caller can keep
/// the treemap hierarchy mutually exclusive with it. Cache-only until a pair passes; soft-fails to
/// `None` (with a note where a candidate was found but rejected) matching the other builders.
fn sankey_panel(
    args: &Args,
    input: &str,
    stats: &[crate::cmd::stats::StatsData],
    col_sems: &[ColSemantics],
    eligible: &std::collections::HashSet<usize>,
    is_map_col: impl Fn(usize) -> bool,
) -> CliResult<Option<(Panel, (usize, usize))>> {
    let path = bivariate_csv_path(input);
    let rows = match parse_bivariate_csv(&path) {
        Ok(rows) => rows,
        // the association panels already warned on a bad sidecar; stay quiet here.
        Err(_) => return Ok(None),
    };
    if rows.is_empty() {
        return Ok(None);
    }

    let field_idx: HashMap<&str, usize> = stats
        .iter()
        .enumerate()
        .map(|(i, s)| (s.field.as_str(), i))
        .collect();

    let label_of = |idx: usize| -> String {
        let sem = &col_sems[idx];
        if !sem.label.is_empty() {
            sem.label.clone()
        } else if stats[idx].field.is_empty() {
            format!("col {}", idx + 1)
        } else {
            stats[idx].field.clone()
        }
    };

    struct Cand {
        i:           usize,
        j:           usize,
        // semantic ranking tier from the dictionary concepts (lower = preferred; neutral without a
        // dictionary). See `sankey_semantic_tier`. Primary sort key so a concept-meaningful flow
        // outranks a geo-context spatial-overlap pair even if the latter's U is more central.
        tier:        u8,
        // distance of the pair's mean directed U from the band center. A Sankey is most compelling
        // when the mapping is genuinely many-to-many (ribbons cross) — that is the MIDDLE of the
        // band, not the top: a pair whose U sits near `SANKEY_U_MAX` is near-functional (almost
        // parallel ribbons, little crossing), which the treemap/hierarchy conveys just as well. So
        // candidates are ranked by proximity to the band center, not by maximum association.
        center_dist: f64,
        n_pairs:     u64,
    }
    let u_center = f64::midpoint(SANKEY_U_MIN, SANKEY_U_MAX);
    let mut cands: Vec<Cand> = Vec::new();
    for row in &rows {
        let (Some(&i), Some(&j)) = (
            field_idx.get(row.field1.as_str()),
            field_idx.get(row.field2.as_str()),
        ) else {
            continue;
        };
        if i == j || !eligible.contains(&i) || !eligible.contains(&j) {
            continue;
        }
        if is_map_col(i) || is_map_col(j) {
            continue;
        }
        // need both directed uncertainty coefficients (a cardinality-skipped or degenerate pair
        // leaves them empty) sitting inside the many-to-many band.
        let (Some(u_2_1), Some(u_1_2)) = (row.u_2_given_1, row.u_1_given_2) else {
            continue;
        };
        let (u_min, u_max) = (u_2_1.min(u_1_2), u_2_1.max(u_1_2));
        if u_min < SANKEY_U_MIN || u_max > SANKEY_U_MAX {
            continue;
        }
        cands.push(Cand {
            i,
            j,
            tier: sankey_semantic_tier(&col_sems[i].concept, &col_sems[j].concept),
            center_dist: (f64::midpoint(u_2_1, u_1_2) - u_center).abs(),
            n_pairs: row.n_pairs,
        });
    }
    if cands.is_empty() {
        return Ok(None);
    }

    // relative co-occurrence support gate (mirrors the top-relationships bar): drop pairs backed by
    // a thin sliver of rows relative to the best-supported candidate.
    let max_n = cands.iter().map(|c| c.n_pairs).max().unwrap_or(0);
    #[allow(
        clippy::cast_precision_loss,
        clippy::cast_possible_truncation,
        clippy::cast_sign_loss
    )]
    let min_support = (max_n as f64 * BIVARIATE_MIN_SUPPORT_RATIO).round() as u64;
    cands.retain(|c| c.n_pairs >= min_support);
    if cands.is_empty() {
        return Ok(None);
    }

    // Rank: semantic tier first (a concept-meaningful flow beats a geo-context overlap), then the
    // statistical "most genuinely many-to-many" (mean U closest to the band center), then higher
    // co-occurrence support, then column order for determinism.
    cands.sort_by(|a, b| {
        a.tier
            .cmp(&b.tier)
            .then_with(|| {
                a.center_dist
                    .partial_cmp(&b.center_dist)
                    .unwrap_or(std::cmp::Ordering::Equal)
            })
            .then_with(|| b.n_pairs.cmp(&a.n_pairs))
            .then_with(|| (a.i, a.j).cmp(&(b.i, b.j)))
    });
    let best = &cands[0];

    // Orient source → target. Concept-aware first: a `category.status` outcome is the natural flow
    // TARGET (e.g. complaint_type → status), so the reader sees categories flowing INTO their
    // resolution. Without that signal, fall back to source = higher-cardinality dimension → target
    // = lower (many types → fewer buckets); equal cardinality falls back to column order.
    let is_status = |idx: usize| col_sems[idx].concept.trim() == "category.status";
    let (s_idx, t_idx) = if is_status(best.j) && !is_status(best.i) {
        (best.i, best.j)
    } else if is_status(best.i) && !is_status(best.j) {
        (best.j, best.i)
    } else {
        let (ci, cj) = (stats[best.i].cardinality, stats[best.j].cardinality);
        if ci > cj || (ci == cj && best.i < best.j) {
            (best.i, best.j)
        } else {
            (best.j, best.i)
        }
    };

    let Some((node_labels, link_source, link_target, link_value, other_src_frac, other_tgt_frac)) =
        build_smart_sankey_arrays(args, s_idx, t_idx)?
    else {
        return Ok(None);
    };
    if other_src_frac > SANKEY_MAX_OTHER_FRACTION || other_tgt_frac > SANKEY_MAX_OTHER_FRACTION {
        viz_note(&format!(
            "viz smart: skipping the '{} → {}' Sankey panel — after keeping the top \
             {SANKEY_TOP_N_PER_SIDE} categories per side, the collapsed 'Other' bucket dominates \
             a side (>{:.0}% of its flow), which would misrepresent the distribution.",
            label_of(s_idx),
            label_of(t_idx),
            SANKEY_MAX_OTHER_FRACTION * 100.0
        ));
        return Ok(None);
    }

    let title = format!("{} → {}", label_of(s_idx), label_of(t_idx));
    let panel = Panel::new(
        title,
        PanelKind::Sankey {
            node_labels,
            link_source,
            link_target,
            link_value,
            value_order: args.flag_sankey_value_order,
        },
    );
    Ok(Some((panel, (s_idx, t_idx))))
}

/// For line charts, order points by their x value so the connecting line is drawn in the
/// correct sequence: numeric x is sorted numerically (fixing 1, 10, 2), while categorical x
/// preserves input order. The numeric gate uses `parse_finite_f64` semantics: a "nan"/"inf"
/// literal makes the column categorical rather than poisoning the sort comparator (NaN keys
/// break total ordering, which misorders the line and can panic Rust's sort).
fn sort_line_xy(xs: Vec<String>, ys: Vec<f64>) -> (Vec<String>, Vec<f64>) {
    let Some(keys) = (!xs.is_empty())
        .then(|| {
            xs.iter()
                .map(|s| parse_finite_f64(s))
                .collect::<Option<Vec<f64>>>()
        })
        .flatten()
    else {
        return (xs, ys);
    };
    let mut pairs: Vec<(f64, String, f64)> = keys
        .into_iter()
        .zip(xs.into_iter().zip(ys))
        .map(|(k, (x, y))| (k, x, y))
        .collect();
    pairs.sort_by(|a, b| a.0.total_cmp(&b.0));
    let mut out_x = Vec::with_capacity(pairs.len());
    let mut out_y = Vec::with_capacity(pairs.len());
    for (_, x, y) in pairs {
        out_x.push(x);
        out_y.push(y);
    }
    (out_x, out_y)
}

// ===== viz smart: cache-driven auto-dashboard =====

/// A single Data Schematic panel: a column and the chart chosen for it.
struct Panel {
    name:            String,
    /// Optional secondary line shown beneath the title (smaller, muted). Carries the
    /// dictionary's human-readable label so `name` can stay the raw field name (which also
    /// feeds trace names and status messages, so it must stay plain — no markup).
    subtitle:        Option<String>,
    kind:            PanelKind,
    /// Whether this panel's VALUE axis renders logarithmic (box panels only, resolved at
    /// classification time from the cached stats + `--log-scale`; frequency bars decide from
    /// their counts at render time instead — see `panel_is_log`).
    value_log:       bool,
    /// Per-axis `(x, y)` log verdict for the RELATIONSHIP panels (scatter pair, density contour),
    /// whose two axes are both data-valued and skew independently — unlike `value_log`, which
    /// speaks for a distribution panel's single value axis (issue #4223). Resolved at panel-build
    /// time by `relationship_axis_log` from the plotted values + `--log-scale`.
    axis_log:        (bool, bool),
    /// How much this panel is worth keeping when the Data Schematic overflows `--max-charts`:
    /// per-column panels get a stats-driven `panel_interest` score; overview panels
    /// (correlation, time-series, map, …) keep the `Panel::new` default of `f64::INFINITY`
    /// so they always survive. Ties (and the no-overflow case) preserve document order.
    interest:        f64,
    /// Reverse-geocoded spatial-extent metadata for a `Map`/`Geo` panel (the 4 bounding-box
    /// corners + center, plus a consolidated jurisdiction summary). `None` for non-map panels,
    /// when geocoding was skipped (e.g. antimeridian-spanning data), or when the lookup failed
    /// (offline/missing index) — the map then renders without the overlay.
    #[cfg(feature = "geocode")]
    geo_meta:        Option<GeoMeta>,
    /// `--geojson` region boundaries + labels to overlay on a `Map`/`Geo` panel (no geocoding
    /// needed). `None` for non-map panels or when no `--geojson` was supplied.
    geojson_overlay: Option<GeoJsonOverlay>,
    /// The column's Data Dictionary (anchor id, description), populated only under
    /// `--dict-info` on HTML output (so image exports never grow icons). Drives the
    /// panel-title info icon: its hover box shows the description, and clicking jumps to the
    /// anchor's entry in the embedded Data Dictionary drawer. The anchor is computed from the
    /// RAW field name at classification time — the panel `name` can be decorated later
    /// (e.g. "foo (sampled)") and would no longer match the page's section ids.
    dict_info:       Option<(String, String)>,
    /// Index into the `stats` slice of the column this panel charts, recorded at classification
    /// time. `None` for overview panels that aren't tied to a single column. This is the ONLY
    /// sound way to rejoin a panel to its stats row: `name` is decorated after construction
    /// (shape hints, "(sampled)") and is a positional "col N" placeholder on headerless input,
    /// so neither it nor `s.field` can carry the identity. See `build_kpi_row`.
    stat_idx:        Option<usize>,
}

impl Panel {
    /// Construct a panel with no geo metadata (the common case for non-map panels). Defaults to
    /// a linear value axis and infinite `interest` (never dropped on overflow) — per-column
    /// panels override both via `with_value_log`/`with_interest`.
    fn new(name: String, kind: PanelKind) -> Self {
        Panel {
            name,
            subtitle: None,
            kind,
            value_log: false,
            axis_log: (false, false),
            interest: f64::INFINITY,
            #[cfg(feature = "geocode")]
            geo_meta: None,
            geojson_overlay: None,
            dict_info: None,
            stat_idx: None,
        }
    }

    /// Record which `stats` row this panel charts, so downstream consumers (the KPI row) can
    /// rejoin panel → stats without depending on the display title.
    fn with_stat_idx(mut self, stat_idx: usize) -> Self {
        self.stat_idx = Some(stat_idx);
        self
    }

    /// Set whether this panel's value axis is logarithmic (box panels, per `--log-scale`).
    fn with_value_log(mut self, value_log: bool) -> Self {
        self.value_log = value_log;
        self
    }

    /// Set the per-axis `(x, y)` log verdict for a relationship panel (issue #4223).
    fn with_axis_log(mut self, axis_log: (bool, bool)) -> Self {
        self.axis_log = axis_log;
        self
    }

    /// Set the panel's interestingness score used for overflow survival ranking.
    fn with_interest(mut self, interest: f64) -> Self {
        self.interest = interest;
        self
    }

    /// Attach an optional subtitle (the dictionary label) shown beneath the title.
    fn with_subtitle(mut self, subtitle: Option<String>) -> Self {
        self.subtitle = subtitle;
        self
    }

    /// Attach the column's Data Dictionary (anchor id, description) — `--dict-info` on HTML
    /// output only — which turns on the panel-title info icon.
    fn with_dict_info(mut self, dict_info: Option<(String, String)>) -> Self {
        self.dict_info = dict_info;
        self
    }

    /// The on-screen title markup: the field name, with the dictionary label as a smaller muted
    /// subtitle line when present. Plotly renders `<br>`/`<span style>` in both annotation and
    /// title text. `self.name` itself stays plain (it also feeds trace names / status messages).
    fn display_title(&self) -> String {
        self.display_title_with_icon(false)
    }

    /// `display_title`, optionally with a trailing muted ⓘ on the TITLE line (before any
    /// subtitle line) when a `--dict-info` description is attached. Used by the panel-title
    /// annotations of BOTH render paths (`panel_title_annotation`), whose hover box / click
    /// event carry the dictionary lookup.
    fn display_title_with_icon(&self, icon: bool) -> String {
        let icon_sfx = if icon && self.dict_info.is_some() {
            format!(" <span style=\"font-size:12px;color:{MUTED_COLOR}\">\u{24D8}</span>")
        } else {
            String::new()
        };
        match &self.subtitle {
            Some(sub) => format!(
                "{}{icon_sfx}<br><span style=\"font-size:11px;color:{}\">{}</span>",
                html_escape(&self.name),
                MUTED_COLOR,
                html_escape(sub)
            ),
            // escape here too, NOT at the source: `name` must stay plain (it also feeds trace
            // names and stderr status messages), so both arms escape at this render sink.
            None => format!("{}{icon_sfx}", html_escape(&self.name)),
        }
    }
}

/// An observed min/max lat/lon bounding box. For a `viz smart` map this is the CORE extent — the
/// box of the non-outlier points (geographic outliers are excluded so a few strays don't inflate
/// it); `map_extent` itself just computes min/max over whatever coordinates it's given.
#[cfg(feature = "geocode")]
#[derive(Clone, Copy)]
struct MapExtent {
    min_lat: f64,
    max_lat: f64,
    min_lon: f64,
    max_lon: f64,
}

/// One reverse-geocoded point of a map's spatial extent (a corner or the center).
#[cfg(feature = "geocode")]
struct GeoPoint {
    /// Short positional tag: "NW", "NE", "SW", "SE", or "Center".
    tag:   &'static str,
    lat:   f64,
    lon:   f64,
    /// The reverse-geocoded location, or `None` when no city matched (e.g. over open water).
    label: Option<crate::cmd::geocode::GeoLabel>,
}

/// Reverse-geocoded spatial-extent metadata attached to a `Map`/`Geo` panel.
#[cfg(feature = "geocode")]
struct GeoMeta {
    /// The CORE extent (non-outlier bounding box) — drives the (filled) overlay box, corner
    /// markers, and the tight default framing.
    extent:      MapExtent,
    /// The FULL extent (core + all geographic outliers). `Some` only when there are outliers (and
    /// the box doesn't wrap the antimeridian); drawn as a second, no-fill dotted box so the
    /// strays' span is legible alongside the core box.
    full_extent: Option<MapExtent>,
    /// The 4 core-extent corners + center, in order NW, NE, SW, SE, Center.
    points:      Vec<GeoPoint>,
    /// Consolidated one-line jurisdiction summary (e.g. "New York & New Jersey, United States"),
    /// with an outlier call-out appended when there are geographic outliers (e.g.
    /// "… — 3 outliers (Pennsylvania)"). Empty when no core point resolved to a city.
    summary:     String,
}

/// Which hierarchical part-to-whole chart a `Hierarchy` panel renders as. Auto-selected by
/// depth (shallow → `Treemap` for accurate size comparison, deep → `Sunburst` for tracing
/// structural paths), or forced via the standalone subcommands / `--hierarchy-style`.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
enum HierStyle {
    Treemap,
    Sunburst,
    Icicle,
}

/// One KPI "big number" tile in a `PanelKind::KpiRow`. `value` is the headline number (a dataset
/// summary or an aggregated measure); `format` is a plotly d3 value-format string. `gauge` carries
/// a validated `[min, max]` domain (drawn as an angular gauge) and `target` a delta reference
/// (drawn as a "vs target" delta); both are optional and gate the indicator `mode`.
#[derive(Clone, Debug)]
struct KpiTile {
    label:  String,
    value:  f64,
    format: String,
    gauge:  Option<[f64; 2]>,
    target: Option<f64>,
}

enum PanelKind {
    /// A leading full-width row of KPI "big number" tiles (plotly `Indicator` traces) summarizing
    /// the dataset — record count, field count, completeness, and the headline numeric measures.
    /// A tile shows a gauge when its dictionary supplies a validated `gauge_range`, a "vs target"
    /// delta when it supplies a `target`, else a plain number. Rendered domain-positioned so N
    /// tiles split the row width.
    KpiRow { tiles: Vec<KpiTile> },
    /// Box plot drawn from precomputed quartiles (no raw data).
    BoxStats {
        q1:     f64,
        median: f64,
        q3:     f64,
        lower:  Option<f64>,
        upper:  Option<f64>,
        mean:   Option<f64>,
    },
    /// Box plot drawn from the raw column values (via `--box-points`, or the size-based heuristic),
    /// so plotly can render true Tukey whiskers and overlay the sample points. `idx` is the source
    /// column index; the (downsampled) values are gathered in the same batched pass as `Histogram`
    /// and looked up by `idx` at render time. `points` is this panel's resolved overlay mode.
    BoxRaw { idx: usize, points: BoxPoints },
    /// Cache-quartile box for a large (> `SMART_BOX_OUTLIERS_MAX` row) column that HAS Tukey
    /// outliers. The box is drawn from the precomputed q1/median/q3 (like `BoxStats`), but its
    /// whiskers end at the observed in-fence extremes and the out-of-fence outlier values are
    /// overlaid as native box points (plotly does NOT recompute the box). `idx` keys the
    /// post-pass `OutlierStats` side table (whisker ends + the capped outlier values); the box
    /// stats are carried on the panel and `fence_low`/`fence_high` are the Tukey inner fences used
    /// to filter the single collection pass.
    BoxOutliers {
        idx:        usize,
        q1:         f64,
        median:     f64,
        q3:         f64,
        mean:       Option<f64>,
        fence_low:  f64,
        fence_high: f64,
    },
    /// Violin (KDE density silhouette + inner quartile box) drawn from the raw column values —
    /// the DEFAULT representation for a continuous column under `--violin auto` (see
    /// `wants_violin` for the guards that keep a box instead). Same raw-values contract as
    /// `BoxRaw`: `idx` keys the batched-pass value map and `points` is this panel's resolved
    /// `--box-points` overlay mode (converted to `ViolinPoints` at render time) — capped at
    /// `Outliers` unless the flag was explicit, since the KDE silhouette already shows what the
    /// size-based `all` point cloud would. A violin has no precomputed-quartile path, so a column
    /// with more than `SMART_BOX_OUTLIERS_MAX` non-null values is drawn from a deterministic
    /// sample instead: `sample_stride > 1` makes the collection pass keep every stride-th
    /// non-null value (at most `VIOLIN_SAMPLE_MAX` of them), and such panels carry NO point
    /// overlay (`points: False`) since a strided sample misses the true extremes.
    Violin {
        idx:           usize,
        points:        BoxPoints,
        sample_stride: usize,
    },
    /// Frequency bar chart; `idx` is the source column index.
    FreqBar { idx: usize },
    /// Line chart of a numeric column over a date/datetime column, sorted chronologically.
    /// Carries the precomputed (already date-sorted) x date strings and y values so the render
    /// loop stays a pure assembly step. `categorical` marks Quarter/Year-bucketed panels whose
    /// x labels ("2024-Q1", "2024") are not date-parseable: they render on a category axis
    /// (evenly spaced, no empty-period gaps) instead of the usual date axis (issue #4216).
    TimeSeries {
        y_label:     String,
        xs:          Vec<String>,
        ys:          Vec<f64>,
        categorical: bool,
    },
    /// Animated scatter of the strongest-correlated numeric pair, revealed cumulatively over time
    /// and colored by time bucket (early → late) — the temporal companion to `ScatterPair`. The
    /// static pair shows the relationship; this shows the relationship AND its drift over time in
    /// one view (the color encoding is what makes an animated 2-var scatter more informative than
    /// its static form). `xs`/`ys`/`bucket` are parallel and sorted by `bucket` ascending, so a
    /// cumulative frame is a contiguous prefix; `bucket_labels` names each time bucket in
    /// chronological order (one per distinct bucket). Axis ranges are pinned globally so nothing
    /// jumps. Rendered as its own inline Plot (slider + Play/Pause), so (like map/geo/3D) its
    /// presence forces the inline Data Schematic path and it is HTML-only (no static-export
    /// composition).
    AnimatedScatterPair {
        xs:            Vec<f64>,
        ys:            Vec<f64>,
        /// per-point time-bucket index (parallel to xs/ys, ascending)
        bucket:        Vec<usize>,
        /// chronological label per distinct bucket (drives the slider steps + color legend ends)
        bucket_labels: Vec<String>,
        x_label:       String,
        y_label:       String,
        /// the date column's label, shown as the slider's current-value prefix
        frame_label:   String,
        x_range:       (f64, f64),
        y_range:       (f64, f64),
    },
    /// Animated geographic point map: dated points on a `ScatterGeo` projection basemap, revealed
    /// CUMULATIVELY over time buckets (dated events accumulating across the map reads naturally).
    /// Only built for continental/global extents (where `viz smart` already uses a `ScatterGeo`
    /// projection, which animates natively — unlike a MapLibre tile map). Its `geo` subplot can't
    /// share the typed x/y grid, so it forces the inline render path and is HTML-only.
    /// `bucket_lats`/`bucket_lons` are parallel-per-bucket point clouds (index = bucket);
    /// `bucket_labels` names each bucket chronologically. Reuses the standalone animated-geo core.
    AnimatedGeo {
        bucket_lats:   Vec<Vec<f64>>,
        bucket_lons:   Vec<Vec<f64>>,
        bucket_labels: Vec<String>,
        frame_label:   String, // the date column's label (slider prefix)
    },
    /// Gapminder-style animated bubble chart: one bubble per ENTITY (a low-cardinality categorical
    /// dimension), each tracing a path through a 2-measure space over time, sized by a third
    /// aggregate (per-cell record count) and colored by entity. Per (entity,bucket): x/y centroid +
    /// size. Non-cartesian only in that it's inline+HTML-only (it IS an x/y scatter, but the
    /// per-frame full-replacement + per-entity trace structure needs the inline Plot path).
    /// `data` carries the `[entity][bucket]` matrices (NaN where an entity is absent in a bucket),
    /// the frame labels and the pinned axis ranges — the same payload the standalone
    /// `viz scatter --slider` path builds, so both render through
    /// [`add_bubble_traces_and_frames`].
    AnimatedBubble {
        data:        EntityBucketAgg,
        x_label:     String,
        y_label:     String,
        size_label:  String,
        frame_label: String,
    },
    /// Cyclic "seasonality" profile: a date/datetime column folded onto a repeating phase —
    /// hour-of-day, day-of-week, or month-of-year — with record volume as the radial value. Exposes
    /// periodicity (rush-hour, weekly, seasonal) that the absolute `TimeSeries` timeline hides.
    /// Drawn as a `ScatterPolar`; like `Scatter3D`, a polar subplot can't share the typed x/y grid,
    /// so it is HTML-only and rendered via the inline path. `theta` are the phase labels and `r`
    /// the matching counts, both CLOSED (first element repeated at the end) so the polar ring
    /// connects.
    CyclicProfile { theta: Vec<String>, r: Vec<f64> },
    /// Correlation heatmap over the dataset's numeric columns. Carries precomputed data (labels +
    /// matrix) so the render loop stays a pure assembly step. `spearman` records WHICH coefficient
    /// the cells hold — Spearman's rank rho on a mostly heavy-tailed table, else Pearson's r
    /// (`corr_prefers_spearman`, issue #4220) — so the hover label matches the panel title.
    CorrHeatmap {
        labels:   Vec<String>,
        matrix:   Vec<Vec<f64>>,
        spearman: bool,
    },
    /// `viz smart --bivariate`: a normalized mutual information (NMI) heatmap over every
    /// surviving column pair from moarstats' bivariate sidecar — unlike `CorrHeatmap` (Pearson,
    /// numeric-only), NMI is defined for numeric AND categorical pairs, so this can chart
    /// associations `CorrHeatmap` can't see at all. `matrix` is masked to the lower triangle
    /// (`f64::NAN` elsewhere, via `mask_to_lower_triangle`) like `CorrHeatmap`. `hover_suffix` is
    /// the same shape as `matrix`; a non-empty cell appends a nonlinearity warning (Pearson vs
    /// Spearman diverge past `NONLINEAR_DIVERGENCE_THRESHOLD`) to that pair's hover.
    AssocHeatmap {
        labels:       Vec<String>,
        matrix:       Vec<Vec<f64>>,
        hover_suffix: Vec<Vec<String>>,
    },
    /// `viz smart --bivariate`: a ranked bar of the strongest column-pair associations (by NMI,
    /// descending, capped to `--limit`) — built only when the association heatmap has more than
    /// `CORR_INCELL_MAX_N` labels (past that point the heatmap's in-cell numeric annotations turn
    /// off, so a ranked top-N view is the more legible way to see the strongest pairs). Unlike the
    /// heatmap (which shows every survivor pair), the ranking is gated on co-occurrence support
    /// (`BIVARIATE_MIN_SUPPORT_RATIO`) so a technically-perfect NMI backed by only a sparse,
    /// narrow slice of rows can't crowd out a more broadly meaningful association. `labels` are
    /// `"FieldA × FieldB"`; `hover_suffix` parallels `labels`/`values` and always leads with the
    /// pair's co-occurring row count (`<br>n=1234`), followed by the same nonlinearity-warning
    /// convention as `AssocHeatmap::hover_suffix` when applicable. Rendered as a horizontal
    /// multivariate lollipop: `values` (NMI) drives the dot's position on a zoomed value axis,
    /// `supports` (each pair's `n_pairs`, raw) the dot SIZE, and `nonlinear` the dot COLOR (amber
    /// for a Pearson/Spearman-divergent numeric pair, the panel color otherwise). All five vectors
    /// are parallel and in descending-NMI rank order.
    TopRelationships {
        labels:       Vec<String>,
        values:       Vec<f64>,
        supports:     Vec<f64>,
        nonlinear:    Vec<bool>,
        hover_suffix: Vec<String>,
    },
    /// Scatter of the most strongly correlated numeric pair — a drill-down for the correlation
    /// heatmap. Carries the two columns' precomputed, row-aligned values. `sizes`, when present,
    /// carries a third numeric column's row-aligned RAW values (the axis MOST associated with the
    /// pair, the complement of the least-redundant axis `Scatter3D` picks) to encode as marker size
    /// (bubble chart) — a 3-variable view that, unlike the HTML-only `Scatter3D`, also renders in
    /// static image export. `scale_bubble_sizes` rescales to pixels at render time. `None` keeps a
    /// plain 2-D scatter. `x_label`/`y_label`/`size_label` are the source column names, carried so
    /// the per-point hover can name each value (the cell has no axis titles) — `size_label` is
    /// `Some` iff `sizes` is.
    ScatterPair {
        xs:         Vec<f64>,
        ys:         Vec<f64>,
        sizes:      Option<Vec<f64>>,
        x_label:    String,
        y_label:    String,
        size_label: Option<String>,
    },
    /// Lorenz curve for an additive measure with income/wealth-style inequality — the plot whose
    /// geometry IS the Gini coefficient (Gini = 2× the area between the curve and the equality
    /// diagonal). Built only under `--smarter` (which populates `gini_coefficient`) for a
    /// non-negative additive measure whose cached Gini clears `LORENZ_GINI_MIN`; see
    /// `is_inequality_candidate`. `pop`/`share` are the precomputed cumulative population fraction
    /// (x, ascending) and cumulative value share (y), both 0→1 and row-aligned; the equality
    /// diagonal is the trivial (0,0)→(1,1) reference line, added as a second trace at render.
    /// `gini` is the CACHED coefficient (shown in the title, never recomputed here, so it matches
    /// the box hint / pivotp / scoresql); `label` is the measure's human label for the hover. The
    /// panel subtitle carries the framing caveat (`lorenz_caveat`) that keeps a high Gini over
    /// heterogeneous rows — and a flat run that is really a pile of zeros — from reading as an
    /// equity finding (issue #4222). A
    /// plain cartesian Scatter (lines), so it composes with the typed subplot grid and static
    /// image export.
    Lorenz {
        pop:   Vec<f64>,
        share: Vec<f64>,
        gini:  f64,
        label: String,
    },
    /// Ordered pipeline funnel over process STAGES — planned → committed → spent, impressions →
    /// clicks → conversions (issue #4222). Built ONLY from a dictionary declaration
    /// (`x-qsv.relationships`, `kind: "pipeline"`); see `PipelineSpec` for why the direction
    /// cannot be inferred from names or statistics, and `build_funnel_panel` for the encoding.
    ///
    /// All four vectors are parallel and UPSTREAM-FIRST; the render arm reverses them, because
    /// plotly places category index 0 at the axis bottom. `totals` drives the bar length.
    ///
    /// `reached` and `n_complete` mean DIFFERENT things per `shape`, which is exactly why `shape`
    /// is carried rather than inferred: for `Columns` they are complete-case counts over the
    /// listwise join (so the sums do NOT match `stats.sum`, disclosed unconditionally in the
    /// subtitle), while for the row encodings `reached` IS the per-stage row count and sums to
    /// `n_complete`. Reusing the column wording there would claim a 100% complete-case rate that
    /// means nothing.
    Funnel {
        stages:     Vec<String>,
        labels:     Vec<String>,
        totals:     Vec<f64>,
        reached:    Vec<usize>,
        n_complete: usize,
        shape:      FunnelShape,
        form:       PipelineForm,
    },
    /// 2D density contour of the most strongly correlated numeric pair — used INSTEAD of
    /// `ScatterPair` for large datasets (>= `SMART_CONTOUR_MIN_POINTS`), where a scatter overplots.
    /// Carries the precomputed bin-center axes and count grid so the render loop stays pure.
    /// `x_label`/`y_label` are the two measures' human labels, carried for the cell hover
    /// (`contour_hover_template`) the same way `ScatterPair` carries its own — the panel title is
    /// decorated with r/rho and log-scale cues, so it cannot be parsed back into axis names.
    ContourPair {
        x:       Vec<f64>,
        y:       Vec<f64>,
        z:       Vec<Vec<f64>>,
        x_label: String,
        y_label: String,
    },
    /// A numeric measure aggregated BY a low-cardinality categorical dimension (e.g. mean amount by
    /// region) — the (dimension, measure) pair with the strongest association (correlation ratio
    /// η²) above `MEASURE_BY_DIM_MIN_ETA_SQUARED`. Bars are sorted by aggregated value descending
    /// and capped to the top-N (`--limit`). The title carries the chosen pair, aggregation, and
    /// η². A plain `Bar`, so it composes with the typed subplot grid and static image export.
    MeasureByDim {
        labels: Vec<String>,
        values: Vec<f64>,
    },
    /// Grouped violin (one KDE density silhouette per category) showing the DISTRIBUTION of a
    /// numeric measure split by a low-cardinality categorical dimension — the distribution
    /// companion to `MeasureByDim`'s aggregate bar. Carries its own collected data (parallel
    /// arrays: `groups[i]` is the category label for `values[i]`), gathered in one data pass at
    /// build time rather than via the batched `hist` map. `points` is the resolved `--box-points`
    /// overlay mode. A plain `Violin::new_xy`, so it composes with the typed subplot grid and
    /// static image export.
    GroupedViolin {
        groups: Vec<String>,
        values: Vec<f64>,
        points: BoxPoints,
    },
    /// 3D scatter of the three numeric columns that form the strongest-correlation triple — a
    /// spatial drill-down for the correlation heatmap. Carries the three columns' precomputed,
    /// row-aligned values plus their axis labels. Like `Map`, a 3D `scene` doesn't compose with the
    /// typed x/y subplot grid, so a Data Schematic containing this panel always renders via the
    /// inline path.
    Scatter3D {
        xs:     Vec<f64>,
        ys:     Vec<f64>,
        zs:     Vec<f64>,
        labels: (String, String, String),
    },
    /// Histogram of a continuous numeric column, chosen INSTEAD of a box plot when moarstats
    /// flagged the column as bimodal/multimodal (a box plot would hide the multiple peaks).
    /// `idx` is the source column index; the (downsampled) values are gathered in a single
    /// batched pass and looked up by `idx` at render time — same side-table pattern as `FreqBar`.
    Histogram { idx: usize },
    /// Geographic point map over an auto-detected latitude/longitude column pair. Carries the
    /// precomputed, row-aligned coordinates (already downsampled to `MAX_SMART_POINTS`). `density`
    /// requests a heatmap render instead of discrete markers when the source had many rows.
    /// MapLibre `map` subplots don't compose with the typed x/y subplot grid, so a Data Schematic
    /// containing this panel always renders via the inline path.
    /// `outlier_lats`/`outlier_lons` carry the geographic outliers (far from the cluster centroid),
    /// drawn as a distinct marker trace on top of the core points.
    Map {
        lats:               Vec<f64>,
        lons:               Vec<f64>,
        density:            bool,
        /// Whether the CORE marker trace is cluster-ELIGIBLE, resolved from the `--cluster` mode
        /// (see `ClusterMode`): under `auto`, set for a dense plain-marker core (no `density`
        /// heatmap, no bubble-size `sizes`, >= `SMART_CLUSTER_MIN_POINTS` points); `on` sets it
        /// for any marker core; `off` clears it. When set, the core bakes `cluster.enabled
        /// = false` (the map OPENS on individual points) and a single "Clusters/Points" toggle
        /// button is added to switch clustering on/off. Never applied to the outlier
        /// call-out trace, and always mutually exclusive with `density` (a heatmap has no
        /// markers to cluster).
        cluster:            bool,
        outlier_lats:       Vec<f64>,
        outlier_lons:       Vec<f64>,
        /// Pre-rendered per-point hover label (row-aligned to `lats`/`lons`): the row's identifier
        /// (and, under `--smarter`, a dictionary/stats-chosen field combination). No coordinate
        /// line — render appends it via the trace-level `MAP_HOVER_TEMPLATE`, keeping coordinates
        /// out of every embedded string. Empty when no identifier could be chosen, in which case
        /// render falls back to plotly's default lat/lon hover. Attached via `text_array` +
        /// `hover_template` (see `map_hover_text_values`).
        hover_text:         Vec<String>,
        /// Per-point hover labels for the outlier markers (aligned to
        /// `outlier_lats`/`outlier_lons`).
        outlier_hover_text: Vec<String>,
        /// When `Some` (only under `--smarter` with a dictionary-tagged map measure —
        /// `MAP_MEASURE_CONCEPTS`), the row-aligned RAW measure value per CORE point, encoded as
        /// marker size (bubble map). `scale_bubble_sizes` rescales to pixels at render time;
        /// missing values draw at the smallest size. Never applied to the outlier trace or
        /// in density mode.
        sizes:              Option<Vec<f64>>,
        /// MapLibre pan constraint (`layout.map.bounds`), computed from the FULL coordinate cloud
        /// BEFORE `lats`/`lons` are downsampled — so a stride-dropped spatial extreme can't shrink
        /// the box below the true extent that the "Full extent" zoom button and the geocoded
        /// extent overlay reference. `None` for near-global / antimeridian extents (see
        /// `map_pan_bounds`), where no constraint is applied.
        pan_bounds:         Option<MapBounds>,
        /// `--photos` only: each point's image URLs, `|`-joined, row-aligned to `lats`/`lons`
        /// (and `outlier_photos` to `outlier_lats`/`outlier_lons`). Emitted as the trace's
        /// `customdata`, which the hover-dwell lightbox JS reads; an empty string means that row
        /// has no photo and dwelling on it does nothing. BOTH vectors are empty when `--photos`
        /// is off, and no `customdata` is emitted at all — so a default Data Schematic embeds no
        /// third-party URLs and its viewers' browsers request nothing off-origin.
        photos:             Vec<String>,
        outlier_photos:     Vec<String>,
        /// Join key to the data viewer drawer: each point's 0-based DATA-row ordinal as a string,
        /// row-aligned to `lats`/`lons` (and `outlier_row_ids` to the outlier coords), emitted as
        /// the trace's plotly `ids`. An EMPTY string means the row exists but sits past the
        /// drawer's `--preview-threshold` prefix, so it has no row to select — distinct from a
        /// trace with no `ids` at all (the extent/geojson overlays), which is not selectable.
        /// BOTH vectors are empty when the cross-link is off (image export, or the drawer
        /// disabled), and no `ids` is emitted at all.
        row_ids:            Vec<String>,
        outlier_row_ids:    Vec<String>,
    },
    /// Geographic point map drawn on a `ScatterGeo` projection basemap (coastlines/land/countries,
    /// no network tiles) instead of a MapLibre tile map — used for `viz smart` when the coordinates
    /// span a continental/global extent. Like `Map`, the `geo` subplot doesn't compose with the
    /// typed x/y grid, so a Data Schematic containing this panel always renders via the inline
    /// path. `outlier_lats`/`outlier_lons` carry the geographic outliers (see `Map`).
    Geo {
        lats:               Vec<f64>,
        lons:               Vec<f64>,
        outlier_lats:       Vec<f64>,
        outlier_lons:       Vec<f64>,
        /// Pre-rendered per-point hover label (row-aligned to `lats`/`lons`); see
        /// `Map::hover_text`.
        hover_text:         Vec<String>,
        /// Per-point hover labels for the outlier markers (aligned to
        /// `outlier_lats`/`outlier_lons`).
        outlier_hover_text: Vec<String>,
        /// Row-aligned RAW measure per CORE point for bubble sizing; see `Map::sizes`.
        sizes:              Option<Vec<f64>>,
        /// Data viewer join key; see `Map::row_ids`. Empty for the static-image `Map` -> `Geo`
        /// coercion, which renders without any JS.
        row_ids:            Vec<String>,
        outlier_row_ids:    Vec<String>,
    },
    /// Filled-region `Choropleth` aggregate drawn on the projection `geo` subplot — added beside
    /// the point Map/Geo panel when geocode resolves the coordinates to 2+ distinct countries,
    /// coloring each country by its row count. Carries precomputed `locations` (ISO-3 codes)
    /// and `z` (counts). Like `Geo`, the `geo` subplot doesn't compose with the typed x/y grid,
    /// so a Data Schematic containing this panel always renders via the inline path.
    Choropleth {
        locations:      Vec<String>,
        z:              Vec<f64>,
        location_mode:  LocationMode,
        /// User GeoJSON for a point-in-polygon (`geojson-id`) panel; `None` for the built-in
        /// geocode-derived iso3/usa-states path. Carried as a value so render does no I/O.
        geojson:        Option<serde_json::Value>,
        feature_id_key: Option<String>,
        /// Pre-rendered per-region hover label (aligned to `locations`): name+id, labeled count,
        /// share of total, rank. Attached via `hover_text_array` + `HoverInfo::Text` at render.
        hover_text:     Vec<String>,
        /// Colorbar title / measure name (e.g. "count", "median PRICE"). Parameterized so a
        /// region-code summary choropleth can label its aggregate instead of a hardcoded "count".
        measure_label:  String,
        /// `Some` only for a region-code SUMMARY choropleth (a real CSV region column exists):
        /// the region column's CSV index plus a matched-feature-id -> raw cell value(s) map
        /// (only entries where the raw spelling differs from the feature id, e.g. zero-padded
        /// zips). Drives the region-click -> data-viewer SearchBuilder filter; `None` for the
        /// geocode-derived and point-in-polygon paths, where no CSV column holds region values.
        region_filter:  Option<(usize, HashMap<String, Vec<String>>)>,
    },
    /// Filled-region choropleth drawn on a MapLibre tile basemap (`map` subplot) instead of the
    /// projection `geo` subplot — used for a `viz smart` `--geojson` point-in-polygon panel whose
    /// matched regions span a metro-scale extent (< `SMART_CHOROPLETH_MIN_SPAN_DEG` in BOTH dims),
    /// where the projection basemap's coastline/street detail is too coarse. Carries the user
    /// GeoJSON plus a precomputed center/zoom so render frames the metro area without re-scanning
    /// geometry. Like `Choropleth`, the `map` subplot doesn't compose with the typed x/y grid, so a
    /// Data Schematic containing this panel always renders via the inline path.
    ChoroplethMap {
        locations:      Vec<String>,
        z:              Vec<f64>,
        geojson:        serde_json::Value,
        feature_id_key: String,
        hover_text:     Vec<String>,
        /// Map center + zoom precomputed from the matched-region bbox union (degrees; `Center` is
        /// rebuilt at render as `Center::new(center_lat, center_lon)`).
        center_lon:     f64,
        center_lat:     f64,
        zoom:           f64,
        /// Colorbar title / measure name (e.g. "count", "median PRICE"). See `Choropleth`.
        measure_label:  String,
        /// Region column index + feature-id -> raw values map. See `Choropleth::region_filter`.
        region_filter:  Option<(usize, HashMap<String, Vec<String>>)>,
    },
    /// Categorical part-to-whole hierarchy (`Treemap` or `Sunburst`, per `style`) over 2–3
    /// nested low-cardinality dimensions. Carries the fully precomputed flat plotly arrays
    /// (`labels`/`parents`/`values` keyed by path-joined `ids`) so the render loop stays a pure
    /// assembly step. Like `Map`/`Scatter3D`, a domain-based trace doesn't compose with the typed
    /// x/y subplot grid, so a Data Schematic containing this panel always renders via the inline
    /// path.
    Hierarchy {
        style:   HierStyle,
        labels:  Vec<String>,
        parents: Vec<String>,
        values:  Vec<f64>,
        ids:     Vec<String>,
    },
    /// `viz smart --bivariate` directed-flow overview: a Sankey between two associated categorical
    /// dimensions (source on the left, target on the right), weighted by co-occurrence count. Node
    /// labels are precomputed with source nodes first, then target nodes (a bipartite index space),
    /// and links reference them by position. Non-cartesian (domain-based) like the hierarchy/map
    /// panels, so it is rendered on the inline path only (HTML output), never through
    /// `panel_trace`.
    Sankey {
        node_labels: Vec<String>,
        link_source: Vec<usize>,
        link_target: Vec<usize>,
        link_value:  Vec<f64>,
        /// Rank nodes by flow (largest at top) instead of plotly's default snap order.
        value_order: bool,
    },
    /// `viz smart` parallel-categories (parcats) overview: ribbons flowing across 3-4 associated
    /// categorical dimensions. Rows are aggregated into distinct category `tuples` (one value per
    /// dimension), weighted by `counts`. Domain-based (no cartesian axes) like the Sankey panel, so
    /// it renders on the inline path only.
    Parcats {
        dim_labels: Vec<String>,
        tuples:     Vec<Vec<String>>,
        counts:     Vec<f64>,
    },
}

/// Tukey inner fences (lower = q1 - 1.5*IQR, upper = q3 + 1.5*IQR) for a numeric column, used to
/// identify outliers for a large-dataset box panel. Prefers the values precomputed in the stats
/// cache; falls back to computing them from q1/q3/IQR. Returns `None` when the quartiles are
/// unavailable or the IQR is non-positive (a constant / near-constant column, where every value
/// would falsely flag as an outlier).
fn box_fences(s: &crate::cmd::stats::StatsData) -> Option<(f64, f64)> {
    let (q1, q3) = (s.q1?, s.q3?);
    let iqr = s.iqr.unwrap_or(q3 - q1);
    if iqr <= 0.0 {
        return None;
    }
    let lo = s.lower_inner_fence.unwrap_or(q1 - 1.5 * iqr);
    let hi = s.upper_inner_fence.unwrap_or(q3 + 1.5 * iqr);
    Some((lo, hi))
}

/// Tukey IQR multiplier applied to the distance-from-centroid distribution when flagging
/// geographic outliers. 3.0 is the "far out" (extreme-outlier) fence, deliberately stricter than
/// the 1.5 inner fence so only points GENUINELY far from the cluster flag — a broad single-region
/// distribution's edge points stay core rather than swamping the map.
const GEO_OUTLIER_DIST_IQR_MULT: f64 = 3.0;

/// Absolute floor (in centroid-scaled degrees, ~111 km/deg) for the zero-IQR point-mass fallback
/// fence in [`partition_geo_outlier_indices`]. When ≥75% of rows share one exact coordinate
/// (e.g. a geocoder's city-centroid default), `q1 == q3 == 0` and a bare `d > q3` fence would
/// flag EVERY non-duplicate point — including real addresses meters away — as a geographic
/// outlier. Requiring at least ~1° keeps same-metro scatter core while still catching genuine
/// strays (another state/country, hundreds of km out).
const GEO_OUTLIER_MIN_FENCE_DEG: f64 = 1.0;

/// Antimeridian-aware median longitude. Mirrors the largest-gap unwrapping of
/// [`lon_center_and_span`] but takes the median (robust to strays) rather than a trimmed midpoint.
/// A plain nearest-rank median can land on a NON-cluster stray: a dateline cluster split across
/// +179°/-179° plus a lone point near 0° makes 0° the middle element, so the real cluster then
/// reads as ~180° away (and the stray as central) and outlier detection inverts. Unwrapping around
/// the largest empty gap places the centroid inside the cluster. Input must be ascending-sorted;
/// returns a longitude in (-180, 180]. Reduces to a plain median when the data doesn't cross the
/// antimeridian (the common case, including all single-hemisphere data).
fn circular_median_lon(sorted: &[f64]) -> f64 {
    let n = sorted.len();
    if n == 0 {
        return 0.0;
    }
    // largest gap between adjacent longitudes (the empty wedge the data does NOT cover)
    let mut max_gap = 0.0_f64;
    let mut gap_idx = 0; // gap lies between sorted[gap_idx] and sorted[gap_idx + 1]
    for i in 0..n.saturating_sub(1) {
        let gap = sorted[i + 1] - sorted[i];
        if gap > max_gap {
            max_gap = gap;
            gap_idx = i;
        }
    }
    // the wrap gap closes the circle from the easternmost point back to the westernmost
    let wrap_gap = (sorted[0] + 360.0) - sorted[n - 1];
    if wrap_gap >= max_gap {
        // doesn't cross the antimeridian -> plain median
        return sorted_quantile(sorted, 0.5);
    }
    // crosses the seam: unwrap the cluster into a contiguous ascending range (the arc east of the
    // gap, then the points west of it shifted +360), take the median, normalize back to (-180, 180]
    let mut unwrapped: Vec<f64> = Vec::with_capacity(n);
    unwrapped.extend_from_slice(&sorted[gap_idx + 1..]);
    unwrapped.extend(sorted[..=gap_idx].iter().map(|v| v + 360.0));
    let mut med = sorted_quantile(&unwrapped, 0.5);
    if med > 180.0 {
        med -= 360.0;
    }
    med
}

/// Partition row-aligned (lat, lon) pairs into a core set and a geographic-outlier set by distance
/// from the cluster centroid: a point is an outlier when its distance from the (robust, per-axis
/// median) centroid exceeds the Tukey far-out fence `q3 + 3*IQR` of all points' distances. Distance
/// is an equirectangular approximation with longitude scaled by `cos(centroid_lat)` so it's roughly
/// isotropic in degrees, and longitude deltas are wrapped across the ±180° antimeridian so a
/// dateline-straddling cluster isn't split. Unlike a per-axis percentile, this flags only points
/// genuinely far from the bulk (true strays), not the distribution's tails. When the distance IQR
/// is zero (a point-mass core, e.g. many duplicate coordinates) the fence falls back to the
/// point-mass band with an absolute floor (`max(q3, GEO_OUTLIER_MIN_FENCE_DEG)`), so a lone far
/// stray is flagged even with very few duplicates while legitimately-nearby points stay core;
/// with too few points (< 4) or a fully degenerate (all-identical) distance spread, nothing is
/// flagged and the whole set is core. Returns `(core_indices, outlier_indices)` into the input
/// slices, so callers can carry row-aligned side data (e.g. hover labels) through the same
/// partition. `partition_geo_outliers` wraps this to gather the lat/lon vecs directly.
fn partition_geo_outlier_indices(lats: &[f64], lons: &[f64]) -> (Vec<usize>, Vec<usize>) {
    let n = lats.len();
    if n < 4 {
        // too few points to characterize a cluster -> everything is core, zero outliers
        return ((0..n).collect(), Vec::new());
    }
    // robust centroid: per-axis medians (so a few strays don't drag the center toward them)
    let mut sorted_lats = lats.to_vec();
    let mut sorted_lons = lons.to_vec();
    sorted_lats.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
    sorted_lons.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
    let c_lat = sorted_quantile(&sorted_lats, 0.5);
    // antimeridian-aware median: a plain median can land on a non-cluster stray near 0° when the
    // real cluster straddles ±180°, inverting outlier detection. circular_median_lon places the
    // centroid inside the cluster; it's a no-op for single-hemisphere data.
    let c_lon = circular_median_lon(&sorted_lons);
    // scale longitude so a degree of lon counts like a degree of lat at this latitude
    let lon_scale = c_lat.to_radians().cos().abs().max(0.01);

    let dist = |lat: f64, lon: f64| -> f64 {
        let dlat = lat - c_lat;
        // wrap the longitude delta into (-180, 180] so points on opposite sides of the ±180°
        // antimeridian are measured as physically adjacent (e.g. +179° and -179° are 2° apart, not
        // 358°) — otherwise a dateline-straddling cluster's minority lobe is spuriously flagged.
        let dlon = ((lon - c_lon + 180.0).rem_euclid(360.0) - 180.0) * lon_scale;
        (dlat * dlat + dlon * dlon).sqrt()
    };

    let dists: Vec<f64> = lats
        .iter()
        .zip(lons)
        .map(|(&la, &lo)| dist(la, lo))
        .collect();
    let mut sorted_dists = dists.clone();
    sorted_dists.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
    let q1 = sorted_quantile(&sorted_dists, 0.25);
    let q3 = sorted_quantile(&sorted_dists, 0.75);
    let iqr = q3 - q1;
    // a zero IQR means the middle 50% of distances are identical — a point-mass core, common when
    // many rows share the exact same coordinate (duplicate geocodes). The Tukey fence would then
    // collapse and a scale-based fallback (mean/std) is itself inflated by the candidate stray, so
    // it misses a lone stray when there are few duplicates. Instead flag anything beyond the
    // point-mass band (`d > q3`) AND beyond an absolute floor (`GEO_OUTLIER_MIN_FENCE_DEG`):
    // `q3` is robust (unmoved by the stray), so a single far stray is caught regardless of how
    // few duplicates form the mass, while the floor keeps legitimately-nearby points (real
    // addresses scattered around a duplicated city-centroid geocode) from ALL being flagged.
    let fence = if iqr > 0.0 {
        q3 + GEO_OUTLIER_DIST_IQR_MULT * iqr
    } else {
        q3.max(GEO_OUTLIER_MIN_FENCE_DEG)
    };

    let (mut core_idx, mut out_idx) = (Vec::with_capacity(n), Vec::new());
    for (i, &d) in dists.iter().enumerate() {
        if d > fence {
            out_idx.push(i);
        } else {
            core_idx.push(i);
        }
    }
    (core_idx, out_idx)
}

/// Partition row-aligned (lat, lon) pairs into a core set and a geographic-outlier set, returning
/// `(core_lats, core_lons, outlier_lats, outlier_lons)`. Thin wrapper over
/// `partition_geo_outlier_indices` (see it for the full algorithm). Only the index form is used in
/// production (so hover labels follow the split); this lat/lon form backs the partition unit tests.
#[cfg(test)]
fn partition_geo_outliers(lats: &[f64], lons: &[f64]) -> (Vec<f64>, Vec<f64>, Vec<f64>, Vec<f64>) {
    let (core_idx, out_idx) = partition_geo_outlier_indices(lats, lons);
    let gather = |idx: &[usize]| -> (Vec<f64>, Vec<f64>) {
        (
            idx.iter().map(|&i| lats[i]).collect(),
            idx.iter().map(|&i| lons[i]).collect(),
        )
    };
    let (core_lats, core_lons) = gather(&core_idx);
    let (out_lats, out_lons) = gather(&out_idx);
    (core_lats, core_lons, out_lats, out_lons)
}

/// The panel a column is routed to once a describegpt Data Dictionary's semantic signals are
/// folded into `viz smart`. Distilled from `role` / `concept` / `content_type` by
/// `derive_semantics`. `Defer` (no usable signal, or any column absent from the dictionary) falls
/// back to the statistical `classify`, so a Data Schematic built without `--dictionary` is
/// unchanged.
#[derive(Clone, Copy, PartialEq, Eq, Debug, Default)]
enum Route {
    /// No semantic signal -> defer to the statistical heuristic (`classify`).
    #[default]
    Defer,
    /// A categorical/code field -> frequency bar, even when stored as a numeric code or when its
    /// cardinality exceeds the usual bar threshold. This is what lets administrative codes stored
    /// as numbers (ward, `police_zone`, `census_tract`, ...) become bars instead of being misread
    /// as continuous measures.
    Dimension,
    /// A genuine continuous measure -> box / histogram / correlation / time-series y. This is the
    /// positive "this IS a measure" signal the statistical heuristic cannot express.
    Measure,
    /// A date/datetime field -> handled by the time-series overview panel; no per-column
    /// distribution panel. Stops high-cardinality timestamps from becoming top-N frequency bars.
    Temporal,
    /// A latitude/longitude coordinate -> consumed by the map panel; never boxed/barred.
    MapCoord,
    /// An identifier / PII / free-text field -> not a meaningful distribution to chart; skipped.
    Skip,
    /// A projected planar coordinate (state-plane easting/northing) -> not mappable (not degrees)
    /// and never a dimension to bar. Charted as a distribution ONLY when no point map rendered,
    /// mirroring `MapCoord`'s "rather than let it vanish" fallback: when lat/lon ARE mapped the
    /// same spatial spread is already on screen, so a 1-D Tukey fence on an easting adds nothing
    /// (and its "outliers" are the edges of the extent, not bad data). Always excluded from the
    /// bivariate/association pools, where a near-unique coordinate yields only NMI artifacts.
    ProjectedCoord,
}

/// A column's resolved charting verdict: where it goes (`route`), how to aggregate it over time
/// (`agg`: `Some(Sum)` for additive counts/amounts, `Some(Mean)` for rates/ratios, `None` for the
/// trend panel's default raw value), its dictionary `concept` (kept for map-pairing / dedup / the
/// guardrail), and its human `label` (kept for panel titles). Built per column by
/// `derive_semantics`; `Agg` is the existing chart-aggregation enum, reused here.
#[derive(Clone, PartialEq, Eq, Debug, Default)]
struct ColSemantics {
    route:   Route,
    agg:     Option<Agg>,
    concept: String,
    label:   String,
}

/// One column's semantic signals parsed from a describegpt Data Dictionary.
#[derive(Clone, Debug, Default)]
struct DictRow {
    content_type: String,
    role:         String,
    concept:      String,
    label:        String,
    description:  String,
    /// Optional `[min, max]` domain for a KPI gauge tile (`x-qsv.gauge_range`), set only for
    /// numeric measures whose concept implies a canonical scale (percent, ratio, rating, score).
    /// The KPI builder still validates that the observed value falls inside the range before
    /// drawing a gauge (else it falls back to a plain number tile).
    gauge_range:  Option<[f64; 2]>,
    /// Optional KPI delta reference (`x-qsv.target`): a semantically-justified TARGET/goal (e.g.
    /// 100% completeness, 0 error-rate), never a fabricated prior-period baseline. Rendered as a
    /// "vs target" delta.
    target:       Option<f64>,
}

/// Which form a declared pipeline is drawn as (issue #4222).
///
/// The dictionary declares the SEMANTICS — that these columns are one ordered pipeline — and that
/// declaration is always honoured. It does not, and cannot, declare that the stages actually nest:
/// that is a measurable property of the data, and it decides the FORM.
///
/// A funnel is a containment chart; its band widths ARE the claim that each stage is a subset of
/// the one above. Drawing one for stages that grow renders a band wider than its predecessor — an
/// hourglass that asserts the opposite of what the numbers say, which no subtitle can retract,
/// because the geometry is read before the caption is.
#[derive(Clone, Debug, PartialEq, Eq)]
enum PipelineForm {
    /// Stage totals never increase: the containment claim holds, so the funnel is honest.
    Funnel,
    /// At least one stage total exceeds its predecessor. The panel bridges the signed differences
    /// between consecutive totals instead — same declared order, same numbers, but the form
    /// asserts arithmetic rather than containment.
    Bridge,
}

impl PipelineForm {
    /// Funnel only while the declared totals are monotonically non-increasing.
    ///
    /// Deliberately NOT the per-row containment share: that measures how many individual rows
    /// break the nesting and is reported in the subtitle, whereas this decides what the picture
    /// claims. A pipeline can have violating rows that still net out to a shrinking total (a
    /// funnel remains truthful), or perfectly nesting rows whose totals grow (it does not).
    fn for_totals(totals: &[f64]) -> Self {
        if totals.windows(2).all(|w| {
            let [a, b] = w else { return true };
            b <= a
        }) {
            Self::Funnel
        } else {
            Self::Bridge
        }
    }
}

/// How a built funnel's numbers should be read — the render arm's hover text depends on it.
///
/// Kept as data on the panel rather than re-derived, because `reached`/`n_complete` are
/// numerically indistinguishable between the shapes while meaning different things.
#[derive(Clone, Debug, PartialEq, Eq)]
enum FunnelShape {
    /// Stages are separate measure COLUMNS; counts are complete-case rows with a positive value.
    Columns,
    /// Stages are row values; `totals` are sums of the named measure column.
    RowsMeasure { value_label: String },
    /// Stages are row values; `totals` ARE the row counts, so they equal `reached`.
    RowsCount,
}

/// A pipeline declared by the dictionary (`x-qsv.relationships`, `kind: "pipeline"`), in either
/// of the two encodings describegpt emits — see `describegpt::dictionary::parse_llm_relationships`.
///
/// This is the ONLY way `viz smart` learns that a set of measures forms a process. Issue #4222
/// framed the question correctly: which columns are stages, and in what direction, is SEMANTICS,
/// not a statistic. No column-name vocabulary can settle it (`township` reads as a shipping
/// stage) and no statistic can either (`total_pop`/`male_pop`/`female_pop` nests perfectly yet is
/// a partition, not a pipeline). So the dictionary declares it, an LLM proposes it, and a human
/// can refine it.
#[derive(Clone, Debug, PartialEq, Eq)]
enum PipelineSpec {
    /// Stages are separate measure COLUMNS, listed upstream-first. Note this is the OPPOSITE
    /// direction from a `kind: "ordered"` group, which ascends.
    Columns { members: Vec<String> },
    /// Stages are VALUES of one category column. `value_column` names the measure to sum per
    /// stage; when absent the funnel counts rows.
    Rows {
        stage_column: String,
        stages:       Vec<String>,
        value_column: Option<String>,
    },
}

/// A parsed describegpt Data Dictionary: per-column semantic rows keyed by field name, plus the
/// dataset `grain` ("one row = one X"). Drives `viz smart` semantic routing.
#[derive(Clone, Debug, Default)]
struct DictData {
    rows:                HashMap<String, DictRow>,
    grain:               Option<String>,
    /// The bare entity noun behind `grain` ("311 service request"), emitted by describegpt as
    /// `x-qsv.grain_unit` and rendered VERBATIM as the subject of count-over-time chart titles.
    /// Under `--language xx` describegpt writes it in the target language, so unlike the prose
    /// `grain` it can be interpolated straight into a localized title (issue #4321). `None` for
    /// dictionaries generated before the field existed — `count_unit_from_grain` then falls back
    /// to parsing `grain`.
    grain_unit:          Option<String>,
    /// Validated temporal cadence token (`x-qsv.cadence`, describegpt's controlled vocabulary:
    /// daily/weekly/monthly/quarterly/annual). Sets the trend panel's bucket floor so e.g.
    /// quarterly data never renders with empty weekly gaps (issue #4216). Unknown tokens are
    /// ignored at the consumer (`bucket_floor_from_cadence`), mirroring the PR #4321 discipline.
    cadence:             Option<String>,
    /// Pipelines declared via `x-qsv.relationships`. Parsed leniently here (shape only); the
    /// funnel builder does the strict validation against real columns and types, mirroring how
    /// `gauge_range` is shape-parsed here and range-checked at the KPI tile.
    pipelines:           Vec<PipelineSpec>,
    /// Dataset-level prose description (jsonschema top-level `description`, produced by
    /// describegpt --description). Legacy json dictionaries don't carry one.
    dataset_description: Option<String>,
    /// Provenance blurb from the jsonschema top-level `x-qsv.generated_by` (model, timestamp).
    generated_by:        Option<String>,
    /// ISO 639-3 language code that describegpt's whatlang pass detected for the dataset
    /// (jsonschema top-level `x-qsv.detected_language_code`, e.g. "spa"). describegpt OMITS
    /// the field entirely below its 0.8 confidence threshold, so absence is normal and
    /// presence already implies confidence -- viz applies no threshold of its own. Drives the
    /// Data Schematic UI language unless `--language` overrides it.
    detected_language:   Option<String>,
}

/// Map a `concept` token (the most specific, highest-confidence signal) to a route. Returns `None`
/// for an unrecognized namespace so the caller falls through to `role`. Concepts are namespaced
/// `ns.leaf` (see `describegpt::dictionary::CONCEPT_VOCAB`).
fn route_from_concept(concept: &str) -> Option<(Route, Option<Agg>)> {
    let (ns, leaf) = concept.split_once('.').unwrap_or((concept, ""));
    let routed = match ns {
        "geo" => match leaf {
            "latitude" | "longitude" | "coordinate_pair" => (Route::MapCoord, None),
            // State-plane coordinates are continuous planar positions, not place keys: they can't
            // be mapped (not degrees) and a frequency bar of near-unique coordinates is garbage.
            // `ProjectedCoord` charts their distribution only when no point map rendered.
            "crs_stateplane_x" | "crs_stateplane_y" => (Route::ProjectedCoord, None),
            // geo *keys* (zip, census_tract, city, state, country, street_address) name a place;
            // they are dimensions to bar, never continuous measures. This is the signal that fixes
            // census_tract even when describegpt defaulted its numeric `role` to `measure`.
            _ => (Route::Dimension, None),
        },
        "time" => match leaf {
            // a duration is a SPAN (a magnitude), not a point in time: it has no place on a time
            // axis and can't be the time-series x column (`canonical_date_col` only accepts stats
            // type Date/DateTime), so routing the whole `time` namespace to Temporal dropped
            // duration columns from the Data Schematic entirely. describegpt deliberately keeps
            // `role: measure` for durations (see is_temporal_content_type in
            // cmd/describegpt/dictionary.rs, issue #4177), but concept outranks role here, so the
            // leaf has to be special-cased. Mean, not Sum: summing "days to close" across records
            // is meaningless in the way a total revenue is not.
            "duration" => (Route::Measure, Some(Agg::Mean)),
            _ => (Route::Temporal, None),
        },
        "id" | "pii" => (Route::Skip, None),
        "org" | "category" | "nyc" => (Route::Dimension, None),
        "measure" => match leaf {
            // ratios/percentages average over time; counts/amounts are additive.
            "ratio" => (Route::Measure, Some(Agg::Mean)),
            _ => (Route::Measure, Some(Agg::Sum)),
        },
        // unknown namespace (or the bare "unknown" token) -> defer to role
        _ => return None,
    };
    Some(routed)
}

/// Map a `role` token (the coarse fallback when no concept resolves) to a route.
fn route_from_role(role: &str) -> Option<(Route, Option<Agg>)> {
    let routed = match role {
        "timestamp" => (Route::Temporal, None),
        "identifier" => (Route::Skip, None),
        "dimension" => (Route::Dimension, None),
        // additive by default; a measure.* concept refines this to Mean for ratios.
        "measure" => (Route::Measure, Some(Agg::Sum)),
        _ => return None,
    };
    Some(routed)
}

/// Map a describegpt `content_type` token to a route (the legacy fallback). Date/datetime/duration
/// tokens may carry a `:<suffix>`, so match on the base token before the first `:`. Unknown/empty
/// yields `Defer` so the column falls back to statistics.
fn route_from_content_type(content_type: &str) -> (Route, Option<Agg>) {
    let base = content_type
        .split_once(':')
        .map_or(content_type, |(b, _)| b)
        .trim();
    let route = match base {
        "category" | "state" | "state_abbr" | "country" | "country_code" | "currency_code"
        | "mime_type" | "color_hex" | "industry" | "job_title" | "profession" | "time_zone" => {
            Route::Dimension
        },
        "latitude" | "longitude" => Route::MapCoord,
        "date" | "datetime" | "time" | "duration" => Route::Temporal,
        "unknown" | "" => Route::Defer,
        // identifier / PII / address / technical / free-text -> not a meaningful distribution
        _ => Route::Skip,
    };
    (route, None)
}

/// The DMY-vs-MDY parsing preference a dictionary column declares, if any (issue #4303).
///
/// A dictionary's `x-qsv.content_type` carries the column's own strftime format as a suffix
/// (`"date:%m/%d/%Y"`, `"datetime:%m/%d/%Y %I:%M:%S %p"`). `route_from_content_type` only needs the
/// base token and throws the suffix away; this reads it and turns it into the single knob
/// `qsv_dateparser::parse_with_preference` accepts.
///
/// Returns `None` — meaning "fall back to `QSV_PREFER_DMY`" — when there is no dictionary row, no
/// format suffix, the format is not a `date:`/`datetime:` one, or the layout expresses no
/// actionable preference (year-leading formats such as `%Y-%m-%d`; see
/// `strftime_dmy_preference`). Note that the preference bit only steers slash-separated `d/m`
/// values, so a declaration like `date:%d-%m-%Y` is inert by design — do not "fix" that here.
///
/// Deliberately NOT gated on the column's resolved `Route`: `derive_semantics` resolves the route
/// from `concept`, then `role`, then `content_type`, so a column carrying a `concept` never reaches
/// the `content_type` arm — yet its declared format is still authoritative for parsing.
fn dict_dmy_preference(row: Option<&DictRow>) -> Option<bool> {
    use crate::cmd::describegpt::dictionary::{content_type_date_format, strftime_dmy_preference};

    // the JSON Schema path stores `x-qsv.content_type` verbatim and the format lookup is an exact
    // `strip_prefix`, so trim before matching or a stray leading space defeats it.
    let fmt = content_type_date_format(row?.content_type.trim())?;
    strftime_dmy_preference(fmt)
}

/// Defend against describegpt's numeric `role` defaulting to `measure` (see
/// `describegpt::dictionary::coerce_role_concept`): downgrade a `Measure` verdict to a `Dimension`
/// (bar) when the column is really a discrete code or ordinal scale — few distinct integer values
/// spread over many rows — rather than a continuous quantity. Only ever touches an integer
/// `Measure`; a non-integer or a (near-)unique column is trusted as-is.
///
/// Two thresholds, because the trust owed to the signal differs:
/// - A *role-defaulted* measure (no `measure.*` concept) is only weakly a measure, so any
///   categorical-cardinality integer code (ward, `police_zone`, `census_tract`, ...) is barred, up
///   to `CATEGORICAL_MAX_CARDINALITY`.
/// - An *explicit* `measure.*` concept is a considered LLM verdict, so it is trusted for anything
///   with real spread — EXCEPT a tiny fixed integer scale (`RATING_MAX_CARDINALITY`). The measure
///   concept vocab has no `rating`/`ordinal` leaf, so a 1-5 satisfaction score lands on
///   `measure.count`/`measure.amount` and would otherwise box; a box plot of a handful of discrete
///   levels puts quartiles on category boundaries and turns every value into an "outlier" point, so
///   a frequency bar of the levels is far more honest. (Tradeoff: a genuine small-range count
///   tagged `measure.count`, e.g. items-per-order 1-6, is also charted as a bar — which is arguably
///   an improvement anyway.)
fn guardrail(mut sem: ColSemantics, s: &crate::cmd::stats::StatsData) -> ColSemantics {
    // a zero-padded numeric code (zip/FIPS/ICD-9, per stats' zero_padded_numeric detection) can
    // never be a measure, whatever the dictionary said: leading zeros only survive in codes,
    // which is harder evidence than an LLM role/concept tag. The flag is only ever emitted for
    // String-typed columns (`is_zpn` in stats.rs requires `typ == TString` — padded integer
    // codes AND zero-padded decimal codes like `007.1` both infer as String), so without this
    // downgrade a dictionary-mistagged code column wouldn't box — it would be DROPPED entirely
    // (`classify_measure` finds no quartiles on a String column); with it, the column charts as
    // the frequency bar it deserves.
    if sem.route == Route::Measure && s.zero_padded_numeric == Some(true) {
        sem.route = Route::Dimension;
        return sem;
    }
    if sem.route != Route::Measure || s.r#type.as_str() != "Integer" {
        return sem;
    }
    let ratio = s.uniqueness_ratio;
    if ratio.is_some_and(|r| r > 0.95) {
        return sem; // genuinely (near-)continuous, e.g. a monetary integer
    }
    // `<=` (not `<`) so a rating that lands exactly on the boundary still bars: a 1-5 scale over
    // 100 rows or a 0-10 scale over 220 rows both give a uniqueness ratio of exactly 0.05.
    let spread_over_many = ratio.is_some_and(|r| r <= 0.05);
    let explicit_measure = sem.concept.starts_with("measure.");
    // role-defaulted numeric measure on a categorical-cardinality integer code -> bar.
    if !explicit_measure && s.cardinality <= CATEGORICAL_MAX_CARDINALITY && spread_over_many {
        sem.route = Route::Dimension;
        return sem;
    }
    // a tiny fixed integer scale (ordinal rating) -> bar, even with an explicit measure.* concept.
    if s.cardinality <= RATING_MAX_CARDINALITY && spread_over_many {
        sem.route = Route::Dimension;
    }
    sem
}

/// Recognize an *intensive* (non-additive) measure — temperature, percentages/ratios, indices or
/// scores, elevation, per-capita/density figures, and pre-aggregated (average/median) columns —
/// from its human label and field name. Summing these across a group is meaningless (you can't add
/// °C from two cities), so `viz smart` averages them instead. Additive amounts/counts (revenue,
/// population, packages) are NOT matched and keep their Sum aggregation. describegpt's `measure.*`
/// concept vocab (count/amount/ratio) is too coarse to express this distinction, so key off the
/// label/field text. Deliberately conservative: bare "rate" is omitted because it is a substring of
/// additive names like "generated"/"aggregate", and `pct`/`ratio`/`percent` already cover most
/// proportion columns.
///
/// A COUNT is additive by definition regardless of what it counts, so a count-named column
/// (`review_score_count`, `reviewScoreCount`, `index_count`) is never intensive even when it embeds
/// an intensive token like "score" or "index" — the count guard wins. That guard is **token-aware**
/// (`count` must be a whole token, split on non-alphanumeric boundaries AND camelCase transitions)
/// so it catches snake_case, spaced, and camelCase names alike while NOT suppressing genuinely
/// intensive fields that merely embed the letters, e.g. `discount_pct`, `account_score`,
/// `county_index`.
/// Tokenize `"<label> <field>"` on non-alphanumeric boundaries AND lower/digit -> upper
/// (camelCase/PascalCase) transitions, lower-cased, so a token is recognized in `order_count`,
/// `Order Count`, and `reviewScoreCount` alike — without matching the substring inside
/// `discount`/`account`/`county`.
fn field_name_tokens(label: &str, field: &str) -> Vec<String> {
    let raw = format!("{label} {field}");
    let mut tokens: Vec<String> = Vec::new();
    let mut cur = String::new();
    let mut prev_lower_or_digit = false;
    for c in raw.chars() {
        if !c.is_alphanumeric() {
            if !cur.is_empty() {
                tokens.push(std::mem::take(&mut cur));
            }
            prev_lower_or_digit = false;
            continue;
        }
        if c.is_uppercase() && prev_lower_or_digit && !cur.is_empty() {
            tokens.push(std::mem::take(&mut cur));
        }
        cur.push(c.to_ascii_lowercase());
        prev_lower_or_digit = c.is_ascii_lowercase() || c.is_ascii_digit();
    }
    if !cur.is_empty() {
        tokens.push(cur);
    }
    tokens
}

/// True when the column's NAME marks it as an identifier/code/key rather than a distributable
/// amount — an `id`/`code`/`key`/`uuid`/`guid`/`identifier` token (`customer_id`, `productCode`,
/// `order_key`, …). Token-boundary matched (see `field_name_tokens`) so `grid`, `medicaid`,
/// `decode`, and `county` don't trip it. A trailing digit run is stripped per token first, so
/// numbered twins (`customer_id2`, `code1`, `productCode2` → `id`/`code`) are caught too. Used on
/// the stats-only inequality path to keep a skewed numeric ID/code out of the Lorenz overview (a
/// dictionary-routed measure bypasses this).
fn is_identifier_name(label: &str, field: &str) -> bool {
    field_name_tokens(label, field).iter().any(|t| {
        matches!(
            t.trim_end_matches(|c: char| c.is_ascii_digit()),
            "id" | "ids" | "code" | "codes" | "key" | "keys" | "uuid" | "guid" | "identifier"
        )
    })
}

fn is_intensive_measure(label: &str, field: &str) -> bool {
    let hay = format!("{label} {field}").to_ascii_lowercase();
    let tokens = field_name_tokens(label, field);
    let is_count = tokens
        .iter()
        .any(|t| matches!(t.as_str(), "count" | "counts" | "cnt" | "num" | "tally"))
        || hay.contains("number of");
    if is_count {
        return false;
    }
    // Matched as whole TOKENS, not substrings. A substring test here is actively wrong: "ratio"
    // is a substring of the entire `-ration` word family, so `duration`, `generation`,
    // `operation`, `registration` and `remuneration` all matched and were silently downgraded
    // from Sum to Mean. (This is the same hazard the doc comment above already records for bare
    // "rate".) Plural/derived forms are listed explicitly rather than inferred, so `indices` and
    // `densities` work without a stemming rule that would reopen the substring problem.
    const INTENSIVE_TOKENS: &[&str] = &[
        "temperature",
        "temperatures",
        "celsius",
        "fahrenheit",
        "average",
        "averages",
        "avg",
        "median",
        "medians",
        "percent",
        "percentage",
        "percentages",
        "pct",
        "ratio",
        "ratios",
        // A rate is a per-unit quantity — summing "failure_rate" across a group produces a number
        // with no meaning. Token-matched like the rest, so the additive `generation`/`migration`
        // family is untouched, and so is a `rating` (a different word, tokenized whole).
        "rate",
        "rates",
        "index",
        "indexes",
        "indices",
        "score",
        "scores",
        "elevation",
        "elevations",
        "altitude",
        "altitudes",
        "depth",
        "depths",
        "density",
        "densities",
        // Logarithmic and other non-linear scales, where even the SUM's unit is meaningless:
        // adding two Richter magnitudes does not describe anything. `magnitude` joins the
        // physical-measurement group above because it fails the same way -- a dictionary that
        // tags it `role: measure` (as an LLM readily does, magnitude being a number with no
        // obvious rate/percent tell in its name) would otherwise yield "Total Earthquake
        // Magnitude", which is exactly the KPI this guard exists to prevent.
        "magnitude",
        "magnitudes",
        // Durations and ages/tenures. A per-record SPAN is not additive the way an amount is:
        // "Total Account Age (Days)" sums a per-customer state into a number that means nothing,
        // and the same applies to elapsed/latency/runtime style measures. This also keeps the
        // label heuristic consistent with `route_from_concept`, which already routes the
        // `time.duration` concept to Mean rather than Sum.
        //
        // Token matching is load-bearing here: "age" is a SUBSTRING of agency, package, storage,
        // usage, message, coverage and mileage, every one of which is additive. Matching these as
        // substrings would repeat exactly the `ratio`-inside-`duration` bug fixed above.
        "age",
        "ages",
        "tenure",
        "seniority",
        "duration",
        "durations",
        "elapsed",
        "runtime",
        "uptime",
        "latency",
    ];
    // phrases and symbols can't survive tokenization (which splits on non-alphanumerics), so
    // these stay substring tests — none of them is a substring of a common additive word.
    const INTENSIVE_SUBSTRINGS: &[&str] = &["\u{b0}c", "\u{b0}f", "per capita", "per_capita"];
    tokens
        .iter()
        .any(|t| INTENSIVE_TOKENS.contains(&t.as_str()))
        || INTENSIVE_SUBSTRINGS.iter().any(|kw| hay.contains(kw))
}

/// Distill a column's `StatsData` + optional dictionary `row` into one charting verdict.
///
/// Precedence: **concept -> role -> `content_type` -> statistics**. `concept` is the most specific,
/// highest-confidence signal (it fixes a numeric admin code that describegpt defaulted to
/// `role: measure`); `role` is the coarse fallback; `content_type` is the legacy mapping; and a
/// column with no usable signal `Defer`s to `classify`. A guardrail wraps every `Measure` verdict.
/// With no dictionary `row`, returns the default (`Defer`) so stats-only behavior is unchanged.
fn derive_semantics(s: &crate::cmd::stats::StatsData, row: Option<&DictRow>) -> ColSemantics {
    let Some(row) = row else {
        return ColSemantics::default();
    };
    let label = row.label.trim().to_string();
    let concept = row.concept.trim();
    let make = |route: Route, agg: Option<Agg>| {
        // An intensive measure (temperature, rate, index, pre-averaged value) tagged additive by
        // the coarse measure.* concept vocab must be averaged, not summed, across a group. An
        // explicit `measure.count` is additive by definition (a count sums, never averages), so
        // it is never downgraded even if its label embeds an intensive token.
        let agg = if route == Route::Measure
            && agg == Some(Agg::Sum)
            && concept != "measure.count"
            && is_intensive_measure(&label, &s.field)
        {
            Some(Agg::Mean)
        } else {
            agg
        };
        guardrail(
            ColSemantics {
                route,
                agg,
                concept: concept.to_string(),
                label: label.clone(),
            },
            s,
        )
    };

    // 1. concept — most specific
    if !concept.is_empty()
        && concept != "unknown"
        && let Some((route, agg)) = route_from_concept(concept)
    {
        return make(route, agg);
    }
    // 2. role — coarse fallback
    if let Some((route, agg)) = route_from_role(row.role.trim()) {
        return make(route, agg);
    }
    // 3. content_type — legacy mapping
    let ct = row.content_type.trim();
    if !ct.is_empty() && ct != "unknown" {
        let (route, agg) = route_from_content_type(ct);
        return make(route, agg);
    }
    // 4. statistics floor — carry the label so panel titles still benefit
    ColSemantics {
        route: Route::Defer,
        agg: None,
        concept: String::new(),
        label,
    }
}

/// The continuous-measure arm of `classify`: a box plot from precomputed quartiles, or a histogram
/// when moarstats flagged the column bimodal/multimodal. Shared by `classify` (stats path) and
/// `classify_with_semantics` (a dictionary `measure` verdict) so a measure is charted the same way
/// however it was identified. Returns `None` when the column lacks quartiles.
fn classify_measure(idx: usize, s: &crate::cmd::stats::StatsData) -> Option<PanelKind> {
    let (Some(q1), Some(median), Some(q3)) = (s.q1, s.q2_median, s.q3) else {
        return None;
    };
    if s.cardinality <= 1 {
        return None;
    }
    // a box plot hides multiple peaks; a histogram tells the truth — but only flag bimodal when
    // Sarle's BC clears the threshold AND the distribution is PLATYKURTIC (negative excess
    // kurtosis). The kurtosis guard rejects BC's well-known false positive on heavily skewed
    // UNIMODAL data: a long tail inflates BC through skewness, yet such columns are leptokurtic and
    // are far better shown as a box with its outlier points than as a one-tall-bar histogram. A
    // genuine two-peaked column is flat-topped (negative excess kurtosis), so it still becomes a
    // histogram. `bimodality_coefficient`/`kurtosis` come from moarstats (`--smarter`) or
    // `enrich_bimodality` (plain smart); when either is absent (e.g. too few rows) it's a box plot.
    if s.bimodality_coefficient
        .is_some_and(|bc| bc >= BIMODALITY_COEFFICIENT_THRESHOLD)
        && s.kurtosis.is_some_and(|k| k < 0.0)
    {
        return Some(PanelKind::Histogram { idx });
    }
    // Observed min/max as whisker endpoints (NOT Tukey fences, which are computed thresholds that
    // need not be observed values) — honest for a precomputed, no-rescan box.
    let lower = s.min.as_deref().and_then(|v| v.trim().parse::<f64>().ok());
    let upper = s.max.as_deref().and_then(|v| v.trim().parse::<f64>().ok());
    Some(PanelKind::BoxStats {
        q1,
        median,
        q3,
        lower,
        upper,
        mean: s.mean,
    })
}

/// Populate `bimodality_coefficient` for continuous-numeric columns that don't already have one —
/// the plain `viz smart` path, where the stats cache carries `skewness` but not the moarstats-only
/// `kurtosis`/`bimodality_coefficient`. Without it a bimodal column (two separated peaks) renders
/// as a box plot whose median can sit in the empty gap BETWEEN the peaks — actively misleading;
/// `classify_measure` upgrades a flagged column to a histogram instead, which shows the peaks.
///
/// Sarle's bimodality coefficient is computed EXACTLY as moarstats does — `BC = (skewness² + 1) /
/// (kurtosis + 3)` with the same qsv-stats sample excess-kurtosis definition and the SAME cached
/// `mean`/`variance`/`skewness` inputs — so a column classifies identically with or without
/// `--smarter`. The 4th central moment is accumulated in a single streaming pass (O(1) memory per
/// column, no value buffering) using each column's cached mean. Columns moarstats already enriched
/// (BC present) are skipped, so this is a no-op — with no data pass at all — under `--smarter` or
/// when no column qualifies.
fn enrich_bimodality(args: &Args, stats: &mut [crate::cmd::stats::StatsData]) -> CliResult<()> {
    // candidates: columns that would become a box plot (continuous numeric with quartiles) and lack
    // a bimodality coefficient. Mirrors classify()/classify_measure()'s box-vs-bar conditions so we
    // don't pay for columns that chart as frequency bars or are skipped as ID-like.
    let candidates: Vec<usize> = stats
        .iter()
        .enumerate()
        .filter(|(_, s)| {
            let near_unique = s.uniqueness_ratio.is_some_and(|r| r > 0.95);
            let low_card = s.cardinality <= CATEGORICAL_MAX_CARDINALITY && !near_unique;
            s.bimodality_coefficient.is_none()
                && matches!(s.r#type.as_str(), "Integer" | "Float")
                && s.cardinality > 1
                && !low_card
                && s.q1.is_some()
                && s.q2_median.is_some()
                && s.q3.is_some()
                && s.skewness.is_some()
                && s.mean.is_some()
                && s.variance.is_some_and(|v| v > 0.0)
        })
        .map(|(i, _)| i)
        .collect();
    if candidates.is_empty() {
        return Ok(());
    }

    let means: Vec<f64> = candidates
        .iter()
        .map(|&i| stats[i].mean.unwrap_or_default())
        .collect();
    let mut counts = vec![0_u64; candidates.len()];
    let mut sum4 = vec![0.0_f64; candidates.len()]; // Σ(x - mean)⁴ per candidate column

    let rconfig = Config::new(args.arg_input.as_ref())
        .delimiter(args.flag_delimiter)
        .no_headers_flag(args.flag_no_headers);
    let mut rdr = rconfig.reader()?;
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        for (k, &col) in candidates.iter().enumerate() {
            let Some(cell) = record.get(col) else {
                continue;
            };
            let cell = crate::cmd::frequency::trim_bs_whitespace(cell);
            if cell.is_empty() {
                continue;
            }
            if let Some(x) = std::str::from_utf8(cell)
                .ok()
                .and_then(|s| s.parse::<f64>().ok())
                && x.is_finite()
            {
                let d2 = (x - means[k]) * (x - means[k]);
                sum4[k] = d2.mul_add(d2, sum4[k]);
                counts[k] += 1;
            }
        }
    }

    for (k, &col) in candidates.iter().enumerate() {
        let n = counts[k];
        // qsv-stats requires >= 4 observations for a defined sample excess kurtosis.
        if n < 4 {
            continue;
        }
        let (Some(skew), Some(variance)) = (stats[col].skewness, stats[col].variance) else {
            continue;
        };
        let variance_sq = variance * variance;
        if variance_sq == 0.0 {
            continue;
        }
        #[allow(clippy::cast_precision_loss)]
        let nf = n as f64;
        // sample excess kurtosis, matching qsv-stats `kurtosis` with precalc mean+variance:
        // (n(n+1) Σ(x-mean)⁴) / ((n-1)(n-2)(n-3) variance²) - 3(n-1)²/((n-2)(n-3))
        let denominator = (nf - 1.0) * (nf - 2.0) * (nf - 3.0);
        let adjustment = 3.0 * (nf - 1.0) * (nf - 1.0) / ((nf - 2.0) * (nf - 3.0));
        let kurtosis =
            (nf * (nf + 1.0) * sum4[k]).mul_add(1.0 / (denominator * variance_sq), -adjustment);
        // Sarle's bimodality coefficient (moarstats `compute_bimodality_coefficient`).
        let bc = skew.mul_add(skew, 1.0) / (kurtosis + 3.0);
        if bc.is_finite() && kurtosis.is_finite() {
            // store both so classify_measure's platykurtic guard (BC high AND excess kurtosis < 0)
            // sees the same pair moarstats would have written under `--smarter`.
            stats[col].kurtosis = Some(kurtosis);
            stats[col].bimodality_coefficient = Some(bc);
        }
    }
    Ok(())
}

/// Combine a column's `ColSemantics` verdict with the statistical `classify`. The dictionary
/// verdict wins for the routes it speaks to; `Defer` falls back to `classify` — also the path for
/// every column when no `--dictionary` is given, so stats-only behavior is unchanged.
///
/// Note on `MapCoord`: the caller already skips coordinates that the map panel ACTUALLY consumed
/// (via `is_map_col`) before reaching here, so a `MapCoord` column that arrives was NOT mapped
/// (e.g. only one of lat/lon present, out-of-range values, or no map rendered). Rather than let it
/// vanish, fall back to `classify` so it's still charted as a distribution — matching the
/// no-dictionary behavior for named-but-unmappable coordinates.
///
/// `ProjectedCoord` (state-plane easting/northing) follows the same principle from the other side:
/// it can never BE the map, so it is gated on whether a point map rendered at all (`map_rendered`,
/// i.e. a lat/lon pair was resolved). With a map, the column's spatial spread is already on screen
/// in two dimensions and a 1-D box of it is redundant; without one, fall back to `classify_measure`
/// rather than let the only spatial column in the dataset vanish.
fn classify_with_semantics(
    idx: usize,
    s: &crate::cmd::stats::StatsData,
    sem: &ColSemantics,
    map_rendered: bool,
) -> Option<PanelKind> {
    match sem.route {
        Route::Defer | Route::MapCoord => classify(idx, s),
        // a categorical/code with no observed values is still nothing to chart
        Route::Dimension => (s.r#type.as_str() != "NULL" && s.cardinality >= 1)
            .then_some(PanelKind::FreqBar { idx }),
        Route::Measure => classify_measure(idx, s),
        // `classify_measure`, NOT `classify`: the latter drops near-unique INTEGER columns as
        // ID-like, and a state-plane easting/northing is integer feet — near-unique on any finely
        // spread dataset. Routing through `classify` would silently re-drop exactly the columns
        // this route exists to rescue. The dictionary already told us this is a coordinate, not a
        // key, so go straight to the continuous-distribution arm.
        Route::ProjectedCoord => (!map_rendered).then(|| classify_measure(idx, s)).flatten(),
        Route::Temporal | Route::Skip => None,
    }
}

/// Lenient shape-parse of `x-qsv.relationships` into the pipelines `viz smart` can chart.
///
/// Structural only, by design: no column-existence, type or eligibility check happens here. That
/// mirrors `xq_range`'s split — shape at parse time, validity at the consumption site — and it
/// matters because dictionaries are hand-editable. A malformed or stale declaration must degrade
/// to "no funnel", never to an error; this is a Data Schematic, not a validator.
///
/// JSONSchema-only. The legacy flat `{"fields":[…]}` dictionary has nowhere to put a row-encoded
/// declaration, so supporting it there would be half a feature.
fn xq_pipelines(v: &serde_json::Value) -> Vec<PipelineSpec> {
    let Some(rels) = v
        .get("x-qsv")
        .and_then(|x| x.get("relationships"))
        .and_then(serde_json::Value::as_array)
    else {
        return Vec::new();
    };
    let str_at = |o: &serde_json::Value, k: &str| {
        o.get(k)
            .and_then(serde_json::Value::as_str)
            .map(str::trim)
            .filter(|s| !s.is_empty())
            .map(ToString::to_string)
    };
    let str_array = |o: &serde_json::Value, k: &str| -> Vec<String> {
        o.get(k)
            .and_then(serde_json::Value::as_array)
            .map(|a| {
                a.iter()
                    .filter_map(serde_json::Value::as_str)
                    .map(str::trim)
                    .filter(|s| !s.is_empty())
                    .map(ToString::to_string)
                    .collect()
            })
            .unwrap_or_default()
    };

    let mut out = Vec::new();
    for rel in rels {
        if rel.get("kind").and_then(serde_json::Value::as_str) != Some("pipeline") {
            continue;
        }
        // `stage_column` discriminates the two encodings, exactly as on the emitting side.
        if let Some(stage_column) = str_at(rel, "stage_column") {
            let stages = str_array(rel, "stages");
            if stages.len() >= 2 {
                out.push(PipelineSpec::Rows {
                    stage_column,
                    stages,
                    value_column: str_at(rel, "value_column"),
                });
            }
        } else {
            let members = str_array(rel, "members");
            if members.len() >= 2 {
                out.push(PipelineSpec::Columns { members });
            }
        }
    }
    out
}

/// Parse a describegpt Data Dictionary into per-column semantic rows + the dataset grain.
///
/// Accepts BOTH shapes so `--dictionary <path>` works with either:
///   * a `--format jsonschema` document — `properties.<col>` carries the human label as `title` and
///     the semantic tokens in `x-qsv.{content_type,role,concept}`; the dataset grain is the
///     top-level `x-qsv.grain`. (This is what `--dictionary infer` produces.)
///   * a legacy `--format json` dictionary — `{"Dictionary":{"response":{"fields":[...]}}}` (or a
///     bare `{"fields":[...]}`); fields carry `name`/`content_type`/`label` (and role/concept if a
///     future emitter adds them). No grain.
///
/// Returns `None` when neither shape yields a usable column, so the caller degrades to the
/// statistical heuristic rather than erroring.
fn parse_dictionary_semantics(json_text: &str) -> Option<DictData> {
    let v: serde_json::Value = serde_json::from_str(json_text).ok()?;

    // JSON Schema shape: a top-level `properties` object keyed by column name.
    if let Some(props) = v.get("properties").and_then(serde_json::Value::as_object) {
        let mut rows = HashMap::with_capacity(props.len());
        for (name, prop) in props {
            if name.is_empty() {
                continue;
            }
            let xq = prop.get("x-qsv");
            let from_xq = |k: &str| {
                xq.and_then(|x| x.get(k))
                    .and_then(serde_json::Value::as_str)
                    .unwrap_or_default()
                    .to_string()
            };
            // KPI hints: `x-qsv.gauge_range` (a 2-element numeric array) and `x-qsv.target` (a
            // number). Optional; absent on legacy dictionaries and on non-measure fields.
            let xq_num = |k: &str| {
                xq.and_then(|x| x.get(k))
                    .and_then(serde_json::Value::as_f64)
            };
            let xq_range = |k: &str| -> Option<[f64; 2]> {
                let a = xq
                    .and_then(|x| x.get(k))
                    .and_then(serde_json::Value::as_array)?;
                match a.as_slice() {
                    [lo, hi] => Some([lo.as_f64()?, hi.as_f64()?]),
                    _ => None,
                }
            };
            let label = prop
                .get("title")
                .and_then(serde_json::Value::as_str)
                .unwrap_or_default()
                .to_string();
            let description = prop
                .get("description")
                .and_then(serde_json::Value::as_str)
                .unwrap_or_default()
                .to_string();
            rows.insert(
                name.clone(),
                DictRow {
                    content_type: from_xq("content_type"),
                    role: from_xq("role"),
                    concept: from_xq("concept"),
                    label,
                    description,
                    gauge_range: xq_range("gauge_range"),
                    target: xq_num("target"),
                },
            );
        }
        if rows.is_empty() {
            return None;
        }
        let top_str = |val: Option<&serde_json::Value>| {
            val.and_then(serde_json::Value::as_str)
                .map(str::trim)
                .filter(|s| !s.is_empty())
                .map(ToString::to_string)
        };
        let grain = top_str(v.get("x-qsv").and_then(|x| x.get("grain")));
        let grain_unit = top_str(v.get("x-qsv").and_then(|x| x.get("grain_unit")));
        let cadence = top_str(v.get("x-qsv").and_then(|x| x.get("cadence")));
        // describegpt appends its attribution/provenance footer to the dataset description AND
        // stores it separately in `x-qsv.generated_by` (which renders as the page footer). Strip
        // it from the description so it isn't shown twice.
        let dataset_description = top_str(v.get("description"))
            .map(|d| strip_dataset_attribution(&d))
            .filter(|s| !s.is_empty());
        let generated_by = top_str(v.get("x-qsv").and_then(|x| x.get("generated_by")));
        // describegpt writes this only when whatlang cleared its confidence threshold, so a
        // missing field means "unknown", never "English".
        let detected_language =
            top_str(v.get("x-qsv").and_then(|x| x.get("detected_language_code")));
        return Some(DictData {
            rows,
            grain,
            grain_unit,
            cadence,
            pipelines: xq_pipelines(&v),
            dataset_description,
            generated_by,
            detected_language,
        });
    }

    // Legacy plain-json dictionary shape.
    let fields = v
        .get("Dictionary")
        .and_then(|d| d.get("response"))
        .and_then(|r| r.get("fields"))
        .or_else(|| v.get("fields"))
        .and_then(serde_json::Value::as_array)?;
    let mut rows = HashMap::with_capacity(fields.len());
    for f in fields {
        let Some(name) = f.get("name").and_then(serde_json::Value::as_str) else {
            continue;
        };
        if name.is_empty() {
            continue;
        }
        let from_field = |k: &str| {
            f.get(k)
                .and_then(serde_json::Value::as_str)
                .unwrap_or_default()
                .to_string()
        };
        rows.insert(
            name.to_string(),
            DictRow {
                content_type: from_field("content_type"),
                role:         from_field("role"),
                concept:      from_field("concept"),
                label:        from_field("label"),
                description:  from_field("description"),
                // legacy plain-json dictionaries don't carry KPI gauge/target hints
                gauge_range:  None,
                target:       None,
            },
        );
    }
    (!rows.is_empty()).then_some(DictData {
        rows,
        ..Default::default()
    })
}

/// Deterministic 8-hex-digit FNV-1a hash. Stable across runs and toolchains (unlike
/// `DefaultHasher`), so tests can assert dictionary anchor ids and each Data Schematic gets a
/// reproducible dictionary-tab window name.
fn dict_short_hash(s: &str) -> String {
    let mut h: u32 = 0x811c_9dc5;
    for b in s.as_bytes() {
        h ^= u32::from(*b);
        h = h.wrapping_mul(0x0100_0193);
    }
    format!("{h:08x}")
}

/// Stable element id for a column's Data Dictionary entry — shared by the dictionary page's
/// per-column sections and the panel info icons that jump to them. Sanitized for use as an
/// HTML id / JS string literal; the hash suffix keeps sanitization collisions apart
/// (`"a b"` vs `"a_b"`) and non-ASCII names unique.
fn dict_anchor_id(col: &str) -> String {
    let mut slug: String = col
        .chars()
        .map(|c| {
            if c.is_ascii_alphanumeric() || c == '-' {
                c
            } else {
                '_'
            }
        })
        .collect();
    slug.truncate(40);
    format!("qsvdict-{slug}-{}", dict_short_hash(col))
}

/// The `--dict-info` chrome `<script>`. `qsvOpenDict(anchor)` opens the embedded Data Dictionary
/// document (the `<script type="text/html" id="qsv-dict-src">` template) in an in-page slide-over
/// The `--dict-info` payload (anchor id, description) for `field`, or None when the dictionary
/// has no usable description for it. Skips empty descriptions and a "<name> column" placeholder
/// that just echoes the field name. Callers gate `dict` on `--dict-info` + HTML output (see
/// `dict_icons` in `build_smart`), so image exports never grow icons.
fn dict_info_for_field(dict: Option<&DictData>, field: &str) -> Option<(String, String)> {
    dict.and_then(|d| d.rows.get(field))
        .map(|r| r.description.trim())
        .filter(|desc| !desc.is_empty() && !desc.eq_ignore_ascii_case(&format!("{field} column")))
        .map(|desc| (dict_anchor_id(field), desc.to_string()))
}

/// DRAWER so the plots stay visible beside it; `qsvOpenDictTab(anchor)` (the drawer's ⧉ button,
/// and the fallback when the drawer can't be built) opens the same document in a NAMED browser
/// tab instead — the title link and every info icon reuse ONE drawer / ONE tab. Both scroll to
/// (and flash-highlight, via `qsvDictReveal`) the requested per-column anchor; the tab path also
/// syncs the opener's light/dark mode. The drawer adopts the template's `.qsv-dict-wrap` +
/// `<style>` via `DOMParser` (which never executes scripts), strips the standalone page's inline
/// `onclick` handlers (they target `window.opener`, meaningless in-page), hides "View chart"
/// links with no matching `data-qsv-dict` panel element (the typed grid is one plot with no
/// per-panel elements), and handles ToC / "View chart" clicks by delegation. Opening adds
/// `body.qsv-dict-open` (a margin-right the drawer slides into) and fires a window `resize` so
/// plotly's responsive plots re-fit the narrowed column. The trailing IIFE hooks
/// `plotly_clickannotation` on the typed-grid plot AND every inline panel div, so clicking a
/// title's info icon annotation jumps to its dictionary entry; the `qsvdict-` name-prefix filter
/// keeps other captured annotations from opening the drawer. Polling mirrors
/// `FULLSCREEN_SCRIPT`: plotly renders in a deferred module script, so `gd.on` appears only
/// after render (capped so a render error can't loop forever). `__QSVDICTKEY__` is substituted
/// at page-build time so two open dashboards don't clobber each other's dictionary tab. Raw
/// string (like `SCRIPT_TEMPLATE`) so the brace-heavy JS needs no escaping.
/// Localize `DICT_SCRIPT_TEMPLATE`'s drawer-bar chrome. All three placeholders are HTML written
/// inside single-quoted JS strings, so all three use [`html_in_js_text`].
fn dict_script_template() -> String {
    DICT_SCRIPT_TEMPLATE
        .replace(
            "__QSVI18N_DICT_TITLE__",
            &html_in_js_text(&t!("viz.dict.title")),
        )
        .replace(
            "__QSVI18N_DICT_POPOUT__",
            &html_in_js_text(&t!("viz.dict.open_own_tab")),
        )
        .replace(
            "__QSVI18N_DICT_CLOSE__",
            &html_in_js_text(&t!("viz.dict.close_esc")),
        )
        // Same message as __QSVI18N_DICT_TITLE__ but in a JS-value position (setAttribute), so
        // it takes js_string_literal instead. One key, two positions -- the bar heading and the
        // accessible name must never disagree.
        .replace(
            "__QSVI18N_DICT_TITLE_JS__",
            &js_string_literal(&t!("viz.dict.title")),
        )
}

const DICT_SCRIPT_TEMPLATE: &str = r##"<script>
function qsvOpenDictTab(anchor) {
  var src = document.getElementById("qsv-dict-src");
  if (!src) return false;
  var w = window.open("", "qsv_dict___QSVDICTKEY__");
  if (!w) return false;
  var needsWrite = true;
  // reading a foreign document (user navigated the tab elsewhere) throws: rewrite in that case.
  try { needsWrite = !w.document.getElementById("qsv-dict-root"); } catch (e) {}
  try {
    if (needsWrite) { w.document.open(); w.document.write(src.textContent); w.document.close(); }
  } catch (e) {
    // cross-origin document.open also throws; fall back to a fresh unnamed tab.
    w = window.open("", "_blank");
    if (!w) return false;
    w.document.open(); w.document.write(src.textContent); w.document.close();
  }
  try {
    w.document.body.classList.toggle("qsv-dark", document.body.classList.contains("qsv-dark"));
    if (anchor) qsvDictReveal(w.document.getElementById(anchor));
  } catch (e) {}
  w.focus();
  return false;
}
// scroll a dictionary section into view and (re)run its flash highlight
function qsvDictReveal(el) {
  if (!el) return;
  el.scrollIntoView();
  el.classList.remove("qsv-dict-hl");
  void el.offsetWidth; // restart the animation when re-jumping to the same section
  el.classList.add("qsv-dict-hl");
}
// build the in-page drawer from the embedded template on first use
function qsvDictDrawer() {
  var drawer = document.getElementById("qsv-dict-drawer");
  if (drawer) return drawer;
  var src = document.getElementById("qsv-dict-src");
  if (!src || !window.DOMParser) return null;
  var doc = new DOMParser().parseFromString(src.textContent, "text/html");
  var wrap = doc.querySelector(".qsv-dict-wrap");
  if (!wrap) return null;
  var style = doc.querySelector("style");
  if (style && !document.getElementById("qsv-dict-style")) {
    var st = document.createElement("style");
    st.id = "qsv-dict-style";
    st.textContent = style.textContent;
    document.head.appendChild(st);
  }
  drawer = document.createElement("aside");
  drawer.id = "qsv-dict-drawer";
  drawer.setAttribute("aria-label", __QSVI18N_DICT_TITLE_JS__);
  var bar = document.createElement("div");
  bar.className = "qsv-dict-drawer-bar";
  bar.innerHTML = '<span>__QSVI18N_DICT_TITLE__</span><span class="qsv-dict-drawer-btns">' +
    '<a href="#" class="qsv-dict-pop" title="__QSVI18N_DICT_POPOUT__">&#x29c9;</a>' +
    '<a href="#" class="qsv-dict-hide" title="__QSVI18N_DICT_CLOSE__">&#x2715;</a></span>';
  var content = document.createElement("div");
  content.className = "qsv-dict-drawer-content qsv-dict-doc qsv-dict-embedded";
  content.appendChild(document.adoptNode(wrap));
  // the standalone page's inline handlers target window.opener; meaningless in-page
  content.querySelectorAll("[onclick]").forEach(function (el) { el.removeAttribute("onclick"); });
  // "View chart" only where a matching panel element exists (the typed grid is ONE plot
  // with no per-panel elements, so its dictionary hides the links entirely)
  content.querySelectorAll(".qsv-dict-viewchart").forEach(function (a) {
    var sel = '[data-qsv-dict="' + a.getAttribute("data-anchor") + '"]';
    if (!document.querySelector(sel)) a.style.display = "none";
  });
  // lift the whole download row (the "Export JSONSchema" link plus one per sidecar — all plain
  // <a download>, no onclick) out of the scrolling content into its own fixed strip under the
  // title bar, so it stays reachable however far the reader scrolls. It gets a strip of its own
  // rather than a seat beside the ⧉/✕ icons because up to five links don't fit on that line.
  // Moving the CONTAINER (not each anchor) is what keeps every link, not just the first; CSS then
  // swaps each full filename for its one-word tag to fit the drawer's width.
  var downloads = content.querySelector(".qsv-dict-downloads");
  drawer.appendChild(bar);
  if (downloads) drawer.appendChild(downloads);
  drawer.appendChild(content);
  document.body.appendChild(drawer);
  bar.querySelector(".qsv-dict-hide").addEventListener("click", function (e) {
    e.preventDefault();
    qsvCloseDict();
  });
  bar.querySelector(".qsv-dict-pop").addEventListener("click", function (e) {
    e.preventDefault();
    qsvCloseDict();
    qsvOpenDictTab("");
  });
  content.addEventListener("click", function (e) {
    var a = e.target && e.target.closest ? e.target.closest("a") : null;
    if (!a) return;
    if (a.classList.contains("qsv-dict-viewchart")) {
      e.preventDefault();
      var p = document.querySelector('[data-qsv-dict="' + a.getAttribute("data-anchor") + '"]');
      if (p) p.scrollIntoView({ behavior: "smooth", block: "center" });
      return;
    }
    var href = a.getAttribute("href") || "";
    if (href.charAt(0) === "#" && href.length > 1) {
      e.preventDefault();
      qsvDictReveal(document.getElementById(href.slice(1)));
    }
  });
  return drawer;
}
function qsvOpenDict(anchor) {
  var drawer = qsvDictDrawer();
  if (!drawer) return qsvOpenDictTab(anchor);
  document.body.classList.add("qsv-dict-open");
  drawer.classList.add("open");
  // plotly's responsive plots re-fit the narrowed content column on window resize
  window.dispatchEvent(new Event("resize"));
  if (anchor) qsvDictReveal(document.getElementById(anchor));
  return false;
}
function qsvCloseDict() {
  var drawer = document.getElementById("qsv-dict-drawer");
  if (drawer) drawer.classList.remove("open");
  document.body.classList.remove("qsv-dict-open");
  window.dispatchEvent(new Event("resize"));
}
document.addEventListener("keydown", function (e) {
  if (e.key === "Escape") qsvCloseDict();
});
(function () {
  var tries = 0;
  function hook() {
    // the typed-grid plot plus every inline panel div; a page only ever has one kind
    var gds = document.querySelectorAll('#qsv-viz-smart-grid, [id^="qsv-viz-panel-"]');
    var pending = false;
    gds.forEach(function (gd) {
      if (gd.__qsvDictHooked) return;
      // Wait for BOTH the plotly render (gd.on) AND the fullscreen enhancer's one-time
      // Plotly.newPlot re-render (marked gd.__qsvFs): attaching before that re-render would
      // silently lose the listener when the plot is re-created. If the enhancement never
      // completes (render error), fall back and attach anyway after ~10s so dictionary
      // clicks aren't disabled with it.
      if (!gd.on || (!gd.__qsvFs && tries < 100)) { pending = true; return; }
      gd.__qsvDictHooked = true;
      gd.on("plotly_clickannotation", function (ev) {
        var a = ev.annotation;
        if (a && a.name && a.name.indexOf("qsvdict-") === 0) qsvOpenDict(a.name);
      });
    });
    if (pending && ++tries < 200) setTimeout(hook, 100);
  }
  // published so the theme toggle can rebind after a MapLibre Plotly.newPlot re-render (which
  // silently drops gd.on listeners; the toggle clears gd.__qsvDictHooked first).
  window.__qsvDictRehook = hook;
  hook();
})();
// Open the drawer by default so the Data Dictionary is visible beside the plots on load.
// Build it directly (rather than via qsvOpenDict) so a drawer that can't be built never
// auto-pops a browser tab; the plots read the already-narrowed body width on first render.
(function () {
  var drawer = qsvDictDrawer();
  if (!drawer) return;
  document.body.classList.add("qsv-dict-open");
  drawer.classList.add("open");
  window.dispatchEvent(new Event("resize"));
})();
</script>"##;

/// The data viewer drawer chrome for `viz smart` HTML dashboards (issue #4283): a bottom drawer
/// holding the underlying rows in a `DataTable`, opened from the "(Explore)"/"(Preview)" link next
/// to the metadata row count. Static CSS for the drawer shell + the script that lazily builds it
/// on first open. `__QSVDATATITLE__` is replaced with the drawer-bar title ("all N rows" /
/// "first N of M rows (preview)") at page build.
///
/// Mirrors the Data Dictionary drawer's mechanics (`DICT_SCRIPT_TEMPLATE`: lazy build, `.open` +
/// `body.qsv-*-open` classes, a synthetic `resize` so plotly re-fits, Esc-to-close) but on the
/// bottom edge, WITHOUT the dictionary's DOMParser/style-lifting indirection — that exists only
/// because the dictionary doubles as a standalone page; this drawer is single-purpose, so its CSS
/// ships directly below. z-index 1090 sits just under the dictionary drawer (1100); the two open
/// on different axes and compose. Everything is lazy: the DataTables library (when gzip-embedded)
/// is inflated and installed only on first open, so an unused drawer costs no page-load time.
///
/// The page footer (credits + Theme toggle + logo) is in the normal flow, so `body.qsv-data-open`'s
/// `margin-bottom` lifts it clear of the open drawer on its own — no per-widget offsets.
/// The DataTables `language:` block inside [`DATA_DRAWER_SCRIPT`], captured verbatim.
///
/// §4 of the localization plan splices a vendored DataTables i18n JSON in here by replacing this
/// exact text. That has NOT happened yet, so the two `searchBuilder.button` strings are
/// deliberately still English literals rather than `t!` calls: converting them now would change
/// the very text this needle has to match, and the mismatch would be silent. Convert them in the
/// same commit that introduces the vendored JSON, and keep DataTables' own `%d` verbatim in the
/// locale values — that is a DataTables placeholder, not a rust-i18n one.
///
/// `assert_data_drawer_lang_needle_is_current` fails if anyone edits that block, so the needle can
/// never silently stop matching.
///
/// Now consumed in production by [`datatables_language_block`] as well as by the guard test.
const DT_LANG_NEEDLE: &str = r#"        language: {
          searchBuilder: { button: { 0: "Advanced Filter", _: "Advanced Filter (%d)" } }
        },"#;

/// Localize the qsv-authored strings in `DATA_DRAWER_SCRIPT`.
///
/// All placeholders sit in BARE JS-value positions, so they use [`js_string_literal`] (see
/// [`fullscreen_script`] for the convention). The DataTables-supplied strings (pagination,
/// search, "no data") are NOT here — they come from the library and stay English until the §4
/// vendored i18n JSON lands.
/// The DataTables `language:` block for the active locale, spliced over [`DT_LANG_NEEDLE`].
///
/// English returns `None` and the block in `DATA_DRAWER_SCRIPT` is left exactly as written — the
/// library's own defaults ARE English, so there is nothing to vendor and an English page keeps
/// its previous bytes. For a curated language the vendored file supplies every DataTables string
/// (pagination, search, "no matching records"), and qsv applies TWO splice-time overrides on top:
///
///  * `searchBuilder.button`, for EVERY locale. The vendored files render it as a literal "search
///    builder" ("Constructor de busqueda" in es); qsv deliberately names that control "Advanced
///    Filter" instead, because it sits alongside the per-column `ColumnControl` widgets and what
///    distinguishes it is the cross-column AND/OR logic -- so the qsv-authored name is translated
///    from qsv's own catalog, not taken from DataTables.
///  * `searchBuilder.conditions.date.{after,before}`, for `zh-CN` ONLY. That one is a correction
///    rather than a preference: upstream ships the two inverted. See the comment at the override.
///
/// Both live here rather than in the vendored files, which must stay byte-faithful copies of the
/// CDN -- `assets/i18n/VENDORED.md` explains why, and how to send such a fix upstream.
///
/// DataTables' `%d` in the counted form is a DataTables placeholder, NOT a rust-i18n one; it must
/// reach the page verbatim. rust-i18n only interpolates `%{name}`, so it passes through untouched.
fn datatables_language_block() -> Option<String> {
    let row = viz_i18n::active_locale();
    let vendored = match row.bcp47 {
        "en" => return None,
        "es" => include_str!("assets/i18n/datatables/es.json"),
        "fr" => include_str!("assets/i18n/datatables/fr.json"),
        "de" => include_str!("assets/i18n/datatables/de.json"),
        // it.json omits `lengthLabels` / `orderClear`, and pt-BR.json omits the whole
        // `columnControl` group -- DataTables falls back to its English defaults for a key the
        // language object does not provide, so those few controls stay English. Recorded per
        // file in assets/i18n/VENDORED.md.
        "it" => include_str!("assets/i18n/datatables/it.json"),
        "pt-BR" => include_str!("assets/i18n/datatables/pt-BR.json"),
        // ja.json and zh-CN.json (upstream's Simplified `zh.json`) each omit columnControl,
        // lengthLabels and orderClear, so those controls keep their English defaults too.
        "ja" => include_str!("assets/i18n/datatables/ja.json"),
        "zh-CN" => include_str!("assets/i18n/datatables/zh-CN.json"),
        // Curated language with no vendored DataTables file yet: keep the English defaults
        // rather than emitting a half-translated table.
        _ => return None,
    };

    let mut lang: serde_json::Value = serde_json::from_str(vendored).ok()?;
    // Overriding through `get_mut` (rather than rebuilding the object) keeps every other key the
    // vendored file provides, including ones added by future DataTables versions.
    let obj = lang.as_object_mut()?;
    obj.entry("searchBuilder")
        .or_insert_with(|| serde_json::json!({}))
        .as_object_mut()?
        .insert(
            "button".to_string(),
            serde_json::json!({
                "0": t!("viz.drawer.advanced_filter"),
                "_": t!("viz.drawer.advanced_filter_counted"),
            }),
        );

    // UPSTREAM DEFECT, corrected here rather than in the vendored file. DataTables' zh translation
    // has the SearchBuilder *date* conditions inverted: `after` reads 早于 ("earlier than") and
    // `before` reads 晚于 ("later than"), so a reader picking a date filter gets the opposite
    // comparison to the one applied. Its `number` conditions (大于/小于) are correct, as are every
    // other vendored locale's dates, so this is specific to upstream zh.
    //
    // Fixed at splice time because `assets/i18n/datatables/zh-CN.json` must stay a byte-faithful
    // copy of the CDN file -- hand-editing it would be reverted by the next version-bump
    // re-download, silently reintroducing the bug. `zh_date_condition_labels_are_corrected` pins
    // the corrected output AND asserts the vendored file still needs correcting, so this override
    // is removed rather than left shadowing a fixed upstream string.
    if row.bcp47 == "zh-CN"
        && let Some(date) = obj
            .get_mut("searchBuilder")
            .and_then(|sb| sb.get_mut("conditions"))
            .and_then(|conditions| conditions.get_mut("date"))
            .and_then(serde_json::Value::as_object_mut)
    {
        date.insert("after".to_string(), serde_json::json!("晚于"));
        date.insert("before".to_string(), serde_json::json!("早于"));
    }

    Some(format!(
        "        language: {},",
        serde_json::to_string(&lang).ok()?
    ))
}

fn data_drawer_script() -> String {
    let mut script = DATA_DRAWER_SCRIPT
        .replace(
            "__QSVI18N_ROW_SELECTED_ONE__",
            &js_string_literal(&t!("viz.drawer.row_selected_one")),
        )
        .replace(
            "__QSVI18N_ROW_SELECTED_MANY__",
            &js_string_literal(&t!("viz.drawer.row_selected_many")),
        )
        .replace(
            "__QSVI18N_CLOSE_VIEWER__",
            &js_string_literal(&t!("viz.drawer.close_viewer")),
        )
        // The "(n)" counter form is built by JS concatenation, so the key carries the opening
        // paren and the closing one stays in the template.
        .replace(
            "__QSVI18N_CLEAR_FILTERS_PREFIX__",
            &js_string_literal(&t!("viz.drawer.clear_filters_prefix")),
        )
        .replace(
            "__QSVI18N_CLEAR_FILTERS__",
            &js_string_literal(&t!("viz.drawer.clear_filters")),
        )
        .replace(
            "__QSVI18N_VIEWER_FAILED__",
            &js_string_literal(&t!("viz.drawer.viewer_failed")),
        );
    // Splice the vendored DataTables i18n over the English `language:` block. English returns
    // None and the block is left byte-for-byte as authored.
    if let Some(lang) = datatables_language_block() {
        script = script.replace(DT_LANG_NEEDLE, &lang);
    }
    script
}

const DATA_DRAWER_SCRIPT: &str = r##"<style>
  /* effective drawer height: the grip persists --qsv-data-h in absolute px, so re-clamp it in
     CSS against the CURRENT viewport — a later window resize / device rotation can otherwise
     leave the persisted value outside the 20vh-90vh bounds. Every consumer (drawer height,
     body margin, fixed-widget pushes) uses this one clamped var so they can never disagree. */
  /* --qsv-data-vh is one vh worth of px. It defaults to the real thing, so a standalone page
     behaves exactly as if the vh unit were used directly; an EMBEDDED page has the parent
     overwrite it with a hundredth of the parent's viewport (see the qsvVizViewport handshake).
     That indirection exists because vh inside an auto-sized iframe resolves against the IFRAME,
     which the gallery has grown to the Data Schematic's full content height — so "55vh" meant 55% of
     the whole document and the drawer opened far taller than the window it had to fit in. */
  /* declared on :root, NOT on the body rule below — the handshake writes this var to
     documentElement's inline style (as --qsv-data-h already does), and re-declaring it on body
     would beat that inherited value and silently pin every page back to the default */
  :root { --qsv-data-vh: 1vh; }
  body { --qsv-data-h-eff: clamp(calc(20 * var(--qsv-data-vh)), var(--qsv-data-h, min(calc(55 * var(--qsv-data-vh)), 560px)), calc(90 * var(--qsv-data-vh))); }
  #qsv-data-drawer { position: fixed; left: 0; right: 0; bottom: 0; height: var(--qsv-data-h-eff); transform: translateY(103%); transition: transform 0.22s ease; z-index: 1120; display: flex; flex-direction: column; color: var(--qsv-page-ink, #1a1a1a); background: var(--qsv-page-bg, #ffffff); border-top: 1px solid rgba(127, 127, 127, 0.4); box-shadow: 0 -4px 18px rgba(0, 0, 0, 0.18); }
  #qsv-data-drawer.open { transform: none; }
  #qsv-data-drawer:focus { outline: none; }
  .qsv-data-drawer-grip { flex: none; height: 10px; cursor: ns-resize; touch-action: none; display: flex; align-items: center; justify-content: center; }
  .qsv-data-drawer-grip::before { content: ""; width: 44px; height: 4px; border-radius: 2px; background: rgba(127, 127, 127, 0.55); }
  .qsv-data-drawer-bar { display: flex; align-items: center; justify-content: space-between; padding: 0 14px 6px; font-size: 13px; font-weight: 600; border-bottom: 1px solid rgba(127, 127, 127, 0.25); }
  .qsv-data-drawer-bar a { color: inherit; text-decoration: none; font-weight: 400; font-size: 15px; opacity: 0.75; }
  .qsv-data-drawer-bar a:hover { opacity: 1; }
  .qsv-data-drawer-content { flex: 1; min-height: 0; display: flex; flex-direction: column; padding: 4px 14px 10px 14px; }
  /* only the table's own layout row scrolls; the controls row above and the info/paging row
     below stay fixed, and the sticky thead pins the header row inside the scroll region */
  #qsv-data-drawer div.dt-container { flex: 1; min-height: 0; display: flex; flex-direction: column; }
  #qsv-data-drawer div.dt-container > div.dt-layout-row { flex: none; }
  /* align-items MUST override the center that .dt-layout-table inherits from DataTables'
     .dt-layout-row flex rule: a taller-than-viewport table gets vertically CENTERED in the
     scrollport, which puts half the overflow ABOVE it — unreachable by scrolling, so the
     first rows sit hidden behind the sticky thead */
  #qsv-data-drawer div.dt-container > div.dt-layout-table { flex: 1 1 auto; min-height: 0; overflow: auto; align-items: flex-start; }
  /* Under scrollX the header is no longer inside the body table: DataTables lifts it into its
     own `div.dt-scroll-head` sitting above `div.dt-scroll-body`, and keeps the two aligned by
     mirroring the body's horizontal scroll onto the head. So the head handles the horizontal
     axis itself — but only the body scrolls, and the drawer's vertical scroller is the OUTER
     `.dt-layout-table`, which would carry the whole head off the top with it. Pinning the head
     restores a fixed header on the vertical axis while leaving DataTables' horizontal mirroring
     untouched. Sticking this one element also pins both header rows (titles + ColumnControl's
     search row) together, with none of the per-row offset measurement that pinning cells
     individually would need.

     `!important` is load-bearing, not cargo cult: DataTables writes `position: relative` as an
     INLINE style on this element ("overflow: hidden; position: relative; border: 0; width:
     100%"), and an inline declaration outranks any selector, however specific. */
  #qsv-data-drawer div.dt-scroll-head { position: sticky !important; top: 0 !important; z-index: 5; background: var(--qsv-page-bg, #ffffff); }
  /* the footer row (credits + Theme toggle + logo) is in flow, so this one margin lifts it clear
     of the open drawer — no per-widget offset rules needed */
  body.qsv-data-open { margin-bottom: var(--qsv-data-h-eff); }
  /* ColumnControl's dropdown is positioned against the viewport but rendered inside the drawer,
     which is itself a stacking context above the page — keep the panel over the sticky thead */
  div.dtcc-dropdown { z-index: 20; }
  #qsv-data-drawer table.dataTable { color: inherit; }
  /* An unbroken long token (URL, base64 blob, hash) has no wrap opportunity, so the column
     grows to the token's full width. Under scrollX that is reachable — it just makes one column
     absurdly wide and pushes everything else out past it. `anywhere` lets such a token break
     mid-token; ordinary prose is unaffected because it still wraps at its spaces first. Note
     this cannot be done with `max-width`: browsers treat that as advisory on table cells under
     `table-layout: auto`. */
  #qsv-data-table td, #qsv-data-table thead th { overflow-wrap: anywhere; }
  /* Total width for the scrollX workaround (see syncTableMinWidth). Deliberately applied through
     an ID-SCOPED RULE reading a custom property, NOT as an inline `style.minWidth`.
     DataTables' sizing pass measures a SHALLOW `cloneNode()` of the table, which copies the style
     ATTRIBUTE; it clears the width attribute and sets `width: auto`, but nothing clears
     `min-width`. An inline min-width therefore rides into the very measurement that is supposed
     to recompute it, flooring the clone at the previous total — so the value could only ever
     ratchet upward, and a table first opened on a wide viewport stayed pinned wide after the
     reader narrowed it, showing a horizontal scrollbar over columns that now had room.
     The clone is stripped of its `id`, so this rule cannot match it (the same property that keeps
     `overflow-wrap: anywhere` above out of the measurement); the custom property rides along on
     the clone harmlessly, because nothing there declares `min-width` from it. */
  #qsv-data-table { min-width: var(--qsv-data-minw, 0px); }
  /* theme-aware: left to the UA default this is #0000EE, then #551A8B once visited — both
     unreadable on dark paper, and href="#" means one click marks it visited forever */
  .qsv-viz-meta a.qsv-data-link, .qsv-viz-meta a.qsv-data-link:visited { font-size: 0.9em; font-weight: 600; color: var(--qsv-link, #0a5fb4); text-decoration: none; display: inline-flex; align-items: center; gap: 4px; }
  .qsv-viz-meta a.qsv-data-link:hover, .qsv-viz-meta a.qsv-data-link:focus-visible { color: var(--qsv-link-hover, #084b8f); text-decoration: underline; }
  /* Selected rows. Derived from --qsv-link with color-mix rather than hardcoded, so the tint
     tracks the theme automatically (the link color flips with body.qsv-dark) instead of needing
     a separate dark-mode rule that could drift. DataTables' own Select CSS paints a solid grey
     that is nearly invisible on the dark theme, hence the override; the !important is needed
     because that rule is more specific and ships inside the vendored bundle. The left border
     is the redundant, non-color cue: a row stays identifiable as selected when the tint is lost
     to a colorblind reader, a high-contrast mode, or a print stylesheet. */
  #qsv-data-table tbody tr.selected > *, #qsv-data-table tbody tr.selected:hover > * { box-shadow: none !important; background: color-mix(in srgb, var(--qsv-link, #0a5fb4) 22%, transparent) !important; color: inherit !important; }
  #qsv-data-table tbody tr.selected:hover > * { background: color-mix(in srgb, var(--qsv-link, #0a5fb4) 30%, transparent) !important; }
  #qsv-data-table tbody tr.selected > *:first-child, #qsv-data-table tbody tr.selected:hover > *:first-child { box-shadow: inset 3px 0 0 0 var(--qsv-link, #0a5fb4) !important; }
  /* count of selected rows, appended to DataTables' own info line */
  .qsv-data-selinfo { opacity: 0.85; }
  /* Transient note (e.g. a clicked map point whose row is past the preview prefix). Sits in the
     drawer bar, not over the table, so it never covers data; aria-live lets a screen reader
     announce it, since it is the only feedback that a click did something. */
  .qsv-data-note { margin-left: 10px; margin-right: auto; font-weight: 400; font-size: 12px; opacity: 0; transition: opacity 0.15s ease; color: var(--qsv-link, #0a5fb4); pointer-events: none; }
  .qsv-data-note.show { opacity: 1; }
</style>
<script>
(function () {
  var dt = null;
  // ---------------------------------------------------------------------------------------
  // Row selection.
  //
  // Hand-rolled rather than using the DataTables Select extension, which is omitted from the
  // bundle by choice — see DATATABLES_CDN_COMBO for the cost/benefit. (Through the 3.x line it
  // also could not have worked here: it hard-required a global jQuery, which this vanilla-mode
  // page does not provide. Select 4.0.0 dropped that requirement, so the omission is now a
  // decision, not a constraint.)
  //
  // What matters for the code below is that Select is ABSENT: with no Select registered,
  // `rows({selected:true})` returns EVERY row rather than none, so selection state cannot be
  // read back off the table and is tracked here instead.
  //
  // Selection is stored as DATA row indexes (`row().index()`), never DOM nodes: `deferRender`
  // discards off-page rows, and sorting/filtering/paging all reshuffle the DOM, so a node-based
  // set would silently lose rows. The visual state is therefore re-applied on every `draw`.
  var selRows = new Set();
  // data index of the last non-shift click, the fixed end of a shift-range
  var selAnchor = null;

  // Repaint the `selected` class on the rows currently in the DOM. Cheap: only the current page.
  function paintSelection() {
    if (!dt) return;
    dt.rows({page: "current"}).every(function () {
      var node = this.node();
      if (node) node.classList.toggle("selected", selRows.has(this.index()));
    });
  }

  // Append/refresh the "N rows selected" count on DataTables' own info line, which is where a
  // user already looks for row counts — and the only feedback when a selected row is on another
  // page or filtered out of view.
  function paintSelectionInfo() {
    var info = document.querySelector("#qsv-data-drawer div.dt-info");
    if (!info) return;
    var el = info.querySelector(".qsv-data-selinfo");
    if (!selRows.size) { if (el) el.remove(); return; }
    if (!el) {
      el = document.createElement("span");
      el.className = "qsv-data-selinfo";
      info.appendChild(el);
    }
    el.textContent = " — " + selRows.size + (selRows.size === 1 ? __QSVI18N_ROW_SELECTED_ONE__ : __QSVI18N_ROW_SELECTED_MANY__);
  }

  // Single funnel for every selection change, so the map bridge sees exactly one event per
  // change regardless of which side caused it. `source` lets a listener ignore its own echo.
  function selectionChanged(source) {
    paintSelection();
    paintSelectionInfo();
    document.dispatchEvent(new CustomEvent("qsv-data-selection", {
      detail: { indexes: Array.from(selRows), source: source }
    }));
  }

  // --- public seam, used by the map<->rows bridge (MAP_SELECT_CHROME) ---
  // Replace (or extend) the selection with the given DATA indexes. Out-of-range indexes are
  // dropped and reported, so a map point past the drawer's preview prefix cannot select row 0.
  window.__qsvDataSelect = function (indexes, extend) {
    if (!dt) return 0;
    var list = Array.isArray(indexes) ? indexes : [indexes];
    // rows().count(), NOT data().count() — the latter counts CELLS (rows x columns), so on a
    // 41-column preview it reported 20500 for 500 rows and waved through every out-of-range index
    var total = dt.rows().count();
    if (!extend) selRows.clear();
    var kept = 0;
    list.forEach(function (i) {
      i = +i;
      if (Number.isInteger(i) && i >= 0 && i < total) { selRows.add(i); selAnchor = i; kept++; }
    });
    selectionChanged("map");
    return kept;
  };
  window.__qsvDataSelectedRows = function () { return Array.from(selRows); };
  // Raw cells of one DATA row (file column order, exactly as embedded), or null when the drawer
  // has not been built or the index is out of range. The map bridge reads a row's coordinates
  // through this rather than reaching into DataTables itself. A DATE column's cell is a
  // `[display, sortKey]` pair rather than a bare string, so callers must expect either.
  window.__qsvDataRowCells = function (i) {
    if (!dt) return null;
    i = +i;
    if (!Number.isInteger(i) || i < 0 || i >= dt.rows().count()) return null;
    return dt.row(i).data() || null;
  };

  // breathing room left above/below a row that had to be scrolled to, so it does not sit flush
  // against the header or the bottom edge
  var ROW_REVEAL_MARGIN = 8;

  // Scroll a row into the drawer's visible band.
  //
  // The vertical scroller is the OUTER `div.dt-layout-table` (the drawer's own flex rule), NOT
  // `div.dt-scroll-body` — under `scrollX: true` that one scrolls horizontally only, since
  // `scrollY` is never set. The header is pinned inside the scroller with `position: sticky`, so
  // its height has to come off the top of the band; scrolling a row flush to the top would park it
  // underneath the header instead of revealing it.
  //
  // Nothing moves while the row is already fully visible: a map click that lands on a row that is
  // on screen anyway should not jerk the table out from under the reader.
  //
  // `behavior: "instant"` is deliberate and is NOT interchangeable with "auto". The page sets
  // `scroll-behavior: smooth`, "auto" inherits it, and a smooth programmatic scroll can be
  // cancelled at frame 0 by the layout work the same click triggers — arriving nowhere, silently.
  function revealRow(node) {
    var scroller = document.querySelector("#qsv-data-drawer div.dt-container > div.dt-layout-table");
    if (!scroller || !node) return;
    var head = document.querySelector("#qsv-data-drawer div.dt-scroll-head");
    var headH = head ? head.getBoundingClientRect().height : 0;
    var sr = scroller.getBoundingClientRect(), nr = node.getBoundingClientRect();
    var above = nr.top - (sr.top + headH), below = nr.bottom - sr.bottom, delta = 0;
    if (above < 0) delta = above - ROW_REVEAL_MARGIN;
    else if (below > 0) delta = below + ROW_REVEAL_MARGIN;
    if (delta) scroller.scrollTo({top: scroller.scrollTop + delta, behavior: "instant"});
  }

  // Bring the row with this DATA index into view, in the CURRENT sort/filter: page to it, then
  // scroll it inside the drawer's scrollport (paging alone only guarantees the right PAGE — on a
  // 25-row page the row can still sit well below the fold). Returns false when the row is filtered
  // out (caller decides whether to explain) — never clears the user's filters.
  window.__qsvDataPageTo = function (i) {
    if (!dt) return false;
    var order = dt.rows({order: "applied", search: "applied"}).indexes().toArray();
    var pos = order.indexOf(+i);
    if (pos < 0) return false;
    var len = dt.page.len();
    // len is -1 when the user picked "All" — everything is already on one page
    if (len > 0) dt.page(Math.floor(pos / len)).draw(false);
    // `draw` is synchronous and materializes the page's rows, so the node is normally attached by
    // now; the deferred retry covers a `deferRender` pass that has not caught up.
    var node = dt.row(+i).node();
    if (node) revealRow(node);
    else setTimeout(function () { revealRow(dt.row(+i).node()); }, 0);
    return true;
  };

  // DataTables' native dark palette keys on `:root.dark` (the <html> element); qsv's theme
  // toggle flips `body.qsv-dark`. Mirror the body class onto <html> so the drawer widgets
  // follow the qsv theme — nothing else in the page styles off `html.dark`.
  function syncDark() {
    document.documentElement.classList.toggle("dark", document.body.classList.contains("qsv-dark"));
  }
  syncDark();
  new MutationObserver(syncDark).observe(document.body, { attributes: true, attributeFilter: ["class"] });
  // Ensure the DataTables library + stylesheet are installed. The payload tags come in three
  // shapes (see `datatables_lib_block`): CDN tags (nothing to do — `window.DataTable` exists),
  // plain inline (`<script>` already executed / real `<style>` already applied), or gzip-b64
  // payloads that must be inflated here on first use.
  function ensureLib() {
    var jobs = [];
    var cssEl = document.getElementById("qsv-data-css");
    if (cssEl && cssEl.tagName === "SCRIPT") {
      jobs.push(window.__qsvGunzip(cssEl).then(function (css) {
        var st = document.createElement("style");
        st.textContent = css;
        cssEl.parentNode.replaceChild(st, cssEl);
      }));
    }
    if (!window.DataTable) {
      var lib = document.getElementById("qsv-data-lib");
      if (lib && lib.getAttribute("type") === "application/gzip-b64") {
        jobs.push(window.__qsvGunzip(lib).then(function (src) {
          lib.textContent = "";
          (0, eval)(src);
        }));
      }
    }
    return Promise.all(jobs);
  }
  function payloadText(el) {
    return el.getAttribute("type") === "application/gzip-b64"
      ? window.__qsvGunzip(el).then(function (s) { el.textContent = ""; return s; })
      : Promise.resolve(el.textContent);
  }
  // Embedded reveal helpers. In the gallery's auto-sized iframes the iframe itself never
  // scrolls (the PARENT page does), so position:fixed pins the drawer to the bottom of the
  // FULL iframe height — often far below the visible viewport — and scrollIntoView cannot
  // help: it is a no-op on fixed-position elements. Instead scroll the parent directly via
  // frameElement geometry. Same-origin only (frameElement access throws cross-origin, and a
  // file://-opened gallery has opaque frames) — then we quietly skip, which is today's
  // behavior. Standalone pages (self === top) never scroll.
  function revealBottom() {
    if (window.self === window.top) return;
    // primary channel: ask a cooperating parent (the qsv gallery) to scroll. postMessage works
    // even when the frames are cross-origin — e.g. a file://-opened gallery, where every direct
    // parent-window access below throws — the same reason the gallery's iframe auto-sizing
    // rides postMessage rather than reading the child document.
    try { window.parent.postMessage({ qsvVizReveal: "bottom" }, "*"); } catch (e) {}
    // fallback for same-origin embedders without the gallery listener: scroll the parent
    // directly. In the gallery both paths fire and converge on the same position.
    try {
      var fr = window.frameElement.getBoundingClientRect();
      var pw = window.parent;
      if (fr.bottom > pw.innerHeight) {
        // instant, NOT smooth: on an iframe-heavy gallery a smooth animation can be canceled
        // at its first frame by concurrent layout work and never arrive (the gallery's jump
        // stabilizer documents the same lesson) — and "auto" would inherit the gallery's
        // scroll-behavior:smooth CSS
        pw.scrollTo({ top: pw.scrollY + fr.bottom - pw.innerHeight, behavior: "instant" });
      }
    } catch (e) {}
  }
  function revealTop(el) {
    if (window.self === window.top) return;
    // the iframe viewport is the whole (unscrolled) document, so the element's viewport rect
    // IS its document position within the iframe
    var off = el.getBoundingClientRect().top;
    try { window.parent.postMessage({ qsvVizReveal: "top", qsvVizOffset: off }, "*"); } catch (e) {}
    try {
      var fr = window.frameElement.getBoundingClientRect();
      var y = fr.top + window.parent.scrollY + off - 80;
      window.parent.scrollTo({ top: Math.max(0, y), behavior: "instant" });
    } catch (e) {}
  }
  // Any reader input over the Data Schematic invalidates a pending reveal re-align series: the
  // reader has taken over scrolling, and a late re-align would yank them back down even with
  // the drawer still open. The arming click's own pointerdown fires BEFORE show() captures its
  // sequence number, so it never cancels the series it starts.
  // Height of the viewport the drawer actually has to fit inside. Standalone that is this
  // window; embedded it is the PARENT's, which only the parent can tell us — an auto-sized
  // iframe has no viewport of its own to measure. Until (or unless) it answers we use the
  // local value, which is the pre-handshake behavior.
  var parentVh = 0;
  function viewportH() { return parentVh || window.innerHeight; }
  // Asked at load AND again on every open. The load-time ask races the embedder: the gallery
  // installs its listener far below the iframe markup, so a Data Schematic near the top of the page
  // can post before anything is listening, and a dropped answer has no other recovery — the
  // drawer would stay sized to the iframe until the parent happened to resize.
  function askViewport() {
    if (window.self === window.top) return;
    try { window.parent.postMessage({ qsvVizWantViewport: 1 }, "*"); } catch (e) {}
  }
  if (window.self !== window.top) {
    addEventListener("message", function (ev) {
      var d = ev.data;
      if (!d || typeof d.qsvVizViewport !== "number" || !(d.qsvVizViewport > 0)) return;
      parentVh = d.qsvVizViewport;
      // feed the CSS clamp in px-per-vh units
      document.documentElement.style.setProperty("--qsv-data-vh", (parentVh / 100) + "px");
      // the drawer may already be open (parent resize): re-fit the table to its new height
      if (dt) dt.columns.adjust();
    });
    // the parent also broadcasts on its own resize; this is the initial ask
    askViewport();
  }
  var revealSeq = 0;
  ["wheel", "touchstart", "keydown", "pointerdown"].forEach(function (t) {
    addEventListener(t, function () { revealSeq++; }, { capture: true, passive: true });
  });
  function show(drawer) {
    // covers a load-time ask that went unheard, and picks up any change since the last open
    askViewport();
    drawer.classList.add("open");
    document.body.classList.add("qsv-data-open");
    window.dispatchEvent(new Event("resize"));
    // move keyboard focus into the drawer (its tabindex="-1" makes it programmatically focusable)
    drawer.focus({ preventScroll: true });
    revealBottom();
    // Opening grows the page (body margin) and the embedding gallery then auto-grows the
    // iframe, and on a still-settling gallery OTHER dashboards' iframes keep growing too —
    // each layout shift above the target cancels an in-flight smooth scroll (the same churn
    // the gallery's jump stabilizer exists to fight). Re-align a few times; once the drawer
    // region is visible the parent's reveal handler is a no-op, so the series converges.
    // Each callback re-checks that the drawer is still open (a close has already revealed the
    // top) AND that no reader input arrived since (revealSeq unchanged).
    var seq = ++revealSeq;
    [450, 1200, 2200].forEach(function (t) {
      setTimeout(function () {
        var d = document.getElementById("qsv-data-drawer");
        if (revealSeq === seq && d && d.classList.contains("open")) revealBottom();
      }, t);
    });
  }
  function build() {
    var rowsEl = document.getElementById("qsv-data-rows");
    var colsEl = document.getElementById("qsv-data-cols");
    if (!rowsEl || !colsEl) return Promise.reject(new Error("data payload missing"));
    return ensureLib().then(function () { return payloadText(rowsEl); }).then(function (rowsTxt) {
      var rows = JSON.parse(rowsTxt);
      var cols = JSON.parse(colsEl.textContent);
      var drawer = document.createElement("aside");
      drawer.id = "qsv-data-drawer";
      drawer.setAttribute("tabindex", "-1");
      // resize grip on the top edge: drag to make the drawer taller/shorter. The height rides
      // the --qsv-data-h CSS var so the body margin and the fixed logo/theme widgets follow.
      var grip = document.createElement("div");
      grip.className = "qsv-data-drawer-grip";
      grip.setAttribute("aria-hidden", "true");
      grip.addEventListener("pointerdown", function (ev) {
        ev.preventDefault();
        grip.setPointerCapture(ev.pointerId);
        var raf = 0, lastH = 0;
        function onMove(mv) {
          // the raw height stays in THIS frame's coordinates — the drawer is pinned to the
          // bottom of this document and mv.clientY is measured in it — but the 20%/90% bounds
          // are about the viewport the drawer must fit inside, which when embedded is the
          // parent's (same reason as --qsv-data-vh; window.innerHeight here is the iframe's
          // full content height, so the bounds would let the drawer grow past the window)
          var vpH = viewportH();
          lastH = Math.min(Math.max(window.innerHeight - mv.clientY, vpH * 0.2), vpH * 0.9);
          if (!raf) raf = requestAnimationFrame(function () {
            raf = 0;
            document.documentElement.style.setProperty("--qsv-data-h", Math.round(lastH) + "px");
          });
        }
        function onUp() {
          grip.removeEventListener("pointermove", onMove);
          grip.removeEventListener("pointerup", onUp);
          grip.removeEventListener("pointercancel", onUp);
          // re-fit plotly panels to the shrunken/grown page and re-balance the table columns
          window.dispatchEvent(new Event("resize"));
          if (dt) dt.columns.adjust();
        }
        grip.addEventListener("pointermove", onMove);
        grip.addEventListener("pointerup", onUp);
        grip.addEventListener("pointercancel", onUp);
      });
      var bar = document.createElement("div");
      bar.className = "qsv-data-drawer-bar";
      var title = document.createElement("span");
      title.textContent = "__QSVDATATITLE__";
      var close = document.createElement("a");
      close.href = "#";
      close.textContent = "✕";
      close.setAttribute("aria-label", __QSVI18N_CLOSE_VIEWER__);
      close.addEventListener("click", function (ev) { ev.preventDefault(); window.qsvCloseData(); });
      bar.appendChild(title);
      // transient status line, flashed by window.__qsvDataNote (see below). `margin-right: auto`
      // keeps it beside the title and still pushes the close button to the far right.
      var note = document.createElement("span");
      note.className = "qsv-data-note";
      note.id = "qsv-data-note";
      note.setAttribute("aria-live", "polite");
      bar.appendChild(note);
      bar.appendChild(close);
      var content = document.createElement("div");
      content.className = "qsv-data-drawer-content";
      var table = document.createElement("table");
      table.id = "qsv-data-table";
      table.className = "display compact";
      table.style.width = "100%";
      // ONE header row: ColumnControl puts the ordering and per-column search widgets inside the
      // title cell itself, so there is no second filter row to build, keep aligned with
      // offset for sticky positioning, or strip back out of the
      // CSV export.
      var thead = document.createElement("thead");
      var titleRow = document.createElement("tr");
      cols.forEach(function (c) {
        var th = document.createElement("th");
        th.textContent = c.title;
        titleRow.appendChild(th);
      });
      thead.appendChild(titleRow);
      table.appendChild(thead);
      content.appendChild(table);
      drawer.appendChild(grip);
      drawer.appendChild(bar);
      drawer.appendChild(content);
      document.body.appendChild(drawer);
      // Clear Filters enablement: the button lights up only while there is something to clear,
      // and its label counts the active filters across all three sources. SearchBuilder pushes
      // its share through the filterChanged hook below (criteria edits do not always redraw);
      // the global box and the ColumnControl widgets are re-read on every draw.
      var sbCount = 0;
      var updateClearFilters = null;
      // the region-click bridge applies its filter through searchBuilder.rebuild(), which does
      // NOT fire the filterChanged hook below — it announces the new criteria count over this
      // seam instead so Clear Filters stays honest
      document.addEventListener("qsv-data-sb-changed", function (ev) {
        sbCount = (ev.detail && ev.detail.count) || 0;
        if (updateClearFilters) updateClearFilters();
      });
      dt = new DataTable(table, {
        data: rows,
        // render.text(): cell values are DATA, not markup — without it DataTables injects them
        // as HTML, so a cell containing markup would render (or execute) instead of display
        columns: cols.map(function (c) {
          var text = DataTable.render.text();
          // A date cell may arrive as [raw, sortKey] when its source text is not one the
          // browser's Date.parse reads correctly (see collect_datatable_rows). Split it back
          // apart: the reader still sees the source text, while ordering AND searching take the
          // ISO key. Plain-string cells in the same column pass straight through, so this has to
          // handle both shapes. Only date columns are ever paired, so the array branch stays
          // inside this arm — every other column keeps the plain escaping renderer.
          if (c.type !== "date") return { type: c.type, render: text };
          // render.text() is an ORTHOGONAL MAP ({display, filter}), not a callable, so the
          // date column supplies a map too. "_" is the fallback the un-keyed types ("type",
          // and the "export" Buttons could ask for) resolve through, which keeps them on the
          // source text exactly as an un-rendered column would be.
          var raw = function (d) { return Array.isArray(d) ? d[0] : d; };
          var key = function (d) { return Array.isArray(d) ? d[1] : d; };
          // The YYYY-MM-DD prefix of an ISO-leading value, or null for anything else (a blank,
          // or a cell this column's typing did not produce a key for). Mirrors the Rust
          // `is_iso_leading_date`, and the anchored digit pattern is what makes the slice safe
          // to hand back unescaped: what it returns is only ever digits and dashes.
          var isoDay = function (v) {
            return typeof v === "string" && /^\d{4}-\d{2}-\d{2}/.test(v) ? v.slice(0, 10) : null;
          };
          return { type: c.type, render: {
            _: raw,
            sort: key,
            // display stays inside render.text() — dropping it here would put an unescaped cell
            // value back into the page.
            display: function (d) { return text.display(raw(d)); },
            // FILTER TAKES THE ISO DAY, NOT THE SOURCE TEXT. ColumnControl's searchDateTime
            // compares epochs: it parses its picker value as ISO (UTC midnight) and the cell
            // through Date.parse. A slash-formatted cell parses as LOCAL midnight, so the two
            // differ by the viewer's UTC offset and NOTHING ever matches; a day-first cell
            // (25/07/2026) is month 25 and parses to NaN. Feeding the ISO key puts both sides
            // on UTC midnight, which is what makes equals/before/after work on non-ISO date
            // columns at all.
            //
            // Truncating to the DAY matters for DateTime columns. The picker is a bare calendar
            // with no time input, so the only value a reader can express is a day; comparing it
            // against a full timestamp makes `equals` unsatisfiable (2011-02-05T22:25:21 is
            // never 2011-02-05T00:00:00) and a picked day would silently return nothing. Both
            // branches truncate, so an all-ISO DateTime column — whose cells are never paired,
            // because they are already ISO-leading — behaves the same as a month-first one.
            // Ordering is unaffected: `sort` keeps the full-precision key.
            filter: function (d) {
              var day = isoDay(key(d));
              return day === null ? text.filter(raw(d)) : day;
            }
          } };
        }),
        deferRender: true,
        // no initial sort: rows show in file order until the user orders a column
        order: [],
        pageLength: 25,
        lengthMenu: [10, 25, 50, 100],
        // Horizontal scrolling rather than Responsive's collapse-into-a-child-row.
        //
        // Responsive and scrollX are mutually exclusive, and Responsive is the one that had to
        // go: it fights ColumnControl. Every column it collapses takes that column's search
        // widget off the screen with it, so on a wide table the per-column controls the drawer
        // exists to offer are exactly the ones a reader cannot reach — they are only available
        // by widening the window until Responsive gives the column back. Its child-row detail
        // view shows the values but carries no widgets at all. scrollX keeps every column, and
        // every column's widget, permanently reachable by scrolling.
        scrollX: true,
        // ColumnControl across TWO header rows, the layout its own manual demonstrates: row 0
        // keeps the column title and the ordering control, row 1 holds the search widget.
        // Crowding all of it into one cell squeezes the title into a two- or three-line stack
        // as soon as the icons and the label compete for the same width. `target: 1` is a row
        // ColumnControl creates itself (one cell per column) — unlike the hand-rolled filter
        // row this replaced, it owns that row, so it also keeps it in step with ordering.
        //
        // "search" auto-detects the widget from the column's `type` — searchDateTime (a calendar
        // picker) for dates, searchNumber for numerics, searchText otherwise.
        columnControl: [
          { target: 0, content: ["order"] },
          { target: 1, content: ["search"] }
        ],
        // DataTables' own sort arrows would duplicate the ordering control now sitting in row 0,
        // so the indicators are off; clicking the header still sorts (handler left on).
        ordering: { indicators: false },
        // Auto-detect never chooses searchList on its own, so the columns qsv's stats say are
        // low-cardinality categoricals ask for it explicitly: a checkbox dropdown of the
        // distinct values, which beats typing a substring you have to already know. A per-column
        // columnControl REPLACES the table-wide one rather than merging, so the override has to
        // restate row 0. `list` is only set when the preview holds every row
        // (see datatable_columns_json).
        columnDefs: cols.map(function (c, i) {
          // The value list goes in a DROPDOWN (the nested array), not inline like the text and
          // date inputs: rendered flat it puts "Select / Deselect / Search…" in the header cell
          // and forces the column as wide as those three controls.
          return c.list ? { targets: i, columnControl: [
            { target: 0, content: ["order"] },
            { target: 1, content: [["searchList"]] }
          ] } : null;
        }).filter(Boolean),
        // SearchBuilder rides in a Buttons popover ("Advanced Filter (n)") on the same controls
        // row as the page-length selector and the global search box, instead of a permanent
        // pane. It stays alongside ColumnControl: the per-column widgets cannot express the
        // cross-column AND/OR logic SearchBuilder is for — which is what "Advanced" names.
        language: {
          searchBuilder: { button: { 0: "Advanced Filter", _: "Advanced Filter (%d)" } }
        },
        layout: { topStart: ["pageLength", { buttons: [{
          extend: "searchBuilder",
          // filterChanged hands over the same criteria count the button's own "(n)" shows
          // (topGroup.count() — incomplete rows included, which is right for Clear Filters
          // too: rebuild({}) removes those as well, so they ARE something to clear). The
          // stock button never uses this hook — it re-reads the count itself on draw — so
          // taking it clobbers nothing. It fires on criteria edits that do not redraw
          // (adding or deleting a still-incomplete row), which draw-only recounting misses.
          config: { filterChanged: function (n) {
            sbCount = n;
            if (updateClearFilters) updateClearFilters();
          } }
        }, {
          text: __QSVI18N_CLEAR_FILTERS__,
          className: "qsv-clear-filters",
          init: function (dt, node, conf) {
            var btn = this;
            updateClearFilters = function () {
              var n = (dt.search() ? 1 : 0) + sbCount;
              dt.columns().every(function () {
                // ColumnControl applies a widget's search as a NAMED FIXED search on the
                // column — "dtcc" for the text/number/date inputs, "dtcc-list" for
                // searchList — which is the only place the widget state is visible:
                // column().search() stays empty. This is the same test ColumnControl's own
                // ccSearchClear button runs (minus its __ccSearch/__ccList init flags,
                // which are its serverSide path — the drawer embeds its data). A cleared
                // widget leaves "" behind, which is falsy, so it does not count.
                if (this.search.fixed("dtcc") || this.search.fixed("dtcc-list")) n++;
              });
              btn.enable(n > 0);
              btn.text(n > 0 ? __QSVI18N_CLEAR_FILTERS_PREFIX__ + n + ")" : __QSVI18N_CLEAR_FILTERS__);
            };
            // every filter source redraws when it applies a change, so draw is the one
            // signal that sees the global box and the ColumnControl widgets
            dt.on("draw", updateClearFilters);
            updateClearFilters();
          },
          // Resets EVERY filter the drawer offers, because a reader who has narrowed the table
          // three different ways should not have to remember which three. The three live in
          // separate places and none of them clears the others:
          //   - the global search box            -> search("")
          //   - ColumnControl's per-column widgets -> columnControl.searchClear()
          //   - SearchBuilder's criteria          -> searchBuilder.rebuild({})
          // columns().search("") is NOT redundant with searchClear(): ColumnControl applies its
          // own search through its own mechanism (a filtered column still reports an empty
          // column().search()), so clearing one leaves the other standing. Belt and braces.
          action: function (e, dt) {
            dt.search("");
            dt.columns().search("");
            if (dt.columns().columnControl) {
              dt.columns().columnControl.searchClear();
            }
            if (dt.searchBuilder) {
              dt.searchBuilder.rebuild({});
            }
            dt.draw();
          }
        }, {
          extend: "csv",
          text: "__QSVDATAEXPORTTEXT__",
          // substituted as a complete JS string literal: the stem is a user-supplied file name
          filename: __QSVDATAEXPORTNAME__,
          // Columns and orthogonal are deliberately left at their defaults, because the two
          // overrides that look obviously right are both wrong. columns=":visible" already
          // means every column. orthogonal="display" is the fidelity-preserving
          // choice, not "export": Buttons strips markup and decodes entities on the way out,
          // which mangles a raw cell but exactly inverts the escaping DataTable.render.text()
          // applied for display. Over a markup/entity/quote/tab/emoji fixture "display" was
          // 10/10 exact where "export" corrupted 4/10. Row scope is the default too: search
          // applies, paging does not, so the file holds every filtered row, not the page.
          //
          // NOTE: this default is only safe because the DataTables **Select extension is not
          // loaded** (omitted by choice; see DATATABLES_CDN_COMBO). With Select present, Buttons
          // silently narrows the export to "the selected rows, if any" — so if it is ever added,
          // this button MUST pin `exportOptions.modifier.selected = null`. The hand-rolled row
          // selection below deliberately does not hook Buttons, so the export stays all-rows.
          exportOptions: {
            // csvHtml5 carries this in its OWN default exportOptions, and supplying the object
            // replaces that default rather than merging into it. Restated so the CSV-injection
            // guard (a leading =, +, - or @ in a cell) cannot be lost by accident.
            escapeExcelFormula: true,
            // The thead has TWO rows — the titles and ColumnControl's search row — and the CSV
            // writer serializes every row of headerStructure, so the search row lands in the
            // file as a blank line between the header and the first record. Drop it.
            //
            // Matched on `data-dt-order="disable"`, which ColumnControl sets on the row it
            // creates (that row must not attach ordering handlers). That is a semantic marker
            // rather than a row index or an emptiness test, so it keeps working if the header
            // gains further rows. Note this hook belongs to exportData's options, NOT to the
            // button config: sitting a level up it is silently never called.
            customizeData: function (d) {
              if (!d.headerStructure) return;
              d.headerStructure = d.headerStructure.filter(function (row) {
                return !row.some(function (c) {
                  return c.cell && c.cell.parentNode &&
                    c.cell.parentNode.getAttribute("data-dt-order") === "disable";
                });
              });
            }
          }
        }] }] }
      });
      // Row selection, "os" idiom: plain click replaces the selection (and clicking the only
      // selected row clears it), ctrl/cmd-click toggles one row, shift-click extends a range.
      // Delegated from the table so it keeps working as deferRender recycles row nodes, and
      // scoped to tbody so the ColumnControl search row in the thead is untouched.
      table.addEventListener("click", function (ev) {
        var tr = ev.target.closest("tbody tr");
        if (!tr || !dt || !table.contains(tr)) return;
        var idx = dt.row(tr).index();
        // the "no matching records" placeholder is a tbody tr with no backing data row
        if (idx === undefined || idx === null) return;
        if (ev.shiftKey && selAnchor !== null) {
          // range over the CURRENT sort+filter, so it matches what the user sees
          var order = dt.rows({order: "applied", search: "applied"}).indexes().toArray();
          var a = order.indexOf(selAnchor), b = order.indexOf(idx);
          if (a >= 0 && b >= 0) {
            selRows.clear();
            for (var i = Math.min(a, b); i <= Math.max(a, b); i++) selRows.add(order[i]);
          }
          // shift-click also starts a native text selection across the rows; drop it so the
          // range reads as a row selection rather than a smear of highlighted text
          var ws = window.getSelection();
          if (ws) ws.removeAllRanges();
        } else if (ev.ctrlKey || ev.metaKey) {
          if (selRows.has(idx)) selRows.delete(idx); else selRows.add(idx);
          selAnchor = idx;
        } else {
          var onlyThis = selRows.size === 1 && selRows.has(idx);
          selRows.clear();
          if (!onlyThis) selRows.add(idx);
          selAnchor = idx;
        }
        selectionChanged("rows");
      });
      // deferRender builds row nodes on demand and every draw re-writes the tbody, so the
      // `selected` class has to be re-applied rather than set once at click time.
      dt.on("draw", function () { paintSelection(); paintSelectionInfo(); });
      // Restate the table's total width as a `min-width` ON THE TABLE, because scrollX's own
      // mechanism is not portable.
      //
      // Under scrollX DataTables leaves the table at `width: 100%` and holds the columns open
      // with `min-width` on each <col>. But CSS 2.1 s17.6.1 limits the properties that apply to
      // column elements to border, background, width and visibility -- `min-width` is not among
      // them. Blink honours it as an extension; WebKit does not. So in Safari nothing enforces
      // those widths: the table collapses to its container and there is nothing left to scroll.
      // It collapses ALL the way rather than stopping somewhere readable because the
      // `overflow-wrap: anywhere` rule above drives every column's min-content contribution down
      // to about one character. The next scroll draw then re-measures the squished row and
      // freezes the squished widths back onto the <col>s, so the collapse is self-sustaining.
      //
      // `min-width` on the TABLE is plain spec'd CSS and is honoured everywhere, so it carries
      // the same total that the <col> min-widths were supposed to.
      //
      // The per-column figures are DataTables' own sizing-pass measurements. That pass measures
      // an id-less clone of the table, so `overflow-wrap: anywhere` (which is keyed on
      // #qsv-data-table) never applies to it -- the numbers describe unbroken text and are
      // engine-independent. In Chrome their sum equals the width the table already renders at,
      // including on a table whose widest column is an unbroken 90-char token, which makes this
      // a no-op there rather than a second, competing sizing mechanism.
      //
      // Recomputed on every sizing pass rather than set once: that pass STRETCHES the columns to
      // fill the container when their natural total is smaller than it, so a value captured on a
      // wide window would pin the table wide after the reader narrows it -- inventing horizontal
      // scroll where none is warranted.
      function syncTableMinWidth() {
        if (!dt) return;
        // settings()[0].columns is internal API, hence the defensive shape check: the bundle is
        // vendored and pinned, but a future bump should degrade to today's Chrome-only behaviour
        // rather than throw inside a draw handler.
        var st = dt.settings()[0];
        if (!st || !st.columns || !st.columns.length) return;
        var total = 0;
        st.columns.forEach(function (c) {
          // a column hidden after init keeps its last measured width, which would inflate the sum
          if (c.visible !== false) total += parseFloat(c.width) || 0;
        });
        if (!(total > 0)) return;
        // FLOOR, never ceil. When the columns' natural total is smaller than the container the
        // sizing pass stretches them to fill it exactly, so this sum lands on the container's own
        // width -- give or take a sub-pixel. Rounding that UP would push the table one pixel past
        // its container and hang a horizontal scrollbar under every narrow table, which is the
        // opposite of the bug being fixed. Rounding down can only ever leave the table a sub-pixel
        // narrower than it could be, which is invisible.
        var px = Math.floor(total) + "px";
        // Also what breaks the adjust() -> column-sizing -> syncTableMinWidth() loop: the second
        // pass computes the same value and stops here.
        //
        // Written as a custom property consumed by the `#qsv-data-table` rule in the stylesheet,
        // never as `table.style.minWidth` -- an inline min-width is copied onto the sizing pass's
        // clone and would floor its own recomputation. See that rule for the full reasoning.
        if (table.style.getPropertyValue("--qsv-data-minw") === px) return;
        table.style.setProperty("--qsv-data-minw", px);
        // re-run the scroll draw so the header's inner table is re-widened to match the body's
        // new width; without it the two stay misaligned until the next unrelated redraw
        dt.columns.adjust();
      }
      dt.on("column-sizing", syncTableMinWidth);
      syncTableMinWidth();
      // Publish the table for the map<->rows selection bridge. The drawer is built LAZILY on
      // first open, so a bridge that merely read `window.__qsvDataDT` at page load would always
      // find nothing — it listens for this event instead (and re-checks the global, in case it
      // hooked after the drawer was already built).
      window.__qsvDataDT = dt;
      document.dispatchEvent(new CustomEvent("qsv-data-dt-ready"));
      // let the just-appended drawer paint once closed, so the open transition animates
      requestAnimationFrame(function () { show(drawer); });
    });
  }
  window.qsvOpenData = function () {
    var drawer = document.getElementById("qsv-data-drawer");
    if (drawer) { show(drawer); return false; }
    var lib = document.getElementById("qsv-data-lib");
    var gz = (lib && lib.getAttribute("type") === "application/gzip-b64") ||
      document.getElementById("qsv-data-rows").getAttribute("type") === "application/gzip-b64";
    if (gz && !window.DecompressionStream) {
      if (window.__qsvNoDecompress) window.__qsvNoDecompress();
      return false;
    }
    build().catch(function (e) { console.error(__QSVI18N_VIEWER_FAILED__, e); });
    return false;
  };
  // Flash a short message in the drawer bar for ~2s. Used by the map<->rows bridge to explain a
  // click that could not do anything visible (a map point whose row lies past the preview
  // prefix). A no-op before the drawer is built, so callers need no guard.
  var noteTimer = null;
  window.__qsvDataNote = function (msg) {
    var el = document.getElementById("qsv-data-note");
    if (!el) return;
    el.textContent = msg;
    el.classList.add("show");
    clearTimeout(noteTimer);
    noteTimer = setTimeout(function () { el.classList.remove("show"); }, 2000);
  };
  window.qsvCloseData = function () {
    var drawer = document.getElementById("qsv-data-drawer");
    if (drawer) drawer.classList.remove("open");
    document.body.classList.remove("qsv-data-open");
    window.dispatchEvent(new Event("resize"));
    // return focus to the Explore/Preview link in the metadata table — the top of the
    // Data Schematic — which also scrolls it back into view
    var link = document.querySelector("a.qsv-data-link");
    if (link) {
      link.focus();
      // embedded: scroll the parent back to the top of this Data Schematic (see revealTop)
      revealTop(link);
    }
    return false;
  };
  // Esc closes the data drawer; the dictionary drawer registers its own Esc handler, so with
  // both open a single Esc closes both — acceptable, they are independent overlays.
  document.addEventListener("keydown", function (ev) {
    if (ev.key === "Escape") {
      // an open Buttons popover (the SearchBuilder "Filter" panel) owns this Esc — Buttons
      // dismisses it itself; only a bare Esc closes the drawer
      if (document.querySelector("div.dt-button-collection")) return;
      var drawer = document.getElementById("qsv-data-drawer");
      if (drawer && drawer.classList.contains("open")) window.qsvCloseData();
    }
  });
})();
</script>"##;

/// Strip a describegpt attribution/provenance footer from a dictionary's dataset description.
/// The same provenance is carried separately in `x-qsv.generated_by` (rendered as the page
/// footer), so it would otherwise appear twice.
///
/// Anchors on a *footer line* whose signature is `Generated by … describegpt` (after an optional
/// leading `Attribution:` / markdown-emphasis prefix, matching describegpt's markdown/TOON footer
/// forms). Requiring the `describegpt` signature on the line means a description that merely
/// mentions "Generated by" mid-sentence is left intact. Any dangling `**Attribution**` heading or
/// horizontal-rule scaffolding just above the footer is trimmed too.
fn strip_dataset_attribution(desc: &str) -> String {
    // Find the byte offset where the footer line begins.
    let mut cut = None;
    let mut offset = 0;
    for line in desc.split_inclusive('\n') {
        let body = line.trim();
        // Drop an optional attribution prefix: markdown emphasis (`*`/`_`) then `Attribution:`.
        let after_prefix = body.trim_start_matches(['*', '_']).trim_start();
        let after_prefix = after_prefix
            .strip_prefix("Attribution:")
            .map_or(after_prefix, str::trim_start);
        if after_prefix.starts_with("Generated by ") && after_prefix.contains("describegpt") {
            cut = Some(offset);
            break;
        }
        offset += line.len();
    }
    let Some(pos) = cut else {
        return desc.trim_end().to_string();
    };
    // Trim trailing separator/blank lines and any dangling "Attribution" heading before the footer.
    let mut head = desc[..pos].trim_end();
    loop {
        let (rest, last) = match head.rfind('\n') {
            Some(i) => (head[..i].trim_end(), head[i + 1..].trim()),
            None => ("", head.trim()),
        };
        let bare = last
            .trim_matches(['*', '_', '#', ' '])
            .trim_end_matches(':')
            .trim();
        let is_attribution = bare.eq_ignore_ascii_case("attribution");
        let is_rule = last.len() >= 3 && last.chars().all(|c| matches!(c, '-' | '*' | '_'));
        if is_attribution || is_rule {
            head = rest;
        } else {
            break;
        }
    }
    head.to_string()
}

/// Render a small, safe subset of Markdown to HTML for `--dict-info` descriptions
/// (per-column and dataset-level). The text is untrusted describegpt/LLM output that
/// gets embedded inside the Data Schematic's `<script type="text/html">` template, so this
/// MUST NOT emit any raw/passthrough HTML: every literal run is `html_escape`d and only
/// a fixed whitelist of tags (`p`, `ul`, `li`, `strong`, `em`, `code`, `a`) is ever
/// produced. That single invariant guarantees both that no injected HTML/JS can execute
/// AND that a literal `</script>` can never appear to break out of the embedding template
/// (we never generate a `script` tag, and any `<`/`>` in the content is escaped). Link
/// URLs are scheme-checked (`sanitize_url`); an unsafe scheme renders as plain text
/// instead of an anchor.
///
/// Supported: paragraphs (blank-line separated), bullet lists (`-`/`*`/`+`),
/// `**bold**`/`__bold__`, `*italic*`/`_italic_`, `` `inline code` ``, and `[text](url)`
/// links. Everything else (tables, headings, code fences, nested lists, images) degrades
/// to escaped literal text.
fn render_dict_markdown(md: &str) -> String {
    let normalized = md.replace("\r\n", "\n").replace('\r', "\n");
    let lines: Vec<&str> = normalized.lines().collect();
    let mut out = String::new();
    let mut i = 0;
    while i < lines.len() {
        // Skip blank lines separating blocks.
        if lines[i].trim().is_empty() {
            i += 1;
            continue;
        }
        // ATX heading (`#`..`######`): single-line block, normalized to <h4> so it stays
        // subordinate to the page's <h1>/<h2> structure.
        if let Some(h) = heading_content(lines[i]) {
            out.push_str("<h4 class=\"qsv-dict-mdh\">");
            out.push_str(&md_inline(&localize_dict_heading(h)));
            out.push_str("</h4>\n");
            i += 1;
            continue;
        }
        // Bullet-list block: a run of consecutive bullet lines.
        if is_bullet_line(lines[i]) {
            out.push_str("<ul>");
            while i < lines.len() && is_bullet_line(lines[i]) {
                out.push_str("<li>");
                out.push_str(&md_inline(bullet_content(lines[i])));
                out.push_str("</li>");
                i += 1;
            }
            out.push_str("</ul>\n");
            continue;
        }
        // Paragraph block: consecutive non-blank, non-bullet, non-heading lines joined by
        // spaces (CommonMark soft-break behavior).
        let mut para: Vec<&str> = Vec::new();
        while i < lines.len()
            && !lines[i].trim().is_empty()
            && !is_bullet_line(lines[i])
            && heading_content(lines[i]).is_none()
        {
            para.push(lines[i].trim());
            i += 1;
        }
        out.push_str("<p>");
        out.push_str(&md_inline(&para.join(" ")));
        out.push_str("</p>\n");
    }
    out
}

/// A bullet-list line: `-`, `*`, or `+` followed by a space and at least one more char.
fn is_bullet_line(line: &str) -> bool {
    let t = line.trim_start();
    (t.starts_with("- ") || t.starts_with("* ") || t.starts_with("+ ")) && t.len() > 2
}

/// Localize the one describegpt section heading that is deliberately emitted in English.
///
/// `describegpt`'s Description response is lead prose followed by a "Notable Characteristics"
/// section, and that literal is LOAD-BEARING: `okf_short_description` (describegpt.rs) uses it as
/// the cut marker that separates the lead prose it needs for OKF's frontmatter `description:` key.
/// So the heading must stay English in the stored schema, and translating it at the prompt would
/// silently break OKF short-description extraction on every non-English run.
///
/// Translating it HERE -- at render time, after all marker parsing -- localizes the drawer without
/// touching the schema or describegpt's parser. Only the curated locales are covered (an uncurated
/// `--language` keeps the English heading), and the match is exact, so an LLM that words the
/// heading differently simply falls through unchanged. Both degrade to today's behavior.
fn localize_dict_heading(heading: &str) -> String {
    if heading.trim() == "Notable Characteristics" {
        return t!("viz.dict.notable_characteristics").to_string();
    }
    heading.to_string()
}

/// Localize the `describegpt` attribution block rendered at the foot of the Data Dictionary
/// drawer.
///
/// `x-qsv.generated_by` is authored by `describegpt` when the dictionary is generated and stored
/// verbatim in the schema, so its labels are English no matter what `viz --language` selects --
/// the dictionary is written once, the Data Schematic locale is chosen later and may differ.
/// Translating it HERE, at render time, localizes the drawer without touching the schema: the
/// stored provenance stays a stable machine-readable record, `parse_dict_model` still reads its
/// `Model:` line out of the raw JSON, and the "Export `JSONSchema`" button still serves the
/// original bytes.
///
/// Only the LABELS move. Every value -- the command line, the prompt-file kind, the model, the
/// API URL, the language code, the RFC3339 timestamp -- rides through byte-for-byte, so the
/// footer stays a faithful, copy-pasteable record of how the dictionary was built. A `Language:`
/// value can even carry describegpt's own English `(dataset-detected, NN.N%)` annotation; that is
/// part of a value, so it is deliberately left alone.
///
/// Same contract as `localize_dict_heading`: the match is EXACT and line-oriented, so a
/// hand-written dictionary, an older describegpt layout, or any line this does not recognize
/// passes through unchanged, and an uncurated `--language` keeps today's English.
///
/// The `-- ` comment prefix describegpt uses for `AttributionFormat::SqlComment` is deliberately
/// NOT handled: that prefix is empty for every other format, and JSON Schema -- the only format
/// that yields an `x-qsv.generated_by` for this drawer to render -- is one of them.
fn localize_dict_provenance(text: &str) -> String {
    // English *is* the catalog value, so the loop below is already an identity under English --
    // returning early just makes that byte-identity unconditional, whatever shape the input takes.
    if viz_i18n::active_locale().bcp47 == "en" {
        return text.to_string();
    }

    // `split('\n')` rather than `str::lines`: it round-trips exactly. `lines` swallows a trailing
    // newline and strips a `\r`, either of which would silently reflow a block this function is
    // otherwise careful to preserve.
    let mut out = String::with_capacity(text.len());
    for (i, line) in text.split('\n').enumerate() {
        if i > 0 {
            out.push('\n');
        }
        out.push_str(&localize_dict_provenance_line(line));
    }
    out
}

/// One line of [`localize_dict_provenance`].
///
/// Split out, and written as a chain of literal arms rather than a `(prefix, key)` table, because
/// `every_t_key_used_in_this_file_exists_in_the_catalog` scans this file for LITERAL `t!` keys and
/// hard-fails on a computed one -- a table would have to build its key at runtime.
///
/// The label arms keep `rest` (which still carries the separating space) verbatim. The five
/// `WARNING:` arms mirror describegpt's `PromptType` match; all five are covered even though a
/// self-inferred dictionary can only produce the two Dictionary ones, because `--dictionary
/// <file>` accepts any describegpt output and one lone English sentence under an otherwise
/// translated block reads as a bug. A reworded warning simply falls through to English.
fn localize_dict_provenance_line(line: &str) -> String {
    // The lead line is a TEMPLATE, not a literal: describegpt interpolates the running binary's
    // name and version (qsv / qsvlite / qsvdp / qsvmcp), so the middle rides through as an
    // argument instead of being parsed apart.
    if let Some(rest) = line.strip_prefix("Generated by ")
        && let Some(tool) = rest.strip_suffix(" describegpt")
    {
        return t!("viz.dict.prov_generated_by", q_tool = tool).to_string();
    }

    if let Some(rest) = line.strip_prefix("Command line:") {
        return format!("{}{rest}", t!("viz.dict.prov_command_line"));
    }
    if let Some(rest) = line.strip_prefix("Prompt file:") {
        return format!("{}{rest}", t!("viz.dict.prov_prompt_file"));
    }
    if let Some(rest) = line.strip_prefix("Model:") {
        return format!("{}{rest}", t!("viz.dict.prov_model"));
    }
    if let Some(rest) = line.strip_prefix("LLM API URL:") {
        return format!("{}{rest}", t!("viz.dict.prov_llm_api_url"));
    }
    if let Some(rest) = line.strip_prefix("Language:") {
        return format!("{}{rest}", t!("viz.dict.prov_language"));
    }
    if let Some(rest) = line.strip_prefix("Timestamp:") {
        return format!("{}{rest}", t!("viz.dict.prov_timestamp"));
    }

    if line == DG_WARNING_DICTIONARY_CONTENT_TYPE {
        return t!("viz.dict.prov_warning_dictionary_content_type").to_string();
    }
    if line == DG_WARNING_DICTIONARY {
        return t!("viz.dict.prov_warning_dictionary").to_string();
    }
    if line == DG_WARNING_DESCRIPTION {
        return t!("viz.dict.prov_warning_description").to_string();
    }
    if line == DG_WARNING_TAGS {
        return t!("viz.dict.prov_warning_tags").to_string();
    }
    if line == DG_WARNING_GENERIC {
        return t!("viz.dict.prov_warning_generic").to_string();
    }

    line.to_string()
}

/// The five warning sentences `describegpt` can append to an attribution block, one per
/// `PromptType` arm (`src/cmd/describegpt.rs`, `warning_message`). Named consts rather than
/// inline literals so `dict_provenance_warnings_match_describegpt` can pin them to describegpt's
/// own source: a reword there must fail a test, not silently drop this footer back to English.
const DG_WARNING_DICTIONARY_CONTENT_TYPE: &str = "WARNING: Label, Description and Content Type \
                                                  generated by an LLM and may contain \
                                                  inaccuracies. Verify before using!";
const DG_WARNING_DICTIONARY: &str = "WARNING: Label and Description generated by an LLM and may \
                                     contain inaccuracies. Verify before using!";
const DG_WARNING_DESCRIPTION: &str =
    "WARNING: Description generated by an LLM and may contain inaccuracies. Verify before using!";
const DG_WARNING_TAGS: &str =
    "WARNING: Tags generated by an LLM and may contain inaccuracies. Verify before using!";
const DG_WARNING_GENERIC: &str =
    "WARNING: Generated by an LLM and may contain inaccuracies. Verify before using!";

/// ATX-heading content: 1–6 leading `#` then a space (e.g. `### Notable Characteristics`,
/// common in describegpt descriptions). Returns the trimmed text after the marker, with an
/// optional ATX closing `#` sequence stripped. `None` when the line isn't a heading.
fn heading_content(line: &str) -> Option<&str> {
    let t = line.trim_start();
    let hashes = t.bytes().take_while(|&b| b == b'#').count();
    if !((1..=6).contains(&hashes) && t.as_bytes().get(hashes) == Some(&b' ')) {
        return None;
    }
    let text = t[hashes + 1..].trim();
    // Strip an optional ATX closing sequence: a trailing run of '#'s must be preceded by
    // whitespace (CommonMark). A '#' that isn't (e.g. `C#`) is content and kept.
    let without_close = text.trim_end_matches('#');
    if without_close.len() < text.len() && without_close.ends_with(char::is_whitespace) {
        Some(without_close.trim_end())
    } else {
        Some(text)
    }
}

/// The content of a bullet line, with the marker (`- `/`* `/`+ `) stripped.
fn bullet_content(line: &str) -> &str {
    line.trim_start()[2..].trim()
}

/// Render inline Markdown (`code`, links, bold, italic) from `text` to safe HTML.
/// Literal runs are `html_escape`d; only whitelisted tags are emitted. Recurses into
/// emphasis and link-text spans (each strictly shorter, so termination is guaranteed).
fn md_inline(text: &str) -> String {
    md_inline_depth(text, 0)
}

/// Depth-limited core of `md_inline`. Each emphasis span and each link label recurses one frame
/// deeper, with no natural bound: dictionary text is LLM-generated and explicitly untrusted here,
/// and a description of deeply nested `**...**` drove ~25k nested frames — enough to exhaust the
/// main thread's stack and abort the process. Past the limit the remaining text is emitted as
/// escaped literal characters, which is exactly how unmatched markers already degrade.
const MD_INLINE_MAX_DEPTH: usize = 16;

fn md_inline_depth(text: &str, depth: usize) -> String {
    if depth >= MD_INLINE_MAX_DEPTH {
        return html_escape(text);
    }
    let b = text.as_bytes();
    let n = b.len();
    let mut out = String::new();
    let mut lit = String::new();
    let mut i = 0;

    while i < n {
        let c = b[i];
        match c {
            b'`' => {
                // Inline code: content up to the next backtick, escaped, not re-parsed.
                if let Some(rel) = text[i + 1..].find('`') {
                    let end = i + 1 + rel;
                    push_escaped(&mut out, &mut lit);
                    out.push_str("<code>");
                    out.push_str(&html_escape(&text[i + 1..end]));
                    out.push_str("</code>");
                    i = end + 1;
                    continue;
                }
                lit.push('`');
                i += 1;
            },
            b'[' => {
                if let Some((html, consumed)) = try_parse_link(&text[i..], depth) {
                    push_escaped(&mut out, &mut lit);
                    out.push_str(&html);
                    i += consumed;
                    continue;
                }
                lit.push('[');
                i += 1;
            },
            b'*' | b'_' => {
                let marker_char = c as char;
                // For '_', avoid intraword emphasis (e.g. `first_name`): an opener may
                // not follow an alphanumeric.
                if marker_char == '_' && text[..i].chars().next_back().is_some_and(is_alnum) {
                    lit.push('_');
                    i += 1;
                    continue;
                }
                let double = i + 1 < n && b[i + 1] == c;
                let mlen = if double { 2 } else { 1 };
                let marker = &text[i..i + mlen];
                if let Some(rel) = find_closing(&text[i + mlen..], marker, marker_char == '_') {
                    let end = i + mlen + rel;
                    push_escaped(&mut out, &mut lit);
                    let (open, close) = if double {
                        ("<strong>", "</strong>")
                    } else {
                        ("<em>", "</em>")
                    };
                    out.push_str(open);
                    out.push_str(&md_inline_depth(&text[i + mlen..end], depth + 1));
                    out.push_str(close);
                    i = end + mlen;
                    continue;
                }
                lit.push(marker_char);
                i += 1;
            },
            _ => {
                let ch = text[i..].chars().next().unwrap();
                let l = ch.len_utf8();
                lit.push_str(&text[i..i + l]);
                i += l;
            },
        }
    }
    push_escaped(&mut out, &mut lit);
    out
}

fn is_alnum(c: char) -> bool {
    c.is_alphanumeric()
}

/// Flush the pending literal buffer into `out`, HTML-escaped.
fn push_escaped(out: &mut String, lit: &mut String) {
    if !lit.is_empty() {
        out.push_str(&html_escape(lit));
        lit.clear();
    }
}

/// Byte offset within `hay` of the first `marker` that qualifies as a closing emphasis
/// delimiter (non-empty content). For underscore emphasis, the char immediately after
/// the closer must not be alphanumeric (avoids intraword matches like `a_b_c`).
fn find_closing(hay: &str, marker: &str, underscore: bool) -> Option<usize> {
    let mut search = 0;
    while let Some(rel) = hay[search..].find(marker) {
        let pos = search + rel;
        if pos == 0 {
            // Empty content (marker immediately follows the opener); keep looking.
            search = pos + marker.len();
            continue;
        }
        if underscore
            && hay[pos + marker.len()..]
                .chars()
                .next()
                .is_some_and(is_alnum)
        {
            search = pos + marker.len();
            continue;
        }
        return Some(pos);
    }
    None
}

fn try_parse_link(s: &str, depth: usize) -> Option<(String, usize)> {
    // s starts with '['.
    let close_br = 1 + s[1..].find(']')?;
    let rest = &s[close_br + 1..];
    if !rest.starts_with('(') {
        return None;
    }
    let url_body = &rest[1..];
    let close_par = find_url_end(url_body)?;
    let consumed = close_br + 2 + close_par + 1;
    let inner = md_inline_depth(&s[1..close_br], depth + 1);
    let url = &url_body[..close_par];
    if let Some(safe) = sanitize_url(url) {
        Some((
            format!(
                "<a href=\"{}\" target=\"_blank\" rel=\"noopener noreferrer\">{inner}</a>",
                html_escape(&safe)
            ),
            consumed,
        ))
    } else {
        Some((inner, consumed))
    }
}

/// Byte index within `s` (which starts just after the opening `(`) of the `)` that closes
/// the link URL, accounting for balanced nested parens (e.g. Wikipedia `.../Foo_(bar)` URLs).
fn find_url_end(s: &str) -> Option<usize> {
    let mut depth = 1usize;
    for (idx, ch) in s.char_indices() {
        match ch {
            '(' => depth += 1,
            ')' => {
                depth -= 1;
                if depth == 0 {
                    return Some(idx);
                }
            },
            _ => {},
        }
    }
    None
}

/// Allow only safe URL schemes for rendered links: absolute `http(s)`/`mailto`, in-page
/// anchors (`#…`), root/relative paths, and scheme-less relative URLs. Returns `None` for
/// `javascript:`, `data:`, `vbscript:`, and any other explicit scheme.
fn sanitize_url(url: &str) -> Option<String> {
    let u = url.trim();
    if u.is_empty() {
        return None;
    }
    let lower = u.to_ascii_lowercase();
    if lower.starts_with("http://")
        || lower.starts_with("https://")
        || lower.starts_with("mailto:")
        || u.starts_with('#')
        || u.starts_with('/')
    {
        return Some(u.to_string());
    }
    // Reject anything with an explicit scheme (a ':' before the first path separator);
    // allow scheme-less relative URLs.
    let scheme_sep = u.find(':');
    let path_sep = u.find(['/', '?', '#']);
    match (scheme_sep, path_sep) {
        (Some(c), Some(p)) if c < p => None,
        (Some(_), None) => None,
        _ => Some(u.to_string()),
    }
}

/// CSS class suffix for a dictionary role's ToC chip tint. Matches loosely (the role string is
/// free LLM text): the four routing roles get distinct accents, anything else stays neutral.
fn dict_role_slug(role: Option<&str>) -> &'static str {
    let Some(role) = role else {
        return "other";
    };
    let r = role.to_ascii_lowercase();
    if r.contains("identifier") {
        "identifier"
    } else if r.contains("measure") {
        "measure"
    } else if r.contains("timestamp") || r.contains("time") || r.contains("date") {
        "timestamp"
    } else if r.contains("dimension") {
        "dimension"
    } else {
        "other"
    }
}

/// Render the human-friendly Data Dictionary document for `--dict-info`: a standalone,
/// script-tag-free HTML page embedded in the Data Schematic (inside a `<script type="text/html">`
/// template), shown in the in-page drawer by `qsvOpenDict` and opened in its own browser tab
/// by `qsvOpenDictTab`.
///
/// Renders straight from the raw dictionary JSON so it can show fields (ranges, cardinality,
/// examples) that the lean routing structs deliberately don't carry; `dict` supplies the
/// shape-normalized dataset description / grain / provenance. Handles both dictionary shapes
/// `parse_dictionary_semantics` accepts (JSON Schema `properties` and legacy `fields`). Sections
/// follow `column_order` (the dataset's header order) so output is deterministic — never the
/// `rows` `HashMap` order; dictionary-only columns are appended sorted. Every dynamic string is
/// `html_escape`d (this is LLM text!), which also guarantees the document can never contain a
/// literal `</script>` that would terminate the embedding template.
fn render_dict_page_html(
    dict_json: &serde_json::Value,
    dict_json_text: &str,
    dict: &DictData,
    dataset_title: &str,
    column_order: &[String],
    view_chart_anchors: &std::collections::HashSet<String>,
    sidecars: &[SidecarDownload],
) -> String {
    use serde_json::Value;

    // compact human rendering of a JSON scalar/array meta value
    fn disp(v: &Value) -> String {
        const MAX_LIST: usize = 12;
        match v {
            Value::String(s) => s.clone(),
            Value::Array(a) => {
                let mut parts: Vec<String> = a.iter().take(MAX_LIST).map(disp).collect();
                if a.len() > MAX_LIST {
                    parts.push(format!("… ({} total)", a.len()));
                }
                parts.join(", ")
            },
            Value::Null => String::new(),
            other => other.to_string(),
        }
    }

    // like `disp`, but comma-groups JSON numbers so stats read as 12,345; anything
    // non-numeric (date/text ranges, enum values, examples) falls through to `disp`.
    fn disp_num(v: &Value) -> String {
        match v {
            Value::Number(n) => {
                let s = n.to_string();
                // serde_json renders extreme-magnitude floats in scientific notation
                // (e.g. `1e-7`, `1e+100`); grouping that would mangle it (`1,e-7`), so
                // leave the exponent form intact and only group plain decimal strings.
                if s.contains(['e', 'E']) {
                    s
                } else {
                    group_thousands(&s)
                }
            },
            _ => disp(v),
        }
    }

    // Index `x-qsv.example_counts` (describegpt's newline-separated "VALUE [count]" string) by
    // bare value, so `disp_examples` can annotate each example with how often it occurs. Lines
    // without a bracketed suffix - notably the `<ALL_UNIQUE>` sentinel - contribute no entry.
    // `rfind` so values that themselves contain " [" aren't truncated mid-value.
    fn example_count_map(raw: &str) -> std::collections::HashMap<&str, &str> {
        raw.lines()
            .filter_map(|line| {
                let line = line.trim_end();
                let idx = line.rfind(" [")?;
                if !line.ends_with(']') {
                    return None;
                }
                Some((line[..idx].trim(), &line[idx + 2..line.len() - 1]))
            })
            .collect()
    }

    // like `disp`, but annotates each example with its occurrence count when `counts` has one,
    // comma-grouped for readability: `LABRADOR RETRIEVER (3,361)`. Values are matched by their
    // rendered form - describegpt writes the same truncation ellipses and date reformatting into
    // both the `examples` array and `example_counts` - and a miss falls back to the bare value.
    //
    // Deliberately driven by the `examples` array, NOT by `example_counts`: for numeric columns
    // describegpt filters the non-numeric "Other…" aggregation-bucket sentinel out of `examples`
    // but keeps it in `example_counts`, so sourcing from the latter would change WHICH entries
    // show, not just add counts.
    fn disp_examples(v: &Value, counts: &std::collections::HashMap<&str, &str>) -> String {
        const MAX_LIST: usize = 12;
        let Value::Array(a) = v else {
            return disp(v);
        };
        let mut parts: Vec<String> = a
            .iter()
            .take(MAX_LIST)
            .map(|e| {
                let val = disp(e);
                match counts.get(val.as_str()) {
                    Some(c) if c.bytes().all(|b| b.is_ascii_digit()) => {
                        format!("{val} ({})", group_thousands(c))
                    },
                    Some(c) => format!("{val} ({c})"),
                    None => val,
                }
            })
            .collect();
        if a.len() > MAX_LIST {
            parts.push(format!("… ({} total)", a.len()));
        }
        parts.join(", ")
    }

    let props = dict_json.get("properties").and_then(Value::as_object);
    let legacy_fields = dict_json
        .get("Dictionary")
        .and_then(|d| d.get("response"))
        .and_then(|r| r.get("fields"))
        .or_else(|| dict_json.get("fields"))
        .and_then(Value::as_array);
    let entry_for = |col: &str| -> Option<&Value> {
        if let Some(p) = props {
            return p.get(col);
        }
        legacy_fields?
            .iter()
            .find(|f| f.get("name").and_then(Value::as_str) == Some(col))
    };

    // display order: the dataset's header order first, then dictionary-only columns (sorted).
    let mut order: Vec<String> = column_order
        .iter()
        .filter(|c| !c.is_empty() && entry_for(c).is_some())
        .cloned()
        .collect();
    let mut extras: Vec<String> = dict
        .rows
        .keys()
        .filter(|k| !order.contains(k))
        .cloned()
        .collect();
    extras.sort_unstable();
    order.extend(extras);

    let mut sections = String::new();
    let mut toc = String::new();
    // Hoisted above the per-column loop: these nine labels are identical for every column, so
    // looking them up per column would be ~9xN catalog lookups and allocations on a wide
    // dictionary. Borrowed as &str below, which also keeps `meta` allocation-free for labels.
    let (
        lbl_type,
        lbl_content_type,
        lbl_role,
        lbl_concept,
        lbl_range,
        lbl_cardinality,
        lbl_null_count,
        lbl_values,
        lbl_examples,
    ) = (
        t!("viz.dict.field_type"),
        t!("viz.dict.field_content_type"),
        t!("viz.dict.field_role"),
        t!("viz.dict.field_concept"),
        t!("viz.dict.field_range"),
        t!("viz.dict.field_cardinality"),
        t!("viz.dict.field_null_count"),
        t!("viz.dict.field_values"),
        t!("viz.dict.field_examples"),
    );
    let role_unknown = t!("viz.dict.role_unknown");
    for col in &order {
        let Some(entry) = entry_for(col) else {
            continue;
        };
        let xq = entry.get("x-qsv");
        let get = |keys: &[&str]| -> Option<&Value> {
            keys.iter()
                .find_map(|k| entry.get(k).filter(|v| !v.is_null()))
        };
        let get_xq = |k: &str| xq.and_then(|x| x.get(k)).filter(|v| !v.is_null());

        let label = get(&["title", "label"])
            .and_then(Value::as_str)
            .unwrap_or_default();
        let description = get(&["description"])
            .and_then(Value::as_str)
            .unwrap_or_default();

        // Labels borrow from the hoisted locals above (no longer `&'static`, since they now
        // come from the message catalog rather than being literals).
        let mut meta: Vec<(&str, String)> = Vec::new();
        if let Some(v) = get(&["type"]) {
            meta.push((&lbl_type, disp(v)));
        }
        if let Some(v) = get_xq("content_type").or_else(|| get(&["content_type"])) {
            meta.push((&lbl_content_type, disp(v)));
        }
        let role = get_xq("role").or_else(|| get(&["role"])).map(disp);
        if let Some(role) = &role {
            meta.push((&lbl_role, role.clone()));
        }
        if let Some(v) = get_xq("concept").or_else(|| get(&["concept"])) {
            meta.push((&lbl_concept, disp(v)));
        }
        // jsonschema numerics ride as minimum/maximum; dates as x-qsv.min/max; legacy as min/max.
        let min = get(&["minimum", "min"]).or_else(|| get_xq("min"));
        let max = get(&["maximum", "max"]).or_else(|| get_xq("max"));
        if min.is_some() || max.is_some() {
            // Range min/max are actual column values, not counts: comma-group them only for
            // genuine quantitative measures. For identifiers, geo codes (a ZIP 15003 is not
            // "15,003"), years and other dimensions, thousands separators corrupt the value,
            // so those render verbatim via `disp`. Cardinality/null_count below are always
            // counts and stay grouped regardless of role.
            let group_range = role
                .as_deref()
                .is_some_and(|r| r.eq_ignore_ascii_case("measure"));
            let fmt_range = |v: &Value| if group_range { disp_num(v) } else { disp(v) };
            meta.push((
                &lbl_range,
                format!(
                    "{} \u{2013} {}",
                    min.map(fmt_range).unwrap_or_default(),
                    max.map(fmt_range).unwrap_or_default()
                ),
            ));
        }
        if let Some(v) = get_xq("cardinality").or_else(|| get(&["cardinality"])) {
            meta.push((&lbl_cardinality, disp_num(v)));
        }
        if let Some(v) = get_xq("null_count").or_else(|| get(&["null_count"])) {
            meta.push((&lbl_null_count, disp_num(v)));
        }
        if let Some(v) = get(&["enum", "enumeration"]) {
            meta.push((&lbl_values, disp(v)));
        }
        if let Some(v) = get(&["examples"]) {
            let raw_counts = get_xq("example_counts")
                .or_else(|| get(&["example_counts"]))
                .and_then(Value::as_str)
                .unwrap_or_default();
            meta.push((
                &lbl_examples,
                disp_examples(v, &example_count_map(raw_counts)),
            ));
        }

        let anchor = dict_anchor_id(col);
        // ToC chip: jump target + a role-tinted left border, so the dataset's semantic shape
        // (identifiers / dimensions / measures / timestamps) is scannable at a glance.
        toc.push_str(&format!(
            "<a class=\"qsv-dict-chip qsv-dict-role-{}\" href=\"#{anchor}\" title=\"{}\">{}</a>\n",
            dict_role_slug(role.as_deref()),
            html_escape(role.as_deref().unwrap_or(&role_unknown)),
            html_escape(col)
        ));
        // "View chart ↗" — reverse navigation to the panel built from this column, rendered
        // only when a matching `data-qsv-dict` cell will exist (the inline render path; the
        // typed grid is ONE plot with no per-panel elements, and dictionary-only or
        // capped-out columns have no cell either) — see `view_chart_anchors` at the call
        // site. In the drawer, click delegation scrolls the cell into view; in the standalone
        // tab the inline handler walks `window.opener`. The selector value needs no quotes:
        // `dict_anchor_id` emits a valid CSS identifier.
        let viewchart = if view_chart_anchors.contains(&anchor) {
            format!(
                "<a class=\"qsv-dict-viewchart\" data-anchor=\"{anchor}\" href=\"#\" \
                 onclick=\"var o=window.opener;if(o){{try{{var \
                 p=o.document.querySelector('[data-qsv-dict={anchor}]');if(p)p.\
                 scrollIntoView({{block:'center'}});}}catch(e){{}}o.focus();}}return \
                 false\">{}&#8599;</a>\n",
                t!("viz.dict.view_chart")
            )
        } else {
            String::new()
        };
        sections.push_str(&format!(
            "<section class=\"qsv-dict-col\" id=\"{anchor}\">\n{viewchart}<h2>{}</h2>\n",
            html_escape(col)
        ));
        if !label.is_empty() && !label.eq_ignore_ascii_case(col) {
            sections.push_str(&format!(
                "<p class=\"qsv-dict-label\">{}</p>\n",
                html_escape(label)
            ));
        }
        if !description.is_empty() {
            sections.push_str(&format!(
                "<div class=\"qsv-dict-desc\">{}</div>\n",
                render_dict_markdown(description)
            ));
        }
        if !meta.is_empty() {
            sections.push_str("<dl class=\"qsv-dict-meta\">\n");
            for (k, v) in &meta {
                if v.is_empty() {
                    continue;
                }
                sections.push_str(&format!("<dt>{k}</dt><dd>{}</dd>\n", html_escape(v)));
            }
            sections.push_str("</dl>\n");
        }
        sections.push_str("</section>\n");
    }

    let title = html_escape(dataset_title);
    let mut intro = String::new();
    if let Some(desc) = dict.dataset_description.as_deref() {
        intro.push_str(&format!(
            "<div class=\"qsv-dict-dataset-desc\">{}</div>\n",
            render_dict_markdown(desc)
        ));
    }
    if let Some(grain) = dict.grain.as_deref() {
        intro.push_str(&format!(
            "<p class=\"qsv-dict-grain\"><strong>{}</strong> {}</p>\n",
            t!("viz.dict.grain"),
            html_escape(grain)
        ));
    }
    let provenance = dict
        .generated_by
        .as_deref()
        .map(|g| {
            // Localize FIRST, escape once at the sink -- the page-wide "escaped exactly once per
            // sink" rule. Escaping first would hand the localizer entity references to match on.
            format!(
                "<footer class=\"qsv-dict-prov\"><pre>{}</pre></footer>\n",
                html_escape(&localize_dict_provenance(g))
            )
        })
        .unwrap_or_default();

    let toc_nav = if toc.is_empty() {
        String::new()
    } else {
        format!("<nav class=\"qsv-dict-toc\">\n{toc}</nav>\n")
    };

    // The download row: "Export JSONSchema" plus one link per generated sidecar this run consumed.
    //
    // Every link is a script-free `<a download>` whose bytes ride as a base64 `data:` URI — base64
    // can't contain `</script>`, so it's safe inside the embedding `<script type="text/html">`
    // template, and needs no JS (which this page can't carry). Authored inline here so the
    // popped-out standalone tab gets them too; `qsvDictDrawer` relocates the whole
    // `.qsv-dict-downloads` container into the drawer's fixed button bar (where the text labels are
    // hidden, leaving just the glyphs).
    //
    // The dictionary export itself is offered ONLY when the dictionary is a JSON Schema (top-level
    // `properties`); legacy `fields` dicts aren't JSON Schema, so they get no button.
    let export_link = if props.is_some() {
        let b64 = base64_simd::STANDARD.encode_to_string(dict_json_text.as_bytes());
        // filename slug: lowercase alphanumerics, everything else -> single underscore, trimmed.
        let mut slug = String::new();
        let mut prev_us = false;
        for c in dataset_title.chars() {
            if c.is_ascii_alphanumeric() {
                slug.push(c.to_ascii_lowercase());
                prev_us = false;
            } else if !prev_us {
                slug.push('_');
                prev_us = true;
            }
        }
        let slug = slug.trim_matches('_');
        let fname = format!(
            "{}.schema.json",
            if slug.is_empty() { "data" } else { slug }
        );
        format!(
            "<a class=\"qsv-dict-export\" download=\"{fname}\" \
             href=\"data:application/json;base64,{b64}\" title=\"{tip}\">&#x2913; <span \
             class=\"qsv-dict-export-label\">{label}</span><span \
             class=\"qsv-dict-export-short\">{short}</span></a>\n",
            tip = html_escape(&t!("viz.dict.export_tooltip")),
            label = t!("viz.dict.export_label"),
            short = t!("viz.dict.export_short")
        )
    } else {
        String::new()
    };

    // one anchor per consumed sidecar, using the same script-free `<a download>` + base64 `data:`
    // pattern. The `download` attribute is the sidecar's own basename (never the title slug), so
    // several downloads from one page can't collide.
    let mut sidecar_links = String::new();
    for sc in sidecars {
        let b64 = base64_simd::STANDARD.encode_to_string(&sc.bytes);
        let fname = html_escape(&sc.fname);
        sidecar_links.push_str(&format!(
            "<a class=\"qsv-dict-export\" download=\"{fname}\" href=\"data:{};base64,{b64}\" \
             title=\"{tip}\">&#x2913; <span class=\"qsv-dict-export-label\">{fname}</span><span \
             class=\"qsv-dict-export-short\">{}</span></a>\n",
            sc.mime,
            html_escape(sc.short),
            // Interpolate the RAW file name and escape once, at the end. Feeding the
            // already-escaped `fname` in here would escape it twice, so a file named `a&b.csv`
            // would surface a literal `&amp;` in the rendered tooltip. `fname` stays escaped for
            // the `download=` attribute and the label span, which are separate contexts.
            tip = html_escape(&t!(
                "viz.dict.sidecar_download_tooltip",
                q_file = sc.fname.clone()
            ))
        ));
    }

    // wrap the export + sidecar links in ONE container: it keeps them on a single wrapping row in
    // the page, and gives `qsvDictDrawer` a single element to relocate into the drawer's button bar
    // (a per-anchor move would leave all but the first behind).
    let downloads = if export_link.is_empty() && sidecar_links.is_empty() {
        String::new()
    } else {
        format!("<div class=\"qsv-dict-downloads\">\n{export_link}{sidecar_links}</div>\n")
    };

    // Fixed neutral light/dark palettes keyed on `qsv-dark` (set on the standalone page's body
    // by `qsvOpenDictTab`, inherited from the Data Schematic body in the drawer), mirroring the
    // Data Schematic's theme-toggle convention. Every rule is scoped under `.qsv-dict-doc` /
    // `.qsv-dict-body` / `#qsv-dict-drawer`, because the drawer adopts this whole `<style>` into
    // the Data Schematic's head — an unscoped `body` rule would restyle the Data Schematic itself.
    // The drawer rules ride along here (inert in the standalone page) so the drawer and the
    // page stay one document with one stylesheet. NOTE: this page must never contain a literal
    // `</script>` (it is embedded in a `<script type="text/html">` template), so it is
    // script-tag-free by design — standalone-tab interactivity is inline `onclick` handlers
    // only, which the drawer strips.
    let ui_lang = viz_i18n::active_locale().bcp47;
    let dict_page_title = t!("viz.dict.title");
    let back_to_dashboard = t!("viz.dict.back_to_dashboard");
    format!(
        r##"<!doctype html>
<html lang="{ui_lang}">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>{dict_page_title} — {title}</title>
<style>
  body.qsv-dict-body {{ font-family: {FONT_FAMILY}; margin: 0; padding: 24px; background: #ffffff; color: #222222; }}
  body.qsv-dict-body.qsv-dark {{ background: #14171c; color: #d7dce2; }}
  .qsv-dict-doc {{ font-family: {FONT_FAMILY}; }}
  .qsv-dict-wrap {{ max-width: 860px; margin: 0 auto; }}
  .qsv-dict-doc h1 {{ font-size: 20px; font-weight: 600; margin: 8px 0 12px; }}
  .qsv-dict-back {{ display: inline-block; font-size: 13px; color: #2563eb; text-decoration: none; margin-bottom: 4px; }}
  .qsv-dark .qsv-dict-back {{ color: #6ea8ff; }}
  .qsv-dict-dataset-desc {{ font-size: 14px; line-height: 1.5; }}
  .qsv-dict-grain {{ font-size: 13px; }}
  /* the download row: the JSONSchema export plus one link per sidecar this Data Schematic was built
     from, wrapping onto as many lines as they need */
  .qsv-dict-downloads {{ display: flex; flex-wrap: wrap; gap: 6px; margin: 8px 0 4px; }}
  .qsv-dict-export {{ display: inline-block; font-size: 12px; padding: 3px 11px; border: 1px solid rgba(128, 128, 128, 0.45); border-radius: 6px; color: inherit; text-decoration: none; }}
  .qsv-dict-export:hover {{ background: rgba(128, 128, 128, 0.15); }}
  /* the one-word tag is for the narrow drawer strip only; the page shows the full filename */
  .qsv-dict-export-short {{ display: none; }}
  /* in the drawer the row becomes a fixed strip under the title bar: full filenames give way to
     the one-word tags (five of them fit on a line, five filenames don't), and the strip stays put
     while the dictionary scrolls beneath it */
  #qsv-dict-drawer > .qsv-dict-downloads {{ flex: none; padding: 8px 16px; margin: 0; gap: 6px 8px; border-bottom: 1px solid rgba(128, 128, 128, 0.35); }}
  #qsv-dict-drawer > .qsv-dict-downloads .qsv-dict-export {{ padding: 2px 9px; white-space: nowrap; }}
  #qsv-dict-drawer > .qsv-dict-downloads .qsv-dict-export-label {{ display: none; }}
  #qsv-dict-drawer > .qsv-dict-downloads .qsv-dict-export-short {{ display: inline; }}
  .qsv-dict-toc {{ display: flex; flex-wrap: wrap; gap: 6px; margin: 14px 0 10px; }}
  .qsv-dict-chip {{ font-size: 12px; padding: 2px 10px; border: 1px solid rgba(128, 128, 128, 0.4); border-left: 4px solid #9ca3af; border-radius: 999px; color: inherit; text-decoration: none; }}
  .qsv-dict-chip:hover {{ background: rgba(128, 128, 128, 0.15); }}
  .qsv-dict-role-identifier {{ border-left-color: #8b5cf6; }}
  .qsv-dict-role-dimension {{ border-left-color: #2563eb; }}
  .qsv-dict-role-measure {{ border-left-color: #059669; }}
  .qsv-dict-role-timestamp {{ border-left-color: #d97706; }}
  section.qsv-dict-col {{ border-top: 1px solid rgba(128, 128, 128, 0.35); padding: 12px 0 8px; scroll-margin-top: 8px; }}
  section.qsv-dict-col h2 {{ font-size: 15px; font-weight: 600; margin: 4px 0; }}
  /* flash the section a reader just jumped to: `:target` covers native anchor navigation
     (the ToC chips in the standalone tab), `.qsv-dict-hl` the scripted jumps (info icons,
     drawer chips) via qsvDictReveal */
  section.qsv-dict-col:target, section.qsv-dict-col.qsv-dict-hl {{ animation: qsv-dict-flash 1.6s ease-out 1; }}
  @keyframes qsv-dict-flash {{ 0% {{ background: rgba(37, 99, 235, 0.22); }} 100% {{ background: transparent; }} }}
  .qsv-dict-viewchart {{ float: right; font-size: 12px; margin: 6px 0 0 12px; color: #2563eb; text-decoration: none; }}
  .qsv-dark .qsv-dict-viewchart {{ color: #6ea8ff; }}
  .qsv-dict-label {{ font-size: 13px; color: #888888; margin: 2px 0 6px; }}
  .qsv-dict-desc {{ font-size: 13px; line-height: 1.5; margin: 4px 0 8px; }}
  .qsv-dict-desc h4.qsv-dict-mdh, .qsv-dict-dataset-desc h4.qsv-dict-mdh {{ font-size: 13.5px; font-weight: 600; margin: 10px 0 4px; }}
  .qsv-dict-desc p, .qsv-dict-dataset-desc p {{ margin: 0 0 6px; }}
  .qsv-dict-desc p:last-child, .qsv-dict-dataset-desc p:last-child {{ margin-bottom: 0; }}
  .qsv-dict-desc ul, .qsv-dict-dataset-desc ul {{ margin: 4px 0 6px; padding-left: 20px; }}
  .qsv-dict-desc li, .qsv-dict-dataset-desc li {{ margin: 1px 0; }}
  .qsv-dict-desc code, .qsv-dict-dataset-desc code {{ font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; font-size: 0.92em; padding: 1px 4px; border-radius: 3px; background: rgba(128, 128, 128, 0.16); }}
  .qsv-dict-desc a, .qsv-dict-dataset-desc a {{ color: #2563eb; text-decoration: underline; }}
  .qsv-dark .qsv-dict-desc a, .qsv-dark .qsv-dict-dataset-desc a {{ color: #6ea8ff; }}
  dl.qsv-dict-meta {{ display: grid; grid-template-columns: max-content 1fr; gap: 2px 14px; font-size: 12px; margin: 4px 0 8px; }}
  dl.qsv-dict-meta dt {{ color: #888888; }}
  dl.qsv-dict-meta dd {{ margin: 0; overflow-wrap: anywhere; }}
  footer.qsv-dict-prov {{ margin-top: 20px; font-size: 11px; color: #888888; }}
  footer.qsv-dict-prov pre {{ white-space: pre-wrap; font-family: inherit; }}
  /* ---- in-page drawer chrome (adopted into the Data Schematic head on first open; inert here) ---- */
  #qsv-dict-drawer {{ position: fixed; top: 0; right: 0; bottom: 0; width: min(480px, 42vw); transform: translateX(103%); transition: transform 0.22s ease; z-index: 1100; display: flex; flex-direction: column; background: #ffffff; color: #222222; border-left: 1px solid rgba(128, 128, 128, 0.4); box-shadow: -6px 0 18px rgba(0, 0, 0, 0.18); font-family: {FONT_FAMILY}; }}
  .qsv-dark #qsv-dict-drawer {{ background: #14171c; color: #d7dce2; }}
  #qsv-dict-drawer.open {{ transform: none; }}
  .qsv-dict-drawer-bar {{ flex: none; display: flex; align-items: center; justify-content: space-between; padding: 10px 16px; border-bottom: 1px solid rgba(128, 128, 128, 0.35); font-size: 14px; font-weight: 600; }}
  .qsv-dict-drawer-btns a {{ color: inherit; text-decoration: none; font-size: 15px; margin-left: 14px; opacity: 0.7; }}
  .qsv-dict-drawer-btns a:hover {{ opacity: 1; }}
  .qsv-dict-drawer-content {{ flex: 1; overflow-y: auto; padding: 0 16px 16px; }}
  .qsv-dict-drawer-content .qsv-dict-wrap {{ max-width: none; }}
  /* the drawer bar already titles the panel; the page header (h1 + back link) is tab-only */
  .qsv-dict-embedded .qsv-dict-head {{ display: none; }}
  /* the footer row (credits + Theme toggle + logo) is in flow, so this one margin keeps it — and
     the whole page — beside the open drawer rather than under it */
  body.qsv-dict-open {{ margin-right: min(480px, 42vw); }}
  @media (max-width: 900px) {{ body.qsv-dict-open {{ margin-right: 0; }} #qsv-dict-drawer {{ width: min(480px, 94vw); }} }}
</style>
</head>
<body id="qsv-dict-root" class="qsv-dict-body">
<div class="qsv-dict-wrap qsv-dict-doc">
<header class="qsv-dict-head">
<a class="qsv-dict-back" href="#" onclick="var o=window.opener;if(o&&!o.closed){{try{{o.focus();}}catch(e){{}}}}window.close();return false">&#8592; {back_to_dashboard}</a>
<h1>{dict_page_title} — {title}</h1>
</header>
{downloads}{intro}{toc_nav}{sections}{provenance}</div>
</body>
</html>"##
    )
}

/// The panel-title annotation for typed-grid cells (cartesian and geo) AND `--dict-info` inline
/// panels (via `inline_panel_title`). When the panel carries
/// a `--dict-info` description, the title line grows a muted ⓘ, the annotation hover box shows
/// the description, and `capture_events` + a `qsvdict-` prefixed `name` route its click through
/// the `plotly_clickannotation` hook to the Data Dictionary tab. No OTHER annotation may set
/// `capture_events`: the hook's name-prefix filter plus this exclusivity keep corr-cell and
/// geo-meta annotations from opening the tab.
fn panel_title_annotation(panel: &Panel, title_x: f64, title_y: f64, font: Font) -> Annotation {
    let mut ann = Annotation::new()
        .text(panel.display_title_with_icon(true))
        .x(title_x)
        .y(title_y)
        .x_ref("paper")
        .y_ref("paper")
        .x_anchor(Anchor::Center)
        .y_anchor(Anchor::Bottom)
        .show_arrow(false)
        .font(font);
    if let Some((anchor, desc)) = &panel.dict_info {
        ann = ann
            .hover_text(html_escape(desc))
            .capture_events(true)
            .name(anchor.clone());
    }
    ann
}

fn load_dictionary_semantics(args: &Args) -> CliResult<Option<(DictData, String)>> {
    let Some(spec) = args.flag_dictionary.as_deref() else {
        if args.flag_dictionary_context.is_some() {
            viz_note("viz smart --dictionary-context: ignored without --dictionary infer.");
        }
        return Ok(None);
    };
    let Some(input) = args.arg_input.as_deref() else {
        return Ok(None);
    };

    let is_infer = spec.eq_ignore_ascii_case("infer");
    if args.flag_dictionary_context.is_some() && !is_infer {
        viz_note(
            "viz smart --dictionary-context: only applies to `--dictionary infer` (a generated \
             dictionary); ignored when reading an existing dictionary file.",
        );
    }

    // Set to the sidecar path ONLY when we infer a fresh dictionary this run, so we persist it
    // after a successful parse (and never overwrite a reused/hand-edited one).
    let mut persist_to: Option<std::path::PathBuf> = None;
    // Captured describegpt stderr + wall-clock elapsed, set ONLY on a fresh infer, so the
    // post-parse summary can report the model/tokens/elapsed of what the LLM just produced.
    let mut infer_meta: Option<(String, std::time::Duration)> = None;
    // Human label for the dictionary source, used in the "not recognizable" fallback message so a
    // broken hand-edited sidecar points the user at the file to fix (not the literal "infer").
    let mut source_label = spec.to_string();

    let json_text = if is_infer {
        let sidecar = dictionary_sidecar_path(input);
        if sidecar.exists() && !viz_dict_fresh() {
            // Reuse the on-disk dictionary from a prior infer (or the user's fine-tuned edit),
            // skipping the LLM entirely. `QSV_VIZ_DICT_FRESH` opts out of this reuse (see
            // `viz_dict_fresh`), re-inferring even when the sidecar is present.
            match std::fs::read_to_string(&sidecar) {
                Ok(text) => {
                    // surface the model the reused dictionary was generated with, when it carries
                    // one (infer-generated files do; hand-written ones may not).
                    let model_note = parse_dict_model(&text)
                        .map(|m| format!(", model {m}"))
                        .unwrap_or_default();
                    viz_note(&format!(
                        "viz smart --dictionary infer: reusing existing dictionary \
                         '{}'{model_note} (delete it to re-infer).",
                        sidecar.display()
                    ));
                    // (only reachable on the reuse path, so a `QSV_VIZ_DICT_FRESH` run - which
                    // skips reuse entirely - honors --dictionary-context rather than ignoring it.)
                    if args.flag_dictionary_context.is_some() {
                        viz_note(&format!(
                            "viz smart --dictionary-context: ignored when reusing the existing \
                             dictionary '{}' (the dictionary is already built); delete it to \
                             re-infer with context.",
                            sidecar.display()
                        ));
                    }
                    source_label = sidecar.display().to_string();
                    text
                },
                Err(e) => {
                    viz_note(&format!(
                        "viz smart --dictionary infer: could not read existing dictionary '{}' \
                         ({e}); building the Data Schematic from statistics alone.",
                        sidecar.display()
                    ));
                    return Ok(None);
                },
            }
        } else {
            // forward --dictionary-context to describegpt as --context-file when present, so the
            // LLM's role/concept/label/grain inference is conditioned on the user's domain context.
            let mut dg_args: Vec<&str> = vec![
                "--dictionary",
                "--description",
                "--infer-content-type",
                "--two-pass",
                "--format",
                "jsonschema",
            ];
            if let Some(ctx) = args.flag_dictionary_context.as_deref() {
                dg_args.push("--context-file");
                dg_args.push(ctx);
            }
            // An explicit --language must reach describegpt too, or the inferred field
            // titles/descriptions come back in the dataset's own language while the chrome
            // renders in the requested one -- a worse result than either alone. Only
            // forwarded when the user asked: left to itself, describegpt autodetects, which
            // is exactly the behavior the "auto" default wants. describegpt documents
            // language NAMES ("Spanish"), so forward that rather than the BCP-47 tag.
            // Keyed on "was the flag explicitly set", NOT on "is it non-English": an
            // explicit `--language en` over a Spanish dataset must also pin describegpt to
            // English, or the labels come back Spanish inside English chrome.
            if args
                .flag_language
                .as_deref()
                .map(str::trim)
                .is_some_and(|f| !f.is_empty() && !f.eq_ignore_ascii_case("auto"))
                && let viz_i18n::Resolution::Curated(row) =
                    viz_i18n::resolve(args.flag_language.as_deref(), None)
            {
                dg_args.push("--language");
                dg_args.push(row.english_name);
            }
            if viz_dict_fresh() {
                // Bypass describegpt's completion cache too, not just the sidecar — otherwise a
                // re-infer replays the cached completion and yields a byte-identical dictionary,
                // making the whole re-infer a silent no-op.
                dg_args.push("--fresh");
            }
            let infer_start = Instant::now();
            match util::run_qsv_cmd(
                "describegpt",
                &dg_args,
                input,
                "Generated a Data Dictionary via describegpt for `viz smart --dictionary infer`",
            ) {
                Ok((stdout, stderr)) => {
                    persist_to = Some(sidecar);
                    infer_meta = Some((stderr, infer_start.elapsed()));
                    stdout
                },
                Err(e) => {
                    viz_note(&format!(
                        "viz smart --dictionary infer: describegpt failed ({e}); building the \
                         Data Schematic from statistics alone (no semantic hints)."
                    ));
                    return Ok(None);
                },
            }
        }
    } else {
        match std::fs::read_to_string(spec) {
            Ok(text) => text,
            Err(e) => {
                viz_note(&format!(
                    "viz smart --dictionary: could not read dictionary file '{spec}' ({e}); \
                     building the Data Schematic from statistics alone (no semantic hints)."
                ));
                return Ok(None);
            },
        }
    };

    let data = parse_dictionary_semantics(&json_text);

    // Report a fresh inference the user just paid for: dictionary name, and — when available —
    // the model, elapsed time, and token usage. `infer_meta` is Some only on a fresh describegpt
    // run (never on reuse); gate on a usable parse so we don't advertise unreadable output.
    if let (Some((stderr, elapsed)), Some(path), Some(_)) =
        (infer_meta.as_ref(), persist_to.as_ref(), data.as_ref())
    {
        let name = path
            .file_name()
            .map(|s| s.to_string_lossy().into_owned())
            .unwrap_or_else(|| path.display().to_string());
        let mut summary = format!(
            "viz smart: inferred data dictionary '{name}' in {}",
            HumanDuration(*elapsed)
        );
        if let Some(model) = parse_dict_model(&json_text) {
            summary.push_str(&format!(", model {model}"));
        }
        if let Some(tokens) = sum_dict_tokens(stderr) {
            summary.push_str(&format!(", {} tokens", HumanCount(tokens)));
        }
        viz_note(&format!("{summary}."));
    }

    // Persist a freshly-inferred dictionary next to the input so users can fine-tune it and later
    // runs can reuse it. Only when the parse succeeded (never persist output the loader can't read)
    // and only for a fresh infer (never clobber a reused/hand-edited file) - except under
    // `QSV_VIZ_DICT_FRESH`, whose whole point is to re-infer and overwrite the sidecar in place.
    // Best-effort: warn on a write failure, matching this function's soft-fail-to-stats philosophy.
    // The write is atomic (see `write_dictionary_sidecar`) so a failure here leaves any existing
    // dictionary intact rather than truncated.
    if let (Some(path), Some(_)) = (persist_to.as_ref(), data.as_ref()) {
        match write_dictionary_sidecar(path, &json_text) {
            Ok(()) => viz_note(&format!(
                "viz smart --dictionary infer: saved inferred dictionary to '{}' (edit it to \
                 fine-tune; reused on later runs).",
                path.display()
            )),
            Err(e) => viz_note(&format!(
                "viz smart --dictionary infer: could not save dictionary to '{}' ({e}); \
                 continuing (it will be re-inferred next run).",
                path.display()
            )),
        }
    }

    if data.is_none() {
        viz_note(&format!(
            "viz smart --dictionary: '{source_label}' is not a recognizable describegpt \
             dictionary (JSON Schema or JSON); building the Data Schematic from statistics alone."
        ));
    }
    // The raw JSON rides along so --dict-info can render the full dictionary page from fields
    // (descriptions, ranges, examples) that the lean routing structs deliberately don't carry.
    Ok(data.map(|d| (d, json_text)))
}

/// Decide which chart (if any) suits a column, from its computed statistics.
fn classify(idx: usize, s: &crate::cmd::stats::StatsData) -> Option<PanelKind> {
    let ty = s.r#type.as_str();
    if ty == "NULL" {
        return None; // all-empty column
    }
    let near_unique = s.uniqueness_ratio.is_some_and(|r| r > 0.95);
    let low_cardinality =
        s.cardinality >= 1 && s.cardinality <= CATEGORICAL_MAX_CARDINALITY && !near_unique;

    if ty == "Boolean" {
        return Some(PanelKind::FreqBar { idx });
    }

    if matches!(ty, "Integer" | "Float") {
        // near-unique integers are almost certainly IDs/keys - not meaningful to chart
        if ty == "Integer" && near_unique {
            return None;
        }
        // low-cardinality numeric (codes/ratings) -> frequency bar, NOT a box plot
        if low_cardinality {
            return Some(PanelKind::FreqBar { idx });
        }
        // continuous numeric -> box plot / histogram from precomputed quartiles. Shared with the
        // dictionary `measure` verdict (see `classify_measure`) so a measure is charted the same
        // way however it was identified; returns None when the column lacks quartiles.
        // (Zero-padded numeric CODES need no exclusion here: stats only emits
        // `zero_padded_numeric` for String-typed columns — see `is_zpn` in stats.rs — so a
        // flagged column never reaches this numeric arm; it takes the categorical path below.
        // The flag matters in `guardrail`, where it overrides a dictionary measure verdict.)
        return classify_measure(idx, s);
    }

    // String / Date / DateTime -> frequency bar when low-cardinality
    if low_cardinality {
        return Some(PanelKind::FreqBar { idx });
    }
    // moarstats refinement: a high-cardinality categorical is normally skipped as ID-like noise.
    // But normalized_entropy distinguishes "near-uniform (truly noise)" from "concentrated (a few
    // dominant categories)". When moarstats says the distribution is concentrated, a top-N
    // frequency bar is still informative, so chart it. Absent moarstats (None), keep today's
    // behavior and skip.
    if !near_unique
        && s.normalized_entropy
            .is_some_and(|e| e < HIGH_CARD_ENTROPY_NOISE_THRESHOLD)
    {
        return Some(PanelKind::FreqBar { idx });
    }
    None // high-cardinality / ID-like text
}

/// Max number of fields (identifier + extras) shown in a `viz smart --smarter` map-point hover.
const MAP_HOVER_MAX_FIELDS: usize = 4;
// describegpt concept tokens (see `describegpt::dictionary::CONCEPT_VOCAB`) used to pick map-hover
// fields, in descending preference within each category.
const MAP_ID_CONCEPTS: &[&str] = &[
    "id.natural_key",
    "id.uuid",
    "id.surrogate_key",
    "id.foreign_key",
];
const MAP_NAME_CONCEPTS: &[&str] = &[
    "pii.full_name",
    "org.company",
    "org.agency",
    "geo.city",
    "geo.street_address",
];
const MAP_MEASURE_CONCEPTS: &[&str] = &["measure.amount", "measure.count", "measure.ratio"];
const MAP_CATEGORY_CONCEPTS: &[&str] = &["category.status", "category.type", "category.channel"];
const MAP_GEO_CONTEXT_CONCEPTS: &[&str] = &["geo.country", "geo.state", "geo.city"];

/// Observed numeric range (`|max - min|`) of a column from its stats, or 0.0 when unparseable.
fn stat_range(s: &crate::cmd::stats::StatsData) -> f64 {
    let parse = |o: &Option<String>| o.as_deref().and_then(|v| v.trim().parse::<f64>().ok());
    match (parse(&s.min), parse(&s.max)) {
        (Some(lo), Some(hi)) => (hi - lo).abs(),
        _ => 0.0,
    }
}

/// A column is unavailable for a hover field when it's the lat/lon pair or already chosen.
fn map_field_used(i: usize, lat_idx: usize, lon_idx: usize, chosen: &[usize]) -> bool {
    i == lat_idx || i == lon_idx || chosen.contains(&i)
}

/// First available (not lat/lon, not already chosen) column whose `concept` exactly matches one of
/// `preferred`, scanning `preferred` in order.
fn first_col_by_concepts(
    col_sems: &[ColSemantics],
    preferred: &[&str],
    lat_idx: usize,
    lon_idx: usize,
    chosen: &[usize],
) -> Option<usize> {
    preferred.iter().find_map(|c| {
        col_sems
            .iter()
            .enumerate()
            .find(|(i, s)| !map_field_used(*i, lat_idx, lon_idx, chosen) && s.concept == *c)
            .map(|(i, _)| i)
    })
}

fn near_unique_col(s: &crate::cmd::stats::StatsData) -> bool {
    s.uniqueness_ratio.is_some_and(|r| r > 0.95)
}

/// Pick the single best human-readable identifier column for a `viz smart` map-point hover, or
/// `None` when nothing suitable exists. Priority (lat/lon always excluded; ties → lowest index):
/// dictionary `id.*` concept, then a name-like concept (`pii.full_name`/`org.*`/`geo.city`/
/// `geo.street_address`), then dictionary `role == "identifier"`, then a statistical fallback —
/// the near-unique column, preferring a `String` (a readable name) over an `Integer` (an opaque
/// key). The statistical fallback runs whether or not a dictionary is present, so plain
/// `viz smart` still labels points.
fn select_map_identifier(
    stats: &[crate::cmd::stats::StatsData],
    col_sems: &[ColSemantics],
    dict: Option<&DictData>,
    lat_idx: usize,
    lon_idx: usize,
) -> Option<usize> {
    let is_coord = |i: usize| i == lat_idx || i == lon_idx;

    // 1 & 2: dictionary concept — id.* then name-like, in preference order
    if let Some(i) = first_col_by_concepts(
        col_sems,
        &[MAP_ID_CONCEPTS, MAP_NAME_CONCEPTS].concat(),
        lat_idx,
        lon_idx,
        &[],
    ) {
        return Some(i);
    }

    // 3: dictionary role == "identifier" (keyed by field name)
    if let Some(d) = dict
        && let Some(i) = stats.iter().enumerate().find_map(|(i, s)| {
            (!is_coord(i)
                && d.rows
                    .get(&s.field)
                    .is_some_and(|r| r.role.eq_ignore_ascii_case("identifier")))
            .then_some(i)
        })
    {
        return Some(i);
    }

    // 4: statistical fallback — near-unique column, prefer String over Integer
    for want in ["String", "Integer"] {
        if let Some(i) = stats.iter().enumerate().find_map(|(i, s)| {
            (!is_coord(i) && s.r#type == want && near_unique_col(s)).then_some(i)
        }) {
            return Some(i);
        }
    }
    None
}

/// Common image file extensions recognized by `looks_like_image_url`. Kept deliberately narrow —
/// a false positive here puts a broken `<img>` in the preview card (and makes the page reference
/// an unrelated host), while a false negative merely leaves `--photos` inert on that column.
const IMAGE_URL_EXTS: &[&str] = &[
    ".jpg", ".jpeg", ".png", ".gif", ".webp", ".avif", ".bmp", ".tif", ".tiff",
];

/// The URL up to (but not including) any `#fragment` or `?query`. Used for image-extension
/// DETECTION only — never for the embedded payload, which must keep the query (see
/// `strip_fragment`).
fn strip_url_suffix(url: &str) -> &str {
    url.split(['#', '?']).next().unwrap_or(url)
}

/// The URL with any `#fragment` removed but the `?query` PRESERVED. This is what the embedded
/// payload uses: presigned S3/CloudFront (and similar) image links carry required auth in the
/// query string, so dropping it would embed a URL that 404s or fetches the wrong object. A
/// fragment (Boston 311's `#spot=<uuid>`) is never sent to the server, so it is safe to drop —
/// and dropping it keeps identical images from looking distinct.
fn strip_fragment(url: &str) -> &str {
    url.split('#').next().unwrap_or(url)
}

/// Whether a raw cell value looks like a link to an image: an `http(s)` URL whose path ends in one
/// of `IMAGE_URL_EXTS`.
///
/// Two shapes in real data have to be handled before the extension test:
///   * **multi-value cells** — Boston 311's `closed_photo` packs several URLs into one cell
///     separated by ` | `, so only the FIRST segment is examined.
///   * **fragment / query suffixes** — the same feed appends `#spot=<uuid>`, which would otherwise
///     hide the `.jpg`. Both `#...` and `?...` are trimmed before matching.
fn looks_like_image_url(value: &str) -> bool {
    let first = value.split('|').next().unwrap_or("").trim();
    if !(first.starts_with("http://") || first.starts_with("https://")) {
        return false;
    }
    let lower = strip_url_suffix(first).to_ascii_lowercase();
    IMAGE_URL_EXTS.iter().any(|ext| lower.ends_with(ext))
}

/// Split a raw cell into its individual image URLs, dropping only the `#fragment` from each (the
/// `?query` is preserved — see `strip_fragment` — so presigned links keep working). Used both to
/// build the embedded per-point payload and (via `looks_like_image_url`) to decide whether a
/// column qualifies at all.
fn split_image_urls(value: &str) -> Vec<String> {
    value
        .split('|')
        .map(str::trim)
        .filter(|s| looks_like_image_url(s))
        .map(|s| strip_fragment(s).to_string())
        .collect()
}

/// Columns holding image URLs, detected from the stats cache ALONE so `--photos` works for a bare
/// `qsv viz smart data.csv` with no Data Dictionary (where `content_type` is empty and the
/// semantic precedence chain falls through to statistics).
///
/// `min`/`max` are the only real cell values the stats cache carries per column, and they are used
/// in preference to `mode`/`antimode` because those two arrive wrapped in qsv's own display
/// formatting (a `*PREVIEW:` prefix, `|`-joined ties) that would have to be unwrapped first.
///
/// An all-NULL column is inherently excluded: it stats as type `NULL` with empty min/max, so
/// nothing matches. That is the desired behavior — Boston 311's `submitted_photo` is empty across
/// all 267k rows, and detecting it from its NAME would produce a lightbox that never loads.
fn image_url_cols(stats: &[crate::cmd::stats::StatsData]) -> Vec<usize> {
    stats
        .iter()
        .enumerate()
        .filter(|(_, s)| {
            s.min.as_deref().is_some_and(looks_like_image_url)
                || s.max.as_deref().is_some_and(looks_like_image_url)
        })
        .map(|(i, _)| i)
        .collect()
}

/// Choose the ordered list of column indices to show in a `viz smart` map-point hover. Always
/// starts with the identifier from `select_map_identifier` (0 or 1 column). Without `--smarter`,
/// that's all. With `--smarter`, appends the best additional fields (capped at
/// `MAP_HOVER_MAX_FIELDS` total), driven by the dictionary when present — a primary measure, a key
/// category, a time field, and geo context — or by statistics otherwise (the highest-variation
/// numeric measure, then the lowest-cardinality category). lat/lon and already-chosen columns are
/// never repeated.
fn select_map_hover_fields(
    stats: &[crate::cmd::stats::StatsData],
    col_sems: &[ColSemantics],
    dict: Option<&DictData>,
    lat_idx: usize,
    lon_idx: usize,
    smarter: bool,
) -> Vec<usize> {
    let mut chosen: Vec<usize> = Vec::new();
    if let Some(id) = select_map_identifier(stats, col_sems, dict, lat_idx, lon_idx) {
        chosen.push(id);
    }
    if !smarter {
        return chosen;
    }

    let room = |chosen: &[usize]| chosen.len() < MAP_HOVER_MAX_FIELDS;

    if dict.is_some() {
        // measure: concept priority, else any `measure.*` (tie → largest observed range)
        if room(&chosen) {
            let m =
                first_col_by_concepts(col_sems, MAP_MEASURE_CONCEPTS, lat_idx, lon_idx, &chosen)
                    .or_else(|| {
                        col_sems
                            .iter()
                            .enumerate()
                            .filter(|(i, s)| {
                                !map_field_used(*i, lat_idx, lon_idx, &chosen)
                                    && s.concept.starts_with("measure.")
                            })
                            .max_by(|(ia, _), (ib, _)| {
                                stat_range(&stats[*ia])
                                    .partial_cmp(&stat_range(&stats[*ib]))
                                    .unwrap_or(std::cmp::Ordering::Equal)
                            })
                            .map(|(i, _)| i)
                    });
            if let Some(i) = m {
                chosen.push(i);
            }
        }
        // category: concept priority, else any `category.*`/`org.*` (tie → lowest cardinality)
        if room(&chosen) {
            let c =
                first_col_by_concepts(col_sems, MAP_CATEGORY_CONCEPTS, lat_idx, lon_idx, &chosen)
                    .or_else(|| {
                        col_sems
                            .iter()
                            .enumerate()
                            .filter(|(i, s)| {
                                !map_field_used(*i, lat_idx, lon_idx, &chosen)
                                    && (s.concept.starts_with("category.")
                                        || s.concept.starts_with("org."))
                            })
                            .min_by_key(|(i, _)| stats[*i].cardinality)
                            .map(|(i, _)| i)
                    });
            if let Some(i) = c {
                chosen.push(i);
            }
        }
        // time: any `time.*`
        if room(&chosen)
            && let Some(i) = col_sems.iter().enumerate().find_map(|(i, s)| {
                (!map_field_used(i, lat_idx, lon_idx, &chosen) && s.concept.starts_with("time."))
                    .then_some(i)
            })
        {
            chosen.push(i);
        }
        // geo context: country/state/city
        if room(&chosen)
            && let Some(i) = first_col_by_concepts(
                col_sems,
                MAP_GEO_CONTEXT_CONCEPTS,
                lat_idx,
                lon_idx,
                &chosen,
            )
        {
            chosen.push(i);
        }
    } else {
        // no dictionary: primary measure by coefficient of variation (fallback range)
        if room(&chosen) {
            let m = stats
                .iter()
                .enumerate()
                .filter(|(i, s)| {
                    !map_field_used(*i, lat_idx, lon_idx, &chosen)
                        && matches!(s.r#type.as_str(), "Integer" | "Float")
                        && !near_unique_col(s)
                })
                .max_by(|(_, sa), (_, sb)| {
                    let score = |s: &crate::cmd::stats::StatsData| {
                        s.cv.map(f64::abs).unwrap_or_else(|| stat_range(s))
                    };
                    score(sa)
                        .partial_cmp(&score(sb))
                        .unwrap_or(std::cmp::Ordering::Equal)
                })
                .map(|(i, _)| i);
            if let Some(i) = m {
                chosen.push(i);
            }
        }
        // then a low-cardinality category (lowest cardinality first)
        if room(&chosen) {
            let c = stats
                .iter()
                .enumerate()
                .filter(|(i, s)| {
                    !map_field_used(*i, lat_idx, lon_idx, &chosen)
                        && s.cardinality >= 1
                        && s.cardinality <= CATEGORICAL_MAX_CARDINALITY
                        && !near_unique_col(s)
                })
                .min_by_key(|(_, s)| s.cardinality)
                .map(|(i, _)| i);
            if let Some(i) = c {
                chosen.push(i);
            }
        }
    }
    chosen
}

/// Parse a stats-cache min/max cell (stored as a string) as a finite f64.
fn parse_stat_f64(v: Option<&str>) -> Option<f64> {
    v.and_then(|v| v.trim().parse::<f64>().ok())
        .filter(|v| v.is_finite())
}

/// Pearson's second skewness coefficient for a column: moarstats' precomputed value when present
/// (`--smarter`), else derived from the base stats cache with the same formula moarstats uses —
/// `3 * (mean - median) / stddev` — so plain `viz smart` gets the same signal with no extra pass.
fn pearson_skewness_stat(s: &crate::cmd::stats::StatsData) -> Option<f64> {
    s.pearson_skewness.or_else(|| {
        let (Some(mean), Some(median), Some(stddev)) = (s.mean, s.q2_median, s.stddev) else {
            return None;
        };
        (stddev > 0.0).then(|| 3.0 * (mean - median) / stddev)
    })
}

/// Whether a column is pronouncedly RIGHT-skewed (`pearson_skewness_stat >= RIGHT_SKEW_MIN`): a
/// dense low bulk under a long high tail. Backs the box panel's "right-skewed" title cue and its
/// log value axis (`box_log_skew_fallback`). A column with no derivable skew (no moarstats value
/// AND an incomplete base cache) reads as NOT skewed, keeping callers on their prior default.
fn is_right_skewed(s: &crate::cmd::stats::StatsData) -> bool {
    pearson_skewness_stat(s).is_some_and(|sk| sk >= RIGHT_SKEW_MIN)
}

/// Whether a column's MEAN is set by its tail rather than describing a typical row — the test
/// behind `viz smart`'s robust-statistics choices (issue #4220): summing instead of averaging a
/// group (`measure_by_dim_panel`) and correlating on ranks instead of values
/// (`corr_prefers_spearman`).
///
/// Deliberately NOT `is_right_skewed`. Pearson's second skewness coefficient
/// (`3 * (mean - median) / stddev`) SATURATES on exactly the distributions this targets: an
/// extreme tail inflates `stddev` faster than it inflates `mean - median`, so the ratio stays
/// small however lopsided the column gets. Measured on issue #4223/#4220's NYC capital-projects
/// repro, `totalplannedcommit` scores 0.38 — under the moderate-skew threshold — while its mean
/// ($15.2M) is **41x** its median ($367K) and its Gini is 0.96. A coefficient that calls that
/// column un-skewed cannot gate a decision about whether its mean is meaningful.
///
/// The mean/median ratio is read directly off the always-present base stats cache and says the
/// thing that actually matters to a reader: how far the "average" sits from the middle row.
///
/// Two guards keep it to the additive, positive-tail case it is written for:
///   - a column with more than `MEAN_OUTLIER_MAX_NEG_SHARE` NEGATIVES is excluded. The ratio is
///     meaningless when values straddle zero, and a symmetric signed column (profit/loss, an
///     anomaly series) has a median near 0 that the zero-inflation branch below would otherwise
///     misread as tail-dominated. The share is a small tolerance rather than zero because real
///     additive money columns carry a few reversals — the repro's `totalplannedcommit` holds 179
///     negative rollbacks in 12,587 rows (1.4%) and is emphatically still an additive amount. This
///     mirrors `box_log_skew_fallback`'s `BOX_LOG_NONPOS_MAX` tolerance.
///   - a non-positive median on such a column is tail-dominated BY DEFINITION: at least half the
///     rows are zero or below, so every bit of a positive mean comes from the minority that isn't —
///     the zero-inflated money column (45–60% zeros) the issue reports.
fn mean_is_outlier_driven(s: &crate::cmd::stats::StatsData) -> bool {
    /// at most this share of a column's values may be negative
    const MEAN_OUTLIER_MAX_NEG_SHARE: f64 = 0.05;

    let (Some(mean), Some(median)) = (s.mean, s.q2_median) else {
        return false;
    };
    if !mean.is_finite() || mean <= 0.0 {
        return false;
    }
    let (Some(neg), Some(zero), Some(pos)) = (s.n_negative, s.n_zero, s.n_positive) else {
        return false;
    };
    let total = neg + zero + pos;
    if total == 0 {
        return false;
    }
    #[allow(clippy::cast_precision_loss)]
    let neg_share = neg as f64 / total as f64;
    if neg_share > MEAN_OUTLIER_MAX_NEG_SHARE {
        return false;
    }
    if median <= 0.0 {
        return true;
    }
    mean >= median * MEAN_MEDIAN_RATIO_MIN
}

/// Zero share at or above which zeros stop being incidental and start reshaping how the
/// distribution must be read. Shared by `box_shape_hint`'s "% zeros" title part and the Lorenz
/// panel's flat-run caveat (`lorenz_caveat`) so the two can never disagree about whether the same
/// column is zero-inflated — they are routinely shown on the same Data Schematic.
const ZERO_SHARE_MIN: f64 = 0.30;

/// Share of zero values among a numeric column's non-null values, from the streaming sign
/// counts (`n_negative`/`n_zero`/`n_positive`) in the base stats cache. `None` for non-numeric
/// columns (the counts are only computed for Integer/Float) or when there are no numeric values.
fn zero_share(s: &crate::cmd::stats::StatsData) -> Option<f64> {
    let (Some(neg), Some(zero), Some(pos)) = (s.n_negative, s.n_zero, s.n_positive) else {
        return None;
    };
    let total = neg + zero + pos;
    #[allow(clippy::cast_precision_loss)]
    (total > 0).then(|| zero as f64 / total as f64)
}

/// Build a short parenthetical title hint for a box/histogram panel from cached statistics.
/// Parts are gathered in priority order — data-quality first (nulls and zeros are invisible in
/// a box and change how its quartiles should be read), then distribution shape — and capped at
/// `MAX_PARTS` so titles stay readable:
///
/// 1. null share (`sparsity`, base cache)
/// 2. zero share (streaming sign counts, base cache)
/// 3. skew direction (moarstats `pearson_skewness`, or derived from the base cache — see
///    `pearson_skewness_stat` — so plain `viz smart` gets it too)
/// 4. outlier share (moarstats scan), with a `mad_stddev_ratio` "heavy tails" fallback when the
///    outlier scan is absent
/// 5. outlier impact on the mean (moarstats `outlier_impact_ratio`)
/// 6. concentration (moarstats `gini_coefficient`)
///
/// Returns None when nothing notable — the title is left unchanged.
fn box_shape_hint(s: &crate::cmd::stats::StatsData) -> Option<String> {
    // nulls below ~a third of the column read as ordinary; above, they reshape the box. Zeros
    // use the shared `ZERO_SHARE_MIN`, which the Lorenz flat-run caveat reads too.
    const NULL_SHARE_MIN: f64 = 0.30;
    // |Pearson skewness| below this reads as ~symmetric; outlier shares below 1% are negligible.
    const SKEW_MIN_ABS: f64 = 0.5;
    const OUTLIER_MIN_PCT: f64 = 1.0;
    // outliers shifting the mean by less than 5% aren't worth a title callout.
    const MEAN_IMPACT_MIN_ABS: f64 = 0.05;
    const GINI_MIN: f64 = 0.6;
    // MAD/stddev is ~0.6745 for a normal distribution; well below it means stddev is inflated
    // by heavy tails. Only consulted when the moarstats outlier scan (`outliers_percentage`)
    // is absent, so it never duplicates the outlier-share part.
    const MAD_STDDEV_HEAVY_TAILS_MAX: f64 = 0.4;
    const MAX_PARTS: usize = 2;

    // The parts form a comma-separated LIST of noun phrases, not a sentence, so each is its own
    // catalog string and the ", " joiner stays literal.
    let mut parts: Vec<String> = Vec::new();
    if let Some(sp) = s.sparsity
        && (NULL_SHARE_MIN..=1.0).contains(&sp)
    {
        parts.push(t!("viz.notes.box_null", q_pct = format!("{:.0}", sp * 100.0)).into_owned());
    }
    if let Some(z) = zero_share(s)
        && z >= ZERO_SHARE_MIN
    {
        parts.push(t!("viz.notes.box_zeros", q_pct = format!("{:.0}", z * 100.0)).into_owned());
    }
    if let Some(skew) = pearson_skewness_stat(s)
        && skew.abs() >= SKEW_MIN_ABS
    {
        parts.push(
            if skew > 0.0 {
                t!("viz.notes.box_right_skewed")
            } else {
                t!("viz.notes.box_left_skewed")
            }
            .into_owned(),
        );
    }
    match s.outliers_percentage {
        Some(pct) if pct >= OUTLIER_MIN_PCT => {
            parts.push(t!("viz.notes.box_outliers", q_pct = format!("{pct:.1}")).into_owned());
        },
        Some(_) => {},
        None => {
            if let Some(r) = s.mad_stddev_ratio
                && r > 0.0
                && r <= MAD_STDDEV_HEAVY_TAILS_MAX
            {
                parts.push(t!("viz.notes.box_heavy_tails").into_owned());
            }
        },
    }
    if let Some(r) = s.outlier_impact_ratio
        && r.is_finite()
        && r.abs() >= MEAN_IMPACT_MIN_ABS
    {
        parts.push(
            t!(
                "viz.notes.box_mean_impact",
                q_pct = format!("{:+.0}", r * 100.0)
            )
            .into_owned(),
        );
    }
    if let Some(g) = s.gini_coefficient
        && g >= GINI_MIN
    {
        parts.push(t!("viz.notes.box_gini", q_gini = format!("{g:.2}")).into_owned());
    }
    parts.truncate(MAX_PARTS);
    if parts.is_empty() {
        None
    } else {
        Some(format!("({})", parts.join(", ")))
    }
}

/// Minimum Gini coefficient for an additive measure to earn a dedicated Lorenz-curve panel.
/// 0.5 matches `scoresql`'s "high inequality" line and comfortably catches the flagship cases
/// (US income ≈ 0.45–0.5, wealth 0.6–0.8). Safe at 0.5 — lower than `box_shape_hint`'s cosmetic
/// `GINI_MIN` of 0.6 — because a candidate must ALSO be an additive amount, so a merely skewed
/// intensive column can't trip it; a dedicated inequality panel that skipped income would miss
/// the point.
const LORENZ_GINI_MIN: f64 = 0.5;

/// Cap on how many Lorenz-curve panels a single smart Data Schematic adds (strongest Gini first),
/// so a wide finance table can't spawn a full-width overview panel — and a data pass — per column.
const LORENZ_MAX_PANELS: usize = 3;

/// Vertex cap for a rendered Lorenz curve. The cumulative curve is computed EXACTLY over every
/// non-null value (so its shape matches the cached Gini); only the finished, monotone curve is
/// then thinned (via `downsample_pair`, endpoints kept) to bound HTML size — thinning vertices of
/// an already-computed convex curve leaves its area, hence its visual Gini, essentially unchanged.
const LORENZ_CURVE_MAX_POINTS: usize = 2000;

/// Whether a column is a good candidate for an inequality (Lorenz-curve) panel: an ADDITIVE
/// measure — revenue/population/income/wealth, NOT an intensive rate/ratio/index/temperature that
/// can't meaningfully be "distributed" — whose cached Gini coefficient clears `LORENZ_GINI_MIN`.
///
/// The Gini value is produced only by `--smarter` (moarstats `--advanced`), so without it this is
/// always false and plain `viz smart` is unchanged. `gini_coefficient.is_some()` already implies a
/// non-negative column (the `qsv-stats` `gini` returns `None` for any negative value — Gini is
/// undefined there), exactly the domain a Lorenz curve needs. The additive test uses the
/// dictionary's aggregation verdict when present (`Agg::Sum` = additive amount, `Agg::Mean` =
/// intensive ratio) and falls back to the field/label name heuristic (`is_intensive_measure`)
/// when no dictionary routed the column.
fn is_inequality_candidate(sem: &ColSemantics, s: &crate::cmd::stats::StatsData) -> bool {
    let additive = match sem.agg {
        // dictionary-tagged additive amount/count (revenue, population, …)
        Some(Agg::Sum) => true,
        // dictionary-tagged intensive/other aggregation (ratio/rate/index) → not distributable
        Some(Agg::Mean | Agg::Count | Agg::Min | Agg::Max) => false,
        // no dictionary signal: fall back to the field/label name test. Exclude intensive
        // (rate/ratio/index) names AND identifier/code/key names — without a dictionary, a
        // skewed near-unique numeric ID/code would otherwise clear the Gini gate and get a
        // prominent Lorenz overview (`classify` skips such near-unique integers as unchartable,
        // but the Lorenz filter runs independently of `classify`).
        None => {
            !is_intensive_measure(&sem.label, &s.field) && !is_identifier_name(&sem.label, &s.field)
        },
    };
    additive && s.gini_coefficient.is_some_and(|g| g >= LORENZ_GINI_MIN)
}

fn lorenz_caveat(s: &crate::cmd::stats::StatsData) -> String {
    // The zero-inflation variant takes the unit clause as a SLOT rather than repeating it, so a
    // translation keeps one copy of the sentence and still controls where it sits.
    let unit_caveat = t!("viz.notes.lorenz_unit");

    zero_share(s).filter(|z| *z >= ZERO_SHARE_MIN).map_or_else(
        || unit_caveat.to_string(),
        |z| {
            t!(
                "viz.notes.lorenz_zeros",
                q_pct = format!("{:.0}", z * 100.0),
                q_unit = unit_caveat.to_string()
            )
            .into_owned()
        },
    )
}

/// Declared stage names that denote a COMPLEMENT or residual rather than a stage the process
/// passed through.
///
/// This is the one hazard that disclosing containment cannot replace, which is why the list
/// outlived the name-matching vocabulary it was born in. `unspent_balance` nests PERFECTLY inside
/// `planned` — it reports 0% violations — while rendering a funnel that means the opposite of the
/// truth. Every other bad declaration shows up as a violation share the subtitle can name; this
/// one does not.
///
/// It WARNS, it does not veto. An explicit declaration outranks a name heuristic, and the
/// collision half of the old list (`review`, `contractor`, `wholesale`, `lead_time`, …) was
/// deleted with the vocabulary: those existed solely as antidotes to greedy substring matching,
/// so warning on a declared `contractor_amt` would be a pure false alarm.
const FUNNEL_COMPLEMENT_MARKERS: &[&str] = &[
    "unspent",
    "uncommitted",
    "unplanned",
    "unpaid",
    "unobligated",
    "undisbursed",
    "unshipped",
    "remaining",
    "variance",
    "shortfall",
    "overrun",
    "difference",
    "delta",
    "balance",
];

/// Containment-violation share above which the subtitle names it. Below this it is float noise or
/// a handful of corrections, not a finding.
const FUNNEL_VIOLATION_NOTE_MIN: f64 = 0.01;

/// Minimum share of the table's rows that must survive the listwise-complete join for a
/// COLUMN-shaped funnel's totals to be worth showing at all. Deliberately not applied to the row
/// encoding — see `build_row_funnel_panel`.
const FUNNEL_MIN_COMPLETE_FRAC: f64 = 0.50;

/// Cap on funnel stages, so a wide finance table can't produce an unreadable twelve-band funnel.
const FUNNEL_MAX_STAGES: usize = 6;

/// Whether a declared stage name reads as a complement/residual (see `FUNNEL_COMPLEMENT_MARKERS`).
fn is_complement_name(label: &str, field: &str) -> bool {
    let hay = format!("{label} {field}").to_ascii_lowercase();
    FUNNEL_COMPLEMENT_MARKERS.iter().any(|m| hay.contains(m))
}

/// Share of row-aligned pairs satisfying `downstream <= upstream`. The tolerance is relative, so a
/// float-noise equality on large money values doesn't read as a violation. Returns 0.0 for empty
/// input (an empty pipeline is not a contained one).
fn containment_fraction(upstream: &[f64], downstream: &[f64]) -> f64 {
    let n = upstream.len().min(downstream.len());
    if n == 0 {
        return 0.0;
    }
    let ok = (0..n)
        .filter(|&r| {
            let (u, d) = (upstream[r], downstream[r]);
            d <= u + 1e-9 * u.abs().max(1.0)
        })
        .count();
    #[allow(clippy::cast_precision_loss)]
    let frac = ok as f64 / n as f64;
    frac
}

/// Sum a value column (or count rows) per declared stage value, in one pass.
///
/// Returns `(totals, counts, matched)` parallel to `wanted`, plus the number of rows that landed
/// in ANY declared stage — the row encoding's denominator.
///
/// Stage matching is trimmed and case-insensitive, with the first declared stage winning a
/// collision. The LLM transcribes these values from the frequency distribution, so case drift is
/// a realistic failure that would otherwise silently zero a band. A declared stage with no
/// matching rows is legitimate and stays as a zero-height band; dropping it would imply the
/// process ends at the last stage that happens to have data.
fn read_stage_totals(
    args: &Args,
    stage_idx: usize,
    value_idx: Option<usize>,
    wanted: &[String],
) -> CliResult<(Vec<f64>, Vec<usize>, usize)> {
    let lookup: HashMap<String, usize> = wanted
        .iter()
        .enumerate()
        .rev() // reverse so the FIRST declared stage wins a case-insensitive collision
        .map(|(k, s)| (s.trim().to_ascii_lowercase(), k))
        .collect();

    let (mut rdr, _headers, _nh) = reader_and_headers(args)?;
    let mut totals = vec![0.0_f64; wanted.len()];
    let mut counts = vec![0_usize; wanted.len()];
    let mut matched = 0_usize;
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let raw = cell_to_string(record.get(stage_idx));
        let key = raw.trim().to_ascii_lowercase();
        let Some(&k) = lookup.get(&key) else {
            continue;
        };
        let inc = match value_idx {
            Some(i) => match parse_f64(record.get(i)) {
                Some(v) => v,
                // a row whose measure doesn't parse contributes to neither total nor count,
                // so the disclosed denominator stays honest
                None => continue,
            },
            None => 1.0,
        };
        totals[k] += inc;
        counts[k] += 1;
        matched += 1;
    }
    Ok((totals, counts, matched))
}

/// Render a completeness share for the funnel subtitle, never rounding UP to 100%.
///
/// The clause this feeds exists specifically to warn that the funnel's totals will not reconcile
/// with `stats.sum`. Plain `{:.0}` defeats that: a dataset dropping 21 of 12,587 rows is 99.83%
/// complete, which renders as "100% of rows" — asserting the opposite of the disclosure's whole
/// purpose, while a nine-figure sum quietly goes missing. So anything short of genuinely complete
/// is FLOORED to one decimal; only an exact 1.0 prints a bare "100".
fn complete_pct_str(complete_frac: f64) -> String {
    if complete_frac >= 1.0 {
        return "100".to_string();
    }
    // floor, not round: 99.96% must not become "100.0%"
    let tenths = (complete_frac * 1000.0).floor() / 10.0;
    format!("{tenths:.1}")
}

/// The caveat line beneath a funnel panel's title (issue #4222).
///
/// Clause 1 is UNCONDITIONAL, following `lorenz_caveat`'s philosophy. The funnel's totals are
/// summed over the listwise-complete join `read_numeric_columns` produces — rows where ANY kept
/// numeric column was blank are absent — so they will NOT match `stats.sum`, and a reader
/// reconciling the two deserves to be told why without having to notice a discrepancy first.
///
/// The remaining clauses fire only when they apply, and both describe the same hazard from
/// different angles: a downstream stage that outruns its predecessor. Per-row violations mean
/// individual overruns; a larger downstream TOTAL means the overruns dominate the column. Capped
/// at 3 clauses like `lorenz_caveat`, so the line stays one readable row.
fn funnel_subtitle(
    stages: &[String],
    totals: &[f64],
    violations: &[f64],
    n_complete: usize,
    complete_frac: f64,
    form: &PipelineForm,
) -> Option<String> {
    // Each clause is a whole sentence with the stage names as slots, so word order belongs to the
    // translator; only the " · " joiner stays literal.
    let mut parts: Vec<String> = vec![
        t!(
            "viz.notes.funnel_complete",
            q_n = HumanCount(n_complete as u64).to_string(),
            q_pct = complete_pct_str(complete_frac)
        )
        .into_owned(),
    ];

    // worst per-row violation, if any is worth naming
    if let Some((k, v)) = violations
        .iter()
        .enumerate()
        .skip(1)
        .max_by(|a, b| a.1.total_cmp(b.1))
        .filter(|(_, v)| **v >= FUNNEL_VIOLATION_NOTE_MIN)
        && let (Some(here), Some(prev)) = (stages.get(k), stages.get(k - 1))
    {
        parts.push(
            t!(
                "viz.notes.funnel_violation",
                q_here = here.clone(),
                q_prev = prev.clone(),
                q_pct = format!("{:.0}", v * 100.0)
            )
            .into_owned(),
        );
    }

    // On a bridge, name the form choice: a reader who expected the declared "pipeline" to be a
    // funnel is owed the reason it is not one, in the same line that reports the violation.
    if matches!(form, PipelineForm::Bridge) {
        parts.push(t!("viz.notes.funnel_bridge").into_owned());
    }

    // a stage whose TOTAL outruns its predecessor: the bar order is kept (vocabulary order is
    // never sorted away), so the inverted band is the finding and needs naming.
    if let Some(k) = (1..totals.len()).find(|&k| totals[k] > totals[k - 1])
        && let (Some(here), Some(prev)) = (stages.get(k), stages.get(k - 1))
    {
        parts.push(
            t!(
                "viz.notes.funnel_inverted",
                q_here = here.clone(),
                q_prev = prev.clone()
            )
            .into_owned(),
        );
    }

    parts.truncate(3);
    Some(parts.join(" \u{b7} "))
}

/// APPROXIMATE share of the most frequent value, reconstructed entirely from the stats cache:
/// `mode_occurrences / record_count`, where the record count is recovered as
/// `cardinality / uniqueness_ratio` (its defining ratio) so no data pass is needed.
///
/// Heuristic-quality only — used solely as a `panel_interest` ranking penalty, never for a
/// user-facing claim: the mode tracker includes empty cells (a NULL-heavy column's "mode" is
/// the empty bucket — penalizing that as dominance is fine for ranking, since such a column is
/// equally uninformative), and the cached `uniqueness_ratio` is rounded, so the reconstructed
/// count drifts on large low-cardinality columns. The user-facing dominance TITLE hint uses
/// exact counts instead (see `dominant_category_hint`).
fn dominant_share_stat(s: &crate::cmd::stats::StatsData) -> Option<f64> {
    let occurrences = s.mode_occurrences?;
    let ur = s.uniqueness_ratio?;
    if ur <= 0.0 || !ur.is_finite() || s.cardinality == 0 {
        return None;
    }
    #[allow(clippy::cast_precision_loss)]
    let record_count = s.cardinality as f64 / ur;
    #[allow(clippy::cast_precision_loss)]
    let share = occurrences as f64 / record_count;
    share.is_finite().then_some(share)
}

/// Score how much a per-column panel is worth keeping when the Data Schematic overflows
/// `--max-charts`, from the column's cached statistics alone (no data pass). This is a
/// heuristic, not a science: the goal is that on a wide dataset the surviving panels are the
/// informative ones, not the leftmost ones.
///
/// - every panel starts at 1.0, discounted by its null share (a mostly-null column has little to
///   show whatever its chart);
/// - a histogram means the column was flagged bimodal — always notable;
/// - a box/histogram measure earns interest from relative spread (|cv|), outlier share (moarstats,
///   when present), and skew;
/// - a frequency bar earns interest from its category count (a 2-bar panel says less than a 12-bar
///   one) and loses it when a single category dominates the rows.
///
/// Overview panels (correlation, map, time-series, …) never come through here — they keep
/// `Panel::new`'s infinite default and always survive.
fn panel_interest(s: &crate::cmd::stats::StatsData, kind: &PanelKind) -> f64 {
    let mut score = 1.0_f64;
    if let Some(sp) = s.sparsity
        && sp.is_finite()
    {
        // keep a small floor so a sparse-but-remarkable column can still outrank a constant one
        score *= (1.0 - sp.clamp(0.0, 1.0)).max(0.05);
    }
    match kind {
        // flagged bimodal/multimodal — a shape worth seeing
        PanelKind::Histogram { .. } => score += 1.5,
        PanelKind::BoxStats { .. }
        | PanelKind::BoxRaw { .. }
        | PanelKind::BoxOutliers { .. }
        | PanelKind::Violin { .. } => {
            if let Some(cv) = s.cv
                && cv.is_finite()
            {
                // cv is a percentage; 100% relative spread buys 1 point, capped at 2
                score += (cv.abs() / 100.0).min(2.0);
            }
            if let Some(pct) = s.outliers_percentage
                && pct.is_finite()
            {
                score += (pct / 10.0).clamp(0.0, 1.0);
            }
            if let Some(skew) = pearson_skewness_stat(s)
                && skew.is_finite()
            {
                score += (skew.abs() / 2.0).min(1.0);
            }
        },
        PanelKind::FreqBar { .. } => {
            #[allow(clippy::cast_precision_loss)]
            let card = s.cardinality.min(CATEGORICAL_MAX_CARDINALITY) as f64;
            #[allow(clippy::cast_precision_loss)]
            {
                score += card / CATEGORICAL_MAX_CARDINALITY as f64;
            }
            if let Some(dom) = dominant_share_stat(s) {
                score -= dom.clamp(0.0, 1.0) * 0.75;
            }
        },
        _ => {},
    }
    score.max(0.0)
}

/// If `name` (already lower-cased) is a key/code twin of a sibling — ending in `_code`/`_id`/`_key`
/// — return the base name it twins (e.g. `subject_code` -> `subject`); otherwise None.
fn code_twin_base(name: &str) -> Option<&str> {
    for suffix in ["_code", "_id", "_key"] {
        if let Some(base) = name.strip_suffix(suffix)
            && !base.is_empty()
        {
            return Some(base);
        }
    }
    None
}

/// Identify "key twin" columns to suppress so a code/label pair (subject + `subject_code`, street +
/// `street_id`) charts only the human-readable member. A column is suppressed when it routes to a
/// frequency bar (`Dimension`) AND its name is `<base>_code`/`_id`/`_key` AND a sibling column
/// named `<base>` ALSO routes to a Dimension bar. Conservative on purpose: it only fires between
/// two charted dimensions, so a lone `*_code` (whose label isn't itself charted) is kept; and the
/// caller applies it only when a dictionary is present, so a stats-only Data Schematic is
/// unchanged.
fn dimension_code_twins(
    stats: &[crate::cmd::stats::StatsData],
    sems: &[ColSemantics],
) -> std::collections::HashSet<usize> {
    use std::collections::{HashMap, HashSet};

    let dim_names: HashMap<String, usize> = stats
        .iter()
        .enumerate()
        .filter(|(i, _)| sems.get(*i).is_some_and(|s| s.route == Route::Dimension))
        .map(|(i, s)| (s.field.to_lowercase(), i))
        .collect();

    let mut suppress = HashSet::new();
    for (lname, &i) in &dim_names {
        if let Some(base) = code_twin_base(lname)
            && dim_names.contains_key(base)
        {
            suppress.insert(i);
        }
    }
    suppress
}

/// Identify categorical columns that are **1:1** with another charted categorical column — each
/// value of one maps to exactly one value of the other and vice versa — and return the redundant
/// members to suppress, keeping one representative per group (issue #4221).
///
/// The reported case is a code/label pair like `magencyacro` ↔ `magencyname` (`DPR` ↔ `Department
/// of Parks & Recreation`). Charted separately they produce byte-identical frequency bars, and the
/// parcats overview spends two of its axes on the same variable with trivial straight-through
/// ribbons between them.
///
/// This is the DATA-DRIVEN counterpart of `dimension_code_twins`, which recognizes the same
/// redundancy from NAMES alone (`subject` + `subject_code`) and only with a dictionary present.
/// Neither `magencyacro` nor `magencyname` is the other's `<base>_code`, and the reporting
/// Data Schematic has no dictionary, so the name rule cannot see this pair. Both feed the same
/// suppression set, so every downstream consumer — the per-column panel loop, `dim_indices`, and
/// `eligible_categorical_dims` (hence parcats and the hierarchy panel) — inherits the verdict.
///
/// # Algorithm: partition refinement
///
/// Two columns are 1:1 exactly when they induce the SAME PARTITION of the rows — the value names
/// are irrelevant, only which rows group together. So each column canonicalizes its values to ids
/// by first appearance, and the candidate columns are refined into groups that still agree:
/// every row splits each group by its members' ids, and a group that survives every row is a set
/// of columns whose partitions never diverged. That is the definition of the property, checked
/// directly.
///
/// This replaced an earlier per-pair formulation that compared distinct-value counts against
/// distinct-pair counts. That version accumulated three separate counters whose null semantics had
/// to be kept in agreement by hand, and grew a correctness bug each time one of them was reasoned
/// about slightly differently (see the guards below, each of which was earned by a real failure).
/// Refinement removes the class: presence is part of a column's identity rather than a side
/// condition, so there is nothing left to keep in sync.
///
/// It is also cheaper. The per-pair form was O(k²) per row in a bucket of k same-cardinality
/// columns, allocating a set for every pair up front; this is O(k), allocates per column, and —
/// because groups only ever shrink — **stops reading as soon as every group is a singleton**. On
/// the overwhelmingly common input, where no two columns are 1:1, that is a handful of rows rather
/// than the whole file.
///
/// # Guards
///
/// **Free pre-filter.** Two columns can only be 1:1 if their cardinalities match, so groups are
/// seeded from the stats cache's `cardinality` and the data pass happens ONLY when at least two
/// eligible categoricals share one. The common case pays nothing.
///
/// **Strict.** "Near-1:1" is deliberately not detected: a tolerance risks suppressing a column
/// that carries real information, and a false positive here DELETES a panel.
///
/// **Empty is not a value, but it IS part of the pattern.** An empty cell canonicalizes to its own
/// reserved id, so two columns blank on DIFFERENT rows separate immediately — they are not the
/// same variable however well their populated values line up. Distinct values and support are
/// counted over populated rows only, so two columns that merely share a blank region cannot
/// collapse on the strength of their blanks.
///
/// **Support.** A bijection observed over a handful of rows is coincidence, not a functional
/// dependency: two unrelated flags (`is_active`, `has_paid`) can easily agree on every one of four
/// rows. A group must therefore average at least `MIN_ROWS_PER_CATEGORY` populated rows per
/// category. The guard is on SUPPORT, not on cardinality — a two-category pair is not inherently
/// suspect, and gating on cardinality would permanently blind this to the real binary code/label
/// pairs (`M`/`F` against `Male`/`Female`) that are squarely the redundancy being targeted.
fn one_to_one_categorical_twins(
    args: &Args,
    stats: &[crate::cmd::stats::StatsData],
    sems: &[ColSemantics],
) -> CliResult<std::collections::HashSet<usize>> {
    use std::collections::{HashMap, HashSet};

    /// Populated rows a category must average before a group counts as a functional dependency
    /// rather than a small-sample coincidence. A 4-row agreement tells you nothing.
    const MIN_ROWS_PER_CATEGORY: u64 = 4;
    /// Canonical id for an empty cell, distinct from every real value's id.
    const EMPTY_ID: u32 = u32::MAX;

    let mut suppress = HashSet::new();

    let eligible: Vec<usize> = stats
        .iter()
        .enumerate()
        .filter(|(i, s)| {
            sems.get(*i)
                .is_some_and(|sem| matches!(sem.route, Route::Defer | Route::Dimension))
                && s.r#type == "String"
                && s.cardinality >= 2
                && s.cardinality <= CATEGORICAL_MAX_CARDINALITY
        })
        .map(|(i, _)| i)
        .collect();

    // seed the groups from the free pre-filter: unequal cardinality rules out a bijection outright
    let mut by_card: HashMap<u64, Vec<usize>> = HashMap::new();
    for &i in &eligible {
        by_card.entry(stats[i].cardinality).or_default().push(i);
    }
    let mut groups: Vec<Vec<usize>> = by_card.into_values().filter(|v| v.len() >= 2).collect();
    if groups.is_empty() {
        return Ok(suppress);
    }
    // `into_values` yields buckets in HashMap order; sort so notes and the pass are reproducible
    for g in &mut groups {
        g.sort_unstable();
    }
    groups.sort();

    // per-column canonical value -> id, assigned in order of first appearance. Two columns agree
    // on a row when their ids match, whatever their values are named.
    let mut ids: HashMap<usize, HashMap<String, u32>> = HashMap::new();
    // populated (non-empty) row count per column — the support its share of a group rests on
    let mut present: HashMap<usize, u64> = HashMap::new();

    let (mut rdr, _headers, _nh) = reader_and_headers(args)?;
    let mut record = csv::ByteRecord::new();
    let mut row_id: HashMap<usize, u32> = HashMap::new();
    while rdr.read_byte_record(&mut record)? {
        // only columns still in a live group can matter: once two columns separate they can never
        // rejoin, so a column that has fallen out of every group is never looked at again
        row_id.clear();
        for &i in groups.iter().flatten() {
            let v = cell_to_string(record.get(i));
            let id = if v.trim().is_empty() {
                EMPTY_ID
            } else {
                *present.entry(i).or_default() += 1;
                let map = ids.entry(i).or_default();
                let next = map.len() as u32;
                *map.entry(v).or_insert(next)
            };
            row_id.insert(i, id);
        }
        // refine: split every group by its members' ids on this row, keeping only the sub-groups
        // that still have someone to disagree with
        let mut refined: Vec<Vec<usize>> = Vec::with_capacity(groups.len());
        for g in &groups {
            let mut by_id: HashMap<u32, Vec<usize>> = HashMap::new();
            for &i in g {
                by_id
                    .entry(row_id.get(&i).copied().unwrap_or(EMPTY_ID))
                    .or_default()
                    .push(i);
            }
            refined.extend(by_id.into_values().filter(|sub| sub.len() > 1));
        }
        groups = refined;
        if groups.is_empty() {
            // nothing can become 1:1 again — stop reading
            break;
        }
    }

    for g in &mut groups {
        g.sort_unstable();
    }
    groups.sort();

    // Keep the column with the SHORTEST values: every member carries the same information, so the
    // one that fits a bar label or a treemap tile wins ("DPR" over "Department of Parks &
    // Recreation", which truncates on an axis). Column index breaks a tie so the pick is stable.
    for g in groups {
        let distinct = |i: usize| ids.get(&i).map_or(0, HashMap::len);
        let populated = |i: usize| present.get(&i).copied().unwrap_or(0);
        // a single-value column is not a dimension anyone charts against, and its "bijection" is
        // vacuous; and the mapping must be corroborated enough times to be a dependency at all
        let n_distinct = g.iter().map(|&i| distinct(i)).max().unwrap_or(0);
        let min_populated = g.iter().map(|&i| populated(i)).min().unwrap_or(0);
        if n_distinct < 2 || min_populated < MIN_ROWS_PER_CATEGORY * n_distinct as u64 {
            continue;
        }
        let mean_len = |&i: &usize| {
            ids.get(&i).map_or(usize::MAX, |vs| {
                if vs.is_empty() {
                    usize::MAX
                } else {
                    vs.keys().map(String::len).sum::<usize>() / vs.len()
                }
            })
        };
        let Some(&keep) = g.iter().min_by_key(|i| (mean_len(i), **i)) else {
            continue;
        };
        let name = |i: usize| {
            let s = &stats[i];
            if s.field.is_empty() {
                format!("col {}", i + 1)
            } else {
                s.field.clone()
            }
        };
        let dropped: Vec<String> = g.iter().filter(|&&i| i != keep).map(|&i| name(i)).collect();
        for &i in &g {
            if i != keep {
                suppress.insert(i);
            }
        }
        viz_note(&format!(
            "viz smart: charting only {} for {} — they are 1:1 ({n_distinct} categories), so \
             separate panels and parcats axes would be duplicates",
            name(keep),
            dropped.join(", "),
        ));
    }

    Ok(suppress)
}

/// Canonical-timestamp priority for a date column's dictionary concept: a smaller rank wins as the
/// time-series x-axis. Event/created timestamps lead; closed/updated/due are secondary; a bare date
/// or no concept is last. Lets `viz smart` trend "requests over `created_date`" rather than over a
/// modified timestamp. Columns without a dictionary all rank 3, so selection falls back to the
/// first date column, exactly as before.
fn timestamp_rank(concept: &str) -> u8 {
    match concept {
        "time.event_timestamp" => 0,
        "time.created_at" => 1,
        "time.closed_at" | "time.updated_at" | "time.due_at" => 2,
        _ => 3,
    }
}

/// Sortedness tiebreak for the canonical-date choice: a date column the file is physically
/// ordered by (ascending or descending — e.g. an export ordered newest-first) is more likely
/// the primary event timestamp than an unsorted one (an edited/backfilled `modified_date`), so
/// among equal-concept-rank candidates it wins. Matched case-insensitively: stats currently
/// emits "Ascending"/"Descending"/"Unsorted" (`STATS_DEFINITIONS.md` documents the uppercase
/// forms). Heuristic caveat: stats' `sort_order` is evaluated on the raw values, so a
/// chronologically-sorted column in a non-lexicographic date format (e.g. `M/D/YYYY`) may read
/// unsorted — the tiebreak then simply doesn't fire and selection falls back to column order,
/// exactly as before.
fn sort_order_rank(s: &crate::cmd::stats::StatsData) -> u8 {
    match s.sort_order.as_deref() {
        Some(order)
            if order.eq_ignore_ascii_case("ascending")
                || order.eq_ignore_ascii_case("descending") =>
        {
            0
        },
        _ => 1,
    }
}

/// Time bucket granularity for the trend panel, widened as the date span grows so a multi-year
/// dataset doesn't render thousands of daily points. Variant order is fineness order (finest
/// first), so `Ord::max` picks the coarser of two buckets — the span default vs the cadence floor
/// (issue #4216); a unit test pins this ordering.
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Debug)]
enum TsBucket {
    Day,
    Week,
    Month,
    Quarter,
    Year,
}

/// Pick a bucket granularity from the span (in days) between the date column's observed min and
/// max. This is only the span-based ceiling on point count; the final bucket may be coarsened
/// further by the data's cadence (see `cadence_floor`).
fn ts_bucket_for_span(span_days: i64) -> TsBucket {
    if span_days <= 370 {
        TsBucket::Day
    } else if span_days <= 1825 {
        TsBucket::Week
    } else if span_days <= 7300 {
        TsBucket::Month
    } else if span_days <= 18250 {
        TsBucket::Quarter
    } else {
        TsBucket::Year
    }
}

/// Truncate a date to its bucket's representative day (the day itself, its ISO-week Monday, the
/// first of its month/quarter/year) so rows in the same period share one key.
fn ts_bucket_key(date: chrono::NaiveDate, bucket: TsBucket) -> chrono::NaiveDate {
    use chrono::{Datelike, Duration};
    match bucket {
        TsBucket::Day => date,
        TsBucket::Week => date - Duration::days(i64::from(date.weekday().num_days_from_monday())),
        TsBucket::Month => date.with_day(1).unwrap_or(date),
        TsBucket::Quarter => date
            .with_day(1)
            .and_then(|d| d.with_month(d.month0() / 3 * 3 + 1))
            .unwrap_or(date),
        TsBucket::Year => date
            .with_day(1)
            .and_then(|d| d.with_month(1))
            .unwrap_or(date),
    }
}

/// Display label for a bucket key: ISO date for Day/Week, `YYYY-MM` for Month, `YYYY-Qn` for
/// Quarter, `YYYY` for Year. Quarter/Year labels are NOT date-parseable, so those panels render
/// on a category axis (see `PanelKind::TimeSeries::categorical`).
fn ts_bucket_label(key: chrono::NaiveDate, bucket: TsBucket) -> String {
    use chrono::Datelike;
    match bucket {
        TsBucket::Day | TsBucket::Week => key.format("%Y-%m-%d").to_string(),
        TsBucket::Month => key.format("%Y-%m").to_string(),
        TsBucket::Quarter => format!("{}-Q{}", key.year(), key.month0() / 3 + 1),
        TsBucket::Year => key.format("%Y").to_string(),
    }
}

/// A short word for the bucket granularity, used in axis titles ("records per day").
fn ts_bucket_word(bucket: TsBucket) -> String {
    match bucket {
        TsBucket::Day => t!("viz.title.bucket_day"),
        TsBucket::Week => t!("viz.title.bucket_week"),
        TsBucket::Month => t!("viz.title.bucket_month"),
        TsBucket::Quarter => t!("viz.title.bucket_quarter"),
        TsBucket::Year => t!("viz.title.bucket_year"),
    }
    .into_owned()
}

/// Number of bucket periods between two dates, inclusive (both endpoints' periods count).
/// Exact integer arithmetic on calendar indices — no floating-point day averaging.
fn periods_in_span(first: chrono::NaiveDate, last: chrono::NaiveDate, bucket: TsBucket) -> i64 {
    use chrono::Datelike;
    let month_index = |d: chrono::NaiveDate| i64::from(d.year()) * 12 + i64::from(d.month0());
    match bucket {
        TsBucket::Day => (last - first).num_days() + 1,
        TsBucket::Week => {
            let a = ts_bucket_key(first, TsBucket::Week);
            let b = ts_bucket_key(last, TsBucket::Week);
            (b - a).num_days() / 7 + 1
        },
        TsBucket::Month => month_index(last) - month_index(first) + 1,
        TsBucket::Quarter => month_index(last) / 3 - month_index(first) / 3 + 1,
        TsBucket::Year => i64::from(last.year()) - i64::from(first.year()) + 1,
    }
}

/// Occupancy threshold (percent) for `cadence_floor`: the finest bucket whose periods are at
/// least this occupied wins. 45% lets genuinely daily/monthly data with holiday/weekend gaps
/// keep its fine bucket, while quarterly data (~12 of 156 weeks ≈ 8%) falls through to Quarter.
const CADENCE_MIN_OCCUPANCY_PCT: i64 = 45;

/// The finest bucket justified by the data's own cadence: the finest candidate whose periods
/// between the first and last observed day are at least `CADENCE_MIN_OCCUPANCY_PCT` occupied.
/// Span-based selection alone renders quarterly data as a comb of spikes separated by empty
/// weeks (issue #4216); occupancy equals mode-of-intervals for regular cadences while degrading
/// gracefully for gappy data. `day_keys` must be sorted and de-duplicated distinct days.
fn cadence_floor(day_keys: &[chrono::NaiveDate]) -> TsBucket {
    let (Some(&first), Some(&last)) = (day_keys.first(), day_keys.last()) else {
        return TsBucket::Day;
    };
    for bucket in [
        TsBucket::Day,
        TsBucket::Week,
        TsBucket::Month,
        TsBucket::Quarter,
        TsBucket::Year,
    ] {
        let mut distinct: Vec<chrono::NaiveDate> =
            day_keys.iter().map(|&d| ts_bucket_key(d, bucket)).collect();
        distinct.dedup(); // day_keys sorted -> bucket keys monotonic, dedup suffices
        let total = periods_in_span(first, last, bucket);
        if distinct.len() as i64 * 100 >= total * CADENCE_MIN_OCCUPANCY_PCT {
            return bucket;
        }
    }
    TsBucket::Year
}

/// Map a dictionary cadence token (describegpt's `x-qsv.cadence`, from its controlled
/// vocabulary) to the equivalent bucket floor. Unknown/absent tokens map to `None` — the
/// consumer-side counterpart of the vocab-membership discipline (PR #4321): never trust prose.
fn bucket_floor_from_cadence(token: &str) -> Option<TsBucket> {
    match token {
        "daily" => Some(TsBucket::Day),
        "weekly" => Some(TsBucket::Week),
        "monthly" => Some(TsBucket::Month),
        "quarterly" => Some(TsBucket::Quarter),
        "annual" => Some(TsBucket::Year),
        _ => None,
    }
}

/// The trend panel's bucket floor: the data's own scanned cadence, coarsened by the dictionary's
/// cadence token when one is present.
///
/// The token may only COARSEN the scan, never refine it. It is derived from aggregate stats
/// (min/max/cardinality), which cannot prove regular spacing — clustered dates can average into
/// a cadence band — whereas `cadence_floor` scans the actual observed days. Any bucket finer
/// than the scanned floor is by construction under `CADENCE_MIN_OCCUPANCY_PCT` occupied, i.e.
/// the comb of spikes issue #4216 removed, so a mistaken token must not be able to reintroduce
/// it. Coarsening is still the token's to decide: it encodes the publisher's declared reporting
/// period, which outranks occupancy when the two disagree in that direction.
fn trend_bucket_floor(cadence: Option<&str>, day_keys: &[chrono::NaiveDate]) -> TsBucket {
    let scan_floor = cadence_floor(day_keys);
    cadence
        .and_then(bucket_floor_from_cadence)
        .map_or(scan_floor, |token_floor| token_floor.max(scan_floor))
}

/// Character cap for a structured count unit (`x-qsv.grain_unit`) before it is rejected as a
/// runaway. 40 matches the intent of the legacy grain path's 40-BYTE cap for ASCII, while leaving
/// a CJK unit — which would blow a 40-byte budget at ~13 characters — comfortable room.
const COUNT_UNIT_MAX_CHARS: usize = 40;

/// Best-effort entity name for a count-over-time panel.
///
/// Precedence, most to least trustworthy:
///
/// 1. `grain_unit` — describegpt's STRUCTURED count unit (`x-qsv.grain_unit`), the bare entity noun
///    with no sentence around it. Under `--language xx` the model is asked to write it in the
///    target language, so it drops straight into a localized title template.
/// 2. `grain` — the prose sentence, split on the English literal `" one "` ("one row = one 311
///    service request" -> "311 service request"). LEGACY: this is the only path for dictionaries
///    generated before `grain_unit` existed. It is deliberately left exactly as it was, English
///    assumption included — a pre-`grain_unit` dictionary keeps today's behavior verbatim rather
///    than silently changing, and the real fix for such a dictionary is to regenerate it (issue
///    #4321).
/// 3. the localized `viz.chart.records` fallback, so the label is always sensible.
fn count_unit_from_grain(grain_unit: Option<&str>, grain: Option<&str>) -> String {
    let fallback = || t!("viz.chart.records").into_owned();

    // The structured field wins outright when present and plausible.
    if let Some(unit) = grain_unit.map(str::trim).filter(|u| !u.is_empty())
        && plausible_count_unit(unit)
    {
        return unit.to_string();
    }

    let Some(g) = grain else {
        return fallback();
    };
    // describegpt phrases grain as "one row = one <X>" / "each row is one <X>"; take the text after
    // the last " one ".
    let entity = g
        .rsplit_once(" one ")
        .map(|(_, tail)| tail)
        .unwrap_or("")
        .trim()
        .trim_end_matches('.')
        .trim();
    if entity.is_empty() || entity.len() > 40 {
        fallback()
    } else {
        entity.to_string()
    }
}

/// Runaway guard for a count unit: it has to fit in a chart title.
///
/// Counted in CHARACTERS, not bytes. The legacy grain path caps at 40 *bytes*, which is a
/// reasonable proxy only for ASCII — a perfectly ordinary Japanese or Chinese unit
/// ("ステーションごとの日次観測記録") exhausts 40 bytes in ~14 characters, so a byte cap would
/// reject exactly the localized units this field exists to carry.
fn plausible_count_unit(unit: &str) -> bool {
    unit.chars().count() <= COUNT_UNIT_MAX_CHARS
}

/// Parse a row's date cell into a `DateTime<Utc>` under the caller's DMY preference — the
/// column's dictionary-declared date format when it expresses one, otherwise `QSV_PREFER_DMY`
/// (see `SmartCtx::dmy_pref`, issue #4303). That can differ from the preference `stats` typed the
/// column under, which is deliberate: an explicit per-column declaration outranks the global env
/// flag. Returns `None` for a missing/blank/unparseable cell.
fn parse_record_date(
    record: &csv::ByteRecord,
    idx: usize,
    prefer_dmy: bool,
) -> Option<chrono::DateTime<chrono::Utc>> {
    let text = std::str::from_utf8(record.get(idx)?).ok()?.trim();
    if text.is_empty() {
        return None;
    }
    qsv_dateparser::parse_with_preference(text, prefer_dmy).ok()
}

/// Pick the canonical date/datetime column for temporal panels (time-series overview, animated
/// scatter). When a dictionary tagged timestamps, prefer the event/created one (`timestamp_rank`);
/// among equal ranks prefer a column the file is physically sorted by (`sort_order_rank` — likely
/// the primary event time); remaining ties break on column order. `stats` emits "Date"/"DateTime"
/// once dates are inferred. Returns the column index and whether it carries a time-of-day
/// component.
fn canonical_date_col(
    stats: &[crate::cmd::stats::StatsData],
    sems: &[ColSemantics],
) -> Option<(usize, bool)> {
    stats
        .iter()
        .enumerate()
        .filter_map(|(i, s)| match s.r#type.as_str() {
            "Date" => Some((i, false)),
            "DateTime" => Some((i, true)),
            _ => None,
        })
        .min_by_key(|&(i, _)| {
            (
                timestamp_rank(sems.get(i).map_or("", |s| s.concept.as_str())),
                sort_order_rank(&stats[i]),
                i,
            )
        })
}

fn build_timeseries_panel(
    args: &Args,
    stats: &[crate::cmd::stats::StatsData],
    dmy_prefs: &[bool],
    map_cols: Option<(usize, usize)>,
    sems: &[ColSemantics],
    grain_unit: Option<&str>,
    grain: Option<&str>,
    cadence: Option<&str>,
    dict_icons: Option<&DictData>,
) -> CliResult<Option<Panel>> {
    use std::collections::BTreeMap;

    let Some((date_idx, is_datetime)) = canonical_date_col(stats, sems) else {
        return Ok(None);
    };
    // the date column is picked here, so resolve its parsing preference here too (issue #4303).
    let prefer_dmy = dmy_prefs.get(date_idx).copied().unwrap_or(false);
    // --dict-info: the overview trend anchors its info icon on the date column's entry (the
    // panel is derived from it, and timestamp entries typically explain timezone/pairing).
    let dict_info = dict_info_for_field(dict_icons, &stats[date_idx].field);

    // y-axis candidate: the first continuous numeric column (preferring Float), excluding map
    // coordinates and any dictionary-tagged non-measure. Near-unique columns are deliberately
    // allowed — a measurement like revenue is often near-unique yet makes the most meaningful
    // trend. A confirmed Measure or an un-tagged (Defer) column qualifies; without a dictionary
    // every column is Defer, exactly as before.
    let is_map_col = |idx: usize| map_cols.is_some_and(|(la, lo)| idx == la || idx == lo);
    let continuous_numeric = |s: &crate::cmd::stats::StatsData| {
        if !matches!(s.r#type.as_str(), "Integer" | "Float") || s.cardinality <= 1 {
            return false;
        }
        let near_unique = s.uniqueness_ratio.is_some_and(|r| r > 0.95);
        let low_cardinality = s.cardinality <= CATEGORICAL_MAX_CARDINALITY && !near_unique;
        !low_cardinality
    };
    let y_eligible = |i: usize| {
        sems.get(i)
            .is_none_or(|s| matches!(s.route, Route::Defer | Route::Measure))
    };
    let y_idx = stats
        .iter()
        .enumerate()
        .filter(|(i, s)| {
            *i != date_idx && !is_map_col(*i) && y_eligible(*i) && continuous_numeric(s)
        })
        .min_by_key(|(_, s)| usize::from(s.r#type != "Float"))
        .map(|(i, _)| i);

    // Aggregation mode:
    //   * a dictionary `measure` (route == Measure) carries an additive/mean `agg`: bucket the
    //     value by calendar period and sum (counts/amounts) or average (ratios) it.
    //   * an un-tagged (Defer) numeric: plot the raw values over time (today's behavior, LTTB).
    //   * no eligible numeric at all: count records per period — the "volume over time" overview,
    //     the single most valuable view for event datasets (e.g. 311 requests per day).
    enum Mode {
        Raw(usize),
        AggValue(usize, Agg),
        Count,
    }
    let mode = match y_idx {
        Some(i) => match sems.get(i).and_then(|s| s.agg) {
            Some(agg) => Mode::AggValue(i, agg),
            None => Mode::Raw(i),
        },
        None => Mode::Count,
    };

    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    // prefer the dictionary's human label for axis/panel titles (like the per-column panels),
    // falling back to the raw header.
    let label_for = |idx: usize| {
        sems.get(idx)
            .map(|s| s.label.as_str())
            .filter(|l| !l.is_empty())
            .map_or_else(|| col_label(&headers, idx, nh), ToString::to_string)
    };
    let date_label = label_for(date_idx);
    let mut record = csv::ByteRecord::new();

    // The raw path plots individual points (chronologically sorted) over time, exactly as before.
    if let Mode::Raw(y_idx) = mode {
        let y_label = label_for(y_idx);
        let mut points: Vec<(i64, String, f64)> = Vec::new();
        while rdr.read_byte_record(&mut record)? {
            // skip non-finite y (NaN/inf): parse_f64 accepts "NaN"/"inf", but a single non-finite
            // value would poison LTTB's bucket averages and area comparisons (and render as a gap)
            let Some(y) = parse_f64(record.get(y_idx)).filter(|v| v.is_finite()) else {
                continue;
            };
            let Some(dt) = parse_record_date(&record, date_idx, prefer_dmy) else {
                continue;
            };
            let label = if is_datetime {
                dt.format("%Y-%m-%dT%H:%M:%S").to_string()
            } else {
                dt.format("%Y-%m-%d").to_string()
            };
            points.push((dt.timestamp_millis(), label, y));
        }
        // a line needs at least two points
        if points.len() < 2 {
            return Ok(None);
        }
        points.sort_by_key(|p| p.0);
        // points are chronologically sorted (monotonic x), so LTTB can downsample by triangle area,
        // preserving spikes/peaks a uniform stride would step over.
        let (xs, ys) = if points.len() > *MAX_SMART_POINTS {
            let ts: Vec<f64> = points.iter().map(|p| p.0 as f64).collect();
            let yv: Vec<f64> = points.iter().map(|p| p.2).collect();
            let keep = lttb_indices(&ts, &yv, *MAX_SMART_POINTS);
            let xs = keep.iter().map(|&i| points[i].1.clone()).collect();
            let ys = keep.iter().map(|&i| points[i].2).collect();
            (xs, ys)
        } else {
            (
                points.iter().map(|p| p.1.clone()).collect(),
                points.iter().map(|p| p.2).collect(),
            )
        };
        return Ok(Some(
            Panel::new(
                t!("viz.title.trend_raw", q_y = y_label, q_date = date_label).into_owned(),
                PanelKind::TimeSeries {
                    y_label,
                    xs,
                    ys,
                    categorical: false,
                },
            )
            .with_dict_info(dict_info),
        ));
    }

    // Bucketed paths (AggValue / Count): group rows by calendar period. Rows are first folded
    // into distinct days (memory O(distinct days), never O(rows)); the final bucket is then
    // chosen from the observed days themselves — the span-based ceiling coarsened to the data's
    // cadence floor — and the day map is re-folded. (sum, n) folds losslessly for Sum/Mean/Count,
    // so the two-stage fold changes no aggregate. Span-based selection alone rendered quarterly
    // data as a comb of spikes separated by empty weeks (issue #4216).
    let value_idx = if let Mode::AggValue(i, _) = mode {
        Some(i)
    } else {
        None
    };
    let mut days: BTreeMap<chrono::NaiveDate, (f64, u64)> = BTreeMap::new();
    while rdr.read_byte_record(&mut record)? {
        let Some(dt) = parse_record_date(&record, date_idx, prefer_dmy) else {
            continue;
        };
        let y = match value_idx {
            Some(i) => match parse_f64(record.get(i)).filter(|v| v.is_finite()) {
                Some(v) => v,
                // a row whose value is missing/non-finite contributes to neither sum nor count
                None => continue,
            },
            None => 0.0,
        };
        let entry = days.entry(dt.date_naive()).or_insert((0.0, 0));
        entry.0 += y;
        entry.1 += 1;
    }
    let day_keys: Vec<chrono::NaiveDate> = days.keys().copied().collect();
    let bucket = {
        let span_bucket = match (day_keys.first(), day_keys.last()) {
            (Some(&lo), Some(&hi)) => ts_bucket_for_span((hi - lo).num_days().max(0)),
            _ => TsBucket::Day,
        };
        // the span-based ceiling still applies on top of the cadence floor, so a 40-year daily
        // dataset doesn't render 14k points
        span_bucket.max(trend_bucket_floor(cadence, &day_keys))
    };
    let mut buckets: BTreeMap<chrono::NaiveDate, (f64, u64)> = BTreeMap::new();
    for (&day, &(sum, n)) in &days {
        let entry = buckets
            .entry(ts_bucket_key(day, bucket))
            .or_insert((0.0, 0));
        entry.0 += sum;
        entry.1 += n;
    }
    // a line needs at least two periods (evaluated on the FINAL fold, not the day map)
    if buckets.len() < 2 {
        return Ok(None);
    }
    let word = ts_bucket_word(bucket);
    let xs: Vec<String> = buckets
        .keys()
        .map(|k| ts_bucket_label(*k, bucket))
        .collect();

    match mode {
        Mode::AggValue(y_idx, agg) => {
            let y_label = label_for(y_idx);
            let (agg_word, ys): (String, Vec<f64>) = if agg == Agg::Mean {
                (
                    t!("viz.title.agg_mean").into_owned(),
                    buckets
                        .values()
                        .map(|&(s, n)| if n > 0 { s / n as f64 } else { 0.0 })
                        .collect(),
                )
            } else {
                // counts/amounts are additive -> sum per period
                (
                    t!("viz.title.agg_sum").into_owned(),
                    buckets.values().map(|&(s, _)| s).collect(),
                )
            };
            Ok(Some(
                Panel::new(
                    t!(
                        "viz.title.trend_agg",
                        q_y = y_label,
                        q_agg = agg_word,
                        q_date = date_label
                    )
                    .into_owned(),
                    PanelKind::TimeSeries {
                        y_label: t!(
                            "viz.title.trend_axis_agg",
                            q_y = y_label,
                            q_agg = agg_word,
                            q_bucket = word
                        )
                        .into_owned(),
                        xs,
                        ys,
                        categorical: matches!(bucket, TsBucket::Quarter | TsBucket::Year),
                    },
                )
                .with_dict_info(dict_info),
            ))
        },
        // Count: one point per period = number of records (rows with a parseable date).
        _ => {
            let unit = count_unit_from_grain(grain_unit, grain);
            let ys: Vec<f64> = buckets.values().map(|&(_, n)| n as f64).collect();
            Ok(Some(
                Panel::new(
                    t!("viz.title.trend_count", q_unit = unit, q_date = date_label).into_owned(),
                    PanelKind::TimeSeries {
                        y_label: t!("viz.title.trend_axis_count", q_unit = unit, q_bucket = word)
                            .into_owned(),
                        xs,
                        ys,
                        categorical: matches!(bucket, TsBucket::Quarter | TsBucket::Year),
                    },
                )
                .with_dict_info(dict_info),
            ))
        },
    }
}

/// Read one additive-measure column in full, compute its EXACT Lorenz curve (via `lorenz_curve`),
/// and build a `Lorenz` overview panel labeled with the CACHED Gini (`gini`, never recomputed — so
/// the picture and the number agree with the box hint / pivotp / scoresql). Only non-negative
/// finite values are kept, matching the domain moarstats' Gini was computed over. Returns `None`
/// for a degenerate column (see `lorenz_curve`).
///
/// `stat` is the column's cached stats row, used ONLY to compose the panel's framing caveat
/// (`lorenz_caveat`) from the already-cached zero share — no statistic is recomputed here.
fn build_lorenz_panel(
    args: &Args,
    sems: &[ColSemantics],
    idx: usize,
    gini: f64,
    stat: &crate::cmd::stats::StatsData,
) -> CliResult<Option<Panel>> {
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let label = sems
        .get(idx)
        .map(|s| s.label.as_str())
        .filter(|l| !l.is_empty())
        .map_or_else(|| col_label(&headers, idx, nh), ToString::to_string);

    let mut values: Vec<f64> = Vec::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        // non-negative finite values only: a Lorenz curve is defined on a non-negative
        // distribution (the same set moarstats' Gini was computed over); nulls/blanks and any
        // stray negative are skipped.
        if let Some(v) = parse_f64(record.get(idx)).filter(|v| v.is_finite() && *v >= 0.0) {
            values.push(v);
        }
    }

    let Some((pop, share)) = lorenz_curve(values) else {
        return Ok(None);
    };

    Ok(Some(
        Panel::new(
            format!("{label} \u{2014} Lorenz curve (Gini {gini:.2})"),
            PanelKind::Lorenz {
                pop,
                share,
                gini,
                label,
            },
        )
        .with_subtitle(Some(lorenz_caveat(stat))),
    ))
}

/// Compute the exact Lorenz curve for a set of non-negative values: sort ascending, then return
/// the cumulative population fraction (`x`, `k/n`) and cumulative value share (`y`, `Σv[0..k]/Σv`)
/// as row-aligned vectors, both led by the `(0,0)` origin so the curve starts on the equality
/// diagonal and ending at `(1,1)`. Returns `None` for a degenerate distribution (fewer than two
/// values, or a non-positive total — nothing a Gini of ~0 doesn't already say). The finished
/// monotone curve is thinned (endpoints kept, via `downsample_pair`) to `LORENZ_CURVE_MAX_POINTS`
/// vertices to bound embedded-HTML size; dropping vertices of an already-computed convex curve
/// leaves its area — the visual Gini — essentially unchanged.
fn lorenz_curve(mut values: Vec<f64>) -> Option<(Vec<f64>, Vec<f64>)> {
    if values.len() < 2 {
        return None;
    }
    values.sort_unstable_by(f64::total_cmp);
    let sum: f64 = values.iter().sum();
    if sum <= 0.0 {
        return None;
    }
    let n = values.len();
    #[allow(clippy::cast_precision_loss)]
    let n_f = n as f64;
    let mut pop = Vec::with_capacity(n + 1);
    let mut share = Vec::with_capacity(n + 1);
    pop.push(0.0);
    share.push(0.0);
    let mut cum = 0.0_f64;
    for (k, v) in values.iter().enumerate() {
        cum += *v;
        #[allow(clippy::cast_precision_loss)]
        pop.push((k + 1) as f64 / n_f);
        share.push(cum / sum);
    }
    Some(downsample_pair(&pop, &share, LORENZ_CURVE_MAX_POINTS))
}

/// How `viz smart` should treat the `--slider` flag: whether to auto-animate an eligible panel.
enum SmartSliderMode {
    /// never animate
    Off,
    /// animate whenever a candidate exists (a strong pair + a temporal axis), a lower bar than Auto
    On,
    /// animate only on a strong signal — the default
    Auto,
}

/// Resolve `--slider` for `viz smart`. `off`/`on`/`auto` (case-insensitive) are mode keywords;
/// absent defaults to `auto`; any other value (a column name) forces animation (`on`). A custom
/// frame column is not yet honored in smart (v1 always animates over the canonical date axis) —
/// it only turns animation on.
fn smart_slider_mode(args: &Args) -> SmartSliderMode {
    match args.flag_slider.as_deref().map(str::trim) {
        None | Some("") => SmartSliderMode::Auto,
        Some(s) if s.eq_ignore_ascii_case("off") => SmartSliderMode::Off,
        Some(s) if s.eq_ignore_ascii_case("on") => SmartSliderMode::On,
        Some(s) if s.eq_ignore_ascii_case("auto") => SmartSliderMode::Auto,
        // a column name forces animation on (over the canonical date axis, for v1)
        Some(_) => SmartSliderMode::On,
    }
}

/// A numeric axis range padded 5% on each side (a degenerate single-value span pads by 5% of the
/// magnitude, or a unit when the value is ~0). Empty input yields a `(0, 1)` fallback.
fn padded_range(vs: &[f64]) -> (f64, f64) {
    let (lo, hi) = vs
        .iter()
        .fold((f64::INFINITY, f64::NEG_INFINITY), |(a, b), &v| {
            (a.min(v), b.max(v))
        });
    if !lo.is_finite() || !hi.is_finite() {
        return (0.0, 1.0);
    }
    let span = (hi - lo).abs();
    let pad = if span > f64::EPSILON {
        span * 0.05
    } else {
        hi.abs().max(1.0) * 0.05
    };
    (lo - pad, hi + pad)
}

/// The row-aligned data for an [`PanelKind::AnimatedScatterPair`] panel: the pair's `xs`/`ys`
/// values with a per-point time-bucket index (all sorted by bucket ascending), the chronological
/// bucket labels, and the globally-pinned axis ranges.
struct ScatterPairAnim {
    xs:            Vec<f64>,
    ys:            Vec<f64>,
    bucket:        Vec<usize>,
    bucket_labels: Vec<String>,
    x_range:       (f64, f64),
    y_range:       (f64, f64),
}

/// Max animation frames / slider steps for `viz smart` time animations: each distinct calendar
/// bucket is one slider step, so the count is capped for legibility (see `choose_anim_bucket`).
const SMART_ANIM_MAX_FRAMES: usize = 30;

/// Frame cap for the EXPLICIT `--slider` animations (`viz <xy>`/`viz geo`), as opposed to the
/// `viz smart` animations that auto-coarsen to `SMART_ANIM_MAX_FRAMES`. One frame is built per
/// distinct slider value and each embeds its own copy of the plotted data — under
/// `--slider-cumulative` frame k re-copies every row from frames 0..=k, so total embedded data is
/// O(rows x frames). Uncapped, a timestamp column with 100k distinct values is unbounded. This
/// ceiling is deliberately generous (a year of daily frames fits); the user picked the column, so
/// exceeding it errors with guidance rather than silently re-bucketing behind their back.
const SLIDER_MAX_FRAMES: usize = 365;

/// Choose the finest calendar bucket (Day → Week → Month → Quarter → Year) whose distinct-period
/// count stays within `SMART_ANIM_MAX_FRAMES`, returning it with the sorted, de-duplicated
/// distinct bucket keys. Shared by the animated readers and the curvature selector so every time
/// animation agrees on the same frame axis. Falls back to Year (coarsest) when even quarterly
/// exceeds the cap — with yearly frames available, only spans beyond ~30 years overflow now
/// (issue #4216 widened the ladder, which previously topped out at Month). That residual
/// overflow is still ABOVE the cap, so `viz smart` callers go through
/// `choose_smart_anim_bucket`, which declines instead; the explicit `--slider` path keeps this
/// raw fallback because it enforces its own, far larger `SLIDER_MAX_FRAMES` ceiling.
fn choose_anim_bucket(dates: &[chrono::NaiveDate]) -> (TsBucket, Vec<chrono::NaiveDate>) {
    let distinct_for = |bkt: TsBucket| -> Vec<chrono::NaiveDate> {
        let mut d: Vec<chrono::NaiveDate> = dates.iter().map(|&x| ts_bucket_key(x, bkt)).collect();
        d.sort_unstable();
        d.dedup();
        d
    };
    [
        TsBucket::Day,
        TsBucket::Week,
        TsBucket::Month,
        TsBucket::Quarter,
        TsBucket::Year,
    ]
    .into_iter()
    .map(|b| (b, distinct_for(b)))
    .find(|(_, d)| d.len() <= SMART_ANIM_MAX_FRAMES)
    .unwrap_or_else(|| (TsBucket::Year, distinct_for(TsBucket::Year)))
}

/// `choose_anim_bucket` for the `viz smart` animations, which must honor `SMART_ANIM_MAX_FRAMES`
/// as a hard cap rather than a best effort: `choose_anim_bucket` falls back to Year when even
/// quarterly overflows, so a span beyond ~30 years still yields more steps than the cap allows.
/// Returning `None` there makes the caller decline the animation (falling back to the static
/// panel) instead of emitting a slider that ignores its own documented cap. The EXPLICIT
/// `--slider` path deliberately does NOT use this: it tolerates up to `SLIDER_MAX_FRAMES` and
/// errors with guidance past that, because the user picked the column.
fn choose_smart_anim_bucket(
    dates: &[chrono::NaiveDate],
) -> Option<(TsBucket, Vec<chrono::NaiveDate>)> {
    let (bkt, distinct) = choose_anim_bucket(dates);
    (distinct.len() <= SMART_ANIM_MAX_FRAMES).then_some((bkt, distinct))
}

/// Population standard deviation of a slice (0.0 for < 2 values or non-finite results). Used to put
/// centroid-path curvature in per-axis standardized units.
fn population_stddev(v: &[f64]) -> f64 {
    if v.len() < 2 {
        return 0.0;
    }
    let n = v.len() as f64;
    let mean = v.iter().sum::<f64>() / n;
    let var = v.iter().map(|&x| (x - mean).powi(2)).sum::<f64>() / n;
    let sd = var.sqrt();
    if sd.is_finite() { sd } else { 0.0 }
}

/// Centroid-path curvature for a pair, in standardized units (see `PAIR_CURVATURE_MIN`).
/// Standardize each per-time-bucket centroid `(cx[k], cy[k])` to z-space using the axis global
/// mean/stddev, then return the perpendicular spread of that path about its own best-fit line — √
/// of the smaller eigenvalue of the 2×2 covariance of the standardized path points. A straight path
/// (a stable or tautological relationship, or two independent trends) yields ~0; only a path that
/// genuinely bends over time scores high. Needs ≥ 3 buckets to bend at all (the caller guarantees
/// this).
fn pair_path_curvature(
    cx: &[f64],
    cy: &[f64],
    mean_x: f64,
    std_x: f64,
    mean_y: f64,
    std_y: f64,
) -> f64 {
    let n = cx.len();
    if std_x <= 0.0 || std_y <= 0.0 || cy.len() != n || n < 3 {
        return 0.0;
    }
    let nf = n as f64;
    let zx: Vec<f64> = cx.iter().map(|&x| (x - mean_x) / std_x).collect();
    let zy: Vec<f64> = cy.iter().map(|&y| (y - mean_y) / std_y).collect();
    let mx = zx.iter().sum::<f64>() / nf;
    let my = zy.iter().sum::<f64>() / nf;
    let sxx = zx.iter().map(|&x| (x - mx).powi(2)).sum::<f64>() / nf;
    let syy = zy.iter().map(|&y| (y - my).powi(2)).sum::<f64>() / nf;
    let sxy = zx
        .iter()
        .zip(&zy)
        .map(|(&x, &y)| (x - mx) * (y - my))
        .sum::<f64>()
        / nf;
    // smaller eigenvalue of [[sxx, sxy], [sxy, syy]] = variance perpendicular to the path's own
    // axis
    let tr = sxx + syy;
    let det = sxx * syy - sxy * sxy;
    let disc = (tr * tr / 4.0 - det).max(0.0);
    let lam_min = (tr / 2.0 - disc.sqrt()).max(0.0);
    let curv = lam_min.sqrt();
    if curv.is_finite() { curv } else { 0.0 }
}

/// One pass over the candidate numeric columns (`kept_indices`, in correlation-matrix order) plus
/// the canonical date column, returning the per-`(bucket, column)` mean for the frame axis chosen
/// by `choose_anim_bucket`. Rows missing the date or ANY column are skipped (listwise, matching the
/// correlation matrix). `None` when fewer than `min_frames` distinct buckets result. Feeds the
/// centroid-path-curvature pair selector.
fn read_bucketed_means(
    args: &Args,
    kept_indices: &[usize],
    date_idx: usize,
    prefer_dmy: bool,
    min_frames: usize,
) -> CliResult<Option<Vec<Vec<f64>>>> {
    let (mut rdr, _headers, _nh) = reader_and_headers(args)?;
    let ncol = kept_indices.len();
    let mut rows: Vec<(chrono::NaiveDate, Vec<f64>)> = Vec::new();
    let mut record = csv::ByteRecord::new();
    'row: while rdr.read_byte_record(&mut record)? {
        let Some(dt) = parse_record_date(&record, date_idx, prefer_dmy) else {
            continue;
        };
        let mut vals = Vec::with_capacity(ncol);
        for &ci in kept_indices {
            let Some(v) = parse_f64(record.get(ci)).filter(|v| v.is_finite()) else {
                continue 'row;
            };
            vals.push(v);
        }
        rows.push((dt.date_naive(), vals));
    }
    if rows.len() < 2 {
        return Ok(None);
    }
    let dates: Vec<chrono::NaiveDate> = rows.iter().map(|r| r.0).collect();
    // decline rather than emit more steps than the cap allows (see `choose_smart_anim_bucket`)
    let Some((bkt, distinct)) = choose_smart_anim_bucket(&dates) else {
        return Ok(None);
    };
    if distinct.len() < min_frames {
        return Ok(None);
    }
    let nb = distinct.len();
    let mut sums = vec![vec![0.0_f64; ncol]; nb];
    let mut counts = vec![0_usize; nb];
    for (d, vals) in &rows {
        let key = ts_bucket_key(*d, bkt);
        let bi = distinct
            .binary_search(&key)
            .expect("key came from distinct");
        counts[bi] += 1;
        for (acc, &v) in sums[bi].iter_mut().zip(vals) {
            *acc += v;
        }
    }
    let means: Vec<Vec<f64>> = sums
        .iter()
        .zip(&counts)
        .map(|(s, &c)| {
            let cf = c.max(1) as f64;
            s.iter().map(|v| v / cf).collect()
        })
        .collect();
    Ok(Some(means))
}

/// Decoupled animation-pair selector: pick the numeric pair whose per-time-bucket centroid path
/// BENDS the most (`pair_path_curvature`), requiring the curvature to clear `PAIR_CURVATURE_MIN`.
/// Deliberately NOT `strongest_pair`: the most-correlated pair is the most tautological — a near-
/// line whose centroids can't leave it (curvature ~0) — so it is the LEAST worth animating. Needs
/// no |r| band: only a genuinely evolving 2-D relationship (a curved centroid path) clears the bar;
/// a tautological line or two independent trends (a straight path) does not. Returns `(i, j,
/// signed_r)` in matrix/`columns` index space, or `None`. `bucket_means` is `[bucket][column]` from
/// `read_bucketed_means`.
fn select_drifting_pair(
    matrix: &[Vec<f64>],
    columns: &[Vec<f64>],
    bucket_means: &[Vec<f64>],
) -> Option<(usize, usize, f64)> {
    let n = matrix.len();
    let means: Vec<f64> = columns
        .iter()
        .map(|c| {
            if c.is_empty() {
                0.0
            } else {
                c.iter().sum::<f64>() / c.len() as f64
            }
        })
        .collect();
    let stds: Vec<f64> = columns.iter().map(|c| population_stddev(c)).collect();
    let mut best: Option<(usize, usize, f64, f64)> = None; // (i, j, r, curvature)
    for i in 0..n {
        for j in (i + 1)..n {
            let r = matrix[i][j];
            if r.is_nan() {
                continue;
            }
            let cx: Vec<f64> = bucket_means.iter().map(|b| b[i]).collect();
            let cy: Vec<f64> = bucket_means.iter().map(|b| b[j]).collect();
            let curv = pair_path_curvature(&cx, &cy, means[i], stds[i], means[j], stds[j]);
            if curv >= PAIR_CURVATURE_MIN && best.is_none_or(|(.., b)| curv > b) {
                best = Some((i, j, r, curv));
            }
        }
    }
    best.map(|(i, j, r, _)| (i, j, r))
}

/// Read the strongest-pair columns (`x_idx`/`y_idx`) together with the canonical date column in one
/// pass and bucket the points by calendar period for an animated, time-colored cumulative scatter.
/// The bucket granularity is chosen as the finest (Day through Year) that keeps the
/// frame/slider-step count legible. Returns `None` (no animation) when fewer than `min_frames`
/// distinct buckets result or too few points survive — the caller then keeps the static
/// `ScatterPair`.
fn read_scatter_pair_anim(
    args: &Args,
    x_idx: usize,
    y_idx: usize,
    date_idx: usize,
    prefer_dmy: bool,
    min_frames: usize,
) -> CliResult<Option<ScatterPairAnim>> {
    let (mut rdr, _headers, _nh) = reader_and_headers(args)?;
    // (bucket date, x, y) for every row with a finite x, a finite y, AND a parseable date
    let mut pts: Vec<(chrono::NaiveDate, f64, f64)> = Vec::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let Some(x) = parse_f64(record.get(x_idx)).filter(|v| v.is_finite()) else {
            continue;
        };
        let Some(y) = parse_f64(record.get(y_idx)).filter(|v| v.is_finite()) else {
            continue;
        };
        let Some(dt) = parse_record_date(&record, date_idx, prefer_dmy) else {
            continue;
        };
        pts.push((dt.date_naive(), x, y));
    }
    if pts.len() < 2 {
        return Ok(None);
    }
    // Each distinct bucket is a SLIDER STEP, so pick the finest granularity (Day through Year)
    // that keeps the step count legible — shared with the drift selector via `choose_anim_bucket`
    // so both agree on the frame axis. Unlike the trend LINE (`ts_bucket_for_span`,
    // span-based), this is frame-count-based: a daily series over 8 months (205 daily buckets)
    // coarsens to ~weekly.
    let dates: Vec<chrono::NaiveDate> = pts.iter().map(|p| p.0).collect();
    // decline rather than emit more steps than the cap allows (see `choose_smart_anim_bucket`)
    let Some((bkt, distinct)) = choose_smart_anim_bucket(&dates) else {
        return Ok(None);
    };
    if distinct.len() < min_frames {
        return Ok(None);
    }

    // (bucket_index, x, y) sorted by bucket ascending so a cumulative frame is a contiguous prefix
    let mut triples: Vec<(usize, f64, f64)> = pts
        .iter()
        .map(|p| {
            let key = ts_bucket_key(p.0, bkt);
            let bi = distinct
                .binary_search(&key)
                .expect("key came from distinct");
            (bi, p.1, p.2)
        })
        .collect();
    triples.sort_by_key(|t| t.0);
    // cap the embedded points (rarely hit at MAX_SMART_POINTS); a stride keep preserves bucket
    // order
    if triples.len() > *MAX_SMART_POINTS {
        let stride = triples.len().div_ceil(*MAX_SMART_POINTS);
        triples = triples.into_iter().step_by(stride).collect();
    }

    let xs: Vec<f64> = triples.iter().map(|t| t.1).collect();
    let ys: Vec<f64> = triples.iter().map(|t| t.2).collect();
    let bucket: Vec<usize> = triples.iter().map(|t| t.0).collect();
    let bucket_labels: Vec<String> = distinct.iter().map(|k| ts_bucket_label(*k, bkt)).collect();
    let x_range = padded_range(&xs);
    let y_range = padded_range(&ys);
    Ok(Some(ScatterPairAnim {
        xs,
        ys,
        bucket,
        bucket_labels,
        x_range,
        y_range,
    }))
}

/// The per-bucket point clouds for an [`PanelKind::AnimatedGeo`] panel: `bucket_lats[k]` /
/// `bucket_lons[k]` are the coordinates first seen in bucket `k` (render accumulates prefixes for
/// the cumulative reveal), and `bucket_labels` names each bucket chronologically.
struct GeoAnim {
    bucket_lats:   Vec<Vec<f64>>,
    bucket_lons:   Vec<Vec<f64>>,
    bucket_labels: Vec<String>,
}

/// How many points an `AnimatedGeo` panel actually serializes, given each bucket's point count.
///
/// The renderer emits a base trace holding every point, then one CUMULATIVE frame per bucket
/// (buckets `0..=k`), so a point in bucket `b` is repeated once per frame from `b` onward — the
/// serialized total is `total + Σ_b len(b) * (nb - b)`, roughly `total * (nb + 1) / 2`, NOT the
/// distinct-point count. Budgeting the point cap against the distinct count instead overshoots by
/// that factor (30 buckets => ~15x).
fn cumulative_embedded_points(buckets: &[Vec<f64>]) -> usize {
    let nb = buckets.len();
    let total: usize = buckets.iter().map(Vec::len).sum();
    total.saturating_add(
        buckets
            .iter()
            .enumerate()
            .map(|(b, pts)| pts.len().saturating_mul(nb - b))
            .sum::<usize>(),
    )
}

/// Read dated lat/lon rows in one pass and partition them into calendar buckets for an animated,
/// cumulative geographic reveal. Only fires for a continental/global extent (where `viz smart`
/// draws a `ScatterGeo` projection, which animates natively) — a city-scale cloud returns `None`,
/// so T2 never tries to animate a MapLibre tile map (blocked by issue #4155). Returns `None` when
/// the extent isn't global, when fewer than `min_frames` distinct buckets result, or when too few
/// rows survive.
fn read_geo_anim(
    args: &Args,
    lat_idx: usize,
    lon_idx: usize,
    date_idx: usize,
    prefer_dmy: bool,
    min_frames: usize,
) -> CliResult<Option<GeoAnim>> {
    let (mut rdr, _headers, _nh) = reader_and_headers(args)?;
    // (bucket date, lat, lon) for every row with valid coordinates AND a parseable date
    let mut pts: Vec<(chrono::NaiveDate, f64, f64)> = Vec::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let (Some(lat), Some(lon)) = (
            parse_f64(record.get(lat_idx)),
            parse_f64(record.get(lon_idx)),
        ) else {
            continue;
        };
        if !(-90.0..=90.0).contains(&lat) || !(-180.0..=180.0).contains(&lon) {
            continue;
        }
        let Some(dt) = parse_record_date(&record, date_idx, prefer_dmy) else {
            continue;
        };
        pts.push((dt.date_naive(), lat, lon));
    }
    if pts.len() < 2 {
        return Ok(None);
    }

    // require a continental/global extent: the FULL-cloud trimmed lon span >= SMART_GEO_MIN_LON_
    // SPAN_DEG OR trimmed lat span >= SMART_GEO_MIN_LAT_SPAN_DEG (the same thresholds
    // `geo_framing`/`build_map_panel` use to switch to the world projection). A city-scale cloud
    // falls through to the (non-animated) MapLibre map.
    let lons: Vec<f64> = pts.iter().map(|p| p.2).collect();
    let (_, lon_span) = lon_center_and_span(&lons, MAP_FRAME_TRIM_FRAC);
    let mut lats_sorted: Vec<f64> = pts.iter().map(|p| p.1).collect();
    lats_sorted.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
    let lat_span = sorted_quantile(&lats_sorted, 1.0 - MAP_FRAME_TRIM_FRAC)
        - sorted_quantile(&lats_sorted, MAP_FRAME_TRIM_FRAC);
    let global = lon_span >= SMART_GEO_MIN_LON_SPAN_DEG || lat_span >= SMART_GEO_MIN_LAT_SPAN_DEG;
    if !global {
        return Ok(None);
    }

    // bucket the dates: each distinct bucket is one slider step, so pick the finest granularity
    // that keeps the step count legible.
    let dates: Vec<chrono::NaiveDate> = pts.iter().map(|p| p.0).collect();
    // decline rather than emit more steps than the cap allows (see `choose_smart_anim_bucket`)
    let Some((bkt, distinct)) = choose_smart_anim_bucket(&dates) else {
        return Ok(None);
    };
    if distinct.len() < min_frames {
        return Ok(None);
    }
    let nb = distinct.len();
    let mut bucket_lats: Vec<Vec<f64>> = vec![Vec::new(); nb];
    let mut bucket_lons: Vec<Vec<f64>> = vec![Vec::new(); nb];
    for (d, lat, lon) in &pts {
        let key = ts_bucket_key(*d, bkt);
        let bi = distinct
            .binary_search(&key)
            .expect("key came from distinct");
        bucket_lats[bi].push(*lat);
        bucket_lons[bi].push(*lon);
    }

    // Keep the EMBEDDED point count bounded. The renderer emits one CUMULATIVE frame per bucket
    // (buckets `0..=k`), plus a base trace holding every point, so the serialized total is NOT the
    // distinct-point count: a point in bucket `b` is repeated once per frame from `b` onward.
    // Budgeting against the distinct count instead would overshoot the cap by ~nb/2 (30 buckets =>
    // ~15x), which is how a nominal 150k-point Data Schematic grew to tens of MB of HTML.
    let embedded = cumulative_embedded_points(&bucket_lats);
    if embedded > *MAX_SMART_POINTS {
        // A uniform per-bucket stride does NOT shrink the total proportionally: `step_by` keeps
        // the first element of every bucket, so a long sparse timeline (many buckets smaller than
        // the stride) can stay above the cap no matter how large the stride gets. Re-measure and
        // escalate until the cumulative embedded count is actually met, and stop if a round makes
        // no further progress (every bucket down to its single kept point).
        // Re-derive from the ORIGINALS each round so `stride` stays an absolute sampling rate;
        // striding the already-strided buckets would compound unpredictably.
        let (src_lats, src_lons) = (bucket_lats.clone(), bucket_lons.clone());
        let mut stride = embedded.div_ceil(*MAX_SMART_POINTS).max(2);
        loop {
            for (i, (la, lo)) in bucket_lats
                .iter_mut()
                .zip(bucket_lons.iter_mut())
                .enumerate()
            {
                *la = src_lats[i].iter().copied().step_by(stride).collect();
                *lo = src_lons[i].iter().copied().step_by(stride).collect();
            }
            if cumulative_embedded_points(&bucket_lats) <= *MAX_SMART_POINTS
                || bucket_lats.iter().all(|b| b.len() <= 1)
            {
                break;
            }
            stride = stride.saturating_mul(2);
        }
        // Striding cannot shrink a bucket below its single first point, so with enough buckets the
        // cumulative count is irreducible. `choose_anim_bucket` now tops out at Year (issue
        // #4216), so this is rare — but a dense multi-decade dataset can still overflow via
        // per-frame point counts. Decline the animation rather than emit a panel that ignores
        // its own cap — the caller
        // falls back to a static map, which shows the same points without the frame repetition.
        if cumulative_embedded_points(&bucket_lats) > *MAX_SMART_POINTS {
            viz_note(&format!(
                "viz smart: skipped the animated map — {} time buckets would embed more than the \
                 {} point budget even fully downsampled. Use a coarser date column to animate it.",
                nb, *MAX_SMART_POINTS
            ));
            return Ok(None);
        }
    }

    let bucket_labels: Vec<String> = distinct.iter().map(|k| ts_bucket_label(*k, bkt)).collect();
    Ok(Some(GeoAnim {
        bucket_lats,
        bucket_lons,
        bucket_labels,
    }))
}

/// The per-`[entity][bucket]` aggregates for an [`PanelKind::AnimatedBubble`] panel: the x/y
/// position and the size measure per surviving cell (`f64::NAN` where a cell was dropped or
/// absent), the ordered frame labels, and the globally-pinned axis ranges. Each value is its
/// cell's `--agg` aggregate — the mean (i.e. the centroid) unless the caller asked otherwise.
struct EntityBucketAgg {
    entities:      Vec<String>,
    xs:            Vec<Vec<f64>>,
    ys:            Vec<Vec<f64>>,
    sizes:         Vec<Vec<f64>>,
    bucket_labels: Vec<String>,
    x_range:       (f64, f64),
    y_range:       (f64, f64),
}

/// The raw per-`(entity, bucket)` cell values, kept UNAGGREGATED so `--agg` is applied exactly
/// once at materialization (see [`read_entity_bucket_agg`]).
#[derive(Default)]
struct CellVals {
    xs:    Vec<f64>,
    ys:    Vec<f64>,
    sizes: Vec<f64>,
}

/// Aggregate a cell's raw values into the single number a bubble plots. Mirrors the per-x
/// [`aggregate`] used by the bar/line paths, but over a whole cell rather than grouped by x.
fn agg_values(vals: &[f64], agg: Agg) -> f64 {
    match agg {
        Agg::Count => vals.len() as f64,
        Agg::Sum => vals.iter().sum(),
        Agg::Mean => {
            if vals.is_empty() {
                0.0
            } else {
                vals.iter().sum::<f64>() / vals.len() as f64
            }
        },
        Agg::Min => vals.iter().copied().fold(f64::INFINITY, f64::min),
        Agg::Max => vals.iter().copied().fold(f64::NEG_INFINITY, f64::max),
    }
}

/// Resolve the animation's frame axis from the raw `--slider` cells, returning the per-ROW bucket
/// index (`None` for a row whose cell doesn't belong on the axis) and the ordered frame labels.
///
/// THE CALENDAR PATH REQUIRES NEGLIGIBLE NUMERIC COVERAGE, and that is load-bearing rather than
/// cosmetic. `qsv_dateparser` reads a bare number as EPOCH SECONDS — "2010" becomes
/// 1970-01-01T00:33:30 — so a year column sent down the calendar path collapses every frame into a
/// single 1970 bucket and the animation dies outright. Even where it survives, calendar-bucketing
/// a year would relabel the frame "2020-01-01", disagreeing with `viz bar --slider year`, which
/// frames on distinct values.
///
/// So the two coverages are tested SEPARATELY (>=90% temporal AND <=10% numeric) rather than
/// against each other. A "more dates than numbers" comparison is not enough, because a numeric
/// cell also counts as a parsed date: 20 bare years plus one real "2024-01-01" gives 21 dates vs
/// 20 numbers, which would hand a fully numeric column to the calendar path and re-collapse it
/// into 1970. Requiring numeric coverage to be negligible makes the misclassification impossible
/// in that direction, and failing to raw mode is the safe fallback anyway — it is exactly what
/// `viz bar --slider` does.
///
/// This CANNOT divert `viz smart` off the calendar path it has always taken: smart only ever passes
/// a column `canonical_date_col` accepted, and that hard-filters on a stats type of
/// "Date"/"DateTime" — while `qsv stats` types an all-numeric column as Integer/Float even when it
/// looks like a date (`20230101`, epoch seconds, a bare year) and even under
/// `--dates-whitelist all`. So a smart-supplied frame column has zero numeric coverage.
fn resolve_frame_axis(
    cells: &[&str],
    prefer_dmy: bool,
) -> CliResult<Option<(Vec<Option<usize>>, Vec<String>)>> {
    let n_numeric = cells
        .iter()
        .filter(|c| parse_finite_f64(c).is_some())
        .count();
    {
        // temporal only when nearly every cell parses — a stats-typed date column (what `viz
        // smart` always passes) is ~100%, while a stray parseable token in a text column is not
        // enough to make the whole column a timeline
        let parsed: Vec<Option<chrono::NaiveDate>> = cells
            .iter()
            .map(|c| {
                qsv_dateparser::parse_with_preference(c, prefer_dmy)
                    .ok()
                    .map(|dt| dt.date_naive())
            })
            .collect();
        let n_parsed = parsed.iter().filter(|p| p.is_some()).count();
        if n_parsed >= 2 && n_parsed * 10 >= cells.len() * 9 && n_numeric * 10 <= cells.len() {
            let dates: Vec<chrono::NaiveDate> = parsed.iter().filter_map(|p| *p).collect();
            let (bkt, distinct) = choose_anim_bucket(&dates);
            if distinct.len() > SLIDER_MAX_FRAMES {
                return fail_incorrectusage_clierror!(
                    "--slider column yields {} animation frames; the maximum is \
                     {SLIDER_MAX_FRAMES}.\nEvery frame embeds its own copy of the plotted data, \
                     so this would produce an unusably large file. Bucket the column first (e.g. \
                     `qsv datefmt` to a coarser grain).",
                    distinct.len()
                );
            }
            let keys = parsed
                .iter()
                .map(|p| p.and_then(|d| distinct.binary_search(&ts_bucket_key(d, bkt)).ok()))
                .collect();
            let labels = distinct.iter().map(|k| ts_bucket_label(*k, bkt)).collect();
            return Ok(Some((keys, labels)));
        }
    }

    // distinct raw values, first-seen then ordered numerically when every value parses as finite,
    // else lexically — the same policy `read_xy_slider` uses, so `--slider year` frames identically
    // on every chart type
    let mut order: Vec<String> = Vec::new();
    let mut seen: HashSet<String> = HashSet::new();
    for c in cells {
        if seen.insert((*c).to_string()) {
            order.push((*c).to_string());
        }
    }
    if order.len() > SLIDER_MAX_FRAMES {
        return fail_incorrectusage_clierror!(
            "--slider column has {} distinct values; the maximum is {SLIDER_MAX_FRAMES}.\nEvery \
             frame embeds its own copy of the plotted data, so this would produce an unusably \
             large file. Bucket the column first (e.g. `qsv datefmt` to a coarser grain, or `qsv \
             apply`), or pick a lower-cardinality --slider column.",
            order.len()
        );
    }
    // ordering (as opposed to the mode choice above) still requires EVERY value to be numeric —
    // exactly `read_xy_slider`'s policy, so a mixed column sorts lexically on both paths
    if order.iter().all(|v| parse_finite_f64(v).is_some()) {
        order.sort_by(|a, b| {
            parse_finite_f64(a)
                .unwrap_or(f64::NAN)
                .partial_cmp(&parse_finite_f64(b).unwrap_or(f64::NAN))
                .unwrap_or(std::cmp::Ordering::Equal)
        });
    } else {
        order.sort();
    }
    let pos: HashMap<&str, usize> = order
        .iter()
        .enumerate()
        .map(|(i, s)| (s.as_str(), i))
        .collect();
    let keys = cells.iter().map(|c| pos.get(*c).copied()).collect();
    Ok(Some((keys, order)))
}

/// Knobs for [`read_entity_bucket_agg`]. The two callers differ only here: `viz smart` AUTO-selects
/// its bubble panel, so it sets strict gates that drop a sparse/jittery panel entirely; the
/// standalone `viz scatter --slider` was asked for explicitly, so it relaxes the gates and WARNS
/// instead of silently refusing to draw.
struct EntityBucketOpts {
    /// bubble-SIZE measure — Gapminder's third data variable. `None` falls back to the per-cell
    /// row COUNT so a 2-measure dataset still animates.
    size_idx:        Option<usize>,
    /// how each `(entity, bucket)` cell collapses to one point; `Mean` is the centroid the smart
    /// panel has always drawn
    agg:             Agg,
    /// frame k aggregates buckets `0..=k` (a RUNNING aggregate) instead of bucket k alone
    cumulative:      bool,
    prefer_dmy:      bool,
    /// minimum distinct frames on the axis, else there is no animation to draw
    min_frames:      usize,
    /// maximum distinct frames before the animation is DECLINED (`Ok(None)`), the counterpart of
    /// `min_frames`. `viz smart` passes `SMART_ANIM_MAX_FRAMES` for the same legibility reason
    /// its other animated panels do (see `choose_smart_anim_bucket`) — the bubble panel resolves
    /// its frame axis through `resolve_frame_axis` rather than the bucket ladder, so it needs the
    /// cap applied here. The standalone path passes `SLIDER_MAX_FRAMES`, which
    /// `resolve_frame_axis` has already enforced with an actionable error, making this a no-op
    /// there: the user picked the column, so they get the error and the guidance, never a
    /// silent refusal.
    max_frames:      usize,
    /// absolute floor on how many frames an entity must be dense in, applied alongside
    /// `min_dense_pct`. Kept SEPARATE from `min_frames` on purpose: `viz smart` sets both to the
    /// same value (its completeness bound subsumes the floor at typical bucket counts), but the
    /// standalone path wants a 2-frame animation while still drawing an entity that appears in
    /// only one of them.
    min_dense_floor: usize,
    min_cell_rows:   usize,
    /// minimum surviving entities, else the whole chart is refused
    min_entities:    usize,
    /// PERCENT of buckets an entity must be dense in to survive (smart 80, standalone 0). Integer
    /// percent, not a float ratio: `(nb * 80).div_ceil(100)` is exact, whereas
    /// `(nb as f64 * 0.8).ceil()` silently disagrees at nb=5 (0.8 has no exact binary
    /// representation, so 5*0.8 lands just above 4.0 and ceils to 5).
    min_dense_pct:   usize,
    /// report skipped entities / empty cells on stderr rather than silently dropping them
    warn:            bool,
}

/// Read one pass of (entity, x, y, size, frame) rows and aggregate them into per-`(entity, bucket)`
/// values for a Gapminder-style animated bubble chart. Returns `None` when the frame axis or the
/// surviving entities can't support an animation; `opts` decides how strict that is (see
/// [`EntityBucketOpts`]).
fn read_entity_bucket_agg(
    args: &Args,
    entity_idx: usize,
    x_idx: usize,
    y_idx: usize,
    frame_idx: usize,
    opts: &EntityBucketOpts,
) -> CliResult<Option<EntityBucketAgg>> {
    let (mut rdr, _headers, _nh) = reader_and_headers(args)?;
    // (entity, raw frame cell, x, y, size) for every row with a non-empty entity, finite x/y (and
    // a finite size when a size measure is requested) and a non-empty frame cell
    let mut rows: Vec<(String, String, f64, f64, f64)> = Vec::new();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let entity = cell_to_string(record.get(entity_idx));
        if entity.trim().is_empty() {
            continue;
        }
        let Some(x) = parse_f64(record.get(x_idx)).filter(|v| v.is_finite()) else {
            continue;
        };
        let Some(y) = parse_f64(record.get(y_idx)).filter(|v| v.is_finite()) else {
            continue;
        };
        // when sizing by a measure, require it finite too so the cell aggregate is well-defined
        // over the same rows the count/x/y use; the count fallback ignores this (size = row count).
        let size = match opts.size_idx {
            Some(si) => match parse_f64(record.get(si)).filter(|v| v.is_finite()) {
                Some(v) => v,
                None => continue,
            },
            None => 0.0,
        };
        let frame_cell = cell_to_string(record.get(frame_idx));
        if frame_cell.trim().is_empty() {
            continue;
        }
        rows.push((entity, frame_cell, x, y, size));
    }
    if rows.len() < 2 {
        return Ok(None);
    }

    // Resolve the frame axis. Numeric columns are checked FIRST and deliberately: a bare year like
    // "2020" also parses as a date, and coarsening it to a calendar bucket would label the frame
    // "2020-01-01" — so `viz scatter --slider year` would disagree with `viz bar --slider year`,
    // which frames on the column's distinct values. Only a genuinely temporal column takes the
    // calendar path.
    let frame_cells: Vec<&str> = rows.iter().map(|r| r.1.as_str()).collect();
    let (bucket_keys, bucket_labels) = match resolve_frame_axis(&frame_cells, opts.prefer_dmy)? {
        Some(axis) => axis,
        // an unparseable/degenerate frame axis is only reachable from the standalone path (smart
        // always hands us a stats-typed date column), and `build_slider_bubble_plot` turns the
        // `None` into an actionable error naming the column
        None => return Ok(None),
    };
    let nb = bucket_labels.len();
    if nb < opts.min_frames || nb > opts.max_frames {
        return Ok(None);
    }

    // Accumulate the RAW per-cell values (not running sums) so `--agg` is applied exactly once, at
    // materialization — the same discipline `read_xy_slider` uses. It also makes min/max correct
    // (they can't be folded into a single running scalar alongside a mean) and makes the cumulative
    // window a plain concatenation of the cells in `0..=k`.
    let mut acc: HashMap<(String, usize), CellVals> = HashMap::new();
    let mut entity_order: Vec<String> = Vec::new();
    let mut seen: HashSet<String> = HashSet::new();
    for ((entity, _, x, y, size), bi) in rows.iter().zip(&bucket_keys) {
        // a row whose frame cell fell off the axis (unparseable date in an otherwise temporal
        // column) is skipped, exactly as the date-only reader always did
        let Some(bi) = *bi else {
            continue;
        };
        if seen.insert(entity.clone()) {
            entity_order.push(entity.clone());
        }
        let e = acc.entry((entity.clone(), bi)).or_default();
        e.xs.push(*x);
        e.ys.push(*y);
        e.sizes.push(*size);
    }
    drop(rows);

    // Drop sparse cells, then keep only entities whose surviving (>= min_cell_rows) cells cover a
    // dense-enough panel. `min_dense_ratio` is what separates the two callers: `viz smart` passes
    // 0.8 because it AUTO-selects this panel and a bubble that flickers in and out reads as
    // sampling noise rather than motion; the standalone command passes 0.0 and instead WARNS,
    // because a user who spelled out --series/--size/--slider asked for this chart specifically.
    let min_dense = opts
        .min_dense_floor
        .max((nb * opts.min_dense_pct).div_ceil(100));
    let dense_count = |ent: &String| -> usize {
        (0..nb)
            .filter(|&bi| {
                acc.get(&(ent.clone(), bi))
                    .is_some_and(|c| c.xs.len() >= opts.min_cell_rows)
            })
            .count()
    };
    let total_entities = entity_order.len();
    let surviving: Vec<String> = entity_order
        .into_iter()
        .filter(|ent| dense_count(ent) >= min_dense)
        .collect();
    if surviving.len() < opts.min_entities {
        return Ok(None);
    }
    if opts.warn {
        let dropped = total_entities - surviving.len();
        if dropped > 0 {
            viz_note(&format!(
                "note: {dropped} of {total_entities} --series values have usable data in fewer \
                 than {min_dense} of the {nb} frames and were skipped."
            ));
        }
        let gaps: usize = surviving.iter().map(|e| nb - dense_count(e)).sum();
        if gaps > 0 {
            viz_note(&format!(
                "note: {gaps} of {} (series x frame) cells have no data; those bubbles are hidden \
                 in the affected frames.",
                surviving.len() * nb
            ));
        }
    }

    // build [entity][bucket] value matrices; NaN where a cell was dropped or absent. Each cell
    // aggregates its raw values with `--agg` (default mean, matching the centroid the smart panel
    // has always drawn). Size aggregates the measure the same way when one was supplied, else it
    // is the per-cell row COUNT.
    let mut xs: Vec<Vec<f64>> = Vec::with_capacity(surviving.len());
    let mut ys: Vec<Vec<f64>> = Vec::with_capacity(surviving.len());
    let mut sizes: Vec<Vec<f64>> = Vec::with_capacity(surviving.len());
    let (mut xv, mut yv): (Vec<f64>, Vec<f64>) = (Vec::new(), Vec::new());
    for ent in &surviving {
        let mut ex = Vec::with_capacity(nb);
        let mut ey = Vec::with_capacity(nb);
        let mut es = Vec::with_capacity(nb);
        for bi in 0..nb {
            // a cumulative frame aggregates every surviving cell at or below it, so the bubble
            // shows a RUNNING aggregate rather than that frame's value alone
            let lo = if opts.cumulative { 0 } else { bi };
            let mut cx: Vec<f64> = Vec::new();
            let mut cy: Vec<f64> = Vec::new();
            let mut cs: Vec<f64> = Vec::new();
            for prev in lo..=bi {
                if let Some(c) = acc.get(&(ent.clone(), prev))
                    && c.xs.len() >= opts.min_cell_rows
                {
                    cx.extend_from_slice(&c.xs);
                    cy.extend_from_slice(&c.ys);
                    cs.extend_from_slice(&c.sizes);
                }
            }
            if cx.is_empty() {
                ex.push(f64::NAN);
                ey.push(f64::NAN);
                es.push(f64::NAN);
                continue;
            }
            let px = agg_values(&cx, opts.agg);
            let py = agg_values(&cy, opts.agg);
            ex.push(px);
            ey.push(py);
            es.push(if opts.size_idx.is_some() {
                agg_values(&cs, opts.agg)
            } else {
                cs.len() as f64
            });
            xv.push(px);
            yv.push(py);
        }
        xs.push(ex);
        ys.push(ey);
        sizes.push(es);
    }
    if xv.is_empty() {
        return Ok(None);
    }
    let x_range = padded_range(&xv);
    let y_range = padded_range(&yv);
    Ok(Some(EntityBucketAgg {
        // `entities` is display-only downstream (legend name + hover), never a key again — the
        // `acc` lookups above are done — so escape once here, matching the read-time escaping of
        // the other hover-bound cell values. plotly renders both sinks as pseudo-HTML.
        entities: surviving.iter().map(|e| escape_hover(e)).collect(),
        xs,
        ys,
        sizes,
        bucket_labels,
        x_range,
        y_range,
    }))
}

/// A bubble-size scaler over a WHOLE `[entity][bucket]` matrix.
///
/// The global min/max is load-bearing and is why this exists instead of [`scale_bubble_sizes`]:
/// every bubble is its own single-point trace, so a per-trace rescale would see one value, land it
/// at `t = 0.5`, and render every bubble mid-sized — silently destroying the size encoding. Scaling
/// once over all cells keeps an entity's size comparable across frames AND across entities.
fn bubble_size_px(sizes: &[Vec<f64>]) -> impl Fn(f64) -> usize {
    let (smin, smax) = sizes
        .iter()
        .flatten()
        .filter(|v| v.is_finite())
        .fold((f64::INFINITY, f64::NEG_INFINITY), |(lo, hi), &v| {
            (lo.min(v), hi.max(v))
        });
    let sspan = smax - smin;
    move |v: f64| -> usize {
        let t = if !v.is_finite() {
            0.0
        } else if sspan > 0.0 {
            (v - smin) / sspan
        } else {
            0.5
        };
        (BUBBLE_MIN_PX + t * (BUBBLE_MAX_PX - BUBBLE_MIN_PX)).round() as usize
    }
}

/// Add a Gapminder bubble animation's traces and frames to `plot`: one single-point trace per
/// entity, plus one per-frame FULL REPLACEMENT per bucket (every entity trace updates).
///
/// Shared by the `viz smart` bubble panel and the standalone `viz scatter --slider` chart; the
/// caller owns the layout, which genuinely differs between them (panel height/title vs
/// `--title`/`--x-title`/`--slider-speed`).
///
/// Two invariants here are easy to "clean up" and must not be:
/// - an absent cell emits EMPTY coordinate vectors, not NaN, so the entity simply vanishes that
///   frame while the trace count and index order stay constant (plotly matches frame traces by
///   index; a varying count leaves stale traces on screen);
/// - the hover escaping is deliberately asymmetric — entities are already `escape_hover`'d at read
///   time and must NOT be escaped twice (-> "&amp;amp;"), while the three labels are raw CSV
///   headers and take the full `escape_template_pct(escape_hover(..))` composition. Pinned by the
///   `escape_composition_is_once_per_sink` unit test.
fn add_bubble_traces_and_frames(
    plot: &mut Plot,
    data: &EntityBucketAgg,
    x_label: &str,
    y_label: &str,
    size_label: &str,
) {
    let EntityBucketAgg {
        entities,
        xs,
        ys,
        sizes,
        bucket_labels,
        ..
    } = data;
    let n_entities = entities.len();
    let size_px = bubble_size_px(sizes);
    let bubble = |e: usize, k: usize| {
        let (px, py, sz) = (xs[e][k], ys[e][k], sizes[e][k]);
        let (pxv, pyv) = if px.is_finite() && py.is_finite() {
            (vec![px], vec![py])
        } else {
            (Vec::new(), Vec::new())
        };
        // format the size measure cleanly: integral values (e.g. a row count) show with no
        // decimal, fractional measures (e.g. a mean population) keep up to 2 places — do NOT
        // cast to an integer, which would truncate a decimal measure and misreport it.
        let size_disp = if sz.is_finite() {
            format!("{}", (sz * 100.0).round() / 100.0)
        } else {
            "0".to_string()
        };
        Scatter::new(pxv, pyv)
            .mode(Mode::Markers)
            // already `escape_hover`d at read time; the legend renders markup but treats `%`
            // literally, so it needs no further escaping here
            .name(entities[e].clone())
            .marker(Marker::new().size(size_px(sz)).opacity(MAP_POINT_OPACITY))
            // a hovertemplate additionally parses `%{...}`, so every interpolated string needs
            // its literal `%` doubled.
            .hover_template(format!(
                "{}<br>{}: %{{x}}<br>{}: %{{y}}<br>{}: {size_disp}<extra></extra>",
                escape_template_pct(&entities[e]),
                escape_template_pct(&escape_hover(x_label)),
                escape_template_pct(&escape_hover(y_label)),
                escape_template_pct(&escape_hover(size_label)),
            ))
    };
    // base = bucket 0 positions (one trace per entity, in entity order)
    for e in 0..n_entities {
        plot.add_trace(bubble(e, 0));
    }
    // one per-frame replacement per bucket: every entity trace updates (traces 0..n)
    for (k, label) in bucket_labels.iter().enumerate() {
        let mut td = Traces::new();
        for e in 0..n_entities {
            td.push(bubble(e, k));
        }
        plot.add_frame(
            Frame::new()
                .name(label.clone())
                .data(td)
                .traces((0..n_entities).collect()),
        );
    }
}

/// The repeating phase a `CyclicProfile` folds timestamps onto.
#[derive(Clone, Copy, PartialEq, Eq)]
enum CyclicAxis {
    HourOfDay,
    DayOfWeek,
    MonthOfYear,
}

impl CyclicAxis {
    /// Number of phase buckets in the cycle.
    fn len(self) -> usize {
        match self {
            CyclicAxis::HourOfDay => 24,
            CyclicAxis::DayOfWeek => 7,
            CyclicAxis::MonthOfYear => 12,
        }
    }

    /// 0-based bucket index for a timestamp.
    fn bucket(self, dt: &chrono::DateTime<chrono::Utc>) -> usize {
        use chrono::{Datelike, Timelike};
        match self {
            CyclicAxis::HourOfDay => dt.hour() as usize,
            CyclicAxis::DayOfWeek => dt.weekday().num_days_from_monday() as usize,
            CyclicAxis::MonthOfYear => dt.month0() as usize,
        }
    }

    /// Human label for each bucket (length == `self.len()`).
    fn labels(self) -> Vec<String> {
        match self {
            CyclicAxis::HourOfDay => (0..24).map(|h| format!("{h:02}h")).collect(),
            // Spelled out with LITERAL keys rather than mapping a runtime key over
            // a `&str` array: `every_t_key_used_in_this_file_exists_in_the_catalog`
            // scans for literal keys, and a runtime one slips past it -- leaving a
            // typo to render as raw key text on the axis with no error anywhere.
            // (That guard reads this file as TEXT, so don't write the macro-call
            // token in a comment either; doing so is itself reported as a
            // non-literal call site.)
            CyclicAxis::DayOfWeek => vec![
                t!("viz.cycle.day_mon").into_owned(),
                t!("viz.cycle.day_tue").into_owned(),
                t!("viz.cycle.day_wed").into_owned(),
                t!("viz.cycle.day_thu").into_owned(),
                t!("viz.cycle.day_fri").into_owned(),
                t!("viz.cycle.day_sat").into_owned(),
                t!("viz.cycle.day_sun").into_owned(),
            ],
            CyclicAxis::MonthOfYear => vec![
                t!("viz.cycle.month_jan").into_owned(),
                t!("viz.cycle.month_feb").into_owned(),
                t!("viz.cycle.month_mar").into_owned(),
                t!("viz.cycle.month_apr").into_owned(),
                t!("viz.cycle.month_may").into_owned(),
                t!("viz.cycle.month_jun").into_owned(),
                t!("viz.cycle.month_jul").into_owned(),
                t!("viz.cycle.month_aug").into_owned(),
                t!("viz.cycle.month_sep").into_owned(),
                t!("viz.cycle.month_oct").into_owned(),
                t!("viz.cycle.month_nov").into_owned(),
                t!("viz.cycle.month_dec").into_owned(),
            ],
        }
    }

    /// Word for the panel title.
    fn word(self) -> String {
        match self {
            CyclicAxis::HourOfDay => t!("viz.title.cyclic_hour_of_day"),
            CyclicAxis::DayOfWeek => t!("viz.title.cyclic_day_of_week"),
            CyclicAxis::MonthOfYear => t!("viz.title.cyclic_month"),
        }
        .into_owned()
    }
}

/// Minimum populated phase buckets for a `CyclicProfile` to be worth drawing: fewer than this isn't
/// a cycle, just a couple of points scattered on a ring.
const CYCLIC_MIN_POPULATED_BUCKETS: usize = 3;

/// Build the cyclic-seasonality polar panel: fold the canonical date/datetime column onto a
/// repeating phase (hour-of-day for datetimes; day-of-week or month-of-year for dates, by span) and
/// plot record volume per phase. Complements `build_timeseries_panel` (absolute timeline) by
/// surfacing periodicity. Returns `Ok(None)` when there's no date column or too few phase buckets
/// are populated to show a real cycle.
fn build_cyclic_panel(
    args: &Args,
    stats: &[crate::cmd::stats::StatsData],
    dmy_prefs: &[bool],
    map_cols: Option<(usize, usize)>,
    sems: &[ColSemantics],
    dict_icons: Option<&DictData>,
) -> CliResult<Option<Panel>> {
    use qsv_dateparser::parse_with_preference;

    // pick the canonical date/datetime column, same ranking as the time-series panel.
    let is_map_col = |idx: usize| map_cols.is_some_and(|(la, lo)| idx == la || idx == lo);
    let Some((date_idx, is_datetime)) = stats
        .iter()
        .enumerate()
        .filter(|(i, _)| !is_map_col(*i))
        .filter_map(|(i, s)| match s.r#type.as_str() {
            "Date" => Some((i, false)),
            "DateTime" => Some((i, true)),
            _ => None,
        })
        .min_by_key(|&(i, _)| {
            (
                timestamp_rank(sems.get(i).map_or("", |s| s.concept.as_str())),
                sort_order_rank(&stats[i]),
                i,
            )
        })
    else {
        return Ok(None);
    };
    // the date column is picked here, so resolve its parsing preference here too (issue #4303).
    let prefer_dmy = dmy_prefs.get(date_idx).copied().unwrap_or(false);

    // choose the cycle: intraday timestamps -> hour-of-day; date-only -> month-of-year when the
    // span is wide enough to show a yearly shape, else the weekly rhythm.
    let axis = if is_datetime {
        CyclicAxis::HourOfDay
    } else {
        let parse_bound = |o: &Option<String>| {
            o.as_deref()
                .map(str::trim)
                .filter(|t| !t.is_empty())
                .and_then(|t| parse_with_preference(t, prefer_dmy).ok())
        };
        let span_days = match (
            parse_bound(&stats[date_idx].min),
            parse_bound(&stats[date_idx].max),
        ) {
            (Some(lo), Some(hi)) => (hi - lo).num_days().max(0),
            _ => 0,
        };
        if span_days >= 60 {
            CyclicAxis::MonthOfYear
        } else {
            CyclicAxis::DayOfWeek
        }
    };

    let mut counts = vec![0u64; axis.len()];
    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        if let Some(dt) = parse_record_date(&record, date_idx, prefer_dmy) {
            counts[axis.bucket(&dt)] += 1;
        }
    }

    if counts.iter().filter(|&&c| c > 0).count() < CYCLIC_MIN_POPULATED_BUCKETS {
        return Ok(None);
    }

    // close the ring: repeat the first phase/value at the end so the polar line connects around.
    let mut theta = axis.labels();
    theta.push(theta[0].clone());
    let mut r: Vec<f64> = counts.iter().map(|&c| c as f64).collect();
    r.push(r[0]);

    let date_label = sems
        .get(date_idx)
        .map(|s| s.label.as_str())
        .filter(|l| !l.is_empty())
        .map_or_else(|| col_label(&headers, date_idx, nh), ToString::to_string);

    Ok(Some(
        Panel::new(
            t!(
                "viz.title.cyclic_records_by",
                q_axis = axis.word(),
                q_date = date_label
            )
            .into_owned(),
            PanelKind::CyclicProfile { theta, r },
        )
        // --dict-info: like the trend panel, anchor on the driving date column's entry
        .with_dict_info(dict_info_for_field(dict_icons, &stats[date_idx].field)),
    ))
}

/// Detect the latitude/longitude column index pair by header name + numeric stats type. Shared by
/// the map-panel builder and the `viz smart` classifier so that columns recognized as geographic
/// coordinates are charted on the map panel only — not redundantly as per-column distribution
/// panels, a correlation-matrix axis, or the time-series y. Name detection needs headers, so this
/// returns None under `--no-headers` (field names are then "0","1",... and won't match).
fn latlon_indices(stats: &[crate::cmd::stats::StatsData]) -> Option<(usize, usize)> {
    let find = |names: &[&str]| {
        stats
            .iter()
            .enumerate()
            .find(|(_, s)| {
                matches!(s.r#type.as_str(), "Integer" | "Float")
                    && names.contains(&s.field.to_ascii_lowercase().as_str())
            })
            .map(|(i, _)| i)
    };
    match (
        find(&["lat", "latitude"]),
        find(&["lon", "long", "lng", "longitude"]),
    ) {
        (Some(lat), Some(lon)) => Some((lat, lon)),
        _ => None,
    }
}

// Rough bounding box of the United States (including Alaska and Hawaii) for the `viz smart` static
// geo map's `albers usa` heuristic.
const US_LON_MIN: f64 = -170.0;
const US_LON_MAX: f64 = -66.0;
const US_LAT_MIN: f64 = 18.0;
const US_LAT_MAX: f64 = 72.0;
// `albers usa` is only chosen when the data spans a continental fraction of the US, so a single
// US city renders as a fitted Mercator view rather than a US-wide composite.
const US_SPAN_MIN_LON_DEG: f64 = 15.0;
const US_SPAN_MIN_LAT_DEG: f64 = 8.0;

// Plotly.js `layout.geo.scope` continent vocabulary and approximate (lon_min, lon_max, lat_min,
// lat_max) bounding boxes, aligning the `viz smart` continent-framing heuristic with the enumerated
// scopes at https://plotly.com/javascript/reference/layout/geo/#layout-geo-scope. "usa", "world"
// and "antarctica" are intentionally omitted: US extents are handled by the `albers usa` heuristic
// above, a world-spanning extent stays the NaturalEarth overview, and Antarctica has no `viz smart`
// use case. Boxes overlap where continents abut (e.g. Eurasia); `continent_scope` only frames to a
// continent when the data falls within EXACTLY one box, so a straddling extent stays the overview.
const CONTINENT_SCOPES: &[(&str, f64, f64, f64, f64)] = &[
    ("africa", -20.0, 52.0, -38.0, 38.0),
    ("europe", -25.0, 45.0, 35.0, 72.0),
    ("asia", 26.0, 180.0, -11.0, 82.0),
    ("north america", -170.0, -50.0, 7.0, 84.0),
    ("south america", -82.0, -34.0, -56.0, 14.0),
    ("oceania", 110.0, 180.0, -50.0, 0.0),
];
// padding (as a fraction of the span, with a small floor) added around a fitted local geo map so
// points aren't flush against the frame.
const GEO_FIT_PAD_FRAC: f64 = 0.1;
const GEO_FIT_PAD_MIN_DEG: f64 = 0.5;

/// Choose the projection and (for local extents) the fitted lon/lat axis ranges for a `viz smart`
/// static geo map. Data that spans a continental fraction of the US uses `albers usa` (a fixed
/// composite that frames the US — CONUS plus Alaska/Hawaii insets — so no axis ranges are set);
/// other continental/global extents use the world `NaturalEarth` projection; everything else uses
/// `Mercator` fit to the data's trimmed extent so a city-scale dataset renders as a zoomed-in map
/// instead of a dot on the world. Returns the longitude and latitude ranges separately so a local
/// extent that wraps the antimeridian can still fit latitude while leaving longitude full-width.
/// Mirrors the antimeridian-aware, trimmed framing semantics that `build_map_panel`/the map view
/// use.
fn geo_framing(
    lats: &[f64],
    lons: &[f64],
    trim_frac: f64,
) -> (ProjectionType, Option<Axis>, Option<Axis>) {
    // `trim_frac` is how much of each tail to drop before framing: `MAP_FRAME_TRIM_FRAC` for
    // point data (a stray coordinate shouldn't dominate the view), but `0.0` for custom GeoJSON,
    // whose every vertex is intentional geometry — trimming there would clip legitimate edge or
    // island polygons out of the rendered choropleth.
    let (lon_center, lon_span) = lon_center_and_span(lons, trim_frac);
    let mut lats_sorted = lats.to_vec();
    lats_sorted.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
    let lat_lo = sorted_quantile(&lats_sorted, trim_frac);
    let lat_hi = sorted_quantile(&lats_sorted, 1.0 - trim_frac);
    let lat_span = lat_hi - lat_lo;

    let lon_lo = lon_center - lon_span / 2.0;
    let lon_hi = lon_center + lon_span / 2.0;

    // US-spanning extent: `albers usa` frames the US itself (CONUS + Alaska/Hawaii insets) and
    // ignores lon/lat axis ranges, so don't set them. Checked BEFORE the global fallback so a
    // coast-to-coast dataset that includes Alaska/Hawaii — and therefore exceeds the global span
    // thresholds — still gets albers usa rather than a NaturalEarth world overview.
    let within_us = lon_lo >= US_LON_MIN
        && lon_hi <= US_LON_MAX
        && lat_lo >= US_LAT_MIN
        && lat_hi <= US_LAT_MAX;
    if within_us && lon_span >= US_SPAN_MIN_LON_DEG && lat_span >= US_SPAN_MIN_LAT_DEG {
        return (ProjectionType::AlbersUsa, None, None);
    }

    // other continental/global extent: a fitted view would be unwieldy, so use the world
    // projection.
    if lon_span >= SMART_GEO_MIN_LON_SPAN_DEG || lat_span >= SMART_GEO_MIN_LAT_SPAN_DEG {
        return (ProjectionType::NaturalEarth, None, None);
    }

    // local extent: Mercator fit to the padded data bounds, clamped to valid coordinate ranges.
    let lat_pad = (lat_span * GEO_FIT_PAD_FRAC).max(GEO_FIT_PAD_MIN_DEG);
    let lataxis = Axis::new().range(vec![
        (lat_lo - lat_pad).max(-90.0),
        (lat_hi + lat_pad).min(90.0),
    ]);

    // a cluster that wraps the antimeridian yields fitted longitudes outside [-180, 180]; clamping
    // them would crop the points on the far side of +/-180, so skip the fitted longitude range
    // (let the projection show the full width) and fit latitude only.
    if lon_lo < -180.0 || lon_hi > 180.0 {
        return (ProjectionType::Mercator, None, Some(lataxis));
    }

    let lon_pad = (lon_span * GEO_FIT_PAD_FRAC).max(GEO_FIT_PAD_MIN_DEG);
    let lonaxis = Axis::new().range(vec![
        (lon_lo - lon_pad).max(-180.0),
        (lon_hi + lon_pad).min(180.0),
    ]);
    (ProjectionType::Mercator, Some(lonaxis), Some(lataxis))
}

/// If every point falls within exactly one plotly.js geo `scope` continent box (see
/// `CONTINENT_SCOPES`), returns that scope name so the `viz smart` geo panel can frame the map to
/// that continent instead of a world overview. Returns `None` when the points span no single
/// continent (world-spanning data) or straddle more than one (e.g. a Europe/Asia mix), leaving the
/// existing `NaturalEarth` overview in place.
fn continent_scope(lats: &[f64], lons: &[f64]) -> Option<&'static str> {
    if lats.is_empty() || lats.len() != lons.len() {
        return None;
    }
    let mut found: Option<&'static str> = None;
    for &(scope, lon_min, lon_max, lat_min, lat_max) in CONTINENT_SCOPES {
        let all_in = lats
            .iter()
            .zip(lons)
            .all(|(&la, &lo)| la >= lat_min && la <= lat_max && lo >= lon_min && lo <= lon_max);
        if all_in {
            if found.is_some() {
                // the extent fits more than one continent box (continents abut, e.g. Eurasia) —
                // ambiguous, so keep the world overview.
                return None;
            }
            found = Some(scope);
        }
    }
    found
}

/// Detect the latitude/longitude column pair from describegpt dictionary signals — concept
/// `geo.latitude`/`geo.longitude`, or `content_type` `latitude`/`longitude` — so coordinates with
/// non-standard headers (e.g. `X Coordinate`/`Y Coordinate`) still drive the map panel. Each must
/// be a numeric stats type. Returns None unless BOTH a latitude and a longitude column are found,
/// so the caller falls back to the header-name heuristic (`latlon_indices`).
fn semantic_latlon(
    stats: &[crate::cmd::stats::StatsData],
    dict: &DictData,
) -> Option<(usize, usize)> {
    // Some(true) = latitude, Some(false) = longitude, None = not a coordinate
    let axis_of = |row: &DictRow| -> Option<bool> {
        let concept = row.concept.trim();
        let ct = row
            .content_type
            .split_once(':')
            .map_or(row.content_type.as_str(), |(b, _)| b)
            .trim();
        if concept == "geo.latitude" || ct == "latitude" {
            Some(true)
        } else if concept == "geo.longitude" || ct == "longitude" {
            Some(false)
        } else {
            None
        }
    };
    let mut lat = None;
    let mut lon = None;
    for (i, s) in stats.iter().enumerate() {
        if !matches!(s.r#type.as_str(), "Integer" | "Float") {
            continue;
        }
        let Some(row) = dict.rows.get(&s.field) else {
            continue;
        };
        match axis_of(row) {
            Some(true) if lat.is_none() => lat = Some(i),
            Some(false) if lon.is_none() => lon = Some(i),
            _ => {},
        }
    }
    Some((lat?, lon?))
}

/// Count occurrences of each non-empty key, returning them in first-seen order alongside the count
/// map (matches `aggregate`'s first-seen ordering).
#[cfg(feature = "geocode")]
fn tally(keys: impl Iterator<Item = String>) -> (Vec<String>, HashMap<String, f64>) {
    let mut order: Vec<String> = Vec::new();
    let mut counts: HashMap<String, f64> = HashMap::new();
    for key in keys {
        if key.is_empty() {
            continue;
        }
        counts
            .entry(key.clone())
            .and_modify(|c| *c += 1.0)
            .or_insert_with(|| {
                order.push(key);
                1.0
            });
    }
    (order, counts)
}

/// Build a `viz smart` choropleth overview panel from already-collected map coordinates: reverse-
/// geocode the points (reusing qsv's geocode engine), then choose the breakdown from what actually
/// resolved — when every resolved point is in the USA, a per-US-state fill (the informative view,
/// needs 2+ states); otherwise a per-ISO-3-country fill (needs 2+ countries). Returns `None` when
/// neither has 2+ regions (a single-region or metro dataset stays point-only). Deciding from the
/// resolved countries — not the broad US bounding box — keeps multi-country datasets that happen to
/// fall inside that box (e.g. US + Mexico/Canada/Caribbean) on the per-country panel.
/// Geocode-gated.
#[cfg(feature = "geocode")]
fn build_smart_choropleth_panel(lats: &[f64], lons: &[f64]) -> Option<Panel> {
    let points: Vec<(f64, f64)> = lats.iter().copied().zip(lons.iter().copied()).collect();
    let regions: Vec<crate::cmd::geocode::GeoRegion> =
        crate::cmd::geocode::reverse_geocode_regions(&points, None)
            .ok()?
            .into_iter()
            .flatten()
            .collect();

    // decide the all-USA case from every resolved region's ISO-2 country code, NOT from the ISO-3
    // tally: a GeoRegion can carry a valid `iso2` ("MX") yet an empty `iso3` when the country-info
    // lookup fails, and the ISO-3 tally skips empties — so a non-US point could vanish and leave an
    // all-"USA" set that wrongly renders US states (silently dropping that point). `iso2` is set
    // straight from the matched record, so it's the reliable signal.
    let all_usa = !regions.is_empty() && regions.iter().all(|r| r.iso2 == "US");
    let (name, location_mode, order, counts) = if all_usa {
        let (state_order, state_counts) =
            tally(regions.iter().filter_map(|r| r.us_state_code.clone()));
        if state_order.len() < 2 {
            // all-USA but a single state: nothing a point map doesn't already show.
            return None;
        }
        (
            t!("viz.title.us_states").into_owned(),
            LocationMode::UsaStates,
            state_order,
            state_counts,
        )
    } else {
        let (country_order, country_counts) = tally(regions.iter().map(|r| r.iso3.clone()));
        if country_order.len() < 2 {
            return None;
        }
        (
            t!("viz.title.countries").into_owned(),
            LocationMode::Iso3,
            country_order,
            country_counts,
        )
    };

    let z: Vec<f64> = order.iter().map(|key| counts[key]).collect();
    let hover_text = choropleth_hover_text(&order, &z, None, &t!("viz.chart.count"), true);
    Some(Panel::new(
        name.to_string(),
        PanelKind::Choropleth {
            locations: order,
            z,
            location_mode,
            geojson: None,
            feature_id_key: None,
            hover_text,
            measure_label: t!("viz.chart.count").into_owned(),
            region_filter: None,
        },
    ))
}

/// Build a `viz smart` choropleth panel by point-in-polygon binning the core lat/lon points into a
/// user-supplied GeoJSON (`--geojson`). Each point is assigned to the region whose polygon contains
/// it, or — when `snap` — to the nearest region within the snap-distance cap (an explicit
/// `--snap-max-dist` verbatim, else auto-derived from the GeoJSON's median region size and the
/// stats cache's lat/lon coordinate precision; see [`resolve_snap_cap`]), else dropped; the panel
/// is colored by per-region counts. Mirrors the `viz choropleth` command's cap: far strays are
/// dropped (surfaced in the panel title + per-region hover) rather than snapped to an arbitrary
/// distant region, and any snapping is surfaced in the panel title with the cap applied. Unlike
/// [`build_smart_choropleth_panel`] this needs no geocode engine. Errors when the explicit
/// `--geojson` can't be loaded/parsed or the `--feature-id-key` matches nothing; returns
/// `Ok(None)` only when valid input yields fewer than 2 regions with points.
fn build_smart_pip_choropleth_panel(
    geojson_spec: &str,
    feature_id_key: &str,
    feature_name_key: Option<&str>,
    lats: &[f64],
    lons: &[f64],
    snap: bool,
    snap_max_km: Option<f64>,
    coord_decimals: Option<u32>,
    loaded_geojson: Option<&serde_json::Value>,
) -> CliResult<Option<Panel>> {
    // an explicit --geojson is explicit intent: a missing file, bad GeoJSON, or wrong
    // --feature-id-key is a hard error, not a silently-dropped panel. `Ok(None)` is reserved for
    // valid inputs that simply don't yield enough regions (the <2 check below).
    let owned;
    let geojson = match loaded_geojson {
        Some(v) => v,
        None => {
            owned = load_geojson(geojson_spec)?;
            &owned
        },
    };
    let features = build_pip_features(geojson, feature_id_key, feature_name_key)?;
    let cap = resolve_snap_cap(snap_max_km, &features, coord_decimals);
    let mut order: Vec<String> = Vec::new();
    let mut counts: std::collections::HashMap<String, f64> = std::collections::HashMap::new();
    // per-region tally of snapped-in points (keyed by feature id), surfaced in the region's hover.
    let mut snapped_by_id: std::collections::HashMap<String, usize> =
        std::collections::HashMap::new();
    let mut total = 0_usize;
    let mut snapped = 0_usize;
    let mut dropped = 0_usize;
    for (&lat, &lon) in lats.iter().zip(lons.iter()) {
        if !((-90.0..=90.0).contains(&lat) && (-180.0..=180.0).contains(&lon)) {
            continue;
        }
        total += 1;
        // smart dashboards resolve the snap-distance cap exactly like the `viz choropleth`
        // command: an outside point snaps only to a region within the cap, else it is dropped.
        let fi = match pip_assign(&features, lat, lon, snap, cap.km) {
            PipOutcome::Inside(fi) => fi,
            PipOutcome::Snapped(fi) => {
                *snapped_by_id.entry(features[fi].id.clone()).or_insert(0) += 1;
                snapped += 1;
                fi
            },
            PipOutcome::Outside => {
                dropped += 1;
                continue;
            },
        };
        let id = &features[fi].id;
        if let Some(c) = counts.get_mut(id) {
            *c += 1.0;
        } else {
            counts.insert(id.clone(), 1.0);
            order.push(id.clone());
        }
    }
    if order.len() < 2 {
        // Total mismatch is the one case worth reporting even though no panel renders: it is not
        // "valid data that happens to yield <2 regions", it means the boundary file does not
        // cover the points at all (wrong --geojson, or lat/lon swapped so everything lands in the
        // ocean). Without this the run is completely silent — the boundary overlay still draws,
        // so the Data Schematic shows outlined regions with no fill and no explanation.
        if total > 0 && dropped == total {
            viz_note(&format!(
                "viz smart: none of the {total} points fell inside any --geojson region, so no \
                 region panel was built. Check that the boundary file covers this data and that \
                 --lat/--lon are not swapped."
            ));
        }
        return Ok(None);
    }
    // only report after we know a panel will actually render, so the note never describes a map the
    // Data Schematic drops.
    if snapped > 0 {
        viz_note(&format!(
            "viz smart: {snapped} of {total} points were snapped to the nearest region (cap {} \
             km, {}).",
            fmt_measure(cap.km),
            cap.basis_phrase_en()
        ));
    }
    if dropped > 0 {
        let why = if snap {
            format!("no region within {} km", fmt_measure(cap.km))
        } else {
            "--no-snap".to_string()
        };
        viz_note(&format!(
            "viz smart: {dropped} of {total} points fell outside every region and were dropped \
             ({why})."
        ));
    }
    let z: Vec<f64> = order.iter().map(|key| counts[key]).collect();
    let names = aligned_region_names(&features, &order);
    let mut hover_text =
        choropleth_hover_text(&order, &z, names.as_deref(), &t!("viz.chart.count"), true);
    // flag each region's snapped-in points as a subset of its count (matches the command path).
    for (h, loc) in hover_text.iter_mut().zip(&order) {
        if let Some(&c) = snapped_by_id.get(loc)
            && c > 0
        {
            h.push_str(&format!("<br>{}", t!("viz.hover.snap_includes", q_n = c)));
        }
    }
    // a sub-panel can't carry a below-map annotation, so surface the snap metadata (count + the
    // cap applied) and dropped points in the panel title — beside the map, never over it.
    let mut title_parts: Vec<String> = Vec::new();
    if snapped > 0 {
        title_parts.push(
            t!(
                "viz.title.snap_snapped",
                q_n = HumanCount(snapped as u64).to_string(),
                q_km = fmt_measure(cap.km)
            )
            .into_owned(),
        );
    }
    if dropped > 0 {
        let why = if snap {
            t!("viz.title.snap_why_over", q_km = fmt_measure(cap.km)).into_owned()
        } else {
            "--no-snap".to_string()
        };
        title_parts.push(
            t!(
                "viz.title.snap_dropped",
                q_n = HumanCount(dropped as u64).to_string(),
                q_total = HumanCount(total as u64).to_string(),
                q_why = why
            )
            .into_owned(),
        );
    }
    let panel_name = if title_parts.is_empty() {
        t!("viz.title.regions").into_owned()
    } else {
        t!("viz.title.regions_with", q_parts = title_parts.join("; ")).into_owned()
    };
    // Frame the basemap to the matched regions' bbox union (PipFeature::bbox is
    // [min_lon, min_lat, max_lon, max_lat]). A metro-scale extent — under
    // SMART_CHOROPLETH_MIN_SPAN_DEG in BOTH dimensions, the inverse of the country-scale threshold
    // build_smart_choropleth_panel uses — renders on a MapLibre tile basemap (fine street/coastline
    // detail) instead of the projection `geo` basemap (whose coastlines are too coarse at city
    // scale). Continental/global extents stay on the projection basemap (offline, no tile fetches).
    let matched: std::collections::HashSet<&str> = order.iter().map(String::as_str).collect();
    let mut min_lon = f64::INFINITY;
    let mut min_lat = f64::INFINITY;
    let mut max_lon = f64::NEG_INFINITY;
    let mut max_lat = f64::NEG_INFINITY;
    // a matched feature crossing the ±180° seam stores its bbox in a [0, 360) frame, so a raw
    // min/max union would be meaningless; fall back to the projection basemap in that case.
    let mut any_wrap = false;
    for f in &features {
        if matched.contains(f.id.as_str()) {
            min_lon = min_lon.min(f.bbox[0]);
            min_lat = min_lat.min(f.bbox[1]);
            max_lon = max_lon.max(f.bbox[2]);
            max_lat = max_lat.max(f.bbox[3]);
            any_wrap |= f.wraps_antimeridian;
        }
    }
    let lon_span = max_lon - min_lon;
    let lat_span = max_lat - min_lat;
    let use_tiles = !any_wrap
        && lon_span < SMART_CHOROPLETH_MIN_SPAN_DEG
        && lat_span < SMART_CHOROPLETH_MIN_SPAN_DEG;

    // the panel embeds the geometry, so it needs an owned document here. This clone replaces
    // what used to be a whole extra fetch+parse of the same file, and is the only copy made.
    let kind = if use_tiles {
        PanelKind::ChoroplethMap {
            locations: order,
            z,
            geojson: geojson.clone(),
            feature_id_key: feature_id_key.to_string(),
            hover_text,
            center_lon: (min_lon + max_lon) / 2.0,
            center_lat: mercator_lat_center(min_lat, max_lat),
            zoom: f64::from(fitbounds_zoom(
                min_lat,
                max_lat,
                lon_span,
                MAP_PANEL_ASSUMED_WIDTH_PX,
                MAP_PANEL_USABLE_HEIGHT_PX,
            )),
            measure_label: t!("viz.chart.count").into_owned(),
            region_filter: None,
        }
    } else {
        PanelKind::Choropleth {
            locations: order,
            z,
            location_mode: LocationMode::GeoJsonId,
            geojson: Some(geojson.clone()),
            feature_id_key: Some(feature_id_key.to_string()),
            hover_text,
            measure_label: t!("viz.chart.count").into_owned(),
            region_filter: None,
        }
    };
    Ok(Some(Panel::new(panel_name, kind)))
}

/// Resolve a region-code cell value to the matching GeoJSON feature id, or `None` when it matches
/// no boundary. Trims first, then prefers an exact match ("15129" -> "15129", "BAKERSTOWN" stays
/// as-is). Failing that, a numeric code that reads shorter than its zero-padded feature id (e.g. a
/// state FIPS `1` vs id `01`, a county FIPS `3` vs `003`, or a zip `7936` vs `07936`) is
/// left-padded to each numeric feature-id width in `numeric_id_widths` wider than the code —
/// shortest viable width first — and the first padded form found is returned. This keeps the
/// matcher correct for the broader geo concepts the summary choropleth accepts (zip/postal, county,
/// state, `census_tract`, fips), not just 5-digit zips. `numeric_id_widths` must be the ascending,
/// deduped digit-widths of the all-numeric feature ids, precomputed once by the caller so this
/// stays O(widths) per cell rather than rescanning every feature id. Used by
/// [`build_smart_summary_choropleth_panels`].
fn match_region_code(
    raw: &str,
    feature_ids: &std::collections::HashSet<&str>,
    numeric_id_widths: &[usize],
    lowercased_ids: &std::collections::HashMap<String, String>,
) -> Option<String> {
    let raw = raw.trim();
    if raw.is_empty() {
        return None;
    }
    // Exact match wins (already-padded zips, alphanumeric ids, etc.).
    if feature_ids.contains(raw) {
        return Some(raw.to_string());
    }
    // For an all-digit code that reads shorter than the zero-padded feature ids (numeric zip/FIPS
    // columns drop leading zeros), try padding to each numeric feature-id width wider than the
    // code.
    if raw.bytes().all(|b| b.is_ascii_digit()) {
        for &width in numeric_id_widths {
            if width <= raw.len() {
                continue;
            }
            let padded = format!("{raw:0>width$}");
            if feature_ids.contains(padded.as_str()) {
                return Some(padded);
            }
        }
    }
    // Last resort: ASCII-case-insensitive. Alphabetic region codes routinely differ in case
    // between a CSV column and the boundary file ("Ca"/"ca" vs STUSPS "CA"), which otherwise
    // matched nothing at all and silently cost the panel. Canonicalize to the FEATURE's spelling
    // so the returned key still indexes the GeoJSON.
    lowercased_ids.get(&raw.to_ascii_lowercase()).cloned()
}

/// Build `viz smart` summary choropleth panel(s) keyed off a region-code DIMENSION column (e.g. a
/// `zip_code` column) matched against a user `--geojson` boundary file via `--feature-id-key`,
/// independent of any lat/lon columns. Emits up to two panels: one colored by the per-region row
/// count (always), and — when the dictionary tags a measure column (a `MAP_MEASURE_CONCEPTS`
/// concept, else any `role=measure` column) — one colored by the per-region MEDIAN of that measure
/// (median, not mean, since real-world measures like sale price are heavily right-skewed).
///
/// The key column is auto-chosen: in one pass, every geo region-code candidate column's trimmed
/// cell values are matched against the GeoJSON feature-id set; the column whose matched-row
/// fraction is highest (and clears `BIVARIATE_MIN_SUPPORT_RATIO`) wins, tie-breaking on the lowest
/// column index. Numeric zip/fips codes read as "15129" match string feature ids "15129"; short
/// all-digit codes are left-padded to 5 so "7936" matches a zero-padded "07936".
///
/// Returns `Ok(None)` when there is no `--geojson`, no geo region-code candidate, no candidate
/// clears the overlap threshold, or the winner yields fewer than 2 matched regions. An unloadable
/// `--geojson` or a `--feature-id-key` matching nothing stays a hard error (explicit intent).
/// The returned `usize` is the winning region column's index, so the caller can suppress its
/// redundant frequency bar (via `is_map_col`).
fn build_smart_summary_choropleth_panels(
    args: &Args,
    stats: &[crate::cmd::stats::StatsData],
    col_sems: &[ColSemantics],
    loaded_geojson: Option<&serde_json::Value>,
) -> CliResult<Option<(Vec<Panel>, usize)>> {
    // explicit intent: only build when the user supplied a --geojson boundary file.
    let Some(spec) = args.flag_geojson.as_deref() else {
        return Ok(None);
    };

    // region-code candidate columns: geo dimensions that NAME a boundary region (zip, county,
    // state, ...), excluding point coordinates and address/city fields that don't key polygons.
    // Canonical describegpt geo leaves plus lenient aliases for hand-curated dictionaries
    // ("zip"/"postal_code" for the canonical "zip_code", "fips" for a county/tract code).
    const REGION_CODE_LEAVES: &[&str] = &[
        "zip_code",
        "zip",
        "postal_code",
        "census_tract",
        "county",
        "county_fips",
        "state",
        "state_fips",
        "country",
        "fips",
    ];
    let candidates: Vec<usize> = col_sems
        .iter()
        .enumerate()
        .filter(|(i, s)| {
            stats.get(*i).is_some_and(|st| st.cardinality >= 2)
                && s.concept
                    .strip_prefix("geo.")
                    .is_some_and(|leaf| REGION_CODE_LEAVES.contains(&leaf))
        })
        .map(|(i, _)| i)
        .collect();
    if candidates.is_empty() {
        return Ok(None);
    }

    // load + validate the boundary file once (already resolved/validated in `run`); build the
    // feature-id set the candidate columns are matched against.
    let owned;
    let geojson = match loaded_geojson {
        Some(v) => v,
        None => {
            owned = load_geojson(spec)?;
            &owned
        },
    };
    let feature_id_key = args.flag_feature_id_key.as_deref().unwrap_or("id");
    let features = build_pip_features(
        geojson,
        feature_id_key,
        args.flag_feature_name_key.as_deref(),
    )?;
    let feature_ids: std::collections::HashSet<&str> =
        features.iter().map(|f| f.id.as_str()).collect();
    // distinct widths of the all-digit feature ids, ascending — precomputed once so the per-row
    // `match_region_code` does not rescan every feature id for each unmatched numeric cell (which
    // would make matching `rows * candidates * features`).
    let numeric_id_widths: Vec<usize> = {
        let mut w: Vec<usize> = feature_ids
            .iter()
            .filter(|id| !id.is_empty() && id.bytes().all(|b| b.is_ascii_digit()))
            .map(|id| id.len())
            .collect();
        w.sort_unstable();
        w.dedup();
        w
    };
    // lower-cased id -> canonical feature id, for the case-insensitive fallback in
    // `match_region_code`. Built from the ORDERED feature list, not the HashSet: iterating the set
    // would pick an arbitrary winner when two ids differ only by case, so the same input could
    // aggregate under a different region between runs. A folded key that maps to more than one
    // distinct id is genuinely ambiguous, so it is REMOVED rather than resolved — exact and
    // zero-padded matching still run first, so this only forfeits the fuzzy fallback.
    let lowercased_ids: std::collections::HashMap<String, String> = {
        let mut m: std::collections::HashMap<String, String> = std::collections::HashMap::new();
        let mut ambiguous: std::collections::HashSet<String> = std::collections::HashSet::new();
        for f in &features {
            let folded = f.id.to_ascii_lowercase();
            match m.get(&folded) {
                Some(existing) if *existing != f.id => {
                    ambiguous.insert(folded);
                },
                Some(_) => {},
                None => {
                    m.insert(folded, f.id.clone());
                },
            }
        }
        for k in ambiguous {
            m.remove(&k);
        }
        m
    };

    // optional measure column to color the second panel: prefer a MAP_MEASURE_CONCEPTS concept,
    // else any `role=measure` column (catches an untagged-concept amount like PRICE whose
    // dictionary role is still "measure"). Never a candidate region column.
    let measure_idx: Option<usize> = first_col_by_concepts(
        col_sems,
        MAP_MEASURE_CONCEPTS,
        usize::MAX,
        usize::MAX,
        &candidates,
    )
    .or_else(|| {
        col_sems
            .iter()
            .enumerate()
            .find(|(i, s)| s.route == Route::Measure && !candidates.contains(i))
            .map(|(i, _)| i)
    });

    // one pass over the rows: for every candidate column tally matched-row count and (per matched
    // region id) the row count and the measure values, plus the matched/total ratio used to pick
    // the key column.
    let n_c = candidates.len();
    let mut order: Vec<Vec<String>> = vec![Vec::new(); n_c];
    let mut counts: Vec<HashMap<String, f64>> = vec![HashMap::new(); n_c];
    let mut values: Vec<HashMap<String, Vec<f64>>> = vec![HashMap::new(); n_c];
    let mut matched_rows: Vec<u64> = vec![0; n_c];
    let mut total_rows: Vec<u64> = vec![0; n_c];
    // per matched feature id, the raw cell spellings that differ from it (zero-padded zips,
    // case-folded codes) — the region-click filter searches the data viewer by RAW value, so it
    // needs the reverse of what `match_region_code` normalized away. Identity matches are
    // omitted to keep the map sparse; the JS side UNIONS the feature id back in (a column can
    // hold both spellings of the same region, so neither list alone is sufficient).
    let mut raws_by_cand: Vec<HashMap<String, Vec<String>>> = vec![HashMap::new(); n_c];

    let (mut rdr, _headers, _nh) = reader_and_headers(args)?;
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        let measure = measure_idx.and_then(|mi| parse_f64(record.get(mi)));
        for (ci, &di) in candidates.iter().enumerate() {
            let cell = cell_to_string(record.get(di));
            let raw = cell.trim();
            if raw.is_empty() {
                continue;
            }
            total_rows[ci] += 1;
            // key aggregates under the matched geojson feature id; an unmatched value still counts
            // toward total_rows so a poorly-overlapping column scores low.
            let Some(k) = match_region_code(raw, &feature_ids, &numeric_id_widths, &lowercased_ids)
            else {
                continue;
            };
            matched_rows[ci] += 1;
            if raw != k {
                let variants = raws_by_cand[ci].entry(k.clone()).or_default();
                if !variants.iter().any(|v| v == raw) {
                    variants.push(raw.to_string());
                }
            }
            match counts[ci].get_mut(&k) {
                Some(c) => *c += 1.0,
                None => {
                    counts[ci].insert(k.clone(), 1.0);
                    order[ci].push(k.clone());
                },
            }
            if let Some(v) = measure {
                values[ci].entry(k).or_default().push(v);
            }
        }
    }

    // pick the candidate with the highest matched-row fraction (tie-break lowest column index,
    // preserved by only replacing on a strictly greater ratio while scanning ascending).
    let mut best: Option<(f64, usize)> = None; // (ratio, candidate slot)
    for ci in 0..n_c {
        if total_rows[ci] == 0 {
            continue;
        }
        let ratio = matched_rows[ci] as f64 / total_rows[ci] as f64;
        if best.is_none_or(|(b, _)| ratio > b) {
            best = Some((ratio, ci));
        }
    }
    let Some((ratio, ci)) = best else {
        return Ok(None);
    };
    if ratio < BIVARIATE_MIN_SUPPORT_RATIO {
        viz_note(&format!(
            "viz smart: --geojson supplied but no region-code column overlapped the boundary ids \
             (best {:.0}%); skipping the summary choropleth.",
            ratio * 100.0
        ));
        return Ok(None);
    }
    let region_idx = candidates[ci];
    let region_raws = std::mem::take(&mut raws_by_cand[ci]);

    // resolve a human label for a column: dictionary label, else header, else positional.
    let label_of = |idx: usize| {
        let s = &col_sems[idx];
        if !s.label.is_empty() {
            s.label.clone()
        } else if stats[idx].field.is_empty() {
            format!("col {}", idx + 1)
        } else {
            stats[idx].field.clone()
        }
    };
    let region_label = label_of(region_idx);

    // shared assembly: frame to the matched-region bbox union, choose a MapLibre tile basemap for a
    // metro-scale extent (fine street/coastline detail) or the projection `geo` basemap otherwise —
    // mirroring `build_smart_pip_choropleth_panel`'s tail.
    let make_panel = |locs: Vec<String>,
                      z: Vec<f64>,
                      measure_label: String,
                      include_pct: bool,
                      title: String| {
        let names = aligned_region_names(&features, &locs);
        let hover_text =
            choropleth_hover_text(&locs, &z, names.as_deref(), &measure_label, include_pct);
        let matched: std::collections::HashSet<&str> = locs.iter().map(String::as_str).collect();
        let mut min_lon = f64::INFINITY;
        let mut min_lat = f64::INFINITY;
        let mut max_lon = f64::NEG_INFINITY;
        let mut max_lat = f64::NEG_INFINITY;
        let mut any_wrap = false;
        for f in &features {
            if matched.contains(f.id.as_str()) {
                min_lon = min_lon.min(f.bbox[0]);
                min_lat = min_lat.min(f.bbox[1]);
                max_lon = max_lon.max(f.bbox[2]);
                max_lat = max_lat.max(f.bbox[3]);
                any_wrap |= f.wraps_antimeridian;
            }
        }
        let lon_span = max_lon - min_lon;
        let lat_span = max_lat - min_lat;
        let use_tiles = !any_wrap
            && lon_span < SMART_CHOROPLETH_MIN_SPAN_DEG
            && lat_span < SMART_CHOROPLETH_MIN_SPAN_DEG;
        let kind = if use_tiles {
            PanelKind::ChoroplethMap {
                locations: locs,
                z,
                geojson: geojson.clone(),
                feature_id_key: feature_id_key.to_string(),
                hover_text,
                center_lon: (min_lon + max_lon) / 2.0,
                center_lat: mercator_lat_center(min_lat, max_lat),
                zoom: f64::from(fitbounds_zoom(
                    min_lat,
                    max_lat,
                    lon_span,
                    MAP_PANEL_ASSUMED_WIDTH_PX,
                    MAP_PANEL_USABLE_HEIGHT_PX,
                )),
                measure_label,
                region_filter: Some((region_idx, region_raws.clone())),
            }
        } else {
            PanelKind::Choropleth {
                locations: locs,
                z,
                location_mode: LocationMode::GeoJsonId,
                geojson: Some(geojson.clone()),
                feature_id_key: Some(feature_id_key.to_string()),
                hover_text,
                measure_label,
                region_filter: Some((region_idx, region_raws.clone())),
            }
        };
        Panel::new(title, kind)
    };

    // count panel (always): per-region row count over the matched regions, first-seen order.
    let count_locs = order[ci].clone();
    if count_locs.len() < 2 {
        return Ok(None);
    }
    let count_z: Vec<f64> = count_locs.iter().map(|k| counts[ci][k]).collect();
    let mut out = vec![make_panel(
        count_locs,
        count_z,
        t!("viz.chart.count").into_owned(),
        true,
        t!("viz.title.count_by_region", q_region = region_label).into_owned(),
    )];

    // median-measure panel (when a measure column exists): per-region median of the buffered
    // values, skipping regions with no parsed measure.
    if let Some(mi) = measure_idx {
        let measure_name = label_of(mi);
        let mut med_locs: Vec<String> = Vec::new();
        let mut med_z: Vec<f64> = Vec::new();
        for k in &order[ci] {
            let Some(vs) = values[ci].get_mut(k) else {
                continue;
            };
            if vs.is_empty() {
                continue;
            }
            vs.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
            let m = vs.len() / 2;
            let median = if vs.len() % 2 == 1 {
                vs[m]
            } else {
                (vs[m - 1] + vs[m]) / 2.0
            };
            med_locs.push(k.clone());
            med_z.push(median);
        }
        if med_locs.len() >= 2 {
            out.push(make_panel(
                med_locs,
                med_z,
                format!("median {measure_name}"),
                false,
                format!("median {measure_name} by {region_label}"),
            ));
        }
    }

    Ok(Some((out, region_idx)))
}

/// Max `--geojson` features to label on a map overlay. Above this, the region boundaries are still
/// drawn but the per-region text labels are suppressed to avoid a cluttered map.
const GEOJSON_OVERLAY_LABEL_MAX: usize = 60;

/// Remote `--geojson` fetch guards. reqwest's default blocking client has no timeout at all and
/// `Response::bytes()` buffers the whole body unchecked, so a stalling server hangs qsv forever
/// and an endless body drives it to OOM. The timeout is a `util::timeout_secs` default (so
/// `QSV_TIMEOUT` overrides it); the size cap is a safety CEILING, not a target — real boundary
/// files (e.g. census tracts) legitimately run to a few hundred MB.
const GEOJSON_FETCH_TIMEOUT_SECS: u16 = 30;
const GEOJSON_MAX_BYTES: usize = 512_000_000;

/// Outline color for `--geojson` region boundaries overlaid on a smart map. A teal, distinct from
/// the warm data points, the purple core-extent box, and the magenta full-extent box.
const GEOJSON_OVERLAY_LINE_COLOR: &str = "#00695c";

/// Outline width (px) for the `--geojson` region boundary overlay.
const GEOJSON_OVERLAY_LINE_WIDTH: f64 = 1.5;

/// Region boundaries (as one gap-separated polyline) plus per-feature labels for overlaying a
/// user `--geojson` on a `viz smart` map panel. Built once in `build_map_panel`, rendered by
/// [`geojson_overlay_map_traces`] / [`geojson_overlay_geo_traces`] on both the map and the
/// offline `ScatterGeo` (global / static-export) paths.
struct GeoJsonOverlay {
    /// All feature ring vertices; a `NaN` gap separates consecutive rings so plotly breaks the
    /// line there and a single trace draws every boundary.
    boundary_lats: Vec<f64>,
    boundary_lons: Vec<f64>,
    /// Per-feature label anchors (a representative interior point; see [`feature_label_anchor`])
    /// and text (HTML-escaped). Empty when there are more than `GEOJSON_OVERLAY_LABEL_MAX`
    /// features.
    label_lats:    Vec<f64>,
    label_lons:    Vec<f64>,
    labels:        Vec<String>,
}

/// Absolute planar (shoelace) area of a polygon ring, in squared degrees. Used only to pick the
/// largest part of a `MultiPolygon` for label placement, so the planar/`deg²` approximation is fine
/// at the local/regional scales this overlay targets. Rings may be open or closed (first==last).
fn ring_area(ring: &[[f64; 2]]) -> f64 {
    let n = ring.len();
    if n < 3 {
        return 0.0;
    }
    let mut a = 0.0;
    let mut j = n - 1;
    for i in 0..n {
        let [xi, yi] = ring[i];
        let [xj, yj] = ring[j];
        a += (xj + xi) * (yj - yi);
        j = i;
    }
    (a / 2.0).abs()
}

/// Area-weighted centroid (shoelace) of a polygon ring, as `[lon, lat]`. `None` when the ring is
/// degenerate (near-zero area). Rings may be open or closed.
fn ring_centroid(ring: &[[f64; 2]]) -> Option<[f64; 2]> {
    let n = ring.len();
    if n < 3 {
        return None;
    }
    let (mut a2, mut cx, mut cy) = (0.0, 0.0, 0.0);
    let mut j = n - 1;
    for i in 0..n {
        let [xi, yi] = ring[i];
        let [xj, yj] = ring[j];
        let cross = xj * yi - xi * yj;
        a2 += cross;
        cx += (xj + xi) * cross;
        cy += (yj + yi) * cross;
        j = i;
    }
    if a2.abs() < 1e-12 {
        return None;
    }
    Some([cx / (3.0 * a2), cy / (3.0 * a2)])
}

/// Point guaranteed on the polygon's surface at latitude `y`: intersect the horizontal line with
/// every ring edge, sort the crossings, and return the midpoint of the widest interior (even-odd)
/// span as `[lon, lat]`. `None` when the line misses the polygon entirely.
fn point_on_surface(rings: &[Vec<[f64; 2]>], y: f64) -> Option<[f64; 2]> {
    let mut xs: Vec<f64> = Vec::new();
    for ring in rings {
        let n = ring.len();
        if n < 3 {
            continue;
        }
        let mut j = n - 1;
        for i in 0..n {
            let [xi, yi] = ring[i];
            let [xj, yj] = ring[j];
            if (yi > y) != (yj > y) {
                xs.push((xj - xi) * (y - yi) / (yj - yi) + xi);
            }
            j = i;
        }
    }
    xs.sort_by(f64::total_cmp);
    let mut best: Option<f64> = None;
    let mut best_w = f64::NEG_INFINITY;
    let mut k = 0;
    while k + 1 < xs.len() {
        let w = xs[k + 1] - xs[k];
        if w > best_w {
            best_w = w;
            best = Some((xs[k] + xs[k + 1]) / 2.0);
        }
        k += 2;
    }
    best.map(|x| [x, y])
}

/// A representative interior point (`[lon, lat]`) of a single polygon (rings: `[0]` exterior, rest
/// holes): the area-weighted centroid when it lands inside, otherwise a point-on-surface at the
/// centroid's latitude (falling back to the ring's vertical midline). `None` for a degenerate ring.
fn representative_point(rings: &[Vec<[f64; 2]>]) -> Option<[f64; 2]> {
    let exterior = rings.first()?;
    let centroid = ring_centroid(exterior)?;
    if point_in_polygon(rings, centroid[0], centroid[1]) {
        return Some(centroid);
    }
    // concave or holed: the centroid fell outside — fall back to an on-surface scanline point.
    let mid_y = {
        let (mut lo, mut hi) = (f64::INFINITY, f64::NEG_INFINITY);
        for &[_, y] in exterior {
            lo = lo.min(y);
            hi = hi.max(y);
        }
        (lo + hi) / 2.0
    };
    point_on_surface(rings, centroid[1]).or_else(|| point_on_surface(rings, mid_y))
}

/// Label anchor (`[lon, lat]`) for a `--geojson` feature: a representative interior point of its
/// LARGEST sub-polygon (so a `MultiPolygon` labels its main part, not the empty gap between parts),
/// falling back to the whole-feature bounding-box center when the geometry is degenerate.
fn feature_label_anchor(feature: &PipFeature) -> [f64; 2] {
    let largest = feature
        .polygons
        .iter()
        .filter(|poly| poly.first().is_some_and(|r| r.len() >= 3))
        .max_by(|a, b| ring_area(&a[0]).total_cmp(&ring_area(&b[0])));
    if let Some(poly) = largest
        && let Some(pt) = representative_point(poly)
    {
        return pt;
    }
    let [min_lon, min_lat, max_lon, max_lat] = feature.bbox;
    [(min_lon + max_lon) / 2.0, (min_lat + max_lat) / 2.0]
}

/// Build the [`GeoJsonOverlay`] for a `--geojson` map overlay: every feature's boundary rings as
/// one `NaN`-gap-separated polyline, plus a per-feature label (its `--feature-name-key` value,
/// falling back to the id) at a representative interior point (see [`feature_label_anchor`]) —
/// suppressed above `GEOJSON_OVERLAY_LABEL_MAX` features. Best-effort: returns `None` when the file
/// can't be loaded/parsed (the choropleth-panel builder already hard-errors on a bad `--geojson`
/// before this runs) or yields no drawable rings. Antimeridian-crossing features are drawn from
/// their stored (continuity-unwrapped) vertices, so a region straddling ±180° may draw imperfectly
/// — a non-issue for the local/regional GeoJSON this overlay targets.
fn build_geojson_overlay(
    spec: &str,
    feature_id_key: &str,
    feature_name_key: Option<&str>,
    loaded_geojson: Option<&serde_json::Value>,
) -> Option<GeoJsonOverlay> {
    let owned;
    let geojson = match loaded_geojson {
        Some(v) => v,
        None => {
            owned = load_geojson(spec).ok()?;
            &owned
        },
    };
    let features = build_pip_features(geojson, feature_id_key, feature_name_key).ok()?;
    if features.is_empty() {
        return None;
    }
    let label_them = features.len() <= GEOJSON_OVERLAY_LABEL_MAX;
    let mut boundary_lats: Vec<f64> = Vec::new();
    let mut boundary_lons: Vec<f64> = Vec::new();
    let mut label_lats: Vec<f64> = Vec::new();
    let mut label_lons: Vec<f64> = Vec::new();
    let mut labels: Vec<String> = Vec::new();
    for feature in &features {
        for polygon in &feature.polygons {
            for ring in polygon {
                if ring.len() < 2 {
                    continue;
                }
                if !boundary_lats.is_empty() {
                    // gap so plotly breaks the line between this ring and the previous one
                    boundary_lats.push(f64::NAN);
                    boundary_lons.push(f64::NAN);
                }
                for &[lon, lat] in ring {
                    boundary_lats.push(lat);
                    boundary_lons.push(lon);
                }
            }
        }
        if label_them {
            let [lon, lat] = feature_label_anchor(feature);
            label_lons.push(lon);
            label_lats.push(lat);
            // `name` is HTML-escaped at construction; the id fallback is raw, so escape it here.
            labels.push(
                feature
                    .name
                    .clone()
                    .unwrap_or_else(|| escape_hover(&feature.id)),
            );
        }
    }
    if boundary_lats.is_empty() {
        return None;
    }
    Some(GeoJsonOverlay {
        boundary_lats,
        boundary_lons,
        label_lats,
        label_lons,
        labels,
    })
}

/// Shared line style for the `--geojson` region boundary overlay.
fn geojson_overlay_line() -> Line {
    Line::new()
        .color(GEOJSON_OVERLAY_LINE_COLOR)
        .width(GEOJSON_OVERLAY_LINE_WIDTH)
}

/// Shared text font for the on-map `--geojson` region labels on the `ScatterGeo` projection
/// (global / static export), where text glyphs render reliably: small, in the overlay's teal.
fn geojson_overlay_label_font() -> Font {
    Font::new()
        .family(FONT_FAMILY)
        .size(10)
        .color(GEOJSON_OVERLAY_LINE_COLOR)
}

/// Small centroid marker carrying each region's name as hover text on the MapLibre tile map. Raster
/// basemaps cull colliding on-map text glyphs (the same reason the extent corner points use
/// haloed circles, not text), so the region label is delivered on hover of this dot instead — a
/// white-haloed teal circle that reads on both light and dark basemaps.
/// Markers for the row pin (see `MAP_ROW_PIN_NAME`) — the magenta core and the white disc behind
/// it whose exposed rim forms the ring.
fn row_pin_marker() -> Marker {
    Marker::new()
        .color(MAP_SELECTED_POINT_COLOR)
        .size(MAP_ROW_PIN_SIZE)
        .opacity(1.0)
}

fn row_pin_halo_marker() -> Marker {
    Marker::new()
        .color("#ffffff")
        .size(MAP_ROW_PIN_HALO_SIZE)
        .opacity(1.0)
}

fn geojson_overlay_label_marker() -> Marker {
    Marker::new()
        .color(GEOJSON_OVERLAY_LINE_COLOR)
        .size(7)
        .opacity(0.9)
        .line(Line::new().color("#ffffff").width(1.0))
}

/// Boundary + label traces for a `--geojson` overlay on a MapLibre tile map (`ScatterMap`). The
/// boundaries are one gap-separated line trace; the labels are centroid hover-markers (the raster
/// basemap culls colliding on-map text, so the region name is shown on hover of a small dot
/// instead).
fn geojson_overlay_map_traces(overlay: &GeoJsonOverlay) -> Vec<Box<dyn Trace>> {
    let mut out: Vec<Box<dyn Trace>> = Vec::with_capacity(2);
    let boundary: Box<dyn Trace> =
        ScatterMap::new(overlay.boundary_lats.clone(), overlay.boundary_lons.clone())
            .name("regions")
            .mode(Mode::Lines)
            .line(geojson_overlay_line())
            .hover_info(HoverInfo::Skip)
            .show_legend(false);
    out.push(boundary);
    if !overlay.labels.is_empty() {
        let label_trace: Box<dyn Trace> =
            ScatterMap::new(overlay.label_lats.clone(), overlay.label_lons.clone())
                .name("region labels")
                .mode(Mode::Markers)
                .marker(geojson_overlay_label_marker())
                .hover_text_array(overlay.labels.clone())
                .hover_info(HoverInfo::Text)
                .show_legend(false);
        out.push(label_trace);
    }
    out
}

/// Boundary + label traces for a `--geojson` overlay on an offline `ScatterGeo` projection (global
/// HTML + static image export). `subplot` binds each trace to a specific `geo{n}` cell when set
/// (the typed-subplot / static path); pass `None` for the single-panel inline plot.
fn geojson_overlay_geo_traces(
    overlay: &GeoJsonOverlay,
    subplot: Option<&str>,
) -> Vec<Box<dyn Trace>> {
    let mut out: Vec<Box<dyn Trace>> = Vec::with_capacity(2);
    let mut boundary =
        ScatterGeo::new(overlay.boundary_lats.clone(), overlay.boundary_lons.clone())
            .name("regions")
            .mode(Mode::Lines)
            .line(geojson_overlay_line())
            .hover_info(HoverInfo::Skip)
            .show_legend(false);
    if let Some(sp) = subplot {
        boundary = boundary.subplot(sp);
    }
    let boundary: Box<dyn Trace> = boundary;
    out.push(boundary);
    if !overlay.labels.is_empty() {
        let mut label_trace =
            ScatterGeo::new(overlay.label_lats.clone(), overlay.label_lons.clone())
                .name("region labels")
                .mode(Mode::Text)
                .text_array(overlay.labels.clone())
                .text_font(geojson_overlay_label_font())
                .hover_info(HoverInfo::Skip)
                .show_legend(false);
        if let Some(sp) = subplot {
            label_trace = label_trace.subplot(sp);
        }
        let label_trace: Box<dyn Trace> = label_trace;
        out.push(label_trace);
    }
    out
}

/// Detect a latitude/longitude column pair and, if a usable pair exists, build a `viz smart` map
/// panel. The pair comes from `coord_hint` (dictionary `geo.latitude`/`geo.longitude` signals) when
/// supplied, else the header-name heuristic (`latlon_indices`). Does one extra data pass to collect
/// the in-range coordinates (the stats cache holds no geometry) and, per point, a hover label: the
/// row's identifier and — under `--smarter` — a dictionary/stats-chosen field combination (see
/// `select_map_hover_fields`); when the `geocode` feature is available each point is also reverse-
/// geocoded so the hover names the resolved place (City, Admin1, Country), with dataset fields
/// winning over duplicate geocoded components. Returns `None` when no pair is found, the columns
/// aren't numeric, or no row has valid coordinates. On success returns the panel together with the
/// (lat, lon) column indices it consumed, so the caller can exclude exactly those columns from the
/// other panels — and only when a map is actually rendered. The trailing `usize` is the full
/// mappable point count (before outlier split / downsampling) that drove the density decision, so
/// the caller's heatmap note reports the source count rather than the downsampled rendered count.
/// Without a dictionary hint, name detection needs headers, so this is a no-op under
/// `--no-headers`.
fn build_map_panel(
    args: &Args,
    stats: &[crate::cmd::stats::StatsData],
    col_sems: &[ColSemantics],
    dict: Option<&DictData>,
    coord_hint: Option<(usize, usize)>,
    cluster_mode: ClusterMode,
    loaded_geojson: Option<&serde_json::Value>,
    // Emit each plotted point's 0-based DATA-row ordinal as the trace's `ids`, so the data viewer
    // drawer can cross-link rows and map points. Off for image export (no JS) and when the drawer
    // is disabled (`--preview-threshold 0`), which keeps those payloads byte-identical.
    capture_row_ids: bool,
) -> CliResult<Option<(Panel, Option<Panel>, (usize, usize), usize)>> {
    let Some((lat_idx, lon_idx)) = coord_hint.or_else(|| latlon_indices(stats)) else {
        return Ok(None);
    };
    // Rows past the drawer's preview prefix have no row to select, so their ordinal is emitted as
    // an empty string rather than a number: same positional alignment, far fewer bytes on a big
    // map, and the bridge can still tell "beyond the preview" (empty entry, worth a note) from
    // "not a selectable trace at all" (no `ids` array — the extent/geojson overlays).
    let row_id_cap = args.flag_preview_threshold as u64;

    // pick the hover fields once: [identifier, extras...] (extras only under --smarter). When this
    // is empty AND no point reverse-geocodes, the trace keeps plotly's default lat/lon hover.
    let hover_field_idxs =
        select_map_hover_fields(stats, col_sems, dict, lat_idx, lon_idx, args.flag_smarter);
    let dataset_has_fields = !hover_field_idxs.is_empty();
    // Resolve the identifier independently — `select_map_hover_fields` can return ONLY `--smarter`
    // extras (measure/category) when no identifier exists, so the first field is not necessarily
    // the id. Treating it as one would bold a measure and (via `has_id`) suppress the geocoded
    // place that should serve as the identifier. Everything that isn't the id becomes an extra.
    let id_idx = select_map_identifier(stats, col_sems, dict, lat_idx, lon_idx);
    let has_id = id_idx.is_some();
    let extra_idxs: Vec<usize> = hover_field_idxs
        .iter()
        .copied()
        .filter(|&i| Some(i) != id_idx)
        .collect();

    // optional bubble-size measure: a dictionary-tagged map measure (amount/count/ratio). `None`
    // without a dictionary (concepts are empty), so non-dictionary maps stay fixed-size markers —
    // sizing by an untagged numeric would be an arbitrary, misleading encoding.
    let size_idx = first_col_by_concepts(col_sems, MAP_MEASURE_CONCEPTS, lat_idx, lon_idx, &[]);

    // opt-in `--photos`: the image-URL columns whose per-row values ride along as `customdata` for
    // the hover-dwell lightbox. Empty unless the flag is set, so a default Data Schematic embeds no
    // off-origin URL and its viewers' browsers request nothing from a third-party image host.
    let photo_idxs: Vec<usize> = if args.flag_photos {
        image_url_cols(stats)
    } else {
        Vec::new()
    };

    let (mut rdr, headers, nh) = reader_and_headers(args)?;
    // friendly labels for the extra fields: dictionary label, else the column header.
    let extra_labels: Vec<String> = extra_idxs
        .iter()
        .map(|&idx| {
            col_sems
                .get(idx)
                .map(|s| s.label.as_str())
                .filter(|l| !l.is_empty())
                .map_or_else(|| col_label(&headers, idx, nh), ToString::to_string)
        })
        .collect();

    let mut lats: Vec<f64> = Vec::new();
    let mut lons: Vec<f64> = Vec::new();
    // dataset portion of each point's hover (bold id + extra fields, NO coords/geo), row-aligned
    // with lats/lons; empty string when the dataset contributes no fields.
    let mut dataset_lines: Vec<String> = Vec::new();
    // raw bubble-size measure per in-range point, row-aligned with lats/lons; left empty unless a
    // map measure was found. A missing/non-numeric value is NaN (drawn at the smallest size).
    let mut sizes_raw: Vec<f64> = Vec::new();
    // `--photos`: this point's image URLs, `|`-joined (the separator the source data already uses
    // for multi-photo cells) and row-aligned with lats/lons. Empty string for a row with no photo,
    // and the whole vector stays empty when the flag is off.
    let mut photo_urls: Vec<String> = Vec::new();
    // 0-based DATA-row ordinal of each kept point, row-aligned with lats/lons. Empty unless
    // `capture_row_ids`. This is the join key to the drawer's DataTable, whose row order is the
    // same single pass over the same Config (see `collect_datatable_rows`).
    let mut row_ids: Vec<u64> = Vec::new();
    let build_dataset_lines = id_idx.is_some() || !extra_idxs.is_empty();
    let mut record = csv::ByteRecord::new();
    // counts EVERY data row read, including ones dropped below for unparseable/out-of-range
    // coordinates — those rows still occupy an ordinal in the drawer, so skipping them here
    // would shift every subsequent point onto the wrong row.
    let mut row_ordinal: u64 = 0;
    while rdr.read_byte_record(&mut record)? {
        let ordinal = row_ordinal;
        row_ordinal += 1;
        let (Some(lat), Some(lon)) = (
            parse_f64(record.get(lat_idx)),
            parse_f64(record.get(lon_idx)),
        ) else {
            continue;
        };
        if (-90.0..=90.0).contains(&lat) && (-180.0..=180.0).contains(&lon) {
            lats.push(lat);
            lons.push(lon);
            if capture_row_ids {
                row_ids.push(ordinal);
            }
            if let Some(si) = size_idx {
                sizes_raw.push(parse_f64(record.get(si)).unwrap_or(f64::NAN));
            }
            if !photo_idxs.is_empty() {
                // one row can carry photos in several columns (Boston 311 has a submitted and a
                // closed photo), and each cell can itself hold several `|`-separated URLs — so
                // flatten both levels into one ordered list for this point's lightbox.
                let urls: Vec<String> = photo_idxs
                    .iter()
                    .filter_map(|&i| record.get(i))
                    .flat_map(|b| split_image_urls(&String::from_utf8_lossy(b)))
                    .collect();
                photo_urls.push(urls.join("|"));
            }
            // with no identifier and no hover fields the formatted line is always empty, so
            // building it allocates a Vec plus a String per row for a result that is discarded.
            // The push still happens so `dataset_lines` stays index-aligned with lats/lons.
            if build_dataset_lines {
                let cell = |i: usize| -> String {
                    record
                        .get(i)
                        .map(|b| String::from_utf8_lossy(b).into_owned())
                        .unwrap_or_default()
                };
                let id_val = id_idx.map(&cell).unwrap_or_default();
                let extra: Vec<(String, String)> = extra_idxs
                    .iter()
                    .zip(&extra_labels)
                    .map(|(&i, label)| (label.clone(), cell(i)))
                    .collect();
                dataset_lines.push(format_map_dataset_line(
                    (!id_val.is_empty()).then_some(id_val.as_str()),
                    &extra,
                ));
            } else {
                dataset_lines.push(String::new());
            }
        }
    }
    if lats.is_empty() {
        return Ok(None);
    }
    // decide density vs. markers from the full row count, then cap the embedded points so a huge
    // dataset doesn't bloat the HTML or freeze the browser on pan/zoom. A `--heatmap-density` of 0
    // disables the heatmap entirely (always render individual markers, whatever the point count).
    // The user-facing heatmap note is emitted by the caller, but only once it's certain a
    // DensityMap will actually render (i.e. a local-extent HTML `Map` panel) — a global extent
    // becomes a `ScatterGeo` world-overview and static image export coerces `Map` -> `Geo`, so
    // neither draws a heatmap despite `density` being true here. Capture the FULL mappable count
    // now — `lats` is shadowed by the downsampled, outlier-excluded core below, so the note must
    // report this source count (the number the threshold was actually compared against).
    let mappable_count = lats.len();
    let density = args.flag_heatmap_density > 0 && mappable_count >= args.flag_heatmap_density;

    // continental/global extents render as an offline `ScatterGeo` projection world-overview
    // (no network tiles, better whole-world context) rather than a zoomed MapLibre tile map.
    // Use the same robust framing semantics as the map view (computed on the full in-range set,
    // before downsampling): an antimeridian-aware, trimmed longitude span and a trimmed latitude
    // span, so a cluster straddling +/-180 or a single far in-range outlier doesn't misclassify a
    // local dataset as global.
    let (_, lon_span) = lon_center_and_span(&lons, MAP_FRAME_TRIM_FRAC);
    let mut lats_sorted = lats.clone();
    lats_sorted.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
    let lat_span = sorted_quantile(&lats_sorted, 1.0 - MAP_FRAME_TRIM_FRAC)
        - sorted_quantile(&lats_sorted, MAP_FRAME_TRIM_FRAC);
    // density/global classification stays on the FULL in-range set (above), so removing outliers
    // doesn't perturb which basemap or render path is chosen.
    let global = lon_span >= SMART_GEO_MIN_LON_SPAN_DEG || lat_span >= SMART_GEO_MIN_LAT_SPAN_DEG;

    // split into the core cluster vs. geographic outliers (points far from the cluster centroid)
    // BEFORE downsampling, so the centroid/distances see the full population and true strays are
    // never stride-sampled away first. Not geocode-gated: the distinct outlier markers render in
    // every `viz` build; only the jurisdiction naming/suppression below needs geocode. Partition by
    // INDEX so the row-aligned dataset hover lines follow the same split.
    let (core_idx, out_idx) = partition_geo_outlier_indices(&lats, &lons);
    let gather_f64 =
        |idx: &[usize], src: &[f64]| -> Vec<f64> { idx.iter().map(|&i| src[i]).collect() };
    let gather_str = |idx: &[usize], src: &[String]| -> Vec<String> {
        idx.iter().map(|&i| src[i].clone()).collect()
    };
    let core_lats = gather_f64(&core_idx, &lats);
    let core_lons = gather_f64(&core_idx, &lons);
    let out_lats = gather_f64(&out_idx, &lats);
    let out_lons = gather_f64(&out_idx, &lons);
    let core_lines = gather_str(&core_idx, &dataset_lines);
    let out_lines = gather_str(&out_idx, &dataset_lines);
    // `--photos` payloads follow the SAME index partition as the hover lines, so a point's photos
    // can never drift onto a different point's marker. A filler of empty strings when the flag is
    // off keeps one packing path below rather than two (two is where alignment bugs come from).
    let photos_or_blank = |idx: &[usize], n: usize| -> Vec<String> {
        if photo_idxs.is_empty() {
            vec![String::new(); n]
        } else {
            gather_str(idx, &photo_urls)
        }
    };
    let core_photos_raw = photos_or_blank(&core_idx, core_lats.len());
    let out_photos_raw = photos_or_blank(&out_idx, out_lats.len());
    // row ordinals follow the SAME index partition (and, below, the same downsample stride) as
    // the hover lines and photos — one packing path, so a point can never acquire another row's
    // ordinal. Filler zeros when the feature is off; they are discarded after downsampling.
    let gather_u64 =
        |idx: &[usize], src: &[u64]| -> Vec<u64> { idx.iter().map(|&i| src[i]).collect() };
    let ids_or_blank = |idx: &[usize], n: usize| -> Vec<u64> {
        if capture_row_ids {
            gather_u64(idx, &row_ids)
        } else {
            vec![0_u64; n]
        }
    };
    let core_ids_raw = ids_or_blank(&core_idx, core_lats.len());
    let out_ids_raw = ids_or_blank(&out_idx, out_lats.len());

    // reverse-geocode the CORE bounding box into the spatial-extent overlay + summary, plus a
    // representative sample of the outliers for the call-out (one batched engine load). Degrades to
    // None when the geocode feature is off, the lookup fails, or the extent spans the antimeridian.
    #[cfg(feature = "geocode")]
    let geo_meta = build_geo_meta(
        map_extent(&core_lats, &core_lons),
        &out_lats,
        &out_lons,
        out_lats.len(),
        args.flag_smarter,
    );

    // downsample coords AND the row-aligned dataset hover lines together (pack the line into the
    // pair's `Y` so the same stride applies), keeping each line aligned to its (lat, lon).
    // core: pack lon + hover line + (optional) bubble measure so one stride keeps them aligned.
    let core_sizes_raw: Vec<f64> = if size_idx.is_some() {
        gather_f64(&core_idx, &sizes_raw)
    } else {
        vec![f64::NAN; core_lats.len()]
    };
    // move the lines in rather than cloning the whole set: `core_lines` is rebuilt from the
    // downsampled result below, so the original vector is dead after this
    let core_packed: Vec<(f64, String, f64, String, u64)> = core_lons
        .iter()
        .copied()
        .zip(core_lines)
        .zip(core_sizes_raw.iter().copied())
        .zip(core_photos_raw)
        .zip(core_ids_raw)
        .map(|((((lo, line), sz), photo), rid)| (lo, line, sz, photo, rid))
        .collect();
    let (lats, core_pl) = downsample_pair(&core_lats, &core_packed, *MAX_SMART_POINTS);
    let lons: Vec<f64> = core_pl.iter().map(|t| t.0).collect();
    let core_lines: Vec<String> = core_pl.iter().map(|t| t.1.clone()).collect();
    // size only when a measure was tagged AND at least one core point has a finite value (else
    // uniform/empty sizing adds nothing — fall back to fixed-size markers).
    let core_sizes: Option<Vec<f64>> = size_idx
        .map(|_| core_pl.iter().map(|t| t.2).collect::<Vec<f64>>())
        .filter(|v| v.iter().any(|x| x.is_finite()));
    let out_packed: Vec<(f64, String, String, u64)> = out_lons
        .iter()
        .copied()
        .zip(out_lines)
        .zip(out_photos_raw)
        .zip(out_ids_raw)
        .map(|(((lo, line), photo), rid)| (lo, line, photo, rid))
        .collect();
    let (outlier_lats, out_pl) = downsample_pair(&out_lats, &out_packed, SMART_GEO_OUTLIER_CAP);
    let outlier_lons: Vec<f64> = out_pl.iter().map(|t| t.0).collect();
    let out_lines: Vec<String> = out_pl.iter().map(|t| t.1.clone()).collect();
    // resolved AFTER downsampling so each surviving point keeps its own photos; left empty when
    // `--photos` is off so the trace emits no `customdata` at all.
    let (core_photos, outlier_photos): (Vec<String>, Vec<String>) = if photo_idxs.is_empty() {
        (Vec::new(), Vec::new())
    } else {
        (
            core_pl.iter().map(|t| t.3.clone()).collect(),
            out_pl.iter().map(|t| t.2.clone()).collect(),
        )
    };
    // Resolved AFTER downsampling, like the photos, so each surviving point keeps its OWN
    // ordinal. Rendered here (not at emit time) because the preview cap is what turns an ordinal
    // into either a selectable id or the empty "beyond the preview" marker.
    let row_id_label = |ordinal: u64| -> String {
        if ordinal < row_id_cap {
            ordinal.to_string()
        } else {
            String::new()
        }
    };
    let (core_row_ids, outlier_row_ids): (Vec<String>, Vec<String>) = if capture_row_ids {
        (
            core_pl.iter().map(|t| row_id_label(t.4)).collect(),
            out_pl.iter().map(|t| row_id_label(t.3)).collect(),
        )
    } else {
        (Vec::new(), Vec::new())
    };

    // per-point reverse-geocoded place (City, County, Admin1, Country [+ FIPS under --smarter]),
    // gap-filtered so it never duplicates a dataset geo.* column already in the hover. One batched
    // engine load for core+outliers; degrades to no place on failure. Empty strings when the
    // geocode feature is off or a point didn't resolve.
    #[cfg(feature = "geocode")]
    let (core_places, out_places): (Vec<String>, Vec<String>) = {
        // which geocoded components are already supplied by a chosen dataset column (gap-filling)
        let mut sup_city = false;
        let mut sup_admin1 = false;
        let mut sup_country = false;
        for &idx in &hover_field_idxs {
            match col_sems.get(idx).map(|s| s.concept.as_str()).unwrap_or("") {
                "geo.city" => sup_city = true,
                "geo.state" => sup_admin1 = true,
                "geo.country" => sup_country = true,
                _ => {},
            }
        }
        // Always reverse-geocode when a map renders: even if the dataset already supplies
        // city/state/country, the lookup still contributes the county (always-on) and — under
        // --smarter — the US FIPS code. `geocode_hover_place` suppresses (via the sup_* flags) any
        // component the dataset already shows, so nothing is duplicated.
        let mut pts: Vec<(f64, f64)> = Vec::with_capacity(lats.len() + outlier_lats.len());
        pts.extend(lats.iter().zip(&lons).map(|(&a, &o)| (a, o)));
        pts.extend(
            outlier_lats
                .iter()
                .zip(&outlier_lons)
                .map(|(&a, &o)| (a, o)),
        );
        let labels = crate::cmd::geocode::reverse_geocode_points(&pts, None).unwrap_or_default();
        // dataset_line lets the place drop any component the dataset already shows for this
        // point (e.g. a city-name identifier).
        let place = |i: usize, dataset_line: &str| -> String {
            labels
                .get(i)
                .and_then(Option::as_ref)
                .map(|l| {
                    geocode_hover_place(
                        l,
                        sup_city,
                        sup_admin1,
                        sup_country,
                        args.flag_smarter,
                        dataset_line,
                    )
                })
                .unwrap_or_default()
        };
        let core: Vec<String> = core_lines
            .iter()
            .enumerate()
            .map(|(i, l)| place(i, l))
            .collect();
        let out: Vec<String> = out_lines
            .iter()
            .enumerate()
            .map(|(j, l)| place(lats.len() + j, l))
            .collect();
        (core, out)
    };
    #[cfg(not(feature = "geocode"))]
    let (core_places, out_places): (Vec<String>, Vec<String>) = (
        vec![String::new(); lats.len()],
        vec![String::new(); outlier_lats.len()],
    );

    // build hover text only when something beyond the bare coordinates was contributed (a dataset
    // field or a geocoded place); otherwise leave the vecs empty so render keeps plotly's default
    // lat/lon hover.
    let geo_contributed = core_places.iter().chain(&out_places).any(|p| !p.is_empty());
    // `--photos` alone is enough to warrant a hover string: the photo-count affordance is the only
    // thing that makes the dwell preview discoverable, so it must appear even on a map whose
    // points would otherwise fall back to plotly's bare lat/lon hover.
    let has_photo_payload = !core_photos.is_empty() || !outlier_photos.is_empty();
    // The affordance may only appear where a dwell can actually deliver a photo, else it
    // promises something the page cannot do:
    //   * a `global` extent renders as `PanelKind::Geo`, which carries no photos at all — so
    //     NEITHER of its traces can show one.
    //   * in `density` mode the CORE is a `DensityMap` heat surface: no discrete markers, no
    //     `customdata`. Its geographic-outlier companion is still a marker trace with photos, so
    //     outliers keep the hint even in density mode.
    let core_hint = !global && !density;
    let outlier_hint = !global;
    let (hover_text, outlier_hover_text) =
        if dataset_has_fields || geo_contributed || has_photo_payload {
            let assemble = |lines: &[String], places: &[String], photos: &[String], hint: bool| {
                lines
                    .iter()
                    .enumerate()
                    .zip(places)
                    .map(|((i, line), place)| {
                        // `photos` is empty when --photos is off, and an entry is "" for a point
                        // with no photo — both yield a count of 0, i.e. no
                        // affordance line.
                        let n = if hint {
                            photos.get(i).map_or(0, |p| {
                                if p.is_empty() {
                                    0
                                } else {
                                    p.split('|').count()
                                }
                            })
                        } else {
                            0
                        };
                        assemble_map_hover(line, has_id, place, n)
                    })
                    .collect::<Vec<String>>()
            };
            (
                assemble(&core_lines, &core_places, &core_photos, core_hint),
                assemble(&out_lines, &out_places, &outlier_photos, outlier_hint),
            )
        } else {
            (Vec::new(), Vec::new())
        };

    // geocode-gated companion: a per-region choropleth overview drawn beside the point map. Gate it
    // on the cheap min/max-extent SPAN (already computed above) — a per-region breakdown only makes
    // sense at country scale, so a cluster smaller than `SMART_CHOROPLETH_MIN_SPAN_DEG` in BOTH
    // axes is treated as a single metro/region and skipped, short-circuiting the expensive
    // all-row reverse-geocode pass (and its index load) for the common high-row-count city
    // dataset. Unlike gating on the geocoded bounding-box corners, this still fires for global
    // extents that wrap the antimeridian. When the gate passes, build_smart_choropleth_panel
    // reverse-geocodes the FULL core set (pre-downsampling) and picks per-US-state vs
    // per-country from what actually resolved — accurate even above `MAX_SMART_POINTS`,
    // embedding only the aggregates (never the raw points), and its own 2-region check drops
    // single-region data.
    // A user `--geojson` switches the companion to a point-in-polygon choropleth (ungated — no
    // geocode engine needed). It bypasses the span gate: an explicit `--geojson` is explicit
    // intent, so a small custom-district file shouldn't be span-suppressed. Without
    // `--geojson`, fall back to the geocode-derived iso3/US-state panel (gated, span-gated as
    // before).
    let (choropleth_panel, geojson_overlay) = if let Some(spec) = args.flag_geojson.as_deref() {
        let snap = !args.flag_no_snap;
        // smart resolves the cap exactly like the command path: an explicit --snap-max-dist
        // (validated up front in run(): non-negative finite, not with --no-snap) wins verbatim,
        // else resolve_snap_cap derives it from the GeoJSON's region scale plus the stats cache's
        // lat/lon precision. Do NOT short-circuit this to f64::INFINITY — a real cap is deliberate
        // here; the `smart_pip_panel_caps_snap` unit test guards the threading.
        let coord_decimals = stats_coord_decimals(stats, lat_idx, lon_idx);
        let key = args.flag_feature_id_key.as_deref().unwrap_or("id");
        let name_key = args.flag_feature_name_key.as_deref();
        let panel = build_smart_pip_choropleth_panel(
            spec,
            key,
            name_key,
            &core_lats,
            &core_lons,
            snap,
            args.flag_snap_max_dist,
            coord_decimals,
            loaded_geojson,
        )?;
        // overlay the region boundaries + labels directly on the point map (best-effort; the panel
        // builder above already hard-errored on a bad --geojson). Independent of the choropleth's
        // 2-region minimum — even a single region is worth outlining on the map.
        let overlay = build_geojson_overlay(spec, key, name_key, loaded_geojson);
        (panel, overlay)
    } else {
        #[cfg(feature = "geocode")]
        let choropleth = (lon_span >= SMART_CHOROPLETH_MIN_SPAN_DEG
            || lat_span >= SMART_CHOROPLETH_MIN_SPAN_DEG)
            .then(|| build_smart_choropleth_panel(&core_lats, &core_lons))
            .flatten();
        #[cfg(not(feature = "geocode"))]
        let choropleth = None;
        (choropleth, None)
    };

    // pan bounds from the FULL (pre-downsample) core + outlier coordinates, so a stride-dropped
    // extreme can't shrink the box below the true extent the "Full extent" button / overlay use.
    // Only the tile `Map` render honors it; the `geo` projection ignores map.bounds.
    let pan_bounds = map_pan_bounds(&[
        (core_lats.as_slice(), core_lons.as_slice()),
        (out_lats.as_slice(), out_lons.as_slice()),
    ]);
    // Enable native point clustering on the core markers when they'd otherwise overplot. Never
    // possible in density mode — the core is a `DensityMap` heatmap, a different trace type. Under
    // `--cluster auto` (the default) also require a plain-marker core (no bubble-size measure,
    // whose per-point size a cluster bubble would override) that is dense enough to benefit
    // (>= SMART_CLUSTER_MIN_POINTS). `--cluster on` overrides those two soft guards; `--cluster
    // off` never clusters, so dense points draw plainly. `lats` here is the downsampled core.
    let cluster = !density
        && match cluster_mode {
            ClusterMode::Off => false,
            ClusterMode::On => true,
            ClusterMode::Auto => core_sizes.is_none() && lats.len() >= SMART_CLUSTER_MIN_POINTS,
        };
    // Downsampling honesty cue, shown as the panel subtitle: when the embedded points are a
    // stride sample of the mappable set, say so ON the panel — the HTML reader (unlike the CLI
    // user, who sees the stderr notes) otherwise has no way to know the map is thinned while
    // the Regions choropleth beside it tallies every row. The outlier clause tells a careful
    // reader the tails are exhaustive (or how thinned) even though the body is sampled. None —
    // no subtitle — when every mappable point is embedded.
    let sample_note: Option<String> = {
        let rendered = lats.len() + outlier_lats.len();
        (rendered < mappable_count).then(|| {
            let n = |v: usize| group_thousands(&v.to_string());
            // density aggregates the sampled points into a heatmap (unless the global extent
            // rendered them as ScatterGeo markers anyway)
            let verb = if density && !global {
                "aggregating"
            } else {
                "showing"
            };
            let mut note = format!(
                "{verb} {} of {} points (sampled)",
                n(rendered),
                n(mappable_count)
            );
            if !out_lats.is_empty() {
                if outlier_lats.len() == out_lats.len() {
                    let noun = if out_lats.len() == 1 {
                        "outlier"
                    } else {
                        "outliers"
                    };
                    note.push_str(&format!(
                        "; all {} geographic {noun} kept",
                        n(out_lats.len())
                    ));
                } else {
                    note.push_str(&format!(
                        "; {} of {} geographic outliers",
                        n(outlier_lats.len()),
                        n(out_lats.len())
                    ));
                }
            }
            note
        })
    };
    let kind = if global {
        PanelKind::Geo {
            lats,
            lons,
            outlier_lats,
            outlier_lons,
            hover_text,
            outlier_hover_text,
            sizes: core_sizes,
            row_ids: core_row_ids,
            outlier_row_ids,
        }
    } else {
        PanelKind::Map {
            lats,
            lons,
            density,
            cluster,
            outlier_lats,
            outlier_lons,
            hover_text,
            outlier_hover_text,
            sizes: core_sizes,
            pan_bounds,
            photos: core_photos,
            outlier_photos,
            row_ids: core_row_ids,
            outlier_row_ids,
        }
    };
    Ok(Some((
        Panel {
            name: t!("viz.title.map").into_owned(),
            subtitle: sample_note,
            kind,
            value_log: false,
            axis_log: (false, false),
            interest: f64::INFINITY,
            dict_info: None,
            // a map panel charts a lat/lon PAIR, not a single stats row
            stat_idx: None,
            #[cfg(feature = "geocode")]
            geo_meta,
            geojson_overlay,
        },
        choropleth_panel,
        (lat_idx, lon_idx),
        mappable_count,
    )))
}

/// Vertical paper-coordinate offset below a map subplot's plotting area at which the
/// consolidated location-summary annotation is anchored (typed-`Plot` grid + `GridJson` paths).
#[cfg(feature = "geocode")]
const GEO_META_OFFSET: f64 = 0.02;

/// Compute the observed bounding box (min/max lat/lon, no trimming) of a map panel's
/// in-range coordinates.
#[cfg(feature = "geocode")]
fn map_extent(lats: &[f64], lons: &[f64]) -> MapExtent {
    let mut e = MapExtent {
        min_lat: f64::INFINITY,
        max_lat: f64::NEG_INFINITY,
        min_lon: f64::INFINITY,
        max_lon: f64::NEG_INFINITY,
    };
    for (&lat, &lon) in lats.iter().zip(lons) {
        e.min_lat = e.min_lat.min(lat);
        e.max_lat = e.max_lat.max(lat);
        e.min_lon = e.min_lon.min(lon);
        e.max_lon = e.max_lon.max(lon);
    }
    e
}

/// Expand a bounding box to also include the given coordinates (used to grow the core extent into
/// the full extent that covers the geographic outliers too).
#[cfg(feature = "geocode")]
fn extent_including(core: MapExtent, lats: &[f64], lons: &[f64]) -> MapExtent {
    let mut e = core;
    for (&lat, &lon) in lats.iter().zip(lons) {
        e.min_lat = e.min_lat.min(lat);
        e.max_lat = e.max_lat.max(lat);
        e.min_lon = e.min_lon.min(lon);
        e.max_lon = e.max_lon.max(lon);
    }
    e
}

/// Whether a map extent is too wide to box meaningfully — more than 180° of longitude. This fires
/// for any extent wider than a hemisphere (the dataset is effectively global), the most common
/// cause being an antimeridian crossing; in either case the box corners + center are meaningless
/// and reverse-geocoding is skipped.
#[cfg(feature = "geocode")]
fn extent_spans_antimeridian(e: &MapExtent) -> bool {
    e.max_lon - e.min_lon > 180.0
}

/// Build the closed-loop (lat, lon) polyline tracing a bounding box, in order
/// NW -> NE -> SE -> SW -> NW, for drawing the spatial extent on the map.
#[cfg(feature = "geocode")]
fn extent_box_latlon(e: &MapExtent) -> (Vec<f64>, Vec<f64>) {
    let lats = vec![e.max_lat, e.max_lat, e.min_lat, e.min_lat, e.max_lat];
    let lons = vec![e.min_lon, e.max_lon, e.max_lon, e.min_lon, e.min_lon];
    (lats, lons)
}

/// Build a *dashed* bounding-box outline as a single gapped polyline: each edge is split into short
/// dash segments separated by `NaN` breaks (which serialize to JSON `null`, rendering as gaps).
/// `scattermap` ignores `line.dash` (MapLibre can't render dashed lines on raster tiles), so the
/// dashes have to be drawn as geometry; this gives the full-extent box a dashed look on the tile
/// map. Dash spacing is proportional to the box size so it reads as dashes at any zoom.
#[cfg(feature = "geocode")]
fn dashed_box_latlon(e: &MapExtent) -> (Vec<f64>, Vec<f64>) {
    let span = (e.max_lat - e.min_lat).max(e.max_lon - e.min_lon);
    // target dash+gap cell length (~40 cells across the longest edge -> ~20 dashes)
    let cell = (span / 40.0).max(f64::MIN_POSITIVE);
    let corners = [
        (e.max_lat, e.min_lon),
        (e.max_lat, e.max_lon),
        (e.min_lat, e.max_lon),
        (e.min_lat, e.min_lon),
        (e.max_lat, e.min_lon),
    ];
    let (mut lats, mut lons) = (Vec::new(), Vec::new());
    for edge in corners.windows(2) {
        let &[(la0, lo0), (la1, lo1)] = edge else {
            unreachable!("windows(2) always yields exactly two elements")
        };
        let len = ((la1 - la0).powi(2) + (lo1 - lo0).powi(2)).sqrt();
        // even cell count so each edge ends on a gap, not a dash bleeding into the corner
        let mut cells = ((len / cell).round() as usize).max(2);
        if cells % 2 == 1 {
            cells += 1;
        }
        // draw the even cells (dashes), skip the odd ones (gaps)
        for c in (0..cells).step_by(2) {
            let (t0, t1) = (c as f64 / cells as f64, (c + 1) as f64 / cells as f64);
            lats.push(la0 + (la1 - la0) * t0);
            lons.push(lo0 + (lo1 - lo0) * t0);
            lats.push(la0 + (la1 - la0) * t1);
            lons.push(lo0 + (lo1 - lo0) * t1);
            lats.push(f64::NAN); // break -> gap between dashes
            lons.push(f64::NAN);
        }
    }
    (lats, lons)
}

/// Styled line for the spatial-extent bounding box: a thick dotted purple frame, so it reads as a
/// boundary annotation rather than a data series.
#[cfg(feature = "geocode")]
fn extent_box_line() -> Line {
    Line::new()
        .color(GEO_EXTENT_LINE_COLOR)
        .width(GEO_EXTENT_LINE_WIDTH)
        .dash(plotly::common::DashType::Dot)
}

/// Styled line for the FULL-extent box (core + outliers): a dotted vivid-magenta frame. Magenta
/// stays high-contrast on every basemap layer (tan land, green, blue water, white) — unlike the
/// amber it replaced, which washed out over light terrain — and is distinct from the purple core
/// box, the warm data points, and the amber outlier markers.
#[cfg(feature = "geocode")]
fn full_extent_box_line() -> Line {
    Line::new()
        .color(GEO_FULL_EXTENT_LINE_COLOR)
        .width(GEO_FULL_EXTENT_LINE_WIDTH)
        .dash(plotly::common::DashType::Dot)
}

/// Marker for the extent corner/center points on a MapLibre tile map: a large, fully opaque purple
/// circle with a white halo. MapLibre raster basemaps can't render custom marker symbols (no
/// sprite) or reliably show text glyphs (label collision culls them), so a haloed circle is used
/// here.
#[cfg(feature = "geocode")]
fn extent_marker_map() -> Marker {
    Marker::new()
        .color(GEO_EXTENT_LINE_COLOR)
        .size(GEO_EXTENT_MARKER_SIZE)
        .opacity(1.0)
        .line(Line::new().color(GEO_EXTENT_MARKER_BORDER).width(2.5))
}

/// Like `extent_marker_map`, but a diamond — `ScatterGeo` (offline projection / static export)
/// renders the standard plotly marker symbols, so the points get a distinctive shape there.
#[cfg(feature = "geocode")]
fn extent_marker_geo() -> Marker {
    extent_marker_map().symbol(plotly::common::MarkerSymbol::Diamond)
}

/// Map (center lat, center lon, zoom) that frames an extent bounding box as tightly as possible
/// while keeping the whole box inside the panel, via the aspect-aware `fitbounds_zoom`. A `viz
/// smart` MAP panel is full grid width but a FIXED short height (`MAP_PANEL_USABLE_HEIGHT_PX`),
/// i.e. wide and short, so the latitude (height) fit and longitude (width) fit are computed
/// separately and the tighter one wins — replacing the old `floor(log2(360/max(latSpan, lonSpan)))`
/// that ignored the panel aspect (over-zooming square core boxes so they clipped, under-zooming
/// wide full extents). Width is `responsive`/unknown at generation time, so a conservative-small
/// `MAP_PANEL_ASSUMED_WIDTH_PX` is assumed; verify visually if the panel sizing changes.
#[cfg(feature = "geocode")]
fn extent_center_zoom_raw(e: &MapExtent) -> (f64, f64, u8) {
    let lat_center = mercator_lat_center(e.min_lat, e.max_lat);
    let lon_center = (e.min_lon + e.max_lon) / 2.0;
    let zoom = fitbounds_zoom(
        e.min_lat,
        e.max_lat,
        e.max_lon - e.min_lon,
        MAP_PANEL_ASSUMED_WIDTH_PX,
        MAP_PANEL_USABLE_HEIGHT_PX,
    );
    (lat_center, lon_center, zoom)
}

/// Map center + zoom framing the (core) extent — so the default view fills with the core cluster
/// rather than zooming out to include far-flung outliers (those are drawn as distinct markers,
/// named in the label, and reachable via the "Full extent" zoom button). Thin wrapper over
/// `extent_center_zoom_raw`.
#[cfg(feature = "geocode")]
fn extent_center_zoom(e: &MapExtent) -> (Center, u8) {
    let (lat, lon, zoom) = extent_center_zoom_raw(e);
    (Center::new(lat, lon), zoom)
}

/// Build the "Core extent" / "Full extent" zoom buttons for a `viz smart` MapLibre map. Each
/// button relayouts the `map` subplot center+zoom to frame the respective extent. Only used when
/// the panel has geographic outliers (so the two views actually differ). The map opens at the core
/// view, so "Core extent" is the active button.
#[cfg(feature = "geocode")]
fn extent_zoom_menu(core: &MapExtent, full: &MapExtent) -> UpdateMenu {
    let button = |label: &str, e: &MapExtent| {
        let (lat, lon, zoom) = extent_center_zoom_raw(e);
        Button::new()
            .label(label)
            .method(ButtonMethod::Relayout)
            .args(serde_json::json!([{
                "map.center": { "lat": lat, "lon": lon },
                "map.zoom": zoom,
            }]))
    };
    UpdateMenu::new()
        .ty(UpdateMenuType::Buttons)
        .direction(UpdateMenuDirection::Right)
        .buttons(vec![
            button(&t!("viz.menu.extent_core"), core),
            button(&t!("viz.menu.extent_full"), full),
        ])
        .x(0.02)
        .x_anchor(Anchor::Left)
        .y(0.98)
        .y_anchor(Anchor::Top)
        .show_active(true)
        .active(0)
        .background_color(NamedColor::White)
        .border_color(NamedColor::Gray)
        .border_width(1)
        // the button pill is always white (over the light map tiles), so pin the label color to
        // ink rather than inheriting `layout.font.color` — otherwise the dark-mode toggle (or a
        // dark `--theme`) flips the label to a light color and it vanishes on the white pill.
        .font(Font::new().family(FONT_FAMILY).color(INK).size(11))
}

/// A single "Clusters/Points" toggle button for the `viz smart` map's cluster-eligible core trace.
/// The map OPENS as individual points (the core trace bakes `cluster.enabled = false`); this
/// button, a native plotly toggle, restyles `cluster.enabled` on trace 0 — `args` (first click,
/// from the inactive state) turns clustering on, `args2` (next click, from the active state) turns
/// it back off. Only trace 0 is targeted, so the geographic-outlier call-out trace (also a
/// `scattermap`, but never clustered) is left untouched. Starts inactive (`active(-1)`) to match
/// the points-first default. Added only when the core is cluster-eligible.
fn cluster_toggle_menu() -> UpdateMenu {
    let toggle = Button::new()
        .label(&t!("viz.menu.cluster_toggle"))
        .method(ButtonMethod::Restyle)
        .args(serde_json::json!([{ "cluster.enabled": true }, [0]]))
        .args2(serde_json::json!([{ "cluster.enabled": false }, [0]]));
    UpdateMenu::new()
        .ty(UpdateMenuType::Buttons)
        .buttons(vec![toggle])
        // stacked just below the Core/Full extent menu (top-left), clear of the top-right modebar.
        .x(0.02)
        .x_anchor(Anchor::Left)
        .y(0.86)
        .y_anchor(Anchor::Top)
        .show_active(true)
        // start inactive: the map opens on individual points, click to collapse into clusters.
        .active(-1)
        .background_color(NamedColor::White)
        .border_color(NamedColor::Gray)
        .border_width(1)
        // pin the label to ink so a dark theme/toggle can't flip it invisible on the white pill.
        .font(Font::new().family(FONT_FAMILY).color(INK).size(11))
}

/// A single "Basemap labels" toggle for a `viz smart` MapLibre choropleth panel. The metro-scale
/// choropleth opens on the LABELED carto basemap (place names in the inter-polygon gaps aid
/// orientation); this native plotly toggle relayouts the `map` subplot's `style` between the
/// labeled and label-free carto variants of the SAME theme. `args` (first click from the active
/// state's inverse) restores labels, `args2` hides them — the button starts ACTIVE (`active(0)`)
/// so the highlighted state reads as "labels shown", matching the labeled-basemap default.
///
/// The two style strings are baked for the RENDER-time theme, so after a runtime light/dark flip
/// they would point at the wrong theme's basemap. The theme-toggle JS keeps them honest: on every
/// `apply()` it rewrites any `map.style` updatemenu arg to the target theme's variant, preserving
/// the `-nolabels` suffix (see `SCRIPT_TEMPLATE`).
fn basemap_labels_toggle_menu(labeled_style: MapStyle, nolabels_style: MapStyle) -> UpdateMenu {
    let toggle = Button::new()
        .label(&t!("viz.menu.basemap_labels"))
        .method(ButtonMethod::Relayout)
        .args(serde_json::json!([{ "map.style": labeled_style }]))
        .args2(serde_json::json!([{ "map.style": nolabels_style }]));
    UpdateMenu::new()
        .ty(UpdateMenuType::Buttons)
        .buttons(vec![toggle])
        // top-left of the panel (this panel has no Core/Full extent menu, so the top slot is free).
        .x(0.02)
        .x_anchor(Anchor::Left)
        .y(0.98)
        .y_anchor(Anchor::Top)
        .show_active(true)
        // start active: the map opens WITH basemap labels, click to hide them.
        .active(0)
        .background_color(NamedColor::White)
        .border_color(NamedColor::Gray)
        .border_width(1)
        // pin the label to ink so a dark theme/toggle can't flip it invisible on the white pill.
        .font(Font::new().family(FONT_FAMILY).color(INK).size(11))
}

/// Padded longitude + latitude `geo` axis ranges that frame the (core) extent box as tightly as
/// possible, for the offline `ScatterGeo` (local Mercator) path — the analogue of
/// `extent_center_zoom` for projection maps. Unlike the tile map, these are exact ranges, so the
/// small `GEO_EXTENT_FIT_PAD_*` padding is the only slack.
#[cfg(feature = "geocode")]
fn extent_geo_axes(e: &MapExtent) -> (Axis, Axis) {
    let lat_pad =
        ((e.max_lat - e.min_lat) * GEO_EXTENT_FIT_PAD_FRAC).max(GEO_EXTENT_FIT_PAD_MIN_DEG);
    let lon_pad =
        ((e.max_lon - e.min_lon) * GEO_EXTENT_FIT_PAD_FRAC).max(GEO_EXTENT_FIT_PAD_MIN_DEG);
    let lon = Axis::new().range(vec![
        (e.min_lon - lon_pad).max(-180.0),
        (e.max_lon + lon_pad).min(180.0),
    ]);
    let lat = Axis::new().range(vec![
        (e.min_lat - lat_pad).max(-90.0),
        (e.max_lat + lat_pad).min(90.0),
    ]);
    (lon, lat)
}

/// Join the non-empty components of a reverse-geocoded label into "City, Admin1, Country".
#[cfg(feature = "geocode")]
fn format_geo_label(label: &crate::cmd::geocode::GeoLabel) -> String {
    [
        label.city.as_str(),
        label.admin1.as_str(),
        label.country.as_str(),
    ]
    .iter()
    .filter(|s| !s.trim().is_empty())
    .copied()
    .collect::<Vec<_>>()
    .join(", ")
}

/// Hover text for a single extent point marker, e.g. "NW: Newark, New Jersey, United States"
/// (or "NW: no nearby city" when the coordinate didn't resolve).
#[cfg(feature = "geocode")]
fn point_hover_text(p: &GeoPoint) -> String {
    match &p.label {
        // escape: geocoded place names can carry `&`/`<` and this feeds HoverInfo::Text,
        // which plotly renders as pseudo-HTML — matching every other hover path
        Some(label) => format!("{}: {}", p.tag, escape_hover(&format_geo_label(label))),
        None => format!("{}: no nearby city", p.tag),
    }
}

/// Distinct, first-seen order, case-insensitive dedup of non-empty trimmed values.
#[cfg(feature = "geocode")]
fn distinct_jurisdictions<'a>(vals: impl Iterator<Item = &'a str>) -> Vec<String> {
    let mut seen = std::collections::HashSet::new();
    let mut out = Vec::new();
    for v in vals {
        let v = v.trim();
        if v.is_empty() {
            continue;
        }
        if seen.insert(v.to_lowercase()) {
            out.push(v.to_string());
        }
    }
    out
}

/// Join a list of names as "A" | "A & B" | "A, B & C".
#[cfg(feature = "geocode")]
fn join_jurisdictions(items: &[String]) -> String {
    match items {
        [] => String::new(),
        [a] => a.clone(),
        [a, b] => format!("{a} & {b}"),
        [rest @ .., last] => format!("{} & {last}", rest.join(", ")),
    }
}

/// Concise jurisdiction list for the outlier call-out: distinct admin1s (states/regions) if any
/// resolved, else distinct countries, joined "A" / "A & B" / "A, B & C"; more than 3 collapses to a
/// count. Returns "" when no outlier point resolved (the call-out then omits the parenthetical).
#[cfg(feature = "geocode")]
fn outlier_jurisdictions(labels: &[Option<crate::cmd::geocode::GeoLabel>]) -> String {
    let resolved: Vec<&crate::cmd::geocode::GeoLabel> =
        labels.iter().filter_map(|l| l.as_ref()).collect();
    if resolved.is_empty() {
        return String::new();
    }
    let admin1s = distinct_jurisdictions(resolved.iter().map(|l| l.admin1.as_str()));
    let names = if admin1s.is_empty() {
        distinct_jurisdictions(resolved.iter().map(|l| l.country.as_str()))
    } else {
        admin1s
    };
    match names.len() {
        0 => String::new(),
        n if n > 3 => t!("viz.map.geo_n_areas", q_n = n).into_owned(),
        _ => join_jurisdictions(&names),
    }
}

/// Whether the geographic outliers fall within the SAME jurisdiction(s) as the core extent — in
/// which case they're not meaningful "elsewhere" strays (just the cluster's far edge), so the
/// outlier call-out is suppressed. True when every resolved outlier admin1 is already among the
/// core's admin1s AND every resolved outlier country is among the core's countries; also true when
/// no outlier point resolved to a jurisdiction (we can't claim a different place). False as soon as
/// an outlier introduces a new admin1/country (e.g. a Pennsylvania stray beside a NY/NJ core).
#[cfg(feature = "geocode")]
fn outliers_share_core_region(
    core: &[GeoPoint],
    outlier_labels: &[Option<crate::cmd::geocode::GeoLabel>],
) -> bool {
    use std::collections::HashSet;
    let lower_set = |it: &mut dyn Iterator<Item = &str>| -> HashSet<String> {
        it.map(str::trim)
            .filter(|s| !s.is_empty())
            .map(str::to_lowercase)
            .collect()
    };
    let core_admin1 = lower_set(
        &mut core
            .iter()
            .filter_map(|p| p.label.as_ref())
            .map(|l| l.admin1.as_str()),
    );
    let core_country = lower_set(
        &mut core
            .iter()
            .filter_map(|p| p.label.as_ref())
            .map(|l| l.country.as_str()),
    );
    let out_admin1 = lower_set(
        &mut outlier_labels
            .iter()
            .filter_map(|l| l.as_ref())
            .map(|l| l.admin1.as_str()),
    );
    let out_country = lower_set(
        &mut outlier_labels
            .iter()
            .filter_map(|l| l.as_ref())
            .map(|l| l.country.as_str()),
    );

    if out_admin1.is_empty() && out_country.is_empty() {
        // outliers didn't resolve to any place -> can't claim they're elsewhere; suppress
        return true;
    }
    out_admin1.is_subset(&core_admin1) && out_country.is_subset(&core_country)
}

/// Assemble the spatial-extent summary string from the core jurisdiction summary, the (true)
/// outlier count, and their consolidated jurisdiction. With zero outliers the core summary is
/// returned unchanged (byte-identical to the pre-feature behavior). Otherwise an em-dash call-out
/// is appended, e.g. "New York & New Jersey, United States — 3 outliers (Pennsylvania)"; the
/// parenthetical is omitted when no outlier point resolved to a jurisdiction.
#[cfg(feature = "geocode")]
fn outlier_summary(core: &str, n_outliers: usize, jurisdictions: &str) -> String {
    if n_outliers == 0 {
        return core.to_string();
    }
    let noun = if n_outliers == 1 {
        t!("viz.map.geo_outlier_one")
    } else {
        t!("viz.map.geo_outlier_many")
    };
    let n_outliers = HumanCount(n_outliers as u64);
    if jurisdictions.is_empty() {
        t!(
            "viz.map.geo_outliers",
            q_core = core,
            q_n = n_outliers.to_string(),
            q_noun = noun
        )
        .into_owned()
    } else {
        t!(
            "viz.map.geo_outliers_where",
            q_core = core,
            q_n = n_outliers.to_string(),
            q_noun = noun,
            q_where = jurisdictions
        )
        .into_owned()
    }
}

/// Post-process the up-to-5 reverse-geocoded extent points into one concise jurisdiction
/// line. Collapses shared country/state instead of repeating it; lists up to 3 distinct
/// regions/countries, else falls back to a count. Returns "" when no point resolved.
#[cfg(feature = "geocode")]
fn consolidate_geo(points: &[GeoPoint]) -> String {
    let labels: Vec<&crate::cmd::geocode::GeoLabel> =
        points.iter().filter_map(|p| p.label.as_ref()).collect();
    if labels.is_empty() {
        return String::new();
    }

    let countries = distinct_jurisdictions(labels.iter().map(|l| l.country.as_str()));
    if countries.len() > 3 {
        return t!("viz.map.geo_n_countries", q_n = countries.len()).into_owned();
    }
    if countries.len() >= 2 {
        return join_jurisdictions(&countries);
    }
    // 0 or 1 distinct country
    let country = countries.first();
    let suffix = country.map(|c| format!(", {c}")).unwrap_or_default();

    let admin1s = distinct_jurisdictions(labels.iter().map(|l| l.admin1.as_str()));
    if admin1s.is_empty() {
        return country.cloned().unwrap_or_default();
    }
    if admin1s.len() > 3 {
        return t!(
            "viz.map.geo_n_regions",
            q_n = admin1s.len(),
            q_suffix = suffix
        )
        .into_owned();
    }
    if admin1s.len() >= 2 {
        return format!("{}{suffix}", join_jurisdictions(&admin1s));
    }
    // single admin1: name the city only when the whole extent resolved to one city
    let admin1 = &admin1s[0];
    let cities = distinct_jurisdictions(labels.iter().map(|l| l.city.as_str()));
    if cities.len() == 1 {
        return format!("{}, {admin1}{suffix}", cities[0]);
    }
    // multiple cities but a single county: name the county for tighter context (mirrors the
    // single-city rule). A multi-county metro keeps the plain "Admin1, Country" form.
    let counties = distinct_jurisdictions(labels.iter().map(|l| l.admin2.as_str()));
    if counties.len() == 1 {
        return format!("{}, {admin1}{suffix}", counties[0]);
    }
    format!("{admin1}{suffix}")
}

/// Max outlier points reverse-geocoded for the call-out's jurisdiction list. The TRUE outlier count
/// is reported separately (it's just the partition size); this only bounds how many we look up to
/// NAME the jurisdictions, so a far-flung set is still represented without ballooning the lookup.
#[cfg(feature = "geocode")]
const GEO_OUTLIER_GEOCODE_SAMPLE: usize = 12;

/// Build the `--smarter` country-context annotation appended to a map's extent summary: the shared
/// continent of the resolved extent countries. `labels` must be EXACTLY the jurisdictions named in
/// the summary (the core points, plus the outliers when their call-out is present) so the continent
/// can't contradict a cross-continent outlier. Returns `None` when geocoding is unavailable,
/// nothing resolves, or the named extent spans more than one continent. Costs one extra batched
/// engine load over the (<=5) distinct countries — the continent is country-level, not on the point
/// lookup.
#[cfg(feature = "geocode")]
fn country_context_note(labels: &[&crate::cmd::geocode::GeoLabel]) -> Option<String> {
    let iso2s = distinct_jurisdictions(labels.iter().map(|l| l.country_iso2.as_str()));
    if iso2s.is_empty() {
        return None;
    }
    let refs: Vec<&str> = iso2s.iter().map(String::as_str).collect();
    let ctxs = crate::cmd::geocode::country_context(&refs).ok()?;
    let continents = distinct_jurisdictions(ctxs.iter().flatten().map(|c| c.continent.as_str()));
    if continents.len() != 1 {
        // no continent resolved, or the named extent spans multiple continents — skip the note.
        return None;
    }
    Some(format!(" · {}", continents[0]))
}

/// Reverse-geocode the CORE bounding box (4 corners + center) into the `GeoMeta` overlay + summary,
/// plus a capped, stride-sampled set of the outlier points — all in ONE batched lookup (the engine
/// loads once). The summary names the core jurisdictions and, when there are outliers that resolve
/// to a DIFFERENT jurisdiction than the core, appends a call-out with the count + the outliers'
/// jurisdiction; outliers within the core's own jurisdiction(s) are suppressed (just the cluster's
/// far edge, not strays elsewhere), and with zero outliers the result is byte-identical to the
/// pre-feature behavior. Returns `None` (so the map renders without the
/// overlay) when the core extent spans the antimeridian (corner/center labels would be
/// meaningless), the geocode lookup fails (offline/missing index), or no core point resolves to a
/// city.
#[cfg(feature = "geocode")]
fn build_geo_meta(
    core_extent: MapExtent,
    outlier_lats: &[f64],
    outlier_lons: &[f64],
    n_outliers: usize,
    smarter: bool,
) -> Option<GeoMeta> {
    // a dataset straddling +/-180 yields a near-global box whose center lands mid-ocean; skip.
    if extent_spans_antimeridian(&core_extent) {
        return None;
    }
    let c_lat = (core_extent.min_lat + core_extent.max_lat) / 2.0;
    let c_lon = (core_extent.min_lon + core_extent.max_lon) / 2.0;
    let coords = [
        ("NW", core_extent.max_lat, core_extent.min_lon),
        ("NE", core_extent.max_lat, core_extent.max_lon),
        ("SW", core_extent.min_lat, core_extent.min_lon),
        ("SE", core_extent.min_lat, core_extent.max_lon),
        ("Center", c_lat, c_lon),
    ];

    // batch the 5 core corner/center lookups with a stride-sampled set of the outlier points so the
    // whole call-out resolves in a single engine load.
    let (sample_lats, sample_lons) =
        downsample_pair(outlier_lats, outlier_lons, GEO_OUTLIER_GEOCODE_SAMPLE);
    let mut query: Vec<(f64, f64)> = coords.iter().map(|&(_, lat, lon)| (lat, lon)).collect();
    query.extend(
        sample_lats
            .iter()
            .zip(&sample_lons)
            .map(|(&lat, &lon)| (lat, lon)),
    );

    // a lookup failure (e.g. offline first-use) degrades to no metadata rather than failing viz.
    let mut labels = crate::cmd::geocode::reverse_geocode_points(&query, None).ok()?;
    // split: first 5 -> core corners/center, remainder -> the outlier sample
    let outlier_labels = labels.split_off(coords.len());

    let points: Vec<GeoPoint> = coords
        .iter()
        .zip(labels)
        .map(|(&(tag, lat, lon), label)| GeoPoint {
            tag,
            lat,
            lon,
            label,
        })
        .collect();

    let core_summary = consolidate_geo(&points);
    if core_summary.is_empty() {
        return None;
    }
    // suppress the call-out when the outliers fall within the core's own jurisdiction(s) — they're
    // the cluster's far edge, not strays "elsewhere", so naming them would just be redundant noise.
    let outliers_in_summary =
        n_outliers != 0 && !outliers_share_core_region(&points, &outlier_labels);
    let mut summary = if outliers_in_summary {
        outlier_summary(
            &core_summary,
            n_outliers,
            &outlier_jurisdictions(&outlier_labels),
        )
    } else {
        core_summary
    };
    // under `--smarter`, annotate with the extent's shared continent. A second, small batched
    // engine load over the <=5 distinct extent countries. Derive the context from EXACTLY the
    // jurisdictions the summary names — the core plus the outliers when their call-out is
    // present — so a cross-continent outlier suppresses the note rather than contradicting it
    // (e.g. a US core with a European outlier).
    if smarter {
        let mut ctx_labels: Vec<&crate::cmd::geocode::GeoLabel> =
            points.iter().filter_map(|p| p.label.as_ref()).collect();
        if outliers_in_summary {
            ctx_labels.extend(outlier_labels.iter().filter_map(Option::as_ref));
        }
        if let Some(note) = country_context_note(&ctx_labels) {
            summary.push_str(&note);
        }
    }
    // the full extent (core + all outliers) is drawn as a second, no-fill box. Only meaningful when
    // there ARE outliers; skip it if a stray pushes the box across the antimeridian (it would
    // wrap).
    let full_extent = (n_outliers > 0)
        .then(|| extent_including(core_extent, outlier_lats, outlier_lons))
        .filter(|fe| !extent_spans_antimeridian(fe));
    Some(GeoMeta {
        extent: core_extent,
        full_extent,
        points,
        summary,
    })
}

/// Outline/symbol color for the spatial-extent overlay drawn on `viz smart` maps. A deep purple,
/// deliberately distinct from the warm (red/orange) palette the data points/density use.
#[cfg(feature = "geocode")]
const GEO_EXTENT_LINE_COLOR: &str = "#6a1b9a";
/// Translucent fill for the spatial-extent bounding box (kept faint so it doesn't obscure data).
#[cfg(feature = "geocode")]
const GEO_EXTENT_FILL_COLOR: &str = "rgba(106, 27, 154, 0.12)";
/// White halo around extent markers, for contrast against both the basemap and the data points.
#[cfg(feature = "geocode")]
const GEO_EXTENT_MARKER_BORDER: &str = "#ffffff";
/// Marker size (px) for the reverse-geocoded extent corner/center points — deliberately large so
/// they read as annotations, not data.
#[cfg(feature = "geocode")]
const GEO_EXTENT_MARKER_SIZE: usize = 16;
/// Bounding-box line width (px) for the spatial-extent frame.
#[cfg(feature = "geocode")]
const GEO_EXTENT_LINE_WIDTH: f64 = 3.0;
/// Line width (px) for the full-extent box.
#[cfg(feature = "geocode")]
const GEO_FULL_EXTENT_LINE_WIDTH: f64 = 2.5;
/// Line color for the full-extent box: a vivid magenta/fuchsia that stays visible on every basemap
/// layer (light land, green, blue water) and is distinct from the purple core box and amber
/// outliers.
#[cfg(feature = "geocode")]
const GEO_FULL_EXTENT_LINE_COLOR: &str = "#e6007e";
/// Fractional padding added to each side of the (core) extent box for the `ScatterGeo` exact-range
/// fit. Small, so the projection map frames the core cluster tightly.
#[cfg(feature = "geocode")]
const GEO_EXTENT_FIT_PAD_FRAC: f64 = 0.04;
/// Minimum absolute padding (degrees) for the `ScatterGeo` extent fit, so a near-degenerate box
/// still gets a sliver of margin.
#[cfg(feature = "geocode")]
const GEO_EXTENT_FIT_PAD_MIN_DEG: f64 = 0.1;

/// Add the spatial-extent bounding box (dotted filled outline) plus reverse-geocoded corner/center
/// points (drawn as hover-labeled diamond glyphs) to a MapLibre map `Plot`. When the panel has
/// geographic outliers, a second no-fill dotted magenta box marking the full extent (core +
/// outliers) is drawn underneath.
#[cfg(feature = "geocode")]
fn add_extent_overlay_map(plot: &mut Plot, meta: &GeoMeta) {
    // the full-extent box (core + outliers) is drawn first, underneath, with no fill. The raster
    // basemap ignores line.dash, so the dashed look is drawn as a gapped polyline.
    if let Some(full) = &meta.full_extent {
        let (flat, flon) = dashed_box_latlon(full);
        plot.add_trace(
            ScatterMap::new(flat, flon)
                .name(&t!("viz.map.full_extent"))
                .mode(Mode::Lines)
                .line(full_extent_box_line())
                .hover_info(HoverInfo::Skip)
                .show_legend(false),
        );
    }
    let (blat, blon) = extent_box_latlon(&meta.extent);
    plot.add_trace(
        ScatterMap::new(blat, blon)
            .name(&t!("viz.map.spatial_extent"))
            .mode(Mode::Lines)
            .line(extent_box_line())
            .fill(plotly::traces::scatter_map::Fill::ToSelf)
            .fill_color(GEO_EXTENT_FILL_COLOR)
            .hover_info(HoverInfo::Skip)
            .show_legend(false),
    );
    let mlat: Vec<f64> = meta.points.iter().map(|p| p.lat).collect();
    let mlon: Vec<f64> = meta.points.iter().map(|p| p.lon).collect();
    let htext: Vec<String> = meta.points.iter().map(point_hover_text).collect();
    plot.add_trace(
        ScatterMap::new(mlat, mlon)
            .name(&t!("viz.map.extent_points"))
            .mode(Mode::Markers)
            .marker(extent_marker_map())
            .hover_text_array(htext)
            .hover_info(HoverInfo::Text)
            .show_legend(false),
    );
}

/// Like `add_extent_overlay_map` (including the optional full-extent box), but for an offline
/// `ScatterGeo` projection `Plot` (used for continental/global extents and static image export).
/// Hover labels are inert in static images, so the static export simply shows the box + diamond
/// glyphs, as intended.
#[cfg(feature = "geocode")]
fn add_extent_overlay_geo(plot: &mut Plot, meta: &GeoMeta) {
    // the full-extent box (core + outliers) is drawn first, underneath, with no fill
    if let Some(full) = &meta.full_extent {
        let (flat, flon) = extent_box_latlon(full);
        plot.add_trace(
            ScatterGeo::new(flat, flon)
                .name(&t!("viz.map.full_extent"))
                .mode(Mode::Lines)
                .line(full_extent_box_line())
                .hover_info(HoverInfo::Skip)
                .show_legend(false),
        );
    }
    let (blat, blon) = extent_box_latlon(&meta.extent);
    plot.add_trace(
        ScatterGeo::new(blat, blon)
            .name(&t!("viz.map.spatial_extent"))
            .mode(Mode::Lines)
            .line(extent_box_line())
            .fill(plotly::traces::scatter_geo::Fill::ToSelf)
            .fill_color(GEO_EXTENT_FILL_COLOR)
            .hover_info(HoverInfo::Skip)
            .show_legend(false),
    );
    let mlat: Vec<f64> = meta.points.iter().map(|p| p.lat).collect();
    let mlon: Vec<f64> = meta.points.iter().map(|p| p.lon).collect();
    let htext: Vec<String> = meta.points.iter().map(point_hover_text).collect();
    plot.add_trace(
        ScatterGeo::new(mlat, mlon)
            .name(&t!("viz.map.extent_points"))
            .mode(Mode::Markers)
            .marker(extent_marker_geo())
            .hover_text_array(htext)
            .hover_info(HoverInfo::Text)
            .show_legend(false),
    );
}

/// Marker color for geographic outlier points on `viz smart` maps. A vivid amber, deliberately
/// distinct from BOTH the warm (red/orange) data points AND the deep-purple extent overlay, so all
/// three layers read as separate. Not geocode-gated — outlier styling works in every `viz` build.
const GEO_OUTLIER_COLOR: &str = "#f9a825";
/// Marker size (px) for outlier points — larger than the data points so the handful of strays stand
/// out at a glance, even when they sit outside the default (core-fit) viewport.
const GEO_OUTLIER_MARKER_SIZE: usize = 11;
/// White halo around outlier markers, for contrast against both tile and projection basemaps.
const GEO_OUTLIER_MARKER_BORDER: &str = "#ffffff";

/// Outlier marker for MapLibre tile maps: a haloed amber circle. MapLibre can't render custom
/// symbols, so the distinction comes from size + color versus the data points.
fn outlier_marker_map() -> Marker {
    Marker::new()
        .color(GEO_OUTLIER_COLOR)
        .size(GEO_OUTLIER_MARKER_SIZE)
        .opacity(0.95)
        .line(Line::new().color(GEO_OUTLIER_MARKER_BORDER).width(1.5))
}

/// Like `outlier_marker_map`, but an X glyph — `ScatterGeo` renders plotly symbols, giving the
/// outliers a distinct SHAPE in addition to color (and a readable glyph in inert static images).
fn outlier_marker_geo() -> Marker {
    outlier_marker_map().symbol(plotly::common::MarkerSymbol::X)
}

/// Maximum number of tiles in the leading `viz smart` KPI overview row (dataset-summary tiles +
/// headline measures). Keeps the row readable and each gauge/number a legible size.
const KPI_MAX_TILES: usize = 6;

/// Choose a plotly d3 value-format for a KPI number: SI-suffixed for large magnitudes ("1.05M",
/// "258k"), a grouped integer for whole numbers, else two decimals.
fn kpi_number_format(value: f64) -> String {
    let a = value.abs();
    if a >= 10_000.0 {
        ".3~s".to_string()
    } else if value.fract() == 0.0 {
        ",.0f".to_string()
    } else if a < 0.01 && a > 0.0 {
        // ",.2f" renders anything under 0.005 as a flat "0.00" — a dictionary-tagged ratio with
        // gauge_range [0, 0.01] and mean 0.0004 showed the headline "0.00" beside a needle
        // sitting at 4% of its scale. Use significant digits instead of fixed decimals.
        ".3~g".to_string()
    } else {
        ",.2f".to_string()
    }
}

/// Build one KPI `Indicator` trace for a tile, positioned by paper-domain `[x0,x1] × [y0,y1]`.
/// A validated `gauge` adds an angular gauge, a `target` adds a "vs target" delta; the `mode`
/// follows whichever are present (always including the big number). The tile LABEL is NOT set as
/// the indicator's built-in title (which renders large, above the number, and overruns narrow
/// tiles for long auto-generated labels) — the caller instead draws it as a smaller, word-wrapped
/// subtitle annotation BELOW the tile via `kpi_label_annotation`, so the indicator here owns only
/// the number/gauge/delta and leaves the bottom band of its domain for that label.
fn kpi_indicator(tile: &KpiTile, x_domain: [f64; 2], y_domain: [f64; 2]) -> Box<dyn Trace> {
    let mode = match (tile.gauge.is_some(), tile.target.is_some()) {
        (true, true) => IndicatorMode::NumberAndDeltaAndGauge,
        (true, false) => IndicatorMode::NumberAndGauge,
        (false, true) => IndicatorMode::NumberAndDelta,
        (false, false) => IndicatorMode::Number,
    };
    let mut ind = Indicator::new(tile.value)
        .mode(mode)
        .domain(Domain::new().x(&x_domain).y(&y_domain))
        .number(Number::new().value_format(tile.format.clone()));
    if let Some([g_lo, g_hi]) = tile.gauge {
        ind = ind.gauge(Gauge::new().axis(GaugeAxis::new().range([g_lo, g_hi])));
    }
    if let Some(target) = tile.target {
        // a "vs target" delta — the reference is a semantically-justified goal from the dictionary,
        // never a fabricated prior-period baseline.
        ind = ind.delta(Delta::new().reference(target));
    }
    ind
}

/// Greedy word-wrap a KPI tile label into `<br>`-separated lines of at most `max_chars`
/// characters, so a long auto-generated label (e.g. "Mean Profit Margin Percentage") renders as a
/// compact multi-line subtitle under its tile instead of overrunning the tile width and colliding
/// with its neighbours. A single word longer than `max_chars` is left intact (never hard-split
/// mid-word — a broken token reads worse than a slightly-wide one).
fn wrap_kpi_label(text: &str, max_chars: usize) -> String {
    let max = max_chars.max(1);
    let mut lines: Vec<String> = Vec::new();
    let mut cur = String::new();
    for word in text.split_whitespace() {
        if cur.is_empty() {
            cur.push_str(word);
        } else if cur.chars().count() + 1 + word.chars().count() <= max {
            cur.push(' ');
            cur.push_str(word);
        } else {
            lines.push(std::mem::take(&mut cur));
            cur.push_str(word);
        }
    }
    if !cur.is_empty() {
        lines.push(cur);
    }
    lines.join("<br>")
}

/// Marks a KPI tile's subtitle annotation so [`KPI_REWRAP_SCRIPT`] can find it in `gd.layout`
/// without depending on annotation INDICES, which differ between the two emission sites (the smart
/// grid interleaves KPI labels into a shared `annotations` vec; the standalone KPI row owns its
/// own). `name` is a real plotly annotation attribute, so it survives the round-trip into the page.
const KPI_LABEL_MARK: &str = "qsv-kpi-label";

/// The word-wrapped subtitle label for one KPI tile, centered under the tile at paper-y `y_top`
/// (anchored at its top so it hangs down into the band reserved below the indicator). Smaller than
/// the number it captions; `font` follows the Data Schematic's themed/non-themed convention.
///
/// `max_chars` is a CHARACTER budget, which cannot track the tile's pixel width — the same page
/// renders its KPI row anywhere from ~580px wide (narrow window, or the `--dict-info` drawer open)
/// to ~1400px. This wrap is therefore the pre-JS baseline only; [`KPI_REWRAP_SCRIPT`] re-wraps
/// against the measured width once the page is live.
fn kpi_label_annotation(
    label: &str,
    x_center: f64,
    y_top: f64,
    max_chars: usize,
    font: Font,
) -> Annotation {
    Annotation::new()
        .text(wrap_kpi_label(label, max_chars))
        .name(KPI_LABEL_MARK)
        .x(x_center)
        .y(y_top)
        .x_ref("paper")
        .y_ref("paper")
        .x_anchor(Anchor::Center)
        .y_anchor(Anchor::Top)
        .show_arrow(false)
        .font(font)
}

/// Build the leading KPI-tile row for `viz smart`: the headline numeric measures (capped at
/// `KPI_MAX_TILES`). A measure tile becomes a gauge when its dictionary supplies a `gauge_range`
/// that CONTAINS the observed value (else it falls back to a plain number — never a misleading
/// gauge), and a "vs target" delta when the dictionary supplies a `target` (a semantically
/// justified goal, never a fabricated prior-period baseline). Returns `None` when no measure tile
/// results (dataset completeness now lives in the Data Schematic header, not as a KPI tile, so the
/// row is worth drawing only when there is at least one measure to headline).
fn build_kpi_row(
    stats: &[crate::cmd::stats::StatsData],
    panels: &[Panel],
    dict: Option<&DictData>,
) -> Option<Panel> {
    if stats.is_empty() {
        return None;
    }
    let mut tiles: Vec<KpiTile> = Vec::with_capacity(KPI_MAX_TILES);
    // Record/field counts and overall completeness are intentionally NOT tiles — record/field
    // counts duplicate the "Rows"/"Columns" header lines, and completeness (near 100% for most
    // datasets) now rides quietly in the header table below "Columns:". The KPI row leads purely
    // with the headline MEASURES.
    //
    // headline measures: the numeric columns the Data Schematic already treats as continuous
    // measures (i.e. those that earned a box/violin/histogram distribution panel), summed
    // (extensive) or averaged (intensive), up to the remaining tile budget. Keying off the
    // built panels keeps the KPI measures consistent with the charts and works at any data size
    // (no re-classification).
    for p in panels {
        if tiles.len() >= KPI_MAX_TILES {
            break;
        }
        if !matches!(
            p.kind,
            PanelKind::BoxStats { .. }
                | PanelKind::BoxRaw { .. }
                | PanelKind::BoxOutliers { .. }
                | PanelKind::Violin { .. }
                | PanelKind::Histogram { .. }
        ) {
            continue;
        }
        // join on the recorded stats index, NOT on `p.name` — classification decorates the title
        // after construction (box shape hints, "(sampled)"), and on headerless input `name` is a
        // positional "col N" while `s.field` is empty, so a name-based lookup silently misses.
        let Some(s) = p.stat_idx.and_then(|i| stats.get(i)) else {
            continue;
        };
        if !matches!(s.r#type.as_str(), "Integer" | "Float") {
            continue;
        }
        let row = dict.and_then(|d| d.rows.get(&s.field));
        let label = match row {
            Some(r) if !r.label.is_empty() => r.label.clone(),
            // headerless input has an empty `field`, which would render a bare "Total " — fall
            // back to the same positional name the panel itself uses.
            _ if s.field.is_empty() => format!("col {}", p.stat_idx.map_or(0, |i| i + 1)),
            _ => s.field.clone(),
        };
        // A `gauge_range` describes a bounded per-record quantity (a 0–5 rating, a 0–1 ratio), so
        // its headline value is the MEAN — summing it would blow past the gauge scale. Otherwise
        // fall back to the extensive/intensive keyword heuristic.
        let has_range = row.and_then(|r| r.gauge_range).is_some();
        let intensive = has_range || is_intensive_measure(&label, &s.field);
        let Some(value) = (if intensive { s.mean } else { s.sum }).filter(|v| v.is_finite()) else {
            continue;
        };
        // gauge only when a range is supplied AND actually contains the value (guardrail against a
        // misleading gauge); delta only when a finite target is supplied.
        let gauge = row
            .and_then(|r| r.gauge_range)
            .filter(|[lo, hi]| lo < hi && value >= *lo && value <= *hi);
        let target = row.and_then(|r| r.target).filter(|t| t.is_finite());
        tiles.push(KpiTile {
            label: if intensive {
                t!("viz.title.kpi_mean", q_label = label)
            } else {
                t!("viz.title.kpi_total", q_label = label)
            }
            .into_owned(),
            value,
            format: kpi_number_format(value),
            gauge,
            target,
        });
    }

    // No measures to headline → no KPI row (completeness alone no longer justifies one).
    if tiles.is_empty() {
        return None;
    }

    Some(Panel::new(
        "Overview".to_string(),
        PanelKind::KpiRow { tiles },
    ))
}

/// Resolved inputs for `build_smart`, produced by `smart_prepare`.
struct SmartPrep {
    /// the caller's `Args` with `--bivariate`'s implications applied (it forces `--smarter`, and
    /// infers a `--dictionary` when none was given).
    args:                    Args,
    /// the path the STATS cache is keyed on: the validated input path (never stdin), with a `dc:`
    /// handle resolved ONCE here so the stats load and the `--dict-info` download row can't drift
    /// onto different files.
    stats_input:             String,
    /// non-default parsing (`--no-headers` / `--delimiter`) forces the stats cache to regenerate.
    force_stats_regen:       bool,
    /// set only when THIS run's moarstats `--bivariate` subprocess wrote a fresh sidecar.
    bivariate_sidecar_fresh: bool,
}

/// Validate `viz smart`'s input, apply `--bivariate`'s flag implications, and run the optional
/// `--smarter` moarstats enrichment pass (which rewrites the stats sidecar `build_smart` reads).
fn smart_prepare(args: &Args, progress: &ProgressBar, show_progress: bool) -> CliResult<SmartPrep> {
    let mut args = args.clone();

    let Some(input) = args.arg_input.clone() else {
        return fail_incorrectusage_clierror!(
            "`viz smart` requires a file input (it derives charts from the dataset's statistics)."
        );
    };
    if input == "-" {
        return fail_incorrectusage_clierror!(
            "`viz smart` cannot read from stdin; pass a CSV/TSV file path."
        );
    }
    // A dictionary is keyed by COLUMN NAME (`dict.rows.get(&s.field)`), but under `--no-headers`
    // the fields are positional (`0`, `1`, ...), so no entry can ever match: no semantic routing,
    // no KPI tiles, no dict subtitles. Say so and drop the flag, rather than let it look applied.
    // Dropping it matters beyond the notice — `--dictionary infer` would otherwise pay for a full
    // (LLM-backed) describegpt pass whose every verdict is then discarded.
    //
    // A notice rather than an error, matching the dictionary loader's fail-to-stats philosophy:
    // the Data Schematic still renders from stats alone (see
    // `viz_smart_bivariate_soft_fails_with_no_headers`).
    if args.flag_no_headers && args.flag_dictionary.is_some() {
        viz_note(
            "viz smart --dictionary: a data dictionary maps entries by column name, but \
             --no-headers names columns positionally (0, 1, 2, ...), so no entry can match. \
             Ignoring --dictionary and charting from statistics alone.",
        );
        args.flag_dictionary = None;
    }

    // `--bivariate` blows up combinatorially with wide datasets (50 choose 2 = 1,225 pairs at the
    // cap), and can't be honored at all under non-default parsing (its moarstats subprocess is
    // skipped just like `--smarter`'s, below). A cheap header-only read (no stats pass) decides
    // the column-count cap. Computed BEFORE the `--bivariate` implication block below so that a
    // column-cap/no-headers skip also skips the (potentially LLM-backed) `--dictionary infer`
    // implication -- otherwise a wide or `--no-headers` input would still pay for an LLM call
    // whose result the disabled bivariate panels would never use.
    let bivariate_enabled =
        if args.flag_bivariate && !(args.flag_no_headers || args.flag_delimiter.is_some()) {
            let ncols = reader_and_headers(&args)?.1.len();
            if ncols > BIVARIATE_MAX_COLUMNS {
                viz_note(&format!(
                    "viz smart --bivariate: {ncols} columns exceeds the \
                     {BIVARIATE_MAX_COLUMNS}-column cap (~{} pairs at the cap); skipping the \
                     bivariate association panels. Run `qsv moarstats --bivariate` directly for \
                     the full pairwise output.",
                    BIVARIATE_MAX_COLUMNS * (BIVARIATE_MAX_COLUMNS - 1) / 2
                ));
                false
            } else {
                true
            }
        } else {
            false
        };

    // `--bivariate` is a turnkey convenience: it forces `--smarter` (the moarstats subprocess
    // that writes the bivariate sidecar `--bivariate` itself enables below) and, unless the
    // caller already gave an explicit `--dictionary`, infers one too (so the new panels'
    // identifier/PII exclusion has a `Route::Skip` signal to work with). The dictionary-infer
    // implication is gated on `bivariate_enabled` (not the raw flag) so a column-cap/no-headers
    // skip doesn't still trigger an unused LLM call.
    if args.flag_bivariate {
        args.flag_smarter = true;
    }
    if bivariate_enabled && args.flag_dictionary.is_none() {
        args.flag_dictionary = Some("infer".to_string());
    }
    // (the caller receives `args` by value; returning it is what freezes further mutation, the
    // role the original inline code gave a `let args = &args;` rebinding here.)

    // Non-default parsing (--no-headers / custom --delimiter) forces the stats path to
    // regenerate. get_stats_records keys its `.stats.csv.data.jsonl` cache only by mtime + stat
    // sufficiency, NOT by parsing options, so without this a current default-parsing cache (e.g.
    // from an earlier headered `qsv stats` run) would be reused and the Data Schematic would render
    // from incorrectly parsed stats (row 1 as headers vs data, or a mis-split delimiter).
    let force_stats_regen = args.flag_no_headers || args.flag_delimiter.is_some();

    // set only when THIS run's moarstats --bivariate subprocess succeeds, gating the association
    // panels below. The sidecar path is deterministic from the input stem, so without this a
    // stale sidecar left by a prior run (on since-changed data) — or a failed/pair-less current
    // run — could be read as if it described the current input.
    let mut bivariate_sidecar_fresh = false;

    if args.flag_smarter {
        // moarstats --advanced computes its advanced stats by RE-READING the input itself: the
        // KGA pass via `Config::new(...).reader_file()` and the entropy pass via a `frequency`
        // subprocess, both using extension-based delimiter detection and assuming a header row
        // (moarstats has no --delimiter/--no-headers flags). So when viz smart is given a custom
        // --delimiter or --no-headers, moarstats would parse the data differently than viz does:
        // for --no-headers the base stats name columns 1,2,... while the KGA reader treats row 1
        // as headers, so the advanced columns never attach; for a custom delimiter the advanced
        // readers mis-split the rows entirely. In either case skip enrichment and let the standard
        // get_stats_records path below build a correct, freshly-regenerated cache
        // (force_stats_regen above) that honors the parsing flags.
        if args.flag_no_headers || args.flag_delimiter.is_some() {
            viz_note(
                "viz smart --smarter: moarstats enrichment is only applied with default parsing; \
                 --no-headers / --delimiter inputs use the standard Data Schematic instead.",
            );
        } else {
            // Enrich the stats cache with moarstats advanced distribution-shape stats so
            // classify()/box_shape_hint() can upgrade box->histogram, surface concentrated
            // high-cardinality FreqBars, and annotate box titles. moarstats rewrites the
            // .stats.csv.data.jsonl sidecar that get_stats_records reads below.
            //
            // moarstats' top-level --force guarantees the base `qsv stats` subprocess re-runs
            // with the full default --stats-options (cardinality/quartiles/skewness); without it
            // a stale, lean stats cache would be reused and the advanced columns viz needs
            // (bimodality_coefficient, normalized_entropy) would silently not be added.
            //
            // --stats-options mirrors moarstats' own default plus --dates-whitelist sniff, so the
            // regenerated cache infers dates the same way viz smart does (for the time-series
            // panel). Soft-fail: a moarstats error degrades to the plain ProfileSchema Data
            // Schematic.
            let stats_opts = "--infer-dates --infer-boolean --cardinality --mode --mad \
                              --quartiles --percentiles --force --stats-jsonl --dates-whitelist \
                              sniff";
            let mut moar_argv: Vec<&str> =
                vec!["--advanced", "--force", "--stats-options", stats_opts];
            // `--bivariate`: extend the SAME moarstats subprocess (no second invocation) with the
            // bivariate sidecar request. Always asks for nmi (the association heatmap's value,
            // works for numeric AND categorical pairs), u (Theil's directed uncertainty
            // coefficient, gates the directed-flow Sankey panel), plus pearson/spearman (their
            // divergence flags a numeric pair as nonlinear).
            // freshness bookkeeping for the bivariate sidecar, read after the moarstats run below;
            // defaults are never read unless `bivariate_enabled`.
            let mut sidecar_before: Option<std::time::SystemTime> = None;
            let mut sidecar_cleared = false;
            if bivariate_enabled {
                moar_argv.extend(["--bivariate", "--bivariate-stats", "nmi,u,pearson,spearman"]);
                // Remove any stale bivariate sidecar from a prior run BEFORE moarstats runs.
                // moarstats (re)writes it only when the fresh run yields pairs, so clearing it
                // first means a pair-less or failed current run leaves NO sidecar for the Data
                // Schematic to read (it then soft-fails) instead of charting
                // association stats computed for since-changed data. Snapshot the
                // pre-run mtime FIRST so that even if the removal fails (e.g. a
                // locked/read-only file), a stale copy whose mtime never advances is
                // still caught as not-fresh below rather than trusted.
                let sidecar = bivariate_csv_path(&input);
                sidecar_before = std::fs::metadata(&sidecar).and_then(|m| m.modified()).ok();
                sidecar_cleared = match std::fs::remove_file(&sidecar) {
                    Ok(()) => true,
                    Err(e) => e.kind() == std::io::ErrorKind::NotFound,
                };
            }
            // moarstats --advanced (+ --bivariate) is by far the longest phase (a full stats
            // recompute + advanced/pairwise passes over every row). On an interactive terminal,
            // stream moarstats' OWN live progress (its per-phase status lines and progress bars)
            // instead of a single static spinner line: suspend our spinner and let the child render
            // to the inherited stderr (its stdout is discarded so it can't corrupt ours). When
            // progress is disabled or output is piped/redirected, fall back to the capturing path
            // so nothing is emitted. Either way viz only needs the on-disk
            // stats/bivariate sidecars.
            let status_msg =
                "Enriched stats cache via moarstats --advanced for `viz smart --smarter`";
            let stream_moarstats = show_progress && std::io::stderr().is_terminal();
            progress.set_message("Analyzing column relationships…");
            let moar_result: CliResult<()> = if stream_moarstats {
                progress.suspend(|| {
                    util::run_qsv_cmd_streaming("moarstats", &moar_argv, &input, status_msg)
                })
            } else {
                util::run_qsv_cmd("moarstats", &moar_argv, &input, status_msg).map(|_| ())
            };
            if let Err(e) = &moar_result {
                viz_note(&format!(
                    "viz smart --smarter: moarstats enrichment failed ({e}); falling back to \
                     standard stats. Data Schematic will omit advanced refinements."
                ));
            }
            // Trust the sidecar only when THIS run actually wrote it: moarstats must have succeeded
            // AND a sidecar must now exist that is provably fresh — either the path was cleared
            // before the run (so any file present was written just now) or, if the pre-delete
            // failed, its mtime advanced past the pre-run snapshot. A present-but-unrefreshed file
            // is a stale copy the delete couldn't remove: skip the panels (with a notice) rather
            // than read the wrong data. An absent sidecar means no pairs — silently skip.
            if bivariate_enabled && moar_result.is_ok() {
                let sidecar_after = std::fs::metadata(bivariate_csv_path(&input))
                    .and_then(|m| m.modified())
                    .ok();
                bivariate_sidecar_fresh =
                    bivariate_sidecar_is_fresh(sidecar_before, sidecar_after, sidecar_cleared);
                // a sidecar present but NOT fresh is a stale copy the pre-delete couldn't remove;
                // warn so the missing panels aren't a mystery (an absent sidecar just means no
                // pairs — stay silent there).
                if !bivariate_sidecar_fresh && sidecar_after.is_some() {
                    viz_note(
                        "viz smart --bivariate: a stale bivariate sidecar could not be removed \
                         and was not refreshed by moarstats; skipping the association panels.",
                    );
                }
            }
        }
    }

    Ok(SmartPrep {
        // resolved HERE, once, so the stats load and the `--dict-info` download row are anchored
        // on the same file by construction rather than by two resolutions happening to agree
        stats_input: resolve_stats_input_path(&input),
        args,
        force_stats_regen,
        bivariate_sidecar_fresh,
    })
}

/// Resolve a `dc:<name>` disk-cache handle to the materialized CSV the stats cache is keyed on,
/// mirroring `util::get_stats_records`. A no-op for every ordinary path.
///
/// Call this EXACTLY ONCE per run, in the data phase (`smart_prepare`), and reuse the result:
/// `diskcache::resolve_dc_path` auto-refreshes a stale entry — a network fetch that can
/// materialize a DIFFERENT CSV — so two resolutions within one run can disagree. Feeding the
/// result to BOTH `get_stats_records` (which then has no `dc:` prefix left to re-resolve) and the
/// `--dict-info` stats anchor is what makes the offered sidecars provably the ones the Data
/// Schematic's stats came from.
///
/// A resolution failure falls back to the raw input, which simply won't canonicalize later — no
/// stats downloads, rather than a failed Data Schematic.
fn resolve_stats_input_path(input: &str) -> String {
    #[cfg(feature = "get")]
    if let Some(dc_name) = input.strip_prefix("dc:") {
        return match crate::diskcache::resolve_dc_path(dc_name) {
            Ok(p) => p.to_string_lossy().into_owned(),
            Err(_) => input.to_string(),
        };
    }
    input.to_string()
}

/// Per-panel data gathered once the panel set is final, consumed by the render dispatch.
struct SmartPanelData {
    freq:          FreqMap,
    raw_values:    HashMap<usize, Vec<f64>>,
    outlier_stats: HashMap<usize, OutlierStats>,
    title_text:    String,
    dict_page:     Option<String>,
    metadata_html: Option<String>,
    /// the data viewer drawer chrome (issue #4283): library + payloads + drawer script.
    /// HTML-only, `None` when disabled via `--preview-threshold 0`.
    data_chrome:   Option<String>,
}

/// Shared working set threaded through `build_smart`'s phases.
///
/// `build_smart` accumulates a large, interdependent set of locals (the stats/dictionary inputs,
/// the growing `panels` vector, the deferred diagnostics). Holding them in one value lets the
/// phases below be methods at a readable arity, instead of free functions taking 15-20 positional
/// parameters — which would move the complexity into the signatures rather than remove it.
struct SmartCtx<'a> {
    args:       &'a Args,
    out_format: OutFormat,
    progress:   &'a ProgressBar,

    stats:          Vec<crate::cmd::stats::StatsData>,
    col_sems:       Vec<ColSemantics>,
    /// per-column DMY-vs-MDY parsing preference: the dictionary's declared date format when it
    /// expresses one, otherwise `QSV_PREFER_DMY` (issue #4303). See `dmy_pref`.
    dmy_prefs:      Vec<bool>,
    dict_data:      Option<DictData>,
    dict_json_text: Option<String>,
    twin_suppress:  std::collections::HashSet<usize>,

    /// the geographic overview panels, built up front and prepended last so they lead the
    /// Data Schematic and survive the panel cap.
    map_panel:         Option<(Panel, (usize, usize))>,
    choropleth_panel:  Option<Panel>,
    summary_choros:    Option<(Vec<Panel>, usize)>,
    map_cols:          Option<(usize, usize)>,
    summary_choro_col: Option<usize>,

    /// set only when THIS run's moarstats `--bivariate` subprocess wrote a fresh sidecar; gates
    /// the association/Sankey panels.
    bivariate_sidecar_fresh: bool,

    /// the source→target pair claimed by the directed-flow (Sankey) panel, if one was built —
    /// keeps the 2-level hierarchy mutually exclusive with it.
    sankey_pair:    Option<(usize, usize)>,
    /// at most ONE animated panel is shown; precedence is T3 (bubble) > T2 (geo) > T1 (scatter).
    bubble_panel:   Option<Panel>,
    geo_anim_panel: Option<Panel>,
    /// the pipeline funnel (issue #4222), built during the correlation pass while the
    /// row-aligned columns are live and inserted later with the other overview panels.
    funnel_panel:   Option<Panel>,

    explicit_box_points: Option<BoxPoints>,
    count_conf:          Config,
    /// the path the stats cache was loaded from — `dc:` resolved ONCE in `smart_prepare`, so the
    /// `--dict-info` download row anchors on exactly the file `get_stats_records` read.
    stats_input:         String,
    nrows:               Option<u64>,
    log_scale:           LogScale,
    violin_mode:         ViolinMode,

    panels:  Vec<Panel>,
    skipped: Vec<String>,

    sentinel_suspects:   Vec<String>,
    sentinel_hints:      Vec<String>,
    nonnumeric_measures: Vec<String>,

    /// set by `finalize_panels`: HTML output rendered as an inline-div grid rather than the typed
    /// subplot grid.
    inline: bool,
}

impl<'a> SmartCtx<'a> {
    /// Load the dataset's statistics and optional data dictionary, derive per-column semantics,
    /// and build the geographic panels up front (the map panel's resolved lat/lon pair decides
    /// which columns the per-column phases must skip).
    fn new(
        prep: &'a SmartPrep,
        out_format: OutFormat,
        progress: &'a ProgressBar,
        loaded_geojson: Option<&serde_json::Value>,
    ) -> CliResult<Self> {
        let args = &prep.args;

        let schema_args = util::SchemaArgs {
            flag_enum_threshold:  0,
            flag_ignore_case:     false,
            flag_strict_dates:    false,
            flag_strict_formats:  false,
            flag_pattern_columns: SelectColumns::parse("").expect("empty selection is valid"),
            flag_dates_whitelist: "sniff".to_string(),
            flag_prefer_dmy:      false,
            flag_force:           prep.force_stats_regen,
            flag_stdout:          false,
            flag_jobs:            None,
            flag_polars:          false,
            flag_no_headers:      args.flag_no_headers,
            flag_delimiter:       args.flag_delimiter,
            arg_input:            Some(prep.stats_input.clone()),
            flag_memcheck:        false,
            flag_output:          None,
        };

        // Keep the spinner live: get_stats_records reuses the cache or runs `qsv stats` as a
        // captured subprocess, so nothing here writes to the terminal to clash with the bar.
        progress.set_message("Loading statistics…");
        let (_headers, mut stats) =
            util::get_stats_records(&schema_args, util::StatsMode::ProfileSchema)?;
        if stats.is_empty() {
            return fail_clierror!(
                "Could not compute statistics for `viz smart`. The input must be a regular \
                 CSV/TSV file (not stdin or a compressed/special format)."
            );
        }

        // Plain `viz smart` (no `--smarter`): the stats cache has `skewness` but not the
        // moarstats-only `kurtosis`/`bimodality_coefficient`, so a bimodal column would
        // render as a box plot that hides its peaks. Compute Sarle's bimodality coefficient
        // in one streaming pass so `classify_measure` can upgrade such a column to a
        // histogram WITHOUT requiring `--smarter`. No-op (and no pass) when moarstats
        // already enriched the cache or no column qualifies. Soft-fail to a box plot.
        if let Err(e) = enrich_bimodality(args, &mut stats) {
            viz_note(&format!(
                "viz smart: bimodality detection failed ({e}); bimodal columns may render as box \
                 plots. Use --smarter for moarstats-based enrichment."
            ));
        }

        // Optional describegpt Data Dictionary (--dictionary): per-column semantic verdicts
        // (concept -> role -> content_type -> stats) that override the statistical guess
        // for the cases they speak to. When --dictionary is absent, dict_data is None and
        // every column's route resolves to Defer, so classification is identical to today's
        // stats-only behavior. Keep the spinner live during describegpt (the slowest
        // phase): its output is captured by run_qsv_cmd, and load_dictionary_semantics
        // suspends the bar only around its own messages.
        progress.set_message("Inferring data dictionary…");
        let (dict_data, dict_json_text) = match load_dictionary_semantics(args)? {
            Some((data, json_text)) => (Some(data), Some(json_text)),
            None => (None, None),
        };

        // Second (lower-precedence) half of the language resolution begun in run(): now that a
        // dictionary exists, let its detected language pick the UI language -- but only if
        // --language was not given, which `resolve` enforces by checking the flag first.
        //
        // ORDERING INVARIANT: this must stay ahead of ALL HTML assembly. The locale is
        // process-global, so anything that calls `t!` or `active_locale()` before this point
        // renders in the pre-dictionary language, and the failure is invisible -- English
        // strings inside a Spanish page read as a missing translation, not as a bug. Both
        // `<html lang>` sites and every chrome string are built downstream of here;
        // viz_smart_dict_page_shares_the_dashboard_language pins that for both pages.
        //
        // A
        // language qsv has no translations for is a note, not an error: the data being in
        // Vietnamese is not a user mistake.
        match viz_i18n::resolve(
            args.flag_language.as_deref(),
            dict_data
                .as_ref()
                .and_then(|d| d.detected_language.as_deref()),
        ) {
            viz_i18n::Resolution::Curated(row) => viz_i18n::set_active(row),
            viz_i18n::Resolution::UncuratedDetected(code) => {
                progress.suspend(|| {
                    winfo!(
                        "Note: detected language '{code}' is not in the curated localization set \
                         ({}); rendering the Data Schematic in English.",
                        viz_i18n::curated_list()
                    );
                });
            },
            // Already rejected in run() before any work started.
            viz_i18n::Resolution::UnknownRequested(_) => {},
        }
        let col_sems: Vec<ColSemantics> = stats
            .iter()
            .map(|s| derive_semantics(s, dict_data.as_ref().and_then(|d| d.rows.get(&s.field))))
            .collect();

        // Resolve the date-parsing preference per column ONCE (issue #4303). A dictionary that
        // declares a column's strftime format (`content_type: "date:%m/%d/%Y"`) is authoritative
        // for that column; QSV_PREFER_DMY is the fallback wherever the dictionary is silent, so a
        // Data Schematic built without a dictionary is byte-identical to before.
        let env_dmy = util::get_envvar_flag("QSV_PREFER_DMY");
        let dmy_prefs: Vec<bool> = stats
            .iter()
            .map(|s| {
                dict_dmy_preference(dict_data.as_ref().and_then(|d| d.rows.get(&s.field)))
                    .unwrap_or(env_dmy)
            })
            .collect();

        // De-duplicate code/label twins (subject + subject_code -> chart only "subject"). Gated on
        // a dictionary being present so a stats-only Data Schematic is byte-identical; timezone
        // twins and IDs are already de-duplicated by the Temporal/Skip routing above.
        let mut twin_suppress = if dict_data.is_some() {
            dimension_code_twins(&stats, &col_sems)
        } else {
            std::collections::HashSet::new()
        };
        // ... and the same redundancy detected from the DATA rather than the names: two
        // categorical columns in a 1:1 correspondence chart as identical bars and waste a parcats
        // axis on the same variable (issue #4221). Deliberately NOT dictionary-gated — the name
        // rule above can only fire on a `<base>_code` spelling, and the reporting Data Schematic
        // has neither that spelling nor a dictionary. Soft-fail: a Data Schematic with a
        // duplicate panel beats no Data Schematic.
        match one_to_one_categorical_twins(args, &stats, &col_sems) {
            Ok(dupes) => twin_suppress.extend(dupes),
            Err(e) => viz_note(&format!(
                "viz smart: 1:1 dimension detection failed ({e}); redundant categorical panels \
                 may appear."
            )),
        }

        // Build the geographic map panel up front (one data pass) so we can learn which lat/lon
        // columns it ACTUALLY consumed. Those columns are charted on the map only —
        // excluded from per-column distribution panels, the correlation matrix, and the
        // time-series y so a map Data Schematic doesn't redundantly box/histogram its
        // coordinates or plot e.g. latitude vs time.
        //
        // The map panel is built for BOTH HTML and static image output. The MapLibre tile basemap
        // (ScatterMap/DensityMap) needs a live browser + network tiles, so it can't be statically
        // exported — but the offline `ScatterGeo` projection basemap CAN (it's how the standalone
        // `viz geo` command exports images). So for image output a local-extent `Map` panel is
        // coerced to the `Geo` form, fit to the data's extent (see `geo_framing`). When
        // build_map_panel returns None (no usable lat/lon pair), map_cols stays None and
        // those columns are charted normally. (map_panel, choropleth_panel): the point map
        // plus an optional geocode-derived per-country choropleth overview. The choropleth
        // is non-cartesian and HTML-only, so it's dropped for image export; the point map
        // is coerced from Map -> Geo for image export (only the offline ScatterGeo basemap
        // can be statically exported).
        let (map_panel, choropleth_panel) = {
            // prefer dictionary geo.latitude/geo.longitude (handles non-standard coord names like
            // `X Coordinate`/`Y Coordinate`), falling back to the header-name heuristic.
            let coord_hint = dict_data.as_ref().and_then(|d| semantic_latlon(&stats, d));
            let cluster_mode = parse_cluster_mode(&args.flag_cluster)?;
            progress.set_message("Building map panel…");
            match build_map_panel(
                args,
                &stats,
                &col_sems,
                dict_data.as_ref(),
                coord_hint,
                cluster_mode,
                loaded_geojson,
                // the cross-link is pure JS chrome: an image export has no drawer to link to, and
                // `--preview-threshold 0` disables the drawer outright. Gating here keeps both
                // payloads byte-identical to before the feature existed.
                matches!(out_format, OutFormat::Html) && args.flag_preview_threshold != 0,
            )? {
                None => (None, None),
                Some((p, choro, cols, mappable_count)) => {
                    if out_format.is_image() {
                        let kind = match p.kind {
                            PanelKind::Map {
                                lats,
                                lons,
                                outlier_lats,
                                outlier_lons,
                                hover_text,
                                outlier_hover_text,
                                sizes,
                                ..
                            } => PanelKind::Geo {
                                lats,
                                lons,
                                outlier_lats,
                                outlier_lons,
                                hover_text,
                                outlier_hover_text,
                                sizes,
                                // a static image has no drawer to cross-link to; `capture_row_ids`
                                // is already false on this path, so these are empty either way
                                row_ids: Vec::new(),
                                outlier_row_ids: Vec::new(),
                            },
                            other => other,
                        };
                        let p = Panel {
                            name: p.name,
                            // the coerced Geo renders MARKERS, never a density heatmap, so the
                            // sampling subtitle's "aggregating" verb (chosen by `sample_note` for
                            // the HTML DensityMap render) must follow the actual render mode.
                            subtitle: p.subtitle.map(|s| match s.strip_prefix("aggregating ") {
                                Some(rest) => format!("showing {rest}"),
                                None => s,
                            }),
                            kind,
                            value_log: p.value_log,
                            axis_log: p.axis_log,
                            interest: p.interest,
                            dict_info: p.dict_info,
                            stat_idx: p.stat_idx,
                            #[cfg(feature = "geocode")]
                            geo_meta: p.geo_meta,
                            geojson_overlay: p.geojson_overlay,
                        };
                        (Some((p, cols)), None)
                    } else {
                        // HTML output: a local-extent density panel actually renders as a
                        // DensityMap here (global extents are `Geo`
                        // markers, image export was coerced above), so this
                        // is the point at which the heatmap/cluster note is truthful. Report
                        // `mappable_count` (the full source count that drove the density decision),
                        // not the panel's downsampled/outlier-excluded
                        // `lats`.
                        if matches!(p.kind, PanelKind::Map { density: true, .. }) {
                            viz_note(&format!(
                                "viz smart: map has {mappable_count} mappable points (>= \
                                 --heatmap-density {}); drawing the core as a density heatmap \
                                 with per-point hover. Set --heatmap-density 0 (the default) to \
                                 draw interactive clustered markers instead.",
                                args.flag_heatmap_density
                            ));
                        } else if matches!(p.kind, PanelKind::Map { cluster: true, .. }) {
                            viz_note(&format!(
                                "viz smart: map has {mappable_count} mappable points; the core \
                                 opens as individual points with a \"Clusters/Points\" toggle \
                                 (click it to collapse dense areas into native MapLibre count \
                                 bubbles that expand again on zoom-in). Hover a bubble for its \
                                 point count, or click it to zoom to where it breaks apart. Set \
                                 --cluster off to omit the toggle, or --heatmap-density <n> to \
                                 draw a density heatmap at or above <n> points instead."
                            ));
                        }
                        (Some((p, cols)), choro)
                    }
                },
            }
        };
        let map_cols = map_panel.as_ref().map(|(_, cols)| *cols);

        // region-code summary choropleth(s): drive per-region aggregates (count + median measure)
        // directly from a geo dimension column (e.g. a zip column), independent of lat/lon. Only
        // when build_map_panel emitted no choropleth companion (avoid two competing region
        // fills), and never for image output (choropleths are HTML-only, like the PIP
        // companion above).
        let summary_choros = if choropleth_panel.is_none() && !out_format.is_image() {
            build_smart_summary_choropleth_panels(args, &stats, &col_sems, loaded_geojson)?
        } else {
            None
        };
        let summary_choro_col = summary_choros.as_ref().map(|(_, c)| *c);

        // Box-points handling for `viz smart`. A continuous-numeric column is normally a cache-only
        // quartile box (no data re-scan). When points should be overlaid, it instead becomes a raw
        // box (one extra batched pass, below) so plotly can draw true Tukey whiskers and
        // the sample points. The overlay mode is the user's explicit `--box-points` when
        // given, otherwise a size-based heuristic decides per the dataset's row count (see
        // `smart_box_points`). The row count is pulled once from the stats/index cache, and
        // only when the first box panel actually needs it.
        let explicit_box_points: Option<BoxPoints> = match args.flag_box_points.as_deref() {
            Some(s) => Some(parse_box_points(Some(s))?),
            None => None,
        };
        let count_conf = Config::new(args.arg_input.as_ref())
            .delimiter(args.flag_delimiter)
            .no_headers_flag(args.flag_no_headers);
        let nrows: Option<u64> = None;
        // resolved here (not at render time) because box panels bake their value-axis log verdict
        // into the Panel during classification below.
        let log_scale = parse_log_scale(&args.flag_log_scale)?;
        let violin_mode = parse_violin_mode(&args.flag_violin)?;

        Ok(SmartCtx {
            args,
            out_format,
            progress,
            stats,
            col_sems,
            dmy_prefs,
            dict_data,
            dict_json_text,
            twin_suppress,
            map_panel,
            choropleth_panel,
            summary_choros,
            map_cols,
            summary_choro_col,
            bivariate_sidecar_fresh: prep.bivariate_sidecar_fresh,
            sankey_pair: None,
            bubble_panel: None,
            geo_anim_panel: None,
            funnel_panel: None,
            explicit_box_points,
            count_conf,
            stats_input: prep.stats_input.clone(),
            nrows,
            log_scale,
            violin_mode,
            panels: Vec::new(),
            skipped: Vec::new(),
            sentinel_suspects: Vec::new(),
            sentinel_hints: Vec::new(),
            nonnumeric_measures: Vec::new(),
            inline: false,
        })
    }

    /// The dataset's row count, pulled once from the stats/index cache and memoized — the lazy
    /// fetch shared by the box-points heuristic, the metadata table and the panel builders.
    fn row_count(&mut self) -> u64 {
        // field-precise destructuring: `nrows` is borrowed mutably while `count_conf` is read.
        let Self {
            nrows, count_conf, ..
        } = self;
        *nrows.get_or_insert_with(|| util::count_rows(count_conf).unwrap_or(0))
    }

    /// A column consumed by the point map (lat/lon) OR by the summary choropleth (region key) is
    /// suppressed from the per-column distribution panels and the correlation/bivariate pools.
    fn is_map_col(&self, idx: usize) -> bool {
        self.map_cols.is_some_and(|(la, lo)| idx == la || idx == lo)
            || self.summary_choro_col == Some(idx)
    }

    /// The DMY-vs-MDY preference to parse column `idx`'s raw date cells under (issue #4303):
    /// the dictionary's declared format when it expresses one, otherwise `QSV_PREFER_DMY`.
    /// Resolved once in `SmartCtx::new`; see the `dmy_prefs` field.
    fn dmy_pref(&self, idx: usize) -> bool {
        self.dmy_prefs.get(idx).copied().unwrap_or(false)
    }

    /// `--dict-info` icons are HTML-gated at build time (image exports never grow them); the
    /// per-field lookups share `dict_info_for_field`.
    fn dict_icons(&self) -> Option<&DictData> {
        (self.args.flag_dict_info && matches!(self.out_format, OutFormat::Html))
            .then_some(self.dict_data.as_ref())
            .flatten()
    }

    /// Under `--dict-info`, the geographic overview panels anchor their info icon on the driving
    /// coordinate column's dictionary entry (lat first) — the natural place a map reader looks for
    /// the coordinates' provenance and caveats.
    fn geo_dict_info(&self) -> Option<(String, String)> {
        self.map_cols.and_then(|(la, lo)| {
            dict_info_for_field(self.dict_icons(), &self.stats[la].field)
                .or_else(|| dict_info_for_field(self.dict_icons(), &self.stats[lo].field))
        })
    }

    /// Classify each column into a Data Schematic panel, accumulating the panel set, the
    /// skipped-column list, and the three deferred sentinel/measure diagnostics.
    ///
    /// Destructures the context up front so the loop can push to `panels` while iterating `stats`
    /// and still reach the lazy `nrows` row-count -- disjoint field borrows the borrow checker
    /// accepts, where `&mut self` method calls would not.
    fn classify_columns(&mut self) {
        let Self {
            args,
            out_format,
            stats,
            col_sems,
            dict_data,
            twin_suppress,
            map_cols,
            summary_choro_col,
            explicit_box_points,
            count_conf,
            nrows,
            log_scale,
            violin_mode,
            panels,
            skipped,
            sentinel_suspects,
            sentinel_hints,
            nonnumeric_measures,
            ..
        } = self;
        let (args, out_format, log_scale, violin_mode) =
            (*args, *out_format, *log_scale, *violin_mode);
        let (map_cols, summary_choro_col) = (*map_cols, *summary_choro_col);
        // the same two derived values `SmartCtx::is_map_col` / `dict_icons` compute; recreated as
        // locals here because `self` is destructured for the duration of the loop.
        let is_map_col = |idx: usize| {
            map_cols.is_some_and(|(la, lo)| idx == la || idx == lo)
                || summary_choro_col == Some(idx)
        };
        let dict_icons: Option<&DictData> = (args.flag_dict_info
            && matches!(out_format, OutFormat::Html))
        .then_some(dict_data.as_ref())
        .flatten();

        for (idx, s) in stats.iter().enumerate() {
            if is_map_col(idx) {
                continue;
            }
            let sem = &col_sems[idx];
            // the panel title is the raw field name (or a positional name when headerless); the
            // dictionary's human label becomes a subtitle beneath it, dropped when it's empty or
            // just echoes the header.
            let name = if s.field.is_empty() {
                format!("col {}", idx + 1)
            } else {
                s.field.clone()
            };
            let subtitle = (!sem.label.is_empty() && !sem.label.eq_ignore_ascii_case(&name))
                .then(|| sem.label.clone());
            // --dict-info: the column's dictionary description + pre-computed anchor.
            let dict_info = dict_info_for_field(dict_icons, &s.field);
            // a code/key twin (e.g. subject_code beside subject) is redundant with its label
            // sibling
            if twin_suppress.contains(&idx) {
                skipped.push(name);
                continue;
            }
            // `map_cols` (the resolved lat/lon pair), NOT `is_map_col` — a summary choropleth keyed
            // off a region CODE puts no coordinates on screen, so a projected coord is
            // still worth charting.
            match classify_with_semantics(idx, s, sem, map_cols.is_some()) {
                Some(mut kind) => {
                    // a cache-only quartile box becomes a raw box (with an overlay mode) when the
                    // explicit flag or the size heuristic calls for points (<=
                    // SMART_BOX_OUTLIERS_MAX rows). Above that, a column that
                    // HAS Tukey outliers becomes a `BoxOutliers` (precomputed
                    // box + native outlier-point overlay via a single fence-filtered
                    // pass); a column with no outliers stays a cheap cache-only `BoxStats` (no
                    // pass).
                    if let PanelKind::BoxStats {
                        q1,
                        median,
                        q3,
                        lower,
                        upper,
                        mean,
                    } = kind
                    {
                        let n =
                            *nrows.get_or_insert_with(|| util::count_rows(count_conf).unwrap_or(0));
                        // tier on the column's actual point count, not the dataset row count:
                        // only the non-null values are collected, embedded and rendered, so a
                        // mostly-null column shouldn't be denied an overlay tier (or the raw
                        // pass) by rows it doesn't have.
                        let n_points = n.saturating_sub(s.nullcount);
                        // the violin verdict needs the panel's value-axis log verdict (a KDE and a
                        // log axis mix poorly); it is recomputed cheaply below when the panel bakes
                        // in `value_log`.
                        let would_log = box_panel_logs(
                            log_scale,
                            s,
                            parse_stat_f64(s.min.as_deref()),
                            parse_stat_f64(s.max.as_deref()),
                        );
                        if let Some(points) =
                            smart_box_points(explicit_box_points.as_ref(), n_points)
                        {
                            // the raw values will be collected in full — exact violin (the default
                            // representation; see `wants_violin`) or raw box.
                            kind = if wants_violin(violin_mode, s, would_log, n_points) {
                                // the KDE silhouette already shows what a full point cloud would,
                                // so the size-based `all` upgrade (valuable on a box, whose five
                                // numbers hide the distribution) is redundant clutter on a violin:
                                // only an explicit --box-points mode overlays more than the
                                // outliers, which the KDE's smoothed tail can't show individually.
                                let points = if explicit_box_points.is_some() {
                                    points
                                } else {
                                    BoxPoints::Outliers
                                };
                                PanelKind::Violin {
                                    idx,
                                    points,
                                    sample_stride: 1,
                                }
                            } else {
                                PanelKind::BoxRaw { idx, points }
                            };
                        } else if wants_violin(violin_mode, s, would_log, n_points) {
                            // `smart_box_points` returned None for one of two reasons, and a violin
                            // still renders in both: an explicit `--box-points none` only means "no
                            // point overlay" (a violin's KDE doesn't need points; the cache-only
                            // escape hatch for that flag is `--violin off`), or the column has too
                            // many values to embed exactly — a violin doesn't need them all: draw
                            // the KDE + inner box from a deterministic stride sample (at most
                            // VIOLIN_SAMPLE_MAX values), with no point overlay either way (a
                            // strided sample misses the true extremes, so sampled "outliers"
                            // would mislead).
                            #[allow(clippy::cast_possible_truncation)]
                            let sample_stride = if n_points > SMART_BOX_OUTLIERS_MAX {
                                n_points.div_ceil(*VIOLIN_SAMPLE_MAX).max(1) as usize
                            } else {
                                1
                            };
                            kind = PanelKind::Violin {
                                idx,
                                points: BoxPoints::False,
                                sample_stride,
                            };
                        } else if explicit_box_points.is_none()
                            && n_points > SMART_BOX_OUTLIERS_MAX
                            && let Some((fence_low, fence_high)) = box_fences(s)
                            && (lower.is_some_and(|lo| lo < fence_low)
                                || upper.is_some_and(|hi| hi > fence_high))
                        {
                            // `lower`/`upper` are the column's observed min/max — outside the
                            // Tukey fences means real outliers exist, so overlay them.
                            kind = PanelKind::BoxOutliers {
                                idx,
                                q1,
                                median,
                                q3,
                                mean,
                                fence_low,
                                fence_high,
                            };
                        }
                    }
                    // for box/histogram panels, append cache-derived shape hints (null/zero share,
                    // skew direction, outlier share, …) to the panel title when notable.
                    // Cache-only, no cost; when nothing is notable the hint is
                    // None and the title is unchanged.
                    let name = match &kind {
                        PanelKind::BoxStats { .. }
                        | PanelKind::BoxRaw { .. }
                        | PanelKind::BoxOutliers { .. }
                        | PanelKind::Violin { .. }
                        | PanelKind::Histogram { .. } => match box_shape_hint(s) {
                            Some(hint) => format!("{name} {hint}"),
                            None => name,
                        },
                        _ => name,
                    };
                    // an honesty cue for violins drawn from a stride sample rather than every value
                    let name = if matches!(&kind, PanelKind::Violin { sample_stride, .. } if *sample_stride > 1)
                    {
                        t!("viz.title.sampled", q_name = name).into_owned()
                    } else {
                        name
                    };
                    // box panels resolve their value-axis log verdict now, while the cached
                    // observed min/max are at hand (frequency bars decide from
                    // their counts at render time)
                    let value_log = match &kind {
                        PanelKind::BoxStats { lower, upper, .. } => {
                            box_panel_logs(log_scale, s, *lower, *upper)
                        },
                        PanelKind::BoxRaw { .. }
                        | PanelKind::BoxOutliers { .. }
                        | PanelKind::Violin { .. } => box_panel_logs(
                            log_scale,
                            s,
                            parse_stat_f64(s.min.as_deref()),
                            parse_stat_f64(s.max.as_deref()),
                        ),
                        _ => false,
                    };
                    let interest = panel_interest(s, &kind);
                    panels.push(
                        Panel::new(name, kind)
                            .with_subtitle(subtitle)
                            .with_dict_info(dict_info)
                            .with_value_log(value_log)
                            .with_interest(interest)
                            .with_stat_idx(idx),
                    );
                },
                None => {
                    if s.r#type == "String" {
                        // exactly one parsing endpoint => a numeric range interrupted by a token
                        let one_endpoint_parses = parse_stat_f64(s.min.as_deref()).is_some()
                            != parse_stat_f64(s.max.as_deref()).is_some();
                        match (sem.route, one_endpoint_parses) {
                            (Route::Measure, true) => sentinel_suspects.push(name.clone()),
                            (Route::Measure, false) => nonnumeric_measures.push(name.clone()),
                            (Route::Defer, true) => sentinel_hints.push(name.clone()),
                            _ => {},
                        }
                    }
                    skipped.push(name);
                },
            }
        }
    }

    /// Build the relationship panels: the `--bivariate` association/Sankey set, the animated
    /// panels (T2 geo, T3 bubble), and the correlation heatmap with its scatter/contour/3D
    /// drill-downs.
    fn add_relationship_panels(&mut self) -> CliResult<()> {
        // Pipeline funnel (issue #4222, ask 3). Built FIRST and STASHED rather than inserted,
        // because the correlation heatmap's own `insert(0, ..)` further down would otherwise land
        // on top of it. It takes its own data pass: piggybacking the correlation read looks free
        // but silently drops any declared stage the correlation pre-filter excluded (a near-unique
        // money column), and force-including one would perturb the Pearson matrix and shrink the
        // listwise join for every other correlation-derived panel.
        let total_rows = self.row_count() as usize;
        self.funnel_panel = self.build_funnel_panel(total_rows)?;

        // `--bivariate`: NMI-driven association heatmap + ranked top relationships, built from
        // moarstats' bivariate sidecar (already produced by the `--smarter` block above, since
        // `--bivariate` forces `--smarter`). Inserted here — BEFORE the correlation-heatmap block
        // below — so that block's own `panels.insert(0, ..)` ends up on top: the LAST
        // `insert(0, ..)` to run wins the front slot, so final visual order top-to-bottom is
        // Correlation -> Association -> Top Relationships. (Inserting TopRelationships first, then
        // AssocHeatmap, puts AssocHeatmap above TopRelationships within that pair.)
        if self.bivariate_sidecar_fresh {
            let input = self.args.arg_input.as_deref().unwrap_or_default();
            let (assoc_panel, top_panel) =
                bivariate_panels(self.args, input, &self.stats, &self.col_sems, |i| {
                    self.is_map_col(i)
                });
            if let Some(panel) = top_panel {
                self.panels.insert(0, panel);
            }
            if let Some(panel) = assoc_panel {
                self.panels.insert(0, panel);
            }

            // directed-flow Sankey overview. Eligible dimensions are the same String freq-bar
            // columns the hierarchy panel nests (so the two panels stay mutually exclusive on a
            // shared pool). HTML-only: a Sankey is a domain-based trace that can't compose with the
            // typed x/y grid used for static image export — so, like the hierarchy/3D/map panels,
            // build it only for non-image output.
            if !self.out_format.is_image() {
                let sankey_eligible: std::collections::HashSet<usize> = self
                    .panels
                    .iter()
                    .filter_map(|p| match p.kind {
                        PanelKind::FreqBar { idx } => {
                            let s = &self.stats[idx];
                            (s.r#type == "String"
                                && (SANKEY_MIN_DIM_CARDINALITY..=CATEGORICAL_MAX_CARDINALITY)
                                    .contains(&s.cardinality))
                            .then_some(idx)
                        },
                        _ => None,
                    })
                    .collect();
                let built = sankey_panel(
                    self.args,
                    input,
                    &self.stats,
                    &self.col_sems,
                    &sankey_eligible,
                    |i| self.is_map_col(i),
                );
                match built {
                    Ok(Some((panel, pair))) => {
                        self.sankey_pair = Some(pair);
                        self.panels.insert(0, panel);
                    },
                    Ok(None) => {},
                    Err(e) => viz_note(&format!(
                        "viz smart: directed-flow (Sankey) panel skipped ({e})"
                    )),
                }
            }
        }

        // Animated panels (T2 geo, T3 bubble). At most ONE animated panel is shown; precedence is
        // T3 (bubble) > T2 (geo) > T1 (scatter pair). `geo_anim_panel` is filled HERE — outside the
        // numeric block, since T2 needs no numeric pair and a global dated-points dataset often has
        // a single numeric (making the numeric block below a no-op) — while `bubble_panel` is
        // filled INSIDE the numeric block (T3 needs a measure pair). The T1 insertion consults
        // both, and the winner is inserted by `add_overview_panels`.
        {
            let mode = smart_slider_mode(self.args);
            // respect `--slider off` and image output (animations are HTML-only), like T1
            if !self.out_format.is_image()
                && !matches!(mode, SmartSliderMode::Off)
                && let (Some((lat_idx, lon_idx)), Some((date_idx, _))) = (
                    self.map_cols.or_else(|| latlon_indices(&self.stats)),
                    canonical_date_col(&self.stats, &self.col_sems),
                )
            {
                // `auto` (default) only fires on a strong signal (>= 3 buckets); `on` lowers the
                // bar
                let min_frames = if matches!(mode, SmartSliderMode::On) {
                    2
                } else {
                    3
                };
                let prefer_dmy = self.dmy_pref(date_idx);
                if let Some(data) = read_geo_anim(
                    self.args, lat_idx, lon_idx, date_idx, prefer_dmy, min_frames,
                )? {
                    let frame_label = self
                        .col_sems
                        .get(date_idx)
                        .map(|s| s.label.as_str())
                        .filter(|l| !l.is_empty())
                        .map_or_else(|| self.stats[date_idx].field.clone(), ToString::to_string);
                    self.geo_anim_panel = Some(Panel::new(
                        t!("viz.title.locations_over_time").into_owned(),
                        PanelKind::AnimatedGeo {
                            bucket_lats: data.bucket_lats,
                            bucket_lons: data.bucket_lons,
                            bucket_labels: data.bucket_labels,
                            frame_label,
                        },
                    ));
                }
            }
        }

        // when 2+ continuous numeric columns exist, prepend a correlation-heatmap panel. This is
        // the one panel that re-scans the data (a single extra pass), since Pearson correlations
        // are not in the stats cache; it's prepended so it survives the panel cap below.
        let numeric_indices: Vec<usize> = self
            .stats
            .iter()
            .enumerate()
            .filter(|(i, s)| {
                !self.is_map_col(*i)
                // a dictionary-tagged dimension/geo/temporal/id numeric is NOT a continuous
                // measure, so exclude it from the Pearson matrix (and thus from the corr drill-down
                // and the implicit box-plot pool). A confirmed Measure, or an un-tagged (Defer)
                // numeric, still qualifies — without a dictionary every column is Defer, exactly
                // as before.
                && matches!(self.col_sems[*i].route, Route::Defer | Route::Measure)
                && matches!(s.r#type.as_str(), "Integer" | "Float")
                && s.cardinality > 1
                && !s.uniqueness_ratio.is_some_and(|r| r > 0.95)
            })
            .map(|(i, _)| i)
            .collect();
        if numeric_indices.len() >= 2 {
            self.progress.set_message("Computing correlations…");
            let (mut rdr, headers, nh) = reader_and_headers(self.args)?;
            let (labels, columns, kept_indices) =
                read_numeric_columns(&mut rdr, &headers, nh, &numeric_indices, false)?;
            // need 2+ numeric columns AND 2+ complete rows for a meaningful correlation matrix
            if labels.len() >= 2 && columns.first().is_some_and(|c| c.len() >= 2) {
                // Pearson is always computed: the nonlinearity note below is DEFINED as a
                // Pearson-vs-Spearman divergence, so it needs an r even when the panel itself
                // reports rho. On a mostly heavy-tailed table the displayed matrix (and every
                // selection driven by it) switches to Spearman's rank correlation, whose cells a
                // few extreme rows cannot dominate (issue #4220).
                let pmatrix = pearson_matrix(&columns);
                let spearman_corr = corr_prefers_spearman(&self.stats, &kept_indices);
                let matrix = if spearman_corr {
                    spearman_matrix(&columns)
                } else {
                    pmatrix.clone()
                };
                // the most strongly correlated pair drills into the heatmap's headline
                // relationship, but only when it's at least moderately correlated (else it's a
                // noise cloud). Strength is judged by the SAME coefficient the heatmap shows, so
                // the drill-down is the headline cell; the pair's Pearson r is then read off
                // `pmatrix` for the title and the nonlinearity check, which are Pearson-defined.
                // All the drill-downs below reuse the columns already read for the matrix (no
                // extra data pass).
                let pair = strongest_pair(&matrix)
                    .filter(|&(_, _, c)| c.abs() >= SCATTER_PAIR_MIN_ABS_R)
                    .map(|(i, j, _)| (i, j, pmatrix[i][j]));

                // Animated relationship-over-time drill-down (T1), DECOUPLED from the strongest
                // pair. The strongest pair is usually the most tautological (a near-line whose 2-D
                // shape can't evolve), so animating it adds nothing over a static time-colored
                // scatter. Instead pick the pair whose per-time-bucket centroid path BENDS
                // the most — a genuinely evolving 2-D relationship
                // (`select_drifting_pair`) — and animate that. HTML only; needs a canonical
                // date column; `--slider off` suppresses. Computed BEFORE the correlation
                // heatmap consumes `labels`/`matrix`; returns the selected (i, j) plus the built
                // Panel so the static drill-down below can de-dupe when they coincide.
                let anim_sel: Option<(usize, usize, Panel)> = 'anim: {
                    let mode = smart_slider_mode(self.args);
                    if self.out_format.is_image() || matches!(mode, SmartSliderMode::Off) {
                        break 'anim None;
                    }
                    let Some((date_idx, _)) = canonical_date_col(&self.stats, &self.col_sems)
                    else {
                        break 'anim None;
                    };
                    // `auto` (default) only fires on a strong signal (>= 3 buckets); `on` lowers
                    // the bar
                    let min_frames = if matches!(mode, SmartSliderMode::On) {
                        2
                    } else {
                        3
                    };
                    let prefer_dmy = self.dmy_pref(date_idx);
                    let Some(bucket_means) = read_bucketed_means(
                        self.args,
                        &kept_indices,
                        date_idx,
                        prefer_dmy,
                        min_frames,
                    )?
                    else {
                        break 'anim None;
                    };
                    let Some((i, j, _)) = select_drifting_pair(&matrix, &columns, &bucket_means)
                    else {
                        break 'anim None;
                    };
                    // `select_drifting_pair` hands back the coefficient from the matrix it was
                    // given — which is Spearman's rho on a tail-dominated table. The title below
                    // and its nonlinearity note are Pearson-defined, so read r off `pmatrix`, as
                    // the static drill-down does. Taking the returned value instead would label a
                    // rho as "r" AND collapse the Pearson-vs-Spearman gap to zero, so the note
                    // could never fire in exactly the mode that most needs it.
                    let r = pmatrix[i][j];
                    // large-N contours aren't animatable (a scatter would overplot into a solid
                    // mass)
                    if columns[i].len() >= SMART_CONTOUR_MIN_POINTS {
                        break 'anim None;
                    }
                    let Some(data) = read_scatter_pair_anim(
                        self.args,
                        kept_indices[i],
                        kept_indices[j],
                        date_idx,
                        prefer_dmy,
                        min_frames,
                    )?
                    else {
                        break 'anim None;
                    };
                    let frame_label = self
                        .col_sems
                        .get(date_idx)
                        .map(|s| s.label.as_str())
                        .filter(|l| !l.is_empty())
                        .map_or_else(|| col_label(&headers, date_idx, nh), ToString::to_string);
                    let rho = spearman_rho(&columns[i], &columns[j]);
                    let rel = if rho.abs() - r.abs() >= SMART_NONLINEAR_MIN_GAP {
                        t!(
                            "viz.title.pair_r_nonlinear",
                            q_x = labels[i],
                            q_y = labels[j],
                            q_r = format!("{r:.2}"),
                            q_rho = format!("{rho:.2}")
                        )
                        .into_owned()
                    } else {
                        t!(
                            "viz.title.pair_r",
                            q_x = labels[i],
                            q_y = labels[j],
                            q_r = format!("{r:.2}")
                        )
                        .into_owned()
                    };
                    let panel = Panel::new(
                        t!("viz.title.pair_over_time", q_pair = rel).into_owned(),
                        PanelKind::AnimatedScatterPair {
                            xs: data.xs,
                            ys: data.ys,
                            bucket: data.bucket,
                            bucket_labels: data.bucket_labels,
                            x_label: labels[i].clone(),
                            y_label: labels[j].clone(),
                            frame_label,
                            x_range: data.x_range,
                            y_range: data.y_range,
                        },
                    );
                    Some((i, j, panel))
                };
                let anim_ij = anim_sel.as_ref().map(|&(i, j, _)| (i, j));

                // T3 (Gapminder bubble): fill bubble_panel INSIDE the numeric block (it needs a
                // measure pair) and BEFORE the T1 insertion, so the arbitration T3 > T1 can be
                // applied. `matrix` is still alive here (consumed by `mask_to_lower_triangle` only
                // at the CorrHeatmap insert below). Requires a low-cardinality categorical
                // entity, a measure pair (the strongest pair in the [0.5, 0.9) |r| band —
                // strong enough to relate, not a tautological near-line — else the first
                // two numerics), a canonical date, and — via `read_entity_bucket_agg`'s
                // strict cell/entity gates — enough non-sparse per-(entity,bucket) support
                // (this is what rejects sparse zone-per-month datasets).
                {
                    let mode = smart_slider_mode(self.args);
                    if self.bubble_panel.is_none()
                        && !self.out_format.is_image()
                        && !matches!(mode, SmartSliderMode::Off)
                        && let Some((date_idx, _)) = canonical_date_col(&self.stats, &self.col_sems)
                    {
                        let n = self.row_count();
                        // first (lowest-idx) eligible categorical dimension is the bubble entity
                        let entity =
                            eligible_categorical_dims(&self.panels, &self.stats, &self.col_sems, n)
                                .into_iter()
                                .min_by_key(|&(idx, ..)| idx);
                        // measure pair: the strongest pair in [SCATTER_PAIR_MIN_ABS_R, 0.9), else
                        // the first two numerics
                        let measure_pair = strongest_pair(&matrix)
                            .filter(|&(_, _, r)| (SCATTER_PAIR_MIN_ABS_R..0.9).contains(&r.abs()))
                            .map(|(i, j, _)| (i, j))
                            .or(Some((0, 1)));
                        if let (Some((e_idx, _, entity_label)), Some((i, j))) =
                            (entity, measure_pair)
                        {
                            let min_frames = if matches!(mode, SmartSliderMode::On) {
                                2
                            } else {
                                3
                            };
                            // bubble SIZE = Gapminder's third data variable: the leftover numeric
                            // least redundant with the (x,y) pair (so size adds new information,
                            // not a near-copy of an axis). `None` for a 2-numeric dataset — the
                            // reader then falls back to the per-cell row count and the label stays
                            // "records".
                            let size_k = least_redundant_third(&matrix, i, j);
                            let size_idx = size_k.map(|k| kept_indices[k]);
                            let size_label = size_k
                                .map(|k| labels[k].clone())
                                .unwrap_or_else(|| t!("viz.chart.records").into_owned());
                            let prefer_dmy = self.dmy_pref(date_idx);
                            // STRICT gates: this panel is AUTO-selected, so a sparse cell or an
                            // entity that flickers in and out reads as sampling noise rather than
                            // motion. The explicit `viz scatter --slider` path relaxes these and
                            // warns instead (see `EntityBucketOpts`).
                            let opts = EntityBucketOpts {
                                size_idx,
                                agg: Agg::Mean,
                                cumulative: false,
                                prefer_dmy,
                                min_frames,
                                max_frames: SMART_ANIM_MAX_FRAMES,
                                min_dense_floor: min_frames,
                                min_cell_rows: 3,
                                min_entities: 3,
                                min_dense_pct: 80,
                                warn: false,
                            };
                            if let Some(data) = read_entity_bucket_agg(
                                self.args,
                                e_idx,
                                kept_indices[i],
                                kept_indices[j],
                                date_idx,
                                &opts,
                            )? {
                                let (x_label, y_label) = (labels[i].clone(), labels[j].clone());
                                let frame_label = self
                                    .col_sems
                                    .get(date_idx)
                                    .map(|s| s.label.as_str())
                                    .filter(|l| !l.is_empty())
                                    .map_or_else(
                                        || self.stats[date_idx].field.clone(),
                                        ToString::to_string,
                                    );
                                self.bubble_panel = Some(Panel::new(
                                    t!(
                                        "viz.title.bubble_over_time",
                                        q_y = y_label,
                                        q_x = x_label,
                                        q_entity = entity_label
                                    )
                                    .into_owned(),
                                    PanelKind::AnimatedBubble {
                                        data,
                                        x_label,
                                        y_label,
                                        size_label,
                                        frame_label,
                                    },
                                ));
                            }
                        }
                    }
                }

                // Static pair drill-down beside the heatmap: the STRONGEST correlated pair as a 2D
                // density contour for large datasets (a scatter would overplot into a solid mass,
                // and the contour embeds only a fixed grid) or a static bubble scatter. The title
                // carries Pearson r, plus Spearman rho when the two diverge enough to mean the
                // relationship is monotonic-but-curved (nonlinear) — so the reader doesn't take the
                // single r as proof of a linear relationship. Suppressed when the animated
                // drill-down above already covers the very same pair (a moderate strongest pair
                // that also drifts), so we don't show both a static and an animated scatter of one
                // relationship — but ONLY when that animated pair (T1) will actually be
                // inserted. If a higher-priority bubble (T3) or geo (T2) animation won
                // arbitration, T1 is discarded, so suppressing the static pair too would drop
                // the strongest-pair drill-down entirely.
                let t1_wins = self.bubble_panel.is_none() && self.geo_anim_panel.is_none();
                // copied out so the panel-building closures below don't hold a borrow of `self`
                let log_scale = self.log_scale;
                let pair_panel = pair
                    .filter(|&(i, j, _)| !(t1_wins && anim_ij == Some((i, j))))
                    .and_then(|(i, j, r)| {
                        let rho = spearman_rho(&columns[i], &columns[j]);
                        let name = if rho.abs() - r.abs() >= SMART_NONLINEAR_MIN_GAP {
                            t!(
                                "viz.title.pair_r_nonlinear",
                                q_x = labels[i],
                                q_y = labels[j],
                                q_r = format!("{r:.2}"),
                                q_rho = format!("{rho:.2}")
                            )
                            .into_owned()
                        } else {
                            t!(
                                "viz.title.pair_r",
                                q_x = labels[i],
                                q_y = labels[j],
                                q_r = format!("{r:.2}")
                            )
                            .into_owned()
                        };
                        if columns[i].len() >= SMART_CONTOUR_MIN_POINTS {
                            // A density grid that collapses into one cell is dropped outright (or
                            // retried in log space over the positive subset) — the contour-branch
                            // counterpart of the `degenerate` scatter guard below (issue #4223).
                            let panel = smart_contour_panel(
                                &columns[i],
                                &columns[j],
                                &name,
                                log_scale,
                                (&labels[i], &labels[j]),
                            );
                            if panel.is_none() {
                                viz_note(&format!(
                                    "viz smart: density panel for {} vs {} skipped (its \
                                     distribution collapses into a single bin)",
                                    labels[i], labels[j]
                                ));
                            }
                            panel
                        } else {
                            let (xs, ys) =
                                downsample_pair(&columns[i], &columns[j], *MAX_SMART_POINTS);
                            // Each axis decides its own scale from the points it will actually
                            // show, so a heavy-tailed measure isn't squashed against the origin
                            // (issue #4223). A single zero or negative keeps that axis linear.
                            let axis_log = (
                                relationship_axis_log(log_scale, &xs),
                                relationship_axis_log(log_scale, &ys),
                            );
                            // Drop a DEGENERATE scatter (essentially no 2-D spread — e.g. a heavily
                            // zero-skewed measure collapses every point onto one axis): it would
                            // render as an empty box, so omit the panel entirely. A merely sparse-
                            // but-informative scatter is kept and just gets a compact height (see
                            // `panel_render_height`).
                            //
                            // Judge that in the space the panel will RENDER in, not in linear
                            // space (issue #4276). A heavy-tailed but strictly positive pair
                            // collapses against the origin on linear axes while spreading cleanly
                            // once logged, and this guard used to run first and drop it before the
                            // log axes above were ever consulted — so the panel disappeared in
                            // exactly the case log scaling exists to rescue, and `--log-scale on`
                            // could not override it. This mirrors the contour branch, which
                            // already retries a collapsed grid in log space before giving up.
                            //
                            // `scatter_fill_plotted` also backs the render-height branch, so the
                            // drop and the compact-height decisions judge the same space.
                            if scatter_fill_plotted(&xs, &ys, axis_log).degenerate {
                                return None;
                            }
                            // Encode a third numeric (the axis MOST associated with the pair, so
                            // size shows a real gradient — the complement of the least-redundant
                            // axis `Scatter3D` picks) as marker size: a 3-variable bubble
                            // view that, unlike the HTML-only 3D panel, also survives
                            // static image export. Aligned to xs/ys because
                            // `downsample_pair` picks the same row indices for the same (n, cap).
                            let (name, sizes, size_label) =
                                match most_associated_third(&matrix, i, j) {
                                    Some(k) => {
                                        let (_, sizes) = downsample_pair(
                                            &columns[i],
                                            &columns[k],
                                            *MAX_SMART_POINTS,
                                        );
                                        (
                                            t!(
                                                "viz.title.pair_size",
                                                q_name = name,
                                                q_col = labels[k]
                                            )
                                            .into_owned(),
                                            Some(sizes),
                                            Some(labels[k].clone()),
                                        )
                                    },
                                    None => (name, None, None),
                                };
                            // These panels carry no axis titles (the pair is named in the panel
                            // title), so the log cue goes in the title too — naming WHICH axis,
                            // which a bare axis-side "log scale" label could not.
                            let name = match axis_log {
                                (true, true) => {
                                    t!("viz.title.pair_log_xy", q_name = name).into_owned()
                                },
                                (true, false) => {
                                    t!("viz.title.pair_log_x", q_name = name).into_owned()
                                },
                                (false, true) => {
                                    t!("viz.title.pair_log_y", q_name = name).into_owned()
                                },
                                (false, false) => name,
                            };
                            Some(
                                Panel::new(
                                    name,
                                    PanelKind::ScatterPair {
                                        xs,
                                        ys,
                                        sizes,
                                        x_label: labels[i].clone(),
                                        y_label: labels[j].clone(),
                                        size_label,
                                    },
                                )
                                .with_axis_log(axis_log),
                            )
                        }
                    });

                // 3D drill-down: with 3+ numeric columns, a Scatter3D of the strongest pair plus a
                // third axis chosen to be the LEAST redundant with that pair (minimizing
                // max(|r_ik|, |r_jk|)), so the cloud genuinely uses all three dimensions instead of
                // collapsing onto the pair's plane. Skipped entirely when the strongest pair is
                // itself near-collinear (|r| >= SMART_3D_COLLINEAR_MAX_ABS_R): two near-identical
                // axes can't form a non-degenerate 3D, and the 2D pair drill-down already shows
                // that relationship. A 3D scene can't share the typed x/y subplot grid, so it's
                // built ONLY for HTML output (which uses the inline render path, like the map
                // panel). Static image export goes through the typed grid, where a 3D panel would
                // hit `panel_trace`'s unreachable arm.
                let scatter3d_panel = pair
                    .filter(|&(_, _, r)| r.abs() < SMART_3D_COLLINEAR_MAX_ABS_R)
                    .filter(|_| columns.len() >= 3 && !self.out_format.is_image())
                    .and_then(|(i, j, _)| {
                        let third = least_redundant_third(&matrix, i, j);
                        third.and_then(|k| {
                            // both downsamples share (n, cap) so they pick the same row indices ->
                            // aligned
                            let (xs, ys) =
                                downsample_pair(&columns[i], &columns[j], *MAX_SMART_POINTS);
                            let (_, zs) =
                                downsample_pair(&columns[i], &columns[k], *MAX_SMART_POINTS);
                            // Legibility (issue #4223): a 3D cloud is only worth rotating if it has
                            // spread in every plane. Judge it by its three 2-D projections with the
                            // same `degenerate` test that drops an unreadable scatter — a
                            // heavily-skewed triple collapses ALL of them onto the origin, which is
                            // the "mass at the origin with a few outliers floating" report. A 3D
                            // scene also can't be read off a static image, so there's no fallback
                            // rendering worth keeping.
                            if scatter_fill_stats(&xs, &ys).degenerate
                                || scatter_fill_stats(&xs, &zs).degenerate
                                || scatter_fill_stats(&ys, &zs).degenerate
                            {
                                viz_note(&format!(
                                    "viz smart: 3D panel for {} / {} / {} skipped (its points \
                                     collapse onto the origin in at least one plane)",
                                    labels[i], labels[j], labels[k]
                                ));
                                return None;
                            }
                            Some(Panel::new(
                                t!(
                                    "viz.title.scatter3d",
                                    q_x = labels[i],
                                    q_y = labels[j],
                                    q_z = labels[k]
                                )
                                .into_owned(),
                                PanelKind::Scatter3D {
                                    xs,
                                    ys,
                                    zs,
                                    labels: (
                                        labels[i].clone(),
                                        labels[j].clone(),
                                        labels[k].clone(),
                                    ),
                                },
                            ))
                        })
                    });

                // Show only the lower triangle: a correlation matrix mirrors across the diagonal
                // (and the diagonal is a trivial 1.0), so masking the upper half + diagonal drops
                // redundant cells. `matrix` was already consumed by `strongest_pair`/the 3D
                // third-axis pick above (which need the full square), so masking here only affects
                // the rendered panel.
                // The coefficient is named in the title (and in every cell's hover): a Spearman
                // matrix reported as plain "Correlation" would be read as Pearson's r.
                let corr_title = if spearman_corr {
                    t!("viz.title.corr_spearman")
                } else {
                    t!("viz.title.corr_pearson")
                };
                self.panels.insert(
                    0,
                    Panel::new(
                        corr_title.into_owned(),
                        PanelKind::CorrHeatmap {
                            labels,
                            matrix: mask_to_lower_triangle(matrix),
                            spearman: spearman_corr,
                        },
                    ),
                );
                // place the drill-down panels right after the heatmap: the animated
                // relationship-over-time scatter (when a drifting pair was found) leads, then the
                // static strongest-pair drill-down (suppressed above if it is the same pair), then
                // the 3D scatter.
                // ARBITRATION (§5): at most one animated panel; T3 (bubble) > T2 (geo) > T1. The T1
                // scatter-pair animation is inserted only when neither T3 nor T2 won (the static
                // pair/3D drill-downs still show, keyed off `anim_ij`, as before).
                let anim_panel = if self.bubble_panel.is_none() && self.geo_anim_panel.is_none() {
                    anim_sel.map(|(_, _, panel)| panel)
                } else {
                    None
                };
                let mut at = 1;
                if let Some(panel) = anim_panel {
                    self.panels.insert(at, panel);
                    at += 1;
                }
                if let Some(panel) = pair_panel {
                    self.panels.insert(at, panel);
                    at += 1;
                }
                if let Some(panel) = scatter3d_panel {
                    self.panels.insert(at, panel);
                }
            }
        }
        Ok(())
    }

    /// Build the pipeline-funnel panel from a dictionary declaration (issue #4222, ask 3), or
    /// `None` when none was declared or none validates.
    ///
    /// **A funnel is drawn only when the dictionary declares one.** Which columns are stages, and
    /// in which direction, is SEMANTICS — no column-name vocabulary settles it (`township` reads
    /// as a shipping stage) and no statistic does either (`total_pop`/`male_pop`/`female_pop`
    /// nests perfectly yet is a partition). The predecessor of this code guessed with a hardcoded
    /// word list and then used row-wise containment as a gate to catch its own bad guesses. Issue
    /// #4222 rejected exactly that shape of solution for its sibling asks — "it is semantics, not
    /// a statistic, so it deserves explicit treatment rather than a guessed proxy" — and this
    /// follows suit.
    ///
    /// Containment is therefore still MEASURED but never REFUSES: a declared pipeline whose
    /// stages do not nest is drawn with the violation share named in the subtitle. That is more
    /// informative than silence, and it is why the NYC CPDB dataset — three aggregates on
    /// different accounting bases — now renders where it previously was declined.
    ///
    /// `total_rows` is the table's row count, used only to disclose how much of it the funnel
    /// actually covers.
    fn build_funnel_panel(&self, total_rows: usize) -> CliResult<Option<Panel>> {
        let specs = match self.dict_data.as_ref() {
            Some(d) if !d.pipelines.is_empty() => d.pipelines.clone(),
            _ => return Ok(None),
        };
        // First declaration that validates wins; a Data Schematic with two funnels is out of scope.
        for spec in &specs {
            let built = match spec {
                PipelineSpec::Columns { members } => {
                    self.build_column_funnel(members, total_rows)?
                },
                PipelineSpec::Rows {
                    stage_column,
                    stages,
                    value_column,
                } => self.build_row_funnel(
                    stage_column,
                    stages,
                    value_column.as_deref(),
                    total_rows,
                )?,
            };
            if built.is_some() {
                return Ok(built);
            }
        }
        Ok(None)
    }

    /// Resolve a declared column name to its stats index.
    fn stage_index(&self, name: &str) -> Option<usize> {
        self.stats.iter().position(|s| s.field == name)
    }

    /// A stage column's display label — the dictionary `title` when it supplied one.
    fn stage_label(&self, idx: usize) -> String {
        self.col_sems
            .get(idx)
            .map(|s| s.label.clone())
            .filter(|l| !l.is_empty())
            .unwrap_or_else(|| self.stats[idx].field.clone())
    }

    /// Funnel over stages held in SEPARATE COLUMNS, listed upstream-first.
    fn build_column_funnel(
        &self,
        members: &[String],
        total_rows: usize,
    ) -> CliResult<Option<Panel>> {
        // Resolve and screen the declared members. The split between hard-skip and warn-only is
        // deliberate: a declaration outranks a NAME heuristic (so an identifier-looking or
        // complement-looking name only warns), but it cannot make an arithmetically meaningless
        // chart sensible — summing a rate or a coordinate is nonsense whoever asked for it.
        let mut indices: Vec<usize> = Vec::with_capacity(members.len());
        for name in members {
            let Some(idx) = self.stage_index(name) else {
                viz_note(&format!(
                    "viz smart: pipeline funnel — declared stage '{name}' is not a column in this \
                     file"
                ));
                return Ok(None);
            };
            let s = &self.stats[idx];
            let sem = &self.col_sems[idx];
            if !matches!(s.r#type.as_str(), "Integer" | "Float") || self.is_map_col(idx) {
                viz_note(&format!(
                    "viz smart: pipeline funnel skipped — declared stage '{name}' is not a \
                     numeric amount"
                ));
                return Ok(None);
            }
            if matches!(sem.agg, Some(Agg::Mean)) || is_intensive_measure(&sem.label, &s.field) {
                viz_note(&format!(
                    "viz smart: pipeline funnel skipped — declared stage '{name}' is an average \
                     or rate, which cannot be summed into a funnel"
                ));
                return Ok(None);
            }
            if is_complement_name(&sem.label, &s.field) {
                // Warn, don't refuse. A complement nests PERFECTLY, so it shows 0% violations —
                // the subtitle cannot surface this one and nothing else will.
                viz_note(&format!(
                    "viz smart: pipeline funnel — '{name}' reads as a complement or remainder, \
                     not a stage the process passed through; the funnel may be inverted"
                ));
            }
            indices.push(idx);
        }
        if indices.len() < 2 {
            return Ok(None);
        }
        if indices.len() > FUNNEL_MAX_STAGES {
            let dropped = indices.len() - FUNNEL_MAX_STAGES;
            indices.truncate(FUNNEL_MAX_STAGES);
            viz_note(&format!(
                "viz smart: pipeline funnel shows the first {FUNNEL_MAX_STAGES} stages ({dropped} \
                 more not shown)"
            ));
        }

        // Dedicated pass over exactly the declared stages, so the complete-case denominator is
        // the funnel's own rather than one diluted by unrelated numeric columns.
        self.progress.set_message("Reading pipeline stages…");
        let (mut rdr, headers, nh) = reader_and_headers(self.args)?;
        let (labels, columns, kept) =
            read_numeric_columns(&mut rdr, &headers, nh, &indices, false)?;
        if kept.len() != indices.len() || columns.first().is_none_or(Vec::is_empty) {
            viz_note(
                "viz smart: pipeline funnel skipped — a declared stage held no usable numeric data",
            );
            return Ok(None);
        }

        let n_complete = columns[0].len();
        #[allow(clippy::cast_precision_loss)]
        let complete_frac = if total_rows == 0 {
            0.0
        } else {
            n_complete as f64 / total_rows as f64
        };
        if complete_frac < FUNNEL_MIN_COMPLETE_FRAC {
            viz_note(&format!(
                "viz smart: pipeline funnel skipped — only {:.0}% of rows are complete across \
                 every stage",
                complete_frac * 100.0
            ));
            return Ok(None);
        }

        let totals: Vec<f64> = columns.iter().map(|c| c.iter().sum()).collect();
        let reached: Vec<usize> = columns
            .iter()
            .map(|c| c.iter().filter(|v| **v > 0.0).count())
            .collect();
        // Measured for DISCLOSURE only -- this no longer gates anything.
        let violations: Vec<f64> = (0..columns.len())
            .map(|k| {
                if k == 0 {
                    0.0
                } else {
                    1.0 - containment_fraction(&columns[k - 1], &columns[k])
                }
            })
            .collect();

        let stages: Vec<String> = indices.iter().map(|&i| self.stage_label(i)).collect();
        Ok(self.finish_funnel(
            stages,
            labels,
            totals,
            reached,
            &violations,
            n_complete,
            complete_frac,
            FunnelShape::Columns,
        ))
    }

    /// Funnel over stages held as VALUES of one category column.
    fn build_row_funnel(
        &self,
        stage_column: &str,
        stages: &[String],
        value_column: Option<&str>,
        total_rows: usize,
    ) -> CliResult<Option<Panel>> {
        let Some(stage_idx) = self.stage_index(stage_column) else {
            viz_note(&format!(
                "viz smart: pipeline funnel — declared stage column '{stage_column}' is not a \
                 column in this file"
            ));
            return Ok(None);
        };
        let value_idx = match value_column {
            Some(vc) => {
                let Some(vi) = self.stage_index(vc) else {
                    viz_note(&format!(
                        "viz smart: pipeline funnel — declared value column '{vc}' is not a \
                         column in this file"
                    ));
                    return Ok(None);
                };
                if !matches!(self.stats[vi].r#type.as_str(), "Integer" | "Float") {
                    viz_note(&format!(
                        "viz smart: pipeline funnel skipped — value column '{vc}' is not numeric"
                    ));
                    return Ok(None);
                }
                Some(vi)
            },
            None => None,
        };
        for s in stages {
            if is_complement_name("", s) {
                viz_note(&format!(
                    "viz smart: pipeline funnel — stage '{s}' reads as a complement or remainder \
                     rather than a stage the process passed through"
                ));
            }
        }

        let mut wanted: Vec<String> = stages.to_vec();
        if wanted.len() > FUNNEL_MAX_STAGES {
            let dropped = wanted.len() - FUNNEL_MAX_STAGES;
            wanted.truncate(FUNNEL_MAX_STAGES);
            viz_note(&format!(
                "viz smart: pipeline funnel shows the first {FUNNEL_MAX_STAGES} stages ({dropped} \
                 more not shown)"
            ));
        }

        self.progress.set_message("Reading pipeline stages…");
        let (totals, counts, matched) =
            read_stage_totals(self.args, stage_idx, value_idx, &wanted)?;
        if counts.iter().filter(|c| **c > 0).count() < 2 {
            viz_note(&format!(
                "viz smart: pipeline funnel skipped — fewer than two of the declared stages of \
                 '{stage_column}' appear in the data"
            ));
            return Ok(None);
        }

        #[allow(clippy::cast_precision_loss)]
        let complete_frac = if total_rows == 0 {
            0.0
        } else {
            matched as f64 / total_rows as f64
        };
        // No FUNNEL_MIN_COMPLETE_FRAC gate here, deliberately. Rows outside the declared stages
        // are simply other statuses -- a legitimately different thing from a row whose stage
        // amount was blank -- so a 50% bar would reject a real funnel over a busy status column.
        let labels = vec![self.stage_label(stage_idx); wanted.len()];
        let shape = match value_idx {
            Some(vi) => FunnelShape::RowsMeasure {
                value_label: self.stage_label(vi),
            },
            None => FunnelShape::RowsCount,
        };
        // Stages partition the rows, so per-row containment is undefined; the subtitle's
        // total-inversion clause still fires and remains a genuine finding.
        let violations = vec![0.0; wanted.len()];
        Ok(self.finish_funnel(
            wanted,
            labels,
            totals,
            counts,
            &violations,
            matched,
            complete_frac,
            shape,
        ))
    }

    /// Shared tail: the refusals that apply to every funnel, then the panel.
    #[allow(clippy::too_many_arguments)]
    fn finish_funnel(
        &self,
        stages: Vec<String>,
        labels: Vec<String>,
        totals: Vec<f64>,
        reached: Vec<usize>,
        violations: &[f64],
        n_complete: usize,
        complete_frac: f64,
        shape: FunnelShape,
    ) -> Option<Panel> {
        if totals.iter().any(|t| *t < 0.0) {
            viz_note(
                "viz smart: pipeline panel skipped \u{2014} a stage total is negative, which \
                 neither a funnel nor a bridge can represent",
            );
            return None;
        }
        if totals.windows(2).all(|w| {
            let [a, b] = w else { return true };
            (a - b).abs() < f64::EPSILON
        }) {
            viz_note(
                "viz smart: pipeline funnel skipped \u{2014} every stage total is identical (the \
                 columns are copies, not stages)",
            );
            return None;
        }
        // The declaration says these columns are one pipeline; the NUMBERS decide whether a
        // containment form can honestly represent it. See `PipelineForm`.
        let form = PipelineForm::for_totals(&totals);
        let subtitle = funnel_subtitle(
            &stages,
            &totals,
            violations,
            n_complete,
            complete_frac,
            &form,
        );
        let noun = match form {
            PipelineForm::Funnel => "funnel",
            PipelineForm::Bridge => "bridge",
        };
        let title = match &shape {
            FunnelShape::Columns => {
                format!("Pipeline {noun}: {}", labels.join(" \u{2192} "))
            },
            _ => format!(
                "Pipeline {noun}: {} ({})",
                labels.first().cloned().unwrap_or_default(),
                stages.join(" \u{2192} ")
            ),
        };
        Some(
            Panel::new(
                title,
                PanelKind::Funnel {
                    stages,
                    labels,
                    totals,
                    reached,
                    n_complete,
                    shape,
                    form,
                },
            )
            .with_subtitle(subtitle),
        )
    }

    /// Prepend the cross-column overview panels: the winning animated panel, measure-by-dimension,
    /// grouped violin, Lorenz curves, parcats, hierarchy, time-series, cyclic profile, and finally
    /// the geographic panels. Each is `insert(0, ..)`-ed, so the LAST one inserted leads the
    /// Data Schematic — the order of these blocks IS the final top-to-bottom order.
    fn add_overview_panels(&mut self) -> CliResult<()> {
        // ARBITRATION (§5): insert the single winning animated panel — T3 (bubble) first, else T2
        // (geo) — as a leading overview row. Prepending here (before the measure-by-dim / grouped-
        // violin / hierarchy `insert(0, ..)` overview panels below) lands it among the correlation
        // drill-downs, just like the T1 slot it supersedes.
        if let Some(winner) = self.bubble_panel.take().or(self.geo_anim_panel.take()) {
            self.panels.insert(0, winner);
        }

        // prepend a "measure by dimension" bar when a low-cardinality categorical dimension
        // strongly explains one of the numeric measures (correlation ratio η² above the gate) —
        // e.g. mean amount by region. The panel builder does one extra data pass to compute η² and
        // the per-group aggregates.
        //
        // Measure candidates are computed separately from `numeric_indices` (the correlation list):
        // a genuine per-row measure like revenue/amount is often near-unique, and the correlation
        // list drops near-unique columns to keep IDs out of the matrix. Here we exclude near-unique
        // columns ONLY for untagged (`Defer`) stats-only columns (the same ID-safety heuristic); a
        // column the dictionary explicitly routes as `Measure` qualifies regardless of uniqueness —
        // mirroring the time-series panel, which deliberately allows near-unique measures.
        let measure_indices: Vec<usize> = self
            .stats
            .iter()
            .enumerate()
            .filter(|(i, s)| {
                let route = self.col_sems[*i].route;
                !self.is_map_col(*i)
                    && matches!(route, Route::Defer | Route::Measure)
                    && matches!(s.r#type.as_str(), "Integer" | "Float")
                    && s.cardinality > 1
                    && (route == Route::Measure || !s.uniqueness_ratio.is_some_and(|r| r > 0.95))
            })
            .map(|(i, _)| i)
            .collect();
        let dim_indices: Vec<usize> = self
            .stats
            .iter()
            .enumerate()
            .filter(|(i, s)| {
                !self.is_map_col(*i)
                    && !self.twin_suppress.contains(i)
                    && matches!(self.col_sems[*i].route, Route::Defer | Route::Dimension)
                    && s.r#type.as_str() == "String"
                    && s.cardinality >= 2
                    && s.cardinality <= CATEGORICAL_MAX_CARDINALITY
            })
            .map(|(i, _)| i)
            .collect();
        if !measure_indices.is_empty() && !dim_indices.is_empty() {
            let top_n = self.args.flag_limit.max(1);
            // bound the immutable borrows of `self` to this statement so the insert below can take
            // `&mut self.panels`
            let built = measure_by_dim_panel(
                self.args,
                &self.stats,
                &self.col_sems,
                &measure_indices,
                &dim_indices,
                top_n,
            );
            match built {
                Ok(Some(panel)) => self.panels.insert(0, panel),
                Ok(None) => {},
                Err(e) => {
                    viz_note(&format!(
                        "viz smart: measure-by-dimension panel skipped ({e})"
                    ));
                },
            }
        }

        // prepend a grouped-violin panel — the distribution of the primary measure split by the
        // coarsest categorical dimension (one violin per category) — whenever a categorical
        // dimension and a numeric measure both exist. Inserted here (after the measure-by-dim bar,
        // before the hierarchy) so the final top-to-bottom order is hierarchy → grouped-violin →
        // measure-by-dim: part-to-whole counts, then distribution-by-category, then the aggregate.
        // A plain cartesian Violin::new_xy, so (unlike the hierarchy) it composes with the typed
        // grid and static export.
        if let Some(&m_idx) = measure_indices.first()
            && !dim_indices.is_empty()
        {
            let top_n = self.args.flag_limit.max(1);
            // the measure's non-null count drives the collection stride so the pass never buffers
            // more than ~VIOLIN_SAMPLE_MAX values (the same per-violin budget as the single-column
            // violin, which tiers on `n - nullcount`). Row count is pulled once from the
            // stats/index cache (shared with the box-points heuristic's lazy fetch).
            let n = self.row_count();
            let measure_nonnull = n.saturating_sub(self.stats[m_idx].nullcount);
            let built = grouped_violin_panel(
                self.args,
                &self.stats,
                &self.col_sems,
                &measure_indices,
                &dim_indices,
                top_n,
                self.violin_mode,
                measure_nonnull,
                self.explicit_box_points.as_ref(),
            );
            match built {
                Ok(Some(panel)) => self.panels.insert(0, panel),
                Ok(None) => {},
                Err(e) => {
                    viz_note(&format!("viz smart: grouped-violin panel skipped ({e})"));
                },
            }
        }

        // prepend Lorenz inequality curves: for the most unequal ADDITIVE measures (highest cached
        // Gini — populated only under --smarter), add the plot whose geometry IS the Gini:
        // cumulative population share vs cumulative value share, against the equality diagonal.
        // Capped at LORENZ_MAX_PANELS so a wide finance table can't spawn a full-width panel (and a
        // data pass) per column. Near-unique measures are deliberately allowed — a whole-dollar
        // income or integer population column is often near-unique yet is the flagship inequality
        // case (`classify` skips near-unique integers as IDs, so the Lorenz filter runs
        // independently to surface them). ID/code columns are instead kept out by name:
        // `is_inequality_candidate` excludes intensive- AND identifier-named columns on the
        // stats-only path (the high-Gini gate alone would not).
        {
            let mut lorenz_candidates: Vec<(usize, f64)> = self
                .stats
                .iter()
                .enumerate()
                .filter(|(i, s)| {
                    !self.is_map_col(*i)
                        && matches!(self.col_sems[*i].route, Route::Defer | Route::Measure)
                        && matches!(s.r#type.as_str(), "Integer" | "Float")
                        && s.cardinality > 1
                        && is_inequality_candidate(&self.col_sems[*i], s)
                })
                .map(|(i, s)| (i, s.gini_coefficient.unwrap_or_default()))
                .collect();
            // strongest inequality first, then keep at most LORENZ_MAX_PANELS
            lorenz_candidates.sort_by(|a, b| b.1.total_cmp(&a.1));
            let dropped = lorenz_candidates.len().saturating_sub(LORENZ_MAX_PANELS);
            lorenz_candidates.truncate(LORENZ_MAX_PANELS);
            if dropped > 0 {
                viz_note(&format!(
                    "viz smart: {dropped} additional high-inequality measure(s) not shown as \
                     Lorenz curves (cap {LORENZ_MAX_PANELS})"
                ));
            }
            // insert weakest-first so the strongest inequality ends up topmost among them
            for (idx, gini) in lorenz_candidates.into_iter().rev() {
                let built =
                    build_lorenz_panel(self.args, &self.col_sems, idx, gini, &self.stats[idx]);
                match built {
                    Ok(Some(panel)) => self.panels.insert(0, panel),
                    Ok(None) => {},
                    Err(e) => viz_note(&format!("viz smart: Lorenz panel skipped ({e})")),
                }
            }
        }

        // prepend a parallel-categories (parcats) overview when 3-4 associated low-cardinality
        // categorical dimensions exist. Reserved for 3-4 dims: 2-dim relationships are shown as a
        // directed Sankey, and the hierarchy panel below is suppressed on any dimension set parcats
        // claims (a ribbon flow conveys the same 3-4-way relationship without implying a
        // part-to-whole nesting). HTML-only, like the hierarchy/Sankey panels.
        // An explicit `--hierarchy-style treemap|sunburst|icicle` is a deliberate request for the
        // part-to-whole hierarchy panel, so it suppresses parcats entirely (the user chose the
        // nested view) and, below, bypasses the hierarchy's independence/Sankey screens.
        let explicit_hierarchy_style = matches!(
            self.args
                .flag_hierarchy_style
                .as_deref()
                .map(str::to_ascii_lowercase)
                .as_deref(),
            Some("treemap" | "sunburst" | "icicle")
        );
        let mut parcats_dims: Option<Vec<usize>> = None;
        if !self.out_format.is_image() && !explicit_hierarchy_style {
            // eligible dims = the same non-degenerate String freq-bar columns the hierarchy panel
            // nests (see `eligible_categorical_dims`), so the two panels stay mutually exclusive on
            // a shared pool and neither flows a near-constant (mostly-empty / single-value) column.
            let n = self.row_count();
            let parcats_eligible =
                eligible_categorical_dims(&self.panels, &self.stats, &self.col_sems, n);
            let built = parcats_panel(self.args, parcats_eligible);
            match built {
                Ok(Some((panel, dim_idxs))) => {
                    parcats_dims = Some(dim_idxs);
                    self.panels.insert(0, panel);
                },
                Ok(None) => {},
                Err(e) => viz_note(&format!(
                    "viz smart: parallel-categories (parcats) panel skipped ({e})"
                )),
            }
        }

        // prepend a categorical part-to-whole hierarchy (treemap/sunburst) when 2+ low-cardinality
        // dimensions exist. HTML-only: like Scatter3D/Map, a domain-based trace can't compose with
        // the typed x/y subplot grid, so it forces the inline render path. The chosen dimensions
        // also keep their individual frequency-bar panels — the hierarchy is a cross-dimensional
        // overview, not a replacement. The chart type is auto-selected by depth (treemap for
        // shallow, sunburst for deep) unless --hierarchy-style forces one. Prepended so it survives
        // the panel cap.
        if !self.out_format.is_image() {
            // eligible dims = genuine categorical (String) freq-bar columns with enough distinct
            // values to be worth nesting (HIER_MIN_DIM_CARDINALITY..=CATEGORICAL_MAX_CARDINALITY).
            // Restricting to String type excludes numeric codes (low-card Integer/Float also become
            // freq bars) and booleans, which make poor — and surprising — hierarchy levels. Sort
            // ascending by cardinality so the coarsest grouping is the outermost level/ring.
            let n = self.row_count();
            let mut dims = eligible_categorical_dims(&self.panels, &self.stats, &self.col_sems, n);
            if dims.len() >= HIER_MIN_DIMS {
                dims.sort_by_key(|&(idx, card, _)| (card, idx));
                dims.truncate(HIER_MAX_DEPTH);
                let depth = dims.len();
                let style =
                    resolve_hierarchy_style(self.args.flag_hierarchy_style.as_deref(), depth)?;
                let dim_idxs: Vec<usize> = dims.iter().map(|&(idx, ..)| idx).collect();
                let title = dims
                    .iter()
                    .map(|(.., name)| name.as_str())
                    .collect::<Vec<_>>()
                    .join(" › ");
                // An explicit `--hierarchy-style treemap|sunburst` is a deliberate request that
                // bypasses both the independence screen and the Sankey mutual-exclusion below
                // (computed once above, before the parcats panel).
                let explicit_style = explicit_hierarchy_style;
                // Mutual exclusivity with the directed-flow Sankey panel: a 2-level treemap of
                // exactly the pair already drawn as a Sankey is redundant, so skip it (the flow
                // view conveys that many-to-many relationship better). Deeper 3-level sunbursts are
                // unaffected. Checked BEFORE the hierarchy's own data pass so the pass is never
                // wasted.
                let sankey_owns_pair = depth == 2
                    && !explicit_style
                    && self
                        .sankey_pair
                        .is_some_and(|(a, b)| dim_idxs.contains(&a) && dim_idxs.contains(&b));
                // Mutual exclusivity with the parallel-categories (parcats) panel: parcats owns 3-4
                // associated dimensions, so if it already claimed these columns, skip the
                // hierarchy — the ribbon flow shows the same relationship without implying
                // part-to-whole nesting.
                let parcats_owns = !explicit_style
                    && parcats_dims
                        .as_ref()
                        .is_some_and(|pd| dim_idxs.iter().all(|i| pd.contains(i)));
                if parcats_owns {
                    viz_note(&format!(
                        "viz smart: skipping the '{title}' hierarchy panel — its dimensions are \
                         already shown as a parallel-categories (parcats) panel."
                    ));
                } else if sankey_owns_pair {
                    viz_note(&format!(
                        "viz smart: skipping the '{title}' hierarchy panel — its two dimensions \
                         are already shown as a directed-flow (Sankey) panel, which conveys their \
                         many-to-many relationship better."
                    ));
                } else {
                    // Don't AUTO-nest statistically independent dimensions: a treemap/sunburst of
                    // independent categoricals just replicates each level's marginal at every
                    // branch, conveying nothing the separate frequency bars don't already show more
                    // legibly. The association is read off the joint counts already in hand (no
                    // re-scan).
                    let leaves = collect_hierarchy_counts(self.args, &dim_idxs, None)?;
                    let assoc = max_pairwise_cramers_v(&leaves, depth);
                    if !explicit_style && assoc < HIER_MIN_ASSOCIATION_CRAMERS_V {
                        viz_note(&format!(
                            "viz smart: skipping the '{title}' hierarchy panel — its dimensions \
                             are statistically independent (max Cramér's V={assoc:.2} < \
                             {HIER_MIN_ASSOCIATION_CRAMERS_V:.2}); the per-column frequency bars \
                             convey the same information. Use --hierarchy-style \
                             treemap|sunburst|icicle to force it."
                        ));
                    } else if let Some((labels, parents, values, ids)) = hierarchy_arrays(
                        &leaves,
                        depth,
                        HIER_TOP_N_PER_LEVEL,
                        HIER_MAX_NODES,
                        "All",
                    ) {
                        self.panels.insert(
                            0,
                            Panel::new(
                                title,
                                PanelKind::Hierarchy {
                                    style,
                                    labels,
                                    parents,
                                    values,
                                    ids,
                                },
                            ),
                        );
                    }
                }
            }
        }

        // prepend a time-series trend panel when the data has a date/datetime column and a
        // continuous numeric column. Like the correlation panel, it does one extra data pass and
        // is prepended so it survives the panel cap.
        // Parse dates under the per-column preference resolved in `SmartCtx::new`: a
        // dictionary-declared format wins for its column, otherwise QSV_PREFER_DMY (issue #4303).
        // NOTE this can now legitimately differ from the preference `stats` typed the column
        // under — `viz smart` builds stats with flag_prefer_dmy = false and stats ORs in
        // QSV_PREFER_DMY, and there is no per-column knob to hand it. The consequence is bounded:
        // the raw cells are read per the dictionary; bucket granularity is derived from those
        // same parsed cells (issue #4216), so it follows the dictionary preference too.
        let built = {
            let grain = self.dict_data.as_ref().and_then(|d| d.grain.as_deref());
            let grain_unit = self
                .dict_data
                .as_ref()
                .and_then(|d| d.grain_unit.as_deref());
            let cadence = self.dict_data.as_ref().and_then(|d| d.cadence.as_deref());
            build_timeseries_panel(
                self.args,
                &self.stats,
                &self.dmy_prefs,
                self.map_cols,
                &self.col_sems,
                grain_unit,
                grain,
                cadence,
                self.dict_icons(),
            )?
        };
        if let Some(panel) = built {
            self.panels.insert(0, panel);
        }

        // prepend a cyclic-seasonality polar panel (volume by hour-of-day / weekday / month) when a
        // date column has periodicity worth showing. Polar is non-cartesian (like Scatter3D/map),
        // so it can't be statically exported — build it for HTML only.
        if !self.out_format.is_image() {
            let built = build_cyclic_panel(
                self.args,
                &self.stats,
                &self.dmy_prefs,
                self.map_cols,
                &self.col_sems,
                self.dict_icons(),
            )?;
            if let Some(panel) = built {
                self.panels.insert(0, panel);
            }
        }

        // prepend the pipeline funnel (issue #4222), built during the correlation pass. Inserted
        // after the cyclic/time-series blocks and before the geographic ones, so it sits directly
        // beneath the maps and above the correlation heatmap, Lorenz curves and parcats: geo still
        // leads, but the governance readout is top-of-fold rather than buried among the
        // distribution panels.
        if let Some(panel) = self.funnel_panel.take() {
            self.panels.insert(0, panel);
        }

        // prepend the geographic overview panels (built up front, above) so they lead the Data
        // Schematic and survive the panel cap. Insert the per-country choropleth first,
        // then the point map at index 0, yielding [map, choropleth, ...] — the point map
        // for spatial detail, the choropleth beside it for the per-jurisdiction aggregate.
        if let Some(mut panel) = self.choropleth_panel.take() {
            panel.dict_info = self.geo_dict_info();
            self.panels.insert(0, panel);
        } else if let Some((sc_panels, region_idx)) = self.summary_choros.take() {
            // the region-code summary choropleth(s) replace the (absent) lat/lon choropleth
            // companion. No coordinate column drives them, so anchor --dict-info on the region key
            // column instead of geo_dict_info() (which is None here). Insert in reverse so the
            // leading Data Schematic order is [count, median, …].
            let info = dict_info_for_field(self.dict_icons(), &self.stats[region_idx].field);
            for mut panel in sc_panels.into_iter().rev() {
                panel.dict_info = info.clone();
                self.panels.insert(0, panel);
            }
        }
        if let Some((mut panel, _)) = self.map_panel.take() {
            panel.dict_info = self.geo_dict_info();
            self.panels.insert(0, panel);
        }
        Ok(())
    }

    /// Decide the render path, apply the `--max-charts` interest ranking, and emit the deferred
    /// per-column diagnostics. Errors when no chartable column survived.
    fn finalize_panels(&mut self) -> CliResult<()> {
        // decide the rendering path. Up to MAX_SUBPLOTS panels always use the typed subplot grid
        // (the only form that supports static image export). For HTML output, more than that
        // switches to an inline-div grid (up to MAX_PANELS_INLINE). Image export can't assemble
        // multiple plots, so it stays capped at MAX_SUBPLOTS (with a warning if more were
        // eligible).
        //
        // `--max-charts 0` (the default) means "auto": fit as many panels as the data warrants —
        // every eligible column for HTML (bounded by MAX_PANELS_INLINE), or MAX_SUBPLOTS for image
        // export. An explicit `--max-charts N` caps the panel count to N instead.
        let is_html = matches!(self.out_format, OutFormat::Html);

        // non-cartesian panels (map subplot, geo projection, or 3D scatter) can't share the typed
        // x/y subplot grid, so any of them forces the inline render path. All are HTML-only (never
        // built for image export above).
        let has_noncartesian = self.panels.iter().any(|p| {
            matches!(
                p.kind,
                PanelKind::Map { .. }
                    | PanelKind::Geo { .. }
                    | PanelKind::Choropleth { .. }
                    | PanelKind::ChoroplethMap { .. }
                    | PanelKind::Scatter3D { .. }
                    | PanelKind::CyclicProfile { .. }
                    | PanelKind::Hierarchy { .. }
                    | PanelKind::Sankey { .. }
                    | PanelKind::Parcats { .. }
                    | PanelKind::AnimatedScatterPair { .. }
                    | PanelKind::AnimatedGeo { .. }
                    | PanelKind::AnimatedBubble { .. }
            )
        });

        let eligible = self.panels.len();
        // default (--max-charts 0) draws every eligible panel up to MAX_PANELS_INLINE, for both
        // HTML and static image export. Static export of >MAX_SUBPLOTS panels is rendered via
        // raw-JSON domain-positioned axes (see render_smart_grid_json), so it's no longer capped
        // at 8.
        let requested = if self.args.flag_max_charts == 0 {
            MAX_PANELS_INLINE
        } else {
            self.args.flag_max_charts
        };
        let want = requested.min(eligible);
        // HTML with >MAX_SUBPLOTS panels (or any non-cartesian panel) uses the inline-div grid.
        // Static image export always uses a single composition (typed grid for ≤MAX_SUBPLOTS, raw
        // JSON beyond), so it never takes the inline path; non-cartesian panels stay HTML-only.
        self.inline = is_html && (want > MAX_SUBPLOTS || has_noncartesian);

        let max_panels = requested.min(MAX_PANELS_INLINE);

        if self.panels.len() > max_panels {
            // rank by interestingness rather than dropping the rightmost columns: overview panels
            // (correlation, map, time-series, …) carry infinite interest so they always survive;
            // per-column panels keep the stats-driven `panel_interest` score computed at
            // classification time. Survivors are re-emitted in their original display order, so
            // panel ordering — and everything when nothing overflows — is unchanged; only WHICH
            // panels survive changes. Ties fall back to document order (also the pre-ranking
            // behavior when every score ties).
            let mut order: Vec<usize> = (0..self.panels.len()).collect();
            order.sort_by(|&a, &b| {
                self.panels[b]
                    .interest
                    .partial_cmp(&self.panels[a].interest)
                    .unwrap_or(std::cmp::Ordering::Equal)
                    .then_with(|| a.cmp(&b))
            });
            order.truncate(max_panels);
            let keep: std::collections::HashSet<usize> = order.into_iter().collect();
            let mut kept: Vec<Panel> = Vec::with_capacity(max_panels);
            for (i, p) in self.panels.drain(..).enumerate() {
                if keep.contains(&i) {
                    kept.push(p);
                } else {
                    self.skipped.push(p.name);
                }
            }
            self.panels = kept;
        }

        if !self.sentinel_suspects.is_empty() {
            viz_note(&format!(
                "viz smart: {} column(s) declared a `measure` in the data dictionary were typed \
                 String by stats, so they have no quartiles to chart: {}. Their values span a \
                 numeric range interrupted by a non-numeric token, most often a null sentinel (a \
                 literal \"NULL\", \"N/A\", ...). Confirm with `qsv denull -s {}`.",
                self.sentinel_suspects.len(),
                self.sentinel_suspects.join(", "),
                self.sentinel_suspects.join(",")
            ));
        }
        if !self.sentinel_hints.is_empty() {
            viz_note(&format!(
                "viz smart: {} skipped column(s) may be numeric data held back by a non-numeric \
                 token such as a literal \"NULL\": {}. `qsv denull` decides by scanning the \
                 values.",
                self.sentinel_hints.len(),
                self.sentinel_hints.join(", ")
            ));
        }
        if !self.nonnumeric_measures.is_empty() {
            viz_note(&format!(
                "viz smart: {} column(s) declared a `measure` in the data dictionary hold \
                 non-numeric content and were skipped: {}. Their dictionary role/concept looks \
                 wrong for the column's actual content.",
                self.nonnumeric_measures.len(),
                self.nonnumeric_measures.join(", ")
            ));
        }
        if self.panels.is_empty() {
            // The diagnostics above ran FIRST on purpose. When every column is skipped there is
            // no Data Schematic, and a bare "no chartable columns" told the user nothing about the
            // most likely cause - a null sentinel holding every numeric column back.
            let denull_hint = if self.sentinel_suspects.is_empty() && self.sentinel_hints.is_empty()
            {
                String::new()
            } else {
                " Some skipped columns look like numeric data held back by a non-numeric token; \
                 run `qsv denull` on this file."
                    .to_string()
            };
            return fail_clierror!(
                "No chartable columns found for `viz smart` (all columns were empty or too \
                 high-cardinality to summarize).{denull_hint}"
            );
        }
        if !self.skipped.is_empty() {
            viz_note(&format!(
                "viz smart: charting {} column(s); skipped {}: {}",
                self.panels.len(),
                self.skipped.len(),
                self.skipped.join(", ")
            ));
        }

        // a jittered all-points overlay reads fine at 2-panel size but turns to noise at
        // postage-stamp cell size: in a dense grid (counted AFTER the --max-charts trim, so it
        // reflects what actually renders), demote the size-based `all` box overlay to
        // outliers-only. An explicit --box-points mode is the user's call and is never demoted;
        // violin panels already default to outliers (their KDE shows the distribution).
        if self.explicit_box_points.is_none() && self.panels.len() > SMART_ALL_POINTS_MAX_PANELS {
            for p in &mut self.panels {
                if let PanelKind::BoxRaw { points, .. } = &mut p.kind
                    && matches!(points, BoxPoints::All)
                {
                    *points = BoxPoints::Outliers;
                }
            }
        }
        Ok(())
    }

    /// Gather the per-panel data the renderers need: frequency counts, raw/outlier values, the
    /// Data Schematic title, and the optional dictionary page + metadata table.
    fn gather_panel_data(&mut self) -> CliResult<SmartPanelData> {
        // gather frequency counts for the bar panels in a single pass
        let bar_indices: Vec<usize> = self
            .panels
            .iter()
            .filter_map(|p| match p.kind {
                PanelKind::FreqBar { idx } => Some(idx),
                PanelKind::KpiRow { .. }
                | PanelKind::BoxStats { .. }
                | PanelKind::BoxRaw { .. }
                | PanelKind::BoxOutliers { .. }
                | PanelKind::Violin { .. }
                | PanelKind::GroupedViolin { .. }
                | PanelKind::CorrHeatmap { .. }
                | PanelKind::AssocHeatmap { .. }
                | PanelKind::TopRelationships { .. }
                | PanelKind::TimeSeries { .. }
                | PanelKind::AnimatedScatterPair { .. }
                | PanelKind::AnimatedGeo { .. }
                | PanelKind::AnimatedBubble { .. }
                | PanelKind::ScatterPair { .. }
                | PanelKind::Lorenz { .. }
                | PanelKind::Funnel { .. }
                | PanelKind::ContourPair { .. }
                | PanelKind::Scatter3D { .. }
                | PanelKind::Histogram { .. }
                | PanelKind::MeasureByDim { .. }
                | PanelKind::CyclicProfile { .. }
                | PanelKind::Map { .. }
                | PanelKind::Geo { .. }
                | PanelKind::Choropleth { .. }
                | PanelKind::ChoroplethMap { .. }
                | PanelKind::Hierarchy { .. }
                | PanelKind::Sankey { .. }
                | PanelKind::Parcats { .. } => None,
            })
            .collect();
        let top_n = self.args.flag_limit.max(1);
        if !bar_indices.is_empty() {
            self.progress.set_message("Computing frequencies…");
        }
        // whether the frequency cache was actually READ (not merely present) — the `--dict-info`
        // download row offers the cache only on a genuine hit.
        let mut freq_cache_used = false;
        let freq = if bar_indices.is_empty() {
            HashMap::new()
        } else {
            // Prefer a pre-existing `frequency` cache (no data pass); fall back to a
            // single-pass recompute when it's absent/stale/incompatible.
            match freq_from_cache(self.args, &self.stats, &bar_indices, top_n) {
                Some(cached) => {
                    freq_cache_used = true;
                    cached
                },
                None => count_values(self.args, &bar_indices, top_n)?,
            }
        };

        // dominant-category title hint: when one real category holds nearly all the rows, the bar
        // chart is one tall bar — say so in the title so the panel reads at a glance. Appended here
        // (not at classification) because it needs the finalized per-bar counts.
        {
            // field-precise destructuring: the title loop mutates `panels` while the hint's lazy
            // row-count callback needs `nrows`/`count_conf` (so `self.row_count()` can't be used).
            let Self {
                panels,
                nrows,
                count_conf,
                ..
            } = self;
            for p in panels.iter_mut() {
                if let PanelKind::FreqBar { idx } = p.kind
                    && let Some(bars) = freq.get(&idx)
                    && let Some(hint) = dominant_category_hint(bars, || {
                        // exact row count, shared with the box-points heuristic's lazy fetch above
                        // and only computed when a panel's drawn share already clears the threshold
                        *nrows.get_or_insert_with(|| util::count_rows(count_conf).unwrap_or(0))
                    })
                {
                    p.name = format!("{} {hint}", p.name);
                }
            }
        }

        // gather raw values for the panels that need them — histogram panels (moarstats-flagged
        // bimodal columns), raw box panels and violin panels — in a single batched pass. Taken only
        // when at least one such panel exists. Sampled violins carry a per-column stride so the
        // pass keeps every stride-th non-null value (bounding both memory and the embedded HTML).
        let raw_indices: Vec<usize> = self
            .panels
            .iter()
            .filter_map(|p| match p.kind {
                PanelKind::Histogram { idx }
                | PanelKind::BoxRaw { idx, .. }
                | PanelKind::Violin { idx, .. } => Some(idx),
                _ => None,
            })
            .collect();
        let raw_strides: HashMap<usize, usize> = self
            .panels
            .iter()
            .filter_map(|p| match p.kind {
                PanelKind::Violin {
                    idx, sample_stride, ..
                } if sample_stride > 1 => Some((idx, sample_stride)),
                _ => None,
            })
            .collect();
        // large box panels that have outliers: collect ONLY their out-of-fence values, keyed by the
        // Tukey fences resolved during classification (folded into the same pass as `raw_indices`).
        let fence_bounds: HashMap<usize, (f64, f64)> = self
            .panels
            .iter()
            .filter_map(|p| match p.kind {
                PanelKind::BoxOutliers {
                    idx,
                    fence_low,
                    fence_high,
                    ..
                } => Some((idx, (fence_low, fence_high))),
                _ => None,
            })
            .collect();
        let (raw_values, outlier_stats) = if raw_indices.is_empty() && fence_bounds.is_empty() {
            (HashMap::new(), HashMap::new())
        } else {
            self.progress.set_message("Preparing panel data…");
            collect_smart_values(self.args, &raw_indices, &raw_strides, &fence_bounds)?
        };

        // Data Schematic title: the user's --title, else the dataset's file name
        let title_text = self.args.flag_title.clone().unwrap_or_else(|| {
            let dataset = std::path::Path::new(self.args.arg_input.as_deref().unwrap_or("data"))
                .file_name()
                .and_then(|s| s.to_str())
                .unwrap_or("data");
            t!("viz.title.data_overview", q_dataset = dataset).into_owned()
        });

        // --dict-info: render the embedded Data Dictionary page when a usable dictionary is present
        // and the output is HTML; soft no-op (with a note) otherwise, matching the dictionary
        // loader's fail-to-stats philosophy.
        let dict_page: Option<String> = if !self.args.flag_dict_info {
            None
        } else if !matches!(self.out_format, OutFormat::Html) {
            viz_note(
                "viz smart --dict-info: only applies to HTML output; ignored for image export.",
            );
            None
        } else if let (Some(dict), Some(json_text)) =
            (self.dict_data.as_ref(), self.dict_json_text.as_deref())
        {
            match serde_json::from_str::<serde_json::Value>(json_text) {
                Ok(dict_json) => {
                    // section order = the dataset's header order (dictionary-only columns follow)
                    let column_order: Vec<String> =
                        self.stats.iter().map(|s| s.field.clone()).collect();
                    // "View chart" targets: only the inline path emits per-panel `data-qsv-dict`
                    // cells, and only for panels that survived the cap — the typed grid is one
                    // plot with nothing to scroll to, so its dictionary renders no such links.
                    let view_chart_anchors: std::collections::HashSet<String> = if self.inline {
                        self.panels
                            .iter()
                            .filter_map(|p| p.dict_info.as_ref().map(|(a, _)| a.clone()))
                            .collect()
                    } else {
                        std::collections::HashSet::new()
                    };
                    // downloads for the generated sidecars this run actually consumed, offered
                    // beside the "Export JSONSchema" link so a reader of the standalone HTML can
                    // pull down the inputs the Data Schematic was built from.
                    // `stats_input` is the exact string handed to `get_stats_records`, resolved
                    // once in `smart_prepare`. Anchoring on it means the offered stats sidecars
                    // are the ones the Data Schematic's statistics were actually read from — not a
                    // second `dc:` resolution that could have refreshed a stale entry in between.
                    let sidecars = collect_sidecar_downloads(
                        self.args.arg_input.as_deref(),
                        Some(std::path::Path::new(&self.stats_input)),
                        &self.stats,
                        &freq,
                        freq_cache_used,
                        self.bivariate_sidecar_fresh,
                    );
                    Some(render_dict_page_html(
                        &dict_json,
                        json_text,
                        dict,
                        &title_text,
                        &column_order,
                        &view_chart_anchors,
                        &sidecars,
                    ))
                },
                // unreachable in practice: dict_data only parses from valid JSON
                Err(_) => None,
            }
        } else {
            viz_note(
                "viz smart --dict-info: no usable data dictionary (see --dictionary); showing the \
                 Data Schematic without dictionary info.",
            );
            None
        };

        // top-of-Data Schematic metadata table (HTML output only, mirroring --dict-info's HTML-only
        // scope). Rows/Columns/Compiled always render, so the table always has >=3 rows;
        // Description and PID are conditional. Image-export paths carry no table (would require
        // Plotly-layout annotations).
        let mut data_viewer: Option<(String, String)> = None;
        let metadata_html: Option<String> = if matches!(self.out_format, OutFormat::Html) {
            let mut rows = String::new();
            // Description: first paragraph of the dictionary's dataset-level description, only when
            // --dict-info is set and a non-empty description exists.
            if self.args.flag_dict_info
                && let Some(desc) = self
                    .dict_data
                    .as_ref()
                    .and_then(|d| d.dataset_description.as_deref())
            {
                let first_para = first_description_paragraph(desc);
                if !first_para.is_empty() {
                    rows.push_str(&format!(
                        "<tr><td class=\"qsv-viz-meta-k\">{}</td><td>{}</td></tr>\n",
                        t!("viz.meta.description"),
                        html_escape(&first_para)
                    ));
                }
            }
            // Rows: reuse the lazily-computed dataset row count. When the data viewer drawer is
            // enabled (issue #4283), the count links to it: "(Explore)" when every row is
            // embedded, "(Preview)" when only the first --preview-threshold rows are.
            let n = self.row_count();
            data_viewer = build_data_viewer_chrome(self.args, &self.stats, n, &self.dmy_prefs)?;
            let data_link = match &data_viewer {
                Some((_, label)) => format!(
                    " <a href=\"#\" class=\"qsv-data-link\" onclick=\"return \
                     qsvOpenData()\">{label}{DATA_LINK_ICON}</a>"
                ),
                None => String::new(),
            };
            rows.push_str(&format!(
                "<tr><td class=\"qsv-viz-meta-k\">{}</td><td>{}{data_link}</td></tr>\n",
                t!("viz.meta.rows"),
                HumanCount(n)
            ));
            // Columns
            rows.push_str(&format!(
                "<tr><td class=\"qsv-viz-meta-k\">{}</td><td>{}</td></tr>\n",
                t!("viz.meta.columns"),
                HumanCount(self.stats.len() as u64)
            ));
            // Completeness: share of non-empty cells across the whole dataset. A quiet header stat
            // rather than a KPI gauge — it's near 100% for most datasets, so a permanent gauge tile
            // just crowded the overview row.
            let ncols = self.stats.len();
            if n > 0 && ncols > 0 {
                let total_cells = n as f64 * ncols as f64;
                let total_null: f64 = self.stats.iter().map(|s| s.nullcount as f64).sum();
                let completeness = (1.0 - total_null / total_cells).clamp(0.0, 1.0);
                rows.push_str(&format!(
                    "<tr><td class=\"qsv-viz-meta-k\">{}</td><td>{:.1}%</td></tr>\n",
                    t!("viz.meta.completeness"),
                    completeness * 100.0
                ));
            }
            // Compiled: Data Schematic build timestamp (makes smart HTML output non-deterministic
            // by design).
            rows.push_str(&format!(
                "<tr><td class=\"qsv-viz-meta-k\">{}</td><td>{}</td></tr>\n",
                t!("viz.meta.compiled"),
                chrono::Utc::now().format("%Y-%m-%d %H:%M UTC")
            ));
            // Generated by: the producing tool AND its version. A Data Schematic must be
            // attributable to the thing that drew it — a reader who cannot tell which qsv wrote a
            // panel cannot re-derive it, and panel-selection heuristics change between releases.
            // Covers MUST 9 (Provenance) of docs/DATA_SCHEMATIC.md in a single row.
            rows.push_str(&format!(
                "<tr><td class=\"qsv-viz-meta-k\">{}</td><td>qsv {}</td></tr>\n",
                t!("viz.meta.generated_by"),
                env!("CARGO_PKG_VERSION")
            ));
            // PID: a clickable link, only when --dataset-pid is set. A full URL such as a DOI is
            // expected. The TEXT is html-escaped and the HREF additionally goes through
            // `sanitize_url` — html-escaping alone leaves `javascript:` intact, and this page may
            // be generated by a service and viewed by someone other than whoever passed the flag. A
            // rejected scheme still shows the value as plain text rather than dropping it silently.
            if let Some(pid) = self.args.flag_dataset_pid.as_deref() {
                let pid_esc = html_escape(pid);
                let cell = match sanitize_url(pid) {
                    Some(href) => format!(
                        "<a href=\"{}\" target=\"_blank\" rel=\"noopener \
                         noreferrer\">{pid_esc}</a>",
                        html_escape(&href)
                    ),
                    None => pid_esc,
                };
                rows.push_str(&format!(
                    "<tr><td class=\"qsv-viz-meta-k\">{}</td><td>{cell}</td></tr>\n",
                    t!("viz.meta.pid")
                ));
            }
            Some(format!("<table class=\"qsv-viz-meta\">\n{rows}</table>"))
        } else {
            None
        };

        Ok(SmartPanelData {
            freq,
            raw_values,
            outlier_stats,
            title_text,
            dict_page,
            metadata_html,
            data_chrome: data_viewer.map(|(chrome, _)| chrome),
        })
    }

    /// Prepend the KPI overview row and dispatch to the render path chosen by `finalize_panels`.
    fn render(mut self, data: SmartPanelData) -> CliResult<SmartRender> {
        let SmartPanelData {
            freq,
            raw_values,
            outlier_stats,
            title_text,
            dict_page,
            metadata_html,
            data_chrome,
        } = data;

        // a Geo panel in image mode must use the raw-JSON grid: it injects a domain-positioned
        // `geo` subplot, which the typed `Layout` (a single `.geo()`, no per-cell domain) can't
        // express.
        let has_geo_image = self.out_format.is_image()
            && self
                .panels
                .iter()
                .any(|p| matches!(p.kind, PanelKind::Geo { .. }));

        // Leading KPI overview row, prepended LAST (after every panel-count-dependent decision —
        // the `--max-charts` ranking, the "charting N columns" note, the box-overlay demotion, and
        // the render-path choice) so it stays invisible to them and simply lands at index 0, on top
        // of the Data Schematic. HTML only: Indicator tiles are domain-positioned and never enter a
        // static image.
        if !self.out_format.is_image()
            && let Some(panel) = build_kpi_row(&self.stats, &self.panels, self.dict_data.as_ref())
        {
            self.panels.insert(0, panel);
        }

        self.progress.set_message("Rendering Data Schematic…");
        if self.inline {
            Ok(SmartRender::Inline(render_smart_inline(
                self.args,
                &self.panels,
                &freq,
                &raw_values,
                &outlier_stats,
                &title_text,
                self.log_scale,
                dict_page.as_deref(),
                metadata_html.as_deref(),
                data_chrome.as_deref(),
                self.map_cols,
            )))
        } else if self
            .panels
            .iter()
            .filter(|p| !matches!(p.kind, PanelKind::KpiRow { .. }))
            .count()
            > MAX_SUBPLOTS
            || has_geo_image
        {
            // static image export of >MAX_SUBPLOTS panels (or any panel count with a geo subplot):
            // plotly's typed Layout only has xaxis1..xaxis8 and a single typed `geo`, so assemble
            // the grid as raw JSON with domain-positioned xaxis9+ and (for geo panels) geo/geo2+
            // subplots.
            render_smart_grid_json(
                self.args,
                &self.panels,
                &freq,
                &raw_values,
                &outlier_stats,
                &title_text,
                self.log_scale,
            )
        } else {
            render_smart_grid(
                self.args,
                &self.panels,
                &freq,
                &raw_values,
                &outlier_stats,
                &title_text,
                self.log_scale,
                dict_page,
                metadata_html,
                data_chrome,
            )
        }
    }
}

/// Build the `viz smart` auto-dashboard from the dataset's statistics + frequency data.
/// Classifies columns into panels, then renders either a single-`Plot` subplot grid (≤8 panels,
/// or any image export) or a self-contained inline-div HTML page (>8 panels, HTML output).
fn build_smart(
    args: &Args,
    out_format: OutFormat,
    progress: &ProgressBar,
    show_progress: bool,
    loaded_geojson: Option<&serde_json::Value>,
) -> CliResult<SmartRender> {
    let prep = smart_prepare(args, progress, show_progress)?;
    let mut ctx = SmartCtx::new(&prep, out_format, progress, loaded_geojson)?;
    ctx.classify_columns();
    ctx.add_relationship_panels()?;
    ctx.add_overview_panels()?;
    ctx.finalize_panels()?;
    let panel_data = ctx.gather_panel_data()?;
    ctx.render(panel_data)
}

/// The equality-diagonal reference line for a `Lorenz` panel: a muted, dotted (0,0)→(1,1) line the
/// caller pushes as a SECOND trace right after the curve (both bound to the same subplot axes when
/// `axes` is `Some`). A perfectly equal distribution would trace this diagonal; the gap below it is
/// the inequality the Gini measures. `HoverInfo::Skip` keeps it out of the hover, and the global
/// `show_legend(false)` keeps it off the legend.
fn lorenz_diagonal_trace(axes: Option<(String, String)>) -> Box<dyn Trace> {
    let mut d = Scatter::new(vec![0.0_f64, 1.0], vec![0.0_f64, 1.0])
        .mode(Mode::Lines)
        .name("equality")
        .line(
            Line::new()
                .color(MUTED_COLOR)
                .dash(plotly::common::DashType::Dot),
        )
        .hover_info(HoverInfo::Skip);
    if let Some((x, y)) = &axes {
        d = d.x_axis(x.clone()).y_axis(y.clone());
    }
    d
}

/// Build the plotly trace for one smart-Data Schematic panel. `axes` carries the subplot axis refs
/// when rendering into the typed grid; pass `None` for a standalone inline-div plot (which uses
/// the default x/y axes). Returns the trace plus, for bar panels, the tallest bar value (used
/// to add vertical headroom for the outside value labels) and whether that panel's y-axis
/// should be logarithmic (per `--log-scale`; bar panels decide from their counts, box panels
/// from the verdict baked in at classification time; `false` for every other panel kind).
///
/// One `#[inline(never)]` function per panel kind, dispatched here: a single body with
/// every arm needs far more stack at `opt-level=0` (no slot reuse across arms), which
/// overflowed Windows' 1 MB debug main-thread stack (issue #4328).
fn panel_trace(
    panel: &Panel,
    color: &'static str,
    freq: &FreqMap,
    hist: &HashMap<usize, Vec<f64>>,
    outliers: &HashMap<usize, OutlierStats>,
    axes: Option<(String, String)>,
    theme: Option<BuiltinTheme>,
    log_scale: LogScale,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    match &panel.kind {
        PanelKind::BoxStats { .. } => panel_trace_box_stats(panel, color, axes),
        PanelKind::BoxRaw { .. } => panel_trace_box_raw(panel, color, hist, axes),
        PanelKind::Violin { .. } => panel_trace_violin(panel, color, hist, axes),
        PanelKind::GroupedViolin { .. } => panel_trace_grouped_violin(panel, color, axes),
        PanelKind::BoxOutliers { .. } => panel_trace_box_outliers(panel, color, outliers, axes),
        PanelKind::FreqBar { .. } => {
            panel_trace_freq_bar(panel, color, freq, axes, theme, log_scale)
        },
        PanelKind::Histogram { .. } => panel_trace_histogram(panel, color, hist, axes),
        PanelKind::TimeSeries { .. } => panel_trace_time_series(panel, color, axes),
        PanelKind::ScatterPair { .. } => panel_trace_scatter_pair(panel, color, axes),
        PanelKind::Lorenz { .. } => panel_trace_lorenz(panel, color, axes),
        PanelKind::Funnel {
            form: PipelineForm::Bridge,
            ..
        } => panel_trace_funnel_bridge(panel, color, axes),
        PanelKind::Funnel { .. } => panel_trace_funnel(panel, color, axes),
        PanelKind::ContourPair { .. } => panel_trace_contour_pair(panel, axes),
        PanelKind::CorrHeatmap { .. } => panel_trace_corr_heatmap(panel, axes),
        PanelKind::AssocHeatmap { .. } => panel_trace_assoc_heatmap(panel, axes),
        PanelKind::TopRelationships { .. } => panel_trace_top_relationships(panel, axes),
        PanelKind::MeasureByDim { .. } => {
            panel_trace_measure_by_dim(panel, color, axes, theme, log_scale)
        },
        // map / geo / 3D / hierarchy panels use a non-cartesian layout (map, geo projection,
        // 3D scene, or domain-based treemap/sunburst) that can't share the typed x/y subplot grid,
        // so they are rendered entirely by `smart_inline_panel_plot` and never reach this
        // assembler.
        PanelKind::KpiRow { .. }
        | PanelKind::Map { .. }
        | PanelKind::Geo { .. }
        | PanelKind::Choropleth { .. }
        | PanelKind::ChoroplethMap { .. }
        | PanelKind::Scatter3D { .. }
        | PanelKind::CyclicProfile { .. }
        | PanelKind::Hierarchy { .. }
        | PanelKind::Sankey { .. }
        | PanelKind::Parcats { .. }
        | PanelKind::AnimatedScatterPair { .. }
        | PanelKind::AnimatedGeo { .. }
        | PanelKind::AnimatedBubble { .. } => {
            unreachable!(
                "map/geo/choropleth/3D/polar/hierarchy/sankey/parcats/animated-scatter panels are \
                 rendered via the inline path, not panel_trace"
            )
        },
    }
}

#[inline(never)]
fn panel_trace_box_stats(
    panel: &Panel,
    color: &'static str,
    axes: Option<(String, String)>,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    let log_y;
    let PanelKind::BoxStats {
        q1,
        median,
        q3,
        lower,
        upper,
        mean,
    } = &panel.kind
    else {
        unreachable!("panel_trace dispatches on panel.kind")
    };
    let trace: Box<dyn Trace> = {
        // value-axis log verdict resolved at classification time (see `box_panel_logs`)
        log_y = panel.value_log;
        // `Panel::name` is stored RAW (it also feeds stderr), so every markup sink escapes
        // it itself — escaped exactly once per sink. Do NOT hoist this into `Panel::new`.
        let mut b = BoxPlot::new(Vec::<f64>::new())
            .name(escape_hover(&panel.name))
            .q1(vec![*q1])
            .median(vec![*median])
            .q3(vec![*q3])
            .marker(Marker::new().color(color))
            // show only the y stats in the hover ("median: 202.771k"), not plotly's default
            // "(<trace name>, median: ...)" which repeats the (long) column name on every
            // statistic line — the column name is already the panel title.
            .hover_info(HoverInfo::Y);
        if let Some((x, y)) = &axes {
            b = b.x_axis(x.clone()).y_axis(y.clone());
        }
        if let Some(l) = lower {
            // clamp a non-positive lower whisker up to q1 on a log axis (issue #4219)
            b = b.lower_fence(vec![log_safe_lower_fence(log_y, *l, *q1)]);
        }
        if let Some(u) = upper {
            b = b.upper_fence(vec![*u]);
        }
        if let Some(m) = mean {
            b = b.mean(vec![*m]);
        }
        if let Some((fill, outline)) = log_distribution_body_cue(log_y) {
            b = b.fill_color(fill).line(Line::new().color(outline));
        }
        b
    };
    (trace, None, log_y)
}

#[inline(never)]
fn panel_trace_box_raw(
    panel: &Panel,
    color: &'static str,
    hist: &HashMap<usize, Vec<f64>>,
    axes: Option<(String, String)>,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    let log_y;
    let PanelKind::BoxRaw { idx, points } = &panel.kind else {
        unreachable!("panel_trace dispatches on panel.kind")
    };
    let trace: Box<dyn Trace> = {
        // opt-in / heuristic raw box: plotly computes the quartiles + true Tukey whiskers from
        // the values and overlays the sample points per this panel's chosen --box-points mode
        log_y = panel.value_log;
        // plotly derives this box's whiskers from the values, so non-positive values must not
        // reach it on a log axis (issue #4219 — see `log_safe_values`)
        let values = log_safe_values(log_y, hist.get(idx).cloned().unwrap_or_default());
        let mut b = BoxPlot::new(values)
            .name(escape_hover(&panel.name))
            .quartile_method(QuartileMethod::Linear)
            .box_points(points.clone())
            .marker(Marker::new().color(color))
            // show only the y stats in the hover ("median: 202.771k"), not plotly's default
            // "(<trace name>, median: ...)" which repeats the (long) column name on every
            // statistic line — the column name is already the panel title.
            .hover_info(HoverInfo::Y);
        if let Some((x, y)) = &axes {
            b = b.x_axis(x.clone()).y_axis(y.clone());
        }
        if let Some((fill, outline)) = log_distribution_body_cue(log_y) {
            b = b.fill_color(fill).line(Line::new().color(outline));
        }
        b
    };
    (trace, None, log_y)
}

#[inline(never)]
fn panel_trace_violin(
    panel: &Panel,
    color: &'static str,
    hist: &HashMap<usize, Vec<f64>>,
    axes: Option<(String, String)>,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    let log_y;
    let PanelKind::Violin { idx, points, .. } = &panel.kind else {
        unreachable!("panel_trace dispatches on panel.kind")
    };
    let trace: Box<dyn Trace> = {
        // the default distribution panel: plotly estimates the KDE silhouette from the
        // (possibly stride-sampled) values and draws the quartile box + mean line inside
        // it, with the sample points overlaid per this panel's resolved --box-points mode
        // (no overlay at all for sampled violins)
        log_y = panel.value_log;
        // plotly estimates the KDE + inner box from the values, so non-positive values must
        // not reach it on a log axis (issue #4219 — see `log_safe_values`)
        let values = log_safe_values(log_y, hist.get(idx).cloned().unwrap_or_default());
        let mut v = Violin::new(values)
            .name(escape_hover(&panel.name))
            .quartile_method(QuartileMethod::Linear)
            .box_plot(ViolinBox::new().visible(true))
            .mean_line(MeanLine::new().visible(true))
            .points(violin_points(points))
            .marker(Marker::new().color(color))
            .line(Line::new().color(color))
            // clamp the KDE to the observed data range so the silhouette can't bleed past a
            // natural bound (density below 0 for a count, above 100 for a percentage); the
            // default "soft" span extends a bandwidth beyond the extremes, which misleads on
            // bounded/discrete columns.
            .span_mode(SpanMode::Hard)
            // show only the y stats in the hover, not plotly's default "(<trace name>,
            // ...)" which repeats the (long) column name — it's already the panel title.
            .hover_info(HoverInfo::Y);
        if let Some((x, y)) = &axes {
            v = v.x_axis(x.clone()).y_axis(y.clone());
        }
        if let Some((fill, outline)) = log_distribution_body_cue(log_y) {
            v = v.fill_color(fill).line(Line::new().color(outline));
        }
        v
    };
    (trace, None, log_y)
}

#[inline(never)]
fn panel_trace_grouped_violin(
    panel: &Panel,
    color: &'static str,
    axes: Option<(String, String)>,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    let PanelKind::GroupedViolin {
        groups,
        values,
        points,
    } = &panel.kind
    else {
        unreachable!("panel_trace dispatches on panel.kind")
    };
    let trace: Box<dyn Trace> = {
        // distribution-by-category overview: one KDE silhouette per category (categories on
        // the shared x-axis, the measure on y). A single `new_xy` trace, so it composes with
        // the typed subplot grid and static image export like the MeasureByDim bar.
        let mut v = Violin::new_xy(groups.clone(), values.clone())
            .quartile_method(QuartileMethod::Linear)
            .box_plot(ViolinBox::new().visible(true))
            .mean_line(MeanLine::new().visible(true))
            .points(violin_points(points))
            .marker(Marker::new().color(color))
            .line(Line::new().color(color))
            // clamp each category's KDE to its observed range (see the single-column violin
            // above) so a bounded/discrete measure can't render density past a natural limit.
            .span_mode(SpanMode::Hard);
        if let Some((x, y)) = &axes {
            v = v.x_axis(x.clone()).y_axis(y.clone());
        }
        v
    };
    (trace, None, false)
}

#[inline(never)]
fn panel_trace_box_outliers(
    panel: &Panel,
    color: &'static str,
    outliers: &HashMap<usize, OutlierStats>,
    axes: Option<(String, String)>,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    let log_y;
    let PanelKind::BoxOutliers {
        idx,
        q1,
        median,
        q3,
        mean,
        ..
    } = &panel.kind
    else {
        unreachable!("panel_trace dispatches on panel.kind")
    };
    let trace: Box<dyn Trace> = {
        // large column WITH outliers: a precomputed quartile box whose whiskers end at the
        // observed in-fence extremes, with the out-of-fence values overlaid as NATIVE box
        // points. The outliers are passed as a 2D `y` (`[[...]]`, via `Y = Vec<f64>`), which
        // makes plotly draw them as points WITHOUT recomputing the box from them — a 1D `y`
        // renders the box but drops the points.
        log_y = panel.value_log;
        let stats = outliers.get(idx);
        // the precomputed whisker is clamped below, but these are raw VALUES handed to
        // plotly — a column granted a log axis by box_log_skew_fallback can carry
        // zero/negative outliers, which are log-undefined exactly like the BoxRaw/Violin
        // case (issue #4219).
        let pts = log_safe_values(log_y, stats.map(|o| o.outliers.clone()).unwrap_or_default());
        // clamp a non-positive lower whisker up to q1 on a log axis (issue #4219), else the
        // log-undefined fence breaks the box and only the outlier points survive
        let whisker_low = log_safe_lower_fence(log_y, stats.map_or(*q1, |o| o.whisker_low), *q1);
        let whisker_high = stats.map_or(*q3, |o| o.whisker_high);
        let mut b = BoxPlot::<f64, Vec<f64>>::new(vec![pts])
            .name(escape_hover(&panel.name))
            .q1(vec![*q1])
            .median(vec![*median])
            .q3(vec![*q3])
            .lower_fence(vec![whisker_low])
            .upper_fence(vec![whisker_high])
            .box_points(BoxPoints::All)
            .point_pos(0.0)
            .jitter(0.0)
            .marker(Marker::new().color(color).size(4))
            .hover_info(HoverInfo::Y);
        if let Some(m) = mean {
            b = b.mean(vec![*m]);
        }
        if let Some((x, y)) = &axes {
            b = b.x_axis(x.clone()).y_axis(y.clone());
        }
        if let Some((fill, outline)) = log_distribution_body_cue(log_y) {
            b = b.fill_color(fill).line(Line::new().color(outline));
        }
        b
    };
    (trace, None, log_y)
}

#[inline(never)]
fn panel_trace_freq_bar(
    panel: &Panel,
    color: &'static str,
    freq: &FreqMap,
    axes: Option<(String, String)>,
    theme: Option<BuiltinTheme>,
    log_scale: LogScale,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    let bar_max: Option<f64>;
    let log_y;
    // bar value-label font: omit the explicit color so the label inherits the layout font color
    // (qsv's ink in the unthemed look, the template's font when themed) -- this lets the dark/light
    // toggle's `font.color` relayout flip the labels instead of leaving them dark on a dark page.
    let label_font = {
        let f = Font::new().size(9);
        if theme.is_some() {
            f
        } else {
            f.family(FONT_FAMILY)
        }
    };
    let PanelKind::FreqBar { idx } = &panel.kind else {
        unreachable!("panel_trace dispatches on panel.kind")
    };
    let trace: Box<dyn Trace> = {
        let bars = freq.get(idx).cloned().unwrap_or_default();
        // x = each bar's distinct category-axis key (real value, or aggregate sentinel) so
        // distinct categories never collapse onto the same plotly category; the friendly,
        // truncated tick labels are applied separately via freq_bar_tick_text.
        let xs: Vec<String> = bars.iter().map(|b| b.x_key.clone()).collect();
        let ys: Vec<f64> = bars.iter().map(|b| b.count as f64).collect();
        // muted-grey the aggregate "(NULL)" / "Other (N)" buckets so they read as summary
        // bars, visually distinct from the palette-colored real categories. Color is driven
        // by the bar's kind (not its label), so a real category named like an aggregate is
        // never mis-colored.
        let bar_colors: Vec<&'static str> = bars
            .iter()
            .map(|b| match b.kind {
                FreqBarKind::Aggregate => MUTED_COLOR,
                FreqBarKind::Category => color,
            })
            .collect();
        bar_max = Some(ys.iter().copied().fold(0.0_f64, f64::max));
        // high dynamic range (a dominating "(NULL)"/"Other" bucket) -> log y-axis so the
        // small real categories stay visible; gated by the resolved --log-scale mode.
        log_y = panel_is_log(panel, freq, log_scale);
        // on a log y-axis, hatch the muted-grey aggregate bars as a redundant cue
        // (alongside the "count (log)" axis title) that the axis is non-linear
        let mut marker = Marker::new().color_array(bar_colors);
        if let Some(shapes) = freq_bar_pattern_shapes(&bars, log_y) {
            marker = marker.pattern(Pattern::new().shape_array(shapes));
        }
        // The category axis shows TRUNCATED tick labels, and plotly's default hover reads
        // that same tick text — so the hover carries each bar's FULL label explicitly.
        // `label` (not `x_key`): the aggregate buckets' axis keys are sentinel-padded.
        let hover_labels: Vec<String> = bars.iter().map(|b| escape_hover(&b.label)).collect();
        let mut bar = Bar::new(xs, ys)
            .name(escape_hover(&panel.name))
            .marker(marker)
            .hover_text_array(hover_labels)
            .hover_template(&t!("viz.hover.count_by_label"))
            // value labels above each bar, SI-formatted ("258k", "1.05M") to match
            // the axis ticks
            .text_template("%{y:.3s}")
            .text_position(TextPosition::Outside)
            // don't clip the outside label of the tallest bar at the cell's top edge. On a
            // log axis the fixed top headroom shrinks to a few pixels when counts span many
            // decades (a dominant "(NULL)"/"Other" bucket), so its value label would otherwise
            // be cut off; let it draw into the inter-row gap instead.
            .clip_on_axis(false)
            .text_font(label_font);
        if let Some((x, y)) = &axes {
            bar = bar.x_axis(x.clone()).y_axis(y.clone());
        }
        bar
    };
    (trace, bar_max, log_y)
}

#[inline(never)]
fn panel_trace_histogram(
    panel: &Panel,
    color: &'static str,
    hist: &HashMap<usize, Vec<f64>>,
    axes: Option<(String, String)>,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    let PanelKind::Histogram { idx } = &panel.kind else {
        unreachable!("panel_trace dispatches on panel.kind")
    };
    let trace: Box<dyn Trace> = {
        let values = hist.get(idx).cloned().unwrap_or_default();
        // the cell has no x-axis title (panel.name is only a cell annotation), so name the
        // binned value and its count in the hover, both comma-grouped.
        // NOTE: `escape_hover` only, no `escape_template_pct` -- preserved from before this
        // string was localized. A `%` in the panel name is mis-read by plotly here, unlike in
        // the contour/Lorenz hovers which take the full composition. Pre-existing; changing it
        // would be a behavior change, not a translation.
        let hover = t!("viz.hover.histogram", q_col = escape_hover(&panel.name)).into_owned();
        let mut h = Histogram::new(values)
            .name(escape_hover(&panel.name))
            .marker(Marker::new().color(color))
            .hover_template(hover);
        if let Some((x, y)) = &axes {
            h = h.x_axis(x.clone()).y_axis(y.clone());
        }
        h
    };
    (trace, None, false)
}

#[inline(never)]
fn panel_trace_time_series(
    panel: &Panel,
    color: &'static str,
    axes: Option<(String, String)>,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    let PanelKind::TimeSeries {
        y_label,
        xs,
        ys,
        categorical: _,
    } = &panel.kind
    else {
        unreachable!("panel_trace dispatches on panel.kind")
    };
    let trace: Box<dyn Trace> = {
        let mut t = Scatter::new(xs.clone(), ys.clone())
            .mode(Mode::Lines)
            .name(escape_hover(y_label.as_str()))
            .line(Line::new().color(color));
        if let Some((x, y)) = &axes {
            t = t.x_axis(x.clone()).y_axis(y.clone());
        }
        t
    };
    (trace, None, false)
}

#[inline(never)]
fn panel_trace_scatter_pair(
    panel: &Panel,
    color: &'static str,
    axes: Option<(String, String)>,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    let PanelKind::ScatterPair {
        xs,
        ys,
        sizes,
        x_label,
        y_label,
        size_label,
    } = &panel.kind
    else {
        unreachable!("panel_trace dispatches on panel.kind")
    };
    let trace: Box<dyn Trace> = {
        let mut marker = Marker::new().color(color);
        if let Some(sizes) = sizes {
            marker = marker.size_array(scale_bubble_sizes(sizes));
        }
        // The marker encodes SCALED pixel sizes, so `%{marker.size}` can't surface the real
        // third value — pre-render a per-point label naming x/y (the cell has no axis titles)
        // and, when present, the size column's RAW value. All comma-grouped via `fmt_measure`.
        let xl = escape_hover(x_label);
        let yl = escape_hover(y_label);
        let sl = size_label.as_deref().map(escape_hover);
        let hover: Vec<String> = xs
            .iter()
            .zip(ys.iter())
            .enumerate()
            .map(|(pt, (x, y))| {
                let mut label = format!("{xl}: {}<br>{yl}: {}", fmt_measure(*x), fmt_measure(*y));
                if let (Some(sl), Some(s)) = (sl.as_ref(), sizes.as_ref().and_then(|sz| sz.get(pt)))
                {
                    label.push_str(&format!("<br>{sl}: {}", fmt_measure(*s)));
                }
                label
            })
            .collect();
        let mut t = Scatter::new(xs.clone(), ys.clone())
            .mode(Mode::Markers)
            .name(escape_hover(&panel.name))
            .marker(marker)
            .hover_text_array(hover)
            .hover_info(HoverInfo::Text);
        if let Some((x, y)) = &axes {
            t = t.x_axis(x.clone()).y_axis(y.clone());
        }
        t
    };
    (trace, None, false)
}

#[inline(never)]
fn panel_trace_lorenz(
    panel: &Panel,
    color: &'static str,
    axes: Option<(String, String)>,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    let PanelKind::Lorenz {
        pop,
        share,
        gini,
        label,
    } = &panel.kind
    else {
        unreachable!("panel_trace dispatches on panel.kind")
    };
    let trace: Box<dyn Trace> = {
        // the cumulative curve; the muted equality diagonal is a SECOND trace added by the
        // caller (grid + inline) right after this one.
        let hover = lorenz_hover_template(label, *gini);
        let mut t = Scatter::new(pop.clone(), share.clone())
            .mode(Mode::Lines)
            .name(escape_hover(&panel.name))
            .line(Line::new().color(color))
            .hover_template(hover);
        if let Some((x, y)) = &axes {
            t = t.x_axis(x.clone()).y_axis(y.clone());
        }
        t
    };
    (trace, None, false)
}

#[inline(never)]
fn panel_trace_funnel_bridge(
    panel: &Panel,
    color: &'static str,
    axes: Option<(String, String)>,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    let bar_max: Option<f64>;
    let PanelKind::Funnel {
        stages,
        labels,
        totals,
        reached,
        n_complete,
        shape,
        form: PipelineForm::Bridge,
    } = &panel.kind
    else {
        unreachable!("panel_trace dispatches on panel.kind")
    };
    let trace: Box<dyn Trace> = {
        // The declared stages do not nest, so the panel bridges the signed differences
        // between consecutive totals instead of asserting containment (see `PipelineForm`).
        //
        // Drawn VERTICALLY, unlike the funnel. A waterfall is an ordinary bar-like trace, so
        // category index 0 lands at the axis BOTTOM — the opposite of a funnel — and the
        // arrays cannot simply be reversed to compensate, because a waterfall accumulates in
        // array order and index 0 must be the `Absolute` that seeds the running total.
        // Vertical sidesteps that entirely, and is the conventional orientation for a bridge.
        //
        // Each step is `stage[k] - stage[k-1]`, labelled as exactly that: an ARITHMETIC
        // difference between two independent totals, never "converted" or "lost", because
        // the whole reason this is not a funnel is that no flow between them is claimed.
        let mut cats: Vec<String> = Vec::with_capacity(stages.len() * 2 - 1);
        let mut vals: Vec<f64> = Vec::with_capacity(stages.len() * 2 - 1);
        let mut measures: Vec<Measure> = Vec::with_capacity(stages.len() * 2 - 1);
        let mut hover: Vec<String> = Vec::with_capacity(stages.len() * 2 - 1);

        // A stage bar's hover, worded per `shape` exactly as the funnel arm words it: the
        // number means something different in each encoding, so calling it "Amount"
        // everywhere would label a ROW COUNT as an amount, and would drop the declared
        // value-column label a row-encoded measure pipeline was given. `RowsCount` folds the
        // value and the count into one line because they are the same number.
        let stage_hover = |k: usize| -> String {
            #[allow(clippy::cast_precision_loss)]
            let pct = if *n_complete == 0 {
                0.0
            } else {
                reached[k] as f64 / *n_complete as f64 * 100.0
            };
            match shape {
                FunnelShape::Columns => format!(
                    "{}<br>Stage: {}<br>Amount: {}<br>Rows reached: {} of {} ({pct:.0}% of \
                     complete cases)",
                    escape_hover(&labels[k]),
                    escape_hover(&stages[k]),
                    fmt_measure(totals[k]),
                    HumanCount(reached[k] as u64),
                    HumanCount(*n_complete as u64),
                ),
                FunnelShape::RowsMeasure { value_label } => format!(
                    "{}<br>Stage: {}<br>{}: {}<br>Rows in stage: {} of {} ({pct:.0}% of rows in \
                     declared stages)",
                    escape_hover(&labels[k]),
                    escape_hover(&stages[k]),
                    escape_hover(value_label),
                    fmt_measure(totals[k]),
                    HumanCount(reached[k] as u64),
                    HumanCount(*n_complete as u64),
                ),
                FunnelShape::RowsCount => format!(
                    "{}<br>Stage: {}<br>Rows: {} of {} ({pct:.0}% of rows in declared stages)",
                    escape_hover(&labels[k]),
                    escape_hover(&stages[k]),
                    HumanCount(reached[k] as u64),
                    HumanCount(*n_complete as u64),
                ),
            }
        };

        cats.push(stages[0].clone());
        vals.push(totals[0]);
        measures.push(Measure::Absolute);
        hover.push(stage_hover(0));

        for k in 1..stages.len() {
            let delta = totals[k] - totals[k - 1];
            cats.push(format!("{} \u{2212} {}", stages[k], stages[k - 1]));
            vals.push(delta);
            measures.push(Measure::Relative);
            hover.push(format!(
                "{} \u{2212} {}<br>Difference: {}<br>{} is {} than {}",
                escape_hover(&stages[k]),
                escape_hover(&stages[k - 1]),
                fmt_measure(delta),
                escape_hover(&stages[k]),
                if delta < 0.0 { "lower" } else { "higher" },
                escape_hover(&stages[k - 1]),
            ));

            // `Total` re-derives its own height from the running total; the real amount goes
            // in the slot anyway so the arrays stay parallel and the JSON self-documenting.
            cats.push(stages[k].clone());
            vals.push(totals[k]);
            measures.push(Measure::Total);
            hover.push(stage_hover(k));
        }

        bar_max = vals
            .iter()
            .zip(&measures)
            .filter(|(_, m)| !matches!(m, Measure::Relative))
            .map(|(v, _)| *v)
            .fold(None, |acc: Option<f64>, v| {
                Some(acc.map_or(v, |a| f64::max(a, v)))
            });

        // Per-bar templates, because no single one is right for every bar: a step bar's
        // number is its DELTA, a stage bar's is the running TOTAL. plotly stays the
        // formatter (so the labels cannot drift from the values it draws), and `.3s` matches
        // the SI precision the KPI row and the axes already use — its own `textinfo` renders
        // seven significant figures.
        let templates: Vec<String> = measures
            .iter()
            .map(|m| {
                if matches!(m, Measure::Relative) {
                    "%{delta:.3s}".to_string()
                } else {
                    "%{final:.3s}".to_string()
                }
            })
            .collect();
        let mut w = Waterfall::new(cats, vals)
            .name(escape_hover(&panel.name))
            .measure(measures)
            .text_template_array(templates)
            .text_position(TextPosition::Outside)
            .increasing(MeasureStyle::new().marker(WaterfallMarker::new().color(BRIDGE_UP_COLOR)))
            .decreasing(MeasureStyle::new().marker(WaterfallMarker::new().color(BRIDGE_DOWN_COLOR)))
            .totals(MeasureStyle::new().marker(WaterfallMarker::new().color(color)))
            .hover_text_array(hover)
            .hover_template("%{hovertext}<extra></extra>");
        if let Some((x, y)) = &axes {
            w = w.x_axis(x.clone()).y_axis(y.clone());
        }
        w
    };
    (trace, bar_max, false)
}

#[inline(never)]
fn panel_trace_funnel(
    panel: &Panel,
    color: &'static str,
    axes: Option<(String, String)>,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    let PanelKind::Funnel {
        stages,
        labels,
        totals,
        reached,
        n_complete,
        shape,
        form: _,
    } = &panel.kind
    else {
        unreachable!("panel_trace dispatches on panel.kind")
    };
    let trace: Box<dyn Trace> = {
        // A horizontal funnel: amount on the value axis, stage on the category axis.
        // A funnel trace draws index 0 at the TOP and works downward — the opposite of a
        // plain category axis, and the reason these arrays are fed UPSTREAM-FIRST, exactly
        // as the panel carries them. Feeding them reversed (the lollipop's convention)
        // renders the funnel upside down, widening toward the bottom.
        //
        // The bar text is left to plotly's own `textinfo`: it computes "percent previous"
        // from the values themselves, so the conversion figures can never drift from the
        // bars. The hover is fully pre-rendered into `hover_text`, which keeps every literal
        // `%` in DATA rather than in the template — so `escape_template_pct` is unnecessary
        // here and only the column label needs `escape_hover`.
        let hover: Vec<String> = (0..stages.len())
            .map(|k| {
                let pct = if *n_complete == 0 {
                    0.0
                } else {
                    #[allow(clippy::cast_precision_loss)]
                    let p = reached[k] as f64 / *n_complete as f64 * 100.0;
                    p
                };
                // The counts mean different things per shape, so the wording must too: for
                // the row encodings `reached` IS the stage's row count and sums to
                // `n_complete`, so "of complete cases" would assert a 100% completeness rate
                // that means nothing.
                match shape {
                    FunnelShape::Columns => format!(
                        "{}<br>Stage: {}<br>Amount: {}<br>Rows reached: {} of {} ({pct:.0}% of \
                         complete cases)",
                        escape_hover(&labels[k]),
                        escape_hover(&stages[k]),
                        fmt_measure(totals[k]),
                        HumanCount(reached[k] as u64),
                        HumanCount(*n_complete as u64),
                    ),
                    FunnelShape::RowsMeasure { value_label } => format!(
                        "{}<br>Stage: {}<br>{}: {}<br>Rows in stage: {} of {} ({pct:.0}% of rows \
                         in declared stages)",
                        escape_hover(&labels[k]),
                        escape_hover(&stages[k]),
                        escape_hover(value_label),
                        fmt_measure(totals[k]),
                        HumanCount(reached[k] as u64),
                        HumanCount(*n_complete as u64),
                    ),
                    FunnelShape::RowsCount => format!(
                        "{}<br>Stage: {}<br>Rows: {} of {} ({pct:.0}% of rows in declared stages)",
                        escape_hover(&labels[k]),
                        escape_hover(&stages[k]),
                        HumanCount(reached[k] as u64),
                        HumanCount(*n_complete as u64),
                    ),
                }
            })
            .collect();
        let xs: Vec<f64> = totals.clone();
        let ys: Vec<String> = stages.clone();
        let mut f = Funnel::new(xs, ys)
            .orientation(Orientation::Horizontal)
            .name(escape_hover(&panel.name))
            .marker(Marker::new().color(color))
            // ONLY the stage-to-stage conversion: plotly computes it from the values, so it
            // can never drift from the bars. The absolute amounts deliberately stay out of
            // the band — "value+percent previous" is two long lines that plotly clips inside
            // a short band — and live in the KPI row above and this panel's hover instead.
            .text_info("percent previous")
            .text_position(TextPosition::Outside)
            .connector(FunnelConnector::new().visible(true))
            .hover_text_array(hover)
            .hover_template("%{hovertext}<extra></extra>");
        if let Some((x, y)) = &axes {
            f = f.x_axis(x.clone()).y_axis(y.clone());
        }
        f
    };
    (trace, None, false)
}

#[inline(never)]
fn panel_trace_contour_pair(
    panel: &Panel,
    axes: Option<(String, String)>,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    let PanelKind::ContourPair {
        x,
        y,
        z,
        x_label,
        y_label,
    } = &panel.kind
    else {
        unreachable!("panel_trace dispatches on panel.kind")
    };
    let trace: Box<dyn Trace> = {
        // standalone (inline) panels show the colorbar; grid cells hide it to avoid clutter
        let hover = contour_hover_template(x_label, y_label);
        let mut c = Contour::new(x.clone(), y.clone(), z.clone())
            .color_scale(ColorScale::Palette(ColorScalePalette::Viridis))
            .show_scale(axes.is_none())
            // Contour::name takes &str (unlike the other traces' impl AsRef<str>)
            .name(&escape_hover(&panel.name))
            .hover_template(&hover);
        if let Some((xa, ya)) = &axes {
            c = c.x_axis(xa.as_str()).y_axis(ya.as_str());
        }
        c
    };
    (trace, None, false)
}

#[inline(never)]
fn panel_trace_corr_heatmap(
    panel: &Panel,
    axes: Option<(String, String)>,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    let PanelKind::CorrHeatmap {
        labels,
        matrix,
        spearman,
    } = &panel.kind
    else {
        unreachable!("panel_trace dispatches on panel.kind")
    };
    let trace: Box<dyn Trace> = corr_heatmap_trace(
        truncate_labels_unique(labels, CORR_LABEL_MAX_CHARS),
        matrix.clone(),
        axes.clone(),
        // standalone (inline) panels show the colorbar; grid panels use in-cell labels
        axes.is_none(),
        *spearman,
    );
    (trace, None, false)
}

#[inline(never)]
fn panel_trace_assoc_heatmap(
    panel: &Panel,
    axes: Option<(String, String)>,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    let PanelKind::AssocHeatmap {
        labels,
        matrix,
        hover_suffix,
    } = &panel.kind
    else {
        unreachable!("panel_trace dispatches on panel.kind")
    };
    let trace: Box<dyn Trace> = assoc_heatmap_trace(
        truncate_labels_unique(labels, CORR_LABEL_MAX_CHARS),
        matrix.clone(),
        hover_suffix.clone(),
        axes.clone(),
        // standalone (inline) panels show the colorbar; grid panels use in-cell labels
        axes.is_none(),
    );
    (trace, None, false)
}

#[inline(never)]
fn panel_trace_top_relationships(
    panel: &Panel,
    axes: Option<(String, String)>,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    let PanelKind::TopRelationships {
        labels,
        values,
        supports,
        nonlinear,
        hover_suffix,
    } = &panel.kind
    else {
        unreachable!("panel_trace dispatches on panel.kind")
    };
    let trace: Box<dyn Trace> = {
        // Horizontal multivariate lollipop: value (NMI) on x, pair on the category y-axis
        // (the zoomed value-axis range and category axis are built by the grid assembler for
        // this panel kind, NOT via `bar_max`/`styled_y_axis`). Three encodings per dot:
        // x = association strength, size = co-occurrence support, color = nonlinear flag.
        // Rank #1 must sit at the TOP, but plotly places category index 0 at the axis BOTTOM,
        // so every parallel array is fed weakest-first (reverse of the descending rank order).
        let (floor, _ceil) = lollipop_value_range(values);
        let xs: Vec<f64> = values.iter().rev().copied().collect();
        // FULL (untruncated) pair labels are the category y-values so each pair gets its own
        // row — two distinct pairs that share a long prefix must NOT collapse onto one line.
        // The category axis (built by the assembler) truncates only the DISPLAYED tick text.
        let ys: Vec<String> = labels.iter().rev().cloned().collect();
        let sizes: Vec<f64> = supports.iter().rev().copied().collect();
        // stems: each extends leftward from its dot down to the (zoomed) axis floor, drawn as
        // an asymmetric x error bar with no rightward arm and no end caps (`width(0)`).
        let stem_minus: Vec<f64> = xs.iter().map(|v| (v - floor).max(0.0)).collect();
        let colors: Vec<&'static str> = nonlinear
            .iter()
            .rev()
            .map(|&nl| {
                if nl {
                    NONLINEAR_MARKER_COLOR
                } else {
                    REGULAR_MARKER_COLOR
                }
            })
            .collect();
        // per-dot hover carries the FULL (untruncated) pair label plus n= and any nonlinearity
        // note; the y-axis tick shows only the truncated label. escape: this is a plotly
        // hoverTEMPLATE, so a raw header containing `<extra>` or `<b>` would otherwise
        // terminate/format the template instead of displaying literally (matching the
        // escaped Histogram/ScatterPair hovers).
        let templates: Vec<String> = labels
            .iter()
            .rev()
            .zip(hover_suffix.iter().rev())
            .map(|(full, suffix)| {
                format!(
                    "{}<br>NMI = %{{x:.3f}}{suffix}<extra></extra>",
                    escape_hover(full)
                )
            })
            .collect();
        let marker = Marker::new()
            .size_array(scale_bubble_sizes(&sizes))
            .color_array(colors);
        let mut sc = Scatter::new(xs.clone(), ys)
            .name(escape_hover(&panel.name))
            .mode(Mode::Markers)
            .marker(marker)
            .error_x(
                ErrorData::new(ErrorType::Data)
                    .symmetric(false)
                    .array(vec![0.0_f64; xs.len()])
                    .array_minus(stem_minus)
                    .width(0)
                    .thickness(1.5)
                    .color(MUTED_COLOR),
            )
            .hover_template_array(templates);
        if let Some((x, y)) = &axes {
            sc = sc.x_axis(x.clone()).y_axis(y.clone());
        }
        sc
    };
    (trace, None, false)
}

#[inline(never)]
fn panel_trace_measure_by_dim(
    panel: &Panel,
    color: &'static str,
    axes: Option<(String, String)>,
    theme: Option<BuiltinTheme>,
    log_scale: LogScale,
) -> (Box<dyn Trace>, Option<f64>, bool) {
    let mut bar_max: Option<f64> = None;
    let log_y;
    // bar value-label font: omit the explicit color so the label inherits the layout font color
    // (qsv's ink in the unthemed look, the template's font when themed) -- this lets the dark/light
    // toggle's `font.color` relayout flip the labels instead of leaving them dark on a dark page.
    let label_font = {
        let f = Font::new().size(9);
        if theme.is_some() {
            f
        } else {
            f.family(FONT_FAMILY)
        }
    };
    let PanelKind::MeasureByDim { labels, values } = &panel.kind else {
        unreachable!("panel_trace dispatches on panel.kind")
    };
    let trace: Box<dyn Trace> = {
        // a high-dynamic-range measure (one big group over much smaller ones) squashes on a
        // linear axis; a log value axis spreads the bars out — but only when every bar is
        // strictly positive (see `measure_by_dim_logs`).
        log_y = measure_by_dim_logs(log_scale, values);
        // headroom only frames the LINEAR range (`0..=max*1.15`); a log value axis autoranges
        // like a box panel (it can't assume counts >= 1, so `bar_max` stays None → the "log
        // scale" value cue, not "count (log)").
        if !log_y {
            bar_max = Some(values.iter().copied().fold(0.0_f64, f64::max));
        }
        // full category names in the hover — the axis ticks are truncated, and plotly's
        // default hover would otherwise echo that truncated tick text
        let hover_labels: Vec<String> = labels.iter().map(|l| escape_hover(l)).collect();
        let mut bar = Bar::new(labels.clone(), values.clone())
            .name(escape_hover(&panel.name))
            .marker(Marker::new().color(color))
            .hover_text_array(hover_labels)
            .hover_template("%{hovertext}<br>%{y:,}<extra></extra>")
            // value labels above each bar, SI-formatted ("258k", "1.05M") to match the ticks
            .text_template("%{y:.3s}")
            .text_position(TextPosition::Outside)
            .clip_on_axis(false)
            .text_font(label_font);
        if let Some((x, y)) = &axes {
            bar = bar.x_axis(x.clone()).y_axis(y.clone());
        }
        bar
    };
    (trace, bar_max, log_y)
}

/// The reusable building blocks of a smart-Data Schematic subplot grid, shared by the
/// typed-`Layout` renderer (`render_smart_grid`, ≤ `MAX_SUBPLOTS` panels) and the raw-JSON renderer
/// (`render_smart_grid_json`, used for static image export of more panels). Each axis pair in
/// `axes` already has its paper-domain and cross-anchor applied; `base_layout` carries the
/// page chrome (size/margins/fonts/background) but neither the per-subplot axes nor the
/// annotations, so each assembler can stamp those on in its own way.
struct SmartGridParts {
    traces:      Vec<Box<dyn Trace>>,
    axes:        Vec<(usize, Axis, Axis)>,
    // geo panels don't use cartesian x/y axes: each carries a domain-positioned `geo{pos}` layout
    // object (as raw JSON, since the typed `Layout` has only one `.geo()`), injected by
    // `render_smart_grid_json`. Empty for the typed-`Layout` grid (geo dashboards never use it).
    geos:        Vec<(usize, serde_json::Value)>,
    annotations: Vec<Annotation>,
    base_layout: Layout,
    theme:       Option<BuiltinTheme>,
    dims:        (usize, usize),
}

/// Build the shared pieces of a smart-Data Schematic subplot grid: one trace per panel (each
/// referencing its own `x{n}`/`y{n}` axes), the domain-positioned + cross-anchored axis pairs,
/// the Data Schematic + per-panel title annotations, and the base page layout. We lay out each
/// cell's axes with explicit paper-domains (rather than a plotly `grid`) so we can (a) scale the
/// plot height with the row count, (b) reserve a band at the top for the Data Schematic title, and
/// (c) place each column's name as a title *above* its panel.
fn smart_grid_parts(
    args: &Args,
    panels: &[Panel],
    freq: &FreqMap,
    hist: &HashMap<usize, Vec<f64>>,
    outliers: &HashMap<usize, OutlierStats>,
    title_text: &str,
    log_scale: LogScale,
) -> SmartGridParts {
    let cols = args.flag_grid_cols.clamp(1, panels.len());
    // each leading overview panel takes a full-width row; the rest pack into `cols`-wide rows.
    // `fig_height` accounts for a shorter leading KPI row (else it is `rows * ROW_HEIGHT_PX`).
    let (geoms, _rows, fig_height) = smart_grid_layout(panels, cols);

    // widen the left margin when a correlation panel is present so its (long) numeric-column
    // tick labels — truncated to CORR_LABEL_MAX_CHARS — aren't clipped against the page edge.
    let left_margin = panels
        .iter()
        .filter_map(|p| match &p.kind {
            PanelKind::CorrHeatmap { labels, .. } | PanelKind::AssocHeatmap { labels, .. } => {
                Some(labels.as_slice())
            },
            PanelKind::KpiRow { .. }
            | PanelKind::BoxStats { .. }
            | PanelKind::BoxRaw { .. }
            | PanelKind::BoxOutliers { .. }
            | PanelKind::Violin { .. }
            | PanelKind::GroupedViolin { .. }
            | PanelKind::FreqBar { .. }
            | PanelKind::TimeSeries { .. }
            | PanelKind::ScatterPair { .. }
            | PanelKind::Lorenz { .. }
            | PanelKind::Funnel { .. }
            | PanelKind::ContourPair { .. }
            | PanelKind::Scatter3D { .. }
            | PanelKind::MeasureByDim { .. }
            | PanelKind::TopRelationships { .. }
            | PanelKind::CyclicProfile { .. }
            | PanelKind::Histogram { .. }
            | PanelKind::AnimatedScatterPair { .. }
            | PanelKind::AnimatedGeo { .. }
            | PanelKind::AnimatedBubble { .. }
            | PanelKind::Map { .. }
            | PanelKind::Geo { .. }
            | PanelKind::Choropleth { .. }
            | PanelKind::ChoroplethMap { .. }
            | PanelKind::Hierarchy { .. }
            | PanelKind::Sankey { .. }
            | PanelKind::Parcats { .. } => None,
        })
        .flat_map(<[String]>::iter)
        .map(|l| l.chars().count().min(CORR_LABEL_MAX_CHARS))
        .max()
        .map_or(DEFAULT_LEFT_MARGIN_PX, |longest| {
            (longest * CORR_LABEL_PX_PER_CHAR + 24).max(DEFAULT_LEFT_MARGIN_PX)
        });
    // the horizontal Top Relationships lollipop puts its (truncated) pair labels on the y-axis, at
    // the shared page-left edge (it's a full-width overview panel); widen the margin to fit them.
    let toprel_margin = panels
        .iter()
        .filter_map(|p| match &p.kind {
            PanelKind::TopRelationships { labels, .. } => labels
                .iter()
                .map(|l| l.chars().count().min(TOPREL_LABEL_MAX_CHARS))
                .max(),
            // the funnel is the other horizontal panel: its category ticks are the short
            // canonical stage names, which still need left room reserved
            PanelKind::Funnel {
                stages,
                form: PipelineForm::Funnel,
                ..
            } => stages
                .iter()
                .map(|l| l.chars().count().min(TOPREL_LABEL_MAX_CHARS))
                .max(),
            _ => None,
        })
        .max()
        .map_or(0, |longest| longest * CORR_LABEL_PX_PER_CHAR + 24);
    let left_margin = left_margin.max(toprel_margin);
    // log freq panels carry a rotated y-axis title cue; reserve extra left room (shared across
    // the single-Plot grid) so it isn't clipped against the page edge.
    let left_margin = if panels.iter().any(|p| panel_is_log(p, freq, log_scale)) {
        left_margin + LOG_AXIS_TITLE_MARGIN_PX
    } else {
        left_margin
    };

    // when a theme is set, let its plotly template drive backgrounds/fonts/axis chrome;
    // otherwise apply qsv's built-in look. `themed` gates the explicit overrides below.
    let theme = args.theme();
    let themed = theme.is_some();
    // a Data Schematic text font: set the family but never an explicit color, so the text inherits
    // the layout font color (qsv's ink unthemed, the template's font when themed). Inheriting
    // -- rather than baking in ink -- lets the dark/light toggle's `font.color` relayout flip
    // these labels.
    let ann_font = |size: usize| {
        let f = Font::new().size(size);
        if themed { f } else { f.family(FONT_FAMILY) }
    };

    let mut base_layout = Layout::new()
        .show_legend(false)
        .height(fig_height)
        // vertical modebar, clear of the Data Schematic title and the top-right cell's panel title
        // (matching the inline panels — see `inline_panel_title`)
        .mode_bar(ModeBar::new().orientation(Orientation::Vertical))
        .margin(
            Margin::new()
                .top(TOP_MARGIN_PX)
                .bottom(BOTTOM_MARGIN_PX)
                .left(left_margin)
                .right(40)
                .pad(4),
        );
    if !themed {
        base_layout = base_layout
            .font(Font::new().family(FONT_FAMILY).color(INK).size(12))
            .paper_background_color(PAPER_BG)
            .plot_background_color(PAPER_BG);
    }

    // annotations: the Data Schematic title (in the reserved top strip) plus one title per panel
    let mut annotations: Vec<Annotation> = Vec::with_capacity(panels.len() + 1);
    annotations.push(
        Annotation::new()
            .text(title_text)
            .x(0.5)
            .y(1.0)
            .x_ref("paper")
            .y_ref("paper")
            .x_anchor(Anchor::Center)
            .y_anchor(Anchor::Bottom)
            .show_arrow(false)
            .font(ann_font(20)),
    );

    let mut traces: Vec<Box<dyn Trace>> = Vec::with_capacity(panels.len());
    let mut axes: Vec<(usize, Axis, Axis)> = Vec::with_capacity(panels.len());
    let mut geos: Vec<(usize, serde_json::Value)> = Vec::new();

    // The leading KPI row (index 0, when present) is domain-positioned and takes NO cartesian axis
    // slot, so cartesian panels must number their axes from 1 as if it weren't there — otherwise 8
    // real charts behind it would run to x9/y9, which the typed `Layout` (x1..x8) silently drops
    // (`assign_typed_axis`). The KPI row and geo panels never coexist here (geo forces the inline
    // path for HTML; image exports carry no KPI row), so a single leading offset is sufficient.
    let axis_offset = usize::from(matches!(
        panels.first().map(|p| &p.kind),
        Some(PanelKind::KpiRow { .. })
    ));

    for (n, panel) in panels.iter().enumerate() {
        let pos = (n + 1).saturating_sub(axis_offset);
        let xref = axis_ref('x', pos);
        let yref = axis_ref('y', pos);
        let color = PALETTE[n % PALETTE.len()];

        // KPI row: a full-width band of domain-positioned Indicator tiles (no cartesian axes).
        // Sub-divide this panel's paper x-domain into N tiles, emit each as an Indicator, then skip
        // the cartesian axis assignment below. No section title — each tile carries its own
        // word-wrapped subtitle label below the number (`kpi_label_annotation`).
        if let PanelKind::KpiRow { tiles } = &panel.kind {
            let geom = &geoms[n];
            let x0 = geom.x_domain.first().copied().unwrap_or(0.0);
            let x1 = geom.x_domain.get(1).copied().unwrap_or(1.0);
            let y0 = geom.y_domain.first().copied().unwrap_or(0.0);
            let y1 = geom.y_domain.get(1).copied().unwrap_or(1.0);
            let count = tiles.len().max(1) as f64;
            let width = (x1 - x0) / count;
            let gutter = width * 0.06;
            // Indicator-domain bottom and the shared label baseline, as fractions of this KPI
            // cell's band (mapped into `[y0, y1]`). Same tuning as the inline path
            // (`smart_inline_panel_plot`): number tiles center the number in the domain; gauge
            // tiles render it lower (at the dial pivot), so a gauge row pins the one shared label
            // baseline just under the gauge number. The cell band is already height-matched to the
            // tiles (`panel_row_scale`), so the label sits snug with no dead band below.
            let has_gauge = tiles.iter().any(|t| t.gauge.is_some());
            let (ind_frac, label_frac) = if has_gauge {
                (0.34_f64, 0.30_f64)
            } else {
                (0.10_f64, 0.24_f64)
            };
            let band = y1 - y0;
            let ind_y0 = y0 + band * ind_frac;
            let label_y = y0 + band * label_frac;
            // narrower tiles (more of them) wrap sooner; the label font fits ~2 lines in the band.
            let max_chars = ((150.0 / count) as usize).clamp(12, 28);
            for (i, tile) in tiles.iter().enumerate() {
                let lo = x0 + i as f64 * width + gutter / 2.0;
                let hi = x0 + (i as f64 + 1.0) * width - gutter / 2.0;
                traces.push(kpi_indicator(tile, [lo, hi], [ind_y0, y1]));
                annotations.push(kpi_label_annotation(
                    &tile.label,
                    f64::midpoint(lo, hi),
                    label_y,
                    max_chars,
                    ann_font(13),
                ));
            }
            continue;
        }

        // geo panels use a `geo{pos}` projection subplot (no cartesian x/y axes), only reachable on
        // the raw-JSON static-export path. Build the ScatterGeo trace, fit the projection to the
        // data extent, and emit the domain-positioned geo layout object as raw JSON (the typed
        // `Layout` has only a single `.geo()` with no per-cell domain).
        if let PanelKind::Geo {
            lats,
            lons,
            outlier_lats,
            outlier_lons,
            hover_text,
            outlier_hover_text,
            sizes,
            // static-export path only: no drawer, so no cross-link ids to emit
            ..
        } = &panel.kind
        {
            let geom = geoms[n].clone();
            // a static image can't be panned/zoomed, so when there are outliers the projection AND
            // framing must cover the FULL plotted extent (core + outliers), not the core alone —
            // otherwise a stray outside the US could leave the projection on `albers-usa` (a fixed
            // US composite, no fitted axes) and clip it. Derive the frame from the actual plotted
            // coords so this holds even without geocode metadata (the outlier markers render either
            // way). Pass the bounding-box CORNERS (not the raw points) so geo_framing's 2.5% trim
            // can't drop the handful of outliers. Without outliers, frame from the core points.
            let (frame_lats, frame_lons) = if outlier_lats.is_empty() {
                (lats.clone(), lons.clone())
            } else {
                let lat_iter = || lats.iter().chain(outlier_lats.iter()).copied();
                let lon_iter = || lons.iter().chain(outlier_lons.iter()).copied();
                let min_lat = lat_iter().fold(f64::INFINITY, f64::min);
                let max_lat = lat_iter().fold(f64::NEG_INFINITY, f64::max);
                let min_lon = lon_iter().fold(f64::INFINITY, f64::min);
                let max_lon = lon_iter().fold(f64::NEG_INFINITY, f64::max);
                (
                    vec![max_lat, max_lat, min_lat, min_lat, max_lat],
                    vec![min_lon, max_lon, max_lon, min_lon, min_lon],
                )
            };
            let (projection, lonaxis, lataxis) =
                geo_framing(&frame_lats, &frame_lons, MAP_FRAME_TRIM_FRAC);
            // for a fitted (Mercator) extent, refine the axis ranges with the tighter extent-fit
            // padding — but only from a concrete extent that already covers everything plotted: the
            // full extent if we have it, else the core extent ONLY when there are no outliers. With
            // outliers but no full extent (e.g. it was antimeridian-filtered), keep the
            // plotted-coords frame so the outlier markers aren't cropped back to the core. Whole-
            // region projections (albers-usa / natural-earth) return no axes and need no refining.
            #[cfg(feature = "geocode")]
            let (lonaxis, lataxis) = match &panel.geo_meta {
                Some(meta) if lonaxis.is_some() || lataxis.is_some() => {
                    match meta
                        .full_extent
                        .as_ref()
                        .or_else(|| outlier_lats.is_empty().then_some(&meta.extent))
                    {
                        Some(ext) => {
                            let (lon, lat) = extent_geo_axes(ext);
                            (Some(lon), Some(lat))
                        },
                        None => (lonaxis, lataxis),
                    }
                },
                _ => (lonaxis, lataxis),
            };
            let (geo_land, geo_water, geo_bg) = geo_palette(theme);
            // when the world-overview framing applies but every point sits in a single plotly
            // continent, frame the map to that continent's `scope` instead of the whole world.
            let continent = matches!(projection, ProjectionType::NaturalEarth)
                .then(|| continent_scope(&frame_lats, &frame_lons))
                .flatten();
            let mut geo = LayoutGeo::new()
                .projection(Projection::new().projection_type(projection))
                .resolution(GeoResolution::OneOverFiftyMillion)
                .showland(true)
                .landcolor(geo_land)
                .showocean(true)
                .oceancolor(geo_water)
                .showlakes(true)
                .lakecolor(geo_water)
                .showcountries(true)
                .bgcolor(geo_bg);
            if let Some(scope) = continent {
                geo = geo.scope(scope);
            }
            if let Some(lonaxis) = lonaxis {
                geo = geo.lonaxis(lonaxis);
            }
            if let Some(lataxis) = lataxis {
                geo = geo.lataxis(lataxis);
            }
            let mut core_marker = Marker::new().color(color).opacity(MAP_POINT_OPACITY);
            if let Some(sizes) = sizes {
                core_marker = core_marker.size_array(scale_bubble_sizes(sizes));
            }
            let mut core_trace = ScatterGeo::new(lats.clone(), lons.clone())
                .mode(Mode::Markers)
                .marker(core_marker)
                .subplot(geo_ref(pos));
            if !hover_text.is_empty() {
                core_trace = core_trace
                    .text_array(map_hover_text_values(hover_text))
                    .hover_template(MAP_HOVER_TEMPLATE);
            }
            traces.push(core_trace);
            // geographic outliers as a distinct amber/X marker trace on top of the core points
            if !outlier_lats.is_empty() {
                let mut out_trace = ScatterGeo::new(outlier_lats.clone(), outlier_lons.clone())
                    .name(&t!("viz.map.geographic_outliers"))
                    .mode(Mode::Markers)
                    .marker(outlier_marker_geo())
                    .show_legend(false)
                    .subplot(geo_ref(pos));
                if !outlier_hover_text.is_empty() {
                    out_trace = out_trace
                        .text_array(map_hover_text_values(outlier_hover_text))
                        .hover_template(MAP_HOVER_TEMPLATE);
                }
                traces.push(out_trace);
            }
            // spatial-extent overlay (bounding box + reverse-geocoded corner/center markers),
            // both bound to this cell's `geo{pos}` subplot, plus a consolidated summary
            // annotation below the cell. Hover labels are inert in static images.
            #[cfg(feature = "geocode")]
            if let Some(meta) = &panel.geo_meta {
                // full-extent box (core + outliers), no fill, drawn first/underneath
                if let Some(full) = &meta.full_extent {
                    let (flat, flon) = extent_box_latlon(full);
                    traces.push(
                        ScatterGeo::new(flat, flon)
                            .name(&t!("viz.map.full_extent"))
                            .mode(Mode::Lines)
                            .line(full_extent_box_line())
                            .hover_info(HoverInfo::Skip)
                            .show_legend(false)
                            .subplot(geo_ref(pos)),
                    );
                }
                let (blat, blon) = extent_box_latlon(&meta.extent);
                traces.push(
                    ScatterGeo::new(blat, blon)
                        .name(&t!("viz.map.spatial_extent"))
                        .mode(Mode::Lines)
                        .line(extent_box_line())
                        .fill(plotly::traces::scatter_geo::Fill::ToSelf)
                        .fill_color(GEO_EXTENT_FILL_COLOR)
                        .hover_info(HoverInfo::Skip)
                        .show_legend(false)
                        .subplot(geo_ref(pos)),
                );
                let mlat: Vec<f64> = meta.points.iter().map(|p| p.lat).collect();
                let mlon: Vec<f64> = meta.points.iter().map(|p| p.lon).collect();
                let htext: Vec<String> = meta.points.iter().map(point_hover_text).collect();
                traces.push(
                    ScatterGeo::new(mlat, mlon)
                        .name(&t!("viz.map.extent_points"))
                        .mode(Mode::Markers)
                        .marker(extent_marker_geo())
                        .hover_text_array(htext)
                        .hover_info(HoverInfo::Text)
                        .show_legend(false)
                        .subplot(geo_ref(pos)),
                );
                if !meta.summary.is_empty() {
                    annotations.push(
                        Annotation::new()
                            .text(&t!("viz.map.geo_extent_caption", q_summary = meta.summary))
                            .x(geom.title_x)
                            .y((geom.y_domain[0] - GEO_META_OFFSET).max(0.0))
                            .x_ref("paper")
                            .y_ref("paper")
                            .x_anchor(Anchor::Center)
                            .y_anchor(Anchor::Top)
                            .show_arrow(false)
                            .font(ann_font(11)),
                    );
                }
            }
            if let Some(overlay) = &panel.geojson_overlay {
                traces.extend(geojson_overlay_geo_traces(overlay, Some(&geo_ref(pos))));
            }
            let mut geo_json = serde_json::to_value(&geo).unwrap_or(serde_json::Value::Null);
            if let Some(obj) = geo_json.as_object_mut() {
                obj.insert(
                    "domain".to_string(),
                    serde_json::json!({ "x": geom.x_domain, "y": geom.y_domain }),
                );
            }
            geos.push((pos, geo_json));
            annotations.push(panel_title_annotation(
                panel,
                geom.title_x,
                geom.title_y,
                ann_font(13),
            ));
            continue;
        }

        let is_box = matches!(
            panel.kind,
            PanelKind::BoxStats { .. }
                | PanelKind::BoxRaw { .. }
                | PanelKind::BoxOutliers { .. }
                | PanelKind::Violin { .. }
        );
        // A Quarter/Year-bucketed trend carries non-date-parseable labels ("2024-Q1"): render it
        // on a category axis via the tick_text mechanism instead of the date axis (issue #4216).
        let (is_date, ts_ticks) = match &panel.kind {
            PanelKind::TimeSeries {
                categorical, xs, ..
            } => (!*categorical, categorical.then(|| xs.clone())),
            _ => (false, None),
        };
        let (trace, bar_max, log_y) = panel_trace(
            panel,
            color,
            freq,
            hist,
            outliers,
            Some((xref.clone(), yref.clone())),
            theme,
            log_scale,
        );
        traces.push(trace);
        // a Lorenz panel needs its equality-diagonal reference line as a second trace, bound to
        // this cell's axes.
        if matches!(panel.kind, PanelKind::Lorenz { .. }) {
            traces.push(lorenz_diagonal_trace(Some((xref.clone(), yref.clone()))));
        }

        // build this subplot's styled, domain-positioned, cross-anchored axes and add its title
        // above the cell. x-axis anchors to its paired y-axis and vice versa.
        let geom = geoms[n].clone();
        // the Top Relationships lollipop is the one horizontal panel: value (NMI) on a zoomed x,
        // pair on the category y — the opposite axis roles from every other (vertical) panel.
        let (x_axis, y_axis) = if let PanelKind::Funnel {
            stages,
            totals,
            form: PipelineForm::Funnel,
            ..
        } = &panel.kind
        {
            // the funnel is horizontal like the lollipop, but its value axis is anchored at
            // zero and its categories run TOP-DOWN, matching the upstream-first arrays the
            // trace feeds (see the `panel_trace` arm).
            let ticks: Vec<String> = stages.clone();
            let max = totals.iter().copied().fold(0.0_f64, f64::max);
            (
                funnel_value_axis(max, theme)
                    .domain(&geom.x_domain)
                    .anchor(yref.clone()),
                lollipop_category_axis(theme, &ticks)
                    .domain(&geom.y_domain)
                    .anchor(xref.clone()),
            )
        } else if let PanelKind::Funnel { totals, .. } = &panel.kind {
            // a bridge is VERTICAL (see the `panel_trace` arm): categories on x, amounts on y,
            // with the ordinary bar headroom so the outside delta labels are not clipped.
            let max = totals.iter().copied().fold(0.0_f64, f64::max);
            (
                styled_x_axis(false, false, false, theme, None)
                    .domain(&geom.x_domain)
                    .anchor(yref.clone()),
                styled_y_axis(Some(max), false, theme)
                    .domain(&geom.y_domain)
                    .anchor(xref.clone()),
            )
        } else if let PanelKind::TopRelationships { values, labels, .. } = &panel.kind {
            let (floor, ceil) = lollipop_value_range(values);
            // bottom-to-top (weakest-first), matching the reversed y-values the trace feeds
            let ticks: Vec<String> = labels
                .iter()
                .rev()
                .map(|l| truncate_label(l, TOPREL_LABEL_MAX_CHARS))
                .collect();
            (
                lollipop_value_axis(floor, ceil, theme)
                    .domain(&geom.x_domain)
                    .anchor(yref.clone()),
                lollipop_category_axis(theme, &ticks)
                    .domain(&geom.y_domain)
                    .anchor(xref.clone()),
            )
        } else {
            // a relationship panel's per-axis log verdict (issue #4223). The y side sets the
            // axis TYPE only — `styled_y_axis`'s "log scale" title is the cue for an unlabeled
            // distribution axis, while these panels name the logged axis in the panel title.
            let (x_log, y_log) = panel.axis_log;
            let mut y = styled_y_axis(bar_max, log_y, theme);
            if y_log {
                y = y.type_(AxisType::Log);
            }
            (
                styled_x_axis(
                    is_box,
                    is_date,
                    x_log,
                    theme,
                    ts_ticks.or_else(|| freq_bar_tick_text(panel, freq)),
                )
                .domain(&geom.x_domain)
                .anchor(yref.clone()),
                y.domain(&geom.y_domain).anchor(xref.clone()),
            )
        };
        axes.push((pos, x_axis, y_axis));
        annotations.push(panel_title_annotation(
            panel,
            geom.title_x,
            geom.title_y,
            ann_font(13),
        ));

        // in-cell `r` value labels for the correlation panel, drawn only when the matrix is
        // small enough to stay legible in one Data Schematic cell. Category axes index annotations
        // by serial number (0-based), and the text flips to white on the dark high-|r| cells
        // for contrast against the RdBu scale.
        if let PanelKind::CorrHeatmap { matrix, .. } | PanelKind::AssocHeatmap { matrix, .. } =
            &panel.kind
            && matrix.len() <= CORR_INCELL_MAX_N
        {
            for ann in corr_incell_annotations(matrix, &xref, &yref) {
                annotations.push(ann);
            }
        }
    }

    SmartGridParts {
        traces,
        axes,
        geos,
        annotations,
        base_layout,
        theme,
        dims: (cols * SMART_COL_WIDTH_PX, fig_height),
    }
}

fn render_smart_grid(
    args: &Args,
    panels: &[Panel],
    freq: &FreqMap,
    hist: &HashMap<usize, Vec<f64>>,
    outliers: &HashMap<usize, OutlierStats>,
    title_text: &str,
    log_scale: LogScale,
    dict_page: Option<String>,
    metadata: Option<String>,
    data_chrome: Option<String>,
) -> CliResult<SmartRender> {
    let SmartGridParts {
        traces,
        axes,
        // geo dashboards are routed to `render_smart_grid_json` (a geo subplot can't live in the
        // typed `Layout`), so `geos` is always empty here.
        geos: _,
        annotations,
        mut base_layout,
        theme,
        dims,
    } = smart_grid_parts(args, panels, freq, hist, outliers, title_text, log_scale);

    let mut plot = Plot::new();
    for trace in traces {
        plot.add_trace(trace);
    }
    for (pos, x_axis, y_axis) in axes {
        assign_typed_axis(&mut base_layout, pos, x_axis, y_axis);
    }
    base_layout = base_layout.annotations(annotations);

    plot.set_layout(apply_theme(base_layout, theme));
    Ok(SmartRender::Grid {
        plot: Box::new(plot),
        dims,
        title: title_text.to_string(),
        theme,
        dict_page,
        metadata,
        data_chrome,
    })
}

fn render_smart_grid_page(
    mut plot: Plot,
    theme: Option<BuiltinTheme>,
    title_text: &str,
    dict_page: Option<&str>,
    meta_table: Option<&str>,
    data_chrome: Option<&str>,
) -> String {
    // match the responsiveness the single-`Plot` HTML path applies in `run`.
    plot.set_configuration(Configuration::new().responsive(true));
    // raw strings (actual newlines, real quotes) so rustfmt's `format_strings` can't split a `\n`
    // or `\"` escape across a line wrap and corrupt the markup (see `smart_html_page`).
    let extra_style = r"  .qsv-viz-grid { width: 100%; }
  .qsv-viz-plot { width: 100%; }";
    let inner = plot.to_inline_html(Some("qsv-viz-smart-grid"));
    let body = format!(
        r#"<div class="qsv-viz-grid">
      <div class="qsv-viz-plot">
{inner}
      </div>
</div>"#
    );
    // the typed plot already carries the Data Schematic title in its layout, so suppress the page
    // <h1>.
    smart_html_page(
        title_text,
        theme,
        extra_style,
        &body,
        false,
        dict_page,
        meta_table,
        // never: `--photos` rides on the map panel's customdata, and a `PanelKind::Map` forces
        // the inline render path (see `has_noncartesian`), so it can't reach this typed grid.
        false,
        data_chrome,
        // ...and for the same reason no MapLibre basemap tile is ever fetched from this page.
        false,
        // ...nor is there a map panel to cross-link the data viewer's rows to.
        None,
        // ...and no choropleth either (they force the inline path), so no region-click filter.
        None,
    )
}

/// Render the Data Schematic as a raw Plotly JSON value with domain-positioned axes, for static
/// image export of > `MAX_SUBPLOTS` panels. plotly's typed `Layout` only has `x_axis1..x_axis8`,
/// but plotly.js itself supports any number of axes — so we serialize the plot, then inject each
/// cell's `xaxis{n}`/`yaxis{n}` (the traces already reference them by name). The result is fed to
/// `StaticExporter::write_fig`, which renders an arbitrary JSON value through the same
/// headless-browser backend as `Plot::write_image`.
fn render_smart_grid_json(
    args: &Args,
    panels: &[Panel],
    freq: &FreqMap,
    hist: &HashMap<usize, Vec<f64>>,
    outliers: &HashMap<usize, OutlierStats>,
    title_text: &str,
    log_scale: LogScale,
) -> CliResult<SmartRender> {
    let SmartGridParts {
        traces,
        axes,
        geos,
        annotations,
        base_layout,
        theme,
        dims,
    } = smart_grid_parts(args, panels, freq, hist, outliers, title_text, log_scale);

    let mut plot = Plot::new();
    for trace in traces {
        plot.add_trace(trace);
    }
    plot.set_layout(apply_theme(base_layout.annotations(annotations), theme));

    let mut value: serde_json::Value = serde_json::from_str(&plot.to_json()).map_err(|e| {
        crate::CliError::Other(format!(
            "viz smart: could not assemble Data Schematic JSON: {e}"
        ))
    })?;
    let Some(layout_obj) = value
        .get_mut("layout")
        .and_then(serde_json::Value::as_object_mut)
    else {
        return fail_clierror!("viz smart: assembled Data Schematic JSON has no layout object");
    };
    for (pos, x_axis, y_axis) in &axes {
        layout_obj.insert(
            axis_json_key('x', *pos),
            serde_json::to_value(x_axis).map_err(|e| {
                crate::CliError::Other(format!("viz smart: could not serialize x-axis {pos}: {e}"))
            })?,
        );
        layout_obj.insert(
            axis_json_key('y', *pos),
            serde_json::to_value(y_axis).map_err(|e| {
                crate::CliError::Other(format!("viz smart: could not serialize y-axis {pos}: {e}"))
            })?,
        );
    }
    // geo panels: inject each domain-positioned `geo{pos}` layout object (its trace already carries
    // a matching `subplot` reference). Pre-serialized as JSON because the typed `Layout` can't hold
    // more than one geo subplot nor a per-cell domain.
    for (pos, geo) in geos {
        layout_obj.insert(geo_ref(pos), geo);
    }

    Ok(SmartRender::GridJson {
        value: Box::new(value),
        dims,
    })
}

/// In-cell `r` value annotations for a correlation matrix, referenced to the given axes (use
/// "x"/"y" for a standalone plot). Undefined correlations (heatmap gaps) get no label; the text
/// flips to white on dark high-|r| cells for contrast against the `RdBu` scale.
fn corr_incell_annotations(matrix: &[Vec<f64>], xref: &str, yref: &str) -> Vec<Annotation> {
    let mut out = Vec::new();
    for (i, row_vals) in matrix.iter().enumerate() {
        for (j, &r) in row_vals.iter().enumerate() {
            if !r.is_finite() {
                continue;
            }
            let text_color = if r.abs() >= 0.5 { "#FFFFFF" } else { INK };
            out.push(
                Annotation::new()
                    .text(format!("{r:.2}"))
                    .x(j as f64)
                    .y(i as f64)
                    .x_ref(xref)
                    .y_ref(yref)
                    .x_anchor(Anchor::Center)
                    .y_anchor(Anchor::Middle)
                    .show_arrow(false)
                    .font(Font::new().family(FONT_FAMILY).color(text_color).size(9)),
            );
        }
    }
    out
}

/// Apply the panel title to an inline-path layout. Without `--dict-info` this is a plain layout
/// title. With a dictionary description attached, the title instead becomes a
/// `panel_title_annotation` (exactly like the typed grid): the title line grows the muted ⓘ,
/// hovering shows the description in a styled plotly hover box, and the `qsvdict-`-named
/// `capture_events` click routes to the Data Dictionary drawer via the `plotly_clickannotation`
/// hook. `extra` carries any annotations the panel already needs (corr in-cell labels), since
/// `Layout::annotations` replaces rather than appends.
fn inline_panel_title(
    layout: Layout,
    panel: &Panel,
    themed: bool,
    extra: Vec<Annotation>,
) -> Layout {
    inline_panel_title_shifted(layout, panel, themed, extra, DEFAULT_TITLE_Y_SHIFT)
}

/// `inline_panel_title` with an explicit title y-shift (px above the plot area), for panels whose
/// subplot paints outside its plot area and would otherwise collide with the default title band.
fn inline_panel_title_shifted(
    layout: Layout,
    panel: &Panel,
    themed: bool,
    extra: Vec<Annotation>,
    y_shift: f64,
) -> Layout {
    // The hover-revealed modebar defaults to a wide horizontal strip across the very band the
    // centered title (and its `--dict-info` ⓘ) occupies, so hovering a panel buried them; a
    // vertical strip down the right edge stays clear of the title band. Applied to every
    // inline panel (not just dict-info ones) so the hover chrome is uniform.
    let layout = layout.mode_bar(ModeBar::new().orientation(Orientation::Vertical));
    let mut anns = extra;
    let layout = if panel.dict_info.is_some() {
        // family-only font (no explicit color) so the annotation inherits the layout font color
        // and the dark/light toggle's `font.color` relayout keeps flipping it (see `ann_font`).
        let font = {
            let f = Font::new().size(16);
            if themed { f } else { f.family(FONT_FAMILY) }
        };
        // y=1 + a small upward shift parks the title in the top-margin band where the layout
        // title would render; anchored bottom, so a subtitle line grows upward.
        anns.push(panel_title_annotation(panel, 0.5, 1.0, font).y_shift(y_shift));
        layout
    } else {
        layout.title(Title::with_text(panel.display_title()))
    };
    if anns.is_empty() {
        layout
    } else {
        layout.annotations(anns)
    }
}

/// Build a standalone themed `Plot` for one panel, used as a cell in the inline-div Data Schematic.
///
/// One function per panel kind rather than one big body: at `opt-level=0` every
/// `Plot`/`Layout` local gets its own stack slot with no reuse across branches, and a
/// single body building all panel kinds needs megabytes of stack -- overflowing
/// Windows' 1 MB main-thread default in debug builds (issue #4328). `#[inline(never)]`
/// on the per-kind builders keeps release builds from re-merging them into one frame.
fn smart_inline_panel_plot(
    panel: &Panel,
    color: &'static str,
    freq: &FreqMap,
    hist: &HashMap<usize, Vec<f64>>,
    outliers: &HashMap<usize, OutlierStats>,
    theme: Option<BuiltinTheme>,
    log_scale: LogScale,
) -> Plot {
    match &panel.kind {
        PanelKind::Map { .. } => inline_panel_plot_map(panel, color, theme),
        PanelKind::Geo { .. } => inline_panel_plot_geo(panel, theme),
        PanelKind::ChoroplethMap { .. } => inline_panel_plot_choropleth_map(panel, theme),
        PanelKind::Choropleth { .. } => inline_panel_plot_choropleth(panel, theme),
        PanelKind::Scatter3D { .. } => inline_panel_plot_scatter3d(panel, color, theme),
        PanelKind::CyclicProfile { .. } => inline_panel_plot_cyclic_profile(panel, theme),
        PanelKind::KpiRow { .. } => inline_panel_plot_kpi_row(panel, theme),
        PanelKind::Hierarchy { .. } => inline_panel_plot_hierarchy(panel, theme),
        PanelKind::Sankey { .. } => inline_panel_plot_sankey(panel, theme),
        PanelKind::Parcats { .. } => inline_panel_plot_parcats(panel, theme),
        PanelKind::AnimatedGeo { .. } => inline_panel_plot_animated_geo(panel, theme),
        PanelKind::AnimatedBubble { .. } => inline_panel_plot_animated_bubble(panel, theme),
        PanelKind::AnimatedScatterPair { .. } => {
            inline_panel_plot_animated_scatter_pair(panel, theme)
        },
        _ => inline_panel_plot_cartesian(panel, color, freq, hist, outliers, theme, log_scale),
    }
}

// map panels use a MapLibre `map` layout (tile basemap, framed to the points) instead of
// cartesian x/y axes, so they're assembled here rather than through the shared
// `panel_trace`/axis path.
#[inline(never)]
fn inline_panel_plot_map(panel: &Panel, color: &'static str, theme: Option<BuiltinTheme>) -> Plot {
    let themed = theme.is_some();
    let row_height = panel_render_height(&panel.kind, panel.axis_log);
    let PanelKind::Map {
        lats,
        lons,
        density,
        cluster,
        outlier_lats,
        outlier_lons,
        hover_text,
        outlier_hover_text,
        sizes,
        pan_bounds,
        photos,
        outlier_photos,
        row_ids,
        outlier_row_ids,
    } = &panel.kind
    else {
        unreachable!("smart_inline_panel_plot dispatches on panel.kind")
    };
    // smart auto panel: trim outliers so a few bad geocodes don't blow up the default view
    #[cfg_attr(not(feature = "geocode"), expect(unused_mut))]
    let (mut center, mut zoom) = map_center_zoom(
        lats,
        lons,
        MAP_FRAME_TRIM_FRAC,
        MAP_PANEL_ASSUMED_WIDTH_PX,
        MAP_PANEL_USABLE_HEIGHT_PX,
    );
    // frame the tight CORE extent (outliers are drawn distinctly and reachable via the "Full
    // extent" zoom button below).
    #[cfg(feature = "geocode")]
    let mut extent_menu: Option<UpdateMenu> = None;
    #[cfg(feature = "geocode")]
    if let Some(meta) = &panel.geo_meta {
        let (c, z) = extent_center_zoom(&meta.extent);
        center = c;
        zoom = z;
        // with geographic outliers, offer Core/Full extent zoom buttons: the map opens tight on
        // the core, and "Full extent" reveals the strays without manual panning/zooming.
        if let Some(full) = &meta.full_extent {
            extent_menu = Some(extent_zoom_menu(&meta.extent, full));
        }
    }
    let mut plot = Plot::new();
    if *density {
        // many points overplot into a solid mass as markers, so aggregate into a heatmap.
        // The density weight is a uniform 1.0, so surface the same per-point labels the marker
        // branch uses — via hovertemplate, since density traces ignore hovertext.
        let mut dens = DensityMap::new(lats.clone(), lons.clone(), vec![1.0_f64; lats.len()])
            .radius(MAP_SMART_DENSITY_RADIUS_PX);
        if !hover_text.is_empty() {
            dens = dens
                .text_array(map_hover_text_values(hover_text))
                .hover_template(MAP_HOVER_TEMPLATE);
        }
        plot.add_trace(dens);
    } else {
        let mut core_marker = Marker::new().color(color).opacity(MAP_POINT_OPACITY);
        if let Some(sizes) = sizes {
            core_marker = core_marker.size_array(scale_bubble_sizes(sizes));
        }
        let mut core_trace = ScatterMap::new(lats.clone(), lons.clone())
            .mode(Mode::Markers)
            .marker(core_marker);
        if !hover_text.is_empty() {
            core_trace = core_trace
                .text_array(map_hover_text_values(hover_text))
                .hover_template(MAP_HOVER_TEMPLATE);
        }
        // `--photos`: per-point image URLs for the hover-dwell lightbox. Inert everywhere
        // except the page's own JS — plotly never renders `customdata` unless a hovertemplate
        // references it, and `MAP_HOVER_TEMPLATE` deliberately does not.
        if !photos.is_empty() {
            core_trace = core_trace.custom_data(photos.clone());
        }
        // Data-viewer cross-link: the row ordinal per point rides in `ids`, NOT `customdata`
        // (which `--photos` already owns as a flat string array). `ids` is inert to rendering
        // and to the hover template — plotly only uses it for animation object-constancy —
        // and the bridge reads it as `gd.data[curve].ids[pointNumber]`. The selected/
        // unselected styles are baked here so the bridge only has to set `selectedpoints`.
        if !row_ids.is_empty() {
            core_trace = core_trace
                .ids(row_ids.clone())
                .selected(
                    MapSelection::new()
                        .size(MAP_SELECTED_POINT_SIZE)
                        .color(MAP_SELECTED_POINT_COLOR),
                )
                .unselected(MapSelection::new().opacity(MAP_UNSELECTED_POINT_OPACITY));
        }
        if *cluster {
            // native MapLibre client-side clustering, configured but DISABLED at load so the
            // map opens on individual points; the "Clusters/Points" toggle (see
            // `cluster_toggle_menu`) flips `cluster.enabled` on. `max_zoom` is
            // baked now so toggling on honors it. No point is ever dropped —
            // clustering only changes rendering, not the embedded data.
            core_trace = core_trace.cluster(
                Cluster::new()
                    .enabled(false)
                    .max_zoom(SMART_CLUSTER_MAX_ZOOM),
            );
        }
        plot.add_trace(core_trace);
    }
    // geographic outliers as a distinct amber marker trace on top of the core points/heatmap.
    // Outliers are always markers (even in density mode), so they carry hover labels too.
    if !outlier_lats.is_empty() {
        let mut out_trace = ScatterMap::new(outlier_lats.clone(), outlier_lons.clone())
            .name(&t!("viz.map.geographic_outliers"))
            .mode(Mode::Markers)
            .marker(outlier_marker_map())
            .show_legend(false);
        if !outlier_hover_text.is_empty() {
            out_trace = out_trace
                .text_array(map_hover_text_values(outlier_hover_text))
                .hover_template(MAP_HOVER_TEMPLATE);
        }
        if !outlier_photos.is_empty() {
            out_trace = out_trace.custom_data(outlier_photos.clone());
        }
        // outliers are selectable too; the base amber identity is untouched, only the
        // selected/unselected states differ (see the core trace above)
        if !outlier_row_ids.is_empty() {
            out_trace = out_trace
                .ids(outlier_row_ids.clone())
                .selected(
                    MapSelection::new()
                        .size(MAP_SELECTED_POINT_SIZE)
                        .color(MAP_SELECTED_POINT_COLOR),
                )
                .unselected(MapSelection::new().opacity(MAP_UNSELECTED_POINT_OPACITY));
        }
        plot.add_trace(out_trace);
    }
    #[cfg(feature = "geocode")]
    if let Some(meta) = &panel.geo_meta {
        add_extent_overlay_map(&mut plot, meta);
    }
    if let Some(overlay) = &panel.geojson_overlay {
        for trace in geojson_overlay_map_traces(overlay) {
            plot.add_trace(trace);
        }
    }
    // The row pin, LAST so it draws over the points and any overlay. Gated on the same
    // condition as the cross-link itself (`row_ids` non-empty): with no drawer there is
    // nothing to select, so nothing could ever fill it in, and an image export -- which
    // passes `capture_row_ids: false` -- stays byte-identical.
    if !row_ids.is_empty() {
        // halo first so the ring renders beneath the core; it never answers a hover, so the
        // pin's own label is the only one a reader can get
        plot.add_trace(
            ScatterMap::new(Vec::<f64>::new(), Vec::<f64>::new())
                .name(MAP_ROW_PIN_HALO_NAME)
                .mode(Mode::Markers)
                .marker(row_pin_halo_marker())
                .hover_info(HoverInfo::Skip)
                .show_legend(false),
        );
        plot.add_trace(
            ScatterMap::new(Vec::<f64>::new(), Vec::<f64>::new())
                .name(MAP_ROW_PIN_NAME)
                .mode(Mode::Markers)
                .marker(row_pin_marker())
                // baked now so the bridge only ever has to set lat/lon/text
                .hover_template(MAP_HOVER_TEMPLATE)
                .show_legend(false),
        );
    }
    // Carto tiles work from local files (no Referer header required). OSM enforces a Referer
    // policy and returns 403 when the HTML is opened directly from disk. A density heatmap is a
    // data overlay, so it gets the label-free basemap; a point map keeps labels for context.
    let mut layout_map = LayoutMap::new()
        .style(smart_map_basemap(is_dark_theme(theme), *density))
        .center(center)
        .zoom(f64::from(zoom));
    // constrain panning to the data's footprint so the map can't be dragged off into empty
    // ocean. Computed in `build_map_panel` from the FULL (pre-downsample) coordinate cloud, so
    // the box always encloses the true extent the "Full extent" zoom button / overlay use — a
    // stride-dropped extreme can't shrink it below the rendered core/outlier points.
    if let Some(bounds) = pan_bounds {
        layout_map = layout_map.bounds(bounds.clone());
    }
    let mut layout = Layout::new()
        .show_legend(false)
        .height(row_height)
        .margin(Margin::new().top(48).bottom(20).left(20).right(20).pad(4))
        .map(layout_map);
    let mut menus: Vec<UpdateMenu> = Vec::new();
    #[cfg(feature = "geocode")]
    if let Some(menu) = extent_menu {
        menus.push(menu);
    }
    if *cluster {
        menus.push(cluster_toggle_menu());
    }
    if !menus.is_empty() {
        layout = layout.update_menus(menus);
    }
    if !themed {
        layout = layout
            .font(Font::new().family(FONT_FAMILY).color(INK).size(12))
            .paper_background_color(PAPER_BG);
    }
    let layout = inline_panel_title(layout, panel, themed, Vec::new());
    plot.set_layout(apply_theme(layout, theme));
    plot.set_configuration(Configuration::new().responsive(true));
    plot
}

// geo panels use a `geo` projection layout (no tiles, fully offline) instead of cartesian
// x/y axes, so they're assembled here like the MapLibre map panel above.
#[inline(never)]
fn inline_panel_plot_geo(panel: &Panel, theme: Option<BuiltinTheme>) -> Plot {
    let themed = theme.is_some();
    let row_height = panel_render_height(&panel.kind, panel.axis_log);
    let PanelKind::Geo {
        lats,
        lons,
        outlier_lats,
        outlier_lons,
        hover_text,
        outlier_hover_text,
        sizes,
        row_ids,
        outlier_row_ids,
    } = &panel.kind
    else {
        unreachable!("smart_inline_panel_plot dispatches on panel.kind")
    };
    let mut plot = Plot::new();
    // a fixed high-contrast marker (warm fill + thin white outline) instead of the palette
    // accent: the `geo` projection paints a light-blue ocean and light-gray land, and the
    // palette's first color is a blue that disappears against the ocean (e.g. coastal/island
    // points on a world overview). Crimson reads on both land and water and in dark themes.
    let mut core_marker = Marker::new()
        .color(NamedColor::Crimson)
        .opacity(MAP_POINT_OPACITY)
        .line(Line::new().color(NamedColor::White).width(0.5));
    if let Some(sizes) = sizes {
        core_marker = core_marker.size_array(scale_bubble_sizes(sizes));
    }
    let mut core_trace = ScatterGeo::new(lats.clone(), lons.clone())
        .mode(Mode::Markers)
        .marker(core_marker);
    if !hover_text.is_empty() {
        core_trace = core_trace
            .text_array(map_hover_text_values(hover_text))
            .hover_template(MAP_HOVER_TEMPLATE);
    }
    // data-viewer cross-link; see the ScatterMap core trace above. ScatterGeo renders as SVG
    // rather than WebGL, so selection restyling here is ordinary plotly behavior.
    if !row_ids.is_empty() {
        core_trace = core_trace
            .ids(row_ids.clone())
            .selected(
                GeoSelection::new()
                    .size(MAP_SELECTED_POINT_SIZE)
                    .color(MAP_SELECTED_POINT_COLOR),
            )
            .unselected(GeoSelection::new().opacity(MAP_UNSELECTED_POINT_OPACITY));
    }
    plot.add_trace(core_trace);
    // geographic outliers as a distinct amber/X marker trace on top of the core points
    if !outlier_lats.is_empty() {
        let mut out_trace = ScatterGeo::new(outlier_lats.clone(), outlier_lons.clone())
            .name(&t!("viz.map.geographic_outliers"))
            .mode(Mode::Markers)
            .marker(outlier_marker_geo())
            .show_legend(false);
        if !outlier_hover_text.is_empty() {
            out_trace = out_trace
                .text_array(map_hover_text_values(outlier_hover_text))
                .hover_template(MAP_HOVER_TEMPLATE);
        }
        if !outlier_row_ids.is_empty() {
            out_trace = out_trace
                .ids(outlier_row_ids.clone())
                .selected(
                    GeoSelection::new()
                        .size(MAP_SELECTED_POINT_SIZE)
                        .color(MAP_SELECTED_POINT_COLOR),
                )
                .unselected(GeoSelection::new().opacity(MAP_UNSELECTED_POINT_OPACITY));
        }
        plot.add_trace(out_trace);
    }
    #[cfg(feature = "geocode")]
    if let Some(meta) = &panel.geo_meta {
        add_extent_overlay_geo(&mut plot, meta);
    }
    if let Some(overlay) = &panel.geojson_overlay {
        for trace in geojson_overlay_geo_traces(overlay, None) {
            plot.add_trace(trace);
        }
    }
    // the row pin, on top of everything (see the `PanelKind::Map` arm). A geo panel never moves
    // its camera for a selection, but it still shows the pin -- downsampling drops rows here
    // too, so without it a click on such a row would do nothing at all.
    if !row_ids.is_empty() {
        plot.add_trace(
            ScatterGeo::new(Vec::<f64>::new(), Vec::<f64>::new())
                .name(MAP_ROW_PIN_HALO_NAME)
                .mode(Mode::Markers)
                .marker(row_pin_halo_marker())
                .hover_info(HoverInfo::Skip)
                .show_legend(false),
        );
        plot.add_trace(
            ScatterGeo::new(Vec::<f64>::new(), Vec::<f64>::new())
                .name(MAP_ROW_PIN_NAME)
                .mode(Mode::Markers)
                .marker(row_pin_marker())
                .hover_template(MAP_HOVER_TEMPLATE)
                .show_legend(false),
        );
    }
    let (geo_land, geo_water, geo_bg) = geo_palette(theme);
    // when every plotted point (core + outliers) sits in a single plotly continent, frame the
    // world overview to that continent's `scope` instead of showing the whole globe.
    let continent = {
        let mut all_lats = lats.clone();
        all_lats.extend_from_slice(outlier_lats);
        let mut all_lons = lons.clone();
        all_lons.extend_from_slice(outlier_lons);
        continent_scope(&all_lats, &all_lons)
    };
    let mut geo = LayoutGeo::new()
        .projection(Projection::new().projection_type(ProjectionType::NaturalEarth))
        .resolution(GeoResolution::OneOverFiftyMillion)
        .showland(true)
        .landcolor(geo_land)
        .showocean(true)
        .oceancolor(geo_water)
        .showlakes(true)
        .lakecolor(geo_water)
        .showcountries(true)
        .bgcolor(geo_bg);
    if let Some(scope) = continent {
        geo = geo.scope(scope);
    }
    let mut layout = Layout::new()
        .show_legend(false)
        .height(row_height)
        .margin(Margin::new().top(48).bottom(20).left(20).right(20).pad(4))
        .geo(geo);
    if !themed {
        layout = layout
            .font(Font::new().family(FONT_FAMILY).color(INK).size(12))
            .paper_background_color(PAPER_BG);
    }
    let layout = inline_panel_title(layout, panel, themed, Vec::new());
    plot.set_layout(apply_theme(layout, theme));
    plot.set_configuration(Configuration::new().responsive(true));
    plot
}

// metro-scale `--geojson` choropleth: a MapLibre tile basemap (`map` subplot) supplies the fine
// street/coastline detail the projection basemap lacks at city scale. Center/zoom are
// precomputed from the matched-region bbox, so render frames the metro area without re-scanning
// geometry. Token-free carto tiles (load from local files; no Referer header) chosen by theme,
// matching the point `Map` panel.
//
// NOTE: plotly's `fitbounds` is a `layout.geo`-only property (the projection Choropleth below
// uses it) — the MapLibre `map` subplot has no fitbounds/auto-fit equivalent, so we frame it
// manually via the precomputed center/zoom (and re-aim the GL camera on resize/fullscreen).
#[inline(never)]
fn inline_panel_plot_choropleth_map(panel: &Panel, theme: Option<BuiltinTheme>) -> Plot {
    let themed = theme.is_some();
    let row_height = panel_render_height(&panel.kind, panel.axis_log);
    let PanelKind::ChoroplethMap {
        locations,
        z,
        geojson,
        feature_id_key,
        hover_text,
        center_lon,
        center_lat,
        zoom,
        measure_label,
        region_filter,
    } = &panel.kind
    else {
        unreachable!("smart_inline_panel_plot dispatches on panel.kind")
    };
    let mut plot = Plot::new();
    let mut cm_trace = ChoroplethMap::new(locations.clone(), z.clone())
        .geojson(geojson.clone())
        .feature_id_key(feature_id_key.clone())
        .color_scale(ColorScale::Palette(ColorScalePalette::Viridis))
        .show_scale(true)
        .color_bar(ColorBar::new().title(escape_hover(measure_label)))
        .marker(
            ChoroplethMarker::new()
                .line(Line::new().width(0.5))
                .opacity(CHOROPLETH_MAP_FILL_OPACITY),
        )
        // insert the fill ABOVE every basemap layer. Plotly's default drops it just above
        // the water layers, so the basemap's roads render on TOP of the regions and muddy
        // the read; "" lifts it over the roads (a near-opaque fill then reads cleanly, with
        // the inter-polygon gaps still revealing enough basemap to orient).
        .below("")
        .hover_text_array(hover_text.clone())
        .hover_info(HoverInfo::Text);
    // region column index rides as trace `meta` so the region-click filter chrome can find
    // this panel (and its column) without any DOM sentinel.
    if let Some((rc, _)) = region_filter {
        cm_trace = cm_trace.meta(*rc);
    }
    plot.add_trace(cm_trace);
    // keep the LABELED basemap: since the fill is inserted above every basemap layer
    // (`below("")` on the trace), place names can no longer bleed over the regions — they sit
    // under the fill and show only in the gaps between polygons (water, harbor, parks, and the
    // uncovered land around the metro), where they aid orientation. Passing `false` (not a
    // label-suppressing overlay); the theme-toggle JS keys off the `-nolabels` suffix, so a
    // labeled style here stays labeled across a light/dark switch.
    let layout_map = LayoutMap::new()
        .style(smart_map_basemap(is_dark_theme(theme), false))
        .center(Center::new(*center_lat, *center_lon))
        .zoom(*zoom);
    // opt-in "Basemap labels" toggle: hides/shows the carto basemap's place names (default on).
    // The two style strings are baked for the render-time theme; the theme-toggle JS rewrites
    // them to the active theme on each flip (preserving the `-nolabels` suffix).
    let labeled_style = smart_map_basemap(is_dark_theme(theme), false);
    let nolabels_style = smart_map_basemap(is_dark_theme(theme), true);
    let mut layout = Layout::new()
        .show_legend(false)
        .height(row_height)
        .margin(Margin::new().top(48).bottom(20).left(20).right(20).pad(4))
        .update_menus(vec![basemap_labels_toggle_menu(
            labeled_style,
            nolabels_style,
        )])
        .map(layout_map);
    if !themed {
        layout = layout
            .font(Font::new().family(FONT_FAMILY).color(INK).size(12))
            .paper_background_color(PAPER_BG);
    }
    let layout = inline_panel_title(layout, panel, themed, Vec::new());
    plot.set_layout(apply_theme(layout, theme));
    plot.set_configuration(Configuration::new().responsive(true));
    plot
}

// choropleth panels fill whole regions on a `geo` projection layout (no tiles), colored by the
// per-region point count; assembled here like the geo point map above.
#[inline(never)]
fn inline_panel_plot_choropleth(panel: &Panel, theme: Option<BuiltinTheme>) -> Plot {
    let themed = theme.is_some();
    let row_height = panel_render_height(&panel.kind, panel.axis_log);
    let PanelKind::Choropleth {
        locations,
        z,
        location_mode,
        geojson,
        feature_id_key,
        hover_text,
        measure_label,
        region_filter,
    } = &panel.kind
    else {
        unreachable!("smart_inline_panel_plot dispatches on panel.kind")
    };
    let mut plot = Plot::new();
    let mut trace = Choropleth::new(locations.clone(), z.clone())
        .location_mode(location_mode.clone())
        .color_scale(ColorScale::Palette(ColorScalePalette::Viridis))
        .show_scale(true)
        .color_bar(ColorBar::new().title(escape_hover(measure_label)))
        .marker(ChoroplethMarker::new().line(Line::new().width(0.5)))
        .hover_text_array(hover_text.clone())
        .hover_info(HoverInfo::Text);
    // a point-in-polygon panel carries its own GeoJSON polygons (geojson-id mode); the
    // built-in geocode-derived panels (iso3 / usa-states) carry neither.
    if let (Some(gj), Some(key)) = (geojson, feature_id_key) {
        trace = trace.geojson(gj.clone()).feature_id_key(key.clone());
    }
    // region column index rides as trace `meta` so the region-click filter chrome can find
    // this panel (and its column) without any DOM sentinel.
    if let Some((rc, _)) = region_filter {
        trace = trace.meta(*rc);
    }
    plot.add_trace(trace);
    // frame to the FILLED REGION GEOMETRIES, not the source points (whose bounding box clips
    // the countries/states at the edges — e.g. a city near a country's center).
    // US-states use the albers-usa composite (CONUS + AK/HI insets), which self-frames
    // the US; everything else uses natural-earth with `fitbounds: "locations"`, so
    // Plotly fits the view to the union of the rendered region polygons — a
    // European/Asian/etc. cluster zooms to those countries, global data stays
    // world-scale.
    let (geo_land, _geo_water, geo_bg) = geo_palette(theme);
    let mut geo = LayoutGeo::new()
        .resolution(GeoResolution::OneOverFiftyMillion)
        .showland(true)
        .landcolor(geo_land)
        .showcountries(true)
        // the choropleth paints no ocean, so geo.bgcolor is the sea; it carries the theme so a
        // dark map has a dark sea. The smart toggle also relayouts geo.bgcolor for live
        // switching.
        .bgcolor(geo_bg);
    if matches!(location_mode, LocationMode::UsaStates) {
        // scope:"usa" restricts the basemap (showland, showcountries) to the US extent,
        // preventing Canadian land (e.g. British Columbia) from bleeding into the
        // albers-usa composite canvas above Washington State.
        geo = geo
            .projection(Projection::new().projection_type(ProjectionType::AlbersUsa))
            .scope("usa");
    } else {
        geo = geo
            .projection(Projection::new().projection_type(ProjectionType::NaturalEarth))
            .fitbounds(GeoFitBounds::Locations);
    }
    let mut layout = Layout::new()
        .show_legend(false)
        .height(row_height)
        .margin(Margin::new().top(48).bottom(20).left(20).right(20).pad(4))
        .geo(geo);
    if !themed {
        layout = layout
            .font(Font::new().family(FONT_FAMILY).color(INK).size(12))
            .paper_background_color(PAPER_BG);
    }
    let layout = inline_panel_title(layout, panel, themed, Vec::new());
    plot.set_layout(apply_theme(layout, theme));
    plot.set_configuration(Configuration::new().responsive(true));
    plot
}

// 3D scatter panels use a `scene` (3D) layout instead of cartesian x/y axes, so they're
// assembled here as well.
#[inline(never)]
fn inline_panel_plot_scatter3d(
    panel: &Panel,
    color: &'static str,
    theme: Option<BuiltinTheme>,
) -> Plot {
    let themed = theme.is_some();
    let row_height = panel_render_height(&panel.kind, panel.axis_log);
    let PanelKind::Scatter3D { xs, ys, zs, labels } = &panel.kind else {
        unreachable!("smart_inline_panel_plot dispatches on panel.kind")
    };
    let (x_label, y_label, z_label) = labels;
    // plotly's default 3D hover labels the coordinates with the bare axis letters x/y/z (it
    // does NOT read the scene axis titles), so name each dimension explicitly. escape: this
    // is a hoverTEMPLATE — a raw header containing `<extra>`/`<b>` would otherwise
    // terminate/format the template instead of displaying literally.
    let hover = format!(
        "{}: %{{x:,.3f}}<br>{}: %{{y:,.3f}}<br>{}: %{{z:,.3f}}<extra></extra>",
        escape_hover(x_label),
        escape_hover(y_label),
        escape_hover(z_label)
    );
    let mut plot = Plot::new();
    plot.add_trace(
        Scatter3D::new(xs.clone(), ys.clone(), zs.clone())
            .mode(Mode::Markers)
            .marker(Marker::new().color(color).opacity(MAP_POINT_OPACITY))
            .hover_template(hover),
    );
    let scene = LayoutScene::new()
        // axis titles render plotly pseudo-HTML just like hovers and panel titles, and
        // these labels are raw CSV headers
        .x_axis(Axis::new().title(Title::with_text(escape_hover(x_label))))
        .y_axis(Axis::new().title(Title::with_text(escape_hover(y_label))))
        .z_axis(Axis::new().title(Title::with_text(escape_hover(z_label))));
    let mut layout = Layout::new()
        .show_legend(false)
        .height(row_height)
        .margin(Margin::new().top(48).bottom(20).left(20).right(20).pad(4))
        .scene(scene);
    if !themed {
        layout = layout
            .font(Font::new().family(FONT_FAMILY).color(INK).size(12))
            .paper_background_color(PAPER_BG);
    }
    let layout = inline_panel_title(layout, panel, themed, Vec::new());
    plot.set_layout(apply_theme(layout, theme));
    plot.set_configuration(Configuration::new().responsive(true));
    plot
}

// cyclic-seasonality panels use a `polar` subplot — plotly auto-creates it from the
// ScatterPolar trace (as the standalone `viz radar` does), so they own their whole Plot
// here too.
#[inline(never)]
fn inline_panel_plot_cyclic_profile(panel: &Panel, theme: Option<BuiltinTheme>) -> Plot {
    let themed = theme.is_some();
    let row_height = panel_render_height(&panel.kind, panel.axis_log);
    let PanelKind::CyclicProfile { theta, r } = &panel.kind else {
        unreachable!("smart_inline_panel_plot dispatches on panel.kind")
    };
    let mut plot = Plot::new();
    plot.add_trace(
        ScatterPolar::new(theta.clone(), r.clone())
            // lines+markers so each phase (hour/day/month) is its own hoverable vertex, not
            // just a single hover on the ring.
            .mode(Mode::LinesMarkers)
            .fill(Fill::ToSelf)
            .name(escape_hover(&panel.name))
            .hover_template(&t!("viz.hover.polar_records")),
    );
    // A polar subplot draws its angular tick labels ("06h", "Mon", "Jan") OUTSIDE its plot
    // area — ~22px past the top and bottom edges — and this plotly version's `LayoutPolar`
    // exposes no `domain` to rein the ring in. So the panel's own margins do the work: a
    // taller top band (with the title lifted to sit above the spill, see
    // POLAR_TITLE_Y_SHIFT) keeps the title clear of the 12-o'clock label, and a taller
    // bottom band keeps the 6-o'clock label from being clipped by the panel edge.
    let mut layout = Layout::new()
        .show_legend(false)
        .height(row_height)
        .margin(Margin::new().top(64).bottom(44).left(20).right(20).pad(4));
    if !themed {
        layout = layout
            .font(Font::new().family(FONT_FAMILY).color(INK).size(12))
            .paper_background_color(PAPER_BG);
    }
    let layout = inline_panel_title_shifted(layout, panel, themed, Vec::new(), POLAR_TITLE_Y_SHIFT);
    plot.set_layout(apply_theme(layout, theme));
    plot.set_configuration(Configuration::new().responsive(true));
    plot
}

// hierarchy (treemap/sunburst) panels are domain-based — no cartesian x/y axes — so, like the
// map/geo/3D panels above, they own their whole Plot here.
// KPI row: a full-width band of domain-positioned `Indicator` tiles (no cartesian axes), so it
// owns its whole Plot here like the hierarchy/sankey panels below.
#[inline(never)]
fn inline_panel_plot_kpi_row(panel: &Panel, theme: Option<BuiltinTheme>) -> Plot {
    let themed = theme.is_some();
    let row_height = panel_render_height(&panel.kind, panel.axis_log);
    let PanelKind::KpiRow { tiles } = &panel.kind else {
        unreachable!("smart_inline_panel_plot dispatches on panel.kind")
    };
    let mut plot = Plot::new();
    let count = tiles.len().max(1) as f64;
    // even horizontal split with a small inter-tile gutter so adjacent gauges/numbers breathe
    let gutter = 0.02_f64;
    // Indicator-domain bottom (`ind_y0`) and the shared label baseline (`label_y`), as paper
    // fractions of this panel's plot (0 = bottom). Number tiles center the number in the
    // domain; gauge tiles render it lower, at the dial pivot (~30px below a plain number at the
    // gauge height). The whole row is height-matched to its content (`panel_render_height`),
    // so the domain fills the plot and the label sits snug under the numbers with no dead band
    // below. Anchor ALL labels at one baseline so they share a clean line (no per-tile step);
    // in a mixed row the gauge number is binding, so it is pinned just under the gauge number
    // and the plain-number tiles carry a slightly larger (still tidy) number-to-label gap.
    let has_gauge = tiles.iter().any(|t| t.gauge.is_some());
    let (ind_y0, label_y) = if has_gauge {
        (0.34_f64, 0.30_f64)
    } else {
        (0.10_f64, 0.24_f64)
    };
    let max_chars = ((150.0 / count) as usize).clamp(12, 28);
    let ann_font = |size: usize| {
        let f = Font::new().size(size);
        if themed { f } else { f.family(FONT_FAMILY) }
    };
    let mut kpi_labels: Vec<Annotation> = Vec::with_capacity(tiles.len());
    for (i, tile) in tiles.iter().enumerate() {
        let lo = (i as f64 / count) + gutter / 2.0;
        let hi = ((i + 1) as f64 / count) - gutter / 2.0;
        plot.add_trace(kpi_indicator(tile, [lo, hi], [ind_y0, 1.0]));
        kpi_labels.push(kpi_label_annotation(
            &tile.label,
            f64::midpoint(lo, hi),
            label_y,
            max_chars,
            ann_font(13),
        ));
    }
    // no panel title — each tile carries its own word-wrapped subtitle label below its number
    // (`kpi_label_annotation`), so reclaim the top band a title would occupy. Vertical hover
    // modebar only.
    let mut layout = Layout::new()
        .show_legend(false)
        .height(row_height)
        .margin(Margin::new().top(20).bottom(20).left(20).right(20).pad(4))
        .mode_bar(ModeBar::new().orientation(Orientation::Vertical))
        .annotations(kpi_labels);
    if !themed {
        layout = layout
            .font(Font::new().family(FONT_FAMILY).color(INK).size(12))
            .paper_background_color(PAPER_BG);
    }
    plot.set_layout(apply_theme(layout, theme));
    plot.set_configuration(Configuration::new().responsive(true));
    plot
}

#[inline(never)]
fn inline_panel_plot_hierarchy(panel: &Panel, theme: Option<BuiltinTheme>) -> Plot {
    let themed = theme.is_some();
    let row_height = panel_render_height(&panel.kind, panel.axis_log);
    let PanelKind::Hierarchy {
        style,
        labels,
        parents,
        values,
        ids,
    } = &panel.kind
    else {
        unreachable!("smart_inline_panel_plot dispatches on panel.kind")
    };
    let mut plot = Plot::new();
    plot.add_trace(hierarchy_trace(*style, labels, parents, values, ids));
    let mut layout = Layout::new()
        .show_legend(false)
        .height(row_height)
        .margin(Margin::new().top(48).bottom(20).left(20).right(20).pad(4));
    if !themed {
        layout = layout
            .font(Font::new().family(FONT_FAMILY).color(INK).size(12))
            .paper_background_color(PAPER_BG);
    }
    let layout = inline_panel_title(layout, panel, themed, Vec::new());
    plot.set_layout(apply_theme(layout, theme));
    plot.set_configuration(Configuration::new().responsive(true));
    plot
}

// Sankey flow panels are domain-based (bipartite nodes + weighted links), like the hierarchy
// panel above — no cartesian x/y axes — so they own their whole Plot here.
#[inline(never)]
fn inline_panel_plot_sankey(panel: &Panel, theme: Option<BuiltinTheme>) -> Plot {
    let themed = theme.is_some();
    let row_height = panel_render_height(&panel.kind, panel.axis_log);
    let PanelKind::Sankey {
        node_labels,
        link_source,
        link_target,
        link_value,
        value_order,
    } = &panel.kind
    else {
        unreachable!("smart_inline_panel_plot dispatches on panel.kind")
    };
    let mut plot = Plot::new();
    let positions = sankey_value_positions(node_labels.len(), link_source, link_target, link_value);
    plot.add_trace(sankey_trace(
        node_labels,
        link_source,
        link_target,
        link_value,
        (*value_order).then_some(()).and(positions.as_ref()),
    ));
    let mut layout = Layout::new()
        .show_legend(false)
        .height(row_height)
        .margin(Margin::new().top(48).bottom(20).left(20).right(20).pad(4));
    // on-screen "node order" toggle (offered whichever order the panel opened in).
    if let Some((xs, ys)) = &positions {
        layout = layout.update_menus(vec![sankey_order_toggle_menu(xs, ys, *value_order)]);
    }
    if !themed {
        layout = layout
            .font(Font::new().family(FONT_FAMILY).color(INK).size(12))
            .paper_background_color(PAPER_BG);
    }
    let layout = inline_panel_title(layout, panel, themed, Vec::new());
    plot.set_layout(apply_theme(layout, theme));
    plot.set_configuration(Configuration::new().responsive(true));
    plot
}

// Parallel categories: a domain-based ribbon flow (like the Sankey above) — no cartesian axes.
#[inline(never)]
fn inline_panel_plot_parcats(panel: &Panel, theme: Option<BuiltinTheme>) -> Plot {
    let themed = theme.is_some();
    let row_height = panel_render_height(&panel.kind, panel.axis_log);
    let PanelKind::Parcats {
        dim_labels,
        tuples,
        counts,
    } = &panel.kind
    else {
        unreachable!("smart_inline_panel_plot dispatches on panel.kind")
    };
    let (dimensions, line, ordered) = parcats_dimensions_and_line(dim_labels, tuples, counts);
    let mut trace = Parcats::new()
        .dimensions(dimensions)
        .counts_array(counts.clone())
        .arrangement(ParcatsArrangement::Perpendicular)
        .bundle_colors(true);
    if let Some(line) = line {
        trace = trace.line(line);
    }
    let mut plot = Plot::new();
    plot.add_trace(trace);
    let mut layout = Layout::new()
        .show_legend(false)
        .height(row_height)
        .margin(Margin::new().top(48).bottom(20).left(20).right(20).pad(4))
        .update_menus(vec![parcats_order_toggle_menu(&ordered)]);
    if !themed {
        layout = layout
            .font(Font::new().family(FONT_FAMILY).color(INK).size(12))
            .paper_background_color(PAPER_BG);
    }
    let layout = inline_panel_title(layout, panel, themed, Vec::new());
    plot.set_layout(apply_theme(layout, theme));
    plot.set_configuration(Configuration::new().responsive(true));
    plot
}

// Animated geographic point map: dated points on a ScatterGeo projection basemap, revealed
// cumulatively over time buckets. Reconstructs the geo layout inline (like the static Geo block
// above — this render fn has no `args`, so `geo_base_layout` isn't reachable) and adds a scrub
// slider + Play/Pause. Reuses the standalone animated-geo core (native ScatterGeo animation).
#[inline(never)]
fn inline_panel_plot_animated_geo(panel: &Panel, theme: Option<BuiltinTheme>) -> Plot {
    let themed = theme.is_some();
    let row_height = panel_render_height(&panel.kind, panel.axis_log);
    let PanelKind::AnimatedGeo {
        bucket_lats,
        bucket_lons,
        bucket_labels,
        frame_label,
    } = &panel.kind
    else {
        unreachable!("smart_inline_panel_plot dispatches on panel.kind")
    };
    const SMART_SLIDER_SPEED_MS: u64 = 700;
    // a fixed high-contrast marker (warm fill + thin white outline), matching the static Geo
    // panel, so points read on both land and ocean and in dark themes.
    let geo_marker = || {
        Marker::new()
            .color(NamedColor::Crimson)
            .opacity(MAP_POINT_OPACITY)
            .line(Line::new().color(NamedColor::White).width(0.5))
    };
    let mut plot = Plot::new();
    // base = ALL points (every bucket flattened) so the initial/static view shows the full
    // cloud; Play then reveals it accumulating chronologically.
    let all_lats: Vec<f64> = bucket_lats.iter().flatten().copied().collect();
    let all_lons: Vec<f64> = bucket_lons.iter().flatten().copied().collect();
    plot.add_trace(
        ScatterGeo::new(all_lats, all_lons)
            .mode(Mode::Markers)
            .marker(geo_marker()),
    );
    // one CUMULATIVE frame per bucket: points from buckets 0..=k concatenated
    for (k, label) in bucket_labels.iter().enumerate() {
        let mut la: Vec<f64> = Vec::new();
        let mut lo: Vec<f64> = Vec::new();
        for b in 0..=k {
            la.extend_from_slice(&bucket_lats[b]);
            lo.extend_from_slice(&bucket_lons[b]);
        }
        let mut td = Traces::new();
        td.push(
            ScatterGeo::new(la, lo)
                .mode(Mode::Markers)
                .marker(geo_marker()),
        );
        plot.add_frame(Frame::new().name(label.clone()).data(td).traces(vec![0]));
    }
    let (geo_land, geo_water, geo_bg) = geo_palette(theme);
    let geo = LayoutGeo::new()
        .projection(Projection::new().projection_type(ProjectionType::NaturalEarth))
        .resolution(GeoResolution::OneOverFiftyMillion)
        .showland(true)
        .landcolor(geo_land)
        .showocean(true)
        .oceancolor(geo_water)
        .showlakes(true)
        .lakecolor(geo_water)
        .showcountries(true)
        .bgcolor(geo_bg);
    let mut layout = Layout::new()
        .show_legend(false)
        .height(row_height)
        // reserve a band below the map for the slider + Play/Pause (the panel height added the
        // same allowance), so the controls sit UNDER the map rather than overlaying/shrinking
        // it; usable draw height then equals a static map's (MAP_ROW_HEIGHT_PX - 48
        // - 20)
        .margin(
            Margin::new()
                .top(48)
                .bottom(20 + SLIDER_CONTROL_ALLOWANCE_PX)
                .left(20)
                .right(20)
                .pad(4),
        )
        .geo(geo);
    if let (Ok(slider), Ok(menu)) = (
        slider_control(frame_label, bucket_labels, SMART_SLIDER_SPEED_MS),
        play_pause_menu(SMART_SLIDER_SPEED_MS),
    ) {
        layout = layout.sliders(vec![slider]).update_menus(vec![menu]);
    }
    if !themed {
        layout = layout
            .font(Font::new().family(FONT_FAMILY).color(INK).size(12))
            .paper_background_color(PAPER_BG);
    }
    let layout = inline_panel_title(layout, panel, themed, Vec::new());
    plot.set_layout(apply_theme(layout, theme));
    plot.set_configuration(Configuration::new().responsive(true));
    plot
}

// Gapminder-style animated bubble chart: one trace per entity, each a single (x,y) bubble that
// moves + resizes per frame (per-frame full replacement). Rendered as its own inline Plot with
// a scrub slider + Play/Pause and a legend of entities. Bubble sizes are scaled GLOBALLY over
// all cells so an entity's size is comparable across frames and entities (a per-point
// scale_bubble_sizes call would collapse each single value to mid-size).
#[inline(never)]
fn inline_panel_plot_animated_bubble(panel: &Panel, theme: Option<BuiltinTheme>) -> Plot {
    let themed = theme.is_some();
    let row_height = panel_render_height(&panel.kind, panel.axis_log);
    let PanelKind::AnimatedBubble {
        data,
        x_label,
        y_label,
        size_label,
        frame_label,
    } = &panel.kind
    else {
        unreachable!("smart_inline_panel_plot dispatches on panel.kind")
    };
    const SMART_SLIDER_SPEED_MS: u64 = 700;
    let bucket_labels = &data.bucket_labels;
    let (x_range, y_range) = (data.x_range, data.y_range);
    let mut plot = Plot::new();
    add_bubble_traces_and_frames(&mut plot, data, x_label, y_label, size_label);
    // pin axes globally so bubbles don't reframe as they move
    let x_axis = Axis::new()
        // axis titles are a plotly markup sink; these labels are raw CSV headers
        .title(Title::with_text(escape_hover(x_label)))
        .range(vec![x_range.0, x_range.1]);
    let y_axis = Axis::new()
        .title(Title::with_text(escape_hover(y_label)))
        .range(vec![y_range.0, y_range.1]);
    let mut layout = Layout::new()
        .show_legend(true)
        .height(row_height)
        .margin(Margin::new().top(48).bottom(60).left(60).right(30).pad(4))
        .x_axis(x_axis)
        .y_axis(y_axis);
    if let (Ok(slider), Ok(menu)) = (
        slider_control(frame_label, bucket_labels, SMART_SLIDER_SPEED_MS),
        play_pause_menu(SMART_SLIDER_SPEED_MS),
    ) {
        layout = layout.sliders(vec![slider]).update_menus(vec![menu]);
    }
    if !themed {
        layout = layout
            .font(Font::new().family(FONT_FAMILY).color(INK).size(12))
            .paper_background_color(PAPER_BG)
            .plot_background_color(PAPER_BG);
    }
    let layout = inline_panel_title(layout, panel, themed, Vec::new());
    plot.set_layout(apply_theme(layout, theme));
    plot.set_configuration(Configuration::new().responsive(true));
    plot
}

// Animated scatter pair: the strongest numeric pair revealed cumulatively over time, colored by
// time bucket (early → late), rendered as its own inline Plot (scrub slider + Play/Pause).
// Forces the inline Data Schematic path (see has_noncartesian) and is HTML-only. Reuses the
// standalone animation core (slider_control/play_pause_menu).
#[inline(never)]
fn inline_panel_plot_animated_scatter_pair(panel: &Panel, theme: Option<BuiltinTheme>) -> Plot {
    let themed = theme.is_some();
    let row_height = panel_render_height(&panel.kind, panel.axis_log);
    let PanelKind::AnimatedScatterPair {
        xs,
        ys,
        bucket,
        bucket_labels,
        x_label,
        y_label,
        frame_label,
        x_range,
        y_range,
    } = &panel.kind
    else {
        unreachable!("smart_inline_panel_plot dispatches on panel.kind")
    };
    const SMART_SLIDER_SPEED_MS: u64 = 700;
    let n_buckets = bucket_labels.len();
    // pin the color scale to the full bucket range so a point keeps its color across frames
    let cmax = n_buckets.saturating_sub(1) as f64;
    // a two-tick color legend showing the earliest → latest bucket the gradient spans
    let colorbar = || {
        ColorBar::new()
            .thickness(10)
            .tick_mode(TickMode::Array)
            .tick_vals(vec![0.0, cmax])
            .tick_text(vec![
                bucket_labels.first().cloned().unwrap_or_default(),
                bucket_labels.last().cloned().unwrap_or_default(),
            ])
    };
    // one scatter over points whose bucket index is in `[start, end)` — a contiguous slice,
    // since points are sorted by bucket ascending — colored by each point's bucket index
    let scatter = |start: usize, end: usize| {
        Scatter::new(xs[start..end].to_vec(), ys[start..end].to_vec())
            .mode(Mode::Markers)
            .name(escape_hover(y_label.as_str()))
            .marker(
                Marker::new()
                    .color_array(bucket[start..end].iter().map(|&b| b as f64).collect())
                    .color_scale(ColorScale::Palette(ColorScalePalette::Viridis))
                    .cmin(0.0)
                    .cmax(cmax)
                    .show_scale(true)
                    .color_bar(colorbar())
                    .opacity(MAP_POINT_OPACITY),
            )
    };
    let mut plot = Plot::new();
    // base = the full cloud so the initial paint and any non-playing/static view degrade to the
    // static ScatterPair + time color; Play then reveals a TRAILING WINDOW migrating over time.
    plot.add_trace(scatter(0, xs.len()));
    // one trailing-window frame per bucket: show the current bucket plus the
    // `ANIM_TRAILING_WINDOW - 1` preceding buckets so the cloud is seen to MIGRATE, not merely
    // accumulate (a cumulative reveal hides motion — the point of a drift-selected pair is the
    // motion). Both bounds are `partition_point`s on the ascending bucket vector. Every frame
    // keeps the same single trace index so plotly leaves no stale traces on screen.
    for (k, label) in bucket_labels.iter().enumerate() {
        let lo_bucket = k.saturating_sub(ANIM_TRAILING_WINDOW - 1);
        let start = bucket.partition_point(|&b| b < lo_bucket);
        let end = bucket.partition_point(|&b| b <= k);
        let mut td = Traces::new();
        td.push(scatter(start, end));
        plot.add_frame(Frame::new().name(label.clone()).data(td).traces(vec![0]));
    }
    // pin axes globally so the cloud doesn't reframe as points accumulate
    let x_axis = Axis::new()
        // axis titles are a plotly markup sink; these labels are raw CSV headers
        .title(Title::with_text(escape_hover(x_label)))
        .range(vec![x_range.0, x_range.1]);
    let y_axis = Axis::new()
        .title(Title::with_text(escape_hover(y_label)))
        .range(vec![y_range.0, y_range.1]);
    let mut layout = Layout::new()
        .show_legend(false)
        .height(row_height)
        .margin(Margin::new().top(48).bottom(60).left(60).right(30).pad(4))
        .x_axis(x_axis)
        .y_axis(y_axis);
    if let (Ok(slider), Ok(menu)) = (
        slider_control(frame_label, bucket_labels, SMART_SLIDER_SPEED_MS),
        play_pause_menu(SMART_SLIDER_SPEED_MS),
    ) {
        layout = layout.sliders(vec![slider]).update_menus(vec![menu]);
    }
    if !themed {
        layout = layout
            .font(Font::new().family(FONT_FAMILY).color(INK).size(12))
            .paper_background_color(PAPER_BG)
            .plot_background_color(PAPER_BG);
    }
    let layout = inline_panel_title(layout, panel, themed, Vec::new());
    plot.set_layout(apply_theme(layout, theme));
    plot.set_configuration(Configuration::new().responsive(true));
    plot
}

// every cartesian (x/y-axis) panel kind: traces come from the shared `panel_trace` path.
#[inline(never)]
fn inline_panel_plot_cartesian(
    panel: &Panel,
    color: &'static str,
    freq: &FreqMap,
    hist: &HashMap<usize, Vec<f64>>,
    outliers: &HashMap<usize, OutlierStats>,
    theme: Option<BuiltinTheme>,
    log_scale: LogScale,
) -> Plot {
    // when a theme is set, its template drives backgrounds/fonts; otherwise apply qsv's look.
    let themed = theme.is_some();
    // overview panels (map/geo, correlation, time-series, …) render a little taller than the
    // per-column box/bar/histogram panels.
    let row_height = panel_render_height(&panel.kind, panel.axis_log);
    let is_box = matches!(
        panel.kind,
        PanelKind::BoxStats { .. }
            | PanelKind::BoxRaw { .. }
            | PanelKind::BoxOutliers { .. }
            | PanelKind::Violin { .. }
    );
    let is_corr = matches!(
        panel.kind,
        PanelKind::CorrHeatmap { .. } | PanelKind::AssocHeatmap { .. }
    );
    // A Quarter/Year-bucketed trend carries non-date-parseable labels ("2024-Q1"): render it
    // on a category axis via the tick_text mechanism instead of the date axis (issue #4216).
    let (is_date, ts_ticks) = match &panel.kind {
        PanelKind::TimeSeries {
            categorical, xs, ..
        } => (!*categorical, categorical.then(|| xs.clone())),
        _ => (false, None),
    };
    let is_toprel = matches!(panel.kind, PanelKind::TopRelationships { .. });
    let (trace, bar_max, log_y) =
        panel_trace(panel, color, freq, hist, outliers, None, theme, log_scale);

    let mut plot = Plot::new();
    plot.add_trace(trace);
    // a Lorenz panel needs its equality-diagonal reference line as a second trace (default axes).
    if matches!(panel.kind, PanelKind::Lorenz { .. }) {
        plot.add_trace(lorenz_diagonal_trace(None));
    }

    // correlation cells need extra left room for tick labels and right room for the colorbar;
    // the horizontal Top Relationships lollipop needs left room for its (truncated) pair labels;
    // log freq cells need a little extra left room for the rotated y-axis title cue.
    let (left, right) = if is_corr {
        (110, 90)
    } else if is_toprel {
        (TOPREL_LABEL_MAX_CHARS * CORR_LABEL_PX_PER_CHAR + 24, 30)
    } else if let PanelKind::Funnel {
        stages,
        form: PipelineForm::Funnel,
        ..
    } = &panel.kind
    {
        // size from the funnel's OWN ticks, not the lollipop's budget: the lollipop reserves room
        // for 56-char truncated column-PAIR labels, while a funnel's ticks are short stage names
        // ("Planned", "Committed"). Borrowing its constant reserved 416px for labels needing ~90,
        // leaving the headline panel squeezed into the left half of a full-width cell.
        // Right room as well: plotly's in-band text can overflow the widest band.
        (funnel_left_margin(stages), 60)
    } else if log_y {
        (60 + LOG_AXIS_TITLE_MARGIN_PX, 30)
    } else {
        (60, 30)
    };
    // the lollipop is the one horizontal panel: value (NMI) on a zoomed x, pair on the category y
    // — the opposite axis roles from every other (vertical) inline panel.
    let (x_axis, y_axis) = if let PanelKind::Funnel {
        stages,
        totals,
        form: PipelineForm::Funnel,
        ..
    } = &panel.kind
    {
        let ticks: Vec<String> = stages.clone();
        let max = totals.iter().copied().fold(0.0_f64, f64::max);
        (
            funnel_value_axis(max, theme),
            lollipop_category_axis(theme, &ticks),
        )
    } else if let PanelKind::TopRelationships { values, labels, .. } = &panel.kind {
        let (floor, ceil) = lollipop_value_range(values);
        let ticks: Vec<String> = labels
            .iter()
            .rev()
            .map(|l| truncate_label(l, TOPREL_LABEL_MAX_CHARS))
            .collect();
        (
            lollipop_value_axis(floor, ceil, theme),
            lollipop_category_axis(theme, &ticks),
        )
    } else {
        // per-axis log for a relationship panel (issue #4223); see the grid path for why the y
        // side sets only the axis type and not `styled_y_axis`'s title cue.
        let (x_log, y_log) = panel.axis_log;
        let mut y = styled_y_axis(bar_max, log_y, theme);
        if y_log {
            y = y.type_(AxisType::Log);
        }
        (
            styled_x_axis(
                is_box,
                is_date,
                x_log,
                theme,
                ts_ticks.or_else(|| freq_bar_tick_text(panel, freq)),
            ),
            y,
        )
    };
    let mut layout = Layout::new()
        .show_legend(false)
        .height(row_height)
        .margin(
            Margin::new()
                .top(48)
                .bottom(60)
                .left(left)
                .right(right)
                .pad(4),
        )
        .x_axis(x_axis)
        .y_axis(y_axis);
    if !themed {
        layout = layout
            .font(Font::new().family(FONT_FAMILY).color(INK).size(12))
            .paper_background_color(PAPER_BG)
            .plot_background_color(PAPER_BG);
    }

    // in-cell `r` labels ride along as `extra` — `Layout::annotations` replaces rather than
    // appends, and `inline_panel_title` may add the clickable title annotation to the same set.
    let extra = if let PanelKind::CorrHeatmap { matrix, .. }
    | PanelKind::AssocHeatmap { matrix, .. } = &panel.kind
        && matrix.len() <= CORR_INCELL_MAX_N
    {
        corr_incell_annotations(matrix, "x", "y")
    } else {
        Vec::new()
    };
    let layout = inline_panel_title(layout, panel, themed, extra);
    plot.set_layout(apply_theme(layout, theme));
    plot.set_configuration(Configuration::new().responsive(true));
    plot
}

/// Minimum array length worth converting to a base64 typed array: tiny arrays (extent-overlay
/// boxes, corner markers, small maps) stay plain JSON for readability, and below this size the
/// `{"dtype":..,"bdata":..}` wrapper outweighs the savings.
const BDATA_MIN_LEN: usize = 64;

/// Minimum serialized figure size (bytes) before a map/geo panel's figure JSON is embedded as a
/// gzip+base64 payload instead of plain JSON. See `map_panel_inline_html`.
const MAP_FIG_GZ_MIN_BYTES: usize = 64 * 1024;

/// Re-encode one numeric JSON array as a plotly base64 typed array
/// (`{"dtype":"float32","bdata":"<base64 little-endian bytes>"}`), decoded natively by the
/// bundled plotly.js (3.x, since 3.0). float32 keeps ~7 significant digits — for lat/lon a
/// worst-case error of ~1 m, invisible at any map zoom (the hover template shows 4-dp ≈ 11 m
/// coordinates anyway) — at ~5.33 bytes/value versus ~17 for full-precision decimal text.
/// Leaves the value untouched when it isn't a numeric array or is too short to profit.
fn bdata_encode_f32(value: &mut serde_json::Value) {
    use base64_simd::STANDARD as BASE64;
    let Some(arr) = value.as_array() else {
        return;
    };
    if arr.len() < BDATA_MIN_LEN {
        return;
    }
    let mut bytes = Vec::with_capacity(arr.len() * 4);
    for v in arr {
        // a non-numeric entry can't ride in a typed array — keep the plain JSON form
        let Some(f) = v.as_f64() else {
            return;
        };
        bytes.extend_from_slice(&(f as f32).to_le_bytes());
    }
    *value = serde_json::json!({
        "dtype": "float32",
        "bdata": BASE64.encode_to_string(&bytes),
    });
}

/// Walk a serialized figure and convert the bulky per-point numeric arrays of its map/geo
/// traces (`lat`/`lon`, densitymap `z` weights, per-point bubble `marker.size`) to float32
/// base64 typed arrays. Only these trace families carry `MAX_SMART_POINTS`-scale arrays;
/// everything else in the figure stays human-readable.
fn bdata_encode_geo_traces(figure: &mut serde_json::Value) {
    let Some(traces) = figure
        .get_mut("data")
        .and_then(serde_json::Value::as_array_mut)
    else {
        return;
    };
    for trace in traces {
        let keys: &[&str] = match trace.get("type").and_then(serde_json::Value::as_str) {
            Some("scattermap" | "scattergeo") => &["lat", "lon"],
            Some("densitymap") => &["lat", "lon", "z"],
            _ => continue,
        };
        for key in keys {
            if let Some(v) = trace.get_mut(*key) {
                bdata_encode_f32(v);
            }
        }
        if let Some(size) = trace.get_mut("marker").and_then(|m| m.get_mut("size")) {
            bdata_encode_f32(size);
        }
    }
}

/// Inline-HTML emitter for a map/geo panel, equivalent to the plotly crate's `to_inline_html`
/// but with the figure run through `bdata_encode_geo_traces` first (the typed builder API can't
/// express base64 typed arrays, so the figure is mutated as serialized JSON). Matches the
/// crate's askama `tojson` output by escaping `<`/`>`/`&` to `\u003c`-style sequences — which
/// is what keeps a pathological `</script>` inside embedded data from terminating the script
/// block (a `\u`-escaped `<` can never appear in base64 or structural JSON). Falls back
/// to the crate's own renderer if the figure doesn't serialize.
fn map_panel_inline_html(plot: &Plot, div_id: &str) -> String {
    let Ok(mut figure) = serde_json::to_value(plot) else {
        return plot.to_inline_html(Some(div_id));
    };
    let compress = viz_compress();
    // QSV_VIZ_NO_COMPRESS means "fully readable output": skip the typed-array re-encoding too,
    // so every coordinate/size stays a greppable decimal in the emitted JSON.
    if compress {
        bdata_encode_geo_traces(&mut figure);
    }
    let json_plain = figure.to_string();
    // compressed pages carry a LARGE figure as a gzip+base64 payload that `qsvNewPlotGz` (from
    // `GZ_PRELUDE_SCRIPT`, emitted with the bundle block) inflates and renders — hover strings
    // and place names compress 3-4x. Small figures stay plain JSON: below the floor the savings
    // are noise next to the bundle, and a readable figure keeps small dashboards greppable.
    // Default gzip level: this runs per panel.
    if compress && json_plain.len() >= MAP_FIG_GZ_MIN_BYTES {
        let b64 = gzip_b64(json_plain.as_bytes(), flate2::Compression::default());
        if !b64.is_empty() {
            return format!(
                "<div id=\"{div_id}\" class=\"plotly-graph-div\" style=\"height:100%; \
                 width:100%;\"></div>\n<script id=\"{div_id}-fig\" \
                 type=\"application/gzip-b64\">{b64}</script>\n<script \
                 type=\"text/javascript\">qsvNewPlotGz(\"{div_id}\");</script>"
            );
        }
    }
    let json = json_plain
        .replace('&', "\\u0026")
        .replace('<', "\\u003c")
        .replace('>', "\\u003e");
    format!(
        "<div id=\"{div_id}\" class=\"plotly-graph-div\" style=\"height:100%; \
         width:100%;\"></div>\n<script type=\"text/javascript\">\n    \
         Plotly.newPlot(\"{div_id}\", {json});\n</script>"
    )
}

fn render_smart_inline(
    args: &Args,
    panels: &[Panel],
    freq: &FreqMap,
    hist: &HashMap<usize, Vec<f64>>,
    outliers: &HashMap<usize, OutlierStats>,
    title_text: &str,
    log_scale: LogScale,
    dict_page: Option<&str>,
    meta_table: Option<&str>,
    data_chrome: Option<&str>,
    // The resolved (latitude, longitude) CSV column indexes behind the map panel. They index the
    // data viewer's rows directly — `collect_datatable_rows` emits every CSV column in file order
    // — which is how the bridge reads the coordinates of a row that has no plotted point of its
    // own (downsampled away) in order to pin it.
    map_cols: Option<(usize, usize)>,
) -> String {
    let cols = args.flag_grid_cols.clamp(1, panels.len().max(1));
    let theme = args.theme();

    // raw strings (actual newlines, real quotes) throughout so rustfmt's `format_strings` can't
    // split a `\n`/`\"` escape across a line wrap and corrupt the markup (see `smart_html_page`).
    let mut cells = String::new();
    for (n, panel) in panels.iter().enumerate() {
        let color = PALETTE[n % PALETTE.len()];
        let plot = smart_inline_panel_plot(panel, color, freq, hist, outliers, theme, log_scale);
        let div_id = format!("qsv-viz-panel-{n}");
        // leading overview panels span the full page width (their own grid row). Under
        // `--dict-info` the cell carries its dictionary anchor as `data-qsv-dict` — the jump
        // target for the dictionary page's "View chart" reverse links (the info icon itself
        // rides on the panel's plotly title annotation, see `inline_panel_title`).
        let fw = if is_overview_panel(&panel.kind) {
            " full-width"
        } else {
            ""
        };
        match &panel.dict_info {
            Some((anchor, _)) => cells.push_str(&format!(
                "    <div class=\"qsv-viz-cell{fw}\" data-qsv-dict=\"{anchor}\">\n"
            )),
            None => cells.push_str(&format!("    <div class=\"qsv-viz-cell{fw}\">\n")),
        }
        // wrap the plot in a fixed-height box so plotly's responsive `height:100%` div resolves to
        // a concrete height instead of expanding to fill the whole cell (which would otherwise
        // overlap/clip any caption rendered below it).
        let plot_height = panel_render_height(&panel.kind, panel.axis_log);
        cells.push_str(&format!(
            r#"      <div class="qsv-viz-plot" style="height:{plot_height}px">
"#
        ));
        // map/geo panels route through the JSON pipeline so their per-point coordinate arrays
        // embed as float32 base64 typed arrays (~3x smaller); other panels keep the crate's
        // renderer. Static-export paths serialize the typed `Plot` and are untouched by this.
        if matches!(panel.kind, PanelKind::Map { .. } | PanelKind::Geo { .. }) {
            cells.push_str(&map_panel_inline_html(&plot, &div_id));
        } else {
            cells.push_str(&plot.to_inline_html(Some(&div_id)));
        }
        cells.push_str("\n      </div>\n");
        // reverse-geocoded spatial-extent summary caption, shown below a map panel.
        #[cfg(feature = "geocode")]
        if let Some(meta) = &panel.geo_meta
            && !meta.summary.is_empty()
        {
            // escape the COMPOSED caption, not just the summary: that keeps the
            // escaping exactly-once (the English prefix has no metacharacters, so
            // this stays byte-identical) while also covering a translation that
            // legitimately contains `&`.
            cells.push_str(&format!(
                r#"      <div class="qsv-viz-geo-meta">{}</div>
"#,
                html_escape(&t!("viz.map.geo_extent_caption", q_summary = meta.summary))
            ));
        }
        cells.push_str("    </div>\n");
    }

    let extra_style = format!(
        r"  .qsv-viz-grid {{ display: grid; grid-template-columns: repeat({cols}, minmax(0, 1fr)); gap: 16px; }}
  .qsv-viz-cell {{ min-width: 0; position: relative; }}
  .qsv-viz-cell.full-width {{ grid-column: 1 / -1; }}
  .qsv-viz-plot {{ width: 100%; }}"
    );
    let body = format!(
        r#"<div class="qsv-viz-grid">
{cells}</div>"#
    );
    // Inject the lightbox chrome only when a map panel actually embedded per-point image URLs —
    // not merely because `--photos` was passed. A dataset with no image column (or no lat/lon)
    // then still produces a Data Schematic with no off-origin reference anywhere in it.
    let has_photos = panels.iter().any(|p| {
        matches!(&p.kind, PanelKind::Map { photos, outlier_photos, .. }
            if !photos.is_empty() || !outlier_photos.is_empty())
    });
    // Only the two MapLibre panel kinds pull OpenStreetMap/CARTO tiles at view time. `Geo`,
    // `Choropleth` and `AnimatedGeo` render on plotly's own `geo` subplot (natural-earth /
    // albers-usa projections) with no tile server involved, so they earn no basemap attribution.
    let has_basemap = panels.iter().any(|p| {
        matches!(
            &p.kind,
            PanelKind::Map { .. } | PanelKind::ChoroplethMap { .. }
        )
    });
    // Whether any point panel carries the per-point row ordinals the rows<->map bridge joins on.
    // Both panel kinds can have them; `row_ids` is empty when the cross-link was gated off.
    let has_map_select = panels.iter().any(|p| {
        matches!(&p.kind, PanelKind::Map { row_ids, outlier_row_ids, .. }
            | PanelKind::Geo { row_ids, outlier_row_ids, .. }
            if !row_ids.is_empty() || !outlier_row_ids.is_empty())
    });
    // the bridge needs the coordinate columns too, so it can pin a selected row that has no
    // plotted point; without them there is nothing to emit it for
    let map_select = if has_map_select { map_cols } else { None };
    // region-click -> drawer filter: a summary choropleth panel carries the region column plus
    // the feature-id -> raw-spelling map (`None` on the geocode-derived and PIP paths, where no
    // CSV column holds region values). Both summary panels share one column, so first wins.
    let choro_filter = panels.iter().find_map(|p| match &p.kind {
        PanelKind::Choropleth {
            region_filter: Some((c, m)),
            ..
        }
        | PanelKind::ChoroplethMap {
            region_filter: Some((c, m)),
            ..
        } => Some((*c, m)),
        _ => None,
    });
    // panels carry no overall title, so the Data Schematic title is shown as the page <h1>.
    smart_html_page(
        title_text,
        theme,
        &extra_style,
        &body,
        true,
        dict_page,
        meta_table,
        has_photos,
        data_chrome,
        has_basemap,
        map_select,
        choro_filter,
    )
}

/// Minimal HTML-escaping for text interpolated into the inline Data Schematic page.
fn html_escape(s: &str) -> String {
    s.replace('&', "&amp;")
        .replace('<', "&lt;")
        .replace('>', "&gt;")
        .replace('"', "&quot;")
}

/// Escape a translated string destined for HTML markup that is itself written inside a
/// **single-quoted JS string literal** — the `innerHTML` builders in `PHOTO_CHROME` and
/// `DICT_SCRIPT_TEMPLATE`.
///
/// There are really THREE nested contexts (JS string literal -> HTML parse -> attribute or text
/// node), and HTML entities neutralize all of them at once:
///
/// * `"` would close the HTML attribute,
/// * `'` would close the JS string literal — the character fr/it translations are full of
///   ("l'ensemble"), and the one `html_escape` does not cover,
/// * `\` is consumed by the JS string literal *before HTML ever sees it*, so a translation
///   containing a Windows path or a regex would silently lose it — `'C:\backup'` parses as `C:` +
///   U+0008 BACKSPACE. This is invisible in the rendered page rather than an error, which is
///   exactly why it is escaped here rather than left to reviewer vigilance.
/// * `<`/`>`/`&` could restructure the markup.
///
/// Ordering matters: `html_escape` runs first so its `&` pass cannot double-escape the `&#39;`
/// and `&#92;` entities introduced afterwards.
///
/// NOTE the difference from [`js_string_literal`]: this adds **no** surrounding quotes, because
/// the placeholder sits *inside* quotes that already exist in the template. The two helpers are
/// therefore not interchangeable — see each JS block's localizer for which convention it uses.
fn html_in_js_text(s: &str) -> String {
    html_escape(s).replace('\'', "&#39;").replace('\\', "&#92;")
}

/// Extract the first meaningful paragraph from a Markdown dataset description for
/// the top-of-Data Schematic metadata cell.
///
/// LLM-generated data dictionaries often prepend a Markdown heading (e.g. an
/// ATX `# Description` or a bold-only `**Description**` line) before the actual
/// summary. A naive `split("\n\n").next()` would surface that heading verbatim,
/// so we skip leading blank/heading-only lines and return the first block of
/// consecutive content lines, joined with spaces (equivalent to the whitespace
/// collapse an HTML table cell applies anyway). Returns an empty string when the
/// description has no content lines.
fn first_description_paragraph(desc: &str) -> String {
    let mut para: Vec<&str> = Vec::new();
    for line in desc.lines() {
        let trimmed = line.trim();
        if para.is_empty() {
            // still skipping leading blank/heading-only lines
            if trimmed.is_empty() || is_markdown_heading_line(trimmed) {
                continue;
            }
            para.push(trimmed);
        } else if trimmed.is_empty() {
            // blank line ends the first paragraph
            break;
        } else {
            para.push(trimmed);
        }
    }
    para.join(" ")
}

/// Whether a single (already-trimmed) line is a Markdown heading that should be
/// skipped when picking the first description paragraph.
///
/// Recognizes an ATX heading (`#`, `##`, …) or a *heading-like* emphasized line
/// (`**Description**`, `__Summary__`, `*Notable Characteristics*`, …). LLM data
/// dictionaries label sections with bold text rather than ATX `#`, so emphasis
/// must count — but only when the emphasized text reads like a heading label,
/// not a sentence. A heading label is terse and has no sentence-terminating
/// punctuation, so a fully-bold *summary sentence* (e.g.
/// `**This dataset contains 50,013 records.**`) is preserved rather than dropped.
/// Setext underlines and horizontal rules are intentionally not treated as
/// headings to avoid false positives.
fn is_markdown_heading_line(line: &str) -> bool {
    if line.starts_with('#') {
        return true;
    }
    // emphasis-only line: wholly wrapped in a single emphasis marker with no
    // further occurrence of that marker inside. Check the two-char markers first
    // so `**Heading**` matches on `**` rather than the `*` fallback.
    for marker in ["**", "__", "*", "_"] {
        if line.len() > 2 * marker.len() && line.starts_with(marker) && line.ends_with(marker) {
            let inner = line[marker.len()..line.len() - marker.len()].trim();
            if !inner.is_empty() && !inner.contains(marker) {
                return is_heading_like_label(inner);
            }
        }
    }
    false
}

/// Whether `label` (the inner text of an emphasized line) reads like a section
/// heading rather than a prose sentence: no sentence-terminating punctuation and
/// only a handful of words. Keeps `Description`/`Notable Characteristics` as
/// headings while rejecting a fully-emphasized summary sentence.
fn is_heading_like_label(label: &str) -> bool {
    if matches!(label.chars().last(), Some('.' | '!' | '?')) {
        return false;
    }
    label.split_whitespace().count() <= 5
}

/// Try to satisfy the bar-panel frequency counts from a pre-existing `frequency`
/// JSONL cache (`qsv frequency --frequency-jsonl`), avoiding the extra full-data
/// pass `count_values` does. Returns `None` — so the caller falls back to
/// `count_values` — when no compatible cache exists or any requested bar column
/// is absent from it (e.g. a high-cardinality/all-unique sentinel column).
///
/// On a hit, each column's non-null pairs are re-sorted (count desc, then value asc),
/// truncated to `top_n`, and given the same aggregate `(NULL)` / `Other (N)` buckets
/// (via `finalize_freq_bars`) as the `count_values` path, so cached and recomputed bars
/// are identical. The cache stores complete per-value data including the empty/null bucket,
/// which `read_frequency_cache_view` surfaces as `FreqCacheColumn::null_count`. (The cache is
/// always whitespace-trimmed — `--frequency-jsonl` is incompatible with `--no-trim` — matching
/// the trim `count_values` applies.)
fn freq_from_cache(
    args: &Args,
    stats: &[crate::cmd::stats::StatsData],
    bar_indices: &[usize],
    top_n: usize,
) -> Option<FreqMap> {
    let path = args.arg_input.as_ref()?;
    let rconfig = Config::new(Some(path))
        .delimiter(args.flag_delimiter)
        .no_headers_flag(args.flag_no_headers);
    let no_headers = rconfig.no_headers;

    // `viz` never sets --no-nulls on the underlying cache read, so it can only reuse a
    // default (nulls-kept) cache; the null bucket is then surfaced as the "(NULL)" bar
    // (unless the user passed --no-nulls, which `finalize_freq_bars` honors).
    let view = crate::cmd::frequency::read_frequency_cache_view(
        std::path::Path::new(path),
        false,
        no_headers,
        args.flag_delimiter,
    )?;

    let mut out = HashMap::with_capacity(bar_indices.len());
    for &idx in bar_indices {
        // In --no-headers mode the cache keys columns positionally ("1", "2", …),
        // matching how `frequency` (and stats) name them; otherwise use the field
        // name. Any column missing from the cache abandons the fast path entirely
        // (return None) so output never silently mixes cached and recomputed bars.
        let key = if no_headers {
            (idx + 1).to_string()
        } else {
            stats.get(idx)?.field.clone()
        };
        let col = view.columns.get(&key)?;
        let mut pairs = col.pairs.clone();
        pairs.sort_by(|a, b| b.1.cmp(&a.1).then_with(|| a.0.cmp(&b.0)));
        out.insert(
            idx,
            finalize_freq_bars(
                pairs,
                col.null_count,
                top_n,
                args.flag_no_nulls,
                args.flag_no_other,
            ),
        );
    }
    Some(out)
}

/// Per-bar pattern (hatch) shapes for a frequency panel, or `None` when no hatching applies.
/// On a log y-axis, the dominant `(NULL)`/`Other` aggregate bars are diagonally hatched as a
/// redundant cue that the axis is non-linear; real categories stay flat-filled. Returns `None`
/// on linear panels so the marker is built exactly as before. Hatching is keyed on the bar's
/// `kind` (not its label), so a real category literally named "Other" is never hatched.
fn freq_bar_pattern_shapes(bars: &[FreqBar], log_y: bool) -> Option<Vec<PatternShape>> {
    if !log_y {
        return None;
    }
    Some(
        bars.iter()
            .map(|b| match b.kind {
                FreqBarKind::Aggregate => PatternShape::RightDiagonalLine,
                FreqBarKind::Category => PatternShape::None,
            })
            .collect(),
    )
}

/// Body styling cue for a box/violin distribution panel drawn on a log value axis:
/// `Some((fill_color, outline_color))` = a faint grey wash + grey outline, the redundant
/// "this axis is log" signal analogous to the freq bar's hatch (`freq_bar_pattern_shapes`).
/// Plotly's box/violin traces support neither a fill pattern (hatch) nor a dashed line — only
/// `fillcolor`, `line.color`, and `line.width` render on the body — so a muted fill + outline
/// is the strongest cue available on these trace types. Returns `None` on a linear axis, so the
/// panel keeps its own palette color and the trace is built exactly as before.
const fn log_distribution_body_cue(log_y: bool) -> Option<(&'static str, &'static str)> {
    if log_y {
        Some((LOG_DISTRIB_FILL, MUTED_COLOR))
    } else {
        None
    }
}

/// Build the "(dominated by X, NN%)" title hint for a frequency bar panel whose single most
/// frequent REAL category holds at least `DOMINANT_MIN_SHARE` of all rows — the one-tall-bar
/// case, worth saying in the title so the panel reads at a glance.
///
/// The share is EXACT on both sides: the numerator is the top real bar's frequency count, and
/// the denominator is the dataset's true row count (`row_count`, fetched lazily — see below).
/// Deliberately NOT derived from the rendered bar total (`--no-nulls`/`--no-other` remove
/// aggregate bars, shrinking that denominator and inflating an 85%-of-rows category to a false
/// "100%") nor from the stats cache's `mode`/`uniqueness_ratio` (the mode tracker includes
/// empty cells, so a NULL-heavy column's mode is the empty bucket; and the cached ratio is
/// rounded to 4 decimals, which materially undercounts rows on large low-cardinality columns).
///
/// `row_count` is a lazy thunk: the drawn bar total is a LOWER bound on the true row count, so
/// when the top category can't clear the threshold even against the drawn bars it can't clear
/// it against the (>=) true count either, and the row count is never fetched.
fn dominant_category_hint(bars: &[FreqBar], row_count: impl FnOnce() -> u64) -> Option<String> {
    const DOMINANT_MIN_SHARE: f64 = 0.90;
    let drawn_total: u64 = bars.iter().map(|b| b.count).sum();
    if drawn_total == 0 {
        return None;
    }
    let top = bars
        .iter()
        .filter(|b| b.kind == FreqBarKind::Category)
        .max_by_key(|b| b.count)?;
    #[allow(clippy::cast_precision_loss)]
    if (top.count as f64) < DOMINANT_MIN_SHARE * drawn_total as f64 {
        return None;
    }
    let total = row_count();
    if total == 0 {
        return None;
    }
    #[allow(clippy::cast_precision_loss)]
    let share = top.count as f64 / total as f64;
    (share >= DOMINANT_MIN_SHARE).then(|| {
        t!(
            "viz.title.dominated_by",
            q_value = truncate_label(&top.label, BAR_LABEL_MAX_CHARS),
            q_pct = format!("{:.0}", share * 100.0)
        )
        .into_owned()
    })
}

/// Turn a column's non-null `(value, count)` pairs into the panel's frequency bars, appending
/// the aggregate `(NULL)` and `Other (N)` buckets to match `qsv frequency`'s default output.
/// `counts` is the full set of non-null pairs (already sorted: count desc, then value asc);
/// `null_count` is the number of empty cells. The top `top_n` non-null categories are kept,
/// and the rest are summarized as a single `Other (N)` bar where N = the count of distinct
/// categories dropped. Both aggregate buckets are appended AFTER the real categories (`(NULL)`
/// first, then `Other`) and can be suppressed with `no_nulls` / `no_other`.
///
/// Exception: when exactly ONE category is left over, it is charted as itself rather than as an
/// `Other (1)` bar. The bar occupies the same slot and has the same height either way, so naming
/// it is strictly more informative. `no_other` still wins — it means "no bar beyond `top_n`".
fn finalize_freq_bars(
    counts: Vec<(String, u64)>,
    null_count: u64,
    top_n: usize,
    no_nulls: bool,
    no_other: bool,
) -> Vec<FreqBar> {
    let keep = if !no_other && counts.len() == top_n + 1 {
        counts.len()
    } else {
        top_n
    };
    let other_unique = counts.len().saturating_sub(keep);
    let other_count: u64 = counts.iter().skip(keep).map(|(_, c)| *c).sum();
    let mut bars: Vec<FreqBar> = counts
        .into_iter()
        .take(keep)
        .map(|(label, count)| FreqBar {
            x_key: label.clone(),
            label,
            count,
            kind: FreqBarKind::Category,
        })
        .collect();

    // derive the aggregate bars' axis keys against the real category keys (and each other) so
    // they're provably unique — see `push_aggregate_bar`.
    let mut used: std::collections::HashSet<String> =
        bars.iter().map(|b| b.x_key.clone()).collect();
    if !no_nulls && null_count > 0 {
        push_aggregate_bar(&mut bars, &mut used, null_text(), null_count);
    }
    if !no_other && other_count > 0 {
        push_aggregate_bar(
            &mut bars,
            &mut used,
            other_text(other_unique as u64),
            other_count,
        );
    }
    bars
}

/// Append a synthetic aggregate frequency bar (a `(NULL)` or `Other (N)` bucket) to `bars`.
/// Its category-axis key is derived from the display label plus an `AGG_KEY_SENTINEL` suffix,
/// extended (sentinel repeated) until it is provably distinct from every key already in `used`
/// — the real category keys plus any earlier aggregate. This guarantees the aggregate can never
/// collapse onto a real category's bar on the plotly axis, even if a real value happens to equal
/// the sentinel-suffixed form. `used` is updated with the chosen key.
fn push_aggregate_bar(
    bars: &mut Vec<FreqBar>,
    used: &mut std::collections::HashSet<String>,
    label: String,
    count: u64,
) {
    let mut x_key = format!("{label}{AGG_KEY_SENTINEL}");
    while used.contains(&x_key) {
        x_key.push(AGG_KEY_SENTINEL);
    }
    used.insert(x_key.clone());
    bars.push(FreqBar {
        x_key,
        label,
        count,
        kind: FreqBarKind::Aggregate,
    });
}

/// Count value occurrences for the given column indices in a single pass, returning the
/// top-N `(value, count)` pairs per column (sorted by count desc, then value asc), plus the
/// aggregate `(NULL)` / `Other (N)` buckets appended by `finalize_freq_bars` (honoring
/// `--no-nulls` / `--no-other`). Values are ASCII-whitespace-trimmed before counting, matching
/// `qsv frequency`'s default, so whitespace-only cells count as NULL and the raw-scan path
/// stays consistent with the always-trimmed frequency cache.
fn count_values(args: &Args, indices: &[usize], top_n: usize) -> CliResult<FreqMap> {
    let rconfig = Config::new(args.arg_input.as_ref())
        .delimiter(args.flag_delimiter)
        .no_headers_flag(args.flag_no_headers);
    let mut rdr = rconfig.reader()?;

    let mut maps: HashMap<usize, HashMap<Vec<u8>, u64>> =
        indices.iter().map(|&i| (i, HashMap::new())).collect();
    // empty cells are tallied here (not in `maps`, so they're never labeled "") and surfaced
    // as a "(NULL)" bar by `finalize_freq_bars`.
    let mut null_counts: HashMap<usize, u64> = indices.iter().map(|&i| (i, 0_u64)).collect();

    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        for &i in indices {
            if let Some(cell) = record.get(i) {
                // trim ASCII whitespace before the empty/null check (and before counting),
                // matching `qsv frequency`'s default trim — otherwise whitespace-only cells
                // would become a literal blank category here but "(NULL)" from the cache.
                let cell = crate::cmd::frequency::trim_bs_whitespace(cell);
                if cell.is_empty() {
                    if let Some(n) = null_counts.get_mut(&i) {
                        *n += 1;
                    }
                    continue;
                }
                if let Some(m) = maps.get_mut(&i) {
                    // hot inner loop of the no-frequency-cache fallback: probe before
                    // allocating, else every counted cell heap-allocates a key that is
                    // immediately dropped.
                    if let Some(c) = m.get_mut(cell) {
                        *c += 1;
                    } else {
                        m.insert(cell.to_vec(), 1);
                    }
                }
            }
        }
    }

    let mut out = HashMap::with_capacity(maps.len());
    for (i, m) in maps {
        let mut counts: Vec<(String, u64)> = m
            .into_iter()
            .map(|(k, c)| (String::from_utf8_lossy(&k).into_owned(), c))
            .collect();
        counts.sort_by(|a, b| b.1.cmp(&a.1).then_with(|| a.0.cmp(&b.0)));
        let null_count = null_counts.get(&i).copied().unwrap_or(0);
        out.insert(
            i,
            finalize_freq_bars(
                counts,
                null_count,
                top_n,
                args.flag_no_nulls,
                args.flag_no_other,
            ),
        );
    }
    Ok(out)
}

/// Single-pass accumulation of leaf-path values for a categorical hierarchy (treemap/sunburst).
///
/// For each row, the `dims` columns (outer level first) are read, ASCII-whitespace-trimmed, and
/// empty cells mapped to `(NULL)` — matching the freq-bar panels — to form a `dims.len()`-segment
/// path. The path's accumulator is incremented by `1.0` when `value_idx` is `None` (count of rows)
/// or by the parsed numeric `value_idx` cell otherwise (sum; unparseable/empty cells contribute 0).
/// Returns a map of full-depth path → aggregated value, which `hierarchy_arrays` turns into the
/// flat plotly arrays.
///
/// Hierarchy values must be ADDITIVE (count or sum) so a parent equals the sum of its children
/// under plotly's `branchvalues="total"`; mean/min/max don't roll up a tree and are rejected by the
/// caller. Memory is bounded by the product of the `dims` cardinalities — safe for `viz smart`
/// (each dim ≤ `CATEGORICAL_MAX_CARDINALITY`); the standalone subcommands trust the user's column
/// choice.
fn collect_hierarchy_counts(
    args: &Args,
    dims: &[usize],
    value_idx: Option<usize>,
) -> CliResult<HashMap<Vec<String>, f64>> {
    let rconfig = Config::new(args.arg_input.as_ref())
        .delimiter(args.flag_delimiter)
        .no_headers_flag(args.flag_no_headers);
    let mut rdr = rconfig.reader()?;
    accumulate_hierarchy_counts(&mut rdr, dims, value_idx)
}

/// Accumulate leaf-path values from an already-open reader (positioned past the header). Split out
/// from `collect_hierarchy_counts` so the standalone `viz treemap`/`viz sunburst` subcommands can
/// reuse the single reader from `reader_and_headers` (the header is needed first to resolve
/// `--cols`/`--value`), reading streamed stdin exactly once.
fn accumulate_hierarchy_counts(
    rdr: &mut csv::Reader<Box<dyn std::io::Read + Send>>,
    dims: &[usize],
    value_idx: Option<usize>,
) -> CliResult<HashMap<Vec<String>, f64>> {
    let mut leaves: HashMap<Vec<String>, f64> = HashMap::new();
    // value mode (--value): track usable vs unusable measure cells so a typo'd / non-numeric
    // measure column fails loudly instead of silently producing a blank or misleading chart.
    let mut valid_values = 0_usize;
    let mut invalid_values = 0_usize;
    let mut record = csv::ByteRecord::new();
    // reused across rows: the path is only cloned into the map on a genuine MISS, so a
    // low-cardinality hierarchy (the common case) stops allocating almost entirely after the
    // first few rows instead of building dims.len() Strings for every row.
    let mut path: Vec<String> = Vec::with_capacity(dims.len());
    while rdr.read_byte_record(&mut record)? {
        path.clear();
        for &d in dims {
            let seg = match record.get(d) {
                Some(cell) => {
                    let cell = crate::cmd::frequency::trim_bs_whitespace(cell);
                    if cell.is_empty() {
                        null_text()
                    } else {
                        String::from_utf8_lossy(cell).into_owned()
                    }
                },
                None => null_text(),
            };
            path.push(seg);
        }
        match value_idx {
            // count mode: every row contributes 1 to its leaf.
            None => {
                if let Some(v) = leaves.get_mut(path.as_slice()) {
                    *v += 1.0;
                } else {
                    leaves.insert(path.clone(), 1.0);
                }
            },
            // value mode: sum a finite, non-negative measure. An empty cell is a benign missing
            // measure (skipped, no leaf created); a non-numeric / negative / non-finite cell can't
            // size an area/angle, so it's tallied and the pass errors afterwards. This avoids
            // silently dropping rows (which would misstate every part-to-whole proportion) or
            // coercing bad data to 0 and rendering a blank/misleading sector.
            Some(vi) => {
                let cell = record
                    .get(vi)
                    .map(crate::cmd::frequency::trim_bs_whitespace)
                    .unwrap_or_default();
                if cell.is_empty() {
                    continue;
                }
                match std::str::from_utf8(cell)
                    .ok()
                    .and_then(|s| s.parse::<f64>().ok())
                {
                    Some(v) if v.is_finite() && v >= 0.0 => {
                        valid_values += 1;
                        if let Some(acc) = leaves.get_mut(path.as_slice()) {
                            *acc += v;
                        } else {
                            leaves.insert(path.clone(), v);
                        }
                    },
                    _ => invalid_values += 1,
                }
            },
        }
    }
    if value_idx.is_some() {
        if invalid_values > 0 {
            // A part-to-whole chart that silently dropped these rows would misstate every
            // proportion, so any unusable measure cell is a hard error (not a warning).
            return fail_clierror!(
                "the --value column has {invalid_values} cell(s) that are non-numeric, negative, \
                 or non-finite; a treemap/sunburst would drop them and misstate the totals. Clean \
                 or filter the data first."
            );
        }
        if valid_values == 0 {
            return fail_clierror!(
                "the --value column has no usable values (need finite, non-negative numbers) — \
                 cannot size the chart."
            );
        }
    }
    Ok(leaves)
}

/// Maximum bias-corrected (Bergsma 2013) Cramér's V across every pair of the `ndims` hierarchy
/// dimensions, computed by marginalizing the joint leaf counts to each 2-way contingency table — so
/// it needs NO extra data pass (the counts are already in hand). `viz smart` uses this to avoid
/// auto-nesting statistically independent dimensions: when no pair of levels is associated, a
/// treemap/sunburst merely repeats each level's marginal at every branch and a set of separate bars
/// says the same thing far more legibly.
///
/// The Bergsma bias correction matters here precisely because the joint table can be sparse (the
/// product of cardinalities): the naive V is upward-biased for large, sparse tables, so the
/// correction subtracts the expected-under-independence inflation and shrinks the effective table
/// size. When the correction is undefined for a tiny table (n close to the table size), the pair
/// falls back to the uncorrected Cramér's V so a small-but-associated hierarchy isn't wrongly read
/// as independent. Returns 0.0 only when there are fewer than two dimensions or no pair has a
/// 2x2-or-larger table with n > 1.
fn max_pairwise_cramers_v(leaves: &HashMap<Vec<String>, f64>, ndims: usize) -> f64 {
    if ndims < 2 {
        return 0.0;
    }
    let mut max_v = 0.0_f64;
    for a in 0..ndims {
        for b in (a + 1)..ndims {
            // marginalize the joint counts down to the (a, b) contingency table.
            let mut cell: HashMap<(&str, &str), f64> = HashMap::new();
            let mut row_tot: HashMap<&str, f64> = HashMap::new();
            let mut col_tot: HashMap<&str, f64> = HashMap::new();
            let mut n = 0.0_f64;
            for (path, &v) in leaves {
                if path.len() != ndims {
                    continue; // defensive: ignore malformed paths
                }
                let ra = path[a].as_str();
                let cb = path[b].as_str();
                *cell.entry((ra, cb)).or_insert(0.0) += v;
                *row_tot.entry(ra).or_insert(0.0) += v;
                *col_tot.entry(cb).or_insert(0.0) += v;
                n += v;
            }
            let (r, c) = (row_tot.len(), col_tot.len());
            // need a 2x2 table and at least as many observations as df+1 for a defined corrected V.
            if r < 2 || c < 2 || n <= 1.0 {
                continue;
            }
            // Pearson chi-square — summed over ALL r*c cells, including structural zeros (an
            // observed-0 cell still contributes (0 - e)^2 / e = e), so iterate the label cross
            // product rather than just the observed cells.
            let mut chi2 = 0.0_f64;
            for (&ra, &rt) in &row_tot {
                for (&cb, &ct) in &col_tot {
                    let e = rt * ct / n;
                    if e > 0.0 {
                        let o = cell.get(&(ra, cb)).copied().unwrap_or(0.0);
                        let d = o - e;
                        chi2 += d * d / e;
                    }
                }
            }
            // Bergsma (2013) bias-corrected Cramér's V.
            #[allow(clippy::cast_precision_loss)]
            let (rf, cf) = (r as f64, c as f64);
            let phi2 = chi2 / n;
            let phi2_corr = (phi2 - (rf - 1.0) * (cf - 1.0) / (n - 1.0)).max(0.0);
            let r_corr = rf - (rf - 1.0) * (rf - 1.0) / (n - 1.0);
            let c_corr = cf - (cf - 1.0) * (cf - 1.0) / (n - 1.0);
            let denom = (r_corr - 1.0).min(c_corr - 1.0);
            let v = if denom > 0.0 {
                (phi2_corr / denom).sqrt()
            } else {
                // The bias correction is undefined for a tiny table (n close to the table size
                // shrinks the effective dimensions to <= 1). Falling through here would leave a
                // perfectly-associated small hierarchy reading as V=0 (independent) and wrongly
                // skip it, so fall back to the UNCORRECTED Cramér's V, which is
                // always defined for r,c >= 2 and n > 1 (denom_raw >= 1.0).
                let denom_raw = (rf - 1.0).min(cf - 1.0);
                (phi2 / denom_raw).sqrt()
            };
            max_v = max_v.max(v);
        }
    }
    max_v
}

/// Turn a map of full-depth `leaf path → value` into the flat
/// `(labels, parents, values, ids)` arrays plotly's `Treemap`/`Sunburst` consume.
///
/// A synthetic root (`HIER_ROOT_ID`, label `root_label`, value = grand total) parents the level-1
/// categories. Subtree totals are rolled up to every ancestor (so parent = sum of children, valid
/// under `branchvalues="total"`). Per parent, only the top `top_n` children (value desc, then label
/// asc — same comparator as `count_values`) are kept; any remainder collapses into a single
/// `Other (k)` leaf so the kept children plus `Other` still sum to the parent. `ids` are
/// path-joined with `HIER_PATH_SEP`, so the same child label under two different parents never
/// collides (plotly keys `parents` on `ids`). Expansion stops once `max_nodes` nodes are emitted
/// (capped nodes render as leaves). Returns `None` for degenerate input (no real split anywhere).
fn hierarchy_arrays(
    leaves: &HashMap<Vec<String>, f64>,
    depth: usize,
    top_n: usize,
    max_nodes: usize,
    root_label: &str,
) -> Option<(Vec<String>, Vec<String>, Vec<f64>, Vec<String>)> {
    if leaves.is_empty() || depth == 0 {
        return None;
    }

    // tree[parent_prefix][child_segment] = subtree total under that child.
    let mut tree: HashMap<Vec<String>, HashMap<String, f64>> = HashMap::new();
    // Roll up in a DETERMINISTIC order. Iterating `leaves` (a HashMap) directly makes the float
    // accumulation order vary between processes — Rust seeds HashMap's hasher randomly — so the
    // same input produced subtly different parent sums run to run (37616.58 vs 37616.579999999994
    // for the same treemap). Harmless numerically, but it means every regeneration of a committed
    // artifact churns, and it makes a chart's own output unreproducible.
    let mut ordered: Vec<(&Vec<String>, f64)> = leaves.iter().map(|(p, &v)| (p, v)).collect();
    ordered.sort_unstable_by(|a, b| a.0.cmp(b.0));
    for (path, v) in ordered {
        if path.len() != depth {
            continue; // defensive: ignore malformed paths
        }
        for l in 0..path.len() {
            *tree
                .entry(path[..l].to_vec())
                .or_default()
                .entry(path[l].clone())
                .or_insert(0.0) += v;
        }
    }

    // a meaningful breakdown needs at least one node that actually splits into 2+ children.
    if !tree.values().any(|children| children.len() >= 2) {
        return None;
    }

    let root_children = tree.get(&Vec::<String>::new())?;
    let total: f64 = root_children.values().sum();

    let mut labels = vec![root_label.to_string()];
    let mut parents = vec![String::new()];
    let mut values = vec![total];
    let mut ids = vec![HIER_ROOT_ID.to_string()];
    let mut node_count = 1_usize;

    // plotly keys `parents` on `ids`, so ids must be GLOBALLY unique. The naive
    // `prefix_id + SEP + seg` join isn't collision-proof against adversarial segment values: a
    // category value containing a literal HIER_PATH_SEP (U+001F is legal in a CSV field) makes
    // paths like ["A\u{1F}B"] and ["A","B"] join identically, and a real child value equal to
    // "<sentinel>other" collides with its parent's Other-bucket id. Disambiguate with a
    // used-set + sentinel loop (same approach as `push_aggregate_bar`); children are queued with
    // their RESOLVED id, so `parents` always references the final unique id.
    let mut used_ids: std::collections::HashSet<String> =
        std::collections::HashSet::from([HIER_ROOT_ID.to_string()]);
    let mut unique_id = |base: String| -> String {
        let mut candidate = base;
        while !used_ids.insert(candidate.clone()) {
            candidate.push(AGG_KEY_SENTINEL);
        }
        candidate
    };

    let mut queue: std::collections::VecDeque<(Vec<String>, String, usize)> =
        std::collections::VecDeque::new();
    queue.push_back((Vec::new(), HIER_ROOT_ID.to_string(), 0));

    while let Some((prefix, prefix_id, level)) = queue.pop_front() {
        if level >= depth || node_count >= max_nodes {
            continue; // leaf level reached, or global cap hit → leave as a leaf
        }
        let Some(children) = tree.get(&prefix) else {
            continue;
        };

        let mut ranked: Vec<(&String, f64)> = children.iter().map(|(k, &v)| (k, v)).collect();
        ranked.sort_by(|a, b| {
            b.1.partial_cmp(&a.1)
                .unwrap_or(std::cmp::Ordering::Equal)
                .then_with(|| a.0.cmp(b.0))
        });

        // when exactly one child would be dropped, keep it instead: it occupies the same tile as
        // the `Other (1)` bucket would, and naming it is strictly more informative (mirrors
        // `finalize_freq_bars`). It expands like any other kept child.
        let keep = if ranked.len() == top_n + 1 {
            ranked.len()
        } else {
            top_n.min(ranked.len())
        };
        let mut other_value = 0.0;
        let mut other_drops = 0_usize;
        for (i, (seg, v)) in ranked.into_iter().enumerate() {
            if i < keep {
                let child_id = unique_id(format!("{prefix_id}{HIER_PATH_SEP}{seg}"));
                let mut child_path = prefix.clone();
                child_path.push(seg.clone());
                labels.push(seg.clone());
                parents.push(prefix_id.clone());
                values.push(v);
                ids.push(child_id.clone());
                node_count += 1;
                queue.push_back((child_path, child_id, level + 1));
            } else {
                other_value += v;
                other_drops += 1;
            }
        }
        // always emit the Other bucket when dropping, so kept children + Other == parent
        // (keeps `branchvalues="total"` consistent; otherwise plotly shows a phantom gap).
        if other_drops > 0 {
            labels.push(other_text(other_drops as u64));
            parents.push(prefix_id.clone());
            values.push(other_value);
            ids.push(unique_id(format!(
                "{prefix_id}{HIER_PATH_SEP}{AGG_KEY_SENTINEL}other"
            )));
            node_count += 1;
        }
    }

    // root + at least two real nodes, else it's not worth a panel.
    if node_count < 3 {
        return None;
    }
    Some((labels, parents, values, ids))
}

/// Construct the plotly domain-based trace for a hierarchy panel/chart from its precomputed flat
/// arrays. A treemap is sorted largest-first (best practice for size comparison via area); a
/// sunburst relies on plotly's lineage colorway for deep paths. Both use `branchvalues="total"`,
/// matching the rolled-up subtree totals `hierarchy_arrays` emits. Both label their tiles/rings
/// with the richer `label+value+percent parent`; the sunburst additionally caps the initial view to
/// two rings (see the per-branch comment below for why), with deeper levels reachable via
/// click-to-zoom.
fn hierarchy_trace(
    style: HierStyle,
    labels: &[String],
    parents: &[String],
    values: &[f64],
    ids: &[String],
) -> Box<dyn Trace> {
    match style {
        HierStyle::Treemap => Treemap::new(labels.to_vec(), parents.to_vec())
            .ids(ids.to_vec())
            .values(values.to_vec())
            .branch_values(BranchValues::Total)
            // treemap-specific marker (plotly.rs#406): rounded corners + a thin white tile
            // outline and inner padding so nested rectangles read as distinct, legible tiles.
            // NOTE: the top pad is intentionally left UNSET. plotly draws each parent's label in
            // the tile's top padding band; pinning `top` to a few px collapses that band so the
            // top-level grouping (e.g. plan: Basic/Plus/Pro) renders as bare color with no header.
            // Omitting `top` lets plotly auto-size a header band that fits the label.
            .marker(
                TreemapMarker::new()
                    .corner_radius(4.0)
                    .pad(Pad::new().left(3.0).right(3.0).bottom(3.0))
                    .line(Line::new().width(1.0).color(NamedColor::White)),
            )
            .sort(true)
            .text_info("label+value+percent parent"),
        // the sunburst's initial view is capped to two rings (the first ring is the root, the
        // second ring is the top-level categories) so it isn't overwhelming; deeper levels are
        // still accessible via click-to-zoom.
        HierStyle::Sunburst => Sunburst::new(labels.to_vec(), parents.to_vec())
            .ids(ids.to_vec())
            .values(values.to_vec())
            .branch_values(BranchValues::Total)
            .text_info("label+value+percent parent")
            // radial in-sector text (plotly.js 3.6) lets the label+value+percent run along each
            // ring's spoke, so deep-path sectors stay legible instead of clipping tangential text.
            .inside_text_orientation(InsideTextOrientation::Radial)
            .max_depth(SUNBURST_MAXDEPTH),
        // an icicle is the rectangular sibling of the sunburst: the same rolled-up subtree totals
        // laid out as stacked bars per level. We leave `tiling.orientation` at plotly's default of
        // horizontal ("h"), so the root sits on the LEFT and children fan out to the right — which
        // keeps deep paths and many small leaves more legible than a treemap's nested rectangles
        // while preserving strict level alignment for cross-branch comparison. Same
        // `label+value+percent parent` tile text.
        HierStyle::Icicle => Icicle::new(labels.to_vec(), parents.to_vec())
            .ids(ids.to_vec())
            .values(values.to_vec())
            .branch_values(IcicleBranchValues::Total)
            .text_info("label+value+percent parent"),
    }
}

/// Convert a `#RRGGBB` `PALETTE` hex string into a `rgba(r,g,b,a)` string plotly accepts, so a
/// node's solid color can tint its outgoing link ribbons at reduced opacity.
fn hex_to_rgba(hex: &str, alpha: f64) -> String {
    let h = hex.trim_start_matches('#');
    let r = u8::from_str_radix(h.get(0..2).unwrap_or("00"), 16).unwrap_or(0);
    let g = u8::from_str_radix(h.get(2..4).unwrap_or("00"), 16).unwrap_or(0);
    let b = u8::from_str_radix(h.get(4..6).unwrap_or("00"), 16).unwrap_or(0);
    format!("rgba({r},{g},{b},{alpha})")
}

/// Compute value-ordered node positions for a Sankey: nodes are grouped into columns by role
/// (source-only on the left, target-only on the right, both-roles in the middle) and, within each
/// column, ranked by total throughput so the largest node sits at the top (`y = 0`). Returned as
/// parallel `(x, y)` vectors indexed by node. Used with `Arrangement::Snap`, which keeps this
/// order while sizing each node by its flow — so even spacing here is fine, snap does the packing.
/// Returns `None` for an empty graph.
fn sankey_value_positions(
    n_nodes: usize,
    link_source: &[usize],
    link_target: &[usize],
    link_value: &[f64],
) -> Option<(Vec<f64>, Vec<f64>)> {
    if n_nodes == 0 {
        return None;
    }
    let mut out_sum = vec![0.0_f64; n_nodes];
    let mut in_sum = vec![0.0_f64; n_nodes];
    let mut is_source = vec![false; n_nodes];
    let mut is_target = vec![false; n_nodes];
    for ((&s, &t), &v) in link_source.iter().zip(link_target).zip(link_value) {
        out_sum[s] += v;
        is_source[s] = true;
        in_sum[t] += v;
        is_target[t] = true;
    }
    // column 0 = source-only (x≈0), 2 = target-only (x≈1), 1 = both roles (middle); a tiny inset
    // keeps the end columns off the exact plot edge so node rectangles don't clip.
    let column = |k: usize| -> u8 {
        match (is_source[k], is_target[k]) {
            (true, true) => 1,
            (false, true) => 2,
            _ => 0,
        }
    };
    let column_x = |c: u8| -> f64 {
        match c {
            2 => 0.999,
            1 => 0.5,
            _ => 0.001,
        }
    };
    let throughput = |k: usize| out_sum[k].max(in_sum[k]);
    let mut xs = vec![0.0_f64; n_nodes];
    let mut ys = vec![0.0_f64; n_nodes];
    for c in [0_u8, 1, 2] {
        let mut members: Vec<usize> = (0..n_nodes).filter(|&k| column(k) == c).collect();
        if members.is_empty() {
            continue;
        }
        // largest throughput first (top of plot); stable tiebreak on node index
        members.sort_by(|&a, &b| {
            throughput(b)
                .partial_cmp(&throughput(a))
                .unwrap_or(std::cmp::Ordering::Equal)
                .then(a.cmp(&b))
        });
        let m = members.len() as f64;
        for (rank, &k) in members.iter().enumerate() {
            xs[k] = column_x(c);
            ys[k] = (rank as f64 + 0.5) / m;
        }
    }
    Some((xs, ys))
}

/// Construct the plotly Sankey trace shared by the standalone `sankey` chart and the
/// `viz smart --bivariate` flow panel, from node labels and parallel link arrays (source/target
/// are 0-based indices into `node_labels`; value is the aggregated flow weight). Nodes are colored
/// from `PALETTE` and each link ribbon is tinted from its SOURCE node's color (at
/// `SANKEY_LINK_ALPHA`) so flows are traceable. When `value_order` is set, nodes are arranged in
/// descending-flow order per column (largest at top) via `sankey_value_positions` + snap; otherwise
/// plotly's default crossing-minimizing snap decides the vertical order.
fn sankey_trace(
    node_labels: &[String],
    link_source: &[usize],
    link_target: &[usize],
    link_value: &[f64],
    positions: Option<&(Vec<f64>, Vec<f64>)>,
) -> Box<dyn Trace> {
    let label_refs: Vec<&str> = node_labels.iter().map(String::as_str).collect();
    let node_colors: Vec<String> = (0..node_labels.len())
        .map(|i| PALETTE[i % PALETTE.len()].to_string())
        .collect();
    let link_colors: Vec<String> = link_source
        .iter()
        .map(|&s| hex_to_rgba(PALETTE[s % PALETTE.len()], SANKEY_LINK_ALPHA))
        .collect();

    let mut node = Node::new()
        .label(label_refs)
        .pad(15)
        .thickness(20)
        .color_array(node_colors);
    // `positions` present = render value-ordered (explicit per-column node x/y); absent = let snap
    // arrange. Either way `arrangement=snap` is set so the on-screen toggle can flip between the
    // two at runtime (clearing node.x/y hands the layout back to snap; re-setting them restores
    // order).
    if let Some((xs, ys)) = positions {
        node = node.x(xs.clone()).y(ys.clone());
    }
    let link = Link::new()
        .source(link_source.to_vec())
        .target(link_target.to_vec())
        .value(link_value.to_vec())
        .color_array(link_colors);
    Sankey::new()
        .node(node)
        .link(link)
        .arrangement(Arrangement::Snap)
}

/// The on-graph "node order" toggle button baked into every Sankey chart (standalone `viz sankey`
/// and the `viz smart` flow panel). A native plotly `updatemenus` button that restyles the sole
/// sankey trace (index 0): value-order sets explicit per-column node positions (`xs`/`ys` from
/// `sankey_value_positions`), snap clears them so plotly's crossing-minimizer takes over. `args`
/// fires on the FIRST click so it must be the OPPOSITE of the initial render — click 1 always flips
/// the layout, click 2 restores it. `value_order_initial` = whether the chart opens value-ordered.
fn sankey_order_toggle_menu(xs: &[f64], ys: &[f64], value_order_initial: bool) -> UpdateMenu {
    let value_args = serde_json::json!([{ "node.x": [xs], "node.y": [ys] }, [0]]);
    let snap_args = serde_json::json!([{ "node.x": [null], "node.y": [null] }, [0]]);
    let (args, args2) = if value_order_initial {
        (snap_args, value_args)
    } else {
        (value_args, snap_args)
    };
    let toggle = Button::new()
        .label(&t!("viz.menu.node_order"))
        .method(ButtonMethod::Restyle)
        .args(args)
        .args2(args2);
    UpdateMenu::new()
        .ty(UpdateMenuType::Buttons)
        .buttons(vec![toggle])
        // top-left of the plot, clear of the centered title and the top-right modebar.
        .x(0.0)
        .x_anchor(Anchor::Left)
        .y(1.0)
        .y_anchor(Anchor::Bottom)
        // a stateless flip: no persistent "pressed" highlight (both orders are valid states).
        .show_active(false)
        .active(-1)
        .background_color(NamedColor::White)
        .border_color(NamedColor::Gray)
        .border_width(1)
        // pin the label to ink so a dark theme/toggle can't flip it invisible on the white pill.
        .font(Font::new().family(FONT_FAMILY).color(INK).size(11))
}

/// Per-column result for a `BoxOutliers` panel, produced by the single `collect_smart_values`
/// pass: the out-of-fence outlier values (capped, rendered as native box points), the most
/// extreme IN-fence values (the honest Tukey whisker endpoints), and the true uncapped outlier
/// count (for the overflow log).
struct OutlierStats {
    outliers:      Vec<f64>,
    whisker_low:   f64,
    whisker_high:  f64,
    outlier_count: usize,
}

/// One streaming pass serving both `viz smart` raw-value panels and outlier-box panels:
/// - `full_indices` (histograms / small raw boxes / violins) collect numeric values, each
///   downsampled to at most `MAX_SMART_POINTS` via uniform stride (preserving the distribution
///   shape). An entry in `strides` (sampled violins over large columns) keeps only every stride-th
///   non-null value DURING the scan, bounding memory as well as the embedded HTML — the stride is
///   precomputed from the column's cached non-null count.
/// - `fence_bounds` (large boxes that have outliers) collect ONLY the out-of-fence values (capped
///   at `SMART_BOX_OUTLIERS_CAP`; never uniform-downsampled, which would drop the extremes) and
///   track the in-fence min/max for honest whisker ends.
///
/// Empty and non-numeric cells are skipped. Folding both into one reader loop means a Data
/// Schematic mixing histograms and outlier boxes still scans the data only once.
fn collect_smart_values(
    args: &Args,
    full_indices: &[usize],
    strides: &HashMap<usize, usize>,
    fence_bounds: &HashMap<usize, (f64, f64)>,
) -> CliResult<(HashMap<usize, Vec<f64>>, HashMap<usize, OutlierStats>)> {
    let rconfig = Config::new(args.arg_input.as_ref())
        .delimiter(args.flag_delimiter)
        .no_headers_flag(args.flag_no_headers);
    let mut rdr = rconfig.reader()?;

    let mut full: HashMap<usize, Vec<f64>> =
        full_indices.iter().map(|&i| (i, Vec::new())).collect();
    let mut outliers: HashMap<usize, OutlierStats> = fence_bounds
        .keys()
        .map(|&i| {
            (
                i,
                OutlierStats {
                    outliers:      Vec::new(),
                    whisker_low:   f64::INFINITY,
                    whisker_high:  f64::NEG_INFINITY,
                    outlier_count: 0,
                },
            )
        })
        .collect();

    let mut seen: HashMap<usize, u64> = full_indices.iter().map(|&i| (i, 0)).collect();
    let mut record = csv::ByteRecord::new();
    while rdr.read_byte_record(&mut record)? {
        for &i in full_indices {
            if let Some(v) = parse_f64(record.get(i))
                && let Some(col) = full.get_mut(&i)
            {
                let k = strides.get(&i).copied().unwrap_or(1).max(1) as u64;
                let c = seen.entry(i).or_insert(0);
                if c.is_multiple_of(k) {
                    col.push(v);
                }
                *c += 1;
            }
        }
        for (&i, &(lo, hi)) in fence_bounds {
            if let Some(v) = parse_f64(record.get(i))
                && let Some(os) = outliers.get_mut(&i)
            {
                if v < lo || v > hi {
                    os.outlier_count += 1;
                    if os.outliers.len() < SMART_BOX_OUTLIERS_CAP {
                        os.outliers.push(v);
                    }
                } else {
                    os.whisker_low = os.whisker_low.min(v);
                    os.whisker_high = os.whisker_high.max(v);
                }
            }
        }
    }

    // downsample full columns so the embedded HTML stays small (outlier columns are already capped)
    for col in full.values_mut() {
        if col.len() > *MAX_SMART_POINTS {
            let (sampled, _) = downsample_pair(col, col, *MAX_SMART_POINTS);
            *col = sampled;
        }
    }

    // degenerate guard: if a column had NO in-fence value (essentially impossible — q1..q3 is
    // within the fences by construction — but be safe), fall the whisker back to the fence.
    for (&i, os) in &mut outliers {
        if !os.whisker_low.is_finite() {
            os.whisker_low = fence_bounds[&i].0;
        }
        if !os.whisker_high.is_finite() {
            os.whisker_high = fence_bounds[&i].1;
        }
        if os.outlier_count > SMART_BOX_OUTLIERS_CAP {
            log::info!(
                "viz smart: column index {i} has {} outliers; showing the first {}",
                os.outlier_count,
                SMART_BOX_OUTLIERS_CAP
            );
        }
    }

    Ok((full, outliers))
}

/// Below this many serialized bytes the data viewer's row payload embeds as plain JSON even when
/// compression is on — gzip+base64 overhead isn't worth it (mirrors `MAP_FIG_GZ_MIN_BYTES`).
const DATATABLE_GZ_MIN_BYTES: usize = 4096;

/// Read up to `limit` rows x ALL columns of the input, stream-serialized as a JSON
/// array-of-arrays for the data viewer drawer (issue #4283). Returns the JSON and the number of
/// rows actually embedded.
///
/// Cells are embedded as raw strings, which DataTables sorts by the per-column `type` the stats
/// provide (`datatable_columns_json`). Date columns are the exception: DataTables orders its
/// `date` type through the browser's `Date.parse`, which reads ISO and month-first text but
/// rejects day-first (`25/07/2026` is month 25), so those columns fell back to source order.
/// `date_cols` carries `Some(prefer_dmy)` for the columns where such a cell is instead embedded as
/// a `[raw, sort-key]` pair (`None` = not a date column), the sort key being qsv-dateparser's
/// reading under that column's own DMY preference — the column's dictionary-declared date format
/// when it expresses one, otherwise `QSV_PREFER_DMY` (issue #4303). The display keeps the source
/// text and ordering follows qsv's own parse. Cells that already lead with an ISO date sort
/// correctly as they are and stay plain strings, so an ISO column costs nothing extra.
fn collect_datatable_rows(
    args: &Args,
    limit: u64,
    date_cols: &[Option<bool>],
) -> CliResult<(String, u64)> {
    let rconfig = Config::new(args.arg_input.as_ref())
        .delimiter(args.flag_delimiter)
        .no_headers_flag(args.flag_no_headers);
    let mut rdr = rconfig.reader()?;

    // stream-serialize into one growing String: the JSON text is the only whole-dataset
    // allocation this pass makes
    let mut out = String::with_capacity(64 * 1024);
    out.push('[');
    let mut record = csv::ByteRecord::new();
    let mut n: u64 = 0;
    while n < limit && rdr.read_byte_record(&mut record)? {
        if n > 0 {
            out.push(',');
        }
        out.push('[');
        for (i, cell) in record.iter().enumerate() {
            if i > 0 {
                out.push(',');
            }
            // serde_json escapes quotes/control chars; HTML-sensitive chars (&<>) are handled
            // at embed time — the gzip-b64 path needs nothing, the plain path \u-escapes them
            let text = String::from_utf8_lossy(cell);
            let sort_key = date_cols
                .get(i)
                .copied()
                .flatten()
                .and_then(|prefer_dmy| datatable_date_sort_key(&text, prefer_dmy));
            if let Some(key) = sort_key {
                out.push('[');
                out.push_str(&serde_json::to_string(&text)?);
                out.push(',');
                out.push_str(&serde_json::to_string(&key)?);
                out.push(']');
            } else {
                out.push_str(&serde_json::to_string(&text)?);
            }
        }
        out.push(']');
        n += 1;
    }
    out.push(']');
    Ok((out, n))
}

/// The sort key for one cell of a date column, or `None` when the raw text can stand on its own
/// (blank, already ISO-leading, or unparseable — an unparseable cell has no better ordering to
/// offer than itself). Keys are RFC 3339 in UTC, so they order lexicographically, which is how
/// DataTables' `date` type will compare them.
fn datatable_date_sort_key(text: &str, prefer_dmy: bool) -> Option<String> {
    let trimmed = text.trim();
    if trimmed.is_empty() || is_iso_leading_date(trimmed) {
        return None;
    }
    qsv_dateparser::parse_with_preference(trimmed, prefer_dmy)
        .ok()
        .map(|dt| dt.to_rfc3339())
}

/// Whether `text` opens with a `YYYY-MM-DD` date. Such a value is both `Date.parse`-readable and
/// lexicographically ordered, so it needs no sort key.
fn is_iso_leading_date(text: &str) -> bool {
    let [y0, y1, y2, y3, dash1, m0, m1, dash2, d0, d1, ..] = *text.as_bytes() else {
        return false;
    };
    dash1 == b'-'
        && dash2 == b'-'
        && [y0, y1, y2, y3, m0, m1, d0, d1]
            .iter()
            .all(u8::is_ascii_digit)
}

/// The per-column DataTables config for the data viewer: `[{"title":..,"type":..,"list":..},..]`.
/// Column types come from the whole-dataset stats — client-side type detection would only see the
/// (possibly partial) embedded subset. The `type` values are DataTables', driving sorting, the
/// SearchBuilder condition set, and which `ColumnControl` search widget the header offers.
///
/// `list` opts a column into `ColumnControl`'s `searchList` (a checkbox dropdown of the distinct
/// values) instead of a text box. It is deliberately conservative on two axes:
///
/// - **Cardinality**: only low-cardinality (`CATEGORICAL_MAX_CARDINALITY`) string columns. Dates
///   get the date picker and numbers get the numeric conditions, both of which beat a value list; a
///   high-cardinality list would be a thousand-entry dropdown.
/// - **Preview completeness** (`full_preview`): `searchList` builds its options client-side from
///   the EMBEDDED rows, while `cardinality` is a whole-dataset statistic. On a truncated preview
///   the two disagree — a column with 12 distinct values dataset-wide might show 4 in the list,
///   with nothing to tell the reader the dropdown is short. So a truncated preview gets text search
///   on every column, which is honest about being a substring match over what is present.
fn datatable_columns_json(stats: &[crate::cmd::stats::StatsData], full_preview: bool) -> String {
    let mut cols = String::with_capacity(stats.len() * 32);
    cols.push('[');
    for (i, s) in stats.iter().enumerate() {
        if i > 0 {
            cols.push(',');
        }
        let dt_type = match s.r#type.as_str() {
            "Integer" | "Float" => "num",
            "Date" | "DateTime" => "date",
            _ => "string",
        };
        let list = full_preview
            && dt_type == "string"
            && s.cardinality > 0
            && s.cardinality <= CATEGORICAL_MAX_CARDINALITY;
        cols.push_str(&format!(
            "{{\"title\":{},\"type\":\"{dt_type}\",\"list\":{list}}}",
            serde_json::to_string(&s.field).unwrap_or_else(|_| "\"\"".to_string())
        ));
    }
    cols.push(']');
    cols
}

/// A complete JS string literal for `s`, safe to substitute into inline `<script>` source.
/// JSON string syntax is a subset of JS, so `serde_json` does the quoting/escaping; the `&<>`
/// trio is then escaped as in `inline_json_script` so the value cannot close the script tag.
fn js_string_literal(s: &str) -> String {
    serde_json::to_string(s)
        .unwrap_or_else(|_| "\"\"".to_string())
        .replace('&', "\\u0026")
        .replace('<', "\\u003c")
        .replace('>', "\\u003e")
}

/// A plain (uncompressed) inline JSON `<script>` payload tag: `&`/`<`/`>` become `\u00XX` inside
/// the JSON string literals so no cell value can smuggle a `</script>` into the page (same
/// escape trio as `map_panel_inline_html`'s plain branch).
fn inline_json_script(id: &str, json: &str) -> String {
    let safe = json
        .replace('&', "\\u0026")
        .replace('<', "\\u003c")
        .replace('>', "\\u003e");
    format!("<script id=\"{id}\" type=\"application/json\">{safe}</script>")
}

/// Build the data viewer chrome for a smart HTML Data Schematic: the DataTables library block, the
/// row + column payload tags, and the drawer CSS/script — plus the "(Explore)"/"(Preview)" label
/// for the metadata Rows link. `None` when the viewer is disabled (`--preview-threshold 0`) or
/// there is nothing to show. Callers gate on HTML output; image exports never pay the data pass.
fn build_data_viewer_chrome(
    args: &Args,
    stats: &[crate::cmd::stats::StatsData],
    total_rows: u64,
    dmy_prefs: &[bool],
) -> CliResult<Option<(String, String)>> {
    let threshold = args.flag_preview_threshold as u64;
    if threshold == 0 || total_rows == 0 || stats.is_empty() {
        return Ok(None);
    }
    // which columns get a date sort key — the same Date/DateTime typing that
    // `datatable_columns_json` turns into the DataTables "date" column type — each carrying the
    // preference to parse it under (dictionary-declared format, else QSV_PREFER_DMY; issue #4303)
    let date_cols: Vec<Option<bool>> = stats
        .iter()
        .enumerate()
        .map(|(i, s)| {
            matches!(s.r#type.as_str(), "Date" | "DateTime")
                .then(|| dmy_prefs.get(i).copied().unwrap_or(false))
        })
        .collect();
    let (rows_json, embedded) =
        collect_datatable_rows(args, total_rows.min(threshold), &date_cols)?;
    let full = embedded >= total_rows;
    // Two whole-sentence keys rather than a shared prefix plus an "of N" fragment: the word
    // order of "first X of Y rows" is not universal, so a translator needs the full sentence.
    let link_label = if full {
        t!("viz.viewer.link_explore")
    } else {
        t!("viz.viewer.link_preview")
    };
    let title = if full {
        t!(
            "viz.viewer.title_all",
            q_rows = HumanCount(embedded).to_string()
        )
        .to_string()
    } else {
        t!(
            "viz.viewer.title_preview",
            q_shown = HumanCount(embedded).to_string(),
            q_total = HumanCount(total_rows).to_string()
        )
        .to_string()
    };
    // The CSV export can only ever contain the rows that were embedded, so when the viewer is
    // showing a truncated preview both the button and the file name have to say so — a
    // download silently missing most of the dataset is worse than no download at all.
    // The stem is not always a plain file name — a `dc:<name>` cache input keeps its prefix, so
    // the raw stem can carry a colon, which is illegal in a Windows filename. Keep only the
    // characters every platform accepts and fall back when nothing survives (stdin included,
    // where there is no input path at all).
    let raw_stem = std::path::Path::new(args.arg_input.as_deref().unwrap_or("data"))
        .file_stem()
        .map_or_else(|| "data".to_string(), |s| s.to_string_lossy().into_owned());
    let mut export_stem: String = raw_stem
        .chars()
        .map(|c| {
            if c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '-') {
                c
            } else {
                '-'
            }
        })
        .collect();
    if export_stem.trim_matches('-').is_empty() {
        export_stem = "data".to_string();
    }
    let export_name = if full {
        export_stem
    } else {
        format!("{export_stem}-preview")
    };
    let export_text = if full {
        t!("viz.viewer.export_csv")
    } else {
        t!("viz.viewer.export_csv_preview")
    };

    let mut rows_tag = String::new();
    if viz_compress() && rows_json.len() >= DATATABLE_GZ_MIN_BYTES {
        let b64 = gzip_b64(rows_json.as_bytes(), flate2::Compression::default());
        if !b64.is_empty() {
            rows_tag = format!(
                "<script id=\"qsv-data-rows\" type=\"application/gzip-b64\">{b64}</script>"
            );
        }
    }
    if rows_tag.is_empty() {
        rows_tag = inline_json_script("qsv-data-rows", &rows_json);
    }
    let cols_tag = inline_json_script("qsv-data-cols", &datatable_columns_json(stats, full));

    let chrome = format!(
        "{lib}\n{rows_tag}\n{cols_tag}\n{script}",
        lib = datatables_lib_block(),
        script = data_drawer_script()
            .replace("__QSVDATATITLE__", &title)
            .replace("__QSVDATAEXPORTTEXT__", &export_text)
            .replace("__QSVDATAEXPORTNAME__", &js_string_literal(&export_name)),
    );
    Ok(Some((chrome, link_label.into_owned())))
}

/// The plotly axis reference string for subplot `pos` (1-based): "x"/"y" for the first,
/// "x2"/"y2", … thereafter.
fn axis_ref(prefix: char, pos: usize) -> String {
    if pos <= 1 {
        prefix.to_string()
    } else {
        format!("{prefix}{pos}")
    }
}

/// Plot-area height (pixels) of a `rows`-row Data Schematic: total height minus the top/bottom
/// margins. Floored at 1 to avoid division by zero.
fn smart_plot_area_h(row_units: f64) -> f64 {
    (row_units * ROW_HEIGHT_PX as f64 - (TOP_MARGIN_PX + BOTTOM_MARGIN_PX) as f64).max(1.0)
}

/// Top of the subplot band (paper coords) for a `rows`-row Data Schematic. The strip above it
/// reserves a fixed `TITLE_BAND_PX` pixels (converted to a paper fraction via the plot-area
/// height) for the top row's panel titles, so they always clear the Data Schematic title — for
/// both short (one-row) and tall (eight-row) dashboards. When any panel carries a subtitle
/// (a dictionary label under the title), the band grows by one extra text line so the
/// two-line title has room. Capped at half the area.
fn smart_grid_top(row_units: f64, has_subtitles: bool) -> f64 {
    let band_px = if has_subtitles {
        TITLE_BAND_PX + TITLE_SUBTITLE_EXTRA_PX
    } else {
        TITLE_BAND_PX
    };
    let band = (band_px as f64 / smart_plot_area_h(row_units)).min(0.5);
    1.0 - band
}

/// Paper-fraction gap between a cell's top and its title, fixed at `TITLE_OFFSET_PX` pixels
/// so it doesn't scale with Data Schematic height (which would consume the reserved band).
fn smart_title_offset(row_units: f64) -> f64 {
    (TITLE_OFFSET_PX as f64 / smart_plot_area_h(row_units)).min(0.05)
}

/// Geometry (in paper coordinates, 0..1) for one subplot cell in the Data Schematic grid.
#[derive(Clone)]
struct SubplotGeometry {
    x_domain: Vec<f64>,
    y_domain: Vec<f64>,
    title_x:  f64,
    title_y:  f64,
}

/// Whether a panel is a leading "overview" summary — map/geo, correlation heatmap and its
/// scatter/contour/3D drill-downs, or the time-series trend — that should span the full Data
/// Schematic width, as opposed to a per-column distribution panel (box/bar/histogram) that flows in
/// the multi-column grid below.
fn is_overview_panel(kind: &PanelKind) -> bool {
    match kind {
        PanelKind::KpiRow { .. }
        | PanelKind::CorrHeatmap { .. }
        | PanelKind::AssocHeatmap { .. }
        | PanelKind::TopRelationships { .. }
        | PanelKind::ScatterPair { .. }
        | PanelKind::Lorenz { .. }
        | PanelKind::Funnel { .. }
        | PanelKind::ContourPair { .. }
        | PanelKind::Scatter3D { .. }
        | PanelKind::MeasureByDim { .. }
        | PanelKind::GroupedViolin { .. }
        | PanelKind::CyclicProfile { .. }
        | PanelKind::TimeSeries { .. }
        | PanelKind::AnimatedScatterPair { .. }
        | PanelKind::AnimatedGeo { .. }
        | PanelKind::AnimatedBubble { .. }
        | PanelKind::Map { .. }
        | PanelKind::Geo { .. }
        | PanelKind::Choropleth { .. }
        | PanelKind::ChoroplethMap { .. }
        | PanelKind::Hierarchy { .. }
        | PanelKind::Sankey { .. }
        | PanelKind::Parcats { .. } => true,
        PanelKind::BoxStats { .. }
        | PanelKind::BoxRaw { .. }
        | PanelKind::BoxOutliers { .. }
        | PanelKind::Violin { .. }
        | PanelKind::FreqBar { .. }
        | PanelKind::Histogram { .. } => false,
    }
}

/// Fill classification of a `ScatterPair`'s point cloud, used to right-size (or drop) the panel.
/// `degenerate` = the cloud carries essentially no 2-D information (fewer than 3 distinct points,
/// no spread on an axis, or nearly every point collapsed onto a single edge) — such a panel is an
/// empty box and is omitted at build time. `sparse` = the cloud hugs one axis edge or has very few
/// distinct points (e.g. a heavily zero-skewed measure) — informative enough to keep, but it gets a
/// compact height instead of the full overview height.
struct ScatterFill {
    degenerate: bool,
    sparse:     bool,
}

/// `scatter_fill_stats` over the values as the panel will actually PLOT them: log-transformed on
/// whichever axes resolved to a log scale (issue #4276). Both verdicts a scatter's fill drives —
/// the build-time drop (`degenerate`) and the render-height branch (`sparse`) — go through here,
/// so they stay in agreement about which space the cloud is judged in, exactly as
/// `scatter_fill_stats` promises about its thresholds.
///
/// A cloud squashed against the origin on linear axes can be perfectly well spread once logged;
/// judging the height on the raw values would hand a rescued panel the compact height meant for a
/// genuinely sparse one. `relationship_axis_log` only returns true for strictly positive values,
/// so `ln` is finite wherever it is applied, and the all-linear case borrows the caller's slices
/// and allocates nothing.
fn scatter_fill_plotted(xs: &[f64], ys: &[f64], axis_log: (bool, bool)) -> ScatterFill {
    let logged = |on: bool, vs: &[f64]| -> Option<Vec<f64>> {
        on.then(|| vs.iter().map(|v| v.ln()).collect())
    };
    let (lx, ly) = (logged(axis_log.0, xs), logged(axis_log.1, ys));
    scatter_fill_stats(lx.as_deref().unwrap_or(xs), ly.as_deref().unwrap_or(ys))
}

/// Classify a scatter's fill from its row-aligned `xs`/`ys` in one O(n) pass. Thresholds live here
/// (one place) so the build-time drop guard and the render-height branch stay in agreement.
fn scatter_fill_stats(xs: &[f64], ys: &[f64]) -> ScatterFill {
    let n = xs.len().min(ys.len());
    // fewer than 3 points can't form the 3 distinct cells the degenerate check below requires, so
    // they're degenerate outright; 3+ points fall through to the real spread/distinct/edge checks
    // (a valid 3-point, non-collinear scatter with axis spread is NOT dropped).
    if n < 3 {
        return ScatterFill {
            degenerate: true,
            sparse:     true,
        };
    }
    let (mut xmin, mut xmax) = (f64::INFINITY, f64::NEG_INFINITY);
    let (mut ymin, mut ymax) = (f64::INFINITY, f64::NEG_INFINITY);
    for k in 0..n {
        let (x, y) = (xs[k], ys[k]);
        if x.is_finite() {
            xmin = xmin.min(x);
            xmax = xmax.max(x);
        }
        if y.is_finite() {
            ymin = ymin.min(y);
            ymax = ymax.max(y);
        }
    }
    let (xr, yr) = (xmax - xmin, ymax - ymin);
    // an axis with no spread makes this a 1-D line, not a 2-D relationship → nothing to show
    if !xr.is_finite() || !yr.is_finite() || xr <= 0.0 || yr <= 0.0 {
        return ScatterFill {
            degenerate: true,
            sparse:     true,
        };
    }
    let (ex, ey) = (xr * 0.01, yr * 0.01);
    // points hugging each axis's low/high edge (within 1% of an extreme): a large share on one edge
    // means the cloud has collapsed to a line (e.g. every event with killed == 0)
    let (mut x_lo, mut x_hi, mut y_lo, mut y_hi) = (0usize, 0usize, 0usize, 0usize);
    // distinct points on a coarse 100×100 grid over the bounding box
    let mut cells: std::collections::HashSet<(i32, i32)> = std::collections::HashSet::new();
    for k in 0..n {
        let (x, y) = (xs[k], ys[k]);
        if !x.is_finite() || !y.is_finite() {
            continue;
        }
        if x - xmin <= ex {
            x_lo += 1;
        }
        if xmax - x <= ex {
            x_hi += 1;
        }
        if y - ymin <= ey {
            y_lo += 1;
        }
        if ymax - y <= ey {
            y_hi += 1;
        }
        let gx = (((x - xmin) / xr) * 100.0) as i32;
        let gy = (((y - ymin) / yr) * 100.0) as i32;
        cells.insert((gx, gy));
    }
    let nf = n as f64;
    let edge_collapse = [x_lo, x_hi, y_lo, y_hi]
        .into_iter()
        .map(|c| c as f64 / nf)
        .fold(0.0_f64, f64::max);
    let distinct_ratio = cells.len() as f64 / nf;
    let degenerate = cells.len() < 3 || edge_collapse > 0.98;
    let sparse = degenerate || edge_collapse > 0.70 || distinct_ratio < 0.12;
    ScatterFill { degenerate, sparse }
}

/// Inline-Data Schematic render height (px) for a panel, CONTEXT-SENSITIVE to the panel's data:
/// geographic panels get a taller `MAP_ROW_HEIGHT_PX` (an animated map adds a
/// `SLIDER_CONTROL_ALLOWANCE_PX` band for its controls); parcats/sankey scale with their
/// category/node count (clamped); a sparse scatter gets a compact height; hierarchy gets the tall
/// `HIER_ROW_HEIGHT_PX`; the leading KPI row a short content-matched height; other overview panels
/// `OVERVIEW_ROW_HEIGHT_PX`; everything else `ROW_HEIGHT_PX`.
fn panel_render_height(kind: &PanelKind, axis_log: (bool, bool)) -> usize {
    match kind {
        PanelKind::Hierarchy { .. } => HIER_ROW_HEIGHT_PX,
        PanelKind::Parcats {
            dim_labels, tuples, ..
        } => {
            // tallest dimension's distinct-category count (small parcats still get the floor)
            let max_cat = (0..dim_labels.len())
                .map(|pos| {
                    tuples
                        .iter()
                        .filter_map(|t| t.get(pos))
                        .collect::<std::collections::HashSet<_>>()
                        .len()
                })
                .max()
                .unwrap_or(0);
            (max_cat * PARCATS_PX_PER_CATEGORY).clamp(PARCATS_HEIGHT_MIN_PX, PARCATS_HEIGHT_MAX_PX)
        },
        PanelKind::Sankey {
            link_source,
            link_target,
            ..
        } => {
            // tallest column = max of distinct source nodes vs distinct target nodes
            let src = link_source
                .iter()
                .collect::<std::collections::HashSet<_>>()
                .len();
            let tgt = link_target
                .iter()
                .collect::<std::collections::HashSet<_>>()
                .len();
            (src.max(tgt) * SANKEY_PX_PER_NODE).clamp(SANKEY_HEIGHT_MIN_PX, SANKEY_HEIGHT_MAX_PX)
        },
        PanelKind::KpiRow { tiles } => kpi_row_height(tiles),
        PanelKind::Map { .. }
        | PanelKind::Geo { .. }
        | PanelKind::Choropleth { .. }
        | PanelKind::ChoroplethMap { .. } => MAP_ROW_HEIGHT_PX,
        PanelKind::AnimatedGeo { .. } => MAP_ROW_HEIGHT_PX + SLIDER_CONTROL_ALLOWANCE_PX,
        PanelKind::ScatterPair { xs, ys, .. } if scatter_fill_plotted(xs, ys, axis_log).sparse => {
            SCATTER_SPARSE_HEIGHT_PX
        },
        _ if is_overview_panel(kind) => OVERVIEW_ROW_HEIGHT_PX,
        _ => ROW_HEIGHT_PX,
    }
}

/// Content-matched height (px) for a KPI overview row: a gauge tile renders its number lower (at
/// the dial pivot) and needs the taller `KPI_ROW_GAUGE_HEIGHT_PX` so the shared label baseline
/// clears it; a pure-number row uses the shorter `KPI_ROW_HEIGHT_PX`.
fn kpi_row_height(tiles: &[KpiTile]) -> usize {
    if tiles.iter().any(|t| t.gauge.is_some()) {
        KPI_ROW_GAUGE_HEIGHT_PX
    } else {
        KPI_ROW_HEIGHT_PX
    }
}

/// Fractional row height (in units of `ROW_HEIGHT_PX`) for a panel in the STATIC-export grid,
/// mirroring the inline path's `panel_render_height` so PNG/SVG/PDF exports match the interactive
/// HTML Data Schematic. One row unit is `ROW_HEIGHT_PX`; a taller overview/map/parcats panel scales
/// above 1.0, a short KPI row below it, a sparse scatter below it. At `1.0` for every row the
/// weighted grid geometry reduces to the original uniform layout.
fn panel_row_scale(kind: &PanelKind, axis_log: (bool, bool)) -> f64 {
    panel_render_height(kind, axis_log) as f64 / ROW_HEIGHT_PX as f64
}

/// Compute the paper-space domains for a cell at grid position (`row`, `col`) in a `rows`×`cols`
/// grid that occupies the vertical band `0.0..=top` (the strip above `top` is left for the
/// Data Schematic title), plus the (x, y) anchor for the panel's own title, placed `title_offset`
/// (a paper fraction) above the cell. When `full_width` is set the cell spans the entire page width
/// (`x_domain == [0.0, 1.0]`, used for the leading overview panels) rather than a single column.
///
/// The inter-cell gaps are fixed paper fractions for typical dashboards, but the *total* gap is
/// capped (`MAX_TOTAL_HGAP`/`MAX_TOTAL_VGAP`) so cells never collapse to a negative size: a fixed
/// per-gap fraction overflows the `0..=top` band once there are enough rows (e.g. a 42-panel,
/// 21-row Data Schematic would need 1.8 of vertical gap alone).
///
/// `row_scales[r]` is row `r`'s height as a fraction of a full `ROW_HEIGHT_PX` row (its tallest
/// panel's `panel_row_scale`), so a row can be taller (map/parcats overview) or shorter (KPI,
/// sparse scatter) than the rest. With every entry `1.0` the per-row weighting collapses back to a
/// uniform grid, so a Data Schematic of only same-height rows is byte-identical to the old layout.
/// `row_scales.len()` must equal `rows`.
fn cell_geometry(
    row: usize,
    col: usize,
    rows: usize,
    cols: usize,
    full_width: bool,
    top: f64,
    title_offset: f64,
    row_scales: &[f64],
) -> SubplotGeometry {
    const HGAP_BASE: f64 = 0.08;
    const VGAP_BASE: f64 = 0.09;
    const MAX_TOTAL_HGAP: f64 = 0.5;
    const MAX_TOTAL_VGAP: f64 = 0.4;

    let cols_f = cols as f64;
    let rows_f = rows as f64;

    let hgap = if cols > 1 {
        HGAP_BASE.min(MAX_TOTAL_HGAP / (cols_f - 1.0))
    } else {
        0.0
    };
    let vgap = if rows > 1 {
        VGAP_BASE.min(MAX_TOTAL_VGAP / (rows_f - 1.0))
    } else {
        0.0
    };

    let cell_w = (1.0 - hgap * (cols_f - 1.0)) / cols_f;
    // Row-height weights: each row counts as its `row_scales[r]` units. The total drawable band
    // (`top` minus the fixed vertical gaps) is divided by the summed weight to get one unit's
    // height, so a taller row (a map/parcats overview) claims more of the band and the rows below
    // it move down. With every scale `1.0` this is `(top - vgaps) / rows` — the uniform grid.
    let row_units: f64 = row_scales.iter().sum();
    let unit_h = (top - vgap * (rows_f - 1.0)) / row_units;
    let this_h = unit_h * row_scales[row];
    // paper-weight of every row strictly above this one
    let weight_above: f64 = row_scales[..row].iter().sum();

    // a full-width overview cell spans the whole page; otherwise it occupies its single column.
    let (x0, x1) = if full_width {
        (0.0, 1.0)
    } else {
        let x0 = col as f64 * (cell_w + hgap);
        (x0, x0 + cell_w)
    };
    let y1 = top - unit_h * weight_above - row as f64 * vgap; // top edge of the cell
    let y0 = y1 - this_h;

    SubplotGeometry {
        x_domain: vec![x0, x1],
        y_domain: vec![y0, y1],
        title_x:  (x0 + x1) / 2.0,
        title_y:  (y1 + title_offset).min(1.0),
    }
}

/// Lay out the Data Schematic panels: each leading overview panel (see `is_overview_panel`) takes
/// its own full-width row, while the remaining per-column distribution panels pack into `cols`-wide
/// rows below. Returns the per-panel geometry (indexed like `panels`), the total row count, and
/// the figure height in pixels (shorter than `rows * ROW_HEIGHT_PX` when a leading KPI row shrinks
/// its own row).
fn smart_grid_layout(panels: &[Panel], cols: usize) -> (Vec<SubplotGeometry>, usize, usize) {
    // assign each panel a (row, col, full_width) placement
    let mut placements: Vec<(usize, usize, bool)> = Vec::with_capacity(panels.len());
    let mut row = 0;
    let mut col_in_row = 0;
    for panel in panels {
        if is_overview_panel(&panel.kind) {
            // close any partially-filled grid row, then take a full-width row of its own
            if col_in_row > 0 {
                row += 1;
                col_in_row = 0;
            }
            placements.push((row, 0, true));
            row += 1;
        } else {
            if col_in_row == cols {
                row += 1;
                col_in_row = 0;
            }
            placements.push((row, col_in_row, false));
            col_in_row += 1;
        }
    }
    let rows = (if col_in_row > 0 { row + 1 } else { row }).max(1);

    // a leading KPI row (always prepended at index 0, so it lands on row 0) gets a shorter row;
    // every other layout keeps full-height rows (scale 1.0 -> the original uniform grid). The
    // scale mirrors the inline path's height branch: a gauge KPI tile renders its number lower (at
    // the dial pivot) and needs the taller `KPI_ROW_GAUGE_HEIGHT_PX` for the shared label baseline
    // to clear it; a pure-number row uses the shorter `KPI_ROW_HEIGHT_PX`.
    // per-row height scale: each row takes its tallest panel's `panel_row_scale`, so the static
    // grid mirrors the context-sensitive inline heights (taller maps/parcats/sankey, shorter KPI
    // and sparse-scatter rows). With all rows the same height this reduces to the uniform grid.
    let mut row_scales = vec![0.0_f64; rows];
    for (panel, &(r, _, _)) in panels.iter().zip(placements.iter()) {
        row_scales[r] = row_scales[r].max(panel_row_scale(&panel.kind, panel.axis_log));
    }
    for s in &mut row_scales {
        if *s <= 0.0 {
            *s = 1.0; // defensive: a row with no panel (shouldn't happen) keeps a full unit
        }
    }
    let row_units: f64 = row_scales.iter().sum();

    let has_subtitles = panels.iter().any(|p| p.subtitle.is_some());
    let top = smart_grid_top(row_units, has_subtitles);
    let title_offset = smart_title_offset(row_units);
    let geoms = placements
        .iter()
        .map(|&(r, c, full)| cell_geometry(r, c, rows, cols, full, top, title_offset, &row_scales))
        .collect();
    let height_px = (row_units * ROW_HEIGHT_PX as f64).round() as usize;
    (geoms, rows, height_px)
}

/// For a categorical bar panel, the display-only x-axis tick labels (in the same order as the
/// bar's x data), which ALSO force the axis into category mode. Two panel kinds need this:
/// - `FreqBar`: long category names (full agency names, datetime strings) would otherwise rotate
///   into tall labels that squeeze the plot area and clip the top value labels, so they're
///   truncated here.
/// - `MeasureByDim`: its x-labels are grouping-dimension VALUES that can look numeric (e.g.
///   magnitude "5.3", "6"). Without a forced category axis, plotly infers a LINEAR x-axis and drops
///   every non-numeric label (and its bar) — which silently hid the one meaningful bar when the
///   dominant group's label was non-numeric (e.g. "6 SR").
///
/// Returns `None` for non-bar panels (their axes are left untouched).
fn freq_bar_tick_text(panel: &Panel, freq: &FreqMap) -> Option<Vec<String>> {
    match &panel.kind {
        PanelKind::FreqBar { idx } => {
            let labels = freq
                .get(idx)?
                .iter()
                .map(|b| truncate_label(&b.label, BAR_LABEL_MAX_CHARS))
                .collect();
            Some(labels)
        },
        PanelKind::MeasureByDim { labels, .. } => Some(
            labels
                .iter()
                .map(|l| truncate_label(l, BAR_LABEL_MAX_CHARS))
                .collect(),
        ),
        _ => None,
    }
}

/// A styled x-axis for a Data Schematic panel: no vertical gridlines, a light baseline, and
/// small tick labels. For single-box panels (`is_box`), the lone "0" category tick is
/// meaningless, so its labels and baseline are hidden. For time-series panels (`is_date`),
/// the axis is typed as a date axis so plotly spaces ticks chronologically. For relationship
/// panels whose x values span orders of magnitude (`log`), the axis is typed logarithmic — the
/// panel title names which axis was logged (issue #4223). `tick_text`, when
/// present (frequency-bar panels, and Quarter/Year-bucketed trend panels whose "2024-Q1" labels
/// no date axis can parse — issue #4216), supplies display-only category labels and forces the
/// category axis — for bars, the x data keeps the full names, so distinct categories never
/// collapse onto one tick.
fn styled_x_axis(
    is_box: bool,
    is_date: bool,
    log: bool,
    theme: Option<BuiltinTheme>,
    tick_text: Option<Vec<String>>,
) -> Axis {
    let mut a = Axis::new()
        .show_grid(false)
        .zero_line(false)
        .show_line(true);
    // when themed, let the template style the axis lines/ticks/fonts
    if theme.is_none() {
        a = a
            .line_color(AXIS_LINE)
            .tick_color(AXIS_LINE)
            // no explicit font color: inherit the layout font (ink) so the toggle can flip it.
            .tick_font(Font::new().family(FONT_FAMILY).size(10));
    }
    if is_box {
        a = a.show_tick_labels(false).show_line(false);
    }
    if is_date {
        a = a.type_(AxisType::Date);
    }
    // a relationship panel whose x values span orders of magnitude (issue #4223). Mutually
    // exclusive with the date/category modes above by construction: those panels carry no
    // `axis_log`, and this one is neither dated nor categorical. The cue is the axis title, set by
    // the caller (which knows the column name) rather than here.
    if log {
        a = a.type_(AxisType::Log);
    }
    // display-only truncated labels for categorical bar panels (freq bars, measure-by-dim). Force
    // the axis to category mode:
    // a category axis positions categories at integer indices 0..n in x-data order regardless
    // of how the category strings look, so our integer `tickvals` line up with the bars. Without
    // this, numeric ("10", "2") or date-like ("2026-06-21") category values could be inferred as
    // a linear/date axis, leaving the ticks misaligned. The full x category values are unchanged.
    if let Some(labels) = tick_text {
        let positions: Vec<f64> = (0..labels.len()).map(|i| i as f64).collect();
        a = a
            .type_(AxisType::Category)
            .tick_mode(TickMode::Array)
            .tick_values(positions)
            .tick_text(labels);
    }
    a
}

/// A styled y-axis for a Data Schematic panel: light horizontal gridlines only, small ticks.
/// When `headroom_max` is given (bar panels), the range is fixed so the tallest bar's outside
/// value label has room and isn't clipped at the cell top: `0..=max*1.15` on a linear axis, or
/// (when `log`) a log10 range from just under 1 up to `log10(max)` plus a margin. A `log` axis
/// also carries a title as the visual cue that it's logarithmic: `LOG_AXIS_TITLE` ("count
/// (log)") for bar panels, `VALUE_LOG_AXIS_TITLE` ("log scale") for box value axes.
fn styled_y_axis(headroom_max: Option<f64>, log: bool, theme: Option<BuiltinTheme>) -> Axis {
    let mut a = Axis::new()
        .show_grid(true)
        .grid_width(1)
        .zero_line(false)
        .show_line(false)
        // match the SI prefixes qsv's own value labels use (`%{y:.3s}` on bars, `.3~s` on
        // indicators). Plotly's untouched default is `exponentformat: "B"`, which renders 10^9 as
        // "B" while the bar sitting against that gridline is labelled "G" — one chart, two
        // suffixes for the same magnitude.
        .exponent_format(ExponentFormat::SI);
    // when themed, let the template style the gridlines/ticks/fonts
    if theme.is_none() {
        a = a
            .grid_color(GRID_COLOR)
            .tick_color(AXIS_LINE)
            // no explicit font color: inherit the layout font (ink) so the toggle can flip it.
            .tick_font(Font::new().family(FONT_FAMILY).size(10));
    }
    match (log, headroom_max) {
        // log axis: plotly ranges are in log10 units. Start a hair below 1 (counts are >= 1) so
        // a count-of-1 bar still draws a visible sliver, and add ~0.15 in log10 (~1.4x) of top
        // headroom for the outside value labels.
        (true, Some(m)) if m > 0.0 => {
            a = a.type_(AxisType::Log).range(vec![-0.05, m.log10() + 0.15]);
        },
        // log axis without a known max: a box panel's value axis (bar panels always pass their
        // tallest bar). Let plotly autorange the log domain.
        (true, _) => {
            a = a.type_(AxisType::Log);
        },
        (false, Some(m)) if m > 0.0 => {
            a = a.range(vec![0.0, m * 1.15]);
        },
        (false, _) => {},
    }
    if log {
        // the visual cue: only log panels get a y-axis title, so a log scale is never mistaken
        // for linear. Linear panels stay title-less to keep the cells compact. Bar panels (which
        // always carry a headroom max) label the axis as a log COUNT; box panels label it as a
        // log VALUE axis.
        let mut title_font = Font::new().size(11);
        if theme.is_none() {
            // no explicit color: inherit the layout font (ink) so the toggle can flip it.
            title_font = title_font.family(FONT_FAMILY);
        }
        let cue = if headroom_max.is_some() {
            log_axis_title()
        } else {
            value_log_axis_title()
        };
        a = a.title(Title::with_text(cue).font(title_font));
    }
    a
}

/// Zoomed value-axis range `[floor, ceil]` for the "Top Relationships" lollipop (`viz smart
/// --bivariate`), given the ranked NMI values. Anchors the floor near the smallest shown value
/// (not 0) so near-ceiling pairs stay separable, but never narrower than `TOPREL_MIN_SPAN` (which
/// guards the all-tied-at-1.0 case from collapsing to a sliver) and never below 0; the top is
/// padded and capped at `TOPREL_CEIL_CAP`. Empty input (never produced by the builder, but kept
/// total) falls back to the full `[0, 1]` NMI domain.
fn lollipop_value_range(values: &[f64]) -> (f64, f64) {
    let max = values
        .iter()
        .copied()
        .filter(|v| v.is_finite())
        .fold(f64::NEG_INFINITY, f64::max);
    let min = values
        .iter()
        .copied()
        .filter(|v| v.is_finite())
        .fold(f64::INFINITY, f64::min);
    if !max.is_finite() || !min.is_finite() {
        return (0.0, 1.0);
    }
    // pad proportionally to the data span (so the dots spread across the panel), but at least
    // TOPREL_MIN_PAD absolute — that absolute floor is what rescues a near-tied cluster (span ≈ 0)
    // from collapsing against the panel edge.
    let pad = ((max - min) * TOPREL_PAD_FRAC).max(TOPREL_MIN_PAD);
    let ceil = (max + pad).min(TOPREL_CEIL_CAP);
    let floor = (min - pad).max(0.0);
    // degeneracy guard only: an all-EXACTLY-tied top-N (span 0, pad = TOPREL_MIN_PAD) still yields
    // a >0 span here, but keep a hard minimum so the axis can never render zero-width.
    if ceil - floor < TOPREL_MIN_SPAN {
        return ((ceil - TOPREL_MIN_SPAN).max(0.0), ceil);
    }
    (floor, ceil)
}

/// The zoomed, linear VALUE (x) axis for a horizontal "Top Relationships" lollipop — light
/// vertical gridlines, small ticks, range fixed to the top-N band via `lollipop_value_range`.
/// Mirrors `styled_y_axis`'s chrome so the panel reads consistently with the rest of the grid.
fn lollipop_value_axis(floor: f64, ceil: f64, theme: Option<BuiltinTheme>) -> Axis {
    let mut a = Axis::new()
        .show_grid(true)
        .grid_width(1)
        .zero_line(false)
        .show_line(false)
        .range(vec![floor, ceil]);
    if theme.is_none() {
        a = a
            .grid_color(GRID_COLOR)
            .tick_color(AXIS_LINE)
            .tick_font(Font::new().family(FONT_FAMILY).size(10));
    }
    a
}

/// Left margin (px) reserved for a funnel panel's category ticks in the INLINE render path.
///
/// Mirrors the per-label sizing `smart_grid_parts` already does for its shared margin, rather
/// than borrowing `TOPREL_LABEL_MAX_CHARS` — that budget is sized for the lollipop's 56-char
/// truncated column-PAIR labels, and a funnel's ticks are short stage names.
fn funnel_left_margin(stages: &[String]) -> usize {
    stages
        .iter()
        .map(|l| l.chars().count().min(TOPREL_LABEL_MAX_CHARS))
        .max()
        .map_or(60, |longest| longest * CORR_LABEL_PX_PER_CHAR + 24)
}

/// The VALUE (x) axis for a horizontal pipeline funnel (issue #4222).
///
/// **Symmetric about zero, because plotly centers funnel bands on the value zero** — each band
/// spans `-amount/2 ..= +amount/2`. A `0..=ceil` range therefore puts the band centre on the
/// LEFT EDGE of the plot area and draws half of every band outside it. That was survivable only
/// because the panel happened to reserve a 416px left margin wide enough to absorb the overflow;
/// sizing that margin to the actual tick labels (which is what it is for) clipped the widest band.
///
/// So the range is half-width either side of zero: the widest band still occupies `1/1.35` of the
/// plot area, exactly as before, but centred rather than half off-screen — and the remaining
/// headroom leaves room for plotly's `textinfo` beside the widest band.
///
/// Tick labels are hidden: with a symmetric range they would read `-100G … 100G` for amounts that
/// are never negative, and a funnel's value axis carries no information the panel doesn't already
/// state — the band text gives stage-to-stage conversion and the hover gives exact amounts. The
/// standalone `viz funnel` likewise draws no value axis.
///
/// Never logged — a log value axis destroys the proportional reading the form depends on.
fn funnel_value_axis(max: f64, theme: Option<BuiltinTheme>) -> Axis {
    let half = if max > 0.0 { max * 1.35 / 2.0 } else { 1.0 };
    let mut a = Axis::new()
        .show_grid(false)
        .zero_line(false)
        .show_line(false)
        .show_tick_labels(false)
        // SI prefixes, matching the rest of the Data Schematic's value formatting (see
        // `styled_y_axis`)
        .exponent_format(ExponentFormat::SI)
        .range(vec![-half, half]);
    if theme.is_none() {
        a = a.tick_color(AXIS_LINE);
    }
    a
}

/// The CATEGORY (y) axis for a horizontal "Top Relationships" lollipop — one tick per ranked pair,
/// no gridlines, small ticks. The trace's y-values are the FULL pair labels (so each pair is a
/// distinct row); `tick_text` overrides only what's DISPLAYED with the caller's already-truncated
/// labels (`tick_text[i]` labeling category position `i`, bottom-to-top). The full name still rides
/// in each dot's hover template. `tick_text` is bottom-to-top (weakest-first), matching the
/// reversed order the trace feeds its y-values.
fn lollipop_category_axis(theme: Option<BuiltinTheme>, tick_text: &[String]) -> Axis {
    let mut a = Axis::new()
        .type_(AxisType::Category)
        .show_grid(false)
        .zero_line(false)
        .show_line(false)
        .tick_mode(TickMode::Array)
        .tick_values((0..tick_text.len()).map(|i| i as f64).collect())
        .tick_text(tick_text.to_vec());
    if theme.is_none() {
        a = a
            .tick_color(AXIS_LINE)
            .tick_font(Font::new().family(FONT_FAMILY).size(10));
    }
    a
}

/// Assign subplot `pos`'s prebuilt axis pair (domains + cross anchors already applied by
/// `smart_grid_parts`) to the typed `Layout` axis fields, which only exist up to 8 (matching
/// `MAX_SUBPLOTS`). Positions beyond 8 can't be expressed by the typed `Layout`; those grids
/// go through `render_smart_grid_json` instead, so this silently drops them as a safeguard.
///
/// Takes `&mut Layout` (via `mem::take`) rather than moving through the by-value builder:
/// at `opt-level=0` each per-arm builder chain would otherwise pin two more full `Layout`
/// stack slots per arm in the caller's loop frame (issue #4328).
fn assign_typed_axis(layout: &mut Layout, pos: usize, x_axis: Axis, y_axis: Axis) {
    let (x, y) = (x_axis, y_axis);
    let l = std::mem::take(layout);
    *layout = match pos {
        1 => l.x_axis(x).y_axis(y),
        2 => l.x_axis2(x).y_axis2(y),
        3 => l.x_axis3(x).y_axis3(y),
        4 => l.x_axis4(x).y_axis4(y),
        5 => l.x_axis5(x).y_axis5(y),
        6 => l.x_axis6(x).y_axis6(y),
        7 => l.x_axis7(x).y_axis7(y),
        8 => l.x_axis8(x).y_axis8(y),
        _ => l,
    };
}

/// The plotly.js layout key for axis `pos` (1-based): `xaxis`/`yaxis` for the first cell,
/// `xaxis{pos}`/`yaxis{pos}` thereafter — matching the `serde(rename = ...)` the typed `Layout`
/// uses, and the `x{pos}`/`y{pos}` refs `axis_ref` stamps on each trace.
fn axis_json_key(prefix: char, pos: usize) -> String {
    if pos <= 1 {
        format!("{prefix}axis")
    } else {
        format!("{prefix}axis{pos}")
    }
}

/// The plotly.js subplot id / layout key for geo panel `pos` (1-based): `geo` for the first geo
/// subplot, `geo{pos}` thereafter. The same string is used both as the trace's `subplot` value
/// and as the layout object key (plotly.js uses identical names for the two).
fn geo_ref(pos: usize) -> String {
    if pos <= 1 {
        "geo".to_string()
    } else {
        format!("geo{pos}")
    }
}

#[cfg(test)]
mod tests {
    use super::*;

    /// Pin the process-global locale to English for a test that ASSERTS on localized output.
    ///
    /// Observers need this as much as mutators do. The locale is process-global, cargo runs unit
    /// tests on parallel threads, and one test calls `set_active(spanish)` -- so a test asserting
    /// English can interleave with it, pass in isolation, and fail only in the full suite. Every
    /// time a helper gained a `t!` call during localization, the tests reading its output silently
    /// acquired this dependency; that has now happened four separate times, which is why this is a
    /// named helper rather than two lines copied around.
    ///
    /// Bind the result -- `let _locale = english_locale();`. Dropping it immediately (`let _ =`)
    /// releases the lock and reinstates exactly the race it exists to prevent.
    #[must_use = "bind this guard to a named variable; dropping it immediately reinstates the race"]
    fn english_locale() -> std::sync::MutexGuard<'static, ()> {
        let guard = viz_i18n::lock_locale();
        viz_i18n::reset_active();
        guard
    }

    // The atomic sidecar write must be mode-transparent: replacing `fs::write` with a
    // tempfile+rename must not change what permissions the sidecar ends up with, in EITHER
    // direction. tempfile creates 0600, so a naive port narrows a new sidecar; forcing a fixed
    // 0644 instead would widen a deliberately private one and ignore a restrictive umask.
    #[cfg(unix)]
    #[test]
    fn dictionary_sidecar_write_is_mode_transparent() {
        use std::os::unix::fs::PermissionsExt;

        let dir = tempfile::tempdir().unwrap();

        // What `fs::write` produces for a NEW file here, measured rather than assumed so the
        // test holds under any umask.
        let probe = dir.path().join("probe.json");
        std::fs::write(&probe, "{}").unwrap();
        let want_new_mode = std::fs::metadata(&probe).unwrap().permissions().mode();

        let sidecar = dir.path().join("data.schema.json");
        write_dictionary_sidecar(&sidecar, r#"{"a":1}"#).unwrap();
        assert_eq!(std::fs::read_to_string(&sidecar).unwrap(), r#"{"a":1}"#);
        assert_eq!(
            std::fs::metadata(&sidecar).unwrap().permissions().mode(),
            want_new_mode,
            "a new sidecar should land with the same mode fs::write would have given it"
        );

        // An EXISTING sidecar the user deliberately made private must survive the replace with
        // its mode intact.
        std::fs::set_permissions(&sidecar, std::fs::Permissions::from_mode(0o600)).unwrap();
        write_dictionary_sidecar(&sidecar, r#"{"b":2}"#).unwrap();
        assert_eq!(
            std::fs::read_to_string(&sidecar).unwrap(),
            r#"{"b":2}"#,
            "the replace should fully overwrite the previous contents"
        );
        assert_eq!(
            std::fs::metadata(&sidecar).unwrap().permissions().mode() & 0o777,
            0o600,
            "replacing an existing sidecar must not widen its permissions"
        );

        // A mode with bits the umask would strip at creation time (group-write is removed by the
        // common 022): the post-write `set_permissions` has to put them back, so this fails if
        // creation alone is relied on.
        std::fs::set_permissions(&sidecar, std::fs::Permissions::from_mode(0o660)).unwrap();
        write_dictionary_sidecar(&sidecar, r#"{"c":3}"#).unwrap();
        assert_eq!(
            std::fs::metadata(&sidecar).unwrap().permissions().mode() & 0o777,
            0o660,
            "a mode carrying bits the umask strips must still be restored exactly"
        );

        // the tempfile is renamed into place, never left lying next to the sidecar
        let strays: Vec<_> = std::fs::read_dir(dir.path())
            .unwrap()
            .filter_map(Result::ok)
            .map(|e| e.file_name().to_string_lossy().into_owned())
            .filter(|n| n.starts_with(".qsv-dict-"))
            .collect();
        assert!(strays.is_empty(), "temp files left behind: {strays:?}");
    }

    fn stat(ty: &str, cardinality: u64, uniqueness: Option<f64>) -> crate::cmd::stats::StatsData {
        crate::cmd::stats::StatsData {
            r#type: ty.to_string(),
            cardinality,
            uniqueness_ratio: uniqueness,
            ..Default::default()
        }
    }

    // map / choropleth basemap framing must honor --width/--height for static image exports but use
    // the default aspect for responsive HTML, so a non-default static aspect doesn't crop the
    // GeoJSON extent (the wiring shared by build_map_plot and build_choropleth_plot).
    #[test]
    fn fit_dims_honors_image_aspect_only() {
        // HTML ignores --width/--height and always frames for the representative default aspect
        assert_eq!(
            fit_dims(Some(400), Some(900), OutFormat::Html),
            (DEFAULT_IMG_WIDTH, DEFAULT_IMG_HEIGHT)
        );
        // image exports honor an explicit (here narrow/tall) aspect ...
        assert_eq!(fit_dims(Some(400), Some(900), OutFormat::Png), (400, 900));
        // ... and a wide/short one ...
        assert_eq!(fit_dims(Some(1600), Some(300), OutFormat::Svg), (1600, 300));
        // ... falling back to the defaults for whichever dimension is omitted
        assert_eq!(
            fit_dims(None, Some(900), OutFormat::Jpeg),
            (DEFAULT_IMG_WIDTH, 900)
        );

        // the aspect actually changes the fit zoom: a wide extent frames tighter (higher zoom) in a
        // wide-short panel than in a tall-narrow one, so passing the wrong dims would mis-zoom.
        let lats = [30.0, 45.0];
        let lons = [-124.0, -72.0];
        let (_, zoom_wide) = map_center_zoom(&lats, &lons, 0.0, 1600.0, 300.0);
        let (_, zoom_tall) = map_center_zoom(&lats, &lons, 0.0, 300.0, 1600.0);
        assert!(
            zoom_wide > zoom_tall,
            "wide-short panel should frame a wide extent tighter than a tall-narrow one \
             (zoom_wide={zoom_wide}, zoom_tall={zoom_tall})"
        );
    }

    #[test]
    fn first_description_paragraph_skips_markdown_heading() {
        // bold-only heading line (the LLM-generated dictionary case, roborev #3502)
        assert_eq!(
            first_description_paragraph("**Description**\n\nThe real summary sentence."),
            "The real summary sentence."
        );
        // ATX heading
        assert_eq!(
            first_description_paragraph("# Description\n\nThe real summary sentence."),
            "The real summary sentence."
        );
        // no heading: first paragraph returned unchanged (backward compatible)
        assert_eq!(
            first_description_paragraph(
                "First paragraph of the summary.\n\nSecond paragraph is dropped."
            ),
            "First paragraph of the summary."
        );
        // multi-line first paragraph is joined with spaces; only the first paragraph is kept
        assert_eq!(
            first_description_paragraph(
                "**Description**\n\nLine one.\nLine two.\n\n**Notable**\n\n- item"
            ),
            "Line one. Line two."
        );
        // leading blank lines and a heading combined
        assert_eq!(
            first_description_paragraph("\n\n**Description**\n\n\nActual text."),
            "Actual text."
        );
        // a fully-emphasized FIRST paragraph that is a real summary sentence is kept,
        // not mistaken for a heading (roborev #3504): it ends with sentence punctuation.
        assert_eq!(
            first_description_paragraph(
                "**This dataset contains 50,013 records.**\n\nMore detail."
            ),
            "**This dataset contains 50,013 records.**"
        );
        // a long emphasized first line with no terminal punctuation is still kept (not terse).
        assert_eq!(
            first_description_paragraph(
                "*The dataset enumerates every licensed dog across the county each year*\n\nMore."
            ),
            "*The dataset enumerates every licensed dog across the county each year*"
        );
        // empty / heading-only descriptions yield an empty string
        assert_eq!(first_description_paragraph("**Description**"), "");
        assert_eq!(first_description_paragraph("   "), "");
    }

    #[cfg(feature = "geocode")]
    fn gp(tag: &'static str, city: &str, admin1: &str, country: &str) -> GeoPoint {
        GeoPoint {
            tag,
            lat: 0.0,
            lon: 0.0,
            label: Some(crate::cmd::geocode::GeoLabel {
                city: city.to_string(),
                admin1: admin1.to_string(),
                country: country.to_string(),
                ..Default::default()
            }),
        }
    }

    #[cfg(feature = "geocode")]
    #[test]
    fn consolidate_geo_single_city() {
        let pts = vec![
            gp("NW", "New York City", "New York", "United States"),
            gp("NE", "New York City", "New York", "United States"),
            gp("Center", "New York City", "New York", "United States"),
        ];
        assert_eq!(
            consolidate_geo(&pts),
            "New York City, New York, United States"
        );
    }

    #[cfg(feature = "geocode")]
    #[test]
    fn consolidate_geo_two_states_one_country() {
        let pts = vec![
            gp("NW", "Newark", "New Jersey", "United States"),
            gp("NE", "Brooklyn", "New York", "United States"),
        ];
        // multiple distinct states within a single country collapse to "A & B, Country"
        assert_eq!(
            consolidate_geo(&pts),
            "New Jersey & New York, United States"
        );
    }

    #[cfg(feature = "geocode")]
    #[test]
    fn consolidate_geo_multi_country() {
        let pts = vec![
            gp("NW", "Seattle", "Washington", "United States"),
            gp("NE", "Vancouver", "British Columbia", "Canada"),
        ];
        assert_eq!(consolidate_geo(&pts), "United States & Canada");
    }

    #[cfg(feature = "geocode")]
    #[test]
    fn consolidate_geo_rolls_up_beyond_three_distinct() {
        // Both roll-up branches are localized and NO golden fixture reaches
        // either, so this is their only coverage -- hence the English pin
        // (roborev 3955).
        let _locale = english_locale();
        // >3 distinct countries -> the country roll-up, no region tail
        let countries = vec![
            gp("A", "Seattle", "Washington", "United States"),
            gp("B", "Vancouver", "British Columbia", "Canada"),
            gp("C", "Mexico City", "CDMX", "Mexico"),
            gp("D", "Lima", "Lima", "Peru"),
        ];
        assert_eq!(consolidate_geo(&countries), "4 countries");
        // >3 distinct admin1s inside ONE country -> the region roll-up, which
        // keeps the ", <country>" tail
        let regions = vec![
            gp("A", "Seattle", "Washington", "United States"),
            gp("B", "Portland", "Oregon", "United States"),
            gp("C", "Boise", "Idaho", "United States"),
            gp("D", "Reno", "Nevada", "United States"),
        ];
        assert_eq!(consolidate_geo(&regions), "4 regions, United States");
    }

    #[cfg(feature = "geocode")]
    fn gp_county(
        tag: &'static str,
        city: &str,
        admin2: &str,
        admin1: &str,
        country: &str,
    ) -> GeoPoint {
        GeoPoint {
            tag,
            lat: 0.0,
            lon: 0.0,
            label: Some(crate::cmd::geocode::GeoLabel {
                city: city.to_string(),
                admin1: admin1.to_string(),
                country: country.to_string(),
                admin2: admin2.to_string(),
                ..Default::default()
            }),
        }
    }

    #[cfg(feature = "geocode")]
    #[test]
    fn consolidate_geo_single_county() {
        // multiple cities but a single county within one state -> name the county for tight context
        let pts = vec![
            gp_county(
                "NW",
                "Pittsburgh",
                "Allegheny County",
                "Pennsylvania",
                "United States",
            ),
            gp_county(
                "NE",
                "McKeesport",
                "Allegheny County",
                "Pennsylvania",
                "United States",
            ),
            gp_county(
                "SE",
                "Bethel Park",
                "Allegheny County",
                "Pennsylvania",
                "United States",
            ),
        ];
        assert_eq!(
            consolidate_geo(&pts),
            "Allegheny County, Pennsylvania, United States"
        );
    }

    #[cfg(feature = "geocode")]
    #[test]
    fn consolidate_geo_multi_county_no_regression() {
        // several counties in one state -> stays the plain "State, Country" form (no county name)
        let pts = vec![
            gp_county(
                "NW",
                "Pittsburgh",
                "Allegheny County",
                "Pennsylvania",
                "United States",
            ),
            gp_county(
                "NE",
                "Greensburg",
                "Westmoreland County",
                "Pennsylvania",
                "United States",
            ),
        ];
        assert_eq!(consolidate_geo(&pts), "Pennsylvania, United States");
    }

    #[cfg(feature = "geocode")]
    #[test]
    fn consolidate_geo_all_unresolved_is_empty() {
        let pts = vec![
            GeoPoint {
                tag:   "NW",
                lat:   0.0,
                lon:   0.0,
                label: None,
            },
            GeoPoint {
                tag:   "Center",
                lat:   0.0,
                lon:   0.0,
                label: None,
            },
        ];
        assert!(consolidate_geo(&pts).is_empty());
    }

    #[cfg(feature = "geocode")]
    #[test]
    fn map_extent_min_max() {
        let e = map_extent(&[1.0, 3.0, 2.0], &[-5.0, 10.0, 2.0]);
        assert_eq!(
            (e.min_lat, e.max_lat, e.min_lon, e.max_lon),
            (1.0, 3.0, -5.0, 10.0)
        );
    }

    #[cfg(feature = "geocode")]
    #[test]
    fn extent_including_grows_to_cover_points() {
        let core = MapExtent {
            min_lat: 40.0,
            max_lat: 41.0,
            min_lon: -75.0,
            max_lon: -74.0,
        };
        // a far north-west stray and a far south-east stray expand the box on all four sides
        let full = extent_including(core, &[44.0, 38.0], &[-78.0, -70.0]);
        assert_eq!(
            (full.min_lat, full.max_lat, full.min_lon, full.max_lon),
            (38.0, 44.0, -78.0, -70.0)
        );
        // no points -> unchanged
        let same = extent_including(core, &[], &[]);
        assert_eq!(
            (same.min_lat, same.max_lat, same.min_lon, same.max_lon),
            (40.0, 41.0, -75.0, -74.0)
        );
    }

    #[test]
    fn partition_geo_outliers_flags_far_from_centroid() {
        // a small spread cluster (10x10 grid in a ~0.09deg box, so the distance IQR is non-zero),
        // plus two points far from the centroid — only the two strays exceed the far-out fence.
        let mut lats: Vec<f64> = Vec::new();
        let mut lons: Vec<f64> = Vec::new();
        for i in 0..10 {
            for j in 0..10 {
                lats.push(40.70 + f64::from(i) * 0.01);
                lons.push(-74.00 + f64::from(j) * 0.01);
            }
        }
        lats.push(45.0); // far north
        lons.push(-74.0);
        lats.push(40.7); // far east
        lons.push(-69.0);
        let (clat, clon, olat, olon) = partition_geo_outliers(&lats, &lons);
        assert_eq!(olat.len(), 2, "only the two far strays flagged");
        assert_eq!(olon.len(), 2);
        assert_eq!(
            clat.len() + olat.len(),
            lats.len(),
            "every point accounted for"
        );
        assert_eq!(clon.len(), clat.len());
        assert!(olat.contains(&45.0));
        assert!(olon.contains(&-69.0));
    }

    #[test]
    fn partition_geo_outliers_too_few_all_core() {
        // too few points to characterize a cluster -> everything is core, zero outliers
        let lats = vec![1.0, 2.0, 3.0];
        let lons = vec![1.0, 2.0, 3.0];
        let (clat, _clon, olat, olon) = partition_geo_outliers(&lats, &lons);
        assert_eq!(clat.len(), 3);
        assert!(olat.is_empty());
        assert!(olon.is_empty());
    }

    #[test]
    fn partition_geo_outliers_duplicates_plus_stray() {
        // a very common shape: a few duplicate coordinates (zero distance IQR) plus one bad
        // geocode. the point-mass (d > q3) fallback must flag the lone far stray even with
        // < 10 duplicates, where a mean/std fence (inflated by the stray itself) would not.
        let mut lats: Vec<f64> = vec![40.70; 5];
        let mut lons: Vec<f64> = vec![-74.00; 5];
        lats.push(60.0); // a distant stray
        lons.push(-74.0);
        let (clat, _clon, olat, olon) = partition_geo_outliers(&lats, &lons);
        assert_eq!(
            olat.len(),
            1,
            "the lone far stray is flagged despite zero IQR"
        );
        assert!(olat.contains(&60.0));
        assert_eq!(clat.len(), 5, "the duplicate core points stay core");
        assert_eq!(olon, vec![-74.0]);
    }

    #[test]
    fn partition_geo_outliers_point_mass_keeps_nearby_points_core() {
        // ≥75% duplicate coordinates (q1 == q3 == 0) + legitimately-nearby scatter: the
        // zero-IQR fallback must NOT flag every non-duplicate point — only genuine strays
        // beyond the absolute floor. 800:200 city-centroid-default vs real-address shape.
        let mut lats: Vec<f64> = vec![40.70; 16];
        let mut lons: Vec<f64> = vec![-74.00; 16];
        // 4 real addresses scattered ~0.05-0.3° (a few km to ~30 km) from the centroid
        for (dlat, dlon) in [(0.05, 0.1), (-0.2, 0.05), (0.3, -0.1), (-0.1, -0.25)] {
            lats.push(40.70 + dlat);
            lons.push(-74.00 + dlon);
        }
        // plus one genuine stray hundreds of km out
        lats.push(45.0);
        lons.push(-74.0);
        let (clat, _clon, olat, _olon) = partition_geo_outliers(&lats, &lons);
        assert_eq!(olat, vec![45.0], "only the genuine stray is flagged");
        assert_eq!(clat.len(), 20, "nearby real addresses stay core");
    }

    #[test]
    fn partition_geo_outliers_constant_flags_nothing() {
        // a fully constant (zero-spread) distribution has zero distance IQR -> nothing flagged
        let lats = vec![10.0; 50];
        let lons = vec![20.0; 50];
        let (_clat, _clon, olat, olon) = partition_geo_outliers(&lats, &lons);
        assert!(olat.is_empty());
        assert!(olon.is_empty());
    }

    #[cfg(feature = "geocode")]
    #[test]
    fn outlier_summary_count_and_jurisdiction() {
        // the outlier call-out is localized -- pin English, or these assertions
        // race the Spanish-setting tests on the process-global locale.
        let _locale = english_locale();
        let core = "New York & New Jersey, United States";
        // zero outliers -> core summary unchanged
        assert_eq!(outlier_summary(core, 0, "Pennsylvania"), core);
        // singular / plural
        assert_eq!(
            outlier_summary(core, 1, "Pennsylvania"),
            "New York & New Jersey, United States \u{2014} 1 outlier (Pennsylvania)"
        );
        assert_eq!(
            outlier_summary(core, 3, "Pennsylvania"),
            "New York & New Jersey, United States \u{2014} 3 outliers (Pennsylvania)"
        );
        // empty jurisdiction -> no parenthetical
        assert_eq!(
            outlier_summary(core, 2, ""),
            "New York & New Jersey, United States \u{2014} 2 outliers"
        );
        // large counts are humanized with thousands separators
        assert_eq!(
            outlier_summary(core, 1_234, "Pennsylvania"),
            "New York & New Jersey, United States \u{2014} 1,234 outliers (Pennsylvania)"
        );
        assert_eq!(
            outlier_summary(core, 12_345, ""),
            "New York & New Jersey, United States \u{2014} 12,345 outliers"
        );
    }

    #[cfg(feature = "geocode")]
    #[test]
    fn outlier_jurisdictions_lists_admin1_then_countries() {
        let label = |admin1: &str, country: &str| {
            Some(crate::cmd::geocode::GeoLabel {
                city: String::new(),
                admin1: admin1.to_string(),
                country: country.to_string(),
                ..Default::default()
            })
        };
        // single admin1
        assert_eq!(
            outlier_jurisdictions(&[label("Pennsylvania", "United States")]),
            "Pennsylvania"
        );
        // two distinct admin1s
        assert_eq!(
            outlier_jurisdictions(&[
                label("Pennsylvania", "United States"),
                label("Ohio", "United States"),
            ]),
            "Pennsylvania & Ohio"
        );
        // no admin1 -> falls back to countries
        assert_eq!(
            outlier_jurisdictions(&[label("", "Canada"), label("", "Mexico")]),
            "Canada & Mexico"
        );
        // nothing resolved -> empty
        assert_eq!(outlier_jurisdictions(&[None, None]), "");
        // more than 3 distinct -> the localized "N areas" roll-up rather than a
        // long list. This value is interpolated into `geo_outliers_where`'s
        // parenthetical, so an unlocalized roll-up here would leave English
        // inside an otherwise-translated caption (roborev 3955).
        let _locale = english_locale();
        assert_eq!(
            outlier_jurisdictions(&[
                label("Pennsylvania", "United States"),
                label("Ohio", "United States"),
                label("Maine", "United States"),
                label("Nevada", "United States"),
            ]),
            "4 areas"
        );
        // the same roll-up on the country fallback path
        assert_eq!(
            outlier_jurisdictions(&[
                label("", "Canada"),
                label("", "Mexico"),
                label("", "Brazil"),
                label("", "Peru"),
            ]),
            "4 areas"
        );
    }

    #[cfg(feature = "geocode")]
    #[test]
    fn outliers_share_core_region_suppression() {
        let label = |admin1: &str, country: &str| {
            Some(crate::cmd::geocode::GeoLabel {
                city: String::new(),
                admin1: admin1.to_string(),
                country: country.to_string(),
                ..Default::default()
            })
        };
        let core = vec![
            gp("NW", "Newark", "New Jersey", "United States"),
            gp("NE", "Brooklyn", "New York", "United States"),
        ];
        // outliers in a core admin1 -> share region (suppress)
        assert!(outliers_share_core_region(
            &core,
            &[label("New York", "United States")]
        ));
        // outliers in a NEW admin1 (Pennsylvania) -> real strays (keep call-out)
        assert!(!outliers_share_core_region(
            &core,
            &[label("Pennsylvania", "United States")]
        ));
        // outliers in a new country -> real strays
        assert!(!outliers_share_core_region(
            &core,
            &[label("Ontario", "Canada")]
        ));
        // unresolved outliers -> can't claim elsewhere -> suppress
        assert!(outliers_share_core_region(&core, &[None]));
    }

    #[cfg(feature = "geocode")]
    #[test]
    fn extent_box_is_closed_loop_nw_ne_se_sw() {
        let e = MapExtent {
            min_lat: 1.0,
            max_lat: 3.0,
            min_lon: -5.0,
            max_lon: 10.0,
        };
        let (lats, lons) = extent_box_latlon(&e);
        assert_eq!(lats, vec![3.0, 3.0, 1.0, 1.0, 3.0]);
        assert_eq!(lons, vec![-5.0, 10.0, 10.0, -5.0, -5.0]);
    }

    #[cfg(feature = "geocode")]
    #[test]
    fn extent_zoom_menu_core_and_full_buttons() {
        // the zoom-button labels are localized -- pin English, or the label
        // assertions below race the Spanish-setting tests on the global locale.
        let _locale = english_locale();
        let core = MapExtent {
            min_lat: 40.70,
            max_lat: 40.80,
            min_lon: -74.05,
            max_lon: -73.95,
        };
        // full extent reaches well beyond the core (a far stray)
        let full = MapExtent {
            min_lat: 40.20,
            max_lat: 41.20,
            min_lon: -76.90,
            max_lon: -73.90,
        };
        let v = serde_json::to_value(extent_zoom_menu(&core, &full)).unwrap();
        let btns = v["buttons"].as_array().expect("buttons array");
        assert_eq!(btns.len(), 2);
        assert_eq!(btns[0]["label"], "Core extent");
        assert_eq!(btns[1]["label"], "Full extent");
        // each button is a relayout of the MapLibre `map` subplot center + zoom
        let core_args = &btns[0]["args"][0];
        assert!(core_args["map.center"]["lat"].is_number());
        assert!(core_args["map.center"]["lon"].is_number());
        let cz = core_args["map.zoom"].as_f64().expect("core zoom");
        let fz = btns[1]["args"][0]["map.zoom"].as_f64().expect("full zoom");
        // the full extent is larger, so its fit zoom is further out (<=) than the core's
        assert!(
            fz <= cz,
            "full-extent zoom {fz} should be <= core-extent zoom {cz}"
        );
    }

    #[cfg(feature = "geocode")]
    #[test]
    fn dashed_box_has_gaps_within_bounds() {
        let e = MapExtent {
            min_lat: 40.0,
            max_lat: 42.0,
            min_lon: -76.0,
            max_lon: -74.0,
        };
        let (lats, lons) = dashed_box_latlon(&e);
        assert_eq!(lats.len(), lons.len());
        // NaN breaks produce the dash gaps
        assert!(lats.iter().any(|v| v.is_nan()), "expected gap (NaN) breaks");
        // every finite dash vertex lies on the box bounds (a small epsilon for float math)
        for (&la, &lo) in lats.iter().zip(&lons) {
            if la.is_nan() {
                assert!(lo.is_nan(), "lat/lon gaps must be paired");
                continue;
            }
            assert!((40.0 - 1e-9..=42.0 + 1e-9).contains(&la));
            assert!((-76.0 - 1e-9..=-74.0 + 1e-9).contains(&lo));
        }
    }

    #[cfg(feature = "geocode")]
    #[test]
    fn antimeridian_guard() {
        let crossing = MapExtent {
            min_lat: -10.0,
            max_lat: 10.0,
            min_lon: -179.0,
            max_lon: 179.0,
        };
        let local = MapExtent {
            min_lat: 40.0,
            max_lat: 41.0,
            min_lon: -74.5,
            max_lon: -73.5,
        };
        assert!(extent_spans_antimeridian(&crossing));
        assert!(!extent_spans_antimeridian(&local));
    }

    #[test]
    fn classify_null_skipped() {
        assert!(classify(0, &stat("NULL", 0, None)).is_none());
    }

    #[test]
    fn classify_boolean_is_bar() {
        assert!(matches!(
            classify(0, &stat("Boolean", 2, Some(0.001))),
            Some(PanelKind::FreqBar { idx: 0 })
        ));
    }

    #[test]
    fn classify_near_unique_integer_id_skipped() {
        // a sequential id column (Integer, ~100% unique) is not meaningful to chart
        assert!(classify(0, &stat("Integer", 1000, Some(1.0))).is_none());
    }

    #[test]
    fn classify_low_card_numeric_is_bar_not_box() {
        // a 1-5 rating has quartiles but should be a frequency bar, not a box plot
        let mut s = stat("Integer", 5, Some(0.05));
        s.q1 = Some(2.0);
        s.q2_median = Some(3.0);
        s.q3 = Some(4.0);
        assert!(matches!(
            classify(2, &s),
            Some(PanelKind::FreqBar { idx: 2 })
        ));
    }

    #[test]
    fn classify_continuous_float_is_box() {
        // a near-unique continuous float (e.g. measurements) is a box plot, not skipped
        let mut s = stat("Float", 500, Some(0.99));
        s.q1 = Some(1.0);
        s.q2_median = Some(2.0);
        s.q3 = Some(3.0);
        s.min = Some("0.5".to_string());
        s.max = Some("3.5".to_string());
        assert!(matches!(classify(1, &s), Some(PanelKind::BoxStats { .. })));
    }

    fn dict_row(content_type: &str, role: &str, concept: &str, label: &str) -> DictRow {
        DictRow {
            content_type: content_type.to_string(),
            role: role.to_string(),
            concept: concept.to_string(),
            label: label.to_string(),
            ..Default::default()
        }
    }

    #[test]
    fn route_from_concept_maps_namespaces() {
        // coordinates -> map; other geo keys (census_tract, zip, ...) -> dimension bar
        assert_eq!(
            route_from_concept("geo.latitude"),
            Some((Route::MapCoord, None))
        );
        assert_eq!(
            route_from_concept("geo.coordinate_pair"),
            Some((Route::MapCoord, None))
        );
        assert_eq!(
            route_from_concept("geo.census_tract"),
            Some((Route::Dimension, None))
        );
        // state-plane planar coordinates: neither mappable nor a place key -> ProjectedCoord,
        // which charts a distribution only when no point map rendered (see
        // `classify_with_semantics_routes`)
        assert_eq!(
            route_from_concept("geo.crs_stateplane_x"),
            Some((Route::ProjectedCoord, None))
        );
        assert_eq!(
            route_from_concept("geo.crs_stateplane_y"),
            Some((Route::ProjectedCoord, None))
        );
        assert_eq!(
            route_from_concept("nyc.bbl"),
            Some((Route::Dimension, None))
        );
        // temporal
        assert_eq!(
            route_from_concept("time.created_at"),
            Some((Route::Temporal, None))
        );
        // identifiers / PII -> skip
        assert_eq!(
            route_from_concept("id.surrogate_key"),
            Some((Route::Skip, None))
        );
        assert_eq!(route_from_concept("pii.email"), Some((Route::Skip, None)));
        // measures: amount/count additive (Sum), ratio averaged (Mean)
        assert_eq!(
            route_from_concept("measure.amount"),
            Some((Route::Measure, Some(Agg::Sum)))
        );
        assert_eq!(
            route_from_concept("measure.count"),
            Some((Route::Measure, Some(Agg::Sum)))
        );
        assert_eq!(
            route_from_concept("measure.ratio"),
            Some((Route::Measure, Some(Agg::Mean)))
        );
        // unknown namespace / bare unknown -> None (fall through to role)
        assert_eq!(route_from_concept("weather.temp"), None);
        assert_eq!(route_from_concept("unknown"), None);
    }

    #[test]
    fn intensive_measure_detection() {
        // intensive quantities -> averaged, matched by label or field
        assert!(is_intensive_measure(
            "Average Annual Temperature (\u{b0}C)",
            "avg_annual_temp_c"
        ));
        assert!(is_intensive_measure("Elevation", "elevation_m"));
        assert!(is_intensive_measure("Literacy", "literacy_pct"));
        assert!(is_intensive_measure("Happiness Index", "happiness_index"));
        assert!(is_intensive_measure("Population Density", "pop_density"));
        // additive amounts/counts stay additive
        assert!(!is_intensive_measure(
            "Metro Population (millions)",
            "metro_population_m"
        ));
        assert!(!is_intensive_measure("Revenue", "revenue"));
        assert!(!is_intensive_measure("Packages", "packages"));
        assert!(!is_intensive_measure("Order Count", "order_count"));
        // a COUNT wins over an embedded intensive token — it is additive by definition
        assert!(!is_intensive_measure(
            "Review Score Count",
            "review_score_count"
        ));
        assert!(!is_intensive_measure("Index Count", "index_count"));
        assert!(!is_intensive_measure("Number of Ratings", "num_ratings"));
        // camelCase / PascalCase count fields are recognized even with an empty/unhelpful label
        assert!(!is_intensive_measure("", "reviewScoreCount"));
        assert!(!is_intensive_measure("", "indexCount"));
        assert!(!is_intensive_measure("", "numRatings"));
        // the count guard is token-aware: intensive fields that merely EMBED "count" as a
        // substring (discount, account, county) are still detected as intensive, in snake_case
        // and camelCase alike
        assert!(is_intensive_measure("Discount", "discount_pct"));
        assert!(is_intensive_measure("Account Score", "account_score"));
        assert!(is_intensive_measure("County Index", "county_index"));
        assert!(is_intensive_measure("", "discountPct"));
        assert!(is_intensive_measure("", "accountScore"));
    }

    #[test]
    fn derive_semantics_averages_intensive_measure() {
        let numeric = stat("Float", 300, Some(0.9));
        // temperature tagged as an additive amount by the coarse vocab -> downgraded to Mean
        let temp = derive_semantics(
            &numeric,
            Some(&dict_row(
                "",
                "measure",
                "measure.amount",
                "Average Annual Temperature (\u{b0}C)",
            )),
        );
        assert_eq!(temp.route, Route::Measure);
        assert_eq!(temp.agg, Some(Agg::Mean));
        // a genuine additive amount keeps Sum
        let pop = derive_semantics(
            &numeric,
            Some(&dict_row(
                "",
                "measure",
                "measure.amount",
                "Metro Population",
            )),
        );
        assert_eq!(pop.agg, Some(Agg::Sum));
        // an explicit measure.count stays additive even when its label embeds an intensive token
        // ("score"/"index") — a count is summed, never averaged.
        let counts = stat("Integer", 300, Some(0.9));
        let score_count = derive_semantics(
            &counts,
            Some(&dict_row(
                "",
                "measure",
                "measure.count",
                "Review Score Count",
            )),
        );
        assert_eq!(score_count.agg, Some(Agg::Sum));
    }

    #[test]
    fn route_from_content_type_legacy_mapping() {
        assert_eq!(route_from_content_type("category").0, Route::Dimension);
        assert_eq!(route_from_content_type("state_abbr").0, Route::Dimension);
        assert_eq!(route_from_content_type("latitude").0, Route::MapCoord);
        // temporal tokens, with/without an strftime or length suffix
        assert_eq!(route_from_content_type("date").0, Route::Temporal);
        assert_eq!(
            route_from_content_type("datetime:%Y-%m-%dT%H:%M:%S").0,
            Route::Temporal
        );
        assert_eq!(route_from_content_type("duration:3600").0, Route::Temporal);
        // identifiers / PII / free-text -> skip
        assert_eq!(route_from_content_type("unique_id").0, Route::Skip);
        assert_eq!(route_from_content_type("email").0, Route::Skip);
        assert_eq!(route_from_content_type("free_text").0, Route::Skip);
        // explicit / empty -> defer to stats
        assert_eq!(route_from_content_type("unknown").0, Route::Defer);
        assert_eq!(route_from_content_type("").0, Route::Defer);
    }

    #[test]
    fn dict_dmy_preference_reads_the_declared_date_format() {
        let with_ct = |ct: &str| DictRow {
            content_type: ct.to_string(),
            ..DictRow::default()
        };

        // no dictionary row at all -> fall back to the env preference
        assert_eq!(dict_dmy_preference(None), None);

        // declared month-first / day-first formats are authoritative
        assert_eq!(
            dict_dmy_preference(Some(&with_ct("date:%m/%d/%Y"))),
            Some(false)
        );
        assert_eq!(
            dict_dmy_preference(Some(&with_ct("date:%d/%m/%Y"))),
            Some(true)
        );
        // the format's own colons must not truncate it
        assert_eq!(
            dict_dmy_preference(Some(&with_ct("datetime:%m/%d/%Y %I:%M:%S %p"))),
            Some(false)
        );
        // content_type is stored verbatim, so leading whitespace must be trimmed
        assert_eq!(
            dict_dmy_preference(Some(&with_ct("  datetime:%m/%d/%Y %I:%M:%S %p  "))),
            Some(false)
        );

        // no actionable preference -> fall back to the env preference
        assert_eq!(dict_dmy_preference(Some(&with_ct("date"))), None);
        assert_eq!(dict_dmy_preference(Some(&with_ct("date:"))), None);
        assert_eq!(dict_dmy_preference(Some(&with_ct("date:%Y-%m-%d"))), None);
        assert_eq!(dict_dmy_preference(Some(&with_ct("date:%Q"))), None); // invalid strftime
        assert_eq!(dict_dmy_preference(Some(&with_ct("time:%H:%M"))), None); // not date/datetime
        assert_eq!(dict_dmy_preference(Some(&with_ct("unknown"))), None);
        assert_eq!(dict_dmy_preference(Some(&with_ct(""))), None);
    }

    #[test]
    fn derive_semantics_precedence_and_label() {
        // no dictionary row -> Defer (stats-only behavior unchanged)
        assert_eq!(
            derive_semantics(&stat("Integer", 5, None), None).route,
            Route::Defer
        );

        // concept beats a defaulted `measure` role: a numeric census_tract that describegpt left
        // role=measure is still routed to a Dimension bar by its geo.census_tract concept.
        let tract = stat("Integer", 33, Some(0.00003));
        let row = dict_row("", "measure", "geo.census_tract", "Census Tract");
        let sem = derive_semantics(&tract, Some(&row));
        assert_eq!(sem.route, Route::Dimension);
        assert_eq!(sem.label, "Census Tract"); // label carried for the panel title

        // role used when concept is absent/unknown
        let r = derive_semantics(
            &stat("String", 8, Some(0.01)),
            Some(&dict_row("", "dimension", "", "")),
        );
        assert_eq!(r.route, Route::Dimension);

        // content_type used when neither concept nor role resolve
        let c = derive_semantics(
            &stat("Float", 99, Some(0.5)),
            Some(&dict_row("latitude", "", "", "")),
        );
        assert_eq!(c.route, Route::MapCoord);

        // all-empty dictionary row -> Defer, but the label still rides along
        let d = derive_semantics(
            &stat("String", 9, Some(0.02)),
            Some(&dict_row("", "", "", "Notes")),
        );
        assert_eq!(d.route, Route::Defer);
        assert_eq!(d.label, "Notes");
    }

    fn csem(concept: &str) -> ColSemantics {
        ColSemantics {
            concept: concept.to_string(),
            ..Default::default()
        }
    }

    // a numeric measure stat with an explicit coefficient of variation (for the statistical
    // hover-combination picker).
    fn measure_stat(cv: f64) -> crate::cmd::stats::StatsData {
        crate::cmd::stats::StatsData {
            r#type: "Float".to_string(),
            cardinality: 100,
            uniqueness_ratio: Some(0.5),
            cv: Some(cv),
            ..Default::default()
        }
    }

    #[test]
    fn lorenz_curve_all_equal_is_diagonal() {
        // a perfectly equal distribution traces the equality diagonal: share == pop at every point
        let (pop, share) = lorenz_curve(vec![5.0; 4]).unwrap();
        assert_eq!(pop.len(), share.len());
        for (p, s) in pop.iter().zip(&share) {
            assert!(
                (p - s).abs() < 1e-12,
                "equal dist should lie on the diagonal"
            );
        }
        assert_eq!((pop[0], share[0]), (0.0, 0.0));
        assert!((pop.last().unwrap() - 1.0).abs() < 1e-12);
        assert!((share.last().unwrap() - 1.0).abs() < 1e-12);
    }

    #[test]
    fn lorenz_curve_max_inequality_hugs_axis() {
        // one entity holds everything: the bottom (n-1)/n of records hold 0% of the total, so the
        // curve hugs the x-axis then jumps to (1,1) — maximal inequality. Zeros are legitimate.
        let (pop, share) = lorenz_curve(vec![0.0, 0.0, 0.0, 100.0]).unwrap();
        let last = share.len() - 1;
        assert_eq!(share[last - 1], 0.0);
        assert!((share[last] - 1.0).abs() < 1e-12);
        assert!((pop[last] - 1.0).abs() < 1e-12);
    }

    #[test]
    fn lorenz_curve_rejects_degenerate() {
        assert!(lorenz_curve(vec![]).is_none());
        assert!(lorenz_curve(vec![42.0]).is_none()); // a single value can't form a curve
        assert!(lorenz_curve(vec![0.0, 0.0, 0.0]).is_none()); // zero total
    }

    #[test]
    fn lorenz_curve_thins_to_cap() {
        // a large column is thinned to at most LORENZ_CURVE_MAX_POINTS vertices, endpoints kept
        let values: Vec<f64> = (1..=(LORENZ_CURVE_MAX_POINTS as u32 * 3))
            .map(f64::from)
            .collect();
        let (pop, share) = lorenz_curve(values).unwrap();
        assert!(pop.len() <= LORENZ_CURVE_MAX_POINTS);
        assert_eq!((pop[0], share[0]), (0.0, 0.0));
        assert!((pop.last().unwrap() - 1.0).abs() < 1e-12);
        assert!((share.last().unwrap() - 1.0).abs() < 1e-12);
    }

    #[test]
    fn is_inequality_candidate_gates() {
        let stat = |field: &str, gini: Option<f64>| crate::cmd::stats::StatsData {
            r#type: "Float".to_string(),
            field: field.to_string(),
            cardinality: 100,
            gini_coefficient: gini,
            ..Default::default()
        };
        let sem = |agg: Option<Agg>, label: &str| ColSemantics {
            route: Route::Measure,
            agg,
            label: label.to_string(),
            ..Default::default()
        };

        // dictionary additive amount + high Gini -> candidate
        assert!(is_inequality_candidate(
            &sem(Some(Agg::Sum), "revenue"),
            &stat("revenue", Some(0.62))
        ));
        // dictionary intensive (ratio -> Mean) -> never a candidate, whatever the Gini
        assert!(!is_inequality_candidate(
            &sem(Some(Agg::Mean), "growth_rate"),
            &stat("growth_rate", Some(0.90))
        ));
        // no dictionary: additive-looking field name + high Gini -> candidate
        assert!(is_inequality_candidate(
            &sem(None, ""),
            &stat("net_worth", Some(0.75))
        ));
        // no dictionary: intensive field name (temperature) -> not a candidate
        assert!(!is_inequality_candidate(
            &sem(None, ""),
            &stat("avg_temperature", Some(0.75))
        ));
        // Gini below the threshold -> not a candidate
        assert!(!is_inequality_candidate(
            &sem(Some(Agg::Sum), "revenue"),
            &stat("revenue", Some(0.40))
        ));
        // Gini absent (plain viz smart, no --smarter) -> never a candidate
        assert!(!is_inequality_candidate(
            &sem(Some(Agg::Sum), "revenue"),
            &stat("revenue", None)
        ));

        // no dictionary: a skewed near-unique numeric ID/code must NOT get a Lorenz panel even at
        // high Gini (the id/code/key NAME excludes it — the Gini gate alone would not).
        assert!(!is_inequality_candidate(
            &sem(None, ""),
            &stat("customer_id", Some(0.72))
        ));
        assert!(!is_inequality_candidate(
            &sem(None, ""),
            &stat("productCode", Some(0.72))
        ));
        // FLAGSHIP (must not regress): a non-id-named additive amount with high Gini stays a
        // candidate on the stats-only path even though such a column is typically near-unique.
        assert!(is_inequality_candidate(
            &sem(None, ""),
            &stat("population", Some(0.60))
        ));
        // a dictionary-tagged additive measure is NEVER excluded by the id-name heuristic (the
        // name guard is stats-only): an explicit Agg::Sum wins even with an id-ish name.
        assert!(is_inequality_candidate(
            &sem(Some(Agg::Sum), "grant_id"),
            &stat("grant_id", Some(0.60))
        ));
    }

    #[test]
    fn is_identifier_name_token_boundary() {
        // id/code/key/uuid tokens (incl. camelCase + suffixes) are identifiers
        assert!(is_identifier_name("", "customer_id"));
        assert!(is_identifier_name("Customer ID", "customerId"));
        assert!(is_identifier_name("", "product_code"));
        assert!(is_identifier_name("", "order_key"));
        assert!(is_identifier_name("", "record_uuid"));
        assert!(is_identifier_name("", "guid"));
        // numbered twins: a trailing digit run is stripped per token before matching
        assert!(is_identifier_name("", "customer_id2"));
        assert!(is_identifier_name("", "code1"));
        assert!(is_identifier_name("", "productCode2"));
        assert!(is_identifier_name("", "order_key3"));
        // substrings inside real words must NOT trip it (token-boundary, not substring)
        assert!(!is_identifier_name("", "grid"));
        assert!(!is_identifier_name("", "medicaid_payment"));
        assert!(!is_identifier_name("", "decode_ratio"));
        assert!(!is_identifier_name("County", "county"));
        assert!(!is_identifier_name("", "population"));
        assert!(!is_identifier_name("Net Worth", "net_worth"));
        // a digit-suffixed non-identifier token must still be admitted (co2 -> "co", not an id)
        assert!(!is_identifier_name("", "co2_emissions"));
    }

    #[test]
    fn select_map_identifier_dictionary_beats_stats() {
        // idx0 has an id.* concept; idx3 is a near-unique String. The dictionary concept wins.
        let stats = vec![
            stat("Integer", 1000, Some(0.99)), // 0: id column
            stat("Float", 900, Some(0.5)),     // 1: lat
            stat("Float", 900, Some(0.5)),     // 2: lon
            stat("String", 1000, Some(0.99)),  // 3: a near-unique name
        ];
        let sems = vec![
            csem("id.natural_key"),
            csem("geo.latitude"),
            csem("geo.longitude"),
            csem(""),
        ];
        assert_eq!(select_map_identifier(&stats, &sems, None, 1, 2), Some(0));
    }

    #[test]
    fn select_map_identifier_name_like_concept() {
        // no id.* concept -> a name-like concept (geo.city) is the identifier.
        let stats = vec![
            stat("Float", 9, Some(0.5)),   // 0: lat
            stat("Float", 9, Some(0.5)),   // 1: lon
            stat("String", 50, Some(0.6)), // 2: city
            stat("Integer", 5, Some(0.1)), // 3: a code
        ];
        let sems = vec![
            csem("geo.latitude"),
            csem("geo.longitude"),
            csem("geo.city"),
            csem(""),
        ];
        assert_eq!(select_map_identifier(&stats, &sems, None, 0, 1), Some(2));
    }

    #[test]
    fn select_map_identifier_statistical_prefers_string_over_integer() {
        // no dictionary concepts: the near-unique String (a readable name) beats the near-unique
        // Integer (an opaque key); lat/lon are excluded even though near-unique.
        let stats = vec![
            stat("Integer", 1000, Some(0.99)), // 0: opaque key
            stat("Float", 900, Some(0.99)),    // 1: lat (near-unique, excluded)
            stat("Float", 900, Some(0.99)),    // 2: lon (near-unique, excluded)
            stat("String", 1000, Some(0.99)),  // 3: a readable name
        ];
        let sems = vec![csem(""), csem(""), csem(""), csem("")];
        assert_eq!(select_map_identifier(&stats, &sems, None, 1, 2), Some(3));
    }

    #[test]
    fn select_map_identifier_none_when_nothing_suitable() {
        // all low-cardinality numerics, no concepts -> no identifier.
        let stats = vec![
            stat("Float", 9, Some(0.5)),   // 0: lat
            stat("Float", 9, Some(0.5)),   // 1: lon
            stat("Integer", 5, Some(0.1)), // 2: a low-card code
        ];
        let sems = vec![csem(""), csem(""), csem("")];
        assert_eq!(select_map_identifier(&stats, &sems, None, 0, 1), None);
    }

    /// `--photos` column detection reads only the stats cache, so it must recognize the real
    /// shapes the Boston 311 feed produces: a bare image URL, one carrying the `#spot=<uuid>`
    /// fragment, and a `|`-joined multi-photo cell.
    #[test]
    fn looks_like_image_url_accepts_real_shapes() {
        assert!(looks_like_image_url("https://example.org/a/b/photo.jpg"));
        assert!(looks_like_image_url("http://example.org/photo.PNG"));
        assert!(looks_like_image_url(
            "https://spot-boston-res.cloudinary.com/image/upload/v1/boston/x.jpg#spot=abc-123"
        ));
        assert!(looks_like_image_url("https://example.org/p.webp?w=200"));
        // multi-value cell: the FIRST segment decides
        assert!(looks_like_image_url(
            "https://example.org/a.jpg#spot=1 | https://example.org/b.jpg#spot=2"
        ));
    }

    /// Non-image URLs and non-URLs must not qualify — a false positive would put a broken
    /// `<img>` in the lightbox and, worse, make the page reference an unrelated host.
    #[test]
    fn looks_like_image_url_rejects_non_images() {
        assert!(!looks_like_image_url(""));
        assert!(!looks_like_image_url("(NULL)"));
        assert!(!looks_like_image_url("https://example.org/page.html"));
        assert!(!looks_like_image_url("https://example.org/report.pdf"));
        assert!(!looks_like_image_url("https://data.boston.gov/dataset/311"));
        // a bare filename is not a fetchable source
        assert!(!looks_like_image_url("photo.jpg"));
        // non-http schemes are never embedded
        assert!(!looks_like_image_url("file:///tmp/photo.jpg"));
        assert!(!looks_like_image_url("javascript:alert(1)//x.jpg"));
    }

    /// Each cell is flattened into its individual URLs with the `#fragment` dropped but the
    /// `?query` PRESERVED (presigned links need it), and non-image segments are discarded rather
    /// than emitted as broken sources.
    #[test]
    fn split_image_urls_flattens_and_strips() {
        assert_eq!(
            split_image_urls(
                "https://e.org/a.jpg#spot=1 | https://e.org/b.jpg#spot=2 | https://e.org/c.png"
            ),
            vec![
                "https://e.org/a.jpg".to_string(),
                "https://e.org/b.jpg".to_string(),
                "https://e.org/c.png".to_string(),
            ]
        );
        // a required query string is KEPT (S3/CloudFront presigned auth) — only the fragment goes
        assert_eq!(
            split_image_urls("https://e.org/a.jpg?X-Amz-Signature=abc123&e=900"),
            vec!["https://e.org/a.jpg?X-Amz-Signature=abc123&e=900".to_string()]
        );
        assert_eq!(
            split_image_urls("https://e.org/a.jpg?w=1#frag"),
            vec!["https://e.org/a.jpg?w=1".to_string()]
        );
        // a cell with nothing image-like contributes no photos at all
        assert!(split_image_urls("(NULL)").is_empty());
        assert!(split_image_urls("").is_empty());
        // mixed: only the image segment survives
        assert_eq!(
            split_image_urls("https://e.org/page.html | https://e.org/a.gif"),
            vec!["https://e.org/a.gif".to_string()]
        );
    }

    /// Detection is driven by the stats cache's `min`/`max` (the only real cell values it carries),
    /// which is what lets `--photos` work with no Data Dictionary. An all-NULL column — Boston
    /// 311's `submitted_photo` — has no min/max and so is deliberately NOT detected: tagging it
    /// would produce a lightbox that never loads.
    #[test]
    fn image_url_cols_detects_from_stats_min_max() {
        let mut photo = stat("String", 100, Some(0.9));
        photo.min = Some("https://e.org/a.jpg#spot=1".to_string());
        photo.max = Some("https://e.org/z.jpg#spot=9".to_string());
        let mut all_null = stat("NULL", 1, None);
        all_null.min = None;
        all_null.max = None;
        let mut link = stat("String", 50, Some(0.5));
        link.min = Some("https://e.org/page.html".to_string());
        link.max = Some("https://e.org/other.html".to_string());
        let stats = vec![stat("Float", 9, Some(0.5)), all_null, photo, link];
        assert_eq!(image_url_cols(&stats), vec![2]);
    }

    /// Only ONE of min/max needs to look like an image: a column whose lexicographic extremes
    /// straddle a null sentinel still qualifies on the other end.
    #[test]
    fn image_url_cols_detects_on_either_bound() {
        let mut s = stat("String", 10, Some(0.5));
        s.min = Some("(NULL)".to_string());
        s.max = Some("https://e.org/z.jpeg".to_string());
        assert_eq!(image_url_cols(&[s]), vec![0]);
    }

    #[test]
    fn select_map_hover_fields_default_is_identifier_only() {
        let stats = vec![
            stat("Float", 9, Some(0.5)),
            stat("Float", 9, Some(0.5)),
            stat("String", 1000, Some(0.99)),
        ];
        let sems = vec![csem("geo.latitude"), csem("geo.longitude"), csem("")];
        // smarter = false -> just the identifier
        assert_eq!(
            select_map_hover_fields(&stats, &sems, None, 0, 1, false),
            vec![2]
        );
    }

    #[test]
    fn select_map_hover_fields_smarter_dictionary_combination() {
        // id + measure + category, in that order, capped.
        let stats = vec![
            stat("String", 1000, Some(0.99)), // 0: id
            stat("Float", 9, Some(0.5)),      // 1: lat
            stat("Float", 9, Some(0.5)),      // 2: lon
            stat("Float", 100, Some(0.5)),    // 3: measure
            stat("String", 4, Some(0.1)),     // 4: category
        ];
        let sems = vec![
            csem("id.natural_key"),
            csem("geo.latitude"),
            csem("geo.longitude"),
            csem("measure.amount"),
            csem("category.status"),
        ];
        let dict = DictData::default();
        assert_eq!(
            select_map_hover_fields(&stats, &sems, Some(&dict), 1, 2, true),
            vec![0, 3, 4]
        );
    }

    #[test]
    fn select_map_hover_fields_smarter_statistical_combination() {
        // no dictionary: id (near-unique String) + highest-CV measure + lowest-cardinality
        // category.
        let stats = vec![
            stat("String", 1000, Some(0.99)), // 0: id
            stat("Float", 9, Some(0.5)),      // 1: lat
            stat("Float", 9, Some(0.5)),      // 2: lon
            measure_stat(0.2),                // 3: low-CV measure
            measure_stat(0.9),                // 4: high-CV measure (preferred)
            stat("String", 4, Some(0.1)),     // 5: category
        ];
        let sems = vec![csem(""), csem(""), csem(""), csem(""), csem(""), csem("")];
        assert_eq!(
            select_map_hover_fields(&stats, &sems, None, 1, 2, true),
            vec![0, 4, 5]
        );
    }

    #[test]
    fn format_and_assemble_map_hover() {
        // dataset line: bold id + escaped extra
        let line = format_map_dataset_line(
            Some("Tokyo"),
            &[("Magnitude".to_string(), "5.2".to_string())],
        );
        assert_eq!(line, "<b>Tokyo</b><br>Magnitude: 5.2");
        // markup in values is escaped
        assert_eq!(
            format_map_dataset_line(Some("<b>x"), &[]),
            "<b>&lt;b&gt;x</b>"
        );

        // with an identifier, the geocoded place is a context line (no coord line — that's
        // appended at render by MAP_HOVER_TEMPLATE from the lat/lon data arrays)
        assert_eq!(
            assemble_map_hover("<b>Tokyo</b>", true, "Tokyo, Japan", 0),
            "<b>Tokyo</b><br>Tokyo, Japan"
        );
        // no identifier column: the geocoded place becomes the bold identifier
        assert_eq!(
            assemble_map_hover("", false, "Tokyo, Japan", 0),
            "<b>Tokyo, Japan</b>"
        );
        // empty when neither dataset nor geocode contributed — the template's coordinate
        // hover stands alone
        assert_eq!(assemble_map_hover("", false, "", 0), "");

        // --photos affordance: only points that HAVE photos get the hint line, and it is the
        // last line so the identifier still leads. Singular/plural both covered.
        assert_eq!(
            assemble_map_hover("<b>Tokyo</b>", true, "", 1),
            "<b>Tokyo</b><br><i>1 photo - keep hovering to view</i>"
        );
        assert_eq!(
            assemble_map_hover("<b>Tokyo</b>", true, "Tokyo, Japan", 3),
            "<b>Tokyo</b><br>Tokyo, Japan<br><i>3 photos - keep hovering to view</i>"
        );
        // a photo-less point is untouched, so the ~69% of Boston 311 cases closed without one
        // keep a clean hover
        assert_eq!(
            assemble_map_hover("<b>Tokyo</b>", true, "", 0),
            "<b>Tokyo</b>"
        );
        // photos with no dataset line and no place: the hint stands alone above the coords
        assert_eq!(
            assemble_map_hover("", true, "", 2),
            "<i>2 photos - keep hovering to view</i>"
        );

        // map_hover_text_values: non-empty entries get the trailing <br> separator the
        // template relies on; empty entries stay empty (no leading blank line)
        assert_eq!(
            map_hover_text_values(&["<b>Tokyo</b>".to_string(), String::new()]),
            vec!["<b>Tokyo</b><br>".to_string(), String::new()]
        );
    }

    #[test]
    fn bdata_typed_array_roundtrip() {
        // below the size floor: left as a plain JSON array
        let mut small = serde_json::json!([1.0, 2.0]);
        bdata_encode_f32(&mut small);
        assert!(small.is_array(), "short arrays must stay plain JSON");

        // at the floor: becomes {"dtype":"float32","bdata":..} whose little-endian bytes
        // round-trip each value within f32 precision (~1e-4 deg at lat/lon magnitudes)
        let vals: Vec<f64> = (0..BDATA_MIN_LEN)
            .map(|i| 40.0 + i as f64 * 0.000_123_4)
            .collect();
        let mut v = serde_json::to_value(&vals).unwrap();
        bdata_encode_f32(&mut v);
        assert_eq!(v["dtype"], "float32");
        let bytes = base64_simd::STANDARD
            .decode_to_vec(v["bdata"].as_str().unwrap())
            .unwrap();
        assert_eq!(bytes.len(), vals.len() * 4);
        for (i, chunk) in bytes.chunks_exact(4).enumerate() {
            let f = f32::from_le_bytes([chunk[0], chunk[1], chunk[2], chunk[3]]);
            assert!(
                (f64::from(f) - vals[i]).abs() < 1e-4,
                "coordinate {i} drifted beyond f32 tolerance"
            );
        }

        // a non-numeric entry keeps the plain array (typed arrays can't carry gaps)
        let mut mixed = serde_json::to_value(vec![1.0_f64; BDATA_MIN_LEN]).unwrap();
        mixed.as_array_mut().unwrap()[10] = serde_json::Value::Null;
        bdata_encode_f32(&mut mixed);
        assert!(mixed.is_array(), "arrays with nulls must stay plain JSON");
    }

    #[test]
    fn map_panel_inline_html_encodes_and_escapes() {
        // a figure big enough to cross BDATA_MIN_LEN, with hover text containing a script
        // terminator — the emitted inline HTML must carry typed arrays and must never contain
        // a literal `</script>` inside the embedded JSON (askama-equivalent `\u003c` escaping).
        let n = BDATA_MIN_LEN;
        let lats: Vec<f64> = (0..n).map(|i| 40.0 + i as f64 * 0.001).collect();
        let lons: Vec<f64> = (0..n).map(|i| -80.0 + i as f64 * 0.001).collect();
        let texts: Vec<String> = (0..n).map(|_| "</script>x".to_string()).collect();
        let mut plot = Plot::new();
        plot.add_trace(
            ScatterMap::new(lats, lons)
                .mode(Mode::Markers)
                .text_array(texts)
                .hover_template(MAP_HOVER_TEMPLATE),
        );
        let html = map_panel_inline_html(&plot, "qsv-viz-panel-test");
        assert!(
            html.contains(r#""dtype":"float32""#) && html.contains(r#""bdata":""#),
            "lat/lon must embed as base64 typed arrays; html: {html}"
        );
        // exactly the two structural tags — none leaked from the embedded data
        assert_eq!(html.matches("</script>").count(), 1);
        assert_eq!(html.matches("<script").count(), 1);
        assert!(html.contains(r#"Plotly.newPlot("qsv-viz-panel-test""#));
    }

    #[cfg(feature = "geocode")]
    #[test]
    fn geocode_hover_place_gap_fills() {
        let label = crate::cmd::geocode::GeoLabel {
            city: "Kinshasa".to_string(),
            admin1: "Kinshasa City".to_string(),
            country: "Democratic Republic of Congo".to_string(),
            ..Default::default()
        };
        // nothing suppressed, empty dataset line -> the full place
        assert_eq!(
            geocode_hover_place(&label, false, false, false, false, ""),
            "Kinshasa, Kinshasa City, Democratic Republic of Congo"
        );
        // the city already appears in the dataset line (it's the identifier) -> dropped
        assert_eq!(
            geocode_hover_place(&label, false, false, false, false, "<b>Kinshasa</b>"),
            "Kinshasa City, Democratic Republic of Congo"
        );
        // concept suppression (a chosen geo.country column) drops the country
        assert_eq!(
            geocode_hover_place(&label, false, false, true, false, ""),
            "Kinshasa, Kinshasa City"
        );

        // county (admin2) is always-on, between admin1 and country
        let us = crate::cmd::geocode::GeoLabel {
            city: "Pittsburgh".to_string(),
            admin1: "Pennsylvania".to_string(),
            country: "United States".to_string(),
            admin2: "Allegheny County".to_string(),
            us_state_fips: "42".to_string(),
            us_county_fips: "42003".to_string(),
            ..Default::default()
        };
        assert_eq!(
            geocode_hover_place(&us, false, false, false, false, ""),
            "Pittsburgh, Allegheny County, Pennsylvania, United States"
        );
        // FIPS appended only when verbose (--smarter): prefer the 5-digit county FIPS
        assert_eq!(
            geocode_hover_place(&us, false, false, false, true, ""),
            "Pittsburgh, Allegheny County, Pennsylvania, United States (FIPS 42003)"
        );
        // verbose but only a state FIPS -> the 2-digit state FIPS
        let state_only = crate::cmd::geocode::GeoLabel {
            city: "Somewhere".to_string(),
            admin1: "Pennsylvania".to_string(),
            country: "United States".to_string(),
            us_state_fips: "42".to_string(),
            ..Default::default()
        };
        assert_eq!(
            geocode_hover_place(&state_only, false, false, false, true, ""),
            "Somewhere, Pennsylvania, United States (FIPS 42)"
        );
        // non-US place under verbose -> no FIPS tail
        assert_eq!(
            geocode_hover_place(&label, false, false, false, true, ""),
            "Kinshasa, Kinshasa City, Democratic Republic of Congo"
        );
    }

    #[test]
    fn guardrail_downgrades_only_code_like_measures() {
        // a low-cardinality integer over many rows, role-defaulted to measure with no concept ->
        // downgraded to a Dimension bar (the fact-#2 defense).
        let code = stat("Integer", 6, Some(0.00001));
        assert_eq!(
            derive_semantics(&code, Some(&dict_row("", "measure", "", ""))).route,
            Route::Dimension
        );
        // a tiny fixed integer scale (ordinal rating) is barred even with an EXPLICIT measure.*
        // concept: a 1-5 satisfaction score lands on measure.count/amount but charts far better as
        // a frequency bar of its levels than as a box plot.
        assert_eq!(
            derive_semantics(&code, Some(&dict_row("", "measure", "measure.count", ""))).route,
            Route::Dimension
        );
        let rating = stat("Integer", 5, Some(0.01));
        assert_eq!(
            derive_semantics(
                &rating,
                Some(&dict_row("", "measure", "measure.amount", ""))
            )
            .route,
            Route::Dimension
        );
        // boundary: a 1-5 rating over exactly 100 rows -> uniqueness ratio == 0.05. The `<=`
        // threshold bars it (a `<` would leave it a box).
        assert_eq!(
            derive_semantics(
                &stat("Integer", 5, Some(0.05)),
                Some(&dict_row("", "measure", "measure.amount", ""))
            )
            .route,
            Route::Dimension
        );
        // an EXPLICIT measure.* concept whose cardinality is above RATING_MAX_CARDINALITY but still
        // categorical (13..=30) is trusted as a genuine measure -> stays Measure. The tighter
        // rating threshold only bars a handful of discrete levels, unlike the role-defaulted path.
        assert_eq!(
            derive_semantics(
                &stat("Integer", 20, Some(0.001)),
                Some(&dict_row("", "measure", "measure.count", ""))
            )
            .route,
            Route::Measure
        );
        // a float is ~never a code -> stays Measure
        assert_eq!(
            derive_semantics(
                &stat("Float", 6, Some(0.00001)),
                Some(&dict_row("", "measure", "", ""))
            )
            .route,
            Route::Measure
        );
        // a near-unique integer (genuine continuous) -> stays Measure
        assert_eq!(
            derive_semantics(
                &stat("Integer", 100_000, Some(0.99)),
                Some(&dict_row("", "measure", "", ""))
            )
            .route,
            Route::Measure
        );
        // residual gap (documented): cardinality above CATEGORICAL_MAX_CARDINALITY is left to the
        // concept layer, NOT the guardrail -> stays Measure.
        assert_eq!(
            derive_semantics(
                &stat("Integer", 50, Some(0.00001)),
                Some(&dict_row("", "measure", "", ""))
            )
            .route,
            Route::Measure
        );
    }

    #[test]
    fn classify_with_semantics_routes() {
        // Defer falls back to classify (low-card string -> frequency bar)
        let sem_defer = ColSemantics::default();
        assert!(matches!(
            classify_with_semantics(0, &stat("String", 5, Some(0.001)), &sem_defer, false),
            Some(PanelKind::FreqBar { idx: 0 })
        ));
        // Dimension -> frequency bar even for a many-distinct integer code
        let sem_dim = ColSemantics {
            route: Route::Dimension,
            ..Default::default()
        };
        assert!(matches!(
            classify_with_semantics(2, &stat("Integer", 500, Some(0.4)), &sem_dim, false),
            Some(PanelKind::FreqBar { idx: 2 })
        ));
        // Measure -> box plot from quartiles (via classify_measure)
        let mut m = stat("Integer", 500, Some(0.4));
        m.q1 = Some(10.0);
        m.q2_median = Some(20.0);
        m.q3 = Some(30.0);
        let sem_meas = ColSemantics {
            route: Route::Measure,
            ..Default::default()
        };
        assert!(matches!(
            classify_with_semantics(3, &m, &sem_meas, false),
            Some(PanelKind::BoxStats { .. })
        ));
        // Temporal / Skip draw no per-column panel
        for route in [Route::Temporal, Route::Skip] {
            let sem = ColSemantics {
                route,
                ..Default::default()
            };
            assert!(
                classify_with_semantics(0, &stat("DateTime", 9, Some(0.6)), &sem, false).is_none()
            );
        }
        // MapCoord falls back to classify: a coordinate the map did NOT consume (the caller already
        // skips consumed ones via is_map_col) must still be charted, not vanish. A near-unique
        // float with quartiles -> box.
        let mut coord = stat("Float", 5000, Some(0.99));
        coord.q1 = Some(1.0);
        coord.q2_median = Some(2.0);
        coord.q3 = Some(3.0);
        let sem_coord = ColSemantics {
            route: Route::MapCoord,
            ..Default::default()
        };
        assert!(matches!(
            classify_with_semantics(7, &coord, &sem_coord, false),
            Some(PanelKind::BoxStats { .. })
        ));
        // ProjectedCoord (state-plane easting/northing) is gated on whether a point map rendered:
        // with a map its spatial spread is already on screen in 2-D, so no redundant 1-D box;
        // without one it falls back to classify_measure rather than vanish.
        let sem_planar = ColSemantics {
            route: Route::ProjectedCoord,
            ..Default::default()
        };
        let mut planar = stat("Integer", 8130, Some(0.81));
        planar.q1 = Some(993_041.5);
        planar.q2_median = Some(1_005_000.0);
        planar.q3 = Some(1_018_194.0);
        assert!(classify_with_semantics(8, &planar, &sem_planar, true).is_none());
        assert!(matches!(
            classify_with_semantics(8, &planar, &sem_planar, false),
            Some(PanelKind::BoxStats { .. })
        ));
        // ... and it must go through `classify_measure`, NOT `classify`, which would drop a
        // near-unique INTEGER column as ID-like. State-plane coords are integer feet, so on a
        // finely-spread dataset uniqueness_ratio clears 0.95 — the regression this route exists to
        // prevent. (Regression test for roborev 3536.)
        let mut planar_uniq = stat("Integer", 2000, Some(1.0));
        planar_uniq.q1 = Some(947_825.0);
        planar_uniq.q2_median = Some(982_500.0);
        planar_uniq.q3 = Some(1_017_175.0);
        assert!(
            classify(9, &planar_uniq).is_none(),
            "classify drops it as ID-like"
        );
        assert!(
            matches!(
                classify_with_semantics(9, &planar_uniq, &sem_planar, false),
                Some(PanelKind::BoxStats { .. })
            ),
            "ProjectedCoord must bypass the near-unique-integer ID guard"
        );
        // still suppressed when a map rendered, regardless of uniqueness
        assert!(classify_with_semantics(9, &planar_uniq, &sem_planar, true).is_none());
    }

    #[test]
    fn classify_measure_box_histogram_none() {
        // quartiles present -> box plot
        let mut s = stat("Float", 500, Some(0.9));
        s.q1 = Some(1.0);
        s.q2_median = Some(2.0);
        s.q3 = Some(3.0);
        assert!(matches!(
            classify_measure(1, &s),
            Some(PanelKind::BoxStats { .. })
        ));
        // flagged bimodal AND platykurtic (negative excess kurtosis) -> histogram
        s.bimodality_coefficient = Some(BIMODALITY_COEFFICIENT_THRESHOLD + 0.01);
        s.kurtosis = Some(-1.0);
        assert!(matches!(
            classify_measure(1, &s),
            Some(PanelKind::Histogram { .. })
        ));
        // high BC but LEPTOKURTIC (skewed unimodal, e.g. a long-tailed/outlier column) stays a box
        s.kurtosis = Some(5.0);
        assert!(matches!(
            classify_measure(1, &s),
            Some(PanelKind::BoxStats { .. })
        ));
        // no quartiles -> nothing to chart
        assert!(classify_measure(1, &stat("Integer", 500, Some(0.9))).is_none());
    }

    #[test]
    fn parse_dictionary_semantics_jsonschema_shape() {
        // a describegpt --format jsonschema doc: label rides as `title`, semantic tokens in
        // x-qsv, dataset grain at the top-level x-qsv.
        let schema = r#"{
          "$schema": "https://json-schema.org/draft/2020-12/schema",
          "type": "object",
          "properties": {
            "census_tract": { "type": ["integer","null"], "title": "Census Tract",
              "x-qsv": { "qsv_type": "Integer", "content_type": "category",
                         "role": "dimension", "concept": "geo.census_tract" } },
            "amount": { "type": "number", "title": "Amount",
              "x-qsv": { "qsv_type": "Float", "role": "measure", "concept": "measure.amount" } },
            "notes": { "type": "string", "x-qsv": { "qsv_type": "String" } }
          },
          "x-qsv": { "grain": "one row = one 311 service request",
                     "grain_unit": "311 service request" }
        }"#;
        let data = parse_dictionary_semantics(schema).expect("schema should parse");
        let tract = data.rows.get("census_tract").expect("census_tract row");
        assert_eq!(tract.role, "dimension");
        assert_eq!(tract.concept, "geo.census_tract");
        assert_eq!(tract.content_type, "category");
        assert_eq!(tract.label, "Census Tract");
        let amount = data.rows.get("amount").expect("amount row");
        assert_eq!(amount.concept, "measure.amount");
        // a property with a bare x-qsv (no role/concept) still parses with empty signals
        let notes = data.rows.get("notes").expect("notes row");
        assert!(notes.role.is_empty() && notes.concept.is_empty());
        assert_eq!(
            data.grain.as_deref(),
            Some("one row = one 311 service request")
        );
        assert_eq!(data.grain_unit.as_deref(), Some("311 service request"));

        // end-to-end: the parsed census_tract row routes to a Dimension bar.
        assert_eq!(
            derive_semantics(&stat("Integer", 33, Some(0.00003)), Some(tract)).route,
            Route::Dimension
        );
    }

    #[test]
    fn xq_pipelines_parses_both_encodings_and_skips_the_rest() {
        let schema = r#"{
          "properties": { "a": { "x-qsv": { "qsv_type": "Float" } } },
          "x-qsv": { "relationships": [
            {"kind": "pipeline", "members": ["planned", "committed", "spent"]},
            {"kind": "pipeline", "stage_column": "status",
             "stages": ["Impression", "Click"], "value_column": "revenue"},
            {"kind": "pipeline", "stage_column": "status", "stages": ["only_one"]},
            {"kind": "pipeline", "members": ["solo"]},
            {"kind": "ordered", "members": ["created", "closed"]},
            {"kind": "joint", "members": ["city", "state"]},
            "not even an object"
          ]}
        }"#;
        let data = parse_dictionary_semantics(schema).expect("schema should parse");
        assert_eq!(
            data.pipelines,
            vec![
                PipelineSpec::Columns {
                    members: vec![
                        "planned".to_string(),
                        "committed".to_string(),
                        "spent".to_string()
                    ],
                },
                PipelineSpec::Rows {
                    stage_column: "status".to_string(),
                    stages:       vec!["Impression".to_string(), "Click".to_string()],
                    value_column: Some("revenue".to_string()),
                },
            ],
            "only well-formed `pipeline` entries survive -- ordered/joint, too-short and \
             malformed ones are skipped rather than erroring"
        );
    }

    #[test]
    fn xq_pipelines_absent_or_legacy_yields_none() {
        let no_rels = r#"{"properties":{"a":{"x-qsv":{"qsv_type":"Float"}}}}"#;
        assert!(
            parse_dictionary_semantics(no_rels)
                .expect("schema should parse")
                .pipelines
                .is_empty()
        );
        // pipelines are JSONSchema-only: the legacy flat shape has nowhere to put a
        // row-encoded declaration, so it never yields one.
        let legacy = r#"{"fields":[
            {"name":"a","type":"Float"},{"name":"b","type":"Float"}
        ]}"#;
        assert!(
            parse_dictionary_semantics(legacy)
                .expect("legacy dict should parse")
                .pipelines
                .is_empty()
        );
    }

    #[test]
    fn render_dict_page_html_groups_stats_and_guards_sci_notation() {
        // Renders localized strings -- must share LOCALE_LOCK with the locale-mutating
        // tests, or a concurrent Spanish test flips the labels asserted below.
        let _locale = english_locale();
        // Range min/max grouping is role-gated: only genuine `measure` columns get thousands
        // separators on their range. Identifiers, geo codes (ZIP) and years are dimensions, so
        // their ranges render verbatim (a ZIP 15003 must never read "15,003", a year 2099 never
        // "2,099"). Cardinality/null_count are always counts and stay grouped regardless of role.
        // Extreme-magnitude float min/max serialize in scientific notation via serde_json
        // (`1e-7`, `1e+100`); grouping must leave those intact rather than mangle them into
        // `1,e-7` / `1e+,100`.
        let schema = r#"{
          "$schema": "https://json-schema.org/draft/2020-12/schema",
          "type": "object",
          "properties": {
            "account_id": { "type": "integer", "title": "Account ID",
              "minimum": 1000001, "maximum": 3000003,
              "x-qsv": { "role": "identifier", "cardinality": 1234567, "null_count": 4210 } },
            "owner_zip": { "type": "integer", "title": "Owner ZIP",
              "minimum": 15003, "maximum": 47909,
              "x-qsv": { "role": "dimension", "concept": "geo.zip_code" } },
            "exp_year": { "type": "integer", "title": "Expiration Year",
              "minimum": 2099, "maximum": 2099,
              "x-qsv": { "role": "dimension", "concept": "unknown" } },
            "revenue": { "type": "integer", "title": "Revenue",
              "minimum": 4000004, "maximum": 9000009,
              "x-qsv": { "role": "measure", "concept": "measure.value" } },
            "tiny": { "type": "number", "title": "Tiny",
              "minimum": 1e-7, "maximum": 1e100,
              "x-qsv": { "role": "measure", "concept": "measure.value", "cardinality": 3, "null_count": 0 } }
          },
          "x-qsv": { "grain": "one row = one account" }
        }"#;
        let dict_json: serde_json::Value = serde_json::from_str(schema).unwrap();
        let order = vec![
            "account_id".to_string(),
            "owner_zip".to_string(),
            "exp_year".to_string(),
            "revenue".to_string(),
            "tiny".to_string(),
        ];
        let html = render_dict_page_html(
            &dict_json,
            schema,
            &DictData::default(),
            "Accounts",
            &order,
            &std::collections::HashSet::new(),
            &[],
        );

        // a JSON Schema dictionary (top-level `properties`) offers a script-free "Export
        // JSONSchema" download: a plain <a download> carrying the verbatim schema as a base64
        // data: URI, with a title-derived filename.
        assert!(
            html.contains(r#"class="qsv-dict-export""#),
            "jsonschema dict offers an export link"
        );
        assert!(
            html.contains(r#"download="accounts.schema.json""#),
            "export filename derives from the dataset title"
        );
        assert!(
            html.contains("href=\"data:application/json;base64,"),
            "export rides as a base64 data: URI"
        );

        // measure range IS comma-grouped; cardinality and null_count are always grouped
        assert!(
            html.contains("4,000,004 \u{2013} 9,000,009"),
            "measure range grouped"
        );
        assert!(html.contains("<dd>1,234,567</dd>"), "cardinality grouped");
        assert!(html.contains("<dd>4,210</dd>"), "null_count grouped");

        // non-measure ranges render verbatim: identifier, ZIP code and year are never grouped
        assert!(
            html.contains("1000001 \u{2013} 3000003"),
            "identifier range not grouped"
        );
        assert!(!html.contains("1,000,001"), "identifier range not mangled");
        assert!(
            html.contains("15003 \u{2013} 47909"),
            "ZIP range not grouped"
        );
        assert!(!html.contains("15,003"), "ZIP not comma-mangled");
        assert!(
            html.contains("2099 \u{2013} 2099"),
            "year range not grouped"
        );
        assert!(!html.contains("2,099"), "year not comma-mangled");

        // scientific-notation floats are preserved verbatim (serde_json emits `1e+100`),
        // and never appear in the comma-mangled forms group_thousands would otherwise yield.
        assert!(
            html.contains("1e-7 \u{2013} 1e+100"),
            "exponent range preserved"
        );
        assert!(
            !html.contains("1,e-7"),
            "no comma-mangled negative exponent"
        );
        assert!(
            !html.contains("1e+,100"),
            "no comma-mangled positive exponent"
        );
    }

    #[test]
    fn render_dict_page_html_legacy_fields_dict_has_no_jsonschema_export() {
        // Renders localized strings -- must share LOCALE_LOCK with the locale-mutating
        // tests, or a concurrent Spanish test flips the labels asserted below.
        let _locale = english_locale();
        // A legacy `fields` dictionary is NOT JSON Schema, so the "Export JSONSchema" control
        // must not appear (the label would be a lie); only top-level `properties` dicts get it.
        let legacy = r#"{
          "fields": [
            { "name": "account_id", "role": "identifier", "content_type": "integer" },
            { "name": "revenue", "role": "measure", "content_type": "float" }
          ]
        }"#;
        let dict_json: serde_json::Value = serde_json::from_str(legacy).unwrap();
        let order = vec!["account_id".to_string(), "revenue".to_string()];
        let html = render_dict_page_html(
            &dict_json,
            legacy,
            &DictData::default(),
            "Accounts",
            &order,
            &std::collections::HashSet::new(),
            &[],
        );
        // the stylesheet always carries `.qsv-dict-export` rules; assert on the ANCHOR markup
        // (and the data: URI) so we're checking the control, not the CSS.
        assert!(
            !html.contains(r#"class="qsv-dict-export""#),
            "legacy fields dict renders no export anchor"
        );
        assert!(
            !html.contains("data:application/json;base64,"),
            "legacy fields dict embeds no schema download"
        );
        assert!(
            !html.contains(r#"class="qsv-dict-downloads""#),
            "with neither an export nor a sidecar, no download row is emitted at all"
        );
    }

    #[test]
    fn sidecar_tooltip_escapes_the_filename_exactly_once() {
        // Regression guard. The tooltip is built by interpolating the file name into a
        // TRANSLATED sentence and escaping the result. An earlier version fed in the
        // already-escaped `fname`, so the name was escaped twice and a file called `a&b.csv`
        // rendered a visible `a&amp;b` in the browser tooltip. Only the raw name may enter
        // `t!`; `fname` stays escaped for the `download=` attribute and the label span, which
        // are different contexts and must keep their own single escaping.
        let _locale = english_locale();

        let schema = r#"{ "properties": { "account_id": { "type": "integer" } } }"#;
        let dict_json: serde_json::Value = serde_json::from_str(schema).unwrap();
        let order = vec!["account_id".to_string()];
        // both hazards at once: `&` starts an entity, `"` would close the title attribute
        let sidecars = vec![SidecarDownload {
            fname: r#"a&b "q".stats.csv"#.to_string(),
            short: "stats",
            mime:  "text/csv",
            bytes: b"a,b\n1,2\n".to_vec(),
        }];
        let html = render_dict_page_html(
            &dict_json,
            schema,
            &DictData::default(),
            "Accounts",
            &order,
            &std::collections::HashSet::new(),
            &sidecars,
        );

        assert!(
            html.contains(r#"title="Download a&amp;b &quot;q&quot;.stats.csv, a sidecar this Data Schematic was built from""#),
            "the tooltip should escape the file name exactly once; all title= values were: {:?}",
            // report EVERY tooltip: the page also carries the JSONSchema export tooltip, and
            // naively taking the first match reports that one and misleads the reader.
            html.match_indices("title=\"")
                .map(|(i, m)| {
                    let rest = &html[i + m.len()..];
                    rest.split('"').next().unwrap_or("")
                })
                .collect::<Vec<_>>()
        );
        for double in ["&amp;amp;", "&amp;quot;"] {
            assert!(
                !html.contains(double),
                "found a double-escaped sequence ({double}) -- the file name is being escaped \
                 twice somewhere in the download row"
            );
        }
        // the OTHER contexts must still be escaped (once): `"` cannot be left raw in an
        // attribute, or it would terminate `download="..."` early.
        assert!(
            html.contains(r#"download="a&amp;b &quot;q&quot;.stats.csv""#),
            "the download attribute keeps its own single escaping"
        );
    }

    #[test]
    fn render_dict_page_html_renders_a_download_row_per_sidecar() {
        // Renders localized strings -- must share LOCALE_LOCK with the locale-mutating
        // tests, or a concurrent Spanish test flips the labels asserted below.
        let _locale = english_locale();
        // Every consumed sidecar joins the JSONSchema export inside ONE `.qsv-dict-downloads`
        // container — the container is what `qsvDictDrawer` relocates into the drawer's button
        // bar, so links outside it (or a per-anchor move) would be dropped there.
        let schema = r#"{ "properties": { "account_id": { "type": "integer" } } }"#;
        let dict_json: serde_json::Value = serde_json::from_str(schema).unwrap();
        let order = vec!["account_id".to_string()];
        let sidecars = vec![
            SidecarDownload {
                fname: "accounts.stats.csv.data.jsonl".to_string(),
                short: "stats",
                mime:  "application/x-ndjson",
                bytes: b"{\"field\":\"account_id\"}\n".to_vec(),
            },
            SidecarDownload {
                fname: "accounts.stats.bivariate.csv".to_string(),
                short: "bivariate",
                mime:  "text/csv",
                bytes: b"col1,col2,nmi\na,b,0.5\n".to_vec(),
            },
        ];
        let html = render_dict_page_html(
            &dict_json,
            schema,
            &DictData::default(),
            "Accounts",
            &order,
            &std::collections::HashSet::new(),
            &sidecars,
        );

        assert!(
            html.contains(r#"<div class="qsv-dict-downloads">"#),
            "the export + sidecar links share one container"
        );
        // one anchor for the schema export plus one per sidecar
        assert_eq!(
            html.matches(r#"class="qsv-dict-export""#).count(),
            3,
            "one anchor per download"
        );
        // `download` is the sidecar's OWN basename, never the title slug, so they can't collide
        assert!(
            html.contains(r#"download="accounts.stats.csv.data.jsonl""#)
                && html.contains(r#"download="accounts.stats.bivariate.csv""#),
            "each sidecar downloads under its own filename"
        );
        // each link also carries a one-word tag, shown INSTEAD of the filename in the drawer's
        // narrow strip — without it several downloads there would be identical ⤓ glyphs
        assert!(
            html.contains(r#"<span class="qsv-dict-export-short">schema</span>"#)
                && html.contains(r#"<span class="qsv-dict-export-short">stats</span>"#)
                && html.contains(r#"<span class="qsv-dict-export-short">bivariate</span>"#),
            "every download carries a distinguishing short tag"
        );
        // per-extension MIME on the data: URI
        assert!(
            html.contains("href=\"data:application/x-ndjson;base64,"),
            "jsonl sidecar rides as NDJSON"
        );
        assert!(
            html.contains("href=\"data:text/csv;base64,"),
            "csv sidecar rides as text/csv"
        );
        // base64 payloads are round-trippable and complete
        let b64 = base64_simd::STANDARD.encode_to_string(b"col1,col2,nmi\na,b,0.5\n");
        assert!(html.contains(&b64), "bivariate bytes embedded verbatim");
        // the page is embedded in a <script type="text/html"> template, so it must never carry a
        // literal closing script tag — base64 can't produce one, but assert the invariant.
        assert!(
            !html.contains("</script>"),
            "dictionary page stays script-tag-free"
        );
    }

    #[test]
    fn redact_sidecar_paths_strips_absolute_paths_from_metadata() {
        // A Data Schematic is made to be SHARED, so the metadata sidecars must not carry the
        // author's directory layout — but they must still say WHICH file they describe.
        let obj = br#"{"arg_input":"/Users/jane/clients/acme/data.csv",
            "canonical_input_path":"/Users/jane/clients/acme/data.csv",
            "canonical_stats_path":"/Users/jane/clients/acme/data.stats.csv.json",
            "record_count":10}"#;
        let out = redact_sidecar_paths(obj.to_vec(), SidecarRedaction::JsonObject);
        let v: serde_json::Value = serde_json::from_slice(&out).unwrap();
        assert_eq!(v["arg_input"], "data.csv");
        assert_eq!(v["canonical_input_path"], "data.csv");
        assert_eq!(v["canonical_stats_path"], "data.stats.csv.json");
        assert_eq!(v["record_count"], 10, "non-path fields survive untouched");
        assert!(
            !String::from_utf8_lossy(&out).contains("/Users/jane"),
            "no absolute path survives"
        );

        // JSONL: only the FIRST line is metadata; every data line passes through verbatim
        let jsonl = b"{\"canonical_input_path\":\"/Users/jane/data.csv\",\"column_count\":2}\n\
            {\"field\":\"a\",\"frequencies\":[]}\n{\"field\":\"b\",\"frequencies\":[]}\n";
        let out = redact_sidecar_paths(jsonl.to_vec(), SidecarRedaction::JsonlFirstLine);
        let text = String::from_utf8(out).unwrap();
        let mut lines = text.lines();
        let meta: serde_json::Value = serde_json::from_str(lines.next().unwrap()).unwrap();
        assert_eq!(meta["canonical_input_path"], "data.csv");
        assert_eq!(meta["column_count"], 2);
        assert_eq!(
            lines.collect::<Vec<_>>(),
            vec![
                "{\"field\":\"a\",\"frequencies\":[]}",
                "{\"field\":\"b\",\"frequencies\":[]}"
            ],
            "data lines are passed through byte-for-byte"
        );

        // pure-data sidecars and unparseable input are left exactly as they are — a download we
        // don't understand is better untouched than mangled
        let csv = b"col1,col2\na,b\n".to_vec();
        assert_eq!(
            redact_sidecar_paths(csv.clone(), SidecarRedaction::None),
            csv
        );
        let junk = b"not json at all".to_vec();
        assert_eq!(
            redact_sidecar_paths(junk.clone(), SidecarRedaction::JsonObject),
            junk
        );
    }

    #[test]
    fn charted_frequency_csv_captures_what_the_panels_show() {
        // The charted-frequency download is what the READER SEES: only charted columns, in header
        // order, bars in plotted order, aggregate buckets included and flagged.
        let stats = |name: &str| crate::cmd::stats::StatsData {
            field: name.to_string(),
            ..Default::default()
        };
        let all = [stats("agency"), stats("uncharted"), stats("status")];
        let bar = |label: &str, count: u64, kind: FreqBarKind| FreqBar {
            x_key: label.to_string(),
            label: label.to_string(),
            count,
            kind,
        };
        let mut freq = FreqMap::new();
        freq.insert(
            0,
            vec![
                bar("NYPD", 9, FreqBarKind::Category),
                // a comma in a label must be quoted, not split the row
                bar("HPD, Brooklyn", 4, FreqBarKind::Category),
                bar("Other (3)", 2, FreqBarKind::Aggregate),
            ],
        );
        freq.insert(2, vec![bar("Closed", 7, FreqBarKind::Category)]);

        let sc = charted_frequency_csv(std::path::Path::new("/tmp/nyc.csv"), &all, &freq).unwrap();
        assert_eq!(sc.fname, "nyc.viz-frequency.csv");
        assert_eq!(sc.mime, "text/csv");
        assert_eq!(
            String::from_utf8(sc.bytes).unwrap(),
            concat!(
                "column,value,count,bucket\n",
                "agency,NYPD,9,category\n",
                "agency,\"HPD, Brooklyn\",4,category\n",
                "agency,Other (3),2,aggregate\n",
                "status,Closed,7,category\n"
            ),
            "header order kept, uncharted column skipped, aggregates flagged, commas quoted"
        );

        // no frequency panel -> no file at all
        assert!(
            charted_frequency_csv(std::path::Path::new("x.csv"), &all, &FreqMap::new()).is_none()
        );
    }

    #[test]
    fn collect_sidecar_downloads_only_offers_what_was_used() {
        use std::io::Write;

        // stdin has no ON-DISK sidecars; with no frequency panel either, nothing is offered
        assert!(collect_sidecar_downloads(None, None, &[], &FreqMap::new(), true, true).is_empty());
        assert!(
            collect_sidecar_downloads(Some("-"), None, &[], &FreqMap::new(), true, true).is_empty()
        );

        let dir = std::env::temp_dir().join("qsv_viz_sidecar_dl_test");
        let _ = std::fs::remove_dir_all(&dir);
        std::fs::create_dir_all(&dir).unwrap();
        let input = dir.join("accounts.csv");
        std::fs::write(&input, "account_id,revenue\n1,2\n").unwrap();
        let input_str = input.to_string_lossy().into_owned();

        // no sidecars on disk yet -> nothing offered even with both gates open
        assert!(
            collect_sidecar_downloads(
                Some(&input_str),
                Some(&input),
                &[],
                &FreqMap::new(),
                true,
                true
            )
            .is_empty()
        );

        // the stats cache + its metadata are offered on existence (viz always reads them)
        std::fs::write(
            dir.join("accounts.stats.csv.data.jsonl"),
            "{\"field\":\"account_id\"}\n",
        )
        .unwrap();
        std::fs::write(
            dir.join("accounts.stats.csv.json"),
            "{\"qsv_version\":\"0\"}",
        )
        .unwrap();
        // ...but the frequency cache and bivariate CSV are gated on ACTUAL USE, not existence:
        // both exist here, yet both gates are closed.
        std::fs::write(dir.join("accounts.freq.csv.data.jsonl"), "{}\n").unwrap();
        std::fs::write(dir.join("accounts.stats.bivariate.csv"), "col1,col2\na,b\n").unwrap();

        let names =
            |v: Vec<SidecarDownload>| -> Vec<String> { v.into_iter().map(|s| s.fname).collect() };
        let got = names(collect_sidecar_downloads(
            Some(&input_str),
            Some(&input),
            &[],
            &FreqMap::new(),
            false,
            false,
        ));
        assert_eq!(
            got,
            vec![
                "accounts.stats.csv.data.jsonl".to_string(),
                "accounts.stats.csv.json".to_string()
            ],
            "an unused frequency cache / stale bivariate CSV is never offered"
        );

        // opening both gates picks them up
        let got = names(collect_sidecar_downloads(
            Some(&input_str),
            Some(&input),
            &[],
            &FreqMap::new(),
            true,
            true,
        ));
        assert!(
            got.contains(&"accounts.freq.csv.data.jsonl".to_string())
                && got.contains(&"accounts.stats.bivariate.csv".to_string()),
            "a cache that WAS read and a freshly-written bivariate CSV are offered: {got:?}"
        );

        // `.stats.csv` is NEVER offered, however fresh. viz doesn't read it, so it can't be shown
        // to be the stats behind this Data Schematic — an explicit `qsv stats --select … -o
        // <stem>.stats.csv` leaves a file newer than the source that describes a different
        // computation than the cache viz used, and no mtime test separates the two.
        let stats_csv = dir.join("accounts.stats.csv");
        std::fs::write(&stats_csv, "field,type\n").unwrap();
        assert!(
            !names(collect_sidecar_downloads(
                Some(&input_str),
                Some(&input),
                &[],
                &FreqMap::new(),
                false,
                false
            ))
            .contains(&"accounts.stats.csv".to_string()),
            "the unverifiable .stats.csv is never offered"
        );

        // over-cap sidecars are skipped rather than slurped into the page
        let mut big = std::fs::File::create(dir.join("accounts.stats.csv.data.jsonl")).unwrap();
        let chunk = vec![b'x'; 1024 * 1024];
        for _ in 0..=(MAX_SIDECAR_EMBED_BYTES / (1024 * 1024)) {
            big.write_all(&chunk).unwrap();
        }
        big.flush().unwrap();
        drop(big);
        assert!(
            !names(collect_sidecar_downloads(
                Some(&input_str),
                Some(&input),
                &[],
                &FreqMap::new(),
                false,
                false
            ))
            .contains(&"accounts.stats.csv.data.jsonl".to_string()),
            "a sidecar over the embed cap is skipped"
        );

        // The stats sidecars follow `stats_anchor` — the path the RUN already resolved — not the
        // raw input string. That separation is what lets a `dc:` handle use the CSV
        // `Config::new` materialized without this function re-resolving it (a re-resolve can
        // refresh a stale entry and silently swap in a different file mid-render).
        let other = dir.join("resolved.csv");
        std::fs::write(&other, "a,b\n1,2\n").unwrap();
        std::fs::write(
            dir.join("resolved.stats.csv.data.jsonl"),
            "{\"field\":\"a\"}\n",
        )
        .unwrap();
        // a real frequency panel, so the charted-frequency download is actually produced and its
        // filename can be checked
        let anchor_stats = [crate::cmd::stats::StatsData {
            field: "a".to_string(),
            ..Default::default()
        }];
        let mut anchor_freq = FreqMap::new();
        anchor_freq.insert(
            0,
            vec![FreqBar {
                x_key: "x".to_string(),
                label: "x".to_string(),
                count: 1,
                kind:  FreqBarKind::Category,
            }],
        );
        let got = names(collect_sidecar_downloads(
            Some("dc:accounts"),
            Some(&other),
            &anchor_stats,
            &anchor_freq,
            false,
            false,
        ));
        assert!(
            got.contains(&"resolved.stats.csv.data.jsonl".to_string()),
            "stats sidecars come from the anchor the run resolved: {got:?}"
        );
        assert!(
            !got.iter().any(|n| n.starts_with("accounts.stats")),
            "the raw input string never locates the stats sidecars: {got:?}"
        );
        assert!(
            got.contains(&"resolved.viz-frequency.csv".to_string()),
            "the charted-frequency download is named off the anchor, never the dc: string: {got:?}"
        );

        let _ = std::fs::remove_dir_all(&dir);
    }

    #[test]
    fn parse_dictionary_semantics_captures_descriptions() {
        // jsonschema shape: per-field `description`, dataset description at the top level
        // (describegpt --description folds it there under --format jsonschema), provenance
        // in x-qsv.generated_by.
        let schema = r#"{
          "$schema": "https://json-schema.org/draft/2020-12/schema",
          "type": "object",
          "description": "311 service requests for 2026.",
          "properties": {
            "census_tract": { "type": ["integer","null"], "title": "Census Tract",
              "description": "The 2020 census tract containing the incident.",
              "x-qsv": { "role": "dimension", "concept": "geo.census_tract" } },
            "amount": { "type": "number", "title": "Amount" }
          },
          "x-qsv": { "grain": "one row = one request",
            "generated_by": "Generated by qsv describegpt\nModel: test-model" }
        }"#;
        let data = parse_dictionary_semantics(schema).expect("schema should parse");
        assert_eq!(
            data.rows["census_tract"].description,
            "The 2020 census tract containing the incident."
        );
        assert_eq!(data.rows["amount"].description, "");
        assert_eq!(
            data.dataset_description.as_deref(),
            Some("311 service requests for 2026.")
        );
        assert!(
            data.generated_by
                .as_deref()
                .is_some_and(|g| g.contains("Model: test-model"))
        );

        // legacy shape: per-field `description` only; no dataset description / provenance.
        let legacy = r#"{"fields":[
            {"name":"a","content_type":"category","description":"Column A holds categories."}
        ]}"#;
        let data = parse_dictionary_semantics(legacy).expect("legacy should parse");
        assert_eq!(data.rows["a"].description, "Column A holds categories.");
        assert!(data.dataset_description.is_none());
        assert!(data.generated_by.is_none());
    }

    #[test]
    fn render_dict_markdown_basic_formatting() {
        // bold, italic, inline code
        assert_eq!(
            render_dict_markdown("A **bold** and *italic* and `code` word."),
            "<p>A <strong>bold</strong> and <em>italic</em> and <code>code</code> word.</p>\n"
        );
        // __bold__ / _italic_ variants
        assert_eq!(
            render_dict_markdown("__strong__ then _emph_"),
            "<p><strong>strong</strong> then <em>emph</em></p>\n"
        );
    }

    #[test]
    fn parse_dictionary_semantics_strips_attribution_from_dataset_description() {
        // describegpt appends its "Generated by … WARNING:" provenance to the dataset
        // description; it must be stripped (it's shown separately as the page footer).
        let schema = r#"{
          "type": "object",
          "description": "Municipal service requests.\n\n### Notable Characteristics\n\n* High volume.\n\nGenerated by qsv v21.1.0 describegpt\nModel: test-model\n\nWARNING: Description generated by an LLM and may contain inaccuracies. Verify before using!",
          "properties": { "a": { "type": "string", "title": "A" } },
          "x-qsv": { "generated_by": "Generated by qsv v21.1.0 describegpt\nModel: test-model" }
        }"#;
        let data = parse_dictionary_semantics(schema).expect("schema should parse");
        let desc = data
            .dataset_description
            .expect("dataset description present");
        assert!(desc.contains("Municipal service requests."));
        assert!(desc.contains("### Notable Characteristics"));
        assert!(desc.contains("High volume."));
        assert!(!desc.contains("Generated by"), "attribution not stripped");
        assert!(!desc.contains("WARNING"), "warning not stripped");
        // the footer still carries the provenance separately
        assert!(data.generated_by.is_some_and(|g| g.contains("test-model")));

        // a mid-sentence "Generated by" that is NOT a line-anchored describegpt footer must
        // be preserved (no truncation).
        let mid = r#"{
          "type": "object",
          "description": "This dataset was Generated by the city's 311 system in 2015.",
          "properties": { "a": { "type": "string" } }
        }"#;
        let data = parse_dictionary_semantics(mid).expect("schema should parse");
        assert_eq!(
            data.dataset_description.as_deref(),
            Some("This dataset was Generated by the city's 311 system in 2015.")
        );
    }

    #[test]
    fn strip_dataset_attribution_footer_forms() {
        // plain line-anchored footer (the jsonschema `description` form)
        assert_eq!(
            strip_dataset_attribution(
                "Some description.\n\nGenerated by qsv v20.1.0 describegpt\nWARNING: ..."
            ),
            "Some description."
        );
        // markdown footer: `*Attribution: Generated by … describegpt …*`
        assert_eq!(
            strip_dataset_attribution(
                "Some description.\n\n*Attribution: Generated by qsv v20.1.0 \
                 describegpt*\nWARNING: ...*"
            ),
            "Some description."
        );
        // TOON footer: `Attribution: Generated by …`
        assert_eq!(
            strip_dataset_attribution(
                "Some description.\n\nAttribution: Generated by qsv v20.1.0 describegpt"
            ),
            "Some description."
        );
        // dangling `---` rule and `**Attribution**` heading above the footer are trimmed
        assert_eq!(
            strip_dataset_attribution(
                "Some description.\n\n---\n\n**Attribution**\n\nGenerated by qsv v20.1.0 \
                 describegpt\nWARNING: ..."
            ),
            "Some description."
        );
        // a "Generated by" line WITHOUT the describegpt signature is preserved (not a footer)
        assert_eq!(
            strip_dataset_attribution("Generated by the city's 311 system.\n\nMore prose."),
            "Generated by the city's 311 system.\n\nMore prose."
        );
        // no footer at all: only trailing whitespace trimmed
        assert_eq!(strip_dataset_attribution("Just text.\n"), "Just text.");
    }

    #[test]
    fn render_dict_markdown_atx_headings() {
        // `Notable Characteristics` is localized at render time by `localize_dict_heading`, so this
        // assertion is locale-sensitive and must pin English against concurrently-running tests.
        let _locale = english_locale();
        // describegpt descriptions commonly use `### Notable Characteristics` sub-headings.
        let html = render_dict_markdown("Intro.\n\n### Notable Characteristics\n\n* one\n* two");
        assert_eq!(
            html,
            "<p>Intro.</p>\n<h4 class=\"qsv-dict-mdh\">Notable \
             Characteristics</h4>\n<ul><li>one</li><li>two</li></ul>\n"
        );
        // all levels normalize to <h4>; inline markup inside the heading still renders.
        assert!(
            render_dict_markdown("# Big **title**")
                .contains("<h4 class=\"qsv-dict-mdh\">Big <strong>title</strong></h4>")
        );
        // not a heading without the trailing space (e.g. a hashtag)
        assert_eq!(render_dict_markdown("#nothashtag"), "<p>#nothashtag</p>\n");
        // a trailing '#' that isn't a space-preceded ATX closing sequence is content
        assert_eq!(
            render_dict_markdown("### C#"),
            "<h4 class=\"qsv-dict-mdh\">C#</h4>\n"
        );
        // a proper closing sequence (preceded by a space) IS stripped
        assert_eq!(
            render_dict_markdown("### Heading ###"),
            "<h4 class=\"qsv-dict-mdh\">Heading</h4>\n"
        );
    }

    #[test]
    fn notable_characteristics_heading_is_localized_at_render_time() {
        let _guard = viz_i18n::lock_locale();
        viz_i18n::set_active(viz_i18n::parse_lang("es").unwrap());

        // The heading is LLM output carried verbatim in the schema's `description`, and it is
        // deliberately English there: describegpt's `okf_short_description` finds this exact
        // literal to cut the lead prose it needs for OKF's frontmatter. Localizing it at render
        // time is what lets the drawer read Spanish without breaking that parser.
        let html = render_dict_markdown("Intro.\n\n### Notable Characteristics\n\n* uno");
        assert!(
            html.contains("<h4 class=\"qsv-dict-mdh\">Características destacadas</h4>"),
            "the section heading should render in the active locale, got: {html}"
        );

        // Any OTHER heading is LLM prose already written in the target language, so it passes
        // through untouched -- only the one structural English literal is swapped.
        let other = render_dict_markdown("### Resumen general");
        assert!(
            other.contains("<h4 class=\"qsv-dict-mdh\">Resumen general</h4>"),
            "non-marker headings must not be rewritten, got: {other}"
        );
    }

    /// A realistic `x-qsv.generated_by`, shaped exactly as `describegpt --format jsonschema`
    /// bakes it into a dictionary. Built from the warning const so a reword can only be missed
    /// in one place, which `dict_provenance_warnings_match_describegpt` then catches.
    fn prov_fixture() -> String {
        [
            "Generated by qsv v21.1.0 describegpt",
            "Command line: target/debug/qsv describegpt --dictionary --description data.csv \
             --language es",
            "Prompt file: Default v8.0.0",
            "Model: google/gemma-4-26b-a4b",
            "LLM API URL: http://localhost:1234/v1",
            "Language: es",
            "Timestamp: 2026-07-31T03:09:38.316096+00:00",
            "",
            DG_WARNING_DICTIONARY_CONTENT_TYPE,
        ]
        .join("\n")
    }

    #[test]
    fn dict_provenance_labels_are_localized_at_render_time() {
        let _guard = viz_i18n::lock_locale();
        viz_i18n::set_active(viz_i18n::parse_lang("es").unwrap());

        let out = localize_dict_provenance(&prov_fixture());

        for expected in [
            "Generado por qsv v21.1.0 describegpt",
            "Línea de comandos: target/debug/qsv describegpt",
            "Archivo de prompt: Default v8.0.0",
            "Modelo: google/gemma-4-26b-a4b",
            "URL de la API del LLM: http://localhost:1234/v1",
            "Idioma: es",
            "Marca de tiempo: 2026-07-31T03:09:38.316096+00:00",
        ] {
            assert!(out.contains(expected), "missing {expected:?} in:\n{out}");
        }

        // The WARNING sentence must arrive IN FULL. Asserting only that the English one is gone
        // would also pass for an arm that returned an empty string.
        assert!(
            out.contains(
                "ADVERTENCIA: La etiqueta, la descripción y el tipo de contenido fueron generados \
                 por un LLM y pueden contener imprecisiones. ¡Verifíquelos antes de usarlos!"
            ),
            "warning sentence not translated in full:\n{out}"
        );

        // No English LABEL survives -- scoped to the labels on purpose. Under "labels only" a
        // VALUE may legitimately read English: describegpt can emit
        // `Language: es (dataset-detected, 94.2%)`, and that parenthetical rides through by
        // design, so a blanket English sweep over the whole block would be wrong.
        for english_label in [
            "Generated by ",
            "Command line:",
            "Prompt file:",
            "Model:",
            "LLM API URL:",
            "Language:",
            "Timestamp:",
            "WARNING:",
        ] {
            assert!(
                !out.contains(english_label),
                "English label {english_label:?} survived:\n{out}"
            );
        }

        // Every VALUE is byte-preserved -- the footer stays a faithful record of the run.
        for value in [
            "target/debug/qsv describegpt --dictionary --description data.csv --language es",
            "Default v8.0.0",
            "google/gemma-4-26b-a4b",
            "http://localhost:1234/v1",
            "2026-07-31T03:09:38.316096+00:00",
        ] {
            assert!(
                out.contains(value),
                "value {value:?} was not preserved:\n{out}"
            );
        }

        // ... and the blank separator line survives, so the block keeps its shape.
        assert_eq!(out.split('\n').count(), 9, "line count changed:\n{out}");

        viz_i18n::reset_active();
    }

    #[test]
    fn dict_provenance_is_byte_identical_in_english() {
        let _locale = english_locale();

        let blob = prov_fixture();
        assert_eq!(localize_dict_provenance(&blob), blob);

        // Degenerate shapes round-trip too: `split('\n')` was chosen over `str::lines` precisely
        // so a trailing newline is neither swallowed nor doubled.
        assert_eq!(localize_dict_provenance("Model: m\n"), "Model: m\n");
        assert_eq!(localize_dict_provenance("\n\n"), "\n\n");
        assert_eq!(localize_dict_provenance(""), "");
    }

    #[test]
    fn dict_provenance_passes_unrecognized_lines_through() {
        let _guard = viz_i18n::lock_locale();
        viz_i18n::set_active(viz_i18n::parse_lang("es").unwrap());

        // A hand-written dictionary carries no describegpt layout at all.
        let hand = "Compiled by hand from the 2019 codebook.\nContact: data@example.org";
        assert_eq!(localize_dict_provenance(hand), hand);

        // A reworded warning falls through whole rather than being half-translated ...
        let reworded = "WARNING: Everything here was written by a robot.";
        assert_eq!(localize_dict_provenance(reworded), reworded);

        // ... and so does a lead line whose tail is some other qsv command.
        let other_tool = "Generated by qsv v21.1.0 schema";
        assert_eq!(localize_dict_provenance(other_tool), other_tool);

        // A label-looking prefix that is not one of ours is left alone.
        let near_miss = "Models: three";
        assert_eq!(localize_dict_provenance(near_miss), near_miss);

        viz_i18n::reset_active();
    }

    /// The five warnings are matched by EXACT equality, so a reword in describegpt would silently
    /// drop this footer's last line back to English on every localized Data Schematic. Pin the
    /// consts to describegpt's own source, and to the English catalog values, so such a reword
    /// fails here instead of shipping.
    #[test]
    fn dict_provenance_warnings_match_describegpt() {
        let _locale = english_locale();

        let warnings = [
            (
                t!("viz.dict.prov_warning_dictionary_content_type"),
                DG_WARNING_DICTIONARY_CONTENT_TYPE,
            ),
            (
                t!("viz.dict.prov_warning_dictionary"),
                DG_WARNING_DICTIONARY,
            ),
            (
                t!("viz.dict.prov_warning_description"),
                DG_WARNING_DESCRIPTION,
            ),
            (t!("viz.dict.prov_warning_tags"), DG_WARNING_TAGS),
            (t!("viz.dict.prov_warning_generic"), DG_WARNING_GENERIC),
        ];

        // English *is* the catalog value, so a drifted en.yml would make the localizer replace a
        // matched sentence with a DIFFERENT one on the English page.
        for (catalog, konst) in &warnings {
            assert_eq!(
                catalog.as_ref(),
                *konst,
                "en.yml has drifted from the matched literal"
            );
        }

        // Undo rustfmt's string-literal line continuations (`\` + newline + indent) so each
        // literal reads as the single line describegpt actually emits.
        //
        // The CRLF normalization is load-bearing, not defensive: `.gitattributes` sets
        // `* text=auto` and does NOT exclude `.rs`, so a Windows checkout stores this file with
        // CRLF. The continuation marker is then `\` + CRLF, `split("\\\n")` matches nothing, no
        // literal is ever rejoined, and all five assertions below fail -- which is exactly how
        // this test first broke, on Windows CI only.
        fn unwrap_continuations(src: &str) -> String {
            let src = src.replace("\r\n", "\n");
            let mut joined = String::with_capacity(src.len());
            for (i, part) in src.split("\\\n").enumerate() {
                if i == 0 {
                    joined.push_str(part);
                } else {
                    joined.push_str(part.trim_start());
                }
            }
            joined
        }

        let src = std::fs::read_to_string(
            std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("src/cmd/describegpt.rs"),
        )
        .expect("src/cmd/describegpt.rs must be readable");
        let joined = unwrap_continuations(&src);

        // Pin the normalization itself, so a Unix-only run still catches its removal rather than
        // leaving Windows CI as the sole detector. Normalize to LF BEFORE re-expanding to CRLF --
        // this must build the same CRLF text whether the checkout was already CRLF or not.
        let as_crlf = src.replace("\r\n", "\n").replace('\n', "\r\n");
        assert_eq!(
            unwrap_continuations(&as_crlf),
            joined,
            "continuation unwrapping must be line-ending agnostic (Windows checks out CRLF)"
        );

        for (_, konst) in &warnings {
            assert!(
                joined.contains(*konst),
                "describegpt no longer emits this warning verbatim, so the drawer footer would \
                 silently fall back to English: {konst:?}"
            );
        }
    }

    #[test]
    fn kpi_labels_carry_the_rewrap_marker() {
        // KPI_REWRAP_SCRIPT finds its labels by this `name`, not by annotation index -- the two
        // emission sites produce different index layouts (shared `annotations` vec vs a dedicated
        // KPI-row layout), so an index-based hook would silently target the wrong annotation.
        let ann = kpi_label_annotation("Total de Conteo", 0.5, 0.24, 28, Font::new().size(13));
        let json = serde_json::to_value(&ann).expect("annotation should serialize");
        assert_eq!(
            json.get("name").and_then(serde_json::Value::as_str),
            Some(KPI_LABEL_MARK),
            "the marker must survive serialization into the page's layout"
        );
        assert!(
            KPI_REWRAP_SCRIPT.contains(KPI_LABEL_MARK),
            "the script and the annotation must agree on the marker string"
        );
    }

    #[test]
    fn render_dict_markdown_bullet_list() {
        let html = render_dict_markdown("Intro line.\n\n- first\n- second **bold**\n- third");
        assert_eq!(
            html,
            "<p>Intro line.</p>\n<ul><li>first</li><li>second \
             <strong>bold</strong></li><li>third</li></ul>\n"
        );
    }

    #[test]
    fn render_dict_markdown_paragraphs_and_softbreaks() {
        // blank line splits paragraphs; single newline collapses to a space.
        let html = render_dict_markdown("Line one\nstill one.\n\nSecond para.");
        assert_eq!(html, "<p>Line one still one.</p>\n<p>Second para.</p>\n");
    }

    #[test]
    fn render_dict_markdown_safe_links() {
        assert_eq!(
            render_dict_markdown("See [docs](https://example.com/a?b=1)."),
            "<p>See <a href=\"https://example.com/a?b=1\" target=\"_blank\" rel=\"noopener \
             noreferrer\">docs</a>.</p>\n"
        );
        // relative + anchor links are allowed
        assert!(render_dict_markdown("[x](#anchor)").contains("href=\"#anchor\""));
        assert!(render_dict_markdown("[x](/page)").contains("href=\"/page\""));
    }

    #[test]
    fn render_dict_markdown_underscores_are_not_intraword_emphasis() {
        // A snake_case identifier must NOT become emphasized.
        assert_eq!(
            render_dict_markdown("The first_name column."),
            "<p>The first_name column.</p>\n"
        );
    }

    #[test]
    fn render_dict_markdown_escapes_raw_html_no_xss() {
        // Raw HTML is escaped, never passed through — no live tag, no </script> breakout.
        let html = render_dict_markdown("<img src=x onerror=alert(1)> and </script> here");
        assert!(!html.contains("<img"));
        assert!(!html.contains("</script>"));
        assert!(html.contains("&lt;img"));
        assert!(html.contains("&lt;/script&gt;"));
    }

    #[test]
    fn render_dict_markdown_rejects_dangerous_url_schemes() {
        // javascript:/data:/vbscript: links drop the anchor and render as plain text.
        let js = render_dict_markdown("[click](javascript:alert(1))");
        assert!(!js.contains("href"));
        assert!(!js.contains("javascript:"));
        assert_eq!(js, "<p>click</p>\n");

        let data = render_dict_markdown("[x](data:text/html,<script>alert(1)</script>)");
        assert!(!data.contains("href"));
        assert!(!data.contains("<script>"));

        let vb = render_dict_markdown("[x](vbscript:msgbox(1))");
        assert!(!vb.contains("href"));
    }

    #[test]
    fn render_dict_markdown_escapes_special_chars_in_text() {
        assert_eq!(
            render_dict_markdown(r#"a & b < c > d "e""#),
            "<p>a &amp; b &lt; c &gt; d &quot;e&quot;</p>\n"
        );
    }

    #[test]
    fn parse_dictionary_semantics_legacy_and_garbage() {
        // legacy --format json dictionary (content_type only, no role/concept/grain)
        let json = r#"{
          "Dictionary": { "response": { "fields": [
            { "name": "status", "type": "String", "content_type": "category", "label": "Status" },
            { "name": "id", "type": "String" }
          ] } }
        }"#;
        let data = parse_dictionary_semantics(json).expect("legacy json should parse");
        let status = data.rows.get("status").expect("status row");
        assert_eq!(status.content_type, "category");
        assert_eq!(status.label, "Status");
        assert!(status.role.is_empty() && status.concept.is_empty());
        // a field with no content_type is still present (carries empty signals)
        assert!(data.rows.contains_key("id"));
        assert!(data.grain.is_none());

        // a bare {"fields":[...]} envelope also works
        let bare = r#"{"fields":[{"name":"x","content_type":"category"}]}"#;
        assert!(parse_dictionary_semantics(bare).is_some());

        // garbage / wrong-shaped JSON yields None (caller degrades gracefully)
        assert!(parse_dictionary_semantics("not json").is_none());
        assert!(parse_dictionary_semantics(r#"{"foo":1}"#).is_none());
    }

    #[test]
    fn timestamp_rank_prefers_event_and_created() {
        // event/created lead; closed/updated/due secondary; bare date or no concept last
        assert!(timestamp_rank("time.event_timestamp") < timestamp_rank("time.created_at"));
        assert!(timestamp_rank("time.created_at") < timestamp_rank("time.closed_at"));
        assert!(timestamp_rank("time.closed_at") < timestamp_rank("time.date"));
        assert_eq!(
            timestamp_rank("time.updated_at"),
            timestamp_rank("time.due_at")
        );
        assert_eq!(timestamp_rank(""), timestamp_rank("time.date")); // no concept == bare date
    }

    #[test]
    fn ts_bucket_span_thresholds_and_keys() {
        use chrono::NaiveDate;
        assert_eq!(ts_bucket_for_span(0), TsBucket::Day);
        assert_eq!(ts_bucket_for_span(370), TsBucket::Day);
        assert_eq!(ts_bucket_for_span(371), TsBucket::Week);
        assert_eq!(ts_bucket_for_span(1825), TsBucket::Week);
        assert_eq!(ts_bucket_for_span(1826), TsBucket::Month);
        assert_eq!(ts_bucket_for_span(7300), TsBucket::Month);
        assert_eq!(ts_bucket_for_span(7301), TsBucket::Quarter);
        assert_eq!(ts_bucket_for_span(18250), TsBucket::Quarter);
        assert_eq!(ts_bucket_for_span(18251), TsBucket::Year);

        // variant order IS fineness order: bucket selection coarsens via `Ord::max`
        // (span ceiling vs cadence floor), so this ordering is load-bearing.
        assert!(TsBucket::Day < TsBucket::Week);
        assert!(TsBucket::Week < TsBucket::Month);
        assert!(TsBucket::Month < TsBucket::Quarter);
        assert!(TsBucket::Quarter < TsBucket::Year);

        // a Wednesday truncates to its ISO-week Monday, and to the 1st of its month
        let wed = NaiveDate::from_ymd_opt(2024, 5, 1).unwrap(); // 2024-05-01 is a Wednesday
        assert_eq!(ts_bucket_key(wed, TsBucket::Day), wed);
        assert_eq!(
            ts_bucket_key(wed, TsBucket::Week),
            NaiveDate::from_ymd_opt(2024, 4, 29).unwrap() // Monday
        );
        assert_eq!(
            ts_bucket_key(wed, TsBucket::Month),
            NaiveDate::from_ymd_opt(2024, 5, 1).unwrap()
        );
        // quarters snap to their quarter-start month (1/4/7/10); years to Jan 1
        assert_eq!(
            ts_bucket_key(wed, TsBucket::Quarter),
            NaiveDate::from_ymd_opt(2024, 4, 1).unwrap()
        );
        let dec = NaiveDate::from_ymd_opt(2024, 12, 31).unwrap();
        assert_eq!(
            ts_bucket_key(dec, TsBucket::Quarter),
            NaiveDate::from_ymd_opt(2024, 10, 1).unwrap()
        );
        assert_eq!(
            ts_bucket_key(dec, TsBucket::Year),
            NaiveDate::from_ymd_opt(2024, 1, 1).unwrap()
        );
        // labels: ISO date for day/week, YYYY-MM for month, YYYY-Qn for quarter, YYYY for year
        assert_eq!(ts_bucket_label(wed, TsBucket::Day), "2024-05-01");
        assert_eq!(ts_bucket_label(wed, TsBucket::Month), "2024-05");
        assert_eq!(ts_bucket_label(wed, TsBucket::Quarter), "2024-Q2");
        assert_eq!(ts_bucket_label(dec, TsBucket::Quarter), "2024-Q4");
        assert_eq!(ts_bucket_label(dec, TsBucket::Year), "2024");
    }

    #[test]
    fn cadence_floor_detects_native_spacing() {
        use chrono::NaiveDate;
        let d = |y: i32, m: u32, dd: u32| NaiveDate::from_ymd_opt(y, m, dd).unwrap();

        // 3 years of quarter-start dates: Week occupancy ~12/157 fails, Quarter (12/12) wins.
        let quarterly: Vec<_> = (0..12i32)
            .map(|i| d(2021 + i / 4, (i as u32 % 4) * 3 + 1, 1))
            .collect();
        assert_eq!(cadence_floor(&quarterly), TsBucket::Quarter);

        // daily data (with weekend gaps) keeps Day: 5 of every 7 days ≈ 71% ≥ 45%.
        let daily: Vec<_> = (0..60)
            .map(|i| d(2024, 1, 1) + chrono::Duration::days(i))
            .filter(|dt| {
                use chrono::Datelike;
                dt.weekday().num_days_from_monday() < 5
            })
            .collect();
        assert_eq!(cadence_floor(&daily), TsBucket::Day);

        // monthly data with a couple of missing months still reads Monthly.
        let monthly: Vec<_> = (0..24i32)
            .filter(|i| *i != 5 && *i != 13)
            .map(|i| d(2023 + i / 12, i as u32 % 12 + 1, 1))
            .collect();
        assert_eq!(cadence_floor(&monthly), TsBucket::Month);

        // annual data: everything finer than Year is mostly empty.
        let annual: Vec<_> = (0..6).map(|i| d(2019 + i, 6, 30)).collect();
        assert_eq!(cadence_floor(&annual), TsBucket::Year);

        // degenerate inputs: empty and single-date default to Day.
        assert_eq!(cadence_floor(&[]), TsBucket::Day);
        assert_eq!(cadence_floor(&[d(2024, 5, 1)]), TsBucket::Day);
    }

    #[test]
    fn bucket_floor_from_cadence_tokens() {
        assert_eq!(bucket_floor_from_cadence("daily"), Some(TsBucket::Day));
        assert_eq!(bucket_floor_from_cadence("weekly"), Some(TsBucket::Week));
        assert_eq!(bucket_floor_from_cadence("monthly"), Some(TsBucket::Month));
        assert_eq!(
            bucket_floor_from_cadence("quarterly"),
            Some(TsBucket::Quarter)
        );
        assert_eq!(bucket_floor_from_cadence("annual"), Some(TsBucket::Year));
        // out-of-vocab tokens (LLM prose, typos, future tokens) are ignored, never an error
        assert_eq!(bucket_floor_from_cadence("Quarterly"), None);
        assert_eq!(bucket_floor_from_cadence("every 3 months"), None);
        assert_eq!(bucket_floor_from_cadence(""), None);
    }

    #[test]
    fn trend_bucket_floor_token_can_only_coarsen_the_scan() {
        let d = |y, m, dd| chrono::NaiveDate::from_ymd_opt(y, m, dd).unwrap();

        // 3 years of quarterly observations: the scan reads Quarter.
        let quarterly: Vec<_> = (0..12i32)
            .map(|i| d(2021 + i / 4, (i as u32 % 4) * 3 + 1, 1))
            .collect();
        assert_eq!(cadence_floor(&quarterly), TsBucket::Quarter);

        // a wrong-because-too-fine token (aggregate stats cannot prove regular spacing) must NOT
        // drag the floor below the scan -- that is the comb of spikes issue #4216 removed.
        assert_eq!(
            trend_bucket_floor(Some("daily"), &quarterly),
            TsBucket::Quarter
        );
        assert_eq!(
            trend_bucket_floor(Some("weekly"), &quarterly),
            TsBucket::Quarter
        );

        // coarsening is still the token's call: the publisher's declared reporting period
        // outranks occupancy in that direction.
        assert_eq!(
            trend_bucket_floor(Some("annual"), &quarterly),
            TsBucket::Year
        );

        // absent and out-of-vocab tokens fall through to the scan
        assert_eq!(trend_bucket_floor(None, &quarterly), TsBucket::Quarter);
        assert_eq!(
            trend_bucket_floor(Some("every 3 months"), &quarterly),
            TsBucket::Quarter
        );
    }

    #[test]
    fn choose_anim_bucket_widens_to_quarter_and_year() {
        use chrono::NaiveDate;
        // 40 years of one date per month = 480 monthly buckets: Month (480) and Quarter (160)
        // both exceed the 30-frame cap, Year (40) still does — the ladder now tops out at Year
        // instead of overflowing at Month (issue #4216).
        let dates: Vec<NaiveDate> = (0..480)
            .map(|i| NaiveDate::from_ymd_opt(1980 + i / 12, i as u32 % 12 + 1, 15).unwrap())
            .collect();
        let (bucket, distinct) = choose_anim_bucket(&dates);
        assert_eq!(bucket, TsBucket::Year);
        assert_eq!(distinct.len(), 40);

        // 6 years of monthly dates (72 > 30 frames) coarsens to Quarter (24 ≤ 30).
        let dates: Vec<NaiveDate> = (0..72)
            .map(|i| NaiveDate::from_ymd_opt(2018 + i / 12, i as u32 % 12 + 1, 15).unwrap())
            .collect();
        let (bucket, distinct) = choose_anim_bucket(&dates);
        assert_eq!(bucket, TsBucket::Quarter);
        assert_eq!(distinct.len(), 24);
    }

    #[test]
    fn choose_smart_anim_bucket_declines_past_the_frame_cap() {
        use chrono::NaiveDate;
        // 40 yearly buckets is still above SMART_ANIM_MAX_FRAMES even at the coarsest rung, so
        // the smart wrapper declines rather than emit a slider that ignores its own cap. The
        // raw ladder still returns Year for the explicit --slider path, which enforces the far
        // larger SLIDER_MAX_FRAMES instead.
        let dates: Vec<NaiveDate> = (0..480)
            .map(|i| NaiveDate::from_ymd_opt(1980 + i / 12, i as u32 % 12 + 1, 15).unwrap())
            .collect();
        assert!(choose_anim_bucket(&dates).1.len() > SMART_ANIM_MAX_FRAMES);
        assert!(choose_smart_anim_bucket(&dates).is_none());

        // within the cap, the wrapper is transparent
        let dates: Vec<NaiveDate> = (0..72)
            .map(|i| NaiveDate::from_ymd_opt(2018 + i / 12, i as u32 % 12 + 1, 15).unwrap())
            .collect();
        let (bucket, distinct) = choose_smart_anim_bucket(&dates).unwrap();
        assert_eq!(bucket, TsBucket::Quarter);
        assert_eq!(distinct.len(), 24);
    }

    #[test]
    fn count_unit_from_grain_extracts_or_falls_back() {
        // the "records" fallback is localized -- pin English, or this races the
        // Spanish-setting tests on the process-global locale.
        let _locale = english_locale();

        // --- structured `grain_unit` wins (issue #4321) ---------------------------------
        // It is preferred even when a parseable grain sentence disagrees, because it is the
        // field describegpt writes in the target language.
        assert_eq!(
            count_unit_from_grain(
                Some("registro de coleta"),
                Some("one row = one product collection record per location")
            ),
            "registro de coleta"
        );
        // ...and it is used verbatim, with surrounding whitespace trimmed.
        assert_eq!(
            count_unit_from_grain(Some("  permit application  "), None),
            "permit application"
        );
        // A CJK unit must survive: it blows a 40-BYTE budget at ~13 chars, so the cap has to
        // count characters. This is the case the legacy byte cap would have silently eaten.
        let cjk = "ステーションごとの日次観測記録";
        assert!(cjk.len() > 40, "fixture must exceed the legacy byte cap");
        assert!(
            cjk.chars().count() <= COUNT_UNIT_MAX_CHARS,
            "fixture must still be within the char cap"
        );
        assert_eq!(count_unit_from_grain(Some(cjk), None), cjk);
        // Blank/whitespace-only grain_unit falls THROUGH to the grain parse, not to `records`.
        assert_eq!(
            count_unit_from_grain(Some("   "), Some("one row = one order")),
            "order"
        );
        // A runaway grain_unit is rejected on chars, then falls through to the grain parse.
        let runaway = "x".repeat(COUNT_UNIT_MAX_CHARS + 1);
        assert_eq!(
            count_unit_from_grain(Some(&runaway), Some("one row = one order")),
            "order"
        );

        // --- legacy grain parsing, unchanged --------------------------------------------
        assert_eq!(
            count_unit_from_grain(None, Some("one row = one 311 service request")),
            "311 service request"
        );
        assert_eq!(
            count_unit_from_grain(None, Some("each row is one order.")),
            "order"
        );
        // no " one " pattern -> fallback
        assert_eq!(
            count_unit_from_grain(None, Some("each row represents a person")),
            "records"
        );
        // absent grain -> fallback
        assert_eq!(count_unit_from_grain(None, None), "records");
        // implausibly long tail -> fallback (guards against a runaway grain sentence)
        let long = format!("one row = one {}", "x".repeat(60));
        assert_eq!(count_unit_from_grain(None, Some(&long)), "records");
    }

    #[test]
    fn code_twin_base_strips_key_suffixes() {
        assert_eq!(code_twin_base("subject_code"), Some("subject"));
        assert_eq!(code_twin_base("street_id"), Some("street"));
        assert_eq!(code_twin_base("region_key"), Some("region"));
        // no key suffix, or suffix-only -> None
        assert_eq!(code_twin_base("subject"), None);
        assert_eq!(code_twin_base("_code"), None);
        // bare "id"/"code" (no underscore) is NOT a twin suffix
        assert_eq!(code_twin_base("zipcode"), None);
    }

    #[test]
    fn dimension_code_twins_suppresses_only_paired_keys() {
        // build a 4-column frame: subject (dim) + subject_code (dim) + street (dim) + lonely_id
        // (dim)
        let stats = [
            {
                let mut s = stat("String", 20, Some(0.01));
                s.field = "subject".to_string();
                s
            },
            {
                let mut s = stat("Integer", 20, Some(0.01));
                s.field = "subject_code".to_string();
                s
            },
            {
                let mut s = stat("String", 20, Some(0.01));
                s.field = "street".to_string();
                s
            },
            {
                let mut s = stat("Integer", 20, Some(0.01));
                s.field = "lonely_id".to_string();
                s
            },
        ];
        let dim = ColSemantics {
            route: Route::Dimension,
            ..Default::default()
        };
        let sems = [dim.clone(), dim.clone(), dim.clone(), dim.clone()];
        let suppress = dimension_code_twins(&stats, &sems);
        // subject_code is suppressed (its label sibling `subject` is charted) ...
        assert!(suppress.contains(&1));
        // ... but subject, street, and lonely_id (no `lonely` sibling) are kept
        assert!(!suppress.contains(&0));
        assert!(!suppress.contains(&2));
        assert!(!suppress.contains(&3));

        // a *_code with no charted label sibling is NOT suppressed
        let orphan_stats = [{
            let mut s = stat("Integer", 20, Some(0.01));
            s.field = "ward_code".to_string();
            s
        }];
        let orphan_sems = [dim];
        assert!(dimension_code_twins(&orphan_stats, &orphan_sems).is_empty());
    }

    #[test]
    fn log_scale_parse() {
        assert!(matches!(parse_log_scale("auto"), Ok(LogScale::Auto)));
        assert!(matches!(parse_log_scale("AUTO"), Ok(LogScale::Auto)));
        assert!(matches!(parse_log_scale("on"), Ok(LogScale::On)));
        assert!(matches!(parse_log_scale("true"), Ok(LogScale::On)));
        assert!(matches!(parse_log_scale("off"), Ok(LogScale::Off)));
        assert!(matches!(parse_log_scale("false"), Ok(LogScale::Off)));
        assert!(parse_log_scale("bogus").is_err());
    }

    #[test]
    fn freq_panel_logs_auto_triggers_on_high_dynamic_range() {
        // a dominating bucket (10000) dwarfs the small categories (>= 50x) -> log
        let dominated = [10_000, 120, 90, 30];
        assert!(freq_panel_logs(LogScale::Auto, &dominated));
        // a balanced distribution (max/min < 50x) stays linear under auto
        let balanced = [100, 90, 80, 70];
        assert!(!freq_panel_logs(LogScale::Auto, &balanced));
        // auto needs at least 3 positive bars (a lone tall bar isn't "dominating" anything)
        let two_bars = [10_000, 1];
        assert!(!freq_panel_logs(LogScale::Auto, &two_bars));
    }

    #[test]
    fn freq_panel_logs_on_and_off_modes() {
        let balanced = [100, 90, 80];
        // `on` forces log even when the range is low (needs 2+ positive bars)
        assert!(freq_panel_logs(LogScale::On, &balanced));
        assert!(!freq_panel_logs(LogScale::On, &[5]));
        // `off` never logs, even on a wildly skewed distribution
        assert!(!freq_panel_logs(LogScale::Off, &[10_000, 100, 10, 1]));
    }

    #[test]
    fn measure_by_dim_logs_needs_all_positive_and_dynamic_range() {
        // a dominating positive group (550) over much smaller positive groups (>= 50x) -> log
        let dominated = [550.0, 8.0, 3.0, 2.0];
        assert!(measure_by_dim_logs(LogScale::Auto, &dominated));
        // a balanced spread (max/min < 50x) stays linear under auto
        let balanced = [100.0, 80.0, 60.0];
        assert!(!measure_by_dim_logs(LogScale::Auto, &balanced));
        // auto needs at least 3 positive bars
        assert!(!measure_by_dim_logs(LogScale::Auto, &[10_000.0, 1.0]));
        // ANY non-positive bar rules out a log value axis (a log axis can't place 0 or negatives)
        // -- this is the common "one tall bar over a pile of zeros" degenerate case, which
        // stays LINEAR
        assert!(!measure_by_dim_logs(
            LogScale::Auto,
            &[550.0, 0.0, 0.0, 0.0]
        ));
        assert!(!measure_by_dim_logs(LogScale::On, &[550.0, 0.0]));
        assert!(!measure_by_dim_logs(LogScale::Auto, &[100.0, -5.0, 3.0]));
        // `on` forces log for 2+ all-positive bars regardless of range; `off` never logs
        assert!(measure_by_dim_logs(LogScale::On, &[100.0, 90.0]));
        assert!(!measure_by_dim_logs(LogScale::Off, &[10_000.0, 100.0, 1.0]));
    }

    #[test]
    fn classify_low_card_string_is_bar() {
        assert!(matches!(
            classify(0, &stat("String", 3, Some(0.1))),
            Some(PanelKind::FreqBar { idx: 0 })
        ));
    }

    #[test]
    fn classify_high_card_string_skipped() {
        assert!(classify(0, &stat("String", 9999, Some(0.99))).is_none());
    }

    #[test]
    fn box_whiskers_use_observed_min_max_not_fences() {
        // Whiskers must be the observed min/max, never the Tukey fences - even when a fence
        // falls INSIDE the data range (a fence value need not exist in the dataset).
        let mut s = stat("Float", 100, Some(0.8));
        s.q1 = Some(10.0);
        s.q2_median = Some(15.0);
        s.q3 = Some(20.0);
        s.lower_inner_fence = Some(12.0); // inside [8, 25] - must be ignored
        s.upper_inner_fence = Some(22.0); // inside [8, 25] - must be ignored
        s.min = Some("8.0".to_string());
        s.max = Some("25.0".to_string());
        match classify(0, &s) {
            Some(PanelKind::BoxStats { lower, upper, .. }) => {
                assert_eq!(lower, Some(8.0)); // observed minimum
                assert_eq!(upper, Some(25.0)); // observed maximum
            },
            _ => panic!("expected BoxStats"),
        }
    }

    #[test]
    fn classify_bimodal_continuous_is_histogram_not_box() {
        // a continuous column flagged bimodal (BC >= 0.555) AND platykurtic (excess kurtosis < 0)
        // should become a histogram (which shows the two peaks), not a box plot (which hides them).
        let mut s = stat("Float", 500, Some(0.9));
        s.q1 = Some(1.0);
        s.q2_median = Some(2.0);
        s.q3 = Some(3.0);
        s.min = Some("0.0".to_string());
        s.max = Some("4.0".to_string());
        s.bimodality_coefficient = Some(0.7); // >= 0.555 threshold
        s.kurtosis = Some(-1.5); // platykurtic (two-peaked / flat-topped)
        assert!(matches!(
            classify(4, &s),
            Some(PanelKind::Histogram { idx: 4 })
        ));
    }

    #[test]
    fn classify_skewed_high_bc_leptokurtic_stays_box() {
        // a heavily-skewed UNIMODAL column (long tail / outliers) can have a high BC purely from
        // skewness, but it's leptokurtic — the platykurtic guard keeps it a box (with outlier
        // points) rather than a misleading one-tall-bar histogram.
        let mut s = stat("Float", 500, Some(0.9));
        s.q1 = Some(1.0);
        s.q2_median = Some(2.0);
        s.q3 = Some(3.0);
        s.min = Some("0.0".to_string());
        s.max = Some("9999.0".to_string());
        s.bimodality_coefficient = Some(0.98); // high, but driven by skew
        s.kurtosis = Some(10.0); // leptokurtic (heavy tail)
        assert!(matches!(classify(0, &s), Some(PanelKind::BoxStats { .. })));
    }

    #[test]
    fn classify_unimodal_continuous_stays_box() {
        // bimodality below the threshold (or absent) keeps the cache-only box plot.
        let mut s = stat("Float", 500, Some(0.9));
        s.q1 = Some(1.0);
        s.q2_median = Some(2.0);
        s.q3 = Some(3.0);
        s.min = Some("0.0".to_string());
        s.max = Some("4.0".to_string());
        s.bimodality_coefficient = Some(0.40); // unimodal
        assert!(matches!(classify(0, &s), Some(PanelKind::BoxStats { .. })));
    }

    #[test]
    fn parse_violin_mode_accepts_auto_on_off() {
        assert!(matches!(parse_violin_mode("auto"), Ok(ViolinMode::Auto)));
        assert!(matches!(parse_violin_mode("AUTO"), Ok(ViolinMode::Auto)));
        assert!(matches!(parse_violin_mode("on"), Ok(ViolinMode::On)));
        assert!(matches!(parse_violin_mode("true"), Ok(ViolinMode::On)));
        assert!(matches!(parse_violin_mode("off"), Ok(ViolinMode::Off)));
        assert!(matches!(parse_violin_mode("false"), Ok(ViolinMode::Off)));
        assert!(parse_violin_mode("bogus").is_err());
    }

    #[test]
    fn wants_violin_auto_defaults_to_violin_with_guards() {
        // a violin is the DEFAULT representation for a continuous column: any shape, no
        // bimodality/kurtosis signal required (given enough distinct values AND observations)
        let s = stat("Float", 100, Some(0.9));
        assert!(wants_violin(ViolinMode::Auto, &s, false, 1_000));

        // too few distinct values for a meaningful KDE (a lumpy comb) -> box
        let mut few = stat("Float", 100, Some(0.9));
        few.cardinality = VIOLIN_MIN_CARDINALITY - 1;
        assert!(!wants_violin(ViolinMode::Auto, &few, false, 1_000));

        // too few observations for an honest KDE (the curve would be bandwidth artifact) -> box,
        // even when the distinct-value count clears its guard
        assert!(!wants_violin(
            ViolinMode::Auto,
            &s,
            false,
            VIOLIN_MIN_POINTS - 1
        ));

        // a would-log panel keeps its box under auto (the KDE is estimated in linear
        // space, so a log axis distorts the silhouette); `on` is the user's explicit
        // choice and still wins
        assert!(!wants_violin(ViolinMode::Auto, &s, true, 1_000));
        assert!(wants_violin(ViolinMode::On, &s, true, 1_000));
    }

    #[test]
    fn wants_violin_on_off_override_guards() {
        // `on`/`off` ignore the guards entirely
        let bland = stat("Float", 5, Some(0.9)); // fails the cardinality AND min-points guards
        assert!(wants_violin(ViolinMode::On, &bland, false, 5));
        assert!(!wants_violin(
            ViolinMode::Off,
            &stat("Float", 100, Some(0.9)),
            false,
            1_000
        ));
    }

    #[test]
    fn panel_interest_violin_scores_like_raw_box() {
        // violins are the default representation, not a special signal — they must rank
        // exactly like the equivalent raw box so --max-charts overflow is shape-neutral
        let mut s = stat("Float", 100, Some(0.9));
        s.cv = Some(85.0);
        let violin = PanelKind::Violin {
            idx:           0,
            points:        BoxPoints::Outliers,
            sample_stride: 1,
        };
        let boxraw = PanelKind::BoxRaw {
            idx:    0,
            points: BoxPoints::Outliers,
        };
        let (vi, bi) = (panel_interest(&s, &violin), panel_interest(&s, &boxraw));
        assert!((vi - bi).abs() < f64::EPSILON, "violin {vi} != boxraw {bi}");
    }

    #[test]
    fn classify_high_card_concentrated_string_is_bar() {
        // high-cardinality text is normally skipped, but a low normalized_entropy (concentrated
        // distribution, a few dominant categories) makes a top-N bar worthwhile.
        let mut s = stat("String", 9999, Some(0.6));
        s.normalized_entropy = Some(0.4); // concentrated, below the 0.95 noise cutoff
        assert!(matches!(
            classify(7, &s),
            Some(PanelKind::FreqBar { idx: 7 })
        ));
    }

    #[test]
    fn classify_high_card_uniform_string_still_skipped() {
        // high cardinality AND near-uniform entropy => genuinely noise, still skipped.
        let mut s = stat("String", 9999, Some(0.6));
        s.normalized_entropy = Some(0.99); // near-uniform
        assert!(classify(0, &s).is_none());
    }

    #[test]
    fn finalize_freq_bars_appends_null_and_other() {
        // asserts English output from a `t!`-backed helper -- pin the locale (see `english_locale`)
        let _locale = english_locale();
        // 12 distinct categories (a..l), descending counts, plus 7 empty cells. With top_n=10,
        // the top 10 are kept and the remaining 2 distinct roll up into "Other (2)"; the null
        // bucket becomes "(NULL)". Both are appended after the real categories, tagged Aggregate.
        let counts: Vec<(String, u64)> = (0..12)
            .map(|i| (((b'a' + i) as char).to_string(), (100 - i as u64)))
            .collect();
        let out = finalize_freq_bars(counts, 7, 10, false, false);
        assert_eq!(out.len(), 12); // 10 real + (NULL) + Other
        assert_eq!(out[9].label, "j"); // last real category kept
        assert_eq!(out[9].kind, FreqBarKind::Category);

        assert_eq!(out[10].label, "(NULL)");
        assert_eq!(out[10].count, 7);
        assert_eq!(out[10].kind, FreqBarKind::Aggregate);

        // the two dropped categories ("k"=90, "l"=89) aggregate to 179, labeled by distinct count
        assert_eq!(out[11].label, "Other (2)");
        assert_eq!(out[11].count, 179);
        assert_eq!(out[11].kind, FreqBarKind::Aggregate);

        // aggregate bars get a sentinel-suffixed x_key so they can't collide with a real
        // category sharing their display label; real categories key on their value
        assert_eq!(out[9].x_key, "j");
        assert_eq!(out[10].x_key, format!("(NULL){AGG_KEY_SENTINEL}"));
        assert_ne!(out[10].x_key, out[10].label);
    }

    #[test]
    fn freq_bar_pattern_shapes_hatches_aggregates_only_on_log() {
        // 10 real categories + (NULL) + Other, the last two tagged Aggregate.
        let counts: Vec<(String, u64)> = (0..12)
            .map(|i| (((b'a' + i) as char).to_string(), (100 - i as u64)))
            .collect();
        let bars = finalize_freq_bars(counts, 7, 10, false, false);

        // linear panel -> no hatching at all
        assert!(freq_bar_pattern_shapes(&bars, false).is_none());

        // log panel -> diagonal hatch on aggregates, none on real categories
        let shapes = freq_bar_pattern_shapes(&bars, true).unwrap();
        assert_eq!(shapes.len(), bars.len());
        // PatternShape has no PartialEq; compare via serde (empty "" = no pattern, "/" = hatch)
        for (bar, shape) in bars.iter().zip(shapes.iter()) {
            let want = match bar.kind {
                FreqBarKind::Aggregate => "/",
                FreqBarKind::Category => "",
            };
            assert_eq!(
                serde_json::to_value(shape).unwrap(),
                serde_json::json!(want)
            );
        }
    }

    #[test]
    fn log_distribution_body_cue_only_on_log() {
        // linear box/violin panels keep their palette color -> no cue
        assert!(log_distribution_body_cue(false).is_none());

        // a log panel gets the faint grey wash + muted grey outline (plotly box/violin can't be
        // hatched or dashed, so this fill+outline pair is the log cue)
        let (fill, outline) = log_distribution_body_cue(true).expect("log panel should get a cue");
        assert_eq!(fill, LOG_DISTRIB_FILL);
        assert_eq!(outline, MUTED_COLOR);
    }

    #[test]
    fn finalize_freq_bars_respects_opt_out_flags() {
        let counts: Vec<(String, u64)> = (0..12)
            .map(|i| (((b'a' + i) as char).to_string(), (100 - i as u64)))
            .collect();
        // suppress both buckets -> just the top 10 real categories remain
        let out = finalize_freq_bars(counts, 7, 10, true, true);
        assert_eq!(out.len(), 10);
        assert!(out.iter().all(|b| b.kind == FreqBarKind::Category));
    }

    #[test]
    fn finalize_freq_bars_names_lone_leftover_category() {
        // asserts English output from a `t!`-backed helper -- pin the locale (see `english_locale`)
        let _locale = english_locale();
        // 11 distinct categories with top_n=10: the single leftover ("k") is charted under its
        // own name as a real category, NOT as an opaque "Other (1)" aggregate.
        let counts: Vec<(String, u64)> = (0..11)
            .map(|i| (((b'a' + i) as char).to_string(), (100 - i as u64)))
            .collect();
        let out = finalize_freq_bars(counts.clone(), 0, 10, false, false);
        assert_eq!(out.len(), 11);
        assert!(out.iter().all(|b| b.kind == FreqBarKind::Category));
        assert_eq!(out[10].label, "k");
        assert_eq!(out[10].x_key, "k"); // real key, no aggregate sentinel
        assert_eq!(out[10].count, 90);
        assert!(!out.iter().any(|b| b.label.starts_with("Other")));

        // --no-other still means "no bar beyond top_n": the leftover is dropped, not promoted
        let out = finalize_freq_bars(counts.clone(), 0, 10, false, true);
        assert_eq!(out.len(), 10);
        assert!(!out.iter().any(|b| b.label == "k"));

        // two leftovers still roll up into the "Other (2)" aggregate
        let mut counts12 = counts;
        counts12.push(("l".to_string(), 89));
        let out = finalize_freq_bars(counts12, 0, 10, false, false);
        assert_eq!(out.len(), 11);
        assert_eq!(out[10].label, "Other (2)");
        assert_eq!(out[10].kind, FreqBarKind::Aggregate);
        assert_eq!(out[10].count, 179);
    }

    #[test]
    fn finalize_freq_bars_omits_empty_buckets() {
        // no nulls and fewer distinct than top_n -> no aggregate bars even with flags off
        let counts = vec![("a".to_string(), 5_u64), ("b".to_string(), 3)];
        let out = finalize_freq_bars(counts, 0, 10, false, false);
        let labels: Vec<(String, u64)> = out.iter().map(|b| (b.label.clone(), b.count)).collect();
        assert_eq!(labels, vec![("a".to_string(), 5), ("b".to_string(), 3)]);
        assert!(out.iter().all(|b| b.kind == FreqBarKind::Category));
    }

    #[test]
    fn finalize_freq_bars_aggregate_key_never_collides() {
        // asserts English output from a `t!`-backed helper -- pin the locale (see `english_locale`)
        let _locale = english_locale();
        // pathological data: a real category whose value already equals the sentinel-suffixed
        // form of the NULL bucket's key. The aggregate key must extend its sentinel until it is
        // distinct, so the synthetic (NULL) bar can never collapse onto that real category.
        let collider = format!("(NULL){AGG_KEY_SENTINEL}");
        let counts = vec![("apple".to_string(), 10_u64), (collider.clone(), 4)];
        let out = finalize_freq_bars(counts, 3, 10, false, false);

        // real category keeps its value as the key; the NULL aggregate is forced to a longer key
        let real = out.iter().find(|b| b.label == collider).unwrap();
        assert_eq!(real.x_key, collider);
        assert_eq!(real.kind, FreqBarKind::Category);

        let null_bar = out.iter().find(|b| b.label == "(NULL)").unwrap();
        assert_eq!(null_bar.kind, FreqBarKind::Aggregate);
        assert_ne!(null_bar.x_key, collider);
        // all x_keys across the panel are unique
        let keys: std::collections::HashSet<&String> = out.iter().map(|b| &b.x_key).collect();
        assert_eq!(keys.len(), out.len());
    }

    #[test]
    fn box_shape_hint_reports_skew_and_outliers() {
        // asserts English output from a `t!`-backed helper -- pin the locale (see `english_locale`)
        let _locale = english_locale();
        let mut s = stat("Float", 100, Some(0.8));
        s.pearson_skewness = Some(1.2); // right skew
        s.outliers_percentage = Some(4.2);
        assert_eq!(
            box_shape_hint(&s).as_deref(),
            Some("(right-skewed, 4.2% outliers)")
        );

        // symmetric, no notable outliers => no hint
        let mut s2 = stat("Float", 100, Some(0.8));
        s2.pearson_skewness = Some(0.1);
        s2.outliers_percentage = Some(0.2);
        assert_eq!(box_shape_hint(&s2), None);

        // no moarstats stats at all => no hint
        assert_eq!(box_shape_hint(&stat("Float", 100, Some(0.8))), None);
    }

    #[test]
    fn box_shape_hint_derives_skew_from_base_stats() {
        // asserts English output from a `t!`-backed helper -- pin the locale (see `english_locale`)
        let _locale = english_locale();
        // no moarstats pearson_skewness: derived from the base cache as 3*(mean-median)/stddev
        let mut s = stat("Float", 100, Some(0.8));
        s.mean = Some(100.0);
        s.q2_median = Some(50.0);
        s.stddev = Some(100.0); // -> +1.5
        assert_eq!(box_shape_hint(&s).as_deref(), Some("(right-skewed)"));

        s.mean = Some(50.0);
        s.q2_median = Some(100.0); // -> -1.5
        assert_eq!(box_shape_hint(&s).as_deref(), Some("(left-skewed)"));

        // an explicit moarstats value wins over the derived one
        s.pearson_skewness = Some(2.0);
        assert_eq!(box_shape_hint(&s).as_deref(), Some("(right-skewed)"));

        // zero spread -> no derived skew, no hint
        let mut flat = stat("Float", 1, Some(0.01));
        flat.mean = Some(5.0);
        flat.q2_median = Some(5.0);
        flat.stddev = Some(0.0);
        assert_eq!(box_shape_hint(&flat), None);
    }

    #[test]
    fn box_shape_hint_puts_data_quality_first_and_caps_parts() {
        // asserts English output from a `t!`-backed helper -- pin the locale (see `english_locale`)
        let _locale = english_locale();
        // nulls + zeros outrank (and, at the 2-part cap, crowd out) the shape parts
        let mut s = stat("Integer", 500, Some(0.5));
        s.sparsity = Some(0.40);
        s.n_negative = Some(0);
        s.n_zero = Some(50);
        s.n_positive = Some(50); // zero share 50%
        s.pearson_skewness = Some(1.2);
        s.outliers_percentage = Some(4.2);
        assert_eq!(box_shape_hint(&s).as_deref(), Some("(40% null, 50% zeros)"));

        // below the thresholds neither fires and the shape parts show through
        s.sparsity = Some(0.05);
        s.n_zero = Some(10);
        s.n_positive = Some(90);
        assert_eq!(
            box_shape_hint(&s).as_deref(),
            Some("(right-skewed, 4.2% outliers)")
        );
    }

    #[test]
    fn box_shape_hint_reports_moarstats_extras() {
        // asserts English output from a `t!`-backed helper -- pin the locale (see `english_locale`)
        let _locale = english_locale();
        // outliers inflating the mean
        let mut s = stat("Float", 100, Some(0.8));
        s.outlier_impact_ratio = Some(0.07);
        assert_eq!(
            box_shape_hint(&s).as_deref(),
            Some("(mean +7% from outliers)")
        );

        // concentration via Gini
        let mut g = stat("Float", 100, Some(0.8));
        g.gini_coefficient = Some(0.87);
        assert_eq!(box_shape_hint(&g).as_deref(), Some("(Gini 0.87)"));

        // heavy-tails fallback fires only when the outlier scan is absent...
        let mut h = stat("Float", 100, Some(0.8));
        h.mad_stddev_ratio = Some(0.3);
        assert_eq!(box_shape_hint(&h).as_deref(), Some("(heavy tails)"));
        // ...and stays silent when outliers_percentage is present (even a negligible one)
        h.outliers_percentage = Some(0.2);
        assert_eq!(box_shape_hint(&h), None);
    }

    #[test]
    fn dominant_category_hint_fires_only_on_real_dominance() {
        // `dominant_category_hint` is localized -- pin English, or this races the
        // Spanish-setting tests on the process-global locale.
        let _locale = english_locale();
        let bar = |label: &str, count: u64, kind: FreqBarKind| FreqBar {
            x_key: label.to_string(),
            label: label.to_string(),
            count,
            kind,
        };
        // one real category holds 95% of the 100 rows
        let dominated = vec![
            bar("Yes", 95, FreqBarKind::Category),
            bar("No", 5, FreqBarKind::Category),
        ];
        assert_eq!(
            dominant_category_hint(&dominated, || 100).as_deref(),
            Some("(dominated by Yes, 95%)")
        );
        // balanced -> no hint, and the drawn-share pre-check means the (potentially costly)
        // row count is never even fetched
        let balanced = vec![
            bar("Yes", 60, FreqBarKind::Category),
            bar("No", 40, FreqBarKind::Category),
        ];
        assert_eq!(
            dominant_category_hint(&balanced, || unreachable!("pre-check must skip row count")),
            None
        );
        // the denominator is the TRUE row count, so bars filtered by --no-nulls/--no-other
        // can't inflate the share: 85 of 100 rows (15 nulls suppressed from the bars) is NOT
        // dominant, even though the surviving bar is 100% of what's drawn
        let only_bar = vec![bar("active", 85, FreqBarKind::Category)];
        assert_eq!(dominant_category_hint(&only_bar, || 100), None);
        // a dominant AGGREGATE bucket (NULL/Other) is not a dominant category: with >90% blank
        // rows the tallest REAL category (4 of 100) stays far below the threshold, even though
        // the stats cache's mode for such a column is the empty bucket
        let null_heavy = vec![
            bar("(NULL)", 96, FreqBarKind::Aggregate),
            bar("A", 3, FreqBarKind::Category),
            bar("B", 1, FreqBarKind::Category),
        ];
        assert_eq!(
            dominant_category_hint(&null_heavy, || unreachable!(
                "pre-check must skip row count"
            )),
            None
        );
        // no real category bars, or no bars at all -> no hint
        let only_aggregates = vec![bar("(NULL)", 100, FreqBarKind::Aggregate)];
        assert_eq!(dominant_category_hint(&only_aggregates, || 100), None);
        assert_eq!(dominant_category_hint(&[], || 100), None);
    }

    #[test]
    fn box_panel_logs_requires_positive_values_and_range() {
        // no skew signals in a default stats cache => only the strictly-positive fast path applies
        let s = crate::cmd::stats::StatsData::default();
        // Auto: fast path needs min > 0 and max/min >= LOG_SCALE_MIN_RATIO
        assert!(box_panel_logs(LogScale::Auto, &s, Some(1.0), Some(100.0)));
        assert!(!box_panel_logs(LogScale::Auto, &s, Some(1.0), Some(10.0)));
        // min <= 0 with no skew signals => the fallback declines => linear
        assert!(!box_panel_logs(LogScale::Auto, &s, Some(0.0), Some(1000.0)));
        assert!(!box_panel_logs(
            LogScale::Auto,
            &s,
            Some(-5.0),
            Some(1000.0)
        ));
        assert!(!box_panel_logs(LogScale::Auto, &s, None, Some(1000.0)));
        // On: forces log on a strictly-positive column, but a zero-min column with no skew
        // signals still can't render on log
        assert!(box_panel_logs(LogScale::On, &s, Some(2.0), Some(3.0)));
        assert!(!box_panel_logs(LogScale::On, &s, Some(0.0), Some(3.0)));
        // Off: always linear
        assert!(!box_panel_logs(LogScale::Off, &s, Some(1.0), Some(1e9)));
    }

    #[test]
    fn box_log_skew_fallback_rescues_skewed_positive_columns() {
        // mirrors CMS medical_payment: 7 zeros in ~1.24M values, q1 well above 0, max ~76M, strong
        // right skew — a linear box/violin needles, so the fallback takes a log axis (issue #4219).
        let skewed = crate::cmd::stats::StatsData {
            n_negative: Some(0),
            n_zero: Some(7),
            n_positive: Some(1_235_750),
            q1: Some(10_000.0),
            max: Some("76000000".to_string()),
            pearson_skewness: Some(3.0),
            ..Default::default()
        };
        // min == 0 (fast path declines), but negligible zeros + wide right-skewed range => log
        assert!(box_panel_logs(
            LogScale::Auto,
            &skewed,
            Some(0.0),
            Some(76_000_000.0)
        ));

        // negative control 1 — zero-inflated (CPDB-like): >5% non-positive keeps it LINEAR
        let zero_inflated = crate::cmd::stats::StatsData {
            n_zero: Some(600_000),
            n_positive: Some(635_757),
            ..skewed.clone()
        };
        assert!(!box_panel_logs(
            LogScale::Auto,
            &zero_inflated,
            Some(0.0),
            Some(76_000_000.0)
        ));

        // negative control 2 — weak skew stays linear
        let mild = crate::cmd::stats::StatsData {
            pearson_skewness: Some(0.2),
            ..skewed.clone()
        };
        assert!(!box_panel_logs(
            LogScale::Auto,
            &mild,
            Some(0.0),
            Some(76_000_000.0)
        ));

        // negative control 3 — box body not positive (q1 == 0) can't render on log
        let q1_zero = crate::cmd::stats::StatsData {
            q1: Some(0.0),
            ..skewed.clone()
        };
        assert!(!box_panel_logs(
            LogScale::Auto,
            &q1_zero,
            Some(0.0),
            Some(76_000_000.0)
        ));

        // negative control 4 — narrow positive range: Auto declines (max/q1 = 38 < 50), On forces
        let narrow = crate::cmd::stats::StatsData {
            q1: Some(2_000_000.0),
            ..skewed.clone()
        };
        assert!(!box_panel_logs(
            LogScale::Auto,
            &narrow,
            Some(0.0),
            Some(76_000_000.0)
        ));
        assert!(box_panel_logs(
            LogScale::On,
            &narrow,
            Some(0.0),
            Some(76_000_000.0)
        ));
    }

    #[test]
    fn is_intensive_measure_matches_tokens_not_substrings() {
        // the -ration word family all CONTAIN "ratio"; a substring test silently downgraded these
        // additive amounts from Sum to Mean, and also dropped them from the inequality overview.
        // ("duration" belongs to this family too, but it is now intensive on its own merits as a
        // time SPAN — see the span cases below — so it can no longer serve as a control here.)
        for additive in [
            "generation",
            "operation",
            "operations",
            "registration",
            "registrations",
            "remuneration",
            "corporations",
            "migration",
            "iteration",
            // "rate" is now an intensive token; "rating" is a different word and must survive
            // token-whole matching
            "rating",
            "ratings",
        ] {
            assert!(
                !is_intensive_measure(additive, additive),
                "{additive} is additive, must not be treated as intensive"
            );
        }
        // genuinely intensive columns still match, including plural/derived forms
        for intensive in [
            "ratio",
            "ratios",
            "debt_to_equity_ratio",
            "rate",
            "rates",
            "failure_rate",
            "interestRate",
            "percent",
            "percentage",
            "pct_complete",
            "avg_price",
            "median_income",
            "score",
            "scores",
            "review_score",
            "index",
            "indices",
            "density",
            "densities",
            "elevation",
            "temperature",
        ] {
            assert!(
                is_intensive_measure(intensive, intensive),
                "{intensive} should be intensive"
            );
        }
        // durations/ages are spans, not amounts: summing them is meaningless
        for span in [
            "age",
            "account_age_days",
            "Account Age (Days)",
            "tenure",
            "duration",
            "call_duration",
            "elapsed",
            "latency_ms",
            "runtime",
        ] {
            assert!(
                is_intensive_measure(span, span),
                "{span} is a span/state, must not be summed"
            );
        }
        // ...but "age" is a substring of these ADDITIVE words, so token matching must protect them
        for additive in [
            "agency",
            "Agency Name",
            "packages",
            "storage",
            "usage",
            "message_count",
            "coverage",
            "mileage",
            "garage",
        ] {
            assert!(
                !is_intensive_measure(additive, additive),
                "{additive} merely contains 'age' and must stay additive"
            );
        }
        // phrases/symbols stay substring-matched (they can't survive tokenization)
        assert!(is_intensive_measure(
            "income per capita",
            "income_per_capita"
        ));
        assert!(is_intensive_measure("temp \u{b0}C", "temp_c"));
        // the count guard still wins over an embedded intensive token
        assert!(!is_intensive_measure(
            "review score count",
            "review_score_count"
        ));
    }

    #[test]
    fn route_from_concept_treats_duration_as_a_measure() {
        // a duration is a span, not a point in time: routing it Temporal dropped it from the
        // Data Schematic entirely (no distribution panel, and canonical_date_col won't take it
        // either)
        assert_eq!(
            route_from_concept("time.duration"),
            Some((Route::Measure, Some(Agg::Mean)))
        );
        // the rest of the `time` namespace is unchanged
        assert_eq!(
            route_from_concept("time.timestamp"),
            Some((Route::Temporal, None))
        );
        assert_eq!(route_from_concept("time"), Some((Route::Temporal, None)));
    }

    #[test]
    fn mercator_lat_center_is_the_visual_midpoint() {
        // round-trip
        for lat in [-80.0, -45.0, 0.0, 12.5, 45.0, 80.0] {
            assert!(
                (inverse_mercator_y(mercator_y(lat)) - lat).abs() < 1e-9,
                "{lat}"
            );
        }
        // at the equator Mercator is locally symmetric, so it agrees with the arithmetic mean
        assert!((mercator_lat_center(-10.0, 10.0) - 0.0).abs() < 1e-9);
        // poleward it does NOT: the visual centre sits north of the arithmetic mean, which is
        // exactly why a Mercator-fitted zoom left the top of an arctic extent off-screen
        let (lo, hi) = (66.0, 84.5);
        let arithmetic = (lo + hi) / 2.0;
        let visual = mercator_lat_center(lo, hi);
        assert!(
            visual > arithmetic + 0.5,
            "visual={visual} arithmetic={arithmetic}"
        );
        // and it is genuinely equidistant in Mercator space
        let (a, b, c) = (mercator_y(lo), mercator_y(visual), mercator_y(hi));
        assert!(((a - b) - (b - c)).abs() < 1e-9);
    }

    #[test]
    fn ring_signed_area_includes_the_closing_edge() {
        let ring = |pts: &[[f64; 2]]| serde_json::json!(pts);
        // a 10x10 square offset from the origin, written UNCLOSED: dropping the closing term
        // makes the shoelace sum wildly wrong (it used to compute 950 instead of 100)
        let unclosed = ring(&[
            [100.0, 100.0],
            [110.0, 100.0],
            [110.0, 110.0],
            [100.0, 110.0],
        ]);
        assert!((ring_signed_area(&unclosed).abs() - 100.0).abs() < 1e-9);
        // the already-closed spelling must give the same answer
        let closed = ring(&[
            [100.0, 100.0],
            [110.0, 100.0],
            [110.0, 110.0],
            [100.0, 110.0],
            [100.0, 100.0],
        ]);
        assert!((ring_signed_area(&closed).abs() - ring_signed_area(&unclosed).abs()).abs() < 1e-9);
        // and an unclosed exterior must still out-rank a genuine hole, which is what
        // repair_polygon_rings orders on
        let hole = ring(&[
            [102.0, 102.0],
            [105.0, 102.0],
            [105.0, 105.0],
            [102.0, 105.0],
        ]);
        assert!(ring_signed_area(&unclosed).abs() > ring_signed_area(&hole).abs());
        // degenerate rings stay zero
        assert!((ring_signed_area(&ring(&[[0.0, 0.0], [1.0, 1.0]])).abs() - 0.0).abs() < 1e-12);
    }

    #[test]
    fn kpi_number_format_keeps_small_magnitudes_visible() {
        // a ratio like 0.0004 rendered as a flat "0.00" next to a gauge needle at 4% of scale
        assert_eq!(kpi_number_format(0.0004), ".3~g");
        assert_eq!(kpi_number_format(-0.0004), ".3~g");
        // the existing bands are untouched
        assert_eq!(kpi_number_format(0.25), ",.2f");
        assert_eq!(kpi_number_format(42.0), ",.0f");
        assert_eq!(kpi_number_format(1_250_000.0), ".3~s");
        // exact zero is a whole number, not a small magnitude
        assert_eq!(kpi_number_format(0.0), ",.0f");
    }

    #[test]
    fn match_region_code_falls_back_to_case_insensitive() {
        let ids: std::collections::HashSet<&str> = ["CA", "NY", "06075"].into_iter().collect();
        let widths = vec![5_usize];
        let lower: std::collections::HashMap<String, String> = ids
            .iter()
            .map(|i| ((*i).to_ascii_lowercase(), (*i).to_string()))
            .collect();
        let m = |raw: &str| match_region_code(raw, &ids, &widths, &lower);
        // exact still wins
        assert_eq!(m("CA").as_deref(), Some("CA"));
        // differently-cased alphabetic codes now match, canonicalized to the FEATURE's spelling
        assert_eq!(m("ca").as_deref(), Some("CA"));
        assert_eq!(m("Ca").as_deref(), Some("CA"));
        // numeric zero-padding is unaffected and still preferred
        assert_eq!(m("6075").as_deref(), Some("06075"));
        // a genuine non-member still misses
        assert!(m("ZZ").is_none());
    }

    #[test]
    fn md_inline_caps_recursion_depth() {
        // untrusted LLM dictionary text: deeply nested emphasis used to recurse without bound and
        // could exhaust the stack. Past the cap the remainder is emitted as escaped literal text.
        let deep = "*".repeat(4000) + "x" + &"*".repeat(4000);
        let out = md_inline(&deep);
        assert!(!out.is_empty());
        // no raw markup escapes, whatever the nesting did
        assert!(!out.contains("<script"));
        // a pathological run of unmatched markers terminates and stays literal
        let unmatched = "*".repeat(5000);
        assert!(!md_inline(&unmatched).is_empty());
        // ordinary emphasis still renders
        assert!(md_inline("plain *em* text").contains("<em>"));
        assert!(md_inline("plain **strong** text").contains("<strong>"));
    }

    #[test]
    fn match_region_code_drops_ambiguous_case_folds() {
        // two feature ids differing only by case make the folded key ambiguous: resolving it from
        // HashSet iteration order would aggregate the same CSV value under a different region
        // between runs, so the fuzzy fallback is forfeited instead (exact matching still works).
        let ids: std::collections::HashSet<&str> = ["CA", "ca", "NY"].into_iter().collect();
        let widths: Vec<usize> = Vec::new();
        let mut lower: std::collections::HashMap<String, String> = std::collections::HashMap::new();
        let mut ambiguous: std::collections::HashSet<String> = std::collections::HashSet::new();
        for id in ["CA", "ca", "NY"] {
            let f = id.to_ascii_lowercase();
            match lower.get(&f) {
                Some(e) if e != id => {
                    ambiguous.insert(f);
                },
                Some(_) => {},
                None => {
                    lower.insert(f, id.to_string());
                },
            }
        }
        for k in ambiguous {
            lower.remove(&k);
        }
        // exact matches are unaffected by the ambiguity
        assert_eq!(
            match_region_code("CA", &ids, &widths, &lower).as_deref(),
            Some("CA")
        );
        assert_eq!(
            match_region_code("ca", &ids, &widths, &lower).as_deref(),
            Some("ca")
        );
        // but a case that matches NEITHER exactly has no deterministic answer, so none is given
        assert!(match_region_code("Ca", &ids, &widths, &lower).is_none());
        // an unambiguous id still folds
        assert_eq!(
            match_region_code("ny", &ids, &widths, &lower).as_deref(),
            Some("NY")
        );
    }

    #[test]
    fn cumulative_embedded_points_budget_converges_on_sparse_buckets() {
        // `step_by` keeps the first element of EVERY bucket, so with many near-empty buckets a
        // single stride pass cannot reach the cap however large the stride is. This models that
        // shape: 400 buckets of 1 point each embeds 400 + ~80k, far above a small cap.
        let sparse: Vec<Vec<f64>> = (0..400).map(|_| vec![0.0; 1]).collect();
        let embedded = cumulative_embedded_points(&sparse);
        assert!(
            embedded > 400,
            "cumulative repetition dominates: {embedded}"
        );
        // striding cannot shrink single-point buckets at all, which is why the production loop
        // stops on the all-buckets-<=1 condition rather than spinning — and why, past that point,
        // it declines the animation instead of emitting a panel that ignores its own cap.
        let strided: Vec<Vec<f64>> = sparse
            .iter()
            .map(|b| b.iter().copied().step_by(1000).collect())
            .collect();
        assert_eq!(
            cumulative_embedded_points(&strided),
            embedded,
            "single-point buckets are irreducible by striding"
        );
        // enough single-point buckets exceed even the default budget on repetition alone
        let many: Vec<Vec<f64>> = (0..700).map(|_| vec![0.0; 1]).collect();
        assert!(cumulative_embedded_points(&many) > 150_000);
    }

    #[test]
    fn pearson_matrix_matches_pairwise_pearson() {
        // the precomputed/parallel form must agree with the per-pair reference exactly, including
        // its NaN semantics for degenerate columns
        let cols = vec![
            vec![1.0, 2.0, 3.0, 4.0, 5.0],
            vec![2.0, 4.1, 5.9, 8.2, 9.8], // near-linear
            vec![5.0, 3.0, 4.0, 1.0, 2.0], // negatively related
            vec![7.0, 7.0, 7.0, 7.0, 7.0], // constant -> undefined
        ];
        let m = pearson_matrix(&cols);
        for i in 0..cols.len() {
            for j in 0..cols.len() {
                let want = pearson(&cols[i], &cols[j]);
                let got = m[i][j];
                assert_eq!(
                    want.is_nan(),
                    got.is_nan(),
                    "NaN disagreement at ({i},{j}): want {want} got {got}"
                );
                if !want.is_nan() {
                    assert!(
                        (want - got).abs() < 1e-12,
                        "({i},{j}): want {want} got {got}"
                    );
                }
            }
        }
        // symmetry and the self-correlation diagonal
        assert!((m[0][0] - 1.0).abs() < 1e-12);
        assert!(m[3][3].is_nan(), "constant column diagonal must stay NaN");
        assert!((m[0][2] - m[2][0]).abs() < 1e-12);
        // degenerate shapes
        assert!(pearson_matrix(&[]).is_empty());
        let single = pearson_matrix(&[vec![1.0]]);
        assert!(single[0][0].is_nan(), "n<2 observations is undefined");
    }

    #[test]
    fn spearman_matrix_matches_pairwise_spearman_rho() {
        // ranking once per column must give exactly what the per-pair `spearman_rho` reference
        // does, including its NaN semantics for a constant (all-ties) column
        let cols = vec![
            vec![1.0, 2.0, 3.0, 4.0, 5.0],
            // monotonic but sharply curved: rho == 1 while Pearson r is well below it
            vec![1.0, 2.0, 4.0, 40.0, 4000.0],
            vec![5.0, 3.0, 4.0, 1.0, 2.0], // negatively related
            vec![7.0, 7.0, 7.0, 7.0, 7.0], // constant -> all ranks tie -> undefined
        ];
        let m = spearman_matrix(&cols);
        for i in 0..cols.len() {
            for j in 0..cols.len() {
                let want = spearman_rho(&cols[i], &cols[j]);
                let got = m[i][j];
                assert_eq!(
                    want.is_nan(),
                    got.is_nan(),
                    "NaN disagreement at ({i},{j}): want {want} got {got}"
                );
                if !want.is_nan() {
                    assert!(
                        (want - got).abs() < 1e-12,
                        "({i},{j}): want {want} got {got}"
                    );
                }
            }
        }
        // the point of the switch: a monotone-but-curved pair is a perfect rank correlation, but
        // its Pearson r understates it badly
        assert!((m[0][1] - 1.0).abs() < 1e-12, "monotone pair must be rho=1");
        assert!(
            pearson_matrix(&cols)[0][1] < 0.75,
            "the curved pair's Pearson r should be far below its rho"
        );
        assert!(spearman_matrix(&[]).is_empty());
    }

    #[test]
    fn mean_is_outlier_driven_sees_tails_the_skewness_coefficient_misses() {
        // `neg` out of 1000 values; the rest split zero/positive around the median
        let col = |mean: f64, median: f64, stddev: f64, neg: u64| crate::cmd::stats::StatsData {
            r#type: "Float".to_string(),
            mean: Some(mean),
            q2_median: Some(median),
            stddev: Some(stddev),
            n_negative: Some(neg),
            n_zero: Some(0),
            n_positive: Some(1000 - neg),
            ..Default::default()
        };

        // The regression that motivated this predicate: `totalplannedcommit` from issue #4220's
        // NYC capital-projects repro. Pearson's second skewness is 0.38 — BELOW RIGHT_SKEW_MIN —
        // yet the mean is 41x the median. `is_right_skewed` must miss it and this must catch it;
        // if the two ever agree here, someone has collapsed them back into one gate.
        let cpdb = col(15_243_802.733, 367_000.0, 117_585_520.06, 0);
        assert!(
            !is_right_skewed(&cpdb),
            "the skewness coefficient saturates on this column — that is the whole point"
        );
        assert!(mean_is_outlier_driven(&cpdb));

        // zero-inflated: over half the rows are zero, so a positive mean is carried entirely by
        // the minority tail (the 45-60%-zeros money columns the issue reports)
        assert!(mean_is_outlier_driven(&col(
            2_258_404.0,
            0.0,
            31_116_974.0,
            0
        )));

        // a typical-looking column: the mean sits near the middle row
        assert!(!mean_is_outlier_driven(&col(59.5, 59.5, 30.0, 0)));
        // just under / at the 2x ratio
        assert!(!mean_is_outlier_driven(&col(19.0, 10.0, 5.0, 0)));
        assert!(mean_is_outlier_driven(&col(20.0, 10.0, 5.0, 0)));

        // a SIGNED series (profit/loss, an anomaly index) centred near zero must NOT be read as
        // tail-dominated just because its median lands on 0 — half its rows are negative
        assert!(!mean_is_outlier_driven(&col(0.4, 0.0, 12.0, 500)));
        // the negative tolerance is a small share, not zero: the repro's `totalplannedcommit`
        // carries 1.4% negative rollbacks and is still an additive amount
        assert!(mean_is_outlier_driven(&col(
            15_243_802.7,
            367_000.0,
            117_585_520.0,
            14
        )));
        // ... but a column that is one-fifth negative is not an additive positive-tail measure
        assert!(!mean_is_outlier_driven(&col(500.0, 10.0, 900.0, 200)));

        // a non-positive mean is not an additive positive-tail measure
        assert!(!mean_is_outlier_driven(&col(0.0, 0.0, 1.0, 0)));
        // an incomplete cache keeps the caller on its prior default
        assert!(!mean_is_outlier_driven(&crate::cmd::stats::StatsData {
            r#type: "Float".to_string(),
            n_negative: Some(0),
            ..Default::default()
        }));
        // sign counts missing entirely -> cannot judge -> prior default
        assert!(!mean_is_outlier_driven(&crate::cmd::stats::StatsData {
            r#type: "Float".to_string(),
            mean: Some(100.0),
            q2_median: Some(1.0),
            ..Default::default()
        }));
    }

    #[test]
    fn corr_prefers_spearman_needs_a_majority_of_outlier_driven_columns() {
        let col = |mean: f64, median: f64| crate::cmd::stats::StatsData {
            r#type: "Float".to_string(),
            mean: Some(mean),
            q2_median: Some(median),
            stddev: Some(1.0),
            n_negative: Some(0),
            n_zero: Some(0),
            n_positive: Some(1000),
            ..Default::default()
        };
        let heavy = col(100.0, 1.0); // mean 100x the median
        let tame = col(10.0, 9.0);
        assert!(mean_is_outlier_driven(&heavy));
        assert!(!mean_is_outlier_driven(&tame));

        let stats = vec![heavy.clone(), tame.clone(), heavy, tame];
        // exactly half clears the >= majority share
        assert!(corr_prefers_spearman(&stats, &[0, 1, 2, 3]));
        // one of three is below it
        assert!(!corr_prefers_spearman(&stats, &[0, 1, 3]));
        // both heavy
        assert!(corr_prefers_spearman(&stats, &[0, 2]));
        // no columns at all -> Pearson, never a divide-by-zero
        assert!(!corr_prefers_spearman(&stats, &[]));
        // `kept_indices` addresses the ORIGINAL stats, not matrix positions: an out-of-range index
        // must not panic and must not count
        assert!(!corr_prefers_spearman(&stats, &[1, 99]));
    }
    #[test]
    fn candidate_lons_probes_across_the_seam_for_adjacent_features() {
        // a feature hugging the dateline WITHOUT crossing it carries no wrap flag, but a query
        // just the other side is a fraction of a degree away going east — measured raw it reads
        // as ~359.5 degrees and the point is dropped as "outside every region".
        let near_seam = PipFeature {
            id:                 "S".to_string(),
            name:               None,
            polygons:           Vec::new(),
            bbox:               [179.5, -1.0, 180.0, 1.0],
            wraps_antimeridian: false,
        };
        let lons: Vec<f64> = near_seam
            .candidate_lons(-179.999)
            .into_iter()
            .flatten()
            .collect();
        assert!(
            lons.iter().any(|l| (*l - 180.001).abs() < 1e-9),
            "expected a +360 probe, got {lons:?}"
        );
        // an ordinary interior feature gets no second probe (it would only cost time)
        let interior = PipFeature {
            id:                 "I".to_string(),
            name:               None,
            polygons:           Vec::new(),
            bbox:               [-75.0, 39.0, -74.0, 41.0],
            wraps_antimeridian: false,
        };
        let lons: Vec<f64> = interior
            .candidate_lons(-74.5)
            .into_iter()
            .flatten()
            .collect();
        assert_eq!(lons.len(), 1, "interior features need only the raw probe");
        // ... and a query far to the EAST of a western feature probes -360
        let lons: Vec<f64> = interior
            .candidate_lons(179.0)
            .into_iter()
            .flatten()
            .collect();
        assert!(
            lons.iter().any(|l| (*l - (-181.0)).abs() < 1e-9),
            "expected a -360 probe, got {lons:?}"
        );
        // a genuinely wrapping feature keeps its existing both-representations behaviour
        let wrapping = PipFeature {
            id:                 "W".to_string(),
            name:               None,
            polygons:           Vec::new(),
            bbox:               [179.5, -1.0, 180.5, 1.0],
            wraps_antimeridian: true,
        };
        assert_eq!(
            wrapping
                .candidate_lons(-179.0)
                .into_iter()
                .flatten()
                .count(),
            2
        );
    }

    #[test]
    fn plotly_cdn_tag_carries_the_pinned_integrity_hash() {
        // What is verifiable offline is the WIRING: that the emitted tag uses the pinned
        // constants rather than some other value, and that the two CDN paths agree. Whether the
        // digest is the true hash of the served file cannot be checked here — the only oracle is
        // the CDN itself, and the vendored bundle is a different version — so that is verified at
        // the constant (see the regeneration command on PLOTLY_CDN_VERSION).
        let tag = plotly_cdn_only();
        assert!(
            tag.contains(&format!("plotly-{PLOTLY_CDN_VERSION}.min.js")),
            "emitted tag is not the pinned version: {tag}"
        );
        assert!(
            tag.contains(&format!("integrity=\"{PLOTLY_CDN_SRI}\"")),
            "emitted tag does not carry the pinned digest: {tag}"
        );
        assert!(tag.contains(r#"crossorigin="anonymous""#), "{tag}");
        // exactly one integrity attribute — the injection must not double-apply
        assert_eq!(tag.matches("integrity=").count(), 1, "{tag}");

        // the hand-written fallback must agree with the injected form on both values, else the
        // fallback path would serve an unverified (or wrongly-verified) bundle
        assert!(PLOTLY_CDN_FALLBACK.contains(&format!("plotly-{PLOTLY_CDN_VERSION}.min.js")));
        assert!(PLOTLY_CDN_FALLBACK.contains(PLOTLY_CDN_SRI));
        assert!(PLOTLY_CDN_FALLBACK.contains(r#"crossorigin="anonymous""#));

        // shape of the digest itself: sha384 is 48 bytes -> 64 base64 chars
        let payload = PLOTLY_CDN_SRI
            .strip_prefix("sha384-")
            .expect("SRI must declare its algorithm");
        assert_eq!(payload.len(), 64, "sha384 digest length: {payload}");
        assert!(
            payload
                .bytes()
                .all(|b| b.is_ascii_alphanumeric() || b == b'+' || b == b'/' || b == b'='),
            "digest is not base64: {payload}"
        );
    }

    #[test]
    fn generator_scripts_pin_the_same_plotly_version() {
        // `examples/viz/gen_gallery.py` and `scripts/gen_benchmark_viz.py` build their own pages
        // (the gallery frame and the benchmark Data Schematic shell) rather than going through
        // `smart_html_page`, so they carry a HAND-COPIED third-party notice naming the plotly
        // version. Nothing else ties those strings to `PLOTLY_CDN_VERSION`: bumping the constant
        // here would leave both scripts silently claiming the old version in every page they emit.
        // This assertion is that missing link.
        //
        // If this fails after a plotly bump, update the `THIRD_PARTY_COMMENT` literal in both
        // scripts to the new version.
        let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR"));
        for rel in [
            "examples/viz/gen_gallery.py",
            "scripts/gen_benchmark_viz.py",
        ] {
            let path = root.join(rel);
            // Neither script is in `Cargo.toml`'s package `include` list — they are repo tooling,
            // not crate content, and shipping them in the tarball would be pointless bloat. So
            // skip rather than fail when running from packaged sources; in a repo checkout (CI and
            // local dev — the only places this can actually catch drift) they are always present.
            if !path.is_file() {
                continue;
            }
            let src =
                std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("cannot read {rel}: {e}"));
            let needle = format!("plotly.js {PLOTLY_CDN_VERSION} -");
            assert!(
                src.contains(&needle),
                "{rel} does not pin plotly {PLOTLY_CDN_VERSION}. Its third-party notice is stale \
                 — update the THIRD_PARTY_COMMENT literal in that script to match \
                 PLOTLY_CDN_VERSION."
            );
        }
    }

    #[test]
    fn plotly_cdn_sri_matches_the_embedded_bundle() {
        use base64_simd::STANDARD as BASE64;
        use sha2::{Digest, Sha384};

        // The crate's vendored `resource/plotly.min.js` — the bundle this build INLINES — is
        // byte-identical to the file the pinned CDN URL serves, so it is a genuine offline oracle
        // for the integrity hash: this asserts the digest is the CORRECT one, not merely
        // well-formed. Crucially it fails the moment a plotly.rs bump changes the embedded
        // bundle without PLOTLY_CDN_SRI being regenerated, which is exactly the regression that
        // would otherwise ship a hash the browser rejects.
        let tag = plotly_js_only();
        let body = tag
            .find('>')
            .map(|i| &tag[i + 1..])
            .and_then(|rest| rest.strip_suffix("</script>"))
            .expect("plotly_js_only should emit a single <script>...</script>");
        let digest = BASE64.encode_to_string(Sha384::digest(body.as_bytes()));
        assert_eq!(
            format!("sha384-{digest}"),
            PLOTLY_CDN_SRI,
            "PLOTLY_CDN_SRI is stale: the embedded plotly.js bundle hashes to sha384-{digest}. \
             Regenerate it (see the command on PLOTLY_CDN_VERSION) and update PLOTLY_CDN_VERSION \
             to match the bundle."
        );
    }

    #[test]
    fn datatables_bundle_is_plain_embed_safe() {
        // The plain (QSV_VIZ_NO_COMPRESS) path inlines the vendored bundle with NO escaping, so
        // neither file may contain its own close tag. (The gz-b64 path is `</script>`-safe by
        // construction — base64 has no `<`.)
        assert!(
            !DATATABLES_JS.contains("</script"),
            "vendored datatables.min.js contains `</script` — plain inline embedding would \
             truncate the tag"
        );
        assert!(
            !DATATABLES_CSS.contains("</style"),
            "vendored datatables.min.css contains `</style` — plain inline embedding would \
             truncate the tag"
        );
        // the download-builder header comment must confirm the pinned component versions
        for component in DATATABLES_CDN_COMBO.split('/') {
            let (code, version) = component
                .split_once('-')
                .expect("combo segments are code-version");
            let name = match code {
                "dt" => "DataTables",
                "b" => "Buttons",
                "cc" => "ColumnControl",
                "date" => "DateTime",
                "r" => "Responsive",
                "sb" => "SearchBuilder",
                other => panic!("unknown combo component {other}"),
            };
            let marker = format!("{name} {version}");
            assert!(
                DATATABLES_JS.contains(&marker),
                "vendored datatables.min.js header lacks \"{marker}\" — bundle and \
                 DATATABLES_CDN_COMBO are out of sync"
            );
            assert!(
                DATATABLES_CSS.contains(&marker),
                "vendored datatables.min.css header lacks \"{marker}\" — bundle and \
                 DATATABLES_CDN_COMBO are out of sync"
            );
        }
    }

    /// The bundle's component inventory is written out in THREE places that have to agree, and
    /// nothing tied them together — so bumping the bundle to add `ColumnControl` updated two of
    /// them and silently left the third describing the old combo (roborev 3916):
    ///
    /// 1. `THIRD_PARTY_NOTICES.md` — the repo-root notice, and the one that matters most: it is
    ///    what `THIRD_PARTY_NOTICES_URL` points at, which every generated Data Schematic carries in
    ///    its footer and HTML comment. Stale here means the notice readers are *sent to*
    ///    under-reports what the page actually ships.
    /// 2. `src/cmd/assets/LICENSE-DataTables.txt` — the vendored license + inventory.
    /// 3. `third_party_comment` — the short in-page attribution line.
    ///
    /// `DATATABLES_CDN_COMBO` is the source of truth; `datatables_bundle_is_plain_embed_safe`
    /// already pins it to the vendored builder header, so checking the prose against the combo
    /// transitively checks it against the actual bytes.
    #[test]
    fn datatables_attribution_matches_the_pinned_combo() {
        // Whitespace is normalised before matching because these are prose/tabular files: the
        // license aligns its inventory into columns ("DataTables    3.0.0") and the notices
        // markdown wraps mid-list, so a literal "Name X.Y.Z" would not match either as written.
        fn flatten(s: &str) -> String {
            s.split_whitespace().collect::<Vec<_>>().join(" ")
        }

        let manifest = env!("CARGO_MANIFEST_DIR");
        let notices_raw = std::fs::read_to_string(format!("{manifest}/THIRD_PARTY_NOTICES.md"))
            .expect("THIRD_PARTY_NOTICES.md is readable");
        let license_raw =
            std::fs::read_to_string(format!("{manifest}/src/cmd/assets/LICENSE-DataTables.txt"))
                .expect("src/cmd/assets/LICENSE-DataTables.txt is readable");
        let notices = flatten(&notices_raw);
        let license = flatten(&license_raw);
        // `datatables: true` is the branch that emits the DataTables credit line
        let in_page = flatten(&third_party_comment(true, false));

        for (label, text) in [
            ("THIRD_PARTY_NOTICES.md", &notices),
            ("src/cmd/assets/LICENSE-DataTables.txt", &license),
        ] {
            assert!(
                text.contains(DATATABLES_CDN_COMBO),
                "{label} does not name the pinned download-builder combination \
                 \"{DATATABLES_CDN_COMBO}\" — it still describes an older bundle. Update it to \
                 match DATATABLES_CDN_COMBO."
            );
        }

        for component in DATATABLES_CDN_COMBO.split('/') {
            let (code, version) = component
                .split_once('-')
                .expect("combo segments are code-version");
            // same code -> display-name mapping as datatables_bundle_is_plain_embed_safe
            let name = match code {
                "dt" => "DataTables",
                "b" => "Buttons",
                "cc" => "ColumnControl",
                "date" => "DateTime",
                "r" => "Responsive",
                "sb" => "SearchBuilder",
                other => panic!("unknown combo component {other}"),
            };
            let marker = format!("{name} {version}");
            for (label, text) in [
                ("THIRD_PARTY_NOTICES.md", &notices),
                ("src/cmd/assets/LICENSE-DataTables.txt", &license),
            ] {
                assert!(
                    text.contains(&marker),
                    "{label} does not list \"{marker}\" — the bundle ships {name} {version} but \
                     the notice's component inventory omits it (or pins another version)."
                );
            }
            // The in-page line is deliberately terse: it prints the CORE version once and then
            // slash-joins the extension names without versions. So the core is held to the full
            // "DataTables <version>" marker — that version is a hardcoded literal in
            // `third_party_comment` and is the one number a reader sees on the page, so a
            // name-only check there would let it drift from the combo unnoticed. Extensions,
            // whose versions the line genuinely omits, are checked by name.
            let expected = if code == "dt" {
                marker.clone()
            } else {
                name.to_string()
            };
            assert!(
                in_page.contains(&expected),
                "third_party_comment's DataTables credit is missing \"{expected}\" — a Data \
                 Schematic would attribute a different component set (or DataTables version) than \
                 it actually embeds. Line was: {in_page}"
            );
        }
    }

    #[test]
    fn datatables_cdn_sri_matches_the_vendored_bundle() {
        use base64_simd::STANDARD as BASE64;
        use sha2::{Digest, Sha384};

        // The version-pinned cdn.datatables.net URLs serve exactly the vendored bytes, so the
        // vendored files are an offline oracle for the SRI digests — this fails the moment the
        // bundle is bumped without the hashes (or DATATABLES_CDN_COMBO) being regenerated.
        let js_digest = BASE64.encode_to_string(Sha384::digest(DATATABLES_JS.as_bytes()));
        assert_eq!(
            format!("sha384-{js_digest}"),
            DATATABLES_CDN_JS_SRI,
            "DATATABLES_CDN_JS_SRI is stale: the vendored datatables.min.js hashes to \
             sha384-{js_digest}. Regenerate it (see the command on DATATABLES_JS)."
        );
        let css_digest = BASE64.encode_to_string(Sha384::digest(DATATABLES_CSS.as_bytes()));
        assert_eq!(
            format!("sha384-{css_digest}"),
            DATATABLES_CDN_CSS_SRI,
            "DATATABLES_CDN_CSS_SRI is stale: the vendored datatables.min.css hashes to \
             sha384-{css_digest}. Regenerate it (see the command on DATATABLES_JS)."
        );
    }

    #[test]
    fn escape_template_pct_doubles_literal_percent() {
        // a hovertemplate parses `%{...}`, so a literal `%` in an interpolated label must be
        // doubled or plotly swallows the following text as a data token
        assert_eq!(escape_template_pct("50% of total"), "50%% of total");
        assert_eq!(escape_template_pct("Pct % complete"), "Pct %% complete");
        // a value that already looks like a token is neutralized rather than honored
        assert_eq!(escape_template_pct("%{x}"), "%%{x}");
        // no-op on percent-free text
        assert_eq!(escape_template_pct("plain label"), "plain label");
        assert_eq!(escape_template_pct(""), "");
    }

    #[test]
    fn escape_composition_is_once_per_sink() {
        // the two helpers compose in one fixed order for template sinks (see the choropleth hover
        // and the AnimatedBubble template); markup first, then `%`-doubling
        let nasty = "R&D <b>50%</b>";
        let composed = escape_template_pct(&escape_hover(nasty));
        assert_eq!(composed, "R&amp;D &lt;b&gt;50%%&lt;/b&gt;");
        // a legend name renders markup but does NOT parse `%`, so it takes escape_hover ALONE —
        // `%`-doubling there would surface a literal "%%" to the reader
        assert_eq!(escape_hover(nasty), "R&amp;D &lt;b&gt;50%&lt;/b&gt;");
        // escaping twice is the bug this ordering guards against
        assert_ne!(escape_hover(&escape_hover(nasty)), escape_hover(nasty));
        assert!(escape_hover(&escape_hover(nasty)).contains("&amp;amp;"));
    }

    #[test]
    fn contour_hover_template_neutralizes_percent_bearing_headers() {
        // this helper reads the process-global locale now that its template lives in the catalog,
        // so an English assertion has to pin the locale -- cargo runs unit tests on parallel
        // threads and a locale-mutating test would otherwise make this fail nondeterministically
        let _locale = english_locale();

        // the labels are raw column headers dropped into a template that parses `%{...}`, so a
        // header carrying `%` -- or one that already looks like a token -- must be neutralized
        // rather than honored, and markup must be escaped before the `%`-doubling
        let t = contour_hover_template("% of total", "%{x}");
        assert!(
            t.starts_with("%% of total: %{x:.3s}"),
            "a literal `%` in a header must be doubled, got: {t}"
        );
        assert!(
            t.contains("%%{x}: %{y:.3s}"),
            "a token-shaped header must be neutralized, got: {t}"
        );
        // the template's OWN tokens survive intact, and the trace-name box stays suppressed
        assert!(t.ends_with("%{z:,} rows<extra></extra>"), "got: {t}");

        // markup first, then `%`-doubling -- the same one-order composition as the other sinks
        let m = contour_hover_template("R&D 50%", "plain");
        assert!(m.starts_with("R&amp;D 50%%: "), "got: {m}");
    }

    #[test]
    fn lorenz_hover_template_neutralizes_percent_bearing_labels() {
        // same process-global locale dependency as the contour test above -- pin it before
        // asserting English
        let _locale = english_locale();

        // the Lorenz hover interpolates a raw measure label into a template that parses `%{...}`,
        // exactly as the contour hover does -- `%` in a header is NOT an intensive-measure token
        // (tokenization splits on non-alphanumerics), so such a column really can reach a Lorenz
        // panel and really can carry a `%` into the template
        let t = lorenz_hover_template("% of total", 0.87);
        assert!(
            t.starts_with("%% of total (Gini 0.87)"),
            "a literal `%` in the label must be doubled, got: {t}"
        );
        // a token-shaped label is neutralized rather than honored
        assert!(
            lorenz_hover_template("%{x}", 0.5).starts_with("%%{x} (Gini 0.50)"),
            "a token-shaped label must be neutralized"
        );
        // markup first, then `%`-doubling -- the one fixed order shared by every template sink
        assert!(
            lorenz_hover_template("R&D 50%", 0.5).starts_with("R&amp;D 50%% "),
            "markup must be escaped before the `%`-doubling"
        );
        // the template's OWN tokens survive intact, and the trace-name box stays suppressed
        assert!(
            t.ends_with("bottom %{x:.0%} of records hold %{y:.0%}<extra></extra>"),
            "got: {t}"
        );
    }

    /// Pins the rust-i18n interpolation semantics that localizing a hover template depends on.
    /// `t!` substitutes its arguments via `rust_i18n::replace_patterns`, which uses the SAME
    /// `%{...}` token syntax plotly does -- so a localized hover template is parsed twice: once by
    /// rust-i18n when the Data Schematic is rendered, once by plotly in the browser. Three
    /// properties make that cohabitation safe. The fourth is a trap, and it fails silently.
    #[test]
    fn rust_i18n_interpolation_cohabits_with_plotly_tokens() {
        use rust_i18n::replace_patterns;

        // 1. a token rust-i18n was given no argument for is emitted verbatim, so plotly's own
        //    tokens survive a `t!` round trip untouched. This is what makes the `q_` argument
        //    prefix sufficient to keep the two namespaces from colliding.
        assert_eq!(
            replace_patterns(
                "%{q_col}: %{x:.3s}<br>%{z:,} rows",
                &["q_col"],
                &["Revenue".to_string()]
            ),
            "Revenue: %{x:.3s}<br>%{z:,} rows"
        );

        // 2. a doubled literal `%` -- what `escape_template_pct` emits -- passes through untouched,
        //    so the existing `%`-escaping composition still means what it meant before.
        assert_eq!(
            replace_patterns("50%% of %{q_col}", &["q_col"], &["total".to_string()]),
            "50%% of total"
        );

        // 3. an argument VALUE is never re-scanned, so a token-shaped column header interpolated as
        //    an argument cannot smuggle in a second round of substitution.
        assert_eq!(
            replace_patterns("%{q_col} rows", &["q_col"], &["%{x}".to_string()]),
            "%{x} rows"
        );

        // 4. THE TRAP. The scanner resets to "seeking `{`" on EVERY `%`, so a plotly token carrying
        //    a literal `%` inside its own braces -- `%{x:.0%}`, the Lorenz axis format -- never
        //    registers its closing brace and mis-pairs with the NEXT `{` in the string. Every
        //    `%{q_*}` argument downstream of such a token is then silently left unsubstituted: no
        //    panic, no warning, no failing key-coverage check, just the raw key in a user's
        //    tooltip. Argument tokens MUST precede any percent-formatted plotly token.
        assert_eq!(
            replace_patterns(
                "bottom %{x:.0%} of %{q_col}",
                &["q_col"],
                &["sales".to_string()]
            ),
            "bottom %{x:.0%} of %{q_col}",
            "known hazard: an argument placed after a percent-formatted plotly token is NOT \
             substituted -- keep `%{{q_*}}` ahead of any `%{{..%}}` token"
        );
        // the same argument ahead of that token resolves normally -- ordering is the whole fix
        assert_eq!(
            replace_patterns(
                "%{q_col}: bottom %{x:.0%}",
                &["q_col"],
                &["sales".to_string()]
            ),
            "sales: bottom %{x:.0%}"
        );
    }

    #[test]
    fn complement_marker_warns_but_does_not_veto() {
        // `unspent_balance` is the one bad declaration that disclosing containment cannot catch:
        // a complement nests PERFECTLY inside its predecessor, so it reports 0% violations while
        // rendering a funnel that means the opposite of the truth.
        for name in [
            "unspent",
            "unspent_balance",
            "remaining_budget",
            "variance_amt",
            "fund_balance",
            "committed_delta",
        ] {
            assert!(
                is_complement_name("", name),
                "{name} should be flagged as a complement"
            );
        }
        // The COLLISION half of the old marker list died with the name-matching vocabulary it
        // existed to defend. Those words only ever mattered because a greedy substring match
        // could mistake them for stages; warning on a column the dictionary explicitly DECLARED
        // would be a pure false alarm.
        for name in [
            "review_count",
            "contractor_amt",
            "wholesale_total",
            "lead_time_days",
            "replaced_units",
        ] {
            assert!(
                !is_complement_name("", name),
                "{name} is an ordinary declared column, not a complement"
            );
        }
        // and a genuine stage is never flagged
        assert!(!is_complement_name("", "spentamt"));
        assert!(!is_complement_name("", "totalplannedcommit"));
    }

    #[test]
    fn containment_fraction_tolerates_float_noise_and_counts_violations() {
        // exact equality on large values must not read as a violation
        let big = vec![1e12, 2e12, 3e12];
        assert!((containment_fraction(&big, &big) - 1.0).abs() < f64::EPSILON);

        // 1 of 4 rows breaks the nesting
        let up = vec![10.0, 10.0, 10.0, 10.0];
        let down = vec![1.0, 2.0, 3.0, 99.0];
        assert!((containment_fraction(&up, &down) - 0.75).abs() < 1e-12);

        // an empty pipeline is not a contained one
        assert!((containment_fraction(&[], &[]) - 0.0).abs() < f64::EPSILON);
    }

    #[test]
    fn pipeline_form_follows_the_totals_not_the_rows() {
        use PipelineForm::{Bridge, Funnel};
        // monotonically shrinking: the containment claim holds
        assert_eq!(PipelineForm::for_totals(&[100.0, 60.0, 40.0]), Funnel);
        // flat stages still nest (a plateau is not growth)
        assert_eq!(PipelineForm::for_totals(&[100.0, 100.0, 40.0]), Funnel);
        // one growing step is enough to make a funnel dishonest -- this is the CPDB shape,
        // where Spent (81G) outruns Committed (28.4G)
        assert_eq!(
            PipelineForm::for_totals(&[191.75, 28.38, 80.98]),
            Bridge,
            "a stage that outruns its predecessor must not be drawn as containment"
        );
        // growth anywhere counts, including the last step only
        assert_eq!(PipelineForm::for_totals(&[100.0, 60.0, 61.0]), Bridge);
        // degenerate inputs are funnels by default -- nothing contradicts containment
        assert_eq!(PipelineForm::for_totals(&[]), Funnel);
        assert_eq!(PipelineForm::for_totals(&[42.0]), Funnel);
    }

    #[test]
    fn complete_pct_str_never_rounds_up_to_100() {
        // the real CPDB case: 12,566 of 12,587 rows. `{:.0}` rendered this as "100", asserting
        // completeness while a nine-figure sum was silently dropped.
        assert_eq!(complete_pct_str(12_566.0 / 12_587.0), "99.8");

        // rounding would reach 100 well before completeness does; flooring must not
        assert_eq!(complete_pct_str(0.9996), "99.9");
        assert_eq!(complete_pct_str(0.999_999), "99.9");

        // only genuine completeness prints a bare 100
        assert_eq!(complete_pct_str(1.0), "100");

        // and the ordinary cases still read naturally
        assert_eq!(complete_pct_str(0.5), "50.0");
        assert_eq!(complete_pct_str(0.0), "0.0");
    }

    #[test]
    fn funnel_left_margin_sizes_to_the_stage_labels() {
        let stages: Vec<String> = ["Planned", "Committed", "Spent"]
            .iter()
            .map(|s| (*s).to_string())
            .collect();
        // "Committed" is 9 chars -> 9*7 + 24. The bug was borrowing the lollipop's 56-char
        // budget (56*7 + 24 = 416px) for labels that need a fraction of it.
        assert_eq!(funnel_left_margin(&stages), 9 * CORR_LABEL_PX_PER_CHAR + 24);
        assert!(funnel_left_margin(&stages) < TOPREL_LABEL_MAX_CHARS * CORR_LABEL_PX_PER_CHAR + 24);

        // an absurdly long stage name is still capped at the lollipop budget
        let long = vec!["x".repeat(200)];
        assert_eq!(
            funnel_left_margin(&long),
            TOPREL_LABEL_MAX_CHARS * CORR_LABEL_PX_PER_CHAR + 24
        );

        // no stages: fall back to the default left margin rather than zero
        assert_eq!(funnel_left_margin(&[]), 60);
    }

    #[test]
    fn funnel_value_axis_is_symmetric_about_zero() {
        // plotly centres funnel bands on the value zero, so a 0..=ceil range would draw half of
        // every band outside the plot area (only survivable while an oversized left margin
        // absorbed the overflow).
        let axis = serde_json::to_value(funnel_value_axis(100.0, None)).unwrap();
        let range = axis["range"].as_array().unwrap();
        let lo = range[0].as_f64().unwrap();
        let hi = range[1].as_f64().unwrap();
        assert!(lo < 0.0 && hi > 0.0, "range must straddle zero: {lo}..{hi}");
        assert!(
            (lo + hi).abs() < 1e-9,
            "range must be symmetric: {lo}..{hi}"
        );
        // the widest band still occupies 1/1.35 of the plot area
        assert!((hi - 100.0 * 1.35 / 2.0).abs() < 1e-9);
        // and its tick labels stay hidden: a symmetric range would read as negative amounts
        assert_eq!(axis["showticklabels"], serde_json::json!(false));
    }

    #[test]
    fn panel_title_escapes_markup_with_and_without_subtitle() {
        // a hostile column header must not reach plotly's markup renderer as live markup on
        // EITHER branch — the no-subtitle branch is the default (no --dictionary) path
        let hostile = "<a href=\"https://evil.example\">Revenue</a>";
        let bare = Panel::new(
            hostile.to_string(),
            PanelKind::BoxRaw {
                idx:    0,
                points: BoxPoints::Outliers,
            },
        );
        let title = bare.display_title();
        assert!(
            !title.contains("<a href"),
            "raw anchor markup leaked into the panel title: {title}"
        );
        assert!(
            title.contains("&lt;a href"),
            "expected escaped markup: {title}"
        );

        // with a subtitle, the title line is still escaped and the subtitle keeps its own escaping
        let with_sub = Panel::new(
            hostile.to_string(),
            PanelKind::BoxRaw {
                idx:    0,
                points: BoxPoints::Outliers,
            },
        )
        .with_subtitle(Some("R&D <b>label</b>".to_string()));
        let title = with_sub.display_title();
        assert!(!title.contains("<a href"));
        assert!(title.contains("&lt;a href"));
        assert!(title.contains("R&amp;D &lt;b&gt;label&lt;/b&gt;"));
        // the styling markup the title itself emits must survive as real markup
        assert!(title.contains("<br><span style="));
    }

    #[test]
    fn panel_trace_name_escapes_markup_exactly_once() {
        // issue #4331: the panel TITLE was escaped (test above) but the trace NAME was not, so a
        // hostile CSV header still reached plotly's markup renderer via the legend/hover box.
        // `Panel::name` is stored raw (it also feeds stderr), so the escape belongs at this sink.
        let hostile = "R&D <b>Revenue</b>";
        let panel = Panel::new(
            hostile.to_string(),
            PanelKind::BoxStats {
                q1:     1.0,
                median: 2.0,
                q3:     3.0,
                lower:  Some(0.0),
                upper:  Some(4.0),
                mean:   Some(2.0),
            },
        );
        let (trace, ..) = panel_trace_box_stats(&panel, "#4c78a8", None);
        let json = trace.to_json();

        // rendered as literal text, not markup
        assert!(
            json.contains(r"R&amp;D &lt;b&gt;Revenue&lt;/b&gt;"),
            "trace name must be escaped, got: {json}"
        );
        // ...and escaped exactly ONCE. A double-escape is invisible to the golden check
        // (no committed fixture header carries `&`/`<`/`>`), so it has to be pinned here.
        assert!(
            !json.contains(r"&amp;amp;"),
            "trace name must not be double-escaped, got: {json}"
        );
    }

    #[test]
    fn log_safe_values_drops_non_positive_only_on_log_axis() {
        // issue #4219, raw-value half: `box_log_skew_fallback` grants a log axis to a skewed
        // column that still holds a few zeros/negatives. BoxRaw/Violin hand these values to
        // plotly, which derives the whisker/KDE from them — a single 0 re-creates the
        // log-undefined fence and breaks the whole trace.
        let vals = vec![-5.0, 0.0, 1.0, 10.0, 1000.0];
        assert_eq!(
            log_safe_values(true, vals.clone()),
            vec![1.0, 10.0, 1000.0],
            "log axis must not receive non-positive values"
        );
        // linear axis: untouched, including the non-positive tail
        assert_eq!(log_safe_values(false, vals.clone()), vals);
        // all-positive input is unchanged on either axis
        let pos = vec![1.0, 2.0, 3.0];
        assert_eq!(log_safe_values(true, pos.clone()), pos);
        assert_eq!(log_safe_values(false, pos.clone()), pos);
    }

    #[test]
    fn build_kpi_row_joins_on_stat_idx_not_decorated_title() {
        // KPI tile labels are localized -- pin English, or the "Total amount"
        // assertion below races the Spanish-setting tests on the global locale.
        let _locale = english_locale();
        // `build_smart` decorates a panel title AFTER construction ("amount (right-skewed)" from
        // `box_shape_hint`, "(sampled)" for strided violins), so a name-based stats join silently
        // missed and the KPI tile — often the whole KPI row — disappeared.
        let stats = vec![crate::cmd::stats::StatsData {
            field: "amount".to_string(),
            r#type: "Float".to_string(),
            sum: Some(5000.0),
            mean: Some(10.0),
            ..Default::default()
        }];
        let kind = || PanelKind::BoxStats {
            q1:     1.0,
            median: 2.0,
            q3:     3.0,
            lower:  Some(0.0),
            upper:  Some(9.0),
            mean:   Some(2.0),
        };

        // decorated title + recorded stat_idx => tile still built
        let decorated =
            vec![Panel::new("amount (right-skewed)".to_string(), kind()).with_stat_idx(0)];
        let row = build_kpi_row(&stats, &decorated, None).expect("KPI row for decorated panel");
        let PanelKind::KpiRow { tiles } = &row.kind else {
            panic!("expected a KpiRow");
        };
        assert_eq!(tiles.len(), 1);
        assert_eq!(tiles[0].label, "Total amount");
        assert!((tiles[0].value - 5000.0).abs() < f64::EPSILON);

        // headerless input: `name` is a positional placeholder and `s.field` is empty, so only the
        // index can carry the identity
        let headerless_stats = vec![crate::cmd::stats::StatsData {
            field: String::new(),
            ..stats[0].clone()
        }];
        let positional = vec![Panel::new("col 1".to_string(), kind()).with_stat_idx(0)];
        assert!(
            build_kpi_row(&headerless_stats, &positional, None).is_some(),
            "headerless columns must still produce a KPI tile"
        );

        // negative control: an overview panel carries no stats row, so it contributes no tile
        let overview = vec![Panel::new("amount".to_string(), kind())];
        assert!(build_kpi_row(&stats, &overview, None).is_none());
    }

    #[test]
    fn cumulative_embedded_points_counts_frame_repetition() {
        // 3 buckets x 10 points: base trace (30) + frames holding 10, 20, 30 => 90
        let buckets = vec![vec![0.0; 10], vec![0.0; 10], vec![0.0; 10]];
        assert_eq!(cumulative_embedded_points(&buckets), 30 + 60);
        // the distinct count (30) is what the old budget used — it undercounts by 3x here, and by
        // ~nb/2 in general, which is how a 150k-point cap produced multi-MB panels
        let distinct: usize = buckets.iter().map(Vec::len).sum();
        assert!(cumulative_embedded_points(&buckets) > distinct * 2);

        // front-loaded buckets are repeated the most
        let front = vec![vec![0.0; 30], vec![0.0; 0], vec![0.0; 0]];
        let back = vec![vec![0.0; 0], vec![0.0; 0], vec![0.0; 30]];
        assert!(cumulative_embedded_points(&front) > cumulative_embedded_points(&back));

        // degenerate inputs
        assert_eq!(cumulative_embedded_points(&[]), 0);
        assert_eq!(cumulative_embedded_points(&[vec![0.0; 5]]), 10);
    }

    #[test]
    fn panel_interest_prefers_informative_panels() {
        let boxkind = PanelKind::BoxStats {
            q1:     1.0,
            median: 2.0,
            q3:     3.0,
            lower:  Some(0.5),
            upper:  Some(10.0),
            mean:   Some(2.5),
        };
        // a high-spread measure outranks a near-constant one
        let mut spread = stat("Float", 1000, Some(0.5));
        spread.cv = Some(150.0);
        let mut constant = stat("Float", 1000, Some(0.5));
        constant.cv = Some(1.0);
        assert!(panel_interest(&spread, &boxkind) > panel_interest(&constant, &boxkind));

        // a mostly-null column is discounted
        let mut sparse = spread.clone();
        sparse.sparsity = Some(0.9);
        assert!(panel_interest(&spread, &boxkind) > panel_interest(&sparse, &boxkind));

        // a multi-category bar outranks a single-category one dominated by its mode
        let barkind = PanelKind::FreqBar { idx: 0 };
        let varied = stat("String", 12, Some(0.01));
        let mut dominated = stat("String", 2, Some(0.002));
        dominated.mode_occurrences = Some(950);
        dominated.uniqueness_ratio = Some(0.002); // 2/0.002 = 1000 rows -> 95% dominance
        assert!(panel_interest(&varied, &barkind) > panel_interest(&dominated, &barkind));

        // a bimodal histogram is always notable
        let hist = PanelKind::Histogram { idx: 0 };
        let plain = stat("Float", 1000, Some(0.5));
        assert!(panel_interest(&plain, &hist) > panel_interest(&plain, &boxkind));
    }

    #[test]
    fn guardrail_bars_zero_padded_measures_whatever_the_dictionary_says() {
        // stats only emits zero_padded_numeric for String-typed columns (leading zeros force
        // String inference), so the realistic mistag is an LLM calling a String code column a
        // measure. An explicit measure.* concept normally wins, but leading zeros are harder
        // evidence: the guardrail downgrades to a Dimension bar. Without it the column would be
        // DROPPED (a String column has no quartiles for the measure path).
        let mut code = stat("String", 200, Some(0.2));
        code.zero_padded_numeric = Some(true);
        assert_eq!(
            derive_semantics(&code, Some(&dict_row("", "measure", "measure.amount", ""))).route,
            Route::Dimension
        );
        // without the flag, the same explicit measure verdict is trusted as-is
        let plain = stat("Float", 200, Some(0.2));
        assert_eq!(
            derive_semantics(&plain, Some(&dict_row("", "measure", "measure.amount", ""))).route,
            Route::Measure
        );
    }

    #[test]
    fn sort_order_rank_prefers_physically_sorted_columns() {
        let with_order = |order: Option<&str>| {
            let mut s = stat("Date", 100, Some(0.5));
            s.sort_order = order.map(String::from);
            s
        };
        // a column the file is ordered by (either direction) outranks an unsorted/unknown one,
        // whatever the casing (stats emits "Ascending"; STATS_DEFINITIONS.md says "ASCENDING")
        assert_eq!(sort_order_rank(&with_order(Some("Ascending"))), 0);
        assert_eq!(sort_order_rank(&with_order(Some("ASCENDING"))), 0);
        assert_eq!(sort_order_rank(&with_order(Some("Descending"))), 0);
        assert_eq!(sort_order_rank(&with_order(Some("DESCENDING"))), 0);
        assert_eq!(sort_order_rank(&with_order(Some("Unsorted"))), 1);
        assert_eq!(sort_order_rank(&with_order(None)), 1);
    }

    #[test]
    fn strongest_pair_picks_max_abs_r_and_skips_nan() {
        // off-diagonal magnitudes: |(0,1)|=0.2, |(0,2)|=0.9, (1,2)=NaN (skipped).
        // strongest by |r| is the (0,2) pair, returned with its signed r.
        let m = vec![
            vec![1.0, 0.2, -0.9],
            vec![0.2, 1.0, f64::NAN],
            vec![-0.9, f64::NAN, 1.0],
        ];
        assert_eq!(strongest_pair(&m), Some((0, 2, -0.9)));

        // every off-diagonal cell undefined => no pair
        let all_nan = vec![vec![f64::NAN, f64::NAN], vec![f64::NAN, f64::NAN]];
        assert_eq!(strongest_pair(&all_nan), None);

        // fewer than two columns => no pair
        assert_eq!(strongest_pair(&[vec![1.0]]), None);
    }

    #[test]
    fn least_redundant_third_skips_undefined_correlations() {
        // Pair (0,1) is chosen. Candidates are cols 2 and 3.
        // col 2: max(|r02|,|r12|) = max(0.8, 0.7) = 0.8
        // col 3: max(|r03|,|r13|) = max(0.1, 0.2) = 0.2  <- least redundant
        let m = vec![
            vec![1.0, 0.9, 0.8, 0.1],
            vec![0.9, 1.0, 0.7, 0.2],
            vec![0.8, 0.7, 1.0, 0.3],
            vec![0.1, 0.2, 0.3, 1.0],
        ];
        assert_eq!(least_redundant_third(&m, 0, 1), Some(3));

        // col 3 became constant on the complete rows => NaN correlations.
        // Even though it would score "lowest" via the Equal fallback, it must be
        // excluded; the only finite candidate (col 2) is picked instead.
        let m_nan = vec![
            vec![1.0, 0.9, 0.8, f64::NAN],
            vec![0.9, 1.0, 0.7, f64::NAN],
            vec![0.8, 0.7, 1.0, f64::NAN],
            vec![f64::NAN, f64::NAN, f64::NAN, 1.0],
        ];
        assert_eq!(least_redundant_third(&m_nan, 0, 1), Some(2));

        // Every third-axis candidate is undefined => no usable third axis.
        let m_all_nan = vec![
            vec![1.0, 0.9, f64::NAN],
            vec![0.9, 1.0, f64::NAN],
            vec![f64::NAN, f64::NAN, 1.0],
        ];
        assert_eq!(least_redundant_third(&m_all_nan, 0, 1), None);
    }

    #[test]
    fn mask_to_lower_triangle_blanks_upper_and_diagonal() {
        let m = vec![
            vec![1.0, 0.5, 0.2],
            vec![0.5, 1.0, 0.9],
            vec![0.2, 0.9, 1.0],
        ];
        let masked = mask_to_lower_triangle(m);
        for r in 0..3 {
            for c in 0..3 {
                if c >= r {
                    assert!(
                        masked[r][c].is_nan(),
                        "upper/diagonal cell ({r},{c}) should be NaN"
                    );
                } else {
                    assert!(
                        masked[r][c].is_finite(),
                        "lower cell ({r},{c}) should be kept"
                    );
                }
            }
        }
        // a kept lower-triangle value is preserved unchanged
        assert!((masked[2][1] - 0.9).abs() < 1e-9);
    }

    #[test]
    fn spearman_rho_sees_monotonic_curve_pearson_misses() {
        // y = x^5 is perfectly monotonic but strongly curved: Spearman = 1.0 while Pearson ~0.82,
        // so the |rho| - |r| gap flags the nonlinearity a single Pearson number hides.
        let x: Vec<f64> = (0..20).map(f64::from).collect();
        let y: Vec<f64> = x.iter().map(|v| v.powi(5)).collect();
        let rho = spearman_rho(&x, &y);
        let r = pearson(&x, &y);
        assert!(
            (rho - 1.0).abs() < 1e-9,
            "spearman should be 1.0, got {rho}"
        );
        assert!(
            r < 0.95,
            "pearson should trail spearman for a curve, got {r}"
        );
        assert!(rho.abs() - r.abs() >= SMART_NONLINEAR_MIN_GAP);

        // tie-averaging: equal values share the mean of the ranks they span.
        let ranks = average_ranks(&[10.0, 10.0, 20.0]);
        assert!((ranks[0] - 1.5).abs() < 1e-9 && (ranks[1] - 1.5).abs() < 1e-9);
        assert!((ranks[2] - 3.0).abs() < 1e-9);
    }

    #[test]
    fn geo_framing_picks_projection_by_extent() {
        // a tight local cluster (LA area, ~1 deg span) -> Mercator fit to padded bounds
        let lats: Vec<f64> = (0..20).map(|i| 34.0 + i as f64 * 0.05).collect();
        let lons: Vec<f64> = (0..20).map(|i| -118.0 + i as f64 * 0.05).collect();
        let (proj, lonaxis, lataxis) = geo_framing(&lats, &lons, MAP_FRAME_TRIM_FRAC);
        assert!(matches!(proj, ProjectionType::Mercator));
        assert!(
            lonaxis.is_some() && lataxis.is_some(),
            "local extent should be fit to bounds on both axes"
        );

        // coordinates spanning the continental US -> albers usa, no fitted ranges
        let us_lats = [40.7_f64, 34.0, 41.9, 29.8, 47.6, 25.8, 39.7];
        let us_lons = [-74.0_f64, -118.2, -87.6, -95.4, -122.3, -80.2, -105.0];
        let (proj, lonaxis, lataxis) = geo_framing(&us_lats, &us_lons, MAP_FRAME_TRIM_FRAC);
        assert!(matches!(proj, ProjectionType::AlbersUsa));
        assert!(
            lonaxis.is_none() && lataxis.is_none(),
            "albers usa frames the US, so no axis ranges"
        );

        // a US extent that includes Alaska/Hawaii spans >90 deg of longitude, exceeding the global
        // threshold; the US heuristic must still win (albers usa), not fall through to
        // NaturalEarth.
        let akhi_lats = [21.3_f64, 61.2, 34.0, 41.9, 40.7, 42.4];
        let akhi_lons = [-157.8_f64, -149.9, -118.2, -87.6, -74.0, -168.0];
        let (proj, ..) = geo_framing(&akhi_lats, &akhi_lons, MAP_FRAME_TRIM_FRAC);
        assert!(
            matches!(proj, ProjectionType::AlbersUsa),
            "a US extent with Alaska/Hawaii should still use albers usa"
        );

        // a global spread -> NaturalEarth world overview, no fitted ranges
        let g_lats = [-40.0_f64, 51.5, 35.7, -33.9, 1.3, 64.1];
        let g_lons = [174.8_f64, -0.1, 139.7, 151.2, 103.8, -21.9];
        let (proj, lonaxis, lataxis) = geo_framing(&g_lats, &g_lons, MAP_FRAME_TRIM_FRAC);
        assert!(matches!(proj, ProjectionType::NaturalEarth));
        assert!(lonaxis.is_none() && lataxis.is_none());

        // a local cluster straddling the +/-180 antimeridian: the fitted longitude range would land
        // outside [-180, 180] and clamping it would crop the far-side points, so longitude is left
        // unfit (full width) while latitude is still fit.
        let am_lats = [10.0_f64, 10.5, 11.0, 11.5];
        let am_lons = [179.0_f64, 179.5, -179.5, -179.0];
        let (proj, lonaxis, lataxis) = geo_framing(&am_lats, &am_lons, MAP_FRAME_TRIM_FRAC);
        assert!(matches!(proj, ProjectionType::Mercator));
        assert!(
            lonaxis.is_none(),
            "antimeridian-wrapped longitude must not be clamped (would crop points)"
        );
        assert!(
            lataxis.is_some(),
            "latitude is still fit for a local extent"
        );
    }

    #[test]
    fn geo_framing_full_extent_avoids_albers_usa_outside_us() {
        // a US-spanning core alone picks albers-usa (a fixed US composite with no fitted axes). The
        // static-export path feeds geo_framing the FULL-extent box corners (core + outliers)
        // instead, so once a stray outside the US (here far south, lat 10) expands the box, the
        // projection must NOT stay albers-usa — otherwise the full-extent box and the stray would
        // be clipped in a non-pannable image. Corners built the same way the static-export
        // path does.
        let (min_lat, max_lat, min_lon, max_lon) = (10.0, 47.6, -122.3, -74.0);
        let flat = vec![max_lat, max_lat, min_lat, min_lat, max_lat];
        let flon = vec![min_lon, max_lon, max_lon, min_lon, min_lon];
        let (proj, ..) = geo_framing(&flat, &flon, MAP_FRAME_TRIM_FRAC);
        assert!(
            !matches!(proj, ProjectionType::AlbersUsa),
            "a full extent reaching outside the US must not use albers-usa"
        );
    }

    #[test]
    fn continent_scope_frames_single_continent() {
        // European cities (London, Paris, Berlin, Madrid, Rome) -> "europe"
        let eu_lats = [51.5_f64, 48.9, 52.5, 40.4, 41.9];
        let eu_lons = [-0.1_f64, 2.4, 13.4, -3.7, 12.5];
        assert_eq!(continent_scope(&eu_lats, &eu_lons), Some("europe"));

        // African cities (Cairo, Lagos, Nairobi, Johannesburg) -> "africa"
        let af_lats = [30.0_f64, 6.5, -1.3, -26.2];
        let af_lons = [31.2_f64, 3.4, 36.8, 28.0];
        assert_eq!(continent_scope(&af_lats, &af_lons), Some("africa"));

        // Australian/NZ cities (Sydney, Melbourne, Perth, Auckland) -> "oceania"
        let oc_lats = [-33.9_f64, -37.8, -32.0, -36.8];
        let oc_lons = [151.2_f64, 145.0, 115.9, 174.8];
        assert_eq!(continent_scope(&oc_lats, &oc_lons), Some("oceania"));
    }

    #[test]
    fn continent_scope_none_for_world_or_straddle() {
        // a world-spanning spread sits in no single continent box -> None (NaturalEarth overview)
        let g_lats = [-40.0_f64, 51.5, 35.7, -33.9, 1.3, 64.1];
        let g_lons = [174.8_f64, -0.1, 139.7, 151.2, 103.8, -21.9];
        assert_eq!(continent_scope(&g_lats, &g_lons), None);

        // London + Tokyo fall in different continent boxes, so no single box holds both -> None
        let split_lats = [51.5_f64, 35.7];
        let split_lons = [-0.1_f64, 139.7];
        assert_eq!(continent_scope(&split_lats, &split_lons), None);

        // a point in the Europe/Asia overlap (e.g. Ankara) sits in BOTH boxes -> ambiguous -> None
        let overlap_lats = [40.0_f64];
        let overlap_lons = [35.0_f64];
        assert_eq!(continent_scope(&overlap_lats, &overlap_lons), None);

        // empty / mismatched inputs -> None
        assert_eq!(continent_scope(&[], &[]), None);
        assert_eq!(continent_scope(&[10.0], &[10.0, 20.0]), None);
    }

    #[test]
    fn parse_theme_accepts_names_case_and_hyphens() {
        // all 8 canonical names resolve
        assert!(matches!(
            parse_theme("default"),
            Some(BuiltinTheme::Default)
        ));
        assert!(matches!(
            parse_theme("plotly_white"),
            Some(BuiltinTheme::PlotlyWhite)
        ));
        assert!(matches!(
            parse_theme("plotly_dark"),
            Some(BuiltinTheme::PlotlyDark)
        ));
        assert!(matches!(
            parse_theme("seaborn"),
            Some(BuiltinTheme::Seaborn)
        ));
        assert!(matches!(
            parse_theme("seaborn_whitegrid"),
            Some(BuiltinTheme::SeabornWhitegrid)
        ));
        assert!(matches!(
            parse_theme("seaborn_dark"),
            Some(BuiltinTheme::SeabornDark)
        ));
        assert!(matches!(
            parse_theme("matplotlib"),
            Some(BuiltinTheme::Matplotlib)
        ));
        assert!(matches!(
            parse_theme("plotnine"),
            Some(BuiltinTheme::Plotnine)
        ));

        // case-insensitive, hyphens accepted as separators, surrounding whitespace trimmed
        assert!(matches!(
            parse_theme("Plotly-Dark"),
            Some(BuiltinTheme::PlotlyDark)
        ));
        assert!(matches!(
            parse_theme("  SEABORN_WHITEGRID  "),
            Some(BuiltinTheme::SeabornWhitegrid)
        ));
        // short aliases
        assert!(matches!(
            parse_theme("dark"),
            Some(BuiltinTheme::PlotlyDark)
        ));
        assert!(matches!(
            parse_theme("white"),
            Some(BuiltinTheme::PlotlyWhite)
        ));

        // unknown names are rejected
        assert!(parse_theme("bogus").is_none());
        assert!(parse_theme("").is_none());
    }

    #[test]
    fn aggregate_preserves_input_order() {
        // first-seen order is preserved (not lexicographic 1, 10, 2)
        let xs = vec!["1".to_string(), "10".to_string(), "2".to_string()];
        let ys = vec![1.0, 2.0, 3.0];
        let (out_x, _out_y) = aggregate(xs, ys, Agg::Sum);
        assert_eq!(out_x, vec!["1", "10", "2"]);
    }

    #[test]
    fn sort_line_xy_orders_numeric_x() {
        let xs = vec!["1".to_string(), "10".to_string(), "2".to_string()];
        let ys = vec![1.0, 2.0, 3.0];
        let (out_x, out_y) = sort_line_xy(xs, ys);
        assert_eq!(out_x, vec!["1", "2", "10"]);
        assert_eq!(out_y, vec![1.0, 3.0, 2.0]);
    }

    #[test]
    fn sort_line_xy_nonfinite_literal_falls_back_to_categorical() {
        // "nan"/"inf" parse as f64 but are non-finite: they must NOT flip the axis numeric
        // (NaN sort keys break total ordering; inf serializes as null). Input order preserved.
        let xs = vec![
            "3".to_string(),
            "1".to_string(),
            "nan".to_string(),
            "2".to_string(),
        ];
        let ys = vec![30.0, 10.0, 99.0, 20.0];
        let (out_x, out_y) = sort_line_xy(xs.clone(), ys.clone());
        assert_eq!(out_x, xs);
        assert_eq!(out_y, ys);

        let xs = vec!["1".to_string(), "inf".to_string()];
        let (out_x, _) = sort_line_xy(xs.clone(), vec![1.0, 2.0]);
        assert_eq!(out_x, xs);
    }

    #[test]
    fn truncate_labels_unique_disambiguates_collisions() {
        // shared 15+-char prefix truncates identically -> would collapse plotly heatmap
        // categories; the helper must keep every label distinct
        let labels = vec![
            "total_precipitation_mm".to_string(),
            "total_precipitation_in".to_string(),
            "short".to_string(),
            "total_precipitation_cm".to_string(),
        ];
        let out = truncate_labels_unique(&labels, 16);
        assert_eq!(out.len(), 4);
        let distinct: std::collections::HashSet<&String> = out.iter().collect();
        assert_eq!(
            distinct.len(),
            4,
            "labels must be pairwise distinct: {out:?}"
        );
        assert_eq!(out[2], "short");
        assert!(out[0].starts_with("total_precipita"));
        // identical full names (duplicate headers) also get disambiguated
        let dupes = vec!["a".to_string(), "a".to_string()];
        let out = truncate_labels_unique(&dupes, 16);
        assert_ne!(out[0], out[1]);
    }

    #[test]
    fn sort_line_xy_preserves_categorical_order() {
        let xs = vec!["b".to_string(), "a".to_string(), "c".to_string()];
        let ys = vec![1.0, 2.0, 3.0];
        let (out_x, _) = sort_line_xy(xs, ys);
        assert_eq!(out_x, vec!["b", "a", "c"]);
    }

    #[test]
    fn cell_geometry_cells_are_within_bounds_and_titled_above() {
        // a 4-panel, 2-column Data Schematic -> 2 rows, occupying the full band (top = 1.0)
        let (rows, cols, top, offset) = (2, 2, 1.0, 0.01);
        for n in 0..4 {
            let g = cell_geometry(
                n / cols,
                n % cols,
                rows,
                cols,
                false,
                top,
                offset,
                &vec![1.0; rows],
            );
            // domains stay inside the paper area
            assert!(g.x_domain[0] >= 0.0 && g.x_domain[1] <= 1.0 + 1e-9);
            assert!(g.y_domain[0] >= -1e-9 && g.y_domain[1] <= 1.0 + 1e-9);
            // each title is horizontally centered on its cell and sits at/above the cell top
            assert!((g.title_x - (g.x_domain[0] + g.x_domain[1]) / 2.0).abs() < 1e-9);
            assert!(g.title_y >= g.y_domain[1] - 1e-9);
        }
        // top row is higher on the page than the bottom row
        let upper = cell_geometry(0, 0, rows, cols, false, top, offset, &vec![1.0; rows]);
        let lower = cell_geometry(1, 0, rows, cols, false, top, offset, &vec![1.0; rows]);
        assert!(upper.y_domain[0] > lower.y_domain[1]);
        // the two columns don't overlap horizontally
        let left = cell_geometry(0, 0, rows, cols, false, top, offset, &vec![1.0; rows]);
        let right = cell_geometry(0, 1, rows, cols, false, top, offset, &vec![1.0; rows]);
        assert!(left.x_domain[1] <= right.x_domain[0] + 1e-9);
        // a full-width cell spans the whole page regardless of column
        let full = cell_geometry(0, 0, rows, cols, true, top, offset, &vec![1.0; rows]);
        assert_eq!(full.x_domain, vec![0.0, 1.0]);
        assert!((full.title_x - 0.5).abs() < 1e-9);
    }

    #[test]
    fn smart_title_band_fits_every_row_count() {
        // For every Data Schematic size up to the 8-panel max (including a tall single-column
        // 8-row layout), the top-row panel title — offset above the cell plus its rendered
        // glyph height — must stay at/below y=1 so it never overlaps the Data Schematic title.
        const GLYPH_PX: f64 = 20.0; // generous estimate of the 13px title's rendered height
        for rows in 1..=MAX_SUBPLOTS {
            let area = smart_plot_area_h(rows as f64);
            let g = cell_geometry(
                0,
                0,
                rows,
                1,
                false,
                smart_grid_top(rows as f64, false),
                smart_title_offset(rows as f64),
                &vec![1.0; rows],
            );
            let title_top = g.title_y + GLYPH_PX / area;
            assert!(
                title_top <= 1.0 + 1e-9,
                "rows={rows}: title_top={title_top} crosses y=1 (would overlap Data Schematic \
                 title)"
            );
            // the title still sits above its own cell
            assert!(g.title_y >= g.y_domain[1] - 1e-9);
        }

        // the reserved band is a real pixel size even for a short one-row Data Schematic
        let band_px = (1.0 - smart_grid_top(1.0, false)) * smart_plot_area_h(1.0);
        assert!(band_px >= 30.0, "one-row title band too thin: {band_px}px");
    }

    #[test]
    fn cell_geometry_stays_positive_for_many_rows() {
        // Regression: a fixed per-gap paper fraction (VGAP=0.09) overflowed the 0..=top band once
        // there were ~6+ rows, yielding negative cell heights and a garbled Data Schematic. Now
        // that static image export draws >MAX_SUBPLOTS panels, verify every cell stays
        // positive and in-bounds for tall grids (e.g. a 42-panel, 21-row Data Schematic),
        // with rows not overlapping.
        for panels in [9usize, 16, 30, 42, 64] {
            let cols = 2;
            let rows = panels.div_ceil(cols);
            let top = smart_grid_top(rows as f64, false);
            let offset = smart_title_offset(rows as f64);
            for n in 0..panels {
                let g = cell_geometry(
                    n / cols,
                    n % cols,
                    rows,
                    cols,
                    false,
                    top,
                    offset,
                    &vec![1.0; rows],
                );
                assert!(
                    g.y_domain[1] > g.y_domain[0],
                    "panels={panels} n={n}: non-positive cell height {:?}",
                    g.y_domain
                );
                assert!(
                    g.y_domain[0] >= -1e-9 && g.y_domain[1] <= 1.0 + 1e-9,
                    "panels={panels} n={n}: y-domain out of bounds {:?}",
                    g.y_domain
                );
                assert!(
                    g.x_domain[1] > g.x_domain[0] && g.x_domain[1] <= 1.0 + 1e-9,
                    "panels={panels} n={n}: bad x-domain {:?}",
                    g.x_domain
                );
                // each cell sits strictly below the one directly above it (n - cols)
                if n >= cols {
                    let m = n - cols;
                    let above = cell_geometry(
                        m / cols,
                        m % cols,
                        rows,
                        cols,
                        false,
                        top,
                        offset,
                        &vec![1.0; rows],
                    );
                    assert!(
                        g.y_domain[1] <= above.y_domain[0] + 1e-9,
                        "panels={panels} n={n}: row overlaps the one above"
                    );
                }
            }
        }
    }

    #[test]
    fn smart_grid_layout_gives_overview_panels_full_width_rows() {
        // one overview panel (correlation heatmap) followed by three distribution bars, 2 columns:
        // the heatmap takes a full-width row, the three bars pack into 2 rows below (2 + 1).
        let panels = vec![
            Panel::new(
                "corr".to_string(),
                PanelKind::CorrHeatmap {
                    labels:   vec!["a".to_string(), "b".to_string()],
                    matrix:   vec![vec![1.0, 0.5], vec![0.5, 1.0]],
                    spearman: false,
                },
            ),
            Panel::new("c1".to_string(), PanelKind::FreqBar { idx: 0 }),
            Panel::new("c2".to_string(), PanelKind::FreqBar { idx: 1 }),
            Panel::new("c3".to_string(), PanelKind::FreqBar { idx: 2 }),
        ];
        let (geoms, rows, _height) = smart_grid_layout(&panels, 2);
        // 1 full-width overview row + ceil(3/2) = 2 grid rows
        assert_eq!(rows, 3);
        // the overview panel spans the full page width and leads the Data Schematic (top band)
        assert_eq!(geoms[0].x_domain, vec![0.0, 1.0]);
        // the distribution bars are NOT full width (two columns)
        assert!(geoms[1].x_domain[1] < 1.0 - 1e-9);
        assert!(geoms[2].x_domain[0] > 1e-9);
        // the first bar row sits below the overview row
        assert!(geoms[1].y_domain[1] <= geoms[0].y_domain[0] + 1e-9);
        // the two bars on the same row share a y-band but occupy different columns
        assert_eq!(geoms[1].y_domain, geoms[2].y_domain);
        assert!(geoms[1].x_domain[1] <= geoms[2].x_domain[0] + 1e-9);
        // the third bar wraps to its own (last) row, below the first bar row
        assert!(geoms[3].y_domain[1] <= geoms[1].y_domain[0] + 1e-9);
    }

    #[test]
    fn panel_row_scale_is_content_aware() {
        let tile = |gauge: Option<[f64; 2]>| KpiTile {
            label: "m".to_string(),
            value: 1.0,
            format: ",".to_string(),
            gauge,
            target: None,
        };
        let plain = Panel::new(
            "kpi".to_string(),
            PanelKind::KpiRow {
                tiles: vec![tile(None), tile(None)],
            },
        );
        let gauged = Panel::new(
            "kpi".to_string(),
            PanelKind::KpiRow {
                tiles: vec![tile(None), tile(Some([0.0, 1.0]))],
            },
        );

        let plain_scale = panel_row_scale(&plain.kind, (false, false));
        let gauge_scale = panel_row_scale(&gauged.kind, (false, false));

        // the scale is the content-matched inline height as a fraction of a normal row
        assert!((plain_scale - KPI_ROW_HEIGHT_PX as f64 / ROW_HEIGHT_PX as f64).abs() < 1e-9);
        assert!((gauge_scale - KPI_ROW_GAUGE_HEIGHT_PX as f64 / ROW_HEIGHT_PX as f64).abs() < 1e-9);
        // a gauge row is taller than a pure-number row (its number sits lower), and both are
        // shorter than a full chart row
        assert!(gauge_scale > plain_scale);
        assert!(gauge_scale < 1.0 && plain_scale < 1.0);
        // a non-KPI panel keeps a full-height row (uniform grid unchanged)
        let bar = Panel::new("c".to_string(), PanelKind::FreqBar { idx: 0 });
        assert!((panel_row_scale(&bar.kind, (false, false)) - 1.0).abs() < 1e-9);

        // the gauge KPI first-row band is genuinely taller in the assembled grid geometry
        let with_kpi = |lead: Panel| {
            vec![
                lead,
                Panel::new("c1".to_string(), PanelKind::FreqBar { idx: 0 }),
                Panel::new("c2".to_string(), PanelKind::FreqBar { idx: 1 }),
            ]
        };
        let (pg, _pr, ph) = smart_grid_layout(&with_kpi(plain), 2);
        let (gg, _gr, gh) = smart_grid_layout(&with_kpi(gauged), 2);
        let band = |g: &SubplotGeometry| g.y_domain[1] - g.y_domain[0];
        assert!(band(&gg[0]) > band(&pg[0]));
        // and the taller KPI row makes the whole figure taller
        assert!(gh > ph);
    }

    #[test]
    fn map_render_heights_are_taller_and_reserve_a_slider_band() {
        // the inline map usable draw height is the fixed map row minus the plotly top+bottom
        // margins (48 + 20); the animated map reserves an EXTRA control band so the slider
        // sits below the map
        assert!((MAP_PANEL_USABLE_HEIGHT_PX - (MAP_ROW_HEIGHT_PX as f64 - 68.0)).abs() < 1e-9);

        // a static geographic panel renders at the tall map row height
        let geo = PanelKind::Geo {
            lats:               vec![0.0, 1.0],
            lons:               vec![0.0, 1.0],
            outlier_lats:       vec![],
            outlier_lons:       vec![],
            hover_text:         vec![String::new(), String::new()],
            outlier_hover_text: vec![],
            sizes:              None,
            row_ids:            vec![],
            outlier_row_ids:    vec![],
        };
        // an animated map adds the control-band allowance on top of the map row
        let anim = PanelKind::AnimatedGeo {
            bucket_lats:   vec![vec![0.0], vec![1.0]],
            bucket_lons:   vec![vec![0.0], vec![1.0]],
            bucket_labels: vec!["a".to_string(), "b".to_string()],
            frame_label:   "t".to_string(),
        };
        assert_eq!(panel_render_height(&geo, (false, false)), MAP_ROW_HEIGHT_PX);
        assert_eq!(
            panel_render_height(&anim, (false, false)),
            MAP_ROW_HEIGHT_PX + SLIDER_CONTROL_ALLOWANCE_PX
        );
        assert!(
            panel_render_height(&anim, (false, false)) > panel_render_height(&geo, (false, false))
        );
    }

    #[test]
    fn parcats_height_grows_with_category_count_and_clamps() {
        // one dimension carries `n` distinct categories; the other is constant → max_cat == n
        let parcats = |n: usize| PanelKind::Parcats {
            dim_labels: vec!["d0".to_string(), "d1".to_string()],
            tuples:     (0..n)
                .map(|i| vec![format!("c{i}"), "x".to_string()])
                .collect(),
            counts:     vec![1.0; n],
        };
        let small = panel_render_height(&parcats(3), (false, false)); // 3*44=132 -> floored
        let mid = panel_render_height(&parcats(15), (false, false)); // 15*44=660 -> between
        let big = panel_render_height(&parcats(40), (false, false)); // 40*44=1760 -> ceiled

        assert_eq!(small, PARCATS_HEIGHT_MIN_PX);
        assert_eq!(mid, 15 * PARCATS_PX_PER_CATEGORY);
        assert_eq!(big, PARCATS_HEIGHT_MAX_PX);
        // monotonic non-decreasing in category count
        assert!(small <= mid && mid <= big);
        // the mid case is genuinely interior to the clamp band
        assert!(mid > PARCATS_HEIGHT_MIN_PX && mid < PARCATS_HEIGHT_MAX_PX);
    }

    #[test]
    fn sankey_height_grows_with_node_count_and_clamps() {
        // `n` distinct source nodes all flowing to a single target → tallest column == n
        let sankey = |n: usize| PanelKind::Sankey {
            node_labels: (0..=n).map(|i| format!("n{i}")).collect(),
            link_source: (0..n).collect(),
            link_target: vec![n; n],
            link_value:  vec![1.0; n],
            value_order: false,
        };
        let small = panel_render_height(&sankey(3), (false, false)); // 3*40=120 -> floored
        let mid = panel_render_height(&sankey(18), (false, false)); // 18*40=720 -> between
        let big = panel_render_height(&sankey(40), (false, false)); // 40*40=1600 -> ceiled

        assert_eq!(small, SANKEY_HEIGHT_MIN_PX);
        assert_eq!(mid, 18 * SANKEY_PX_PER_NODE);
        assert_eq!(big, SANKEY_HEIGHT_MAX_PX);
        assert!(small <= mid && mid <= big);
        assert!(mid > SANKEY_HEIGHT_MIN_PX && mid < SANKEY_HEIGHT_MAX_PX);
    }

    #[test]
    fn scatter_fill_classifies_zero_pile_and_well_spread() {
        // a well-spread cloud filling its bounding box: many distinct cells, no edge pileup
        let xs: Vec<f64> = (0..100).map(|i| i as f64).collect();
        let ys: Vec<f64> = (0..100).map(|i| ((i * 37) % 100) as f64).collect();
        let spread = scatter_fill_stats(&xs, &ys);
        assert!(!spread.degenerate && !spread.sparse);
        let spread_panel = PanelKind::ScatterPair {
            xs:         xs.clone(),
            ys:         ys.clone(),
            sizes:      None,
            x_label:    "x".to_string(),
            y_label:    "y".to_string(),
            size_label: None,
        };
        // a well-filled ScatterPair keeps its full overview-panel height
        assert_eq!(
            panel_render_height(&spread_panel, (false, false)),
            OVERVIEW_ROW_HEIGHT_PX
        );
        // and a sparse one is shorter than a well-filled one
        assert!(SCATTER_SPARSE_HEIGHT_PX < OVERVIEW_ROW_HEIGHT_PX);

        // sparse-but-not-degenerate: most points pile against the low x/y edge, a few spread out
        let mut sx = vec![0.0_f64; 75];
        let mut sy = vec![0.0_f64; 75];
        for i in 0..25 {
            sx.push(i as f64 * 4.0);
            sy.push(i as f64 * 4.0);
        }
        let sparse = scatter_fill_stats(&sx, &sy);
        assert!(sparse.sparse && !sparse.degenerate);
        let sparse_panel = PanelKind::ScatterPair {
            xs:         sx,
            ys:         sy,
            sizes:      None,
            x_label:    "x".to_string(),
            y_label:    "y".to_string(),
            size_label: None,
        };
        assert_eq!(
            panel_render_height(&sparse_panel, (false, false)),
            SCATTER_SPARSE_HEIGHT_PX
        );

        // A heavy-tailed but strictly positive cloud reads as sparse on LINEAR axes -- the bulk
        // piles against the origin under a far outlier -- yet spreads cleanly once logged. Such a
        // panel is rescued from the build-time drop by its log axes (issue #4276), so its HEIGHT
        // must be judged in the same space: sizing it off the raw values would hand a
        // well-filled cloud the compact height meant for a genuinely sparse one (roborev 3821).
        let mut hx: Vec<f64> = Vec::new();
        let mut hy: Vec<f64> = Vec::new();
        for i in 0..99 {
            let v = 1.0 + f64::from(i % 20);
            hx.push(v);
            hy.push(v * 3.0);
        }
        hx.push(1.0e6);
        hy.push(3.0e6);
        let raw = scatter_fill_stats(&hx, &hy);
        assert!(
            raw.sparse,
            "the linear view of a heavy tail should read as sparse"
        );
        let plotted = scatter_fill_plotted(&hx, &hy, (true, true));
        assert!(!plotted.sparse, "logging it should spread the cloud out");
        let heavy_panel = PanelKind::ScatterPair {
            xs:         hx,
            ys:         hy,
            sizes:      None,
            x_label:    "x".to_string(),
            y_label:    "y".to_string(),
            size_label: None,
        };
        assert_eq!(
            panel_render_height(&heavy_panel, (true, true)),
            OVERVIEW_ROW_HEIGHT_PX,
            "a log-rescued scatter must get the full height, not the sparse one"
        );
        // ... while the same panel drawn on linear axes still gets the compact height
        assert_eq!(
            panel_render_height(&heavy_panel, (false, false)),
            SCATTER_SPARSE_HEIGHT_PX
        );

        // degenerate: a flat line (no y-spread) collapses to 1-D → dropped at build time
        let flat_x: Vec<f64> = (0..50).map(|i| i as f64).collect();
        let flat_y = vec![7.0_f64; 50];
        let degen = scatter_fill_stats(&flat_x, &flat_y);
        assert!(degen.degenerate && degen.sparse);

        // exactly 3 non-collinear points with real 2-D spread are NOT degenerate (the early cutoff
        // is n < 3, matching the "3 distinct cells" degenerate threshold — see roborev 3691); fewer
        // than 3 points can't form a cloud and stay degenerate.
        assert!(!scatter_fill_stats(&[0.0, 10.0, 5.0], &[0.0, 3.0, 10.0]).degenerate);
        assert!(scatter_fill_stats(&[0.0, 10.0], &[0.0, 10.0]).degenerate);
    }

    #[test]
    fn density_concentration_flags_one_cell_collapse() {
        // a uniform 2x2 grid: each of the 4 cells holds a quarter of the mass -> 0.25
        let uniform = vec![vec![1.0, 1.0], vec![1.0, 1.0]];
        assert!((density_concentration(&uniform) - 0.25).abs() < 1e-9);

        // all mass in one cell (the "single dark cell at the origin" symptom) -> 1.0
        let collapsed = vec![vec![99.0, 0.0], vec![0.0, 0.0]];
        assert!((density_concentration(&collapsed) - 1.0).abs() < 1e-9);
        assert!(density_concentration(&collapsed) > SMART_CONTOUR_MAX_BIN_SHARE);

        // a well-spread grid stays under the gate; NaN cells (empty bins) are ignored, not summed
        let spread = vec![vec![10.0, 12.0, f64::NAN], vec![11.0, 9.0, 13.0]];
        assert!(density_concentration(&spread) < SMART_CONTOUR_MAX_BIN_SHARE);

        // an empty grid reads as "not concentrated" (0.0), never tripping the gate on its own
        assert_eq!(density_concentration(&[vec![0.0, 0.0]]), 0.0);
    }

    #[test]
    fn positive_subset_pair_keeps_substantial_positive_mass() {
        // 1000 rows, all strictly positive on both axes: nothing dropped
        let xs: Vec<f64> = (1..=1000).map(|i| i as f64).collect();
        let ys: Vec<f64> = (1..=1000).map(|i| (i * 2) as f64).collect();
        let (px, py, dropped) = positive_subset_pair(&xs, &ys).expect("all-positive kept");
        assert_eq!(px.len(), 1000);
        assert_eq!(py.len(), 1000);
        assert!(dropped.abs() < 1e-9);

        // ~50% zeros but 1000 survivors and 50% share: kept, with the dropped share reported
        let mut zx = vec![0.0_f64; 1000];
        let mut zy = vec![0.0_f64; 1000];
        for i in 1..=1000 {
            zx.push(i as f64);
            zy.push(i as f64);
        }
        let (_, _, dropped) = positive_subset_pair(&zx, &zy).expect("half positive kept");
        assert!((dropped - 0.5).abs() < 1e-9);

        // too few survivors (below SMART_CONTOUR_LOG_MIN_POINTS): declined even though the share
        // is high
        let few_x: Vec<f64> = (1..=100).map(|i| i as f64).collect();
        let few_y = few_x.clone();
        assert!(positive_subset_pair(&few_x, &few_y).is_none());

        // enough absolute survivors but a tiny share (600 positives in 100k rows): declined, so a
        // rounding-error sliver is never dressed up as the distribution
        let mut sparse_x = vec![0.0_f64; 100_000];
        let mut sparse_y = vec![0.0_f64; 100_000];
        for i in 1..=600 {
            sparse_x.push(i as f64);
            sparse_y.push(i as f64);
        }
        assert!(positive_subset_pair(&sparse_x, &sparse_y).is_none());
    }

    #[test]
    fn relationship_axis_log_engages_only_on_wide_positive_range() {
        // Auto: a wide positive dynamic range (>= LOG_SCALE_MIN_RATIO) engages log
        let wide: Vec<f64> = vec![1.0, 10.0, 100.0, 1000.0];
        assert!(relationship_axis_log(LogScale::Auto, &wide));

        // Auto: a modest range stays linear (no spurious log on well-behaved data)
        let modest: Vec<f64> = vec![10.0, 12.0, 15.0, 18.0];
        assert!(!relationship_axis_log(LogScale::Auto, &modest));

        // a single zero (zero-inflated column) rules out log outright, even with a wide range
        let zeroed: Vec<f64> = vec![0.0, 1.0, 100.0, 1000.0];
        assert!(!relationship_axis_log(LogScale::Auto, &zeroed));

        // On forces log for any 2+ strictly-positive values regardless of range;
        // Off never logs
        assert!(relationship_axis_log(LogScale::On, &modest));
        assert!(!relationship_axis_log(LogScale::Off, &wide));
    }

    #[test]
    fn bin_2d_log_axes_use_geometric_centers() {
        // three decades of positive data, log-binned on both axes. The returned bin CENTERS are
        // raw values (a plotly log axis consumes raw coords), so they must grow multiplicatively
        // across the axis rather than in equal linear steps.
        let xs: Vec<f64> = vec![1.0, 10.0, 100.0, 1000.0];
        let ys = xs.clone();
        let (xc, _, z) = bin_2d(&xs, &ys, 10, (true, true));
        // every point counted exactly once (4 points -> total mass 4)
        let total: f64 = z.iter().flatten().sum();
        assert!((total - 4.0).abs() < 1e-9);
        // consecutive centers grow by a roughly constant RATIO (geometric), not a constant delta
        let ratios: Vec<f64> = xc.windows(2).map(|w| w[1] / w[0]).collect();
        let first = ratios[0];
        assert!(ratios.iter().all(|r| (r - first).abs() < 1e-6));
        // and the linear deltas are NOT constant (proving the spacing isn't linear)
        let d0 = xc[1] - xc[0];
        let dn = xc[xc.len() - 1] - xc[xc.len() - 2];
        assert!(dn > d0 * 2.0);

        // linear binning of the same data keeps constant linear DELTAS (regression: viz contour
        // must be unaffected by the new parameter)
        let (xc_lin, _, _) = bin_2d(&xs, &ys, 10, (false, false));
        let deltas: Vec<f64> = xc_lin.windows(2).map(|w| w[1] - w[0]).collect();
        let d_first = deltas[0];
        assert!(deltas.iter().all(|d| (d - d_first).abs() < 1e-6));
    }

    #[test]
    fn smart_contour_panel_honors_log_scale_mode() {
        // the log-space retry's title is localized -- pin English, or this races the
        // Spanish-setting tests on the process-global locale. The "a vs b" passed in
        // below is arbitrary fixture data, not observed output; the assertions on
        // `.name` are what need the pin.
        let _locale = english_locale();
        // a collapsed linear pair: 1200 zeros pile into one bin (0.6 > gate), plus 800 strictly-
        // positive points spread across three decades so the log-space grid is legible.
        let mut xs = vec![0.0_f64; 1200];
        let mut ys = vec![0.0_f64; 1200];
        for k in 0..800 {
            let v = 10_f64.powf((k as f64 / 799.0) * 5.0); // 1 .. 1e5, geometric
            xs.push(v);
            ys.push(v);
        }

        // Auto (default): linear collapses -> log retry over the positive subset. Both axes log,
        // and the omitted-zeros share is stated in the title.
        let auto = smart_contour_panel(&xs, &ys, "a vs b", LogScale::Auto, ("a", "b"))
            .expect("auto retries collapsed linear density in log space");
        assert_eq!(auto.axis_log, (true, true));
        assert!(
            auto.name.contains("omitted"),
            "auto title states dropped share: {}",
            auto.name
        );

        // Off: the user disabled log, so a collapsed linear grid has no legible fallback -> no
        // panel, and NEVER a log axis (regression guard for the review finding).
        assert!(
            smart_contour_panel(&xs, &ys, "a vs b", LogScale::Off, ("a", "b")).is_none(),
            "--log-scale off must not emit a log contour"
        );

        // On: forces the log view even though the linear grid here is collapsed anyway.
        let on = smart_contour_panel(&xs, &ys, "a vs b", LogScale::On, ("a", "b"))
            .expect("on forces the log contour");
        assert_eq!(on.axis_log, (true, true));

        // An ALL-positive pair that fills its box on BOTH scales (a uniform 30x30 grid): neither
        // the linear nor the log grid collapses. `On` still forces log (nothing dropped -> plain
        // "log scale" cue, no "omitted"); `Off` leaves it linear.
        let (mut gx, mut gy) = (Vec::new(), Vec::new());
        for a in 1..=30 {
            for b in 1..=30 {
                gx.push(a as f64);
                gy.push(b as f64);
            }
        }
        let on_pos = smart_contour_panel(&gx, &gy, "a vs b", LogScale::On, ("a", "b"))
            .expect("on forces log");
        assert_eq!(on_pos.axis_log, (true, true));
        assert!(on_pos.name.contains("log scale") && !on_pos.name.contains("omitted"));
        let off_pos = smart_contour_panel(&gx, &gy, "a vs b", LogScale::Off, ("a", "b"))
            .expect("off keeps the legible linear contour");
        assert_eq!(off_pos.axis_log, (false, false));
        assert!(!off_pos.name.contains("log"));

        // A tiny non-positive share must still be DISCLOSED, not hidden behind the bare "log
        // scale" cue: 5 zeros in 5005 rows rounds to 0% but reads as "<1% of rows omitted".
        let mut tx: Vec<f64> = (0..5000)
            .map(|k| 10_f64.powf((k as f64 / 4999.0) * 5.0))
            .collect();
        let mut ty = tx.clone();
        for _ in 0..5 {
            tx.push(0.0);
            ty.push(0.0);
        }
        let tiny = smart_contour_panel(&tx, &ty, "a vs b", LogScale::On, ("a", "b"))
            .expect("on forces log over the positive subset");
        assert!(
            tiny.name.contains("<1% of rows omitted"),
            "sub-1% omission is disclosed, not hidden: {}",
            tiny.name
        );
    }

    #[test]
    fn measure_by_dim_forces_category_x_axis() {
        // MeasureByDim x-labels can look numeric (magnitude "5.3", "6") while also including a
        // non-numeric label ("6 SR") that carries the meaningful value. freq_bar_tick_text must
        // return the labels so styled_x_axis pins a CATEGORY axis; otherwise plotly infers a
        // linear x-axis and drops the non-numeric label's bar. Regression guard for the
        // "killed by magnitude looked empty" bug (the 550 bar was rendered at zero width).
        let panel = Panel::new(
            "killed by magnitude".to_string(),
            PanelKind::MeasureByDim {
                labels: vec!["6 SR".to_string(), "5.3".to_string(), "(NULL)".to_string()],
                values: vec![550.0, 0.0, 2.0],
            },
        );
        let freq: FreqMap = std::collections::HashMap::new();
        let ticks =
            freq_bar_tick_text(&panel, &freq).expect("MeasureByDim must supply category ticks");
        assert_eq!(ticks, vec!["6 SR", "5.3", "(NULL)"]);

        // a non-bar panel still leaves its axis untouched (returns None → autodetected axis)
        let other = Panel::new(
            "c".to_string(),
            PanelKind::CyclicProfile {
                theta: vec![],
                r:     vec![],
            },
        );
        assert!(freq_bar_tick_text(&other, &freq).is_none());
    }

    #[test]
    fn axis_layout_keys_align_with_trace_refs_beyond_eight() {
        // render_smart_grid_json lets static export exceed the typed Layout's 8-axis limit only if
        // each cell's injected layout key (xaxis{n}) matches the axis its trace references (x{n}).
        // Verify the two naming schemes agree for the first dozen positions — including the pos-1
        // special case ("xaxis"/"x", not "xaxis1"/"x1") — so panels 9+ aren't silently orphaned.
        for (prefix, axis_word) in [('x', "xaxis"), ('y', "yaxis")] {
            assert_eq!(axis_json_key(prefix, 1), axis_word);
            assert_eq!(axis_ref(prefix, 1), prefix.to_string());
            for pos in 2..=12usize {
                let key = axis_json_key(prefix, pos);
                let reference = axis_ref(prefix, pos);
                assert_eq!(key, format!("{axis_word}{pos}"));
                assert_eq!(reference, format!("{prefix}{pos}"));
                // a layout key like "xaxis9" is referenced by its trace as "x9"
                assert_eq!(reference.trim_start_matches(prefix), pos.to_string());
            }
        }

        // a domain-positioned, cross-anchored axis serializes to the JSON shape plotly.js
        // expects under the injected key: a 2-element `domain` array and an `anchor` string.
        let axis = styled_y_axis(None, false, None)
            .domain(&[0.0, 0.5])
            .anchor("x9");
        let v = serde_json::to_value(&axis).unwrap();
        assert_eq!(
            v.get("domain").and_then(|d| d.as_array()).map(Vec::len),
            Some(2)
        );
        assert_eq!(
            v.get("anchor").and_then(serde_json::Value::as_str),
            Some("x9")
        );
    }

    #[test]
    fn parse_map_style_resolves_supported_styles() {
        // every supported MapLibre style resolves (all token-free); MapStyle has no PartialEq,
        // so compare its kebab-case serde value.
        for (name, expected) in [
            ("open-street-map", "open-street-map"),
            ("osm", "open-street-map"),
            ("carto-positron", "carto-positron"),
            ("carto-positron-nolabels", "carto-positron-nolabels"),
            ("carto-darkmatter", "carto-darkmatter"),
            ("carto-dark-matter", "carto-darkmatter"),
            ("carto-darkmatter-nolabels", "carto-darkmatter-nolabels"),
            ("carto-dark-matter-nolabels", "carto-darkmatter-nolabels"),
            ("carto-voyager", "carto-voyager"),
            ("carto-voyager-nolabels", "carto-voyager-nolabels"),
            ("white-bg", "white-bg"),
            ("basic", "basic"),
            ("streets", "streets"),
            ("outdoors", "outdoors"),
            ("light", "light"),
            ("dark", "dark"),
            ("satellite", "satellite"),
            ("satellite-streets", "satellite-streets"),
        ] {
            let style = parse_map_style(name).unwrap();
            assert_eq!(
                serde_json::to_value(style).unwrap(),
                serde_json::json!(expected),
                "{name} should resolve to {expected}"
            );
        }
        // matching is case-insensitive
        assert!(parse_map_style("Carto-Positron").is_ok());
    }

    #[test]
    fn parse_map_style_dropped_stamen_errors() {
        // the Stamen basemaps have no MapLibre equivalent and were dropped
        for name in ["stamen-terrain", "stamen-toner", "stamen-watercolor"] {
            assert!(
                parse_map_style(name).is_err(),
                "{name} should be unsupported"
            );
        }
    }

    #[test]
    fn parse_map_style_unknown_errors() {
        assert!(parse_map_style("not-a-style").is_err());
    }

    #[test]
    fn parse_choropleth_map_style_resolves_nolabels() {
        // the choropleth --map style parser also accepts the label-free carto variants
        for (name, expected) in [
            ("carto-positron-nolabels", "carto-positron-nolabels"),
            ("carto-darkmatter-nolabels", "carto-darkmatter-nolabels"),
            ("carto-voyager-nolabels", "carto-voyager-nolabels"),
        ] {
            let style = parse_choropleth_map_style(name).unwrap();
            assert_eq!(
                serde_json::to_value(style).unwrap(),
                serde_json::json!(expected),
                "{name} should resolve to {expected}"
            );
        }
    }

    #[test]
    fn smart_map_basemap_picks_nolabels_for_overlays() {
        // `data_overlay` picks the label-free carto variant (used by the density map and the
        // choropleth's label-OFF toggle state); the labeled variant is the default everywhere else
        let cases = [
            (false, false, "carto-positron"),
            (false, true, "carto-positron-nolabels"),
            (true, false, "carto-darkmatter"),
            (true, true, "carto-darkmatter-nolabels"),
        ];
        for (is_dark, overlay, expected) in cases {
            assert_eq!(
                serde_json::to_value(smart_map_basemap(is_dark, overlay)).unwrap(),
                serde_json::json!(expected),
                "is_dark={is_dark} overlay={overlay}"
            );
        }
    }

    #[test]
    fn map_pan_bounds_pads_and_clamps() {
        // a tight NYC-ish cluster yields a padded box (>= the data extent, still within valid
        // lat/lon) that includes an outlier passed in a second coordinate group
        let bounds = map_pan_bounds(&[
            (
                [40.70_f64, 40.80].as_slice(),
                [-74.02_f64, -73.95].as_slice(),
            ),
            ([41.50_f64].as_slice(), [-73.50_f64].as_slice()),
        ])
        .expect("a well-formed local extent yields bounds");
        let v = serde_json::to_value(&bounds).unwrap();
        // south below the min lat, north above the max lat (outlier included)
        assert!(v["south"].as_f64().unwrap() < 40.70);
        assert!(v["north"].as_f64().unwrap() > 41.50);
        assert!(v["west"].as_f64().unwrap() < -74.02);
        assert!(v["east"].as_f64().unwrap() > -73.50);
        // clamped to valid ranges
        assert!(v["south"].as_f64().unwrap() >= -90.0);
        assert!(v["north"].as_f64().unwrap() <= 90.0);
    }

    #[test]
    fn map_pan_bounds_skips_global_and_empty() {
        // near-global longitude span (or antimeridian-crossing) -> no constraint
        assert!(
            map_pan_bounds(&[([10.0_f64, -10.0].as_slice(), [-170.0_f64, 170.0].as_slice())])
                .is_none()
        );
        // empty / all-nonfinite clouds -> no constraint
        assert!(map_pan_bounds(&[([].as_slice(), [].as_slice())]).is_none());
        assert!(map_pan_bounds(&[([f64::NAN].as_slice(), [f64::NAN].as_slice())]).is_none());
    }

    #[test]
    fn map_center_zoom_centers_on_bounding_box() {
        // a tight cluster around (40, -75) centers there and zooms in
        let lats = [39.9, 40.1, 40.0];
        let lons = [-75.1, -74.9, -75.0];
        let (center, zoom) = map_center_zoom(&lats, &lons, 0.0, 1000.0, 600.0);
        let v = serde_json::to_value(&center).unwrap();
        // the centre latitude is the MERCATOR midpoint, not the arithmetic one: the zoom is
        // fitted in Mercator y, so the centre has to be taken in the same space or the box is not
        // actually centred in the viewport (visible as a clipped poleward edge on tall extents).
        // Over a 0.2-degree cluster the two agree to ~1e-5, hence the loosened tolerance here plus
        // the exact assertion below.
        assert!((v["lat"].as_f64().unwrap() - 40.0).abs() < 1e-3);
        assert!((v["lat"].as_f64().unwrap() - mercator_lat_center(39.9, 40.1)).abs() < 1e-12);
        // longitude is linear in Mercator, so it stays the exact arithmetic midpoint
        assert!((v["lon"].as_f64().unwrap() - (-75.0)).abs() < 1e-9);
        // small span -> high zoom; large span -> low zoom. Use a genuinely wide (non-wrapping)
        // longitude range so this exercises the plain bounding-box path, not antimeridian wrap.
        let (_, world_zoom) = map_center_zoom(&[-60.0, 70.0], &[-80.0, 80.0], 0.0, 1000.0, 600.0);
        assert!(
            zoom > world_zoom,
            "tight cluster should zoom in more than world"
        );
    }

    #[test]
    fn map_center_zoom_single_point() {
        // a zero-span (single point) dataset gets a sensible non-extreme zoom
        let (_, zoom) = map_center_zoom(&[51.5], &[-0.12], 0.0, 1000.0, 600.0);
        assert!((1..=16).contains(&zoom));
    }

    #[test]
    fn map_center_zoom_handles_antimeridian() {
        // a tight cluster straddling the 180° line (179 and -179) is ~2° wide, not ~358°, so it
        // centers near the date line and zooms in rather than framing the whole globe at lon 0.
        let lats = [18.0, 16.0];
        let lons = [179.0, -179.0];
        let (center, zoom) = map_center_zoom(&lats, &lons, 0.0, 1000.0, 600.0);
        let lon = serde_json::to_value(&center).unwrap()["lon"]
            .as_f64()
            .unwrap();
        assert!(
            lon.abs() > 170.0,
            "center lon should be near 180, got {lon}"
        );
        assert!(
            zoom >= 5,
            "tight antimeridian cluster should zoom in, got {zoom}"
        );
    }

    #[test]
    fn lon_center_and_span_non_wrapping() {
        // ordinary western-hemisphere cluster: plain midpoint + span, no wrap
        let (center, span) = lon_center_and_span(&[-75.1, -74.9, -75.0], 0.0);
        assert!((center - (-75.0)).abs() < 1e-9);
        assert!((span - 0.2).abs() < 1e-9);
    }

    #[test]
    fn lon_center_and_span_empty_is_degenerate() {
        // empty input must not panic on the sorted[0]/sorted[n - 1] indexing -> degenerate (0, 0)
        assert_eq!(lon_center_and_span(&[], 0.0), (0.0, 0.0));
    }

    #[test]
    fn lon_center_and_span_crossing_trims_outlier() {
        // a tight cluster straddling +/-180 plus one far in-range longitude outlier (80). The
        // crossing branch must trim like the non-crossing one: untrimmed, the unwrapped span runs
        // from the outlier (~80) to the cluster (~182) and reads as global; trimming the outlier
        // away keeps the span local.
        let mut lons = vec![80.0_f64]; // single far outlier
        for k in 0..100 {
            lons.push(178.0 + (k % 2) as f64); // 178 / 179
            lons.push(-179.0 + (k % 2) as f64); // -179 / -178
        }
        let (_, untrimmed) = lon_center_and_span(&lons, 0.0);
        let (_, trimmed) = lon_center_and_span(&lons, MAP_FRAME_TRIM_FRAC);
        assert!(
            untrimmed >= 90.0,
            "untrimmed crossing span should include the outlier, got {untrimmed}"
        );
        assert!(
            trimmed < 10.0,
            "trimmed crossing span should exclude the outlier, got {trimmed}"
        );
    }

    #[test]
    fn sorted_quantile_nearest_rank() {
        let s: Vec<f64> = (0..=100).map(f64::from).collect(); // 0..=100
        assert_eq!(sorted_quantile(&s, 0.0), 0.0);
        assert_eq!(sorted_quantile(&s, 1.0), 100.0);
        assert_eq!(sorted_quantile(&s, 0.5), 50.0);
        assert_eq!(sorted_quantile(&s, 0.01), 1.0);
        assert_eq!(sorted_quantile(&s, 0.99), 99.0);
        assert_eq!(sorted_quantile(&[], 0.5), 0.0); // empty -> 0.0
    }

    #[test]
    fn downsample_pair_caps_and_preserves_shape() {
        let xs: Vec<f64> = (0..1000).map(f64::from).collect();
        let ys: Vec<f64> = xs.iter().map(|v| v * 2.0).collect();
        let (dx, dy) = downsample_pair(&xs, &ys, 100);
        assert_eq!(dx.len(), 100);
        assert_eq!(dy.len(), 100);
        // row alignment is preserved (ys == 2*xs throughout)
        assert!(dx.iter().zip(&dy).all(|(x, y)| (y - x * 2.0).abs() < 1e-9));
        // endpoint-inclusive: both the first AND the last observation are retained, so a
        // chronologically sorted time-series keeps its latest point
        assert_eq!(dx[0], 0.0);
        assert_eq!(*dx.last().unwrap(), 999.0);
        assert_eq!(*dy.last().unwrap(), 1998.0);
        // single-element cap doesn't divide by zero and returns the first point
        assert_eq!(downsample_pair(&xs, &ys, 1), (vec![0.0], vec![0.0]));
        // already within cap -> returned unchanged
        let (sx, sy) = downsample_pair(&xs[..50], &ys[..50], 100);
        assert_eq!(sx.len(), 50);
        assert_eq!(sy.len(), 50);
        // cap == 0 -> unchanged (no divide-by-zero)
        assert_eq!(downsample_pair(&xs, &ys, 0).0.len(), 1000);
    }

    #[test]
    fn lttb_indices_keeps_endpoints_and_spike() {
        // a flat baseline with a single tall spike that sits BETWEEN uniform-stride samples
        let n = 1000;
        let cap = 50;
        let spike = 333usize; // 333 * (n-1) / (cap-1) is not an integer -> stride would miss it
        let xs: Vec<f64> = (0..n).map(|i| i as f64).collect();
        let ys: Vec<f64> = (0..n)
            .map(|i| if i == spike { 1000.0 } else { 0.0 })
            .collect();

        // baseline: uniform stride steps right over the spike
        let stride: Vec<usize> = (0..cap).map(|i| i * (n - 1) / (cap - 1)).collect();
        assert!(
            !stride.contains(&spike),
            "test precondition: stride must miss the spike"
        );
        let (_, stride_y) = downsample_pair(&xs, &ys, cap);
        assert!(
            stride_y.iter().all(|&y| y < 1.0),
            "uniform stride drops the spike"
        );

        // LTTB selects by triangle area, so the spike survives
        let keep = lttb_indices(&xs, &ys, cap);
        assert_eq!(keep.len(), cap);
        assert_eq!(keep[0], 0, "first point always kept");
        assert_eq!(*keep.last().unwrap(), n - 1, "last point always kept");
        assert!(
            keep.contains(&spike),
            "LTTB preserves the peak a uniform stride misses"
        );
        // indices are strictly increasing (monotonic selection)
        assert!(keep.windows(2).all(|w| w[0] < w[1]));

        // edge caps: no panic, sensible output
        assert_eq!(lttb_indices(&xs, &ys, 0), (0..n).collect::<Vec<_>>());
        assert_eq!(lttb_indices(&xs, &ys, 1), vec![0]);
        assert_eq!(lttb_indices(&xs, &ys, 2), vec![0, n - 1]);
        // already within cap -> all indices unchanged
        assert_eq!(
            lttb_indices(&xs[..10], &ys[..10], 50),
            (0..10).collect::<Vec<_>>()
        );
    }

    /// A tight NYC cluster plus a handful of far-west bad coordinates, used to contrast the
    /// trimmed (`viz smart`) framing against the full-extent (`viz map`) framing.
    fn nyc_cluster_with_outliers() -> (Vec<f64>, Vec<f64>) {
        let mut lats = vec![40.7_f64; 200];
        let mut lons = vec![-73.95_f64; 200];
        // jitter so the cluster has a small but non-zero span
        for (i, (la, lo)) in lats.iter_mut().zip(lons.iter_mut()).enumerate() {
            *la += (i % 10) as f64 * 0.005;
            *lo += (i % 10) as f64 * 0.005;
        }
        // ~2.5% outliers dragging the raw bounding box out to Pennsylvania
        for _ in 0..5 {
            lats.push(40.5);
            lons.push(-77.5);
        }
        (lats, lons)
    }

    #[test]
    fn map_center_zoom_trimmed_ignores_outliers() {
        // robust (percentile-trimmed) framing must center on the cluster, not the outliers.
        let (lats, lons) = nyc_cluster_with_outliers();
        let (center, _zoom) = map_center_zoom(&lats, &lons, MAP_FRAME_TRIM_FRAC, 1000.0, 600.0);
        let lon = serde_json::to_value(&center).unwrap()["lon"]
            .as_f64()
            .unwrap();
        // raw min/max midpoint would be ~(-73.95 + -77.5)/2 = -75.7 (in Pennsylvania); the trimmed
        // framing must stay over the NYC cluster instead.
        assert!(
            lon > -74.3,
            "trimmed framing should not be pulled west of NYC, got lon {lon}"
        );
    }

    #[test]
    fn map_center_zoom_full_extent_includes_outliers() {
        // standalone `viz map` passes trim_frac == 0.0, so every coordinate frames the view; the
        // far-west outliers pull the center toward the raw midpoint (regression guard for the fix
        // that kept the trimming scoped to `viz smart`).
        let (lats, lons) = nyc_cluster_with_outliers();
        let (center, _zoom) = map_center_zoom(&lats, &lons, 0.0, 1000.0, 600.0);
        let lon = serde_json::to_value(&center).unwrap()["lon"]
            .as_f64()
            .unwrap();
        assert!(
            lon < -75.0,
            "full-extent framing must include the western outliers, got lon {lon}"
        );
    }

    #[test]
    fn auto_hierarchy_style_by_depth() {
        // best-practice rule: shallow (≤2 levels) → treemap, deep (3+ levels) → sunburst.
        assert_eq!(auto_hierarchy_style(2), HierStyle::Treemap);
        assert_eq!(auto_hierarchy_style(3), HierStyle::Sunburst);
        assert_eq!(auto_hierarchy_style(4), HierStyle::Sunburst);
    }

    #[test]
    fn wrap_kpi_label_wraps_on_word_boundaries() {
        // short labels stay on one line (no <br> injected), so exact-text consumers/tests still see
        // the whole label.
        assert_eq!(wrap_kpi_label("Completeness", 40), "Completeness");
        assert_eq!(
            wrap_kpi_label("Mean Customer Rating", 40),
            "Mean Customer Rating"
        );
        // a long label wraps at the last word boundary that fits within max_chars.
        assert_eq!(
            wrap_kpi_label("Mean Profit Margin Percentage", 20),
            "Mean Profit Margin<br>Percentage"
        );
        // a single word longer than max_chars is left intact rather than hard-split mid-word.
        assert_eq!(
            wrap_kpi_label("Supercalifragilistic", 8),
            "Supercalifragilistic"
        );
        // multiple wraps.
        assert_eq!(wrap_kpi_label("a b c d e", 3), "a b<br>c d<br>e");
    }

    #[test]
    fn resolve_hierarchy_style_flag() {
        assert_eq!(
            resolve_hierarchy_style(None, 2).unwrap(),
            HierStyle::Treemap
        );
        assert_eq!(
            resolve_hierarchy_style(Some("auto"), 3).unwrap(),
            HierStyle::Sunburst
        );
        // case-insensitive explicit override wins over the depth rule
        assert_eq!(
            resolve_hierarchy_style(Some("Treemap"), 3).unwrap(),
            HierStyle::Treemap
        );
        assert_eq!(
            resolve_hierarchy_style(Some("sunburst"), 2).unwrap(),
            HierStyle::Sunburst
        );
        // icicle is an explicit-only override (auto never picks it)
        assert_eq!(
            resolve_hierarchy_style(Some("icicle"), 3).unwrap(),
            HierStyle::Icicle
        );
        assert_eq!(
            resolve_hierarchy_style(Some("ICICLE"), 2).unwrap(),
            HierStyle::Icicle
        );
        assert!(resolve_hierarchy_style(Some("pie"), 2).is_err());
    }

    #[test]
    fn hierarchy_arrays_rolls_up_and_keeps_ids_unique() {
        // 2-level hierarchy; "Widgets"/"Gadgets" repeat under two regions to exercise id
        // uniqueness.
        let mut leaves: HashMap<Vec<String>, f64> = HashMap::new();
        leaves.insert(vec!["East".into(), "Widgets".into()], 150.0);
        leaves.insert(vec!["East".into(), "Gadgets".into()], 30.0);
        leaves.insert(vec!["West".into(), "Widgets".into()], 80.0);
        leaves.insert(vec!["West".into(), "Gadgets".into()], 20.0);
        leaves.insert(vec!["West".into(), "Gizmos".into()], 10.0);

        let (labels, parents, values, ids) =
            hierarchy_arrays(&leaves, 2, 12, 200, "All").expect("hierarchy");

        // synthetic root: parentless, labeled, value == grand total.
        assert_eq!(ids[0], HIER_ROOT_ID);
        assert_eq!(parents[0], "");
        assert_eq!(labels[0], "All");
        let total: f64 = leaves.values().sum();
        assert!((values[0] - total).abs() < 1e-9);

        // every non-root parent reference resolves to a real id.
        let id_set: std::collections::HashSet<&String> = ids.iter().collect();
        for (i, p) in parents.iter().enumerate().skip(1) {
            assert!(
                id_set.contains(p),
                "parent {p} of node {} not in ids",
                ids[i]
            );
        }
        // path-joined ids are unique despite repeated child labels.
        assert_eq!(id_set.len(), ids.len(), "ids must be unique");

        // a level-1 node's value == sum of its level-2 children (East = 150 + 30).
        let east_id = format!("{HIER_ROOT_ID}{HIER_PATH_SEP}East");
        let east_val = values[ids.iter().position(|x| *x == east_id).unwrap()];
        assert!((east_val - 180.0).abs() < 1e-9);
    }

    #[test]
    fn hierarchy_arrays_ids_survive_adversarial_segments() {
        // (a) a segment containing a literal HIER_PATH_SEP must not join-collide with the
        // equivalent split path; (b) a real child value equal to "<sentinel>other" must not
        // collide with the parent's Other-bucket id. All ids stay globally distinct and every
        // parent ref resolves.
        let mut leaves: HashMap<Vec<String>, f64> = HashMap::new();
        leaves.insert(vec![format!("A{HIER_PATH_SEP}B"), "x".to_string()], 10.0);
        leaves.insert(vec!["A".to_string(), "B".to_string()], 20.0);
        leaves.insert(
            vec!["A".to_string(), format!("{AGG_KEY_SENTINEL}other")],
            5.0,
        );
        leaves.insert(vec!["A".to_string(), "C".to_string()], 1.0);
        // top_n=2 under "A" forces an Other bucket alongside the literal "<sentinel>other" child
        let (labels, parents, values, ids) = hierarchy_arrays(&leaves, 2, 2, 100, "root").unwrap();
        assert_eq!(labels.len(), parents.len());
        assert_eq!(labels.len(), values.len());
        assert_eq!(labels.len(), ids.len());
        let distinct: std::collections::HashSet<&String> = ids.iter().collect();
        assert_eq!(
            distinct.len(),
            ids.len(),
            "ids must be globally unique: {ids:?}"
        );
        // every non-root parent reference resolves to an emitted id
        for p in parents.iter().skip(1) {
            assert!(ids.contains(p), "dangling parent ref {p:?}");
        }
    }

    #[test]
    fn hierarchy_arrays_folds_remainder_into_other() {
        // asserts English output from a `t!`-backed helper -- pin the locale (see `english_locale`)
        let _locale = english_locale();
        // parent "R" has 4 children; top_n = 2 keeps a,b and folds c,d into "Other (2)".
        let mut leaves: HashMap<Vec<String>, f64> = HashMap::new();
        for (seg, v) in [("a", 40.0), ("b", 30.0), ("c", 20.0), ("d", 10.0)] {
            leaves.insert(vec!["R".into(), seg.into()], v);
        }
        // a second top-level category so the root itself splits (required for a real hierarchy).
        leaves.insert(vec!["S".into(), "x".into()], 5.0);

        let (labels, _parents, values, _ids) =
            hierarchy_arrays(&leaves, 2, 2, 200, "All").expect("hierarchy");

        let other_pos = labels
            .iter()
            .position(|l| l == "Other (2)")
            .expect("Other bucket");
        assert!((values[other_pos] - 30.0).abs() < 1e-9); // 20 + 10 dropped

        // kept children (40 + 30) + Other (30) == R's rolled-up value (100).
        let r_pos = labels.iter().position(|l| l == "R").unwrap();
        assert!((values[r_pos] - 100.0).abs() < 1e-9);
    }

    #[test]
    fn hierarchy_arrays_names_lone_dropped_child() {
        // asserts English output from a `t!`-backed helper -- pin the locale (see `english_locale`)
        let _locale = english_locale();
        // parent "R" has 3 children with top_n = 2: the single leftover ("c") is emitted under
        // its own name rather than as an "Other (1)" tile, and still expands to its own child.
        let mut leaves: HashMap<Vec<String>, f64> = HashMap::new();
        leaves.insert(vec!["R".into(), "a".into(), "a1".into()], 40.0);
        leaves.insert(vec!["R".into(), "b".into(), "b1".into()], 30.0);
        leaves.insert(vec!["R".into(), "c".into(), "c1".into()], 20.0);
        leaves.insert(vec!["S".into(), "x".into(), "x1".into()], 5.0);

        let (labels, _parents, values, _ids) =
            hierarchy_arrays(&leaves, 3, 2, 200, "All").expect("hierarchy");

        assert!(!labels.iter().any(|l| l.starts_with("Other")));
        let c_pos = labels
            .iter()
            .position(|l| l == "c")
            .expect("named leftover");
        assert!((values[c_pos] - 20.0).abs() < 1e-9);
        // the promoted child expanded like any other kept child
        assert!(labels.iter().any(|l| l == "c1"));
    }

    #[test]
    fn hierarchy_arrays_degenerate_returns_none() {
        // a single chain (no node splits into 2+) isn't worth a panel.
        let mut leaves: HashMap<Vec<String>, f64> = HashMap::new();
        leaves.insert(vec!["only".into(), "one".into()], 5.0);
        assert!(hierarchy_arrays(&leaves, 2, 12, 200, "All").is_none());
    }

    #[test]
    fn cramers_v_zero_for_independent_dims() {
        // Perfectly independent 2x2 joint (cell = row_tot*col_tot/n): chi-square is 0, so the
        // (bias-corrected) V is 0 — below HIER_MIN_ASSOCIATION_CRAMERS_V, so `viz smart` would
        // SKIP the hierarchy (this is exactly the sales_sample region/payment/category case).
        let mut leaves: HashMap<Vec<String>, f64> = HashMap::new();
        leaves.insert(vec!["a1".into(), "b1".into()], 30.0);
        leaves.insert(vec!["a1".into(), "b2".into()], 30.0);
        leaves.insert(vec!["a2".into(), "b1".into()], 20.0);
        leaves.insert(vec!["a2".into(), "b2".into()], 20.0);
        let v = max_pairwise_cramers_v(&leaves, 2);
        assert!(
            v < HIER_MIN_ASSOCIATION_CRAMERS_V,
            "independent V={v} should be ~0"
        );
    }

    #[test]
    fn cramers_v_high_for_nested_dims() {
        // Perfect nesting: each child category lives under exactly one parent (a real hierarchy),
        // so association is near-total and V clears the threshold — `viz smart` builds the panel.
        let mut leaves: HashMap<Vec<String>, f64> = HashMap::new();
        leaves.insert(vec!["a1".into(), "b1".into()], 25.0);
        leaves.insert(vec!["a1".into(), "b2".into()], 25.0);
        leaves.insert(vec!["a2".into(), "b3".into()], 25.0);
        leaves.insert(vec!["a2".into(), "b4".into()], 25.0);
        let v = max_pairwise_cramers_v(&leaves, 2);
        assert!(
            v >= HIER_MIN_ASSOCIATION_CRAMERS_V,
            "nested V={v} should be high"
        );
    }

    #[test]
    fn cramers_v_tiny_table_falls_back_to_uncorrected() {
        // 3x3 perfectly one-to-one table with only 3 rows: the Bergsma correction is undefined
        // (r_corr/c_corr collapse to 1.0, denom 0), but the dims ARE perfectly associated. The
        // uncorrected fallback must report a high V so this small hierarchy isn't wrongly skipped.
        let mut leaves: HashMap<Vec<String>, f64> = HashMap::new();
        leaves.insert(vec!["a1".into(), "b1".into()], 1.0);
        leaves.insert(vec!["a2".into(), "b2".into()], 1.0);
        leaves.insert(vec!["a3".into(), "b3".into()], 1.0);
        let v = max_pairwise_cramers_v(&leaves, 2);
        assert!(
            v >= HIER_MIN_ASSOCIATION_CRAMERS_V,
            "tiny perfectly-associated table V={v} should not read as independent"
        );
    }

    #[test]
    fn cramers_v_max_over_pairs_and_guards() {
        // fewer than two dims is undefined -> 0.0
        let mut single: HashMap<Vec<String>, f64> = HashMap::new();
        single.insert(vec!["x".into()], 10.0);
        assert_eq!(max_pairwise_cramers_v(&single, 1), 0.0);

        // 3 dims: levels 0&1 independent, but level 2 is perfectly nested under level 0; the MAX
        // over all pairs must surface that association so the panel isn't wrongly skipped.
        let mut leaves: HashMap<Vec<String>, f64> = HashMap::new();
        leaves.insert(vec!["a1".into(), "b1".into(), "c1".into()], 25.0);
        leaves.insert(vec!["a1".into(), "b2".into(), "c1".into()], 25.0);
        leaves.insert(vec!["a2".into(), "b1".into(), "c2".into()], 25.0);
        leaves.insert(vec!["a2".into(), "b2".into(), "c2".into()], 25.0);
        let v = max_pairwise_cramers_v(&leaves, 3);
        assert!(
            v >= HIER_MIN_ASSOCIATION_CRAMERS_V,
            "max-pair V={v} should catch nested level"
        );
    }

    #[test]
    fn sankey_concept_family_splits_on_dot() {
        assert_eq!(sankey_concept_family("geo.city"), Some("geo"));
        assert_eq!(sankey_concept_family("category.status"), Some("category"));
        assert_eq!(sankey_concept_family("nyc.complaint_type"), Some("nyc"));
        // no dot -> whole token is the family
        assert_eq!(sankey_concept_family("org"), Some("org"));
        // un-tagged columns yield no semantic signal
        assert_eq!(sankey_concept_family(""), None);
        assert_eq!(sankey_concept_family("  "), None);
        assert_eq!(sankey_concept_family("unknown"), None);
    }

    #[test]
    fn sankey_semantic_tier_ranks_bridges_over_geo_overlap() {
        // cross-family bridge (a real flow) -> preferred
        assert_eq!(
            sankey_semantic_tier("nyc.complaint_type", "category.status"),
            0
        );
        assert_eq!(sankey_semantic_tier("org.agency", "category.status"), 0);
        // both geo-context (spatial overlap) -> deprioritized
        assert_eq!(sankey_semantic_tier("geo.city", "geo.zip_code"), 2);
        // same family, or either side un-tagged -> neutral
        assert_eq!(sankey_semantic_tier("category.type", "category.status"), 1);
        assert_eq!(sankey_semantic_tier("", "category.status"), 1);
        assert_eq!(sankey_semantic_tier("unknown", "unknown"), 1);
    }

    #[test]
    fn sankey_side_topn_collapses_tail_into_other() {
        // more than SANKEY_TOP_N_PER_SIDE categories: the smallest collapse into "Other (k)".
        let mut tot: HashMap<String, f64> = HashMap::new();
        let n = SANKEY_TOP_N_PER_SIDE + 3;
        for i in 0..n {
            // descending weights so the last 3 are the collapsed tail
            tot.insert(format!("c{i:02}"), (n - i) as f64);
        }
        let (kept, kept_set, other, other_frac) = sankey_side_topn(&tot);
        assert_eq!(kept.len(), SANKEY_TOP_N_PER_SIDE);
        assert_eq!(kept_set.len(), SANKEY_TOP_N_PER_SIDE);
        assert_eq!(other.as_deref(), Some("Other (3)"));
        assert!(other_frac > 0.0 && other_frac < 1.0);
        // the heaviest category is kept and ordered first
        assert_eq!(kept[0], "c00");

        // at or below the cap: everything kept, no "Other".
        let mut small: HashMap<String, f64> = HashMap::new();
        small.insert("a".into(), 3.0);
        small.insert("b".into(), 1.0);
        let (kept, _set, other, other_frac) = sankey_side_topn(&small);
        assert_eq!(kept.len(), 2);
        assert_eq!(other, None);
        assert_eq!(other_frac, 0.0);

        // exactly one over the cap: the lone leftover keeps its own node instead of becoming an
        // opaque "Other (1)" node.
        let mut one_over: HashMap<String, f64> = HashMap::new();
        let n = SANKEY_TOP_N_PER_SIDE + 1;
        for i in 0..n {
            one_over.insert(format!("c{i:02}"), (n - i) as f64);
        }
        let (kept, kept_set, other, other_frac) = sankey_side_topn(&one_over);
        assert_eq!(kept.len(), n);
        assert_eq!(kept_set.len(), n);
        assert_eq!(other, None);
        assert_eq!(other_frac, 0.0);
        assert_eq!(kept[n - 1], format!("c{:02}", n - 1));
    }

    #[test]
    fn hex_to_rgba_parses_palette_hex() {
        // PALETTE[0] is #4C78A8 -> (76, 120, 168)
        assert_eq!(hex_to_rgba("#4C78A8", 0.45), "rgba(76,120,168,0.45)");
        // leading '#' is optional and lowercase hex parses too
        assert_eq!(hex_to_rgba("ff9da6", 0.5), "rgba(255,157,166,0.5)");
    }

    #[test]
    fn sankey_value_positions_ranks_by_flow_per_column() {
        // Bipartite: sources {0:A, 1:B}, targets {2:X, 3:Y}. A is the heavier source, X the
        // heavier target, so each should sit at the TOP (smaller y) of its column.
        //   A->X = 10, A->Y = 1, B->X = 2   (throughput: A=11, B=2, X=12, Y=1)
        let src = [0_usize, 0, 1];
        let tgt = [2_usize, 3, 2];
        let val = [10.0_f64, 1.0, 2.0];
        let (xs, ys) = sankey_value_positions(4, &src, &tgt, &val).unwrap();
        // source-only nodes hug the left, target-only nodes hug the right
        assert!(xs[0] < 0.5 && xs[1] < 0.5, "sources on the left");
        assert!(xs[2] > 0.5 && xs[3] > 0.5, "targets on the right");
        // within each column the heavier node is higher (smaller y)
        assert!(ys[0] < ys[1], "A (heavier source) above B");
        assert!(ys[2] < ys[3], "X (heavier target) above Y");
        // empty graph yields no positions
        assert!(sankey_value_positions(0, &[], &[], &[]).is_none());
    }

    #[test]
    fn pip_inside_outside_and_snap() {
        // one 0..10 square keyed by properties.id
        let gj = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "properties": {"id": "A"},
                "geometry": {"type": "Polygon", "coordinates":
                    [[[0.0, 0.0], [0.0, 10.0], [10.0, 10.0], [10.0, 0.0], [0.0, 0.0]]]}
            }]
        });
        let feats = build_pip_features(&gj, "properties.id", None).unwrap();
        assert_eq!(feats.len(), 1);
        assert_eq!(feats[0].id, "A");
        // (lat, lon) inside the square
        assert_eq!(
            pip_assign(&feats, 5.0, 5.0, false, f64::INFINITY),
            PipOutcome::Inside(0)
        );
        // far outside, no snap -> dropped
        assert_eq!(
            pip_assign(&feats, 50.0, 50.0, false, f64::INFINITY),
            PipOutcome::Outside
        );
        // far outside, snap with no distance cap -> nearest (the only) feature
        assert_eq!(
            pip_assign(&feats, 50.0, 50.0, true, f64::INFINITY),
            PipOutcome::Snapped(0)
        );
        // same point, but a tight cap is farther than any boundary -> dropped despite snap
        assert_eq!(
            pip_assign(&feats, 50.0, 50.0, true, 10.0),
            PipOutcome::Outside
        );
    }

    // the --snap-max-dist cap is an equirectangular km distance: a point ~5.6 km outside a region
    // snaps under a 10 km cap but drops under a 4 km cap (and always snaps when unbounded).
    #[test]
    fn pip_snap_distance_cap() {
        // unit square straddling the equator, where 1 degree ~ 111.32 km.
        let gj = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "properties": {"id": "A"},
                "geometry": {"type": "Polygon", "coordinates":
                    [[[0.0, 0.0], [0.0, 1.0], [1.0, 1.0], [1.0, 0.0], [0.0, 0.0]]]}
            }]
        });
        let feats = build_pip_features(&gj, "properties.id", None).unwrap();
        // 0.05 deg east of the boundary at the equator ~ 5.6 km out
        let (lat, lon) = (0.5, 1.05);
        assert_eq!(
            pip_assign(&feats, lat, lon, true, 10.0),
            PipOutcome::Snapped(0),
            "within the 10 km cap -> snaps"
        );
        assert_eq!(
            pip_assign(&feats, lat, lon, true, 4.0),
            PipOutcome::Outside,
            "beyond the 4 km cap -> dropped"
        );
        assert_eq!(
            pip_assign(&feats, lat, lon, true, f64::INFINITY),
            PipOutcome::Snapped(0),
            "unbounded cap always snaps"
        );
    }

    // the cap is an EQUIRECTANGULAR km distance: longitude degrees shrink by cos(lat). At lat 60.5,
    // a point 0.5 deg east of a boundary is ~27 km away (0.5 * 111.32 * cos(60.5) ~ 27.4) — NOT the
    // ~56 km a missing-cos(lat) bug would compute. The two-sided bracket (snaps under 35 km, drops
    // under 20 km) pins the longitude scaling: a no-cos bug would give 56 km and drop under 35.
    #[test]
    fn pip_snap_cap_applies_longitude_cos_scaling() {
        let gj = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "properties": {"id": "A"},
                "geometry": {"type": "Polygon", "coordinates":
                    [[[0.0, 60.0], [0.0, 61.0], [1.0, 61.0], [1.0, 60.0], [0.0, 60.0]]]}
            }]
        });
        let feats = build_pip_features(&gj, "properties.id", None).unwrap();
        // 0.5 deg east of the east edge at lat 60.5 -> ~27.4 km out
        let (lat, lon) = (60.5, 1.5);
        assert_eq!(
            pip_assign(&feats, lat, lon, true, 35.0),
            PipOutcome::Snapped(0),
            "~27 km < 35 km cap -> snaps (would be ~56 km and DROP without cos(lat) scaling)"
        );
        assert_eq!(
            pip_assign(&feats, lat, lon, true, 20.0),
            PipOutcome::Outside,
            "~27 km > 20 km cap -> dropped"
        );
    }

    // the `viz smart` GeoJSON choropleth panel honors the snap-distance cap: a point in the gap
    // between two regions (far from both) is dropped under the default cap — surfaced in the panel
    // title — but snapped when the cap is unbounded. Exercised at the unit level because the
    // Data Schematic's upstream geographic-column detection / outlier removal make an end-to-end
    // fixture brittle; this calls the panel builder directly with explicit coordinates.
    #[test]
    fn smart_pip_panel_caps_snap() {
        // the "Regions (...)" panel title is localized -- pin English. `--no-snap`
        // inside it is a FLAG NAME and stays literal in every locale, which is
        // why the third assertion below still expects it verbatim.
        let _locale = english_locale();
        // two regions separated by a wide gap: A at lon 0..4, C at lon 16..20 (both lat 0..10).
        let gj = r#"{"type":"FeatureCollection","features":[
            {"type":"Feature","properties":{"id":"A"},"geometry":{"type":"Polygon",
             "coordinates":[[[0.0,0.0],[0.0,10.0],[4.0,10.0],[4.0,0.0],[0.0,0.0]]]}},
            {"type":"Feature","properties":{"id":"C"},"geometry":{"type":"Polygon",
             "coordinates":[[[16.0,0.0],[16.0,10.0],[20.0,10.0],[20.0,0.0],[16.0,0.0]]]}}]}"#;
        let path = std::env::temp_dir().join("qsv_smart_pip_cap_test.geojson");
        std::fs::write(&path, gj).unwrap();
        let spec = path.to_str().unwrap();
        // 2 points in A, 2 in C, 1 in the lon-10 gap (~6 deg / hundreds of km from either edge).
        let lats = vec![5.0, 4.0, 5.0, 4.0, 5.0];
        let lons = vec![2.0, 3.0, 18.0, 17.0, 10.0];

        // an explicit 10 km cap drops the gap point; the panel still renders (2 regions) and
        // flags the drop.
        let panel = build_smart_pip_choropleth_panel(
            spec,
            "properties.id",
            None,
            &lats,
            &lons,
            true,
            Some(10.0),
            None,
            None,
        )
        .unwrap()
        .expect("2 regions keep points, so a panel renders");
        assert_eq!(panel.name, "Regions (1 of 5 dropped, >10 km)");

        // an unbounded cap snaps the gap point instead -> nothing dropped; the title surfaces the
        // snap metadata (count + cap) rather than staying bare.
        let snapped = build_smart_pip_choropleth_panel(
            spec,
            "properties.id",
            None,
            &lats,
            &lons,
            true,
            Some(f64::INFINITY),
            None,
            None,
        )
        .unwrap()
        .expect("panel renders");
        assert!(
            snapped.name.starts_with("Regions (1 snapped"),
            "snapped points must surface in the title, got {:?}",
            snapped.name
        );

        // --no-snap drops the gap point regardless of distance; the title reports the reason.
        let no_snap = build_smart_pip_choropleth_panel(
            spec,
            "properties.id",
            None,
            &lats,
            &lons,
            false,
            Some(10.0),
            None,
            None,
        )
        .unwrap()
        .expect("panel renders");
        assert_eq!(no_snap.name, "Regions (1 of 5 dropped, --no-snap)");

        // the continental-extent regions above (20 deg lon / 10 deg lat span, both >=
        // SMART_CHOROPLETH_MIN_SPAN_DEG) stay on the projection `geo` basemap, NOT tiles.
        assert!(
            matches!(panel.kind, PanelKind::Choropleth { .. }),
            "continental extent uses the projection Choropleth, got {:?}",
            std::mem::discriminant(&panel.kind)
        );

        let _ = std::fs::remove_file(&path);
    }

    // The SPANISH counterpart of `smart_pip_panel_caps_snap`, and the reason it has
    // to exist: an English-pinned test cannot guard localization. Revert any `t!`
    // below to the `format!` it replaced and every English assertion still passes,
    // because English IS the catalog's value -- so the pin proves byte-identity, not
    // that the string is localized at all (roborev 3959). This test fails on that
    // revert. It covers the panel title, the count measure label, the snapped-in
    // hover suffix, and the pct/rank hover lines.
    #[test]
    fn smart_pip_panel_localizes_title_and_hover() {
        let _guard = viz_i18n::lock_locale();
        viz_i18n::set_active(viz_i18n::parse_lang("es").unwrap());

        // same geometry as the English test: regions A (lon 0..4) and C (lon 16..20)
        // separated by a wide gap, plus one point in the gap at lon 10.
        let gj = r#"{"type":"FeatureCollection","features":[
            {"type":"Feature","properties":{"id":"A"},"geometry":{"type":"Polygon",
             "coordinates":[[[0.0,0.0],[0.0,10.0],[4.0,10.0],[4.0,0.0],[0.0,0.0]]]}},
            {"type":"Feature","properties":{"id":"C"},"geometry":{"type":"Polygon",
             "coordinates":[[[16.0,0.0],[16.0,10.0],[20.0,10.0],[20.0,0.0],[16.0,0.0]]]}}]}"#;
        // a DISTINCT temp path: this runs in parallel with the English test, which
        // removes its own file at the end.
        let path = std::env::temp_dir().join("qsv_smart_pip_es_test.geojson");
        std::fs::write(&path, gj).unwrap();
        let spec = path.to_str().unwrap();
        let lats = vec![5.0, 4.0, 5.0, 4.0, 5.0];
        let lons = vec![2.0, 3.0, 18.0, 17.0, 10.0];

        let build = |snap: bool, cap: Option<f64>| {
            build_smart_pip_choropleth_panel(
                spec,
                "properties.id",
                None,
                &lats,
                &lons,
                snap,
                cap,
                None,
                None,
            )
            .unwrap()
            .expect("2 regions keep points, so a panel renders")
        };

        // dropped branch: title shell, the drop clause and the ">N km" reason all localized
        assert_eq!(
            build(true, Some(10.0)).name,
            "Regiones (1 de 5 descartados, >10 km)"
        );

        // snapped branch: the snap clause, plus every localized hover fragment
        let snapped = build(true, Some(f64::INFINITY));
        assert!(
            snapped.name.starts_with("Regiones (1 ajustados"),
            "the snap clause must be localized, got {:?}",
            snapped.name
        );
        let PanelKind::Choropleth { hover_text, .. } = &snapped.kind else {
            panic!("continental extent builds a projection Choropleth");
        };
        let hover = hover_text.join("\n");
        for expected in [
            "recuento:",                       // viz.chart.count, the measure label
            "incluye 1 ajustados desde fuera", // viz.hover.snap_includes
            "del total",                       // viz.hover.pct_of_total
            "puesto ",                         // viz.hover.rank_of
        ] {
            assert!(
                hover.contains(expected),
                "hover should contain localized {expected:?}, got {hover:?}"
            );
        }
        assert!(
            !hover.contains("count:") && !hover.contains("snapped from outside"),
            "no English hover fragment should survive, got {hover:?}"
        );

        // `--no-snap` is a FLAG NAME, not a word: it must survive verbatim in every
        // locale, or the title would name a flag that does not exist.
        assert_eq!(
            build(false, Some(10.0)).name,
            "Regiones (1 de 5 descartados, --no-snap)"
        );

        let _ = std::fs::remove_file(&path);
        viz_i18n::reset_active();
    }

    // a metro-scale `--geojson` choropleth (matched regions span < SMART_CHOROPLETH_MIN_SPAN_DEG in
    // BOTH dims) switches to the MapLibre tile basemap (`ChoroplethMap`), framed to the region
    // bbox, so city-scale coastline/street detail isn't lost on the coarse projection basemap.
    #[test]
    fn smart_pip_panel_metro_extent_uses_tile_basemap() {
        // two small adjacent NYC-scale regions: ~0.05 deg each, union ~0.1 deg lon / 0.05 deg lat.
        let gj = r#"{"type":"FeatureCollection","features":[
            {"type":"Feature","properties":{"id":"A"},"geometry":{"type":"Polygon",
             "coordinates":[[[-74.05,40.70],[-74.05,40.75],[-74.00,40.75],[-74.00,40.70],[-74.05,40.70]]]}},
            {"type":"Feature","properties":{"id":"B"},"geometry":{"type":"Polygon",
             "coordinates":[[[-74.00,40.70],[-74.00,40.75],[-73.95,40.75],[-73.95,40.70],[-74.00,40.70]]]}}]}"#;
        let path = std::env::temp_dir().join("qsv_smart_pip_metro_test.geojson");
        std::fs::write(&path, gj).unwrap();
        let spec = path.to_str().unwrap();
        // 2 points in A, 2 in B.
        let lats = vec![40.72, 40.73, 40.72, 40.73];
        let lons = vec![-74.02, -74.03, -73.97, -73.98];

        let panel = build_smart_pip_choropleth_panel(
            spec,
            "properties.id",
            None,
            &lats,
            &lons,
            true,
            Some(10.0),
            None,
            None,
        )
        .unwrap()
        .expect("2 regions keep points, so a panel renders");
        match panel.kind {
            PanelKind::ChoroplethMap {
                center_lon,
                center_lat,
                zoom,
                ..
            } => {
                // centered on the matched-region bbox union (lon -74.05..-73.95, lat 40.70..40.75).
                assert!(
                    (center_lon - (-74.0)).abs() < 1e-9,
                    "center_lon = {center_lon}"
                );
                // the centre latitude is the MERCATOR midpoint (the zoom is fitted in Mercator
                // y, so the centre must be taken in the same space or the box is not actually
                // centred). Over a 0.05-degree box that is ~5e-6 from the arithmetic mean.
                assert!(
                    (center_lat - 40.725).abs() < 1e-4,
                    "center_lat = {center_lat}"
                );
                assert!(
                    (center_lat - mercator_lat_center(40.70, 40.75)).abs() < 1e-9,
                    "center_lat = {center_lat}"
                );
                // a ~0.1 deg metro box zooms in well past the world view, within the fit clamp.
                assert!(
                    (9.0..=16.0).contains(&zoom),
                    "metro zoom should be street-level, got {zoom}"
                );
            },
            other => panic!(
                "metro extent should use the tile basemap, got {:?}",
                std::mem::discriminant(&other)
            ),
        }

        let _ = std::fs::remove_file(&path);
    }

    // as-written decimal counting for the command path's precision signal: exponent-aware,
    // trailing fractional zeros trimmed (matching the stats cache's shortest-format semantics).
    #[test]
    fn byte_decimal_places_counts() {
        assert_eq!(byte_decimal_places(b"12"), 0);
        assert_eq!(byte_decimal_places(b"-40.71"), 2);
        assert_eq!(byte_decimal_places(b"40.7100"), 2, "trailing zeros trimmed");
        assert_eq!(byte_decimal_places(b" 40.5 "), 1, "whitespace trimmed");
        assert_eq!(byte_decimal_places(b"1.5e-3"), 4, "0.0015 -> 4 decimals");
        assert_eq!(byte_decimal_places(b"7e-2"), 2, "0.07 -> 2 decimals");
        assert_eq!(byte_decimal_places(b"1.5E2"), 0, "150 -> 0 decimals");
        // trailing zeros must be trimmed AFTER the exponent is applied: a trailing-zero mantissa
        // cancels fractional positions of the shifted value (roborev #3488).
        assert_eq!(byte_decimal_places(b"150e-2"), 1, "1.5 -> 1 decimal");
        assert_eq!(byte_decimal_places(b"4070e-2"), 1, "40.7 -> 1 decimal");
        assert_eq!(byte_decimal_places(b"1000e-3"), 0, "1 -> 0 decimals");
        assert_eq!(byte_decimal_places(b"50.0e-1"), 0, "5 -> 0 decimals");
        assert_eq!(byte_decimal_places(b"0.150"), 2, "0.15 -> 2 decimals");
        // an all-zero mantissa is zero: 0 decimal places regardless of notation (roborev #3489).
        assert_eq!(byte_decimal_places(b"0e-3"), 0, "0 -> 0 decimals");
        assert_eq!(byte_decimal_places(b"0.0e-3"), 0, "0 -> 0 decimals");
        assert_eq!(byte_decimal_places(b"0.000"), 0, "0 -> 0 decimals");
        assert_eq!(byte_decimal_places(b""), 0);
        assert_eq!(byte_decimal_places(b"n/a"), 0);
    }

    /// Build `PipFeatures` from equator-anchored square boxes `side_deg` degrees on a side (one per
    /// `count`), for exercising the region-size term of the auto snap cap.
    fn square_features(side_deg: f64, count: usize) -> Vec<PipFeature> {
        let features: Vec<serde_json::Value> = (0..count)
            .map(|i| {
                // space the boxes apart so they don't merge visually; geometry only matters for
                // the bbox diagonal.
                let x0 = (i as f64) * (side_deg * 2.0);
                let (x1, y1) = (x0 + side_deg, side_deg);
                serde_json::json!({
                    "type": "Feature",
                    "properties": {"id": format!("R{i}")},
                    "geometry": {"type": "Polygon", "coordinates":
                        [[[x0, 0.0], [x0, y1], [x1, y1], [x1, 0.0], [x0, 0.0]]]}
                })
            })
            .collect();
        let gj = serde_json::json!({"type": "FeatureCollection", "features": features});
        build_pip_features(&gj, "properties.id", None).unwrap()
    }

    // ward-scale regions (~3 km bbox diagonal) with precise coordinates derive a sub-km cap from
    // the region-size term — the old fixed 10 km default would snap across several wards.
    #[test]
    fn snap_cap_ward_scale_region_fraction() {
        // 0.02 deg squares at the equator: diag = hypot(2.2264, 2.2264) ~ 3.149 km;
        // region term = 0.315 km; p=5 precision floor ~ 0.0016 km is negligible.
        let feats = square_features(0.02, 3);
        let cap = resolve_snap_cap(None, &feats, Some(5));
        assert!(
            (cap.km - 0.31).abs() < 1e-9,
            "ward-scale cap should be ~0.31 km, got {}",
            cap.km
        );
        assert!(matches!(
            cap.basis,
            SnapCapBasis::Auto {
                region_km:    Some(_),
                precision_km: Some(_),
            }
        ));
    }

    // country-scale regions clamp to the absolute ceiling instead of growing without bound.
    #[test]
    fn snap_cap_country_scale_clamps_max() {
        // 10 deg squares: diag ~ 1574 km -> region term ~ 157 km -> clamped to 100 km.
        let feats = square_features(10.0, 3);
        let cap = resolve_snap_cap(None, &feats, Some(4));
        assert!(
            (cap.km - SNAP_CAP_MAX_KM).abs() < f64::EPSILON,
            "continent-scale cap should clamp to {SNAP_CAP_MAX_KM}, got {}",
            cap.km
        );
    }

    // coarse coordinates dominate small regions: 1-decimal lat/lon quantizes by kilometers, so
    // the stats-aware precision floor (not the region term) sets the cap.
    #[test]
    fn snap_cap_precision_floor_dominates() {
        // region term ~ 0.315 km; p=1 floor = 2 * 111.32 * (1/sqrt(2)) * 0.1 ~ 15.74 -> 16 km.
        let feats = square_features(0.02, 3);
        let cap = resolve_snap_cap(None, &feats, Some(1));
        assert!(
            (cap.km - 16.0).abs() < 1e-9,
            "1-decimal coords should floor the cap at ~16 km, got {}",
            cap.km
        );
    }

    // The stderr phrase keeps its literals in Rust; the below-map note reads the catalog. Nothing
    // but this assertion stops the two from drifting apart, and only ONE of the five phrases is
    // asserted by an integration test -- the other four rest solely here. A missing or misspelled
    // key fails too: rust-i18n renders the key itself, which never equals the literal.
    #[test]
    fn snap_basis_catalog_matches_english_phrase() {
        let _locale = english_locale();
        for basis in [
            SnapCapBasis::Explicit,
            SnapCapBasis::Auto {
                region_km:    Some(12.0),
                precision_km: Some(3.0),
            },
            SnapCapBasis::Auto {
                region_km:    Some(12.0),
                precision_km: None,
            },
            SnapCapBasis::Auto {
                region_km:    None,
                precision_km: Some(3.0),
            },
            // never leaves resolve_snap_cap (it returns Fallback); covered for exhaustiveness.
            SnapCapBasis::Auto {
                region_km:    None,
                precision_km: None,
            },
            SnapCapBasis::Fallback,
        ] {
            let cap = SnapCap { km: 1.0, basis };
            assert_eq!(
                cap.basis_phrase_localized(),
                cap.basis_phrase_en(),
                "catalog value drifted from the English literal for {basis:?}"
            );
        }
    }

    // with neither signal available the cap falls back to the fixed default.
    #[test]
    fn snap_cap_fallback_default() {
        let cap = resolve_snap_cap(None, &[], None);
        assert!((cap.km - DEFAULT_SNAP_MAX_KM).abs() < f64::EPSILON);
        assert_eq!(cap.basis, SnapCapBasis::Fallback);
    }

    // an explicit --snap-max-dist is used verbatim: no clamping, no 2-sig-fig rounding.
    #[test]
    fn snap_cap_explicit_wins() {
        let feats = square_features(0.02, 3);
        let cap = resolve_snap_cap(Some(3.456), &feats, Some(1));
        assert!((cap.km - 3.456).abs() < f64::EPSILON);
        assert_eq!(cap.basis, SnapCapBasis::Explicit);
    }

    // stats-cache coordinate precision: Float uses max_precision, Integer counts as 0 decimals,
    // and the coarser (min) column drives; anything else disables the floor.
    #[test]
    fn stats_coord_decimals_prefers_coarser_column() {
        use crate::cmd::stats::StatsData;
        let float_col = |p: Option<u32>| StatsData {
            r#type: "Float".to_string(),
            max_precision: p,
            ..Default::default()
        };
        let int_col = || StatsData {
            r#type: "Integer".to_string(),
            ..Default::default()
        };
        let string_col = || StatsData {
            r#type: "String".to_string(),
            ..Default::default()
        };

        let stats = vec![float_col(Some(5)), float_col(Some(2))];
        assert_eq!(stats_coord_decimals(&stats, 0, 1), Some(2));

        let stats = vec![float_col(Some(5)), int_col()];
        assert_eq!(
            stats_coord_decimals(&stats, 0, 1),
            Some(0),
            "an Integer coordinate column is 0-decimal"
        );

        let stats = vec![float_col(None), float_col(Some(2))];
        assert_eq!(
            stats_coord_decimals(&stats, 0, 1),
            None,
            "a stale cache without max_precision disables the floor"
        );

        let stats = vec![string_col(), float_col(Some(2))];
        assert_eq!(stats_coord_decimals(&stats, 0, 1), None);
    }

    #[test]
    fn pip_multipolygon_and_holes() {
        let gj = serde_json::json!({
            "type": "FeatureCollection",
            "features": [
                {"type": "Feature", "properties": {"id": "M"}, "geometry": {
                    "type": "MultiPolygon", "coordinates": [
                        [[[0.0, 0.0], [0.0, 2.0], [2.0, 2.0], [2.0, 0.0], [0.0, 0.0]]],
                        [[[10.0, 10.0], [10.0, 12.0], [12.0, 12.0], [12.0, 10.0], [10.0, 10.0]]]
                    ]}},
                {"type": "Feature", "properties": {"id": "H"}, "geometry": {
                    "type": "Polygon", "coordinates": [
                        [[20.0, 20.0], [20.0, 30.0], [30.0, 30.0], [30.0, 20.0], [20.0, 20.0]],
                        [[23.0, 23.0], [23.0, 27.0], [27.0, 27.0], [27.0, 23.0], [23.0, 23.0]]
                    ]}}
            ]
        });
        let feats = build_pip_features(&gj, "properties.id", None).unwrap();
        let m = feats.iter().position(|f| f.id == "M").unwrap();
        let h = feats.iter().position(|f| f.id == "H").unwrap();
        // inside the SECOND polygon of the MultiPolygon
        assert_eq!(
            pip_assign(&feats, 11.0, 11.0, false, f64::INFINITY),
            PipOutcome::Inside(m)
        );
        // inside H's filled band (between exterior and hole)
        assert_eq!(
            pip_assign(&feats, 21.0, 21.0, false, f64::INFINITY),
            PipOutcome::Inside(h)
        );
        // inside H's HOLE -> not contained; no snap -> dropped
        assert_eq!(
            pip_assign(&feats, 25.0, 25.0, false, f64::INFINITY),
            PipOutcome::Outside
        );
        // same hole point, snap with no cap -> snaps back to H (nearest boundary)
        assert_eq!(
            pip_assign(&feats, 25.0, 25.0, true, f64::INFINITY),
            PipOutcome::Snapped(h)
        );
    }

    #[test]
    fn pip_feature_id_top_level_numeric_and_missing_skipped() {
        // first feature has a numeric top-level id; second has no top-level id (skipped under "id")
        let gj = serde_json::json!({
            "type": "FeatureCollection",
            "features": [
                {"type": "Feature", "id": 7, "properties": {}, "geometry": {
                    "type": "Polygon", "coordinates":
                        [[[0.0, 0.0], [0.0, 1.0], [1.0, 1.0], [1.0, 0.0], [0.0, 0.0]]]}},
                {"type": "Feature", "properties": {"name": "x"}, "geometry": {
                    "type": "Polygon", "coordinates":
                        [[[5.0, 5.0], [5.0, 6.0], [6.0, 6.0], [6.0, 5.0], [5.0, 5.0]]]}}
            ]
        });
        let feats = build_pip_features(&gj, "id", None).unwrap();
        assert_eq!(feats.len(), 1, "feature without a top-level id is skipped");
        assert_eq!(feats[0].id, "7", "numeric id coerces to string");
        assert_eq!(
            pip_assign(&feats, 0.5, 0.5, false, f64::INFINITY),
            PipOutcome::Inside(0)
        );
    }

    #[test]
    fn pip_build_errors_when_no_feature_matches_key() {
        let gj = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "properties": {"name": "x"},
                "geometry": {"type": "Polygon", "coordinates":
                    [[[0.0, 0.0], [0.0, 1.0], [1.0, 1.0], [1.0, 0.0], [0.0, 0.0]]]}
            }]
        });
        // no feature has properties.id -> usable set is empty -> error
        assert!(build_pip_features(&gj, "properties.id", None).is_err());
    }

    #[test]
    fn escape_hover_escapes_markup() {
        assert_eq!(escape_hover("A & <B>"), "A &amp; &lt;B&gt;");
        // & must be escaped first so an already-escaped entity isn't double-escaped wrongly
        assert_eq!(escape_hover("<b>"), "&lt;b&gt;");
    }

    #[test]
    fn fmt_measure_whole_vs_decimal() {
        assert_eq!(fmt_measure(65.0), "65");
        assert_eq!(fmt_measure(3.27), "3.27");
        assert_eq!(fmt_measure(3.3630), "3.363");
        assert_eq!(fmt_measure(0.5), "0.5");
        // thousands separators are inserted into the integer part only
        assert_eq!(fmt_measure(1_234_567.0), "1,234,567");
        assert_eq!(fmt_measure(1234.5), "1,234.5");
        assert_eq!(fmt_measure(-1_234_567.0), "-1,234,567");
        assert_eq!(fmt_measure(-12_345.678), "-12,345.678");
        assert_eq!(fmt_measure(999.0), "999");
    }

    #[test]
    fn build_pip_features_autodetects_and_overrides_name() {
        // properties.name is auto-detected when no key is given
        let gj = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "properties": {"id": "A", "name": "Alpha", "label": "Other"},
                "geometry": {"type": "Polygon", "coordinates":
                    [[[0.0, 0.0], [0.0, 1.0], [1.0, 1.0], [1.0, 0.0], [0.0, 0.0]]]}
            }]
        });
        let feats = build_pip_features(&gj, "properties.id", None).unwrap();
        assert_eq!(feats[0].name.as_deref(), Some("Alpha"));
        // explicit --feature-name-key overrides auto-detect
        let feats = build_pip_features(&gj, "properties.id", Some("properties.label")).unwrap();
        assert_eq!(feats[0].name.as_deref(), Some("Other"));
        // no name property anywhere -> None (universal hover, no name line)
        let gj2 = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "properties": {"id": "A"},
                "geometry": {"type": "Polygon", "coordinates":
                    [[[0.0, 0.0], [0.0, 1.0], [1.0, 1.0], [1.0, 0.0], [0.0, 0.0]]]}
            }]
        });
        let feats = build_pip_features(&gj2, "properties.id", None).unwrap();
        assert_eq!(feats[0].name, None);
    }

    #[test]
    fn build_pip_features_resolves_top_level_name_foreign_member() {
        // a top-level (foreign-member) "name" — TopoJSON-style exports carry names there, not
        // under properties — must auto-detect via the bare "name" candidate
        let gj = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "name": "Beta",
                "properties": {"id": "B"},
                "geometry": {"type": "Polygon", "coordinates":
                    [[[0.0, 0.0], [0.0, 1.0], [1.0, 1.0], [1.0, 0.0], [0.0, 0.0]]]}
            }]
        });
        let feats = build_pip_features(&gj, "properties.id", None).unwrap();
        assert_eq!(feats[0].name.as_deref(), Some("Beta"));
        // and an explicit non-properties --feature-name-key resolves it too
        let feats = build_pip_features(&gj, "properties.id", Some("name")).unwrap();
        assert_eq!(feats[0].name.as_deref(), Some("Beta"));
    }

    #[test]
    fn build_pip_features_escapes_name() {
        let gj = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "properties": {"id": "A", "name": "R&D <x>"},
                "geometry": {"type": "Polygon", "coordinates":
                    [[[0.0, 0.0], [0.0, 1.0], [1.0, 1.0], [1.0, 0.0], [0.0, 0.0]]]}
            }]
        });
        let feats = build_pip_features(&gj, "properties.id", None).unwrap();
        assert_eq!(feats[0].name.as_deref(), Some("R&amp;D &lt;x&gt;"));
    }

    #[test]
    fn choropleth_hover_text_content_and_alignment() {
        // the "% of total" / "rank N of N" lines are localized -- pin English.
        // The "count" label is passed IN below and so needs no pin itself.
        let _locale = english_locale();
        // z deliberately unsorted so rank (by descending z) differs from index order
        let locs = vec!["JP01".to_string(), "JP02".to_string(), "JP03".to_string()];
        let z = vec![10.0, 30.0, 60.0];
        let names = vec!["Hokkaido".to_string(), String::new(), "Okinawa".to_string()];
        let h = choropleth_hover_text(&locs, &z, Some(&names), "count", true);
        assert_eq!(h.len(), 3);
        // named region: name (id), labeled count, share, rank (10/100 = 10%, lowest -> rank 3)
        assert_eq!(
            h[0],
            "<b>Hokkaido</b> (JP01)<br>count: 10<br>10.0% of total<br>rank 3 of 3"
        );
        // empty name -> bold id fallback; 30/100 = 30%, rank 2
        assert_eq!(
            h[1],
            "<b>JP02</b><br>count: 30<br>30.0% of total<br>rank 2 of 3"
        );
        // highest z -> rank 1
        assert_eq!(
            h[2],
            "<b>Okinawa</b> (JP03)<br>count: 60<br>60.0% of total<br>rank 1 of 3"
        );
    }

    #[test]
    fn choropleth_hover_text_no_pct_when_not_count_or_sum() {
        // asserts the localized "rank N of N" line and the ABSENCE of the
        // localized "% of total" line -- both need English pinned.
        let _locale = english_locale();
        let locs = vec!["A".to_string(), "B".to_string()];
        let z = vec![3.5, 1.5];
        // include_pct = false (mean/min/max): no "% of total" line, value still labeled + ranked
        let h = choropleth_hover_text(&locs, &z, None, "magnitude", false);
        assert_eq!(h[0], "<b>A</b><br>magnitude: 3.5<br>rank 1 of 2");
        assert_eq!(h[1], "<b>B</b><br>magnitude: 1.5<br>rank 2 of 2");
        assert!(!h[0].contains("% of total"));
    }

    #[test]
    fn match_region_code_pads_and_matches() {
        // ascending, deduped digit-widths of the all-numeric ids, as the caller precomputes.
        fn numeric_widths(ids: &std::collections::HashSet<&str>) -> Vec<usize> {
            let mut w: Vec<usize> = ids
                .iter()
                .filter(|id| !id.is_empty() && id.bytes().all(|b| b.is_ascii_digit()))
                .map(|id| id.len())
                .collect();
            w.sort_unstable();
            w.dedup();
            w
        }

        let ids: std::collections::HashSet<&str> = ["07936", "15129", "15007", "BAKERSTOWN"]
            .into_iter()
            .collect();
        let widths = numeric_widths(&ids);
        let lower_ids: std::collections::HashMap<String, String> = ids
            .iter()
            .map(|i| ((*i).to_ascii_lowercase(), (*i).to_string()))
            .collect();

        // exact 5-digit match, with surrounding whitespace trimmed
        assert_eq!(
            match_region_code(" 15129 ", &ids, &widths, &lower_ids).as_deref(),
            Some("15129")
        );
        // short all-digit code left-padded to 5 to match a zero-padded id
        assert_eq!(
            match_region_code("7936", &ids, &widths, &lower_ids).as_deref(),
            Some("07936")
        );
        // non-numeric code matched verbatim
        assert_eq!(
            match_region_code("BAKERSTOWN", &ids, &widths, &lower_ids).as_deref(),
            Some("BAKERSTOWN")
        );
        // empty / whitespace-only -> no match
        assert_eq!(match_region_code("   ", &ids, &widths, &lower_ids), None);
        // a value absent from the boundary set -> no match (counts against overlap ratio)
        assert_eq!(match_region_code("99999", &ids, &widths, &lower_ids), None);
        // a short code whose padded form is absent -> no match (not blindly padded-and-kept)
        assert_eq!(match_region_code("123", &ids, &widths, &lower_ids), None);

        // non-zip widths: the code is padded to whatever numeric feature-id widths exist, not a
        // hardcoded 5. State FIPS (2-wide) and county FIPS (3-wide) both resolve.
        let fips: std::collections::HashSet<&str> =
            ["01", "06", "003", "037", "42003"].into_iter().collect();
        let fips_widths = numeric_widths(&fips);
        let lower_fips: std::collections::HashMap<String, String> = fips
            .iter()
            .map(|i| ((*i).to_ascii_lowercase(), (*i).to_string()))
            .collect();
        // state FIPS 1 -> 01 (2-wide id)
        assert_eq!(
            match_region_code("1", &fips, &fips_widths, &lower_fips).as_deref(),
            Some("01")
        );
        // county FIPS 3 -> 003 (3-wide id), preferring the shortest viable width first
        assert_eq!(
            match_region_code("3", &fips, &fips_widths, &lower_fips).as_deref(),
            Some("003")
        );
        // already-wide numeric id matches exactly
        assert_eq!(
            match_region_code("42003", &fips, &fips_widths, &lower_fips).as_deref(),
            Some("42003")
        );
        // a width with no matching padded form -> no match
        assert_eq!(
            match_region_code("9", &fips, &fips_widths, &lower_fips),
            None
        );
    }

    #[test]
    fn parse_f64_rejects_non_finite() {
        // finite values parse, with surrounding whitespace trimmed
        assert_eq!(parse_f64(Some(b"3.5")), Some(3.5));
        assert_eq!(parse_f64(Some(b"  -2.0 ")), Some(-2.0));
        // Rust's f64::from_str accepts these; viz must reject them so they can't poison min/max
        // folds (infinite axis spans, ±inf aggregate bars)
        for bad in ["inf", "-inf", "+inf", "infinity", "Infinity", "NaN", "nan"] {
            assert_eq!(
                parse_f64(Some(bad.as_bytes())),
                None,
                "should reject {bad:?}"
            );
        }
        // empty / non-numeric still None
        assert_eq!(parse_f64(Some(b"")), None);
        assert_eq!(parse_f64(Some(b"abc")), None);
    }

    #[test]
    fn partition_geo_outliers_ignores_antimeridian_wrap() {
        // a single cluster straddling the ±180° antimeridian: 5 points just east of +179°, 3 just
        // west of -179°. They are all physically within ~3° of each other, so none is a geographic
        // outlier. Before the wrap fix, the minority lobe measured ~358° away and was mis-flagged.
        let lats = [10.0, 10.1, 9.9, 10.2, 9.8, 10.0, 10.1, 9.9];
        let lons = [179.0, 179.2, 179.4, 179.6, 179.8, -179.8, -179.6, -179.4];
        let (core_lats, _core_lons, out_lats, out_lons) = partition_geo_outliers(&lats, &lons);
        assert!(
            out_lats.is_empty() && out_lons.is_empty(),
            "dateline-straddling cluster must yield zero geographic outliers, got {out_lons:?}"
        );
        assert_eq!(core_lats.len(), lats.len());
    }

    #[test]
    fn partition_geo_outliers_centroid_ignores_stray_across_seam() {
        // a dateline cluster split across +179°/-179° plus ONE stray near 0°. A plain median lon
        // picks the stray (0°) as the centroid, making the real cluster read ~180° away and the
        // stray look central -> the cluster would be flagged and the stray kept. The circular
        // centroid sits in the cluster, so only the stray is flagged.
        let lats = [10.0, 10.1, 9.9, 10.2, 9.8, 10.1, 9.9, 10.0, 10.0];
        let lons = [
            179.0, 179.2, 179.4, 179.6, -179.6, -179.4, -179.2, -179.0, 0.5,
        ];
        let (core_lats, _core_lons, _out_lats, out_lons) = partition_geo_outliers(&lats, &lons);
        assert_eq!(
            out_lons.len(),
            1,
            "exactly the lone mid-ocean stray should be flagged, got {out_lons:?}"
        );
        assert!(
            out_lons[0].abs() < 1.0,
            "the flagged outlier should be the ~0° stray, got {}",
            out_lons[0]
        );
        assert_eq!(core_lats.len(), 8);
    }

    #[test]
    fn pip_assign_handles_antimeridian_crossing_feature() {
        // a Polygon spanning lon 170° .. -170° (crossing +180°), lat 0° .. 10°, in raw unsplit
        // GeoJSON coordinates (a 170->-170 edge jumps 340° => detected as antimeridian-crossing).
        let geojson = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "id": "PACIFIC",
                "properties": {"name": "Pacific"},
                "geometry": {
                    "type": "Polygon",
                    "coordinates": [[
                        [170.0, 0.0], [-170.0, 0.0], [-170.0, 10.0], [170.0, 10.0], [170.0, 0.0]
                    ]]
                }
            }]
        });
        let features = build_pip_features(&geojson, "id", None).unwrap();
        assert!(
            features[0].wraps_antimeridian,
            "feature crossing ±180° must be flagged as wrapping"
        );
        // points on BOTH sides of the seam fall inside; a point far away (Greenwich) does not
        assert_eq!(
            pip_assign(&features, 5.0, 175.0, false, f64::INFINITY),
            PipOutcome::Inside(0)
        );
        assert_eq!(
            pip_assign(&features, 5.0, -175.0, false, f64::INFINITY),
            PipOutcome::Inside(0)
        );
        assert_eq!(
            pip_assign(&features, 5.0, 0.0, false, f64::INFINITY),
            PipOutcome::Outside
        );
        // snap path: just north of the box (lat 0..10) on the negative-lon side — outside every
        // polygon but ~555 km from the top edge, within the 2000 km cap -> Snapped. Exercises
        // candidate_lons in the distance loop (#4092 snap code), not just the containment path.
        assert_eq!(
            pip_assign(&features, 15.0, -175.0, true, 2_000.0),
            PipOutcome::Snapped(0)
        );
    }

    #[test]
    fn pip_assign_wide_polygon_is_not_treated_as_crossing() {
        // a coarse rectangle spanning lon -100°..100°, lat 0°..10°: its horizontal edges span 200°
        // but are centered on the prime meridian, NOT the antimeridian. It must NOT be normalized,
        // so a Greenwich point (lon 0°) inside the rectangle still bins inside it.
        let geojson = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "id": "WIDE",
                "geometry": {
                    "type": "Polygon",
                    "coordinates": [[
                        [-100.0, 0.0], [100.0, 0.0], [100.0, 10.0], [-100.0, 10.0], [-100.0, 0.0]
                    ]]
                }
            }]
        });
        let features = build_pip_features(&geojson, "id", None).unwrap();
        assert!(
            !features[0].wraps_antimeridian,
            "a prime-meridian-centered wide polygon must not be flagged as antimeridian-crossing"
        );
        // Greenwich (lon 0°) is inside the -100..100 rectangle
        assert_eq!(
            pip_assign(&features, 5.0, 0.0, false, f64::INFINITY),
            PipOutcome::Inside(0)
        );
        // a point well east of +100° is outside
        assert_eq!(
            pip_assign(&features, 5.0, 150.0, false, f64::INFINITY),
            PipOutcome::Outside
        );
    }

    #[test]
    fn pip_assign_handles_wide_antimeridian_crossing_feature() {
        // a WIDER Pacific box spanning lon 160° .. -160° (crossing +180°), lat 0°..10°. Its
        // endpoints are 20° off the seam, but only 40° apart across it, so it IS a dateline
        // crossing and must be normalized — unlike the -100..100 case (160° across the seam).
        let geojson = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "id": "PACIFIC",
                "geometry": {
                    "type": "Polygon",
                    "coordinates": [[
                        [160.0, 0.0], [-160.0, 0.0], [-160.0, 10.0], [160.0, 10.0], [160.0, 0.0]
                    ]]
                }
            }]
        });
        let features = build_pip_features(&geojson, "id", None).unwrap();
        assert!(
            features[0].wraps_antimeridian,
            "a 160°->-160° box (40° across the seam) must be flagged as antimeridian-crossing"
        );
        // points on both sides of the seam are inside; Greenwich is not
        assert_eq!(
            pip_assign(&features, 5.0, 175.0, false, f64::INFINITY),
            PipOutcome::Inside(0)
        );
        assert_eq!(
            pip_assign(&features, 5.0, -170.0, false, f64::INFINITY),
            PipOutcome::Inside(0)
        );
        assert_eq!(
            pip_assign(&features, 5.0, 0.0, false, f64::INFINITY),
            PipOutcome::Outside
        );
    }

    #[test]
    fn pip_assign_handles_exact_90deg_seam_gap_feature() {
        // boundary case: a box spanning lon 135° .. -135° has EXACTLY a 90° short-way seam gap,
        // matching the inclusive "within 90°" rule, so it must be treated as crossing (the cutoff
        // is >=, not >). A point near the seam bins inside; Greenwich does not.
        let geojson = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "id": "PACIFIC",
                "geometry": {
                    "type": "Polygon",
                    "coordinates": [[
                        [135.0, 0.0], [-135.0, 0.0], [-135.0, 10.0], [135.0, 10.0], [135.0, 0.0]
                    ]]
                }
            }]
        });
        let features = build_pip_features(&geojson, "id", None).unwrap();
        assert!(
            features[0].wraps_antimeridian,
            "a 135°->-135° box (exactly 90° across the seam) must be flagged as crossing"
        );
        assert_eq!(
            pip_assign(&features, 5.0, 180.0, false, f64::INFINITY),
            PipOutcome::Inside(0)
        );
        assert_eq!(
            pip_assign(&features, 5.0, -150.0, false, f64::INFINITY),
            PipOutcome::Inside(0)
        );
        assert_eq!(
            pip_assign(&features, 5.0, 0.0, false, f64::INFINITY),
            PipOutcome::Outside
        );
    }

    #[test]
    fn pip_assign_pole_cut_antarctica_class_keeps_raw_frame() {
        // Natural-Earth-style Antarctica: a pole-enclosing polygon pre-CUT along the seam — its
        // ring runs along both ±180° meridians and closes along lat -90, so it contains a
        // |Δlon| = 360 edge that flags it as "crossing". Continuity-unwrapping can't close such a
        // ring (±360 residue), so it must be kept RAW: its planar form is already correct, and
        // the old shift-all-negative-lons normalization corrupted ray-cast parity for points
        // near the prime meridian (0.4° -> -0.2° edges became fake ~359°-long edges).
        let geojson = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "id": "ANT",
                "geometry": {
                    "type": "Polygon",
                    "coordinates": [[
                        [-180.0, -60.0], [-90.0, -65.0], [-0.2, -60.0], [0.4, -60.0],
                        [90.0, -65.0], [180.0, -60.0], [180.0, -90.0], [-180.0, -90.0],
                        [-180.0, -60.0]
                    ]]
                }
            }]
        });
        let features = build_pip_features(&geojson, "id", None).unwrap();
        assert!(
            !features[0].wraps_antimeridian,
            "a pole-cut ring must revert to its raw planar frame, not unwrap"
        );
        // coastal points on BOTH sides of the prime meridian bin inside
        assert_eq!(
            pip_assign(&features, -75.0, 0.2, false, f64::INFINITY),
            PipOutcome::Inside(0)
        );
        assert_eq!(
            pip_assign(&features, -75.0, -0.1, false, f64::INFINITY),
            PipOutcome::Inside(0)
        );
        // and near the seam
        assert_eq!(
            pip_assign(&features, -80.0, 179.0, false, f64::INFINITY),
            PipOutcome::Inside(0)
        );
        // north of the coast is outside
        assert_eq!(
            pip_assign(&features, -50.0, 0.3, false, f64::INFINITY),
            PipOutcome::Outside
        );
    }

    #[test]
    fn pip_assign_feature_crossing_both_meridians_unwraps_contiguously() {
        // a (non-polar) band spanning 90°E across the antimeridian and onward across the prime
        // meridian to 5°E: continuity unwrapping keeps BOTH crossings contiguous (the old
        // shift-negatives normalization broke the -10° -> 5° prime-meridian edge into a fake
        // ~345°-long edge and left the feature's interior near 0° unreachable by frame_lon).
        let geojson = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "id": "BAND",
                "geometry": {
                    "type": "Polygon",
                    "coordinates": [[
                        [90.0, 0.0], [170.0, 0.0], [-170.0, 0.0], [-90.0, 0.0],
                        [-10.0, 0.0], [5.0, 0.0], [5.0, 10.0], [-10.0, 10.0],
                        [-90.0, 10.0], [-170.0, 10.0], [170.0, 10.0], [90.0, 10.0],
                        [90.0, 0.0]
                    ]]
                }
            }]
        });
        let features = build_pip_features(&geojson, "id", None).unwrap();
        assert!(features[0].wraps_antimeridian);
        // interior points: east of the seam, west of the seam, and past the prime meridian
        for lon in [100.0, 179.0, -179.0, -30.0, -0.1, 4.0] {
            assert_eq!(
                pip_assign(&features, 5.0, lon, false, f64::INFINITY),
                PipOutcome::Inside(0),
                "lon {lon} must be inside the band"
            );
        }
        // outside the band's longitude range
        assert_eq!(
            pip_assign(&features, 5.0, 40.0, false, f64::INFINITY),
            PipOutcome::Outside
        );
    }

    #[test]
    fn geojson_rings_to_closed_drops_degenerate_rings() {
        // documented contract: rings with fewer than 3 DISTINCT vertices are dropped. A valid
        // (open) triangle is kept and auto-closed; a 2-distinct ring ([[0,0],[1,1],[0,0]]) and a
        // 1-distinct point ring are degenerate (zero area) and dropped.
        let p = |x: f64, y: f64| -> geojson::Position { vec![x, y].into() };
        let poly: Vec<Vec<geojson::Position>> = vec![
            vec![p(0.0, 0.0), p(1.0, 0.0), p(0.0, 1.0)], // 3 distinct -> kept
            vec![p(5.0, 5.0), p(6.0, 6.0), p(5.0, 5.0)], // 2 distinct -> dropped
            vec![p(9.0, 9.0), p(9.0, 9.0), p(9.0, 9.0)], // 1 distinct -> dropped
        ];
        let rings = geojson_rings_to_closed(&poly);
        assert_eq!(rings.len(), 1, "only the non-degenerate ring survives");
        // the kept ring is closed (first == last) so windows(2) enumerates every edge
        assert_eq!(rings[0].first(), rings[0].last());
        assert_eq!(rings[0].len(), 4); // 3 vertices + the appended closing vertex
    }

    #[test]
    fn sanitize_geojson_reorders_malformed_exterior_ring() {
        // a real-world malformation (seen in an NYC-neighborhoods GeoJSON): a polygon whose first
        // ring — the GeoJSON "exterior" — is a zero-area sliver (2 distinct vertices), with the
        // true boundary listed AFTER it as a "hole". Plotly assumes ring[0] is the exterior
        // and throws on this, blanking the whole choropleth panel. The sanitizer must drop
        // the degenerate ring and promote the largest-area ring to ring[0].
        let big = serde_json::json!([
            [0.0, 0.0],
            [10.0, 0.0],
            [10.0, 10.0],
            [0.0, 10.0],
            [0.0, 0.0]
        ]);
        let small = serde_json::json!([[2.0, 2.0], [4.0, 2.0], [4.0, 4.0], [2.0, 4.0], [2.0, 2.0]]);
        // sliver: 2 distinct vertices (a, b, a, a) — zero area, degenerate.
        let sliver = serde_json::json!([[1.0, 1.0], [1.5, 1.5], [1.0, 1.0], [1.0, 1.0]]);
        let mut geojson = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "id": "bad",
                "properties": {"name": "bad"},
                // malformed order: sliver(exterior), small, big
                "geometry": {"type": "MultiPolygon", "coordinates": [[sliver, small, big]]}
            }]
        });
        sanitize_geojson_for_render(&mut geojson);

        let rings = &geojson["features"][0]["geometry"]["coordinates"][0];
        let rings = rings.as_array().expect("polygon rings array");
        // the degenerate sliver is dropped; the two real rings survive.
        assert_eq!(rings.len(), 2, "degenerate sliver ring dropped");
        // ring[0] is now the largest-area ring (the true exterior), not the inner small ring.
        assert_eq!(
            ring_signed_area(&rings[0]).abs(),
            100.0,
            "largest ring promoted to exterior"
        );
        assert_eq!(
            ring_signed_area(&rings[1]).abs(),
            4.0,
            "smaller ring becomes the hole"
        );
    }

    #[test]
    fn sanitize_geojson_leaves_conforming_polygon_untouched() {
        // a well-formed polygon (exterior first and largest, hole strictly inside) keeps its ring
        // COUNT and ORDER: no ring is dropped, reordered, or rebuilt. Only winding is normalized
        // for d3-geo (see repair_polygon_rings), which reverses each ring's vertex sequence in
        // place — so identity is asserted as "same vertices", modulo direction.
        let exterior = serde_json::json!([
            [0.0, 0.0],
            [10.0, 0.0],
            [10.0, 10.0],
            [0.0, 10.0],
            [0.0, 0.0]
        ]);
        let hole = serde_json::json!([[2.0, 2.0], [4.0, 2.0], [4.0, 4.0], [2.0, 4.0], [2.0, 2.0]]);
        let mut geojson = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "id": "ok",
                "geometry": {"type": "Polygon", "coordinates": [exterior.clone(), hole.clone()]}
            }]
        });
        sanitize_geojson_for_render(&mut geojson);
        let rings = geojson["features"][0]["geometry"]["coordinates"]
            .as_array()
            .expect("polygon rings array");
        assert_eq!(rings.len(), 2);
        // same vertex sets, still exterior-then-hole (a reversed ring compares equal once one side
        // is flipped, so this catches any drop/reorder/rebuild while tolerating the winding fix).
        let same_ring = |got: &serde_json::Value, want: &serde_json::Value| {
            let g = got.as_array().expect("ring array");
            let w = want.as_array().expect("ring array");
            g == w || g.iter().rev().eq(w.iter())
        };
        assert!(
            same_ring(&rings[0], &exterior),
            "exterior kept its vertices and is still first"
        );
        assert!(
            same_ring(&rings[1], &hole),
            "hole kept its vertices and is still second"
        );
        // ...and the winding is what d3-geo needs.
        assert!(ring_signed_area(&rings[0]) < 0.0, "exterior clockwise");
        assert!(ring_signed_area(&rings[1]) > 0.0, "hole counter-clockwise");
    }

    #[test]
    fn load_geojson_invokes_sanitizer() {
        // wiring guard: this bug is invisible at qsv runtime (the binary exits 0 and writes
        // valid-looking HTML; the crash only happens later in the browser). The unit tests above
        // cover the sanitizer LOGIC, but nothing else asserts that `load_geojson` actually CALLS it
        // — drop that one line in a refactor and every other test still passes while the bug
        // returns. So load a malformed-ring file through the real entry point and assert it
        // comes back fixed.
        let dir = std::env::temp_dir();
        let path = dir.join("qsv_viz_load_geojson_sanitize_wiring.geojson");
        let malformed = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "id": "bad",
                "geometry": {"type": "MultiPolygon", "coordinates": [[
                    // degenerate sliver listed first (the bogus "exterior") ...
                    [[1.0, 1.0], [1.5, 1.5], [1.0, 1.0], [1.0, 1.0]],
                    // ... then the real, larger boundary listed as a "hole"
                    [[0.0, 0.0], [10.0, 0.0], [10.0, 10.0], [0.0, 10.0], [0.0, 0.0]]
                ]]}
            }]
        });
        std::fs::write(&path, serde_json::to_vec(&malformed).unwrap()).unwrap();

        let loaded = load_geojson(path.to_str().unwrap()).expect("load_geojson");
        let _ = std::fs::remove_file(&path);

        let rings = loaded["features"][0]["geometry"]["coordinates"][0]
            .as_array()
            .expect("polygon rings array");
        assert_eq!(
            rings.len(),
            1,
            "load_geojson must drop the degenerate sliver ring"
        );
        assert_eq!(
            ring_signed_area(&rings[0]).abs(),
            100.0,
            "load_geojson must promote the real boundary to the exterior"
        );
    }

    #[test]
    fn sanitize_geojson_drops_collinear_zero_area_rings() {
        // a ring can have 3+ DISTINCT vertices yet zero area when they are collinear
        // ([[0,0],[1,1],[2,2]] closed). The distinct-vertex count alone does NOT catch it, but it
        // is just as non-renderable: as a sub-polygon's only ring it leaves plotly's
        // MultiPolygon centroid with no positive-area sub-polygon and reproduces the
        // blank-panel crash. So it must be dropped, and a feature left with no renderable
        // polygon must be dropped too.
        let collinear = serde_json::json!([[0.0, 0.0], [1.0, 1.0], [2.0, 2.0], [0.0, 0.0]]);
        let real = serde_json::json!([
            [0.0, 0.0],
            [10.0, 0.0],
            [10.0, 10.0],
            [0.0, 10.0],
            [0.0, 0.0]
        ]);
        let collinear_hole = serde_json::json!([[3.0, 3.0], [4.0, 4.0], [5.0, 5.0], [3.0, 3.0]]);
        let mut geojson = serde_json::json!({
            "type": "FeatureCollection",
            "features": [
                // a MultiPolygon whose only ring is collinear -> no renderable polygon -> dropped.
                {"type": "Feature", "id": "collinear_only",
                 "geometry": {"type": "MultiPolygon", "coordinates": [[collinear]]}},
                // a valid polygon carrying a collinear (zero-area) hole -> hole stripped, kept.
                {"type": "Feature", "id": "valid",
                 "geometry": {"type": "Polygon", "coordinates": [real, collinear_hole]}}
            ]
        });
        sanitize_geojson_for_render(&mut geojson);

        let feats = geojson["features"].as_array().expect("features array");
        assert_eq!(
            feats.len(),
            1,
            "the collinear-only feature is dropped entirely"
        );
        assert_eq!(feats[0]["id"], "valid", "the renderable feature survives");
        let rings = feats[0]["geometry"]["coordinates"]
            .as_array()
            .expect("polygon rings array");
        assert_eq!(rings.len(), 1, "the collinear zero-area hole is stripped");
        assert_eq!(
            ring_signed_area(&rings[0]).abs(),
            100.0,
            "the real exterior is kept"
        );
    }

    #[test]
    fn repair_polygon_rings_normalizes_winding_for_d3_geo() {
        // plotly's `choropleth` projects with d3-geo, which needs a CLOCKWISE exterior — the
        // OPPOSITE of RFC 7946. So spec-conforming (CCW) input must come back reversed, or the
        // fill floods the whole map and the regions punch out of it. Measured on a donut fixture.
        let ring = |pts: &[[f64; 2]]| serde_json::json!(pts);
        // RFC 7946 form: CCW exterior, CW hole.
        let ccw_exterior = ring(&[
            [0.0, 0.0],
            [10.0, 0.0],
            [10.0, 10.0],
            [0.0, 10.0],
            [0.0, 0.0],
        ]);
        let cw_hole = ring(&[[2.0, 2.0], [2.0, 8.0], [8.0, 8.0], [8.0, 2.0], [2.0, 2.0]]);
        assert!(
            ring_signed_area(&ccw_exterior) > 0.0,
            "fixture: CCW exterior"
        );
        assert!(ring_signed_area(&cw_hole) < 0.0, "fixture: CW hole");

        let fixed = repair_polygon_rings(vec![ccw_exterior.clone(), cw_hole.clone()])
            .expect("polygon survives");
        assert_eq!(fixed.len(), 2, "both rings kept");
        assert!(
            ring_signed_area(&fixed[0]) < 0.0,
            "exterior must be normalized to CLOCKWISE"
        );
        assert!(
            ring_signed_area(&fixed[1]) > 0.0,
            "hole must be normalized to COUNTER-CLOCKWISE"
        );
        // reversal must MOVE coordinates, never rebuild them: same vertex set, reversed order.
        let orig: Vec<_> = ccw_exterior.as_array().unwrap().iter().rev().collect();
        let got: Vec<_> = fixed[0].as_array().unwrap().iter().collect();
        assert_eq!(
            orig, got,
            "exterior is the input ring reversed, vertex-wise"
        );

        // already-normalized input is a no-op (byte-identical), so conforming files never churn.
        let already = repair_polygon_rings(fixed.clone()).expect("polygon survives");
        assert_eq!(already, fixed, "normalizing twice changes nothing");
    }

    #[test]
    fn load_geojson_normalizes_winding() {
        // wiring guard, same rationale as `load_geojson_invokes_sanitizer`: an inverted fill is
        // invisible at qsv runtime (exit 0, valid-looking HTML) and only shows up in the browser.
        let path = std::env::temp_dir().join("qsv_viz_load_geojson_winding.geojson");
        let rfc_conforming = serde_json::json!({
            "type": "FeatureCollection",
            "features": [{
                "type": "Feature",
                "properties": {"name": "ccw"},
                // RFC 7946 §3.1.6-conforming counter-clockwise exterior
                "geometry": {"type": "Polygon", "coordinates": [[
                    [0.0, 0.0], [10.0, 0.0], [10.0, 10.0], [0.0, 10.0], [0.0, 0.0]
                ]]}
            }]
        });
        std::fs::write(&path, serde_json::to_vec(&rfc_conforming).unwrap()).unwrap();
        let loaded = load_geojson(path.to_str().unwrap()).expect("load_geojson");
        let _ = std::fs::remove_file(&path);

        let rings = loaded["features"][0]["geometry"]["coordinates"]
            .as_array()
            .expect("polygon rings array");
        assert!(
            ring_signed_area(&rings[0]) < 0.0,
            "load_geojson must hand d3-geo a CLOCKWISE exterior"
        );
    }

    #[test]
    fn lollipop_value_range_zooms_near_tied_cluster() {
        // near-ceiling cluster (the common bivariate case): the axis must NOT start at 0 — it
        // should zoom in so the dots separate, while staying below the smallest value.
        let (floor, ceil) = lollipop_value_range(&[0.9949, 0.999, 1.0]);
        assert!(
            floor > 0.9,
            "floor should zoom in near the data, got {floor}"
        );
        assert!(
            floor < 0.9949,
            "floor must sit below the smallest value, got {floor}"
        );
        assert!(ceil <= TOPREL_CEIL_CAP, "ceil must be capped, got {ceil}");
        assert!(ceil >= 1.0, "ceil must clear the max value, got {ceil}");
    }

    #[test]
    fn lollipop_value_range_guards_all_tied() {
        // an all-EXACTLY-tied top-N (span 0) must still yield a finite, non-degenerate, in-bounds
        // range (identical values legitimately overlap; the axis must not render zero-width).
        let (floor, ceil) = lollipop_value_range(&[1.0, 1.0, 1.0]);
        assert!(floor >= 0.0 && ceil <= TOPREL_CEIL_CAP);
        assert!(
            ceil - floor >= TOPREL_MIN_SPAN - f64::EPSILON,
            "span {} must be at least TOPREL_MIN_SPAN",
            ceil - floor
        );
        assert!(ceil >= 1.0, "the tied value must be inside the range");
    }

    #[test]
    fn lollipop_value_range_wide_spread() {
        // a genuinely spread top-N pads proportionally around the data and never dips below 0.
        let (floor, ceil) = lollipop_value_range(&[0.3, 0.6, 0.9]);
        assert!(floor >= 0.0);
        assert!(floor < 0.3, "floor pads below the min, got {floor}");
        assert!(
            ceil > 0.9 && ceil <= TOPREL_CEIL_CAP,
            "ceil pads above the max, got {ceil}"
        );
    }

    #[test]
    fn bivariate_sidecar_fresh_when_path_cleared_and_written() {
        // pre-delete succeeded (cleared) and a file exists now => moarstats wrote it this run.
        let t0 = std::time::UNIX_EPOCH;
        assert!(bivariate_sidecar_is_fresh(Some(t0), Some(t0), true));
        // even with no pre-existing file, a present-after file with cleared path is fresh.
        assert!(bivariate_sidecar_is_fresh(None, Some(t0), true));
    }

    #[test]
    fn bivariate_sidecar_not_fresh_when_absent_after_run() {
        // moarstats produced no pairs (no sidecar written); nothing to read.
        let t0 = std::time::UNIX_EPOCH;
        assert!(!bivariate_sidecar_is_fresh(Some(t0), None, true));
        assert!(!bivariate_sidecar_is_fresh(None, None, false));
    }

    #[test]
    fn bivariate_sidecar_stale_when_delete_failed_and_mtime_unchanged() {
        // the pre-delete FAILED (not cleared) and moarstats did not rewrite the sidecar, so its
        // mtime is unchanged from the pre-run snapshot: this is the stale copy we must NOT trust.
        let t0 = std::time::UNIX_EPOCH;
        assert!(!bivariate_sidecar_is_fresh(Some(t0), Some(t0), false));
    }

    #[test]
    fn bivariate_sidecar_fresh_when_delete_failed_but_mtime_advanced() {
        // the pre-delete failed but moarstats still overwrote the sidecar in place: the advanced
        // mtime proves the current run wrote it, so it is fresh.
        let before = std::time::UNIX_EPOCH;
        let after = before + std::time::Duration::from_secs(1);
        assert!(bivariate_sidecar_is_fresh(Some(before), Some(after), false));
    }

    // A convex polygon (square): the area-weighted centroid is inside, so it IS the label anchor.
    #[test]
    fn geojson_label_anchor_convex_is_centroid() {
        let square = vec![vec![
            [0.0, 0.0],
            [0.0, 10.0],
            [10.0, 10.0],
            [10.0, 0.0],
            [0.0, 0.0],
        ]];
        let pt = representative_point(&square).expect("anchor");
        assert!((pt[0] - 5.0).abs() < 1e-9 && (pt[1] - 5.0).abs() < 1e-9);
        assert!(point_in_polygon(&square, pt[0], pt[1]));
    }

    // A concave U-shaped polygon: the bbox center AND the naive centroid fall in the notch
    // (outside), so the anchor must be pulled onto the surface (guaranteed inside).
    #[test]
    fn geojson_label_anchor_concave_stays_inside() {
        // a "U": full width at the bottom, two prongs up the sides, hollow center-top.
        let u = vec![vec![
            [0.0, 0.0],
            [10.0, 0.0],
            [10.0, 10.0],
            [7.0, 10.0],
            [7.0, 3.0],
            [3.0, 3.0],
            [3.0, 10.0],
            [0.0, 10.0],
            [0.0, 0.0],
        ]];
        // sanity: the whole-bbox center (5,5) is in the hollow notch => outside
        assert!(!point_in_polygon(&u, 5.0, 5.0));
        let pt = representative_point(&u).expect("anchor");
        assert!(
            point_in_polygon(&u, pt[0], pt[1]),
            "anchor {pt:?} must be inside the U"
        );
    }

    // A MultiPolygon (a tiny part + a large part with a wide gap between them): the anchor must
    // land inside the LARGE part, never in the empty gap that a whole-feature bbox center would
    // pick.
    #[test]
    fn geojson_label_anchor_multipolygon_picks_largest_part() {
        let small = vec![vec![
            [0.0, 0.0],
            [0.0, 1.0],
            [1.0, 1.0],
            [1.0, 0.0],
            [0.0, 0.0],
        ]];
        let large = vec![vec![
            [50.0, 50.0],
            [50.0, 60.0],
            [60.0, 60.0],
            [60.0, 50.0],
            [50.0, 50.0],
        ]];
        let feature = PipFeature {
            id:                 "m".to_string(),
            name:               Some("Multi".to_string()),
            polygons:           vec![small, large.clone()],
            bbox:               [0.0, 0.0, 60.0, 60.0],
            wraps_antimeridian: false,
        };
        let [lon, lat] = feature_label_anchor(&feature);
        // inside the large square, not in the (0,0)-(60,60) union-bbox gap (~30,30)
        assert!(
            point_in_polygon(&large, lon, lat),
            "anchor ({lon},{lat}) not in large part"
        );
    }

    #[test]
    fn localized_js_placeholders_are_all_substituted() {
        // A leaked `__QSVI18N_*__` renders as literal garbage in the page rather than
        // failing a build, so the only guard is a test. This one covers every JS-const
        // block that carries placeholders.
        for (name, rendered) in [
            ("FULLSCREEN_SCRIPT", fullscreen_script()),
            ("MAP_SELECT_CHROME", map_select_chrome(3, 4)),
            ("PHOTO_CHROME", photo_chrome()),
            ("DICT_SCRIPT_TEMPLATE", dict_script_template()),
            ("DATA_DRAWER_SCRIPT", data_drawer_script()),
        ] {
            assert!(
                !rendered.contains("__QSVI18N"),
                "an i18n placeholder survived substitution into {name}"
            );
        }
        // the coordinate columns ride the same substitution machinery and leak the same way
        let chrome = map_select_chrome(3, 4);
        assert!(
            !chrome.contains("__QSV_LAT_COL__") && !chrome.contains("__QSV_LON_COL__"),
            "a coordinate-column placeholder survived substitution into MAP_SELECT_CHROME"
        );
        assert!(chrome.contains("var LAT_COL = 3, LON_COL = 4;"));
    }

    #[test]
    fn every_t_key_used_in_this_file_exists_in_the_catalog() {
        // rust-i18n renders the KEY ITSELF when a key is missing -- no panic, no warning, no
        // compile error. A real instance: three keys were appended to the wrong YAML section, so
        // the dictionary page shipped the literal text "viz.dict.back_to_dashboard" to users.
        //
        // Key-set parity between en.yml and es.yml cannot catch this (both locales agreed, and
        // both were wrong). The only reliable check is call-site-driven: scan this file for the
        // keys actually passed to `t!` and confirm each resolves to something other than itself.
        //
        // The scan must tolerate rustfmt putting the key on its own line -- an earlier version
        // keyed on the literal `t!("` and silently skipped all three multi-line calls, one of
        // which was the very key whose omission motivated this test. So rather than trusting a
        // count floor, this asserts FULL coverage: every `t!(` site in the file must present a
        // string-literal first argument, and every such literal must resolve.
        let _locale = english_locale();

        // Scan PRODUCTION code only. This module's own source contains a `t!` call with a
        // runtime key (the resolution check below) and the literal "t!(" the scanner searches
        // for, both of which would register as unverifiable sites.
        let whole = include_str!("viz.rs");
        // The CRLF arm is not hypothetical: `.gitattributes` declares `* text=auto`, so a Windows
        // checkout hands include_str! a file whose newlines are \r\n. The LF-only anchor then
        // fails to match, the scan does NOT stop at the test module, and it picks up this
        // module's own deliberately-runtime `t!` keys -- reporting them as unverifiable sites and
        // failing the assertion below on Windows only.
        let src = whole
            .split_once("\n#[cfg(test)]\nmod tests {")
            .or_else(|| whole.split_once("\r\n#[cfg(test)]\r\nmod tests {"))
            .map_or(whole, |(before, _)| before);
        let bytes = src.as_bytes();
        let (mut checked, mut sites) = (0usize, 0usize);
        let mut missing: Vec<&str> = Vec::new();
        let mut non_literal: Vec<usize> = Vec::new();

        for (idx, _) in src.match_indices("t!(") {
            // skip `assert!(`, `debug_assert!(` and friends, which contain `t!(` as a substring
            if idx > 0 && matches!(bytes[idx - 1], b'a'..=b'z' | b'A'..=b'Z' | b'0'..=b'9' | b'_') {
                continue;
            }
            sites += 1;
            let rest = src[idx + 3..].trim_start();
            let Some(after_quote) = rest.strip_prefix('"') else {
                // a non-literal first argument cannot be verified statically; there are none
                // today, and the assertion below makes adding one a deliberate decision.
                non_literal.push(idx);
                continue;
            };
            let Some(key) = after_quote.split('"').next() else {
                continue;
            };
            checked += 1;
            if rust_i18n::t!(key) == key {
                missing.push(key);
            }
        }

        assert!(
            non_literal.is_empty(),
            "{} `t!` call(s) use a non-literal key and are therefore unverifiable here; if that \
             is intentional, exempt them explicitly rather than letting coverage drop silently",
            non_literal.len()
        );
        assert_eq!(
            checked, sites,
            "every `t!` site should have yielded a literal key to check"
        );
        assert!(
            sites > 60,
            "expected many `t!` sites by scanning the source, found {sites} — the scan probably \
             broke rather than the catalog shrinking"
        );
        assert!(
            missing.is_empty(),
            "these t! keys are missing from the catalog and would render as their own key name in \
             the page: {missing:?}"
        );
    }

    #[test]
    fn english_emits_no_localization_artifacts() {
        let _locale = english_locale();
        // The single most important property of the whole feature: an English page must be
        // byte-identical to one built before viz learned to localize. Both vendored-asset
        // injections are empty-for-English, and BOTH of them shipped a bug where the empty
        // substitution still left a separator behind (a stray blank line) -- caught only by the
        // golden check. These assertions make that class of regression fail here instead.
        viz_i18n::reset_active();
        assert!(plotly_locale_block().is_empty());
        assert!(plotly_setlocale_js().is_empty());
        assert!(datatables_language_block().is_none());

        // ...and the placeholder must vanish without disturbing the surrounding bytes.
        let bootstrap = PLOTLY_GZ_BOOTSTRAP.replace("__QSVI18N_PLOTLY_SETLOCALE__", "");
        assert!(
            bootstrap.contains("    (0, eval)(src);\n    q.forEach(function (args) {"),
            "an empty locale substitution must restore the bootstrap byte-for-byte"
        );
        assert!(!bootstrap.contains("__QSVI18N"));
    }

    #[test]
    fn plotly_locale_bundle_precedes_the_queue_stub() {
        // REGRESSION GUARD. The locale file self-registers by branching on
        // `typeof Plotly === "undefined"`. Emitted AFTER `PLOTLY_STUB_SCRIPT` it sees the stub's
        // `window.Plotly`, calls `register()` on a queue object that throws the dictionary away,
        // and the bootstrap then deletes the stub -- leaving `setPlotConfig` reporting locale
        // "es" while every built-in modebar tooltip renders in English.
        //
        // This shipped and passed EVERY static check (asset embedded, setPlotConfig emitted,
        // eval<setLocale<replay ordering); only opening the page in a browser revealed it. Hence
        // an explicit ordering assertion rather than a presence one.
        let _guard = viz_i18n::lock_locale();
        viz_i18n::set_active(viz_i18n::parse_lang("es").unwrap());

        let block = plotly_js_block();
        let locale_at = block
            .find(r#"name:"es""#)
            .expect("the vendored locale bundle should be present for es");
        let stub_at = block
            .find("__qsvPlotQ")
            .expect("the gzip path should emit the queue stub");
        assert!(
            locale_at < stub_at,
            "the plotly locale bundle must precede the queue stub, else it registers against the \
             stub and is silently discarded (locale@{locale_at}, stub@{stub_at})"
        );

        viz_i18n::reset_active();
    }

    #[test]
    fn spanish_injects_both_vendored_assets() {
        let _guard = viz_i18n::lock_locale();
        viz_i18n::set_active(viz_i18n::parse_lang("es").unwrap());

        let locale = plotly_locale_block();
        assert!(
            locale.contains(r#"name:"es""#),
            "the vendored plotly locale bundle should be emitted"
        );
        assert!(
            plotly_setlocale_js().contains(r#"setPlotConfig({locale: "es"})"#),
            "registering a locale does not select it -- setPlotConfig must be emitted too"
        );

        let lang = datatables_language_block().expect("es has a vendored DataTables file");
        let parsed: serde_json::Value = serde_json::from_str(
            lang.trim_start()
                .trim_start_matches("language:")
                .trim_end_matches(','),
        )
        .expect("the spliced language block must be valid JSON");
        // vendored strings survive...
        assert!(parsed.get("zeroRecords").is_some());
        // ...and qsv's own name for SearchBuilder overrides the vendored "search builder" wording,
        // with DataTables' %d placeholder intact.
        let button = &parsed["searchBuilder"]["button"];
        assert_eq!(button["0"], "Filtro avanzado");
        assert_eq!(button["_"], "Filtro avanzado (%d)");

        viz_i18n::reset_active();
    }

    /// The date SearchBuilder conditions DataTables ships for zh are inverted, so an unpatched
    /// drawer applies the opposite comparison to the one the reader picked. `datatables_language_
    /// block` corrects them at splice time; this pins that, and pins that the correction is still
    /// needed.
    #[test]
    fn zh_date_condition_labels_are_corrected() {
        let _guard = viz_i18n::lock_locale();
        viz_i18n::set_active(viz_i18n::parse_lang("zh-CN").unwrap());

        let lang = datatables_language_block().expect("zh-CN has a vendored DataTables file");
        let parsed: serde_json::Value = serde_json::from_str(
            lang.trim_start()
                .trim_start_matches("language:")
                .trim_end_matches(','),
        )
        .expect("the spliced language block must be valid JSON");
        let date = &parsed["searchBuilder"]["conditions"]["date"];
        assert_eq!(date["after"], "晚于", "`after` must read 'later than'");
        assert_eq!(date["before"], "早于", "`before` must read 'earlier than'");
        // the sibling conditions must survive the surgical edit untouched
        assert_eq!(date["between"], "介于");
        assert_eq!(
            parsed["searchBuilder"]["conditions"]["number"]["gt"],
            "大于"
        );

        // Canary: when upstream fixes this, the override turns into dead code that shadows a
        // now-correct string. Fail then, so it gets removed rather than quietly kept forever.
        let vendored: serde_json::Value =
            serde_json::from_str(include_str!("assets/i18n/datatables/zh-CN.json"))
                .expect("the vendored zh-CN DataTables file must be valid JSON");
        let raw = &vendored["searchBuilder"]["conditions"]["date"];
        assert!(
            raw["after"] == "早于" && raw["before"] == "晚于",
            "the vendored zh-CN DataTables file no longer has its date conditions inverted -- \
             upstream looks fixed, so drop the override in datatables_language_block"
        );

        viz_i18n::reset_active();
    }

    /// The `pt` vs `pt-BR` trap, generalized across every curated language.
    ///
    /// A vendored plotly locale registers itself under the id baked into the FILE, while
    /// `plotly_setlocale_js` selects by the `LOCALES` TAG. When those disagree the locale is
    /// registered and then never selected, so a fully translated Data Schematic keeps an English
    /// modebar -- no panic, no console error, and a test that only checks "a block was emitted"
    /// sails straight past it. plotly publishes no generic `pt` locale (only `pt-br`, id "pt-BR"),
    /// which is exactly how this bites in practice.
    #[test]
    fn every_vendored_locale_registers_the_id_it_later_selects() {
        let _guard = viz_i18n::lock_locale();
        let mut plotly_checked = 0_usize;
        let mut datatables_checked = 0_usize;

        for row in viz_i18n::LOCALES {
            viz_i18n::set_active(row);
            let bundle = plotly_locale_block();
            if bundle.is_empty() {
                // No vendored plotly file: the documented graceful fallback. Selecting a locale
                // nothing registered would be worse than staying English, so assert it doesn't.
                assert!(
                    plotly_setlocale_js().is_empty(),
                    "{} selects a plotly locale that was never registered",
                    row.bcp47
                );
            } else {
                plotly_checked += 1;
                assert!(
                    bundle.contains(&format!(r#"name:"{}""#, row.bcp47)),
                    "the vendored plotly locale for {} registers an id different from the tag \
                     setPlotConfig asks for -- its modebar would silently stay English",
                    row.bcp47
                );
                assert!(
                    plotly_setlocale_js()
                        .contains(&format!(r#"setPlotConfig({{locale: "{}"}})"#, row.bcp47)),
                    "{} registers a plotly locale but never selects it",
                    row.bcp47
                );
            }

            if let Some(lang) = datatables_language_block() {
                datatables_checked += 1;
                let parsed: serde_json::Value = serde_json::from_str(
                    lang.trim_start()
                        .trim_start_matches("language:")
                        .trim_end_matches(','),
                )
                .unwrap_or_else(|e| {
                    panic!(
                        "{}'s spliced DataTables block is not valid JSON: {e}",
                        row.bcp47
                    )
                });
                assert!(
                    parsed.get("zeroRecords").is_some(),
                    "{}'s vendored DataTables strings did not survive the splice",
                    row.bcp47
                );
                // qsv overrides searchBuilder.button from its OWN catalog, so an untranslated
                // catalog entry shows up here as the English default.
                assert_ne!(
                    parsed["searchBuilder"]["button"]["0"], "Advanced Filter",
                    "{}'s advanced-filter label was not translated",
                    row.bcp47
                );
            }
        }

        assert!(
            plotly_checked >= 5 && datatables_checked >= 5,
            "expected every vendored locale to be exercised, saw plotly={plotly_checked} \
             datatables={datatables_checked}"
        );
        viz_i18n::reset_active();
    }

    #[test]
    fn assert_data_drawer_lang_needle_is_current() {
        // DT_LANG_NEEDLE must keep matching DATA_DRAWER_SCRIPT verbatim: the vendored
        // DataTables i18n JSON will be spliced in by replacing exactly this text, and a needle
        // that silently stopped matching would leave the table strings English forever with no
        // error. If this fails, someone edited the `language:` block -- re-copy it into the const.
        assert!(
            DATA_DRAWER_SCRIPT.contains(DT_LANG_NEEDLE),
            "DT_LANG_NEEDLE no longer matches DATA_DRAWER_SCRIPT's `language:` block"
        );
        // ...and DataTables' own %d placeholder must survive verbatim -- it is NOT a rust-i18n
        // token (rust-i18n only interpolates %{name}), so nothing may "helpfully" rewrite it.
        assert!(
            DT_LANG_NEEDLE.contains("Advanced Filter (%d)"),
            "DataTables' %d placeholder must stay verbatim"
        );
    }

    #[test]
    fn html_in_js_text_neutralizes_both_nesting_levels() {
        // Used where translated text lands in HTML markup that is itself inside a
        // SINGLE-quoted JS string. A `"` would close the HTML attribute, a `'` would close the
        // JS string, and a `\` is swallowed by the JS string literal before the HTML parser
        // ever runs -- `'C:\backup'` parses as "C:" + U+0008 BACKSPACE, losing text silently
        // with no error anywhere. All must become entities -- and unlike js_string_literal,
        // this helper must NOT add surrounding quotes.
        let out = html_in_js_text(r#"l'ensemble "des" <b>& C:\backup"#);
        assert!(!out.starts_with('"'), "must not add quotes: {out}");
        for bad in ['\'', '"', '<', '>', '\\'] {
            assert!(
                !out.contains(bad),
                "{bad:?} survived escaping and could break out of its context: {out}"
            );
        }
        assert!(
            out.contains("&#39;"),
            "apostrophe should become &#39;: {out}"
        );
        assert!(
            out.contains("&quot;"),
            "double quote should become &quot;: {out}"
        );
        // A bare `&` must be escaped FIRST so the other entities are not double-escaped.
        assert!(
            out.contains("&amp;"),
            "ampersand should become &amp;: {out}"
        );
        assert!(
            out.contains("&#92;"),
            "backslash should become &#92; or it is eaten by the JS string: {out}"
        );
        for entity in ["&amp;#39;", "&amp;#92;", "&amp;quot;"] {
            assert!(
                !out.contains(entity),
                "entities must not be double-escaped ({entity}): {out}"
            );
        }
    }

    #[test]
    fn js_string_literal_neutralizes_translation_hostile_characters() {
        // The whole JS-const localization scheme rests on this: placeholders sit in BARE
        // positions and `js_string_literal` supplies the quotes AND the escaping. Real
        // translations routinely contain apostrophes (fr "l'ensemble"), and a value able to
        // emit a raw `<` could close the surrounding `<script>`.
        let apostrophe = js_string_literal("l'ensemble des donnees");
        assert!(
            apostrophe.starts_with('"') && apostrophe.ends_with('"'),
            "the helper must supply its own surrounding quotes: {apostrophe}"
        );
        assert!(
            apostrophe.contains("l'ensemble"),
            "an apostrophe is safe inside a double-quoted JS string and must survive intact"
        );

        let quoted = js_string_literal(r#"say "hi""#);
        assert!(
            quoted.contains(r#"\""#),
            "an embedded double quote must be backslash-escaped, got {quoted}"
        );

        // `<`/`>`/`&` become \u00XX so no translation can smuggle a closing script tag.
        let tagish = js_string_literal("</script><b>&");
        assert!(
            !tagish.contains('<') && !tagish.contains('>') && !tagish.contains('&'),
            "angle brackets and ampersands must be unicode-escaped, got {tagish}"
        );
        assert!(tagish.contains("\\u003c") && tagish.contains("\\u003e"));
    }
}
