# Changelog

All notable changes to the qsv Agent Skills (MCP Server) project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Changed
- **`qsvmcp` now builds with plain `viz` instead of `viz_static`.** No functional change for MCP callers: `tool-handlers.ts` already rejects a non-HTML `--output` (`.png/.svg/.pdf/.jpeg/.webp`) *before* qsv runs, so the browser/webdriver-backed static-export path was never reachable over MCP — it remains CLI-only. The build-side effect is that `plotly_static`, `fantoccini`, `webdriver-downloader`, `native-tls` and ~50 other crates are no longer compiled into `qsvmcp`, which is what makes the binary buildable on the Windows publish targets: `publish.yml` builds qsvmcp with `--features=qsvmcp`, and `viz_static` does not finish a Windows release build inside GitHub's 6h job cap ([run 31243109948](https://github.com/dathere/qsv/actions/runs/31243109948)). Skill count unchanged at 55; no skill JSON regeneration needed (`viz`'s USAGE is not feature-gated).
- **`qsv-viz` skill regenerated for the new `--preview-threshold` flag** ([#4283](https://github.com/dathere/qsv/issues/4283)). `viz smart` HTML dashboards gain an interactive data viewer: next to the rowcount in the dashboard metadata, an "(Explore)" link (dataset ≤ threshold; all rows embedded) or "(Preview)" link (first `<n>` rows embedded) opens the underlying table in a bottom DataTables drawer with global search, per-column filters, and a stats-configured SearchBuilder. `--preview-threshold <n>` (default 50000) sets the cut; `0` disables the viewer. The regen also picked up the previously-unregenerated `funnel` subcommand and pipeline-relationship docs from [#4281](https://github.com/dathere/qsv/pull/4281)/[#4282](https://github.com/dathere/qsv/pull/4282) (viz + describegpt skills). Skill count unchanged at 55.
- **`qsv-viz` skill regenerated for the new `choropleth` chart type** ([#302](https://github.com/dathere/qsv/issues/302)). `viz` gains a `choropleth` subcommand that fills geographic regions colored by a value (or per-region row count), and `viz smart` auto-adds a per-country choropleth panel beside the point map when coordinates resolve to ≥2 countries. Regions are matched by `--location-mode` (`iso3`, `usa-states`, `country-names`, or a GeoJSON feature id via `geojson-id`); `--map` switches from the token-free geo projection to a MapLibre basemap, using `--geojson` (local file or remote URL) + `--feature-id-key`. When qsv is built with the `geocode` feature, `--geocode` derives region codes from the local Geonames index — reverse from `--lat`/`--lon`, or forward from a place/country-name column. Skill count unchanged at 55.
- **`qsv-viz` skill regenerated for `viz smart` reverse-geocoded spatial-extent metadata** ([#302](https://github.com/dathere/qsv/issues/302)). When qsv is built with the `geocode` feature, a `viz smart` dashboard that detects a lat/long pair now reverse-geocodes the map's spatial extent — the 4 bounding-box corners + center — against the local Geonames index and draws it on the map as a bounding box with labeled points, plus a consolidated location summary below the map (e.g. "New York & New Jersey, United States"). In HTML the points reveal city/state/country on hover; static image exports show the box without hover. Degrades gracefully (the map renders without the overlay) when geocoding is unavailable, and skips extents spanning the antimeridian. Skill count unchanged at 55.
- **`qsv-viz` skill regenerated for the new `--no-nulls` / `--no-other` flags** ([#302](https://github.com/dathere/qsv/issues/302)). `viz smart` frequency bar charts now show a `(NULL)` bar for empty cells and an `Other (N)` aggregate bar collecting the categories beyond `--limit` (N = how many distinct categories were rolled up), matching `qsv frequency`'s default output; both aggregate bars are drawn in a muted grey so they read as summaries. `--no-nulls` and `--no-other` suppress the respective bars. Skill count unchanged at 55.
- **`qsv-viz` skill regenerated for the new `--theme` flag** ([#302](https://github.com/dathere/qsv/issues/302)). `viz` gains a `--theme <name>` option that applies a Plotly built-in theme (`default`, `plotly_white`, `plotly_dark`, `seaborn`, `seaborn_whitegrid`, `seaborn_dark`, `matplotlib`, `plotnine`; case-insensitive, hyphens accepted) to all chart types including `smart` dashboards. The theme drives backgrounds, fonts and axis chrome; dark themes also drive the inline dashboard's page chrome. Skill count unchanged at 55.
- **`qsv-describegpt` skill regenerated for multimodal `--context-file`**. The `--context-file` description now advertises non-text context: CSV and Excel/ODS spreadsheets are extracted to text locally, while PDF and image files (JPEG/PNG/WebP/GIF) are sent to the LLM as a base64 multimodal attachment (needs a multimodal model & endpoint; ~32 MB max); Word/PowerPoint are rejected. The file type is sniffed from contents, and context is injected into the user message. Skill count unchanged at 55.
- **`qsv-describegpt` skill regenerated for the new `--format okf` output** ([#4018](https://github.com/dathere/qsv/issues/4018)). The describegpt skill now advertises **OKF (Open Knowledge Format)** as a `--format` choice alongside SemanticMd — a leaner, vendor-neutral plain-markdown-plus-YAML-frontmatter Data Dictionary document (`type`/`title`/`description`/`resource`/`timestamp`/`tags` frontmatter + a `Column | Type | Description` Schema table). Adds the `--okf-type` flag (sets the required `type` frontmatter key, default `"CSV Table"`); `--ds-source`/`--ds-updated` additionally map to OKF's `resource`/`timestamp`. Skill count unchanged at 55.

### Added
- **New `visual-data-dictionary` repo-local agent skill** (NOT packaged — it lives at the top of `.claude/skills/` beside `build-dashboard`/`release-prep`, which `package-plugin.js` does not archive; only `.claude/skills/skills/` ships). It drives the **qsv CLI** directly rather than the `mcp__qsv__*` tools, so it needs `Bash` + `python3` and is available when working in the qsv repo. `/visual-data-dictionary <input.csv>` builds an interactive `viz smart` dashboard whose panels are routed by an LLM-inferred JSON Schema data dictionary, with that dictionary embedded beside the charts. Four stages, in order: `denull` (blank null sentinels so numeric columns are actually numeric) → `describegpt --dictionary --description --two-pass --infer-content-type --format JSONSchema` → optional GeoJSON feature-id-key discovery → `viz smart --smarter --bivariate --dictionary <schema> --dict-info`. The order is load-bearing: a dictionary built from dirty data documents a `String` column that is really a number, and `viz smart` then charts it as a category or skips it. Detects a local LLM endpoint (LM Studio / ollama / `QSV_LLM_BASE_URL`) and prompts only when none is found. The GeoJSON step inspects the file and ranks candidate `--feature-id-key` properties by uniqueness *and* meaningfulness, demoting geometry/bookkeeping fields (`shape_area`, `intptlon10`, `last_edited_date`) that are unique but useless as region labels. Passing `--dictionary` explicitly is required because `--bivariate` otherwise implies `--dictionary infer`, costing a second LLM call.
- **`qsv-viz` exposed as an MCP skill** ([#302](https://github.com/dathere/qsv/issues/302)). The Plotly charting/auto-dashboard command (subcommands `smart`, `bar`, `line`, `scatter`, `histogram`, `box`, `pie`, `heatmap`, `candlestick`, `ohlc`, `sankey`, `radar`, `map`) is now generated as a skill and is **search-discoverable** via `qsv_search_tools` (not preloaded in `COMMON_COMMANDS`). Skill count grows 54 → 55. Output is **HTML-only** over MCP: viz produces a self-contained interactive HTML artifact that is **always saved to the working directory** (never inlined into chat or deleted, regardless of size) and its `.html` path returned. A non-HTML `--output` (`.png/.svg/.pdf/.jpeg/.webp`) is **rejected** before viz runs, so MCP callers can't trigger the browser/webdriver-backed static-export path — static export remains CLI-only. `viz` is registered in `ALWAYS_FILE_COMMANDS` and `NON_TABULAR_COMMANDS`; the auto-created output file and the saved working-dir artifact both use the `.html` extension so the chart is correctly typed.

## [21.1.0] - 2026-06-14

Companion release to qsv 21.1.0 ("Leaner & Faster"). The MCP server tracks the qsv binary version per the policy adopted in 20.0.0. A performance-and-polish release: no new MCP skills and no MSRV or Polars bump. The minimum qsv binary is raised to 21.1.0, so update your qsv binary first; existing pipelines are otherwise unaffected.

### Changed
- **Minimum qsv version raised to 21.1.0** — single-sourced in `manifest.json` (`_meta.com.dathere.qsv.minimum_qsv_version`); `src/config.ts` and `scripts/cowork-setup.cjs` read it dynamically at runtime. Required because the regenerated skills advertise 21.1.0-only flags.
- **Skill JSONs regenerated from qsv 21.1.0 USAGE text.** Skill count unchanged at 54 (no new top-level commands). Materially updated skills:
  - **`qsv-diff`** — new `--drop-equal-columns` flag (drop columns with no differences anywhere; combinable with `--drop-equal-fields`), plus examples ([#2000](https://github.com/dathere/qsv/issues/2000); [#3998](https://github.com/dathere/qsv/pull/3998)).
  - **`qsv-safenames`** — new "safer" `s`/`S` modes plus `--collapse` and `--unicode` flags for finer header-sanitization control ([#1921](https://github.com/dathere/qsv/issues/1921); [#3975](https://github.com/dathere/qsv/pull/3975)).
  - **`qsv-describegpt`** — new `--context-file` option to enrich prompts with additional dataset context ([#3969](https://github.com/dathere/qsv/issues/3969); [#3972](https://github.com/dathere/qsv/pull/3972)).
  - **`qsv-stats`** — description refreshed for the faster, leaner stats engine (~40% faster unindexed, ~42% faster indexed `--everything`, ~43% less peak memory) ([#3970](https://github.com/dathere/qsv/pull/3970)).
  - **`qsv-sample`** — minor description/flag refinements.
  - All other skill JSONs carry a version-stamp bump to 21.1.0.

### Notes
- **Transparent auto-decompression now benefits MCP/Cowork users.** qsv 21.1.0 auto-detects and decompresses `.zip` inputs and lookup tables (local) plus compressed remote sources (`.gz`/`.zlib`/`.zst`) for lookup tables and `get`/`dc:` ingests — no pre-processing step ([#1417](https://github.com/dathere/qsv/issues/1417)). `get`/`fetch` remain intentionally NOT exposed as MCP skills.

## [21.0.0] - 2026-06-07

Companion release to qsv 21.0.0. The MCP server tracks the qsv binary version per the policy adopted in 20.0.0.

### Added
- **Environment-variables documentation** — `README-MCP.md` now documents the `QSV_MCP_*` environment variables supported by the qsv MCP server.

### Changed
- **Minimum qsv version raised to 21.0.0** — single-sourced in `manifest.json` (`_meta.com.dathere.qsv.minimum_qsv_version`); `src/config.ts` and `scripts/cowork-setup.cjs` read it dynamically at runtime ([#3897](https://github.com/dathere/qsv/pull/3897) made it a single source of truth).
- **Minimum Node.js version raised to 20.0.0** (from 18.0.0); all docs swept for the Node 18 → 20 change.
- **Command counts: qsvmcp 64 → 65, full `qsv` binary 73 → 74** — the new `get` command (disk-cache fetch with cloud sources, qsv [#2263](https://github.com/dathere/qsv/issues/2263)) joins both the `qsvmcp` and full `qsv` feature sets; reflected in `manifest.json`, `README-MCP.md`, and `README-MCPB.md`. `get`/`fetch` are intentionally NOT exposed as MCP skills, so the skill count is unaffected.
- **Skill count: 55 → 54** — reflected in `manifest.json`, `.claude-plugin/plugin.json`, root `.claude-plugin/marketplace.json`, `README-MCP.md`, `README.md`, `README-MCPB.md`, `GEMINI.md`, `cowork-CLAUDE.md`, and `skills/csv-wrangling/SKILL.md`.
- **Skill JSONs regenerated from qsv USAGE text** — beyond dropping `luau`, the regeneration synced two drifted skills: `qsv-describegpt.json` (neuro-symbolic rename) and `qsv-sniff.json` (`--sample` distributed-sampling note when a CSV index is present), plus an updated `stats` skill description.

### Fixed
- **Symlink bypass closed + Parquet fallback warning sanitized** — hardened input path handling against a symlink-based bypass and sanitized the Parquet fallback warning message.

### Removed
- **`luau` skill dropped from qsvmcp** — the `qsvmcp` binary no longer enables the `luau` feature, so `qsv-luau.json` is no longer part of the generated skill set. Luau requires `panic = "unwind"`, which is incompatible with qsvmcp's `panic = "abort"` release build (a Luau callback error would abort the process instead of surfacing — see qsv [#3937](https://github.com/dathere/qsv/issues/3937)). The `luau` command remains available in the full `qsv` binary.

## [20.1.0] - 2026-05-18

Companion release to qsv 20.1.0 ("Synthetic Data"). MCP server tracks the qsv binary version per the policy adopted in 20.0.0.

### Added
- **`synthesize` command skill** — new `qsv-synthesize.json` (category: `generation`), exposing the 20.1.0 `qsv synthesize` command for statistically-faithful synthetic CSV generation, including `--locale` for 14 fake-rs locales, `--consistent-fakes` for stable source→fake mapping, string-length stats for unstructured text columns, and Content Type inference (street_name, license_plate, industry, profession, ipv6_address, etc.) when paired with `describegpt`. Skill JSONs auto-regenerated from qsv 20.1.0 USAGE text.

### Changed
- **Minimum qsv version raised to 20.1.0** across all three enforcement points (`src/config.ts`, `manifest.json`, `scripts/cowork-setup.cjs`); required for the `synthesize` skill and the `describegpt --infer-content-type` / `--two-pass` features it builds on.
- **Skill count: 54 → 55** — reflected in `manifest.json`, `.claude-plugin/plugin.json`, root `.claude-plugin/marketplace.json`, `README-MCP.md`, `README.md`, `cowork-CLAUDE.md`, and `skills/csv-wrangling/SKILL.md`.
- **qsv CLI command count: 72 → 73** in `manifest.json` features array (qsvmcp 63 → 64); the new `synthesize` command joins `distrib_features` and the `qsvmcp` feature set.
- **`feat(mcp): Compute qsv skill count dynamically`** (65af4bf65) — skill count is now derived at build time rather than hardcoded.
- **`chore(mcp): update luau skill JSON with Luau version`** (d384893fc) — pin Luau 0.720 in skill metadata.
- **README.md totals refreshed** — examples 174 → 221, options 604 → 658, arguments 87 → 98; new `generation` category row; `utility` 23 → 24 (includes `implode`).

### Fixed
- **Version drift fixed across publish artifacts** — `.claude-plugin/plugin.json` and the repo-root `.claude-plugin/marketplace.json` (both `metadata.version` and `plugins[0].version`) were stale at 20.0.0 after the qsv 20.1.0 binary bump; now aligned at 20.1.0.
- **`MINIMUM_QSV_VERSION` enforcement points re-synced** — `src/config.ts` and `scripts/cowork-setup.cjs` were stale at 20.0.0 while `manifest.json` had already moved to 20.1.0; all three now agree.

## [20.0.0] - 2026-05-03

### Added
- **`implode` command skill** — inverse of `explode`; auto-regenerated from qsv 20.0.0 USAGE text (#3733)

### Changed
- **BREAKING: MCP server version synced to qsv binary** — versioning policy change. MCP server and qsv binary versions now advance together starting at 20.0.0. Consumers pinning the MCP server independently (e.g., `^19.1.0`) will see a major-version jump from 19.1.1 → 20.0.0.
- **Minimum qsv version raised to 20.0.0** across all three enforcement points (`src/config.ts`, `manifest.json`, `scripts/cowork-setup.cjs`); required for the `implode` command and other 20.0.0 features
- **Regenerated all skill JSONs from qsv 20.0.0 USAGE text** — beyond adding `implode`, the regeneration also picked up examples-parser fixes (#3802) for `qsv-sqlp`, `qsv-split`, `qsv-exclude` and minor description/argument refinements across other skills (#3793, #3734)
- **Hardened MCP loader, executor, concurrency, and audit logging** (#3801) — pre-abort signal to avoid unrefed-timer hang in CI, async binary lookup so hook hard-timeout actually fires, child-process kill on hook timeout
- **Sharpened agent definitions and unbroken plugin marketplace install** (#3811) — fixes fresh marketplace installs and reconciles command counts
- **Command counts updated** for qsv 20.0.0 — `qsv` 71→72, `qsvmcp` 62→63 in README-MCP.md and manifest.json (skill-based count remains 54)
- **README category table rebuilt** from skill JSON categories (#3716 follow-up); doc-drift fixes in MCP guidance
- **Guidance made data-driven** with truncated-log markers (#3716)

### Fixed
- **Examples parser alignment and pipe-prefixed commands** (#3802) — corrects `extract_examples` parsing of multi-line examples and pipe chains; shell-escapes the apostrophe in the sqlp dollar-quoting example
- **Notification buffering** until handlers attach (#3711 follow-up)

## [19.1.1] - 2026-04-14

### Changed
- **Minimum qsv version raised to 19.1.0** across all enforcement points (config.ts, cowork-setup.cjs, manifest.json)
- **Full MCP-qualified tool names** in skill/agent content for correct Cowork/plugin resolution (#3711)
- **Compact allowed-tools lists** in SKILL.md files
- **Command guidance migrated to YAML** file for easier maintenance (#3708)

### Fixed
- **Command count discrepancies** — reconciled stale counts (51, 70, "53+") to actual 53 skill-based commands across plugin.json, marketplace.json, cowork-CLAUDE.md, csv-wrangling, and manifest.json
- **Agent version drift** — bumped agent frontmatter from 19.0.0 to 19.1.1
- **Author field inconsistency** — harmonized plugin.json and marketplace.json to include both `url` and `email`
- **Stale MINIMUM_QSV_VERSION comment** removed from config.ts
- **Release prep skill** updated with missing checklist items (agent versions, config.ts minimum version, package-lock sync, command count verification)

## [19.1.0] - 2026-04-13

### Added
- **`blake3` and `to` skills** added; `pivotp` & `moarstats` skills updated (#3703)

### Changed
- **Command guidance migrated to YAML** — extracted inline guidance objects to `qsv/command-guidance.yaml` for easier maintenance (#3708)
- **Server decomposition** — centralized constants, added error sanitization, extracted modules from `mcp-server.ts` (#3705)
- **Executor refactor** — extracted `spawnWithTimeout`, gated debug logging behind `QSV_MCP_DEBUG` (#3694)
- **Dedicated Serena config** for MCP Cowork project (#3704)
- **Documentation** — added AI tools rationale, StatsSighting link, qsv MCP tools README, streamlined quick-start, updated pivotp/sortcheck skill docs, Rationale in FAQ

### Dependencies
- Bumped `hono` 4.12.7 → 4.12.12 and `@hono/node-server`

## [19.0.0] - 2026-04-06

### Added
- **FBI Crime Data MCP Server** integration with CDE homepage summary & spillover tools (#3673, #3675)
- **Project reproducibility manifest** for Cowork plugin (#3657)

### Changed
- **Modular refactor** — split `mcp-tools.ts` into focused modules (#3682)
- **Parquet conversion** — replaced sqlp fallback with `to parquet` (#3681)
- **Guards and logging** added to MCP tools/sampling
- **`data-profile` skill** improved (#3668)
- **`tojsonl` skill** updated
- **Plugin agents** — policy-analyst uses MoarStats and BLS MCP Server (#3634, #3638); replaced b3sum dependency with built-in qsv blake3 (#3659)
- **Plugin skills** — migrated commands to skills (#3655); refined joinp asof guidance (#3633)
- **Documentation** — updated tool counts, Census/BLS/FBI jurisdictional notes, Cowork troubleshooting links, spreadsheet format clarification, intentional use of low-level Server API documented

### Fixed
- **Plugin** — corrected plugin wording and skill count in START_HERE guide; policy-analyst references infer-ontology workflow directly; use quadruple backticks for ONTOLOGY.md template

## [18.0.5] - 2026-03-27

### Added
- **WebSearch/WebFetch logging** — log results for citation tracking (#3645)
- **Stop and SessionEnd hooks** — log each turn and session for observability (#3643)
- **`review-respond` skill** — batch-process PR review comments in a single pass

### Changed
- **DuckDB support enabled by default** — no longer requires explicit opt-in
- **UserPrompt logging** refactored to use hooks for more reliable capture (#3642)
- **Documentation** — simplified MCPB install flow (6 steps → 3), replaced CLI-based Cowork plugin install with GUI marketplace workflow, consolidated and audited docs for accuracy (#3646)
- **Plugin agents** — agents now check for ONTOLOGY.md and skip redundant profiling; policy-analyst ontology compilation refined

### Fixed
- **START_HERE.md** — corrected "qsv tab" wording to "qsv plugin"; fixed domain skill count from 5 to 6 (added missing bls-query)
- **Plugin** — corrected data-analyst ontology skip target (step 6 → step 7); inlined unreachable skill/command references into agent prompts
- **Documentation version footers** — updated footer/version blocks from 18.0.4 → 18.0.5 in `README.md`, `docs/reference/AUTO_UPDATE.md`, and `docs/reference/UPDATE_SYSTEM.md`

## [18.0.4] - 2026-03-23

### Changed
- **Tool dispatch refactored** from if-else cascade to dispatch map in `mcp-server.ts` — extracted `handleSetup`, `handleListFiles`, `handleSetWorkingDir`, `handleBrowseDirectoryDispatch` private methods; adding new tools now requires only a map entry + handler method
- **Version module refactored** — extracted `resolveProjectRoot` and `readVersionFromJson` helpers from monolithic `getVersion()`; both exported for testability
- **Documentation** consolidated install guides into single START_HERE.md (#3641)
- **data-profile command** replaced describegpt with LLM-based profiling docs
- **data-analyst agent** no longer uses describegpt
- **policy-analyst agent** now infers ontology by default, references infer-ontology workflow directly

### Fixed
- **Version sync validation** — `version.ts` now validates `manifest.json` version matches `package.json` at startup, logging a warning on mismatch and a diagnostic when `manifest.json` is absent (suppressed in test runs via `NODE_TEST` guard)
- **Version test skip pattern** — manifest sync test uses `{ skip: !MANIFEST_AVAILABLE }` instead of silent early return, consistent with project test conventions

### Added
- **`infer-ontology` command** for plugin — infers domain ontology from data
- **9 version module tests** covering `resolveProjectRoot`, `readVersionFromJson` edge cases (missing file, invalid JSON, non-string version, empty version), and package.json/manifest.json sync validation

## [18.0.3] - 2026-03-23

### Added
- **Cowork/plugin-mode server instructions** — conditional COWORK/PLUGIN-MODE NOTES appended to server instructions when running in plugin mode, covering host path architecture, sequential operations, large file guidance, and context window management

### Changed
- **Workflow guide deployment** moved from SessionStart hook (which doesn't fire in Cowork) to MCP server startup as a fire-and-forget operation; deploys `CLAUDE.md` and `.cowork-instructions.md` to working directory
- **Workflow guide deployer** refactored: static imports for `readFile`/`writeFile`, extracted `deployIfMissing` helper to deduplicate logic
- **Documentation** updated stale claims: command counts (65 qsvmcp / 75 qsv), core tools count (11), smart commands list (added describegpt/moarstats, removed diff), manifest description (75 commands)

### Fixed
- **completedDirResult** returned when MCP Apps enabled (browse directory)
- **package.json** included in MCPB archive for runtime version detection
- **Hardcoded timeout** in plugin-mode notes now says "default 10 min, configurable via QSV_MCP_OPERATION_TIMEOUT_MS"

## [18.0.2] - 2026-03-22

### Added
- **Policy analyst agent** for evidence-based policymaking with BLS MCP Server integration
- **BLS query skill** (`bls-query`) for querying Bureau of Labor Statistics data series
- **Build dashboard skill** documentation for interactive HTML dashboard creation
- **Cowork plugin support** with Quick Start guides, marketplace, and setup hooks (#3621)
- **MCP Apps support** — inline App SDK shim, macOS Finder and Windows PowerShell native directory pickers (#3604, #3605, #3617)
- **qsv auto-installer** — automatic download and installation of qsvmcp and MCP binaries (#3608, #3609, #3613)
- **scoresql** added to MCP server as COMMON_COMMAND (#3614)
- **describegpt LLM passthrough** for direct LLM API access (#3581)
- **PII/PHI screening** in data-profile command with regex-based detection
- **Plugin commands, skills & agents** — data-profile, data-describe, csv-query, data-join, data-clean, data-convert (#3630)

### Changed
- **Policy analyst agent** now uses moarstats for enhanced statistical analysis (#3634)
- **joinp asof guidance** refined for clearer usage patterns (#3633)
- **Consolidate data-quality into data-profile** and trim agent duplication
- **Guidance alignment** — refine stats-aware SQL, exclude unbundled commands (#3623, #3625)
- **describegpt LLM passthrough** tweaks (#3582, #3585)
- **Suppress index --output option** from agent view
- **Parse polars with multi-part suffixes** for version detection
- **Plugin binary validation** — detect qsv binary and warn if missing, validate env parsing
- **Documentation** — consolidate getting started guides, fix 25 stale claims (#3620, #3628), doc audit (#3586)
- **Markdown frontmatter** fully populated across all skill/command/agent files
- **MCP/Plugin publish skill** updated

### Fixed
- **LAUS series ID prefix** and BLS MCP server key corrected in policy analyst configuration
- **PII regex patterns** hardened — add word boundaries to Amex, IBAN, and email patterns
- **Relative path** in data-profile and skill file links in agent docs
- **App picker** prevented when setting working directory
- **Cowork plugin** — fix loading and sync marketplace version

## [17.0.0] - 2026-03-03

### Added
- **TSV output format** for token-efficient agent responses (configurable via `QSV_MCP_OUTPUT_FORMAT`)
- **`qsv_log` tool** and comprehensive audit logging with `QSV_MCP_LOG_LEVEL` env var
- **DuckDB-powered CSV-to-Parquet conversion** when DuckDB binary available
- **Auto-run `moarstats`** after `stats` execution for enriched statistical context
- **Reserved cache filename guard** prevents `--output` from overwriting `.stats.csv` and `.freq.csv` cache files
- **Interactive directory picker via MCP Elicitation**: Call `qsv_set_working_dir` without arguments to show a form-based directory picker (when the MCP client supports elicitation)
  - Discovers well-known directories (Downloads, Documents, Desktop, Home) and offers them as selectable options
  - Includes a custom path field for arbitrary directory entry
  - First-tool-use trigger: automatically prompts for directory selection before the first data-processing tool call when no working directory has been confirmed
  - Gracefully falls back to existing behavior when the client doesn't support elicitation
- **Stats/frequency cache-aware guidance** for `sqlp`, `joinp`, `pivotp`
- **Asof join guidance** for `joinp` command
- **Polars SQL engine header** for clear engine differentiation
- **Auto-deploy CLAUDE.md** to Cowork working folder on session start
- **Require Polars feature** in qsv binary validation
- **Operation timeout guidance** in server instructions
- **Show complete command line** in error messages for agent diagnostics

### Changed
- **Optimize skills for qsvmcp binary**: Removed 5 non-qsvmcp commands (`apply`, `fetch`, `fetchpost`, `foreach`, `to`) from skill generation and TypeScript hardcoded references, reducing skill count from 56 to 51
- Rust skill generator now auto-cleans stale `qsv-*.json` files after generation
- Updated `qsv_command` generic tool description to reference `luau` instead of `apply`
- **Deduplicate utilities**, extract shared pipeline, remove dead code
- **SQL translation hardening** — unique `_tbl_N` aliases, string literal protection, pre-scan qualified refs
- **Fine-tune Parquet conversion logic**
- **Comprehensive logging** across all tool handlers
- **Absolute path resolution** for all file-path arguments

### Fixed
- `translateSql` — remove double-parens, prevent alias collision, protect string literals
- Accept `"input"`/`"output"` as aliases for `input_file`/`output_file` parameters
- Correct server instructions and tool guidance inaccuracies
- Ensure CWD is set to qsv Working Directory for `runQsvSimple` helper
- Harden TSV output logic and skip binary output formats
- Harden moarstats auto-run with null checks and robust content lookup
- Make `QsvSkill.examples` optional and strengthen type guard
- Consistent engine header ordering for sqlp results

### Removed
- 5 non-qsvmcp commands from skill generation (`apply`, `fetch`, `fetchpost`, `foreach`, `to`)

## [16.1.2] - 2026-02-18

### Added
- **Auto-sync working directory from MCP client roots** — Automatically detects the client's root directory via MCP `roots/list` protocol and sets it as the working directory, eliminating manual `qsv_set_working_dir` calls when using Claude Cowork's "Work in a folder" feature
  - Listens for mid-session root changes via `RootsListChangedNotification`
  - Manual `qsv_set_working_dir` overrides auto-sync; pass `"auto"` to re-enable
  - Re-entrancy guard and retry cap (max 3) prevent unbounded recursion from rapid notification bursts
  - Directory validation, `file://` URI parsing, and robust JSON-RPC error code handling

### Removed
- Census integration prompt (`qsv_census_integration`) and complementary server hints from tool descriptions

### Fixed
- Harden roots sync error handling: validate root path is a directory, normalize JSON-RPC error codes, wrap pending re-sync in try/catch
- Remove `sqlp` reference for UNPIVOT (not supported in Polars SQL); document DuckDB UNPIVOT as external tool for wide-to-long reshape
- Correct inaccurate command references in SKILL.md files (`fixlengths`, `apply emptyreplace` arg order, replaced `to parquet` with `to ods/xlsx`)
- Clarify `fixlengths` behavior (pads short rows) and add `--frequency-jsonl` cache hint
- Fix download URL version in README-MCPB

### Tests
- 9 roots sync contract tests covering directory validation, URI parsing, error code normalization, manual override, re-entrancy guard, and retry cap

### Dependencies
- Bumped `ajv` from 8.17.1 to 8.18.0

## [16.1.1] - 2026-02-17

### Added
- **DuckDB integration** — Automatic DuckDB routing for SQL queries when DuckDB binary is detected
  - New `src/duckdb.ts` module with lazy detection, SQL translation, and query execution
  - Parquet-first strategy: CSV inputs are auto-converted to Parquet before SQL execution
  - DuckDB preferred for PostgreSQL-compatible SQL; falls back to Polars SQL (sqlp) when unavailable
  - Unsupported output formats (Arrow, Avro) automatically fall back to sqlp
  - New env vars: `QSV_MCP_DUCKDB_BIN_PATH`, `QSV_MCP_USE_DUCKDB`
  - DuckDB status shown in `qsv_config` output
  - Engine indicator (`🦆 Engine: DuckDB vX.Y.Z`) prepended to DuckDB query results
  - MCPB Desktop Extension: DuckDB settings exposed in user config UI (`duckdb_path`, `use_duckdb`)

### Fixed
- CSV truncated-read edge case in file parsing
- AM/PM date format detection in Polars schema generation

### Tests
- DuckDB live integration tests with self-contained NYC 311 fixture (8 tests)

### Docs
- macOS Quick Start guide for installing Claude Desktop with qsv, Census, and Wikidata MCP servers
- Wikidata MCP Server section added to Quick Start guide

## [16.1.0] - 2026-02-15

### Added
- **pragmastat skill** - New `qsv-pragmastat.json` skill with examples and `--memcheck` flag for conservative memory pre-checks
- **Auto-enable --frequency-jsonl** for frequency command (version guard for qsv >= 16.1.0, skips stdin)
- **Frequency cache options** added to `qsv-frequency.json` skill definition
- **Comprehensive mcp-server test suite** - `mcp-server.test.ts` with 35 tests
- **Google Gemini CLI support** - Extended plugin mode detection + `QSV_MCP_PLUGIN_MODE` override
- **c8 code coverage** - `test:coverage` script
- **Stderr size limit** - Prevents memory issues from excessive error output

### Changed
- **Comprehensive code deduplication** — Refactored MCP server removing ~625 lines across 19 files
  - Consolidated 3 `runQsv` implementations into shared `runQsvSimple` with `onSpawn`/`onExit` callbacks
  - Merged 7 guidance tables into single `COMMAND_GUIDANCE` map
  - Decomposed `handleToolCall` into 4 focused functions
  - Extracted `errorResult`/`successResult` helpers and shared test helpers
  - Optimized SkillLoader with parallel reads and single-pass `getStats`
  - Replaced `Record<string, any>` with `unknown` for type safety
  - Removed dead code (`getFileContent`, `pipelineToShellScript`)
- **Renamed 'analysis' category to 'documentation'** for AI-powered documentation commands
- **Removed unused `binary` field** from CommandSpec type
- **Extracted filesystem tools** into `mcp-filesystem.ts` module
- **Improved COMMAND_GUIDANCE** — Enhanced per-command guidance hints
- **Hardcoded min version** for frequency-jsonl guard (16.1.0)
- **Regenerated skills** with normalized USAGE text, sans unused binary property
- **Minimum qsv version** set to 16.0.0 in manifest metadata

### Fixed
- Skip stats/frequency cache when reading from stdin
- Non-null assertion removal in executeWithFile
- normalizeOptionKey edge case for already-prefixed options
- NaN for invalid versions instead of incorrect comparison
- Stale documentation (core tool counts, pipeline references, artifact counts)
- Timing-dependent test fixes and tightened error assertions
- Windows 8.3 path compatibility in plugin mode tests
- Copilot review fixes for plugin mode validation
- Typo: `diretory` → `directory`

### Dependencies
- Bumped `qs` from 6.14.1 to 6.14.2

### Removed
- `qsv_pipeline` tool — agents should call tools sequentially for better error visibility, or use `qsv_sqlp` for multi-step queries
- `QSV_MCP_MAX_PIPELINE_STEPS` environment variable
- `pipeline.ts` fluent API and `mcp-pipeline.ts` tool handler
- Stale documentation: `PROFILE_CACHE.md`, `COPILOT_REVIEW_FIXES*.md`, `CHANGELOG_FILESYSTEM.md`, `examples/pipeline.js`

## [16.0.0] - 2026-02-08

### Added
- **CSV-to-Parquet Core Tool** - New `qsv_to_parquet` tool converts CSV files to Parquet format for optimized SQL operations
  - Auto-generates stats and Polars schema (`.pschema.json`) for correct data type inference
  - Sniffs CSV for Date/DateTime columns via `--infer-dates --dates-whitelist sniff` for automatic temporal type detection
  - Skips stats/schema regeneration when already up-to-date (checks file modification times)
  - Supports comprehensive CSV dialect coverage (delimiter, quoting, encoding)
  - Outputs Snappy-compressed Parquet with same file stem in working directory
  - Active guidance recommends Parquet for CSV files >10MB needing SQL queries
- **Server Instructions** - MCP Server Instructions sent during initialization for cross-tool workflow guidance
  - Covers workflow ordering, stats cache acceleration, file handling, tool composition, and memory limits
  - Injected into system prompt by compatible MCP clients (Claude Desktop, Claude Code, etc.)
  - Configurable via `QSV_MCP_SERVER_INSTRUCTIONS` env var or MCPB settings; empty = built-in defaults
- **Claude Plugin Layer** - Full Claude Code and Cowork integration via plugin manifest
  - Plugin manifest (`.claude-plugin/plugin.json`) for plugin discovery
  - MCP server config (`.mcp.json`) with server key `"qsv"` and `QSV_MCP_EXPOSE_ALL_TOOLS=true`
  - 5 slash commands: `/data-profile`, `/data-clean`, `/csv-query`, `/data-convert`, `/data-join`
  - 2 subagents: `data-analyst` (read-only analysis), `data-wrangler` (data transformation)
  - 3 domain knowledge skills: `csv-wrangling`, `data-quality`, `qsv-performance`
  - Commands include `allowed-tools` restrictions and `argument-hint` for natural invocation
  - Agents have explicit tool lists and reference domain knowledge skills
  - Skills are concise reference tables optimized for Claude scanning
- **Active Context-Sensitive Guidance** - Beyond passive tool descriptions, added structured guidance dictionaries
  - `WHEN_TO_USE_GUIDANCE`: Per-command hints for when to choose each tool (e.g., sqlp vs joinp, join vs joinp)
  - `COMMON_PATTERNS`: Workflow composition patterns (e.g., "index → stats → frequency", "convert to Parquet → query with DuckDB")
  - `ERROR_PREVENTION_HINTS`: Proactive warnings about common mistakes (OOM on large files, high-cardinality columns, subcommand requirements)
  - `COMPLEMENTARY_SERVERS`: Cross-server integration hints (Census MCP for geocoded US data)
  - Moarstats bivariate stats: guidance clarifies `--bivariate` writes to separate `<FILESTEM>.stats.bivariate.csv` file
- **Promoted `qsv_index` and `qsv_stats` to Core Tools** - Core tools increased from 7 to 10
  - `qsv_index`, `qsv_stats`, and `qsv_to_parquet` join the 7 existing core tools
  - Skill definitions properly loaded for deferred/core-only modes
  - These foundational tools are now always available without search discovery
- **Executor Timeout Handling** - `runQsv()` now enforces timeout on spawned qsv processes
  - Default timeout: 10 minutes via `config.operationTimeoutMs` (per-call override via `params.timeoutMs`)
  - Graceful termination: sends SIGTERM, then SIGKILL after 1 second
  - Returns exit code 124 (standard timeout code) with descriptive error message
  - Prevents hanging processes from blocking the MCP server
- **Gemini CLI Integration** - New documentation and support for Google's Gemini CLI
  - Integration guide at `docs/guides/GEMINI_CLI.md`
  - Template variable expansion support for `${PWD}`

### Changed
- **Reduced MCP Skills** - Excluded `behead`, `edit`, `flatten`, `pro`, and `snappy` commands from MCP skills generation, reducing from 60 to 55 skills
  - `behead` - trivial header removal, not needed for AI agent workflows
  - `edit` - interactive command not suitable for AI agent use
  - `flatten` - not suitable for AI agent use
  - `pro` - contains interactive/terminal-dependent subcommands (lens, workflow)
  - `snappy` - compression utility not needed for AI agents
- **Signal Termination Handling** - Proper signal handling in executor close handler
  - Maps signal termination to conventional exit codes (128 + signal number)
  - SIGTERM→143, SIGKILL→137, SIGINT→130, SIGHUP→129, SIGQUIT→131
  - Signal information added to stderr for debugging
- **Tool Guidance Updates** - Updated command categorization for better accuracy
  - `ALWAYS_FILE_COMMANDS` expanded to 25 commands: added slice, sample, template, geocode, schema, validate, diff, cat, transpose, partition, split, explode
  - `METADATA_COMMANDS` refined to 4 commands: count, headers, index, sniff
  - Enhanced guidance for sqlp (Parquet-first workflow for large CSVs), moarstats (bivariate stats), and geocode (FIPS examples)
- **Reduced Default Concurrent Operations** - `QSV_MCP_MAX_CONCURRENT_OPERATIONS` default changed from 10 to 1
- **Skill Version Sync** - Updated all 55 skill JSON files to version 16.0.0
- **Comprehensive Code Deduplication** - Refactored MCP server codebase removing 625 lines across 19 files
  - Consolidated 3 `runQsv` implementations into shared `runQsvSimple` with `onSpawn`/`onExit` callbacks
  - Merged 7 guidance tables into single `COMMAND_GUIDANCE` map
  - Decomposed `handleToolCall` into 4 focused functions
  - Extracted `errorResult`/`successResult` helpers and shared test helpers
  - Optimized SkillLoader with parallel reads and single-pass `getStats`
  - Replaced `Record<string, any>` with `unknown` for type safety
  - Removed dead code (`getFileContent`, `pipelineToShellScript`)

### Fixed
- **Cowork Compatibility via Plugin Mode** - MCP server now works correctly inside Claude Cowork's VM
  - Cowork mounts workspace at a VM-internal path via symlink, causing `realpathSync()` to resolve
    to a canonical path outside the configured `allowedDirs`, blocking all file operations
  - Added **plugin mode** detection: active when `CLAUDE_PLUGIN_ROOT` env var is set AND
    `MCPB_EXTENSION_MODE` is NOT enabled (i.e., running as a Claude Plugin, not a Desktop Extension)
  - In plugin mode, directories are auto-added to `allowedDirs` instead of being rejected,
    since Cowork/Code already provides filesystem isolation
  - Working directory defaults to `${PWD}` in plugin mode (vs `${DOWNLOADS}` otherwise)
  - `qsv_config` tool now shows three deployment modes: Plugin, Extension, and Legacy
- **isTemporaryFile regex fix** - Corrected regex to match 16-char hex filenames (not 36-char UUID)
- **Process tracking for shutdown** - Restored `Set<ChildProcess>` tracking so graceful shutdown kills active processes
- **Concurrency counter scope** - Moved counter to wrap entire `handleToolCall` body for accurate limiting
- **Version comparison with pre-release metadata** - `compareVersions` now strips pre-release/build metadata
- **Non-blocking path validation** - `isPathAllowed` uses `isDirectory` hint to avoid blocking `statSync`
- **Already-prefixed option keys** - `buildSkillExecParams` handles options that already have `--` prefix
- **Cross-device file moves** - Added EXDEV fallback for file moves across filesystem boundaries
- **Moarstats help guard** - Bivariate stats block now guarded for `--help` requests without `input_file`
- **Search guidance typo** - Fixed "to to" duplicate word in search tool guidance

### Security
- **Windows Cross-Drive Path Traversal Fix** - Security fix for path validation on Windows
  - `path.relative()` returns absolute path when comparing paths on different drives
  - Added `isAbsolute()` validation in `setWorkingDirectory()`, `resolvePath()`, and `validatePath()`
  - Prevents potential path traversal attacks across drive boundaries

### Removed
- **Removed client-detector.ts** - Eliminated client auto-detection that bypassed deferred loading
  - Previously, Claude clients were auto-detected and got all tools exposed immediately
  - Now deferred loading is consistent for ALL clients (~85% token reduction by default)
  - Users who want all tools immediately can set `QSV_MCP_EXPOSE_ALL_TOOLS=true`
  - Simplifies codebase and matches 15.3.0 documentation
- **Removed `qsv_data_profile` tool** - The tool produced ~60KB output for a 40-column file, filling Claude's context window and making it impractical
  - Tool guidance now recommends using `qsv stats --cardinality --stats-jsonl` instead
  - Removed profile cache manager and related configuration options
- **Removed `QSV_MCP_TIMEOUT_MS`** - Dead code with 5-minute default removed
  - Consolidated to single timeout source: `QSV_MCP_OPERATION_TIMEOUT_MS` (10-minute default)
  - Removed profile cache environment variables: `QSV_MCP_PROFILE_CACHE_ENABLED`, `QSV_MCP_PROFILE_CACHE_SIZE_MB`, `QSV_MCP_PROFILE_CACHE_TTL_MS`

### Dependencies
- Bumped `@modelcontextprotocol/sdk` from ^1.25.2 to ^1.26.0

## [15.3.0] - 2026-01-31

### Added
- **BM25 Search Integration** - Upgraded tool search from substring matching to BM25 relevance ranking
  - Uses `wink-bm25-text-search` library for proper probabilistic information retrieval
  - Field-weighted search: name (3x), category (2x), description (1x), examples (0.5x)
  - Text preprocessing with stemming, lowercasing, and negation propagation
  - Falls back to substring search if BM25 index not yet built
- **Deferred Tool Loading** - Implements Anthropic's Tool Search Tool pattern
  - Only 7 core tools loaded initially (reduces token usage ~85%)
  - Tools discovered via `qsv_search_tools` are dynamically added to tool list
  - Core tools (always loaded): `qsv_search_tools`, `qsv_config`, `qsv_set_working_dir`, `qsv_get_working_dir`, `qsv_list_files`, `qsv_pipeline`, `qsv_command`
  - Searched tools are marked as loaded and appear in subsequent ListTools responses

### Changed
- **SkillLoader BM25 Integration** - `search()` method now uses BM25 with limit parameter
  - `isBM25Indexed()` method to check if index is ready
  - BM25 index built automatically after `loadAll()`
- **handleSearchToolsCall** - Now marks found tools as loaded for deferred loading pattern

### Dependencies
- Added `wink-bm25-text-search` ^3.0.0 (MIT licensed) for BM25 search algorithm
- Added `wink-nlp-utils` ^2.1.0 for text preprocessing (stemming, tokenization)

## [15.2.0] - 2026-01-31

### Added
- **Dataset Profiling** - New `qsv_data_profile` tool profiles CSV files to help Claude make informed decisions
  - Uses `qsv frequency --toon` to generate column statistics in TOON format (token-efficient for LLMs)
  - Shows data types, cardinality, uniqueness_ratio, null counts, sparsity, sort_order, and value distributions
  - Helps Claude optimize operations across multiple commands:
    - `sqlp`: Choose optimal JOIN order, GROUP BY columns, WHERE selectivity
    - `joinp`: Determine optimal table order (smaller cardinality on right)
    - `frequency`: Identify high-cardinality columns (`<ALL_UNIQUE>`) to exclude
    - `dedup`: Check if uniqueness_ratio=1 (dedup would be a no-op for that key)
    - `sort`: Check if data is already sorted (sort_order field)
    - `pivotp`: Verify pivot column cardinality to avoid overly wide output
- **Profile Caching** - Configurable cache for TOON profiles to avoid redundant profiling
  - `QSV_MCP_PROFILE_CACHE_ENABLED` - Enable/disable caching (default: true)
  - `QSV_MCP_PROFILE_CACHE_SIZE_MB` - Maximum cache size (default: 10 MB)
  - `QSV_MCP_PROFILE_CACHE_TTL_MS` - Cache TTL (default: 1 hour)
- **Profile-Aware Tool Guidance** - Tool descriptions now include 📊 hints recommending `qsv_data_profile` first

### Changed
- **Documentation Reorganization** - Consolidated markdown files into organized `docs/` subdirectories
  - `docs/guides/` - User guides (QUICK_START, CLAUDE_CODE, DESKTOP_EXTENSION, FILESYSTEM_USAGE)
  - `docs/reference/` - Technical reference (AUTO_UPDATE, CI, SKILLS_API, UPDATE_SYSTEM)
  - `docs/desktop/` - Desktop extension docs (README-MCPB)
  - Root now contains only README.md, README-MCP.md, CLAUDE.md, and CHANGELOG.md

## [15.1.1] - 2026-01-28

### Changed
- **Skill Version Sync** - Updated all 56 skill JSON files to version 15.1.1 to align with MCP server release
- **Documentation Update** - Revised README-MCP.md installation instructions to reference latest MCPB version 15.1.1
- `sniff` is now properly recognized as a METADATA_COMMAND (i.e. uses stdout as they're short outputs)

## [15.1.0] - 2026-01-27

### Changed
- **Simplified Tool Guidance** - Removed redundant feature requirement hints (Polars, Luau) from tool descriptions
- **DuckDB Fallback** - Added guidance to use DuckDB as an alternative when sqlp encounters errors with complex queries
- **Expanded Error Prevention** - Added cat, dedup, sort, and searchset to commands with common mistake warnings
- **Streamlined Descriptions** - Removed verbose optimization hints that are now handled automatically

## [15.0.0] - 2026-01-26

### Added
- **Tool Search Support** - New `qsv_search_tools` for discovering qsv commands by keyword, category, or regex
- **Expose-All-Tools Mode** - Auto-detects Claude clients (Desktop, Code, Cowork) for automatic tool exposure
- **US Census MCP Integration** - Census MCP server awareness with integration guides
- **Agent-Understandable Examples** - Improved usage examples optimized for AI agent comprehension

### Changed
- **Token Optimization** - 66-76% reduction in tool description token usage
- **Streaming Executor** - Uses `spawn` instead of `execFileSync` for better output handling
- **Output Size Limits** - 50MB stdout limit prevents memory issues on large outputs
- **Added `cat` and `geocode` to common tools** - More robust subcommand handling

### Fixed
- Critical MCP server stability errors resolved
- Windows EPERM retry logic with exponential backoff for file locking
- Pass working directory to spawned qsv processes
- Cross-platform test runner improvements

### Removed
- qsv UI commands that are not useful for MCP server (e.g., `lens`, `prompt`)

## [14.1.0] - 2026-01-13

### Added
- Versioned MCPB packaging - `.mcpb` files now include version in filename (e.g., `qsv-mcp-server-14.1.0.mcpb`)
- Version display in packaging script output for better release tracking
- Retry logic with exponential backoff for Windows EPERM errors in cache save operations

### Changed
- Optimized tool definition token consumption by 66-76% through more concise descriptions
- Removed `test_file` fields from skill JSON files to reduce token usage
- MCPB packaging script now reads version from `package.json` automatically

### Fixed
- Resolved critical MCP server errors affecting stability
- Removed redundant semicolons in code for cleaner formatting
- Fixed flaky Windows CI test by adding retry logic for file rename operations (EPERM errors)

## [14.0.0] - 2026-01-12

### Added
- **MCP Desktop Extension (MCPB)** - One-click installation bundle for Claude Desktop
  - Auto-detection of qsv binary path
  - Template variable expansion (`$HOME`, `${HOME}`) in configuration paths
  - Cross-platform support (macOS, Windows, Linux)
  - User-friendly installation workflow
- **Enhanced Tool Descriptions** - Intelligent guidance system for optimal tool selection
  - 💡 USE WHEN - Specific use-case recommendations
  - 📋 COMMON PATTERNS - Workflow patterns and command combinations
  - ⚠️ CAUTION - Memory limits, file size constraints, feature requirements
  - 🚀 PERFORMANCE - Index acceleration tips and cache strategies
- **Stats Cache Auto-Generation** - Automatically enables `--stats-jsonl` when running stats command
- **Production CI/CD** - Comprehensive testing across Node.js 20, 22, 24 on all platforms
- **Update Checker** - Monitors qsv binary versions and notifies of available updates
- **qsv Command Detection** - Automatically detects available qsv commands at runtime
- **Total Memory Context** - Exposes system memory information to help LLM with planning
- **Graceful Shutdown** - Proper cleanup and shutdown handling for MCP server
- **MCP Prompts** - Welcome message and examples prompt for better user onboarding
- Comprehensive documentation in `README-MCPB.md` for Desktop Extension
- Claude Code integration guide in `CLAUDE_CODE.md`

### Changed
- **Token Optimization** - Concise descriptions extracted from README command table instead of verbose USAGE text
- **Security Enhancement** - Replaced `execSync` with `execFileSync` to prevent command injection attacks
- Improved file metadata display when listing supported files
- Enhanced installer experience with better validation and error messages
- Optimized for local file access with improved directory restrictions
- More robust output processing using temp output files and intelligent stdout handling
- Improved error handling and validation throughout codebase
- Updated to `@modelcontextprotocol/sdk` v1.25.2

### Fixed
- Cross-platform test runner compatibility for Node.js 20
- Windows-specific CI test issues
- Promise-based deduplication for metadata cache to prevent race conditions
- Lock file cleanup in converted file cache system
- Template variable expansion in config paths
- Manifest.json compliance with MCP Bundle spec v0.3
- Various issues identified in GitHub Copilot reviews

### Removed
- `applydp` skill (deprecated command)
- Old Phase implementation comments from LIFO converted file cache
- Unnecessary null byte checks
- Unreachable code after cache size cap implementation

## [13.0.0 to 14.0.0] - Development Phase

### Added
- **Initial MCP Server Implementation** - Full Model Context Protocol server for qsv
  - 25 MCP Tools: 20 common commands + 1 generic + 1 pipeline + 3 filesystem tools
  - Natural language interface for all 67 qsv commands
  - Local file access with directory restrictions via `QSV_MCP_ALLOWED_DIRS`
- **Skills Auto-Generation System**
  - Integrated `qsv --update-mcp-skill` command (requires `mcp` feature flag)
  - Automatic parsing of qsv USAGE text using qsv-docopt
  - Performance hint extraction (📇 indexed, 🤯 memory-intensive, 😣 proportional)
  - Concise descriptions from README command table
- **Multi-Format Support**
  - Native: CSV, TSV, SSV, Snappy-compressed variants
  - Auto-converted: Excel (.xls, .xlsx, .xlsm, .xlsb), OpenDocument (.ods), JSONL/NDJSON
  - Transparent file conversion with format auto-detection
- **Pipeline System** - Chain multiple qsv commands into efficient workflows
  - Automatic intermediate file management
  - Automatic indexing between steps
  - Atomic operations with rollback on failure
  - Performance optimization
- **Converted File Manager**
  - LIFO (Last In, First Out) cache for converted files
  - Configurable cache size (default 1GB, range 0.1-100GB)
  - Automatic cleanup of oldest conversions
  - Conversion metadata tracking
- **Filesystem Tools**
  - `qsv_list_files` - List tabular data files in directories
  - `qsv_set_working_dir` - Change working directory
  - `qsv_get_working_dir` - Get current working directory
- **Resource Limits** - DoS prevention and resource management
  - Operation timeout (default 120s)
  - Max file listings (default 1000)
  - Max pipeline steps (default 50)
  - Max concurrent operations (default 10)
- **20 Common Command Tools**
  - `qsv_select`, `qsv_stats`, `qsv_frequency`, `qsv_search`, `qsv_sort`
  - `qsv_dedup`, `qsv_join`, `qsv_count`, `qsv_headers`, `qsv_slice`
  - `qsv_apply`, `qsv_rename`, `qsv_schema`, `qsv_validate`, `qsv_sample`
  - `qsv_moarstats`, `qsv_index`, `qsv_template`, `qsv_diff`, `qsv_cat`
- **Generic Command Tool** - `qsv_command` for remaining 47 qsv commands
- Configuration system with environment variables
- Automated installation script (`scripts/install-mcp.js`)
- MCPB packaging script (`scripts/package-mcpb.js`)
- Comprehensive test suite with cross-platform support
- Example scripts demonstrating usage patterns

### Changed
- Architecture optimized for local file access
- Improved qsv binary discovery with timeout
- Enhanced error messages for better troubleshooting
- More robust parameter handling and validation
- Smart pagination per MCP specification
- Deterministic example generation with IndexMap
- Shell-safe argument formatting for CLI commands

### Security
- Command injection prevention via `execFileSync`
- Directory access control with path validation
- Symlink resolution and `..` traversal prevention
- Resource limits to prevent DoS attacks
- Binary trust verification requirements

### Documentation
- Added comprehensive `README-MCP.md` for MCP Server
- Added `CLAUDE.md` with development guidelines
- Added `FILESYSTEM_USAGE.md` for file operations
- Added `AUTO_UPDATE.md` for update system
- Added design documentation in `docs/design/`
- Added API documentation for Agent Skills

## Version Numbering

The qsv Agent Skills project follows qsv's version numbering scheme. Version numbers match the qsv release they are designed to work with:

- **14.0.x** - Compatible with qsv 14.0.x
- **13.0.x** - Compatible with qsv 13.0.x

Patch versions (x.y.Z) may be released independently for bug fixes and minor improvements to the MCP server without requiring a qsv update.

## Links

- [qsv Repository](https://github.com/dathere/qsv)
- [MCP Specification](https://modelcontextprotocol.io/)
- [Claude Desktop](https://claude.ai/desktop)
- [NPM Package](https://www.npmjs.com/package/@qsv/agent-skills) (future)

---

**Note**: This changelog covers changes to the qsv Agent Skills / MCP Server component specifically. For changes to the main qsv toolkit, see the [qsv CHANGELOG](../../CHANGELOG.md).
