# Markdown Hybrid Search MCP — Design (v1)

## Summary

This subproject will implement an MCP server that builds a **one-shot, ephemeral local index** for Markdown corpora under one or more root folders provided via CLI arguments. The server provides **hybrid retrieval** by combining:

- **Vector retrieval** over semantic embeddings, and
- **Full-text retrieval** (BM25-style) over tokenized text.

The system is optimized for repeated queries during a single server run. It avoids brute-force full rescans per query.

This is the **first** version of the system and should remain **minimal**: prioritize correctness and a small feature set over extensibility.

Implementation constraints (v1):

- Implementation language/runtime: **Python**.
- MCP SDK/framework: **FastMCP**.

## Goals

- Provide an MCP server that can be started with one or more corpus roots.
- Discover Markdown documents under those roots and build a local **temporary** index at startup.
- Support hybrid search with sensible defaults.
- Address **document length imbalance** so long documents do not dominate results.
- Handle Markdown that embeds code blocks (no multimodal/image support).

## Non-goals

- Image/multimodal indexing.
- Remote hosted databases or distributed indexing.
- Complex access control/tenanting.

## Interface

### CLI / Process arguments

The server process will accept:

- `--root <path>`: repeatable. One or more folder roots to scan.
    - Startup validation (v1): each `--root` MUST exist at startup, otherwise the server MUST exit with an error.
- `--work-dir <path>`: optional parent directory for temporary index storage (default: OS temp).
    - The server MUST create a unique, process-owned subdirectory under `--work-dir`.
    - On process exit, the server MUST attempt best-effort deletion of the process-owned subdirectory.
    - The server MUST NOT delete user-owned directories.

Azure OpenAI (Entra ID auth):

- `--azure-openai-endpoint <https://...>`
- `--azure-openai-embedding-deployment <deployment>`
- `--azure-openai-rewrite-deployment <deployment>`
- `--azure-openai-api-version <YYYY-MM-DD|preview>`
- Optional: `--azure-tenant-id <tenant-guid>`

Azure OpenAI API surface (v1):

- Use the **Responses API** for query rewrite.
- The rewrite stage MUST use **JSON output** (schema-constrained; no surrounding prose).
- The rewrite deployment MUST be backed by **GPT-5.2** (`gpt-5.2`, version `2025-12-11`).
- The server MUST fail at startup if the configured rewrite deployment/API does not support **structured outputs** (JSON schema, strict mode).
- API version note (v1): the Responses API examples use the `/openai/v1/` endpoint with `api-version=preview`. For v1, `--azure-openai-api-version` SHOULD be set to `preview`.

Compatibility envelope (v1, design intent):

- DuckDB: `duckdb >= 1.4.3` with `fts` and `vss` extensions available; no fallback.
- Rewrite: Azure OpenAI Responses API + structured outputs; rewrite deployment is `gpt-5.2`.
- Startup fail-fast: if DuckDB capabilities or structured outputs are not available, exit before serving MCP tools.

Platform support (v1):

- v1 MUST support Windows from day 1.

Discovery parameters (v1):

- v1 does not provide include/exclude glob flags.
- Discovery uses `fd -e md` and therefore indexes `*.md` files only.
    - V1 intentionally excludes `.markdown`, `.mdx`, and other Markdown-like extensions.

Required in v1:

- `--azure-openai-embedding-deployment` (embeddings)
- `--azure-openai-rewrite-deployment` (LLM-assisted query rewrite/expansion)

Discovery rules (v1):

- Always ignore `.git/` directories.
- Ignore behavior is delegated to the installed `fd` implementation.
- Follow symlinks/junctions during enumeration.
- Enforce `--root` folders as hard boundaries on **real paths** (see “Symlink boundary rule”).
- Deduplicate files by normalized real path.

Implementation note (v1): record `fd --version` and the exact `fd` invocation in logs and/or `index_meta`.

Symlink boundary rule (v1):

- Follow symlinks/junctions, but only index a file if its `realpath` is within at least one provided `--root`.
- The stored/returned `path` is the **normalized real path** (as defined by `normalized_real_path`).
- Path-based filters such as `roots` and `path_prefix` apply to the normalized real path.

"Within root" definition (v1):

- Let `R` be the normalized real path of a root, and `P` be the normalized real path of a candidate.
- The candidate is within the root iff either:
    - `P == R`, OR
    - `R` is an ancestor directory of `P` on a path-segment boundary.
- This must be implemented as a path-component comparison (not a naive string prefix test).

Credential preference (resolved requirement):

- Prefer Azure CLI > Interactive Browser > Device Code.
- Do not attempt Managed Identity.

### MCP tools

Expose an MCP tool set (names are placeholders; final names should align with repo conventions):

1. `search`
    - Input: `query: str`, `top_k: int`, optional filters (`roots`, `path_prefix`, `updated_after`, etc.)
    - Output: ranked results containing (at minimum):
        - `doc_id`, `path`, `title` (if derivable), `snippet`, `score`.
        - Channel diagnostics (recommended for debuggability):
            - `text_rank` (nullable), `text_bm25` (nullable)
            - `vec_rank` (nullable), `vec_distance` (nullable)

    Filter semantics (v1):
    - `updated_after` (if provided) MUST be an RFC3339 timestamp with a timezone.
        - Inputs without a timezone MUST be rejected.
        - Inputs with a timezone offset (e.g., `+08:00`) MUST be accepted and normalized to UTC before comparison.
    - The filter matches documents with `mtime_unix_ms > updated_after_unix_ms`.
    - Filter application point (v1): `roots`, `path_prefix`, and `updated_after` MUST be applied inside the DuckDB candidate queries (FTS and VSS), not only after union/fusion.
    - `path_prefix` must use a path-segment boundary rule on the normalized real path (e.g., `path = prefix OR path starts with prefix + '/'`).
        - The server MUST normalize `path_prefix` inputs (including OS-native separators on Windows) to match the indexed normalized real path representation.
    - Time is compared against the indexed snapshot (not live filesystem reads).

2. `get_document`
    - Input: `doc_id` or `path`
    - Output: stored metadata and optionally raw Markdown content (or a content window).
    - Defaults and limits (v1):
        - Content format: raw Markdown.
        - Optional windowing parameters (recommended): `offset_chars` (default: 0) and `max_chars`.
        - Hard cap: `max_chars <= 200000`.
        - Include `truncated: bool` and `returned_chars: int` in the response when content is returned.
    - Consistency (v1):
        - `get_document` returns content from the indexed snapshot (not live filesystem reads).

3. `stats`
    - Output: index stats (doc count, last indexing time, embedding deployment id, embedding dimensions, etc.).

## Data model

### Document-only indexing

For v1, the system indexes at **document** granularity only.

- Document: a Markdown file.
- Record metadata: `doc_id`, `path`, `mtime`, `size`, plus derived fields used for search.

Snapshot content (v1):

- The raw Markdown content is stored as `content_md` in the index.
- All derived text fields (`text_all`, `text_no_code`, `code_signals`) are produced from `content_md` during indexing.

`doc_id` (v1):

- `doc_id` is the SHA-256 hex digest of the UTF-8 bytes of `normalized_real_path`.
- `normalized_real_path` is defined as:
    1. `realpath(path)` (after following symlinks/junctions)
    2. Convert path separators to `/`
    3. On Windows: case-fold the full path (equivalent to `normcase`)
    4. Apply Unicode normalization (NFC)

Derived fields (recommended):

- `content_md`: raw Markdown snapshot of the document (source of truth for `get_document`).
- `text_all`: Markdown rendered/normalized to plain text, including code blocks.
- `text_no_code`: a code-light representation for semantic embeddings.
- `code_signals`: a deterministic, compact representation of code blocks (commands, flags, errors, identifiers).
- `title`: best-effort title (e.g., first H1), optional.

### Storage backends

We need both full-text and vector indexes.

- **Unified storage (v1, no fallback)**: DuckDB (>= v1.4.3; validated by startup self-test) in an owned temporary directory for the lifetime of the process. Use DuckDB extensions to support full-text search and vector similarity search (`fts` and `vss`).
- It is allowed to download/enable DuckDB extensions at runtime even if that requires network access (including on Windows).
- If DuckDB FTS/vector capabilities cannot be enabled/used, the server should **exit with an error** (no alternative backends in v1).

DuckDB compatibility contract (v1):

- The implementation must perform a startup self-test:
    - create/open the DuckDB database in the process-owned work directory,
    - install/load required extensions,
    - create schema,
    - insert 1–2 synthetic documents (including one synthetic embedding vector),
    - execute one FTS query and one VSS query.
- If any step fails, the server must exit with an error before serving MCP tools.
- Any SQL examples below are **illustrative** unless explicitly marked “validated in self-test”.
- The SQL and schema used by the running server MUST be validated by (and kept consistent with) the startup self-test. If the environment does not support the validated shape (extensions, types, functions), the server MUST exit with an error.
- The server SHOULD record in `index_meta`:
    - DuckDB engine version,
    - loaded extension versions/status (fts/vss),
    - a stable identifier for the validated DDL/query templates (e.g., a hash).
- The server SHOULD validate that the vector query uses the intended index access path. Plan-string matching is brittle; prefer an allow-list of acceptable plan node labels and a deny-list for obvious full scans.

DuckDB schema and query shape (v1):

- Extensions:
    - `INSTALL fts; LOAD fts;`
    - `INSTALL vss; LOAD vss;`
- Tables (minimum):
    - `docs` (document-level, snapshot-based):
        - `doc_id VARCHAR PRIMARY KEY`
        - `path VARCHAR`
        - `mtime_unix_ms BIGINT`
            - Document mtime captured at indexing time from filesystem metadata.
            - Stored in UTC as Unix milliseconds since epoch.
        - `size BIGINT`
        - `title VARCHAR`
        - `content_md VARCHAR` (raw Markdown snapshot)
        - `text_all VARCHAR` (plain-text view used for FTS)
        - `text_no_code VARCHAR` (code-light view used for embeddings)
        - `code_signals VARCHAR` (compact signals extracted from code blocks)
        - `embedding FLOAT[3072]`
            - `3072` is required for `text-embedding-3-large`.
            - Dimension mismatch MUST be treated as a fatal error (fail fast).
    - `index_meta` (single row): corpus roots, build timestamp, embedding deployment name, dimension, counts, plus `fd` version and discovery invocation.

Schema placement (v1):

- The implementation SHOULD create the `docs` table in schema `main`.
- This keeps DuckDB FTS-generated schema names stable/predictable (e.g., for `main.docs`, the FTS schema is `fts_main_docs`).

Validated SQL invariant (v1):

- The server MUST use the exact same DDL and query templates as the startup self-test.
- If the environment cannot reproduce the validated capabilities (extensions, types, functions, index scan shape), the server MUST exit with an error before serving MCP tools.
- Full-text search (FTS): - Create index:
  `PRAGMA create_fts_index('main.docs', 'doc_id', 'title', 'text_all', stopwords = 'none', stemmer = 'none', lower = 1, strip_accents = 1, ignore = '([^0-9A-Za-z_\-./:\+#=@])+' );` - Note (v1): the ignore pattern is tuned to preserve developer-centric tokens (e.g., `C++`, `C#`, `key=value`, `@decorator`). - Note (v1): tokenization remains intentionally tuned for ASCII/English and code-like tokens. Non-ASCII text may be dropped or tokenized poorly. - Query candidates:
  `SELECT doc_id, score FROM (SELECT *, fts_main_docs.match_bm25(doc_id, :q) AS score FROM main.docs) WHERE score IS NOT NULL ORDER BY score DESC LIMIT :k_text;`
- Vector search (VSS): - Create HNSW index:
  `CREATE INDEX docs_embedding_hnsw ON main.docs USING HNSW (embedding) WITH (metric = 'cosine');` - Query candidates (cosine metric requires `array_cosine_distance`):
  `SELECT doc_id, array_cosine_distance(embedding, :qvec::FLOAT[3072]) AS distance FROM main.docs ORDER BY distance ASC LIMIT :k_vec_oversample;` - The startup self-test MUST validate that the HNSW index is used for the validated query shape (e.g., `EXPLAIN` includes an `HNSW_INDEX_SCAN` node).

Vector filtering with ANN (v1):

- To preserve HNSW index usage while still applying `roots/path_prefix/updated_after` during candidate generation, v1 MUST use ANN oversampling: - compute `k_vec_oversample = k_vec * N`, - run ANN for `k_vec_oversample`, - apply filters, then truncate back to `k_vec`.
- Fixed constant (v1): `N = 10`.

Filter requirement (v1): the running server MUST apply `roots/path_prefix/updated_after` constraints inside both candidate queries above (FTS and VSS), rather than filtering only after fusion.

## Index build pipeline

1. **Discovery**
    - Discovery is performed by invoking `fd` (rather than implementing recursive traversal in-process).
        - For each `--root`, run a command equivalent to:
          `fd --type file --follow --hidden --exclude .git -e md . <root>`
    - Rationale:
        - `fd` follows symlinks/junctions (`--follow`).
        - Ignore behavior is delegated to the installed `fd` implementation.
        - `--exclude .git` enforces the “always ignore `.git/`” rule.
    - Guardrails (v1):
        - V1 does not impose a hard cap on discovered files.
        - The server SHOULD log the discovered file count before indexing starts, and SHOULD warn if the corpus is exceptionally large.
        - If `fd` is not available on PATH, exit with an error.
        - If any provided `--root` does not exist, exit with an error.
    - Precedence (v1):
        - The server does not attempt to override or re-implement ignore semantics.
    - Root boundary enforcement (v1):
        - After enumeration, compute `realpath` for each candidate.
        - Skip a candidate if its `realpath` is not within at least one provided `--root`.
        - Define “within root” using the same normalization rules as `normalized_real_path` applied to both the candidate and each root (i.e., compare normalized real paths with a path-segment boundary; see “Within root” definition above).
    - Deduplicate files by **normalized real path**.
    - Record `path` (normalized real path), `mtime`, `size`, and a stable `doc_id` (SHA-256 of normalized real path).
    - Preflight (v1, recommended): before any Azure embedding calls begin, scan the discovered file metadata and fail fast if any file exceeds the size threshold (e.g., > 10MB).

2. **Parsing (document-level)**
    - Fail-fast on very large files.
        - Recommended threshold (v1): if file size is > 10MB (raw bytes), treat this as a fatal indexing error.
        - Because v1 is no-partial-index, the process MUST exit with an error (do not silently skip).
    - Read the raw Markdown file and store it as `content_md` (snapshot).
        - File decoding policy (v1): decode Markdown as UTF-8.
        - Decode errors MUST be treated as fatal (fail the indexing run).
    - Derive all other text views from `content_md`.
    - Produce two derived text views:
        - `text_all`: includes code blocks.
        - `text_no_code`: excludes (or strongly down-weights) fenced code blocks.
    - Extract deterministic `code_signals` from fenced code blocks.
    - Optional (future): store code blocks with stable identifiers for separate retrieval/display.
      For v1, code blocks are retrievable via `content_md` and do not require a separate table.

    Markdown handling (v1):
    - Use a CommonMark-compatible Markdown parser/library.
    - Indented code blocks: best effort. If detected, include them in `text_all` and optionally in `code_signals`; otherwise treat as normal text.

    `code_signals` extraction (v1, minimal):
    - Source: fenced code blocks in `content_md`.
    - For each fenced code block:
        - Include the info string (language) if present.
        - Extract tokens and normalize:
            - identifiers: `[A-Za-z_][A-Za-z0-9_\-\.]*`
            - CLI flags: `--[A-Za-z0-9\-]+`
            - error-like tokens (best effort): `0x[0-9A-Fa-f]+`, `E\d+`, `Exception`, `Traceback`
        - Sort + de-duplicate tokens to ensure determinism.

    `code_signals` size caps (v1):
    - To avoid pathological growth on log-heavy code blocks, the implementation SHOULD cap:
        - maximum distinct tokens stored (e.g., 2000), and/or
        - maximum total characters stored (e.g., 200000).
    - If capped, the implementation SHOULD record that truncation occurred (e.g., in `index_meta` or per-doc metadata).

3. **Embedding (document-level)**
    - Compute embeddings using Azure OpenAI embeddings.
    - Default embedding input: `text_no_code`.
    - For `mode=mixed` (future option): embed `text_no_code + code_signals`.
    - Store embedding model identity and dimensions in index metadata for observability.

    Embedding model (v1):
    - The embedding deployment must be backed by **text-embedding-3-large**.

    Embedding context limit (v1):
    - The embedding model context length is **8191 tokens**.
    - Token counting should use the model-compatible tokenizer (e.g., `tiktoken` with `cl100k_base`).

    Length imbalance mitigation for embeddings (no chunking):
    - Embed a **length-normalized representation** of the document (cap tokens/chars).
    - Suggested strategy: keep headings + first N tokens of prose (optionally plus tail), drop or down-weight code.

    Embedding input construction (v1 defaults):
    - Build `text_no_code` from the Markdown document with fenced code blocks removed (or strongly down-weighted).
    - Exclude YAML front matter (best effort) to reduce noise.
    - Construct a bounded `embedding_input` with a token budget (using `tiktoken` `cl100k_base`):
        - Total cap: **6000 tokens**.
        - Head: **5000 tokens** from the start (prefer title/headings, then prose blocks).
        - Tail: **1000 tokens** from the end (to capture conclusions/FAQs).
    - Because the embedding input is capped below 8191 tokens, v1 does not require any additional over-limit splitting logic.

4. **Full-text indexing (document-level)**
    - Index `text_all` for keyword lookup (error messages, commands, API identifiers).

5. **Vector indexing (document-level)**
    - Index document embeddings.

6. **No incremental updates (v1)**
    - Build once at startup; do not watch or update during the process lifetime.

7. **Failure policy (v1)**
    - Retry Azure OpenAI calls several times (with exponential backoff + jitter).
    - If indexing still fails after retries, **exit with error** (do not build a partial index).
    - Because v1 does not impose a hard corpus cap, the implementation MUST be robust under throttling:
        - limit concurrent embedding requests,
        - treat rate limiting as a first-class retryable failure.

## Handling length imbalance (bias control)

Problem: if documents have very different lengths, vector embeddings computed from raw long text can become less representative, and text scoring can skew toward verbose documents.

Mitigations (document-level):

1. **Length-normalized embedding input**
    - Do not embed the entire file verbatim.
    - Use a bounded representation (headings + capped prose) so long documents do not dilute semantics.

2. **Length-aware scoring adjustment (optional)**
    - If needed, apply a mild penalty based on document length:
      $$\text{score} = \text{hybrid\_score} \cdot \frac{1}{1 + \alpha \log(1 + \text{doc\_chars})}$$

3. **Diversification (optional)**
    - Apply a light MMR-like diversification step over final results.

## Hybrid retrieval strategy

### Candidate generation

- Retrieve `k_text` candidates from full-text search over documents.
- Retrieve `k_vec` candidates from vector search over document embeddings.

Defaults (v1):

- `top_k = 10`
- `k_text = 50`
- `k_vec = 50`

VSS oversampling (v1):

- `k_vec_oversample = k_vec * 10`

### Coarse reranking via RRF (rank fusion)

Use Reciprocal Rank Fusion (RRF) to combine multiple ranked lists (e.g., full-text rank list and vector rank list) into a single coarse ranking.

For each candidate document $d$, compute:

$$\mathrm{rrf}(d) = \sum_{i \in \{\text{text},\text{vec}\}} \frac{1}{k + r_i(d)}$$

Where:

- $r_i(d)$ is the 1-based rank of document $d$ in channel $i$ (missing documents contribute 0 for that channel),
- $k$ is a smoothing constant (typical values are 50–60).

Notes:

- RRF is robust when channel score scales differ and avoids manual weight tuning.
- This design does not use a separate rerank model for fine reranking.

Default RRF constant (v1):

- `k = 60`

Final scoring contract (v1):

- `score` in tool output is the final fused score (higher is better), computed as RRF across available channels.
- Optional penalties (e.g., length-aware penalty) MUST be applied only after fusion and should be documented if enabled.

### Result shaping

- Results are already document-level.
- Provide a snippet derived from `text_all` around the best-matching region (implementation detail).
- Snippet size limit (v1 default): **<= 800 characters**.

Snippet construction (v1):

- Use the rewritten keyword query (from the LLM rewrite stage) to build a token list.
- Find the first occurrence of any token (case-insensitive) in `text_all`.
- Return an 800-character window centered around the first hit (best effort).
- If no token is found, return the first 800 characters of `text_all`.

## Code blocks

- Code blocks are indexed in `text_all` for full-text search.
- For vector search, compute embeddings from `text_no_code` by default.
- Additionally compute and store `code_signals` (deterministic, local extraction).
- For v1, code blocks are retrievable from the stored `content_md` snapshot and do not require separate `code_block_id` pointers.
- Provide (future) query option `mode: {"docs","code","mixed"}`. Semantics: `docs` emphasizes prose; `mixed` includes code signals; `code` emphasizes code signals and relies on full-text for exact matching.

## Observability & operations

- Maintain an `index_meta` record:
    - corpus roots,
    - build timestamp,
    - embedding deployment id/name, dimension,
    - counts.
    - recommended additions:
        - duckdb version,
        - loaded extensions + versions (fts/vss),
        - `fd --version` and the exact discovery invocation,
        - rewrite deployment name,
        - validated schema/query template identifier (e.g., a hash) to ensure runtime and self-test stay aligned.
- Emit structured logs for:
    - index build duration,
    - docs discovered,
    - docs indexed,
    - embedding calls and retry counts,
    - query latency breakdown (fts, vector, fusion).

Operational visibility requirement (v1):

- Because v1 does not impose a hard cap on discovery/embedding, the server SHOULD surface cost/scale early:
    - log discovered doc count before embedding begins,
    - log an estimate of embedding input size (e.g., total chars; optional approximate tokens).

## LLM-assisted query rewrite (required, minimal)

Use Azure OpenAI **Responses API**. For v1, keep this minimal and use it for **query rewrite / expansion**.

Rewrite deployment requirements (v1):

- The rewrite deployment MUST be backed by **GPT-5.2** (`gpt-5.2`, version `2025-12-11`).
- The implementation MUST enforce schema-constrained JSON output using structured outputs (JSON schema, strict mode), not prompt-only.
- Startup fail-fast: before serving MCP tools, the server MUST run a small rewrite preflight that verifies structured outputs work end-to-end (request accepted, output parses, and schema constraints are respected). If the preflight fails, the server MUST exit with an error.

Model identity verification (v1):

- The server MUST NOT depend on being able to prove which model a deployment is backed by.
- A successful structured-output preflight (schema-valid JSON output) is sufficient for capability verification.

Structured outputs schema constraints (v1):

- The JSON schema used for rewrite MUST conform to the structured outputs subset (e.g., `additionalProperties: false` for objects; all fields required).
- The server SHOULD validate the schema locally at startup to avoid hard-to-debug runtime failures.

Stages:

1. **Query rewrite / expansion**
    - Turn the user query into:
        - a keyword query for full-text, and
        - a semantic query for embeddings.
    - Expand abbreviations and normalize product/resource names.
    - v1 requirement: the rewrite stage MUST translate the query to **English**.
    - User input queries MAY be any language.

Rewrite output contract (v1):

- The model MUST output JSON only (no surrounding prose).
- The implementation MUST enforce this via schema-constrained JSON output (not prompt-only enforcement).
- Proposed schema (strict):
    - `keyword_query: string` (English; max 256 chars)
    - `semantic_query: string` (English; max 512 chars)
    - `keywords: string[]` (required; 0..16 items; English; each max 32 chars)
    - `language: "en"` (constant)

Determinism and validation (v1 defaults):

- Set temperature to 0 (or the lowest supported value).
- Validate JSON strictly. If parsing fails, retry with a repair prompt up to 2 times.
- Reject outputs exceeding the length caps.

Data minimization principle:

- It is acceptable (privacy/compliance) to send messages to Azure OpenAI. Prefer sending only the user query and small candidate snippets when possible.

Failure handling (v1):

- If query rewrite/expansion fails, `search` must return a tool error (no fallback to using the raw query).
- The error should clearly indicate that the rewrite stage failed (distinct from indexing/storage failures), and logs should include enough detail to troubleshoot.

## Risks and constraints

- DuckDB full-text/vector capabilities and packaging may require careful dependency choices on Windows.
- Azure OpenAI embeddings and chat models require network connectivity and Entra ID credentials.
- One-shot indexing means startup time is the key UX constraint.

## Lifecycle

- The index must be **ephemeral** and stored under a temporary directory.
- The server creates and owns a unique work directory for the process lifetime.
    - If `--work-dir` is provided, the work directory is a subdirectory under `--work-dir`.
    - Otherwise, the work directory is a subdirectory under the OS temp directory.
- On process exit, the server MUST attempt best-effort deletion of the process-owned work directory.
- The server MUST NOT delete user-owned directories.

## Corpus observations (sample)

These are sample statistics collected via a CLI sampling pass (see `CLARIFY_v1_0.md` for exact figures and method). The key takeaway is that fenced code blocks are common, so code-aware indexing improves both keyword and semantic retrieval.

## Milestones

1. Scaffolding (this change): project layout + design.
2. Index primitives: metadata + discovery + document parsing.
3. Full-text index (document-level).
4. Vector index backend + Azure OpenAI embeddings.
5. Hybrid fusion + bias control + MCP tools.
