---
phase: 25.1-parser-robustness-security-hardening
plan: "02"
type: execute
wave: 2
depends_on:
  - "25.1-01"
files_modified:
  - src/parse.rs
  - TECH-DEBT.md
autonomous: true
requirements: []

must_haves:
  truths:
    - "CREATE<tab>SEMANTIC<tab>VIEW v (...) is detected and rewritten correctly"
    - "CREATE<newline>SEMANTIC<newline>VIEW v (...) is detected and rewritten correctly"
    - "All 7 DDL forms work with arbitrary inter-keyword ASCII whitespace (spaces, tabs, newlines, CRs, mixed)"
    - "Error positions remain accurate after token-based prefix matching (byte offsets not shifted)"
    - "TECH-DEBT.md item 4 is marked resolved"
    - "cargo test passes (all TEST-01 through TEST-08 green)"
    - "just test-all passes"
  artifacts:
    - path: "src/parse.rs"
      provides: "match_keyword_prefix helper + token-based detect_ddl_kind + decoupled callers"
      contains: "match_keyword_prefix"
      min_lines: 50
    - path: "TECH-DEBT.md"
      provides: "Item 4 marked resolved"
      contains: "✅ DDL prefix whitespace"
  key_links:
    - from: "detect_ddl_kind (src/parse.rs)"
      to: "match_keyword_prefix"
      via: "private fn detect_ddl_prefix returns Option<(DdlKind, usize)>"
      pattern: "detect_ddl_prefix"
    - from: "rewrite_ddl (src/parse.rs)"
      to: "detect_ddl_prefix"
      via: "replaces prefix_len(kind) call with usize from detect_ddl_prefix"
      pattern: "detect_ddl_prefix"
    - from: "validate_and_rewrite (src/parse.rs)"
      to: "detect_ddl_prefix"
      via: "replaces prefix_len(kind) call with usize from detect_ddl_prefix"
      pattern: "detect_ddl_prefix"
    - from: "extract_ddl_name (src/parse.rs)"
      to: "detect_ddl_prefix"
      via: "replaces prefix_len(kind) call with usize from detect_ddl_prefix"
      pattern: "detect_ddl_prefix"
---

<objective>
Implement token-based keyword matching in `detect_ddl_kind` and decouple all callers from the hardcoded `prefix_len()` function. Turn the RED proptest suite GREEN. Close TECH-DEBT.md item 4.

Purpose: The current `starts_with_ci` approach fails silently when tabs, newlines, carriage returns, or multiple spaces appear between CREATE, SEMANTIC, and VIEW (and across all 7 DDL prefixes). Token-based matching fixes this without normalizing the raw query bytes — byte offsets for error position reporting are preserved.

Output:
- `match_keyword_prefix(input: &[u8], keywords: &[&[u8]]) -> Option<usize>` private helper
- `detect_ddl_prefix(query: &str) -> Option<(DdlKind, usize)>` private helper returning (kind, consumed_bytes)
- `detect_ddl_kind` refactored to call `detect_ddl_prefix` (public API unchanged: `Option<DdlKind>`)
- All callers of `prefix_len(kind)` updated to use the dynamic byte count from `detect_ddl_prefix`
- `prefix_len` function removed (or kept private and narrowed if any remaining use)
- TECH-DEBT.md item 4 updated from ⚠ to ✅
</objective>

<execution_context>
@/Users/paul/.claude/get-shit-done/workflows/execute-plan.md
@/Users/paul/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/phases/25.1-parser-robustness-security-hardening/25.1-RESEARCH.md
@.planning/phases/25.1-parser-robustness-security-hardening/25.1-01-SUMMARY.md
@src/parse.rs
@TECH-DEBT.md

<interfaces>
<!-- Full function signature inventory for src/parse.rs callers of prefix_len -->
<!-- These 4 functions all call prefix_len(kind) and must be updated: -->

1. rewrite_ddl (line ~204):
   ```rust
   let plen = prefix_len(kind);
   ```
   Must become: call detect_ddl_prefix on the trimmed query, destructure the usize.

2. extract_ddl_name (line ~241):
   ```rust
   let plen = prefix_len(kind);
   ```
   Same pattern.

3. validate_and_rewrite (line ~550):
   ```rust
   let plen = prefix_len(kind);
   ```
   Same pattern.

4. parse_ddl_text (line ~151):
   ```rust
   let prefix_len = match kind { ... }  // local variable, hardcoded constants
   ```
   Must also switch to detect_ddl_prefix output.

<!-- Critical invariant: byte offsets must not shift -->
<!-- validate_and_rewrite uses trim_offset = query.len() - query.trim_start().len() -->
<!-- This trim_offset is ADDED to the plen to compute error positions. -->
<!-- After the fix: error_position = trim_offset + consumed_bytes_from_detect_ddl_prefix(trimmed_query) -->
<!-- The consumed_bytes is measured from the START of the trimmed_no_semi string, -->
<!-- just like the old prefix_len was. This is correct because detect_ddl_prefix -->
<!-- operates on the trimmed string (no leading whitespace). -->

<!-- The position_invariant_missing_name_drop proptest (TEST-03) checks: -->
<!-- pos == trim_offset + prefix_len ("drop semantic view".len() == 18) -->
<!-- After the fix, "drop semantic view" with single spaces = 18 bytes consumed -->
<!-- so the invariant is preserved for single-space inputs. -->
<!-- For "drop\tsemantic\tview" (tabs), plen would be 20, and the test uses -->
<!-- single spaces only, so the existing TEST-03 proptest still passes. -->
</interfaces>
</context>

<tasks>

<task type="auto" tdd="true">
  <name>Task 1: Implement match_keyword_prefix + detect_ddl_prefix, refactor detect_ddl_kind</name>
  <files>src/parse.rs</files>
  <behavior>
    - match_keyword_prefix(b"CREATE\tSEMANTIC\tVIEW rest", &[b"create", b"semantic", b"view"]) returns Some(22) — the number of bytes consumed by the prefix
    - match_keyword_prefix(b"CREATE  SEMANTIC  VIEW rest", same keywords) returns Some(26) — double spaces counted
    - match_keyword_prefix(b"SELECT * FROM t", &[b"create"]) returns None
    - match_keyword_prefix(b"", any keywords) returns None
    - detect_ddl_kind("CREATE\tSEMANTIC\tVIEW v (...)") returns Some(DdlKind::Create)
    - detect_ddl_kind("CREATE\nOR\nREPLACE\nSEMANTIC\nVIEW v (...)") returns Some(DdlKind::CreateOrReplace)
    - detect_ddl_kind("SHOW\tSEMANTIC\tVIEWS") returns Some(DdlKind::Show)
    - detect_ddl_kind("SELECT * FROM t") returns None
    - All existing TEST-01 through TEST-06 proptests continue to pass
  </behavior>
  <action>
    In src/parse.rs, insert BEFORE the existing `detect_ddl_kind` function:

    **1. Add `match_keyword_prefix` private helper:**

    ```rust
    /// Match a fixed sequence of keyword tokens at the start of `input`, tolerating
    /// arbitrary ASCII whitespace between tokens.
    ///
    /// Returns `Some(bytes_consumed)` if all keywords matched (case-insensitively),
    /// where `bytes_consumed` is the number of bytes consumed by the keyword prefix
    /// (including inter-keyword whitespace). Returns `None` otherwise.
    ///
    /// The match anchors at position 0. Leading whitespace in `input` is consumed
    /// as part of the match (counted in the returned byte count). If the caller has
    /// already trimmed leading whitespace, the returned count is from offset 0 of
    /// the trimmed slice.
    ///
    /// Anti-pattern avoided: does NOT scan at increasing offsets (no O(n²) behavior).
    /// If keyword[0] doesn't match at the start (after whitespace), returns None.
    ///
    /// Note: only handles ASCII whitespace (0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x20).
    /// Unicode whitespace is handled by DuckDB's StripUnicodeSpaces before the hook fires.
    fn match_keyword_prefix(input: &[u8], keywords: &[&[u8]]) -> Option<usize> {
        let mut pos = 0;
        for (i, &kw) in keywords.iter().enumerate() {
            // Skip ASCII whitespace (but not before the first keyword — caller is
            // responsible for leading whitespace; we skip INTER-keyword whitespace
            // only for i > 0).
            if i > 0 {
                // Require at least one whitespace character between keywords.
                if pos >= input.len() || !input[pos].is_ascii_whitespace() {
                    return None;
                }
                while pos < input.len() && input[pos].is_ascii_whitespace() {
                    pos += 1;
                }
            }
            // Match keyword case-insensitively.
            if input.len() < pos + kw.len() {
                return None;
            }
            if !input[pos..pos + kw.len()].eq_ignore_ascii_case(kw) {
                return None;
            }
            pos += kw.len();
        }
        Some(pos)
    }
    ```

    **2. Add `detect_ddl_prefix` private helper:**

    ```rust
    /// Detect the DDL kind and consumed prefix byte count from a query string.
    ///
    /// The input must already be trimmed of leading/trailing whitespace and
    /// trailing semicolons. Returns `Some((DdlKind, consumed_bytes))` where
    /// `consumed_bytes` is the number of bytes consumed by the matched prefix
    /// (including any inter-keyword whitespace in the input). Returns `None`
    /// if no prefix matches.
    ///
    /// Longest-first ordering prevents prefix overlap.
    fn detect_ddl_prefix(trimmed: &str) -> Option<(DdlKind, usize)> {
        let b = trimmed.as_bytes();

        // CREATE OR REPLACE SEMANTIC VIEW (5 keywords) — before CREATE SEMANTIC VIEW
        if let Some(n) = match_keyword_prefix(b, &[b"create", b"or", b"replace", b"semantic", b"view"]) {
            return Some((DdlKind::CreateOrReplace, n));
        }
        // CREATE SEMANTIC VIEW IF NOT EXISTS (6 keywords) — before CREATE SEMANTIC VIEW
        if let Some(n) = match_keyword_prefix(b, &[b"create", b"semantic", b"view", b"if", b"not", b"exists"]) {
            return Some((DdlKind::CreateIfNotExists, n));
        }
        // CREATE SEMANTIC VIEW (3 keywords)
        if let Some(n) = match_keyword_prefix(b, &[b"create", b"semantic", b"view"]) {
            return Some((DdlKind::Create, n));
        }
        // DROP SEMANTIC VIEW IF EXISTS (5 keywords) — before DROP SEMANTIC VIEW
        if let Some(n) = match_keyword_prefix(b, &[b"drop", b"semantic", b"view", b"if", b"exists"]) {
            return Some((DdlKind::DropIfExists, n));
        }
        // DROP SEMANTIC VIEW (3 keywords)
        if let Some(n) = match_keyword_prefix(b, &[b"drop", b"semantic", b"view"]) {
            return Some((DdlKind::Drop, n));
        }
        // DESCRIBE SEMANTIC VIEW (3 keywords)
        if let Some(n) = match_keyword_prefix(b, &[b"describe", b"semantic", b"view"]) {
            return Some((DdlKind::Describe, n));
        }
        // SHOW SEMANTIC VIEWS (3 keywords)
        if let Some(n) = match_keyword_prefix(b, &[b"show", b"semantic", b"views"]) {
            return Some((DdlKind::Show, n));
        }

        None
    }
    ```

    **3. Rewrite `detect_ddl_kind` to call `detect_ddl_prefix`:**

    Replace the current body of `detect_ddl_kind` (the 7 `starts_with_ci` branches) with:

    ```rust
    pub fn detect_ddl_kind(query: &str) -> Option<DdlKind> {
        let trimmed = query.trim().trim_end_matches(';').trim();
        detect_ddl_prefix(trimmed).map(|(kind, _)| kind)
    }
    ```

    The public API is unchanged: still returns `Option<DdlKind>`. The `starts_with_ci` function can remain (it is still used in other places). Do NOT remove it.

    Run `cargo test` after this step — TEST-01 through TEST-06 should pass, TEST-07 should now PASS (GREEN), TEST-08 adversarial tests should pass. If position invariant tests (TEST-03) fail, see the note in the interfaces section about trim_offset.
  </action>
  <verify>
    <automated>cargo test -- --test-output immediate 2>&1 | tail -20</automated>
  </verify>
  <done>cargo test passes. detect_ddl_kind uses detect_ddl_prefix. match_keyword_prefix exists in src/parse.rs. TEST-07 proptest is GREEN.</done>
</task>

<task type="auto">
  <name>Task 2: Decouple all prefix_len callers + close TECH-DEBT.md item 4</name>
  <files>src/parse.rs, TECH-DEBT.md</files>
  <action>
    **Part A: Update callers of `prefix_len` in src/parse.rs**

    Four call sites use `prefix_len(kind)` to compute how many bytes to slice past the detected prefix. All must be updated to use `detect_ddl_prefix` instead.

    The pattern for each update is:

    ```rust
    // OLD (before this task):
    let kind = detect_ddl_kind(trimmed).ok_or_else(|| "...")?;
    let plen = prefix_len(kind);

    // NEW (after this task):
    let (kind, plen) = detect_ddl_prefix(trimmed).ok_or_else(|| "...")?;
    ```

    Apply this transformation to ALL four call sites:

    **1. `rewrite_ddl` (~line 204):**
    ```rust
    // Replace:
    let kind = detect_ddl_kind(trimmed).ok_or_else(|| "Not a semantic view DDL statement".to_string())?;
    let fn_name = function_name(kind);
    let plen = prefix_len(kind);

    // With:
    let (kind, plen) = detect_ddl_prefix(trimmed).ok_or_else(|| "Not a semantic view DDL statement".to_string())?;
    let fn_name = function_name(kind);
    ```

    **2. `extract_ddl_name` (~line 241):**
    ```rust
    // Replace:
    let kind = detect_ddl_kind(trimmed).ok_or_else(|| "Not a semantic view DDL statement".to_string())?;
    let plen = prefix_len(kind);

    // With:
    let (kind, plen) = detect_ddl_prefix(trimmed).ok_or_else(|| "Not a semantic view DDL statement".to_string())?;
    ```

    **3. `validate_and_rewrite` (~line 550):**
    ```rust
    // Replace:
    let Some(kind) = detect_ddl_kind(query) else { return Ok(None); };
    let plen = prefix_len(kind);

    // With:
    let trimmed_for_detect = query.trim().trim_end_matches(';').trim();
    let Some((kind, plen)) = detect_ddl_prefix(trimmed_for_detect) else { return Ok(None); };
    ```
    Note: `validate_and_rewrite` already computes `trim_offset` and `trimmed_no_semi`. The detect must operate on the trimmed string so that `plen` is relative to the start of the trimmed content (same as before). The error position formula `trim_offset + plen` remains correct.

    **4. `parse_ddl_text` (~line 151):**
    ```rust
    // Replace the local variable prefix_len computation:
    let kind = detect_ddl_kind(trimmed).ok_or_else(|| "...")?;
    let prefix_len = match kind { ... hardcoded lengths ... };

    // With:
    let (kind, prefix_len) = detect_ddl_prefix(trimmed).ok_or_else(|| "Not a CREATE SEMANTIC VIEW statement".to_string())?;
    if !matches!(kind, DdlKind::Create | DdlKind::CreateOrReplace | DdlKind::CreateIfNotExists) {
        return Err("Not a CREATE SEMANTIC VIEW statement".to_string());
    }
    ```

    **After all callers are updated:**

    Remove or mark the `prefix_len` function as dead code. Prefer removal — if any test or code still needs it, the compiler will report it. If `starts_with_ci` has no remaining callers either, note it but leave it in place (it is private and harmless).

    Confirm `cargo test` still passes after this change (all tests green, including TEST-03 position invariants).

    **Part B: Update TECH-DEBT.md**

    In TECH-DEBT.md, under `## Test Coverage Gaps`, find item 4:

    ```markdown
    ### 4. ⚠ DDL prefix whitespace not normalized (`detect_ddl_kind`)
    ```

    Change the marker from `⚠` to `✅` and update the content to reflect resolution:

    Replace the body with:
    ```markdown
    ### 4. ✅ DDL prefix whitespace — RESOLVED in Phase 25.1

    - **Origin:** Phase 25 proptest surfaced this
    - **Resolution:** Phase 25.1 replaced `starts_with_ci` literal prefix matching with
      `match_keyword_prefix` token-based scanning. All 7 DDL prefix forms now tolerate
      arbitrary ASCII whitespace (space, tab, newline, carriage return, vertical tab,
      form feed) between keywords. The `prefix_len()` static function was replaced by
      the dynamic byte count returned by `detect_ddl_prefix(query)`.
    - **Scope:** ASCII whitespace only (6 characters: 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x20).
      Unicode whitespace is handled by DuckDB before the hook fires.
    ```

    Also update the date line at the bottom of TECH-DEBT.md to `2026-03-12` if not already current.

    Run `just test-all` as the final gate for this plan.
  </action>
  <verify>
    <automated>cargo test && echo "cargo test PASSED" && just test-all && echo "just test-all PASSED"</automated>
  </verify>
  <done>All 4 prefix_len call sites updated to use detect_ddl_prefix. prefix_len function removed from src/parse.rs (or confirmed no longer called). TECH-DEBT.md item 4 marked ✅. cargo test passes. just test-all passes.</done>
</task>

</tasks>

<verification>
Full phase gate:
1. `cargo test` — all tests pass including TEST-07 (whitespace proptests now GREEN) and TEST-08 (adversarial tests)
2. `just test-all` — Rust unit + proptest + sqllogictest + DuckLake CI all pass
3. `grep -n "match_keyword_prefix\|detect_ddl_prefix" src/parse.rs` — both helpers present
4. `grep -n "prefix_len" src/parse.rs` — no remaining calls to the old prefix_len function (function itself either removed or dead)
5. `grep "✅ DDL prefix whitespace" TECH-DEBT.md` — item 4 marked resolved
</verification>

<success_criteria>
1. `cargo test` passes (including new TEST-07 proptest covering all 7 DDL forms with tab/newline/mixed inter-keyword whitespace)
2. `just test-all` passes (Rust + SQL + DuckLake CI)
3. detect_ddl_kind("CREATE\tSEMANTIC\tVIEW v (...)") returns Some(DdlKind::Create)
4. validate_and_rewrite("DROP\nSEMANTIC\nVIEW v") returns Ok(Some("SELECT * FROM drop_semantic_view('v')"))
5. Error positions remain accurate: position_invariant_* proptests (TEST-03) still pass
6. TECH-DEBT.md item 4 is ✅ resolved
7. No new Cargo dependencies added
</success_criteria>

<output>
After completion, create `.planning/phases/25.1-parser-robustness-security-hardening/25.1-02-SUMMARY.md`
</output>
