---
phase: 23-parser-proptests-and-caret-integration-tests
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - tests/parse_proptest.rs
autonomous: true
requirements:
  - TEST-01
  - TEST-02
  - TEST-03
  - TEST-04
  - TEST-05

must_haves:
  truths:
    - "DDL detection is case-insensitive for all 7 forms under random case variation"
    - "DDL rewrite produces correct function call for all 7 forms with random view names"
    - "Validation error positions always point to correct byte in original query regardless of leading whitespace"
    - "Near-miss detection does not false-positive on normal SQL statements"
    - "Bracket validation handles nested structures with embedded strings correctly"
  artifacts:
    - path: "tests/parse_proptest.rs"
      provides: "Property-based tests for all public parser functions"
      min_lines: 200
  key_links:
    - from: "tests/parse_proptest.rs"
      to: "src/parse.rs"
      via: "use semantic_views::parse::*"
      pattern: "use semantic_views::parse"
---

<objective>
Create property-based tests for the parser module's public API functions using proptest.

Purpose: The parser module (`src/parse.rs`) has 79 hand-written unit tests covering specific cases, but does not exercise the combinatorial space of whitespace variations, case mixing, special characters in view names, or edge-case lengths. Proptest will systematically fuzz all 7 public functions with randomized inputs following the established pattern from `expand_proptest.rs`.

Output: `tests/parse_proptest.rs` with property-based tests covering detection, rewriting, position invariants, near-miss safety, and bracket validation.
</objective>

<execution_context>
@/Users/paul/.claude/get-shit-done/workflows/execute-plan.md
@/Users/paul/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/23-parser-proptests-and-caret-integration-tests/23-RESEARCH.md

@tests/expand_proptest.rs (established proptest pattern in this codebase)
@src/parse.rs (module under test)
@src/lib.rs (module exports — `pub mod parse` available without feature gate)

<interfaces>
<!-- Key types and contracts the executor needs from src/parse.rs -->

```rust
pub const PARSE_NOT_OURS: u8 = 0;
pub const PARSE_DETECTED: u8 = 1;

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum DdlKind {
    Create,
    CreateOrReplace,
    CreateIfNotExists,
    Drop,
    DropIfExists,
    Describe,
    Show,
}

pub struct ParseError {
    pub message: String,
    pub position: Option<usize>,
}

// 7 public functions to test:
pub fn detect_ddl_kind(query: &str) -> Option<DdlKind>;
pub fn detect_semantic_view_ddl(query: &str) -> u8;
pub fn rewrite_ddl(query: &str) -> Result<String, String>;
pub fn extract_ddl_name(query: &str) -> Result<Option<String>, String>;
pub fn validate_and_rewrite(query: &str) -> Result<Option<String>, ParseError>;
pub fn validate_clauses(body: &str, body_offset: usize, query: &str) -> Result<(), ParseError>;
pub fn detect_near_miss(query: &str) -> Option<ParseError>;
```

DDL prefix to DdlKind mapping (longest-first ordering):
- "create or replace semantic view" -> CreateOrReplace
- "create semantic view if not exists" -> CreateIfNotExists
- "create semantic view" -> Create
- "drop semantic view if exists" -> DropIfExists
- "drop semantic view" -> Drop
- "describe semantic view" -> Describe
- "show semantic views" -> Show

Rewrite targets (function_name mapping):
- Create -> "create_semantic_view"
- CreateOrReplace -> "create_or_replace_semantic_view"
- CreateIfNotExists -> "create_semantic_view_if_not_exists"
- Drop -> "drop_semantic_view"
- DropIfExists -> "drop_semantic_view_if_exists"
- Describe -> "describe_semantic_view"
- Show -> "list_semantic_views"

Known clause keywords: ["tables", "relationships", "dimensions", "metrics"]
Near-miss threshold: Levenshtein distance <= 3
</interfaces>
</context>

<tasks>

<task type="auto" tdd="true">
  <name>Task 1: Create parse_proptest.rs with detection and rewrite properties</name>
  <files>tests/parse_proptest.rs</files>
  <behavior>
    - Property: Any case variation of each of the 7 DDL prefixes + valid suffix is detected as the correct DdlKind
    - Property: Leading whitespace (0-20 spaces) + valid DDL prefix still detects correctly
    - Property: Trailing semicolons (0-3) do not affect detection
    - Property: Non-DDL SQL statements (SELECT, INSERT, UPDATE, DELETE, ALTER) always return None from detect_ddl_kind
    - Property: detect_semantic_view_ddl returns PARSE_DETECTED for all 7 forms and PARSE_NOT_OURS for non-DDL
    - Property: rewrite_ddl for CREATE forms produces SQL starting with "SELECT * FROM create_semantic_view(" (or or_replace/if_not_exists variant) and containing the view name as a quoted string
    - Property: rewrite_ddl for DROP/DESCRIBE forms produces SQL starting with "SELECT * FROM drop_semantic_view(" or "describe_semantic_view(" and containing the view name
    - Property: rewrite_ddl for SHOW produces "SELECT * FROM list_semantic_views()"
    - Property: extract_ddl_name returns the correct view name for all 7 forms with random valid identifiers
  </behavior>
  <action>
Create `tests/parse_proptest.rs` following the `expand_proptest.rs` pattern. Structure:

1. **Strategy helpers:**
   - `arb_case_variant(prefix: &str)` -- generates random upper/lower case of each character using `proptest::collection::vec(bool, len).prop_map(...)`. This is the core strategy for case-insensitive testing.
   - `arb_whitespace()` -- generates `"[ \t]{0,10}"` via `string_regex`.
   - `arb_view_name()` -- generates `"[a-z_][a-z0-9_]{0,29}"` via `string_regex`. Keep names short to avoid slow tests.
   - `arb_semicolons()` -- generates `";{0,3}"` via `string_regex`.

2. **DDL prefix/kind pairs as a constant array** for parameterized properties:
   ```rust
   const DDL_FORMS: &[(&str, DdlKind)] = &[
       ("create semantic view", DdlKind::Create),
       ("create or replace semantic view", DdlKind::CreateOrReplace),
       ("create semantic view if not exists", DdlKind::CreateIfNotExists),
       ("drop semantic view", DdlKind::Drop),
       ("drop semantic view if exists", DdlKind::DropIfExists),
       ("describe semantic view", DdlKind::Describe),
       ("show semantic views", DdlKind::Show),
   ];
   ```

3. **Detection properties (TEST-01):**
   - For each of the 7 DDL forms, generate case-variant prefix + appropriate suffix (name for name-only forms, name+body for CREATE forms, nothing for SHOW). Assert `detect_ddl_kind` returns the expected `DdlKind`.
   - Use a single `proptest!` block with multiple `#[test]` functions, one per DDL form, to ensure shrinking is effective per-form.
   - Test leading whitespace + detection (combine `arb_whitespace()` with case variant prefix).
   - Test trailing semicolons (append `arb_semicolons()` after DDL).
   - Test non-DDL rejection: generate `prop_oneof!["SELECT", "INSERT", "UPDATE", "DELETE", "ALTER"]` prefixed strings, assert `detect_ddl_kind` returns `None`.

4. **Rewrite properties (TEST-02):**
   - For CREATE forms: generate valid DDL with random view name + minimal valid body (`tables := [], dimensions := []`). Assert rewrite starts with `SELECT * FROM {function_name}(` and contains `'{name}'`.
   - For DROP/DESCRIBE forms: generate DDL prefix + random view name. Assert rewrite starts with `SELECT * FROM {function_name}(` and contains `'{name}'`.
   - For SHOW: assert rewrite is exactly `SELECT * FROM list_semantic_views()`.
   - Test extract_ddl_name returns the correct name for all 7 forms.

5. **Test count target:** ~15-20 properties across the detection and rewrite groups. Use default 256 cases per property.

Do NOT test validate_and_rewrite or validate_clauses in this task -- those go in Task 2.
  </action>
  <verify>
    <automated>cd /Users/paul/Documents/Dev/Personal/duckdb-semantic-views && cargo test parse_proptest --test parse_proptest -- --nocapture 2>&1 | tail -20</automated>
  </verify>
  <done>All detection and rewrite property tests pass. Each of the 7 DDL forms has case-insensitive detection verified. Rewrite output structure is verified for all forms. Non-DDL inputs are correctly rejected.</done>
</task>

<task type="auto" tdd="true">
  <name>Task 2: Add validation, position invariant, near-miss, and bracket properties</name>
  <files>tests/parse_proptest.rs</files>
  <behavior>
    - Property: validate_and_rewrite error positions always point to the correct byte in the original query when leading whitespace varies (TEST-03)
    - Property: For DDL with a typo in a clause keyword (e.g., "tbles" instead of "tables"), the error position points at the start of the typo
    - Property: detect_near_miss returns None for common SQL statements that are not close to any DDL prefix (TEST-04)
    - Property: detect_near_miss returns Some for inputs within Levenshtein distance 3 of a known DDL prefix
    - Property: validate_clauses with balanced nested brackets (e.g., `tables := [{'a': 'b'}]`) does not produce a bracket error (TEST-05)
    - Property: validate_clauses with unbalanced brackets produces an error with a position pointing at the bracket
  </behavior>
  <action>
Append to `tests/parse_proptest.rs` (same file from Task 1):

1. **Position invariant properties (TEST-03):**
   - Generate CREATE DDL with varying leading whitespace (0-20 spaces) and a deliberately invalid clause name (e.g., "tbles" instead of "tables"). Call `validate_and_rewrite`. Assert the error position + 5 bytes equals "tbles" in the original query string. Use: `&query[pos..pos+5] == "tbles"`.
   - Generate CREATE DDL with empty body `()`. Call `validate_and_rewrite`. Assert error position points at the byte right after `(`.
   - Generate DROP DDL with no view name after prefix. Assert error position equals `trim_offset + prefix_len`.

2. **Near-miss safety properties (TEST-04):**
   - Generate common SQL statements: `prop_oneof!` from ["SELECT * FROM t", "INSERT INTO t VALUES (1)", "UPDATE t SET x=1", "DELETE FROM t", "ALTER TABLE t ADD COLUMN x INT", "BEGIN TRANSACTION", "COMMIT", "ROLLBACK"]. Assert `detect_near_miss` returns `None` for all of these. This ensures no false positives on normal SQL.
   - Generate strings that ARE near-misses: Take a known prefix, apply a single character substitution (e.g., "cretae semantic view" -- transposition). Assert `detect_near_miss` returns `Some` with a suggestion message containing the correct prefix.

3. **Bracket validation properties (TEST-05):**
   - Generate bodies with balanced brackets: use a strategy that produces strings like `tables := [{}], dimensions := [{}]` where the brackets are guaranteed balanced by construction. Assert `validate_clauses` returns `Ok(())`.
   - Generate bodies with an extra `[` (append `[` to a valid body). Assert `validate_clauses` returns an error.
   - Generate bodies with embedded string literals containing brackets (e.g., `tables := [{'name': 'a[b]c'}], dimensions := [{}]`) -- these should NOT trigger bracket errors because the brackets are inside string literals. Note: check if the current implementation handles this; if it does simple counting, this test documents the behavior.

4. **Constrain string lengths** per research pitfall: use `"[\\x20-\\x7E]{1,100}"` for random ASCII printable strings to avoid slow Levenshtein computation. Use `"[a-z_]{1,30}"` for view names.

5. **Test count target:** ~8-12 additional properties. Use default 256 cases.
  </action>
  <verify>
    <automated>cd /Users/paul/Documents/Dev/Personal/duckdb-semantic-views && cargo test parse_proptest --test parse_proptest -- --nocapture 2>&1 | tail -20</automated>
  </verify>
  <done>All validation, position, near-miss, and bracket property tests pass. Position invariants verified across whitespace variations. Near-miss detection has no false positives on common SQL. Bracket validation handles nested/string-embedded cases.</done>
</task>

</tasks>

<verification>
Run the full proptest suite for parse_proptest:
```bash
cargo test parse_proptest --test parse_proptest
```

Run the full test suite to confirm no regressions:
```bash
just test-all
```
</verification>

<success_criteria>
- `tests/parse_proptest.rs` exists with 20+ property-based tests covering all 7 parser public functions
- `cargo test parse_proptest` passes in under 10 seconds
- `just test-all` passes (no regressions)
- All 5 TEST requirement IDs (TEST-01 through TEST-05) have at least one property exercising them
</success_criteria>

<output>
After completion, create `.planning/phases/23-parser-proptests-and-caret-integration-tests/23-01-SUMMARY.md`
</output>
