---
phase: 19-parser-hook-validation-spike
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - test/sql/phase19_parser_hook_validation.test
  - .planning/phases/19-parser-hook-validation-spike/19-SPIKE-RESULTS.md
autonomous: true
requirements: []

must_haves:
  truths:
    - "Each of the 7 DDL prefixes has been tested against DuckDB with the extension loaded"
    - "For each prefix, the error type is recorded: Parser Error (triggers hook) or Catalog Error (bypasses hook)"
    - "A concrete scope decision is documented: which statements get native syntax in v0.5.1"
  artifacts:
    - path: "test/sql/phase19_parser_hook_validation.test"
      provides: "Empirical proof that all 7 DDL prefixes produce Parser Error (not Catalog Error)"
      contains: "DROP SEMANTIC VIEW"
    - path: ".planning/phases/19-parser-hook-validation-spike/19-SPIKE-RESULTS.md"
      provides: "Scope decision document for v0.5.1 DDL coverage"
      contains: "Scope Decision"
  key_links:
    - from: "test/sql/phase19_parser_hook_validation.test"
      to: ".planning/phases/19-parser-hook-validation-spike/19-SPIKE-RESULTS.md"
      via: "Test results inform scope decision"
      pattern: "Parser Error"
---

<objective>
Empirically confirm which DDL prefixes trigger the DuckDB parser fallback hook, then document the scope decision for v0.5.1.

Purpose: Phase 19 is a scope-determination spike. Research (19-RESEARCH.md) predicts HIGH confidence that all 7 DDL prefixes produce Parser Errors (triggering the hook), but the roadmap demands empirical confirmation before Phase 20 implementation begins. This prevents wasted effort on statements that would require a different approach.

Output: A sqllogictest proving all 7 prefixes trigger parser errors, and a SPIKE-RESULTS.md documenting the scope decision.
</objective>

<execution_context>
@/Users/paul/.claude/get-shit-done/workflows/execute-plan.md
@/Users/paul/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/19-parser-hook-validation-spike/19-RESEARCH.md

Source files for understanding current parser hook:
@src/parse.rs
@cpp/src/shim.cpp
@test/sql/phase16_parser.test

<interfaces>
<!-- Current parser detection returns PARSE_NOT_OURS for anything except CREATE SEMANTIC VIEW -->

From src/parse.rs:
```rust
pub const PARSE_NOT_OURS: u8 = 0;
pub const PARSE_DETECTED: u8 = 1;

pub fn detect_create_semantic_view(query: &str) -> u8;
pub fn rewrite_ddl_to_function_call(query: &str) -> Result<String, String>;
```

From cpp/src/shim.cpp:
```cpp
// sv_parse_stub calls sv_parse_rust -> returns PARSE_NOT_OURS for non-CREATE prefixes
// -> DuckDB shows its original Parser Error (the error we need to confirm)
extern "C" uint8_t sv_parse_rust(const char *query, size_t query_len);
```

<!-- Existing target functions (already registered, all will work once detection is extended in Phase 20): -->
From src/lib.rs (registered functions):
- `create_semantic_view(name, tables:=, dimensions:=, metrics:=)` -- replace=false, if_not_exists=false
- `create_or_replace_semantic_view(name, tables:=, dimensions:=, metrics:=)` -- replace=true
- `create_semantic_view_if_not_exists(name, tables:=, dimensions:=, metrics:=)` -- if_not_exists=true
- `drop_semantic_view(name)` -- if_exists=false
- `drop_semantic_view_if_exists(name)` -- if_exists=true
- `list_semantic_views()` -- returns all views
- `describe_semantic_view(name)` -- returns view details
</interfaces>
</context>

<tasks>

<task type="auto">
  <name>Task 1: Create sqllogictest validating all 7 DDL prefixes trigger parser errors</name>
  <files>test/sql/phase19_parser_hook_validation.test</files>
  <action>
Create a sqllogictest file that loads the semantic_views extension and tests all 7 DDL prefixes empirically. The test proves each prefix produces a "Parser Error" (meaning DuckDB's native parser could not handle it, so the fallback hook path is reachable).

The test structure:

1. `require semantic_views` header
2. Create a source table for the CREATE variants that need valid DDL bodies
3. Create a semantic view via the existing working native DDL (`CREATE SEMANTIC VIEW`) so DROP/DESCRIBE have a target
4. For each of the 6 non-CREATE prefixes, use `statement error` with a regex matching "Parser Error" or the extension's own error (proving the hook was invoked):

   - `DROP SEMANTIC VIEW test_view` -- currently unhandled by the extension, so sv_parse_rust returns PARSE_NOT_OURS, and DuckDB shows "Parser Error: syntax error at or near SEMANTIC". Use `statement error` with pattern matching "Parser Error" OR "syntax error".
   - `DROP SEMANTIC VIEW IF EXISTS test_view` -- same pattern
   - `CREATE OR REPLACE SEMANTIC VIEW test_view (...)` -- The detection function currently only checks "create semantic view" prefix. "CREATE OR REPLACE SEMANTIC VIEW" does NOT match this prefix (it starts with "create or"). So sv_parse_rust returns PARSE_NOT_OURS and DuckDB shows its Parser Error. Use `statement error` with pattern.
   - `CREATE SEMANTIC VIEW IF NOT EXISTS test_view (...)` -- This DOES match the "create semantic view" prefix (it starts with "create semantic view"). sv_parse_rust returns PARSE_DETECTED. Then the rewrite function `parse_ddl_text` tries to parse it and extracts the name as "IF" (first token after prefix). This will either produce a function call error or succeed with wrong name. Test with `statement error` or observe the behavior.
   - `DESCRIBE SEMANTIC VIEW test_view` -- Parser Error from DuckDB
   - `SHOW SEMANTIC VIEWS` -- Parser Error from DuckDB

5. Confirm `CREATE SEMANTIC VIEW test_view (...)` still works (prefix 7 -- already proven, but include for completeness)

IMPORTANT: The goal is NOT to make these statements work (that's Phase 20). The goal is to prove each one triggers a Parser Error (not a Catalog Error or Binder Error), confirming the fallback hook path is reachable.

For the `CREATE SEMANTIC VIEW IF NOT EXISTS` case specifically: since "create semantic view" IS a prefix of "CREATE SEMANTIC VIEW IF NOT EXISTS", the current detection WILL match it and the current rewrite code will attempt to process it. Test what actually happens -- the rewrite will extract "IF" as the view name and fail or produce an unexpected result. Document the actual behavior in the test comments. This is expected and informative for Phase 20.

After writing the test, run `just build && just test-sql` to execute it. If any test case produces an unexpected error type (e.g., Catalog Error instead of Parser Error), that is a critical finding to record.

After the sqllogictest passes, also run `just test-all` to confirm no regressions.
  </action>
  <verify>
    <automated>cd /Users/paul/Documents/Dev/Personal/duckdb-semantic-views && just build && just test-all</automated>
  </verify>
  <done>
    - All 7 DDL prefixes tested via sqllogictest
    - Each test confirms Parser Error (not Catalog Error) for unhandled prefixes
    - CREATE SEMANTIC VIEW IF NOT EXISTS behavior documented (prefix overlap case)
    - Full test suite passes with no regressions
  </done>
</task>

<task type="auto">
  <name>Task 2: Document spike results and scope decision</name>
  <files>.planning/phases/19-parser-hook-validation-spike/19-SPIKE-RESULTS.md</files>
  <action>
Create `19-SPIKE-RESULTS.md` documenting the empirical findings from Task 1 and the scope decision for v0.5.1.

Structure:

```markdown
# Phase 19: Parser Hook Validation Spike — Results

**Tested:** {date}
**Extension version:** v0.5.0 (DuckDB v1.4.4 amalgamation)

## Empirical Results

| # | DDL Statement | Error Type | Error Message (excerpt) | Hook Triggered? |
|---|--------------|------------|------------------------|-----------------|
| 1 | DROP SEMANTIC VIEW x | {actual} | {actual message} | {YES/NO} |
| 2 | DROP SEMANTIC VIEW IF EXISTS x | {actual} | {actual message} | {YES/NO} |
| 3 | CREATE OR REPLACE SEMANTIC VIEW x (...) | {actual} | {actual message} | {YES/NO} |
| 4 | CREATE SEMANTIC VIEW IF NOT EXISTS x (...) | {actual} | {actual message} | {YES/NO} |
| 5 | DESCRIBE SEMANTIC VIEW x | {actual} | {actual message} | {YES/NO} |
| 6 | SHOW SEMANTIC VIEWS | {actual} | {actual message} | {YES/NO} |
| 7 | CREATE SEMANTIC VIEW x (...) | Success | (already working) | YES (proven) |

## Analysis

{For each prefix, explain what happened and why. Note any surprises vs. the research predictions.}

### Prefix Overlap: CREATE SEMANTIC VIEW IF NOT EXISTS

{Document the special case where "CREATE SEMANTIC VIEW IF NOT EXISTS" matches the current "create semantic view" prefix detection. What did the rewrite produce? What error occurred? This informs how Phase 20 must order its prefix detection (longer prefixes first).}

## Scope Decision

**All 7 DDL prefixes CAN use native syntax via the parser fallback hook.**

{Or if any prefix produced a Catalog Error instead of Parser Error, document which ones cannot use native syntax and must remain function-only.}

### v0.5.1 Native DDL Scope

| DDL Statement | v0.5.1 Approach | Requirement |
|--------------|-----------------|-------------|
| CREATE SEMANTIC VIEW | Native DDL (existing) | — |
| CREATE OR REPLACE SEMANTIC VIEW | Native DDL | DDL-05 |
| CREATE SEMANTIC VIEW IF NOT EXISTS | Native DDL | DDL-06 |
| DROP SEMANTIC VIEW | Native DDL | DDL-03 |
| DROP SEMANTIC VIEW IF EXISTS | Native DDL | DDL-04 |
| DESCRIBE SEMANTIC VIEW | Native DDL | DDL-07 |
| SHOW SEMANTIC VIEWS | Native DDL | DDL-08 |

### Implementation Notes for Phase 20

- Detection function must check longer prefixes first ("CREATE OR REPLACE SEMANTIC VIEW" before "CREATE SEMANTIC VIEW") to avoid prefix overlap
- {Any other findings relevant to Phase 20 implementation}

## Conclusion

{Brief summary: research predictions confirmed/invalidated, Phase 20 can proceed with full/partial native DDL scope.}
```

Fill in ALL {actual} values from the sqllogictest output in Task 1. Run the statements manually with `duckdb` CLI if needed to capture exact error messages for the table.

To capture exact error messages, run each statement individually:
```bash
just build
echo "LOAD 'build/debug/extension/semantic_views/semantic_views.duckdb_extension'; DROP SEMANTIC VIEW nonexistent;" | ./duckdb
```
(Adjust path to duckdb binary as needed -- check justfile for the correct path.)
  </action>
  <verify>
    <automated>test -f /Users/paul/Documents/Dev/Personal/duckdb-semantic-views/.planning/phases/19-parser-hook-validation-spike/19-SPIKE-RESULTS.md && grep -q "Scope Decision" /Users/paul/Documents/Dev/Personal/duckdb-semantic-views/.planning/phases/19-parser-hook-validation-spike/19-SPIKE-RESULTS.md && grep -c "YES\|NO" /Users/paul/Documents/Dev/Personal/duckdb-semantic-views/.planning/phases/19-parser-hook-validation-spike/19-SPIKE-RESULTS.md</automated>
  </verify>
  <done>
    - SPIKE-RESULTS.md exists with all 7 prefixes documented with actual error types
    - Scope decision clearly states which DDL statements get native syntax in v0.5.1
    - Implementation notes for Phase 20 include prefix ordering guidance
    - Prefix overlap case (CREATE SEMANTIC VIEW IF NOT EXISTS) analyzed
  </done>
</task>

</tasks>

<verification>
Phase 19 success criteria from ROADMAP.md:
1. Each of the 7 DDL prefixes has been tested against DuckDB with the extension loaded -- verified by sqllogictest passing
2. For each prefix, the error type is recorded: Parser Error or Catalog Error -- verified in SPIKE-RESULTS.md table
3. A concrete scope decision is documented -- verified by "Scope Decision" section in SPIKE-RESULTS.md

Quality gate: `just test-all` passes (per CLAUDE.md)
</verification>

<success_criteria>
- sqllogictest file exists at test/sql/phase19_parser_hook_validation.test and passes
- SPIKE-RESULTS.md documents all 7 DDL prefixes with empirical error types
- Scope decision clearly states which statements get native DDL in v0.5.1
- `just test-all` passes with no regressions
</success_criteria>

<output>
After completion, create `.planning/phases/19-parser-hook-validation-spike/19-01-SUMMARY.md`
</output>
