---
phase: 17-ddl-execution
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - src/parse.rs
  - cpp/src/shim.cpp
  - src/lib.rs
  - test/sql/phase16_parser.test
autonomous: true
requirements: [DDL-01, DDL-02, DDL-03, BUILD-03]

must_haves:
  truths:
    - "CREATE SEMANTIC VIEW sales (...) creates a view in the catalog"
    - "View created via native DDL is queryable via semantic_view()"
    - "Existing function-based DDL (create_semantic_view) still works"
    - "Extension loads in DuckDB CLI and Python client"
  artifacts:
    - path: "src/parse.rs"
      provides: "DDL text parsing and rewriting functions + FFI execution entry point"
      contains: "pub fn parse_ddl_text"
    - path: "cpp/src/shim.cpp"
      provides: "Real plan function with DDL-executing TableFunction"
      contains: "sv_plan_function"
    - path: "src/lib.rs"
      provides: "DDL connection creation and passing to C++ hooks"
      contains: "ddl_conn"
    - path: "test/sql/phase16_parser.test"
      provides: "End-to-end native DDL integration test"
      contains: "CREATE SEMANTIC VIEW"
  key_links:
    - from: "cpp/src/shim.cpp (sv_ddl_bind)"
      to: "src/parse.rs (sv_execute_ddl_rust)"
      via: "extern C FFI call"
      pattern: "sv_execute_ddl_rust"
    - from: "cpp/src/shim.cpp (sv_plan_function)"
      to: "cpp/src/shim.cpp (sv_ddl_bind)"
      via: "TableFunction bind callback"
      pattern: "sv_ddl_bind"
    - from: "src/lib.rs (init_extension)"
      to: "cpp/src/shim.cpp (sv_register_parser_hooks)"
      via: "extern C call passing ddl_conn"
      pattern: "sv_register_parser_hooks.*ddl_conn"
    - from: "src/parse.rs (rewrite_ddl_to_function_call)"
      to: "create_semantic_view table function"
      via: "SQL rewrite executed on ddl_conn"
      pattern: "create_semantic_view"
---

<objective>
Wire the parser hook chain to existing catalog code so `CREATE SEMANTIC VIEW name (tables := [...], dimensions := [...], metrics := [...])` creates a semantic view that is immediately queryable via `semantic_view()`.

Purpose: This is the core deliverable of Phase 17 -- making the parser extension spike actually functional. Phase 16 proved detection works; this plan makes DDL execution work end-to-end.

Output: Working native DDL syntax that creates views via statement rewriting to the existing `create_semantic_view()` table function.
</objective>

<execution_context>
@/Users/paul/.claude/get-shit-done/workflows/execute-plan.md
@/Users/paul/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/17-ddl-execution/17-RESEARCH.md
@.planning/phases/16-parser-hook-registration/16-01-SUMMARY.md

@src/parse.rs
@cpp/src/shim.cpp
@src/lib.rs
@src/ddl/define.rs
@test/sql/phase16_parser.test

<interfaces>
<!-- Key types and contracts the executor needs. Extracted from codebase. -->

From src/parse.rs (Phase 16 -- existing):
```rust
pub const PARSE_NOT_OURS: u8 = 0;
pub const PARSE_DETECTED: u8 = 1;
pub fn detect_create_semantic_view(query: &str) -> u8;

#[cfg(feature = "extension")]
#[no_mangle]
pub extern "C" fn sv_parse_rust(query_ptr: *const u8, query_len: usize) -> u8;
```

From cpp/src/shim.cpp (Phase 16 -- existing):
```cpp
struct SemanticViewParseData : public ParserExtensionParseData {
    string query;  // raw query text carried from parse to plan
    // ...
};

extern "C" {
    uint8_t sv_parse_rust(const char *query, size_t query_len);
}

// sv_parse_stub -- parse function, delegates to Rust, returns SemanticViewParseData with query text
// sv_plan_stub -- TO BE REPLACED by sv_plan_function
// sv_register_parser_hooks(duckdb_database) -- registers hooks on DBConfig

extern "C" {
    bool sv_register_parser_hooks(duckdb_database db_handle);
}
```

From src/lib.rs (init_extension -- existing):
```rust
fn init_extension(con: &Connection, db_handle: ffi::duckdb_database) -> Result<(), Box<dyn Error>> {
    // ...
    let persist_conn: Option<ffi::duckdb_connection> = if db_path.as_ref() != ":memory:" { ... };
    // ...
    if !unsafe { sv_register_parser_hooks(db_handle) } { ... }
}
```

From src/ddl/define.rs (existing -- reused via SQL rewrite):
```rust
pub struct DefineState {
    pub catalog: CatalogState,
    pub persist_conn: Option<ffi::duckdb_connection>,
    pub or_replace: bool,
    pub if_not_exists: bool,
}
// create_semantic_view table function: parses named params, inserts into catalog
```
</interfaces>
</context>

<tasks>

<task type="auto" tdd="true">
  <name>Task 1: Rust DDL text parsing, rewriting, and FFI execution function</name>
  <files>src/parse.rs</files>
  <behavior>
    - parse_ddl_text("CREATE SEMANTIC VIEW sales (tables := [...])") -> Ok(("sales", "tables := [...]"))
    - parse_ddl_text("create semantic view My_View (...)") -> Ok(("My_View", "..."))
    - parse_ddl_text("  CREATE SEMANTIC VIEW x (a := 1);") -> Ok(("x", "a := 1")) (whitespace + semicolon)
    - parse_ddl_text("SELECT 1") -> Err("Not a CREATE SEMANTIC VIEW statement")
    - parse_ddl_text("CREATE SEMANTIC VIEW") -> Err("Missing view name")
    - parse_ddl_text("CREATE SEMANTIC VIEW x") -> Err("Expected '(' after view name")
    - rewrite_ddl_to_function_call("CREATE SEMANTIC VIEW sales (tables := [...], dimensions := [...])") -> Ok("SELECT * FROM create_semantic_view('sales', tables := [...], dimensions := [...])")
    - rewrite_ddl_to_function_call handles view names with single quotes (escaping)
    - sv_execute_ddl_rust FFI function: receives query text + connection, rewrites DDL, executes via duckdb_query, returns view name or error
  </behavior>
  <action>
    Add two pure functions to src/parse.rs (NOT feature-gated, testable under cargo test):

    1. `pub fn parse_ddl_text(query: &str) -> Result<(&str, &str), String>`:
       - Trim input, strip trailing semicolons
       - Case-insensitive prefix match for "create semantic view" (reuse existing pattern from detect_create_semantic_view)
       - Extract view name: next whitespace-delimited token after prefix (stop at '(' or whitespace)
       - Extract body: everything between first '(' and last ')' (rfind for last paren -- handles nested parens in STRUCT literals)
       - Return (name, body) or descriptive error

    2. `pub fn rewrite_ddl_to_function_call(query: &str) -> Result<String, String>`:
       - Call parse_ddl_text to extract name and body
       - Escape single quotes in view name (replace ' with '')
       - Return: `SELECT * FROM create_semantic_view('{safe_name}', {body})`

    3. Add feature-gated FFI function `sv_execute_ddl_rust`:
       ```rust
       #[cfg(feature = "extension")]
       #[no_mangle]
       pub extern "C" fn sv_execute_ddl_rust(
           query_ptr: *const u8, query_len: usize,
           exec_conn: ffi::duckdb_connection,
           name_out: *mut u8, name_out_len: usize,
           error_out: *mut u8, error_out_len: usize,
       ) -> u8
       ```
       - Wrap in catch_unwind (same pattern as sv_parse_rust)
       - Convert query_ptr/len to &str
       - Call rewrite_ddl_to_function_call
       - Execute rewritten SQL via ffi::duckdb_query on exec_conn
       - On success: extract view name from parse_ddl_text, write to name_out buffer (null-terminated, truncated to name_out_len-1)
       - On failure: write error message to error_out buffer, return non-zero
       - Helper fn write_to_buffer(buf: *mut u8, len: usize, s: &str) for safe buffer writes

    Write unit tests for parse_ddl_text and rewrite_ddl_to_function_call in the existing #[cfg(test)] mod tests block. Follow TDD: write tests first (RED), then implement (GREEN).

    The sv_execute_ddl_rust function is feature-gated so it won't be tested via cargo test -- it will be tested via sqllogictest in Task 3.
  </action>
  <verify>
    <automated>cd /Users/paul/Documents/Dev/Personal/duckdb-semantic-views && cargo test parse</automated>
  </verify>
  <done>parse_ddl_text correctly extracts view name and body from DDL text. rewrite_ddl_to_function_call produces correct SQL rewrites. sv_execute_ddl_rust FFI function exists (feature-gated). All parse module tests pass.</done>
</task>

<task type="auto">
  <name>Task 2: C++ plan function and DDL connection wiring</name>
  <files>cpp/src/shim.cpp, src/lib.rs</files>
  <action>
    **In cpp/src/shim.cpp:**

    1. Add extern "C" declaration for the new Rust FFI function:
       ```cpp
       extern "C" {
           uint8_t sv_execute_ddl_rust(
               const char *query_ptr, size_t query_len,
               duckdb_connection exec_conn,
               char *name_out, size_t name_out_len,
               char *error_out, size_t error_out_len);
       }
       ```

    2. Add file-scope static for DDL connection:
       ```cpp
       static duckdb_connection sv_ddl_conn = nullptr;
       ```

    3. Replace sv_stub_bind, SvStubGlobalState, sv_stub_init_global, sv_stub_execute, sv_plan_stub with:

       a. `SvDdlBindData` struct (extends FunctionData): holds `string view_name`. Implement Copy() and Equals().

       b. `sv_ddl_bind` function: declare return type (VARCHAR "view_name"), extract query from input.inputs[0] via StringValue::Get, call sv_execute_ddl_rust with sv_ddl_conn, throw BinderException on error, return SvDdlBindData with view name.

       c. `SvDdlGlobalState` struct (extends GlobalTableFunctionState): `bool done = false`.

       d. `sv_ddl_init_global` function: return make_uniq<SvDdlGlobalState>().

       e. `sv_ddl_execute` function: if done, SetCardinality(0) return. Otherwise set done=true, get view_name from bind_data, SetCardinality(1), SetValue(0, 0, Value(view_name)).

       f. `sv_plan_function`: cast parse_data to SemanticViewParseData, construct ParserExtensionPlanResult with TableFunction("sv_ddl_internal", {LogicalType::VARCHAR}, sv_ddl_execute, sv_ddl_bind, sv_ddl_init_global). Push sv_data.query as VARCHAR parameter. Set requires_valid_transaction = true and return_type = StatementReturnType::QUERY_RESULT.

    4. Modify sv_register_parser_hooks signature to accept ddl_conn:
       ```cpp
       bool sv_register_parser_hooks(duckdb_database db_handle, duckdb_connection ddl_conn)
       ```
       Store ddl_conn in the file-scope static: `sv_ddl_conn = ddl_conn;`
       Change ext.plan_function from sv_plan_stub to sv_plan_function.

    **In src/lib.rs:**

    5. Update the extern "C" declaration for sv_register_parser_hooks to include ddl_conn parameter:
       ```rust
       extern "C" {
           fn sv_register_parser_hooks(db_handle: ffi::duckdb_database, ddl_conn: ffi::duckdb_connection) -> bool;
       }
       ```

    6. In init_extension, create a dedicated DDL connection for the parser hook path. This connection is ALWAYS created (even for in-memory databases) because the native DDL path always rewrites to `SELECT * FROM create_semantic_view(...)` which needs a separate connection to avoid deadlock:
       ```rust
       let mut ddl_conn: ffi::duckdb_connection = ptr::null_mut();
       let rc = unsafe { ffi::duckdb_connect(db_handle, &mut ddl_conn) };
       if rc != ffi::DuckDBSuccess {
           return Err("Failed to create DDL connection for parser hook".into());
       }
       ```

    7. Pass ddl_conn to sv_register_parser_hooks:
       ```rust
       if !unsafe { sv_register_parser_hooks(db_handle, ddl_conn) } {
           return Err("Failed to register parser hooks via C++ helper".into());
       }
       ```

    IMPORTANT: The DDL connection created here is separate from persist_conn. persist_conn writes to the _definitions catalog table. ddl_conn executes the rewritten SQL (which internally goes through create_semantic_view's bind, which uses persist_conn if available). For in-memory databases, persist_conn is None but ddl_conn is always created because the rewrite still needs a separate connection to avoid the ClientContext deadlock.
  </action>
  <verify>
    <automated>cd /Users/paul/Documents/Dev/Personal/duckdb-semantic-views && just build 2>&1 | tail -5</automated>
  </verify>
  <done>shim.cpp compiles with real plan function. sv_register_parser_hooks accepts ddl_conn. init_extension creates DDL connection and passes it. `just build` succeeds with no linker errors.</done>
</task>

<task type="auto">
  <name>Task 3: Integration tests for native DDL execution</name>
  <files>test/sql/phase16_parser.test</files>
  <action>
    Update test/sql/phase16_parser.test to test real DDL execution instead of stub behavior. The test must exercise DDL-01, DDL-02, DDL-03, and BUILD-03.

    Replace the file contents with a test that:

    1. Keeps the "Normal SQL still works" test (SELECT 42).

    2. Creates a source table for the semantic view:
       ```sql
       CREATE TABLE sales (region VARCHAR, amount DOUBLE);
       INSERT INTO sales VALUES ('East', 100.0), ('East', 200.0), ('West', 150.0);
       ```

    3. **DDL-01**: Creates a semantic view via native DDL syntax:
       ```sql
       CREATE SEMANTIC VIEW sales_view (
           tables := [{alias: 'sales', table: 'sales'}],
           dimensions := [{name: 'region', expr: 'region', source_table: 'sales'}],
           metrics := [{name: 'total_amount', expr: 'SUM(amount)', source_table: 'sales'}]
       );
       ```
       This should return the view name "sales_view" as a VARCHAR result.

    4. **DDL-02**: Queries the view created via native DDL:
       ```sql
       SELECT * FROM semantic_view('sales_view', dimensions := ['region'], metrics := ['total_amount']) ORDER BY region;
       ```
       Expected: East 300.0, West 150.0

    5. **DDL-03**: Verifies function-based DDL still works by creating another view:
       ```sql
       FROM create_semantic_view('sales_fn_view',
           tables := [{alias: 'sales', table: 'sales'}],
           dimensions := [{name: 'region', expr: 'region', source_table: 'sales'}],
           metrics := [{name: 'total_amount', expr: 'SUM(amount)', source_table: 'sales'}]
       );
       ```
       Then queries it to verify it works.

    6. **Case insensitive** DDL (regression for PARSE-02): lowercase `create semantic view` should also work.

    7. Clean up: drop tables.

    Use sqllogictest syntax: `statement ok` for DDL that should succeed, `query T` for single-column VARCHAR results, `query TR` for mixed-type results with sort mode, etc. Consult existing test files for syntax patterns.

    After updating the test, run `just build && just test-sql` to verify all sqllogictests pass. Then run `just test-all` to verify no regressions.
  </action>
  <verify>
    <automated>cd /Users/paul/Documents/Dev/Personal/duckdb-semantic-views && just build && just test-all</automated>
  </verify>
  <done>phase16_parser.test exercises native DDL creation (DDL-01), querying via semantic_view (DDL-02), function-based DDL coexistence (DDL-03). `just test-all` passes: cargo test, sqllogictest, and DuckLake CI all green. BUILD-03 verified by extension loading successfully in test runners.</done>
</task>

</tasks>

<verification>
Phase-level verification (run after all tasks complete):

1. `just test-all` passes (cargo test + sqllogictest + DuckLake CI)
2. Native DDL creates a queryable view (covered by updated phase16_parser.test)
3. Function-based DDL still works (covered by existing phase2_ddl.test + phase4_query.test + updated phase16_parser.test)
4. Extension loads in both CLI and Python (covered by sqllogictest runner using CLI, DuckLake CI using Python)
</verification>

<success_criteria>
- `CREATE SEMANTIC VIEW sales (tables := [...], dimensions := [...], metrics := [...])` creates a view and returns the view name
- `FROM semantic_view('sales', dimensions := ['region'], metrics := ['total_amount'])` returns correct aggregated results for a view created via native DDL
- `FROM create_semantic_view(...)` continues to work unchanged
- `just test-all` green: 90+ Rust tests, 4+ sqllogictests, 6 DuckLake CI tests
</success_criteria>

<output>
After completion, create `.planning/phases/17-ddl-execution/17-01-SUMMARY.md`
</output>
