---
phase: 02-storage-and-ddl
plan: "02"
type: execute
wave: 2
depends_on: ["02-01"]
files_modified:
  - src/ddl/mod.rs
  - src/ddl/define.rs
  - src/ddl/drop.rs
  - src/ddl/list.rs
  - src/ddl/describe.rs
  - src/lib.rs
autonomous: true
requirements:
  - DDL-01
  - DDL-02
  - DDL-03
  - DDL-04

must_haves:
  truths:
    - "define_semantic_view scalar function accepts (VARCHAR name, VARCHAR json) and returns a VARCHAR confirmation message"
    - "drop_semantic_view scalar function accepts (VARCHAR name) and returns a VARCHAR confirmation message or an error"
    - "list_semantic_views table function returns (name VARCHAR, base_table VARCHAR) — one row per registered view"
    - "describe_semantic_view table function accepts VARCHAR name and returns one row: (name, base_table, dimensions, metrics, filters, joins) all VARCHAR"
    - "All four functions are registered on the DuckDB connection in the extension entrypoint"
  artifacts:
    - path: "src/ddl/define.rs"
      provides: "DefineSemanticView VScalar implementation"
      contains: "impl VScalar for DefineSemanticView"
    - path: "src/ddl/drop.rs"
      provides: "DropSemanticView VScalar implementation"
      contains: "impl VScalar for DropSemanticView"
    - path: "src/ddl/list.rs"
      provides: "ListSemanticViewsVTab VTab implementation"
      contains: "impl VTab for ListSemanticViewsVTab"
    - path: "src/ddl/describe.rs"
      provides: "DescribeSemanticViewVTab VTab implementation"
      contains: "impl VTab for DescribeSemanticViewVTab"
    - path: "src/lib.rs"
      provides: "Extension entrypoint registering all 4 DDL functions"
      contains: "register_scalar_function_with_state"
  key_links:
    - from: "src/ddl/define.rs"
      to: "src/catalog.rs"
      via: "catalog_insert called via CatalogState"
      pattern: "catalog_insert"
    - from: "src/ddl/drop.rs"
      to: "src/catalog.rs"
      via: "catalog_delete called via CatalogState"
      pattern: "catalog_delete"
    - from: "src/ddl/list.rs"
      to: "src/catalog.rs"
      via: "CatalogState read in VTab bind"
      pattern: "get_extra_info"
    - from: "src/lib.rs"
      to: "src/catalog.rs"
      via: "init_catalog called in entrypoint"
      pattern: "init_catalog"
---

<objective>
Implement the four DDL functions as duckdb-rs VScalar and VTab types, and wire them into the extension entrypoint.

Purpose: This plan delivers the user-facing DDL surface — the four functions users call to define, remove, list, and inspect semantic views. It builds on the model and catalog layer from plan 02-01.

Output: `src/ddl/` module with four function implementations, `src/lib.rs` updated to initialize the catalog and register all functions.
</objective>

<execution_context>
@/Users/paul/.claude/get-shit-done/workflows/execute-plan.md
@/Users/paul/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/02-storage-and-ddl/02-CONTEXT.md
@.planning/phases/02-storage-and-ddl/02-RESEARCH.md
@.planning/phases/02-storage-and-ddl/02-01-SUMMARY.md
@Cargo.toml
@src/lib.rs
@src/catalog.rs
@src/model.rs
</context>

<tasks>

<task type="auto">
  <name>Task 1: Implement define_semantic_view and drop_semantic_view as VScalar types</name>
  <files>src/ddl/mod.rs, src/ddl/define.rs, src/ddl/drop.rs</files>
  <action>
Create `src/ddl/` directory with `mod.rs`, `define.rs`, and `drop.rs`.

**CRITICAL architectural note from RESEARCH.md Open Question 1:** `duckdb::Connection` is not `Send + Sync` — it cannot be stored in `CatalogState`. The VScalar `invoke` method cannot perform catalog writes using a stored connection. Solution: open a fresh `duckdb::Connection` inside `invoke` using the same database file path stored in state. For v0.1, store `Option<String>` (the database path) alongside the HashMap in a wrapper state type. For in-memory databases, skip the catalog write path (use `None` for the path — this matches the integration test setup where the catalog is initialized before function registration).

**Actually use a cleaner approach:** Per RESEARCH.md Pattern 4, the entrypoint receives a `Connection` — store a `Arc<duckdb::Connection>` is not possible (not Send). Instead, store the DB file path as `Arc<str>` in a struct alongside the `CatalogState`, and inside `invoke`, open a new connection to the path. For in-memory databases (used in tests), the path is `":memory:"` — a second connection to `":memory:"` creates a separate DB; this means catalog writes from within invoke would be to a different DB. This is the core tension in RESEARCH.md.

**Resolution for v0.1:** Pass both the `CatalogState` (HashMap) and a `Connection` path through a single wrapper. For the scalar functions, perform catalog writes via a fresh `Connection::open(path)` inside `invoke`. Accept that in-process integration tests will either use a file-backed DB or mock the catalog. The integration SQL logic tests (plan 02-03) will use a file-backed DB.

Implement `src/ddl/define.rs`:

```rust
use std::sync::Arc;

use duckdb::{
    core::{DataChunkHandle, LogicalTypeId, WritableVector},
    vscalar::{ScalarFunctionSignature, VScalar},
    Connection, Result,
};
use duckdb_string_t (import from duckdb::ffi or use DuckString helper per RESEARCH.md code examples)
```

Wait — there is an important nuance from RESEARCH.md code examples. Reading VARCHAR from input in VScalar uses:
```rust
let col = input.flat_vector(0);
let slice = col.as_slice_with_len::<duckdb_string_t>(input.len());
let name = DuckString::new(&mut { slice[i] }).as_str().to_string();
```

This requires `duckdb_string_t` and `DuckString` from duckdb-rs. These are in `duckdb::ffi::duckdb_string_t` and `duckdb::core::DuckString` (check actual import paths at compile time; the research examples show them being used but import paths must be resolved).

Create `src/ddl/define.rs`:

```rust
use std::sync::Arc;

use duckdb::{
    core::{DataChunkHandle, LogicalTypeId, WritableVector},
    vscalar::{ScalarFunctionSignature, VScalar},
    Connection,
};

use crate::catalog::{catalog_insert, CatalogState};

/// Shared state for define_semantic_view: the in-memory catalog plus db path for writes.
pub struct DefineState {
    pub catalog: CatalogState,
    pub db_path: Arc<str>,
}

pub struct DefineSemanticView;

impl VScalar for DefineSemanticView {
    type State = DefineState;

    fn signatures() -> Vec<ScalarFunctionSignature> {
        vec![ScalarFunctionSignature::exact(
            vec![
                LogicalTypeId::Varchar.into(), // view name
                LogicalTypeId::Varchar.into(), // definition JSON
            ],
            LogicalTypeId::Varchar.into(), // confirmation message
        )]
    }

    unsafe fn invoke(
        state: &Self::State,
        input: &mut DataChunkHandle,
        output: &mut dyn WritableVector,
    ) -> Result<(), Box<dyn std::error::Error>> {
        let out = output.flat_vector();
        for i in 0..input.len() {
            // Read VARCHAR inputs — use the method shown in RESEARCH.md code examples
            // The exact types (duckdb_string_t, DuckString) must be resolved at compile time.
            // Use input.flat_vector(col).get_value::<String>(i) if available in duckdb-rs 1.4.4,
            // otherwise use the as_slice_with_len pattern from RESEARCH.md.
            // Resolve the actual working approach at compile time and note in SUMMARY.md.
            let name = read_varchar(input, 0, i);
            let json = read_varchar(input, 1, i);

            // Open a fresh connection for catalog writes (Connection is not Send)
            let con = Connection::open(state.db_path.as_ref())?;
            match catalog_insert(&con, &state.catalog, &name, &json) {
                Ok(()) => {
                    let msg = format!("Semantic view '{}' registered successfully", name);
                    out.insert(i, msg.as_str());
                }
                Err(e) => return Err(e),
            }
        }
        Ok(())
    }
}

/// Read a VARCHAR value from a DataChunk column at the given row index.
/// Resolves the correct duckdb-rs 1.4.4 API at compile time.
/// See RESEARCH.md code examples for the as_slice_with_len pattern.
fn read_varchar(input: &DataChunkHandle, col: usize, row: usize) -> String {
    // Implementation: use the FlatVector read pattern from RESEARCH.md
    // Exact API: input.flat_vector(col).as_slice_with_len::<duckdb_string_t>(n)[row]
    // then DuckString::new(&mut copy).as_str().to_string()
    // Resolve actual imports at compile time.
    todo!("resolve at compile time")
}
```

**NOTE:** The `read_varchar` function has a `todo!` placeholder. The executor MUST resolve the correct duckdb-rs 1.4.4 API for reading VARCHAR from a `DataChunkHandle` before the task is done. The research provides the pattern (RESEARCH.md "Scalar Function Registration" code example). The executor should try compiling with the `as_slice_with_len::<duckdb_string_t>` pattern first, check what imports are needed, and iterate until it compiles with zero warnings.

Create `src/ddl/drop.rs` with the same structure but for `drop_semantic_view`:
- One VARCHAR input parameter (name)
- Calls `catalog_delete(&con, &state.catalog, &name)`
- Success message: `"Semantic view 'X' removed successfully"`
- Error from `catalog_delete` propagates as the VScalar error

Create `src/ddl/mod.rs`:
```rust
pub mod define;
pub mod drop;
pub mod list;
pub mod describe;
```

Add `pub mod ddl;` to `src/lib.rs`.

Run `cargo build` (not just `cargo check`) to verify the cdylib compiles. Run `cargo clippy -- -D warnings`.
  </action>
  <verify>
    <automated>cd /Users/paul/Documents/Dev/Personal/duckdb-semantic-views && cargo build 2>&1 | tail -10</automated>
    <manual>Confirm cargo build exits 0. Confirm the `read_varchar` helper is fully implemented (no `todo!` macros remaining). Confirm `DefineSemanticView` and `DropSemanticView` both implement `VScalar`.</manual>
  </verify>
  <done>
    - `cargo build` exits 0 — no `todo!` stubs remaining, full implementation in place
    - `cargo clippy -- -D warnings` exits 0
    - `src/ddl/define.rs` contains `impl VScalar for DefineSemanticView`
    - `src/ddl/drop.rs` contains `impl VScalar for DropSemanticView` (or analogous type name)
    - Both functions use `catalog_insert` / `catalog_delete` from `src/catalog.rs`
  </done>
</task>

<task type="auto">
  <name>Task 2: Implement list_semantic_views and describe_semantic_view as VTab types; wire all functions in entrypoint</name>
  <files>src/ddl/list.rs, src/ddl/describe.rs, src/lib.rs</files>
  <action>
Create `src/ddl/list.rs` implementing `ListSemanticViewsVTab` using the `VTab` trait pattern from RESEARCH.md Pattern 2.

`list_semantic_views()` output schema: `(name VARCHAR, base_table VARCHAR)`.
- At bind time: call `bind.get_extra_info::<CatalogState>()`, clone the Arc, read all entries, extract `base_table` from each JSON value using `serde_json`, collect into `Vec<(String, String)>` stored in `BindData`.
- At func time: emit one row per entry. Use `AtomicBool` done flag in `InitData`.
- No parameters (`parameters()` returns `None`).

Create `src/ddl/describe.rs` implementing `DescribeSemanticViewVTab`.

`describe_semantic_view('name')` output schema: `(name VARCHAR, base_table VARCHAR, dimensions VARCHAR, metrics VARCHAR, filters VARCHAR, joins VARCHAR)`.
- One VARCHAR parameter.
- At bind time: look up the name in the CatalogState, parse JSON, extract fields. If name not found, return error "semantic view 'X' does not exist".
- Store parsed fields in `BindData`: `name: String, base_table: String, dimensions: String, metrics: String, filters: String, joins: String` — serialize the arrays back to JSON strings for the VARCHAR columns.
- At func time: emit one row with all fields.
- `parameters()` returns `Some(vec![LogicalTypeId::Varchar.into()])`.

**IMPORTANT from RESEARCH.md Pitfall 3:** When calling `bind.get_extra_info::<CatalogState>()`, the return is a `*const CatalogState`. Use `unsafe { (*ptr).clone() }` to get an owned `Arc` clone. Do NOT take ownership.

**IMPORTANT from RESEARCH.md Pitfall 6:** Use `LogicalTypeId::Varchar` for all JSON-containing columns (dimensions, metrics, filters, joins) — do NOT use `LogicalTypeId::Json` as it may not be available in all builds.

Update `src/lib.rs` entrypoint:

```rust
use duckdb::{duckdb_entrypoint_c_api, Connection, Result};
use std::error::Error;
use std::sync::Arc;

mod catalog;
mod ddl;
mod model;

use catalog::init_catalog;
use ddl::{
    define::{DefineSemanticView, DefineState},
    drop::{DropSemanticView, DropState},
    list::ListSemanticViewsVTab,
    describe::DescribeSemanticViewVTab,
};

/// Extension entry point — called by DuckDB when the extension is loaded.
///
/// # Safety
///
/// This function is called by DuckDB across an FFI boundary. The `con` parameter
/// is provided by DuckDB and is guaranteed to be a valid connection handle for
/// the duration of the call. The `#[duckdb_entrypoint_c_api]` macro handles the
/// unsafe C FFI bridging and panic-catching automatically.
#[duckdb_entrypoint_c_api()]
pub unsafe fn extension_entrypoint(con: Connection) -> Result<(), Box<dyn Error>> {
    // Initialize the catalog: creates schema/table if needed, loads existing rows
    let catalog_state = init_catalog(&con)?;

    // Resolve database file path for catalog writes from within scalar functions.
    // Use ":memory:" as sentinel for in-memory databases.
    let db_path: Arc<str> = Arc::from(
        con.path().unwrap_or(":memory:")
    );

    // Register scalar functions (DDL mutations)
    con.register_scalar_function_with_state::<DefineSemanticView>(
        "define_semantic_view",
        &DefineState { catalog: catalog_state.clone(), db_path: db_path.clone() },
    )?;
    con.register_scalar_function_with_state::<DropSemanticView>(
        "drop_semantic_view",
        &DropState { catalog: catalog_state.clone(), db_path: db_path.clone() },
    )?;

    // Register table functions (DDL reads)
    con.register_table_function_with_extra_info::<ListSemanticViewsVTab>(
        "list_semantic_views",
        &catalog_state,
    )?;
    con.register_table_function_with_extra_info::<DescribeSemanticViewVTab>(
        "describe_semantic_view",
        &catalog_state,
    )?;

    Ok(())
}
```

**NOTE:** `con.path()` may not exist in duckdb-rs 1.4.4. At compile time, resolve how to obtain the database file path from a `Connection`. Check `Connection` methods in docs. If `path()` is not available, use an alternative: store the path during init via a thread-local or use a different approach. Document the resolution in SUMMARY.md.

**NOTE:** If `register_table_function_with_extra_info` does not exist in 1.4.4 (see RESEARCH.md Open Question 2), fall back to using a `static OnceLock<CatalogState>` that is set before function registration and accessed from within VTab bind via the static. Document the fallback in SUMMARY.md.

Run `cargo build` and `cargo clippy -- -D warnings`.
  </action>
  <verify>
    <automated>cd /Users/paul/Documents/Dev/Personal/duckdb-semantic-views && cargo build 2>&1 | tail -10</automated>
    <manual>Confirm cargo build exits 0. Check that all four functions are registered in extension_entrypoint. Verify no `todo!` stubs remain in list.rs or describe.rs.</manual>
  </verify>
  <done>
    - `cargo build` exits 0
    - `cargo clippy -- -D warnings` exits 0
    - `src/lib.rs` registers: `define_semantic_view`, `drop_semantic_view`, `list_semantic_views`, `describe_semantic_view`
    - `describe_semantic_view` returns 6 VARCHAR columns: name, base_table, dimensions, metrics, filters, joins
    - `list_semantic_views` returns 2 VARCHAR columns: name, base_table
  </done>
</task>

</tasks>

<verification>
After both tasks:
- `cargo build` exits 0 — extension cdylib compiles with all four functions
- `cargo clippy -- -D warnings` exits 0 — zero lint violations
- `cargo test` exits 0 — all unit tests from plan 02-01 still pass
- All four DDL function implementations are complete (no `todo!` macros)
- `src/lib.rs` entrypoint calls `init_catalog` and registers all four functions
</verification>

<success_criteria>
- `src/ddl/define.rs`, `src/ddl/drop.rs`, `src/ddl/list.rs`, `src/ddl/describe.rs` all exist with full implementations
- Extension builds as cdylib with zero warnings and zero clippy violations
- Entrypoint wires catalog init + all four function registrations
- `describe_semantic_view` output schema is `(name, base_table, dimensions, metrics, filters, joins)` — all VARCHAR
- `list_semantic_views` output schema is `(name, base_table)` — both VARCHAR
</success_criteria>

<output>
After completion, create `.planning/phases/02-storage-and-ddl/02-02-SUMMARY.md`
</output>
