---
phase: 56-yaml-export
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - src/model.rs
  - src/render_yaml.rs
  - src/lib.rs
  - src/ddl/read_yaml.rs
  - src/ddl/mod.rs
  - test/sql/phase56_yaml_export.test
  - tests/yaml_proptest.rs
autonomous: true
requirements:
  - YAML-04
  - YAML-08

must_haves:
  truths:
    - "SELECT READ_YAML_FROM_SEMANTIC_VIEW('name') returns a valid YAML string for a stored semantic view"
    - "Fully qualified names (database.schema.view_name) resolve to the correct stored view"
    - "Exported YAML includes materializations when declared"
    - "Exported YAML omits internal fields (column_type_names, column_types_inferred, created_on, database_name, schema_name)"
    - "Exported YAML fed back through CREATE SEMANTIC VIEW ... FROM YAML produces an identical semantic view"
    - "Nonexistent view names produce a clear error"
  artifacts:
    - path: "src/render_yaml.rs"
      provides: "YAML export function with field stripping"
      exports: ["render_yaml_export"]
    - path: "src/ddl/read_yaml.rs"
      provides: "VScalar implementation for READ_YAML_FROM_SEMANTIC_VIEW"
      exports: ["ReadYamlFromSemanticViewScalar"]
    - path: "test/sql/phase56_yaml_export.test"
      provides: "sqllogictest integration tests for YAML export and round-trip"
  key_links:
    - from: "src/ddl/read_yaml.rs"
      to: "src/render_yaml.rs"
      via: "calls render_yaml_export()"
      pattern: "render_yaml_export"
    - from: "src/ddl/read_yaml.rs"
      to: "src/catalog.rs"
      via: "reads CatalogState HashMap to get stored JSON"
      pattern: "guard\\.get"
    - from: "src/lib.rs"
      to: "src/ddl/read_yaml.rs"
      via: "register_scalar_function_with_state for read_yaml_from_semantic_view"
      pattern: "ReadYamlFromSemanticViewScalar"
---

<objective>
Add YAML export capability via a `READ_YAML_FROM_SEMANTIC_VIEW('name')` scalar function that converts stored semantic view definitions to clean YAML output suitable for version control and round-trip workflows.

Purpose: Completes the YAML definition lifecycle -- users can define views from YAML (phases 51-53) and export existing views back to YAML for portability and version control.

Output: `render_yaml.rs` (export logic), `ddl/read_yaml.rs` (VScalar), sqllogictest integration tests, proptest extension for round-trip fidelity.
</objective>

<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/56-yaml-export/56-RESEARCH.md

@src/model.rs
@src/render_ddl.rs
@src/ddl/get_ddl.rs
@src/ddl/mod.rs
@src/lib.rs
@src/catalog.rs
@tests/yaml_proptest.rs
@test/sql/phase45_get_ddl.test
@test/sql/phase52_yaml_ddl.test

<interfaces>
<!-- Key types and contracts the executor needs. Extracted from codebase. -->

From src/catalog.rs:
```rust
pub type CatalogState = Arc<RwLock<HashMap<String, String>>>;
```

From src/model.rs:
```rust
#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)]
pub struct SemanticViewDefinition {
    pub tables: Vec<TableRef>,
    pub dimensions: Vec<Dimension>,
    pub metrics: Vec<Metric>,
    pub joins: Vec<Join>,
    pub facts: Vec<Fact>,
    pub materializations: Vec<Materialization>,
    pub column_type_names: Vec<String>,       // internal, no skip_serializing_if
    pub column_types_inferred: Vec<u32>,       // internal, no skip_serializing_if
    pub created_on: Option<String>,            // internal, no skip_serializing_if
    pub database_name: Option<String>,         // internal, no skip_serializing_if
    pub schema_name: Option<String>,           // internal, no skip_serializing_if
    pub comment: Option<String>,               // has skip_serializing_if = "Option::is_none"
}
```

From src/ddl/get_ddl.rs (template for VScalar):
```rust
pub struct GetDdlScalar;
impl VScalar for GetDdlScalar {
    type State = CatalogState;
    unsafe fn invoke(state, input, output) -> Result<...> { ... }
    fn signatures() -> Vec<ScalarFunctionSignature> { ... }
}
```

From src/lib.rs (registration pattern, line 564):
```rust
con.register_scalar_function_with_state::<GetDdlScalar>("get_ddl", &catalog_state)?;
```

From src/lib.rs (import pattern, lines 303-323):
```rust
use crate::ddl::{
    get_ddl::GetDdlScalar,
    // ... other imports
};
```
</interfaces>
</context>

<tasks>

<task type="auto" tdd="true">
  <name>Task 1: YAML export function and model cleanup</name>
  <files>src/model.rs, src/render_yaml.rs, src/lib.rs</files>
  <read_first>src/model.rs, src/render_ddl.rs, src/lib.rs, tests/yaml_proptest.rs</read_first>
  <behavior>
    - Test 1: render_yaml_export strips column_type_names (empty vec not in output)
    - Test 2: render_yaml_export strips column_types_inferred (empty vec not in output)
    - Test 3: render_yaml_export strips created_on (None not in output)
    - Test 4: render_yaml_export strips database_name (None not in output)
    - Test 5: render_yaml_export strips schema_name (None not in output)
    - Test 6: render_yaml_export preserves user-facing fields (tables, dimensions, metrics, joins, facts, materializations, comment)
    - Test 7: render_yaml_export output can be deserialized back via from_yaml and produce semantically equal definition (internal fields zeroed on both sides)
    - Test 8: render_yaml_export handles empty definition (no dimensions, no metrics)
    - Test 9: render_yaml_export handles definition with materializations
    - Test 10: render_yaml_export handles definition with all metadata annotations (comment, synonyms, access modifiers)
  </behavior>
  <action>
**Step 1: Add `skip_serializing_if` to internal fields in `src/model.rs`**

On `SemanticViewDefinition`, modify these 5 field annotations (adding `skip_serializing_if` while keeping existing `#[serde(default)]`):

- `column_type_names: Vec<String>` -- change `#[serde(default)]` to `#[serde(default, skip_serializing_if = "Vec::is_empty")]`
- `column_types_inferred: Vec<u32>` -- change `#[serde(default)]` to `#[serde(default, skip_serializing_if = "Vec::is_empty")]`
- `created_on: Option<String>` -- change `#[serde(default)]` to `#[serde(default, skip_serializing_if = "Option::is_none")]`
- `database_name: Option<String>` -- change `#[serde(default)]` to `#[serde(default, skip_serializing_if = "Option::is_none")]`
- `schema_name: Option<String>` -- change `#[serde(default)]` to `#[serde(default, skip_serializing_if = "Option::is_none")]`

This is backward-compatible because: (a) these fields already have `#[serde(default)]` for deserialization, so missing fields deserialize correctly; (b) stored JSON without these fields already loads fine; (c) empty/None is the user-facing default anyway.

**Step 2: Create `src/render_yaml.rs`**

Create a new module parallel to `render_ddl.rs`. The module is NOT feature-gated (always compiled, unit-testable under `cargo test`).

```rust
//! YAML export: renders a [`SemanticViewDefinition`] as a YAML string
//! suitable for round-trip through `CREATE SEMANTIC VIEW ... FROM YAML $$ ... $$`.
//!
//! Internal fields populated at define time (column types, timestamps,
//! database/schema context) are stripped before serialization. The render
//! logic lives here (always compiled, unit-tested under `cargo test`).
//! The extension-only VScalar wrapper lives in [`crate::ddl::read_yaml`].

use crate::model::SemanticViewDefinition;

/// Export a semantic view definition as a YAML string.
///
/// Clones the definition and strips internal runtime fields that are
/// repopulated at define time:
/// - `column_type_names` / `column_types_inferred` (DDL-time type inference)
/// - `created_on` (DDL-time timestamp)
/// - `database_name` / `schema_name` (connection context)
///
/// After stripping, `serde(skip_serializing_if)` on these fields ensures
/// they are omitted from the YAML output entirely.
pub fn render_yaml_export(def: &SemanticViewDefinition) -> Result<String, String> {
    let mut export = def.clone();
    export.column_type_names.clear();
    export.column_types_inferred.clear();
    export.created_on = None;
    export.database_name = None;
    export.schema_name = None;

    yaml_serde::to_string(&export)
        .map_err(|e| format!("YAML serialization error: {e}"))
}
```

Add unit tests in a `#[cfg(test)] mod tests` block within `render_yaml.rs`:
- Test field stripping (verify internal fields absent from output using string contains checks)
- Test user-facing field preservation (tables, dims, metrics, joins, facts, materializations, comment all present)
- Test round-trip: `render_yaml_export` -> `from_yaml` -> compare stripped versions via PartialEq
- Test with materializations present
- Test with all metadata annotations (comment, synonyms, access Private)
- Test empty definition

**Step 3: Register module in `src/lib.rs`**

Add `pub mod render_yaml;` after the existing `pub mod render_ddl;` line (line 10). This line is NOT feature-gated (same as `render_ddl`).

**Step 4: Extend proptest in `tests/yaml_proptest.rs`**

Add a new proptest that verifies render_yaml_export round-trip for arbitrary definitions:

```rust
#[test]
fn yaml_export_roundtrip(def in arb_definition()) {
    let yaml_str = semantic_views::render_yaml::render_yaml_export(&def)
        .expect("YAML export should succeed");
    let reimported = SemanticViewDefinition::from_yaml("proptest", &yaml_str)
        .expect("Re-import should succeed");

    // Strip internal fields from original for comparison
    let mut expected = def.clone();
    expected.column_type_names.clear();
    expected.column_types_inferred.clear();
    expected.created_on = None;
    expected.database_name = None;
    expected.schema_name = None;

    prop_assert_eq!(expected, reimported);
}
```
  </action>
  <verify>
    <automated>cargo test render_yaml && cargo test yaml_export_roundtrip && cargo test yaml_proptest</automated>
  </verify>
  <acceptance_criteria>
    - src/render_yaml.rs exists and contains `pub fn render_yaml_export`
    - src/render_yaml.rs contains `export.column_type_names.clear()`
    - src/render_yaml.rs contains `export.column_types_inferred.clear()`
    - src/render_yaml.rs contains `export.created_on = None`
    - src/render_yaml.rs contains `export.database_name = None`
    - src/render_yaml.rs contains `export.schema_name = None`
    - src/model.rs contains `skip_serializing_if = "Vec::is_empty"` on column_type_names field
    - src/model.rs contains `skip_serializing_if = "Vec::is_empty"` on column_types_inferred field
    - src/model.rs contains `skip_serializing_if = "Option::is_none"` on created_on field
    - src/model.rs contains `skip_serializing_if = "Option::is_none"` on database_name field
    - src/model.rs contains `skip_serializing_if = "Option::is_none"` on schema_name field
    - src/lib.rs contains `pub mod render_yaml;`
    - tests/yaml_proptest.rs contains `yaml_export_roundtrip`
    - `cargo test render_yaml` exits 0
    - `cargo test yaml_export_roundtrip` exits 0
  </acceptance_criteria>
  <done>
    render_yaml_export correctly strips internal fields and produces clean YAML.
    Round-trip (export -> from_yaml -> compare) passes for arbitrary definitions.
    model.rs serde annotations updated for clean serialization.
    All unit tests and proptests pass.
  </done>
</task>

<task type="auto">
  <name>Task 2: VScalar function and integration tests</name>
  <files>src/ddl/read_yaml.rs, src/ddl/mod.rs, src/lib.rs, test/sql/phase56_yaml_export.test</files>
  <read_first>src/ddl/get_ddl.rs, src/ddl/mod.rs, src/lib.rs, src/render_yaml.rs, test/sql/phase45_get_ddl.test, test/sql/phase52_yaml_ddl.test</read_first>
  <action>
**Step 1: Create `src/ddl/read_yaml.rs`**

Create a new VScalar implementation following the `GetDdlScalar` pattern exactly. Key differences from `GetDdlScalar`:
- Takes 1 argument (VARCHAR name), not 2 (GetDdlScalar takes object_type + name)
- Calls `render_yaml_export` instead of `render_create_ddl`
- Supports fully qualified names via `rsplit('.')` bare name extraction

```rust
//! READ_YAML_FROM_SEMANTIC_VIEW scalar function: wraps
//! [`crate::render_yaml::render_yaml_export`] as a `VScalar` so that
//! `SELECT READ_YAML_FROM_SEMANTIC_VIEW('name')` works inside DuckDB.
//!
//! The render logic itself lives in [`crate::render_yaml`] (always compiled,
//! unit-tested under `cargo test`). This module adds the extension-only
//! VScalar registration.

use duckdb::core::{DataChunkHandle, Inserter, LogicalTypeHandle, LogicalTypeId};
use duckdb::types::DuckString;
use duckdb::vscalar::{ScalarFunctionSignature, VScalar};
use duckdb::vtab::arrow::WritableVector;
use libduckdb_sys::duckdb_string_t;

use crate::catalog::CatalogState;
use crate::model::SemanticViewDefinition;
use crate::render_yaml::render_yaml_export;

/// Extract the bare view name from a potentially qualified name.
/// Supports: `"view_name"`, `"schema.view_name"`, `"database.schema.view_name"`.
fn resolve_bare_name(input: &str) -> &str {
    input.rsplit('.').next().unwrap_or(input)
}

pub struct ReadYamlFromSemanticViewScalar;

impl VScalar for ReadYamlFromSemanticViewScalar {
    type State = CatalogState;

    unsafe fn invoke(
        state: &Self::State,
        input: &mut DataChunkHandle,
        output: &mut dyn WritableVector,
    ) -> Result<(), Box<dyn std::error::Error>> {
        crate::util::catch_unwind_to_result(std::panic::AssertUnwindSafe(|| {
            let len = input.len();
            let name_vec = input.flat_vector(0);
            let names = name_vec.as_slice_with_len::<duckdb_string_t>(len);
            let out_vec = output.flat_vector();

            for i in 0..len {
                let raw_name = DuckString::new(&mut { names[i] }).as_str().to_string();
                let bare_name = resolve_bare_name(&raw_name);

                let guard = state
                    .read()
                    .map_err(|_| Box::<dyn std::error::Error>::from("catalog lock poisoned"))?;
                let json = guard
                    .get(bare_name)
                    .ok_or_else(|| format!("semantic view '{}' does not exist", bare_name))?;
                let def: SemanticViewDefinition = serde_json::from_str(json)?;
                let yaml = render_yaml_export(&def)
                    .map_err(|e| -> Box<dyn std::error::Error> { e.into() })?;
                out_vec.insert(i, yaml.as_str());
            }
            Ok(())
        }))
    }

    fn signatures() -> Vec<ScalarFunctionSignature> {
        vec![ScalarFunctionSignature::exact(
            vec![LogicalTypeHandle::from(LogicalTypeId::Varchar)], // name only (1 arg)
            LogicalTypeHandle::from(LogicalTypeId::Varchar),       // return: YAML string
        )]
    }
}
```

**Step 2: Register module in `src/ddl/mod.rs`**

Add `pub mod read_yaml;` after the existing `pub mod get_ddl;` line.

**Step 3: Register scalar function in `src/lib.rs`**

In the import block (around line 313), add:
```rust
read_yaml::ReadYamlFromSemanticViewScalar,
```
after the `get_ddl::GetDdlScalar,` line.

In `init_extension` (after the GET_DDL registration around line 564), add:
```rust
// Register READ_YAML_FROM_SEMANTIC_VIEW scalar function.
con.register_scalar_function_with_state::<ReadYamlFromSemanticViewScalar>(
    "read_yaml_from_semantic_view",
    &catalog_state,
)?;
```

**Step 4: Create `test/sql/phase56_yaml_export.test`**

Create sqllogictest integration tests covering:

1. **Basic export**: Create a simple semantic view, call `READ_YAML_FROM_SEMANTIC_VIEW('name')`, verify output contains expected YAML keys (`tables:`, `dimensions:`, `metrics:`)
2. **Internal fields absent**: Verify output does NOT contain `column_type_names`, `column_types_inferred`, `created_on`, `database_name`, `schema_name`
3. **With materializations**: Create a view with MATERIALIZATIONS clause, verify `materializations:` appears in YAML output
4. **With metadata annotations**: Create a view with COMMENT, SYNONYMS, PRIVATE metric, verify they appear in YAML output
5. **Fully qualified name**: Call with `memory.main.view_name` format, verify it returns same result as bare name
6. **Nonexistent view error**: Call with nonexistent view name, verify error
7. **Round-trip verification**: Create view A, export YAML, drop view A, create view B from the exported YAML, query both to verify identical results. Since DuckDB sqllogictest has no dynamic EXECUTE, the round-trip is verified by: (a) checking exported YAML format is valid, and (b) creating a new view from manually-expected YAML and querying it

Test file structure:
```sql
# Phase 56: READ_YAML_FROM_SEMANTIC_VIEW
#
# Verifies YAML export scalar function and round-trip fidelity:
#   YAML-04: Returns a valid YAML string for a stored semantic view
#   YAML-08: Exported YAML round-trips correctly
#
# Requirements covered: YAML-04, YAML-08

require semantic_views

# Setup tables, create views, run SELECT READ_YAML_FROM_SEMANTIC_VIEW queries
```

Use `query I` with `LIKE '%pattern%'` checks (same pattern as phase45_get_ddl.test) for substring matching. Use `statement error` for error cases.
  </action>
  <verify>
    <automated>just build && just test-sql && cargo test</automated>
  </verify>
  <acceptance_criteria>
    - src/ddl/read_yaml.rs exists and contains `pub struct ReadYamlFromSemanticViewScalar`
    - src/ddl/read_yaml.rs contains `fn resolve_bare_name`
    - src/ddl/read_yaml.rs contains `rsplit('.')`
    - src/ddl/read_yaml.rs contains `render_yaml_export`
    - src/ddl/mod.rs contains `pub mod read_yaml;`
    - src/lib.rs contains `read_yaml::ReadYamlFromSemanticViewScalar`
    - src/lib.rs contains `"read_yaml_from_semantic_view"`
    - test/sql/phase56_yaml_export.test exists and contains `require semantic_views`
    - test/sql/phase56_yaml_export.test contains `READ_YAML_FROM_SEMANTIC_VIEW`
    - test/sql/phase56_yaml_export.test contains `statement error` for nonexistent view
    - test/sql/phase56_yaml_export.test contains `memory.main.` for FQN test
    - `just build` exits 0
    - `just test-sql` exits 0
    - `just test-all` exits 0
  </acceptance_criteria>
  <done>
    READ_YAML_FROM_SEMANTIC_VIEW scalar function is registered and works in DuckDB.
    Bare names and fully qualified names both resolve correctly.
    Nonexistent views produce clear errors.
    YAML output is clean (no internal fields) and includes materializations when present.
    All sqllogictest integration tests pass.
    Full test suite (just test-all) is green.
  </done>
</task>

</tasks>

<threat_model>
## Trust Boundaries

| Boundary | Description |
|----------|-------------|
| User SQL -> scalar function | View name string from user query enters the scalar function |

## STRIDE Threat Register

| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-56-01 | Tampering | resolve_bare_name | accept | Name used only as HashMap key lookup in CatalogState; no SQL interpolation. rsplit('.') is safe string operation. Low risk. |
| T-56-02 | Information Disclosure | render_yaml_export | mitigate | Field stripping removes internal runtime fields (timestamps, database context) from exported YAML. Implemented via clone + clear + skip_serializing_if. |
| T-56-03 | Denial of Service | yaml_serde::to_string | accept | Output size bounded by stored definition size; no amplification possible. Stored definitions already passed size cap at ingestion time (YAML_SIZE_CAP = 1 MiB). |
</threat_model>

<verification>
1. `cargo test render_yaml` -- unit tests for YAML export function
2. `cargo test yaml_export_roundtrip` -- proptest round-trip via export
3. `just build && just test-sql` -- sqllogictest integration tests
4. `just test-all` -- full test suite (unit + proptest + sqllogictest + DuckLake CI)
</verification>

<success_criteria>
- `SELECT READ_YAML_FROM_SEMANTIC_VIEW('name')` returns clean YAML for any stored semantic view
- Fully qualified names (`database.schema.view_name`) resolve correctly
- Internal fields are absent from exported YAML
- Materializations appear in exported YAML when declared
- Exported YAML can recreate an identical semantic view via FROM YAML
- All tests pass: `just test-all` exits 0
</success_criteria>

<output>
After completion, create `.planning/phases/56-yaml-export/56-01-SUMMARY.md`
</output>
