---
phase: 29-facts-clause-hierarchies
plan: 02
type: execute
wave: 2
depends_on: ["29-01"]
files_modified:
  - src/expand.rs
  - src/ddl/describe.rs
  - test/sql/phase29_facts_hierarchies.test
  - tests/parse_proptest.rs
autonomous: true
requirements:
  - FACT-02
  - FACT-05
  - HIER-03

must_haves:
  truths:
    - "Metric expressions referencing fact names expand with inlined fact expressions (parenthesized)"
    - "Multi-level fact chains (fact A refs fact B) resolve correctly via topological inlining order"
    - "Word-boundary substitution prevents substring collisions (net_price vs net_price_total)"
    - "DESCRIBE SEMANTIC VIEW shows facts column with JSON array of fact definitions"
    - "DESCRIBE SEMANTIC VIEW shows hierarchies column with JSON array of hierarchy definitions"
    - "End-to-end DDL with FACTS + HIERARCHIES creates, queries, describes, and drops correctly"
    - "Proptest covers adversarial FACTS and HIERARCHIES clause input"
  artifacts:
    - path: "src/expand.rs"
      provides: "Fact expression inlining before metric expansion"
      contains: "inline_facts"
    - path: "src/ddl/describe.rs"
      provides: "Facts and hierarchies columns in DESCRIBE output (8 columns total)"
      contains: "facts"
    - path: "test/sql/phase29_facts_hierarchies.test"
      provides: "End-to-end sqllogictest for FACTS + HIERARCHIES DDL and query"
    - path: "tests/parse_proptest.rs"
      provides: "Proptest generators for FACTS and HIERARCHIES clauses"
      contains: "arb_facts_clause"
  key_links:
    - from: "src/expand.rs"
      to: "src/graph.rs"
      via: "toposort_facts for inlining order"
      pattern: "toposort_facts|inline_facts"
    - from: "src/expand.rs"
      to: "src/model.rs"
      via: "Fact struct fields (name, expr, source_table)"
      pattern: "fact\\.name|fact\\.expr"
    - from: "src/ddl/describe.rs"
      to: "stored JSON"
      via: "def[\"facts\"] and def[\"hierarchies\"] JSON extraction"
      pattern: "def\\[.facts.\\]|def\\[.hierarchies.\\]"
---

<objective>
Implement fact expression inlining in the expansion pipeline, update DESCRIBE output to include facts and hierarchies, and create end-to-end tests.

Purpose: Complete the runtime behavior for facts (metrics reference fact names, expansion inlines them with parenthesization) and the metadata exposure for both features. End-to-end tests validate the full pipeline from DDL through query and describe.

Output: expand.rs inlines facts into metric expressions; describe.rs outputs 8 columns (adding facts, hierarchies); sqllogictest covers the full lifecycle; proptests cover adversarial input.
</objective>

<execution_context>
@/Users/paul/.claude/get-shit-done/workflows/execute-plan.md
@/Users/paul/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/29-facts-clause-hierarchies/29-RESEARCH.md
@.planning/phases/29-facts-clause-hierarchies/29-01-SUMMARY.md

<interfaces>
<!-- Interfaces from Plan 01 that this plan depends on. -->

From src/model.rs (after Plan 01):
```rust
pub struct Fact {
    pub name: String,
    pub expr: String,
    pub source_table: Option<String>,
}

pub struct Hierarchy {
    pub name: String,
    pub levels: Vec<String>,
}

pub struct SemanticViewDefinition {
    // ... existing fields ...
    pub facts: Vec<Fact>,
    pub hierarchies: Vec<Hierarchy>,  // added in Plan 01
}
```

From src/graph.rs (after Plan 01):
```rust
pub fn validate_facts(def: &SemanticViewDefinition) -> Result<(), String>;
pub fn validate_hierarchies(def: &SemanticViewDefinition) -> Result<(), String>;
// find_fact_references and is_word_boundary_char helpers exist
```

From src/expand.rs (current):
```rust
pub fn expand(view_name: &str, def: &SemanticViewDefinition, req: &QueryRequest)
    -> Result<String, ExpandError>;
// expand() builds SELECT with metric expressions -- fact inlining must happen
// BEFORE the metric expr is placed into the SELECT clause
```

From src/ddl/describe.rs (current -- 6 columns):
```rust
pub struct DescribeBindData {
    name: String,
    base_table: String,
    dimensions: String,   // JSON
    metrics: String,      // JSON
    filters: String,      // JSON
    joins: String,        // JSON
    // facts and hierarchies fields MUST be added
}
// bind() adds 6 columns, func() writes 6 flat_vectors
// Must become 8 columns with facts (idx 6) and hierarchies (idx 7)
```

From test/sql/phase28_e2e.test (DESCRIBE format):
```
query TTTTTT
DESCRIBE SEMANTIC VIEW p28_analytics;
----
p28_analytics	p28_orders	[dims_json]	[metrics_json]	[]	[joins_json]
```
Phase 29 changes this to `query TTTTTTTT` (8 T's) with facts and hierarchies columns appended.
</interfaces>
</context>

<tasks>

<task type="auto" tdd="true">
  <name>Task 1: Implement fact expression inlining in expand.rs</name>
  <files>src/expand.rs</files>
  <behavior>
    - Test: inline_facts with no facts returns expr unchanged
    - Test: inline_facts replaces fact name in metric expr with parenthesized fact expr: SUM(net_price) -> SUM((price * (1 - discount)))
    - Test: inline_facts with qualified name: SUM(li.net_price) -> SUM((li.price * (1 - li.discount)))
    - Test: inline_facts word boundary: "net_price" does NOT match in "net_price_total" or "total_net_price_x"
    - Test: inline_facts word boundary: "net_price" matches in "SUM(net_price)", "net_price + tax", "(net_price)", "net_price"
    - Test: inline_facts multi-level: fact A = price * qty, fact B = A * (1 - discount) -> B inlines A first, then B is fully resolved
    - Test: inline_facts preserves parenthesization for operator precedence: fact = a + b inlined into expr * fact -> expr * (a + b)
    - Test: replace_word_boundary at string boundaries: needle at start, at end, entire string
  </behavior>
  <action>
1. **expand.rs** -- Add fact inlining functions:

   `fn replace_word_boundary(haystack: &str, needle: &str, replacement: &str) -> String`:
   - Scan `haystack` for occurrences of `needle` (case-sensitive since fact names are identifiers)
   - At each match position, check:
     - Character before match (if any) is NOT alphanumeric or underscore
     - Character after match (if any) is NOT alphanumeric or underscore
     - If both checks pass, replace this occurrence with `replacement`
   - Return new string with all replacements applied
   - Edge cases: needle at position 0 (no char before), needle at end (no char after), needle is the entire haystack

   `fn toposort_facts(facts: &[Fact]) -> Result<Vec<usize>, String>`:
   - Build a dependency graph: for each fact, scan its expr for references to other fact names using word-boundary matching
   - Use Kahn's algorithm (same pattern as RelationshipGraph::toposort in graph.rs)
   - Return indices into the original `facts` slice in topological order (leaf facts first)
   - Return Err if cycle detected (should not happen since validate_facts already checked, but defensive)

   `fn inline_facts(expr: &str, facts: &[Fact], topo_order: &[usize]) -> String`:
   - Start with a map of fact_name -> resolved_expr (initially empty)
   - Process facts in topo_order:
     - For each fact at index i, take its expr and inline any already-resolved facts into it
     - Try qualified form first (source_table.name), then unqualified form (name)
     - Store resolved expr (with parenthesization) in the map: `format!("({})", resolved_expr)`
   - Then apply the map to the input `expr`:
     - For each resolved fact, replace qualified and unqualified forms with the parenthesized resolved expr
   - Return the final expr string

2. **Wire into expand()** function:
   - Near the top of `expand()`, after validation but before building the SELECT clause:
     ```rust
     // Inline facts into metric expressions before expansion.
     let topo_order = toposort_facts(&def.facts).unwrap_or_default();
     ```
   - When building the metric SELECT items, before using a metric's `expr`, inline facts:
     ```rust
     let resolved_expr = if def.facts.is_empty() {
         met.expr.clone()
     } else {
         inline_facts(&met.expr, &def.facts, &topo_order)
     };
     ```
   - Use `resolved_expr` instead of `met.expr` when building the SELECT clause

3. **Add unit tests** in expand.rs tests module:
   - Test replace_word_boundary with various edge cases (see behavior block)
   - Test toposort_facts with independent facts, chained facts
   - Test inline_facts with single fact, multi-level facts, qualified names
   - Test end-to-end expand() with facts: define a view with facts, query a metric that references a fact, verify expanded SQL contains the inlined expression
  </action>
  <verify>
    <automated>cd /Users/paul/Documents/Dev/Personal/duckdb-semantic-views && cargo test expand::tests:: -- --nocapture 2>&1 | tail -10</automated>
  </verify>
  <done>
    - replace_word_boundary correctly handles word boundaries at all positions
    - toposort_facts produces correct topological order for fact dependencies
    - inline_facts resolves single and multi-level fact chains with parenthesization
    - expand() uses inline_facts on metric expressions before building SELECT clause
    - Word-boundary substitution prevents substring collisions
    - All existing expand.rs tests still pass
  </done>
</task>

<task type="auto">
  <name>Task 2: Update DESCRIBE output to include facts and hierarchies columns</name>
  <files>src/ddl/describe.rs, test/sql/phase28_e2e.test</files>
  <action>
1. **describe.rs** -- Add facts and hierarchies columns:
   - Add `facts: String` and `hierarchies: String` fields to DescribeBindData
   - In bind(), add two new column declarations after "joins":
     ```rust
     bind.add_result_column("facts", LogicalTypeHandle::from(LogicalTypeId::Varchar));
     bind.add_result_column("hierarchies", LogicalTypeHandle::from(LogicalTypeId::Varchar));
     ```
   - In bind(), extract facts and hierarchies from the parsed JSON:
     ```rust
     let facts = serde_json::to_string(&def["facts"]).unwrap_or_else(|_| "[]".to_string());
     let hierarchies = serde_json::to_string(&def["hierarchies"]).unwrap_or_else(|_| "[]".to_string());
     ```
   - Store in DescribeBindData: `facts, hierarchies`
   - In func(), add two new flat_vector writes:
     ```rust
     let facts_vec = output.flat_vector(6);
     let hierarchies_vec = output.flat_vector(7);
     facts_vec.insert(0, bind_data.facts.as_str());
     hierarchies_vec.insert(0, bind_data.hierarchies.as_str());
     ```
   - Update the doc comment on DescribeSemanticViewVTab to reflect 8 columns

2. **phase28_e2e.test** -- Update DESCRIBE assertion:
   - Change `query TTTTTT` to `query TTTTTTTT` (8 T's) on the DESCRIBE test
   - Append `\t[]\t[]` to the expected output line (empty facts and hierarchies arrays for the p28_analytics view which has no facts or hierarchies)
   - The full expected line becomes: `p28_analytics\t{base_table}\t{dims}\t{metrics}\t{filters}\t{joins}\t[]\t[]`
  </action>
  <verify>
    <automated>cd /Users/paul/Documents/Dev/Personal/duckdb-semantic-views && cargo test -- --nocapture 2>&1 | tail -5 && just build && just test-sql 2>&1 | tail -10</automated>
  </verify>
  <done>
    - DESCRIBE SEMANTIC VIEW returns 8 columns: name, base_table, dimensions, metrics, filters, joins, facts, hierarchies
    - Views without facts/hierarchies show empty JSON arrays ([])
    - phase28_e2e.test DESCRIBE assertion updated and passing
    - All existing sqllogictests pass with the new column count
  </done>
</task>

<task type="auto">
  <name>Task 3: End-to-end sqllogictest + proptests for FACTS and HIERARCHIES</name>
  <files>test/sql/phase29_facts_hierarchies.test, tests/parse_proptest.rs</files>
  <action>
1. **phase29_facts_hierarchies.test** -- Create comprehensive end-to-end sqllogictest:

   Test structure:
   ```sql
   -- Setup: create physical tables
   CREATE TABLE p29_orders (id INT, customer_id INT, region VARCHAR);
   CREATE TABLE p29_line_items (id INT, order_id INT, extended_price DECIMAL(10,2), discount DECIMAL(3,2), tax_rate DECIMAL(3,2));
   CREATE TABLE p29_customers (id INT, name VARCHAR, country VARCHAR, state VARCHAR, city VARCHAR);

   -- Insert test data with known arithmetic values for fact verification
   INSERT INTO p29_orders VALUES (1, 1, 'East'), (2, 2, 'West');
   INSERT INTO p29_line_items VALUES (1, 1, 100.00, 0.10, 0.05), (2, 1, 200.00, 0.20, 0.08), (3, 2, 150.00, 0.00, 0.10);
   INSERT INTO p29_customers VALUES (1, 'Alice', 'US', 'NY', 'NYC'), (2, 'Bob', 'US', 'CA', 'LA');
   ```

   Test cases:
   a. **CREATE with FACTS and HIERARCHIES** -- statement ok
   b. **Query metric that references a fact** -- verify arithmetic is correct after inlining
      - net_price = extended_price * (1 - discount)
      - SUM(li.net_price) should produce correct totals
   c. **Query metric with multi-level fact chain** -- fact A refs fact B
      - tax_amount = net_price * tax_rate (refs net_price fact)
      - SUM(li.tax_amount) should produce correct totals
   d. **DESCRIBE shows facts and hierarchies** -- query TTTTTTTT, verify JSON arrays contain correct entries
   e. **Error: fact cycle** -- statement error, two facts referencing each other
   f. **Error: unknown fact reference** -- statement error, fact references non-existent fact
   g. **Error: hierarchy references non-existent dimension** -- statement error
   h. **Error: facts source table not in TABLES** -- statement error
   i. **FACTS without HIERARCHIES** -- statement ok (hierarchies optional)
   j. **HIERARCHIES without FACTS** -- statement ok (facts optional)
   k. **DROP and cleanup** -- DROP SEMANTIC VIEW, DROP TABLE

   Arithmetic verification:
   - Line item 1: extended_price=100, discount=0.10 -> net_price = 100 * 0.90 = 90.00, tax = 90 * 0.05 = 4.50
   - Line item 2: extended_price=200, discount=0.20 -> net_price = 200 * 0.80 = 160.00, tax = 160 * 0.08 = 12.80
   - Line item 3: extended_price=150, discount=0.00 -> net_price = 150 * 1.00 = 150.00, tax = 150 * 0.10 = 15.00
   - Total net_price (East/order1): 90 + 160 = 250.00; (West/order2): 150.00
   - Total tax_amount (East/order1): 4.50 + 12.80 = 17.30; (West/order2): 15.00

2. **parse_proptest.rs** -- Add proptest generators for FACTS and HIERARCHIES:
   - Add `arb_fact_entry()` strategy: generates `alias.name AS expr` where alias/name are `arb_identifier()` and expr is `arb_expr()` (reuse existing proptest strategies)
   - Add `arb_facts_clause()` strategy: generates `FACTS (entry1, entry2, ...)` with 0-5 fact entries
   - Add `arb_hierarchy_entry()` strategy: generates `name AS (dim1, dim2, dim3)` with 1-5 level names (each is `arb_identifier()`)
   - Add `arb_hierarchies_clause()` strategy: generates `HIERARCHIES (entry1, entry2, ...)` with 0-3 hierarchy entries
   - Update the existing `arb_keyword_body()` strategy (or equivalent) to optionally include FACTS and HIERARCHIES clauses between RELATIONSHIPS and DIMENSIONS
   - Add proptest: `parse_keyword_body(generated_body)` either succeeds or returns a well-formed ParseError (no panics, no crashes)

3. **Fuzz target** -- If `fuzz/fuzz_targets/fuzz_ddl_parse.rs` exists, add seed corpus entries with FACTS and HIERARCHIES clauses to the corpus directory. If no fuzz target exists, note this for future work.
  </action>
  <verify>
    <automated>cd /Users/paul/Documents/Dev/Personal/duckdb-semantic-views && cargo test -- --nocapture 2>&1 | tail -10 && just build && just test-sql 2>&1 | tail -10</automated>
  </verify>
  <done>
    - phase29_facts_hierarchies.test covers: DDL with facts/hierarchies, query with fact inlining (arithmetic verified), multi-level fact chains, DESCRIBE output, error cases (cycles, unknown refs, unknown dims)
    - Proptests generate adversarial FACTS and HIERARCHIES clause input without panics
    - just test-all passes (full quality gate)
  </done>
</task>

</tasks>

<verification>
After all tasks:
- `just test-all` passes (full quality gate per CLAUDE.md)
- End-to-end: CREATE SEMANTIC VIEW with FACTS and HIERARCHIES -> query metrics referencing facts -> DESCRIBE shows all 8 columns -> DROP
- Arithmetic correctness: fact inlining produces correct aggregation results (verified by known test data)
- Error cases: fact cycles, unknown fact refs, unknown hierarchy dims all rejected at define time
- Backward compatibility: all phase28_e2e.test tests pass with updated DESCRIBE column count
- Proptests: adversarial FACTS/HIERARCHIES input does not crash parser
</verification>

<success_criteria>
1. Metric expressions with fact references expand to correct SQL with parenthesized inlined expressions
2. Multi-level fact chains (fact refs fact) resolve via topological order
3. Word-boundary substitution prevents net_price matching net_price_total
4. DESCRIBE SEMANTIC VIEW returns 8 columns including facts and hierarchies JSON arrays
5. sqllogictest verifies full lifecycle with arithmetic correctness
6. Proptests cover FACTS/HIERARCHIES clause parsing with adversarial input
7. `just test-all` passes
</success_criteria>

<output>
After completion, create `.planning/phases/29-facts-clause-hierarchies/29-02-SUMMARY.md`
</output>
