---
phase: quick-260321-i40
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - docs/_ext/sqlgrammar_lexer.py
  - docs/_static/css/sqlgrammar.css
  - docs/conf.py
  - docs/reference/create-semantic-view.rst
  - docs/reference/drop-semantic-view.rst
  - docs/reference/describe-semantic-view.rst
  - docs/reference/show-semantic-views.rst
  - docs/reference/semantic-view-function.rst
  - docs/reference/explain-semantic-view-function.rst
autonomous: false
requirements: []

must_haves:
  truths:
    - "Syntax sections in reference docs render with 4 distinct colors for keywords, placeholders, optional brackets, and string literals"
    - "Example sections in reference docs continue to render with standard SQL highlighting unchanged"
    - "Sphinx build completes without warnings related to the lexer or code blocks"
  artifacts:
    - path: "docs/_ext/sqlgrammar_lexer.py"
      provides: "Custom Pygments lexer for SQL grammar notation"
      min_lines: 30
    - path: "docs/_static/css/sqlgrammar.css"
      provides: "Custom CSS for 4-color token styling"
      min_lines: 10
  key_links:
    - from: "docs/conf.py"
      to: "docs/_ext/sqlgrammar_lexer.py"
      via: "sys.path insert + add_lexer() in setup()"
      pattern: "add_lexer.*sqlgrammar"
    - from: "docs/reference/*.rst"
      to: "docs/_ext/sqlgrammar_lexer.py"
      via: "code-block:: sqlgrammar directive in Syntax sections"
      pattern: "code-block:: sqlgrammar"
    - from: "docs/conf.py"
      to: "docs/_static/css/sqlgrammar.css"
      via: "html_css_files or html_static_path"
      pattern: "sqlgrammar.css"
---

<objective>
Create a custom Pygments lexer for SQL grammar syntax highlighting in reference docs, register it in Sphinx, apply custom CSS for 4-color rendering, and update all reference doc Syntax sections to use it.

Purpose: Syntax sections in reference docs currently render as plain SQL, but they contain grammar notation (angle-bracket placeholders, optional brackets, ellipsis) that SQL lexers mishandle. A dedicated lexer produces clean, consistent, Snowflake-style 4-color highlighting.

Output: Custom lexer, CSS, updated conf.py, 6 updated .rst files, verified Sphinx build.
</objective>

<execution_context>
@/Users/paul/.claude/get-shit-done/workflows/execute-plan.md
@/Users/paul/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@docs/conf.py
@docs/pyproject.toml
@docs/reference/create-semantic-view.rst
@docs/reference/drop-semantic-view.rst
@docs/reference/describe-semantic-view.rst
@docs/reference/show-semantic-views.rst
@docs/reference/semantic-view-function.rst
@docs/reference/explain-semantic-view-function.rst
</context>

<tasks>

<task type="auto">
  <name>Task 1: Create the custom Pygments lexer, CSS, and register in conf.py</name>
  <files>docs/_ext/sqlgrammar_lexer.py, docs/_static/css/sqlgrammar.css, docs/conf.py</files>
  <action>
**1. Create `docs/_ext/sqlgrammar_lexer.py`:**

Create directory `docs/_ext/` (with empty `__init__.py`). Create a Pygments `RegexLexer` subclass named `SqlGrammarLexer` with:

- `name = 'sqlgrammar'`
- `aliases = ['sqlgrammar']`

Token rules (in `tokens['root']` list, ORDER MATTERS -- put more specific patterns first):

1. **Whitespace**: `r'\s+'` -> `Token.Text`
2. **String literals** (single-quoted): `r"'[^']*'"` -> `Token.Literal.String`
3. **Placeholders** (angle-bracket): `r'<[a-zA-Z_][a-zA-Z0-9_]*>'` -> `Token.Name.Variable`
4. **Optional brackets and ellipsis**: `r'[\[\]]'` -> `Token.Name.Tag`, and `r'\.\.\.'` -> `Token.Name.Tag`
5. **SQL keywords** (case-insensitive, word-boundary): Use `words()` helper from `pygments.lexer`:
   ```python
   (words((
       'CREATE', 'OR', 'REPLACE', 'SEMANTIC', 'VIEW', 'IF', 'NOT', 'EXISTS',
       'AS', 'TABLES', 'RELATIONSHIPS', 'FACTS', 'DIMENSIONS', 'METRICS',
       'PRIMARY', 'KEY', 'UNIQUE', 'REFERENCES', 'USING',
       'DROP', 'DESCRIBE', 'SHOW',
       'SELECT', 'FROM', 'WHERE', 'ORDER', 'BY', 'LIMIT',
   ), prefix=r'\b', suffix=r'\b'), Token.Keyword),
   ```
6. **Everything else** (punctuation, identifiers, commas, parens, semicolons): `r'[^\s]+'` -> `Token.Text` (fallback, renders unstyled)

Import from `pygments.lexer` (RegexLexer, words) and `pygments.token` (Token).

**2. Create `docs/_static/css/sqlgrammar.css`:**

Create directory `docs/_static/css/`. Add CSS rules that target Pygments-generated spans. Pygments wraps tokens in `<span class="X">` where X is the short token name. Use these selectors scoped to `.highlight-sqlgrammar` (Sphinx adds `highlight-{lang}` class to the wrapper div):

```css
/* SQL Grammar syntax highlighting — 4-color scheme */
.highlight-sqlgrammar .k  { color: #2E7D32; font-weight: bold; }  /* Keyword — green */
.highlight-sqlgrammar .nv { color: #1565C0; font-style: italic; } /* Name.Variable (placeholders) — blue */
.highlight-sqlgrammar .nt { color: #6A1B9A; }                     /* Name.Tag (brackets, ellipsis) — purple */
.highlight-sqlgrammar .s  { color: #C62828; }                     /* String — red */
```

Notes:
- `.k` = Keyword token class
- `.nv` = Name.Variable token class
- `.s` = String (also covers `.s1` for single-quoted)
- `.nt` = Name.Tag token class
- Test the actual CSS classes by building and inspecting. Pygments may use `.s1` for single-quoted strings; if so, add `.highlight-sqlgrammar .s1` too.

Also add a dark-mode variant for Shibuya's auto color mode:
```css
[data-theme="dark"] .highlight-sqlgrammar .k  { color: #66BB6A; font-weight: bold; }
[data-theme="dark"] .highlight-sqlgrammar .nv { color: #64B5F6; font-style: italic; }
[data-theme="dark"] .highlight-sqlgrammar .nt { color: #CE93D8; }
[data-theme="dark"] .highlight-sqlgrammar .s,
[data-theme="dark"] .highlight-sqlgrammar .s1 { color: #EF9A9A; }
```

**3. Update `docs/conf.py`:**

Add these changes to the existing conf.py (do NOT remove existing content):

a. At the top of the file, add:
```python
import sys
import os
sys.path.insert(0, os.path.abspath("_ext"))
```

b. Add `html_static_path` and `html_css_files` after the theme options:
```python
html_static_path = ["_static"]
html_css_files = ["css/sqlgrammar.css"]
```

c. Add a `setup()` function at the bottom:
```python
def setup(app):
    from sqlgrammar_lexer import SqlGrammarLexer
    app.add_lexer("sqlgrammar", SqlGrammarLexer)
```

Note: `add_lexer()` in modern Sphinx (7.x+) takes the lexer CLASS, not an instance. Do NOT pass `SqlGrammarLexer()` -- pass the class itself.
  </action>
  <verify>
    <automated>cd /Users/paul/Documents/Dev/Personal/duckdb-semantic-views/docs && python -c "import sys; sys.path.insert(0, '_ext'); from sqlgrammar_lexer import SqlGrammarLexer; lex = SqlGrammarLexer(); tokens = list(lex.get_tokens('CREATE SEMANTIC VIEW <name> AS')); print(tokens); assert any('Keyword' in str(t[0]) for t in tokens), 'No keyword tokens'; assert any('Variable' in str(t[0]) for t in tokens), 'No variable tokens'; print('OK')"</automated>
  </verify>
  <done>SqlGrammarLexer importable and tokenizes SQL grammar notation into 4 token types. CSS file exists with light and dark mode rules. conf.py registers the lexer and CSS.</done>
</task>

<task type="auto">
  <name>Task 2: Update Syntax sections in all 6 reference docs</name>
  <files>docs/reference/create-semantic-view.rst, docs/reference/drop-semantic-view.rst, docs/reference/describe-semantic-view.rst, docs/reference/show-semantic-views.rst, docs/reference/semantic-view-function.rst, docs/reference/explain-semantic-view-function.rst</files>
  <action>
Change `.. code-block:: sql` to `.. code-block:: sqlgrammar` ONLY in the "Syntax" sections of each file. Leave ALL other code blocks (in Examples, Clauses, Query Modes, Filtering, Ordering, etc.) as `.. code-block:: sql`.

Specific edits (each file has exactly ONE syntax block to change):

1. **`create-semantic-view.rst`** line 15: change `.. code-block:: sql` to `.. code-block:: sqlgrammar`
   - This is the block under the `Syntax` heading (line 12-13), directly after `.. _ref-create-syntax:` label
   - Do NOT change any of the other 8+ code blocks in this file (they are all in Clauses and Examples sections)

2. **`drop-semantic-view.rst`** line 15: change `.. code-block:: sql` to `.. code-block:: sqlgrammar`
   - Under the `Syntax` heading. Do NOT change the Examples code block.

3. **`describe-semantic-view.rst`** line 15: change `.. code-block:: sql` to `.. code-block:: sqlgrammar`
   - Under the `Syntax` heading. Do NOT change any code blocks in Examples.

4. **`show-semantic-views.rst`** line 15: change `.. code-block:: sql` to `.. code-block:: sqlgrammar`
   - Under the `Syntax` heading. Do NOT change code blocks in Examples.

5. **`semantic-view-function.rst`** line 15: change `.. code-block:: sql` to `.. code-block:: sqlgrammar`
   - Under the `Syntax` heading (after `.. _ref-sv-syntax:`). Do NOT change any of the many code blocks in Query Modes, Filtering, Ordering, Name Resolution, or Examples sections.

6. **`explain-semantic-view-function.rst`** line 15: change `.. code-block:: sql` to `.. code-block:: sqlgrammar`
   - Under the `Syntax` heading (after `.. _ref-explain-syntax:`). Do NOT change the Examples code blocks.

**Verification approach**: After editing, grep for `sqlgrammar` to confirm exactly 6 occurrences across the 6 files, and grep for remaining `code-block:: sql` to confirm they are all in non-Syntax sections.
  </action>
  <verify>
    <automated>cd /Users/paul/Documents/Dev/Personal/duckdb-semantic-views && grep -rn "code-block:: sqlgrammar" docs/reference/ | wc -l | tr -d ' ' && test "$(grep -rn 'code-block:: sqlgrammar' docs/reference/ | wc -l | tr -d ' ')" = "6" && echo "OK: exactly 6 sqlgrammar blocks"</automated>
  </verify>
  <done>Exactly 6 Syntax section code blocks changed to sqlgrammar. All other code blocks remain as sql.</done>
</task>

<task type="checkpoint:human-verify" gate="blocking">
  <name>Task 3: Verify rendered output</name>
  <files>docs/_build/html/reference/create-semantic-view.html</files>
  <action>
Build the docs and present to user for visual verification. Run: `cd docs && uv run sphinx-build -b html . _build/html`

Confirm build succeeds with no lexer-related warnings. Then pause for human verification of the rendered output.
  </action>
  <verify>Sphinx build exits 0 with no lexer warnings. User confirms visual output.</verify>
  <done>User confirms 4-color syntax highlighting renders correctly in Syntax sections and Examples sections are unchanged.</done>
  <what-built>Custom Pygments lexer for SQL grammar notation with 4-color highlighting applied to all 6 reference doc Syntax sections.</what-built>
  <how-to-verify>
    1. Build the docs: `cd docs && uv run sphinx-build -b html . _build/html`
    2. Open `docs/_build/html/reference/create-semantic-view.html` in a browser
    3. Verify the Syntax section shows 4 distinct colors:
       - SQL keywords (CREATE, SEMANTIC, VIEW, AS, TABLES, etc.) in green/bold
       - Placeholders (name, column, expression) in blue/italic
       - Optional brackets [ ] and ellipsis ... in purple
       - String literals (if any) in red
    4. Scroll to Examples section -- verify it still has standard SQL highlighting (different from the Syntax section)
    5. Spot-check 1-2 other reference pages (e.g., drop-semantic-view.html, semantic-view-function.html)
    6. Toggle dark mode (Shibuya theme toggle) and verify colors still look good
  </how-to-verify>
  <resume-signal>Type "approved" or describe any color/styling issues to fix</resume-signal>
</task>

</tasks>

<verification>
- `cd docs && python -c "import sys; sys.path.insert(0, '_ext'); from sqlgrammar_lexer import SqlGrammarLexer; print('Lexer OK')"` -- lexer is importable
- `grep -rn 'code-block:: sqlgrammar' docs/reference/ | wc -l` returns 6
- `cd docs && uv run sphinx-build -b html . _build/html 2>&1 | grep -i 'warning\|error' | grep -v 'intersphinx'` returns no lexer-related warnings
</verification>

<success_criteria>
- 4-color syntax highlighting visible in all 6 reference doc Syntax sections
- Example sections unchanged (standard SQL highlighting)
- Sphinx build clean (no lexer warnings)
- Dark mode colors work
</success_criteria>

<output>
After completion, create `.planning/quick/260321-i40-custom-pygments-lexer-for-sql-grammar-sy/260321-i40-SUMMARY.md`
</output>
