---
title: "Architecture"
description: "How mehen's parsers, analyzers, metric pipeline, and reporters compose."
keywords: ["architecture", "design", "internals"]
---

mehen's internals are organized around a clean separation between parsing, analyzing, and reporting.

## Crate layout

| Crate | Responsibility |
|---|---|
| `mehen-cli` | CLI binary — entry point, command routing, exit codes. |
| `mehen-engine` | Pipeline orchestration — `run_diff`, `run_top_offenders`, registry, language detection. |
| `mehen-core` | Parser-neutral domain types and the `LanguageAnalyzer` trait. |
| `mehen-metrics` | Shared metric formulas, accumulators, finalizers (Halstead, cyclomatic, cognitive, MI, ABC, LOC, NOM, NPA, NPM, WMC). |
| `mehen-<lang>` | Per-language analyzer crates. Each owns parsing and metric interpretation. |
| `mehen-tree-sitter` | Shared tree-sitter wrapper and CST traversal helpers, used only by the tree-sitter-backed languages. |
| `mehen-antlr` | Shared support for ANTLR-backed analyzers (Kotlin, Java, C#) — re-exports the runtime and provides char→byte span conversion, recovered-error diagnostics, and hidden-channel comment (CLOC) extraction. |
| `mehen-markdown` | Markdown analyzer (pulldown-cmark) with embedded-code dispatch via `LanguageDispatcher`. |
| `mehen-sql` | SQL analyzer (sqruff). Publishes a dedicated `sql.*` metric family instead of the source-code families; the sqruff CST is confined behind a parser-neutral `SqlFileFacts` adapter. |
| `mehen-git` | Git operations for `mehen diff`. |
| `mehen-report` | JSON, YAML, TOML, and GitHub-Markdown rendering. |
| `xtask` | Developer-only commands (kind-enum codegen, AST dumps, audits). |

## Parser diversity, single contract

A core architectural decision: **each language uses the parser best suited to it**, but every analyzer
returns the same `LanguageAnalysis` shape. mehen does not force a single AST model across languages.

| Language | Parser | What it gives us |
|---|---|---|
| Python | [Ruff](https://docs.astral.sh/ruff/) | Modern Python syntax (match, exception groups, f-strings, async); typed AST plus semantic model. |
| TS / JS / TSX / JSX | [Oxc](https://oxc.rs/) | Decorators, class fields, parameter properties, JSX nesting, `satisfies`, dynamic import. |
| PHP | [Mago](https://github.com/carthage-software/mago) | Attributes, promoted properties, enums, traits, readonly, null-safe calls, `match`. |
| Ruby | [Prism](https://ruby.github.io/prism/) | Blocks, lambdas, numbered params, rescue modifiers, endless methods, pattern matching. |
| Rust | [`ra_ap_syntax`](https://docs.rs/ra_ap_syntax/) | The same syntax library rust-analyzer uses internally. |
| Kotlin | [ANTLR](https://www.antlr.org/) (official [Kotlin spec grammar](https://github.com/Kotlin/kotlin-spec)) | Semantically-named rules for `when` entries, elvis `?:`, safe-call `?.`, `catch` blocks, labeled jumps, and property accessors — richer than a tree-sitter CST. |
| Java | [ANTLR](https://www.antlr.org/) ([grammars-v4 Java grammar](https://github.com/antlr/grammars-v4/tree/master/java/java)) | Records, sealed types, switch expressions, text blocks, pattern matching, modules — as first-class, semantically-named rules. |
| C# | [ANTLR](https://www.antlr.org/) ([Roslyn's generated grammar](https://github.com/dotnet/roslyn/blob/main/src/Compilers/CSharp/Portable/Generated/CSharp.Generated.g4)) | Generated from the same `Syntax.xml` model as the compiler's syntax nodes, so it tracks C# as implemented: records, `is not`, `and`/`or`/relational patterns, list patterns, collection expressions, raw strings, primary constructors, `required` members, and the C# 14 `field` and extension blocks — plus properties/accessors, indexers, events, local functions, expression-bodied members, `when` guards, interpolated strings, and `#if` directives. |
| SQL | [sqruff](https://github.com/quarylabs/sqruff) | Dialect-aware SQL CST (postgres, T-SQL, snowflake, bigquery, …) for CTE/join/subquery structure and object-touch analysis. |
| Markdown | [pulldown-cmark](https://github.com/pulldown-cmark/pulldown-cmark) | CommonMark + GFM with byte-accurate spans. |
| Go, C, PowerShell | tree-sitter | Mature grammars where tree-sitter's coverage is the best available. |

`mehen-metrics` owns the math (Halstead formulas, MI variants, cyclomatic accumulators); each
per-language analyzer owns the *interpretation* — which syntax constructs count as decisions, which
tokens classify as Halstead operators, which members count as public methods.

## Pipeline

```text
file path
  ↓
language detection (extension-driven)
  ↓
parser (per-language: Ruff / Oxc / Mago / Prism / ra_ap_syntax /
        ANTLR / sqruff / pulldown-cmark / tree-sitter)
  ↓
analyzer.analyze(tree)              ← per-language metric interpretation
  ↓
LanguageAnalysis  (Send + 'static)  ← parser-neutral, owned, no parser-arena lifetimes
  ↓
metric finalizers (mehen-metrics)
  ↓
reporter (mehen-report)             ← JSON / Markdown / YAML / TOML
```

The key constraint: `LanguageAnalysis` is parser-neutral and `Send + 'static`. That lets the engine
analyze files in an ignore-aware worker pool without arena-backed parsers (Oxc, Mago) leaking
lifetimes across thread boundaries. Directory discovery uses the same `.gitignore` semantics as
ripgrep, normalizes roots once, and excludes generated, vendored, and binary paths using each
repository's Git attributes. It processes each file directly in a traversal worker, so a producer
cannot accumulate an unbounded queue while parsers are busy.

## Markdown is special

The Markdown analyzer (`mehen-markdown`) parses the document into a block/inline AST and dispatches
code fences back through `LanguageDispatcher`. That is how
[Markdown Halstead](/metrics/markdown/halstead) and [MCC](/metrics/markdown/mcc) credit embedded code
blocks without duplicating language logic.

## ANTLR-backed analyzers

Kotlin, Java, and C# run through ANTLR v4 grammars generated to Rust ahead of time and checked in.
A normal `cargo build` uses those modules without compiling codegen; `cargo xtask antlr generate`
links the workspace-pinned `antlr-rust-codegen` crate directly. The shared `mehen-antlr` crate
carries the concerns every ANTLR analyzer needs but tree-sitter and the language-specific parsers
handle differently:

- **Span conversion.** ANTLR reports positions in character offsets; mehen's spans are byte offsets.
  `mehen-antlr` converts once, centrally.
- **Comments live on a hidden channel.** They are absent from the parse tree, so CLOC is recovered
  from the token stream rather than by walking nodes.
- **No parent pointers.** ANTLR rule contexts cannot look upward, so analyzers thread parent-dependent
  context (e.g. else-if detection) top-down as they descend the tree.

Despite those differences, the analyzers still return the same parser-neutral `LanguageAnalysis` as
every other backend. See [Add a new language → ANTLR](/developers/new-language) for the full flow.

## SQL is its own metric family

SQL does not fit the function/class-centric model that the source-code metrics assume — a declarative
`SELECT` has no methods, classes, or imperative branches to count. `mehen-sql` therefore publishes a
dedicated `sql.*` metric namespace (CTE graphs, join/subquery structure, object-touch risk, SQL
Halstead, composite scores) instead of the shared code families. The [sqruff](https://github.com/quarylabs/sqruff)
CST is confined to the crate's `facts.rs` behind a parser-neutral `SqlFileFacts` adapter, so the rest
of the pipeline never sees a sqruff type. Dialect resolution (directive-or-inference, validated before
sqruff ever runs) also lives inside the crate. See [SQL metrics](/metrics/sql/overview).

## See also

- [Spaces](/concepts/spaces) — the language-aware container model.
- [Output formats](/concepts/output-formats) — how each crate's output is rendered.
- [Developers → Add a new language](/developers/new-language).
