[FEATURE] Constrain codebase to statically typed Python subset
websocketwampenhancement
## Summary
Constrain the **autobahn-python** codebase to a *statically typed Python subset*, enabling:
1. **All objects have a static type** — either automatically inferred by the type checker or explicitly annotated
2. **Implicit `Any` is forbidden** — must be eliminated or explicitly justified
3. **Public API surface is fully typed** and safely inferable by tooling
4. **Alignment with modern (2025) Python typing best practices** and PEP-compliant conventions
This constraint is a prerequisite for future work toward **SLSA Level 4** — compiling typed Python to WASM components for reproducible, verifiable builds.
See also:
* https://github.com/crossbario/txaio/issues/214
---
## Related: PR #1838
This issue supersedes and extends the work started in **PR #1838** (@bblommers).
PR #1838 adds type hints to a subset of the codebase:
- `autobahn/twisted/websocket.py`
- `autobahn/websocket/protocol.py`
- `autobahn/wamp/message.py`
- `autobahn/wamp/types.py`
This issue establishes the **comprehensive style guide and acceptance criteria** that PR #1838 and all future typing work should align with.
See [PR #1838 Style Analysis](#pr-1838-style-analysis) below for detailed comparison.
---
## Strategic Context
This issue is part of a broader initiative to enable **deterministic, reproducible compilation** of the WAMP Python ecosystem (txaio, autobahn-python, zlmdb, cfxdb, wamp-xbr, crossbar) to WebAssembly.
The key architectural principle is:
> **Python is treated as a *source language*, not a runtime platform.**
Type checking is not merely a linting step — it is the **first stage of compilation**. To use type tools as a **compiler frontend**, we must ensure **every symbol's type is statically known**.
See design documents:
- [Toward SLSA Level 4: Compiling Typed Python to WASM Components](https://github.com/typedefint/system-and-software-assurance/blob/main/MIL/plane2/TYPED_PYTHON_TO_WASM.md) (private)
- [Toward SLSA Level 4: Reproducible Application Builds via WASM and WASI](https://github.com/typedefint/system-and-software-assurance/blob/main/MIL/plane2/UNIV_BINARY_SLSA_FUTURE_L4.md) (private)
---
## Why autobahn-python?
autobahn-python is our core WAMP client library for Python, and fundamental to [crossbar](https://github.com/crossbario/crossbar) as our WAMP router as well:
* Implements WAMP protocol (RPC, PubSub) over WebSocket and RawSocket transports
* Supports both Twisted and asyncio
* Large codebase with significant public API surface
* Foundation for crossbar and downstream applications
The codebase currently has:
* Partial type annotations (inconsistent coverage)
* Mixed typing styles (legacy `Optional`, `Union` alongside modern syntax)
* Unannotated functions, especially in older modules
* Implicit attribute creation patterns
This must be addressed systematically across all modules.
---
## PR #1838 Style Analysis
PR #1838 by @bblommers is a valuable contribution that adds type hints to several files. Below is an analysis of the style choices made and their alignment with this project's style guide.
### Files Modified in PR #1838
| File | Scope |
|------|-------|
| `autobahn/twisted/websocket.py` | Twisted WebSocket adapter |
| `autobahn/websocket/protocol.py` | Core WebSocket protocol |
| `autobahn/wamp/message.py` | WAMP message types |
| `autobahn/wamp/types.py` | WAMP type definitions |
### Style Choices: Aligned with Style Guide
| Choice | Example from PR | Status |
|--------|-----------------|--------|
| Union syntax `X \| Y` | `str \| None`, `int \| str` | **Aligned** (PEP 604) |
| Lowercase generics | `dict[str, Any]`, `tuple[str, dict]`, `list[bytes]` | **Aligned** (PEP 585) |
| `@overload` decorator | Used in `check_or_raise_uri()`, `Timings.diff()` | **Aligned** (PEP 484) |
| `Literal` for overloads | `Literal[True]`, `Literal[False]` | **Aligned** (PEP 586) |
| Remove `:type:` from docstrings | Removed redundant type annotations | **Aligned** |
| Return type annotations | `-> None`, `-> str`, `-> tuple[...]` | **Aligned** |
### Style Choices: Divergent from Style Guide
| Issue | Current in PR | Required by Style Guide | Action Required |
|-------|---------------|------------------------|-----------------|
| **`from __future__ import annotations`** | Not present | Required in every module | Add to all files |
| **Legacy `typing` imports** | `from typing import Optional` still present | Should be removed (use `X \| None`) | Remove and migrate |
| **Mixed `Optional` usage** | `Optional[str]` in some places | Use `str \| None` everywhere | Migrate all |
| **Incomplete parameter types** | `payload` untyped in several methods | All parameters must be typed | Add missing types |
### Specific Examples of Divergence
**1. Missing `from __future__ import annotations`:**
```python
# CURRENT (autobahn/twisted/websocket.py line 28)
from typing import Any, Optional
# REQUIRED
from __future__ import annotations
from typing import Any # Optional no longer needed
```
**2. Legacy `Optional` still used:**
```python
# CURRENT (autobahn/twisted/websocket.py)
peer: Optional[str] = None
is_server: Optional[bool] = None
# REQUIRED
peer: str | None = None
is_server: bool | None = None
```
**3. Untyped parameters:**
```python
# CURRENT (autobahn/websocket/protocol.py)
def _onMessageFrameData(self, payload) -> None: # payload untyped
def _onMessageFrame(self, payload) -> None: # payload untyped
def _onPing(self, payload) -> None: # payload untyped
# REQUIRED
def _onMessageFrameData(self, payload: bytes) -> None:
def _onMessageFrame(self, payload: bytes) -> None:
def _onPing(self, payload: bytes) -> None:
```
**4. Partially typed return in `__iter__`:**
```python
# CURRENT (autobahn/websocket/protocol.py)
def __iter__(self) -> Iterable[str]:
return self._timings.__iter__()
# BETTER (more precise)
def __iter__(self) -> Iterator[str]:
return iter(self._timings)
```
### Coverage Gap
PR #1838 covers only **4 files** out of the full autobahn-python codebase. Major modules still requiring typing:
| Module | Status | Priority |
|--------|--------|----------|
| `autobahn/wamp/protocol.py` | Untyped | High |
| `autobahn/wamp/component.py` | Untyped | High |
| `autobahn/wamp/serializer.py` | Untyped | Medium |
| `autobahn/wamp/auth.py` | Untyped | Medium |
| `autobahn/wamp/cryptosign.py` | Untyped | Medium |
| `autobahn/asyncio/websocket.py` | Untyped | High |
| `autobahn/asyncio/wamp.py` | Untyped | High |
| `autobahn/twisted/wamp.py` | Untyped | High |
| `autobahn/websocket/compress*.py` | Untyped | Low |
| `autobahn/rawsocket/*.py` | Untyped | Medium |
### Recommendation for PR #1838
To align PR #1838 with this style guide:
1. Add `from __future__ import annotations` to all modified files
2. Replace all `Optional[X]` with `X | None`
3. Remove unused `typing` imports (`Optional`, `Union`, `Dict`, `Tuple`)
4. Add missing parameter types (especially `payload: bytes`)
5. Run `ruff check --select ANN,UP` and fix violations
---
## What "Statically Typed Subset" Means
### Required Typing Discipline
**All public functions and methods must have:**
- Parameter type annotations
- Explicit return type annotation
```python
async def call(
self,
procedure: str,
*args: Any,
**kwargs: Any,
) -> Any:
...
```
**Every class must declare instance attributes upfront:**
```python
class Session:
_transport: ITransport | None
_session_id: int | None
_realm: str | None
_authid: str | None
_authrole: str | None
def __init__(self) -> None:
self._transport = None
self._session_id = None
...
```
**All module-level globals must be typed:**
```python
_log: Logger = make_logger()
WAMP_SERIALIZERS: Final[dict[str, type[Serializer]]] = {...}
```
**All containers must have explicit type parameters:**
```python
_subscriptions: dict[int, Subscription] = {}
_registrations: dict[int, Registration] = {}
_pending_calls: dict[int, Future[Any]] = {}
```
### Forbidden Patterns
**Implicit `Any`:**
```python
# BAD
items = []
# GOOD
items: list[Message] = []
```
**Dynamic attribute creation after `__init__`:**
```python
# BAD
self.foo = 1
# GOOD
class Bar:
foo: int
def __init__(self) -> None:
self.foo = 1
```
**Runtime type hacks:**
```python
# FORBIDDEN
eval("...")
exec("...")
getattr(obj, dynamic_name) # where dynamic_name is not a literal
```
---
## Python Typing Style Guide
This project follows **modern Python 3.11+ typing conventions** aligned with official PEPs.
### Minimum Python Version
**Python 3.11+** is required. This enables:
- `Self` type (PEP 673)
- `Required` / `NotRequired` for TypedDict (PEP 655)
- ExceptionGroup and `except*` syntax
- Native union syntax without `from __future__ import annotations`
### Required Import
**Every module must begin with:**
```python
from __future__ import annotations
```
This enables:
- Forward references without quotes (PEP 563)
- Consistent annotation behavior
- Future compatibility with PEP 649 (Python 3.14+)
### Union Types (PEP 604)
**Use `X | Y` syntax, not `Union[X, Y]` or `Optional[X]`:**
```python
# GOOD
def process(value: str | None) -> int | str:
...
# BAD
def process(value: Optional[str]) -> Union[int, str]:
...
```
### Built-in Generic Types (PEP 585)
**Use lowercase built-in generics, not `typing` module equivalents:**
```python
# GOOD
items: list[int] = []
mapping: dict[str, bytes] = {}
pair: tuple[int, str] = (1, "a")
# BAD
items: List[int] = []
mapping: Dict[str, bytes] = {}
```
### Type Aliases
```python
# GOOD
Callback: TypeAlias = Callable[[int], None]
# For Python 3.12+, prefer PEP 695 syntax:
type Callback = Callable[[int], None]
```
### TypeVar and Generics
```python
from typing import TypeVar
T = TypeVar("T")
def identity(x: T) -> T:
return x
```
### Protocol for Structural Typing (PEP 544)
**Prefer `Protocol` over ad-hoc duck typing:**
```python
from typing import Protocol
class ITransport(Protocol):
def send(self, data: bytes) -> None: ...
def close(self) -> None: ...
```
### Constants with Final (PEP 591)
```python
from typing import Final
WAMP_VERSION: Final[int] = 2
DEFAULT_REALM: Final[str] = "realm1"
```
### Literal Types (PEP 586)
```python
from typing import Literal
def set_mode(mode: Literal["json", "msgpack", "cbor"]) -> None:
...
```
### Overloads for Conditional Return Types (PEP 484)
```python
from typing import Literal, overload
@overload
def get_session(create: Literal[True]) -> Session: ...
@overload
def get_session(create: Literal[False]) -> Session | None: ...
def get_session(create: bool = False) -> Session | None:
...
```
### Self Type (PEP 673)
```python
from typing import Self
class ComponentConfig:
def with_realm(self, realm: str) -> Self:
self._realm = realm
return self
```
### Avoiding `Any`
**`Any` defeats static analysis and must be avoided.**
If unavoidable:
1. Explicitly justify in a comment
2. Isolate to minimal scope
3. Wrap in a typed facade if possible
```python
# ACCEPTABLE: WAMP payload can be any JSON-serializable value
# This is fundamental to the protocol design
payload: Any
```
### Docstrings
**Remove redundant `:type:` and `:rtype:` annotations when type hints are present:**
```python
# GOOD
def connect(host: str, port: int) -> Connection:
"""
Establish a connection.
:param host: The hostname or IP address.
:param port: The port number.
:returns: An established connection.
"""
# BAD (redundant)
def connect(host: str, port: int) -> Connection:
"""
:param host: The hostname.
:type host: str # REMOVE
:rtype: Connection # REMOVE
"""
```
---
## Type Checking Configuration
### Primary Tool: `ty` (strict mode)
[ty](https://github.com/astral-sh/ty) is the authoritative type checker for this project.
**Configuration in `pyproject.toml`:**
```toml
[tool.ty]
python-version = "3.11"
```
**Strict mode invocation:**
```bash
ty check --warn any-type
```
### Linting: `ruff`
```toml
[tool.ruff]
target-version = "py311"
line-length = 120
[tool.ruff.lint]
select = [
"ANN", # flake8-annotations
"I", # isort
"E", # pycodestyle errors
"F", # pyflakes
"W", # pycodestyle warnings
"UP", # pyupgrade (modernize syntax)
"TCH", # flake8-type-checking (imports)
]
[tool.ruff.lint.flake8-annotations]
mypy-init-return = true
suppress-none-returning = false
allow-star-arg-any = false
[tool.ruff.lint.isort]
required-imports = ["from __future__ import annotations"]
```
---
## Implementation Workflow
### Phase 1: Configuration
1. Update `pyproject.toml` with ruff and ty configuration
2. Add `from __future__ import annotations` to all modules
3. Update `justfile` with type checking recipes
### Phase 2: Align PR #1838
1. Update PR #1838 to match style guide
2. Merge PR #1838 as foundation
### Phase 3: Progressive Typing
Priority order:
1. **Core WAMP protocol** (`wamp/protocol.py`, `wamp/component.py`)
2. **Asyncio bindings** (`asyncio/websocket.py`, `asyncio/wamp.py`)
3. **Twisted bindings** (`twisted/wamp.py`)
4. **Serializers and auth** (`wamp/serializer.py`, `wamp/auth.py`)
5. **WebSocket internals** (`websocket/*.py`)
6. **RawSocket** (`rawsocket/*.py`)
### Phase 4: CI Integration
1. Add type checking to CI workflow
2. Gate PRs on passing type checks
3. Document typed subset contract
---
## Scope and Constraints
**In scope:**
- Add type annotations to all public APIs
- Declare class-level attributes
- Type all containers explicitly
- Add `from __future__ import annotations` to all files
- Migrate legacy typing syntax (`Optional` → `X | None`, etc.)
**Out of scope (for this issue):**
- Runtime behavior changes
- Logic rewrites (unless required for stable types)
- Test typing (tracked separately)
---
## Acceptance Criteria
- [ ] All public functions/methods have parameter and return type annotations
- [ ] All classes declare instance attributes at class level
- [ ] All containers have explicit type parameters
- [ ] No implicit `Any` (explicit `Any` only with justification)
- [ ] `from __future__ import annotations` in every module
- [ ] No legacy `Optional`, `Union`, `Dict`, `List`, `Tuple` imports
- [ ] `ty check --warn any-type` passes with zero errors
- [ ] `ruff check .` passes with zero errors (ANN rules enabled)
- [ ] Type checking added to CI
- [ ] PR #1838 aligned and merged
---
## Related Work
- **PR #1838**: Initial type hints contribution by @bblommers — to be aligned with this style guide
- **txaio typed subset**: Foundation library, being typed in parallel
- **SLSA Level 3 implementation**: Current focus on provenance; typed subset enables future Level 4
- **WASM compilation roadmap**: Typed subset is prerequisite for Python → WASM compiler frontend
---
## References
### PEPs
| PEP | Title | Relevance |
|-----|-------|-----------|
| [PEP 484](https://peps.python.org/pep-0484/) | Type Hints | Foundation |
| [PEP 526](https://peps.python.org/pep-0526/) | Variable Annotations | Class attributes |
| [PEP 544](https://peps.python.org/pep-0544/) | Protocols: Structural subtyping | Interface typing |
| [PEP 563](https://peps.python.org/pep-0563/) | Postponed Evaluation of Annotations | `from __future__ import annotations` |
| [PEP 585](https://peps.python.org/pep-0585/) | Type Hinting Generics In Standard Collections | `list[T]` vs `List[T]` |
| [PEP 586](https://peps.python.org/pep-0586/) | Literal Types | `Literal["a", "b"]` |
| [PEP 591](https://peps.python.org/pep-0591/) | Adding a final qualifier | `Final[T]` |
| [PEP 604](https://peps.python.org/pep-0604/) | Union Operators | `X \| Y` syntax |
| [PEP 612](https://peps.python.org/pep-0612/) | Parameter Specification Variables | `ParamSpec` |
| [PEP 655](https://peps.python.org/pep-0655/) | Required and NotRequired for TypedDict | TypedDict fields |
| [PEP 673](https://peps.python.org/pep-0673/) | Self Type | `Self` return type |
| [PEP 695](https://peps.python.org/pep-0695/) | Type Parameter Syntax | Python 3.12+ `type` statement |
### Tools
- [ty](https://github.com/astral-sh/ty) — Astral's type checker (strict mode)
- [ruff](https://github.com/astral-sh/ruff) — Fast Python linter with annotation rules
- [pyright](https://github.com/microsoft/pyright) — Alternative type checker (reference)
---
## Checklist
- [x] I have searched existing issues to avoid duplicates
- [x] I have described the problem clearly
- [x] I have provided use cases
- [x] I have considered alternatives
- [x] I have assessed impact and breaking changes
0 条评论