Adopt uv workspaces for shared internal Python modules and a single publish flow
enhancementc/dxpythonc/clientc/infrapython:uv
Extend the Python side of the monorepo to a first-class **uv workspace** so we can factor out internal modules (a `phoenix-config` equivalent, shared rate limiting, shared executors) and publish everything through one flow.
## Where we are today
A uv workspace already exists, but it only links the packages we *already* ship:
```toml
# pyproject.toml:453
[tool.uv.workspace]
members = ["packages/*"]
exclude = ["packages/phoenix-sqlean"]
[tool.uv.sources]
arize-phoenix-client = { workspace = true }
arize-phoenix-evals = { workspace = true }
arize-phoenix-otel = { workspace = true }
```
Every member is a public PyPI distribution with its own `release-please` entry. There is no lane for a module that several packages need but that isn't a product in its own right, so the fallback has been copy-paste.
## The cost of no shared lane
Two files are duplicated between `arize-phoenix-client` and `arize-phoenix-evals`, and they have **already drifted**:
| Module | client | evals |
|---|---|---|
| `rate_limiters.py` | 282 lines | 297 lines |
| `executors.py` | 670 lines | 655 lines |
Concrete drift in `rate_limiters.py`: the evals copy carries a `printif()` tqdm helper the client copy lacks, and the two initialize the token bucket differently (`self.tokens = 1.0` in evals vs `0.0` in client). A rate-limiter fix landed in one package silently does not reach the other.
## Goal
- Make the uv workspace the unit of Python development: one lockfile, one resolution, `uv sync` gives every package plus every internal module wired up.
- Add internal workspace members that are **not** product surfaces — starting with shared rate limiting / executors, and a Python analogue of `@arizeai/phoenix-config`.
- Keep a single publish flow: adding a workspace member should not mean hand-editing `.github/workflows/publish.yaml`.
## Design questions to settle first
1. **How do internal modules reach end users?** uv workspace sources are dev-time only — a published wheel needs a real dependency. Options:
- publish the shared package to PyPI under its own name (e.g. `arize-phoenix-common`) and depend on it normally — simplest, but it becomes a public, versioned surface we have to support;
- vendor it into each wheel at build time via a hatch build hook — keeps it private, but complicates the build and can produce duplicate module names on `sys.path`.
2. **Versioning.** Does the shared package get its own `release-please` entry and semver, or is it pinned lockstep to the packages that consume it? A shared package that changes on every release forces a release of all consumers.
3. **Namespace.** All packages already publish under the `phoenix.*` namespace (`phoenix.client`, `phoenix.evals`, `phoenix.otel`). Where does shared code live — `phoenix.common`? — without colliding with the root `phoenix` package in `src/`.
4. **Publish flow.** `list-python-packages` in `publish.yaml` derives its matrix from `.release-please-manifest.json` + a PyPI existence check, so any new manifest entry becomes publishable by default. Confirm that's the behavior we want for internal members, or add an explicit opt-out.
5. **Does `packages/phoenix-sqlean` join?** It's currently excluded as a `setup.py`-based C extension with its own build matrix; leaving it out is probably still right.
## Scope
- [ ] Decide distribution strategy for internal modules (question 1) and write it down
- [ ] Create the first shared internal package and move `rate_limiters.py` / `executors.py` into it, reconciling the drift between the client and evals copies
- [ ] Update `arize-phoenix-client` and `arize-phoenix-evals` to consume it, keeping their public re-exports intact so downstream imports don't break
- [ ] Add a Python `phoenix-config` equivalent as the second consumer, mirroring `js/packages/phoenix-config`
- [ ] Verify the publish flow handles the new member without workflow edits, or extend it deliberately
- [ ] Document the workspace layout and how to add a new member
0 条评论