ITADN

Adopt uv workspaces for shared internal Python modules and a single publish flow

#15163Openmikeldking 创建于 12 天前
enhancementc/dxpythonc/clientc/infrapython:uv
M
mikeldkingcommented
Extend the Python side of the monorepo to a first-class **uv workspace** so we can factor out internal modules (a `phoenix-config` equivalent, shared rate limiting, shared executors) and publish everything through one flow. ## Where we are today A uv workspace already exists, but it only links the packages we *already* ship: ```toml # pyproject.toml:453 [tool.uv.workspace] members = ["packages/*"] exclude = ["packages/phoenix-sqlean"] [tool.uv.sources] arize-phoenix-client = { workspace = true } arize-phoenix-evals = { workspace = true } arize-phoenix-otel = { workspace = true } ``` Every member is a public PyPI distribution with its own `release-please` entry. There is no lane for a module that several packages need but that isn't a product in its own right, so the fallback has been copy-paste. ## The cost of no shared lane Two files are duplicated between `arize-phoenix-client` and `arize-phoenix-evals`, and they have **already drifted**: | Module | client | evals | |---|---|---| | `rate_limiters.py` | 282 lines | 297 lines | | `executors.py` | 670 lines | 655 lines | Concrete drift in `rate_limiters.py`: the evals copy carries a `printif()` tqdm helper the client copy lacks, and the two initialize the token bucket differently (`self.tokens = 1.0` in evals vs `0.0` in client). A rate-limiter fix landed in one package silently does not reach the other. ## Goal - Make the uv workspace the unit of Python development: one lockfile, one resolution, `uv sync` gives every package plus every internal module wired up. - Add internal workspace members that are **not** product surfaces — starting with shared rate limiting / executors, and a Python analogue of `@arizeai/phoenix-config`. - Keep a single publish flow: adding a workspace member should not mean hand-editing `.github/workflows/publish.yaml`. ## Design questions to settle first 1. **How do internal modules reach end users?** uv workspace sources are dev-time only — a published wheel needs a real dependency. Options: - publish the shared package to PyPI under its own name (e.g. `arize-phoenix-common`) and depend on it normally — simplest, but it becomes a public, versioned surface we have to support; - vendor it into each wheel at build time via a hatch build hook — keeps it private, but complicates the build and can produce duplicate module names on `sys.path`. 2. **Versioning.** Does the shared package get its own `release-please` entry and semver, or is it pinned lockstep to the packages that consume it? A shared package that changes on every release forces a release of all consumers. 3. **Namespace.** All packages already publish under the `phoenix.*` namespace (`phoenix.client`, `phoenix.evals`, `phoenix.otel`). Where does shared code live — `phoenix.common`? — without colliding with the root `phoenix` package in `src/`. 4. **Publish flow.** `list-python-packages` in `publish.yaml` derives its matrix from `.release-please-manifest.json` + a PyPI existence check, so any new manifest entry becomes publishable by default. Confirm that's the behavior we want for internal members, or add an explicit opt-out. 5. **Does `packages/phoenix-sqlean` join?** It's currently excluded as a `setup.py`-based C extension with its own build matrix; leaving it out is probably still right. ## Scope - [ ] Decide distribution strategy for internal modules (question 1) and write it down - [ ] Create the first shared internal package and move `rate_limiters.py` / `executors.py` into it, reconciling the drift between the client and evals copies - [ ] Update `arize-phoenix-client` and `arize-phoenix-evals` to consume it, keeping their public re-exports intact so downstream imports don't break - [ ] Add a Python `phoenix-config` equivalent as the second consumer, mirroring `js/packages/phoenix-config` - [ ] Verify the publish flow handles the new member without workflow edits, or extend it deliberately - [ ] Document the workspace layout and how to add a new member
0 条评论