EvalPort adapter for Phoenix datasets/experiments now exists (external package, no code changes needed here)
triageintegration
Posting this as a heads-up rather than a feature request, since I saw in CONTRIBUTING.md that you're not taking on new feature work right now — this doesn't need any changes to Phoenix's own codebase.
[EvalPort](https://github.com/adhabnr-ux/evalport) is an open interchange format (Apache-2.0) for portable LLM evaluation datasets — test cases, graders, eval suites, and result sets as plain JSON, framework-agnostic. It's already got adapters for AutoGen, CrewAI, Ragas, LangSmith, Braintrust, MLflow, Opik, and Langfuse, plus core converters for Inspect AI, Promptfoo, DeepEval, and OpenAI Evals.
I built and shipped a `phoenix-openeval-adapter` as a standalone package (not a Phoenix dependency, not touching this repo): [adapters/phoenix-openeval-adapter](https://github.com/adhabnr-ux/evalport/tree/main/adapters/phoenix-openeval-adapter). It's real, installable, and tested against `arize-phoenix-client` (21 tests passing against the real SDK types, not mocks):
```python
from phoenix.client import Client
from phoenix_openeval_adapter import to_openeval, from_openeval, experiment_to_openeval
client = Client()
dataset = client.datasets.get_dataset(dataset="my-dataset")
# Phoenix dataset examples -> portable EvalPort suite
suite = to_openeval(dataset.examples, suite_id="my_dataset_eval")
# EvalPort suite -> Phoenix-uploadable examples
examples = from_openeval(suite)
client.datasets.create_dataset(name="from-evalport", examples=examples)
# A ran experiment (task_runs + evaluation_runs) -> EvalPort ResultSet
result_set = experiment_to_openeval(ran_experiment, suite_id="my_dataset_eval")
```
Because Phoenix's dataset examples carry `input`/`output` as arbitrary JSON mappings rather than flat strings, `to_openeval()` auto-detects the input/expected-output field by common naming conventions and always preserves the full raw mapping under `metadata["phoenix"]`, so nothing is silently dropped even when the heuristic guesses differently than you'd expect — details are in the [adapter README](https://github.com/adhabnr-ux/evalport/tree/main/adapters/phoenix-openeval-adapter#what-round-trips-losslessly-and-what-doesnt).
The only actual ask, and it's a small one: if this seems useful, would you be open to a one-line mention/link in the integrations section of your docs? No pressure either way — wanted to flag that it exists now in case a user asks about moving eval datasets between Phoenix and another tool, and happy to answer questions about the mapping if useful.
Spec: https://github.com/adhabnr-ux/evalport/blob/main/spec/SPEC.md
0 条评论