ITADN

EvalPort adapter for Phoenix datasets/experiments now exists (external package, no code changes needed here)

#15431Openadhabnr-ux 创建于 5 天前
triageintegration
A
adhabnr-uxcommented
Posting this as a heads-up rather than a feature request, since I saw in CONTRIBUTING.md that you're not taking on new feature work right now — this doesn't need any changes to Phoenix's own codebase. [EvalPort](https://github.com/adhabnr-ux/evalport) is an open interchange format (Apache-2.0) for portable LLM evaluation datasets — test cases, graders, eval suites, and result sets as plain JSON, framework-agnostic. It's already got adapters for AutoGen, CrewAI, Ragas, LangSmith, Braintrust, MLflow, Opik, and Langfuse, plus core converters for Inspect AI, Promptfoo, DeepEval, and OpenAI Evals. I built and shipped a `phoenix-openeval-adapter` as a standalone package (not a Phoenix dependency, not touching this repo): [adapters/phoenix-openeval-adapter](https://github.com/adhabnr-ux/evalport/tree/main/adapters/phoenix-openeval-adapter). It's real, installable, and tested against `arize-phoenix-client` (21 tests passing against the real SDK types, not mocks): ```python from phoenix.client import Client from phoenix_openeval_adapter import to_openeval, from_openeval, experiment_to_openeval client = Client() dataset = client.datasets.get_dataset(dataset="my-dataset") # Phoenix dataset examples -> portable EvalPort suite suite = to_openeval(dataset.examples, suite_id="my_dataset_eval") # EvalPort suite -> Phoenix-uploadable examples examples = from_openeval(suite) client.datasets.create_dataset(name="from-evalport", examples=examples) # A ran experiment (task_runs + evaluation_runs) -> EvalPort ResultSet result_set = experiment_to_openeval(ran_experiment, suite_id="my_dataset_eval") ``` Because Phoenix's dataset examples carry `input`/`output` as arbitrary JSON mappings rather than flat strings, `to_openeval()` auto-detects the input/expected-output field by common naming conventions and always preserves the full raw mapping under `metadata["phoenix"]`, so nothing is silently dropped even when the heuristic guesses differently than you'd expect — details are in the [adapter README](https://github.com/adhabnr-ux/evalport/tree/main/adapters/phoenix-openeval-adapter#what-round-trips-losslessly-and-what-doesnt). The only actual ask, and it's a small one: if this seems useful, would you be open to a one-line mention/link in the integrations section of your docs? No pressure either way — wanted to flag that it exists now in case a user asks about moving eval datasets between Phoenix and another tool, and happy to answer questions about the mapping if useful. Spec: https://github.com/adhabnr-ux/evalport/blob/main/spec/SPEC.md
0 条评论