---
title: "Your First Traces"
description: "Build a support agent and trace every LLM call, tool execution, and RAG retrieval"
---

import StartPhoenix from "../../../snippets/start-phoenix.mdx";
import PhoenixAuthNote from "../../../snippets/phoenix-auth-note.mdx";

When building agents and LLM applications, it's hard to see what's actually going on under the hood. Even if you set up a comprehensive agent architecture with multiple prompts, descriptive tools, and data retrievals, you're left answering questions like:

- Why did the agent choose that tool instead of this one?
- What context was actually passed to the LLM when it generated that response?
- Where is all the latency coming from - is it the model, the retrieval, or something else?
- The user got a wrong answer, but which step in the pipeline failed?

In this tutorial, with just a few additional lines of code, you'll be able to monitor every LLM call, tool execution, and retrieval operation that powers your agents. You'll learn how to debug, monitor, and analyze your agents more effectively and efficiently, transforming them from personal projects to production-ready applications.

**Follow along with code**

Throughout the tutorial we will include key code snippets, but to see the full implementation, check out the companion projects below.

<Columns cols={2}>
  <Card title="TypeScript Tutorial" icon="js" href="https://github.com/Arize-ai/phoenix/tree/main/js/examples/apps/tracing-tutorial">
    Companion TypeScript project with runnable examples
  </Card>

  <Card title="Python Tutorial" icon="python" href="https://colab.research.google.com/github/Arize-ai/phoenix/blob/main/tutorials/tracing/python_tracing_quickstart.ipynb">
    Companion Python project with runnable examples
  </Card>
</Columns>

# SupportBot

Our sample support agent for this tutorial:

1. **Classifies** incoming queries (order status vs. FAQ)
2. **Routes** to the appropriate handler:
   - **Order Status**: Use a tool to look up order information, then summarize for the customer
   - **FAQ**: Search a knowledge base with embeddings, then generate an answer using RAG

The issue is that our users are complaining. Responses are slow, answers are wrong, but we have no idea why. Our agent is a black box - we can see what the user asked, and see how the agent replied, but we don't have visibility into the individual components of our agent that actually ran.

Let's set up tracing to gain visibility.

# Setting Up Tracing

First, install the dependencies and configure OpenTelemetry to send traces to Phoenix.

## Install Dependencies

<Tabs>
  <Tab title="TypeScript" icon="js">
    ```bash
    npm install ai @ai-sdk/openai @ai-sdk/otel @arizeai/phoenix-otel zod
    ```

    <Note>
    AI SDK v7 requires Node.js 22 or newer. `@arizeai/phoenix-otel` bundles the OpenTelemetry setup and the `@arizeai/openinference-vercel` span processors that translate AI SDK spans for Phoenix; `@ai-sdk/otel` emits those spans.
    </Note>
  </Tab>

  <Tab title="Python" icon="python">
    ```bash
    pip install "arize-phoenix-otel>=0.16.0" arize-phoenix-client openai openinference-instrumentation-openai numpy
    ```

    <Note>
    `arize-phoenix-otel` **0.16.0+** is required to import `SpanAttributes` and the OpenInference context managers directly from `phoenix.otel`. On older versions, import them from `openinference.instrumentation` and `openinference.semconv.trace` instead.
    </Note>
  </Tab>
</Tabs>

## Start Phoenix

In order to send traces to Phoenix, you need a Phoenix instance running.

<StartPhoenix />

Now point your application at it:

<Tabs>
  <Tab title="TypeScript" icon="js">
    Create a `.env` file in your project root:

    ```bash
    PHOENIX_COLLECTOR_ENDPOINT=http://localhost:6006
    OPENAI_API_KEY=<ENTER YOUR OPENAI API KEY>
    ```
  </Tab>

  <Tab title="Python" icon="python">
    ```python
    import os

    os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
    os.environ["OPENAI_API_KEY"] = "<ENTER YOUR OPENAI API KEY>"
    ```
  </Tab>
</Tabs>

<PhoenixAuthNote />

## Configure Tracing

<Tabs>
  <Tab title="TypeScript" icon="js">
    Create an `instrumentation.ts` file:

    ```typescript
    import { OpenTelemetry } from "@ai-sdk/otel";
    import { registerTelemetry } from "ai";
    import { register } from "@arizeai/phoenix-otel";

    // Register with Phoenix - this handles all the OpenTelemetry boilerplate
    export const provider = register({
      projectName: "support-bot",
    });

    registerTelemetry(
      new OpenTelemetry({
        tracer: provider.getTracer("@arizeai/phoenix-otel/ai-sdk"),
        headers: false,
      })
    );
    ```

    Import this file at the top of your application to enable tracing.
  </Tab>

  <Tab title="Python" icon="python">
    Setting `auto_instrument=True` in the `register` function looks at the installed [OpenInference](https://github.com/Arize-ai/openinference) packages and automatically instruments them, so you don't need to do manual configuration for each library.

    ```python
    from phoenix.otel import register

    tracer_provider = register(
        project_name="support-bot",
        auto_instrument=True
    )
    ```
  </Tab>
</Tabs>

# Tracing LLM Calls

Every LLM call is a decision point. What prompt did the model receive? What did it output? How long did it take, and how many tokens did it use?

Without tracing, you're forced to build your own logging or debugging, and therefore miss out on key data that would block you from full observability. With tracing, you get a complete record of every LLM interaction, including

- input messages (system, user, assistant prompt)
- LLM output
- model name, model provider
- invocation parameters
- token counts
- latency

For SupportBot, tracing LLM calls will give us observability into the classification stage. We'll see exactly what class all our support queries are, and what lead to that classification.

It will also give us observability into the final generation stage. How was the final output delivered to the user, and what context went into that final LLM call that generated the final output?

<Tabs>
  <Tab title="TypeScript" icon="js">
    Once the Phoenix provider and AI SDK integration are registered at startup, AI SDK calls are traced without per-call configuration:

    ```typescript
    import { generateText } from "ai";
    import { openai } from "@ai-sdk/openai";

    const result = await generateText({
      model: openai.chat("gpt-4o-mini"),
      instructions: "Classify the query as 'order_status' or 'faq'",
      prompt: userQuery,
    });
    ```
  </Tab>

  <Tab title="Python" icon="python">
    Calls will be automatically traced when instrumentation is enabled:

    ```python
    user_query = "Where is my order?"

    result = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "Classify the query as 'order_status' or 'faq'"},
            {"role": "user", "content": user_query},
        ],
    )
    ```
  </Tab>
</Tabs>

**In Phoenix, you'll see:**

<video controls className="w-full aspect-video rounded-xl" src="https://storage.googleapis.com/arize-phoenix-assets/assets/images/classification-trace.mp4" />

# Tracing Tool Calls

Tools allow your agent to interact with databases, APIs, external systems. In order to gain insight into how your tools are performing, you need to answer questions like

- Did the LLM decide to call the right tool?
- Did it extract the parameters correctly?
- Did the tool return what you expected?

With tracing, you see the complete chain, including the LLM's decision, the exact parameters passed, and the tool's response, without having to guess which step broke.

When your LLM calls tools, those executions are automatically traced as child spans. With Phoenix, you can see the complete chain, including the LLM's decision, the exact parameters passed, and the tool's response.

<Tabs>
  <Tab title="TypeScript" icon="js">
    With the AI SDK, you can simply define your tools using the AI SDK configuration — tool executions are traced automatically:

    ```typescript
    import { generateText, isStepCount, tool } from "ai";

    const result = await generateText({
      model: openai.chat("gpt-4o-mini"),
      prompt: userQuery,
      tools: {
        lookupOrderStatus: tool({
          description: "Look up order status by order ID",
          inputSchema: z.object({
            orderId: z.string(),
          }),
          execute: async ({ orderId }) => {
            // Your tool logic here
            return orderDatabase[orderId];
          },
        }),
      },
      stopWhen: isStepCount(2),
    });
    ```
  </Tab>

  <Tab title="Python" icon="python">
    For the OpenAI Python SDK, you need a helper function to execute tool calls and manually create spans to trace tool execution. 

    ```python
    tools = [
      # tools defined here
    ]

    # Helper function to execute tools
    def execute_tool_call(tool_call, database):
        """Execute a tool call and return the result."""
        function_name = tool_call.function.name
        function_args = json.loads(tool_call.function.arguments)

        with tracer.start_as_current_span(
            function_name,
            attributes={
                SpanAttributes.OPENINFERENCE_SPAN_KIND: "TOOL",
                SpanAttributes.TOOL_NAME: function_name,
                SpanAttributes.TOOL_PARAMETERS: json.dumps(function_args),
                SpanAttributes.INPUT_VALUE: json.dumps(function_args),
            },
        ) as tool_span:
          # Helper function logic here
      

    messages = [
        {
            "role": "system",
            "content": "You are a helpful customer support agent....",
        },
        {"role": "user", "content": user_query},
    ]

    result = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages,
        tools=tools,
        tool_choice="auto",
    )

    message = result.choices[0].message
    messages.append(message)

    # Execute tool if called, then get final response
    if message.tool_calls:
        for tool_call in message.tool_calls:
            tool_result = execute_tool_call(tool_call, ORDER_DATABASE)
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": json.dumps(tool_result),
            })

        # Final LLM call with tool result
        final_result = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=messages,
        )
        final_response = final_result.choices[0].message.content
    else:
        final_response = message.content
    ```
  </Tab>
</Tabs>

**In Phoenix, you'll see:**

<video controls className="w-full aspect-video rounded-xl" src="https://storage.googleapis.com/arize-phoenix-assets/assets/images/tool-call.mp4" />

1. **LLM Span**: Model decides to call `lookupOrderStatus`
2. **Tool Span**: Shows the tool name, input (`orderId`), and output
3. **LLM Span**: Model summarizes the result

# Tracing RAG Pipelines

RAG pipelines can fail in many places. The embedding might not capture the query's intent, the retrieval might return irrelevant documents, or the LLM might misuse good context. When a user gets a bad answer, which step failed? With tracing, you can see the full pipeline, including which documents were retrieved, what context was injected into the prompt, and how the LLM used it. You can pinpoint exactly where things went wrong.

For RAG, trace both the embedding calls and the generation call. Each `embed` call becomes its own span:

<Tabs>
  <Tab title="TypeScript" icon="js">
    ```typescript
    // Embed the user's query - traced automatically
    const { embedding } = await embed({
      model: openai.embedding("text-embedding-ada-002"),
      value: userQuery,
    });

    // ... semantic search logic ...

    // Generate with retrieved context - traced automatically
    const { text } = await generateText({
      model: openai.chat("gpt-4o-mini"),
      instructions: `Answer using ONLY this context:\n\n${retrievedContext}`,
      prompt: userQuery,
    });
    ```
  </Tab>

  <Tab title="Python" icon="python">
    ```python wrap
    # Embed the query - traced automatically
    embedding_response = client.embeddings.create(
        model="text-embedding-ada-002",
        input=user_query
    )
    query_embedding = embedding_response.data[0].embedding

    # Generate with retrieved context - traced automatically
    rag_result = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": f"You are a helpful customer support agent. Answer the user's question using ONLY the information provided in the context below. Be friendly and concise.\n\nContext:\n{rag_context}",
            },
            {"role": "user", "content": user_query},
        ],
    )
    ```
  </Tab>
</Tabs>

**In Phoenix, you'll see:**

<video controls className="w-full aspect-video rounded-xl" src="https://storage.googleapis.com/arize-phoenix-assets/assets/images/rag-initial.mp4" />

The generation span shows the retrieved context in the system prompt, so you can immediately see if retrieval found the right documents.

# Grouping Operations with Parent Spans

A single user request might trigger multiple LLM calls, tool executions, and retrievals. Let's allocate all of these under one parent span, so all operations for one request are nested together. Click on the parent span and see the entire execution tree: classification, tool calls, retrieval, generation, all in one view, with timing relationships visible at a glance.

<Info>
  See the entire agent with grouped tracing [here](https://github.com/Arize-ai/phoenix/tree/main/js/examples/apps/tracing-tutorial/support-agent.ts).
</Info>

To see all operations for a single request as one trace, wrap them in a parent span using the OpenTelemetry API:

<Tabs>
  <Tab title="TypeScript" icon="js">
    ```typescript
    // trace and SpanStatusCode are re-exported by @arizeai/phoenix-otel, so
    // no direct dependency on @opentelemetry/api is needed
    import { trace, SpanStatusCode } from "@arizeai/phoenix-otel";

    const tracer = trace.getTracer("support-agent");

    async function handleSupportQuery(userQuery: string) {
      return tracer.startActiveSpan(
        "support-agent",
        { attributes: { "openinference.span.kind": "AGENT", "input.value": userQuery } },
        async (span) => {
          try {
            // All LLM calls, tool executions, and embeddings inside here
            // will appear as children of this span
            const result = await processQuery(userQuery);
            
            span.setAttribute("output.value", result);
            span.setStatus({ code: SpanStatusCode.OK });
            return result;
          } catch (error) {
            span.setStatus({ code: SpanStatusCode.ERROR });
            throw error;
          } finally {
            span.end();
          }
        }
      );
    }
    ```
  </Tab>

  <Tab title="Python" icon="python">
    ```python
    from opentelemetry import trace
    from phoenix.otel import SpanAttributes

    def handle_support_query(user_query: str) -> AgentResponse:
        tracer = trace.get_tracer("support-agent")

        with tracer.start_as_current_span(
            "support-agent",
            attributes={
                SpanAttributes.OPENINFERENCE_SPAN_KIND: "AGENT",
                SpanAttributes.INPUT_VALUE: user_query,
            },
        ) as agent_span:
            try:
                # All LLM calls, tool executions, and embeddings inside here
                # will appear as children of this span
                
                agent_span.set_attribute(SpanAttributes.OUTPUT_VALUE, result)
                agent_span.set_status(trace.Status(trace.StatusCode.OK))
                return result
            except Exception as error:
                agent_span.set_status(trace.Status(trace.StatusCode.ERROR))
                raise
    ```
  </Tab>
</Tabs>

# Running the Demo

The final SupportBot agent combines the classifier, the order status tool, and the FAQ retrieval into a single agent. 

The tutorial code runs 7 test queries against the agent:

```typescript
const queries = [
  "What's the status of order ORD-12345?",  // Order found
  "How can I get a refund?",                 // FAQ in knowledge base
  "Where is my order ORD-67890?",            // Order found
  "I forgot my password",                    // FAQ in knowledge base
  "What's the status of order ORD-99999?",   // Order NOT found
  "How do I upgrade to premium?",            // FAQ NOT in knowledge base
  "Can you help me with something?",         // Vague request
];
```

<Columns cols={2}>
  <Card title="Complete TypeScript Tutorial" icon="js" href="https://github.com/Arize-ai/phoenix/tree/main/js/examples/apps/tracing-tutorial/support-agent.ts" description="End-to-end TypeScript tutorial">
  </Card>

  <Card title="Complete Python Tutorial" icon="python" href="https://colab.research.google.com/github/Arize-ai/phoenix/blob/main/tutorials/tracing/python_tracing_quickstart.ipynb" description="End-to-end Python tutorial">
  </Card>
</Columns>

The code may prompt for feedback on each response - you can skip this for now (press `s`) and focus on the traces.

# Viewing Your Traces

Open Phoenix at [http://localhost:6006](http://localhost:6006). You'll see 7 `support-agent` traces - one for each query.

<video controls className="w-full aspect-video rounded-xl" src="https://storage.googleapis.com/arize-phoenix-assets/assets/images/final-first.mp4" />

Click into any trace to see the full execution tree. Let's look at two interesting cases:

### Trace 1: "Can you help me with something random?"

Our support query classifier gave the following classification:

```
{
  "category": "faq",
  "confidence": "low",
  "reasoning": "The query is vague and doesn't specify a relevant topic, but it suggests a need for assistance, placing it within a general support context."
}
```

Confidence: low is a huge red flag! This tells us that our support query classifier was unable to confidently classify the user's support query, indicating the query may be out of scope for our agent.

The last span shows the most relevant context retrieved, which is

```
Context:
Q: How do I reset my password?
A: Go to Settings > Security > Reset Password. You'll receive an email with a reset link that expires in 24 hours.

Q: How do I update my profile information?
A: Go to Account Settings > Profile. You can update your name, email, phone number, and address there.
```

This context is not relevant to the user's question at all.

Therefore, our traces have given us proper insight into why the final answer was:

```
I’d be happy to help, but I can only assist with questions related to account settings, passwords, and profile information. Let me know if you need help with those!
```

### Trace 2: "What's the status of order ORD-99999?"

Our support query classifier gave us the following classification:

```
{
  "category": "order_status",
  "confidence": "high",
  "reasoning": "The query directly asks about the status of a specific order, indicating it is related to order tracking."
}
```

Hmm. Seems like our classifier thinks this question accurately falls within the scope of our agent. Let's keep going.

Our support agent LLM span chose the following tool call:

```
lookupOrderStatus("{\"orderId\":\"ORD-99999\"}")
```

Seems good...

The lookOrderStatus tool call gave us:

```
{"error":"Order ORD-99999 not found in our system"}
```

Aha! Seems like ORD-99999 is an invalid order number!

That's why the final output was:

```
Hi there! I checked on your order with the ID ORD-99999, but it seems that I couldn't find any details at the moment. If you could provide me with more information or check back later, I'd be happy to assist you further!
```

# Summary

Congratulations! In this tutorial, you learned how to:

- **Trace LLM calls** - Capture inputs, outputs, tokens, and latency automatically once tracing is registered
- **Trace tool calls** - See tool decisions, parameters, and responses as child spans
- **Trace RAG pipelines** - Monitor embeddings and see retrieved context in generation prompts
- **Group with parent spans** - Nest all operations for a request into one trace
- **View and analyze traces** - Debug agent behavior by exploring execution trees in Phoenix

# Next Steps

You can see inside your application now - every LLM call, tool execution, and retrieval is visible. We spent some time manually analyzing traces. But how can we **automate** this analysis, over thousands of traces? How can we store this analysis in Phoenix, so that we can build **metrics** that measure our application?

In the [next chapter](/docs/phoenix/tracing/tutorial/annotations-and-evaluations), you'll learn to:

- Annotate traces to mark quality issues
- Capture user feedback (thumbs up/down) and attach it to traces
- Run automated LLM-as-Judge evaluations to find patterns in what's failing
