best llm trace dashboards
Summary
- A good LLM trace dashboard does two jobs: it charts aggregate health (requests, latency, errors, token usage, quality) and it lets you click through to the individual trace and span tree behind any spike.
- In open source, MLflow's Overview tab (available since MLflow 3.9.0) gives every experiment Usage, Quality and Tool Calls views over all logged traces.
- MLflow's quality charts are built from the assessments on your traces. Logging traces without scorers or human feedback gives you operational charts only.
- The MLflow dashboard needs a SQL-backed tracking store (SQLite, PostgreSQL or MySQL). The default file-based store won't render the charts.
- On Databricks, traces stored in Unity Catalog are OpenTelemetry-format Delta tables, so you can build AI/BI Dashboards on them with SQL next to the managed MLflow UI.
best llm trace dashboards
The best LLM trace dashboard ties aggregate charts to the traces underneath them. A latency spike or a dip in quality scores is only useful if one click shows the exact requests, spans and tool calls that caused it. In open source, MLflow's experiment Overview tab does this out of the box, with Usage, Quality and Tool Calls views over every trace in the experiment. When you need dashboards that join trace data with business data, store traces as tables and chart them with SQL. That's the approach Databricks takes with traces in Unity Catalog.
What is an LLM trace dashboard?
A trace is the record of one request through an LLM application. It's made of spans, one for each step: a model call, a retrieval, a tool call. MLflow Tracing "captures the inputs, outputs, and metadata associated with each intermediate step of a request", including latency and token usage at each step. A trace dashboard aggregates those records over time so you can see volume, speed, failures, spend and quality at a glance, then drill into a single trace when something looks wrong.
What to look for in a trace dashboard
| Criterion | Why it matters |
|---|---|
| Requests, latency and error charts | The first signal that a deploy or upstream model changed behavior |
| Token usage and cost | Spend attribution per app, model or prompt version |
| Quality charts fed by scorers | Operational health alone misses confident wrong answers |
| Tool-call performance | Agents often fail at the tool step, not the model step |
| Drill-down to the span tree | Gets you from a chart to the cause in one click |
| Time range and granularity controls | Separates a one-off blip from a trend |
| Open trace format | OpenTelemetry-compatible traces avoid lock-in to one viewer |
| SQL access to raw traces | Custom dashboards and joins with your own data |
MLflow's open-source trace dashboard
The MLflow Dashboard (Overview) docs describe three sub-tabs:
- Usage: Requests, Latency, Errors, and Token Usage & Token Stats.
- Quality: Quality Summary and Quality Insights, "dynamically generated based on the assessments available in your traces."
- Tool Calls: Statistics Cards, Tool Performance Summary, Tool Usage & Latency, and Tool Error Rate.
Every tab has a time range selector and a time unit selector. The dashboard requires a SQL-backed tracking store, and the MLflow blog dates the Overview tab to MLflow 3.9.0.
Token and cost charts depend on your instrumentation. Per MLflow's token usage and cost docs, token usage tracking needs MLflow 3.2.0 or later, and cost tracking needs 3.10.0 or later with the tracking server started with the [genai] extra.
With a tracking server in place, autologging feeds the dashboard, and mlflow.search_traces() pulls the same data into pandas for custom charts:
import mlflow
mlflow.set_experiment("support-agent")
mlflow.langchain.autolog() # one span per LangChain step
# Later: pull successful traces into a DataFrame for your own analysis
traces = mlflow.search_traces(filter_string="trace.status = 'OK'")
How Databricks approaches trace dashboards
On Databricks you get the same open-source MLflow, fully managed: the tracing overview says there's no tracking server to run and no version to maintain yourself.
The trace UI. In an experiment's Traces tab you can describe the traces you want in plain English with Filter with AI, open a trace to see its span tree with each step's inputs, outputs, timing and errors, and click Assess to add a rating or an expected answer (Observe and find issues).
Traces as tables. When you store traces in Unity Catalog, setup creates <table_prefix>_otel_spans, _otel_logs, _otel_metrics and _otel_annotations Delta tables in OpenTelemetry format. You query them through a SQL warehouse, build AI/BI Dashboards on them, or process them with Spark. Unity Catalog schema and table permissions control who sees what. The Databricks tracing blog describes native experiment dashboards for trace volume, errors, latency, token usage and cost, with custom AI/BI Dashboards on the trace tables for everything else.
Natural-language analysis. The tracing overview also points to Genie Code for asking questions of trace data without writing SQL.
Model-level dashboards. For spend across all apps, Unity Gateway offers a built-in Usage Dashboard that account admins create from the Govern menu, with a Cost Analysis page for Databricks cost and external model spend.
FAQs
Why is my MLflow Overview tab empty?
Check that traces are logged to the selected experiment and that you're on a SQL-backed tracking store. The file-based store won't render the charts.
Why are the quality charts blank when usage charts work?
Quality charts come from assessments on traces. Run scorers in an evaluation, or add human feedback, to populate them.
Can I build my own trace dashboard?
Yes. On Databricks, chart the Unity Catalog trace tables with AI/BI Dashboards. In open-source MLflow, mlflow.search_traces() returns a pandas DataFrame you can plot.
Do trace dashboards work for agents outside Databricks?
Yes. MLflow Tracing is fully compatible with OpenTelemetry, and Databricks documents trace ingestion that doesn't depend on where the agent runs.
The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.