What metrics should I use to evaluate new AI models?

Summary

• Evaluate a new model on your own workload, not on general benchmarks. Use the same evaluation dataset, scorers and prompts you use for the model it would replace, so the comparison is fair.
• Quality metrics come first: correctness against ground truth, groundedness in retrieved context, relevance, safety and adherence to your guidelines. For agents, add tool-call correctness and efficiency.
• Operational metrics decide whether a model is usable: latency percentiles, time to first byte, error and fallback rates, and tokens and cost per request on real traffic.
• On Databricks, MLflow runs built-in and custom LLM judges over evaluation datasets and compares runs. Unity Gateway traffic splitting lets you A/B test a candidate model on live traffic and measure it in the usage table.

What metrics should I use to evaluate new AI models?

Use two sets of metrics. Quality metrics measure whether the model gets your task right: correctness against known answers, groundedness in the context you supply, relevance, safety, and adherence to your own rules. For agents, add whether the model picks the right tools with the right arguments. Operational metrics measure whether it's usable in production: tail latency, time to first byte, error rate, and tokens and cost per request. A new model earns a switch only when it holds quality on your own evaluation set, with the same scorers as the current model, at latency and cost you can accept. A strong general benchmark doesn't tell you that.

Quality and operational metrics to compare

Metric	What it tells you	How to measure it
Correctness	Does the answer match the expected facts?	LLM judge against ground-truth expectations
Groundedness	Is the answer supported by retrieved context, or hallucinated?	Judge over the retrieved documents
Relevance	Does it address what the user asked?	Judge over input and output
Safety	Is it free of harmful or toxic content?	Safety judge on every example
Guideline adherence	Does it follow your tone, format and policy rules?	Natural-language guideline judges
Format validity	Is the JSON or schema output parseable?	Code-based scorer
Tool-call correctness	Right tools, right arguments?	Judge with expected tool calls
Tool-call efficiency	Any redundant calls?	Judge over the trace
Latency and time to first byte	Is it responsive, including the tail?	Percentiles on live or replayed traffic
Error and fallback rate	How often does the provider fail?	Status codes and routing logs
Tokens and cost per request	What does a good answer cost?	Token counts, including reasoning tokens

Two rules keep the comparison honest. Freeze the dataset and scorers before you test, and judge on examples that reflect real traffic, including the failures you already know about.

How Databricks approaches model evaluation

Build the evaluation set from real usage. MLflow evaluation datasets can be built from traces of a deployed app, stored in Unity Catalog. That gives you inputs your users actually send, plus the expectations your reviewers attach.
Score with judges you can trust. MLflow built-in judges include Correctness and RetrievalSufficiency (both need ground truth), RetrievalGroundedness, RelevanceToQuery, Safety and Guidelines. For agents, ToolCallCorrectness and ToolCallEfficiency cover tool use. Multi-turn judges such as ConversationCompleteness and UserFrustration score whole conversations. Custom LLM judges handle domain-specific criteria, and code-based scorers handle deterministic checks such as exact match or format validation. Databricks validates its judges against human expert judgment and tracks Cohen's kappa, accuracy and F1. You can align judges with your own reviewers' feedback, and pick the model that powers a judge with the model argument.
Compare versions side by side. Run mlflow.genai.evaluate() with the same dataset and judges for the current model and the candidate, each wrapped as a predict_fn. Select both runs in the MLflow UI and choose Compare, then drill into the specific examples where a metric regressed. Averages hide regressions, so check them per example.
Test on live traffic through Unity Gateway. Put the current and candidate models behind one Unity Gateway model service and use traffic splitting to send each a share of requests, which is the documented way to roll out a new model gradually or run an A/B test. Session affinity keeps a session on one destination. Routing decisions land in the routing_information field of system.ai_gateway.usage, next to latency_ms, time_to_first_byte_ms, status_code and token counts, including output_reasoning_tokens. The built-in usage dashboard's Performance tab shows latency percentiles and error rates per service. After the switch, production monitoring (Beta) can keep running the same scorers on a sample of your agent's live MLflow traces.

A repeatable evaluation workflow

1. Score the baseline. Run the current model through your frozen dataset and judges.
2. Score the candidate. Same dataset, same judges, same prompt version.
3. Review regressions. Read the examples where any judge's verdict got worse, not just the averages.
4. Canary it. Route a small share of traffic to the candidate through a traffic split. Compare latency, errors, tokens and cost by destination.
5. Promote and keep watching. Shift traffic once both quality and operations hold, and keep the scorers running in production.

FAQs

Can I rely on LLM judges to evaluate a new model?

Use them, with checks. Databricks validates its built-in judges against human expert judgment, and you can align a judge to your domain with reviewer feedback. Spot-check a sample of verdicts yourself before you trust a switch decision.

Do I need ground-truth answers?

Only for some metrics. Correctness, RetrievalSufficiency and ToolCallCorrectness need expectations, while RelevanceToQuery, Safety, RetrievalGroundedness and Guidelines don't.

How do I test a new model without changing application code?

Add it as a destination on the existing Unity Gateway model service and split traffic. Callers keep using the same model service, and the usage table records which destination served each request.

Which operational metric matters most?

Tail latency and cost per request for agents, because both compound across every step of a multi-call run. Also watch reasoning tokens, which the usage table reports separately from regular output tokens.