Skip to main content

tools for evaluating llm apps

Summary

  • Evaluating an LLM app takes three things: a dataset of inputs (and, where you have them, expected outputs), scorers that grade each response, and a harness that runs your app over the dataset so you can compare versions.
  • MLflow's open-source GenAI evaluation provides all three through mlflow.genai.evaluate(), with built-in LLM judges such as Correctness, RelevanceToQuery, Safety and RetrievalGroundedness, natural-language Guidelines judges, and custom @scorer functions.
  • Combine LLM judges for qualities that need judgment (relevance, grounding, tone) with code-based scorers for deterministic checks (format, length, exact match).
  • On Databricks, evaluation datasets live in Unity Catalog tables, you can score existing traces from the UI with no code, and domain experts label traces in the Review App to align LLM judges with their standards.

tools for evaluating llm apps

The tools for evaluating LLM apps are an evaluation dataset, a set of scorers, and a harness that runs your app against both and records the results. Spot-checking a handful of chat transcripts tells you very little, because a change that fixes one answer can quietly break others. MLflow's open-source GenAI evaluation brings the three pieces into one API, mlflow.genai.evaluate(), and stores every result as a trace you can open and inspect.

What does LLM app evaluation involve?

MLflow defines each evaluation by three components: a dataset of inputs and expectations, scorers that encode the evaluation criteria, and a predict function that generates outputs for the dataset. A scorer receives a trace, extracts the fields it needs, runs its assessment, and returns the result as feedback attached to that trace. Because results live on traces, you can see not only that a response failed, but which retrieval or model call caused it.

Types of scorers to evaluate

Scorer type Use it for What it needs
Built-in judges RelevanceToQuery and Safety Whether the response addresses the request and avoids harmful content Inputs and outputs
Built-in judges RetrievalRelevance and RetrievalGroundedness Retrieval quality and hallucination in RAG apps A trace with retrieval spans
Built-in judges Correctness and RetrievalSufficiency Matching or covering a known correct answer Expected answers or facts
Guidelines judges Pass/fail checks against natural-language rules, such as style The rules you write
Custom judges with make_judge() Domain-specific criteria and your own grade types Instructions with template variables such as {{ inputs }} and {{ outputs }}
Code-based scorers with @scorer Deterministic checks: exact match, format validation, length A Python function
Third-party scorers Specialized metrics from open-source evaluation frameworks That framework's package

Start with built-in judges for a quick baseline. Add Guidelines judges for rules specific to your product, then code-based scorers for anything a regular expression or a parser can decide. LLM judges call a model for every assessment, so use a code-based scorer wherever a simple rule is enough.

Running an evaluation with MLflow

import mlflow
from mlflow.genai.scorers import Correctness, Guidelines, scorer

dataset = [
    {"inputs": {"question": "Can MLflow manage prompts?"},
     "expectations": {"expected_response": "Yes!"}},
]

@scorer
def short_enough(outputs) -> bool:
    return len(outputs.split()) < 120

def predict_fn(question: str) -> str:
    return my_app(question)  # your LLM app

results = mlflow.genai.evaluate(
    data=dataset,
    predict_fn=predict_fn,
    scorers=[Correctness(), Guidelines(name="is_english", guidelines="The answer must be in English"), short_enough],
)

Run the same dataset and scorers against each new version of your prompt or app, then open both evaluation runs in the MLflow UI to see which test cases changed. mlflow.genai.evaluate() creates one trace per dataset row, annotated with each judge's feedback and rationale.

How Databricks approaches LLM app evaluation

MLflow on Databricks runs the same evaluation API, with a few managed pieces around it:

  • Evaluation datasets in Unity Catalog. mlflow.genai.datasets.create_dataset() creates a dataset stored as a Unity Catalog table, named in catalog.schema.table_name form. You can pull real traces with mlflow.search_traces() and add them with merge_records(), so test cases come from actual usage.
  • No-code scoring. In an experiment's Traces tab, select traces, choose Evaluate, and run pre-built judges or your registered scorers. Results attach to each trace as feedback and show up under Evaluation runs.
  • Hosted judge models. By default, each judge uses a Databricks-hosted LLM designed for GenAI quality assessment. You can change it with the model argument in <provider>:/<model-name> format.
  • Domain experts in the loop. The Review App lets subject-matter experts label traces and define what a good response looks like. Judge alignment then uses that feedback: a judge's align() method produces a new judge tuned to your experts' assessments.

See Evaluate and improve and Scorers and LLM judges.

FAQs

Is MLflow GenAI evaluation open source?

Yes. mlflow.genai.evaluate(), the built-in judges and custom scorers are part of open-source MLflow, which is Apache 2.0 licensed. Databricks runs the same MLflow as a managed service.

Do I need ground-truth answers to evaluate an LLM app?

No. Judges such as RelevanceToQuery, Safety and RetrievalGroundedness work without ground truth. Correctness and RetrievalSufficiency need expected answers or expected facts.

Can I use metrics from other open-source evaluation libraries?

Yes. MLflow wraps metrics from several open-source evaluation frameworks as scorers, and you can combine them with built-in judges in a single mlflow.genai.evaluate() call.

How do I compare two versions of my app?

Evaluate both versions against the same dataset with the same scorers. Each run is recorded separately, so you can compare aggregate results and open individual traces where one version passed and the other failed.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.