How do I choose an agent harness for production AI agents?

Summary

• The agent loop is the easy part — a production harness is what surrounds it: evaluation, observability, governance, memory, and managed deployment. Databricks Agent Bricks provides all of these so agents run reliably in production, not just in a demo.
• Framework-agnostic by design. Build with the framework you already know — LangGraph, the OpenAI Agents SDK, LangChain, LlamaIndex, CrewAI, AutoGen, or pure Python — and wrap it in the MLflow ResponsesAgent interface for portability across the platform.
• Managed, serverless deployment. Custom Agents run as fully managed Databricks Apps on serverless compute with scale-to-zero — no infrastructure to provision or operate.
• Built-in evaluation and tracing. MLflow records every LLM call, tool invocation, and agent decision, and Agent Evaluation runs built-in and custom LLM judges against evaluation datasets in a build → evaluate → iterate loop.
• Governed and secure. Agents inherit user identity through on-behalf-of authentication, are governed by Unity Catalog, and carry end-to-end audit trails, with production-grade memory powered by Lakebase.

How do I choose an agent harness for production AI agents?

Writing the basic agent loop — a model that reasons and calls tools — is the easy part. Choosing an agent harness means choosing everything that surrounds that loop and makes it safe to run against real business data: how you evaluate quality before shipping, how you trace and debug what the agent did, how access is governed, where state and memory live, and how the agent is deployed and scaled. Databricks Agent Bricks is the enterprise platform that provides these layers so you can move an agent from prototype to production.
The first decision is managed vs. custom. Default to managed Agent Bricks components — Knowledge Assistant for document Q&A, Genie for text-to-SQL over structured data, and the Supervisor Agent for multi-agent orchestration — because they handle evaluation, optimization, and governance for you. Move to a Custom Agent when you need to standardize on a specific framework, require deterministic coded control flow rather than LLM routing, need persistent memory across steps and sessions, or are embedding the agent in a larger app with custom UI and business logic.

Why Databricks Agent Bricks for production agent harnesses

A production harness needs more than a framework. Agent Bricks brings the surrounding layers together:

• Framework-agnostic. Build agents with your preferred framework — LangGraph, the OpenAI Agents SDK, LangChain, LlamaIndex, CrewAI, AutoGen, or pure Python — and wrap them in the MLflow ResponsesAgent interface. That single interface gives you automatic compatibility with the AI Playground, Agent Evaluation, and deployment, so you are not locked into one authoring style.
• Observability built in. MLflow provides automatic, end-to-end tracing that records every LLM call, tool invocation, and agent decision, with a standard endpoint for querying deployed agents. This is the difference between debugging an agent and guessing.
• Evaluation as a first-class step. Agent Evaluation lets you build evaluation datasets with expectations and score agents with built-in and custom LLM judges across quality, cost, and latency — the build → evaluate → iterate loop that separates a shipped agent from a proof of concept.
• Governance through Unity Catalog. Agents inherit user identity through on-behalf-of authentication, so they can only access what the requesting user is authorized to use, and every run carries access controls and audit trails governed in one place with your data and models.
• Managed serverless deployment. Custom Agents deploy as fully managed Databricks Apps on serverless compute with auto-scaling and scale-to-zero — no infrastructure to manage.
• Production-grade memory. State and conversation history are powered by Lakebase, keeping agents context-aware across steps and sessions.

Getting started

• Prototype in the AI Playground, then export to production code and develop locally with full IDE support.
• Wrap your agent with the MLflow ResponsesAgent interface to make it portable across the AI Playground, evaluation, and deployment. Follow the agent quickstart.
• Evaluate before you ship using Agent Evaluation datasets and LLM judges, iterating on quality, cost, and latency.
• Deploy as a Databricks App on serverless compute, then govern and monitor it through Unity Catalog and MLflow. See Build and deploy enterprise-grade agents and the Agent Bricks documentation.

FAQs

Which agent frameworks does Databricks support?

You can build with any framework — including LangGraph, the OpenAI Agents SDK, LangChain, LlamaIndex, CrewAI, AutoGen, and pure Python — by wrapping your agent in the MLflow ResponsesAgent interface, which provides compatibility with the AI Playground, evaluation, and deployment.

Should I use a managed Agent Bricks component or a custom agent?

Default to managed components (Knowledge Assistant, Genie, Supervisor Agent) because they handle evaluation, optimization, and governance for you. Choose a Custom Agent when you need a specific framework, deterministic coded control flow, persistent memory across sessions, or a custom application UI and business logic.

What makes an agent production-ready on Databricks?

Production agents pass an evaluation loop with LLM judges and evaluation datasets, carry automatic MLflow tracing of every call and decision, inherit user identity and access controls through Unity Catalog governance, and deploy on serverless compute as managed Databricks Apps.

How are agents deployed and scaled?

Custom Agents deploy as fully managed Databricks Apps on serverless compute with auto-scaling and scale-to-zero, so there is no infrastructure to provision or operate.

Where do agents store memory and state?

Production-grade memory is powered by Lakebase, which stores agent state and conversation history so agents stay context-aware across steps and sessions.