How to build custom agents

Summary

• Databricks Agent Bricks is a platform for building, optimizing, evaluating, and deploying custom AI agents and multi-agent systems.
• Build in your preferred way: declarative builders (Knowledge Assistant for domain Q&A, Supervisor Agent for orchestration), fully custom Python agents using popular frameworks, or the no-code AI Playground for prototyping.
• Evaluate and improve with MLflow Tracing, Agent Evaluation (quality, cost, and latency with LLM judges), custom judges, and review apps for stakeholder feedback.
• Deploy to Model Serving endpoints or Databricks Apps on serverless compute, with built-in agent memory and enterprise data connectivity — no infrastructure to manage.
• Unity Catalog governs agent tools and functions with the same security that protects tables and models, and the Supervisor Agent coordinates multi-agent systems.

How to build custom agents

On Databricks you build custom AI agents with Agent Bricks, following a standardized build → evaluate → iterate → deploy loop. You can use your preferred models, frameworks, and tools while relying on Databricks for managed governance, so you don't have to re-architect code or manage infrastructure to get an agent into production.

Why Databricks Agent Bricks for building custom agents

• Multiple build paths. Use declarative, configuration-driven builders — a Knowledge Assistant for domain-specific Q&A and a Supervisor Agent for orchestrating multiple agents — or write fully custom agents in Python with popular authoring frameworks such as LangGraph or LlamaIndex. The no-code AI Playground lets you prototype and tune prompts and parameters before exporting to code.
• Automatic evaluation and optimization. Agent Bricks applies current agentic research to automatically build evaluations and optimize agent quality, including foundation-model fine-tuning and prompt-optimization techniques.
• Built-in evaluation. MLflow Tracing records and analyzes agent behavior for debugging; Agent Evaluation measures quality, cost, and latency using LLM judges and custom metrics; built-in and custom judges score responses against guidelines; and review apps collect feedback from subject-matter experts.
• Deploy without managing infrastructure. Deploy agents to scalable Model Serving endpoints or as Databricks Apps on serverless compute, with production-grade built-in memory for conversation history and agent state, plus direct, governed connectivity to enterprise data and systems.
• Consistent governance. Unity Catalog governs agent tools and functions with the same security framework that protects your tables and models, and supports on-behalf-of-user access so agents operate with the right permissions.
• Multi-agent orchestration. The Supervisor Agent coordinates multiple agents, Genie Agents, Unity Catalog functions, managed MCP servers, and custom agents for complex, centrally coordinated workflows.

Getting started

• Read Custom agents are now available on Databricks.
• Review the developer documentation: Build and deploy enterprise-grade agents and the agents overview.
• Learn the workflow with new Databricks training for mastering AI agents and connect tools via managed MCP servers.

FAQs

Do I have to use a specific framework to build a custom agent?

No. You can build fully custom agents in Python with your preferred authoring framework, use the declarative Knowledge Assistant and Supervisor Agent builders, or prototype no-code in the AI Playground.

How do I evaluate a custom agent?

Use MLflow Tracing to inspect agent behavior and Agent Evaluation to measure quality, cost, and latency with built-in and custom LLM judges, plus review apps to gather expert feedback.

How do I deploy a custom agent?

Deploy to Model Serving endpoints or as a Databricks App on serverless compute, with built-in memory and enterprise data connectivity, all governed by Unity Catalog.

Can I build a multi-agent system?

Yes. The Supervisor Agent orchestrates multiple agents, Genie Agents, Unity Catalog functions, managed MCP servers, and custom agents into a coordinated system.