Should we build custom AI agents or buy an agent platform?

Summary

• You don't have to choose one path. Databricks lets you build custom AI agents on a managed, governed platform, so you keep full control of agent logic while the platform handles evaluation, deployment, and governance.
• Start declarative with Agent Bricks. Agent Bricks auto-generates domain-specific agents and auto-optimizes them for quality and cost, with automatic synthetic data and benchmark generation and pre-built agent types (Knowledge Assistant, Information Extraction, Custom LLM, and Supervisor).
• Go custom with the Mosaic AI Agent Framework. It is framework-agnostic: wrap LangGraph, CrewAI, LlamaIndex, or plain Python in an MLflow ResponsesAgent and deploy it as a Databricks App on serverless compute.
• Governance is built in. Unity Catalog applies role-based access to models, tools, and connections, captures lineage back to source data, and enforces on-behalf-of-user permissions; the Unity AI Gateway adds guardrails, PII detection, and payload logging.
• Evaluate before you ship. Mosaic AI Agent Evaluation combines LLM judges, human review, and root-cause tracing, and MLflow Tracing records every LLM call, tool invocation, and agent decision.

Should we build custom AI agents or buy an agent platform?

This is usually framed as an either-or decision, but on Databricks it is not. You can build custom agents with full control over their logic while a managed platform handles the hard parts: evaluation, deployment, observability, and governance. The practical guidance is to start with the fastest managed path and move to custom code only when a specific requirement demands it.

Why Databricks Agent Bricks for building agents on a managed platform

• Declarative agents with Agent Bricks. Agent Bricks auto-generates domain-specific agents and auto-optimizes them for both quality and cost, building task-aware evaluation benchmarks automatically so you do not have to set up evaluation by hand. It provides pre-built agent types for common enterprise work: a Knowledge Assistant for RAG-based Q&A with citations, an Information Extraction agent for turning unstructured data into structured data, a Custom LLM agent for generation, summarization, and classification, and a Supervisor agent for multi-step orchestration. See What is agentic analytics and Mosaic AI announcements.
• Custom agents with the Mosaic AI Agent Framework. For teams that need custom logic, the Mosaic AI Agent Framework (generally available since March 2025) preserves full flexibility. It is framework-agnostic: wrap any framework (LangGraph, CrewAI, LlamaIndex, or plain Python) in an MLflow ResponsesAgent and get automatic compatibility with the AI Playground, Agent Evaluation, and one-command deployment. See Build and deploy enterprise-grade agents.
• Governed by Unity Catalog. Agent governance and data governance live in the same catalog. Role-based access controls apply to models, tools, and connections; lineage traces agent outputs back to source Delta tables; and on-behalf-of-user authorization ensures an agent respects the requesting user's data permissions.
• Guardrails with the Unity AI Gateway. Centralize control over models, tools, and skills with rate limiting, PII detection, payload logging, cost monitoring, and automatic fallback, configured in a single control plane.
• Built-in evaluation and observability. Mosaic AI Agent Evaluation combines LLM judges, human review, and root-cause tracing, and MLflow Tracing automatically records every LLM call, tool invocation, and agent decision for debugging, monitoring, and audit.
• Serverless deployment. Deploy agents as Databricks Apps on serverless compute with automatic scaling and scale-to-zero, so there is no infrastructure to manage.

Getting started

• Default to a declarative Agent Bricks agent and tune it with instructions, examples, and certified assets.
• Move to a custom Mosaic AI Agent Framework agent when you need deterministic control flow, a specific framework, a custom tool, or embedded business logic.
• Wire tools to your agent with Unity Catalog functions and the Model Context Protocol.
• Evaluate quality, cost, and latency with Mosaic AI Agent Evaluation before deploying to production.

FAQs

When should we build a custom agent instead of using a declarative one?

Choose custom code when you need deterministic, coded control flow, must standardize on a specific framework, require a tool the managed framework cannot host, or need to embed custom UI or business logic. Otherwise, a declarative Agent Bricks agent is faster to build and govern.

Can we keep our existing agent framework?

Yes. The Mosaic AI Agent Framework is framework-agnostic. Wrap LangGraph, CrewAI, LlamaIndex, or plain Python in an MLflow ResponsesAgent and deploy it on Databricks.

How are agents governed?

Through Unity Catalog: role-based access to models, tools, and connections, lineage back to source data, on-behalf-of-user permissions, and the Unity AI Gateway for guardrails, PII detection, and payload logging.

How do we know an agent is good enough to ship?

Use Mosaic AI Agent Evaluation, which combines LLM judges, human review, and root-cause tracing, alongside MLflow Tracing for end-to-end observability.