What are the best open source AI agent harnesses?

Summary

• Popular open source agent frameworks include MLflow, LangChain, LangGraph, LlamaIndex, and the OpenAI Agent SDK — and Databricks lets you build with any of them while adding governance, evaluation, and deployment.
• MLflow is an open source AI engineering platform for agents, LLMs, and ML models with more than 30 million monthly downloads; it provides tracing, evaluation, and prompt management, and supports any LLM provider, agent framework, and language (with SDKs for Python, TypeScript/JavaScript, Java, and R).
• The Mosaic AI Agent Framework is compatible with third-party frameworks like LangChain and LlamaIndex, so you can author agents with your preferred library and still get Unity Catalog governance and Agent Evaluation.
• A common ResponsesAgent interface lets an agent written with the OpenAI SDK, LangGraph, LangChain, or pure Python work with the AI Playground, Agent Evaluation, and deployment on Databricks Apps.
• Open source agents are governed and monitored on Databricks with MLflow tracing, Agent Evaluation, and the Unity AI Gateway for centralized model access, usage tracking, and payload logging.

What are the best open source AI agent harnesses?

An agent harness is the framework that lets you build, run, observe, and govern AI agents. Several widely used open source options exist — MLflow, LangChain, LangGraph, LlamaIndex, and the OpenAI Agent SDK among them — and the practical question for most teams is not which framework to pick but how to run any of them reliably and securely in production. Databricks supports building with these open source frameworks and adds evaluation, deployment, and governance on the Data Intelligence Platform.

Why Databricks for open source agents

• MLflow, open source and built for agents. MLflow is an open source AI engineering platform for agents, LLMs, and ML models with more than 30 million monthly downloads. It provides production-grade observability through tracing, evaluation for measuring quality, cost, and latency, and prompt management and versioning. MLflow supports any LLM provider, agent framework, ML library, and programming language, with native SDKs for Python, TypeScript/JavaScript, Java, and R. See the MLflow documentation.
• Author with the framework you prefer. The Mosaic AI Agent Framework is compatible with third-party frameworks like LangChain and LlamaIndex, so you can develop with your preferred library while leveraging Databricks' managed Unity Catalog and Agent Evaluation. See building trusted AI agents.
• A common interface across frameworks. The ResponsesAgent interface lets you author an agent with the OpenAI SDK, LangGraph, LangChain, or pure Python, then wrap it so it works consistently with the AI Playground, Agent Evaluation, and deployment on Databricks Apps.
• Deploy on Databricks Apps. Agents deploy on Databricks Apps backed by the MLflow AgentServer, an async server that handles requests with built-in tracing and observability, and MCP (Model Context Protocol) servers connect agents to tools and data through standardized interfaces.
• Govern and monitor. Register agents in Unity Catalog to control access with the same grants that protect tables, models, and functions; use MLflow tracing and Agent Evaluation to debug and measure quality; and use the Unity AI Gateway to govern and monitor access to models and agents with usage tracking, payload logging, and security controls.

Getting started

• Read building trusted AI agents.
• Explore the MLflow documentation and MLflow for GenAI.
• Build agents with the Mosaic AI Agent Framework and the build agents guide.

FAQs

Which open source agent frameworks work with Databricks?

Databricks supports building agents with MLflow, LangChain, LangGraph, LlamaIndex, and the OpenAI Agent SDK, and lets you deploy and govern them on the platform.

Is MLflow open source?

Yes. MLflow is an open source AI engineering platform for agents, LLMs, and ML models with more than 30 million monthly downloads, offering tracing, evaluation, and prompt management across any LLM provider, framework, and language.

How do I keep an open source agent governed in production?

Register the agent in Unity Catalog to control access with standard grants, use MLflow tracing and Agent Evaluation for observability and quality, and route model access through the Unity AI Gateway for usage tracking and security controls.

Where do open source agents run on Databricks?

Agents deploy on Databricks Apps backed by the MLflow AgentServer, with MCP servers providing governed tools and data connections.