Which models are best for agentic applications?

Summary

• Agentic applications require models strong at tool-calling, function invocation, and reasoning over multiple steps. Claude (Sonnet/Opus) and GPT-4o lead on proprietary side; Llama and Mistral Mixtral offer best open-weight options.
• Task type determines model choice: retrieval-augmented chat, structured extraction, text-to-SQL, and multi-step planning each favor different model strengths.
• Most production agents use a dual-model strategy: a fast, efficient model for planning/filtering, and a capable model for complex reasoning or tool decisions.
• Latency, cost, context window, and compliance constraints all shape which model tier you pick. Databricks Agent Bricks lets you evaluate candidate models on your actual workflows with MLflow tracing.
• Proprietary models excel at tool use and handling edge cases; open-weight models give you lower costs and deployment flexibility if you self-host.

Which models are best for agentic applications?
The best model for an agentic application depends on your task type, latency requirements, and infrastructure constraints. Proprietary models like Claude Sonnet and GPT-4o lead on tool-calling accuracy and reasoning; open-weight models like Llama and Mistral Mixtral excel when cost or self-hosting matters. Most production agents use a tiered approach: a lightweight model for planning and filtering, paired with a high-capability model for complex decisions.

What is an agentic application?

An agentic application is an LLM-powered system that autonomously takes multi-step actions, uses tools or APIs, and handles long-running workflows. Agents make decisions, invoke functions, fetch data, and coordinate across systems. Common examples include support assistants that route tickets, document-processing systems that extract and validate information, text-to-SQL analytics assistants, and workflow automation agents that orchestrate business processes.

Key capabilities to evaluate

When comparing models for agentic use, assess these dimensions:

Capability	Why It Matters
Tool-calling / function-calling support	Agents must reliably invoke external APIs and tools. Models must accurately parse function names, parameters, and handle tool responses without hallucination.
Context window (token length)	Long documents or multi-step workflows require a model that can hold and reason over extended context without losing fidelity.
Instruction-following consistency	Agents need deterministic, repeatable behavior. Models must adhere to structured-output requirements and system prompts without drift.
Chain-of-thought reasoning	Multi-step tasks benefit from models that can plan internally and correct course if a tool call fails.
Latency and throughput	Real-time agent UIs need sub-second responses; background workflows can tolerate higher latency but must scale to concurrent requests.
Safety and structured outputs	Production agents require models that can enforce schema validation, prevent injection attacks, and audit tool usage.

How Databricks approaches agentic applications

Databricks Agent Bricks is the platform for building, evaluating, and deploying production agents. It integrates model selection, governance, and evaluation into a single lifecycle:
Model flexibility. Agent Bricks works with foundational models via the Databricks Foundation Model API, including both hosted proprietary models and open-weight models deployed on serverless compute. This lets you prototype with a fast model, then graduate to a more capable one without refactoring your agent code.
Evaluation and iteration. Use MLflow Tracing to record every agent step, from tool invocation to model output. This creates a feedback loop: capture baseline metrics on a representative test set, try new models or prompt variants, and measure the lift on correctness, latency, and cost. The Agent Bricks platform automates this compare-and-select workflow.
Structured outputs and validation. Define the schema your agent should produce (e.g., extracted fields from a PDF, SQL query correctness). Agent Bricks validates model outputs and can enforce constraints, reducing hallucination and runtime errors.
Tiered model strategies. Deploy a cascade of models: a lightweight model handles most queries and routes only high-uncertainty or high-risk cases to a larger, more capable model. This keeps costs down while maintaining quality on edge cases.
Governance and auditability. All agent actions, tool invocations, and model outputs are logged to Unity Catalog. Access to models and agent operations is governed by Databricks' role-based access control, meeting enterprise compliance requirements.

Common use cases

Retrieval-augmented QA. Support agents over knowledge bases need good instruction-following and context. Claude Sonnet or GPT-4o excel; Llama 3.x or Mixtral work with careful tuning.
Document extraction. Extracting structured data from PDFs requires multimodal support and schema validation. Pair vision models for layout understanding with extractors for structured output, validated in Agent Bricks.
Text-to-SQL analytics. Natural-language to SQL queries benefit from code-tuned models paired with Genie text-to-SQL patterns. Validate generated SQL before execution.
Multi-step workflows. Agents orchestrating business processes need reliable planning and tool-calling. Pair capable models (Claude Opus, GPT-4o) for orchestration with smaller models for subtasks.

FAQs

What is the difference between a planner and an actor model in agentic applications?

In a two-model strategy, the planner decides which tools to call. The actor executes complex interactions. This split reduces costs and improves reliability.

How do I choose between a proprietary and open-weight model for my agent?

Choose proprietary (Claude, GPT) if you prioritize tool-calling accuracy, minimal prompt engineering, and don't need to self-host. Choose open-weight (Llama, Mixtral) if you need lower inference costs, control over model updates, or can't send data to third-party APIs. Many teams run a hybrid strategy.

Can I switch models without rewriting my agent code?

Yes, if you use Databricks Agent Bricks. The framework abstracts model calls, so you can swap a Llama model for Claude without changing your agent logic. Use MLflow to track how each model performs on your test set, then iterate.

What context window do agentic applications typically need?

It depends on your task. A simple RAG agent over short documents might need 4K tokens. Long-document processing or multi-step workflows with conversation history benefit from 16K–32K or longer context windows. If your agent must handle longer context, use retrieval chunking plus the model's full window, or choose a model with extended context support.

How do I know if my model is hallucinating in tool calls?

Instrument your agent with MLflow Tracing and audit tool invocations: log which function the model chose, what parameters it passed, and whether the function succeeded. Create a test set with edge cases and measure failure rates. Validate structured outputs in Agent Bricks against your schema to catch invalid tool calls before execution.