When should I use frontier models versus smaller AI models?
Summary
- Use a frontier model when the task is open-ended, multi-step or high-stakes: complex reasoning, long-horizon agentic work, deep design exploration, nuanced judgment. Use a smaller model when the task is well-scoped and repeatable, runs at high volume, or needs fast responses.
- Decide with measurements on your own tasks, not with token prices. Keep the lightest model that clears your quality bar, and remember that a larger model can still cost less per task when it finishes with fewer tokens.
- A common pattern is to use both: a router sends routine work to a smaller model and escalates hard cases to a frontier model.
- On Databricks, Unity Gateway puts Databricks-hosted and external models behind one governed interface, MLflow evaluation compares candidates on your data, and Smart Routing (Beta) picks the lowest-cost capable model for coding-agent tasks.
When should I use frontier models versus smaller AI models?
Use a frontier model when the work needs deep reasoning, long-horizon planning or broad knowledge, and a wrong answer is expensive or hard to spot. Use a smaller model when the task is well-scoped, has a clear right answer, runs at high volume, or sits on a latency-sensitive path. Defaulting everything to the biggest model wastes money, and defaulting everything to the smallest one quietly lowers quality. Databricks' engineering team found that the most intelligent models "are very effective at solving all kinds of problems" but expensive, while "Medium and lower intelligence models are still highly effective at the common tasks." The practical rule: start from the requirement, evaluate candidates on your own data, and keep the lightest model that meets the bar.
What separates a frontier model from a smaller one?
"Frontier" describes the most capable general-purpose models available at a given moment. They're strongest on ambiguity, multi-step reasoning and tool use, and they cost the most per token. Smaller models trade some of that capability for lower latency and price. The line keeps moving: in Databricks' own coding benchmark, open models were "now able to handle even the highest level of task difficulty," and models clustered into rough capability tiers.
Frontier vs. smaller models: how they compare
| Factor | Frontier models | Smaller models |
|---|---|---|
| Best fit | Complex reasoning, nuanced understanding, high-autonomy agents | High-volume, straightforward tasks with tight latency or cost limits |
| Typical work | Design exploration, multi-step research, large refactors, long agent sessions | Task classification, field extraction, routing, routine code edits |
| Price per token | Highest | Lowest |
| Cost per completed task | Can be lower than expected, because strong models may use fewer tokens | Low when the model succeeds first time; rises with retries and repairs |
| Latency | Slower responses, more so with extended reasoning | Faster responses, suited to real-time paths |
| Typical failure | Overspending on routine work | Missing edge cases on ambiguous or multi-step work |
The cost row is the one teams get wrong. Databricks' benchmark concluded that "The token price of a model is a poor indicator of actual costs incurred on end-to-end tasks," and that the harness a model runs in "dramatically impacts cost and quality."
How to decide for a specific workload
Work through four questions before picking a default:
- What does a wrong answer cost? If errors are expensive or hard to detect, start with a frontier model or add a frontier check on a smaller model's output.
- Is there a clear right answer? Extraction, classification and routine edits usually have one, which makes them strong candidates for a smaller model.
- What are the volume and latency targets? High-volume and real-time paths favor smaller models.
- Do you have an evaluation set? If not, build one first. Without it, every choice is a guess.
Then pick a starting strategy. Efficiency-first means starting with a small model and upgrading only for proven capability gaps. Capability-first means starting with the strongest model, reaching the quality bar, then stepping down to cheaper models where evaluations allow.
Combine both instead of choosing one
Routing by task is an increasingly common pattern. A dispatcher can hand each task to a different harness and model "based on the complexity of the task," or a single harness can pair "an expensive, high-intelligence model and a cheap worker model," with the cheap model escalating when a task needs more, or the expensive model outsourcing work to the cheap one. Databricks' Smart Routing works this way for coding agents: the router "defaults to a medium-sized model" and escalates "when the task demands frontier-level capability."
How Databricks approaches model choice
Unity Gateway gives one governed entry point for both sizes. Foundation Model APIs provide pay-per-token access to leading models "with no infrastructure to run," and external providers connect through the same access controls. A model service lets you change the underlying model without changing the name the application calls, and traffic splitting tests a smaller model on a share of live traffic before a full switch.
MLflow evaluation scores each candidate with the same dataset and judges, and the system.ai_gateway.usage table records tokens and latency per request so cost and speed sit next to quality. Smart Routing, in Beta, "selects the lowest-cost model capable of handling your task" for coding agents run through Unity Gateway. For high-volume offline work, AI Functions run batch inference with any AI or ML model.
FAQs
Are smaller models always cheaper?
Per token, yes. Per completed task, not always: a smaller model that needs more steps, retries or human repair can end up costing more than a frontier model that finishes cleanly.
Can a smaller model match a frontier model on my task?
Often, for well-scoped work. The only reliable way to know is to evaluate both on your own examples with the same scorers.
How often should I revisit the choice?
Whenever a candidate model ships or your workload changes. Re-run the same evaluation, and switch only when the results justify it.
The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.