How can I reduce LLM costs without limiting developer access?

Summary

• You reduce LLM cost by governing how models get used, not by cutting developers off: route every model, agent, and tool call through one control point so spend becomes visible, cappable, and attributable.
• On Databricks that control point is the Unity Gateway. It logs every request to the system.ai_gateway.usage system table, so you can find the applications and users driving the bill and fix those instead of throttling everyone.
• Rate limits in requests per minute and tokens per minute apply on an endpoint or per user, and a per-user limit overrides the endpoint ceiling, so you cap runaway service accounts while giving developers headroom.
• Budgets track spend across Databricks-hosted models, external providers, and coding agents from one view, alert as a threshold approaches, and can block usage once crossed; pair them with rate limits for a firm ceiling.
• Model routing keeps routine calls on lower-cost models and reserves premium models for the work that needs them, so teams spend less without giving up capability.

How can I reduce LLM costs without limiting developer access?

Cut LLM cost by governing how models get used, not by taking access away. Route every model, agent, and tool call through one control point and you can see what each application spends, cap runaway usage, route work to the right model, and attribute the bill to a team, all while developers keep access to the models they need. The mistake is controlling cost by locking developers to one model or a blunt quota, which slows real work and pushes people toward shadow tools. On Databricks, that control point is the Unity Gateway, a single governed entry for model, agent, and tool traffic.

Why cost and access pull against each other

Give developers open, direct access to frontier models and spend is invisible and uncapped: an agent stuck in a loop, or a batch job on an oversized model, runs up a bill no one sees until it lands. Clamp down with a company-wide quota and you starve real work. A control point between your applications and the models they call is the way out, because centralizing traffic makes spend visible and lets you apply limits precisely rather than across the board.

Cost levers that keep access open

Lever	What it does	Why access stays open
Usage visibility	Logs every request with token counts and caller identity	You fix the few expensive workloads instead of throttling all traffic
Cost attribution	Tags spend by user, team, tool, or use case	Spend gets an owner, so limits land where the cost is
Model routing	Sends routine calls to lower-cost models	Developers keep capability without paying premium rates for everything
Rate limits	Caps requests and tokens per minute, per user or endpoint	Power users get headroom while runaway accounts are capped
Budgets	Alert and can block when a threshold is crossed	Teams self-manage a budget instead of hitting one hard cap
Guardrails	Screen requests before they reach a model	Block wasteful or unsafe calls, not legitimate traffic

How Databricks approaches this with the Unity Gateway

The Unity Gateway is the single control point for model, agent, and tool traffic on Databricks. Developers reach models from multiple providers, plus coding agents, through one governed endpoint, so you don't have to restrict them to a single model to stay in control.
See it and attribute it. Every request is logged to the system.ai_gateway.usage system table, capturing token counts, latency, and caller identity, and usage rolls up by request tag so spend maps to a user, team, tool, or use case. That is how you answer which applications are driving the LLM bill and act on the few that matter.
Cap it precisely. Rate limits are set in requests per minute and tokens per minute, globally on an endpoint or per user, and a per-user limit overrides the endpoint ceiling, so you tighten a service principal while giving a developer headroom. Account budgets track spend across Databricks-hosted models, external providers, and coding agents from one view, set by user, team, or tag. A budget can alert while access continues, or block usage once the threshold is crossed. Enforcement is approximate and spend can overshoot before requests stop, so pair budgets with rate limits for a firm ceiling; note that budgets don't yet track provisioned throughput or bring-your-own-key provider usage.
Right-size it. Routing keeps routine calls on lower-cost models and reserves premium models for the tasks that need them, so teams spend less without losing capability. See how to govern a coding agent's model access.
The Unity Gateway also applies guardrails and service policies that screen traffic, such as blocking unsafe content, with an allow, ask, or deny decision on each request. These capabilities are still evolving, so confirm the current lifecycle state of guardrails and service policies with your Databricks account team before you design around them. For the current feature set, see what's new in the Unity Gateway and expanding agent governance with the Unity Gateway.

Use cases

• Contain agent runaway: rate limits and budgets stop a looping agent from running up a surprise bill, without touching other teams.
• Give power users headroom: set higher per-user rate limits for developers while capping service accounts and batch jobs.
• Chargeback by team: request tags attribute spend to the team that drove it, so each team manages its own budget.

FAQs

How do I reduce LLM cost without slowing developers down?

Route calls through the Unity Gateway and apply precise controls: per-user rate limits, team budgets, and model routing. You cap runaway usage and steer routine work to cheaper models while developers keep access to the models they need.

How do I find which applications drive my LLM bill?

Query the system.ai_gateway.usage system table. It logs every request with token counts and caller identity, and request tags group spend by application, user, or team, so you can target the expensive workloads.

Can a budget stop developers from working?

It depends how you set it. A budget can alert while access continues, or block usage once the threshold is crossed, so per-team budgets and per-user rate limits contain cost without one hard cap on everyone.