Do AI gateways provide models or just route requests?

Summary

• An AI gateway routes and governs requests to models. The models run behind it, hosted by Databricks or by an external provider.
• Unity Gateway is the Databricks AI gateway and the control plane for AI in Unity Catalog. It routes requests, limits AI spend, manages fallbacks, and monitors usage.
• Models behind Unity Gateway come from Foundation Model APIs (models Databricks hosts), external model providers you connect, and serving endpoints.
• Callers reach a model through a model service named catalog.schema.name, and the EXECUTE privilege controls who can call it.
• Unity Gateway has been generally available since August 4, 2026. Some capabilities, including service policies and agent services, remain in Beta.

Do AI gateways provide models or just route requests?

AI gateways route and govern requests to models, and the models run behind them. Databricks describes Unity Gateway as an AI gateway that "routes requests, limits AI spend, manages fallbacks, monitors usage, and connects coding agents to model capacity." Databricks hosts the latest partner and open-weight models, external providers serve theirs, and Unity Gateway is how applications and agents reach them. Databricks also provides ready-to-use, pay-per-token model services in system.ai, so a team can start without creating its own model service.

How does a request move through Unity Gateway?

When an agent calls a model, Databricks authenticates the request, authorizes it against Unity Catalog, and routes it through Unity Gateway before it reaches any external system. To get started you authenticate with a personal access token, and OAuth is recommended for production use cases. The gateway exposes an OpenAI-compatible endpoint, and you pick the model by passing its fully qualified name.

What does Unity Gateway do between your application and the model?

Capability	What the documentation says
Routing	A model service is an endpoint that translates and routes inference requests to one or more models
Traffic splitting	Distributes requests across multiple model backends behind one model service, for example to roll out a new model gradually or run A/B tests
Fallbacks	If a request fails, the system retries it against the configured fallback destinations in the exact order specified
Access control	Models, model services, model providers, MCPs, and skills are Unity Catalog securables, and EXECUTE lets a principal call a model service
Rate limits and spend caps	Set rate limits and spend caps to protect capacity and control cost
Usage tracking	Track model and MCP usage, including who called what and when, in governed usage tables
Payload logging	Inspect full request and response payloads in inference tables
Provider credentials	A model provider connection routes requests to an external provider using centrally managed credentials

Where do the models behind Unity Gateway come from?

Databricks documents three sources.

• Foundation Model APIs. Databricks hosts the latest partner and open-weight models, and a model service can route to Databricks-served models using pay-per-token or provisioned throughput.
• External providers. You can use Unity Gateway with your own model provider account. The Custom provider type routes to an OpenAI-compatible provider.
• Serving endpoints. The Unity Gateway overview lists serving endpoints as a way to query models. Databricks documents separate serving-endpoint query options for endpoints you host yourself.

How does Databricks approach AI governance for model access?

The governance guide says AI governance keeps the AI your organization uses safe, compliant, and cost-controlled, and that on Databricks Unity Gateway is the control plane for AI. It spans three dimensions. Asset governance: Unity Catalog manages every model, MCP server, function, and connection as a securable object. Traffic governance: Unity Gateway routes every model service and MCP service request from a central control plane, and enforces rate limits, budgets, and usage tracking. Behavior governance: service policies allow or deny individual requests and responses, and they are in Beta.

FAQs

Where are the models actually hosted?

Databricks hosts the models available through Foundation Model APIs, and external providers serve the models you reach through a model provider connection. Unity Gateway is the access and control layer in front of both.

Can I fall back to another model if a request fails?

Yes. A model service supports fallbacks. If a request fails, for example with a 429 or 5xx error, the system retries against the configured fallback destinations in the exact order specified, until one succeeds or all are exhausted. Fallbacks are independent of traffic splitting.

Do I have to use Unity Gateway to call a model on Databricks?

Databricks recommends model services in Unity Gateway when you are getting started and want to call an LLM. They give governed access to foundation models served natively by Databricks without configuring a serving endpoint. For endpoints you host yourself, the documentation describes separate serving-endpoint query options.

Can I connect my own provider account?

Yes. A model provider is registered as a model provider service in Unity Catalog, and you govern access using your organization's credentials. By default only the model provider service owner can query it, so grant EXECUTE to let others call it.

Can I cap what a team spends on models?

Yes. The governance guide says to set rate limits and spend caps to protect capacity and control cost. The Unity Gateway overview adds that you can monitor usage and cost across models, MCP servers, and coding agents.

Does Unity Gateway govern anything besides model calls?

Yes. Unity Gateway routes every model service and MCP service request from a central control plane, and the overview says you can restrict access to models, MCP servers, and skills.