What is an AI gateway?

Summary

• An AI gateway is a control layer between applications or agents and the AI services they call. It handles access, routing, guardrails, and usage tracking in one place, so each application doesn't rebuild them.
• Unity Gateway is the Databricks AI gateway. The docs call it "the Databricks governance solution for enterprise AI." It is built on Unity Catalog and generally available, while service policies (guardrails) and agent services remain in Beta.
• Models, model services, model providers, MCP servers, and skills are Unity Catalog securables with catalog.schema.name names, so you grant access to them the way you grant access to data.
• A model service routes requests to one or more models with traffic splitting and fallbacks, and you can change models or fallbacks without changing the name your application calls.
• The system.ai_gateway.usage system table captures request and response details for model services, model provider services, and MCP services, and inference tables can store raw payloads in Unity Catalog Delta tables.

What is an AI gateway?

An AI gateway is a governed layer between the applications and agents that call AI and the models and tools they call. Instead of every application holding its own provider credentials, limits, safety checks, and logs, calls pass through one path where access is checked, traffic is routed, guardrails can run, and usage is recorded.
Databricks' AI gateway is Unity Gateway, which the docs call "the Databricks governance solution for enterprise AI." Built on Unity Catalog, it extends governance "beyond your data and AI assets to the runtime interactions between models, agents, MCP servers, and tools."

What does an AI gateway do?

This table lists the core jobs and how Unity Gateway documents each one.

Job	What it means	In Unity Gateway
Access control	Decide who can call which models and tools	EXECUTE lets a principal call a model service, a model provider, or an MCP, and you grant it through Unity Catalog
Routing	Send each request to a model, with backups	A model service routes to one or more models, with traffic splitting and fallbacks
Rate limits	Cap request volume	Requests per minute and tokens per minute on model services, requests per minute on MCP services; none are set by default
Guardrails	Check what goes in and what comes out	Service policies (Beta) evaluate the request before the service is invoked and the response after it answers
Usage tracking	Record who called what	The system.ai_gateway.usage system table
Payload logging	Keep the raw exchange	Inference tables store raw JSON request and response payloads in Unity Catalog Delta tables

How does a call move through Unity Gateway?

A call to a model service follows the sequence the docs describe:

1. The caller invokes the service by its catalog.schema.name name, across workspaces or from outside Databricks.
2. Unity Catalog checks privileges. The caller needs EXECUTE on the service, plus USE CATALOG and USE SCHEMA on its catalog and schema.
3. Attached service policies evaluate the request before Databricks invokes the service. If a policy denies it, Databricks returns an HTTP 200 response that names the blocking policy instead of an error status.
4. The service sends the request to a destination chosen by its traffic split. If that attempt fails, for example with a 429 or 5xx error, Unity Gateway retries against the fallbacks in the priority order you define.
5. After the service responds, service policies evaluate the response.

Unity Gateway also captures request and response details for model services, model provider services, and MCP services in system.ai_gateway.usage.

How is this different from calling a provider directly?

When an application calls a provider on its own, that application holds the credential. In Unity Gateway, a model provider service holds it instead. Databricks encrypts and stores the credential, doesn't return it on read, and callers "don't see the stored credentials." Model choice is decoupled the same way: "You can change models or fallbacks without changing the name your application calls."

FAQs

Does an AI gateway add latency?

A gateway sits in the request path, so measure it for your own workloads. The system.ai_gateway.usage table includes latency_ms and time_to_first_byte_ms columns, which gives you the data to do that.

Can one gateway cover more than one model provider?

Yes. A service can route to Databricks-served models, using pay-per-token or provisioned throughput, or to external models through a model provider, and you can mix these destinations in one service.

Does an AI gateway govern tools as well as models?

In Unity Gateway it does. MCP servers are Unity Catalog securables, so callers need EXECUTE on an MCP. Administrators can allowlist individual tools, and MCP policies (Beta) can allow, block, or require approval for a call.

Is Unity Gateway generally available?

Yes. The August 4, 2026 release note says Unity Gateway "is now generally available," and notes that some capabilities, including service policies and agent services, remain in Beta.