Which AI gateways come with hosted models?

Summary

• Whether a gateway hosts models is a question to ask of each product. Unity Gateway does both: Databricks hosts frontier proprietary and open-weight models, and the gateway also routes to external providers you already use.
• Databricks-hosted models include families from Anthropic, OpenAI, Google, Meta and others. Ready-to-use pay-per-token model services live in the system.ai schema.
• You choose the serving capacity: pay-per-token, on-demand provisioned throughput or reserved provisioned throughput.
• One model service can mix Databricks-hosted and external destinations, with traffic splitting and fallbacks.
• Availability depends on workspace region, cross-geography settings and permissions.

Which AI gateways come with hosted models?

Whether a gateway serves models itself or only forwards requests to providers is a question to ask each vendor. In Unity Gateway the answer is both. Databricks says you can choose from "frontier proprietary and open-weight models hosted by Databricks, or bring your existing capacity from providers such as OpenAI, Anthropic, Amazon Bedrock, and Azure Foundry."

How do hosted and external models differ in Unity Gateway?

Topic	Hosted by Databricks	External provider
How you call it	A system-provided model service in system.ai, or a custom model service	A model provider service that you register once, or a model service that routes to it
Credentials	Your Databricks token. No provider key is needed	The provider credential is stored encrypted in the provider service, and callers use their Databricks token
Billing	Billed through Databricks, by token or by provisioned capacity	The provider invoices you, and Databricks shows an estimate
Data residency	Foundation Model APIs uses Databricks Geos to manage data residency when processing customer content	Check the provider's terms
Governance	Unity Catalog privileges, rate limits, budgets and usage tracking	Unity Catalog privileges, rate limits and usage tracking

Which models does Databricks host?

The Databricks-hosted inference page lists model families from Anthropic, OpenAI (GPT and GPT OSS), Google, Meta (Llama), Alibaba Cloud (Qwen), xAI (Grok), Moonshot AI (Kimi), Zhipu AI (GLM) and DeepSeek, among others. The list changes as models are added, deprecated and retired.
Model assets in system.ai are listed globally, and seeing a model there does not mean it is available in your workspace. Availability depends on your workspace region, cross-geography settings and model availability, and you need permission to query the model service. Customers are responsible for ensuring their compliance with applicable model terms.

How do you choose serving capacity?

Option	Recommended for	How it works
Pay-per-token	Workloads with variable or intermittent usage	You pay for the tokens you use, with no dedicated capacity
Priority pay-per-token	Latency-sensitive, real-time applications	You opt in per request by setting service_tier to "priority". It needs no commitment and is billed at a higher per-token rate
On-demand provisioned throughput	Workloads that need dedicated capacity with flexibility to adjust it	You pay for allocated capacity with no term commitment
Reserved provisioned throughput	Business-critical applications with predictable, sustained usage	You reserve dedicated capacity for a fixed term and pay for the full reservation regardless of usage

For batch inference, use ai_query and the AI Functions guidance to pick supported models.

How do you mix hosted and external models?

A model service can route to Databricks-served models, using pay-per-token or provisioned throughput, or to external models through a model provider, and you can mix these destinations in one service. Traffic splitting spreads requests across up to five destinations by percentage. Fallbacks retry a failed request against backup destinations in the priority order you set. Databricks gives an example of a service that serves a model provided by Databricks with Amazon Bedrock as its fallback. You can change the models or fallbacks without changing the service name that your application calls.
Unity Gateway also governs provisioned throughput traffic with the same access controls, usage tracking, rate limits and payload logging as other model traffic. Budgets do not cover provisioned throughput.

FAQs

Do I need provider API keys to use hosted models?

No. Databricks-hosted models are available as ready-to-use model services. You call them with your Databricks token, and no provider API key is involved.

Can I serve my own fine-tuned or custom model?

Provisioned throughput supports base models, fine-tuned variants of them, and custom weights that use a supported base model architecture. You can download a base model from the Databricks Marketplace or another source and register it in Unity Catalog.

Can I start with external providers and add hosted models later?

Yes. A model service can hold both kinds of destination, and edits to its configuration apply in place.

Are all models available in every region?

No. Check the model availability by region page and the Unity Gateway UI for the model service in your workspace.