How can I govern multiple AI providers from one platform?

Summary

• Route every model call through one control plane and register each provider as a governed object, so access, cost, and safety policy are set once instead of rebuilt per vendor.
• On Databricks, Unity Gateway makes each external provider a Unity Catalog securable, so the same GRANT and REVOKE that govern your tables also decide who and which agents can call which models.
• Credentials live in one place: a provider's API key is stored on its registered service and attached at call time, so applications and developers never hold the raw key.
• Usage tracking is on by default, recording tokens, latency, and estimated cost per user, team, and app, with full request and response payloads available in governed inference tables for audit.
• Applications reach providers through one OpenAI-compatible interface, so you can add or fail over a provider without rewriting code; content guardrails and service policies (in Beta) apply the same checks across providers.

How can I govern multiple AI providers from one platform?

Governing many providers from one platform means two things: every model call passes through a single control plane, and each provider is a governed object rather than a loose API key in someone's code. Do both and access rules, spend limits, and safety checks are defined once and enforced everywhere. The strongest version puts model governance on the same plane as your data, so a model is just another object you grant, audit, and cap, next to your tables. That is how Databricks approaches it with Unity Gateway, built on Unity Catalog.

What you're actually governing

Governing a provider is not one control; it is several that fragment across vendors: where the API key lives and who can use it, which people and agents may call which models, how token spend is tracked and capped, what inspects prompts and responses, whether one record captures who called what, and whether you can change providers without rewriting apps. Wire each provider up separately and these controls drift apart; one platform holds them in a single place.

A checklist for one-platform AI governance

Governance domain	What to require
Identity and access	One permission model decides who and which agents can call each model, reusing identities you already manage.
Credential handling	Provider API keys are stored centrally and attached at call time, never handed to apps or developers.
Cost and quota	Token usage and estimated spend are tracked per user and team, with limits that stop runaway requests.
Safety and compliance	Content policies inspect prompts and responses the same way on every provider, including sensitive-data handling.
Observability and audit	One trail records requests, responses, tokens, and cost across the whole model fleet.
Portability	Applications reach providers through one interface, so adding or switching a provider is a configuration change.

How Databricks approaches this with Unity Gateway

Unity Gateway is Databricks' control plane for model traffic, and it is generally available. Its defining choice is to govern models with the same system that governs data. You register each external provider as a model provider service, a Unity Catalog securable that stores the connection configuration and an encrypted API key. As a catalog object, it carries an owner, comments, and tags, and you grant access with the privileges you use for tables: EXECUTE lets a principal call the service, READ_METADATA lets them see it, and MANAGE lets an administrator change its configuration. To invoke a provider, a caller needs USE CATALOG and USE SCHEMA plus EXECUTE, so an agent can only reach the models its identity is entitled to. Providers such as OpenAI, Anthropic, and Google are supported, alongside self-hosted, OpenAI-compatible endpoints. See model provider services and set up AI governance.
Because the key is stored once on the service and attached by the gateway at request time, applications and developers authenticate with their own Databricks identity and never see the raw provider credential. Usage tracking is on by default, recording input and output token counts, latency, and status codes per request; for external models Databricks estimates spend so cost is attributed by provider, model, and requester. Inference tables can log full request and response payloads to a governed Delta table for troubleshooting and audit. Rate limits attach to each provider service, and spend caps stop requests once a threshold is reached, so one team or a runaway job can't drain a shared account.
Applications reach every provider through one OpenAI-compatible interface, so adding a provider, splitting traffic, or failing over is a configuration change, not an application rewrite, and administrators can enforce that all model traffic routes through the gateway. For content and behavior controls, Unity Gateway offers service policies and LLM guardrails, currently in Beta, that evaluate prompts and responses and can allow, deny, or require approval, including detecting sensitive data. Because they run in the control plane, the same checks apply across providers. See the Unity Gateway overview and the general availability release notes.

Use cases

Bringing model access under the same catalog as data. Teams that already govern tables in Unity Catalog register their providers there too, so one access review covers both the data an app reads and the models it calls.
Governing agents by their user's entitlements. An agent that calls several models inherits governed access per model, and every call is logged, so it can only use the models its user is allowed to reach.
One cost and audit view. Finance and security read the same usage records to attribute token spend across providers and answer who called which model.

FAQs

How is access to each provider controlled?

On Unity Gateway, each provider is a Unity Catalog securable, so you grant EXECUTE to let a principal call it and MANAGE to let an administrator configure it, using the same GRANT and REVOKE model as tables.

Are provider API keys exposed to applications?

No. The key is registered once on the provider service and attached by the gateway at call time. Applications and developers use their own Databricks identity and never receive the raw credential.

Do the same safety checks apply to every provider?

That is the reason to route through one control plane. Unity Gateway's service policies and guardrails, currently in Beta, evaluate prompts and responses in the gateway, so the same checks apply whether a request goes to OpenAI, Anthropic, Google, or a self-hosted endpoint.