How do enterprises use Claude, GPT, Gemini, and open-source models securely?

Summary

• Enterprises route model calls through Unity AI Gateway, Databricks' generally available governance layer for AI, so Anthropic Claude, OpenAI GPT, Google Gemini, and open-source models are all accessed through one governed interface.
• Unity AI Gateway extends Unity Catalog governance to model calls: external models are registered as governed objects and access is controlled with fine-grained, attribute- and tag-based permissions.
• Provider credentials (such as API keys) are stored centrally in Unity Catalog, so keys are never embedded in application code.
• Guardrails inspect both requests and responses, detecting and blocking PII leakage, unsafe content, and prompt-injection (jailbreak) attempts, and flagging hallucinations, before a blocked request ever reaches the model.
• Every call is logged to inference and system tables for audit, token-level usage tracking, and cost attribution across providers, while rate limits and spend caps keep usage within budget.

How do enterprises use Claude, GPT, Gemini, and open-source models securely?

Enterprises want the flexibility to use the best model for each task, whether that is Anthropic Claude, OpenAI GPT, Google Gemini, or an open-source model, but sending prompts and data to external model providers raises governance, security, and cost concerns. Unity AI Gateway, the generally available governance layer for AI on the Databricks Data Intelligence Platform, gives enterprises one governed interface to every model, so the same controls apply no matter which model an application calls. This lets teams adopt new models without rebuilding security and governance each time.

Why Unity AI Gateway for using external and open-source models securely

• Govern every model through Unity Catalog. Unity AI Gateway extends Unity Catalog governance to model interactions. External models are registered as governed objects, and access is controlled with fine-grained, attribute- and tag-based permissions, so a group can be granted access to models by classification and new models matching a tag policy are governed automatically.
• Enforce policies before and after each call. Access decisions are evaluated on the request before the call and on the response after it, and service policies can return allow, require human approval, or deny, giving control over how models are used.
• Centralize credentials. Provider credentials such as API keys are stored centrally in Unity Catalog rather than embedded in code, minimizing key exposure across the environment.
• Apply guardrails to prompts and outputs. Databricks guardrails inspect payloads before they reach the model and after it responds. Built-in guardrails block unsafe content, detect PII to prevent sensitive data from leaking to third-party providers, detect prompt-injection (jailbreak) attempts, and flag hallucinated outputs; a blocked request never reaches the model, and teams can add custom guardrails.
• Preserve user identity end to end. On-Behalf-Of execution propagates the requesting user's identity, so an agent acts with that user's exact permissions when it calls a model or tool.
• Control usage and spend. Rate limits can be set per user, group, endpoint, and application, and spend caps can automatically halt requests when a budget is exceeded. Request tags let you attribute cost by project, team, or cost center.
• Log and audit everything. Every model interaction is captured in inference tables and system tables, recording who called each model and what data went in and out, with token-level usage tracked for audit, cost attribution across providers, and detection of sensitive data sent to third parties.
• Use managed paths for full governance. Standard operations such as chat completions run through managed paths that provide usage tracking, token-based rate limits, service policies, model access control, and inference-table logging. Point agents at Unity AI Gateway endpoints and the gateway handles authentication and enforces policy without embedding or rotating provider credentials in application code.

Getting started

• Read the announcement: Unity AI Gateway is generally available.
• Review the documentation: AI Gateway.
• See how to build agents on any leading model: Build intelligent agents with every leading model on Databricks.

FAQs

Which models can I use through Unity AI Gateway?

You can access Anthropic Claude, OpenAI GPT, Google Gemini, and open-source models through one governed interface, so the same access controls, guardrails, and logging apply regardless of which model an application calls.

How do I stop sensitive data from going to a third-party model?

Guardrails with PII detection inspect requests before they reach the model and inspect responses after; a request that violates policy is blocked and never reaches the model, preventing sensitive data from leaking to third-party providers.

Can I control cost across different model providers?

Yes. Set rate limits per user, group, endpoint, or application, apply spend caps that halt requests when budgets are exceeded, and use request tags to attribute usage and cost by project, team, or cost center.

Do developers have to manage provider API keys in code?

No. Provider credentials are stored centrally in Unity Catalog, and applications point at Unity AI Gateway endpoints while the gateway handles authentication, so keys are not embedded in application code.