How can we use GPT, Claude, and Gemini securely on our own data?

Summary

• Yes. The secure pattern is provider-agnostic: call GPT, Claude, or Gemini through a governed Databricks endpoint rather than a provider's public API, so your data stays in your governed lakehouse and every request is logged.
• Data governed by Unity Catalog stays under your control. Row filters and column masks decide what a query can read before any content is placed in a prompt, and lineage and audit logs record access.
• Model Serving external model endpoints hold provider keys centrally and encrypted, so application code never touches raw credentials and you can change models without a rewrite.
• Unity Gateway, generally available, governs the traffic: grant and revoke model access with EXECUTE, set rate limits, track usage in system tables, and log full request and response payloads to inference tables.
• Guardrails and service policies can redact or block sensitive data before a prompt reaches an external provider, which gives regulated teams a defensible audit trail.

How can we use GPT, Claude, and Gemini securely on our own data?

Yes, and the secure pattern is the same no matter which model you choose: route every request through a governed endpoint on Databricks rather than calling a provider's public API directly. Your data stays in your governed lakehouse under Unity Catalog, the model is reached through Model Serving external model endpoints, and Unity Gateway applies access control, rate limits, and logging to each call. A model sees only the content you deliberately send it, provider keys never live in application code, and every request is attributable and auditable.

What secure access to an external model requires

The hard part isn't the model, it's the plumbing around it. Call a provider API directly from many apps and you end up with scattered keys, no record of what data left your walls, and safety rules that differ from team to team. Securing external models means putting one governed layer between your applications and every provider, so access, data handling, and audit are decided once and enforced the same way for GPT, Claude, and Gemini alike.

Key capabilities to evaluate

Capability	Why it matters
Governed model access	Decide who can call which model with the same permission model you use for data, not a spread of raw API keys.
Credential isolation	Provider keys are held centrally and encrypted, never embedded in notebooks or application code.
Data residency and lineage	Source data stays in your governed store, and you can trace what was read and sent.
Input and output screening	Detect and redact sensitive fields before a prompt reaches an external provider.
Usage and cost tracking	Attribute tokens, latency, and cost by user, team, and application.
Full payload audit	Keep a queryable record of requests and responses across every provider.
Provider portability	Add or change models without rewriting applications.

How Databricks approaches this

On Databricks, external models run behind governed endpoints, and the data they act on stays in Unity Catalog.

• External model endpoints. Register GPT from OpenAI, Claude from Anthropic, and Gemini from Google as external models in Model Serving. Applications call one stable Databricks endpoint, and the provider key is stored encrypted in Databricks rather than handled by client code. Endpoints are OpenAI-compatible, so changing the served model is a configuration change, not a rewrite. See What are external models?.
• Data stays in the governed lakehouse. Your data lives as Delta tables in Unity Catalog. Row filters and column masks decide what a query can read before any of it is placed in a prompt, and lineage and audit logs record who accessed what. The provider receives only what a request carries, not access to your tables.
• Runtime governance with Unity Gateway. Unity Gateway, generally available, governs the traffic. External providers are registered as governed objects you grant and revoke with EXECUTE, rate limits protect spend and availability, usage is tracked in system tables, and full request and response payloads can be logged to inference tables as governed Delta tables. See AI governance with Unity Gateway and Govern external model providers.
• Screen sensitive data before it leaves. Guardrails and service policies can evaluate each request and response and redact or block sensitive content, so a prompt is cleaned of what a provider should not see. Registering external providers as governed objects in Unity Catalog is a newer capability, so confirm its current availability for your cloud before you depend on it.

Use governed endpoints as the default, since they carry the access control, rate limits, usage tracking, and payload logging. Direct passthrough to a provider loses that observability and should stay a temporary exception.

Use cases

Prototype on real data without exporting it. A data science team can point a frontier model at governed customer records to test a summarization or extraction task, without copying data out to an external service.
Meet compliance for regulated data. A team handling personal or financial data can redact sensitive fields before any prompt leaves, then show auditors a full log of what was sent and returned.
Run many models from one control plane. A platform team can let different groups use different frontier models through one governed endpoint, each with its own budget, rate limit, and audit trail.

FAQs

Does our data get used to train the external model?

Only the content you place in a request reaches the provider. Databricks does not send your tables to a model; a request carries only the prompt you construct. Review each provider's data-use terms, and use guardrails to strip sensitive fields before they leave.

Where are the provider API keys stored?

Centrally and encrypted in Databricks. Applications authenticate to the governed endpoint rather than to the provider, so keys are not copied across notebooks, jobs, and application code.

Can we switch models without rewriting our apps?

Yes. Applications call a stable governed endpoint, and the served model is set in configuration. Because the endpoints are OpenAI-compatible, the same client code can target GPT, Claude, or Gemini.

How do we prove what was sent to a model?

Usage tracking records tokens, latency, and cost per user and application in system tables, and inference tables log full request and response payloads to governed Delta tables for audit and debugging.