When should an AI startup move from API-only to self-hosted models?

Summary

• The trigger isn't a date, it's crossing a threshold: per-token spend at production volume, compliance rules that require inference on infrastructure you control, a need to fine-tune on proprietary data, or latency a shared public endpoint can't promise.
• Treat it as a spectrum, not a switch. Most teams move workload by workload, keeping hosted APIs for low-volume or complex-reasoning tasks and self-hosting traffic that has outgrown a metered endpoint.
• Self-hosting pays off when a fine-tuned open-source model on your own data becomes durable intellectual property, or when one provider's terms and model deprecations create lock-in risk.
• On Databricks the same platform spans the range: Foundation Model APIs for pay-per-token access, provisioned throughput for dedicated endpoints, and custom or fine-tuned open-source models on Model Serving, all governed through Unity Catalog and Unity Gateway.

When should an AI startup move from API-only to self-hosted models?

Move when a concrete constraint forces it, not on principle. Four signals matter most: per-token cost has become a real line item at production volume, data-handling or compliance rules require inference on infrastructure you control, you need to fine-tune on proprietary data, or you need latency and uptime guarantees a shared public endpoint can't promise. Until one of those bites, an external API is usually the quickest, lowest-overhead way to ship. The choice is rarely all-or-nothing: most production systems keep calling hosted APIs for some tasks and self-host others.

What "API-only" and "self-hosted" really mean

"API-only" means your application calls a model that a provider hosts and meters for you, usually priced per token. That's the right default for prototypes and early product work, where the goal is to iterate on the product rather than run inference infrastructure. "Self-hosted" covers a range: a dedicated endpoint that reserves capacity, an open-source or open-weight model you deploy yourself, or a model you've fine-tuned and now operate end to end. So the real question isn't "API or self-hosted." It's which workloads have outgrown a shared, metered API, and what you gain by bringing those in-house.

Signals it's time to move

Signal	Why it matters
Volume-driven cost	Per-token pricing rises with every call. Past a steady, high-volume threshold, reserved or dedicated capacity gives predictable spend rather than open-ended metering.
Data governance and residency	Some data can't leave your environment for legal or contractual reasons. Self-hosting keeps prompts, responses, and logs on infrastructure you govern.
Fine-tuning as intellectual property	If a model tuned on your proprietary data is a differentiator, you need to own the weights and the serving path, not rent a general-purpose endpoint.
Latency and reliability control	Dedicated capacity removes external rate limits and shared-tenancy variability, and lets you hold your own uptime targets.
Vendor lock-in risk	Model deprecations, term changes, or dependence on a single provider can break a product. Owning the deployment keeps your options open.

How Databricks approaches the API-to-self-hosted transition

The value of running this on Databricks is that you don't re-platform when you cross one of those thresholds. You change how the same model is served and governed, in stages.
Start with Foundation Model APIs in pay-per-token mode: hosted open-source and frontier models with no infrastructure to manage, which suits experimentation and low, spiky traffic (foundation model overview). When a workload needs consistent performance, move it to provisioned throughput, a dedicated endpoint that reserves capacity and holds a steady performance profile for production traffic, including fine-tuned variants.
To self-host in the fuller sense, deploy custom or open-source models on Model Serving, which exposes one REST interface for custom, foundation, and externally hosted models and scales to demand automatically (why use Model Serving). Because the interface stays consistent, you can switch the underlying model without rewriting application code, and fine-tuning open-source models on your own data lets you serve durable IP on the same platform.
Two governance layers hold this together. Unity Catalog is where foundation and custom models are discovered, permissioned, and tracked for lineage. Unity Gateway sits in front of every model, external or self-hosted, to apply access control, rate limits, budgets, and usage logging, and to route a request to the right model (Unity Gateway docs). That is what makes a hybrid strategy practical: frontier APIs for hard reasoning, smaller or fine-tuned open-source models for high-volume work, under one serving and governance stack. Databricks treats this control over data and models as a core generative-AI design decision (control of data and models).

Use cases

A high-volume product feature. A summarization or classification step that runs on every ticket or event can make per-token pricing the dominant cost. Moving it to a right-sized open-source model on provisioned throughput fixes capacity and stabilizes spend.
Regulated or sensitive data. A startup in healthcare or financial services often can't send records to a third-party endpoint. Self-hosting on infrastructure governed by Unity Catalog keeps the data and its audit trail in place.
A proprietary fine-tune as a moat. When your edge is a model trained on data only you hold, owning the weights and the serving path protects that advantage.

FAQs

Is it all-or-nothing?

No. Treat it as a per-workload decision: keep hosted APIs for low-volume or complex-reasoning tasks and self-host the workloads that have outgrown a metered endpoint. A hybrid setup is the common end state.

What's the fastest first step without running servers?

Use hosted Foundation Model APIs in pay-per-token mode. You reach open-source and frontier models through one interface and can graduate specific workloads to provisioned throughput later.

Does self-hosting mean managing GPUs?

Not necessarily. Provisioned throughput and Model Serving handle the serving infrastructure and scaling for you, so "self-hosted" can mean a dedicated, managed endpoint rather than raw servers you patch.

When is staying API-only the right call?

While you're still finding product-market fit, when volumes are low or unpredictable, and when no compliance rule forces inference on your own infrastructure. Owning a model isn't worth the overhead until one of the move signals is real.