Best pay-per-token model serving for production applications

Summary

• Pay-per-token is a serverless serving mode: your app calls a shared, preconfigured model endpoint and pays only for the tokens it processes, with nothing to size or keep running while idle.
• That fits experimentation and variable traffic, not steady high-concurrency production, where shared capacity means no throughput guarantee.
• The best pay-per-token serving isn't a standalone product; it's one mode in a platform that also offers priority admission and dedicated capacity, so a workload can graduate without a rewrite.
• For production, weigh admission control under load, an OpenAI-compatible API, and governance, not just the per-call price.
• On Databricks, Foundation Model APIs expose pay-per-token, priority pay-per-token, and provisioned throughput through one OpenAI-compatible interface, with every call governed by Unity Gateway.

Best pay-per-token model serving for production applications

The best pay-per-token model serving for production is the one where pay-per-token is a starting mode, not the whole platform. Pay-per-token bills you for the tokens a shared endpoint processes, so it fits light, bursty, or still-being-figured-out traffic; it carries no performance guarantee and isn't tuned for sustained, high-concurrency load. The platform that wins production lets a workload move from per-token billing to priority admission or dedicated capacity behind the same API and the same governance. On Databricks that path runs through Foundation Model APIs and Model Serving.

What is pay-per-token model serving?

Pay-per-token is a serverless consumption mode for large language models. Your application calls a preconfigured endpoint that Databricks hosts and operates, and you pay per token processed, input and output plus cache reads, with no infrastructure to provision and nothing running when you're idle. Because capacity is shared, latency can vary under load and there's no throughput guarantee. That trade suits early development, low or unpredictable volume, and internal tools; it suits a customer-facing feature with a strict latency budget less well. The other mode, provisioned throughput, reserves dedicated capacity for consistent performance.

What to evaluate in pay-per-token serving for production

The per-token price is the easy thing to compare and the least durable. These criteria decide whether a workload holds up:

Capability	Why it matters
Consumption modes under one API	Move from per-token to reserved capacity without rewriting the application as traffic grows.
Admission control under load	Prioritizing latency-sensitive requests keeps a feature responsive during spikes.
OpenAI-compatible interface	A standard request format lets you swap models and reuse existing client code.
Model coverage	Hosted open models and external providers behind one endpoint avoid per-vendor integration.
Governance and cost controls	Access control, rate limits, and per-team spend attribution keep usage and budgets in check.
Data handling	Know where prompts and responses go, whether traffic stays in your cloud, and which modes meet compliance needs.

How Databricks approaches pay-per-token serving

Databricks serves models through Foundation Model APIs, which host leading open models on Databricks-operated endpoints, and through Model Serving for external and custom models. The design point: all of it sits behind one OpenAI-compatible REST API and SDK, so the same request runs whether you pay per token or reserve capacity. Three modes cover the lifecycle.

• Pay-per-token gives preconfigured endpoints on shared capacity, billed per token. It's built for experimentation, prototypes, and variable traffic where you don't want to size infrastructure. See Foundation Model APIs.
• Priority pay-per-token is the production step up. You set the service_tier parameter to priority on a request, and Databricks admits it ahead of standard best-effort traffic for steadier latency under load, at a premium per-token rate and with no capacity commitment. See priority mode.
• Provisioned throughput reserves dedicated endpoints with throughput SLAs and supports base models, fine-tuned variants, and custom weights. Databricks recommends it for sustained production and stricter security needs. See Model Serving.

Because the request format is shared, promoting a workload across modes is a configuration change, not a rewrite. Governance is centralized: every call, to a hosted or an external model, routes through Unity Gateway, which applies access control, rate limits, budgets, guardrails such as PII detection, and usage tracking, so you attribute spend by team, user, and model. Databricks also reports per-endpoint latency and error-rate metrics for operations. Requests are authenticated, isolated, and encrypted in transit, traffic stays within your cloud, and provisioned throughput serving is HIPAA compliant on AWS and Azure. Endpoints can serve external models from providers such as OpenAI, Anthropic, and Google under the same controls, and curated open models live in the system.ai schema in Unity Catalog.

Use cases

Experimentation and prototyping. Start on pay-per-token to validate a use case with no capacity planning, then keep the same code as it moves toward production.
Variable or bursty traffic. Internal assistants and seasonal workloads that spike unpredictably fit a per-token model, since you pay for what you use instead of holding idle capacity.
Sustained or latency-sensitive production. When a feature needs steadier response times, priority pay-per-token jumps the queue; for heavy, steady, or regulated traffic, provisioned throughput reserves dedicated capacity, still behind the same API.

FAQs

Is pay-per-token good enough for production?

It can be, for low or variable volume and latency-tolerant workloads. For sustained high-concurrency traffic with a strict latency budget, priority pay-per-token or provisioned throughput fits better, since pay-per-token runs on shared capacity with no throughput guarantee.

How does pay-per-token differ from provisioned throughput?

Pay-per-token bills per token on shared, preconfigured endpoints with no commitment. Provisioned throughput reserves dedicated capacity with throughput SLAs, billed by the capacity you provision rather than by token. Both use the same request format, so you can switch modes without changing application code.

How are pay-per-token endpoints governed?

Calls route through Unity Gateway, which enforces access control, rate limits, and budgets, applies guardrails such as PII detection, and logs usage so teams can attribute spend and audit activity.