What is an open model API and when should we use one?

Summary

• Open model APIs provide unified access to high-quality open-source language models like DBRX, Llama-3, and others through a standardized, OpenAI-compatible interface, enabling developers to switch between models without code changes.
• They run on a pay-per-token basis and stay within your security perimeter, allowing rapid prototyping and experimentation without the overhead of managing model deployments.
• Use open models when you need cost-effective performance, data privacy, or flexibility across workloads. They often deliver strong performance at lower cost while keeping your data secure within Databricks.
• Databricks Foundation Model APIs provide unified OpenAI-compatible access across open models, proprietary models (Claude, Gemini), and external third-party models from a single interface.
• Access is available through multiple interfaces: the AI Playground for exploration, REST APIs for application integration, or SQL using ai_query for batch processing.

What is an open model API and when should we use one?

An open model API is a managed service that gives you access to high-quality open-source language models through a unified, programmatic interface. On Databricks, Foundation Model APIs provide OpenAI-compatible endpoints to query high-quality open models like DBRX, Llama-3, and others without building or maintaining your own model infrastructure. You query these APIs on a pay-per-token basis directly from your Databricks Workspace, and your data stays within Databricks' security perimeter throughout.

Understanding open models and their role

Open models are large language models released publicly with transparent architectures and weights available for download or use. Open models can be studied, modified, and deployed anywhere you choose. They offer distinct advantages including no vendor lock-in, full transparency of model design, and the ability to run them from your local machine to enterprise infrastructure. They excel at tasks like summarization, code generation, and reasoning.
A Foundation Model API abstracts away the complexity of managing model deployments. Instead of provisioning compute, containerizing models, and handling scaling yourself, you make API calls and pay only for tokens consumed. This is the managed equivalent of running open models on your own infrastructure.

Key capabilities to evaluate in open model APIs

Capability	Why it matters
Model selection and switching	Can you choose different models for different workloads? Do you need to rewrite code when switching? Standardized interfaces (like OpenAI-compatible APIs) let you experiment without refactoring.
Data privacy and governance	Are requests logged externally? Does the provider have access to your queries? Databricks-hosted models keep all data and logs within your workspace.
Cost-to-capability ratio	Do the available models provide performance sufficient for your use case? Open models often deliver strong cost-to-performance ratios.
Integration breadth	Can you use the API from applications, scripts, dashboards, and notebooks? Support for REST APIs, Python SDKs, and SQL queries ensures flexibility across your stack.
Latency and availability	Do response times meet your application's needs? Is the service available during your peak usage windows?
Batch and real-time paths	Do you need sub-second responses (real-time) or can batch processing (higher throughput, lower cost per token) work for your use case?

When to use open model APIs

Rapid prototyping and development: Open models deliver strong performance and keep your data secure within Databricks' security perimeter. You can experiment quickly without concerns about external data transit or vendor lock-in.
Cost efficiency and performance tuning: Open models provide strong cost-to-performance ratios. You can select the best model for each specific workload, switching between models for different tasks without code changes. This flexibility lets you optimize for both quality and cost.
Data privacy and governance: When your organization has strict data residency or privacy requirements, Databricks-hosted open models ensure your data and queries never leave your workspace. You get unified authentication, security, and governance controls across all model access.
Flexibility and provider independence: The unified OpenAI-compatible interface lets you move fluidly between models for real-time and batch use cases. If a particular model doesn't meet your needs, switching to another takes only a configuration change, not a code rewrite.
Integration with existing tooling: If your applications already use OpenAI clients or SDKs, you can redirect requests to Databricks-hosted open models with only configuration changes. No application code modifications needed.

How Databricks approaches open model APIs

Databricks Foundation Model APIs provide a unified, OpenAI-compatible interface across open models (DBRX, Llama-3, Llama-2), proprietary models (Claude, Gemini), and external third-party models. This standardization means you can experiment across different model families and providers using the same code.
Three access patterns meet different use cases:

1. AI Playground: An interactive web interface for exploring models and testing prompts without writing code. Ideal for quick experimentation and learning.
2. REST APIs: Direct HTTP endpoints compatible with standard OpenAI client libraries and SDKs. Integrate into applications, services, and batch jobs directly.
3. SQL integration via ai_query: Query open models directly from SQL notebooks and Lakeflow Pipelines. This enables batch processing of large datasets and integration with existing SQL workflows.

All three paths respect your data security and governance: models run within your workspace, all queries stay internal, and requests are not used for model training or external logging. Requests are priced on a pay-per-token basis with no minimum commitments.

Use cases

Content generation and summarization: Batch-process customer feedback, support tickets, or documents to extract summaries, sentiment, or key themes without sending data to external APIs.
Code generation and refactoring: Generate unit tests, documentation, or refactored code suggestions while keeping proprietary code internal to your organization.
Semantic search and embeddings: Use open models to generate embeddings for your data, then store and search them within Databricks AI Search, all without data leaving your workspace.

FAQs

Which model should I use for my workload?

Databricks Foundation Model APIs let you access both open and proprietary models. Choose based on your specific requirements: cost-efficiency, data privacy, model availability for your use case, or integration needs. The flexibility of trying multiple models through the same API means you can test and select the best fit for each application.

How much does it cost to use open model APIs?

Foundation Model APIs are priced on a pay-per-token basis with no minimum commitments. Pricing varies by model. See current pricing on the Databricks pricing page.