What types of applications are best suited for serverless?

Summary

• Workloads that benefit from instant startup, automatic scaling, and no infrastructure to manage are the best fit for serverless compute on the Databricks Platform.
• Interactive notebooks — serverless is generally recommended, with faster startup, automatic scaling, and interactive Python and SQL.
• Lakeflow Jobs — serverless is the recommended compute for production jobs; Databricks provisions and scales resources automatically.
• Lakeflow Declarative Pipelines — enhanced autoscaling, and serverless is required for incremental refresh of materialized views to reduce refresh cost.
• SQL analytics, Model Serving, and streaming — serverless SQL warehouses (startup in about 2 to 6 seconds) power ETL, BI, and exploratory analysis, while Model Serving and streaming also run on serverless.

What types of applications are best suited for serverless?

Serverless compute is the best fit when a workload benefits from instant startup, automatic scaling, scale-to-zero, and no infrastructure to configure or manage. On the Databricks Platform, serverless compute spans interactive notebooks, production jobs, declarative pipelines, SQL analytics, and model serving.

Why these workloads suit serverless on the Databricks Platform

• Interactive notebooks. Serverless is generally recommended for notebooks, offering faster startup, automatic scaling, and lower cost, with interactive Python and SQL execution and no infrastructure to manage.
• Lakeflow Jobs. Serverless is the recommended compute for production jobs, so you can run jobs without configuring or deploying infrastructure — Databricks automatically provisions and scales compute.
• Lakeflow Declarative Pipelines. Serverless pipelines eliminate manual cluster configuration and use enhanced autoscaling that scales both horizontally and vertically in response to demand. Serverless is also required for incremental refresh of materialized views, which lowers refresh cost.
• SQL analytics and BI. Serverless SQL warehouses are optimized for SQL-based analytics and reporting and perform well for ETL, business intelligence, and exploratory analysis, with startup in about 2 to 6 seconds and intelligent autoscaling.
• Model Serving. Model Serving provides unified deployment and governance of AI models with automatic scaling on serverless infrastructure.
• Streaming. Streaming workloads are supported on serverless compute (with some limitations).
• Reliability at scale. Serverless has achieved a 99.998% success rate across billions of workloads, according to Databricks.

Getting started

• Read what is serverless compute? to understand the model and its coverage.
• Use the guidance on choosing compute for interactive notebooks to move notebooks to serverless.
• Follow the serverless architectures best practices to plan jobs, pipelines, and model serving.
• See how serverless compute is transforming notebooks and Lakeflow Jobs.

FAQs

When should I choose serverless?

Choose serverless when you want instant startup, automatic scaling, and no clusters to configure — typical of interactive notebooks, jobs, pipelines, SQL analytics, and model serving.

Which workloads can run serverless?

Serverless compute is available for SQL warehouses, jobs, notebooks, Lakeflow Declarative Pipelines, and Model Serving.

How fast do serverless SQL warehouses start?

Serverless SQL warehouses typically start in about 2 to 6 seconds.

Is serverless reliable at scale?

Yes. Databricks reports a 99.998% success rate for serverless across billions of workloads.