How easy is it to scale up and scale down with serverless?

Summary

• With serverless compute on the Databricks Data Intelligence Platform, scaling is automatic: you don't add or remove machines or tune autoscaling rules, Databricks does it for you as demand changes.
• Intelligent Workload Management predicts resource needs and scales clusters up as work arrives and back down when demand falls, including scaling inactive compute to zero so you stop paying for idle capacity.
• Compute starts in seconds because Databricks keeps warm pools of pre-initialized instances ready, so there is no cluster startup wait when you scale up.
• Serverless is built to absorb traffic spikes without over-provisioning, and it automatically recovers from out-of-memory errors by restarting work on larger instances with no manual intervention.
• You choose a performance goal instead of instance types or Spark settings, and you pay only for the compute a workload actually consumes.

How easy is it to scale up and scale down with serverless?

With serverless compute, scaling is essentially hands-off. On the Databricks Data Intelligence Platform, you don't provision machines, pick instance types, or write autoscaling rules; Databricks manages capacity for you and adjusts it continuously as your workloads change. Compute scales up when work arrives and back down, all the way to zero, when it doesn't, so you get elasticity without operational effort and pay only for what you use.

Why serverless on the Databricks Platform makes scaling easy

• Automatic scale up and scale down. Serverless uses Intelligent Workload Management, an AI-powered system that predicts resource requirements as queries and jobs arrive. It provisions additional capacity as demand rises and scales resources back down as demand falls, with no manual intervention. For SQL, it scales both horizontally (adding clusters) and vertically (larger clusters) based on queue depth.
• Scale-to-zero. When compute is inactive it scales down to zero and suspends automatically, so idle capacity is not left running and you are not billed for it. See What is serverless computing?.
• Instant startup with warm pools. Databricks maintains warm pools of pre-initialized instances, so serverless compute starts in seconds rather than minutes, removing the traditional cluster startup wait when you scale up. serverless SQL warehouses typically start in about 2 to 6 seconds.
• Built to absorb traffic spikes. The intelligent autoscaler continuously analyzes workload patterns and system signals rather than relying on static rules, so it adds capacity for spikes without over-provisioning and releases it afterward. If a task hits an out-of-memory error, the autoscaler detects it, restarts the task on a larger instance, and continues the job automatically.
• No manual configuration. You don't select instance types, set autoscaling parameters, or tune Spark configurations. You choose a performance goal, such as performance-optimized (fastest startup) or standard (cost-optimized), and Databricks fine-tunes the underlying infrastructure to match. See Use serverless architectures.
• Pay only for what you use. Because compute is provisioned precisely when needed and scales to zero when idle, you pay only for the compute a workload actually consumes, with no idle cluster costs. Most customers see a 25% or greater reduction in serverless compute spend.

Getting started

• Turn on serverless SQL warehouses for interactive analytics and BI with near-instant startup and automatic scaling.
• Move jobs, notebooks, and pipelines to serverless so they scale without cluster-creation or tuning work. See Announcing GA of serverless compute for notebooks, workflows, and Lakeflow Declarative Pipelines.
• Review Use serverless architectures for platform best practices.

FAQs

Do I have to configure autoscaling with serverless?

No. Serverless compute scales automatically based on demand. You do not choose instance types, set autoscaling parameters, or tune Spark configurations; you pick a performance goal and Databricks manages capacity for you.

Does serverless scale down when I'm not using it?

Yes. Inactive serverless compute scales to zero and suspends automatically, so you are not billed for idle capacity and do not need to shut anything down manually.

How does serverless handle sudden traffic spikes?

Intelligent Workload Management continuously analyzes workload patterns and provisions additional capacity as demand rises, then releases it as demand falls, so spikes are absorbed without over-provisioning.

How fast does serverless compute start when it scales up?

Serverless compute starts in seconds because Databricks keeps warm pools of pre-initialized instances. Serverless SQL warehouses typically start in about 2 to 6 seconds.