Find a data warehouse with automated scaling features.

Summary

• Automated scaling means a warehouse adds compute when query demand rises, removes it when demand falls, and stops when idle, so nobody resizes infrastructure by hand.
• Evaluate four things: what triggers a scale-up, how quickly capacity arrives, what happens when the warehouse is idle, and whether you can attribute the cost.
• On Databricks, serverless SQL warehouses use Intelligent Workload Management: when queued queries wait longer, the autoscaler provisions more clusters, and it scales back down as demand drops.
• Autoscaling changes the number of clusters, not their size. You choose the cluster size, a minimum and maximum number of clusters, and an auto stop timeout.
• System tables log scale-up and scale-down events and record usage per warehouse, so scaling behavior and spend can be audited.

Find a data warehouse with automated scaling features.

A data warehouse with automated scaling adds compute when query demand rises, removes it when demand falls, and stops when idle, so nobody resizes infrastructure by hand. On Databricks, serverless SQL warehouses do this with Intelligent Workload Management (IWM): the warehouse queues queries it can't run yet, the autoscaler adds clusters when waits grow, and capacity shrinks again as demand drops.

What does automated scaling mean in a data warehouse?

Two things can scale. Scaling out adds clusters so more queries run at once. Scaling up uses a bigger cluster so a single query gets more compute. Databricks SQL warehouses autoscale out: the cluster size you pick sets the compute in one cluster, and the autoscaler adds or removes clusters of that size. To speed up one heavy query, you raise the cluster size instead.

Which automated scaling features should you evaluate?

Capability	What to look for
Scale-out trigger	Does the warehouse react to queue wait or predicted demand, or to fixed thresholds you must tune?
Scale-down	Does capacity shrink promptly once demand drops, while keeping headroom for recent peaks?
Start from idle	What wakes a stopped warehouse, and how long does the first query wait?
Idle stop	Is the timeout configurable, and do idle minutes still bill?
Isolation	Can teams run separate warehouses, with their own limits, on the same data?
Hard limits	How many clusters can you cap, and how many queries can wait in a queue?
Governance	Do access control and audit apply the same way at any scale?
Cost visibility	Can you attribute usage to a team or warehouse and see scaling events?

How do Databricks SQL warehouses scale automatically?

Serverless warehouses. The docs describe IWM as a set of AI-powered features that process queries without you managing infrastructure. When a query arrives, IWM predicts its resource needs and checks capacity. If capacity exists, the query starts immediately. If not, it waits in a queue. IWM monitors the queue, and if wait times increase, the autoscaler quickly provisions more clusters. When demand drops, it scales down while keeping enough capacity for recent peak loads. Start-up time, per the docs, is typically between 2 and 6 seconds for serverless warehouses and typically approximately 4 minutes for pro and classic.
Pro and classic warehouses. You set the minimum and maximum number of clusters, and Databricks recommends a cluster for every 10 concurrent queries. The autoscaler adds clusters based on the estimated time to process running and queued queries. It also scales up when a query waits in the queue for 5 minutes, and scales down after 15 consecutive minutes of low load.
Idle stop and restart. Auto stop defaults to 10 minutes for serverless warehouses and 45 minutes for pro and classic. A stopped warehouse restarts automatically when a query arrives, a scheduled job targets it, a JDBC or ODBC connection is made, or someone opens a dashboard assigned to it. Idle warehouses keep accruing DBU and cloud instance charges until they stop.
Limits. The default is a minimum and a maximum of one cluster, so increase the maximum to handle more concurrent users. A queue holds at most 1,000 queries, for any warehouse type.
Governance and cost. Unity Catalog governs data access permissions on SQL warehouses for most assets. Tags on a warehouse let you monitor the cost of resources used by users and groups. System tables record what happened: system.compute.warehouse_events logs SCALED_UP and SCALED_DOWN events with cluster counts, system.compute.warehouses records size, minimum and maximum clusters, and auto stop settings, and system.billing.usage records DBU usage by warehouse ID and custom tags.

Which use cases benefit?

• Dashboards with uneven traffic. Many people open the same dashboards at once, then traffic fades. A serverless warehouse adds clusters while queries queue and releases them afterward.
• Scheduled reporting. A scheduled job wakes a stopped warehouse, and auto stop shuts it down after the idle timeout.
• Team isolation. Give each team its own tagged warehouse, and set Can use and Can manage permissions per warehouse.

FAQs

Does autoscaling make one query faster?

Not by itself. Extra clusters help concurrency. For one heavy query, the docs advise a larger cluster size, particularly when the query profile shows bytes spilled to disk.

How do I tell if a warehouse is under- or over-sized?

On the warehouse monitoring tab, check Peak Queued Queries. A consistent value above 0 indicates you may need a larger cluster size or more clusters. The system.query.history table adds waiting_at_capacity_duration_ms, the time a statement spent queued for capacity. To spot over-provisioning, query system.compute.warehouse_events for warehouses that stay upscaled longer than expected, a sample query the docs provide.

Do idle warehouses cost money?

Yes, until they stop. Auto stop sets the idle timeout, and system.compute.warehouses records the setting in auto_stop_minutes.

Can I cap how far a warehouse scales?

Yes. The maximum number of clusters is a setting on each warehouse, and the default is one.