Are there limits on execution time or memory?

Summary

• Databricks compute is elastic: serverless compute autoscales to match workload demand and scales down when idle, so you are not fixed to a single memory or execution-time envelope.
• Execution time: serverless compute has a maximum run duration of 7 days; runs that exceed it are stopped, and longer workloads can be split into smaller runs or moved to classic compute.
• Configurable timeouts: you can set an execution timeout for job queries with the spark.databricks.execution.timeout property, and set job-level timeouts on your workloads.
• Memory: serverless intelligent autoscaling detects out-of-memory conditions and automatically retries tasks on larger VMs; on classic compute you choose instance types and worker counts to size memory.
• Documented limits for serverless compute are published in the Databricks serverless compute limitations reference.

Are there limits on execution time or memory?

Databricks compute is elastic, so execution time and memory are largely governed by how you configure and scale compute rather than by fixed hardware you provision up front. There are a few documented platform limits — most notably a maximum run duration for serverless compute — but memory scales with your workload and execution timeouts are configurable. The authoritative, current values are published in the serverless compute limitations reference.

How Databricks Platform handles execution time and memory

• Serverless maximum runtime. Serverless compute has a maximum run duration of 7 days. Runs that exceed 7 days are terminated by the platform and are not retried. To run longer, break the work into smaller runs or use classic compute.
• Configurable execution timeouts. By default, serverless jobs have no query execution timeout, but you can set one using the spark.databricks.execution.timeout property. You can also configure timeouts at the job level so a workload is stopped if it runs longer than expected.
• Elastic memory with autoscaling. Serverless compute uses intelligent autoscaling that continuously analyzes workload patterns, automatically detects out-of-memory errors, and restarts affected tasks on larger VMs without manual intervention. Resources scale down when demand decreases, so idle capacity is not held.
• Sizing memory on classic compute. On classic compute you size memory directly by choosing instance types (vertical scaling) and the number of workers (horizontal scaling), following the performance efficiency best practices for scaling.
• Check the documented limits. Specific, current limits — including serverless job limitations — are maintained in the serverless compute limitations documentation, which is the source of truth as the platform evolves.

Getting started

• Review the serverless compute limitations reference for current execution-time and job limits.
• Read What is serverless computing? to understand how autoscaling handles memory and demand.
• Apply the performance efficiency best practices to size compute for memory-intensive workloads.

FAQs

Is there a maximum runtime for a Databricks serverless job?

Yes. Serverless compute has a maximum run duration of 7 days; runs that exceed it are terminated and not retried. Longer workloads can be split into smaller runs or run on classic compute.

Can I set a timeout on a query or job?

Yes. You can set an execution timeout for job queries with the spark.databricks.execution.timeout property and configure timeouts at the job level.

What happens if a task runs out of memory?

On serverless compute, intelligent autoscaling detects out-of-memory errors and automatically restarts the affected tasks on larger VMs. On classic compute, you size memory by selecting instance types and worker counts.