What is a trigger, and what types exist?

Summary

• A trigger is the timing setting of a streaming query. It decides when Structured Streaming runs the next micro-batch, or whether the query runs in continuous mode.
• The Spark guide lists unspecified (default), fixed-interval micro-batches, one-time micro-batch (deprecated), available-now micro-batch and continuous (experimental). Spark 4.1.0 added real-time mode, and PySpark support arrived in 4.2.0.
• The default starts each micro-batch as soon as the previous one finishes. A fixed interval waits out the interval unless the batch overran. Available-now processes the available data in one or more micro-batches, then stops.
• On Databricks, the continuous trigger is not supported, and Lakeflow Pipelines sets timing through triggered or continuous pipeline mode.

What is a trigger, and what types exist?

The Spark guide says the trigger settings of a streaming query "define the timing of streaming data processing, whether the query is going to be executed as micro-batch query with a fixed batch interval or as a continuous processing query". You set one with trigger(...) on writeStream. In PySpark, "Only one trigger can be set" per call.

Which trigger types exist?

Trigger	PySpark argument	What it does
Unspecified (default)	none	Generates the next micro-batch as soon as the previous one has completed processing
Fixed interval micro-batches	processingTime='2 seconds'	Kicks off micro-batches at the interval you set
One-time micro-batch (deprecated)	once=True	Processes all available data in one micro-batch, then stops
Available-now micro-batch	availableNow=True	Processes all available data in possibly several micro-batches, then stops
Continuous (experimental)	continuous='1 second'	Runs the low-latency continuous processing mode with a checkpoint interval
Real-time mode	realTime='5 seconds'	Runs a real-time mode query with batches of the given duration

The first five rows are the Spark guide's list. Real-time mode is newer and is covered below.

How do the micro-batch triggers behave?

Default. With no trigger set, PySpark's docs say the query runs "as fast as possible", equivalent to processingTime='0 seconds'.
Fixed interval. If the previous micro-batch finishes within the interval, "the engine will wait until the interval is over". If it takes longer, "the next micro-batch will start as soon as the previous one completes". If no new data is available, no micro-batch starts.
One-time versus available-now. Spark deprecated Trigger.Once in 3.4.0 and points to available-now, added in 3.3.0, which "provides the better guarantee of processing, fine-grained scale of batches, and better gradual processing of watermark advancement including no-data batch". Available-now splits the work into micro-batches using source options such as maxFilesPerTrigger or maxBytesPerTrigger. It processes all data available at execution time, uncommitted batches first. If any source does not support it, Spark falls back to a one-time micro-batch.

# Default trigger
df.writeStream.format("console").start()

# Fixed interval
df.writeStream.format("console").trigger(processingTime='2 seconds').start()

# Available-now
df.writeStream.format("console").trigger(availableNow=True).start()

What are continuous and real-time mode?

The continuous trigger is "a new, experimental streaming execution mode introduced in Spark 2.3" with at-least-once guarantees, where micro-batch mode offers exactly-once. The docs list only map-like operations (projections and selections), Kafka or rate sources, and Kafka, memory or console sinks, and note there are no automatic retries of failed tasks.
Real-time mode (RTM) arrived in Apache Spark 4.1.0. The release notes call it the "First official support for Structured Streaming queries running in real-time mode for continuous, sub-second latency processing", initially for Scala stateless queries. The Scala and Java API is Trigger.RealTime(...), with a default batch duration of 5 minutes, and Spark 4.2.0 added the trigger to PySpark (SPARK-54660). The Structured Streaming guide's trigger table does not list it yet, so check the API reference for current limits.

How do triggers work on Databricks?

Databricks documents the unspecified, processingTime, availableNow, real-time and continuous triggers. The unspecified trigger is equivalent to a processingTime trigger with a zero interval, and the examples are .trigger(processingTime='10 seconds'), .trigger(availableNow=True) and .trigger(realTime='5 minutes'). The continuous trigger is listed as "Not supported", with a pointer to real-time mode instead. On serverless compute, "only Trigger.AvailableNow() and Trigger.Once() are supported". For low-latency operational streaming, the production guide recommends a processingTime trigger with a 0 interval and advises against AvailableNow, Once and Continuous for operational workloads. A Databricks blog post dated May 19, 2026 says real-time mode, an evolution of the Spark Structured Streaming engine, is generally available on AWS, Azure and GCP.
Lakeflow Pipelines exposes timing through pipeline mode. Triggered mode "stops after refreshing all tables based on the data available when the update started", and continuous mode "processes new data as it arrives in data sources". The pipelines.trigger.interval property controls the trigger interval for a flow or an entire pipeline, applies only to continuous pipelines, and defaults to five seconds for streaming queries.

FAQs

Can one application use different triggers?

Yes, through separate queries. Each writeStream carries its own trigger, and "you can start any number of queries in a single SparkSession", running concurrently and sharing the cluster resources.

Is Trigger.Once still available?

It is deprecated since Spark 3.4.0 in favor of AvailableNow(). Databricks lists it among the two triggers supported on serverless compute.

Does available-now matter for stateful queries?

Yes. The guide says the watermark "gets advanced per each batch" and a no-data batch runs before termination if the last batch advances the watermark, which helps keep state size smaller and predictable.