Does Spark only do micro batch? Does it handle streaming as well?

Summary

• No. Spark handles streaming with Structured Streaming. It runs micro-batches by default and also offers Continuous Processing (experimental) and, since Spark 4.1.0, Real-Time Mode.
• You use the same Dataset/DataFrame operations as in batch queries and pick the mode with a trigger.
• Micro-batch has exactly-once guarantees. Continuous Processing has at-least-once guarantees and supports only map-like operations.
• Databricks doesn't support Continuous Processing and points to real-time mode for low latency.
• Serverless compute supports only Trigger.AvailableNow() and Trigger.Once().

Does Spark only do micro batch? Does it handle streaming as well?

No. Spark handles streaming with Structured Streaming, "a scalable and fault-tolerant stream processing engine built on the Spark SQL engine". You express a streaming computation "the same way you would express a batch computation on static data", using the Dataset/DataFrame API in Scala, Java, Python or R.
Micro-batch is the default engine, but Spark also has Continuous Processing, experimental since Spark 2.3, and Real-Time Mode, which got its first official support in Spark 4.1.0. The Spark docs add: "Without changing the Dataset/DataFrame operations in your queries, you will be able to choose the mode based on your application requirements."

How does Structured Streaming run by default?

The Spark docs say that "by default, Structured Streaming queries are processed using a micro-batch processing engine, which processes data streams as a series of small batch jobs", with "exactly-once fault-tolerance guarantees". If you set no trigger, "micro-batches will be generated as soon as the previous micro-batch has completed processing". A fixed schedule comes from a processing-time trigger, where micro-batches "will be kicked off at the user-specified intervals".

Which processing modes does Spark offer?

Mode	How it works	Status and guarantee
Micro-batch	Processes the stream as a series of small batch jobs	Default engine. Exactly-once fault-tolerance guarantees
Continuous Processing	Long-running tasks continuously read data from sources, process it and write to sinks	Experimental since Spark 2.3. At-least-once guarantees
Real-Time Mode (RTM)	Queries run in real-time mode for continuous, sub-second latency processing	First official support in Spark 4.1.0 (Scala, stateless). PySpark trigger support in Spark 4.2.0

Continuous Processing is narrow. The Spark docs list only map-like operations as supported: projections such as select, map and flatMap, and selections such as where and filter. Aggregation functions are not supported. The documented sources are Kafka and Rate, and the sinks are Kafka, Memory and Console. The docs also say: "There are currently no automatic retries of failed tasks."

What is Real-Time Mode?

The Spark 4.1.0 release notes describe "First official support for Structured Streaming queries running in real-time mode for continuous, sub-second latency processing." That release covers Scala stateless queries (SPARK-53736). The Spark 4.2.0 notes add "Real-time Mode (RTM) trigger support to PySpark" (SPARK-54660).
On Databricks, "Real-time mode is a trigger type for Structured Streaming that enables ultra-low latency data processing", and Databricks announced general availability on March 19, 2026. It runs on classic compute (dedicated access mode, or standard access mode for Python only) with Databricks Runtime 16.4 LTS and above. Serverless compute is not supported, and real-time triggers support update output mode only. Setup also requires setting spark.databricks.streaming.realTimeMode.enabled to true and turning off autoscaling, Photon and spot instances.

query = (
    spark.readStream
        .format("kafka")
        .option("kafka.bootstrap.servers", broker_address)
        .option("subscribe", input_topic)
        .load()
        .writeStream
        .format("kafka")
        .option("kafka.bootstrap.servers", broker_address)
        .option("topic", output_topic)
        .option("checkpointLocation", checkpoint_location)
        .outputMode("update")
        .trigger(realTime="5 minutes")
        .start()
)

Which trigger should I use on Databricks?

Trigger	What it does	On Databricks
Unspecified (default)	Starts each micro-batch as soon as the previous one finishes	General-purpose streaming. Not supported on serverless compute
processingTime	Starts micro-batches at an interval you set	Not supported on serverless compute
availableNow	Processes all available data, possibly in several micro-batches, then stops	Scheduled incremental batch processing. Recommended on serverless compute
Once	Processes all available data in one micro-batch, then stops	Deprecated since Databricks Runtime 11.3 LTS. Use availableNow
continuous	Continuous Processing	Databricks doesn't support or recommend it. Use real-time mode
realTime	Real-time mode	Classic compute, Databricks Runtime 16.4 LTS and above

How do I run streaming workloads on Databricks?

Lakeflow Jobs can start a run when "source tables are updated" (the table update trigger), and its continuous trigger keeps a job "always running" by starting another run "whenever a job run completes or fails". Lakeflow Pipelines runs in triggered or continuous mode. A triggered pipeline "stops after refreshing all tables based on the data available when the update started", and a continuous pipeline "processes new data as it arrives in data sources".
On serverless compute, the Databricks docs say to run continuous streaming workloads with a pipeline in continuous mode, or with Trigger.AvailableNow() in a job that runs continuously. A query with an unsupported trigger raises the error INFINITE_STREAMING_TRIGGER_NOT_SUPPORTED.

When should I use real-time mode or micro-batch?

• Real-time mode: sub-second latency, such as blocking or flagging a credit card transaction when a fraud score exceeds a threshold, and operational decision-making, where systems trigger immediate actions based on incoming data.
• Micro-batch (the default trigger): analytical processing such as ETL pipelines, data transformations and medallion architecture implementations, plus cost-sensitive workloads, because "real-time mode requires dedicated compute resources".
• availableNow: scheduled incremental batch processing.

FAQs

Is Spark Structured Streaming real time?

By default it runs micro-batches. Real-Time Mode, supported from Spark 4.1.0 and generally available on Databricks, is the mode built for continuous, sub-second latency processing.

Can I use Continuous Processing on Databricks?

No. The docs say: "Databricks doesn't support or recommend this mode. Use real-time mode instead for low-latency use cases."

Should I still use Trigger.Once?

No. Spark says the trigger "is deprecated and users are encouraged to migrate to Available-now micro-batch". Databricks says to use Trigger.AvailableNow for all incremental batch processing workloads.