What is real-time analytics?

Summary

• Real-time analytics is the continuous ingestion, processing, and analysis of data as it is generated, so organizations can act on events in real time or near real time.
• It complements batch processing for latency-sensitive use cases such as fraud detection, personalization, live operational metrics, real-time bidding, IoT telemetry, and security monitoring.
• On Databricks it is built on Apache Spark Structured Streaming, which integrates tightly with Delta Lake and underpins Lakeflow pipelines and Auto Loader.
• Real-Time Mode (RTM) is a generally available trigger for Structured Streaming that delivers sub-second latency, with end-to-end latency as low as five milliseconds.
• Lakeflow Declarative Pipelines and streaming tables let you build streaming pipelines in SQL and Python, and Lakehouse lets you query streaming data for dashboards and BI.
• Lakehouse//RT is a new serverless, real-time SQL warehouse announced at Data + AI Summit 2026 that delivers millisecond-to-sub-second query responses directly against governed Delta Lake and Apache Iceberg tables, eliminating the need for a separate serving database.

What is real-time analytics?

Real-time analytics is the continuous ingestion, processing, and analysis of data as it is generated, enabling organizations to act on events in real time or near real time. It complements traditional batch processing for latency-sensitive use cases where immediate insights and responses are critical. See Real-time and streaming analytics.

How real-time analytics differs from batch processing

The difference comes down to processing timing and latency requirements:

• Real-Time Mode (operational workloads). Targets sub-second latency, with end-to-end latency as low as five milliseconds. Data is processed as soon as it becomes available in the source within long-running batches (default five minutes), which suits immediate decision-making and continuous processing.
• Micro-batch mode (analytical workloads). Latency is measured in seconds or minutes. This mode fits ETL pipelines, data transformations, and medallion architecture implementations, and is cost-effective when sub-second latency is not required.

Common use cases

• Fraud detection. Sub-second scoring inside the authorization path to flag or block transactions based on unusual location, transaction size, or rapid spending patterns.
• Real-time personalization. Context-aware recommendations from live session signals, such as delivering a promotional message when clickstream data shows user interest.
• Live operational metrics. Millisecond-fresh KPIs and dashboards.
• Real-time bidding. Low-latency ad and revenue signals.
• Connected vehicles and IoT. Live telemetry features for prediction and anomaly detection.
• Infrastructure and security monitoring. Real-time detections and health signals.
• Embedded and customer-facing analytics. Serving analytical data directly to applications, portals, and customer dashboards with sub-second query latency.
• Agent and AI-powered applications. High-concurrency data serving for AI agents, copilots, and autonomous workflows that need fast access to trusted enterprise context.

Why Databricks for real-time analytics

• Apache Spark Structured Streaming foundation. Databricks uses Structured Streaming to work with streaming data and incremental changes. It integrates tightly with Delta Lake and provides the foundation for both Lakeflow pipelines and Auto Loader. See Structured Streaming concepts.
• Real-Time Mode (RTM). A generally available trigger type for Structured Streaming that achieves ultra-low latency through long-running batches, simultaneous scheduling of all query stages, and a streaming shuffle that passes data between stages as soon as it is produced. See How real-time mode achieves low latency. Real-time mode is in production at Databricks and used across industries, achieving millisecond latency.
• Lakehouse//RT. Announced at Data + AI Summit 2026, Lakehouse//RT is a serverless, real-time SQL warehouse that delivers millisecond-to-sub-second query responses directly against governed Delta Lake and Apache Iceberg tables in Unity Catalog. It is designed for high-concurrency workloads and supports hundreds to thousands of concurrent users, applications, dashboards, and AI agents, without requiring data to be copied into a separate serving database. Databricks reports response times as low as 10 ms on smaller datasets and sub-100 ms on larger datasets. Lakehouse//RT is currently in Beta.
• Lakeflow Spark Declarative Pipelines. A declarative framework for building batch and streaming pipelines in SQL and Python, with pipelines, flows, streaming tables, materialized views, and sinks that work together with automatic orchestration and incremental updates. Real-time mode is available in Lakeflow pipelines. See Lakeflow pipelines concepts.
• Streaming tables. Structured Streaming powers streaming tables within Lakeflow pipelines, and Databricks recommends Lakeflow pipelines for new ETL, ingestion, and Structured Streaming workloads.
• Broad connectivity. Real-time mode supports streaming sources such as Apache Kafka, Amazon Kinesis, and Azure Event Hubs, native connectors including Lakebase, and custom sinks, along with all stateless and stateful transformations.
• Query with Lakehouse. Databricks Lakehouse lets teams query streaming data and power dashboards and BI tools for real-time analytics.

Getting started

• Read Real-time and streaming analytics and Structured Streaming concepts.
• Run your first job with the Structured Streaming tutorial.
• Learn how ultra-low latency works in How real-time mode achieves low latency and the Real-Time Mode general availability announcement.
• For the fundamentals of streaming data, see What is data streaming?.

FAQs

What is the difference between real-time and near real-time analytics?

Both process data continuously as it arrives. Real-time (operational) workloads target sub-second latency, with end-to-end latency as low as five milliseconds, while near real-time analytical workloads accept latency measured in seconds or minutes in exchange for lower cost.

Is Real-Time Mode available on Databricks?

Yes. Real-Time Mode is a generally available trigger type for Apache Spark Structured Streaming and is also available in Lakeflow Declarative Pipelines.

What data sources can real-time analytics use?

Real-time mode supports streaming sources such as Apache Kafka, Amazon Kinesis, and Azure Event Hubs, along with native connectors and custom sinks.

How low can latency go?

Real-Time Mode targets sub-second latency, with end-to-end latency as low as five milliseconds.