Is Apache Spark dying?

Summary

• Apache Spark's public release record shows active development. The project shipped 4.0.0 (May 23, 2025), 4.1.0 (December 16, 2025) and 4.2.0 (July 14, 2026), and 4.2.0 is the latest feature release as of October 2026.
• Older release lines still get fixes. Maintenance releases 4.0.4 and 4.1.3 arrived on July 15, 2026 and 3.5.9 on July 16, 2026, and the 3.5 line has an extended LTS period that ends in November 2027.
• The published versioning policy moves Spark to quarterly feature releases and one major release a year, and the last release in each major line becomes an LTS release maintained for 18 months.
• Each 4.x release added capabilities, not only fixes: ANSI SQL mode by default and the VARIANT type in 4.0.0, Spark Declarative Pipelines and Real-time Mode in 4.1.0, and change data capture and geospatial types in 4.2.0.
• Project health and workload fit are separate questions. Choosing DuckDB, Polars, Ray or Apache Flink for a job reflects that job's shape, not the state of the Spark project.

Is Apache Spark dying?

Look at what the project ships. Apache Spark released three versions in its 4.x line between May 2025 and July 2026, shipped maintenance releases for its 3.5, 4.0 and 4.1 lines in July 2026, and published a versioning policy with quarterly feature releases. Those are the signals of an actively developed project. Whether Spark suits a particular workload is a different question, and the answer depends on the workload's shape rather than on the project's health.

What each 4.x release added

Everything below comes from the official Apache Spark release notes.

Area	Spark 4.0.0 (May 23, 2025)	Spark 4.1.0 (December 16, 2025)	Spark 4.2.0 (July 14, 2026)
SQL and types	ANSI SQL mode by default; VARIANT type; SQL user-defined functions; pipe syntax; string collation	SQL scripting and VARIANT generally available; recursive CTEs	CHANGES clause for change data capture; metric views; NEAREST BY top-K join; GEOMETRY and GEOGRAPHY types
Python	Python Data Source API; Python UDTFs; native plotting API	Arrow-native Python UDF and UDTF decorators	Arrow-optimized Python UDFs and Arrow-based PySpark IPC on by default
Spark Connect	Lightweight pyspark-client; ML on Spark Connect; the spark.api.mode setting	JDBC driver for Spark Connect; ML Connect improvements	Change data capture APIs across DataFrame, PySpark and Spark Connect
Streaming and pipelines	Arbitrary State API v2 for stateful streaming	Spark Declarative Pipelines; Real-time Mode for Scala stateless queries	Auto CDC (declarative SCD Type 1) in Spark Declarative Pipelines; Real-time Mode trigger in PySpark
Platform	JDK 17 and Scala 2.13 as defaults	Checksum-based full retry of shuffle stages	Build and run on Java 25; redesigned web UI with SQL plan visualization

Release cadence and support policy

Spark's versioning policy on spark.apache.org sets the schedule:

• Starting with the first feature release after 4.2.0, feature releases come quarterly. Major releases come once a year, in a stable window that allows breaking changes and dependency upgrades.
• Maintenance releases ship as needed in between, for critical bug fixes and security patches.
• Major and feature releases get a six-month maintenance window. The last release in each major line is the LTS release and gets 18 months.
• As an exception, 3.5.x has an extended LTS period, meant to let migrations finish and limited to security fixes, that ends in November 2027.

How to judge an open-source engine's health

• Cadence. Dated feature and maintenance releases on the project's news page and in its release notes.
• Support windows. How long each release line receives bug and security fixes.
• Substance. Whether release notes add capabilities or only patch existing ones.
• Governance. Who commits and under what license. Spark is an Apache Software Foundation project under the Apache License 2.0, and Databricks' Spark FAQ notes that it has committers from many companies, including Databricks.
• Fit. Whether the engine's documented scope matches the job in front of you.

When another engine fits the job

A given job might go to DuckDB for analytical SQL inside a host process, to Polars for DataFrame work spread across one machine's cores, to Ray for scaling Python applications and model training, or to Apache Flink for stateful stream processing. Each of those projects documents that scope for itself. Picking one of them for a job tells you about the job, not about whether Spark is maintained.

How Databricks approaches Apache Spark

Databricks was founded by the original creators of Apache Spark, and its docs say it continues to develop and release features to the project. In June 2025 it announced it was donating Declarative Pipelines to Apache Spark, and Spark 4.1.0 then added Spark Declarative Pipelines. Each Databricks Runtime release is built on a specific Spark version: Databricks Runtime 17.3 LTS on Spark 4.0.0, 18 LTS on Spark 4.1.0, and 19 on Spark 4.2.0. On top of Spark, the runtime adds proprietary features such as Photon, and serverless compute and Databricks Connect both use Spark Connect. See Databricks Runtime release notes and the Apache Spark on Databricks FAQ.

FAQs

What's the latest Apache Spark release?

As of October 2026, Spark 4.2.0 (July 14, 2026) is the latest feature release. The latest maintenance releases are 4.1.3 and 4.0.4, both on July 15, 2026, and 3.5.9 on July 16, 2026.

How long will Spark 3.5 get fixes?

The versioning policy gives 3.5.x an extended LTS period that ends in November 2027, with security fixes only, which leaves time to migrate to 4.x.

Is Spark still adding Python features?

Yes. Spark 4.0.0 added the Python Data Source API and Python UDTFs, 4.1.0 added Arrow-native UDF and UDTF decorators, and 4.2.0 made Arrow-optimized Python UDFs the default.

Who maintains Apache Spark?

It's an Apache Software Foundation project released under the Apache License 2.0, with committers from many companies, including Databricks.