What is Apache Hudi?

Summary

• Apache Hudi is an open-source data lakehouse platform built on an open table format: it brings tables, transactions, upserts and deletes, and indexes to files on a data lake.
• Every change is recorded as an action on the table's timeline, which drives snapshot, time travel and incremental queries.
• Copy-on-write tables rewrite base files when records change; merge-on-read tables write changes to log files and merge them at query time or through compaction.
• Hudi integrates with Apache Spark and Apache Flink; the 1.2.x line supports Spark 3.3.x through 4.1.x.
• It has been an Apache Software Foundation Top-Level Project since 2020.

What is Apache Hudi?

Apache Hudi (pronounced "hoodie") is an open-source data lakehouse platform built on an open table format. It "brings core database functionality directly to a data lake": tables, transactions, efficient upserts and deletes, advanced indexes, ingestion services, clustering and compaction, and concurrency control, "all while keeping your data in open file formats." Hudi is aimed at data that changes after it lands: records you update, delete or receive as change data capture, which downstream jobs then read incrementally.

What does Hudi add to a data lake?

Concept	What it is	Why it matters
Timeline	"a log of all actions performed on the table at different instants (points in time)"	It's the "source of truth for the state of the table" that queries read
Transactions	"Atomic writes, with relational/streaming data consistency models, snapshot isolation and non-blocking concurrency controls"	Readers and writers get snapshot isolation on a data lake
Upserts and deletes	Record-level updates and deletes by key	Apply changes by record key
Indexes	"a mapping from a given key (record key + optionally partition path) consistently to a file id"	Upserts avoid "the need to join against the entire dataset to determine which files to rewrite"
Table services	Clustering, compaction, cleaning, file sizing and indexing that Hudi schedules	File layout stays maintained as data changes
Time travel	Query "a snapshot of a table as of a given instant in the past"	Debug earlier versions or roll back to one
Incremental queries	"only return new data written to the table since an instant on the timeline"	Downstream jobs process changes, not the whole table

How do copy-on-write and merge-on-read differ?

Hudi has two table types, chosen when you create the table. In Spark SQL that's type = 'cow' (the default) or type = 'mor'.

Aspect	Copy-on-write	Merge-on-read
Updates and deletes	New base files in the file group; "there are no log files written"	Written "to log files" and "merged dynamically with base files during query execution"
Write latency	Higher	Lower
Query latency	Lower	Higher
Update cost	Rewrites entire base files	Appends to the delta log
Read-optimized queries	Not applicable	Read only the columnar base files
Typical workloads (per the docs)	Batch ETL, data warehousing on data lakes, slowly changing data	CDC pipelines, streaming ingestion, frequent updates and deletes

Compaction later reconciles a merge-on-read table's log files with its base files.

How do you query and write Hudi tables?

Hudi supports snapshot queries, which "see the latest snapshot of the table as of the latest completed action"; time travel queries; incremental queries; CDC queries, which "provide database like change data capture streams"; and, on merge-on-read tables only, read-optimized queries, which can miss the newest log data that a snapshot query sees.
In Spark SQL, after you add the Hudi Spark bundle and the HoodieSparkSessionExtension and HoodieCatalog settings from the quick start:

CREATE TABLE trips (
  ts BIGINT, uuid STRING, rider STRING, fare DOUBLE, city STRING
) USING HUDI
TBLPROPERTIES (type = 'mor', primaryKey = 'uuid', orderingFields = 'ts')
PARTITIONED BY (city);

UPDATE trips SET fare = 25.0 WHERE rider = 'rider-D';

-- Time travel to an instant, then read changes since the earliest commit
SELECT * FROM trips TIMESTAMP AS OF '20220307091628793';
SELECT * FROM hudi_table_changes('db.trips', 'latest_state', 'earliest');

Apache Flink can also write and read Hudi tables through the 'connector' = 'hudi' table option, including streaming reads of records "that changed since a given commit timestamp."

How does Hudi relate to Delta Lake and Apache Iceberg?

Delta Lake and Apache Iceberg are also open-source table formats for data lakes, and each documents its own transaction model, update path and engine support. Compare them on your workload, such as how often records change and which engines must read and write the tables, using each project's own documentation.

How Databricks approaches open table formats

On Databricks, Delta Lake is "the default format for reading and writing data and tables," and you can "read and write Iceberg tables and interoperate with external Iceberg engines." Iceberg reads (UniForm), in Databricks Runtime 14.3 LTS and above, "configure Delta Lake tables to automatically generate Iceberg metadata, so Iceberg clients can read Delta Lake data without rewriting files." Hudi isn't among the open table formats listed on the Databricks data format options page.

FAQs

Is Hudi production-ready?

The Apache Software Foundation announced Hudi as a Top-Level Project on June 4, 2020, and the latest release is 1.2.1 as of October 2026. Readiness for your workload depends on your engines, so check the supported Spark and Flink versions for the release you deploy.

Do I need Spark to use Hudi?

No. Hudi also integrates with Apache Flink, and Hudi 1.2.x supports Flink 1.17.x through 2.1.x as well as Spark 3.3.x through 4.1.x. Hudi's docs also list Apache Hive among the engines that query its tables.

Can I convert an existing table to Hudi?

Yes. Hudi's bootstrap operation brings an existing table under Hudi management, either as METADATA_ONLY, where "the full record data is not copied into Hudi," or as FULL_RECORD, where it "is copied into hudi and metadata columns are added." You can also use Hudi for new partitions only, or rewrite the data with the Spark datasource writer.