How do I recover a Delta table after a failed write?

Summary

• A Delta write is atomic. If it fails before the commit, the table stays at its last committed version, and the data files the attempt wrote remain untracked until VACUUM deletes them. Run DESCRIBE HISTORY to see what actually committed.
• If a bad write did commit, inspect an earlier version with VERSION AS OF or TIMESTAMP AS OF, then run RESTORE TABLE ... TO VERSION AS OF or TO TIMESTAMP AS OF.
• RESTORE fails if the older version's data files were deleted manually or by VACUUM. Delta's defaults keep log history for 30 days and let VACUUM remove unreferenced files after 7 days.
• RESTORE is a data-changing commit, so a streaming job that reads the table can process the restored rows again.

How do I recover a Delta table after a failed write?

First work out whether the write committed. A Delta write has two phases: it writes data files, then commits by adding a new entry to the transaction log. A write that fails before the commit leaves the table at its last committed version, so there is nothing to undo. A write that committed bad data, or a multi-statement job that stopped partway through, needs RESTORE.

What does a failed write leave behind?

Possibly data files that no table version references, if the attempt got as far as writing them. The Databricks docs say they "will not corrupt the table state, but the files will not become part of the table", and VACUUM deletes untracked files, including leftovers from failed transactions. If two writers conflict, the Delta docs say the write fails with a concurrent modification exception, such as ConcurrentAppendException, "rather than corrupting the table". By default each SQL statement is its own atomic transaction against one table, so a job that runs several statements can fail after earlier ones committed.

How do I check what the table contains now?

DESCRIBE HISTORY my_table;                        -- newest operation first
DESCRIBE HISTORY my_table LIMIT 1;                -- last committed operation
SELECT count(*) FROM my_table VERSION AS OF 15;   -- inspect an older version

DESCRIBE HISTORY returns columns including version, timestamp, userName, operation, operationParameters, isBlindAppend and operationMetrics. Find the last version before the bad job, then query it with VERSION AS OF or TIMESTAMP AS OF. Querying an older version doesn't change the current table.

How do I restore a known-good version?

RESTORE TABLE my_table TO VERSION AS OF 15;
RESTORE TABLE my_table TO TIMESTAMP AS OF '2026-10-08 15:30:00';

The timestamp format is yyyy-MM-dd HH:mm:ss, or a yyyy-MM-dd date. In Python, DeltaTable.forName(spark, "my_table").restoreToVersion(15) does the same. You can restore an already restored table. On Databricks you need MODIFY permission, and the command reports metrics such as num_removed_files and num_restored_files.

Which symptom needs which fix?

Symptom	What happened	What to do
Write failed, table unchanged	The commit never happened; staged files are untracked	Fix the cause and rerun. VACUUM removes the leftover files
ConcurrentAppendException or ConcurrentDeleteReadException	A concurrent operation conflicted at commit	Rerun. To avoid repeats, partition by the columns your conditions filter on so operations touch disjoint files
Job reported success, data is wrong	The commit succeeded	DESCRIBE HISTORY, then RESTORE to the version before the job
Multi-statement job stopped midway	Earlier statements committed	RESTORE to the last good version, or run the statements as one transaction next time
RESTORE fails	Data files for that version were deleted manually or by VACUUM	Restore a newer version. Setting spark.sql.files.ignoreMissingFiles to true allows a partial restore

What limits RESTORE?

• Retention. Time travel needs both the log and the data files. delta.logRetentionDuration defaults to 30 days, and delta.deletedFileRetentionDuration (the VACUUM threshold) defaults to 7 days. Raise both to keep more history; storage costs can go up.
• Streaming. RESTORE writes log entries with dataChange set to true, so a streaming query reading the table processes the restored data as new.
• Scope. RESTORE reverts the whole table. For specific rows, query the earlier version and INSERT or MERGE them back.

How does Databricks help?

Catalog Explorer shows table history on the History tab. On Databricks Runtime 18.0 and above, time travel queries are blocked for versions older than deletedFileRetentionDuration (default 7 days), and Unity Catalog managed tables apply this on 12.2 and above. Predictive optimization runs VACUUM on managed tables and keeps data files at least 7 days, so set delta.deletedFileRetentionDuration first if you need longer. For jobs with several statements, Unity Catalog managed Delta tables with catalog commits enabled support transactions (BEGIN ATOMIC ... END;): if any statement fails, all changes roll back.

FAQs

Can I restore to a version from months ago?

Only if the log entry and the data files still exist. With the defaults, history older than 30 days is cleaned up after checkpoints, and a VACUUM run removes unreferenced data files older than 7 days.

What's the difference between RESTORE and ROLLBACK?

RESTORE reverts a table after a change has committed. On Databricks, ROLLBACK discards changes inside an interactive transaction before COMMIT.

Can I restore just some rows?

Not with RESTORE. The Databricks docs show the pattern: INSERT INTO my_table SELECT * FROM my_table TIMESTAMP AS OF date_sub(current_date(), 1) WHERE userId = 111.