Best file format for multimodal AI data such as images, video, and audio

Summary

• There is no single best file format for multimodal AI data. Keep images, video, and audio in their native formats, and use a table format for the paths, labels, and embeddings that describe them.
• On Databricks, Unity Catalog volumes govern non-tabular files of any format, and Delta Lake, which extends Parquet data files with a transaction log, is the default table format.
• The FILE type (Beta) stores a governed reference to an unstructured file in a Delta Lake table column, as either FILE MANAGED or FILE EXTERNAL.
• Lance is an open source format built for multimodal AI that stores binary objects next to tabular data, but Unity Catalog documentation does not list it as a table format.
• Embeddings belong in an array[float] column that a Databricks AI Search index can sync from.

Best file format for multimodal AI data such as images, video, and audio

There is no single best file format, because multimodal data has two parts with different needs. The media itself (images, video, audio) stays in its native encoding, and a table format describes it with paths, labels, timestamps, and the embeddings derived from it. On Databricks, the documented building blocks are Unity Catalog volumes for the files and Delta Lake tables for the metadata. Lance is a purpose-built alternative that keeps media bytes and tabular columns in one format, so check platform support before you adopt it.

Which options cover which needs?

Need	Option	What the documentation says
Govern raw image, video, and audio files	Unity Catalog volume	Holds non-tabular data of any format, managed or external
Query metadata, labels, and embeddings with transactions	Delta Lake table	Extends Parquet data files with a transaction log for ACID transactions
Tie a table row to a governed file	FILE type (Beta)	Stores a reference with uri, offset, size, content_type, and checksum
Load file bytes into Spark	Binary file data source	One record per file with path, modificationTime, length, and content
Keep media bytes and tabular columns in one open format	Lance	Describes itself as an open source lakehouse format for multimodal AI
Search embeddings	Databricks AI Search	Delta Sync Index over a table with precomputed embeddings

How does Databricks store and govern multimodal data?

Unity Catalog volumes "enable governance over non-tabular datasets" and hold "non-tabular data of any format, including structured, semi-structured, or unstructured," such as image, audio, video, and PDF files. A managed volume lives in Unity Catalog managed storage, and an external volume registers an existing cloud storage path.
Delta Lake is "open source software that extends Parquet data files with a file-based transaction log for ACID transactions and scalable metadata handling," and it is the default format on Databricks. Parquet is "an open source, column-oriented data file format designed for efficient data storage and retrieval."
The FILE type (Beta) "stores a governed reference to an unstructured file, with metadata such as path and size." With FILE MANAGED, Unity Catalog stores copies of the files and manages them with the table. With FILE EXTERNAL, the column references existing files in a volume. The SQL reference lists Delta Lake tables on Databricks Runtime 18 LTS and above, and the docs note that automatic garbage collection of unreferenced files isn't supported in Beta.

CREATE TABLE attachments (id BIGINT, attachment FILE EXTERNAL);

INSERT INTO attachments
SELECT row_number() OVER (ORDER BY file.uri), file
FROM read_files('/Volumes/my_catalog/my_schema/my_volume/', format => 'file');

SELECT attachment.uri, attachment.size, attachment.checksum FROM attachments;

To load file bytes instead, the binary file data source reads each file into one record: spark.read.format("binaryFile").load("/Volumes/<catalog>/<schema>/<volume>/").

Where does Lance fit?

Lance describes itself as "a modern, open source lakehouse format for multimodal AI." Its documentation says it stores images, videos, audio, text, and embeddings alongside tabular data in one format, and its blob columns hold large binary objects with lazy, file-like reads. The Lance file format is "a columnar container optimized for cloud object stores, random access, and Arrow-native processing."
Check platform support first. Unity Catalog documentation says managed tables must use delta or iceberg, and external tables can use delta, CSV, JSON, avro, parquet, ORC, or text. Lance is not on that list. Lance's own Databricks guide, last updated February 19, 2026, describes a Classic Compute setup tested on Databricks Runtime 16.4 LTS. It lists Unity Catalog on Classic Compute and custom data sources on SQL warehouses as not supported.

How do embeddings fit in?

Databricks AI Search is "a search solution that is built into the Databricks Data + AI Platform," and its indexes "appear in and are governed by Unity Catalog." A Delta Sync Index with self-managed embeddings reads pre-calculated embeddings from a source table, and the documentation says the precomputed embedding column should be array[float]. On standard endpoints, the source table needs a change data feed, and tables with row tracking enabled use one automatically.

Use cases

Computer vision (illustrative): A team keeps camera clips in a volume and records each clip's path, labels, and frame embeddings in a Delta Lake table.
Speech archives (illustrative): Audio files stay in a volume, and a table holds one row per recording with its transcript and an embedding column.

FAQs

Should I store media bytes in the table or reference the files?

Databricks documents both. The binary file data source loads each file's bytes into a content column, while the FILE type stores a governed reference with metadata instead of the bytes. A reference keeps files in their original format in a volume, and the table holds metadata, labels, and embeddings.

Can Unity Catalog manage Lance tables?

The Unity Catalog documentation does not list Lance among supported table formats, and Lance's Databricks guide lists Unity Catalog on Classic Compute as not supported.

Can I index embeddings stored in a Delta table?

Yes. Create an AI Search Delta Sync Index with self-managed embeddings over the table. It needs a primary key column and an embedding column in array[float] format.