Governing unstructured data and files with Unity Catalog volumes

Summary

• Unity Catalog volumes are the securable object that brings governance to non-tabular data: images, audio, video, PDFs, documents, model artifacts, and raw files that don't fit in a table.
• A volume adds a governed path over cloud object storage in the same catalog and schema namespace as your tables, so files and structured data share one access model, one lineage graph, and one audit trail.
• Access uses familiar privileges: READ VOLUME to read files and WRITE VOLUME to add, change, or delete them, granted directly or inherited from the schema or catalog.
• Managed volumes let Databricks handle the storage lifecycle; external volumes point at a location you control for interoperability with outside systems.
• Volumes are the standard place to land the unstructured data behind AI: RAG source documents, ML training sets, and MLflow artifacts all stay governed and traceable.

Governing unstructured data and files with Unity Catalog volumes

Most enterprise data governance was built for tables, but the data behind modern AI is mostly files: PDFs and contracts, images and audio, video, sensor logs, and model artifacts. Unity Catalog volumes close that gap. A volume is a securable object that governs a directory of files in cloud object storage under the same catalog and schema as your tables, so unstructured data gets the same access control, lineage, and audit as everything else in the lakehouse. You grant a volume privilege once and every notebook, job, query, and app honors it, without separate bucket policies or IAM roles per location.

What are Unity Catalog volumes?

A volume represents a logical volume of storage in a cloud object storage location. It sits alongside tables and views in Unity Catalog's three-level namespace (catalog.schema.volume) and can hold data in any format: structured, semi-structured, or unstructured. Files in a volume are reachable by a stable POSIX-style path under /Volumes/, so the same reference works across users, workspaces, and tools without hard-coded cloud URIs.
There are two types. Managed volumes let Databricks manage the storage location and file lifecycle inside your cloud account, with governance applied automatically. External volumes point to a location you register and control, which is the right choice when systems outside Databricks also need to read or write the files. Because external systems reach that storage directly, they run outside Unity Catalog's enforcement, so reserve external volumes for real interoperability needs.

What to evaluate when governing unstructured data

Governing files well takes more than a place to put them. These are the capabilities that matter:

Capability	Why it matters
Unified namespace with tabular data	Files and tables live in one catalog, so teams discover and secure both the same way
File-level access control	Separate read from write and delete, so consumers and producers get only what they need
Privilege inheritance	Grants at a schema or catalog cascade to current and future volumes, cutting per-bucket policy sprawl
Managed and external options	Match Databricks-managed simplicity or bring your own governed location for interoperability
Lineage across data and AI	See which files feed which tables, models, and agents, down to the asset
Audit logging	Record who read or wrote each file, for compliance and investigation
Secure sharing	Distribute file collections across teams, clouds, and organizations without copying

How Databricks approaches it

In Unity Catalog, a volume is a first-class securable object, and access uses the same ANSI SQL privilege model as tables. A user needs USE CATALOG on the catalog, USE SCHEMA on the schema, and then READ VOLUME to read files or WRITE VOLUME to add, modify, or delete them. Grant those privileges at the schema or catalog level and they inherit to every volume beneath, so you govern a whole class of files with one rule.
You work with volume files the way you already work: browse, upload, and download in Catalog Explorer; read and write with Apache Spark, pandas, or SQL; call dbutils.fs, the Databricks CLI, the Files REST API, or the SDKs. Every access runs through Unity Catalog, which enforces permissions, records lineage as files flow into tables, models, and agents, and writes activity to audit logs and system tables. Volumes also share through Delta Sharing and the Databricks Marketplace, so you can distribute PDF, image, audio, and video collections across clouds and accounts alongside tables and models.
For grounding, see the Unity Catalog volumes documentation, the tutorial on working with unstructured data in volumes, and the volumes general availability announcement.

Use cases

• RAG and generative AI: land source documents (PDFs, HTML, transcripts) in a volume, then parse and index them with AI functions and Databricks AI Search while access stays governed.
• Machine learning: store training and test datasets plus MLflow model artifacts in volumes, so experiment inputs and outputs stay under one governance layer.
• Data ingestion: use a volume as the landing zone for raw files, then load them into Delta tables with Auto Loader or COPY INTO.
• Operational files: keep libraries, configuration, checkpoints, and logs in governed locations instead of ungoverned storage.

FAQs

What is the difference between a managed and an external volume?

Databricks manages the storage location and file lifecycle for a managed volume, applying governance automatically. An external volume points to a location you register and control, which suits cases where systems outside Databricks also need to access the files.

What privileges control access to files in a volume?

READ VOLUME grants read access to files and directories; WRITE VOLUME grants the ability to add, modify, and delete files. Users also need USE CATALOG and USE SCHEMA on the parents, and privileges can be inherited from the schema or catalog.

How do volumes support AI and RAG workloads?

Volumes are the governed home for unstructured AI data: RAG source documents, ML training data, and model artifacts. Because access, lineage, and audit are enforced by Unity Catalog, retrieval and training pipelines run on governed files rather than ungoverned buckets.

Can I share files in a volume with other teams or organizations?

Yes. Volumes can be shared through Delta Sharing and listed in the Databricks Marketplace, so collections of PDFs, images, audio, video, and other files can be distributed across clouds, regions, and accounts alongside tables and models.