Unity Catalog REST API and Iceberg REST catalog compatibility

Summary

• Unity Catalog implements the Apache Iceberg REST Catalog API, a standardized, open interface that enables any Iceberg-compatible engine to read Unity Catalog-managed tables.
• The REST API endpoint at /api/2.1/unity-catalog/iceberg-rest is generally available and supports Apache Spark, Trino, Snowflake, Flink, DuckDB, PyIceberg, and many other Iceberg-compatible clients.
• Credential vending through the REST API allows external systems to access tables securely without requiring broad storage permissions.
• Catalog federation enables bidirectional interoperability: Unity Catalog can read Iceberg tables from external catalogs while allowing those platforms to access Databricks data.
• UniForm (Universal Format) enables write-once Delta Lake, read-as-Iceberg from any compatible engine, eliminating data duplication and replication pipelines.

Unity Catalog REST API and Iceberg REST catalog compatibility
Unity Catalog implements the Apache Iceberg REST Catalog API standard, enabling any Iceberg-compatible engine to access Databricks-managed tables. This open interface decouples catalog implementation from clients, allowing organizations to read and share data across a heterogeneous ecosystem of compute engines and platforms. The REST API is generally available and production-ready.

The Iceberg REST Catalog API standard

Apache Iceberg defines a REST API specification for accessing catalog operations (listing tables, reading metadata, committing transactions) without requiring direct access to a catalog's internal implementation. Unity Catalog conforms to this standard through the /api/2.1/unity-catalog/iceberg-rest endpoint. Because it is standards-based, any client library that speaks Iceberg REST can connect to Unity Catalog and read tables, regardless of whether the client was built by Databricks.
This separation of concerns has a key advantage: Databricks manages catalog logic, governance, and storage integration, while external systems only need to implement the REST API contract. Updates to either side don't require coordinated releases.

Supported clients and engines

The following engines and frameworks can read from Unity Catalog via the Iceberg REST API:

Engine	Use Case	Auth	Data Format
Apache Spark	Distributed SQL and batch processing	Service principals, OAuth	Delta or Iceberg
Apache Trino	Federated SQL queries across data systems	Configured credentials	Iceberg via catalog
Snowflake	Cloud data warehouse queries	REST API catalog integration	Iceberg tables
Apache Flink	Streaming and batch workloads	Configured credentials	Iceberg
DuckDB	In-process analytical queries	Configured credentials	Iceberg
PyIceberg	Python-based data engineering workflows	REST API credentials	Iceberg
Custom clients	Domain-specific applications	REST API authentication	Iceberg

All clients access the same catalog namespace and inherit the same fine-grained permissions defined in Unity Catalog. An external engine reading a table respects row-level and column-level access controls without additional configuration.

How credential vending works

Unity Catalog's Iceberg REST API extends credential vending to external clients. When an external system requests a table, Unity Catalog evaluates the request against access policies, generates a short-lived credential scoped to that access, and returns it to the client. The client uses that temporary credential to read data from storage (S3, Azure Blob Storage, GCS). The credential is valid only for the specific table and operation, and it expires automatically. This improves security by avoiding persistent API keys and applying fine-grained governance at read time.

Catalog federation for multi-catalog enterprises

Catalog federation enables bidirectional interoperability: Unity Catalog can read Iceberg tables from external catalogs (AWS Glue, Google Cloud Lakehouse Runtime, Snowflake Horizon, others), and those systems can read from Unity Catalog. When enabled, Unity Catalog appears as a catalog to external systems, and external catalogs appear as schemas within Unity Catalog. Queries can span both: join a table from Unity Catalog with a table from an external catalog in a single query. Access control remains local to each catalog, but the query engine coordinates the reads.

UniForm for Delta and Iceberg alignment

Many organizations have invested in Delta Lake. UniForm (Universal Format) is a Databricks feature that enables Delta tables to be read as Iceberg tables by any Iceberg-compatible engine without data replication.
How it works: when you enable UniForm on a Delta table in Databricks, Databricks maintains an Iceberg metadata layer alongside the Delta metadata. External engines (Snowflake, Trino, etc.) can read that Iceberg metadata and access the underlying data files directly. From the external engine's perspective, it's just an Iceberg table. From Databricks' perspective, it's still a Delta table with all Delta's transaction guarantees.
This eliminates a common challenge: replicating data to different formats to support different engines. With UniForm and the REST API, you write once (in Delta) and read anywhere (as Iceberg), maintaining a single source of truth and governance through Unity Catalog.

Use cases

Analytics warehouse federation: Organizations with data across multiple platforms use catalog federation to query both systems in a single query without ETL replication.
Multi-engine ML pipelines: Data teams use Spark for ETL, Python (PyIceberg) for inference, and DuckDB for exploration. All read from the same Databricks catalog with unified access control and lineage tracking.
Third-party integrations: Grant external analytics tools read-only access via the REST API. Credential vending issues temporary tokens scoped to allowed tables, eliminating permanent credential exchange.

FAQs

Do I need special configuration to enable the REST API?

The Iceberg REST API endpoint is available by default on all Unity Catalog-enabled workspaces. External clients can begin using it immediately as long as they have valid credentials (service principal, OAuth token, or API token) and appropriate table permissions.

What metadata is exposed through the REST API?

The REST API exposes table schema, partitioning information, and the location of data files. It does not expose column statistics, table comments, or business metadata beyond the table name. For that, use the Unity Catalog metadata API.

Can I use the REST API for writes?

Yes. The Iceberg REST API supports both reads and writes. External systems can insert, update, or delete data through the REST API with write permissions on the table.

How does credential vending work with external systems?

Credential vending is automatic. The client authenticates to Databricks once (via OAuth, API token, or service principal), and thereafter receives temporary credentials for each table access. External systems never need persistent Databricks credentials.