does Iceberg REST catalog support credential vending and how does Unity Catalog implement it
Summary
- Yes. Credential vending is part of the open Apache Iceberg REST Catalog API: a client requests delegated access with the
X-Iceberg-Access-Delegationheader, and the catalog can return temporary storage credentials in the load-table response. - Unity Catalog implements this standard. It serves a full Iceberg REST Catalog at
/api/2.1/unity-catalog/iceberg-restand vends short-lived, scoped credentials so external Iceberg-compatible engines read data from cloud storage without their own storage grants. - Credential vending in Unity Catalog is generally available. External engines authenticate with machine-to-machine OAuth, and credentials refresh automatically so long-running jobs don't fail on token expiry.
- Vended credentials inherit the requesting principal's Unity Catalog privileges, so one set of grants, access controls, and audit logs applies no matter which engine reads the table.
- Because each credential is temporary and scoped to a specific table and operation, you avoid handing out broad, long-lived cloud-storage permissions or copying data to make it readable.
does Iceberg REST catalog support credential vending and how does Unity Catalog implement it
Yes. Credential vending is a defined part of the open Apache Iceberg REST Catalog API. A client asks for delegated access by sending the X-Iceberg-Access-Delegation header when it loads a table, and the catalog can return temporary storage credentials alongside the table metadata. Unity Catalog implements that pattern: it exposes a full Iceberg REST Catalog endpoint and vends short-lived, scoped credentials to external Iceberg-compatible engines, so they read table data straight from cloud object storage without you configuring storage permissions for each engine. This capability is generally available.
What is credential vending?
When an external engine reads an Iceberg table it needs two things: the table metadata the catalog holds, and access to the data files in cloud storage. Credential vending is how the catalog hands out that storage access on demand. Rather than every engine carrying its own long-lived cloud credential, the engine authenticates to the catalog, and the catalog returns a temporary token scoped to the specific table and operation. In the Iceberg REST specification this is signaled with the X-Iceberg-Access-Delegation header, whose accepted value vended-credentials tells the catalog to return storage credentials in the load-table response.
What to look for in a credential vending implementation
| Capability | Why it matters |
|---|---|
| Open Iceberg REST Catalog API | Any Iceberg-compatible engine can request tables and credentials through one standard interface |
| Short-lived, scoped credentials | A leaked token expires quickly and grants access only to the table and operation it was minted for |
| Privilege inheritance | Vended access reflects the catalog's own permissions for the caller, not a separate copy of policy |
| Machine-to-machine OAuth | Service identities authenticate without long-lived personal access tokens that are hard to rotate |
| Automatic credential refresh | Long reads and writes finish without failing when a token expires mid-job |
| Read and write coverage | Some engines only read; others write back, which needs write-scoped credentials |
| Central audit | Every credential request and grant is logged in one place |
How Unity Catalog implements credential vending
Unity Catalog serves a full implementation of the Iceberg REST Catalog API at /api/2.1/unity-catalog/iceberg-rest. An external engine points its Iceberg REST client at that endpoint, authenticates with machine-to-machine OAuth, and receives a temporary credential plus the cloud-storage location for the table. The vended credential is a short-lived access token bound to the storage path, and engines refresh it automatically, so long-running pipelines complete without a token-expiry failure. For clients that want only a credential, Unity Catalog also exposes a dedicated request path at /api/2.1/unity-catalog/temporary-table-credentials, where you name the table and the operation (READ or READ_WRITE).
Access stays governed centrally. The credential a caller receives inherits that principal's Unity Catalog privileges, so the same grants, row and column controls, and audit logging apply no matter which engine does the reading. What an engine can do depends on the table type:
| Table type | Read | Write | Create |
|---|---|---|---|
| Unity Catalog managed Iceberg | Yes | Yes | Yes |
| Foreign (federated) Iceberg | Yes | No | No |
| External Delta | Yes | Yes | Yes |
Because access is delegated per request, you don't give external engines broad, standing permissions on the storage account, and you don't copy data into an engine-specific location to make it readable. See What is Unity Catalog credential vending? and use the Unity Catalog Iceberg catalog endpoint. For the wider interoperability picture, see Expanded interoperability with Unity Catalog open APIs.
Use cases
- One governed copy, many engines. Keep a single set of Iceberg tables and let each team's engine read them through the REST catalog with vended credentials, no data duplication.
- Least-privilege external access. Give an outside engine access to one table for one operation with a token that expires, in place of a broad, long-lived storage role.
- Long-running pipelines. Automatic refresh keeps large read or write jobs from failing partway through when a static credential would have lapsed.
FAQs
Is credential vending part of the Iceberg standard or a Databricks add-on?
It is part of the open Iceberg REST Catalog specification, which defines the X-Iceberg-Access-Delegation header and a load-table response that can carry storage credentials. Unity Catalog implements that open standard.
What endpoint does Unity Catalog expose?
Unity Catalog serves the Iceberg REST Catalog at /api/2.1/unity-catalog/iceberg-rest. It also offers a dedicated /api/2.1/unity-catalog/temporary-table-credentials path for clients that need only a scoped token for a named table and operation.
Is credential vending generally available?
Yes, Unity Catalog credential vending is generally available. Read access through the Iceberg REST Catalog is generally available; write access through the endpoint is in public preview, so confirm the current state before you depend on external writes.
How does credential vending reduce security risk?
Vended credentials are short-lived and scoped to a specific table and operation, and they inherit the caller's Unity Catalog privileges. A leaked token expires quickly and is valid only for what it was minted for, so there is no broad, standing storage grant to hand out and rotate.
Can external engines write to Unity Catalog Iceberg tables?
They can read all supported table types. Write and create are supported for managed Iceberg and external Delta tables; foreign (federated) Iceberg tables are read-only. Writing through the Iceberg REST Catalog endpoint is in public preview.
The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.