Skip to main content

data governance platforms for multi-cloud

Summary

  • Multi-cloud data governance applies one set of identities, access policies, lineage, and audit practices to data in more than one cloud, and gives you a governed way to share data between clouds.
  • The real test is where a platform draws its boundaries. On Databricks, Unity Catalog is built into the platform on AWS, Azure, and Google Cloud, a metastore is regional, and a multi-cloud deployment runs one metastore per region in each cloud.
  • OpenSharing (formerly Delta Sharing) shares live data and AI assets between metastores across clouds and regions without replication, although your cloud vendor may charge egress fees.
  • Access control combines privileges, tag-driven ABAC policies, and row filters and column masks, and Databricks recommends ABAC for consistent policies across many tables.
  • Open formats keep data portable: managed tables are always Delta or Apache Iceberg, and the Iceberg REST catalog lets supported clients read Unity Catalog tables.

data governance platforms for multi-cloud

Multi-cloud data governance means applying consistent identities, access policies, lineage, and audit practices to data that lives in more than one cloud, and sharing that data across clouds without losing control. Ask any platform where its governance boundary sits: a workspace, a region, an account, or a cloud. On Databricks, Unity Catalog is the unified governance layer for data and AI built into the platform, and its boundary is the regional metastore. A multi-cloud deployment runs one metastore per region in each cloud and connects them with OpenSharing.

What should you look for in a multi-cloud governance platform?

Capability What to check
Governance scope Whether one model covers data and AI assets in every cloud you run, and where its boundaries sit
Cross-cloud sharing How governed data reaches another cloud: live access or copies, who grants access on the receiving side, and what egress costs
Open formats Whether other engines can read governed tables through open standards and APIs
Fine-grained access control Privileges, tag-driven policies, row filters, and column masks
Identity Whether users and groups are provisioned from your identity provider, and into which accounts
Lineage and audit Whether lineage is captured automatically and where audit logs can be read

How does Databricks approach multi-cloud data governance?

  • The same governance layer on each cloud. Unity Catalog is available on Databricks on AWS, Azure Databricks, and Databricks on Google Cloud, and also as an open-source implementation. Tables, views, volumes, functions, models, and services follow a three-level namespace, catalog.schema.object.
  • Regional metastores. You can have only one metastore per region, and all workspaces in that region share it. A metastore stores metadata for securable objects and the permissions that govern access to them. Catalogs, not metastores, are the usual unit of data isolation.
  • Sharing across clouds. Databricks-to-Databricks OpenSharing shares data between metastores across clouds and regions. It does not require data replication, but your cloud vendor may charge egress fees when you share across clouds or regions. If the cost gets too high, the deployment guide suggests a pipeline that synchronizes tables between regions. To share with users on any computing platform, Databricks-to-Open sharing covers tabular data you manage in Unity Catalog.
  • Fine-grained access. Access control combines privileges and ownership, ABAC policies based on governed tags, table-level row filters and column masks, and workspace bindings. Databricks recommends ABAC to centralize and scale access control.
  • Identity. Principals must be defined at the Databricks account level to receive privileges. Databricks recommends provisioning them from your identity provider with automatic identity management, or SCIM provisioning if the provider does not support it.
  • Lineage and audit. Lineage is captured automatically for queries run on Databricks, down to the column level, and aggregated across all workspaces attached to the metastore. Audit events are available through the audit log system table (Public Preview), and most audit logs are only accessible from the region in which they are recorded.
  • Open formats. Managed tables are always in Delta or Apache Iceberg format. Unity Catalog implements the Iceberg REST catalog API, so supported clients can read and write managed Iceberg tables, and read Delta tables that have Iceberg reads (UniForm) enabled.

Each metastore stores its own permissions, so policies are not shared between metastores automatically. For decentralized governance models, the deployment guide suggests managing permissions through automated deployment pipelines, such as Terraform.

FAQs

How does multi-cloud governance differ from single-cloud governance?

The pattern is the same: one metastore per region, with OpenSharing between metastores. In a multi-cloud deployment the metastores sit in different clouds, so more data access crosses cloud boundaries, where egress fees can apply.

Can I move data between clouds without re-creating governance policies?

Not automatically. A metastore stores the permissions for its own objects, so data in another metastore needs its own grants. With OpenSharing, the recipient creates a catalog from the share and grants access to their team. Do not register shared tables as external tables in more than one metastore, because schema and property changes made through one metastore will not register in the other.

What is the difference between identity and access control?

Identity says who a principal is. Users, groups, and service principals are defined at the Databricks account level, and Databricks recommends provisioning them from your identity provider. Access control says what they can do: privileges granted on securable objects, plus ABAC policies, row filters, and column masks.

What does open mean for Unity Catalog?

Unity Catalog is available as an open-source implementation. Managed tables use Delta or Apache Iceberg, and Unity Catalog serves the Iceberg REST catalog API at /api/2.1/unity-catalog/iceberg-rest. External access requires enabling external data access on the metastore and granting the EXTERNAL USE SCHEMA privilege.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.