How can I govern data across Databricks and Snowflake together?

Summary

• Unity Catalog, Databricks' centralized governance layer, can extend across multiple compute platforms and data systems, enabling policy enforcement on external data sources including those in Snowflake.
• Databricks connects to Snowflake as a federated data source, allowing queries against Snowflake tables with Unity Catalog governance applied on the Databricks side.
• Cross-workspace data sharing via Databricks shares data objects without physical movement, using credential delegation and fine-grained access control.
• Both platforms support ACID transactions, lineage tracking, and role-based access control on governed data objects.
• A hybrid architecture keeps data in place while centralizing governance policies: define roles and permissions once in Unity Catalog, apply them across both platforms using connectors and external catalogs.

How can I govern data across Databricks and Snowflake together?

Governance across Databricks and Snowflake means applying consistent policies (access control, tagging, lineage, and audit trails) to data wherever it lives. Unity Catalog, Databricks' centralized metadata layer, can govern data in both systems by connecting to Snowflake tables as external data sources and applying policies on the Databricks side. Snowflake independently enforces its own governance framework. A hybrid approach keeps data in place, integrates both systems' native governance capabilities, and maintains a clear audit trail across both platforms.

Governance capabilities: how they compare

Capability	Databricks (Unity Catalog)	Snowflake
Access control	Role-based (RBAC) with column- and row-level filtering	Role-based (RBAC) with column- and row-level masking
Data lineage	Tracks source/transformation/consumer across jobs and notebooks	Tracks query lineage in account-level audit logs
Tagging and discovery	Business and technical tags searchable in Catalog Explorer	Native tags and object tagging for classification
Audit logging	All access and mutations logged to audit event tables in Unity Catalog	All queries and DDL logged to QUERY_HISTORY and ACCOUNT_USAGE views
Data classification	Tags enforce PII/sensitive data labels; can block access based on classification	Native data classification via masking policies and role definitions
Cross-platform sharing	Databricks shares via credential delegation; external tables reference Snowflake	Snowflake shares via secure data shares; read-only for consumers
ACID compliance	ACID transactions on Delta tables; consistency on writes	ACID transactions on Iceberg and native table formats

Architecting governance across both systems

Data residency and connectors: Data stays where it lives. Databricks connects to Snowflake using the native Snowflake connector, issuing federated queries that Snowflake executes. Unity Catalog governs the results on the Databricks side, applying row and column filtering based on the Databricks user's role.
Credential delegation: Connect Databricks to Snowflake using credential delegation so queries execute with the Databricks user's implicit credentials. This ensures Snowflake enforces its own role-based policies as well. Audit trails on both sides show which user accessed what, when, and why.
Cross-workspace sharing: If both systems run Databricks workspaces, use Databricks shares to ship objects (tables, views, volumes) from one workspace to another without copying data. Recipients access via a read-only external location, and the producer controls permissions. Unity Catalog policies travel with the share.
Unified policy layer: Define a master governance policy in Unity Catalog: tag datasets with sensitivity levels (public, internal, sensitive, PII), assign roles based on job function, map credentials to roles. Apply the same logic on both Databricks and Snowflake to maintain consistency. Use Databricks connectors to discover and govern Snowflake assets; use Snowflake's QUERY_HISTORY to audit who accessed what.

Implementing a hybrid governance strategy

1. Inventory: Catalog all datasets in both systems. For Snowflake, document table location and access patterns. For Databricks, register tables in Unity Catalog metastore.
2. Define roles and policies: Create role definitions (data-engineer, analyst, data-scientist) and map them to both platforms. Specify which roles can read, write, and manage specific datasets. Use the principle of least privilege.
3. Enable connectors: Configure Databricks Snowflake connector with credential delegation. Query Snowflake tables from Databricks SQL or notebooks; results inherit Databricks user permissions.
4. Tag and classify: Assign governance tags to all objects (technical tags like owner, project; business tags like pii, confidential). Use tags to drive access policies: restrict pii-tagged tables to authorized roles only.
5. Audit and monitor: Set up dashboards on QUERY_HISTORY (Snowflake) and audit event tables (Unity Catalog) to monitor access patterns, spot anomalies, and validate compliance. Use alerts to flag privilege escalation attempts.

Use cases

Enterprise with multi-cloud analytics: Analytics teams query from both Databricks and Snowflake. Define a unified access policy: analysts see marketing data only; finance teams see revenue tables only. Databricks and Snowflake each enforce the policy on their data. A compliance dashboard queries both audit systems to show who accessed what across the entire enterprise.
Healthcare system protecting patient data: Patient records live in Snowflake; research pipelines run in Databricks. Tag all patient ID and medical data as pii. Databricks governance masks sensitive columns for researchers; Snowflake governance prevents direct queries on patient tables. Audit logs show every access, fulfilling regulatory requirements.

FAQs

Can Unity Catalog govern data that physically resides in Snowflake?

Yes. Databricks can read Snowflake tables via the Snowflake connector, apply Unity Catalog policies (row/column filtering, tagging) in the Databricks query context, and return results to authorized users. Both systems maintain independent audit trails.

Do I need to replicate data between Databricks and Snowflake to govern it?

No. Use external table definitions or federated queries. Databricks queries Snowflake tables in place. Each system enforces its own governance; Databricks adds an additional policy layer on top when Snowflake tables are accessed from Databricks.

How do I audit access across both systems?

Monitor Snowflake's QUERY_HISTORY view for direct Snowflake access and Databricks' audit event tables for Databricks access. Correlate events using user credentials and timestamps to reconstruct who accessed what, when, and via which system.

What happens if a user has a role in Databricks but not in Snowflake?

Credential delegation ensures the Databricks user's identity is passed to Snowflake. If the identity has no Snowflake role, the query fails on the Snowflake side. Both systems enforce their own policies independently; the most restrictive policy wins.

Can I use a single metadata store to govern both platforms?

Partially. Unity Catalog governs Databricks data and can reference external Snowflake tables. Snowflake maintains its own governance model in parallel. For true unified governance, map policies between the systems (same role names, equivalent column filtering rules) and orchestrate both with consistent tagging and access definitions.