security best practices for Databricks on Azure

Summary

• Follow the Define, Deploy, Monitor lifecycle. Define controls with the Databricks security best practices guide, deploy them with the Security Reference Architecture (SRA) Terraform templates, and continuously monitor with the Security Analysis Tool (SAT).
• Choose the right network architecture. Azure Databricks offers a progressive model: managed security by default, hardened connectivity, and a fully isolated environment for regulated workloads.
• Isolate the network. Use VNet injection to deploy into your own virtual network, add Azure Private Link for private control-plane connectivity, enable secure cluster connectivity (no public IP), and configure IP access lists and network exfiltration protections.
• Protect data with encryption. Configure customer-managed keys (CMK) for managed services and storage, encrypt traffic between worker nodes, use Azure Managed Identities for storage, and keep production data out of DBFS.
• Centralize identity and prove compliance. Enforce Microsoft Entra ID single sign-on with MFA, automate provisioning with SCIM, govern all assets with Unity Catalog, and rely on SOC 2 Type II, ISO certifications, and HIPAA configuration guidance.

Security best practices for Databricks on Azure

Securing Azure Databricks starts with a defense-in-depth architecture and a repeatable lifecycle. Databricks recommends a Define, Deploy, Monitor approach that addresses the threat categories most relevant to a data and AI platform: account takeover, data exfiltration, insider threats, supply chain attacks, ransomware, and resource abuse. The practices below map to that lifecycle and to the Azure-native controls you can layer on top.

Why Databricks Platform for security on Azure

• A structured security lifecycle. Define the controls appropriate for a standard or a security-focused deployment using the security best practices guide; deploy them programmatically with the Security Reference Architecture (SRA) Terraform templates; and monitor adherence with the Security Analysis Tool (SAT), which scans workspaces against customizable best practices and lets you visualize or export findings. See Define, deploy and monitor your data and AI workloads with security best practices.
• A progressive network architecture. Azure Databricks supports three levels: managed security (secure defaults with Unity Catalog controls), hardened connectivity (tightened ingress and egress for auditability and access control), and an isolated environment (all access made private for regulated industries with strict data exfiltration requirements).
• Network isolation with Azure controls. Use VNet injection to deploy Azure Databricks into your own virtual network (minimum /18 address space, with public and private subnets delegated to Microsoft.Databricks/workspaces and network security groups controlling traffic). Add Azure Private Link for private connectivity to the control plane and web application, enable secure cluster connectivity so cluster nodes have no public IP, use a VPN Gateway or ExpressRoute for on-premises links, and configure IP access lists and network exfiltration protections. See the security, compliance, and privacy best practices.
• Encryption and customer-managed keys. Configure customer-managed keys (CMK) for managed services and storage, encrypt traffic between cluster worker nodes, use Azure Managed Identities to access storage, configure Azure Storage firewalls and soft deletes, keep production data out of DBFS, and rely on credential redaction to keep secrets out of logs and outputs.
• Unified governance with Unity Catalog. Govern data and AI assets centrally with fine-grained access controls at the catalog, schema, table, and column levels, automatic table- and column-level lineage, and comprehensive audit logs. Isolate sensitive workloads into separate workspaces and assign Unity Catalog securables to specific workspaces.
• Identity with Microsoft Entra ID. Enforce single sign-on at the account level (enabled by default on Azure), require MFA through your identity provider, use OAuth federation for API access, and automate user and group synchronization with SCIM. Configure SSO emergency access with MFA and monitor authentication logs for unusual activity. See your identity security modernization journey.
• Compliance and monitoring. Enable verbose audit logging and system tables, manage code through Git folders and CI/CD, and provision infrastructure as code. Databricks provides a due-diligence package with ISO certifications and annual penetration test letters, a SOC 2 Type II report, HIPAA configuration guidance, and a compliance security profile, all available through the Security and Trust Center.

Getting started

• Review the security best practices guide and choose the standard or security-focused control set.
• Pick your network architecture (managed, hardened, or isolated) and deploy with the Security Reference Architecture Terraform templates.
• Install the Security Analysis Tool (SAT) to monitor configuration drift on an ongoing basis.
• Read Data exfiltration protection with Azure Databricks to plan private, exfiltration-resistant connectivity.

FAQs

What security framework does Databricks recommend on Azure?

A Define, Deploy, Monitor lifecycle: define controls with the best practices guide, deploy them with the Security Reference Architecture Terraform templates, and monitor adherence with the Security Analysis Tool.

How do I isolate the network for Azure Databricks?

Use VNet injection to deploy into your own virtual network, add Azure Private Link for private control-plane connectivity, enable secure cluster connectivity so nodes have no public IP, and configure IP access lists and network exfiltration protections.

How does identity work for Azure Databricks?

Enforce Microsoft Entra ID single sign-on at the account level with MFA, use OAuth federation for API access, and automate user and group provisioning with SCIM.

What compliance evidence is available?

Databricks provides a due-diligence package with ISO certifications and annual penetration test letters, a SOC 2 Type II report, and HIPAA configuration guidance through the Security and Trust Center.