security best practices for Databricks on AWS

Summary

• Follow the Define, Deploy, Monitor lifecycle. Define controls with the Databricks security best practices guide, deploy them with the Security Reference Architecture (SRA) Terraform templates, and continuously monitor with the Security Analysis Tool (SAT).
• Govern every data and AI asset with Unity Catalog. Use fine-grained access control, attribute-based access control (ABAC), row filters, and column masks, and route all access through Unity Catalog so grants, lineage, and audit logs are enforced.
• Isolate the network. Deploy a customer-managed VPC, use AWS PrivateLink for front-end and back-end connectivity, keep secure cluster connectivity on (the default), configure IP access lists, and apply data exfiltration protection.
• Protect data with encryption. Configure customer-managed keys (CMK) for managed services and for storage, and encrypt data in transit and at rest.
• Centralize identity and prove compliance. Enforce account-level single sign-on with MFA, automate users and groups with SCIM, apply least privilege, and rely on SOC 2 Type II, ISO 27001, HIPAA, and GDPR alignment.

Security best practices for Databricks on AWS

Securing Databricks on AWS starts with a defense-in-depth architecture and a repeatable lifecycle. Databricks recommends a Define, Deploy, Monitor approach that addresses the threat categories most relevant to a data and AI platform: account takeover, data exfiltration, insider threats, supply chain attacks, ransomware, and resource abuse. The practices below map to that lifecycle so you can harden a workspace and keep it hardened.

Why Databricks Platform for security on AWS

• A structured security lifecycle. Define the controls appropriate for a standard or a highly secure deployment using the security best practices guide; deploy them programmatically with the Security Reference Architecture (SRA) Terraform templates; and monitor ongoing adherence with the Security Analysis Tool (SAT), which compares account and workspace configurations against curated best practices, prioritizes findings by severity, and can scan notebooks for unobfuscated secrets. See Define, deploy and monitor your data and AI workloads with security best practices.
• Unified governance with Unity Catalog. Govern data and AI assets centrally with fine-grained access control, attribute-based access control (ABAC), and standard GRANT/REVOKE permissions, plus row filters and column masks for table- and column-level security. Assign Unity Catalog securables to specific workspaces and plan your data isolation model up front. See the security, compliance, and privacy best practices.
• Network isolation. Use a customer-managed VPC to control network configuration, and add AWS PrivateLink for both front-end (user) and back-end (workload) connectivity so traffic stays on trusted private networks with no public internet exposure. Keep secure cluster connectivity enabled (the default so cluster nodes have no public IP addresses), configure IP access lists, and apply data exfiltration protection.
• Encryption and customer-managed keys. Classify data by sensitivity, then configure customer-managed keys (CMK) for managed services to protect control-plane data and CMK for storage to encrypt data at rest in your own cloud storage. Encrypt data in transit and at rest using industry-standard protocols.
• Identity, SSO, and least privilege. Authenticate all users through account-level single sign-on (SSO) with multi-factor authentication (MFA), automate user and group provisioning with SCIM, and use OAuth tokens for applications and service principals. Limit account admins to a small number of trusted individuals and manage every grant by the principle of least privilege. Databricks-managed passwords reached end-of-life on July 10, 2024. See your identity security modernization journey.
• Secrets and monitoring. Store credentials in the secrets manager rather than in notebooks, enable verbose audit logging, and use system tables to monitor activity. The Databricks AI Security Framework (DASF) extends these controls to AI and machine learning workloads.
• Compliance and certifications. Databricks maintains SOC 2 Type II, ISO 27001, HIPAA, and GDPR alignment, with a due-diligence compliance package available through the Security and Trust Center.

Getting started

• Review the security best practices guide and pick the standard or highly secure control set for your workspace.
• Deploy a hardened workspace with the Security Reference Architecture Terraform templates.
• Install the Security Analysis Tool (SAT) to monitor configuration drift on an ongoing basis.
• Plan your Unity Catalog data isolation and governance model.

FAQs

What security framework does Databricks recommend on AWS?

A Define, Deploy, Monitor lifecycle: define controls with the best practices guide, deploy them with the Security Reference Architecture Terraform templates, and monitor adherence with the Security Analysis Tool.

How do I keep Databricks network traffic private on AWS?

Use a customer-managed VPC with AWS PrivateLink for front-end and back-end connectivity, keep secure cluster connectivity enabled, configure IP access lists, and apply data exfiltration protection.

What compliance certifications does Databricks hold?

Databricks maintains SOC 2 Type II, ISO 27001, HIPAA, and GDPR alignment, with a due-diligence compliance package available through the Security and Trust Center.

How do I monitor my security posture over time?

Run the Security Analysis Tool (SAT), which compares your account and workspace configurations against curated best practices, prioritizes gaps by severity, and can scan notebooks for unobfuscated secrets.