Skip to main content

How do I manage Lakebase projects, branches, and endpoints with Terraform or Databricks Declarative Automation Bundles?

Summary

  • Both Terraform and Databricks Declarative Automation Bundles enable infrastructure-as-code management of Lakebase resources, letting you version and automate project, branch, and endpoint deployments.
  • Lakebase resources follow a parent-child hierarchy: projects contain branches, branches contain endpoints. Databricks automatically provisions a production branch with a primary endpoint when you create a project.
  • Declarative Automation Bundles use the postgres_projects resource and deploy via the CLI; Terraform uses the databricks_postgres_project resource and state files. Both support configuration of auto-created default resources with replace_existing semantics.
  • Key operational concerns: soft-delete retention windows, Terraform state sharing in multi-user environments, and resource naming constraints. Deletion can be set to immediate purge if needed.

How do I manage Lakebase projects, branches, and endpoints with Terraform or Databricks Declarative Automation Bundles?
Both Terraform and Databricks Declarative Automation Bundles (the Databricks infrastructure-as-code framework) let you codify your Lakebase infrastructure, turning manual project setup into repeatable, version-controlled deployments. Choose based on your existing tooling: Terraform if you already use it for cloud resources; Declarative Automation Bundles if you want a Databricks-native, CLI-first experience.

Resource hierarchy and lifecycle

Lakebase uses a three-level hierarchy: a project is the top-level container; branches are isolated copies of a project's schema and role structure; endpoints are connection points to a branch that handle read-write or read-only workloads. When you create a project, Databricks automatically provisions a production branch and a primary read-write endpoint; you declare these in code using replace_existing: true to take over their configuration.

Declarative Automation Bundles approach

With Declarative Automation Bundles, the core resource is postgres_projects. A minimal bundle defines your project, then branches and endpoints beneath it:

bundle:
  name: lakebase-app

resources:
  postgres_projects:
    my_app:
      project_id: 'my-app'
      display_name: 'My Application'
      pg_version: 17

  postgres_branches:
    dev_branch:
      parent: ${resources.postgres_projects.my_app.id}
      branch_id: 'dev'
      no_expiry: true

  postgres_endpoints:
    dev_endpoint:
      parent: ${resources.postgres_branches.dev_branch.id}
      endpoint_id: 'primary'
      endpoint_type: 'ENDPOINT_TYPE_READ_WRITE'
      autoscaling_limit_min_cu: 0.5
      autoscaling_limit_max_cu: 2
      replace_existing: true

Deploy the bundle with:

databricks bundle validate
databricks bundle deploy

The validate step checks syntax; deploy creates or updates the resources. If you need to configure the auto-created production branch or primary endpoint, declare them in your bundle with replace_existing: true to override defaults.

Terraform approach

With Terraform, use the databricks_postgres_project resource from the Databricks provider. A basic project:

resource "databricks_postgres_project" "app" {
  project_id = "my-app"
  spec = {
    pg_version            = 17
    display_name          = "My Application"
    enable_pg_native_login = false
  }
}

Then run:

terraform fmt
terraform apply

Like bundles, Terraform automatically creates a production branch with a primary endpoint. To configure either, declare a databricks_postgres_branch or databricks_postgres_endpoint resource with replace_existing = true.

Comparison: Bundles vs. Terraform

Aspect Declarative Automation Bundles Terraform
Configuration format YAML, bundler semantics HCL, state file
State management CLI-managed, implicit Explicit state file (local or S3/remote)
Learning curve Lower if using Databricks CLI already Lower if familiar with Terraform
Multi-environment Built-in environment targets (dev, prod) Use workspaces or dynamic naming
Sharing resources Shared state in workspace metadata S3 or Terraform Cloud for state locking
Cloud dependencies Databricks-only Works with any cloud or hybrid setup

Key operational details

Deletion behavior: By default, removing a project soft-deletes it and retains it for 7 days. To delete immediately and permanently, add purge_on_delete: true in Declarative Automation Bundles or purge_on_delete = true in Terraform, or pass --purge to the CLI delete command.
Terraform state in multi-user teams: Lakebase autoscaling configuration does not detect drift from manual changes outside Terraform. Use shared remote state (S3, Terraform Cloud, or Databricks state management) and enable state locking to prevent accidental overwrites when multiple engineers manage the same resources.
Resource naming constraints: Project IDs, branch IDs, and endpoint IDs must be 1 to 63 characters, using only lowercase letters, numbers, and hyphens.
Network and security: Both tools work with Databricks IAM and Unity Catalog for row-level and column-level access control. Use Terraform or bundle variables to keep credentials and configuration out of version control.

FAQs

Can I mix Declarative Automation Bundles and Terraform for the same Lakebase infrastructure?

Not recommended. Mixing tools on the same infrastructure can cause state conflicts and unpredictable drift. Pick one approach per project and stick with it.

What happens to my data if I delete a Lakebase project via Terraform or Declarative Automation Bundles?

By default, the project soft-deletes with a 7-day grace period. If you use purge_on_delete: true or --purge, the deletion is immediate and permanent. Data is not recoverable after permanent deletion, so use caution in production.

How do I set up staging and production environments with Terraform?

Use Terraform workspaces or variable files to maintain separate state per environment. For example, create dev.tfvars and prod.tfvars with different project IDs and configurations, then apply with -var-file dev.tfvars or -var-file prod.tfvars.

Can I manage a Lakebase project created manually through the UI with Terraform or Declarative Automation Bundles?

Yes, but you must import the existing project into your infrastructure-as-code setup. Terraform provides an import command; Declarative Automation Bundles requires manual state setup. After import, manage it fully through code.

How does autoscaling configuration in Terraform prevent drift?

Autoscaling settings (min and max compute units) are stored in your Terraform state file. If someone manually changes the endpoint autoscaling through the UI, Terraform sees the diff on the next plan. Run terraform apply to enforce the declared configuration or update the code to match reality.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.