Skip to main content

What are the best alternatives to running a separate analytics database alongside Postgres?

Summary

  • Teams run a separate analytics database because transactional and analytical workloads pull in opposite directions: transactions touch a few rows at a time, while analytics scans and aggregates large volumes.
  • On Databricks, Lakebase keeps standard Postgres for transactions and shares one logical copy of its data with the lakehouse through Lake Transactional/Analytical Processing (LTAP), so analytics runs on a different engine without a copy pipeline you build.
  • Lakehouse Federation queries an existing Postgres database in place and read-only. It pushes work down to that database, so it avoids a copy, but the source still runs the pushed-down work.
  • Postgres read replicas move reads off the primary, and because streaming replication is asynchronous by default, a replica can lag.

What are the best alternatives to running a separate analytics database alongside Postgres?

The best alternative depends on where the data lives and how much load analytics may place on it. For data in Lakebase, Databricks' LTAP architecture, whose capabilities sit at different release stages, lets Postgres serve transactions while the lakehouse serves analytics from one logical copy, with no pipeline for you to build. For an existing Postgres database elsewhere, Lakehouse Federation queries it in place, and Lakeflow Connect copies it into Delta tables when data volume or latency matters (its PostgreSQL connector is in Public Preview). Lakebase is built on Neon technology.

Why do teams run a separate analytics database?

Databricks' LTAP documentation describes transactional (OLTP) workloads as acting on a few rows at a time and analytical (OLAP) workloads as aggregating and joining many rows, and says "These patterns pull in opposite directions." Keeping two systems in sync means running change data capture, streaming pipelines, and read replicas. In the docs' words, that infrastructure "is fragile, it adds latency between when data is written and when it can be analyzed, and it competes for resources with the primary transactional database."

How do the alternatives compare?

Approach How analytics runs Effect on the source Postgres Notes
Postgres read replica A hot standby serves read-only queries Reads move off the primary Streaming replication is asynchronous by default, so the standby can lag
Lakebase read replica Independent read-only compute on the same data Reads run on separate compute attached to the same storage Asynchronous, so eventually consistent
Lakebase with LTAP Lakehouse//RT reads the bulk of the data from a columnar copy in object storage and asks Postgres only for the current log sequence number Isolated from the primary Postgres instance Lakehouse//RT is Beta; Unity Catalog registration is GA
Lakehouse Federation Query pushed down to the external Postgres over JDBC The source still runs the pushed-down work Read-only; suited to ad hoc reporting, BI, and proofs of concept
Lakeflow Connect CDC Data copied into Delta streaming tables and queried on Databricks An ingestion gateway extracts snapshots and change logs from the source The PostgreSQL connector is Public Preview

How does Lakebase approach analytics on Postgres data?

Lakebase separates stateless Postgres compute from a storage layer of safekeepers, pageservers, and cloud object storage. LTAP adds one step: as storage materializes data into object storage, it transcodes row-oriented Postgres data into Parquet's columnar layout, readable through open table formats such as Delta and Iceberg. Transcoding runs in the storage layer, isolated from the primary Postgres instance. Lakebase remains standard Postgres, and Postgres indexes stay in their original representation.
Databricks documents three capabilities on this foundation, described in the LTAP architecture page:

  • Register Lakebase in Unity Catalog (GA): govern analytical access and run cross-source queries from the lakehouse.
  • Lakehouse//RT querying Lakebase (Beta): transactionally consistent OLAP queries on live Postgres data, without impacting Lakebase OLTP performance.
  • Lakebase Change Data Feed (Public Preview): stores row-level changes from Postgres tables as Unity Catalog Delta tables.

Lakebase also lists autoscaling, scale to zero, branches, and instant restore, which creates a new branch from any point in time within your history window.

When should I choose each approach?

  • Lakebase with LTAP: your application's data lives in Lakebase and analytics needs it fresh.
  • Lakehouse Federation: you need ad hoc reporting, BI, or a proof of concept on an existing database without moving data. When a source supports both Lakehouse Federation and Lakeflow Connect, Databricks recommends Lakeflow Connect if higher data volumes and lower latency are priorities.
  • Read replica: you want reads off the primary compute and can accept that replicas are eventually consistent.

FAQs

Does Lakehouse Federation take analytics load off my production Postgres?

Not entirely. Federated queries are pushed down to the external database over JDBC, and the docs say the query runs on both Databricks and remote compute, so the source database still does part of the work. Access is read-only.

Do large analytical queries lock Postgres tables and block writes?

Ordinary reads do not. The Postgres documentation says that under MVCC "reading never blocks writing and writing never blocks reading." The cost Databricks describes is different: the workloads differ in shape, and the sync infrastructure competes for resources with the primary database.

Can Lakebase run analytics inside the Postgres engine?

Databricks describes purpose-built engines: "Postgres serves transactions and the Lakehouse serves analytics, and neither compromises the other."

Can I query Lakebase data from the lakehouse today?

Yes, by registering the database in Unity Catalog. That creates a read-only catalog that you query with a serverless SQL warehouse, and you can join it with lakehouse tables. Unity Catalog permissions control queries through SQL warehouses, while direct connections use Postgres roles.

Can Lakehouse Federation reach a Postgres database that I run myself?

Yes, for PostgreSQL data that is not managed by Databricks, as long as your compute has network connectivity to the database.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.