Postgres or CockroachDB for AI agent workloads?
Summary
- For most AI agent workloads, choose Postgres, and run it on Lakebase, the serverless Postgres database in the Databricks Data + AI Platform: agents get standard Postgres plus compute that starts from an API call, scales to zero, branches instantly and connects to lakehouse data.
- CockroachDB is a distributed SQL database designed to accept reads and writes on every node, run SERIALIZABLE transactions by default and place data across regions. It speaks the PostgreSQL wire protocol and supports most PostgreSQL syntax.
- The deciding question is topology. If an agent application's writes must be served from many nodes or regions at once, CockroachDB is built for that. If agents mostly need fast, isolated, short-lived Postgres environments with lakehouse context, Lakebase fits.
- Both store agent state as relational rows with JSONB and run similarity search on pgvector-compatible vectors.
Postgres or CockroachDB for AI agent workloads?
Choose Postgres for most AI agent workloads, and run it on Lakebase, the serverless Postgres database in the Databricks Data + AI Platform. Agent workloads are bursty, short-lived and experimental: an agent creates a database mid-task, writes state and tool results, tests a change and moves on. Lakebase gives that workload standard Postgres on stateless compute that starts, stops, branches and reads lakehouse data. CockroachDB is a distributed SQL database for applications that need every node to accept writes and data to live in several regions, and it's the right tool when an agent application has that requirement.
What agent workloads ask of a database
| Agent behavior | What the database needs |
|---|---|
| Creates databases or environments from code | Provisioning through an API, with no capacity planning |
| Works in bursts, then goes quiet | Compute that scales up under load and down to zero when idle |
| Tries risky writes and migrations | Isolated, disposable copies of production data |
| Saves chat history, checkpoints and tool results | ACID transactions and flexible JSON columns |
| Recalls context by meaning | Vector search next to the relational data |
| Decides with business history, features and scores | Governed access to analytical data, not a stale copy |
Postgres (on Lakebase) vs CockroachDB: how they compare
| Capability | Postgres on Lakebase | CockroachDB |
|---|---|---|
| What it is | The open source object-relational database, run by Lakebase as serverless Postgres in the Databricks Data + AI Platform | A source-available distributed SQL database designed to be scalable and consistent |
| SQL and drivers | Native Postgres SQL, wire protocol, drivers and extensions | PostgreSQL wire protocol and most PostgreSQL syntax; works with most PostgreSQL drivers and ORMs |
| Transactions | ACID, with every SQL-standard isolation level including Serializable | ACID across arbitrary tables and rows; SERIALIZABLE by default, with READ COMMITTED available |
| Write topology | One primary read-write compute per branch, plus read replicas | Every node accepts reads and writes; data is split into replicated ranges |
| Consensus | The write-ahead log replicates across a quorum of safekeepers | Raft: a quorum of a range's replicas acknowledges each write |
| Geography | High availability across availability zones in a region | Multi-region databases with survival goals and table locality |
| How you run it | Fully managed in Databricks; compute autoscales within a set range and scales to zero when idle | Deploy in any environment, or use CockroachDB Cloud: Basic, Standard, Advanced or bring-your-own-cloud |
| Copies and history | Copy-on-write branches from current data or any point in the parent's history | Historical reads with AS OF SYSTEM TIME |
| Agent state | jsonb columns with GIN indexes | JSONB with GIN indexes |
| Vector search | pgvector, plus Lakebase Search for vector and keyword search | A VECTOR type compatible with pgvector, with vector indexes for approximate search |
How Lakebase runs Postgres for agents
A Databricks interview on LTAP describes "databases built to keep up with how agents actually work," and that is the design goal behind Lakebase. The LTAP engineering blog explains the mechanism: Lakebase moves the write-ahead log and data files off the database machine into independent storage services, so Postgres compute owns no data and "can scale up instantly under load and scale all the way down to zero when idle." It is still "real Postgres, so the wire protocol, SQL, drivers, and extensions all work as-is."
For an agent platform, the topology stays simple: one primary compute per branch, with read replicas for reads.
- Provisioning. An agent platform can create projects, branches and endpoints with the REST API, Databricks CLI or SDKs.
- Bursts. Scale to zero suspends idle compute and reactivates it on the next query. Pairing it with high availability isn't supported, because scale to zero isn't available on high availability computes.
- Isolation. Branches are copy-on-write, and their Postgres roles and grants stay isolated, so a migration can run against a copy of production and be discarded.
- Lakehouse context. Registering Lakebase in Unity Catalog applies permissions, lineage and audit logs alongside lakehouse data, and synced tables serve features and scores into Postgres at low latency.
For agent state, Databricks' managed agent sessions and agent memory run on Lakebase, are in Beta, and are the recommended path for new agents. The agent's own business records, such as orders or cases, live in ordinary Lakebase tables. LTAP, the architecture for one governed copy of data, is still being rolled out. See the Lakebase architecture docs.
FAQs
When should an agent application use CockroachDB?
When its writes must be accepted on many nodes or in several regions at once. CockroachDB is designed as an always-on database that accepts reads and writes on all nodes, with multi-region survival goals.
Is Lakebase a fork of Postgres?
No. Lakebase runs the open source Postgres engine, not a fork or a new SQL dialect, so standard Postgres clients, drivers and ORMs connect to it.
Can agents store vectors and state in the same Postgres database?
Yes. With pgvector, Postgres stores vectors with the rest of the data, alongside ACID transactions and joins. Lakebase also offers Lakebase Search for vector, keyword and hybrid search in the same database.
The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.