When should an AI application use Lakebase Search instead of Databricks AI Search?

Summary

• Use Lakebase Search when your application keeps its operational data in Lakebase Postgres and needs vector, BM25 keyword, or hybrid search in the same SQL statement as filters and joins against live tables.
• Use Databricks AI Search when you want a managed search engine over lakehouse data, with Databricks-managed embeddings, indexes that sync from Delta tables, hybrid search, and reranking.
• Databricks' own guidance is to pick Lakebase Search when you want all your operational and search data in one database, and AI Search when you want high-quality retrieval out of the box without manual tuning.
• Lakebase Search is a set of Postgres extensions (lakebase_vector, lakebase_text, lakebase_tokenizer). It needs Postgres 16 or later, and enabling it on a project is irreversible.
• Lakebase Search is generally available on AWS and Azure, and lakebase_vector is a drop-in companion to pgvector, so the same vector types, operators, and query syntax apply.

When should an AI application use Lakebase Search instead of Databricks AI Search?

Choose Lakebase Search when search is one more query over rows your application already reads and writes in Lakebase Postgres. Databricks' Lakebase Search announcement describes the fit as wanting "all your operational and search data in one database": a single SQL statement can fuse vector and BM25 keyword results, apply filter predicates, and join against live operational tables. Choose Databricks AI Search when you want a managed search engine over lakehouse data that delivers high-quality retrieval out of the box, without manual tuning. The deciding factors are where your data lives and how you want the index maintained.

What is Lakebase Search?

Lakebase Search adds vector, keyword, and hybrid search to Lakebase projects through three Postgres extensions.

• lakebase_vector provides approximate nearest-neighbor search with the lakebase_ann index. It is a drop-in companion to pgvector: the same vector types, distance operators, and query syntax work without modification.
• lakebase_text provides BM25 full-text search with the lakebase_bm25 index and works with PostgreSQL's standard tsvector type. One documented caveat: BM25 global statistics are not MVCC-versioned, so scores and rankings can change within a REPEATABLE READ transaction.
• lakebase_tokenizer adds configurable tokenization.

Lakebase Search needs Postgres 16 or later. Enabling it restarts all computes in the project and can't be turned off.

What is Databricks AI Search?

Databricks AI Search is "a search solution that is built into the Databricks Data + AI Platform and integrated with its governance and productivity tools." Its indexes appear in and are governed by Unity Catalog. It supports hybrid keyword-similarity search, filtering, and reranking, and you query it with the Python SDK, the REST API, or the SQL vector_search() function.

Lakebase Search vs Databricks AI Search: how do they compare?

	Lakebase Search	Databricks AI Search
Where it runs	Postgres extensions inside a Lakebase project	Managed endpoints and indexes in the Databricks platform
How you query	SQL, with pgvector operators (<->, <=>, <#>) and BM25 ranking through <@>	Python SDK, REST API, or SQL vector_search()
Search types	Vector, BM25 keyword, and hybrid	Hybrid keyword-similarity search, filtering, and reranking
Embeddings	You generate them with an embedding model and store them in a VECTOR column	Databricks can compute them for a Delta Sync index, or you supply pre-calculated ones
Keeping data current	Search runs on rows in Postgres tables; synced tables bring in Unity Catalog tables	A Delta Sync index can sync automatically when its source table updates
Combining with other data	Filters and joins against live operational tables in one SQL statement	Filters apply to indexed columns, and results return to your application
Setup	Postgres 16 or later; enabling is irreversible	Creating an index requires CREATE TABLE on the target schema

Which application fits which product?

If your application...	Start with	Documented reason
Filters or joins search results against live operational tables	Lakebase Search	One SQL statement can fuse vector and BM25 results, apply filter predicates, and join against live tables
Already runs pgvector queries	Lakebase Search	lakebase_vector accepts the same vector types, distance operators, and query syntax
Searches lakehouse tables and needs the index to follow source updates	Databricks AI Search	A Delta Sync index can sync automatically when the source table updates
Wants Databricks to compute embeddings	Databricks AI Search	Databricks-managed embeddings are calculated for a Delta Sync index
Wants managed retrieval quality without tuning	Databricks AI Search	Databricks positions it for high-quality retrieval out of the box, and reranking is a documented capability

FAQs

Is Lakebase Search the same as pgvector?

No. pgvector (the vector extension) is listed on its own in the Lakebase extensions table. Lakebase Search adds lakebase_vector, lakebase_text, and lakebase_tokenizer, which are listed as requiring Lakebase Search to be enabled. Installing lakebase_vector with CASCADE installs pgvector as a dependency.

Who generates the embeddings?

In Lakebase Search, you generate embeddings with an embedding model and store them in a VECTOR column. On Databricks you can query an embedding model through Model Serving, for example with ai_query, then insert the vectors into Lakebase. AI Search can compute embeddings for a Delta Sync index or use a source that already contains them.

How does hybrid search work in each?

In Lakebase Search you write a SQL query. The docs show one that retrieves top candidates from each search and combines them with Reciprocal Rank Fusion. In AI Search you set the query_type parameter to hybrid, and it runs approximate nearest-neighbor and full-text searches in parallel, then merges the results with Reciprocal Rank Fusion.

How do I get lakehouse data into Lakebase Search?

Use synced tables to sync a Unity Catalog table into Lakebase, and map its columns to search-ready Postgres types during the sync.