How difficult is it to migrate real-time workloads from ClickHouse, Druid, or Snowflake to Lakehouse//RT?

Summary

• Migration has two parts. Lakehouse//RT is a Beta serverless SQL warehouse type that runs SELECT queries only. Ingestion and transformation move to other Databricks compute that writes Unity Catalog managed Delta Lake or Apache Iceberg tables, and the serving queries move to Lakehouse//RT.
• Table types decide what can move. Lakehouse//RT reads managed Delta Lake and Iceberg tables, materialized views, streaming tables and metric views. Hive metastore tables, foreign tables and Lakehouse Federation are not supported, and external tables are not yet supported.
• Clients change. Lakehouse//RT only accepts connections that use the Statement Execution API, and a driver that connects another way receives a 501 error.
• Rewrites go construct by construct. ClickHouse array lambdas, Snowflake colon paths, insert-time aggregation and layout settings each have a Databricks SQL counterpart, and Lakehouse//RT runs in ANSI mode only.
• Tooling coverage differs. Lakebridge lists Snowflake in its supported sources. ClickHouse and Druid are not in that table, so plan your own query conversion for them.

How difficult is it to migrate real-time workloads from ClickHouse, Druid, or Snowflake to Lakehouse//RT?

The effort depends on how much of the workload is query serving and how many source-specific features it uses. Lakehouse Real-Time (Lakehouse//RT) is a serverless compute built for low-latency, high-concurrency use cases. It is in Beta, and it serves reads only. So the migration has two parts. Ingestion and transformations run on other Databricks compute and land in Unity Catalog tables, then queries and client applications point at a Lakehouse//RT warehouse. Source-specific functions, insert-time aggregation, drivers and access controls are where to look first.

What does Lakehouse//RT support and not support?

Lakehouse//RT is a SQL warehouse type. A workspace admin enables the Beta under Previews, in a supported region, and you create the warehouse under Compute > SQL Warehouses by choosing Real-Time. You can't upgrade an existing SQL warehouse to it, so plan on a new one. The docs say the supported feature set will change before general availability. These documented limits matter most:

Area	Documented behavior
Statements	SELECT only. No INSERT, UPDATE, DELETE, MERGE, CTAS, DDL, GRANT or REVOKE
Tables	Unity Catalog managed Delta Lake and Apache Iceberg tables (recommended for best performance), materialized views, streaming tables, metric views
Not supported	Hive metastore tables, foreign tables and Lakehouse Federation, temporary tables, and tables in CSV, JSON, Parquet or other file formats. External tables and tables in Unity Catalog default storage are not yet supported
Clients	Statement Execution API connections only. Legacy Thrift connections get a 501 error
SQL	ANSI mode only. No Python UDFs, AI functions or GEOGRAPHY and GEOMETRY types
Governance	No ABAC, row-level security or column masking

How does each source system's ingestion map to Databricks?

Source	Today (vendor docs)	On Databricks
ClickHouse	The Kafka table engine acts as a Kafka consumer, and a materialized view moves its rows into a target table	Structured Streaming kafka source written to a managed Delta table with toTable
Druid	Supervisors on the Overlord manage Kafka indexing tasks that read by partition and offset	The same Kafka source
Snowflake	Snowpipe loads staged files, Snowpipe Streaming writes rows, and the Snowflake Connector for Kafka ingests topics	Kafka source for topics, Auto Loader (cloudFiles) for files arriving in cloud storage

Run the Databricks stream next to the existing consumer. By default Spark's Kafka source generates a unique consumer group id for each query and tracks offsets itself instead of committing them to Kafka, so the original pipeline keeps running. Lakeflow Pipelines is the declarative option, and a pipeline runs in triggered or continuous mode. Backfill history with a batch job on other compute, because Lakehouse//RT can't write. For Snowflake history, the Databricks Runtime includes a Snowflake connector.

Which queries need rewriting?

Construct	Source system	Databricks SQL
Array lambdas	ClickHouse arrayMap and arrayFilter	transform and filter
JSON and VARIANT paths	Snowflake src:salesperson.name	Colon operator, jsonExpr : jsonPath, on VARIANT or JSON strings
Pre-aggregation	ClickHouse materialized views run at insert time, often into SummingMergeTree or AggregatingMergeTree targets. Druid rollup, on by default, combines rows with identical dimension values and timestamps at ingestion	Materialized view over the landed table, refreshed incrementally where possible
Data layout	ClickHouse ORDER BY and PARTITION BY, Snowflake CLUSTER BY	Liquid clustering with CLUSTER BY, which replaces partitioning and ZORDER

Freshness becomes a design choice. ClickHouse describes its materialized views as triggers that run as data is inserted. A Databricks materialized view is refreshed on a schedule or with REFRESH MATERIALIZED VIEW, so match the cadence, or use a continuous pipeline, to each table's freshness need. Lakehouse//RT also runs read queries in ANSI mode only, so queries that relied on non-ANSI behavior might raise errors or return different results. When a query uses an unsupported feature, Lakehouse//RT returns an error naming it, and the docs say to use a serverless SQL warehouse instead. Replay your real query set before cutover.

Which tools help?

Lakebridge, a Databricks Labs toolkit, covers assessment, SQL conversion and data reconciliation. Snowflake is a listed source. ClickHouse and Druid are not, so plan your own query conversion and result comparisons for them.

FAQs

Can the old system and Lakehouse//RT run side by side?

Yes. For Kafka-fed pipelines, add a second consumer of the same topic that writes Delta tables, then compare query results from both systems before shifting traffic.

Can Lakehouse//RT handle the ingestion too?

No. It is for read (SELECT) queries only, so use Structured Streaming or Lakeflow Pipelines to write the tables it serves.

Will my existing drivers work?

Only drivers that use the Statement Execution API. The Python connector needs use_kernel=True and JDBC needs UseThriftClient=0. For ODBC, the Simba Spark and Databricks drivers are not supported, so use the Databricks OSS ODBC Driver (Beta).

Is Lakehouse//RT generally available?

No, it is in Beta. Ask your Databricks account team to enable it, then turn it on under Previews.