How should I evaluate the reliability and availability of an embedded CDP?

Summary

• An embedded CDP runs within your data platform, so its reliability depends directly on your data warehouse's uptime, replication, and failover capabilities.
• Look for platforms that use the same underlying infrastructure as your core data warehouse to avoid managing multiple independent availability systems.
• Evaluate automatic failover, read replicas, backup and restore procedures, and whether the CDP supports multi-region or multi-cloud deployment.
• Check service-level agreements for uptime commitments and the platform's recovery time objectives (RTO) and recovery point objectives (RPO) in failure scenarios.
• Ensure the CDP logs all customer data transactions for audit and compliance, and test the platform's disaster recovery procedures before deploying to production.

How should I evaluate the reliability and availability of an embedded CDP?

Reliability and availability are critical for any system managing customer data at scale. An embedded CDP, by definition, relies on your data warehouse's infrastructure for data storage and compute. This means you don't need to manage a separate, independent CDP database and keep it in sync with your warehouse. But it also means the CDP's uptime is directly tied to your warehouse's availability.
The right question to ask is whether the platform uses your data warehouse's built-in reliability features (replication, failover, backup) or requires you to manage additional backup and recovery procedures for CDP-specific data. The former is simpler and more reliable. The latter creates operational complexity and potential consistency gaps.

What is reliability in a CDP?

Reliability in a CDP has two aspects. First, the CDP must correctly process and store customer data without corruption, data loss, or unexpected behavior. This includes handling schema changes, data type conversions, and edge cases (null values, duplicates, malformed records) without silent failures. Second, the CDP must remain available during infrastructure failures, network disruptions, and maintenance windows.
Availability, in turn, refers to the percentage of time the CDP is operational and can process requests. Availability depends on the underlying data warehouse (its replication strategy, failover mechanisms, and geographic distribution), the CDP application itself (whether it has built-in redundancy), and the supporting infrastructure (networking, identity services, API gateways).

Key capabilities to evaluate

Capability	Why it matters
Data warehouse dependency	Does the CDP's availability depend on your warehouse, or does it require its own independent infrastructure?
Automatic failover	If a node fails, does the system automatically failover to a replica, or does it require manual intervention?
Read replicas and scaling	Can the platform read from read replicas to distribute load and improve availability?
Backup and recovery	What backup mechanisms does the CDP support (snapshots, continuous replication, point-in-time restore)?
Multi-region support	Can the CDP run across multiple regions or clouds for geographic redundancy?
RTO and RPO	What is the recovery time objective (time to restore service) and recovery point objective (data recency after recovery)?
Audit and compliance logging	Does the platform log all transactions for audit and compliance verification?
Disaster recovery testing	Can you run disaster recovery drills to verify failover and recovery procedures?

How Databricks approaches reliability

CustomerLake is built on Databricks Lakehouse, which provides enterprise-grade reliability and availability. Because customer data lives in Delta Lake tables managed by your data warehouse, you inherit the warehouse's availability architecture. Databricks provides automatic replication, managed failover, and disaster recovery capabilities within each cloud region.
Identity resolution, enrichment, and other CDP operations run on Databricks Jobs and Lakeflow compute, which can scale horizontally and handle transient failures through automatic retries. If a job fails, it can be re-run with point-in-time consistent data because Delta Lake transactions ensure data consistency.
Databricks supports multi-region replication and backup through Unity Catalog's metastore replication and cloud-native backup services. You can replicate customer data to another region or cloud for disaster recovery, and restore from snapshots if needed.
CustomerLake also provides comprehensive audit logging through Databricks' system table schema, tracking all identity resolution operations, enrichments, and activations. This ensures you can trace every transformation applied to customer data for compliance and debugging.

Use cases

A financial services firm with high availability requirements can deploy CustomerLake on a Databricks workspace with multi-region replication. If the primary region fails, the metastore replica in the secondary region enables quick failover of customer identity and enrichment workloads.
An e-commerce platform handling real-time customer activation can rely on Databricks' automatic failover for Delta Lake. Streaming identity resolution jobs run on Lakeflow and automatically retry on transient failures, ensuring customer data is always current.
A SaaS company managing customer data across multiple tenants can use Delta Lake's transaction guarantees to ensure each tenant's data is isolated and consistent, even during concurrent updates and failover events.

FAQs

What happens to my customer data if a Databricks cluster fails?

Your customer data is stored in Delta Lake tables, which are managed by Databricks and replicated within your region. A cluster failure doesn't affect your data. You can spin up a new cluster and continue working with the same data immediately.

Can I replicate my customer data to another cloud for disaster recovery?

Yes. Databricks supports cross-region and cross-cloud replication through Unity Catalog. You can replicate your customer data to another region or cloud and restore from snapshots if needed.

How can I verify that CustomerLake is processing my customer data correctly?

CustomerLake logs all operations through Databricks system tables. You can query audit logs to see every identity resolution, enrichment, and activation operation, along with timestamps and status. This gives you full transparency into data processing.

Does CustomerLake require me to manage its own separate backups?

No. Because CustomerLake data lives in your Databricks Lakehouse as Delta Lake tables, it inherits Databricks' backup and recovery capabilities. You don't need to manage separate CDP backups.