Introducing Databricks CustomerLake

Summary

• Infinity campaigns: CustomerLake is an agentic CDP replacing one-off campaigns with infinity campaigns—continuous agentic loops that react to customer context in real-time.
• 1:1 personalization: The platform enables enterprises to deliver one-to-one personalized experiences to customers a billion times a day using profile and campaign agents.
• Unified customer data: CustomerLake unifies customer data, AI models, agents, and identity resolution on the open lakehouse, enabling real-time activation across marketing channels.

Introducing Databricks CustomerLake

Databricks enters the marketing industry with CustomerLake, a new Agentic Customer Data Platform (CDP) built natively on Databricks.
CustomerLake is an Agentic CDP that unifies customer data, AI models, agents, identity resolution, audience building, and activation — bringing agentic marketing to the open lakehouse.
CustomerLake replaces one-off campaigns with “infinity campaigns,” continuous agentic loops that react to customer context in real time, enabling enterprises to deliver 1:1 personalized experiences a billion times a day.
CustomerLake offers campaign and profile agents, an open partner ecosystem, and native integrations and reverse ETL to ingest, unify, and activate customer data across the marketing and advertising technology stack.
Learn more about CustomerLake: https://www.databricks.com/product/customerlake-cdp
Timestamps:

Chapters

• 0:00 Introduction
• 3:51 Infinity Campaigns
• 6:19 The Agentic CDP
• 7:30 CustomerLake Overview
• 9:28 Profile Agents Overview
• 11:57 Profile Agents Demo
• 15:36 Campaign Agents Overview
• 17:32 Campaign Agents Demo
• 25:39 Closing

FAQs

What are infinity campaigns?

Infinity campaigns are continuous agentic loops that react to customer context in real-time, replacing traditional waterfall campaign processes that take weeks to months.

What is an agentic CDP?

An agentic CDP (Customer Data Platform) uses agents and AI to continuously react to customer signals, automatically personalizing content, offers, and experiences in real-time.

How can companies give marketing AI agents governed, real-time customer context?

Marketing AI agents need rich customer context (purchase history, preferences, engagement, demographics) to personalize at scale. This video introduces CustomerLake—a Databricks solution that combines customer data from multiple sources, governs access, and serves it in real time to marketing AI agents. CustomerLake consolidates data from CRM, web analytics, transactions, and email systems into a unified customer dataset governed by Unity Catalog. AI agents query this governed customer context to personalize offers, next-best-actions, and communications while maintaining compliance with privacy regulations.

What is an embedded customer data platform?

An embedded customer data platform (like CustomerLake) is a unified system that ingests customer data from multiple sources, unifies customer identities, applies governance policies, and serves enriched customer profiles in real time. Unlike standalone CDPs, an embedded CDP is built into the Databricks Data and AI platform, eliminating data duplication and enabling direct integration with analytics and AI systems. CustomerLake demonstrates this approach: instead of exporting data from Databricks to a separate CDP, customer data is governed and enriched within Databricks, then served to marketing applications, analytics, and AI agents.

How should enterprises prepare customer data for marketing AI agents?

Preparing customer data for marketing AI agents requires: (1) consolidating customer data from multiple sources (CRM, email, web, transactional), (2) unifying customer identities (deduplication, linking), (3) applying governance and consent rules (GDPR, CCPA compliance), (4) enriching with derived attributes (lifetime value, propensity scores), (5) ensuring freshness (real-time updates for active behavior). This video shows CustomerLake handling these steps automatically. By preparing data this way, marketing teams can confidently deploy AI agents with accurate, governed context that respects customer privacy.

Full transcript

0:00 Introduction

Hello. I'm a vice president of engineering here at Data Bricks and I'm here to talk to you about an exciting new product that we're launching. 80% of enterprises say that they compete primarily on the basis of customer experience. But how have data and marketing teams created customer experiences in the past? It was mostly based around the concept of a campaign and campaigns have been this waterfall like process where you start with a plan that usually marketing will do and then the data teams have to make sure the data is in place and then you have to create the right audiences and then them become campaigns and some campaign automation tool that then becomes activation and personalization.
And of course the marketing the marketers have to stay on top of the performance. This has been a process that has worked. However, campaigns historically took weeks to months to design and iterate. And even though the industry has been talking about onetoone personalization for a very long time, in reality, most of these campaigns have been one to many. AI and agents are changing everything. So whereas in the past you had humans dying at human speeds, what we're already seeing happening is humans delegating buying decisions to agents who do research, compare and buy on behalf of those customers.
In addition, these agents are always on, so they can actually react to new information, new data in seconds. And finally, not only do these agents react in real time, but with commerce becoming headless, the number of channels is exploding. So campaigns that are bad for millions of people and take weeks to months are not good anymore. We do have to throw this model out buying agents. And the good that's a lot more real time, a lot more personalized.
So imagine if every customer had a team of agents assigned to them, responding to signals and personalizing only for that one customer. What would it look like? What would it look like for that to happen? Well, as the humans or their agent representatives act and provide create different behaviors, different signals, the agents on the enterprise side can take those signals, make them part of the customer context, evaluate them very quickly and make a decision on whether there is an action that needs to be taken or not.
And when this next best action is determined, it will be a very specific piece of content only for that customer personalized for the right channel at the right time. And this is a process that will keep happening because then that interaction goes back to humans or their agents which create new signals. And that's almost like an infinite loop that happens in real time perpetually. This is a completely new concept that we like to call infinity campaigns.
Infinity campaigns are autonomous engagements loops that run constantly all the time. And the reason we call it infinity is because they can be infinite long in time. They're evergreen. They're always running. They can respond in infinitely fast time. So they're true real time. And there's no one to many anymore. This is the first time that we can envision true onetoone personalization with an infinite number of potential journeys tailored to every individual customer.

3:51 Infinity Campaigns

So how do we achieve something like infinity campaigns? Well, the closest thing in the past was CDPs and CDPs stand for customer data platform. Despite their name, it's actually a tool primarily for marketing and marketing ops. And if you look at their history, they have historically done three things. Collect a copy of the customer data, a customer 360. Give the ability to marketers to create segments of several people, several customers and activate those segments, meaning send that segment information to tools like email, mobile web personalization tool to do what's called activation, meaning to send a message to the customer or create a personalized experience for the customer.
Now, these historically have also been one to many. And if you look at their history, they actually started quite far from the data. The first instance of a CDP was really a customer table in an email tool. But then in the mid2000, CDP became its own category with uh CDPs that were bundled, meaning it had both the data infrastructure and the marketer UI bundled in one. and they would copy take a copy of the customer 360 and provide it for use to marketers.
But then as platforms like lakehouses became more popular, they evolved to being more composable, meaning they could actually start sending some queries down to the lakehouse, but still they were not part of the lakehouse. It was still a separate layer, separate governance, still had to move data out of the lakehouse. So these CDPs work great for their time, but for the age of agents, we believe you need a new type of a CDP that we call the agentic CDP.

6:19 The Agentic CDP

And the agentic CDP has three key conditions. First, it's embedded in the lakehouse. It lives where the data lives. And this is very important because customer data, customer context and decision context and the agents they all have to live in the same place. The second requirement of the agendics, it has to be designed from the ground up for agents. This is not about putting a chat box in the UI of an existing CDP. This is about making the agent the core infrastructure piece of this new platform.
And last but not least, CDPs need to power infinity campaigns, meaning they need to be able to provide a team of autonomous agents that constantly monitor every individual customer and in real time do true one-to-one personalization responding to the customer's needs and intentions. Today I am proud to announce customer lake. Customer lake is a new agendic embedded in the datab bricks lakehouse. Besides being embedded which means that it lives in the same governance security and data infrastructure envelope as the rest of the lakehouse.

7:30 CustomerLake Overview

It is also democratized because it provides a rich interface designed for business users. And it's autonomous because it offers a teams of agents so that marketers can power infinity campaigns for their customers. Customer Lake is comprised of two different products, profile agents and campaign agents. Profile agents can take raw customer data, resolve identity so that every customer is uniquely and properly identified and provide a golden customer record that's ready for business and marketing use out of the box.
Campaign agents provide a team of agents so that marketers can power infinity campaigns offering true realtime always on one-to-one personalization for every single customer. If we look at the architecture of customer lake, we can see that the data flows from sources to profile agents to campaign agents and through a new reverse ETL capability to activation tools that touch the customer. The sources can be any data in the lakehouse and we can also bring in third party data from marketing or CRM tools through our leg flow connect capability.
Profile agents provides all the capabilities needed to take raw customer data and create golden customer profiles that then is leveraged by customer lake to power infinity campaigns with capabilities such as audiencing and measurement and through reverse that data and those campaigns get activated through an arbitrary number of customer channels. Let's double click on profile agents. Profile agents are really agents that do the customer 360 work for you with a prompt that describes the business use case.

9:28 Profile Agents Overview

The agents get busy and they start by ingesting and transforming the data. They enrich it by third party data sources for data hydration and then they value the output. But they don't stop there. they can continue ingesting, transforming, enriching until the output matches the user's expectations. If we focus a little bit more on the first part of this process, the data preparation, one of the key components there is agents ingesting data through Lakeflow connect that allows easy data ingestion from third party data sources.
Obviously any data already in the lakehouse is also eligible to be used. And then our agents will do a semantic tagging of the data so that we can available with one click through our new data marketplace. If we double click on identity resolution, we can see that we have taken a very novel approach to a problem that historically has been tackled with traditional machine learning algorithms. So the goal of identity resolution is to identify that two profiles that may not match exactly actually belong to the same person.
And for that we start with a simple set of rules that agents help define and they're very transparent and easy to understand. But when these rules are inconclusive, we fall back to an LLM that helps make a determination. And in certain cases, we can also fall back to a human to make the final evaluation. And as this process runs, the learnings from every run feed back to the rules and the LLM automatically so that the identity resolution keeps improving with every single run.
We also allow customers to bring in their own machine learning models if they have them and contribute to this process and pipeline. And we also have an identity provider marketplace. So we can bring thirdparty identity resolution capabilities from our partners with one click and put them in our customers hands. Let's move to a demo to see what this looks like. >> Hey, I'm a director of product here at Data Bricks. Let's jump into a demo of profile agents. >> Okay, so here I am on the homepage of my fan 360 and profile agents.

11:57 Profile Agents Demo

Um you can see that I'm in the Fan 360 America's project. If I have different uh projects, I can resolve um different identities and different customer 360s separately. Um but here I'm in my America fan 360. Um I can see that there's already three sources configured. Um there's a set of matching rules that have been defined and the uh result is 4. 2 million unified profiles. So, what I'm going to do, um, we want to integrate the new data set that we got from our thirdparty ticket reseller.
So, I'm going to jump over here and add a new source. Um, these are the tables that are available in Unity. Um, I can go in here and say tickets resale customers. This is the table that I want to integrate. Add this data set. And what's going to happen now is Genie is preparing this data set for identity resolution. It's going to go through and create um automatically these semantic tags. So it's um identifying what's the email columns, the phone numbers, um all the uh inputs that we need for identity resolution.
Um it's also uh defining some rules to filter and normalize the data. So you can see here um that it's automatically defined a couple of uh filters and ways to essentially standardize and clean up the data. So, in this case, it's phone numbers. Um, and all the phone numbers are normalized to a standard format. From here, um, I could go into each one of these and, um, edit the rules or edit the filters. Um, I'm not going to do that here just for the sake of time.
I'm going to trust Genie. Uh, and I'm just going to accept these rules. Um, and then down below here, we get a preview of both the raw data. So, this was the data that was input from my ticket resale table. Um, and then the normalized output. So this is what Genie um did the heavy lifting for me. Normalize this and now I'm ready for the identity resolution process. Um let's mark this as ready and jump into our identity resolution.
So now what's happening is there's a set of existing rules for identity resolution that have already been defined. You can see these here. Um, Genie is now looking at the existing rules and looking at the new data set and defining new rules um to integrate this data into our fan 360. So, you can see the new rule that uh recommended here is to take billing address and uh the full name um and it's doing an exact match on billing address and a fuzzy match on the full accepting this rule.
Um, if I wanted to, I could go in and manually edit this. Um, I could say that, you know, define rules for not matching. And I can also send these to reviews. Um, reviews can be done by both Genie, um, so decide whether something that's on the borderline is a match or not, um, or a human reviewer, um, or both. Um, again, I trust Genie. I'm going to go ahead and accept this uh recommended rule. Um and from here now I've prepped the data.
I've uh defined new rules uh for identity resolution to integrate that data. And now I can run this end to end. So now I'm going to launch this run. Um and I can see that my new identity resolution of and the creation of my fan 360 um is being created. Let's double click on campaign agents. Campaign agents start with a business goal. The goal plus the customer context drives agents to create and suggest an campaign plan for Infinity campaign.

15:36 Campaign Agents Overview

And once that plan is approved by the marketer, Infinity Campaign deploys agents to personalize every single interaction of every customer in real time. And as the campaign runs, the decisioning graph, the decisioning context flows back and informs how to improve that campaign autonomously. A component of a campaign is an audience. Audiences have been part of CDPs forever. And in this case, um, we have taken a very agendic first approach to audiencing.
Audiences can actually be created from the campaign business goals autonomously or users can use Genie to describe what audiences they want and have them be created for them. And of course, an audience can become part of a campaign with a click of a button. We also support extensive test and measurement, performance measurement for campaigns. Um, it's very easy for marketers to see how a campaign is performing. It's easy to test different aspects of the campaigns.
But more importantly, agents always monitor campaigns and they can flag what is going well or not so well for a campaign and suggest improvements in real time. And finally, our new reversatil capability allows data from customer lake to be uploaded to a very large number of activation tools that power customer experiences across email, mobile, advertising, and all kinds of different other channels. Let's move to a demo to see what this looks like. >> Okay, so here we are on the homepage of Customer Lake Campaign Agents.

17:32 Campaign Agents Demo

Um, and what we can see here is we can see a breakdown of the campaigns that I'm tracking. I can see my key audiences. I can see trends over time. Um, but what we're trying to do is we're trying to launch an endto-end campaign. So, what I'm going to do is I'm going to start with my audiences over here. Um, I want to uh target my uh most engaged fans. So, I actually have an audience that's already created of these most engaged fans.
And then I'm going to refine that from here. Um, so, uh, you can see this is, uh, an insights dashboard around this audience. So, I could see the audience size of almost 5 million. I could see how it's trending over time. Um, I can see a breakdown of, uh, what different categories this audience is buying, uh, ticket spend, etc. Um, these insights are actually recommended automatically by Genie based on the underlying data.
Um, you obviously can curate these insights yourself, but this kind of genie recommended insights is a really powerful feature. Um, I don't want to target just the engaged fans. I actually want to refine this from here. So, what I'm going to do is I'm going to go ahead and edit this. And now I can see in here as I click into this, here's all the definitions um that back this audience. They're socially engaged. They have a certain amount of spend.
They have a certain amount of app sessions. Um, and then what I'm going to do is I'm going to modify this. Um, and copy in the prompt, narrow this down to Team USA fans who have not purchased jerseys yet this season. Um, so I'm using Genie to actually refine the definition of this audience. So what Genie is going to do is it's going to take my original audience um, and it's going to add additional filters. So you can see here, this is the definition of the original audience.
Um, this is the new audience definition that Genie is proposing. Um, and I have new filters here. Favorite team is United States. Um, and they haven't purchased any jerseys. Um, so I'm using Genie to essentially create these new audiences on the fly. I'm going to accept that. Um, this looks good. Uh, save this audience. Um, and now again, I get back to the insights page with the new audience definition. Um, you can see just as a gut check here, uh, that breakdown by favorite team is indeed, uh, limited only to the, uh, United States here.
Um, so now what I do is I have my, uh, USA teams fans. You can see Genie also updated the metadata about this audience, um, the description, the name, etc. Um, and I'm going to use this in a new campaign. So, uh, I'm going to jump over to the campaigns tab over here. Um, you can see that this audience was automatically attached to my campaign. Um, and now what I'm going to do is I'm going to prompt uh Genie that I want to increase um Jersey Jersey sales by uh targeting this audience.
Um, very simple prompt. look at the existing campaigns. It's going to look at the audiences that I've had. Um, I could have attached any additional context to that audience, whether it be a campaign brief or an existing dashboard. Again, just to keep it simple. We just want to increase Jersey sales. Um, now what's going to happen is Genie comes back with a series of essentially clarifying questions to get more context.
So, what channels should this campaign use? Um, we'll say use meta. Um, how many messages should the fans receive? Let's make it a three-part journey. Um, on the context that you've given. And now what I'm going to do is I'm going to have Genie draft a campaign brief. So this campaign brief is the source of truth for the campaign moving forward. It gives a summary of the campaign goals, um the targeting strategy, the messaging strategy.
Um and you can see that here. Goals, messaging, message types, the message plan, timing, um any sort of guardrail. So guard rails are um I want to have you know obviously suppress a do not contact list or marketing opt outs or I want to have frequency caps on this um and any sort of assumptions and I can edit this um directly um and again this becomes a source of truth on which the rest of the campaign will be built.
I'm not going to edit this now again I trust Genie. I'm going to go and say build this campaign. So now what Genie is going to do um is it's going to generate all the objects that are needed for this campaign. Um you can see the audience that we generated or created in the previous phase. Team USA fans without jersey purchases. Um it's Genie's created a new conversion metric audience here. USA jersey converters. Um I can click into this and edit this goal by the way.
So I can define custom conversion metrics. Um I can see the actions. I have uh in this case a three-part email campaign, an initial launch, a reminder um and uh you know a last chance um and then also I have uh a uh meta campaign um that's set up alongside my email campaign. Um here's the guard rails that were implemented. Marketing optouts uh don't uh don't target people with open support tickets, etc. Uh, this looks good to me.
So, I'm going to save this as a draft. Um, and now what I see is this would be my uh live campaign dashboard where I'm getting kind of real-time insights onto the profiles, the activations, the conversions, etc. for this campaign. Um now once this campaign is live um our Genie campaign agents are going to evaluate on a onetoone basis for each individual customer that's qualified for this campaign what methods they should be getting what channels they should be getting it on um the frequency the timing of this.
So it is truly onetoone marketing at scale. Um to see how that works, we can actually go into um the decisioning tab here. And because this campaign isn't live yet, we're going to run a simulation on a 100 random profiles that are qualified. Um and once the simulation runs, we can actually click into this and see for each individual profile that was selected for the simulation, what targeting strategy they got, what's the message that they received.
Um in this case, they were skipped in the initial one. Um, but now this profile received the launch campaign for this email. Um, and I can click into this and actually see the reasoning behind uh, Genie's decision here. So, it gives a summary. Um, it gives uh, what rules were considered, what's the rationale, it also gives what what data signals were considered here. Um, and any sort of guard rails that were applied.
And again, Genie is not just looking at this individual campaign. It's also looking at um you know when these customers are being targeted by other campaigns that you're running um as part of campaign agents right um so this was just a simulation but I can now launch this campaign um and as I do that uh now uh this campaign will start running uh and Genie will evaluate for every single customer um what's the best targeting strategy um this campaign just got launched so it doesn't have any data yet but if I were to jump over here into an existing campaign, let's say, um, a ticket to browse retargeting campaign, I'd be able to see the insights.
I could see the profiles, the activations, the conversions, the conversion rate. Um, I can see a breakdown of activations by day. Um, and similar to the simulation, I could actually go into each of these runs and see who was actually targeted for this campaign um, and why were they targeted. So this is true um agent-driven one-to-one marketing at scale um powered by customer-like campaign agents. >> In the past few months, we have been rolling out these products and we are proud to be partnering with some of the world's greatest brands that have already deployed and use customer lake.

25:39 Closing

We're also very proud and honored to be part of a very elite ecosystem of Martekch technology vendors that have partnered with us to integrate their products with customer lake and provide a very easy way for our customers to create a very modern stack that puts agents and infinity campaigns at the center of their customer experience strategy with customer lake. I hope you have seen that we now have the capability to enable enterprises to provide the perfect customer experience a billion times a day.

Learn more about the Databricks Data and AI platform.