Customer Lake: Agentic CDP for AI-Driven Personalization

Summary

Customer Lake is Databricks' embedded agentic customer data platform (CDP), built natively within the Databricks Lakehouse to give AI agents direct access to unified customer data, business context, and governance through Unity Catalog without copying data across systems.
Infinity Campaigns replace static marketing calendars with continuously running agents that personalize campaigns for each customer individually, powered by LLM-driven identity resolution that cleans raw customer data and resolves profiles across data sources.
Embedding the CDP in the lakehouse eliminates duplicate governance layers and enables personalization at the speed agents operate — seconds to minutes instead of the days required by traditional CDP architectures.

Customer Lake: Agentic CDP for AI-Driven Personalization

Traditional customer data platforms disconnect marketers from the data and context needed to compete in an agentic future. Customer Lake is the first embedded agentic CDP, built natively within the Databricks Lakehouse to power real-time, personalized marketing campaigns run by AI agents. Unlike composable CDPs that copy data across systems, Customer Lake gives agents direct access to unified customer data, business context, and governance through Unity Catalog.
Explore how Infinity Campaigns replace static campaign calendars with ever-running agents that know each customer individually, powered by high-quality profile agents that resolve identities using LLMs and clean raw customer data. Learn how embedding the CDP in the lakehouse eliminates duplicate governance layers, accelerates deployment, and enables personalization at the speed agents operate: seconds to minutes instead of days.
🤝

Chapters

FAQs

What is Customer Lake and how is it different from a traditional CDP?

Customer Lake is an agentic CDP built natively inside the Databricks Lakehouse, giving AI agents direct access to unified customer data without copying data across separate systems. Traditional composable CDPs require data movement and separate governance layers, while Customer Lake shares the governance and context already configured in Unity Catalog.

What are Infinity Campaigns in Customer Lake?

Infinity Campaigns replace static, time-boxed campaign calendars with ever-running agents that know each customer individually and act at the optimal moment to engage. Rather than batch campaigns sent on a fixed schedule, each customer has a continuously operating agent managing their personalized marketing experience in real time.

How does Customer Lake use LLMs for identity resolution?

Profile agents use LLMs to clean raw customer data and resolve identities — matching records that represent the same customer across different data sources — to build high-quality unified customer profiles. This LLM-first approach handles the messiness of real-world customer data that rules-based matching systems often miss.

Why does embedding a CDP in the lakehouse improve AI-driven personalization?

When the CDP lives inside the lakehouse, agents can query unified customer data with the same governance and context used by data scientists, without copying data or maintaining separate permission systems. This removes latency and duplication, enabling personalization decisions in seconds to minutes rather than the days typical of traditional CDP architectures.

Full transcript

[00:19] All right, and we're back. Welcome back to Summit Live. I am Jason Pole. I'm the global CTO for BuiltOns. And today I'm swapping out my co-host, so Arya has has been swapped out with Amber. So, Amber, would you would you do the honors of introducing yourself? Yes. Hi everyone. My name's Amber. I was here on Summit Live yesterday.
[00:36] Um and I work on the tech marketing team and I build a lot of demos. Throwing over to Tasso. Hi. Um my name is Tasso. I'm the general manager for Customer Lake, our new product that we launched today. And what is CDP or Customer Lake?
[00:52] Yeah, so Customer Lake is a type of CDP. CDP stands for customer data platform. And is what historically marketers would use to access customer data and turn customer data into campaigns. Mhm. Okay. Okay.
[01:07] And then how is the how is the CDP or the Customer Lake how is that changing the customer experience now? Yeah, that's that's a great question because customer experience is changing whether we like it or not and it's changing because more and more people
[01:23] are using agents to buy. Okay. Right before you want to buy something, you would take your time, right? You would have to do research, ask your friends, go figure out who has the best price, how they can get it shipped to you. Now most people that buy something, they
[01:39] just ask an LLM. Do you do that? Do you ask an LLM? Yeah, I actually I was I was just in a paint store yesterday and I wanted to buy some paint and I was waiting in line to ask the the guy behind the counter, so I pulled up my AI, took a picture of the can of paint and I was able to get
[01:55] an answer from the AI faster and it was actually more accurate than the guy at the counter. Yeah. Yeah. Yeah. So, that's happening everywhere. I think everyone is doing this now, but more and more you won't even have to go to the store, right? You deploy an agent and be like, "Hey, tell me what Here's Here's my problem. Tell me what I should buy. Do the
[02:10] research. Tell me who sells it. Where is it? Is it near me? Do they have it in stock? What's the price?" So, what this is doing is compressing the the say the buying cycle, right? What used to be maybe a few days of research and actually going and buying
[02:25] it now can happen from seconds to minutes to a couple of hours. Um so so things happen a lot faster and also the agents filter everything for you. If marketing sends you something that's not fully personalized for you,
[02:41] you're not going to see. The agent's going to intercept it, right? So, this raises the bar for how fast and how accurate marketing has to be in the next decade. Yeah. Wow. So, the So, customer data platforms have always existed, but now they're just
[02:57] changing because of AI just like most things. Everything, right? Everything. Yeah, so we think the way CDP has worked in the past does not work for the agentic future because they were disconnected from the lakehouse, right? Which is Databricks. And by being disconnected,
[03:13] it means they didn't have access to the data, to the agents, and to the context Yeah. that you now need to do that. So, we introduce a new concept. We call it uh the agentic CDP. And the agentic CDP has three key characteristics. It's embedded
[03:31] in the lakehouse. Yeah. So, it's part of the same platform that's managing your data and agents. Mhm. Um it's match um it powers what we call infinity campaigns, which are campaigns that are run by agents, Yeah. not by humans, and that makes them a lot
[03:46] more personalized and a lot faster. And it will it is born in the era of agents, right? So, it's been built the agentic CDP built from scratch Mhm. to be LLM first. Okay. That's the new evolution of CDPs. And Castle Lake is an instance of an agentic
[04:04] CDP built specifically for the Databricks Lakehouse. Okay. you used the term like uh embedded in the Lakehouse, where with CDPs like uh the big term used to be um composable. Composable. So, that's right. Is how is how is uh being embedded in
[04:19] the Lakehouse different than being composable? question. Yeah, so CDPs have a long history. Um the CDPs as a category is maybe 10 years old. And I think you've had some experience before Databricks. I did. I did. Yes, I had a CDP company and before that I had a database
[04:35] company. So, it's it's Everything is coming together now for me, which is fascinating. Really exciting. Um but even before CDPs were a thing, um you know, like even in the 2000s, email tools would store a table of customer data.
[04:50] Right? So, that was a CDP of the 2000s essentially. It was a table of customer name, email that would live inside your email tool. And but then, you know, as more and more channels came around, CDPs became a separate thing. And they would copy data from the data
[05:07] warehouse. Yep. And then, it was kind of an island like a data mart. And then, when platforms like Databricks came out, they came closer. So, they became composable. What composable means it's still separate, but it can be SQL queries down to Databricks versus copying the data.
[05:23] Got you. But there was still a lot of friction, right? The data separated and governance is a pain. Now, you need to manage the, you know, governance and users in the different places. And then, with context, which can be decision and context, business context, data context,
[05:38] and agents increasingly living inside Databricks, composable is not as close to the data as it seemed. Got you. So, embedded is the next evolution, right? So, we went from a table in an email tool to a bundle CDP that would
[05:53] copy data to composable CDP that was still separate, but we know we'd push some queries down and to embedded. And I feel like embedded is a final home for CDPs. So, you know, it's like CDPs finally came home. They're embedded. There's They're where the data and the
[06:09] context and the agents live, and now they have full access to everything they need to do these new type of campaigns, new type of marketing. And you use the term infinity campaigns. Is that Well, first, what is that exactly? And then, is that the customer's top use case? Yes, infinity campaign is our vision for
[06:26] how marketing is going to evolve. It's kind of what customer like powers. We call it infinity because they can run forever, right? So, before you had to do a campaign the campaign would run for 2 weeks and
[06:41] then you would do another campaign with something else. Maybe you would tweak what you run before and you do something new and then something new. The concept of infinity campaign is what if we would what if we could have one agent for every customer? And that agent is like a personal
[06:57] marketing person dedicated to that just one customer. You could never afford to do that Yeah. with humans, right? You couldn't have like one marketing per customer. That would not scale. But you can have one agent per customer. And what that agent can do is understand your background profile, all your
[07:13] signals and history that you've sent to the brand yourself. And at any given point in time, it knows what's the best next action for you. So, this concept of campaigns or journeys that have a start and at the end and then something new starts that can be replaced with an evergreen,
[07:29] always running campaign Oh, wow. run by agents that constantly evolve their understanding of you and the understanding of the business context as well. And they keep evolving and personalizing better and better, right? So, the notion that you had one campaign then another whole campaign calendar is
[07:45] replaced by this agenting always running campaign that has agents that know the customers really well and can do personalization in a way that was never possible before. Okay. Yeah. And then I saw in your your keynote announcement you talked a little bit about identity resolution and how that's
[08:02] a key part of any I guess CDP. Yes. Um but we've got maybe a little bit of a special take on it. I will have embedded embedded CDP. For sure. So you know a generic CDP like everything depends on high quality data. Yep. Right? So the market and the technology
[08:17] has changed so much Mhm. but the old saying is still true. Garbage in garbage out. And so we don't like garbage in, right? The garbage in is not good. So Customer Life has two products. One is campaigns which is built for marketers as a
[08:33] business UI. This is for marketing, marketing operations, marketing analytics. But then we also wanted to address the problem of how do you feed this very powerful uh engine with high quality data? That's the job of profiles. Profiles is the second part of Customer
[08:49] Life. And what Profile does is it takes raw customer data and turns it into business ready customer data. They they love it, you know. I know. I know. I know. I know. Yeah. Yeah. Yeah. Yeah. Um we have the
[09:05] we have a soccer game going on in the background. Yeah. So we know when there's a when when a goal is scored. So um So the the notion with Profile agents, they're agents that do customer 360 work. So they can take raw customer data
[09:20] understand the semantics of the data, improve the quality, resolve the identities so that you don't think you have three different customers but it's actually the same person Yep. which is horrible for customer experience. Yeah. Yeah. I don't know if you guys have ever gotten two emails from the same brand.
[09:36] Yes. The same email at the same time two times because they think you're different people. Or you actually already buy the thing and then you're still getting emails for the thing you already Exactly. You already bought This is all data problems, right? Data context problems. So, profile agents will ingest the data,
[09:51] Okay. um fix the quality, make sure you can identify the customers uniquely, and then produce business-ready data products that are ready for consumption from the CDP or even for analytics BI. So, profile agents is broader than just
[10:07] CDP. The The promise is it can shorten the time and effort it takes to produce high-quality customer 360 data. So, one of the key features there is identity resolution, and which is I just explained, right? If you have different people, how do you know they're the same?
[10:23] People used to do this with static rules or machine learning, which is very difficult. Yeah. We're doing it with LLMs. Uh-huh. So, it's a modern, fresh approach that's easier to deploy and more accurate and faster, as well.
[10:38] And then, we also have a capability if you have partners that also give you data about customers, and we have a number of partners like um Acxiom, Epsilon, LiveRamp, and others. bit broadly about um you know, any sort of CDP, usually you've got integrations
[10:55] all around for lots of different things. Like how our um our partners are like really excited about this, being able to to plug into it. Um maybe talk a little about like what are these integration points and Yes. So, So, CDP works with data, right? So, the job is to figure out who needs to get what when. Yep.
[11:10] But, the CDP does not actually send the communications out. It doesn't send the email out. We're not an advertising network, right? We're not like Meta or Google. There's no SMS. Um these are all partners that we have. We have a very rich ecosystem, right?
[11:26] Dozens of partners. We had about 20 partners that we launched Customer Lake with. Very exciting. We have integrations with all of them. And so, the CDP once it figures out using the data in collaboration with the human marketer, who do we want to target to have the the
[11:42] best customer experience, relevant content, right? And what are the channels, we will send those customers with the relevant information to those partners, and then those partners will send a hyper-personalized email, so a a very targeted ad. And this is all first-party
[11:57] data, right? This is all data that the customer gave to the brand. Yep. And so, they expect those communications to be personalized, right? I mean, we as Databricks were not in the business of selling or exchanging data. What we're trying to do is to help um our enterprise customers collect the data
[12:13] from their own customers so that they can better understand them, and everything they send back to their own customers is truly personalized, truly relevant, right? So, think the opposite of spam. That's my life's goal. How can we use data, context, and agent so that
[12:29] um enterprises can do less, but much more relevant communications that are very impactful, and they're a win-win both for our enterprise clients, but also for their consumer or the business clients as well. Oh, absolutely. Yeah. That sounds really good. Yeah. And then in terms of like cuz we're also
[12:46] hearing Juneau Ontology, we're hearing AI Gateway, we're hearing um more things for getting security and ontology on top of your data. Um are there any things you recommend to Databricks customers that are using CDP like
[13:01] what what other tools kind of go with that in the Databricks platform that we've seen uh today? I think the um the Juneau Ontology is super relevant, right? Because uh context is very important for us, right? I'll give you a simple example. Um
[13:16] if you want to send a communication to your loyalty customers, right? You need to know who what is the loyalty program, who's there, when did they get Actually, operating a CDP is not that different from trying to get high quality insights out of your data.
[13:32] Right. The difference is when you try to get insights, it's just a human reading the inside. When you're using a CDP, that insight is translated into a campaign, right? Or an audience, something that will help you send more relevant communication. So,
[13:47] the difference between an insight and a CDP is that the inside the consumption is with a human, the CDP the consumption the campaign, that's doing something very relevant. So, a lot of the business concepts like who's in my loyalty program? What's my loyalty program even mean? Who's an
[14:03] active customer? Who's engaged on my website versus they're only coming to my stores. You know, if I'm a big bank, right? Who's a customer who has a credit card, who doesn't, right? Who's a customer of a private bank versus the retail bank.
[14:20] This is all ontology context. Yep. That you need to have well organized. And if you have it well organized, it helps you track business KPIs, it helps you generate insight, it helps you do marketing as well. Yeah. So, that's the beauty of what the thing
[14:36] we're doing here at Databricks is that we're we're bringing together um data, agents, and context, and now applications Mhm. all in the same platform, and the synergies all over the place, right? Whereas if you try to do CDP outside of
[14:52] the lakehouse, you have to figure out all the context for Databricks lakehouse, and then you have to figure out again for your CDP. Yep. Which would be a lot of work, and again, this could be a lot of bottlenecks, right? With the data and everything else in between. Yeah, Unity Catalog has been out for I
[15:08] think 5-plus years now, so people have already been registering all their data sets in there. They've been starting to define the metadata about the descriptions of these data sets even down to the column level. Uh Unity is also been like capturing a lot of like the lineage and the really And so now, you know, all of this uh
[15:23] context that's been getting acquired and curated within Unity is going to be leveraged by the CDP downstream. Exactly. And you know, you're touching to another great point, which is governance. That's right. Because the CDP is accessing some of the most sensitive data of the organization, customer data,
[15:38] right? You don't want anything bad to happen to the customer data. It's a mess, right? If anything happens, you have to be extremely careful. Very sensitive data some of this. Super sensitive data, right? And there's, you know, legal and reputational implications if anything happens. Right. So, the beauty of bringing the CDP and making it embedded inside Databricks is
[15:55] the same governance security mechanisms you're using for how people are accessing this data for reporting or insights can now apply for the CDP. Whereas if the CDP is separated from the lakehouse, you have to recreate a whole new governance layer
[16:11] Yep. in your CDP Yeah. to the lakehouse And then keep it in governance layer. I think you've been seeing, right? And and and this can be First of all, very time-consuming, right? It can take 6 months or a year to approve this new governance layer, right? It can be very slow for business
[16:27] teams. And and and it's also risky, right? So, we feel that's a huge advantage by making the CDP embedded, you get security and governance out of the box. Yeah. Okay. And then Tasso, that almost wraps up our session, but I just really want to know
[16:42] in like one or two sentences when cuz CDP's been around for a while, what do you want people to at home to take away from CDP at Databricks? Yeah, that the CDP, your your lakehouse is now your CDP. Okay. Right? That's the That's the big
[16:58] evolution of the category. I want that on a shirt. Your lakehouse app The lakehouse is your new CDP. Yeah. Perfect. Well, thank you so much for joining us. Thank you. It was a real pleasure. Yeah. Really appreciate it. Thank you. Yeah.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.