Building a Scalable Customer Data Foundation with Identity Resolution

Summary

This video explains how LiveRamp Embedded Identity within Databricks resolves and unifies fragmented customer touchpoints across channels without moving or exposing data outside the existing governed environment.
Three common hurdles addressed are the identity tax from fragmented consumer data, the privacy barrier of sending data outside Databricks for enrichment, and the modeling gap where analytics investments fail to deliver expected outcomes.
By combining a Databricks data foundation with LiveRamp's Ramp ID household-level identity graph, teams can create AI-ready unified customer profiles at scale that reduce wasted media spend and enable agentic AI applications to act on identity-resolved data.

Building a Scalable Customer Data Foundation with Identity Resolution

Identity resolution is foundational to effective customer360, AI, analytics, activation, and measurement. However, frequently shifting privacy requirements and fragmented data make it harder than ever. In this video, learn how organizations are using LiveRamp Embedded Identity within Databricks to resolve and unify customer touchpoints without moving or exposing data. We’ll demonstrate how to extend this solution to enable secure data collaboration, improve match rates across partners and channels, and create a data foundation for AI-powered marketing— grounded in trust. Join us to see how top brands are modernizing their workflows to turn fragmented data into exceptional customer experiences.
Talk By: Lisa Cramer, Director of Product, LiveRamp ;

Chapters

FAQs

What is the identity tax and why does it persist in modern data environments?

The identity tax refers to the wasted cycles and inaccurate analytics that result from fragmented consumer touchpoints arriving from an ever-growing number of channels and interaction points. As described in this video, string matching and probabilistic modeling can approximate identity resolution but rarely achieve the accuracy of a reference identity graph.

How does LiveRamp Embedded Identity work inside Databricks?

LiveRamp Embedded Identity runs the identity resolution process directly within the customer's Databricks environment using packaged clean rooms, so PII and governed data never need to leave the platform. The Ramp ID—a household-level identity graph with best-match metadata—is appended to customer records in place, preserving governance and eliminating the latency of external data transfers.

What business outcomes can organizations expect from identity resolution on Databricks?

According to the chapter labels in this video, identity fragmentation contributes to 20–40% wasted media spend, 48% audience mismatch, and up to 3x customer acquisition cost. Resolving identity into a unified customer profile reduces these inefficiencies and creates an accurate data foundation for AI-powered marketing and agentic applications.

How does identity resolution enable agentic AI applications?

Once customer touchpoints are resolved into a unified identity-enriched profile within Databricks, AI agents can act on that data with the confidence that they are working from an accurate, de-duplicated view of each customer. This video presents identity resolution not as a point solution but as foundational infrastructure that agentic AI applications depend on to make reliable decisions.

Full transcript

[00:10] Okay, I'm excited to get started. Um, so I'm Lisa Kramer. I'm director of product at LiveRamp and I'm really excited to be here today to talk about building a scalable data foundation with identity
[00:22] resolution. Okay. So, I get to talk to clients and partners a lot of the time about how to think about their data foundation. And
[00:34] oftent times when I'm talking to clients or partners that use data bricks, that foundation is set. You have the compute, you have the data, you have the governance in place, but oftent times uh
[00:48] predictable analytics kind of remains out of reach. And so I want to talk about um kind of the three primary hurdles that I hear from folks um throughout our time together today. The
[01:01] first is what I would categorize as the identity tax and that is really around fragmented consumer touch points coming into your data environment. Um and
[01:14] oftentimes what I hear is like we can stitch this data together ourselves. uh we can figure out string matching or we can deploy some um propensity modeling or some sort of probabilistic modeling
[01:27] um to create something that looks like identity resolution. Oftent times what happens is that is cycles and cycles of trying to figure it out and and really never getting to an accurate view like
[01:40] you can with a reference graph. The second challenge I often hear is really around the privacy barrier. You have your data set in data bricks. You
[01:52] have the governance in place and so it really doesn't make sense to kind of take that and send it outside of your own environment to get an identifier appended. Um that introduces risks. It
[02:05] introduces latency. Um and it really doesn't allow you to keep that governance in place. And then the third key challenge I hear um that marketers often feel the most is really around the
[02:19] modeling gap. And this is um really the perception around we've invested a ton of energy and resources into creating this data foundation and we're not seeing the outcomes we'd expect. Um and
[02:33] so those are the three key areas that we're going to talk through um as we go through our time together today um a little bit deeper. So the first one is identity. This is
[02:46] still a challenge. It's been a challenge for a long time. Um but the reason it's still a challenge is identifiers continue to come in through a variety of different sources, more channels, more
[03:00] interaction points with the consumer. and it becomes really hard to understand how to stitch that data together in a way that truly brings value and meaning to the data. So, as an example, you may
[03:13] have a data source that is coming in from instore transactions, a different one from um app activity, a different one from maybe like a promotional offer
[03:25] or something like that. And even if they all collect PII, you still have challenges in being able to stitch that information together.
[03:42] And this creates actual impact. You know, uh 20 to 40% of programmatic spend is wasted. um up to 48% of audiences are mismatched and you can see up to 3x higher customer acquisition costs uh
[03:57] just from fragmented identifiers. So really a a lot of impact for marketing teams that are trying to prove that you know every dollar counts.
[04:09] Okay. So I'm excited to share that live ramp has launched our embedded identity capabilities within data bricks. We are live within the packaged clean room
[04:21] capability. Um and I'll show a demo here in a little bit. What this means is you have access to our identity graph on demand as you need it within your own environment. And this ensures you can
[04:35] connect those fragmented touch points to the best representation of a person or a household. And this also ensures you can use your data foundation in ways that
[04:47] are AI ready, right? You can create more accurate results from your modeling or um thinking about measurement insights. All of that can occur within your own
[05:00] data bricks environment. We're really excited about this for a number of reasons. Um, we've heard from a lot of partners, uh, uh, that, you
[05:12] know, data bicks is where all of their data is. Um, and I think, you know, uh, the data foundation that can be created within data bicks ensures that you know
[05:24] what data you have. It ensures you can query that data and it ensures that um you can have governance around the data. And so that creates a really nice
[05:36] foundation that that really can be deployed effectively downstream once identity is applied. And so data bicks creates the data foundation but live ramp brings our
[05:49] identity capabilities and together uh that ensures you can have scalable impact for your business.
[06:12] Okay. So what does live ramp identity do? Um we are built on an offline reference graph and have been for decades. Um and this allows you to resolve data to the
[06:26] best representation of a person or a household. It also gives you metadata. Um we call it best x for y, best email for this individual or best address for
[06:40] this household. Um, and all of this allows you to create customer profiles that are holistic in nature, right? All in the offline realm. You're able to connect all of those PII
[06:52] touch points um to the individual or household. But then data is more powerful when it can be connected into the ecosystem and that is what ramp ID is built for. Ramp
[07:05] ID is encoded for your specific business, but then can be translated to any other destination or partner within the ecosystem. In fact, Live Ramp works
[07:17] with I'm sorry, my um clicker isn't working very well, but Live Ramp works with over 900 uh partners throughout the ecosystem to
[07:29] help them build their data foundation. This includes all of the walled gardens. This includes all of the major social platforms and includes all of the top brands across these verticals. And so as
[07:42] you think about connecting your own data set into the ecosystem, this could include use cases around activating to downstream destinations. It could include things like collaborating with
[07:55] CMN's or RMN's, creating a CMN yourself and starting to monetize uh or syndicate some of your data. Um and so Ramp ID ensures you can connect across this
[08:08] ecosystem. In fact, it's one of our uh differentiators that we often tout our vast network of marketing and advertising destinations.
[08:21] Okay. Okay. Great. So, we've talked about um that first challenge, the identity tax. The second challenge was around privacy um and really wanting to keep all of
[08:33] your data in place. Well, with our launch in data bricks, you're now able to do so. You can natively resolve any identifier touch point. You can also natively translate that data as well.
[08:47] From there, you can use our data connectors to connect into the platform for activation or other advanced measurement use cases with premium publishers, all from your own
[08:59] environment. cool. So, now I'm going to walk us through a quick demo um of what this
[09:14] looks like. Um, in this demo, we are going to be a a an advertiser um that has a data set that we're going to take a look at here. So, it's a loyalty data set. It has some attribute information
[09:27] and some PII um based touch points. So all of this isn't synthetic data uh for demo purposes only, but um you can see there's a number of attributes in here and a number of different touch
[09:40] points that I want to be able to connect together. As I look through the email based data, I start to realize it's going to be really hard to do string based matching here. Okay, so let's take
[09:54] a look at how many records this table has and we'll use that as we drive an outcome towards the end. Okay, so we have a thousand records here that we're starting with. Again, all synthetic
[10:07] data. Additionally, I need to tell live ramp what to do with this data set. So I have a metadata table that tells some information around an input table, the
[10:19] execution to run PII and what column headers to look at. From there, I go into clean rooms and you can click on a live ramp tile. This means identity resolution is set up with a click of a
[10:33] button. You can actually just come in and um set the clean room up here and that will create um a clean room on the live ramp
[10:45] side. So I'm going to show the right now we're in the customer side. I'm going to show the live ramp side here in a second. And you can see it takes a minute for it to set everything up. Um, what happens during this process is
[10:58] we're setting up an endpoint for authentication and then we're setting up the appropriate permissions. So you can see we had it on the live ramp side as well. Okay. So while that happens, I'm
[11:10] going to go into one that's already set up and we're going to see I have my two input data tables here. This is um the loyalty data that we've already looked at uh includes PII touch points and then
[11:23] this is the metadata table that includes the config. So once I have that in there I can go in here and I need to add a parameter. The parameter is simply the meta table name.
[11:38] This tells live ramp what to execute on my data table. That's all you have to do. And once you do that, you can reuse that metadata table over and over and over as you run more data through this
[11:51] process. Okay. So here you're going to see it's running. Um uh you can start to see um the the progress in the runs tab and
[12:03] start to understand what's happening within the uh overall processing. This again synthetic data and demo but it's built for largecale batch data. Okay, so here we have our output table.
[12:17] Um, and you can see we still have all our attributes. Um, but now instead of PII touch points, we have ramp ID. And so let's take a look at look at the
[12:29] sample data. Um, and you're going to see same thing all of the attributes as well as some ramp IDs appended. Okay, so our original goal with going through this was to understand if there
[12:43] was any room to optimize this data table. And so I'm going to use this um feature to take a look at the data itself. Um and the outcome is going to
[12:57] be actually we have 250 duplicates within this data set. Okay. So 25% of my data is actually duplicates. That means I can
[13:10] consolidate that data. I can start to think about what are those attributes put together actually look like. I can have a better understanding of customer preferences. And then of course I can have more targeted advertising. I'm
[13:23] reducing the amount of wasted media spend by consolidating my data together.
[13:39] so we took a look at the identity tax problem. We took a look a little bit about kind of the privacy barriers. Um, and then we took a look at what it looks like in practice. The third challenge was around that
[13:51] modeling gap and really trying to drive outcomes from your data set. Um and so how I like to think about this is uh data bricks is the data foundation right
[14:03] it has um uh a way to organize data a way to query data it has all the governance that you can apply to your data set it tells you what data you have
[14:16] but identity actually tells you how to connect that data together. It tells you what data should be joined. Uh, and it allows you to have a join key across all of your customer data tables. And so
[14:30] using that on top of that data foundation or in conjunction with that data foundation really sets you up for scalable outcomes within the business. Particularly as you start to think about
[14:44] um agentic workflows, agents are going to start to take action on your data, but the data itself is going to drive the outcome. And so ensuring that you
[14:56] have a really great view of identity on your data foundation will ensure you can optimize uh those outcomes. Okay. Uh and I think this might be my my
[15:10] last slide. So um just a couple things uh to keep in mind. Um identity resolution no longers no longer requires you to move your data. It's available on
[15:23] demand as needed within your own environment. And you saw the setup was really straightforward and simple. Click of a button and you're able to set up the clean room with live ramp for processing.
[15:35] Second, privacy and performance are no longer in conflict. You're able to resolve data to ramp ID and then deploy into any downstream use case needed. Uh
[15:47] you can easily connect it to our platform with our uh data bricks connectors um and activate across our vast network of advertising and marketing destinations.
[15:59] And then finally, this is infrastructure not a point solution. As you think about uh setting the data foundation up for success, being able to understand what data to connect together is extremely
[16:13] important. It ensures you can have outsized impact on the outcomes downstream. Um we're really excited about our capabilities that are now live with data
[16:25] bricks. Um I'd love to hear from some of you. So here's um an email address where you can find me and my team databriggs live.com.
[16:37] Um would love to hear use cases you're thinking about, challenges you're facing um around identity resolution. As I said, building the data foundation with identity ensures you can be set up for
[16:50] success. Thanks so much for coming.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.