lululemon: How We Modernized Our Audience Segmentation and Where It's Taking Us With Agentic AI

Summary

lululemon's CRM Audience Data Science team operates PAVE, an ML platform scoring millions of guest profiles for personalized marketing, and this video describes how adopting Databricks Asset Bundles cut their deployment time from a fragile two-week manual process down to two days.
A multi-stream feature store architecture built on Unity Catalog with YAML-defined metadata, point-in-time features, and always-on features provides a shared foundation for all models, and MLflow 3.0 handles experiment tracking, the model registry, and model promotion across all streams.
The agentic AI roadmap includes automated pipeline failure diagnosis and a shift from threshold-based marketing segmentation to per-guest next-best-action optimization, building on the DABs, Unity Catalog feature store, and MLflow 3.0 foundation.

lululemon: How We Modernized Our Audience Segmentation and Where It's Taking Us With Agentic AI

At lululemon, our CRM Audience Data Science team operates PAVE—an ML platform scoring millions of guest profiles for personalized marketing across email, push, web and paid media. With growth came pain: data scientists managing deployments instead of building models, no path to production, and marketing waiting weeks for new segments. The bottleneck was never the model. We came to DAIS 2025 with these problems and left with a plan: DABS, Unity Catalog, and MLflow 3.0 as the foundation for a live system. Key areas we will cover: DABS: parameterized pipelines that any team member can deploy in one command Multi-stream architecture where each new model type inherits the same pipeline pattern Unity Catalog migration seeding an enterprise feature store now scaling beyond PAVE MLflow 3.0 for unified tracking, registry and model promotion across all streams Agentic AI roadmap: next best action for marketing partners and generative model monitoring
Talk By: domenic fayad, Senior Data Scientist, lululemon ; jeroen ruissen, Sr Manager - Data Science, lululemon athletica

Chapters

FAQs

What is the PAVE model at lululemon and what does it do?

PAVE is lululemon's ML platform operated by the CRM Audience Data Science team that scores millions of guest profiles to determine the likelihood of engagement for personalized marketing campaigns across email, push notifications, web, and paid media channels. The platform produces ranked lists of guests for marketing teams to target with specific communications. This video explains how PAVE grew from a single model into a multi-stream architecture with multiple flagship models sharing common preprocessing and feature infrastructure.

How did Databricks Asset Bundles improve lululemon's deployment process?

Before adopting Databricks Asset Bundles, lululemon's deployment process was a fragile two-week manual effort that required data scientists to manage deployments instead of building models. DABs introduced parameterized pipelines that any team member can deploy in a single command, cutting deployment time from two weeks to two days. Each new model type in the multi-stream architecture inherits the same pipeline pattern, making it straightforward to onboard additional models without starting from scratch.

How is the Unity Catalog feature store structured for the PAVE platform?

The feature store uses a layered architecture with YAML metadata that defines each feature's lineage, refresh schedule, and availability pattern. Features are categorized as either point-in-time—computed at a specific historical moment for training—or always-on, available in real time for scoring. This architecture provides a shared feature foundation for all PAVE models while enabling the Unity Catalog migration to scale beyond PAVE to other data science teams across lululemon.

What is lululemon's agentic AI roadmap for marketing personalization?

lululemon's roadmap includes two main agentic AI directions: automated pipeline failure diagnosis that detects and resolves issues in ML pipelines without manual investigation, and next-best-action optimization that moves beyond threshold-based marketing segments to personalized per-guest recommendations. The per-guest optimization approach would replace assigning guests to segments with a system that determines the single best marketing action for each individual. This video presents these as the natural next evolution beyond the foundation built with DABs, Unity Catalog, and MLflow 3.0.

Full transcript

[00:07] All right. Hi everyone. Uh today we're going to talk to you about how we modernize our audience segmentation. And where does this take us? >> Before we get started, I'm going to start off with something that's a little bit Lululemon. We call it a clearing. So, a clearing is
[00:19] the practice of acknowledging anything that's kind of preventing you from being present. My own personal clearing, it's 5:20. It's been a pretty long day. Pretty hungry, a little tired. Mindful that everyone here might be a little hungry and tired, so we'll try and keep people
[00:32] awake today. Um we have two hopes or intentions for today. The first hope is that we can share some stories that hopefully bring some energy and um can inspire.
[00:44] The second is that you think these guys are like, you know, kind of cool and maybe you go buy some Lululemon when you go home. Okay, so behind me is actually a photo of our very first store. It's from
[00:57] Vancouver, that's where I'm from actually. Uh it's West 4th store in in Kitsilano. You've probably interacted with our brand before. Um you may have known us from our yoga pants. That's kind of was our claim to fame way back when. Uh in that time though, we've
[01:09] matriculated into a whole bunch of things. So, basically, if anything is sweat or community involved, you might find us uh with those products. So, you'll have seen us in golf, in tennis. We've done some collaborations with
[01:21] designers. We do a lot of casual wear. It's actually what we're sporting tonight. You think it's kind of cool. Um you've also seen us at the Olympics. You may have some thoughts and feelings about that. Uh I loved it.
[01:33] Okay. Um who are these guys talking? Uh my name is Dominic. I'm a senior data scientist at Lululemon. I've been here for 5 years. And it might be fun to do like a little bit of an ad-lib, especially with the World Cup. So, my friends call me Dom. And my teams are Canada and Mexico.
[01:46] Uh Canada because that's where I'm from. Mexico because Canada was not very good for a very long time. And my mom is Mexican. Um so, still support them. And my goal is to run the Sea Wheeze, which is uh actually Lululemon's half marathon this
[01:58] August in about an hour 43. I'll pass it to my data scientist senior manager. >> Cool. Thanks, Tom. Hey, everyone. Looking forward to the next uh 40 minutes here. Um
[02:10] No one calls me Yeroon. That's just the closest like kind of phonetic way to pronounce my name. So, think of the J as a Y. Um My World Cup team is the Netherlands, which is where I'm from. Uh I believe
[02:22] we're the team that's made it to the final most often without ever winning the final. Uh and my current goal is to start learning piano just to do something quite different. I'm quite active and just want to like start learning something different and and new.
[02:40] Um So, we're the audience data science team. And what that means is that we build decision uh systems to improve personalization in digital marketing. Um As Lululemon, we're quite different than
[02:52] maybe your typical retailer. And that's because we really own the process, but with that we also own the data from stitch to sale, as we like to say it. So, we really own a lot of data across like our production process, but
[03:05] then we also sell 99% of our product through either our own stores or our own website. Uh and that's obviously quite different than some other retailers like that might be our competitors cuz they might
[03:17] sell a lot through Foot Locker, maybe a little bit of Amazon here and there. Whereas, we have a lot of insight on like what our guests are like currently browsing, what they're looking at, uh what emails they're responding well to, etc. And obviously, that gives us
[03:30] a competitive uh edge if we do use that uh data correctly. So, one of the core models that we want to talk about today and one model that
[03:42] we've like evolved a lot throughout the last year is like our pave model. Um Really, what pave aims to do is like if you look at all of our guests, it tries to like rank guests from like most engaged to
[03:54] least engaged. And then that's something that we pass on to our marketing partners, but also to like our like like app and website teams such that they can like meet the guest where they are currently at and make sure that
[04:06] we're personalizing content for them based on their like current like interests. Obviously, we use a lot of like Databricks like technologies for that. So, we use like Unity Catalog, MLflow, Databricks, and we store a little bit of
[04:18] data in Snowflake. Um what we want to talk about today is um how we use the lessons that we learned last year from the Databricks um Data
[04:31] and AI Summit, went home, and really like modernized our solution. So, last year we were developing PAVE uh to replace like an older model that we had. Uh and we learned a lot last
[04:43] year about like CI/CD, preprocessing, and also like a little bit about agentic AI. Today, we want to talk you through those three topics and like show you how we've kind of took uh taken the lessons from last year
[04:56] and actually implemented them in like a model that's running in production at the moment. So, the first thing we'll talk about is CI/CD. Maybe a show of hands. So, how many people here from Italy?
[05:12] One sort of. How many are from the States? Kind of figured it's I think about this morning. I should have probably picked a different analogy. So, um because it's 5:30, we'll talk about food.
[05:25] So, imagine two friends chatting with each other, and the first one says, "Hey, like I'm I just made some great pizza. Like let me send you the recipe." And the American is like, "Sure, sounds good. What do I need?" So, you just need some pizza dough,
[05:37] mozzarella, tomatoes, and basil. When an Italian says that, they're thinking of this. The American running to the store, making it night,
[05:50] buys pizza dough, buys mozzarella, buys tomatoes, and basil. And obviously, you can imagine there's a
[06:02] bit of a mismatch here between like what they expected and what they're getting. Um so, this is how the pizza turns out. And at this point, obviously, the Italian's thinking, "Oh, this is not at all what I imagined."
[06:15] And this is like this is a simple example that kind of tries to illustrate like putting your model from development into production. Where obviously, you have the recipe,
[06:27] and you can have instructions, but there's so much beyond the recipe that you need, and that's all in config, that's in cluster configurations, that's in environment variables, etc. It's also in ovens.
[06:40] It's in temperature, and whether you measure that in like Celsius or Fahrenheit. And if you think about this analogy, what we'll talk about first is Databricks asset bundles, currently called deployment automation bundles.
[06:53] Um and really what that aims to do is ship the entire oven along with all the ingredients into production, rather than simply sending you the recipe and letting you figure it out.
[07:07] Um so, we implemented Databricks asset bundles to like fix our own handoff problem. So, we're the data science team. We develop like an end-to-end solution, but then, to put it in production, we do hand it off to like
[07:19] the technology team. And obviously, what that means is if you think about like cluster setup, environment variables, etc. Um those are things that aren't captured in the code itself, they're captured in
[07:32] like configuration. So, what that means is the moment you ship your model to production, if we have made any changes to clusters or Spark versions or package versions, etc., then that's something that we need to pass along
[07:44] with the recipe. Um what that means is there's we had slow cycles. Obviously, there's a lot of like manual steps there. So that leads to like human error from time to time. And then we did also find that there's
[07:56] some knowledge transfer like with the technology team. We don't per se have like dedicated capacity. What that means is we have a different engineer supporting us the next week or the next month. We kind of need to like start from scratch a little bit there.
[08:10] So in short, like CICD was hard. Um this is an example of like what it looked like. So this is like the checklist that we wrote that we essentially send along whenever
[08:23] we wanted to put like a new like a model in production. So it goes from like um as you can see like access controls, like uh replicating cluster configuration, running smoke less smoke
[08:35] tests, etc. Obviously, you can imagine that's manual, fragile. It's not version controlled, importantly. Uh and it's something we had to repeat for every every release.
[08:51] Um income uh Databricks asset bundles. That's what it was called up until February. Uh Databricks changed their name, but they kept the abbreviation, smart enough. So currently it's called deployment automation bundles. We still call it DABs.
[09:04] And what DABs aims to do is provide like um provide uh customers like ourselves with a code-first way to build, manage, and deploy like your entire workflows.
[09:16] And because it's deployed as code, that means we kind of tackle some of those challenges that we that we talked about before. So it is deployed as code. It is version controlled. And it we currently
[09:28] use it to like define global variables as well where like there are some tables that might look a little bit differently in terms of like naming or even like column names for us uh between dev and prod. Maybe it's not best practice, but
[09:40] it does happen in real life and we do use like the dev stairs as well. What we found is that the actual step of deployment with that has gone from about 2 weeks to about 2 days. So obviously
[09:54] that's a huge improvement. Just wanted to give you like a little snippet of what that actually means. So it means that we use like three different files that together make up like a Databricks asset bundle. So the
[10:06] first one is databricks.yaml. This is where you like define all your clusters. So as you can see there we have like a different cluster for ETL and then for actual model training. That's just cuz we don't need like the same like compute
[10:19] for ETL as we do for model training. Then we also have a space to like indicate like some environment variables. We define our targets from dev staging to prod. So this is really where we input like specific like
[10:33] Snowflake locations for like those three environments all the way to like specific cluster and like policy configurations for um Then the second part Sorry. The second part is workflow.yaml.
[10:47] So that's a parameterized way to actually define your workflow. So currently in production we have a few different workflows and all of them are defined in workflow.yaml. um And then deploying this solution really
[11:00] is as easy as like shipping the entire asset bundle to QA. Potentially do like a few manual like changes there, but far fewer than before. um
[11:12] And then hitting the deploy button that you see all the way on the right there and then that should deploy it in QA. If all goes well, we can ship it straight to prod.
[11:25] So just kind of rehashing here, but obviously before we had like a very long and manual process that was not version controlled. Currently um, we have all of that captured within like the Databricks asset bundles and that's uh really made our life a lot easier.
[11:38] Um, and it's made our Lululemon a lot happier. With this, I'll pass it on to Dom to talk about our second challenge. >> All right, thanks. All right, so I'm going to talk a little bit about preprocessing.
[11:51] And I think maybe we're still thinking of food. Maybe we're a little tired. Let's wake up a little bit. Um, let's build on the example of the Italian and our American friend. So, American friend has now figured it out. Okay, he's made a really nice pizza. It's got the San Marzano's. It looks
[12:03] beautiful. Gets a nice compliment and maybe takes it a little bit too far and thinks, you know, now that I figured out how to build in the kitchen, I want to build it out in a pizza truck. And so, you go to visit him as the Italian friend in about a week
[12:16] and it looks something like this. And so, the truck looks nice, but maybe the first thing you realize is that there's no menu. So, how do you discover what he's selling? But also, he's got a line. So, obviously, he's attracting some folks. But as you can see, some folks in the
[12:28] line kind of look like they're waiting. It turns out you do have to wait. Took about 30 minutes, which is odd because, you know, the pizza should only be there for like a minute. Eventually, it's your turn and and and you get something that looks like this.
[12:40] I think the point here is that with preprocessing, I think it's you see in practice you can build it for yourself. As soon as you start to scale, it gets quite challenging. And so, we noticed that for preprocessing, we struggled with
[12:53] scaling preprocessing across different models. So, when I say the word preprocessing, it's going to come up a bunch. A few trigger words all I will warn you. Um, consider it the processing of a process of making data ready for machine learning. So, I think a few of us are all of us
[13:07] are probably quite familiar with this, but just to paint the picture for for simplicity, model pipeline looks something like this. We have the model. The first thing we're going to do is ingest data. So, your data could be in transactional sources, it could be in your email data, could be website data,
[13:21] push notifications, you you name it. Preprocessing is the process of cleaning, integrating, transforming, aggregating, testing, subsetting, selecting everything. Everything before the model. So, the next thing you do is hopefully
[13:34] you have a model that's not, you know, some pickle file on your on your local machine and it's uh it's an MLflow and it looks nice. And for inference, it's basically the same thing. You just basically take the model and then you throw your dart. >>
[13:46] >> The thing about pre-processing though is it always looks so nice when you see the table and it's ready to go. It looks amazing, right? You have all your features, it looks beautiful. And most demos that I've seen from from our engineers, it looks like this. The reality though is when you kind of go through the code, it looks very
[13:59] interesting. Sometimes you have like a single hop where it's just like a giant SQL query pulling all your features all at once. Sometimes it's staging tables that you join together. Sometimes it's like three versions of the exact same code that have evolved over like
[14:11] multiple years. So, it gets a little messy. And that's what we started to notice at our organization. So, on our teams, we have several flagship models. We've talked a little bit about PAVE. We have other ones like LTV and and you name it. We basically had the same pre-process
[14:24] pre-processed up high across three different projects. And it was interesting, you you go check it out and you're like, "Oh, that's a version that was from before. That definition actually has changed." And this is quite common across teams. So, to summarize, there's five reasons
[14:37] why we couldn't scale effectively. The first is the time it takes to pre-process is quite a long time, more than just going for lunch. The second is the source data limitations. They could exist in a bunch of different places. How do we build
[14:49] something that's modular and can kind of work across different uh sources? Third is compute duplication. If I'm running something for to run inference and another team's running something to run inference on a Monday and a Friday and we could have just organized, you
[15:01] can see how that can be a pretty efficient process. Onboarding tax. I think everyone's reviewed their own code. It's not usually the most pretty and proud moments. Uh it's even harder when you're showing someone else your code.
[15:13] And then of course the lineage opacity. So, if you can't see what's coming into your sources, it can be quite challenging. So, I'm a data scientist, I'm not an engineer. Um but as a data scientist, I think this perspective is important. So,
[15:25] from my perspective and I think for most data scientists, we want clean, fast, reproducible model and inference-ready features. So, I'm going to say some words that are going to trigger, hopefully. Um I will warn you right now that word is a feature store. And the reason I think
[15:38] it's a bit of a triggering word is because I feel like this word has been coming up in conferences and in conversations for years, and yet it's still kind of not solved fully. So, that's my sincerest apology to talk about feature stores.
[15:53] For those who have been to Japan, I just recently came back from one. I am a 30-year-old man who went for the first time and did the classic big three recently. So, I think that's like kind of something I've seen all my friends do recently. You will see these kinds of machines in some places. Um they're amazing. Um
[16:06] I kind of made me think of what a feature store should feel like as a data scientist. Sometimes you just want to go and order your features, and they're ready to go. It doesn't mean that you have to eat it as is. You can still garnish it. You can still add some things to it if you want. You take things out, but it's the base
[16:19] features that are kind of prepared for you. And so, speaking from the perspective of a data scientist, I'll walk you through the architecture that we built for our feature store. The very first layer we would consider source. So, source can live anywhere.
[16:32] Snowflake, Unity Catalog, AWS. It doesn't really matter. The second stage is the canonical facts. So, the easiest way to think about this is we're going from a really wide or long data set to a little bit wider. So,
[16:44] we're pivoting on some things to help for aggregation. And this is what's This is what our feature store is going to interact with. And the feature store is just Python and YAML is is a good way of thinking about it. So, YAML kind of controls all the definitions, all the tags, anything
[16:57] about the features themselves, and Python essentially just wraps it all and and allows us to do all the things you might want to do with it. Parameterized quite heavily. So, there's nothing actually stored in layer three. Layer three just interacts
[17:09] with layer two. In layer four, we have kind of a A and B situation. Quite frankly, it could just be an If you're thinking of this, this could just be one thing. It very It could be, but I do think from like an illustration
[17:21] purpose perspective, I quite like it separated. In 4A, think of this as the data scientist owned version. There's nothing actually materialized until you call it. So, we're going to go through a demo in a minute and to get a feel of
[17:33] what this looks like, but essentially you're conditioning on the audience and the date. So, you you can basically go to that ramen uh vending machine and make that order based on whatever you choose.
[17:48] The second is definitely not owned by the data scientist. This is owned by the engineering teams and this is just like your always-on inference tables. So, the same set of ingredients, but always on and ready to go.
[18:00] So, you have your point in time, which is your 4A, which I mentioned, and then you have your always-on. The point in time is something you can call and allows you to basically condition as you like, very flexible for a data scientist that may want to train different kinds of models. Always-on is available for everyone.
[18:14] A lot of words, so I'm going to pivot to a demo and show you guys an example of what it looks like in practice.
[18:34] >> Probably killed the presentation. >> Yeah.
[18:53] >> So, I said it's a demo, but I'm essentially walking through a notebook that's already ran, so it kind of takes away some of the potential errors out of this. So, this notebook's a pretty simple notebook. Let's think from the perspective of a data scientist. I just want to observe, see some discoverable
[19:06] features, and then potentially run some basic feature store type functions. So, the first thing I'm going to do like any other time I'm loading up a notebook, is just import my library. So, I'll make this a little bigger so we can actually get a better look here. All I'm
[19:18] doing is importing the library. After I've imported the library to the cluster, I can have a look and just pick the exact uh the methods that I want to keep.
[19:31] And when you're building a model, the very first thing a data scientist is going to have is obviously go through the process of selecting the who and the when. In this case, let's pretend that's in care of us. So, take care of. So, we already have our data set. We have two columns. We have the entity, so the who,
[19:44] and the date. So, here we have 100,000 records. We can see we have almost 100,000 distinct entities. And it's spread across 407 dates. You might want to do your own pre-validation. Here's a quick view of
[19:57] what that could look like. So, here we just kind of done a little bit of a exploration of what we're looking for. So, we're going to look for transactional features, and we're going to look for email features, but based on just those two things that we're conditioning on.
[20:13] Now, it's time to run the point-in-time feature store. So, very simply, as you might hopefully do, is just explore what the functions or the methods are all about. This is a pretty simple uh thing that I think most agents are quite good at, uh summarizing your function and what it's actually doing. So, most
[20:26] data scientists read these kinds of things. I would hope. And there's two ways to run run this feature store that I'm going to show. The first is the lazy method, where I'm essentially just running the feature store and storing it as a PySpark data
[20:38] frame. I'm lazy loading it. So, you can see here I've ran it. I've given it the the spine or the data set I'm conditioning on, and then the domains. So, I'm requesting the transaction feature store. And I'm going to lazy load 10 records. So, you can see
[20:51] the steps it's going through. It has sequences that it's following. As an example, we probably don't want to calculate ratio features before we actually get the actual features. So, this is sequence that matters. After we've gone through all that, there's a bit of a summary. It shows you
[21:04] which features you've selected. In this case, we've requested all the transactional features, and you can see all the features that kind of come from them. So you probably don't want to just grab 10 records and store it in memory all
[21:17] the time. That doesn't make sense for all use cases. Sometimes you're going to have major downstream consumers like pave like we talked about. And so for that case it might make sense to specifically call out which features you want to grab, right? We may have already done a feature selection and found out
[21:29] which ones are important which is versus which ones that aren't. I shouldn't have to run that feature store against all the features if I don't need them all. So you can see here I've already set up this service. We're calling it the pave service. There's the entity that's already
[21:41] described. Essentially the main thing to take away here is that you can parameterize this very heavily. It's Python. It's very flexible. And instead of writing it to memory we're basically going to write it to a table a union catalog table.
[21:55] Basically the same process. The only difference here is instead of calling a random uh a random set of domains we can pre-specify what those are and exactly what we want to look for. For example, we might want to have certain horizons horizons like the last 7 days, 90 days,
[22:08] 180 days, 365 all that kind of stuff. After it runs it'll run both feature stores that we've requested. In this case you saw earlier it was email and transactional. And then it'll basically materialize it
[22:20] for you. And here's a quick little view of what the features kind of look like.
[22:32] Um candidly one of the things I like the most is the metadata uh structure that I learned from the tecton uh stack which I believe is required by Databricks. Pretty confident it was. Um what's nice here is instead of thinking of it like commenting your columns on a table is moving away from
[22:44] that and building like a metadata schema with all the information about your tables. This is quite nice from a discoverability perspective. I don't know anything about features from a specific domain. I can just go to the metadata schema and see what's going on
[22:56] there. So I'll give you a quick example of what that looks like. So you can see here I'm taking a look at a couple of Spark tables, the email registry and the transaction registry. So I'm going to display a few records from that email registry. And what's
[23:09] nice here is I can take a look at every single feature, and rather just seeing like one column description, I can essentially have anything I want to describe this. So here we've tagged it, you can see a description, we've got business logic,
[23:21] the basically anything you'd like to have. This is quite helpful. I should add if you're taking photos, this is all dummy data, so that's quite important to know. And yeah, so the key thing here is it's
[23:34] quite flexible and very easy to use in this feature store, and it enables us to scale across projects.
[24:05] To summarize, we've talked about deployment automation bundles, and how helpful that's been for us with CI/CD. Talked
[24:19] Over the last year, I think last year in particular, we were really impressed by some of the projects we saw here on MLflow 3.0, it was new at the time. And in that time, we've started to use it for our paid flagship models. And what's been quite nice about it is the tagging, quite flexible tagging that allows us to
[24:32] play with the different kinds of tags you want to have on a model, but also the shareability on Unity Catalog. It's been quite nice to have it associated to a schema.
[24:49] Cool. All right. Um we'll finish up by like sharing some thoughts about like agentic solutions that we were, frankly, thinking of implementing. And then both of them were more or less announced by like the CEO this morning of Databricks and we're like, at least we're thinking about the right things. We're maybe
[25:02] we're not moving at the right speeds, but we're thinking about the right things. Um, so when it comes to agentic use cases, there are kind of two that we wanted to talk about. So the first one is like a Monday morning, you wake up,
[25:17] your training pipe- pipeline failed. Obviously, that means you'll probably spend the rest of that day, if not that week, investigating why it failed, trying out a few different solutions,
[25:29] etc. Um, what we were thinking is actually more or less in my in line with what like Databricks like showed this morning. So, currently like
[25:41] our testing notebook set up more or less as a smoke alarm. So, if something goes wrong, it goes off. Monday morning, we come in and we need to like figure out why it went wrong. Where we want to go
[25:54] is something that's more of a fire handler. So, let's say if something goes wrong in the notebook, honestly, I think 95% of the instances, it's because of like an upstream like table that like loaded a little later because they probably had an upstream
[26:06] table that loaded a little later, etc., etc. Um, and these are things that in our view too, like obviously agents can diagnose. Um, say they found out at like 2:30 in
[26:19] the morning that it failed because of like an upstream table, the best thing like this agent can do is obviously try to like try to load data from the table again at 5:00 a.m. If it works 5:00 a.m., just
[26:32] kick off like the model pipeline again. Um, so more or less what we saw this morning is exactly what we were thinking. So like diagnose by like executing like the the queries, like
[26:45] analyzing the data, seeing where it went wrong. Then from there, um, from there obviously there are different criticalities so like some are maybe like warnings
[26:57] uh some are like errors and should be breaking errors and then from there also take actions to like maybe trigger the pipeline again or maybe send like a notification through teams uh to one of us and I think more or less that's what was announced this morning so we have
[27:10] more room on our backlog. >> All right, this is the last slide and we can think about some food after that.
[27:24] So next best action to segment to your seg- segmentation to next best action that's kind of like a pretty broad thing. So I will talk about our journey towards this and try and paint a picture of what it could look like. Um over the last couple of years we built a
[27:36] relationship with our our stakeholders mainly the retention life cycle team and they've been using our models and have matured themselves to using the like the predictive segmentation uh to improve their efficiencies.
[27:48] Basically what it looks like from a stack though is something like a predictive feature store. So I kind of like one of the talks I listened to earlier today from Pepsi where they talked about moving their predictive and their model their predictive features and their models closer to their data. I quite like that and I think that seems
[28:01] like that's the the trend that's going on. We've done something like this as well so we've basically built this predictive feature store for that's enabled us to build some segments. So anything from like you're likely to buy you're likely to engage with channel some distinct shopping
[28:14] preferences. This is very much a process where there's a human in the loop. And it works but there's quite a bit of like give and take. So for example there might be a question about how many folks make up X segment one if we change X Y
[28:28] and Z if we change a threshold what if we include a new attribute and so that process does take quite a bit of time although it is effective.
[28:40] The next step is not as inspirational but is maybe a an idea of what it could look like. So instead of moving towards straight to like next best actioning one intermediary step could just be have having agents reason over threshold curves. So, as an example, if we
[28:52] consider the email channel and working with working with the folks that send out emails, um they currently have segments that are pre-conditioned on some attributes, and one of them might be like a threshold. So, imagine for example, we had the likelihood to
[29:04] purchase over the next n number of day. Setting the threshold to 0.7, we know it's going to have some sort of impact. It's either going to decrease or increase some segments, and there's going to be some sort of effect. And we can use uh methods to go back in
[29:17] time do observational analysis to see what it would have looked like, and that process does include a human in the loop. But, what if it didn't have to? What if it could be a simple UI where we could run the campaigns we're doing what-if analysis on, and provide those those cuts for the for the teams using
[29:29] them. So, not the most revolutionary idea, but perhaps we can go a step further and invite a different kind of question. What if we didn't have uh content, or we didn't have anything as the constraints?
[29:41] So, what if we relax all constraints, what does it look like? Well, maybe we can move away from segments entirely. Maybe we can start asking what does the guest want instead of just what does the channel need? So, as an example, if we had inputs such as paid media, email, push, or we had
[29:54] like a content strategy or content schedule, what if we could pick what is the most efficient thing for the guest for them to increase their lifetime value and their experience with us? We're a premium retailer after all, we need to know our guests, and so that's
[30:06] quite important to us. And so, this is one of the research questions that we're thinking about right now, and how we're trying to support our guests.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.