Sponsored by: Eclerx | From Legacy Workflows to Intelligent Retail Decisioning on Databricks

Summary

Despite having data lakehouses in production, most retailers still make decisions as slowly as they did five years ago because data and decisioning are disconnected—category managers may open 15 dashboards before adjusting a single promotion, often after the opportunity window has closed.
Eclerx's intelligent decisioning framework on the Databricks Data and AI platform layers agentic AI on top of scalable analytics foundations to enable dynamic pricing from live signals, autonomous promotional adjustments, and faster supply chain decisions without relying on overnight batch jobs.
This video presents AI-ready data designed for agents rather than humans, a domain decisioning layer including tools like Spend IQ and digital shelf optimization, and real customer results showing measurable cost reduction and decision speed improvements.

Chapters

FAQs

Why are retail organizations failing to make faster decisions despite having data lakehouses in production?

The problem is a decisioning gap rather than a data gap. As described in this video, retailers often have petabytes of data, hundreds of dashboards, and dozens of trained ML models, yet decisions still move at the same speed as before because there is no layer that translates data insights into autonomous action—a category manager may still review 15 dashboards before making a single promotional change.

What is the estimated cost of slow retail decision-making?

According to IHL Group data cited in this video, slow retail decisions cost the industry approximately $1.7 trillion. This figure reflects lost revenue from missed promotional windows, suboptimal pricing, and supply chain inefficiencies that could be avoided with faster, more contextual decisions on the Databricks Data and AI platform.

What is agentic commerce as described in this video?

Agentic commerce refers to AI agents completing retail decisions autonomously rather than presenting recommendations for humans to approve. Instead of a pricing model suggesting an adjustment that a category manager then manually applies, an agentic system can detect a live signal—like a competitor price change or inventory shift—and execute the optimal response without human intervention.

What does AI-ready data by design mean for retail organizations?

AI-ready data is structured and labeled for agents to consume directly, not just for human analysts building dashboards. This means organizing data with semantic clarity—including clear metric definitions, entity relationships, and decision-relevant context—so that agents on the Databricks Data and AI platform can interpret and act on data without additional transformation steps.

Full transcript

[00:08] Hello everyone. Thanks for joining in. I know it the last 2 days have been action packed for all of you, but obviously thanks for making time and seeing me here. Thank you so much. So yeah, probably boring topic from the
[00:20] namesake, but a little more intelligent decisioning framework that we have enabled for most of our retailers and CPG clients. We'll be talking about that in some details. Uh so definitely uh feel free to you
[00:33] know, then raise questions and queries and comments that you may have for us. More than happy to answer any of those. Cool. So by the time now that we are on day three of Databricks, I'm sure most
[00:45] of you guys would be having a lakehouse up and running in production. Perfect. But to be honest, how many of us are actually utilizing that the data lakehouse which is in production is
[00:57] making uh you know, your decision making faster or uh the business is able to take better decisions because of it. I don't know if if if that's the case and that's the gap that I want to spend the next 17 minutes on to be precise.
[01:11] I'm Rajasi Behere, head of data analytics and AI at Epsilon. We work with around 50 global retail and CPG brands on this exact problem. Across marketing, pricing, supply chain,
[01:24] digital shelf, customer engagement. So today I'll walk you through what we've learned through closing this gap. What's working, what's not, and where this is all heading towards.
[01:38] Cool. So here's something that I've been noticing around for the last 2 years. Almost every retailer we work with this say that okay, infrastructure is solved. Petabytes of data has been ingested, Unity Catalog stood up, governance in
[01:51] place, three dashboards for every business question you could ever ask. 40 ML models trained and waiting. Perfect. And yet, despite of having all
[02:03] of these things, we know that almost nothing actually moves faster than it did 5 years ago. Let me give you a real example. So, there's a category manager I was working with last year, and she runs a large
[02:17] beverage portfolio across North American market. She told me her Monday morning routine, "Rajsee, I come in in my office, I open about 15 different dashboards, 15. So,
[02:29] before she can decide on a single promotional adjustment, uh she's analyzing data, what has worked last week, what has not worked last week. And by the time she's done all of this analysis, the promo window she was
[02:41] deciding on has already closed. So, that's not a data problem. It's that The problem is that she has all the data, but there's a decisioning problem. So, all all of the ML more pilots that,
[02:55] you know, everybody's experimenting with lot of ML and AI experimentation has been done. Everything that that we they want to just go ahead and be live on the production environment, models work beautifully in a notebook, but then they
[03:07] absolutely die down when they hit the real operating environment. So, here's the line I would want you to hold on to for the next 15 minutes, that every every retailer that we have been speaking to,
[03:19] uh yes, they they do have the data lakehouse, almost none of the decisions are actually being enabled via data in a near real-time basis. So, that is something that we're going
[03:31] to be talking about in terms of how have we helped some of our retailers and CPG clients solve that entire problem put together. So, and by the way, again, you know, just don't go by what I'm trying to say.
[03:44] Uh it's it's about uh what what, you know, the research has got up as well. So, the top quartile retailers are now deciding 5 to 10 times faster than the median on things like pricing, assortment, promo. McKinsey and Gartner
[03:58] both track this. And And this is the part that matters, right? Because that velocity gap shows up directly in the margin. There's a measurable spread between the fast retailers and the slow ones, and it's widening every quarter.
[04:12] So, again, Gartner is saying that by 2027, basically next year, agentic decisioning becomes the baseline. It's no more a differentiator. Just the price of being in the game is something that you need to account for starting today.
[04:26] And IHL Group puts a number on the cost of getting this wrong. So, just, you know, about by the fear of not moving faster, what is the cost that we may have to pay around as well is in terms of $1.7 trillion
[04:39] are lost every year by global retail. So, on inventory, pricing, and shelf decisions made on that data, that is already, by the time someone looked at it, they you know, it was it was it was all stale. So, three numbers, one
[04:52] conclusion, standing still is genuinely getting expensive faster. So, what's actually shifting? So, there are four things which are actually shifting, and the catch is they're not
[05:05] happening one after another. They're all happening at once. So, yep, batch processing is now becoming streaming. So, if you've shopped on Amazon recently, you've probably noticed that
[05:17] the price of the item changes every few minutes. Uh So, that that's the dynamic pricing that we're talking about, which is based on the live signals, not overnight batches, and that's the new bar. So, siloed is becoming now more and
[05:31] more unified. Walk into a Sephora store today, the associate already knows what you bought online last week, what you returned, what you searched for for this in the morning. So, one unified customer view across every touchpoint. That used
[05:45] to be like a 5-year road map, and now it's like table stakes. Dashboards. Dashboards are becoming decisions, right? So, again, Walmart's automated replenishment doesn't wait for a manager to look at the chart. The
[05:57] system sees a signal, makes a call, places the order. The output of analytics instead from you know and it's it's really shifted from something a human reads to something a system does. And rules are becoming agentic. So, have
[06:10] you tried uh Klarna's AI assistant recently? You know, it it just answers questions. It actually goes and completes a purchase on your behalf. So, within guardrails, without you clicking through five pages, uh that that's
[06:24] that's uh you know, that that's not science fiction anymore, but that's now already shipped across as well. So, all four at once, which means whatever architecture you build, it has to handle
[06:36] every one of these. You don't get to pick which shift to follow. Yep. And that is the reason why we are just wanting to say that that everything has to move in tandem, and nothing just,
[06:49] you know, moves as slowly. Oops, sorry. So, now here is a little bit of a complex part in terms of how at Eklavya
[07:01] we have been able to solve this. So, this is the part where I just want you to stay a little bit on and try to, you know, just resonate in terms of what you would have also built and seen in your businesses growing as well. So, we'll
[07:14] we'll walk bottom up. Obviously, because that that's how it makes sense. So, at the bottom you do have the data workbench 360. So, that's like an accelerator, and that we have already built on Databricks, which is the foundation. So, lakehouse activation in
[07:27] just 4 weeks. So, ingestion, data quality, governance, cost optimization, self-serve analytics, 100 plus connectors which are pre-built out of the box. To give you a sense of scale,
[07:39] we are We're for one of our CPG clients, which is running over 300 production pipelines through this. And after we activated it, the DBU spend dropped by about 6% in the first quarter. Not
[07:51] because it it throttled anything, just because the cost optimization module that we have built for them that that ensured that the inefficiencies were actually eliminated and we are able to get their cost significantly down. Then
[08:04] then comes the Synapse AI. Think of it as a CDP, composable CDP as we used to call till yesterday, but but for the whole enterprise which this is something that we had built last year
[08:16] for our clients. Not just marketing. It is one customer view, one SKU view, one store view. And here's the important part. It's it's built so that AI agents can read and use this data as well. So
[08:28] they are not just uh you know, built out to be able to create out the entire uh data analytics consumption layer, but also we are making the data AI ready. So it's not just dashboards, agents also
[08:40] are able to use it as well. So this is what we mean and when we say AI ready data, uh it's it's by design and not you know, by fluke. Most lake houses today are optimized by human consumption. Clean
[08:52] tables, named columns, easy joins, but the next gap is making the same data legible to machines. So structured product attributes, normalized identity
[09:05] exposed via APIs that agents can call. We bake that in our AI 2, which is the the the layer that we are talking about and that is not as an afterthought, but we are actually baking in for the
[09:17] foundation part of it right from the beginning. Oops. Sorry.
[09:29] Uh uh On top of Synapse, we talk about the the domain decisioning layer as well. This is where the actual business decisions get made, and now we are talking business outcomes. Uh We are talking about Spend IQ, which is
[09:42] the marketing decisions and always-on MMM, audience decisioning, creative personalization. We do have our module Retail IQ, which runs the digital shelf decisions, search rank, PDP visibility,
[09:55] compliance around content, around hundreds of retailer sites. The pricing agent runs pricing and trade promotions. That is repricing the SKUs based on competitor moves, inventory demand signals.
[10:07] So, what what I'm talking about is that there are multiple agents, be it the forecasting agent, be it the trading agent, be it the pricing agent, they all function on the same foundation. So, different decision engines pick, you
[10:20] know, based on which foundation or which function are you would want to work us on, and then the point being is that it isn't a marketing tool or a digital shelf tool, it's an enterprise decisioning stack because the accelerators you start with is just the
[10:33] door you walk in through. And on top of all of this, the L4 that we are talking about is something that we call is in terms of ideas, the conversational layer. So, ask the question in plain English, get a
[10:46] governed answer plus a recommended action. So, instead of category manager opening 15 dashboards and trying to identify what is the next move that I need to be taking, you know, what how how I can just get an answer about which sales have done
[10:58] better in California last week. Now, the obvious question would be everything can be done through Genie as well. Yes, it could be done, but obviously for Genie also to be effectively functioning and having the context of all of the
[11:11] dynamics within an organization, you need a semantic layer, and that is something that we are able to build in from L1, which is a semantic and contextual layer as well that we have brought in as well. So, that is
[11:23] something that we would want to just, you know, speak about and emphasize that yes, Databricks will give you the platform, Databricks will be able to give you all of the tools, functionalities, but obviously e-Clerx will help you get to
[11:36] the decisioning layer for sure. Cool. So, that was a lot of content. I know that there are pre-built accelerators that we have spoken about, but then how
[11:48] does this everything work all together? How do we actually deliver this? So, yes, the same rhythm. Every accelerator, every function, three beats.
[12:00] Earlier, it was all fragmented, manual, slow, reports waiting for the next business review. But, now we have reimagined those business processes. We have now got unified agent agent tech and closed-loop
[12:14] workflows, and we are now able to detect, diagnose, and recommend, and actually help organizations act in minutes. And the outcome is is here to witness in terms of the measurable, scaled,
[12:26] repeatable. And the actions now have been taken in the downstream systems, and it's not just the slides that get delivered to the leaderships, but it's about the actions and the business outcomes that the organizations are reaping benefits from.
[12:39] So, yes, hold this pattern into your head. So, when it's marketing, pricing, shelf or supply chain, uh it it it follows this exact shape.
[12:51] And we would just want to give a simple example of how we have enabled it for for one of our customers. Uh and and before we get to the case study as well, the the real quick snap on in terms of how this works, and
[13:05] uh when it when it doesn't always even work for everyone. So, what is what is it that you need to actually keep in mind and not just use it as a simple uh playbook in in terms of point one, we decompose. So, we don't
[13:17] start from what the platform can do, but we start from what decision is broken, who makes it, and what's it worth. So, 25 years in retail operations that e-clerks has absolutely been proud to be
[13:31] working as a part of the organization. Absolutely, we know that what's the difference between a problem worth solving and the one that just looks important. Two is we build. So, everything I just showed you on that entire stack of L1, L2, L3, L4, that is
[13:45] pre-built, pre-integrated, pre-governed. So, custom builds that would take about 9 months earlier, we activate that in 9 weeks. So, that's not because we cut corners, it's because the IP already exists. And three that this is this is
[13:58] the one of the most people that that miss around is that we orchestrate as well. That the output isn't just a dashboard because it's an action that takes place in your PIM systems, your OMS, your MarTech, your pricing engine.
[14:10] This is the part where most data and AI projects die. It's where a model that worked in lab fails in the real operating environment altogether. So, you probably know this pattern. The executive deck shows a beautiful POC. 12
[14:23] months later, you're asking, "Why hasn't it scaled?" Almost always the answer is is this layer, the orchestration, because the decisions are being made, it is being put up on the board presentations, everybody clapped around
[14:35] it, but then where's the outcome? Because nobody actioned on the recommendations that were being given by any consulting firm altogether. So, we'll just quickly move on in terms of where we have made the real
[14:47] difference altogether is for a global retail brand and that was something that we did for about 60 plus markets. We worked for three categories predominantly and four channels. So, the marketing budget was obviously running
[15:00] in hundreds of millions and the CMO came to us with a question that honestly I hear in in in some of the version in every single week in terms of how do we move our marketing decisioning from quarterly review cycle to something
[15:13] always-on. Uh well, the challenge was not very simple to be tackled, but from a top-down approach and a bottoms-up approach as a hybrid combo that we have been able to bring up with to be able to say that yes, MMM always on MMM more than happy
[15:27] to help you. So, what was being done quarterly, now we are able to, you know, bring it back in house as well for our customers to be able to get done that in a weekly a rhythm. And it sounds familiar that earlier it was a quarterly
[15:40] cycle, now I want to be able to control in terms of what is it that you know, I can actually optimize on a weekly monthly basis from a campaign spends perspective and what is it that my CMO would want to understand as well.
[15:52] So, I would say that definitely the money went out but the results were not actually optimized when it when they came back. But using Databricks features and all of our advanced analytics capabilities, we have been able to bring
[16:05] that entire piece of being able to get all of the unified data in one place, building an MMM module on top of it and then the the client was able to then get the entire benefit of about improvement
[16:19] about of the met media efficiency gain of 8628%. The 80% plus campaigns were automated because then the decisions that were being mentioned out, they were just not like a black box version, we actually orchestrated for them as well.
[16:32] So, we helped them that the strategy is no longer on the the PowerPoint presentation but we are able to get them into a 90-day absolutely hands-off as a continuous loop altogether. So, that was something
[16:44] that we were able to do for our one of our clients. Obviously, there are tons and other examples as well but this is something that we wanted to mention and now going forward, it's not about what we have already done but also going
[16:57] forward that the gap doesn't always the stay the same, right? It changes shape and the next version is going to be more dramatic. McKinsey estimates 3 to 5 trillion dollars of global retail spend could shift due to you know, agentic
[17:10] channels to to by by 2030. There are about you You to 20% online sales which may flow through agentic channels by 2030, that's also being recognized by Bain. So, you know, those are some of
[17:23] the stats that I just wanted to make sure that we all are prepared towards. And that is something that we would want to say that how are we trying to address all of these changes in the in the in the outlook that we are seeing is through our L1 and L2 of the stack to
[17:36] make sure that you are ready, that your data is ready for the agents that are coming all together. So, yep. Just wanted to give you a quick pre-fix about what Ecolab does in the space. We have about
[17:50] army of 2,500 analytics and AI experts, data engineers and architects and obviously a load of domain expertise of working with retail and CPG clients, global clients, 50 of them. So, we do have
[18:03] pre-built, pre-integrated, pre-governed and we've been able to efficiently reduce the 9-month custom builds compressed into 9 weeks all together. Yep. So, that was it. So, definitely the
[18:17] retailers, I would say who lead the agentic era won't be the ones who got there first, but definitely they'll be the ones who started preparing today. So, if you're ready for what your Monday
[18:31] morning should look like, definitely more than happy to engage in either of the ways to be able to help you diagnose, activate or scale whichever maturity state your organization could be, more than happy to say. And again to reiterate that yes,
[18:46] Databricks will give you a platform and Ecolab will help you accelerate the decision-making. Thank you.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.