One Stack for Retail AI: From Transactional Planning to Agentic Intelligence on Databricks

Summary

Algonomy built a unified retail AI platform entirely on the Databricks Data and AI platform that combines classical ML forecasting and optimization with GenAI and agentic workflows in a single stack, addressing the supply chain exception management gap that traditional planning systems cannot handle.
Hyper-local forecasting creates one model per SKU-store combination by selecting from an ensemble of 200 candidate algorithms, scaling to millions of distinct model combinations using Apache Spark's distributed compute on Databricks without infrastructure sprawl.
A multi-agent architecture called Maya chains orchestrator, specialist, and processing agents to resolve exceptions end to end—converting an unstructured vendor email into a resolved order update in minutes—achieving reported results of 45% fewer out-of-stocks and 100% efficiency improvement.

One Stack for Retail AI: From Transactional Planning to Agentic Intelligence on Databricks

Retail AI platforms are no longer just analytics or ML systems — they must power real-time planning, massive-scale AI and intelligent decisioning on a single foundation. In this video, we share how our retail AI SaaS product is built entirely on Databricks’ unified platform. What we do uniquely on Databricks: We run classic AI (forecasting, optimization) and GenAI + agentic workflows together in a single Databricks stack, eliminating fragmentation between ML pipelines, planning logic and intelligent decision automation. Our hyperlocal forecasting USP (unique model per SKU–Store via ensemble selection) is possible only because Databricks enables massive distributed ML at low cost, scaling to millions of combinations without infrastructure sprawl. Hierarchical forecasting across store, city, region, national levels, combined with genetic optimization for inventory and replenishment, runs at production scale using Apache Spark™’s distributed compute and optimized execution engine.
Talk By: Amit Rohatgi, Vice President Product Management, Algonomy ; Naveen Khattar, AVP Product Consulting, Algonomy Software Pvt Ltd ;

Chapters

FAQs

What retail supply chain problem does Algonomy's Maya agent solve?

Maya addresses supply chain exceptions—disruptions that originate outside the normal retail planning system, such as unstructured vendor emails, nearby events affecting demand, or unexpected supply changes. In a typical supermarket with 10,000 to 15,000 SKUs, a planner handles roughly 50 to 60 such exceptions every day. Maya uses NLP to extract structured information from unstructured inputs and chains agent decisions to resolve exceptions end to end, from master data updates through distribution center notifications.

How does hyper-local forecasting work in the Algonomy platform?

Hyper-local forecasting creates a separate model for each SKU-store combination by selecting the best-performing algorithm from an ensemble of 200 candidate algorithms for that specific combination. This means a large retailer could have millions of distinct models, each tuned to the local demand patterns of a specific product at a specific location. This scale of distributed ML is enabled by Apache Spark's distributed compute on Databricks, which handles millions of model combinations without manual infrastructure management.

What results has the Algonomy platform delivered for retailers?

According to this video, the platform has achieved a 45% reduction in out-of-stock incidents and a 100% efficiency improvement in exception management workflows. These results come from combining accurate hyper-local demand forecasting with automated agent-driven exception resolution that previously required manual planner intervention. The composable, replaceable task agent architecture also allows individual components to be improved or swapped without rebuilding the entire system.

How does the Maya agent architecture distinguish between different agent types?

Maya uses a hierarchical architecture with an orchestrator agent that directs the overall workflow, specialist agents that handle domain-specific decisions like demand simulation or inventory optimization, and processing agents that execute specific tasks such as extracting structure from emails or sending notifications. This separation allows each agent to be specialized, tested independently, and replaced without disrupting the overall pipeline. The composable design is described in this video as key to making the system maintainable as retail use cases evolve.

Full transcript

[00:10] Hi everybody. U thanks for coming in. Um today uh we're going to talk about um a one stack uh that we built on data bricks. It's a one stack for retail AAI and u through this presentation we'll essentially take a scenario and we'll
[00:23] walk you through how uh we essentially um uh execute that scenario on our one stack. Right. So I'm Amit here. I'm the uh VP product management and consultant with Algonomi. We'll talk a little bit
[00:36] about who we are and I've got uh Naveen who's uh VP product um and we both are going to be talking about how our one stack for retail AI um is built on data bricks and we'll walk you through and we'll use an scenario and essentially
[00:49] walk you through how our stack essentially executes that use case. Okay. >> Good afternoon everyone. Uh before we begin, we just wanted to check how many from retail. Uh oh, quite a lot. So
[01:02] good. Uh so we'll have to we can basically go ahead faster on this now. So uh we've been working on retail planning system for years now. I'm sure
[01:15] I think you you you all have worked seen those systems and what we believe and have experienced is most of these systems work for business as usual very well I think and
[01:28] with the current technological advancements. I think things are going even better for all the use cases which are business as usual. But specifically in retail the gap exists in the uh
[01:40] exceptions. Now exceptions typically originate from somewhere outside the retail systems. Maybe something coming from the vendor.
[01:53] Maybe maybe something coming from an event which is happening around the store. Maybe something coming from a lot of other sources which are typically not captured in the retail system. So I think that is the primary gap what uh
[02:07] we're trying to address. While we are saying from a systems perspective the business as usual processes will continue to grow, continue to evolve and become stronger where we talk about better forecast accuracies, better uh
[02:20] predictability of the model, better explanability of the models and so on. But going further, the loop can be closed only if we are able to manage the exceptions well. will
[02:35] to just give you an example or a quantum of the problems what we've seen I think in a typical supermarket uh it's about 10 to 15,000 SQS and as a
[02:47] rule of thumb each SQ go through goes through one exception at least in a cycle of one year considering there are about 200 to 250 working days so a planner is dealing with about 50 60
[03:00] exceptions every day which are again originating out of the system not within the system where they have to actually work right from one system to another system to another system and so on. So
[03:14] there are no connected system and there are very less connected system in this space. If I have any exception in the pricing, I might be as a planner working on a pricing system. Whereas if I have
[03:27] to run a promotion, I might go to a promotion system and maybe some master data update, I will have to go to an ERP or a master data management system which typically do not talk to each other in
[03:39] real time in and seamlessly. So that's what uh I think we'll be talking about.
[03:51] So to get this uh into this problem statement further, we'll just describe one uh scenario what a planner would face. Imagining one planner Sara in Hong
[04:03] Kong uh who gets into a morning commute for her work in a metro gets an email from a supplier which is based out of Middle East and
[04:16] it suggests that the lead time has for all the products will change from 7 days to 14 days because of a disruption in the shipping route. Now this looks like a very simple thing to update which is
[04:28] like I just need to change the lead time in the system in one of the system from 7 days to 14 days. But what are my downstream impacts is something what a planner always needs to imagine. Now
[04:40] since this has originated out of the system. So the planner will have to actually will have no option but to go to the office start probably logging into one system change the lead
[04:52] time in one and then she will have to identify what are the downstream impacts if the lead time is changed in all likelihood what will happen is they will have to carry an additional inventory
[05:05] because the lead time demand would be higher. Now from that perspective uh it may be the case that there are five or six steps before she can even arrive at a situation where she knows
[05:18] how much additional inventory would would the DCB be carrying to support this higher lead time or change in this lead time. So this typically I think she will hop from one system to another
[05:30] system and go into and maybe by the end of the day or within four five hours 6 hours she might not be able to even reach a situation where she will know exactly what is the additional inventory what she would be carrying. Maybe it is
[05:42] next day when the systems sync then only she'll come to know what is the additional inventory what she would be carrying or she will have to compute it manually and could be a situation where she
[05:55] misses this entirely and the order is placed and then what happens is DC all of a sudden either gets the inventory or gets to know about the additional inventory only after the PO is placed the purchase order is placed. Now in
[06:09] that case there is no option for the DC but to accommodate but it is not a planned event and there may be some disruptions for the other orders or other incoming deliveries at the uh at
[06:21] the DC. Now if we imagine a system which first of all can actually capture all these uh
[06:35] exceptions and in our experience what we see in all these exceptions lie in emails. What if a system can actually intelligently read emails and create downstream actions?
[06:48] The all the agents are programmed to basically uh take the downstream actions and finish the task within uh lesser time and with all the steps which
[07:02] are uh uh which are required to be taken to complete that task. Now reimagining that situation where same planner Sara when
[07:15] she receives this email forwards this email to the agent uh and internally for us for Egonomi we call our agent as Maya she basically sends this email to Maya
[07:27] what happens is all the actions what she had taken uh manually and with her own intelligence as a part of a SOP or something uh which she has been doing uh
[07:40] for years. The agent takes over and basically creates those uh sequence of actions and gives her the confirmation of the task and let's see
[07:52] how it does. So this is more of a uh I think agent console where there are multiple emails or multiple use cases or the exceptions which have been
[08:04] forwarded. So first of all what the agent does does is using the NLP engine identifies or understands what uh what does this email mean. Now in
[08:17] this case there is a very unstructured information which is available in the email. First of all, uh as a first step, the NLP engine creates a structured
[08:29] output out of that email which is uh what what vendor is impacted, how many days of lead time is impacted, what SQS are impacted, from which date it is
[08:41] applicable and if it is applicable till a certain date or not. Uh like in this email it was mentioned that it will be applicable for at least 12 weeks. Now uh here also the agent takes a note and
[08:55] makes sure that this date is noted and what action to be taken. We can see uh towards the end of it. Once this information is available uh the structured data is available the
[09:07] agent starts pushing this data into the respective systems and simulates a situation. Right? So first thing is more of a updating the uh master data. Going to the next level is simulating the
[09:21] inventory. What would be the additional orders or the what would be the additional inventory at the DC and when will it come? entire simulation is done by the uh algorithm in the uh back end
[09:35] and finally it actually notifies the DC user or the DC operations to ensure that there is they can actually make space for that additional inventory which is coming. Once the DC confirms
[09:49] it basically goes back and notifies to Sara and eventually I think if if you see the speed at which it is executed she would have just come out of her morning commute out of the metro and her
[10:03] confirmation email that this has been executed is already received. Now there was one open question in that email uh before final execution. uh it would like to confirm from Sara
[10:15] that yes because there is an open question of whether end of 12 weeks the system should or the agent should automatically revert it to 7 weeks or should it actually reconfirm with Sara
[10:27] or the supplier before it uh before it reverts to uh 7 days till 12 weeks. Yeah, this is already executed. Now from
[10:40] uh just a comparison of these two scenarios, I think minimum of in the current scenario or in the as a scenario minimum of about 3 to 5 hours is
[10:52] required or and in lot of systems it would typically require one day of uh data sync to happen before the entire decision can be taken. But with Maya I
[11:04] think it is possible within minutes. And yeah so while in retail like we said there are multiple systems which are uh which are working in silos uh there is forecast
[11:19] there is auto planning segmentation promotions and then there are agents which are working separately. So what we uh are doing is basically creating more of a virtual one stack of the solution
[11:33] where the agent actually speaks to each of those systems and executes what the business planner needs on a day-to-day basis and execute all the exceptions and
[11:45] minimize the gap between business as usual and the exceptions for a 360deree planning and execution of uh the forecast and replenishment with this I'll just hand it over to
[11:58] Amit. >> Yeah, thank thanks Navein. Right. So um so next section what we'll do is uh we'll jump into the technology behind Maya uh which is our uh planning agent.
[12:10] U so before we do that just a quick introduction on who we are. Um we are um Algonomi we um we focus on delivering uh retail applications around analytics, AI
[12:22] and agentic applications for retail. Uh we serve the entire retail value chain. U we uh we deliver solutions for merchandising teams uh for supplier and vendor collaborations. Uh we've got
[12:34] solutions for customer analytics. We've got solutions for customer marketing. Uh we deliver real-time personalization across all sales channels for retail businesses. Um we uh we've been around
[12:48] for 20 years. We work with almost 400 plus brands globally. Uh we uh you know more than 30 billion decisions are essentially executed through our platform every day. Um we are well
[13:00] recognized with the analyst community. So, Gartner, Forester, IDC um all of us recognize for our retail planning solutions, for our customer data platform and our uh for our uh real-time
[13:14] personalization capabilities. So, today you know what Namin was talking about is essentially being uh served through our one unified stack uh completely built on data bricks. Each
[13:26] layer of the stack essentially is leveraging underlying data bricks capabilities. uh we leverage data lake uh delta lake for our data foundations. Uh we execute our forecasting our uh you
[13:38] know customer predictive models or optimization models essentially through containerized execution endpoints through MLflow. Uh we using mosaic AI for our AI and agentic applications and
[13:50] we use AI uh BI geni as natural language interfaces for our retail planners and marketing teams. Now u now before we move forward there's
[14:02] one question that comes up very often is that um we we we've had machine learning models for a long time we've got now generative AI which can do reasoning why do we need agents um isn't the machine
[14:15] learning models and reasoning models good enough why do we need agents for all this so so if you look at you know models are extraordinary and machine learning models are very good at telling
[14:27] us what so they can essentially help us identify hey um you should be ordering 400 units or identify anomaly scores or identify forecast lifts for us u and you
[14:41] know I'm sure most of you you know who are working in retail you know models like this or and sites like this you already have access to uh in fact we've been building uh and deploying models like these for last 20 years now u but
[14:55] if you look at questions on the right hand side u this is like hey uh you know the supplier is delayed by 3 weeks. Should we still order? What should we order? How much should we order? When should we order? Uh the forecast is is
[15:09] showing a spike and you know why is there spike? Is this spike you know real retail demand or is this because of a promotion? Uh you know should we mark down a line when we already are running
[15:21] short nationally for a particular item? Now if you look at these questions these are not data questions. These are all reasoning questions. They require more context. They require judgment calls to be taken. They require you know systems
[15:34] to going into live systems and essentially acting on those insights. Right? So it's more than you know what a model could do. A machine learning model you know is was never designed to answer these questions. It's not a gap that you
[15:47] could fill by creating a better model. It's not something that a machine learning model would do. And a generative AI model is is is that good at reasoning but what we need is essentially you know autonomous agent
[16:02] which essentially can execute sequence of steps till the decision is made and that's why we need agents. Um an agent you know which could which could chain decisions which could take which could
[16:14] chain decision tasks you know. So if you look at the email it's not just one query to one model. It requires extracting the information from the email. It requires updating the master
[16:26] data management. It requires running the supply uh it running the simulation for inventory planning. It requires notifying the DC. So it requires executing chaining of different tasks,
[16:38] you know, different decisions which essentially depend on each other, right? So a generative AI is essentially just meant for answering a query. But an agent is essentially something which chains decisions for you. U a generative
[16:51] AI model would only have access to information that you put it in the prompt for the generative AI model. An agent has access to tools and through tools they can essentially you know
[17:03] query into your ERP they can query the delta lake they can essentially call the forecasting models and they could be also writing uh in the audit logs right so so while a generative AI model is just for answering queries you know an
[17:17] agent essentially uh takes actions right and all of this needs to run autonomously on u on hundreds of decisions and and you still need a capability where those decisions could be paused for human intervention. Right?
[17:31] So that's what an agent does. It it runs an iterative reasoning loop, you know, till the decisions are made. And that's why you need an agents an agents and but an agent would essentially, you know, work and leverage the machine learning
[17:44] models. It would leverage the generative uh AI capabilities. But to but to create and execute a scenario like this, you essentially need agents, right? And so so what we've done is now just going a
[17:57] little deep into a design behind Maya u so if you look at at a high level just three different blocks right u there's an orchestrator that uh
[18:10] essentially not just routes you know different data sets this is the system which essentially holds the entire state of the you know full full workflow state
[18:22] it is a system which essentially decides which tool or which agent to call and it essentially holds the entire context and it uses the context to itself decide what's the next decision to be made.
[18:36] Then you've got specialists. Uh these are specialists. So you'll have a specialist for each job. So you may have a specialist for you know NLP specialist that would read the email, another
[18:48] specialist which would update the master data management. another specialist which would run the simulation. So you'll have a specialist you know a a task agent for each and every task and then there is a human judgment which is
[19:01] preserved because each uh each decision workflow would fall would go through a human decision gate and and you know for the human decision uh the system would not just surface you know raw
[19:13] information it would surface a very structured information which includes the entire insights impact analysis so that the human can take a more informed decision. So this is just a high level how we've structured the solution.
[19:26] So so basically what what's happening is that you've got the orchestrator in the middle in a way which essentially routes which essentially holds the context which essentially decides which tool
[19:39] agent to essentially call. So when the email comes in um it essentially invokes the NLP agent which extracts the information for the email. It understands there are 47 SKs that are impacted. U it takes the information
[19:54] updates the master data management by itself. Uh all 47 SKs are updated. It runs the simulation agent. The end of the simulation agent it identifies that there are 12 SKs that are at risk. So
[20:07] then it forward um you know it pull forward some orders. So it identifies the date. Some of the orders were that were to be made like 3 weeks later or two weeks later. It'll make those orders right now because because your delay is
[20:20] from seven weeks to 14 weeks. So you essentially want to make the orders right now. So it makes the it pulls forward some of these orders and then it identifies the space that will be required in the distribution center and
[20:32] then there is another agent which notifies the distribution center. Uh essentially it tells you the number of pallets that they need to account for and what date would those consignments essentially come. So this is how uh you
[20:44] know you know centralized uh Maya which is the orchestrator is essentially coordinating between uh different uh tools and each of the tools is essentially a specific agent which is
[20:57] completing one retail decision task. Um then u yeah so so all of this essentially is built on uh databicks platform. Now but still the one of the
[21:10] most important things is u you know is the underlying transactional AI it all depends upon uh you know the quality of the agent and the quality of the decision the agent is making finally
[21:22] boils down to your forecasting model if your forecasting model isn't right then your agent is reasoning on a wrong premise itself right so so core part to it is still a forecasting model and so
[21:35] what we have done is Um we use a hyperlocal forecasting model and this is not just one model which is running for uh one store. This is uh essentially uh
[21:47] you know one model per store skew combination. So that's the scale at which this runs. Um and uh you know it's so so the system has access to almost 200 uh candidate algorithms and the
[22:00] system finds the best fit uh algorithm for a stew store skew combination and it picks that up and then it essentially runs a bottomup approach where it reconciles all of those outputs at a
[22:14] store level, a city level and region level. And that's how uh we build our hyper localal forecasting model. For the replenishment, we use a genetic algorithm which is essentially an optimization uh problem where it's you
[22:28] know constrained by uh vendors service level by vendor's lead time by minimum order quantities and uh the holding cost and so we use a genetic algorithm for
[22:40] replenishment over there. Just a glimpse on some of the metrics that we've been able to achieve using the database track and our forecasting model. We've almost
[22:52] received a 45% reduction in out of stocks. And then you know there's almost like a 100% uh improvement in efficiency running on data bricks uh frameworks versus you know traditional frameworks
[23:04] that we've been using in the past. Um so uh we essentially run the whole combination of every skew and store in data bricks environment uh which essentially runs distributed clusters.
[23:17] The same clusters are then used for ML which we use for ML training are also essentially used for agent tool calls. And then we've got a single MLflow observability that is essentially uh you
[23:29] know being working across both forecast and agents. So essentially uh you know there's a huge optimization on the cost but on the other side we've been able to almost you know achieve a 95% on shelf
[23:41] availability and um you know reduce the uh inventory cost by 22%. U just going slightly more deeper into the whole you know design that we talked
[23:55] about this entire thing is composible. Now we picked up one scenario where we were uh essentially handling one exception and u for that exception you know we we we we brought in certain task
[24:09] agents right so basically you know there's an orchestrator the first layer then there is our task agents task agents are specifically domain specific agents you know which would you know
[24:21] carry out one specific decision so for example you could have a task agent for replenishment you could have a task agent for markdown You could have a task agent for promotion decisioning. So, so you've got these task agents and each of
[24:34] these task agents are designed where these are replaceable, individually testable, observable and it's all composible architecture. Tomorrow if you have some other problem, you could just plug in different different agents over
[24:46] there, right? So, completely composable architecture. And uh below the task agents there are processing agents. This is where you run the entire machine learning uh models. Um so for your forecasting optimization RFM models
[24:59] would run essentially on the processing agents it leverages the spark MLB underlying over there and then there are at the lowest level there are tool agents uh these are you know unity catalog functions these are uh you know
[25:13] data delta lake lookups and and all of these functions are essentially registered within unity catalog and any of the upper layers essentially can call the tool agents at any time. So essentially the idea is that u there is
[25:28] no data copy any layer essentially can access the same unified data uh within the delta lake um and um you know all the data and compute essentially is running on one single instance. So
[25:40] that's one big advantage that we uh we are seeing with um with data bricks is because we've got the data and the compute and the governance through the unity catalog all in single place. Um
[25:54] and and so essentially what we've done is u you know uh one of the big assets that we bring in is a rem retail semantic data model which is essentially using the underlying uh data uh
[26:07] capabilities delt like delta lake unity catalog then we've got our own proprietary uh retail uh models which are we are executing through data bricks workflows and um you know spark ml um
[26:20] and then we've essentially now created a retail agentic framework which is completely leveraging Mosk uh AI capabilities within the data bricks right so we you know completely taking
[26:32] advantage of the data bricks capabilities one single stack no data copies uh completely uh governed every layer of it is governed through a unity catalog
[26:44] yeah so I think uh this is what we had to cover and you know the whole um you know advantages that we are seeing are because there's no data fragmentation
[26:56] because the quality of the decisions that your agent would make would essentially depend upon uh you know these you know foundational capabilities. If your data is fragmented sitting in different places obviously
[27:09] it's hard to achieve you know decision qualities and by bringing all of this data together you know data you know compute and governance in one single platform uh we've seen tremendous uh you
[27:22] know um advantages and tremendous benefit um you know in in our capabilities that we deploy for our customers. Right. Well, thank you so much. U any questions if you guys have
[27:35] there two mics over there. Um happy to answer. Uh but we'll be around as well. Um so if anybody of you have any discussions you want to do, happy to answer. Yeah.
[27:48] Thank you so much.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.