Let’s Elevate: An Open Source Model for Data Sharing and Collaboration in Retail and Consumer Goods

Summary

• Elevate industry initiative: An industry alliance is building open-source standards for data sharing and collaboration to reduce costs and simplify integration processes for retailers and suppliers.
• Significant ROI potential: One beverage company projected $300 million in incremental revenue just by reducing data availability time from 4 days to next day with a single regional grocer.
• Democratized collaboration: Elevate aims to simplify data sharing beyond the largest trading partners, addressing the estimated $2 trillion problem of friction in retail and consumer goods collaboration.

Let’s Elevate: An Open Source Model for Data Sharing and Collaboration in Retail and Consumer Goods

Talk by Adrian Bolosan, Sr. Director Data Collaboration , Databricks ; Dag Liodden, Founder & CPO, Crisp, Inc. ; Len Ostroff, Crisp ; Richard Schwartz, President and CEO, Pensa Systems ; Rob Saker, Global VP | Consumer Industries GTM, Databricks ; Sabrina Miller, Director of Value Acceleration , Databricks
Retailers and suppliers face persistent financial and technical challenges to data sharing — including expensive proprietary platforms, complex data integration hurdles, fragmented governance and more — which currently restrict seamless data exchange primarily to their largest trading partners. In this video, we’ll provide an in-depth explanation of Elevate, an industry alliance focused on building open source standards for data sharing and collaboration to drive greater efficiency across the entire ecosystem. this video will highlight proposed standards for data sharing, data models, business cases on the ROI and potential areas of innovation to democratize data sharing, drastically reduce costs, simplify integration processes and foster transparent, trusted collaboration. Learn about the Elevate industry data-sharing initiative and how your company can participate and help guide standards to improve data sharing with your key partners.
Here’s more to explore:
Open data sharing for data, analytics, and AI: https://www.databricks.com/product/delta-sharing
A new approach to data sharing: https://www.databricks.com/resources/ebook/a-new-approach-to-data-sharing
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements

FAQs

What is the Elevate initiative?

Elevate is an industry alliance focused on building open-source standards for data sharing and collaboration to drive efficiency across the retail and consumer goods ecosystem.

Why is data sharing critical in retail?

The retail and consumer goods industry faces a $2 trillion collaboration problem; faster data sharing can unlock significant value (one beverage company projected $300 million from next-day instead of 4-day updates).

What are Elevate's goals?

This video covers proposed standards for data sharing, data models, business cases, and innovation opportunities to democratize data sharing and simplify integration processes.

Who is participating?

This video features Databricks, Crisp and Pensa Systems, with Databricks playing a coordination role as speakers from the retail and consumer goods industry discuss collaboration challenges.

Full transcript

Thanks for joining. I lead the global consumer industries at Databricks. Retail consumer goods, travel. And we're here today to talk about something that um has been a passion of mine for a long time. And you'll hear from our our friends and partners with Crisp and Pens. It's been a passion of theirs for a long time. Um the challenge of driving better collaboration in the retail and consumer goods industry. If you think about what our fundamental problems are in the industry, the biggest enemy is is collaboration.
It's lack of data, it's lack of visibility. And fundamentally that translates into time, time and cost. And I meet with customers and what I hear consistently is that it's too technically expensive, it's too financially expensive to drive collaboration. Um I spent years as a CDO, I spent years in consumer goods leading data transformation. And there's a bunch of different efforts that going on, but fundamentally the challenge is that this is not something that anyone company can solve.
This is something that we as an industry need to come together to establish standards for how we can better work together to reduce the friction of collaboration to really drive these efficiencies. As a reference point, I worked with a large beverage company and they said just by reducing the amount of time it takes from 4 days to next day on data availability between they and a regional grocer in the US, they think it's worth $300 million in terms of incremental revenue.
And that's because of the reduction of out of stocks and things. That's one beverage company with one regional grocer. It's a $2 trillion problem that we have overall and it's fundamentally a problem of friction. So that's what we're talking about here today. Databricks and Pensa and Crisp can't solve this ourselves, but this is something where we we think that we can help to uh drive attention. We can drive focus, bring the right partners together because we do play a role and work with a lot of the leading retailers and consumer goods companies in the industry.
Uh forward-looking statement, of course. Um Agenda. We're going to talk through kind of the momentum that we've seen uh with, my colleague here, in terms of Delta Sharing because fundamentally we think an open-source based approach to data sharing is key to enabling this. And we're seeing massive adoption in that space. We want to talk then about um the use cases and the value opportunity that we're seeing within there.
Uh and going to talk through that. And then Degs going to come up and talk more in terms of like some of the things that we're seeing in terms of how do we bring this to life? What are some of the technical approaches and and more that we want to do? And then we have a call to action at the end. And we're really looking to um to to start making some forward movement on this in the back half of the year. I will be extremely transparent.
This is not something that's going to be solved this year. This is something that's going to be started this year. This is a multi-year journey. This is a very hard thing, but it's a very worthy thing to take on. Uh quick favor to ask. These are customer outcome maps. This is how we think about uh focusing in uh where we focus our attention in the industry for the business problems that are within there. You may have seen this if you saw a session uh earlier this week.
Um but my ask is if you have a phone ready. Ready? Okay, there we go. If you can just take a quick survey and let us know what are your priorities cuz we want to make sure that what we're doing is responsive to the needs of the customers. This is a perfect example. Data collaboration. This is not something that is going to generate Databricks money and revenue. This is something that's going to solve a key business problem and lift the the rising tide lifts all ships in the industry.
We think it's the right thing to do. Uh but we want to invest in this because we know it's a meaningful problem for customers. So, if you take that survey and you tell us what's important to you, it'll tell us where to invest. Thank you. I saw about it. Six of you uh take pictures. I did pay attention to those that didn't. I'm looking in that row right there. Um Anyways, uh here's what we have for speakers today. I'm joined with on stage, my colleague, uh and Dag will be coming up and talking in in just a few.
Uh but I'm going to actually hand it over and um to talk about what we're seeing for the data sharing uh growth and adoption. And you want to Sure. Give us a message. Thanks. So, Adrian Bolosan, um senior director of data collaboration go-to-markets at Databricks, and I have the fortunate pleasure of working with all the industry leaders like and leveraging things like Delta Sharing to solve problems across RCT, media, etc.
But we'll focus obviously today specifically on uh retail consumer goods. And we just want to show again some of the highlights, some of the success and momentum we have around Delta Sharing. So, pulled some recent metrics to show kind of year-over-year growth. This thing in the middle that you guys are all looking at that looks like this Death Star is actually a live shot of all that what we call active edges of our customers.
So, this is a real ecosystem of our customers connecting with each other through the technologies like Delta Sharing. Uh so, year-over-year growth 262%, but what's really interesting is and I pulled by industry, and in RCT actually had the highest growth itself at 327%. Another metric I'll call out I didn't highlight here. We talked about the openness of Delta Sharing. 40% of all these Delta Shares are actually what we call Delta or Databricks to open, meaning our customers are actually sharing to their customers and vendors, etc. , that are aren't on Databricks, right?
So, there is no vendor lock-in, and that's something we believe in. That's really the ethos of Databricks is is open and interoperable. And then here, just obviously, this isn't a full list of all the data providers that are out there, but obviously showing the breadth and depth of all these this ecosystem now. All those dots behind here are really supported by industry-leading data providers that are solving for business use cases or industry outcomes, as we call them, across retail.
And I won't name all these off cuz you guys can see them. We have a small group, but you can see the wide range, everywhere from retailer first-party data all the way to external demand. We were just talking about weather impacts and natural disasters, etc. So, it's really the breadth and depth of this ecosystem system that supports these critical business use cases across RCT. And I'll add in too, if you think about um we had major announcements with SAP that everybody I'm sure has seen in SAP Databricks, and we've had other capabilities and things that are announced within there.
Delta sharing's at the core of that. And so, what we're talking about here, I think about SAP Databricks, if we have a data sharing alliance where you have standards to bring that information in, we've just reduced the friction of making retailer data available within SAP, right? Because it's Delta sharing, there's no more ETL, there's no more data copy. So, this is why we're seeing such a strong adoption of this protocol is that it really eliminates the need for costly, complex ETL integrations and allows us to focus instead in terms of leveraging information, integrating it, and then making decisions on it.
Okay. Uh so, we're going to uh jump in next, and, I think you're coming up uh talk about this. Yeah, so what I wanted to do, thanks. Um what I wanted to do was uh give a quick introduction to Elevate, uh let's elevate. org. And the coalition, which we'll talk about and invite everyone to uh join and participate in, is about fully digital retail, connecting the dots to provide a fully transparent across the supply chain uh view of what's going on with data at each stage that connects to each other.
Set of founding partners, it's an open coalition and we'll talk about that, what it means to um to join and participate in it. See if I go the right direction. Yep. So So um uh overall, there are a lot of gaps across the supply chain. Um at Pensa, for example, we we work with both CPGs and retailers, with e-commerce companies, uh across the board. And there are many cases where data has been developed, models have been developed for one particular use case or one set of vendors um uh and suppliers, and not another set.
They don't fit together. The loops don't close, and that's causing problems for everyone. That's what we'll talk about. As as talked about in general, the effect of all of this The effect of all of this is a a $2 trillion problem. The lack of consistent information across all of that uh is causing a major problem. Um inventory that's in the wrong place, either overstock or understock, the lack of visibility that impacts planning and mistaken decisions across the board. do you want to pick up from there and uh talk a bit?
Thank you. Which way does this go? Um if you go this way, we'll go forward. Otherwise, you'll have to repeat what I said and speak backwards. Actually, since we have a very uh uh quaint audience here, I would love to do a a quick poll. Can I see everybody raise a hand show a hand to those that work for a retailer? Distributor? CPG? Technology? Okay, so we have a good mix of all of them. Great. That's actually perfect.
Um so we think that there are five foundational challenges that uh we we have to resolve to unbreak the supply chain. Um the first is reliable, uniform access to high-quality data. Um when we say reliable, we mean something that uh we can uh expect to be available at the right time, something that's driven by data pipelines um that don't fall over. Uh when we say uniform, we mean um to have a one uh view into the data, whether we are consumers as systems or we're consumers as humans having access to all of these different data sets in in one place.
And then we talk about quality, it's about does this data actually represent the true state of the business and is it complete? Uh this is a pretty big problem in in the space as well. Um as you might know, the current state of affairs in in the CPGs is that data is primarily shared um in Excel and via emails. And that's uh almost uh unforgivable uh in 2025 where we have technologies such as uh Delta Sharing available.
So we believe that this problem can be solved with a standard set of data sets uh and a standard set of data sharing mechanisms, which uh Delta Sharing is is one of them. But even if we have access um to good quality data in a uniform way, we still have some pretty specific problems to the CPG space. And that is that even if we have information, we have data about supply chain, about what's happening in the store, what's selling, what's happening in the in the marketing platforms, and so on.
The concept of products and what a product is is really hard to tackle. So, we were just talking about earlier here, so I think was mentioning there was a a popcorn company that has a lot of collabs with chocolate companies like Mondelēz and Mars and so on. Lots of varieties of popcorn. When you are using identifiers like UPCs, which are not really that universal because they can be specific to certain retailers, they say nothing about the structure of a product.
The product is more than just a UPC. It can be a particular sellable item, but it can also be a pack size. It can be a particular flavor of a product line. It can be a brand. It can be lots of different ways that you can slice it. And unfortunately, a lot of the analysis that we need to do are not on a like a UPC SKU level. We need to level that up. There are also a lot of systems that don't have UPCs at all. When you're running a marketing campaign, a digital marketing campaign, how do you associate the campaign and the messaging with the UPCs across the entire retail footprint that is going to be affected by this marketing campaign?
And how can we imagine that we can spend our marketing dollars correctly if we don't even know which products are being affected by it. So, we need some sort of, something that allows for structuring or bringing structure to product information. And that also includes things like hierarchies. Every retailer has their own view of the data. It does not line up with what the next retailer does. It does not line up with what the Circana and Nielsen use, and it doesn't line up with what the production planner and the marketing manager and a sales manager at the brand's house.
So, huge problem there with the product attribution. The third part is as we we see we've seen Walmart, we've seen Amazon over the past several years. They have really opened up the type of data that they're sharing. Um if any one of you have worked with the Walmart Sentilla program, you know that they're sharing literally hundreds of different data feeds. And Amazon has been doing it for years as well. Um still as so many parts of a CPG business, so many teams that the only thing that they use is the syndicated data from Circana and Nielsen.
There are a lot of good use cases for those data sets still. I'm not saying that those things will go away, but these first-party data sets that retailers are sharing, they're often expensive to purchase. They're also significant investments in technology to even get the data on to the vendors' platforms. And still too many parts of the organization will just click the easy button and continue with doing analysis on syndicated data that is quite narrow, usually not up to date, and not necessarily very exact.
So, as these data sets become available, we're investing into getting them onto the various platforms. We also have to make sure we are ready to actually leverage and get value from those data sets. And speaking of driving value from data, this also we've talked about data silos. Those are well known, but also a lot of decision silos. We hear way too often that a huge campaign a promotion for a a new or an existing product is launched, and then the product goes out of stock within days of being launched, or it's not even available in the stores when the campaigns go live.
And I think anyone that works in CPG knows about this, it's because a lot of teams have different data sets and they don't talk to each other. Um as we start to get these data sharing programs up and running, data should be available to all of the different functions, but we also have to make sure that they're actually collaborating on that data. And this is not just solving the old traditional problems, the ones I just described now, but as we start entering into the the era of AI agents, we need to make sure that in order for us to unlock the true capabilities of the full potential of AI, we have to make sure that these agents not only have access to the full breadth of data so they can make good decisions, but also that they can influence decisions across different organizations.
So having an AI for marketing and an AI for supply chain optimization and those two things talk to each other, but then we're just replicating the same patterns that we've struggled with for decades. And that brings me to the last main challenge. And I think I think it's everyone can agree these days that reporting is not the right answer to almost every question, although that's how we usually present it and I'm sure my company and myself are guilty on on producing reports, but I think there are very few business stakeholders that say I wish I had more reports available to me.
That's not really what people are looking for. Um what business stakeholders or business people want, they want to drive the business forward. Um and similar to what I talked about earlier, like we're we're now in a in a in a in a situation where AI agents hopefully will start helping us. This is a time for us to step back and take a look at the actual business problems that we're driving and we can build for both humans and for agents.
We need to have systems that don't just present numbers or reports. We need to have systems that detect problems, that detect uh, We need to have systems that can help uh recommend what to do to either harness those opportunities or to solve the problems. And we need, um, systems that make it possible to take more programmatic, automated action into the supply chain. Um, this this last part here is I think is a is a great opportunity for Elevate, uh, as kind of a next phase.
Uh, there are way too many business problems that go through analysis and that problem resolution or product detection or root cause analysis and all of that, and then it ends up with a a person at a vendor picking up the phone or sending an email to someone. Uh, we should be able to do better than that. And I think, again, uh, with the AI and agents, uh, the this these is really fertile grounds for doing more development together as well.
And with that, I'll hand it back to, um, to you. Yeah. Thank you. So, as talked about at the outset, um, what we're talking about and the vision for fully connected retail is a is a not a a single activity for any given vendor. Uh, it's not going to be solved this year, but the vision overall is really that fully connected, uh, supply chain and bringing online digital, digitized view of physical inventory as it travels end to end with loops that close through that to be able to look at, um, plans and references, actual implementation.
And to be able to optimize the entire system where there are different use cases and different activities that are responsible for part of that data, part of the process that connect together so that across the board the entire supply chain, all of the vendors, the retailers, the CPGs collaborating together can really solve the problem. Behind that, what we're working toward are a set of additional frameworks, additional models, taxonomies, ways to connect the dots between disparate systems.
As Rob also talked about before, one of the things in the past that's held back the ability to do this is either the high cost of getting data where data isn't available, getting first-party data um where it's either too expensive or too time-consuming to do. As AI comes in, and we'll talk a little bit more about that, it allows some of the that to be filled in. What we're looking to do now is to provide the frameworks to be able to connect all of that and allow the players across the board to be able to develop models, optimize, increase visibility.
Those standards are not standards from us, from any of the vendors or anybody right now in the coalition. It's going to be by working together to fill it in and take different use cases and provide the connectors to be able to bring it together. So, there are many causes for the for the mismatch and the gaps along the way. , you talked about up-leveling. We talked about the fact that some models around demand forecasting and supply chain are about um uh pallets that are delivered.
Pallets lead to cartons, which lead to case packs. Case packs get broken out and put on shelves. There are different shelves for the same products in different parts of a retail store. The original planogram for the shelf planning generally is never the real plan for what's been laid out. So, there are many ways in which the individual models created by different companies for different purposes don't fit together. What we're looking to do is fill that in, develop additional models and frameworks to connect, and then with a fully digital retail end-to-end visibility, then optimize the entire system.
So, the particular vantage point um for Pensap, um my company, has to do with um vision AI. The ability for um computer vision and advanced AI to be able to stare at the at the shelves at retail, literally wave a phone or another camera device as fast as you can walk down an aisle, and understand and digitize what's on the shelf, um how the shelf is organized, the shelving principles, and everything else, and generate a new source of first-party data that didn't exist before because it was too expensive and too time-consuming.
The ability to do that now allows the loop to be closed between the original planning of the of the shelf planograms and what's actually executed. The original plan for um um phantom stock and other things at retail, what's the reality of that? Where is the stock? Where is the shelf plan? Um how do you optimize from the real uh activity in the store across chains? How do you optimize the um space allocation? How do you decide where to send folks for the workforce optimization?
Where is the merchandising going to produce the largest bang for the buck? By closing the loop between planning and execution using new sources of data from AI, there's an opportunity to do this and it heightens the demand for fully digital retail. God, you want to Yeah, thank you. So, Walmart and Amazon been leading the industry in sharing data uh for several years as I mentioned before. Um but they have massive resources to invest into shared data programs like this.
They also have an advantage uh um that the smaller retailers don't have and that is that the vendors um so, Walmart is incredibly meaningful uh to an individual vendor. So, vendors will build to whatever Walmart, Amazon, and the likes uh produce in terms of data. This is not a luxury that the smaller ones have. Uh number one, it's expensive to or at least it could be can seem to be very expensive to build something like this.
And the second part is that if you make it non-standard, um there's no way that the vendors will actually build to your particular spec. So, for the smaller retailers, this is probably one of the reasons why these programs never get started in the first place. But that's really where we think that um a set of standards on delta sharing and some pretty basic data sets can open up a lot of opportunities. So, just call out here that you know a calendar dimension, so a fiscal calendar, a product dimension, stores, distribution centers, sales, and inventory.
With those basic building blocks shared in a standard format on DeltaShare, you can open up a tremendous amount of use cases that can drive meaningful value to the ecosystem. So, Crisp, we've been open-sourcing and sharing what we call blueprints, which are different types of analyses that can be done on top of these basic data primitives. Um and here are just an example some examples of it. Um but anomaly detection, demand forecasting, store clustering is something that a lot of clients want to use.
On-shelf availability detection and out-of-stock detection, of course. Uh we talked about extreme weather impact earlier. Uh and the list goes on and on. And we actually have have added quite a few just this week that we uh have published. All of these are available. You can't see it here, but it's on uh the Crisp GitHub uh page where all these are available. And the reason why we publish these uh is that we want to give uh retailers an idea of what can be done with some pretty simple basic data sharing primitives.
And we also do this because we have a lot of vendors as clients. Um and although we as Crisp we provide a lot of analytics and capabilities like that, we also know that we can't solve all of these types of problems in like a cookie-cutter out-of-the-box product way. So, we just open up the box and say, "Here's a good starting point uh for building these kinds of analyses." Um so, whether you're a retailer or or a vendor, uh just check those out and you can see there's a lot of really really valuable things that can be done with a a fairly small set of um of data primitives.
Right. And that brings us uh back to you. Hopefully. There we go. Um hopefully you all see the why we're We're about this. Like, there's a a a immense potential to derive a lot of value to the organization to your organizations. I'll And this is something though that we can't again do any single company. It's something where we need retailers and suppliers, distributors, wholesalers, and tech companies to come together to really help to prove this out to establish a standard that we can then use.
I mean, if I'm a a supplier right now, if I'm working with retailers, I got data models for every single retailer that I'm working with. And that's just cost prohibitive. That means that I'm going to limit the number of retailers that I partner with. If I'm a retailer, it's in my best interest to lower the cost of collaboration with my suppliers because I want to attract the long-tail suppliers. I I think on average about 10% of a retailer's suppliers will participate in data sharing initiatives.
If you're a grocer, you're going to have minimum 10,000 suppliers. So, you have 1,000 of those who are participating. The opportunity for value creation is in the remainder of like the mid-tail and long-tail for what we do there. So, what we're asking if you're interested, you can grab this QR code in the bottom there. Um uh it will give you my email address. Um and shoot me a note. And if you want to participate with us to do pilots uh in the back half this year.
And these are not heavy lift pilots. But if you are interested in in in learning more, if you're interested in potentially your company um wanting to consider doing pilots with us, if you're a supplier, or if you're a retailer I'd love to hear from you. Um we're going to be working with We already have a few retailers that are interested. We have quite a few suppliers that are interested. But we would love to get pilots going in the back half this year.
And then um start to evolve the standard and build it from go from there. So that's our presentation day. Appreciate you showing up, you talk listening to us, and for finding the room.

Learn more about the Databricks Data and AI platform.