Sponsored by: Artefact | The Workforce You Won't Recognise

Summary

Enterprise AI adoption stalls at twenty to twenty-five percent under the tooling trap, where new platforms are deployed into existing workflows without redesigning the work itself—illustrated by 700 dashboards built for a large European healthcare provider that nobody opened.
Artefact's framework for building a hybrid human-agent workforce treats AI as a product with persistent funding and lean tactical teams, measuring success by cycle time and decision outcomes rather than model count or dashboard quantity.
An e-commerce repricing case study demonstrates that agentic systems can achieve over ninety percent adoption when work is explicitly redesigned before the platform is deployed—delegating coordination-heavy tasks to agents while humans focus on ideation and judgment.

Chapters

FAQs

What is the tooling trap in enterprise AI?

The tooling trap describes the pattern where an enterprise tool is deployed, training is run, adoption peaks at around twenty to twenty-five percent, the organization blames change resistance, more training follows, and a new tool is deployed three to five years later. It occurs because the tool is inserted into existing workflows without redesigning what those workflows should actually look like.

Why did 700 dashboards built for a healthcare provider fail to drive adoption?

Artefact built 700 dashboards for a large European healthcare provider's global e-commerce team, automating thousands of hours of data work across sixty markets. Despite technical excellence on a modern data platform, very few people opened the dashboards because the workflows around them had not been redesigned to integrate the outputs into how people actually did their jobs.

How does treating AI as a product change enterprise adoption outcomes?

Treating AI as a product means giving it persistent funding and lean, accountable tactical teams rather than running one-time transformation programs. This video argues that this approach, common in financial services, creates the accountability structures needed to move from isolated AI experiments to production-grade hybrid workforces.

What does the e-commerce agentic repricing case study show?

The e-commerce repricing case study demonstrates that agentic systems can achieve over ninety percent adoption when work is explicitly redesigned before the platform is deployed. By identifying which tasks to eliminate, augment, or elevate, and building agents around coordination-heavy workflows, teams can move from persistently low adoption to near-universal use.

Full transcript

[00:07] Firstly, thank you for all taking the time to come here. I know it's the last day of the conference. Massively appreciate it. So, hopefully you've all enjoyed the last couple of days. We've seen a lot of fantastic technologies from a whole range of partners and vendors and
[00:19] obviously Databricks themselves. Something we want to talk to you about today that's a little bit different. Actually driving adoption and change within the organization. So, we're going to talk through this new workforce that's arriving, agents. I'd like to give you a bit of an
[00:31] anecdote in a story. So, about 6 months ago we were working with a large healthcare provider out in Europe. 50,000 people, huge e-commerce transformation. We led the development and
[00:44] transformation of about 700 dashboards for the global e-commerce team. And the good news is, it worked. Thousands of hours of dashboards work automated, huge success
[00:56] built on a modern data platform. Or so we thought. The biggest challenge we had ultimately was that um nobody opened it. We had very, very little adoption. So, we had a fundamentally technical excellent
[01:09] platform, but nobody used it and we couldn't understand why. So, about 8, 9 months into the program, we paused cuz we really needed to understand what was going on here. Why had we built and automated all the work
[01:21] that everyone had been complaining about and to give you a sense, these were 60 markets and about 5 to 600 hours a month of just raw like data automation, data wrangling. It's a huge problem, but they managed to fix it
[01:33] and work through. And that kind of comes down to what we call the tooling trap and I think you'll all be familiar with some flavor of this. Where an enterprise tool gets deployed.
[01:45] We run training. We try and drive adoption. We track that adoption. Adoption peaks at around 20, 25%. We struggle to get it further. We blame
[01:57] change resistance. We blame the organization. And then we run more training and inevitably 3 to 5 years later, we deploy another enterprise tool. And the biggest problem here is that the tool that gets deployed is deployed into existing
[02:10] workflows without actually redesigning or thinking about what that means. Now, that was maybe less of an issue before and we'll touch on the whole journey to cloud and what that means and why this is a very different paradigm shift. But ultimately, we're we're designing for
[02:23] something different now. So, let's dig into why why does why does this happen? Why does this fail? It's because you get years and decades of organizational scar tissue that gets encoded in these workflows and software. And I'll give you a sense of the company
[02:36] we worked for that we did this. We really tried to unpick why this had happened. So, if we go back to the original process for the e-commerce team, was developed over 20 years ago. 3 years after that, they developed a whole bunch of compliance patches to
[02:49] comply with the EU regulation. 4 years after that, they implemented a new system large with a a big GSI. Then we had a whole bunch of manual hand-offs that nobody could explain why they were happening. There was another a
[03:01] work around implemented at the beginning of COVID. And then you're putting AI on top of that. So, it's really no wonder why these programs don't actually drive adoption success. So again, you'll all be familiar with this number. Everyone's seen it.
[03:13] Everyone's talked about it, right? And whether it's 100% correct or not, that doesn't really matter. Directionally, it is correct. We're not seeing the adoption and we are expecting to see and it's not a technology problem. Ultimately, the technology can
[03:26] do the job. If it was purely technology, we would have been able to solve it by now. And it ultimately comes down to having an alibi and an excuse. You know, what is your AI program? And I if I had to guess of people in the room and across
[03:38] the conference, it's a combination of you bought a platform, you've nominated a head of AI, a chief AI officer, you've developed a or a developing a center of excellence, you have partnerships with one or multiple vendors, and you have a whole bunch of pilots currently in your
[03:51] portfolio, right? If that sounds familiar, I could ask you to put your hands up, but I I suspect I know the answer, right? Is that a transformation or is that a bit of an excuse?
[04:04] So, the hard truth that we're really here to push today and our conviction is that you've gone and purchased the easy part. You put the platform, you put the technology in place. The 80% of the work redesigning the work in the operating makes model itself,
[04:16] that's the hard stuff that's been avoided. And there's a lot of reasons for that that we'll dig into and then how you can actually work around that to drive success and transformation in your organization. So, a little bit of more about the why cuz I
[04:28] think it's really important we understand these root causes. We've applied a legacy mindset. So, if we go back to the journey to cloud, the reason and you know, maybe we didn't see the huge reduction in IT expense bill that we were expecting, but the reason it was
[04:40] by and large successful is that we were just doing lift and shift, right? We were applying a legacy mindset and it worked cuz we weren't doing cloud native refactoring. Few did, but it wasn't massively different. The new world is, right? It was very much lift and shift.
[04:54] Why this time is different is because what agents do fundamentally is change the coordination bandwidth. There used to be a handoff because analysts couldn't see controllers numbers and they needed to receive that
[05:06] and analyze it. There needed to be a review cuz you didn't know what was going to get sent to you by the juniors and seniors in your organization. And then there had to be a weekly cycle cuz it takes time to review these things. And these this is all embedded within your existing organizational structure
[05:18] and operating model. And again, the reason that some companies and a lot of you will have seen productivity gains and every, you know, every other day we speak to a CIO or a CEO, where do they see success? It's all on productivity tooling. Now,
[05:31] they're spending a lot on tokens and that's a separate problem, but ultimately, it's a faster human they can leverage it and work around. And that's great. Productivity benefits are awesome. But the bigger problem here is that for multi-step work that takes actions,
[05:43] the operating model is no longer fit for purpose. So, we need to ultimately redesign it. And the agents are actually the perfect forcing function for that to take the mandate and really seize that across your organization to drive change.
[05:59] So, if the technology works, what is actually meant to change then? So, one of our big convictions here is that you need to change your operating model. And I'll say that throughout, but how do you go about changing it? Let's start from a principles basis. Product over project, right? You have to
[06:13] fund persistent data um programs. And what I mean by this is we shouldn't see AI as a one-off. We should see it as part of a you know, specific cost center that we can allocate um SLAs to and drive over time. Same as treating
[06:26] AI as a product. The idea here is that it needs to be embedded across the organization. I think one of the biggest mistakes we see is appointing so-called heads of AI or chief AI officers as opposed to
[06:38] decentralizing AI to all the different functions, whether that's your CTO, your CPO, your CRO, or your CMO. AI needs to really be embedded there. And it needs to come from leadership, not just having it, you know, sponsorship. So, to say leadership, and we'll we'll come to that
[06:51] later. So, treat AI as a product, and it's a data product, and we'll come to that later when we talk about how to build good agents. Whoop, sorry. So, then we talk about autonomy with
[07:03] accountability. So, this is about having these like very lean tactical teams that can go and make impact. The companies that are doing the best work right now have deep domain knowledge and sector expertise, whether that's in I'll give
[07:15] you an example of a financial services client. They run the mortgages team. That uh team that's building out AI capabilities is two to three business people who have deep sector expertise, and then two to three AI {{}slash} data engineers, basically, right? And that's
[07:27] the model that works really well. Now, it works well if you can embed them around a platform and you set guardrails, whether that's you know, we'll come to compliance and policy as code. And that's how you control it. Otherwise, you end up having to come back to the mothership all the time.
[07:40] That slows you down, and you lose velocity. Ultimately, velocity is really important in this game. So, you implement policy as code from a platform perspective, you implement explainability because that's what we all want to understand. Why is the model
[07:52] telling us to do this? We need to understand that. But, these are all technological problems that we have solutions for, but it still needs to be factored in. Then, we start looking at value stream KPIs. For each of those domains that we want to build AI in, what are we really
[08:04] trying to measure? And this is a difficult one because sometimes what you measure then becomes the wrong target and you optimize or over optimize for the wrong thing. So, one thing that's been very successful for both internally and with our clients has been measuring
[08:16] cycle time adoption. What do I mean by that? We're not just focused on how many, you know, tokens did we spend or the effort we reduced in a specific sector. We looked at how much faster can we get this specific job to be done? And
[08:28] it's about being very granular around that specific task. Not about the number of models you deploy. And then finally, and again, you'll all be familiar with this, platform as self-service. Make sure everything infra ML tooling, you can
[08:40] serve it as an API to all your, we'll say, customers, but it's effectively your internal teams who can then consume this and use it faster. So again, you can do all that on Databricks and many of you are either very mature or maturing rapidly in that stack, and
[08:53] that's good news because that's that's the starting point, but it is the starting point and we need to move from there. So, as I've said before, you can't really hand this to a chief AI officer. Not that we have any particular misgiving about the role, but that's
[09:06] sponsorship, not leadership. The most successful programs we see, they come directly from the CEO. And the reason for that is that sponsorship, um oops, sorry. Sponsorship has never redesigned a company, and that's really what we're
[09:18] all going to have to try and do if we shift to trying to be AI native. And the firms that are AI native will have an outsized advantage of being able to disrupt us because they don't have any of these legacy systems. And again, that's a somewhat obvious statement, but
[09:30] it can't be overstated enough. So again, the good news, you already own or are on your way to developing the technology. And the point there is in Databricks serves you a fantastic we don't need to get into the architecture.
[09:42] That's not what this is about. There's been a number of fantastic talks on that. But the whole purpose here is you build your platform and you have to see it as the rails. And it's these rails that will accelerate you, but it's also the rails that will keep you compliant
[09:54] and managed and governed. And that's really important cuz without that you're not going to get organizational adoption. Okay, so now we're going to talk about AI native or agent native businesses.
[10:06] What are best practices? So again, as we said treating AI as a product. Treat the agent as a product, right? It ultimately belongs where the data is already governed, ideally in your lakehouse. And that's why it's important to have that center of gravity, data gravity so to
[10:18] speak, making sure that everything, the tools, the evals, the traces, the harnesses is as is becoming the nomenclature, are all centered in the lakehouse. That's really important. Often something we found retrieval actually decides the quality of the
[10:31] outcome, not so much the model. I think we sometimes over index on should I be using this model or that model or some open source thing and that's all well and good and there's definitely a cost optimization story there, but that's not the key takeaway. The key takeaway is
[10:43] good data engineering done correctly with metadata and context management, that's what is that's what ultimately leads to the difference in outcome. And then finally, you can't ship what you can't evaluate. A somewhat obvious statement, but ultimately how many
[10:55] people in the room have really put a lot of thought into what my eval should be. Who's managing that? How are we updating it? What does that mean on a per data product item? Really, really important stuff and this comes to down to when we re-architect and will come down to how
[11:08] we did that within the organization we started speaking about. You need to know what you're measuring and benchmarking. The benefit is that will help you drive down your cost over time. And then again, goes without saying from a cyber perspective and it's
[11:20] been in the news the last week, this opens up a whole new set of attack vectors, which if you're in a large complex organization, what that ultimately means is there will be people who will try and slow this program and these programs down for those very
[11:32] reasons. So, you need to have really robust answers to these problems before you embark on that journey and you do that by having it well governed and well managed from the start. Okay. So, now we're going to come down to how do we actually redesign the work?
[11:45] And the whole point is you redesign the work before the platform. So, in the old world we used to build our platform and then we'd look at processes and begin to ingest them. The way we are meant to be doing it and the way we want to do it now
[12:08] is to go work into the platform. So, the platform is actually the outcome and I think that's a really important thing and a definition that we use within Artifact is the data platform's job is to serve data products. It's not just infrastructure. It's everything that sits around it.
[12:20] It's the socio-technical layer, right? It's the ability to serve it to the end user and that's everything from the creation of the data product technically, designing it, requirements gathering, and then serving it to the end user in a reliable, uh, cost-effective way. And there's a lot
[12:33] of, um, I think let's call it traditional change management that people think it falls into that bucket of and that's absolutely correct. It is really important and I think over the last 3 years there's been a lot of ignorance around that that we could just build and solve it and actually we're not seeing
[12:45] the adoption we expected. So, we've got to go back to brass tacks of how this works. So, when it comes down to redesigning work, there's kind of three key mental models to keep in in your mind. So,
[12:58] eliminate, augment, and elevate and I'll touch on each of them. So, with eliminate the agent kind of owns everything end-to-end and the human inspects some of those exceptions. This is fairly obvious, right? With augmentation, the agent actually does the drafting. The human is just in
[13:11] the loop and there's a spectrum across that, but it's what most of you will be used to with some of the very early stage agentic workflows you're probably seeing in your organization. And then finally elevating it and this is actually the trick. Where you see
[13:24] most adoption is is you elevate the work cuz without that people are going to push back. The people are very much the immune system of your organization. They're used to good and bad change programs, more bad than good, to be honest, and they'll push back on a lot of this. And if they see stuff that is
[13:36] changing their role in a way they don't like, you won't be successful. You need to work with the team on the ground. So, either you automate it completely and you uplift the role and the type of work they do, or it's not going to be successful.
[13:54] Okay. So, again, we'll touch on this, but I'll gloss through it fairly quickly. As you go from the spectrum of pure automation to assisted automation, where the AI is actually doing more and the human is in that loop, augmented decision-making. And if we
[14:06] look at the difference of the actual type of work you do, you go from invoice extraction, document classification, all well-worn tasks I think most of you are hopefully well on your way to doing across your organization. Then we start getting into where you actually need a bit more human
[14:18] judgment and you want the human in the loop. Content moderation, pricing strategy. And then finally, the human leading and the AI extending. And that's when it comes down to creative direction. We see that with a lot of our We work with a whole host of
[14:30] luxury brands, and this is always a key question for them. How do I actually empower my teams without taking away from the various artisans and métiers we work with?
[14:42] So again, don't train people on the tool. Focus on the parts of the job that aren't value added to either to them. Take it out and redesign the work. But this is very much redesigning work. In the same way at the beginning of COVID, we talked about future ways of working,
[14:54] we just shifted remotely. This is genuinely the future ways of working we've been talking about and getting to over the last 6 years. So, what does the actual methodology look like? So firstly, you redesign. You take that elevate, augment, eliminate
[15:07] mental model, and you apply it. But you focus on one loop first. Don't try and boil the ocean. Don't try and do too much. Focus on where your data is very high quality. Focus on where there's going to be a ideally a top line impact to your
[15:19] revenue. And build it and prove it. Not just talk about it on slides. And then build native. Again, you'll all be Databricks native or Databricks customers. Um build it there, build it in the lakehouse, and make sure it's managed and governed. And
[15:31] you can start simple, you know, with agent bricks, and you can move to more complex ways and more advanced ways of doing it over time, but start simple and prove it out. And then from there, you evaluate and govern. And that's the key thing. You need to be able to prove that the outcome is one, correct,
[15:45] and two, how correct is it? So, you can begin to benchmark and ideally drive your token down uh token cost down over time. That's going to be a big conversation over the next 12 to 18 months. So, going back to how we started, apologies for the technical
[15:57] error. Um so, we started with huge uh dashboard transformation on a data platform for an e-commerce business. Hugely technically successful, but failed in terms of adoption. How did we actually go about succeeding? So, we went back to first principles and
[16:09] we redesigned the workflow. So, if we look at a e-commerce trader today, what do they actually do? So, they review the trading pack, they decide what needs to be pricing. There's a trigger, and that sometimes runs once, maybe twice a week, depending on geography. From there, they have to
[16:22] complete a dashboard. They will spend 20 to 30 hours a week wrangling in Excel to try and estimate their pricing. And they're eyeballing this the whole time. It gets sent back to the category team. There's two to three days of negotiation. They export everything to
[16:34] Excel and then have to build their reporting and management pack. Uh and then once that's done, there's really no audit of who came to that decision, but they get to a decision, they push it. And that that worked for the business for a time.
[16:46] What actually led to them using the tools that we built for them was coming down to this agentic process redesign. So, what we did is we built a pricing agent. That was the first thing. And from there, it decided what needed to be repriced, and it did that within certain
[16:58] uh boundaries, and making sure it fell within confidence intervals. So, when a SKU arrived and it was sold out, or a rival dropped price, that was all being monitored in real time. So, again, you were automating a valuable process. The agent was then being um
[17:11] reordering that, shifting spending, or sending it for manual sign-off. And then what happened was everything that would normally happen across the course of the week, that was done automated. And then the human was sent updates. And the areas
[17:25] where actually we don't have confidence this is a accurate enough, make a manual intervention. So one, the boring work got reduced, but more importantly, they spent more time with their suppliers figuring out how to actually drive this change across uh their organization and
[17:38] own their domain specifically. So this was really important. We went from having a faster dashboard to actually taking work off the trader and making it more empowering. So again, same platform, same underlying technology, but because we redesigned the work and through that the workforce,
[17:52] we actually delivered that uh that's adoption. And again, the key thing here is none of that you can purchase. You can't go and buy a SAS platform that does that exactly. You have to design it and you have to build it. So again, back to that core tenant of you've built the
[18:04] 20%, the 30%, you now need to go and redesign the work and do the remainder of it. That's where the bulk of the work will sit for the next two to three years. And that's kind of what we're seeing now in the industry.
[18:16] So let me So I think a couple of key questions to leave you with. Who would have to do their job differently? And if you can't give a specific answer about your targeting, marketing, supply, research, if you
[18:28] can't explain how the job is going to be different, it means you don't know how to transform the organization. It's going to be another excuse and you have to go back to the drawing board. You have to have a very, very clear idea by working with the team on the ground, leadership as well, but you have to have
[18:41] a clear idea around the work that they need to be doing and what's going to benefit the organization and how to re-engineer it around that. So finally, again, it's the org chart that everyone's been unwilling and for very good reasons cuz it's very, very
[18:53] hard work um to redesign. That's where this is going to sit. And this is The companies that tackle this head on and the ones who will be successful are the ones who are going to go into it, I think, full speed and figure things out, not wait. Cuz if you sit on the sidelines, everyone else will beat you
[19:05] to it. This transformation will take time. It's not something you can accelerate because we've got Claude or uh Codex. This is going to take a lot of hard manual work in the trenches with the team. Thank you very much for your time. We're
[19:17] over at booth 6740. If you have any questions, we'll be here for some Q&A afterwards as well. Thank you guys for turning up. >>

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.