Designing for Agents: A New Playbook for AI-Native Product Experiences

Summary

This video introduces three practical frameworks for designing AI-native product experiences, developed by a Databricks staff product designer with three years of hands-on experience building agentic products.
The Levels of AI Autonomy framework (L0–L5) helps product and engineering teams align on how much work an AI agent should perform versus a human, with Databricks Genie used as a real-world illustration of each level.
Designing for agentic experiences requires explicit trust signals, human review checkpoints baked into workflows, and transparent UI surfaces that display agent output—because users must understand and verify what an agent has done before acting on it.

Designing for Agents: A New Playbook for AI-Native Product Experiences

For 30 years, we’ve designed software around a human who needed to be in the driver’s seat — navigating menus, configuring pipelines, running queries. That human is leaving the driver’s seat.
In this video, we explore what happens to product design when AI agents become the primary users of your platform. We’ll look at real workflows being disrupted across industries — from legal to design to data engineering — and introduce the Levels of AI Autonomy framework (L0–L5), a practical tool for aligning product and engineering teams on how much the AI should do versus the human.
You’ll leave with a concrete playbook for deciding when to build a UI, when to get out of the agent’s way, and how to design the trust and control surfaces that make agentic experiences actually work.
Talk By: Ridhhima Gupta, Staff Product Designer, Databricks ;

Chapters

FAQs

What is the Levels of AI Autonomy framework and how does it apply to product design?

The Levels of AI Autonomy framework (L0–L5) is a practical tool for aligning product and engineering teams on how much work an AI agent should do versus a human. L0 represents full human control while L5 represents a fully autonomous agent, and the framework helps teams decide where on that dial a given product feature should sit.

How are AI agents changing the way software products are designed?

For over 30 years, software UIs were designed as translation layers between what a system could do and what a human could understand—because humans were always in the driver's seat. Now agents operate through APIs and MCPs and sometimes navigate UIs themselves, meaning product designers must rethink what role a UI plays when an agent is the primary user.

Why are trust signals important in agentic product design?

Because AI agents are non-deterministic and can hallucinate or leak data, users need visible signals that confirm an agent's actions align with their intent before they act on the output. This video covers trust signal patterns and human-review checkpoints that designers can bake into agentic workflows to build user confidence.

How should designers handle errors and hallucinations in agentic UX?

The video recommends treating error handling, hallucinations, and data leakage as first-class design concerns rather than edge cases. This means building transparent agent output surfaces, providing contextual feedback when an agent is uncertain, and designing explicit human-review steps for irreversible or high-stakes agent actions.

Full transcript

[00:10] right. Hi everyone. Um, it's the last day of days and I hope you've had a fun and exciting conference so far and ready to head into the long weekend because I sure am. Um, so in this session we're
[00:22] going to talk about a framework for designing for AI native experiences. something that I as a product designer at data bicks have been working on for the last three years and today I'm going
[00:34] to share a lot of my knowledge that I've gathered designing for a lot of agentic products within data bricks and share some frameworks with you all as you're developing your own applications. So,
[00:46] quick show of hands. Um, who here has been white coding their own apps in the last month? All right, a lot of you. And when you're
[00:58] white coding these apps or you're building your own apps production grid, um, show off hands when you've you've had an agent integrated as part of the app.
[01:11] Okay. Okay. I can see a few people here. All right. Um but the thing is I I can see that this number is going to keep going up. Um because agents are peripherating and agents are taking up a
[01:25] lot of uh work away from humans in terms of doing a lot of the grunt work that humans used to do previously. And so the key question that a lot of us will face in the next couple of months as we're building our own apps is how do we
[01:38] design experiences in this new agentic world where agents are doing part of the work that humans used to do. And so in this talk we're going to talk about three frameworks that you can use when
[01:50] you're designing agentic experiences. and we'll talk through an example at an end at the end as well so you can understand how this framework can be applied when you're building your own products. Um, so yeah, let's get
[02:03] started. I'm going to skip through this forward-looking statement. I'm sure you've seen several of these throughout the conference. So taking a few uh decades uh taking us a few decades back you know for over 30 plus years we've
[02:17] been designing software with a human at the center of um you know the experience the human is in the driver's seat the human is actually has been making all the decisions had full agency on what's
[02:29] happening in the product as well as they were working in very deterministic systems and UIs were supposed to be this were created to be this translation layer between what the system could do and
[02:41] what the user could understand. But that's changing as we all know. Um, agents are now working through APIs, MCPs, and even navigating UIs themselves and doing the work that humans used to
[02:55] do. And now it almost feels like the agent and the humans are actually sharing the driver's seat. You know, the human is shifting on bit on the side and asking the agent to come with them. And
[03:07] maybe it looks like in the future the agent could potentially take the full driver seat as well. And so as an example, you know, earlier if you had to create a simple Salesforce data pipeline on data bricks, you would open data
[03:20] bricks, find a workspace, pick a cluster, write SQL, you'd go through a lot of these steps manually as a user versus now it's changed more to an intentbased system where you can just
[03:32] tell it tell something like Genie code to set up a pipeline to for um understanding Salesforce data churn and refresh every day at 8 a.m. Right? So
[03:45] the agent does all the rest of the work and you just had to as a user express your intent and um it's not just in the data work that we are seeing this shift. It's happening everywhere. So all sorts of
[03:58] industries are getting disrupted with how agents um like how agents work. So from industries like design so Figma is going through a big shift. Um legal like
[04:10] with Harvey we are seeing a lot of lawyers using agentic applications to do a lot of um legal work much quicker. So is a very interesting example. It's a um
[04:22] it's a music creation application that is one of my favorite applications right now. If you haven't tried it uh highly suggest checking it out. It's it it helps it's using agents to actually build really really compelling music um
[04:36] out of the box and other applications like cap clarna again for video editing and uh customer support. So we are seeing a big shift across all of these products and so I think as um people
[04:51] working and vibe coding our own applications and products uh where we are integrating agents within these applications there's several questions that come up in front of us right so first is which workflows should you
[05:05] offload to an agent when you're designing these products right like how do you actually think about what workflows go to an agent versus which workflows still need the human input at all. Um is there a way to kind of figure
[05:17] it out? Is there a framework that one can use to determine that? And then uh the last qu the last part of it is um I guess as a designer I get asked this a lot that hey do we even need a UI for a
[05:30] specific task anymore? If an agent is already going into using an MCP scrolling through the UI of another software like do we need to build this UI surface? And uh it's a question that
[05:42] I think about deeply and we're going to talk through that as well like what is the role of UIs in the future and how do do we even in which cases do we need a UI and why do we need them anymore
[05:55] um and so a big part of as we try to understand or ask this question is um it's important to shift our thinking from you know we are not just adding AI to our products we have to um now
[06:08] rethink the way products are built in agentic era. And so from my work at data bricks and also brainstorming with lot of other designers in the community as well as some brainstorming with Claude
[06:20] um I'm going to share with you three frameworks that can be useful for you as you design these systems. So um the first one is the autonomy dial. So um what this means is you know
[06:33] it's about trying to pick a pick the right level of autonomy for your agent. And we're going to deep dive into that in a bit. The second is um understanding implications of risk, capability, and
[06:46] trust. So how do you incorporate these various um important aspects of agentic design into your into the app UX that you're building? And the third is um
[06:58] also thinking about these new emerging UX surfaces that are coming up that didn't exist until you know one or two years ago and how can you incorporate that into apps and agents that you're building.
[07:11] So let's begin with the first um framework which is the autonomy dial. So the first design decision um for any agentic feature that you'll want to make is how much the agent should do because
[07:24] that one choice will drive almost everything about the UX of your app, right? And we've actually seen this um before in in a very parallel universe
[07:36] which is for self-driving cars, right? So self-driving cars um uh are a much more evolved industry. They've um you know they've been around for now several years. They it's a very specialized
[07:49] agent designed specifically for self-driving. But one of the cool things about um we can use this as a proxy because self-driving cars also have there's a SAPE levels of driving
[08:01] automation standard for self-driving cars which is an L0 to L5 where L0 is a human driving the car manually. You know they're doing everything manually versus L5 is full automation. uh it's fully
[08:13] autonomous and what we see with the Whimos of the world today but um like I said self-driving cars are more specialized right so um but we can still use these levels of different autonomy
[08:27] of agents as proxy as to how we think about um multi-purpose agents that we are working with today so um here's an adapted version of this framework that I've created um for
[08:41] multi-purpose agentic work. Um, and as you can see, as you again, this again is trying to adapt through the L0 to L5 framework where L0 is everything's manual. The human is in a more
[08:53] deterministic um uh doing more deterministic work, has a lot of control versus L5 is the a the human is basically maybe even just planning at a high level a goal and the agent goes on
[09:06] and plans, executes and corrects itself over time. And as you look at each level, like as you go from level zero to level five, you will see that the work that the agent does versus the work that
[09:19] the human does changes a lot um as you progress through each level. And we go through that real quick.
[09:33] for example, here if you see this is the um long dreaded form of creating a cluster on data bricks. Um but here again the user would be in full control. They're doing everything manually.
[09:45] There's no agent or AI suggesting anything or helping the user in any way. Um level one and level two is where I think the world was maybe 3 or 4 years
[09:58] ago. Um so level one was uh when the agent can maybe give you some suggestions. So a good example that I always think about level one is when you you know maybe 10 years ago when you
[10:10] were writing an email in Gmail uh it would give suggestions of the next words to write and you as a human had the agency to choose or reject those suggestions right but level two um is
[10:25] where the agent actually creates a draft for the user. So for example uh in data bricks um if you notice there is this u about I would say about three years ago we released this feature where you would
[10:39] get a description of a table that um autogenerated and as a user you could accept or deny it. Um same within coding notebooks. We would whenever you ran into an error uh we would generate or
[10:52] help you diagnose the error, suggest improvements in the code. But it was always the human was in the driver's seat. So the human had to accept these suggestions to move forward. The agent was not working on these suggestions
[11:05] automatically. Level three is where level three onwards is where things get super interesting. Um so level three is where the agent does a lot more complex tasks
[11:17] autonomously and the human reviews the outcomes and overrides or corrects it. So this is an example of uh genie um I'm sure a lot of you must have seen it uh throughout the conference but uh the
[11:30] idea that hey if you're working on a simple dashboard um you can again give the prompt to genie and it's it's building the dashboard for you in real time. It's making a lot of decisions for
[11:42] the user. It's executing a lot of these tasks automatically. Um, but the user still has ability to override a lot of the decisions that the the agent has made. Right? So here the UX starts to
[11:57] change from um just manual um selection and manual control to more of a a reviewing surface and where the user can review and override things when needed.
[12:10] So um level four is where the agent can take up much bigger and a lot more complex tasks autonomously and the user's job now changes to monitoring the various tasks and overriding if needed.
[12:24] So and one thing I do want to caveat is these levels are not meant to be you know um they're completely segmented. it definitely a spectrum and but the way to think about level four is like for examp
[12:37] let's talk through it as an example so let's say you were to build a ML recommendation system previously you would have to build a notebook do a bunch of stuff um on data bricks to actually um start building a
[12:50] recommendation system you would might might have u various different assets associated within data bricks but
[13:02] autonomously to work on this complex task and the agent quality is that high that you can actually trust what the agent is doing. And so here the UX starts to change from again manual
[13:14] selection and manual uh configuration to more of a observability and monitoring experience. So what you see on the screen here is that the user is able to look at what are the various things that are configured, can look at the status
[13:27] of it, can monitor if the user needs to look into something, but only when something breaks, they might want to go into the system and see what's actually happening. And finally, level five. Uh again, this
[13:40] is in my opinion highly controversial. uh because in the self-driving cars world, level five is where um you know uh the agent is really is able to do well and is is able to operate
[13:52] autonomously without making a lot of mistakes. But right now again we are working with u not very specialized agents and in that world it does seem like we we are a little bit further away
[14:05] from having an agent who's not you know that's accurate and understands a user's context very well. But um ideally in a level five environment this would again change to um a user just reviewing a lot
[14:19] of alerts and keeping tabs of what's happening in the system where a user might no longer need to go into the details of the system and understand what's going on under the hood. So again, the goal of this level 0 to
[14:32] level five framework is to map out as you're designing your um user experience is to map out various parts of the workflow that you're designing and which level your um agent is. And we'll talk
[14:45] through that um talk through this through an example towards the end. But um a key part of this is also like just because an agent can operate at a level doesn't mean that you should design to have that agent at that level
[14:58] in your application. And I think this is where risk and capability and trust actually come into play. So um here we by we have to consider two questions
[15:10] here right. So one is capability which is can the agent do the task well? Does it make errors? Um or you know um is it does the job fairly accurately or uh and
[15:23] the other axis is risk right? So, how risky is it if the task if the of the agent if it gets it wrong, does it break my system? Does it does it have reputation implications? Does it make
[15:35] any big changes in my organization that were so as we think about it like um one thing that you kind of can consider as you're thinking about what levels to assign for your various agents is where
[15:48] does the task that you're trying to do fall under this axis. So um the top left is actually the holy grail which is the most ideal scenario where there's a lot there's very little risk in the task
[16:00] that the user is doing and the agent also has higher capability of producing a good outcome. So for example, if you're you know prototyping uh an internal dashboard or you're working on
[16:13] a notebook exploration that you perhaps are not going to uh build any production level systems in you can still provide like a higher level um agent for this particular task in your application
[16:26] versus um on the lower end if if there's much higher risk um and you also think that the the agent might make mistakes there's higher probability of making mistakes, you might still want to bake
[16:39] in manual review into your agentic design. So an example of this is like you know if you're managing API keys or if you're creating clusters, setting up budgets, uh these might have higher
[16:51] implications uh for your business and so you might want to as your design bake in the human review. And lastly, the other two quadrants are they're a bit more on the spectrum. um they can vary a lot
[17:05] depending on again your risk appetite um as a business as well as the model's capability. But one quick thing on this is like so on the top right right if you're seeing well my agent can actually
[17:17] do really well on this task it does it perhaps all you know doesn't do a lot of mistakes but there is a higher risk of the agent going uh you know if the agent
[17:29] does make that mistake in 1% of chances there is a higher risk to my business. So for example, if a user if uh you know if you're working on like sharing an application uh in production or you want
[17:42] to change a user's permission on a table, you don't want the agent to just do that automatically. So in those types of cases, you might lower the level of the agent that you deploy. And same for
[17:54] you know if you have lower capability of the agent but um a lower risk you might still want to lower the the autonomy of the agent and bring in a bit more human control.
[18:07] And um another part of this whole um agentic design is also like how do you build trust with the agent. So um big part of this is incorporating you know logs of how the agent is
[18:20] thinking the decision ration uh providing a user with easy overrides showing progress and also asking the user when it's unsure and we are actually seeing a lot of these trust
[18:33] mechanisms in coding um u platforms like um cursor um and codeex. we are seeing that these principles are actually being incorporated. But uh a key thing to
[18:45] again consider is using these principles in your own applications as well. Um and lastly a big part of this um framework a third part of this is like
[18:57] you know as you're designing for agentic experiences also consider incorporating new and emerging UX surfaces in the area. So we're now seeing like this u
[19:10] it's an exciting time for designers and developers because there's a new breed of UX surfaces that are coming up and um these are meant to be just in time and hyperpersonalized to a specific user. So
[19:24] on the left here you would see this is um something that claude code created when I was planning a trip to Vietnam. It actually planned the trip in real time and suggested this highly
[19:36] interactive experience for me as a user uh to go through and um here if you imagine right I didn't have to leave where my work was happening and I could again explore um or or get my task
[19:50] accomplished within that within that agent's experience. And on the right was a simple experiment where I was trying out an SF buy versus rent calculation. And again here also it generated a
[20:03] simple UX surface for me to explore that rather than just you know spitting out text. Um and another part of this is again uh as you can imagine these experiences or
[20:15] widgets that you're seeing can be highly contextual to what the user um who the user is, what their behaviors have been on the platform etc. Um and another part of like this
[20:28] emerging UX is um is that other user that's going to be using your system which is designing for agents. So if you imagine your agent is using the app and an a third party agent is using your app
[20:42] uh it might be worth considering how what's the right user experience you're giving that agent as well right so common things that we've been considering internally within data bricks and uh maybe there's a lot more
[20:54] uh here as well but some of the big things we are seeing is that you know make sure the error messages that you're showing are machine readable the agent can actually understand what's going on
[21:06] uh because previously we made sure that error messages were again more human focused but we want it to be uh machine readable as well. Make sure there's API parity. So every UI action that you're
[21:18] building has a programmatic equivalent by an MCB and uh docs is actually a big part of u designing for agents as well, right? So making sure a lot of docs again are focused more on a user who has
[21:32] a lot of context of your platform um and might have other context available as well. But for an agent crawling through a dock and figuring out what to do in your system, it might be important to
[21:44] maybe running that by an agent and seeing how it's working. Um all right. So another part of this is again as these new surfaces are emerging
[21:56] there were a few guidelines that I was kind of thinking about um here. So one is um instead of designing like full UIs um one of the things to consider is um
[22:08] like we saw in that trip planning experience is that designing more self-contained blocks of UX um where these blocks can be reused over and over for different users but it also has
[22:22] dynamic content. So depending on what the user is doing, what kind of user it is and um where this block might be shown, we might change the content
[22:34] presented here as well. And uh the lastly is again thinking about more hyper personalized UX um for various users on your system.
[22:47] All right. So let's walk through this uh through an example. Uh it's a very simple example but hopefully it can give you an overview of how we might put this into practice.
[22:59] So let's say um you are uh designing an agentic app which helps a user build a simple dashboard and that's your uh prompt. So the way I
[23:13] would think about using this framework is I would go about first laying out what are the various steps that are within this process. So it you know you'll start with like okay I have to
[23:25] find the data in my system. Um the user would perhaps have to write a query. Uh then they might go about designing a layout, choosing the right visualization
[23:37] and also then sharing out the dashboard with someone else when it's ready for review. Um and but here is once you've mapped out the various parts of the journeys
[23:49] where you can start figuring out what is the right levels um that you want to assign to your agents um within this um within this workflow. So for example, let's start with the first one which is
[24:02] about finding data. So in this particular case again it depends on how accurate or what capability your agent has. So in this example, I'm assuming that the agent is right now not that
[24:15] good in figuring out what's the right data to find or to actually use in building this visualization. So it's I've put it a bit lower in the spectrum where you know this there's maybe lower
[24:27] risk but also lower capability um to find uh the right data lower risk because the user can still override this easily but things like where you have to choose the right design layout the right
[24:39] visualization and write a query um I based on the system that I was working in I thought that okay this is something that an agent can do fairly well and there's not a lot of risk involved here
[24:51] because even if things go wrong, it can be overridden really quick easily. But um something like sharing out the dashboard with someone else that perhaps is on a higher risk spectrum because
[25:04] maybe the data is wrong or there's some errors in it. It's showing wrong numbers uh or it's leaking out the data to someone else that shouldn't be where it shouldn't have been shared out to. So
[25:16] that's how I would I would perhaps try to put it in a matrix here and then um try to uh put together this uh dashboard uh workflow again in this levels
[25:30] framework where again the levels don't have to be one and by you know very strict but the idea is that where in the spectrum can they can they lie so in this case again I've tried to um show
[25:42] that spectrum but also if you notice there is this intent aspect now as well that comes in with agentic systems is you a big part of this workflow is like I'm also asking for intent from the user.
[25:57] So a very crude example of this um uh is you know from the here on you can start sketching on a few ways by which you can accomplish that same task. So for
[26:09] example, the first part that that we looked at which was about finding the right tables to use for when you're building a dashboard. So let's say the user inputs like hey I want to build this dashboard for Salesforce data.
[26:22] There could be three different ways you can accomplish this in the UI. So one is you know there's a level one where in the user is still manually selecting the various tables to use and only then the
[26:34] agent goes around and actually designs the system or the the dashboard. Level two is where it's actually suggest it has done um some searching and it has a fair estimate of these are the tables to
[26:47] use and it shows it suggests those tables to the user to use. Uh and level three and four is maybe the agent is good enough where it does make decisions on its own where it's figured out the
[26:59] right tables to use and but it presents those entities to the user to to be able to edit. And so um if you're working in a more conservative system, you know, it
[27:11] might be worth going towards level one or level two. Um but in in case of like for example in Genie we we actually operate in level three level four when we are designing dashboards cuz now the
[27:24] agent is is has a higher capability of figuring out the right tables. Um and as you keep going like based on this framework and based on where this your agent lies in the system um into
[27:37] that spectrum you can figure out what the right UI surfaces could be. And I've intentionally left it at a very high level as a sketch because this can help you first like design the system before you maybe give these sketches to cursor
[27:51] or claude and and start building out your UIs. Um and so yeah um in summary yeah there are three frameworks we talked about. there's agent autonomy, there's trust
[28:04] and um incorporating incorporating trust uh within the system and emerging UX's. Um the other part of it I would say is you know as you look through this framework map your workflows break it
[28:17] down and map your workflows to various levels. Um and a key part of this is there's no one autonomy level that you would have for your entire app. it would keep changing depending on the various tasks within the product. And um another
[28:32] key learning of this is think UI surfaces are not going away. Um the way a UI surfaces is actually might change depending on the level of autonomy of the agent but
[28:45] like a very high level of general purpose autonomy of agents um it's not going away anytime soon. And uh the last piece is again designing for both uh the
[28:57] human and the agent user. So as you're designing systems, don't just think about um one of these personas. And uh yeah, so that's it from me. Um if
[29:10] there are any questions can use the mics and happy to help answer that.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.