From Data Platform to App Platform: Building Production Apps on Lakebase

Summary

Lakebase provides a fully integrated operational database layer on the Databricks Data and AI platform, enabling low-latency reads, transactional writes, and app serving without external databases or sync pipelines.
This video covers three practical app patterns built on Lakebase: read-only analytics surfaces, transactional write-back applications, and AI context stores, each demonstrated with a customer churn command center.
A live demo shows how Lakebase integrates with Unity Catalog governance and Databricks Apps using Streamlit, Flask, or React, with Databricks Asset Bundles enabling multi-environment deployment.

From Data Platform to App Platform: Building Production Apps on Lakebase

Your lakehouse handles analytics beautifully. But the moment someone needs low-latency reads, transactional writes or an app serving layer, you're stitching together external databases and sync pipelines. Lakebase offers a different path — this video shows you how to build on it. A Databricks MVP will walk through practical patterns for building production apps where Lakebase serves as the operational layer, fully integrated with Unity Catalog governance and Lakehouse analytics. We'll cover architecture patterns, the interplay between Lakebase and Databricks Apps, latency and scaling considerations, and the honest gaps practitioners need to plan around. You'll leave with architecture patterns, design decisions and a realistic assessment of production readiness.
Talk By: Doug MacWilliams, Managing Director, West Monroe ;

Chapters

FAQs

What problem does Lakebase solve for teams building apps on Databricks?

Traditional lakehouse platforms handle analytics well but require stitching together external databases and sync pipelines for low-latency reads and transactional writes. Lakebase provides a fully integrated operational database layer within Databricks, eliminating that external infrastructure.

What are the three app patterns demonstrated in this video?

This video covers read-only analytics surfaces for consuming lakehouse data, transactional write-back applications that let users act on insights, and AI context stores that feed agent workflows. Each pattern is illustrated through a customer churn command center demo.

How does Lakebase integrate with Unity Catalog governance?

Lakebase is fully integrated with Unity Catalog, meaning the same governance controls—access policies, lineage, and deployment guardrails—apply to transactional data as to analytical data. This allows teams to enforce consistent data controls across both operational and analytical workloads.

What app frameworks are supported for building on top of Lakebase?

This video covers Streamlit, Flask, and React as front-end frameworks for Databricks Apps, all of which can be backed by Lakebase as the transactional layer. Databricks Asset Bundles are used to manage multi-environment deployment across these application architectures.

Full transcript

[00:08] I appreciate everybody joining today. Um, my name is Doug McWills. I'm a managing director at West Monroe. We're professional services consulting firm. I'm also a data bicks MVP. Uh, my background is helping clients build data strategies. Often it's on data bricks.
[00:22] uh we help them architect those solutions and then I lead those teams to build those uh and and build those technical solutions. So appreciate the opportunity to present with you here today. Today I'm going to be talking about building applications on top of
[00:36] your data platform. So today I'm I'm going to talk and cover why you should be thinking about building applications on top of your platform. Um we'll be talking about kind
[00:49] of the what to build. So considering three different uh patterns with how you can build applications. I've got a simple demo focused on customer churn. So we've uh architected and built and
[01:02] ingested data from different systems. We're going to put in an application and show you how you can take action on your lakehouse data already and we're going to talk about how you can create tasks and send that data out to really operationalize your data and enable your
[01:15] business. And then we're going to take a step back. We're going to talk about how you should be building this. So what app frameworks are relevant um and what to plan for as far as the limitations and some of the considerations before you
[01:28] get started. So maybe before I get started, how many people here by show of hands are are have built applications on top of data bricks? Okay, so probably about half. Great to hear. How many people are using AI to
[01:41] build those applications? It's the pretty much the same number of hands. Yeah. So that's not a surprise and that's that's kind of the the the why now. So I'll I'll start with this. So
[01:53] traditionally building a a data platform uh building uh with your data consolidating your data really enables a number of different things. Traditionally what we're going to do is build a medallion architecture building a data warehouse. We're going to enable
[02:05] BI and analytics. Uh adjacent to that we often build data science and ML. Along the way we need to make sure that data is clean. So we do things like data quality and MDM. And then also we send that data back out to people and other
[02:18] systems. That's traditionally where the data platform has lived. Now with the advent of building with AI with uh lakebase as your Postgress database behind the scenes for OLTP as well as the app uh capabilities on top of data
[02:32] bricks. Building an application has never been easier and building an application really is a new way to think about leveraging your lakehouse as another data asset and operationalizing that for your business.
[02:49] So three different patterns that we see as far as building uh applications on top of your or your lakehouse. So on the left hand side here really uh think of this as a readonly version of your data being able to serve up your data. BI
[03:01] tools are are good can can serve up data but sometimes it takes a long time to do that. um uh you can use applications you can vibe code a version of your data and your and your uh kind of a BI version of that for a readonly provide different
[03:14] interactions and and slicers and capabilities that's something that you can do in the middle here we have the uh the transact so how do you operationalize your data how do you build the rightback functionality how do you understand what's going on your
[03:27] business how do you keep track of that how do you uh tell others to take action and how do you follow up on that that's where our demo is going to be focused on today and then on the on the right hand side here as we think of all the costs around AI agentic building um building
[03:42] context and remembering what's going on with your business is another use for lakebase so you can organize prepare your data for efficient building and and and building applications on top of your lakehouse.
[03:58] So moving along here as we think about this demo here is a simple architecture. So um the scenario we have here is consider a B2B that's going to be ingesting some common data sources from
[04:10] their source systems and preparing it in order to identify customers that are potential to churn. So we're going to be ingesting data from your CRM. We're going to be ingesting data from your billing. We're going to be ingesting data from support and usage. With some
[04:23] of the announcements today around customer data platform, imagine that our lakehouse not only ingests that data but is also to do able to do the ID resolution and be able to knit the data between these different systems and organizing and prepare and calculate a
[04:36] risk score. This risk score and other supporting data is then prepared and presented into lakebase and the data bricks app. We then have this operator which is uh maybe a customer service manager who's able to engage with the
[04:49] app u monitor it uh observe some of the trends and analysis that's presented in the app but then also take action on that with the tasks. So all of this is to be able to take action in a uh uh
[05:02] probably two things I also comment on here. This is a simple pragmatic approach. Um I think this is a a great way to start because this isn't requiring real-time data. This is a batchbased process where you would
[05:15] ingest your data maybe a few times a day and this is also not really using any agentic process. Those are things you could add onto this at a at a subsequent evolution of the solution.
[05:28] Um so again here's a view of the uh I'm going to switch over to the demo here in just a second but again the idea here is you have a customer health uh command center. Uh we're going to be looking at a few select customers. We're going to
[05:41] have a calculated risk score. We're ingesting that data from the CRM from billing support uh into our lakebase. We're going to score that risk and we're going to be uh kind of looking at creating an action plan to resolve any
[05:54] potential customer churn and we're going to be writing that back out. So I'll switch over to the demo here. So hopefully you can see this. Um so on I'm
[06:06] having so on the left hand side on the top there you can see there's three tabs that are part of this simple application. One is called the health queue, one is the action workbench. So once we create tasks that'll go to the
[06:18] action workbench and then finally an admin or data just to monitor and make sure the data has been synced back and forth. Um just below that you'll see the risk threshold and you can see that we can actually adjust uh what threshold we
[06:30] want to uh our customers to pop up on our on our risk score. Right now we have Ridgeway Retail which is uh one of the highest critical scores. That customer is in the middle here. So it has a 100 uh risk score. Again we've analyzed the
[06:45] CRM the billing support and usage here. We've come up with a number of factors that are that are concerning mainly that the they haven't paid their bills, namely that there are there's missed SLAs's within open tickets. And so we
[06:57] can scroll down here a little bit and we can see a bit more about some of those BI capabilities we touched on before and we can see where they're um kind of not paying their bills. And then you can see
[07:09] also here the usage of our service has gone down. So again, this customer looks like they might be trying to find another service and potentially leaving. So knowing we have a playbook for customer retention and we've developed
[07:21] that, we can come up here and we can click um create our our our plan. So what this does is looking at the data we have uh we built this ahead of time. This is a
[07:33] playbook to save this cumber customer or retain this customer. Um we're going to reach out to our customer success. We're going to reach out to our uh uh escalate our support tickets to our support
[07:46] leadership to make sure that they know that this is something they should take action on. We're going to schedule a meeting with Nora Brooks, the executive sponsor here, to say, "Hey, why making sure everything's okay?" So imagine
[07:58] we're taking this data, organizing it, preparing it, but that that doesn't mean that it's all AI that's going to solve the next steps. We're taking this and creating tasks that are sometimes given to people, sometimes given to these source systems. So you can sit at the
[08:11] middle of this command hub and help uh your business operate and and and kind of take that best of both worlds. So scrolling back up to the top, we can see at the top that there are four that are
[08:24] cued for rightback. We'll go over to our our action workbase here or workbench and we can see these tasks that are listed out below for this. So in here we can change the owner of who's who's part
[08:36] of this. We can change the due date and we can even change the priority and kind of organize and make sure that the the details of these tasks are correctly and completely populated. And then we can save these tasks and then send those
[08:49] back out to those systems and these can be a format of different variety things either API integrations back to these source systems you emails to other people. This can all take the place of something else. And again, imagine the
[09:01] simple example, but imagine you can also trigger AI workflows and AI processes that are agentic in nature to actually automate some of these steps. Again, a crawl, walk, run approach. So that really concludes the demo today.
[09:16] I'm going to switch back over here. So now we want to take a step back and think about what are the app frameworks
[09:28] that you could be building with. So uh as we heard this morning data bricks is announcing the the Genie app builder. So today the app that I just showed was really a react version. Um but really
[09:41] when we think about building with applications we think about these three different formats. So on the left hand side we have a streamlet app. This is the simplest thing. This is great for internal applications for your business.
[09:53] This requires very little UI code and understanding and this is great for just a handful of pages. If you graduate from that, have more than just a few pages or have a complex uh route that you need to
[10:06] take, then you would probably look at using Flask. Again, what I used today was React. This is probably the most polished. This has the full uh uh web stack that you could you could imagine and and you can customize and tweak all
[10:19] the buttons and steps that are in there and really this could be really business facing within your organization and really enable those business stakeholders to have the right branding and have the right style. I'll be honest with you with AI agentic
[10:33] process and building of this. We have these these called out with complexity and speed and having these differentiated the application I just showed you was able to spin up in just a couple hours and actually able to iterate on it and and kind of give it
[10:45] feedback. So you the I would say that this complexity and the speed is being compressed by AI and probably the limitation for what you should be thinking about is what skills does your technical team have the democratization
[10:58] of being applications being built by the by the business that's what you probably should be focused on right the the ability to build these things is actually become quite trivial so how do you control that
[11:10] so these are the frameworks that I think would be good to think up. So again because it's so uh because building applications are so easy now
[11:22] governance is the new hard part. So what I would encourage you to think about is how do you um put controls? How do you put processes around this? How do you make sure that different business stakeholders are not building the same
[11:35] thing? Um how do you have uh uh cost controls? and how do you uh not drown in the proliferation of apps and assets that could be built. So the new discipline is how do you put people how
[11:47] do you put process how do you put the technical guard rails think of that as maybe a skills.mmd file that would use and embed here and how do you make sure that your deployment process is robust and resilient to that.
[12:00] So all right so a few considerations uh before you get started um this is the applications today are
[12:13] really going to work with au authenticated users on data bricks so you you need you it doesn't have the anonymous public access yet so you need to make sure that as you grant and and build these applications the end users
[12:25] are able to be on data bricks um the next thing is planning your identity model. So thinking about OBO versus a service principle, you know, when you build these applications, do you want
[12:37] the user to all have the same perspective and have the same access to data, have the same access to buttons, or do you need to build different perspectives and have different controls on what what version of data and what buttons they're able to click or push?
[12:51] Those are really important as you kind of think through and build these. I I did have on here that the app scale uh the app tier doesn't scale to zero. announcing today. Obviously that does scale. So excited about the lower cost um and kind of remediating that
[13:04] challenge. Um the other thing is as you build these applications a lot of times you're writing that back to a lakebase and and you want to sync that into your Delta Lake. There will be a lag of a few seconds between when you write back to
[13:16] lakebase and when that gets synced into Delta Lake. So if you're building analytics on top of the clicks and the actions of your application, just make sure you're also including that in your design so your your analytics doesn't get out of sync with what's being synced
[13:29] across the platform. So uh before you build again the message here is controls and consistency um setting those development guardrails and these are uh kind of the deployment standards um uh these are the the skills
[13:43] the which of these frameworks are you going to use? Are you going to enable all users to use all of these frameworks? Are you going to narrow that down? Uh what data should these groups have access to? The other thing announced today is app spaces. So how do
[13:57] you uh divvy up the app spaces within your business departments or by functions and how do you control where those apps get built and who has access to those apps and then what guardrails do you have on people that are in those
[14:09] spaces. So, another thing is it's exciting to hear all of the announcements here at at the at the summit. Um, but understanding that maybe not all these are GA at the moment. As you think about building these
[14:22] solutions, you I would also encourage you to think about when these are going to be GA because you want to make sure your solutions and your UI and your customers experiences are solid and you don't want to be building something that
[14:34] might be in a preview state or may not be completely ready. So understanding what those conditions are, when those are going to be GA will help you design a solution that's going to be uh have a good customer experience because you
[14:46] really only get one time to build that and if they don't like it, you can really be fighting an uphill battle afterwards. And then finally, as I'm pretty sure 99% of the people here are going to be building with AI, make sure
[14:58] you're also building with DABs because as you deploy this into dev, once you have the assets for dabs, it'll make it much easier to deploy that into the higher environments and uh that'll just make your life easier. So
[15:12] yeah, um I have uh that really concludes the presentation and demo I have now. Um I have a little bit of time for questions or I'm happy to take those afterwards.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.