A Prescription for Success: Leveraging DABs for Faster Deployment and Better Patient Outcomes
Summary
- DABs Framework: This video demonstrates how Databricks Asset Bundles (DABs) accelerate deployment cycles and improve patient outcomes in healthcare through faster delivery.
- Healthcare Applications: Learn how DABs enable rapid iteration and deployment of healthcare solutions on the Databricks Data and AI platform.
- Operational Excellence: Discover how structured deployment frameworks improve reliability, governance, and time-to-value for healthcare organizations.
A Prescription for Success: Leveraging DABs for Faster Deployment and Better Patient Outcomes

Talk by Alex Owen, Sr. Solutions Architect, Databricks ; Brendon Allen, Principal Data Engineer, Health Catalyst
Health Catalyst (HCAT) transformed its CI/CD strategy by replacing a rigid, internal deployment tool with Databricks Asset Bundles (DABs), unlocking greater agility and efficiency. This shift streamlined deployments across both customer workspaces and HCAT's core platform, accelerating time to insights and driving continuous innovation. By adopting DABs, HCAT ensures feature parity, standardizes metric stores across clients, and rapidly delivers tailored analytics solutions. Attendees will gain practical insights into modernizing CI/CD pipelines for healthcare analytics, leveraging Databricks to scale data-driven improvements. HCAT's next-generation platform, Health Catalyst Ignite™, integrates healthcare-specific data models, self-service analytics, and domain expertise—powering faster, smarter decision-making.
Here's more to explore:
Databricks named a leader in the 2024 Gartner® Magic Quadrant™for Cloud DBMS: https://www.databricks.com/resources/analyst-paper/databricks-named-leader-by-gartner
An open, unified approach to your data, BI and AI workloads: https://www.databricks.com/product/databricks-sql
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements
FAQs
What are Databricks Asset Bundles (DABs)?
This video explains that DABs are a framework for bundling and deploying Databricks assets, enabling faster and more reliable deployment cycles.
How do DABs improve patient outcomes?
According to this video, DABs accelerate the deployment of healthcare analytics and AI solutions on the Databricks Data and AI platform, enabling faster insights and interventions.
What deployment benefits do DABs provide?
This video demonstrates how DABs improve deployment speed, consistency, and governance compared to manual deployment approaches.
What healthcare challenges do DABs address?
This video shows how DABs enable healthcare organizations to rapidly deploy analytics solutions that improve operational efficiency and patient care.
Full transcript
Right, everyone. Thank you for coming to our talk on a prescription for success, leveraging data bricks asset bundles for faster deployments and better patient outcomes at Health Catalyst. I'm a solutions architect here at Data Bricks. But first, I want to talk about a forward-looking statement that I did not read. Um everything here is just the best to our knowledge. Uh no guarantees. And then one more slide for me.
And then also please please please to make these presentations better over time please fill out a survey and let us know how we did. So like I was saying I'm uh a solutions architect here at Data Bricks. And I'm excited today to present with my friend and I'll let him introduce himself. I'm uh with Health Catalyst. Recently, I've picked up a new hobby of cycling and wanted to take you guys on a more metaphorical uh ride uh with us about the health catalyst journey leveraging data bricks.
Um I'm so excited to be able to be here today. I've gained so much from the community uh and really wanted to give back. So, thank you to everyone that's ever contributed to a community forum, presented in the past, and those that may present in the future. uh you guys helped inspire this presentation and very grateful to be here today. Uh I'm representing the work of dozens of hardworking engineers at Health Catalyst.
We have four principles of being smart, hardworking, humble, and continuous learners that uh I'm just a humble representative of all the great work that we've put together over the last year at Health Catalyst. Our mission is to be the catalyst for measure massive measurable informed healthcare improvement. Health catalyst empowers organizations to be proactive and future proof their operations using our proven healthcare specific approach to data and analytics.
Most recently we've announced Um, apologies. At Health Catalyst, we have uh over 600 clients that we've uh been able to serve that service more than 250 hospitals uh throughout the nation, servicing 100 million patients each year. Uh we've had 365 uh documented case studies of datadriven improvement. Of that, we've resulted in 2. 2 2 billion in realize uh savings that directly improve by directly improving patient care as well as patient outcome uh and patient experience incentives. uh health catalyst.
We recently announced uh our healthcare ignite uh spark platform. as long as I stay behind. Um, which is a modern data analytic platform that uh allows us to uh announce uh analytics for community, regional and midsize hospital operations through an uh flexible application layer that data bicks is a component that plugs into this. Uh data bricks allows us to uh get the right data at the right time into the right users's hands.
Let me do this. All right. So the impact of the Spark Ignite platform by Health Catalyst just shows what's possible when you build on data bricks. And so Health Catalyst is a perfect example of our built-on program. And this is where we collaborate and partner with software vendors to build and take them take those applications to market and those applications are built on top of data bricks. So we partner with health catalyst to devise product strategies, co-selling motions and joint marketing ventures.
And I think the results are pretty easy to see with the growth at Health Catalyst and the impact that they provide for their customers. So our goal is to provide our partners and their product teams with the ability to focus on delivering healthcare specific analytics, operational insights and not managing infrastructure. So it's really exciting to see health catalyst build customer focused revenue generating SAS products, not prototypes, not internal tools, something that provides value in the market.
And this abstraction layer that they're building on top of data bricks is what sets them apart in a very competitive market. And that's what the build-on program is all about, right? We want to have our customers and our partners have the ability to grow smarter, accelerate their innovation, and stand out in competitive marketplaces. I had to stand on one leg. So what does this built-on strategy look like at health catalyst?
So the speed of deployment is certainly critical to their business and at the heart of this is data bricks asset bundles. So each health catalyst customer has multiple workspaces. So typically when I work with customers I see isolation of environments by dev stage and prod. And health catalyst has taken a really fresh approach to this and they isolate by data product. So this allows them to standardize workflows. They can govern at a high level for all their workspaces and they can onboard not only new customers but new data products to existing customers.
So data bicks is also a purpose-built platform for product development. It's not just for data engineers. It's not just for data teams or MLOps folks. We see engineers collaborating with data scientists, product managers to build customer-f facing apps and they start by ingesting some general data that's meant to support many use cases. Then they enrich it with customer specific data sets and that's what allows them to create these very very tailored data products and data bricks asset bundles allows them to package code resources and environment configs together for each individual data product.
And finally, they're able to iterate on established design patterns so that when a bespoke customer product provides more value, they can seamlessly integrate it to their other product offerings. So building this abstraction layer on top of data bricks allows them to scale innovation, accelerate deployment, and they can start to deliver on these unique customer needs without having to reinvent the platform layer. That's a slide for me.
All right, so show hands. Who here has not worked with data bricks asset bundles? Have not. Okay, I was afraid that everyone had and then this part would be pointless. So, we're going to do a quick overview and I think it provides some value so we're all on the same page. Okay. So, why is dabs important? What's the motivation behind dabs? Why should you care? So, everyone in this room wants to derive some value from their data.
But the tricky part is your data products have to be in production before you can start seeing that value, right? So you have a business problem, you got some data, you're going to throw that data at your business problem, try to solve it, and that's going to be our use case. And every use case is made up of multiple resources. So maybe you have an ETL pipeline, some SQL files, uh, and a dashboard, or maybe you're trying to train and deploy models.
But those resources need to work in conjunction with deployment configurations. So we need to make sure that we're deploying in the right environment. We need to give the right people access to run these things. We need to expose the right data. So resources de deployment configurations working together. But anyone who just loosely follows the software development life cycle has a couple things in common and that's code.
Right? So we're we've got code, we're going to review it, we're going to test it. We have unit tests. We have integration tests. When those tests pass, we're going to merge that to main and we're going to deploy it in the right environment. So, we talked about a little that the typical path is devstage prod, but with health catalyst, we're doing it in a cool way with data product, but the point is is that we need to a target and we need to deploy somewhere.
And that's where this environment separation comes in into play. And then finally, automated deployments. It's really important that you can attach something to a git action. So when you merge, you're deploying in the right place at the right time and there's not a human in the loop to add complexity and possibly some errors. Okay, let's look at the anatomy of a project. So we've got some code. So we've got notebooks, maybe we have a lot of custom dependencies packaged together as a wheel.
We got some SQL files. We have some resources. So we need to deploy this not deploy this code but to orchestrate this code. Maybe we have workflows maybe a robust ETL tool like Delta live tables. But the point is we have some parameterization. We have some orchestration and code and resources need to work together. And then finally we have these environment configs and not just workspaces not just targets but maybe we need to think about compute size or the identity that we want to run in different environments.
Maybe I want to use use my personal user or in production I'll have a service principle. But the point here is that these three components need to work in lock step together as you're deploying to make something work at scale and work fast. Okay. So how do bundles help with all this? Again, we want to package code, resources, and configs all together, but we also want to decouple those assets from the workspace. So or the target.
So we want to write code once deploy in many places. We also need the ability to track the state of our resources. So under the covers dabs data bricks asset bundles is using Terraform state. So we're keeping state. So we not we're not deploying assets that are already deployed. We can do this in an incremental fashion but we can also uh monitor drift. And then finally we want some kind of simple and easy interface to interact with this stuff with.
So if you see down below, we've just got a CLI call and we have an environment parameter that we'll pass to it. Okay, let's think about the journey to production. What's this look like? So I'm a developer. I'm in dev. I'm playing around with some ideas. I have a single node cluster. I'm not setting any schedules. No real orchestration. And I'm going to run these things as my own personal user. And then maybe I'm on to something and I have something of value and I want to test it at scale.
And so I'm going to move it over to staging. And so here since the data volumes are larger, I'm doing more integration tests. I'm going to have a three node cluster. I'm going to put it on a daily schedule. Let it run for a while. Make sure I don't break anything. But I'm going to continue to run this as my user. But then in prod, so we have spiky workloads in prod. So we need some autoscaling. We're going to put it on an hourly schedule so we can keep up with our SLAs's. and we're going to run it as a service principle because it's best practice.
So what does this look like as I'm moving between environments? Again, here's our CLI call, right? So data bricks uh bundle deploy and staging. And then I can immediately run the job I just deployed with another call uh from the CLI. And then in production, you might have guessed it, same thing, right? Except we're just passing a different parameter. Okay, so we talked about a clean, easy to use interface, the CLI. So what's behind this? and at the heart, you know, data bricks asset bundles really is a YAML driven workflow, right?
So, let's see what a DML kind of looks like. So, we've got the name of our bundle, Shark Sightings. We've got a default workspace. So, by default, I'm going to put it in staging. Um, that means if I if that my CLI command doesn't have a parameter, it's just going to go in staging. And so, now the next part of our YAML is going to be our resources. So, here I'm going to define a job. And I'm going to give this job a name, Shark Sightings.
But I'm going to uh parameterize it with the environment. Then my job has one task where I'm going to execute a notebook. And that notebook takes one parameter. Again, I'm going to parameterize that by the environment. I'm just simply defining uh the database where I'm going to write my results. And then I'm going to run a notebook. And then I've defined some cluster configs, small cluster i3 instance. And then finally, I've got some environment-based uh specs.
And so here's where I'm going to overwrite my defaults. So in development, I'm just going to use my default workspace. But then when I move production, I have a new workspace. And I'm also going to give this job a schedule. Again, I'm going to leave all the other defaults just the same at the resource configuration level. And again, I'm parameterizing by the environment. So this is kind of what a YAML looks like that would be driving these DABs workflows.
Okay, now the fun stuff. So now we kind of understand the CLI part. We understand the DABs part. Now we can start doing advanced stuff. And this is where it gets really super fun. So we can dynamically generate these configurations. So we have our base YAML that defines resources, environment, configs, and the name and the default workspace. We can apply a second layer of YAML that can have more fine grained parameterization for each resource or for each environment.
And since this is beginning to look a lot like a metadata driven workflow, we can start scraping metadata from other places. So maybe we have a repo that is listing all the jobs. Maybe we have a table in data bricks that is mapping data sets to pipelines. And we can start writing scripts that output YAML based on a configuration for a resource or a configuration for an environment. So this allows us to start moving really really fast.
Okay, we can also start integrating this with our version control for better governance. So from a data bricks perspective, we can use a service principle to deploy and then a separate service principle to actually run the job. But when we start integrating this with our CIC CICD tools, we can automate this deployment with like git actions or something. And then we can also maybe push that governance up to that git level so that only certain identities can deploy in certain environments.
So we can we can do some more advanced governance. And then finally, and I think this is going to be really interesting to see how Health Catalyst implemented it, and I think it's kind of a secret to some of their success, is they're able to define templates for really common workloads. So you you create a template for a a job that will run on every customer's deployment, and you can even include very specific requirements in those templates.
So now I'm going to turn it over to and we're going to get into the really interesting stuff which is how he works at Help Catalyst. Thank you. So as said, we have three purpose-built workspaces. Instead of the dev stage prod, we have an ingestion workspace, a data products workspace, and a customer workspace. What this facilitates is data governance and redefining data grains. We're able to dynamically work within our unity catalog with our catalog suites of a dev test and prod suite of catalog.
They each have an associated service principle. So for example in the first that top quadrant of ingest we have an ingest dev service principle test service principle and a production service principle. What this facilitates is a lot of flexibility for our end number of sources at a single client. We can then remap or uh come up with a bunch of different flexibil use cases for what we expose in our dev environment and test environment.
For our data product lines, we're able to develop those internally following a very standardized process that we're also leveraging for our customer workspace. We're able to follow a CI/CD pipeline of data martrde dev that builds into datamart test in an internal environment. What we can then use is a parameterized pipeline that deploys that to our end number of clients um which allows us to have the right access at the right time and allows the service principle to own this catalog and schema uh which helps uh specifically or particularly at the uh test in production layers.
We'll get back or I'll circle back to a lesson learned on uh two different approaches for how you can manage your development environment. But what's been amazing is that a single development standardization and template for our data products maps really nicely to our customer workspace experience. So that our internal analysts and our client uh team members um have a standardized repeatable process that they can also leverage.
As we were saying earlier, we have over 600 clients currently. We have of those we're in the process of migrating many of them and we have to scale our operations not just internally but repeatably at scale across our clients as we migrate them from our legacy system. So scaling our deployments uh we have our customer onboarding process which requires reliability across all of our client environments. We can't scale to the level that we need if we have inconsistencies across clients for a product.
It scales for our needs and has facilitated our customer migrations. Um we're able to ingest new data sources uh at scale. um have data quality embedded um for our speed of healthcare. Um we're able to drastically improve our new product introduction as well as our use cases through rapid rapid prototyping and uh go to market strategies for uh upcoming innovations. So taking two steps back is when I first joined the organization about nine years ago um there were a lot of pain points.
The first pain point in our legacy system um was Unttracks changes. We were dealing with files that were stored on fileshares. Things were getting emailed across uh the network, shared on access drives, and this really became a problem during deployments, upgrades, and hot fixes for anyone that had access uh to the environment. Uh we were deploying an a asset that would end up with code overwrites. Whoever wrote the last person that saved in our application was the source of truth.
Uh if someone else came in 30 seconds before me, made a massive update, I made the last change, I was I was the winner. Uh it led to a lot of headache and a quite a bit of frustration. There was no source of truth, no roll back and led to a lot of frustration. Um made deployments very risky. Everything was a very manual process. We had manual intervention every step of the way. We had a process that was about 24 checkpoints of do A B C um all the way down to ensure that something was done successfully.
One misstep will one misstep would cause uh the entire process to throw an error and have to uh figure out what went wrong. Um, typically our business process was deploy our artifacts during business hours and schedule them to run overnight. We'd come next in the next morning to discover things ran successfully or uh unfortunately uh things didn't run correctly. Uh, and this was unfortunate because everyone involved in the process ranging from analysts to our environment admins um were at risk of introducing human error.
So what we've been able to since do is or again sorry um without having a single source of truth one-off updates and fixes were um very common uh to be introduced again through our updates hot fixes uh everyone ranging from analysts engineers and our system admins were dealing with inconsistencies across environments trying to troubleshoot something that showed up at client A wasn't prevalent at pro uh at client B because that fix had been uh inter uh either performed slightly differently um based off of an unknown additional request that came in boing all the pain points of the legacy system down of how DABs have been become vital to and an incredible asset uh pun intended for our uh process is that The absence of scalable deployment framework of for data brick hampers the velocity and increases the risk of inconsistent outcomes across environments without dabs and without assets we wouldn't be able to scale at the velocity that we need to be successful uh in our in our pursuits.
So with consistency with DABS, we're able to leverage source control, shift code and configuration and automate deployments. Each of them come with a vast variety of uh improvements that we've been able to see um in the organization ranging from improved code management improved audit trails um and have really been able to blend the spectrums of data engineering and software engineering where software engineers are becoming uh much stronger data engineers and data engineers are gaining and leveraging skill sets uh that are typically found in software engineering.
So the value of our dabs in our assets um are truly the assets. It's much more than just a set of tables. With uh with our assets, they can range from a single SQL file to something that's much more complex of a series of notebooks, um SQL files, uh AIBI dashboards, and everything in between. We leverage these assets with configuration as code and platform controls that we control through our CI/CD pipeline um in our Azure PIP deployment pipelines where we're able to leverage assets where we can set those flags that you could also introduce in a CLI command that showed earlier where we're able to perform a task, deploy it to dev, have it run as the service principle uh that's been configured, generate the workflow that drives a consistent metric store.
What this facilitates is an organized, flexible, and repeatable uh set of assets that we can leverage both internally as well as deliver additional value to our clients. So, what's next for Health Catalyst? Well, first thing is applying so much of what I've already learned this morning. Um, we want to improve our development workflow. Uh, recently, uh, we've had a bit of a dance between working in a VS Code or a PyCharm or your IDE of choice, running the CLI command in development to deploy that DAB, going out to the data bricks UI for the analyst to go run their DAB, uh, where they can, uh, run and deploy, verify success or verify failure.
Uh some of these processes can be as large as 25 tasks in a single workflow. Actually, that's probably an understatement. Um but for the for this story, we have 25 four tasks. As we run through each of those tasks, if a task dies at 21, uh, task 21 linearly or if multi- uh, faceted, we typically would have to go in, fix the problem at task 21, redeploy the dab, rerun the dab, and it has to start at task one and work its all the way through the system.
Um what we're excited about and what I'm jumping ahead to a future slide is uh recent announcement uh from data bricks that allows us to uh work more uh intuitively with within jobs. Um and we're looking to improve our developer experience um improve our workflow process and get 1% better each day. Um, a part of our success and what I view as success for our team is I'm incredibly grateful that we've been an early adopter of data bricks.
Um, by doing so, we've learned uh when we first started this journey, the only available options was using the workflow guey or YAML files. Um by having that limitation uh the guey file or the guey didn't wasn't scalable for our solution uh because of uh there was no export functionality. Data bricks recently um enhanced the CLI to be able to do an export statement. So you can take an existing workflow job, use the CLI to run an export, and you can generate an asset bundle um from something that you've either deployed through a jobs API or manually through the guey interface. um through learning and being an early adopter of learning that CLI uh we've been able to leverage uh a lot of great and cutting edge um technology as well as created some band-aid solutions that data bricks has since come through and introduced full-fledged features that addresses and reduces some of our tech debt.
Um, as mentioned, custom templates has been a gamecher. As I alluded to, we have a process for our migrations that map to our data pro existing data products and all of our uh client custom configured and bespoked uh data assets that they're generating. Um, we have clients that have thousands of data marts that they've generated over the course of our 14-year history. Um, migrating those clients was originally very daunting.
Frankly, it still is. Uh, but through DABS and this enablement, uh, I'm I sleep a lot better at night knowing that we have a process that's keeping up with the scale that me and our team uh, have been asked to execute against. We've learned that parameters are our friends. We're able to leverage CI/CD much better than any process that I've been able to work with uh, in the past. um we're able to clearly define uh and introduce additional parameters as new features are announced or new functionality that we're providing our clients and then uh combining all that with pipelines.
So I have blown through my slides and I feel like one of them has disappeared. Um and that's okay. uh parting advice that I I really want to emphasize. Um if I was starting over and we were starting this project today, the things I wish I would have known now that I would have done slightly differently was we would start with managed cataloges. Um there's a big pain point that if you don't start with a managed catalog and you attempt to create a managed catalog down the road, it requires you to rebuild the entire catalog. um it's much easier to start with a managed catalog and just don't use a managed uh path um versus the other the other option.
So that's recommendation one. If you're starting from ground zero, start there at minimum. Two, define a uh schema owner strategy. The two strategies that I recommend are a every developer has their own prefixed uh or postfixed name uh that gets generated with their asset. Uh so I can create a schema and can create the same and contribute to the same schema but it writes to two distinct paths so we're not inadvertently overwriting each other.
The second option is invest in a development pipeline process so that every developer on commit to a feature branch can commit to that development branch so that you have a clear pathway for landing your development workflows into a development schema before you go promoting to uh a test or a staging and then from staging into a production environment fell fast. Uh it yeah fell fast. It's a team motto that we have. Um we want to iterate as fastly as we can.
We want to find out what's going to work, what's not going to work. Some of these things have led to internal innovation. Uh including a lesson learned that uh in the early days of dabs, you could deploy a dab and it would generate a workflow. But what was that workflow ID? We had no idea. Um, fortunately we had an amazing developer uh that decided to just continue investigating what they we were doing internally on our local machine before we tried promoting this to prod.
We identified that using after deploying an asset if you ran validate you would get a response back that would tell us that uh in the metadata uh on that response object what the job ID was um with that associated generated asset. Um so while unconventional uh we were able to create a workflow that established and gave us the information that we need needed so that we could then use that job ID in other API calls um disagree and commit um it's okay like there is a dozen ways to approach any of this I am really pleased with the way that we've kind of rethought and reimagined what that workspace strategy looks like it's facilitated a lot of flexibility ility from a uh governance as well as provisioning um standpoint uh as well as deployment of how we move uh across our catalog suite versus moving an asset from um and running a job in a dev workspace and then running it again in a staging workspace and then once again prod with the right provisions the right people are going to see within a product workspace for example the dev runs the test runs and the prod runs.
Those that only need access to the dev runs will only see the dev runs and vice versa. Someone that doesn't need access to see every dev run that's running but is the system administrator for a given client can see the production runs. Um long way of saying we could disagree on the approach internally commit on a solution fail fast and keep moving forward. Don't let perfect become the enemy of good. Um, there's so much innovation within data bricks that it's difficult for me to keep up with all the innovation that they're coming out with, let alone the innovation that my teams are contributing to.
Um, there's always going to be something new. There's always going to be something shiny. Let's focus on what's good, what gets the job done, what drives value to the business and to our customers and to our clients. um without getting caught up in what's perfect, what's the ideal solution. Lastly, please get involved in the data bricks community and um and with your solution architects. I can't tell you how much value I've gained from everyone.
Uh thank you for being here. Um get involved in in the communities. I'm grateful for um the uh I'm from Utah. Uh I've been involved with the Python Utah meetup, big data meetup. uh local data bricks meetup and all of them have provided so much value. Um so thank you for contributing to those type of um environments and communities. Uh it really goes a long way to help fester and facilitate um new ideas and new new innovations.
Um, I don't have the session's name, but if you found this session helpful, if you search DABS within the uh app, uh there was a session at 8 a. m. this morning um that really dived into uh two areas that uh at Health Catalyst were really or we're really excited for and I'm I'm really excited for. One is they've announced Pythonic DABs uh which allows us to be that solves one of our major pain points of not being able to update uh dabs dynamically.
Um and I think and my assumption is is that through a Pythonic DAB we'll be able to interact with that object um and modify those uh responses uh or not necessarily responses but those artifacts um in a more dynamic nature. two, they've also introduced and there was a great uh quick demo on the uh dabs in the uh workspace UI where you're now able to interact with dabs natively within the workspace. I think that's there's incredible value uh especially on our uh analytics teams uh that aren't as comfortable uh while they've come a long ways in bridging that spectrum of data engineering, they're still not as comfortable as maybe many of you.
And even if you find yourself in the camp of not feeling super comfortable with that spectrum, um the work workspace UI is going to I think facilitate a a single um source of truth um while working within a DAB in a centralized workspace environment. Um that workspace UI I think has a lot of promise and is going to uh continue to add value to our developer experience. And with that, thank you so much for joining. Um, we are about six minutes ahead of schedule.
I am uh happy to answer questions. Um, and do look forward to uh meeting you. Um, feel free to reach out to me on LinkedIn or X. I am trying to get more involved in the community. um and be uh more proactive in my involvement uh because again I really do appreciate everyone else's involvement and trying to lead by example. Um thank you again for being here and uh I'll let those exit that want to exit and uh questions for those that want to ask questions.
Okay. Thank you.
Learn more about the Databricks Data and AI platform.
The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.