Harnessing Databricks Asset Bundles: Transforming Pipeline Management at Scale at Stack Overflow
Summary
- Databricks Asset Bundles for pipeline management: This video explores how Stack Overflow optimized data engineering workflows using DABs for scalable and efficient multi-environment pipeline deployments on the Databricks Data and AI platform.
- Modular design principles: Learn DRY-principled modular architecture, code reusability and bundle variables that ensure clarity, maintainability and data isolation across projects on the Databricks Data and AI platform.
- Enterprise infrastructure deployment: Discover how to leverage enterprise infrastructure and Unity Catalog security strategies to streamline deployment across development, staging and production environments on the Databricks Data and AI platform.
Harnessing Databricks Asset Bundles: Transforming Pipeline Management at Scale at Stack Overflow

Talk by Chelsea Zhang, Staff Data Engineer, Stack Overflow
Discover how Stack Overflow optimized its data engineering workflows using Databricks Asset Bundles (DABs) for scalable and efficient pipeline deployments. this video explores the structured pipeline architecture, emphasizing code reusability, modular design and bundle variables to ensure clarity and data isolation across projects. Learn how the data team leverages enterprise infrastructure to streamline deployment across multiple environments. Key topics include DRY-principled modular design, essential DAB features for automation and data security strategies using Unity Catalog. Designed for data engineers and teams managing multi-project workflows, this talk offers actionable insights on optimizing pipelines with Databricks evolving toolset.
Here’s more to explore:
Production ready data pipelines for analytics and AI: https://www.databricks.com/solutions/data-engineering
The Big Book of Data Engineering: https://www.databricks.com/resources/ebook/big-book-data-engineering-2nd-edition
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements
Chapters
0:00Introduction and challenges4:08Design goals and strategy12:28Five-layer architecture22:39Notebook and module structure26:07Evolution of the platform30:54Data isolation and deployment35:02Takeaways and future vision
FAQs
What are Databricks Asset Bundles and why use them?
DABs provide a structured framework for organizing and deploying data engineering workflows at scale on the Databricks Data and AI platform. Stack Overflow used DABs to achieve code reusability, maintainability and efficient multi-project deployment.
How do I design modular pipelines with DABs?
This video demonstrates DRY-principled modular design patterns using DABs on the Databricks Data and AI platform. Learn how to structure code for reusability across projects while maintaining clarity and reducing duplication.
How do DABs help with multi-environment deployment?
Learn how bundle variables enable you to deploy the same pipeline configuration across development, staging and production environments on the Databricks Data and AI platform while maintaining data isolation and environment-specific settings.
How does Unity Catalog integrate with DABs?
This video shows how to leverage Unity Catalog security policies within DAB deployments on the Databricks Data and AI platform to ensure consistent governance, data security and compliance across all pipeline deployments.
Full transcript
0:00 Introduction and challenges
Hello everyone. Um, it's a long day, but I want to share something really exciting with you guys. That's a lot of experience on the data platform team uh from Stack Overflow and I know that's a brand everybody really have your heart in. So, thank you for the support and thank you for showing up here. Uh next slide is just you know looking for statement make sure you know everybody understand this is for information only and of course um do the survey and that will be very helpful for us uh to understand how to improve the quality of our talk.
All right let's get started. So is there any Lego fan in the crowd? Yes. Yes. Me included. Okay. But I'm only like so addicted to Legos like for six months because I got my first set like for my birthday in January and I'm like, "Okay, yeah, it's it's a decor and I might enjoy it or not. They're collecting dust." So I got it. I was like, "Oh my god, I you know, I want a second set." So that was like two weeks in. All right, I just power through it.
And then right now like I'm on my eighth set and my coworker knows I have like full-blown collection behind me just in the span of six months. So I got really addicted to the process of like dumping out bricks and like yeah okay this is kind of overwhelming but you know what I'm just like going through the flow and enjoy you know see things getting built and then like but once things get built guess what I do next I buy another set and I keep just dumping out bricks and keep building and I start thinking about this why why am I doing this?
Why am I spending so much money on those toys? I think this is actually something really close to what engineer do like on a daily basis. It's like we kind of just like run towards kiosk and we create kiosk and we love it. We just like okay the process of actually recognize pattern and you know polish your pattern and like get better and do it again. That's the thing that's why that's the thing make us engineers. So I mean that's the same case for our data platform team.
We have four data engineers you know that's the kind of the experience we ran through the last year and a half. So I think this is like I mean you know we we turn chaotic bricks challenge into pattern and you know we build things smarter and now we get to share about this with you guys with a friend. So I want to make sure that you know it's a long day you know I want to make sure you actually get something from this talk.
If you never heard about DAB data bricks asset bundle that's okay. Um you know that's just something like help us to deploy data bricks pipeline uh in a more streamlined way. I think it's gonna be helpful for you to get some use case out of this presentation and you know now you understand okay maybe that is the tool for me and I'll just you know go read up the data bricks documentation go from there they have really good documentation and also if you already have some experience with data bricks asset bundle you will realize there are some difference um the way we use it um versus you know the classic documentation provided and that's where you're going to focus on the tweaks, right?
Um and then if you're in a third category, you're in a small team just like us, we have four DE and you feel like, oh my god, I got more pipeline to build than the number of engineers I got on my hand focus on where we do the scaling and the streamlining. So that's where we you know get the all the learning fruits you know from this one and a half year. So let's start with looking you know just turn the time back for a year and a half the challenge that we have um and also how we actually thinking about like what are the requirements to overcome these challenge you know thinking about like what are the goals for us to you know have it have the data platform built so that will make things a little bit easier for our team and then eventually how we come up with you know a list the strategy attack strategy and principles to follow for the next year and a half of building process.
4:08 Design goals and strategy
So first of all you guys probably have a good idea of what stack overflow is hopefully um and uh what stack exchange is. So that is the for the serving of public community right you have stack overflow for programming and then you know stack is chain there are all kinds of topics over hundreds of topics right so that's just the public part and also we offer stack overflow team so that is the enterprise solution basically you have your own private stack overflow and you have this knowledge you know uh repository to share you know inside of your company's network and we have our team to support that.
So that's you know classic dual missions. So as data platform team we need to support how can we polish these uh product to make things a little bit more u you know favor um a little bit better for everybody to use and also put that aside. We have growing demand from multiple departments because they have their own favorite tool to use multiple tools that is just pumping out data on daily basis. So there are like third party you know data re data sets that we need to also curate into our data pipeline and then have that available in the warehouse so that once everything is in one single spot it could be joined and you know analytics people can start making datadriven insights stuff like that.
So as you can see the pain points from this nature of you know environment that we're in is one is we need to make sure of data isolation right so because we have multiple products multiple pro product line we we make sure we we don't mix apples and orange together that's very important so that means you know you're going to have to have tons of pro repository uh based on the classic documentation like if you're deploying one pipeline, you're going to have one asset bundle, another one, another one.
That's not going to happen. We only have four people. So, and then you have to make sure you scale, right? We we need to deal with like a large data volume. We're talking about Stack Overflow, guys. And then about a year and a I think a year and a half ago, we were using DBX. So it was not very uh easy to learn. Um the learning curve is kind of steep uh for new data engineers and we have to do a lot of documentation.
So and luckily um data bricks come up with you know dab and it gets things a lot easier for us. So that's where we start our journey. You know we were able to lower this learning curve for new data engineers. So we have a list of goal to achieve in this new build repo. We want to make sure we have code reusability tuned to the maximum because we have no not enough people. So we're going to have to speed up the pipeline creation process with maximum maximize shared components, right?
We never build the same thing twice and now you're going to have to deal with, you know, inconsistency. That's tough. And then we have to want to want to have clear structure meaning you don't actually have to read through like a whole big documentation about how when you know when you're on boarding a data team you look at a repo you kind of get it look at a few you know feature branch commits you get it how to add a new pipeline and the repo is just very structurally clear and you understand okay maybe a little bit after after a few pipeline in you're like oh I start connecting the dots.
That's how things connected together. That's that's how Dev learned. And then we want to make it flexible so that we're not like locked down to a certain pattern. And this repo has to be work for multiple products, right? Because multiple products meaning there's different business requirements. So you got to have to give give it space to breathe, right? In your repository. So you got to make sure you have the flexibility to break apart from the previous pattern, but it can coexist in the same spot.
And then you that's going to be everybody what everybody's favorite is minimize context switching. I hate context switching. If you telling me that I'm going to work well, you do the math, right? We're 4DE and they have hundreds of pipeline. If I have to switch from one repo to another to another on on one day of job, I'm going nuts. Like I'm crazy. I cannot do my work efficiently. Right? That's why we want one repo for all the pipelines.
The pipeline that goes to mult deployed to multip products for multiple clients even towards multi-tenants and it can be done. So here are the three bullet points of the structural principle that we end up want to focus on. One is we're going to use dab as a foundation because it provide a lot of flexibility for the team to you know prosper build up our own pattern and second one is we made clear pipeline boundary. So three things client meaning the data owner because you don't want to mix client A's data with client B's data right they need to be processed in separate pipeline the second one is the product meaning what different the business process you're going to have different data model it's going to be different so apparently you're going to have to two different pipeline for two different products and of course environment so you understand that and then for we want we want to make it dynamic.
That's what DAB offer us is allow us to really let the configurable value to drip through the multiple layers which I'm going to talk about that in detail later so that it actually make this multi-tenant flexible multi-target multi-pipeline deployments from one single pipeline a real thing. So to sum up uh you know for the challenge to our design principle before is for us was it was just hundreds of pipeline to build very overwhelming we don't have enough people stretch team and uh there is a longer onboarding time and of course you know for us to build anything new it just feels like it's not that fast from you know you're typing out code and you see the it's working and you're just iteratively building this cycle is not fast enough.
And then after that we have one repo deploys all the pipeline and we have reusable module and it's your choice you want to use it use it. If you want to add new module great because people going to use it later and then we have we achieve a balance between you reusability and the flexibility. If you remember about the flexibility that was talking about earlier, if you wanted to de like decouple from existing pattern because you're adopting a new product for your pipeline repo, you can do that.
It's okay, right? So that and also we can effortlessly scale for multi-tenant. So that's the part is going to uh work with our infrastructure team uh enterprise level configuration. They use octopus cloud. So in this next session I'm going to talk about this five layer of the pipeline architecture. So you're going to see some code blocks. That's okay. I know it's it's a late for the day. Take a picture of it and you know I will make sure the structure is pretty much the same you know and then I will show you where you're going to focus on.
12:28 Five-layer architecture
Okay how that's going to be different from the classic example from data bricks tab. So let's go on the first layer. So this top layer is our CI/CD workflow. We use GitHub uh the top layer GitHub workflow and then we have this twole workflow. So it's one like a GitHub workflow call another okay simple. So you would see this first part is blinking on the top right. We call that release bundle pipeline. That's going to be the workflow that actually has all the steps that you need to deploy a pipeline using that which is the shared portion of you know the git github uh uh git uh github workflow actions and then we have the second box.
So these are going to be the GitHub workflow actually has all the parameter values to pass onto this common one to the first one and it's going to be determined by which product which environment which client remember that boundary I was talking about earlier. So that's where the first this you got your first taste of reusability right there. So in the shared steps which is the first blue box over there we I mean it's pretty pretty straightforward right first your Azure logged in and make sure you use a service principle do not use your developers account to do that because once that person left you're going to have overhead because somebody deployed the pipeline they're going to own it if can you imagine when that person left and their account get deactivated that's going to be a lot of work right we don't want to do that use service principle save you some time and then the bundle validation which is the uh data bricks bundle validate so it will tell you everything checked out you know the pipeline definition looks good and sometimes you know if you find oh I don't know like based on the if something goes wrong during the validation I don't understand what's going wrong turn on the debug flag it will give you more verbose um log uh log content so it will help you that actually helped me in the And then you either do deploy or destroy.
So deploy everybody knows that destroy is basically you know scratched all the stuff that deployed before. show cleaning slated and also there's another thing um set the datab bricks underscorehost set that URL in your uh product in your like the GitHub workflow uh that has all the parameters that will allow you to deploy your pipeline into multiple data bricks workspace that's very helpful because I remember when I first started with DAB in example they actually have the workflow URL inside of the next layer which is the datab brick configuration so that means all your pipeline is going to deploy to the same workspace you can can do that right so that's the trick to do that all right let's move on to the next layer that's the bundle definition layer and where it blink is where your it's called datab bricksyl and it's always going to look like that and uh in the bundle definition.
The first section is you're going to have a b bunch of bundle variables. So those are going to be uh the bundle variables that configure for this bundle specifically and it will expect these value to be received from your first layer which is your GitHub workflow and there's of course there is a few categories of those right the boundary what client what what products it which environments goes into and then the connectivity so where is your up upstream right if it's if the data coming from Azure event hub what's the connectivity do not save any secrets in there.
Let's use uh data vault backed uh secret scope. Okay, in data let's not put any secrets in your uh GitHub workflow. That's bad. Um and you know warehouse stuff like that and also job related stuff like you know your Chrome schedule for job run um web hooks um for notification from your workspace and also access. So those are going to include a security group and also the permission you want to grant them um what kind of you know what kind of like a manage or read or stuff like that and then you're going to have this section it's called deployment target.
So for us we use three targets dev prod that's easy to understand and then user. So user is where we actually plugged in the GitHub actor. So when we actually triggered a GitHub workflow, we can capture a GitHub actor which is developers handle on a GitHub account. So that it allow us to have a separate deployed files for these developer. That means they're send it's like a sandbox environment. They're they have their own pipeline.
So even you know when testing in a dev workspace their pipeline is not going to mess with other people's pipeline that deploy from the same thing. So that's that's actually pretty nice. Um and of course you know we you you see we actually baked in the boundary in our file uh the file path which is environment bundle name client and product. So that is enforced uh file structure for the deployed pipeline um files on the work workspace so that it's very easy and it's very consistent.
Remember that's the structure the clear structure very predictable structure so that our developer always know how to go through those directory to find their stuff. Where is their stuff? Right, that's where. And also for the layer two, make sure you include the sync pass. SRC in it. So what it does is if you take a close look at the uh screenshot of um our repo structure, you will find out we actually have multiple bundles under the bundles folder.
So that's the that's how we able to basically stuff multiple bundle into one repo is we have the src folder is go beyond the b each individual bundle folder. So what do src do is when it's get deployed it tells dab where to find all the source file. I'm going to talk about source file a little bit later but that's how that's how we do it. that's the trick and I we actually get this inspiration for uh of this from work.
So I'm going to include a slide at the very end of this. Um he had a great session year ceremony. I went that it's it's amazing. Uh and also this is what's going to look like in your workspace after you hit deploy. So you notice the src folder is underneath your bundles uh file directory. That's the result of sync. SRC do force if you choose to put SRC on the same level of your individual B bundle. Okay, we're moving on to layer three resource definition.
So what is resource? Resource is job definition, pipeline, pipeline meaning the LT pipeline definition. So those are the orchestrator of your lower logic, right? Like notebooks and Python and stuff like that. So that's where it goes. They goes under the each individual bundle folder and under the resource folder. You see here I have job. ymo and the DT_1. You can keep everything in one, but for me I I just like to have them separate so it's kind of easy.
Otherwise you end up having a really long. I hate that. Um so in a typical resource definition YAML file you will have your you know workflow pipeline uh you know make sure you use the variables right so for example the name for your workflow that's how our job like all the job that deploy from this repo going to look like it's just going to be feed with you know different client product and environment and it's all follow this naming convention.
So it's going to be very predictable to find and also make sure you use a tag. Uh tag is also going to help you to find if you have tons of data pipeline deployed to a workspace use tag wisely so that you can filter them out by your tagging and of course we use this uh variables for navigating to the note for notebook path. So that's going to be the next layer um that become dynamic. So we keep we pass the variable through spark config.
Um this is how we do it. It's just you know you have a dollar sign and then the variable name this variable name and then in that way your next layer which is a notebook it will get these variable from all the way from your what GitHub workflow right it drips through that's how every all the variable pass through these layers let's move on to layer four so layer four is your notebook definition which is going to be called or driven by your last layer which is the resource the job the LT pipeline so make sure so here's a thing like I'm trying to show you where the dots are connected so in your databicks. ymmo YL which is your layer 2 you will have this line called workspace root path that's where you define where your file is going to deployed in your workspace using variables and then in the layer number three in the resource which is your job or pipeline definition you have the spark config it pass this workspace file path through with the slash src what it does is it passed this string into the notebook.
22:39 Notebook and module structure
Remember the spark config is how we pass the value into a notebook. And now we do cy. pass uh append get the bundle. source pass. In that way it would append the src directory to the python to the for the python interpreter so that it will find your module that is defined under src. That's how things tie together. So if you look at a structure of a typical notebook, you will have your imports which is your standard library and then your custom module which is you know stuff under your src that you built and then you're going to have the variable read.
So basically reading a bundle of variable from the spark config and then your function. I know a lot of people they will write like a long really long notebook right uh and you guys probably heard about the medallion. So what we do is just to keep this structurally clear we actually have um just function call right for raw bronze silver gold and then we have all those medallion logic defined in the next layer which is in the Python modules.
So this is the last layer and then you will notice under src we have like different subfolder. So clients meaning like if when the pipeline has to deal with different type of you know external like connectivity. So we have like API data bricks that's that's one category and also we have common which is for some helper function apparently and there's a configuration. So those are like project specifically a product specifically configuration logic.
How is it going to read from this config? So that's going to be there. And then we also have our uh the medallion logic as you see at the at the very bottom of this blue box on the right. We have raw bronze silver gold. That's where the sharable reusable medallion logic goes. So this is the last layer. So this entire thing it follows do not repeat yourself principle the drive principle it makes sure we have reusable components for fast pipeline de uh development and there's inter they're all interchangeable through the notebook function calls.
So in that way we begin this mobility if you guys see this it's it's quite apparent here. So lots of code was written, right? You saw lots of code in there. But it took over a year for us to get here from the beginning of you know that becoming um and now we're sharing this with you guys. And I I no doubt I know some of you guys probably have way more advanced you know usage for DAB and I always hear about those during the office hour for DAB.
26:07 Evolution of the platform
It's amazing. And then the thing is when are you going to start all this if you're at either stage right I mentioned earlier um how you going to start this and when you going to start our S sur he I like this he always says sooner is better than never is now but you're not going to write this whole repo like in one night right was not built in the day neither a data platform So this is this is like how the last a year and a half boiled down to.
So if you don't have anything start you have a single notebook you have lots of implementation you don't even have function extracted fine right just it does the thing and they get the result for you you're happy so now you want to productionize it you're going to start getting organized stuff right start extract function put the function on push it to the top of a notebook you will start organizing it and now the next step is you're going to make some common notebook so that maybe for a different product you can use some common notebook just for configuration right now you move from one notebook to two notebooks or one to multiple because guess what this configuration is going to work for all your product you're not you don't you're not going to write it multiple times and third step is scale with your CI/CD because all the hard work you did Right?
All the notebook you don't want to just use it for one product. You want to use it for all the product. That's where you start getting gaining speed from adding more of those CI/CD workflow definition with all the parameters, all the DAB variables. And step four is a DEP for a specific need. So let's say if you have a very specific product that just completely it's not even doing medallion fine you don't have to do medallion if the product says so then just use the you can actually create the notebooks for your product specific folder for your layer 4 which is notebook layer and it's still calling that from your sharable resource which is the pipeline definition that still connects right that's why the layers It's all interchangeable.
And then you're going to keep refactoring the common notebook extract function into Python module. So now you get more mobility. You're no long your logic is no longer live in your notebook. Notebook is a higher orchestrating from you know it's beyond your pipeline. So beyond the pipeline definition. So it's just a call for example called each medallion layer. It doesn't really define a medallion layer, right? And now your Python module which is the last layer, it's all swappable and you can add more.
You can make it more tighten up. It's it's you know you can do whatever you want in that layer. Switch switch it out if it doesn't work. And then the last step is keep growing, keep iterating, grow without breaking. So the trick that I share earlier was the user sandbox pipeline, right? Have those little thing built in your uh CI/CD workflow. So to make sure your team is not going to step on each other's toe. That's important for us.
That ensure you actually go the full speed you want and you're not you know, stopping blocking each other. And every time we put in new stuff, we look around, do some minimal refactoring, right? Clean up after yourself, keep everything stable. Um, stable meaning t actually you're able to test something easily that user sandbox guarantee that. So in the next session this is a little bit breakoff from the previous the more technical breakdown is we are going to talk about how this repo is going to work with the other part of the stack overflow which is our infrastructure team.
30:54 Data isolation and deployment
So apparently they they maintain multi-tenants environment for the stack overflow for team and apparently we don't want to mix client A's data with client B's data. Everybody knows that. So data is isolation and scaling is in the requirements. Unity catalog was really really helpful for us to ensure that we establish clear data boundary which we defined at the very beginning of this project which is client product environment.
So this is how the schema naming for all our stuff in the unity catalog. So we have the client and environment on the catalog level. Then we have the product on the schema level and then we have the table name and then postfix was the medallion structure. So we we it's like a naming convention everybody has to learn once and we just understand it right and then this is going to be very secure for us because you know data bricks unity catalog take care of all of this um isolate data with access control this is scalable for multi-tenants uh which I'm going to talk about in the next slide uh and it's automated schema structure enforcement to make make sure there's consistency in it.
So I mentioned earlier that our infrastructure team use octopus cloud and the challenge was we have hundreds of clients for stack for team and then that means you got to have to hundreds have hundreds of catalog and hundreds of pipeline deployed for every product. That sounds kind of daunting, right? That's scary. We only have four people. So the solution was the client the multi-tenant uh specific variables are hosted by our infrastructure team.
So what this is this is actually their work. They proposed that for each application that want to deploy in a multi-tenant environment, we create an image when we deploy the GitHub workflow. Uh and then it would expecting to receive the tenant related the tenant specific variable from at the deployment time from Octopus cloud. So that what it does is it combine the two parts the variable that the data platform team holds and the and the variable that uh tenants related and combine it and push towards the same workflow that we have for the pipeline that own by data uh data platform.
So it's just like one extra layer beyond right. So in that way we were able to deploy hundreds of pipeline and it just completely because we cannot go in there and just kickstart everything. It deployed it runs it cleans up and creates own its own environment. It's all hands-free after the deployment button is getting hit hit. So that's how we did it for one product. So that's for stack overflow for team. We have m we have hundreds of pipeline deployed just from one single click.
So there are some deployment steps that we need to follow. The pre-install is we combine the variables do the Azure log and the bundle validation and the install is the bundle deployment and uninstall is a bundle destroy for cleanup just in case there's some client churning happening. So the benefit of this is we actually meet with our infrastructure in the middle. We leverage the existing infrastructure team deployment pattern that they chose Octopus cloud and then we still scale with them and automate deployment pro process across hundreds of tenants which is a huge success for us.
So a few key takeaway why that right one is the dynamic variable that is the same is it helps us to achieve to actually build this pattern that's that's the same make it possible for us it's portable reusable workflow across the pipeline and then we also leverage data bricks ecosystem the unity catalog data governs all the good stuff and they keep building on top of it we're just taking everything they they build We have a lot of you know the DLT serverless come in.
35:02 Takeaways and future vision
It just makes things happening even faster and also they have building documentation the bundle config schema JSON it saves us time for reference when we actually building some pipelines and also flexible targets that allows us to do fast testing with confidence so we can you know iterates through the development cycle. So the impact or the gain for the stack overflow data platform team is we scale from one to many products within about like seven months.
That's amazing. And keep in mind each product that actually goes to some of them they actually goes to hundreds of pipeline because of this multi-tenant setup. And then we have minimized uh we have we have maximized the code reuse and minimize the redundancy in the in in the in in the repo. So we we're still working on it. I mean we're still creating kio still dumping out the chaotic bricks all the time but that's okay.
So we have a lot on our plate right some of those actually already done. I made this slides like a couple months ago. So we have a lot of data integration tasks to do. Stack is change. We have hundreds of sites. So if we need to know what's going on how you know if the communities are happy or how to make things better for them we need to really tap in. We have to have this integration going on so that our analysts can actually see it almost live as soon as possible right if something happened and then the site analytics and also experiments.
So these are the stuff make sure we actually capture user behavior user preference doing like different sites experiment and then it has to get to the analyst fast. So that's something we're working on right now and also marketing campaign. Um so in that way we actually can understand how the emails reach how the messaging between the users and the platform happens during certain campaign because the dev service is coming up.
We want to know what's going on and we also wanted to expand the data platform service offering. One is data cataloging. We want to leverage the crowdsourcing. So we allow the team, we allow the stacker who actually have the knowledge of our data. They do labeling so that we don't have to get into a meeting room to ask things again and again. They just label it and then we use it. If we need to tag the legal department say hey can you answer this question for us they will see it.
So that avoids a lot of communication time waste for us. we can focus we can minimize you know the context switching and also the self-service style in creating sharable assets. So data platform team it's it's like a shared service team we have to face a lot of client a lot of customer within the sack workflow. So how to make things more consistent and instant and easy for them is something we always keep in our mind when we when we keep designing polishing our platform and also we want to make sure our pipelines are keep getting per tuned for performance and lower our cost.
So the goal for our data platform team is we want to make flexible enterprisegrade data platform and it's always that stay ahead of company's growing needs to support all our developers. So we start with kiosk. It's the raw materials the stuff the business requirements in front of us and we go through iteration and we discover pattern along the way and then we keep doing refactoring and we adopt pattern learn our lesson and then we share with you guys and accelerate a community and we grow together and you know build stuff faster and smarter.
That's it. Thank you guys for being here and uh have this fun session with us. This is the slides that I promised. Um, this is the inspiration where this all comes from again. He's he's great. So, take a picture of that. All right. Uh, the floor is open for questions.
Learn more about the Databricks Data and AI platform.
The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.