Tech Industry Session: Optimizing Costs and Controls to Democratize Data and AI

Summary

• Scale democratization safely: Enable thousands of users—from engineers to business teams—to access Databricks Data and AI platform tools for analytics, AI, and BI without sacrificing governance or cost control.
• Unified governance at scale: Implement systems tables, tagging, and governance frameworks for granular control, ensuring data quality and compliance as user adoption grows.
• Cost-efficient large workloads: Balance scalability with financial responsibility by optimizing resource usage and implementing governance controls that prevent unexpected cost overruns.

Tech Industry Session: Optimizing Costs and Controls to Democratize Data and AI

Talk by Anup Segu, Chief Architect, YipitData ; Miranda Luna, Director, Manager Product Management, Databricks ; Vivek Srivastava, SVP Product and Engineering, OT Technology, LLC
Join us for this video focused on how leading tech companies are enabling data intelligence across their organizations while maintaining cost efficiency and governance. Hear the successes and the challenges when Databricks empowers thousands of users—from engineers to business teams—by providing scalable tools for AI, BI and analytics. Topics include: Combining AI/BI and Lakehouse Apps to streamline workflows and accelerate insights Implementing systems tables, tagging and governance frameworks for granular control Democratizing data access while optimizing costs for large-scale analytical workloads Hear from customers and Databricks experts, followed by a customer panel featuring industry leaders. Gain insights into how Databricks helps tech innovators scale their platforms while maintaining operational excellence.
Intelligent analytics for everyone: https://www.databricks.com/product/business-intelligence
BI meets AI: https://www.databricks.com/resources/ebook/business-intelligence-meets-ai
Explore the only unified platform for agent systems: https://www.databricks.com/product/artificial-intelligence
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements

FAQs

How can we democratize data access without losing control?

Use governance frameworks with systems tables and tagging to implement granular access controls and compliance policies while scaling to thousands of users.

What happens when we scale Databricks to many users?

This video covers how leading tech companies successfully onboarded large user bases, adding costs and complexity—but shares strategies for managing both.

Can we optimize costs across BI and analytics workloads?

Yes, by combining AI/BI with Lakehouse Apps and implementing governance systems that align resource usage with business needs and team chargebacks.

What do other tech companies do at scale?

Hear directly from customer panelists on how YipitData, OT Technology, and others democratized data while maintaining cost efficiency and operational excellence.

Full transcript

All right, welcome, welcome, welcome everybody. Thank you for coming in. I know it's the lunch hour, so it's a busy one, but we are so excited for you to be here with us today. Um, we're going to be talking about optimizing costs, controls as we democratize data and AI, right? One of the things that's kind of the big scary or the big question mark as you're onboarding many more folks, right? Democratizing AI sounds amazing.
Um, until you get down to the nuts and bolts of like, oh shoot, I have 200,000 more people using data bricks or with access to data bricks in some way. What am I going to do now? So, we're going to talk a little bit about data bicks's evolution and then we're going to have a couple of our customers kind of tell you directly what their evolution has looked like as they've scaled and adopted and kind of grown in their data bricks usage.
It's not going to be just me. Uh, and we're going to get into some Q&A. Um, and hopefully we'll leave with a little bit better understanding of some of the kind of things to look out for, best practices, um, and things to put into practice. So, let's go ahead and get started. I am a product manager here at Data Bricks. I focus on the AIBI side. So, when we talk about kind of democratizing data, scaling user access, uh, usually some of the products I work on are behind the motivation for that. um and very keen to kind of make sure that as we're building and helping you scale that we're doing so in a you know controlled way that gives you all of the all of the kind of governance knobs that you need.
Great. So we want to democratize data and AI for all of our employees, right? It's the dream. We want everyone to be as effective as they can. We want self-s serve. We want everybody to be able to kind of get off and running without bottlenecking on any one particular analyst. But the reality of what data bicks has been in the past 10 years, if we think about its evolution, it started for like a tiny sliver of the company, right?
Data bicks, as mentioned this morning, uh started as a managed Spark company for data engineers, right? There were about five people working together in a workspace. They all sat in the same room. Somebody needed access to something, they just went like, "Oh, hey can you give me access to that?" And then, "Oh, yeah, sure. No problem. Oh, it's over here in this directory." Right? You didn't even need search. Your search was your friend. um that was a very different world right so we started with that kind of a data bricks that was built for those scenarios and that scale super easy to govern right you're governing the five people sitting around you um but then when we wanted to think about how is data bricks evolving like we started to add the notebook we started to add some ML capabilities okay now you have some different departments in data bricks right you have data scientists you have uh some ML engineers and now you're trying to figure out okay well I've now tripled you know the departments using data bricks.
They can't just talk to each other across the room anymore. Um, you know, I really need to think about what are my processes as I'm adding more users. What are the policies I want to put into place for how they're using data bricks, how they're collaborating with each other. Um, and it's a whole new set of challenges, right? It's not just a go crazy, you know, it's coming out of your budget and good luck. Um, so as data bricks has expanded kind of its capabilities and led to more of that kind of opportunity to democratize data within the organization, there have been some new challenges, right?
And those are important to think through. And as we started, you know, we started with just the data engineers, added on our machine learning engineers and our data scientists. And then we added a data warehouse, right? So all of our SQL fluent analysts came in. That was another order of magnitude growth. And with that investment was also paired Unity catalog. Um that was a big transition for a lot of our existing customers.
Um but was critical kind of foundational work as you try to kind of scale. So what is kind of the next phase of scaling we're thinking about and what do we want to make sure we're supporting you as you kind of grow through and that's AIBI. So both the dashboards and Genie side of the house kind of our business intelligence platform, right? The promise of Genie, the promise of dashboards is to kind of have a seamless single source of truth that is accessible by anyone in your organization.
But again, when we say anyone in your organization, that's a little bit of a scary thing, right? How do I actually get everybody in here securely? How do I make sure that they're not all spinning up infinity compute? How do I make sure that they're kind of adhering to the data governance policies that I want to enforce in Unity, etc. And it's a challenge, right? It's something that's really important for us to think through.
You know, growth is good, but you know, doing it in a stable and safe way is critical and that is what we're going to go ahead and talk about a little bit more today. So, with that, I'm going to invite from One Trust to come up and share a little bit about their journey. VC, thank you. Good afternoon, everyone. Uh, how many of you have heard of uh One Trust? Some of some of you some of you definitely and uh how many of you are data bricks customers today and how many of you have actually tried to democratize data and ran into a problem where because of cost concerns or governance issues not able to scale it.
So there are a few of you definitely right. Uh so. I am the SVP of engineering at one trust. Uh and I'll talk about the journey we have taken uh in the process of going from uh from a long time to data bricks and where we are and talk about uh how we have actually democratized data and open access to most of our for our users and customers as well. uh so on one trust if you don't know one trust provides a platform to responsibly collect govern and use your data and if I have to simplify that if you think about you have you go to say McDonald or you're using a looking at an NFL game altogether on any of the devices OTD devices mobile devices laptops you provide some data about you and the data is collected and you provide it for a specific purpose how do You make sure the data is collected for the right purpose, governed correctly, risks are identified and you provide access controls on the data.
That's all powered by one trust behind the scene for several large customers. And I'll show a slide on the number of customers that we have here. We have about 14,000 customers, 11 million users, 3. 7 million vendors, 5. 5 billion consent uh profiles, and three billion consent transactions per day. It's a large volume of data we're talking about. Last year, we processed about 10 trillion HTTP requests. 10 trillion HTTP requests.
And that trillion request holds contain about 200 pabytes of data transferred. Uh so it's a lot of data we talking about here. So we have solution for every stage where we actually go and collect the data. We govern the data and use the data responsibly. We have uh behind these all of these solutions that we have it that you see in the screen there are 26 different models. I would say that behind so or data models behind that we say that to suppose and uh and service these requests that we're going to talk about in second year uh we have 14,000 customers and we said that you can see the customers across every industry whether it's media, healthcare, finance, technology everywhere we have 120 environments.
There are 60 environments that are production environment, 60 non-production environments across 14 countries across the globe. We have to do it primarily because of regulatory requirements where the data transfers policies are enforced and we are not taking the data outside the country. So that's why we need to have all of these environments. So to truly scale to this uh large volume of data for three billion transactions 120 environments we need to ensure the data is manage managed effectively costs are kept at bay and we put right controls and access governance around this data but so in SAS environment when customers start to use this data so I didn't talk about so 14 environments we have lot of these customers or name brand large customers they have their own specific dedicated envir environments.
So you're talking about 120 environments, dedicated environments which is a lot of volumes going through this data altogether. But customers what they're looking for the data we collected they're also looking at this data to be married with their systems altogether to drive downstream uh downstream applications altogether. One of the example could be looking at the marketing right so we have provided this consent and they want to go and look at this data say okay I want to drive my marketing campaigns using the consent data that you have provided.
So we have to provide the data back to them or ask them to provide their data so that we can provide this analytics and dashboard to govern and provide this analytics they're looking for. uh there is a a lot of expectations from customers about I want to have a self-service BI and AI right so BI was important earlier people talked about BI have have a dashboard I should be able to build my own dashboards build my own KPIs but now people are looking for AI capabilities as well they're looking for how can I actually go and build my own chatbot and ask questions on my data set altogether so and then there's lots of regulatory requirements as well from a GDPR CCPA across the board so we have to make Sure we are also when we are cap capturing this PII data personal information we capture this data and provide governance around these regulations as well and we need to provide security across all this data.
So that's the expectation from all of this thing that we do it and the customers have this expectation. So we started our journey in 2018 with Uh the originally back in the day back in the day if you rewind seven years ago the expectation was if you can provide me some my data give me some analytics build me a dashboards or a report I think it'll be good. Uh but as the data starts to grow for us we need to start thinking about how can we provide little bit more analytics on top of it self-service BI capabilities and scale to the volume that I talked about earlier.
So we made the decision by evaluating several other vendors and decided to go with data bricks and I'm glad we made the decision to go to data bricks because what you have heard earlier today that's going to power us going to the future of what we trying to do it from a governance from a from a self service capability AIBI building agents on top of that so whole lot of activities that we can do it on on on data bricks going forward also so we built all these dashboards earlier but the customers now expectation is now I AI capabilities, NLP capabilities.
So now we are starting to explore natural language capabilities within data bricks and start to provide the capability of self-service AI, self-service BI and you can so I'll show a demo in a bit here. So from an architecture perspective, think about again going back to 120 environments and we have to deploy this architecture across those 120 environments. Our primary data store is SQL. This is running on Azure ecosystem.
Data bricks and uh our primary applications all run in Azure ecosystem. And we have large volume of customers. So a lot of these environments are actually also multi-tenant environments. So the 14 environments that I talked about earlier, they are across geos and they're multi-tenant environments with thousands of customers in one environment. So we need to make sure that we are not creating a noisy enable problem when we are trying to get data from SQL into into data bricks altogether.
So we have to start to create some dedicated lanes for that data. We need to make sure the data is aggregated uh in in bronze cleanse aggregated and we service the data through PowerBI. In SQL our architecture is actually we have one database per tenant. But when we get the data into uh into the bronze layer it's all coming into the same workspace altogether. But we need to make sure when we go back to cleanse the data and make it consumable consumable for the customer.
The data is actually going back to its own workspace altogether. So or containers. So that's what we do it. We use PowerBI embedded uh to build all these reports and analytics on that. So this architecture talking about billions of transactions deployed across all of these large environments altogether had to scale and has to be cost effective and governed properly. So we ran into several challenges during the process.
Uh as you can imagine talking about 160 environments it means deploying data bricks Kafka SQL everything has to be deployed. the infrastructure cost, the storage cost, uh the VM cost, all that is going to be start to add up slowly. We also looking at data sets where we don't have access controls and governance. We're talking about 26 different applications that they talked about. Everybody has their different access controls on that data set.
So how can we start to build this access controls on that so that everybody has consistent access the right governance in process altogether and and since we are in the privacy industry we need to definitely make sure that we also handling all the privacy readiness is there and also looking at auditability of the data is there across the entire data set how who used it when used it how was it used what was the purpose of using it and build trust across the data.
So we have to look at multiple options to go and optimize cost. So first one we started look at how do we actually go and look at when the data lands into the bronze layer. So a lot of data is coming in. We tal we saw the three billion transactions and I only talked about a small subset of the data that the three billion transactions specifically what we are collecting it but we also have data that we have captured from metadata that we have captured by scanning different environments altogether. uh and also looking at privacy data a lot of other data that comes into risk data lot of data comes into our environment.
So we need to make sure that we are looking at the data and keeping the data relevant uh for a duration that is required. So we start to look at uh different environments and say okay we need to start look at how do we keep the data and look at time travel requirements. Sometimes we have to go and replay because there are challenges where the data in SQL may not be reflecting correctly exactly every single time because something failed in between and it may not be reflecting correctly into uh in data bricks.
So we need to go and make sure the process is identified to go and do that sanity check of the data and then and and replace those events if required. But in certain environments we said do we need to have 15 days a month of uh of data for time travel or we can actually go and reduce the data by a significant volume. So we start to clean the data and by environment by environment we start to look at what exactly should be our data retention policy should be should be on those environments and specifically our internal environment starts to even reduce it to to one day or even the lowest number that we can go down to.
Also looking at uh they talked about multi-tenency. So we have in one of our larger environments we have 5,000 customers or 5,000 tenants in one environment. This is large number of tenants altogether and all of these are large ten large companies. You saw some of the media companies they have to actually look at consent data pushed into one trust constantly and this data is coming in that data need to be actually go and cleanse and we need to make sure this these large customers are not impacting these smaller customers by noise creating a noisy enabled problem.
So we start to create by the data segregation start segregating the data by what kind of data that we are trying to send it and then we go and start to to group those tenants separately so that the data is is is coming through come constantly the live data is coming the migrated data is coming in a different stream altogether and the tenants that are in a lower priority are not getting impacted. We started to look at also from a pilot light perspective.
What does that mean? So we are not on unity catalog yet. We are actually in the process of migrating to Unity catalog but uh we have to start to think about when the servers are up there uh if there's no use or less use how can we actually go and reduce uh the composition of the server the VM size altogether or even from a PowerBI side how can we scale it down automatically so that it is running at it low capacity altogether and whenever the first stream of data starts to come we start to actually go and go and ramp it up.
So we started to look at building these pilot light jobs and uh have some of these environments where they are shut down or slowed down and then when the first stream of work comes and this talking about 5,000 tenants they are across geos so never know when the volume of data is starting to surge uh we always have peaks and valleys across across the entire day and then we look at the weekend shutdown for some of those environments as well and and looked at serverless capacity capabilities also to start to say completely go and shut down some of those environments and uh they'll come back up uh as and when needed.
So with making all of these changes uh we were able to save about $150,000 per month uh on on our environments across these environments. So it's a significant uh significant cost savings uh we were able to achieve by making these changes. uh we were able to so we have scaled from uh 5,000 customers to 25,000 customers. So if you look at on the left hand bar we have about $200,000 per month expense on uh on this VM and data brick SQL uh cost altogether.
Uh but when we start to scale up to 25,000 customer environments uh we were able to actually scale to 25 five times the customer by only increasing the cost by 50%. That's a significant ramp up and we were able to see scale up the customers without impacting uh our cost drastically there. Uh but to do all that we need to be proactive about how do we monitor our data? How are we looking at cost? So education is extremely required to make sure everybody in the company understand what are the where exactly you are spending your cost what the cost components are and specifically in case of this environment we need to look at PowerBI cost we need to look at VM's cost networking cost DBSQL cost all those costs has to be looked at across all of these environments on a week-by-eek basis and we go and look at it if there are any changes that has happened across these environments if the cost is gone up how can we start to go go and monitor and see what could have constituted to increasing the cost and then we go and go and decrease the cost after that or or make conscious effort to identify the problem and fix it.
So for doing the cost analysis we used Azure cost analysis as well as cloud zero. But uh to optimize the cost we need to start looking at uh Azure workbooks and we have built our own PowerBI dashboards to go and look at across every environment what is the cost per week and how do we actually monitor and see what could have constituted uh this increase in cost and reduce that. uh when we went to data bricks uh we were able to also provide uh almost about 5020 different dashboards scanned dashboards to our customers directly uh with 40 plus uh marketplace visuals that are available that customers can pull it and build their own custom visuals. uh we were able to scale to scale to billions of data points and that was required because we have this large volume of data that's coming in across the board and uh uh complex data models because we talked about 26 different uh data models or uh uh schemas that we have behind the scene.
So we need to make sure all the data is flowing into data bricks and these relationships are built correctly so that customers can actually go and build an aggregated dashboard across these different data models to provide board level reporting which goes across the entire suite of applications. So data bricks made all that possible for us because of the scalability uh changes that we were able to accomplish there. Uh here you are seeing the some of our dashboards that we have built.
On the left hand side it is the dashboard that we have to build it. Homegrown dashboards that we have it earlier. Uh this was uh built for On the right hand side these are the PowerBI dashboards we have built. So as you can see it here we were able to slice and dice the data. We can drill down and provide a much more compelling visual where people can go and slice and dice the data depending on their needs. Uh this is for a risk management solution and the next one is for consent management solution.
So once we have gotten all of this data one of the requirement for us to go and look at okay now we have all of this how can we actually go and provide AI or NLP capabilities on that. So I talked about a little bit I talked about regulatory intelligence earlier. We have a framework for regulatory intelligence and the content for regulatory intelligence called data datag guidance. com and we can actually go and provide uh the content on what exactly the regulatory requirements are there for your data if the data moves or the data breach happens.
So we have built a chatbot on there. So that provides the regulatory intelligence and the same chatbot is now extended to use NLP capabilities and you can now go and find out exactly if you want to query specifically about your data you can use nature language and query the data and provide the regulatory intelligence or ask any specific questions on your data right in the chat bar there. So we have built it using large language models.
Again, we are not in AIBI or Genie yet, but it is all all homegrown build using uh AIBI and translating SQL to NLP to SQL. So key takeaways. So if you have to talk about democratings, democratization of data must be a cost discipline is required. I did talk about a couple of things earlier saying how critical is to have a security rigor as well as the cost discipline. uh what we have done it earlier we did not pay quite quite a bit of attention earlier for looking at the cost and we decided to look at the cost when it cost starts overrun later on so I so the one of the key takeaway for us was to start to look at cost on an ongoing basis analyze and educate everybody in the company and say okay this is the cost going to be and how you can actually start to experiment and look at look at these changes on an ongoing basis and governance also comes into play saying that governance going to sometimes have an impact on on your innovation altogether.
So governance is not a blocker in our case. Sometimes block governance is required to make sure you have the right controls in place and that's enable business accelerator. So you don't actually go and build and duplicate the work multiple times and everything starts with the with the with good data. You have to build the right data set quality of the data and use data responsibly. Uh if you want to continue the conversation, we are at booth 322.
Please stop by there and if you want to have a chat with me or connect with me on LinkedIn, please feel free to do so and we will uh continue the conversation there. With that, I'll hand it over again to. Thank you very much. And I'd love to welcome up to the stage as well from another one of our customers, Yep. You guys go ahead and take a seat. I'll get us transitioned over here. All right. So, let the music quiet down.
Awesome. Great. So, thank you guys both for joining us today. I know there's a lot going on in real life, a lot going on at data and AI summit, too. Um, but I'd love to start, you know, Anoop, I know got to introduce himself. Perhaps you could give everyone an idea kind of yourself, what you do, your journey with data bricks. Yeah, absolutely. Hey, everyone. I'm the chief architect at Yepit Data. Uh I've been at the company for almost nine years.
Um we've been a long-standing data bricks customer and definitely gone through a lot of the journey of data democrat democratization. Um just a little bit of the background about what yepa data does. Uh we analyze billions of data points every day to provide insights that are both accurate and timely that cover thousands of companies, multiple industries. Our primary customers are institutional investors and fortune 500 companies.
And our goal is to just help them make better decisions whether it's investment decisions strategic decisions. Uh we want them to have the right data to make that information. We acquire uh multiple different types of data sets, license them or build them ourselves. We bring it all together, make sense of it. That's really where um our value proposition is in that analytical capability and we use data bricks to enable that.
So, um, kind of like what Brando was getting at for us, we've been going through a huge data democratization journey over the last several years partnering with data bricks. Um, at this point, roughly 60% of our employees either actively use data bricks, uh, on a weekly basis and probably 80% of our employees are probably interacting with some data artifact that's coming from data bricks. So, it's super embedded into all of our data operations. as a data business you can kind of see there just a lot of natural overlaps with the platform data bricks provides and how we can kind of operationalize it.
Awesome. Um, so An you mentioned, you know, you've got what 60% of the business in data bricks frequently, 80% kind of interacting with artifacts regularly. Can how do kind of some of the different roles uh interact with data bricks? What are they what are they kind of doing to get the right data access and how are you controlling to make sure they're not going crazy? Yeah, absolutely. And I think you know they touched about this too, but like data access really important to get right.
Um, and cost controls play hand inand with that. We have a 20 pabyte data lake. So we can't just have all of our you know 60% of our employees just querying that on a regular basis. Um for us like we think about data access as what is your role and what are your business needs and that kind of basically dictates you know what kind of data sets you're going to work with and you know what types of services are you going to interact with that.
I think with data bricks you have so many tools in front of you. So you have to be kind of smart about who gets access to what and there's a lot of ways to kind of control that. The way we think about some of our personas like we have our kind of at our core and like more entrenched in data bricks are data engineers, data scientists. They're going to be using a variety of products from data bricks where it's the notebooks.
Um you know there's you know classic compute. Uh they'll also be doing a lot of stuff with like model serving and endpoints and the data science tools that are there. Um they're going to be more technical focused and they'll typically have broader data access because they need to do a lot of that data transformation work. Uh we'll also have data analysts and they'll typically you know get more involved more at like the silver and gold level of our data sets and they will be sometimes using notebooks sometimes using AIBI dashboards.
Um they're kind of doing that final level analytics kind of providing the artifacts that go to our customers. Um but then we have a lot of data operational folks that have been kind of growing over the last several years especially as more of our data sets have been enriched by ML models and techniques like that. um we need people like humans in the loop looking at that data. Now they don't necessarily need to be technical in the sense of like they need to know how to code and so that's where we've started to shift more and more to data bricks apps to kind of create you know standardized experiences where they can interact with subsets of our data.
They often are working with more sensitive data sets, but they don't need to see everything about that. And that's where data bricks apps can be a great way to get them the right access to what they need to do, be able to interact with that data, take kind of actions to augment or correct that data, but in a very controlled and guided manner. And so those been kind of like our different layers to kind of control data access.
Makes sense. And you know, I love that slide you showed us that had, you know, growing 5x, but the cost only grew 0. 5x. So I think that was like a huge takeaway but I feel like there was a lot of work behind there right it looks very easy on the slide like oh yeah we just grew massively but our costs stayed under control uh can you tell us a little bit more about some of the kind of actual challenges you face what you had to push through to get that sort of kind of efficiency and growth that's a good question right so originally our intent was to start to migrate data from uh to data bricks and our whole promise was okay we need to go and optimize for getting the exact same thing that we do it in in go build the same dashboards uh and provide uh experience more focused on developer experience rather than the business outcome and business experience and what we want to achieve.
It first was move really fast get everything done and we'll figure that out what what goes on there. Uh in the process we didn't even think about what the cost implications are going to be because now we're talking about not move lift and shift but also looking at a lot more data and a lot more capabilities that need to be need to be added. So we started to focus more and more on how do we optimize optimize cost what controls we need to put into into into that and the lesson that we learned I would say that one of the biggest lessons that we learned in the process was as a saying goes it says uh we need to if you want to go uh go fast go alone if you want to go go far go with the team and we actually actually really applied that we started when think about okay need to bring all the stakeholders together to get together whether it's looking from a fin perspective whether it's looking at business people whether it's looking at sales Everybody's customers, everybody engaged all together in the entire process and see how can we how can we get get faster to what we look looking for and achieve the business outcome.
Also in the process we identified there's lots of skill gaps. So the expectations were okay we will build a dashboard but is it the dashboards are really achieving what the customers is looking for or the sales team is looking for anybody looking for. So let's go and marry the uh marry that and bridge the gap between and find out okay we have the data let's go and find out the right insight that is going to be extremely critical for everybody to do that.
That makes a lot of sense. That makes a lot of sense. And you guys have talked a lot about kind of working with like kind of matching the right capabilities and access to the right folks in the org based on persona. Talking about kind of bringing everyone on the journey. Were there any tools or kind of e like kind of thirdparty pieces and systems you integrated with that helped with this journey? I don't know an if maybe you guys used any at.
Yeah, I think you know there's definitely been a number of tools. Um for us I think a lot of it started with Unity catalog you kind of talked about at the beginning and you know we very much have seen that play out for us very well. Um you have to you we have so many data assets with like probably over 200,000 tables organizing that and kind of managing that you need a central centralized layer. So all data access for us is going through Unity catalog.
That's kind of the foundation. Uh the thing with Unity catalog and just with data brick services is they are flexible. You kind of have to design a system that makes sense for your organization. And for us, you know, we kind of realized we need to model access control based on what our organization looks like. Like what is our org structure? Who are those groups? Model that in data bricks. So that you know when people ask for access you they're not necessarily thinking about oh am I, you know, in the finance department or am I a member of this corporate uh solutions team.
They are just looking for like can I get access to this table? And that's just fine. That's kind of like how they're going to go about their day. But we have to think more from a centralized perspective of like okay if that person's going to ask access I know his teammate's going to ask access for it tomorrow and so on so on. So we need to get ahead of that and I think that's where you know data bricks unity are really work well and kind of you can set up the group structure that makes sense give everything give everyone access based on the groups they have don't try and go down the world of like hey this person gets access to this get this person gets access to that.
I think like those kind of principles have been very effective. Um, in terms of other tools, I think some of the services and products that data bricks offers, we talked a little bit about data bricks apps. I think that's been really effective. Um, one thing I think that maybe sometimes is surprising and we've kind of learned this over the last year with data bricks apps in general is um, there's a lot of built-in frameworks that it comes with.
Um I think in today's day and age with coding assistants and just how prevalent like lower lower code application frameworks that exist you can actually create data apps very quickly have it connect your entire lakehouse and choose very surgically what kind of data sets you want to control and what kind of insights you want to bring in. think the, you know, keynotes today was a great great example of how many different technologies you can just package together in one experience for someone.
And so that's been a really effective tool where we have teams that need a range of access to different data sets, but they don't actually need to work with the data every day or like work with it in a notebook or query, you know, a terabyte of data. Like they just need to be able to pull out certain insights um and really not do anything else. And so we're able to kind of really tailor what are their operations and what are they supposed to do through data bricks apps and I think that's been like a very good other end of the spectrum from like you have your engineers and data scientists really just working in a notebook environment and then somewhere in between are kind of like the AI BI dashboards and Genie which we've been actively moving to where you want to bring in everyone else in the or that just needs to get you know a live read or have certain questions that you know would be better than a shoulder tap or a slack ping.
Um, and so I think that's been, you know, our kind of mix of tools that have been really effective for us. Nice. Makes a lot of sense. Um, I guess kind of as we look at how you guys have grown, I mean, and you've talked about this a little bit already, but it feels like there's always a little bit of a tension between governance and innovation. Um, certainly I think some of the latest developments in Genai have made it more of a flash point, right?
Everybody's always like, "Yes, I want both." But I don't think everybody always knows how to get there. And when when that path isn't clear, it falls back to like I'm going to hold on to my governance, right? Um so VC I don't know maybe you could tell us a little bit about how you guys think about that at one trust and you know have you found the best of both worlds yet? Yeah, it's I I'll again use the analogy here, right?
So, if all of us has a fast running car and this only has an accelator, how many of us will actually go and drive the car? Probably none, right? So, we need we need a break in the car and we need a we need accelerator in the car uh to be in the stay in the lane and follow the governance around that, right? The right use of this is called responsible use of the car, right? So, same thing we do responsible use of data.
So, we need to actually go and have the right governance and I don't think the governance is actually a distractor. It is actually an inn innovator or supporter in in all of these cases because we need to make sure have the right guard rails we follow the right practices we use the ethically we are using the right data and secure the data so it is actually enabler in my my opinion that's what we have seen it so we actually start to bring in a lot of people who are responsible for making sure the data is actually governed correctly we have a responsible AI committee they come and talk about every use case that we have done it and actually help us enable move faster and think about things that could matter and could actually be distracted down the line so think about early on so we don't have to go and change our course.
So it's a Got it. That makes sense. Yeah. I mean, best of both worlds, right? Gas break. Okay. Um well, I guess, you know, I know we're we're wrapping up here, but Anupc I know everybody here kind of has some some questions about cost controls and you know, learnings. I guess if we were going to distill it all down to one piece of advice, what would be your like top piece of advice for everybody here? I can start. Um yeah, you know, I think for us what we've realized over these past few years on this journey for us um data democratization in general uh shouldn't be a technologyled initiative and I think it kind of gets to what was just saying in the just the last question like you need to find that alignment with your business stakeholders and treat it more as a collaborative exercise. uh it's going to be so hard to it's just pulling teeth to try and get people to get excited about this stuff, but until they realize like how it actually impacts them and it's going to solve their problems.
And so, you know, what we've seen as a really effective pattern is if you're starting new in this area, focus on a specific use case, a specific business use case that's going to have real impact and make sure that business leader, the project stakeholder, like they really kind of understand the vision, are bought in and kind of design a solution with them. And then you can kind of socialize that win and kind of get momentum.
And that's been the story for us where when we first started with data bricks was only covering a certain part of our business. But as people started to see how quickly you could be productive, generate insights, generate business, and answer questions for our customers, it kind of created that flywheel effect internally where everyone else was raising their hands. And part of that is, you know, you have to figure out how to make this transparent to everyone.
Both the outcomes, but also, you know, I think the cost side of it as well. Um, you don't want everyone to be like, hey, I want to use data bras. It's just a magical thing. and not really understand okay there is a bill on the other side of it there are resources return on investment that they need to make but they should understand that and so we've spent a lot of time over these years really kind of um tuning how we do that cost attri attribution tag everything tag everything sensibly so it's like something that makes sense to the business stakeholders um and it's gotten to this place where like you know we have like AI API dashboards that go straight to our CFO and now he understands exactly every line item in data bricks cost that we're generating what it's related to what business initiatives.
So if there's a spike, he knows who to yell at and it's not going to be us on the engineering side, it's going to be, you know, that business person because he understands all these things are actually generating business on the data platform. Um, and you know, there's business needs involved and there's just a trade-off that we're all going to have to balance. Awesome. What about you? Yeah, I'll tag into what Anup was talking about, right?
Is the cost is extremely critical, right? So if you look at cost the percentage of the revenue the more controls you can put in the cost that's important so that we can have more more revenue and benefits from the uh from that alto together also in in my opinion I think it's more important to also look at from a culture as much as architecture because we focused a whole lot on culture uh on architecture and not on culture earlier data literacy is important how do we make sure everybody understands uh what we are doing changing the mindset and aligning the definition that's another critical thing that we encountered was a taxonomy across all of those business units that we have it.
Exactly. Same speaking the same language all together and build build the trust in all the teams that you work with. Yeah, makes sense. Everybody's got to be on board or nobody's on board essentially. Exactly. Yeah. Awesome. Um well, I want to say thank you guys both for coming and kind of sharing all these stories. You know, we build features after talking to you, but I feel like the story behind the features gets lost.
So, really appreciate you both coming and kind of highlighting your journeys. Um and certainly I'm sure folks can come up and ask some questions after if they're interested. Awesome. Absolutely. Fantastic. Well, thank you guys all for joining us today. Um we're all going to hang around a little bit if you want to do a little bit of questions kind of off the cuff afterwards and we really appreciate you guys coming and learning a little bit more about the journeys.
Thanks so much. Thanks all.

Learn more about the Databricks Data and AI platform.