How to Manage High-Quality, Secure Data and Cost Visibility for AI

Summary

• Data quality for AI readiness: This video covers managing high-quality, trustworthy data at scale, emphasizing the importance of understanding and securing data from the beginning of its lifecycle.
• Tokenization for security: Learn about tokenization as a highly efficient method for securing sensitive data without compromising AI performance, essential for protecting data during training and inference.
• Cost visibility and optimization: Discover how integrating compute cost visibility and optimization helps manage the hidden costs of AI, enabling data-driven decisions on infrastructure spending.

How to Manage High-Quality, Secure Data and Cost Visibility for AI

Talk by Laura Case, DIrector, Product Management, Capital One Software (Sponsor Speaker); Yudhish Batra, Distinguished Engineer, Capital One (Sponsor Speaker)
Companies need robust data management capabilities to build and deploy AI. Data needs to be easy to find, understandable, and trustworthy. And it’s even more important to secure data properly from the beginning of its lifecycle, otherwise it can be at risk of exposure during training or inference. Tokenization is a highly efficient method for securing data without compromising performance. In this video, we’ll share tips for managing high-quality, well-protected data at scale that are key for accelerating AI. In addition, we’ll discuss how to integrate visibility and optimization into your compute environment to manage the hidden cost of AI — your data.
Here’s more to explore:
Unified and open governance for data and AI: https://www.databricks.com/product/unity-catalog
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements

FAQs

Why is data security important from the start of the data lifecycle?

This video emphasizes that data security must be integrated from the beginning rather than applied later, as improperly protected data can be at risk of exposure during AI training or inference, compromising compliance and trust.

What is tokenization and how does it protect data for AI?

According to this video, tokenization is a highly efficient method for securing data without compromising AI performance, replacing sensitive information with tokens while maintaining analytical utility for training and inference.

How does this video address the hidden costs of AI?

This video discusses how to integrate visibility and optimization into compute environments to manage the often-overlooked costs associated with data management in AI workloads, not just compute resources.

What best practices does this video share from Capital One?

This video shares Capital One's approach to data architecture transformation, security solutions, and best practices for managing high-quality, well-protected data at scale to accelerate AI deployment.

Full transcript

All right. So I think it's time to kick off the session. Hello everyone. Good morning. Uh uh I work at data bricks as a solution architect. So I'm actually based in Germany and uh still like trying to dealing with my jet lag but super happy to be here. This is also for me personally the very first uh DICE event that I've been participating with. So today I have the honor to you know welcome you and then uh give a short introduction to our speakers here from Capital One Software.
So I'm with Utish and today I think they are going to show us uh how to manage the high quality secured data and also manage cost visibility for AI which I think is a super relevant and then interesting topic for you. So thanks for coming and with that I'm going to hand it over now to and to the stage. Hey, good morning everyone. Um, welcome. Uh, I'm a distinguished engineer at Capital One Software and I lead the architecture for our product Capital One Slingshot.
And I'm, director of product um, the product lead for Data Bolt Partnerships. So, let's just dive into today's agenda. First, we're going to tell you a little bit about Capital One and our tech transformation. Then we're going to talk a uh in about the main focus of our presentation today, data principles for AI. We're going to talk about how to manage highquality data that's trustworthy. Then we're going to discuss how to secure data for it to be AI ready.
And then once you're ready to build and deploy AI models, how do you sure ensure you do this cost efficiently? And at the end, we'll wrap up with some key takeaways. So you may know Capital One as a bank, but we are actually a technology company. We've undergone a dramatic tech transformation to operate with the speed and agility required to succeed in the digital age. In 2016, we declared that we were going to be allin on the cloud to allow us to harness the power of all of the massive amounts of data that we've collected.
In 2017, we had to start rearchitecting our entire data ecosystem on the cloud in order to manage data and AI at a larger scale scale to create our foundation for AI analytics and machine learning. With this, we started to adopt cloud technologies such as data bricks. And in 2018, we assessed very different um security solutions to be able to protect our data in the cloud. And we found that there wasn't a solution on the market that would meet the scalability and performance needs of our organization.
So we started to build our own in-house tokenization solution. As of 2020, we shut down our last on-prem data center and had moved all of our data to the cloud. And along that journey, we learned a lot of lessons about how to manage data at scale in the cloud. And with that, one of the things that we developed internally was a way to manage our costs. And um as we worked on this product called Slingshot, we realized that other enterprises could take advantage of our lessons on how to more effectively manage the cost of their data.
And so we launched Capital One Software. And Capital One software is a B2B enterprise business within Capital One that takes products that we've built internally for our own needs and commercializes those. And then two months ago, we announced the latest product uh which is a commercialized version of our tokenization solution, also known as data bolt. Along our journey, we've learned that highquality and well-managed data powers our ability to build proprietary AI tools.
Some examples are here of different tools that we've built. The first is Chat Concierge. This is a a multi-agentic AI solution that connects car buyers and car sellers to one another to improve the car buying experience on both sides. It also takes action on customers behalf or on customers behalf based on the requests that they make through the tool. We've also developed our own in-house Genai driven uh agent servicing product.
This allows our thousands of agents to efficiently and effectively manage our customers relationships when they need to connect with uh our customer service lines. And then lastly, we harness the power of our data to provide AIdriven personalization to improve our products. Um this proprietary AI system enables rapid experimentation and iteration for our 100 million customers. Our use of these AI tools creates a flywheel that drives increased customer engagement.
Our application of AI powers exceptional customer experiences which in turn enhances customer engagement and interactions with us. Customer engagement provides insights into how satisfying the experience the experiences are and also opportunities for us to improve them. And then these insights lead to more engagement and continued refinements of the experiences we provide. Through this flywheel, we apply even more customized realtime solutions to unlock impact for our customers and for the business.
Now, research shows that there are three common barriers to adopting AI in data management. The first, trusted data is a top hurdle to implementing AI. Data is everywhere, growing rapidly. How do we find the data we need and trust that data when it's put into a model? And then second, data and data security hinder adoption. Customers trust us with their sensitive data and we want to make sure models or chat bots aren't going to leak any of that information.
And then lastly, cost. It can be incredibly expensive to build and maintain models. So, how do we reduce the cost barrier to get more value out of the data? At Capital One, we face these same challenges in our journey to adopt AI. And our presentation today will focus on each of these pillars. First, how do we get to trusted data? Second, how do we secure data in an ever growing data landscape? And then last, how can we be costefficient and get as much value out of our data as possible?
Now, the biggest lesson that we've learned in our journey is that worldclass AI starts with worldclass data. I've been in the data and analytics space for a couple of decades. And we've said this very differently with our data teams, which is garbage in, garbage out. Your analytics, machine learning, your AI is only as good as the data that you feed into it. Having strong fundamentals and a commitment to a strong data foundation is the baseline for AI readiness.
And we've learned some best practices on how to turn that phrase around um and to get to good AI. So, let's dig into what that means. First, we want to talk about how to manage high quality data, also what we call trusted data. Trusted data is the foundation to ensuring data is ready to use for AI use cases. Our experience has shown that there are foundational pieces you need for a data to be AI ready. First, you need highquality metadata.
You need to understand what data you have feeding into your models, and that requires understanding a lot more about your metadata. And then you also need highquality data. Again, we're trying to avoid garbage in, garbage out. And then the last thing you need is end to end lineage. Um, we're going to drill a little bit deeper into each of these so that um, we're all working with a common understanding of what we mean.
So, high quality metadata, what do we mean? What we mean is first, where is the data? Um, is it in data bricks? Is it in shared drives? Is it stored on somebody's laptop? Um, do we have it in S3 buckets? We really need a solid inventory of where that data lives that you want to feed into models. And it's even better if you have an inventory of everywhere you store data across your data landscape. And then once we know where the data is, we need more details about that data.
What is the database schema table column? Do you have any data classification of it? What type of data is it? And then do we have any policies around that data, its usage and its storage? Um is it protected health information? Is it PII? Are there any regulations around using that data? Having really good quality metadata lets you pick the best data available for your AI models. And then once we know where our data is, we want to know how good the data is.
And again, all of this in the avoiding the garbage in garbage out. Um data quality assessments can be done in multiple ways. So some options are to scan the data either manually or automatically. Um you can also use data profiling, data quality tools. Um there are many different methods to assess the quality of your data. And then sometimes the data comes to you in a trusted format such as data sets you purchase. Um or maybe you have an internal system with trusted data such as your master data management system um that is coming to you that you know the quality of that data.
And then lastly, data lineage. So data has a life cycle. very rarely do you see data come in and have it not change at all. So, first data can change over time. Um, simple example like myself, my date of birth has never changed. Um, however, I do have twin teenagers and in their teens, uh, my hair very quickly moved from brown to gray. Um, and then obviously over time as well, my address has changed. Moved to a slightly larger house with a growing family.
Um data also changes as it moves from system to system. So you may ingest the data and then you run some type of transforming processes on it. Um or like we mentioned before, you know, you're running data quality checks and fixing that data and resolving some data issues. And so really understanding your data at every step in its life cycle is critical to understand what data you're feeding into your model. And then when choosing your data, um, you really want to make sure that you're choosing the right data for your use case.
And all of this requires a fundamental understanding of the lineage of that data. Now, once you have a solid foundation of highquality metadata, it's time to secure that data. So, there are many different ways that you can protect data. So, the first three on this screen are really about non-reversible forms. Um and then we also have reversible forms. So let's just talk about each of these. Um and a lot of these lessons are lessons learned from our own data journey.
Um and you know this is an overall spectrum. So the first piece is deletion. So if you don't need that data, you're paying to store it, you're paying to process it, you're paying to have re people on your team look at it. Um first, you know, basically deleting data that you don't need is really the first way to um protect it. But obviously if you delete data that's not going to be usable for all of your AI models. Um and then masking data.
So masking is where you convert this into another format such as um using nine X's for a social security number or using the first uh six to mask your social security number. Um again this is a valuable way to protect your data but largely leaves it unusable for models. And then redaction which largely applies to unstructured data is another way to protect data. So then we get into the two options that actually allow you to reverse them which is encryption and tokenization.
Um people generally are more familiar with encryption and so we wanted to talk a little bit about tokenization how that works and specifically API based tokenization. So at the core both encryption and tokenization use cryptographic methods to convert clear text to cipher text. Um encryption requires a key called a data encryption key to decrypt and or to encrypt and possibly decrypt the data. Um so API based tokenization which is what we use at Capital One.
Um the way that it works is that an application receives some type of sensitive data. Then they send a request to convert the data to from clear text to cipher text and then the tokenization layer receives the API permissions and if approved sends the token back to the application so the application can store the token and then the reverse is also true if the application has the ability to detokenize the data. Now, as I mentioned during our data protection journey, Capital One chose tokenization and we to chose this for a few different reasons.
First, tokenization requires less key management than encryption. Um, it also protects against brute force compromise. Uh, second, tokens can have embedded metadata that allow for bad token detection as well as help when the tokens need to be rotated. Um tokens can also be format preserving and so the do data can be tokenized and then we can continue to use it um without modifying the underlying schema or the applications that use it.
And then most importantly um tokenization allows for the utilization of that data when it's tokenized allowing us to still get full um value out of our tokenized data in analytics and AI use cases. So, Capital One tokenizes our data at scale. Along the way, we've learned some best practices in data protection. You'll notice that these best practices are very familiar as we walk through what we've learned. First, we learned that having high quality metadata is key.
If you want to protect data at scale, you need to know what data you have and where it is. And to be able to tokenize a scale, we need an inventory of what data we have. And we use scanning tools to scan all of our data. The next best practice we learned is that we need to understand the policies that are applied to data. We're in a regulated industry, so we needed to comply with many different regulations. We also considered whether we could use any of the previously mentioned data protection methods.
Was there any data that we could delete? Could we use masking? Could we use redaction? What data do we need to be able to use? The next back process that we learned is that we need to understand the policies that applied to that data. I think this might be a repeat or we're frozen, unable. All right, perfect. Um, so once we got to the point where we wanted to protect our data, we found that there's not a one-sizefits-all solution to this problem.
We require that our business implement the highest row that is technically possible for their use case. U with structured data, we have a lot more options and tokenization is our preferred method for structured data. With unstructured data, much of this flexibility is removed and encryption becomes the preferred method of protecting sensitive data. Our next best practice is that we recommend to use robust data access controls.
We leverage cloudnative features for data access control. Uh we use identity and access management which allow for the granular definition of permissions at the user group and role level ensuring that access to data is appropriately restricted to authorized users. And we rely on the native functionality of AWS's access control features paired with our custom access policies within our tokenization solution to lead to more scalable, performant, and secure operations.
And our last best practice is that data protection is never done. It is continuous process. Um, for us, data protection involves continuously improving our process of scanning data, optimizing the efficiency of our tools, scanning newly acquired data, and refining our data ingestion and tokenization process. Now, with all of these learnings, that brings me back to Capital One software. So, we recently announced the launch of our tokenization solution, Data Vault.
Um, our solution offers a few features. Um, our security model ensures that sensitive data never leaves the customer's environment. Our cloudnative architecture offers a flexible deployment model that can fit a customer's unique infrastructure. And we offer lightning fast performance um with a throughput of up to 4 million tokens per second. Now, you're all at a data bricks conference. So, let's talk about Capital One data bolt for data bricks.
So we also seamlessly integrate with data bricks to offer customers high performance and convenience while securing their data. Um we integrate with unity catalog data bricks workflows and datab bricks role-based access control as well as data bricks userdefined functions to provide an endto-end tokenization solution. Um we leverage Unity catalog metadata information and data bricks workflows to allow customers to choose what data to tokenize and to facilitate simple tokenization.
Um, customers deploy the tokenization engine directly in their environment to decrease latency and tokenization functionality is accessed through UDFs that are simple to interact with and set up automatically during installation. And we leverage data bricks uh user groups to configure data bricks arvback allowing users to tokenize and detokenize data. Now that we've protected the data, we'll the next uh practice is to um yeah to manage your costs.
Thank you. Um now that we have seen how to uh build highquality secure data um and spend more money, let me go back and save some money. So uh we'll share some best practices on how we can build a cost-effective uh data ecosystem. Um but before we get get into the how part let's talk about the what uh like what is the total cost of ownership in in today's world of data and AI and the options that we are getting the total cost of ownership is changing and evolving.
So let's talk about that. Um the first and foremost is the cost of uh data and this is further divided into three different categories. So data management infrastructure and what do I mean by that is um it's any tooling that you leverage with your data ecosystem. It could be maybe injection tools uh scheduling tools that you leverage in the data ecosystem. Um second is the data preparation and engineering cost. Um so as as spoke about building highquality secure data all that cost will be in this bucket which is basically how you are preparing your data getting your data ready for AI and consumption by the analyst and securing it.
Um and last but not the least is the storage cost. Um this is the actual storage cost for the data that you pay to the cloud providers. Um it could be small right now but as the amount of data is growing it could really balloon very quickly. So we got to keep eye on what is used, how much is used, uh and can we delete it and kind of keep a check on the cost. Uh the next is the compute cost. Um this usually would be the major chunk of cost that you will incur in your data ecosystem.
Uh and this is it includes the cloud infrastructure which is basically any compute that you are uh provisioning for processing your data. Um in data bricks it could be the job compute APC um and if we look at the AI tech infrastructure so as we have new models coming in we are building models training fine-tuning them um any sort of GPU that is leveraged would fall in this uh category of AI tech infrastructure. The next is the model built cost.
Um so this is basically if if you decide to train your own model own base model uh you would incur a heavy upfront cost of training the model. Uh but if you want to leverage a model that is available out of the box in the market u then you would have some cost in fine-tuning the model basically training the model with your data so that you can answer specific questions about your business. Um the next is and last is the model operations.
This is basically once you have the model ready I want to serve the model to the customers. Uh so all the cost that is incurred in serving the model um is in this bucket as well. Um we have the model monitoring which basically keeps a check uh on the model to see if there is any bias or if there is any degradation in the performance of the model. Um so now that we understand the total cost of ownership uh let's look at key areas um which we need to invest in early uh to make the most out of uh our data.
So first is the governance um and what do I mean by governance is that we need to put the right controls checks and balances in the data ecosystem uh to make sure it is utilized properly. There is no abuse. there is no overprovision compute um um and there are controls and checks in place where the data platform owners can feel confident in scaling the use of the data platform across the organization. Um the next um and the key is optimized compute.
So as I said compute is a major chunk of cost. So you want to make sure that you have the right size compute at the right time for the jobs. Um and the last is the monitoring and observability. Having right monitors, right controls in place will make sure there is no wastage in the system. Um and you are proactive in responding to issues. So let's dive into each of these categories. U first is building a well-governed data environment.
So um at Capital One we adopted um the cloud data warehouse technology early and there were very few tools that were available in the market and we decided to build something for our own. Um and we have lot of lines of business in capital one which leverage these tools and then there is a central team which manages the data ecosystem. Uh so we decided to go with a federated ownership model where we have controls defined by the central team. uh whereas lines of business are kind of uh always within those boundaries and using.
So we we chose the federated ownership model. Um next is defining the budgets. Um having the right cost controls um um was the key to make sure that we don't overrun our budgets our allocated uh money to the data platforms. And last in this was having the reviews and approvals, having the ownership u uh defined and getting the right approvals before allocating the money to a particular use case. Now this is this is how we did the federated uh ownership and the governance um the herical governance.
So if you look at the bottom um we have a group of compute owners uh which are responsible for certain projects u they belong to a business unit let's say analytics team um and then in a business unit of data engineering. So um we have admins in this uh group which are responsible for their own organization their own project teams. They have their own defined guardrails and then on the top sits the platform admins which have defined the overall governance for the data platform and they are responsible for managing the entire data ecosystem in our organization.
So u this is how we implemented the hierarchical governance in um Capital One to federate and scale our data usage across the organization and um we are Capital One software so we kind of package that solution which was built built internally and it's available um as Capital One Slingshot. So you can configure it for your own business. You can choose how you want to use the approval system, how you want to use the controls in the system and um you should be able to leverage it for your own um um company.
So um and we recently launched it for data bricks as well. So we support data bricks uh within capital one slingshot. Now next is optimizing the compute infrastructure. Um so the first and foremost in optimizing your compute infrastructure is the compute specific settings like what type of compute do you want to leverage um you need to pick the right size right type of the compute for your workload. So if you are running analytics queries you might want to choose a SQL warehouse that is available.
If you are running jobs um the job cluster is the right compute instead of APC. So that's that's another key thing that we have to keep in mind that we pick the right size right type compute for the workloads to get the maximum value. Um and the last is leveraging the platform features um that are provided by all the providers. Uh so in data bricks um they have GPUs available and u other feature which databicks provides.
So um now moving on again um in Slingshot uh we have built a job optimization uh solution which basically makes sure that your job cluster is always u fit for your workload and it is meeting the SLA. So uh this is available in our product and it it is basically a MLbased solution where it continuously monitors your jobs and right sizes your compute uh for your jobs and uh saves you money. Um next is implementing monitoring and observability.
Um so um strong foundation requires uh monitoring in multiple aspects. We need to monitor multiple aspects of the data ecosystem. And the first is monitoring the data pipelines. So as your data pipelines are running, you want to make sure that they are healthy, they're not failing. Um and if there are failures, somebody is alerted. This would ensure that uh the data arrives timely. U the data available to the downstream models is accurate.
Um and the business insights that you're deriving from the models are correct. The next is the data quality checks. Um as mentioned um we need to monitor for uh we need to have data quality checks in place plus we need to monitor that the data quality checks are passing as they go along. Uh if there are failures somebody is actioning on the failures and making sure that the issues are fixed in a timely manner. Um and the last is the cost visibility by workloads.
So Laura touched on the data lineage piece which is basically how the data is flowing through the ecosystem. Uh we also want to keep track of cost by workloads so that we can derive the return on investment. So let's say you're building a pipeline the data flows through multiple steps. It gets transformed and finally gets used in um in a BI report or in a AI model. We want to have the visibility of cost across the workloads just like lineage so that we can capture uh what's the cost of this data and how it is benefiting the business.
Um and with all that uh it should be done proactively. It it should not be reactive. With proactive you can make sure that your data is accurate uh timely and the insights that are generated from that are uh good and um have good impact on the business. Now getting this uh these things right is the uh key to deploying u and building u AI models. Um and we want to leave you with three key takeaways which is uh trusted data is the um is the key to good AI.
Um and data security is a continuous process. Uh you got to keep monitoring your data uh to make sure it is secure. And last is cost governance should not be an after afterthought which is you want to invest early in making sure you have the right controls before you scale your data ecosystem across the organization. All right, that's it. Thank you so much uh for taking the time and you can visit us at the booth uh E412 uh you can check out both our products data bolt and slingshot.
Thank you.

Learn more about the Databricks Data and AI platform.