Sponsored by: Monte Carlo | Building Trusted AI on Certifiable Data Assets at American Airlines

Summary

American Airlines' head of data governance Andrew Machen shares how his team built a complete governance program—including policies, committees, operating models, and certified data assets—in just over a year, driven by the urgency of the company's AI strategy.
This video explains how data observability tools from Monte Carlo shifted American Airlines from manual, alert-heavy quality monitoring to automated checks that reduce alert fatigue and allow data stewards to focus on delivering business impact rather than manual oversight.
American Airlines designed its governance program with AI in mind from the start, distinguishing between big-G governance (formal organizational structures) and little-G governance (day-to-day data quality practices) to make it understandable and actionable across the business.

Chapters

FAQs

How did American Airlines build a complete data governance program in just over a year?

According to this video, the forcing function was American Airlines' AI strategy, which required governed data to deliver trustworthy AI outcomes. The team moved quickly by establishing clear operating models, policies, committees, and certified data assets in sequence, and by partnering with Monte Carlo for data observability to automate quality monitoring rather than building manual processes.

What are certified data assets and why does American Airlines use them?

Certified data assets are datasets that have met rigorous quality and governance standards defined by American Airlines' governance program, including verified data quality checks, ownership assignment, and lineage tracking. Certification ensures that every dataset used in AI or analytics workloads meets a known standard, giving teams confidence that what they are building on is reliable.

What role does data observability play in American Airlines' governance program?

Data observability, provided through Monte Carlo, allows American Airlines to monitor and validate data health automatically rather than relying on engineers to manually check pipelines. This video describes a shift from a push model—where too many alerts overwhelmed teams—to a pull model where data stewards investigate meaningful anomalies, freeing them to focus on impact rather than noise.

What advice does Andrew Machen give organizations starting their governance journey?

This video closes with advice to make allies across the business and to speak in business language rather than technical governance jargon. Machen emphasizes that governance succeeds when it is simple and understandable across all parts of the organization, and that building relationships with business stakeholders is as important as the technical implementation of policies and quality checks.

Full transcript

[00:10] >> Perfect. Okay, now we can actually get started. >> My name is Bar Moses. I'm the CEO and co-founder at Monte Carlo. Monte Carlo's mission is to enable world the world's enterprises to adopt trusted AI. And what we recognize that the heart and
[00:22] soul of AI is trusted data. And we have the fortune of working with um amazing uh enterprises to help them deliver trusted data and AI products. Um and Andrew is one of them. I've known Andrew
[00:34] for many years now. Um Andrew's title is head of data governance American at American Airlines. I'm I'm reading off the slide, but it may have changed. Uh so you might have to you might have to correct me on that. Um I'm really
[00:46] thrilled for this conversation for a couple a couple of reasons. The first is you all have had a really impressive journey and you've been very intentional in your journey and one of the most I think hardest part in the transition
[00:58] from data to AI is nailing governance and I think if I can say if there's one customer that I've worked with that has really nailed governance or has made strides towards that I think that's you guys. And so with that, Andrew, if you
[01:11] could just start by introducing yourself, um what your latest title is if I got it wrong, maybe a little bit about sort of your role today. >> How do you think about sort of data and um at American Airlines? >> Okay. Well, thank you so much, Bar, for
[01:25] uh for having this conversation with me and uh you know, helping me to represent all the great people that work on data and data governance at American Airlines. It's truly a team effort. Uh
[01:37] my name is Andrew Matchin. I am uh the head of data governance as senior manager at at the group and uh I've been working with American Airlines for almost 15 years and so uh you know it's
[01:50] so yeah we we've been working really hard to try to make to try to make governance work and uh certainly it's been an amazing partnership working bar with you and your team uh you know we uh we're we're truly lucky to have such
[02:03] great partners. >> I love that. So let's get started. Um, Andrew, let's just dive into sort of setting the stage here. >> Um, your team has built a complete data governance or sort of governance program
[02:15] at American Airlines, policies, committees, certified data assets, the whole thing in just over a year. And y'all have done some pretty remarkable things. I remember in your move from to
[02:28] the cloud in a very short amount of time and very quickly shortly after spun up um a governance. Mhm. >> That's a really remarkable pace for a company of America like American Airlines scale and regulatory
[02:40] complexity. >> What was the forcing function for that? I think many companies in the audience that I work with >> would you know kill so to speak to get to move that fast and to be able to
[02:53] accomplish so much in short amount of time. What was the push for that? what was actually was this a you know an executive mandate or gaps in your data like what actually instigated this and helped create the urgency
[03:05] >> well you know first of all I have to say that we've been working really hard on data governance for about four years to make it work the way that it is working now
[03:19] uh we've had a lot of amazing people we've had my predecessor uh Neil Bat that's come in and he he did a tremendous amount of foundational work uh you know partnering with companies like Monte Carlo uh trying to get
[03:33] everything working. Uh I I think I think what you see there is you've got not much problem trying to
[03:45] get people to do big G data governance, right? Assessing the risk, making sure that the audits are in place. Uh taking care taking care of things of that nature is not a hard cell, right? No one wants to get sued, you know? We we have
[03:58] to do these things to make sure that we are that we are mitigating risk. I think for a company that's not a bank or a pharmaceutical company, right? U
[04:10] a company that is not going to need to choose centralized data governance >> as their as their pattern. A company that's going to need to choose more more of a federated or semifederated
[04:22] governance model. >> You can have a lot of issues with little G data governance as we call it, right? making sure that data is, you know, that it's available, that it is properly
[04:35] stewarded, that you're making things, you're making things go fast. And I I think that's where you have the push that AI brings to the equation,
[04:49] right? we we're in a position where you know the company is looking to accomplish you know that they're they're really trying to accomplish great things with AI and
[05:01] you know I I I think it it's helpful for us and it would be helpful for any company to be able to use that push. Uh I think there's a lot that we had to do though. I I think uh for us we we've
[05:13] really had to get a lot better as an IT organization in telling the story the way people need to hear it and you know but once you
[05:25] know once we took that accountability then I I think the combination of that and the push from AI uh you know is definitely something that I think is going to push a lot of people into
[05:37] really looking at data governance and data stewardship in a very in a very different way. Hope that makes sense. Yeah. >> Yeah, it does. And and when you say tell the story for little G, small G, like
[05:50] you mean that's like explaining to folks the ROI on that or why it matters. Is that what you mean? >> Absolutely. Right. >> You know, one of the things about data governance is that like when you're
[06:02] talking about risk, people are bought into that, right? when you talk to leaders about okay we have to do data governance that's an easy thing to say oh yes I'm fully behind data governance u but then when you have to have a
[06:15] conversation okay about well we need resources to steward data in a different way right look at data quality right look at metadata and add metadata that maybe we weren't
[06:28] adding before adding adding the kind of quality of metadata that we have to create the kind of context that is required for good AI outcomes so that you can trust what comes out of it. >> Yeah. >> Right. That's where the conversation
[06:40] gets interesting, right? That's >> that's where we have to dig in and be able to tell a better story there. >> And and I think that's that's one of the things that our leadership team >> has really been able to do well in, you
[06:54] know, in in the last year, year and a half is is to get that conversation through and get people on board. >> Yeah, makes sense. So, let's unpack that a little bit. You know, a lot of sort of the what you all talk about is sort of
[07:06] certified data assets. It's a very deliberate word choice that has to do with the narrative and the story and how to communicate that. Can you explain what does sort of certified data asset mean? Who what exactly happens? Who
[07:19] checks the box? How does a process work like in practice? What does that look like? >> Right? So one of the things that I did whenever I took over data governance was I started to think about
[07:32] the the people that I just mentioned right that'll say okay data governance is really important and then they'll eyes will glaze over when you start talking about it they'll walk away how do we stop that right how do we get people to understand what's really
[07:44] important here >> bringing data governance back >> right you know absolutely and and and changing the conversation I I think what you're seeing in data governance right now is a pivot
[07:57] toward trust, >> right? And that's really what it is. And and so what we what we tried to do and my we it was really we we spent an
[08:09] entire summer with a crossf functional team of engineers, architects, data owners, data analysts, uh leaders in business and IT
[08:23] working together to try to figure out how we encapsulate the idea kind of almost like use a uh a data abstraction concept on
[08:37] data governance to try to figure out how to make it simple and and and un have people understand and really buy in quickly to what we're doing. And we we came up with the concept of certified data assets. I'm sure that it's been
[08:50] done many times before, but this is this is what we came up with. And what it is is it is a list that we start with that is small at first and mainly manual, but
[09:02] it's the basic blocking and tackling of what we know we need to do to trust the data assets that we put forward to customers, >> right? Things like making sure that you
[09:14] have defined stewards, >> right? that you have the type of uh privacy by design and cyber security requirements and very important to us
[09:26] that you have the type of data observability out of the box that you get from data quality as table stakes, >> right? And you know we started to do that. We started to socialize it and
[09:39] people had been able to understand it, right? And so we work through that and and and at this point what we're trying to do there is we're trying to automate it all, right? We're trying to tie everything that we do with that to data
[09:51] contracts so that people don't have to work on it. >> And so that's that's where the concept is now. It's working well for us. It's something that grows. It's not minimum viable governance. It's something that grows with us as we are able to make things easy
[10:04] >> for for the people that we serve. >> Okay. And maybe you can give us just a little bit more context to sort of frame the conversation. Can you give us s a sense of footprint like how many data products are we talking how many pipelines like where you know what is
[10:18] sort of the scope of what you guys cover and where does data observability sit in that stack. So when we're talking about data products, we're being very very intentional about that, right? >> That that's another thing that we talk
[10:30] about. Like a lot of times we find people talk about data products in for us a very ethereal way. Like what is a data product? Like you know, we had a lot of groups come in, you know, paid
[10:42] consulting companies that come in and talk about data products and and that was a question that we asked like what is a data product for us? Mhm. >> Like how do we know that we trust the data product that's going out? How do we
[10:54] measure it in real time? >> Mhm. >> Right. And so our data product count is pretty low right now. Right. We're we're we're on the border of a couple dozen right now, but we're being very very intentional about it and it's
[11:06] underpinned by, >> you know, a few hundred assets under that that cross over data domains and subdomains. And so we're taking it slow right now, but the pipelines are, you
[11:19] know, that we're working with are in the hundreds, right? And and so but uh you know, we're it is absolutely a work in progress. It is always going to be a work in progress because we're always going to be trying to get better there. >> Yeah.
[11:31] >> But >> 100%. >> Yeah. And at that scale, you know, how do you actually you you talked about starting with like a small group of things that define trust, making that
[11:43] manual and then moving to automated one day in the future it'll be, you know, autonomous. We're talking about self-improving agents or sort of, you know, a lot of big, you know, things to come. How do you get your teams to
[11:55] actually trust? I think it's very hard to trust the the um automated part. Like it's actually hard to move from I'm used to seeing this to setting it up myself. Most of the companies that we work, you know, they have like hundreds of rules
[12:08] that they have manually set up. They, you know, there's the one person who's been there for 15 years who've set it up. They know and trust it. And to ask them to move to let me introduce this automated concept or god forbid a black
[12:21] box like how do you help people make that leap, >> right? Well, first of all, you have to involve them, right? You know, I continue to talk to a lot of people that are really concerned about what AI is going to do to the workforce, right? We
[12:34] talk about data as the new oil, as the new gold, but the way I look at it is these people with the acumen to be able to manage the data that we have at our companies, they they're the new platinum, right? They are they are going
[12:47] to be the the force multiplying function behind everything that we do because all of this has to be validated. We need people for context, right? that the types of things that we do are are going
[13:00] to change, right? But these people are more important than ever. I I feel like they're they're always going to be right at the center of what we have. So, the first thing is to make sure that you're involving them,
[13:13] right? That you're that what you're building is to serve the goodness that they bring forward, right? I think u sorry that what was the other part of the question? Yeah. >> Um just how do you help people like make
[13:27] the leap from Yeah. uh manual to automated? Yeah. Yeah. >> And so the the second thing is to make sure that you you're you're holding events to hear
[13:40] what they have to say that you've got an operating model. You've got an operating model in place that puts them in control >> and has them be able to ask the
[13:52] questions and guide the direction of data governance the way that it needs to be. And then the other thing is that you you have to hold the events and have partners like like we have with you >> like town hall committees to have
[14:05] discussions >> or just just technical conversations. We we you know we've done that as well. We had our first we had our first mini conference uh last uh last month that that you know your company participated in and uh those are so important to what
[14:20] we have getting people to understand and speak to each other. Um you know and we've realized that you know those kinds of things we need to run them all the time. uh when it comes to the things
[14:32] that make people's people when it comes to the things that make people's lives easier, they want to lean in, right? Um we've got a lot of people that are really that are really passionate about
[14:44] data quality and observability, right? That that's actually a pretty easy thing for us to sell. uh we've had people in governance come to the to the governance office uh and that like the the quality
[14:57] tools all they want and so so we have to temper that I think you can't you you can't do that but you know the thing that we have to do is we have to help them to understand that
[15:12] uh that they are in control of what gets set there right that there are basic things that we have to do in data quality that have to be set. Um, but then the sky's is really the limit in
[15:24] terms of what they want to monitor and how they see it, right? And so, uh, you know, we're we're trying to hold events that teach everyone to to fish when it comes to, uh, things like observability
[15:38] agents, things like that, because that that's that's been a big one for us recently is making sure that we don't get into alert fatigue, okay? uh when it comes to to data quality and you know I I I think certainly that's
[15:51] been a uh a really good data point for Monte Carlo and how you you know how you and your company have really helped us to educate the people.
[16:04] >> It's probably the number one fear that folks have is that they'll be you know bombarded with alerts and ultimately have alert fatigue. So I think you're sort of addressing, you know, the elephant in the room, you know, head on,
[16:16] >> right? You know, that that's that's the thing we have. You know, our our observability strategy is is getting quicker and quicker by the day. We're we're we're tying all of the alerting
[16:28] that we have for out of the box monitoring to digital data contracts, right? And and what that's going to do is that pushes our data literacy strategy, right? And so that becomes the important thing for us to do. We have to
[16:40] make sure that we're training people and that they understand what's coming in terms of alerts and then with you know uh how the AI functionality of what we have can make people's lives easier and
[16:52] like you know the last time we ran an event like that we had a full room of people that were you know engineers and analysts alike that were ready to take in the information. >> Yeah. And so that feels good because it means that people really understand,
[17:05] right? And and and we've gone uh you know, we we've gone from a push to a pull, right? >> Yeah. I love that. And you know, you're dealing with um you know, customer loyalty data and people's, you know,
[17:18] flight information. And so, you know, obviously the the use cases for data are critically important for people. And so I can understand why folks are passionate about that. Um which is awesome. Um maybe one or two more
[17:30] questions. Um, you know, since you're monitoring sort of this important data like loyalty account transactions, check-in data in production today. >> Mhm. >> You know, the next frontier here is AI
[17:42] agents that might one day soon act on that data, >> right? >> What changes when that happens? >> Listen, so you know that's that's where we are right now. >> Okay. We whenever we started to talk
[17:55] about data governance and the foundation of certified data assets, AI was always in mind. We knew that it was uncharted territory for us and know and I I I work for a fairly large company and so we
[18:08] have AI leadership that's in place now. And so what we did at the time was we worked to set them up for success because we knew that they were they they
[18:20] weren't quite here when we started. They're here now. And you know, it was great. It was a great day because we we had this strategy that we put together. We had tears of certification that we put together. Uh and and when the
[18:33] leadership finally showed up, we were able to sit with them and say, you know, we're so glad you're here. Here's the strategy. We are setting you up to do everything that you need to do with AI governance. We want to be partners with
[18:46] you. And you know the the partnership with them has been great but that we we were very intentional about making data trusted so that they could go and not have to worry about that and worry worry
[18:58] about the core items of AI governance. Right? It's still a work in progress. >> But uh I I I think I think that's the thing that we've been the happiest about
[19:10] was that you know we set them up to continue on with that journey. And so we do it we we do it with linked arms now and and know and certainly as we
[19:22] >> start to work through or we we've been working for several months now time flies uh with things like semantic layer and the directed graphs with the you know with all of the knowledge the context that's going to make AI work. We
[19:35] know that we're doing the right thing and it's because we did it that way. >> Amazing. So let's close with you know people here in the audience um who might have been in sort of similar situation being asked to stand up a data
[19:48] governance or you know soon an AI governance program before their organization is actually deployed in scale. They have urgency from leadership, but they probably have, you know, very skeptical engineers. Um, they
[20:02] have a business that wants AI yesterday, but with no budget and deliver it right now, right? >> Um, what do you what advice do you have? What do you wish you had known when you started this journey?
[20:15] >> I I think it's very important to make as many friends as you possibly can, >> right? Let's let's start with that. Inside the company, outside your company, have as many conversations you
[20:28] as as you possibly can. Talk to those people not not just with the knowledge of analysis or data engineering, but like I said, the people with the data acumen to
[20:40] actually drive results. Keep those people very, very close to you. That's the first thing. It's very nice if you have executive leadership buyin, right?
[20:52] If you don't, then I think it's very important to have uh an ROI leader project for that. I think that's something that helped us quite a bit and have some intentionality
[21:05] about it. Something that your business really really wants, right? And then lead through that and execute. That was that was very important for us to have that because it created a head of steam
[21:17] for us where it turned that you know that conversation from hey you really should be stewarding your data different and we're here to help to you know why can't you talk to me right now I need my
[21:29] data certified right which is you know it's it's hard to believe you know 18 months in that you know now that that's a conversation we have every day >> right um I think it's important to make
[21:41] friends I think it's important important to get good advice. Uh you know I you know definitely get help if you need it. Uh you know create create the basic
[21:53] blocking and tackling the operating models that you need to get people together. Um put your customers first. Um let the technical geniuses at your company do what they're going to do
[22:06] right. um you know but but but create that sense of we that is going to make sure that you have the right thing at the right time. I I think that's that's really the best advice that I can give
[22:19] right now. Certainly it's a work in progress for us. Uh but those are those are the things that I would absolutely do if I started again. >> I love that. I think that's very powerful and you know in the age of AI
[22:31] make sure to make friends. I think that's a great takeaway. Um, Andrew, I can I think I can be asking you questions for a couple more hours now. Uh, sadly, we only have 20 minutes, but thank you so much. I really appreciate
[22:43] and congratulations on everything. Such an exciting time. >> Always a pleasure, Bar. >> Thank you.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.