Sponsored by: Informatica | Govern Like You Mean It: Centralized Governance, Security and Compliance

Summary

Informatica's Cloud Data Governance and Catalog integrates with Unity Catalog through metadata federation, enabling centralized governance across the entire enterprise data estate with business context, lineage tracking, and automated data quality rules.
This video covers four pillars of enterprise data governance—compliance, data quality, democratization, and security—and shows how Informatica extends Unity Catalog's metadata to legacy and external systems to provide a unified governance view.
The demo includes automated lineage scanning across Databricks, Oracle, and data mappings, interactive lineage visualization, and policy push-down that dynamically writes Unity Catalog grants, along with personal data compliance monitoring for GDPR and CCPA.

Chapters

FAQs

How does Informatica Cloud Data Governance integrate with Unity Catalog?

Informatica's Cloud Data Governance and Catalog uses a metadata federation architecture that reads Unity Catalog's metadata and enriches it with business context, lineage, and data quality signals from across the enterprise. It can also push governance policies back down to Unity Catalog by writing grants dynamically, keeping access controls synchronized.

What are the four pillars of governance that this video covers?

This video organizes enterprise data governance around four pillars: compliance, data quality, democratization (sharing data with colleagues and business units), and security. Each pillar is shown as an area where Informatica's platform adds enterprise-wide context on top of what Unity Catalog provides within Databricks.

How does Informatica help with GDPR and CCPA compliance monitoring?

Informatica's platform includes personal data compliance monitoring that identifies and tracks personal data across governed assets, providing targeted quality and access views for sensitive fields. This helps organizations meet GDPR and CCPA requirements by making it clear where personal data lives, who can access it, and whether quality standards are being met.

What is policy push-down and how does it work with Unity Catalog grants?

Policy push-down is the capability shown in this video where Informatica writes Unity Catalog grants dynamically based on governance policies defined in the Informatica catalog. Instead of manually managing permissions in Databricks, access controls defined in the enterprise governance layer are propagated automatically to Unity Catalog, keeping both systems consistent.

Full transcript

[00:09] Good morning. Hopefully you can hear me great. I'm pretty loud, but uh but I'll try not to shout today. My timer's going off, so I'm going to get started today. Uh I'm that guy. I'm Scott Hayes. Uh I'm a product specialist
[00:21] at Informatica. We're going to talk about data governance today. So hopefully you're in the right place. So really quickly, I'm going to go over a
[00:34] couple of slides, but I really want to do some showand tell for us today. So the first thing is we want to talk about data versus context. And I feel like if you're here, you kind of already know the importance of governance, but it's
[00:46] worth, you know, mentioning it really quickly that like we've all got lots of data. And what might be missing in our organization is context. What data is related to other data? what data is related to other business concepts like
[01:00] business terms and processes and projects etc. that context, how things are related to each other. And certainly with the uh with the rise of AI, this has only made this even more valuable,
[01:14] right? When it comes to AI, we're adopting LLMs at a just a just a record pace. We're building our own models as well. And and that's that's great. But
[01:26] from a business perspective, what are those things related to? What's the context? What data are these models relying on? Right? What projects are these models uh uh uh related to? Right?
[01:39] We want to try to avoid duplicate efforts. And even just the simple answer, who owns this? Who owns these projects? Right? So this is where obviously governance comes in.
[01:56] We talk about governance. We kind of talk about these four pillars. Compliance is obviously very gerine. Data quality is very much related to it. Again, especially with AI, if you're if you're training your models on bad data,
[02:09] you're going to get bad models. Democratization, which is really the sharing of data. How do I share data with my uh with my colleagues? And of course, security.
[02:26] We threw a slide in there because I get all sorts of questions about Unity catalog. How do we work with data bricks etc. And so this is a bit of an eye chart but ultimately we can certainly read in what Unity catalog provides. Obviously Unity catalog is providing
[02:40] tons of metadata for the uh for the assets within data bricks and we can make that part of a larger uh uh uh platform of that metadata. So besides just Unity, all these legacy systems
[02:53] that we have and pulling in that information as well. And then likewise, we can push things back down to some of these other resources including data bricks. So one thing that's noted there is I can actually build security
[03:06] policies in my governance platform and push it down into datab bricks where it would live in data bricks. So if I was an administrator in datab bricks, I would see that policy. It's not something that has to live elsewhere. I
[03:19] would see oh somebody has granted access to these tables to this group etc. But it could be orchestrated through a larger governance platform.
[03:40] So naturally I'm showing us uh uh Informatica. I'm logging into my uh my Informatica system here. The first thing I'm looking at is my governance uh and and catalog platform here. I landed on a on a landing page.
[03:53] It's just a custom page that I can add as a landing page for my users. I can customize this highly and I can even decide who gets to see those dashboards as well. And I can have multiple
[04:05] dashboards. Maybe I have one for the security folks. Maybe I have one. In this case, we've uh we've invented a fictitious uh park and this I wanted to focus on that. So, you can see I can customize the heck out of this. But in
[04:18] this use case, the first one I'm going to look at, I'm interested in uh a report, a report that maybe somebody is questioning, hey, what's going on this with this report? So, I could certainly
[04:30] browse or anything like that what I would do in a uh in a uh a catalog, but I'm going to search for it. I know it's something called guest spend. Maybe I can
[04:43] zoom in a little bit so maybe you can read. So of course I can filter things on the left there, but I see it right here. This guest spend summary
[05:00] and pretty typical in any sort of governance. I can do enriched metadata. In this case, I just put a screenshot on there along with my description. Again the concept here is this is for my data consumers. I want to give my data consumers confidence that this is in
[05:12] fact what I meant to look for. Of course a staple of any governance is I want to make sure I understand who the stakeholders are. In this case I can see it's this group called technical
[05:24] governance technical owners. And you know if I want to I can certainly drill down and see exactly who that is. Unfortunately I'm supposed to be the owner of this thing. So, I'm the one that's supposed to know.
[05:36] Let's dig in a little deeper. Of course, certainly when I'm looking at BI reports, a great one to look at is I want to look at lineage. So, the lineage that I'm about to show you is all automated. What
[05:49] that means is I scan data bricks, I scanned Oracle, I scan the notebooks within datab bricks. Uh we've got some Informatica mappings we've done. uh I scan those and it automatically drew
[06:02] this diagram for me. Okay. And as you can see this is all interactive. I can zoom in, zoom out, drag it around. I can even export it if I want to export it as an image if I need to give this to
[06:14] management. Uh a feature that our customers really asked for was this tabular format. So if I need to just get it like from a program standpoint, source target, source target, that kind of stuff.
[06:26] But really quick, I'm going to dig in a little bit. This is This is nice. But if you can see, I zoom out. It's a bit of an eye chart. Let's uh let's let's narrow our focus a little bit. Maybe on this report, I'm interested in specific
[06:39] areas. I'm interested in specific elements. It's going to prompt me which elements are you interested in. In my case, I'll say email and full name. Great.
[06:52] Now, it's going to redraw it and it's a a simpler view. Right. Certainly a little easier. I can even do things like, you know, hide the
[07:04] data processes if I want. So these hops in between, like I can go into here and say, oh wait, hide that.
[07:21] So now all it's showing me is I'm not really interested right now in the mappings that move that data there. I just want to know high level where the data is going to and I'll zoom in just cos so we can see and you'll see that
[07:33] the original system was Oracle. So again, this concept of an an overarching data governance platform, I a lot of my business is going to be in data bricks, but I know it's I'm bringing things in
[07:46] from multiple sources and and of course because Informatica has that heritage of ETL, we've got connectivity to all the all sorts of different things including up to, you know, mainframe as well as uh
[07:59] modern stacks as well. Of course, we talked about those different pillars. One of the things I want to bring up is of course data quality. How does data quality fit into all this? Certainly I can look at data
[08:12] quality and see how things are uh are put together. But in this diagram I want to look at data quality within lineage. So you can see here I'm going to zoom
[08:24] in. You can see here now all the assets it's showing me the data quality results for them. And certainly we could go over at another time like how do I build those rules etc. We have LLMs to build
[08:37] those rules. We of course we have a development platform that we've been using for years but uh that you can still use but yeah we have LLMs to build that a lot of automation to uh to help with that data quality
[08:52] >> right so this kind of gives you an idea so like if data quality how has it been flowing through the organization I can kind of see this one thing that's worth mentioning about that data quality the same rules that I build will work on any
[09:04] system so the rule that I build I'm not building it for data brick I'm not building it for Oracle etc. I build a rule and that rule will work regardless Oracle SQL data bricks even things like
[09:18] CSV file and parquet files same rule. Incidentally those same rules I can use within a pipeline within mappings. Maybe I want to build a pipeline that makes the decision if the row is a bad row
[09:32] redirect it somewhere else make the uh you know alert the owner something of that. So those same rules that I can use, I can use elsewhere.
[09:45] I'm going to look at something else here. I'm going to drill down and look at one of these tables for example. You'll notice, by the way, it opened up a tab on the left. So it didn't, you know, I can I can go down rabbit holes if I want and get it won't get lost.
[09:59] This is really what I wanted to show to start with. Here I can see I'm looking at a table and I can see on the upper left exactly where I'm at, what schema, you know, the sort of normal technical stuff that I'm used to seeing. But you
[10:12] can see our AI automatically attempted to to uh assign glossery terms to those columns. My columns are very nicely named. Guest ID is the glossery guest
[10:24] ID, but we all know that's not the case in uh in a lot of our schemas. So our AI built that. And obviously what this uh allows for us is my business users can start to look at business concepts. If I
[10:38] look at one of these business concepts, for example, uh guest loyalty tier is a nice one. You'll notice this is a business concept. Look at the upper left. There's no tables, there's no
[10:51] schemas, there's nothing like that. This is a business concept. It's showing me the ontology of this uh business term. So my business users can see ah this is the guest loyalty tier and now I can start to relate it. You can see here off
[11:04] to the right it's got a list of these are all the technical objects that are related to that business term. So very quickly I can see where is business loyalty term whatever that means.
[11:18] Likewise I can have data quality. How does a business qu uh concept have data quality? Of course, the the the business term is related to the column. The column has data quality. We've
[11:30] aggregated that up. Likewise, I can start to when we talk about context, put things together. I can start to relate this to reference
[11:42] data. Maybe I have reference data. I know exactly what these tiers should be. I could click on here and see what reference data it is related to. And now it's actually pulling this from an
[11:54] actual reference uh uh product within Informatica that says these are them. I can always if I have permission to do so. I could go in here. I can look at the crosswalks for these uh for these as
[12:06] well. I can also build rules that use these reference objects. Since I know which uh values these should be, I can now build a data quality rule. So when I'm looking, which things are these
[12:18] related to? make sense? So, I'm going to switch gears for a second since we talked about AI and AI is very much top of mind for a lot of folks. I'm going to focus on an AI
[12:31] project. Everything we've looked at so far is pretty, you know, uh very dataentric. I'm looking at data bricks. I'm looking at Oracle, etc. I'm going to go back home and I'm going to look at a
[12:43] different dashboard, a dashboard that I built specifically for AI. Now, I built a couple of very simple AI projects because or actually one AI project and I wanted to uh to to sort of show this off
[12:56] here. Okay, so I created a project called peak pulse guest churn prediction. So ultimately my developers made a model to help predict
[13:09] when there's going to be customer turnurn based on the data, right? Something that they built themselves. So as a business user, I want to make sure I'm relating that to other things. So we started with a project. You'll notice,
[13:22] by the way, in this dashboard, I've got data quality already top of mind. This is the data quality that is related to this these AI projects. So we're going
[13:34] to drill down a little bit really quickly. I'm looking at another business concept. It's a project, right? Obviously, projects can be related to
[13:47] lots of things. I can see who the stakeholders are. And in this case, you can see it's got a related it's related to a policy. Of course, we're dealing with personal data. So, we make sure we have to make sure we uh uh uh conform to
[14:00] our policies related to personal data. And I can see it's related to an AI model. Okay. So when I say it's related to an AI model when we scanned data
[14:12] bricks since this is where we built that model I automatically ingested that metadata of that model including these metrics I got this straight from data bricks. So when we think of metadata we
[14:24] think very much of tables and schemas and columns etc. But when I do a metadata scan I'm capturing this as well. Optionally, I can certainly turn that off or filter it to say which ones.
[14:36] But here now, this is giving me exactly what I uh uh want from that from that uh uh from that model. So now again when we talk about that context, what is this
[14:50] model related to? We specifically have a model lineage view. To me, this is the keys of the kingdom. Now this is a very simple simple one
[15:04] because I wanted to build it for illustration purposes. This is telling me exactly what's training this data. Right? This model is being trained using guest data and ticket revenue data. All
[15:16] right. So the next question of course I would want to ask myself is how good is this data? And certainly I can click on a button here and say
[15:31] now I have an idea exactly how good the quality is for that because that's all related to this information that we already have. And certainly I can drill down if I wanted more information. 96.35 may be decent but it's not the best. Let
[15:45] me drill down. I opened up this table now. And if I want to, I can look at that data quality.
[15:58] I can see exactly what's going on with these. So I can see three of them look reasonably well. I see one of them is in a red. I can start to drill down and see what's going on with that. In this case, you can see I've configured that there
[16:10] were 92 bad rows on that one. And I had it automatically isolate those rows and I had it put into a CSV file. So really context that's the sort of key
[16:22] word. I went from an AI project and I was able to very quickly even if I'm not a technical user be able to see which data are we using for this if in that if I wanted to in that same diagram it
[16:34] would show where is that data flowing to not just training but like maybe it's flowing somewhere I would see that as well even as a business user I'll be able to see that.
[16:51] One last thing I'll leave you with. I have three more minutes, but uh we mentioned a little bit about security. I wanted to bring bring up something here. I made another dashboard that focused specifically on personal data.
[17:05] So again, this is just the perspective of where is all of our personal data. You can see I've got a list of glossery here. This is what we mean by personal data. phone, email, date of birth, etc.
[17:19] And very quickly, I can see the data quality for the personal data. In this case, I customize it to just say, just show me the data quality for our personal data. Again, even a business
[17:31] user can go in here and very quickly say, how is our personal data? Uh, there's some problems. There's I see a band of red. I can drill down, see that. And likewise,
[17:43] you can see I created a a widget here that shows the uh the the users with or the objects that contain personal data. So these are all the tables or flat
[17:56] files or parquet files that contain that that uh personal data. And incidentally, all of this can be uh uh uh uh retrieved via API. And even in the user interface,
[18:09] there's an export button. I can export this immediately as a spreadsheet. So if I want to convey this to somebody else, I can certainly convey that to uh to somebody else. Speaking of security, I
[18:21] mentioned that I can do that push down access control. So right in here I can create policies.
[18:36] So here I created a policy and it's a little difficult to see. Let me see if I can zoom in. But I created the policy down the down upper uh lower left there. All with name BRZ. I want to share all
[18:50] of the bronze tables. So this is providing read and write access to that group. Damn USR manager. You don't have to swear about it. Geez. But uh this is providing access to that and it's based
[19:04] on that search. So what that means is is if tomorrow I onboard another bronze table, it's automatically going to be included in that policy and this gets pushed down into data bricks. So we're
[19:17] literally connecting on the data bricks and issuing the commands grant access to that to that uh group etc. So now when I set this up I don't have to go back to the technical users and say oh wait add
[19:30] this table add that table etc. And the power of that search is I've made a simple one just names uh named BRZ but I could also do this to be based on owner
[19:42] anything that has me as a stakeholder or anything that is related to personal data or even custom attributes. If I create a custom attributes that says you know flagged system or anything like
[19:55] that I can build policies based on that as well. All right, I think that's my time. I really wanted to do uh really wanted to highlight that uh data governance and I
[20:07] really wanted to do showand tell. I'm not a big fan of uh PowerPoint slides. I'll be certainly available over here if you have questions. You want to talk to me afterwards and certainly come visit us in the uh Informatica booth. Uh I don't know off that way or next to the
[20:21] Salesforce booth. What are the odds? Thank you.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.