Extending Unity Catalog to Govern the Data Estate With Informatica Cloud Data Governance & Catalog

Summary

• Unity Catalog governance: Informatica Cloud Data Governance & Catalog (CDGC) integrates with Unity Catalog metadata to provide end-to-end governance and security across enterprise data landscapes.
• Comprehensive data lineage: The integration delivers complete data transformation visibility across multicloud and hybrid environments, supporting holistic cataloging and centralized governance frameworks.
• Data quality assurance: Rich data quality capabilities ensure data is cleansed, validated and trusted for analytics and AI, with centralized access policies and stewardship for regulatory compliance.

Extending Unity Catalog to Govern the Data Estate With Informatica Cloud Data Governance & Catalog

Talk by Ajay GOLLAPALLI, Director, Informatica (Sponsor Speaker)
Join this 20-minute session to learn how Informatica CDGC integrates with and leverages Unity Catalog metadata to provide end-to-end governance and security across an enterprise data landscape. Topics covered will include: Comprehensive data lineage that provides complete data transformation visibility across multicloud and hybrid environments -Broad data source support to facilitate holistic cataloging and a centralized governance framework Centralized access policy management and data stewardship to enable compliance with regulatory standards Rich data quality to ensure data is cleansed, validated and trusted for analytics and AI
Here’s more to explore:
Unified and open governance for data and AI: https://www.databricks.com/product/unity-catalog
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements

FAQs

How does Informatica integrate with Unity Catalog?

Informatica Cloud Data Governance & Catalog leverages Unity Catalog metadata to extend governance capabilities across an enterprise data landscape, providing centralized control and visibility.

What is data lineage and why does it matter?

Data lineage shows the complete path of data transformations across systems, enabling organizations to understand how data flows through their infrastructure and meets compliance requirements.

Which data governance platforms provide comprehensive data lineage?

Informatica's Cloud Data Governance integration with Unity Catalog enables organizations to track data lineage from source systems through transformations to final analytics dashboards, providing complete visibility into data dependencies and quality. The combined platform captures both technical lineage showing data movement and business lineage connecting data to business outcomes. This comprehensive lineage capability helps compliance teams, data stewards, and analysts understand data provenance and impact.

What integration platforms support enterprise data governance?

Enterprise data integration platforms like Informatica need to connect with modern data catalogs like Unity Catalog to enforce governance policies across data movement and consumption. These platforms should support discovery of sensitive data, tracking lineage through complex ETL processes, and preventing unauthorized access or movement. Integration enables organizations to maintain consistent governance standards across legacy systems and modern cloud data platforms.

What observability tools help organizations monitor AI systems in production?

AI observability platforms should track model performance, data quality, and data usage to ensure AI systems remain accurate and compliant as data changes over time. These tools often integrate with data catalogs and governance systems to understand what data AI systems access and how that data flows through business processes. Comprehensive observability enables organizations to detect issues like data drift, bias amplification, or unauthorized data access in AI applications.

Full transcript

Hi all. Uh I am AJ Galapali from Informatica. I run the entire ecosystem partnerships and I have from data bricks. I partner in crime. Hi folks. Thanks for joining us. Uh so. I uh I'm part of the ISV partner team here at data bricks. So, how many of you is like first data in AI summit? Ah, interesting. Good. Awesome. Glad to have you with us. Appreciate your uh the fact that you're joining us today. So, I have to give you we'll talk about Unity catalog and how Informatica integrates with uh data bricks and that modern stack.
A little forwardlooking statement. uh know legally we uh we basically have to share this with you. Whatever is being said here uh is you know a forward-looking statement. So uh you can take a snapshot if you want. There you go. All right. And uh so I'm not Padep. I'm I'm here to replace Padep. Praep is uh also a a peer of mine. We work on the same team. So uh happy to be here and uh stepping in for Padep. he's he's got some uh he had some some is you know some additional conflicts.
So what we'll talk about today is the data bricks and Informatica partnership. We have a very uh healthy and strong partnership. How do we bring value using Informatica in datab bricks uh and then AJ here is going to do some demos and then we'll have some Q&As's. All right. All right. So from a partnership perspective hopefully some of you attended uh Informatica world about a month ago. There was a lot of great uh content that was shared about the Aentic world and how Informatica is going to you know be managing all these agentic workflows.
Uh Informatica is a really strong company. We have a very great partnership. They're selected tiers of ours. So really high in uh in our tiering and category system. So last year they won the partner of the year. Uh kind of got closed this year, right? But uh exactly and we have hundreds of joint customers using data bricks and Informatica as well. Any any of any of you using IDCM right now? There you go. That's great.
So that's good. I'm assuming the rest of you guys are power center users. Either or we have a great integration point. Um and then uh you'll see in a minute here. All right. So this is kind of the landscape. If you look at the left side of this screen, these are all the informat Informatica components that are fully integrated with the data intelligence platform. So there's a very great integration from a Unity catalog perspective providing the governance between your Informatica components and data bricks.
Obviously uh all that starts with uh the uniform data lake foundation the integration with your data formats uh and there's a full stack of integration fully integrated from Informatica and data bricks AJ thank you so much uh for introduction introducing the IDMC to many of us here. Yeah. One question. How many of you all work on data? Just a show of hands. I think everybody here works on data. Everybody works on data.
Doesn't matter what your role is. Are you a data owner, data steward? Are you a data integration engineer or a data scientist? It doesn't matter what your title says, but end of the day, we all work on data. So when you look at from an Informatica perspective, the way we look at data, especially on the crowd data, governance is mainly three pillars. Governance provides the risk and compliance. Every catalog in the world communicates says the same thing that we provide risk and compliance because that is the essence of catalog.
They have to provide risk and compliance. But Informatica looks beyond risk and compliance. The enterprise requires a common data quality observ observability within the data data catalog itself. So we look at as a second pillar called as a data quality within the data governance and this data quality provides us how much the data is complete is the data correct set of data for doing your data science workloads. Is the data even skewed to do like for example if you're doing a model and if you're having an area under the curve is my data providing a 0.
5 or a 0. 91 it changes various factors especially when you have your data cleansed readily available. The third we also look at from an analytical point of view is data sharing. Many of us have this trouble. We go to systems, we go into uh let's say for example, we go into every single environment and then say I want data from SQL. I want data from SAS solution like Salesforce. I I want data from multiple uh locations.
But nobody says actually customer says I want customer data, I want orders data, I want my invoice data. That's what customers ask for. Am I audible? Okay. Thank you. So when the customer says these are the important things that they need, they don't care whether the data is stored in Oracle or whether data is stored in SQL or whether data is in Salesforce, whether data is in any environment. They don't care about it.
So that is where Informatica adds a lot of value in terms of saying follow the data object model rather than following the system object model follow the data object model where we can provide access controls and where we can provide data quality and data cataloging experience on the governance portfolio in an enterprise. That's why Informatica looks at it. We did not do it all by just the data governance alone. We saw the product side of it, a list of seven applications and seven services.
We also built a AI engine called CLA. And if you go back to in 2016 videos, you will see a lot more how CLA was introduced. It was introduced as the next big geeky thing at that time. But now it became common across many places because CLA is changed the data the way people look at data. is the AI governance behind this modern data governance. It provides various algorithms behind it. There are machine learning algorithms to looking at textbased analytics, reinforcement learnings, many of the classification, regression model and NLP to SQL SQL to NLPs.
Everything happens within CLA. The way CLA works is when you ask a question, CLA knows what your intent is and based upon the intent, it provides an answer. It is not a just a chat box or a chat window, but it provides a lot more behind the scenes. It it provides a front end for your catalog. So let's take a look at the demo. Don't believe whatever I say or just look at the demo. So when you look at the demo right in a CLA what we introduced this year called CLA GPT what it does it you can ask any question within the CLA on your data catalog.
You can say I I want to know what is my let's say we are in California. Let's say talk about California consumer privacy acts. I don't know how many you know of it. I I was not aware of it until last week when I started putting this demo. And when I want to know in my enterprise if I have any policies related to CCPA that's California consumer privacy act. It's a mouthful. And what it says is it provides me the organization policies that I as an enterprise need to take consider this policy for my organization.
When I consider this policy, I'm also need to know like this is too much information. This is not how we commonly talk in your enterprise. This is not how I talk. When I talk about it, I say like, hey, what is it? Is it a email ID? Is it a username? Is it a first name? That's the kind of things we notice. So, we want to know what is our enterprise glossery looks like. So our next question will be like most often tell me show me a preview of the policy first and then when you can see the policies you'll be able to see what type of a policy it is and it also provides a glo quality of it.
It provides is it like how good as an organization data related to this policy that we have. Is it a high quality? Is it complete? Is it the right set of data for doing my machine learning models? And I can also ask about the glossaryy terms. It provides the glossaryy terms saying what are the kind of business terms are related to this policy. For example, it says my first name, last name, email address. All those things come under the CLA policies.
And you also can talk chat with this and say like what datab bricks fields are related to this policy because end of the day we are looking at a system what is stored. If you want curious about as a data owner, I want to know what data elements are related to this policy inside the database environment and we can find out what are the columns and we can also find out what is this columns aligned with what is the columns how the columns quality looking like the value distribution and we can also look at beyond that it's saying like which table all these columns found in now we saw normally we go on hierarchy we go from a catalog, schema, table, column names rather I'm going from tell me what columns are those what tables do contain these columns and it provides you what kind of a column it is within the entire environment and what are the related fields for this column itself.
So this CLA GPT provides an entry point for all the data owners making their lives very simple so that they can access the elements within the within the CLA itself. But guess what? We need more about information, right? I said this about cataloges. Why am I talking about this? This is a front end for the catalog. But let's take a look at the back end too. And Dennis, do you want to talk about Unity catalog? So what is Unity catalog, right?
Everybody knows, everybody knows Unity catalog. We always been talking about it in last three uh databicks summits. Unity catalog provides a unified governance for data and AI. It helps enterprises address the four key challenges if you look about it. Unified visibility into data and AI and also provides a kind of governance within the datab bricks environment. It provides the access control policies within the datab bricks environment and it contains the data itself.
If you think about all that thing about catalog, it provides a catalog, a schema, a table name and the actual data is within the catalog itself. That's the main purpose of unity catalog. The way Informatica extends the catalog is by consuming the metadata for the enterprise. So, Informatica takes a enterprise catalog layer. It provides a catalog of catalogs behavior where you have many many catalogs. If you have especially if you have a um uh different kind of services if you're in Azure or AWS you have your own cataloges and then you have unity catalog and then if you have the other u data warehouses in your environments you'll have their own catalogs.
So combining these all these cataloges into one place providing ability to discover data providing semantic intelligence providing glosseries providing root cause analysis impact analysis the relationship between the objects business context across multiple systems combined together. All of the services gets built in with Informatica's cloud data governance catalog. That is the purpose of this catalog of catalogs experience and many many thousands of customers utilize it today to achieve their outcome in the enterprise mechanisms while just like take a look at the demo itself and see like what does this catalog thingy provides us.
So if you look at from a Informatica perspective, Informatica data governance and catalog is one of the chicklets that's on the homepage of the data informatica homepage and it provides you with a dashboard where you can search for data terms all that what we did in CLA GPT. We're going to do almost similar things but now from a data owner perspective not from a data consumer from a data owner perspective. thinking about like how do I look at these terms?
What I showed you earlier was the same experience in a more of a natural language processing. This is more of a technical terms or business terms and glossery terms and search and look up the data itself. It provides a data quality scorecards, data dashboards and lineage items. You can search for a particular term within the datab bricks ecosystem. It's all jointed together with the Unity catalogs, REST API and information schema being consumed within the Informatica's enterprise data catalog.
So this provides for example the various terminologies and we can also see the lineage in a more natural way of looking at saying where is my data journey is the data coming from my enterprise in an Oracle system moving into ADLS moving into data bricks and I'm making the code change within the databicks notebooks and how I move it back into datab bricks in the silver layer and the golden layer all this information it's practically every organization does this today so when you look at the linear age you'll be able to see this where is the data coming from and where how is it being stored in data bricks how is being consumed in your applications and now they have coined a new term called platinum layer where you have visualizing the analytics data with powerbi reports or genie reports everything where you can look at it you can take a look of the entire hierarchy of data itself you can look at from end to end perspective that's a Uber goal of taking saying my data So when it came to the system at 60% quality and I'm doing my analytics I want to be at 90% quality or 100% quality that is the whole lineage that we can see within the data bricks within the Informatica's enterprise catalog experience and not only that you can also see not only the columns and the tables and the fields but also add on top of it what was the quality journey of the data.
So the data journey is quality. If I go back and see like I want to add the different quality layers whether it's aggregated quality or whether I want to look at individual accuracy validity of it or I can look at certifications or business objectives. You can see the journey of the data along with how it has been cured over the period of time. And I can tell you this when you go back to the booth area to see a demo they'll be able to show you that we are the only catalog company that catalog services provider that can scan the notebooks in databicks environment and tell you how the data gets transformed not from source to destination but within the notebook which part of the code component changes the data's journey that is also scanned and provided in the lineage within the data within the enterprise catalog environment And you can ask me why do I need this?
If you do a root cause analysis or impact analysis, uh as a data engineer when I started my journey, it's not an easy process. I have to go back and take a look at my data and then say what is which part of the code is making the change to what part of code what part of the data set data element itself. So I need to do all this analysis. It helps me to make sure that there is redundancy, there is resiliency on the data.
So with this what I'm going to show you is the dashboard which contains all this information where you can be the starting point for the entire enterprise data governance catalog. And what we also did we didn't stop there we put across the blueprint for the enterprise grade geni application. So if if an organization thinking about generative AI, how do I use this catalog? What make me thinking was everybody talks about generative AI?
Take a data set, put a chatbot in front of it, you're generative AI, right? But what will happen if you have multiple systems in play and you want single chatbot for example let's say you think about a supply chain supply chain bot a supply chain co-pilot where the data can to come from salesforce with sales orders where there is data coming from master data management for all parts and components where there is analytical data sitting in data bricks that has to be integrated so what we did is we created this metadata intelligence which will provide metadata so you're querying against the metadata uh rather than the data.
So the metadata goes to the rack chain and then able to go through the entire data journey process for generative AI and able to fetch and provide the Q&A between the chat interface itself. So with this um as I said three important things we talk about governance, democratization and observability. And uh if you're going to attend my next sessions, there are a couple of other sessions. We are talking about how the entire Informatica IDMC is going to talk about what how does it work with Unity data bricks.
And we also talking about tomorrow with the deote how massive data management is providing our customers a eventually a uh golden record on databicks environment. Do try to attend the sessions if you can and thank you for listening to us. Any questions that I can take?

Learn more about the Databricks Data and AI platform.