Sponsored: Informatica | Accelerating Trusted Data for AI: Informatica’s Enterprise Data Management

Summary

Informatica delivers data management agents and an agentic workflow framework that accelerates deploying agents on the Databricks Data and AI platform by unifying enterprise data assets with validation, governance, and business context.
This video demonstrates ClareCopilot generating a CDC data mapping from natural language, claiming a 20x productivity improvement over manual data mapping, and shows MDM data and address validation running inside Genie Code as headless Agent Bricks tools.
Informatica's IDMC platform integrates directly with Unity Catalog across all its services through 300+ connectors and over 50,000 integration combinations, and offers Informatica-managed MCP servers for governance and master data management in agentic workflows.

Chapters

FAQs

What is Informatica's role in accelerating AI agent deployment on Databricks?

Informatica provides data management agents and an agentic workflow framework that helps operationalize agents built on Agent Bricks on the Databricks Data and AI platform. It unifies enterprise data assets into a 360-degree view that is validated, governed, and enriched with business context so agents can be deployed faster with no data left behind.

How does Informatica IDMC integrate with Unity Catalog?

Informatica's IDMC platform offers direct Unity Catalog integration across all of its services, allowing metadata and governed assets to flow between Informatica and Databricks. This federation enables centralized governance while preserving the full context of enterprise data assets within the Databricks Data and AI platform.

What productivity improvement does Informatica claim for data mapping?

According to this video, Informatica claims a 20x productivity improvement over manual data mapping when using ClareCopilot to generate CDC mappings from natural language instructions. This is demonstrated live in the session as part of the IDMC demo.

What is headless data management in the context of Agent Bricks?

Headless data management refers to Informatica services such as MDM and address validation being exposed as tools that Agent Bricks agents can call directly, without a human operating an Informatica UI. In this video, this is shown as MDM data and address validation executing inside Genie Code as on-demand agent tools.

Full transcript

[00:10] Thanks for everyone for joining. Um my name is Shannon Eustus. I'm from Informatica. I've been with Informatica now over 20 years. Uh mainly in the data integration space. Uh but now uh we're having to wear multiple hats inside this world, right? I got to be a data
[00:22] engineer. I got to be a data steward. I got to be a data governance person. I got to be an MDM person. I got to do all wear all these different hats. So, it's fun times uh today. So, the topic that
[00:34] we're going to talk about today is trusted data um inside of this world of the agentic enterprise, right? And it just doesn't happen, right? It's a journey that we have to go through, right? We can't just take a agent,
[00:46] deploy it, and expect it to be perfect, right? We got to go through a process of getting data for the agent to be able to display. We got to make sure that it's trustworthy. It's got the right data quality. It's governed. Uh the right
[00:58] agent needs to have access to the right data. So, we're going to And all this doesn't just magically happen when we snap our fingers, right? So, um it's kind of chaos right now, right?
[01:10] I'm more mentally tired working with Claude every single day. I thought AI was supposed to be this magical panacea that everything that's going to happen is going to be magic. And now at the end of the day, I'm like, I just want to go
[01:22] to sleep. So I I see the smiles. Everybody knows what I'm talking about, right? So why is it? Um I think it really kind of comes down to trust, right? Uh data is fragmented. I got to pull it in from a variety of different
[01:34] systems. I'm not really sure on the quality. How many times have you wrote in a cloud prompt? How did did you really do what I asked you to do? Right? And it's like, yeah, no. Um, so this is
[01:47] what kind of causes that chaos and that mental drain, right? But what happens if the world was not like that, right? If we had a unified data platform with unified data, metadata, everything was trusted and governed and I can get
[02:00] insights immediately, life would be so much easier at that particular point in time. I wouldn't spend as much brain power as I as I am right now doing this, right? And why is that? is because for
[02:12] the longest time we always talked about data silos. How do I get data from all these different uh systems to uh to connect them now we have metadata silos as well right? Right? I have context over here.
[02:24] I have context over there. What's right? Which one am I supposed to be using? Right? So with Informatica, what we're pl you know what we're doing is we're providing that connected enterprise uh context layer. So now
[02:37] where things are heavily fragmented and kind of just weaving themselves all over the place, we can get that into a nice understandable ontology at that point and really just provide the context to the enterprise.
[02:49] Now, how does Informatica solve some of the critical data challenges for AI? Well, the first is being able to understand where is my data at inside the enterprise. Uh, that's the first thing. Once I have that understanding of
[03:01] where the data is and what type of data it is, I can then ingest that data, I can then cleanse it using our data quality capability. I can secure it, master it, and govern it. And this is kind of that life cycle that we see with
[03:13] data uh at that point. And now Informatica can solve all of this inside of one platform. You're not having to go out and have a point vendor for one solution, another point vendor for another solution, another point vendor for another solution. Think of how many
[03:26] vendors we have out here, right? No vendor left behind. Who wants to integrate all this together? Informatica provides that one solution to allow you to do that on top of data bricks to solve both analytical use cases as well
[03:38] as AI use cases. Now, we do this through our IDMC uh intelligent data management cloud. Um, and we do it for a variety of different use case or uh roles. Whether or not
[03:50] you're a data producer, a data consumer, a data owner, all of that um can be delivered through our system intelligence, our context engine. We call it um our catalog per se. And then
[04:04] from there, on top of that, we have purpose-built solutions to help for any type of data management. They're accelerated by our co-pilots and our agents. We'll go into a demo here shortly. And then ultimately that
[04:17] provides you the insights that you're looking for. Now whether or not you're wanting to do this on AWS, Azure, Onprim, GCP, Oracle, all of that, all the hyperscalers are supported um from
[04:30] Informatica to provide you kind of the flexibility that you're looking for. Now, let's kind of go into intraoperability real quick. So, the first kind of thing is once I, you know,
[04:42] I have to understand where my data is and I got to be able to bring it in. Informatica has over 300 plus different connectors at that point to allow you to be able to have that reach that you're looking for to bring data into data bricks. Now, someone did the math. I
[04:55] wasn't the guy that did it, but that gives you over 50,000 different types of combinations in terms of being able to bring data into your environment. Right? Today it might be data bricks, tomorrow
[05:07] it might be something else, but now you're futureproofed in terms of connectivity with Informatica. We have a variety of different scanners uh at our disposal. Uh our uh main uh
[05:20] scanner today is bringing everything into the SQL warehouse. It allows us to do things like SQL push down. We can use cortex as a part of SQL uh not cortex sorry uh the datab bricks AI functions
[05:32] to be able to uh push down all that ETL all the AI functions that hey data bricks has inside of their environment into the mappings uh in real time. What we can also do is connect over to lakebase both from a CDC perspective uh
[05:46] as well as a transformation perspective if I'm doing things more batch at that point. So um from a scanner perspective we call a scanner think of a scanner as kind of like an external connection or external catalog inside of data bricks.
[06:00] So I want to go out and I want to connect over to Salesforce. I have a scanner for Salesforce. I have a scanner for um uh Tableau. I have a scanner for data bricks and unity catalog. There's over 120 different scanners that we have
[06:14] access to to bring data into uh into our catalog at that point. Uh let me skip by that real quick. So since we're talking about the catalog, I'm going a little bit quick because I got a lot of demos
[06:26] go through. Um we have direct integration with Unity catalog. Every one of our services is Unity catalog validated at that point. So you have that trust that Informatica
[06:39] can integrate directly with Unity. Now from here we like to think of ourselves not as a competitor to Unity but as a citizen that sits adjacent to Unity at that point, right? Because everybody's got their metadata catalog and we want
[06:53] to be able to cat be the catalog of cataloges to provide you that enterprise context um out inside of your enterprise. And that could be data that is sitting inside of us. It could be data that's
[07:05] sitting inside of Unity. It could be a competitor of ours um at that point, right? We want to be able to provide that context to you to allow the agents to know what they need to know at that particular point in time, make the right
[07:17] insights and right decisions based on that context. Now, let me go through a couple of different demonstrations uh at this point to show how we can accelerate uh data ingestion, data development uh and
[07:31] as well as data quality development. Okay. So, let me switch gears real quick. All right. Escape. All right. So, the first demonstration that I'm going to go
[07:44] through at this point is introduce um our clear co-pilot capability for data integration. So, what I've done here is I'm a data analyst. I might be a data
[07:56] steward. I might be a data engineer. And I want to take data from Oracle that's on prim. I want to do CDC and I want to bring that over into data bricks. Now to do that I simply create
[08:09] a prompt like this want all natural language. I just want to do data ingestion do an incremental load source from Oracle right over to data bricks. Here's the asset that I
[08:21] want uh you to connect to using from a connection perspective. Here's my schema for the source and target. here's the environment that I want you to run on and here's all the tables or I want you to bring all the tables over. So in a matter of seconds,
[08:34] we plug that into Clare Copilot, right? Just straight copy and paste. Claire's then going to analyze the intent to figure out what's the best way of doing that inside of Informatica. Uh in this case, it's you know, hey, we're trying
[08:46] to do CDC. There's no transformation that they supplied. So Claire was going based on the input that you're telling me. Uh ingestion job is the best way of doing it. Then it analyzed everything
[08:59] else to see if there was any additional questions that it needed to have answered to truly generate that job. So here it analyzed everything to make sure that everything was valid, understood what we wanted and made the
[09:11] recommendation to use a database ingestion replication job to do that. So then it gave me that and said, "All right, do you want me to create this asset?" And I said, "Yes, let's do it." And over here on the left, you see
[09:23] exactly that right there. So in a matter of seconds I've now imported in u started the ingestion of Oracle data over into datab bricks. Now we can do this for applications. We can do this
[09:36] for files. We can also do this for databases. Whatever our PM supports allows us to do that in a quick fashion. Right? So now I'm speeding up ingestion at that point. I may not know Informatica but through Clare I can
[09:48] quickly get that job taken care of. All right. So I was working on the other one. uh time ran out before the clock started going. So I want to go in in this case I want to do uh some ETL work.
[10:02] So the first thing that I did was I created a stub mapping and I'm doing this on purpose instead of oneshotting the prompt so you guys can see all the capability that we have. But in the first prompt what I said was I want to
[10:15] create a sample um a stub mapping of that. I'm going to come back over here to clear and I'm going to create a go home and you guys are seeing this live.
[10:28] So, uh, new yes and and we'll paste. So, what it's going to do is it's going to create a sample uh mapping for me at
[10:41] this particular point in time with just the source and target information that I specified inside of the prompt. So, we'll give it a second.
[10:59] Now when uh while we're waiting on this, I've been developing mappings for over 20 years. All informatica mappings now. You think I would know every syntax, every function, every format of the
[11:11] data. I don't, right? You know, I'm not in it every single day. I kind of timed myself with co-pilot in this approach that we're talking about. I'm now 20 times faster than I was
[11:23] before. Right? That's a 20x uplift in terms of productivity. And the reason I'm doing it is because Claire and the LLMs are helping me out and the agents are helping me helping me out behind the scenes and I'm not having to go back and
[11:36] remember all the syntax or how do I take this Python function and convert that over to Informatica? what is the SQL statement doing and how do I convert that over to Informatica? The LLM are helping me do that behind the scenes.
[11:49] So, let's go ahead and say yes, we want to create this
[12:02] and it'll generate my sub mapping and I'll go ahead and key up the next uh prompt. So, this next prompt I'm getting into more transformation logic. Now, in this situation, I've just put everything inside of a a simple expression because,
[12:14] you know, demos. But what you can do is what if I have a markdown file for a source to target mapping? What if I have SQL statement? What if I have Python? What if I had someone just write something out on a napkin, right? I can
[12:26] take that logic, plug it into the next step, and it will automatically build what I need based on the intent for what was provided. So, in this case, we're going to copy um everything. Make sure I got it all.
[12:40] There we go. Ctrl + C. And we're going to bring it back over. And here, if I click right there, and I go to generate transformations,
[12:53] it'll say, "All right, well, what do you want?" And we're just going to paste a prompt in and hit generate at this point. Now, from there, it's going to look at what I what I specified. And I can have more than one transformation. and I've
[13:06] done probably 15 to 20 different transformations inside of here and had it automatically build this out. The more the complex of the logic, the better your efficiency and your productivity is going to go up at that particular point in time versus what I'm
[13:19] what I've seen because I have the LM doing a lot more. So, let me just clean this up a little bit. And what you can see at this point is now I have all my logic already built out at that particular point in time. So, now I'm no
[13:33] longer having to figure out how to do it. I can take that source to target mapping, build the pipeline out automatically. Now, this is kind of that first step that gets you that 20x. When we were
[13:45] first talking with customers, they were like, look, I just need a 50% uplift in terms of productivity. I just gave you a 20% up, you know, 20x improvement right there. Well, then headless came and open claw
[13:59] and clawed and now the game and the goalpost has shifted and now I got to be even more productive, you know, on top of that. So, what we've done is at Informatica World this year,
[14:11] we announced that we are going headless from a headless data management perspective. What that means is we're releasing a lot of purpose-driven MCPs for only data management uh
[14:23] responsibilities at that point. They could be tools, they could be skills, they could be agents, uh they could be MCPs by themselves. And this is an example of that working together directly inside of Code Genie. What
[14:35] we've done at this particular point in time is you if you go out to the marketplace, uh the data bricks marketplace, you can go in and install the various different MCPs that we have available. There's five today. we're going to have um probably about 20 or so
[14:48] more that are going to be released about the October time frame that you can go in and add those in. So in this example, what we've done is I'm in Code Genie and I've connected over to our MDM MCP at
[15:04] this particular point in time. Now I can bring MDM data directly inside of Code Genie. I can bring our catalog data directly inside of Code Genie and interact with it. So here um I've said hey inside the informatica environment
[15:18] can you provide me information around guest information around Clark Griswell. Our demo was set up around a theme park. We went wild with it. So Clark is kind of our hero uh in this point. So what it
[15:30] did was it said yeah sure I'll go out. I'll find all that information and it'll supply that right back to you at this particular point in time. So from there we can also see any head of household holding. And I can see Ellen is the
[15:42] wife. Audrey and Russ Russ are the sons. But then I can also do other things like this. It brought back his address. Well, what if I want to go out and I want to
[15:54] mail information to him. Is that address actually correct? Informatica has been doing address validation for over uh almost two decades at this particular point in time. So here I can go out
[16:06] analyze that and I can say can you check to see if Clark's address is actually correct. So it'll go out do all of that through the MCP return the results do any enrichment in this case uh I think
[16:18] we only had like zip five. It went ahead and did the zip 9 at that point or 5 plus 4. Yeah. Uh at that point and brought it back. Well I went ahead and said all right well we got all of this
[16:30] information around Clark. Let's talk. Let's see if the same information is valid between Rusty and Audrey. So, it went out and was like, here's the information that I know at this point. Everything's consolidated. This is our
[16:42] golden record, but we see that we have bad zip codes or different zip codes for Russ and Audrey. So, I went in, I said, "All right, are the is that actually correct?" And it came back and was like, "No, that's not correct. Here's the
[16:55] actual values that we need." So now I can come back and you know it's asking me do you want to update this value inside of MDM. So this is how through code genie you can uh start working through uh uh with Informatica and
[17:09] you're not even realizing that you're working with Informatica at that particular point in time. So it's really what we're seeing in this world of MCPs is you're just kind of talking the data. You're not even realizing in some cases where the data is coming from uh at that
[17:22] particular point in time. all you're really focused on is just getting the job done at hand at that point and you know that's kind of the the goal that you know data bricks is looking for is that opened interchange of data and
[17:35] ideas right so with MCPS with code genie with Informatica you're able to do that you're able to get the trust that you're looking for so that is kind of it from a demo perspective let me switch back over
[17:47] to my presentation and we'll just wrap up at that point all All right. So go here. Here's kind of the innovation that we brought over the past four or five
[18:00] years. We've moved on from rag retrieval. We've talked about co-pilots. Now we're in this aenic enterprise. Uh we kind of went over the MCPs at that point. Here's kind of the rough
[18:13] architecture diagram. Yeah. Go ahead and get your cameras out at this particular point in time. So this is as of today. Uh as I said in October we'll have a lot more MCPS uh deployed and available for
[18:26] you to interact with on the marketplace. And I kind of did the demo and here's a QR code for anybody that wants to kind of take a further deeper dive at that point. So with a minute to
[18:39] spare. All right.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.