Sponsored by: S&P Global | Leveraging Genie to Remove Friction From Exploratory Data Analysis (EDA)

Summary

This video covers how S&P Global uses Databricks Genie within Capital IQ Workbench — a Databricks white-label environment — to let both technical and business evaluators explore S&P's large and complex financial datasets using natural language, without writing code.
Setting up Genie for EDA involves defining a context layer with table relationships and cardinality, entity disambiguation configurations to map natural language terms to the correct dataset entities, and example queries to guide the model toward accurate responses.
A live demo shows Genie uncovering a data anomaly through conversational follow-up questions, demonstrating how natural language EDA can surface data quality issues and unexpected patterns that traditional dashboards would miss.

Chapters

FAQs

What is Capital IQ Workbench and how does it use Databricks?

Capital IQ Workbench is S&P Global Market Intelligence's Databricks white-label environment, where clients can trial and evaluate S&P's datasets in a private, managed Databricks instance with all associated data and services configured. Genie is integrated into this environment to enable natural language data exploration, reducing the technical barrier for evaluating potential data purchases.

What EDA pain points does Genie solve for S&P's clients?

Clients typically face four friction-heavy EDA steps that require technical expertise: data discovery, entity disambiguation, distribution analysis, and anomaly detection. Genie shortens this process by allowing evaluators to ask conversational questions and receive curated answers, tables, and visualizations instead of writing SQL queries from scratch.

How does entity disambiguation work in Genie for financial data?

Entity disambiguation is a configuration in the Genie context layer that maps ambiguous natural language terms — such as a company name or product model — to the correct underlying dataset entities, accounting for variations in names, tickers, and classifications. This is especially important for financial data where the same name can refer to multiple instruments, time periods, or entity types.

What business impact has S&P Global seen from using Genie for EDA?

The video describes measurable improvements in evaluation engagement, including more logins and queries run during the data trial phase, and better insight into what clients actually want based on the questions they ask Genie. The approach is described as shortening sales cycles and improving customer conversion rates beyond the previous baseline.

Full transcript

[00:08] Thank you for joining us for today's session. Um and uh I know it's uh at the end of day number one, so everyone's probably done their 10, 20,000 steps um on their health apps. Uh if you notice from the beginning, all the way from the keynote speech actually
[00:20] today, everybody's been talking about Genie, Genie, Genie over and over again. Um and uh fortunately or unfortunately for you, today's uh no different for this session. We're going to be specifically covering how we at S&P are
[00:34] using Genie with our clients to enable them to explore and evaluate all of the different data sets and the different content uh that we have. So, starting with a little bit background about myself. I've been at S&P for about
[00:47] a decade. I've been a product specialist and a solution architect across all of our different products and solutions. Most importantly is probably point number four. I'm a product manager of CIQ Workbench, which is our Data Bricks white label
[01:00] solution. So, any client that wants to trial data or test data, we spin up a private environment for them, uh a Data Bricks environment with all of the bells and whistles. We manage all of the data, all of the services that comes with it. And
[01:13] that kind of ties into where Genie comes in in this presentation. So, uh a couple of key things about the agenda. Uh what does S&P do? So, why do we need uh to accelerate EDA and how it
[01:25] functions? What are the some of the common uh pain points that clients typically have during the evaluation phase of our data? And then setting up Genie, automating Genie for all of the different data sets. Um and then finally, if we have enough
[01:38] time and everything works, hopefully we can do a quick demo of what that looks like. So, uh starting with S&P Global. S&P Global is a big company, five different divisions. Uh specifically, we're focusing on
[01:50] market intelligence. Um my marketing team would probably like to phrase it that market intelligence does data insights and analytics and workflow solutions for capital markets. I would probably compress that and
[02:02] summarize it to we deliver data. For the sake of this speech. So, we deliver data and specifically we deliver a lot of data. Um this slide just shows you the the volume and the variety of data across
[02:16] all of the different data sets, all of the different products, and all of the different solutions um that we have to serve. I've also added a couple of uh interesting points for context um so that you understand the size of the
[02:28] data. Um apart from number three, I couldn't think of a good example for 350 products, so I just said we have more products than trading days uh than there are in a year. So, now that we know the size of the
[02:40] data in terms of analysis and evaluation that we need to focus on, who are some of the clients? What are some of the use cases that we need to cater to? So, every time you evaluate certain data sets, um you have a tremendous difference in
[02:53] terms of personas that operate within different segments. You have individual investors, um academic lecturers, professional services, investment management, private equity. Um all of our clients are very
[03:05] diverse and they use the same data in very different ways. They have many different use cases. Um across the variety of different use cases that they do, we don't The problem that we have is we
[03:18] cannot standardize a a very uh uh we cannot do the same process because of the diversity of clients, data, and use cases. So, if we try to kind of create
[03:30] uh a standardized journey that our clients typically go through in terms of pain points of what happens from the moment they have a use case to until they make a commercial decision, it typically works something like this.
[03:44] And we have a variety of uh of different types of personas. We have a couple of different steps in the journey. The first and most important step of the journey, is they come in with some sort of requirements. They have a use case,
[03:56] they have a problem to solve, they have some sort of a need. We take that need and we map it to the variety of different data sets that we have available. Typically, that revolves mapping with speaking with vendors to provide a
[04:08] catalog that has 80, 90, 100 different data sets that they can choose from. After you make a decision, what data sets are relevant, we then go into a trial. And a trial here involves is bringing on all of the other relevant
[04:20] teams to set up an environment, get the infrastructure ready, create a database, making sure that everything is set up before any sort of data is tried or tested. Uh step number four is the first time
[04:35] that a user will actually touch the data in any sort of exploration. So, the use cases in the personas that we have typically data scientists, engineers, people that would write code and evaluate the data.
[04:47] And the pain points typically across using a lot of many data sets are you need to know how to code, you need to understand the schema structure, is it normalized, is it flatter, you need to be an expert to be able to extract some sort of insights.
[05:00] After the exploration, you validate the use case against the initial use case that you have. And of course, finally, you make a commercial decisions. Um is there in a budget, do you want to purchase the data? So, this is typically
[05:13] of over the last few years, we've probably done maybe three or 400 different trials. And all of the trials roughly map out to the to the same process. Um the types of personas that exist, I would categorize them kind of in two
[05:25] ways, maybe technical and non-technical. Um technical to me at least means is anyone that manipulates data in any language other than English, I would consider technical. Um if you're using just English to to
[05:38] write code and to evaluate data, I would maybe categorize you in one of the one of the areas that it's slightly less technical. Um what that looks like uh on the kind of the reason why I
[05:50] wanted to include this slide is because I think the the Databricks architecture kind of paint a very clear picture. So, on the left-hand side is typically how clients would use data. We
[06:03] we would send them data um as an input as part of the data intelligence platform. And what they would typically have to do is going from left to right is how the
[06:15] value increases for that particular data for that particular content. The closer you get to the users, the closer you get to the use cases, the more you enrich and transform the data for that particular use cases, the more valuable it becomes.
[06:28] So, normally clients that would uh begin evaluation would have to start from the left and work their way through the organizations and build all these services and everything else to get to the end users. Databricks have kindly put Genie
[06:43] as kind of the final mile within the data intelligence platform um before data before the business users have actually begin to explore and validate uh and use the data.
[06:56] For us, with Workbench, which is the platform that we provide for evaluation, which is based on Databricks, what we try to do is get as as close as possible to the right-hand side of that diagram.
[07:08] We create the environment, we add the data, we do the transformation, the manipulation, we create the Genie rooms, and get users, as you see it on the right-hand side, as close as possible to the value without having to bring in the
[07:20] engineers, without spinning up the infrastructure, and everything else. So, um this slide, the only thing that is different is the middle, which is the what is the actual pain point for exploration. The whole goal is you're
[07:32] filtering and you're looking for signal. You're trying to find a signal from the data that proves some sort of value. Um so, this is uh actually instead of I I was actually
[07:44] going to say instead of bloating your contacts which is you can see what I spent time doing. Instead of giving you too much information, I think what we should do is just build it from the ground up. So these are all of the different categories and the main pain points that you
[07:56] typically would have in exploration and evaluation of data. This is before Genie. Before we do anything, this is what the current landscape looks now. So firstly is infrastructure. The main pain points that clients start
[08:09] initially is you would need an environment, you need compute, you need data, you need approvals and sign-offs. We take data bricks, we give you a private solid workspace that we call workbench and all you have is a URL that
[08:22] allows you to access the data bricks workspace. That way, I would say anything in green that you see on the right as a solution solves the first problems that clients would typically have. So they would have access to the data.
[08:36] The next few problems quite common as you begin to use the data. For example, entity disambiguation. If you're looking for IBM, this is something that I've come across with or if you're trying to say model Y, you're trying to find some sort of an entity
[08:49] which does exist in the database but you don't know how to look for it. If you do wildcard searches for IBM, you don't know if IBM is saved as international business machines or some other way.
[09:01] Anything in yellow is we've solved it using these tools but they're not it's not fully solved. So we give you sample queries to cheat that you can customize that give you examples. The semantics mapping the way business terminology is
[09:13] done might be different. We typically send PDF documents, documentation across all of our different data sets that kind of swamps everything down. It's there, it describes the data but it doesn't really
[09:26] solve the semantics problem. And then we also provide notebooks for users to accelerate the learning of the different data sets. Some data sets have 10, 20, 30, 40, 50, 60 different tables.
[09:38] So, we provide notebooks to help accelerate the exploration, but it doesn't solve the problem. And then you have, of course, it's time-intensive. If you can't code, you need business users. You still need to write SQL. You still
[09:50] need to write Python. This is the landscape of exploration at the moment for us, for our users, where we've solved the first initial problems. The data piece, we provide a lot of samples, which we've partially solved.
[10:02] It's still time-intensive, and everything else is still to be solved. So, this is before Genie. Just to show you an example here of how we've solved one of the problems is the my pet peeve is the entity
[10:16] entity recognition. When we first started using Genie for the first time ever, Genie used to do what happened on the left-hand side. You ask for a company. You would say, "I want IBM's latest financials for 2025 on a
[10:28] quarterly basis." Genie would take IBM. It would go to the table that has the companies, and it would show you a list of IBM that's matched across all of the companies. And I promise you, I've trunked it. I've
[10:40] sliced that image. It's actually over 1,000 different entities plus. So, that's just a small example. How we've solved it, we've registered a function that you can do for Genie that allows Genie to effectively use it and cheat by
[10:53] mapping the company names that you provide to the closest possible match that we have in our database. So, you would ask, "I want IBM's financials." Genie would recognize that's a company, and then it would search the function
[11:05] within the data set, take the company key that you see on the right-hand side, and then use it for the rest of the searches in the rest of the conversation. That way, solving the entity recognition, at least for companies,
[11:18] um otherwise, I would probably argue that it's unusable, because if you start if you start wildcard searching across a lot of different entities and fields, it's very difficult to actually get a good answer. So, that solves the entity
[11:30] recognition piece. So, now that's in green. Okay, so what about the rest of the problems? So, the rest of the problems work something like this. We have about 90 or so core
[11:42] data sets. I cannot manually create 90 different Genie rooms manually. I'm not going to do that. So, what we've had to do is figure out a process that allows us to take all of
[11:55] the documentation and all of the metadata and standardize it in a in a systematic pipeline that creates the Genie rooms automatically uh for our clients. On the left-hand side, you can see all
[12:07] of the different components that we've used for us to do so. So, we've taken documentation, metadata, we've created templates, um and we've taken our query library, which contains a couple of thousand different SQL examples,
[12:20] and we've created a process that for each data set, each subscribable unit, each uh unit that clients would typically test, we create, for lack of a better word, an AI bundle. The AI bundle has four components. It
[12:33] has a readme file that describes the structure of the contents. It has the SQL queries for that specific data set. It has a data set uh a YAML file that is effectively, think of it like a semantic
[12:45] layer that describes the table, the relationships, the descriptions, the cardinality, and then a context file um that can be used effectively plug and play with uh if a user wants to ask some questions about uh specific data set,
[12:57] you throw it in Claude, you throw it in OpenAI, and you can begin having a conversation uh with that data set. So, saving time uh learning it. We then take each one of these bundles for each data set, and we've mapped it
[13:09] to the specific categories that Genie has um to effectively get the room as close as possible to be able to produce decent text-to-SQL. Um so, you can see with one single
[13:21] bundle, repeatable process for each data set, we map out to the catalog, the descriptions, um the tables, the descriptions, the metrics, the joins, the synonyms, and so on. Effectively giving us a way to create a
[13:34] Genie room per each data set in addition to the entity recognition service that we built on the side. Okay. So, this is how the landscape looks kind of after Genie and after the bundles that we've
[13:47] created. So, we've taken the semantics, the the schema away, we've abstracted the schema, so users don't have to learn the complexity immediately, they can just ask questions in natural language, reduces time to insight, it's self-serve
[14:00] for business users. And the most important at the top is we we manage the accuracy with benchmarks, give users feedback on how we structure the room, and transparency. So, this is
[14:14] kind of the the end result of what we're trying to achieve. So, I've prepared a quick demo to show you the previous user experience and the current user experience that we're working on. So, I'm not sure if you can see it, but
[14:26] hopefully you can. So, this is an example of how clients currently access when it's a notebook environment without Genie. So, for each data set, we give clients
[14:38] a starting point so they can treat where they can take our pre-built notebooks and customize them within a private environment that they can use for their use cases. So, we go through coverage questions, we go through frequently
[14:50] asked questions, exploring different events. This notebook is built on our machine readable transcripts, just as an example. You have FAQs. Clients will take this copy the questions, copy the queries, customize it,
[15:03] and then customize it for their use case. Instead of giving clients a notebook the first user experience, the first touch point with clients, with with data
[15:15] with a specific domain of data should be something like this. Whether you're technical or non-technical you You a a Genie room um where we describe what the data set is. We've mapped out all of the definitions, the metadata in the catalog
[15:28] um with instructions, with any expressions, the semantic layer, we've added the data. So, whether you're technical or not technical, you start here. If you do want to dig into the data you have you're still in a Databricks environment, you can go into the
[15:40] notebook. Um and then continue to evaluate and validate afterwards. For the sake of time, I because we only have about 2 minutes, I have pre-built a very quick conversation to show you an example of
[15:52] what we would typically experience. Um first thing to note is as you can imagine the uh as you can probably see, the agent mode within this workspace has been disabled for now
[16:04] which means that the responses are let's say a little bit worse than what we typically would get once the agent mode is fully enabled after a few approvals and everything else. But for now, I think this paints a very clear picture of the experience. So, you start in a brand new room.
[16:16] You a user would typically say, "Okay, before exploring it, tell me about this data set." So, it will go through it will go through the instructions and the documentation um transcripts. We provide earning calls for company presentations, speakers,
[16:29] etc. etc. Okay, what types of transcripts do they exist in this data set? Um these are the types of transcript that we capture um that describe what the what the document documents are.
[16:41] Okay, so what does the coverage begin to look like? Um the coverage the first capture date is 2004. The first transcript that's captured begins 2008. I can promise you if you have genie mode on uh agent mode switched on here, it would give you a
[16:54] distribution of all of the different documents and when the most significant coverage begins, but because it's chat is it's a bit more restrained um in its answers. Um okay, can you So, I have to ask you. So, can you show me the distribution per
[17:06] year? We've sliced this data since 2025 um so, that kind of makes sense. It stops kind of Q1 2022. Um and then okay, so now I know the coverage, I know some of the history, I know kind of how it works.
[17:18] The previous example I'm looking for IBM. Uh I would, like before I previously showed you, how it would work. It would verify that it was IBM. It would search for IBM, even though we
[17:30] don't save IBM inside the database. It would find the right relevant company and then can use that company ID for the remainder of the chat. Um so then I said, "Okay, show me every single transcript for a particular company that I'm interested in." Um and
[17:43] you can imagine you can ask for a coverage of 10 to 15 companies. It creates the sample universe and then uses that across for the rest of the conversation. There was thing I wanted to show you. I did see something weird which stated
[17:55] 2007 transcripts, which um was a bit high. So, I will I wasn't going to show you this section, but because this is the point of this exercise, I wanted to dig into what the actual numbers looked like. So, I asked, "Okay, so what are
[18:07] the actual different transcript types? What are how are they broken down?" Um this doesn't really tell us what the what is it a data issue or is it specific type of data. So, then I was trying to push it along, "Okay, so give
[18:19] me some of the descriptions for what those two different transcripts are." And eventually, um it found out that these are specific calls that were canceled and don't have enough information and they were conference types. So, then I finally got a bit annoyed and
[18:31] I I I prop- properly decided to ask it directly, "Why was there significantly more transcripts for IBM for 2018 compared to all the other years? Are they deleted? Are they wrong? What is the problem?" And then we found out
[18:43] after a couple of back and forths with the data, there were 144 transcripts that were conference pres- conference presentations, which IBM doesn't do before or after. It was only for that particular period. Uh which means that
[18:56] the code was correct and uh we were able to uncover why that was. So, I think we're out of time, but just to summarize the final point. We use Genie Rooms for this kind of user experience, where whether you're
[19:08] technical or non-technical, the first touch point with any data for exploration evaluation should be like this. Um, save you time, much faster, and much better user experience for anybody that's doing exploration.
[19:20] And that's it. I want to thank you all for joining us and listening.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.