Mastercard GAME Entity Resolution with Generative AI on Databricks

Summary

Mastercard's GAME (Generative AI Merchant Enrichment) algorithm replaces brittle string similarity rules — which would incorrectly match 'Walmart store' to 'Walter's store' rather than 'WM Supercenter' — with a semantic pipeline that links unstructured transaction descriptors to real-world merchants while prioritizing low false positive rates.
The three-stage architecture uses Llama 3.1 8B for entity recognition, vector search to retrieve up to 160 merchant candidates, LightGBM to re-rank candidates into the top 20, and GPT OSS 120B as an AI judge, running on Databricks with Databricks Apps, serving endpoints, and Unity Catalog.
This video covers responsible AI principles applied to financial entity resolution, including explainability, data sovereignty, continuous improvement loops, benchmarking, and risk-adjusted value measurement for a system that must minimize false positives in production.

Mastercard GAME Entity Resolution with Generative AI on Databricks

Mastercard GAME, or Generative AI Merchant Enrichment, is an entity resolution algorithm built on Databricks to match unstructured transaction descriptors with real-world merchants. It replaces brittle string similarity rules with semantic search, fine-tuned language models and contextual reasoning while prioritizing low false positive rates.
Learn how GAME combines Llama 3.1 8B for entity recognition, vector search for candidate retrieval, LightGBM for re-ranking and GPT OSS 120B as an AI judge. Tomas Drietomsky also explains how Databricks Apps, serving endpoints and Unity Catalog support interactive testing, automated queries and large-batch enrichment through the Mastercard Places API. The session covers model governance, data sovereignty, explainability, benchmarking and risk-adjusted value in financial AI.

Chapters

FAQs

What is Mastercard GAME and what problem does it solve?

GAME stands for Generative AI Merchant Enrichment and is an entity resolution algorithm built to match unstructured transaction descriptors with real-world merchants. Transaction descriptors often contain incomplete or abbreviated information — such as partial merchant names and truncated addresses — which makes string similarity approaches unreliable and causes high false positive rates.

Why does traditional fuzzy matching fail for merchant entity resolution at Mastercard?

String similarity algorithms such as Levenshtein distance measure character proximity rather than semantic meaning, causing incorrect matches — for example, matching 'Walmart store' to 'Walter's store' instead of 'WM Supercenter.' This problem is compounded by merchant aliases, service provider descriptor noise, and the unstructured format of real-world card payment data.

What are the three stages of the GAME architecture on Databricks?

GAME uses a three-stage pipeline: Llama 3.1 8B performs entity recognition on the raw transaction descriptor, vector search retrieves up to 160 candidate merchants for that entity, and LightGBM re-ranks those candidates into the top 20, with GPT OSS 120B acting as an AI judge for final evaluation and quality control.

How does Mastercard approach responsible AI governance for the GAME system?

The GAME system prioritizes low false positive rates over raw match coverage because incorrect merchant enrichment carries financial and reputational risk. Governance is enforced through Unity Catalog for data sovereignty, with continuous improvement loops and benchmarking to ensure model quality is maintained, and explainability requirements addressed as part of the financial AI governance framework.

Full transcript

[00:08] My name is Tomas Drietomsky. Slovakia. And I'm a lead data scientist in MasterCard for 4 years now. Uh today I would like to talk about an algorithm I've been working on for the last 3 years, which is called game, which stands for generative may I generative AI merchant enrichment.
[00:26] Um the agenda for today is the following. We'll start with introduction and a problem definition. Then we'll talk about architecture, how we actually solve the problem. Then we'll talk about learnings and principles for responsible AI. In other words, how to build AI products
[00:42] responsibly. And then finally, we'll talk about the results and continuous improvement. So, before we begin, I've prepared an example. Believe it or not, it's based on real-life scenario. So, we have a query,
[00:57] Walmart store, and we have two options. Do we match it with Walter's store or WM Supercenter? So, who thinks option one is the right choice? Raise your hand. Good. There is no one. So, that's a good thing.
[01:13] Uh but there's still someone who thinks that this is actually option one is the correct match. That someone is string similarity. So, string similarity would say yeah, Walter's store is closer to Walmart store than Walmart Supercenter. So, this practically the problem we've
[01:29] been dealing with at MasterCard for quite some And today we'll discuss how we solved it. So, data set harmonization is a ubiquitous challenge at MasterCard for both internal processes and operations and
[01:44] market-facing services products. The objective is to link two records or the objective is to link records between two or more data sets when no common identifier exists. Um traditional fuzzy matching algorithms
[02:01] suffer from low match coverage and high false positive rates. Today, we'll focus on one of the most impactful use cases of this problem, which is merchant enrichment. So,
[02:17] merchant enrichment is a subset of entity resolution. But instead of matching arbitrary entities, um we are specifically trying to match uh real-world merchants to transaction descriptors.
[02:32] So, what is a transaction descriptor? It's a text that is attached to a card payment that contains information about the merchants, its location, contact information, etc. But it often comes in unstructured format. So, this actually a transaction from my
[02:49] city banking app uh from my trip to New York like 2 years ago. And you can see that the merchant name is Square Sunshine Coffee Ro. Address is Largement, that's it. Right? Uh what would be nice to know is like what's the complete address of the merchant? What's the URL? How can I
[03:06] reach out to them? How can I find them, right? So, this is something that we can solve. Uh so, think about turning the Square Sunshine Coffee Ro Largement into a complete merchant name, so Sunshine Coffee Ro Roasters uh at 1932
[03:22] Palmer Ave, Largement. And complementing it with address information, business category, URL, uh etc. Um Game was built to solve data harmonization more broadly. Uh but one of its most concrete
[03:39] application uh is a product called Places. It's API-based service where both internal and external clients can uh send these raw transaction descriptor to us, and we'll enrich them for you. Um the whole idea was born about Yeah,
[03:56] originally we had the legacy engine based on string similarity, but as you could see on my first slide, it doesn't really work, right? So, we had to come up with a solution, and uh yeah, that that's the main reason I'm here today. Um
[04:16] So, let's talk about how and why the legacy engine misses the mark. So, the legacy approach at Mastercard binds hand hand-crafted rules with normalized Levenshtein distance. If you're not familiar with that, so Levenshtein distance is uh essentially a character-level edit
[04:33] the edit distance metric, and it asks how many insertions, deletions, and substitutions do we need to take to turn one string into another. Uh on paper, that sounds reasonable, uh but it falls apart in many predictable ways.
[04:49] Um The first problem is the unstructured transaction descriptors. You never know what's going to be the entity composition in the descriptor, right? So, do you receive merchant name, street, zip, city, state, country? Do you receive merchant name, URL, and the state? Do you receive
[05:05] merchant name, phone number, and the address information? And in which order, right? That's a tough problem to tackle with string similarity, especially when you don't know what is inside of the transaction descriptor. Um what we have also seen is that even though we would receive structured
[05:20] input, there are still variations of certain entities that string similarity simply cannot cover. For example, New York City versus New York, St. Louis versus St. Louis, right? Uh you have to build rules for this, and the problem with rules is
[05:35] they fail. First, they memorize, and then they go stale, and then you have to build more and more, and it never ends. The second problem that we're dealing with, I call it the cold start problem, is that the transaction descriptor patterns evolve constantly. There are
[05:51] new payment facilitators that process our data, right? There are new merchants that have unique merchant names, they try to be creative. We have um
[06:07] um there are um There are patterns that we simply see in the transactions where merchant looks in certain way at certain time point and then they switch acquirer or they they start to use I know, new PO new POS machine, right?
[06:25] And only they look different, but we need to know that. We still need to link these records. And this is something that String ID has been always struggling with. And at least uh context knowledge. So, entity resolution in financial domain
[06:41] requires context, geography, and world knowledge to generalize at scale. I've prepared a couple of examples that we've been dealing with uh the last couple of years, and I think these are interesting and funny in the same way. So, brand aliases, right? Uh do you
[06:58] remember the example I had on my third slide? So, Walmart versus Walmart Supercenter, Kentucky Fried Chicken versus KFC, New York City versus New York, right? Um You have simply multiple ways of describing one entity. Uh then we have the service providers.
[07:15] So, the service provider usually appends their name before the merchant name. So, Starbucks becomes Square asterisk Starbucks, right? And there's endless amounts of service providers that do this. So, how do we do that in this? Dynamic identifiers. So, we have software vendor vendors such as Amazon,
[07:32] right? And they append their transaction ID to each merchant name. So, to each transaction. So, each transaction looks different, right? And there's many merchants that do this. And these strings can be timestamps, order ID, booking numbers, transaction ID. It can
[07:47] be whatever. And again, we at General we solve this in scale. Duplicates. This is interesting. So, sometimes we just see the same merchant name twice divided by asterisk. Then we have misaligned attributes. This usually happens with e-commerce
[08:02] merchants. So, they send us phone numbers in the city fields. And sometimes they send URLs in the in the city field. So, if you think about string similarity engine, I want to say that, I know, Subway in San Francisco,
[08:18] in California, is the same as Subway with this string in California. Obviously, it's going to fail, right? And you need to know that this is a phone number. It should be easy to detect. Um And then we have partial duplicates. So, sometimes we see
[08:34] a part part of the street address appears in the merchant name, but it's usually just a little like just a small part of it. So, this is again going to in this is going to uh significantly affect how string similarity works.
[08:50] And then finally, truncations. There are two types of truncations. One is when you would take Tomasi's Cafeteria and we are missing the last two letters. It's not going to be that big deal, right? It's still like you can take the longest common sequence and tell that these two strings are the same. But with this one, it's more tricky because this merchant
[09:07] chain is actually supposed to have four digits store IDs. As you can imagine, store ID is unique identifier of certain location. So, if we miss the last two digits, it's going to be different store, right? And big problem. So, how does GAME solve that? So, GAME is an
[09:25] entity resolution algorithm that combines semantic search with an ensemble of large language models. Rather than asking, "How similar are these two strings?" we're asking a question, "Are these two strings the same real world merchant? And that completely
[09:41] completely changes the narrative, right? It was designed specifically to address the systematic limitations of ISO 8583 standard used by global payment networks. Uh the constraints that are baked baked
[09:56] into the standard are not going away. So, rather than fighting it, we taught game how to reason with it. Um and it does that reasoning against Mastercard's near real real-time real-time merchant database as a
[10:12] reference data set. All right. Uh we'll continue with the architecture. So, we're the first team at Mastercard to build an AI-powered API-based products on Databricks end to end. And
[10:29] now I'll explain how we actually did that. So, revisiting my example from earlier, Sunshine Coffee Roasters, right? This is our input. It can be anything though. There could be an street address, URL, phone number, whatever. Doesn't matter. So, this is our starting point. And game comprises
[10:46] of three stage architecture. So, the stage one we call entity recognition. Uh the idea is that we have fine-tuned that LLM on our transactions, so it would actually recognize the entities in the transaction descriptor. So, we know
[11:02] that Square is service provider, we know Sunshine Coffee Roasters is a merchant name, and we know Larsmont is a city. Stage two is what we call candidate retrieval. So, we have embedded Mastercard's merchant database and created a vector search index.
[11:17] Now, we take the output from stage one, which is Sunshine Coffee Roasters Mont, because we don't need Square for the enrichment. It doesn't add value. It actually it creates noise. So, we want to separate the Square. It's We know it was provide was processed by Square, but it's all we need to know. Now we just
[11:33] take the rest and retrieve the candidates. Here you see some made up examples, but they all look similar, right? Uh but that's not it. We still need to know which candidate is actually the real match. And for that, we use entity linkage. So, we have another LLM.
[11:50] We also call it AI as a judge. And we take the output from stage one, we take the output from stage two, and we submit it to AI as a judge. And AI as a judge will determine whether there is a match or not. And if there is, we are going to retrieve
[12:05] uh or return the energy information. Now, let's talk about how it works under the hood. So, we send a request, Sunshine Coffee Roasters. We have the entity recognition, right? So, how does it actually work? So, we have generated labeled
[12:21] transaction descriptors from our transactional table. We have fine-tuned Llama 3 1 8 B on it. And practically, this is our entity recognition engine. So, it takes the unstructured transaction descriptor as an input, and it will generate a JSON with entity labels as an output.
[12:41] Stage number two, candidate retrieval, actually consists of three distinct steps. We start with constructing the vector search index. So, again, we take Mastercard's merchant database, we use BG small EN to generate the embeddings, and then we construct vector search search index from those
[12:57] embeddings, right? But that's not it. The way it works is for each query, the vector search index will return 160 candidates for a match. Candidates that potentially could be the right record that we're looking for. But that's a lot, right? Like AI has to
[13:13] analyze like 160 records, that's not really optimal. So, we have implemented a score model. So, it's practically just LGBM model that we have trained for this exact use case to actually generate score for each candidate pair. So, you take the query, Sunshine Coffee Roasters
[13:30] large merchant, you take those 160 candidates, and you generate score for each pair. Then we filter out the candidates with low score and the candidates are simply not relevant, and we just take top 20 that are relevant, right? The top 20
[13:45] like the refined top 20 candidates. And yeah, then we just re-rank them. That's it. The last step is AI as a judge. So, we take the parsed entities from stage one, we take that refined candidate pool from stage two, we send it to GPT- 4 SS 120B,
[14:04] and it will determine whether there is a match or not. Then we generate the response, which will be either we couldn't find a match or we'll return the energy record. Now, let's talk about accessibility. So, game is accessible to two types of users, MasterCard employees internally
[14:21] and external clients. So, first let's talk about the internal. So, Databricks apps, they're great, right? And they provide a clean intuitive interface designed for testing and exploration. So, someone wants to try game, they want to learn like what is it about, I want
[14:37] to test it in like small sample. That's what you do. You don't need to really like know what's going on there. It's like ChatGPT-like unit interface where you just start sending those transaction descriptors, and you'll get the output back. Then we have a serving endpoint. So, if you need if you need it for an automated
[14:53] workflow, this is what we're using. So, we just expose game to an endpoint, so you can send single queries or all small batches, and we'll simply send the energy records back to you. The last access point for internal use cases is paired entity catalog. So,
[15:09] that's meant just for large batches, cuz as you can imagine, there's like a lot of LLMs involved, right? And it takes quite some time. So, that's why we have also prepared this access point, and the idea is that you have two MasterCard teams that have their own workspaces, right? And you
[15:25] need to connect them somehow. So, we create a shared Unity catalog where both teams have read and write access. So, data can simply prepare the table for us, we take it over, we enrich it, and we just send it back. That's it. This is what we use for the Well, when when it's I I would say mostly millions
[15:41] of millions of records. Then we have uh the external clients. So, the access works through two channels. Uh the API. So, that's the Places API product I've mentioned in the beginning of my talk. And that's the primary integration point. So, clients, doesn't
[15:58] matter internal external, they send the transaction descriptors, we enrich them and send it back as JSON. Easy as that. Uh for the large batches, uh this is still being developed, uh but for large batches, you can just send over a file, take it over, enrich it, and send it
[16:15] back. So, very similar concept as the shared Unity catalog. Okay. Now, let's let's talk about how to build AI product responsibly and how how did we actually do it, right? So, um MasterCard has an AI governance
[16:31] program to assess, measure, and help teams uh mitigate AI risk. So, I want to talk about how we uh adopted uh these principles when building game. So, there are three foundational pillars that we have followed. The first one is
[16:48] governance. We host open LLMs on private serving endpoints rather than calling third-party APIs, which gives us full sovereignty uh over our models. Every inference run is attributable to certain clients, products, even use
[17:05] Another argument I would use to use open models instead of the uh closed models like Gemini, Claude, and GPT is if there's a new version of Claude, let's say, right? I I love the 4.6. Then we had 4.7. Now we have 4.8, I think. Uh
[17:23] hard to keep track. So, uh these they are ultimately going to decommission the earlier versions. But what if you have built your entire workflow around it? What if it's actually changing uh how your algorithm works? That's not going to happen with open models. So, that's why we are really sticking to the
[17:39] Llama 318B and the GPT 120B. So, the sovereignty is like very important concept for us. Um the second is trust. So, um in financial sector, a bad match is not just a wrong answer. It can end up in a
[17:56] credit decision, risk model, or a fraud flag. And this is why minimizing inaccuracies is not just a quality target, but it is a responsibility. And the third is quality control. So, when game makes a match decision, we
[18:12] can trace exactly what was extracted extracted and entity recognition. We know exactly which candidates appeared, and we know what was the reasoning of the AI as a judge to speak certain that or simply understand why there was no match, there was match, there was a
[18:27] false positive. We can always backtrack. Um being able to reconstruct and explain that decision to a compliance team, a client, or an auditor uh is a hard requirement. So, explainability and data sovereignty were
[18:43] not features that we have added in the end, but they were design constraints from the very start. Uh from 2023 when I started to work on this. All right. Let's talk about some model results and continuous improvement. So,
[18:59] this is practically our process of financial quality, right? So, the item number one is assess impact. What does it mean? So, there is a new feature that we want to add. There is a pattern in the transactions where game simply fails and
[19:16] we need to figure out like what to do with it. So, we usually spend a lot of time by assessing the impact because as you can imagine, the architecture is quite complicated, but everything is connected, right? And we have seen that even small change in prompt, small change in training data can have
[19:33] horrible consequences if you make a mistake. Oh, this is why we usually spend a lot of time especially on the SS impact and the data scientist peer review. So, what is data scientist peer review? Like everyone is reviewing a data pull request, right? Like that's kind of obvious, but there is a reason why I
[19:50] added it to my slide and it is exactly the reason that I just mentioned. Even small change to prompt can can have like incredible effect on your model.
[20:06] So, let's talk about the automated test now. So, to cover the wide spectrum of real life scenarios that we see when it comes to anti-fraud solution at MasterCard, we have constructed benchmarks for each module of game. So, one for entity recognition, one for candidate retrieval, and one for AI as a
[20:23] judge. The idea is we have current version of game that is running in production and we know exactly what what is the what is the true positive rate, false positive rate on our benchmark data set. Now, there is a change, there is a new feature or a
[20:39] hot fix. What happened next is that we'll simply rerun all those benchmarks and what we would expect is that it's going to be the same or better, right? If it's worse, then I guess we made a mistake. This is one of the mechanisms that we have implemented to make sure that the model keeps performing well and
[20:55] the users can actually trust that each change that we apply is not going to compromise um their use cases practically. Human over the loop. So, this is again very important. We have a success team that is randomly
[21:12] reviewing matches from game, and they are reporting false positives to us, to data science. What we do next is that we try to assess whether it was just a fire, like one record, which we simply couldn't cover, or if it's a systematic pattern. Because again, if
[21:28] it's a systematic pattern, we have to go through 1 2 3 again, right? Um and only after that we release the feature or the hotfix. Let's talk about model results now. So, we'll start with entity recognition.
[21:48] So, the scope of this benchmark is very straightforward. Unstructured transaction descriptor, JSON with entity labels, and we just want to see how accurate different LLMs are for this task, right? The metric that we use is a precision. And here we have couple of closed models
[22:05] and open models that are available on Databricks System AI catalog. So, obviously, Gemini 2.5 Pro, Claude Sonnet 4, GPT-5 2 performing very well. That's what you would expect, right? They're practically trained on the internet, so we expect them to have very good
[22:22] uh knowledge about brands, geography areas, like they understand that certain zip code return is related to certain city, city is in certain county, county is in certain region, etc., etc. So, yes, the closed models perform well out
[22:37] of the box. Um if we would talk about the model that I actually mentioned that we are using, Llama 3 18B is in the bottom right, like 64% precision. That's not much. So, what can we do about it? And that's practically the next question.
[22:53] Can fine-tuning close that gap? Yes, it can. So, what you can see on this chart is we have added some others open source LLM's that we have fine-tuned on the benchmark data set that I I just talked about and you can see that suddenly we
[23:11] are outperforming Gemini 2.5 Pro, Claude and GPT, right? Uh the idea is not for every task you don't need Opus 4.6, 4.8 for every single task, right? If you have the domain knowledge
[23:27] and you can actually define the boundaries of like what you're trying to solve. In our case, it's the wide spectrum of the transactional data that we're trying to parse. If you can reflect that in your train validation and test set, you can simply fine-tune small models and save a lot of money by doing that, right? Just
[23:44] compare how much would it be to run Opus 4.6 on our task compared to Llama 3.1 8B. So, this is the main value of it. Um
[24:01] Uh one more thing worth highlighting is that we have joined forces with Mastercard Foundry R&D and we have evaluated several of these smaller models for this exact use case and they have published a paper on archive, which you can find in this QR code and this paper practically analyzed
[24:17] the effect of prompt effect on reasoning and how good are these small LLM's for this entity recognition use case. All right. Let's talk about candidate retrieval.
[24:34] So, the scope is Mastercard merchant database millions of merchants and we need to get the top 20 and we need we need to get the correct merchant in the top 20, right? That's That's what we're trying to do. The metric is recall and
[24:51] this is how we are doing it. So, let me explain this heat map. So, the columns are different languages. We have Greek, Bulgarian, uh English, Czech, and German. On the Y axis, we have different embedding models. And what you can
[25:06] notice is that we have purposely uh or we have evaluated very different models on purpose, right? So, what you can see there is the model size in millions of parameters, and then you can also see the embedding dimension. One of the outcomes of our experimentation is that embedding
[25:22] dimension has low impact on recall in our use case. So, why would we use embeddings with one 1,024 dimensions, which will result in slower vector search, higher storage costs, and higher latency, right? If you can just go ahead and use the BG small
[25:39] EN V 1.5, which has marginally worse recall, but it's just more effective. So, this is how we are actually thinking about the problems, right? Like, we're not trying to get the best model possible at any cost. We're always trying to find the right trade-off between the price and the accuracy.
[26:01] The other thing that's very important to mention is that each country has specific nuance. In the US, you have different address model than we have in Europe, right? Or they have in Middle East, India. So, that's why we are seeing very different results across different languages.
[26:16] And the idea is that Game has model architecture, architecture, which means that for each country, we can use different embedding model. And that's what we are actually doing. So, when we're scaling to different markets, we run these benchmarks, make sure we are using the best model possible, and then just integrate it.
[26:36] And finally, let's talk about the entity linkage. So, what's the scope? We take the output from stage one, which is the parsed transaction descriptor, right? We take the top 20 candidates, and we send it over to the AI as the judge. And the AI as the judge has to
[26:51] determine whether there is a match or there's no match. It's as simple as that. It's done based on a very complex prompt. The metric that we are using is F0.5 score because it it penalizes false positives more heavily.
[27:07] And as I said, we are obsessed with false positives. We don't want to see them. So, the whole methodology is built around having minimal false positives but maximizing the true positives. Um So, let's have a look at the results. The frontier models like GPT-2,
[27:25] Gemini, Claude Sonnet, they perform well, right? Opus 4.6 is is obviously the best one. Uh and then if you look at open source models, they're not that much behind. It's not that big difference. But that's why I don't really like this
[27:40] view. I don't like to just sort everything but by the F0.5 score. So, I prefer this view instead. On X axis, we have false positive rate, where lower is better. And on Y axis, we have true positive
[27:55] rate, right? So, as I just said, we try to keep the false positives down as much as possible, but we want to have as many true positives as possible. So, this is how we are deciding which uh which model we will use as AI as a judge.
[28:11] We are using GPT- 120B because it's open and it's apparently the best one from the models that are out there, right? In the past, we were actually Llama 3 370B, but we were actually fighting a lot with hallucinations because the model is trying to please you as a user.
[28:28] So, it's always trying to find a match even if it's not really there. Uh one of the last things that I wanted to cover is re-ranking. So, let me explain this chart. On X axis, we have relative inference time for K1. K1
[28:46] stands for one candidate, right? What if What would happen if you would just take the the first candidate that appears from from the vector search that is returned by the vector search? On the on the Y axis, we have relative hit rate. So, what you can see happens is that
[29:03] when we use K20, we're gaining approximately 35% hit rate or match rate, and the cost for that is approximately 60% increased latency. That's kind of okay. And that's the reason why we're actually using 20 candidates, right? That's our optimal
[29:19] number. Then we can see the line it's we we we start to see the diminishing returns effect. So, what we have changed in the last compared to last year is that originally we're always just taking the 20 candidates. We never had any re-ranking. We were not scoring the the candidate,
[29:35] but this is something we have changed. So, today we take the top 160 candidates. So, you can see that the potential hit rate is actually up to 50% compared to one candidate, right? So, we apply the re-ranking and the filtering to remove noise,
[29:51] and now we have constructed a refined version of the top 20 candidates. What's the result? For the same latency, we have 8.3% hit rate gain for 20 candidates. So, no extra effort, right? We just re-rank the candidates. So, we have complemented the
[30:08] semantic similarity with the LightGBM model, which is practically just built on like custom-built features that for our use case. I know that was ambiguous, but that would take a lot of time to explain. Uh all right.
[30:25] For every attempted match, there are only three possible outcomes. A true positive creates right? So, we match the record. Then we we didn't match the record. So, that has neutral impact, right? So, you send us the Sunshine Coffee Roasters. We couldn't find it,
[30:40] but there's no harm done in that. But if you send us the Sunshine Coasters, and we'll match it with Starbucks, there'll be damage, right? Like as I said earlier, this engine is used ultimately by like other products that are running loyalty or fraud or something like that. So, we
[30:56] simply cannot afford to match two different brands. So, the KPI that we use is the risk-adjusted value. So, there is value created by GAME, but then there's also damage caused by the false positives. And the damage caused by the false positives is much bigger than the value
[31:12] created. And this is practically the reason why we are obsessed with the false positives. And that's practically the only thing that we are focused on, like how to keep them down, and the true positive rate is just growing naturally with it. All right. So, GAME is the new standard for entity
[31:30] resolution. This slide compares the legacy engine with GAME across three levels of input complexity, defined not not just by the completeness of the records, but by how distorted and ambiguous the descriptor is. So, what is the completeness?
[31:46] So, I I think I've mentioned this in the beginning of my talk, but it really matters if the client will send us merchant name, street, zip, city, state, country, URL, phone number, or just a merchant name, right? If you just send us the merchant name, which is like Joe's, it can be anything. It can be
[32:01] Joe's Barber Shop, Joe's Cafe, or just Joe's. You You just don't know. And it really matters like what's the input entity composition. So, let me explain the three levels of the complexity. So, low complexity means that the
[32:17] merchant name is clean, there is enough location signals to determine where approximately it is, and and there is minimal descriptor noise. So, we don't have PayFacs payment facilitators there. We don't have those transaction IDs. We don't have any of
[32:33] that. It's just clean record. And you can see that the engine performs quite well, right? So, what is this Why am I even talking about this? So, string similarity still works to some extent. If the records are clean enough, it works. And I don't
[32:49] think we should replace every single uh entity resolution engine in the world with game. Like, not at all. String similarity still works to some extent, and we should be we should keep using it because it's cheap, it's simple, and it's explainable. So, you can see there isn't that big
[33:05] like performance gain uh of game for the low complexity. Moderate complexity are merchant names that are affected by single distortion such as aliases, truncations, or the service provider prefixes.
[33:22] Right. You Here you can see that the string similarity engine or the legacy engine uh suddenly dropped to just like 40% on our on our benchmark because it simply cannot deal with these things. It doesn't know that Square, Sunshine, Coinstar Sunshine, Coinstar are the same thing. It's just the confidence score
[33:39] that is returned by the legacy engine is just telling you the added distance between the two strings, but doesn't tell you if it's the same entity or not. So, this is like one of the main issues of the string similarity, and game can solve that. And then finally,
[33:55] we have high complexity uh uh samples. So, the merchant signal is ambiguous or hidden. So, we have seen classic descriptor where a person would use PayPal to order McDonald's through DoorDash. Right? So,
[34:11] you see that long string where you have like PayPal star DoorDash star McDonald's 1 2 3 4. Now, what's the merchant? And how is string similarity going to help with that? Well, it's not. It simply doesn't understand that what PayPal or DoorDash is. It's just a string. There's no meaning behind it.
[34:26] And it just doesn't work. So, what we can see here is a clear trend that the more difficult, the more ambiguous, the more complex the descriptors are, both engines start to perform worse, but game can still solve most of them,
[34:43] which is important for us. That's it. Uh thank you very much everyone who chose my talk over the financial forum. Really appreciate it. Uh I also like to thank my team. It won't be possible without you guys, so appreciate each of you. I also
[34:59] appreciate the Databricks team. I don't know if there are somewhere around, but they've been helping us build this thing for the last 3 years. So, I also thank them, and yeah, hope you liked it. If you want Yeah.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.