Scaling Blockchain ML With Databricks: From Graph Analytics to Graph Machine Learning

Summary

• Graph neural networks: Learn how Coinbase uses Graph Neural Networks (GNNs) with Kumo AI to detect fraud patterns that evolve over time.
• Blockchain scale: Analyze billion-node transaction graphs representing Bitcoin and other cryptocurrencies for clustering and fraud detection.
• ML on cryptoassets: Discover how Databricks enables real-time fraud detection and NFT price prediction for cryptocurrency and token networks.

Scaling Blockchain ML With Databricks: From Graph Analytics to Graph Machine Learning

Talk by Indra Rustandi, Staff ML Engineer, Coinbase
Coinbase leverages Databricks to scale ML on blockchain data, turning vast transaction networks into actionable insights. this video explores how Databricks’ scalable infrastructure, powered by Delta Lake, enables real-time processing for ML applications like NFT floor price predictions. We’ll show how GraphFrames helps us analyze billion-node transaction graphs (e.g., Bitcoin) for clustering and fraud detection, uncovering structural patterns in blockchain data. But traditional graph analytics has limits. We’ll go further with Graph Neural Networks (GNNs) using Kumo AI, which learn from the transaction network itself rather than relying on hand-engineered features. By encoding relationships directly into the model, GNNs adapt to new fraud tactics, capturing subtle relationships that evolve over time. Join us to see how Coinbase is advancing blockchain ML with Databricks and deep learning on graphs.
Databricks Named a Leader in the 2025 Gartner® Magic Quadrant™ for Data Science and Machine Learning Platforms: https://www.databricks.com/blog/databricks-named-leader-2025-gartner-magic-quadrant-data-science-and-machine-learning
Build and deploy quality AI agent systems: https://www.databricks.com/product/artificial-intelligence
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements

FAQs

Why are graph neural networks better than traditional features for blockchain ML?

This video explains that blockchain data naturally forms massive graphs with millions or billions of nodes. Traditional feature engineering can't capture the complex relationships in these networks. Graph Neural Networks, implemented with Kumo AI on the Databricks Data and AI platform, learn relationship patterns directly from the data.

How does Coinbase use GraphFrames on Databricks?

This video demonstrates how Coinbase leverages GraphFrames to analyze billion-node transaction graphs from Bitcoin and other blockchains. This enables clustering and fraud detection at scale, uncovering structural patterns that reveal suspicious activity and fraudulent tokens.

What is agentic machine learning?

This video demonstrates scaling blockchain ML with graph analytics and graph machine learning on the Databricks Data and AI platform. Agentic machine learning applies machine learning techniques to understand network patterns and make autonomous decisions. The video shows how to combine graph representation learning with agent systems for analyzing complex blockchain networks and predicting behaviors at scale.

Which ML platforms support both traditional ML and deep learning frameworks?

This video shows how the Databricks Data and AI platform supports both traditional machine learning and deep learning frameworks for graph analytics. The platform integrates with popular libraries for graph processing and supports distributed training of graph neural networks. This unified approach enables seamless workflow from graph analysis to sophisticated machine learning models on the same platform.

Which platforms support reusable features for batch and real-time machine learning?

This video demonstrates feature reuse across batch and real-time workflows on Databricks for blockchain graph analytics. The platform's feature store enables consistent feature definitions that work across both batch model training and real-time inference, eliminating training-serving skew and ensuring that graph-based features are computed consistently across all workflows.

Full transcript

Good afternoon. Thank you for uh coming. So, so, so today I'm going to talk about our work in machine learning at Coinbase applying kind of graph uh analytics and also graph based machine learnings to tackle some of the biggest problems in crypto and blockchain. Um so just uh some agenda here but yeah as you might be aware um blockchain data is quite massive. So we have millions if not billions of entities uh kind of interacting with one another.
We have a record of them in the blockchain. So it's a it's a it's a very big data right. These entities are typically pseudonmous. So kind of hard to find the identity of these entities and also very dynamic because yeah the pattern of activities change over time. Um so gi given all these kind of properties of the blockchain what we we have some questions that we can try to answer and then so my my current focus at Coinbase is on uh fraud detection.
So there's a a bit focus more on the fraud detection angle but but definitely uh it's not isolated to just fraud. So for instance in the fraud case yeah so can we detect fraud real time in the blockchain that's one question let's say right but now going outside the fraud case so we have also this cryptonative tokens uh existing on the blockchain you might have heard things like NFTs can we predict or estimate the value of this kind of cryptonative assets on the blockchain using just purely based on the on the data right so that's another interesting And then going back to the fraud angle with these tokens now also because a lot of these tokens can be fraudulent also.
Can we detect kind of um which ones are fraudulent early enough so that we can protect our users from getting harmed when interacting with this kind of cryptonative assets or tokens. Right? So this is interesting because also I think with the right tools and um methods we can definitely try to tackle some of these questions and get kind of high high impact results right so so that's a blockchain and then um why graph I think because we have all these entities interacting with each other you can kind of make connections naturally among this con among these entities so naturally we can represent the blockchain data as a graph and as you can imagine also it's quite big as I mentioned right so millions billions of uh nodes let's say in the graph right um so in particular at Coinbase let's say what has helped us deal with this kind of scale of the data in the blockchain in order to get something quick it's kind of this library called graph frames uh I don't uh you might be familiar already but this is a sparkbased uh graph analytics library actually available in data bricks and one advantage is actually because if you deal with kind of this kind of scale of graphs right one natural instinct is to say I would need a graph database so uh okay but actually with graph frames you you don't you don't have to do that because I think with the graph frames actually you have uh you have the capability to run standard graph algorithms the library has a function options to do that already.
So you can run it right there and there in data bricks. So you don't have to worry about the complexity of say having to deal with um graph database moving things moving data around to the graph database and so on so forth. So just a quick illustration kind of right if uh this is very simplified but a real world use case that's common on the blockchain money laundering let's say right so very simplified again just just just want to emphasize but usually in this kind of money laundering you have kind of this pattern of uh flow transfers so the intention of the sender is to obuscate the source of the funds so it will transfer all these monies around but at some point it might come back to the sender, right?
So there's kind of this circular pattern. Um how can we detect this? And so let's say again very simplified case but let's say in in graph frames uh you can uh write codes uh similar to what's shown here and then then uh when you run it you can flag you can detect what kind of activity satisfy these patterns and we can then investigate further into whether kind of this is a indeed kind of money laundering case or not.
Right? So this is all all this you can you can uh run uh already it's kind of you don't need graph DB again. So data bricks uh with the graph frames library you can you can run this right. So now uh going to the more kind of again fraud angle uh where so in in fraud one one kind of characteristic in fraud is that the fraud serves are changing their behavior all the time because they they want to evade detection, right?
So, how do we how can we keep up with kind of this kind of um changing patterns in in the kind of uh what the frosters are doing, right? So, well, since I work in machine learning, I might be biased, but I think machine learning is a great tool to actually do this. And in fact kind of in in in the fraud domain um there are already techniques successful techniques that have been applied to do fraud detection. So you might have heard things such as random forest, gradient boosting, deep learning, um neural network, right?
What's kind of common across all these traditional machine learning methods is that uh you have to come up with the features. So you have to do feature engineering and usually this is a manual process, right? So and what happens if your data in this form of a graph? So again you might then there are features from things that kind of natural to a graph like for instance like the nodes of the degree kind of cycles um community structure in the graph maybe yeah note kind of hop uh the hub score or centrality score of the node so on so forth.
But this kind of process as you can imagine it can be quite tedious right and then you you want to come up with the feature evaluate not every feature might be useful for the actual use case so you have to kind of repeat the process so it's it's a very tedious process and also um remember that the fraud patterns also keep changing so you might need to adapt your features over time to to capture the new frost patterns, right?
So, this is a big challenge for this kind of machine learning um traditional machine learning methods. We're kind of uh can we do something about it? Especially when when again when the data is in in a graph. Yeah, I think turns out that there is a solution here what's called graph neural networks. So this is a machine learning technique or family uh um family of techniques that uh kind of naturally can be applied to the graph structure data to generate kind of predictions.
So the way kind of works kind of very briefly is that it's using what's called uh message passing here. Right? So here's an illustration. And so let's say um I want to get uh an idea if uh a node address a is is fraudulent. So get get a score of that. In this kind of graph neural network what happens is that actually there uh it will pass information. So the message is information coming from its neighbors the neighbors of node A. it will combine them all together and including also the information available for node A combine them together so do some kind of aggregation here in order to get some kind of uh what's we call embedding so a representation that can then be used to generate say the prediction that you care about say uh an indication whether uh the address is fraudulent or not.
So this is natural in in the model itself as you can imagine if you have to kind of manually engineer similar features to this kind of what the message passing is it's doing it can be quite tricky right it's it's uh yeah it's it's not obvious. So yeah one feature of um graph neural network uh is that it's very good at capturing uh local information. So um information available from let's say neighbor one hop or two hops away.
Um but it does tend to struggle when let's say the relevant information is available farther away. So kind of let's say can we get uh information from nodes 10 or plus hops away right uh so what what can we do about it? So there's actually a more recent development called graph transformers where if you're familiar you might have heard about transformer already but that's a kind of the architecture underlying large language models.
So it's using kind of this uh what's called attention right and then using this attention applied to the kind of this graph kind of model what it's doing is trying to pay attention to say relevant information in nodes that can be arbitrarily uh distant. So that's just very high level idea kind of right of how we work. So that's kind of so we have a solution also now for let's say if if the information is available farther away.
So yeah the exciting thing so now we have both kind of GNN's and then graph transformers both are available to help us right deal with kind of this graph structure data and then help us kind of reduce the need for manual feature engineering in order to get let's say a very uh perform model like with state-of-the-art performance right okay so now let's say the natural question is okay. So we have all these fancy methods.
Now how can we actually use it in kind of a production use case? Um and especially again remember the fact that our graph is quite big let's say billion plus nodes right. Uh so this is where let's say um one instinct maybe okay let's try to build it ourselves kind of from scratch right um we did some kind of analysis on this uh just to try to get a rough estimate and remember again Coinbase we cannot say assign a a very big team just solely working on this problem let's say 20 people is like too big so yeah for kind of a reasonable team size for us if we have to do Ourself we estimate that let's say at least 6 months we need six months to kind of build the necessary setup in FRA and so on to do training um evaluation and also uh having this model run in production right so and well thanks to actually what uh there is a tool now called kumo um so we have some kumo people here um but kumo helps us to kind uh solve this problem.
So uh quick preview of Kumo. So Kumo allows uh kind of yeah companies like Coinbase apply the graph uh advanced graph models that I've mentioned before GNN's uh graph transformers on kind of their own data um and then to get kind of the most performant model and then and then remove kind of all the complexity that uh that is needed to train and run these models. Right? So with Kumo we we we develop a model on the blockchain kind of graph where in about four weeks we um uh have one uh machine learning engineer working on the model and then we got a state-of-the-art model where yeah we we see double the precision compared to a traditional machine learning model using kind of manually engineered features.
So this is exciting one that we have now a capability to protect our users from losing kind of millions of dollars right so that's kind of on the blockchain right I've mentioned the blockchain data most companies I can imagine uh they don't deal with blockchain right and so what uh what what is it for for like a typical company like this kind methods actually and this is where I think uh you don't need to look kind of hard to find kind of graph in your data.
So when you have kind of tables relational tables which is kind of a typical enterprise typically has that right when you have these tables are joined by kind of this foreign key primary key already like say your customer ID product ID so on so forth when you kind of make these connections you already have a graph and then you can apply things like yeah tools like Kumo to actually say be able to generate kind of predictions based on the graph that's kind of in in your data already in the table.
So with and then with tools like Kumo actually then what what kind of the what the advantage that you can gain is that again you don't have to worry about say setting up the infra uh data preparation and so on. Um because then yeah you can you just need to put say the data in the right kind of place uh that kumo can pick up and you need to define kind of the keys. So for us we use kind of data bricks actually for this and then kumo will take uh the work of turn turning your data into graph and run um autoML basically to discover what's what's the best model in for your use case.
Um so this is kind of already available actually and this is where kind of for Coinbase we are excited to explore more on this where we we we can explore more of not just again the blockchain data but also our users data because here we see the potential of say if you have let's say uh taking us as example Coinbase our our data on our user activities and then if you can join them with kind uh okay the onchain behavior of these users and then if you have some some other data on kind of let's say trends uh the in that happen in the uh crypto all this stuff with kind of kumo I think all it's very easy to integrate all of them together and then uh treat them as a big graph but then be able to kind of generate a model where yeah then you can get kind of state-of-the-art results.
So for instance, let's say right um and again outside fraud this is but if we have this information on the user data then maybe with kind of this holistic big picture we can then make specific targeted kind of recommendations call it for the user in terms of maybe say what what things in crypto that are relevant for them right meaningful for them so I think this is all kind of it's already available let's say in our data and in your use case maybe something different but say kind of or somewhat hidden but I think with again the the nice thing with kumo is that and then this kind of graph techniques is that it allows us to uncover this hidden patterns or that kind of already present in the data.
So I think this is uh we are excited to go this to go in this direction. Yeah. Um so that's all I have. Uh uh thank you and happy to take questions.

Learn more about the Databricks Data and AI platform.