Machine Learning on Ethereum Data: Recommending Tokens
This is not just true for Ethereum, but with all public blockchains. It’s a profound difference from the old world of siloed and permissioned data. As a data scientist you’ve traditionally had access to datasets that are either private (you work at a company that owns the datasets) or made open after-the-fact (e.g. MovieLens).
In the world of Ethereum, data is open by design.
In this post, we will demonstrate that you can build data products on top of Ethereum data. Specifically, we’ll show you how we trained and evaluated a machine-learning model: a recommender system for ERC20 tokens. This is not just an academic exercise — you can head over to Similar Coins to get token recommendations for your wallet now.
We also integrated with 0x Instant so that you can go ahead and seamlessly buy the tokens that are recommended to you. If you do go ahead with a transaction, we take a 0.5% cut. We‘re not doing this to get rich (it’s a humble $0.50 for every $100 you transact) —think of it as a proof of concept for how data products on the blockchain might make money.
Hopefully it goes without saying, but these “recommendations” are not investment advice in any way. They are simply statistical associations based on which tokens frequently co-occur in Ethereum wallets.
Why build a token recommender?
Recommender systems are well-known to be useful in domains with large item inventories. This is simply because they help you find what you (probably) want, faster.
A classic example is Amazon.com and their “people who buy X also buy Y” recommendations. The main reason Jeff Bezos first focused on books was exactly because it’s the retail category with the biggest number of unique items. In other words, you could argue that Amazon would not have gotten to where they are today if it hadn’t been for recommender systems.
In our case, the inventory of tokens (or cryptoassets more broadly) has exploded in the last few years. Coinmarketcap.com lists ~2000 coins, while Etherscan lists nearly 1000 “top ERC20 tokens” and >160,000 deployed token contracts.
We think there might be room for improvement in token discovery, going beyond just sorting by market cap.
Evaluating our token recommender
We won’t go into the implementation in this post — you can check out Evgeny’s detailed post here on our first modeling attempt if you’re interested. You can also just head straight to our repo. In this post we’ll keep it high-level and focus on how we evaluated the recommender system offline, and improved it to the point where it was ready for launch.
In brief, we’re using a collaborative filtering model, and our user-item matrix is effectively a wallet–token matrix. We will often refer to wallets as “users” even if real-world users and wallets are not 1:1 — users can have multiple wallet; wallets can have multiple users.
First attempt: Balances as % of total token supply
Initially we wanted to keep our data pipeline as simple as possible, using only one data source: the blockchain transactions. We wanted to represent a user’s preference for tokens via their token balance. The problem is that tokens can have wildly different supplies, meaning that the raw balances are meaningless, as they cannot be compared across different tokens. Our idea to counter this was to create a standardised balance unit: % of total token supply. For example, a user might hold 0.01% of all the XYZ tokens available, and also 0.002% of all ABC tokens.
When we evaluated precision and recall on a test set with this approach, we found that performance was close to random (that is, the recommender was terrible). A qualitative evaluation revealed that we were pretty much recommending a small set of unknown sh*tcoins to everyone.
So we quickly pivoted and decided we needed to bring in token prices.
Second attempt: Balances in $USD
Balances denoted in $USD intuitively makes a lot more sense, and offline evaluation confirmed our intuition.
The first evaluation with k = 5 recommendations per user gave us a precision of 1.3% and a recall of 5.3%. With about 1000 tokens in our inventory, this is a lot better than (uniform) random performance.
In machine learning, it’s always important to have a meaningful benchmark, and a random benchmark is not really useful in our case. So we instead created a popularity-recommender which simply recommends the k most popular tokens to everyone. If we couldn’t beat this benchmark, we might as well just recommend these top k tokens to everyone instead of shipping a recommender system — that would be a lot easier!
Popularity is a common benchmark for recommender system, and in many domains it’s really not that easy to beat. Indeed, in our case we could only get to a performance that was 40% worse than the popularity-recommender! For k = 5, the popularity had a precision / recall of 2.3% / 9.4%. Compare that to our WALS model which only made it to 1.3% / 5.3%.
Third attempt: Replacing Tensorflow with LightFM
We quickly realized that we would not make big enough leaps by only tuning hyperparameters. So we decided to change the model altogether. We switched to LightFM’s Weighted Approximate-Rank Pairwise (WARP) loss for implicit feedback learning-to-rank. That’s a bit of a mouthful, but essentially learning-to-rank is more directly suitable for what we are trying to do: we want to show you 5 tokens we think you might be interested in.
This was a lot more promising, and after some hyperparameter tuning, our best performing model had a precision / recall of 4.8% / 20.1%. These metrics were more than twice as high as the popularity-recommender. Finally we could actually deliver useful personalized recommendations!
Try our recommender at SimilarCoins.com
All of the above is just offline evaluation. But the real test comes when users start interacting with the recommendations.
So: please visit Similar Coins and test the token recommender for yourself. Let us know what you think! As you might be able to tell, we are data folks — not frontend developers 😅 but please do let us know what you think about the recommender in the comments below.
Reflections on (open) data products for blockchain
It’s worth taking a step back to think about how this type of personalization is different from what you normally see on the web. When the Facebooks, Amazons and Googles of the world provide you with recommended content, they do that using your personal data, stored in their closed databases. In fact, it’s the main excuse they have to collect and store data about you.
The token recommender on the other hand, is basically BYOD — bring your own data. We don’t collect any data about your transactions, and we have no idea who you are. Yet we can provide you with personalized content.
Beyond existing sources of market data and dashboards, the area of open data products for blockchain is largely unexplored. We look forward to seeing more Ethereum data products in the months and years to come.
Thanks for reading — feel free to 👏👏👏 if you liked this article!