[Terra] Bots and success rate
Intro
Determining whether the wallet is a bot or a real user was the hardest part of this bounty because there's no on-chain metadata which indicates this. Therefore, I had to derive a methodology to come up with this categorization.
Methodology
Here is the framework I used to determine a bot from a regular user:
- If the user has high number of transactions, then it's very likely a bot.
- If the user has a low success_rate, then it's likely a bot
- If the user only has few transactions, then it's very likely a user.
- If the success_rate is high and number of transactions are low, then it's likely a user.
Thresholds
The next step was choosing the thresholds for "high number of transactions", "low success rate". Here are the thresholds I chose:
- High number of transactions = 300. I believe no user does 5 transactions a day for 60 days straight. That seems like a bot behavior.
- Similarly, a user doing less than 20 transactions in less than 20 transactions is surely a user.
- If the success_rate is higher than 80%, then that is likely a user but if it's less than 40% then that is most likely a bot.
- For all other cases, I assigned the category as a bot.
Here is the SQL I came up with.
WHEN num_tx >= 300 OR success_rate < 40 THEN 'Bot'
WHEN num_tx <= 20 THEN 'User'
WHEN success_rate > 80 and num_tx < 300 THEN 'User'
ELSE 'Bot'
Observations
With the above criteria, we see that ~98% (~345K) of the wallets were categorized as users and only 2% (~7600) as bots. I think the bot number seems high but with the popularity of Terra, it's quite plausible we have that many bots running.
Conversely, comparing the number of users to the number of transactions, we can see that at every transaction level the number of users seem to be higher.
More importantly there are 986 users classified as "Bot" that have in aggregate done around 3.71M transactions.
10% of the overall failed transactions are by the bot users even through they are only 2% of the overall users as we saw in the first chart.
Now, if we look at the average success rate, we can see that the users are close to 100% - 98.6% to be exact while the bots are around 60%.
Conclusion
Based on the chosen methodology, we concluded that
- 2% of the users in the last 60 days were bots.
- These 2% users accounted for ~10% failed transactions.
- In addition, these bots accounted for 7.5M transactions compared to 5.8M for the average users.
- The average success rate for the bots was 60% compared to