Reliability of Octan Reputation Ranking and Scores
How we prove Octan's Reputation Ranking Syste
Last updated
How we prove Octan's Reputation Ranking Syste
Last updated
To validate RELIABILITY of Octan Reputation Ranking and Scoring algorithms, we use Kendall Tau Correlation (KTC), a mathematical standard to compare and measure the difference and similarity between two ranking lists/systems (higher KTC score means greater similarity and lower difference). KTC score measures how Octan Reputation Ranking (ORR) correlates with other on-chain statistics (i.e. total Txns and Receives, Degree and Indegree which describe oriented connections and interactions between accounts).
Firstly, it is clear that low correlation (under 0.1) means ORR algorithms are not good at extracting social and relative characteristics among accounts and their transactions on-chain. However, too high correlation (over 0.8) means the ranking is also not significantly more useful than or distinguished from simple normalized statistics. ORR essentially has correlation scores between 0.1-0.64 for all comparing pairs, pretty good according to the expectation.
All ranking lists match ORR Intuition (a) with different levels.
Correlation with “Total Txns” is lowest because Txns don’t present exact relations between accounts (e.g. Bod sends 10 transactions to Alice but only worthed 1 degree added for both). Correlation with “Degree” is higher but still low because it includes “OUT-transfers” which sends value to others. Note that by ORR Intuition (d), sending value should result in a lower reputation score.
Correlation with “Total Receives” is pretty good because it matches with ORR Intuition (b). Alice receiving transactions from Bod means that Alice is important to Bob. However, “Total Receives” still count replicated oriented connections. Correlation with “Indegree” is highest because it matches ORR Intuition (b, d) and presents exact relations among accounts (i.e. matching Intuition (a) at highest level).
Correlation scores with samples cut on the ranked tops (not the full lists) are high because the tops have many connections, i.e. the local subgraphs are much denser, while the entire graph is super sparse. It is rather hard to have good handles with extremely sparse partitions.
In this section, we consider ARB token airdrop as a case study to prove the worthiness and use-case of Octan Reputation Scores (RS) applied for airdrop events. The following figure shows a high similarity between Reputation Ranking distribution and airdrop-received distribution. Note that the zero-RS wallets and uncertain EOA group (892,356 wallets ~ 20.1% over all EOAs), and a part of the low-RS group does NOT receive any ARB airdrop. This confirms that Octan Reputation Score scheme is suitable to classify INELIGIBLE wallets (EOAs) for airdrop campaigns.
Octan Team has investigated several scenarios to develop reputation ranking algorithms being resistant to Sybil attacks & manipulation effectively.
Malicious guys can conduct Sybil attack or spamming transactions by creating many satellite accounts, then transferring penny tokens to a targeted account. However, this cannot result in a significant impact in general, because:
Satellite accounts have extremely low reputation, thus they CANNOT boost the reputation score of the targeted account significantly.
Cost (gas & operation) exceeds potential benefits.
Other manipulations:
Reduce Reputation scores of other accounts: NO WAY
Self-transfer within one account to create sinkage: filtered out
Make iterated cycle-transfers: gas cost and algorithm-penalty
