Sybil Report on Gitcoin Grants Rounds 13

tags: gitcoin Reports

Updated by March 2022

Authors: Danilo Lessa Bernardineli

Summary

An total of 11.9% of the Gitcoin users, representing TODO% of the contributions, were flagged during R13. The Sybil Incidence during this round is significantly lower than R12, with an estimate of being 70% of it was before.

The Flagging Efficiency was 84% (lower boundary: 77% and upper boundary: 93%) which means that the combined process is underflagging sybils compared to what humans would do.

Links

• Past reports
  • GR12: https://hackmd.io/HXQ3yWfBQN6QGQX4cmOAxQ
  • GR11: https://hackmd.io/kOEbdDNeSdCkYXGY2wJExQ
  • GR10: https://hackmd.io/XdCMH5DtRHyPNLCGDFZ-oQ
  • GR09: https://hackmd.io/smLQ-QXdTse2dz1c6-Z85w

Parameters

Algorithm Aggressiveness: 30%
Threshold for marking human evals as "Sybil or "Not Sybil": 90%

Algorithm Aggressiveness tunes how much sensitive / specific the flagging model should be. When it is set to 50%, then it will be optimized for accuracy, while closer values to 0% means that it will be optimized for minimizing false detection rate

Statistics

Sybil Incidence & Detection

Sybil Incidence

Estimated Sybil Incidence per Evaluation Round

• Estimated Sybil Incidence: 14.1% +/- 1.3% (95% CI)
• Estimated Sybil Users: 2453 (between 2227 and 2680, 95% CI)

---

• Flagged Users Fraction: 11.9%
• Flagged Sybil Users: 2071

Sybil Flags

• ML generated flags: 53
• Heuristic generated flags: 1067
• Human provided flags: 951
• Total flags: 2071

Sybil Evaluations

Relative distributions of scores per category
TODO: upload histogram

---

Total users evaluated by humans: 6405 (36.8% of total)
Users marked as true by humans: 951 (14.8%)
Users marked as false by humans: 5454 (85.2%)

---

Total users evaluated by heuristics: 1180 (6.8% of total)
Users marked as true by heuristics: 1067 (90.4%)
Users marked as false by heuristics: 113 (9.6%)

---

Total users evaluated by algorithms: 9818 (56.4% of total)
Users marked as true by algorithms: 53 (0.5%)
Users marked as false by algorithms: 9765 (99.5%)

---

Total users evaluated: 17403 (100.0% of total)
Users marked as true: 2071 (11.9%)
Users marked as false: 15332 (88.1%)

---

Comparison between Evaluation / Prediction / Aggregate score

Summary Statistics on Scenarios

Contributors

Total Contributions (original): TODO
Total Contributions (modified): TODO
Total Contributions (removed): TODO

---

Matched Contributions (original): TODO
Matched Contributions (modified): TODO
Matched Contributions (removed): TODO
Change: TODO
Total Contributors (original): TODO
Total Contributors (modified): TODO
Total Contributors (removed): TODO

---

Sum of USDT Amount (original): TODO
Sum of USDT Amount (removed): TODO

---

Median of Median USDT Amount per User (original): TODO
Median of Median USDT Amount per User (modified): TODO
Median of Median USDT Amount per User (removed): TODO

---

Median Contribution Count per User (original): TODO
Median Contribution Count per User (modified): TODO
Median Contribution Count per User (removed): TODO

---

Mean Count per User (original): TODO
Mean Count per User (modified): TODO
Mean Count per User (removed): TODO

---

Histogram of the Contribution per User aggregates