Sybil Report on Gitcoin Grants Rounds 12

tags: gitcoin Reports

Updated by December 2021

Authors: Danilo Lessa Bernardineli

Summary

An total of 27.9% of the Gitcoin users, representing 21.7% of the contributions, were flagged during R12. The Sybil Incidence during this round is significantly higher than R11, with an estimate of being 2.6x higher (lower / upper boundaries being 1.6x to 5.1x).

Links

• Past reports
  • GR11: https://hackmd.io/kOEbdDNeSdCkYXGY2wJExQ
  • GR10: https://hackmd.io/XdCMH5DtRHyPNLCGDFZ-oQ
  • GR09: https://hackmd.io/smLQ-QXdTse2dz1c6-Z85w

Parameters

Algorithm Aggressiveness: 30%
Threshold for marking human evals as "Sybil or "Not Sybil": 90%

Algorithm Aggressiveness tunes how much sensitive / specific the flagging model should be. When it is set to 50%, then it will be optimized for accuracy, while closer values to 0% means that it will be optimized for minimizing false detection rate

Statistics

Sybil Incidence & Detection

Sybil Incidence

Estimated Sybil Incidence per Evaluation Round

---

Scenario A: Incl. squelches

• Estimated Sybil Incidence: 26.3% (between 24.2% and 28.4%, 95% CI)
• Estimated Sybil Users: 7635 (between 7025 and 8245, 95% CI)

---

Scenario B: No squelches

• Estimated Sybil Incidence: 16.4% (between 14.5% and 18.3%, 95% CI)
• Estimated Sybil Users: 4753 (between 4203 and 5304, 95% CI)

---

• Flagged Users Fraction: 27.9%
• Flagged Sybil Users: 8100

Sybil Flags

• ML generated flags: 3986
• Heuristic generated flags: 2811
• Human provided flags: 1571 (at most 850 squelches)
• Total flags: 8100

Sybil Evaluations

Relative distributions of scores per category

Total users evaluated by humans: 4997 (17.2% of total)
Users marked as true by humans: 1571 (31.4%)
Users marked as false by humans: 3426 (68.6%)

---

Total users evaluated by heuristics: 2811 (9.7% of total)
Users marked as true by heuristics: 2543 (90.5%)
Users marked as false by heuristics: 268 (9.5%)

---

Total users evaluated by algorithms: 21179 (73.1% of total)
Users marked as true by algorithms: 3986 (18.8%)
Users marked as false by algorithms: 17193 (81.2%)

---

Total users evaluated: 28987 (100.0% of total)
Users marked as true: 8100 (27.9%)
Users marked as false: 20887 (72.1%)

---

Comparison between Evaluation / Prediction / Aggregate score

Summary Statistics on Scenarios

Contributors

Total Contributions (original): 528233
Total Contributions (modified): 528233
Total Contributions (removed): 114923

---

Matched Contributions (original): 528233
Matched Contributions (modified): 413310
Matched Contributions (removed): 0
Change: 114923
Total Contributors (original): 28416
Total Contributors (modified): 20888
Total Contributors (removed): 7529

---

Sum of USDT Amount (original): 4981968.65
Sum of USDT Amount (removed): 621376.66

---

Median of Median USDT Amount per User (original): 2.00
Median of Median USDT Amount per User (modified): 2.00
Median of Median USDT Amount per User (removed): 1.60

---

Median Contribution Count per User (original): 10.00
Median Contribution Count per User (modified): 10.00
Median Contribution Count per User (removed): 8.00

---

Mean Count per User (original): 18.59
Mean Count per User (modified): 25.29
Mean Count per User (removed): 15.26

---

Histogram of the Contribution per User aggregates