Sybil Report on Gitcoin Grants Rounds 14

Summary

A total of 36% of the Gitcoin users, representing 27% of the contributions, were flagged during R14. The Sybil Incidence during this round is significantly higher than R13, with an estimate of being 1.6x higher (lower / upper boundaries being between 1.2x and 2.0x)

The Flagging Efficiency was 162% (lower boundary: 143% and upper boundary: 187%) which means that the combined process is over-flagging sybils compared to what humans would do.

Notes

As per other rounds, Sybil Incidence seems to be higher during the beginning of it. We had a large jump on the final evaluation and that was not qualitatively detected on other rounds.

Parameters

  • Algorithm Aggressiveness: 20%
  • Threshold for marking human evals as "Sybil or "Not Sybil": 90%

Previous Reports

Statistics

Sybil Incidence & Detection

Sybil Incidence

Estimated Sybil Incidence per Evaluation Round

  • Estimated Sybil Incidence: 22.1% +/- 3.0% (95% CI)
  • Estimated Sybil Users: 9916 (between 8585 and 11247 w/ 95% CI)
  • Flagged Users Fraction: 35.8%
  • Flagged Sybil Users: 16073

Sybil Flags

  • ML generated flags: 9358
  • Heuristic generated flags: 6074
  • Human provided flags: 641
  • Total flags: 16073

Sybil Evaluations

Relative distributions of scores per category
TODO: upload histogram

Total users evaluated by humans: 2729 (6.1% of total)

Users marked as true by humans: 641 (23.5%)
Users marked as false by humans: 2088 (76.5%)

Total users evaluated by heuristics: 6519 (14.5% of total)
Users marked as true by heuristics: 6074 (93.2%)
Users marked as false by heuristics: 445 (6.8%)

Total users evaluated by algorithms: 35638 (79.4% of total)
Users marked as true by algorithms: 9358 (26.3%)
Users marked as false by algorithms: 26280 (73.7%)

Total users evaluated: 44886 (100.0% of total)
Users marked as true: 16073 (35.8%)
Users marked as false: 28813 (64.2%)

Comparison between Evaluation / Prediction / Aggregate score

Summary Statistics on Scenarios

Contributors

Total Contributions (original): 623400
Total Contributions (modified): 623400
Total Contributions (removed): 167988

Matched Contributions (original): 623400
Matched Contributions (modified): 455412
Matched Contributions (removed): 0
Change: 167988
Total Contributors (original): 44870
Total Contributors (modified): 28814
Total Contributors (removed): 16057

Sum of USDT Amount (original): 3507425.77
Sum of USDT Amount (removed): 879342.51

Median of Median USDT Amount per User (original): 1.14
Median of Median USDT Amount per User (modified): 1.20
Median of Median USDT Amount per User (removed): 1.10

Median Contribution Count per User (original): 8.00
Median Contribution Count per User (modified): 9.00
Median Contribution Count per User (removed): 6.00

Mean Count per User (original): 13.89
Mean Count per User (modified): 21.64
Mean Count per User (removed): 10.46

Histogram of the Contribution per User aggregates

Select a repo