(Backlog) Proposed Process Improvements

tags: gitcoin Notes

Updated by October 2021

Authors: Charlie Rice, Nick Hirannet

• Automate the evaluator notification (future)
• Automate distribution of evaluator assignments (future)
• Be able to run SQL queries in reasonable time (less than 10 min to pull 500,000 contributions). Create data lake or warehouse that can be used instead of Metabase. If cannot be done, need other credentialing solution (currently Jesse has credential to access) (in R12 scope)
• Standardize value of amount_median, amount_mean to one currency (dollars?) (done)
• Feature Engineering Process (in R12 scope)
• Model Selection Process (future)
• Loss Function Selection / Cross Validation Process (future)
• Change to Explicit three-valued logic to heuristic: sybil, not-sybil, do not know (future)
• Refine human evaluator instructions (clarify logic of the columns) (in R12 scope)
• Describe the Anti-Sybil process framework - cycles of work (oversight -> operations -> oversight) (in R12 scope)
  • eg. first two months for oversight & slow work and the third month for operational & fast work
    • Oversight: can include rituals & time-frames for reviewing past assumptions (eg, loss functions & models)
    • Operations: defining & preparing for the round