---
# System prepended metadata

title: Solidity Underhanded Contest Judging Process

---

# Solidity Underhanded Contest Judging Process

*Judges Call on 2020-10-20*

Attendees: 
- Alex
- Austin
- Chris
- Franzi
- Goncalo
- Stefan

**Agenda**

1. We only have two submissions so far. There will be no need for much judging if there are no submissions so, please: 
    * Remind all your friends & colleagues that wanted to take part!
    * Spread the word in your forums and social networks!


2. How was judging done in the initial contest in 2017? Does anybody remember? What did you like and didn't like?

*Notes from 2017 judging process:*
- Directory with all submissions
- Google form used for judging?
- Everybody just randomly picked submissions to judge
- Spreadsheet for tracking which submission received how many reviews etc
- Evaluation from Chris' side was clustered into negative / neutral / positive

3. How should we share the submissions amongst each other? Which tool do we want to use?
    * Upload all submissions to a github repo 
    * Use syncthing
    * Other?

--> Github

4. How do we want to split up submissions / how many submissions can you tackle per person?
    * If we receive <10 submissions:
    * If we receive up to 20 submissions:
    * If we receive >20 submissions:

--> How many people should review a submission? At least 2-3 reviewers for each submission. (2 enough if agreement / 3 if no agreement)

5. Which judging criteria will we use (feel free to add to it!)?
    * Plausibility
    * Originality
    * Simplicity (short & clean code)

- How obvious is the exploit? / How well is it hidden?
- Severity of the exploit (mostly giving full control)
- Compiler outputs / warnings? --> Must compile of course; but do we take into account SMT Checker result; third party audit tools: Do we take those in account?

6. Until when can we be done with judging and announce the winners? (Initial target was end of November)



7. Am I missing anything? :) 


Other questions:
- Should we also allow Solidity v0.6?
- 