Spearbook: Audit Process

###### tags: `Spearbook` # Spearbook: Audit Process This document is meant to serve as a **guide for auditors engaging in Spearbit audits.** Note that *client* and *project team* will be used interchangeably. High level breakdown of the process: ![Spearbit megabrain - Frame 1(1)](https://hackmd.io/_uploads/rJqV-3yMA.jpg) ## Roles > Note: > - Information about promotions can be found here: https://github.com/spearbit/proposals/discussions/3 ### ****Junior Security Researcher**** Junior personnel with less than one year of web3 security experience and IT background. Must be guided by SR's and LSR's. - Small InfoSec background, technical knowledge required (junior sysadmin level). - Aids during the report generation process, therefore familiarity with the [report-generation repo](https://github.com/spearbit-audits/report-generator-template_v.2) is required #### Technical skills and ideal knowledge to have: - **Confident** reading and writing Solidity code. - **Confident** with how the Ethereum network works. - POW, POS. - Block construction. - TX propagation and mempool. - Nodes and clients. - Familiar with other EVM blockchains. - **Confident** using Etherscan. - **Confident** using GitHub. - **Confident** using IDEs (Remix, HardHat, Foundry, Brownie, etc..). - **Confident** with blockchain security concepts and common vulnerabilities. - **Confident** with most common ERCs and proxy patterns. - **Confident** with common DeFi mechanics (AMMs, lending, collateral, staking, etc..) and DeFi protocols (i.e., uniswap, compound, bancor, olympus,etc..) - **Familiar** programming back end and front end applications with web3 or ethersJS. - **Familiar** with ethereum clients and how they work. - **Familiar** with basic EVM concepts and compilers. - High level understanding of CALL and STATE. - Difference between solidity > and < 0.8.0 versions. - Gas optimizations. - **Familiar** with basic token economics. - **Completed** Ethernaut, DamnVulnerableDeFi or any other blockchain CTF. - **Completed** SECUREUMs bootcamp and participated on a RACE finding multiple issues. - **Can write** Proof of Concepts for his own findings. - **Can write** recommendations to fix security issues following best practices. - **Can write** reports out of findings. - **Can document** a protocol's architecture. #### Soft Skills | Soft Skills| | -------- | | **Can communicate** his/her point clearly, both written and verbally.| | **Can communicate** professionally with the client.| | **Can take** initiative and be proactive.| | **Can work in a team**, assist SR's and LSR's during an engagement.| |**Familiar** with Blockchain history and culture. (e.g., BTC & ETH whitepapers, hacks, twitter marketing ops, psyops, celebrities and rugpulls...)| #### Responsibilities | Responsibilities | | -------- | | Going through issues when presenting to the client.| | Identifying issues during the engagement. | | Verifying issues are properly formatted according to the standard.| | Generating reports.| | Writing about the protocol's architecture.| |Making questions and learning.| <hr> ### **Associate Security Researcher** Junior to mid tier personnel with more experience than a regular Junior Security Researcher. - Less than a year in blockchain security. - Web3 engineering or InfoSec background. - Technical skills (previous tech job experiences). #### Technical skills and ideal knowledge to have: Builds on top of the Junior Security Researcher's mentioned above with deeper knowledge of each topic. #### Soft Skills | Soft Skills| | -------- | | **Can communicate** his/her point clearly, both written and verbally.| | **Can communicate** professionally with the client.| | **Can take** initiative and be proactive.| | **Can work in a team**, assist SR's and LSR's during an engagement.| |**Familiar** with Blockchain history and culture. (e.g., BTC & ETH whitepapers, hacks, twitter marketing ops, psyops, celebrities and rugpulls...)| #### Responsibilities | Responsibilities | | -------- | | Going through issues when presenting to the client.| | Identifying issues during the engagement. | | Verifying issues are properly formatted according to the standard.| | Writing about the protocol's architecture.| |Asking questions and learning.| |Reviewing client fixes. | <hr> ### **Security Researcher** Professional personnel with more than one year of experience in web3 security and InfoSec or Software Engineering background. Must not be guided. - Web3 engineering or InfoSec background, technical knowledge required (medium sysadmin and smart contract developer level). #### Technical skills and ideal knowledge to have - **Proficient** reading and writing Solidity code. - **Proficient** using GitHub, analyzing code bases and searching for vulnerabilities in open sourced repositories. - **Confident** programming back end and front end applications with web3, ethersJS or other SDK. - **Confident** with Web2 security attacks: node DDOS, server exploitation, phishing, Nation State hacking campaigns, etc.. - **Confident** conducting on-chain investigations using alternative tools (beyond etherscan). - **Confident** with MEV practices: PGAS, front running, arbitrages, liquidations, sandwiching, liquidity sniping, tx censoring, bots... - **Confident** using IDEs (Remix, HardHat, Foundry, Brownie, etc..). - **Confident** writing tests and using tools (Foundry tests, fuzzing campaigns, formal verification,etc..). - **Confident** writing and reproducing web3 exploits. - **Confident** with blockchain security concepts, common vulnerabilities and edge cases. - **Confident** with common ERCs and their use across the ecosystem, proxy patterns, reading EIPs and specifications. - **Confident** with common DeFi mechanics (AMMs, lending, collateral, staking, etc..), DeFi protocols (i.e., uniswap, compound, bancor, olympus,etc..), NFTs, and identifying forks / reused code. - **Confident** with developer tooling. - **Confident** with system monitoring (Tenderly, Forta, OZ defender, etc.. ). - **Familiar** with ethereum clients and how they work. - **Familiar** with advanced EVM concepts and compilers. - Yul. - Understanding of the ethereum yellow paper. - CREATE2, deterministic deployments. - Reverse Engineering, smart contract decompilation and debugging. - Solc and optimizers. - Initialization and Runtime Code. - Stack manipulation and memory management. - Basic opcodes: CALL, STATICALL, DELEGATECALL, REVERT, etc.. - Gas optimizations. - Handle `assembly{}` blocks. - **Confident** understanding protocols token economic design. - **Completed** Ethernaut, DamnVulnerableDeFi or any other blockchain CTF. - **Completed** SECUREUMs bootcamp and participated on a RACE finding multiple issues. - **Can write** Proof of Concepts for his own findings. - **Can write** recommendations to fix security issues following best practices. - **Can write** reports out of findings. - **Can review** bugs and client fixes. - **Can document** a protocol's architecture. #### Soft Skills | Soft Skills | | -------- | |**Can communicate** his/her point clearly, both written and verbally.| | **Can communicate** professionally with the client.| | **Can take** initiative and be proactive.| | **Can work in a team**, guide Junior Security Researchers and follow LSR's lead during an engagement.| |**Can conduct** research, gather information and present conclusions.| | **Familiar** with Blockchain history and culture. (e.g., BTC & ETH whitepapers, hacks, twitter marketing ops, psyops, celebrities and rugpulls...) | **Confident** with offensive and defensive security methodologies. | #### Responsibilities | Responsibilities | | -------- | | Guiding and following LSR's lead.| | Going through issues when presenting to the client.| | Identifying issues during the engagement. | |Ensuring issues are properly formatted according to the standard.| |Reviewing client fixes.| <hr> ### **Lead Security Researcher** Senior personnel with more than four years of experience in web3 security, InfoSec, IT and / or solid web3 background. Guides others. - Extensive experience reading/writing code to spot unusual patterns and estimating risk (probability/impact). #### Technical skills and ideal knowledge to have - **Proficient** reading and writing Solidity code. - **Proficient** using GitHub, analyzing code bases and searching for vulnerabilities in open sourced repositories. - **Proficient** writing tests and using tools (Foundry tests, fuzzing campaigns, formal verification,etc..). - **Proficient** understanding protocols token economic design. - **Proficient** writing and reproducing web3 exploits. - **Proficient** with blockchain security concepts, common vulnerabilities and edge cases. - **Proficient** with common ERCs and their use across the ecosystem, proxy patterns, reading EIPs and specifications. - **Proficient** with common and custom DeFi mechanics (AMMs, lending, collateral, staking, etc..), DeFi protocols (i.e., uniswap, compound, bancor, olympus,etc..), NFTs, and identifying forks / reused code. - **Proficient** conducting on-chain investigations using alternative tools (beyond etherscan). - **Proficient** with MEV practices: PGAS, front running, arbitrages, liquidations, sandwiching, liquidity sniping, tx censoring, bots... - **Confident** with basic computer science algorithms. - **Confident** programming back end and front end applications with web3, ethersJS or other SDK. - **Confident** with Web2 security attacks: node DDOS, server exploitation, phishing, Nation State hacking campaigns, etc.. - **Confident** with developer tooling. - **Confident** with system monitoring (Tenderly, Forta, OZ defender, etc.. ). - **Confident** with ethereum clients and how they work. - **Confident** with advanced EVM concepts and compilers. - Can handle all `assembly{}` blocks without major misses. - Aware of construction gas costs. - **Can write** Proof of Concepts for his own and other's findings. - **Can write** recommendations to fix security issues following best practices. - **Can review** bugs and client fixes. - **Can document** a protocol's architecture. - **Aware** of standards and best practices to compare and recommend. #### Soft Skills | Soft Skills | | -------- | |**Can communicate** in english at business proficiency level.| |**Can communicate** his/her point clearly, both written and verbally.| | **Can communicate** professionally with the client.| |**Can communicate** complex issues in a simple way.| | **Can take** initiative and be proactive.| | **Can perform** managerial tasks, coach Junior Security Researchers and SR during an engagement.| |**Can conduct** research, gather information and present conclusions.| | **Confident** with Blockchain history and culture. (e.g., BTC & ETH whitepapers, hacks, twitter marketing ops, psyops, celebrities and rugpulls...) | **Confident** with offensive and defensive security methodologies. | |**Can answer** team and client questions.| |**Can identify** threat vectors and weak spots.| |**Up to date** with industry news and developments.| |**Up to date** with new exploits and vulnerabilities.| |**Time management** and making sure the engagement progresses according to the timeline.| |**Does not** take anything for granted.| #### Responsibilities | Responsibilities | | -------- | | Guiding Junior Security Researchers and SR's.| | Going through issues when presenting to the client.| | Identifying issues during the engagement.| |Reviewing client fixes.| |Assuring audit quality.| |Assessing major risks and recommending solutions.| |Assessing engagements costs, effort, scope, depth and time.| |Making sure issues are written in time.| <hr> ## Payment **1st weekday of month and 15th of month** sent to the ethereum address you provided to the core team. Payments will be processed for every auditor that has completed an audit since then. ## 1. Offers will be posted in `request-for-auditors`. Auditors usually react with " :eyes: " based on interest and availability. **A beta version of an application dedicated to improve this process is being tested.** Requests follow a standard format and are designed to provide enough information about an offer: > - **Name:** Name of the client (project). > - **Description:** Small and concise description about the project. > - **Timeline:** When will the engagement be conducted. > - **Staff wanted:** Number of auditors needed for the engagement. > - **More Info:** Extra relevant information pertaining the offer and client. :::info **FAQ: Is it possible to clearly define and / or describe:** **1. When will the audit start exactly?** **A:** Not always. Many times starting dates depend on auditor availablity; It is not possible to define a start date before a team has been assembled. **2. The deadline for which every auditor will know if he has been selected or not?** **A:** Same week the offer has been posted. ::: When a request has been crossed i.e: *~~crossed~~*, you can assume the offer has been fullfilled and a team assembled. :::info **Note:** In this context the **core team will only reach out to you if selected.** ::: ## 2. When a match is found, core team members will reach out to you. Selections are based on: - Knowledge compatibility. - Waiting queues. - Personal interests. We try to assemble the best team for a job and include those who have spent more time waiting to get on an audit. **If you have any particular field of interest feel free to reach out and let us know because it is an influencing factor.** :::info **e.g:** If someone has a particular interest (and / or expertise) and wants to learn assembly, EVM or bridges, it will be more likely to be selected for such hypothetical audit than someone who is only interested on auditing NFT projects or heavy math protocols. **Q: What if i dont have any particular interest?** **A:** You may be eligible either ways based on expertise, waiting times and community contributions. ::: ## 3. You will be added to two discord channels to optimize communication. As soon as the SOW (Statement Of Work) has been signed by the client you will be added to 2 channels before the audit starts: - An **internal channel only for the audit team** in the `C:\AUDITORS\AUDITS` category. - Used for internal discussions. - Audit channels can be in one of four phases at a time: - 🟢 Ongoing phase. - 🟡 Stand by / Waiting phase. - 🔵 Fixes and Report generation phase. - 🔴 Ended phase. - example: 🟢-*PROJECT_NAME-audit-team* - A **shared collaboration channel together with the project team** in the `C:\AUDITORS\SHARED` category. - Used to communicate with the client. - 🔄 icon indicates it is a collaboration channel between Spearbit audit team<>Client - example: 🔄-*PROJECT_NAME-collab* ## 4. A kick off call with the project team, core team and auditor team will be scheduled to discuss deadlines, scope and answer relevant questions. Although optional, many auditors like to prepare in advance for the kick off call by looking at relevant documentation and source code (if applicable; Sometimes private audits dont release source code until start day or documentation remains incomplete). - **Example of things to look at, include but are not limited to:** - Project white paper. - Previous audit reports. - Project specification document. - Project documentation both for end users and developers. - GitHub repository. - A list of known external services used by the protocol (Aave, Compound, Balancer...) During the call **you are encouraged to ask any question you may have to the project team.** Some of the points discussed during these meetings may include: 1. Self-introductions. 2. Timeline. 3. Scope. 4. Information about the project. 5. Codebase overview / walkthrough. 6. Protocol high level overview. 7. Attack surface. 8. Focus points. ## 5. When the audit starts you will be added to a GitHub repository created inside the spearbit-audits organization. - This GitHub repo will contain the code in scope as well as any relevant documentation inside `README.md` :::warning Project team members will be invited as well, therefore it is important to maintain a professional attitude and **chose words wisely.** ::: Let's assume for the sake of the following example that the client has requested an audit on 2 extremely important implementations: `MyBank.sol` and `Vault.sol` ### I) spearbit-audits/PROJECTNAME audit repository Inside spearbit-audits, a repository with the name of the client / project name will be created. In this repository you will find different branches often times proportional to the amount of files in scope. If a file has more than 500 lines of code we may split it to minimize latency in Github API requests when pulling issues for copywriting. If applicable, `README.md` will also contain extra documentation about the project being audited. See and read through the [audit template here](https://github.com/spearbit-audits/audit-template). ![](https://i.imgur.com/IzcwNNi.png) ### II) Pull Requests In the **Pull Requests section** you can see the 2 files in scope and number of comments per file. ![](https://i.imgur.com/gdDE8Oc.png) ### III) Comments As soon as you find a potential issue or have a question about some part of the code **you’re encouraged to leave a single comment (not a PR review) on the affected LOC of the PR**. These comments should be short and to the point - they do not need to have the structure of a formal report issue yet. If the question is to the project or a specific person, feel free to @-tag them or ping them in Discord about it with the comment link. ![](https://i.imgur.com/5XrfhBv.png) > ❔ This incremental audit approach of commenting on the code as soon as a discovery is made helps the project team address and clarify issues earlier in the audit process. For them, it’s less overwhelming than receiving all issues at the end, and it helps reduce the waiting time for the fixes as they can start working on them sooner. For us, we have a common place right next to the code to discuss issues. 🔍 You don’t have to review the code on the PR itself. You can of course review it in an IDE with extensions. Some auditors also prefer to initially not look at issues others have found to be less biased. But we ask you to upstream your issues from time to time for the mentioned benefits. ### IV) Issues Once everyone has done their review and left comments on the PRs, **comments are turned into issues**. **When the GitHub issue is created, reply to the PR comment with the issue link and resolve the comment.** The repo is already set up with the correct template and must follow the standard format: - **Context:** Link to the affected LOC. **It should be a GitHub permalink of the audit commit**. **In the repo of the client** (not on the spearbit repo we are doing the PR on), make sure to browse the files of the audit commit, then click on the three dots and choose “Copy permalink”: ![](https://i.imgur.com/wqYVOow.png) - **Description:** Clear and concise description of the issue and Proof Of Concept if applies. - **Recommendation:** If there is a specific code recommendation, use syntax highlighting either by using ```solidity``` or ```diff``` code blocks. Please dedent the code so it starts to the left. - **Project-Name:** Section used to record the project answer pertaining the issue, and / or any other relevant communications, usually after fixes. - **Spearbit:** Spearbits final answer regarding the issue. Has it been fixed, ignored, recommendation followed...? ### V) Labels Mark the issue with the correct label based on severity. **Issues will be reviewed and labeled** as **Fixed** (if the project has fixed the issue) or **Acknowledged** (if the project has Acknowledged the issue but not taken any further action). Mark the issue with **ReadyForReport** to confirm the issue is ready for the client report. <img src="https://i.imgur.com/214Z3my.png" width="130px"> ### VI) Example of an issue fixed by developers, reviewed and ready for report: <img src="https://i.imgur.com/2dPkHDW.png" width="700px"> ## 6. Close Up Call Close up calls can last ~1 hour and are scheduled in order to allow the project team ask the auditor team questions about findings and remediations. :::info **Note:** During these calls the auditor team screenshares and walks through issues while answering any questions posed by the project team. ::: Additionally, if the project team requires it, a draft report can be generated for their internal purposes. ## 7. Fixing period After the close up call, if the client agrees to conduct a fix review, the security researchers will estimate the amount of work it will take to review the fixes. An SOW will be sent to the client for this. :::info **Note:** This phase should not take much effort nor a lot of time have to be spent on it. ::: :::warning Ask the project team to create one PR per issue to make reviewing the fixes easier. ::: ## 8. Final Report After the fixing period, a final report will be generated by **Junior Security Researchers** and copywrited by a core team member before final delivery. ## 9. End The end... or is it? Ready for your next audit?