tags: Spearbook

Spearbook: Audit Process

This document is meant to serve as a guide for auditors engaging in Spearbit audits. Note that client and project team will be used interchangeably.

High level breakdown of the process:

Roles

Note:

• Information about promotions can be found here: https://github.com/spearbit/proposals/discussions/3

Junior Security Researcher

Junior personnel with less than one year of web3 security experience and IT background. Must be guided by SR's and LSR's.

• Small InfoSec background, technical knowledge required (junior sysadmin level).
• Aids during the report generation process, therefore familiarity with the report-generation repo is required

Technical skills and ideal knowledge to have:

• Confident reading and writing Solidity code.
• Confident with how the Ethereum network works.
  • POW, POS.
  • Block construction.
  • TX propagation and mempool.
  • Nodes and clients.
  • Familiar with other EVM blockchains.
• Confident using Etherscan.
• Confident using GitHub.
• Confident using IDEs (Remix, HardHat, Foundry, Brownie, etc..).
• Confident with blockchain security concepts and common vulnerabilities.
• Confident with most common ERCs and proxy patterns.
• Confident with common DeFi mechanics (AMMs, lending, collateral, staking, etc..) and DeFi protocols (i.e., uniswap, compound, bancor, olympus,etc..)
• Familiar programming back end and front end applications with web3 or ethersJS.
• Familiar with ethereum clients and how they work.
• Familiar with basic EVM concepts and compilers.
  • High level understanding of CALL and STATE.
  • Difference between solidity > and < 0.8.0 versions.
  • Gas optimizations.
• Familiar with basic token economics.
• Completed Ethernaut, DamnVulnerableDeFi or any other blockchain CTF.
• Completed SECUREUMs bootcamp and participated on a RACE finding multiple issues.
• Can write Proof of Concepts for his own findings.
• Can write recommendations to fix security issues following best practices.
• Can write reports out of findings.
• Can document a protocol's architecture.

Soft Skills

Soft Skills
Can communicate his/her point clearly, both written and verbally.
Can communicate professionally with the client.
Can take initiative and be proactive.
Can work in a team, assist SR's and LSR's during an engagement.
Familiar with Blockchain history and culture. (e.g., BTC & ETH whitepapers, hacks, twitter marketing ops, psyops, celebrities and rugpulls…)

Responsibilities

Responsibilities
Going through issues when presenting to the client.
Identifying issues during the engagement.
Verifying issues are properly formatted according to the standard.
Generating reports.
Writing about the protocol's architecture.
Making questions and learning.

---

Associate Security Researcher

Junior to mid tier personnel with more experience than a regular Junior Security Researcher.

• Less than a year in blockchain security.
• Web3 engineering or InfoSec background.
• Technical skills (previous tech job experiences).

Technical skills and ideal knowledge to have:

Builds on top of the Junior Security Researcher's mentioned above with deeper knowledge of each topic.

Soft Skills

Soft Skills
Can communicate his/her point clearly, both written and verbally.
Can communicate professionally with the client.
Can take initiative and be proactive.
Can work in a team, assist SR's and LSR's during an engagement.
Familiar with Blockchain history and culture. (e.g., BTC & ETH whitepapers, hacks, twitter marketing ops, psyops, celebrities and rugpulls…)

Responsibilities

Responsibilities
Going through issues when presenting to the client.
Identifying issues during the engagement.
Verifying issues are properly formatted according to the standard.
Writing about the protocol's architecture.
Asking questions and learning.
Reviewing client fixes.

---

Security Researcher

Professional personnel with more than one year of experience in web3 security and InfoSec or Software Engineering background. Must not be guided.

• Web3 engineering or InfoSec background, technical knowledge required (medium sysadmin and smart contract developer level).

Technical skills and ideal knowledge to have

• Proficient reading and writing Solidity code.
• Proficient using GitHub, analyzing code bases and searching for vulnerabilities in open sourced repositories.
• Confident programming back end and front end applications with web3, ethersJS or other SDK.
• Confident with Web2 security attacks: node DDOS, server exploitation, phishing, Nation State hacking campaigns, etc..
• Confident conducting on-chain investigations using alternative tools (beyond etherscan).
• Confident with MEV practices: PGAS, front running, arbitrages, liquidations, sandwiching, liquidity sniping, tx censoring, bots…
• Confident using IDEs (Remix, HardHat, Foundry, Brownie, etc..).
• Confident writing tests and using tools (Foundry tests, fuzzing campaigns, formal verification,etc..).
• Confident writing and reproducing web3 exploits.
• Confident with blockchain security concepts, common vulnerabilities and edge cases.
• Confident with common ERCs and their use across the ecosystem, proxy patterns, reading EIPs and specifications.
• Confident with common DeFi mechanics (AMMs, lending, collateral, staking, etc..), DeFi protocols (i.e., uniswap, compound, bancor, olympus,etc..), NFTs, and identifying forks / reused code.
• Confident with developer tooling.
• Confident with system monitoring (Tenderly, Forta, OZ defender, etc.. ).
• Familiar with ethereum clients and how they work.
• Familiar with advanced EVM concepts and compilers.
  • Yul.
  • Understanding of the ethereum yellow paper.
  • CREATE2, deterministic deployments.
  • Reverse Engineering, smart contract decompilation and debugging.
  • Solc and optimizers.
  • Initialization and Runtime Code.
  • Stack manipulation and memory management.
  • Basic opcodes: CALL, STATICALL, DELEGATECALL, REVERT, etc..
  • Gas optimizations.
  • Handle assembly{} blocks.
• Confident understanding protocols token economic design.
• Completed Ethernaut, DamnVulnerableDeFi or any other blockchain CTF.
• Completed SECUREUMs bootcamp and participated on a RACE finding multiple issues.
• Can write Proof of Concepts for his own findings.
• Can write recommendations to fix security issues following best practices.
• Can write reports out of findings.
• Can review bugs and client fixes.
• Can document a protocol's architecture.

Soft Skills

Soft Skills
Can communicate his/her point clearly, both written and verbally.
Can communicate professionally with the client.
Can take initiative and be proactive.
Can work in a team, guide Junior Security Researchers and follow LSR's lead during an engagement.
Can conduct research, gather information and present conclusions.
Familiar with Blockchain history and culture. (e.g., BTC & ETH whitepapers, hacks, twitter marketing ops, psyops, celebrities and rugpulls…)
Confident with offensive and defensive security methodologies.

Responsibilities

Responsibilities
Guiding and following LSR's lead.
Going through issues when presenting to the client.
Identifying issues during the engagement.
Ensuring issues are properly formatted according to the standard.
Reviewing client fixes.

---

Lead Security Researcher

Senior personnel with more than four years of experience in web3 security, InfoSec, IT and / or solid web3 background. Guides others.

• Extensive experience reading/writing code to spot unusual patterns and estimating risk (probability/impact).

Technical skills and ideal knowledge to have

• Proficient reading and writing Solidity code.
• Proficient using GitHub, analyzing code bases and searching for vulnerabilities in open sourced repositories.
• Proficient writing tests and using tools (Foundry tests, fuzzing campaigns, formal verification,etc..).
• Proficient understanding protocols token economic design.
• Proficient writing and reproducing web3 exploits.
• Proficient with blockchain security concepts, common vulnerabilities and edge cases.
• Proficient with common ERCs and their use across the ecosystem, proxy patterns, reading EIPs and specifications.
• Proficient with common and custom DeFi mechanics (AMMs, lending, collateral, staking, etc..), DeFi protocols (i.e., uniswap, compound, bancor, olympus,etc..), NFTs, and identifying forks / reused code.
• Proficient conducting on-chain investigations using alternative tools (beyond etherscan).
• Proficient with MEV practices: PGAS, front running, arbitrages, liquidations, sandwiching, liquidity sniping, tx censoring, bots…
• Confident with basic computer science algorithms.
• Confident programming back end and front end applications with web3, ethersJS or other SDK.
• Confident with Web2 security attacks: node DDOS, server exploitation, phishing, Nation State hacking campaigns, etc..
• Confident with developer tooling.
• Confident with system monitoring (Tenderly, Forta, OZ defender, etc.. ).
• Confident with ethereum clients and how they work.
• Confident with advanced EVM concepts and compilers.
  • Can handle all assembly{} blocks without major misses.
  • Aware of construction gas costs.
• Can write Proof of Concepts for his own and other's findings.
• Can write recommendations to fix security issues following best practices.
• Can review bugs and client fixes.
• Can document a protocol's architecture.
• Aware of standards and best practices to compare and recommend.

Soft Skills

Soft Skills
Can communicate in english at business proficiency level.
Can communicate his/her point clearly, both written and verbally.
Can communicate professionally with the client.
Can communicate complex issues in a simple way.
Can take initiative and be proactive.
Can perform managerial tasks, coach Junior Security Researchers and SR during an engagement.
Can conduct research, gather information and present conclusions.
Confident with Blockchain history and culture. (e.g., BTC & ETH whitepapers, hacks, twitter marketing ops, psyops, celebrities and rugpulls…)
Confident with offensive and defensive security methodologies.
Can answer team and client questions.
Can identify threat vectors and weak spots.
Up to date with industry news and developments.
Up to date with new exploits and vulnerabilities.
Time management and making sure the engagement progresses according to the timeline.
Does not take anything for granted.

Responsibilities

Responsibilities
Guiding Junior Security Researchers and SR's.
Going through issues when presenting to the client.
Identifying issues during the engagement.
Reviewing client fixes.
Assuring audit quality.
Assessing major risks and recommending solutions.
Assessing engagements costs, effort, scope, depth and time.
Making sure issues are written in time.

---

Payment

1st weekday of month and 15th of month sent to the ethereum address you provided to the core team.

Payments will be processed for every auditor that has completed an audit since then.

1. Offers will be posted in request-for-auditors.

Auditors usually react with "

" based on interest and availability. A beta version of an application dedicated to improve this process is being tested.

Requests follow a standard format and are designed to provide enough information about an offer:

• Name: Name of the client (project).
• Description: Small and concise description about the project.
• Timeline: When will the engagement be conducted.
• Staff wanted: Number of auditors needed for the engagement.
• More Info: Extra relevant information pertaining the offer and client.

FAQ: Is it possible to clearly define and / or describe:

1. When will the audit start exactly?
A: Not always. Many times starting dates depend on auditor availablity; It is not possible to define a start date before a team has been assembled.

2. The deadline for which every auditor will know if he has been selected or not?
A: Same week the offer has been posted.

When a request has been crossed i.e: crossed, you can assume the offer has been fullfilled and a team assembled.

Note: In this context the core team will only reach out to you if selected.

2. When a match is found, core team members will reach out to you.

Selections are based on:

• Knowledge compatibility.
• Waiting queues.
• Personal interests.

We try to assemble the best team for a job and include those who have spent more time waiting to get on an audit. If you have any particular field of interest feel free to reach out and let us know because it is an influencing factor.

e.g: If someone has a particular interest (and / or expertise) and wants to learn assembly, EVM or bridges, it will be more likely to be selected for such hypothetical audit than someone who is only interested on auditing NFT projects or heavy math protocols.

Q: What if i dont have any particular interest?
A: You may be eligible either ways based on expertise, waiting times and community contributions.

3. You will be added to two discord channels to optimize communication.

As soon as the SOW (Statement Of Work) has been signed by the client you will be added to 2 channels before the audit starts:

• An internal channel only for the audit team in the C:\AUDITORS\AUDITS category.
  • Used for internal discussions.
  • Audit channels can be in one of four phases at a time:
    • 🟢 Ongoing phase.
    • 🟡 Stand by / Waiting phase.
    • 🔵 Fixes and Report generation phase.
    • 🔴 Ended phase.
  • example: 🟢-PROJECT_NAME-audit-team

• A shared collaboration channel together with the project team in the C:\AUDITORS\SHARED category.
  • Used to communicate with the client.
  • 🔄 icon indicates it is a collaboration channel between Spearbit audit team<>Client
  • example: 🔄-PROJECT_NAME-collab

4. A kick off call with the project team, core team and auditor team will be scheduled to discuss deadlines, scope and answer relevant questions.

Although optional, many auditors like to prepare in advance for the kick off call by looking at relevant documentation and source code (if applicable; Sometimes private audits dont release source code until start day or documentation remains incomplete).

• Example of things to look at, include but are not limited to:
  • Project white paper.
  • Previous audit reports.
  • Project specification document.
  • Project documentation both for end users and developers.
  • GitHub repository.
  • A list of known external services used by the protocol (Aave, Compound, Balancer…)

During the call you are encouraged to ask any question you may have to the project team. Some of the points discussed during these meetings may include:

1. Self-introductions.
2. Timeline.
3. Scope.
4. Information about the project.
5. Codebase overview / walkthrough.
6. Protocol high level overview.
7. Attack surface.
8. Focus points.

5. When the audit starts you will be added to a GitHub repository created inside the spearbit-audits organization.

• This GitHub repo will contain the code in scope as well as any relevant documentation inside README.md

Project team members will be invited as well, therefore it is important to maintain a professional attitude and chose words wisely.

Let's assume for the sake of the following example that the client has requested an audit on 2 extremely important implementations: MyBank.sol and Vault.sol

I) spearbit-audits/PROJECTNAME audit repository

Inside spearbit-audits, a repository with the name of the client / project name will be created. In this repository you will find different branches often times proportional to the amount of files in scope. If a file has more than 500 lines of code we may split it to minimize latency in Github API requests when pulling issues for copywriting.

If applicable, README.md will also contain extra documentation about the project being audited.

See and read through the audit template here.

II) Pull Requests

In the Pull Requests section you can see the 2 files in scope and number of comments per file.

III) Comments

As soon as you find a potential issue or have a question about some part of the code you’re encouraged to leave a single comment (not a PR review) on the affected LOC of the PR. These comments should be short and to the point - they do not need to have the structure of a formal report issue yet. If the question is to the project or a specific person, feel free to @-tag them or ping them in Discord about it with the comment link.

❔ This incremental audit approach of commenting on the code as soon as a discovery is made helps the project team address and clarify issues earlier in the audit process. For them, it’s less overwhelming than receiving all issues at the end, and it helps reduce the waiting time for the fixes as they can start working on them sooner. For us, we have a common place right next to the code to discuss issues.
🔍 You don’t have to review the code on the PR itself. You can of course review it in an IDE with extensions. Some auditors also prefer to initially not look at issues others have found to be less biased. But we ask you to upstream your issues from time to time for the mentioned benefits.

IV) Issues

Once everyone has done their review and left comments on the PRs, comments are turned into issues. When the GitHub issue is created, reply to the PR comment with the issue link and resolve the comment.
The repo is already set up with the correct template and must follow the standard format:

• Context:
  Link to the affected LOC. It should be a GitHub permalink of the audit commit. In the repo of the client (not on the spearbit repo we are doing the PR on), make sure to browse the files of the audit commit, then click on the three dots and choose “Copy permalink”:
  

• Description:
  Clear and concise description of the issue and Proof Of Concept if applies.

• Recommendation:
  If there is a specific code recommendation, use syntax highlighting either by using solidity or diff code blocks. Please dedent the code so it starts to the left.

• Project-Name:
  Section used to record the project answer pertaining the issue, and / or any other relevant communications, usually after fixes.

• Spearbit:
  Spearbits final answer regarding the issue. Has it been fixed, ignored, recommendation followed…?

V) Labels

Mark the issue with the correct label based on severity. Issues will be reviewed and labeled as Fixed (if the project has fixed the issue) or Acknowledged (if the project has Acknowledged the issue but not taken any further action).

Mark the issue with ReadyForReport to confirm the issue is ready for the client report.

VI) Example of an issue fixed by developers, reviewed and ready for report:

6. Close Up Call

Close up calls can last ~1 hour and are scheduled in order to allow the project team ask the auditor team questions about findings and remediations.

Note: During these calls the auditor team screenshares and walks through issues while answering any questions posed by the project team.

Additionally, if the project team requires it, a draft report can be generated for their internal purposes.

7. Fixing period

After the close up call, if the client agrees to conduct a fix review, the security researchers will estimate the amount of work it will take to review the fixes. An SOW will be sent to the client for this.

Note: This phase should not take much effort nor a lot of time have to be spent on it.

Ask the project team to create one PR per issue to make reviewing the fixes easier.

8. Final Report

After the fixing period, a final report will be generated by Junior Security Researchers and copywrited by a core team member before final delivery.

9. End

The end… or is it? Ready for your next audit?