Comparative Report: Hardhat vs Ape

Both Hardhat and Ape are popular frameworks within the Ethereum development community. They provide tools and features that facilitate smart contract development, deployment, and interaction. This report aims to present an in-depth comparison of these two frameworks to help Lido-on-Ethereum engineering teams make an informed choice.

Background

While working on Lido V2, Lido-on-Ethereum team recognized a series of pain points that need urgent attention. Our existing infrastructure, namely lido-dao and scripts, while functional, has revealed some clear areas for improvement. Addressing these will not only boost our productivity but also enhance the quality of our deployments and voting process. This necessitates evaluating tools and frameworks that can alleviate these issues. Among the potential candidates are Hardhat and Ape Ethereum development frameworks. Before diving into the comparative analysis of these two platforms, let's shed light on the specific challenges we currently face:

For Contract Development:

1. Deploy Whack-a-mole: We're constantly troubleshooting deployment issues, making the process unpredictable and time-consuming.
2. Lack of Conventions or Style Guide: The absence of a unified style guide or convention has led to inconsistencies in code, making collaboration and review processes more challenging.
3. Poor Test Structure: Our current test structure lacks organization and clarity, which affects maintainability and comprehension.
4. Slow Tests: Test execution times are considerably longer than desired, slowing down the development feedback loop.
5. Deficient Test Tooling: We lack sophisticated tooling for gas tests, fuzzing, invariants, etc., which impedes our ability to perform thorough and efficient testing.
6. Suboptimal Developer Experience in JS: The absence of features such as typing, async-await structures, and BigNumber handling makes JavaScript development less intuitive and more error-prone.
7. Messy Utilities: Our utility functions and libraries are disorganized, leading to redundancy and inefficiencies.
8. Dependabot Notifications Overlooked: Critical updates and vulnerabilities flagged by Dependabot often go unnoticed, potentially putting our codebase at risk.

For DAO Operations:

1. Discontinuation of Brownie: Our reliance on Brownie has become a concern since it's no longer being maintained.
2. Brownie-specific Caching Issues: We've encountered caching problems that seem specific to Brownie, causing data retrieval and transaction challenges.
3. Difficulty Parsing Big Omnibuses: Large and complex transactions or omnibuses are hard to parse and interpret.
4. Slow Tests: Just as with contract development, the tests related to DAO operations are sluggish and hinder rapid iteration.
5. Challenges in Integrating Regression Tests: Our current framework complicates the integration of regression tests, which is vital for ensuring our changes don't inadvertently introduce new issues.

Given these challenges, our objective is to determine whether Hardhat or Ape can provide solutions to these pain points, improving our overall development process and infrastructure.

A note on Foundry

Foundry carves a niche in the Ethereum development landscape with its unique offerings tailored to bolster the development, testing, and deployment of decentralized applications and smart contracts.

1. Rust: Unlike many other Ethereum tools built in JavaScript or Python, Foundry is grounded in Rust. Rust, known for its performance efficiency, memory safety, and concurrency capabilities, imparts Foundry with a robust backbone. This choice of language ensures high-performance operations, reducing overhead and potential bottlenecks during development.
2. Versatile Development Environment: While Foundry is rooted in Rust, it provides support for both Solidity and Vyper, the leading languages in Ethereum smart contract development. This dual-language support offers teams a broad spectrum of development options.
3. Modularity is Key: Foundry's design philosophy revolves around modularity. Developers can cherry-pick specific components or tools from the framework, tailoring their development environment to their needs without fully committing to a complete migration.
4. The Anvil Network: An integral part of Foundry, the Anvil network is a specialized development environment. It streamlines smart contract interactions, testing, and debugging. Our team could leverage the Anvil network independently to enhance our contract deployment and interaction capabilities, even if we don't fully adopt Foundry.
5. Emphasis on Testing: Foundry doesn’t cut corners when it comes to testing. It introduces advanced testing capabilities, including fuzz testing. By subjecting the system to a barrage of random data, fuzz testing aims to unearth coding errors and potential security vulnerabilities, ensuring the resilience and security of smart contracts.

Given Foundry's strengths, it stands as a potent tool in the Ethereum development arena. For our team, while a wholesale shift to Foundry might require significant resources due to adapting to its Rust-based environment and nuances in Solidity testing and scripting, there’s scope to partially adopt its offerings. By assimilating components like the Anvil network or its fuzz testing features, we can strategically address and mitigate our existing challenges.

Hardhat vs Ape: Comparative Review

Development Environment and Features

Hardhat

• Local Network: Hardhat comes equipped with a local Ethereum network, allowing developers to test and run their code before deploying it to live networks.
• Node-agnostic: Hardhat does not bind developers to a particular Ethereum node, offering flexibility in development and deployment processes.
• Extensibility via Plugins: The framework can be easily extended with plugins, providing developers with customization options tailored to their needs.
• Rich Ecosystem of First- and Third-party Plugins: Beyond its core offerings, Hardhat boasts a vast ecosystem of both first-party and third-party plugins, enhancing its functionality manifold.
• Supports Dual Setup with Foundry: Developers can set up Hardhat alongside Foundry, maximizing the benefits from both frameworks.
• Test Coverage Works Out-of-the-box: Hardhat offers out-of-the-box test coverage, ensuring developers can readily gauge the quality and reliability of their code.
• In-solidity console.log: Debugging in Hardhat is made simpler with in-built Solidity console logging.
• VS Code Extension: To further streamline the development process, Hardhat offers a dedicated VS Code extension, providing an integrated environment for coding, testing, and deploying.
• Testing through Mocha + Chai: Leveraging popular JavaScript testing frameworks, Hardhat uses Mocha and Chai for its testing suite, offering a familiar environment for many developers.

Ape

• No Local Network: Ape doesn’t offer a built-in local Ethereum network, which might require developers to look for external solutions for preliminary testing, e.g. Hardhat network, Ganache, Anvil.
• Node-agnostic: Like Hardhat, Ape is also node-agnostic, ensuring developers aren't confined to a specific Ethereum node during development.
• Extensible via Plugins: Ape supports extensibility through plugins, allowing developers to expand its functionality as per project requirements.
• Essential First-party Plugins: While Ape might not have as vast an ecosystem as Hardhat, it does come with essential first-party plugins to cater to core development needs.
• Prioritizes Vyper: Ape places a distinct emphasis on the Vyper language, making it a preferred choice for projects that lean heavily on Vyper for smart contract development.
• In-vyper Dev Comments: Developers can leverage in-Vyper comments for code assertions.
• No Solidity Test Coverage Yet: As of now, Ape lacks test coverage for Solidity, which might pose challenges for projects that primarily use Solidity.
• Interactive Debugging: Ape boasts interactive debugging features, simplifying the troubleshooting process and enabling developers to pinpoint and resolve issues efficiently.

Languages

Hardhat Typescript

• Strict Typing with TypeScript + typechain: TypeScript offers a statically typed environment. With the added advantage of typechain, Hardhat allows developers to generate TypeScript typings for smart contracts, ensuring type safety and minimizing runtime errors.
• Autocompletion: Given TypeScript's statically-typed nature, developers can benefit from autocompletion features in IDEs, speeding up the development process and minimizing potential syntactical errors.
• Unreadable Error Traces: One of the drawbacks of TypeScript, especially when combined with certain libraries or complex applications, is that error traces can sometimes be extensive and hard to decipher.
• ES2020 BigInt with ethers v6: With the introduction of BigInt in ES2020 and its integration in ethers v6, developers can handle large integers seamlessly, which is crucial when dealing with cryptocurrency values and computations.
• async-await: The async-await structures introduce complexities in error handling and can lead to unintentionally blocking code if not used judiciously.

Ape Python

• Weak Optional Typing: Python is fundamentally dynamically typed, but with the introduction of type hints, developers can opt for weak optional typing. This offers some of the benefits of static typing while maintaining Python's dynamic nature.
• Expressive: Python is renowned for its readability and expressiveness. With its concise syntax, developers can often write fewer lines of code to achieve the same functionality compared to other languages.
• Error Traces: Python's error traces are generally clear and concise, allowing developers to quickly pinpoint the source of issues and correct them.
• Testing with Pytest: Ape benefits from Python's rich testing ecosystem. Pytest stands out as a versatile testing tool that offers simple syntax for writing tests.
• Fixtures: With Pytest in Ape, developers can use fixtures to set up and tear down test environments, ensuring tests run in a consistent and isolated context.
• Parametrization: Another feature of Pytest, parametrization allows developers to run a test multiple times with different arguments, ensuring comprehensive test coverage with minimal code repetition.

Solidity support

Hardhat

• First-class Solidity Support: Hardhat places Solidity at the forefront of its framework, providing comprehensive support tailored to the needs of Solidity developers.
• Support Starts from 0.4.11: Hardhat's support for Solidity begins from version 0.4.11, covering a broad spectrum of Solidity versions and ensuring compatibility with a wide range of smart contracts.
• Supports Different Hardforks: With Ethereum undergoing several hardforks over the years, Hardhat's ability to support different hardforks ensures developers can adapt their contracts to various network conditions.
• In-solidity console.log: One of the standout features, the in-solidity console.log, facilitates debugging by allowing developers to print messages directly from within their Solidity code.
• Language Support for Solidity via VS Code Plugin: The integration with a VS Code plugin provides enhanced language support for Solidity, offering a more interactive and efficient coding experience.

Ape

• Solidity Compiler Plugin: Ape supports Solidity through a plugin, indicating that while Solidity is supported, it might not be as deeply integrated as in Hardhat.
• No Test Coverage for Solidity: A significant drawback, Ape currently lacks test coverage for Solidity, which could pose challenges in ensuring contract robustness and security.
• Custom Errors are Not Decoded: This could hinder the debugging process, as developers might struggle to understand specific errors thrown by the contract, given that during the V2 delivery, the team adopted the convention of using custom errors instead "require" statements.
• Some Call Trace Issues: We can discovered certain call trace issues, which could complicate the troubleshooting process.

Vyper support

Hardhat

• Vyper Compiler Plugin: Hardhat's support for Vyper is primarily through a plugin, suggesting that while Vyper is supported, it might not be deeply integrated like its support for Solidity.
• Earliest Supported Version 0.2.0: Hardhat's support for Vyper starts from version 0.2.0. While this provides compatibility with many contracts written in recent versions of Vyper, older Vyper contracts might not be compatible.

Ape

• Vyper-first Approach: Ape's emphasis on Vyper is evident, offering a framework tailored for developers primarily working with Vyper. This prioritization could be beneficial for smaller standalone Vyper projects.
• All Vyper Versions Available: Ape provides support for all Vyper versions, making it extremely versatile and ensuring compatibility with any Vyper-based smart contract, regardless of when it was written.
• Test Coverage Added Recently: Ape's recent addition of test coverage for Vyper is a notable improvement, aiding developers in ensuring the robustness and security of their Vyper contracts.
• In-vyper Comments: A unique feature that facilitates inline documentation and explanations within the codebase.

Tracing transactions

Harhat

• Tracing with debug_traceTransaction: Known to crash on larger transactions due to a buffer size limit. Buffer size limitation can be addressed by streaming data, although this might introduce additional complexity.
• Tracing Transactions with Callback Subscriptions: Capable of tracing transactions irrespective of their size, beneficial for contracts with hefty transactions. Exclusive to the Hardhat network. For other networks, it defaults to debug_traceTransaction. The method, while effective, is noted to be a bit hacky and might require developers to be extra vigilant for unexpected behaviors.

Ape

• Tracing via debug_traceTransactions: Capable of processing large transactions without crashing. This robustness can be especially useful for the DAO operations team dealing with complex omnibus votes. Traces don't include contract addresses, which might make it challenging to determine the exact contract interactions in some scenarios. By default, the trace details provided are extensive, which could be overwhelming. We might need to implement filters to make the trace logs more readable and relevant. There's a feature request in the Ape repository for displaying events in the traces, which could enhance its transaction tracing capabilities.

Extracting events

Hardhat

• Stability: No major issues have been discovered in relation to event extraction, suggesting a stable implementation.
• Ease of Use: Event extraction in Hardhat works out-of-the-box, providing developers with a smooth experience and minimal setup overhead.
• Specific Cases: Requires some minor adjustments for LogScriptCall events in omnibuses.

Ape

• Stability: The framework appears to be buggy when it comes to event extraction, which could pose challenges and increase debugging time for developers.
• Missing Events: There are instances where events from delegate calls go missing. One potential solution is loading ABIs dynamically during the process, although this might introduce some complexity. Random events like NodeOperatorAdded are inexplicably absent in some cases.
• Signature Conflicts: There are issues when two events share the same signature but have different event field names.

Node performance

Hardhat

• Node Compatibility: Mostly node-agnostic, supporting a range of nodes including Hardhat network, Ganache, and Anvil. While it is broadly compatible, some issues have been encountered with Ganache.
• Transaction Trace Handling: Ganache can process large transaction traces effectively. Impersonating accounts on Ganache seems to be finicky.
• Performance and Integration with Anvil: Local clean Anvil is noted to be the fastest in terms of performance, though forking is still slow. Perhaps RPC is the bottleneck.

Ape

• Node Compatibility: Node-agnostic with seamless compatibility across the Hardhat network, Ganache, and Anvil.
• Seamless Anvil Integration: Anvil works flawlessly out of the box with Ape, offering developers a hassle-free experience. Anvil completed the V2 upgrade in just 3 minutes with transaction traces and events. This is attributed to the use of debug_traceCall instead of debug_traceTransactions, making it significantly faster than Hardhat and Ganache's 30-minute duration for the same operation.

Quality of Life

Hardhat

• Debugging Capabilities: Features an "in-solidity console.log", enabling developers to insert debug statements directly in their Solidity code.
• Etherscan Verification: Supports programmatic Etherscan verification, aiding in contract authenticity and transparency.
• Ledger Support: Incorporates Ledger hardware wallet support, facilitating secure contract interactions
• Testing: Parallel tests are supported, but not always reliably. Resolving the inconsistencies might require significant Node.js expertise to optimize the setup.
• Dependency Management: Allows different versions of the same dependency. This flexibility is mainly reliant on internal conventions, so adherence to standards is key.

Ape

• Debugging Capabilities: Features a built-in interactive debugger that launches a console when a test fails, providing immediate feedback and easing the debugging process
• Etherscan Verification: Like Hardhat, Ape also supports Etherscan verification, ensuring contract authenticity.
• Performance: Initial startup time is somewhat slow, which could affect rapid iterative development. However, developers have promised improvements in future releases
• Parallel Testing: Supports parallel testing via the pytest-xdistr module, which can significantly reduce test run times for larger projects.
• Community & Support: The Ape team is quick and responsive both on Discord and GitHub, which can be crucial for addressing issues, understanding best practices, or suggesting and contributing new features.

Bugs

Hardhat

• Call Trace Processing: Hardhat tends to crash when dealing with large call traces.

Ape

• Redundant Recompilations: Redundant recompilations have been reported in certain scenarios. This could slow down iterative development and increase the wait time for developers.
• Missing Contracts Address in Call Tree: Contract addresses are missing from call_tree.
• Buggy Interactive Debugging: There are instances where the interactive debugger fails to load the Python context.
• Event Decoding: Ape has shown issues with decoding the NodeOperatorAdded event.

L2 Support

Hardhat

• Broad L2 Compatibility: Supports all EVM-compatible Layer 2 solutions.
• SDKs & Tooling: A majority of L2 Software Development Kits (SDKs) and tooling are written on Hardhat. This synergy ensures seamless integration and utilization of L2-focused tools within the Hardhat environment.

Ape

• L2 Plugins: Directly supports network plugins for Arbitrum and Optimism, two of the prominent L2 solutions in the market
• SDK Porting: Most L2 SDKs are written in TypeScript and need to be ported to Python for seamless integration with Ape. This porting process could introduce delays, potential translation issues, or inconsistencies.
• Non-EVM L2 Operations: There might be challenges when dealing with non-EVM L2-specific operations.

Conclusion

Hardhat, with its track record of reliability, stands out as a well-established platform within the Ethereum ecosystem. Its solid foundation in TypeScript not only streamlines the development process but also introduces a multitude of benefits, such as strict typing and autocompletion. The richness of its ecosystem, characterized by an array of plugins and tools, further accentuates its reliability. An added advantage for our team is the pre-existing use of Hardhat in the lido-dao repository.

Ape, on the other hand, is undeniably promising. While it carves a niche with its Vyper-first approach, making it attractive for specific projects, it does have certain limitations. Some features are yet to be incorporated, and there's a noted presence of bugs which, though fixable, might introduce development hiccups. Nevertheless, the development experience on Ape is commendable, bolstered by tools like its interactive debugger. It's evident that Ape thrives on community contribution, encapsulating the spirit of an enthusiast project. However, this also brings forth concerns about its longevity and consistent backing.

Taking into account the strengths and challenges of both platforms, the team is inclined towards adopting Hardhat with some enhancements from Foundry for our core infrastructure development. Hardhat's proven stability, coupled with the team's existing familiarity, makes it a pragmatic choice. However, the allure of Ape, especially for Vyper-centric endeavors, remains. It's conceivable to harness Ape for smaller isolated Vyper projects, leveraging its unique offerings while also diversifying our toolset.

Links

• Hardhat
• Ape
• Foundry
• Brownie
• lido-dao
• scripts

Appendix

Omnibuses Hardhat Implementation

Declarative & self-descriptive omnibuses

Omnibus definition

Omnibus test

Typed contracts Support

One-place protocol config

Complete omnibus traces without debug_traceTransaction usage

Keystore accounts support

Accounts management

Accounts unlock on omnibus launch

Minimal input required from deployer

Seamless support of Ganache, Hardhat and Anvile nodes

VS Code debugging

Ape

Call trace

Ape on Anvil performance