HackMD
Document automated testing across the Holo organization, including but not limited to:
tags: holo-notes holo-notes-testing
Testing tools and frameworks
- JavaScript & Typescript
- Tryorama
- When is this used: This is for unit testing Holochain DNA's
- Used In:
- Holofuel
- SL
- JCH
- HHA
- dummy-dna
- Used In:
- When is this used: This is for unit testing Holochain DNA's
- tryCP:
- This is used for load testing holochain DNA's locally
- Used In:
- Holofuel
- SL
- JCH
- Used In:
- This is used for load testing holochain DNA's locally
- Mocha or tape
- Testing javascript services and API's
- Used In
- hpos-holochain-api
- …
- Used In
- Testing javascript services and API's
- Tryorama
- Rust Testing frameworks
- Testing rust services and API's
- Used In:
- envoy
- configure-holochain
- …
- Used In:
- Testing rust services and API's
- Pupeteer
- This is used to run integration tests on UI's
- used in
- nightly tests
- Holofuel UI tests
- used in
- This is used to run integration tests on UI's
- nixOS tests
- This is used to run unit tests on nix code i.e. holoport code
- Used In
- holo-nixpkgs
- Used In
- This is used to run unit tests on nix code i.e. holoport code
Nightly tests
Application/Subject Under Test (SUT)
- Holoport setup (not registration)
- Holo hosting
Environment (Dev. QA, Alpha, Master(Prod)
- Nightly tests are only run on Dev
- They are also run before every PR is merged into dev
Brief description of testing
- The Basic holo hosting is tested
- Log-in to host-console
- Publish and hosting a happ
- Hosted user accessing a hosted happ and making zome calls
Framework used
- puppeteer with mocha
URL/Location of Test Suite (Github, Google Drive, etc.)
https://github.com/Holo-Host/nightly-network-test/
Test cases/coverage
- Test of host console UI functions
- Sign in with host credentials
- Check if authenticated calls to holoport apis are successful
- Check if unauthenticated call to holoport api fails with 401
- Sign out host
- Test of hosted happ UI connected to holo network (Dummy DNA and UI)
- Sign out and check there's no more agent present
- Sign in and check if connected to the same host
- Receives a signal
- Calls cell and receives signal from different cell
- Calls second cell
- Check old transactions
Ownership
- Holo Devs that update holo-nixpkgs
Who owns the test suite?
- Holo Devs that update holo-nixpkgs
Who is accountable for maintenance?
- Holo Devs that update holo-nixpkgs
Run schedule
- Every night
- And before every PR merged into develop of holo-nixpkgs
Test Maintenance
- How/when are tests reviewed and updated?
Currently this is to keep tabs on the general working of our eco-system, so when work on holo-nixpkgs are made
Who is responsible for maintaining the testing framework?
~~
How often is the framework updated?
~~
Test Results/Reporting
The nightly runs are reported on the Holo-Hosting channel on MM
Process for reviewing results
Its a pass or fail result.
Who is responsible for reviewing/troubleshooting failures?
Holo devs
When/how are defects/issues reported?
This notifies us of a broken service or a regression, Based on passed experiences it breaks due to know updates and is tracked by devs
How/when are test results/trends reported to a wider audience?#
Not applicable. Only ment for internal dev use
Tryorama tests
Application/Subject Under Test (SUT)
All Holo DNAs
- Holofuel
- Holo-Hosting-app
- servieclogger
- joining-code-factory
- dummy-dna
Environment (Dev. QA, Alpha, Master(Prod)
- Only for Dev use
Brief description of testing
- This is a unit testing framework that is used to test DNA's
- They are a part of the dev life-cycle
- Its spins us multiple local holochain nodes and intracts with each other based in the applications
Framework used
- Tryorama
- Tape
- Languages: Typescript
URL/Location of Test Suite (Github, Google Drive, etc.)
- All DNA's
