# Notes on development of bot for EESSI software layer

- HackMD for meeting in context of MultiXscale: https://hackmd.io/z6hu9gkoR6G1yCichemorA

## Working group members

- Bob Dröge
- Kenneth Hoste (GitHub: @boegel)
- Thomas Röblitz

## Next meeting

- *(none planned yet)*

## Open questions

If you have any questions that should be discussed during the next meeting, add them here

- (Kenneth) This is an example question
- how do we ensure that we can trust artifacts being built by 3rd parties?
- signing content/results
- how do we handle overlapping builds launched simultaneously?
- 1st: ask EB what needs to build, checks if overlapped with ongoing build (PR), logs information somewhere
- what if sources of dependencies change between building for different PRs
- check if existing dependencies might be replaced with any ingest? ... what if the replacement is what has been requested?
- do we want a human to start bot working after a pull request? maybe, at least at the start, e.g., to avoid too large builds
- do we need policies ...
- if bot reacts on easystack file events it would only work with EasyBuild, do we want to make it more generic from the start (or keep that for the future)?

## Meeting notes

### 2022-05-10

slides: https://github.com/EESSI/meetings/blob/main/meetings/eessi-bot-software-layer-meeting-20220510.pdf

#### Attendees

- Thomas (University of Bergen, NESSI)
- Dan (Univ. of Trømso, NESSI)
- Jacob (Univ. of Oslo - NESSI)
- Jörg (Imperial College, EESSI)
- Bob (University of Groningen, EESSI)
- Kenneth (Ghent Univerisity, EESSI)
- Terje (University of Oslo, NESSI/EESSI)

#### Notes

- Kenneth going through slides (link to be added)
- slide 4: current workflow
- credentials: we could use host-based credentials, becomes irrelevant with bot? better keep both to max security
- building full stack or single package? right now new packages are added on top of the existing stack by adding a section in the build script for the full stack; then existing packages are just skipped (EB detects it automatically)
- easystack files being used? only at the very end to detect if anything is missing
- right now script is needed to deal with missing features of easystack files (e.g., no ability to use PRs)
- slide 5: tasks for bot in context of EESSI
- also see https://github.com/EESSI/software-layer/issues/149
- build step also creates tarballs to be ingested --> allows (easier) testing before it is actually ingested
- **build** and **deploy** must have tasks; *test* and *verify* are optional
- slide 6: designing a bot for EESSI
- in the future we may need/want a master bot controlling the "build" bots
- slide 7: bot implementation
- submit jobs for stages? maybe having a local mode and a slurm mode
- current backend works with slurm, but could be adapted to work with other backends (PBS, fork)
- how can we make the bot stateless?
- events with a timeout? cronjob? how to know what to check for? how to handle failures?
- events are just CURL commands?
- PyGHee may need a way to discriminate among different sources of events
- slide 8:
- automatic-build.sh idea was that this would be customisable via site-config file
- slide 9: PyGHee
- helps in focusing on handling events (relieving developer from logistics around)
- multiple events are handled simultaneously?
- possibly `waitress` handles that, multithreading being used?
- should check with a small test that just sleeps for 1 hour and a new event is being created
- idea: name of directory to work in to handle an event contains commit id + PR number
- directory should contain all information, input data, scripts, results for handling the event (good for easier inspection, replay, ...)
- good starting point: play with PyGHee
- slide 11: A bot for EasyBuild, EESSI, and beyond
- maybe focus on EESSI first
- slide 12: development setup
- bot could update itself when its code got updated
- slide 13: development plan
-