---
# System prepended metadata

title: Notes on development of bot for EESSI software layer

---

# Notes on development of bot for EESSI software layer

- HackMD for meeting in context of MultiXscale: https://hackmd.io/z6hu9gkoR6G1yCichemorA

## Working group members

- Bob Dröge
- Kenneth Hoste (GitHub: @boegel)
- Thomas Röblitz

## Next meeting

- *(none planned yet)*

## Open questions

If you have any questions that should be discussed during the next meeting, add them here

- (Kenneth) This is an example question
- how do we ensure that we can trust artifacts being built by 3rd parties?
    - signing content/results
- how do we handle overlapping builds launched simultaneously?
    - 1st: ask EB what needs to build, checks if overlapped with ongoing build (PR), logs information somewhere
    - what if sources of dependencies change between building for different PRs
    - check if existing dependencies might be replaced with any ingest? ... what if the replacement is what has been requested?
- do we want a human to start bot working after a pull request? maybe, at least at the start, e.g., to avoid too large builds
    - do we need policies ...
- if bot reacts on easystack file events it would only work with EasyBuild, do we want to make it more generic from the start (or keep that for the future)?


## Meeting notes

### 2022-05-10

slides: https://github.com/EESSI/meetings/blob/main/meetings/eessi-bot-software-layer-meeting-20220510.pdf

#### Attendees

- Thomas (University of Bergen, NESSI)
- Dan (Univ. of Trømso, NESSI)
- Jacob (Univ. of Oslo - NESSI)
- Jörg (Imperial College, EESSI)
- Bob (University of Groningen, EESSI)
- Kenneth (Ghent Univerisity, EESSI)
- Terje (University of Oslo, NESSI/EESSI)

#### Notes

- Kenneth going through slides (link to be added)
    - slide 4: current workflow
        - credentials: we could use host-based credentials, becomes irrelevant with bot? better keep both to max security
        - building full stack or single package? right now new packages are added on top of the existing stack by adding a section in the build script for the full stack; then existing packages are just skipped (EB detects it automatically)
        - easystack files being used? only at the very end to detect if anything is missing
        - right now script is needed to deal with missing features of easystack files (e.g., no ability to use PRs)
    - slide 5: tasks for bot in context of EESSI
        - also see https://github.com/EESSI/software-layer/issues/149
        - build step also creates tarballs to be ingested --> allows (easier) testing before it is actually ingested
        - **build** and **deploy** must have tasks; *test* and *verify* are optional
    - slide 6: designing a bot for EESSI
        - in the future we may need/want a master bot controlling the "build" bots
    - slide 7: bot implementation
        - submit jobs for stages? maybe having a local mode and a slurm mode
            - current backend works with slurm, but could be adapted to work with other backends (PBS, fork)
        - how can we make the bot stateless?
            - events with a timeout? cronjob? how to know what to check for? how to handle failures?
            - events are just CURL commands?
            - PyGHee may need a way to discriminate among different sources of events
    - slide 8:
        - automatic-build.sh idea was that this would be customisable via site-config file
    - slide 9: PyGHee
        - helps in focusing on handling events (relieving developer from logistics around)
        - multiple events are handled simultaneously?
            - possibly `waitress` handles that, multithreading being used?
            - should check with a small test that just sleeps for 1 hour and a new event is being created
        - idea: name of directory to work in to handle an event contains commit id + PR number
            - directory should contain all information, input data, scripts, results for handling the event (good for easier inspection, replay, ...)
        - good starting point: play with PyGHee
    - slide 11: A bot for EasyBuild, EESSI, and beyond
        - maybe focus on EESSI first
    - slide 12: development setup
        - bot could update itself when its code got updated
    - slide 13: development plan
        - 

