# EUM'23 talk: So you want to implement a bug fix or feature in EasyBuild framework...

- 1h slot
- remote, Alex as speaker (Wed 26 April 2023 at 14:00 CEST)
- talk format: slides or docs?
- new page under "For developers/contributors"

## Contents

- structure of EasyBuild framework
- block diagram of packages/modules
- using Mermaid: https://squidfunk.github.io/mkdocs-material/reference/diagrams
- old overview in slide 11 at https://users.ugent.be/~kehoste/EasyBuild_introduction_JSC-workshop_Oct14.pdf
- main (option parsing, robot to resolve deps + filter + order, iterate over stuff to install one by one, etc.)
- easybuild.main (start point)
- easybuild.tools.options (option parsing)
- easybuild.framework.easyconfig.tools (robot)
- iterate over installatons (build_and_install_software)
- build_and_install_one
- create easyblock instance (app) using parsed easyconfig
- app.run_all_steps
- EasyConfig class
- represent a parsed easyconfig file (self.cfg in an easyblock)
- EasyBlock
- abstract class
- does not implement configure/build/install steps
- step-wise installation procedure
- diagram
- via EasyBlock.run_all_steps
- toolchain mechanism
- sets up build environment
- loads modules for toolchain + (build) deps
- set env vars like $CC, $CFLAGS, ...
- easybuild.tools.toolchain.* (general logic to set up build env)
- toolchain.py
- compiler.py
- mpi.py
- easybuild/toolchains/*.py: toolchain definitions
- toolchain components:
- easybuild.toolchains.{compiler,mpi,linalg,fft}/*.py
- install RPATH wrappers
- interaction with modules tool (Lmod)
- easybuild.tools.modules: talks to Lmod (avail, load, etc.)
- easybuild.tools.module_generator: produces Tcl or Lua text (contents of module file)
- extensions support
- Extension class => instance for each extension
- installation via EasyBlock.install_extensions called by EasyBlock.extensions_step
- Extension.run method (will be renamed in EB 5.0)
- ExtensionEasyblock: "wrapper" class for stuff that can be installed as extension or stand-alone
- PythonPackage
- as extension: ExtensionEasyblock => uses Extension class
- as stand-alone installation: ExtensionEasyblock => uses Easyblock class
- easystack support (experimental)
- easybuild.framework.easystack
- API (easybuild.framework, easybuild.tools, etc.)
- https://docs.easybuild.io/api/easybuild
- easybuild.tools.filetools
- GitHub integration
- easybuild.tools.github
- functions called from easybuild.main
- Hooks
- to customize EasyBuild framework: https://docs.easybuild.io/hooks
- supported via easybuild.tools.hooks
- run_hook spread across the code (mostly in EasyBlock steps)
- reporting bugs
- include traceback/error => easy to search
- retain backward compatiblity at all times (except when fixing bugs)
- tests are a hard requirement for changes to framework
- new feature => test that covers it
- bug fix => test that triggers the bug, and verifies the fix
- test/framework/*.py
- test easyconfigs, dummy easyblock implementation, ...
- end-to-end tests in test.framework.toy_build
- run via:
- python -O -m test.framework.suite test_filter_by_name => run specific tests, by name
- python -O -m test.framework.modules => run entire test subsuite
- example: functions being added into systemtools (from Raxml easyblock)

----

# EasyBuild framework overview

## Structure

## Workflow

how an eb command is processed

## Key packages, modules, classes, functions

### EasyConfig class

### EasyBlock class

### Toolchain mechanism

### EasyBuild configuration

- build_option function
- tools/options.py to define configuration setting
- tools/config.py to be able to query setting from anywhere via build_option function

### Modules tool + module generator

### Extensions

### Easystack support

### GitHub integration features

### Hooks

## Opening issues

### Reporting bugs

- include error messages, tracebacks
- EasyBuild configuration + OS info
- description of what are you trying to do
- how to reproduce the problem

### Feature requests

- ...

## Development

### Fixing bugs

### Implementing features

### Writing tests

----

Example Mermaid diagram

https://squidfunk.github.io/mkdocs-material/reference/diagrams/#using-sequence-diagrams

sequenceDiagram
  autonumber
  Alice->>John: Hello John, how are you?
  loop Healthcheck
      John->>John: Fight against hypochondria
  end
  Note right of John: Rational thoughts!
  John-->>Alice: Great!
  John->>Bob: How about you?
  Bob-->>John: Jolly good!

----

stateDiagram-v2
    easybuild.main --> tools
    
    state "easybuild.tools" as tools{
        [*] --> options
        
        state "easybuild.tools.options
        fa:fa-caret-right parsing of options
        fa:fa-caret-right set_up_configuration()" as options
        
        options --> tools.ec
        
        state "easybuild.framework.easyconfig.tools
        fa:fa-caret-right parse_easyconfigs()" as tools.ec
        
        tools.ec --> robot
        
        state "easybuild.tools.robot
        fa:fa-caret-right resolve_dependencies()" as robot
    }
    
    robot --> build_install
    
    state "build_and_install_software()" as build_install
    
    build_install --> framework
    
    state "easybuild.framework" as framework{
        direction LR
        [*] --> build_install_one

        state easybuild.framework.easyblock {
            
            state "build_and_install_one()" as build_install_one
            
            build_install_one --> get_easyblock
            
            state "EasyBlock
            fa:fa-caret-right run_all_steps()" as easyblock
            
            get_easyblock --> easyblock
        }
        
        state easybuild.framework.easyconfig {
            state "get_easyblock_class()" as get_easyblock
        }
    }
    #easyblock --> hooks
    #hooks --> easyblock