_Environmental Data Science book_ ⛰ 🌳 🏙️ ❄️ 🔥 🌊 Infrastructure maintainers working group [PUBLIC ARCHIVE]

--- tags: eds-book, maintainers-session --- # _Environmental Data Science book_ ⛰ 🌳 🏙️ ❄️ 🔥 🌊 Infrastructure maintainers working group [PUBLIC ARCHIVE] ## Meeting 29-07-22 * [name=Alejandro] * [name=Anne] ### Goal - Draft documentation of the notebook templates ### TODO - move README draft to README - move config.json to .github - have a single use of `notiz-dev/github-action-json-property@release` - create templates for R, Julia - clean templates to remove badges - more mantainer ### Notes - [README](https://docs.google.com/presentation/d/1VT5z2F8foPHAogJyvIjfWOb3r_e_myOaXE2o5J7gbhA/edit?usp=sharing) ## Meeting 12-03-23 * [name=Alejandro] * ENV variables should be passes in docker actions using -E (see example in the Seviri [notebook](https://github.com/Environmental-DS-Book/wildfires-exploration-fires_seviri/blob/2163508ea3c09bb678554f8c5d7582663ae6b004/.github/workflows/render.yaml#L54). * datashader problem with defaults [channel](https://discourse.holoviz.org/t/datashader-import-fails-with-numba-error/5004/3) * add testing, it runs render but file isn't updated ### TODOS * nightly-build (use CRON) > TODO * hash or uniqueID to notebookrepository names, similar to JDH (see [here](https://github.com/jdh-observer/jdh001-SLCj9T3MsrEk)) ## Meeting 29-07-22 * [name=Alejandro] * [name=Anne] ### Goal - [X] test integration with CMIP6 example (: ### Template - Single template - workflow - step 1, personal repo - create a PR - check PR and binder - if it's ok, then merge - transfer - Environmental-DS-Book - step 2, organisation repo - transfer - create manually quay image - add robot permissions, write - set the secrets in the repo, of the robot - QUAY_USERNAME - QUAY_PASSWORD - add/invite the notebook author/contributor in the repository - fill the config file TODO - documentation in README - steps with nice images, see [Pluto](https://github.com/JuliaPluto/static-export-template) - move config.json to .github - have a single use of `notiz-dev/github-action-json-property@release` - create templates for R, Julia - clean templates to remove badges - more mantainer ### Resources - https://github.com/alan-turing-institute/environmental-ds-book/tree/master/book/community/templates - if: github.repository_owner == 'Environmental-DS-Book' - See example: https://github.com/galaxyproject/galaxy/blob/dev/.github/workflows/build_container_image.yaml - Pluto template https://github.com/JuliaPluto/static-export-template ## Meeting 22-07-22 * [name=Anne] * [name=Alejandro] ### Recap * Login * IMAGE_NAME: name of the image. Example - myusername/myContainer. If not supplied, this defaults to <DOCKER_USERNAME>/<GITHUB_REPOSITORY_NAME> or <GITHUB_ACTOR>/<GITHUB_REPOSITORY_NAME>. * Examples * https://github.com/binder-examples/r * https://github.com/binder-examples/demo-julia * Step * create quay repo, same name as GH repo * associate the general robot account * rerun workflow * TODO * only rebuild when changes in the environment/runtime/toml * repo2docker * way to pass repo2docker * add in config.json * kernel * quay repo * Future * Use CLI * Julia & R > jl and R-markdown/R-notebooks > convert to ipynb ## Meeting 15-07-22 ### Sign-up * [name=Anne] * [name=Alejandro] ### Recap - EnvDS book - How to use local images per repository, or better Quay API images - create new repo, new version ### Solutions - Create separate image per notebook - https://github.com/Environmental-DS-Book/test2i2c/blob/9793620380493c288696c68f799a89148bb32a1f/.github/workflows/build.yaml#L26 - Single image, tag notebook name - try persistent identifier, check RoHub, [W3ID](https://w3id.org/) - [ADDITIONAL_TAG](https://github.com/jupyterhub/repo2docker-action#optional-inputs) - `context.repo.repo` - `context.repo.owner` ## Meeting 04-07-22 ### Sign-up * [name=Anne] * [name=Alejandro] ### Recap #### EnvDS book organisation - Add new GH action to ignore build if not changes, otherwise build. - How contributors create image without registering to Quay or DockerHub. - Can we use locally? do we care pushing to Quay? - Automated Quay image creation in `build.yaml` - Set image name and version in GitHub Actions in `render.yaml` ### Goal: - Working action for rendering the notebook without authentication :heavy_check_mark: ### Decide - Option 1: Simple, create binder badge, manually inspect if runs (only one workflow) - Option 2: Complex, caching Binder, run notebook (render) ![](https://i.imgur.com/CkVlMEn.png) ### Resources - [name=Anne]: - https://github.com/jupyterhub/repo2docker-action/blob/master/create_docker_image.sh - https://github.com/addnab/docker-run-action - https://github.com/addnab/docker-run-action#run-an-image-built-by-a-previous-step - addnab/docker-run-action - [name:Alejandro]: - [Scipy](https://github.com/scipy-conference/scipy_proceedings/pull/692) - [Blog post](https://blog.neater-hut.com/scipy-is-partnering-with-joss-part-1.html) ### TODO - Personal repository - Check parsing error when running the local docker in option 2 - EnvDS book - How to use local images per repository, or better Quay API images - create new repo, new version ## Meeting 24-06-22 ### Sign-up * [name=Anne] * [name=Alejandro] ### Recap: - Repo2docker works - Created [render.yaml](https://github.com/Environmental-DS-Book/test2i2c/blob/main/.github/workflows/render.yaml) ### Goal: - Working action for generating rendered file :heavy_check_mark: ### Resources Example [docker-run-action & github-push-action](https://github.com/Torimune29/cppcodegen/blob/484e5932bb1398880ccde2ab15668edb2498e225/.github/workflows/deploy.yml) ### TODO - Add new GH action to ignore build if not changes, otherwise build. - How contributors create image without registering to Quay or DockerHub. - Can we use locally? do we care pushing to Quay? - Automated Quay image creation in `build.yaml` - Set image name and version in GitHub Actions in `render.yaml`