Environmental Data Science book ⛰ 🌳 🏙️ ❄️ 🔥 🌊 Infrastructure maintainers working group [PUBLIC ARCHIVE]

Meeting 29-07-22

• Alejandro
• Anne

Goal

• Draft documentation of the notebook templates

TODO

• move README draft to README
• move config.json to .github
• have a single use of notiz-dev/github-action-json-property@release
• create templates for R, Julia
• clean templates to remove badges
  • more mantainer

Notes

• README

Meeting 12-03-23

• Alejandro

• ENV variables should be passes in docker actions using -E (see example in the Seviri notebook.

• datashader problem with defaults channel

• add testing, it runs render but file isn't updated

TODOS

• nightly-build (use CRON) > TODO
• hash or uniqueID to notebookrepository names, similar to JDH (see here)

Meeting 29-07-22

• Alejandro
• Anne

Goal

• test integration with CMIP6 example (:

Template

• Single template
  • workflow
    • step 1, personal repo
      • create a PR
      • check PR and binder
      • if it's ok, then merge
      • transfer
        • Environmental-DS-Book
    • step 2, organisation repo
      • transfer
        • create manually quay image
          • add robot permissions, write
        • set the secrets in the repo, of the robot
          • QUAY_USERNAME
          • QUAY_PASSWORD
        • add/invite the notebook author/contributor in the repository
        • fill the config file

TODO

• documentation in README
  • steps with nice images, see Pluto
• move config.json to .github
• have a single use of notiz-dev/github-action-json-property@release
• create templates for R, Julia
• clean templates to remove badges
  • more mantainer

### Resources

• https://github.com/alan-turing-institute/environmental-ds-book/tree/master/book/community/templates
• if: github.repository_owner == 'Environmental-DS-Book'
  • See example: https://github.com/galaxyproject/galaxy/blob/dev/.github/workflows/build_container_image.yaml
  • Pluto template
    https://github.com/JuliaPluto/static-export-template

Meeting 22-07-22

• Anne
• Alejandro

Recap

• Login

  • IMAGE_NAME: name of the image. Example - myusername/myContainer. If not supplied, this defaults to <DOCKER_USERNAME>/<GITHUB_REPOSITORY_NAME> or <GITHUB_ACTOR>/<GITHUB_REPOSITORY_NAME>.

• Examples

  • https://github.com/binder-examples/r
  • https://github.com/binder-examples/demo-julia

• Step

  • create quay repo, same name as GH repo
  • associate the general robot account
  • rerun workflow

• TODO

  • only rebuild when changes in the environment/runtime/toml
    • repo2docker
    • way to pass repo2docker
  • add in config.json
    • kernel
    • quay repo

• Future

  • Use CLI
  • Julia & R > jl and R-markdown/R-notebooks > convert to ipynb

Meeting 15-07-22

Sign-up

• Anne
• Alejandro

Recap

• EnvDS book
  • How to use local images per repository, or better Quay API images
    • create new repo, new version

Solutions

• Create separate image per notebook
  • https://github.com/Environmental-DS-Book/test2i2c/blob/9793620380493c288696c68f799a89148bb32a1f/.github/workflows/build.yaml#L26
• Single image, tag notebook name
  • try persistent identifier, check RoHub, W3ID
  • ADDITIONAL_TAG
    • context.repo.repo
    • context.repo.owner

Meeting 04-07-22

Sign-up

• Anne
• Alejandro

Recap

EnvDS book organisation

• Add new GH action to ignore build if not changes, otherwise build.
• How contributors create image without registering to Quay or DockerHub.
  • Can we use locally? do we care pushing to Quay?
• Automated Quay image creation in build.yaml
• Set image name and version in GitHub Actions in render.yaml

Goal:

• Working action for rendering the notebook without authentication
  Image Not Showing Possible Reasons

  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported

  Learn More →

Decide

• Option 1: Simple, create binder badge, manually inspect if runs (only one workflow)
• Option 2: Complex, caching Binder, run notebook (render)
  

Resources

• Anne:
  • https://github.com/jupyterhub/repo2docker-action/blob/master/create_docker_image.sh
  • https://github.com/addnab/docker-run-action
  • https://github.com/addnab/docker-run-action#run-an-image-built-by-a-previous-step
  • addnab/docker-run-action
• [name:Alejandro]:
  • Scipy
  • Blog post

TODO

• Personal repository
  • Check parsing error when running the local docker in option 2
• EnvDS book
  • How to use local images per repository, or better Quay API images
    • create new repo, new version

Meeting 24-06-22

Sign-up

• Anne
• Alejandro

Recap:

• Repo2docker works
• Created render.yaml

Goal:

• Working action for generating rendered file
  Image Not Showing Possible Reasons

  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported

  Learn More →

Resources

Example docker-run-action & github-push-action

TODO

• Add new GH action to ignore build if not changes, otherwise build.
• How contributors create image without registering to Quay or DockerHub.
  • Can we use locally? do we care pushing to Quay?
• Automated Quay image creation in build.yaml
• Set image name and version in GitHub Actions in render.yaml