NeIC AHM 23 - Research Software Development Practice workshop

--- tags: NeIC, Events --- # NeIC AHM 23 - Research Software Development Practice workshop This page: https://hackmd.io/@coderefinery/neic-ahm-2023 Have 210 minutes available for workshop and 30 minutes for waffle break [toc] ## ToDo - [x] add "Richard Darst, Aalto University" to https://indico.neic.no/event/237/page/84-workshops - [x] RB: inform Richard about expectations/preparation - [ ] JM: Change last paragraph in Indico workshop descr: - We hope that during the workshop you will find out what exactly software development is with hands-on examples as well as improve your own research software development skills or contribute your own knowledge to the discussion. - --> software development practices related to DevOps - Jake to change and Maiken will update Indico - Closer to the date add Hedgedoc link to Indico. - [ ] Maiken: Once topics are decided - create an email to participants with info about the topics and call out for contributions - participants can add on this document their topic discussion points - short problems or short solutions - [ ] Maiken: Add hedgedoc document to indico - [ ] prefil some example discussion points - [ ] instruction to participants how to add points: "please scroll down to ... to add your points" - [x] Hedgedoc for audience: https://notes.coderefinery.org/AHM23-DevOpsWS - [ ] @MatiasJJ or @samumantha add schedule and place to - [x] @MatiasJJ share the project + possible questions + goals and dreams :nerd_face: - https://kannu.csc.fi/s/TtrYdNdKD3G9gzY - [ ] RB or @samumantha Prepare questions and suggestions for action points etc. - [ ] RB or @samumantha Prepare maybe motivation for version control # Email suggestion Dear participants of the Research Software Development Practice workshop at NeIC AHM 2023 We are looking forward to seeing you next week! We're reaching out to you for 3 main reasons 1. share updates in the agenda 2. to share a collaborative document that we will be using upfront and during the workshop - https://notes.coderefinery.org/AHM23-DevOpsWS 3. and most importantly to collect some input from you to help make the workshop a collaboration between the presenters and participants. Agenda (1. above) --------- The high-level agenda is the following - Introduction on the workshop - goals and motivation (15 min) - Software development from code-writing to deployment via GitLab/Hub - a rough overview (30 min) - Topic A: Continuous integration and deployment of a software package - Topic B: Github actions+pages for documentation and lessons - Topic C: Releasing a Python package with CI Please find more details in https://notes.coderefinery.org/AHM23-DevOpsWS Collaborative document (2. above) ---------------------------------- We will use this document:https://notes.coderefinery.org/AHM23-DevOpsWS both upfront and during the workshop for collaboration and communication. It is editable by anyone by clicking the pencil-icon. Keep an eye on this document for updates, and use it for interacting with the workshop participants. Discussion/collaboration (3. above) ------------------------------------- Each topic session aims to save 15-25 min for discussions and collaboration at the end of the session. We would like that you as audience have a think about topics A/B/C and see if you have - Own projects or use-cases or any cool things to show related to the topic - Some problem that you have that you have not solved - Some solution that you have found that you want to share - Discussion points of other types In the topic sections in https://notes.coderefinery.org/AHM23-DevOpsWS we have added a couple of examples of how we would like the discussion/interaction points to be added. The main components being - Add your initials or full name (note that anyone with link can access the document) - Describe shortly what you want to discuss/show or what question you might have - Aim at a max 5 min slot for your discussion/interaction point [End email-suggestion] ## Goals The goal of this workshop is that in the style of show-and-tell one learns about different ways of using research software development practices related to DevOps and continuous integration from members of the NeIC community. During the workshop we will hear from 3 contributors on 3 different topics, getting to know how they use Software development and Continuous integration in their job and how it helps them solve problems. The contributors will also bring up unsolved problems or unfinished solutions and engage the audience to solve these and discuss. We hope that during the workshop you will find out how software development practices can be related to DevOps with hands-on examples to improve your skills. Be prepared to ask questions, discuss and contribute to the workshop The current contributors and topics can be found here: https://notes.coderefinery.org/AHM23-DevOpsWS The goal of this workshop is that in the style of show-and-tell one learns about different ways of doing things in Git{Lab,Hub}. We aim at having 1 introduction/overview session and 3 topical demos, each of 45 minutes duration. The introduction session will give an overview of the software development process from keyboard to deployment, and what features in Git{Lab,Hub} are useful in that process. ## Target audience Primarily people that already know about the basics of Git{Lab,Hub} and use it for their software development project. They want to join the workshop to learn about and discuss solutions and cool implementations of tools related to Git{Lab,Hub} or related to other sides of the research software development process. ## How We envisage that each topic has 1 main responsible of that topic-session, but of course that main responsible is free to bring in others to contribute. The actual topics will depend on who gets the slot. Each topic should aim at a 10 minute overview/introduction, 10-20 minutes how-did-I-do-it, and 15-25 min collaboration / problem solving. The idea is to show how a certain goal was reached using the tools of Git{Lab,Hub} in order to show examples of different things one can do related to the software development process. Maybe unsolved problems, or unfinished solutions, or implementations that one would like improved are brough forward, and the idea is that the audience can engage in the development/discussion of that item. We might set up either a test-infrastructure for collaboration/implementation of solutions, and/or make available forks of real software development projects to collaborate on. --- ## Schedule Parallel Workshops are on the Tuesday, Jan 24. Overview - all times in CET: - **Part 1** - 15:00 - 16:45 - **Break** with Waffles, fruit, tea and coffee 16:45 - 17:15 - **Part 2** - 17:15 - 19:00 ### (15 min, Jake and Maiken) Practicalities, introduction, and motivation In this section we should give the idea behind the workshop and why we thought it was necessary, who will be talking and what about, what the agenda is. In the introduction we should also mention the style of the presentations (i.e presenters perspectives "how i did it") and (if) there is space for collaboration, questions and probem solving (although this depends on the speaker and what they wish to do). ### (30 min, Samantha, Matias, and Radovan) Software development from code-writing to deployment via GitLab/Hub This section is now in the hedgedoc: https://notes.coderefinery.org/AHM23-DevOpsWS --- Below older notes that can be incorporated above: Introduction to GitLab/GitHub - an overview of the procedures people can typically use, the different possibilities the tools have, the workflow a developer might follow. H. Seibold's overview slide can be the overview and the "path": from developer coding to PR to comments, to merge conflicts, to history of pushes, to GitLab/Hub CI/CD - webhooks etc. - basically a rough and not too detailed overview of some scenarios of using Git{Lab,Hub}. :::info - Briefly discuss benefits of git and GitHub - Motivation for git: - Revertability - History of what was done - Branches, working on separate features - Reproducibility (Tags?) - .. - Motivation for git*: - Collaboration (With oneself and others) - First step towards public - issues - workflows for testing, deployment etc - .. ::: ### (45 min, Maiken) Topic A: Continuous integration and deployment of a software package - Example: Show how ARC uses GitLab for building the software on multiple platforms, deploying ARC and running some basic functional tests. - 10 min overview - 10-20 min how did I do it - 15-25 min collab / problem solving ### (15 min) Q&A - taking questions from audience - or buffer if we need more time for sessions above ### (30 min) Break with Waffles, fruit, tea and coffee (16:45 - 17:15) ### (45 min, Richard and Maiken) Topic B: Github actions+pages for documentaion and lessons (ideas see below) Maiken has started: https://drive.google.com/drive/folders/1rHH6sLKmmScab-UjogUHvEJcTRD7gdMY?usp=share_link Need shortening - maybe remove the example and Richard can take that part - 10 min overview - What are GH actions and what you can do with them - 10-20 min how did I/we do it - What have CR done with them - 15-25 min collab / problem solving - Get usecases from the audience and set up one maybe (Jake): docs.csc.fi tests: https://github.com/CSCfi/csc-user-guide/tree/master/tests (Uses Travis CI) ### (45 min, Jake & Radovan) Topic C: Releasing a Python package with CI (ideas see below) - 10 min overview - 10-20 min how did I do it - 15-25 min collab / problem solving PyPI has a sample project `setup.py` vs `pyproject.toml` vs `setup.cfg` Pennylane example: how to release 3 python packages across 3 repos all with CI - pytest vs unittest and integration with CI - PyPI vs test-PyPI vs pip installing from GitHub ### (15 min, moderated by Jake) Q&A and wrap-up - Some preparation is needed but we should be flexible on this. - Bigger picture, where to start if one were to start a new project. Should we test, should we automate everything. - where to start in an existing project that does not have any testing/devops/automation yet - what is a good balance - can be moderated round-table - and/or it can be taking questions from audience --- ## Topic ideas (please and and edit) - Maiken: Show how ARC uses GitLab and GitHub to set up a public interactive test-infrastructure for ARC configuration - Radovan: Building websites and generating documentation pages - ?: How can GitLab/GitHub automate testing of software - ?: Building containers with CI - Maiken: GitHub actions - small demo showing how GitHub actions are used in a projec to sync files and folders between forks. (20min) - Jenkins (or Travis CI). Integration tools, are they worth using? - **please add more and add name to where you would like to participate** # Meeting 17-01-23 ## Agenda The goal of this workshop is that in the style of show-and-tell one learns about different ways of using research software development practices and continuous integration from members of the NeIC community. During the workshop we will hear from 3 contributors on 3 different topics, getting to know how they use Software development and Continuous integration in their job and how it helps them solve problems. The contributors will also bring up unsolved problems or unfinished solutions and engage the audience to solve these and discuss. We hope that during the workshop you will find out what exactly software development is with hands-on examples as well as improve your own research software development skills or contribute your own knowledge to the discussion. - Add more tools and devops What is the collab problem solving part Email the participants these are the topics, keep these topics in mind and if you would like to show something. "Be prepared to give feedback on the topics in person and contribute your own ideas"