Try   HackMD

2020-01-21-colchester-workshop

https://bit.ly/codimaEssex

Notes for https://ccp-codima.github.io/2020-01-21-colchester-workshop/

In this notepad, we invite you to share organisational information, and
details about what you want to work on/have worked on during the workshop.
Please contribute to the agenda of the workshop to make the most of it!
We are going to finalise the agenda in January, depending on the audience
skills and priorities, so everything in this document is subject to
discussion - suggestions welcome.

We will also use this or newly create notepads to keep notes from the
workshop, and share linkis and code snippets during the workshop.

Workshop concept

This workshop will be useful to you if you are a mathematician or computer
scientist interested in using GAP and/or SageMath in your research. You do
not necessarily intend to contribute to the development of these systems
or to submit your code to be included in them in a form of packages - what
you need is to write your own code to work with some examples and conduct
computational experiments. To do that efficiently, you not only need to
know the relevant functionality in GAP and/or SageMath, but also know how
to manage the whole process to make it as automated and reproducible as
possible.

You don't need to have any previous knowledge of the tools that will be
presented at the workshop. For collaborative practical sessions, we will
aim at organising participants in groups, so that novices will be paired
up with someone more experiences. We will start with basics in order to
boost up novices' skills and prepare everyone to more advanced topics.

For the practical sessions, you are encouraged to bring your own code from
experiment that you run with GAP or SageMath in the past, or have a
particular problem that you would like to tackle with their help. Please
bring a laptop with a Mac, Linux, or Windows operating system (not a tablet,
Chromebook, etc.) on which you have administrative privileges. A working
installation of GAP and SageMath is desirable. If you have problems with
installing them, we can try to resolve them in the beginning of the workshop,
if time permits.

Installation instructions:
- GAP: https://carpentries-incubator.github.io/gap-lesson/setup/
- SageMath:
- Linux and macOS: http://www.sagemath.org/download.html
- Windows: https://github.com/sagemath/sage-windows/releases
- Git: we suggest to use the standard Software Carpentry setup, see instructions e.g. at https://ccp-codima.github.io/2020-02-20-lancaster/#setup

The main format of workshop sessions will be live coding: instead of
using slides, the instructor will type in the code and/or commands,
run them, and explain their acions. The participants are strongly
encouraged to repeat the same steps with the instructor. We call this
"participatory live coding" or "coding along". This approach is
used by the Carpentries. Howeber, this is not a full-scale official
Software Carpentry workshop: we will use Carpentries pedagogical
approaches a lot, but will not have clearly identified lessons on
Unix shell and Git (which would take a whole day by themselves). Instead, we
will interleave elements of these into the topics and mini-projects
outlined below, making pauses when necessary to explain concepts, and
embedding practice of relevant skills into collaborative exercises.

Attendees of previous CCP-CoDiMa training events will be very welcome to
act as helpers for topics which they already know, and help others to work
through things like installation problems, error messages and unexpected
output. You don’t need to be an expert in the tools we’re covering, just
to have used one or more of them and be comfortable helping people to
troubleshoot.

Participants

If you are coming to the workshop, please briefly intoduce youself below
(keep the order alphabetic). Please indicate briefly if you have any prior
experience of using GAP and/or SageMath.

  • Erik (Madison) Bray (https://iguananaut.net):
    Research software engineer at CNRS / Université Paris-Sud. I'm one of the core maintainers of SageMath, and developer of the Windows port of Sage. Also have contributed a little bit to GAP development. I am not a researcher and do not often use the very software I develop on, but I do know many of the basics of Sage, and can help with all manner of technical issues. I am also an expert at Python, and am a regular Software Carpentry instructor.

  • Chris Jefferson (https://caj.host.cs.st-andrews.ac.uk): I am a reader at the University of St Andrews. My background is in A.I. and Computational Group Theory. I am one of the developers of the internals of GAP, including debugging and profiling code. My main (group theory related) research interestes are computationally hard problems on permutation groups, such as intersection and normaliser.

  • Olexandr Konovalov (https://olexandr-konovalov.github.io/year-archive/):
    I am a lecturer at the School of Computer Science at the University of St Andrews (and an organiser for this workshop). Mathematician by training, I started to use GAP in my research in the theory
    of group rings, and eventually became an active contributor to GAP. I am an
    instructor and an instructor trainer for the Carpentries, and the author of
    the Software Carpentry-style lesson "Programming with GAP"
    (https://carpentries-incubator.github.io/gap-lesson/) which we plan to teach
    at this workshop.

  • Alastair Litterick (https://www.essex.ac.uk/people/litte00606/alastair-litterick)
    I am a lecturer in the Department of Mathematical Sciences at the University of Essex, and a local organiser for this workshop. My primary research interests are in algebraic groups and finite simple groups, broadly understood, and this has included quite a lot of computational algebra, but primarily not in GAP (and I have no SageMath experience at all). I am increasingly interested in algebraic geometry, but have not yet attempted to apply computational methods to this.

  • Jesus Martinez-Garcia (https://jesusmartinezgarcia.net):
    I am a lecturer in mathematics at the University of Essex (and a local organiser for this workshop) and I have a background in computer engineering. I have applied some programming techniques to geometric problems, from rudimentary calculations in linear algebra and symbolic methods to, more recently, creating ad-hoc libraries that solve probelms in Geometric Invariant Theory (AKA GIT). I created a library to describe the GIT quotient of hypersurfaces in projective space in Python and more recently, with collaborators Gallardo, Moon and Swinarski, I have completed a prototype of GIT library in SageMath which describes the GIT quotient of a Projective space by any non-exceptional simple Lie group. I belive this library can be useful to many researchers in moduli spaces. My personal goal for this workshop is to understand what I need to do to make our library be part of the SageMath distribution.

  • Emilio Pierro (https://www.researchgate.net/profile/Emilio_Pierro):
    I am a Guest Teacher at the London School of Economics. My research is in finite simple group theory and I've been using GAP (and Magma) for many years to study various properties of finite groups.

  • Michael Torpey (https://mtorpey.github.io):
    I am an associate lecturer in the School of Computer Science at the University of St Andrews. My background in computational algebra led to a natural interest in GAP, and I am the author of GAP's package manager and the Memoisation package, and a co-author of the Semigroups and Digraphs packages. I have a particular interest in teaching these sorts of technologies to others, and so I'll be delighted to be participating in another Software Carpentry–style workshop!

  • Madeleine Whybrow (https://madeleinewhybrow.wordpress.com/):
    I am a postdoc at the University of Primorska in Slovenia. I use GAP and other computer algebra systems extensively in my research. I will be talking about my work developing algorithms and software for studying axial and Majorana algebras and will be around throughout the workshop to help out and answer any questions you might have.

  • James Williams (https://seis.bristol.ac.uk/~qd19176/)
    I am a postdoc at the University of Bristol. I use GAP to gain insight in my research of p-groups.

Topics

This is the initial list of topics, listed in some kind of logical dependencies
between them. Please add a topic which you would like to learn, or suggest something
what you would like to teach yourself. Some of the topics may be merged, or
incorporated into the exercise sessions as means to achieve other tasks. We will
be making the final programme in mid-January, taking into account these suggestions.

  • How to install our systems and packages (Olexandr, Erik): this may be a short slot in the beginning to check that everyone can run GAP and SageMath, and help if necessary. Some problems may be identified and dealt with later, if they can't be resolved immediately.

  • GAP Software Carpentry lesson (Olexandr, Chris, Madeleine, Michael): this is a Software Carpentry-style introduction to GAP (https://carpentries-incubator.github.io/gap-lesson/). In average, it could be taught over 4.5 hours; depending on the audience, that could be anything between 3 and 6 hours. The suggestion is to spend Day 1 morning on GAP and switch to SageMath after lunch, and then continue GAP training on Day 2. At Day 1, an absolute necessity is to cover episodes 1 and 3 of the GAP lesson from the link above.

  • Introduction to SageMath (Erik):

    Run the notebooks online: https://mybinder.org/v2/gh/ccp-codima/2020-01-21-colchester-workshop/master?filepath=notebooks%2Fintroduction_to_sage%2F

    Git repository: https://github.com/ccp-codima/2020-01-21-colchester-workshop

    Nice Sage notebooks: https://sagemanifolds.obspm.fr/examples.html

  • Working with files and directories in the terminal (Olexandr, Erik, Michael and others): this topic is usually covered at Software Carpentry workshops in more detail. We will not have a separate session on this topic here due to time constraints, but will be using it a lot during live coding sessions, making stops, when necessary, for more detailed explanations. Mastering the command line will save you a lot of time, so please ask questions, if you will need us to repeat something or explain some tricks. Thus, these skills will be reinforced via the transfer of the tacit knowledge, by watching how the instructors will work and repeating their actions on your computer. We expect to achieve this by the end of Day 1.

  • Running tests to check that code works correctly (Olexandr, Erik): for GAP, this will be introduced in the GAP Software Carpentry lesson in the Episode 4, which will be the start the GAP lesson on Day 2. It will be then revisited several times in later sessions, e.g. in the one on publishing reproducible experiments, and another about debugging and profiling in GAP. For SageMath, FIXME.

  • Version control with Git and GitHub: again, we will not run a full half-day course on this topic, like we usually do at Carpentries, but will start to introduce it from Day 2. For this, on Day 2 we should ensure that everyone has installed Git and created a free GitHub account. Then before Episode 4 on GAP, we will establish the repository for our project with the initial version of the lesson's avgord.g file, and then will keep its future versions and a test file under version control. This will provide an opportunity to learn most common Git commands, and practise a typical workflow for making changes. When we are fluent at working with our local repository, we will put the code on GitHub and practise making further changes with pushing and pulling to/from the repository.

  • Continuous integration with Travis CI: after finishing GAP Episode 4, we will have code and tests, but they will not be performed automatically. Now we can activate Travis CI integration, and add our standard setup for regression tests to the repository. We will then continue with Episode 5, checking how our changes are tested on Travis CI after being pushed to GitHub.

    • we can start with a general introduction to Travis before diving into
      the GAP specifics; an input from SageMath developers at that
      point would be great. At the moment I think there is not much state
      of the art on this, though I think a few projects are using Sage in
      Travis; I will look into it (EB)

  • Using Git and GitHub to publish code and collaborate with others (while teaching GAP Episodes 5 and 6):

    • collabrative exercises in pairs to work on the same repository where both people have write access
    • submitting pull requests to other repositories where you do not have write access (e.g. adding your details to the list of workshop participants) using GitHub web interface
    • submitting pull requests to test changes and perform code review before they are getting merged

  • Tips and pitfalls for using GAP (Olexandr): this is a lecture given using a Jupyter notebook hosted on Binder. Participants will be able to open it and run it in their browser. This way, I will combine the lecture about GAP with introducing Jupyter notebooks. Since it will be running remotely, there's no need for local installation.

  • Publishing reproducible experiments using Jupyter notebooks running on Binder (Olexandr can cover both GAP and Sage): we will now add Jupyter notebook and Binder setups to our avgord.g repository

    • Olexandr is now creating a template for publishing reproducible GAP experiments in Jupyter notebooks runnable on Binder: https://github.com/rse-standrewscs/gap-binder-template. It is possible to incorporate it into episodes 4-5 of the GAP Carpentries style lesson, or have a separate session on it, depending on time constraints and the progress achieved on Day 1.

  • Recommendations for reproducible research practices using Jupyter (Olexandr): how to avoid going from spaghetti code to spaghetti notebooks.

  • If there is an interest in using GAP and SageMath in Jupyter locally: walkthrough of the installation process (GAP: Olexandr; SageMath: Erik)

  • Automate tasks with UNIX shell scripts: again, this is something which we will not teach expliciltly, but may come across explaining Travis CI setup or starting GAP or SageMath using a shell script on a university cluster.

  • Debugging and profiling in GAP (Chris): GAP has many ways to help you understand why your code isn't working, or is performing slowly. I will give an overview of various techniques, including the 'profiling' package.

  • Run distributed computations with GAP and SageMath

    • who:

      • Olexandr (GAP, based on https://github.com/olexandr-konovalov/scscp-demo) - this is based on the example from the Software Carpentry GAP lesson, so will naturally be its continuation
      • ??? (Sage) (what do we mean here by distributed? - distributed as in shared memory parallelising vs distributed memory parallelising - AFAIK, there is no multithreading in Sage)

    • JMG: I'm actually very interested in this. I have asked to get Sage installed into our parallel computation centre and my code is potentially parallelisable. It was my understanding that this was possible, see here

  • Success stories: talks by people who have applied GAP or SageMath in their own research

    • who: Madeleine
    • who: James - I would be interested in giving a short talk (20 mins?) on how GAP helped to drive my research in my PhD.
    • who: Chris - Both developing new algorithms for permutation groups, and using these algorithms in A.I.
    • who: Jesus (SageMath for moduli spaces), happy to talk about this for 10' on whiteboard.

  • GAP package manager (Michael): an optional tool for installing and updating GAP packages from within GAP, automating a lot of the more complicated parts. Particularly useful for those needing custom packages or their latest versions ahead of including in GAP release. This might come up in other parts of the workshop, but if there is interest, Michael can present a short demo.

  • Contributing to our systems (overview, sources of information, communication channels, etc.)

    • GAP and its packages: Olexandr, Michael
    • SageMath: Erik

  • "Code is Science" (Olexandr: wrap-up talk about research software): Scientific code needs to be treated as a primary research output. For this reason, we have created the Code Is Science Manifesto, which calls for making genuine efforts to make research code open, correct, sustainable, and citable. I will discuss the main principles of the Manifesto and provide some practical tips on their implementation. The talk is intended for anyone who deals with research software, including researchers, research software engineers, and administrators.

    • also where and how to publish software and/or software papers in computational algebra

Mini-projects

These could be mini-projects to work on during practical sessions e.g.

  • collaboration using GitHub (pull requests, code review)
    • for example, submit a PR to the list of participants to specify
      your website and/or GitHub username
  • set up regression tests and continuous integration for your code
  • bring your code to archive it on Zenodo and publish your reproducible experiment on Binder
  • reviewing and further developing SageMath Software Carpentry lesson

Draft schedule

Tuesday January 21st

09:30 Registration on the NTC (North Teaching Centre) foyer (location here).

  • Since it can be tricky to get oriented around campus, Alastair Litterick will be at the Silberrad Student Centre (AKA The Welcome Centre, location here) at 09:20 to lead a party to the NTC.

Morning [NTC 1.03]

10:00 Start of the workshop

  • Welcome, housekeeping and introductions

  • Checking and helping with GAP and SageMath installations (at least up to the state sufficient to start the GAP training session; the rest of the problems can be identified and dealt with later)

  • Programming with GAP (Olexandr Konovalov): Software Carpentry-style lesson at https://carpentries-incubator.github.io/gap-lesson/

11:15 Break

11:45 Programming with GAP (continued)

Troubleshooting:

  • GapWriteIniFile what to do if it tries to write a file on drive M: which is not mounted
  • GAP error message Error, file <filename> must exist and be readable at path/to/gap/installation/lib/files.gi etc. is very confusing to a beginner - it looks like GAP is trying to find a file in the directory of GAP installation

This is the content of avgorg.g which by the end of the morning GAP session should appear in your home directory:

AvgOrd:=function ( G )
    local sum, g;
    sum := 0;
    for g in G do
        sum := sum + Order( g );
    od;
    return sum / Size( G );
end;

Q: how to enumerate "first" 100000 elements of a free group of rank 2?

A: using iterators

G:=FreeGroup("x","y");
i:=0;
for g in G do 
  Print(i, " : ", g,"\n");
  i:=i+1;
  if i>100000 then 
    break;
  fi;
od;

13:00 Lunch [NTC 1.02]

Afternoon [STEM 4.1]

14:00 Programming with SageMath (Erik Bray)

Troubleshooting Q&A:

Q: How to start SageMath in Jupyter

A: use sage -n jupyter

Q: Where are my ipynb files in Windows?

A: In your home directory in SageMath-X.Y\runtime\home\sage

15:15 Break

15:45 Programming with SageMath (continued)

16:45 Wrapping up

  • Reminder: everyone needs to install Git and create a free GitHub account for tomorrow's exercises

Evening

18:30 Workshop dinner

  • We will be having a workshop dinner on the 21st at 6.30pm in The Old Siege House Bar and Brasserie at 75 East Street, Colchester, Essex CO1 2TS. The 3-course menu is paid for by the organisers. Drinks can be purchased by attendees themselves. Due to the large number of participants we need to provide our dinner choice from the menu in advance – please fill in the form at https://forms.gle/s3rzVn6GbStb9W5c8

Wednesday January 22nd

Morning [Lakeview Room]

10:00 Overview of the agenda for the day

Announcements:

To those staying in the Wivenhoe House Hotel: we provisionally booked a table for 8 people at 8pm tonight. If you're joining us, please add your name to the list, and we will update the reservation:

  • Olexandr
  • Madeleine
  • Michael
  • Erik
  • Dean
  • Awss
  • Diego
  • Chris Jefferson
  • Rhys

10:15

GAP (Olexandr Konovalov)

Links:

Break (11:15-11:45)

11:45

James Williams: Powerfully Nilpotent Groups - Insights due to GAP

Chris Jefferson: Debugging and Profiling in GAP

  • NoMethodFound errors
  • LoadPackage errors (wrong name/uncompiled package)
  • Undocumented and ALLCAPS functions
  • Common syntax error message
  • Using "profiling" package: https://gap-packages.github.io/profiling/

Lunch (13:00-14:00) [NTC 3.02]

Afternoon [NTC 3.02]

14:00 Binderfest

Bring your GAP or SageMath code

Useful links:

Most useful shell commands:

  • pwd
  • cd
  • ls
  • mkdir
  • cp
  • mv
  • rm

Most useful Git comands

  • non-destructive (for inspection, call as many times as you wish)

    • status
    • diff
    • log

  • commands that change the state of the repository

    • add
    • commit
    • push
    • pull

  • setting up: see https://swcarpentry.github.io/git-novice/02-setup/index.html

    • to use "nano" for commit messages (once on your computer)
      • git config --global core.editor "nano -w"
    • to clone a repisitory (once for each clone)
      • git clone

Most useful Git commands: git help and git help <command> (e.g. git help commit)

Break (15:15-15:45)

15:45 Sage Q & A

16:45 Wrapping up (maybe staying a bit longer, if the room remains available)

20:00 Dinner

Some of us staying in the Wivenhoe House Hotel go to the dinner in the hotel's Brasserie. Please add your name to the list (line 298) if you're coming.

What do we want to do tomorrow?

Definitely this:

Optionally this (please put an exclamation mark after the item if you are interested)

  • Olexandr, Michael: walk through the rest of the GAP lesson!
  • Olexandr: Code is Science (wrap-up, see abstract above)!!!
  • Olexandr: Tips and pitfalls for using GAP!!
  • Olexandr: SCSCP (see abstract above)!
  • Help with installation (e.g. of GAP Jupyter notebooks)
  • Time for exercises, questions
  • More practice in using version control/GAP/SageMath (maybe in small groups)
  • More experience with publishing notebooks on Binder
  • Anything else? Please suggest

Thursday January 23rd

Morning [STEM 4.1]

10:00 (with a break at 11:15-11:45):

Lunch (13:00-14:00) [6.300]

Afternoon [STEM 4.1]

14:00 (with a break at 15:15-15:45)

  • Code Is Science: Research Software is a Primary Research Output (Olexandr)

########################################################################
Links for Olexandr's talk:

What does “release” mean?

Sharing reproducible experiments with Jupyter on Binder and Azure Notebooks:

Code Deserves Credit

-As a user:
- Follow recommendations by developers, if they are provided
- Cite accurately. It helps both developers and readers.

Where to publish software papers:

########################################################################

Wrapping up

We anticipate finishing the workshop between 4pm and 5pm.

Olexandr: I plan to leave at 4.30pm by taxi to the Colchester train station for a 5.30pm train. You can join me if you are interested in a lift.

Last changed by 
 
Olexandr Konovalov
0
1615
Published on HackMD