# Software Carpentry in Lancaster

This pad is synchronized as you type, so that everyone viewing this page sees the same text.
This allows us to collaborate seamlessly on documents.

For a different style of pads see
- https://etherpad.wikimedia.org

Users are expected to follow The Carpentries' code of conduct:
https://docs.carpentries.org/topic_folders/policies/code-of-conduct.html

All content is publicly available under the Creative Commons Attribution License:
https://creativecommons.org/licenses/by/4.0/

---

The workshop page is https://ccp-codima.github.io/2020-02-20-lancaster/
for the workshop on 20-21 February 2020 in Lancaster.

This pad is located at

- https://hackmd.io/wXD9sEu9ShO5wSZYQsDKuw

On Monday we were using a different pad, located at

- https://pad.carpentries.org/2020-02-20-lancaster

but all the content from that pad has been migrated here.

After the original pad has been reset, this contend has been migrated there for archiving.

Hashtag for tweets: #codima2020

**Instructors**
- Alexander Konovalov
- I am one of the developers of GAP [https://www.gap-system.org]. I work in the School of Computer Science at the University of St Andrews [https://alex-konovalov.github.io/]. I'm involved in the Carpentries since 2015. I am a co-investigator on the CCP-CoDiMa [https://www.codima.ac.uk/] which organises this workshop.
- Samuel Lelièvre
- Fredrik Strömberg
- I am mainly developing programs for number theory (especially automorphic forms) using Sage (Python and Cython). I am an assistant professor in Number Theory at University of Nottingham.

**Helpers**
- Awss Al-Ogaidi

## Software Carpentry Lesson Materials

SWC site: https://software-carpentry.org/
SWC lessons: https://software-carpentry.org/lessons/

### Thursday morning: Unix shell

Lesson: http://swcarpentry.github.io/shell-novice/
Reference: http://swcarpentry.github.io/shell-novice/reference

A directory is also known as a folder.

Use the up/down arrow keys to go through the command history, e.g. 'up' prepares the previous command.

### Shell commands:
- `ls` lists all the files/directories contained in the current working directory.
- `ls -F` indicates which files are folders and which are not, folders have a trailing slash/
- `ls -f` seems to give an alphabetically unsorted list of files: "do not sort, enable -aU, disable -ls --color"
- `ls -l` the -l flag means "long output". Displays extra information about the files.
- `ls -a` The `-a` flag represents "all"; displays everything including hidden files and folders, `./` and `../`
- `pwd` print the name of the current working directory
- `cd` "change directory". Argument supplied to `cd` must be contained within the current working directory.
- `cd Desktop` changes to the directory 'Desktop'.
- `cd ..` changes directory to the parent of the current working directory.
- `cd ~` changes directory to the home folder, also known as `~`
- `cd -` return to the previously visited directory "Back"
- `cd` The`cd` command with no argument changes to home folder. Equivalent to `cd ~`
- `mkdir name` "make directory". This command would create a folder called 'name'
- `touch filename` creates an empty file called 'filename'.
- `mv` "move"
- `mv filename dir/` will move the file called 'filename' to the directory specified by 'dir'
- `mv oldname newname` will rename the file 'oldname' and save it with the name 'newname'
- `rm` "remove"
- `rm file` deletes a file
- `rm -r dir` deletes a directory and all files contained. `-r` means "recursively"
- `rm -i` "interactive" deletion, prompts for every file.
- `cp` "copy"
- `cp file1 file2 file3 dir/` copies the specified files into the target directory
- `cp -r dir/ backup_dir/` copies the directory 'dir' into the directory 'backup_dir'

#### Wildcards, (patterns/regular expressions)
- `*` wildcard, matches any character string. Given a partial file name, the `*` will stand in for any possible completion.
- `rm p*` will remove all files that begin with 'p'.
- `ls *.txt` will list all files which end '.txt'
- `ls *b* ` will list all files containing anywhere a 'b'
- `?` matches a single character of a file name
- `[AB]` matches either A or B
#### Word count
- `wc` "word count"
- `wc filename` returns the word count, line count, and character count of the file
- `wc -l filename` returns the numbers of lines in the file
#### Filters
- `>` e.g. `wc -l *.pdb > lengths.txt` counts the lines in each file ending '.pdb' and saves this to a file lengths.txt <br> WARNING: `>` will overwrite a file
- `>>` appends to the end of a file
- `cat` outputs the contents of a file, line by line.
- `sort`
- `head -n N filename` returns the first N lines of filename
- `tail -n N filename` returns the last N lines of filename
- `echo`
- `chmod u+x file` adds permission for the user to x-ecute the file
- Should be combined with the header indicating the program to run the file i.e. `#! $(which bash)` e.g. `#! /user/bin/bash`
- `grep expr file` searches 'file' for lines containing expr. Flags:
- `-v` invert the search, returns non-matches
- `-w` returns whole words
`nano` launches the text editor, nano. `Ctrl-X` will close the editor.
`nano file.txt` creates a new text document, file.txt and opens it in nano.

Summary: http://swcarpentry.github.io/shell-novice/reference/

#### Tab completion
Typing just the beginning of a filename, then pressing the 'Tab' key.
If there is a unique file name which matches what was typed, completes the rest of the file name.
Otherwise pressing Tab twice offers choices for all matching filenames.

**Question**: How to copy all .txt files within the current directory and its subdirectories into one folder

**Answers**:

mkdir all-text_files # Avoids unnecessary loops?
cp $(find . -name "*.txt") all_text_files/

for filename in $(find . -name "*.txt") do 
    cp $filename all-in-one/
done

mkdir one_folder # only current folder
for each in $(ls *.txt)
    do cp $each ./one_folder
done

def exercise2(x: int, n:int) -> bool:
    r"""
    Return True if square of first input is a square modulo the second input, otherwise return False.
    
    INPUT:
    
    - ``x`` -- integer
    - ``n`` -- integer
    
    OUTPUT: A boolean description whether the square of first input is a square modulo the second input, otherwise return False.
    
    """
    if not isinstance(x,(int,Integer)):  # Multiple types should be given as a tuple
        raise ValueError(f"Received input of type {type(x)}. This function needs two integers!")
        
    if not isinstance(n,(int,Integer)):  # Multiple types should be given as a tuple
        raise ValueError(f"Received input of type {type(n)}. This function needs two integers!")
        if n < 1:
            raise ValueError(f"Received input {n}. n should be positive!")
    
    return (x**2 % n).is_square()

def is_square_residue(x, n: int) -> bool:
    r"""
    Return `True` if $x$ is congruent to a squared integer modulo $n$, otherwise return `False`
    
    INPUTS:
    - ``x`` -- an integer
    - ``n`` -- a positive integer
    
    OUTPUT: A boolean describing whether there are any solutions $a$ to $x \equiv a^{2} \mod n$ for inputs `x` and `n`
    """
    if not (isinstance(x, (Integer, int)) and isinstance(n, (Integer, int)) and n > 0):
        raise ValueError(f"Received inputs (x, n) = ({x}, {n}) of type ({type(x)}, {type(n)}). Inputs should be integers, with n positive!")
    return (x % n) in set(a**2 for a in range(n))
    
n = 8
for x in range(n):
    print(x, is_square_residue(x, n))

## DAY 2 Morning: Version control with Git

**Configure git**:
- `git config --global user.name "name goes here"`
- `git config --global user.email "[e-mail]"`
- Windows only: `git config --global autocrlf true`
- Mac and Linux : `git config --global autocrlf input`
`- git config --global core.editor "nano -w"`

---

Closing remarks for Git lesson:

How to write a good commit message:
- https://chris.beams.io/posts/git-commit/

Help pages on forking and collaborating with issues and pull requests:
- https://help.github.com/en/github/getting-started-with-github/fork-a-repo
- https://help.github.com/en/github/collaborating-with-issues-and-pull-requests

Version control is not only for coding:
- https://arstechnica.com/tech-policy/2018/11/how-i-changed-the-law-with-a-github-pull-request/
- https://github.com/adrian/irish-constitution

Useful resources:

* Software Sustainability Institute (SSI): https://software.ac.uk
* UK Society of Research Software Engineering: https://society-rse.org/
* The Carpentries: https://carpentries.org/
* Code Is Science Manifesto: https://codeisscience.github.io/manifesto/
* Software Citation Principles: https://doi.org/10.7717/peerj-cs.86
* Code4REF: https://code4ref.github.io/
* Hidden REF: https://hidden-ref.org/

What does “release” mean?
- At least should have a version
- GitHub releases: https://help.github.com/en/articles/creating-releases
- Ideally, should also have a DOI
- Zenodo (https://zenodo.org/)
- Figshare (https://figshare.com/)
- institutional repository
- ...
- Getting a DOI can be automated using GitHub-Zenodo integration: https://guides.github.com/activities/citable-code/
- Have an open source license (helpful: https://choosealicense.com/)

Sharing reproducible experiments with Jupyter on Binder and Azure Notebooks:
- GAP demo/template at https://github.com/rse-standrewscs/gap-binder-template
- Python template at https://github.com/rse-standrewscs/python-binder-template
- R template at https://github.com/rse-standrewscs/r-binder-template

Code Deserves Credit

- Software citation principles: A.M. Smith, D.S. Katz, K.E. Niemeyer, FORCE11 Software Citation Working Group. (2016) Software Citation Principles. PeerJ Computer Science 2:e86. https://doi.org/10.7717/peerj-cs.86
- As a developer:
- Advise your users how to cite it
- Automate sample citation generation
- Provide a CITATION.md file
- Provide a CFF file: https://citation-file-format.github.io/
- As a user:
- Follow recommendations by developers, if they are provided
- Cite accurately. It helps both developers and readers.

Where to publish software papers:
- JOSS (Journal of Open Source Software): https://joss.theoj.org/
- JORS (The Journal of Open Research Software): https://openresearchsoftware.metajnl.com/
- ReScience C: http://rescience.github.io/
- More journals accepting papers that primarily about the software
- https://www.software.ac.uk/which-journals-should-i-publish-my-software
- JSAG (Journal of Software for Algebra and Geometry): https://msp.org/jsag/

---

## Git lesson

We follow the Software Carpentry lesson "Version control with Git":

- http://swcarpentry.github.io/git-novice/

### Configure Git

$ git config --global user.name "Your name"
$ git config --global user.email "[e-mail]"
$ git config --global core.editor "nano -w"
$ git config --global core.autocrlf input

### Create directory and put it under version control

$ mkdir planets
$ cd planets
$ git init

## Status

Create a text file `mars.txt``

nano mars.txt

Write something in the file then save it and exit nano.

## Add, commit

## Checking the diff

$ git diff

$ git diff --staged

## About the log

$ git log --oneline

Imagine you are sending in a revised version
of a manuscript for publication.

If you used meaningful commit messages,
the Git log will show what you revised.

(Avoid commit messages like "Take care of annoying comment by reviewer"!)

## About commit messages

When using a text editor one can format the commit message as

- a one-line description, for `git log --oneline` to display
- then a blank line
- then a more detailed description of the changes.

## Break

(Enjoy snacks and drinks)

## Ignoring files

We want to tell git to never track some files.

For this we create a `.gitignore` file in our Git repo.

We keep this `.gitignore` file under version control.

For example in a git repo for a paper, we want
to track `.tex` files but not `.log`, `.aux`, `.pdf` files.

There might be cases where you still want to record some pdf files (say if they are used for figures in the paper, or just the pdf of the version you submitted to a journal)

Then we can use the `-f` (for "force") option to `git add`:

$ git add -f my_file.pdf

This will override our choice (recorded in the `.gitignore` file for that Git repo) to ignore changes to `.pdf` files.

## Collaboration using GitHub

Create a repository on GitHub.

There is a "GitHub education" scheme which offers for free some of the for-pay features to people in education (people doing or studies or teaching). Thy will ask you to use an academic address and might require some form of confirmation of your status.

GitHub is not the only place where Git repositories can be hosted.

- maybe your university
- github.com
- gitlab.com
- bitbucket.com
- ...

# CCP-CoDiMa workshop github page
Try to submit a pull request to https://github.com/ccp-codima/2020-02-20-lancaster

Maybe even better - to the repository at https://github.com/alex-konovalov/planets
- then it will be actually possible to merge your pull request

## Day 2 Afternoon - SageMath

Remember notebook functionality allowing to re-run all or selection of cells, with or without restart

Formatted strings in Python: https://docs.python.org/3.7/reference/lexical_analysis.html#f-strings

Exercise:

 f=(x+y*z)/sqrt(x^2+y^2+z^2)
g=diff(f,z)
g(0,1,1)=1/4*sqrt(2)

var('s,t,u')
f = s+t*u /sqrt(s**2 + t**2 +u**2)
f.diff(u)
f.substitute(s=0, t=1, u=1)
C

var('s,t,u')
f=(s+t*u)/(sqrt(s^2+t^2+u^2))
z=diff(f,u)
z
z.substitute(s=0,t=1,u=1)
1/4*sqrt(2)

so (e)?
e

var('s','t','u')
f = (s+t*u)/sqrt(s^2+t^2+u^2)
diff(f,u).substitute(s=0,t=1,u=1)

s,t,u=var('s,t,u')
f = (s+t*u)/sqrt(s**2 + t**2 +u**2)
z=diff(f,u)
z(s=0,t=1,u=1)
1/4*sqrt(2)
x - 1/2

def Bp(n):
    var('x','t')
    F = t*exp(x*t)/(exp(t)-1)
    return F.taylor(t,0,n).coefficient(t^n)*factorial(n)

**Exercise**: Recursive calculation of the Pochammer symbol

Will's function:

def RecursivePoch(s,k):
    r"""
    Recursively computes the Pochammer symbol (s)_k.
    INPUTS:
    - s: a complex number
    - k: a non-negative integer
    
    OUTPUTS:
    - The numerical value of the Pochammer symbol (s)_k.
    """
    if isinstance(k,(int,Integer)) == False or k < 0:
        raise ValueError("Second argument should be a non-negative integer.")
    if k == 0:
        return 1
    else:
        return (s+k-1)*Poch(s,k-1)

def poch_symbol(k : int) :
    var('s')  
    if k==1 : 
        return s
    elif k < 0:                                           return 'please enter a positive integer'
    return (s+k-1)*poch_symbol(k-1)

def pochammer(s,k):
    if k==0:
        return 1
    return (s+k-1)*pochammer(s,k-1)

def Pock(s,k):
    if k==0: 
        return 1
    return (s+k-1)*Pock(s+k-2)

def poch(s,k):
    if k==1:
        return s
    return poch(s,k-1)*(s+k-1)

# Classes in SageMath

- Inheritance: a mechanism which allows to create classes one from another without redefining all methods of the newly created class from scratch: https://docs.python.org/3/tutorial/classes.html#inheritance
- "dunder" is a jargon term for "double underscores"
- these are special indentifiers in Python which have format `__name__`
- examples are` __mult__`, `__add__` for standard operations
- docstring: one or more strings elclosed in triple quotes, i.e. """
- private methods:
- starting with an underscore _ (this is merely a convention, not enforced programmatically)
- should not be called by external code outside the class in which they are defined
- How try/except works in Python: https://docs.python.org/3/tutorial/errors.html
- `try` to try to run the code
- `except` to describe the action if this attempt fails
- `raise` to raise an error of a specific type (can then be analysed by` except`)
- Different kinds of testing common in software engineering:
- doctests: are examples in the documentation correct?
- unit tests: tests focussed mainly on single components of the software project
- integration tests foccussed mainly on checking how different components fit together

Please add here more terms which you would like to be explained