# Summer 2022

## Topics

### Command-line/Unix intro

Intro reading: [two](https://www.kevinbrowne.ca/command-line-bullshittery-and-other-realities-of-computing/) [articles](https://medium.com/@eytanadar/on-the-value-of-command-line-bullshittery-94dc19ec8c61) on "command line bullshittery" (the original post by Philip Guo that introduced this term is no longer available, alas ...) (Also see [^1])
- [Software Carpentry 'shell-novice' lesson](https://swcarpentry.github.io/shell-novice/); [setup instructions](https://swcarpentry.github.io/shell-novice/setup.html)
- [Terminus](https://web.mit.edu/mprat/Public/web/Terminus/Web/main.html), a command-line aventure game for practicing file-system navigation
- [an educative.io command-line course](https://www.educative.io/courses/harnessing-power-command-line-interface?hss_channel=tw-3305457991) (the syllabus is a useful catalog of shell skills)
- [Google form](https://forms.gle/2oiX4MPYW2XR8TdT8)
- [commands.dev](https://www.commands.dev/) (catalog of shell-command recipes)
- [Software Carpentry 'shell extras' lesson](https://carpentries-incubator.github.io/shell-extras/)
- [shell cheat sheet](https://cheatography.com/davechild/cheat-sheets/linux-command-line/)

* Jen
+ Ben

## Windows resources

* [Setup notes (started by Darren)](https://hackmd.io/gyHS7AtXRJO46Z92HTzMeA)
## Sessions

- **covered in session 1**: keyboard shortcuts (TAB/`^C`/`^V`), navigation, create/copy/remove, `grep` and basic `find`, basic pipes and filters. `touch`, `wc`, `cat`, `head`/`tail`, `cut` (briefly), `nano`/`vi` (briefly), `man` pages (briefly), `less`/`more` (briefly). $\approx$ SC shell introduction sections 1-4.
- **covered in session 2**:
- overview: shell as a language (vs R/Python/Perl ?)
- simple scripts (`. myscript` vs `./myscript`)
- permissions and paths
- running commands for many files:
- globbing
- `$()`
- `xargs`
- `` (capture result of command)
- (`find ... -exec`)
- `for` loops
- shell and environment variables; command-line arguments
* **covered in session 3**:
- remote access (e.g. [SC extras lesson 2](https://carpentries-incubator.github.io/shell-extras/02-ssh/index.html))
- `ssh`, [including RSA keys](https://medium.com/@jakewies/accessing-remote-machines-using-ssh-55a0fdf5e9d8) (here is [more info than you really wanted on SSH key exchange](https://goteleport.com/blog/ssh-handshake-explained/))
- `scp` vs. `rsync` vs. `wget` vs `curl` ([curl vs wget](https://linuxhint.com/what-is-the-difference-between-wget-vs-curl/))

**session 4:**

- `screen`, `tmux` etc.
- job control (see [SC extras lesson 5](https://carpentries-incubator.github.io/shell-extras/06-job-control/index.html)):
- `&`/`bg`/`fg`/`^Z`
- `jobs`
- `top`, `htop`, `ps`
- misc: `which`

---

- installing stuff
- tarballs (`.tar`, `.tar.gz`, `.tgz`); `tar`, `g(un)zip`, `configure`/`make`/`make install`
- `apt`/`yum` (Linux), [homebrew/MacPorts/fink](https://apple.stackexchange.com/questions/32724/what-are-pros-and-cons-for-macports-fink-and-homebrew)(MacOS), `cygwin`/`WSL`/? (Windows)

### make ([makestuff](https://github.com/dushoff/makestuff)/[shellpipes](https://dushoff.github.io/shellpipes/)/pipeR)

- [Software Carpentry 'make-novice' lesson](http://swcarpentry.github.io/make-novice/)
- [useMakestuff repo](https://github.com/mac-theobio/useMakestuff)

## other workflow tools
- [targets](https://books.ropensci.org/targets/) (an R-specific workflow tool)
- Snakemake (typically Python-specific, but [see here](http://simon.podhajsky.net/blog/2022/cleaning-up-5-years-of-genomics-work-in-36-hours/): also has advice about docker and `renv`)
- comments on tools for [R package version management](https://hrbrmstr.substack.com/p/2022-05-0301)

## containers (Docker etc.)
* Docker is a thin virtual machine (sort of): https://jsta.github.io/r-docker-tutorial/
* [Rocker](https://www.rocker-project.org/) is a set of R-oriented Docker images
* [twitter thread from Péter Sólymos about docker resources](https://twitter.com/psolymos/status/1440344622010548241)
* [Docker Desktop](https://www.docker.com/products/docker-desktop/): a virtual machine + Docker layer for MacOS & Windows
* [starting a shell in a running docker ](https://stackoverflow.com/questions/30172605/how-do-i-get-into-a-docker-containers-shell) (`docker ps; docker exec -it <container_id or name> bash`)
* set permissions for Linux: `sudo groupadd docker; sudo usermod -aG docker $USER; newgrp docker`; (may need to log out & back in/start a new shell?)
* [Dockerfile for r-base/latest on Rocker](https://github.com/rocker-org/rocker/blob/master/r-base/latest/Dockerfile)
* incantations:
* `docker run --rm -ti rocker/r-base` (`--rm` cleans up, `t` == start a terminal, `i` == interactive)
* if you get some message like `the input device is not a TTY. If you are using mintty, try prefixing the command with 'winpty'`, listen to this message and add this prefix
`winpty docker run --rm -ti rocker/r-base`
* `docker run --rm -p 8787:8787 -e PASSWORD=definitelysecure rocker/verse` followed by pointing browser to `localhost:8787` or
* Jonathan
+ Mikael, Steve

## debugging in R
* Michael
+ Mikael
+ Mike L. (not really, but it seemed a shame not to list him)

## purrr

* [Lab meeting recording, Passcode: LWXH9^!w](https://mcmaster.zoom.us/rec/share/QyH_1jjaWo67loEy8PzsBGy7HeKzKDjy0Nq1dIwheXHpLRY56dznnY4y2z367OI6.wYd0CIuuKYf1a-zV
)

`purrr` is an [R package](https://purrr.tidyverse.org/), part of the [tidyverse](https://www.tidyverse.org/). It is the tidyverse's machinery for [functional programming](https://en.wikipedia.org/wiki/Functional_programming). The name "purrr" comes from the idea of "pure functions" (functions that take an argument and return a value and have no *side effects*), which are important in functional programming.

The main functions in `purrr` are `map()` and variations; "map" is a term from functional programming, which means "apply a function to a list of objects". `map()` is the equivalent of `lapply()` ("list apply") in base R. It also has a lot of other commands for working with lists (see the [cheatsheet](https://github.com/rstudio/cheatsheets/blob/main/purrr.pdf))

- [Jenny Bryan purrr tutorial](https://jennybc.github.io/purrr-tutorial/index.html) (uses tidyverse + [repurrrsive package](https://cran.r-project.org/web/packages/repurrrsive/index.html), i.e. recursive list examples)

* Jonathan
+ Bicko

[^1]: and ["In the Beginning was the Command Line"](https://web.archive.org/web/20051220164526/http://www.cryptonomicon.com/command.zip), an optional, ancient, very long essay by Neal Stephenson

# Summer 2023

## debugging in R

- [flow package](https://moodymudskipper.github.io/flow/)
- `debug` package (Bravington, [archived](https://cran.r-project.org/src/contrib/Archive/debug/)
- (`remotes::install_version("debug", "1.3.1"`)
- Wickham [chapter 22](https://adv-r.hadley.nz/debugging.html)
- good old `cat()`
- `traceback()`: understanding the call stack
- `debug()`: debugging mode (also `debugonce`)
- `browser()`: setting breakpoint
- `options(error = recover)`: debug on error
- `options(warn = 2)`: promote warnings to errors
- `setBreakpoint("filename#linenum")` (front end to `trace()`)
- exiting: `Q`, `undebug()`, `options(error = NULL, warn = 1)
- reproducible examples ([reprex package](https://reprex.tidyverse.org/))
- debugging in weird environments (e.g. rmarkdown?)
- [debugging compiled (e.g. C++) code within R](https://blog.davisvaughan.com/posts/2019-04-05-debug-r-package-with-cpp)
- annoyances: editing functions defined within functions, functions that are reassigned (thus losing the debug flag, etc.)

## example from Advanced R

f <- function(a) g(a)
g <- function(b) h(b)
h <- function(c) i(c)
i <- function(d) {
  if (!is.numeric(d)) {
    stop("`d` must be numeric", call. = FALSE)
  }
  d + 10
}

## a more annoying example

f <- function(a) {
   g <- function(b) {
      h <- function(c) {
         i <- function(d) {
              if (!is.numeric(d)) {
                 stop("`d` must be numeric", call. = FALSE)
              }
              d + 10
         }
         i(c)
      } 
      h(b)
   }
   g(a)
}

## more generally

- reading the error message
- googling the error message
- ChatGPT???
- bisection ([git
](https://www.metaltoad.com/blog/beginners-guide-git-bisect-process-elimination))

## testing

Primary reading: [testing chapter](https://discdown.org/rprogramming/testing.html) from the [Digital Science Center](https://discdown.org/) (DiSC) at Universität Innsbruck.

- floating-point issues (`all.equal` etc.; tolerances)
- `waldo::compare`
- test cases
- unit, integration, regression tests
- refactoring (need for established tests)
- testing frameworks: base-R/built-in, `testthat`, `tinytest` (see [here](https://www.avrahamadler.com/2022/01/20/reduce-dependency-hell/)), [tinysnapshot](https://cran.r-project.org/web/packages/tinysnapshot/readme/README.html)
- automation: [GitHub Actions](https://github.com/features/actions) (and [alternatives](https://stackshare.io/github-actions/alternatives))