# USDA Long Term Agroecosystem Research Network
# Data Carpentry
## 22-24 Aug 2018, CSU Morgan Library

## Workshop website: https://btmoyers.github.io/2018-08-22-USDA/

### Welcome!
This is a document that any and all of us can use to take notes and share difficult to type information (e.g. links). To start editing, click on the icons above that look either like a pencil (editing only) or a book (both editing and viewing). You can just start typing anywhere in the dark panel.

Apologies for the issue with sign-ins! The permissions have been reset so that you can edit without signing in.

### Names
* Pat Nash
* Bryan Carlson
* Abby Jackson
* Gerardo Armendariz
* Mara Sedlins
* Stephen Teet
* Laura O'Gan
* Greg Sonnier
* Nicole Kaplan
* Bruce Vandenberg
* Kevin Cole
* Dan Arthur
* Jane Okalebo
* Kerrie Geil
* Lindsey Yasarer
* Steve Van Vactor
* Steve Wagner
* Jennifer Carter
* Earl Keel

### What is tidy data?
1. Each column is **one** variable.
2. Each row is **one** observation.
3. Each cell contains **one** data type.
4. Each table contains **one** type of observation.

To learn more about tidy data, check out [this great paper](https://www.jstatsoft.org/article/view/v059i10).

## 22 August AM
### OpenRefine

OpenRefine is an open source tool for data cleaning. It can be very useful for dealing with typos, whitespace, and other common issues with data cleaning. Even better: it does all of its work without modifying the original data file, which is ideal for data management. [Here](https://datacarpentry.org/OpenRefine-ecology-lesson/) is the lesson and [here](https://datacarpentry.org/OpenRefine-ecology-lesson/setup.html) are setup instructions.

>**Exercise 1**
>1. Using faceting, find out how many years are represented in the census.
>
>2. Is the column formatted as Number, Date, or Text? How does changing the format change the faceting display
>
>3. Which years have the most and least observations?

> **Exercise 2**
> Rename the species column. Did any issues arise?
>

> **Exercise 3**
> 1. You might like to look for trends in your data by month of collection across years.
> 2. How do you sort your data by month?
> 3. How would you do this differently if you were instead trying to see all of your entries in chronological order?

### Review (or intro) to R basics
[Here](https://datacarpentry.org/R-ecology-lesson/) is the lesson and [here](https://datacarpentry.org/R-ecology-lesson/#setup_instructions) are setup instructions.

An example of how to embed code:

foo <- c(1,3)

> What will happen in each of these examples? (hint: use class() to check the data type of your objects):

num_char <- c(1, 2, 3, "a")
num_logical <- c(1, 2, 3, TRUE)
char_logical <- c("a", "b", "c", TRUE)
tricky <- c(1, 2, 3, "4")

> What happens when you combine numbers and characters?
> character wins
> What happens when you combine numbers and logical values?
> numbers win
> What happens when you combine characters and logical values?
> character wins

> Why do you think it happens?
>

## 22 August PM
### Starting with data in R

download.file("https://ndownloader.figshare.com/files/2292169", "data/portal_data_joined.csv")

### Manipulating data with `dplyr`

[Here](https://github.com/rstudio/cheatsheets/raw/master/data-transformation.pdf) is a `dplyr` cheatsheet.

[Here](https://github.com/rstudio/cheatsheets/raw/master/data-import.pdf) is a `tidyr` cheatsheet.

> **Pipe exercise**
> Using pipes, subset the surveys data to include animals collected before 1995 and retain only the columns year, sex, and weight.
>

> **Mutation exercise**
> Create a new data frame from the surveys data that meets the following criteria: contains only the species_id column and a new column called hindfoot_half containing values that are half the hindfoot_length values. In this hindfoot_half column, there are no NAs and all values are less than 30.
> *Hint: think about how the commands should be ordered to produce this data frame*

> **split-apply-combine exercises**
> 1. How many animals were caught in each plot_type surveyed?
> 2. Use group_by() and summarize() to find the mean, min, and max hindfoot length for each species (using species_id). Also add the number of observations (hint: see ?n).
> 3. What was the heaviest animal measured in each year? Return the columns year, genus, species_id, and weight.

## For tomorrow
- install the R packages `tidyverse` and `markdown` and test load using `library()`
- install git (gitbash counts)
- create a GitHub account
- make a lunch order (below)

## Lunch orders
**Menu:** https://www.jimmyjohns.com/downloadable-files/jj_menu_no_prices.pdf
Please type your name, the sandwich you would like to order, and any notes for your order

| Name | Sandwich | Notes on order |
| -------- | -------- | -------- |
| Gerardo | Tukey breast, Provolone, Avo | 8" French Bread | |
| Jennifer | Tukey, Ham, Provolone | 8",tomato,mayomayomayomayo
| Bryan | Original roast beef | Order #377897850 |
| Dan | Club Lulu #16 | 8" French bread, No tomato, Add Provolone cheese
|Greg|Tuna salad|8" French bread
|Lindsey| Tuna salad|unwich lettuce wrap with sprouts and provolone cheese
|Kerrie | original size big italian (#9)|no mayo|
|Steve|#8 |9-grain wheat bread|
| Stephen | #5 | 8" french bread
|Kevin | #5 |8" french bread|
|Jane |#4|8" french bread (plus sprouts)|
|Tobin | #5 Italian| unwich (gluten free)|
|Earl| #9|original all the way|
|Steve W | #14 | load it up|
|Bruce | 8" JJBLT |

### Order your sandwich
#### 1) URL:
https://online.jimmyjohns.com
#### 2) Select Pickup
#### 3) Select this location:
1335 W Elizabeth St
Fort Collins, CO 80521
#### 4) Set you pickup time for 11:30am to 11:45am (8/23/2018)

## 23 August AM
### Visualization with `ggplot2`

[Here](https://www.rstudio.com/wp-content/uploads/2016/05/base-r.pdf) is the base R cheatsheet. [Here](https://www.rstudio.com/wp-content/uploads/2016/01/rstudio-IDE-cheatsheet.pdf) is an RStudio IDE cheatsheet that includes many keyboard shortcuts.

[Here](https://www.rstudio.com/wp-content/uploads/2015/03/ggplot2-cheatsheet.pdf) is the ggplot2 cheatsheet. [Here](https://ggplot2.tidyverse.org/reference/) is the ggplot2 website.

Here is where we made the subset data file for plotting: https://datacarpentry.org/R-ecology-lesson/03-dplyr.html#exporting_data

> **Plot Exercise**
> Use what you just learned to create a scatter (point) plot of weight over species_id with plot_type showing in different colors. Is this a good way to show this type of data?
>

> **Plot Exercise 2**
> 1. Boxplots are useful summaries, but hide the shape of the distribution. For example, if the distribution is bimodal, we would not see it in a boxplot. An alternative to the boxplot is the violin plot, where the shape (of the density of points) is drawn.
> Replace the box plot with a violin plot; see `geom_violin()`.
> 2. In many types of data, it is important to consider the scale of the observations. For example, it may be worth changing the scale of the axis to better distribute the observations in the space of the plot. Changing the scale of the axes is done similarly to adding/modifying other components (i.e., by incrementally adding commands). Try representing weight on the log10 scale; see `scale_y_log10()`.
> 3. So far, we’ve looked at the distribution of weight within species. Try making a new plot to explore the distribution of another variable within each species.
> - Create a boxplot for hindfoot_length. Overlay the boxplot layer on a jitter layer to show actual measurements.
> - Add color to the data points on your boxplot according to the plot from which the sample was taken (plot_id).
>
> Hint: Check the class for plot_id. Consider changing the class of plot_id from integer to factor. Why does this change how R makes the graph?

One great resource for colors: http://www.cookbook-r.com/Graphs/Colors_(ggplot2)/

Here are two more:
http://colorbrewer2.org/
https://coolors.co/

# R markdown

We're going to be building a document that looks like this:

https://github.com/maglet/rmacc-rmarkdown/blob/master/Markdown_Demo.pdf

The script files I worked with are in this git repository:
https://github.com/maglet/rmacc-rmarkdown

But if you want, you can take another graph you made during the ggplot session and document it.

[Here](https://www.rstudio.com/wp-content/uploads/2015/02/rmarkdown-cheatsheet.pdf) is the RStudio rmarkdown cheatsheet.

### Git in RStudio
Slides for the git lesson are [here](https://docs.google.com/presentation/d/13DQR2qIcCCg66DHJgBjultvcaEP1ZrUx2UprhdHrikw/edit?usp=sharing). Here is the [Happy Git and GitHub](http://happygitwithr.com/) tutorial.

### git setup

git config --global user.name "Vlad Dracula"
git config --global user.email "[e-mail]"