---
# System prepended metadata

title: 'RNA-Seq Workshop Sep 21, 2021 Handout'
tags: [RNA-Seq, Workshop]

---

---
title: 'RNA-Seq Workshop Sep 21, 2021 Handout'
disqus: hackmd
---

RNA-Seq Workshop Sep 21, 2021 Handout
===

# Table of Contents

[TOC]

# Session 1 (Alex)

## 1. HTC login

> Before you begin, please make sure your sremote is active


### Mac
**On Mac:** Open Terminal app and type the following

```
$ ssh pittid@htc.crc.pitt.edu
```

Then type your password
```
pittid@htc.crc.pitt.edu's password:
```
### Windows

**On Windows:** Download Putty app and follow the steps below

Download using the link here: https://www.putty.org/

Once downloaded, fill out the red box area will the following credentials. Then click 'Open'. 
You can also save these crednetials by naming the 'Saved Sessions' box and clicking 'Save'

![](https://i.imgur.com/QUZHvaB.png)


## 2. Unix Tutorial: Basic Commands I


| Command | Function | 
| -------- | -------- | 
| pwd     | Present working directory     |
| ls | Display files in list wise fashion |
| mkdir | Make a directory |
| cd | Change directory |
| . | Present Directory |
| \.\. | Previous Directory (One level up, parent directory) |
| tab | Hit <tab> for auto complete |
| clear | Clears the screen |
| up/down arrows | Shows previously used commands |
| history | Displays all your previously used commands |
| man | manual |

### 2.1. Unix Exercise 1: Tree Structure of File System Part 1

```
$ cd ~ 
$ pwd
```
![](https://i.imgur.com/Mx3SwXJ.png)

```
$ mkdir rna_wkshop
$ cd rna_wkshop
$ pwd
```
![](https://i.imgur.com/7adElag.png)

```
$ mkdir Data
$ mkdir QC
$ mkdir Jobs
$ mkdir Mapping 
$ mkdir Counts
$ ls
```
![](https://i.imgur.com/DQl7gSj.png)

```
$ cd Jobs 
$ pwd
```
![](https://i.imgur.com/DTtn52p.png)

```
$ cd .. 
$ pwd
```
![](https://i.imgur.com/5q8QHiE.png)

```
$ cd Data
$ cd
$ cd ~/rna_wkshop
$ pwd
```
![](https://i.imgur.com/Nb3vurX.png)

### 2.2. Unix Exercise 2: Tree Structure of File System Part 2

```
$ cd ~ 
$ pwd
```
![](https://i.imgur.com/xThRjNm.png)

```
$ cd /
$ cd bgfs
$ cd genomics
$ cd workshops
$ cd 2021f
$ cd Introduction_to_Linux_for_NGS
$ pwd
```
![](https://i.imgur.com/cficHD6.png)



```
$ cd
$ pwd
```
![](https://i.imgur.com/CDmt5dm.png)

```
$ cd /bgfs/genomics/workshops/2021f/Introduction_to_Linux_for_NGS
$ pwd
```
![](https://i.imgur.com/6dgmlLc.png)


```
$ cd
$ pwd
```
![](https://i.imgur.com/CDmt5dm.png)


# Session 2 (Alex)

## 3. Unix Tutorial: Basic Commands II

| Command | Function | 
| -------- | -------- | 
| cp     | Used for copying files/directories |
| gzip/gunzip | gzip command is used to compress files and gunzip is used to uncompress files. The compressed files have .gz extension. |
| head | Used to display the beginning of a text file. Default is 10 lines. |
| tail | Used to display the tail end of a text file. Default is 10 lines. |
| grep | Used to search for a string of characters in a file. The text search pattern is called a regular expression. |
| cut | Command line utility for cutting sections from each line of files and writing to standard output |
| less / more| Terminal pager program which allows you to view files BUT NOT edit them, press q to quit out of the program| 
| wc | The program reads either standard input or a list of files and generates one or more of the following statistics: newline count, word count, and byte count |


### 3.1. Unix Exercise 3: Practical File Operations Part 1

```
$ cd ~/rna_wkshop
$ cp -r /bgfs/genomics/workshops/2021f/Introduction_to_Linux_for_NGS/Unix_Practice . 
$ ls
```
![](https://i.imgur.com/sHDqiUZ.png)



```
$ cd Unix_Practice 
$ ls -lh
```
![](https://i.imgur.com/0VC7Kff.png)

```
$ head practice_counts.txt
```
![](https://i.imgur.com/khR8SDO.png)

```
$ tail practice_counts.txt
```
![](https://i.imgur.com/S9aD1zo.png)

```
$ head -30 practice_counts.txt
```
![](https://i.imgur.com/tD9l1Bo.png)

```
$ grep ENSG00000284733 practice_counts.txt
```
![](https://i.imgur.com/q9NERx2.png)

```
$ wc -l practice_counts.txt
```
![](https://i.imgur.com/iFnkB2w.png)

```
$ wc -c practice_counts.txt
```
![](https://i.imgur.com/Tyxx0vH.png)

## 4. Unix Tutorial: Basic Commands III

| Command | Function | 
| -------- | -------- | 
| \| (pipe)     | The standard output to the left of the pipe is sent as standard input of the command on the right.    |
| sort | Sorts the lines of input in ascending order |
| uniq | Utility which, when fed a text file or STDIN, outputs the text with adjacent identical lines collapsed to one, unique line of text |
| find | Used to find files/directories in unix. |
| > | Output to the left of > sign is put into a file, file name is specified after > sign |
| >> | Appending the output to the left of >> to the end of an existing file specified on the right |

### 4.1. Unix Exercise 4: Practical File Operations Part 2

```
$ grep ENSG00000284733 practice_counts.txt | wc -l
```
![](https://i.imgur.com/ffLyQ7V.png)

```
$ cut -f 7 practice_counts.txt | grep -w 0 | wc -l
```
![](https://i.imgur.com/9nkzAYk.png)

```
$ cut -f 6 practice_counts.txt | sort
```
![](https://i.imgur.com/5PtSTUL.png)

*** Picture above only captures a small portion of  output ***

```
$ cut -f 6 practice_counts.txt | sort -r
```
![](https://i.imgur.com/EMlugqo.png)

```
$ cut -f 6 practice_counts.txt | sort -n
```
![](https://i.imgur.com/A6OPnO8.png)

```
$ cut -f 6 practice_counts.txt | sort -n | uniq -c
```
![](https://i.imgur.com/BfZ2QV7.png)

```
$ cd copy1
$ less HelloWorld.txt >> test.txt 
$ less test.txt
```
![](https://i.imgur.com/rZN62mW.png)
![](https://i.imgur.com/6Z84PCR.png)

```
$ cd ..
$ grep ENSG00000284733 practice_counts.txt > gene.txt 
$ less gene.txt
```
![](https://i.imgur.com/b9Aipm7.png)
![](https://i.imgur.com/DvyLU7f.png)


# Session 3 (Vishal)
## 5. Unix Tutorial: Vim Editor
**Insert mode**
| Command | Use | 
| -------- | -------- | 
| i     | Insert before the cursor |
| I | Insert at the beginning of the line |
| a | Insert after the cursor |
| A | Insert at the end of the line |

**Visual mode**
| Command | Use | 
| -------- | -------- | 
| v | Start visual mode |
| V | Start visual mode line wise |
| d | Delete marked text |
| Esc | Exit visual mode |

**Exit commands**
| Command | Use | 
| -------- | -------- | 
| :w | Write (save) the file but don’t exit |
| :wq | Write and quit|
| :q | Quit. Fails if there are changes. |
| :q! | Quit and throw away changes. |

**Misc commands**
| Command | Use | 
| -------- | -------- | 
| yy | Yank. Copy a line. |
| P | Paste after the cursor |
| d | Delete a line |
| /pattern | Search for a pattern |


## 6. Copying Workshop Materials
In this section, we will copy the materials we need to our home directory

```
$ cd ~
$ cp -r /bgfs/genomics/workshops/2021f/Introduction_to_Linux_for_NGS/Text_Practice ./ 
$ cd Text_Practice
```

## 7. File Permissions and Hashes
Adding parameters to the ls command allows us to see the permissions.

```
$ ls -lh
```

## 8. Changing file permissions

If we have been given the ability to, we can change who can access a file.

```
$ chmod 664 old.job
```

After we have transferred a large file, it is good to check that all the data is intact. We can generate a type of file hash called an md5 sum to compare against the md5 sum that was transferred to us.

```
$ md5sum -c hash.md5
```

## 9. Vim Editor

### a. Opening and editing an existing file
Here we will open an existing file, add some text, save it, and quit
```
$ vim old.txt
    i
    [esc] 
    :wq
```
### b. Opening a file and quitting without saving
We can quit a file without saving any changes we may have made
```
$ vim old.txt
    i
    [esc] 
    :q!
```
### c. Creating a new file and saving it
Opening a file with a name that doesn’t exist creates a new file with that name.

```
$ vim new.job 
    i
    [esc] 
    :wq
```

## 10. SLURM Jobs

### a. Finding a module

Find the FastQC module for this job
```
$ module spider fastqc
```

### b. Edit job file

Add module name and version number, number of cores, and email to job script
```
$ vim example.job
    i
    #Add 1 to -c
    #Add your email to --mail-user=
    #Add FastQC module name and number to module load
    [esc] 
    :wq

```

### c. Run the job

Use SBATCH to schedule and execute the job
```
$ sbatch example.job
```


###### tags: `RNA-Seq` `Workshop`

