# Joel
Joel.md
# Ubuntu WSL2 setup
### Basic component installation by `apt-get install`

`sudo apt-get --yes update`
`sudo apt-get --yes upgrade`
`sudo apt-get --yes dist-upgrade`
`sudo apt-get --yes autoremove`

`sudo apt-get --yes install build-essential checkinstall libgtk2.0-dev freeglut3-dev libfreetype6-dev libqhull-dev qhull-bin libqhull7 dvipng ghostscript texlive-latex-base poppler-utils python-gobject-2-dev python-gobject python-gi-dev libffi-dev python-numpy default-jre libmldbm-perl libcdb-file-perl libdb5.3++-dev libdb5.3++ hmmer ncbi-tools-bin blast2 cd-hit libcurl4-openssl-dev libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev libgtk2.0-dev python2.7 libssh2-1-dev last-align texlive texlive-latex-extra texlive-latex-recommended gifsicle ffmpeg imagemagick uuid-dev elinks links2 python-tk mysql-server libmysqlclient-dev liblzma-dev zip binutils-multiarch bison++ byacc cmake make freebsd-buildutils samtools hdf5-tools libhdf5-serial-dev mercurial r-cran-rcpparmadillo libblas-dev liblapack-dev libgd-dev expat pandoc ttf-mscorefonts-installer python-pysam python-htseq liblzo2-dev zlib1g-dev openssh-server`

## Bashrc

bashrc is a shell script located in the user home that is used to customise the terminal session

For example to load a module you can add the line:
`module load <module>`

To modify an environment variable, like PATH, add the line:
`export PATH=$PATH:<path/to/dir>`

to activate a Python environment is as simple as adding the line:
`source <path/to/env>/bin/activate`

### Prompt
`tty -s && export PS1="\[\033[38;5;164m\]\u\[$(tput sgr0)\]\[\033[38;5;15m\] \[$(tput sgr0)\]\[\033[38;5;231m\]@\[$(tput sgr0)\]\[\033[38;5;15m\] \[$(tput sgr0)\]\[\033[38;5;2m\]\h\[$(tput sgr0)\]\[\033[38;5;15m\] \[$(tput sgr0)\]\[\033[38;5;172m\]\t\[$(tput sgr0)\]\[\033[38;5;15m\] \[$(tput sgr0)\]\[\033[38;5;2m\]\w\[$(tput sgr0)\]\[\033[38;5;15m\]\n \[$(tput sgr0)\]"`

## Ln for scratch
https://www.unixtutorial.org/commands/ln#:~:text=ln%20command%20is%20a%20Unix,already%20existing%20files%20or%20directories.

## SCP
the `scp` (secureshell copy) is a command-line utility that allows you to share files between different hosts or locations. It uses ssh for data transfer and provides the same security level - both files and passwords are encrypted.
You can copy files or directory:
* from you local system to a remote system
* from a remote system to your local system
* between two remote systems from your local system

### SCP command syntax
`scp [OPTION] [user@]SRC_HOST:]file1 [user@]DEST_HOST:]file2`
* `OPTION` - scp options [-c cipher|-F ssh_config|-P remote host ssh port|-l limit|...]
* [user@]SRC_HOST:]file1 - source file
* [user@]DEST_HOST:]file2 - destination file
To be able to copy files, you must have at least read permissions on the source file and write permission on the target system

NB: when copying files that share the same name and location on both systems, scp will overwrite files without warning.

Examples:
Copying files
* copy the file "example.txt" from a remote host to the local host
`scp [e-mail]:example.txt /some/local/directory`
* copy the file "example.txt" from the local host to a remote host
`scp example.txt [e-mail]:/some/remote/directory`
* Copy the files "example_1.txt" and "example_2.txt" from the local host to your home directory on the remote host
`scp example_1.txt example_2.txt [e-mail]:~`
* Copy the file "example.txt" from remote host "remotehost_1.edu" to remote host "remotehost_2.edu"
`scp username@remotehost_1.edu:remote/directory/example.txt username@remotehost_2.edu:/remote/directory`

Copying directories
* Copy the directory "example_a" from the local host to a remote host's directory "example_b"
`scp -r example_a [e-mail]:/some/remote/directory/example_b `
* Copy an entire directory from one host to another
`scp -r /local/directory [e-mail]:/remote/directory`

syncing files via rsync
Rsync provides many options for passing arguments to augment it's functionality. In the examples below we will use:

* -a flag: Stands for "archive" and syncs recursively and preserves symbolic links, special and device files, modification times, group, owner, and permissions.
* -z flag: Compresses files during transfer.
* -P flag: Combines the flags --progress and --partial. The first of these gives you a progress bar for the transfers and the second allows you to resume interrupted transfers:

The following command is a "push" operation because it pushes (syncs) a local directory to a remote system.
`rsync -azP ~/some/local/source/directory username@remotehost:/some/remote/destination/directory`

Conversely, a "pull" operation is used to sync a remote directory to the local system.
`rsync -azP username@remote_host:/some/remote/destination/directory ~/some/local/directory`

## CHMOD
File and Directory Permissions
`-rw-r--r-- 1 exampleuser examplegroup 4096 Apr 4 04:20 example.txt`
The file's mode represents the file's permissions and some extra information. There are five parts to the mode.
part 1: - (file or directory)
part 2: rw- (user permissions)
part 3: r-- (group permissions)
part 4: r-- (other permissions)
part 5: (special attributes)

The first character of the mode (part 1) is the file type. It can be a regular file or a directory. Each file and directory has three user based permission groups which break down into three sets: `user` part 2, `group` part 3, and `other` part 4.

part 1: A dash (-) in this position, as in the example, denotes a regular file, meaning that there is nothing special about the file. This is by far the most common kind of file. Directories are also common and are indicated by a d in the file type slot.
part 2: `user (u)` - The Owner permissions apply only the owner of the file or directory, they will not impact the actions of other users.
part 3: `group (g)` - The Group permissions apply only to the group that has been assigned to the file or directory, they will not effect the actions of other users.
part 4: `other (o)` - The All Users permissions apply to all other users on the system, this is the permission group that you want to watch the most.
part 5: `Directory Set Group ID` - There are two special bits in the permissions field of directories.
`s` - Set group ID
`t` - Save text attribute (sticky bit) - The user may delete or modify only those files in the directory that they own or have write permission for.

Each permission set can contain four basic representations:
`r` : the file is readable. Refers to a user's capability to read the contents of the file.
`w` : the file is writable. Refers to a user's capability to write or modify a file or directory.
`x` : the file is executable. Refers to a user's capability to execute a file or view the contents of a directory.
`s` : Some executable files have an s in the user permissions listing instead of an x. This indicates that the executable is setuid, meaning that when you execute the program, it runs as though the file owner is the user instead of you. Many programs use this setuid bit to run as root in order to get the privileges they need to change system files. - : null value (nothing).

Modifying Permissions
`chmod` : Changes the file mode bits of each given file according to mode, which can be either a symbolic representation of changes to make, or an octal number representing the bit pattern for the new mode bits.
`chown` : Changes the user and/or group ownership of each given file.
`chgrp` : Change the group of a file or directory.

chmod - change mode
To change permissions, use the chmod command. The chmod (change mode) command protects files and directories from unauthorized users on the same system, by setting access permissions.
Syntax
`chmod options permissions file name`
`chmod [OPTION]... MODE[,MODE]... FILE...`
`chmod [OPTION]... OCTAL-MODE FILE...`
`chmod [OPTION]... --reference=RFILE FILE...`

Some examples:
Private file for you
`chmod 600 myfile`
`chmod u=rw myfile`
Everyone can read; you can write
`chmod 644 myfile`
`chmod u=rw,g=rw,o=rw myfile`
Private directory for you
`chmod 700 mydir`
Everyone can read; you can write
`chmod 755 mydir`
`chmod u=rwx,g=r,o=r mydir`
* 4 stands for "read",
* 2 stands for "write",
* 1 stands for "execute", and
* 0 stands for "no permission"
So 7 is the combination of permissions 4+2+1 (read, write, and execute), 5 is 4+0+1 (read, no write, and execute), and 4 is 4+0+0 (read, no write, and no execute).

set the ownership of myfile to mrrobot
`chown mrrobot myfile`
set the ownership of the directory and all files in mydirectory to mrrobot
`chown -R mrrobot mydirectory`
set the group ownership of myfile to mrrobot
`chgrp mrrobot myfile`
set the group ownership of the directory and all files in mydirectory to mrrobot
`chgrp -R mrrobot mydirectory`

## VIM
VI

## Sublimetext
move line - `ctrl + shift + arrow`
duplicate - `ctrl + shift + d`
delete line - `ctrl + shift + k`
indent line - `ctrl + [ or ]`
select text -> edit tab -> Line -> Reindent
paste and indent - `ctrl + shift + v`

## HPC

## Sorghum project.
### Weblogo3

1. get sequences from Pronghorn
/data/gpfs/assoc/pgl/data/SDLIM/RING_working/sbi_domain_class.fasta

2. Seperate sequences based on class
RING-C2
RING-G
RING-H2
RING-HC
RING-ST
RING-v

3. Align sequences by Muscle

muscle3.8.31_i86linux32 -in domain_only_seperate3.fasta -out domain_only_seperate4.fasta -maxiters 1 -diags

4. Draw the WebLogo
https://weblogo.threeplusone.com/manual.html#CLI

### Circos

RING-C2
RING-G
RING-H2
RING-HC
RING-ST
RING-v

High to low
#e41a1c
#377eb8
#4daf4a
#984ea3
#ff7f00
#ffff33

https://colorbrewer2.org/#type=qualitative&scheme=Set1&n=6

### JGI
https://data.jgi.doe.gov/refine-download/phytozome?organism=Sbicolor&expanded=454
Sorghum bicolor v3.1.1
Sbicolor_454_v3.0.1.fa.gz - genome file
Sbicolor_454_v3.1.1.gene.gff3.gz - gene coordinate file

1. Chromosome number - egrep ">" Sbicolor_454_v3.0.1.fa
- Scafflod, unanchored etc
2. Chromosome size - samtools faidx
3. Gene location in Sbicolor_454_v3.1.1.gene.gff3.gz

gff3
https://useast.ensembl.org/info/website/upload/gff3.html

ShinyCircos
https://venyao.xyz/shinycircos/
please go to help.

### Results - Weblogo
1. Sequences were obtained via pronghorn: `/data/gpfs/assoc/pgl/data/SDLIM/RING_working/sbi_domain_class.fasta`

2. Sequences were organized base on the 6 different classes:
a)RING-C2
b)RING-G
c)RING-H2
d)RING-HC
e)RING-ST
f)RING-v

3. Sequences were algiend via muscle:
a)`muscle -in Sobic_C2_class - out aligned_C2.Fasta`
b)`muscle -in Sobic_G_class - out aligned_G.Fasta`
c)`muscle -in Sobic_H2_class - out aligned_H2.Fasta`
d)`muscle -in Sobic_HC_class - out aligned_HC.Fasta`
e)`muscle -in Sobic_ST_class - out aligned_ST.Fasta`
f)`muscle -in Sobic_V_class - out aligned_V.Fasta`

4.Result Weblogo
All the Weblogo images has a dimension size of 1920 x 500 pixels
a)Weblogo C2
b)Weblogo G
c)Weblogo H2
d)Weblogo HC
e)Weblogo ST
f)Weblogo V

### Results - Circos
1. The highest Gene number corresponds with these colors (in hexadecimals)
a)RING-C2 : #984ea3
b)RING-G : #ff7f00
c)RING-H2 : #e41a1c
d)RING-HC : #377eb8
e)RING-ST : #ffff33
f)RING-v : #4daf4a

2. Sorbic bicolor genome and genes were download:
https://data.jgi.doe.gov/refine-download/phytozome?organism=Sbicolor&expanded=454
a)Under this url, File were located under:
-> Sorhum bicolor v3.1.1
-> Sbicolor_454_v3.0.1.fa.gz (The genome file containing Chromosome 1 - 10 along with super chromosome)
-> Sbicolor_454_v3.1.1.gene.gff3.gz (Contains the gene/mRNA/CDS/etc information about the Chromosomes)

3. Chromosome size were obtained using Sbicolor_454_v3.0.1.fa.gz as the input file and using the`samtools faidx` function to get the size.
-Note: The Super chromosome ID was removed

4. Chromosome number, gene, starting and ending, and the ID were organized from Sbicolor_454_v3.1.1.gene.gff3.gz file.

5. 2 .csv files were created in order to meet the format of ShinyCircos
a)Chromosome size: Sbicolor_454_v3.0.1.fa.gz
> .csv format = chr, start, end
> Ex) Chr01, 1, 80884392
b)Chromosome information: Sbicolor_454_v3.1.1.gene.gff3.gz
> .csv format = chr, start, end, label
> Ex) Chr01, 1951, 2616, Sobic.001G000100.v3.2

## Scratch location in Pronghorn
your scratch folder location We used to use the location as `~/scratch` by using ln command

`/data/gpfs/assoc/pgl/joel`

https://www.geeksforgeeks.org/ln-command-in-linux-with-examples


https://stackoverflow.com/questions/72798310/adding-extra-track-to-outside-of-circos-plot-circlize-chorddiagram

library(circlize)

links <- data.frame(from = c("A", "B", "C", "B", "C"),
                     to = c("D", "E", "F", "D", "E"),
                    value = c(1, 1, 1, 1, 1))

categories <- data.frame(from = c("D", "E", "F", "D", "E"),
                         to = c("X", "X", "Y", "Y", "Y"),
                         value = c(1, 1, 1, 1, 1))

chordDiagram(links)