---
# System prepended metadata

title: TE divergence Plots
tags: [divergence, TE, plot]

---

# TE divergence Plots
###### tags: `TE`, `divergence`, `plot`
Go to your working dir
`cd /groups/fr2020/...`

Copy the template folder divergencePlot_template in 
/groups/arklab/bin/REPET

`cp -R /groups/arklab/bin/REPET/divergencePlot_template`

- [ ] Take a look at the folder and its contents

- [ ] Read README:


#==============================================================================

This directory contains the scripts and files necessary to output a divergence landscape plot. RepeatMasker's RepBase poorly categorizes animals that aren't mammals or flies (as it appears to me), so using a TEdenovo library is highly suggested:

	1. Copy this folder to a new location.

	2. Symlink the fasta file for the organism's genome to this directory.
`ln -s /groups/fr2020/metagenomes/assembly.fa ./`
`ln -s /TEdenovo_project_directory/{NAME}_Blaster_GrpRecPil_Struct_Map_TEclassif_Filtered_MCL/{NAME}_denovoLibTEs_filtered_MCL.fa ./`


	3. After running TEdenovo on the genome you want to generate a plot for, take the file located in:
	/TEdenovo_project_directory/{NAME}_Blaster_GrpRecPil_Struct_Map_TEclassif_Filtered_MCL/{NAME}_denovoLibTEs_filtered_MCL.fa
	and symlink it to this new location.

	4. Edit "run" to modify the variable names as needed.

	5. Execute the following command: "./run > output.log"

The landscape divergence plot should be labeled {NAME}.html.

#==============================================================================

Notes

The script "extractUnknowns.py" is used to pull out all of the consensuses labeled as "unknown" into a separate file. It also generates a "known" file that contains everything else.

In /groups/arklab/bin/RepeatMasker, there is a script called "createRepeatLandscape_UnknownOnTop.pl" that is a copy of "createRepeatLandscape.pl", with the only difference being that the unknown consensuses are plotted on top of the other results in the divergence plots.

#==============================================================================

Brandon M. Le (Brandon_Le@brown.edu), 2016


OK, step one is done, let's create a symlink (step2) to the fasta file:

ln -s /groups/fr2020/Bkoreanus/Bkoreanus.fa ./

Step 3: Look for the TEdenovo fasta file as described (once TEdenovo is finish) and create a symlink in /groups/fr2020/Bkoreanus/divergencePlot_template

Step: edit `run`:

## Modify these variables as needed
file="DmelChr4"
TElib="DmelChr4_denovoLibTEs_filtered_MCL.fa"

instead of DmelChr, it should be Bkoreanus

We are going to use one of the servers (no clusters) for this run. Then modifie the option "-pa" in
`RepeatMasker -pa 12 -a -low -no_is -e ncbi -lib ${TElib}.classified ${file}.fa`

with the numer of nodes available, but not all of then! It's going to run overnight anyway (so use SCREEN)

Save `run` file

Before run anythig we need to check every commad in run works on your terminal (bash_profile issue). You need to check:

`RepeatClassifier`
`faToTwoBit`
`RepeatMasker`

### Everything ready?

Check which machine are you going to use
`status`
etc...

This step (divergePlot) is better (faster) in the servers:

Generally Available Bioinformatics Servers 

    * arthur               CentOS 7.4    8*96  
    * domino               CentOS 7.4   24*96 
    * flicker              CentOS 7.4   24*250
    * jake                 CentOS 7.4    4*96  
    * luna                 CentOS 7.4    4*96  
    * minnie               CentOS 7.4   32*1000 
    * rocket               CentOS 7.4   12*128 
    * taiga                CentOS 7.4   12*128
    * tern                 CentOS 7.4   12*128


`ssh tern`
Just remener, adjust -pa in ./run

`RepeatMasker -pa 12 -a -low -no_is -e ncbi -lib ${TElib}.classified ${file}.fa`

to the number of processors each machine has (processors*Memory)

Use a screen session
`screen`

and run it
`./run`

If you use one of the clusters
	General Use Clusters:

    * cluster5             CentOS 7.4  2 @ 40*995   
    * cricket              CentOS 7.4  1 @ 40*230  
    * grendel              CentOS 7.4  8@8*96     
                                      3@12*128   
```
ssh cricket
clusterize -log output.log ./run
```



## EXTRA July 9

I would like to check two more divergence plots:
1. Use the combination of cosmid + illumina TEdenovo
`cat Dscosmid_denovoLibTEs_filtered_MCL.fa Dstevenill_denovoLibTEs_filtered_MCL.fa > newname.fa`

For building the plot within the cosmid seqs (Dscosmid.fa) in the divergencePlot_Template 
`customTElib="newname.fa"`

2. And use another custom TE library in 
`/groups/arklab/ostracods/divergencePlot_BL/customLibrary/ostracods_denovoLibTEs_filtered.fa.classified`

copy it in your new divergencePlot_Template folder and:

Run divergencePlot_Template script, but use this script instead: ./run

```
# Genererates a landscape plot through RepeatMasker (RM)

# Modify these variables as needed
file="ostracods"
customTElib="customLibrary/ostracods_denovoLibTEs_filtered.fa.classified"

# Run RM: query against database (Repbase in this scenario) with -a (alignment output .align)
/groups/arklab/bin/RepeatMasker/RepeatMasker -pa 12 -a -low -no_is -lib ${customTElib} ${file}.fa

# Run script calcDivergenceFromAlign.pl using output file (.align) from RM
perl /groups/arklab/bin/RepeatMasker/util/calcDivergenceFromAlign.pl -s ${file}.divsum ${file}.fa.align

# Output file ${file}.divsum which contains Kimura divergence is used to generate the landscape plot
perl /groups/arklab/bin/RepeatMasker/util/createRepeatLandscape_UnknownOnTop.pl -div ${file}.divsum -twoBit ${file}.2bit > ${file}.html
```