###### tags: `Ubuntu` # Usefull commands on UBUNTU ## Conda (miniconda3) ### Activate/deactivate enviroment "base" on console `conda config --set auto_activate_base false` `conda config --set auto_activate_base true` ## FOLDER @ home "cd .."" goto home directory /homes$ ls -list ## Finding directory ``` $ pwd /homes/sabraun ``` ## Porecourse directory ``` $ cd /vol/porecourse vol/porecourse$ ls -l insgesamt 25642889 drwxr-sr-x 2 cruecker porecourse 32 Okt 19 2017 bin drwxrwsr-x 7 jueneman porecourse 551 Sep 14 10:23 DATA -rw-r--r-- 1 nkleinbo porecourse 6441306010 Sep 14 15:41 Data.tar.gz drwx--S--- 5 jueneman porecourse 77 Sep 19 10:38 errorstat drwxr-sr-x 3 nkleinbo porecourse 276 Sep 18 12:58 files drwxr-sr-x 2 nkleinbo porecourse 60 Sep 14 15:01 image drwxrwxr-x 2 nkleinbo porecourse 81616 Nov 7 2017 Nanopore_small -rw-r--r-- 1 121 porecourse 248807246 Feb 20 2018 Nanopore_small.tar.gz drwxrwxr-x 54 nkleinbo porecourse 4007 Nov 23 2017 Results_old -rw-r--r-- 1 nkleinbo porecourse 12107811967 Sep 14 15:07 Results.tar.gz drwxrws--- 3 jueneman porecourse 21 Okt 27 2017 share drwxr-sr-x 3 nkleinbo porecourse 77 Sep 17 10:32 testing drwxr-sr-x 2 tstitzle porecourse 81616 Nov 23 2017 'ubuntu@$' /porecourse$ ls -lrth (-l list r -reverse -t nach Änderungszeit sortieren, neueste zuerst -h -human-readable ) insgesamt 25G drwxr-sr-x 2 cruecker porecourse 32 Okt 19 2017 bin drwxrws--- 3 jueneman porecourse 21 Okt 27 2017 share drwxrwxr-x 2 nkleinbo porecourse 80K Nov 7 2017 Nanopore_small drwxr-sr-x 2 tstitzle porecourse 80K Nov 23 2017 'ubuntu@$' drwxrwxr-x 54 nkleinbo porecourse 4,0K Nov 23 2017 Results_old -rw-r--r-- 1 121 porecourse 238M Feb 20 2018 Nanopore_small.tar.gz drwxrwsr-x 7 jueneman porecourse 551 Sep 14 10:23 DATA drwxr-sr-x 2 nkleinbo porecourse 60 Sep 14 15:01 image -rw-r--r-- 1 nkleinbo porecourse 12G Sep 14 15:07 Results.tar.gz -rw-r--r-- 1 nkleinbo porecourse 6,0G Sep 14 15:41 Data.tar.gz drwxr-sr-x 3 nkleinbo porecourse 77 Sep 17 10:32 testing drwxr-sr-x 3 nkleinbo porecourse 276 Sep 18 12:58 files drwx--S--- 5 jueneman porecourse 77 Sep 19 10:38 errorstat ``` ## The manual of a command ``` man ls ``` >manual of the command "ls" --> list ## Useful commands mkdir --> make directory touch --> create a file cp --> copy file mv --> move/rename a file rm --> delete a file rmdir --> delete a directory ## Create links ``` ln -s ``` >Softlink create a link to a directory or file ``` ln ``` >Hardlink link to /vol/porcourse/sabraun Beispiel: cd /vol/porecourse ``` /vol/porecourse$ mkdir sabraun /vol/porecourse$ cd .. : ln -s /vol/porecourse/sabraun workdir : ls -l insgesamt 14 drwxr-xr-x+ 2 sabraun users 2 Sep 19 10:39 Bilder drwxr-xr-x+ 2 sabraun users 2 Sep 19 10:39 Dokumente drwxr-xr-x+ 2 sabraun users 2 Sep 19 10:39 Downloads drwxr-xr-x+ 2 sabraun users 2 Sep 19 10:39 Musik drwxr-xr-x+ 2 sabraun users 2 Sep 19 10:39 Öffentlich drwxr-xr-x+ 2 sabraun users 2 Sep 19 10:39 Schreibtisch lrwxrwxrwx 1 sabraun users 43 Sep 19 10:39 thindrives -> /var/opt/thinlinc/sessions/sabraun/8/drives drwxr-xr-x+ 2 sabraun users 2 Sep 19 10:39 Videos drwxr-xr-x+ 2 sabraun users 2 Sep 19 10:39 Vorlagen lrwxrwxrwx 1 sabraun users 23 Sep 19 11:38 workdir -> /vol/porecourse/sabraun ``` ## Display file content head -n X >>display first X lines in file tail -n X >>display last X lines in file less >>display file page-wise Beispiel ``` ~/workdir$ head -n 20 seqs.fasta ~/workdir$ tail -n 20 seqs.fasta ~/workdir$ less seqs.fasta mit "/" in Datei suche ``` ## Wildcards "*" any character "?" Exactly one character [A-Z] letter [ACGT] one with this characters {1-20} numbers Beispiel ``` /vol/biotools/bin$ ls blast* /vol/biotools/bin$ ls -l blast? /vol/biotools/bin$ ls -l [ab]*[cd] (alle Dateien mit A und B und endet mit C und D) ``` ## Useful tool grep ``` ~/workdir$ grep ">" sequences_1.fasta >gi|406593966|ref|NC_018627.1|-40485/1 >gi|406593966|ref|NC_018627.1|-40485/2 >gi|406593966|ref|NC_018627.1|-40483/1 >gi|406593966|ref|NC_018627.1|-40483/2 >gi|406593966|ref|NC_018627.1|-40481/1 >gi|406593966|ref|NC_018627.1|-40481/2 >gi|406593966|ref|NC_018627.1|-40479/1 >gi|406593966|ref|NC_018627.1|-40479/2 >gi|406593966|ref|NC_018627.1|-40477/1 >gi|406593966|ref|NC_018627.1|-40477/2 >gi|406593966|ref|NC_018627.1|-40475/1 >gi|406593966|ref|NC_018627.1|-40475/2 >gi|406593966|ref|NC_018627.1|-40473/1 >gi|406593966|ref|NC_018627.1|-40473/2 >gi|406593966|ref|NC_018627.1|-40471/1 >gi|406593966|ref|NC_018627.1|-40471/2 >gi|406593966|ref|NC_018627.1|-40469/1 >gi|406593966|ref|NC_018627.1|-40469/2 >gi|406593966|ref|NC_018627.1|-40467/1 >gi|406593966|ref|NC_018627.1|-40467/2 >gi|406593966|ref|NC_018627.1|-40465/1 >gi|406593966|ref|NC_018627.1|-40465/2 >gi|406593966|ref|NC_018627.1|-40463/1 >gi|406593966|ref|NC_018627.1|-40463/2 >gi|406593966|ref|NC_018627.1|-40461/1 ~/workdir$ grep -c ">" sequences_1.fasta 25 ``` ## Usefool tool: wc ``` ~/workdir$ grep -c "Ch" Araport11_genes.gff -c nur zählen der Zeilen in der Datei ~/workdir$ grep -c "^Chr1" Araport11_genes.gff -c zählen alle zeilen mit Anfang Chr1 ``` Finde "Romeo und Juliet" file ``` /vol/porecourse/files$ grep -r "Romeo and Juliet" congratulations/you/found/me:"Romeo and Juliet" is intended for secondary school students who ``` Kombiniere grep mit umleiten in Datei > and >> ``` grep "^Chr1" Araport11_genes.gff | less grep "^Chr1" Araport11_genes.gff > gene_selection.gff (weiterleiten an neue Datei) grep "^Chr2" Araport11_genes.gff >> gene_selection.gff (weitelten an die selbe datei mit >>) grep "^>" sequences.fasta | head -n 20 | tail -n 10 > headers 11-20.txt grep -v ">" sequences.fasta | wc (alle zeilen ohne ">") ``` ## Usefool tool: cat ``` cat ~/workdir/sequence_1.fasta cat sequences_*.fasta > sequences.fasta #Summary all assembly info files of a flye run cat BC*/ass*.txt >> assembly_info_all.csv ``` ## Root sudo bash ## Vergabe von Rechten an Sascha nur über root chown sascha:sascha abricate/ ## Anlegen von Ordnern mkdir <Name> ## proxy ### Einrichten 1. open terminal 2. type in ``` $ sudo bash $ cd /etc/apt/ $ gedit apt.conf ## copy these two lines in apt.conf : Acquire::http::proxy "http://proxy.clondiag.jena:8080/"; Acquire::https::proxy "https://proxy.clondiag.jena:8080/"; ## ``` 3. save apt.conf 4. restart machine! ### Abfragen https://www.shellhacks.com/linux-proxy-server-settings-set-proxy-command-line/ ``` $ env | grep -i proxy ``` ## Print working directory pwd ## Alle Befhle in Konsole history ## Editor: gedit ## Suchen ausgewählter Zeilen in Fasta grep '>' <Dateiname>.fasta ## Diskusage ``` $ du $ du -sm "name" #(-m MByte, -s summarize) ``` ## Deinstallation ``` $ apt-get purge "packet" $ apt-get remove "packet" ``` List of one named Depian Package ``` $ dpkg -l| grep -i "name" ``` ## kill processes ``` $ ps -ef $ kill "number of process" ``` ## Translate lowercase to uppercase letters ``` tr a-z A-Z < EIEC_chromosome_polished_complete2.fasta > EIEC_chromosome_polished_complete3.fasta ``` ## Counting letters in fastX files ``` grep "letter" filename | wc grep "^letter" filename | wc #only letter at a start of lines ``` ## Zeilenumbrüche oder Zeichen aus Fasta entfernen ``` tr -d "\n" < dateiname > neuer dateinamen tr -d "-" < dateiname > neuer dateinamen ``` ## Leerzeilen eintfernen und single lines erzeugen in FASTA 1. entferne alle Bindestriche mit gedit oder notepadd++ 2. lösche alle leerzeilen mit ``` grep "\S" dateiname > dateinamen1 ``` 3. erzeuge single lines mit awk ``` awk '{if (substr($0,1,1)==">"){print "\n"$0} else printf("%s",$0);p++;}END{print "\n"}' dateinamen2 > dateinamen3 ``` ## Type ``` type --help type "program-name" ``` Shows the path of the executable program. ## screen ### WAS IST DAS? https://wiki.ubuntuusers.de/Screen/ ### install ```sudo apt-get install screen ``` ### Befehle ```screen -S NAME``` new SCREEN ```screen -r NAME``` returen to SCREEN ```screen -ls NAME```list SCREEN ```screen -S NAME``` neues SCREEN ## CPU info ```watch "grep 'cpu MHz' /proc/cpuinfo"``` ```watch -n 0.5 sensors``` ## automatic rename and copy files in different folders to another folder ```for subdir in *; do cp $subdir/assembly.fasta $subdir.fasta; done;``` ## more When there is more than one input file, the more command concatenates them and also includes each filename as a header. To concatenate to a file: ```more *.txt > out.txt``` To concatenate to the terminal: ```more *.txt | cat``` Example output: ``` :::::::::::::: file1.txt :::::::::::::: This is my first file. :::::::::::::: file2.txt :::::::::::::: And this is my second file. ``` ## Delete header in multifasta file via awk ``` Example: >cellulase ATGCTA >gyrase TGATGCT >16s TAGTATG $ awk 'BEGIN {print ">"ARGV[1]};!/^>/{print}' test.fasta > test_noheader.fasta >test_noheader.fasta ATGCTA TGATGCT TAGTATG ``` ## Set default python version for installation Create a symlink from /usr/bin/python2.7 to /usr/bin/python and set the group name as “python”. Later, group name will be used to switch links. Do not forget to set the priority in the end of the command ```sudo update-alternatives --install /usr/bin/python python /usr/bin/python2.7 1 sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.5 2 sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.6 3 ``` Switch between python version with: ``` sudo update-alternatives --config python ``` This works with any python version ## Batch rename files with name-list.txt Note: make a copy of your original files as backup in a TMP folder 1. create a txt-file with all names (same number of lines/names vs. files to rename) - use excel to combine BC + name + ID + .fasta (OVERVIEW_ALL_SEQ_RUNS.xlsx) - replase tabstop by copy/paste \t to underline unsing notepad++ - delete CR at notebook++ via copy/paste \r at the end of each line ``` BC01_Staph-Stamm_Dubai-01_Mediclinic-041_291122__97812_.fasta BC02_Staph-Stamm_Kuwait_122_Z4326_181217__309651_.fasta BC03_Staph-Stamm_Trinidad-Tobago_2020-009_371M_140820__95809_.fasta BC06_AERO_CH_Nord36_R409_IMP-1_14-07-15__240758_.fasta BC07_AERO_CH_Nord37_R412_IMP-2_14-07-15__240759_.fasta BC08_AERO_CH_Nord38_R414_IMP-13_14-07-15__240760_.fasta BC09_AERO_CH_Nord39_R417_IMP-15_14-07-15__240761_.fasta BC10_AERO_CH_Nord40_R418_IMP-19_14-07-15__240762_.fasta BC11_AERO_CH_Nord41_R419_IMP-29_14-07-15__240763_.fasta BC12_SALM_CH_Nord42_R421_IMP-4_14-07-15__240764_.fasta BC13_SMAR_CH_Nord43_R422_IMP-11_14-07-15__240765_.fasta BC14_CIND_CH_Nord44_R424_IND-1_14-07-15__240766_.fasta BC15_CIND_CH_Nord45_R425_IND-2_14-07-15__240767_.fasta BC16_CIND_CH_Nord47_R430_IND-4_14-07-15__240769_.fasta BC17_ACBA_CH_Nord48_R431_NDM-1_14-07-15__240770_.fasta BC18_COLI_CH_Nord50_R452_NDM-1_14-07-15__240772_.fasta BC20_COLI_CH_Nord52_R468_NDM-4_14-07-15__240774_.fasta BC21_COLI_CH_Nord53_R470_NDM-5_14-07-15__240775_.fasta BC22_COLI_CH_Nord54_R473_NDM-7_14-07-15__240776_.fasta BC24_FREU_CH_Nord56_R539_VIM-4_14-07-15__240778_.fasta ``` 2. goto terminal and the folder where the files are and copy the rename.txt in folder 3. use follwing command: ``` paste -d' ' <(ls *.fasta) rename.txt | xargs -n2 mv ``` 4. done ## Create multiple files at once on command line ```touch recovery{0001..0022}.csv``` ## Compress with tar and xz (multithreading) ```tar -c -I 'xz -9 -T0' -f archive.tar.xz [list of files and folders]```