2022 UC Carpentries Fall Workshop (The Unix Shell)

Workshop Details
Dates: September 6th - 13th, 2022
Time: 9am - 12pm

Workshop Agenda:
https://ucsdlib.github.io/2022-09-06-carpentries-uc/

Day 4: The Unix Shell

Software Installation:
Git for Windows - Windows
Windows: https://gitforwindows.org

• download the Git-2.33.0.2-64-bit.exe
• Git win 64 bit Direct download link https://github.com/git-for-windows/git/releases/download/v2.33.0.windows.2/Git-2.33.0.2-64-bit.exe

Mac users will use the built-in Terminal application
required for Unix Shell and Git lesson

Recommended Text editors to use:
Nano

• Part of Git for Windows isntaller (called GitBash)
• MacOS Terminal
  Notepad++ (optional text editor)
• https://notepad-plus-plus.org/downloads/
• Download latest version Notepad++

Lesson Data (download)

• https://librarycarpentry.org/lc-shell/data/shell-lesson.zip
• Download shell-lesson.zip and move the file to your Desktop
• Unzip/extract the file. You should end up with a new folder called shell-lesson on your Desktop.

NOTES:

What is the shell?
Intro slides: https://docs.google.com/presentation/d/1JvAk7a-SgYUN4y8xXtQKB2Z2iwup0hEfJZmsZ-xlJ5Q/edit?usp=sharing

Navigating the filesystem
Webpage for help using the shell on windows machines: http://man.he.net/

Working with files and directories

Automating the tedious with loops
More about shell scripts: https://ryanstutorials.net/bash-scripting-tutorial/

Counting and mining with the shell
Regular Expressions resources:
LC regex cheatsheet: https://librarycarpentry.org/lc-data-intro/reference.html
regex101:https://regex101.com/
regexper: https://regexper.com/
Regex Testing: https://www.regextester.com/

Workshop Day 4

First name and Last Name/Organization/Dept./Email

Name (first & last)	Organization	Dept.	Email
(example) Jane Doe	UCSD	IT	[e-mail]
Tom Le	UCM		[e-mail]
John Thompson	UC Merced	Cell & Molecular Biology	[e-mail]
Sam Erickson	UC Merced	Physics	[e-mail]
Sarina Qin	UC Merced	Nat Sci	[e-mail]
Douglas Zhang	UCSD	Chemistry and Biochemistry	[e-mail]
Kenan Chan	UCSD	SIO	[e-mail]
Bruce Hamilton	UCSD	CMM	[e-mail]
Jonathan Le	UCR	Mathematics	[e-mail]
Amber Heidbrink	UCSD	Cell and Developmental Biology	[e-mail]
Dayana Elizalde	UCR		[e-mail]
Shang Su	U Toledo	Cell and Cancer Biology	[e-mail]
Christopher Gray	UCR	Computer Sciencee	[e-mail]
Oishee Misra	UCSD	Economics	[e-mail]
Ha Vu	UCSD	Economics	[e-mail]
Michelle Wu	UCLA	Neuroscience	[e-mail]
Jay Chi	UCSB	ETS	[e-mail]
Osika Tripathi	UCSD	PUblic Health	[e-mail]
Kazuma Nagatsuka	UCSD	Robotics(Mechanical Engineering)	[e-mail]
Melodi Frey	UCSD	CMM	[e-mail]
Matthew Falcone	UCB	Civil and Environmental Engineering	[e-mail]
Ivan Felix Rios	UCSD	Mathemathics & Economics	[e-mail]
Brett Taylor	UCSD	Biomedical Sciences	[e-mail]
Agnieszka Pluta	UCLA	Psychology	[e-mail]
Jun Tan	UCSD	Economics	[e-mail]
Roberto Silva	UCSD	Scripps	[e-mail]
Vishakha Malhotra	UCSF	Biostatistics and Epidemiology	[e-mail]
Lillie Pennington	UC Merced	Life and Environmental Sciences	[e-mail]
Alexander Frey	UCSD	Rady School of Management	[e-mail]
Cherie Thompson	UCSD Library
Dexin Zhou	UCSD	Mathematics	[e-mail]
Nicole Rosenberg	UCSD	Scripps	[e-mail]
Mugen Blue	UCM	EECS	[e-mail]
Haley Potts	UCSD	Math & Economics	[e-mail]
Aleks Leszczynska	UCSD	Pediatrics	[e-mail]
Junxiao Gao	UCSF	Biostatistics and Epidemiology	[e-mail]
Steven Krehel	UCSD	Economics	[e-mail]
Zhaoning (Johnny) Wang	UCSD	CMM	[e-mail]
Daryl Han	UC Irvine	Student Center and Event Services	[e-mail]
Josiah Piceno	UCM	MBSE	[e-mail]
Apisit Kaewsanit	UCSF	Epidemiology and Biostatistics	[e-mail]
Jacob Ross	UCSD	anesthesiology	[e-mail]
Jay Colond	UCM	sociology	[e-mail]
Anshika Kandhway	UCM	Environmetal System	[e-mail]
Dilawer Ali	UCM	Mechanical Engineering	dali4@
Bineh Ndefru	UCLA	Materials Science	[e-mail]
Mario Cuaya	UCR	Computer Science	[e-mail]
Andrew Chan	UCSD	IGPP	[e-mail]
Andrew Gorin	UC Berkeley	Earth and Planetery Science	[e-mail]
Donald Zarate	UCR	Political Science and Psychology	[e-mail]
Jessica Wu-Woods	UCR	Microbiology	[e-mail]

Day 4 Exercises

1. Copying a file
   Instead of moving a file, you might want to copy a file (make a duplicate), for instance to make a backup before modifying a file. Just like the mv command, the cp command takes two arguments: the old name and the new name. How would you make a copy of the file gulliver.txt called gulliver-backup.txt? Try it!

2. Renaming a directory
   Renaming a directory works in the same way as renaming a file. Try using the mv command to rename the firstdir directory to backup.

3. Moving a file into a directory
   If the last argument you give to the mv command is a directory, not a file, the file given in the first argument will be moved to that directory. Try using the mv command to move the file gulliver-backup.txt into the backup folder.

4. For loop exercise
   Complete the blanks in the for loop below to print the name, first line, and last line of each text file in the current directory.

___ file in *.txt
___
    echo "_file"
    head -n 1 _____
    ____ __ _ _____
___

5. Count, sort and print (faded example)
   To count the total lines in every tsv file, sort the results and then print the first line of the file we use the following:

   wc -l *.tsv | sort -n | head -n 1

   Now let’s change the scenario. We want to know the 10 files that contain the most words. Fill in the blanks below to count the words for each file, put them into order, and then make an output of the 10 files with the most words (Hint: The sort command sorts in ascending order by default).

   __ -w *.tsv | sort __ | ______

6. Counting number of files
   Let’s make a different pipeline. You want to find out how many files and directories there are in the current directory. Try to see if you can pipe the output from ls into wc to find the answer.

7. Counting the number of words
   Check the manual for the wc command (either using man wc or wc –help) to see if you can find out what flag to use to print out the number of words (but not the number of lines and bytes). Try it with the .tsv files.
   If you have time, you can also try to sort the results by piping it to sort. And/or explore the other flags of wc.

8. Case sensitive search in select files
   Search for all case sensitive instances of a word you choose in the ‘America’ and ‘Africa’ .tsv files in this directory. Print your results to the shell.

9. Count words (case sensitive)
   Count all case sensitive instances of a word you choose in the ‘America’ and ‘Africa’ .tsv files in this directory. Print your results to the shell.

10. Case insensitive search in select files (whole word)
    Search for all case insensitive instances of that whole word in the ‘America’ and ‘Africa’ .tsv files in this directory. Print your results to a file results/hero-i.tsv.

Day 4 Reflection

Please enter how you will use the Unix Shell in your work or research here:

1. As a sysadmin, I use the shell everyday with SSH. Move and migrate data all the time with tools like rsync. Getting to the point where I prefer to manage files/folders with shell rather than macOS GUI. Many of the commands from the workshop were familiar, but defintely found some very helpful additional commands to help manage my systems more efficiently (i.e. CTRL-r). Thanks!

Day 4 Survey

Link to survey for today: https://forms.gle/5qgx8X6H3GRMacwD6

Day 4 Questions

Please enter any questions not answered during live session here:
1.

End Day 4