owned this note
owned this note
Published
Linked with GitHub
# Software Caprentry Intro to Python Workshop
### UCSD Biomedical Library Building, Classroom 4 - 9:00am - 4:30pm
### November 27-28, 2018
---
### This HackMD: https://goo.gl/3uiqg6
# Editing this HackMD has been locked by the Instructors.
---
## Workshop Schedule:
https://ucsdlib.github.io/2018-11-27-UCSD/
---
# Please Sign in here:
#### (name, affiliation (faculty, staff, student, post-doc, etc.), department/Office/Lab)
Reid Otsuji, Data Curation Librarian, Library
Stephanie Labou, Data Science Librarian, Library
Rick McCosh, Post-doc, OB/GYN+Repro Sci.
Arden Tran, Research Facilitator, Research IT
Bryan S. Kehr, Library ITS
Saranya Canchi, POst-doc, Neurosciences
Hamanda Badona Cavalheri, Biology, grad student
Maya Becker, graduate student, Scripps Institution of Oceanography
Jesse Wilson, Post-doc, Scripps Institute of Oceanography
Margaret Lindeman, graduate student, SIO
Emelia Bainto, Data Manager, KD Research Lab
Marion Alberty, graduate student, SIO
Luke Colosi, undergraduate student, UCSD
Erik Klimczak, Staff Research Associate, SIO,
Ali Hamidi, Postdoc, SIO
Robina Shaheen, associate research scientist
---
# Add your notes here:
#### (These are collaborative notes, please make sure you do not overwrite others notes)
# Day 1 - Morning
## Unix Shell:
Data files download:
http://swcarpentry.github.io/shell-novice/data/data-shell.zip
Save to your desktop and unzip
Shell file system hierarchy
Start Terminal in MacOS
Start Gitbash in Windows
Print working directory - shows where you are in the file system (file path)
great command to use when you need to know where you are in the system
```
pwd
```
Change directory
```
cd
```
Move up in the filesystem hierarchy
```
cd ..
```
Make a new folder
```
mkdir
```
using `cd` will take you back you to the home directory
```
cd
```
list the contents of a directory
```
ls
```
to change your directory to 'desktop'
```
cd desktop
```
#### Tip: use the `Tab` key to Auto complete
For help you can use:
`man` will list all the flags and usage for the command
```
man [command] #example: $man ls
on Windows (Git Bash) use: $ ls --help
```
#### Great site for looking up shell commands.
http://explainshell.com/
Move back to the user/home folder
```
cd ~
```
Display folders in a `ls` list
```
ls -F
```
Move folder command
```
mv
```
create the folder data-process
```
mkdir data-process
```
Naming files and folders: don’t use spaces!
* Use _ or - instead
* Can also use camelcase: FileName
* Can use numbers, but never as the first character
Remove directory
```
rm
```
Making new files
```
nano
```
Use a file extension to specify. For example, make a new text file in your current directory:
```
nano newfile.txt
```
Word count command - lists lines, words, and characters
useful to see what's in the files
```
wc
```
wildcards using `*` or `?` for wc to fit different patterns. you can use wildcard when referencng a file. e.g. `ls *.pdb`
Return all files with extension .pdb
```
wc *.pdb
```
Return all files ending in "ethane.pdb"
```
wc ?ethane.pdb
```
View contents one page at a time
```
less
```
Return the output sorted
```
sort
```
Sort, specifying by numeric
```
sort -n
```
Sort length.text and write out result to new file, sorted-lengths.txt
```
sort -n length.txt > sorted-lengths.txt
```
```
head -n 1 sorted-lengths.txt 9 methane.pdb
```
```
tail -n 1 sorted-lengths.txt 9 methane.pdb
```
Use the pipe `|` to send the output of one
Use `cat` to print a file to the terminal (cat is short for concatenate)
```
cat salmon.txt
```
`cut` command can be used to break up the file
```
cut -d, -f 2 animals.txt
```
to save output as a new file
```
cut -d, -f 2 animals.txt >> animals-today.txt
```
`>>` append command is used to add text to an existing file
#### creating a loop to rename the files:
basic loop structure `for`,`in`, `do`, `done`
```
for filename in basilist.dat unicorn.dat
>do
>head -n 3 $filename
>done
```
when you start a loop byt typing `for` the prompt will change to a `>` this is how you know you are starting a loop
Example loop:
```
for datafile in *.pdb
do
ls $datafile
done
```
Another loop:
```
for chem in *pdb
do
ls $chem > test.txt
done
```
Note that this loop *overwrites* each output, so that there is only one value in test.txt
We can fix this by appending to test.txt, rather than recreating it each time:
```
for chem in *pdb
do
ls $chem >> test.txt
done
```
#### n.b.: how to save a variable
To save a file as a temporary variable outside of a loop:
```
export VARIABLENAME=filename.xxx
```
example: Give ethane.pdb handle "x" & then display it in the terminal:
```
export x=ethane.pdb
cat $x
```
### To get a list of ALL the commands you’ve used in this session
```
history
```
You can save the output a file
```
history > command_history.txt
```
# Day 1 - Afternoon
# Git
## Basic Git commands
```
git status
git add [file.name]
git commit -m "[short desscriptive message]"
git log
git diff
```
to setup global configuration for Git
```
git config
```
To set your username
```
git config --global user.name "[your name here]"
```
set your email address (make sure it is an email that can be public)
```
git config --global user.name "[your emailaddress that is ok to be public here]"
```
There are line endings for Windows and MacOS machines.
FOr windows set config global to:
```
git config --global core.autocrlf true
```
For Mac set config global to:
```
git config --global core.autocrlf input
```
To track and preserve changes/versions of file
```
git add filename
git commit -m "commit message"
```
`git add` "stages" your file and `git commit` adds that snapshot to your repository
If you run `git status` after you make a change, after `git add` and after `git commit`, you can see how git is tracking your changes (i.e., see change but not tracked, staged for commit, committed and up to date)
show the last commit information
```
git log
```
display short 7 digit identifer list of recent commits
```
git log --oneline
```
view the version changes between commits
```
git diff HEAD [7 alpha# ID] [file.name]
```
Roll back changes to previous version
```
git checkout [7 alpha# ID] [file.name]
```
go back to the latest version
```
git checkout head [file.name]
#for exmple
git checkout head refrerences.txt
```
##### detached head state
to revert back to master state use
```
git checkout master
```
If you're on Mac and want to see branch names in your command prompt like on Stephanie's computer, add the following to `~/.bash_profile` (from https://gist.github.com/joseluisq/1e96c54fa4e1e5647940):
```
# Git branch in prompt.
parse_git_branch() {
git branch 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/ (\1)/'
}
export PS1="\u@\h \W\[\033[32m\]\$(parse_git_branch)\[\033[00m\] $ "
```
Create a new branch
```
git checkout -b [branchname]
```
This will create a new branch *starting from the current state of your repository*
So, if you roll back your repository, then start a new branch, your new branch will start from that stage of your files.
```
# roll back to prior version
git checkout [commitID] [file.name]
# make new branch from this version
git checkout -b [newbranchname]
# get back to master branch
git checkout master
# get back to current state of repo, back to HEAD
git checkout HEAD [file.name]
```
More about branches: https://git-scm.com/book/en/v2/Git-Branching-Basic-Branching-and-Merging
Then open a new terminal window or use `source ~/.bash_profile`
Remove from staging
```
git reset HEAD [file.name]
```
## Git ignore
Create git .ignore file
add `.gitignore.txt` file to your folder
## Github collaboration:
Setting up remotes in GitHub: https://swcarpentry.github.io/git-novice/07-github/index.html
```
git remote add origin [URL]
git push -u origin master
```
"Pull" changes in GitHub to local
```
git pull origin master
```
"Push" local changes to GitHub
```
git push origin master
```
Best practice: always start your local session by pulling from remote repository (incorporate any changes made by collaborators before starting your own edits)
Fix merge conflicts by editing file directly, then add and commit fixed file
Use this link to accept collaboration invitation:
https://github.com/notifications