Software Caprentry Intro to Python Workshop

# Software Caprentry Intro to Python Workshop ### UCSD Biomedical Library Building, Classroom 4 - 9:00am - 4:30pm ### November 27-28, 2018 --- ### This HackMD: https://goo.gl/3uiqg6 # Editing this HackMD has been locked by the Instructors. --- ## Workshop Schedule: https://ucsdlib.github.io/2018-11-27-UCSD/ --- # Please Sign in here: #### (name, affiliation (faculty, staff, student, post-doc, etc.), department/Office/Lab) Reid Otsuji, Data Curation Librarian, Library Stephanie Labou, Data Science Librarian, Library Rick McCosh, Post-doc, OB/GYN+Repro Sci. Arden Tran, Research Facilitator, Research IT Bryan S. Kehr, Library ITS Saranya Canchi, POst-doc, Neurosciences Hamanda Badona Cavalheri, Biology, grad student Maya Becker, graduate student, Scripps Institution of Oceanography Jesse Wilson, Post-doc, Scripps Institute of Oceanography Margaret Lindeman, graduate student, SIO Emelia Bainto, Data Manager, KD Research Lab Marion Alberty, graduate student, SIO Luke Colosi, undergraduate student, UCSD Erik Klimczak, Staff Research Associate, SIO, Ali Hamidi, Postdoc, SIO Robina Shaheen, associate research scientist --- # Add your notes here: #### (These are collaborative notes, please make sure you do not overwrite others notes) # Day 1 - Morning ## Unix Shell: Data files download: http://swcarpentry.github.io/shell-novice/data/data-shell.zip Save to your desktop and unzip Shell file system hierarchy Start Terminal in MacOS Start Gitbash in Windows Print working directory - shows where you are in the file system (file path) great command to use when you need to know where you are in the system ``` pwd ``` Change directory ``` cd ``` Move up in the filesystem hierarchy ``` cd .. ``` Make a new folder ``` mkdir ``` using `cd` will take you back you to the home directory ``` cd ``` list the contents of a directory ``` ls ``` to change your directory to 'desktop' ``` cd desktop ``` #### Tip: use the `Tab` key to Auto complete For help you can use: `man` will list all the flags and usage for the command ``` man [command] #example: $man ls on Windows (Git Bash) use: $ ls --help ``` #### Great site for looking up shell commands. http://explainshell.com/ Move back to the user/home folder ``` cd ~ ``` Display folders in a `ls` list ``` ls -F ``` Move folder command ``` mv ``` create the folder data-process ``` mkdir data-process ``` Naming files and folders: don’t use spaces! * Use _ or - instead * Can also use camelcase: FileName * Can use numbers, but never as the first character Remove directory ``` rm ``` Making new files ``` nano ``` Use a file extension to specify. For example, make a new text file in your current directory: ``` nano newfile.txt ``` Word count command - lists lines, words, and characters useful to see what's in the files ``` wc ``` wildcards using `*` or `?` for wc to fit different patterns. you can use wildcard when referencng a file. e.g. `ls *.pdb` Return all files with extension .pdb ``` wc *.pdb ``` Return all files ending in "ethane.pdb" ``` wc ?ethane.pdb ``` View contents one page at a time ``` less ``` Return the output sorted ``` sort ``` Sort, specifying by numeric ``` sort -n ``` Sort length.text and write out result to new file, sorted-lengths.txt ``` sort -n length.txt > sorted-lengths.txt ``` ``` head -n 1 sorted-lengths.txt 9 methane.pdb ``` ``` tail -n 1 sorted-lengths.txt 9 methane.pdb ``` Use the pipe `|` to send the output of one Use `cat` to print a file to the terminal (cat is short for concatenate) ``` cat salmon.txt ``` `cut` command can be used to break up the file ``` cut -d, -f 2 animals.txt ``` to save output as a new file ``` cut -d, -f 2 animals.txt >> animals-today.txt ``` `>>` append command is used to add text to an existing file #### creating a loop to rename the files: basic loop structure `for`,`in`, `do`, `done` ``` for filename in basilist.dat unicorn.dat >do >head -n 3 $filename >done ``` when you start a loop byt typing `for` the prompt will change to a `>` this is how you know you are starting a loop Example loop: ``` for datafile in *.pdb do ls $datafile done ``` Another loop: ``` for chem in *pdb do ls $chem > test.txt done ``` Note that this loop *overwrites* each output, so that there is only one value in test.txt We can fix this by appending to test.txt, rather than recreating it each time: ``` for chem in *pdb do ls $chem >> test.txt done ``` #### n.b.: how to save a variable To save a file as a temporary variable outside of a loop: ``` export VARIABLENAME=filename.xxx ``` example: Give ethane.pdb handle "x" & then display it in the terminal: ``` export x=ethane.pdb cat $x ``` ### To get a list of ALL the commands you’ve used in this session ``` history ``` You can save the output a file ``` history > command_history.txt ``` # Day 1 - Afternoon # Git ## Basic Git commands ``` git status git add [file.name] git commit -m "[short desscriptive message]" git log git diff ``` to setup global configuration for Git ``` git config ``` To set your username ``` git config --global user.name "[your name here]" ``` set your email address (make sure it is an email that can be public) ``` git config --global user.name "[your emailaddress that is ok to be public here]" ``` There are line endings for Windows and MacOS machines. FOr windows set config global to: ``` git config --global core.autocrlf true ``` For Mac set config global to: ``` git config --global core.autocrlf input ``` To track and preserve changes/versions of file ``` git add filename git commit -m "commit message" ``` `git add` "stages" your file and `git commit` adds that snapshot to your repository If you run `git status` after you make a change, after `git add` and after `git commit`, you can see how git is tracking your changes (i.e., see change but not tracked, staged for commit, committed and up to date) show the last commit information ``` git log ``` display short 7 digit identifer list of recent commits ``` git log --oneline ``` view the version changes between commits ``` git diff HEAD [7 alpha# ID] [file.name] ``` Roll back changes to previous version ``` git checkout [7 alpha# ID] [file.name] ``` go back to the latest version ``` git checkout head [file.name] #for exmple git checkout head refrerences.txt ``` ##### detached head state to revert back to master state use ``` git checkout master ``` If you're on Mac and want to see branch names in your command prompt like on Stephanie's computer, add the following to `~/.bash_profile` (from https://gist.github.com/joseluisq/1e96c54fa4e1e5647940): ``` # Git branch in prompt. parse_git_branch() { git branch 2> /dev/null | sed -e '/^[^*]/d' -e 's/* $.*$/ (\1)/' } export PS1="\u@\h \W\[\033[32m\]\$(parse_git_branch)\[\033[00m\] $ " ``` Create a new branch ``` git checkout -b [branchname] ``` This will create a new branch *starting from the current state of your repository* So, if you roll back your repository, then start a new branch, your new branch will start from that stage of your files. ``` # roll back to prior version git checkout [commitID] [file.name] # make new branch from this version git checkout -b [newbranchname] # get back to master branch git checkout master # get back to current state of repo, back to HEAD git checkout HEAD [file.name] ``` More about branches: https://git-scm.com/book/en/v2/Git-Branching-Basic-Branching-and-Merging Then open a new terminal window or use `source ~/.bash_profile` Remove from staging ``` git reset HEAD [file.name] ``` ## Git ignore Create git .ignore file add `.gitignore.txt` file to your folder ## Github collaboration: Setting up remotes in GitHub: https://swcarpentry.github.io/git-novice/07-github/index.html ``` git remote add origin [URL] git push -u origin master ``` "Pull" changes in GitHub to local ``` git pull origin master ``` "Push" local changes to GitHub ``` git push origin master ``` Best practice: always start your local session by pulling from remote repository (incorporate any changes made by collaborators before starting your own edits) Fix merge conflicts by editing file directly, then add and commit fixed file Use this link to accept collaboration invitation: https://github.com/notifications

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.