# Software Caprentry Intro to Python Workshop
### UCSD Biomedical Library Building, Classroom 4 - 9:00am - 4:30pm
### November 27-28, 2018
---
### This HackMD: https://goo.gl/Spbfq2
# THE HACKMD is Locked from editing.
---
## Workshop Schedule:
https://ucsdlib.github.io/2018-11-27-UCSD/
## Post workshop survey
https://www.surveymonkey.com/r/swc_post_workshop_v1?workshop_id=2018-11-27-UCSD
---
# Please Sign in here:
#### (name, affiliation (faculty, staff, student, post-doc, etc.), department/Office/Lab)
Reid Otsuji, Data Curation Librarian, Library
Ryan Johnson, Metadata Librarian, Library
Stephanie Labou, Data Science Librarian, Library
Erik Klimczak, Staff, SIO
Bryan S. Kehr, Library ITS
Marion Alberty, graduate student, SIO
Margaret Lindeman, graduate student, SIO
Hamanda Badona Cavalheri, grad student, Biology
Rick McCosh, Post-doc, OB/GYN+Repro Science
Jesse Wilson, Post-doc, SIO
Maya Becker, graduate student, SIO
Ali Hamidi, Postdoc, SIO
Emelia Bainto, Staff, Pediatrics
Arden Tran, Research Facilitator, Research IT
Luke Colosi, Undergraduate student, UCSD
Robina Shaheen, Project scientist, UCSD
---
# Add your notes here:
### (Reminder: These are collaborative notes, please make sure you do not overwrite others text!)
# Day 2 - Introduction to Python
## CHEAT SHEETS - courtesy of MAS Data Science & Engineering program, Faye Chou [HDSI], Dr. Yoav Freund [CSE], and Arden Tran [RIT]
Cheat sheets for Python, Bash, MD, Pandas, etc. - a data scientist centric reference set. Download here: https://ucsdcloud-my.sharepoint.com/:b:/g/personal/artran_ucsd_edu/ERm-GDiD9pxAqtrlBu_7vbcBKIUqHW5JeKnLFrwol7h2Uw
Save as a PDF and keep a local copy as this link expires in 7 days. If you need a copy in the future - email me at artran@ucsd.edu (Arden Tran)
data (`data.zip`): http://swcarpentry.github.io/python-novice-inflammation/setup/
you can write python code directly in the terminal.
```
$ python
>>> # you can start writing code after the >>> prompt
```
Start up Jupyter notebook. In the Terminal by typing:
```
$ jupyter notebook
```
Working with Jupyter notebooks: http://ucsd.libguides.com/ld.php?content_id=44671979
### Remember: python is case sensitive.
keep that in mind when working with variables.
### Note: If you close the notebook, you will have to run (import) your libraries again.
Python can be used as a calculator!
```
weight_kg = 60.0
```
Print results:
```
weight_kg_text = 'weight in kilograms:'
print(weight_kg_text, weight_kg)
```
Print results with calculations:
```
print('weight in pounds:', 2.2 * weight_kg)
```
Info about the numpy library:
http://www.numpy.org/
Numpy:
```
import numpy
```
(You may also see `import numpy as np`)
```
numpy.loadtxt(fname='inflammation-01.csv', delimiter=',')
```
```
data = numpy.loadtxt(fname='inflammation-01.csv', delimiter=',')
```
Summary of best practices for scientfic computing
```
import this
```
check
```
type(data)
```
Check shape (rows, columns) of data
```
print(data.shape)
```
**IMPORTANT: Python starts counting at 0**
So an array of 60x40 has rows 0-59 and columns 0-39
Select a value in array using [row.number, colum.number]
```
data[1,2]
```
Can also select a range of rows and columns
```
data[0:3, 1:5]
```
another slicing example:
```
data[1:4,0:3]
```
you can slice backwards by using negative numbers:
```
data[-1,-1]
```
```
print(data[::10,3])
```
```
print(data[::10,0].shape)
```
Stats with numpy
```
numpy.mean(data)
```
Help to find information about functions use `?` after function name:
```
numpy.mean?
```
numpy has tab completion
print all of numpy functions:
```
dir(numpy)
```
---
## plotting in python
importing matplotlib.pyplot
```
import matplotlib.pyplot as plt #plt is a chosen
```
assign matplotlib.pyplot to a chosen name in this example it is plt. be cautious of using names, make sure you do not confuse assigned names as variables or other functions!
display plots in the notebook cell
```
%matplotlib inline
```
Example of creating a plot
```
matplotlib.pyplot.imshow(data)
matplotlib.pyplot.show()
```
You can use markdown in Jupyter! Cell -> Cell Type -> Markdown
Markdown cheatsheet: https://guides.github.com/pdfs/markdown-cheatsheet-online.pdf
```
image = matplotlib.plyplot.imshow(data)
matplotlib.plyplot.colorbar()
matplotlib.plyplot.show()
```
Plotting Average inflammation over time:
```
ave_inflammation = numpy.mean(data, axis=0)
ave_plot = matplotlib.pyplot.plot(ave_inflammation)
matplotlib.pyplot.show()
```
Add labels and titles using .xlabel, .ylabel, and .title
```
ave_plot = matplotlib.pyplot.plot(ave_inflammation, label = 'ave inflammation')
matplotli.pyplot.xlabel('Days', fontsize = 16)
matplotlib.pyplot.ylabel('Average inflammation', fontsize = 16)
matplotlib.pyplot.title('%s', $filename, fontsize = 16)
matplotlib.pyplot.legend()
```
Adding subplots
```
# load data
data = nmpy.loadtxt('inflammation-01.csv', delimiter=',')
# set up figure
fig = matploglib.pyplot.figure(figsize=(10,3))
# this figure is 1 row, 3 columns
ax1 = fig.add_subplot(1, 3, 1)
ax2 = fig.add_subplot(1, 3, 2)
ax3 = fig.add_subplot(1, 3, 3)
# make plots
ax1.plot(numpy.mean(data, axis=0))
ax1.set_ylabel('Average')
ax2.plot(numpy.max(data, axis=0))
ax2.set_ylabel('Max')
ax3.plot(numpy.min(data, axis=0))
ax3.set_ylabel('Min')
# force subplots to fit into figure area
# see: https://matplotlib.org/users/tight_layout_guide.html
fig.tight_layout()
```
**Paste your solution!**
## Make 2x2 subplot of data statistics
## Load libraries
import numpy
import matplotlib.pyplot
## Load data
data = numpy.loadtxt('inflammation-01.csv', delimiter=',')
## Make figure
fig = matplotlib.pyplot.figure(figsize=(10,10))
## Initialize axes
ax1 = fig.add_subplot(2, 2, 1)
ax2 = fig.add_subplot(2, 2, 2)
ax3 = fig.add_subplot(2, 2, 3)
ax4 = fig.add_subplot(2, 2, 4)
## Make first subplot showing mean inflammation of all the patients for each day
ax1.plot(numpy.mean(data, axis=0))
ax1.set_ylabel('Average', fontsize=14)
## Make second subplot showing standard deviation of all the patients for each day
ax2.plot(numpy.std(data, axis=0))
ax2.set_ylabel('Standard Deviation', fontsize=14)
## Make second subplot showing the maximum inflammation of all the patients for each day
ax3.plot(numpy.max(data, axis=0))
ax3.et_ylabel('Maximum', fontsize=14)
## Make third subplot showing the minimum inflamation of all the patients for each day
ax4.plot(numpy.min(data, axis=0))
ax4.set_ylabel('Minimum', fontsize=14)
## Add title for
matplotlib.pyplot.suptitle('Patient Inflammation Statistics')
## Solutions to Walter Munk challenge:
```
full_name = 'Walter Munk'
print(full_name[len(full_name)-4:])
```
```
full_name = "Walter Munk"
full_name[-4:]
```
```
last_four = full_name[(len(full_name))-4:]
print(last_four)
```
## Standard deviation plot
filename = 'inflammation-01.csv'
std_inflammation = numpy.std(data, axis=0)
days = numpy.arange(40)
print(days)
ave_plot = matplotlib.pyplot.plot(days, ave_inflammation, label='ave inflammation')
matplotlib.pyplot.fill_between(days, ave_inflammation-std_inflammation, ave_inflamation+std_inflammation,
color='blue', alpha=0.2)
matplotlib.pyplot.xlabel('Days', fontsize=16)
matplotlib.pyplot.ylabel('Average inflammation', fontsize=14)
matplotlib.pyplot.title('%s' %filename, fontsize=16)
matplotlib.pyplot.legend()
---
## Loops
Example loop
```
word = 'oceanography'
for char in word:
print(char)
```
Indents are important! Use 4 spaces or tab for an indent.
`range` built-in python function creates a sequence of numbers.
creating a basic `for loop`:
```
for i in range(len(word)):
print(i, word[i])
```
```
w1 = 'oce'
w2 = 'anography'
w3 = w1 + w2
print(w3)
```
---
## Paste your exercise solution here:
### Exercise
```
word = 'happy go lucky'
newword = ''
for i in word:
newword = i + newword
print(newword)
string = 'surf'
list = []
for i in string:
list.append(i)
print(list)
```
```
yum = 'pizza'
newyum = ''
for i in range(len(yum)):
newyum = yum[i]+newyum
print(newyum)
```
```
yum[::-1] #this could work too
```
---
# lists
```
my_array = numpy.array(['a',2,3])
print(my_array)
print(my_array.type)
```
lists can define data types
Visual for how to think of lists, and lists of lists: https://swcarpentry.github.io/python-novice-inflammation/fig/indexing_lists_python.png
Making a list
```
empty_list = []
numeric_list = [1, 2, 5, 6]
char_list = ['ocean', 'atmosphere', 'terrestrial']
```
create and append to a list:
```
empty_list.append('hey')
print(empty_list)
```
```
empty_list.append('ho')
print(empty_list)
```
## Paste your solution here:
word = 'surf'
str_list = []
for i in word:
str_list.append(i)
print(str_list)
```
string = 'surf'
list = []
for i in string:
list.append(i)
print(list)
```
## Exercise:
string_for_slicing = "Observation date: 02-Feb-2013"
list_for_slicing = [["fluorine", "F"], ["chlorine", "Cl"], ["bromine", "Br"], ["iodine", "I"], ["astatine", "At"]]
output:
"2013"
[["chlorine", "Cl"], ["bromine", "Br"], ["iodine", "I"], ["astatine", "At"]]
```
print(list_for_slicing[:4])
```
**Important**: can't do math operations with lists the same way you can do with arrays
```
odds = [1, 3, 5]
events = 2 * odds
print(events)
# [1, 3, 5, 1, 3, 5]
```
This duplicates the list rather than multiplying.
## Exercise:
beatles = "In an octopus's garden in the shade"
One last thing about lists:
Use + to concatenate two lists
```
list1 = ['pizza', 'tacos']
list2 = ['quinoa', 'kale']
combo = list1 + list2
# ['pizza', 'tacos', 'quinoa', 'kale']
```
### working with multiple files
Useful library called "glob"
```
import glob
print(glob.glob('inflamatino*.csv'))
```
The returned results aren't necessarily sorted. To force a sort:
```
print(sorted(glob.glob('inflamatino*.csv')))
```
Using a loop to read in files and return plots
```
import numpy
import matplotlib.plyplot
# get all file names, assign to object
filenames = glob.glob('inflammation*.csv')
# get subset of files
filenames3 = filenames[:3]
# use a loop to:
for f in filenames3:
# first, return name of file
print(f)
# load data
data = numpy.loadtxt9f, delimiter = ',')
# set up figure
fig = matplotlib.pyplot.figure(figsize=(10,3))
# specify plots
ax1 = fig.add_subplot(1, 3, 1)
ax2 = fig.add_subplot(1, 3, 2)
ax3 = fig.add_subplot(1, 3, 3)
# create plots, add labels
ax1.set_ylabel('Average')
ax1.plot(numpy.mean(data, axis=0))
ax2.set_ylabel('Maximum')
ax2.plot(numpy.max(data, axis=0))
ax3.set_ylabel('Minimum')
ax3.plot(numpy.min(data, axis=0))
fig.tight_layout()
matplotlib.pyplot.show()
```
## conditional statements
Example
```
num = 53
if num > 100:
print(num, 'is greater than 100')
```
Nothing happens, because the condition is not met (it is false)
Can add an "else" clause in the case that the condition is not true
```
num = -3
if num > 0:
print(num, 'is positive')
else:
print('less than zero')
```
To add a third (or more) scenario:
```
num = -3
if num > 0:
print(num, 'is positive')
elif: num == 0
print('number is zero')
else:
print('less than zero')
```
Test multiple conditions:
```
num = 10
# check if value is positive and even (by having no remainder, a remainder of zero)
if (num > 0) and (numpy.mod(num, 2)==0):
print(num, 'is positive and even')
```
```
num = 7
if (num > 0) or (numpy.mod(num, 2)==0):
print(num, 'at least one condition is satisfied')
```
Can also use & for and, and | for or
## functions
Example:
```
# define function
def fahr_to_celsius(temp):
return ((temp - 32) * (5/9))
# call function
fahr_to_celcius(32)
```
```
print('freezing point of water is', fahr_to_kelvin(32))
```
```
print('boiling point of water is', fahr_to_kelvin(212))
```
```
def analyze(filename):
data = numpy.loadtxt(fname=filename, delimiter=',')
fig = matplotlib.pyplot.figure(figsize=(10.0, 3.0))
axes1 = fig.add_subplot(1, 3, 1)
axes2 = fig.add_subplot(1, 3, 2)
axes3 = fig.add_subplot(1, 3, 3)
axes1.set_ylabel('average')
axes1.plot(numpy.mean(data, axis=0))
axes2.set_ylabel('max')
axes2.plot(numpy.max(data, axis=0))
axes3.set_ylabel('min')
axes3.plot(numpy.min(data, axis=0))
fig.tight_layout()
matplotlib.pyplot.show()
```
```
def detect_problems(filename):
data = numpy.loadtxt(fname=filename, delimiter=',')
if numpy.max(data, axis=0)[0] == 0 and numpy.max(data, axis=0)[20] == 20:
print('Suspicious looking maxima!')
elif numpy.sum(numpy.min(data, axis=0)) == 0:
print('Minima add up to zero!')
else:
print('Seems OK!')
```
Good to test function code before running on set of files.
```
detect_problems('inflammation-01.csv')
```
Run functions analyze and detect_problems in a loop.
```
filenames = glob.glob('inflammation*.csv')
for f in filenames:
print(f)
analyze(f)
detect_problems(f)
```
### Remember: when you create functions make sure to `document and comment` in the code.
### Tip: Keep track of the `line numbers` in the notebook. You may run into the problem of running code out of order.
Google Drive
Gist
Clipboard
Sign in with Wallet
