# WEEK 1 (16-20/11/20)
## Machine learning (Mon, 16/11/20)
## What is data?
Data is a collection of facts (numbers, words, measurements, obsverations,...)
**Quantitative**=numerical information
-Discrete: whole numbers
-Continuous: decimals (stock prices)
**Qualitative**: descriptive observations
* Why we need data? Data is power, give us insights
* How can we collect data? Survey, observation,...
Internet of Things (IOT): The network of physical objects—“things”—that are embedded with sensors, software, and other technologies for the purpose of connecting and exchanging data with other devices and systems over the Internet.
**Census vs. Sample**
Census=whole population
Sample=part of the population
**Structured data**: highly organized, can be input into excel
**Unstructured data**: has no defined format and cannot be organized e.g. voice, audio
## What is data science?
Extracting messy data to make sense of it
**DA vs DE vs DS**
*DA-data analyst*:analyze data to help company make better decision, clean data, visualize, explore insights
Skill: some business skills
*DE-data engineer*: develop, construct, test and maintain complete architecture, develop data pipeline, deploy ML model
Skill: software engineering, big data, programming
*DS-data scientist*: analyze and interpret complex data (focus on models), +DA task, build model
Skill: Statistics, business, ML modeling
## What is machine learning?
ML is a field of study that gives computers the ability to learn without being explicitly programmed.
*AI*: technique enabling computers to mimic human behavior
*ML*: subset of AI, use statistical methods to enable machines to improve with experiences
*Deep learning*: subset of ML which make the computation of multi-layer neural networks feasible
**Supervised learning**: using labelled data
*Classification problem*-discrete output type
(e.g. Is this email Spam or not?)
*Regression problem*-predict continuous quantity output
(e.g. Predict the amount of rainfall)
## How ML works?
Collect data, feed into ML model, ML figure out the rule for you
**Steps to Predictive Modelling**
1. Get data
2. Clean, prepare, manipulate data
3. Train model
4. Test data
5. Improve
---
## Python Basics (Tues+Wed, (17-18)/11/20)
==**Formatting**==
`
print(1,2,3,4,5, sep='+', end='%')
`
Output
`1+2+3+4+5%`
`print(1,2,3,4,5, sep='<', end='|')`
`print(6,7,8,9,10, sep='+', end='*')`
---
`x='Hello'`
`y=' how are you'`
`"{},{}".format(x,y)`
`Hello, how are you`
---
`" ".join([x,y])`
`Hello how are you`
---
```python=
a = 'Tom Huynh'
print("My name is {}".format(a))
```
**Round**
```
round(15678.5634,-3)
```
Output
`16000`
"-3": round to the hundreths
---
`print("{:.2f}".format(1536678.5999))`
`1536678.60`
---
```
help(round(-2.01))
```
Python will return help info on integer cuz `(round(-2.01))` return an integer
str.upper() returns an all-caps version of a string, works for list too
---
The print() function automatically adds a newline character unless we specify a value for the keyword argument end other than the default value of '\n':
```
print("hello")
print("CoderSchool")
print("hello", end='')
print("CoderSchool", end='')
```
Output:
```
hello
CoderSchool
helloCoderSchool
```
---
==**Order of operations**==
**PEMDAS** - Parentheses, Exponents, Multiplication/Division, Addition/Subtraction.
**Min, Max**
`print(min(1, 2, 3))`
`print(max(1,2,3))`
**Absolute values**
`print(abs(-12))`
**Reassignment**
`a, b = 1, 2`
`a, b = b, a`
`print(a, b)`
Output
`2 1`
The // operator gives us a result that's rounded down to the next integer.
`print(75/10)`
`7.5`
`print(75//10`
`7`
**Comparison Operations**
==**Function**==
```
def fun_fun(x):
'''
this is my very fun function will take in one x input and return x + 2
'''
return x + 2
```
**Docstrings(triple quotations)**: explain what the function does
```
def greet(who="CoderSchool"):
print(who, "always be learning!")
greet(who="Alex")
Output
Alex always be learning!
# In this case, we don't need to specify the name of the argument, because it's unambiguous
greet("world")
Output
world always be learning!
```
**Default arguments**:
```
def my_function(age=20):
return(age)
```
**Boolean conversion**
`# All numbers are treated as true, except 0`
`print(bool(1))`
`print(bool(0))`
``# All strings are treated as true, except the empty string ""`
`print(bool("asf"))`
`print(bool(""))`
Generally empty sequences are "falsey" and the rest are "truthy"
==**List**==
`primes = [2, 3, 5, 7]`
`a = ['a', 1, 2]`
`b = [12, 132.43, help]`
```
hands = [
['j','q','k'],
['2','3','4'],
['6','A','K'],
]
```
```
print(hands[2],[1])
```
**Indexing list**
list[start_index:end_index]
[starting_index:ending_index:jump_interval]
`[::-1]` #use for reversing
**Mutating lists**
`planets[3] = 'Malacandra'` #change element 3 into 'Malacandra'
`planets[:3] = ['Mur', 'Vee', 'Ur']`#change first 3 elements into [...]
---
`sorted(planets[:3], reverse=False)` # The planets sorted in alphabetical order
---
`sorted(planets[:3], reverse=True)` # The planets sorted in alphabetical order
---
`primes = [2, 3, 5, 7]
sum(primes)`
`17`
---
`planets.append('Pluto')` #adding an item to the end
---
`a = [1,2,3]`
`b = [4,5,6]`
`c=7`
`a.append(c)`
Output
`[1, 2, 3, 7]`
`a.append(b)`
`[1, 2, 3, 7, [4, 5, 6]]`
`a.extend(b)`
`[1, 2, 3, 4, 5, 6]`
---
list.pop() removes and returns the element that was last added into the list
---
`planets.index('Saturn')` #returning the index of 'Saturn'
---
`"abc" in planets` #determine whether a list contains a particular value
---
==***Tuples***==
`t = (1, 2, 3)` #different to list which use []
* Tuples cannot be **modified** (ie immutable )
```
a = (1,2,3)
b = list(a) #change tuple a into list b
```
---
==**Loops**==
```
s = 'steganograpHy is the practicE of conceaLing a file, message, image, or video within another fiLe, message, image, Or video.'
msg = ''
#print all the uppercase letters in s, one at a time
for char in s:
if char.isupper():
print(char, end='')
```
***While loops***
```
i = 0
while True:
print(i, end=' ')
i += 1
if (i > 10):
break
```
Output
`1 2 3 4 5 6 7 8 9 10`
***List comprehension***
Basic syntax:
[ expression ==for== item ==in list if== conditional ]
Equivalent to:
```
for item in list:
if conditional:
expression
```
```
loud_short_planets = [planet.upper() + '!' for planet in planets if len(planet) < 6]
```
Output
`['VENUS!', 'EARTH!', 'MARS!']`
==**Strings and Dictionaries**==
Some important uses of the backslash(escape) \ character
**String methods**
* str.upper() #allcaps
* str.lower() #alllower
* str.index('fun') #Searching for the first index of a substring
* claim.startswith('fun')
* claim.endswith('fun!')
* str.split()
* '/'.join([month, day, year])
**String format**
```
name = 'milly'
age = 20
print("My name is "+name)
print(f"My name is {name}")
print("My name is {} and my age is {}".format(name, age))
```
==**Dictionaries**==
`numbers = {'one':1, 'two':2, 'three':3}`
In this case `'one'`, `'two'`, and `'three'` are the **keys**, and 1, 2 and 3 are their corresponding **values**.
---> Use curly brackets!
Use the same syntax to add another key-value pair
`numbers[78] = 45`
To return all the keys:
```
for k in numbers:
print(k)
```
To return all the values:
```
for k in numbers:
print(numbers[k])
```
Dictionary comprehension:
```
planets = ['Mercury', 'Venus', 'Earth', 'Mars', 'Jupiter', 'Saturn', 'Uranus', 'Neptune']
planet_to_initial = { planet: planet[0] for planet in planets}
planet_to_initial
```
Output:
```
{'Earth': 'E',
'Jupiter': 'J',
'Mars': 'M',
'Mercury': 'M',
'Neptune': 'N',
'Saturn': 'S',
'Uranus': 'U',
'Venus': 'V'}
```
The in operator tells us whether something is a key in the dictionary:
`'Saturn' in planet_to_initial` ---> `True`
**A for loop over a dictionary will loop over its keys**
dict.keys() and dict.values() to get **keys** and **values**
dict.items():iterate over the keys and values of a dictionary simultaneously (item=**key-value** pair)
**Loop through to get both indexes and keys:**
```
for x, y in enumerate(my_car):
print(x, y)
```
```
0 brand
1 model
2 year
3 driven_by
4 owner
5 (1, 'maintance')
6 (2, 'maintance')
```
==**Regex**== (TO BE CONTINUED)
---
## Bash and Git (Thur, 19/11/20)
**GUI vs CLI**
**CLI advantages:**
* powerful+popular in CS
* best way to use cloud services
* easily work with files contain dataset
* type faster than click
==**Syntax**==
echo"Hello" ---> print Hello
pwd --->print working directory
ls -a ---> list all content
ls -l ---> listing in long format
cd ---> change directory
history ---> show history
==**grep and sed**==
* grep Hello greetings.txt ---> find line with the word “Hello”
* sed ‘s/Hello/Goodbye/‘ greetings.txt ---> substitute “Hello” with “Goodbye” for every line in the file.
* sed printed out changes but not actually change the file.
* echo "Hello" > greetings.txt ---> replaces all of the content (be careful)
* echo "Hello" >> greetings.txt ---> appends to the end of the file
==**Filesystem**==
Files are organized in hierarchical directory structure
**Absolute path**: like latitude, longitude, has the same value no matter where you are
**Relavtive path**: specifies a location relative to where you are
==**Basic command**==
mkdir dir1 dir2 dir 3 ---> creates 3 directories
rmdir dir1 dir2 ---> removes 2 empty directories
touch file1 ---> creates 1 empty file
mv file1 file2 ---> moves or renames 2 files
cp file ---> copies file
rm -R dir ---> deletes non-empty directory
==**Files inspection**==
cat file_name ---> views a file's contents or concatenate several files
less large_file ---> one page is displayed at a time; space bar to page down; q to quit (less serveral files: :n to move to next file, :p to go back, :q to quit
head -n 3 file_name ---> prints the first 3 lines of the file
tail -n 3 file_name ---> prints the last 3 lines of the file
shuf -n 3 file_name ---> prints randomly 3 lines of the file
wc file_name ---> counts number of lines, words, and characters in the file
---
**What git can do:**
* keep track of changes to code
* synchronizes code
* changes to code without loosing the original
**==Git Commands==**
* git clone <url> ---> makes a copy of a repository and stores it on your computer
* git init ---> makes the current folder as a git repository
* git add . ---> add all changed files.
* git status ---> a helpful command, to see what's currently going on in the git repo
* git commit ---> saves the version of the repository.
* git push -u origin master ---> pushes the changes I made on my computer up to the Github repo
* git pull ---> pull the most recent version of the online repository.
* git log ---> to see all the version that you have saved and you can see the commit message is a very good indicator or reminder of what was changed in a particular version
Branching: Branch is a version of the repository. Each branch has its own commit history and current version.
* git branch ---> shows all branches of code
* branch <branch_name> ---> creates a new branch
* git checkout <branch_name> ---> switches to a branch
* git merge <branch_name> ---> merges the branch with current branch
---
## HTML/CSS (Fri, 20/11/20)
**==HTML vs CSS vs Javascript==**
Compulsory components:
1. Web Browser (Google Chrome)
2. Text Editor (Visual Studio Code)
**==What is HTML?==**
* Hyper Text Markup Language
* Every webpage is HTML
* Describe web content
* HTML is made up of elements by tags
**==Tag Syntax==**
<tagname>content</tagname>
<h1>
About Us
</h1>
Attributes:
<tagname attributename="attributevalue">content</tagname>
**Basic Document structure**
**==HTML Tags==**
**Common HTML tags**
* <head>
* <body>
* <span> ---> take only as much space as it needs
**==What is CSS==**
* Cascading Stylesheet
* CCS focuses on presentation, making shit looks nice
* Not a programming language
* Make website layouts + designs
---> Best to put CSS code in external file e.g.
p {
font-size=120%
}
**==CSS Syntax==**
---
**==Semantic tags for CSS==**
---
===**Javascript**==
Make web more dynamic (e.g. changing color after you click a button)
Sign in with Wallet
Connect another wallet