# Python Notes
## Rules
1. Language: English (90%)
2. Notes for one person per week:
* Arrmen >2
* Tsr >2
3. Add name to let us know who u r
4. Can add tags to classify
5. Welcome comments
---
### Arrmen / 111019 [#jupyter]()
This note will give a brief start up for .NET development on Jupyter Notenook.
#### Background
Since Nov., the extension tool Try.Net can support developer write .NET code on Jupyter Notebooks.
#### Enviroment Requirement
* Jupyter Notebooks
* [.NET Core 3.0 SDK](https://dotnet.microsoft.com/download/dotnet-core/3.0)
* [.NET Core 2.1](https://dotnet.microsoft.com/download/dotnet-core/2.1) (for Try.net)
* Try.NET Global Tool 2.1
in .NET core cli

    dotnet tool install -g dotnet-try

* Ananconda Prompt (only for startup)
#### Startup
1. Open Ananconda Prompt to install .NET kernel

    dotnet try jupyter install

2. (Optional) Check to see if the .NET kernel is installed

    jupyter kernelspec list

3. Start a new notebook, u will see C# or F# notebook option, like image below.
#### Reference
* [Microsoft Announcement](https://devblogs.microsoft.com/dotnet/net-core-with-juypter-notebooks-is-here-preview-1/)
* [News from iThome](https://www.ithome.com.tw/news/134068)
### Tsr / 110919 [#python]()
**Difference between * and np.dot or np.matmul**
'* is only for multiplacing corresponding array elements
do not use '*' for matrix multiplcation!
### Arrmen / 110419 [#python]() [#performance]()
This note will give a brief introduction of implementation of python.
<style>
.row-container {
width: 100%;
display: flex;
justify-content: space-around;
align-items: center;
}
.bridge {
text-align: center
}
</style>
<div class="row-container">
<p>code</p>
<p class="bridge">interpreter<br>--------------></p>
<p>byte code</p>
<p class="bridge">complier<br>--------------></p>
<p>machine code</p>
</div>

1. interpreter: byte code can run on different OS, common interpreters like

| Interpreter | Feature |
| ----------- | -------------------------------- |
| CPython | transfer to C byte code |
| Jython | transfer to Java byte code |
| PyPy | transfer to **Python** byte code |
| PythonNet | Mono byte code |
| IronPython | .NET byte code |
2. complier: virtual machine, common compliers like

| Complier | Handle code | Feature |
| -------- | ---------------- | ----------------------------------------------------------- |
| C | C byte code | usually small and fast |
| JVM | Java byte code | equal to Java virtual machine |
| CLR | VB, C# byte code | **only on Windows**; abbreviation of "Common Language Run-time" |
| Mono | VB, C# byte code | **cross OS**. |
* [Mono](https://en.wikipedia.org/wiki/Mono_(software)): a virtual machine based on .Net
* Tips: change interpreter in PyCharm
* In PyCharm(v2019.1.3): `File` > `Settings...` > left navigaton `Project interpreter` > top input of right side `Project interpreter` selector
### Arrmen / 102919 [#python]()
**Python version: ^3.6.0**
This notes will give a brief introduction of sorting function in python.
* **Two function**
1. `<list>.sort(<iterable object>[, key=<sort key function>, reverse=<bool>])`
2. `sorted(<iterable object>[, key=<sort key function>, reverse=<bool>])`
* For Python ^2.2.0, sorting function is **stable**
* Python has used [Timsort](https://en.wikipedia.org/wiki/Timsort) since v2.3.0
* Minimum sorted result has 0 as index without reverse

    example_list_1 = ['Renault', 'Citroen', 'Peugeot', 'Renault']
    example_list_3 = [('Renault', 2), ('Citroen',), ('Peugeot',), ('Renault', 1)]
    example_set = {'Renault', 'Citroen', 'Peugeot'}
    example_tuple = ('Renault', 'Citroen', 'Peugeot')
    
    # sorted() return a <list> without modifiying original data
    example_list_1	         # -> ['Renault', 'Citroen', 'Peugeot', 'Renault']
    example_list_1.sort()       # -> <class 'NoneType'>
    type(example_list_1.sort()) # -> None
    example_list_1	         # -> ['Citroen', 'Peugeot', 'Renault', 'Renault']
    
    # <list>.sort() change data of list, return None
    example_list_1	          # -> ['Renault', 'Citroen', 'Peugeot', 'Renault']
    sorted(example_list_1)       # -> ['Citroen', 'Peugeot', 'Renault', 'Renault']
    type(sorted(example_list_1)) # -> <class 'list'>
    example_list_1	          # -> ['Renault', 'Citroen', 'Peugeot', 'Renault']
    
    # Reverse
    sorted(example_list_1, reverse=True) # -> ['Renault', 'Renault', 'Peugeot', 'Citroen']
    
    # Sort other type iterable object
    sorted(example_set)            # -> ['Citroen', 'Peugeot', 'Renault']
    type(sorted(example_set))      # -> <class 'list'>
    sorted(example_tuple)          # -> ['Citroen', 'Peugeot', 'Renault']
    type(sorted(example_tuple))    # -> <class 'list'>
    sorted('example string')       # -> [' ', 'a', 'e', 'e', 'g', 'i', 'l', 'm', 'n', 'p', 'r', 's', 't', 'x']
    type(sorted('example string')) # -> <clas 'list'>
    
    # Sorted by provided key
    sorted(example_list_1, key=lambda x: x[1])          # -> ['Renault', 'Peugeot', 'Renault', 'Citroen'] -> only sorted by second character
    sorted(example_list_1, key=lambda x: x[1:len(x)-1]) # -> sorted by [1:string end]
    
    # Stable sort probement
    sorted(example_list_3, key=lambda example: example[0]) # -> [('Citroen',), ('Peugeot',), ('Renault', 2), ('Renault', 1)] -> need to limit sorting reference
    
    example_list_3.sort(key=lambda example: example[0])
    example_list_3 # -> [('Citroen',), ('Peugeot',), ('Renault', 2), ('Renault', 1)] -> need to limit sorting reference too

#### Reference
* [3.6.9 Official Document](https://docs.python.org/3.6/howto/sorting.html#sort-stability-and-complex-sorts)
### Tsr / 102519 [#python]()
**(Difference between \*args and \*\*kwargs)**
> def myFun(arg1, *argv):
print ("First argument :", arg1)
for arg in argv:
print("Next argument through *argv :", arg)
myFun(**<u>'Hello'</u>**, 'Welcome', 'to', 'GeeksforGeeks')
vs
def myFun(\*\*kwargs):
for key, value in kwargs.items():
print ("%s == %s" %(key, value))
myFun(**<u>first=</u>**'Geeks', mid ='for', last='Geeks')

**lambda example when used in indexing**
> X.loc[lambda f: f.category_name =='comp.graphics'].iloc[::10][:5]

from lambda f, all rows in X with category name, comp.graphics is fetched
from X.loc[], the resulting rows from previous query is queried and forms a dataframe
this resulting dataframe is iloc using index, a vector of [start: end:interval](parameters of iloc)[start:end](slicing the results)

### Tsr / 102119 [#python]()
**Typical visualisation using matplotlib.pyplot**
- plt.subplots(figsize=(9, 7)) # subplots. can explore other parameters. Figsize is especially useful in **jupyter notebook** so that the diagram fits in the browser
- x = sent_tag_dic.keys() # x values
- y = [len(sent_tag_dic[k]) for k in sent_tag_dic.keys()] # y values
- plt.bar(x, y) # visualisation model
- plt.show() # to show the graphs, can be executed after running a few blocks of (subplot, bar)

**Comments on zip()**
Definition: The zip() function returns an iterator of tuples based on the iterable object.
- If no parameters are passed, zip() returns an empty iterator
- If a single iterable is passed, zip() returns an iterator of 1-tuples. Meaning, the number of elements in each tuple is 1.
- If multiple iterables are passed, ith tuple contains ith Suppose, two iterables are passed; one iterable containing 3 and other containing **5** elements. Then, the returned iterator has **3** tuples. It's because iterator stops when **<u>*shortest*</u>** iterable is exhaused.
### Arrmen / 102019 [#python]()
This note will give some funny ways of quickly initializing an iterable object with default value.
-> iterable object: includes `list`, `tuple`, `set`...

    # 1. Use for-loop
    [i*i for i in range(5)] # -> [0, 1, 4, 9, 16]
    {'Hi' for i in range(5)} # -> {'Hi'} -> because it's a set
    (i*i for i in range(5)) # -> <generator object <genexpr> at 0x...>
    type(i*i for i in range(5)) # -> <class 'generator'>
    tupe(i*i for i in range(5)) # -> (0, 1, 4, 9, 16)
    i*i for i in range(5) # -> SyntaxError: invalid syntax
    
    # 1.5 Complex case: combine with zip(), map(), enumerate()...
    example_list_1 = ['Mazda', 'ISUZU', 'MINI'];
    ['index of {} is {}'.format(car, index) for index, car in enumerate(example_list_1)]
    # -> ['index of Mazda is 0', 'index of ISUZU is 1', 'index of MINI is 2']
    
    # 2. With operator; only for times operator
    # ! This method doesn't support <set> object
    ['Hi'] * 5 # -> ['Hi', 'Hi', 'Hi', 'Hi', 'Hi']
    list('Hi') * 5 # -> ['H', 'i', 'H', 'i', 'H', 'i', 'H', 'i', 'H', 'i']
    [2] * 5 # -> [2, 2, 2, 2, 2]
    list(2) * 5 # -> TypeError: 'int' object is not iterable
    ('Hi',) * 5 # -> ('Hi', 'Hi', 'Hi', 'Hi', 'Hi')
    tuple('Hi') # -> ('H', 'i', 'H', 'i', 'H', 'i', 'H', 'i', 'H', 'i')

#### Reference
* [Thispointer.com](https://thispointer.com/python-how-to-create-a-list-and-initialize-with-same-values/), 03/04/18
### Arrmen / 101519 [#python]() [#file]()
This note will give an introduction of handling **.txt file**
Content in _example.txt_

Bently
Volkswagen

0. Open file `open(<path with file name> [, <mode>])`: 'mode' can be refered from [W3School](https://www.w3schools.com/python/python_file_handling.asp), example

file = open('example.txt', 'a+')
print(type(file)) # -> <class '_io.TextIOWrapper'>`

2. Close file `close()`: example `file.close()`
3. Read: **can't read file multiple times(see below examples), so use a variable to store contents at first**

    file = open('example.txt', 'r')
    # read()
    print(file.read()) # -> Bently /n Vlokswagen
    print(file.read()) # -> if u call this function many times,
                       # it will return nothing after first call,
                       # because file has been read end

    # read a single line: readline()
    print(file.readline(), file.readline()) # -> Bently /n Vlokswagen

    # divide contents by line: readlines() -> <class list>
    # improper solution
    for line in file.readlines():
        print(line) # -> [] -> because file has been read completely at first call 'readlines()'
    # better solution
    line_list = file.readlines()
    for line in line_list:
        print(line) # -> Bently /n Vlokswagen

3. Write

    file = open('example.txt', 'a+')
    # write single value: write(<string>>), return int(length of <string>)
    file.write('\nBugatti') # -> Bently \n Volkswagen \n Bugatti

    # write multiple values at once: writeines(<string list>), return <None>
    file.writelines(['\nSaab', '\nScania']) # -> Bently \n Volkswagen \n Saab \n Scania

4. Delete: needs to import package `os`

    import os
    # remove single file: remove(<path with file name)
    os.remove('example.txt')    

5. Create: can automatically be done when mode is 'a' or 'w', but also can use mode 'x'. Remember add **file extension**

    # Create example.txt if it doesn't exist
    open('example.txt', 'x')
    
    # Create multiple type of file with file extension if it doesn't exist
    open('example.png', 'x') # -> create a .png

* Tips: use `close()` to update file's state

    file_write = open('example.txt', 'a+')
    file_read = open('example.txt', 'r')
    file_write.write('\nSuzuki')
    file_write.close()
    print(file_read.read()) # print Bently \n Volkswagen \n Suzuki
                            # same as content in example.txt
    file_write = open('example.txt', 'a+')
    file_read = open('example.txt', 'r') # open and write
    file_write.write('\nSuzuki')
    print(file_read.read()) # print Bently \n Volkswagen
                            # but content in example.txt is Bently \n Volkswagen \n Suzuki
    file_write = open('example.txt', 'a+')
    file_write.write('\nSuzuki') # write and open
    file_read = open('example.txt', 'r')
    print(file_read.read()) # print Bently \n Volkswagen
                            # but content in example.txt is Bently \n Volkswagen \n Suzuki

#### Reference
* [Python file handling](https://www.guru99.com/reading-and-writing-files-in-python.html)
* [W3School](https://www.w3schools.com/python/python_file_handling.asp)
### Arrmen / 101319 [#python]() [#application]()
This note will give 2 funny functions related to 'list'.
1. Filter `filter(<function>, <iterable object>)`, return a `<class 'filter'>` instead of a `<iterable object>`
* **Usage:** check whether every element satisfies the conditional function. If passed, put it into a 'filter' object
* same as [`map()`](https://hackmd.io/vG-86mkjQrWnsRn3qB1aVA?both#Arrmen--101319-python-application1)

    def example_func(x):
        return x > 3
    
    list_1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
    set_1 = {2, 5, 6, 8, 9}
    tuple_1 = (2, 5, 6, 8, 9)

    filter(example_func, list_1) # -> <filter object at 0x...>
    list(filter(example_func, list_1)) # -> [4, 5, 6, 7, 8, 9, 10]
    
    # Use lambda instead of function
    list(filter(lambda x: x > 5, list_1)) # -> [6, 7, 8, 9, 10]
    
    # Support many iterable objects
    list(filter(lambda x: x > 5, set_1) # -> [6, 8, 9]
    list(filter(lambda x: x > 5, tuple_1) # -> [6, 8, 9]
    list(filter(lambda x:x > 'g', 'example string')) # -> ['x', 'm', 'p', 'l', 's', 't', 'r', 'i', 'n']
    # -> character can compare directly with ascii code

    # Support many output type
    set(filter(lambda x: x > 5, list_1)) # -> {6, 7, 8, 9, 10}
    tuple(filter(lambda x: x > 5, list_1)) # -> (6, 7, 8, 9, 10)

2. Zip `zip(<iterable object>[, <iterable object>, ...])`, return a `<class 'zip'>` instead of a `<iterable object>`
* **Usage:** combine multiple iterable object element with **same index** into **tuple**, and put tuple into 'zip' class

    list_1 = [1, 2, 3, 4, 5]
    list_2 = [6, 7, 8, 9, 10]
    set_1 = {1, 2, 3 ,4, 5}
    tuple_1 = (6, 7, 8, 9, 10)

    zip(list_1, list_2) # -> <zip object at 0x...>
    list(zip(list_1, list_2)) # -> [(1, 6), (2, 7), (3, 8), (4, 9), (5, 10)]
    
    # Support many/multiple iterable objects
    list(zip(set_1, tuple_1)) # -> [(1, 6), (2, 7), (3, 8), (4, 9), (5, 10)]
    list(zip('Luxgen', 'Lamborgnini')) # -> [('L', 'L'), ('u', 'a'), ('x', 'm'), ('g', 'b'), ('e', 'o'), ('n', 'r')]
    list(zip(set_1, tuple_1, 'Lexus')) # -> [(1, 6, 'L'), (2, 7, 'e'), (3, 8, 'x'), (4, 9, 'u'), (5, 10, 's')]

    # Support many output type
    set(zip(list_1, list_2)) # -> {(2, 7), (5, 10), (4, 9), (3, 8), (1, 6)} -> unindex is correct
    tuple(zip(list_1, list_2)) # -> ((1, 6), (2, 7), (3, 8), (4, 9), (5, 10))
    
    # Combine with filter/map
    def compare_function(tuple_param):
        element_from_list_1 = tuple_param[0]
        element_from_list_2 = tuple_param[1]
        return element_from_list_2 > element_from_list_1*2 + 2

    list(filter(compare_function, zip(list_1, list_2))) # -> [(1, 6), (2, 7)]

    # Loop with zip
    for list_1_element, list_2_element in zip(list_1, list_2):
        print(list_1_element, list_2_element) # -> 1 6/2 7/3 8/4 9/5 10

#### Reference
* [zip & filter introduction](https://yangfangs.github.io/2017/08/23/python-map-zip-filter-reduce/)
* [document for zip](https://docs.python.org/3/library/functions.html#zip)
* [document for filter](https://docs.python.org/3/library/functions.html#filter)
### Arrmen / 101319 [#python]() [#application]()
This note will give 2 funny tips related to 'list'.
1. String to list. Use `split(<string>|<None>, <int>)`

    str = 'Nissan Mitsubishi Hyundai'
    str.split() # -> ['Nissan', 'Mitsubishi', 'Hyundai']
    str.split(None) # -> ['Nissan', 'Mitsubishi', 'Hyundai']
    str.split('s') # -> ['Ni', '', 'an Mit', 'ubi', 'hi Hyundai']
    str.split('ss') # -> ['Ni', 'an Mitsubishi Hyundai']
    str.split('5') # -> ['Nissan Mitsubishi Hyundai']

    # second parameter means (the max split - 1)
    str.split(None,1) # -> ['Nissan', 'Mitsubishi Hyundai']
    str.split(None,10000) # -> ['Nissan', 'Mitsubishi', 'Hyundai']

2. Map `map(<function>, <iterable object> [, <iterable object>, ...])`return a `<class 'map'>` instead of a `<iterable object>`
* **Usage:** map every element and put element into a 'map'
* Same as [filter and reduce](https://hackmd.io/vG-86mkjQrWnsRn3qB1aVA?both#Arrmen--101319-python-application)

    def example_func(x):
        return x ** 2

    list1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
    map(example_func, list1) # -> <map object at 0x...>
    list(map(example_func, list1)) # -> [1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

    # Use lambda instead of function
    list(map(lambda x: x **2, list1)) # -> [1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
    
    # > 1 list parameters
    list2 = [10, 20, 30, 40, 50, 60, 70, 80, 90, 100]
    list(map(lambda x, y: x + y, list1, list2)) # -> [11, 22, 33, 44, 55, 66, 77, 88, 99, 110]
    list(map(example_func, list1, list2)) # -> TypeError: <lambda>() takes 1 positional argument but 2 were given
    
    # Use map to create <set> and <tuple>
    set(map(example_func, list1)) # -> {1, 4, 9, 16, 25, 36, 49, 64, 81, 100}
    tuple(map(example_func, list1)) # -> (1, 4, 9, 16, 25, 36, 49, 64, 81, 100)
    
    # Use multiple types as parameters
    tuple1 = {20, 40, 60}
    list(lambda x, y: x + y, tuple1, list1) # -> [21, 42, 63]
    list(lambda x, y: x + str(y), 'example string', list1) # -> ['e1', 'x2', 'a3', 'm4', 'p5', 'l6', 'e7', ' 8', 's9', 't10']
    tuple(map(lambda x: x+1, tuple1)) # -> (21, 41, 61)
    # -> type transformation is <tuple> -> <map> -> <tuple>)

#### Reference
* [string split]('https://www.mkyong.com/python/python-how-to-split-a-string/')
* [map]('https://yangfangs.github.io/2017/08/23/python-map-zip-filter-reduce/')
### Arrmen / 100619 [#python]() [#data-type]()
This notes will give a basic introduction of 'tuple'
1. **Unchangeable**: not like python 'list', value in tuple can't be modified
-> **can't** remove, append, replace, split, slice ...
-> **can** search, count, get length
2. **allows duplicate members**, not like python 'set', python 'dict'
3. type of element in tuple doesn't need to be same; allows tuple in tuple
4. basic method or usage

    # use below as example tuple
    exampleTuple = ("apple", "banana", "cherry", "orange", "kiwi")
    
    # get length: same as 'list'
    len(exampleTuple) # -> '5'
    
    # join tuple: same as 'list'
    tmpTuple1 = ("toyota", "ford")
    tmpTuple2 = ("volvo", "hammer")
    exampleTuple = tmpTuple1 + tmpTuple
    print(exampleTuple) # -> ("toyota", "ford", "volvo", "hammer")
    
    # access by index: same as 'list', but type of return is element itself
    exampleTuple[1] # -> "banana"
    
    # access by range: same as 'list', but type of return is tuple
    exampleTuple[0:2] # -> ("apple", "banana")
    
    # loop: same as 'list'
    for item in exampleTuple:
        print(item) # -> "apple", "banana" ...
    
    # check element is in tuple
    # also can use .count(), .index() to check
    if "apple" in exampleTuple:
        print("true") # -> "true"
        
    # create tuple
    # multiple element case -> declare directly
    exampleTuple = ("bmw", "ferrari")
    # single element case -> add comma
    exampleTuple = ("chrysler")
    type(exampleTuple) # -> "string"
    exampleTuple = ("chrysler",)
    type(exampleTuple) # -> tuple
    
    # count(value): get how many times a element duplicates
    exampleTuple.count("apple") # -> 1
    
    # index(value): get index of first matching element, throw exception as no result
    exampleTuple("banana") # -> 1
    exampleTuple("bananas") # -> error exception

* Tips: use `type()` to check data's type
* Tips: like python 'list', tuple support negative index, but [0] is same as [-0]
* Tips: use `exampleTuple[n:n]` will get `()`
* Tips: use `exampleTuple[n:n+1` will get `(exampleTuple[n],)` -> a tuple
* Tips: change value: can't directly change, but can transform to 'list' and transform the list again to 'tuple''

    print(exampleTuple) # -> ("apple", "banana", "cherry", "orange", "kiwi")
    tmpList = list(exampleTuple)
    list[1] = "porsche"
    exampleTuple = tuple(list)
    print(exampleTuple) # -> ("apple", "porsche", "cherry", "orange", "kiwi")

* Tips: to access double tuple, use the same way as double 'list
#### Reference
* [W3School](https://www.w3schools.com/python/python_tuples.asp)
### Tsr / 100419
- multiple assignment of string chars. e.g. t1,t2,t3='hii' t1='h'
- backslash to escape quotation e.g. circus = 'Monty Python\'s Flying Circus'
- backslash to separate codes onto multi lines (not needed if inside brackets)
- indexes: -1 is the index of the last character
- regex: re.finall(r.......) e.g. re.findall(r'^.*(ing|ly|ed|ious|ies|ive|es|s|ment)$', 'processing') *Remember to prefix regular expressions with the letter r (meaning "raw"), which instructs the Python interpreter to treat the string literally, rather than processing any backslashed characters it contains.*
- string formatting: e.g. '{}->{};'.format ('cat', 3) 'cat->3;'
- aligning:
>>> '{:6}'.format(41) [1]
' 41'
>>> '{:<6}' .format(41) [2]
'41
- sf:
>>> '{:.4f}'.format(math.pi) '3.1416'

### Arrmen / 100219 [#python]() [#windows]() [#install]()
This notes will share how i install python at Windows 10.
[Download here](https://www.python.org/downloads/windows/), according your computer architecture (bits 64 or 32)
* Follow GUI set-up instruction, **choose folder carefully**
* How to use python in terminal?
1. go to **'System'**(`Win` + `PAUSE`)
2. find **'Environment variables setting'**(`Advanced system setting`)
3. click **'Path'** option, click edit
4. add the path to **'python.exe'**(I don't use default path, so i forget it; possibly in ../ProgramFile x86/...)
5. apply, and test on ur terminal
* Tips: GUI = Graphic Unit Interface, CLI = Command Line Interface ~= terminal
* Tips: After initializing get-pip.py (at same folder as python.exe, usually), u can add its path (../Script) into Path too.
#### Reference
* [Set-up tutorial](https://realpython.com/installing-python/)
* [CLI setting](https://superuser.com/questions/143119/how-do-i-add-python-to-the-windows-path)

### Tsr / 100119
- install python using *homebrew* to prevent issues
- [] is a list, index starts from 0
- trimming by indexing, e.g. [1:3] gets index 1&2
- lambda to apply functions to each row, e.g. X['category_name'] = X.category.apply(lambda t: dmh.format_labels(t, twenty_train))