---
# System prepended metadata

title: Python Basic
tags: [language]

---

---
tags: language
---
Python Basic
===
* News & Trend:
    * [Awesome Python Newsletter](https://python.libhunt.com/newsletter)
    * [What do companies expect from Python devs in 2019?](https://hackernoon.com/what-do-companies-expect-from-python-devs-in-2019-f8b585a1ddf)
* Resource：
    * [Big Book of Python](https://www.bigbookofpython.com/)
    * [Practical Python Projects](https://practicalpython.yasoob.me/)
    * [The Hitchhiker’s Guide to Python!](https://docs.python-guide.org/)
    * [Legally Free Python Book List](https://www.pythonkitchen.com/legally-free-python-books-list/)
    * [Learn Programming For Free (All Free Resources)](https://dev.to/nerdjfpb/learn-programming-for-free-all-free-resources-2b3j)
    * [Hypermodern Python](https://cjolowicz.github.io/posts/hypermodern-python-01-setup/)
* Great Tool:
    * [explainshell.com: command line explain tool](https://explainshell.com/)
    * [No Really, Python's Pathlib is Great](https://rednafi.github.io/digressions/python/2020/04/13/python-pathlib.html)
    * [A new kind of Progress Bar, with real time throughput, eta and very cool animations!](https://github.com/rsalmei/alive-progress)


# Best Practices
* [Clean Code in Python](https://testdriven.io/blog/clean-code-python/)
* [The Best of the Best Practices (BOBP) Guide for Python](https://gist.github.com/sloria/7001839)
* [Logging in Python](https://realpython.com/python-logging/)
* [Pysa: An open source static analysis tool to detect and prevent security issues in Python code](https://engineering.fb.com/security/pysa/)
* [Complete Guide How To Code Review](https://dev.to/nickbulljs/complete-guide-how-to-code-review-2837)

## Code Style and Structure
* Python style guide
    * [PEP8：Style Guide for Python Code](https://www.python.org/dev/peps/pep-0008/)
    * [The Elements of Python Style](https://github.com/amontalenti/elements-of-python-style)
    * [How to Write Beautiful Python Code With PEP 8](https://realpython.com/python-pep8/)
    * Tools to check code with PEP8
        * [pylint](https://pypi.org/project/pylint/)
        * [pycodestyle](https://pypi.org/project/pycodestyle/)
    * [PEP257：Docstring Conventions](https://www.python.org/dev/peps/pep-0257/)
* [A Pythonic Guide to SOLID Design Principles](https://realpython.com/simpy-simulating-with-python/)
* [Clean Architectures in Python](https://leanpub.com/clean-architectures-in-python)


## Python Modules
* [Python 的 Import 陷阱](https://medium.com/pyladies-taiwan/python-%E7%9A%84-import-%E9%99%B7%E9%98%B1-3538e74f57e3)
* [Python Modules: Creating, Importing, and Sharing](https://stackabuse.com/python-modules-creating-importing-and-sharing/)
* What is Python modules：
    *  highest level organizational unit in Python
    *  the units that store code and data, provide code-reuse to Python projects
    *  useful in partitioning the system's namespaces in self-contained packages
    *   In fact, any Python file with a .py extension represents a module.
* Basic Concept：
    * **Script**：
        * In computing, the word script is used to refer to a file containing a logical sequence of orders or a batch processing file.
        * A plain text file containing Python code that is intended to be directly executed by the user is usually called scrip
        * script in python is an informal term that means **top-level program file** (__name__ == '__main__')
        *  'top-level' mean any function or variable which is not nested inside another function or class. 
        *  a plain text file, which contains Python code that is designed to be imported and used from another Python file, is called module.
    * Python files which are used to run as a stand-alone Python app (top-level files) are usually designed to run as script
    * import them would actually run the commands in the script
    * Modules which are designed to be imported by other code won't execute any code, but only expose its top-level names as attributes to the imported object.
    * Python modules which could be used for both - importing and running as a top-level script.
* Program Architecture：
    *  In general, we can distinguish three types of files in a multi-file Python application:
        *  top-level file：
        *  user-defined modules：
        *  standard library modules：

## Python Packaging 
* [An Overview of Packaging for Python](https://packaging.python.org/overview/)
* [Python dependency management and packaging made easy.](https://github.com/python-poetry/poetry)

##  Python Deployment
* [Cookiecutter PyPackage Tool](https://github.com/lgiordani/cookiecutter-pypackage)
* deployment questions below:
    * Who are your software’s users? Will your software be installed by other developers doing software development, operations people in a datacenter, or a less software-savvy group?
    * Is your software intended to run on servers, desktops, mobile clients (phones, tablets, etc.), or embedded in dedicated devices?
    * Is your software installed individually, or in large deployment batches?
* PyPI,, ```setup```, ```wheel``` tools：
    * Python’s ecosystem provides for distributing Python code to **developers**
*  Wheel - Built-package format:
    * [Python Wheels](https://pythonwheels.com/)
    * [PEP 427 -- The Wheel Binary Package Format 1.0](https://www.python.org/dev/peps/pep-0427/)
    * pure python code package can use sdist for deployment
    * package include other language code, using **wheel** 
    * wheel is a package format designed to ship libraries with compiled artifacts
    * ```pip``` always prefers wheels because installation is always faster, so even pure-Python packages work better with wheels


### Project Stucture
* [Structuring Your Project](https://docs.python-guide.org/writing/structure/)
In practical terms, **structure** means making clean code whose logic and dependencies are clear as well as how the files and folders are organized in the filesystem.

#### Structure of the Repository:
simple repository example:
```
README.rst
LICENSE
setup.py
requirements.txt
simple/
 -sample/__init__.py  #sample is your module name
 -sample/core.py
 -sample/helpers.py
docs
 -docs/conf.py
 -docs/index.rst
tests
 -tests/test_basic.py
 -tests/test_advanced.py
```
Let's get into some specifics.
* The Actual Module:
    * Location: ```./module_name``` or ```module_name.py```
    * Purpose: clear show the module core
* License:
    * Location:```./LICENSE```
    * Purpose: lawyering up
    * check out [choosealicense.com](https://choosealicense.com/) and pick the full license text and copyright claims 
* Setup file:
    * Location: ```./setup.py```
    * Purpose: Package and distribution management
* Requirement File:
    * Location: ```./requirement.txt```
    * Purpose: Development dependencies
    * check [pip requirements file](https://pip.pypa.io/en/stable/user_guide/#requirements-files) 
* Test Suite:
    * Location: ```./tests.py```or```test_module_name.py```
    * Purpose: Package integration and unit tests
    * How to import module in test file?
        * Expect the package to be installed in site-packages
        * (recommend) Use a simple (but explicit) path modification to resolve the package property
    * Reference: [Testing Your Code](https://docs.python-guide.org/writing/tests/)
* Makefile:
    * Location:```./Makefile```
    * Purpose: Generic management tasks
    * **make** is an incredibly useful tool for defining generic tasks for your project (not just for C) 
```bash
init:
    pip install -r requirements.txt
    
test:
    py.test tests
.PHONE: init test
```
* Doc:
    * [Great tool for Documentation created- Sphinx ](http://www.sphinx-doc.org/en/master/)
 
 
# Testing
* Resource:
    * [Testing Your Code](https://docs.python-guide.org/writing/tests/)
    * [TDD (from IT幫30天)](https://ithelp.ithome.com.tw/users/20010292/ironman/462)
    * [Write Unbreakable Python](https://dev.to/jesterxl/write-unbreakable-python-20ck)
    * [Ultimate Guide to Python Debugging](https://towardsdatascience.com/ultimate-guide-to-python-debugging-854dea731e1b)
    * [Testing Best Practices for Machine Learning Libraries](https://towardsdatascience.com/testing-best-practices-for-machine-learning-libraries-41b7d0362c95)
* Note:
    * Focus on one tiny bit of functionality.
    * Should be fast, but a slow test is better than no test.
    * It often makes sense to have one testcase class for a single class or model

## TDD
* What is TDD?
    * Test-Driven Development (TDD) 
    * TDD forces you to clearly state your goal before you write the code.
    * TDD mantra is “Test first, code later”
* Important keys:
    * Test first, code later
    * Driving by the test, not by the requirements
    * you should try to get to a situation in which only one test fails.
    * test boundary cases 
        * finding the so-called "corner cases"
        * showing the code covers them
    * Write code that passes the test. Then refactor it.
    * A test should fail the first time you run it. If it doesn’t, ask yourself why you are adding it.
    * Never refactor without tests.
        * Refactoring : changing the code without altering the results
        * How can you be sure you are not altering the behaviour of your code? Well, this is what the tests are for.

## Python test tools
### IDE or Editor:
* Break Point: IDE 通常可以設置程式執行時預計要停止的多個中斷點，一段一段執行時一邊檢視變數狀況，不用一直去 print() 。
...

### assert(斷言):
* 目的：用一些表達式測試程式中非預期的偶發狀況，例如某些變數不可以為負的測試...等。
* 用途：安插在程式中，判斷某些 expression 是否為假，如果為假可以回傳訊息並中斷程式。
* 語法：```assert expression, 'how some message'```
```python
>>> product_amount = -1
# 寫下預期都要大於0的 expression, 並在沒通過時秀出訊息
>>> assert product_amount>0, 'product_amount is negative!'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError: product_amount is negative!
```
* 優點：用 if statement 也可以輕易完成的事情，改用 assert 的好處是，我們可以在特定執行下不進入 assert 的判斷，也就是說測試完後就讓 assert 功臣身退，但不用去修改或刪除相關程式碼：
    * 執行 python 時給定參數 -O: ```python -O```
```python
$ python -O
Python 3.7.1 (default, Dec 14 2018, 19:28:38) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> product_amount = -1
>>> assert product_amount>0, 'product_amount is negative!'
>>> 
```
* 進一步說明：
    * __debug__ 是 python 內建變數，一般情況是 True，如果執行時需要最佳化(在執行時加上-O引數)，其會是 False。
    * 一般使用 ```assert expression``` 相當於以下程式片段：
    ```python
    if __debug__:
        if not expression: raise AssertionError
    ```

### pytest
* [pytest:helps you write better programs](https://docs.pytest.org/en/latest/)

### unittest modules
* [unittest.mock](https://docs.python.org/3/library/unittest.mock.html)
* unittest modules 有時稱為 PyUnit, 是 JUnit 的 Pyhton 實現
* 單元測試：測試一個工作單元的行為，其單為基本上要**獨立**，否則會測試到相關交互合作行為
* 以軟體而言，單元測試通常指的是測試某個函式(或方法)
* Python 的 unittest 模組主要包括4個部份：
    * 測試案例(Test case):測試最小單元
    * 測試設備(Test fixture)
    * 測試套件(Test suite)
    * 測試執行器(Test runner)

* unittest.TestCase:
```python
import unittest
  
class TestStringMethods(unittest.TestCase):
    def test_upper(self):
        expected = 'FOO'
        result = 'foo'.upper()
        self.assertEqual(result, expected)
    def test_isupper(self):
        self.assertTrue('FOO'.isupper())
        self.assertFalse('Fool'.isupper())
    def test_split(self):
        s = 'hello world'
        result = s.split()
        expected = ['hello', 'world']
        self.assertEqual(result, expected)

if __name__=='__main__':
    unittest.main()
```
Test result:
![](https://i.imgur.com/k25ocJS.png)

### Other
* [Python Mocking 101: Fake It Before You Make It](https://www.fugue.co/blog/2016-02-11-python-mocking-101)
* doctest modules
* pytest 
* Hypothesis
* fixtures replacement tool
    * [Prefer factories to fixtures.](https://github.com/FactoryBoy/factory_boy)


# Clean Architecture
* [Book: clean architectures in Python](https://leanpub.com/clean-architectures-in-python)
* Talk inwards with simple structures, talk outwards through interfaces.
* Main Layer：
    * Entities：
        * domain model
        * Entities don’t know details about the external interfaces, and they only work with interfaces.
    * Use cases：
        * the processes that happen in your application
        * It is very important to isolate small actions in use cases, as this makes the whole system easier to test, understand and maintain.
        * Use cases know the entities, so they can instantiate them directly and use them.
    * External systems
        * This part of the architecture is made by external systems that implement the interfaces (such as API) defined in the previous layer.
        * The word API is of uttermost importance in a clean architecture.

# Python Foundation
* Python3.5 技術手冊-林信良
## Common Sense
* **lib & lib/pythonx.x** : 放置標準函式庫(原始碼)
    * lib/pythonx.x/site-packages: 第三方程式庫
* ```sys.path``` can show the directory path list of python modules.
    * ```import this``` show the zen of Python
* Python interpernet default encoding is UTF-8(same in **bash on shell**, but MS not)
    * In *.py file, first line can write command to change encoding method:
        ``` python
        # coding= <encoding name>
        # coding: <encoding name>
        ```
        EX:
        ```python
        # coding: Big5
        # coding= UTF-8
        ```
* Basic Modules:
    * a *.py file is a modules, can use ```import``` cammand to include. 
    * ```dir(__builtins__)``` can show the build in modules.
    * After executing *.py file and import some mosules, a "__pycache__" directory and some *.pyc files will be created.  It's a binary file that be created by  interpreter. If there is no any change of *.py, the *.pyc file will be used to accelerate the interpreter speed.
    * ```print(sys.path)```: show whole module path. 
    * ```sys.path.append('/dirctory/path')```: 增加模組 import 路徑。
* ```$ python -c "coding"```: 可以執行一小段程式碼，方便測試。
* [What does if __name__ == “__main__”: do?](https://stackoverflow.com/questions/419163/what-does-if-name-main-do)


## Built-in Types
* The principal built-in types are 
    * numerics 
    * sequences 
    * mappings 
    * classes
    * instances
    * exceptions
* most of built-in objects considered false:
    * constants defined to be false: None, False
    * zeor of any numeric types: 0, 0.0, 0j, Decimal(0), Fraction(0, 1)
    * empty sequences and collections: '', [], (), {}, set(), rnage(0).
*  Boolean operations - and, or, not
    *  and, or, is a short-circuit operator.
    *  not has a lower priority than non-Boolean operators, so ```not a == b``` is interpreted as ```not (a == b)```

### Numeric Data Type: int, float, complex
* integer：int()
    * Python3 後不分 **int** / **long**，長度不受限制（物理限制以外）
    * 轉換進位基底的方式: 
      ```python
      int('010',  2) # 2
      int('010',  8) # 8
      int('010', 16) # 16
      
      0b010 # 2
      0o010 # 8
      0x010 # 16
      
      bin(10) #'0b1010'
      oct(10) #'0o12'
      hex(10) # 0xa
      ```
    * 任何進位的整數都是 int 類別的實例:
      ```type(0o10) #<class 'int'>```
* Booleans: a subtype of intefers
    * bool():可以傳入任何型態，將 None, False, 0, 0.0, 0j, '', (), [], {}, 等傳入都會返回 False, 其他非0直回傳 True 
* float point: float()
    * Floating point numbers are usually implemented using **double** in C
    * ```sys.float_info```：show floating point information 
    * ```1e-1 + 1e-2``` # 0.11
    * complex: ```z=a+bj```. complex numbers have a real and imaginary part, which are each a floating point number. To extract these parts from a complex number z, use ```z.real``` and ```z.imag```.
    * 3 * 0.1 == 0.3 # Got False
        * 解法1:```round(.1*3, 2)==round(.3, 2) # True```
        * 解法2：```from decimal import Decimal``` 此套件重現 10進位經準度
    * some method:
        * ```(0.25).as_integer_ratio() # (1, 4)```
        * ```(1.0).is_integer() # True```

### Bytes
```python
import os
import io
from bitstring import BitArray

from PIL import Image

with io.BufferedReader(io.FileIO('IMAG0733.jpg', "r")) as f:
    print(f)
    img_bytes = f.read()
binary = BitArray(img_bytes).bin

with io.BufferedReader(io.FileIO('IMAG0733.jpg', "r")) as f:
    print(Bits(f))
    img_bytes = f.read()
```
* [Inserting pyodbc.Binary data (BLOB) into SQL Server image column](https://stackoverflow.com/questions/11836924/inserting-pyodbc-binary-data-blob-into-sql-server-image-column)
* [Python SQLite BLOB to Insert and Retrieve file and images](https://pynative.com/python-sqlite-blob-insert-and-retrieve-digital-data/)


### Sequence Types — list, tuple, range:
* String: Text Sequence Type
```
type('string') # <class 'str'>
type("string") # <class 'str'>
print("'string'") # 'string'
print('"string"') # "string"
```
* 字串轉義表示：
    * ```\\ => \```
    * ```\' => '```
    * ```\n, \t, \r```


## Function in Python
* What is First class objects： 
    * in a language are handled uniformly throughout. 
    * They may be stored in data structures, passed as arguments, or used in control structures. 
    * A programming language is said to support first-class functions if it treats functions as first-class objects.  
* Python supports the concept of **First Class functions**：
    * A function is an instance of the Object type.
    * You can store the function in a variable.
    * You can pass the function as a parameter to another function.
    * You can return the function from a function.
    * You can store them in data structures such as hash tables, lists, …


## Deal With Exception
* Introduction
    * 當某些原因使執行中的程式無法繼續，Python 中可以**引發例外(Raise Exception)** 
    * Python 可以在例外發生時加以處理，可以引發例外，自訂例外
* ```try``` & ```except```
    * example:
        ```python
        try:
            numbers = input('Enter multi number with space between them: ').split(' ')
            print('Mean:'sum(int(numbers) for number in numbers) / len(numbers))
        except: ValueError as err:
            print(err)
        ```
    * python 直譯器程式執行 try 區塊中的程式，如果發生例外會跳離發生點，去比對 **except** 宣告的型態，是否符合引發的**例外物件**型態，如果是就執行 except 區塊程式碼
    * 在 pyhton 中，例外並不一定是錯誤，例如，**for in** 語法中其底層就運用例外機制
        * 進一步說明，只要具有 ```__iter__()``` 方法的物件，都可以用 for in 來迭代
        * ```__iter()``` 方法應該傳回一個迭代器(iterator)
        * 該迭代器具有 ```next()``` 方法，每次迭代就會傳回下一個物件，沒下個元素時，則會引發 **StopIteration**
        ![](https://i.imgur.com/tSJ4UWm.png)
        * **for in** 會在遇到 StopIteration 時，默默結束迭代，流程比較類似：
            ```python
            def for_in(iterable, do_it):
                iterator = iter(iterable)
                try:
                    while True:
                        do_it(next(iterator))
                except StopIteration:
                    pass
            ```
    * ```except``` 後可以用 tuple 指定多個例外物件，也可以多個 except，如果 ```except``` 後沒指定例外物件，那麼會捕捉所有引發的物件
        ```python
        import time
        try:
            time.sleep(10)
            num = int(input('Enter a integer: '))
            print('{} is {}'.format(num, 'Odd' if num%2 else 'Even'))
        except ValueError:
            print('Enter a integer!!!')
        except(EOFError, KeyboardInterrupt):
            print('User interrupts the program: ')
        except: # catch all except
            print('Some thing wrong!!')
        ```
## Some tech
* Get modules function list (not string):
    ```python
    import inspect
    modulus_list = [func[0] for func in inspect.getmembers(models, predicate=inspect.isroutine) if callable(getattr(models, func[0]))]
    ```



# Python OOP
* While Python isn’t purely an object-oriented language, it’s flexible enough and powerful enough to allow you to build your applications using the object-oriented paradigm. 

## Python Super() Function
* Reference:
    * [Main Referance: Supercharge Your Classes With Python super()](https://realpython.com/python-super/)
    * [python document](https://docs.python.org/3/library/functions.html#super)
* One of the ways in which Python achieves this is by supporting inheritance, which it does with ```super()```.
* what is ```super()```:
    * ```super()``` alone returns a temporary object of the superclass that then allows you to call that superclass’s methods.
    * a common use case is building classes that extend the functionality of previously built classes
    * Example:
    ```python
    class Rectangle:
        def __init__(self, length=0, width=0):
            self.length = length
            self.width = width

        def area(self):
            return self.length * self.width

    class Square(Rectangle):
        def __init__(self, length):
            return super().__init__(length, length)


    class Cube(Square):
        def surface_area(self):
            one_side_area = super().area()
            return one_side_area * 6

        def volume(self):
            one_side_area = super().area()
            return one_side_area * self.length

    if __name__ == '__main__':
        c = Cube(10)
        print(c.surface_area()) # 600
        print(c.volume()) # 1000
    ```
 * ```super()``` Deep Dive:
     * ```super(subclass, )``` parameter
         1.  the subclass
         2.  an object that is an instance of that subclass
     * in python3: ```super(current_subclass, self)``` = ```super()```
     * You can call ```super()``` with other classes as well:
     ```python
     class Cube(Square):
        def surface_area(self):
            one_side_area = super(Square, self).area()
            return one_side_area * 6

        def volume(self):
            one_side_area = super(Square, self).area()
            return one_side_area * self.length
     ```
    * what second parameter do?
        * Remember, this is an object that is an instance of the class used as the first parameter.
        * By including an instantiated object, super() returns a **bound method**： a method that is bound to the object
        * It gives the method the object’s context such as any instance attributes
        * If this parameter is not included, the method returned is just a function, unassociated with an object’s context.




# Internet by Python
* [Advanced usage of Python requests - timeouts, retries, hooks](https://hodovi.ch/blog/advanced-usage-python-requests-timeouts-retries-hooks/)


