---
tags: language
---
Python Basic
===
* News & Trend:
* [Awesome Python Newsletter](https://python.libhunt.com/newsletter)
* [What do companies expect from Python devs in 2019?](https://hackernoon.com/what-do-companies-expect-from-python-devs-in-2019-f8b585a1ddf)
* Resource：
* [Big Book of Python](https://www.bigbookofpython.com/)
* [Practical Python Projects](https://practicalpython.yasoob.me/)
* [The Hitchhiker’s Guide to Python!](https://docs.python-guide.org/)
* [Legally Free Python Book List](https://www.pythonkitchen.com/legally-free-python-books-list/)
* [Learn Programming For Free (All Free Resources)](https://dev.to/nerdjfpb/learn-programming-for-free-all-free-resources-2b3j)
* [Hypermodern Python](https://cjolowicz.github.io/posts/hypermodern-python-01-setup/)
* Great Tool:
* [explainshell.com: command line explain tool](https://explainshell.com/)
* [No Really, Python's Pathlib is Great](https://rednafi.github.io/digressions/python/2020/04/13/python-pathlib.html)
* [A new kind of Progress Bar, with real time throughput, eta and very cool animations!](https://github.com/rsalmei/alive-progress)

# Best Practices
* [Clean Code in Python](https://testdriven.io/blog/clean-code-python/)
* [The Best of the Best Practices (BOBP) Guide for Python](https://gist.github.com/sloria/7001839)
* [Logging in Python](https://realpython.com/python-logging/)
* [Pysa: An open source static analysis tool to detect and prevent security issues in Python code](https://engineering.fb.com/security/pysa/)
* [Complete Guide How To Code Review](https://dev.to/nickbulljs/complete-guide-how-to-code-review-2837)

## Code Style and Structure
* Python style guide
* [PEP8：Style Guide for Python Code](https://www.python.org/dev/peps/pep-0008/)
* [The Elements of Python Style](https://github.com/amontalenti/elements-of-python-style)
* [How to Write Beautiful Python Code With PEP 8](https://realpython.com/python-pep8/)
* Tools to check code with PEP8
* [pylint](https://pypi.org/project/pylint/)
* [pycodestyle](https://pypi.org/project/pycodestyle/)
* [PEP257：Docstring Conventions](https://www.python.org/dev/peps/pep-0257/)
* [A Pythonic Guide to SOLID Design Principles](https://realpython.com/simpy-simulating-with-python/)
* [Clean Architectures in Python](https://leanpub.com/clean-architectures-in-python)

## Python Modules
* [Python 的 Import 陷阱](https://medium.com/pyladies-taiwan/python-%E7%9A%84-import-%E9%99%B7%E9%98%B1-3538e74f57e3)
* [Python Modules: Creating, Importing, and Sharing](https://stackabuse.com/python-modules-creating-importing-and-sharing/)
* What is Python modules：
* highest level organizational unit in Python
* the units that store code and data, provide code-reuse to Python projects
* useful in partitioning the system's namespaces in self-contained packages
* In fact, any Python file with a .py extension represents a module.
* Basic Concept：
* **Script**：
* In computing, the word script is used to refer to a file containing a logical sequence of orders or a batch processing file.
* A plain text file containing Python code that is intended to be directly executed by the user is usually called scrip
* script in python is an informal term that means **top-level program file** (__name__ == '__main__')
* 'top-level' mean any function or variable which is not nested inside another function or class.
* a plain text file, which contains Python code that is designed to be imported and used from another Python file, is called module.
* Python files which are used to run as a stand-alone Python app (top-level files) are usually designed to run as script
* import them would actually run the commands in the script
* Modules which are designed to be imported by other code won't execute any code, but only expose its top-level names as attributes to the imported object.
* Python modules which could be used for both - importing and running as a top-level script.
* Program Architecture：
* In general, we can distinguish three types of files in a multi-file Python application:
* top-level file：
* user-defined modules：
* standard library modules：

## Python Packaging
* [An Overview of Packaging for Python](https://packaging.python.org/overview/)
* [Python dependency management and packaging made easy.](https://github.com/python-poetry/poetry)

## Python Deployment
* [Cookiecutter PyPackage Tool](https://github.com/lgiordani/cookiecutter-pypackage)
* deployment questions below:
* Who are your software’s users? Will your software be installed by other developers doing software development, operations people in a datacenter, or a less software-savvy group?
* Is your software intended to run on servers, desktops, mobile clients (phones, tablets, etc.), or embedded in dedicated devices?
* Is your software installed individually, or in large deployment batches?
* PyPI,, ```setup```, ```wheel``` tools：
* Python’s ecosystem provides for distributing Python code to **developers**
* Wheel - Built-package format:
* [Python Wheels](https://pythonwheels.com/)
* [PEP 427 -- The Wheel Binary Package Format 1.0](https://www.python.org/dev/peps/pep-0427/)
* pure python code package can use sdist for deployment
* package include other language code, using **wheel**
* wheel is a package format designed to ship libraries with compiled artifacts
* ```pip``` always prefers wheels because installation is always faster, so even pure-Python packages work better with wheels

### Project Stucture
* [Structuring Your Project](https://docs.python-guide.org/writing/structure/)
In practical terms, **structure** means making clean code whose logic and dependencies are clear as well as how the files and folders are organized in the filesystem.

#### Structure of the Repository:
simple repository example:

README.rst
LICENSE
setup.py
requirements.txt
simple/
 -sample/__init__.py  #sample is your module name
 -sample/core.py
 -sample/helpers.py
docs
 -docs/conf.py
 -docs/index.rst
tests
 -tests/test_basic.py
 -tests/test_advanced.py

Let's get into some specifics.
* The Actual Module:
* Location: ```./module_name``` or ```module_name.py```
* Purpose: clear show the module core
* License:
* Location:```./LICENSE```
* Purpose: lawyering up
* check out [choosealicense.com](https://choosealicense.com/) and pick the full license text and copyright claims
* Setup file:
* Location: ```./setup.py```
* Purpose: Package and distribution management
* Requirement File:
* Location: ```./requirement.txt```
* Purpose: Development dependencies
* check [pip requirements file](https://pip.pypa.io/en/stable/user_guide/#requirements-files)
* Test Suite:
* Location: ```./tests.py```or```test_module_name.py```
* Purpose: Package integration and unit tests
* How to import module in test file?
* Expect the package to be installed in site-packages
* (recommend) Use a simple (but explicit) path modification to resolve the package property
* Reference: [Testing Your Code](https://docs.python-guide.org/writing/tests/)
* Makefile:
* Location:```./Makefile```
* Purpose: Generic management tasks
* **make** is an incredibly useful tool for defining generic tasks for your project (not just for C)

init:
    pip install -r requirements.txt
    
test:
    py.test tests
.PHONE: init test

* Doc:
* [Great tool for Documentation created- Sphinx ](http://www.sphinx-doc.org/en/master/)

# Testing
* Resource:
* [Testing Your Code](https://docs.python-guide.org/writing/tests/)
* [TDD (from IT幫30天)](https://ithelp.ithome.com.tw/users/20010292/ironman/462)
* [Write Unbreakable Python](https://dev.to/jesterxl/write-unbreakable-python-20ck)
* [Ultimate Guide to Python Debugging](https://towardsdatascience.com/ultimate-guide-to-python-debugging-854dea731e1b)
* [Testing Best Practices for Machine Learning Libraries](https://towardsdatascience.com/testing-best-practices-for-machine-learning-libraries-41b7d0362c95)
* Note:
* Focus on one tiny bit of functionality.
* Should be fast, but a slow test is better than no test.
* It often makes sense to have one testcase class for a single class or model

## TDD
* What is TDD?
* Test-Driven Development (TDD)
* TDD forces you to clearly state your goal before you write the code.
* TDD mantra is “Test first, code later”
* Important keys:
* Test first, code later
* Driving by the test, not by the requirements
* you should try to get to a situation in which only one test fails.
* test boundary cases
* finding the so-called "corner cases"
* showing the code covers them
* Write code that passes the test. Then refactor it.
* A test should fail the first time you run it. If it doesn’t, ask yourself why you are adding it.
* Never refactor without tests.
* Refactoring : changing the code without altering the results
* How can you be sure you are not altering the behaviour of your code? Well, this is what the tests are for.

## Python test tools
### IDE or Editor:
* Break Point: IDE 通常可以設置程式執行時預計要停止的多個中斷點，一段一段執行時一邊檢視變數狀況，不用一直去 print() 。
...

### assert(斷言):
* 目的：用一些表達式測試程式中非預期的偶發狀況，例如某些變數不可以為負的測試...等。
* 用途：安插在程式中，判斷某些 expression 是否為假，如果為假可以回傳訊息並中斷程式。
* 語法：```assert expression, 'how some message'```

>>> product_amount = -1
# 寫下預期都要大於0的 expression, 並在沒通過時秀出訊息
>>> assert product_amount>0, 'product_amount is negative!'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError: product_amount is negative!

* 優點：用 if statement 也可以輕易完成的事情，改用 assert 的好處是，我們可以在特定執行下不進入 assert 的判斷，也就是說測試完後就讓 assert 功臣身退，但不用去修改或刪除相關程式碼：
* 執行 python 時給定參數 -O: ```python -O```

$ python -O
Python 3.7.1 (default, Dec 14 2018, 19:28:38) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> product_amount = -1
>>> assert product_amount>0, 'product_amount is negative!'
>>> 

* 進一步說明：
* __debug__ 是 python 內建變數，一般情況是 True，如果執行時需要最佳化(在執行時加上-O引數)，其會是 False。
* 一般使用 ```assert expression``` 相當於以下程式片段：

    if __debug__:
        if not expression: raise AssertionError

### pytest
* [pytest:helps you write better programs](https://docs.pytest.org/en/latest/)

### unittest modules
* [unittest.mock](https://docs.python.org/3/library/unittest.mock.html)
* unittest modules 有時稱為 PyUnit, 是 JUnit 的 Pyhton 實現
* 單元測試：測試一個工作單元的行為，其單為基本上要**獨立**，否則會測試到相關交互合作行為
* 以軟體而言，單元測試通常指的是測試某個函式(或方法)
* Python 的 unittest 模組主要包括4個部份：
* 測試案例(Test case):測試最小單元
* 測試設備(Test fixture)
* 測試套件(Test suite)
* 測試執行器(Test runner)

* unittest.TestCase:

import unittest
  
class TestStringMethods(unittest.TestCase):
    def test_upper(self):
        expected = 'FOO'
        result = 'foo'.upper()
        self.assertEqual(result, expected)
    def test_isupper(self):
        self.assertTrue('FOO'.isupper())
        self.assertFalse('Fool'.isupper())
    def test_split(self):
        s = 'hello world'
        result = s.split()
        expected = ['hello', 'world']
        self.assertEqual(result, expected)

if __name__=='__main__':
    unittest.main()

Test result:

### Other
* [Python Mocking 101: Fake It Before You Make It](https://www.fugue.co/blog/2016-02-11-python-mocking-101)
* doctest modules
* pytest
* Hypothesis
* fixtures replacement tool
* [Prefer factories to fixtures.](https://github.com/FactoryBoy/factory_boy)

# Clean Architecture
* [Book: clean architectures in Python](https://leanpub.com/clean-architectures-in-python)
* Talk inwards with simple structures, talk outwards through interfaces.
* Main Layer：
* Entities：
* domain model
* Entities don’t know details about the external interfaces, and they only work with interfaces.
* Use cases：
* the processes that happen in your application
* It is very important to isolate small actions in use cases, as this makes the whole system easier to test, understand and maintain.
* Use cases know the entities, so they can instantiate them directly and use them.
* External systems
* This part of the architecture is made by external systems that implement the interfaces (such as API) defined in the previous layer.
* The word API is of uttermost importance in a clean architecture.

# Python Foundation
* Python3.5 技術手冊-林信良
## Common Sense
* **lib & lib/pythonx.x** : 放置標準函式庫(原始碼)
* lib/pythonx.x/site-packages: 第三方程式庫
* ```sys.path``` can show the directory path list of python modules.
* ```import this``` show the zen of Python
* Python interpernet default encoding is UTF-8(same in **bash on shell**, but MS not)
* In *.py file, first line can write command to change encoding method:

        # coding= <encoding name>
        # coding: <encoding name>

EX:

        # coding: Big5
        # coding= UTF-8

* Basic Modules:
* a *.py file is a modules, can use ```import``` cammand to include.
* ```dir(__builtins__)``` can show the build in modules.
* After executing *.py file and import some mosules, a "__pycache__" directory and some *.pyc files will be created. It's a binary file that be created by interpreter. If there is no any change of *.py, the *.pyc file will be used to accelerate the interpreter speed.
* ```print(sys.path)```: show whole module path.
* ```sys.path.append('/dirctory/path')```: 增加模組 import 路徑。
* ```$ python -c "coding"```: 可以執行一小段程式碼，方便測試。
* [What does if __name__ == “__main__”: do?](https://stackoverflow.com/questions/419163/what-does-if-name-main-do)

## Built-in Types
* The principal built-in types are
* numerics
* sequences
* mappings
* classes
* instances
* exceptions
* most of built-in objects considered false:
* constants defined to be false: None, False
* zeor of any numeric types: 0, 0.0, 0j, Decimal(0), Fraction(0, 1)
* empty sequences and collections: '', [], (), {}, set(), rnage(0).
* Boolean operations - and, or, not
* and, or, is a short-circuit operator.
* not has a lower priority than non-Boolean operators, so ```not a == b``` is interpreted as ```not (a == b)```

### Numeric Data Type: int, float, complex
* integer：int()
* Python3 後不分 **int** / **long**，長度不受限制（物理限制以外）
* 轉換進位基底的方式:

      int('010',  2) # 2
      int('010',  8) # 8
      int('010', 16) # 16
      
      0b010 # 2
      0o010 # 8
      0x010 # 16
      
      bin(10) #'0b1010'
      oct(10) #'0o12'
      hex(10) # 0xa

* 任何進位的整數都是 int 類別的實例:

* Booleans: a subtype of intefers
    * bool():可以傳入任何型態，將 None, False, 0, 0.0, 0j, '', (), [], {}, 等傳入都會返回 False, 其他非0直回傳 True 
* float point: float()
    * Floating point numbers are usually implemented using **double** in C
    * ```sys.float_info```：show floating point information 
    * ```1e-1 + 1e-2``` # 0.11
    * complex: ```z=a+bj```. complex numbers have a real and imaginary part, which are each a floating point number. To extract these parts from a complex number z, use ```z.real``` and ```z.imag```.
    * 3 * 0.1 == 0.3 # Got False
        * 解法1:```round(.1*3, 2)==round(.3, 2) # True```
        * 解法2：```from decimal import Decimal``` 此套件重現 10進位經準度
    * some method:
        * ```(0.25).as_integer_ratio() # (1, 4)```
        * ```(1.0).is_integer() # True```

### Bytes

import os
import io
from bitstring import BitArray

from PIL import Image

with io.BufferedReader(io.FileIO('IMAG0733.jpg', "r")) as f:
print(f)
img_bytes = f.read()
binary = BitArray(img_bytes).bin

with io.BufferedReader(io.FileIO('IMAG0733.jpg', "r")) as f:
print(Bits(f))
img_bytes = f.read()

* [Inserting pyodbc.Binary data (BLOB) into SQL Server image column](https://stackoverflow.com/questions/11836924/inserting-pyodbc-binary-data-blob-into-sql-server-image-column)
* [Python SQLite BLOB to Insert and Retrieve file and images](https://pynative.com/python-sqlite-blob-insert-and-retrieve-digital-data/)


### Sequence Types — list, tuple, range:
* String: Text Sequence Type

type('string') # <class 'str'>
type("string") # <class 'str'>
print("'string'") # 'string'
print('"string"') # "string"

* 字串轉義表示：
    * ```\\ => \```
    * ```\' => '```
    * ```\n, \t, \r```


## Function in Python
* What is First class objects： 
    * in a language are handled uniformly throughout. 
    * They may be stored in data structures, passed as arguments, or used in control structures. 
    * A programming language is said to support first-class functions if it treats functions as first-class objects.  
* Python supports the concept of **First Class functions**：
    * A function is an instance of the Object type.
    * You can store the function in a variable.
    * You can pass the function as a parameter to another function.
    * You can return the function from a function.
    * You can store them in data structures such as hash tables, lists, …


## Deal With Exception
* Introduction
    * 當某些原因使執行中的程式無法繼續，Python 中可以**引發例外(Raise Exception)** 
    * Python 可以在例外發生時加以處理，可以引發例外，自訂例外
* ```try``` & ```except```
    * example:

try:
numbers = input('Enter multi number with space between them: ').split(' ')
print('Mean:'sum(int(numbers) for number in numbers) / len(numbers))
except: ValueError as err:
print(err)

    * python 直譯器程式執行 try 區塊中的程式，如果發生例外會跳離發生點，去比對 **except** 宣告的型態，是否符合引發的**例外物件**型態，如果是就執行 except 區塊程式碼
    * 在 pyhton 中，例外並不一定是錯誤，例如，**for in** 語法中其底層就運用例外機制
        * 進一步說明，只要具有 ```__iter__()``` 方法的物件，都可以用 for in 來迭代
        * ```__iter()``` 方法應該傳回一個迭代器(iterator)
        * 該迭代器具有 ```next()``` 方法，每次迭代就會傳回下一個物件，沒下個元素時，則會引發 **StopIteration**
        ![](https://i.imgur.com/tSJ4UWm.png)
        * **for in** 會在遇到 StopIteration 時，默默結束迭代，流程比較類似：

def for_in(iterable, do_it):
iterator = iter(iterable)
try:
while True:
do_it(next(iterator))
except StopIteration:
pass

    * ```except``` 後可以用 tuple 指定多個例外物件，也可以多個 except，如果 ```except``` 後沒指定例外物件，那麼會捕捉所有引發的物件

import time
try:
time.sleep(10)
num = int(input('Enter a integer: '))
print('{} is {}'.format(num, 'Odd' if num%2 else 'Even'))
except ValueError:
print('Enter a integer!!!')
except(EOFError, KeyboardInterrupt):
print('User interrupts the program: ')
except: # catch all except
print('Some thing wrong!!')

## Some tech
* Get modules function list (not string):

import inspect
modulus_list = [func[0] for func in inspect.getmembers(models, predicate=inspect.isroutine) if callable(getattr(models, func[0]))]

# Python OOP
* While Python isn’t purely an object-oriented language, it’s flexible enough and powerful enough to allow you to build your applications using the object-oriented paradigm. 

## Python Super() Function
* Reference:
    * [Main Referance: Supercharge Your Classes With Python super()](https://realpython.com/python-super/)
    * [python document](https://docs.python.org/3/library/functions.html#super)
* One of the ways in which Python achieves this is by supporting inheritance, which it does with ```super()```.
* what is ```super()```:
    * ```super()``` alone returns a temporary object of the superclass that then allows you to call that superclass’s methods.
    * a common use case is building classes that extend the functionality of previously built classes
    * Example:

class Rectangle:
def __init__(self, length=0, width=0):
self.length = length
self.width = width

def area(self):
return self.length * self.width

class Square(Rectangle):
def __init__(self, length):
return super().__init__(length, length)

class Cube(Square):
def surface_area(self):
one_side_area = super().area()
return one_side_area * 6

def volume(self):
one_side_area = super().area()
return one_side_area * self.length

if __name__ == '__main__':
c = Cube(10)
print(c.surface_area()) # 600
print(c.volume()) # 1000

 * ```super()``` Deep Dive:
     * ```super(subclass, )``` parameter
         1.  the subclass
         2.  an object that is an instance of that subclass
     * in python3: ```super(current_subclass, self)``` = ```super()```
     * You can call ```super()``` with other classes as well:

class Cube(Square):
def surface_area(self):
one_side_area = super(Square, self).area()
return one_side_area * 6

def volume(self):
one_side_area = super(Square, self).area()
return one_side_area * self.length

    * what second parameter do?
        * Remember, this is an object that is an instance of the class used as the first parameter.
        * By including an instantiated object, super() returns a **bound method**： a method that is bound to the object
        * It gives the method the object’s context such as any instance attributes
        * If this parameter is not included, the method returned is just a function, unassociated with an object’s context.




# Internet by Python
* [Advanced usage of Python requests - timeouts, retries, hooks](https://hodovi.ch/blog/advanced-usage-python-requests-timeouts-retries-hooks/)