PyCon TW 2017 Collaborative Talk Notes Day 1 - R1

# PyCon TW 2017 Collaborative Talk Notes Day 1 - R1 > ### Quick Links > - [Portal for Collobration Notes 共筆統整入口](http://beta.hackfoldr.org/pycontw2017/) (hosted by [betw.hackfoldr](http://beta.hackfoldr.org/) and [HackMD](https://hackmd.io/)) > - [Program Schedule 演講列表](https://tw.pycon.org/2017/events/talks/) > - [PyCon TW 2017 Official Site 官網](https://tw.pycon.org/2017/) > > ### How to update this note? > - Everyone can *freely* update this note. 任何人都能自由地更新內容。 > - Please respect all the participants and follow our [code of conduct](https://tw.pycon.org/2017/about/code-of-conduct/) during discussion. 討論、記錄時，請遵守大會的[行為準則](https://tw.pycon.org/2017/about/code-of-conduct/) > ------- ### 10:40-11:10 Talk: [PyOT - The Merge Between Python and the Internet of Things Concept](https://tw.pycon.org/2017/events/talk/337602843466793058/) - Slider: - Speaker: Renaldi Gondosubroto ------- ### 11:45-12:30 Talk: [after VOEZ launch - how to resolve problems of mobile game server development and service maintenance](https://tw.pycon.org/2017/en-us/events/talk/319314010958725199/) - Slider: https://hackmd.io/p/SkuoAv2lZ#/ - Speaker: 郭學聰 - 共筆： #### 作者所用的語言 * Python * Ruby * Golang * C# #### VOEZ Current Status - Profile, save data, achievement, leaderboard - Based on GCP - 曾經想過用AWS - [Teaser](https://www.youtube.com/watch?v=Bh6gQyJHbxI) #### HTTP protocol Request ![](https://i.imgur.com/SDJjHLD.png) Response ![](https://i.imgur.com/65Kr0uz.png) #### Genuine & Purchase verification ##### leaderboard -> fake score * verify request: send request & attach signature ![](https://i.imgur.com/CQpYwXR.png) * RSA ##### game play -> pirated app & server * server授權：RSA request hash 增加 nonce (random number) #### Stability * transaction ##### publish order and request order (Racer condition) * request order * client read game info A, then B, then C * publish order * publish game info C, then B, then A * remove game info A, then B, then C ##### database cache mechanism * database in HDD and SSD: slow * Redis and mem in memory: fast, as cache ##### reliable inter-server request * 由於server之間的溝通導致玩家扣錢失敗 * A: Game server B: 金流伺服器 * 扣款失敗：rollback * 嘗試三次，失敗後要求client重新連線 * 可以讓玩家排隊沒關係！資料庫不能掛！ #### Performance * Python is so SLOW * "skip Python code execution if you can" - HT Kuo 06, 09, 2017 * 支援百萬人上線需要的反應時間: nanosec * 用Google BigQuery取得公開資料,並把結果存成靜態檔案放流,迴避Python執行 * 使用CDN * 全球的玩家連線到各地的CDN * 節省流量及運算資源 * asset files * 每次都用不同URL * 可以準確在換日的時候讓 cache expired * service downtime: 503 * service update: 舊的版本丟410 #### Timezone * 為了準確地在本地換日時間(ex:3 a.m.)更新活動 * TZ = "Asia/Taipei" * 用UTC去計算會碰到日光節約時間的問題 >Q: Why not use AWS? AWS跟中華電信間連線超慢 >Q: 應對百萬使用者的方式? 設定動態開設資源 >Q: Header content type為空? 手動刪除的XD >Q: 阿里雲和GCP共通方式？可以做data sync但中國比較偏向自己有伺服器但是Code共通 >Q:歌曲放上CDN是否會被竊取? 歌曲資料有經過加密 ------- ### 14:55-15:40 Talk: [Building Microservices in Python 個案分享](https://tw.pycon.org/2017/events/talk/348376501365768328/) - Slider: https://www.slideshare.net/jonascheng3/building-microservices-in-python-pycon2017 - Speaker: Jonas Cheng 自我介紹：從趨勢獨立成立的子公司 soocii 平台提供給手遊玩家的價值 - 直播群聊 - 社群分享 - 即時資訊(攻略) 費時<1年 backend工程師*5 推薦大師：Martin Fowler、Sam Newmer ## 使用 microservice 的好處： - 解決傳統monolithic架構的問題傳統架構：（負載平衡）＋（各種認證，註冊，通知等服務...）＋（ＤＢ）改完後=>user用的版本跟後端不相容=>破壞ＵＸ - 革新的設計 - 富有彈性(能夠應對PM的變更需求) - Small Codebase - 相對容易測試與維護 - Easy to Scale - plugin可以隨時抽出或加入，容易擴充 - Easy to Deploy - 部分佈署 - System Resilience - 部分元件異常還是可以正常運作 #### 曾經面對的挑戰系統配置混合雲：AWS + GCP (因為AWS在tokyo，速度不夠快) 使用技術：Python + C micro service 問題：開發環境的準備 - 外部相依性 & 內部相依性 - 全部在本地 - 只有microservice跟business logic在本地 - 只有microservice在本地 - 都在雲端開發 - soocii 採用混合方案 - 本地環境：docker-compose, mock data, 假認證 - 整合環境：docker-compose 參考：Development environments for microservices (https://www.datawire.io/guide/deployment/development-environments-microservices/) - 跨服務的溝通 - 程式溝通方式 - RESTful - 希望達到零溝通，但基本上沒看過 - micro service 要能時刻提供服務，不會影響主要功能 - 理想上：不應該存在部分元件異常導致全部損毀（如：Redis 異常，應該要可以再存取 RDB） - 實際上：很難，像是 Auth 是全部共用模組，難以取代，不過另外一個想法是也不容易壞掉 - 自動化測試（ＣＩ） - 理想：修補後可以馬上上線 - 測試種類：（測越多信心越高） - unit test: functions logics（最快） - service test: classes + mock class - requests.mock, mock.patch - UI test: 整合性測試（最慢） - consumer-driven contract (CDC) - 非同步測試： - celery: task_always_eager - event-driven (good to have) - sleep (bad) - 自動化部署（ＣＤ） - 盡可能延後 - 多版本ＡＰＩ同時並存（透過blueprint找到對應版本） - #### 導入 microservice 的關鍵 - 所有成員要有DevOps mindset - 至少要有CI（自動）/CD（到處都可部署）的雛形 - 每個service有獨立的CI - 監控機制 Q. Instance實際溝通的行為，開發中如何處理？ AWS: (application load balancer) 每個docker註冊在ALB後面 => 確保服務活著，但不清楚是跟哪個docker溝通 Q. 跨伺服器溝通如果逼不得已要交換資料，怎麼處理？只可能盡量避免。 Q. 為什麼要做混合雲？發生什麼事情？直播的frame rate掉太多，內部做profiling本地端最快，最後才採用本地端。 ------- ### 16:10-16:55 Talk: [Understanding Serverless Architecture](https://tw.pycon.org/2017/events/talk/332303505006002268/) - Slide: https://speakerdeck.com/dawny33/understanding-serverless-architectures - Speaker: Jalem Raj Rohit - what is serverless? function as a service, no servers AWS: Lambda > functions > ... - function container - function executed in container - We are not running or maintaining servers - advantages - less time maintaining server - lots of server cost saved - e.g. deep learning - dis(ad-)vantages - allow to run in limit time (AWS: timout setting) - no control over containers - monitoring is very very very difficult - especially, scale out as distributed - lessons learned and pitfalls - limit running time problem - Ansiable to rescue - help provisioning workers - running Ansible in `nohup` mode - no control over container problem - for security - monitoring distributed system is difficult - logging (?) - How to scale? ------- ### 17:20-17:50 Talk: [Python module in Rust](https://tw.pycon.org/2017/events/talk/325227434406314071/) - Slider: [Python module in Rust](https://docs.google.com/presentation/d/1mTw-4buKDTqPNzJS03s2I0apBMal-SaeKk1dHDSE6fk/pub?start=false&loop=false&delayms=3000&slide=id.g22c75fc6c3_0_6) - Speaker: 許邱翔 (Chiu-Hsiang Hsu) (dv) - Rust community love Python most - Rust cares memory safety without GC ; Rust is high performance like C [benchmark](http://benchmarksgame.alioth.debian.org/) - [PyPI of Rust](https://doc.crates.io/) - pyvenv + pip -> Cargo - PEP of Rust -> RFC Q: why the Rust source code need something like "Extend C" A: that is for avoiding default behavior "name mangling" like C++ Q: Rust can share memory buffer with Python? A: yes