### 生成式人工智慧輔助之Python程式設計專題 物理系 C24106082 陳宏彰 https://hackmd.io/@simba-fs/2023-GPT-python ---- # 選課系統剩餘人數監控 --- # 實作細節 ---- ## 新增監控請求 ```flow st=>start: /listen <url> [selector] hasSelector=>condition: 有選擇器？ defaultSelector=>operation: 填入預設選擇器 #A9-table > tbody:nth-child(2) > tr:nth-child(1) > td:nth-child(8) save=>operation: 儲存起來 notify=>end: 傳送訊息通知使用者 st->hasSelector hasSelector(no)->defaultSelector->save hasSelector(yes)->save->notify ``` ---- ## 儲存的資料 ```python { 64926592: { # user telegram chat id 'https://course.ncku.edu.tw/index.php?c=.....': [ # url to motitor '#A9-table > tbody:nth-child(2)...', # selector 0 # previous value ], }, 7593595739:{ 'https://course.ncku.edu.tw/index.php?c=.....': [ '#A9-table > tbody:nth-child(2)...', 0 ] } } ``` ---- ## 爬取數值 ```flow getValue=>start: /listen <url> [selection] isExist=>condition: 有爬到？ log=>operation: 印除錯訊息 doesChange=>condition: 有改變？ save=>operation: 儲存起來 notify=>end: 通知使用者 getValue->isExist isExist(no)->log isExist(yes)->doesChange doesChange(no)->log->notify doesChange(yes)->save->notify ``` ---- # Multithread ```python= t1 = Thread(target=crawler) t2 = Thread(target=bot.infinity_polling) t1.start() t2.start() ``` --- ## 其他不算核心的操作 ---- ### 移除監控請求 ```flow start=>start: /remove <url> doUserExist=>condition: 有使用者？ notify=>operation: 通知使用者 doRecordExist=>condition: 有紀錄？ remove=>operation: 移除監控請求 end=>end: end start->doUserExist doUserExist(no)->notify->end doUserExist(yes)->doRecordExist doRecordExist(no)->notify doRecordExist(yes)->remove->end ``` ---- ### 列出監控請求 ```flow start=>start: /list findData=>operation: 根據使用者 id 找資料 sendToUser=>end: 傳送給使用者 start->findData->sendToUser ``` --- # 爬取資料 ---- ```python request.get('https://course.ncku.edu.tw/...') ``` ---- # but  ---- Selenium、playwright? ---- headers ----  ---- ```python headers = { Host: 'course.ncku.edu.tw', 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/116.0', Accept: '*/*', 'Accept-Language': 'en-US,en;q=0.8,zh-TW;q=0.5,zh;q=0.3', 'Accept-Encoding': 'gzip, deflate, br', DNT: '1', Connection: 'keep-alive', Referer: 'https://course.ncku.edu.tw/index.php?c=qry11215&m=en_query&i=U2RUawA2BjULclF3VzsBP1ZiU3RebAd2CDsHZwU9CzBVblYtVjxWPVM7UzsDPA02WmVVbVJmB2sFfF99Bj8GNgpmUHcIegFjBCQLZQInAjcCPVNsUiFXNV91BjwHeFwrAHJRawAyC2AGdQYlVHANNg0yUGgANQJuUSgHK1MjAztTNFRrACcGaAsrUT9XYwE0VmlTbV5y', Cookie: '_ga=GA1.3.381219810.1673964398; _ga_V5263G30EE=GS1.1.1690885343.325.0.1690885343.0.0.0; _ga_9VR7HBB4SG=GS1.1.1687957937.7.0.1687957937.60.0.0; AMCV_4D6368F454EC41940A4C98A6%40AdobeOrg=-2121179033%7CMCIDTS%7C19523%7CMCMID%7C12671621494609851332063067034236606133%7CMCAID%7CNONE%7CMCOPTOUT-1686749625s%7CNONE%7CMCAAMLH-1687347225%7C11%7CMCAAMB-1687347225%7Cj8Odv6LonN4r3an7LhD3WZrU1bUpAkFkkiY1ncBR96t2PTI%7CMCSYNCSOP%7C411-19530%7CvVersion%7C5.3.0; s_pers=%20v8%3D1686742425397%7C1781350425397%3B%20v8_s%3DMore%2520than%252030%2520days%7C1686744225397%3B%20c19%3Dpr%253Apure%2520portal%253Apersons%253Aview%7C1686744225399%3B%20v68%3D1686742421172%7C1686744225400%3B; _ga_8RY35M6ZMQ=GS1.1.1690853311.10.0.1690853311.0.0.0; _ga_WHBRRJE1JV=GS1.1.1690298802.14.1.1690298834.0.0.0; _ga_YTL27NBKJ1=GS1.1.1686673400.7.1.1686673853.0.0.0; _ga_TKEHRTBPD6=GS1.1.1690760071.15.1.1690760097.0.0.0; _ga_BSPNZQ67NJ=GS1.1.1690760097.14.1.1690760238.0.0.0; _ga_8LGPXZ8D11=GS1.1.1690885960.5.0.1690885963.0.0.0; __dtsu=6D001683526930C7219E3255D732F50D; _ga_7TZLNDHBYS=GS1.1.1683905548.1.0.1683905556.0.0.0; _ga_H1MCL60QVP=GS1.1.1684898230.1.0.1684898236.0.0.0; _ga_FHTS9H202Z=GS1.1.1684918248.2.1.1684921479.0.0.0; _ga_H6F5F49WXK=GS1.1.1687957937.3.0.1687957937.60.0.0; cos_lang=cht; COURSE_WEB=ffffffff8f7cbb1d45525d5f4f58455e445a4a423660; COURSE_CDN=ffffffff8f7ce72345525d5f4f58455e445a4a42cbd9; _gid=GA1.3.1739964840.1690853312; SSO=adc313fd46be8056305dc9a4aa926d71; PHPSESSID=C24106082ff661c220ea6059c78ece6371aa19c20; AMCVS_4D6368F454EC41940A4C98A6%40AdobeOrg=1; s_sess=%20s_cpc%3D0%3B%20s_sq%3D%3B%20s_ppvl%3Dpr%25253Apure%252520portal%25253Apersons%25253Asimilar%252C64%252C64%252C889%252C1920%252C889%252C1920%252C1080%252C1%252CP%3B%20s_ppv%3Dpr%25253Apure%252520portal%25253Apersons%25253Aview%252C77%252C40%252C3913%252C1920%252C889%252C1920%252C1080%252C1%252CP%3B%20e41%3D1%3B%20s_cc%3Dtrue%3B', 'Sec-Fetch-Dest': 'script', 'Sec-Fetch-Mode': 'no-cors', 'Sec-Fetch-Site': 'same-origin' } ``` ---- ```python requests.get( 'https://course.ncku.edu.tw/...', headers=headers, ) ``` --- # demo ---- {%youtube Se_M5uCjh80%} --- # Thanks!