# shopee quick note ## feature engineering * goal: good features to describe env * assumption: sys can be describe by features * assumption: can infer a good action only from current sys (Makovianly) * the dynamics is unknown and non-deterministic * what do others use? * what have we used? * what do we have? * Do we need to consider the Ratings fro, customers? * Do we need to consider inventory? * Do we need to consider shipping fee? ## misc * check whether our DRCR also has the same normalization effect ## learning notes * on-policy/off-policy * > On-policy methods attempt to evaluate or improve the policy that is used to make decisions, whereas off-policy methods evaluate or improve a policy different from that used to generate the data. [ref][R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction. IEEE Transactions on Neural Networks, 9(5):1054–1054, 2018.] * (my words) If we are using delayed updating, replay memory, or explorer-exploiter, then we are using off-policy ## log ### 1st week * Jul 6 * on-board training * Jul 7 * read *"DYNAMIC PRICING ON E-COMMERCE PLATFORM WITH DEEP REINFORCEMENT LEARNING"* * meeting, to know the pricing project * read *"Reinforcement Learning Applications in Dynamic Pricing of Retail Markets"* * simulation the behavior of the Market by shopper-captive model * read *"An Empirical Analysis of Algorithmic Pricing on Amazon Marketplace"* * only read the 1st page, it focuses on the interaction between pricing algorithms * Jul 8 * reading the description of shopee features, trying to do sth for feature engineering * read *"DYNAMIC PRICING ON E-COMMERCE PLATFORM WITH DEEP REINFORCEMENT LEARNING"* * read about pre-training part * *"Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards"* * only the abstract, it addresses those problem without good reward reward function; if uses demonstration as main input for models to learn * read documentation of DDPG https://spinningup.openai.com/en/latest/algorithms/ddpg.html * maybe we can use Gaussian noise for * Jul 9 * setup git on windows etc * learn python's reduce * read the DDPG document https://spinningup.openai.com/en/latest/algorithms/ddpg.html * read dynamic pricing stuff by Peggy on Notion https://docs.google.com/document/d/12RazIqcZGPEVDRke7bK5vuCmKyRnMUrZ2UApMU0w-9Q/edit# * finish the assignment "2020 Shopee Taiwan Summer Internship Goal Setting" * Jul 10 * finish "Leveraging......" * go through Polobao's code ### 2nd week * Jul 13 * 弄直播 * read papers * 上ML課 * Jul 14 * 開pricing會議 * Jul 15 * 創意思考課程 * 設定GCP * plot distribution of features for bluetooth earphone in Feb 2020 * Jul 16 * 弄直播 * 驗證內插可行性, test on mean, mode, and median; we choose mean * Jul 17 * 弄直播 * 學shopee API * 寫內插程式 env.py and utils.py * * BI/DS brainstorming, learning ARIMA from Andy and Polobao ### 3rd week * Jul 20 * 中午直播彩排 * ML training course: DNN, gradient descenting, momentum * Jul 21 * discuss how to design performance metrics for our model w/ Jimmy and Polobao * discuss the use of git on our project w/ Polobao * 中午做直播道具 * 下午開dynamic pricing的會, * a useful idea: one data point per order, traffic record among past several hours * for validation: completely random vs ruled-based experts vs our model * merge to master and use git flow, learn from Sunny * LS_COLORS setting, GOOD * try to ssh between my laptop and shopee's laptop, FAILED * Tarjan's algorithm, NOT FINISHED * rewrite my weekly report * ask how to claim reimbursement at shopee, on process * Jul 22 * fix division-by-zero bug * 中午正式第一次直播 * 想出機率分部內插法 * how to generate a pdf from sampled data points * Jul 23 * 實作機率分部內插法 * 中午討論直播腳本 * examine the minimum number of data points needed to infer the Gaussian probability distribution by which they are sampled * mean: ![](https://i.imgur.com/nYb7gR3.png) * var: ![](https://i.imgur.com/xyuGHkY.png) * 3d: ![](https://i.imgur.com/lMcju0F.png) * learn to plot 3d figure in python * think about a Poisson model to describe shopping behavior * API stuff: http GET in google app script for sheets * google app script: to divide numbers concatenated by a space, in a cell * Jul 24 * read "inferring a gaussian distribution thomas p. minka 1998 (revised 2001)" * 中午討論直播腳本與彩排 * 傍晚直播對焦會議(required by HR instructors and staff from livecasting team) * plot the pdf for our data ### 4th week * Jul 27 * 早上 看一下直播的文件大家有沒有更新東西 並打一點註解 然後開始寫code, for probability interpolation. Try to use more pandas * 中午彩排直播與討論 1200~1400 * 下午續繼寫code * 1600~1700跟polobao討論架構和git flow，畫程式的架構圖，處理我們碰到的overfitting的問題 * 晚上上課: CNN 與 Titanic 資料集，講師的作品分享 * 做 API in G sheets 感謝 Sonny 指導 * `item_list[0].value` * `if (item_list.length === 0)`` * 晚上做機率內插 * Jul 28 * 早上買道具做狗牌寫程式 * 中午正式第二次直播 * 下午寫完架構 * 晚上除蟲 * Jul 29 * 早上除蟲 * 中午觀摩直播 * 下午除蟲做簡報 * Jul 30 * 上午 merge "Develop prob distri interpolation" into "Develop" * 下午參訪楊梅倉庫 * Jul 31 * 上午期中簡報 * 下午去Garena ### 5th week * Aug 3 * 假如機率內插得到的值小於等於零就重新取樣 * 找到讓函數漸漸逼近零的好關鍵字: smoothstep function * 編輯repo讀我檔 * 改寫main.py成一個大for-loop以便我們之後實驗各種超參數 * 新增gaussian noise，跟OU noise並存，並且做成選項；gaussian noise的參數做成可調的超參數 * 新增一個功能，可以依照欄位名稱決定要用哪一種機率分布去取樣 * polobao發現我之前寫的悲劇性錯誤 * Aug 4 * 寫個功能(gaussian noise on action price) 2 hr * 下午開會 1hr * 教人用shopee-api-g-sheet 0.5hr * 編輯shopee-api-g-sheet readme 1hr * 中午直播彩排 1 hr * 傍晚實際直播 1 hr * Aug 5 * fix the bug of extremely large gaussian noise * plot pdf * ![](https://i.imgur.com/vNF9LW3.png) * https://gitlab.com/y56/intern2020/-/commit/49d5440d72ff2f325d0db58c3c627895df39080f * Aug 6 * solve issue: column-wise pdf * create issue: use sql in python to get data * Aug 7 * 中午直播彩排 * get data by sql * adjust utc+0 to utc+8 * get l3d, l7d, and l30d * solve a presicion issue * 全體實習生的直播會議 ## 6th week * Aug 10 * Confirmed the correctness of l3d, l7d, and l30d. * sum(b.pv_ch)/30.0 AS l30d_pv has precision of 0.1, while sum(b.pv_ch)/cast(30.0 as real) AS l30d_pv has full precision. * Replace the traffic_table_LnD_hourly as `f` part in `fetch_data_hourly.py` with updated version, saving as `fetch_data_hourly_new_LnD.py`. * Put the long long sql string as another file. Use "xxx{a}xxx{b}xxx".format(a=a,b=b). Note that start_date = datetime.datetime(2020, month, 1).date() and then f'start_date ' will give '2020-01-01'. * Have to fix empty dataset_2020-06-01_2020-06-30_hourly_all.csv. * Aug 11 * [SQL] Succeeded generating a table w/ zero-valued rows for every hour in a specified range. * [SQL] Found a bug of `date_add()`: It gives a wrong timestamp when adding time to `1981-12-31 23:59:59.999` * https://gist.github.com/y56/f15ea575d53bf13c9c929ab2c2c33a13 * Aug 12 * [SQL] Worked on getting the data. Should have been the desired data. * [SQL] Presto can't declare any timestamps between `1982-01-01 00:00:00.000` and `1982-01-01 00:29:59.999` (inclusively). * https://gist.github.com/y56/848005fb02fabe1c3934a7b2d160876c * Aug 13 * 中午直播當主持人 * Examine the correctness of traffic data * https://shopee.tw/product-i.37137599.905684794 * ![](https://i.imgur.com/1Zbc9nF.jpg) * Aug 14 * 中午直播彩排當場控 * Discuss the skeleton of the pricing daemon w/ Kevin * ![](https://i.imgur.com/hDjTATf.jpg) * Open more issues * Pickle actor networks for every sku * Output a suggested price from a pickled network and market features * Get L1D data * Decide pdf for each market feature ## 7th week * Aug 17 * Plot the 100 skus * The feature-vs-price plot show the benefit of hourly resolution profile: Large number of data points erase the need to abandon price with few data points. * Might need to interpolate in the time series * Meet to discuss the skeleton and daemon ## TODO: * Build the pricing daemon * Plot the behavior of grouped skus * Develop grouping methodology ## notes for midterm presentation * cur condition=? * purpose=? * who have succeeded? * what have we done? and how good? :::danger LSTM RNN GRU ::: I found an interesting feature in our Presto recently and would like to know some more about it. In Presto, ```values from_unixtime(0);``` will give ```1970-01-01 07:30:00.000``` instead the standard ```1970-01-01 00:00:00.000```. The +7.5 hr shift continues until ```values from_unixtime(378664199);``` which gives ```"1981-12-31 23:59:59.000"```. Interestingly, ```values from_unixtime(378664200);``` will give ```"1982-01-01 00:30:00.000"```, which means a +8 hr shift. When doing ```date_add('second', 1, TIMESTAMP '1981-12-31 23:59:59.000')```, we will get ```1982-01-01 00:30:00.000``` instead of ```1982-01-01 00:00:00.000```. This means we get wrong values when doing time addition.