---
# System prepended metadata

title: shopee quick note

---

#  shopee quick note
## feature engineering
* goal: good features to describe env
    * assumption: sys can be describe by features
    * assumption: can infer a good action only from current sys (Makovianly)
    * the dynamics is unknown and non-deterministic 
* what do others use?
* what have we used?
* what do we have?
* Do we need to consider the Ratings fro, customers?
* Do we need to consider inventory?
* Do we need to consider shipping fee?

## misc
* check whether our DRCR also has the same normalization effect

## learning notes
* on-policy/off-policy
    * > On-policy methods attempt to evaluate or improve the policy that is used to make decisions, whereas off-policy methods evaluate or improve a policy different from that used to generate the data. [ref][R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction. IEEE Transactions on Neural Networks, 9(5):1054–1054, 2018.]
    * (my words) If we are using delayed updating, replay memory, or explorer-exploiter, then we are using off-policy

## log
### 1st week
* Jul 6
    * on-board training
* Jul 7
    * read *"DYNAMIC PRICING ON E-COMMERCE PLATFORM WITH DEEP REINFORCEMENT LEARNING"*
    * meeting, to know the pricing project
    * read *"Reinforcement Learning Applications in Dynamic Pricing of Retail Markets"*
        * simulation the behavior of the Market by shopper-captive model
    * read *"An Empirical Analysis of Algorithmic Pricing on Amazon Marketplace"*
        * only read the 1st page, it focuses on the interaction between pricing algorithms
* Jul 8
    * reading the description of shopee features, trying to do sth for feature engineering
    * read *"DYNAMIC PRICING ON E-COMMERCE PLATFORM WITH DEEP REINFORCEMENT LEARNING"*
        *  read about pre-training part
    * *"Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards"*
        * only the abstract, it addresses those problem without good reward reward function; if uses demonstration as main input for models to learn
    * read documentation of DDPG https://spinningup.openai.com/en/latest/algorithms/ddpg.html
        * maybe we can use Gaussian noise for 
* Jul 9 
    * setup git on windows etc
    * learn python's reduce
    * read the DDPG document https://spinningup.openai.com/en/latest/algorithms/ddpg.html
    * read dynamic pricing stuff by Peggy on Notion https://docs.google.com/document/d/12RazIqcZGPEVDRke7bK5vuCmKyRnMUrZ2UApMU0w-9Q/edit#
    * finish the assignment "2020 Shopee Taiwan Summer Internship Goal Setting" 
* Jul 10
    * finish "Leveraging......"
    * go through Polobao's code

### 2nd week
* Jul 13
    * 弄直播
    * read papers
    * 上ML課
* Jul 14
     * 開pricing會議
* Jul 15
    * 創意思考課程
    * 設定GCP
    * plot distribution of features for bluetooth earphone in Feb 2020
* Jul 16
    * 弄直播
    * 驗證內插可行性, test on mean, mode, and median; we choose mean
* Jul 17
    * 弄直播
    * 學shopee API
    * 寫內插程式 env.py and utils.py
    * * BI/DS brainstorming, learning ARIMA from Andy and Polobao
### 3rd week
* Jul 20
    * 中午直播彩排
    * ML training course: DNN, gradient descenting, momentum
* Jul 21
    * discuss how to design performance metrics for our model w/ Jimmy and Polobao
    * discuss the use of git on our project w/ Polobao
    * 中午做直播道具
    * 下午開dynamic pricing的會,
        * a useful idea: one data point per order, traffic record among past several hours
        * for validation: completely random vs ruled-based experts vs our model
    * merge to master and use git flow, learn from Sunny
    * LS_COLORS setting, GOOD
    * try to ssh between my laptop and shopee's laptop, FAILED
    * Tarjan's algorithm, NOT FINISHED 
    * rewrite my weekly report
    * ask how to claim reimbursement at shopee, on process
* Jul 22
    * fix division-by-zero bug
    * 中午正式第一次直播
    * 想出機率分部內插法
        * how to generate a pdf from sampled data points
* Jul 23
    * 實作機率分部內插法
    * 中午討論直播腳本
    * examine the minimum number of data points needed to infer the Gaussian probability distribution by which they are sampled
        * mean: ![](https://i.imgur.com/nYb7gR3.png)
        * var: ![](https://i.imgur.com/xyuGHkY.png)
        * 3d: ![](https://i.imgur.com/lMcju0F.png)
    * learn to plot 3d figure in python
    * think about a Poisson model to describe shopping behavior
    * API stuff: http GET in google app script for sheets
    * google app script: to divide numbers concatenated by a space, in a cell
* Jul 24
    * read "inferring a gaussian distribution thomas p. minka 1998 (revised 2001)"
    * 中午討論直播腳本與彩排
    * 傍晚直播對焦會議(required by HR instructors and staff from livecasting team)
    * plot the pdf for our data
### 4th week
* Jul 27
    * 早上 看一下直播的文件大家有沒有更新東西 並打一點註解 然後開始寫code, for probability interpolation. Try to use more pandas
    * 中午彩排直播與討論 1200~1400
    * 下午續繼寫code
    * 1600~1700跟polobao討論架構和git flow，畫程式的架構圖，處理我們碰到的overfitting的問題
    * 晚上上課: CNN 與 Titanic 資料集，講師的作品分享
    * 做 API in G sheets 感謝 Sonny 指導
        * `item_list[0].value`
        * `if (item_list.length === 0)``
    * 晚上做機率內插
* Jul 28
    * 早上買道具做狗牌寫程式
    * 中午正式第二次直播
    * 下午寫完架構
    * 晚上除蟲
* Jul 29
    * 早上除蟲
    * 中午觀摩直播
    * 下午除蟲做簡報
* Jul 30
    * 上午 merge "Develop prob distri interpolation" into "Develop"
    * 下午參訪楊梅倉庫
* Jul 31
    * 上午期中簡報
    * 下午去Garena
### 5th week
* Aug 3
    * 假如機率內插得到的值小於等於零就重新取樣
        * 找到讓函數漸漸逼近零的好關鍵字: smoothstep function 
    * 編輯repo讀我檔
    * 改寫main.py成一個大for-loop以便我們之後實驗各種超參數
    * 新增gaussian noise，跟OU noise並存，並且做成選項；gaussian noise的參數做成可調的超參數
    * 新增一個功能，可以依照欄位名稱決定要用哪一種機率分布去取樣
    * polobao發現我之前寫的悲劇性錯誤
* Aug 4
    * 寫個功能(gaussian noise on action price)  2 hr
    * 下午開會  1hr
    * 教人用shopee-api-g-sheet  0.5hr
    * 編輯shopee-api-g-sheet readme  1hr
    * 中午直播彩排  1 hr
    * 傍晚實際直播  1 hr
* Aug 5
    * fix the bug of extremely large gaussian noise
    * plot pdf 
    * ![](https://i.imgur.com/vNF9LW3.png)
    * https://gitlab.com/y56/intern2020/-/commit/49d5440d72ff2f325d0db58c3c627895df39080f
* Aug 6
    * solve issue: column-wise pdf
    * create issue: use sql in python to get data
* Aug 7
    * 中午直播彩排
    * get data by sql
        * adjust utc+0 to utc+8
        * get l3d, l7d, and l30d
        * solve a presicion issue
    * 全體實習生的直播會議
## 6th week
* Aug 10
    * Confirmed the correctness of l3d, l7d, and l30d.
    * sum(b.pv_ch)/30.0 AS l30d_pv has precision of 0.1, while sum(b.pv_ch)/cast(30.0 as real) AS l30d_pv has full precision.
    * Replace the traffic_table_LnD_hourly as `f` part in `fetch_data_hourly.py` with updated version, saving as `fetch_data_hourly_new_LnD.py`.
    * Put the long long sql string as another file. Use "xxx{a}xxx{b}xxx".format(a=a,b=b). Note that start_date = datetime.datetime(2020, month, 1).date() and then f'start_date ' will give '2020-01-01'.
    * Have to fix empty dataset_2020-06-01_2020-06-30_hourly_all.csv.
* Aug 11
    * [SQL] Succeeded generating a table w/ zero-valued rows for every hour in a specified range.
    * [SQL] Found a bug of `date_add()`: It gives a wrong timestamp when adding time to `1981-12-31 23:59:59.999`
        * https://gist.github.com/y56/f15ea575d53bf13c9c929ab2c2c33a13
    
* Aug 12
    * [SQL] Worked on getting the data. Should have been the desired data.
    * [SQL] Presto can't declare any timestamps between `1982-01-01 00:00:00.000` and `1982-01-01 00:29:59.999` (inclusively).
        * https://gist.github.com/y56/848005fb02fabe1c3934a7b2d160876c
* Aug 13
    * 中午直播當主持人
    * Examine the correctness of traffic data
        * https://shopee.tw/product-i.37137599.905684794
        * ![](https://i.imgur.com/1Zbc9nF.jpg)
* Aug 14
    * 中午直播彩排當場控
    * Discuss the skeleton of the pricing daemon w/ Kevin 
    * ![](https://i.imgur.com/hDjTATf.jpg)
    * Open more issues
        * Pickle actor networks for every sku
        * Output a suggested price from a pickled network and market features
        * Get L1D data
        * Decide pdf for each market feature
## 7th week
* Aug 17
    * Plot the 100 skus
    * The feature-vs-price plot show the benefit of hourly resolution profile: Large number of data points erase the need to abandon price with few data points.
    * Might need to interpolate in the time series
    * Meet to discuss the skeleton and daemon
## TODO:
* Build the pricing daemon
* Plot the behavior of grouped skus
* Develop grouping methodology


## notes for midterm presentation
* cur condition=?
* purpose=? 
* who have succeeded?
* what have we done? and how good?


:::danger
LSTM RNN GRU

:::




I found an interesting feature in our Presto recently and would like to know some more about it.

In Presto, 
```values from_unixtime(0);``` 
will give 
```1970-01-01 07:30:00.000``` 
instead the standard
```1970-01-01 00:00:00.000```.

The +7.5 hr shift continues until 
```values from_unixtime(378664199);```
which gives
```"1981-12-31 23:59:59.000"```.

Interestingly, 
```values from_unixtime(378664200);``` 
will give
```"1982-01-01 00:30:00.000"```,
which means a +8 hr shift.

When doing 
```date_add('second', 1, TIMESTAMP '1981-12-31 23:59:59.000')```, 
we will get 
```1982-01-01 00:30:00.000```
instead of 
```1982-01-01 00:00:00.000```.
This means we get wrong values when doing time addition.
