---
# System prepended metadata

title: '[Kevin] 2019/7 July daily plan'

---

# [Kevin] 2019/7 July daily plan
## Schedule
- 7/1 ~ 7/9
    - 
    - Plan
        - Debug the code of inference part.
        - Let all the for loop in program is functionable.

- 7/1
    - settle down:
        - Choose my sit and computer, screen, chair.
        - Install the operating system (Ubuntu 16.04) and the developing environment
- 7/2
    - meet and know my student:
        - His name is Achmad Kripton Nugraha. I can call him Kripton.
    - Outcome:
        - Narrow down the problem. Inference has bug because the DataFrame become empty after excuting feature engineering. 
    - Note:
        - df is functionable.
        - df_idx is functionable.
        - df_new is Empty dataframe. So the Error is  happened in feature_engineering.

    - Tomorrow's plan:
        - Check the functions in feature_engineering, try to narrow down the problem.
---------------------------------------------------------
- 7/3 
	- Outcome:
	    - Narrow down the problem. The reason of the bug is df.dropna() drop whole array because there is some columns with all NaN in it.
	- Note:
        - Only need these columns: [Refilling, Hot temp , Cold temp, Warm Temp, Hot Valve, Warm Valve, Cold Vale] 
        - Feature engineering will generate other features I need.
	    - Power saving data will generate by Refilling data. (9th step of feature engineering)
	    - Time shifting will generate the data of yesterday (power saving) and last week (power saving).
	    - After that, time shifting will do df.dropna(), do drop all of NAN data in the data frame.(This is why I got a empty DataFrame, the current input data have some features that is useless and they are full of NAN. It make Whole DataFrame got dropped out.)
	    - DataFrame.shape in feature_engineering process:
	        (30240,23)
            (30240,23)
            (30240,23)
            (21600,25)
            (21600,30)
            (1440,30)
            (1440,30)
            (1440,30)
            (1440,31)
            (0,33) >>> after time shifting.
            
	- Tomorrow's plan:
	    -  Find out which features should be dropped, and where is the code dropping the features.
-----------------------------------------------------------------
- 7/4
	- Outcome:
	    - The module feature_engineering is functionable now.

	- Note:
	    - The inference data I put in was wrong, too many useless columns, drop them first.
		Columns_drop = ['Sterilizing', 'ErrorCode', 'SavingPower', 'ColdTemp_Insulation_Low', 'ColdTemp_Insulation_High', 'HotTemp_Insulation_Low', 'ColdTemp_Insulation', 'WarmTemp_Insulation', 'HotTemp_Insulation_High', 'HotTemp_Insulation', 'WaterLevel', 'TDS', 'Status', 'Consumption', 'Filter_Usage', 'Filter_Hint', 'Usage_CC', 'Usage_L', 'Usage_MT']
	    - After data_engineering, the columns are:		
		Columns = ['Heating', 'Cooling', 'Refilling', 'WarmTemp', 'ColdTemp', 'HotTemp', 'Consump', 'day', 'hour_class', 'cool_time_back23', 'consump_sum_back23', 'hottemp_mean_back23', 'warmtemp_mean_back23', 'coldtemp_mean_back23', 'power_saving', 'Last_week', 'yesterday']
	    - DAYS. For data_engineering, we input 21 days data, 15 days after weekend has dropped, and finally 10 days left after dropna() in time shifting.
	    - FEATURES. We have 9 features at first, drop 3 and then add 11 during data engineering, so we have 17 features after data engineering.
	    - DataFrame.shape in feature_engineering process:
            (30240, 7)
	        (30240, 7)
	        (30240, 7)
	        (21600, 9)
	        (21600, 14)
	        (1440, 14)
	        (1440, 14)
	        (1440, 14)
	        (1440, 15)
	        (960, 17)
	- Tomorrow's plan: 
	    - Write a code to  generate the answer for predicting power saving from raw data.
---------------------------------------------------------
- 7/5
    - Outcome:
        - Know how to maintain the dispensors.
             - step.1 use GreenMeter app to check the meter is working
		    - step.2 use Gateway app to check the Gateway is transfering the data
		    - step.3 use bluetooth to check the WiFi of the Gateway is available
		    - step.4 change the battery of Module if the battery is dead. The Module should hook on the red wire.
    
    - Note:
         - Compare df_real (answer) with df_heatmap (predict) to calculate the percentage of energy we saved.
	    - Answer of poer_saving is from refilling, if refilling >5hrs && <12hrs = 1 ; else = 0  
	    - There is some little different about the code and the definition above.
	    - Learn how to maintain the Dispensors. 
		    
    - Tomorrow's plan: 
        - Write a code to generate the answer for predicting power saving from raw data.
------------------------
- 7/8
    - Outcome: 
        - Finishing the answer of power saving (ps_time.py)
        - Step.1 Read csv file and drop columns and rows.
        - Step.2 Excute ps_time.main to calculate the answer of real_pstime
   	    - Step.3 Drop columns, then reshape ps_time. (row index = time, columns = date)
        - Step.4 Save as csv file. 
    - Note: 
        - The original power_saving code means:
        ```
        block       :s     e     s     e
        refilling   :1...11100000111...1    
        power_saving:....00111111100....
        
        #block means the Consecutive 1 of refilling
        #block size means the length
        ```
        - So I decide to modify the code, don't let power_svaing be 1 at the end and begin of block(111...1) 
    
 
    - Tomorrow's plan:
        -  Modify the code of power_saving to conform the meaning of power_saving we defined. (if refilling >=5hrs && <=12hrs = 1 ; else = 0)
--------------------------
- 7/9
    - Outcome:
        - Modify the code of power_saving and group (in inference_one_month.py) 
    - Note: 
        - The start and end of refilling=1 should not be power_saving = 1
        - The Time range should be 12>= time >=5.
        - Group have tolerate one minute of data. But for counting block size, the data's unit is 15mins, so it shouldn't tolerate once.
        - The code if index==0 is wrong, it should be counting the block size befor first block.
    - Tomorrow's plan:
        - Make sure the final metrics has same size for each columns. So we can dintinguish the output result correctly. 
