# 記憶體的資料映射討論 ## numpy data mapping - resize 1張照片成300x300 (dtype uint8) - 300x300x3 x 1byte = 0.26 MB - resize 1000張照片成300x300 (dtype uint8) - 1000x300x300x3 x 1byte = 257 MB - resize 1000張 照片成300x300 (dtype float32) - 1000x300x300x3 x 4byte = 1030 MB - code ```python= import numpy as np import sys data_1 = np.ones((1, 300, 300, 3), dtype='uint8') data_1000 = np.ones((1000, 300, 300, 3), dtype='uint8') data_1_size = getsizeof(data_1)/1024/1024 data_1000_size = getsizeof(data_1000)/1024/1024 print('data 1 type: {} \t data 1000 type: {}'.format(data_1.dtype, data_1000.dtype)) print('data 1 size: {}MB \t data 1000 size: {}MB'.format( round(data_1_size, 2), round(data_1000_size, 2))) data_1 = data_1.astype('float32') data_1000 = data_1000.astype('float32') data_1_size = getsizeof(data_1)/1024/1024 data_1000_size = getsizeof(data_1000)/1024/1024 print('\ndata 1 type: {} \t data 1000 type: {}'.format(data_1.dtype, data_1000.dtype)) print('data 1 size: {}MB \t data 1000 size: {}MB'.format( round(data_1_size, 2), round(data_1000_size, 2))) ``` - output ![](https://i.imgur.com/Kh1IQu8.png) ## training data mapping ### training input data: - 1000張x300x300x3x4=1030MB (4是因為訓練會用float32佔4byte) ## training weights: - model - 假設使用Xception (84MB) - 模型84MBx2(forward&backward 168MB) - batch operations - batchsize=1, 4, 16, 32, 64 - 記憶體佔 1x168MB - 記憶體佔 4x168MB - 記憶體佔16x168MB=2.6GB - 記憶體佔32x168MB=5.2GB - 記憶體佔64x168MB=10 GB