Competition - HackMD

# Competition ## 12/1 데이터셋 접근 권한 이슈 ## 12/2 데이터셋 다운로드 내일 계획 1. Instance Segmentation 모델 탐색 (SOTA 위주로 + 깃 클론해서 에러 해결까지) 2. 선정된 SOTA에 contest dataset format 변환해서 돌려보기 3. 성능 비교해서 baseline 선정 4. Google colab PRO 결제하기 ## 12/3 ### 모델 선정 & 환경 구축 테스트 Dexter: 1. PANet <- 개 같은 거 갖다버렸음 2. Detectron2 프레임워크 세팅 (mask-rcnn 등 [다양한 모델](https://github.com/facebookresearch/detectron2/blob/main/MODEL_ZOO.md) 실험 가능) - 컴페티션 커스텀 데이터 dataset (loader) 구축 완료 Mirae: 1. Mask r cnn <- 고통 그 자체 ### Code: custom dataset ```python from detectron2.structures import BoxMode classes = ['pet', 'ps', 'pp', 'pe'] def get_contest_dicts(data_dir): image_dir = os.path.join(data_dir, 'image') label_dir = os.path.join(data_dir, 'annotation') dataset_dicts = [] for obj_cls in os.listdir(label_dir): for i, file_name in enumerate(os.listdir(os.path.join(label_dir, obj_cls))): json_file = os.path.join(os.path.join(label_dir, obj_cls), file_name) if i < 3: with open(json_file) as f: label = json.load(f) record = {} record['file_name'] = os.path.join(image_dir, obj_cls, label['images'][0]['file_name']) for value in ['height', 'width']: record[value] = label['images'][0][value] record['image_id'] = label['images'][0]['id'] objs = [] for instance in label['annotations']: obj = { "bbox": instance['bbox'], "bbox_mode": BoxMode.XYWH_ABS, "segmentation": instance['segmentation'], "category_id": instance['category_id'] - 1, } objs.append(obj) record['annotations'] = objs dataset_dicts.append(record) return dataset_dicts for d in ["train", "test"]: DatasetCatalog.register("contest_" + d, lambda d=d: get_contest_dicts("contest_dataset/" + d)) MetadataCatalog.get("contest_" + d).set(thing_classes=classes) contest_metadata = MetadataCatalog.get("contest_train") ``` ### 내일 생각해볼 것: - train / val / test split - model selection ### 12/4 ## 12/5 - 평가기준: COCO API에서 클래스 별 seg AP 0.50 * COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml * iter: 300 (default) | AP | AP50 | AP75 | APs | APm | APl | |:------:|:------:|:------:|:-----:|:-------:|:------:| | 22.252 | 23.833 | 23.833 | nan | 100.000 | 20.955 | | category | AP | category | AP | category | AP | |:-----------|:-------|:-----------|:-------|:-----------|:------| | pet | 37.086 | ps | 51.921 | pp | 0.000 | | pe | 0.000 | | | | | ------------------------------------------------------ * COCO-InstanceSegmentation/mask_rcnn_X_101_32x8d_FPN_3x.yaml | AP | AP50 | AP75 | APs | APm | APl | |:------:|:------:|:------:|:-----:|:-----:|:------:| | 26.815 | 27.640 | 27.640 | nan | nan | 26.815 | | category | AP | category | AP | category | AP | |:-----------|:-------|:-----------|:-------|:-----------|:------| | pet | 40.924 | ps | 66.337 | pp | 0.000 | | pe | 0.000 | | | | | * COCO-InstanceSegmentation/mask_rcnn_X_101_32x8d_FPN_3x.yaml * iter: 3000 * LVISv0.5-InstanceSegmentation/mask_rcnn_X_101_32x8d_FPN_1x.yaml | AP | AP50 | AP75 | APs | APm | APl | |:------:|:------:|:------:|:-----:|:-------:|:------:| | 21.324 | 22.277 | 22.277 | nan | 100.000 | 20.041 | | category | AP | category | AP | category | AP | |:-----------|:-------|:-----------|:-------|:-----------|:------| | pet | 32.574 | ps | 52.723 | pp | 0.000 | | pe | 0.000 | | | | | * LVISv0.5-InstanceSegmentation/mask_rcnn_X_101_32x8d_FPN_1x.yaml * iter: 3000 * batchsize: 256 | AP | AP50 | AP75 | APs | APm | APl | |:------:|:------:|:------:|:-----:|:-------:|:------:| | 35.594 | 35.594 | 35.594 | nan | 100.000 | 35.594 | | category | AP | category | AP | category | AP | |:-----------|:-------|:-----------|:--------|:-----------|:------| | pet | 42.376 | ps | 100.000 | pp | 0.000 | | pe | 0.000 | | | | | --------------------------------------------------------- 여기까지는 12개 데이터만 가지고 학습 후 12개 데이터만 가지고 평가한 결과. --- #### Full dataset result * COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml * iter: 3000 * batchsize: 512 * lr_step: 1000, 2000 | AP | AP50 | AP75 | APs | APm | APl | |:------:|:------:|:------:|:-----:|:------:|:------:| | 13.965 | 15.201 | 14.918 | 0.000 | 17.624 | 14.146 | | category | AP | category | AP | category | AP | |:-----------|:-------|:-----------|:-------|:-----------|:------| | pet | 26.597 | ps | 29.265 | pp | 0.000 | | pe | 0.000 | | | | | ### 앞으로 할 것 1. class별 AP 0 issue (bg까지 포함 class 5개로?) 1. Train / Val split 후 train 시 validation 포함 & 자동화 1. backbone / neck / head 공부 & 조사하고 커스텀 1. Data augmentation 1. LR, LRstep, Optimizer, loss 등 실험 ## 12/6 - [PET, PE, PS, PP 의미](https://to-heaven.tistory.com/762) - PET: 폴리에틸렌 테레프탈레이트 (패트병) - PP: 폴리프로필렌 (요거트 용기, 지퍼락) - PS: 폴리스틸렌 (플라스틱 숟가락, 볼펜심) - PE: 폴리에틸렌 - [Base RCNN](https://github.com/facebookresearch/detectron2/blob/main/configs/Base-RCNN-FPN.yaml) - [Detectron2 config](https://detectron2.readthedocs.io/en/latest/modules/config.html) #### 결과 1. COCO/mask_rcnn_R_101_FPN_3x - Params - batch size: 8 - LR: 0.002 - LR Step: 3000, 6000, 9000 - RPN proposal size: 512 - freeze: 0 - Iter: 1000 | AP | AP50 | AP75 | APs | APm | APl | |:------:|:------:|:------:|:-----:|:------:|:------:| | 14.000 | 14.284 | 14.284 | 0.000 | 94.374 | 13.751 | | category | AP | category | AP | category | AP | |:-----------|:-------|:-----------|:-------|:-----------|:------| | pet | 25.033 | ps | 30.966 | pp | 0.000 | | pe | 0.000 | | | | | - iter: 2000 | AP | AP50 | AP75 | APs | APm | APl | |:------:|:------:|:------:|:-----:|:-----:|:------:| | 13.606 | 13.814 | 13.814 | 0.000 | 7.228 | 13.615 | | category | AP | category | AP | category | AP | |:-----------|:-------|:-----------|:-------|:-----------|:------| | pet | 23.259 | ps | 31.166 | pp | 0.000 | | pe | 0.000 | | | | | - iter: 3000 | AP | AP50 | AP75 | APs | APm | APl | |:------:|:------:|:------:|:-----:|:------:|:------:| | 13.605 | 13.764 | 13.764 | 0.000 | 85.545 | 13.354 | | category | AP | category | AP | category | AP | |:-----------|:-------|:-----------|:-------|:-----------|:------| | pet | 23.052 | ps | 31.367 | pp | 0.000 | | pe | 0.000 | | | | | - iter: 4000 | AP | AP50 | AP75 | APs | APm | APl | |:------:|:------:|:------:|:-----:|:------:|:------:| | 13.151 | 13.305 | 13.305 | 0.000 | 56.634 | 13.110 | | category | AP | category | AP | category | AP | |:-----------|:-------|:-----------|:-------|:-----------|:------| | pet | 21.381 | ps | 31.224 | pp | 0.000 | | pe | 0.000 | | | | | 2. COCO/mask_rcnn_R_101_FPN_3x - Params - batch size: 8 - LR: 0.001 - LR Step: 2000, 4000 - RPN proposal size: 512 - freeze: 0 - image size: 1024 - Iter: 500 ### 앞으로 할 것 1. Data augmentation 1. backbone / neck / head 공부 & 조사하고 커스텀 1. LR, LRstep, Optimizer, loss 등 실험 ## 12/8 ## 12/9 - Trainer class customize - Train, val, test split - Validation while training - Augmentation experiment - Error fixed by deleting output folder (cache), but still some classes becomes 0 mAP later anyway ### 결과 1. COCO/mask_rcnn_R_101_DC5_3x.yaml !!! contest_val로 테스트 진행!!! - Params - batch size: 4 - LR: 0.0001 - LR Step: 2000, 4000 - RPN proposal size: 512 - freeze: 0 - ITER: 5000 - Iter: 1000 | category | AP | category | AP | category | AP | |:-----------|:-------|:-----------|:-------|:-----------|:------| | pet | 12.141 | ps | 13.549 | pp | 0.000 | | pe | 0.000 | | | | | - Iter: 3000 | category | AP | category | AP | category | AP | |:-----------|:-------|:-----------|:-------|:-----------|:------| | pet | 14.206 | ps | 14.187 | pp | 0.000 | | pe | 0.000 | | | | | ## 12/10 ### 결과 1. COCO/mask_rcnn_R_50_DC5_3x.yaml - Aug - Resize - RandomFlip - RandomBrightness - RandomContrast - RandomSaturation - RandomCrop - Params - batch size: 4 - LR: 0.001 - LR Step: 2000, 4000 - RPN proposal size: 512 - freeze: 0 - ITER: 5000 - Iter: 1000 | category | AP | category | AP | category | AP | |:-----------|:-------|:-----------|:-------|:-----------|:------| | pet | 10.643 | ps | 14.109 | pp | 0.000 | | pe | 0.000 | | | | | - Iter: 3000 | category | AP | category | AP | category | AP | |:-----------|:------|:-----------|:-------|:-----------|:------| | pet | 9.061 | ps | 14.378 | pp | 0.000 | | pe | 0.000 | | | | | - Iter: 5000 | category | AP | category | AP | category | AP | |:-----------|:------|:-----------|:-------|:-----------|:------| | pet | 8.819 | ps | 14.377 | pp | 0.000 | | pe | 0.000 | | | | |

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.