---
# System prepended metadata

title: LVIS @ LVLab

---

LVIS @ LVLab
===

## Table of Contents

[TOC]

## Environment Setup

#### 0. Cuda version: **9.1**

#### 1. Donwload and install anaconda 
https://repo.anaconda.com/archive/Anaconda3-2019.07-Linux-x86_64.sh

#### 2. Setup the environment
```shell=
conda create -n mmdet python=3.7 -y
source activate mmdet

pip install torch==1.1.0 torchvision==0.2.0
pip install cython==0.29.12
pip install mmcv matplotlib
pip install terminaltables
```
#### 3. Get code
Download mmdetection-liyu.tar.gz file in the email and unzip it. Then, compile mmdet operations:
```shell=
cd mmdetection-liyu
pip install -v -e .
```

#### 4. Prepare the dataset

**a. For dataset images:**

```shell=
    # Make sure you are in $ROOT/mmdetecion-liyu
    rm -rf data
    mkdir data
    cd data
    mkdir lvis
    mkdir pretrained_models
```
    
- If you already have COCO2017 dataset on your server, it will be great. Link `train2017` and `val2017` folders under `lvis`.
- If you do not have COCO2017 dataset, please download:
        http://images.cocodataset.org/zips/train2017.zip
        http://images.cocodataset.org/zips/val2017.zip
        and Unzip these files and mv them under `lvis`.

**b. For dataset annotations:**
- Download annotations:
        http://calvin.inf.ed.ac.uk/wp-content/uploads/data/cocostuffdataset/stuffthingmaps_trainval2017.zip
        https://s3-us-west-2.amazonaws.com/dl.fbaipublicfiles.com/LVIS/lvis_v0.5_train.json.zip
        https://s3-us-west-2.amazonaws.com/dl.fbaipublicfiles.com/LVIS/lvis_v0.5_val.json.zip
- Unzip all the files and put them under `lvis`, and change the name of folder `stuffthingmaps_trainval2017` to `stuffthingmaps`.

**c. For pretrained models:**
- Download: https://s3.ap-northeast-2.amazonaws.com/open-mmlab/mmdetection/models/htc/htc_dconv_c3-c5_mstrain_400_1400_x101_64x4d_fpn_20e_20190408-0e50669c.pth
- Move this model file to `$ROOT/mmdetection-liyu/data/pretrained_models/`


**After all these operations, the folder `data` should be like this:**
```
    data
    ├── lvis
    │   ├── lvis_v0.5_train.json
    │   ├── lvis_v0.5_val.json
    │   ├── stuffthingmaps
    │   │   ├── train2017
    │   │   │   ├── 000000004134.png
    │   │   │   ├── 000000031817.png
    │   │   │   ├── ......
    │   │   └── val2017
    │   │       ├── 000000424162.png
    │   │       ├── 000000445999.png
    │   │       ├── ......
    │   ├── train2017
    │   │   ├── 000000100582.jpg
    │   │   ├── 000000102411.jpg
    │   │   ├── ......
    │   └── val2017
    │       ├── 000000062808.jpg
    │       ├── 000000119038.jpg
    │       ├── ......
    └── pretrained_models
        └── htc_dconv_c3-c5_mstrain_400_1400_x101_64x4d_fpn_20e_20190408-0e50669c.pth
```

#### 5. Install datasets APIs:
```shell=
pip install 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'
pip install lvis
```

Train the model
---
```shell=
tmux

cd $ROOT/mmdetection-liyu

./tools/dist_train.sh configs/htc/htc_dconv_c3-c5_mstrain_400_1400_x101_64x4d_fpn_20e_lvis.py ${NUMBER_OF_GPUS}

```

If the we have 8 GPUs, the commond should be:
```shell=
./tools/dist_train.sh configs/htc/htc_dconv_c3-c5_mstrain_400_1400_x101_64x4d_fpn_20e_lvis.py 8
```

> We should check if the loss is gradually decreasing. 
> If the loss goes very large after a few steps, pleas contect us immediately.

Evaluation
---
Test with 8 GPUs:
```shell=
cd $ROOT/mmdetection-liyu

./tools/dist_test_lvis.sh configs/htc/htc_dconv_c3-c5_mstrain_400_1400_x101_64x4d_fpn_20e_lvis.py ./work_dirs/htc_dconv_c3-c5_mstrain_400_1400_x101_64x4d_fpn_20e_lvis/latest.pth 8 --out ./work_dirs/htc_dconv_c3-c5_mstrain_400_1400_x101_64x4d_fpn_20e_lvis/results.pkl --eval bbox segm
```

Normally, we should get something like this:
```
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 5000/5000, 11.0 task/s, elapsed: 454s, ETA:     0s
writing results to ./work_dirs/mask_rcnn_r50_fpn_1x_lvis_rpn/results.pkl
Starting evaluate bbox and segm
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=300 catIds=all] = 0.160
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=300 catIds=all] = 0.263
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=300 catIds=all] = 0.168
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=     s | maxDets=300 catIds=all] = 0.132
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=     m | maxDets=300 catIds=all] = 0.223
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=     l | maxDets=300 catIds=all] = 0.278
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=300 catIds=  r] = 0.003
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=300 catIds=  c] = 0.117
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=300 catIds=  f] = 0.276
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 catIds=all] = 0.197
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=     s | maxDets=300 catIds=all] = 0.153
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=     m | maxDets=300 catIds=all] = 0.269
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=     l | maxDets=300 catIds=all] = 0.342
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=300 catIds=all] = 0.157
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=300 catIds=all] = 0.248
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=300 catIds=all] = 0.168
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=     s | maxDets=300 catIds=all] = 0.113
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=     m | maxDets=300 catIds=all] = 0.230
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=     l | maxDets=300 catIds=all] = 0.306
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=300 catIds=  r] = 0.005
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=300 catIds=  c] = 0.121
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=300 catIds=  f] = 0.263
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 catIds=all] = 0.192
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=     s | maxDets=300 catIds=all] = 0.139
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=     m | maxDets=300 catIds=all] = 0.269
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=     l | maxDets=300 catIds=all] = 0.346
```

***Please copy this log infomation to a txt file, like results.txt***


What we need:
-----
1. The aforementioned `results.txt` file.
2. The whole `./work_dirs/htc_dconv_c3-c5_mstrain_400_1400_x101_64x4d_fpn_20e_lvis/` folder.



********************
Update 0924:
=========
**What's new:**
- Add instaboost
- Add image sampling
- Add cosine lr decay strategy

***Please copy the whole mmdetection-liyu folder and the environment to another machine, and do the following steps.***


## Environment update:
#### 1. env
```shell=
pip install instaboost
```
#### 2. code
Please replace the following files with the attachments:
-  `mmdetection/mmdet/datasets/pipelines/loading.py`
-   `mmdetection/mmdet/datasets/custom.py`
- `mmdetection/mmdet/datasets/loader/build_loader.py`
- `mmdetection/mmdet/datasets/loader/sampler.py`
- `mmdetection/mmdet/apis/train.py`

Add the following files to `mmdetection/configs/htc/`
- `htc_dconv_c3-c5_mstrain_400_1400_x101_64x4d_fpn_20e_lvis_insta_cos_imgsampling.py`
- `htc_dconv_c3-c5_mstrain_400_1400_x101_64x4d_fpn_20e_lvis_insta_cos.py`

#### 3. Other files
Please put `class_to_imageid_and_inscount.pt` under `mmdetection-ins/data/lvis/`

## Runs:

#### Run #1:
```shell=
./tools/dist_train.sh configs/htc/htc_dconv_c3-c5_mstrain_400_1400_x101_64x4d_fpn_20e_lvis_insta_cos.py 8
```

#### Run #2:
```shell=
./tools/dist_train.sh configs/htc/htc_dconv_c3-c5_mstrain_400_1400_x101_64x4d_fpn_20e_lvis_insta_cos_imgsampling.py 8
```

**Please contact me anytime if there are any problems.**

## Thanks very much!