---
# System prepended metadata

title: Dense Prediction with Attentive Feature Aggregation
tags: [ETHz]

---

---
title: Dense Prediction with Attentive Feature Aggregation
tags: ETHz
---

## Links
- Submission
    - [NeurIPS 2021](https://www.overleaf.com/project/60915535c48467422f527568)
    - [AAAI 2022](https://www.overleaf.com/4978887446xrqngqtsqptr)
    - [CVPR 2022](https://www.overleaf.com/4353386925yzvzsxxzqfqy)
    - [ECCV 2022](https://www.overleaf.com/project/6222211bd00b0ae7bb079633)
    - [WACV 2023](https://www.overleaf.com/project/62c31643dee60dd4c119de6d)
- Code
    - [Develop Codebase](https://github.com/SysCV/ms-attention-network)
    - [Release Codebase](https://github.com/SysCV/dla-afa)
    - [TIMM](https://github.com/RoyYang0714/pytorch-image-models)
    - [mmsegmentation](https://github.com/RoyYang0714/mmsegmentation)
    - [mmdetection](https://github.com/RoyYang0714/mmdetection)
- [Shared Folder](https://drive.google.com/drive/u/0/folders/1GMR4UGvsoxA9hGHUEl42l366enEVf6H4)
    - [Experiment sheet - Cityscapes](https://docs.google.com/spreadsheets/d/1lcc_HFGmnseXtz_093CB3EMWEemrS3yKOt6GTY8BVQM/edit)
    - [Experiment sheet - BDD100K](https://docs.google.com/spreadsheets/d/1r6Fi6-NXcJSXzDy8vjl6zcsVaQuepgo-XiOnPwcZmjU/edit)
    - [Experiment sheet - Mapillary Vistas](https://docs.google.com/spreadsheets/d/1ySYq0MdRSAGjjOPSQLwgi75L47LRTVQLijdWVBsAGnI/edit)
    - [Experiment sheet - ImageNet-1K](https://docs.google.com/spreadsheets/d/1ZsHIFPhHQNVC3fQiylUK70PqwiReuv0RZXNsUxytEZk/edit#gid=0)
    - [Experiment sheet - COCO](https://docs.google.com/spreadsheets/d/1_4RLpkuL37BZlMvza0cFTR6JiZRxVZgNat9LEhCOxcI/edit#gid=0)
    - [Experiment sheet - Ablation](https://docs.google.com/spreadsheets/d/1TuDQN0s0Y19zzI1MS1oSMWpoztr3ZUDotoZeGNm67v4/edit)

## Contribution
- Attentive Feature Aggregation
    - Improve aggregation method -> AFA
    - Competitive model with lower parameters and operations
    - Scale-Space Rendering Attention
        - SSR solve gradient vanishing -> stronger performance
- Cityscaps
    - Validation: SOTA w/o extra data
    - Testing: 2nd place w/o extra data
- Mapillary Vistas
    - Validation
- BDD100K
    - Validation: SOTA

## To-Do For WACV2023
### Release
- [ ] Cameara-Ready to arXiv
- [ ] Porject Page update
    - [ ] Teaser Update
    - [ ] WACV Title
    - [ ] Video
- [ ] Illustration Video
    - [ ] PPT
    - [ ] Narration
- [ ] Code Release
    - [ ] Checkpoint check
    - [ ] mmsegmentation patch

## To Do for CVPR
### Beyound Deadline
- [ ] Semantic Segmentation
    - [ ] Pytorch-lightning version for DDP
        - [x] Rewrite codebase
        - [x] Reproduce performance
        - [ ] Move mmseg experiments into codebase
    - [ ] SOTA Challenge
        - [ ] Model tuning
            - [ ] Batch Size v.s. Training Epoch / Iterations v.s. Learning Rate
            - [ ] Crop Size v.s. Batch Size
        - [ ] Using MV pre-train
        - [ ] Saving Memory
            - [ ] Interpolation for GT during training
            - [ ] Modifying cuda computation for scaled segmentaion results
- [ ] Classification
    - [ ] AFA-DLA-34

## To Do for AAAI
### Beyond Deadline
- [x] Multi-media
    - [x] NYUDv2 -> Thinner the edge
    - [x] Cityscapes 
    - [x] BDD100K Demo
- [x] Arxiv
    - [x] Extend Ablation
    - [x] Add Supplementary to main
    - [x] Adjust the SSR figure
    - [x] Move attentive vis to main
    - [x] Resolve Thomas's feedback
- [ ] Codebase Release
    - [ ] SysCV Template
    - [x] Project page

### Supplementary
- [x] Technical
    - [x] Quantitative Results
    - [x] Gradient Vanishing
    - [x] Training Loss
    - [x] Training details
    - [x] Attention Visualization
        - [x] AFA-DLA v.s. DLA
        - [x] Binary
        - [x] Multiple
    - [x] Qualitative results
        - [x] Cityscapes
        - [x] BDD100K
        - [x] MV
        - [x] Boundary Detection
- [x] NeurIPS paper
- [x] NeurIPS cover letter
- [x] Multi-media
    - [x] Cityscapes video
- [x] Codebase


## To Do for NeurIPS
### Before Deadline
- [x] DLA
    - [x] Data augmentation
        - [x] Gaussian blur (Vistas)
        - [x] Random rotate (10 degreee)
        - [x] Random Scale (0.5, 2.0)
        - [x] Random Horizontally Flip
    - [x] Learning rate warm up
    - [x] RMI Loss
    - [x] Aux seg head
    - [x] Binary Fusion
        - [x] Spatial Attention
        - [x] Channel Attention
    - [x] L5 Fusion
    - [x] Auto Labelling
    - [x] DLA-Up-OCR

- [x] Scale-Space Rendering Attention
    - [x] abs activation function
    - [x] Self attention trend: train on [0.5,1.0]

- [x] Cityscapes
    - [x] Data Loader
    - [x] Evaluation
    - [x] Benchmark Submission
    - [x] DLA169 final model performance
        - [x] Imagenet pre-trained
        - [x] Mapillart Vistas pre-trained
        - [x] BDD100K pre-trained
    - [x] DLA102x final model performance
        - [x] Imagenet pre-trained
    - [x] Larger Crop Size: (1024, 1024)
    - [x] Larger Batch Size: 16
    - [x] Seg-Fix

- [x] Mapillary Vistas
    - [x] Data Loader
    - [x] Presize: 2176
    - [x] Evaluation
    - [x] DLA169 final model performance
        - [x] Imagenet pre-trained
    - [x] Larger Crop Size: (832, 832)
    - [x] Larger Batch Size: 16

- [x] BDD-100K
    - [x] Data Loader
    - [x] Evaluation
    - [x] DLA169 final model performance
        - [x] Imagenet pre-trained
        - [x] Cityscapes pre-trained
    - [x] DLA34Up performance
    - [x] Larger Crop Size: (720, 1280)
    - [x] Larger Batch Size: 16

- [x] Ablation Study
    - [x] AFA Improvement
    - [x] Multi-scale Attention Comparison

- [x] Benchmark Comparison
    - [x] Cityscapes val
    - [x] Citysapes test
    - [x] Mapillary Vistas val
    - [x] BDD100k val

### Supplementary
- [x] Cityscapes test set results
- [x] Systematic visualizations
- [x] Code submission
    - [x] Seg Fix part
- [x] Training details
    - [x] Add final inference scales
- [x] Licnese: dataset & codebase

## Dataset
### Cityscapes
- Resolution: 2048 x 1024
- Splits
    - Train + val: 2975 + 500
    - Coarse Label: 20000
    - Test: 1525
- Categories:
    - 19 classes

### Mapillary Vistas
- Resolution (most 4:3, mean ratio=1.38)
    |     Height    |      Width     |
    | :-----------: | :------------: |
    | 1000 ~ 6000 px| 2000 ~ 7000 px |
- Splits
    |   Training    |   Validation   |     Testing    |
    | :-----------: | :------------: | :------------: |
    |     18000     |      2000      |      5000      |
- Categories
    - 65 classes

### BDD100K
- Resolution: 1280 x 720
- Splits
    - Training: 7000
    - Validation: 1000
    - Testing: 2000
- Categories:
    - 19 classes

### ImageNet - 2012
- [Task](https://image-net.org/challenges/LSVRC/2012/)