Augmentation Matters: A Simple-yet-Effective Approach to Semi-supervised Semantic Segmentation

CVPR2023
Authors: Zhen Zhao, Lihe Yang, Sifan Long, Jimin Pi, Luping Zhou, Jingdong Wang

Introduction

作者認為現今對於Semi-supervised的訓練方式採取日益複雜的機制，在帶來顯著成效的同時也造成不小的成本代價，所以作者提出一種簡單有效的方式透過data augmentation提升傳統的teacher-student模型的效能。
一般來說auto augmentations是應用在supervised的訓練當中，其目的性旨在找出最佳策略的augmentation方式，而Semi-supervised只為了產生相同輸入的預測分岐，作者希望可以使auto augmentations更貼合Semi-supervised所需要的方向，他簡化現存的randomAug和設計一款基於強度的高隨機增強方式。
此外作者認為使用一般的threshold篩選可用的unlabeled data可能造成沒被挑選到的data被浪費掉，且為了使信任值較低的unlabeled data不會破壞整個模型訓練效能，本篇論文提出了利用labeled data資訊混和過低信賴值的unlabeled data，在unlabeled data中採取隨機裁剪黏貼的data augmentation可對unlabel的訓練效果顯著提升。

Contribution

打破日益複雜的semi-supervised方法提出基於傳統teacher-student框架得到良好的效能
修改auto augmentations使他更貼合SSS的應用，且自適應的注入label data資訊
提出一個簡單而有效的strong baseline提供往後SSS的發展

Random Intensity-based Augmentations

Image Not Showing Possible Reasons

The image was uploaded to a note which you don't have access to
The note which the image was originally uploaded to has been deleted

從augmentation pool中隨機sample出k種augmentation取代傳統固定數量的方式
augmentation的強度在連續空間中做sample
從pool中移除太過強烈的augmentation

Adaptive Label-aided CutMix

Image Not Showing Possible Reasons

The image was uploaded to a note which you don't have access to
The note which the image was originally uploaded to has been deleted

計算信心值ρi的方式:

Image Not Showing Possible Reasons

The image was uploaded to a note which you don't have access to
The note which the image was originally uploaded to has been deleted

區塊混和:

Image Not Showing Possible Reasons

The image was uploaded to a note which you don't have access to
The note which the image was originally uploaded to has been deleted

將unlabeled data重新排列並跟原本序列中的data做比較，信心值小於原本data則跟labeled data部分區塊做CutMix

Image Not Showing Possible Reasons

The image was uploaded to a note which you don't have access to
The note which the image was originally uploaded to has been deleted

對unlabeled data之間做隨機裁剪黏貼

Framwork

Image Not Showing Possible Reasons

The image was uploaded to a note which you don't have access to
The note which the image was originally uploaded to has been deleted

student model:

Bx為labeled data的batch，Lx以pixel-wise方式做一般的cross entropy

Image Not Showing Possible Reasons

The image was uploaded to a note which you don't have access to
The note which the image was originally uploaded to has been deleted

Ag為weak augmentation方式，主要用於teacher模型中
Aa為Adaptive Label-aided CutMix的data augmentation，Ar為Random Intensity-based Augmentations中簡化的RandomAug，T將兩者包在一起用於student模型

Image Not Showing Possible Reasons

The image was uploaded to a note which you don't have access to
The note which the image was originally uploaded to has been deleted

Bu為unlabeled data的batch，Lu將teacher預測結果給student做supervised learning

Image Not Showing Possible Reasons

The image was uploaded to a note which you don't have access to
The note which the image was originally uploaded to has been deleted

teacher model:

Image Not Showing Possible Reasons

The image was uploaded to a note which you don't have access to
The note which the image was originally uploaded to has been deleted

將student的知識用EMA方式給teacher