Global and Local Mixture Consistency Cumulative Learning for Long-tailed Visual Recognitions

作者：Fei Du, Peng Yang, Qi Jia, Fengtao Nan, Xiaoting Chen, Yun Yang
論文連結：https://openaccess.thecvf.com/content/CVPR2023/papers/Du_Global_and_Local_Mixture_Consistency_Cumulative_Learning_for_Long-Tailed_Visual_CVPR_2023_paper.pdf

整理by: chewei

主要貢獻：

• One-Stage training strategy : GLMC
  • 一個Loss: Global and Local mixture consistency Loss
  • 一個逐Epoch累計變化的Loss 參數

$\to$ 主要圍繞在Loss改進

整體架構

• 首先以[1] BBN方式做出inverse sampler 使head->tail和tail->head 的sample相同，目的是為了要增強 Tail Class 的精確度：
  Sample Function:
  
  $$\begin{gathered} P_i=\frac{w_i}{\sum _{j=1}^c{w}_j}, \\ {w}_i=\frac{N_{max}}{N_i} \end{gathered}$$
  
• 接下來對此兩張照片一起做Global Mixture(MixUp)，以及Local Mixture(CutMix)
  Global Mixture Function:
  
  $$\begin{array}{rl}& \lambda ～Beta(\beta ,\beta )\\ & {\tilde{x}}_g=\lambda x_i+(1-\lambda )x_j,\\ & {\tilde{p}}_g=\lambda p_i+(1-\lambda )p_j.\end{array}$$
  Local Mixture Function:
  
  $$\begin{array}{rl}& {\tilde{x}}_l=\text{M}\odot x_i+(\text{1}-\text{M})\odot x_j.\\ \text{Bounding Box :}\\ & B=(r_x,r_y,r_w,r_h)\\ & r_x～Uniform(0,W),r_w=W\sqrt{1-\lambda }\\ & r_y～Uniform(0,H),r_h=H\sqrt{1-\lambda }\end{array}$$
  
• 接著放進Model(ResNet32)後分別算出
  • $L_{CE}(C(\tilde{x}),\tilde{y})$
  • $L_{cb}(C(\tilde{x}),\tilde{w},\tilde{y})$
  • $L_{sim}=sim(u_g,sg(h_l))+sim(u_l,sg(h_g))$

$L_{CE}$爲CrossEntropy Loss：

$$L_{EC}=-\frac{1}{2N}\sum _{i=1}^N({\tilde{p}}_g^i(logf({\tilde{x}}_g^i))+{\tilde{p}}_l^i(logf({\tilde{x}}_l^i)))$$

$L_{CE}$爲Rebalanced Loss：

$$L_{cb}=-\frac{1}{2N}\sum _{i=1}^N{\tilde{w}}^i({\tilde{p}}_g^i(logf({\tilde{x}}_g^i))+{\tilde{p}}_l^i(logf({\tilde{x}}_l^i)))$$

$L_{sim}$爲[2]SimSiam 所提出之 SimSiam Loss：

$$L_{sim}=sim(u_g,sg(h_l))+sim(u_l,sg(h_g))$$

結合方式爲:

$$L_{total}=\alpha L_{CE}+(1-\alpha )L_{cb}+\gamma L_{sim}$$

$\gamma$經過消融實驗後設定爲10

$\alpha$計算方式爲：

$$\alpha =1-(\frac{T}{T_{max}}{)}^2$$

---

整體架構圖：

Loss架構圖：

Related Work

stop-gradient
simulate cosine

參考資料

[1] BBN
[2] siamsim

BBN筆記