{%hackmd SybccZ6XD %}
###### tags: `paper`

# Beyond Self-Attention: External Attention Using Two Linear Layers for Visual Tasks

Goal
> Decrease computational complexity.
>
> Mine potential relationships across the whole dataset.

Self-attention
> 
> 

Simplified self-attention
> 
> 

External-attention
> 
> 

How to Mine potential relationships across the whole dataset?
> self-attention: only considers the relation between elements within a data sample and ignores potential relationships between elements in different samples
>
> External-attention: the similarity between the ith feature and the jth row of M

Ablation study on PASCAL VOC dataset
> 單純換上EA，結果沒有比較好，還有另外調整Norm
> 

Ablation Study on Different Multi-Head Mechanism on ImageNet Dataset
> 從這裡可以看出來，parameter會增加
> 

Experiment
> image classification, object detection, semantic segmentation, instance segmentation, image generation, point cloud classification, and point cloud segmentation tasks