---
# System prepended metadata

title: Filter Spatial Channel Weighting

---

# Filter Spatial Channel Weighting

+ Author : 利醬
+ Date   : 2021/08/13

## Source
International Journal of Pattern Recognition and Artificial Intelligence
2021 - 
Filtering deep convolutional features for image retrieval


## PyTorch Implementation
<https://github.com/leonIceHot/Computer-Vision/tree/master/Filtering-Spatial-Channel-Weighting>

Project Architecture
![](https://i.imgur.com/b0TyQOe.png)


Filtering on spatial channel weighting (FSCW) 
-----
+ ![](https://i.imgur.com/ocwTmlQ.png)

  + Preprocessing
  + Model
  + Filter 
  + Spatial  
  + Channel Weighting  
  + Image Representation

### Preprocssing
+ Dataset : 
  + Oxford5K
    + The Oxford Buildings Dataset
    + <https://www.robots.ox.ac.uk/~vgg/data/oxbuildings/>
  
  + Paris6K 、 Paris106K
  	
+ Holdout validation
  + Total      : 5063 images 
  + Train      : 0.7  (3545)	
  + Validation : 0.3  (1518)
  + Test       : 0.5  (2531)

### Model
Pytorch implementation of cnn network 
+ <https://github.com/shanglianlm0525/PyTorch-Networks>
+ VGG
 + ![](https://i.imgur.com/KelDyLu.png)
 + ![](https://i.imgur.com/ufqD9YB.png)

+ ResNet
 + ![](https://i.imgur.com/thTbu8v.png)


### Filtering the feature maps
![](https://i.imgur.com/nabZn7i.png)
We extract the feature maps in the pool5 layer of VGG16 model, 
Whereas we extract the feature maps in the res5c_relu layer of the ResNet101 model

+ 𝑋 ∈ 𝑅(𝑊×𝐻×𝑁) be the 3-dim feature tensor extracted  
  + N = 512 feature maps
  + W: Width of a feature map
  + H: Height of a feature map

![](https://i.imgur.com/BuDYfaQ.png)

![](https://i.imgur.com/KG9HWXI.png)

![](https://i.imgur.com/aRnQRMd.png)


### Spatial Weighting
![](https://i.imgur.com/gYqGg10.png)

![](https://i.imgur.com/CSp4RHL.png)

![](https://i.imgur.com/t35H8DK.png)

![](https://i.imgur.com/MZc9RQU.png)


### Channel Weight


### Image Representation
![](https://i.imgur.com/0zKJuiv.png)

1. Sum-Pooling
   ![](https://i.imgur.com/6dQn3w4.png)

2. Global representation vector
   ![](https://i.imgur.com/FWo5i6W.png)

3. FSCW descriptor
   + PCA Whitening
     + Theorem
       + <http://ufldl.stanford.edu/tutorial/unsupervised/PCAWhitening/>
       + <http://ufldl.stanford.edu/tutorial/unsupervised/ExercisePCAWhitening/>
       + <https://www.twblogs.net/a/5b7e231c2b717768385582bf>
     + Implementation
       + sklearn.decomposition.PCA 
       + <https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html>
     
   
   + L2 Normalization 

## Experiment

### Query expansion (QE)
