CAM (Class Activation Mapping)
===
###### tags: `PyTorch`, `XAI`
> 使用 [jacobgil 大大](https://github.com/jacobgil/pytorch-grad-cam)所製作 CAM package 的筆記
## :floppy_disk: Install
有好心人 [sugatoray ](https://github.com/jacobgil/pytorch-grad-cam/issues/180)上傳至 conda-forge, 可直接從 conda 安裝
```
conda install -c conda-forge grad-cam
```
亦可使用 pip
```
pip install grad-cam
```
---
## :memo: Using
### 0. **import**
截至 2022/8/9 所支援的各式 CAM algorithm
```python!
from pytorch_grad_cam import GradCAM, HiResCAM, ScoreCAM, GradCAMPlusPlus, AblationCAM, XGradCAM, EigenCAM, FullGrad
```
用於指定 target class (要輸出哪一類別的 CAM)
```python!
from pytorch_grad_cam.utils.model_targets import ClassifierOutputTarget
```
用於將 CAM 疊在原圖上顯示
```python!
from pytorch_grad_cam.utils.image import show_cam_on_image
```
### 1. **input data**
輸入資料與一般 inference 相同
> 以下使用 torchvision.trnasformer, 前處理方式參照 [ResNet50_Weights.IMAGENET1K_V1](https://pytorch.org/vision/main/models/generated/torchvision.models.resnet50.html#torchvision.models.ResNet50_Weights)
```python
transformer = transforms.Compose([
transforms.ToPILImage(),
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])
data = transformer(plt.imread(image_path))
data = torch.unsqueeze(data, 0) # [c, h, w] to [1, c, h, w]
```
### 2. **選擇 target layer**
CAM的基本理論在於抽出 conv layer 結果(feature map), 並將其疊合得到最終 CAM 的結果.
此處即是選擇要抽出哪些 conv layer. (若 target layer 多於一層, 會將它們平均)
>以下為使用 torchvision.models 中的 resnet50, 並選擇 layer4 的輸出為 target layer
```python
model = models.resnet50(pretrained=True)
target_layers = [model.layer4[-1]]
```
### 3. **選擇 target class**
若不指定目標類別, 預設為輸出最大機率值的類
> 此處指定為 [ImageNet](https://deeplearning.cms.waikato.ac.nz/user-guide/class-maps/IMAGENET/) 第 235 類 (German shepherd dog)
```python
targets = [ClassifierOutputTarget(235)]
```
### 4. **CAM**
可搭配 with 語法來使用, 以確保資源有確實釋放.
> 以下使用 GradCAM, 可視需求替換為 package 中提供的任何 CAM-algo.
輸出的 grayscale_cam 為一個 3d-numpy array [b, h, w].
```python
cam = GradCAM(model=model, target_layers=target_layers, use_cuda=torch.cuda.is_available())
grayscale_cam = cam(input_tensor=data, targets=targets)
```
---
## :computer: Full sample code
```python
import cv2
import matplotlib.pyplot as plt
import torch
from torchvision import models, transforms
from pytorch_grad_cam import GradCAM, ScoreCAM, GradCAMPlusPlus, AblationCAM, XGradCAM, EigenCAM, FullGrad
from pytorch_grad_cam.utils.model_targets import ClassifierOutputTarget
from pytorch_grad_cam.utils.image import show_cam_on_image
image_path = r'dog_cat.jpg'
model = models.resnet50(weights=models.ResNet50_Weights.IMAGENET1K_V1)
target_layers = [model.layer4[-1]]
transformer = transforms.Compose([
transforms.ToPILImage(),
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])
data = transformer(cv2.imread(image_path))
data = torch.unsqueeze(data, 0) # [c, h, w] to [1, c, h, w]
with GradCAM(model=model, target_layers=target_layers, use_cuda=torch.cuda.is_available()) as cam:
targets = [ClassifierOutputTarget(235)]
grayscale_cam = cam(input_tensor=data, targets=targets)
grayscale_cam = grayscale_cam[0, :]
rgb_img = cv2.imread(image_path)/255
grayscale_cam = cv2.resize(grayscale_cam, (rgb_img.shape[1], rgb_img.shape[0]))
visualization = show_cam_on_image(rgb_img, grayscale_cam, use_rgb=True)
plt.imshow(visualization)
plt.show()
```
> [原圖來源](https://www.petbacker.com/blog/how-to/tips-on-how-to-make-a-dog-and-cat-become-friends)
Sign in with Wallet
Connect another wallet