# CheXnet With NIH Report

## Data

| | Image Index | Follow-up # | Patient ID | Patient Age | Patient Gender | View Position | Cardiomegaly | Emphysema | Effusion | Hernia | ... | Mass | Nodule | Atelectasis | Pneumothorax | Pleural_Thickening | Pneumonia | Fibrosis | Edema | Consolidation | fold |
| --- | ---------------- | ----------- | ---------- | ----------- | -------------- | ------------- | ------------ | --------- | -------- | ------ | --- | ---- | ------ | ----------- | ------------ | ------------------ | --------- | -------- | ----- | ------------- | ----- |
| 1 | 00000001_000.png | 0 | 1 | 058Y | M | PA | 1.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | train |
| 2 | 00000001_001.png | 1 | 1 | 058Y | M | PA | 1.0 | 1.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | train |
| 3 | 00000002_000.png | 0 | 2 | 081Y | M | PA | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | train |
| 4 | 00000003_000.png | 0 | 3 | 081Y | F | PA | 0.0 | 0.0 | 0.0 | 1.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | train |
| 5 | 00000003_001.png | 1 | 3 | 074Y | F | PA | 0.0 | 0.0 | 0.0 | 1.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | train |
| 6 | 00000003_002.png | 2 | 3 | 075Y | F | PA | 0.0 | 0.0 | 0.0 | 1.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | train |
| 7 | 00000003_003.png | 3 | 3 | 076Y | F | PA | 0.0 | 0.0 | 0.0 | 1.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | train |
| 8 | 00000003_004.png | 4 | 3 | 077Y | F | PA | 0.0 | 0.0 | 0.0 | 1.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | train |
| 9 | 00000003_005.png | 5 | 3 | 078Y | F | PA | 0.0 | 0.0 | 0.0 | 1.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | train |
| 10 | 00000003_006.png | 6 | 3 | 079Y | F | PA | 0.0 | 0.0 | 0.0 | 1.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | trai |
| ... | ... | ... | ... | .. | .. | .. | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |

### All Data:
| columns | num | | columns | num |
| ------------ | ------- | --- | ------------------ | ------- |
| Cardiomegaly | 2772.0 | | Atelectasis | 11535.0 |
| Emphysema | 2516.0 | | Pneumothorax | 5298.0 |
| Effusion | 13307.0 | | Pleural_Thickening | 3385.0 |
| Hernia | 227.0 | | Pneumonia | 1353.0 |
| Infiltration | 19870.0 | | Fibrosis | 1686.0 |
| Mass | 5746.0 | | Edema | 2303.0 |
| Nodule | 6323.0 | | Consolidation | 4667.0 |

|  |  |
| ------------------------------------ | ------------------------------------ |
### Training Data:

| columns | num | | columns | num |
| ------------ | ------- | --- | ------------------ | ------ |
| Cardiomegaly | 1950.0 | | Atelectasis | 7996.0 |
| Emphysema | 1799.0 | | Pneumothorax | 3705.0 |
| Effusion | 9261.0 | | Pleural_Thickening | 2279.0 |
| Hernia | 144.0 | | Pneumonia | 978.0 |
| Infiltration | 13914.0 | | Fibrosis | 1158.0 |
| Mass | 3988.0 | | Edema | 1690.0 |
| Nodule | 4375.0 | | Consolidation | 3263.0 |

|  |  |
| ------------------------------------ | ------------------------------------ |

### Testing Data:

| columns | num | | columns | num |
| ------------ | ------ | --- | ------------------ | ------ |
| Cardiomegaly | 582.0 | | Atelectasis | 2420.0 |
| Emphysema | 509.0 | | Pneumothorax | 1089.0 |
| Effusion | 2754.0 | | Pleural_Thickening | 734.0 |
| Hernia | 42.0 | | Pneumonia | 242.0 |
| Infiltration | 3938.0 | | Fibrosis | 362.0 |
| Mass | 1133.0 | | Edema | 413.0 |
| Nodule | 1335.0 | | Consolidation | 957.0 |

|  |  |
| ------------------------------------ | ------------------------------------ |

### Summary


## Models & Result

### Pneumoni

| Model | ROC |
| --------------------------------------- | ------ |
| Dence121+resnest50+inceptionv4+xceptino | 0.7764 |
| ==CheXnet==(Base Model) | 0.7700 |
| ResNeSt50 | 0.7599 |
| Dence201 | 0.7598 |
| Dence169 | 0.7570 |
| Inceptionv4 | 0.7550 |
| xception | 0.7550 |
| senet154 | 0.7470 |
| Dence101 | 0.7430 |
| Pnasnet | 0.7290 |
| Nasnet | 0.6590 |

|  |  |
| ------------------------------------ | ------------------------------------ |

### Infiltration

| Model | ROC |
| ----------------------- | ----- |
| Dence121 + Attention | 0.855 |
| ==CheXnet==(Base Model) | 0.711 |
| senet154 | 0.704 |
| Pnasnet | 0.672 |
| Nasnet | 0.64 |

|  |  |
| ------------------------------------ | ------------------------------------ |

# CheXnet With KMSH_Announce

## Data

### Info
* Total: 6391
* True Positive: 4334
* True Negative: 2057

#### Confidence Of Each XML
* Red: True Negative
* Blue: True Positive
* X: Confidence of CheXnet with 14 category
* Y: Frequency

|  |  |
| ------------------------------------ | --- |
|  |  |
### Data Preprocess
#### Two Category
1. 引入 NIH 資料來充當 True Negative，將資料製作成 1:1
* 使用 NIH 資料內**無症狀**作為 True Negative 資料，共引入 2297 筆
2. 針對4位醫生所標記的資料，分別 8/2 分當作 train/test
3. 最後分別從 Train & Test 抽取 240 筆資料，共 480 筆做為 validation
#### Seven Category

資料格式：{path, Pneumonia, Normal}

## Result Of CheXnet With 2 Category

### Best Method

* Model: DenseNet 121
* KMSH Only
* Data Preprocess: Without normalize

**Threash hold:** 0.5

| label | auc | precision | recall | f1 | specificity | sensitive |
| --------- | -------- | --------- | -------- | -------- | ----------- | --------- |
| Normal | 0.888744 | 0.691998 | 0.752552 | 0.721006 | 0.841024 | 0.752552 |
| Pneumonia |0.888953 | 0.878430 | 0.841947 | 0.859802 | 0.754496 | 0.841947 |

**Threash hold:** 0.6

| label | auc | precision | recall | f1 | specificity | sensitive |
| --------- | -------- | --------- | -------- | -------- | ----------- | --------- |
| Normal | 0.888744 | 0.749700 | 0.608653 | 0.671854 | 0.903553 | 0.608653 |
| Pneumonia | 0.888953 | 0.916233 | 0.772265 | 0.838111 | 0.851239 | 0.772265 |

**Threash hold:** 0.7

| label | auc | precision | recall | f1 | specificity | sensitive |
| --------- | -------- | --------- | -------- | -------- | ----------- | --------- |
| Normal | 0.888744 | 0.799056 | 0.411764 | 0.543347 | 0.950853 | 0.411764 |
| Pneumonia | 0.888953 | 0.942295 | 0.697046 | 0.904267 | 0.910063 | 0.697046 |

**New Test Data - 2cat**

| label | auc | precision | recall | f1 | specificity | sensitive |
| --------- | ------ | --------- | -------- | -------- | ----------- | --------- |
| Normal | 0.9311 | 0.3333 | 0.15 | 0.543347 | 0.950853 | 0.411764 |
| Pneumonia | 0.932 | 0.9721 | 0.99 | 0.904267 | 0.910063 | 0.697046 |

**New Test Data - 7cat**

| label | auc | precision | recall | f1 | specificity | sensitive |
| --------- | ------ | --------- | -------- | -------- | ----------- | --------- |
| Normal | 0.9311 | 0.3333 | 0.15 | 0.543347 | 0.950853 | 0.411764 |
| Pneumonia | 0.932 | 0.9721 | 0.99 | 0.904267 | 0.910063 | 0.697046 |

| KMSH Data Befor Retrain | KMSH Data After Retrain |
| ------------------------------------ | ------------------------------------ |
|  |  |

## Result Of CheXnet With 7 Category
Train with selecting data like below:

['path', 'Effusion', 'Infiltration', 'Mass', 'Nodule', 'Atelectasis', 'Pneumonia', 'fold']

### Data
* Train / Test / Val

| columns | numbers | | columns | numbers | | columns | numbers |
| ------------ | ------- | --- | ------------ | ------- | --- | ------------ | ------- |
| Effusion | 8420.0 | | Effusion | 792.0 | | Effusion | 250.0 |
| Infiltration | 10403.0 | | Infiltration | 900.0 | | Infiltration | 250.0 |
| Mass | 5730.0 | | Mass | 428.0 | | Mass | 250.0 |
| Nodule | 6197.0 | | Nodule | 542.0 | | Nodule | 250.0 |
| Atelectasis | 7544.0 | | Atelectasis | 843.0 | | Atelectasis | 250.0 |
| Pneumonia | 1753.0 | | Pneumonia | 3694.0 | | Pneumonia | 240.0 |
| Normal | 5000.0 | | Normal | 900.0 | | Normal | 490.0 |


* Without KMSH un-labeling data

| label | auc | precision | recall | F1 |
| ------------ | -------- | --------- | -------- | -------- |
| Atelectasis | 0.899245 | 0.559194 | 0.526690 | 0.542456 |
| Effusion | 0.926817 | 0.555188 | 0.635101 | 0.592462 |
| Infiltration | 0.875477 | 0.455135 | 0.467778 | 0.461370 |
| Mass | 0.891548 | 0.474699 | 0.460280 | 0.467378 |
| Nodule | 0.889967 | 0.446429 | 0.369004 | 0.404040 |
| Normal | 0.841876 | 0.590062 | 0.211111 | 0.310966 |
| Pneumonia | 0.999947 | 0.999444 | 0.973741 | 0.986425 |

* With KMSH un-labeling data

| label | auc | precision | recall | F1 |
| ------------ | -------- | --------- | -------- | -------- |
| Atelectasis | 0.881804 | 0.456790 | 0.482800 | 0.949122 |
| Effusion | 0.913841 | 0.683511 | 0.324495 | 0.987558 |
| Infiltration | 0.888068 | 0.332097 | 0.696667 | 0.866646 |
| Mass | 0.867386 | 0.413249 | 0.306075 | 0.981265 |
| Nodule | 0.877275 | 0.417241 | 0.223247 | 0.982780 |
| Normal | 0.562983 | 0.500000 | 0.004058 | 0.998378 |
| Pneumonia | 0.857607 | 0.684382 | 0.912686 | 0.782216 |

| label | auc | precision | recall | F1 |
| ------------ | -------- | --------- | -------- | -------- |
| Atelectasis | 0.881804 | 0.456790 | 0.482800 | 0.949122 |
| Effusion | 0.913841 | 0.683511 | 0.324495 | 0.987558 |
| Infiltration | 0.888068 | 0.332097 | 0.696667 | 0.866646 |
| Mass | 0.867386 | 0.413249 | 0.306075 | 0.981265 |
| Nodule | 0.877275 | 0.417241 | 0.223247 | 0.982780 |
| Pneumonia | 0.857607 | 0.684382 | 0.912686 | 0.782216 |

:::warning
NIH Data & KMSH Data 會有衝突
Model 會以 KMSH & NIH Data 來做分類，而非真的去找尋 Pneumonia 症狀
因此在不加入 KMSH Un-labeling Data 之前 AUC 可以達到 99，但加入 Un-labeling Data 混淆 Model 後，Auc 下降到 0.85
此部分還待解決，正在找尋是哪個部份引響到整個 model
:::

###### tags: `Media-ai`, `result`