---
# System prepended metadata

title: rc2021-ICJAISV74 Note 2
tags: [mlrc2021]

---

# rc2021-ICJAISV74 Note 2

###### tags: `mlrc2021`

## Roadmap

1. Run RS & US results and compare with [Zhan2021]().
2. Contact with *Zhan* again. Prepare the original process and `sklearn.StandardScaler` (Suggested by *Zhan*).
    - Ask for RS & US query history. Check the AUBC is different or not.
    - How do you run `alipy.BMDR` and `alipy.LAL` for multi-class classification problems. (Comment the `if len(ul) != 2: ...` condition?)
    - Ask for several query stratigies AUBC(acc) results.
    - Ask for the code, if possible.
    - Which `method` of `libact.US-1`? `{'lc', 'sm', 'entropy'}, (default='lc')`? Or other detail in parameters?
    - Detail of using multiple base model for BSO (Suggested by *Zhan*).
    - What is *saturated performance* of datasets in *page 4682*.
    - What is *BSO results on some datasets are not as large as one might expect* in *page 4682*? 
      How can we evaluate it?
    - More detail on base model in QBC.
3. Share my experience.
    - Run multicore (Pool) of QBC and QUIRE.
    - 

## Supplementary

1. Ask for RS & US query history

```python=
(Pdb) p results["indices"][:90]  # iris
array([106,  91,  28, 117,  13, 121, 128,  35,   7,  45, 123, 102, 137,
       142,  27,  22, 125,  92, 129,  34,  23,  29,  15,   6,  33, 140,
        52,   5,  58,  98,  41,  99, 109, 149,  65,  17,  96,  84, 130,
         9,  57,   1,  42,  61,  26,  66,  95,  87,  70,  82, 116,  93,
        32,  56, 143,   2,  10, 131,  31,  20,  21, 135,  12, 108,   0,
        64,  48,  39, 132, 145,  85,  18,  89,  50,  16,  51,  80, 111,
       107,  38,  81,  53,  79,  24, 126,  73, 100,   4,  97, 115])
```

2. How do you run `alipy.BMDR`

```python=
# alipy/query_strategy/query_labels.py#L1089
1089        if len(ul) != 2:
1090            raise ValueError(f"This query strategy is implemented for binary classification only, "
1091                             f"but {len(ul)} classes are detected.")
```

3. Ask for several query stratigies AUBC(acc) results.

See EXCEL---TABLE3.

4. Ask for the code, if possible.

Would you mind to send the partial of example code of RS or US?
I may try to find some difference between us.

5. Detail of using multiple base model for BSO (Suggested by *Zhan*).

    - Where should I train with `SVC(), LinearSVC(), LogisticRegression(), GaussianProcessClassifier()`?
    - What is your `sklearn.__version__`?

## Research Log

I list the results pass test or not of random sampling (RS) in *TABLE 3*.

| dataset      | [Zhan2021] | No preprocessing | sklearn.StandardScaler | sklearn.Normalizer        |
| ------------ | ---------- | ---------------- | ---------------------- | ------------------------- |
| appendicitis | 0.836      | No 0.79          | Yes 0.838              | No 0.786                  |
| sonar        | 0.617      | No 0.67          | No 0.755               | No 0.51                   |
| iris         | 0.835      | No 0.93          | No 0.924               | Yes(?) 0.845              |
| wine         | 0.858      | No 0.95          | No 0.960               | Yes(?) 0.870              |
| parkinsons   | 0.840      | Yes 0.84         | Yes 0.837              | No 0.75                   |
| ex8b         | 0.866      | No 0.835         | No 0.885               | No 0.845                  |
| seeds        | 0.862      | No 0.92          | No 0.907               | No 0.88                   |
| glass        | 0.387      | No 0.48          | No 0.611               | Yes 0.395                 |
| thyroid      | 0.696      | No 0.71          | No 0.919               | Yes 0.695                 |
| heart        | 0.808      | Yes 0.80         | Yes 0.806              | No 0.785                  |
| haberman     | 0.727      | Yes 0.72         | Yes 0.731              | Yes 0.731                 |
| ionosphere   | 0.901      | No 0.87          | No 0.918               | No 0.657                  |
| clean1       | 0.649      | No 0.580         | No 0.819               | No 0.554                  |
| breast       | 0.954      | No 0.959         | Yes 0.956              | No 0.942                  |
| wdbc         | 0.952      | Yes(?) 0.955     | Yes 0.954              | No 0.932                  |
| r15          | 0.877      | No 0.976         | No 0.975               | No 0.437                  |
| australian   | 0.846      | Yes(?) 0.851     | Yes 0.848              | No 0.838                  |
| diabetes     | 0.736      | Yes(?) 0.740     | Yes(?) 0.741           | No 0.706                  |
| mammographic | 0.819      | No 0.772         | No 0.809               | No 0.503                  |
| ex8a         | 0.838      | No 0.696         | No 0.855               | No 0.61                   |
| vehicle      | 0.567      | Yes(?) 0.563     | No 0.691               | No 0.38                   |
| tic-tac-toe  | 0.87       | Yes 0.87         | Yes 0.87               | Yes 0.87                  |
| german       | 0.726      | Yes(?) 0.73      | No 0.732               | No 0.699                  |
| splice       | 0.806      | No 0.793         | Yes 0.807              | No 0.542                  |
| gcloudb      | 0.893      | No 0.882         | Yes(?) 0.895           | No 0.769                  |
| gcloudub     | 0.942      | No 0.928         | Yes(?) 0.944           | No 0.886                  |
| checkerboard | 0.978      | No 0.836         | Yes(?) 0.977           | No 0.988                  |
| phishing     | 0.926      | No 0.934         | No 0.938               | linux15-13/clais-noGPU-50 |
| d31          | 0.582      | No 0.954         | No 0.955               | No 0.519                  |
| spambase     | 0.685      | No 0.914         | No 0.914               | No 0.878                  |
| banana       | 0.895      | No(?) 0.893      | No(?) 0.893            | No 0.738                  |
| phoneme      | 0.822      | Yes(?) 0.821     | Yes(?) 0.821           | No 0.79                   |
| texture      | 0.666      | No 0.966         | No 0.966               | No 0.716                  |
| ringnorm     | 0.976      | Yes(?) 0.976     | Yes(?) 0.976           | linux15-13                |
| twonorm      | 0.976      | No(?) 0.975      | No(?) 0.975            | linux15-13/linux5-20      |

I also list the results pass test or not of uncertainty sampling (US) in *TABLE 3*/*TABLE 5* in [Zhan2020]().

| dataset      | [Zhan2020] | No preprocessing | sklearn.StandardScaler | sklearn.Normalizer |
| ------------ | ---------- | ---------------- | ---------------------- | ------------------ |
| appendicitis | 0.845      | No 0.786         | Yes 0.839              |                    |
| sonar        | 0.76       | No 0.698         | No 0.775               |                    |
| iris         | 0.87       | No 0.928         | No 0.937               |                    |
| wine         | 0.958      | No 0.94          | No 0.966               |                    |
| parkinsons   | #N/A       | 0.854            | 0.854                  |                    |
| ex8b         | 0.892      | No 0.846         | Yes(?) 0.899           |                    |
| seeds        | #N/A       | 0.913            | 0.913                  |                    |
| glass        | #N/A       | 0.491            | 0.612                  |                    |
| thyroid      | 0.702      | No 0.73          | No 0.936               |                    |
| heart        | 0.765      | No 0.814         | No 0.816               |                    |
| haberman     | 0.725      | Yes 0.723        | Yes 0.729              |                    |
| ionosphere   | 0.914      | No 0.878         | No 0.927               |                    |
| clean1       | 0.826      | No 0.624         | No 0.839               |                    |
| breast       | 0.955      | No 0.961         | Yes(?) 0.957           |                    |
| wdbc         | #N/A       | 0.965            | 0.965                  |                    |
| r15          | #N/A       | 0.95             | 0.95                   |                    |
| australian   | 0.846      | Yes(?) 0.851     | Yes(?) 0.85            |                    |
| diabetes     | 0.744      | No 0.754         | Yes 0.745              |                    |
| mammographic | #N/A       | 0.779            | 0.814                  |                    |
| ex8a         | 0.85       | No 0.711         | No 0.879               |                    |
| vehicle      | 0.539      | No 0.559         | No 0.696               |                    |
| tic-tac-toe  | #N/A       | 0.871            | 0.87                   |                    |
| german       | 0.742      | No 0.737         | Yes(?) 0.738           |                    |
| splice       | 0.809      | No 0.803         | No 0.82                |                    |
| gcloudb      | 0.887      | Yes 0.886        | No 0.898               |                    |
| gcloudub     | 0.951      | No 0.936         | Yes(?) 0.953           |                    |
| checkerboard | 0.91       | No 0.85          | No 0.984               |                    |
| Phishing     | #N/A       |                  |                        |                    |
| d31          | #N/A       | cl7-25           | linux7-20              |                    |
| spambase     | #N/A       | cl7-25           | linux7-20              |                    |
| banana       | #N/A       | cl7-25           | linux7-20              |                    |
| phoneme      | #N/A       |                  |                        |                    |
| texture      | #N/A       |                  |                        |                    |
| ringnorm     | #N/A       |                  |                        |                    |
| twonorm      | #N/A       |                  |                        |                    |

Example. **Yes** *appendicitis* `sklearn.StandardScaler`
![](https://i.imgur.com/aMjkfxp.png)

Example. **No** *seeds*
![](https://i.imgur.com/T6ew5Ni.png)

Example. **Yes(?)** *wine* `sklearn.Normalizer`
![](https://i.imgur.com/6b2erjX.png)

Example. **No(?)** *twonorm* `sklearn.StandardScaler`
![](https://i.imgur.com/F8bMFYL.png)
