# Multivariate Anomaly Detection Bugbash

[ToC]

## Introduction
> Multivariate Anomaly Detection(MVAD) is a feature in Anomaly Detector service, which will be generally available(GA) in November. MVAD provides the capability to detect anomalies among multiple variables in complex system or equipment.

This bugbash focus on the MVAD GA version, please follow the contents below to prepare the resources and test on the APIs, report feedbacks through links below.
- [x] :warning: [Report an issue](https://microsoft.sharepoint.com/teams/Kensho/_layouts/OneNote.aspx?id=%2Fteams%2FKensho%2FShared%20Documents%2FMultivariate%20AD&wd=target%28GA.one%7CC8B379E6-F47F-45AD-B1A5-2807DCEAC66A%2F%F0%9F%95%B7%EF%B8%8FIssues%7CFB21446D-70A0-49F4-B3E5-E254672D3A63%2F%29onenote:https://microsoft.sharepoint.com/teams/Kensho/Shared%20Documents/Multivariate%20AD/GA.one#🕷️Issues&section-id={C8B379E6-F47F-45AD-B1A5-2807DCEAC66A}&page-id={FB21446D-70A0-49F4-B3E5-E254672D3A63}&end)
- [x] :bulb: [Submit an idea for improvements](https://microsoft.sharepoint.com/teams/Kensho/_layouts/OneNote.aspx?id=%2Fteams%2FKensho%2FShared%20Documents%2FMultivariate%20AD&wd=target%28GA.one%7CC8B379E6-F47F-45AD-B1A5-2807DCEAC66A%2F%F0%9F%91%80Improvements%7CC0622BA4-B6ED-4CC4-9E60-98621E5DE9EB%2F%29onenote:https://microsoft.sharepoint.com/teams/Kensho/Shared%20Documents/Multivariate%20AD/GA.one#👀Improvements&section-id={C8B379E6-F47F-45AD-B1A5-2807DCEAC66A}&page-id={C0622BA4-B6ED-4CC4-9E60-98621E5DE9EB}&end)

:crown: ***Special gifts for the following rewards:***
* ==E2E No.1==
* ==Best Bug Hunter==
* ==Most Valuable Feedback==
* ==Most Unique Bug==

## Step 1: Create an Anomaly Detector

1. [Create an Anomaly Detector](https://ms.portal.azure.com/#create/Microsoft.CognitiveServicesAnomalyDetector), and
* Select **Cognitive Services Anomaly Finder API** as Subscription
* Select **Central US EUAP** as region
* Click **Identiy** and set status as:on: **On** (only *System assigend managed identity* is supported)
* Select **Review + create**

2. Copy the **endpoint** and **key**, paste them somewhere for further steps.
Get your endpoint:
Get your key (both key1 and key2 work):

## Step 2: Create an Storage Account

1. [Create a Storage Account](https://ms.portal.azure.com/#create/Microsoft.StorageAccount-ARM) ,select **Cognitive Services Anomaly Finder API** as Subscription.

2. Go to your Storage Account resource and set access control to let your Anomaly Detector resource fetch data from your Storage Account.
* Go to **Access Control(IAM)**, and select **+ADD** to **Add role assignment**.

* Search role of **Storage Blob Data Reader**, **click on it** and then select **Next**.

* Select assign access to **Managed identity**, and **Select Members**, then choose the anomaly detector resource that you created earlier, then select **Review + assign**.

## Step 3: Prepare your data

### Download sample data
Multivariate Anomaly Detection supports two types of data schema for training and async inference: **`Multiple Tables`** and **`One Table`**.

* Download [One Table sample data](https://mvaddataset.blob.core.windows.net/public-sample-data/sample_data_5_3000.csv)
* Download [Multiple Tables sample data](https://mvaddataset.blob.core.windows.net/public-sample-data/sample_data_5_3000.zip) and unzip it.

### Upload data to Storage Account

Go to your Storage Account, select **Containers** and create a new container.

Select **Upload** to upload the folder *(for multiple tables)* or the CSV file *(for one table*) that you downloaded earlier.

Click on the data that you uploaded and copy the **Blob URL** to somewhere for further steps.

## Step 4: Test

### API Overview
There are 7 APIs provided in Multivariate Anomaly Deteciton:
* **Training**: Use `Train Model API` to create and train a model, then use `Get Model Status API` to get the status and model metadata.
* **Inference**:
* Use `Async Inference API` to trigger an asynchronous inference process and use `Get Inference results API` to get detection results on a batch of data.
* You could also use `Sync Inference API` to trigger a detection on one timestamp every time.
* **Other operations**: `List Model API` and `Delete Model API` are supported in MVAD for model management.

|API Name| Method | Path | Description |
| ------ | ---- | ----------- | ------ |
|**Train Model**| POST | `{endpoint}`/anomalydetector/v1.1/multivariate/models | Create and train a model |
|**Get Model Status**| GET | `{endpoint}`anomalydetector/v1.1/multivariate/models/`{modelId}` | Get model status and model metadata with `modelId` |
|**Async Inference**| POST | `{endpoint}`/anomalydetector/v1.1/multivariate/models/`{modelId}`:detect-batch | Trigger an asynchronous inference with `modelId` |
|**Get Inference Results**| GET | `{endpoint}`/anomalydetector/v1.1/multivariate/detect-batch/`{resultId}` | Get asynchronous inference resulsts with `resultId` |
|**Sync Inference**| POST | `{endpoint}`/anomalydetector/v1.1/multivariate/models/`{modelId}`:detect-last | Trigger a synchronous inference with `modelId` |
|**List Model**| GET | `{endpoint}`/anomalydetector/v1.1/multivariate/models | List all models |
|**Delete Model**| DELET | `{endpoint}`/anomalydetector/v1.1/multivariate/models/`{modelId}` | Delete model with `modelId` |

### Option 1: Test in Python

- [x] [Fork this repo](https://github.com/jr-MS/mvad-ga-bugbash) and run the notebook, remember to replace Anomaly Detector `endpoint`, `key` and dataSource `url` with your own resources.

### Option 2: Test in Postman
Click this button to fork the API collection in Postman and follow the steps below to test.
[](https://app.getpostman.com/run-collection/18763802-b90da6d8-0f98-4200-976f-546342abcade?action=collection%2Ffork&collection-url=entityId%3D18763802-b90da6d8-0f98-4200-976f-546342abcade%26entityType%3Dcollection%26workspaceId%3De1370b45-5076-4885-884f-e9a97136ddbc#?env%5BMVAD%5D=W3sia2V5IjoibW9kZWxJZCIsInZhbHVlIjoiIiwiZW5hYmxlZCI6dHJ1ZSwidHlwZSI6ImRlZmF1bHQiLCJzZXNzaW9uVmFsdWUiOiJlNjQxZTJlYy01Mzg5LTExZWQtYTkyMC01MjcyNGM4YTZkZmEiLCJzZXNzaW9uSW5kZXgiOjB9LHsia2V5IjoicmVzdWx0SWQiLCJ2YWx1ZSI6IiIsImVuYWJsZWQiOnRydWUsInR5cGUiOiJkZWZhdWx0Iiwic2Vzc2lvblZhbHVlIjoiOGZkZTAwNDItNTM4YS0xMWVkLTlhNDEtMGUxMGNkOTEwZmZhIiwic2Vzc2lvbkluZGV4IjoxfSx7ImtleSI6Ik9jcC1BcGltLVN1YnNjcmlwdGlvbi1LZXkiLCJ2YWx1ZSI6IiIsImVuYWJsZWQiOnRydWUsInR5cGUiOiJzZWNyZXQiLCJzZXNzaW9uVmFsdWUiOiJjNzNjMGRhMzlhOTA0MjgzODA4ZjBmY2E0Zjc3MTFkOCIsInNlc3Npb25JbmRleCI6Mn0seyJrZXkiOiJlbmRwb2ludCIsInZhbHVlIjoiIiwiZW5hYmxlZCI6dHJ1ZSwidHlwZSI6ImRlZmF1bHQiLCJzZXNzaW9uVmFsdWUiOiJodHRwczovL211bHRpLWFkLXRlc3QtdXNjeC5jb2duaXRpdmVzZXJ2aWNlcy5henVyZS5jb20vIiwic2Vzc2lvbkluZGV4IjozfSx7ImtleSI6ImRhdGFTb3VyY2UiLCJ2YWx1ZSI6IiIsImVuYWJsZWQiOnRydWUsInR5cGUiOiJkZWZhdWx0Iiwic2Vzc2lvblZhbHVlIjoiaHR0cHM6Ly9tdmFkZGF0YXNldC5ibG9iLmNvcmUud2luZG93cy5uZXQvc2FtcGxlLW9uZXRhYmxlL3NhbXBsZV9kYXRhXzVfMzAwMC5jc3YiLCJzZXNzaW9uSW5kZXgiOjR9XQ==)

1. Select environment as **MVAD**.

1. Select **Environment**, paste your Anomaly Detector `endpoint`, `key` and dataSource `url` in to the **CURRENT VALUE** column, click **Save** to let the variables take effect.

2. Select **Collections**, and click on the first API - **Create and train a model**, then click **Send**.
:::info
:bulb: **Note:** If your data is one CSV file, please set the dataSchema as **OneTable**, if your data is multiple CSV files in a folder, please set the dataSchema as **MultiTable.**
:::


3. In the response of the first API, copy the modelId and paste it in the `modelId` in **Environments**, click **Save**. Then go to **Collections**, click on the second API - **Get model status**, and click **Send**.

4. Select the third API - **Batch Detection**, and click **Send**. This API will trigger a synchronous inference task, and you should use the Get batch detection results API several times to get the status and the final results.

5. In the response of the third API, copy the resultId and paste it in the `resultId` in **Environments**, click **Save**. Then go to **Collections**, click on the fourth API - Get batch detection results, and click **Send**.

6. For the rest of the APIs, click on each and click Send to test on their request and response.