---
# System prepended metadata

title: Labeling Tool API Document

---


Labeling Tool API Document
===

## User Management

### Login
- request
  + `POST /login`
  + payload
```
{
	
  "email": "peaceful.he@gmail.com",
  "password": "password1"
}
```
- response
```
{
    "token": "a0f571627a2540e5b8ad8705b54839d7"
}
```

### SignUp
- request
  + `POST /signup`
  + payload
```
{
  "email": "peaceful.he@gmail.com",
  "name": "Jing",
  "password": "password1"
}
```

- response
```
{
    "token": "a0f571627a2540e5b8ad8705b54839d7"
}
```

### User Profile
- request
  - `POST /profile`
  - payload
```
{
  "token": "89f48dfc619d44f2806ac0c49d28296c"
}
```

- response
```
{
    "campaigns": List[CampaignObject],
    "name": "Jing",
    "daily_progress": List[ProgressObject]
}
```
## Data Management
### Leaderboard
- request
  - `POST /leaderboard`
  - payload
```
{
  "token": "89f48dfc619d44f2806ac0c49d28296c"
}
```

- response
```
{
  "categories": List[CategoryStatsObject]
  "n50": 118,
  "n70": 71,
  "n85": 23
}
```

### Create a Campaign
- request
  - `POST /start_campaign`
  - payload
```
{
  "token": "a0f571627a2540e5b8ad8705b54839d7",
  "categories": ["GOOD FOR DRY SKIN"]
}
```

- response: a `CampaignObject`


### Modify/Display a Campaign
- request
  - `POST /campaign/<campaign_id>`
  - payload
```
{
  "token": "89f48dfc619d44f2806ac0c49d28296c",
  "update_category_status": [
     {
        "category": "GOOD FOR DRY SKIN",
        "status": "active"
     }
  ]
}
```
  - note: status can be `active` or `deactive`

- response: a `CampaignObject`

### Stop a Campaign
- request
  - `POST /stop_campaign`
  - payload
```
{
  "token": "89f48dfc619d44f2806ac0c49d28296c",
  "campaign_id": 13
}
```

- response: a `CampaignObject`


### Create a Battle
- request
  - `POST /start_battle`
  - payload
```
{
  "campaign_id": 7,
  "token": "a0f571627a2540e5b8ad8705b54839d7",
  "number_of_questions_per_page": _INT_,
  "number_of_examples_per_page": _INT_,
  "size": _INT_,
}
```
  - `number_of_questions_per_page`: for one example, the prediction can be multiple possible categories, this option defines how many questions are displayed for an example; default value is `1`
  - `number_of_examples_per_page`: default value is 1; If this value is set as greater than 1, then one page will show one question with this number of examples;
  - `size`: the maximal number of examples for each category
  - note: only allow one of `number_of_questions_per_page` or `number_of_examples_per_page` be greater than 1

- response: a `BattleObject`


### Disaplay a Battle with Example(s)
- request
  - `POST /battle/<battle_id>`
  - payload
```
{
  "token": "89f48dfc619d44f2806ac0c49d28296c"
}
```

- response: a `BattleObject`

### Submit a Data Example with Labels
- request
  - `POST /submit`
  - payload
```
{
  "token": "89f48dfc619d44f2806ac0c49d28296c",
  "battle_id": 11,
  "examples": [
    {
      "example_id": "very dry skin, yet this powderfoundation applies smoothly",
      "question_answers": [
        {"question": "GOOD FOR DRY SKIN", "answer": 1}
        ]
      
    }
    ]
}
```

- response
```
{
  "token": "a0f571627a2540e5b8ad8705b54839d7",
  "battle_id": 11,
  "examples": [
    {
      "example_id": "very dry skin, yet this powderfoundation applies smoothly",
      "question_answers": [
        {"question": "GOOD FOR DRY SKIN", "answer": 1}
        ]
      
    }
    ]
}
```

### Search Data Examples
- request
  - `POST /example/search`
  - payload
```
{
  "token": "89f48dfc619d44f2806ac0c49d28296c",
  "query": {
      "keywords": {
         "query": _QUERY_STRING_
      },
      "regex": {
        "qury": _REGEX_STRING_
      },
      "categories": {
        "list": _CATEGORY_LIST_,
        "is_negation": _TRUE_OF_FALSE
      }
  },
  "size": _SIZE_INT_,
  "start": _START_INT_,
  "campaign_id": _CAMPAIGN_ID_, # optional
  "battle_id": _BATTLE_ID_, # optional
  "scope": _SCOPE_LIST_
}
```
  - possible scope values can be: `labeled`, `unlabeled`

- response
```
{
  "examples": _EXAMPLE_OBJECT_LIST_
}
```

### Import Data Examples
- request
  - `POST /example/import`
 

- response
```
{
  "status": _STATUS_STR_,
  "number_of_examples": _INT_,
  "errors": [
     {
       "sentence": _SENTENCT_TEXT_,
       "categories": LIST[str],
       "reason": _REASON_STR_
     }
  ]
}
```
### Update Data Example Labels
- request
  - `POST /example/update`
  - payload
```
{
  "token": "89f48dfc619d44f2806ac0c49d28296c",
  "updates": [
     {
        "example_id": _EXAMPLE_ID_,   
        "question_answers": [
          {"question": "GOOD FOR DRY SKIN", "answer": 1}
        ],
        "update_mode": _UPDATE_MODE_
     },
     ...
  ]
}
```
  - `update_mode` can be `overwrite|append`

- response
```
{
  "token": "a0f571627a2540e5b8ad8705b54839d7",
  "examples": _UPDATED_EXAMPLE_OBJECT_LIST_
}
```


### Category Autocomplete
- request
  - `POST /category/autocomplete`
  - payload
```
{
  "token": "89f48dfc619d44f2806ac0c49d28296c",
  "prefix": "a"
}
```

- response
```
{
  "categories": [
     "a little goes a long way",
     "a lot of smells"
  ]
}
```

## Model Management
### Create a Model
- request
  - `POST /start_train`
  - payload
```
{
  "token": "89f48dfc619d44f2806ac0c49d28296c"
}
```

- response: a `ModelObject`

### List models
- request
  - `POST /model`
  - payload
```
{
  "token": "89f48dfc619d44f2806ac0c49d28296c",
  "ids": List[str]
}
```
  - `ids`: it's optional. If it is not provided, then all model information are displayed

- response:
```
{
   "models": List[Model_Object]
}
```

### Run a Model
- request
  - `POST /model/predict`
  - payload
```
{
  "token": "89f48dfc619d44f2806ac0c49d28296c",
  "model_id": _MODEL_ID_STR_,
  "inputs": [
      {
         "text": _TEXT_STR_
      },
      ...
  ]
}
```

- response:
```
{
  "predictions": [
      {
         "text": _TEXT_STR_,
         "categories": List[str],
         "category_scores": [
           {"category": _CATEGORY_NAME_STR_, "score": _SCORE_FLOAT_},
           ...
         ]
      },
      ...
  ]
```

### Model Prediction Results on Datasets
- request
  - `POST /model/predict/{data_type}`
  - payload
```
{
  "token": "89f48dfc619d44f2806ac0c49d28296c",
  "model_id": _MODEL_ID_STR_
}
```
  - `data_type` can be `training` or `validation`

- response: a tsv file including `text`, `labels`, `predictions` fields


### Download a Model (TBD)



## Objects
### CampaignObject
```
{
            "campaign_id": 1,
            "categories": [
                "SOFT BRISTLE",
                "SOFTENS HAIR",
                "SOOTHING",
                "TRAVEL FRIENDLY",
                "TRULY WORKS"
            ],
            "editor_id": "peaceful.he@gmail.com",
            "start_time": "20200813_180318",
            "status": "in progress"
}
```

### CategoryStatsObject
```
{
            "active_campaign_ids": [
            ],
            "category": "HEAT PROTECTANT",
            "index": 155,
            "n": 8,
            "p": 0.625,
            "total": 8,
            "n_availability": 50,
            "status": "active"  # optional, value can be `active` or `deactive`
}
```


### CampaignObject
```
{
    "battles": [
    ],
    "campaign_id": 7,
    "categories": List[CategoryStatsObject],
    "editor_id": "peaceful.he@gmail.com",
    "n50": 1,
    "n70": 1,
    "n85": 0,
    "start_time": "20200819_183902"
}
```

### BattleObject
```
{
    "battle_id": 14,
    "campaign_id": 7,
    "categories": [
      "GOOD FOR DRY SKIN"
    ],
    "editor_id": "peaceful.he@gmail.com",
    "examples": List[ExampleObject],
    "headers": [
        "index",
        "topic",
        "p",
        "n",
        "total"
    ],
    "n50": 0,
    "n70": 0,
    "n85": 0,
    "start_time": "20200826_154957",
    "status": {
        "number_labeled": 0,
        "number_of_category": 1,
        "number_to_label": 8
    }
}

```

### ExampleObject
```
{
            "example_id": "This product is very good for dry skin",
            "options": {
                "correct": 1,
                "incorrect": 0
            },
            "questions": [
                "GOOD FOR DRY SKIN",
                "GOOD FOR OILY SKIN",
                "UNKNOWN"
            ],
            "sentence": "This product is very good for dry skin",
            "images": [{
               "url": _IMAGE_URL_STR_
            }],
            
        }
    ],
}
```

### ModelObject
```
{
   "model_id": _MODEL_ID_STR_,
   "creator_id": _USER_EMAIL_STR_,
   "status": _STATUS_STR_,
   "start_training_time": _DATE_TIME_STR_,
   "finish_training_time": _DATE_TIME_STR_,
   "training_data_profile": [
     {"category": _STR_, "support": _STR_},
     ...
   ],
   "evaluation_result": {
     "micro_p": _FLOAT_,
     "micro_r": _FLOAT_,
     "micro_f1": _FLOAT_,
     "micro_accuracy": _FLOAT_,
     "macro_p": _FLOAT_,
     "macro_r": _FLOAT_,
     "macro_f1": _FLOAT_,
     "macro_accuracy": _FLOAT_,
     "category_result": [
       {"category": _STR_, "p": _FLOAT_, "r": _FLOAT, "f1": _FLOAT_, "support": _INT_},
       ...
     ]
   }
}
```
- status: can be "IN_PROGESS", "SUCCEEDED", "FAILED"

### ProgressObject
```
{
  "editor_id": _EDITOR_ID_STR_,
  "from": _INCLUSIVE_FROM_DATE_YYYY_MM_DD_STR,
  "to": _INCLUSIVE_TO_DATE_YYYY_MM_DD_STR,
  "number_of_labels": _INT_
}
```
