# AMD 比賽 demo system 用法
###### tags: `AMD`
## To Execute Kai's Demo
1. Open three terminals
2. In each terminal, `source .bashrc` and activate the environment via `kai_1`
3. In each terminal, `cd /home/aac/kai/repository/ADAR-MED`
4. In terminal 1, type in
`python3 -m fastchat.serve.controller --host 127.0.0.1`
6. In terminal 2, type in
`python3 -m fastchat.serve.vllm_worker --model-path /home/aac/Ryan/medalpaca --host 127.0.0.1`
This will take a while since we will load a model in this step. When it's done, go step 6.
8. In terminal 3, type in
`python3 -m fastchat.serve.med_chabot_web_server --host 127.0.0.1 --share`

## To Develop

1. Install vllm
Refer to this [website](https://hackmd.io/@unj0M9DkQhqZGOyd71BT5g/HkFNSQEHR).
2. Install fastchat
- You can copy from `/home/aac/kai/repository/FastChat` and then install the packages.

        pip3 install --upgrade pip
        pip3 install -e ".[model_worker,webui]"

- Or install it from [source](https://github.com/lm-sys/FastChat). The code on the github is more latest than that from pip install.

        git clone https://github.com/lm-sys/FastChat.git
        cd FastChat
        pip3 install --upgrade pip  # enable PEP 660 support
        pip3 install -e ".[model_worker,webui]

3. Develop
- There are three files we'll use, `controller.py`, `vllm_worker.py`, and `gradio_web_server.py`.
- Mainly, we'll rewrite the file `gradio_web_server.py` (In my case, I copied and changed the filename to `med_chabot_web_server.py`)
- For example, we can rewrite this part to insert our prompt.

        text = text[:INPUT_CHAR_LEN_LIMIT]  # Hard cut-off
        state.conv.append_message(state.conv.roles[0], text)
        state.conv.append_message(state.conv.roles[1], None)

Like

        text = text[:INPUT_CHAR_LEN_LIMIT]  # Hard cut-off
        text = f"Doctor, {text} Based on the symptoms, can you provide me four suggestions?"
        state.conv.append_message(state.conv.roles[0], text)
        state.conv.append_message(state.conv.roles[1], None)

# Terminal Setup
# Step 1: Create a workload and determine how much GPUs that you need
We use Jupyter Lab in following as example.
Cloud AI : https://aac.amd.com/dashboard
# Step 2: Select AMD Instinct™ MI210 Accelerators
# Step 3: Two connection ways (Jupyter Lab/SSH)
# Step 4: Enter Jupyter Lab to design you project