---
# System prepended metadata

title: AMD 比賽 demo system 用法
tags: [AMD]

---

# AMD 比賽 demo system 用法
###### tags: `AMD`
## To Execute Kai's Demo
1. Open three terminals
2. In each terminal, `source .bashrc` and activate the environment via `kai_1`
3. In each terminal, `cd /home/aac/kai/repository/ADAR-MED`
4. In terminal 1, type in
    `python3 -m fastchat.serve.controller --host 127.0.0.1`
6. In terminal 2, type in
    `python3 -m fastchat.serve.vllm_worker --model-path /home/aac/Ryan/medalpaca --host 127.0.0.1`
    This will take a while since we will load a model in this step. When it's done, go step 6.
8. In terminal 3, type in
    `python3 -m fastchat.serve.med_chabot_web_server --host 127.0.0.1 --share`

## To Develop

1. Install vllm
    Refer to this [website](https://hackmd.io/@unj0M9DkQhqZGOyd71BT5g/HkFNSQEHR).
2. Install fastchat
   - You can copy from `/home/aac/kai/repository/FastChat` and then install the packages.
        ```
        pip3 install --upgrade pip
        pip3 install -e ".[model_worker,webui]"
        ```
    - Or install it from [source](https://github.com/lm-sys/FastChat). The code on the github is more latest than that from pip install.
        ```
        git clone https://github.com/lm-sys/FastChat.git
        cd FastChat
        pip3 install --upgrade pip  # enable PEP 660 support
        pip3 install -e ".[model_worker,webui]
        ```
3. Develop
    - There are three files we'll use, `controller.py`, `vllm_worker.py`, and `gradio_web_server.py`.
    - Mainly, we'll rewrite the file `gradio_web_server.py` (In my case, I copied and changed the filename to `med_chabot_web_server.py`)
    - For example, we can rewrite this part to insert our prompt.
      ```python=340
        text = text[:INPUT_CHAR_LEN_LIMIT]  # Hard cut-off
        state.conv.append_message(state.conv.roles[0], text)
        state.conv.append_message(state.conv.roles[1], None)
      ```
      Like
      ```python=340
        text = text[:INPUT_CHAR_LEN_LIMIT]  # Hard cut-off
        text = f"Doctor, {text} Based on the symptoms, can you provide me four suggestions?"
        state.conv.append_message(state.conv.roles[0], text)
        state.conv.append_message(state.conv.roles[1], None)
      ```


# Terminal Setup
# Step 1: Create a workload and determine how much GPUs that you need
We use Jupyter Lab in following as example.
Cloud AI : https://aac.amd.com/dashboard
![image](https://hackmd.io/_uploads/H1kd_XSYR.png)
![image](https://hackmd.io/_uploads/SkaddQSKC.png)
# Step 2: Select AMD Instinct™ MI210 Accelerators
![image](https://hackmd.io/_uploads/HJaF_QBtC.png)
# Step 3: Two connection ways (Jupyter Lab/SSH)
![image](https://hackmd.io/_uploads/HJFqOQHtR.png)
# Step 4: Enter Jupyter Lab to design you project
![image](https://hackmd.io/_uploads/B1cQtmStC.png)
