---
# System prepended metadata

title: Torchserve(Taipei Machine)

---

# Torchserve(Taipei Machine)
## 建立 TorchServe (not in Docker)
> follow https://github.com/pytorch/serve/blob/master/docs/getting_started.md

### 1. clone the repo
- `git clone https://github.com/pytorch/serve.git`
### 2. check cuda version
- `nvcc --version`
### 3. Deal with ==SSL error==(If not sudoer)
(Objective: To prevent faliure of Step4)
1. use `python3 -m pip config debug` to check if there's **pip.conf** file, if **pip.conf** file found, go to step 3
2. create pip.conf file at the place you like. Recommand to put it as `~/.config/pip/pip.conf` (Create the directory)
3. `gedit pip.conf`
	- put below text in pip.conf
	- if still has SSL error, try to find out which url cause it and add the url to pip.conf
```
[global]
trusted-host = pypi.python.org
			   pypi.org
               files.pythonhosted.org
               download.pytorch.org
```
### 4. Install dependencies
- if have found CUDA in previous step, which cuda version is in: `cu92`, `cu101`, `cu102`, `cu111`, `cu113`
		At `/serve` run `python3 ./ts_scripts/install_dependencies.py --cuda={cuXXX}`
        e.g.`python3 ./ts_scripts/install_dependencies.py --cuda=cu111`
- if no CUDA found, use
		- `python ./ts_scripts/install_dependencies.py`
- if show **no module name pip**, use
	- `sudo apt install python3-pip`

### 5. install torchserve dependencies
- `pip install testresources`
	- an error in step 4 says that this dependencies is missing, so install it here.
- `pip install torchserve torch-model-archiver torch-workflow-archiver`
- if found warning of **'{/input/the/path}' is not on PATH**, try to add it to PATH
	1. `export PATH=$PATH:{/input/the/path}`
        e.g.`export PATH=$PATH:/home/mtk53580/.local/bin`
	2. `source ~/.profile` : run for changes to take immediate effect

### 6. serve a model
> do step 1 & 2 at the path you like, but both of them need to be done in same path, overall file structure shoud be like:
> - serve (step 1)
> - model_store (step 2)
> 	- densenet161-8d451a50.pth (step 3)
> 	- densenet161.mar (step 4)

1. clone TorchServe repository
	- **don't need to do this if had already clone in step 1**
	- `git clone https://github.com/pytorch/serve.git`
3. create directory to store the model
	- `mkdir model_store`
4. download trained model
	1. `cd model_store`
	2. `wget https://download.pytorch.org/models/densenet161-8d451a50.pth`
5. archive the model (stay in folder `model_store`)
	- `torch-model-archiver --model-name densenet161 --version 1.0 --model-file ../serve/examples/image_classifier/densenet_161/model.py --serialized-file densenet161-8d451a50.pth --export-path ./ --extra-files ../serve/examples/image_classifier/index_to_name.json --handler image_classifier`
> --model file and --extra-files used files in TorchServe repository, if you didn't follow the file structure of mine, you need to adjust the path to point to **serve** folder


### 7. install jdk
> step 3 should also install java, if not, try this step
> (Can test the installation with `java -version`)
- install **.tar.gz** file for java11 [link](https://www.oracle.com/java/technologies/javase/jdk11-archive-downloads.html)
- unzip
- put /bin file in it to PATH
	- put this into **~/.profile** `export PATH="$HOME/jdk-11/bin"`


### warning
- if torchserve or other python package not found but installed, put `export PATH="$HOME/.local/bin:$PATH"` to **~/.profile**


### 8. start
- run following command in the parent directory of model_store
	- if your model_store path is: ~/model_store, then you should run command in ~
- `torchserve --start --ncs --model-store model_store --models densenet161.mar`

### 9. test out!
1. download picture for test
	- `curl -O https://raw.githubusercontent.com/pytorch/serve/master/docs/images/kitten_small.jpg`
2. `curl http://127.0.0.1:8080/predictions/densenet161 -T kitten_small.jpg`