---
# System prepended metadata

title: Draft guides for custom models

---


# Design for UI model

## Tasks:

**NOTE:** I ordered the options imo, best is the first one.

- [ ] Define which type of input from users (Model_ID, path, model.json)
- [ ] Create a table of recommendation for users (which hardware for which size)
- [ ] Create UI (@ashley)
  - [ ] Place for user input
  - [ ] Place for user to adjust params -> We can opt out this since we have `right panel`.
- [ ] Adding Docs
- [ ] Adding QA checklist for this

## Teamates to involved
- Ashley: design UI
- James: App pod
- Rex: QA


:::warning
This will be decapriated soon. See below.
:::

# How to use custom models on Jan
As Jan can run any GGUF models on Huggingface hub, please follow step by step

# 1. Choose GGUF model
You can use any models you like. In this guide we will use our latest model [Mistral-7B-Intruct-SLERP](janhq/Mistral-7B-Instruct-v0.2-SLERP-GGUF).

```title=Model_ID
janhq/Mistral-7B-Instruct-v0.2-SLERP-GGUF
```

# 2. Create model configuration file
You will need to navigate to your `jan/models` folder on your computer.

Windows
```title=Path to jan/models
C:/Users/<your_user_name>/jan/models
```

MacOS/ Linux
```title=Path to jan/models
jan/models
```

Create a folder with the name of the model

```Example
mistral-7B-intruct-slerp
```

Create a `model.json` inside the new created folder. The following is an example tempalate for `model.json`

```
{
    "source_url": "https://huggingface.co/janhq/Mistral-7B-Instruct-v0.2-SLERP-GGUF/resolve/main/mistral-7b-instruct-v0.2-slerp.Q4_K_M.gguf",
    "id": "mistral-7B-intruct-slerp",
    "object": "model",
    "name": "Mistral-7B-Intruct-SLERP",
    "version": "1.0",
    "description": "This is a guide for you to run custom model on Jan. Have fun.",
    "format": "gguf",
    "settings": {
      "ctx_len": 2048,
      "prompt_template": "<|im_start|>system\n{system_message}<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant"
    },
    "parameters": {
      "max_tokens": 2048
    },
    "metadata": {
      "author": "Jan",
      "tags": ["7B", "Finetuned"]
    }, 
    "engine": "nitro"
  }
```

:::info 3 Key notes
Ensure the `source_url` is the link to download model

Ensure the `id` is the same with the new created folder

Ensure to choose right `prompt_template`
:::

# 3. Use custom model
You can restart Jan app and look for the your new custom model in the Hub.

![image](https://hackmd.io/_uploads/HJLAqvwI6.png)

There you go. If you have any questions or request GGUF version of any models, please message us on [Jan Discord](https://discord.gg/Dt7MxDyNNZ).





