# Design for UI model

## Tasks:

**NOTE:** I ordered the options imo, best is the first one.

- [ ] Define which type of input from users (Model_ID, path, model.json)
- [ ] Create a table of recommendation for users (which hardware for which size)
- [ ] Create UI (@ashley)
- [ ] Place for user input
- [ ] Place for user to adjust params -> We can opt out this since we have `right panel`.
- [ ] Adding Docs
- [ ] Adding QA checklist for this

## Teamates to involved
- Ashley: design UI
- James: App pod
- Rex: QA

:::warning
This will be decapriated soon. See below.
:::

# How to use custom models on Jan
As Jan can run any GGUF models on Huggingface hub, please follow step by step

# 1. Choose GGUF model
You can use any models you like. In this guide we will use our latest model [Mistral-7B-Intruct-SLERP](janhq/Mistral-7B-Instruct-v0.2-SLERP-GGUF).

janhq/Mistral-7B-Instruct-v0.2-SLERP-GGUF

# 2. Create model configuration file
You will need to navigate to your `jan/models` folder on your computer.

Windows

C:/Users/<your_user_name>/jan/models

MacOS/ Linux

jan/models

Create a folder with the name of the model

mistral-7B-intruct-slerp

Create a `model.json` inside the new created folder. The following is an example tempalate for `model.json`

{
    "source_url": "https://huggingface.co/janhq/Mistral-7B-Instruct-v0.2-SLERP-GGUF/resolve/main/mistral-7b-instruct-v0.2-slerp.Q4_K_M.gguf",
    "id": "mistral-7B-intruct-slerp",
    "object": "model",
    "name": "Mistral-7B-Intruct-SLERP",
    "version": "1.0",
    "description": "This is a guide for you to run custom model on Jan. Have fun.",
    "format": "gguf",
    "settings": {
      "ctx_len": 2048,
      "prompt_template": "<|im_start|>system\n{system_message}<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant"
    },
    "parameters": {
      "max_tokens": 2048
    },
    "metadata": {
      "author": "Jan",
      "tags": ["7B", "Finetuned"]
    }, 
    "engine": "nitro"
  }

:::info 3 Key notes
Ensure the `source_url` is the link to download model

Ensure the `id` is the same with the new created folder

Ensure to choose right `prompt_template`
:::

# 3. Use custom model
You can restart Jan app and look for the your new custom model in the Hub.


There you go. If you have any questions or request GGUF version of any models, please message us on [Jan Discord](https://discord.gg/Dt7MxDyNNZ).