# Jan: File-based approach with OpenAI compatibility

## High Level Goals
- OpenAI-compatible API
- Local File Structure (as the permanent storage soln)
- Composability & Extensibility

*See rational at bottom*

### Nomenclature

- `API`:
- But Devs should be able to just use the API to build externally, e.g. a VSCode extension.
- `SDK`:
- Builds on top of the API
- `Plugins`:
- System-wide plugins, can be called by higher-level functions
- `Assistants`:
- An assistant is _not_ a `Plugin`;
- Assistants are higher-level abstractions that are 1:1 with OpenAI's Assistants
- `Assistants` can compose over `models` and `tools` , and run arbitrary code
- `Tools`:
- Equivalent to OpenAI's "tools" or "functions" (formerly "plugins").
- Installed at the Assistant level as npm imports?

### Deprecated

- `Apps` are deprecated

## Directory Structure

# Sample user folder structure
/models
    /llama-7b                          # Model_id
        model.json                     # Default Model settings
        llama-7b-q4-k-l.gguf           # Raw model file(s)

/assistants
    /jan
        assistant.json                 # e.g. description
        model.jsons                    # MODELS * 
    /homework-helper                   # Assistant_id      
        assistant.json
        model.json                     # Overrides Model Settings, if set
        /files
            answer-sheet.pdf           # Q!: files copied => More exportable
            files.json                 # Q!: referenced => More lightweight
    /my-ai-gf         
        assistant.json
        model_x.json
        model_xx.json                  # Multi model assistant
        # Go workspace convention overrides would also allow:
        /models
            llama-7b
        /threads
            /...
    /stable-diffuser
        assistant.json
        model.json
        index.js                       # Q!: allow SDK impl here?

/threads
    /jan_unixTimestamp
        thread.json                      
        model.json            # Model that user chose for the thread
        messages.json         # Messages.json, message.files point to `/files`
        /files
    /assistantName_unixTimestamp
        thread.json
        model.json        # Overrides Assistant-level ModelFile
        messages.json

Note: The Jan Folder Structure follows Go Workspace Conventions, i.e. `the closest/deepest path import takes precedence`. This means that if there are multiple `/threads` or `/models` within an `/assistant` folder, the one that is imported from within the directory wins.

## Models
- Models refer to the actual, low level, AI models

### model.json (i.e. `Modelfiles`)
- A `ModelFile` is a specific snapshot of "a model's settings"
- `ModelFiles` follow Go Workspace’s conventions:
- A `Model` has a default, Model-level`ModelFile`
- An `Assistant` can have an Assistant-level `ModelFile`, which overrides Model-level
- A `Thread` can have a Thread-level `ModelFile`, which overrides Assistant-level

Open questions:
- Should ModelFiles be in `dockerfile` or `json`?
- Should ModelFiles have a structured format (like Ollama) or unstructured?
- See below:

Structured:

from: "./models/..."
parameter: {... //unstructured params go in here}
template:
system: 

Unstructured:

from: "./models"
param_1: ""
param_2:

## Messages
- `Message` objects contain content, `files`, and other metadata
- See [OpenAI message object](https://platform.openai.com/docs/api-reference/messages)

## Threads
- Root level `/threads` contain convos with Jan assistant, and other assistants by default. Reasons:
- When users delete an assistant, threads should persist
- In the future, we may support 1 thread, many assistants
- Threads are user conversations with `assistants`
- Threads contain `Messages`
- `Threads` can have its own `ModelFile`: When users start a new thread with an `assistant`, they have the option of changing model settings. This creates a new Thread-based `ModelFile`
- Once the user sends a `message` in a `thread`, the `thread` is **locked**:
- Locked: Users cannot change the `assistant` it is talking to.
- Q: should users be able to edit modelfile parameters? e.g. temp?
- Threads do not contain `files`

Open questions:
- How do users attach docs/pdfs within a thread? See OpenAI's [message obj](https://platform.openai.com/docs/api-reference/messages/object#messages/object-file_ids)
- Should we let users modify Thread's `ModelFile` after messages are already sent? I don't see why not...

## Assistants

Mode 1: Typical Assistant

.jan
    /models
    /threads
    /assistants
        assistants.json
        index.js               # SDK customizations
        model.json
        /files

Mode 2: Assistant overrides global threads & models

.jan
    /models
    /threads
    /assistants
        assistants.json        # -f: using locally packaged `model`, `thread`
        /models                # optional & overrides
        /threads               # optional & overrides

- Assistants are higher-level abstractions that compose on top of `threads`, `models`
- Assistants can have a `model.json`
- Assistants can have a knowledge base of files
- If an `Assistant` is deleted, its entire folder and child files are deleted
- If an `Assistant` is deleted, the underlying `model` remains, but the Assistant-level `model.json` is deleted
- [To Discuss] If an `Assistant` is deleted, its previous `threads` remain, but users cannot chat again

## Stress test:

*What if 2 different assistants use the same underlying model? It seems we reference a shared file-path to the model .bin/.gguf? Where is this folder that contains the raw models?*
- It would just be in ./models

*What happens when a user deletes an assistant? Do we delete the model? Separately, do we delete convo history from the app?*
- Only the assets in ./assistants is deleted

*Is model_config the same thing as Ollama's ModelFile? What are the properties in the file? How is it different for gguf vs another model format?*
- Open to discussion

*What if users just want to slightly tweak the system prompt of an assistant? Do they (a) edit the existing assistant or (b) create a new assistant?*
- They would create a `thread` with an `assistant`, edit the settings (Thread-level ModelFile). This MF is saved in /threads. A new assistant is not created.

*How would users chat with a model "out of box"? e.g. a base-case where I want to chat with a default mistral-7bn. So do we automatically create an assistant each time users download a model? Or can users start a thread directly with models, not just assistants?*
- They would chat with a `model`, using the Model-level MF.

*OpenAI allows `file_id` on 2 levels: (1) in an assistant and (2) within individual messages (threads). From the md above, it do we only support the latter?*
- I'd prefer to stay OAI compatible, and support file_id for both `assistant` level and `message` level. But open to discussion.

## Strategic Rationale

### Why OpenAI Compatibility
- We've been struggling to define what is a `model` vs `modelfile` vs `assistant/ai/app`. During retreat, we coincidentally converged on the `Assistant API` that OpenAI released last week. Rather than reinvent the wheel, we can leverage their ERD and default to an industry standard.
- OpenAI has a moat, and are becoming the industry API standard
- OpenAI needs an open source alternative - we can be ActuallyOpenAI - an "escape hatch" that developers can offramp from OpenAI within xx minutes.
- **Our vision remains the same** - be a Personal AI that is local and private

### Why File-based Approach
- `Data service` was quite convoluted, and a bit of legacy from our previous iterations.
- Devs shouldn't have to learn our database to build on Jan
- VSCode & Obsidian's use of local files allows for better performance and flexibility when it comes to data storage and retrieval. Files are faster to access and more lightweight than databases, which can be slower and more resource-intensive. Additionally, files allow for more straightforward data versioning and recovery, since they can be easily backed up and restored. This moves us closer to our vision of an AI "cortical stack".
- **Our strategy remains the same** - build a dev ecosystem on top of Jan. This is a critical devex improvement.