---
# System prepended metadata

title: Nicole Architecture Deepdive

---

# Nicole Architecture Deepdive



## 4 Dec 2023

# 4 Dec Codebase notes

Deep dives: 

- [ ]  Inference extension details
- [ ]  Electron Extension lifecycle: look in /electron/extensions

## High level questions

- How is server supposed to be built? (see below)
    - Will Desktop depend on Server running?
    - How to actually decouple in a clean way?
- Lifecycle of the app? Lifecycle of an extension?
- Whats going on with **events**?
- **Overall devex**: what does adding an extension devex look like?
    - User wants to override existing API - straightforward
    - User wants to introduce new data stores - has to modify core?
    - User wants to add new UI - which files are they touching?
- Core SDK design specifics
    - Core exposes both backend process wrappers (like `fs`) and extension/controller APIs (like `createAssistant`)
    - Why not directly expose the stdlib `fs` methods in `core`? Why write wrappers like `writeFile` / `writeFileSync`? `DownloadManager` / `request`
        - This requires devs to learn our framework, rather than directly using `fs` & `request` which they are already familiar with
    - Events should be an enum on the entity, rather than a separate global EventName Obj.
        - e.g. `[assistant.events](http://assistant.events)` (or a better semantic way)

**What if:**

```jsx
core/
	apis/            // Jan specific entities like models, assistants and their interfaces
	std/             // Our cross-platform, standard library, aka 'helpers' 'wrappers' needs better name
		fs/            // Proposal: directly expose stdlib, e.g. `fs`, `stdout`, `child_process`
		fileManager/	 // We provide additional wrappers, e.g. `writeFile`
		downloadManager/
		eventsManager

  // Framework specific, native OS controls/gestures/menus
	nativeOS/        // Desktop only library
		shellManager/  // e.g. openFileExplorer -> shell.openPath()

extensions/        // 3rd party extensions mostly go here
  assistants/      // Assistant controller logic, i.e. `assistant.json`

infra/
	common/
		fileManager    // Impl fm.writeFile
	electron/        // Handles fs, fileManager/fm.writeFile
	server/          // Handles fs, fileManager/fm.writeFile
```

- **/extensions** implements **APIs**, i.e.  extensions’ controller logic
    - The filesystem logic we have is implemented at this level
    - i.e. `/extensions` implementations should include `/assistants` > `assistant.json` and  `/model` *Atm, `assistant` is in the code, and `/models` nested in root directory?*
- **/infra/[common/electron/server]** implements **stdlib** modules, i.e. modules that use backend-node/main processes
    - Directly expose stdlib objects that run on background processes, e.g. `fs`, `request`
    - Also we can pre-implement a lot of helper libs, e.g. our `fs` wrapper, and `request` objects
    - Implemented shared logic in `/infra/common`

## Desktop Mode with Server

- What actually happens during runtime?
- Linh’s code is just a basic server scaffold? What is architecture vision?

### **Option 1: Current direction?**

- UI routes all actions toward the Server process, not Electron process
- Only NativeOS actions execute on ipcMain.
- Pros:
- Cons:
    - Desktop mode requires running a server / opening PORT
    - Leaky abstractions

```mermaid
graph LR
	User --POST/assistant--> PORT_1337 --> FastifyProcess --> fs_handler --> fs
	User -- useCreateAssistant --> PORT_1337

	User --useDesktopAction--> ElectronMainProcess --> action_handler
```

### **Option 2: Clean implementation**

- Calls run in their respective processes, i.e. when using electron GUI, background processes execute through `ipcMain`
- Common logic is implemented in a single file
    - Controller logic (`createAssistant` in `/extensions/assistant`) and background process logic (in `/common` `fs.makeFile`)
    - Sidenote: this means extension-devs would only have to write controller logic, we should implement all the background process libs… implement once, run everywhere.
- `Electron/server` are just routers to execute  common logic in their respective background processes
- Pros: clean, can easily decouple Server from Electron and vice versa
- Cons: mutex complexity with multiple processes on fs (but this is edge case...)

```mermaid
graph LR
	User -- POST/assistant--> PORT_1337 --> FastifyProcess --> fs_handler --> fs
	User -- useCreateAssistant--> ElectronMainProcess --> fs_hander --> fs
```

- Electron and server share the same fs_handler implementation

```jsx
//infra/common/fs_handler.ts
const writeFile = (event, path: string, data: string): Promise<void> => {
      try {
        await fs.writeFileSync(join(userSpacePath, path), data, 'utf8')
      } catch (err) {
        console.error(`writeFile ${path} result: ${err}`)
      }
    };
```

```jsx

// Controller implementation (infra agnostic)
// extensions/assistants/index.ts
// ----------------------------
async createAssistant() {
	fs.writeFile(...)
}

// Infra handlers
// ----------------------------
// framework/electron/handlers/fs.ts (ElectronAPI)
ipcMain.handle('writeFile', writeFile)

// framework/server/handlers/fs.ts => sse (ServerAPI)
export async function writeFile() { writeFile }         // Is this needed: fastify.post('/writeFile', writeFile)? 

// framework/server/index.ts
// One-time-macro to handle user REST calls e.g. `POST /createAssistant`
// This means users can add additional endpoints via extensions (without explicit definition)
fastify.createEndpointsFor(core.assistant)
```

Our SDK:
- Build AI apps once, works cross platform
- OpenAI compatible entities and interface
- More powerful:
- Events
- Injectable (Turing complete code, not just functions)

At a high level, our SDK has 2 parts:

1. An open AI, compatible objects and interface library (this is the asst framework) 
2. A standard library that is cross platform (this is the local first framework)

-  
- Local first, with helpful libraries for file based data persistence


## 3 Dec 2023

## High level Explanation
- Jan supports 3rd party extensions through a core SDK ( `@janhq/core` (link)).
- Jan is platform agnostic, capable of operating on Desktop, web client-server architecture, and mobile devices, in both online and offline modes.
    - This is achieved through the application of `Clean Architecture` and the `Dependency Inversion Principle`, allowing infrastructure-specific implementations to be injected only at runtime. *Rephrase: Developers can build features wihtout worrying about cross compatibility*
    - Supported application frameworks include an Electron desktop app, web client + server mode, and iOS/Android (coming soon).
    - Supported data storage mechanisms include local filesystems, NoSQL databases, and mobile-specific storage solutions (coming soon).
- By default, Jan operates in a local-first mode, prioritizing offline functionality unless specified otherwise. 
- User data in Jan is structured to be modular, enabling users to bundle their assistants, conversation history, model preferences, and more into a single folder for easy transfer across devices.

## Developer
- Getting Started
    - Build an extension
    - Anatomy of an extension ([example](https://code.visualstudio.com/api/get-started/extension-anatomy))
        - Filestructure
        - Entrypoint
        - Manifest File
        - Lifecycle Build / runtime 
    - Development workflow
    - Desktop development
    - Server/Cloud development
    - Mobile development (coming soon)
    - [Obsidian example](https://docs.obsidian.md/Plugins/Getting+started/Anatomy+of+a+plugin)
- Extension Capabilities 
    - Overview
    - Core capabilities
        - types, 
        - Data Store (vault)
            - There are various ways to store Data, depending on the infrastructure inferred at runtime: `folder diretory` (default mode if no mechanism is specified). A noSQL 
            - [vsscode example](https://code.visualstudio.com/api/extension-capabilities/common-capabilities#data-storage)
    - ([vscode example](https://code.visualstudio.com/api/extension-capabilities/overview))
- Extension Guides
    - store
    - model
    - assistant
    - thread
    - etc.
- Infrastructure
    - ... 
- UI and Themes
    - [Obsidian example](https://docs.obsidian.md/Plugins/User+interface/About+user+interface)
- Publish Extensions
    - Testing
    - Publishing to the Hub
    - [later] Monetizing
- Advanced Topics
    - How Jan is Built
- Sample Extensions
    - Conversational
    - Inference
    - Assistant
    - Monitoring

## Log

### 3 Dec 2023

- Realization: electron doesn't need to implement from core.
- 

- How do 3rd party extensions modify & mount on top of this?
- Users want to create a different `assistant UX`?
- Users want to modify `assistant` object... add a vanity property
    - To persist objects, they have to add to core...
- Users want to modify assistant logic but not touching store


`/core`
```javascript
export assistant // the asst object
export createAssistant // methods on assistants
```

`/infrastructure/electron/handlers/defaults.ts` (not sure about path)
- Keep this layer as light as possible
- Should just call whatever is defined in `/extensions`

```javascript 

ipcMain.handle(
    'writeFile',
    fs.createItem()
)
```

`/extensions/fs-assistant-extension/` (infra & driver agnostic code)
- Implements the folder structure logic, i.e. `janroot/assistants`
- Implements the default global assistant

`index.ts`
```javascript
import {Assistant, Store} from Core
type()
onLoad( createDefaultAssistant() )
onUnload()

async createAssistant(assistant)
    store = getStoreInstance()
    await store.createStorage(path)     // not fs.mkdir()
    await store.createItem(assistant)
```

`/web/hooks/useCreateAssistant.ts`

```javascript
// Keep the same
import Assistant from Core

const createAssistant = extensionManager.get(ext).createAssistant()
export function createAssistant();
```









---

- find extension docs templates
    - first section like obsidian: https://docs.obsidian.md/Plugins/Getting+started/Anatomy+of+a+plugin
    - subsequent sections on extensions like VSCode
- How does the first assistant.json, model.json get created? tehy should sit in the appropriate place. is it under extensions. or in its own folder?

- "Jan's initial extensions are available as a core library @janhq/core"

### 1 Dec 2023

BaseExtesnion
- registerView
- registerSettings

core.api
core.events

Electron/invokers/app.ts: bridge btw app and electron (refactored from preload.js)
	Electron/handlers




### 30 Nov 2023
- https://app.excalidraw.com/s/kFY0dI05mm/2do4B3rI2hY

Q: how to properly refactor fs?
Q: how to properly refactor events?
Q: what's the extension devex?

## Proposal

## Motivation
- @louis-jan's suggestion to go Clean is good.
- We have leaky abstractions, a result of legacy features. This can be (slowly) refactored over time. 
- I suggest that we perfect the specifications ASAP, before we perfect the code.
- The resulting codebase will undoubtedly move us closer to things like a offering the world a `local-first, but truly modular framework`.

> This is a thread to kickoff this discussion && to patch my own misunderstandings.


## Possible directory structure
```sh
core/                     # The Jan SDK: @janhq/core
    domain/               # Object definitions
    application/          # Interface definitions (imports domain)

infrastructure/
    # Frameworks
    electron/
    capacitor/            # Future consideration

    # Adapters, drivers, devices
    fs/
    identity/            # Future consideration
    
web/                     # More or less the same
```

## `/core`
- Core, aka `@janhq/core`, is the Jan SDK
- Core has no `io` concerns (this is where we currently have leaky abstractions)
- Core has no framework concerns (we also have leaky abstractions here)
- The actual implementations for `core sdk` are injected at runtime, per `dependency inversion principle`. This means we can route to implementations inside of `/infrastructure` at runtime and not worry about infra at all at this level.

**Directory**

```sh
/core                                   
    index.ts                            
    /Domain                             # Object layer
        /Common
            objectBase.ts
            globalConstants.ts
            globalEnums.ts
            globalExceptions.ts
        
        # Sample Objects
        /Assistant                      
            assistant.ts                # Define obj: type, const, enums, exceptions, etc.
            events.ts                   # Define events, if any    
        /Store                          # Same as fs.ts
            store.ts                    # Define store
        /Thread
        /Message                        # onMessage events are moved here
            
    /Application                        # Interface Layer
        /Common
            interfaceBase.ts
            executeOnMain.ts            # Isolate the Electron specific stuff here
        # Sample Interface
        /Assistant
            assistant.ts                # Define CRUD interface
            eventHandlers.ts            # Event hander defs
        /Store
            store.ts                    # Define CRUD interface, e.g. writeObj, deleteObj
            dependencyInjection.ts      # Calls window.coreAPI?._()
```

## `/infrastructure`
- Contains the infrastructure-specific implementations on Jan SDK
    - Framework-level: `electron`, `capacitor`, `web-only`
    - Adapter/driver-level: `fs`, `sqlite` (we should add `fsAPI` to `coreAPI`)
    - Adapter/driver-level: `oauth`, `saml`
- Define the cascading default behavior at runtime, i.e. how to determine what framework and drivers are available.

```sh
/infrastructure                         # Implementations
    /Fs                                 # Implements store
    /Electron                           # Current folder, but keep the os-native functions only
    /Identity
```


### Ref
- [Main inspo]( https://github.com/jasontaylordev/CleanArchitecture/tree/main/src)


---

https://www.youtube.com/watch?v=SxJPQ5qXisw&ab_channel=DevTernityConference

https://github.com/jasontaylordev/CleanArchitecture/tree/main/src
- has leaky abstractions though... not end of the world?

**/core**
- domain (just objects)
    - common
    - entities
        - models
        - threads
    - events
    - exceptions
- application (just interfaces, references domain)
    - common
        - interfaces
            - e.g. `IApplicationDbContext` (get,set) LEAKY!
    - models
        - commands
            - CRUD models
        - eventHandlers 
        - queries
    - threads

- infrastructure (implements the app interfaces)
    - files
    - identity
    - persistence
        - e.g. `ApplicationDbContext` (implements interface)
    - services (external io)

- web (handlers, mapping UI to invocation)
    - 

Alternate: 
![image](https://hackmd.io/_uploads/B1dwaeUHT.png)

- Web (entrypoint)
- Electron

- Application
    - Features
        - Entity




---


1. Core shouldn't have to know about `plugins` & `extensions`?
    - Would Extension makers have to edit `/core`?
    - `JanPlugin` -> `JanEntity`?
    - Instead of `conversational.ts plugin` break it down into `threads`, `messages` **entities** instead.
e.g. 
```markdown
core/
    store                # fs/db abstraction
    model                # Exports type, interface, etc.
    thread               # Exports type, interface, etc.
    message
    assistant
```

2. Rename `fs` to `store`? Because data can be in fs or db?
```js
export abstract class Store {
    type Store = {...};
    abstract createObject();     // Formerly writeFile
    abstract getObject();        // Formerly readFile
    abstract listObjects();      // Formerly listFiles    
}
```

3. What's our directory folder structure?

```sh
/core                   # Domain layer
    /entities           # Type, interface definitions
/extensions             # Application layer code
    /chat
        createThread    
        deleteThread
    /hub
        downloadModel   #
    /settings
        ...
/server                 # Traditional server framework
    /fs-adapter         # Impl of core.store.getObject
/client              # User interfaces/presentation layer
    /web             # Most of our current stuff
    /electron        # Electron specific stuff, inherits /web
    /mobile          
/data                # Data access impls
    /local           # fs stuff
    /lite            # random db example
    /remote          # random remote storage option
```

![image](https://hackmd.io/_uploads/HybruABrp.png)

## Appendix

- Louis diagrams: https://excalidraw.com/#room=f6b7902dd2ca16c5aae8,EenOqcSGf0yTsUOxwM-Ebg
Architecture questions: 


1. Clean should map onto folders. What’s the folder structure?

1. Take me through implementations of writing a message to filesystem. From UI component to UI component
	- Do we write to file first?



Core compiles at interface level only
CoreAPI is injected at compile time by the application!

Runtime: application injects actual implementation of coreAPI. 
	then accessible via window.coreAPI.

Electron code is in /electron/handlers

———— 

1. At compile time, window is just an empty object

Moving some stuff to server

—

Electron: native only, events driven 
	/ fs
	/ menu
	/ threads 	
—

Server: refactor most endpoints here
	/models
	/start
	/ threads?

——

### 29 Nov 2023
- [ ] QA https://hackmd.io/qn4CBE_zQUeElYEXuJevfg
- [ ] read: https://hackmd.io/@janhq/HylgdG4Bp
- Multiple inf engine disc: https://github.com/janhq/jan/issues/771


## Questions: 
- Lifecycle of a model extension implementation (download model)
    - Relative to our current codebase
- Both in Electron wrapper and Serverside-only mode


- Overview
    - Overall diagram/explainers of how jan is built
    - Walk through directory structure?
        - Web: all user interfaces and app functionality within a single browser window should be written with the same tools and paradigms that you use on the web (from electrons perspective)
        - `Caveat`: In order to directly include NPM modules in the renderer, you must use the same bundler toolchains (for example, webpack or parcel) that you use on the web.




- Files and Folders: 
    - e.g. https://help.obsidian.md/Files+and+folders/Accepted+file+formats
    - Mention Sandboxing
        – Extension Devs always assume a flat-file system, with a defined JSON
        – Backend service can implement it using a DB (if they want)
        – Backend service can implement using a sandboxed filesystem too
- Extensions
    - Definition / diagrams
        - > Extensions are similar to VSCode and Obsidian extensions. (model page: https://code.visualstudio.com/api)
    - Lifecycles
    - Sample instances
    - e.g. https://help.obsidian.md/Extending+Obsidian/Community+plugins
- OS Modules
    - Definition / diagrams
        - Modules are implementations on native operating system (OS) APIs. For example, filesytem operations (link), inference operations on RAM/VRAM (link) are implemented as Jan Modules. Modules subsequently expose an interface that an `Extension` can implement. 
    - Lifecycles
    - Sample instances

- Concepts
    - Mapping concepts to Electron
    - Mapping concepts to OpenAI

- User interface
    - UI framework
    - UIKit
    - Customizing Jan: https://help.obsidian.md/Customization/Appearance
    
- Extensions List
    - Models Extension
    - Chat Extension

- Modules List
    - etc



## Prereq Concepts (mostly Electron)

- `Native interfaces/APIs`: include the file picker, window border, dialogs, context menus, and more - anything where the UI comes from your operating system and not from your app. The default behavior is to opt into this automatic theming from the OS.

- `Native menu`: a custom menu bar within an Electron-based application that uses **native operating system (OS) APIs** instead of the default Chromium-based menu bar provided by Electron
    - `App menus`: are menu items at the top of mac os bar when u open an app
    - `context menus`: are like things copied to user clipboard, mouse gestures.

- `Main process` 
    - Equivalent to Chrome's main process manager: a single browser process then controls these (renderer/tab-specific) processes, as well as the application lifecycle as a whole
    - Controls renderer processes
    - Controls app lifecycle as a whole.
    - Runs in a **Node.js** environment, meaning it has the ability to require modules and use all of Node.js APIs

- `BrowserWindow`
    - Managed by `main` process.
    -  Each instance of the `BrowserWindow` class creates an application window that loads a web page in a separate renderer process. 
    -  its contents are called `webcontents`
    -  Browser windows are `EventEmitter`!

- `Web embeds`
    - If you want to embed (third-party) web content in an Electron BrowserWindow, there are three options available to you: `<iframe> tags`, `<webview> tags`, and `BrowserViews`.

- `Preload scripts`
    - Main process attaches preload scripts to BrowserWindow instances
    - So that renderer processes get access to node/os features

- `Node vs OS features`
    - Node.js provides several runtime-specific features that are not OS-level. Some examples include:
    - Event Loop: Node.js has an event-driven architecture with an event loop that allows non-blocking I/O operations, making it suitable for building scalable network applications
    - Asynchronous APIs: Node.js provides asynchronous system APIs, allowing developers to perform I/O operations without blocking the execution of other code
    - Worker Threads: Node.js allows the creation of worker threads, which are separate instances of the V8 engine running in parallel, enabling multi-threaded execution within a Node.js application


- `ContextIsolation`
    - Context Isolation means that `preload` scripts are isolated from the `renderer's main world` to avoid leaking any privileged APIs into your web content's code.



## Appendix: 
- Easy Electron / IPC overview: https://www.electronjs.org/docs/latest/tutorial/ipc
- IPC patterns and fuckery: https://www.electronjs.org/docs/latest/tutorial/ipc#pattern-1-renderer-to-main-one-way