# Nicole Architecture Deepdive

## 4 Dec 2023

# 4 Dec Codebase notes

Deep dives:

- [ ] Inference extension details
- [ ] Electron Extension lifecycle: look in /electron/extensions

## High level questions

- How is server supposed to be built? (see below)
- Will Desktop depend on Server running?
- How to actually decouple in a clean way?
- Lifecycle of the app? Lifecycle of an extension?
- Whats going on with **events**?
- **Overall devex**: what does adding an extension devex look like?
- User wants to override existing API - straightforward
- User wants to introduce new data stores - has to modify core?
- User wants to add new UI - which files are they touching?
- Core SDK design specifics
- Core exposes both backend process wrappers (like `fs`) and extension/controller APIs (like `createAssistant`)
- Why not directly expose the stdlib `fs` methods in `core`? Why write wrappers like `writeFile` / `writeFileSync`? `DownloadManager` / `request`
- This requires devs to learn our framework, rather than directly using `fs` & `request` which they are already familiar with
- Events should be an enum on the entity, rather than a separate global EventName Obj.
- e.g. `[assistant.events](http://assistant.events)` (or a better semantic way)

**What if:**

core/
	apis/            // Jan specific entities like models, assistants and their interfaces
	std/             // Our cross-platform, standard library, aka 'helpers' 'wrappers' needs better name
		fs/            // Proposal: directly expose stdlib, e.g. `fs`, `stdout`, `child_process`
		fileManager/	 // We provide additional wrappers, e.g. `writeFile`
		downloadManager/
		eventsManager

  // Framework specific, native OS controls/gestures/menus
	nativeOS/        // Desktop only library
		shellManager/  // e.g. openFileExplorer -> shell.openPath()

extensions/        // 3rd party extensions mostly go here
  assistants/      // Assistant controller logic, i.e. `assistant.json`

infra/
	common/
		fileManager    // Impl fm.writeFile
	electron/        // Handles fs, fileManager/fm.writeFile
	server/          // Handles fs, fileManager/fm.writeFile

- **/extensions** implements **APIs**, i.e. extensions’ controller logic
- The filesystem logic we have is implemented at this level
- i.e. `/extensions` implementations should include `/assistants` > `assistant.json` and `/model` *Atm, `assistant` is in the code, and `/models` nested in root directory?*
- **/infra/[common/electron/server]** implements **stdlib** modules, i.e. modules that use backend-node/main processes
- Directly expose stdlib objects that run on background processes, e.g. `fs`, `request`
- Also we can pre-implement a lot of helper libs, e.g. our `fs` wrapper, and `request` objects
- Implemented shared logic in `/infra/common`

## Desktop Mode with Server

- What actually happens during runtime?
- Linh’s code is just a basic server scaffold? What is architecture vision?

### **Option 1: Current direction?**

- UI routes all actions toward the Server process, not Electron process
- Only NativeOS actions execute on ipcMain.
- Pros:
- Cons:
- Desktop mode requires running a server / opening PORT
- Leaky abstractions

graph LR
	User --POST/assistant--> PORT_1337 --> FastifyProcess --> fs_handler --> fs
	User -- useCreateAssistant --> PORT_1337

	User --useDesktopAction--> ElectronMainProcess --> action_handler

### **Option 2: Clean implementation**

- Calls run in their respective processes, i.e. when using electron GUI, background processes execute through `ipcMain`
- Common logic is implemented in a single file
- Controller logic (`createAssistant` in `/extensions/assistant`) and background process logic (in `/common` `fs.makeFile`)
- Sidenote: this means extension-devs would only have to write controller logic, we should implement all the background process libs… implement once, run everywhere.
- `Electron/server` are just routers to execute common logic in their respective background processes
- Pros: clean, can easily decouple Server from Electron and vice versa
- Cons: mutex complexity with multiple processes on fs (but this is edge case...)

graph LR
	User -- POST/assistant--> PORT_1337 --> FastifyProcess --> fs_handler --> fs
	User -- useCreateAssistant--> ElectronMainProcess --> fs_hander --> fs

- Electron and server share the same fs_handler implementation

//infra/common/fs_handler.ts
const writeFile = (event, path: string, data: string): Promise<void> => {
      try {
        await fs.writeFileSync(join(userSpacePath, path), data, 'utf8')
      } catch (err) {
        console.error(`writeFile ${path} result: ${err}`)
      }
    };

// Controller implementation (infra agnostic)
// extensions/assistants/index.ts
// ----------------------------
async createAssistant() {
	fs.writeFile(...)
}

// Infra handlers
// ----------------------------
// framework/electron/handlers/fs.ts (ElectronAPI)
ipcMain.handle('writeFile', writeFile)

// framework/server/handlers/fs.ts => sse (ServerAPI)
export async function writeFile() { writeFile }         // Is this needed: fastify.post('/writeFile', writeFile)? 

// framework/server/index.ts
// One-time-macro to handle user REST calls e.g. `POST /createAssistant`
// This means users can add additional endpoints via extensions (without explicit definition)
fastify.createEndpointsFor(core.assistant)

Our SDK:
- Build AI apps once, works cross platform
- OpenAI compatible entities and interface
- More powerful:
- Events
- Injectable (Turing complete code, not just functions)

At a high level, our SDK has 2 parts:

1. An open AI, compatible objects and interface library (this is the asst framework)
2. A standard library that is cross platform (this is the local first framework)

-
- Local first, with helpful libraries for file based data persistence

## 3 Dec 2023

## High level Explanation
- Jan supports 3rd party extensions through a core SDK ( `@janhq/core` (link)).
- Jan is platform agnostic, capable of operating on Desktop, web client-server architecture, and mobile devices, in both online and offline modes.
- This is achieved through the application of `Clean Architecture` and the `Dependency Inversion Principle`, allowing infrastructure-specific implementations to be injected only at runtime. *Rephrase: Developers can build features wihtout worrying about cross compatibility*
- Supported application frameworks include an Electron desktop app, web client + server mode, and iOS/Android (coming soon).
- Supported data storage mechanisms include local filesystems, NoSQL databases, and mobile-specific storage solutions (coming soon).
- By default, Jan operates in a local-first mode, prioritizing offline functionality unless specified otherwise.
- User data in Jan is structured to be modular, enabling users to bundle their assistants, conversation history, model preferences, and more into a single folder for easy transfer across devices.

## Developer
- Getting Started
- Build an extension
- Anatomy of an extension ([example](https://code.visualstudio.com/api/get-started/extension-anatomy))
- Filestructure
- Entrypoint
- Manifest File
- Lifecycle Build / runtime
- Development workflow
- Desktop development
- Server/Cloud development
- Mobile development (coming soon)
- [Obsidian example](https://docs.obsidian.md/Plugins/Getting+started/Anatomy+of+a+plugin)
- Extension Capabilities
- Overview
- Core capabilities
- types,
- Data Store (vault)
- There are various ways to store Data, depending on the infrastructure inferred at runtime: `folder diretory` (default mode if no mechanism is specified). A noSQL
- [vsscode example](https://code.visualstudio.com/api/extension-capabilities/common-capabilities#data-storage)
- ([vscode example](https://code.visualstudio.com/api/extension-capabilities/overview))
- Extension Guides
- store
- model
- assistant
- thread
- etc.
- Infrastructure
- ...
- UI and Themes
- [Obsidian example](https://docs.obsidian.md/Plugins/User+interface/About+user+interface)
- Publish Extensions
- Testing
- Publishing to the Hub
- [later] Monetizing
- Advanced Topics
- How Jan is Built
- Sample Extensions
- Conversational
- Inference
- Assistant
- Monitoring

## Log

### 3 Dec 2023

- Realization: electron doesn't need to implement from core.
-

- How do 3rd party extensions modify & mount on top of this?
- Users want to create a different `assistant UX`?
- Users want to modify `assistant` object... add a vanity property
- To persist objects, they have to add to core...
- Users want to modify assistant logic but not touching store

`/core`

export assistant // the asst object
export createAssistant // methods on assistants

`/infrastructure/electron/handlers/defaults.ts` (not sure about path)
- Keep this layer as light as possible
- Should just call whatever is defined in `/extensions`

ipcMain.handle(
    'writeFile',
    fs.createItem()
)

`/extensions/fs-assistant-extension/` (infra & driver agnostic code)
- Implements the folder structure logic, i.e. `janroot/assistants`
- Implements the default global assistant

`index.ts`

import {Assistant, Store} from Core
type()
onLoad( createDefaultAssistant() )
onUnload()

async createAssistant(assistant)
    store = getStoreInstance()
    await store.createStorage(path)     // not fs.mkdir()
    await store.createItem(assistant)

`/web/hooks/useCreateAssistant.ts`

// Keep the same
import Assistant from Core

const createAssistant = extensionManager.get(ext).createAssistant()
export function createAssistant();

---

- find extension docs templates
- first section like obsidian: https://docs.obsidian.md/Plugins/Getting+started/Anatomy+of+a+plugin
- subsequent sections on extensions like VSCode
- How does the first assistant.json, model.json get created? tehy should sit in the appropriate place. is it under extensions. or in its own folder?

- "Jan's initial extensions are available as a core library @janhq/core"

### 1 Dec 2023

BaseExtesnion
- registerView
- registerSettings

core.api
core.events

Electron/invokers/app.ts: bridge btw app and electron (refactored from preload.js)
Electron/handlers

### 30 Nov 2023
- https://app.excalidraw.com/s/kFY0dI05mm/2do4B3rI2hY

Q: how to properly refactor fs?
Q: how to properly refactor events?
Q: what's the extension devex?

## Proposal

## Motivation
- @louis-jan's suggestion to go Clean is good.
- We have leaky abstractions, a result of legacy features. This can be (slowly) refactored over time.
- I suggest that we perfect the specifications ASAP, before we perfect the code.
- The resulting codebase will undoubtedly move us closer to things like a offering the world a `local-first, but truly modular framework`.

> This is a thread to kickoff this discussion && to patch my own misunderstandings.

## Possible directory structure

core/                     # The Jan SDK: @janhq/core
    domain/               # Object definitions
    application/          # Interface definitions (imports domain)

infrastructure/
    # Frameworks
    electron/
    capacitor/            # Future consideration

    # Adapters, drivers, devices
    fs/
    identity/            # Future consideration
    
web/                     # More or less the same

## `/core`
- Core, aka `@janhq/core`, is the Jan SDK
- Core has no `io` concerns (this is where we currently have leaky abstractions)
- Core has no framework concerns (we also have leaky abstractions here)
- The actual implementations for `core sdk` are injected at runtime, per `dependency inversion principle`. This means we can route to implementations inside of `/infrastructure` at runtime and not worry about infra at all at this level.

**Directory**

/core                                   
    index.ts                            
    /Domain                             # Object layer
        /Common
            objectBase.ts
            globalConstants.ts
            globalEnums.ts
            globalExceptions.ts
        
        # Sample Objects
        /Assistant                      
            assistant.ts                # Define obj: type, const, enums, exceptions, etc.
            events.ts                   # Define events, if any    
        /Store                          # Same as fs.ts
            store.ts                    # Define store
        /Thread
        /Message                        # onMessage events are moved here
            
    /Application                        # Interface Layer
        /Common
            interfaceBase.ts
            executeOnMain.ts            # Isolate the Electron specific stuff here
        # Sample Interface
        /Assistant
            assistant.ts                # Define CRUD interface
            eventHandlers.ts            # Event hander defs
        /Store
            store.ts                    # Define CRUD interface, e.g. writeObj, deleteObj
            dependencyInjection.ts      # Calls window.coreAPI?._()

## `/infrastructure`
- Contains the infrastructure-specific implementations on Jan SDK
- Framework-level: `electron`, `capacitor`, `web-only`
- Adapter/driver-level: `fs`, `sqlite` (we should add `fsAPI` to `coreAPI`)
- Adapter/driver-level: `oauth`, `saml`
- Define the cascading default behavior at runtime, i.e. how to determine what framework and drivers are available.

/infrastructure                         # Implementations
    /Fs                                 # Implements store
    /Electron                           # Current folder, but keep the os-native functions only
    /Identity

### Ref
- [Main inspo]( https://github.com/jasontaylordev/CleanArchitecture/tree/main/src)

---

https://www.youtube.com/watch?v=SxJPQ5qXisw&ab_channel=DevTernityConference

https://github.com/jasontaylordev/CleanArchitecture/tree/main/src
- has leaky abstractions though... not end of the world?

**/core**
- domain (just objects)
- common
- entities
- models
- threads
- events
- exceptions
- application (just interfaces, references domain)
- common
- interfaces
- e.g. `IApplicationDbContext` (get,set) LEAKY!
- models
- commands
- CRUD models
- eventHandlers
- queries
- threads

- infrastructure (implements the app interfaces)
- files
- identity
- persistence
- e.g. `ApplicationDbContext` (implements interface)
- services (external io)

- web (handlers, mapping UI to invocation)
-

Alternate:

- Web (entrypoint)
- Electron

- Application
- Features
- Entity

---

1. Core shouldn't have to know about `plugins` & `extensions`?
- Would Extension makers have to edit `/core`?
- `JanPlugin` -> `JanEntity`?
- Instead of `conversational.ts plugin` break it down into `threads`, `messages` **entities** instead.
e.g.

core/
    store                # fs/db abstraction
    model                # Exports type, interface, etc.
    thread               # Exports type, interface, etc.
    message
    assistant

2. Rename `fs` to `store`? Because data can be in fs or db?

export abstract class Store {
    type Store = {...};
    abstract createObject();     // Formerly writeFile
    abstract getObject();        // Formerly readFile
    abstract listObjects();      // Formerly listFiles    
}

3. What's our directory folder structure?

/core                   # Domain layer
    /entities           # Type, interface definitions
/extensions             # Application layer code
    /chat
        createThread    
        deleteThread
    /hub
        downloadModel   #
    /settings
        ...
/server                 # Traditional server framework
    /fs-adapter         # Impl of core.store.getObject
/client              # User interfaces/presentation layer
    /web             # Most of our current stuff
    /electron        # Electron specific stuff, inherits /web
    /mobile          
/data                # Data access impls
    /local           # fs stuff
    /lite            # random db example
    /remote          # random remote storage option


## Appendix

- Louis diagrams: https://excalidraw.com/#room=f6b7902dd2ca16c5aae8,EenOqcSGf0yTsUOxwM-Ebg
Architecture questions:

1. Clean should map onto folders. What’s the folder structure?

1. Take me through implementations of writing a message to filesystem. From UI component to UI component
- Do we write to file first?

Core compiles at interface level only
CoreAPI is injected at compile time by the application!

Runtime: application injects actual implementation of coreAPI.
then accessible via window.coreAPI.

Electron code is in /electron/handlers

————

1. At compile time, window is just an empty object

Moving some stuff to server

—

Electron: native only, events driven
/ fs
/ menu
/ threads
—

Server: refactor most endpoints here
/models
/start
/ threads?

——

### 29 Nov 2023
- [ ] QA https://hackmd.io/qn4CBE_zQUeElYEXuJevfg
- [ ] read: https://hackmd.io/@janhq/HylgdG4Bp
- Multiple inf engine disc: https://github.com/janhq/jan/issues/771

## Questions:
- Lifecycle of a model extension implementation (download model)
- Relative to our current codebase
- Both in Electron wrapper and Serverside-only mode

- Overview
- Overall diagram/explainers of how jan is built
- Walk through directory structure?
- Web: all user interfaces and app functionality within a single browser window should be written with the same tools and paradigms that you use on the web (from electrons perspective)
- `Caveat`: In order to directly include NPM modules in the renderer, you must use the same bundler toolchains (for example, webpack or parcel) that you use on the web.

- Files and Folders:
- e.g. https://help.obsidian.md/Files+and+folders/Accepted+file+formats
- Mention Sandboxing
– Extension Devs always assume a flat-file system, with a defined JSON
– Backend service can implement it using a DB (if they want)
– Backend service can implement using a sandboxed filesystem too
- Extensions
- Definition / diagrams
- > Extensions are similar to VSCode and Obsidian extensions. (model page: https://code.visualstudio.com/api)
- Lifecycles
- Sample instances
- e.g. https://help.obsidian.md/Extending+Obsidian/Community+plugins
- OS Modules
- Definition / diagrams
- Modules are implementations on native operating system (OS) APIs. For example, filesytem operations (link), inference operations on RAM/VRAM (link) are implemented as Jan Modules. Modules subsequently expose an interface that an `Extension` can implement.
- Lifecycles
- Sample instances

- Concepts
- Mapping concepts to Electron
- Mapping concepts to OpenAI

- User interface
- UI framework
- UIKit
- Customizing Jan: https://help.obsidian.md/Customization/Appearance

- Extensions List
- Models Extension
- Chat Extension

- Modules List
- etc

## Prereq Concepts (mostly Electron)

- `Native interfaces/APIs`: include the file picker, window border, dialogs, context menus, and more - anything where the UI comes from your operating system and not from your app. The default behavior is to opt into this automatic theming from the OS.

- `Native menu`: a custom menu bar within an Electron-based application that uses **native operating system (OS) APIs** instead of the default Chromium-based menu bar provided by Electron
- `App menus`: are menu items at the top of mac os bar when u open an app
- `context menus`: are like things copied to user clipboard, mouse gestures.

- `Main process`
- Equivalent to Chrome's main process manager: a single browser process then controls these (renderer/tab-specific) processes, as well as the application lifecycle as a whole
- Controls renderer processes
- Controls app lifecycle as a whole.
- Runs in a **Node.js** environment, meaning it has the ability to require modules and use all of Node.js APIs

- `BrowserWindow`
- Managed by `main` process.
- Each instance of the `BrowserWindow` class creates an application window that loads a web page in a separate renderer process.
- its contents are called `webcontents`
- Browser windows are `EventEmitter`!

- `Web embeds`
- If you want to embed (third-party) web content in an Electron BrowserWindow, there are three options available to you: `<iframe> tags`, `<webview> tags`, and `BrowserViews`.

- `Preload scripts`
- Main process attaches preload scripts to BrowserWindow instances
- So that renderer processes get access to node/os features

- `Node vs OS features`
- Node.js provides several runtime-specific features that are not OS-level. Some examples include:
- Event Loop: Node.js has an event-driven architecture with an event loop that allows non-blocking I/O operations, making it suitable for building scalable network applications
- Asynchronous APIs: Node.js provides asynchronous system APIs, allowing developers to perform I/O operations without blocking the execution of other code
- Worker Threads: Node.js allows the creation of worker threads, which are separate instances of the V8 engine running in parallel, enabling multi-threaded execution within a Node.js application

- `ContextIsolation`
- Context Isolation means that `preload` scripts are isolated from the `renderer's main world` to avoid leaking any privileged APIs into your web content's code.

## Appendix:
- Easy Electron / IPC overview: https://www.electronjs.org/docs/latest/tutorial/ipc
- IPC patterns and fuckery: https://www.electronjs.org/docs/latest/tutorial/ipc#pattern-1-renderer-to-main-one-way