# Handling exceptions in nim-status

For developers who are intersted in attempting to contribute to nim-status and need a better understanding of how nim-status handles exceptions, please read on.

## Structure
The strucutre of `nim-status` is organised as follows (showing only the most relevant files/directorties):

├── examples
│   ├── client
│   └── waku
├── status
│   ├── api
│   ├── api.nim
│   ├── migrations
│   └── private
├── status.nim
└── test

- **examples** - Contains a client application that uses `nim-status` as a library, [example chat client](/LPd8cG9jSS-y_Rk7L8ogIQ), as well as a self-contained version of the [waku v2 chat client](https://github.com/status-im/nim-waku/blob/master/examples/v2/chat2.nim).
- **status** - The `status` directory contains the guts of `nim-status`. It has two directories that are important to understand: `api` and `private`. All **public-facing** API modules live in the `status/api` directory, while all internal modules live in the `status/private` directory.
- **status.nim** - Contains the top-level import file, which simply re-exports `status/api.nim`, which in turn simply re-exports all the API modules contained in `status/api/`. This means that consumers of `nim-status` using the `nimbus-build-system` can simply use `import status` to import all API modules, but *it's recommended to only import the needed modules, ie `import status/api/wallet`.*
- **test** - Contains nim-status tests. There are some examples of how to consume certain modules and APIs, however the tests are in need of some love.

## Public-facing APIs versus private modules
As a consumer of nim-status, in the majority of cases, the APIs exposed in the `status/api` directory will be consumed directly. These API modules consume internal private modules, but those internal modules are only re-exported if they are needed by the caller. So if a caller imports an API module, ie `import status/api/wallet`, that should include everything needed to execute API calls and work with the returned types.

If there is a needed module in the `status/private` directory, it can be imported directly, but it is discouraged as the API modules should be responsible enough to re-export any needed modules to work with the APIs and their return types. The public-facing APIs are simply a collection of internal module calls, wrapped up in a `Result[T, string]` type, so if there is a custom implementation that is needed that is not satisfied by the public-facing API modules, then it is possible for consumer to import private modules and roll their own implementation. Note that private modules will raise exceptions that will need to be caught and handled properly, while public-facing API modules will return `Result[T, string]`. There is, of course, nothing stopping custom implementations from returning custom `Result[T, string]` types, following the pattern set forth by the public-facing API modules. More in the next section.

For compilation efficiency, it is import to import API modules individually as needed (ie `import status/api/auth`), instead of importing all API modules using the top-level `import status`, which is equivalent to `import status/api`, which imports all API modules in the `status/api` directory. The latter will import the needed `auth` module, but it will also import all the other API modules, which in turn import and re-export many other modules. It's best to avoid unneeded imports to keep compilation times down.

## Exception/Defects and Result types
Any public-facing API module in `nim-status` will return a type that is equivalent to `Result[T, string]`, where `T` is the type being returned, and `string` is the error type. Internal/private modules will raise exceptions, with the exception type being determined by the layer, which also must inherit from `StatusError`, exported from `status/common.nim`.

We strive to catch exceptions raised by the *lowest layer* possible, and then translate them in to a layer-appropriate exception. If the layer is a public-facing API layer, it will be translated to a `Result[T, string]` type. Any defects cannot be relied upon to be caught, as they are by definition meant to be raised in panic situations, so we will simply declare them in the `{.raises:[].}` pragma to inform the compiler exception analysis of a possible `Defect` that may be raised.

## Catching exceptions by "layer"
As an example of how to catch exceptions by "layer", let's look more closely at `status/private/contacts.nim` which handles DB persistence of contacts.

### Handling the lowest layer exceptions
At the lowest layer, we are wrapping libs and modules that are vendor depdencies of `nim-status`. We cannot control the exceptions raised by these modules nor can we control their compilation analysis of exceptions raised (in other words, we can't change their `{.raises:[].}` metadata. Look at the `getContacts` proc of `status/private/contacts.nim`:

# status/private/contacts.nim
proc getContacts*(db: DbConn): seq[Contact] {.raises:[ContactDbError,
  Defect].} =

  const errorMsg = "Error getting contacts from the database"
  try:
    var contact: Contact
    let query = fmt"""SELECT *
                      FROM {contact.tableName}"""
    result = db.all(Contact, query)
  except SerializationError as e:
    raise (ref ContactDbError)(parent: e, msg: errorMsg)
  except SqliteError as e:
    raise (ref ContactDbError)(parent: e, msg: errorMsg)
  except ValueError as e:
    raise (ref ContactDbError)(parent: e, msg: errorMsg)

Here, there are three errors caught, which are all translated in to a `ContactDbError` with the original exception being set as the `parent` of `ContactDbError`, so that exception stack tracing can be inspected when needed. The `SerializationError` and `SqliteError` both are raised by `db.all`, which is an vendor dependency, thus at the lowest layer that we can access in `nim-status`. The query formatting string `fmt"""SELECT * FROM {contact.tableName}"""` raises a `ValueError`. This comes from nim's `strformat` module, which again, is a dependency that we cannot control.

All three of these exceptions inherit from `CatchableError`, so we know that catching them here can be relied upon, and we can provide information to consuming layers above that will call `getContacts`, such as the public-facing APIs.

Notice the `{.raises: [ContactDbError, Defect].}`. This tells any consumers of this proc that there are two types of `Exception` that could be raised at the call site: `ContactDbError` and `Defect`. There is a `Defect` being raised in this proc from a lower-level, but we are going to let that bubble up untouched as we are not meant to be able to recover from a raised `Defect`. For `ContactDbError`, it inherits `StatusError` which inherits `CatchableError`, so callers of this proc will know that they will need to catch any `ContactDbError` exceptions so that the application will not crash unexpectedly.

### Handling the next layer

We may want to expose a public-facing API in `status/api/contacts.nim` (doesn't exist yet) to be able to give consumers access to `getContacts`. This proc will call a lower layer of `nim-status`, namely the `getContacts` proc from `status/private/contacts.nim`. To do this, we'd create the following:

# status/api/contacts.nim
{.push raises: [Defect].}

import # vendor libs
  stew/results         # exports Result type

import # nim-status modules
  ../private/contacts, # exports our getContacts proc
  ./common             # API common procs, exports StatusObject
  
export contacts, results

type
  GetContactsResult = Result[seq[Contact], string]

proc getContacts*(self: StatusObject): GetContactsResult =

  const errorMsg = "Error getting contacts: "
  try:
    let contacts = self.userDb.getContacts()
    return GetContactsResult.ok contacts
  except ContactDbError as e:
    return GetContactsResult.err errorMsg & e.msg
  except StatusApiError as e:
    return GetContactsResult.err errorMsg & e.msg
    

All lower layer exceptions from the `getContacts` proc have been wrapped up for us in to `ContactDbError`, and so we need to catch that exception. Additionally, access `self.userDb` could potentially raise a `StatusApiError`, which we also need to catch. Because we are dealing with a public-facing API proc that end consumers (ie chat clients) would consume, we need to return a `Result[T, string]`, so we've created a `GetContactsResult` type, which returns a `seq[Contact]` if the call was successful or a `string` if there was an error.

Notice too, that we export `contacts` such that consumers of this module can get access to the `Contact` type, part of the `seq[Contact]` type returned from `getContacts`.

### The final, consumer layer
The final layer is the layer of the consumer, typically a client like a chat client. The example chat client in the `nim-status` repo is exactly that: an example of how clients should consume `nim-status`

Let's say we have a file in our client called `tasks.nim`, note that this proc should be consumed inside of a task thread, such that it does not block the main thread (see the example chat client on how to do that using `nim-task-runner`. We would be able to consume our `getContacts` proc like so:

{.push raises: [Defect].}

import # nim libs
  std/json
  
import # vendor libs
  status/api/contacts # only need to import the contacts module
                      # for this call

proc printStatusContacts() =

  let
    status = StatusObject.new("dataDir")
      # A StatusObject should be instantiated in tasks already and made
      # accessible to this proc in a thread-safe manner. This is purely
      # for illustration.
    getContactsResult = status.getContacts()
  
  if getContactsResult.isErr:
    # handle the error and return early
    echo "Error encountered while getting status contacts: " &
      getContactsResult.error
    return
    
  let contacts = getContactsResult.get
  echo "Returned contacts: ", (%contacts).pretty
  

## Conclusion
The exception-handling architecture for nim-status can be summarized as follow: any public-facing API will return a `Result[T, string]`, while any internal modules will raise layer-specific exceptions, and each layer above will handle those exceptions, translating them in to layer-specific exceptions.