# Other

> [TOC]

A previous approach attempted was to do an hybrid of status-go and nim-status.
This post is to provide some context on past and current approaches and goals for nim-status.

## Initial Roadmap and Context

Original Roadmap:
* Create a shim in nim-status connecting to status-go with 1-to-1 mapping.
* Desktop AND Mobile using nim-status.
* Development of new features in nim-status instead of status-go.
* Integrate nim-waku in nim-status in such a way a client app can use it.

Some of the original context as well:
* Desktop team was still learning Nim (and Qt) and getting more familiar with systems programming.
* Desktop team getting familiar with status-go & co.
* Meant to be 4-8 devs from different teams, in practice around 2 or 3 devs (or just 1 at times due to the Nix stuff).
* Achieving feature parity in desktop with mobile was very high priority.
* Desktop achieved feature parity around December 2020.

## Development done

Several status-go APIs & functionality were implemented, some examples include Status Accounts, Contacts, Mailserver, Usernames, Identicons, Chat & Messages (& related features), Contact Management, Wallet accounts, tokens, transactions, dapp permissions, web3 provider, etc..
Other functionality needed such as nim-sqlcipher, Protobuffers.
Also work done on NIM's concurrency model, as well as the NIX integration to be able to integrate with status-go & status-react.

## Main obstacles with the main approach (Hybrid)

### Nix and integration with nim-status + status-go

In order to make this library work with mobile (status-react) we tried immediatly to integrate it into status-react's build system so mobile could use it as soon as possible; this effort was originally expected to be quick (1-2 weeks) since after all it's "just" compiling this as a C library, in practice it took several months (+5) due to:
* Integration with Nix scripts turned out to be complex.
* Weird status-go flags to deal with.
* Lack of familiarity with Nix and learning curve.
* Platform specific issues and gotchas (i.e Android, iOS).

### status-go flags
One should set `GOOS` and `GOARCH` environment variables properly before invoking `go build`.

Nix build script is here: https://github.com/status-im/status-react/blob/feature/nim-status-thin-wrapper/nix/nim-status/status-go.nix

#### Android
- Have to use shared library. Pass `-Wl,-soname,libstatus.so` to `CGO_LDFLAGS`.
- Therefore, pass `c-shared` as `-buildmode` flag to `go build`.

#### iOS
- Using static library.
- Therefore, pass `c-archive` as `-buildmode` flag to `go build`.
- Also pass `-tags ios` to `go build`.

### nim-status flags

`config.nims` has to be set up properly. Example:

switch("passC", "-isysroot <path_to_Android_NDK_sysroot> -target aarch64-linux-android23 -fPIC")
switch("passL", "--sysroot <path_to_Android_NDK_sysroot> -target aarch64-linux-android23")
switch("cpu", "arm64")
switch("os", "android")

put "arm64.android.clang.path", "<path_to_Android_NDK_clang>"
put "arm64.android.clang.exe", "clang"
put "arm64.android.clang.linkerexe", "clang"

With these settings, one should also pass `--cc:clang` to `nim c`.

Nim compiler needs `-d:nimEmulateOverflowChecks` flag,
otherwise compiler will complain about undefined nimMulInt/nimAddInt functions (https://github.com/nim-lang/Nim/issues/13645#issuecomment-601037942, https://github.com/nim-lang/Nim/pull/13692)

We have to provide proper `ar` and `ranlib` locations before invoking Nim's cross-compiler (https://github.com/status-im/status-react/blob/900d8d1f0322d4086cafdd0aeffb5056743019c1/nix/nim-status/getFlags.nix#L89). Simply setting `AR` and `RANLIB` environment variables does not seem to work. There is no relevant `config.nims` setting, at least to my knowledge.

#### Android issues
- `-fPIC` compiler flag is required.
- Target-specific (i.e. x86/arm/arm64) binaries should reside in separate folders, as listed here: https://github.com/status-im/status-react/blob/feature/nim-status-thin-wrapper/nix/nim-status/buildAndroid.nix. This also applies to binaries generated by status-go build.

#### iOS issues
- `-miphoneos-version-min=8.0` compiler flag is required.
- Need to create a single multi-arch fat binary using `lipo` tool. Details here: https://github.com/status-im/status-react/blob/feature/nim-status-thin-wrapper/nix/nim-status/buildIos.nix. This also applies to status-go build.

### Concurrency story in Nim w.r.t. status-go

Concurrency mechanics need to be considered carefully: there are issues w.r.t. to the hybrid approach, issues which any Nim library would encounter integrating with status-go or status-react.

- status-go's APIs are generally synchronous (from the perspective of Nim code linking to `libstatus.a|dll|dylib|so`) and don't involve passing around an instantiated "status-go object".
- The Nim shim for status-go is simply imported wherever it's needed in the status-desktop codebase and the shim's synchronous procs are invoked with the assumption that status-go has been started on app login.
- Much of the status-desktop codebase was written without awareness of how integration with nim-status + nim-waku would have to work differently, because the desktop team didn't know better at the time.
- nim-waku uses nim-chronos and has asynchronous functions, so the thread on which nim-status + nim-waku lives must run the chronos event loop, which cannot work correctly if another event loop needs to run on the same thread, e.g. Qt's event loop or Nim's `asyncdispatch` event loop.
- Therefore status-desktop must run nim-status + nim-waku on its own thread and use immutable-message passing to interact with it indirectly.
- There are no built-in Nim facilities that make bidirectional message passing and coordination across threads easy or simple. The default memory model of Nim does not allow any sharing across threads (reading or writing) of any memory that is subject to garbage collection (locks/mutexes are ***not*** a solution; the problem runs deeper and is tied to ref counts of Nim's default per thread GC).
- In addition to a multithreaded immutable-message passing architecture being a necessity for using nim-status + nim-waku in status-desktop (a challenging problem in its own right), it's especially difficult to "dogfood" nim-status + nim-waku in the desktop app in an incremental fashion (as originally intended) because the status-desktop codebase is replete with synchronous calls to status-go that form a dependency chain.

### Sqlcipher database issues

Turns out status-go uses sqlcipher v3 and nim-status uses v4, which caused at first some mysterious issues as the two aren't compatible, furthermore there are difficult problems when attempting to have concurrent connections to the same database from two different libraries simultaneously.

## Integrating nim-waku into nim-status/status-go and desktop/mobile

Several approaches were explored:

### Approach 1 - integrate nim-waku with nim-status, interact with status-go (Hybrid)

This is the Hybrid approach originally considered for nim-status project. Besides the difficult Nix issues, some other issues are the handling of async code in Nim, as well as account management, since both status-go and nim-status would require access to user accounts, which is further complicated by concurrency issues as well as incompatible database versions.

This would also have required modifying status-go to uncouple functionality so the client is forced to use the nim-status API, and also so status-go does not duplicate the work of encoding/decoding messages and inserting info in the database.

### Approach 2 - integrate nim-waku with status-go through nim-status by intercepting certain apis

In theory it makes sense and is simple, but in practice it’s quite complex because of sync vs. async, and in the context of status-desktop the need to run nim-status + nim-waku on a separate thread. See *Concurrency story in Nim w.r.t. status-go* above.

There are no functions that we can use to simply hook nim-status' independent implementations of status-go's equivalent functionality to be replacements for status-go's implementations. status-go’s architecture deals with the concept of services which should follow a specific interface to interact with any transport protocol.

### Approach 3 - use nim-waku as lib in status-go

This would involve compiling nim-waku as a library and using it in status-go:
* Implement the api in `go/protocol/transport/wakuv2`.
* Leverages what was learned and already done in nim-status in regards to using the nimbus-build-system with Nix and incompatibilities of certain libraries like secp256k, sqlcipher, etc.

Unknowns / possible issues:
* Possible async issues or concurrency issues to take into account (unknown).
* Versions of libraries like secp256k1 need to match between status-go and nim-waku.

See *Concurrency story in Nim w.r.t. status-go* above.

### Approach 4 - integrate nim-waku with nim-status and replace status-go completely

Exploration of other approaches (in particular Approach 1) showed that in practice it’s hard to have a nim implementation side by side with status-go to concurrency issues, in particular accessing the database at the same time will cause crashes.
It is far easier to do a "clean state" approach and start with a well thought-out Nim implementation instead of trying to fit the existing mold. However as method to integrate nim-waku this would have taken longer too.

### Choosen approach

In a meeting with members of the mobile, nim-waku, nimbus, and desktop teams, a 5th approach was suggested by nimbus:
* to write a port in go and integrate with status-go directly as this had the least unknowns as concluded in the meeting.
* Continue nim-status as a pure nim implementation and not as a hybrid with status-go. (i.e Approach 4)

### Current State

**go-waku**
Mostly done. Turned out it was much faster to do than originaly thought (we succesfully achieved interop with nim-waku chat2 example), the next steps are to create a bridge and do some initial testing with branches of desktop and mobile using this status-go branch. This branch is still pending review by core team to determine its viability as well as changes to increase code quality.

https://github.com/status-im/status-go/tree/go-waku-store

The following protocols were implemented in go-waku and kept up to date with latest changes in nim-waku.
- Relay
- Store
- LightPush

From these, LightPush is missing integration with desktop. Work is being done in desktop to add a switch to toggle between waku versions.

<video width="740" height="480" src="https://i.imgur.com/wZL2D99.mp4" type="video/mp4" controls />

Things pending to determine are how will peer discovery work with desktop/mobile. WakuRelay does not describe how new peers should be discovered.

**nim-status**

Using Approach 4, Nim-status now continues as a replacement for geth but this time as a pure nim implementation, and will reuse most of the code already done previously that implements a lot of the required functionality. To speed up development the library will be developed with a very simple command line client, and later integrated into desktop/mobile. Goals & Notes can be found [here](https://github.com/status-im/nim-status/discussions/171)