# SQS: Separate Write and Read

## Initial Architecture

In the initial SQS architecture, we combined the SQS web server (the read side) together with the node logic (the write side) in-process.

While this proved to be an attractive architecture for the initial iteration speed, it has also caused impediments stemming from chain repository merge, branch and release processes.

Currently, each SQS change is low-risk. However, it has to go through the same expensive processes as the chain repository.

## Current Release Process

During a chain release process, an SQS change has to be backported to multiple release branches. The process entails:
1. Develop and test a change against `v20.x`
2. Create a copy PR to `main`
3. Seek approval
4. Backport to the current mainline branch `v20.x`
5. Tag and push from `v20.x` branch
6. Backport to the next branch `v21.x`

There are several steps that are redundant.

## Proposed Release Process

By moving SQS web server into a separate repository, we can shorten the process to:

1. Develop against `main` branch of `osmosis-labs/sqs`
2. Create a PR and seek approval (if necessary)
3. Tag and push

By having `osmosis-labs/sqs` pointing to the current `osmosis-labs/osmosis` release branch, we eliminate the branch management overhead.

Additionally, we reduce the need of going through a strict review process in `osmosis-labs/osmosis` by having a more flexible set of guidelines.

## Separating Write and Read

Note that we refer to SQS web server as the "read side" and the chain end-of block ingestion component as the "write side".

By moving SQS web server into its own repository, we separate the read from write.

### Shared Components

Note that the read and the write sides share common code components that are centered around the storage abstraction.

- [ingest/sqs/repository/redis](https://github.com/osmosis-labs/osmosis/tree/main/ingest/sqs/repository/redis)
- [ingest/sqs/pools/repository/redis](https://github.com/osmosis-labs/osmosis/blob/main/ingest/sqs/pools/repository/redis/redis_pools_repository.go)
- [ingest/sqs/repository/router/redis](https://github.com/osmosis-labs/osmosis/tree/main/ingest/sqs/router/repository/redis)
- [ingest/sqs/chain_info/repository/redis](https://github.com/osmosis-labs/osmosis/tree/main/ingest/sqs/chain_info/repository/redis)

Since `osmosis-labs/sqs` inevitably must import the `osmosis-labs/osmosis`, we propose to keep the repository abstraction be located in the `osmosis` repository.

The downside of this approach is that these repository changes will have to go through a chain merge and release cycle. Additionally, we will have to manage the `go.mod` dependecy in `sqs`. However, these components are not expected to be frequent so this trade-off is acceptable.

### Start-Up Logic

#### Chain Node

The chain node is going to be agnostic of SQS web server. It will configure ingestion via `app.toml`. If ingestion is enabled, the node is going to check Redis connection on start up. If Redis connection is not established, the node will fail to start.

#### SQS Web Server

Contrary to the node, the SQS web server will try to validate a connection to both Redis and chain node and start up. If either fails, the web server will fail to start. The reason for such behavior is that SQS web server has no use unless a connection is established.

However, if the connection to either is lost after the initial start-up, SQS will simply fail healthcheck.

## Deployment Updates

Currently, stage is auto-deployed from a tag of this pattern `sqs-vX.X.X` in the `osmosis` repository.

The watchtower detects the tag and auto-deploys via docker-compose.

For production, there is a manual CI job that applies a similar docker-compose file to every node.

We will need to modify the docker-compose files to start up sqs in addition to the chain node.

We will also have to build two Docker images:
1. SQS Web Server
2. Node

## Endpoints

All endpoints and APIs remain unaffected by this change.

## Service Weaver

[Service Weaver](https://serviceweaver.dev/docs.html#what-is-service-weaver) has been proposed to allow us having a monolith within the same repository while being able to deploy to various hosts with flexibiliy.

While Servive Weaver is an attractive solution for splitting a service into multiple distributed components, it does not solve the merge and release overhead discussed at the beginning of this proposal.

We have the ability to come back to evaluating Service Weaver integration in the future, once we begin working on optimizing our deployments.

## Conclusion

### Pros & Cons

#### Pros
- Flexible process management tailored to SQS risk profile and the desired iteration speed
- Fewer steps to release
- Reducing complexity of the chain repository
- Customized CI tailored to SQS
- Decoupling node from web server

#### Cons
- Some overhead with shared components between `osmosis` and `sqs` repositories
- More complex deployments
- 3 day refactor

In conclusion, separating write and read per suggestions outlined in this document will help to customize SQS-specific processes with the goal of simplifying releases.