# Web Sockets

## Goal

The goal of this document is to frame the problem and the possible solutions / mitigations of web socket routing in Alkemio.

## Problem

In Alkemio, we have 2 different orchestration engines with different networking setup. In order to make all scenarios work, we need to be able to have WS work in the following scenarios:

- client (public network) -> traefik (public network) -> oathkeeper (private network) -> alkemio-server (private network) - prod setup
- client (public network) -> traefik (public network) -> oathkeeper (private network) -> alkemio-server (public network) - dev setup

Websocket scheme is ws:// and is different from http(s)://. Websocket handshake uses both schemes! First it sends http(s):// request, then it continues on ws://. In order to be able to route the whole traffic correctly, we need a setup where both traefik and oathkeeper understand and can route web sockets.

## Orchestration Engines

### Docker-compose

We use docker-compose with 2 different setups:
- All services inside an internal network, client outside of it
- Client and server outside of the internal network

In both scenarios traefik is use to route the requests to the internal network. There are requests that are then routed back to the local network interface via oathkeeper, in the 2nd scenario*

* At the moment there is only one traefik config and in both cases oathkeeper routs to localhost. Ideally that setup should be split into two, so we can replicate production setups better.

### Kubernetes

Used in our web deployments, Kubernetes (k8s) has an ingress routing all incoming requests to different services. On top of the ingress, all requests that need to be authenticated against our GraphQL API go through Oathkeeper so they have their cookie mutated to a JWT token that our passport js implementation can use for authentication.

## Reverse Proxies

### Traefik

Traefik [should support proxying web sockets natively](https://github.com/traefik/traefik/issues/1327) without any overhead. We are on version 2+ so that issue should be resolved in our version.

### Oathkeeper

Oathkeeper [should also support proxying web sockets ](https://community.ory.sh/t/socket-server-behind-the-authkeeper-proxy/1170/2). We need to verify whether this works in our setup.

## Possible solution(s)

### k8s

- Option A
- configure separate routes for ws and gql requests
- route ws requests directly via traefik to alkemio-server. This will lack authentication of the web sockets.
- Option B
- try using the same routes for ws and gql requests (as is)
- route ws requests from traefik to oathkeeper. Have the following configuration in oathkeeper:

- id: "alkemio:graphql:protected"
  upstream:
    preserve_host: true
    url: "http://alkemio-server-service:4000"
  match:
    url: "<http|ws>://dev.alkem.io/graphql"
    methods:
      - POST
      - GET
  authenticators:
    - handler: cookie_session
    - handler: noop
  authorizer:
    handler: allow
  mutators:
    - handler: id_token
  errors:
    - handler: redirect
      config:
        to: https://dev.alkem.io/identity/login

- Option C (hybrid)
- split the ws and gql routes
- route the requests traefik --> oathkeeper --> alkemio-server
- have separate oathkeeper rule for the ws - that way we shouldn't have a regression regarding the web socket on e.g. https://dev.alkem.io/graphql, as it should use a route like https://dev.alkem.io/ws/graphql

### Docker-compose - TBD

## Notes

- it seems like most (all?) solutions proxying web sockets use the ws connection to a dedicated port, e.g. - https://github.com/lanzafame/ipfs-cluster-k8s/blob/deploy-k8s-guide/ipfs-cluster-base/cluster-service.yaml