# Design: Peer to Peer Routing **Author**: @sdouglas ## Description (what) Currently all trafic is routed through the controller. Peer to peer routing will change this so that runners can talk directly to each other without involving the controller. ## Motivation (why, optional) There are a few main reasons for doing this: - Performance, it is faster to invoke runners directly rather than adding a proxy step in the middle - Security, if runners talk directly to each other they can leverage security features of Istio such as mTLS and auth policies. This will allow for workload identity to be provided via mTLS, and also allow us to use Istio to limit access. ## Goals - Verb to verb invocations will be handled by the runners talking to each other directly - The HTTP ingress endpoint will talk directly to the runners ### Non-Goals (optional) - No changes to the security model yet ## Design (how) - The ModuleContext will be extended to include information on the routing table. This information will be a list of routing table entries that contain the module name and endpoint URI. If future this may be expanded to also include an optional verb name, the initial implementation will just used module based routing for simplicity. - All communication from the language runtime will now be routed through the runner. The runner will bind to localhost to enable the user application to connect to it, and also bind to a public endpoint to allow for incoming verb invocations. As both endpoints will bind `VerbService` this allows the runner to know if an invocation is an outgoing one that needs to be routed to a peer, or an incoming one to be routed to the application. - The controller will calculate the routing table for a module by looking at it's `+calls` metadata. Only modules that are listed as being called will be added to the route table. This routing table will be added to the `ModuleContext` and streamed to the module as changes are made. - When new versions of a module are deployed and ready the controller will push route table updates out to the runners. - Runners will be responsible for adding their own timeline service entries on call, once the timeline service has been refactored to allow this. For now peer to peer routing means we may loose some timeline events. Once this is ready runners should add all 'inbound' events to the timeline, so all calles are logged at the recieving runner rather than at the call site. - Kube deployments will be changed to use a ServiceAccount per module rather than a ServiceAccount per deployment. As security rules are expressed in terms of ServiceAccount this will reduce the churn in the security rules. - The provisioner will create Istio authorisation policies that allow peer to peer access to modules that are declared to be called in the schema. - Initially system services that make calls such as Cron and HTTP Ingress will continue to route through the controller. This will likely be address in follow up work.