# Design: Federation Protocol

**Author**: @juho
<!-- Status: draft, in-review, etc. -->

## Description

This proposal defines a **Federation Protocol** for FTL, allowing an FTL cluster to:
- Pull or generate **external FTL schemas** from another **realm**.
- Enable **local development** by syncing external schemas.
- Call **verbs** from an external realm without manual imports.

## Motivation

- Enables **cross-realm interoperability** between FTL clusters.
- Supports **distributed services** where different realms contribute to the overall system.
- Allows integration with **non-FTL systems** like gRPC and REST.

### **Example 1: Generating a Realm from gRPC Reflection**
Some systems expose gRPC APIs but don’t provide FTL schemas. Using **gRPC reflection**, an FTL cluster can:
- Expose gRPC service definitions as an FTL realm.
- Call remote gRPC services like native FTL verbs.
- Stay in sync with schema updates automatically.

### **Example 2: Generating a Realm from Another FTL Cluster**
For environments with multiple FTL clusters (e.g., different teams, cloud regions):
- Reference schemas and verbs from another cluster instead of duplicating them.
- Sync changes automatically when the external cluster updates.
- Enable seamless **cross-cluster service calls**.

## Design

### Schema

We shall represent external realms in the schema as top level `schema` elements. All but one realm in schema are marked as `+external` with a metadata marker.

For example, for grpc realm it could look something like

external realm bank {
  module balance {
    export verb withdraw(...) ...
  }
  ...
}

realm <cluster's realm> {
  builtin module builtin { ... }

  module echo { 
    export verb echo(echo.EchoRequest) echo.EchoResponse
      +calls balance.withdraw@bank
    ...
  }
}

The schema of an `+external` realm is fetched from a remote git repo. Each realm is reponsible syncing its schema in file in git repo representing the public API of the realm.

The endpoint the cluster uses to connect to the remote FTL realm is passed as a config value with key `<name>-realm-endpoint`

When referring to schema elements from other realms, a `@<realm>` suffix is used in the referring modules.

### Exporting verbs from FTL cluster

To support exporting verbs to be called from other realms, we change the `export` annotation to support optional export level. For example, in a Go verb, we can define a verb to be exported from the realm like this:

//ftl:verb export:realm
func Echo(...)...

We will export a public schema, which is all `export:realm` verbs and any data used by them by writing a [proto-text](https://protobuf.dev/reference/protobuf/textformat-spec/) file to the project root on every change to the public API. This file will be committed with the code changes so it can be consumed by other realms.

It should be generated by every build FTL does.

To avoid dealing with transitive dependencies, we will also add checks that the public API of a realm does not expose data types defined in external realms.

### Declaring external realms

Locally, we will declare external realms in the `ftl-project.toml` file, with a separate section for each realm.
The configuration will include the git repo to fetch the external realm schema from, as well as the commit used to fetch the schema to keep builds deterministic. For now, we assume that the `ftl-project.toml` is at the root of the external repo. The external schema file in that repo should be defined in `ftl-project.toml`

We will add a new CLI command for adding new realms:

ftl realm add --git-repo=[e-mail]:bank/accounts.git --git-branch=main

This adds an entry to ftl-project.toml for a new realm. We shall also store the latest commit from the branch in the `ftl-project.toml`

This way the config with external realms looks like

[realms]
  [egress]
      [accounts]
        git-repo = "[e-mail]:bank/accounts.git"
        git-branch = "main"
        git-commit = "aac675450010d2b8fc0528239e49e56fd94b5309"
      [taxman]
        ...

To update the commits to the latest, we will add a command

ftl realm update

Finally, we shall add commands for managing realms like `ftl realm list` and `ftl realm remove`

The realms are placed in `[egress]` block to allow us to record schemas of the calling services in the future. Though, not implemented now, in the future we can also write the calling schema to the target repo when adding a realm dependency to maintain 2-way graph of schemas across realm boundaries.

### Local dev experience

For now, any calls to a remote realm locally will fail, but we can make the DX for this better in the future by allowing fake or remote connection based development against these APIs.

### Connectivity

When a verb in FTL cluster calls an external FTL verb, it is routed to a remote `ftl-ingress` service for that realm. The ingress service then routes the call to the runner of the verb.

### Non FTL realms

For any realm not implemented in FTL (like external APIs or a set of gRPC services), the following are needed:
- an FTL proxy working as the `ftl-ingress` endpoint for any external calls. This proxy needs to be able to translate FTL calls to any native format that might be used internally in the realm.
- a git repo with up to date public schema for the realm. For example for gRPC, we need a process that reads the reflection edpoints, or original proto files, and translates those to FTL schema that is then committed to the repo where FTL clusters can consume it

### Runtimes

The runtimes need to be changed to take the realm name into account when generating clients, by using the realm appropriately in the client package path.

### Diagram


## Rejected Alternatives

### Manually Importing schemas to local modules
We could ask users to import proto files for remote grpc services or FTL modules they want to call as local modules.
However, this would be too much maintenance for engineers.

### Updating the schema of external clusters between the cluster directly
The ftl clusters could sync the schema of remote clusters at runtime, but that would require connectivity to a cluster for developers for local development, and would make dealing with breaking changes difficult.