Author: Stuart Douglas Description (what) We should support Verbs that can handle a stream of data, so that the full request / response does not need to fit in memory. Motivation (why, optional) At present FTL can only handle data that arrives in a single request and fits enteirly in memory. There are many use cases that require streaming data (e.g. downloading large files). This will be particularly important for AI which relies on large datasets that are likely to large to handle with our current verb architecture. A core use case for this is to handle large HTTP requests / response that we don't want to load into memory, or long running websocket requests.

Author: @worstell Description Attach/propagate headers (metadata) through verb calls. Headers can be specified at verb, module, or realm level, with automatic propagation through the call chain based on the specification level. For HTTP ingress calls, selected headers are captured and automatically forwarded downstream; the same logic applies to verb-to-verb calls. Goals Support header propagation with the following levels of granularity:Verb level—selected headers are captured through the inbound request and automatically forwarded downstream to any calls that occur within the annotated verb's call lifecycle. Only response-specified headers are returned in the verb's final response Module level—selected headers are automatically propagated through all verb calls in the module; can be specified for propagation on requests and/or responses Realm level—same as module-level, scoped to the entire realm

Author: @juho Description This proposal defines a Federation Protocol for FTL, allowing an FTL cluster to: Pull or generate external FTL schemas from another realm. Enable local development by syncing external schemas. Call verbs from an external realm without manual imports. Motivation

Author: Stuart Douglas Description (what) Egress allows users to specify remote endpoints they need access to. FTL will provision appropriate policies to allow this access. Goals Allow the end user to specify egress in a way that is noted in schema metadata Allow the engress endpoint to be configured differently in different environments Allow the end user to inject the egress URI into their verbs

Author: Stuart Douglas Description (what) Workload identity is how a module can identify its caller. Goals A FTL module should be able to securely determine which service it is being called by This should apply to both inter-module FTL calls and ingress, however ingress is out of scope of this initial design SPIFFE should be used as the identity representation

Author: Stuart Douglas Description (what) Fixtures will allow users to pre-seed the system with values in dev mode, to allow things like default users etc to be setup. Motivation (why, optional) End users will not want to start from a blank database each time, our lack of fixtures will be one of the first things end users run into when writing applications with databases. Goals Allow for the FTL dev database to be pre-seeded with data

Author: @worstell Declaring Databases Databases are declared using discoverable file locations with a required hierarchy: Go db/ ├── mysql/ # must be exactly "mysql" or "postgres" │ └── mydb/ # database name

Introduction FTL is a framework for building distributed systems that are safe to operate and easy to reason about. How is it different? Language agnostic, code first Whole-system static analysis Infrastructure as code, not YAML Fearlessly modify types Incredibly fast development cycle

Author: @aat Description (what) Refactor the remaining backend services that are in the Controller. Motivation (why, optional) There are a few architectural issues remaining: We've now split the Controller into multiple distinct services by responsibility, but we're left with some random functionality here and there that doesn't cleanly fit into any particular bucket. The CLI currently communicates directly with the Controller, Admin and Schema services.

Author: @worstell Cron Requirements: Execute scheduled jobs exactly once Distribute jobs across multiple nodes (balance load) Support automatic failover Raft Integration: Sharded groups to distribute load

Author: @mtoohey Description (what) Deploying modules to a cluster will involve a new process where the a changeset is created that tracks the progress of the change to the cluster. Motivation (why, optional) The schema should model the truth of the cluster. The reality of the cluster is that the it is in flux as deployments are created. We want a clear view to see what is changing in a cluster and it's state.

Author: @worstell Where GetUser is the verb generated from SQL and Caller is a verb that injects this query. Option 1: module Foo { database postgres mydb +migration ...

Author: @mtoohey Description Reimplement FTL's PubSub using Kafka, with topics, partitions and consumer groups. Motivation FTL's PubSub needs to be highly performant and scalable. Goals High performance

Controller Strong consistency.

Author: @aat Description (what) Some options for how we manage runtime metadata in the schema. Motivation (why, optional) We are moving everything into the schema, so we need an extensible and scalable approach to handling runtime metadata. Runtime metadata is information in the schema that exists for the use of runtime systems, and is not part of the static schema type system. Goals Handle multiple systems updating distinct runtime metadata for a single node.

Author: @sdouglas Description (what) Currently all trafic is routed through the controller. Peer to peer routing will change this so that runners can talk directly to each other without involving the controller. Motivation (why, optional) There are a few main reasons for doing this: Performance, it is faster to invoke runners directly rather than adding a proxy step in the middle Security, if runners talk directly to each other they can leverage security features of Istio such as mTLS and auth policies. This will allow for workload identity to be provided via mTLS, and also allow us to use Istio to limit access.

Author: @worstell Description Convert database schemas and queries into FTL types (queries as Verb and models as Data), integrating SQL into the FTL type system. Motivation Ensuring type safety, reducing boilerplate, and simplifying development against DBs. This feature also lays the groundwork for enforcing redaction and encryption policies throughout the FTL ecosystem. Goals Provide ergonomic, type-safe interaction with DBs

Author: @sdouglas Description (what) This document describes the initial design for database migrations in FTL. Motivation (why, optional) FTL needs migrations as part of it's provisioning support. Goals Define how basic dbmate migrations will be handled by FTL.

Author: @jvmakine Description Currently, we store full database DSNs including usernames and passwords to an ftl managed secret under the module accessing the database. However, this is not very secure, and we need to figure a way to connect to a DB while minimizing the risk of exposing the credentials. Motivation There are multiple ways platforms can manage DB credentials. Some of these are: No rotation at all