Design: Profiles

# Design: Profiles **Author**: @aat  ## Description (what) Introduce the concept of profiles to replace `ftl-project.toml`, similar to how profiles work in a number of other similar systems such as [kubectl](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/) contexts. FTL profiles will differ in some ways though, as they will be designed to be checked into a repository. ## Motivation (why) - The current mechanism for switching between contexts is via `--config`, which is both tedious and easy to forget to use. - Because the FTL tooling is only aware of a single configuration file at once, there's no way to build tooling for working across multiple profiles, such as eg. copying secrets/config between profiles, etc. - Supporting multiple config/secret providers per config is confusing. ## Goals - Make it easier to reason about managing multiple clusters. - Simplify using different combinations of features/configuration: - Local vs. remote cluster. - Local vs. remote secrets/config. - Locally, support offline management of secrets/config. - Simplify managing secrets/config: - Remove the ability to mix/match secrets/config providers within a single cluster. Each cluster, whether local or remote, has a single provider for each type. ## Non-goals - It's unclear whether we should have a first class concept of "test" or "integration-test" profiles, but we'll see how that falls out. ## Design (how) Profiles will specify the cluster endpoint, whether it is ephemeral (`ftl dev`) or persistent, and for ephemeral clusters the profile will also specify the secret+config providers to support "offline" management. There will always be a "default" profile which will be used if no other profile is specified. The default profile can be overridden locally (not persisted to git) by issuing a command. Tests will always use the "default" profile unless otherwise specified. ### Code Management of `.ftl-project` will be isolated to an internal package. The public API wrapping this will return a `Profile`, probably an interface with local+remote implementations. For local clusters, the `Profile` will either update secrets/config through the cluster itself if available, or directly if the cluster is not live. The `Profile` interface will look something like this: ```go type Profile interface { // ProjectConfig is static project-wide configuration shared by all profiles, such as realm, module roots, etc.. ProjectConfig() ProjectConfig // ProfileConfig is the static configuration for a Profile. // // This includes the name, endpoint, etc. ProfileConfig() ProfileConfig SecretsManager() *manager.Manager[configuration.Secrets] ConfigurationManager() *manager.Manager[configuration.Configuration] } ``` ### Filesystem layout Currently for local development, secrets+config are tightly coupled with the `ftl-project.toml` file. The new design will separate these into distinct files. To support this we'll introduce a directory hierarchy somewhat like this: ``` .ftl-project/ project.json profiles/ <profile>/ profile.json [ secre] [config.json] ``` `profiles.json` will contain all profile definitions. ### CLI #### Set project-wide configuration ``` $ ftl init --no-git --realm="NAME" --module-roots="DIR,..." ``` #### List profiles List available profiles, whether they're default, local or remote, what secrets+config providers they use, etc. ``` $ ftl profile list ``` #### Set the default profile This will change the _persisted_ default profile: ``` $ ftl profile default <profile> ``` #### Switch profiles Switch the currently active profile on your machine: ``` $ ftl profile switch <profile> ``` This command will write the profile name to `$CACHEDIR/ftl/<project-path-hash>/default-profile`, eg. for a project in `/Users/alec/dev/project` this would end up in `~/Library/Caches/ftl/c0f1410a1a21a4cd452b6097e663057ee6622d88/default-profile`. #### Create a new remote cluster profile ``` $ ftl profile new <name> --remote=<endpoint> ``` #### Create a new local cluster profile ``` $ ftl profile new <name> \ --local \ --secrets=<secret-provider> \ --configuration=<config-provider> ``` Local profiles will default to inline secrets and configuration providers. eg. Create a new local "devel" profile with inline secrets+config: ``` $ ftl profile new devel --local ``` Create a new local "smoketest" profile with secrets stored in 1Password: ``` $ ftl profile new smoketest --local --secrets=1password ``` #### Cloning a profile ``` $ ftl profile clone <from> <to> ``` #### Diffing two profiles ``` $ ftl profile diff <a> <b> ``` #### Dump the state of a profile ``` $ ftl profile dump <profile> ``` ### Required changes (how) - Create a `configuration.FileResolver` which stores secrets/config in `.ftl-project/<role>/<profile>.json` - Add package to manage profiles. - Remove `configuration.ProjectConfigResolver`. - Update go-runtime to support profiles. - Replace `--config` flag with `--profile` for overriding the profile per-command. - Add `ftl profile` commands (see above). - Add `RemoteRouter` + `RemoteProvider` that use gRPC to a cluster. - Refactor `AdminService` so there is no ability to select a provider. - Finally, refactor `Manager` so it only supports a single provider. ## Rejected Alternatives (optional)