# FTL command-line interface **Author**: Wes **Status**: ==draft== / in-review / accepted / rejected ## Description ## Motivation (optional) We want to make the usage of `ftl` easy for people to understand and get help. ## Goals - Ability to build without running the FTL server. - Parallel builds. - One shot builds - exit once all modules are built. - Parallel deploys. ## Use cases There's a bit of confusion now with `ftl serve` and `ftl dev` where things are still running from previous deploys or development cycles ### Development This is the primary interactions most developers will have with FTL during day-to-day development > Discussion: Should this replace other commands like `ftl serve`? ``` ftl dev <dirs>... ``` This will start the ftl cluster, build and deploy all modules in the specified dirs, and hot reload modules changes as needed. Long-term this will also include secrets, database migrations, etc.. ### Testing/CI Should be able to build and deploy everything and then run tests against the running cluster. > Maybe this needs attention. Do we like this solution? The current solution is something like this: ```sh ftl serve --background --recreate ftl deploy backend/modules # run tests here... ftl serve --stop ``` ### Discrete commands Primarily used to break down other commands into discrete steps for diagnosing issues when needed or interacting with production clusters. #### ftl build ``` Usage: ftl build <dirs> ... Build all modules found in the specified directories. Arguments: <dirs> ... Base directories containing modules. ``` #### ftl deploy ``` Usage: ftl deploy <dirs> ... Build and deploy all modules found in the specified directories. Arguments: <dirs> ... Base directories containing modules. ``` #### ftl serve > Maybe we get rid of this altogether? ``` ftl serve --background/--stop (maybe --recreate) (in separate process?) ``` ## Future use cases - Developing single module locally while running against production cluster ``` ftl fork/branch? <module(s)> //create new branch of production ``` ## Non-Goals (optional) - ... ## Design #### Build The `ftl build` command can be used to build an FTL module locally or in a CI environment. This command will build a single module at a time. For building and deploying multiple modules, see `ftl deploy`. ``` ftl build --help Usage: ftl build [<module-dir>] Build an FTL module. Arguments: [<module-dir>] Directory containing ftl.toml ``` #### Deploy Questions: * Do we have a use case for deploying all modules? Seems like it would be useful for CI/testing to deploy everything. However, this will limit the options for a single deploy for things like replicas, etc. Maybe this would be different command? Or a flag on `ftl deploy`. * ``` ftl deploy --help Usage: ftl deploy [<base-dir>] Create a new deployment. Arguments: [<base-dir>] Directory to scan for FTL modules. All modules found, will be built and deployed until everything reconciles (or fails). Flags: --replicas --no-wait Do not wait for the deploy to finish before exiting. ``` #### Dev > specify list of directory `ftl dev` should be a one-stop-shop for developing with FTL. Ideally, the typical developer workflow would be to open a terminal and run just `ftl dev .` which would do the following and more: * Start the FTL cluster * Build, deploy, and hot reload modules * Provide infra such database, migrations, configuration, secrets, etc. ``` ftl dev --help Develop on FTL. This command will start the FTL cluster, build, deploy, and hot reload modules with their dependencies. Arguments: [<base-dir>] Directory to watch for FTL modules Flags: --watch=500ms Watch template directory at this frequency and regenerate on change. --reconnect-delay=1s Delay before attempting to reconnect to FTL. ``` Deprecated flags: ```bash! # Do we need this failure delay? Seems like we would just not rebuild until something related to the failing module changes. Either a schema or filesystem change. --failure-delay=500ms Delay before retrying a failed deploy. # It feels like `dev` is not the right place to have exit after deploy. Dev should be started and left running while developing. Similar to `npm run dev`. --exit-after-deploy Exit after all modules are deployed successfully. ``` > Notes - Break the build system out into its own top level package in the FTL repo. - (1) Add a system for building and optionally deploying a single module - extract schema, (optional?) build, (optional) deploy - (2) Another system for repeatedly building in parallel until everything reconciles. - (3) Live reloading system which will restart builds for individual modules. - One of these systems needs to understand schema dependencies and which modules to rebuild when modules change. - Must support "build and exit", probably just calling (2)? - We will need a bounded work queue to avoid killing the users computer. - We'll want a flag to control concurrency. - Ideally each build system can feedback how concurrent it is. eg. if each Go build uses 10 cores, we don't want to build 10 Go modules at once. - Alternatively, have a way to limit the concurrency of each build. #### ftl serve This command is used to start the FTL cluster. It can be used in local or production environments. For local environments, `ftl serve` will automatically start and remove both controllers and runners as needed. In production environments, this will be managed by k8s or something similar. ```bash ftl serve --help Start the FTL cluster flags: --bind=http://localhost:8892 Starting endpoint to bind to and advertise to. Each controller and runner will increment the port by 1 --allow-origins=ALLOW-ORIGINS,... Allow CORS requests to ingress endpoints from these origins ($FTL_CONTROLLER_ALLOW_ORIGIN). --db-port=54320 Port to use for the database. --recreate Recreate the database even if it already exists. -c, --controllers=1 Number of controllers to start. -r, --runners=0 Number of runners to start. ``` ## Previous discussions... #### ftl build Build a single FTL module. Will look for and use the `ftl.toml` file in the specified `module-dir` to determine which build command to use. ```bash ftl build --help Build an FTL module. Arguments: [<module-dir>] Directory containing ftl.toml ``` #### ftl deploy Build and deploy 1 or more FTL modules. Will recursively scan the `base-dir` for modules (`ftl.toml` files) and leverage `ftl build` to build each module. This command will also use ```bash ftl deploy --help Deploy all modules found within the `base-dir` Arguments: [<base-dir>] Directory to scan for FTL modules to deploy flags: -n, --replicas=1 Number of replicas to deploy. -r, --retry=false Retry failed builds with exponential backoff until they all succeed -w, --wait=false Wait for modules to come online before exiting the command ``` #### ftl dev Run FTL in development mode with hot reloading of modules, etc. * Build and deploy all modules found. * Detect new modules being added or removed update deployments as necessary. * Hot reload and rebuild, redeploy when changes are detected. * Will exponentially back off if builds fail ```bash ftl dev --help Watch a directory for FTL modules. Build, deploy, and hot reload all found modules. Arguments: [<base-dir>] Directory to watch for FTL modules flags: --serve=true Start the FTL cluster --watch=500ms Watch base directory at this frequency and regenerate on change. --reconnect-delay=1s Delay before attempting to reconnect to FTL. --exit-after-deploy Exit after all modules are deployed successfully. ``` Ideas! ```bash! ftl dev // Could be scary to run without knowing which env ftl db <> // New dev command ftl-dev // Flags ftl dev --wipe-user-db ftl dev --endpoint=<> // Prompt if endpoint is not local ftl db wipe --yes <would prompt with "are you sure y/n"> Interactive FTL prompt (less useful in automation, etc.) // Maybe other commands for non-local ftl-prod ftl-remote ``` #### Discussion: `ftl dev` accepts a base directory argument, within which all modules are deployed and watched. Watching _only_ the directory provided in the most recent `ftl dev` run may create confusing experiences for users— - Deployments from prior `ftl dev` runs may not be included in the most recent directory path. These would remain live in the cluster but not watched. - Subsequent deployments via `ftl deploy …` may not be included in the most recent `ftl dev` directory path. These would also be live in the cluster but not watched. Instead, we’re proposing that everything live in the cluster is watched and hot-reloaded, always. Users can consider hot-reloading an inherent feature of FTL and need not be responsible for toggling it across modules. #### Alternatives: 1. With `<base-dir>` and `recreate` arguments: - `ftl dev --recreate <base-dir>` - Cluster is recreated with clean state. - Everything in provided directory is deployed to cluster and watched. - Subsequent deploys performed discretely (`ftl deploy …`) are also watched. - `ftl dev <base-dir>` - Cluster is not recreated. Existing deployments are re-provisioned and watched. - Everything in provided directory is deployed to cluster and watched. - Subsequent deploys (`ftl deploy …`) are also watched. 2. Without `<base-dir>` argument: Deploys are explicitly managed by `ftl deploy` and not by `ftl dev`. The `--recreate` argument operates in the same way as above. All deployed modules are watched/hot-reloaded. 3. Without `recreate` argument: `ftl dev <base-dir>` always recreates the cluster and deploys the modules contained in the specified path. Everything is watched and hot-reloaded. This approach is a true one-stop-shop. ## Rejected Alternatives (optional)