# Workflow pattern extension & application to GTFNExecutor
###### tags: `cycle 14`

- Developers:
- Appetite: 3 weeks 1 FTE
- Status: Done

## Problem

Workflows consist of multiple steps. Each step can have it's own configuration and there is no common configuration interface (nor should there be).
To change a configuration variable of one step means to create a new workflow, since they are immutable (as they should be, we think).

Writing out the complete workflow in code again, with just one value changed is verbose and error prone, in particular if workflows are nested. The current way to avoid this is to store commonly changed step configuration variables in a class, which then builds the workflow. A single variable can be changed and the new workflow built. But this class then becomes the sum of all configuration of all subworkflows. This was recognized as a temporary solution at best already when workflows were first implemented.

Particularly the `GTFNExecutor` / `gtfn_cpu` has become bloated with step-configuration due to that. The `GTFNExecutor` class, which is not directly a workflow step also acts as a blockage to extending or nesting the GTFN compilation workflow.

## Appetite
3 weeks 1 FTE

## Solution

### MVP
As a minimum workflows with named steps together with either of (or both) should be implemented:
- a way to create a new workflow from an existing one by replacing steps (analogous to `dataclasses.replace`)
- a way to create worklow recipes with defaults, from which a new workflow can be instantiated, specifying only non-defaults (analogous to `factoryboy.Factory`)

And the `GTFNExecutor` should become a `workflow.Step` which uses all of these. As a consequence, GTFNExecutor (or it's replacement) will not replicate any step configuration variables (e.g. `enable_itir_transforms`), while creating and using variants of it is still convenient.

### Extensions

If there is time:
- use the workflow patterns inside the sub-steps of GTFNExecutor
- a way to do nested replacements in a flat way (like `factoryboy`)
- a way to extend nested replacements to non-workflow steps (under certain conditions)

Extensions also have implementation in GTFNExecutor as part of their definition of done.

## Rabbit holes

- do not overengineer for potential other usages. The whole system is lightweight enough to allow redesigning later if necessary

## No-gos
- do not start applying the workflow pattern everywhere else

## scratch pad

@program(backend=[gtfn,itir_embedded,fieldview_embedded], transformations=..., debug=...)
def foo():
    ....