---
# System prepended metadata

title: GTC based generator for stencil modules
tags: [cycle 2]

---

# GTC based generator for stencil modules
###### tags: `cycle 2`
shaper: Rico

## Assigned

to Rico first half of cycle

## Appetite
- ~~full cycle, should allow preparing for unstructured toolchain~~
- half cycle, might need to adjust scope if merge conflicts turn up
- ~~two weeks, limited scope~~

## Problem

### Context
The stencil module is the python module that is generated alongside the stencil implementation or containing the stencil implementation for some backends. For GT4Py CLI usage generating it in the former case is optional, whereas for JIT compilation use cases it is the python side entry point for the generated code.

The current procedure for generating stencil modules involves walking and accumulating data from the ImplementationIR. The data is then used to render a jinja2 template.

The eve based IRs provide more elegant ways to achieve the same data collection. An eve-based templated code generator could represent the template file as well as the rules for rendering it from GTIR. This is not to say that it has to be rewritten in this form.

### Problem areas

The switch from DefinitionIR / ImplementationIR to GTIR should mostly concern the abstract `BaseModuleGenerator` and it's concrete subclass `PyExtModuleGenerator` in the `backend.base` module, only minor adjustments should be required in further, backend specific subclasses.

![](https://i.imgur.com/GDwYtwQ.png)

### (1)
* Data for template rendering comes from IIR, change to GTIR
* hard to read (because IIR does not have querying methods that GTIR has and loops were used instead of a visitor)
* Strongly coupled to current implementation of IIR
* The resulting nested dict `args_data` requires trial and error to use from a new custom module generator. It's structure is taillored to the IIR structure and not to the use case.

### (2)
Strong coupling between the indicated and methods across the two classes. Moving all the functionality into the module generator would increase cohesion and decrease coupling

### (3)
* `__call__()` does some code generation work. This blocks some customization in subclasses (they would have to reimplement all the desired parts of `__call__()` as well)
* uses data passed by the backend but also scrapes data from DefinitionIR, which will also be replaced with GTIR.

### (4)
* the only abstract method is `generate_implementation()`, let's keep that!
* some code generation is customizable, some is instead hard coded in `__call__`. It should be checked if that remains desirable.

### Concrete subclasses
All concrete subclasses must be checked and migrated from the `args_data` structure and legacy IR direct access to whatever is chosen to replace those. Known subclasses requiring attention:

* `backend.base.PyExtModuleGenerator`
* `backend.gt_backends.GTCudaPyModuleGenerator`

There may be others.

## Solution

Change the `BaseModuleGenerator` to draw information from GTIR (unless OIR can be shown to be a better choice). Simplify it's API so that the backend calling it at most has to pass the IR. Use the templated generator pattern for generation of code that is used to render the overall module template (optionally represent the whole module template as a templated generator, whichever is easier). Replace explicit iteration of tree nodes with query methods or an explicit visitor.

### Goals

* Eliminate the dependency on ImplementationIR that results from the current stencil module generation process.
* Keep the necessary level of customization low (one child module generator class with a `generate_implementation()` method per backend family).
* Decrease coupling (see Problem area (2)).
* If the abstract class stores data gathered from the IR or is passed such information from the backend (the latter is not recommended), then it should be structured in an IR-independent way and easy to use (documented) from concrete subclasses.

Optional goals:

* increase readability of the data gathering by using well established patterns.
* increase customizability of module generation
* establish lightweight unit tests for the involved code generation functionality

## Potential rabbit holes

Mapping the generation process as a whole to gtc patterns from the start would potentially lead to distractions. Recommended approach is to to start by keeping the file template + rendering class structure while switching to GTIR, verify equivalence and then start porting code generation to a templated generator class.

Might cause merge conflicts with other gtc related projects.

## No-gos

* Rethinking the contents and structuring of the generated stencil module is out of scope for this project.
* Rethinking or changing the structure or behaviour of the generated stencil objects in a backwards incompatible way is out of scope and if necessary, tests should be introduced to avoid such changes.
* Anything that requires more than minimal changes in backends should be avoided.