# Integrating compiled backends

###### tags: `functional cycle 9`

Appetite: Full Cycle
Developer: Rico, Peter

## Work in Progress
- [x] investigate cmake python packages
- [x] inspect CMake project structure in GTFN tests
- [x] explore ways to capture code + metadata + bindings (Peter)
- [x] explore ways to generate CMake and write projects to file (Peter)
- [x] PoC roundtrip and run a copy stencil (Rico)
- [ ] write tests / ficticious examples for the anticipated use cases
- [x] roundtripping to python
- [ ] getting a cmake project in which one could develop a C++ driver
- [ ] design the system (using the explorations from above)

### packages:
* Potential reference for advanced usage: [`cmake-common`](https://pypi.org/project/cmake-common/), [`pybind11-cmake`](https://github.com/safijari/pybind11-cmake), [`generate_cmake`](https://github.com/jpanikulam/experiments/tree/master/pymake/generate_cmake)
* parsing / autoformatting: [`parse_cmake`](https://github.com/wjwwood/parse_cmake), [`cmake-format`](https://pypi.org/project/cmake-format/), [`cmake-tidy`](https://github.com/MaciejPatro/cmake-tidy)
* cmake project inspection API: [`cmake-file-api`](https://github.com/madebr/python-cmake-file-api)
* Not useful (probably): `python-cmake-buildsystem`, `cmake-init`, `clean_cmake_project`, `cmakew`

## Background:

The C++ backend already generates code that can be used outside declarative gt4py but the code it generates can not be executed from python yet.

An architecture for integrating the generated code is outlined in [this hackmd note](https://hackmd.io/qSsXvuffSi2r0C4niHxSTg#Architecture-overview).

The main Idea is for the backend to create a `Project` instance from the generated code, where it is the `Project`'s responsibility to know how to build the code. An instance of `ProjectDecorator` (in the above link named "ProjectTransformer") would then know how to generate and build python bindings for the `Project`.

In practice we will work with `cmake` projects in this iteration, and there will likely be a specialized `CmakeProject` which inherits from / implements the `Project` protocol.

## Goals:
* Execute a single fencil in python using the GTFN backend without manual interference.
* Keep the responsibilities for generating and building code separate (by not supporting arbitrary project layouts)
* Keep the responsibility for generating python bindings separate from the rest (if possible).

## Non-Goals:
* Any Project caching (except a no-op placeholder).

## Known Steps:
* Get input from GTFN backend authors Hannes and Felix on CMake project structure (beyond what's in the tests, specifically bindings through project composition).
* Design the Project protocol / API
* Design and implement a concrete Project class which uses CMake
* Verify composability and implement python bindings project decorator
* [optional] Once single fencil works, put placeholder caching.

## Possible Rabbit holes:
* CMake: use the simplest approach that works for everything.
* Do not try to allow for more flexibility in CMake Project layout than comes for free.
* Do not write a Python CMake API