[DaCe] Task shaping

# [DaCe] Task shaping ###### tags: `cycle 18`  - Shaped by: Christos, Edoardo, Peter, Philip - Appetite (FTEs, weeks): - Developers: ## Introduction The tasks related to DaCe are divided into different categories, each described below in a separate paragraph. In the context of this document, *DaCe backend* refers to the GT4Py backend for ITIR using DaCe for code generation, developed by CSCS. On the other hand, *DaCe library* refers to DaCe as a module, developed and mantained by SPCL. The following paragraphs contain comments from a grooming meeting hold on 2023-10-31. ## DaCe backend software quality The code of the DaCe backend is difficult to read, hard to mantain and generates an initial SDFG which is hard to analize because of multiple levels of nesting and the presence of states in conjunction with temporaries. A proposal (see [here](https://hackmd.io/V74Lk75ORNezvLBSAT6YpQ)) was discussed to refactor the DaCe backend. The outcome of the discussion was that most of its complexity comes from the input and output representations, ITIR and SDFG API respectively. These representations are far away from each other, and there is very small room for code improvement unless we either change the input representation to something else (Fieldview IR, [Tensor IR](https://hackmd.io/YsE0InS_QWKzt2kGhY7YMg#Lowering-to-DaCe)) or we introduce intermediate representation steps: - change the input representation: this decision is not planned in near future; - working on the intermediate step would be a reasonable activity, but a little risky because of unknowns in the future initial representation. One concrete activity is to contribute to DaCE library by making proposals to simplify the SDFG API, whenever we find room for improvement. Additional tasks related to GT4Py development for DaCe backend: - [ITIR pass for field domain offsets](https://hackmd.io/n-uPcaDwTNahIKNwrpZYwQ) - [ITIR pass to replace invalid identifiers](https://hackmd.io/9aV2DpvKRtKtkOuy_7USKA) #### 2023-10-31 meeting notes: - Regarding the ITIR pass to replace invalid identifiers: check how the GTFN backend deals with unicode symbols. - FieldOperator AST (in GT4Py frontend) and JAX IR (see [below](https://hackmd.io/ffupzhO6TDig7nPU_cJpyQ?both#Mid--and-large-size-contributions-to-DaCe-library)) are two options to prototype a better IR for DaCe. - **Decision** to keep the 2 tasks for ITIR passes in scope for next cycle. ## DaCe backend development for icon4py This task is about advancing the support of icon4py stencils on the DaCe backend. The CPU target is the default platform for testing and integration. During cycles 16 and 17, several features were added or extended, so that it was possible to run all stencils in atmosphere dynamical core. The last PR was about adding translation support for removal of lifts by means of temporaries. While working on isolated stencils, this approach requires further development for fused stencils. The outcome of this task is to be able to compile and run the SDFG generated from icon4py stencils. **Question is whether it still is a valid objective**, given the known limitations of the DaCe backend design. #### 2023-10-31 meeting notes: - **Decision** to continue the support of icon4py stencils but limit it to the atmosphere model. Focus on the execution and optimization of fused stencils. ## Small-size contributions to DaCe library A selection of small tasks to help gt4py team get confidence with usage and internal design of DaCe library: - SourceLocationGroup (gt4py): add debug information to SDFG to track the source location in gt4py code. - [2023-10-31 meeting notes] This activity consists of two parts: one on gt4py side, about adding debug information to SDFG; and another part in DaCe, about enabling propagation and merge of debug information in SDFG transformations. **Decision** to keep it in scope. - Error in GPU codegen: see [#1388](https://github.com/spcl/dace/issues/1388). - [2023-10-31 meeting notes] **Decision** to keep it in scope because needed for icon4py stencil support. - Auto-optimize issue / error in CPU codegen: not published yet, currently Edoardo is working on it. - [2023-10-31 meeting notes] **Decision** to keep it in scope because needed for icon4py stencil support. - (from Ben) Not very SDFG or transformations specific, more about the high-level flow of DaCe + its entry point: [#1380](https://github.com/spcl/dace/issues/1380) - [2023-10-31 meeting notes] Already completed by Philip. - (from Ben) More Python specific: [#1375](https://github.com/spcl/dace/issues/1375) - [2023-10-31 meeting notes] This issue could be quite complex. **Decision** to exclude from the scope for next cycle. - (from Ben) Looks like either an SDFG or Transformations bug: [#1372](https://github.com/spcl/dace/issues/1372) - [2023-10-31 meeting notes] This issue could be related to other issues observed in icon4py stencil auto-optimize. **Decision** to keep it in scope. - (from Ben) Bug report by Edoardo, looks like an SDFG or Transformations issue: [#1272](https://github.com/spcl/dace/issues/1272) - [2023-10-31 meeting notes] This issue is more of a conceptual change, than a bug. Not needed now, not a blocking issue. **Decision** to exclude it from the scope. - (from Philip) Bug related to function call [#1389](https://github.com/spcl/dace/issues/1389) - [2023-10-31 meeting notes] Good for learning about DaCe Python-frontend. **Decision** to keep it in scope. ## Mid- and large-size contributions to DaCe library - JAX-to-DaCe (shaped as a CSCS internship proposal [here](https://hackmd.io/84NQjpebS8KXLB9_0PmhIA)) - [2023-10-31 meeting notes] Good candidate to prototype a better IR for DaCe backend. **Decision** to keep it in scope. - Library node for indirection - [2023-10-31 meeting notes] This activity needs further discussion with SPCL team. Check other propsals like indirection-memlets. - (from Ben) Transformation in DaCe relevant for EXCLAIM/climate and weather stencils. Two obvious candidates could be: 1) Stencil inlining 2) overlapped tiling (basically same as inlining, but use GPU shared memory to reduce floating point operations and reuse already computed results). - [2023-10-31 meeting notes] This activity is part of DaCe backend development for icon4py (see [here](https://hackmd.io/ffupzhO6TDig7nPU_cJpyQ?both#DaCe-backend-development-for-icon4py)).