# [Greenline] Research: I/O in python driver - Shaped by: Magdalena - Appetite (FTEs, weeks): - Developers: <!-- Filled in at the betting table unless someone is specifically required here --> ## Problem <!-- The raw idea, a use case, or something we’ve seen that motivates us to work on this --> We need to enable IO for the greenline. The IO system needs work at scale with large paralel datasets on distributed system, and it should somehow integrate in a prospective Weather&Climate platform or workflows at CSCS/C2SM Similar problems have likely been tackled by other scientific communities, many of them relying on Python as a tool of choice for data handling. With this project we want to gather more precise requirements for the system and assess what tools there are that we can use. ## Appetite <!-- Explain how much time we want to spend and how that constrains the solution --> ## Solution <!-- The core elements we came up with, presented in a form that’s easy for people to immediately understand --> - Understand and document minimal requirements of IO system for icon4py greenline - Assess exsting libraries and approaches in hindsight of these requirements - Sketch a first draft of an architecture. - Solution should handle grid and ### sources - [Enriques notes](https://hackmd.io/0x-YtL00Qq6G97C1oVaWkw) - [CF conventions](https://cfconventions.org/) - [MetPy](https://unidata.github.io/MetPy/latest/index.html) ### preliminary investigations #### input - the icon grid file can be loaded in uxarray after patching the contents to follow UX conventions. see - There is interest by groups at MCH and EMPA to have a joint python grid interface instead of everybody doing its own version. It should allow them to access neighbor information. (contact: David Leutwyler, Viktoria Cherkas (MCH), ) ## Rabbit holes <!-- Details about the solution worth calling out to avoid problems --> Do not get lost in too many details. ## No-gos <!-- Anything specifically excluded from the concept: functionality or use cases we intentionally aren’t covering to fit the ## appetite or make the problem tractable --> ## Progress <!-- Don't fill during shaping. This area is for collecting TODOs during building. As first task during building add a preliminary list of coarse-grained tasks for the project and refine them with finer-grained items when it makes sense as you work on them. --> - [x] Task 1 ([PR#xxxx](https://github.com/GridTools/gt4py/pulls)) - [x] Subtask A - [x] Subtask X - [ ] Task 2 - [x] Subtask H - [ ] Subtask J - [ ] Discovered Task 3 - [ ] Subtask L - [ ] Subtask S - [ ] Task 4