Public API Import Path Streamlining

# Public API Import Path Streamlining ###### tags: `cycle 15`  - Shaped by: @ricoh - Appetite (FTEs, days): 1 FTE, 4 days (part of small-batch) - Developers: @ricoh - Available for discussion/review: @egparedes - PR: https://github.com/GridTools/gt4py/pull/1263 - Status: Merged [Original Description](https://hackmd.io/46Oa6MspRoSTiFozHzun5A?view#Public-API-tasks) ## Problem User code must import bits of public API (classes, utilities) from various places. Sometimes some of them are imported indirectly (`from gt4py.next.ffront.fbuiltins import Field`) instead of where they are defined, creating ambiguity and hurting readability and teachability. Goal: Simplify user code boilerplate imports and update tests and documentation to reflect that. We have examples like this: ```python= from gt4py.next.ffront.decorator import field_operator, program from gt4py.next.ffront.fbuiltins import ( Dimension, FieldOffset, Field, float64, ... ) from gt4py.next.common import DimensionKind from gt4py.next.program_processors.runners import gtfn_cpu from gt4py.next.iterator.embedded import np_as_located_field I = Dimension("I") ... @field_operator def ... @program def ... ``` We would prefer usage like (for Example) ```python= from gt4py.next import Dimension, field_operator, Field, ... @field_operator def foo(a: Field[...]) ... ``` Or for users who prefer the `import numpy as np` style ```python= import gt4py.next as gtn I = gtn.Dimension("I") @gtn.field_operator def foo(...) ``` Known examples of public API: - `ffront.fbuiltins`: - dtypes - `FieldOffset` - the builtins - `ffront.decorators`: - `field_operator` - `program` - `common`: - `Dimension` - `DimensionKind` - `Field` - `GridType` - `iterator.embedded`: - `NeighborTableOffsetProvider` (found in `ffront_test_utils.py`) - `StridedNeighborOffsetProvider` (> 5 hits in `icon4py` tests, thanks to @muellch) - `index_field` - `np_as_located_field` (Also often requested to be renamed. Intended to change in upcoming storage PR.) To be determined: - Any exceptions that users might want to test for` - Should we provide (optional) default dimensions (useful for teaching)? ## Appetite This is intended as a small-batch project for the implementation track of the [GT4Py Cleanup](https://hackmd.io/-OGkeMJeQT-CqXPQTzlUCw#Problem) Project. ## Solution The minimal solution is to collect all of the public API in a single namespace by importing it for example into `gt4py/next/__init__.py`. The idea is to follow a similar approach to `numpy`, where the most used utilities can be imported from the top level module. ### Definition of Done - Make sure `gt4py.next` namespace does not get polluted. No star imports (except from submodules that define `__all__` and only after double checking it). - Make sure none of this leads to circular imports. If so resolve them if possible. - Update Coding Guidelines with - convention for importing inside the library (suggested: import from where it's defined) - convention for importing inside tests and docs (suggested: import from `gt4py.next`) - convention for when we put a proxy in the top level and when not - Convert usage in tests and docs as well as "getting started" notebook ### Risks - Importing something at the top level might lead to cirular imports. See [Rabbit holes](https://hackmd.io/sw-N-AV7SVOc6258-63VYg?both#Rabbit-holes) for mitigation. - Updating tests and docs could lead to merge conflicts, which should be easy to resolve. Mitigate: seek out other PRs that look like they might have conflicts and notify the authors before merging. ## Rabbit holes If any circular imports pop up, make sure to timebox trying to fix them. If the timebox is not enough, then postpone the proxy import for the affected entities. It's more important to fulfill the DoD for the existing proxies than to complete the list of code objects. If there is potential for naming something better: If you can make a suggestion and it meets no resistance, ok. The usual name negotiations are not included in the workload estimate. ## No-gos No refactorings. It is recognized that some things should be moved / renamed / interfaces changed. These however are separate projects that need shaping. The only exception is resolving circular imports.