# AiiDA Team Meeting 2022-01-26

###### tags: `team meetings`
###### time: 10am CET

[TOC]

### Present

* Chris
* Francisco
* Giovanni
* Simon
* Jason
* Marnik
* Sebastiaan

### Catch-up round

*Max. 3 minutes each*

Chris
- 🔧 Move to flit for PEP 621 compliant package build (#5312):
- There have been numerous standardisation, to move away from hard-coding the need for setuptools or a setup.py, so you can have a pyproject.toml to "rule them all" (see https://www.python.org/dev/peps/pep-0621, https://www.python.org/dev/peps/pep-0660, https://www.python.org/dev/peps/pep-0517)
- In https://github.com/aiidateam/aiida-core/pull/5312, I have moved aiida-core to use https://github.com/pypa/flit; a build tool that focuses on pure-python packages, and so can avoid some of the legacy baggage and complexity setuptools has.
- I have updated https://github.com/aiidateam/aiida-plugin-cutter to use flit, and you can see an example of this with https://aiida-diff.readthedocs.io.
- I have also update aiida-registry, to pull plugin info directly from PyPI if it is on there (https://github.com/aiidateam/aiida-registry/pull/201), meaning you don't have to switch to flit, but if you do it will still show up fine on there.
- 👌 IMPROVE: Configuration migrations (#5319)
- Added `verdi config downgrade`
- ♻️ REFACTOR: Profile configuration
- DiskObjectStoreRepositoryBackend: use container in context manager
- 👌 IMPROVE: Legacy tar file archive migration performance (#5275)
- 👌 IMPROVE: Add SQLA migration for parity with Django schema (#5097)
- ♻️ REFACTOR: Remove Django storage backend
- Opened https://github.com/aiidateam/aiida-core/issues/5324 (improving aiida terminology), would be interested in feedback
- Personel notes:
- I was on a pocast on 19/01/22, discussing documentation with pip and pycharm maintainers (should appear here soon: https://talkpython.fm/episodes/all)
- I presented at the Jupyter Community call: https://hackmd.io/YIX3UHIiQFy2Uk3FmQA7XA, https://www.youtube.com/playlist?list=PLUrHeD2K9Cmkoamm4NjLmvXC4Y6E1o8SP

Leopold:
* GSOC 2022 participation

Giovanni:
* Sebastiaan and me cannot attend next meeting (Feb 9th): overlaps with MADICES workshop. Can we move it? E.g. at 14:00 (TBD together, also to check if the final program is out and the time of sessions is confirmed)
* Lorenzo Bastonero has taken over the development of the aiida-phonopy plugin, and is refactoring it. Need of a way to run many calculations, each in a different subfolder. Discussed an approach, h implemented it in [this PR](https://github.com/aiidateam/aiida-core/pull/5321). If there is no issue, and this can be included in 2.0 (together with Jason's changes for the double quotes), great (this might be needed for a [future] common-workflow paper).
* SciPy 2022 conference: who wants to participate?

Sebastiaan:

* **General**
* I have closed multiple stale PRs. We have had a significant list of PRs for a long time and I think it is important to clean house a bit. Please have a look at your own PRs and see if they are close to merging (and in that case update them and request a review) or maybe close them for now.
* **Merged PRs**:
* [Dependencies: drop support for Python 3.7](https://github.com/aiidateam/aiida-core/pull/5307)
* [Fix the broken CI pipeline](https://github.com/aiidateam/aiida-core/pull/5301)
* [Dependencies: drop support for NumPy 1.18](https://github.com/aiidateam/aiida-core/commit/471b93801ab764fdbc68a9a77896c2c47438292f)
* [Dependencies: explicitly compile `pymatgen` with compatible `numpy`](https://github.com/aiidateam/aiida-core/commit/d3c9ac697c9413eccfb8c7939189de1dda93cde3)
* [CI: fix failing pre-commit due to `mypy` exception](https://github.com/aiidateam/aiida-core/commit/995f254803ef0fa780e66f43904ae530d89aed3f)
* [Print warning to stdout on `load_profile` for development versions](https://github.com/aiidateam/aiida-core/pull/5311)
* [`Process`: raise when `exposed_outputs` gets non-existing `namespace`](https://github.com/aiidateam/aiida-core/pull/5265): Raise useful message when users misspell the namespace when attaching exposed outputs as outputs.
* [Fix inconsistent behavior of options in `verdi calcjob cleanworkdir`](https://github.com/aiidateam/aiida-core/pull/5209) (Together with Bonan Z.)
* [Transport: store machine keyword under hostname attribute](https://github.com/aiidateam/aiida-core/commit/62c505439a8ca091f5bc7c35b632d99390478519)
* [`RemoteData`: allow passing transport to `_clean` and set extra](https://github.com/aiidateam/aiida-core/commit/9cecc982d0a358c90a5bf1004057cbf84734040d)
* [`verdi calcjob cleanworkdir`: fix filter bug and add filter option](https://github.com/aiidateam/aiida-core/commit/e880ab75102f74043e1b921f8eaba4a4001511a8)
* [`SandboxRepositoryBackend`: move imports out of destructor](https://github.com/aiidateam/aiida-core/pull/5316)
* **Open PRs**:
* [`Dict`: implement the `__contains__` method](https://github.com/aiidateam/aiida-core/pull/5328)
* **AEP**
* Updated the [`aiida-shell` AEP](https://github.com/aiidateam/AEP/pull/32) with some more design considerations after talking to external people wanting to use it.
* New implementation now also allows to run on any remote `Computer`.
* Uses `CalcJob` and `Parser` implementation with a simple wrapper function to simplify the interface

        results_split = engine.shell_job(
            'split',
            ['-l', '2', '{single_file}'],
            files={'single_file': single_file},
            outputs=['x*']
        )

* The wrapper will automatically create a `Code` for the given shell command (if it doesn't yet exist) and will also create and configure the localhost `Computer` if not already present. This makes this snippet runnable directly after installing AiiDA, without having to configure anything. Significantly lowers the barrier to entry again.
* If another computer is required, one can specify it using the well-known `metadata.options`:

        results_split = engine.shell_job(
            'split',
            ['-l', '2', '{single_file}'],
            files={'single_file': single_file},
            outputs=['x*'],
            metadata={'options': {'computer': orm.load_computer('some_remote')}}
        )

Jason:
* More to double quote escaping [PR](https://github.com/aiidateam/aiida-core/pull/5280).
* Separete the `computer_cmdline_params` from `cmdline_params` to set run line parameters related to computer and scheduler only so can configure double quote escaping independently.
* Add `custom_cmdline_string`, when this field is set only this string will print to script and print as it is.
* This ["maximum recursion depth reached" issue](https://github.com/aiidateam/aiida-core/issues/4876) potentially limit the power in running high-throughput calculations. I can easily reproduce it in my workflow and try to debug it as Chris suggest, deserve to give it a close look.

Marnik:
* **Merged PRs**
* [🐛 FIX: reinstate OverridableArgument to public API [#5329]](https://github.com/aiidateam/aiida-core/pull/5329)
*Note*: `cmdline.utils.echo` also had some changes that can break plugin code which relies on it (e.g. `aiida-pseudo`).
* **Open PRs**
* [✨ NEW: Add verdi group move-nodes command [#4428]](https://github.com/aiidateam/aiida-core/pull/4428)
* [Dict: implement items() method [#5333]](https://github.com/aiidateam/aiida-core/pull/5333)
* [✨ NEW: _repr_pretty_ representation for ProcessBuilder [#4970]](https://github.com/aiidateam/aiida-core/pull/4970)
* **Question**: Do we already have some scripts for migrating plugins (e.g. to add the `core` prefix etc)?
* `reentry`:
* There was a small issue in how the hash for the datafile was constructed (see [#67](https://github.com/aiidateam/reentry/issues/67)), that caused the datafile to be the same for different virtual environments that symlink to the same Python binary. Opened [#68](https://github.com/aiidateam/reentry/pull/68) to fix.
* The archival notice suggests users that want to continue to maintain the package to open an Issue, but I think this is not possible if the repository is archived? Maybe we should have them contact `[e-mail]` instead.

Simon:
* Supported Louis with development of the *declarative work chain*.
* Reviewed various dependency and build-related PRs.
* Reviewed "Profile Access PR" [#5270](https://github.com/aiidateam/aiida-core/pull/5270)

Francisco:
* Merged [PR-#5270](https://github.com/aiidateam/aiida-core/pull/5270): a new system to (1) keep track of processes accessing an aiida profiles and (2) lock the access to aiida profiles (meaning: verify that no other process is currently accessing and preventing future processes from accessing until the lock is released).
* The process tracking part was applied as part of this PR but the locking was still not being used by any other AiiDA method (see next PR).
* Submitted [PR-#5331](https://github.com/aiidateam/aiida-core/pull/5331) to use the new locking mechanism when performing maintainance.
* Analyzing [issue-#5323](https://github.com/aiidateam/aiida-core/issues/5323):
* Some of the tests of the tracking/locking mechanism involve submitting sub-processes that load aiida profiles in order to check that the whole chain of methods works properly.
* The submission of the sub-processes seems to be problematic under some setups (too slow or maybe not being launched at all?).
* I think even though these are not strictly speaking "unit tests", I think they are useful to assessing the proper working of the feature and should be kept even if slow (maybe under some nightly build?).
* Slightly unrelated: are we aware of [FastAPI](https://fastapi.tiangolo.com/) for easy restAPI setups?
* [chris]: that is what aiida-restapi uses :wink:
* FFR: :+1:

### Specific Agenda Items