---
# System prepended metadata

title: AiiDA Team Meeting 2022-01-26
tags: [team meetings]

---

# AiiDA Team Meeting 2022-01-26

###### tags: `team meetings`
###### time: 10am CET

[TOC]

### Present

* Chris
* Francisco
* Giovanni
* Simon
* Jason
* Marnik
* Sebastiaan


### Catch-up round

*Max. 3 minutes each*

Chris
- 🔧 Move to flit for PEP 621 compliant package build (#5312):
    - There have been numerous standardisation, to move away from hard-coding the need for setuptools or a setup.py, so you can have a pyproject.toml to "rule them all" (see https://www.python.org/dev/peps/pep-0621, https://www.python.org/dev/peps/pep-0660, https://www.python.org/dev/peps/pep-0517)
    - In https://github.com/aiidateam/aiida-core/pull/5312, I have moved aiida-core to use https://github.com/pypa/flit;  a build tool that focuses on pure-python packages, and so can avoid some of the legacy baggage and complexity setuptools has.
    - I have updated https://github.com/aiidateam/aiida-plugin-cutter to use flit, and you can see an example of this with https://aiida-diff.readthedocs.io.
    - I have also update aiida-registry, to pull plugin info directly from PyPI if it is on there (https://github.com/aiidateam/aiida-registry/pull/201), meaning you don't have to switch to flit, but if you do it will still show up fine on there.
- 👌 IMPROVE: Configuration migrations (#5319)
    - Added `verdi config downgrade` 
- ♻️ REFACTOR: Profile configuration
- DiskObjectStoreRepositoryBackend: use container in context manager 
- 👌 IMPROVE: Legacy tar file archive migration performance (#5275) 
- 👌 IMPROVE: Add SQLA migration for parity with Django schema (#5097)
- ♻️ REFACTOR: Remove Django storage backend
- Opened https://github.com/aiidateam/aiida-core/issues/5324 (improving aiida terminology), would be interested in feedback
- Personel notes:
    - I was on a pocast on 19/01/22, discussing documentation with pip and pycharm maintainers (should appear here soon: https://talkpython.fm/episodes/all)
    - I presented at the Jupyter Community call: https://hackmd.io/YIX3UHIiQFy2Uk3FmQA7XA, https://www.youtube.com/playlist?list=PLUrHeD2K9Cmkoamm4NjLmvXC4Y6E1o8SP

Leopold:
 * GSOC 2022 participation




Giovanni:
 * Sebastiaan and me cannot attend next meeting (Feb 9th): overlaps with MADICES workshop. Can we move it? E.g. at 14:00 (TBD together, also to check if the final program is out and the time of sessions is confirmed)
 * Lorenzo Bastonero has taken over the development of the aiida-phonopy plugin, and is refactoring it. Need of a way to run many calculations, each in a different subfolder. Discussed an approach, h implemented it in [this PR](https://github.com/aiidateam/aiida-core/pull/5321). If there is no issue, and this can be included in 2.0 (together with Jason's changes for the double quotes), great (this might be needed for a [future] common-workflow paper).
 * SciPy 2022 conference: who wants to participate?


Sebastiaan:

 * **General**
     * I have closed multiple stale PRs. We have had a significant list of PRs for a long time and I think it is important to clean house a bit. Please have a look at your own PRs and see if they are close to merging (and in that case update them and request a review) or maybe close them for now.
 *  **Merged PRs**:
     * [Dependencies: drop support for Python 3.7](https://github.com/aiidateam/aiida-core/pull/5307)
     * [Fix the broken CI pipeline](https://github.com/aiidateam/aiida-core/pull/5301)
         * [Dependencies: drop support for NumPy 1.18](https://github.com/aiidateam/aiida-core/commit/471b93801ab764fdbc68a9a77896c2c47438292f)
         * [Dependencies: explicitly compile `pymatgen` with compatible `numpy`](https://github.com/aiidateam/aiida-core/commit/d3c9ac697c9413eccfb8c7939189de1dda93cde3)
         * [CI: fix failing pre-commit due to `mypy` exception](https://github.com/aiidateam/aiida-core/commit/995f254803ef0fa780e66f43904ae530d89aed3f)
     * [Print warning to stdout on `load_profile` for development versions](https://github.com/aiidateam/aiida-core/pull/5311)
     * [`Process`: raise when `exposed_outputs` gets non-existing `namespace`](https://github.com/aiidateam/aiida-core/pull/5265): Raise useful message when users misspell the namespace when attaching exposed outputs as outputs.
     * [Fix inconsistent behavior of options in `verdi calcjob cleanworkdir`](https://github.com/aiidateam/aiida-core/pull/5209) (Together with Bonan Z.)
         * [Transport: store machine keyword under hostname attribute](https://github.com/aiidateam/aiida-core/commit/62c505439a8ca091f5bc7c35b632d99390478519)
         * [`RemoteData`: allow passing transport to `_clean` and set extra](https://github.com/aiidateam/aiida-core/commit/9cecc982d0a358c90a5bf1004057cbf84734040d)
         * [`verdi calcjob cleanworkdir`: fix filter bug and add filter option](https://github.com/aiidateam/aiida-core/commit/e880ab75102f74043e1b921f8eaba4a4001511a8)
     * [`SandboxRepositoryBackend`: move imports out of destructor](https://github.com/aiidateam/aiida-core/pull/5316)
 * **Open PRs**:
     * [`Dict`: implement the `__contains__` method](https://github.com/aiidateam/aiida-core/pull/5328)
 * **AEP**
     * Updated the [`aiida-shell` AEP](https://github.com/aiidateam/AEP/pull/32) with some more design considerations after talking to external people wanting to use it.
         * New implementation now also allows to run on any remote `Computer`.
         * Uses `CalcJob` and `Parser` implementation with a simple wrapper function to simplify the interface
        ```python
        results_split = engine.shell_job(
            'split',
            ['-l', '2', '{single_file}'],
            files={'single_file': single_file},
            outputs=['x*']
        )
        ```
        * The wrapper will automatically create a `Code` for the given shell command (if it doesn't yet exist) and will also create and configure the localhost `Computer` if not already present. This makes this snippet runnable directly after installing AiiDA, without having to configure anything. Significantly lowers the barrier to entry again.
        * If another computer is required, one can specify it using the well-known `metadata.options`:
        ```python
        results_split = engine.shell_job(
            'split',
            ['-l', '2', '{single_file}'],
            files={'single_file': single_file},
            outputs=['x*'],
            metadata={'options': {'computer': orm.load_computer('some_remote')}}
        )
        ```
        
Jason:
  * More to double quote escaping [PR](https://github.com/aiidateam/aiida-core/pull/5280).   
      * Separete the `computer_cmdline_params` from `cmdline_params` to set run line parameters related to computer and scheduler only so can configure double quote escaping independently. 
      * Add `custom_cmdline_string`, when this field is set only this string will print to script and print as it is.
  * This ["maximum recursion depth reached" issue](https://github.com/aiidateam/aiida-core/issues/4876) potentially limit the power in running high-throughput calculations. I can easily reproduce it in my workflow and try to debug it as Chris suggest, deserve to give it a close look.

Marnik:
  * **Merged PRs**
      * [🐛 FIX: reinstate OverridableArgument to public API [#5329]](https://github.com/aiidateam/aiida-core/pull/5329)
*Note*: `cmdline.utils.echo` also had some changes that can break plugin code which relies on it (e.g. `aiida-pseudo`).
  * **Open PRs**
      * [✨ NEW: Add verdi group move-nodes command [#4428]](https://github.com/aiidateam/aiida-core/pull/4428)
      * [Dict: implement items() method [#5333]](https://github.com/aiidateam/aiida-core/pull/5333)
      * [✨ NEW: _repr_pretty_ representation for ProcessBuilder [#4970]](https://github.com/aiidateam/aiida-core/pull/4970)
  * **Question**: Do we already have some scripts for migrating plugins (e.g. to add the `core` prefix etc)?
  * `reentry`:
      * There was a small issue in how the hash for the datafile was constructed (see [#67](https://github.com/aiidateam/reentry/issues/67)), that caused the datafile to be the same for different virtual environments that symlink to the same Python binary. Opened [#68](https://github.com/aiidateam/reentry/pull/68) to fix.
      * The archival notice suggests users that want to continue to maintain the package to open an Issue, but I think this is not possible if the repository is archived? Maybe we should have them contact `developers@aiida.net` instead.


Simon:
  * Supported Louis with development of the *declarative work chain*.
  * Reviewed various dependency and build-related PRs.
  * Reviewed "Profile Access PR" [#5270](https://github.com/aiidateam/aiida-core/pull/5270)


Francisco:
  * Merged [PR-#5270](https://github.com/aiidateam/aiida-core/pull/5270): a new system to (1) keep track of processes accessing an aiida profiles and (2) lock the access to aiida profiles (meaning: verify that no other process is currently accessing and preventing future processes from accessing until the lock is released).
      * The process tracking part was applied as part of this PR but the locking was still not being used by any other AiiDA method (see next PR).
  * Submitted [PR-#5331](https://github.com/aiidateam/aiida-core/pull/5331) to use the new locking mechanism when performing maintainance.
  * Analyzing [issue-#5323](https://github.com/aiidateam/aiida-core/issues/5323):
      * Some of the tests of the tracking/locking mechanism involve submitting sub-processes that load aiida profiles in order to check that the whole chain of methods works properly.
      * The submission of the sub-processes seems to be problematic under some setups (too slow or maybe not being launched at all?).
      * I think even though these are not strictly speaking "unit tests", I think they are useful to assessing the proper working of the feature and should be kept even if slow (maybe under some nightly build?).
  * Slightly unrelated: are we aware of [FastAPI](https://fastapi.tiangolo.com/) for easy restAPI setups?
      * [chris]: that is what aiida-restapi uses :wink: 
      * FFR: :+1: 

### Specific Agenda Items