AiiDA Team Meeting 2021-11-03

# AiiDA Team Meeting 2021-11-03 ###### tags: `team meetings` ###### time: 10am CET [TOC] ### Present * Sebastiaan * Francisco * Marnik * Giovanni * Chris * Louis * Leopold * Jason ### Catch-up round *Max. 3 minutes each* Chris (before seb): * Held meeting on archive and "♻️ REFACTOR: Reorganise Backend class UML" * Agreed in principle * Merged: * 👌 IMPROVE: AuthInfo use in QueryBuilder (#5201) * 👌 IMPROVE: Allow Graph to be used by any backend (#5200) * 👌 IMPROVE: Get session from backend in SqlaModelEntity (#5199) * 👌 IMPROVE: BackendEntity typing and abstraction (#5196) * 👌 IMPROVE: Add AuthInfo joins to QueryBuilder (#5195) * 🔧 MAINTAIN: Move pylint plugin to external package (#5190) * 👌 IMPROVE: Ensure QueryBuilder is passed Backend (#5186) * ✨ NEW: Add Node.objects.iter_repo_keys (#4922) * All towards: https://aiida-archive-demo.readthedocs.io * Share analysis Sebastiaan: * **Merged PRs**: * [Dependencies: update to `click==8.0` and remove `click-completion` ](https://github.com/aiidateam/aiida-core/pull/5111): Would be good if people test the interactive functionality manually as this is the part of the code that changed the most. * [`get_entry_point`: add proper deprecation pathway for old entry points](https://github.com/aiidateam/aiida-core/pull/5206): Improvement over previous fix of supporting deprecated entry points. Before only through factories, now also through low-level `get_entry_point` which should now cover everything. * [`verdi code setup`: validate the uniqueness of the full label](https://github.com/aiidateam/aiida-core/pull/5205): in interactive mode, the command will now reprompt for the label if one is specified that already exists. For non-interactive, an error message is raised. This only fixes the problem for remote codes. Second PR is open for local codes. * **Open PRs**: * [`verdi code setup`: validate the uniqueness of label for local codes](https://github.com/aiidateam/aiida-core/pull/5215): Complement of #5205 to also support local codes. Uniqueness condition here has to be just on the code label, since local codes do not have an associated computer. * [`Node`: add the `is_valid_cache` property setter](https://github.com/aiidateam/aiida-core/pull/5207): Proposed official API to control caching on the node-level. This property allows to mark a single node as invalid for caching. * [`Code`: add additional validation before storing](https://github.com/aiidateam/aiida-core/pull/5204): Checks that a label is actually defined (currently you can happily store a `Code` without label whatsoever) and that the full label is unique. Question on uniqueness condition: should the user be included? * [`Code`: check existence of executable for remote code when storing ](https://github.com/aiidateam/aiida-core/pull/5184): This could help prevent typos when setting up a code and subsequently the calculation failing. * [REST API: make the profile configurable as request parameter ](https://github.com/aiidateam/aiida-core/pull/5054): Has been open for a while. No new progress. Awaiting merging of Django/Sqla refactoring to test if there are bugs in the profile switching functionality. Leopold: * For the coding week: besides working to get 2.0 out, anyone else interested in task farming / REST API? Simon: * _(absent because of urgent MarketPlace meeting)_ * Support for Python 3.10 - [status update](https://github.com/aiidateam/aiida-core/pull/5188#issuecomment-952657394) [[#5188]](https://github.com/aiidateam/aiida-core/pull/5188) * The package is installable with pip (including extras), but some tests fail for Python 3.10. * The conda installation is not yet possible. Jason: * _(absent because of urgent MarketPlace meeting)_ * A very draft AEP for the container code https://github.com/aiidateam/AEP/pull/28. (Please excuse my bad English writing). Very glad if someone can have a look and put some comments. Francisco * Coding week arrangements: we have a place (and a time: 6th-10th of dec)! We are finishing arrangements for lodging at [Alpine Classics](http://www.classic-hotel.ch/) * Leo: confirmed? Almost Marnik * [Node equality comparison](https://github.com/aiidateam/aiida-core/discussions/5187): So far only Giovanni and Seb responded (i.e. no plugin developers). Consensus seems to be to consistently compare Python base types by content, i.e. change `Dict` so this is also the case for this data type. For other types, we won't restrict the user, i.e. leave the choice to the developer whether the data type should compare by content or not. * **Open PR's**: * [👌 IMPROVE: constructor of base data types](https://github.com/aiidateam/aiida-core/pull/5165). * [✨ NEW: `_repr_pretty_` representation for ProcessBuilder](https://github.com/aiidateam/aiida-core/pull/4970). Giovanni * Topics for coding week (Coding): - Support for FirecREST plugins (transport, scheduler) - Also redesign of transport (and scheduler) plugin to optionally use async coroutines? (But still support blocking calls in a back-compatible way); e.g. opens up the way to use asyncssh as a library, and can potentially give a lot of improvement removing some blocking points in the workers - Also extestion of scheduler plugins to avoid the aiida-fireworks-scheduler (and possibly other ones) to have to fetch back and re-parse the submission script via the transport, just to know what was written in there (see my suggested implementation design in [this comment](https://github.com/aiidateam/aiida-core/issues/5084#issuecomment-900278316)) - Support for running containerized codes (e.g. SARUS) - should be easy? * Topics for coding week (Discussions): - Discussion on how sharing should work (push/pull, reducing data duplication between clones (i.e., partial pulls/pushes)), and which steps can bring us there in an "incremental" way (I think we'll need to test it, even in a non-efficient implentation, to see if the User Interface is what users need) - Plans for REST API - REST API to submit - Does current REST API need to be redesigned? (Maybe not?) - Should we have some client library to work via the REST API (e.g. to create a query builder, get instances, ...)? (*IF* we think that at some point in the future we might want to perform a subset of the operations directly via the REST API of a AiiDA server living on a different machine, e.g. a central server where everybody pushes the whole profile, while keeping locally only a subset of it) - Add documentation to disk-objectstore (and update and merge AEP, till in PR) [GP] ### Specific Agenda Items * [GP] Discuss agenda for coding week (and attach names of people interested in the topic for discussions, or wanting to work on this) * Coding week topics: - v2.0 summary/demo/testing (Chris) - transport + scheduler design - FirecREST, async - Task farming (?) - Containerised codes - Data sharing (push/pull) - REST API - New StructureData and mat sci node types - Issue/PR triage lightning round