AiiDA Team Meeting 2023-10-19

# AiiDA Team Meeting 2023-10-19 ###### tags: `team meetings` ###### time: 14:00 CET [TOC] ### Present * Sebastiaan * Marnik * Giovanni * Edan * Miki * Jusong * Kristjan ### Catch-up round *Max. 3 minutes each* Sebastiaan: * **Miscellaneous**: * Revive concept of blogs. [A question from Xing Wang](https://aiida.discourse.group/t/lose-provenance-inside-workchain/126) on Discourse revived the idea of having a place for a blog. Years ago, [`aiida-blog`](https://github.com/aiidateam/aiida-blog) was created for this purpose with Giovanni and Francisco writing the first entry but it was never published. Instead of having a separate repo and location, I propose we simply use the main website. Its news and events entries are already written through a blog system for Sphinx. I have [opened a PR](https://github.com/aiidateam/aiida-website/pull/49) that updates the website to add the blog post of G+F. Adding new blog posts is then as easy as adding a new post through a PR. * Big thanks to Daniel Hollas for excellent work on reducing import times making `verdi` tab-completion very snappy! * **Open Issues**: * [Discussion on desired breaking changes for AiiDA v3.0](https://github.com/aiidateam/aiida-core/issues/6157) * **Merged PRs**: * [ORM: `ProcessNode.is_valid_cache` is `False` for unsealed nodes ](https://github.com/aiidateam/aiida-core/pull/6147): Fixes rare race-condition bug in caching * [`FolderData`: Expose repository API on top-level namespace](https://github.com/aiidateam/aiida-core/pull/6150) * [Dependencies: Update to `disk-objectstore~=1.0`](https://github.com/aiidateam/aiida-core/pull/6132) * [Tests: Refactor transport tests from `unittest` to `pytest`](https://github.com/aiidateam/aiida-core/pull/6152) * **Open PRs**: * [CLI: Add `cached` and `cached_from` projections to `verdi process list`](https://github.com/aiidateam/aiida-core/pull/6156): Example output: ``` sph@invader$ vpla PK Created Process label Cached Process State Process status ------ --------- ------------------------ -------- ---------------- ---------------- 152644 3h ago PwCalculation ⏹ Finished [0] 152652 3h ago PwCalculation ✔ ⏹ Finished [0] 152661 1h ago ShellJob<pw.x@localhost> ⏹ Finished [400] 152668 1h ago ShellJob<pw.x@localhost> ✔ ⏹ Finished [400] Total results: 6 ``` * [Engine: Add the wait argument to submit](https://github.com/aiidateam/aiida-core/pull/6155): Will allow to make `submit(Process, **inputs, wait=True` blocking. This is useful in demos/tutorials in interactive notebooks where we want the persistence and restart safety of submitting to the daemon, but want the intuitiveness of `run` where results are available when cell returns control. * [Add the `SshFabricTransport` plugin](https://github.com/aiidateam/aiida-core/pull/6154): Subclass of `SshTransport` but uses `fabric` to create the connection, which is a library built-on top of `paramiko` but has robust logic to automatically figure out connections parameters from SSH config files. Suggested by Sasha Yakutovich to make the configuration of SSH computer a lot easier, with less parameters that have to be specified, just the hostname. This is an alternative implementation [to his original implementation](https://github.com/aiidateam/aiida-core/pull/6083). He reimplemented `Transport` from scratch, but in my version I reuse `SshTransport` and just override connection part. Since `fabric` uses `paramiko` underneath, this works with minimal changes. Do we want to include this plugin in `aiida-core` or provide it through a separate plugin package? * [Repository: Add the `as_path` context manager ](https://github.com/aiidateam/aiida-core/pull/6151): Allows making contents of node repository as file on local disk. Useful when needing access to files through filepath, e.g. for libraries that don't accept file handles. Simplifies a lot of boilerplate code using `tempfile`. * [ORM: Register `numpy.ndarray` with the `to_aiida_type` to `ArrayData`](https://github.com/aiidateam/aiida-core/pull/6149): Accept `numpy.ndarray` as inputs to process functions and automatically convert to `ArrayData`. * [Dependencies: Update to `sqlalchemy~=2.0`](https://github.com/aiidateam/aiida-core/pull/6146): Ready for review * [Config: Switch from `jsonschema` to `pydantic`](https://github.com/aiidateam/aiida-core/pull/6117) * [Add `verdi profile setup`](https://github.com/aiidateam/aiida-core/pull/6023): Requires previous PR to switch to `pydantic` * [Add the `SqliteDosStorage` storage backend](https://github.com/aiidateam/aiida-core/pull/6148): Two parts of functionality not supported yet: * Use of JSONB query language (not sure if can be supported by sqlite) * Use of `date_trunc`. Currently only used by `QueryBuilder.get_creation_statistics` which is only used by the REST API (for MaterialsCloud) Marnik: * Giving tutorial-like talk at [ADIS 2023](https://www.mpie.de/4903042/Programme) in Tegernsee, Germany. Organised by Jörg Neugebauer and Tilmann Hickel from `pyiron`. Will give an overview of AiiDA, MC3D and some other applications. Miki * preparing a design document for StructureData (user stories), in the next two weeks I will iterate within the team. Kristjan: * fixed a small issue in the QE input parser: https://github.com/aiidateam/qe-tools/pull/71 * integers in cell parameters were not supported * blog post about rest api: might make sense to postpone after (potential?) migration to https://github.com/aiidateam/aiida-restapi * Blog post on backups instead. Jason: * Nothing to report related to `aiida-core`. * Giving a talk in two weeks at Cambridge UK for the Huawei workshop. Will give an overview of AiiDA, Materials Cloud. * Will test ssh-agent with Fabric. Edan: * Nothing to report related to `aiida-core`. * Blog * Managing experimental workflows with AiiDA * Calculation live-monitoring * Will check if `SshFabricTransport` would render `ssh2win` moot * Will review `sqlalchemy~=2.0` PR Giovanni: * Reviewed some PRs by Sebastiaan * Proceeding with hiring (up to 4 people if we are lucky...) ### Specific Agenda Items