# AiiDA Team Meeting 2021-01-13

###### tags: `team meetings`
###### time: 4pm CET

[TOC]

### Present

* Casper
* Simon
* Giovanni
* Sebastiaan
* Chris
* Francisco
* Flaviano
* Leopold
* Sasha

### Catch-up round

*Max. 3 minutes each*

Leopold:
* [AiiDA-powered research questionnaire](https://www.aiida.net/science/): somebody interested in sending this year's edition together with me (March)? Francisco; to be done early March so answers are ready by April for the MARVEL review.
* GSOC 2021 participation: projects and tutors? [timeline](https://summerofcode.withgoogle.com/how-it-works/)
* [Our project proposals for last year](https://github.com/aiidateam/aiida-core/wiki/GSoC-2020-Projects)
* Note: GSOC have halved the expected (=paid) work hours for the projects: 350h=>175. Time period to complete: 12 weeks => 10 weeks (18h week if fully spread out; can participate "part time")
* AiiDA REST API for workflow submission - presentation after the meeting => Moved to Jan 27th at 4pm
* Investigating [memory leak in runners](https://github.com/aiidateam/aiida-core/issues/4603) (confirm accumulating memory references)
* Casper's [/querybuilder REST API endpoint PR](https://github.com/aiidateam/aiida-core/pull/4337) reviewed, almost ready to go! Open questions
* Do we add a hidden `--enable-post` option to the `verdi restapi` cli or do we leave this configuration in the API config.py (note: this will likely become a `--disable-authentication` at some point)
* The REST API currently adds `full_type` to responses, which are then removed again in the `/querybuilder`. Should this be made optional?
* PR contains a hack for dealing with multiple projections.
* REST API: `<id>` sometimes refers to UUID exclusively, sometimes to PK, ... [issue](https://github.com/aiidateam/aiida-core/issues/4650)
* 23 open PRs, major open issues (memory leak in daemon runners; asyncio testing to do, repository testing to do). Schedule next coding days?

Chris:
* Plumpy:
* Merged full type annotations, improved docstrings, minor fixes
* New documentation with Jupyter Notebook tutorial: https://plumpy.readthedocs.io/en/latest/tutorial.html
* Releasing v0.18.2 soon
* aiida-core:
* Add `verdi group delete --delete-nodes` (#4578): a number of improvements to the deletion of nodes API/CLI, and exposes API functions *via* `from aiida.tools import delete_nodes, delete_group_nodes`
* Fixed benchmarking tests for engine asyncio
* version 1.6 release: confirm milestone critical issues https://github.com/aiidateam/aiida-core/milestone/38
* New archive: mainly waiting on [new file repository](https://github.com/aiidateam/aiida-core/pull/4345)

Marnik:
* Presentation at [PRACE workshop on delivering online courses]( https://events.prace-ri.eu/event/1146/).
* PR's:
* **[OPEN]** Docs: Add "How to extend workflows" section [[#4562](https://github.com/aiidateam/aiida-core/pull/4562)].
* QE - **[OPEN]** Add the `create_magnetic_allotrope` function [[#640](https://github.com/aiidateam/aiida-quantumespresso/pull/640)].
* Feedback from Sasha on CP2K por favor 🙏
* QE - **[MERGED]** `PwRelaxWorkChain`: Move `RelaxType` input to `get_builder_from_protocol` [[#634](https://github.com/aiidateam/aiida-quantumespresso/pull/634)].
* Might be nice to get some feedback on API

Simon:
* The 'test-install' workflow is failing infrequently, because of non-deterministic time-outs.
- Sebastiaan: The QE plugin repo has a nightly build with slack notification which might serve as template for implementing notification workflow.

Sebastiaan:
* Pull requests:
* Dependencies: update requirement `plumpy~=0.18.1` to make new `asyncio` implementation compatible with Jupyter [[#4642]](https://github.com/aiidateam/aiida-core/issues/4642)
* CLI: ensure `verdi database version` works even if schema outdated [[#4641]](https://github.com/aiidateam/aiida-core/issues/4641)
* **[OPEN]** `CalcJob`: add the option to stash files after job completion [[#4424]](https://github.com/aiidateam/aiida-core/issues/4424)
* **[OPEN]** Data: add plugin type `RemoteFolderData` and `RemoteStashData` [[#4627]](https://github.com/aiidateam/aiida-core/issues/4627)
* **[OPEN]** `RemoteData`: deprecate automatic opening of a transport [[#4637]](https://github.com/aiidateam/aiida-core/pull/4637)
* **[OPEN]** Implement the new file repository [[#4345]](https://github.com/aiidateam/aiida-core/issues/4345)

Casper:
- Addressed review by Leopold of POST PR [#4337](https://github.com/aiidateam/aiida-core/issues/4337), but still under review.
- Opened [#4660](https://github.com/aiidateam/aiida-core/issues/4660) concerning creating a schema for the `QueryBuilder.queryhelp`.
- Had an issue where all my profiles were not updated with the new RabbitMQ keys.
- **aiida-optimade**
- Add support for CifData.
- **[OPEN PR]** Initialize DB without storing large values (e.g., `cartesian_site_positions`).
This is relevant especially for CifData.
- Currently working on implementing on-the-fly calculations of non-stored fields.
- With the help of Giovanni we'll try to create an index on the JSONB fields in order to improve query times.

Giovanni:
- [Robustness] [improved handler for Quantum ESPRESSO error suggested](https://github.com/aiidateam/aiida-quantumespresso/issues/637)
- [Performance] Converting with Casper the big DB of Leonid's MD to expose it with OPTIMADE (a few tens of thousands of trajectory data, with a total of ~4M steps, converted to StructureData). It worked, now investigating query speed (and e.g. if a GIN index will improve performance or not).
- [Robustness] Checking and reporting a few more (hopefully final!) issues with the new asyncio engine. [#4649](https://github.com/aiidateam/aiida-core/issues/4649), [#4648](https://github.com/aiidateam/aiida-core/issues/4648). See discussion on next goals below.
- [Robustness/performance] Some issues with the repository migration, see [this comment](https://github.com/aiidateam/aiida-core/pull/4345#issuecomment-750552972) - some might be related to the new code (SQLite/SQLA error in the wrong thread) but others maybe just related to the new asyncio?
- [Performance] Migration (of archives) from 0.10 to 0.11 is very slow!

Francisco:
- Just got back from holidays so not much to report.

Flaviano:
- Effective mass calcfunction
- Upload CIF file to import structure via AiiDA-POST
- Overview of AiiDA-POST after the meeting

Sasha:
- Mostly worked on the aiida-common-workflows (CP2K)
- Noticed that (on Quantum Mobile) I can't kill jobs through `verdi process kill`.

### Specific Agenda Items

Giovanni:
* move the meetings earlier, to allow Jason to participate?
* revisit roadmap/timeline; need for everybody to help and test develop, otherwise we don't manage to release it
* Continue keeping the focus on performance (for large datasets/throughput) and robustness

Leopold:
* AiiDA REST API for workflow management - presentation by Flaviano and me after the meeting => moved to Jan 27th at 4pm