AiiDA Team Meeting 2021-01-27

# AiiDA Team Meeting 2021-01-27 ###### tags: `team meetings` ###### time: 10am CET [TOC] ### Present - Chris - Marnik - Giovanni - Flaviano - Leopald - Jason - Simon - Caspa - Seb ### Catch-up round *Max. 3 minutes each* Chris: * Type Checking: all of `aiida/engine` (+bug fixes) [[#4669](https://github.com/aiidateam/aiida-core/pull/4669)] * Fixes two identified bugs, one where `killing` future was None, now leads to warning: now leads `logger.warning(f'killed CalcJob<{node.pk}> but async future was None')` * Created <https://github.com/aiidateam/aiida-integration-tests> for testing/profiling of issues that cannot be captured by unit tests * Informed reproduction of issues "Memory leak in daemon runners?" (#4603), "Ensure that children processes are properly killed when parent killed and clean up nodes priority/important topic/engine topic/processes type/bug" (#3776) * Lead to bug fixes: [[#4682](https://github.com/aiidateam/aiida-core/pull/4682)] fixes the `kill_process` reference and [[#4683](https://github.com/aiidateam/aiida-core/pull/4683)] fixes the `PluginVersionProvider` reference. * Add `to_aiida_type` to public API (import from `aiida.orm`) [[#4672](https://github.com/aiidateam/aiida-core/pull/4672)] Leopold: * Proposal for GSOC 2021 projects ideas: * NEW extension of AiiDA REST API to workflow management: we give the student the design; they implement a well-defined subset of it * NEW extensibility of REST API through python entry points * KEEP Performance optimizations at the ORM level: could limit to object insertion only (potentially even just one db backend); remove task of also moving `verdi import` onto this implementation * KEEP? container support for AiiDA Codes (note: may want to go into conda-forge direction instead) * NEW? Could potentially also include scheduling-related ideas (task farming; basic cross-computer scheduling) but these need robustness and would probably require extensive testing. Perhaps less suitable * Proposal: add jinja2 templating inside code/computer config files? [issue](https://github.com/aiidateam/aiida-core/issues/4680) Simon: * Requesting input on NumFOCUS newsletter update. (result: no updates) * (in progress) Implement Slack notification for test-install GitHub Actions workflow. Casper: - Finally merged [#4337](https://github.com/aiidateam/aiida-core/pull/4337), which introduces the `/querybuilder` endpoint open for POST requests to supply a `QueryBuilder.queryhelp` dictionary, which will perform and return the QueryBuilder result. - Also fixed some CI issue and updated a bit of REST API code in the process. - Two new issues have sprung up due to this PR: [#4676](https://github.com/aiidateam/aiida-core/issues/4676) and [#4660](https://github.com/aiidateam/aiida-core/issues/4660). - Giovanni spent a day with support from myself to test whether retrieving data from JSONB fields in the DB can be optimized. There were mainly two possible solutions: 1. Creating GIN indexes on the Extras column of the Node DB table. 2. Creating a separate table that _only_ contains the necessary Extras data, possibly combined with a GIN index and related to the Node table through the Node PK. The main results can be seen in issue [#4664](https://github.com/aiidateam/aiida-core/issues/4664). Sebastiaan: * Improved new repository implementation migrations after feedback and testing from Giovanni. Should be in a state that would be ready for merging **Need more testers!** [Instructions are on the wiki](https://github.com/aiidateam/aiida-core/wiki/AiiDA-2.0-plugin-migration-guide#testing-the-migration) * Finalized the discussion on stashing of files after calculation job completion together with Marnik, Francisco and Giovanni - Decided to leave the current interface of `RemoteData` as it is and not make passing the `Transport` in required. Reason being that there are still some disadvantages in that design as well and we need to fully work those out before changing the interface. Removed [the issue](https://github.com/aiidateam/aiida-core/issues/4636) from milestone v1.6 - Also decided to not touch `RemoteData` for now. Stashing will simply use a new data type `RemoteStashData` but that is not related to `RemoteData`. The only benefit of making it a subclass was ease of use with querying, but without it it is still possible and the subclassing would have required refactoring, a migration and potential breaking of code, so the advantage did not weigh out the cost. Removed [the issue](https://github.com/aiidateam/aiida-core/issues/4626) from milestone v1.6 - Add tests to the [PR](https://github.com/aiidateam/aiida-core/pull/4424) which is now ready for review. Only documentation is still missing. I very much welcome any help here. Francisco: * [#4678](https://github.com/aiidateam/aiida-core/pull/4678) - Hash calculation for nodes with -0 value was generating different hashes pre-stored and post-stored (it gets stored as 0). * [#4671](https://github.com/aiidateam/aiida-core/pull/4671) - When using `verdi process play -all`, only paused "active" processes will be pinged (opposed to terminated ones). * [#4670](https://github.com/aiidateam/aiida-core/pull/4670) - Improve error message when trying to run jobs on non-configured computers. Marnik: * ABC tutorial on 10 FEB 2021: schedule can be found [here](https://docs.google.com/document/d/1XaKH7oOM4d6F7jDyfU1Pq9Ov7IBkEnue1FVMCjFTkXk/edit#heading=h.29prq8qauryg). Looking for standby, mainly for helping to install. * In progess: AEP 006 - extending stashing functionality. * [QE] `aiida-quantumespresso` release 3.3.0 🎉 [[#641]](https://github.com/aiidateam/aiida-quantumespresso/pull/641) * [QE] Final touches to the protocol. [[#634]](https://github.com/aiidateam/aiida-quantumespresso/pull/634), [[#635]](https://github.com/aiidateam/aiida-quantumespresso/pull/635), [[#638]](https://github.com/aiidateam/aiida-quantumespresso/pull/638) Giovanni: * Performance of DB - good to have focused discussion at some point * Focus of next coding days? * Who tested the new repository implementation? Flaviano: * Nothing to add today. ### Specific Agenda Items #### Coding Days (Feb 8th/9th) * **Target:** release 1.6 * **Preparations:** add any issue or PR that you want included to 1.6 to the corresponding [milestone](https://github.com/aiidateam/aiida-core/milestone/38). *Deadline: 29th of Jan (end of this week)*. * **Note:** all current issues in the milestone are engine related. #### General outstanding problems * Problem with broken links in doc search => RTD is not fixing it ([see issue here](https://github.com/readthedocs/readthedocs.org/issues/7487)). Should we try do something about this on our end? Is there even something we can do? * Problem with jedi and ipython ([see issue here](https://github.com/aiidateam/aiida-core/issues/4668)) => iPython is not yet releasing a fix and this breaks all shells. Should we pin jedi (`jedi==0.17.2`) on the next release?