# AiiDA Team Meeting 2023-09-21

###### tags: `team meetings`
###### time: 14:00 CET

[TOC]

### Present

* Marnik
* Miki
* Xing
* Jusong
* Sebastiaan
* Kristjan
* Edan
* Giovanni

### Catch-up round

*Max. 3 minutes each*

Sebastiaan:
* [`set_extra` gets slower and slower in iterations of loop](https://github.com/aiidateam/aiida-core/issues/1456#issuecomment-1728460721): Revisited issue from 2018 reporting super-linear scaling when setting extras on node in loop over querybuilder result. Added new benchmarks and showed that problem is use of `.all()` instead of `.iterall()`. The former doesn't use a transaction. With manual transaction, `.all()` becomes almost as fast, but still worse for big result sets. Recommendation to use transaction requires [PR #6125](https://github.com/aiidateam/aiida-core/pull/6125) to be fixed as currently storing inside a loop in a transaction will except.
* Would like to update `aiida-core` to SqlAlchemy v2.0. But first need to update `disk-objectstore` as it pins `~=1.4`. It was already getting close to a v1.0 release, so maybe we include this change as well? Opened two PRs:
* [Dependencies: Unpin sqlalchemy](https://github.com/aiidateam/disk-objectstore/pull/158)
* [Devops: Remove deprecations for v1.0](https://github.com/aiidateam/disk-objectstore/pull/160)
* Finished and merged a few outstanding PRs of Chris:
* [CLI: Add `verdi tui`](https://github.com/aiidateam/aiida-core/pull/6071): New optional dependency group `tui`, which when installed, will provide a TUI through `verdi tui`
* [`SqliteTempBackend`: Allow to read/write archives](https://github.com/aiidateam/aiida-core/pull/5658)
* [Typing: Improve process function typing](https://github.com/aiidateam/aiida-core/pull/6077)
* Various PRs merged by Daniel Hollas that improve import time
* Open PRs:
* [Docs: Add important note on using `iterall` and `iterdict`](https://github.com/aiidateam/aiida-core/pull/6126): short and sweet, who can give it a review?
* [`PsqlDosBackend`: Fix `Node.store` excepting when inside a transaction](https://github.com/aiidateam/aiida-core/pull/6125): Ready for review
* [Config: Switch from jsonschema to pydantic](https://github.com/aiidateam/aiida-core/pull/6117): All ready, just waiting for [`emmet-core`](https://github.com/materialsproject/emmet/pull/790) (sub dependency through `pymatgen`) to release compatible version. Anyone want to already review the PR?
* [Deprecation: `aiida.orm.nodes.data.upf` and `verdi data core.upf`](https://github.com/aiidateam/aiida-core/pull/6114): Anything more to do before this can be merged Marnik?
* [`Scheduler`: Refactor interface to make it more generic](https://github.com/aiidateam/aiida-core/pull/6043): _Open Question_: What do we call the new base class. Currently it is called `BashScheduler`. Any better options?
* [Rename workchain subprocess queueing](https://github.com/aiidateam/aiida-core/pull/5838): Deprecating `to_context/ToContext` in favor of `queue_subprocess`. This a long outstanding PR from Chris. I think it is an improvement in the user interface. However, the question is whether it is worth it, since the use of `to_context/ToContext` is so prevalent. The migration path is straightforward, potentially could be automated with `aiida-upgrade`.

Marnik:
* Nothing to report on the `aiida-core` front.

Miki
* [aiida-atomistic](https://github.com/aiidateam/aiida-atomistic) uploaded on github (branch "new_Data"), I can show some slides on the StructureData (just an example, more details [here](https://hackmd.io/IN19UdRDRICDhyDGMdsUUQ))
* discussion on StructureData, Monday 24, 3PM

Xing:
* Had a discussion on aiida-worktree.
* Since it is in the early development stage, Sebastiaan suggested maintaining it in a separate repository rather than adding it to aiida-core, so that it can be updated fast.
* Marnik will try to test in his workflow.

Jason:
* The new aiida-registry is announced, but since it uses the old `aiidateam/aiida-core:lastest` image it test install plugins in aiida-core==2.3.1, see [PR](https://github.com/aiidateam/aiida-registry/pull/280) where I tried with `aiidateam/aiida-core-with-services:edge`. When will the new version released that contains the docker stack implementation?
* I create [`aiida-resource-registry`](https://github.com/aiidateam/aiida-resource-registry) with template support in order to replace aiida-code-registry. The usage of it is [implemented in AiiDAlab widget base](https://github.com/aiidalab/aiidalab-widgets-base/pull/511). Same concept even part of implementation can be used for verdi computer/code setup.
* Do we need a follow up discussion on how to set discource categories, new users are ask questions in "new to AiiDA" channel.

Kristjan:
* Worked on the backup command: https://github.com/aiidateam/aiida-core/pull/6069
* Xing reported a problem of asking ssh passwords multiple times, probably makes sense to add a parameter `--ssh_password` to the cli.
* Will add a restore command when i get time
* `verdi storage backup --path x --restore (--profile y)`
* DOS backup function: https://github.com/aiidateam/aiida-core/blob/backup-script/aiida/storage/psql_dos/backend.py#L429
* to the dos repo?

Giovanni:
* For Sebastiaan: [Issues with things to do (at least) in AiiDA-core after releasing disk-objectstore 1.0](https://github.com/aiidateam/aiida-core/issues/6081). Also, close https://github.com/aiidateam/disk-objectstore/issues/155 - @7wMTWWsmS1i2a1QmzsgXgg let me know if you want to discuss specific decisions such as defaults for the command line in verdi storage maintain, ...

### Specific Agenda Items