AiiDA Team Meeting 2025-02-20

# AiiDA Team Meeting 2025-02-20 ###### tags: `team meetings` ###### time: 14:00 CET [TOC] ### Present - Alex - Julian - Ali - Miki - Edan - Kristjan - Giovanni - Jusong - Xing ### Catch-up round *Max. 3 minutes each* Alex: - Release v2.7.0 progress (see [project](https://github.com/orgs/aiidateam/projects/15)): - Talked with Michail about production tests, still need TODO - Not there yet, 20th February was too ambitious for prerelease. --> Moving it to in 4-6 weeks, if we move it again we do a patch release with some of the database features because @Kristjan could use these now - ssh with password authentication [aiidateam/pull/6761](https://github.com/aiidateam/aiida-core/pull/6761), should we push to v2.7.0 release? There seems to be interest/momentum now, ideally for testing it --> If it slows down the release we drop it, otherwise we include it. - Continuing on the issues mentioned in my talk last week, I tried to identify a solution for creating a while representation in aiida-core that could be used in aiida-workgraph (see [slides showing current state in aiida](https://docs.google.com/presentation/d/1h4rnYwv2vLLH11ogAXNf7sNoO1OhPxVW3xyCQ7E-qo4/edit?usp=sharing)), still a lot of thinking, discussed with @Xing already, will talk with @jusongyu on Friday about it - TODO review PR from Xing regarding the while loop [workgraph/pull/409](https://github.com/aiidateam/aiida-workgraph/pull/409) - [Py3.13 support PR](https://github.com/aiidateam/aiida-core/pull/6600) depends on [aiidateam/disk-objectstore/pull/184](https://github.com/aiidateam/disk-objectstore/pull/184), [180](https://github.com/aiidateam/disk-objectstore/pull/180), [181](https://github.com/aiidateam/disk-objectstore/pull/181) and another one doing releaes for disk-objectstore Julian: - [[6763] `QueryBuilder` Implement `get_creation_statistics` for SQLite backend](https://github.com/aiidateam/aiida-core/pull/6763) by Zisen: - Ready to be merged. Tests pass. But, contains a check: `backend_class == sqlite_dos.backend.SqliteDosStorage`, should we extend that to `SqliteZipBackend` (`SqliteTempBackend`)? Function only used in `src/aiida/restapi/translator/nodes/node.py`. - Work on: [[6723] WIP: Profile data mirroring](https://github.com/aiidateam/aiida-core/pull/6723/): - For now, only mirroring of Processes, PR already rather large - Default behavior is incremental updating of directory based on existing subdirectories with data of nodes that have an `mtime` since the last mirroring time - `DumpLogger` class tracks dumped nodes and paths -> Load from (write to) JSON file at beginning (end) of mirroring - Some of the options: - `--delete-missing`: Deletes directories and enries in log file when node is deleted from AiiDA's DB -> Must be run explicitly. By default, files are kept. - `--symlink-duplicates`: If nodes in multiple collections (or otherwise duplicated e.g. due to caching) create symlinks rather than dumping the same files in a different directory. False by default, so that all subdirectories can be zipped and shared, thus containing all files. - `--update-groups`: Updates the mirrored subdirectory and dump log of a group if group was renamed - Only specific groups can be selected, only nodes in groups, etc. - Would need some testers at this stage :rocket: - GP: Test with larger DB, e.g. 100k, time for first run, and time for next runs -> Doing nothing should be very quick - GP: Possibly also check for files, e.g. if file is corrupt; or possibly write first to sandbox and then move atomically to the final directory - GP: For `verdi process dump`, see if I can use AGE tools, e.g. [see here](https://github.com/aiidateam/aiida-core/pull/3686) - Next blog posts, anybody excited to share something? Xing: - WorkGraph coding week for stable release (Feb. 24th-28th), see [project](https://github.com/orgs/aiidateam/projects/10) - Thursday (27th) MSD group meeting, WorkGraph discussion - [improved if/while syntax](https://github.com/aiidateam/aiida-workgraph/pull/409): provides the syntax sugar by using similar `if_` and `while_` instructions as in the WorkChain. Ali: - Stashing: - My development plans are explained here on issue [6764](https://github.com/aiidateam/aiida-core/issues/6764) Let me know there if you have any comments. - As of now with these two PRs, I'm providing support for compression formats during stashing, `TARGZ`, etc. [PR 6743](https://github.com/aiidateam/aiida-core/pull/6743) and [PR 6746](https://github.com/aiidateam/aiida-core/pull/6746) - Let's make a decision on ``--clean-workdir` - [PR CLI: add option --clean-workdir to verdi node delete #6756](https://github.com/aiidateam/aiida-core/pull/6756) - Blog post tomorrow: - Debugging the AiiDA Daemon Jason: - Discourse, the "annoucement" channel has been set to broadcast only the first post. - Wrap up `keyring-rs` into [`keyringrs`](https://github.com/unkcpz/keyringrs) for managing the password cross-platform. - GSoC is on the way, I put Ali and Julian as co-mentors. Edan - [Pydantic PR](https://github.com/aiidateam/aiida-core/pull/6255) - CLI issue mostly resolved, albeit with the UX regression caviat (see PR discussion) - File handling using base64 encoding - Need to understand how to handle large files (discussing with Kristjan) - Will benchmark pre-merging in aiidalab-qe - Any workflow should provide some info when compared before/after PR - Possible issue with v2.3 query - If I recall correctly, `QueryBuilder`'s `in` operation did not work for sets (which would be most performant) - This was fixed as part of the `fields` [PR](https://github.com/aiidateam/aiida-core/pull/6245), I think, which is unavailable in v2.3 - Noticed during AiiDAlab-QE development (see [this PR](https://github.com/aiidalab/aiidalab-qe/pull/1174) for details) - However, it seems `list({pk-set})` still does not resolve this matter - If indeed an issue, will open a formal one Kristjan - nothing to report Miki - nothing to report Giovanni - do not store passwords in DB (only on file or external keyring) - tested MFA support with Paramiko, @Ali do you want to get the code so we see if/what we can make official? - profile data mirroring (@Julian): - test time for second dumping with no changes, with e.g. 100k processes (both with and without symlinks) - what happens if you CTRL+C? The mtime is "recent", but might be partially written only. Will the file be "fixed/updated" at the next run, or considered "OK"? - @Edan, serving files - can we avoid pass content of files in pydantic and/or serving with the JSON in the REST API? I would just have links, and have direct download of those Discussion: - Move next meeting? - Extend ssh transport by password authentication. Related to discussions in issues [aiida-core/issues/6642](https://github.com/aiidateam/aiida-core/issues/6642), [aiida-core/issues/6688](https://github.com/aiidateam/aiida-core/issues/6688) and PR draft [aiida-core/pull/6761](https://github.com/aiidateam/aiida-core/pull/6761) -