## Note on WorkGraph during the coding week Here you can add what you want to do, and update what you have done. ## Ideas - how easy it is to swap out quite different nodes that do the same thing (Elliott) - Or generalise workgraph builders e.g. high throughput `CalcJob(struct_file, **inputs)` -> `CalcJob(struct_dir, **inputs)` - Easier access to output from tasks e.g. not having to use `wg.tasks[-1].outputs[0].value` for final task (Elliott) - Set the input of a task using `builder` directly. (Riccardo) - Validate all the required inputs before submission (Daniel) - Visualising nested tasks e.g. `add_multiply_if1` in graph builder tutorial? (Elliott) - Timeout for `wg.run` (Elliott) ## Xing ## Julian As commented with Xing: - No `aiidateam` account on pypi.org - For `aiida-core` some maintainers still there, but people who left the group - How to handle it for WorkGraph? ## Elliott Kasoar - Two workgraphs Federica started that I'll be working on are https://github.com/stfc/aiida-mlip/pull/147 and https://github.com/stfc/aiida-mlip/pull/153. - Abstract version of #147 (geomopt -> any calculation): https://github.com/stfc/aiida-mlip/pull/169 Other thoughts/questions/issues: - Clearer distinction with WorkChains would be useful, in terms of: - Difference in features (current/planned) - When both a user and plugin developer should choose each - Similar to the broker discussion, which differences are inherent, and which are simply not yet implemented - Planned future support (is there a chance this will be dropped?) - Maybe obvious in hindsight, but if you don't set `wait=True` on submit, results don't update e.g. `wg.tasks[0].outputs["result"]`. What's the best way to update this? - Not sure if this is workgraph specific, but `submit(wait=True)` can complete before sub-tasks are complete, which is a bit confusing - Probably due to timeout - What context is, and how/when it's set isn't always clear, e.g. knowing you can set a dict (`multiply1.set_context({"result": f"mul.{key}"})`) was hard to find ## Riccardo Bertossa started porting https://github.com/rikigigi/aiida-QECpWorkChain/blob/master/aiida_QECpWorkChain/workflow.py to workgraph. stuff: - type annotation of a calcfunction task: `List[List[float]]` and similar type annotation not working, but have to use `workgraph.Any` - very annoying to use an existing builder to add a task: need a small recursive function to unwrap the port namespaces members. Something like ``` #function that recursively loops over the builder and replaces all the members that have the _data attribute with its content def replace_data(dict_): for key in dict_.keys(): if hasattr(dict_[key],'_data'): dict_[key]=replace_data(dict_[key]._data) elif isinstance(dict_[key],dict): dict_[key]=replace_data(dict_[key]) return dict_ builder_unwrapped = replace_data(builder._data) wg.add_task(CpCalculation, **builder_unwrapped, name=name) ``` - it would be nice to have some operation on lists, for example: - Maps: you have a list `l` of len `len_l` one task with a input with the same type of the list elements. Then you want to produce a list with the same lenght and with content the output of the task: something like `[ task(x) for x in l]` - Reduction operations: list, task(x, task_result_type_of_previous_step) that iterates over the list from left, or from right, and at the end returns the last task_result - note for the unexperienced user: the documentation should try to explain very clearly that each time you need a variable that is produced by a calculation in the code (that is taken from the output of some step), a new workgraph/workfunction is needed to wait for that variable in the worker - note for myself: if you don't use links, don't use workgraph but a plain function that may take an existing workgraph in the argument, and add tasks. Add the links only when you have to - It is annoying to use dict or long list of arguments to pass around in very nested workflows. Is it possible to use https://docs.python.org/3/library/dataclasses.html to define classes of data to pass around (so if I decide for whatever reason to add a new element to the dataclass, it is forwarded everywhere in the graph, and I do not have to manually add it everywhere)? - take ispiration from existing framework: https://github.com/temporalio/samples-python/blob/main/hello/hello_activity.py - successfully ported a very small part of a messy workchain in https://github.com/rikigigi/aiida-QECpWorkChain/blob/test-workgraph/test.py (to run the workflow) and https://github.com/rikigigi/aiida-QECpWorkChain/blob/test-workgraph/aiida_QECpWorkChain/workflow2.py (graph definition) ## Nataliya - Two ways to set inputs, which one is recommended? ## Notes from discussion - `add_while_task`? - Pass class `While`, rather than string `"While"` -> Leverage Generic types - Currently only nested dictionary: Support setting also (similar to submit in AiiDA, where you can provide either the class and the input parameters, _or_ only the builder) -> Rest are GH issues :D