---
tags: modularization, plugin
---

# AiiDAlab-qe plugin implementation

## Overview of QeApp design
QeApp uses the Wizards UI, which divides one calculation into four steps. Each step may contain several sections (panels), as shown below.

<img src=https://hackmd.io/_uploads/Hy8WYsxSn.png />

## Downsides of the existing implementation
Currently, each step corresponds to one Python `class`. For example, in `Step 2`, all configurations (workflow, basic and advanced settings) are written in the `ConfigureQeAppWorkChainStep` class. The settings for different workchains (relax, bands, pdos) are also mixed. The code is bloated and the logic of the code is hard to follow, thus, it is difficult to maintain and review. This problem becomes more serious when we add extensions to the QeApp, e.g., adding advancing settings (charge, Hubbard), or new properties like XPS, XAS, etc.

This problem also exists in the `WorkChain` class. Currently, all sub-workchains (relax, bands, pdos) are written in the `QeAppWorkChain` class. One needs to expose the input and output of the sub-workchains, and handle the builder of the sub-workchains in one `get_builder_from_proptol` method.

Overall, it is becoming challenging to maintain the QeApp code in these big `classes`, and it is not flexible to add new features.

## Improve it by using the plugin implementation
The idea is to make the QeApp more modularized, and pluggable. Take into account the following facts:
- the configuration for a property calculation (e.g. pdos, xps) has its settings unrelated to other properties.
- the sub-workchain of the properties can be run independently.
- the analysis of the results of the properties is also independent.

Thus, we could divide the big `classes` into several small groups, and each group is responsible for one property calculation. For example, we could create a group for the `PDOS` property, which includes its settings, workchain, and result analysis. **The `PDOS` group is only loaded when the user selects to run the `PDOS` property, which means it is pluggable.**

**GUI**
Here is an example of configuration (`Step 2`), when the user selects to run `PDOS` and `XPS`, the panels for the `PDOS` and `XPS` will show.
<img style="border: 1px solid #555" src=https://hackmd.io/_uploads/HJnCnTeB3.gif />

**Workchain**
Here is a example of the `WorkChain`:

<img style="border: 1px solid #555" src=https://hackmd.io/_uploads/BJP2bRlS3.png />

Above is the idea of the plugin. A plugin is just a way to encapsulate a Python module in a way a user can easily utilize. When users enable a plugin, the QeApp can load it and display it with useful information.

Since QeApp is purely a Python package, we could discover the plugin via a well-known Python Entry Point, as shown below.

[options.entry_points]
aiidalab_qe.property =
    bands = aiidalab_qe.bands:property
    pdos = aiidalab_qe.pdos:property
    xps = aiidalab_qe.xps:property
    xas = aiidalab_qe.xas:property

So the developer can maintain their plugin as a separate folder in the QeApp (even a separate pakcage). In this way, the logic of the code is more clear, and it is easy to maintain and review. Besides, the QeApp can be easily extended by adding new features.

## How to write a QeApp Plugin?
A QeApp plugin will typically register panels (setting, result), and workchain. To give an example, here is the simplest plugin to print the formula of the input structure:

**Outline**, it will be shown as a checkbox in the workflow panel, as shown below.

class Outline(OutlinePanel):
    title = "Hello World"

<img style="border: 1px solid #555" src=https://hackmd.io/_uploads/SJxZyeWSh.png />

**Setting**, it will register a new panel in the configuration Step. In this class, one needs to implement the `get_panel_value` and `load_panel_value` methods to tell QeApp how to handle the panel values.

<img style="border: 1px solid #555" src=https://hackmd.io/_uploads/SJavjkWSn.png />

class Setting(Panel):
    """"""
    title = "Hello world"

    def __init__(self, **kwargs):
        self.name = ipw.Text(value="", description="Your name:")
        self.children = [self.name]
        super().__init__(**kwargs)

    def get_panel_value(self):
        return {"name": Str(self.name.value)}

    def load_panel_value(self, input_dict):
        self.name.value = input_dict.get("name", 1)

**Result**, it will register a new panel in the Results Step. In this class, one needs to implement the `_update_view` method to tell QeApp how to show the outputs of the workchain.

<img style="border: 1px solid #555" src=https://hackmd.io/_uploads/Sy3cdWWrn.png />

class Result(ResultPanel):
    title = "Hello world"
    workchain_label = "hello_world"

    def _update_view(self):
        name = self.node.outputs.name.value
        formula = self.node.outputs.structure.get_formula()
        self.summary_view = ipw.HTML(
            f"""<div> <h4>Hello {name}</h4> The input structure is: {formula} </div>""".format()
        )
        self.children = [ipw.HBox(children=[self.summary_view])]

**WorkChain**, it will be loaded into the QeApp workchain. One needs to implement a `get_builder` function to extract the input parameters for the sub-workchain.

def get_builder(codes, structure, parameters):
    """Get the workchain specific parameters
    """
    parameters = parameters.get("hello_world", {})
    builder = HelloWorldWorkChain.get_builder_from_protocol(
                codes=codes,
                structure=structure,
                parameters=parameters,
            )
    return builder


workchain_and_builder = [HelloWorldWorkChain, get_builder]

**Entry point**, here is the entry point for this plugin. One needs to add it to `entry_points` inside the setup file.

property ={
"outline": Outline,
"setting": Setting,
"workchain": workchain_and_builder,
"result": Result,
}

entry_points={
        "aiidalab_qe.property": [
            "hello_world = aiidalab_qe_hello_world:property",
        ],
    },

Note: one plugin does not need to register all the items (settings, workchain, results). The panel in each step is pluggable, which means you could only register one item in a plugin. For example, you can only add a new `Structure` panel in Step 1 without doing any property calculation.

You can add this plugin as a folder in the QeApp package, or create a new package for it.

**Bringing It All Together**, You can find all the code above in this [github repository](https://github.com/superstar54/aiidalab-qe-hello-world).

### Your second QeApp plugin

Here is another example to do equation-of-state calculation. https://github.com/superstar54/aiidalab-qe-eos

## Conclusions
In this new design, QeApp provides the entry point to extend the functionality without having to dive into QeApp core code. QeApp plugin can encapsulate certain functionality neatly to add new extension (e.g., XPS). The new design makes QeApp modular, easily maintainable and extendable.

The example given above is limited, but shows the QeApp Entry Point used for common tasks that you can expand on to write your own tools.

## Feedback from the plugin meeting

- add a search page to show all the calculated properties related to one structure. Consider the possibility for the QEapp plugins to add/update, upon successful completion, an extra in the structure with the uuid of teh workchain. e.g. ...inputs.structure.set_extra('qeapp',[workchain_uuid1]) This will empower users / other developers to write search apps that are structure centred rather than workchain centred. From a structure it is then possible to directly link all properties that have been computed on it and directly link to the visualizers for the specific properties (e.g. opening the pertinent QEapp plugin)

- add caching to reduce duplicate calculations from different sub-workchain. For example, a SCF calculation from XPS and Bands.
- consider already refactored codes.

### Other considerations
- each sub-workchain run in parallel. For large systems, we may want to reuse some duplicate calculations (e.g. scf), so we should also support sequencely run.
- even though, each sub-workchain is run independently, we could set some common variable (e.g. scf_parent_folder) in the `ctx`, we could let the sub-workchain override this common variable.
- If a user has several separate calculations on different properties, but the configuration parameters do not conflict, we could in principle merge the workchains and show the results at the same time.
- try to find a way to link the plugin property with the base step. e.g when the user selects a property to be run, it will reset the state of the "Confirm" button.
- special treatment for the pdos calculation, the smearing should be `tetrahedra` and should not be overridden.

## Step 1, PR for UI

- create a QeApp class
- for each step, the class has a parent attribute

- `workchain_settings` is splited into baisc settings (spin, electronic type ...) and `workflow`.
- `pseudo_family_selector` and `advanced_settings` are merged into `advance_settings`
- link basic protocol to all plugin specific protocols
- removed the `workchain_settings`, `pseudo_family_selector` and `advanced_settings` traitlets because they are not used anymore. They can be read by other steps (e.g. step 3 submit) from the parent.

#### We don't need the override checkbox.
In the advaced settings, the default values are the the values defined by the protocol.

In the old design,

if override:
    use the widget value
else:
    # use the value from the qeapp.yaml, or get from the protocol.
    default_value
# However, the default value of the widget is
default_widget_value = default_value

Therefore, we could use the value of the widget directly in all the cases (override and not override)

### TODO
- Add reset method for panel, how to handle the default parameter from qeapp.yaml?

## Step 2, PR for workchain

## Step 3, PR for nice to have features
- make `StructureSelectionStep` pluginable. However, it uses `StructureManagerWidget` from aiidalab-widget-base.
- make