# DF Designer Intern Tasks

## Frontend

The frontend of DF Designer is a React application written in TypeScript. Its job is to render a graph received from the backend, allow the user to edit it, then send updates to the backend.

### Interdependence between tasks

Tasks 1, 2, and, to an extent, 3 can be done in parallel. Task 4 needs to be done after task 1 and 2 are done. Task 5 (DevOps) needs to be done continously throughout development.

For developing task 1 independently a simple third-party graph renderer should be used to visually test the layout.

For developing task 2 independently a simple third-party graph layout library should be used to generate dummy graphs for testing the rendering.

For developing task 3 independently a few dummy graphs should be hand crafted on which the window can be tested.

### Task 1. Graph Layout

The first challenge to tackle is the automatic layout of the graph. The backend does not provide coordinates for the nodes, just their properties and the edges between them.

Altough there are some ready-made graph layout libraries out there, after some experimentation, none of them seemed to meet ou specific goals (see challenges further down). Nevertheless, it is well worth studying them before building our own. Here's a list of relevant libraries:

- [dagre.js](https://github.com/dagrejs/dagre) - Implements a custom algorithm, as far as I can tell. Unmaintained and in practice produces suboptimal (ie. messy) layout for our purposes.
- [elk.js](https://github.com/kieler/elkjs) - Implements a variety of algorithms and in general has lots of options. The [`Layered` algorithm](https://www.eclipse.org/elk/reference/algorithms/org-eclipse-elk-layered.html) is of paricular interest to us. JS code is transpiled from Java, so it's better to read the original.
- [grandalf](https://github.com/bdcht/grandalf) - Written in Python, implements Sugiyama algorithm.

#### 1.1. Challenges

- Nodes must be laid out automatically
- Nodes must be organized into fixed columns (see image)
- Within a column, nodes must be ordered such that the edges between columns cross each other as little as possible
- Graph updates must be fast (possibly incremental)
- There has to be an option to group nodes in one flow closer together (clustering constraint)
- There has to be option to exclude nodes matching a specific query from the layout (tags filter).
- Support virtual nodes -- nodes that don't come from the graph, but are added in by external services (eg. recommendations from predictors).

This is arguably the most challenging task on the front end side. It requires a good understanding of algorithms, graph theory and data structures, as well as research into the topic.

#### 1.2. To do

1. Research layout algorithms. Read the linked libraries, some of them have references to papers, it is recommended to read those as well.
2. Come up with the custom algorithm. Write a pseudo-code version with explanation and add it to the Architecture page.
3. Implement the algorithm.
4. Write rigorous unit tests. Test edge cases and benchmark performance.
5. Iterate and optimize.

### Task 2. Graph Rendering

The layouted graph needs to be rendered. The current plan is to use [React Flow](https://github.com/wbkd/react-flow/pulse) for this, but the viability of this needs to be tested.

#### 2.2. Challenges

- The graph needs to render fast.
- The graph needs to be pannable/zoomable.
- Nodes need to be deletable.
- There has to be a mechanism for adding new nodes.
- Existing nodes need to be easily connectable.

#### 2.1. To do

1. Create a storybook with the required components:
1. Node blocks
2. Transition blocks
3. Connections
4. Navigation elements
2. Test the viability of React Flow. Benchmark performance and stability for large graphs (~500 nodes, ~1000 edges) with frequent updates.
1. If it in principle proves viable, explore optimiziations, eg. culling not visible nodes.
2. It it does not prove viable, find/build a React based alternative.
1. Describe the planned working of the alternative. Add the description to the Architecture page.
2. Implement the alternative.
3. Write unit tests and add visual regression testing.
3. Add navigation (zoom/pan) to the graph view.
4. Implement a React hook, which maps actions in the UI (node/edge added/removed) to changes in the graph.
5. Unit tests for the hook.

### Task 3. Details window

When a node or a transition is selected in the graph a window must appear on the right side which allows editing (and reading) the properties of the selected object. See Figma drafts for an idea of how this should look like.

#### 3.1. Challenges

- Window must display current properties of the selected object and allow editing them:
- For both transitions and nodes:
- Title
- For transitions (conditions):
- The condition function: either selected from a pre-defined list or suggested based on a en example utterance. The latter case must fetch the suggestion from external annotators (See the Architecture page).
- The arguments of the selected condition function.
- For composite conditions, a nested menu which allows editing each individual condition and adding/removing new ones.
- For nodes:
- Response: either a string or auto-generated (latter is WIP).
- Response parameters: allow adding parameters for the response (MIDAS, sfc, etc.) either manually, or via a classifier (external).
- Comment: a comment to be inserted next to the response.

#### 3.2. To Do

1. Study the graph structure to learn about the properties that the window needs to display/edit. Also read thorugh the external services documentation.
2. Create a storybook with the required components (*TODO: create separate Figma designs for each of these*):
1. Tabbed view
2. Condition selector
3. Argument editor
4. Composite condition editor
5. Annotator selector
6. Response parameter editor
3. Write hooks that connect to the external annotators (see Architecture for details) + tests.
4. Write hooks which map UI actions to changes in the graph + tests.
5. Put the window together
6. Integration and visual regression tests.

### Task 4. Node Filters

There needs to be a way to filter the nodes shown on screen. This should be done in a way similar to how Trello allows filtering cards.

Suggested filters:
- Flow
- Reponse parameters (MIDAS etc.)
- Recommendations

Recommendations could also be implemented as a filter. When the user wishes to see recommendations from a specific predictor, they turn on the specific filter and see the what the predictor recommends for the selected node. The recommended nodes are "virtual", that is they are added by an external service and not the parser.

#### 4.1. Challenges

- Turning filters on/off should be instant
- Filters should be searchable
- Filters should be extensible
- Filters should support virtual nodes by predictors

#### 4.2. To Do

1. Create a hook for the graph renderer which, based on the active filters, filters the nodes that should be shown + tests.
2. Create a storybook with the filter selector window
3. Create a hook for the graph renderer which, based on the active filters, connects to external predictors and adds virtual nodes to the graph based on the response.
4. Add integration tests and benchmark.

### Task 5. DevOps and other

Github actions should be set up during development. All tasks describe what tests they require. As these tests are added, they should be added to CI via GH actions.

Linting and formatting should be added for all source files:
- For TS - ESLint and Prettier
- For Python - Flake8 and Black