# Work Items

# Further Progress since 4 november

* expanded toy benchamrking study
* fitted 10 directional change algorithms on daily M4 benchmarking dataset. Overall, ~61k+ fits.
* Set up infrastructure to use 3 distinct types of directional change algorithms
* Committed changes to repository, updated aggregated results from experiments. For reference see links from the previous work item.

# Progress since 4 November

* Set up benchmarking study
* switched from kotsu to mlflow for logging results
* Fixed bugs in Network Pipeline
* Conducted benchmarking study on M4 daily dataset for 3 directional change algorithms (totalling 12k+ fitted estimators)
* code for benchmarking study available in this repo: https://github.com/ViktorKaz/m4benchmarkstudy
* Logs of results available in this file: https://github.com/ViktorKaz/m4benchmarkstudy/blob/main/runs.csv
* Aggregated results available in this Jupyter Notebook:
https://github.com/ViktorKaz/m4benchmarkstudy/blob/main/analyse_results.ipynb
* entry point for running benchmarking study: https://github.com/ViktorKaz/m4benchmarkstudy/blob/main/orchestration.py
The interface for running the experimentis quite clean:

orchestrator  = Orchestrator(models_gen,data_gen, evaluator)
orchestrator.run()

# Work items Meeting 4 November 2022

## Network pipeline
1. Added the possibility to pass variables directly to the steps of the pipeline
2. Added the possibility to use any function as a step in the pipeline. Previously the algorithms that can be added as steps were limited to transformers and forecasters.
3. Added the possibility to refer to fitted estimators in the steps. Previously the functionality was limited to using the last output of any given step. From a practical point of view this was limiting what can be done at `predict`, i.e. the pipeline could only use the predictions of the previous steps but subsequent steps did not have access to the fitted estimators used to generate these predictions.

## Directional change algorithms

Implemented the following directional change algorithms by using the network pipeline:

1. Directional change reduced to forecasting
2. Directional change reduced to supervised classification
3. Directional change reduced to forecasting with exogenous variables