Mentorship Programme Notes: Shivansh Subramanian
===

###### tags: `mentorship`

This is a collaborative document to keep track of progress during the mentorship programme.

:::info
- **Call time**: 3pm UTC
- **Project name**: Deep Learning for Time Series
- **Mentees**: Shivansh Subramanian
- **Mentor(s)**: Franz Király, Guzal Bulatova, Leonidas Tsaprounis
- **Call joining link**: specify
:::

## References
The sktime mentorship programme is inspired by the [The Turing Way](https://the-turing-way.netlify.app/welcome) project and [Open Life Science](https://openlifesci.org) programme under a CC BY 4.0, Open Life Science (OLS-2), 2020 license.

**Contents**

[TOC]

## Week 0

- For reference: sktime's [mentorship programme call for applications](https://github.com/alan-turing-institute/sktime/wiki/Mentorship-programme)
- Get started with HackMD using this short guide: https://hackmd.io/@openlifesci/OLS-HackMD-guide
- Please read our [Code of Conduct](https://github.com/alan-turing-institute/sktime/blob/master/CODE_OF_CONDUCT.rst)
- Check out sktime's [development roadmap](https://github.com/alan-turing-institute/sktime/issues/228)

**Preferred time and days provided by the mentor and mentee**
Please indicate your preferred day/time for your regular call:

* Friday, 1500 UTC

### Prep work for week 1

**Mentees will:**

1. Read https://ideas.ted.com/are-you-mentorable/
2. Set 1-2 personal development goals for yourself:
...
4. State how your mentors can best support you in your contribution to sktime (e.g., providing code reviews, share useful resources, explain ML concepts):
...
4. Open an issue on the [sktime/mentoring](https://github.com/sktime/mentoring/issues) repo. This can be updated during the mentorship program.

## copy of mentoring issue

**Mentee**:
Name: Shivansh Subramanian (he/him)
Current Role: Computer Science undergraduate at IIIT Hyderabad.
GitHub: @AurumnPegasus

**Mentors**:
Franz Király (@fkiraly)
Guzal Bulatova (@GuzalBulatova)
Leonidas Tsaprounis (@ltsaprounis)

**Why did you join sktime's mentorship program?**
- I am interested in open source, and wanted to contribute to something which will be useful for me, and a lot other people
- Wanted to see similarities in approach towards time series models vs NLP models, and gain more knowledge about sequential datasets
- Learn more about mathematical/statistical approach towards time series questions, and improve my skills in that area

**What topics are you working on?**:
- Improving and Adding to the sktime-dl package
- Merging sktime-dl with core sktime

**What are your learning goals?**:
- Contributing to open source, and large code base, in a way which is understandable by other developers and easy for them to develop upon it in the future.
- Understanding best code practices in multi-developer environment
- Improving upon my knowledge of sequential Deep Learning models by exploring different approaches and understanding the mathematical insights behind them.

**What's next for you after the mentorship program?**:
- completing my research in NLP
- continue contributing to sktime and other open source projects.

## Week 1 - 2022-06-03

### Agenda
* discussing aim of mentoring
* how sktime mentoring programme works
* discussing project to work on & next steps
* developer set-up

### Notes

* discussing aim of mentoring

* how sktime mentoring programme works
* using hackmd
* expect to populate 2 days ahead agenda
* time allocation: preference = 4 hours a day
* planning to finish in Oct 21, 350 hour project
* discussing project to work on & next steps
* suggest to get started with a "simple" sktime PR
* should focus on sth that's similar to interfacing or implementing DL forecasters
* some suggestions in increasing order of difficulty
* https://github.com/alan-turing-institute/sktime/issues/2733
* https://github.com/alan-turing-institute/sktime/issues/929
* https://github.com/alan-turing-institute/sktime/issues/2499
* https://github.com/alan-turing-institute/sktime/issues/2357
* https://github.com/alan-turing-institute/sktime/issues/2059
* https://github.com/alan-turing-institute/sktime/issues/1432
* https://github.com/alan-turing-institute/sktime/issues/1621
* dev guide https://www.sktime.org/en/latest/developer_guide.html
* add new estimator https://www.sktime.org/en/latest/developer_guide/add_estimators.html
* developer set-up

### actions

* work through week 0 prep above until week 2
* pick a good first issue, work on it
* ensure that you can run tests :heavy_check_mark:
* happy to help async
* can post for help in slack/contributors or mentees
* use the error messages (output/python) to understand what's wrong
* suspicion: soft deps are missing
* suspicion 2: maybe pycache folders need deleting
* add suggestion about "[dev]" to the installation instructions (trouble shooting section) :heavy_check_mark:

### development goals

- Contributing to open source, and large code base, in a way which is understandable by other developers and easy for them to develop upon it in the future.
- Understanding best code practices in multi-developer environment
- Improving upon my knowledge of sequential Deep Learning models by exploring different approaches and understanding the mathematical insights behind them.

### project goals

DL integration workstream - to be revisited after first two good first issues (plan based on how this goes)

## Week 2 - 2022-06-10

### work done

* Working installation of sktime and tests.
* conda installation was giving ```Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`.``` error when running ```show_versions()```
* Finally worked when using venv instead of conda
* Do not know why conda is not working :/
* FK: if sth like this happens, you could immediately ask in the #contributors or the #mentees channel. You should not be stuck for longer than 1h, please ask for help.
* Added documentation in troubleshooting for [all_extras,dev] :heavy_check_mark:
* FK: Where is that? Is there a PR? Haven't seen it
* Working on [#2733](https://github.com/alan-turing-institute/sktime/issues/2733) on KNeighborsTimeSeriesClassifier :heavy_check_mark:
* FK: I have already done that, since no one picked it up.
* If you work on an issue, you need to post in it that you are working on it.
* Even though I have now made a PR as well, it would be great do see your PR on this, as a learning opportunity.
* Working on [#929](https://github.com/alan-turing-institute/sktime/issues/929) on VECM
* Existing PR exists but dont know communication behind it, why it was not accepted, whats left etc

### Agenda

* review of work done
* Review of possible bugs/warnings related to ubuntu 21.04+ and conda
* Lots of warnings when doing ```make html``` for website
* Although not an issue as of now, something to keep in mind [Pystan](https://pystan.readthedocs.io/en/latest/faq.html#what-linux-distributions-are-supported)

* FK next proposed steps: picking a specific, small-ish "good first issue", then committing to make (a) a draft PR within a day or so, and (b) a reviewable PR by mid next week.

### Questions

* When to send a PR -> should I open an issue with respect to different changes proposed and then send PR? Do I need to create a new branch for each PR even if changes are minor?
* FK: I'd say, make a PR as soon as you are working, keep it in draft state until its' ready. Then turn it to "ready for review"
* FK: if the changes you are making are separate (conceptually, or code-wise), it is usually better to make separate PR.

### Notes
* Shivansh question: Existing PR on VECM exists but dont know communication behind it, why it was not accepted, whats left etc
* FK: was closed due to inactivity
* existing code may or may not be useful, feel free to use it or start over
* Take over PR for VECM and complete it

* Review of possible bugs/warnings related to ubuntu 21.04+ and conda
* FK: could be a bug - can you open issue?
* SS: Warnings reated to ```make html```, look at it during docsprint

### Actions, short-term goals

* Send the PR for documentation by June 10/11th [#2770](https://github.com/alan-turing-institute/sktime/issues/2770) :heavy_check_mark:
* Send the PR by June 10/11th [#2733](https://github.com/alan-turing-institute/sktime/issues/2733) on KNeighborsTimeSeriesClassifier :heavy_check_mark:
* Working on [#929](https://github.com/alan-turing-institute/sktime/issues/929) on VECM then committing to make (a) a draft PR within a day or so, and (b) a reviewable PR by mid next week.
* Open a BUG issue for conda bug :heavy_check_mark:

## Week 3 - 2022-06-17

### work done

* Open a BUG issue for conda bug [#2785](https://github.com/alan-turing-institute/sktime/issues/2785)
* PR sent for website documentation update [#2770](https://github.com/alan-turing-institute/sktime/issues/2770).
* Minor changes required related to text but should be done by Friday meet
* FK: thanks! issue was not linked to PR, use key phrases like `fixes +2770`, then it is automatically linked
* PR sent and closed for [#2733](https://github.com/alan-turing-institute/sktime/issues/2733) on KNeighborsTimeSeriesClassifier
* Had some extra paramteres.
* Update on [#929](https://github.com/alan-turing-institute/sktime/issues/929)
* Facing some issues since the fit and predict for VECM are part of different classes requiring various different extra parameters
* Cant test since I do not know what those parameters mean. I will try it out but will ask for help by early Thursday.
* Solved the issue early Friday, sent PR

### Agenda

* Review of work done
* FK notes/questions (moving this above sktime-dl due to discussion of DL forecasters work item)
* Discussing approach to sktime-dl and adding DL forecasters.

### Questions

* Review of work done
* FK: can you please link the issue for "conda bug"?
* thanks!
* do we understand what the root cause is?

* Discussing approach to sktime-dl and adding DL forecasters.

FK: is now really the right time to discuss sktime-dl/DL?
looks like the VECM forecaster is not yet interface compliant.

We should not move to DL before we can ensure simpler forecasters work and are compliant with sktime interface.

Useful resource on how to implement sktime estimators:
https://github.com/sktime/sktime-workshop-pydata-london-2022

recommended reading:
* ["testing" section](https://www.sktime.org/en/stable/developer_guide/add_estimators.html#how-to-test-interface-conformance) in "add estimator" dev guide
* vs code generic documentation on running pytest tests
* ensure you can run tests according to regex subsetting
* tutorial at pydata above

### Notes

FK: can you kindly ensure that the title of each meeting includes the date? E.g., `week 4 - 2022-06-24` or similar, and not `week 4 - MMM DD`

### Actions, short-term goals

* Try to reproduce the conda bug on another linux system
* Solve the failed tests for estimators for [#2829](https://github.com/alan-turing-institute/sktime/pull/2829) :heavy_check_mark:
* next week, you should be able to demonstrate how you run tests in vs code or `check_estimators`
* if you get stuck, ask for help in #contributors channel, or #mentees channel, or ping mentors directly

## Week 4 - 2022-06-24

### work done

* All tests pass locally in VECM PR [#2829](https://github.com/alan-turing-institute/sktime/pull/2829)
* Issue was that `self._y` was a pandas dataframe while residual values were numpy nd array (wierd because for VAR it was both pandas by default)
* Should be directly merge-able except for minor issues/comments
* predict_intervals is giving problems wrt conversion of whole datatype into multi indexed pandas dataframe and diff between alpha and coverage

### Agenda

* Review of work done
* What Next: Deep Learning

### Questions

* Questions about dev day meetup
* questions related to visa
* related to gsoc/taxes

### Notes

* Type conversion: VECM requires whole tuples -> multi index pandas
* Guzal: is there already existing type conversion implementation in sktime?
* Dev Days:
* Friday: Whatever you are working on, scope of work and so on.
* Reimbursement: Guzal is not sure
* Visa: maybe tourist / student?
* Taxes: GSoC Mail: If you are a resident in a given country, like Germany, and you will be visiting another country, like the US, for a 2 week vacation, you can opt to not code during that vacation period so that you do not have to fill out a tax form. Your location is still Germany and your stipend amount will reflect that. Your dashboard should also reflect Germany.
Discuss w/ your mentor your vacation timing so that they can help to extend your project length to cover that missed 2 week period.

### Actions, short-term goals

* Implement data type conversion for VECM PR [#2829](https://github.com/alan-turing-institute/sktime/pull/2829)
* Attend DL and Benchmarking meet and figure out work.
* Go through Tony's sktime-dl integration PR and figure out structure / design.

## Week 5 - 2022-07-01

### work done

* Implement data type conversion for VECM PR [#2829](https://github.com/alan-turing-institute/sktime/pull/2829)
* Need to go in a lot more depth, requires to understand whether statsmodel has implemented predict_interval or confidence_interval
* statsmodel only implements this for out of sample fh, so will need to figure out what to do for in sample fh
* Decision taken was to merge VECM with current functionality, and document all discussions / findings.
* Patching `predict_interval` for VECM with assumption that confidence_interval in statsmodel is prediction interval in sktime.
* Merge without `_predict_intervals`
* Implemented basic `_predict_intervals`, failing for some tests. Currently trying to debug it and update should be there soon.
* Go through Tony's sktime-dl integration PR and figure out structure / design.
* Opened [#2894](https://github.com/alan-turing-institute/sktime/issues/2894) to create CNN Regressor (and base regressor) based on Tony's implementation of CNN Classifier [#2447](https://github.com/alan-turing-institute/sktime/pull/2447)
* Opened a draft PR with base class for regressor [#2902](https://github.com/alan-turing-institute/sktime/pull/2902)

### Agenda

* Implement data type conversion for VECM PR [#2829](https://github.com/alan-turing-institute/sktime/pull/2829)
* Completed, need to make a decision on `predict_quantile` (need to edit the base class or implement both quantile and interval)
* [#2894](https://github.com/alan-turing-institute/sktime/issues/2894) to create CNN Regressor (and base regressor) based on Tony's implementation of CNN Classifier [#2447]
* Made the base class and CNN class on top of it, throws an error for idempotent input for some reason (for very minor differences)
* Thoughts on what can be made better
* A utils function to convert input y into one hot / label encoding
* A functionaility to specify number of linear layers and activation functions (for more data samples it might be better to create a deeper network)

### Questions

### Notes

## Week 6 - 2022-07-07

### work done

* Complete conversion functionality used in `_predict_interval` for part 2 of VECM, part 1 is PR [#2829](https://github.com/alan-turing-institute/sktime/pull/2829)
* Test passes for `predict interval` but failes for `predict quantile`. Need to explore why
* Complete the CNN Regressor [#2894](https://github.com/alan-turing-institute/sktime/issues/2894)
* Completed, need to just add an exclude test cases for `idempotent` and `pickle` tests [#2902](https://github.com/alan-turing-institute/sktime/pull/2902)
* Also need to explore why it is happening, currently no idea :(
* Opened an issue related to improvements / changes in existing CNN Classifier [#2935](https://github.com/alan-turing-institute/sktime/issues/2935)
* to act as a baseline for other deep learning modules which we can add later on

### Agenda

* Discuss work done

### Actions, short-term goals

* Start working on improvements in CNN Classifier [#2935](https://github.com/alan-turing-institute/sktime/issues/2935)
* Either complete CNN Regressor PR by excluding tests OR figure out the bug with Tony / other sktime-dl people

### Questions

### Notes

* Check out guzal's message on np random seed
* Might need to explore pickle error later => specific to tf? but why windows :p
* Figure out who works with sktime-dl (also send reminder to Tony about the error)

## Week 7 - 2022-07-15 + Week 8 - 2022-07-22

### work done

* Completed the CNN Regressor [#2894](https://github.com/alan-turing-institute/sktime/issues/2894)
* Completed the issue related to improvements / changes in existing CNN Classifier [#2935](https://github.com/alan-turing-institute/sktime/issues/2935)
* Issues with mac python 3.7 (added comment related to spordiac error)
* Draft PR for adding GPU functionality [#3072](https://github.com/alan-turing-institute/sktime/pull/3072)
* Need to discuss design plan for `BaseDeepClassifier.predict`
* Need to still work on multi-gpu and multi-cpu setting.
* Started working on save/load functionality [#3022](https://github.com/alan-turing-institute/sktime/issues/3022)
* Doubt related to overwriting dunder methods
* Will need to test on all estimators

### Agenda

* Discuss work done
* Questions from "work done"
* design plan for `BaseDeepClassifier.predict`
* overwriting dunder methods
* GSoC timeline

### Notes

* Discuss work done
* Questions from "work done"
* design plan for `BaseDeepClassifier.predict`
* currently automatically detecting
* FK: thinks option to choose device in init may be a good idea!
* default should be automatic detection
* Tan (earlier): need to check what happens if a device not present is selected, e.g., GPU on a computer that does not have one addressable by tensorflow
* overwriting dunder methods
* GSoC timeline
* asks for extension - 22 weeks at 350hours total
* mentors need to do that in gsoc web interface
* action: mentors to discuss and look into this

### Actions, short-term goals

* FK to review CNN regressor
* FK to review base deep classifier additions
* Work on template for DL models
* Complete design and work of implementation of save/load methods.

### Questions

## Week 9 - 2022-07-22

### work done

* FK to review CNN regressor
* Reviewed and merged
* FK to review base deep classifier additions
* issues related to class variables
* Made changes
* custom label encoder / onehot encoder
* Is there a need? Maybe replace it with automatic check or a variable
* VECM pr
* Based on fix provided by FK in [#3105](https://github.com/alan-turing-institute/sktime/pull/3105)
* Working on Issue [#3022](https://github.com/alan-turing-institute/sktime/issues/3022)
* Initial thought was to add the save load functionality in BaseEstimator but thats a sklearn class (realised it a bit late :p)
* Where to add it? BaseClassifier (and hence BaseRegressor and BaseForecaster) ?
* Should we have a BaseDeepEstimator? (deriving from BaseEstimator)
* For now do it in BaseDeepClassifier

<!-- CNN Classifier

BaseDeepClassifier BaseDeepClassifier

Base Classifier Base Deep Estimator

BaseEstimator Base Estimator -->

### Agenda

* Discuss work done

### Notes

### Actions, short-term goals

### Questions

* What should be next: BaseDeepForecastor/CNNForecasters or other Classification/Regression Modules?
* SS: Prefer getting done with classification first (simply because its a checklist done :p)

* GB: we'd like to get a feedback on mentoring: what goes well, what you'd like to change with style/what we're focusing on

* FK: change default meeting time?
* SS: UTC 15:30? FK: doesn't work

## Week 10 - 2022-08-05

### work done

* FK to review base deep classifier additions [#2991](https://github.com/alan-turing-institute/sktime/pull/2991)
* issues related to class variables
* Made changes
* Ready to review
* VECM pr
* Merged
* Adding CNTC network draft pr [#3731](https://github.com/alan-turing-institute/sktime/pull/3171):
* Added the network for contextual networks
* Added the code for transforming the data into required format
* Need to add main class and tests
* Completed save/load functionality PR [#3128](https://github.com/alan-turing-institute/sktime/pull/3128):
* Need review

### Agenda

* discuss work done
* next steps

### Notes

* classifier additions PR ready to review
* Custom Networks:
* User provides child of BaseDeepNetwork
* Need to be compatible with sktime api
* Needs to pass all tests

* testing
* FK: we should be testing `BaseDeepNetwork` children
* suggestion: add a test suite for those
* see how we currently test specific estimators
* see `TestAllClassifiers` and `TestAllForecasters` classes
* we should have, similarly, `TestAllDeepNetworks`
* for this, we need to add the base class `BaseDeepNetwork` to:
* `registry._base_classes`
* `check_estimator` lookup
* https://www.sktime.org/en/stable/developer_guide/testing_framework.html

### Actions, short-term goals

* continue work on
* save/load
* base deep networks
* CNTC
* migrating more classifiers
* open issue about testing `BaseDeepNetwork` descendants, list todos
* think about what needs to be tested
* list 3 examples of `BaseDeepNetwork` descendants, think about commonalities to be tested
* Open an issue about `BaseDeepNetwork` and `BaseClassifier` being parents of `BaseDeepClassifier`
* when migrating classifiers: make clear notes on which aprts are repeating a lot, "boilerplate"
* use that to think about how to best reduce developer boilerplate
* Deadline around 2 weeks.
* Ask on slack for a meeting related to class design (maybe Tuesday)

### Questions

<!-- BaseDeepNetwork -> # network
CNNNetwork

BaseClassifier -> BaseDeepClassifier -> CNNClassifier
BaseDeepRegressor -> CNNRegressor -->

## Week 11 - 2022-08-12

### work done

* FK to review base deep classifier additions [#2991](https://github.com/alan-turing-institute/sktime/pull/2991)
* Merged
* Adding CNTC network PR [#3731](https://github.com/alan-turing-institute/sktime/pull/3171):
* Adding MLP network PR [#3232](https://github.com/alan-turing-institute/sktime/pull/3232):
* Adding FCN network PR [#3233](https://github.com/alan-turing-institute/sktime/pull/3233):
* save/load functionality PR [#3128](https://github.com/alan-turing-institute/sktime/pull/3128):
* Discussions related to designs, need to finalize one.
* Starting LSTMFCN Classifier Issue [#3193](https://github.com/alan-turing-institute/sktime/issues/3193):
* Need to figure out where to put `AttentionLSTM`

### Agenda

* discuss work done
* Approach towards bug/test failure
* Design for `save/load`
* Design related to `LSTMFCNClassifier`
* Updates on design for `BaseDeepNetwork`:
* Lots of boilerplate code
* `build_model` function is same for all
* For testing related to it, wanted to wait for design finalization before proceeding.
* GSoC Review:
* Wanted feedback before mid-evaluations
* Current Progress:
* Merging sktime-dl with sktime: Plan to complete major work by September
* Adding DL forecasters: Plan to do after midterm evaluation
* Adding soft features: Doing side-by-side, adding save/load, GPU, enhancing base structures etc
* Adding some more DL classifiers: Depends on need, will need to ask suggestions of possible classifiers.

### Notes

new `test_all_tags` error in CI:

* network does not have specific tags
* as long as this is the case, add the string to `SCITYPES_WITHOUT_TAGS` in the same file as the test

Need to figure out where to put `AttentionLSTM`
* FK: what is this?
* SS: keras layer
* FK: similar to sklearn classifiers in sktime
* example is `CanonicalIntervalTree` which is an sklearn classifier, inside the `classification` module
* as a network layer, makes most sense in a submodule of `networks`, imo

Updates on design for `BaseDeepNetwork`
* FK: could you write a STEP?
* https://github.com/sktime/enhancement-proposals
* once you have seen a number classifiers
* explain repeated boilerplate and utility code
* try to summarize this in a design
* list high-level useful tests that are generic

Wanted feedback before mid-evaluations
* FK: plan looks very reasonable to me! GB: agree
* classifiers is good to wrap up at mid-term
* let´s finalize with a clean base class design
* other contributors can add more classifiers then
* forecasters is more challenging, but you are up to it!
* gives you opportunity to build sth new

### Actions, short-term goals

### Questions

## Week 12 - 2022-08-19

### work done

* Adding MLP network PR [#3232](https://github.com/alan-turing-institute/sktime/pull/3232):
* Solved the error, needs to be merged before others are
* Adding CNTC network PR [#3731](https://github.com/alan-turing-institute/sktime/pull/3171):
* Adding FCN network PR [#3233](https://github.com/alan-turing-institute/sktime/pull/3233):
* Adding LSTMFCN network PR [#3292](https://github.com/alan-turing-institute/sktime/pull/3292)
* Adding EncoderClassifier network issue [#3293](https://github.com/alan-turing-institute/sktime/issues/3293)
* Wrote the code locally, need to test before sending PR
* For some reason test is taking too much time
* Adding BaseDeepClassifier STEP Document [#26](https://github.com/sktime/enhancement-proposals/pull/26)
* Need to add examples and add some more details in the design

### Agenda

* discuss work done

### Notes

Franz recommendations on next steps:
* get MLP done and maybe one or two other networks - don't spend too much time on individual networks. Ensure that the interface is finalized
* SS: Ready for review
* open the issue with "how to migrate networks" as a good first issue for other contributors. This should include good instructions and a list of networks that are yet to migrate (conditional on the interface finalized)
* SS: Will do once some more classifiers are merged
* start designing the forecasting class
* don't drop the save/load work, please
* SS: Currently in STEP design phase, will make PR soon. Current implementation, although not ideal, is still ready to merge.
* don't drop the STEP document, please
* SS: Working on examples to finalise on a more concrete manner. Will be done in next couple of days.

### Actions, short-term goals

### Questions

## Week 13 - 2022-08-29

### work done

* get MLP done and maybe one or two other networks
* Done and merged [#3232](https://github.com/alan-turing-institute/sktime/pull/3232)
* Adding a different file in module for LSTMFCN Classifier [#3292](https://github.com/alan-turing-institute/sktime/pull/3292)
* Ready to merge FCN Classifier [#3233](https://github.com/alan-turing-institute/sktime/pull/3233)
* Adding a soft dependancy in CNTC Classifier [#3171](https://github.com/alan-turing-institute/sktime/pull/3171)
* open the issue with "how to migrate networks" as a good first issue for other contributors.
* Done, [#3351](https://github.com/alan-turing-institute/sktime/issues/3351)
* Need review on where readability/understanding of instructions can be improved
* Save/Load work:
* Updated design (and tested it works) at [#3128](https://github.com/alan-turing-institute/sktime/pull/3128)
* DL Design work:
* Added examples to existing proposed solution [#26](https://github.com/sktime/enhancement-proposals/pull/26)

### Agenda

* Discuss work done
* Wanted feedback on created issue for migration
* Wanted reviews on proposed design (for save/load and DL Design)
* GSoC Midevals

### Notes

FK notes res LSTMFCN classifier
* can you please be more specific in the PR what you are referring to
* what did I recommend specifically, where
* what are you proposing, in general, to do
* please make sure to add description

FK re CNTC classifier
* please make sure the tests pass
* look at dev guidelines on adding soft dependencies if unclear how

FK re migration:
* try to add an example on how migration works, use running example
* read it with the eyes of someone who does not know already
* avoid vague formulations, say "how" - e.g., how to identify whether file already exists

### Actions, short-term goals

* FK action: look at shivansh comments in [#3128](https://github.com/alan-turing-institute/sktime/pull/3128) and [#26](https://github.com/sktime/enhancement-proposals/pull/26)

* Shivansh to think about feedback to mentors
* learning goals and project goals - where are we?
* positive and critical feedback of mentoring
* any thoughts about the sktime community
* thoughts about next steps

### Questions

## Week 13 - 2022-09-05

### work done

* Changes in instruction for Migrating Estimators Issue [#3351](https://github.com/alan-turing-institute/sktime/issues/3351)
* Add good first issue label
* action on FK
* Made some friends go through it as well, it seems okay now
* If any other changes, let me know
* Need to add special cases (can only do that when they are resolved and merged)
* Soft Dependency: CNTC [#3171](https://github.com/alan-turing-institute/sktime/pull/3171)
* Extra Module: LSTMFCN [#3272](https://github.com/alan-turing-institute/sktime/pull/3292)
* Added save/load STEP design [#27](https://github.com/sktime/enhancement-proposals/pull/27):
* Issue with understanding:
* How does `load_from_path` work in [#3336](https://github.com/alan-turing-institute/sktime/pull/3336): can I simply send path there and do the loading of model in `load_from_path` (and then assign it to cls attribute?)

### Agenda

* Discuss work done
* FK: 3171, 3272, are these work in progress still?
* Discuss Feedback:
* My Feedback: https://hackmd.io/@JjP-uexCSkK27Af-_99qDg/rycZtApyi
* FK: I think we don't have access
* Discuss future work:
* Finalise step design to start working on design/code
* Start looking at DL Forecasters, and how to design them
* Another STEP document?
* Maybe some starters?
* FK: not what I expected from a design document. This is basically selected code files dumped into the PR, not a description of what/why. Look at how the other STEP documents look like.
* the code itself is not too helpful, since without the rest of the code base it also cannot be run, and there is no discussion of intention or context

### Notes

* How does `load_from_path` work in [#3336](https://github.com/alan-turing-institute/sktime/pull/3336): can I simply send path there and do the loading of model in `load_from_path` (and then assign it to cls attribute?)
* FK: idea is not putting attributes in the class
* the idea is locating a method that does file -> new object with the class
* note: what we have to do to get new object from file is different by *class*, but not by *object*
* so we save that as part of the file, and putting it with the class makes it easy to do "similar things" according to the inheritance pattern

FK general advice on writing design documents

* start with problem description and scope definition
* what does the design address?
* when would it be used?
* next: suggest user journey design or usage design
* how would a user or another class use your code?
* think about use cases and workflows
* in this case, example would be "save/load workflow", e.g.,

    # part 1: save the model
    file_loc = "my/dir/file"
    my_model.save(file_loc)

    # part 2: load the saved model
    from somewhere import load
    loaded_model = load(file_loc)

* optimally, now list requirements; explain design principles
* then, provide description of the code structure (not the entire code)
* providing code is fine, but make reference to it in the document
* look at other STEP documents - some are better or less good
* excellent examples: PEP documents (python extension proposals), typically much higher quality than `sktime` ones
* think about what made the issue 3351 clearer - which sequence to explain things for a reader?
* sequence of explaining things
* giving concrete examples
* being precise

DL forecasters
* understand `BaseForecaster` and extension templates
* existing DL forecasters are implemented in `kats`, `darts`, few other packages
* think about commonalities with DL classifiers
* suggest STEP doc

### Actions, short-term goals

* action FK: add good first label to 3351

### Questions

## Week 14 - 2022-09-12

### work done

* Re-wrote BaseDeepClass step design to make it more understandable:
* Messed up the commit history a bit, will just fix it

* Worked on creating base template for DL Estimators [#3028](https://github.com/alan-turing-institute/sktime/issues/3028):
* Added files in extension template, its ready just need to make PR (wanted to confirm once before it)

* Read implementations of `kats`:
* LSTM Forecaster:
* Written in pytorch
* Scales data before it goes through model
* Creates sequences based on time-window (seq-length)
* Fit is more or less the same, predict is a bit more complicated, but should be doable

* Implementation of `darts`:
* RNN Forecaster:
* Need to go in more detail
* written in pytorch
* Looked similar, had a linear layer after RNN seq

### Agenda

* Discuss work done
* holiday schedule

### Notes

* FK: please make sure we track actions properly
* carry over actions from last meetings
* review them in a "review of actions" agenda point at the start of the meeting

### Actions, short-term goals

* make draft PR with current DL extension template by Shivansh
* please look at last couple weeks and search for outstanding actions. Next meeting, we start with review of actions.
* Shivansh: check literature for reference on RNN forecaster
* Hyndman: forecasting - principles and practice
* paper search on google scholar
* let mentors know of dates/times of absence due to holidays

## Week 14 - 2022-09-19

### work done

* Remaining Actions:
* make draft PR with current DL extension template
* Done [#3433](https://github.com/alan-turing-institute/sktime/pull/3433)
* Unsure about the errors related to coverage?
* Update basedeepclass design
* Fixed the PR [#26](https://github.com/sktime/enhancement-proposals/pull/26)
* Update save/load design
* Sagar has done it, I was not able to find solution for adding keras models in zip file for some reason.
* Have code reviewed the PR and suggested tests as he asked.
* Lot more updates on [#27](https://github.com/sktime/enhancement-proposals/pull/27) and [#3425](https://github.com/alan-turing-institute/sktime/pull/3425) by Sagar and Franz
* check literature for reference on RNN forecaster:
* [Tensorflow tutorial](https://www.tensorflow.org/tutorials/structured_data/time_series)
* Linear Layer
* CNN Forecaster with window_size=3
* These are for single step forecasting (1 step in future), other examples exist in tutorial for multiple step in future.
* [Air pollution forecasting using RNN with LSTM](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8512020): Not exactly Time Series but kinda related
* [Deep Learning for Household Load Forecasting—A Novel Pooling Deep RNN](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7885096): More about dataset pooling than RNN, but implementation was fairly standard. Also uses stacked RNNs
* Can either use multi-predictions or single predictions, unsure about pros/cons (or if they are the same which I think is the case). Will read some literature on it.
* CNTC Classifier:
* unsure about how to remove doctests and test
* LSTMFCN Classifier:
* A bit weird case, imports are required but the file is not an estimator.

### Agenda

* Discuss work done

### Notes

### Actions, short-term goals

* CNTC Classifier:
* how to skip the doctest
* G: not specific to sktime, should be google-able
* LSTMFCN Classifier:
* unsure about the design
* check out the writing whole file into another function solution if it works.
* DL Template:
* fix the imports, that is throwing the error
* RNN Forecastors:
* Read about difference between single next prediction vs multi predictions.
* Check: PyData, M5

## Week 15 - 2022-09-26

### work done

* CNTC Classifier [#3171](https://github.com/alan-turing-institute/sktime/pull/3171):
* Completed
* LSTMFCN Classifier:
* check out the writing whole file into another function solution if it works.
* Remaining
* DL EstimatorTemplate [#3433](https://github.com/alan-turing-institute/sktime/pull/3433):
* Should be mergeable
* Based on current existing design, will require changes based on [#26](https://github.com/sktime/enhancement-proposals/pull/26)
* RNN Forecastors:
* [LSTM in M5](https://www.kaggle.com/code/minhajulhoque/deep-learning-rnn-for-m5-forecasting/notebook):
* Not directly LSTM, but it was more of an encoder-decoder architecture
* Currently working on creating an example notebook for forecasting using sktime to see whats existing already [here](https://colab.research.google.com/drive/1eT16Q1hY3Y9c6rcr56qF5-UHy4u6cX-K#scrollTo=0cfwIiPLjAc6)
* Once done, will start making a design STEP document
* FK: is there substantial new design?
* If its required, if not can start with coding
* Update basedeepclass design
* Fixed the PR [#26](https://github.com/sktime/enhancement-proposals/pull/26)
* Need Review, based on design updating existing code should be easy

### Agenda

* Discuss work done

### Notes

### Actions, short-term goals

action Shivansh:
* announce in 3 slack channels that will be meeting & would appreciate input on the DL design
* contributors, sktime-dl, mentees

## Week 16 - 2022-10-03

** Did not have a meeting due to holidays**

## Week 17 - 2022-10-10

### work done

* Went through save/load discussion and CNNRegressor parameters discussion
* Mostly catching up to discussions and work done

### Agenda

* work done
* Discuss actions

### Notes

* Discuss actions
* FK: recommend you focus on forecaster design - this was the ambition in your project?
* migration of estimators is "rote work", do that as filler but not as primary goal
* please bring up if you are blocked, e.g., PR base class - FK will look into it

### Actions, short-term goals

* Complete open PRs/Issues related to sktime-dl migration:
* LSTMFCN
* Encoder
* MCDNN
* MACNN
* Add extra parameters to CNNRegressor: taken up by Sagar:
* based on [#2935](https://github.com/sktime/sktime/issues/2935)
* Convert DL Forecaster notebook into BaseClass and EstimatorClass:
* [#3501](https://github.com/sktime/sktime/pull/3501)

* basedeepclass design
* PR [#26](https://github.com/sktime/enhancement-proposals/pull/26)
* action FK: give feedback

## Week 18 - 2022-10-17

### work done

* Fixed LSTMFCN Classifier [#3292](https://github.com/sktime/sktime/pull/3292)
* Used a workaround to encase the class within a function, had to be done since the defined class was inheriting from keras class (which could not be resolved wrt soft dependancy issue)
* Should be merge-able if all tests pass
* Worked on FCNForecaster [#3501](https://github.com/sktime/sktime/pull/3501)
* Completed FCNForecaster till fit for univariate data (should work for multivariate as well)
* Need to check compatibility with different data formats, if I need to make it more general (currently works with np array)
* Add a function to split it into windows (would go in BaseDeepForecaster)
* Prediction is easy, need to add iterative loop for it
* Also need to check out ForecastHorizon to make it similar to other forecastors (and how neg values of FH should be handled?)
* Will make DeepBaseForecaster once the FCNForecaster is completely done, need to check which functions are redundant once I have a completely working estimator.
* basedeepclass design
* PR [#26](https://github.com/sktime/enhancement-proposals/pull/26)

### Agenda

* work done

### Notes

* Forecaster
* Forecasting Horizon:
* Guzal: its always positive, negative is mostly relative, which is still prediction in future
* There might be multivariate output
* Guzal: generally you take each dimension seperately in such cases
* SS: can predict multivariate output at once with DL Forecasters

### Actions, short-term goals

* BaseDeepClass Design
* Guzal will have a look at it
* Main work for next week:
* Complete predictions in FCNForecast
* Make BaseDeepForecast Design

## Week 19 - 2022-10-24

### work done

* Fixed LSTMFCN Classifier [#3292](https://github.com/sktime/sktime/pull/3292)
* Needs review, all tests pass
* Worked on FCNForecaster [#3501](https://github.com/sktime/sktime/pull/3501)
* BaseDeepForecaster design ongoing
* Need to still integrate predict with ForecastingHorizon
* A lot like regressor design?
* basedeepclass design
* PR [#26](https://github.com/sktime/enhancement-proposals/pull/26)
* Going with Solution 2 (creating BaseDeepEstimator) based on suggestion

### Agenda

* review of work done
* question about `X` in forecasting

### Notes

"work done"
* FK: for anything "done", please ensure there is a PR, or it is not done
* mark work that is still on as "ongoing"

* question about `X` in forecasting
* Shivansh: should there not always be an `X`?
* FK: no, that is not always needed. Look at the tutorial for an explanation of forecasting without an `X`
* read the `sktime` forecasting tutorial

### Actions, short-term goals

* DLForecaster and FCNForecaster
* Read forecastng tutrial to understand forecasting without X
* Complete predict functions based on that
* BaseDeepClass design
* Implement Solution 2 as mentioned in STEP design [#26](https://github.com/sktime/enhancement-proposals/pull/26)
* Add, for both solutions, user journey for adding a new DL estimator class

## Week 20 - 2022-10-31

### work done

* Worked on FCNForecaster [#3501](https://github.com/sktime/sktime/pull/3501)
* Need to still integrate predict with ForecastingHorizon
* Figured out the issue of no X and how to work wtih it
* Difference in what X is considered in DL and in Forecasting
* basedeepclass design [#26](https://github.com/sktime/enhancement-proposals/pull/26)
* Going with Solution 2 (creating BaseDeepEstimator) based on suggestion
* Updated the docs for user journey for adding a new DL estimator class

### Agenda

* review of work done
* GSoC Objectives Check

### Notes

* Keep frequently updating FCNForecaster PR:
* Guzal: if you get stuck ping mentors
* GSoC Objectives:
* SS: Ideal goal should be to finish First DL Forecaster (and BaseDeepForecaster design) by the end
* Once the BaseDeepForecaster is done it would be easier to expand
* Add a bare minimum test suite atleast
* Guzal: Specific test suite for DL Models
* Atleast basic sanity checks
* Some Example:
* Expected output check
* Expected error check

### Actions, short-term goals

* Leo: Review on BaseDeepClass Design
* SS: Can update on work once get approval for user journey
* BaseDeepClass:
* Create PR once Franz and Leo have reviewed and given go ahead
* Test Suite:
* Go through existing test files to just understand how it is supposed to be
* First step would be to start with specific Estimator
* Next would be to go for `test_all_estimators`
* Forecasters:
* Ask for more help if getting stuck/cant understand
* Work on completing the FCNForecaster PR

## Week 21 - 2022-11-07

### work done

* Worked on FCNForecaster [#3501](https://github.com/sktime/sktime/pull/3501)
* Pushed a working implementation of `fit` and `predict`
* Tried some sample datasets:
* The loss is getting stuck after a point
* Might be a network specific issue (or hyperparameter) or might be implementational issue. Most likely implementational
* Currently debugging, have asked for help on thread, will ask on community as well

* basedeepclass design [#26](https://github.com/sktime/enhancement-proposals/pull/26)
* Going with Solution 2 (creating BaseDeepEstimator) based on suggestion

### Agenda

* review of work done
* Decide on a new time

### Notes

* Guzal and Leo: loss stagnation might be specific to network
* Maybe try on a different dataset, with more datapoints
* Leo: is it from a paper? try to find simpler neural network from papers so that we can know definitely if we are going right or wrong.
* airline dataset -> try to get reasonable predictions with a simple network.

* Deciding on a time:
* Leo: generally not available in mornings
* SS: prefer morning
* Going with 0800 UTC

* Guzal: Next meet should be wrap up type for GSoCqd

### Actions, short-term goals

* Try simpler architecture forecaster on larger dataset (airline for eg)

* Go ahead with BaseDeepClass design now

## Week 22 - 2022-11-14

### work done

* FCNForecaster [#3501](https://github.com/sktime/sktime/pull/3501)
* Fixed the training error:
* Tried different datasets, no significant difference from that
* Ended up going with a simpler model (MLP instead of FCN) and was able to debug the network
* Solution works now, still not able to find "paper" benchmarks so just went with tutorial ones.
* Pushed a working implementation with `fit`, `predict` and exogeneous data support.
* Design error:
* Forecasters have a design which require `y` as the main data, and `X` as exog data as input
* DL Models have a design which requires `X` as the training data, and `y` as the target.
* There is a conversion required from forecaster input to DL input within `_fit` which I am currently doing
* Error arrises when I integrate `MLPForecaster` with `BaseDeepForecaster`, since then it expects input in DL format
* As a solution, can either ask the user to pass data in another function `convert_dtype` wh1ich returns `source` and `target` as required by DL estimators, and then pass that to `fit`
* Since the conversion is already done in `fit` in current implementation, its just a matter of putting the code in another function
* So currently have 2 implementations up:
* First, without BaseDeepClass, which works end to end
* Need to just add support for diff input dtypes (numpy and pd)
* Second, with BaseDeepClass, which has a design question to be solved

* BaseDeepEstimator [#3755](https://github.com/sktime/sktime/pull/3755)
* Pushed the code in PR with new `BaseDeepEstimator`
* Implemented Solution 2, no changes required in any estimator, just had to remove code from BaseDeepClassifier, BaseDeepRegressor and add that to BaseDeepEstimator
* Reduces redundancy by a lot, especially for the save/load code
* some issues with pull, will sort it out by tonight/tom.

### Agenda

* review of work done
* review GSoC period

### Notes

* Guzal: For forecaster, check `check_x_y` to see the checks for forecaster.
* Guzal: for pull just check if you are updating across all branches you were working on, might be a silly error.
* Guzal: ping on slack if PR review is required for completion atleast for deadline week.

### Actions, short-term goals

* Mostly work on trying to complete PRs as much as possible by gsoc deadline
* Finish requirements of GSoC and dont miss the deadline!
* Fix a time/date for final feedback session.

## Week 22 - 2022-11-29

feedback session

For Shivansh

* project goals
* learning goals
* interaction with mentors
* points for improvement
* general comments
* interaction with community