# MLC Seminar June 22, 2021: "Statistical downscaling of precipitation using skip connections based models" by Dánnell Quesada

This hackmd document will serve as the show notes for Dánnell Quesada's talk. Please use it for questions or important notes that add to the content of the speaker. After the talk, these notes will be stored and made publicly on the seminar website https://mlcdresden.gitlab.io/pages/

For the entire seminar, please be mindful of your peers and supportive in your communication. We henceforth make following the Dresden code of conduct obligatory for everyone connected:
https://dresden-code-of-conduct.org/de/
In light of this, please mute yourself if you are not talking!

## Community Mixer

We start the event with a community mixer. We will split the group in groups of three. In each group, please take turns where each participant has 3 minutes to talk and answer the following questions:

1. How did you learn Machine Learning?
2. What was the first project that you used Machine Learning for?
3. What is currently your biggest challenge with Machine Learning?

Take notes below which common patterns or unusual findings you see.

### Notes for the Community Mixer

#### Notes room 1
(Isabel, Andre, Falk)

Isabel (computer science):
- lecture at university; theoretical, RFs, SVMs
- first project: course at university: segment images with RFs; worked so so
- biggest challanges: collecing meaningful data for model training and tuning of hyper parameters

Andre (bioinformatics):
- University classes: statistical models in bioinformatics
- Master's project: markov models for biological sequences
- Trying to be more in contact with machine learning, using ML in my work (bioinformatics)

Falk (biologist)
- during PhD developed towards bioinformatics in NGS data analysis
- later worked more on data from pathology (microscopic image data) and learned deep learning with advisors; segmentation, object localization
- lately much work in research managing issues and lost contact/practice in deep learning

#### Notes of room 3
- Nico, Lena, Noreen
Nico:
- informatics master, first 'classic style' then also other ML
- 1st project: mnist classification (linear regression, octave)
- biggest challenge: integrate physical knowledge (e.g. partial differential equations) - "physics informed NNets"
Noreen:
- image analysis, customer requests
- 1st project: random forest applied to image segmentation (cell images)
- biggest challenge: deployment, low-key/user friendly usage of ML
Lena:
- bioinformatics
- 1st project: Hidden Markov Model for analysis of protein/DNA sequences
- biggest challenge: combine data sources

#### Notes for room 4

(dannel, helene, peter)

- how did you get involved in ML?
- master thesis: feed forward NNs -> self-taught
- PhD: use MLP on patient data -> self-taught, online courses (Stanford lecture videos)
- Phd: first witness of SVMs/RFs; much later: reproduce nature paper about adversarial robustness of ML in medicine (self-taught)
- first project
- climate forecasting
- predict time series of leukemia patients
- training instance segmentation (Unet, Stardist, etc)
- current challenges
- reproducibility of results (same seeds :heavy_check_mark:, GPUs ...)
- mediate excitement for ML to youngsters and teenagers
- understand why the heck the model is not learning ;-)

#### room 2
Klemens:
- NN in IDL
- Statistical Downscaling for clima model outputs
- Understand the models. Is existing a priori knowledge relfected in the models.

Christoph:
- University classes, self-studies: Statistics
- unknown
- can we know how well we can estimate/learn: connecting ML and statistics

## Presentation by Dannel

### Notes, Comments & Questions

- slide 8.1: is there a semantic connection between the input channels, T-Q-U-... ? (the conv layers appear to do convolutions inside the frame only, maybe considering 3x3x20 convolutions for each feature map)
- yes, there is
- slide 8.1: if I understand correctly, the output is a single channel "image"?
- 3 output channels

- slide 8.1: what was the loss function used? (bernoulli gamma?)
- by default a Unet is trained with an MSE loss
- slide 9: ubuntu based containers tend to be rather big, was this a problem in practice?

- metrics: which value of the quality metrics have relevance for the task at hand?
- interested in extremes
- does it makes sense to seperate the task of learning the normal conditions/behavior versus the extreme conditions?

- dataset: data taking changed over time?
- if model is able to reproduce changes in climate wrt to baseline, then hopefully this holds for all time ranges
- suggest: play with the amount of training data (e.g. omit last 5 years)
- tricky: need to attend to bias correction for reference predictions

- how to select the best model?
- https://sebastianraschka.com/blog/2016/model-evaluation-selection-part2.html
- tricky question, not sure anyone used these methods for regression/segmentation

- how to optimize for batch size?
- look into omniopt (module available on taurus) -> https://www.scads.de/en/services/omniopt
- other hyperparameter optimizations tools come to mind: [hyperopt](https://github.com/hyperopt/hyperopt), [optuna](https://optuna.org/)
- heuristic based approach from the fastai community: https://towardsdatascience.com/implementing-a-batch-size-finder-in-fastai-how-to-get-a-4x-speedup-with-better-generalization-813d686f6bdf

- feature importance
- https://rss.onlinelibrary.wiley.com/doi/10.1111/j.1467-9868.2010.00740.x
-

## Wrap-up Community Mixer

We will take turns and ask each group to present their findings. Each group gets 2 minutes to talk.