- ML Fundamentals Journal Club·Last edited by Patrik Reizinger on Feb 15, 2021Linked with GitHubContributed by
There is no commentSelect some text and then click Comment, or simply add a comment to this page from below to start a discussion.Deep Residual Learning for Image Recognition
Reading log/further reading
Papers:
- Exploring Randomly Wired Neural Networks for Image Recognition
- Densely Connected Convolutional Networks
- Residual Networks are Exponential Ensembles of Relatively Shallow Networks
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- BlockDrop: Dynamic Inference Paths in Residual Networks
Blogposts:
VGG parameter and memory usage
Learn More →
Naive Inception
Learn More →
Inception with bottleneck
Learn More →
Read more
Reading Group
Rota link
Aug 10, 2023Új témák MLJC-re
Contrastive Learning Contrastive Learning Inverts the Data Generating Process - Roland/Yash/Wieland CPC Contrastive Representation Learning blogpost Meta-learning (Feri) MAML and Reptile [1, 2]
Nov 2, 2021Importrance of Masking in Generative Modeling of Sequences
When learning to generate sequences of symbols $x_1, x_2, \ldots, x_T$, we often do that by defining a pobabilistic generative model, a probability distribution over sequences $p(x_1, \ldots, x_T)$, by making use of the chain rule of probabilities: $$ p(x_1, \ldots, x_T) = p(x_1) p(x_2\vert x_1) p(x_3\vert x_1, x_3) \cdots p(x_T\vert x_1, \ldots, x_{T-1}) $$ This makes computational sense because modeling each of the conditional distributions above is easier as it is a distribution over a single symbol $x_t$, so even though the entire sequence $x_1, \ldots, x_T$ can take combinatorially many values, each component distribution we model $p(x_t\vert x_1, \ldots, x_{t-1})$ is only a distribution over a relatively small number of options, which can be be modelled easily. A generative model that defines a probabilistic model as a product of conditional distributions like above is often called autoregressive.
Jun 7, 2021Periodic Markov-chain example
In this short note (and a corresponding colab notebook) about the example of a Markov-chain whose state distribution does not converge to a stationary distribution. These are probably irrelevant technicalities for reinforcement learning, but an interesting topic to understand nevertheless. Consider an integer $k$ and the homogeneous Markov-chain $S_{t}$ such that the transition probabilities are: $$ \mathbb{P}(S_{t+1} = n+1 \mod 2k \vert S_t = n) = \mathbb{P}(S_{t+1} = n-1 \mod 2k \vert S_t = n) = \frac{1}{2} $$
May 20, 2021
HackMD