Try   HackMD

Új témák MLJC-re

Contrastive Learning

  • Contrastive Learning Inverts the Data Generating Process - Roland/Yash/Wieland
  • CPC
  • Contrastive Representation Learning blogpost

Meta-learning

(Feri)

  • MAML and Reptile [1, 2]
  • implicit function theorem and iMAML [1, 2]
  • Natural Neural Networks and WarpGrad [1, 2]
  • invite: Chelsea Finn

Natural Gradient Descent

(Feri)

  • NGD
  • K-FAC and Exact NGD in Linear Networks
  • Wasserstein natural gradients
  • invite: Michael Arbel

Differential Privacy

  • Algorithmic foundations of Differential Privacy [1: approx. Chapters 1-3]
  • DP-SGD [1]
  • ?
  • invite: ??/Andrew Trask

Geometric Deep Learning

  • Group Equivariant CNNs
  • Spherical CNNs
  • Gauge Invariant CNNs
  • invited speaker: Kondor Risi, Taco Cohen

Inference in Graphical Models

  • Belief Propagation
  • Viterbi, forward-backward
  • ADF and EM
  • Variational Message Passing
  • invite: Tom Minka

Continual Learning

  • EWC
  • Learning without Forgetting
  • Variational Continual Learning
  • invite: Raia Hadsell

PCA Extended

(Patrik)

  • Contrastive PCA paper
  • Probabilistic Contrastive PCA paper
  • Variational Autoencoders Pursue PCA Directions (by Accident) paper

Reinforcement Learning

  • Neu Gergely

Causal Inference

Few-shot/small data

Domain adaptation

VAEs & Co

  • VAEBM
  • PCA-VAE
  • iVAE

EBMs

  • MCMC
  • Score Matching
  • VAEBM
  • How to train your EBM

Network & Data visualization

Last changed by 
 
ML Fundamentals Journal Club
reading group covering the foundations of machine learning
0
205

Read more

Reading Group

Rota link

Aug 10, 2023
Importrance of Masking in Generative Modeling of Sequences

When learning to generate sequences of symbols $x_1, x_2, \ldots, x_T$, we often do that by defining a pobabilistic generative model, a probability distribution over sequences $p(x_1, \ldots, x_T)$, by making use of the chain rule of probabilities: $$ p(x_1, \ldots, x_T) = p(x_1) p(x_2\vert x_1) p(x_3\vert x_1, x_3) \cdots p(x_T\vert x_1, \ldots, x_{T-1}) $$ This makes computational sense because modeling each of the conditional distributions above is easier as it is a distribution over a single symbol $x_t$, so even though the entire sequence $x_1, \ldots, x_T$ can take combinatorially many values, each component distribution we model $p(x_t\vert x_1, \ldots, x_{t-1})$ is only a distribution over a relatively small number of options, which can be be modelled easily. A generative model that defines a probabilistic model as a product of conditional distributions like above is often called autoregressive.

Jun 7, 2021
Periodic Markov-chain example

In this short note (and a corresponding colab notebook) about the example of a Markov-chain whose state distribution does not converge to a stationary distribution. These are probably irrelevant technicalities for reinforcement learning, but an interesting topic to understand nevertheless. Consider an integer $k$ and the homogeneous Markov-chain $S_{t}$ such that the transition probabilities are: $$ \mathbb{P}(S_{t+1} = n+1 \mod 2k \vert S_t = n) = \mathbb{P}(S_{t+1} = n-1 \mod 2k \vert S_t = n) = \frac{1}{2} $$

May 20, 2021
Independent Componet Analysis

Independent Component Analysis We are familiar with the problem that our high-dimensional data sources (images, video, fMRI, etc.) are redundant. And actually, we don't care about the value of pixels/voxels, we need some compact representation. Do you know state-space models? Well, something like that would be great. Of course, we have dimensionality-reduction methods, like Principal Component Analysis (PCA) or Factor Analysis (FA). Why do we need another one then? Someone just tried to get a PhD by tweaking around a bit to get something published? Fortunately, Independent Component Analysis is much more than that. Notation $s$ - signal sources/latents $x$ - signal mixtures/observations $y$ - reconstructed signal sources $g$ - prescribed CDF function

May 19, 2021
Read more from ML Fundamentals Journal Club
Published on HackMD