HackMD
Deep Learning Summit London 2019 - Day 1
tags: RE.WORK Lectures Deep Learning
Day 1
- Date: Sep 19, 2019
- Link to Schedule
Tricks for Deep Learning -BP
Huma Lodhi, Data Scientist
- In industrial Deep Learning, you need to combine numerical features with boolean and categorical ones. Using embeddings for the categorical ones, and simply concatenating the outputs of the lowest layers, leads to successful architectures
DEEP DIVE: Deep Reinforcement Learning - DEEPMIND
Hado Van Hasselt, Senior Staff Research Scientist
(apparently not everyone knows that is an assumption in RL, not a theorem)
-
Traditionally, the term "control" has been used by the control theory community to describe a control signal (external forcing) given to a system in order to mantain a stable trajectory in phase space. In RL it is used more loosely in the sense of optimisation of agent behavior
-
when we talk about state, in RL, we usually mean the state of the agent, which is different from the state of the world (environment). The last one may or may not be fully observable
-
Deep RL is RL where the value function, the policy function and the model for the evolution of the state (if used) are each parametrized by deep neural networks.
-
two well-known DRL algorithms (or families thereof) are DQN and actor-critic
-
if the state is fully observable, the transitions are deterministic and we can compute them exactly (e.g., checkers, chess Go) we can do some pretty amazing things with Deep RL
-
if we don't have a perfect model for the state transitions, (e.g., poker, autonomous driving) we could use model-based reinforcement learning:
-
However, deep model-based RL has a pretty major drawback: algorithms are extremely good at exploiting the inaccuriacies of the state transition model in order to maximize the expected value. Replay is used to mitigate the effect, but it's not easy to understand the tradeoff between model and replay
-
Recently, a seminal paper came out, which introduced a new model-based RL algorithm, which is the first to show competitive performance on ALE (the Atari Learning Environment) with respect to model-free approaches, and of course a much higher sample efficiency.
-
Hado will present a new paper on similar topics at NeurIPS later this year
Transfer Learning in NLP: Concepts, Tools, and Application to Language Generation Tasks - HUGGING FACE
Thomas Wolf, Chief Science Officer
-
Transfer learning radically transformed the Deep Learning NLP landscape in the last 18 months. With sequential transfer learning, you train a huge model for a certain task on a huge (unlabeled) dataset, and you transfer to another task/dataset, where the second (labeled) dataset is much smaller than the first one
-
The best models are all based on the Transformer architecture
-
Usually, the pretraining task is language modeling,i.e., predicting the probability of a word/sentence given prior text. Empirically, this seems to be the task which leads to better transfer learning
-
Good news for practical applications! The fine-tuning phase (after pretraining) seems to be both fast and robust for most downstream tasks. I.e., few iterations are necessary to reach high accuracy on the downstream task, and the accuracy doesn't seem to be highly sensitive to training hyperparameters. The case shown in the slide is a text classification task
-
The pretraining phase requires huge computational resources, thus is beyond the capabilities of most companies (a DGX-1 wouldn't be nearly enough). Thus the current practice is to use pre-trained models, distributed either as libraries, or as model checkpoints (harder to use)
-
PyTorch-Transformers is Hugging Face library for NLP. It features state-of-the-art pretrained models, considerably simplifying the sequential transfer learning worfklow
-
These models are all open-vocabulary models (check how the unknown word "puppeteer" is handled), so they can handle new words easily
-
a model can be built and serialized with a few lines of PyTorch code
-
even if we don't pay the largest computational cost (training huge models on huge datasets), we still have to perform inference with a huge model. This still has a considerable, although much smaller, computational cost. PyTorch-Transformer works around that by featuring DistillBERT, a model distilled from BERT which is smaller and faster
-
NLP models have reached a size where we start to see diminishing returns
-
Hugging Face already collected 5M$ and it's going for another founding round. They are quite obviousy very competent, their GitHub repository has 13k stars and 3k forks, thus they could be a good startup to bet on.
Driving Cars on UK Roads with Deep Reinforcement Learning - WAYVE
Alex Kendall, CTO & Co-Founder
- Wayve is a small UK startup, which made quite an impression in the Autonomous Driving world with this video:
- Their young CTO proposes an End-to-End Deep Learning approach to Autonomous Driving, i.e., an approach in which instead of combining different independently developed subsystems in a bigger autonomous driving system, all the modules are trained toghether. This is a bit different from the usual interpretation of E2E, i.e, a single huge differentiable model which is trained with some learning algorithm.
- Alex says that this is possible today, because of huge progress in CV in the last 4 years
- Imitation learning is used to gather training data from expert drivers, and DDPG (Deep Deterministic Policy Gradients), a Deep RL algorithm, is used to learn from each safety driver intervention.. A single monocular camera and a consumer-grade GPS are all the sensors used:
- The whole approach is extremely data-efficient, using only 20 hours of driving to learn to drive in a completely new street to the level shown in the video.
- Since the reward is very sparse, even DRL is not enough, and simulation is important. To perform domain transfer from simulation to reality, generative models are key
- Alex concludes by listing some of current difficulties with this approach, noting that interpretability will be necessary to satisfy regulatory concerns, but causal inference is not (yet) possible
- overall, the impression is that the results are amazing for a model which doesn't even use maps and works in different atmospheric conditions, but still very preliminary (ony a few kilometers driven, low traffic)
- It will be interesting to see where they arrive in a year or two. Blog post
Machine Learning for Autonomous Driving: Recent Advances and Future Challenges - SCALE AI
Li Erran Li, Scale AI/Columbia University, Head of ML/Adjunct Professor
- This lecture was not very easy to follow, because of the choice of the author started from the most basic concepts
- Autonomous driving is a hot topic, with transportation network companies such as Lyft buying AI startups, especially in the Computer Vision field
- 3D vision is a more complex problem than 2D vision. An interesting, relatively new approach is PointNet:
- In these last 5 years we understood the importance of separating shape and appearance (texture) when learning useful representations for CV
- Human behavior is the main challenge, followed by long tail events (very rare events that happen once every 10000 or more drives, such as for example a truck dropping its payload). Li predicts that we will have fully autonomous cars (L5) in 2025
That's it for Day 1! Click here to go to Day 2. 👋
