Notes from Stanford CS230: Deep Learning | Autumn 2018 | Lecture 8 - Career Advice / Reading Research Papers
===

###### tags: `Capstone`

##### Link: https://www.youtube.com/watch?v=733m6qBH-jI

### Reading research papers
- Compile list of papers (online blogs/medium/github incl.)
- Process might feel like skipping around papers, reading and understanding about 10% of each, then narrowing down on one seminal paper to properly dive into
- From there, you might find a citation for another useful paper to look into, then with the new knowledge, perhaps revisit some of the other papers that were not thoroughly explored
- 5 to 20 papers should give you a basic understanding of an area, do some work, apply some algorithms etc
- 50 to 100 papers and roughly understanding a lot of it will give you a very good understanding of an area. These are just guidelines for the volume of reading if you want to pick up a new area.

### How do you read one paper?
- Bad way to read a paper is to start from the first to the last word
- I [Prof] actually always have a folder of unread papers with me as my personal reading list all the time. On my team I personally lead a reading group discussing about two papers a week (though to select two I will have to read 5 or 6 papers a week)
- Take multiple passes through the paper. Go for the higher efficient content first and the harder material later. Start with title/abstract/figures, maybe one or two key experiment sections, to get a good sense of what the whole paper is about
- A lot of times in deep learning the entire paper can be summarised in one or two figures in the figure caption
- In the second pass you should read carefully into the intro, conclusions, figures, and skim the rest
- Part of publishing papers is to convince reviewers that your paper is worthy of acceptance so you will find often that the abstract, intro and conclusion is often when the authors try to summarise their work really carefully to make a case
- You can skimp the section on related work as usually unless you’re already familiar with the literature, most of the stuff here will sometimes be almost impossible to understand. Sometimes authors try to cite almost everyone that might be reviewing the paper to make them feel good and hopefully accept the paper.
- On the third pass you should read the paper but skip the math and on the fourth, read the whole thing and skip parts that do not make sense
- Sometimes papers can have parts that are phenomenal and influential but also contain a lot of unimportant stuff as you can’t really predict what will be important in the future so it’s fine if you read a paper and some of it doesn’t make sense or will only be realised five years into the future

### As you read the papers, try to answer these questions:
- What did the authors try to accomplish?
- What were the key elements of the approach?
- What can you use yourself?
- What other references do you want to follow?
- Reading more papers will get you better as there will be similar ways across authors how tables or network architectures are presented. I [Prof] maybe need about half hour to understand a paper or 20 mins if it’s a really easy one and newer readers might take an hour
- Sometimes you come across a really difficult paper that can take you a long time to chase down all the references and learn a lot of other stuff and might take you 3 or 4 hours
- Depending on how much time you want to spend reading papers, you can actually learn a lot

### Source of papers
- State of the art is evolving very rapidly. Twitter is an important place to find out about new things. @kiankatan @AndrewYNg
- There’s an ML subreddit that’s actually pretty good. A lot of noise but important stuff gets mentioned there
- Some important conferences are NIPS/ICML/ICLR
- Community/friends
- Some people use arxiv sanity
- If you really want to get into the math, rederive it from scratch. Same with code. Run the open source code if available and to go further, reimplement from scratch
- Longer term advice: steady reading, as opposed to short bursts