# Information for Prospective PhD Candidates Mark van der Wilk, November 2023 **[Update September 2023: I have moved institutions, and am now at the CS dept in Oxford. The PhD degree in Oxford is called a "DPhil", and the terms are used interchangeably below.]** ## Our Research See our [website](https://mvdw.uk) and [research overview](https://mvdw.uk/research-overview/) for all details. Three research questions are central to our research group: - How do we find general patterns that allow generalization beyond the training set? I.e. how do we find inductive bias, rather than building it in? - How can we create neurons that automatically assemble their structure, with as little global communication as possible? - How do we use predictions to build decisions-making systems that are safe and reliable? We explore solutions that develop deep learning in a different direction than the "scale up" approach that is common today. This builds towards a different vision of what deep learning could look like. One where training a model needs 1) less data, 2) less human intervention, and 3) less energy. While we explore many paths to these goals, we take significant inspiration from Bayesian statistics, and particularly from Bayesian model selection (underappreciated in ML). So far, this has led to methods for selecting inductive bias (e.g. invariance), as well as identifying causal direction. Bayesian uncertainty estimation may also help to inspire new learning rules, which do not require communication across a whole network like backpropagation does. Overall, many topics contribute towards better answers to these three questions, including: - Bayesian model selection & Neural architecture search - Causality & relations to disentanglement - Out of distribution generalisation - Geometric deep learning and invariances - Meta-learning - Continual learning - Local learning rules - Approximate Bayesian inference & uncertainty quantification - Gaussian process models, Deep GPs, and relations to Neural Nets - Bayesian optimisation & experiment design - Generative modelling, semi-supervised learning, self-supervised learning - Capsule networks - Model-based reinforcement learning - Analysis of generalisation and learning in deep networks, generalisation bounds, connections to Bayesian inference. ## The PhD and our Research Group It's great that you're interested in joining our research group. A PhD is a journey where you develop your understanding to the boundary of human knowledge, and beyond. This is a significant undertaking that usually takes 3-4 years. This makes the research groups you apply to an important choice. I would say that the most important criteria in choosing a group are 1) how well _your_ research interests fit with those of the group and the prospective supervisor, and 2) whether the group feels like a supportive and pleasant place to work. The people in the research group you join will be the people who you brainstorm with, who you whiteboard maths with, who you will code up projects with, and who will give you constructive feedback on your work. So it's important to have an environment that you're happy with! In our group, we aim to understand existing machine learning methods, and develop new methods that work *reliably* and *without human supervision*. Our research is motivated by a wide variety of problems, from improving the reliability of large-scale neural networks, to more statistical problems where data is scarce. We also work on more applied problems, most often through collaborations with domain-experts at Oxford, Imperial College London, and beyond. We started in 2020, and we currently consist of myself, and 8 PhD candidates. There is a nice mix of expertise and perspectives (e.g. from Bayesian statistics to deep learning engineering), and motivating applications (biology to robotics). The aim is to have a group with a diverse collection of expertises that overlap roughly around taking inspiration from probabilistic inference. To widen the expertise that we can learn from, we also collaborate with researchers at Oxford, Imperial College London, in the UK, and internationally. To get an impression of who we are and what we do, I recommend that you look at: - [the overview of our research](https://mvdw.uk/research-overview/) [1], - [members of the research group](https://mvdw.uk/people/) [2], - and perhaps a few of our [publications](https://mvdw.uk/publication/) [3]. If you like the look of our research directions, the next step is to develop your research proposal (tips below), and to send in your application to Oxford. During the interview process, we will have the opportunity to chat informally as well, and you can meet some of the current group members. <!-- ## Admission Requirements The two most important things when applying to a PhD are: - Your research proposal, and how well it fits to the research of the group you are applying to. - Evidence of a strong academic background, with particularly strong mathematical skills. A strong mathematical background is usually demonstrated by a first-class (or equivalent) degree in information or electrical engineering, physics, maths, or computer science. A background in e.g. linear algebra, probability, statistics, and optimisation are particularly important. You can demonstrate alignment with my research interests in your research statement that outlines **1)** what problem you are interested in, **2)** why this problem is interesting or important, and **3)** what techniques you think will be useful or necessary for reaching your goals. --> ## Application Process ### In short In principle, all you need to do is follow the [Department of Computer Science application process](https://www.ox.ac.uk/admissions/graduate/courses/dphil-computer-science) [4]. The main application deadline is **mid-day December 1**. Submitting for this date is necessary for your application to be automatically considered for all funding opportunities. You may still submit an application after the deadline, as there may still be funding spots available, but meeting the deadline significantly increases your chances. You may **also** apply to CDTs [7], which have separate funding pots, deadlines and requirements for the applications. See the respective CDT websites [7] for more information. ### Getting in Touch If you have any specific issues with the application, then do get in touch. To draw my attention to your e-mail, please use the subject "PhD Inquiry" (I may miss it otherwise). Please do read the tips on here carefully though. If you provide a short summary of some of your research interests, I can give an impression as to whether this fits within the group. I can give a better indication of the fit of your research interests if you include your research proposal. However, I cannot give feedback to proposals or detailed suggestions on how to modify them. I am happy to answer questions about our research directions. Reading some of the group's recent papers is the best way to, and the research overview on my [website](https://mvdw.uk) is a good way to understand what we work on. If you do send an e-mail, please include a CV, a transcript of your courses, and a draft of your research proposal (if you have it already), and of course, the specific question you want answered. I do try to respond to all inquiries, but sometimes I do miss some due to large volumes of e-mails. **If you apply, your application will be considered carefully.** ### The Wrong Way to Get in Touch Academics (myself included) get quite a lot of open enquiries for internship / PhD positions. Many of these give a general overview of the experience of the person getting in touch, but do not contain: - an indication of future research interests, - a research proposal, - an indication of understanding of the group's existing research, - any specific question. It is hard to respond to these e-mails, since 1) it often looks like they were sent in bulk to many recipients, and 2) since there are no specific points/questions to respond to. Having a clear and specific question is the best way to get a conversation going, as is anything you can include about your specific interests and ideas. ### Research Proposal As part of your application, you will need to write a research proposal. This proposal is an outline of something that you would like to investigate. However, we will not hold you to what you write, and many PhD candidates work on completely different things to their original proposal! So the proposal is not meant to be a detailed plan of exactly what you will do. The goal is more to determine **1)** that your reserach interests fit with the group so we can support your journey, and **2)** that you are curious in a targeted way. With "targeted", I mean that you appreciate how your proposal fits in the field, why the open problem is important, and that you have some idea about the technical obstacles that you will need to overcome. Good proposals include: 1. An identification of one or more open problems that you want to work on, and an understanding of why this is important to the field. 2. A suggestion on how the broad problem can be broken up into specific sub-problems, with some measure of success. Including _some_ mathematical detail can help convey your ideas in a precise way. 3. An understanding of the technical obstacles, and some idea of a first approach on how to solve them. In terms of length, I agree with the departmental guidance [4] of 1000 words (mathematical formulas don't count). Keep in mind that a research proposal is not a literature review! The departmental guidance [4] also suggests you include relevant skills and experience, reasons for applying to Oxford in particular, and principal resons why you consider yourself a strong application. I would recommend to keep this short (since this is also in your CV) so you can focus on using the words for discussing your research interests. Many PhD applicants (myself included, when I was applying) find writing a research statement difficult, particularly if they have come straight from undergrad. The level of detail that is expected will depend on your experience, and other parts of your application will be considered as well. The most important thing to convey in your research statement, is what *you* are *interested* in. ### Interview Once we receive your application, you may be invited for a formal interview with myself and another academic. During the interview, you will have the opportunity to tell us about yourself, and to discuss your interests and what you want to work on in your PhD. This discussion will largely be based around your research proposal. We will also assess your technical knowledge, usually by discussing a paper that we will send you a few days before the interview. In some cases, I may ask you to meet some of the current group members as well. ### Post-interview Usually, we will interview several candidates around the same time. After interviewing all candidates, I will get in touch to let you know whether I think I could act as your supervisor. If so, we can discuss any questions and considerations that you have. I can also arrange for you to meet current group members, for you to get a first-hand idea of what the PhD is like. Most applicants will also be applying for funding, which is a separate decision that is made centrally by the department. However, this does not require any separate applications. If I do not think I can act as your PhD supervisor, I may be able to recommend other researchers. Applying for a PhD can be a difficult process. Since I only have a very limited number of spaces, it is common that I cannot take on excellent candidates. So I do recommend that you apply to several places. ## Funding The CDTs and departmental PhD programmes both have funding available for accepted students. If you are accepted in the process described above, you will automatically be considered for funding. The university keeps a [list](https://www.ox.ac.uk/admissions/graduate/fees-and-funding/oxford-funding#content-tab--2) [5] of all scholarships available. Do take a look there to check what scholarships you may be eligible for. In some cases, applicants may have already found their own external funding before applying. This is but good thing (although rare), since it removes the hurdle of this separate funding process. However, the most important consideration is still the research fit, which is assessed in the usual interview process. External funding sources: - Commonwealth PhD scholarship [6]: For students from commonwealth countries who otherwiese could not study. - [Martingale Foundation](https://martingale.foundation) [8] support during their studies. ## PhD Programmes and CDTs When applying for a PhD, you can choose to apply through two different routes: - The main departmental PhD degree (_Computing Research_ in the application). - Centres for Doctoral Training (CDTs) e.g. [StatML](statml.io) and [AI4Health](ai4health.io). The CDTs provide a more structured PhD, that includes a cohort of fellow-students, a teaching programme, and perhaps some structure to get you started on your projects (e.g. "miniprojects"). The main departmental PhD does not provide as much structure, and so it is more up to you to determine how you structure your early years. Both routes do lead you to the same destination. You will develop your skills, collaborate on projects with the group, and make your reserach contributions to the field. Do consider the information on the CDT website and consider whether the set-up appeals to you. **You can apply to both routes.** ## Good luck I hope this answered some questions about the process. Like I said at the start, do [get in touch](https://mvdw.uk) if you have any questions. To summarise: 1. Take a look at our [research](https://mvdw.uk/research-overview/) to see whether you're interested in it. 1. If so, think about what you want to research, and prepare a research proposal. 2. Follow the [Dept of CS instructions](https://www.ox.ac.uk/admissions/graduate/courses/dphil-computer-science) [4] and submit your application. 3. We invite certain candidates for interviews. We are looking forward to receiving your application! ## References & Links [1] Overview of the group's research: https://mvdw.uk/research-overview/ [2] Research group members: https://mvdw.uk/people/ [3] Our publications: https://mvdw.uk/publication/ [4] Instructions for applying to the Dept of Computer Science at the University of Oxford: https://www.ox.ac.uk/admissions/graduate/courses/dphil-computer-science [5] Available scholarships: https://www.ox.ac.uk/admissions/graduate/fees-and-funding/oxford-funding#content-tab--2 [6] Commonwealth PhD scholarship: https://cscuk.fcdo.gov.uk/scholarships/commonwealth-phd-scholarships-for-least-developed-countries-and-fragile-states/ [7] There are several suitable CDTs, like [AIMS](https://aims.robots.ox.ac.uk), and [StatML](https://statml.io). [8] https://martingale.foundation