Information for Prospective DPhil/PhD Candidates

Mark van der Wilk, August 2024

Thanks for your interest in doing a PhD/DPhil (in Oxford, DPhil and PhD are synonymous) in our group. This guide will tell you a bit about my group's approach to research, our interests, and tips on how to apply.

Getting in Touch

If you have any specific issues with the application, then do get in touch. I am happy to have a brief chat about your research interests, if there is overlap with my own. If you do send an e-mail, please include a CV, a transcript of your courses, your research interests, and a draft of your research proposal (if you have it already), and of course, any specific questions you want answered.

To draw my attention to your e-mail, please use the subject "PhD Inquiry" (I may miss it otherwise). Please do read the tips on here carefully though, to avoid asking something that is already answered. I do not have the capacity to give feedback to proposals or detailed suggestions on how to modify them.

I am happy to answer questions about our research directions. Reading some of the group's recent papers is the best way to, and the research overview on my website is a good way to understand what we work on.

I do try to respond to all inquiries, but sometimes I do miss some due to large volumes of e-mails. If you apply, your application will be considered carefully.

Our Research

See our website and research overview for details and explanations with different emphases. Three research questions are central to our research group:

• How do we find general patterns that allow generalization beyond the training set? I.e. how do we find inductive bias, rather than building it in?
• How can we create neurons that automatically assemble their structure, with as little global communication as possible?
• How do we use predictions to build decisions-making systems that are safe and reliable?

We explore solutions that develop deep learning in a different direction than the "scale up" approach that is common today. This builds towards a different vision of what deep learning could look like. One where training a model needs 1) less data, 2) less human intervention, and 3) less energy.

While we explore many paths to these goals, we take significant inspiration from Bayesian statistics, and particularly from Bayesian model selection (underappreciated in ML). So far, this has led to methods for selecting inductive bias (e.g. invariance), as well as identifying causal direction. Bayesian uncertainty estimation may also help to inspire new learning rules, which do not require communication across a whole network like backpropagation does.

Overall, many topics contribute towards better answers to these three questions, including:

• Bayesian model selection & Neural architecture search
• Causality & relations to disentanglement
• Out of distribution generalisation
• Geometric deep learning and invariances
• Meta-learning
• Continual learning
• Local learning rules
• Approximate Bayesian inference & uncertainty quantification
• Gaussian process models, Deep GPs, and relations to Neural Nets
• Bayesian optimisation & experiment design
• Generative modelling, semi-supervised learning, self-supervised learning
• Capsule networks
• Model-based reinforcement learning
• Analysis of generalisation and learning in deep networks, generalisation bounds, connections to Bayesian inference.

If your research interests match any of these, then you may be a good fit for the research group.

The PhD and our Research Group

Given that you're interested in joining our research group, you probably have a view of what you want to get out of it, although for clarity, here is a summary of what you can expect. A PhD is a journey where you develop your understanding to the boundary of human knowledge, and beyond. This is a significant undertaking that usually takes 3-4 years. This makes the research groups you apply to an important choice.

I would say that the most important criteria in choosing a group are 1) how well your research interests fit with those of the group and the prospective supervisor, and 2) whether the group feels like a supportive and pleasant place to work. The people in the research group you join will be the people who you brainstorm with, who you whiteboard maths with, who you will code up projects with, and who will give you constructive feedback on your work. So it's important to have an environment that you're happy with!

In our group, we aim to understand existing machine learning methods, and develop new methods that work reliably and without human supervision. Our research is motivated by a wide variety of problems, from improving the reliability of large-scale neural networks, to more statistical problems where data is scarce. We also work on more applied problems, most often through collaborations with domain-experts at Oxford, Imperial College London, and beyond.

We started in 2020, and we currently consist of myself, a post-doc, and 8 PhD candidates. There is a nice mix of expertise and perspectives (e.g. from Bayesian statistics to deep learning engineering), and motivating applications (biology to robotics). The aim is to have a group with a diverse collection of expertises that overlap roughly around taking inspiration from probabilistic inference. To widen the expertise that we can learn from, we also collaborate with researchers at Oxford, Imperial College London, in the UK, and internationally.

To get an impression of who we are and what we do, I recommend that you look at:

• the overview of our research [1],
• members of the research group [2],
• and perhaps a few of our publications [3].

If you like the look of our research directions, the next step is to develop your research proposal (tips below), and to send in your application to Oxford. During the interview process, we will have the opportunity to chat informally as well, and you can meet some of the current group members.

Admission Requirements

Applying to a PhD is quite different than applying to a masters course. An MSc will accept many (~100) students onto a cohort. The large numbers make it easier to give an indication of what the admission requirements typcially look like. When applying for a
DPhil, you will be applying to one or several individual supervisors, each of which may only have space for at most a couple of students. This means that there are no consistent rules for who can be admitted. I myself will be able to take 1-2 students per year, and many factors weigh into the admission decision.

As a minimum, you will need evidence of a strong academic background, with evidence of mathematical skill [9]. Basically any STEM degree can show this, i.e. engineering/CS/maths/stats/physics. While exam results certainly aren't everything, successful applicants typically have strong exam results (e.g. top 10% of their year). Interest in our research interests also weights heavily, as does your research statement.

Application Process

In short

In principle, all you need to do is follow the Department of Computer Science application process [4]. The main application deadline is mid-day December 1. Submitting for this date is necessary for your application to be automatically considered for all funding opportunities. You may still submit an application after the deadline, as there may still be funding spots available, but meeting the deadline significantly increases your chances.

If you are interested in doing a PhD in my group, I recommend that you definitely apply to the Department of Computer Science. You may also apply to CDTs [7], which have separate funding pots, deadlines and requirements for the applications. See the respective CDT websites [7] for more information. Once you are accepted to the CDT, you can then arrange to do a project with me.

Research Proposal

As part of your application, you will need to write a research proposal. This proposal is an outline of something that you would like to investigate. However, we will not hold you to what you write, and many PhD candidates work on completely different things to their original proposal! So the proposal is not meant to be a detailed plan of exactly what you will do.

The goal is more to determine 1) that your reserach interests fit with the group so we can support your journey, and 2) that you are curious in a targeted way. With "targeted", I mean that you appreciate how your proposal fits in the field, why the open problem is important, and that you have some idea about the technical obstacles that you will need to overcome. Good proposals include:

1. An identification of one or more open problems that you want to work on, and an understanding of why this is important to the field.
2. A suggestion on how the broad problem can be broken up into specific sub-problems, with some measure of success. Including some mathematical detail can help convey your ideas in a precise way.
3. An understanding of the technical obstacles, and some idea of a first approach on how to solve them.

In terms of length, I agree with the departmental guidance [4] of 1000 words (mathematical formulas and references don't count). A research proposal is not a literature review, so you are not expected to summarise reading you have done. However, you can include a reference to a paper you find important to show where you got your inspiration from, or to contrast your approach to an existing paper.

The departmental guidance [4] also suggests you include relevant skills and experience, reasons for applying to Oxford in particular, and principal resons why you consider yourself a strong applicant. I would recommend to keep this short (since this is also in your CV) so you can focus on using the words for discussing your research interests.

Many PhD applicants (myself included, when I was applying) find writing a research statement difficult, particularly if they have come straight from undergrad. The level of detail that is expected will depend on your experience, and other parts of your application will be considered as well. The most important thing to convey in your research statement, is what you are interested in.

Interview

Once we receive your application, you may be invited for a formal interview with myself and another academic. During the interview, you will have the opportunity to tell us about yourself, and to discuss your interests and what you want to work on in your PhD. This discussion will largely be based around your research proposal. We will also assess your technical knowledge, usually by discussing a paper that we will send you a few days before the interview.

In some cases, I may ask you to meet some of the current group members as well.

Post-interview

Usually, we will interview several candidates around the same time. After interviewing all candidates, I will get in touch to let you know whether I think I could act as your supervisor. If so, we can discuss any questions and considerations that you have. I can also arrange for you to meet current group members, for you to get a first-hand idea of what the PhD is like.

Most applicants will also be applying for funding, which is a separate decision that is made centrally by the department. However, this does not require any separate applications.

If I do not think I can act as your PhD supervisor, I may be able to recommend other researchers. Applying for a PhD can be a difficult process. Since I only have a very limited number of spaces, it is common that I cannot take on excellent candidates. So I do recommend that you apply to several places.

Funding

The CDTs and departmental PhD programmes both have funding available for accepted students. If you are accepted in the process described above, you will automatically be considered for funding. The university keeps a list [5] of all scholarships available. Do take a look there to check what scholarships you may be eligible for.

In some cases, applicants may have already found their own external funding before applying. This is but good thing (although rare), since it removes the hurdle of this separate funding process. However, the most important consideration is still the research fit, which is assessed in the usual interview process.

External funding sources:

• Commonwealth PhD scholarship [6]: For students from commonwealth countries who otherwiese could not study.
• Martingale Foundation [8] support during their studies.

PhD Programmes and CDTs

When applying for a PhD, you can choose to apply through two different routes:

• The main departmental PhD degree [4].
• Centres for Doctoral Training (CDTs) e.g. StatML and AI4Health.

The CDTs provide a more structured PhD, that includes a cohort of fellow-students, a teaching programme, and perhaps some structure to get you started on your projects (e.g. "miniprojects"). The main departmental PhD does not provide as much structure, and so it is more up to you to determine how you structure your early years.

Both routes do lead you to the same destination. You will develop your skills, collaborate on projects with the group, and make your reserach contributions to the field. Do consider the information on the CDT website and consider whether the set-up appeals to you. You can apply to both routes. If you want to do your PhD with me, I recommend that you also apply through the departmental PhD programme.

Good luck

I hope this answered some questions about the process. Like I said at the start, do get in touch if you have any questions.

To summarise:

1. Take a look at our research to see whether you're interested in it.
2. If so, think about what you want to research, and prepare a research proposal.
3. Follow the Dept of CS instructions [4] and submit your application.
4. We invite certain candidates for interviews.

We are looking forward to receiving your application!

References & Links

[1] Overview of the group's research: https://mvdw.uk/research-overview/
[2] Research group members: https://mvdw.uk/people/
[3] Our publications: https://mvdw.uk/publication/
[4] Instructions for applying to the Dept of Computer Science at the University of Oxford:
https://www.ox.ac.uk/admissions/graduate/courses/dphil-computer-science
[5] Available scholarships: https://www.ox.ac.uk/admissions/graduate/fees-and-funding/oxford-funding#content-tab–2
[6] Commonwealth PhD scholarship: https://cscuk.fcdo.gov.uk/scholarships/commonwealth-phd-scholarships-for-least-developed-countries-and-fragile-states/
[7] There are several suitable CDTs, like AIMS, and StatML.
[8] https://martingale.foundation
[9] There are many different kinds of mathematical skill. For example, a maths degree gives deep, detailed & ground-up understanding, and develops the ability to prove things. An engineering degree typically has less emphasis on proofs, but does develop the ability to mathematically analyse situations, or make numerical models. Both of these skills are extremely useful in a machine learning PhD. The important thing is that you can show that you enjoy taking on mathematical challenges, and that you can independently make progress when faced with a mathematical problem of any of these types.