A Reading List

###### tags: `one-offs` `references` # A Reading List **Overview**: In 2020, I posted a short reading list of some textbooks which I've found to be useful as a learner. I document the list here, with some present-day commentary ## General Criteria The basic motivating criteria for the books to be included in this list were that: 1) if I had found the book earlier, then I would have understood certain topics more quickly, 2) the book contributed to me liking the subject, and 3) I still enjoy reading the book. ## The List A good place to start is Luc Devroye's "Non-Uniform Random Variate Generation", which is available freely and in full at http://luc.devroye.org/rnbookindex.html. I like this book because it covers a lot of the basic tricks which are foundational to simulation. It also distinguishes itself by nicely paying attention to simple implementation-level details which can help to save time in practice. If you read this book cover-to-cover, then you will be well-equipped to solve all sorts of direct simulation problems efficiently. Another terrific reference is Art Owen's "Monte Carlo theory, methods and examples", which is available freely at https://artowen.su.domains/mc/. While it is still technically a work-in-progress, the content is in a very advanced state. The presentation is very thorough without becoming too technical, gives an excellent broad introduction to Monte Carlo, and contains many useful references. It gives a great picture of the richness of the field, and helps you to appreciate why certain innovations have been so impactful. A slightly more recent release is is Nicolas Chopin and Omiros Papaspiliopoulos' "An Introduction to Sequential Monte Carlo" (https://link.springer.com/book/10.1007/978-3-030-47845-2). This text provides an accessible introduction to the topic (which was much-needed at the time), and also provides very good coverage of modern developments (particle MCMC, SMC samplers, and so on). In terms of mathematical sophistication, it certainly goes up a level from the previous two recommendations, though not dramatically so; the exposition guides the reader well. From my point of view, the cost of understanding SMC well necessitates engaging with the measure-theoretic formulation of the problem at some level, and I think that this is a worthwhile investment. In any case, I think that this text provides a nice change of pace relative to the more out-and-out theoretical picture provided by (any of the) relevant textbooks of P. Del Moral. As concerns well-established MCMC methods, I would say that for readers with a Statistics-centric background who are in search of an Algorithm-centric (as opposed to Theory-centric) introduction to MCMC, one can recommend either the book of Casella & Robert (https://springer.com/gp/book/9780387212395) or of Liu (https://springer.com/gp/book/9780387763699). Both are quite reliable resources, though perhaps not the most modern. (Actually, on this point: when it comes to recent developments in MCMC, textbooks have arguably not entirely "caught up", in that there are many important ideas from the past 10-15 years which have not yet shown up in textbooks. For these topics, I would currently recommend surveys, tutorials, etc. instead. Some newer resources may have shown up in the interim, though.) As an MCMC enthusiast, one should occasionally see how such methods are used away from your primary application domain. As a concrete example, it is often worth dipping into the physics literature. Some nice starters in this direction are the texts of Kalos & Whitlock (https://onlinelibrary.wiley.com/doi/book/10.1002/9783527626212), of Krauth (https://global.oup.com/ukhe/product/statistical-mechanics-algorithms-and-computations-9780198515364?cc=gb&lang=en&), and of Lelièvre, Rousset & Stoltz (https://worldscientific.com/worldscibooks/10.1142/p579), each coming from slightly different viewpoints. On a few topics which are a bit more distant from my usual interests, * 'Density Ratio Estimation in Machine Learning' by Sugiyama, Suzuki, and Kanamori (https://cambridge.org/core/books/density-ratio-estimation-in-machine-learning/BCBEA6AEAADD66569B1E85DDDEAA7648) is a very concise and readable account of a topic which admits a very elegant treatment. * 'Interacting Multiagent Systems' (https://global.oup.com/academic/product/interacting-multiagent-systems-9780199655465?cc=gb&lang=en&) by Lorenzo Pareschi and Giuseppe Toscani is a relatively friendly introduction to particle systems of kinetic and / or collisional type. It covers a nice mix of relevant analytical tools, simulation methods, and applications. * 'Molecular Dynamics' by Leimkuhler & Matthews (https://springer.com/gp/book/9783319163741) is a great text on the titular topic, with a thorough treatment of the relevant theory of ODEs, SDEs, Numerical Integration, and more - a perfect setting for understanding several useful computational tools. Some later additions: * With regard to (Convex) Optimisation, * A (deservedly) standard recommendation is Boyd and Vandenberghe (https://web.stanford.edu/~boyd/cvxbook/). Stephen Boyd is also a co-author on some other recent monographs on more specific topics (e.g. https://web.stanford.edu/~boyd/papers/admm_distr_stats.html, https://web.stanford.edu/~boyd/papers/prox_algs.html) which I have also found to be extremely readable and very useful, particularly in view of modern advances in _non-smooth_ convex optimisation. * With a slightly different flavour, I also enjoyed Nesterov's 'Lectures on Convex Optimization' (https://springer.com/gp/book/9783319915777) which is very precise, and gives a very strong sense for how the author thinks about systematically formulating and solving these problems. * Some time ago, I enjoyed taking part in a reading group on Martin Wainwright's 'High-Dimensional Statistics' (https://cambridge.org/core/books/highdimensional-statistics/8A91ECEEC38F46DAB53E9FF8757C7A4E, which develops a range of useful analytical tools, and then applies them to several interesting problems in contemporary statistics. * Another nice statistical textbook for some related topics is Alexandre Tsybakov's 'Introduction to Nonparametric Estimation' (https://springer.com/gp/book/9780387790510), which is again precise, thorough, and systematic.