 HackMD
Diffusion Models: From Theory to Practice (6.S982): Spring '25, MIT
Prerequisites đź“š: Machine learning (6.7900 or similar), probability (6.3700, 18.600 or similar), linear algebra (18.06, 6.C06[J] or similar), and calculus (18.02 or similar).
Meeting time đź•‘: Tuesdays, 1-4 p.m.
Location đź“Ť: E25-111
Satisfies: II, AAGS, Concentration subject in AI
Instructors 🧑‍🏫: Costis Daskalakis and Giannis Daras
Teaching Assistant 🎓: Vardis Kandiros
Office Hours đź•”: Tuesdays 5-7 pm (after class) or by appointment.
Description 📖: Deep generative models have found a plethora of applications in Machine Learning, and various other scientific and applied fields, where they are used for sampling complex, high-dimensional distributions and leveraged in downstream analyses involving such distributions. This course focuses on the foundations, applications and frontier challenges of diffusion-based generative models, which over the recent years have become the prominent approach to generative modeling across a wide range of data modalities and form the backbone of industry-scale systems like AlphaFold 3, DALL-E, and Stable Diffusion. Topics include mathematical aspects of diffusion-based models (including forward and inverse diffusion processes, Fokker-Planck equations, computational and statistical complexity aspects of score estimation), the use of diffusion models in downstream analyses tasks (such as inverse problems), extensions of diffusion models (including rectified flows, stochastic interpolants, and Schrödinger bridges), and frontier challenges motivated by practical considerations (including consistency models, guidance, training with noisy data).
Sylabus đź“’:
Week 1: Introduction to generative models and their applications (GANs, VAEs, Flows, Diffusion Models, and Inverse Problems)
Week 2: Langevin Dynamics
Week 3: Introduction to Diffusion Models (definition of the forward process, reversibility, relationship between Conditional Expectation and MMSE, Tweedie’s Formula).
Week 4: Deeper dive into diffusion models (Fokker-Planck Equations, Deterministic Sampling, Latent Diffusion Models)
Week 5: Special Topics I: Diffusion Models and Inverse Problems
Week 6: Special Topics II: Rectified Flow, Stochastic Interpolants, and Schrödinger Bridges.
Week 7: Special Topics III: Generative Models from Corrupted Data (Stein’s Unbiased Risk Estimate, Noise2X, Ambient Diffusion)
Week 8: Guest Lecture
Week 9-12: Student Presentations.
Grading đź“Š: 50% group project, 25% paper presentation, 25% quizzes.
Contact 📧: Questions about the class? Send an email at costis[at]mit[dot]edu or gdaras[at]mit[dot]edu.
